Generator sizing 3p 480/277 or 208/120

So then it is OK to install a 100A MB in the existing 208/120V panel and feed the ATS.
Use A 20KW 208/120 3p generator, install a 208/120v 3ph panel off the ATS and put the 8 120v loads in that panel.

I was concerned that taking 100A to the ATS would be too much. It would be equal to the MBR of the existing 100A 3P 120/208v panel.
the load supplied is all that really matters. Your mentioned items though on 8 different branch circuits are the type of load that doesn't draw much and is possible they could be all on just one maybe two branch circuits. If that is the case a 20 or 30 amp feeder is sufficient to handle them and so would be a generator with only 20 or 30 amp output capacity.

At same time even if you fed this with a 200 amp conductor it still only draws less than 30 amps and a 30 amp generator would still have no problem with supplying it.

Lets say your panel supplied by a 100 amp feeder only had a total connected load of 58 amps. A generator capable of delivering 58 amps is really all that is needed even if you still ran 100 amp conductors to it. It won't deliver any more than what is demanded by the connected loads and overcurrent protection will keep it from being overloaded should something happen where it does try to deliver more than it is rated for.
 
I am doing a very similar project right now.

208 system and all the customer wants powered is essential safety and parking lot 120V LEDs and 1 208V compressor to keep air in the fire system.

We are using a 14 kw generator, and a 100A ATS because that is the smallest they make. The generator breaker is 50 amps and the breaker coming out of the panel to feed normal power to the ATS is also 50 amps. Then there is a small subpanel with all the essential loads in it.

Actual load will be much less than 50A.
 
I came here to say this. If the system is 120/208 then use a 120/208 generator and transfer switch.

I have seen way too many 208 systems backed up by a 240 single phase generator and it just leads to all sorts of weirdness.

People just seem afraid to special order the generator and ATS they actually need, it is not that hard.
I’ve done a couple of Generac’s that way, they respond slower than everyone else, so the voltage difference is not an issue with them, at least the older ones.
 
Within reason the generator could not be "too big". It si not going to force more current than the load requires.
You mentioned a 20kw and 100 amps. Note on a 3 phase system that would be closer to 55 amps per phase,
For simplicity sake I would install a 100 amp ATS on the existing 100 amp panel and use the 20kw generator the customer desires.
If you elect to go smaller changes are they will decide to add more load than currently anticipated and , with the 20kw, you will be prepared.
I know you qualified the "too big" comment with "within reason", but just to provide some more context for other readers, there are (at least) two detrimental effects on the system that can come with oversizing a generator: excessive fault current and wet-stacking. In a fault scenario, the fault will only be limited by the system impedance and the sub-transient reactance of the generator. A larger generator can produce more fault current which may result in exceeding SCCR/AIC ratings on certain devices. Wet-stacking is unburnt fuel deposits in the exhaust and caused when a generator does not run near it's rated output. Special equipment may be needed to appropriately load the generator once a year. Both these effects will probably only present themselves in a severely oversized application. For example, I'm stuck with a 1.5 MW generator on a facility that never pulls more than 300 KVA. I'm looking down the barrel of a very expensive load test and all my AFC calcs had to use the generator's fault current for the worst case scenario.
 
Wet-stacking is unburnt fuel deposits in the exhaust and caused when a generator does not run near it's rated output.
Wet stacking is pretty much only a problem with diesel engines as well. A diesel engine not loaded to ~50% or greater can pass these unburned fuel deposits because they aren't running at high enough operating temperature to cleanly burn all the fuel, and the diesel that passes through is oily in nature which helps cause these deposits. Gasoline, LP, NG engines all tend to run at higher temperature (once initially warmed up) even if not loaded much and this tends to burn all the fuel brought in to the combustion chamber more efficiently, plus if those would pass unburned fuel through it isn't oily in nature and wouldn't collect on surfaces in the exhaust in same manner as it could with diesel engine.
 
Top