All installations are a trade-off between cost and performance.
Apparently during the WW II period, in some areas it was common to wire circuits using a single wire and the conduit as the grounded conductor. This is simply using a shared grounded conductor and EGC and taking it to an extreme.
When you ask 'why 277 volt lighting' the big objection seems to be the shared neutral. If you used 277V lighting but without the shared neutral this objection goes away, at the cost of more copper usage.
You could take this further, and use only 120V lighting with no shared neutrals. Such would be minutely safer, but would be much more expensive to install.
IMHO shared neutral circuits are perfectly acceptable if properly/carefully installed, and 480/277V for lighting is perfectly acceptable in commercial situations where qualified individuals are expected to work on them. I would not want 277V lighting in my home
-Jon