Simply put, the connected Wye point will force the primary windings to remain at 120V which prevents the 480V from truly accommodating an unbalance loading, like you may experience when starting large motors.
I do not know of a single transformer manufacturer that does not warn against connecting the Wye when a transformer is run in reverse.
There is only a major problem when the load side winding is delta or there is a delta tertiary winding. And it results from input voltage imbalance, not necessarily related to output loading.
Utilities often use wye-delta transformers, in part because of ferro-resonance problems with wye-wye, but they have complete control over the balance of the input phase voltages on the wye side and can thus avoid the problem.
The delta winding, being a closed loop, forces the sum of the phase voltage phasors to be zero. Anything else will cause large circulating currents in the winding and corresponding unwanted currents on the line side. But this phasor sum condition is not automatically met in the windings on the wye side. If the X0 is left floating (not connected to
either neutral or ground), the voltage at the wye point will be free to vary, allowing the input side constraint equation to be met. The current sum condition (no current through X0 when left floating) is automatically met because of the delta winding, regardless of phase load imbalance on that side.