The reason you don't count a neutral as a current carrying conductor is because the adjustment factors are based on the I²R losses in the conductor that cause heat. The adjustment factor is to limit the heat. Since the heat is I²R we can look at it this way for a 3 phase circuit, not including non-linear circuits.
If we have circuits loaded to 20 amps and all of the loads are line to neutral we can look at the heat a number of ways
A phase current 20 amps, 400R watts of heat
B phase current 20 amps, 400R watts of heat
C phase current 20 amps, 400R watts of heat
Neutral current 0 amps, 0 watts of heat
Total heat 1200R watts of heat
A phase current 20 amps, 400R watts of heat
B phase current 20 amps, 400R watts of heat
C phase current 20 amps, 0 watts of heat
Neutral current 20 amps, 400R watts of heat
Total heat 1200R watts of heat
No matter what combination of current on the phase conductors and the neutral, the total heat will never exceed the heat created by all three phase conductors being loaded at 20 amps.