As usual, the NEC is not easy to read and understand. I see:
220.61(a) saying to calculate the neutral as the maximum unbalanced load (so remove all your 208V loads with no neutral and the highest phase current would most likely be your calculated neutral current). This probably keeps the neutral load still around 40A since you only have one 208V single phase load.
220.61(b) says certain appliances get to be counted at 70% as can the load over 200A. Neither of those probably apply in your case.
220.61(c) says back in part (a) you can't exclude circuits that are 208/120 single phase (e.g. 2 hots and a neutral) or any of the feeder that is supplying non-linear loads.
This last one can be difficult, as you may have no idea what type of loads are connected (office computers would be non-linear, but coffee pots not). Regardless, if your calculated load comes to 40A, your #4 neutral can easily carry that even if everything is considered harmonic. The issue will affect someone adding more load in the future. If they add things that are harmonic, they'll have to watch their calculation. If they add purely 3 phase things, then it won't be an issue. Remember that the neutral doesn't need the 125% continuous factor applied, so that there buys you some headroom (you'll need to do the original calculation again with the 125% factor removed to see what the neutral load is).
Bottom line, if these are copper conductors, you have a lot of margin (non-continuous factor exempt on the neutral, 75C ampacity allows 85A on #4 CU). If they are aluminum, you'll want to ensure the neutral load is under 65A.