So the X from the neutral–4v–was subtracted from the X of Phase A–16v–and the Y of the neutral– -v√3–was added to the Y of Phase A–0?
Basically, although I would say that it's subtraction in both cases, using the proper sign conventions.
now thinking about how to apply that to my 2nd diagram seems like a nightmare,
As you have specified a single conductor size throughout, It's not that bad if you assume the voltage waveforms all stay 120 degrees apart and the load currents are constant and in phase with the voltages, as I did.
Just draw your network out as a one line graph, with a node at the source, at each fixture (indicate phase letter), and everywhere the conductors branch. Label each edge of the graph with the length of the run between nodes. If you have a fixture at a branching point, I think it will be simpler to separate those two conditions with an edge of length 0.
Now starting at the leafs of the tree, label edge with the lines currents (or rather number of unit currents for simplicity), using vector math on the neutral, where 1/a + 1/b + 1/c = 0. Each fixtures add a unit current pair, and the supply side of each branching node is the sum of the currents from the other edges incident to the node. [After you do this, if the segments with multiple downstream fixtures don't have neutral current near 0, adjust the phase choices of the downstream fixtures appropriately, and repeat.]
Finally, starting at the supply, add another set of labels to each edge, which is the cumulative total of the segment currents times the segment length (using vector math for the neutral). Then at each leaf of the tree, determine the phase X with label A for which A/X + the vector neutral label N has largest vector magnitude. [If the neutral currents have been kept near 0, I expect that will just be determined by the phase labels.] Take the biggest of these for all leafs, and plug that number as I * d into the unidirectional voltage drop formula C
mil = K * I * d / V
d.
Cheers, Wayne