The general idea is fine by the NEC. Utilities have their own rules about how many PV disconnects they'll allow and their location requirements so I can't speak to that; you'll have to check with the utility if you don't already know their rules.
If the 400A disconnects shown at each building are fused and immediately adjacent (or even just close enough) to the tap points, I don't think the conductors between the taps and the disconnects need to be upsized. The taps to the PV disconnects should follow the tap rules in 240.21(B). If a single fused disconnect for each system cannot be placed to meet the taps rules, utility requirements, and 690.15 requirements for disconnects at the inverters, then you might need additional disconnects.
One other spot bearing some additional scrutiny is the conductors between the transformer @ building 2 and the tap point at Building 3. What transformer secondary rule (240.21C) do these follow? It may be that you cannot tap these conductors without putting an additional overcurrent device ahead of the tap. (Or maybe one already exists but is just not shown?)
Utility requirements aside, the general idea to interconnect at each building to avoid trenching is sound by the NEC, and these other details I've mentioned are just details.