Using a VFD will typically require a 40hp drive for a 20hp motor. Somewhere around 110a 240/1ph full load. You would be pulling paralleled 350 aluminum to be getting 5% voltage drop assuming a 160a max load (110 for motor and 50a for barn).
IMHO that is not quite correct.
To run a 20hp motor from a single phase supply, you need to oversize the drive in order to have a large enough capacitor bank and input rectifier. However the drive is still only drawing 20hp (at full load) from the supply. So you are probably looking at 55-60A (at full load) being used by the VFD and pump.
Might be significantly cheaper to go with a 480 single phase service, and bringing it up to your pump location, getting a 480v drive for the pump. Get a small transformer to feed the barn.
This is a good point. Going to 480V means half the current to deliver the same power to the load, and twice the available voltage to tolerate the voltage drop. So for the same % drop you get away with much smaller conductors.
A 480V drive will typically cost less than a 240V drive of the same capacity.
This needs to be balanced against how much the utility will charge for a 480V service (will you have a separate monthly base change, will they charge for bringing in the transformer) and the ongoing losses in the step down transformer, and possibly needing to convert the pump motor to higher voltage.
As I see it the list of reasonable options are:
1) bring single phase primary service to the location and use a 240V VFD
2) use fat conductors to mitigate voltage drop
3) tolerate large voltage drop
4) get a 480V single phase service and use a 480V VFD with a step down transformer to supply the barn.
-Jon