The derating is needed for the portion of the run that has (20) conductors inside of a conduit, where the heat builds up. Once the conductor is in the panel gutter, the heat load dissapates, and does not use the OCPD as a heat sink, making the OCPD trip at inaccurate load points. The same is true at the other (load) end, depending on what kind of device is being fed. I doubt you're running (20) #6's into a wall box, and you can splice/downsize the conductors in downstream portions of the run, to #8, #10, #12, whatever is required for the ampacity derating for that portion of the circuit.
This makes sense for ampacity derating--voltage drop is a different consideration.