First of all, I think this is definitely an area where the code is genuinely ambiguous. The code doesn't even define 'conductor' clearly enough to say that the conductive parts of a disconnect aren't conductors.
With that said, to me the key to your question is that you mentioned an unfused disconnect. If it were a fused disconnect, then the application of the 120% rule would be separately calculated for the conductors on either side of that disconnect, depending on the other overcurrent devices at the other ends of those conductors. But for an unfused disconnect, the same 120% calculation would apply to the conductors on both sides of the disconnect. It simply seems illogical that it wouldn't also apply to the disconnect itself.
If you are oversizing conductors for voltage drop and not the 120% rule that is really a different situation.