I've done probably 40 panel changes in the past year, all recessed in drywall.
Some EMT (houses built in 60's when men were men and electrical contractors actually made money) and some Romex. I can honestly say that a 6 hour average time is pretty accurate and that is with a good electrician taking very few if any breaks.
Sure you will get a few that you can do in 4 hours ( surface mount panel, all wires come in from same side etc..) but those are not common.you will also get some that take 8 or more ( Macmikeman's example of the rats nest with studs too close together)
As for materials, unless you are using absolute garbage, your not going to put together a new 40 ckt panel with new breakers and other misc materials ( polaris taps, boxes, ground bars , etc..) for less than $350 and that don't include arc faults or any mark up at all.