I grew up with metric.
Both metric and imperial (fractional) are just as easy once you learn them.
True, imperial takes a little longer to sink in, but once it sink in that is half of half of half it can be easier than metric for some things.
Do you have problems with the 12/60/60 set up of clocks? I'm sure you don't.
Yes, especially in March and November. Oh wait, that's DST cause someone doesn't want to maybe miss a TV show!
I'll bet if we all went GMT for 0 hour and converted to a 10-hour day you'd do just as well or better. They might even redefine the second to better match SI units.