Aside from minimum sizing one of the other big reasons to go bigger is this. At startup a non-VFD/soft start motor can pull up to 6-10x FLA. So if you have one or two very large motors and the rest of the loads are quite small starting can be a big issue. So this is where you may want to do a motor sizing style calculation say 250% of largest motor plus 100% of other loads with a breaker and use that for panel sizing.
Also above about 1000 A or so fuses get expensive, non-stocked, and very large. Class T and L exists but this is a point where insulated case breakers are usually your best bet. In addition arc flash is going to get obnoxiously high and be a serious issue which is much easier to address with either a fast breaker with a maintenance switch or better yet a virtual main breaker scheme. So my immediate response to the overall design is NO, don’t do this.