The NEC states that its purpose is all about safety. The fact that the NEC does not prohibit what you are describing is the only proof you need that its authors do not consider it unsafe. You need to maintain the working space clear per 110.26. You need to follow PPE rules (including keeping uninvolved personnel away from the gear when operating breakers). That is all the code has to say on the subject.
However, let's be clear about what you mean by "safe." Do you consider it safe to enter an intersection on a green light, not knowing for certain whether the car to your right has noticed his red light? Do consider it safe to walk outdoors, not knowing if there is a person nearby with a highly contagious disease? Do you consider it safe to live on a planet that is struck by lightning 3000 times every day? No electrical component is immune to failure, and the gear in the control room can explode without warning. But then, so too can the control panels that the operators sit in front of all day.