Yep. That's a big no-no.
If those caps are downstream from the soft starter, chances are they have caused a short in one or more of your SCRs. If that happens you get a high DC component in the starting power output while ramping. The motor fights itself until the bypass contactor kicks in or the ramp time expires and the soft starter goes to full voltage. Might be time to rethink the application. Did someone not read the instructions?
It's possible that you got lucky and the caps merely caused self-commutation of the SCRs (meaning they turned on by themselves when they weren't supposed to), and they will be OK when you remove the caps. Check the SCR stacks with an ohmmeter going line-to-load on each phase (L1-T1, L2-T2, L3-T3, power off of course). If you have anything less than 10k ohms, you have a short. If you do, replace them all. If the caps damaged one, they probably stressed the rest to the point of near failure.