Yes, putting the OL aux contact in series with the Stop button will serve to shut it down on an overload.
Just so you know what happens when you connect a motor in Delta when it was supposed to be Wye.
The effective voltage across the windings will be (in this case) 173% higher than the motor is designed for. The motor will still spin at the desired speed because speed is based on frequency, not voltage, but you will have been over saturating the windings continuously. That over saturation becomes heat in the motor, but because of the way that drive is programmed, it likely would not respond to that correctly and allow it to continue running. That would ex[lain why you lost 3 motors running it this way.
UNLESS...
There is a "trick" used when running a motor at ABOVE its base speed using a VFD, wherein you WIRE the motor at the lower voltage and APPLY the higher voltage, BUT you program the VFD as if it is the LOWER voltage again. That way, when you get to the full rated frequency and you want to make it go faster with the VFD, you can continue to do so, because the VFD thinks it is the lower voltage, and the motor is happy, up unitl (in this case) 173% speed or roughly 100Hz. So it MIGHT be that someone originally intended for this saw to be run like that, but later on, someone didn't understand that and reprogrammed the drive for the higher voltage, which then started killing the motors.