Is contactor and overload sized for the motor it is driving? If sized for a 240 volt or even a three phase motor of the same horsepower it could be too small to handle the 120 volt motor.
It doesn't hurt to run the neutral through the contactor and overload, as long as any interruption of the neutral is simultaneous to interruption of the ungrounded conductor, but also isn't necessary.
You only have to use all three poles of the overload if it is a type that is designed to detect phase loss, or if accuracy of the device depends on heating from all three units. Melting alloy overload units usually use a different selection chart for a single element than for multiple elements in the same enclosure, check the instructions of the overload unit for settings or element selection.