I think getting some of the timers right has to be the hardest, followed by making sure the transmitter FIFO is kept full when it's supposed to be full. At lower bit rates there's plenty of time, and things like the T3.5 timer is fairly long. The last Modbus stack I wrote I made sure to test all the way to 460kBaud to make sure it would work properly at saner speeds.
The problem is increased reliance on general purpose operating systems as the platform, rather than a small RTOS on a dedicated microcontroller.
My personal peeve is using TCP instead of UDP. Datagrams are so much nicer than byte streams, and multicast UDP seems like a natural approach to doing Modbus on a large network with lots of devices.