Of course any one can withdraw with your account/routing numbers, but that's fraud and most bank accounts are protected from this and the bank must credit your account within a few days once you file a claim. Then the bank will investigate and make a determination and leave you with the money or transfer the money back if it is found to be a legitimate transaction.
And until it is all straightened out, someone likely wiped out all you had, leaving you with other hardships, even if they are temporary. It is still probably a good idea to have an account with limited funds that is used for these transactions and move money in/out as necessary to prevent someone from taking an amount that will create a hardship for you. If all your money is in one account - then it can all be taken and maybe even more than you actually have. Electronic payments made when there are insufficient funds are not necessarily automatically rejected just like a check with insufficient funds is not always returned unpaid. You still have the payment taken from your account and do end up paying overdraft charges as determined by banks policies.
Reputable companies will not take funds from your account if they were not authorized to do so. OP has someone he does not know or trust, once they have account information, they can take funds. If they are legitimate they likely will not make a disappearing act, and maybe if they took unauthorized funds it may just be a mistake. Someone that does a disappearing act likely was intending to take you for whatever they can. Even if you get reimbursed somehow it created a hardship.
Same goes for writing a check or using a credit card - once someone has your account info - they have a place to start if they want to try something. Look at fine print details for your account to find what limitations may be for your account in the event fraudulent activity happens, it is not necessarily the same for all accounts or even credit card accounts.