Well, if you accept Visa or Mastercard (MC to save my typing) it is in your agreement that you wont add surcharges to customers who pay using the card. Amex doesn't prohibit surcharging, but requires that your card policy is non-discriminatory, the effect of which is that if you only accept Amex you can surcharge, but if you accept Visa or MC as well then you cant surcharge Amex, as you cant surcharge Visa or MC.
Thats what you signed up to, look in your agreement, you'll find wording like this example:
Additionally, I believe that if you are conducting business in the states of California, Colorado, Connecticut, Florida, Kansas, Maine, Massachusetts, New York, Oklahoma, Texas then state law prohibits credit card surcharges.
The goal is simple; the merchant (thats you) pays for the convenience of the customer being able to pay you, not the customer. You get paid the discounted amount. As noted in another posting, thats what you buy into when you accept plastic. Plus a per monthly charge for your terminal, plus the possibility that you may get "charged back" for transactions that the cardowner disputes. This is very likely if you put charges through without having the plastic in your hand and the customer sign the chit, the "customer not present" nightmare.
Offset from this is that accepting plastic when the other guy doesn't can get you the job. If the client doesn't have the funds available to do the work now, he may put the job on credit. Sure, one day the piper will require to be paid, but you've got your money by then, its the customers problem.
A quick google gives me this from the Texas Attorney General:
