As someone that has had to manage quite a few people, including under the supervision of a large corporate HR department, I'll offer my take on the situation. As with everything, take what you will and do what you feel is right.
First, at his age you can't simply terminate him as if this were the 1950's. As I have been educated in the matter, and I am sure that I am not all-knowing about the subject, the only way you can properly terminate him is to provide him several months of opportunities to improve while documenting his expectations and failures. Each document should be signed by you, him, and a witness. Every meeting or discussion on the matter should have at least one witness that understands the situation. Even then, it is surprising what people can get away with in court. The only up side I have found to this process is that you rarely have to terminate someone because they will get the picture long before they reach the end of the rope.
If this process comes as a bit of a surprise I would recommend reading up on the matter so that there are no surprises when you fire that special person that feels compelled to make a legal stand.
Secondarily, I don't know your company well enough to know how many people you have working for you but it can matter in this situation. People are always a mixed bag, and the better the mix (fewer weaknesses, more strengths) the more the employee is worth. In the end the mix of employees need to balance out to meet the company's demands, with one employee's strength balancing out another's weakness. The more people in a company the easier it tends to be to live with extreme weaknesses. If you can get the things done you need done, and get them done well, then it would seem that your mix of employees works. Unless you really feel confident you can replace him with someone else who will help your company do substantially better or will work for substantially less pay there isn't much of a reason to terminate him.
There are two exceptions for which I would consider terminating him even if the business is working as it should. One would be if his attitude, when it is degrading, is being passed to other employees. This is a bad situation that can cause major issues and take a tremendous effort to repair. I have had to repair such damage in a company before and the cure took over a year of constant effort along with turning over about a fourth of the people affected. After having administered the cure I would be quick to prevent the disease.
The other exception is if this is simply not a situation you can deal with yourself. I consider coping with the personality differences and discomforts part of being a manager. Honestly, if I only hired people that I really liked I would be likely to build an unbalanced company consisting of employees with similar strengths and weaknesses to my own. Still, jokes aside, managers are humans and there are limits to what they can cope with before becoming a problem themselves. If this is the situation it can be trickier to properly terminate someone because the bigger reason for terminating the employee is the fact that he pushes the boss's buttons.
I have a similar situation myself, and as of yet I have not terminated the guy. The only difference is that instead of an attitude problem his work quality just goes through the floor. To make a long story short, in the end we get done what we need to get done, I keep pushing him on ways to improve, and for his pay all I can do is exchange him for another employee with equal but different weaknesses. In my company I already have enough people who would be more like what I would typically get for his pay, so I really *need* him to stay and balance them out. Over time I am trying to improve other people as well so that, perhaps, one day I can replace him with a more typical person.
As an old guy I work with likes to point out, only people who have never had to manage people think managing people is a great thing.
