When I was chair, I had to deal with a similar situation. My adjunct faculty member was in his 80s, already badly slipping mentally, and then underwent heavy chemo for cancer, which further drastically altered him mentally. He would get lost in the hallways, forget where he was going, spend much of the first 10 minutes of class packing and unpacking his briefcase and fumbling with papers, either reteach the same material several times or skip entire sections, couldn't reliably put two sensible sentences together (and he was teaching our lowest dev ed, who couldn't afford to have someone who didn't have it together as their teacher). But, of course, he was an old crony of the prior chair and beloved among the "in" clique. I limped along with him that first fall, then "didn't have any classes" in spring (and our enrollment was truly down); come fall, I didn't call him back, and caught holy hell from him, cronies, and all for taking out my (non-existent) revenge against prior chair on this guy, being ageist, etc. etc. etc.
The real trigger came when I did an observation of his class one day, and he said any number of incredibly offensive off-color things that, had his students been savvy enough, could have gotten him, our dept., and the school sued to high heaven for racial discrimination/hate speech. (I kept jumping in to rephrase/correct/redirect him, but finally ended up taking over the class and sending him to the office on the ruse that he had to leave early for a dr. appointment--awful, I know, but the best I could come up with on the spot. I then had a long talk w/the class to smooth things over.)
On the other hand, I'd also been this guy's supervisor in prior years. He was OK, but not great, then. I knew from those days, and through that fall, that students weren't being served, but neither the chair nor dean wanted to hear it because they didn't want to deal with it. I had his students in my office regularly--and I mean, at least one a week, often more--complaining about his teaching and/or scared to death he was going to drop over dead in the classroom, and what should they do if he did? (They weren't being cruel: this guy looked to be at death's door, and I similarly worried.) When I became chair, I asked if I could get him out of the classroom, and the dean was glad to let me--she knew it needed to be done. Of course, this was just one of the many sins I committed as chair, and that's fine: he had no business "teaching" anymore.
Personally, I felt like hell about having to do this. I really liked him, and to this day (if he can remember me) I'm sure he hates me. Still, my replacement won great acclaim by immediately hiring him back--and what the hell, they're just students, right? In any event, I did what had to be done for the good of all concerned, but it would have been miserable, had I not just put it at arms'-length and done it (i.e., if I'd given a damn). A good chair does what has to be done, though, even if it's hard, and even if it's not popular.
I guess I just wanted to offer a perspective from the other side of the door. And I do agree: the chair/dean need to step in. Being nice is not always being kind; often it isn't.