Thats a shame. I managed the caretaker on normal with using quen and attacking at the right time, when he has his axe stuck etc. Iris greatest fear killed me several times, till I got lazy and turned it down to easy. On easy I had no problem killing him at all. Are you high enough level ?Maybe...