Felt the same way but was playing on Death March, so didn't want to change difficulty level. For those in the same boat, I offer up my solution. Even after reading about the good advice to counterpunch, followed by fast punch and quick back up, I would inevitably get pinned in a corner and take too much damage trying to get out of that situation. Hence, I figured it was a problem of not enough vitality to outlast him in the fight, unless I had a perfect run where I kept him on that center strip (something that never happened because PC keys are difficult to manage in fistfights in tight quarters). Luckily I had a potion with me that enabled reassigning the skill points and I went back to my hard save after the first tournament with the halfling. I then put everything into the green alchemy tree until I had maxed out the upper level skill that gives a tremendous boost to vitality by strengthening the mutagen effect (SYNERGY). With greater green mutagens attached to a full green build it boosted my vitality quite a bit. While this did not solve my fighting problem, it gave me the luxury of a few mistakes, and I defeated him within a few tries. One thing I picked up from a fellow frustrated poster was the observation that the red bar glows at the moment you should counterpunch (quick right click), because relying on the animation is not productive. Very annoying challenge to say the least, especially since I needed to respec back afterwards. Luckily though, money was not an issue for my character and it was well worth the 2000 in coin to move forward. Hell, if that had been an option I would of paid it upfront rather than spend another minute trying to defeat the arrogant elf with the self-satisfied head bob finishing animation. Through the whole process I kept feeling like this was beta tested by a group of teenagers with decked out controllers. One thing is for sure, they certainly weren't trying it on my PC and old mouse.