To be honest, I just wish we could get into fist fights with certain civilians if we wanted to. Like, if you, without a weapon drawn, threaten a burly looking dockhand in Novigrad, you might get in a fight.
That, or I wish we can just attack guards suddenly instead of having to "provoke" them first.
Hell, I would love a crap ton more RDR style dynamic encounters involving bandits and raids to get more human on human fights. God knows there are enough wolves and drowners to fight..
There are certain things that should remain lore based, like Geralts sexuality or hair color for instance. But some things can be changed of the sake of gameplay.
It would suck if you couldn't get into fights with guards. Even though the actual Geralt wouldn't go around picking fights with guards like that. It is there for the sake of gameplay.
Civillians is a different story. With armed guards you have more wiggle room..but for civilians..I think they made the right decision keeping us from harming them.