Yes, I think Novigrad is pretty much the most crowded area in the game. Maybe parts of Oxenfurt can compare to it, but even that I doubt.
I assume that certain street corner is somehow bugged, in a way that the streaming system of the game makes NPC spawn where they are no needed, e.g. some...