Why do we need "boss encounter"?
Good question. I am about to show you how you answered your own question though:
The boss fight is epitome of the level/game. It's made challenging to make you feel like you have achieved something great once you overcome it. I strongly believe it is a relic of ancient times from beginning of gaming, when there was nothing but gameplay. But games have outgrown it and have become much more than just the actions we do. I feel that narrative is the driving force behind games especially when it comes to role playing games. What we are trying to achieve from the boss fight is satisfactory conclusion to the narrative.
I believe you are partly right, 'Boss Fights' are incorporated into many forms of media, (video games, book, movies and TV series among them,) to provide a sense of satisfacton and closure. They provide a 'claimax' or sorts. I for one do not think the concept is outdated at all.
Heres why...
Cyberpunk, in my opinion, is the game which is in its essense about people and stories of their struggle with unfair, unjust and uncontrollable world. A story where they try and get something done right for once, mostly for themselves or those close to them like Strange Days, Judge Dredd, Robcop or Appleseed. And like it was said before, Cyberpunk is a world of "Bosses" set in equilibrium and a new guy would be torn to pieces, the other side of the coin is that none of the bosses are safe from each other and all the player character need to do is expose the weakness and watch as other bosses stream to it like sharks to blood. In such a scenario sending an email or making a phone call is as much of resolution as anything else. Bullet to the head might as well be mercy killing at that point.
Strange Days - the Hotel scene towards the end is an unorthodox 'Boss' encounter.
Judge Dredd - (1995) Dredd fights Rico. (2012) Ma-Ma's apartment. Both classic 'Boss' material.
RoboCop - Boddicker get's arrested and ED-209 gets shot up. Classic 'Boss' encounters again.
Appleseed - (1988) A. J. Sebastian & Calon - classic stuff right there. (2004) Save the D-Tank by stoping the mobile fortresses - unorthodox, but still a 'Boss' enounter. (2007) Kestner is a bit of an odd one, but Halcon is blatantly 'Boss' material.
Yes, Cyberpunk 2020 is chock full of 'bosses', from lowly Gang Leaders and Corrupt Officials to Mega Corp CEO's and Heads of Secret Societies. I would not use the word equilibrium, because the powerbases of 2020 are always in a state of flux. New guys rise and the old guard fall all the time. Trusted Executives often orchestrate their rise by 'eliminating the competion' and 'freeing some space at the top'. Cyberpunk is
THE Dog-eat-Dog World.
The means used by various groups and organisation to achieve this sort of goal varies wildly. However, for the player to be a part of it, they need to be taking some form of action. This could be an execution, (which could take many forms and is a hot topic in and of itself,) an economic, (such as depriving someone of their assets or otherwise bankrupting them,) or perhaps a more intellectual approach, (such as blackmailing the target or framing them for something.) These are just some of the possibilities I would like to see for dealing with situations.
All in all this comes down to a very simple, though often overlooked, proverb:
"Use the right tool for the right job."
There is a place for strong opponents and difficult combat situation but they shouldn't be the "Boss fights" at the end of the level.
There shouldn't be any 'levels' in 2077, but missions, (or whatever other name you wish to use for the tasks the player should be seeking to complete,) do need an objective. I am certainly not advocating the use of a pitched battle between the player and some 'super bad guy' at the end of every task, but this sort of conflict does have it's place, as you said, and I believe it does have a place in Cyberpunk.
It shouldn't always be the "crescendo" of a mission. I admit that element of the 'Boss' encounter is outdated. Perhaps taking out a 'Boss' could be the first step of a fight, then the player would have to fight their way through irrate thugs to get away. How about a 'Hitman' style approch where the player must infiltrate a location and plant a device or some 'evidence' to kill or incriminate the target?
TL;DR
Bosses are not the problem, how they are dealt with is.