agree with you here, what I'd like to see implemented is dots on the horizon so you can just run or ride to it. A yellow dot for current objectives and a green dot for waypoint marker. Maybe even use witcher sense for them to show up. But like you say, a quick button/toggle option for the map would be great. I play without the minimap whatsoever so another option would be very welcomed
I also tried playing without the minimap, but just like the newest ES, the quests are built with an extremely extensive use of compass and minimap in mind, so much so they don't even give you clues as location of stuff sometimes, so it becomes just frustrating in some situations.
I then switched to minimap always ON for lack of middle options, but especially for those playing on a plasma like me, it's also not very good for the panel, due to possible retention issues. Thankfully there's often cutscenes/meditation/whathaveyou interrupting the image, so it's a very minor issue.
Still a quick toggle option would be best. I find tracking and traveling in third person to be much harder than first, so I'm unable to just open the map and then mentally plan my voyage. Also, the roads in Witcher 3 are realistically uneven and curvy, and there's abundant vegetation/hills/bumps and lack of land markers (again very realistic), making it ever harder (unlike let's say Skyrim, where it's easy to travel the world without any map at all after a few hours)