i don't know why but i think its only happening when i use my Dual sense Edge controller, when i more around with my keyboard everything seems fine
Unlikely, I used my DS Edge plenty of time and had zero issues.
I can also confirm GOG does not remove everything.
Do a clean install as indicated in
@LeKill3rFou's link.
Do note that your saves might not work with the vanilla game. Or they might be extremely buggy. Try loading your save after the clean install and see what happens.
If your save is borked, you have two choices:
Reinstall your mods and risk recreating the bug
Or
Start over.
Also note that a lot of mods for CP2077 can't simply be disabled. Once they've made changes to your game/save file, these changes can be permanent. Hence the extra issues when you remove them.