Try stripping all of your equipped gear off, then dumping everything in your inventory into a stash. Travel to a new region (e.g. Velen to Skellige). Retrieve your stuff from the stash.
It could be a sign that some of your items may have very long itemID labels. That can slow down the inventory or create weirdness. It can happen after the game has been saved and reloaded a large number of times.
Also -- title edited for clarity.