I've been following this post, silently, since the beginning and refreshing it every day, reading the input here. THere's a lot of good information here, however, I feel like everyone was so quick to dismiss Sigilfey for his hot take about how to work around the issue, that they failed to even consider the valuable information he provided in one of his responses. I know a lot of people want to hate on the mods and the company right now, but, anyone interested in the history of this issue should read this.
Sigilfey linked to a post from The Witcher 3 forums made in 2017 that highlights the extensive testing done to uncover why saved games were going corrupt. Back then, they weren't hitting a size limit, but, they were hitting something else just as terminal, and it all stems from the REDengines handling of itemIDs. Back then, during the testing, the user that did all of this testing called this a runestone bomb, because it was best duplicated with runestones.
In The Witcher 3, the REDengine assigns a unique itemID for every single item you come across, even swords that are the same sword with the same stats, even stackable items. Every runestone got it's own itemID. The engine didn't see a stack of 1000 runestones as one item stacked 1000 times, but a stack of 1000 unique itemIDs. This is only part of the problem. Another issue the engine has, is, whenever you get rid of items, or keep them in your inventory whatever, every time you save and reload your game, that stack of 1000 runestones? It got 1000 MORE unique itemIDs, and the old itemIDs were not released from your save file inventory. They existed as ghost items.
So what players in The Witcher 3 were running in to, was an issue where, over time and after hoarding everything not nailed down, their save would surpass the 16bit integer used for itemIDs, which is 65,536. When this happened, your save would be corrupted because it would cause an itemID overflow, rendering the save file unreadable.
It's important to know about the history of this issue because I believe it plays a critical role in the current issue we're having now. Players who are getting corrupted saves smaller than 8MB are potentially surpassing the 65,536 limit for itemIDs on their saves, rendering their saves unreadable. Everyone else is hitting the max file size for saves, as noted in earlier posts where the EOF is not being written to the save file, like it magically gets cut off.
The problem, is that every unique itemID is causing save file bloat. The solution is obviously some kind of garbage collector process that removes these ghost itemIDs attached to our character inventories.
However, CDPR has known about this issue, because it was brought to their attention during The Witcher3 days, and yet the issue persisted when they improved REDengine 3 to 4. Why? Both answers will likely make your heart sink.
Either they CAN'T fix it, because it's some kind of inherent flaw tied to the engine that will completely break the way it handles inventory, or, they WON'T fix it, because doing so would cause irreparable harm to the way the inventory system functions, so much so that it would cost them a lot of time and money to fix, more so than correcting the issues on console.
I don't believe this issue was left in maliciously. I believe that, if they were even aware of this issue, they never imagined that people would either 1.) Hit the save file size limit, or 2.) Would exceed the 65,536 itemID limit. While it's short-sighted, again, I don't believe this is a malicious thing. I believe there's a reason for why the engine handles itemIDs this way, and I also believe that there's a reason why this issue has persisted since the days of TW3, maybe even earlier.
All of this is a hypothesis based on the testing done in the post SigilFey linked to way earlier in this post. You can find that post
HERE and I would advise anyone and everyone read this post, because I believe this engine related issue is at the very core of what is going on.
Thank you for coming to my Ted Talk.
