Save files are corrupted

[Acksios]

I did some extra testing. I'm going to conclude this for now. It's been interesting. I've missed my work, so a little puzzling was alright, but I'm expected back at work tomorrow so I'm going to enjoy the last couple of hours of my evening

The posts with testing:

Starting off:
https://forums.cdprojektred.com/ind...-are-corrupted.11052596/page-14#post-12479720
Testing bloating and trying to change save file size:
https://forums.cdprojektred.com/index.php?threads/save-files-are-corrupted.11052596/page-15#post-12481214
Some other tests:
https://forums.cdprojektred.com/ind...-are-corrupted.11052596/page-16#post-12482888

I think most people shouldn't be too worried:

Save files are corrupted

My save is 32mb, I've dismantled every piece of junk and gun since lv1, only crafted something like 10 items however.. I've had friends craft for money, it seems a case to case basis and RNG whether or not you get fuckerino. Maybe you mean it is 3.2 mb?

forums.cdprojektred.com

I've looked through TW3 files. Not much to see there. My TW3 saves are way smaller, so I'm not even going to bother to try and replicate stuff there. No configurations. There was something about purging items from memory on an interval, so I figured I test disassembling and selling again but then wait till the game autosaved. Didn't change a thing.

Save files for Witcher games were mostly elusive too. So I didn't really find anything special.

I saw some posts about people not being able to open their stash. They had to open the stash on their vehicles to get it back to work. So a far stretch was that something was getting loaded back in again. Tried some moving around from my inventory to house and vehicle stash. Throwing away half the grenade stack and then moving the other items around a bit. That also didn't really change anything. There's definitely something weird going on when transfering items from and to your stash, but that's a whole other issue I guess.

So; for now:

Straight and fair: Most users won't encounter this situation. The users that do encounter this situation can encounter it while playing the game in a normal way. You don't need to be duplicating items or anything. If you love crafting and on top of that love throwing grenades, you're in a group of people that might encounter this issue. You'll need to, in my personal opinion, use the crafting option eeeeeeeeeextensively. I can definitely imagine people doing that. And it's a feature in this game. So it has got to be dealt with.

Without any further breakthroughs and no code at my finger tips I'm fairly certain I won't find exact reasoning and won't find a work-around either.

Cheers

I have to reply and give a big thanks for all your work! Much appreciated!!
Cheers

 

[igrvks]

If this is your argument then better rework the whole crafting tree perk.

There's a perk when you disassemble items with 15% chance to get a free component, there's a gain more components while disassembling perk, reduce cost by 30%, reduce upgrade cost by 20%, grants 20% to craft an item for free, grants 10% of free upgrade, reduce upgrade costs, chance to acquire free component while upgrading and there's still perk progression rewards every other level that adds the percentage in this. So then they better destroy this whole damn tree.

This is the most mind boggling aspect of this entire oversight in my opinion as well. If this crafting system was indeed designed and implemented like this how is it possible no one at QA thought to try out what happens if you really push these perks to the limit? We have a community member in this very topic who was able to find out exactly what happens in one hour of testing (Huge thanks to FlorettaV by the way) so we are not even talking about weeks of stress testing or anything.

And if these crafting perks are not something that the team intended one must ask how is it possible it was ever implemented, because just reading through those perk descriptions tell you right away it's a literal money and resource printer that completely breaks the game. Also when you consider the rate of XP accumulation within the perk tree it's obvious that a player has to craft literally thousands of items to reach level cap for Crafting skill.

If there is indeed some hidden design intention of CDPR that players should not play this game for hundreds of hours, do much crafting or even gather resources I must join Ignacioantonio in his statement that the entire crafting skill must be either redesigned to follow the design ethos of CDPR or scrapped entirely.

As for all the community members in this thread doing the amazing work of testing I have not only my infinite gratitude and respect for devoting your free time and labor for looking into a issue that CDPR should had, but also two questions you might want to consider while testing:

- Does the amount of shards you pick up affect the save size? I must have picked up tons of duplicate shards during my playthrough now since the game doesn't tell you they have been already read and I'm really into the lore so I want to read everything, and since we have determined the entire ItemID issue are those duplicate shards also contributing to the bloat even though you can only see one of each in your inventory? Should the players stop looking at in game lore as well while CDPR determines if they might do something about this?

- Since it has been stated that enemy bodies etc. left within the game world might affect the bloat as well would it be worth testing a scenario where player creates two new saves and plays for example one hour into the story with other save going lethal and other going ghost (no knockouts or kills, so no bodies either) following the exact same route? Should the players stop killing or incapacitating enemies as well while CDPR determines if they might do something about this?

 

[Rawls]

Several posts deleted. Again, the topic of the thread is NOT "the quality of other users' posts." People who keep up with the personal back and forth are gonna get kicked.

 

[ahtis89]

Please fix this. My save file is already over 7mb and don't wanna progress no more if this happens to me also.

 

[Duskguy]

be aware that the game runs fine. It will just not load anymore. Meaning if you've hit the hard cap at the start of a session, you could play an entire day, erase all your loadable quicksaves, autosaves, and if you only save manually at the end of your session, you could easily lose 10+ hours of gameplay.
If they're particularly sticklers for space, they may even delete their old manual saves, and only have 1 or even worse, they may only rely on autosaves and quicksaves, and suddenly have lost their entire playthrough.

Lost 114 hours doing exactly this. I've had previous games that take longer to load with more save files.

My screen was hanging at a black screen before actually loading. I had 10 manual saves all the quick saves and all the auto saves. So, I deleted most of them without thinking of backing them up.
What could go wrong?

Yeah, then I died in a mission hour and hours later... save file damaged.

I've since loaded and am playing a file from, oh, about 110 hours earlier.

Moral is, if you enjoy the game, back up your save files. Dont delete them, just move then to another location just in case.

Being so early, I did some tests, (check back on to around page 11 on this thread). Basically:

Stashing items still counts as inventory. Indeed it seemed like placing items in and out of the stash marginally increased the file size. Perhaps thinking each stack of components was "new".

Crafting items, even a few, seems to offset the component cost (when building a few green snipers). Until you start pumping out more than a few. Consumables such as grenades still increased the file size. Using those grenades shrunk the file a little, but the file was still larger. Selling the sniper rifles and then resetting the vendor showed a still small increase in file size.

FlorettaV has since done quite a bit more testing with much more components that seems to show the same thing. Check around pages 15 and 16.

Hopefully we get an answer about this topic now that so many review sites are picking it up. Even though most of them are focusing on bashing the dev's, or that it's due to cheating rather than the actual issue. Saw one site claim moderators were speaking as cdpr devs. Another claimed Tom's hardware discovered the issue first (though it's not even the first article published on the issue).

In the mean time I'm going to continue to play the game and keep track of my save file sizes based on what I do. And the most recent was afk fist fighting an elevator wall overnight. (Stupid athleticism skill). That save file didnt change size before and after.

 

[TalentX]

Without any further breakthroughs and no code at my finger tips I'm fairly certain I won't find exact reasoning and won't find a work-around either.

Cheers

Great job over the testing back then. Saved me quite some time, as I was about to do the testing too.

However, the reason why it gets corrupted is kind of obvious, I will try to explain:

So it is pretty much confirmed, that there is a cap at 8MB file size on the save files.
Once you reach a state within your progression that exceeds 8MB, the information saved into the save file will, counter intuitively, cut at 8MB leaving the rest in "/dev/null".
However, the missing information in the save file is relevant, as the game, upon loading the save file, is expecting information at the end of the lines (think of it as information written in text form sorted in lines) where it was cut due to the 8MB limit ultimately leading to a corruption notice.

And that's pretty much it causing the corrupted save file. In order to fix this issue, the limit of 8MB has to be removed or the game has to track down the save file in a way that it can never exceed the file size of 8MB. Least would also mean that you will never be able to have items beyond the predefined limits, for example 999999 Eurodollars, or 999 stacks of items etc. depending on what types with which limits they use (strings, integers, floats, boolean etc.).
Another Edit: In addition items that the player can store must be limited too, using actual slot. Storage as well as inventories must have predefined slots for how many items the player is capable of keeping around (Not the weight number, but the acutal amount of different items - the weight could be done in addition to the slots which normally is even the case in most other RPGs). As of now there is only a weight limit that only determines when you cannot move your character anymore, but inventory as well as storage is offering space that is infinite which automatically scales depending on how many different items you carry along.

Concerns regarding long loading screens, and removed/used items from the progression still taking up their spaces in the file size is another matter (which also has to be addressed of course, but these are not causing the corruption initially).

As you mentioned in on of your earlier posts, we have to wait for a fix and keep a track on the save file size during progression every now and then I guess.

 

[vegetaalex66]

[...] It took my playthrough (about 120 hours, completed all missions including NCPD ones) an extra 1 hour and 15 minutes of fully exploiting the weakness in the save files of this game to come close to reaching the file limit.

[...] My initial save was at about 6.229KB. That was my playthrough where I looted everything I came across. [...]

I'm doing a completionist run and am also picking up absolutely everything. I was worried if I have to change that habit, so these points were really reassuring for me. Thank you Also for all the effort you put into testing things, of course.

btw as a PSA for everyone, disassembling individual green MaxDocs gives a huge amount of crafting xp, so if you still want to level crafting to make a few legendaries here and there it seems doable without bloating the save files too much.

 

[Ershin]

Can someone share an almost corrupted save?

 

[EnnyRose]

Can someone share an almost corrupted save?

Sure, here's one at 8.185KB only 7 short of the deadzone
This is as close as I've gotten.

 

[botwinka]

So what I've been reading here and what I know regarding file IO handling (I'm a programmer and a game developer, just not at CDPR), this looks like an issue of buffer overflowing.

If that's true, then it would explain everything, starting from the magic number 8,192kb (2^13), through corrupted savefiles not being recoverable, as per official response, and why the game is entering unresponsive state.

1) 8,192kb (2^13) is a neat number, a clean number and a power of 2 compliant; programmers like power of 2s, so that makes me believe it's a hardcoded limit: a buffer size;

2) buffer is an intermediary between save file and game memory. First the game loads byte by byte into the buffer, then it reads the buffer and pick up words to make up the gamestate. And saving is simply putting gamestate word by word into the buffer, than byte by byte into the file. So if the gamestate exceeds buffer size, the information outside of the 8MB limit is forever lost if saved to a file; hence saved files that are of 8,192kbs are forever corrupted, which holds true to the released official statement.

3) why the game enters endless unresponsive state is basically the how a parser is made. A kind of save file reader, if you will. Common practice is making an endless (while) loop, which would read word by word from the save file, until reaching a special character called EOF: End of File. Then it would exit the loop and restore the gamestate from that savefile. But because buffer was overflown while saving, that means that special character is not present, and never reached; hence endless reading.

That is all IF the buffer size assumption is true. I don't know for sure if it's the case here, and what I've written above is nothing but an educated guess. Rembember, I'm not a CDPR dev, so I don't have the source code or the tools to test my hypothesis.

 

[Weaver2077]

[SaveConfig]
AutoSaveEnabled = false

This should do the trick.

In what file is this option?

 

[guticogo]

Well guys honestly it's your fault you should not use crafting or loot anything, you should not even start the game or even buy it, just watch a stream, preferably of one of our sponsored streams like CohhCarnage.

 

[TalentX]

So what I've been reading here and what I know regarding file IO handling (I'm a programmer and a game developer, just not at CDPR), this looks like an issue of buffer overflowing.

If that's true, then it would explain everything, starting from the magic number 8,192kb (2*2^12), through corrupted savefiles not being recoverable, as per official response, and why the game is entering unresponsive state.

1) 8,192kb (2*2^12) is a neat number, a clean number and a power of 2 compliant; programmers like power of 2s, so that makes me believe it's a hardcoded limit: a buffer size;

2) buffer is an intermediary between save file and game memory. First the game loads byte by byte into the buffer, then it reads the buffer and pick up words to make up the gamestate. And saving is simply putting gamestate word by word into the buffer, than byte by byte into the file. So if the gamestate exceeds buffer size, the information outside of the 8MB limit is forever lost if saved to a file; hence saved files that are of 8,192kbs are forever corrupted, which holds true to the released official statement.

3) why the game enters endless unresponsive state is basically the how a parser is made. A kind of save file reader, if you will. Common practice is making an endless (while) loop, which would read word by word from the save file, until reaching a special character called EOF: End of File. Then it would exit the loop and restore the gamestate from that savefile. But because buffer was overflown while saving, that means that special character is not present, and never reached; hence endless reading.

That is all IF the buffer size assumption is true. I don't know for sure if it's the case here, and what I've written above is nothing but an educated guess. Rembember, I'm not a CDPR dev, so I don't have the source code or the tools to test my hypothesis.

I don't know if you have read my earlier post as well, but that's exactly what I thought too. I am pretty sure it is the lead.

The way how to solve this however might be a concern that may not be fixable so fast, because apparently once the file reaches a specific size, the game has trouble loading through it (not yet talking about reaching the buffer size limit, but it generally takes longer when it's larger than 5,5MB if I remember it correctly) and also items that are no longer present within the character's environment, nor the world environment (Shops reset themselves after 2 in-game days) the size doesn't get smaller at all.

Cheers
TalentX

 

[Tracido]

So what I've been reading here and what I know regarding file IO handling (I'm a programmer and a game developer, just not at CDPR), this looks like an issue of buffer overflowing.

If that's true, then it would explain everything, starting from the magic number 8,192kb (2*2^12), through corrupted savefiles not being recoverable, as per official response, and why the game is entering unresponsive state.

1) 8,192kb (2*2^12) is a neat number, a clean number and a power of 2 compliant; programmers like power of 2s, so that makes me believe it's a hardcoded limit: a buffer size;

2) buffer is an intermediary between save file and game memory. First the game loads byte by byte into the buffer, then it reads the buffer and pick up words to make up the gamestate. And saving is simply putting gamestate word by word into the buffer, than byte by byte into the file. So if the gamestate exceeds buffer size, the information outside of the 8MB limit is forever lost if saved to a file; hence saved files that are of 8,192kbs are forever corrupted, which holds true to the released official statement.

3) why the game enters endless unresponsive state is basically the how a parser is made. A kind of save file reader, if you will. Common practice is making an endless (while) loop, which would read word by word from the save file, until reaching a special character called EOF: End of File. Then it would exit the loop and restore the gamestate from that savefile. But because buffer was overflown while saving, that means that special character is not present, and never reached; hence endless reading.

That is all IF the buffer size assumption is true. I don't know for sure if it's the case here, and what I've written above is nothing but an educated guess. Rembember, I'm not a CDPR dev, so I don't have the source code or the tools to test my hypothesis.

Hmm, so even if a line of code is found to alter this, it may not function without patching, is how this sounds..

I'm doing a completionist run and am also picking up absolutely everything. I was worried if I have to change that habit, so these points were really reassuring for me. Thank you Also for all the effort you put into testing things, of course.

btw as a PSA for everyone, disassembling individual green MaxDocs gives a huge amount of crafting xp, so if you still want to level crafting to make a few legendaries here and there it seems doable without bloating the save files too much.

I haven't even opened the city from lockdown (just met Jackie at the bar for the Dex meet) and I'm nearing 3MB. If I were you, I'd just keep your eye on it, where your saves are located.

 

[Duskguy]

So what I've been reading here and what I know regarding file IO handling (I'm a programmer and a game developer, just not at CDPR), this looks like an issue of buffer overflowing.

If that's true, then it would explain everything, starting from the magic number 8,192kb (2*2^12), through corrupted savefiles not being recoverable, as per official response, and why the game is entering unresponsive state.

1) 8,192kb (2*2^12) is a neat number, a clean number and a power of 2 compliant; programmers like power of 2s, so that makes me believe it's a hardcoded limit: a buffer size;

2) buffer is an intermediary between save file and game memory. First the game loads byte by byte into the buffer, then it reads the buffer and pick up words to make up the gamestate. And saving is simply putting gamestate word by word into the buffer, than byte by byte into the file. So if the gamestate exceeds buffer size, the information outside of the 8MB limit is forever lost if saved to a file; hence saved files that are of 8,192kbs are forever corrupted, which holds true to the released official statement.

3) why the game enters endless unresponsive state is basically the how a parser is made. A kind of save file reader, if you will. Common practice is making an endless (while) loop, which would read word by word from the save file, until reaching a special character called EOF: End of File. Then it would exit the loop and restore the gamestate from that savefile. But because buffer was overflown while saving, that means that special character is not present, and never reached; hence endless reading.

That is all IF the buffer size assumption is true. I don't know for sure if it's the case here, and what I've written above is nothing but an educated guess. Rembember, I'm not a CDPR dev, so I don't have the source code or the tools to test my hypothesis.

To help you, or anyone else interested in looking, I'm uploading one of my "damaged" files from patch 1.04.
Also, can confirm that since patch 1.05, the "damaged file" message no longer occurs and the endless black screen does.

Note: a completely different save that has nothing to do with the uploaded one is reporting as corrupted in the load game menu. It's a recent one from the new playthrough... no idea why it's corrupted. Luckily I'm now constantly making new manual saves instead of overwriting old ones.

 

[botwinka]

I don't know if you have read my earlier post as well, but that's exactly what I thought too. I am pretty sure it is the lead.

it appears I was in the middle of writing my own post when it came up.

 

[warbaby2]

It's a legacy RED engine bug... they should have known that would happen when implementing that lackluster crafting system. Ludicrous...
...well, I guess we aren't supposed to play the game for too long or get involved with the systems and world too much. Consume main story - wait for DLC... hey, if that was the original intention for how the game was meant to played, then I get all the missing stuff, you wouldn't get to see it all anyways before your saves go bye bye.

 

[TalentX]

Note: a completely different save that has nothing to do with the uploaded one is reporting as corrupted in the load game menu. It's a recent one from the new playthrough... no idea why it's corrupted. Luckily I'm now constantly making new manual saves instead of overwriting old ones.

While I generally recommend to do new manual saves regardless, because sometimes you may want to go back further, instead of being stuck with your "always-current" progress, it shouldn't matter whether creating new manual saves or replacing old ones in this concern. The game always writes a new save file internally (replacing an old save bascially deletes it and writes a new file with the same name).
I think the problem here is the amount of saves present in the CP77 save game folder and something related with the "user.gls" file located there, that may cause additional corruption on overwriting save files.

 

[Tracido]

While I generally recommend to do new manual saves regardless, because sometimes you may want to go back further, instead of being stuck with your "always-current" progress, it shouldn't matter whether creating new manual saves or replacing old ones in this concern. The game always writes a new save file internally (replacing an old save bascially deletes it and writes a new file with the same name).
I think the problem here is the amount of saves present in the CP77 save game folder and something related with the "user.gls" file located there, that may cause additional corruption on overwriting save files.

Oh boy, I remember this old issue from The Witcher 1.. I had to make folders to keep extra saves in to not break the game when re-loading, and I save so much, because of glitches in the first place which break games, it was a nightmare, still have lost folders all over on backup drives from back then, hah.. That's never been dealt with? Sigh..

 

[selticus1978]

I never had any trouble with The Witcher 3 on the ps4 no crashes no corrupted saves.