Save files are corrupted

[pagefault404]

Stop with the fear mongering please.

He's not wrong.

It's not fear mongering. Fear mongering is encouraging the masses to be afraid of something that either isn't true at all, or by only giving out partial information and encouraging fear based on the conclusions that the masses are allowed to jump to with this limited information. For example, claiming that all aliens exist and will wipe us out as a species when they find us is fear mongering. It's giving you the information that aliens exist, and allowing you to jump to the conclusion that ALL of them will want to exterminate us. (Did anyone else hear a Dalek when they read that? Anyways)

We never were able to confirm what kind of black magic consoles are using to write the player data in memory to file. All we know for sure is that we almost never heard from console users unless it was for unrelated issues. The PlayStation console encrypts their saves, and allocates 12.5MB for the save file. We don't know if that's PER save, or if that's for manual saves and auto saves, because we would need to decrypt the file to find out, and that requires modding your PlayStation console which voids your warranty. (To be fair, voiding a warranty for modding a product you purchased is stupid, but, that's an entirely different topic altogether.) I don't think we have anyone here willing to void their warranty by modding the console for the sake of testing this.

The Xbox does things differently. It bundles all of your save files together into one large file in the preview, so you only get an overview of the total size of the files in the folder, instead of being able to drill down and see individual file sizes. Xbox users CAN, however, delete all saves but their manual save and it will show them the size of that one save file. However, everyone who has done this has never produced a save file size larger than 5.5MB - 6.0MB

So, because we were never able to confirm anything in regards to how consoles handle their player data in memory when compressing to file, we were never able to confirm whether or not this could happen to the consoles. So it's not outside of the realm of possibility that this could happen to consoles further down the line. Hell, it's not even far fetched that this could happen to consoles.

He never said it WILL happen. He alluded to the POSSIBILITY of it happening.

 

[CyberBossMan]

I understand your point but I say let CDPR handle it. They were made aware of the PC I'm sure they will update consoles if/when they need it. It's pointless to still be "concerned" about something that may not even be an issue. Let people enjoy their holiday.

Cheers!

 

[EnnyRose]

@EnnyRose I took a look at the uncompressed sizes of your save file nodes compared to a "normal" human friend of mine, listing here the ten largest uncompressed node types of the save files:
C:\Code\cyberpunk\buddy\AutoSave-3\sav.dat
PersistencySystem: 5.13 MB
PSData: 5.12 MB
CommunitySystem: 1.08 MB
questSystem: 263.49 KB
StatsSystem: 255.98 KB
DeviceSystem: 195.36 KB
DS_DynamicConnections: 195.35 KB
TransformAnimator: 132.15 KB
Animations: 132.15 KB
customArray: 96.96 KB
C:\Code\cyberpunk\EnnyRoseNearlyBrokenSave\ManualSave-29\sav.dat
StatsSystem: 22.35 MB
PersistencySystem: 7.22 MB
PSData: 7.21 MB
CommunitySystem: 1.18 MB
DeviceSystem: 433.27 KB
DS_DynamicConnections: 433.27 KB
questSystem: 308.3 KB
TransformAnimator: 219.98 KB
Animations: 219.98 KB
ScriptableSystemsContainer: 207.03 KB

I think they're not quite as far in the game as you but most things are proportional in size, but your StatsSystem nodes are just bananas.

Note that these are uncompressed sizes and I have yet to look at how they contribute to the sizes of the compressed chunks.

So uh, would you be interested in like 32 saves where all I did was craft right from the moment I could?
I was creating those saves to analyze the crafting bloat.

Stopped since the fix came out, but it could be useful if you're analyzing saves for the bloat.

 

[IAmMrSoda]

I understand your point but I say let CDPR handle it. They were made aware of the PC I'm sure they will update consoles if/when they need it. It's pointless to still be "concerned" about something that may not even be an issue. Let people enjoy their holiday.

Cheers!

I get that. And like pagefault404, I'm not saying it will. Nor am I trying to fear monger. But with so little evidence console side, we don't know it the bug is still there. So I'm trying to keep a realistic reminder that this may not be rntirely over yet. Maybe it is. Maybe console us fine. I don't know nor does anyone. It went uncomfirmable. But look at how many PC users were upset they lost their saves or are upset theirs couldn't be fixed. It will be the same once again -IF- console users ultimately get hit by the bug, likely with the added frustration of feeling that no one at CDPR tried to test if it was on console to fix it before it ever struck. If it does, yes I'm sure CDPR will fix it quick since they got an idea what to do. But it won't change any lost saves for console players who went on thinking they were safe because they were of the understanding it wasca PC only thing.

I'm all for people enjoying the game and the holiday with it. But I also don't want to see a repeat of this bug if it could have been fixed the first for all platforms. And the fact that many are still confused if it even is on console is enough for us to at least accept the possibility, for now, that it may still occur.

So I disagree with you on it being pointless. Because there's a lot on unknowns and I think it's reasonably safe for console players to at least keep an eye open for this. If it doesn't occur by mid-January then I would say the bug was only PC because by that point I have to think console would inevitably have hit it by then.

EDIT: For the record, I am a console player. I'm on the One X. So that's why I'm so bothered.

 

[LaikeBogard]

Just want to thank the dev team for addressing this problem quickly. I just wanted to know they were working on it, I never expected them to get a hotfix out before Christmas. I hope they have a great holiday.

 

[Tizerak]

It doesn't seem like they've fixed the issue with PC save files themselves. They probably just coded the game to allow 8MB+ save files instead of calling them corrupt, but there's still the issue of 6MB+ saves having a long black screen before the game begins to load. I went back and pulled up my old save folder to test the 1.06 patch and on one of the saves I loaded which was ~7MB it still had a much longer load time w/the black screen before loading as opposed to earlier saves around the ~3MB ballpark that begin loading immediately.


And before one of the millions of uninformed morons comes in claiming (and spreading misinformation) my save file is big because I "gLitChEd" or duped or hacked or whatever, please just off yourself now. It's already been confirmed that a large contributor to save file bloat is crafting and NEWS FLASH upgrading items that you enjoy and want to keep relevant requires LOTS of parts and the best way to get parts? TA DA! CRAFTING! Specifically, crafting lots and lots of mods & anti personnel grenades (in my case) then breaking them down with a fully decked out lvl 20 crafting tree. The game encourages this method with the crafting perks available and is a more reliable method of getting materials then praying to RNG to drop the materials needed, especially when that pair of legendary pair of boots you got back at level 5 that looks cool and you want to keep now requires 200+ legendary components to keep upgrading.

 

[cIass101]

Yeah it is very possible there is much to do on performance of saves files.

A good test case is this one, you get a blackscreen for one minute or so before the save starts to load.

[External link] to an 8,137Kb file,
in order to corrupt it you just have to craft some stuff, any stuff.

 

[Vinthir]

So regarding the damaged save data issue/ 8MB limit, and the 1.06 fix, I'd like to clarify a bit:

• For now we only addressed the limit, as it was the safest thing to do just before Christmas holiday. We'll continue working on improving saving in the new year.
• Hitting the 8MB limit was actually not easy at all. E.g. in one of our tests 4355 weapons & 30000 kg load were needed in the inventory to bloat a normal save this way.
• So how does it happen? Our investigation points to crafting large numbers of items (going well into tens of thousands). The modifier indicating if an item is crafted will keep being added to item type in the StatsSystem save node, gradually accumulating space in the save file. We are working on addressing this but still, hitting these numbers is not easy at all during a normal playthrough, gets easier though if one were to e.g. use a duplication glitch or a mod making crafting instant.
• To my knowledge we had no reports of exactly this issue happening on consoles, and it would be risky to deploy a just in case fix before Christmas.

And having said that, Happy Holidays, all, see you next year!

 

[EnnyRose]

So regarding the damaged save data issue/ 8MB limit, and the 1.06 fix, I'd like to clarify a bit:

• For now we only addressed the limit, as it was the safest thing to do just before Christmas holiday. We'll continue working on improving saving in the new year.
• Hitting the 8MB limit was actually not easy at all. E.g. in one of our tests 4355 weapons & 30000 kg load were needed in the inventory to bloat a normal save this way.
• So how does it happen? Our investigation points to crafting large numbers of items (going well into tens of thousands). The modifier indicating if an item is crafted will keep being added to item type in the StatsSystem save node, gradually accumulating space in the save file. We are working on addressing this but still, hitting these numbers is not easy at all during a normal playthrough, gets easier though if one were to e.g. use a duplication glitch or a mod making crafting instant.
• To my knowledge we had no reports of exactly this issue happening on consoles, and it would be risky to deploy a just in case fix before Christmas.

And having said that, Happy Holidays, all, see you next year!

Surprisingly easy to hit when you've created a macro to automate crafting, and having netflix on your second monitor to make money
Did take like 60h total gametime to hit though (before I realized I could set the time it took to finish a craft)

 

[Duskguy]

Surprisingly easy to hit when you've created a macro to automate crafting, and having netflix on your second monitor to make money
Did take like 60h total gametime to hit though (before I realized I could set the time it took to finish a craft)

I was rewatching Stargate SG-1 on netflix lol. Just crafting bounce backs and max docs.

 

[IAmMrSoda]

So regarding the damaged save data issue/ 8MB limit, and the 1.06 fix, I'd like to clarify a bit:

• For now we only addressed the limit, as it was the safest thing to do just before Christmas holiday. We'll continue working on improving saving in the new year.
• Hitting the 8MB limit was actually not easy at all. E.g. in one of our tests 4355 weapons & 30000 kg load were needed in the inventory to bloat a normal save this way.
• So how does it happen? Our investigation points to crafting large numbers of items (going well into tens of thousands). The modifier indicating if an item is crafted will keep being added to item type in the StatsSystem save node, gradually accumulating space in the save file. We are working on addressing this but still, hitting these numbers is not easy at all during a normal playthrough, gets easier though if one were to e.g. use a duplication glitch or a mod making crafting instant.
• To my knowledge we had no reports of exactly this issue happening on consoles, and it would be risky to deploy a just in case fix before Christmas.

And having said that, Happy Holidays, all, see you next year!

First off I want to say it's awesome to finally hear a bit of detail from you guys about this. Gives us more an idea of what it was and how it occurs. Mass crafting and the duping/glitching being big factors in how quick or easy it happens, with duping/glitching being the one to do it fastest, something I believe was determined to be the case by the community as well. So good to see some clarification and official confirmation there. So kudos and props to you guys for getting it fixed. Always good to see.

But if anyone could, can someone explain this bullet point to me?

• So how does it happen? Our investigation points to crafting large numbers of items (going well into tens of thousands). The modifier indicating if an item is crafted will keep being added to item type in the StatsSystem save node, gradually accumulating space in the save file. We are working on addressing this but still, hitting these numbers is not easy at all during a normal playthrough, gets easier though if one were to e.g. use a duplication glitch or a mod making crafting instant.

To me that interprets as each crafted item gets a unique modifier tag for being crafted and regardless of if that crafted item is then sold or dismantled, its crafted tag isn't deleted and the item's entry stays listed in the save. But for a normal looted or bought item, it gets its entry removed after a few in game days of being sold or dismantled. Basically crafted items continue to "exist" even after being sold or dismantled, but non-crafted items are properly removed from the entry after a few in game daysfrom when they were sold/dismantled. Is that correct?

Also while I'm glad a console related comment was made(bullet point below), I'm torn on it. I feel like this doesn't really help in determing the status of this bug on console still. Which I won't lie, disheartens me. I feel like maybe a little testing just to see if it was even possible on console should have been done. That way console users could be warned until CDPR can test more and, if need be, deploy a fix which we understand would likely not be until January(which is fine). But this just kinda comes across as because they didn't see console reports they didn't even try testing for it on consoles. And personally given then seriousness of this bug, I sorta think not at least minimally testing for the possibility on console was even more risky overall. It could have, as I said, at least given console users a heads up to keep being careful. Right now I feel there's a risk that, if the bug does show up on console, people may feel mislead or feel that console testing should have been done too like I said.

• To my knowledge we had no reports of exactly this issue happening on consoles, and it would be risky to deploy a just in case fix before Christmas.

I don't know. I enjoy the and accept its bugs and all. And trust they will get the bugs and performance fixed as best they can. But this is one point, the 8MB on console, I feel they made a wrong assessment on risk.

 

[mahony0]

So regarding the damaged save data issue/ 8MB limit, and the 1.06 fix, I'd like to clarify a bit:

• For now we only addressed the limit, as it was the safest thing to do just before Christmas holiday. We'll continue working on improving saving in the new year.
• Hitting the 8MB limit was actually not easy at all. E.g. in one of our tests 4355 weapons & 30000 kg load were needed in the inventory to bloat a normal save this way.
• So how does it happen? Our investigation points to crafting large numbers of items (going well into tens of thousands). The modifier indicating if an item is crafted will keep being added to item type in the StatsSystem save node, gradually accumulating space in the save file. We are working on addressing this but still, hitting these numbers is not easy at all during a normal playthrough, gets easier though if one were to e.g. use a duplication glitch or a mod making crafting instant.
• To my knowledge we had no reports of exactly this issue happening on consoles, and it would be risky to deploy a just in case fix before Christmas.

And having said that, Happy Holidays, all, see you next year!

At least game is playable for us now, thank you for quick patch. Happy holidays!

 

[FlorettaV]

So regarding the damaged save data issue/ 8MB limit, and the 1.06 fix, I'd like to clarify a bit:

• For now we only addressed the limit, as it was the safest thing to do just before Christmas holiday. We'll continue working on improving saving in the new year.
• Hitting the 8MB limit was actually not easy at all. E.g. in one of our tests 4355 weapons & 30000 kg load were needed in the inventory to bloat a normal save this way.
• So how does it happen? Our investigation points to crafting large numbers of items (going well into tens of thousands). The modifier indicating if an item is crafted will keep being added to item type in the StatsSystem save node, gradually accumulating space in the save file. We are working on addressing this but still, hitting these numbers is not easy at all during a normal playthrough, gets easier though if one were to e.g. use a duplication glitch or a mod making crafting instant.
• To my knowledge we had no reports of exactly this issue happening on consoles, and it would be risky to deploy a just in case fix before Christmas.

And having said that, Happy Holidays, all, see you next year!

This is an amazing reply. I had hoped the max size change was just part of the full fix, and I am really happy to see CDPR going out of their way to post their actual thought and action process in this forum post. It helps me understand and see that there is more in the works. And to have patience, because it sounds like the situation is in good hands.

Praise where praise is due. Incredibly amazing. Thank you very much!

Happy holidays to everyone!

 

[Rhyzur]

So regarding the damaged save data issue/ 8MB limit, and the 1.06 fix, I'd like to clarify a bit:

• For now we only addressed the limit, as it was the safest thing to do just before Christmas holiday. We'll continue working on improving saving in the new year.
• Hitting the 8MB limit was actually not easy at all. E.g. in one of our tests 4355 weapons & 30000 kg load were needed in the inventory to bloat a normal save this way.
• So how does it happen? Our investigation points to crafting large numbers of items (going well into tens of thousands). The modifier indicating if an item is crafted will keep being added to item type in the StatsSystem save node, gradually accumulating space in the save file. We are working on addressing this but still, hitting these numbers is not easy at all during a normal playthrough, gets easier though if one were to e.g. use a duplication glitch or a mod making crafting instant.
• To my knowledge we had no reports of exactly this issue happening on consoles, and it would be risky to deploy a just in case fix before Christmas.

And having said that, Happy Holidays, all, see you next year!

I'm making grenades right now as I'm typing this. Crafting is extremely tedious. I'm sure crafting and then destroying purple grenades might seem like an exploit, but just to get one purple mat, you need to click and hold hundreds of times if you were to convert materials as the game intended, and costs bloat way too much even if you spec heavy into crafting.

 

[Extreme96PL]

So regarding the damaged save data issue/ 8MB limit, and the 1.06 fix, I'd like to clarify a bit:

• For now we only addressed the limit, as it was the safest thing to do just before Christmas holiday. We'll continue working on improving saving in the new year.
• Hitting the 8MB limit was actually not easy at all. E.g. in one of our tests 4355 weapons & 30000 kg load were needed in the inventory to bloat a normal save this way.
• So how does it happen? Our investigation points to crafting large numbers of items (going well into tens of thousands). The modifier indicating if an item is crafted will keep being added to item type in the StatsSystem save node, gradually accumulating space in the save file. We are working on addressing this but still, hitting these numbers is not easy at all during a normal playthrough, gets easier though if one were to e.g. use a duplication glitch or a mod making crafting instant.
• To my knowledge we had no reports of exactly this issue happening on consoles, and it would be risky to deploy a just in case fix before Christmas.

And having said that, Happy Holidays, all, see you next year!

Almost no crafting at all i only crafted ammo for my revolver once and upgraded it twice however i noticed that most grown of save file was when i started Delamain quest and when i was driving around the map unlocking fast travel points etc. I also noticed that sometimes bodies dont dissapear same with loot but sometimes both bodies and loot dissapear but loot is still visible on minimap. My latest save hitted 5.1mb i did some side quests, gigs and NCPD events and i am in act 2 need to meet with Takemura before parade. Anyway thanks for removing the limit at least i am not worried now that my save will corrupt before ending the game merry christmas

 

[lex34]

My game crash today again with 1.06 patch, i played 2 hours, ps5 user

 

[Zao.540]

• So how does it happen? Our investigation points to crafting large numbers of items (going well into tens of thousands). The modifier indicating if an item is crafted will keep being added to item type in the StatsSystem save node, gradually accumulating space in the save file. We are working on addressing this but still, hitting these numbers is not easy at all during a normal playthrough, gets easier though if one were to e.g. use a duplication glitch or a mod making crafting instant.

Thank you for sharing this, it definitely resonates well with the hex soup I'm staring at.

It's always super fun when data that's supposed to be reasonably sized grows beyond assumptions and expectations.

In most of the saves I've got lying around this node is like 50-200 KiB uncompressed, while Enny's at 22.35 MiB and Molte's linked by class101 above sits at 13.75 MiB. Even when they compress to like 20% of the size, that's still a hefty piece of data to sneak under the limit.

If we (without evidence) assume that the number of entries in that node are roughly proportional to the size of the node, that's around a hundredfold increase in entry count for the fun saves. I'm speculating wildly here of course, but if loading a save does some amount of work for each entry, 100x more entries could mean 100x more work.

I'm sure the clever folks at CDPR will figure out a way to improve load times, even if it might involve changing the way such things are represented and persisted to file, but it's my impression that it's way more complicated work than what I assume is relatively straightforward disk I/O routines.

Everything around persistent data is precarious and I'm impressed that a robust fix came out this soon, well done!

 

[IAmMrSoda]

Thank you for sharing this, it definitely resonates well with the hex soup I'm staring at.

It's always super fun when data that's supposed to be reasonably sized grows beyond assumptions and expectations.

In most of the saves I've got lying around this node is like 50-200 KiB uncompressed, while Enny's at 22.35 MiB and Molte's linked by class101 above sits at 13.75 MiB. Even when they compress to like 20% of the size, that's still a hefty piece of data to sneak under the limit.

If we (without evidence) assume that the number of entries in that node are roughly proportional to the size of the node, that's around a hundredfold increase in entry count for the fun saves. I'm speculating wildly here of course, but if loading a save does some amount of work for each entry, 100x more entries could mean 100x more work.

I'm sure the clever folks at CDPR will figure out a way to improve load times, even if it might involve changing the way such things are represented and persisted to file, but it's my impression that it's way more complicated work than what I assume is relatively straightforward disk I/O routines.

Everything around persistent data is precarious and I'm impressed that a robust fix came out this soon, well done!

Would you be willing to explain that bullet point to me? Lol I'm a console user, but I still would like clarification on what it means. Least for my own understanding. I posted what I thought it meant above in another post, but I'll copy my understanding below and hopefully you can help correct me where I'm wrong(which is likely a lot lol).

My interpretation from earlier post:

To me that interprets as each crafted item gets a unique modifier tag for being crafted and regardless of if that crafted item is then sold or dismantled, its crafted tag isn't deleted and the item's entry stays listed in the save. But for a normal looted or bought item, it gets its entry removed after a few in game days of being sold or dismantled. Basically crafted items continue to "exist" even after being sold or dismantled, but non-crafted items are properly removed from the entry after a few in game daysfrom when they were sold/dismantled. Is that correct?

 

[Rhyzur]

Would you be willing to explain that bullet point to me? Lol I'm a console user, but I still would like clarification on what it means. Least for my own understanding. I posted what I thought it meant above in another post, but I'll copy my understanding below and hopefully you can help correct me where I'm wrong(which is likely a lot lol).

My interpretation from earlier post:

To me that interprets as each crafted item gets a unique modifier tag for being crafted and regardless of if that crafted item is then sold or dismantled, its crafted tag isn't deleted and the item's entry stays listed in the save. But for a normal looted or bought item, it gets its entry removed after a few in game days of being sold or dismantled. Basically crafted items continue to "exist" even after being sold or dismantled, but non-crafted items are properly removed from the entry after a few in game daysfrom when they were sold/dismantled. Is that correct?

Basically the game keeps receipts. So if you sell something you got money for that and it accounts for how you got that money. Or if you destroy something it keeps track that you destroyed something and what you destroyed in order to get that something in return. Basically rather than having a clean slate every single time you load in it just keeps printing receipts.

 

[Zao.540]

@IAmMrSoda I don't have enough information gathered to say much about that and there's room for multiple interpretations from what was being said.

Some information about some aspects of items or their types is kept around or duplicated, either for some gameplay reason or by accident.

@Rhyzur's analogy is quite sound.