Overcoming the Mod Limit (maybe)

+
Overcoming the Mod Limit (maybe)

First of all if this has been talked about before please link it, because I have not been able to find anything on what i am specifically talking about.

Whilst working on my Vlad UI Mod I was experimenting around with wcc_lite after I noticed that the vanilla game tends to divide up its bundles in categories, there is an r4gui.bundle, xml.bundle etc... etc..
So i attempted to do the same for a mod:

I took one of my older mods and bundled all the xmls in one bundle and all the redswfs in another.
I generated metadata for the bundles folder and wcc_lite detected both bundles and added it to the mod.
Folder structure was like so:

-modName
-----content
---------bundles
-------------xml.bundle
-------------r4gui.bundle
-------------metadata.store
---------texture.cache
---------en.w3strings

Game launches no problem, both bundles are loaded and i can use the mod perfectly fine.
---------------------------------

Because of this I had the hypothesis that what actually causes a 'mod limit' has nothing to do with the number of bundles. The vanilla game already has ~50.
I think it's to do with the number of metadata.store files (which actually references each bundle and the files within it.) Obviously with a large number of mods you begin to add a lot of these metadata files, hence you can reach the mod limit.

To test this I started to create a 'merged' mod. I took quite a few of the mods I had lying around, and moved the blob0.bundle from each one into a single mods bundles folder (modMerged/content/bundles) and renamed it to the name of the mod. Delete the existing metadata file from the mod, and move the rest into the mods folder like you would any other.
I would then generate a metatdate.store file for this new merged mod bundles folder, which went fine, wcc_lite reported the bundles in the cmd prompt, and successfully generated the file.

Launching the game had no issues and I can detect a file from every bundle in the merged mod. Either by command or by seeing an actual in-game change.

So i took this to the extreme and eventually I got up to 35 bundles in content/bundles, each from a separate mod. Metadata was generated no problem, wcc_lite reported 35 bundles and again launching the game had no issues and I could detect a file from each bundle in the merged mod.

---------------------------------

Again I'm not sure if this has been done before, and whether or not this truly overcomes the mod limit. To me it does because I have definitely run into the mod limit with fewer than 35 mods (with bundles) installed.
Not to mention this also is a far simpler method of merging two or more mods together without the need to unpack each mods' blob0.bundle, merge the unpacked files, and repack.

---------------------------------

P.S I do not want this thread to turn into a direct discussion over the mod limit itself, i,e why it exists and complaining about it. If you want to do that you can do it here.
 
Last edited:
you might be on to something here! Maybe a short tutorial video so others can try to recreate this more easily and share their results?
Great work
 
This is just a basic tutorial to merge two mods, but it is very simple to do anyway.
Ofc this will only work if both mods bundles do not have any conflicting files inside.

For explanation sake I will use two simple mods: modA and modB.

-modA
-----content
---------blob0.bundle
---------metadata.store

-modB
-----content
---------blob0.bundle
---------metadata.store

Make a new mod folder, which i will call modMerged
Give it a folder structure like so:

-modMerged
-----content
---------bundles

Copy the blob0.bundle from modA into the bundles folder and rename it to identify it ( I will just call it 'a' ).
Do the same with modB.

So you should have this:

-modMerged
-----content
---------bundles
-------------a.bundle
​​​​​​-------------b.bundle

You will now need to run wcc_lite and execute metadatastore over the bundles folder to generate one.

You can use the following code in a batch fiie:
Code:
cd <path-to-exe-dir>
call wcc_lite metadatastore -path="%~dp0bundles"
pause
Create a new txt file in modMerged/content (eg, generatemetadata.txt)
Copy that code in, making sure to replace <path-to-exe-dir> with the dir of the wcc_lite exe.
Save, exit and rename the extension from .txt to .bat.
Run it and you should see wcc_lite do its thing.
It should detect both bundles and generate a metadata file.
Code:
Building the metadata store took 0.0 sec ( 2 bundles, 8 file entries )

Once done you should finally have this:

-modMerged
-----content
---------bundles
-------------a.bundle
​​​​​​-------------b.bundle
-------------metadata.store

Copy that into your W3 Mods folder.

That's pretty much it, you can now start the game and both mods you merged should work.
This should also work to DLC mods, it's the same process, just treat my example as DLCA, DLCB and DLCMerged.

P.S If the mods you are merging have other stuff, like w3string files texture caches etc... then create a new folder in modMerged/content,
call it something like 'modAcontent' / 'modBcontent' and copy the rest of the two mods into those.

Will look something like this:
-modMerged
-----content
---------bundles
-------------a.bundle
-------------b.bundle
-------------metadata.store
---------modAcontent
-------------en.w3strings
-------------texture.cache
---------modBcontent
-------------en.w3strings
-------------texture.cache
 
Last edited:
Thank you very much for detailed describred, will definitely try out, one question though.
What about buffers0.bundle, how should I handle with these?
Should I move them to ModXYZContent aswell?
 

Guest 3841499

Guest
It would be great if someone would update Mod Merger to do all this automatically as it would make merging so much faster. I am not sure how conflicts would work within different bundles.

Since Mod Merger allows custom commands, maybe that can be used to have it merge only the metadata files?

 
Amazing ! I've managed to merge the following mods :
-aerondight redone (not the mod itself, but the scabbard addon)
-brighter toussaint
-clear toussaint lighting
-dodge like e3 witcher (the blob0 first, but some animations were missing (geralt was just sliding to the next location) so I merged the "buffer0" too and it works fine)
-Scabbards (same, blob0 and buffer0, all working fine)
-tweaks
-WLM

It run smoothly with other mods, here is what I've done :
Install every mod I wanted (approximatly 90-100)
Script merge all
Test the mods mergeable with mod merger and move them to another folder
Merge the simple mods only containing a blob0, metadata and sometimes buffer0, all whilme keeping the unmergeable ones (nitpicker, rct, bootstrap, sharedimports...) in a modmerged file with your technique
adding again the other mergeable mods and merging them through mod merger.

It took some time but it worked in the end, game launched, save loaded and no crash ingame (appart two times when I entered the inventory a bit quick, don't know why).

THEN, seeing all was working fine, I tried installing another mod, mergeable with the mod merger : Un-merge my previous pack, added the mod and re-merged all.
Now the game won't launch (infinite loading in task manager), so I tried merging some DLC files (since the mod I added worked with one) and now the game launches in the task manager but closes after a
 
*few seconds.
I'm launching it through gog, like before. The "play" icon turns grey, and back to green after a few seconds and I can see the game opening and closing in the task manager.

Anyone got the same problems ?
My main one is basically the infinite background loading, not the save loading.
 
There are some way to merge mods like Mod Merger. I've never encountered script mod limit but only texture mod limit. so Mod merger bypass it
 
There is supposedly a limit on characters per script, that's tied to some windows wonkyness. But I can't find anything that backs up that statement.

E3 Dodge is one of the buggiest mod used by people now days. I've heard loads of problems with that.

Also, @Overay35 Why are you posting on this thread? Because it seems you are using Mod Merger, and not the Lim3zer0's merger method, which is this whole thread about. The method is like 10 times better than Mod Merger anyway in my humble opinion.
 
There is supposedly a limit on characters per script, that's tied to some windows wonkyness. But I can't find anything that backs up that statement.

E3 Dodge is one of the buggiest mod used by people now days. I've heard loads of problems with that.

Also, @Overay35 Why are you posting on this thread? Because it seems you are using Mod Merger, and not the Lim3zer0's merger method, which is this whole thread about. The method is like 10 times better than Mod Merger anyway in my humble opinion.

At some point, the game shits itself if a single function is too big, but we're talking really really big. Had this happen in W3EE, where enemies get their stats assigned in the scripts. The function that held all those stats got too big at some point, which led to occasional crashes when npc's got spawned in. Fixed by simply splitting that one function in multiple parts.

I really doubt any other mod has functions that large.

Beyond that, I strongly doubt there's a limit to how much script a mod can have. I think Reaperrz would have found it ages ago if that were the case.
 
Okay, I did everything on your list that has to do with the bundles. I tried it with 44 mods, fist spoiler tag has screenshots of the folder structure.

modMerged Folder
modMerged Folder.png

modMerged_Content Folder
modMerged_Content Folder.png

modMerged_Content_Bundles Folder
modMerged_Content_Bundles Folder.png
So after setting up the folder structure like you suggested I then created batch file and ran it. Below, in the next spoiler tag, is the result of running it.
D:\modMerged>cd D:\Witcher3ModTools\bin\x64

D:\Witcher3ModTools\bin\x64>call wcc_lite metadatastore -path="D:\modMerged\bundles"
[2018.08.12 07:57:03][Info][WCC] Starting commandlet 'metadatastore'
[2018.08.12 07:57:03][Info][WCC] ---------------------------------------------------------------
[2018.08.12 07:57:03][Info][WCC] Bundle input directory: 'D:\modMerged\bundles\'
[2018.08.12 07:57:03][Info][WCC] Store output path: 'D:\modMerged\bundles\metadata.store'
[2018.08.12 07:57:03][Info][WCC] Loaded 0 bundles with 0 entries
[2018.08.12 07:57:03][Info][WCC] Ignored censorship for 0 files
[2018.08.12 07:57:03][Info][WCC] Analyzing buffers....
[2018.08.12 07:57:03][Info][WCC] Number of buffers: 0
[2018.08.12 07:57:03][Info][WCC] Number of files with buffers: 0
[2018.08.12 07:57:03][Info][WCC] Max buffer ID: 0
[2018.08.12 07:57:03][Info][WCC] Exported 0.00 KB of text (discardable)
[2018.08.12 07:57:03][Info][WCC] Exported 0 bundles (0.02 KB)
[2018.08.12 07:57:03][Info][WCC] Exported 0 entries (0.02 KB)
[2018.08.12 07:57:03][Info][WCC] Exported 0 files (0.03 KB)
[2018.08.12 07:57:03][Info][WCC] Created 0 buffers (0.00 KB)
[2018.08.12 07:57:03][Info][WCC] Created 0.01 KB in initialization data
[2018.08.12 07:57:03][Info][WCC] -------------------------------------------------------
[2018.08.12 07:57:03][Info][WCC] -- Per extension stats for normal files
[2018.08.12 07:57:03][Info][WCC] -------------------------------------------------------
[2018.08.12 07:57:03][Info][WCC] -------------------------------------------------------
[2018.08.12 07:57:03][Info][WCC] -------------------------------------------------------
[2018.08.12 07:57:03][Info][WCC] -- Per extension stats for buffer files
[2018.08.12 07:57:03][Info][WCC] -------------------------------------------------------
[2018.08.12 07:57:03][Info][WCC] -------------------------------------------------------
[2018.08.12 07:57:03][Info][WCC] -------------------------------------------------------
[2018.08.12 07:57:03][Info][WCC] -- Compression statistics for totals
[2018.08.12 07:57:03][Info][WCC] -------------------------------------------------------
[2018.08.12 07:57:03][Info][WCC] -------------------------------------------------------
[2018.08.12 07:57:03][Info][WCC] Building the metadata store took 0.0 sec ( 0 bundles, 0 file entries )
[2018.08.12 07:57:03][Info][WCC] ---------------------------------------------------------------

D:\Witcher3ModTools\bin\x64>pause
Press any key to continue . . .

As you can see, it didn't bundle any of the mods and it created a new folder inside of the modMerged folder. As seen in the next spoiler tag

New Bundles Folder Auto Created.png

So, what have I done wrong?
 

Attachments

  • modMerged_Content_Bundles Folder.png
    modMerged_Content_Bundles Folder.png
    82 KB · Views: 356
Your Directory structure is wrong in these pictures.

In modMerged/content/bundles you should have all your bundles, no additional folders.

Rename blob and buffer files:
AdditionalArmorStands_blob.bundle
AdditionalArmorStands_buffers.bundle

and move them DIRECTLY to modMerged/content/bundles/

And for the rest of the files texture.caches, .strings and such in logic like:
modMerged/content/modAdditionalArmorStands_content/ (I think you have it already like this)

And run the wcc with this and not the one that you did:

wcc_lite metadatastore -path="D:\modMerged\content"

Delete D:\modMerged\bundles that's propably empty right?
 
Your Directory structure is wrong in these pictures.

In modMerged/content/bundles you should have all your bundles, no additional folders.

Rename blob and buffer files:
AdditionalArmorStands_blob.bundle
AdditionalArmorStands_buffers.bundle

and move them DIRECTLY to modMerged/content/bundles/

Ahh, I was reading it as they need to each be in their own folders. Now it makes sense


And for the rest of the files texture.caches, .strings and such in logic like:
modMerged/content/modAdditionalArmorStands_content/ (I think you have it already like this)

Yes

And run the wcc with this and not the one that you did:

wcc_lite metadatastore -path="D:\modMerged\content"

Okay, I'll change it in the by converting it back to a text file, then back to a Batch file

Delete D:\modMerged\bundles that's probably empty right?

Should Be

Thanks for your reply and help
 
Top Bottom