I believe it was a guesstimate for how many "big" mods it would take to put the engine into a state that might prevent it from loading. Or, perhaps that's the maximum number of remaining functions the engine can keep track of by design. It's sort of moot as you're arguing from massive hindsight. When the REDKit was released, there were no tools like Mod Merger, no one had yet tried to cook various assets together, and the actual toolset provided did not include the ability to do that. I have to say that they might have underestimated people's dedication to getting their mods working with the game.MonarchX;n10520322 said:Yes, and the question is whether this whole load/offload aggregation is related to the 12-mod limit? Why do you think they did it?
SigilFey;n10515592 said:Most notably, there is a finite amount of memory (RAM) and other system resources that any engine is capable of utilizing. This is not governed by the amount of RAM on your system, but by the amount of RAM that the engine itself is coded to use.
SigilFey;n10520632 said:the maximum RAM required to load the game assets + mod assets ever exceeds the maximum RAM available in the engine at any instant
Because these functions were introduced by modders -- that's not the way the mod kit was designed to work. You're arguing hindsight again. (Meaning: you're superimposing your present day understanding of modding, following years of tools and techniques being developed by the community, and saying, "So why didn't CDPR do it this way from the beginning?" That's a fallacy of reasoning. Kind of like saying, "Well, why didn't Napoleon just create armored tanks, then he could have won at Waterloo!" Because tanks didn't exist back then.) Even though CDPR has never explained exactly what the situation with the REDKit was, I get the distinct impression that it didn't work out as originally intended. I also get the impression that, at some point during the development of TW3, the decision was made to pull people off of the project, likely before it was"ready". This is all speculation on my part. As stated, CDPR has not made any sort of formal statement about this. But these things happen: Limited hours, limited people, limited funds. Contracts, limitations, agreements. Terms, deadlines, and legal concerns. Like all business in any industry. Can't have everything. If the REDKit had been developed further, perhaps it would have included such tools and workarounds by default. Maybe the engine itself would have been re-worked to allow for higher limit, or the limit could have been based on the total "size" of assets that needed to be simultaneously loaded. Who knows? In the end, that's not what happened.MonarchX;n10520692 said:No, still makes no sense because it's all under an assumption that mod COUNT has more influence on the game performance/function than mod CONTENT. 15 mods with a single 128x128 texture each will prevent the game from loading, but my giant 3.2GB MergedPack makes the game load faster than vanilla (due to one of the mods disabling store videos). Where's the logic in that? Besides, you can cripple performance with scripts alone or even settings.
Possible, and this would be leaning toward the "upper engine limit" idea. Most engines have a maximum number of functions / variables that can be loaded into it at one time, and TW3 is definitely putting the engine through its paces under vanilla conditions. It's also somewhat dependent on the "worst-case scenario" the game can produce. (Meaning: the busiest scene with the most processes simultaneously running under the heaviest graphical load that player will ever experience throughout the course of the game.) Limits on engines are sometimes set against such situations, as no mods can ever exceed the limits of what the engine is capable of doing for that particular situation...even though a larger selection of mods would work just fine in every other part of the game.MonarchX;n10520692 said:There is some kind of PER-BUNDLE limit. For example, HD Reworked Project 5.0 (HDRP5) has very large and heavy textures. If you try to uncook it, you get OUT OF MEMORY error during the process. There is no way to merge it to mere mortals, but those who seek the way tend to find it. The way around that is to use Cache Viewer...
Was this a question?Murzinio;n10523232 said:
https://github.com/freebsd/freebsd/blob/master/sys/contrib/octeon-sdk/cvmx-malloc/malloc.cSigilFey;n10524312 said:How much RAM will I have available in the game? Answer: LESS than 8 GB. The engine must obviously load its operating environment, and that alone is going to gobble up, say, 1 GB of RAM. Permanently. That means that even though my system has 32 GB, the engine is only written to recognize and utilize a maximum of 8 GB. It's 100% impossible to tell a program, "Just use unlimited RAM, loading unlimited processes into memory." The program won't know when to stop and will max out all available RAM at the speed of electrons, almost instantly crashing the whole system. I must always program a minimum operating environment and a maximum operating environment. Have to. Computers can't "think". Only calculate mathematical equations.
It's still an upper limit. This is another fallacy that has wound up causing problems for the gaming industry...since...forever. "256 TB of RAM!!! That's...insane. It's so much that we'll never hit that limit! It's seriously not even worth thinking about."Murzinio;n10525292 said:...unless you have more than 256TB of RAM or you're using a 32bit OS.
Right. It can. And there are engines that will do that. TW3 doesn't. Obviously, there is reason for that. We might not like this reason (whatever it actually is). We might want to argue with that reason. We might even be able to come up with an alternative solution to get around the limit (which many already have). But that does not mean that there is "no reason", nor that the existing reason is invalid.Murzinio;n10525292 said:C++ allows you to either write wrappers or overload the new/delete operators to make your own management system, and it can be easily implemented and made configurable for different platforms/future versions of the engine. And with some config tweaks you can bump the RAM and VRAM usage way over vanilla values so it's either adjustable (there are budget options in config files, though I don't remember if they affect anything in the release build) or set higher for PCs to the point of not being an issue.
Two totally different considerations. The hindsight involves TW3 modding when the REDkit was released vs. as it is now. Whatever the reasons / motivations were (which we don't know), the fact remains that the tools for present day modding techniques did not exist when it was released. Especially if modding was never considered / set aside / not possible to include in the dev process, that's even more of an argument for what I was saying: that the devs involved never considered such approaches, even though have become commonplace between 2015 and 2018. And the Napolean analogy stands -- as modding for TW3 is not seemingly a direct parallel for modding anything else. Hence, all the trouble authors are having. That means that new things had to be created to deal effectively with it. We lost our "Battle of Waterloo" back in the day, but we've got some mean tanks now.Murzinio;n10525292 said:Besides, the scripts and leftovers in wcc clearly show that the modding support for the game wasn't even seriously considered, it's not like modding exists since yesterday so they couldn't make the design decisions with it in mind, so the comparisions with Napoleon and tanks make absolutely no sense.
Probably a lot of them. I don't code. What little "scripting" I've done has been self-taught and largely by wrote. (I never learned the foundations, just reverse-engineered the pieces I needed, trial-and-error style, until I got them to do what I wanted.) I'm completely amateur, but still wrote a bunch of stuff for Morrowind between ~2003-2007. I was able to pull off some pretty cool stuff, too! But I never studied the vocabulary in any depth. If there are any glaring issues, feel free to PM and I'll edit. (Hopefully the gist gets through.)Murzinio;n10525292 said:Also you're confusing different technical terms in your examples...
That's a long read, and I'm not familiar with the "code-y" parts of it. (Only read the first bits, then skimmed around a bit.) From what I gather it's a memory manager for the 64-bit era? Basically, it ensures that no chunks of RAM are wasted?traderain;n10525392 said:https://github.com/freebsd/freebsd/b...alloc/malloc.c
Meh. I oversimplified -- like I said I was going to. (You were warned! ) But in the early Pentium days, limiting the Hz (frequencies) is exactly what he had to. Even DOSBox still offers the ability to downclock the CPU by a certain percentage, as running the actual legacy programs still suffer from the same issues. In the beginning, games were created to run "optimally" on very particular hardware. The only way the engine would perform 100% correctly was to have that precise Hz of CPU and the exact amount of RAM it was designed for. So the engine was designed for a specific amount of resources, and having more / less could and would cause issues (even though most were minor enough to be ignored easily until the hardware got waaay more advanced).Murzinio;n10525532 said:Also, you can't really "design" a program to run at specified clock speed independent of the hardware. Unless you maybe set the clocks when it runs but it would be totally pointless. Different CPUs will have different "speed" on the same clocks, there's much more to that than the frequency. So that would be a terrible idea to rely on for your game time. If you want to limit the framerate you use precise timers to measure the time on each tick in the game loop and render the frame when the desired time has passed, like 1/60 of a second to get at most 60 fps... And for game logic/physics usually you just use a fixed time step. So again, that's not how it works.
And what does that have to do with anything? Where did I imply any of that? You completely missed the point, which was meant to show that current games are far from the adress space limit. That is 48bits btw, if we will need to we will expand to full 64bits which allows for 16EB adress space, so you don't have to worry about that.SigilFey;n10526252 said:It's still an upper limit. This is another fallacy that has wound up causing problems for the gaming industry...since...forever. "256 TB of RAM!!! That's...insane. It's so much that we'll never hit that limit! It's seriously not even worth thinking about." Kind of like a conversation I had with a bud back in the early 2004. "8 GIGAbytes of RAM!!! Why the @#$%! would you waste so much money on that, man!? There's no way any game is going to come out that requires 8 GB of RAM! You're just wasting your money. That's...insane." I can also go back to the early 1990s, when my father brought home an IBM computer from Kodak with the 286 microprocessor (which weren't released on the open market yet) and 2 MEGAbytes of "above-board memory". And it was so amazingly powerful that all of the games we had could run, like, perfectly. It was...insane. And yet it was never more that 3 years before these "insane" systems were starting to show their age, and newer games with "absolutely mind-blowing features" came out that required me to turn down graphics options in order to get them to run smoothly. Assuming that the way computer systems will work is somehow going to change because "we've broken all boundaries and achieved a whole new plane of technology" will pretty directly lead to disappointment. There are always limits. We have from absolute zero to the speed of light in this existence...and that's it. (That'll be plenty for our gaming needs. )
Maybe you should read my post again. The only way these limits could exist is when you implement them yourself. So W3 engine already "does that". I already explained that you can see for yourself reaching memory usage much higher than console hardware limits/PC vanilla is possible with some config tweaks, which shows that any limits are not likely to be a problem causing the mod limit issue.SigilFey;n10526252 said:Right. It can. And there are engines that will do that. TW3 doesn't. Obviously, there is reason for that. We might not like this reason (whatever it actually is). We might want to argue with that reason. We might even be able to come up with an alternative solution to get around the limit (which many already have). But that does not mean that there is "no reason", nor that the existing reason is invalid.
SigilFey;n10526252 said:I don't code.
No offence but I can clearly see that, yet you don't have problems discussing topics you don't understand and making completely wrong statements about them. You should read up design on patterns used in games (most importantly the game loop) http://gameprogrammingpatterns.com/ and try implementing them by making some simple games in C++ (and make sure you understand the memory management) with something like SFML (pretty much a wrapper for OpenGL), when you finish and read your posts again you will see you had a lot of strange misconceptions.SigilFey;n10526252 said:I'm completely amateur
Wrong guess. malloc is a heap allocation function from C standard library. (C++ improved malloc by adding type safety and handling the size automatically) You can use it to write a program with 5-6 lines of code that will allocate entire available memory on your PC without any black magic.SigilFey;n10526252 said:That's a long read, and I'm not familiar with the "code-y" parts of it. (Only read the first bits, then skimmed around a bit.) From what I gather it's a memory manager for the 64-bit era? Basically, it ensures that no chunks of RAM are wasted?
That's not the case for a long time now. It's not an oversimplication, it's simply wrong... We're discussing a 2,5 year old game, old software you're talking about had completely different issues. Game development is much different today.SigilFey;n10526332 said:Meh. I oversimplified -- like I said I was going to. (You were warned! ) But in the early Pentium days, limiting the Hz (frequencies) is exactly what he had to. Even DOSBox still offers the ability to downclock the CPU by a certain percentage, as running the actual legacy programs still suffer from the same issues. In the beginning, games were created to run "optimally" on very particular hardware. The only way the engine would perform 100% correctly was to have that precise Hz of CPU and the exact amount of RAM it was designed for. So the engine was designed for a specific amount of resources, and having more / less could and would cause issues (even though most were minor enough to be ignored easily until the hardware got waaay more advanced).
If you don't have your framerate independent from game logic/physics then you're doing it wrong or you're still talking about old games when no one bothered to consider that. Which is not the case today, the same games run on at least 30-144FPS on different platforms and often you can go much higher if you have the hardware.SigilFey;n10526332 said:Most notoriously, unlimited FPS, which throws games for a loop sometimes since it's running too fast and falling out of sync with anything requiring frame-timing.
Again, it's not something you "must" do. Memory budgets are only helpful to avoid running out of memory if you don't have a lot of it available, but then you can easily adjust them. If you want you can write an engine without that. And you can write a game aiming for unlimited framerate (where frame is presented as soon as it is ready) without a problem, programmers figured out problems like that years ago.SigilFey;n10526332 said:Engine must set min/max levels.
...even remotely relates to anything that I was suggesting about the current state of TW3's mod limit. In fact...you've said exactly what I was saying using precise numbers.Murzinio;n10527332 said:...current games are far from the adress space limit. That is 48bits btw, if we will need to we will expand to full 64bits which allows for 16EB adress space, so you don't have to worry about that...
"it" = engines that allow for more RAM, higher / larger "address space limits", more powerful CPUs, [more-better-gamey-stuff-that-goes-real-fast-like], etc.SigilFey;n10526252 said:What you say about future versions is spot-on! It's definitely possible, and it will eventually be implemented in future titles. That's what tends to happen as the industry creates new engines and games!
I never once said that things could not be done differently. I never once said that things could not be done better. I simply said that engines can and do contain limitations. If those limitations are in place in the code, they become an upper limit. For example, there's a limit on TW3's modding ability, and it's there for a reason. Really insisting that it should have been done differently does not negate the fact that it was done the way it was done.Murzinio;n10528702 said:1: The numbers I mentioned are hardware limitations, not engine limitations. I don't want to explain the same stuff 3 times...
2: Sound channels - ok I guess. 32/64 bit - you just change the compilation target and you can compile both 32bit and 64bit versions. If you have all the dependencies available for both and don't mess up the code it's not a problem. So kinda true, but you just need to compile it with different target, and if you follow good practices for writing code you won't have to change anything in it.
Memory usage is not fixed at the compilation by itself, you can allocate memory dynamically depending on size of a file that program opens for example, you don't have to know anything about it beforehand, it's a really basic thing to do, without that software would be very limited... And it's called "dynamic memory allocation" for a reason, one of it's uses is precisly when you don't know how much memory you will need when you write the code. Like I said you can write a few lines of code program that will demonstrate this, without specifying anything about RAM. And your program doesn't need to "recognize" the RAM, the operating system handles that, manages the available memory, checks for access violations etc. C++ only provides an abstraction, you don't allocate every single byte manually.
Not being able to use faster CPUs doesn't make sense either, if the CPU can execute the same instructions faster, the code will run faster and even if you have locked framerate, it will just reduce the CPU time the game uses because the computations will be ready sooner and you will only check the timers. For unlimited framerate you will just get more fps. If what you say is true you wouldn't be able to see a difference on newer CPUs or even after overclocking. That only applies to being able to utilize different instruction sets or number of threads but you definitely get a difference with higher frequency or newer, optimized CPU generations.
I know I've seen this rock before!Cercaphus;n10528692 said:I think we came full circle now
Because I tried to explain to you how all of it works... And you continue to argue with points that I didn't even made.SigilFey;n10528892 said:I asked two questions above, very clearly, but you have not directly responded to either of them. You seem tunnel-visioned on criticizing the simple reality of the situation by repeatedly arguing the way it could have been. Obviously, that's not the way TW3 was built. There's a mod limit. If you truly want to reject everything I've suggested because I used a wrong term or over-simplified the explanation of a process, then you're welcome to reject it. You seem to be convinced.
It's not some "method" you can use or not, it's the way to allocate something on heap in C or with new in C++ (either directly or with smart pointers, but you have to use it if you want something on heap, no way around that). And it doesn't work by filling your entire memory, that was an example to show it can if you want it to.SigilFey;n10528892 said:However...if it's so simple, why do we have so many different methods being used throughout the industry? Why doesn't every single developer under the sun use malloc with perfect efficiency to manage their heap allocation? Why isn't every engine 100% flawless in its execution, able to make complete use of 128 GB of RAM?