TW3 Texture Compression Guide - Reach for Higher Quality

+

Guest 3841499

Guest
The Witcher 3 Texture Compression Guide
Part 1

It came to my attention that many modders do not know that TW3 supports 100% lossless/uncompressed texture utilization and compression methods newer and better than common DXT5 that is used when Diffuse, Specular, Normal, Emissive, Particle, and several other texture types are selected in W3Oven (or for wcc_lite.exe). This guide explains in some detail the types of compression TW3 supports and the differences between them. There are no screenshots available at the moment. I asked several modders to provide me with their HQ import TGA/PSD or whichever files for comparison between different texture types. This topic has to stay objective. If I am mistaken/incorrect, then I welcome your correction, but please don’t shame and embarrass yourself by jumping into this thread with “ZOMFG! Monarch posted something and he’s wrong! Fake info! Enemy of modders! Let’s get on it, boys! Get your E3 Sails and let’s ride this red wave! Kek! Kek! Lock him up! Lock him up!” attitude and failing to create a negative emotional impact with expectations of some response.

DXT5 Compression:
This is a very old method and it massacres quality and details in red and blue texture channels, especially for textures with smooth gradients. It also reduces quality of green channel, but not as much as does in green and blue channels. I think it leaves alpha channel in-tact (same as import), but I could be wrong about that. There is no way to actually produce a black & white texture with DXTA compression. It will always come with color noise. It isn’t noticeable in many cases and is fine for most textures, but there are cases where imported high quality textures are visibly worse due to compression. That is why CDPR used a different, more modern type of compression for some textures.

RGBA Functions and Values:
I am not very knowledgeable in technical aspects of textures, but AFAIK, green is responsible for luminance in diffuse textures. I think that is why it was decided not to degrade green channel as much as red and blue by whoever created DXT5 standard. However, in normal maps, RGB = XYZ, where red (X) is responsible for the horizontal part of "3D-like" effect, green (Y) is responsible for vertical "3D-like" effect, and blue (Z) is like depth effect, but blue channel is not necessary for normal maps, which makes it confusing. The point here is that normal maps compressed with DXT5 provide a slightly distorted "3D-like" effect with vertical (green channel) being more accurate and refined than horizontal effect.

CharacterDiffuse/Normal vs. WorldDiffuse/Normal:
Diffuse texture type selections, be they World or Character or whatever else, use the same exact compression. Why name them differently then? I believe there is a file somewhere in game directory that lists texture loading priority and it lists Character, World, etc. types. In other words, when the game is loading textures, those sub-types determine whether the texture is loaded first or last or somewhere in the middle. Ultimately, on today’s PC’s, cooking Character vs. World would make no difference quality or performance-wise. The difference between Diffuse vs. DiffuseWithAlpha is obvious and the same applies to WorldNormal (no alpha channel) vs. NormalmapGloss (alpha channel data exists). If you cook a texture with alpha channel content using incorrect texture type that does not support alpha channel, then the process will either discard alpha channel data or corrupt the texture. This is also why I advice to never leave "Texture Type" menu selection box empty in W3Oven and always perform manual selection.

DirectX 11 Block Compression 7 (BC7):
This compression method produces significantly less noise with identical amount and placement in each channel, which makes black & white textures without color noise a possibility. It is also vastly superior compressing textures with smooth gradients, regardless of the channel, resulting in less banding and noise. CDPR used this compression method for their clouds and several other FX textures. It looks like they used this method more frequently as the game was developed after release. I think you can find more of such textures in EP1 and BOB directories than in vanilla game directory. Here's a visual comparison between Uncompressed, BC3 (same as DXT5), and BC7:
bc3_bc7.png


To import and cook textures with BC7, you need to select QualityColor for Texture Type in W2Oven. TW3 also supports several sub-types of QualityColor (BC7):
- QualityTwoChannels (BC5) – 100% lossless (I think), but blue channel must be 100% black and alpha channel is not supported
- QualityOneChannel (BC4) - 100% lossless, black & white textures only, alpha channel is not supported and only red channel is used (in other words, black & white when viewed in as a file, but red when viewed in the game – very unique and specific for FX textures).

There is an issue with QualityColor and that is alpha channel compression noise that is mild, but degrades alpha channel quality more than DXT5 does. On top of that, wcc_lite.exe has problems with compressing QualityColor textures that can result in alpha channel producing corrupt tiny clusters of 100% white pixels not meant to be there. Such clusters rarely cause any visual/detectable abnormality in the game and depending on the texture, you may get 1-5 of such tiny clusters. See here - https://drive.google.com/file/d/1xLZlnUx6I-rVor9zBxTHXJYow6z2oltN/view?usp=sharing . The only way to fix such spots is to go back to the import texture, find that same location (that resulted in corruption on the compressed texture) and alter/edit/change those pixels in some way (any tiny change would work, no difference in visible outcome has to occur). The more complex the texture and alpha channel data, the more likely you are to have such corruptions in alpha channel with QualityColor setting. I think Texture Cache Reader/Parser by Traderain (and possibly WolvenKit) cannot export/extract DDS versions of QualityColor textures from texture.cache without corruption or simply empty/no-data/back-only files. There is no problem with uncooking such files. Alpha channel degradation could be TW3-only or wcc_lite.exe-only issue, not part of BC7 standard in general.

Extra findings:
When I cooked a file with only red channel and alpha channel data (green and blue channels were 100% black) using QualityColor texture type (not QualityColorOneChannel), there was 0% compression noise in both red and alpha channels. It appears DirectX 11 BC7 Block compression noise can be affected by the number and/or channels utilized.

100% Lossless Storage/Compression:
Finally, you can have 100% lossless texture loaded in the game in TW3 if you select the right texture type. Ironically, it’s called Default. I don’t mean default texture type settings that W3Oven auto-selects based on “_n”, “_d” file name or when W3Oven leaves that field blank. There is actually a texture type labeled “Default, see screenshot here - https://drive.google.com/file/d/1GdftbBtPMrrODEXWKDIRzRcHPvGO3nfo/view?usp=sharing . Weird eh? CDPR used such lossless compression for some small-sized FX textures (radial masks, I think) and eye shadow texture. In W3Edit and WolvenKit, Default textures can be spotted in XBM files that do not display any compression type parameters. Uncompressed XBM texture files are always greater in size that TGA import files and usually 4x greater in size than textures compressed with DXT5. Be careful and always uncook mods that you cook because TW3 bundles have some sort of size limit for texture.cache and texture size and/or capacity that can result in non-uncookable mods, like HD Reworked Project. In such cases, you have to split your mods into several mod packages.

Hardware Resource Utilization and Performance Impact:
VRAM use can be greatly affected by the use of large 4K uncompressed textures, but performance in terms of framerate is generally not affected by texture compression type. My cloud mod with 4K uncompressed textures and uncompressed 4K water wave textures increased VRAM utilization by some 500MB in certain situations (reached 5GB at 1440p at some point), but with 11GB of VRAM, it was never an issue.

Personal notes, opinions, and findings:
Personally, I do not see any reason, except for saving space, not to use QualityColor for almost all textures. It’s higher quality than DXT5 and saves just as much space. If alpha channel corruption occurs, I think 100% uncompressed textures would be a better choice for normal map textures than for diffuse/specular/emissive ones. Why? Because I think visual impact is more significant and easier observed. For example, using existing alpha channel in common character details directory, I re-created the same detail normal maps with identical intensity as vanilla, but cooked them without compression and without upscaling (vanilla resolution). They were small, mostly 512x512 textures. The results were obvious if you knew what to look for – facial and body skin pores, fabric details on clothing, scratches on metal helmets, etc. As I mentioned, DXT5 severely degrades red and blue channel quality. Assuming red produces horizontal normal map effect, and blue adds to the depth of the effect, DXT5 reduces accuracy and quality of those aspects. Most textures in TW3 are cooked with DXT5 compression, which means that the vast majority of normal maps have disproportionate amount and quality of horizontal vs. vertical of normal map effect! Most games use DXT5... That’s sad…
 
Last edited by a moderator:
Hi,I did tests on the clouds and to adjust the pixelisation , it is necessary to correct each layer( red green blue and alpha) to compress then with WorldDiffuseWithAlpha.
i use Photoshop and grimp.
 

Guest 3841499

Guest
Part 2​

Flare Type Compression:
Flare texture type, just like Default, is 100% lossless storage of textures, but it does not use texture.cache. Why and what does it mean? I think no texture.cache means the texture is loaded into the game at all times, it’s never pre-cached, which makes complete sense given that the flare texture is used all the time in/for all kinds of FX, light sources, etc. There are also some other textures cooked using that method for the same exact reason. Dust Depth of Field and Focus Mode are good examples of Flare type textures that aren’t actually flares. There are also tiny 32x32 or smaller textures out there, mostly proxy textures, that wcc_lite.exe does not store in texture.cache due to their size. Un Flare type textures, those tiny proxy textures do get compressed. At the moment, with current tools, textures can be cooked using Flare type, but they cannot be uncooked to any viewable format (TGA/PNG).

TexArray Compression:
You have to use the texture type specified by the texarray.texarray file, otherwise textures end up corrupt in the game and when uncooked. There may be a possible workaround. I think at some point, I cooked some WorldDiffuseWithAlpha array as QualityColor, then unpacked already cooked texarray.texarray file, changed its parameters from WorldDiffuseWithAlpha to QualityColor and repacked it. I think it worked in the game, but it was not merge-friendly because wcc_lite.exe spit out CRC errors. It did, however, uncook TGA’s with non-corrupt textures.

DDS-Only Vegetation 32bit Compression:
To correctly cook grass and tree textures that come only in DDS format for error-free merge-friendly uncooking, you must include all the SRT files found in the directory of modded files and all the remaining (if any left/exist) of non-modded DDS files of the same directories. DDS textures must be compressed to DXT5 format to be uncookable and be placed in Uncooked directory. However, you can also use higher quality ARGB 8x8x8x8 32bit unassigned format compression (via NVidia DDS plugin for Photoshop, unsure about GIMP) to cook those textures. Such textures end up 100% black when uncooked, therefore not merge-friendly, but fully functional in the game.
Post automatically merged:

Hi,I did tests on the clouds and to adjust the pixelisation , it is necessary to correct each layer( red green blue and alpha) to compress then with WorldDiffuseWithAlpha.
i use Photoshop and grimp.

I don't know what you are talking about, but WorldDiffuseWithAlpha is DXT5 low-quality compression that severely degrades red and blue channels. It is not capable of producing smooth gradients and adds a lot of noise. It's low quality. It sucks.
 
Last edited by a moderator:

Guest 3841499

Guest
Part 3
The Unknown and Questions:
1. There are SpecialQuestDiffuse and SpecialQuestNormal texture types, where SpecialQuestNormal moves red channel data into alpha channel, leaves green channel as green channel, and it looks like it erases data in blue and red channels... I have no idea what they are all about. I do know there is BC6, a compression meant for HDR images, but have no idea if SpecialQuest texture type has anything to do with that. Any ideas?
2. In normal maps, X/Red = horizontal, Y/Green = Vertical, and Z/Blue = "Depth", but blue channel is not even necessary. Some special normal maps in TW3, like the one sofr certain clouds, have 100% black blue channel. What do you think it does? Increase image "depth / intensity"?
3. I have a hard time understanding tink mask textures ("_a.xbm/tga" texture files)... I Googled, but couldn't get a conctrete answer, especially since some sources stated that tink mask textures are either black & white or the same info can be stored in alpha channel... I noticed that Geralt's head "_a.xbm/tga" texture shows smoother gradients in red channel, but compression messed them with rectangular noise. For best quality, do I want red channel as smooth as possible? Some of these tint masks are overall pink, others blue & white, etc. It would be really appreciated if someone could explain to me per-channel meaning and function of these tint masks.
 
Last edited by a moderator:

Guest 3841499

Guest
I think this should be stickied and so should my HDRP Shampoo guide because it describes common with wcc_lite's shortcomings and workarounds. Given the state of TW3 modding, especially with the most popular mod (HDRP) being non-uncookable using normal procedures, info on workarounds in that thread and quality-related info in this one cannot be found anywhere else.
 
I'm not sure I understood well.

So Is -texturegroup=Default the best way to cook texture?

And Is it find to apply Default to every Diffuse, Specular, Normal, Emissive, Particle, etc?

Be careful and always uncook mods that you cook because TW3 bundles have some sort of size limit for texture.cache and texture size and/or capacity that can result in non-uncookable mods, like HD Reworked Project. In such cases, you have to split your mods into several mod packages.

So Does Default make trouble texture.cache? How to split mods into several mod packages?

Thanks again. It's really helpful.
 
Last edited:

Guest 3841499

Guest
I am unsure about the command interface, but Default is good for ALL, regardless of texture type - Diffuse, Particle, Normal. It simply does not compess, which makes large bundle and texture.cache sizes that cannot be uncooked without errors. You have to divide into several packages using trial and error and make sure each one can be uncooked to be merge-friendly.
 
Thank you for answer. I'm also agree with your opinion. I tested some textures using Default that are originally Diffuse, Normal, Specular, etc and it's better look and quality.
 

Guest 3841499

Guest
I would really appreciate some in-game screenshots that show the difference, even if they are close-ups.
 
Hello, MonarchX.

I have a question.
When I'm cooking textures using Default, does it still have corruption problem like QualityColor?

If do so, how to fix corruption problem?
 

Guest 3841499

Guest
Nope! Default = no corruption. To fix QualityColor corruption, you have to find it first, then go back to the original texture in Modded folder, find that exact spot (x and y coordinates) and something to it - blur it or sharpen it, just make a tiny difference, then import again, etc. It's a MAJOR PAIN IN THE ASS...
 
Oh... I don't even wanna think about MAJOR PAIN IN THE ASS... :D

So Default is the best and has no corruption.

Thanks, Master!
 
Top Bottom