I think there is a problem with the rendering and occlusion cameras not being perfectly synchronised, leading to objects streaming in when they should have been already loaded, or it might be that the game's streaming system is not properly configured to deliver objects and textures on time.
THIS. @Verrenus You are absolutely correct. I searched on Youtube about Umbra 3 engine and how it works and it all makes sense now. This is the reason for the pop-ins. However, I'm thinking CDPR didn't tweak and tune the software fully which is why we're getting these pop-ins. Makes sense too cause on almost every patch, they talk about "Optimization" when in fact, they're probably tweaking Umbra 3 engine so it works better on Witcher 3.
Now I tried to fiddle with the .INI files for anything that had to do with Umbra 3 rendering. Unfortunately it seems CDPR has hard locked Umbra 3's engine so no matter how much value you input in the .INI file, it doesn't change anything. Words like "Streaming", "Rendering", "LOD", "Distance" and all the values associated with it are for the most part, related to Umbra 3 engine. If only we could find a way to unlock the engine, we could tweak the values and possibly get more distance LOD, more NPC spawning and no more popping effect. Why wouldn't CDPR unlock these stuff and let us tweak it on PC is beyond me.