If I may clear some things up - Geralt (and probably every witcher) was wearing only the steel sword on his back (grip above the right shoulder

). As somebody said, the silver sword was a very expensive, rare and delicate weapon (silver is a soft metal, so he never used it with humans and non-magical monsters, as it would quickly get chiped out). Besides - if he would show it to everyone he met, he'd quickly run into trouble (Oh my gods! It's made out of silver! Let's get it!)
And does that mean that he wore his steel sword on his back normally?
Yes, it does. If there was any chance of getting into trouble (outside, in the city, in an inn etc) of course. Not in the bath or when having sex

Just use common sense - Sapkowski writes a very "realistic" fantasy, so think about it - if you would wear a sword, Geralt would probably too.