It was happening to me at idle when the temp's were below 5 deg. Heat was still coming out though. Regardless, mid-point on our coolant gagues in 190 i think. So basically, if your coolant temp sensor is above one single notch off cold, you should still be able to get air warmer than ~70 degrees.
Coolant temp sensors changed position on the B5 like crazy. The 96 was different than late 97's and it changed again for 99's and again at least once for the b6 platform IIRC.
On mine its behind the block and extremely easy to change. I had to replace mine at about 110000 miles.
A bad thermostat is a possibility, but i would imagine that you'd only see temps spike cold (thermo stuck open), or an overheating issue (thermo stuck closed). Not some sort of interesting medium.
Same for a broken waterpump.
Adjusting your mix could be an interesting solution. I don't know the heat characteristics of coolant, but... If having a water rich mixture or a coolant rich mixtures causes the system to be less efficient at heat transfer that could help (in cold temps).