One Doctor in particular was famous for his scarf-wearing.
But general people ought on the street probably won’t recognize his iconic scarf or its bold colors if you just wear it around. Whovians will. Whovians will love it. But if you want average joes to understand where you stand fandom-wise, your hint might need to be a bit more obvious.
Consider, say, this TARDIS Scarf if you’re after a little geekery to keep you warm. The blue color, Police Box design, and tassels are sure to draw the eye. Also the TARDIS is a bit more iconic, so it’s likely average people will have a better idea that they’re dealing with a Whovian. Also you’ll be nice and toasty, which is a win-win. Warning: This scarf does not have the ability to bend space and time.