I quite like hats but I don't wear them, unless the weather is really bad. I like hats on women too, but I notice that few wear them. Perhaps they have the same reaction to them as I do, namely that they make your head scratchy.
I notice too that some of the glitterati like to wear, what we in England used to call flat hats or cheese cutters. They were always the province of the working class or the upper class who wore them while shooting. I've got one sculling aorund but I wouldn't dare wear it. I used to wear a gray bowler a long time ago, but eventually I felt stupid and stopped. It was a good trademark for a while though.
It's worth noting that baseball hats seem to need instructions on how to wear them. Young men in particular feel it necessary to put them on incorrectly; let's all repeat "The Peak Goes in the Front!" I think we've run the gamut of trying it to the side and the back and various places in between. It's over, let's get back to normal!