I don't think it's just a football thing. My impression has always been that the people who most strongly identify with the paraphernalia of their nation; whether that be English, Scottish, French, American, Iranian, whatever; tend to also be the most bigoted. I don't know whether it's because it lends you a rose-tinted view of your countries achievements and actions or what, but when I think of traits I'd associated with an idealised patriot (of any nation), words like "insular" and "racist" spring to mind.

This may be effectively the same thing - a minority spoiling it for the rest - but I'm not sure it's entirely to do with football hooliganism.

I'd also like to point out that I do have a national pride - I'm intensely proud of being born and living in the UK - but I'm not ashamed to say I find romantic displays of patriotism distasteful.