The restrictive stereotyping of special interests is unfortunately too widespread to intrinsically treat it as a red flag, but:
it's directly and observably an artefact of diagnostic design, which is directly and personally attributable to the likes of Simon "of course I don't want prenatal gene testing to enable wholesale abortion of autistic kids! Why, I don't want it for the specific ones who could go on to be professors of mathematics at my alma mater, Cambridge!" Baron-Cohen.
It's a key component of the "diagnosis solely by stereotype → demographic imbalance in diagnosed individuals" feedback loop. (Remember clinicians inventing the wholly fictitious "female-presenting autism" from whole cloth to excuse themselves, the instant a research study went "look at these motherfuckers provably refusing to dx women"?)
If you trace it back to the roots — and, given the state of the autism research field, you don't have to trace very fucking far — we come squarely back to Asperger, literally sorting his employment-exploitable Little Professors from the Lebensunwerten destined for the camps.
It's too widespread to treat it as the "oh, I'm talking to a neo-Nazi" red flag it fucking should be, but god it should be. Special interests are not about trains or code or Fallout New Vegas; not about what the interest is at all. It's about the way you engage with it, a qualitative judgement that you do and feel about your hobbies in a characteristic way.