Their nickname for themselves? Cumberbitches.
"It's funny and I'm flattered," Cumberbatch recently told PEOPLE during a photo shoot at the storied Algonquin Hotel in New York City. "But on a serious note, come on, give yourself more power as women," he added. "Don't just call yourself bitches!"
The 38-year-old actor, currently starring in The Imitation Game, said he knows that the fans are just being cheeky, but he still hopes the moniker doesn't totally stick.
"I know it was a joke, and more of an empowering thing. The majority of [the fans] are smart and savvy and just having fun, but it's getting out there," he said. "It sounds a bit [rude]."
"Cumbersomes? Cumbercollective? Cumberfans?" he said. "There's lots of stuff!"
Cumberbatch, who is already garnering Oscar buzz for portraying the mathematician genius Alan Turing in The Imitation Game, recently broke those fans' hearts by getting engaged to his girlfriend, theater director and actress Sophie Hunter.
So how does she handle his popularity with the female masses?
"She's just so in command of it," he explained. "It could be a difficult thing, but she's just really cool. It's really impressive."
"I like a traditional Christmas, the whole nine yards," he said. "It's all about just shutting down and shutting off for a bit. Going off the grid."
He said he may also try to squeeze in a short holiday before returning to his hectic filming schedule – but he won't divulge where.
"I have my favorite spots, but I can't tell you where they are in case someone reading this pops up and ruins it!" he said, laughing. "I do value my privacy, and so any place that has an element of that is great. But I'm partial to the Mediterranean."
For more from our interview with Cumberbatch, pick up the new issue of PEOPLE, on newsstands now