"It's silly," he tells PEOPLE with a laugh. "First of all, it's completely arbitrary. Point me to 50 people online who think I'm super sexy, I'll point you to 50 more who say, 'He's old and looks like my dad.' It's not universal. If you buy into that, you're crazy."
What is universal? The fact that Hamm, who will bid farewell to his iconic character when Mad Men takes its final bow in 2015, keeps fans watching. And the actor, who stars in Million Dollar Arm, out this Friday, has no plans to stop anytime soon.
"It's been a long road," Hamm says. "And it's coming to an end. But I can't express anything but the deepest gratitude for having worked on Mad Men. It's made me a better actor. And it's taught me how to deal with a lot of stuff in life in general."
In Hamm's case, it's a life most fans don’t know much about. The actor largely shies away from the spotlight (you won't find him posting selfies on Instagram) and relishes time at home with his longtime girlfriend, filmmaker Jennifer Westfeldt, and their beloved dog, Cora.
"L.A. is a crazy city," he says. "You want to have a place in your life that you can get to that feels much more real. Mostly I try to keep my head down and go."
Still, Hamm admits, "It's nice that people tell you, 'I like what you do.' A big portion of getting to the place [I've] gotten in my career is recognizing that there is a tremendous amount of good fortune and luck. You can never feel entitled."
For exclusive photos and much more of Hamm's revealing interview, pick up the latest issue of PEOPLE, on newsstands Friday