"I'm not going to be soft on [the contestants]," Weir told PEOPLE Tuesday at the launch of the Kardashian pre-paid Mastercard at Pacha N.Y.C. "I suppose I'd be Simon Cowell, but with a lot of Paula [Abdul] underneath."
Furthermore, the 26-year-old U.S. figure skating champ says he'll be a stickler for details. "I'm all for the theatrics and the beauty of figure skating, but at the same time, I'm an Olympian twice-over," he says. "I'm going to judge the technique the way it should be judged."
And while Weir is under strict rules not to talk to the contestants or their professional skating partners, he still has some suggestions for how the celebs can avoid injury.
"The worst thing you can do is be afraid to go down, because that's when you actually get hurt," he says. "If you're completely free, you'll be fine. That's the secret of ice skating."
But if anyone does get injured, they shouldn't expect sympathy from their judge. "The thing about figure skating is that you have to push through any injury. Nobody cares if you're hurt," he adds. "I'm not going to care if anyone's hurt. I’ve had to do it. That's what you have to do as a figure skater. So I hope that celebrities can get into that full mind set of what it is to be a figure skater and tell the world how hard it really is."
Skating commentator Dick Button will join Weir on the judges' panel with Skating premieres Nov. 22.