With Steven Tyler, Randy Jackson and Jennifer Lopez on board, Lythgoe says the dynamic is "new and fresh, which is a big difference."
"[There's] real warmth and real humor," Lythgoe, 61, tells PEOPLE. "It's just a lot of fun to work together."
He says "the girls just love" Lopez, and Idol hopefuls are "dumbstruck" when they see the panel. But when it comes to filling the roles left vacant by previous judges, the Idol newbies are taking their own cues.
"They're all kind, but they're tough," he says. "You've got to look at their backgrounds and what they've been through. So they're not going to be cruel. They're not going to be like, 'Pack your suitcase – you're going home.' The integrity is going to be there."
But what about Simon Cowell's role as the top dog at the table?
"[Randy's] the one who knows what he's doing," he says. "But … every one of them is an alpha."
Aside from the fresh faces on the panel, Lythgoe says viewers can expect a few more changes, especially in terms of contestant standards.
"We're going to have all different styles," he says. "No one is going to be allowed to hide behind their guitar. A little more variety – a little more polish before they get to the American public."
Reporting by MICHELLE WARD