Asked about how he handles being between two big personalities on Idol – he literally sits between Mariah Carey and Nicki Minaj – the country star jokes, "Yeah, yeah, Randy [Jackson] and Ryan [Seacrest] are huge personalities. I do my best to keep them civil."
"Oh, and Mariah and Nicki are in there, too," he said with a hearty laugh during an interview with PEOPLE at the Television Critics Association event on Tuesday.
Still, Urban, 45, says there are bound to be disagreements given the judges' different tastes and styles.
"First of all, it's four passionate people who really care about the artists," he says. "Secondly, it's four people who just got thrown onto a dance floor without any sort of choreography and were told 'dance together.' So some toes get stepped on – and we're figuring it out."
Jackson, the only judge who has been on Idol since season 1, says it happens with every group of judges.
"Listen, since this show started there have been very different, distinct personalities," he says. "It's always been a very happy, funny, dysfunctional family."
But all is well now – mostly. "Everything is great. We're getting along," Jackson tells PEOPLE. "Me and Keith and Ryan have been fighting a lot – but that's just today."
American Idol's new season debuts Jan. 16 at 8 p.m. on Fox.