Paula Abdul & Nicole Scherzinger Won't Be Back on The X Factor
updated 01/31/2012 AT 01:00 PM EST
•originally published 01/31/2012 AT 09:45 AM EST
The shakeup that's shaping up for season 2 of the Fox singing competition.
On the heels of the news that Steve Jones was leaving the show as host, judge Nicole Scherzinger is out as well, according to EW.com. And Paula Abdul was also let go from the show, a source confirms to PEOPLE.
Abdul had a tough time mentoring the groups, becoming the first judge without contestants on season 1. And Scherzinger faced serious backlash after Crow's elimination.
With the women off the judges' panel, record producer L.A. Reid and Simon Cowell, who is also the executive producer of the talent competition, are the only familiar faces that remain. There is no word yet on their replacements.
Speculation about a possible casting change began brewing in December, although Jones originally brushed off talk of his exit as "a fabrication."
A week later, Cowell told PEOPLE he would be making some changes for the sophomore season. And so he did.
A U.K. rep for Cowell could not comment on reports of the departures but said that "discussions and negotiations" are standard after a first season ends. "This happens after every show or series," he added. "No decisions have been made that I am aware of."