03/14/2012 at 07:00 AM EDT
Everyone from Katy Perry
to Janet Jackson has been rumored for the two open judge seats
on The X Factor
But at least one oft-mentioned name can be taken off the list: Avril Lavigne.
"I'm making a record. I want to make movies," she told PEOPLE on Tuesday. "No, it's not happening."
At an L.A. event launching her Abbey Dawn fashion line on JustFabulous, Lavigne, 27, who performed on the program, says of the speculation, "It's flattering. It's a great show."
That now leaves Britney Spears
as the apparent frontrunner
– buzz that producer Simon Cowell
has done nothing to quell.
"You'll have to wait and see," Cowell said in an appearance on Good Day New York,
adding that a "very famous singer" and one "very famous actress" also have emerged as potential judges.
"I can't confirm or deny anything right now," he says. "The truth is, we are talking to a number of people. It changes on a daily basis to be honest with you."