TV Watch

L.A. Reid: I'm Leaving The X Factor

12/13/2012 at 07:10 PM EST

The latest season of The X Factor is down to the wire – but L.A. Reid is already looking ahead to next season. And he's not planning to come back.

"I have decided that I will not return to The X Factor next year," Reid, who is chairman and CEO of Epic Records, tells Access Hollywood. "I have a company to run that I've kind of neglected."

Of his decision to leave the singing competition, he says, "It saddens me a little bit, but only a little bit."

RELATED: The X Factor: Will Demi Lovato Be Back Next Season?

One of the things he's sure to miss is spending so much time with his pal and fellow judge Simon Cowell. "I love Simon. We have a great relationship," says Reid. "We have fun together, we talk, we laugh, we do bad things, we're friends."

Reid says being on the show, which also currently stars judges Britney Spears and Demi Lovato, was "a nice break" from his day job, "but now I've got to go back to work."

Reid has been a judge on the show for two seasons with Cowell. Nicole Scherzinger and Paula Abdul lasted only one season as judges before leaving.

RELATED: X Factor Shocker! Find Out Who's Going to the Finale