The pop superstar is leaving The X Factor after a single season as a judge alongside Simon Cowell, Demi Lovato and L.A. Reid.
"She has made the decision that she won't be returning," a source close to the singer tells PEOPLE. "She wants to concentrate on her music."
(Reid announced in December that he is also leaving the show.)
As the coach of the teen group, Spears's top contestant, Carly Rose Sonenclar, was a fan favorite and made it to the finale, where she finished in second place.
But Spears, 31, earned mixed reviews as a judge, with PEOPLE's TV critic saying she brought plenty of charm yet lacked the rejuvenating presence that Steven Tyler and Jennifer Lopez earlier brought to American Idol.
Fox had no comment on Spears's status for upcoming seasons.
News of her decision comes just days after Fox entertainment chairman Kevin Reilly told reporters he was pleased with her work on the show.
"I thought it went very well," Reilly said Tuesday of her stint. "It was fraught with unknowns. I think [Britney] did a good job. People are fascinated with her and always will be. Maybe some people were waiting for more drastic displays – those did not happen. The ratings were higher last year but I think the show was better [this year]. She fit in on that panel nicely."
Spears told PEOPLE last month that working on the show had been a blessing.
"It's no secret I'm a shy person," she said. "I feel doing this show over the past year has allowed me to come out of my shell a lot more. Everyone on the show has been amazing and made me feel really comfortable."
Reporting by JENNIFER GARCIA and PATRICK GOMEZ