Camille Grammer and Adrienne Maloof
03/01/2013 AT 05:00 PM EST
Don't look for Camille Grammer
on the upcoming Real Housewives of Beverly Hills
reunion – nor on season 4 for that matter: The reality star has decided to call it quits, a source close to Grammer tells PEOPLE.
"There were some good times and some not so good times, but it's very hard to live your life out on the screen," the source says, referencing her divorce from Kelsey Grammer
, which played out on the show's first season.
Grammer's decision comes at a time when there is great uncertainty regarding the upcoming season's cast.
is also considering leaving the show, but she has not made a definitive decision as of yet," the source tells PEOPLE.
But a source close to Maloof tells PEOPLE that she is not appearing on the reunion special as well – for legal reasons.
"She's painted into a legal corner," the source says. "Because of the judgement in her divorce proceedings
she is limited in what she can say – and she will be unable to defend herself. She's very unhappy about how the show tried to paint her and made her into their evil person for simply trying to protect her kids and family's privacy." (TMZ.com
first reported the news that Maloof was not appearing on the reunion episode.)
Adds the source: "She thinks [the show has] gotten silly and petty, and she's frankly ready to move on. ... Don't expect her back for season 4."
As for Grammer, however, she has no regrets about the show says the source close to her: "She is definitely a much stronger person now than when she started."