Gilbert Flores/Celebrity Photo
As the world continues to get to know a new side
of Charlie Sheen, where are his four young children in the midst of all this?
Authorities removed his twin boys
, Max and Bob, nearly 2, from his care Tuesday night and returned them to their mother, Brooke Mueller, after she won a restraining order
. Soon afterward, Sheen urged Mueller
on NBC's Today
show to "tell me where our sons are."
A source close to Mueller tells PEOPLE the boys are currently living in an L.A.-area rental apartment with her, along with Mueller's mother Moira Fiore and a nanny while Fiore waits to complete a purchase on a condo.
"Charlie kept the kids away from Brooke on purpose, and now that they are back with Brooke she is very happy," a family source close to Brooke tells PEOPLE. (Sheen has called her claims "a frigging lie"
As for his daughters – Sam, 7 and Lola, 6 – they're at home with their mom, Denise Richards
"Sam and Lola have not been to Charlie's house in quite some time, and they have not met [Sheen's live-in girlfriends] the Goddesses
," a source tells PEOPLE. "They live with Denise. They're doing well. They aren't aware of what’s going on. They're continuing with their normal routine."
This all been "very upsetting" for Richards, says the source.
"It’s a very difficult situation," the source explains. "This isn't the man she married, he’s a completely different person. She’s supportive and she'll always be there when he needs her help, but it’s really hard."
Reporting by LINDA MARXFor much more on Charlie Sheen, including the childhood influence of his father's Vietnam war epic
Apocalypse Now, his thoughts on his exes and his prediction about the future of
Two and a Half Men, pick up this week's issue of PEOPLE, on newsstands nowPlease note: Comments have been suspended temporarily as we explore better ways to serve you. Your opinion is important to us; you can find current discussions at facebook.com/peoplemag.