08/12/2013 at 03:40 PM EDT
is taking a break from his baby drama.
The X Factor
judge, 53, who is expecting a child
with still-married Lauren Silverman, 36, is "happy to be on vacation," a source tells PEOPLE. "It's obvious that the drama between Lauren and [ husband] Andrew that involved Simon, makes him a bit upset."
Cowell was spotted boarding a yacht in the South of France. He "has been in contact with Lauren since her divorce turned messy
but "won't see Lauren in person until things calm down," the source says.
Adds the insider, "He cares about her and has been checking in on her and the baby. He still encourages Lauren to quickly figure things out with Andrew so they can move forward with their divorce."
Despite reports, the source says, "Simon still has no plans to move in with Lauren," but Cowell, who owns several properties in Beverly Hills, "can put up Lauren in one of [them when he's working in L.A.] so he can spend time with his baby."