Simon Cowell and Lauren Silverman
08/01/2013 at 12:00 PM EDT
When rumors began circulating last May that Simon Cowell
was in a relationship with his friend's wife, Lauren Silverman, the TV host's lawyer responded with a strong denial.
"Any implication or statement that my client is engaged in adultery and has allegedly broken up the marriage of Ms. Silverman is not only false but reckless," Martin Singer told the New York Post
at the time.
The high-profile Hollywood attorney also branded as "false and defamatory" any suggestion that the X-Factor
host's relationship with Silverman "has become more serious such that she is now getting a divorce so that Simon and she can be together."
Silverman, 36, is now pregnant
with Cowell's child and the TV host is reportedly named in divorce papers filed July 15 by her husband, real estate magnate Andrew Silverman, alleging adultery.
Neither Cowell nor any of his reps, including Singer, have commented since the pregnancy news broke.
A source tells PEOPLE
that Lauren Silverman was "definitely unhappy" in her marriage at the time she met Cowell, 53, and that their relationship wasn't the only factor in her split with her husband.
"It was probably not the best relationship when she first met Simon," the source says of the Silvermans' marriage. "It wasn't like they were blissfully in love at the time."