08/02/2013 at 08:50 AM EDT
faced a ballroom full of television reporters and finally confronted the news about getting his friend's wife pregnant.
The X Factor
judge first tried deflecting the issue with a quip. "I haven't read the newspapers," he said Thursday at the Television Critics Association meeting in Beverly Hills.
Then he offered a no comment: "Unfortunately at this moment I have to keep it private."
His X Factor
cohort Demi Lovato
jumped to his defense. "I don't think you have to answer that!" she said.
Cowell later told PEOPLE: "There are a lot of things I will eventually clear up when the time is right, but I really have to be sensitive because there's a lot of people's feelings involved here.”
Cowell, 53, is expecting his first child with New York socialite Lauren Silverman, 36, the wife of Cowell's friend Andrew Silverman.
Andrew Silverman, who has reportedly filed divorce papers
in New York listing "adultery" as the reason for their split, issued a statement
Thursday saying his "top priority" was his 7-year-old son " while working to resolve all outstanding issues with my wife."