09/30/2011 AT 04:15 PM EDT
A day in the life of Simon Cowell
starts with breakfast in bed, ends with "a lot of beers," and includes 500 push ups in between.
"I'll do 500 push ups in a day religiously, and if I can I like to have a workout," the host of The X Factor
tells PEOPLE. "Then I have a steam and a bath but I always have breakfast in bed."
For Cowell, 51, the most important meal of the day consists of a cup of hot water with lemon ("very good to flush out all the crap in your system," he explains), fresh papaya juice with squeezed lime, and oatmeal.
Because a typical workday starts around 1 or 2 in the afternoon and ends after 9 p.m., Cowell says he's become quite a night owl. But there's one thing he can always count on to wind down: "Beer. A lot of beers. Sapporo is a favorite."
"I have a weird routine," he admits. "I'll have a massage and then I'll have four or five beers ice cold … then I'll carry on until 4 or 5 in the morning and that kind of keeps me awake."
And it's in those wee hours that his creative juices really start flowing.
"I always know what time I normally fall asleep based on my last text," Cowell says. "I'm actually very on it at that time of day. … Last night I was sending thoughts on promos, edits. I mean, I like doing that."
Apparently Cowell's fastidious nature on his TV shows
extends to his personal hygiene.
"I'm obsessed with cleanliness for myself, so I will take a bath three times a day, sometimes a steam twice a day in addition to that," Cowell says. "And the same thing in the house. … "I don't like clutter. That's why I think I'd find it difficult to have kids."
"Your house, I think, defines your mind," he added. "If you come into something which is chaotic, it means your head is chaotic, and I can't work that way."
For more photos inside Cowell's L.A. retreat and a look at where Lauren Conrad, Cher, Tia and Tamera Mowry and more call home, pick up the latest issue of PEOPLE, on newsstands now