"I'll be honest, we have it good," Silverman tells PEOPLE of her five-year relationship with the host of Jimmy Kimmel Live!
But don't expect the couple to hit the altar anytime soon.
"I don't want the government involved in our love," says Silverman, 37, who's headlining a show benefiting Project A.L.S. in Manhattan next Friday, May 9. "When my friends get married I think it's romantic, I cry, whatever," she says. "But it's not for me."
When Silverman and Kimmel aren't making provocatively hilarious Internet-sensation videos – hers about f------ Damon, his about f------ Ben Affleck – the couple spend quiet time at home in Los Angeles.
"Jimmy and I spend our mornings and nights together," Silverman says. "We set our alarms an hour earlier in the morning just so we can lie in bed and listen to Howard Stern, and at night we're watching Lost or 24."
As to whether or not kids are in their future, time will tell, she says: "I might adopt, but we're going to wait. If you're going to have kids, you need to have time to give them undivided attention."
One of PEOPLE's Most BeautifulTime is one thing the actress doesn't have at the moment, as she puts in 14-hour days on the set of the Sarah Silverman Program, which returns to Comedy Central in October.
But next week Silverman will spend a brief hiatus in Manhattan, where – in addition to her Project A.L.S. performance – she'll also attend the Metropolitan Museum Costume Institute's annual star-studded gala, where she'll likely wear a design by Dolce and Gabbana.
Despite being named one of PEOPLE's 100 Most Beautiful, Silverman says she's nervous for her fitting.
"Who knows if they'll have my size!" she jokes. "I'm a size eight – that's fat in the fashion world!"