The Modern Family actor, 46, has been married to wife Holly for 13 years and wouldn't dare risk jeopardizing his relationship.
"It really matters whom you're with," Burrell – who usually prefers to remain private about his personal life – tells Playboy in its March issue, on newsstands Friday.
"My wife and I … have two great kids now. I know I've stumbled into the right relationship."
But even before he was married, the Oregon native was no stranger to serious relationships.
"Luckily or unluckily, I don't have a ton of exes," he says. "Before my wife, I was in only two relationships – one for five years and another for three-and-a-half years. I'm a serial monogamist."
That said, luck with the ladies didn't always come easy.
"When I was 28 to 29 or so, I would stay with my mentors, husband-and-wife professors, and then sleep on their porch for a while, and then I'd live in the van," Burrell recalls. "I may have had one date during that whole period, which was about a year on and off."
Explains the star: "Basically, when my date figured out I was living in my van, I didn't hear from her again. The funniest thing was I was confused about why. I was like, 'What's the problem?' I didn't realize I was the creepy guy in a van. What could possibly be holding up this relationship? The van pretty much eliminated my dating life completely."