"I'd love to have a nice bus so I could bring my kids racing with me," the Grey's Anatomy star, 43, told PEOPLE after a qualifying run in his Mazda RX-8 for Friday's Grand-Am Rolex Sports Car Series in Watkins Glen, N.Y.
For the actor – who qualified for Saturday's Sahlen's Six Hours of The Glen at position No. 9, the best in his racing career – balancing his passions with his role as a family man (he has a daughter Talula, 7, and 2-year-old twin sons Darby and Sullivan with makeup-artist wife Jillian) is a delicate balancing act.
"I would be so happy if they were with me when I race," says Dempsey. "The kids are very young so I need to find a great furnished vehicle for all of us so they can be with me on the track."
Off to Le MansDempsey, who is traveling around the world to race in different venues – including next weekend's 24 Hours of Le Mans in France – on his summer break from filming Grey's Anatomy, likens racing to acting.
"I am very focused," he says. "I sit and wait for a race the same way I wait for a scene to be shot on TV or in film. I pace myself emotionally and calm down so when I am on, I am really on."
Even though he has signed for two more years with Grey's Anatomy, when the show breaks for hiatus, Dempsey's professional life is totally devoted to his racing career, which often coincides with fund-raising for favorite charities like breast cancer.
"When I am not with the kids, I spend all of my off time from the show working toward improving my racing skills," he says. "I love it. I love the healthy competition."
Though his wife and children weren't with him at Saturday's race, his mother, sister and other family members cheered him on. "I grew up ski-racing with my family and to them being here feels like I've come home," he says. "I am looking forward to the day my own kids can be at my side."