10/16/2012 at 08:35 AM EDT
After two run-ins with the law in 14 months, Matthew Fox wants to set the record straight. Yes, he drove drunk
. No, he didn't hit a woman
"It's been a long, trying year of sitting on my hands on a lot of negative s--- said about me, when they're the furthest things from who I am," the former Lost
actor tells Men's Fitness
"In the 46 years I've been breathing on this planet, I have never hit a woman before. Never have, never will," he says. "Out of all the negative s---, the only thing that's true is that I was arrested for a DUI."
Heather Bormann, the Cleveland bus driver who claimed Fox assaulted her in August 2011, withdrew her lawsuit
this year. But Fox admits the incident hurt his reputation. It also didn't help that Fox's former Lost
costar Dominic Monaghan labeled him a "woman beater" on Twitter.
"I think there's still gonna be a lot of people out there who'll think it's true no matter what," Fox says.
As for the DUI this past May
, Fox says his behavior was "stupid" and something he'll never repeat.
"I will bet you any amount of money that I will never get behind the wheel of a car after even one drink," he says. "The rest of it, the Cleveland situation, the Monaghan situation – was a pile of bulls**t, and I'm not gonna waste too much breath on that."
, the movie Fox was filming in Cleveland when the bus episode happened, opens this Friday.