"If I heard someone saying this, I would probably doubt it as well," London, 37, tells PEOPLE in an exclusive interview.
But he insists it happened, and shares how he survived. "I was acting my way through the whole thing, telling myself, 'Just play like you are one of them,' " he says.
As London tells it, on June 10 several men helped change his flat tire in Palm Springs, Calif., when they pulled a gun on him, drove him around for 12 hours while making him take ecstasy and smoke meth – then stole his car. A 26-year-old man has been arrested in the case.
At the time, London was in Palm Springs visiting his estranged wife, Melissa Cunningham, 39, and the couple's 3-year-old son, Lyrik. As part of his fight to regain partial custody of Lyrik, the actor – a recovering addict – says he's been sober since last September and has been undergoing regular urine tests.
"I haven't tested dirty ever," he says.
But London's family members doubt his story. "After hearing Jeremy's account of his alleged ordeal, it only brings up a million more questions about what really happened," his twin brother Jason tells PEOPLE.
For more on London's story, pick up this week's new issue of PEOPLE, on newsstands Friday