04/08/2009 AT 11:15 AM EDT
Just days after appearing
on the Tyra Banks Show
, Levi Johnston, Bristol Palin's former fiancé and father to her son Tripp, talked more on CBS' The Early Show
about his difficulties with the family of former vice presidential candidate Sarah Palin.
One point of disagreement: Whether Johnston, 18, ever lived at the Palin's home. "They said I didn't live there," he said. "[They say] I 'stayed' there. I was like, 'Okay, whatever you want to call it' – I had all my stuff there. So, if you wanna call it staying there, that's fine."
"So, they're lying," said anchor Maggie Rodriguez. "Yeah," Johnston replied.
After Palin lost the election and Johnston and Bristol broke up
, Johnston says there has been tension during his visits to see 3-month-old Tripp. "I can go over there whenever I want and see my baby," says Johnston. "But I'm not allowed to take him, and it's not as comfortable going over there anymore with that family sitting down on the couch. It's awkward. I don't know. They just, like, they don't think of me the same way anymore."
He adds, "I don't wanna go over there every day and sit there when her family's there and try and play with my baby. I want to be able to take him and do what – our own thing with my family."
Levi wasn't the only Johnston to defend his story. His mother, Sherry, and sister Mercede also weighed in.
"I mean, look what they're doing," said Mercede. "They're lying, trying to save themselves when they're the ones that asked for it. They brought him to the campaign. They should have known what was coming. They can't turn around and try and take it back now."
As for why Levi is speaking out now, he explained, "We're not trying to cash in on their name, you know. I'm just trying to get my side of the story out there. Letting people know who I am. Not what the magazines say I am."
The biggest misconception? "Umm, probably that my family's white trash," he said.