"I haven't lost an ounce of sleep," LaRoche, 36, told ABC News in an interview that aired on Good Morning America on Tuesday.
The veteran athlete decided to walk away from from a reported $13 million contract after team officials asked him to stop bringing his 14-year-old son Drake into the clubhouse everyday.
White Sox Executive Vice President Ken Williams said last month that he had only asked LaRoche to "dial it back" with Drake, a conversation that the player told ABC News led him to consider retiring just "20 minutes later."
LaRoche added that he didn't remember any of his teammates having an issue with Drake's presence – in fact, the teenager had his own locker and often helped with team chores like cleaning cleats.
The athlete, who also has a daughter named Montana with wife Rachel, insisted to ABC News that his hefty paycheck was "not a huge" factor in his decision to retire.
"I learned a long time ago, no matter how much we have, it's never enough," he explained, adding, "Who doesn't need 13 million? ... I get that it's an absurd amount of money."
Nonetheless, he hasn't missed the huge salary – yet. Drake, however, said he wished he had more time "hanging out with the guys. They're awesome."
LaRoche called it a "privilege" to have Drake around the game for most of his life, noting, "he gets to grow up from this and hopefully learn from my screw ups and good decisions."
WATCH: Twitter Goes Crazy After Man Saves a Boy From Being Hit in Face by Baseball Bat
A trip last year to southeast Asia, where LaRoche went undercover with a non-profit organization that helps victims of sex trafficking, also factored in his decision to make family first.
"I think having my own kid, having two kids of my own, especially a 12-year-old daughter, it's impossible not to picture, you know, 'What if this was my daughter?' " LaRoche explained to ABC News.
The first basemen, who signed his retirement papers in March, said he doesn't "hold a grudge" against Williams and other White Sox team officials.
"I don't hate anybody over there. You know, it just made my decision easy," LaRoche shared. "Honestly, it's not the end of the world to me. And I thank my parents for that. The way I was raised. Because baseball – and I've said it before, I don't want to be defined by this game. I know there's a lot more to life."
And he's made sure Drake feels no responsibility for his decision: "I think he knows, deep down, that baseball was never, like, my life or my world or everything to me."