01/08/2013 AT 02:30 PM EST
is slowly getting stronger after his incredible weight loss.
But his workout regimen doesn't include much heavy lifting as a new father.
Asked at Monday night's New York Film Critics Circle Awards dinner if he's running home to change diapers, the actor, 43, told reporters: "I was there this morning, so I guess I am running home. But I'm not changing the diapers."
So, who is?
"Somebody," he said.
For now, the father of 11-day-old son Livingston
– as well as older siblings Levi, 4, and Vida, 3 – is focused on getting his own body back to normal.
Asked if he misses his old hunky physique, he replied: "I have less [physical] leverage, but my mind is way sharper … You know, I love to wrestle. It's a good time to wrestle me now. I was 135 lbs., and now I'm up to 160 lbs, so I'm done with that and on my way back."
McConaughey says his target weight is 165 lbs., so he's only 5 lbs. shy of that.
The actor lost the weight for a role in Dallas Buyers Club
through an intense diet. When that was over, he knew exactly what he wanted to eat first.
"A cheeseburger," he said. "That was it. I got the cheeseburger, and then I sat there for 25 minutes and doozied it up."
Reporting by MICHELLE WARD