His first child, Madeline Leigh DeMarcus, came into the world at 4:49 p.m., three weeks before her due date.
Mom Allison DeMarcus, a former Miss Tennessee and CMT personality, is doing great, Jay tells PEOPLE in an interview, and the 6 lb., 9 oz. baby with a full head of dark curls "is tiny, but perfect in every way! I'm just kissing her little face off."
Madeline Leigh puts Jay in the Rascal Flatts' Dads Club with bandmates Joe Don Rooney and Gary LeVox, both experienced fathers of two.
"Gary told me to strap in and hold on for the wildest ride of my life," said the tired but happy dad. "I didn't get much sleep last night. I kept waking up and thinking 'Oh my gosh, I'm a father!"
The couple, who met when Allison appeared in a Rascal Flatts video and married in 2004, started thinking of baby names as soon as they found out they were having a girl.
"Picking a name for a girl was hard. There are so many beautiful names, and so many pretty combinations. Leigh is Ali's middle name, and we just loved Madeline, though I'm already calling her Maddy."
While Mom and Maddie rested in the hospital, Dad went back to the studio to contribute vocals to McCurdy's upcoming debut country album.