"I woke up with a married woman in my bed this morning," Followill, 30, Tweeted on Sunday morning. "Luckily she was mine."
The ceremony took place at Brentwood's Wolf Den Farm at 5 p.m., with dinner (catered by Nashville restaurant Miel) and dancing following in a barn on the property, the band's spokesman tells PEOPLE. The couple's first dance was to an instrumental Hawaiian lap steel guitar tune.
"It was a gorgeous 70 degree day and the wedding was beautiful," the spokesman says. "It was a very love-affirming wedding with lots of warmth and happiness."
Guests included Followill's Kings of Leon bandmates Caleb and Jared Followill (his brothers) and Matthew Followill (his cousin), as well as Scarlett Johansson, who went to high school with the bride, and country singer Billy Currington.
After the wedding, Followill, who met Baylin, 25, a singer/songwriter signed to Verve Records, at Bonnaroo in 2006, Tweeted a picture of his gold wedding band. Appropriate since the night before, he sent out this message:
"This time tomorrow night I will be wearing some new jewelry. I'm so excited for tomorrow. A new day, a new life, and a new wife. Thanks Jess."