Ryan Reynolds and Blake Lively
Splash News Online
celebrated her 25th birthday a few days early on Thursday at Serendipity 3 in New York City with boyfriend Ryan
The couple, who were recently spotted hitting the gym
, both donned hats – Reynolds, 34, in a newsboy hat, and Lively in a fedora – as they indulged in one of the restaurant's trademark Peanut Butter Frrrozen Hot Chocolate and hot fudge sundaes with sides of peanut butter.
"They were very attentive to each other, sitting close and very into each other," an onlooker told PEOPLE.
"They were oblivious to what was going on around them," adds another source.