The former couple, who were spotted together with their three children on Saturday, are spending most of the summer in Atlanta, Georgia, where Garner is currently filming the drama Miracles from Heaven.
"He's going to be there for most of the summer," a source close to the couple tells PEOPLE of Affleck, adding: "That was the plan. He's happy to be there with the family."
The source says the pair, who plan to divide their assets via mediation, aren't in a "rush" to file for divorce and instead are "concentrating on being a family right now."
"[Garner]'s finishing a film, so there hasn't been like, 'Okay, let's get this done now,' because they always were like, 'This is happening and still moving forward,' " the source says, concluding, "I don't think they're talking about it."
Both the stars were spotted wearing their wedding rings on Saturday as Garner, 43, ran errands and Affleck, 42, left a gym. Later in the day, the exes got together to take Violet, 9, Seraphina, 6, and Samuel, 3, to a local school fair, spending time in a bouncy castle, a batting game and a face-paint tent.
"This is not easy on him," the source tells PEOPLE of how Affleck is handling the split. "I'm sure it's not easy on Jen either, but this is hard for him ... and he's happy to be with the family."
VIDEO: What Wearing Wedding Bands Means for Ben Affleck and Jennifer Garner
Affleck and Garner, who were married for 10 years, will continue to live on the same property when they return to Los Angeles. They're renting actress Brooke Shields' home while theirs undergoes extensive renovations.