"Ben's friendly, but it's not always meant to be flirty," a source close to Affleck tells PEOPLE in this week's new cover story. "This was a very close relationship with someone who was taking care of the most important people in his life. "
Speaking of the actor's friendliness, the source adds, "People want to fault him for that."
Affleck has strongly denied allegations of an affair, though multiple sources have told PEOPLE that Ouzounian told friends she and Affleck slept together. Sources close to Jennifer Garner say she found out Affleck and Ouzouzian were intimate: "To Jen, this is the ultimate betrayal," one friend tells PEOPLE. The source close to Affleck maintains the two had only a "work relationship and a friendship."
Affleck and Garner, who announced their plans to divorce on June 30, did not split because of the alleged affair, the Affleck source adds, disputing a Garner insider's contention that it was "the final straw."
"This was not one of the reasons for the breakup," the source tells PEOPLE. "There was no physical relationship before or after the breakup." The source also says that Affleck and Garner were separated for eight months, which Garner sources dispute.
A source close to the couple points to Garner's "constant nitpicking" as a reason the marriage failed.
"When you are made to feel like an inadequate husband over and over again," the source says, "your friends are going to say maybe this isn't working out."
Despite the painful split, Affleck has continued to spend time with Garner and their children while she films Miracles from Heaven in Atlanta. The source close to him says that the actor has been spending time "with [his] kids as much as possible" and is "leaning on his mom and friends," although he hasn't been spotted at the guest house on her rental property since news broke of the alleged affair. Last weekend, Affleck hosted a barbecue party for 35 guests at his house on Hampton Island, Georgia, a source tells PEOPLE.
"This is not easy for him," the source tells PEOPLE. "It's hard not to pay attention to the stories that are being written about you, but he continues to focus on what is most important, his family."
Reporting by LIZ MCNEIL
For much more on Affleck and Garner's difficult split, the nanny allegations and how Garner is coping, pick up this week's issue of PEOPLE, on newsstands Friday