Anderson was back in her ex's arms to watch son Dylan's baseball game on May 8 in Malibu. "The boys miss Pam and me together," Lee tells PEOPLE. "When we're together, they're so happy. At the end of the day, that's truly what my heart wants. I want us all to be together."