Fox, 29, showed off her baby bump in a colorful print maxi dress while her estranged husband Green wore all black for their day trip out.
While they may have not been holding hands, the couple appeared to be in good spirits as they walked side by side together.
A source confirmed to PEOPLE that Green is "of course" the father and that the pair "never stopped loving each other and they will never stop being devoted parents to their boys."
The pregnancy came as a surprise for the couple who are "very excited" and are "continuing to work on their marriage and are living together again," another source said.
The New Girl star filed for divorce in August after five years of marriage and 11 years together. According to legal filings, the divorce is still pending.
Reps for both stars have not commented.
Charley Gallay / Wireimage
The couple, who are already parents to Noah, 3½, and Bodhi, 2, are back together and while Megan has been busy working on Teenage Mutant Ninja Turtles, Brian has stayed with their boys.
"They both love being parents," adds the source.
The exes have remained amicable since announcing their split and have often been spotted together with their two children and have also been seen running errands together without the kids.