"What makes a family is love, and we all love each other," O'Donnell, 47, tells PEOPLE. "We're gonna stay together and take care of each other as a family unit."
The definition of a family is at the heart of the comedian's documentary, which premieres this Sunday on HBO at 7 pm ET and features interviews with kids who come from all different types of families: ones headed up by a mom and a dad, two moms, two dads, as well as a mother and a grandmother. Adoption and surrogacy are also discussed.
"It's just children telling the truth about how they live and how it is for them," says O'Donnell.
A Family also includes new songs by artists including Ziggy Marley and They Might Be Giants, covering everything from love to in-vitro fertilization.
Adjustments and ChallengesO'Donnell says she was inspired by the popular 1970s album and TV special Free To Be … You and Me, created by Marlo Thomas. "We took that model and just applied it to family instead of equal rights," explains O'Donnell.
The comedian herself appears briefly in the documentary with her daughter Vivienne, 7 (her other kids Parker, 14, Chelsea, 12, and Blake, 10, are seen in video footage), and addresses her split with Carpenter. She also acknowledges how it's "difficult for families to adjust and changes are challenging." (Her next documentary, she reveals, will tackle divorce.)
These days, the exes live near each other in Nyack, N.Y. – Carpenter also has an apartment in New York City – and are raising the kids "equally," says O'Donnell, who hosts "Rosie Radio" on Sirius XM. (She also penned the forward to the upcoming book The Sound of Hope by Lois Kam Heymann, a speech-language pathologist. Heymann has worked with Blake, who has auditory processing disorder, a learning disability.)
Adds O'Donnell, "You have to really look at the four kids and realize that family is forever, which we both believe and it makes it a little bit easier."
Her family may be expanding, too: She says her girlfriend Texas-based artist Tracy Kachtick-Anders, 45, is "hopefully moving to Nyack" with her six children, ages 6 to 18.
"Our group of kids together is like the Brady Bunch on steroids," O'Donnell jokes. "Or the Gay-dy Bunch."
With reporting by RONKE IDOWU REEVES
For more on O'Donnell, her family and life with her new girlfriend, pick up the latest issue of PEOPLE, on newsstands now