That's quite literally where her mother, Eunice Kennedy Shriver, found the inspiration to create Camp Shriver, a summer camp for people with intellectual disabilities that eventually evolved into the Special Olympics.
Eunice's older sister Rosemary Kennedy was born with an intellectual disability and "watching her sister be ostracized by the world" led Eunice to hold the first Camp Shriver event on her Maryland farm in 1962, Maria says.
"Mummy's legacy started in the backyard. It really started in her own home, in her own heart, in her own relationship with her sister, and her parents and her siblings," Maria tells PEOPLE. "Then it went to the yard, then it went to a stadium, and then it went to another country and then it just dominoed."
The Special Olympics have come a long way since the first games in 1968. Maria, 59, recalls working the 1984 games in Los Angeles when "everybody left town. Now everybody here stays in town. Now I see families sitting in traffic jams, waiting in line, filling the stands. It makes me really proud of the city. It makes me really proud of this community."
She's also proud of her children – Patrick, Christopher, Christina and Katherine Schwarzenegger – for getting involved in the games.
"It's a great thing to be involved in something that you can do with your kids," says Maria. "There are so many things that take you away from your family, and I think that's the one thing my mother – and my father and my uncles, everybody – really pushed: family loyalty, family unity."
Maria has instilled the same values in her own children, as well as a commitment to doing volunteer work.
"I tell them they don't have to necessarily work for Special Olympics or Best Buddies [another Kennedy family nonprofit] but they have to find something that allows them to broaden their consciousness and make the world a better place – whether it's animal rescue or a Social Impact Summit or something," Maria says. "Regardless of who you are, who you're brought up by – you don't have to build it so big but you can do something."
For her part, Maria is doing her best to carry on her mother's legacy, which includes releasing the Shriver Report Snapshot, a new national poll offering insight into Americans' perceptions of people with intellectual disabilities.
Of all the results, Maria says she was most surprised by "how much isolation still exists for people with intellectual disabilities, how few people have a friend who has an intellectual disability."
"Even though half of the country says they know someone [with an intellectual disability], such a small percentage (13 percent) of people actually say, 'I'm friendly with or I have a friend who has an intellectual disability.' So that tells me that inclusion works, that experience with this community matters, but that we still have a long way to go in recognizing people with intellectual disabilities to be full and complete human beings, and putting them in our social circles and giving them the same rights that we give our other friends."
"This movement I think is in many ways where the gay [rights] movement or the LGBT community was 20 years ago," she adds, "where people said they didn't know somebody who was gay and that how over time, that experience changed. I think that's what's happening today with people with intellectual disabilities."
Maria says the Special Olympics have played a key role in helping people become more understanding and empathetic towards people with intellectual disabilities.
"These games have been game-changing," she says, crediting ESPN and ABC for their in-depth coverage of the Special Olympics.
"I think people will be taking a second take on this community, and I hope in seeing the results of the Snapshot, they will also say, 'Wow, I really should reach out to someone who has an intellectual disability. I should get my kids involved. I should update my perceptions, my beliefs and my views.' "