"They were going to do something in this world," Badger tells PEOPLE. "I love them so much. I said, 'I'm going to do it for them.'"
He's fulfilling that promise with his nonprofit Lily Sarah Grace Fund, which focuses on bringing arts to underfunded public schools. All three girls had dyslexia, and art was a creative outlet and learning tool.
The girls and their maternal grandparents perished in a horrific fire at the Stamford, Conn., home of their mother, Matthew's ex-wife, Madonna Badger, reported The New York Times. Madonna Badger survived, as did her boyfriend, Michael Borcina – a contractor who was renovating the house and who, according to investigators, left a bag of fireplace ash in the mudroom.
Prosecutors announced they have decided not to file criminal charges, CNN reported earlier this month. Badger, meanwhile, has hired a private investigator and is exploring a civil lawsuit against the city and its officials.
So far, the fund has raised $530,000 and funded 400 arts projects in public schools. The fund will host a Broadway benefit hosted by Whoopi Goldberg on June 21.
"It is giving me purpose," Badger says. "I'm in the process of making Lily, Sarah and Grace make a mark on the planet and help people and let people know how wonderful they were."
For Matthew Badger's complete interview, pick up the latest issue of PEOPLE, on newsstands now
Courtesy Matthew Badger