Three of the modest wooden structures, built by Los Angeles man Elvis Summers, were confiscated earlier this month and eight were marked for removal Thursday, Summers tells PEOPLE.
"It's just devastating," Summers, 38, says. "These houses are all these people had that was safe and secure."
The houses were slated for removal as part of a street cleanup requested by the office of city councilman Curren Price, a sanitation spokeswoman told the Los Angeles Times. She added that the three structures that were removed earlier this month will be destroyed.
While Summers was able to relocate the houses marked for removal, putting them in storage still left the homes' former occupants to sleep on the streets.
"The city made these people completely homeless again," he says.
Since April, Summers and a team of volunteers have built 37 tiny houses on wheels for homeless people around Los Angeles through his non-profit My Tiny House Project LA. The Seattle, Washington native started the organization after a video he created showing how he a tiny house for his homeless neighbor, 60-year-old Irene "Smokie" McGhee, went viral and sparked an outpouring of support. Summers then began fundraising to build additional houses – ultimately raising over $95,000.
Suddenly, the good deed he completed in a weekend turned in to a full-time job. Summers and his supporters began working seven days a week building homes.
"[If] I have to do it singlehandedly 'til it's done – building one tiny house at a time until there's no more homelessness – then that's what I'll do," he told PEOPLE in June.
The project was dealt its first blow from the city in August, when the Los Angeles City Council declared that structures were unsuitable for human habitation.
"The only legal use for these [structures] is for dogs," Los Angeles City Councilman Joe Buscaino said at the time, according to the San Jose Mercury News. "This is not the way we treat people who are homeless in our city."
The city has cited the structures' lack of electricity, running water and toilets as the reason for this decision. Summers says he is close to securing a piece of land where a compound of the homes could share bathroom and laundry facilities.
"The tiny houses are an essential step for right now," Summers says. "Nobody seems to be addressing the tens of thousands of people that need somewhere to go tonight."
A spokeswoman for Mayor Eric Garcetti told The Times that he is committed to getting people into permanent housing. However, that will do little for the immediate needs of those who have slept in the tiny houses for months and now face a return to sleeping on the street.
"Their hearts are broken," he continues. "Half of them are still just sitting on the concrete where their houses once stood."