"I'm very sorry that he left, and I'm very upset," Williams, 90, tells PEOPLE. "Certainly I want him to go back. He needs help."
Julia says she heard from family that Ted went to California, perhaps to Hollywood, to record children's books. She says she's optimistic he'll keep busy and not return to a life on the streets, but doesn't think he can stay sober without professional help.
"I know he doesn't stick at things for long. Whatever it is, he soon tires of it," she says. "He thinks he can make it on his own, without rehab. And I'm sure he can't."
She adds: "It's a very sad story."
A family friend told UPI that Williams was disturbed by the sight of cameras at the Origins Recovery Center, by a medication regimen so heavy that "when he wasn't in [therapy] class, he was asleep," and by more interview requests from Dr. Phil McGraw, who had convinced him to enter rehab in the first place.
Williams became an overnight sensation earlier this month after a videographer with the Columbus Dispatch spotted him by a freeway off-ramp holding a sign claiming he had "a God-given gift of voice" and that he had fallen on hard times.