While working on the 1999 film Bowfinger at the beginning of his career, Kennedy recalls meeting Coleman, whom he describes as a "pop icon," in one of the most unexpected places in Hollywood.
"We were filming in downtown Los Angeles and I pulled up to the [set] and looked over and saw this security guard. It was a little guy," Kennedy, 40, tells PEOPLE.
When he realized the guard was Coleman, "I said, 'Gary, how you doing, man?' and he said, 'You know, trying to put the pieces back together," says Kennedy. At the time, Coleman was facing financial and legal trouble.
"I said, 'Why are you here?'" recalled Kennedy. "He said, 'I need a job, man.' I said, 'But you’re Gary Coleman.' And he said, 'That don’t mean nothing.'"
The chance meeting with Coleman, who died Friday at age 42, was "so humbling," says the Ghost Whisperer actor. "[Diff'rent Strokes] was a pop culture phenomenon and here he was guarding my trailer. I remember just looking at him. I used to watch him all the time. I used to recite all his lines. He was one of the reasons I wanted to get into Hollywood."
But Coleman, Kennedy says, had no gripes or shame about his temporary job. "He was really cool about it. He was a sweetheart."
"I just thought, 'Enjoy your moment and be humble, because anything could happen,'" says Kennedy. "And now the poor guy is gone."