Silverman, 43, known for her own provocative style of comedy, joined her ex, Jimmy Kimmel, on his talk show Thursday to remember the legendary Rivers, who died earlier in the day.
"She was a hero to me, and I loved her very much," Silverman said. "I think a lot of people, when they die at 81, you go, 'Well, she was 81. She had an amazing life.' But she wasn't done. She, right now, was at her most vital.
"I mean, oh my God, I tuned into Fashion Police every Friday because they were the most hard-core jokes on TV," she added. "You just can't believe that she's saying them."
"We lost a comedy legend today," said Kimmel, 46. "And besides being a pioneer for women in comedy, for everyone in comedy, Joan was a very lucky person because she loved her job so much, she never wanted to stop. And she didn't have to stop, because she was still great at it.
"We will miss you Joan," he added.
As a tribute of sorts, Silverman and Kimmel then traded comic insults.
"If she was here," said Silverman, "she'd probably want me to say, like, 'Nice tie. Who made it, Calvin Clown?' 'I like that shirt. Does it come in men's?' 'Oh my God, Jimmy, I love your hair. You have to tell me where you bought it.' "
To which Kimmel replied: "You know what, I bought it from your back."