The actor's widow, Deborah Lin, and their 8-month-old daughter, Liliana, were among the first mourners to arrive, having flown in from California, reports New York's Daily News.
"It's a nightmare for the whole family. It's a horrible, horrible thing, everybody is just devastated,” Bob Price, whose wife is a cousin of the late star, told the newspaper. "This man was everybody's hero. It's just the worst thing that could have happened."
James Gandolfini Dies at Age 51
Friends and family members left flowers at the Robert Spearing Funeral Home in Park Ridge, N.J., near where Gandolfini had grown up. Broadway theaters also paid tribute to him, dimming their marquee lights in his memory this week.
Gandolfini, who won three Emmys playing Tony Soprano, also was nominated for a Tony Award in 2009 for his role in the comedy-drama God of Carnage.
The Sopranos star, 51, died of a heart attack in Rome on June 19. His 13-year-old son was vacationing with him, and Gandolfini's body was returned to the States last weekend.
A funeral service for Gandolfini will be held Thursday morning at the Cathedral Church of Saint John the Divine on Manhattan's Upper West Side.
According to a spokesperson, the 90-minute service for will be led by the Very Rev. Dr. James A. Kowalski, Dean, the Cathedral Church of Saint John the Divine.
Remembrances will be provided by Deborah Lin Gandolfini; family friends Thomas Richardson and Susan Aston; and David Chase, the creator and executive producer of The Sopranos.
Music will range from traditional hymns to contemporary selections.