Ledger's father so mishandled the management of their grandfather's estate 15 years ago in Australia that Kim was removed as executor, the uncles claim.
"It plunged into enormous debt," Mike Ledger, one of Kim's brothers, tells PEOPLE.
Heath died in January at age 28 of an accidental mixture of prescription drugs – leaving behind a daughter, Matilda, by his ex, actress Michelle Williams. His will, written before his relationship with Williams, leaves everything to his parents and sisters.
Ledger's uncle Mike insists the family has nothing to gain by removing Kim: "Our only vested interest is to assure that Matilda is well looked after."
Mike added that Kim's recent statement that Matilda "will be taken care of" was not enough assurance. "When you are talking about large sums of money like this," he says, "it should be an independent executor, but Kim hasn't chosen that way."
Williams's Father Raises QuestionsAnother of Kim's brothers, Hayden, tells PEOPLE that Williams's father, Larry, asked him if Kim was capable of managing the actor's estate. "I just said, 'No, not if he handles it like ours,' " he says.
The bitter dispute among the brothers dates back more than 15 years, when Kim became an executor of their grandfather Sir Frank Ledger's estate, which they estimate was worth about $2 million in the mid 1980s. The family patriarch was a respected engineer in Perth.
According to the newspaper The Australian, Frank's daughters filed a motion in Supreme Court asking to have Kim removed as the executor. The submission to the court stated that Kim's handling of his grandfather's estate "placed the assets at risk." A report in Australia's The Sunday Times says that a Supreme Court judge eventually removed Kim from overseeing the estate.
Ledger Family Friend Defends KimRobert John Collins – an Australian executor of Heath's will who has known the Ledger family for 30 years – said last week that Mike and Hayden's allegations about Kim were "rubbish."
Kim and his brothers have remained estranged since the dispute. In fact, Mike says that 17 family members weren't even invited to attend Heath's memorial and funeral in Perth.
"I'm his godfather," Mike says. "We didn't even have the ability or the opportunity to pay our last respects to Heath. And I tell you, that is terrible – that is so sad."
Reporting by MELENIE AMBROSE
Like this story? Click on the Yahoo Buzz! button below to Buzz it up!