Sam Lutfi and Britney Spears
Fame; PA Photos/Landov
02/23/2009 AT 03:15 PM EST
A manager and confidante to Britney Spears
when her life was at the peak of turmoil, Sam Lutfi insists that in these calmer days he's no threat to her safety.
Fighting back against a restraining order
, Lutfi 34, not only denies he's been harassing Spears, but claims that it's been Spears reaching out to him
– not the other way around, sending him text messages even though court orders forbid her from doing so.
"I don't think anyone would want to have on public record allegations of behavior that weren't committed," Lutfi's lawyer Bryan Freedman tells PEOPLE. "A restraining order to is protect someone against stalkers and people who want to commit violence, and that description doesn't fit my client."
In court documents filed Friday, Lutfi does admit that he's passed messages on to her through her hairdresser and others, including her ex-boyfriend, Adnan Ghalib, 36. But Lutfi argues that the messages weren't harmful in any way.
"I have not, in any alleged communications with Britney, her hairdresser or any other person, threatened any act of violence against Britney or her family," he states in the papers.
Lutfi's lawyer agues in the filing that the singer, 27, and her father haven't provided evidence that she's suffered emotional distress from Lutfi's indirect contact. As co-conservators, Jamie and lawyer Andrew Wallet hold indefinite control over her affairs since her two forced hospitalizations in early 2008.
In his request for the restraining orders, Jamie Spears accused Lutfi of working with Ghalib and a lawyer to "disrupt the conservatorship with utter disregard for Ms. Spears's health and well being," court documents show. Lutfi allegedly sent a text message to the hairdresser saying, "I've done everything I can to free her from [the conservatorship]. Very close to getting her free now."
Lutfi has since sued
the Spears family for defamation and battery. Jamie Spears is set to testify at a 1:30 p.m. hearing on Monday to uphold the restraining order.