Kanye West is the subject of a Beverly Hills Police investigation after a man claimed on Monday that the rapper assaulted him.
"The named suspect was identified as Kanye West by the victim and several witnesses," says a police statement. "Mr. West had left the location prior to officers' arrival. As of this writing, BHPD detectives are conducting a follow-up investigation."
A source close to Kim Kardashian says it was the bystander – who has not been identified – who instigated the event and threatened the reality star.
"Kim was walking into a furniture meeting for their new home," the source tells PEOPLE. "Kanye was there ahead of her for the meeting. So Kim gets there and walks into the building with a million paparazzi trailing her."
The source says a Caucasian man began yelling racial obscenities to the paparazzi, then when Kardashian told him "that wasn't cool," the man began hurling epithets at her about her and West.
"At this point Kim was seriously afraid for her life," the source continues. "He was going nuts. Kanye came down to help her but then the guy ran into a chiropractic office so Kim and Kanye walked in after him."
The man verbally confronted West with obscenities, and at one point West hit him – "barely," the source insists.
A lawyer for West wasn't immediately available for comment.
West has a history of scuffles: In 2008, he was arrested after getting into a fracas with photographers at LAX, but charges were eventually dismissed after he completed anger management classes. Also that year, the rapper was arrested in the UK after a similar brush, but he was released with no charges filed. Last July, West tangled with a photographer again at LAX, a case which is still pending.
Additional reporting by JENNIFER GARCIA