The confrontation began Tuesday when Michael Scotto, a reporter for New York cable news station NY1, asked Grimm about a Justice Department investigation into his campaign finances.
After cutting the interview short, Grimm told Scotto, "You ever do that to me again I'll throw you off this (expletive) balcony."
He also threatened to "break (Scotto) in half."
NY1 posted video of the incident on its website.
Grimm later issued a statement saying he was "extremely annoyed" with Scotto and doubted he was the "first member of Congress to tell off a reporter."
NY1 political director Bob Hardt demanded a "full apology" from Grimm.