Amelie Mucci/Splash News Online
03/24/2012 at 02:50 PM EDT
You wouldn't mess with Don Draper around the office at Sterling Cooper Draper Pryce. And you shouldn't mess with Jon Hamm
, the man behind those suits and ties, on the Mad Men
But his costars thought about it.
John Slattery, who plays troubled executive Roger Sterling, and Elisabeth Moss, who plays secretary-turned-ad copywriter Peggy Olson on the AMC series – whose season 5 premiere
is Sunday – attempted to play a prank on Hamm when he was making his directorial debut for the show's April 1 episode.
"We were going to do this thing when you just mouth the words with no volume, no sound, coming out, and it wasn't even that funny," Slattery, 49, tells PEOPLE about the abandoned idea to trick Hamm. "Elisabeth [Moss] would say that I chickened out, but I think I was just trying to concentrate on what I was doing."
They were not the only ones celebrating April Fools' Day a little early on-set, and Slattery says Vincent Kartheiser, who suits up as young exec Pete Campbell, also tried a hand at tricking Hamm.
"That didn't work either," Slattery says. "Hamm is too smart. He kind of is un-Punkable
Hamm, after all, had to use those smarts in the directing chair. "It was not my choice to come in and go, 'I'm gonna do my thing. Here's a whole different attitude!' " Hamm, 41, explains. "My job was to not drive that train off the tracks, but keep it going as smoothly as I could."
So was he happy with the final product? "I think I did an okay job," he says. "We'll see."