Nothing's more miserable than the set of a comedy," declares Colin Firth when he sits down with PEOPLE to look back on 30 years of charming audiences, most often with self-deprecating wit. "It's rampant with anxiety as you try to be funny." Not that he ever looks as if he's trying. At 53, the star of Bridget Jones's Diary and The King's Speech (for which he won an Oscar) knows how to win hearts - and get laughs. But his latest role is darker fare. In The Railway Man Firth plays Eric Lomax, a real WWII British Army officer tortured by the Japanese while working on the Burma railroad. "More than 100,000 died" in the effort, he says. "Why aren't we hearing about their heroism?" Tough stuff, but Firth and costar Nicole Kidman buoyed each other's spirits. "We weren't rolling in the aisles, but there was relief." And, thinking it over, Firth allows that keeping things light isn't so bad after all. "What I said about comedy was complete nonsense," he decides, looking at a photo of himself with Mamma Mia! pals Pierce Brosnan and Stellan Skarsgard. "We had a whale of a time. I said, 'Do you ever sometimes feel overpaid?' They nearly threw me in the sea."

HIS BIG BREAK, 1984

After jumping from the London stage to the screen, Firth, then 23, found himself at what he recalls - not entirely fondly - as his first magazine shoot, for Britain's The Face. "I felt idiotic. I still do, actually. I felt like I'd walked into the wrong party."

LOST EMPIRES, WITH LAURENCE OLIVIER, 1986

Firth appeared with Olivier (and John Castle, at right) in a British TV film about touring performers. "He was 84 and frail. When someone said, 'Action!' it was like he was 25. Suddenly this power and vigor came into him." Like many of his generation, Firth was brought up "on the accepted wisdom of this man as the greatest actor in the world." When he asked Olivier who he thought was the greatest, the acting legend told him without pause, "Marlon Brando." Will Firth still be acting at 84? "Goodness. If I make it to 84, I might as well be."

HIS FIRST MOVIE: ANOTHER COUNTRY, 1984

"We didn't like each other and went through that photo shoot without a word exchanged," Firth says of costar Rupert Everett (left), also in his first film role. "We started off as friends, then it went horribly wrong about two weeks in. He was incredibly unpleasant. He described me as a 'ghastly guitar-playing socialist.'" (Everett has said that "I fancied him for the first five minutes. Then he got the guitar out.") Firth counters, "I did not have a guitar." But they patched up their differences and reteamed in other films, including Shakespeare in Love. Says Firth: "We're great friends now."

OUT OF THE BLUE WITH CATHERINE ZETA-JONES, 1991

"This picture I have never, ever seen," says Firth. "This was a TV film, rather good one. She was enchanting, and not a movie star yet. As you can see, I was [dramatic pause] younger." Did he know she would be a star? "I don't think I made a prediction, but it seemed pretty inevitable. She was clearly talented and utterly stunning."

PRIDE AND PREJUDICE, 1995

The wildly successful mini-series (with Jennifer Ehle) launched the Jane Austen renaissance and made Firth the definitive Mr. Darcy. Last year a 12-ft. statue (inset) recalling his most iconic Pride scene - after the hero has swum clothed in a lake - went up in London's Hyde Park. The mortified star says the figure is "what I hope is a composite of the character." Not you, then? "No." Sure looks like you. "It was a temporary installation," he protests. "Thank God for Londoners. And everybody."

BRIDGET JONES: THE EDGE OF REASON, 2004

Underscoring his Darcy-ness, Firth played Helen Fielding's version of the dashing suitor in the two movie versions of her bestsellers. But - spoiler alert! - the author kills off the character in her latest book. That doesn't worry Firth, who hasn't ruled out reuniting the cast (Renée Zellweger, above, and Hugh Grant). "It would be like herding cats," he says, adding, "I've always thought it would be more interesting to do when we've all deteriorated a bit - a story of a different generation."

THE KING'S SPEECH, 2010

The reception for Firth's Oscar-winning film was glowing, which, he says, surprised those who made it. During filming, costar Geoffrey Rush "joked that the tagline will be 'Two Middle-Aged Men Make Friends.'" That the story about King George VI (the current Queen's father) resonated with audiences, including Britain's royal family, is, says Firth, "very gratifying. I've heard that the Queen has seen it and that she was moved. I spoke to Prince Charles, who was appreciative. And Camilla Parker Bowles said she saw it three times!"

HIS LEADING LADY, 2013

Married 16 years, Colin and Livia (at the Toronto International Film Festival) have a red carpet routine: "You put your body armor on and do it. But nothing's nicer than getting off-stage and coming home."

THE RAILWAY MAN, 2014

The film explores war's toll on families, whom Firth calls "the secondary casualties." After three films together, costar Nicole Kidman is his unofficial "work wife." On the shoot, "Keith [Urban, her husband] visited Scotland. I got to know him a bit and met the kids." (Firth has two sons with wife Livia and a third with his ex Meg Tilly.)