They're big shoes to fill. But some names are already being floated as potential successors. Let's look at some of the next would-be after-hours kings and queens.
The Scotsman, 51, would be a logical candidate, as he's been hosting The Late Late Show, which airs after Letterman on CBS, since 2005. Ferguson has seemed conflicted at times about the 11:35 slot, saying in 2009: "I don't know if I want to be any more famous than I am now."
It could be perfect timing for Handler, 39, who is ending her eight-year run as host of E!'s Chelsea Lately this coming December. Her manager said this week that she's actively looking for another gig, and is open to late night.
PEOPLE TV critic Tom Gliatto says, "My ideal choice would be Stephen Colbert from Comedy Central's Colbert Report: incredibly funny, self-effacing, charming, intelligent – someone who seems to function as an actual human being (that is, who's like the viewer) and not just as a comedian who would kill to host a talk show or the next big entertainment gig. Plus, he might be tired of the political persona he created for Report."
Nikki Finke of Deadline Hollywood claims the job is Colbert's to lose.
Tina Fey and Amy Poehler
Here's an idea: Not one host but two. Fey, 43, and Poehler, 42, have been the perfect pair as Golden Globes hosts. They could follow former SNL castmate Jimmy Fallon into the talk-show game – perhaps hosting on alternating nights so they can still pursue their acting careers.
Neil Patrick Harris
The actor, 40, has more time now that How I Met Your Mother has wrapped up after almost a decade. He's no stranger to hosting, either, having emceed the Tony Awards and the Emmy Awards multiple times with great success.
Another accomplished host – of her own talk show since 2003, and of the Academy Awards twice as well as the Emmys once – DeGeneres would be a fantastic get for CBS. Still, she's got a pretty good day job as it is.
No stranger to late night, O'Brien, 50, once had Letterman's old 12:35 p.m. time slot at NBC before famously having a brief, ill-fated run on The Tonight Show. Now with his own late-night show at TBS, would he want to brave the networks again?
The comedian, 50, had a three-episode arc on his FX show Louie in which his character mistakenly believed he'd been asked to replace Letterman. Could life imitate art?
The comedian and former eponymous sitcom superstar, 59, has been chugging along nicely with his Emmy-nominated web series, Comedians in Cars Getting Coffee.
Anyone named Jimmy?
Jimmy Kimmel and Jimmy Fallon have the other two big 11:35 p.m. slots filled. Anyone talked to Jimmy Smits lately?
Now, your turn: Who do you think should get the job? Tell us below.