Pitt also said that the globe-trotting clan (who recently traveled to Panama) will add to their frequent-flier miles, temporarily relocating to New Orleans for the next several months to film The Curious Case of Benjamin Button.
"I do like it there," Pitt said. "It's one of our most unique cities and it certainly needs some help right now. I know it will do fine, and the people are just fantastic."
Pitt has visited New Orleans for the Hurricane Katrina recovery effort, cosponsoring with Global Green USA a contest to find an eco-friendly plan for housing and community centers. He also has been working with organizations helping AIDS orphans.
"We wrapped at 6 in the morning, got on a plane and got our crusty asses here," he told PEOPLE.
Jolie and the children stopped over in Los Angeles while Pitt headed to Palm Springs with Blanchett for the film festival's awards gala. The festival honored the Babel cast as well as Blanchett for career achievement and director Alejandro González Iñárritu. (The movie is also nominated for several Golden Globes, including a best supporting actor nod for Pitt.)
"For us, it's about the ensemble award," said Pitt. "Given the international nature of our cast, it's a really nice honor."
During the gala, Pitt joined the cast on stage and the 1,800 people in the audience in singing happy birthday to costar Rinko Kikuchi, who turned 26 on Saturday.