The paper reports that the security chief for Pitt and Jolie gave a local journalist a statement in which the couple asked for privacy.
"We love Africa and to be here in Namibia with our family is very special for us," the statement said. "To the local people who have been so kind and gracious, thank you for making us feel at home.
"As for the press, we kindly ask for privacy so that we can enjoy this beautiful country with our children."
The Sunday Times reported that its own photographer, along with three others from France, were ordered to leave Namibia or face arrest. Journalists must have proper accreditation to work in the country.
Namibian Prime Minister Nahas Angula defended the crackdown. "This lady is expecting," he told the Sunday Times. "You guys are harassing her. Why don't you allow her some privacy? Harassment is not allowed in Namibia."
He continued, "If a person says they don't want to be photographed, then of course that person deserves protection."
Pitt, 42, and Jolie, 30, have been vacationing in the Southern Africa country with kids Maddox, 4, and Zahara, 1, at a seaside resort hotel where they've rented all the rooms.
According to the Sunday Times, the family recently visited a local KFC, a pet store (where they reportedly bought a turtle), and a jewelry shop.