The Chinese government sent tanks to quell the June 4, 1989 protests, and has never released a death toll. Estimates from human rights groups and witnesses range from several hundred to several thousand.
The Tiananmen crackdown is a taboo subject in China and 29 years later it remains a point of contention between China and many Western countries.
In a statement on Sunday the recently appointed Pompeo said he remembered "the tragic loss of innocent lives".
"As Liu Xiaobo wrote in his 2010 Nobel Peace Prize speech, delivered in absentia, 'the ghosts of June 4th have not yet been laid to rest'," Pompeo said referring to the Chinese dissident who died last year while still in custody.
"We join others in the international community in urging the Chinese government to make a full public accounting of those killed, detained or missing," Pompeo added.
China's Foreign Ministry did not immediately respond to a request for comment and there was no mention of the day in state media.
However, Hu Xijin, editor of nationalistic tabloid the Global Times, called Pompeo's statement a "meaningless stunt" that "represents a wish of the Western world to meddle in China's political process".
The Chinese Communist Party no longer mentions the Tiananmen incident in order to help Chinese society move on, which it has successfully done, Hu added, writing in English on Twitter, which is blocked in China.
Tens of thousands of people are expected to gather later in the day in Hong Kong to mark the anniversary, the only place in China where such large-scale public commemorations happen.
On Tiananmen Square, security was tight as is usual for the anniversary, with no signs of any protests or other memorial events.
Foreigners' passports were checked by Chinese police at a checkpoint nearly a kilometer from the square. A Reuters reporter was turned away and told that unapproved "interview activities" were forbidden in the square on Monday.
In their annual open letter, the Tiananmen Mothers, who represent the families of those who died, said the government was guilty of serious disrespect by ignoring their requests for redress.
"Such a powerful proletarian dictatorship apparatus is afraid of us: the old, the sick, and the weakest and most vulnerable of our society," they wrote in a letter addressed to Chinese President Xi Jinping.
In Taiwan, the democratic and self-ruled island China claims as its own, former president Ma Ying-jeou said in a statement it was important to face up to history to help heal the families' wounds.
"Only by doing this can the Chinese communists be seen by the world as a real great power," wrote Ma, under whose administration ties with China dramatically improved.
(Reporting by Ben Blanchard and Christian Shepherd; Editing by Michael Perry; Additional reporting by Jess Macy Yu in TAIPEI; Editing by Ryan Woo and Michael Perry)