Tillerson said all State Department employees would undergo mandatory training to be completed by June 1, speaking in Cairo to about 150 U.S. embassy staff outside the ambassador's residence.
Tillerson is in Cairo at the start of a Middle East tour that will also take him to Kuwait, Lebanon, Jordan and Turkey.
Tillerson said that respect for one another is a key value at the State Department, noting that a prime example is the problem of sexual harassment.
"There is no form of disrespect for the individual that I can identify, anything more demeaning than for someone to suffer this kind of treatment," he said.
"It's not okay if you're seeing it happening and just look away. You must do something. You must notify someone. You must step in and intervene," he added.
Tillerson's comments come amid a chorus of sexual misconduct accusations against powerful men in media, business and politics in the United States.
In November, some 200 women who worked in national security, including at the State Department, signed a letter saying that sexual harassment policies were weak and often unenforced. They made several recommendations, including mandatory training and channels to report abuse without fear of retribution.
The White House has been rocked by similar allegations. Last week, two White House aides left the administration after domestic violence allegations surfaced against them.
Both men have denied the accusations. Reuters has not independently verified either case.
"(People's) lives are being shattered and destroyed by a mere allegation. Some are true and some are false ... Is there no such thing any longer as Due Process?" Trump said on Twitter on Sunday.
(Reporting by Yara Bayoumy; Writing by Makini Brice; Editing by Jeffrey Benkoe)