Zuma, who is battling corruption allegations, has been in a weakened position since he was replaced as leader of the ANC in December by Cyril Ramaphosa, the deputy president.
The ANC did not say what had been decided at the meeting between Zuma and the party's "top six" leadership team at Zuma's residence in Pretoria, but it summoned its National Working Committee (NWC) to an urgent meeting on Monday afternoon.
The ANC's NWC conducts the day-to-day running of the party. It would need to call a meeting of the party's National Executive Committee to force Zuma to quit.
Opposition leader Julius Malema, a former ANC member, said Zuma had refused to resign at the meeting late on Sunday.
"He refused to resign and he told them to take a decision to remove him if they wish to do so because he didn't do anything wrong to the country," Malema wrote on Twitter.
Malema was one of the first to disclose that Ramaphosa had won the race to succeed Zuma as ANC leader in December.
Zuma has been deserted by several prominent allies in the ANC since Ramaphosa became leader of what is the only party to govern South Africa since the end of apartheid.
Ramaphosa, who is in a strong position to become the next president of South Africa, has been lobbying for Zuma's removal.
Zuma has not said whether he will step down voluntarily before his second term as president ends next year. Opposition parties want Zuma to resign before his state of the nation address to parliament, which is scheduled for Thursday.
The speaker of parliament said on Friday that Zuma would face a fresh no-confidence motion on Feb. 22.
Zuma has survived several no-confidence votes during his rule thanks to loyal voting by ANC lawmakers, but support for his leadership is on the wane. Although Zuma retains the support of a faction within the ANC, he no longer holds a top post.
(Reporting by Tiisetso Motsoeneng and Alexander Winning; Editing by Toby Chopra and Peter Cooney)