"We've had some disagreements with the leader of the ruling party," Kvirikashvili said in a televised statement. "I think this is the moment now when the leader of the party should be given an opportunity to form a new cabinet."
Kvirikashvili, 50, has been prime minister since 2015.
Under the constitution, the whole cabinet has to resign along with the prime minister.
The ruling party, Georgian Dream, must then submit a new cabinet list to Georgia's president within seven days. President Giorgi Margvelashvili then has a further seven days to submit the new cabinet to parliament for approval.
Ivanishvili, 62, a former prime minister, made a political comeback in May, becoming again the chairman of Georgian Dream amid tensions among party members over a range of issues.
His Georgian Dream coalition won control of parliament in October 2012, defeating the party of ex-president Mikheil Saakashvili, leader of the 2003 Rose Revolution.
Ivanishvili's coalition later split, but Georgian Dream won a parliamentary election in 2016, cementing its status as the ruling party in the country of 3.7 million people.
Ivanishvili stepped down from the posts of prime minister and party chairman in 2013, though critics say he has continued to run the country from behind the scenes.
(This version of the story was refiled to fix typo in paragraph six)
(Reporting by Margarita Antidze; Editing by Gareth Jones)