"The peace negotiation that will be held with [the Oromo Liberation Army] will start the day after tomorrow in Tanzania," Abiy said Sunday.
"The Ethiopian government and people will need this negotiation very much. [...] I call upon everybody to play (their) part, stated Abiy."
The OLA has been fighting the Ethiopian federal government since it split in 2018 with the historic Oromo Liberation Front (OLF) when it renounced armed struggle.
Abiy spoke at a gathering of parties to the Tigray peace process which saw a deal on November 2 end a two-year conflict between the federal government and rebelling regional authorities.
Neither Abiy nor the OLA gave details on the format of the new talks, who would mediate or where they would be held.
The OLA confirmed in a statement "that the Ethiopian regime has accepted our terms for peace negotiations, which include the involvement of an independent third-party mediator and a commitment to maintain transparency throughout the process.
"This is a crucial and positive step towards establishing a lasting peace in the region," it said. "Since the beginning of this conflict, the OLA has consistently called for peaceful dialogue as the only viable solution. [...] It is heartening to see the regime finally come to the same realisation."
The split of the Oromia rebel groups gave rise to a string of armed groups claiming to be one, but with loose ties.
The OLA's strength, estimated at a few thousand men in 2018, has increased significantly in recent years, though observers believe it is insufficiently organised or well-armed to pose a real threat to the federal government.