Plateau State, sitting on the dividing line between Nigeria's mostly Muslim north and predominantly Christian south, has seen a surge in tit-for-tat attacks in recent months.
Two cattle herders were killed on Sunday before 21 farmers were killed in attacks later in several villages, Plateau State police spokesman Alfred Alabo told AFP.
"The two herders were killed first while the second attacks were launched later in the night," he said. "The commissioner of police has visited the scene of the incident. We are investigating the cause of the attack."
In a statement, Plateau State Governor Caleb Mutfwang said he was saddened by the killings and said the security situation had become "alarming".
"The security architecture has become like an old abandoned engine that needs to be reserviced and retooled," the official.
In May, local officials said more than 100 people were killed and several thousand more displaced by attacks in Plateau's Mangu district.
Clashes between nomadic herders and farmers over land, resources and water are common in several parts of Nigeria, including the central and northwestern states.
But violence often spirals into attacks and mass kidnappings by heavily armed gangs and vigilante groups set up to protect communities.
The local leader of cattle breeders association of Nigeria (MACBAN), Bello Tafawa, told AFP two herders were shot dead on Sunday evening on their way home in an unprovoked attack.
"We have already reported the case to the security agencies in the area," he said.
A local youth community leader Rwang Tengwong said 21 persons were killed by suspected "herders" in various villages later on Sunday.
"Twenty one persons were killed and several others injured on Sunday during coordinated attacks," he said, blaming ethnic Fulani herdsmen.
President Bola Ahmed Tinubu, who began his term last month, has called on Nigeria's armed forces to better coordinate to manage the country's multiple security threats.
Security forces are still battling to end a 14-year-long jihadist conflict in the northeast, intercommunal violence, mass kidnappings for ransom and also separatist tensions in the country's southeast.