Children foraging for firewood outside the village of Kabanga near Mpigi town, about 40 kilometers (24 miles) west of the capital Kampala, made the grisly discovery on Sunday, according to local residents.
Shocked villagers told police there were metallic boxes containing what appeared to be skulls in an underground chamber beneath one of three ruined shrines on a sleepy hillside.
"We swiftly moved in and dug up the place, and so far we have recovered 17 human skulls," regional police spokesman Majid Karim told AFP. "We are conducting more excavations to ensure there are no more skulls other than what we have so far recovered."
The underground chamber "was well-built" and was more than two meters (seven feet) deep, police said.
Karim added that the remains, discovered in four boxes, were being examined to determine their age and sex, as well as when they may have been buried.
Three skulls are those of children, Karim said after an initial examination.
"When there are skulls... there must be other parts of the body like the legs," he said, calling on residents to volunteer information that may lead to the discovery of other body parts.
He also urged the public to remain calm, with officers working to investigate the incident.
It is not clear how the skulls ended up at the site, located on a steep hill that has prevented the use of heavy machinery, slowing the search.
Father-of-four Bruno Serunkuma Mubumbi, 42, told AFP he had received an urgent call on Sunday informing him of the discovery of the skulls at the site close to his home.
"Our worry (is) if police have recovered the skulls, where could the rest of the body parts be, like the legs, the hands?" said Mubumbi, a motorcycle spare parts trader. "We are shocked. We never expected this to happen in our neighborhood."
Restaurant manager Gideon Muhindo, 43, said he was "scared" at the discovery, describing it as "something surprising .. It is something not good for the community. It is strange for the community."
Local media reported residents as saying that people had previously gathered at the location to worship.
Police said the suspected owner of the site is on the run, having been linked to a separate case involving the murder of a prominent traditional Baganda leader, entrepreneur Daniel Bbosa.
The killing in Kampala was reportedly carried out by hired gunmen as Bbosa returned home from work in February of this year.
Mpigi district, although densely populated, remains semi-rural, with agriculture dominating local trade.
Coffee and bananas are the major cash crops and staple foods, and there is a major road linking the villages to the neighboring countries of the Democratic Republic of the Congo, Rwanda, and Tanzania.