The rebels, who had seized the building last week, reportedly left early on Wednesday. The reason is not clear.
Pro-Russia separatists have captured official buildings in dozens of towns in eastern Ukraine in recent weeks.
The government has accused Russia of helping the rebels and sent troops to restore Kiev’s authority.
Russia’s Interfax news agency quoted a pro-Moscow militiaman in Mariupol as saying government forces had taken over the city hall.
“No shots were heard,” he said.
“All the militia members were removed from the building.”
He added that government forces had started to clear the barricade on the central square with a bulldozer.
There were also reports of clashes between Ukrainian troops and pro-Russia militants in Sloviansk, a rebel stronghold, early on Wednesday.
The Ukrainian government has rejected the pro-Russian activists’ demands for greater autonomy for eastern regions, fearing they could lead to the break-up of the country or more regions being annexed.
Kiev has called a presidential election on May 25, and said it was ready to back new international talks in Geneva, as long as Moscow supported the poll.
On Tuesday, Russian Foreign Minister Sergei Lavrov ruled out holding another round of international talks unless pro-Russian opposition groups were involved.
The separatists in Donetsk have proclaimed a “People’s Republic” and are preparing to hold an independence referendum on Sunday.
ypTXO8osXTMImage source: Wikimedia Commons President-elect Donald Trump celebrated his election victory at the Ultimate Fighting…
Millions of voters across the US chose to return Donald Trump to the White House…
Donald Trump declares victory in the US election as he addresses jubilant supporters in Florida.…
Stocks around the world are rising as Donald Trump appears to be on the cusp…
Donald Trump has won Pennsylvania, North Carolina and Georgia and taken a lead over Kamala…
Quincy Jones, the celebrated musician and producer who worked with Michael Jackson, Frank Sinatra, Ray…