Capt. Amadou Sanogo, whose troops seized power in a coup last week, said in a statement that his soldiers had ceded control to avoid fighting in residential areas.
There are reports of casualties but no figures have been given.
The loss of Gao is a serious blow to the coup leaders.
They deposed the president in protest at what they saw as the poor conduct of the fight against the Tuareg rebels.
The historic city of Timbuktu is now the only major northern town that remains under the control of the Malian army.
Rebel sources say they are already positioned in its outskirts and residents fear fighting could erupt soon.
Regional group Ecowas has put 2,000 troops on standby in case of a possible intervention in Mali.
It has threatened to close land borders, freeze assets and impose a financial blockade if the army does not stand down before Monday.
The rebels took Gao hours after another town, Kidal, fell to them.
Witnesses reported heavy gunfire coming from the main military camp in Gao and helicopters engaging rebel fighters.
Before the coup, Malian forces had struggled to drive back the rebels and officers had complained that the army needed more equipment to fight.
Capt. Amadou Sanogo has asked for foreign help to fight the rebels but has been condemned over the coup.
Members of the military leadership are in neighboring Burkina Faso for talks with President Blaise Compaore, who is mediating in the crisis.
Separatist rebels seeking to carve out a desert homeland began a rebellion in the West African state in January.
The National Movement for the Liberation of Azawad (MNLA) want an independent north while a smaller, Islamist group called the Ansar Edine wants to impose Sharia law.
Azawad is the Tuareg name for their home region in the Sahara Desert.
The Tuaregs have launched several rebellions over the years, complaining that the government in Bamako ignores them.
The conflict has been fuelled by the return of Tuareg fighters from Libya last year after fighting for the late Muammar Gaddafi or his opponents.
It appears these fighters are heavily armed with looted weapons.
Analysts say the rebels have taken advantage of Mali’s military coup to move swiftly across the north.
The US House Ethics Committee has voted to release its report on former Republican Representative…
ABC News has agreed to pay $15 million to President-elect Donald Trump to settle a…
South Korea’s parliament has voted to impeach President Yoon Suk Yeol over his failed attempt…
Israeli war planes have carried out more than 100 air strikes in Syria on December…
President-elect Donald Trump has threatened to impose 100% tariffs on the BRICS countries if they…
Syrian troops have withdrawn from the city of Aleppo following an offensive by rebels opposed…