Capt. Amadou Sanogo, whose troops seized power in a coup last week, said in a statement that his soldiers had ceded control to avoid fighting in residential areas.
There are reports of casualties but no figures have been given.
The loss of Gao is a serious blow to the coup leaders.
They deposed the president in protest at what they saw as the poor conduct of the fight against the Tuareg rebels.
The historic city of Timbuktu is now the only major northern town that remains under the control of the Malian army.
Rebel sources say they are already positioned in its outskirts and residents fear fighting could erupt soon.
Regional group Ecowas has put 2,000 troops on standby in case of a possible intervention in Mali.
It has threatened to close land borders, freeze assets and impose a financial blockade if the army does not stand down before Monday.
The rebels took Gao hours after another town, Kidal, fell to them.
Witnesses reported heavy gunfire coming from the main military camp in Gao and helicopters engaging rebel fighters.
Before the coup, Malian forces had struggled to drive back the rebels and officers had complained that the army needed more equipment to fight.
Capt. Amadou Sanogo has asked for foreign help to fight the rebels but has been condemned over the coup.
Members of the military leadership are in neighboring Burkina Faso for talks with President Blaise Compaore, who is mediating in the crisis.
Separatist rebels seeking to carve out a desert homeland began a rebellion in the West African state in January.
The National Movement for the Liberation of Azawad (MNLA) want an independent north while a smaller, Islamist group called the Ansar Edine wants to impose Sharia law.
Azawad is the Tuareg name for their home region in the Sahara Desert.
The Tuaregs have launched several rebellions over the years, complaining that the government in Bamako ignores them.
The conflict has been fuelled by the return of Tuareg fighters from Libya last year after fighting for the late Muammar Gaddafi or his opponents.
It appears these fighters are heavily armed with looted weapons.
Analysts say the rebels have taken advantage of Mali’s military coup to move swiftly across the north.
Quincy Jones, the celebrated musician and producer who worked with Michael Jackson, Frank Sinatra, Ray…
Misleading allegations, rumours and outright lies about voting and fraud are flooding online spaces in…
At least 158 people have died in Spain's worst flooding disaster in generations. On October…
Google has been fined two undecillion (a two followed by 36 zeroes) roubles by a…
Embarking on a home remodel is an exciting journey, promising enhanced comfort, increased property value,…
The US presidential candidates continued to campaign across key swing states on October 20. Footage…