Two army helicopters were scrambled in response, a local official told AFP news agency by phone.
Gao, with a population of 87,000, more than twice the size of Kidal, hosts one of the biggest garrisons in the north.
Separatist rebels seeking to carve out a desert homeland began a rebellion in the west African state in January.
A regional group, the Economic Community of West African States (Ecowas), has placed on alert a peacekeeping force of 2,000 soldiers, Ivory Coast President Alassane Ouattara was quoted as saying by AFP news agency.
After a coup by disgruntled military officers in Mali a week ago, Ecowas has threatened to close land borders, freeze assets and impose a financial blockade if the army does not stand aside before Monday.
“We can hear heavy fire coming in the direction of the main military camp,” a Reuters reporter said.
“People here are running all over the place and all the shops are closing.”
Mahamane Diakite, an aide to the governor of Gao, told AFP: “We can hear heavy weapons fire. We have also seen two helicopters taking off to shoot. Rebels have entered the town.”
Malians with family members in Gao say the city is under attack from multiple rocket launchers, the Associated Press reports.
Rebels can expect to meet tougher resistance in Gao, where the majority of troops are from the Bambara tribe, unlike Kidal, where the majority of troops were Tuareg.
Before the coup, Mali’s government forces had struggled to drive back the rebels.
The mid-ranking officers who overthrew the government said the army needed more equipment to fight.
Their leader, Capt. Amadou Sanogo, has asked for foreign help to tackle the rebels but has been condemned over the coup.
Three members of the military leadership have gone to neighboring Burkina Faso for talks with President Blaise Compaore, who is mediating in the crisis.
One of the three – junta chief of staff Colonel Moussa Sinko Coulibaly – said after the talks that he agreed on the need to restore normality.
“There should be a regular and normal constitutional life, and now, the way to establish this, that is what we are going to discuss,” he said.
The Tuareg fought side by side with Islamist fighters to take over Kidal.
However, it is not clear how they will share their success.
Rebels from the National Movement for the Liberation of Azawad (MNLA) want an independent north while a smaller, Islamist group called the Ansar Edine wants to impose Sharia law.
Azawad is the Tuareg name for their home region in the Sahara Desert.
The Tuaregs have launched several rebellions over the years, complaining that the government in Bamako ignores them.
The conflict has been fuelled by the return of Tuareg fighters from Libya last year after fighting for the late Muammar Gaddafi or his opponents.
It appears these fighters are heavily armed with looted weapons.
Analysts say the rebels have taken advantage of the recent military coup to move swiftly from target to target across the north.
If Gao falls, the only major town in government hands in the north will be Timbuktu.
Image source: Wikimedia Commons President-elect Donald Trump celebrated his election victory at the Ultimate Fighting…
Millions of voters across the US chose to return Donald Trump to the White House…
Donald Trump declares victory in the US election as he addresses jubilant supporters in Florida.…
Stocks around the world are rising as Donald Trump appears to be on the cusp…
Donald Trump has won Pennsylvania, North Carolina and Georgia and taken a lead over Kamala…
Quincy Jones, the celebrated musician and producer who worked with Michael Jackson, Frank Sinatra, Ray…