The agency did not give details, but reports suggest the accusations involve prostitution in Cartagena, which is hosting the Summit of the Americas, and at least one agent was found to be cavorting with prostitutes.
Agency spokesman Edwin Donovan told said the agents had been replaced and Barack Obama’s security would not be affected.
Barack Obama arrived in Colombia on Friday to attend the summit.
In a statement, Edwin Donovan said the agency took allegations of misconduct “very seriously” and said an investigation had been launched.
“There have been allegations of misconduct made against Secret Service personnel in Cartagena, Colombia prior to the President’s trip,” he said.
“Because of this, those personnel are being relieved of their assignments, returned to their place of duty, and are being replaced by other Secret Service personnel.”
According to The Washington Post, 12 agents were recalled, and at least one was suspected of involvement with prostitutes.
More than 30 leaders are in the Colombian port city for the summit, which will see talks on economic policy and trade.
Even before the incident with the agents, the summit seemed likely to be an awkward one for the US government, with debates on the legalization of drugs and sanctions against Cuba promised.
Ecuadorean President Rafael Correa is boycotting the summit because of Cuba’s exclusion, and the attendance of Venezuelan President Hugo Chavez is in doubt.
Earlier, Colombian police said two small explosions took place in Cartagena, but caused no casualties or damage.
Those explosions followed at least one other blast in Bogota near the US embassy, officials said.
Quincy Jones, the celebrated musician and producer who worked with Michael Jackson, Frank Sinatra, Ray…
Misleading allegations, rumours and outright lies about voting and fraud are flooding online spaces in…
At least 158 people have died in Spain's worst flooding disaster in generations. On October…
Google has been fined two undecillion (a two followed by 36 zeroes) roubles by a…
Embarking on a home remodel is an exciting journey, promising enhanced comfort, increased property value,…
The US presidential candidates continued to campaign across key swing states on October 20. Footage…