The agency did not give details, but reports suggest the accusations involve prostitution in Cartagena, which is hosting the Summit of the Americas, and at least one agent was found to be cavorting with prostitutes.
Agency spokesman Edwin Donovan told said the agents had been replaced and Barack Obama’s security would not be affected.
Barack Obama arrived in Colombia on Friday to attend the summit.
In a statement, Edwin Donovan said the agency took allegations of misconduct “very seriously” and said an investigation had been launched.
“There have been allegations of misconduct made against Secret Service personnel in Cartagena, Colombia prior to the President’s trip,” he said.
“Because of this, those personnel are being relieved of their assignments, returned to their place of duty, and are being replaced by other Secret Service personnel.”
According to The Washington Post, 12 agents were recalled, and at least one was suspected of involvement with prostitutes.
More than 30 leaders are in the Colombian port city for the summit, which will see talks on economic policy and trade.
Even before the incident with the agents, the summit seemed likely to be an awkward one for the US government, with debates on the legalization of drugs and sanctions against Cuba promised.
Ecuadorean President Rafael Correa is boycotting the summit because of Cuba’s exclusion, and the attendance of Venezuelan President Hugo Chavez is in doubt.
Earlier, Colombian police said two small explosions took place in Cartagena, but caused no casualties or damage.
Those explosions followed at least one other blast in Bogota near the US embassy, officials said.
Image source: Wikimedia Commons President-elect Donald Trump celebrated his election victory at the Ultimate Fighting…
Millions of voters across the US chose to return Donald Trump to the White House…
Donald Trump declares victory in the US election as he addresses jubilant supporters in Florida.…
Stocks around the world are rising as Donald Trump appears to be on the cusp…
Donald Trump has won Pennsylvania, North Carolina and Georgia and taken a lead over Kamala…
Quincy Jones, the celebrated musician and producer who worked with Michael Jackson, Frank Sinatra, Ray…