Eleven US Secret Service agents and five soldiers were recalled to Washington last week after nearly all of them allegedly brought prostitutes back to their rooms at a luxury hotel in Colombia.

The agents were in Cartagena, Colombia, to scout out the security situation ahead of President Barack Obama’s trip there for the Summit of the Americas this weekend.

But the agents – who have now been placed on leave – reportedly capped off a week of heavy drinking at the beachfront Hotel Caribe in Cartagena by cavorting with prostitutes.

The partying was exposed when one of the women caused a commotion after an agent refused to pay her. Local police and hotel security were called.

Five U.S. military personnel, who were working with the Secret Service, could also be involved in misconduct and have been confined to their rooms and ordered not to have contact with others. They face possible disciplinary charges.

Rep Peter King, who was briefed on the incident, said 11 agents, many of them married, were in the team that was recalled to the U.S. – instead of 12 as originally reported – and that “close to” all of them were involved.

Peter King said he was told that anyone visiting the hotel overnight was required to leave identification at the front desk and leave the hotel by 7:00 a.m.

When a woman failed to do so, it raised questions among hotel staff and police, who investigated.

They found the woman with the agent in the hotel room and a dispute arose over whether the agent should have paid her. Peter King said he was told that the agent did eventually pay the woman.

The incident was reported to the U.S. embassy, prompting further investigation.

During their week-long stay at the five-star hotel in Cartagena, the agents were seen drinking heavily, according to waiters there.

A number of the White House staff and traveling press corps were also staying at the hotel.

The White House said Barack Obama had been briefed about the incidents but would not comment on his reaction.

“The President does have full confidence in the United States Secret Service,” presidential spokesman Jay Carney said when asked.

Jay Carney insisted the matter was more a distraction for the media than Barack Obama. But Secret Service assistant director Paul Morrissey said in a statement: “We regret any distraction from the Summit of the Americas this situation has caused.”

The Washington Post reported that Jon Adler, president of the Federal Law Enforcement Officers Association, said the accusations related to at least one agent having involvement with prostitutes.

The association represents federal law enforcement officers, including the Secret Service.

Ronald Kessler, a former Post reporter and the author of a book about the Secret Service, told the newspaper that he had learned that among the agents involved, several are married.

Although prostitution is legal in parts of Colombia, the Secret Service is said to consider solicitation inappropriate behavior for its agents.

Colombia has become known as “the Thailand of Latin America” for its loose laws on prostitution and the easy availability of sex workers.

The incident threatened to overshadow Barack Obama’s economic and trade agenda at the Summit of the Americas and embarrass the U.S.

Secret Service spokesman Ed Donovan would not confirm that prostitution was involved, saying: “The Secret Service takes all allegations of misconduct seriously.”

“These personnel changes will not affect the comprehensive security plan that has been prepared in advance of the President’s trip,” said Ed Donovan of the conference in the Colombian port city attended by Barack Obama and more than 30 world leaders.

He said the allegations of misconduct were related to activity before the president’s arrival on Friday night.

Barack Obama was attending a leaders’ dinner on Friday night at Cartagena’s historic Spanish fortress.

He was due to attend summit meetings with regional leaders on Saturday and Sunday.

Those involved had been sent back to their permanent place of duty and were being replaced by other agency personnel, Ed Donovan said.

The matter was turned over to the agency’s Office of Professional Responsibility, which handles the agency’s internal affairs.

On the steamy streets of Cartagena, a resort city with a teeming prostitution trade, there was condemnation for the Secret Service agents for what residents saw as abusing their station and dishonoring their country.

Edwin Yepes, a souvenir vendor, said: “They are supposed to come here and set an example.

“We are an inferior culture, and so it’s better if they don’t come than if they damage our image of them.”