Mark Shand, 62, was in New York when the incident happened on Tuesday night.
“The duchess, the Prince of Wales and all her family members are utterly devastated by this sudden and tragic loss,” Clarence House said.
He was described as a man of “extraordinary vitality, a tireless campaigner and conservationist”.
Mark Shand was chairman of the Elephant Family, a movement which exists to save the endangered Asian elephant.
In its statement, Clarence House said it was with “deep sadness” that it had to confirm Mark Shand’s death.
It said his “incredible work through the Elephant Family and beyond remained his focus right up until his death”.
The death comes as the Duchess of Cornwall and the Prince Charles, currently understood to be in Scotland, prepare for a high-profile tour of Canada next month.
Mark Shand, the duchess’s only brother, reportedly slipped while lighting a cigarette and hit his head on the pavement outside an after-party for a charity event.
According to the New York Police Department, the emergency services picked him up where he fell outside the Rose Bar and took him to Bellevue Hospital, where he was pronounced dead on Wednesday.
Mark Shand married French actress Clio Goldsmith – the niece of billionaire entrepreneur James Goldsmith and the cousin of Jemima Khan and Zac Goldsmith – in 1990. They have a daughter together called Ayesha, aged 19.
Jemima Khan tweeted her condolences: “RIP Mark Shand: Best man, godfather to my son & my cousin’s ex.”
Her ex-husband, former Pakistan international cricketer and politician Imran Khan also took to the social media site to pay his respects.
Imran Khan tweeted: “Shocked & saddened to learn of the death of one of my closest English friends, Mark Shand, in New York where he was raising funds 4 charity.”
Mark Shand had been in New York for a charity auction at Sotheby’s, which was in aid of unprivileged children and the Elephant Family.
The event was the finale of the month-long Big Egg Hunt NYC by Faberge, where egg sculptures were sold to the highest bidder.
It raised $1.6 million for charity and was reportedly also attended by Princess Eugenie.
In a statement, the Elephant Family described Mark Shand as the head of its family and “a true force for conservation”.
5-jzoHANr20Image source: Wikimedia Commons President-elect Donald Trump celebrated his election victory at the Ultimate Fighting…
Millions of voters across the US chose to return Donald Trump to the White House…
Donald Trump declares victory in the US election as he addresses jubilant supporters in Florida.…
Stocks around the world are rising as Donald Trump appears to be on the cusp…
Donald Trump has won Pennsylvania, North Carolina and Georgia and taken a lead over Kamala…
Quincy Jones, the celebrated musician and producer who worked with Michael Jackson, Frank Sinatra, Ray…