Around 140,000 people were killed in Hiroshima when the US dropped its atomic bomb in 1945.
John Kerry was joined by foreign ministers from the G7 group of nations who are holding talks in the city.
They laid wreaths at the memorial and observed a minute of silence.
The ministers also visited the Bomb Dome, over which the A-bomb exploded, and the nearby Hiroshima museum, which tells the personal stories of people who died.
John Kerry wrote in the museum guestbook that it was “a stark, harsh, compelling reminder not only of our obligation to end the threat of nuclear weapons, but to rededicate all our effort to avoid war itself”.
At 08:10 local time on August 6, 1945, the US B-29 bomber the Enola Gay dropped a uranium bomb nicknamed “Little Boy” on Hiroshima.
The bomb exploded 1,800ft above what is now the Hiroshima Peace Dome.
About 70,000 people died immediately. At least 140,000 people had died by the end of the year through injury and the effects of radiation.
The bombing, and a second bomb dropped on Nagasaki three days later, forced Japan to surrender, initiating the end of World War Two.
In 2008, House of Representatives Speaker Nancy Pelosi visited Hiroshima, but US diplomats have largely avoided official visits.
Many in the US believe the bombing was necessary to end the war, and do not want their leaders to take any action which might be seen as an apology.
John Kerry previously said his time in Hiroshima would “revisit the past and honor those who perished” but stressed that his trip was “about the present and the future”.
It also comes amid efforts to strengthen the relationship between the US and Japan, particularly with growing concern about China’s assertiveness in territorial disputes in Asia, affecting Japan and other US allies.
President Barack Obama is attending a G7 leaders’ summit elsewhere in Japan in May, and there are reports he is considering a stop in Hiroshima.
If it happens, it will be the first time a sitting US president visits Hiroshima.
The US House Ethics Committee has voted to release its report on former Republican Representative…
ABC News has agreed to pay $15 million to President-elect Donald Trump to settle a…
South Korea’s parliament has voted to impeach President Yoon Suk Yeol over his failed attempt…
Israeli war planes have carried out more than 100 air strikes in Syria on December…
President-elect Donald Trump has threatened to impose 100% tariffs on the BRICS countries if they…
Syrian troops have withdrawn from the city of Aleppo following an offensive by rebels opposed…