Xi Jinping and Shinzo Abe met on the sidelines of the Asia-Pacific Economic Co-operation (APEC) summit in Beijing.
Their first meeting included a public handshake with little sign of warmth.
In a speech to APEC, President Barack Obama has meanwhile announced big changes to visa arrangements with China.
Multiple entry short-term visas for businessmen and tourists will be extended to 10 years – up from one year.
Those for students rise from one year to five.
Barack Obama also stressed the importance of ties between China and the US, saying “the US welcomes the rise of a prosperous, peaceful and stable China.”
His comments come amid underlying tension between the US and China over Beijing’s growing regional influence.
Relations between China and Japan have long been soured by a row over islands in the East China Sea.
The uninhabited but strategically important islands, known as Diaoyu by China and Senkaku by Japan, are controlled by Japan but also claimed by China.
Tokyo’s decision to purchase three of them from their private Japanese owner in September 2012 led to an escalation in a dispute that has rumbled quietly for years.
The Chinese and Japanese leaders interacted awkwardly as they posed for an unsmiling photo after their talks.
Shinzo Abe said the meeting – which came three days after the two sides agreed to work to prevent the territorial dispute from escalating – was “the first step for improving ties by returning to mutually beneficial relations based on common strategic interests”.
He also said they had agreed to start preparations to establish a maritime crisis mechanism.
There have been fears that a clash – accidental or otherwise – between Chinese and Japanese paramilitary vessels patrolling waters around the disputed islands could trigger a conflict.
Xi Jinping told Shinzo Abe that China hoped Japan would follow a path of peaceful development and adopt prudent military and security policies.
Relations have also been hampered by what China sees as Japan’s failure to adequately acknowledge its war-time conduct.
hVep_ICdqooImage source: Wikimedia Commons President-elect Donald Trump celebrated his election victory at the Ultimate Fighting…
Millions of voters across the US chose to return Donald Trump to the White House…
Donald Trump declares victory in the US election as he addresses jubilant supporters in Florida.…
Stocks around the world are rising as Donald Trump appears to be on the cusp…
Donald Trump has won Pennsylvania, North Carolina and Georgia and taken a lead over Kamala…
Quincy Jones, the celebrated musician and producer who worked with Michael Jackson, Frank Sinatra, Ray…