Official results showed Maithripala Sirisena, a former ally of the incumbent, had won 51.3% of the vote.
Mahinda Rajapaksa, in office since 2005, said on Twitter he looked forward to a peaceful transition of power.
His supporters credit him with ending the civil war and boosting the economy, but critics say he had become increasingly authoritarian and corrupt.
Maithripala Sirisena had already received promises of support from Tamil and Muslim leaders before the election.
However, the result shows Maithripala Sirisena also picked up a significant portion of the majority Sinhalese vote, most of whom solidly supported Mahinda Rajapaksa in previous elections.
Maithripala Sirisena, who is due to be sworn in at 18:00 local time, was surrounded by supporters and photographers as he left his offices earlier.
“We must allow all Sri Lankans to enjoy this victory, therefore avoid hurting or insulting people,” he said.
“I ask for everyone’s support to take this process forward with compassion and love.”
Mahinda Rajapaksa was seeking a third term in office after he changed the constitution to scrap the two-term limit.
But before the results were announced, Mahinda Rajakpaksa’s press officer said the president “concedes defeat and will ensure a smooth transition of power bowing to the wishes of the people”.
Both Mahinda Rajapaksa and Maithripala Sirisena are Sinhalese, the majority ethnic group in Sri Lanka.
They were allies until November when Maithripala Sirisena, the health minister in Mahinda Rajapaksa’s government, announced his surprise candidacy.
Turnout in many areas was above 70%, roughly in line with previous elections, with no reports of major incidents disrupting the voting process.
In Jaffna and Trincomalee, two of the main Tamil strongholds, turnout was higher than previous national elections.
Mahinda Rajapaksa was last elected in 2010 when he defeated his former army chief Sarath Fonseka, who was later jailed on charges of implicating the government in war crimes.
NG-2gtqPuE8Image source: Wikimedia Commons President-elect Donald Trump celebrated his election victory at the Ultimate Fighting…
Millions of voters across the US chose to return Donald Trump to the White House…
Donald Trump declares victory in the US election as he addresses jubilant supporters in Florida.…
Stocks around the world are rising as Donald Trump appears to be on the cusp…
Donald Trump has won Pennsylvania, North Carolina and Georgia and taken a lead over Kamala…
Quincy Jones, the celebrated musician and producer who worked with Michael Jackson, Frank Sinatra, Ray…