The Social Democrats and the Greens won a single-seat majority in the state legislature, a region of 8 million people in north-western Germany.
The election is seen as a bellwether for national elections in September.
The Lower Saxony defeat has set alarm bells ringing for the chancellor.
Angela Merkel’s CDU coalition has lost a number of state elections as she seeks a third term as Germany’s chancellor.
Sunday night’s knife-edge finish saw the SPD and Greens winning a combined 46.3% of the vote to the centre right’s 45.9%.
David McAllister, the incumbent leader of Lower Saxony’s government and close ally of Chancellor Merkel, had been hoping for re-election.
He was born in Berlin to a German mother and a Scottish father and is seen as a possible successor to Chancellor Merkel as CDU leader.
The Social Democrats (SPD) enjoyed a comfortable lead over the incumbents in the run-up to the poll, but it evaporated as polling day approached.
The SPD leader in Lower Saxony, Stephan Weil, said before the result that a victory in the state polls as a sign that his party will be taken seriously in September’s national elections.
Since Angela Merkel’s re-election as chancellor in 2009, the CDU has suffered setbacks in recent state elections, and have lost power to the SPD and Greens in four other states.
There was also concern that the CDU’s coalition partners, the Free Democrats, would not win enough votes to maintain the coalition.
They require 5% of the vote to gain seats in the state legislature – exit polls suggested they had 10%.
Angela Merkel appeared several times on the campaign trail with David McAllister, who has played heavily on his Scottish roots.
Known as “Mac”, he used bagpipes in his election broadcasts. He speaks English with a broad Scottish accent.
vhRbcQjZLRwThree Indian nationals have been arrested and charged over the killing of Sikh separatist leader…
President Joe Biden has urged pro-Palestinian protesters on university campuses to uphold the rule of…
Blue Ivy Carter has joined the voice cast of The Lion King prequel Mufasa: The…
At least five people, including a four-month-old baby, have been killed after dozens of tornadoes…
Harvey Weinstein has been hospitalized just days after his 2020 rape conviction in New York…
Hamas has published a video showing the first proof of life of US and Israeli…