The Social Democrats and the Greens won a single-seat majority in the state legislature, a region of 8 million people in north-western Germany.
The election is seen as a bellwether for national elections in September.
The Lower Saxony defeat has set alarm bells ringing for the chancellor.
Angela Merkel’s CDU coalition has lost a number of state elections as she seeks a third term as Germany’s chancellor.
Sunday night’s knife-edge finish saw the SPD and Greens winning a combined 46.3% of the vote to the centre right’s 45.9%.
David McAllister, the incumbent leader of Lower Saxony’s government and close ally of Chancellor Merkel, had been hoping for re-election.
He was born in Berlin to a German mother and a Scottish father and is seen as a possible successor to Chancellor Merkel as CDU leader.
The Social Democrats (SPD) enjoyed a comfortable lead over the incumbents in the run-up to the poll, but it evaporated as polling day approached.
The SPD leader in Lower Saxony, Stephan Weil, said before the result that a victory in the state polls as a sign that his party will be taken seriously in September’s national elections.
Since Angela Merkel’s re-election as chancellor in 2009, the CDU has suffered setbacks in recent state elections, and have lost power to the SPD and Greens in four other states.
There was also concern that the CDU’s coalition partners, the Free Democrats, would not win enough votes to maintain the coalition.
They require 5% of the vote to gain seats in the state legislature – exit polls suggested they had 10%.
Angela Merkel appeared several times on the campaign trail with David McAllister, who has played heavily on his Scottish roots.
Known as “Mac”, he used bagpipes in his election broadcasts. He speaks English with a broad Scottish accent.
vhRbcQjZLRwThe US House Ethics Committee has voted to release its report on former Republican Representative…
ABC News has agreed to pay $15 million to President-elect Donald Trump to settle a…
South Korea’s parliament has voted to impeach President Yoon Suk Yeol over his failed attempt…
Israeli war planes have carried out more than 100 air strikes in Syria on December…
President-elect Donald Trump has threatened to impose 100% tariffs on the BRICS countries if they…
Syrian troops have withdrawn from the city of Aleppo following an offensive by rebels opposed…