Franck Kessie put Ivory Coast ahead in the 30th minute, finishing from close range after Manuel Neuer had saved Amad Diallo's initial effort, and the African side held that lead deep into the second half.
Undav, introduced just after the hour mark, levelled in the 68th minute with a volley from a Nadiem Amiri cross, before completing his turnaround with a stoppage-time winner in the 94th minute, latching onto a pass from Felix Nmecha to drill home from inside the box.
The win means Germany have reached the knockout stages for the first time since 2014, after group-stage exits at the previous two World Cups, with a game still to play.
Defender Jonathan Tah praised his side's mentality after the match.
"We had exactly what it takes to succeed in a tournament. A winning mentality, team spirit. We didn't give up; we just kept going. The guys coming off the bench brought new energy to the game. Deniz deserves special mention. He was outstanding," Tah said.
Ivory Coast, who looked set for a famous win for long periods, will now have to win their final group game against Curacao to secure their own place in the knockout rounds for the first time in the country's history.
Germany, who thrashed Curacao 7-1 in their opening match, will secure top spot in Group E with a point against Ecuador in their final fixture, or sooner if Ecuador fail to beat Curacao.
It marked a dramatic turnaround for a Germany side that had looked second-best for much of the contest, with Ivory Coast's disciplined defensive shape and direct counterattacking threat briefly putting one of the tournament's bigger upsets in sight.
Undav's introduction shifted the balance decisively, his second goal of the tournament arriving in stoppage time to spare Germany's blushes and stretch his remarkable scoring record to 10 goals in just 11 appearances for his country.