Julian Nagelsmann's Germany had gone ahead and appeared in control before Paraguay drew level and the match could not be separated after extra time. Orlando Gill's heroics in the shootout, combined with Jose Canale's decisive kick, secured a famous victory.
Paraguay captain Gustavo Gomez was overcome with pride after the win.
"I think the feeling is hard to explain," he said. "I'm very proud of my teammates, of the group. I think I said in the last interview that this group deserved one more game, and with everything we've been through, I'm most impressed by our unity. We have incredible strength to face any situation.
"Germany knew they needed to work incredibly hard to beat us."
Gill, who was central to Paraguay's survival, dedicated the win to his nation.
"An immense thrill. It was a tough match. We managed to hold on," he said. "We opened the scoring, they equalised, but then we managed to keep it up. Obviously, we analysed every player and every detail of the penalty takers. Thank God I was able to save two penalties. This is a privilege; we eliminated a champion. This is dedicated to all Paraguayans."
Nagelsmann was clearly devastated and delivered a frank assessment of where Germany fell short.
"There's a real sense of dejection in the dressing room," he said. "Unfortunately, that's how football works sometimes, some teams can win with simple means. And you have to defend against those means consistently. We took too long to make our opponent chase the game.
"We could have put the ball into the box far more often. We have to settle the game before it gets to penalties. Our build-up play was too slow."
Arsenal midfielder Kai Havertz was equally blunt in his self-criticism.
"I'm a little lost for words," Havertz said. "This is my second World Cup and both times it came to nothing. All I can do is apologise. I thought we didn't play bad football at the last few tournaments, but something was always missing. And it was the same today. We have to take a hard look at ourselves, especially the players, and I'm leaving the coach out of that."