Nico González's 87th-minute strike from 30 yards secured a place in the final so making City the first club in history to reach four consecutive FA Cup finals.
The Championship side, who were relegated from the Premier League last month, gave a robust account of themselves at Wembley. Finn Azaz curled into the top corner in the 79th minute to give Southampton a shock lead, only for Jeremy Doku to level within four minutes with a deflected effort.
González, on from the bench, then produced the decisive moment — a piledriver from outside the box that flew into the top corner with three minutes of normal time remaining.
Speaking after the game, the Spanish midfielder reflected on what it meant.
"I don't remember scoring a better goal, maybe last season, but it wasn't that good to be fair. It's amazing to score at the end. It's been a long time without playing. And being here, in this amazing stadium, with this atmosphere, it's an amazing feeling."
He added: "I think I had a few chances to shoot because they were defending so deep. We were playing really good in the second half and making them be in their own area. I was waiting for the chance and I had it, so I gave it everything, I shot with everything and it went in."
González also put the result in the context of a remarkable seven days for the club.
"It's been a really important week for us, it's been an amazing three wins. We're alive in the league which was maybe one month ago, we never thought to arrive with this chance. But it's amazing to arrive to another final."
Manchester City will face Leeds United or Chelsea in the final at Wembley next month. Their opponents will be decided on Sunday.