Anan Khalaili's early strike put Crystal Palace ahead before Emersonn levelled and Axel Disasi was sent off for catching Jack Clarke with his studs, leaving the hosts to play the majority of the match with 10 men.
Leif Davis headed Ipswich in front on the stroke of half-time, and although Jorgen Strand Larsen equalised with 15 minutes remaining, substitute Zian Flemming struck a stoppage-time winner to complete a dramatic 3-2 comeback for the visitors at Selhurst Park.
Gary O'Neil was delighted with his side's resilience in what was already Ipswich Town's second away win of the season.
"Winning away in the Premier League is tough against the teams playing in Europe this season, so I'm really pleased to have two wins on the board after four matches, it's a big plus for us," O'Neil said.
"I'm really pleased with the group, what they gave, lots of emotion, lots of effort to respond to setbacks, respond to positives.
"It wasn't perfect, of course, but we won. We found a way to win. We deserved to win as well, which is important."
O'Neil praised his side's response after conceding an equaliser similar to one they had shipped against Liverpool.
"We conceded a goal not too dissimilar to the Liverpool one, but we scored a fantastic equaliser," he said. "Even at the end, once it goes 2-2, the lads just jump straight back on and started probing again. It takes a lot to win away in the Premier League."
Pierre Sage was left to rue individual moments that shaped the contest.
"We are really frustrated because of the context of the game," Sage said. "We conceded with their first opportunity and the red card meant I had to re-organise the team in a 3-5-1. In the second half we came back with more possession, we created chances and had the ball, and we got back to 2-2. We had the opportunity for the 3-2 but we missed it. And then at the end it's really hard to concede in this manner. I'm frustrated for the players because they gave a lot on the pitch."
Sage acknowledged a recurring defensive issue despite his side conceding fewer clear-cut chances than usual.
"We saw the same goal three times, a cross and then not winning the ball in our box," he said. "We have conceded too many crosses and are not winning the ball in the box, this is a new problem. It's clear we have to be more efficient in that box."
Sage also praised Strand Larsen's energetic display and said captain Dean Henderson, out with a foot injury, should return for the visit of Nottingham Forest.