An own goal from Pedro Henrique on 25 minutes gave Palace the lead on the night, converting Daniel Muñoz's driven cross-shot past Shakhtar goalkeeper Dmytro Riznyk. Shakhtar levelled through a superb Eguinaldo strike to give brief hope of a comeback, before Ismaila Sarr settled the tie in the second half.
The Senegalese forward, who has now scored nine Conference League goals this season to become the competition's outright top scorer, ensured a historic night at SE25.
Crystal Palace will face Rayo Vallecano in the final in Leipzig on May 27.
Head coach Oliver Glasner, who is leaving the club at the end of the season, spoke with evident emotion to TNT Sports after the final whistle.
"It was really exciting. Everyone could see how good Shakhtar are — so huge credit to my players. They stuck to the plan and worked so hard."
He pointed to a moment in the second half that he said defined the collective character of his squad.
"You look at the second goal — three players got fouled but they always continued play. This shows everything about this team."
Glasner revealed he had spoken to chairman Steve Parish before the match about just how far this group has come under his watch.
"I said to him: 'Do you ever think we would be playing in the semi-final of a European competition at Selhurst Park?' He said 'no'. I said 'when we talked for the first time three years ago, I didn't think about it either.' This is what you can achieve if you work hard and stick together in tough moments."
He described the dressing room atmosphere he has built as the foundation of everything Palace have achieved.
"There was always more noise around us rather than inside. When you stick together in difficult moments, it is possible — and you get the reward."
Earlier in the week, Glasner had shown the squad three clips from the first leg in his pre-match briefing — none of them tactical.
"They were not football topics. We showed them mentality — how they fought for each other, the commitment for the team. We showed them when the three centre-backs and Dean Henderson were jumping chest to chest, celebrating together. This is amazing."
It is the 50th win of Glasner's Palace reign and the greatest achievement in the south London club's 151-year history. He will leave in the summer regardless of what happens in Leipzig, making the final his farewell and, potentially, the defining moment of an extraordinary three-year chapter at Selhurst Park.