The 20-year-old spent the second half of last season on loan at the Bundesliga club, making 11 appearances.
City will receive a fee in the region of €6million for the centre-back, according to reports.
Simpson-Pusey joined Manchester City's academy at Under-9 level and progressed through the club's youth ranks over more than a decade.
He captained City's Under-18s to FA Youth Cup glory in 2023-24 and was later named Premier League 2 Player of the Season.
The defender made his senior City debut in the 2024-25 Carabao Cup, coming on as a substitute in a defeat at Tottenham Hotspur, and went on to make five further first-team appearances.
He spent the first half of last season on loan at Celtic before that move was cut short and he was sent to Köln in January instead.
Cologne managing director Thomas Kessler said the club were pleased to secure Simpson-Pusey's long-term future after his impact during the loan spell.
"Jahmai has settled in very quickly since his move in the winter and immediately embraced the challenge at this level. His performances have shown the potential he possesses. We are all the more pleased that we have now been able to secure his services long-term," Kessler said.
Simpson-Pusey said his experience in Germany convinced him to commit to the club permanently.
"I played a lot of minutes and the club put a lot of trust in me. I'm thankful for that and that's why I wanted to stay in Köln," he said.
Simpson-Pusey thanked Manchester City as he brought his decade-long association with the club to a close.
"After just over a decade at Manchester City, my time as a Blue has come to an end. To have made my senior debut for City will always hold a special place in my heart," he said.
The move continues City's pattern of cashing in on academy graduates once their development pathway to the first team becomes less clear.