Chelsea led 3-0 inside half an hour through Romeo Lavia, Pedro Neto and João Pedro, before Malick Yalcouyé pulled one back before the break. Brighton drew level to within a goal when João Pedro turned a Pascal Gross cross into his own net just after the hour, but Cole Palmer restored the two-goal cushion before Gross's stoppage-time header made for a nervy finish.
Fabian Hürzeler was left to rue his side's inability to hold their nerve.
"Frustrating part is that we controlled the game, scored three goals away and had a lot of chances but we lost. We didn't find the right balance between controlling the game and being good defensively. It was small margins and Chelsea had so much quality to punish us," he said.
"Frustrating but a lot of positives to take away as well. We came here to win the game and the boys showed that attitude even at 3-0 down. We will get stronger because of this game."
Hürzeler singled out debutant Malick Yalcouyé for special praise.
"He had an impressive performance today. He showed attitude and personality at a tough place to go. In the end, we would have been more happy if we could take points away from this game. But it's important to emphasise these kind of performances," he said, adding that midfielder Yasin Ayari had suffered "a small issue with his ankle" but should be back the following weekend.
Alonso was delighted with the result but acknowledged his side's dip in control after the opening half hour.
"Hard fought win but this is the Premier League. We knew that Brighton were tough to beat recently in these games. It was good to finish with the win but many things happened," he said. "It was a brilliant first half hour. We could have scored more than three. But we were sitting too deep after that. This is a process. We need to develop with these wins which give you the confidence."
The Chelsea boss was full of praise for João Pedro despite his own goal.
"He has been terrific in all the games he has played this season and also the pre-season. He has the desire to play each game 100 per cent and have a terrific season. We need to help him but it's important he has this hunger to do it," Alonso said.
On new signing Martinez, who conceded three on his debut, he added: "It's great to have him. We saw that even today. Even if we conceded three goals, he had a straight impact and he helped the team really well. He is a great personality and a great goalkeeper. His expression and his charisma influence the rest of the team." Asked about the transfer window, Alonso said: "We will see what happens. We are ready for whatever. Most important is the team."