The former Everton striker had drawn his opening two games, first earning a point at Coleraine before his side were held by Carrick Rangers. His long-awaited home debut then appeared to be heading towards another frustrating result after Bangor equalised late on.
Jordan Stewart had given Glentoran the lead, celebrating his recent contract extension with the opening goal. But Ben Cushnie brought Bangor level in the 83rd minute, setting up a tense finish.
Weatherup provided the decisive moment with three minutes remaining, racing onto a long ball before finishing between the legs of goalkeeper Patrick Solis Grogan.
Ferguson was delighted to finally get his first three points in front of the home supporters.
"I'm delighted to get my first three points at home, I'm absolutely made up," he said.
"The boys kept on going until the very end."
The manager's first scheduled match, against Ballymena United on 24 October, had been postponed because of Storm Boelo and a waterlogged pitch. His eventual debut came away to Coleraine, where Stewart scored a late equaliser to secure a 1-1 draw.
A goalless encounter at Carrick Rangers followed, leaving Ferguson still waiting for a win before Friday's meeting with Bangor.
Stewart's goal had put Glentoran in a promising position, but Ferguson was unhappy with how his team responded as the visitors pushed forward after the break.
The manager felt his side had allowed Bangor back into the contest by dropping too deep, with former Glentoran player Cushnie taking advantage to level the score.
Weatherup, introduced from the bench in place of Pat Hoban, then supplied the finish that Ferguson's side needed. The 18-year-old's goal was particularly satisfying for his manager, who praised the character shown by his players after the equaliser.
"We showed good character. We showed a bit of resilience," Ferguson said.
"They put us under pressure in the second half but that will happen. You're not going to dominate a game for the whole game at any level.
"I was delighted the boys showed character and resilience to get back into the game."
The result gives Ferguson a first victory in Northern Ireland after a difficult introduction to the Irish Premiership, although he knows there is still plenty of work to do.
Glentoran remain some distance behind the leading sides, with Linfield and Coleraine both winning on Friday. Larne also collected three points, leaving Ferguson's team with ground to make up if they are to challenge towards the top of the table.
He has been encouraged by the defensive work in his opening matches but wants to see more from his players in possession and in the final third.
"I think we need to get better overall," he said.
"We're not conceding many goals, but we just have to create more.
"That's the hardest part of the game, but we look solid and that's the main thing.
"I'm happy with the group and the league is very close together, from top to bottom."
Ferguson is not looking too far ahead as he settles into his new job, preferring to concentrate on building momentum one match at a time.
"We've just got to keep going. I think that is seven or eight games undefeated now," he added.
"We have to keep going and keep winning. We're a big club and the fans want us to keep winning, and I'm glad we got the three points for them today."
The win gives Ferguson the home start he had been waiting for and a platform to build on as Glentoran look to close the gap on the teams above them.