After a goalless and largely uninspiring first half at Turf Moor, Arsenal brought on the likes of Jack Wilshere and Danny Welbeck as the north London side chased a winner.
And it was Alexis Sanchez who ended up being the hero, when he converted his penalty in the second half of stoppage time after Aaron Ramsey was fouled in the box.
The victory lifted Arsenal up into fourth place in the Premier League table and above the likes of Liverpool FC and Tottenham Hotspur.
Arsenal shareholder Morgan was clearly a happy man as he took to social media during the game, but he was not very impressed by the first-half showing.
Morgan tweeted: “72 minutes into an Arsenal game & I have nothing to say. #BFCvAFC.”
Then, after Arsenal were awarded the penalty late on, he added: “Whooaaaaaa. #afc. BOOOM! 1-0. #sanchez.”
Arsenal will look to make it three wins in a row in the Premier League when they take on Huddersfield Town at The Emirates on Wednesday night.
The north London side are aiming to challenge for the Premier League title this season after having finished fifth in the table last term.
Arsenal have not won the title since their famous unbeaten triumph in the 2003-04 campaign.
MORE: Man United latest news
MORE: Arsenal latest news
MORE: Chelsea latest news
MORE: Liverpool latest news