The north London side looked to be heading on their way to a comfortable victory over the South Wales team after Theo Walcott put them 2-0 up in the first half.
However, the visitors – playing their first game under new boss Bob Bradley – fought back and made it 2-1 before half-time thanks to Gylfi Sigurdsson’s goal.
Birthday boy Mesut Ozil then made it 3-1 with a fine goal in the 57th minute at The Emirates, but Borja Baston tapped home to set up a tense finale.
Arsenal’s task was made more difficult after new signing Granit Xhaka was shown a straight red card for a petulant trip.
However, the Gunners held on to take all three points and move level with Manchester City at the top of the table.
Morgan tweeted after the final whistle:
— Piers Morgan (@piersmorgan) 15 October 2016
Arsenal take on Ludogorets in the Champions League at The Emirates on Wednesday night.
MORE: Man United latest news
MORE: Arsenal latest news
MORE: Chelsea latest news
MORE: Liverpool latest news