Robin van Persie netted against his former employers in the third minute to give the hosts the perfect start, before Wayne Rooney missed a penalty just before half-time.
Despite dominating a subdued Arsenal, United failed to build on their lead until the 67th minute, when Patrice Evra headed home.
Arsenal midfielder Jack Wilshere was then sent off for a second bookable offence, before Santi Cazorla netted a consolation deep into stoppage time.
“It looked as if it was going to be self destruction,” Ferguson, whose side moved top of the table, told Sky Sports. “It only takes a second to lose a goal.
“They had some possession without really threatening us, but thank god we got the second one because the fact they scored right on full-time would have been an embarrassment [if we’d only have been 1-0 up].
“It was a strange game and the chances we missed… well, we should have put the game to bed a long, long time ago. If you see the scoreline you’d think it was a close game, but it wasn’t.
“There was a lack of urgency in our game throughout. We were far too casual. It was an unusual Manchester United/Arsenal game. There were a lot of bookings but it never really got anywhere near the Manchester United/Arsenal games of the past.
“I was just really disappointed in our ability to hammer home the advantage,” he added.
“I spoke about goal difference to the players at the start of the season. We lost [the title] on goal difference last season and I don’t want to see that again. We had an opportunity to add to the goal tally today.”
MORE: Man United latest news
MORE: Arsenal latest news
MORE: Chelsea latest news
MORE: Liverpool latest news