Rickie Lambert headed the Saints in front after 16 minutes, but Van Persie restored parity with a clinical left-footed strike.
The home side took the lead again when Morgan Schneiderlin nodded past Anders Lindegaard in the second half, but Jos Hooiveld’s reckless challenge on Van Persie resulted in a spot-kick, and an opportunity for the former Gunners captain to level.
However, the Netherlands international’s speculative chipped attempt was palmed wide by Saints keeper Kelvin Davis.
With United edging towards a second league defeat in three games, Van Persie netted a late brace to complete his hat-trick and the Red Devils’ comeback, much to the striker’s relief.
“I’m so relieved,” said Van Persie.
“When I missed that penalty, I was so gutted. I try to reach a certain standard in my game, and that’s not part of my idea of that standard.
“I want to play football well, respect it, and something went wrong there. I wanted to hit it hard and in the corner like I always do.
“But something went wrong. It’s just me to blame.”
The 29-year-old also paid credit to second-half substitute Paul Scholes, who helped mastermind the visitors’ comeback and ensure the Red Devils kept pace with title rivals Manchester City.
“Definitely, Scholes is unbelievable,” added Van Persie. “He’s a world-class player.
“We lost grip of the game, even when we were 2-1 down, we weren’t really chasing the game.
“When he came on, every pass was spot on: 30 or 40 metres and short passes. With him in the game, as a striker, you’re always on your toes because anything can happen at any certain time.
“When he came on he changed the game, he’s my man of the match today.”
MORE: Man United latest news
MORE: Arsenal latest news
MORE: Chelsea latest news
MORE: Liverpool latest news