The Red Devils suffered a blow in the first half after Juan Mata was sent off for the first time in his career for a second bookable offence in 26th minute.
Tony Pulis’s men took 66 minutes to finally make the breakthrough against 10-man United thanks to Salomon Rondon’s winner.
Former Blackburn Rovers and Leicester City midfielder Savage had his say on West Brom’s slender win following United’s first defeat in five outings.
“It’s always such a big moment when you beat Manchester United, no matter what place they are in the league,” Savage told BBC Sport.
“When you are sitting in that changing room after the game its such a great feeling. For these players out there who haven’t beaten them before, they are really close now this is a huge final few minutes.”
United are in fifth place and three points behind Manchester City, who occupy the final Champions League spot.
The Red Devils had beaten Shrewsbury Town, FC Midtjylland, Arsenal and Watford in their four games ahead of Sunday’s trip to The Hawthorns.
Van Gaal’s side make the trip to Liverpool in the first leg of their Europa League last-16 clash On Thursday night.
MORE: Man United latest news
MORE: Arsenal latest news
MORE: Chelsea latest news
MORE: Liverpool latest news