Centre-half Sébastien Bassong gave the home side an early lead before a smart finish from Anthony Pilkington put the Canaries in charge at Carrow Road.
However, Craig Gardner gave the Black Cats hope of a comeback just before the interval, with Matthew Kilgallon guilty of wasting the best chance to rescue a point for the visitors in the second half.
“We’ve conceded some very, very poor goals,” O’Neill told Sky Sports. “It gave us an uphill task.
“But from the moment we conceded the second goal, we played very strongly, and in the second half in particular, we were really excellent and created some decent chances – we should have scored two or three.
“But we didn’t manage any [goals] in the second half. If we show the same spirit as we did in the second half for the rest of the season, we’ll be fine.
“But if we show anything like we did in the first half, it could be a tough battle.”
O’Neill was forced to take off Sunderland’s top goal scorer Steven Fletcher at half-time and the Black Cats boss is unsure about the severity of the injury.
“I’m not sure. At this moment, he is sore and the doctor are with him. I won’t know properly until Monday or Tuesday,” he added.
MORE: The latest football news