Ramires and Didier Drogba struck either side of half-time to hand Chelsea the advantage after Kenny Dalglish’s side made a lacklustre start to the Wembley showpiece.
Andy Carroll’s introduction mid-way through the second half transformed Liverpool and the striker halved the Blues’ lead before coming close to a dramatic equaliser.
“It is so difficult to talk about now,” Agger told Liverpool’s website.
“I really don’t know why we played as we did for the first 60 minutes. But we didn’t turn up and you cannot afford to do that in an FA Cup final.
“We know these opportunities do not come along that often and that is why you have to take them when they do.
“This will haunt us. We didn’t show up for the first hour and that is not good enough in an FA Cup final.
“When Andy came on he did well and we played in a different way.
“I don’t know if it was over the line. I was at the other end of the pitch and I haven’t seen a replay but that wasn’t the reason why we lost.
“The reason why we lost was that we didn’t play well in the first hour. We didn’t lose because Chelsea played well but I think because we started the game really badly. We can only blame ourselves.
“There’s not much more to say right now. We are all just so disappointed.”
The 27-year-old, who was part of the Liverpool side which won the Carling Cup back in February, insisted the Reds are keen to win their final two games to end the season on a positive note.
“That is how we will start to move forward. It is difficult right now to think about anything else,” added Agger.
“Can we challenge the top teams next season? Yes, of course. We have moved forward a lot this season but right now the only feelings are of disappointment.
“When you get into an FA Cup final, you really want to win it. Nobody ever remembers the runners-up. Nobody likes to be a loser.”
MORE: Man United latest news
MORE: Arsenal latest news
MORE: Chelsea latest news
MORE: Liverpool latest news