The Merseyside outfit took the lead when Glen Johnson’s impressive long-range strike left West Ham goalkeeper Jussi Jasskerlainan stranded.
But Rodgers men managed to secure three points and move into the top half of the Premier League table for the first time this term courtesy of Joe Cole’s close-range finish and Collins’ error in the 79th minute.
“It was a brilliant win for us. West Ham have been terrific this year. Sam has done a great job. I thought our team were outstanding,” Rodgers told BBC Sport.
“We were unfortunate to concede the penalty, I don’t know what Joe Allen can do to avoid that. Maybe the roar of the crowd made the referee change his mind. It was a very harsh decision for us.
“We’ve threatened to score three, four and five goals through most of this season, but through not being ruthless or having bad luck we haven’t scored as many as we would have liked.”
888Sport are offering new customers 21/2 on West Ham to beat Liverpool and Valencia to score any time on your first bet. T&Cs apply. Click here to sign up now.
MORE: The latest football news