The Reds headed into the game looking to build on their impressive start to the new season and fresh from their 2-1 comeback win over Tottenham Hotspur last weekend.
However, the Reds fell behind in the 21st minute when Trezeguet finished from close range to put the hosts ahead.
Liverpool FC controversially had a goal disallowed by VAR in the 27th minute for offside after Roberto Firmino netted.
But the Merseyside outfit were able to find an equaliser in the 87th minute when Andy Robertson headed home.
And Sadio Mane netted an incredible winner for the visitors in the fourth minute of stoppage time to keep Liverpool FC six points clear at the top of the Premier League table.
Tweeting after Robertson’s equaliser, Lineker wrote: “Bloody hell! Robertson has equalised for @LFC.”
And after Mane’s winner, the England legend added in a separate tweet: “What a day. @LFC win it at the very death. Mane with the goal. You might want to watch @BBCMOTD tonight. Tasty. Very tasty.”
With Manchester City also coming from behind to beat Southampton 2-1, Liverpool FC’s win left them six points clear at the top of the table after 11 games.
Next up for the Reds in the Premier League is a crunch clash against champions Manchester City at Anfield next weekend as they look to extend their lead at the top of the table.
Before that is a home clash against Genk in the Champions League on Tuesday night.
MORE: Man United latest news
MORE: Arsenal latest news
MORE: Chelsea latest news
MORE: Liverpool latest news