The Team Sky rider headed into Wednesday’s stage from Ypres to Arenberg already nursing a damaged wrist and grazing to his left hip following a fall during racing on Tuesday, although doctors cleared him to carry on.
But with heavy rain making conditions difficult for all riders, the 29-year-old fell 30km into the stage and while getting back on the saddle, he went down again with 69km to go.
Clearly in pain, he consulted with team doctor Alan Farrell and sports director Nicolas Portal before climbing into the team car to leave the race.
His withdrawal came before the stage had reached the first of several cobbled sections which had been shortened to seven instead of nine due to the adverse weather.
Team Sky principal Brailsford said: “Crashing yesterday and then crashing before the cobbles is obviously very very disappointing and devastating for Chris but that is sport and we keep going.
“Chris was in pain before the stage, there is no denying that. You couldn’t have had a worse moment to hurt your wrist the day before cobbles. It was going to be challenging and it will play on your mind but he was brave and was up for it and he wanted to fight for it.”
Froome’s exit is the second by a Brit in this year’s tour after Mark Cavendish was ruled out injured after a crash on the sprint to the line during the opening stage from Leeds to Harrogate.
The stage five win eventually went to Lars Boom of the Netherlands while Italian Vincenzo Nibali retained the yellow jersey by two seconds.
Meanwhile, Froome’s Team Sky colleague Geraint Thomas is now the highest placed Brit in 14th overall.
MORE: The latest football news
MORE: The latest tennis news
BIOGRAPHY: Ethan Hazard
BIOGRAPHY: Daniel Sturridge