After replacing David Skrela after just three minutes James produced an imperious display with the boot, kicking 17 points to send the French side into the last four.
Wales full-back Lee Byrne crossed for the only try of the game for Clermont, a score made by the power and guile of Rougerie.
England’s Six Nations kicking hero Owen Farrell kicked Sarries’ only points in a game dominated by James’ boot, and some stoic defending from the French outfit.
Elsewhere Ruan Pienaar was the star of the show as Ulster upset their RaboDirect PRO12 rivals Munster 22-16 to book themselves a spot in the Heineken Cup’s last four and a clash with Edinburgh.
The South African World Cup-winning scrum half kicked 14 impressive points as Ulster survived a fightback from their Irish rivals to take the win.
A try from Simon Zebo and 11 points from the boot of Ronan O’Gara seemingly set up a grandstand finish for Munster in the second half.
But stirling work from the Ulster defence – led by man-of-the-match Stephen Ferris – ultimately saw them home.
MORE: The latest football news
MORE: The latest tennis news
BIOGRAPHY: Adam Lallana
BIOGRAPHY: Nacho Monreal