Meanwhile Orient's slide down the table continued despite a late flurry and they have now won just one of their last seven games.
Town quickly stamped their authority on the match and could not have wished for a better start. The game was just 17 minutes old when the dangerous Steve Guinan beat Gaby Zakauni and Justin Miller down the left. His cross into the area found Finnigan who rolled his effort into the net from 14 yards past Orient keeper Glen Morris.
Two minutes later Town stumped Orient with a second as defender Shane Duff powerfully headed home Grant McCann's cleverly flighted free-kick to leave Morris stranded.
Kayode Odejayi almost headed a third before the break as Cheltenham finished firmly in control.
Orient manager Martin Ling made a double change at half-time as he threw Tom Youngs and Derek Duncan into the fray and the move looked to have paid dividends just five minutes after the restart.
Town paid the price for failing to clear a corner and Duncan whipped the ball back into the box for Gary Alexander to head home his first league goal since October from close range.
Orients revival lasted just three minutes as Finnigan ghosted past a static Orient defence to beat Morris from 16 yards for the Robins' third.
After that the game became a bad-tempered affair with a rash of bookings before Orient threw themselves a late lifeline.
The hard-working Michael Simpson fired in a shot that was drifting wide but it was diverted into the path of Orient's central defender John Mackie, who blasted home his second goal of the season.
Orient, desperate not to slip to their fifth home defeat of the campaign, threw bodies forward in bid to snatch late leveller and almost succeeded, but Lee Steele's shot was blocked.