The visitors had twice gone ahead through Ian Sharps and Will Hoskins but each time Orient hit back as Adam Chambers and Justin Miller headed levellers.
But as the home side pressed for a winner they were caught by Wisemen's sucker punch which settled this five-goal thriller.
After a slow start it was Orient who went close to breaking the deadlock as Matthew Lockwood and Gary Alexander sent efforts just wide.
But it was the visitors who grabbed the lead when Orient failed to clear a corner and when Michael Keane's snap-shot from the edge of the box cannoned off Glyn Garner, skipper Sharps reacted first to head home from close range.
But orient hit back and levelled 60 seconds later when Wayne Corden's right-wing cross was headed home by Chambers.
Orient striker Alexander blazed over from six yards and as the game swung from end to end Garner was forced to palm away a close-range header from Hoskins.
Miller went close for the home side but it was Rotherham who struck a deadly blow in first-half stoppage time when Clayton Fortune and Lockwood failed to deal with Neil Cutler's massive clearance and Hoskins nipped in to lob over the advancing Garner.
After the interval Orient almost levelled when Cordon flashed an effort against the woodwork, then Miller had an effort saved by Cutler before Alexander headed over.
So it came as no surprise when Orient finally levelled. Poor marking allowed defender Miller to head home at the far post following Lockwood's corner from the left.
As Orient went hunting for the winner, substitute Lee Steele almost netted with his first touch and then Lockwood fired a chance just wide.
But eight minutes from time substitute Wiseman smacked home a winner that took a wicked deflection off Lockwood and ended up in the net via an upright.
Orient almost levelled in stoppage time but Cutler turned Steele's close-range effort on to the bar.