Rushden started well and on four minutes Drewe Broughton was just over with a header from a Darren Caskey free-kick. A minute later Glen Morris made a fine save to deny Tyrone Berry.
However Rushden took the lead in the seventh minute when Wayne Hatswell headed back Caskey's cross for Broughton to nod home.
On 13 minutes Berry was clear on the right but the winger shot weakly and Morris was able to make a comfortable save.
Towards the middle of the half play became scrappy with neither side making clear cut chances.
However on 39 minutes Broughton pulled a ball back from the by-line but it just avoided the on-coming Simeon Jackson.
On the stroke of half time Caskey saw his cross elude all defenders and the ball scraped the post. Seconds later Ronnie Bull was just over for the home side.
Orient looked more positive in the second half after making changes at the break bringing on strikers Gary Alexander and Lee Steele. And on 54 minutes Orient had the ball in the net through sub Steele but he was ruled offside.
Orient keeper Morris had to head to safety on 64 minutes but Broughton was unable to put the ball into an empty goal firing inches wide.
On 70 minutes Orient went close when full-back Justin Miller was inches wide with a fierce shot from the edge of the box.
Orient battled hard to level and Paul Connor should have equalised on 82 but he blazed over with only Scott Tynan to beat.
Rushden lost defender Phillip Gulliver on 84 minutes and he had to be replaced by player-manager Barry Hunter, and with his experience Rushden held on to win.