The O's recorded their third straight league and cup win when the summer signing from Yeovil broke the deadlock following Wayne Corden's excellent left-wing cross.
The win might have been a little more comfortable had Adam Boyd not seen his first-half spot-kick saved by Clayton Ince.
The result left newly-promoted Walsall still searching for their first victory, but had they not frittered away three glorious second-half chances they would have returned home with at least a point.
Chances were few and far between in a disjointed first half and veteran striker Tommy Mooney shot wide for the visitors after just five minutes.
Scott Dann then went close with a header before Orient finally clicked into gear.
The home side gradually spluttered into life and Boyd had a shot saved from distance by the impressive Ince.
The visiting keeper then saved at the feet of Sean Thornton, but it was his penalty save that stole the show on 36 minutes.
Orient midfielder Adam Chambers sliced through the Walsall rearguard only to be fouled by Dann who was booked.
But Boyd's spot-kick to the keeper's right was pushed away and he reacted first to keep out Cordon's follow-up attempt.
After the break Mooney missed a glorious chance to open the scoring and it proved to be a costly error.
Shortly afterwards Gray had the home supporters on their feet when he headed home to break the deadlock.
Orient should have extended their advantage as Thornton and Gray both went close to adding to the home side's tally.
But Walsall came storming back strongly in the closing stages and Paulo Carneiro blasted wastefully wide 15 minutes from time.
The win was the perfect present for Orient manager Martin Ling, who was celebrating his 200th game in charge.