Whaley fired home from 18 yards past former Bury goalkeeper Glyn Garner nine minutes into the second half after Orient defender Brian Saah was caught in possession by Jake Speight, who delivered a telling cross.
That decisive strike sent Orient crashing to a second home defeat in a week as they dropped out of the top three in League Two.
The home side struggled to find their form and it was Bury, inspired by David Flitcroft, who carved out the early chances.
Whaley tested Garner nine minutes, but the former Bury keeper dived to his right and tipped the ball away for a corner.
Flitcroft also tried his luck from distance but, again, Garner pushed his shot around the post.
It was Orient who created the best chance of the half only for striker Gary Alexander, without a goal in 11 games, to fire wastefully wide from eight yards.
As Orient stepped up the pace Craig Easton headed just over the bar as Bury had to dig deep to survive until the break.
In the second half Easton was denied by Bury keeper Neil Edwards who tipped his shot from 20 yards over the bar.
But the visitors caught Orient on the break a few minutes later and once Whaley's decisive strike put his side in front Orient simply fell apart.
Shortly after Bury boss Chris Casper was sent to the stands by referee Andy Woolmer, but it was the visitors who kept their cool to hang on for the points.
Orient manager Martin Ling pulled off top scorer Alexander and Shane Tudor in a bid to spark his side, but it was the visitors who looked dangerous on the break.
As Orient looked a pale shadow of the side who had topped the table earlier in the season, Bury duo Danny Reet and substitute Nicky Adams could easily have stretched their side's advantage late in the game.