Orient, giantkillers themselves last season, were given a taste of their own medicine by Torquay, who are currently 91st in the Football League.
Robertson, on loan from Sheffield United, struck after 52 minutes and, although Orient's James Walker levelled shortly afterwards, Robertson grabbed the winner nine minutes from time.
In an instantly forgettable first half, Orient failed to impose themselves on their League Two visitors and genuine chances were few and far between.
Orient's Gary Alexander had an early header saved by Nathan Abbey, who was on loan Orient last season.
Alexander then saw a shot deflected over the bar midway through a miserable first half.
Little was seen of Torquay as an attacking force, but they created the best two chances of the opening 45 minutes.
Just past the half-hour mark Chris McPhee raced clean through as Orient hesitated waiting for a flag that never came.
But the Torquay player failed to find a finish and saw his shot pushed away by Orient keeper Glyn Garner.
The visitors were forced into an early substitution when Carl Motteram replaced Kevin Hill and he made a quick impression when he crossed from the left and Robertson fired an effort against the bar.
Orient responded and Alexander's smart snap-shot from 16 yards was well saved by Abbey just before half-time.
After the break it was the visitors who grabbed the shock lead when Alexander's poor defensive header fell to Robertson, who raced clear and coolly lobbed over Garner.
The reverse stung Orient into action and, after Justin Miller had a shot pushed away by Abbey, the O's levelled on 58 minutes.
Miller's neat ball into Walker saw the on loan striker swivel and curl the ball into the corner of the net.
But Torquay refused to buckle and on the hour the livewire Robertson went close again.
The game had sprung into life and Alexander had a shot saved by Abbey as Orient went looking for the winner.
Miller fired a good chance wide for the home side before the visitors struck the all-important killer blow nine minutes from time.
As Orient's defender Alton Thelwell pulled up injured, Robertson had a clear run at goal and made no mistake as he swept the ball past Garner to settle the tie.
The shock success earned the League Two side a home tie with Southampton in the next round.