The former Reading man - a club-record £5.5m signing from the Madejski Stadium last summer - put a nightmare 13 months behind him to net a stunning 25-yard effort to open his Potters account.
Orient should have put the Premier League side to bed in normal time with Adrian Patulea and Andros Townsend missing gilt-edged chances.
But Kitson broke the deadlock five minutes into extra-time to spare Stoke's blushes.
Potters boss Tony Pulis made wholesale changes to the side which draw 0-0 with Birmingham - but it didn't seem to affect the Potters early on as Kitson twice chanced his arm.
At the other end, Townsend - on loan from Premier League high-fliers Tottenham - was starting to find his range and stung the palms of keeper Steve Simonsen with a rasping drive.
And it was the League One side who had the last word in the first half as Charlie Daniels' free-kick bounced agonisingly wide of the mark before Jason Demetriou forced a smart stop from Simonsen.
Patulea should have broken the deadlock for the O's three minutes into the second-half when the Stoke defence went missing at the back post, but he could only head wide.
But Kitson was a constant menace for the Orient defence and was unlucky not to notch his first goal in the 84th minute - his header cleared off the line by John Melligan.
Extra-time always looked likely but when it did arrive, it produced the best moment of the match. Kitson lashed at a loose ball in the 95rd minute which flew into the top corner of the net with keeper Glenn Morris stranded.