It was Connor's third goal since his £40,000 move from Swansea, but it was rough justice on visitors Cheltenham, who deserved at least a point for their efforts.
Cheltenham's Brian Wilson was guilty of missing a hat-trick of chances while Orient keeper Glyn Garner pulled off two stunning saves to ensure the visitors went home empty handed.
Orient Boss Martin Ling dropped leading scorer Gary Alexander to the bench and brought Jabo Ibehre into the starting line up and also handed 18-year-old Jason Demetriou his first full start on the right.
And it was Demetriou who set up Ibehre for the first chance of the match but Town keeper Shane Hicks made a smart stop.
Cheltenham then took control of the game as Wilson lobbed over from 12 yards on 14 minutes. The unfortunate Town player then skied an effort from inside the six-yard box over the bar following an excellent cross from Steve Gillespie.
Orient hit back and it was Connor who broke the deadlock on 23minutes. Demetriou sent in a deep cross from the right and Joe Keith's shot was deflected in by the Orient striker.
That should have been the signal for Orient to impose themselves on the game but it was the visitors who continued to carry the greatest threat.
Promotion-chasing Orient had to thank Garner for preserving their advantage as he made two stunning saves, low to his right to keep out shots from the impressive Grant McCann.
After the interval Orient improved and Ibehre almost stretched their advantage when he burst through but was denied by the alert Higgs.
Cheltenham continued to press and create the cleaner chances, but the luckless Wilson fired wide again before Gillespie toe poked a last minute effort wide of the target.