Cheltenham wasted a glorious chance to draw level in the second half when Grant McCann missed a penalty, which just about summed up the visitors' miserable afternoon.
Orient could not have wished for a better start as the game was only six minutes old when a high ball into the area was controlled by Alexander, who swivelled and fired home from six yards as the visiting defence stood off.
It was a dream opening for the basement boys and from then on they put their foot on the gas and almost added a second when Alexander sent Peter Till racing clear, and his right-wing cross was headed wide by Michael Simpson.
But Cheltenham, inspired by skipper John Finnigan, fought their way back into the contest and only a stunning save from home goalkeeper Glyn Garner kept out Shane Duff's close-range header.
As Cheltenham continued to look dangerous, Brian Wilson fired a shot across the face of the Orient goal, but an impressive debut from loan signing Alton Thelwell helped the hosts ride out the storm.
Garner made a smart stop to keep out Damien Spencer's header early in the second half, before what proved to be the game's crucial moment.
Referee Gary Sutton awarded the visitors a 51st-minute spot-kick following a handball by Jabo Ibehre, but Garner flung himself to his left to block McCann's effort.
Orient hit back to take a grip on the game on 67 minutes when Finnigan handled Simpson's cross and Lockwood stepped up to fire home from the spot for his sixth goal of the season.
It was a killer moment for the visitors and after that Orient won the game at the canter.
Home substitute Paul Connor could have stretched the home side's advantage as he went close on two occasions in the last ten minutes.