The Bulls had looked in complete control after early goals from Febian Brandy and Steve Guinan.
But Leyton Orient skipper Tamika Mkandawire's goal just before the interval inspired his side to a strong show after the break.
Hereford got off to a great start when Matt Done's pass found Brandy in the area after just four minutes. The Manchester United youngster turned neatly before firing a low shot into the bottom corner of the net.
Brandy earned Hereford a penalty shortly afterwards when he went down under a challenge from Alton Thelwell.
Guinan drilled his penalty past goalkeeper Jamie Jones with a minimum of fuss for his 11th goal of the season.
Hereford should have been three or four ahead at the interval, with Brandy guilty of missing two great chances, one with only keeper Jones to beat.
But Orient gave themselves hope when former Bull Mkandawire slid the ball home from a Charlie Daniels free-kick a minute before the break.
Hereford began the second period in some style with Guinan having a shot saved and Brandy flashing a shot just wide.
But, as the half progressed, the visitors grew stronger and forced Hereford back for long periods.
Only an excellent block from Richard Jackson prevented Scott McLeish from levelling the scores in the 66th minute.
The former Wycombe striker was out of luck again moments later when Hereford keeper Peter Gulacsi pulled off an outstanding point-blank save to turn the ball around an upright.
Substitute Ryan Jarvis headed agonisingly wide in injury time, as Orient faced the long journey back to East London without a point for the first time in Geraint Williams' short reign.