It takes a special grade of carbide, very rigid tooling and a lot of RPM's to turn hardened material. We used to have special machines for it called "hard turn lathes". Your skinny bar hanging out isn't the optimal setup. Your cutter is cutting in "reverse" because it's sharper in the back than in the front and the harmonics are better due to dragging a tail rather than pushing it.
If you ever have to do it again make a boring bar 3/4 as big around as the hole (for rigidity) and no longer than it needs to be to meet the depth requirement. Put a stub of a carbide lathe bit sideways in the end of the bar. Either crank the speed up to the max of your machine and take light cuts, or slow the RPM's way down and take heavier ones.