Hi Rob B,
Welcome to the group!
You don't say how it's being held; so I'll guess in a chuck with no tail-stock centre.
Check the chuck for "bell-mouth" where the jaws are further apart at the tips than the base.
With each pass of your tool bit the work gets pushed back and as it rotates it works itself even further out of the chuck.
Does the work get loose in the chuck and need periodic re-tightening?
See if a tail-stock centre support makes an immediate difference.
Long term, the bell-mouth can be corrected by grinding the jaws, but you need a way to "load" the jaws while doing it.
-brino