The CORRECT way to face is to start from the center(though I usually start from the outside!) The advice to start from the center is good,as it will eliminate the problem.
The reason I usually start from the outside is that I can quickly determine the depth of cut needed to clean off,say,a bandsawed piece of large dia. metal. You could do the same by putting your cutter deep enough to clean off the metal by trying it against the outer edge. Note where your compound dial is set so it will clean off all the bandsaw cuts. Then,move to the center of the bar,and move your compound into the rotating metal by that same amount. Then,proceed to face the metal off. Actually,I never face with just 1 cut. I take 2,just in case the uneven depth of cut as the material revolves deflects the tool in and out at all on that initial cut.
I hope this made sense.