I'm guessing you are using a self centering chuck that is not centering perfectly. If so that is normal since they are not perfect, however some are much better than others. I you want it to line up you will have to use a four jaw chuck and dial gauge. Have fun!
I agree. Chances are the repeatability (or lack of it) of your scroll chuck is what you are seeing. A simple test would be to turn one end, remove it from the chuck then put it back in the chuck the same way without flipping it. Take another cut and see how concentric it is.
Standard procedure using a 3 jaw scroll chuck is to do all the cutting before removing the work. Once you remove the work don't count on the work being held the same a second time.