I've never worked with a colchester , so no expert by any means.
It sounds like If the wheel itself is moving to and fro on the shaft, maybe you are missing a washer or spacer from behind the hand wheel.
Thus allowing it to jiggle around some.
Or, Could be some wear on the shaft end causing play. Or the lead screw might be simply mounted to far outboard, leaving a gap at the end where the hand wheel mounts ( This occurred on my standard modern lathe one time ). If you loosen up the lead screw and wheel enough you maybe can inspect the area behind the wheel to see if it looks worn. a spacer might take up the play.
Or perhaps adjusting the lead screw position so the hand wheel seats more snugly against the saddle/ casting.
Glenn