I store mine on the head of the post of my mill. I have several bases that are handy for lots of things, like holding welding and clamping jigs or on the shop press.
It might be worth noting, if you haven't noticed, that strong magnets don't stick to sheet metal as strongly as chunks of iron with more mass. So, stuck to the side of sheet metal toolbox or even the shroud of the lathe head, they can be knocked off inadvertently. Or, if stuck to sheet metal when using the indicators, they can scoot.