I have a spring loaded layout punch I mount in the tailstock. Depending where your located, they are dirt cheap, about
5 bucks at places like harbour feight. They are the spring loaded type that you just push the tip on your layout mark, and when it gets to the end of the travel, it clicks, and makes a punch mark. Great for quick punch marks for drilling, or use as a spring loaded guide for your taps.
Then comes those times when the tap you have, doesnt have the drilled dimple for the spring loaded device. This is when you need one of those tap holders that have the ferral to tighten the tap into it, and the other end has the little round stock handle. Most are allready drilled on the centre of the handle end, and the spring loaded device fits in perfect.
Those are allso available dirt cheap to.
Even if the handle end isnt drilled, just chuck it you lathe, and take a centre drill and put your own dimple in it. Many tap sets come with these as well as the standard handle.
You dont need any thing fancy , or expensive, just the spring loaded punch, and a few sizes of the tap handles. The type with the ferral to tighten the tap. Then regardless if the tap wil take the sping loaded point, or not, you have a way to get them started and get it straight.
As a matter of fact, I had to use both items tonight, the 1/4-20 tap had not been drilled to take the point.
I should have taken a pic while it was setup, it would have made it more clear.
One other way you could do it with out any extra tooling in a pinch, is remove the chuck from the tailstock, and put the tailstock barell up against your tap handle. It will help keep things straight, provided your tap handle is flat on that side.