I keep it on the slowest speed. Drilling with smaller sizes is not a problem but, depending on the material, 3/8" and up can get dicey.
Good luck finding a slo-speed attachment. I've been looking for years. It can be made though.
Someone once told me there were plans for on in a Popular Mechanics (or other type magazine). I've searched but to only come up empty.