you do realize that most of your cutting is going to be about 1000rpm ?? there abouts. Except if you are cutting with very small diameter tools, then you would need the high rpms. I rarely go to my 3250rpm.. I also added a VFD recently. My lowest is not really where I want it, and neither is my high. But I have different belt combos, so that resolves most issues. I can dial what I need mostly.
Consider a stepped pulley for your needs too. Both spindle and motor. It will help when you need it. But if you need to be at 5000rpm go for it. I don't think you need it. As far as the VFD, you are undersized if you are lower than your motor's rating. I went with a bigger VFD than the motor label, the manufacturer recommended I up it to the next level.