Very likely true. Not many "need" the speed control of the VFD. The primary reason I believe most hobbyists use them is to deal with the single phase vs three phase problem. The variable is merely a bonus. I've thought about one for my bp clone, it's a step pulley, but as long as I've used it like it is, I wouldn't know how to act with a VFD. Plus I have three phase in the shop. Maybe someday I'll find a machine where I'll see enough advantage to get one.
People building CNC mills who intend to use a rigid tapping cycle would need one if their control didn't include a spindle encoder and accurate speed control. I've seen it done with the Reeves type drive, and it's a little iffy dialing in the speed. And you will break a few taps if you don't have the speed close. Even with an accurate VFD, without an encoder, you're limited to one shot tapping.