In general, ER collets have a larger clamping range, approx 1mm, in comparison to many other collet systems, which I think is why many prefer them. There are other benefits as well, such as having a through hole so that you can work on stock which is significantly longer than the collet - you can't do that in an R8 collet.
As for the various ER sizes, it depends on the general size of work you do which would be preferable; I prefer ER40, as the largest collet size is close to the hole through my lathe's spindle. If I chose a smaller size I would limit myself on max diameter and chosing larger I'm not really gaining much....for what I do.
I also have r8 on the mill and I use them quite a bit, but due to needing a drawbar, no through hole, etc, I don't see them as especially useful elsewhere. I don't tend to use my ER collets on the mill mostly because an ER chuck takes up height, I just go with the R8 there. For any given range, I need more R8 collets to hold the same range of sizes, but *for me*, I tend to use only a very limited number of R8 collets, though I use them very often.
IMO, each has advantages and disadvantages and serve differing functions in my shop. I don't often do much workholding in the R8 collets, instead it's toll holding which doesn't need the through hole. I often hold long stock in my lathe chuck, so the length limitation of R8 would be a problem. My collet blocks are ER40 to take advantage of the through hole and to take advantage of the collets I already have.