The problem is that these holes are tapped radially and then a taper is cut on the OD. Even on a straight surface countersinking a hole will produce an oval countersink. Here countersinking before tapping won't work, as that countersink will disappear when the taper is cut, and countersinking afterwards would have to be perpendicular to the taper, not inline with the tapped hole.
I don't think there's really much else to do than touch them up with a burr or small stone. From a design perspective, if these are designed to hold something (presumably these are some kind of tapered threaded extension?) then it'll be tough finding set screws short enough to not protrude above the surface of the taper and still have some means of tightening them. If having the set screws protrude above the surface then it would make more sense to have a steeper taper after the last threaded hole.