D shaped broaches are commercially available, so it can be done..... Just like Clickspring's square broach. The pilot hole would have to be tangent to and centered on the flat, and as big as you could make it. You'd probably have to alternately interrupt the cut so it you weren't trying to cut the whole circular part in one giant chip. All the cutting forces are going to balance each other, EXCEPT the part of the broach directly across from the flat. That will force it to follow (without cutting) the point on your pilot hole which is tangent to the the desired D shape.
I'm not sure how difficult it might become to do at 8mm, (probably not worth it in your case or mine), but how much does presentation takes presidence over function.... I made a pulley for a half inch D shaft motor, I made a round hole and keyed it, and put set screws into the key slot. I just overcut the key slot until the key would just barely go into the key slot in the pulley when it was aligned over the flat. Then used the set screws to drive that key hard against the flat. My thinking is that I won't bugger the motor shaft that way. And the fact that a D shaft is not ever meant to be a high torque connection (at least not just by it's self), so set screws on the flat are the most typical retention method, so you can figure back from there that the retention forces required just aren't that high.