I'd cut the die on the mill first. Rotary table work unless you have CNC. Use a drill bit on location to rough the center hole area, move to the other locations with the RT and hand wheels and drill those. Then end mill work to the profile.
For the punch, I'd scribe the lines using the die as a pattern. I have a die filer which I'd use to get to the scribed line after rough milling close.
I'd also drill a hole in the punch for the screw hole and put an alignment pin in place.
In use, hand punch the screw hole and slip it over the pin in the punch. The punch out the felt.
Bruce