Just remember to take it slow. Multiple passes, even for brass. Aluminum you can get away with a bit more, but not much. It won't take much punishment, and if you push it, it will let you know (probably by breaking), so keep the speeds up, the feeds low, and the depth of cut to just a whisker.
But it sure beats milling on a lathe!