Yes a metric set works for imperial sizes. Metric collets are numbered for a range in millimeters something like 5-4, 6-5, 7-6, etc. For example, to hold a 3/8ths round piece it would take a 10-9 metric collet.
A 23 collet set should cover any Imperial size up to and including an inch but please calculate it out for yourself to be sure.