You need to determine if it is an R8 or MT spindle Looks like it might be MT Morse Taper
Can you get a better picture of the spindle bore something doesn't look right.
First we have to answer the questions above. If the R8 collets or MT end mill holder don't fit properly then the only way is to find out what the spindle taper is actually is to measure it. Bear in mind that there are different sizes of MT tapers. An MT2 tool will not fit an MT3 socket.