[X86][MC] Reject out-of-range segment and debug registers encoded with APX

APX specification states that the high bits found in REX2 used to encode GPRs can also be used to encode segment and debug registers, although all of them will #UD. Therefore, when disassembling we reject attempts to create segment or debug registers with a value of 16 or more.

Author: anematode

llvm/lib/Target/X86/Disassembler/X86Disassembler.cpp

@@ -819,8 +819,12 @@ static int readModRM(struct InternalInstruction *insn) {
         *valid = 0;                                                            \
       return prefix##_ES + (index & 7);                                        \
     case TYPE_DEBUGREG:                                                        \
+      if (index > 15)                                                          \
+        *valid = 0;                                                            \
       return prefix##_DR0 + index;                                             \
     case TYPE_CONTROLREG:                                                      \
+      if (index > 15)                                                          \
+        *valid = 0;                                                            \
       return prefix##_CR0 + index;                                             \
     case TYPE_MVSIBX:                                                          \
       return prefix##_XMM0 + index;                                            \

llvm/test/MC/Disassembler/X86/x86-64-err.txt

@@ -5,6 +5,10 @@
 # 32: into
 0xce
 
+# 64: invalid instruction encoding
+0xd5,0xc5,0x20,0xef
+# 64: invalid instruction encoding
+0xd5,0xc5,0x21,0xef
 # 64: invalid instruction encoding
 0xc4,0x62,0xf9,0x18,0x20
 # 64: invalid instruction encoding