Document mitigation for Intel MDS security flaws

In May 2019, four new microprocessor security flaws, known as "MDS"
(Microarchitectural Data Sampling) have been discovered.  These flaws
affect unpatched Nova Compute nodes and instances running on Intel
x86_64 CPUs.  The said security flaws are also referred to as "RIDL"
(Rogue In-Flight Data Load) and "Fallout".

Refer to the following pages for further details:

 - https://access.redhat.com/security/vulnerabilities/mds
 - https://mdsattacks.com/
 - https://zombieloadattack.com/

            * * *

If we're adding the guide for "MDS" flaws, then it begs the
question: "What about mitigation guides for previous vulnerabilities?"

Two points:

(a) Write the mitigation document for rest of the previous
    vulnerabilities too, for completeness' sake. (In April 2018 I wrote
    this doc[1] for Meltdown — polish it and submit it. Parts of that
    document's content is already incorporated into the help text for
    the config attribute `cpu_model_extra_flags`.)

(b) For now, we can live with the cliché, "something is better than
    nothing"; we'll add the other docs "when we get to it".  Meanwhile,
    operators get mitigation details from various other places —
    processor vendors, Linux distributions, etc.

[1] https://kashyapc.fedorapeople.org/Reducing-OpenStack-Guest-Perf-Impact-from-Meltdown.txt

Change-Id: I1bb472c3438cc9a91945999d2350b2c59fa6a1f3
Signed-off-by: Kashyap Chamarthy <[email protected]>

Author: kashyapc

doc/source/admin/index.rst

@@ -53,3 +53,4 @@ operating system, and exposes functionality over a web-based API.
    support-compute.rst
    system-admin.rst
    secure-live-migration-with-qemu-native-tls.rst
+   mitigation-for-Intel-MDS-security-flaws.rst

doc/source/admin/mitigation-for-Intel-MDS-security-flaws.rst

@@ -0,0 +1,127 @@
+======================================================================
+Mitigation for MDS ("Microarchitectural Data Sampling") Security Flaws
+======================================================================
+
+Issue
+~~~~~
+
+In May 2019, four new microprocessor flaws, known as `MDS
+<https://access.redhat.com/security/vulnerabilities/mds>`_ , have been
+discovered.  These flaws affect unpatched Nova compute nodes and
+instances running on Intel x86_64 CPUs.  (The said MDS security flaws
+are also referred to as `RIDL and Fallout <https://mdsattacks.com/>`_ or
+`ZombieLoad <https://zombieloadattack.com>`_).
+
+
+Resolution
+~~~~~~~~~~
+
+To get mitigation for the said MDS security flaws, a new CPU flag,
+`md-clear`, needs to be exposed to the Nova instances.  It can be done
+as follows.
+
+(1) Update the following components to the versions from your Linux
+    distribution that have fixes for the MDS flaws, on all compute nodes
+    with Intel x86_64 CPUs:
+
+    - microcode_ctl
+    - kernel
+    - qemu-system-x86
+    - libvirt
+
+(2) When using the libvirt driver, ensure that the CPU flag ``md-clear``
+    is exposed to the Nova instances.  It can be done so in one of the
+    three following ways, given that Nova supports three distinct CPU
+    modes:
+
+    a. ``[libvirt]/cpu_mode = host-model``
+
+       When using ``host-model`` CPU mode, the ``md-clear`` CPU flag
+       will be passed through to the Nova guests automatically.
+
+       This mode is the default, when ``virt_type=kvm|qemu`` is
+       set in ``/etc/nova/nova-cpu.conf`` on compute nodes.
+
+    b. ``[libvirt]/cpu_mode = host-passthrough``
+
+       When using ``host-passthrough`` CPU mode, the ``md-clear`` CPU
+       flag will be passed through to the Nova guests automatically.
+
+    c. A specific custom CPU model — this can be enabled using the
+       Nova config attributes: ``[libvirt]/cpu_mode = custom`` plus a
+       particular named CPU model, e.g. ``[libvirt]/cpu_model =
+       IvyBridge``
+
+       (The list of all valid named CPU models that are supported by
+       your host, QEMU, and libvirt can be found by running the
+       command ``virsh domcapabilities``.)
+
+       When using a custom CPU mode, you must *explicitly* enable the
+       CPU flag ``md-clear`` to the Nova instances, in addition to the
+       flags required for previous vulnerabilities, using the
+       ``cpu_model_extra_flags``.  E.g.::
+
+           [libvirt]
+           cpu_mode = custom
+           cpu_model = IvyBridge
+           cpu_model_extra_flags = spec-ctrl,ssbd,md-clear
+
+(3) Reboot the compute node for the fixes to take effect.  (To minimize
+    workload downtime, you may wish to live migrate all guests to
+    another compute node first.)
+
+Once the above steps have been taken on every vulnerable compute
+node in the deployment, each running guest in the cluster must be
+fully powered down, and cold-booted (i.e. an explicit stop followed
+by a start), in order to activate the new CPU model.  This can be done
+by the guest administrators at a time of their choosing.
+
+
+Validate that the fixes are in effect
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+After applying relevant updates, administrators can check the kernel's
+"sysfs" interface to see what mitigation is in place, by running the
+following command (on the host)::
+
+    # cat /sys/devices/system/cpu/vulnerabilities/mds
+    Mitigation: Clear CPU buffers; SMT vulnerable
+
+To unpack the message "Mitigation: Clear CPU buffers; SMT vulnerable":
+
+  - The ``Mitigation: Clear CPU buffers`` bit means, you have the "CPU
+    buffer clearing" mitigation enabled (which is mechanism to invoke a
+    flush of various exploitable CPU buffers by invoking a CPU
+    instruction called "VERW").
+
+  - The ``SMT vulnerable`` bit means, depending on your workload, you may
+    still be vulnerable to SMT-related problems.  You need to evaluate
+    whether your workloads need SMT (also called "Hyper-Threading") to
+    be disabled or not.  Refer to the guidance from your Linux
+    distribution and processor vendor.
+
+To see the other possible values for the sysfs file,
+``/sys/devices/system/cpu/vulnerabilities/mds``, refer to the `MDS
+system information
+<https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html#mds-system-information>`_
+section in Linux kernel's documentation for MDS.
+
+On the host, validate that KVM is capable of exposing the ``md-clear``
+flag to guests::
+
+    # virsh domcapabilities kvm | grep md-clear
+        <feature policy='require' name='md-clear'/>
+
+Also, refer to the 'Diagnosis' tab in this security notice document
+`here <https://access.redhat.com/security/vulnerabilities/mds>`_
+
+
+Performance Impact
+~~~~~~~~~~~~~~~~~~
+
+Refer to this section titled "Performance Impact and Disabling MDS" from
+the security notice document `here
+<https://access.redhat.com/security/vulnerabilities/mds>`_, under the
+'Resolve' tab.  (Note that although the article referred to is from Red
+Hat, the findings and recommendations about performance impact apply
+for other distributions as well.)

doc/source/admin/security.rst

@@ -48,3 +48,13 @@ streams of a nova instance—i.e. guest RAM, device state, and disks (via
 NBD) when using non-shared storage.  For further details on how to set
 this up, refer to the
 :doc:`secure-live-migration-with-qemu-native-tls` document.
+
+
+Mitigation for MDS (Microarchitectural Data Sampling) security flaws
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+It is strongly recommended to patch all compute nodes and nova instances
+against the processor-related security flaws, such as MDS (and other
+previous vulnerabilities).  For details on applying mitigation for the
+MDS flaws, refer to the :doc:`mitigation-for-Intel-MDS-security-flaws`
+document.