bpo-38686: fix HTTP Digest handling in request.py (#17045)

* fix HTTP Digest handling in request.py

There is a bug triggered when server replies to a request with `WWW-Authenticate: Digest` where `qop="auth,auth-int"` rather than mere `qop="auth"`. Having both `auth` and `auth-int` is legitimate according to the `qop-options` rule in §3.2.1 of [[https://www.ietf.org/rfc/rfc2617.txt|RFC 2617]]:
>      qop-options       = "qop" "=" <"> 1#qop-value <">
>      qop-value         = "auth" | "auth-int" | token
> **qop-options**: [...] If present, it is a quoted string **of one or more** tokens indicating the "quality of protection" values supported by the server.  The value `"auth"` indicates authentication; the value `"auth-int"` indicates authentication with integrity protection

This is description confirmed by the definition of the [//n//]`#`[//m//]//rule// extended-BNF pattern defined in §2.1 of [[https://www.ietf.org/rfc/rfc2616.txt|RFC 2616]] as 'a comma-separated list of //rule// with at least //n// and at most //m// items'.

When this reply is parsed by `get_authorization`, request.py only tests for identity with `'auth'`, failing to recognize it as one of the supported modes the server announced, and claims that `"qop 'auth,auth-int' is not supported"`.

* 📜🤖 Added by blurb_it.

* bpo-38686 review fix: remember why.

* fix trailing space in Lib/urllib/request.py

Co-Authored-By: Brandt Bucher <[email protected]>

Author: 2 people

Lib/urllib/request.py

@@ -1136,15 +1136,17 @@ def get_authorization(self, req, chal):
         A2 = "%s:%s" % (req.get_method(),
                         # XXX selector: what about proxies and full urls
                         req.selector)
-        if qop == 'auth':
+        # NOTE: As per  RFC 2617, when server sends "auth,auth-int", the client could use either `auth`
+        #     or `auth-int` to the response back. we use `auth` to send the response back.
+        if 'auth' in qop.split(','):
             if nonce == self.last_nonce:
                 self.nonce_count += 1
             else:
                 self.nonce_count = 1
                 self.last_nonce = nonce
             ncvalue = '%08x' % self.nonce_count
             cnonce = self.get_cnonce(nonce)
-            noncebit = "%s:%s:%s:%s:%s" % (nonce, ncvalue, cnonce, qop, H(A2))
+            noncebit = "%s:%s:%s:%s:%s" % (nonce, ncvalue, cnonce, 'auth', H(A2))
             respdig = KD(H(A1), noncebit)
         elif qop is None:
             respdig = KD(H(A1), "%s:%s" % (nonce, H(A2)))

Misc/NEWS.d/next/Library/2019-11-06-15-26-15.bpo-38686.HNFBce.rst

@@ -0,0 +1 @@
+Added support for multiple ``qop`` values in :class:`urllib.request.AbstractDigestAuthHandler`.