bpo-30008: OpenSSL 1.1 compatibility without using deprecated API #3943

markwright · 2017-10-10T12:40:12Z

Note: RAND_pseudo_bytes() is deprecated so RAND_bytes() is used when pseudo is requested.

https://bugs.python.org/issue30008

Note: RAND_pseudo_bytes() is deprecated so RAND_bytes() is used when pseudo is requested.

the-knights-who-say-ni · 2017-10-10T12:40:15Z

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept your contribution by verifying you have signed the PSF contributor agreement (CLA).

Unfortunately our records indicate you have not signed the CLA. For legal reasons we need you to sign this before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

Thanks again to your contribution and we look forward to looking at it!

tiran

Thanks,

you also need to sign the contributor agreement and add a news file with blurb.

tiran · 2017-10-10T12:44:20Z

Modules/_hashopenssl.c

@@ -1012,7 +1012,7 @@ PyInit__hashlib(void)
 {
    PyObject *m, *openssl_md_meth_names;

-#ifndef OPENSSL_VERSION_1_1
+#if (OPENSSL_VERSION_NUMBER < 0x10100000L) || defined(LIBRESSL_VERSION_NUMBER)


Good catch, but this is an unrelated fix and should be tracked in a separate issue.

Thanks, however it was the compiler that told me I needed to make this change
to correct this compiler error that occurs building Python 3.7.0a2 with openssl
1.1.0g built with disable-deprecated:

/var/tmp/portage/dev-lang/python-3.7.0_alpha2/work/Python-3.7.0a2/Modules/_hashopenssl.c: In function 'PyInit__hashlib': /var/tmp/portage/dev-lang/python-3.7.0_alpha2/work/Python-3.7.0a2/Modules/_hashopenssl.c:1017:5: error: implicit declaration of function 'OPENSSL_add_all_algorithms_noconf' [-Werror=implicit-function-declaration] OPENSSL_add_all_algorithms_noconf(); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /var/tmp/portage/dev-lang/python-3.7.0_alpha2/work/Python-3.7.0a2/Modules/_hashopenssl.c:1018:5: error: implicit declaration of function 'ERR_load_crypto_strings'; did you mean 'ERR_load_ERR_strings'? [-Werror=implicit-function-declaration] ERR_load_crypto_strings(); ^~~~~~~~~~~~~~~~~~~~~~~ ERR_load_ERR_strings

tiran · 2017-10-10T12:44:37Z

Modules/_ssl.c

@@ -63,6 +63,7 @@ static PySocketModule_APIObject PySocketModule;
 #include "openssl/err.h"
 #include "openssl/rand.h"
 #include "openssl/bio.h"
+#include "openssl/dh.h"


Why is dh.h required?

To fix thie compiler error that occurs building Python 3.7.0a2 with openssl 1.1.0g built with disable-deprecated:

/var/tmp/portage/dev-lang/python-3.7.0_alpha2/work/Python-3.7.0a2/Modules/_ssl.c: In function '_ssl__SSLContext_load_dh_params': /var/tmp/portage/dev-lang/python-3.7.0_alpha2/work/Python-3.7.0a2/Modules/_ssl.c:3757:5: error: implicit declaration of function 'DH_free'; did you mean 'bt_free'? [-Werror=implicit-function-declaration] DH_free(dh); ^~~~~~~ bt_free

tiran · 2017-10-10T12:47:04Z

Modules/_ssl.c

@@ -5264,7 +5277,9 @@ PyInit__ssl(void)
        return NULL;
    PySocketModule = *socket_api;

-#ifndef OPENSSL_VERSION_1_1
+#ifdef OPENSSL_VERSION_1_1
+    OPENSSL_init_ssl(0, NULL);


No need to call this, OpenSSL calls this function internally and automatically.

https://www.openssl.org/docs/man1.1.0/ssl/OPENSSL_init_ssl.html

Thanks, reverted in a separate commit. I can squash all the commits into 1 later if you like.
PS: I didn't expect the Spanish Inquisition!

tiran · 2017-10-10T13:05:47Z

Modules/_ssl.c

@@ -4625,7 +4634,11 @@ PySSL_RAND(int len, int pseudo)
    if (bytes == NULL)
        return NULL;
    if (pseudo) {
+#ifdef OPENSSL_VERSION_1_1
+        ok = RAND_bytes((unsigned char*)PyBytes_AS_STRING(bytes), len);


I don't think it is safe to replace RAND_pseudo_bytes with RAND_bytes. A RAND engine may implement bytes and pseudobytes differently, e.g. bytes may not fill the buffer to the end in case of an error.

The function https://docs.python.org/3/library/ssl.html#ssl.RAND_pseudo_bytes has been deprecated. We can drop when OpenSSL 1.2.0 removes the feature.

Thanks, I replaced it with:

ok = (_PyOS_URandom((unsigned char*)PyBytes_AS_STRING(bytes), len) == 0 ? 1 : 0);

bedevere-bot · 2017-10-10T13:14:28Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I didn't expect the Spanish Inquisition!. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

ivvolnov-radargeo · 2017-11-03T06:39:28Z

This patch is not working for me:

*** WARNING: renaming "_ssl" since importing it failed: build/lib.linux-x86_64-3.6/_ssl.cpython-36m-x86_64-linux-gnu.so: undefined symbol: TLSv1_method

markwright · 2017-11-04T03:31:22Z

@ivvolnov-radargeo The patch was split into 2 pull requests, you need both, see:
https://bugs.python.org/issue30008#msg304028

to call this, OpenSSL calls this function internally and automatically.

markwright · 2017-11-08T11:40:45Z

I didn't expect the Spanish Inquisition!

bedevere-bot · 2017-11-08T11:40:47Z

Nobody expects the Spanish Inquisition!

@tiran: please review the changes made to this pull request.

devurandom · 2018-03-07T07:55:32Z

@tiran What's the status of this?

Backport of PRs python#3934 and python#3943 Signed-off-by: Eneas U de Queiroz <[email protected]>

neheb · 2019-11-03T01:38:08Z

This PR rebased to Python 3.8.0: https://raw.githubusercontent.com/neheb/packages/1acf047c7f5c9d807ad49b9787aaaad543734be8/lang/python/python3/patches/020-ssl-module-emulate-tls-methods.patch

Additional fixes:

--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -47,6 +47,7 @@ static PySocketModule_APIObject PySocketModule;
 
 /* Include OpenSSL header files */
 #include "openssl/rsa.h"
+#include "openssl/dh.h"
 #include "openssl/crypto.h"
 #include "openssl/x509.h"
 #include "openssl/x509v3.h"
@@ -128,13 +129,13 @@ static void _PySSLFixErrno(void) {
 #include "_ssl_data.h"
 
 #if (OPENSSL_VERSION_NUMBER >= 0x10100000L) && !defined(LIBRESSL_VERSION_NUMBER)
-#  define OPENSSL_VERSION_1_1 1
-#  define PY_OPENSSL_1_1_API 1
+# define OPENSSL_VERSION_1_1 1
+# define PY_OPENSSL_1_1_API 1
 #endif
 
 /* LibreSSL 2.7.0 provides necessary OpenSSL 1.1.0 APIs */
 #if defined(LIBRESSL_VERSION_NUMBER) && LIBRESSL_VERSION_NUMBER >= 0x2070000fL
-#  define PY_OPENSSL_1_1_API 1
+# define PY_OPENSSL_1_1_API 1
 #endif
 
 /* Openssl comes with TLSv1.1 and TLSv1.2 between 1.0.0h and 1.0.1
@@ -201,6 +202,11 @@ static void _PySSLFixErrno(void) {
 #define TLS_method SSLv23_method
 #define TLS_client_method SSLv23_client_method
 #define TLS_server_method SSLv23_server_method
+#define X509_get0_notBefore X509_get_notBefore
+#define X509_get0_notAfter X509_get_notAfter
+#define OpenSSL_version_num SSLeay
+#define OpenSSL_version SSLeay_version
+#define OPENSSL_VERSION SSLEAY_VERSION
 
 static int X509_NAME_ENTRY_set(const X509_NAME_ENTRY *ne)
 {
@@ -1615,7 +1621,7 @@ _decode_certificate(X509 *certificate) {
     ASN1_INTEGER *serialNumber;
     char buf[2048];
     int len, result;
-    ASN1_TIME *notBefore, *notAfter;
+    const ASN1_TIME *notBefore, *notAfter;
     PyObject *pnotBefore, *pnotAfter;
 
     retval = PyDict_New();
@@ -1677,7 +1683,7 @@ _decode_certificate(X509 *certificate) {
     Py_DECREF(sn_obj);
 
     (void) BIO_reset(biobuf);
-    notBefore = X509_get_notBefore(certificate);
+    notBefore = X509_get0_notBefore(certificate);
     ASN1_TIME_print(biobuf, notBefore);
     len = BIO_gets(biobuf, buf, sizeof(buf)-1);
     if (len < 0) {
@@ -1694,7 +1700,7 @@ _decode_certificate(X509 *certificate) {
     Py_DECREF(pnotBefore);
 
     (void) BIO_reset(biobuf);
-    notAfter = X509_get_notAfter(certificate);
+    notAfter = X509_get0_notAfter(certificate);
     ASN1_TIME_print(biobuf, notAfter);
     len = BIO_gets(biobuf, buf, sizeof(buf)-1);
     if (len < 0) {
@@ -3235,7 +3241,7 @@ _ssl__SSLContext_impl(PyTypeObject *type, int proto_version)
        conservative and assume it wasn't fixed until release. We do this check
        at runtime to avoid problems from the dynamic linker.
        See #25672 for more on this. */
-    libver = SSLeay();
+    libver = OpenSSL_version_num();
     if (!(libver >= 0x10001000UL && libver < 0x1000108fUL) &&
         !(libver >= 0x10000000UL && libver < 0x100000dfUL)) {
         SSL_CTX_set_mode(self->ctx, SSL_MODE_RELEASE_BUFFERS);
@@ -6403,10 +6409,10 @@ PyInit__ssl(void)
         return NULL;
 
     /* OpenSSL version */
-    /* SSLeay() gives us the version of the library linked against,
+    /* OpenSSL_version_num() gives us the version of the library linked against,
        which could be different from the headers version.
     */
-    libver = SSLeay();
+    libver = OpenSSL_version_num();
     r = PyLong_FromUnsignedLong(libver);
     if (r == NULL)
         return NULL;
@@ -6416,7 +6422,7 @@ PyInit__ssl(void)
     r = Py_BuildValue("IIIII", major, minor, fix, patch, status);
     if (r == NULL || PyModule_AddObject(m, "OPENSSL_VERSION_INFO", r))
         return NULL;
-    r = PyUnicode_FromString(SSLeay_version(SSLEAY_VERSION));
+    r = PyUnicode_FromString(OpenSSL_version(OPENSSL_VERSION));
     if (r == NULL || PyModule_AddObject(m, "OPENSSL_VERSION", r))
         return NULL;

github-actions · 2022-02-25T00:08:43Z

This PR is stale because it has been open for 30 days with no activity.

neheb · 2022-02-25T00:19:51Z

I believe Python 3 compiles without deprecated OpenSSL 1.1 APIs now. 3.0 is a much different story.

github-actions · 2022-08-13T00:12:54Z

This PR is stale because it has been open for 30 days with no activity.

pquentin · 2023-05-10T07:13:42Z

This pull request can be closed, per this comment:

PR is partly based on and supersedes #3943

Originally posted by @tiran in #20397 (comment)

github-actions · 2023-06-10T00:10:39Z

This PR is stale because it has been open for 30 days with no activity.

arhadthedev · 2023-06-10T05:49:43Z

Closing after #3943 (comment).

bpo-30008: OpenSSL 1.1 compatibility without using deprecated API

5aed182

Note: RAND_pseudo_bytes() is deprecated so RAND_bytes() is used when pseudo is requested.

the-knights-who-say-ni added the CLA not signed label Oct 10, 2017

bedevere-bot added the awaiting review label Oct 10, 2017

tiran requested changes Oct 10, 2017

View reviewed changes

bedevere-bot added awaiting changes and removed awaiting review labels Oct 10, 2017

markwright added 2 commits November 5, 2017 17:09

Replace OpenSSL 1.1.0 deprecated RAND_pseudo_bytes with _PyOS_URandom

4809f4c

Revert calling OPENSSL_init_ssl(), tiran advises: No need

82d3a39

to call this, OpenSSL calls this function internally and automatically.

the-knights-who-say-ni added CLA signed and removed CLA not signed labels Nov 5, 2017

Add a news file with blurb for removing OpenSSL 1.1 deprecated API

8b5d5c2

markwright force-pushed the fix-issue-30008 branch from 264b6f3 to 8b5d5c2 Compare November 5, 2017 06:54

bedevere-bot removed the awaiting changes label Nov 8, 2017

bedevere-bot added the awaiting change review label Nov 8, 2017

cotequeiroz added a commit to cotequeiroz/cpython that referenced this pull request Jun 5, 2018

Python 2.7: update API deprecated in openssl 1.1

b16e07e

Backport of PRs python#3934 and python#3943 Signed-off-by: Eneas U de Queiroz <[email protected]>

cotequeiroz mentioned this pull request Sep 13, 2018

python: Support building without deprecated APIs openwrt/packages#7006

Closed

jefferyto mentioned this pull request Jan 14, 2020

python3: Fix compilation without deprecated APIs openwrt/packages#11001

Merged

tiran mentioned this pull request May 25, 2020

bpo-30008: Fix OpenSSL no-deprecated compilation (GH-20397) #20397

Merged

github-actions bot added the stale Stale PR or inactive for long period of time. label Feb 25, 2022

github-actions bot removed the stale Stale PR or inactive for long period of time. label Feb 27, 2022

floppym mannequin mentioned this pull request May 18, 2022

OpenSSL 1.1.0 deprecated functions #74194

Open

ezio-melotti removed the CLA signed label Jul 13, 2022

github-actions bot added the stale Stale PR or inactive for long period of time. label Aug 13, 2022

github-actions bot removed the stale Stale PR or inactive for long period of time. label May 11, 2023

github-actions bot added the stale Stale PR or inactive for long period of time. label Jun 10, 2023

arhadthedev closed this Jun 10, 2023

Uh oh!

bpo-30008: OpenSSL 1.1 compatibility without using deprecated API #3943

bpo-30008: OpenSSL 1.1 compatibility without using deprecated API #3943

Uh oh!

Conversation

markwright commented Oct 10, 2017 • edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

the-knights-who-say-ni commented Oct 10, 2017

Uh oh!

tiran left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bedevere-bot commented Oct 10, 2017

Uh oh!

ivvolnov-radargeo commented Nov 3, 2017

Uh oh!

markwright commented Nov 4, 2017

Uh oh!

markwright commented Nov 8, 2017

Uh oh!

bedevere-bot commented Nov 8, 2017

Uh oh!

devurandom commented Mar 7, 2018

Uh oh!

neheb commented Nov 3, 2019

Uh oh!

github-actions bot commented Feb 25, 2022

Uh oh!

neheb commented Feb 25, 2022

Uh oh!

github-actions bot commented Aug 13, 2022

Uh oh!

pquentin commented May 10, 2023

Uh oh!

github-actions bot commented Jun 10, 2023

Uh oh!

arhadthedev commented Jun 10, 2023

Uh oh!

Uh oh!

markwright commented Oct 10, 2017 •

edited by bedevere-bot

Loading