bpo-24048: Save the live exception during import.c's remove_module() #13005

ZackerySpytz · 2019-04-29T09:29:54Z

Save the live exception during the course of remove_module().

https://bugs.python.org/issue24048

Save the live exception during the course of remove_module().

vstinner

LGTM. But maybe the NEWS entry to explain what was the problem rathe than explaining the fix.

Python/import.c

vstinner

LGTM.

Python/import.c

brettcannon · 2019-05-06T16:54:18Z

Any reason you didn't merge, @vstinner ?

vstinner · 2019-05-10T01:40:54Z

Python/import.c

    if (PyMapping_DelItem(modules, name) < 0) {
-        if (!PyMapping_HasKey(modules, name)) {
-            return;
-        }
        Py_FatalError("import:  deleting existing key in "
                      "sys.modules failed");


By the way, it would be nice to chain the two exceptions rather than calling the nasty Py_FatalError() function :-(

miss-islington · 2020-05-29T19:17:19Z

Thanks @ZackerySpytz for the PR, and @ncoghlan for merging it 🌮🎉.. I'm working now to backport this PR to: 3.7.
🐍🍒⛏🤖

miss-islington · 2020-05-29T19:17:19Z

Thanks @ZackerySpytz for the PR, and @ncoghlan for merging it 🌮🎉.. I'm working now to backport this PR to: 3.6.
🐍🍒⛏🤖

miss-islington · 2020-05-29T19:17:20Z

Thanks @ZackerySpytz for the PR, and @ncoghlan for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

bedevere-bot · 2020-05-29T19:17:33Z

GH-20521 is a backport of this pull request to the 3.7 branch.

miss-islington · 2020-05-29T19:17:34Z

Sorry, @ZackerySpytz and @ncoghlan, I could not cleanly backport this to 3.6 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 94a64e9cd411a87514b68082c1c437eb3b49dfb9 3.6

miss-islington · 2020-05-29T19:17:35Z

Sorry @ZackerySpytz and @ncoghlan, I had trouble checking out the 3.8 backport branch.
Please backport using cherry_picker on command line.
cherry_picker 94a64e9cd411a87514b68082c1c437eb3b49dfb9 3.8

…ythonGH-13005) Save the live exception during the course of remove_module(). (cherry picked from commit 94a64e9) Co-authored-by: Zackery Spytz <[email protected]>

…H-13005) Save the live exception during the course of remove_module(). (cherry picked from commit 94a64e9) Co-authored-by: Zackery Spytz <[email protected]>

bedevere-bot · 2020-05-29T21:31:17Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable LTO + PGO 3.7 has failed when building commit 5aa40e5.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/429/builds/110) and take a look at the build logs.
Check if the failure is related to this commit (5aa40e5) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/429/builds/110

Summary of the results of the build (if available):

Click to see traceback logs

remote: Enumerating objects: 8, done.        
remote: Counting objects:  12% (1/8)        
remote: Counting objects:  25% (2/8)        
remote: Counting objects:  37% (3/8)        
remote: Counting objects:  50% (4/8)        
remote: Counting objects:  62% (5/8)        
remote: Counting objects:  75% (6/8)        
remote: Counting objects:  87% (7/8)        
remote: Counting objects: 100% (8/8)        
remote: Counting objects: 100% (8/8), done.        
remote: Total 9 (delta 7), reused 7 (delta 7), pack-reused 1        
From https://github.com/python/cpython
 * branch                  3.7        -> FETCH_HEAD
Reset branch '3.7'

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make[2]: [Makefile:1722: clean] Error 1 (ignored)
Objects/longobject.c: In function ‘_PyLong_Frexp’:
Objects/longobject.c:2826:33: warning: ‘x_digits[0]’ may be used uninitialized in this function [-Wmaybe-uninitialized]
 2826 |                     x_digits[0] |= 1;
      |                                 ^~
In file included from Python/ast.c:8:
Python/ast.c: In function ‘ast_for_stmt’:
./Include/node.h:34:29: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
   34 | #define LINENO(n)       ((n)->n_lineno)
      |                         ~~~~^~~~~~~~~~~
In function ‘assemble_lnotab’,
    inlined from ‘assemble_emit’ at Python/compile.c:5247:25,
    inlined from ‘assemble’ at Python/compile.c:5525:18:
Python/compile.c:5201:19: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
 5201 |         *lnotab++ = k;
      |         ~~~~~~~~~~^~~
Python/compile.c: In function ‘compiler_visit_stmt’:
Python/compile.c:2970:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
 2970 | compiler_visit_stmt(struct compiler *c, stmt_ty s)
      | ^~~~~~~~~~~~~~~~~~~
Modules/main.c: In function ‘_PyCoreConfig_Copy’:
Modules/main.c:2495:47: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
 2495 | #define COPY_ATTR(ATTR) config->ATTR = config2->ATTR
      |                                        ~~~~~~~^~~~~~
 2496 | #define COPY_STR_ATTR(ATTR) \
      | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                  
 2497 |     do { \
      |     ~~~~~~                                     
 2498 |         if (config2->ATTR != NULL) { \
      |         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~         
 2499 |             config->ATTR = _PyMem_RawWcsdup(config2->ATTR); \
      |             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 2500 |             if (config->ATTR == NULL) { \
      |             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~      
 2501 |                 return -1; \
      |                 ~~~~~~~~~~~~                   
 2502 |             } \
      |             ~~~                                
 2503 |         } \
      |         ~~~                                    
 2504 |     } while (0)
      |     ~~~~~~~~~~~                                
 2505 | #define COPY_WSTRLIST(LEN, LIST) \
      | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~             
 2506 |     do { \
      |     ~~~~~~                                     
 2507 |         if (config2->LIST != NULL) { \
      |         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~         
 2508 |             config->LIST = copy_wstrlist(config2->LEN, config2->LIST); \
      |             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 2509 |             if (config->LIST == NULL) { \
      |             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~      
 2510 |                 return -1; \
      |                 ~~~~~~~~~~~~                   
 2511 |             } \
      |             ~~~                                
 2512 |         } \
      |         ~~~                                    
 2513 |         config->LEN = config2->LEN; \
      |         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~          
 2514 |     } while (0)
      |     ~~~~~~~~~~~                                
 2515 | 
      |                                                
 2516 |     COPY_ATTR(install_signal_handlers);
      |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~          
Modules/main.c: In function ‘_PyCoreConfig_Copy’:
Modules/main.c:2516:5: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
 2516 |     COPY_ATTR(install_signal_handlers);
      |     ^
Modules/main.c: In function ‘_PyCoreConfig_Copy’:
Modules/main.c:2516:5: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
 2516 |     COPY_ATTR(install_signal_handlers);
      |     ^
In function ‘assemble_lnotab’,
    inlined from ‘assemble_emit’ at Python/compile.c:5247:0,
    inlined from ‘assemble’ at Python/compile.c:5525:0:
Python/compile.c:5201: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
 5201 |         *lnotab++ = k;
      | 
In function ‘assemble_lnotab’,
    inlined from ‘assemble_emit’ at Python/compile.c:5247:0,
    inlined from ‘assemble’ at Python/compile.c:5525:0:
Python/compile.c:5201: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
 5201 |         *lnotab++ = k;
      | 
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/socketmodule.c: In function ‘getsockaddrarg’:
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/socketmodule.c:2194:9: warning: ‘strncpy’ specified bound 64 equals destination size [-Wstringop-truncation]
 2194 |         strncpy((char *)sa->salg_name, name, sizeof(sa->salg_name));
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/socketmodule.c: In function ‘sock_sendmsg’:
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/socketmodule.c:4230:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
 4230 | sock_sendmsg(PySocketSockObject *s, PyObject *args)
      | ^~~~~~~~~~~~
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/socketmodule.c: In function ‘sock_sendmsg’:
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/socketmodule.c:4230:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
 4230 | sock_sendmsg(PySocketSockObject *s, PyObject *args)
      | ^
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/sha256module.c: In function ‘SHA256Type_copy’:
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/sha256module.c:78:22: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
   78 |     dest->local = src->local;
      |                   ~~~^~~~~~~
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/sha256module.c: In function ‘SHA256Type_copy’:
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/sha256module.c:78:22: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
   78 |     dest->local = src->local;
      |                      ^
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/sha512module.c: In function ‘SHA512Type_copy’:
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/sha512module.c:87:22: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
   87 |     dest->local = src->local;
      |                   ~~~^~~~~~~
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/sha512module.c: In function ‘SHA512Type_copy’:
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/sha512module.c:87:22: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
   87 |     dest->local = src->local;
      |                      ^
In file included from /home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/pyexpat.c:71:
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/clinic/pyexpat.c.h: In function ‘pyexpat_ParserCreate’:
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/clinic/pyexpat.c.h:243:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  243 | pyexpat_ParserCreate(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
      | ^~~~~~~~~~~~~~~~~~~~
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/expat/xmlparse.c: In function ‘build_node’:
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/expat/xmlparse.c:6803:39: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
 6803 |   dest->type = dtd->scaffold[src_node].type;
      |                ~~~~~~~~~~~~~~~~~~~~~~~^~~~~
In file included from /home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/expat/expat_config.h:8,
                 from /home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/expat/xmltok.c:49:
./pyconfig.h:1526: warning: "_POSIX_C_SOURCE" redefined
 1526 | #define _POSIX_C_SOURCE 200809L
      | 
In file included from /usr/include/bits/libc-header-start.h:33,
                 from /usr/include/string.h:26,
                 from /home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/expat/xmltok.c:34:
/usr/include/features.h:310: note: this is the location of the previous definition
  310 | # define _POSIX_C_SOURCE 199506L
      | 
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/expat/xmlparse.c: In function ‘build_node’:
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/expat/xmlparse.c:6803:39: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
 6803 |   dest->type = dtd->scaffold[src_node].type;
      |                                       ^
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/_ctypes/_ctypes.c: In function ‘PyCSimpleType_new’:
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/_ctypes/_ctypes.c:2013:21: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
 2013 |     stgdict->length = 0;
      |     ~~~~~~~~~~~~~~~~^~~
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/_ctypes/_ctypes.c: In function ‘PyCSimpleType_new’:
/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Modules/_ctypes/_ctypes.c:2013:21: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
 2013 |     stgdict->length = 0;
      |                     ^
Task was destroyed but it is pending!
task: <Task pending coro=<<async_generator_athrow without __name__>()>>
Task was destroyed but it is pending!
task: <Task pending coro=<<async_generator_athrow without __name__>()>>
stty: 'standard input': Inappropriate ioctl for device
Exception in thread Thread-1563:
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/test/test_ssl.py", line 2392, in run
    msg = self.read()
  File "/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/test/test_ssl.py", line 2369, in read
    return self.sslconn.read()
  File "/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/ssl.py", line 931, in read
    return self._sslobj.read(len)
ssl.SSLError: [SSL: PEER_DID_NOT_RETURN_A_CERTIFICATE] peer did not return a certificate (_ssl.c:2546)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/home/buildbot/buildarea/3.7.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/test/test_ssl.py", line 2478, in run
    raise ssl.SSLError('tlsv13 alert certificate required')
ssl.SSLError: ('tlsv13 alert certificate required',)

bpo-24048: Save the live exception during import.c's remove_module()

f935e6e

Save the live exception during the course of remove_module().

ZackerySpytz requested a review from a team April 29, 2019 09:29

the-knights-who-say-ni added the CLA signed label Apr 29, 2019

bedevere-bot added the awaiting review label Apr 29, 2019

vstinner approved these changes Apr 29, 2019

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting review labels Apr 29, 2019

serhiy-storchaka reviewed May 2, 2019

View reviewed changes

Python/import.c Show resolved Hide resolved

vstinner approved these changes May 2, 2019

View reviewed changes

Python/import.c Show resolved Hide resolved

ncoghlan merged commit 94a64e9 into python:master May 8, 2019

bedevere-bot removed the awaiting merge label May 8, 2019

vstinner reviewed May 10, 2019

View reviewed changes

serhiy-storchaka added needs backport to 3.6 labels May 29, 2020

bedevere-bot removed the needs backport to 3.7 label May 29, 2020

miss-islington assigned ncoghlan May 29, 2020

serhiy-storchaka removed needs backport to 3.6 labels May 29, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

bpo-24048: Save the live exception during import.c's remove_module() #13005

bpo-24048: Save the live exception during import.c's remove_module() #13005

Uh oh!

ZackerySpytz commented Apr 29, 2019 •

edited by bedevere-bot

Loading

Uh oh!

vstinner left a comment

Uh oh!

Uh oh!

vstinner left a comment

Uh oh!

Uh oh!

brettcannon commented May 6, 2019

Uh oh!

vstinner May 10, 2019

Uh oh!

miss-islington commented May 29, 2020

Uh oh!

miss-islington commented May 29, 2020

Uh oh!

miss-islington commented May 29, 2020

Uh oh!

bedevere-bot commented May 29, 2020

Uh oh!

miss-islington commented May 29, 2020

Uh oh!

miss-islington commented May 29, 2020

Uh oh!

bedevere-bot commented May 29, 2020

Uh oh!

Uh oh!

Uh oh!

bpo-24048: Save the live exception during import.c's remove_module() #13005

bpo-24048: Save the live exception during import.c's remove_module() #13005

Uh oh!

Conversation

ZackerySpytz commented Apr 29, 2019 • edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

brettcannon commented May 6, 2019

Uh oh!

vstinner May 10, 2019

Choose a reason for hiding this comment

Uh oh!

miss-islington commented May 29, 2020

Uh oh!

miss-islington commented May 29, 2020

Uh oh!

miss-islington commented May 29, 2020

Uh oh!

bedevere-bot commented May 29, 2020

Uh oh!

miss-islington commented May 29, 2020

Uh oh!

miss-islington commented May 29, 2020

Uh oh!

bedevere-bot commented May 29, 2020

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

Uh oh!

ZackerySpytz commented Apr 29, 2019 •

edited by bedevere-bot

Loading