Skip to content

[3.5] bpo-30675: Fix multiprocessing code in regrtest #2220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 15, 2017
Merged

[3.5] bpo-30675: Fix multiprocessing code in regrtest #2220

merged 1 commit into from
Jun 15, 2017

Conversation

vstinner
Copy link
Member

  • Rewrite code to pass slaveargs from the master process to worker
    processes: reuse the same code of the Python master branch
  • Move code to initialize tests in a new setup_tests() function,
    similar change was done in the master branch
  • In a worker process, call setup_tests() with the namespace built
    from slaveargs to initialize correctly tests

Before this change, warm_caches() was not called in worker processes
because the setup was done before rebuilding the namespace from
slaveargs. As a consequence, the huntrleaks feature was unstable. For
example, test_zipfile reported randomly false positive on reference
leaks.

else:
msvcrt.CrtSetReportMode(m, 0)
if ns.wait:
input("Press any key to continue...")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to do this on subprocesses?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. It's not done in worker processes in master neither, so I moved this code out of setup_tests() again.

try:
result = runtest(*args, **kwargs)
result = runtest(testname, ns.verbose, ns.quiet,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use_resources is not passed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, nice catch, fixed.

It's a bug in my change. I copied the runtest() call from above, but above, there is a first "support.use_resources = ns.use_resources" call.

* Rewrite code to pass slaveargs from the master process to worker
  processes: reuse the same code of the Python master branch
* Move code to initialize tests in a new setup_tests() function,
  similar change was done in the master branch
* In a worker process, call setup_tests() with the namespace built
  from slaveargs to initialize correctly tests

Before this change, warm_caches() was not called in worker processes
because the setup was done before rebuilding the namespace from
slaveargs. As a consequence, the huntrleaks feature was unstable. For
example, test_zipfile reported randomly false positive on reference
leaks.
@vstinner vstinner merged commit 33cf0c4 into python:3.5 Jun 15, 2017
@vstinner vstinner deleted the regrtest_slaveargs branch June 15, 2017 22:08
@vstinner
Copy link
Member Author

I fixed the two bugs that you spotted @serhiy-storchaka, thanks for your review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants