Skip to content

bpo-38045: Avoid quadratic complexity when creating composite IntFlags. #16291

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

anntzer
Copy link
Contributor

@anntzer anntzer commented Sep 19, 2019

When one creates all possible combos of an IntFlag, e.g. with

from enum import Flag
F = Flag("F", list("abcdefghijklm"))
for idx in range(2**len(F) - 1):
    F(idx)

the previous version would exhibit quadratic complexity because
_value2member_map_ would grow bigger over each iteration.

Iterate over "true" flag members rather than _value2member_map_,
to avoid this problem. For a 13-member flag (so 8192 combinations),
the speedup is more than 100x.

https://bugs.python.org/issue38045

When one creates all possible combos of an IntFlag, e.g. with

    from enum import Flag
    F = Flag("F", list("abcdefghijklm"))
    for idx in range(2**len(F) - 1):
        F(idx)

the previous version would exhibit quadratic complexity because
_value2member_map_ would grow bigger over each iteration.

Iterate over "true" flag members rather than _value2member_map_,
to avoid this problem.  For a 13-member flag (so 8192 combinations),
the speedup is more than 100x.
@the-knights-who-say-ni
Copy link

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed the PSF contributor agreement (CLA).

Recognized GitHub username

We couldn't find a bugs.python.org (b.p.o) account corresponding to the following GitHub usernames:

@anntzer

This might be simply due to a missing "GitHub Name" entry in one's b.p.o account settings. This is necessary for legal reasons before we can look at this contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

You can check yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

@hongweipeng
Copy link
Contributor

I think it's better not to create members in _decompose(). Just like its name, it only does decomposition work.

@hongweipeng
Copy link
Contributor

I created a commit without creating members hongweipeng@a5f0601 . the speedup is aslo more than 100x in the example and passes test_enum.py.

@anntzer
Copy link
Contributor Author

anntzer commented Sep 26, 2019

Your patch fixes the problem with Flag, but not with IntFlag (replace Flag by IntFlag in the original example and the slow behavior still exists with your patch, but not with mine).

@hongweipeng
Copy link
Contributor

@anntzer Thanks for your response. You are right. The previous commit did not resolve IntFlag, but I think there are still ways to solve Flag and IntFlag without creating members. So I create another commit, see in hongweipeng@9ae31f4 .

@anntzer anntzer closed this Nov 27, 2019
@anntzer
Copy link
Contributor Author

anntzer commented Nov 27, 2019

Superseded by #16483.

@anntzer anntzer deleted the quadratic-intflags branch November 27, 2019 06:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants