Skip to content

bpo-46400: Update libexpat from 2.4.1 to 2.4.4 #31022

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 12, 2022
Merged

bpo-46400: Update libexpat from 2.4.1 to 2.4.4 #31022

merged 3 commits into from
Feb 12, 2022

Conversation

jouve
Copy link
Contributor

@jouve jouve commented Jan 30, 2022

@the-knights-who-say-ni
Copy link

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed the PSF contributor agreement (CLA).

Recognized GitHub username

We couldn't find a bugs.python.org (b.p.o) account corresponding to the following GitHub usernames:

@jouve

This might be simply due to a missing "GitHub Name" entry in one's b.p.o account settings. This is necessary for legal reasons before we can look at this contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

You can check yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

@corona10 corona10 self-requested a review January 31, 2022 04:54
@corona10
Copy link
Member

Please sign up the CLA first

@corona10 corona10 requested a review from vstinner January 31, 2022 04:55
@jouve
Copy link
Contributor Author

jouve commented Jan 31, 2022

Please sign up the CLA first

sent it already , waiting for its validation :)

@corona10
Copy link
Member

corona10 commented Feb 5, 2022

@jouve I will take a look soon :)

@vstinner
Copy link
Member

vstinner commented Feb 8, 2022

How did you create this PR?

@jouve
Copy link
Contributor Author

jouve commented Feb 8, 2022

I imported code from https://github.com/libexpat/libexpat at R_2_4_4 tag

I re-introduced the 3 lines here https://github.com/python/cpython/blob/main/Modules/expat/expat_external.h#L70:

/* Namespace external symbols to allow multiple libexpat version to
   co-exist. */
#include "pyexpatns.h"

the diffs about #include <expat_config.h> locations were introduced in python's version in commits like : d413c50 and in libexpat in libexpat/libexpat@59734d6

In both case by @corona10, and according libexpat/libexpat#513 for the same reason. I kept the one from libexpat.

The rest of the changes are relevant to the diff/changelog between R_2_4_1 and R_2_4_4.

I also arrived to the same result exporting the relevant patches in libexpat and re-applying them in cpython.

@corona10
Copy link
Member

corona10 commented Feb 9, 2022

@hartwork Can you please take a look?

@jouve
Copy link
Contributor Author

jouve commented Feb 9, 2022

If you want/it's easier to review, I can regenerate this PR using the 2nd method I described (export the patches & re-apply), it will generate about 15 commits instead of one.

@hartwork
Copy link
Contributor

hartwork commented Feb 9, 2022

@hartwork Can you please take a look?

@corona10 I just have, and it looks sane to me, but don't have CPython glasses. What I would suggest a person with CPython glasses does, is this:

## 1. Understand CPython's deviation from libexpat 2.4.1 upstream
# git clone --depth 1 https://github.com/python/cpython  # i.e. main
# git clone --depth 1 --branch R_2_4_1 https://github.com/libexpat/libexpat libexpat_2_4_1
# diff --color=always -r -u libexpat_2_4_1/expat/lib/ cpython/Modules/expat/ | less

## 2. Understand jouve CPython's deviation from libexpat 2.4.4 upstream
# git clone --depth 1 --branch expat_R_2_4_4 https://github.com/jouve/cpython jouve_cpython
# git clone --depth 1 --branch R_2_4_4 https://github.com/libexpat/libexpat libexpat_2_4_4
# diff --color=always -r -u libexpat_2_4_4/expat/lib/ jouve_cpython/Modules/expat/ | less

The diffs are small and the diff between the diffs is also small. Again, looks good to me, but someone with CPython glasses should check for themselves as well please.

If you want/it's easier to review, I can regenerate this PR using the 2nd method I described (export the patches & re-apply), it will generate about 15 commits instead of one.

@jouve my vote for sticking to status quo, because the approach above likely works for others too and I'm not sure all these commits (with different paths and different SHA1s) add value to the Git history of CPython. Just my 2 cents.
Thanks for the PR! 👍

Copy link
Member

@corona10 corona10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm with minor comment

@corona10 corona10 added needs backport to 3.9 only security fixes needs backport to 3.10 only security fixes labels Feb 12, 2022
@corona10
Copy link
Member

@hartwork Thanks, I double-check also, it looks good to me too :)

@@ -0,0 +1 @@
expat: Update libexpat from 2.4.1 to 2.4.4 to fix security issue (CVE-2021-45960)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please note that CVE-2021-45960 is only 1 of 10 CVEs addressed since 2.4.1, see https://github.com/libexpat/libexpat/blob/master/expat/Changes . I would either go all in and mention all 10 or not mention CVEs at all, just my 2 cents.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@corona10 corona10 changed the title bpo-46400: vendor expat 2.4.4 bpo-46400: Update libexpat from 2.4.1 to 2.4.4 Feb 12, 2022
@corona10 corona10 merged commit 8aaaf7e into python:main Feb 12, 2022
@miss-islington
Copy link
Contributor

Thanks @jouve for the PR, and @corona10 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9, 3.10.
🐍🍒⛏🤖

@miss-islington
Copy link
Contributor

Sorry @jouve and @corona10, I had trouble checking out the 3.10 backport branch.
Please backport using cherry_picker on command line.
cherry_picker 8aaaf7e182e22026c3487a3b86d4d7d4f0f5f778 3.10

@bedevere-bot
Copy link

GH-31295 is a backport of this pull request to the 3.9 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.9 only security fixes label Feb 12, 2022
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Feb 12, 2022
(cherry picked from commit 8aaaf7e)

Co-authored-by: Cyril Jouve <[email protected]>
@miss-islington
Copy link
Contributor

Thanks @jouve for the PR, and @corona10 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.7.
🐍🍒⛏🤖

@miss-islington
Copy link
Contributor

Thanks @jouve for the PR, and @corona10 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

@miss-islington
Copy link
Contributor

Sorry, @jouve and @corona10, I could not cleanly backport this to 3.7 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 8aaaf7e182e22026c3487a3b86d4d7d4f0f5f778 3.7

@miss-islington
Copy link
Contributor

Sorry @jouve and @corona10, I had trouble checking out the 3.8 backport branch.
Please backport using cherry_picker on command line.
cherry_picker 8aaaf7e182e22026c3487a3b86d4d7d4f0f5f778 3.8

corona10 pushed a commit to corona10/cpython that referenced this pull request Feb 12, 2022
@bedevere-bot bedevere-bot removed the needs backport to 3.10 only security fixes label Feb 12, 2022
@bedevere-bot
Copy link

GH-31296 is a backport of this pull request to the 3.10 branch.

@bedevere-bot
Copy link

GH-31297 is a backport of this pull request to the 3.8 branch.

corona10 pushed a commit to corona10/cpython that referenced this pull request Feb 12, 2022
corona10 pushed a commit to corona10/cpython that referenced this pull request Feb 12, 2022
@bedevere-bot
Copy link

GH-31298 is a backport of this pull request to the 3.7 branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants