Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-46400: Update libexpat from 2.4.1 to 2.4.4 #31022

Merged
merged 3 commits into from Feb 12, 2022
Merged

Conversation

@jouve
Copy link
Contributor

@jouve jouve commented Jan 30, 2022

includes changes from 2.4.1 to 2.4.4 https://github.com/libexpat/libexpat/blob/R_2_4_4/expat/Changes

This will fix the CI issues like https://buildbot.python.org/all/#/builders/271/builds/1233

https://bugs.python.org/issue46400

@the-knights-who-say-ni
Copy link

@the-knights-who-say-ni the-knights-who-say-ni commented Jan 30, 2022

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed the PSF contributor agreement (CLA).

Recognized GitHub username

We couldn't find a bugs.python.org (b.p.o) account corresponding to the following GitHub usernames:

@jouve

This might be simply due to a missing "GitHub Name" entry in one's b.p.o account settings. This is necessary for legal reasons before we can look at this contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

You can check yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

@jouve jouve force-pushed the expat_R_2_4_4 branch from ea592c8 to d438a97 Jan 30, 2022
@corona10 corona10 self-requested a review Jan 31, 2022
@corona10
Copy link
Member

@corona10 corona10 commented Jan 31, 2022

Please sign up the CLA first

@corona10 corona10 requested a review from vstinner Jan 31, 2022
@jouve
Copy link
Contributor Author

@jouve jouve commented Jan 31, 2022

Please sign up the CLA first

sent it already , waiting for its validation :)

@corona10
Copy link
Member

@corona10 corona10 commented Feb 5, 2022

@jouve I will take a look soon :)

@vstinner
Copy link
Member

@vstinner vstinner commented Feb 8, 2022

How did you create this PR?

@jouve
Copy link
Contributor Author

@jouve jouve commented Feb 8, 2022

I imported code from https://github.com/libexpat/libexpat at R_2_4_4 tag

I re-introduced the 3 lines here https://github.com/python/cpython/blob/main/Modules/expat/expat_external.h#L70:

/* Namespace external symbols to allow multiple libexpat version to
   co-exist. */
#include "pyexpatns.h"

the diffs about #include <expat_config.h> locations were introduced in python's version in commits like : d413c50 and in libexpat in libexpat/libexpat@59734d6

In both case by @corona10, and according libexpat/libexpat#513 for the same reason. I kept the one from libexpat.

The rest of the changes are relevant to the diff/changelog between R_2_4_1 and R_2_4_4.

I also arrived to the same result exporting the relevant patches in libexpat and re-applying them in cpython.

@corona10
Copy link
Member

@corona10 corona10 commented Feb 9, 2022

@hartwork Can you please take a look?

@jouve
Copy link
Contributor Author

@jouve jouve commented Feb 9, 2022

If you want/it's easier to review, I can regenerate this PR using the 2nd method I described (export the patches & re-apply), it will generate about 15 commits instead of one.

@hartwork
Copy link
Contributor

@hartwork hartwork commented Feb 9, 2022

@hartwork Can you please take a look?

@corona10 I just have, and it looks sane to me, but don't have CPython glasses. What I would suggest a person with CPython glasses does, is this:

## 1. Understand CPython's deviation from libexpat 2.4.1 upstream
# git clone --depth 1 https://github.com/python/cpython  # i.e. main
# git clone --depth 1 --branch R_2_4_1 https://github.com/libexpat/libexpat libexpat_2_4_1
# diff --color=always -r -u libexpat_2_4_1/expat/lib/ cpython/Modules/expat/ | less

## 2. Understand jouve CPython's deviation from libexpat 2.4.4 upstream
# git clone --depth 1 --branch expat_R_2_4_4 https://github.com/jouve/cpython jouve_cpython
# git clone --depth 1 --branch R_2_4_4 https://github.com/libexpat/libexpat libexpat_2_4_4
# diff --color=always -r -u libexpat_2_4_4/expat/lib/ jouve_cpython/Modules/expat/ | less

The diffs are small and the diff between the diffs is also small. Again, looks good to me, but someone with CPython glasses should check for themselves as well please.

If you want/it's easier to review, I can regenerate this PR using the 2nd method I described (export the patches & re-apply), it will generate about 15 commits instead of one.

@jouve my vote for sticking to status quo, because the approach above likely works for others too and I'm not sure all these commits (with different paths and different SHA1s) add value to the Git history of CPython. Just my 2 cents.
Thanks for the PR! 👍

Copy link
Member

@corona10 corona10 left a comment

lgtm with minor comment

@corona10
Copy link
Member

@corona10 corona10 commented Feb 12, 2022

@hartwork Thanks, I double-check also, it looks good to me too :)

@@ -0,0 +1 @@
expat: Update libexpat from 2.4.1 to 2.4.4 to fix security issue (CVE-2021-45960)
Copy link
Contributor

@hartwork hartwork Feb 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please note that CVE-2021-45960 is only 1 of 10 CVEs addressed since 2.4.1, see https://github.com/libexpat/libexpat/blob/master/expat/Changes . I would either go all in and mention all 10 or not mention CVEs at all, just my 2 cents.

Copy link
Member

@corona10 corona10 Feb 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@corona10 corona10 changed the title bpo-46400: vendor expat 2.4.4 bpo-46400: Update libexpat from 2.4.1 to 2.4.4 Feb 12, 2022
@corona10 corona10 merged commit 8aaaf7e into python:main Feb 12, 2022
12 checks passed
@miss-islington
Copy link
Contributor

@miss-islington miss-islington commented Feb 12, 2022

Thanks @jouve for the PR, and @corona10 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9, 3.10.
🐍🍒🤖

@miss-islington
Copy link
Contributor

@miss-islington miss-islington commented Feb 12, 2022

Sorry @jouve and @corona10, I had trouble checking out the 3.10 backport branch.
Please backport using cherry_picker on command line.
cherry_picker 8aaaf7e182e22026c3487a3b86d4d7d4f0f5f778 3.10

@bedevere-bot
Copy link

@bedevere-bot bedevere-bot commented Feb 12, 2022

GH-31295 is a backport of this pull request to the 3.9 branch.

miss-islington added a commit to miss-islington/cpython that referenced this issue Feb 12, 2022
(cherry picked from commit 8aaaf7e)

Co-authored-by: Cyril Jouve <jv.cyril@gmail.com>
@miss-islington
Copy link
Contributor

@miss-islington miss-islington commented Feb 12, 2022

Thanks @jouve for the PR, and @corona10 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.7.
🐍🍒🤖

@miss-islington
Copy link
Contributor

@miss-islington miss-islington commented Feb 12, 2022

Thanks @jouve for the PR, and @corona10 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒🤖

@miss-islington
Copy link
Contributor

@miss-islington miss-islington commented Feb 12, 2022

Sorry, @jouve and @corona10, I could not cleanly backport this to 3.7 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 8aaaf7e182e22026c3487a3b86d4d7d4f0f5f778 3.7

@miss-islington
Copy link
Contributor

@miss-islington miss-islington commented Feb 12, 2022

Sorry @jouve and @corona10, I had trouble checking out the 3.8 backport branch.
Please backport using cherry_picker on command line.
cherry_picker 8aaaf7e182e22026c3487a3b86d4d7d4f0f5f778 3.8

@bedevere-bot
Copy link

@bedevere-bot bedevere-bot commented Feb 12, 2022

GH-31296 is a backport of this pull request to the 3.10 branch.

@bedevere-bot
Copy link

@bedevere-bot bedevere-bot commented Feb 12, 2022

GH-31297 is a backport of this pull request to the 3.8 branch.

@bedevere-bot
Copy link

@bedevere-bot bedevere-bot commented Feb 12, 2022

GH-31298 is a backport of this pull request to the 3.7 branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants