Expose Expat XML attack protection APIs

@gpshead

BPO	46793
Nosy	@gpshead, @hartwork, @corona10

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2022-02-18.21:27:31.809>
labels = ['type-feature', '3.11']
title = 'expose expat XML billion laughs attack mitigation APIs'
updated_at = <Date 2022-02-25.02:16:52.654>
user = 'https://github.com/gpshead'

bugs.python.org fields:

activity = <Date 2022-02-25.02:16:52.654>
actor = 'sping'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2022-02-18.21:27:31.809>
creator = 'gregory.p.smith'
dependencies = []
files = []
hgrepos = []
issue_num = 46793
keywords = []
message_count = 2.0
messages = ['413513', '413955']
nosy_count = 3.0
nosy_names = ['gregory.p.smith', 'sping', 'corona10']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue46793'
versions = ['Python 3.11']

Linked PRs

@picnixz

By orphan I meant without a parent parser.

@picnixz oh I see!

Considering the failure can only happen when either the parser has a parent or the factor is invalid as a number, I'm going to simply remake those checks on Python's side so that I can raise appropriate exceptions (ideally I would have loved having a return code being returned by expat so that I know what exactly happened but it's also fine).

I'm reading that as doing a value range check in CPython "manually" — 1.0 to +infinity (both inclusive) for amplification and 0 to 2^64-1 (both inclusive) for the threshold — and treating failure to set after passed input validation as "must have failed because there was a parent parser present". Is that what you had in mind?

I could introduce a new API function XML_Parser XML_ParserGetParent(XML_Parser parser) just for this for Expat 2.7.3 (and aim at getting it released before — say — 2025-10-20), if it changes the picture for CPython significantly, what do you think?

I'm reading that as doing a value range check in CPython "manually"

Yes.

I would like to return a meaningful error when someone uses an invalid factor so that users know that it was the factor that was wrong and I also want to have a meaningful error when users use the function for parsers with a parent.

Ideally I would like the functions to return an error code if they failed (instead of XML_FALSE) and that error code should be different depending on the error that happened (so that I can distinguish between a "bad" parser and a "bad" factor).

I don't want to burden Expat with utility functions only for us, so I also don't mind raising a generic error and document what could be the reason.

@picnixz

@picnixz okay 👍

gh-90949: add Expat API to prevent XML deadly allocations (CVE-2025-5…

[3.14] pythongh-90949: add Expat API to prevent XML deadly allocations (

gh-90949: amend GH-139234 in prevision of future mitigation API (#139366

pythongh-90949: amend pythonGH-139234 in prevision of future mitigati…

gh-90949: expose Expat API to tune exponential expansion protections (#…

pythongh-90949: amend pythonGH-139234 in prevision of future mitigati…

[3.12] pythongh-90949: add Expat API to prevent XML deadly allocations (

pythongh-90949: amend pythonGH-139234 in prevision of future mitigati…

pythongh-90949: add Expat API to prevent XML deadly allocations (CVE-…

pythongh-90949: amend pythonGH-139234 in prevision of future mitigati…

pythongh-90949: add Expat API to prevent XML deadly allocations (CVE-…

pythongh-90949: amend pythonGH-139234 in prevision of future mitigati…

gh-90949: Fix an "unused function" compiler warning introduced in GH-…

pythongh-90949: Fix an "unused function" compiler warning introduced in

@hartwork

Ok, so here's the plan for the future.

We have PRs for tuning the allocation limiter in pyexpat for 3.10 up to 3.14. main already has the feature. @hartwork, could you go through the PRs and update them accordingly? TiA.
Next, we should create backports for tuning the billion laughs protection in pyexpat for 3.10 up to 3.14. main already has the feature. For that, I'm going to handle the backports after the PRs above are done.
Finally, we should design an API for tuning those protections from etree parsers as it was done for .flush(). I don't know if we need to do it for SAX parsers though as I now don't remember if parser deferral couold be tuned on SAX parsers (maybe it can because of inheriting interfaces).

What I want is that, at any point of the time, security branches and release branches have the same APIs. As discused, 3.9 won't have those APIs and will live with the defaults. I don't know when 3.10/3.11 (@pablogsal) and 3.12 (@Yhg1s) will have their security releases but if it's in the next days, I'm going to skip those releases and wait for the next one. Then we can merge the first batch of PRs (or we can already do so if (1) is ready). Then, we merge (2) but only if we are able to make it before (it depends on when (1) is done). Note that I'll be unavailable next week so if there is a release planned for the next 10 days, I would prefer skipping the introduction of those APIs and add them in the one after. In the meantime, we can work on (3) so that we have something to put in main.

@picnixz

Ok, so here's the plan for the future.

We have PRs for tuning the allocation limiter in pyexpat for 3.10 up to 3.14. main already has the feature. @hartwork, could you go through the PRs and update them accordingly? TiA.

@picnixz they are all ready to review (and merge) as far as I am concerned. Previous workarounds for issue #139400 have been been dropped from all of them.

Next, we should create backports for tuning the billion laughs protection in pyexpat for 3.10 up to 3.14. main already has the feature. For that, I'm going to handle the backports after the PRs above are done.

Yes.

Finally, we should design an API for tuning those protections from etree parsers as it was done for .flush(). I don't know if we need to do it for SAX parsers though as I now don't remember if parser deferral couold be tuned on SAX parsers (maybe it can because of inheriting interfaces).

Yes.

What I want is that, at any point of the time, security branches and release branches have the same APIs. As discused, 3.9 won't have those APIs and will live with the defaults. I don't know when 3.10/3.11 (@pablogsal) and 3.12 (@Yhg1s) will have their security releases but if it's in the next days, I'm going to skip those releases and wait for the next one. Then we can merge the first batch of PRs (or we can already do so if (1) is ready). Then, we merge (2) but only if we are able to make it before (it depends on when (1) is done). Note that I'll be unavailable next week so if there is a release planned for the next 10 days, I would prefer skipping the introduction of those APIs and add them in the one after. In the meantime, we can work on (3) so that we have something to put in main.

The new API was made with hasattr in mind, so I personally do not see value in synchronisation across backports. What is the value bought by these costs?

I agree that with hasattr checks we won't need to bother, but it's more for syncing on our side. But we can also try to avoid this. Now, however, I wonder whether we should also ~~not~~ indicate hasattr checks for the new Expat interface in the docs. Because, while we released 3.14.0, the new methods will only be present in 3.14.1, so we should also indicate this in the docs. What do you think?

@picnixz

Now, however, I wonder whether we should also not indicate hasattr checks for the new Expat interface in the docs.

@picnixz is the "not" by accident here ⬆️ and you meant to say the opposite?

Because, while we released 3.14.0, the new methods will only be present in 3.14.1, so we should also indicate this in the docs. What do you think?

Do we have any places mis-indicating they would have arrived in 3.14.0? I'm not aware of any but if we have some, let's make those say 3.14.1 and backports, yes. Past PR #116278 is related.

is the "not" by accident here ⬆️ and you meant to say the opposite?

Yes my bad. I meant "we should add this".

Do we have any places mis-indicating they would have arrived in 3.14.0?

I don't think so. Strictly speaking, with .. versionadded:: next, we should be fine because it will be marked ("added in version 3.14.1") in the online docs. But to stress this, it would be good if the attack section (of pyexpat) mentions the "hasattr" trick.

@picnixz

I meant "we should add this".

@picnixz alright 👍

But to stress this, it would be good if the attack section (of pyexpat) mentions the "hasattr" trick.

@picnixz the attack section will probably be overlooked by someone just reading the docs of the four specific founctions, so I would personally want to copy the approach from https://github.com/python/cpython/pull/116278/files#diff-a739ba3914e4e44366492e0305c5000673e34e958ed59e267ae0b678673bca6fR220 instead both for visibility and for symmetrie (or consistency). Let me demo a pull request for that variant, upcoming.
Also, #139460 would cause conflicts otherwise. I'm still hoping for review on that one from someone. I believe it's keeping things short enough while still giving some more guidance users.

gh-90949: Recommend `hasattr` with Expat security methods (#139800)

[3.14] gh-90949: add Expat API to prevent XML deadly allocations (CVE…

[3.13] gh-90949: add Expat API to prevent XML deadly allocations (CVE…

[3.10] gh-90949: add Expat API to prevent XML deadly allocations (CVE…

[3.11] gh-90949: add Expat API to prevent XML deadly allocations (CVE…

[3.12] gh-90949: add Expat API to prevent XML deadly allocations (CVE…

[3.14] pythongh-90949: expose Expat API to tune exponential expansion…

[3.14] gh-90949: expose Expat API to tune exponential expansion prote…

[3.13] gh-90949: expose Expat API to tune exponential expansion prote…

gh-90949: Fix copy-paste typo in pyexpat capsule API initialization (#…

[3.15] gh-90949: Fix copy-paste typo in pyexpat capsule API initializ…

[3.12] gh-90949: expose Expat API to tune exponential expansion prote…

[3.11] gh-90949: expose Expat API to tune exponential expansion prote…

[3.10] gh-90949: expose Expat API to tune exponential expansion prote…

Navigation Menu

Repository navigation

Expose Expat XML attack protection APIs #90949

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Expose Expat XML attack protection APIs #90949

Description

Linked PRs

Activity

hartwork commented on Sep 17, 2025

picnixz commented on Sep 17, 2025

hartwork commented on Sep 17, 2025

picnixz commented on Oct 8, 2025

hartwork commented on Oct 8, 2025

picnixz commented on Oct 8, 2025

hartwork commented on Oct 8, 2025

picnixz commented on Oct 8, 2025

hartwork commented on Oct 8, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions