Repository navigation
avoid counting all siblings for :first-child and :nth-*() bounds - #223
Merged
Merged
Conversation
:first-child, :last-child, :only-child, the -of-type variants and the bounds of :nth-*() compared count(preceding-sibling::*) (or following-sibling) with a constant. count() walks every sibling for every element it is evaluated on, so matching was quadratic in the number of siblings: li:first-child took about 3 s on a list of 32000 items. Test count >= k as ***-sibling::***[k] instead, which stops at the kth sibling. Only the modulo of :nth-*(an+b) with |a| > 1 still needs count().
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #223 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 3 3
Lines 1019 1022 +3
Branches 170 171 +1
=========================================
+ Hits 1019 1022 +3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
AdrianAtZyte
approved these changes
Nov 4, 2026
wRAR
approved these changes
Nov 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
:first-child, :last-child, :only-child, the -of-type variants and the bounds of :nth-*() are translated to
count(preceding-sibling::*)(orfollowing-sibling) compared with a constant, and count() walks every sibling for every element it's evaluated on, so matching is quadratic in the number of siblings. on lxml 6.1.3li:first-child,li:last-child,li:nth-child(2)andli:nth-child(-n+3)each take ~3.1 s of CPU on a 281 KiB page of 32000<li>(0.76 s at 16000) andli:only-of-type4.1 s, so a page with one long flat list is a cheap way to stall a scrapy/parsel spider that runs any of these (an algorithmic-complexity DoS where the attacker only has to serve the page). testingcount >= kas***-sibling::***[k]lets libxml2 stop at the kth sibling, e.g.:first-childbecomesnot(preceding-sibling::*[1]), and the same selectors take 4.5 to 11 ms on that page. only the modulo of:nth-*(an+b)with|a| > 1still needs count(), XPath 1.0 has no sibling index to use instead. old and new translations select the same elements on ~35k random selectors (every one of these forms, namespaced types, inside :not()/:has()/:is() and after combinators) and test_nth_child still passes;:nth-child(0)now short-circuits to[0]like-ndoes.