gh-90949: expose Expat API to tune exponential expansion protections

Doc/library/pyexpat.rst

@@ -238,16 +238,71 @@ XMLParser Objects
  .. versionadded:: 3.13
 
 
-:class:`!xmlparser` objects have the following methods to mitigate some
-common XML vulnerabilities.
+:class:`!xmlparser` objects have the following methods to tune protections
+against some common XML vulnerabilities.
+
+.. method:: xmlparser.SetBillionLaughsAttackProtectionActivationThreshold(threshold, /)
+
+ Sets the number of output bytes needed to activate protection against
+ `billion laughs`_ attacks.
+
+ The number of output bytes includes amplification from entity expansion
+ and reading DTD files.
+
+ Parser objects usually have a protection activation threshold of 8 MiB,
+ but the actual default value depends on the underlying Expat library.
+
+ An :exc:`ExpatError` is raised if this method is called on a
+ |xml-non-root-parser| parser.
+ The corresponding :attr:`~ExpatError.lineno` and :attr:`~ExpatError.offset`
+ should not be used as they may have no special meaning.
+
+ .. note::
+
+ Activation thresholds below 4 MiB are known to break support for DITA 1.3
+ payload and are hence not recommended.
+
+ .. versionadded:: next
+
+.. method:: xmlparser.SetBillionLaughsAttackProtectionMaximumAmplification(max_factor, /)
+
+ Sets the maximum tolerated amplification factor for protection against
+ `billion laughs`_ attacks.
+
+ The amplification factor is calculated as ``(direct + indirect) / direct``
+ while parsing, where ``direct`` is the number of bytes read from
+ the primary document in parsing and ``indirect`` is the number of
+ bytes added by expanding entities and reading of external DTD files.
+
+ The *max_factor* value must be a non-NaN :class:`float` value greater than
+ or equal to 1.0. Peak amplifications of factor 15,000 for the entire payload
+ and of factor 30,000 in the middle of parsing have been observed with small
+ benign files in practice. In particular, the activation threshold should be
+ carefully chosen to avoid false positives.
+
+ Parser objects usually have a maximum amplification factor of 100,
+ but the actual default value depends on the underlying Expat library.
+
+ An :exc:`ExpatError` is raised if this method is called on a
+ |xml-non-root-parser| parser or if *max_factor* is outside the valid range.
+ The corresponding :attr:`~ExpatError.lineno` and :attr:`~ExpatError.offset`
+ should not be used as they may have no special meaning.
+
+ .. note::
+
+ The maximum amplification factor is only considered if the threshold
+ that can be adjusted by :meth:`.SetBillionLaughsAttackProtectionActivationThreshold`
+ is exceeded.
+
+ .. versionadded:: next
 
 .. method:: xmlparser.SetAllocTrackerActivationThreshold(threshold, /)
 
  Sets the number of allocated bytes of dynamic memory needed to activate
  protection against disproportionate use of RAM.
 
-By default, parser objects have an allocation activation threshold of 64 MiB,
-or equivalently 67,108,864 bytes.
+Parser objects usually have an allocation activation threshold of 64 MiB,
+but the actual default value depends on the underlying Expat library.
 
  An :exc:`ExpatError` is raised if this method is called on a
  |xml-non-root-parser| parser.
@@ -271,7 +326,8 @@ common XML vulnerabilities.
  near the start of parsing even with benign files in practice. In particular,
  the activation threshold should be carefully chosen to avoid false positives.
 
- By default, parser objects have a maximum amplification factor of 100.0.
+ Parser objects usually have a maximum amplification factor of 100,
+ but the actual default value depends on the underlying Expat library.
 
  An :exc:`ExpatError` is raised if this method is called on a
  |xml-non-root-parser| parser or if *max_factor* is outside the valid range.
@@ -1010,4 +1066,6 @@ The ``errors`` module has the following attributes:
  not. See https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl
  and https://www.iana.org/assignments/character-sets/character-sets.xhtml.
 
+
+.. _billion laughs: https://en.wikipedia.org/wiki/Billion_laughs_attack
 .. |xml-non-root-parser| replace:: :ref:`non-root <xmlparser-non-root>`

Doc/whatsnew/3.15.rst

@@ -558,10 +558,18 @@ xml.parsers.expat
 
 * Add :meth:`~xml.parsers.expat.xmlparser.SetAllocTrackerActivationThreshold`
  and :meth:`~xml.parsers.expat.xmlparser.SetAllocTrackerMaximumAmplification`
- to :ref:`xmlparser <xmlparser-objects>` objects to prevent use of
- disproportional amounts of dynamic memory from within an Expat parser.
+ to :ref:`xmlparser <xmlparser-objects>` objects to tune protections against
+ disproportional amounts of dynamic memory usage from within an Expat parser.
  (Contributed by Bénédikt Tran in :gh:`90949`.)
 
+* Add :meth:`~xml.parsers.expat.xmlparser.SetBillionLaughsAttackProtectionActivationThreshold`
+ and :meth:`~xml.parsers.expat.xmlparser.SetBillionLaughsAttackProtectionMaximumAmplification`
+ to :ref:`xmlparser <xmlparser-objects>` objects to tune protections against
+ `billion laughs`_ attacks.
+ (Contributed by Bénédikt Tran in :gh:`90949`.)
+
+ .. _billion laughs: https://en.wikipedia.org/wiki/Billion_laughs_attack
+
 
 zlib
 ----

Include/pyexpat.h

@@ -57,6 +57,11 @@ struct PyExpat_CAPI
  XML_Parser parser, unsigned long long activationThresholdBytes);
  XML_Bool (*SetAllocTrackerMaximumAmplification)(
  XML_Parser parser, float maxAmplificationFactor);
+ /* might be NULL for expat < 2.4.0 */
+ XML_Bool (*SetBillionLaughsAttackProtectionActivationThreshold)(
+ XML_Parser parser, unsigned long long activationThresholdBytes);
+ XML_Bool (*SetBillionLaughsAttackProtectionMaximumAmplification)(
+ XML_Parser parser, float maxAmplificationFactor);
  /* always add new stuff to the end! */
 };
 

Lib/test/test_pyexpat.py

@@ -958,6 +958,64 @@ def test_set_maximum_amplification__fail_for_subparser(self):
  self.assert_root_parser_failure(setter, 123.45)
 
 
+@unittest.skipIf(expat.version_info < (2, 4, 0), "requires Expat >= 2.4.0")
+class ExpansionProtectionTest(AttackProtectionTestBase, unittest.TestCase):
+
+ def assert_rejected(self, func, /, *args, **kwargs):
+ """Check that func(*args, **kwargs) hits the allocation limit."""
+ msg = (
+ r"limit on input amplification factor \(from DTD and entities\) "
+ r"breached: line \d+, column \d+"
+ )
+ self.assertRaisesRegex(expat.ExpatError, msg, func, *args, **kwargs)
+
+ def set_activation_threshold(self, parser, threshold):
+ return parser.SetBillionLaughsAttackProtectionActivationThreshold(threshold)
+
+ def set_maximum_amplification(self, parser, max_factor):
+ return parser.SetBillionLaughsAttackProtectionMaximumAmplification(max_factor)
+
+ def test_set_activation_threshold__threshold_reached(self):
+ parser = expat.ParserCreate()
+ # Choose a threshold expected to be always reached.
+ self.set_activation_threshold(parser, 3)
+ # Check that the threshold is reached by choosing a small factor
+ # and a payload whose peak amplification factor exceeds it.
+ self.assertIsNone(self.set_maximum_amplification(parser, 1.0))
+ payload = self.exponential_expansion_payload(ncols=10, nrows=4)
+ self.assert_rejected(parser.Parse, payload, True)
+
+ def test_set_activation_threshold__threshold_not_reached(self):
+ parser = expat.ParserCreate()
+ # Choose a threshold expected to be never reached.
+ self.set_activation_threshold(parser, pow(10, 5))
+ # Check that the threshold is reached by choosing a small factor
+ # and a payload whose peak amplification factor exceeds it.
+ self.assertIsNone(self.set_maximum_amplification(parser, 1.0))
+ payload = self.exponential_expansion_payload(ncols=10, nrows=4)
+ self.assertIsNotNone(parser.Parse(payload, True))
+
+ def test_set_maximum_amplification__amplification_exceeded(self):
+ parser = expat.ParserCreate()
+ # Unconditionally enable maximum activation factor.
+ self.set_activation_threshold(parser, 0)
+ # Choose a max amplification factor expected to always be exceeded.
+ self.assertIsNone(self.set_maximum_amplification(parser, 1.0))
+ # Craft a payload for which the peak amplification factor is > 1.0.
+ payload = self.exponential_expansion_payload(ncols=1, nrows=2)
+ self.assert_rejected(parser.Parse, payload, True)
+
+ def test_set_maximum_amplification__amplification_not_exceeded(self):
+ parser = expat.ParserCreate()
+ # Unconditionally enable maximum activation factor.
+ self.set_activation_threshold(parser, 0)
+ # Choose a max amplification factor expected to never be exceeded.
+ self.assertIsNone(self.set_maximum_amplification(parser, 1e4))
+ # Craft a payload for which the peak amplification factor is < 1e4.
+ payload = self.exponential_expansion_payload(ncols=1, nrows=2)
+ self.assertIsNotNone(parser.Parse(payload, True))
+
+
 @unittest.skipIf(expat.version_info < (2, 7, 2), "requires Expat >= 2.7.2")
 class MemoryProtectionTest(AttackProtectionTestBase, unittest.TestCase):
 

Misc/NEWS.d/next/Library/2025-09-22-14-40-11.gh-issue-90949.UM35nb.rst

@@ -1,5 +1,5 @@
 Add :meth:`~xml.parsers.expat.xmlparser.SetAllocTrackerActivationThreshold`
 and :meth:`~xml.parsers.expat.xmlparser.SetAllocTrackerMaximumAmplification`
-to :ref:`xmlparser <xmlparser-objects>` objects to prevent use of
-disproportional amounts of dynamic memory from within an Expat parser.
+to :ref:`xmlparser <xmlparser-objects>` objects to tune protections against
+disproportional amounts of dynamic memory usage from within an Expat parser.
 Patch by Bénédikt Tran.

Misc/NEWS.d/next/Library/2025-09-26-18-04-28.gh-issue-90949.YHjSzX.rst

@@ -0,0 +1,7 @@
+Add
+:meth:`~xml.parsers.expat.xmlparser.SetBillionLaughsAttackProtectionActivationThreshold`
+and
+:meth:`~xml.parsers.expat.xmlparser.SetBillionLaughsAttackProtectionMaximumAmplification`
+to :ref:`xmlparser <xmlparser-objects>` objects to tune protections against
+`billion laughs <https://en.wikipedia.org/wiki/Billion_laughs_attack>`_ attacks.
+Patch by Bénédikt Tran.