gh-149468: Add option to validate ElementTree during serialization by serhiy-storchaka · Pull Request #149469 · python/cpython

serhiy-storchaka · 2026-05-06T19:25:48Z

Issue: Add option to validate ElementTree during serialization #149468

serhiy-storchaka · 2026-05-06T19:30:37Z

This PR also fixes some bugs in serialization to HTML. I am going to extract them into separate issue before merging this PR. But for now, it is more convenient to review them together.

sethmlarson · 2026-05-06T20:30:50Z

@@ -0,0 +1,3 @@
+Add the *validate* option to :mod:`xml.etree.ElementTree` serialization
+functions, which allows to validate the element or element tree before


Suggestion on wording: "...which validates the element or element tree names and values only contain allowed/escaped characters prior to serialization"

* The content of comments, processing instructions and elements "xmp", "iframe", "noembed", "noframes", and "plaintext" is no longer escaped. * The "plaintext" element no longer have the closing tag. * Add support of empty attributes (with value None).

serhiy-storchaka · 2026-05-07T17:28:25Z

Rebased it onto #149490.

vstinner · 2026-05-13T11:58:37Z



+xml.etree.ElementTree
+---------------------


You should now move this entry to Doc/whatsnew/3.16.rst.

ezio-melotti · 2026-05-30T18:54:29Z

   generate a Unicode string (otherwise, a bytestring is generated).  *method*
   is either ``"xml"``, ``"html"`` or ``"text"`` (default is ``"xml"``).
-   *xml_declaration*, *default_namespace* and *short_empty_elements* has the same
+   *xml_declaration*, *default_namespace*, *validate* and *short_empty_elements* has the same


Suggested change

*xml_declaration*, *default_namespace*, *validate* and *short_empty_elements* has the same

*xml_declaration*, *default_namespace*, *validate* and *short_empty_elements* have the same

ezio-melotti · 2026-05-30T18:54:54Z

   generate a Unicode string (otherwise, a bytestring is generated).  *method*
   is either ``"xml"``, ``"html"`` or ``"text"`` (default is ``"xml"``).
-   *xml_declaration*, *default_namespace* and *short_empty_elements* has the same
+   *xml_declaration*, *default_namespace*, *validate* and *short_empty_elements* has the same


Suggested change

*xml_declaration*, *default_namespace*, *validate* and *short_empty_elements* has the same

*xml_declaration*, *default_namespace*, *validate* and *short_empty_elements* have the same

ezio-melotti · 2026-05-30T18:55:31Z

+   .. versionchanged:: next
+      Added the *validate* parameter.
+
   .. versionchanged:: 3.8


Suggested change

.. versionchanged:: next

Added the *validate* parameter.

.. versionchanged:: 3.8

.. versionchanged:: 3.8

This is duplicated below.

ezio-melotti · 2026-05-30T19:43:27Z

+      If *validate* is true, check that all characters are legal XML or HTML
+      characters, depending on *method*, element and attribute names are
+      valid, and the content of comments, processing instructions and
+      HTML elements like ``<script>`` do not contain illegal sequences,
+      and raise :exc:`ValueError` otherwise.
+      By default, no validation is performed.


This sentence is a bit of a mouthful, but it's a nice compromise between conciseness and clarity.
It could be improved with a couple of minor tweaks:

Suggested change

If *validate* is true, check that all characters are legal XML or HTML

characters, depending on *method*, element and attribute names are

valid, and the content of comments, processing instructions and

HTML elements like ``<script>`` do not contain illegal sequences,

and raise :exc:`ValueError` otherwise.

By default, no validation is performed.

If *validate* is true, check that all characters are legal XML or HTML

characters (depending on *method*), that element and attribute names are

valid, and that the content of comments, processing instructions and

HTML elements like ``<script>`` do not contain illegal sequences.

Raise :exc:`ValueError` if any check fails.

By default, or if *method* is ``"text"``, no validation is performed.

or (if the method applies to all checks):

Suggested change

If *validate* is true, check that all characters are legal XML or HTML

characters, depending on *method*, element and attribute names are

valid, and the content of comments, processing instructions and

HTML elements like ``<script>`` do not contain illegal sequences,

and raise :exc:`ValueError` otherwise.

By default, no validation is performed.

If *validate* is true, check that all characters are legal,

that element and attribute names are valid, and that the content

of comments, processing instructions and HTML elements

like ``<script>`` do not contain illegal sequences according

to the selected *method* (``"xml"`` or ``"html"``).

Raise :exc:`ValueError` if any check fails.

By default, or if *method* is ``"text"``, no validation is performed.

Another option is to list the checks:

Suggested change

If *validate* is true, check that all characters are legal XML or HTML

characters, depending on *method*, element and attribute names are

valid, and the content of comments, processing instructions and

HTML elements like ``<script>`` do not contain illegal sequences,

and raise :exc:`ValueError` otherwise.

By default, no validation is performed.

If *validate* is true and *method* is either ``"xml"`` or ``"html"``,

check that:

* all characters are legal XML or HTML characters

* element and attribute names are valid

* the content of comments, processing instructions and HTML elements

like ``<script>`` do not contain illegal sequences

and raise :exc:`ValueError` otherwise.

By default, or if *method* is ``"text"``, no validation is performed.

ezio-melotti · 2026-05-30T19:46:32Z

+        self.check(ET.Element(ET.QName('\ufffe', 'tag')))
+
+class HTMLValidationTest(unittest.TestCase):


Suggested change

self.check(ET.Element(ET.QName('\ufffe', 'tag')))

class HTMLValidationTest(unittest.TestCase):

self.check(ET.Element(ET.QName('\ufffe', 'tag')))

class HTMLValidationTest(unittest.TestCase):

ezio-melotti · 2026-05-30T19:50:06Z

+    def test_invalid_attr_value(self):
+        self.check(ET.Element('tag', attrib={'key': '\x00'}))
+        self.check(ET.Element('tag', attrib={'key': '\ud8ff'}))
+        self.check(ET.Element('tag', attrib={'key': '\ufffe'}))
+        self.check(ET.Element('tag', attrib={'key': ET.QName('\x00')}))
+        self.check(ET.Element('tag', attrib={'key': ET.QName('\ud8ff')}))
+        self.check(ET.Element('tag', attrib={'key': ET.QName('\ufffe')}))


This and several other methods could use subTests if you think it's an improvement, e.g.:

Suggested change

def test_invalid_attr_value(self):

self.check(ET.Element('tag', attrib={'key': '\x00'}))

self.check(ET.Element('tag', attrib={'key': '\ud8ff'}))

self.check(ET.Element('tag', attrib={'key': '\ufffe'}))

self.check(ET.Element('tag', attrib={'key': ET.QName('\x00')}))

self.check(ET.Element('tag', attrib={'key': ET.QName('\ud8ff')}))

self.check(ET.Element('tag', attrib={'key': ET.QName('\ufffe')}))

@support.subTests('value', ('\x00', '\ud8ff', '\ufffe'))

def test_invalid_attr_value(self, value):

self.check(ET.Element('tag', attrib={'key': value}))

self.check(ET.Element('tag', attrib={'key': ET.QName(value)}))

ezio-melotti · 2026-05-30T19:50:22Z

+        self.check(elem)
+
 class IterparseTest(unittest.TestCase):


Suggested change

self.check(elem)

class IterparseTest(unittest.TestCase):

self.check(elem)

class IterparseTest(unittest.TestCase):

ezio-melotti · 2026-05-30T19:53:39Z

        write("<!--%s-->" % text)
    elif tag is ProcessingInstruction:
+        if validate:
+            m = re.search('[ \t\r\n]', text)


Is there a test for this?

ezio-melotti · 2026-05-30T20:03:02Z

            if text:
+                if validate:
+                    if '\0' in text:
+                        raise ValueError('invalid characters')


Suggested change

raise ValueError('invalid characters')

raise ValueError('invalid character ("\\0")')

Similarly the other error messages could specify what is invalid.

serhiy-storchaka requested a review from AA-Turner as a code owner May 6, 2026 19:25

bedevere-app Bot mentioned this pull request May 6, 2026

Add option to validate ElementTree during serialization #149468

Open

bedevere-app Bot added the awaiting core review label May 6, 2026

sethmlarson reviewed May 6, 2026

View reviewed changes

merwok reviewed May 6, 2026

View reviewed changes

Comment thread Lib/xml/etree/ElementTree.py

serhiy-storchaka requested review from a team, FFY00, ZeroIntensity, corona10, emmatyping, encukou, ericsnowcurrently and erlend-aasland as code owners May 7, 2026 17:18

serhiy-storchaka added 2 commits May 7, 2026 20:21

pythongh-149468: Add option to validate ElementTree during serialization

a134c0b

serhiy-storchaka force-pushed the xml-etree-validate branch from 73e1b24 to a134c0b Compare May 7, 2026 17:26

vstinner reviewed May 13, 2026

View reviewed changes

ned-deily removed the request for review from a team May 16, 2026 04:55

Merge branch 'main' into xml-etree-validate

c81fe70

serhiy-storchaka force-pushed the xml-etree-validate branch from ca19970 to c81fe70 Compare May 29, 2026 21:42

ezio-melotti reviewed May 30, 2026

View reviewed changes

		@@ -0,0 +1,3 @@
		Add the validate option to :mod:`xml.etree.ElementTree` serialization
		functions, which allows to validate the element or element tree before

	xml_declaration, default_namespace, validate and short_empty_elements has the same
	xml_declaration, default_namespace, validate and short_empty_elements have the same

-      If *validate* is true, check that all characters are legal XML or HTML
-      characters, depending on *method*, element and attribute names are
-      valid, and the content of comments, processing instructions and
-      HTML elements like ``<script>`` do not contain illegal sequences,
-      and raise :exc:`ValueError` otherwise.
-      By default, no validation is performed.
+      If *validate* is true, check that all characters are legal,
+      that element and attribute names are valid, and that the content
+      of comments, processing instructions and HTML elements
+      like ``<script>`` do not contain illegal sequences according
+      to the selected *method* (``"xml"`` or ``"html"``).
+      Raise :exc:`ValueError` if any check fails.
+      By default, or if *method* is ``"text"``, no validation is performed.

		self.check(ET.Element(ET.QName('\ufffe', 'tag')))

		class HTMLValidationTest(unittest.TestCase):

	raise ValueError('invalid characters')
	raise ValueError('invalid character ("\\0")')



		xml.etree.ElementTree
		---------------------

Uh oh!

Conversation

serhiy-storchaka commented May 6, 2026 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

serhiy-storchaka commented May 6, 2026

Uh oh!

sethmlarson May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

serhiy-storchaka commented May 7, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

serhiy-storchaka commented May 6, 2026 •

edited by bedevere-app Bot

Loading

sethmlarson May 6, 2026 •

edited

Loading