How to Parse XML in Python: ElementTree, Namespaces, Large Files, lxml
Rewritten as a tested reference. Every example was executed by the example package against small fixture files and one generated 23.8 MB file; the output blocks are copied from that run. The 2024 version's performance numbers were not measured and are replaced by a measured table; its xml.etree.cElementTree recommendation (a deprecated alias since Python 3.3; xml.etree.ElementTree already uses the C accelerator), its codecs.open encoding advice (wrong; the equivalent open(..., encoding="utf-8") is shown failing below) and a stray "Meta Description" section are gone; namespaces, large-file streaming, writing, xmltodict, pandas.read_xml and the current security picture are new.
The standard library does this without installing anything. Given a file books.xml with a <catalog> of <book> elements:
import xml.etree.ElementTree as ET
root = ET.parse("fixtures/books.xml").getroot()
for book in root.findall("book"):
print(book.get("id"), book.find("title").text, book.findtext("price", default="n/a"))
bk101 XML Developer's Guide 44.95
bk102 Midnight Rain 5.95
bk103 Maeve Ascendant n/a
That covers most "read an XML file" tasks. The rest of this page is the things that go wrong next, each with the real output: elements you cannot find because of a namespace, a file too big for parse(), encoding and syntax errors, and when to reach for lxml, xmltodict or pandas instead.
Video Tutorial
Recorded for the 2024 version of this page; the code below supersedes it where they differ.
The fixtures, and how the outputs were captured
The package ships a small catalogue (books.xml: three books, attributes, a CDATA section, one book without a price), a sitemap.xml in the sitemaps namespace with an xhtml:link alternate, a Latin-1 file with its encoding declaration, a broken file, an XSD, and two attack files. The snippets assume import xml.etree.ElementTree as ET, from lxml import etree and import defusedxml.ElementTree, and the third-party packages from the package's requirements.txt (pip install lxml xmltodict untangle beautifulsoup4 defusedxml pandas). Sections 2, 4 and 10 show failures: there, each line is an expression the package evaluates through a small helper that prints the expression's value, or the exception's type and message instead of raising, so the script continues; pasted into a plain file, a failing line raises with the same text at the end of the traceback and stops. In sections 4 and 10 the lines that read like ParseError.position …, XMLSyntaxError.lineno … and the expat … | libxml2 … header are printed by short try/except and print statements in the scripts that the blocks do not repeat. Section 7 omits the package's small iterparse demo (it is shown on the large file in section 5). Everything else prints directly and is shown as run. Tested with Python 3.12.11 (expat 2.6.3), lxml 6.1.3 (libxml2 2.14.6), xmltodict 1.0.4, untangle 1.2.1, beautifulsoup4 4.15.0, defusedxml 0.7.1, pandas 3.0.5.
1. ElementTree: tags, attributes, text, paths
import xml.etree.ElementTree as ET
tree = ET.parse("fixtures/books.xml")
root = tree.getroot()
print(root.tag, root.attrib)
for book in root.findall("book"):
print(book.get("id"), book.find("title").text, book.findtext("price", default="n/a"))
first = root.find("book")
print(first.find("price").attrib, first.find("price").text)
print([t.text for t in root.iter("tag")]) # every <tag> anywhere below root
print(root.find("book[@id='bk103']/description").text) # a path with a predicate; CDATA comes back as text
print(root.find("book/isbn")) # missing element -> None, not an exception
data = open("fixtures/books.xml", "rb").read()
print(ET.fromstring(data).tag, ET.fromstring("<a><b>x</b></a>").find("b").text) # from bytes, from a str
catalog {'updated': '2026-09-16'}
bk101 XML Developer's Guide 44.95
bk102 Midnight Rain 5.95
bk103 Maeve Ascendant n/a
{'currency': 'USD'} 44.95
['xml', 'reference', 'fantasy']
Contains <b>markup</b> & ampersands
None
catalog x
The API is small: find returns the first match or None, findall a list, findtext the text with a default, iter walks the whole subtree, .get() reads an attribute, .text the content. Paths accept a subset of XPath (book[@id='bk103']/description), enough for most documents. ET.fromstring takes bytes or str, so ET.fromstring(response.content) is the whole story for an HTTP response.
2. Strings, bytes and encodings
Give the parser bytes and let it read the encoding declaration. Decoding the file yourself is where the classic mistakes live. The fixture is Latin-1 with <?xml version="1.0" encoding="ISO-8859-1"?>:
import xml.etree.ElementTree as ET
from lxml import etree
ET.parse("fixtures/latin1.xml").getroot().find("to").text # file -> bytes, declaration honoured
ET.fromstring(open("fixtures/latin1.xml", "rb").read()).find("body").text
ET.fromstring(open("fixtures/latin1.xml", encoding="utf-8").read()).find("to").text # decoded as the wrong encoding
ET.fromstring(open("fixtures/latin1.xml", encoding="latin-1").read()).find("to").text # decoded as the right one
etree.fromstring(open("fixtures/latin1.xml", encoding="latin-1").read()).find("to").text # lxml, str with a declaration
etree.fromstring(open("fixtures/latin1.xml", "rb").read()).find("to").text # lxml, bytes
Jürgen
Grüße aus München
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 55: invalid start byte
Jürgen
ValueError: Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration.
Jürgen
Two rules fall out of this. Open XML files in binary mode ("rb") or pass the filename, never open(..., encoding=...); the declaration inside the file is the parser's job. And lxml refuses a str that still carries a declaration, which is exactly what response.text gives you, so pass response.content to lxml and ElementTree alike.
3. Namespaces: why findall("url") returns nothing
Sitemaps, Atom, RSS extensions, SOAP, office formats: much of the XML you meet on the web is namespaced, and a namespaced tag's real name is {uri}local. The sitemap fixture:
root = ET.parse("fixtures/sitemap.xml").getroot()
print("root.tag:", root.tag)
print("findall('url'):", root.findall("url"))
NS = "{http://www.sitemaps.org/schemas/sitemap/0.9}"
print("1 braces:", [u.find(NS + "loc").text for u in root.findall(NS + "url")])
ns = {"sm": "http://www.sitemaps.org/schemas/sitemap/0.9", "xhtml": "http://www.w3.org/1999/xhtml"}
print("2 prefix map:", [u.findtext("sm:loc", namespaces=ns) for u in root.findall("sm:url", ns)])
print("3 wildcard:", [u.findtext("{*}loc") for u in root.findall("{*}url")])
for u in root.findall("sm:url", ns):
alt = u.find("xhtml:link", ns)
print(u.findtext("sm:loc", namespaces=ns), u.findtext("sm:lastmod", namespaces=ns), alt.get("hreflang") if alt is not None else None)
print(ET.tostring(root.find("sm:url", ns), encoding="unicode").strip().splitlines()[0]) # before registering prefixes
ET.register_namespace("", "http://www.sitemaps.org/schemas/sitemap/0.9")
ET.register_namespace("xhtml", "http://www.w3.org/1999/xhtml")
print(ET.tostring(root.find("sm:url", ns), encoding="unicode").strip().splitlines()[0]) # after
root.tag: {http://www.sitemaps.org/schemas/sitemap/0.9}urlset
findall('url'): []
1 braces: ['https://example.com/', 'https://example.com/blog/python-parse-xml', 'https://example.com/pricing']
2 prefix map: ['https://example.com/', 'https://example.com/blog/python-parse-xml', 'https://example.com/pricing']
3 wildcard: ['https://example.com/', 'https://example.com/blog/python-parse-xml', 'https://example.com/pricing']
https://example.com/ 2026-09-01 de
https://example.com/blog/python-parse-xml 2026-09-16 None
https://example.com/pricing 2026-08-20 None
<ns0:url xmlns:html="http://www.w3.org/1999/xhtml" xmlns:ns0="http://www.sitemaps.org/schemas/sitemap/0.9">
<url xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml">
Three working forms: the {uri} prefix on every tag, a prefix map passed as namespaces= (the prefixes are yours, they do not have to match the document's), or the {*} wildcard (Python 3.8+) when you do not care which namespace. The prefix map is the one that scales, because the same dictionary serves find, findall, findtext and, in lxml, xpath. register_namespace only affects output: without it, serialised elements get generated ns0: prefixes, as the second-to-last line shows.
4. When parsing fails
import xml.etree.ElementTree as ET
from lxml import etree
ET.parse("fixtures/broken.xml")
issubclass(ET.ParseError, SyntaxError)
etree.parse("fixtures/broken.xml")
etree.tostring(etree.parse("fixtures/broken.xml", etree.XMLParser(recover=True)).getroot()).decode() # lxml can carry on
ET.parse("fixtures/missing.xml")
xml.etree.ElementTree.ParseError: mismatched tag: line 5, column 4
issubclass(ET.ParseError, SyntaxError): True
ParseError.position (line, column): (5, 4)
lxml.etree.XMLSyntaxError: Opening and ending tag mismatch: title line 4 and book, line 5, column 10 (broken.xml, line 5)
XMLSyntaxError.lineno: 5 error_log entries: 4
recover=True: <catalog>
<book id="bk101">
<title>Unclosed title
</title>
</book>
</catalog>
FileNotFoundError: [Errno 2] No such file or directory: 'fixtures/missing.xml'
Catch ET.ParseError (a SyntaxError subclass) and read .position; with lxml, XMLSyntaxError carries .lineno and an error_log with every problem libxml2 saw. recover=True makes lxml close the unclosed tag and continue, which is right for scraping a feed someone hand-edited and wrong for anything where the structure matters. A missing file is a plain FileNotFoundError, not a parse error.
5. Large files: iterparse and the measured cost of each approach
parse() builds the whole tree in memory. For files of hundreds of megabytes, stream instead: iterparse yields elements as their end tags arrive, and you clear each one after using it. The standard library version:
n = 0
for event, elem in ET.iterparse("generated/large.xml"):
if elem.tag == "record":
n += elem.get("region") == "us"
elem.clear() # children were already parsed; clearing the record drops them too
and the lxml version, which also deletes the already-processed siblings (the loop from the lxml documentation's parsing tutorial, which explains it as necessary because filtering with tag= skips over elements that would otherwise stay in the tree):
from lxml import etree
n = 0
for event, elem in etree.iterparse("generated/large.xml", tag="record"):
n += elem.get("region") == "us"
elem.clear()
while elem.getprevious() is not None:
del elem.getparent()[0]
The package generates a 23.8 MB file of 200,000 <record> elements and runs the same task, counting records with region="us", once per library, each in its own process so the peak memory is per approach:
file: generated/large.xml, 23.8 MB, 200,000 <record> elements; task: count records with region="us"; one subprocess per approach
approach seconds peak RSS MB count
ET.parse + findall 0.67 254 66667
ET.iterparse + clear 0.76 34 66667
ET.iterparse + clear + root.clear 0.80 17 66667
lxml.parse + xpath 0.35 370 66667
lxml.iterparse + clear 0.29 22 66667
xml.sax handler 0.74 23 66667
xml.dom.minidom 3.18 751 66667
xmltodict.parse 1.55 215 66667
untangle.parse 3.55 386 66667
BeautifulSoup(features='xml') 5.10 872 66667
Measured on an Apple Silicon Mac; your absolute numbers will differ, the ratios are the point. A full tree costs 10 to 16 times the file size in memory (ET.parse 254 MB, lxml 370 MB on a 23.8 MB file) and minidom about thirty times; the streaming approaches sit at 17 to 34 MB; lxml is the fastest either way; BeautifulSoup and untangle are the slowest, and BeautifulSoup and minidom the largest. xmltodict also has a streaming mode (item_depth with item_callback) that the table does not exercise.
One row needs explaining. ET.iterparse + clear uses 34 MB where lxml and sax use 22 to 23, because clear() empties each record but leaves the empty element attached to the root, so the root grows by one stub per record. The third row fixes that by also clearing the root as it goes:
n = 0
it = ET.iterparse("generated/large.xml", events=("start", "end"))
_, root = next(it) # the first event is the root's start tag
for event, elem in it:
if event == "end" and elem.tag == "record":
n += elem.get("region") == "us"
elem.clear()
root.clear() # drop the stubs of the records already seen
The package then doubles the file to check that this is real and not a rounding artefact:
file: generated/large2.xml, 47.7 MB, 400,000 <record> elements; task: count records with region="us"; one subprocess per approach
approach seconds peak RSS MB count
ET.parse + findall 1.31 491 133334
ET.iterparse + clear 1.60 53 133334
ET.iterparse + clear + root.clear 1.59 17 133334
lxml.parse + xpath 0.62 718 133334
lxml.iterparse + clear 0.57 22 133334
xml.sax handler 1.48 23 133334
The tree approaches double with the file (254 → 491 MB, 370 → 718 MB); ET.iterparse with only elem.clear() grows (34 → 53 MB); the root-clearing version, lxml.iterparse with the sibling loop, and sax do not move (17, 22, 23 MB). That is the property you want from streaming.
6. Writing and modifying
tree = ET.parse("fixtures/books.xml"); root = tree.getroot()
book = root.find("book[@id='bk103']")
book.set("lang", "en"); book.find("title").text = "Maeve Ascendant (2nd ed.)"
price = ET.SubElement(book, "price", currency="EUR"); price.text = "9.99"
root.remove(root.find("book[@id='bk102']"))
ET.indent(root, space=" ") # Python 3.9+
tree.write("generated/books-modified.xml", encoding="utf-8", xml_declaration=True)
new = ET.Element("urlset", xmlns="http://www.sitemaps.org/schemas/sitemap/0.9")
for loc in ("https://example.com/", "https://example.com/a"):
u = ET.SubElement(new, "url"); ET.SubElement(u, "loc").text = loc
ET.indent(new)
print(ET.tostring(new, encoding="unicode"))
The written file (the CDATA section came back out as escaped text, which is equivalent XML) and the built sitemap:
<?xml version='1.0' encoding='utf-8'?>
<catalog updated="2026-09-16">
<book id="bk101" lang="en">
<title>XML Developer's Guide</title>
<author>Gambardella, Matthew</author>
<price currency="USD">44.95</price>
<tags>
<tag>xml</tag>
<tag>reference</tag>
</tags>
</book>
<book id="bk103" lang="en">
<title>Maeve Ascendant (2nd ed.)</title>
<author>Corets, Eva</author>
<description>Contains <b>markup</b> & ampersands</description>
<tags />
<price currency="EUR">9.99</price>
</book>
</catalog>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://example.com/</loc>
</url>
<url>
<loc>https://example.com/a</loc>
</url>
</urlset>
7. lxml: full XPath, schema validation, source lines
pip install lxml. Same ElementTree API plus real XPath 1.0, XSD validation and speed:
from lxml import etree
tree = etree.parse("fixtures/books.xml")
print("xpath titles:", tree.xpath("//book[@lang='en']/title/text()"))
print("xpath count:", tree.xpath("count(//book)"), "sum prices:", tree.xpath("sum(//price)"))
print("xpath attr:", tree.xpath("//book[price > 10]/@id"))
print("sourceline of bk102:", tree.xpath("//book[@id='bk102']")[0].sourceline)
sm = etree.parse("fixtures/sitemap.xml")
ns = {"sm": "http://www.sitemaps.org/schemas/sitemap/0.9", "xhtml": "http://www.w3.org/1999/xhtml"}
print("sitemap locs:", sm.xpath("//sm:url/sm:loc/text()", namespaces=ns))
print("alternates:", sm.xpath("//xhtml:link/@href", namespaces=ns))
schema = etree.XMLSchema(etree.parse("fixtures/books.xsd"))
print("books.xml valid:", schema.validate(etree.parse("fixtures/books.xml")))
bad = etree.parse("fixtures/books-invalid.xml")
print("books-invalid.xml valid:", schema.validate(bad))
print("first error:", schema.error_log.filter_from_errors()[0].message)
try:
etree.parse("fixtures/books-invalid.xml", etree.XMLParser(schema=schema)) # validate while parsing
except etree.XMLSyntaxError as e:
print("XMLParser(schema=...):", type(e).__name__, str(e).rsplit(" (", 1)[0])
print(etree.tostring(tree.xpath("//book[@id='bk102']")[0], pretty_print=True).decode().strip())
xpath titles: ["XML Developer's Guide", 'Midnight Rain']
xpath count: 3.0 sum prices: 50.900000000000006
xpath attr: ['bk101']
sourceline of bk102: 10
sitemap locs: ['https://example.com/', 'https://example.com/blog/python-parse-xml', 'https://example.com/pricing']
alternates: ['https://example.com/de/']
books.xml valid: True
books-invalid.xml valid: False
first error: Element 'price': This element is not expected. Expected is ( author ).
XMLParser(schema=...): XMLSyntaxError Element 'price': This element is not expected. Expected is ( author ).
<book id="bk102" lang="en">
<title>Midnight Rain</title>
<author>Ralls, Kim</author>
<price currency="USD">5.95</price>
<tags><tag>fantasy</tag></tags>
</book>
XPath functions (count, sum, comparisons on text), attribute selection with /@id, and namespaces through the same prefix map. sourceline tells you where an element came from, useful when validating other people's exports. Note that xpath() returns floats for numeric functions and that sum on decimal strings shows binary floating point; convert with Decimal if you are summing prices for real.
8. The other standard-library parsers
minidom, sax and the raw expat binding are still there. minidom gives a W3C DOM (getElementsByTagName, firstChild.data), sax and expat are event callbacks with no tree at all:
from xml.dom import minidom
import xml.sax, xml.parsers.expat
doc = minidom.parse("fixtures/books.xml")
titles = doc.getElementsByTagName("title")
print("minidom:", len(titles), [t.firstChild.data for t in titles])
print(doc.getElementsByTagName("book")[0].getAttribute("id"), doc.documentElement.tagName)
class Handler(xml.sax.ContentHandler):
def __init__(self): self.count = {}
def startElement(self, name, attrs): self.count[name] = self.count.get(name, 0) + 1
xml_handler = Handler(); xml.sax.parse("fixtures/books.xml", xml_handler)
print("sax element counts:", xml_handler.count)
p = xml.parsers.expat.ParserCreate(); seen = []
p.StartElementHandler = lambda name, attrs: seen.append((name, attrs.get("id")) if name == "book" else None)
with open("fixtures/books.xml", "rb") as f: p.ParseFile(f)
print("expat books:", [s for s in seen if s])
minidom: 3 ["XML Developer's Guide", 'Midnight Rain', 'Maeve Ascendant']
bk101 catalog
sax element counts: {'catalog': 1, 'book': 3, 'title': 3, 'author': 3, 'price': 2, 'tags': 3, 'tag': 3, 'description': 1}
expat books: [('book', 'bk101'), ('book', 'bk102'), ('book', 'bk103')]
In the table above sax matches iterparse on memory and ElementTree on time, at the cost of writing a handler class; minidom is the slowest and largest of the standard options. Use minidom only when you need DOM semantics, sax/expat when you already have callback-shaped code; otherwise ElementTree and iterparse cover both cases.
9. A dict, an object or a table instead of a tree
import xmltodict, untangle, pandas as pd
from bs4 import BeautifulSoup
d = xmltodict.parse(open("fixtures/books.xml", "rb").read())
print("xmltodict keys:", list(d["catalog"].keys()), "first title:", d["catalog"]["book"][0]["title"])
print("xmltodict price:", d["catalog"]["book"][0]["price"])
print("xmltodict unparse:", xmltodict.unparse({"root": {"item": ["a", "b"]}}, pretty=False))
print("xmltodict sitemap:", [u["loc"] for u in xmltodict.parse(open("fixtures/sitemap.xml", "rb").read())["urlset"]["url"]])
o = untangle.parse("fixtures/books.xml")
print("untangle:", o.catalog["updated"], o.catalog.book[0]["id"], o.catalog.book[0].title.cdata, o.catalog.book[1].price["currency"])
soup = BeautifulSoup(open("fixtures/sitemap.xml", "rb").read(), features="xml")
print("bs4 xml:", [loc.text for loc in soup.find_all("loc")], "alternates:", [l["href"] for l in soup.find_all("link")])
print("bs4 recover broken:", BeautifulSoup(open("fixtures/broken.xml", "rb").read(), features="xml").find("title").text.strip())
df = pd.read_xml("fixtures/books.xml", xpath="//book")
print("pandas columns:", list(df.columns)); print(df[["id", "lang", "title", "price"]].to_string(index=False))
print("pandas sitemap:", pd.read_xml("fixtures/sitemap.xml", xpath="//sm:url", namespaces={"sm": "http://www.sitemaps.org/schemas/sitemap/0.9"})["loc"].tolist())
xmltodict keys: ['@updated', 'book'] first title: XML Developer's Guide
xmltodict price: {'@currency': 'USD', '#text': '44.95'}
xmltodict unparse: <?xml version="1.0" encoding="utf-8"?>
<root><item>a</item><item>b</item></root>
xmltodict sitemap: ['https://example.com/', 'https://example.com/blog/python-parse-xml', 'https://example.com/pricing']
untangle: 2026-09-16 bk101 XML Developer's Guide USD
bs4 xml: ['https://example.com/', 'https://example.com/blog/python-parse-xml', 'https://example.com/pricing'] alternates: ['https://example.com/de/']
bs4 recover broken: Unclosed title
pandas columns: ['id', 'lang', 'title', 'author', 'price', 'tags', 'description']
id lang title price
bk101 en XML Developer's Guide 44.95
bk102 en Midnight Rain 5.95
bk103 de Maeve Ascendant NaN
pandas sitemap: ['https://example.com/', 'https://example.com/blog/python-parse-xml', 'https://example.com/pricing']
xmltodict turns the document into nested dicts (attributes as @name, mixed content as #text) and unparse goes the other way; it is the quickest route to JSON. It does no namespace processing by default: a default namespace disappears from the keys, as in the sitemap, while prefixed tags keep their prefix; pass process_namespaces=True to expand them. untangle gives attribute-style access and .cdata for text. BeautifulSoup(features="xml") uses lxml underneath, ignores namespaces in find_all, and tolerates broken markup; it builds its own Python object tree on top of the parse, which is where its position at the bottom of the memory table comes from. pandas.read_xml (an XPath and a namespace map, lxml by default) is the shortest path from repeated elements to a DataFrame, with NaN for the missing price. One caveat with xmltodict: a single <book> would come back as a dict, not a one-item list; pass force_list=("book",) when the count can vary.
10. Untrusted XML: what the parsers do today
The 2024 version said the standard library is "vulnerable by default". The Python documentation is more precise, and has moved: the 3.12 page carries a table whose footnote says expat 2.4.1 and newer is not vulnerable to billion laughs or quadratic blowup, with external entity expansion marked safe for every module; the current page (3.14) drops the table and says that "Expat versions lower than 2.7.2 may be vulnerable to the “billion laughs”, “quadratic blowup” and “large tokens” vulnerabilities, or to disproportional use of dynamic memory". The Python tested here reports expat 2.6.3 (a system-provided build, exactly the case the documentation warns about), below that newer threshold, so what follows is what 2.6.3 actually did with the two attack files (the "large tokens" case is not exercised):
import xml.etree.ElementTree as ET
from lxml import etree
import defusedxml.ElementTree as DET
len(ET.parse("fixtures/billion-laughs.xml").getroot().text)
ET.parse("fixtures/xxe.xml").getroot().text # <!ENTITY leak SYSTEM "secret.txt">
len(etree.parse("fixtures/billion-laughs.xml").getroot().text) # lxml default parser
etree.parse("fixtures/xxe.xml").getroot().text
etree.parse("fixtures/xxe.xml", etree.XMLParser(resolve_entities=True)).getroot().text
etree.parse("fixtures/xxe.xml", etree.XMLParser(resolve_entities=False)).getroot().text
len(etree.parse("fixtures/billion-laughs.xml", etree.XMLParser(huge_tree=True)).getroot().text)
DET.parse("fixtures/billion-laughs.xml")
DET.parse("fixtures/xxe.xml")
DET.parse("fixtures/books.xml").getroot().tag
expat expat_2.6.3 | libxml2 2.14.6
xml.etree.ElementTree.ParseError: limit on input amplification factor (from DTD and entities) breached: line 14, column 6
xml.etree.ElementTree.ParseError: undefined entity &leak;: line 3, column 6
lxml.etree.XMLSyntaxError: Maximum entity amplification factor exceeded, see xmlCtxtSetMaxAmplification., line 1, column 7 (<string>, line 1)
lxml.etree.XMLSyntaxError: Entity 'leak' not defined, line 3, column 13 (xxe.xml, line 3)
this-is-the-secret-file-content
None
lxml.etree.XMLSyntaxError: Maximum entity amplification factor exceeded, see xmlCtxtSetMaxAmplification., line 1, column 7 (<string>, line 1)
defusedxml.common.EntitiesForbidden: EntitiesForbidden(name='lol', system_id=None, public_id=None)
defusedxml.common.EntitiesForbidden: EntitiesForbidden(name='leak', system_id='secret.txt', public_id=None)
catalog
On these versions ElementTree rejects both files outright and lxml's default parser (resolve_entities='internal' since lxml 5) rejects both too; the one line that leaks the local file is lxml with resolve_entities=True set explicitly, so never set it for input you did not write. huge_tree=True lifts libxml2's size limits but not the amplification limit. defusedxml (pip install defusedxml) still works and refuses any entity declaration by default, which is the conservative choice when you cannot control the expat or libxml2 your code will run on: Python may use a system-provided expat, and the documentation's threshold moved from 2.4.1 to 2.7.2 between releases. Check yours with pyexpat.EXPAT_VERSION and lxml.etree.LIBXML_VERSION.
Which one
| Need | Use |
|---|---|
| Read values from a normal-sized file, string or response | xml.etree.ElementTree |
| Namespaced document (sitemap, feed, SOAP) | ElementTree with a prefix map; {*} when the namespace does not matter |
| File larger than memory allows (from the table: 10–16× file size for a tree) | ET.iterparse or lxml.iterparse with clear(); xml.sax if you prefer callbacks |
Real XPath, XSD validation, recover=True, speed | lxml |
| Nested dicts or JSON | xmltodict (force_list for repeatable elements) |
| A DataFrame from repeated elements | pandas.read_xml |
| Input you do not control, on a Python or libxml2 you do not control | defusedxml, or current expat/libxml2 with defaults; never resolve_entities=True |
When ScrapingAnt is not needed, and when it is
Parsing needs no service: a local export, a feed or a sitemap your own IP can fetch is requests.get(url).content into any of the parsers above. Where the site refuses your requests, the fetch is the problem, not the parsing. Setting browser=false performs the request without a headless browser (no JavaScript rendering); the docs describe it for static content, and such a request costs 1 API credit through a datacenter proxy. Then parse the body with the same code. See the request and response format.
Limitations
- Timings and peak memory are from one machine and one file shape (flat records); a deeply nested document changes the ratios. The package prints the same table for your machine.
- The API was not called for an XML URL (no key in the run); the product paragraph stays within the documented parameters.
xml.dom.pulldomandxmltodict's streaming mode are not in the table.
Examples tested on 2026-09-16 with Python 3.12.11 (expat 2.6.3), lxml 6.1.3 (libxml2 2.14.6), xmltodict 1.0.4, untangle 1.2.1, beautifulsoup4 4.15.0, defusedxml 0.7.1, pandas 3.0.5. Code: scrapingant-examples/examples/python-parse-xml.
This article was drafted with AI assistance from a tested evidence packet and reviewed by the named author, who is responsible for the code, measurements and corrections.