Repository navigation
Remove old-deprecated ElementTree features (part 2) #80724
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2020-06-10.16:04:24.719>
created_at = <Date 2019-04-06.16:06:22.348>
labels = ['expert-XML', 'type-feature', '3.9']
title = 'Remove old-deprecated ElementTree features (part 2)'
updated_at = <Date 2021-01-05.17:46:41.989>
user = 'https://github.com/serhiy-storchaka'bugs.python.org fields:
activity = <Date 2021-01-05.17:46:41.989>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2020-06-10.16:04:24.719>
closer = 'scoder'
components = ['XML']
creation = <Date 2019-04-06.16:06:22.348>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 36543
keywords = ['patch']
message_count = 27.0
messages = ['339533', '339535', '350938', '364802', '365190', '368083', '368105', '368108', '368111', '368113', '368118', '368120', '368121', '368122', '368123', '368127', '368135', '371210', '371212', '371214', '384390', '384399', '384400', '384401', '384412', '384415', '384419']
nosy_count = 8.0
nosy_names = ['scoder', 'christian.heimes', 'eli.bendersky', 'petr.viktorin', 'serhiy.storchaka', 'hroncok', 'miss-islington', 'rahul-kumi']
pr_nums = ['12707', '18937', '19108', '19188', '19920', '19921', '20117', '20780']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue36543'
versions = ['Python 3.9']
Activity
serhiy-storchaka commentedon Apr 6, 2019
The proposed PR removes old-deprecated ElementTree features.
Methods Element.getchildren(), Element.getiterator() and ElementTree.getiterator() deprecated in 2.7 and 3.2. They were deprecated in the documentation only, and started to emit a warning in 3.8. Use list(elem) or iteration instead of getchildren(), methods iter() instead of getiterator().
The xml.etree.cElementTree module deprecated in 3.3. It was deprecated documentation only because adding a runtime warning will cause more harm than removing it because of the common idiom of using it since Python 2:
try:
import xml.etree.cElementTree as ET
except ImportError:
import xml.etree.ElementTree as ET
TODO: Add a What's New entry after the start of developing 3.9.
serhiy-storchaka commentedon Apr 6, 2019
See also bpo-29209.
serhiy-storchaka commentedon Sep 1, 2019
New changeset f02ea62 by Serhiy Storchaka in branch 'master':
bpo-36543: Remove old-deprecated ElementTree features. (GH-12707)
f02ea62
serhiy-storchaka commentedon Mar 22, 2020
New changeset b33e525 by Serhiy Storchaka in branch 'master':
bpo-36543: Remove the xml.etree.cElementTree module. (GH-19108)
b33e525
serhiy-storchaka commentedon Mar 27, 2020
New changeset 6467134 by Miro Hrončok in branch 'master':
bpo-36543: What's new: Document how to replace xml.etree.cElementTree (GH-19188)
6467134
tiran commentedon May 4, 2020
For the record, I'm quite unhappy that xml.etree.cElementTree was removed without going through a proper active deprecation cycle with plenty of head start. The removal came as a surprise to me -- and I'm a core dev, author of PEP-594 and owner of the defusedxml package. xml.etree.cElementTree module was also never added to PEP-4 and is still not mentioned as removed in PEP-4.
This changeset broke defusedxml, a package with over 10M downloads a month and 57k dependencies on Github. I'm sure that the removal is going to cause more trouble for other packages, too.
In PEP-594 I deliberately included a long deprecation phase for all packages with at least one release deprecation warnings. I gave users plenty of time to plan for removal.
scoder commentedon May 5, 2020
Christian, I understand your complaint, but I've actually never seen code in the wild that didn't provide a fallback for the import, usually something like what Serhiy spelled out and explained above:
This makes it inconvenient for users to emit a deprecation warning for the import, because it would impact perfectly future proof code like the above, without a good way for users to work around it by adapting their code, because the above import order is actually what they want. And there's a lot of such code.
I personally believe that the breakage will be quite limited. But that's a belief, not a fact. I don't have numbers to back it.
serhiy-storchaka commentedon May 5, 2020
xml.etree.cElementTree should be treated in the same way as cStringIO and cPickle -- they where separate modules in the past, but now the acceleration is used by default in io.StringIO and pickle. It looks an oversign that it was not removed in 3.0.
xml.etree.cElementTree was deprecated since 3.3. I think 6 releases (3.3-3.8) is long enough period. It was not possible to emit a deprecation warning at runtime for reasons mentioned above. And we did not have deprecation warnings for cStringIO and cPickle for the same reasons.
defusedxml is the only known to me package which does not fall back to xml.etree.ElementTree. I hope it is easy to fix it. In Python xml.etree.cElementTree is the same as xml.etree.ElementTree, so you can just import xml.etree.ElementTree. And you can use the above mentioned idiom if you still support Python 2.
hroncok commentedon May 5, 2020
Some stats: See https://bugzilla.redhat.com/showdependencytree.cgi?id=PYTHON39&hide_resolved=0 and Ctrl+F for cElementTree.
scoder commentedon May 5, 2020
Thanks Miro, that's beautiful. Good to know that I'm not the only one who doesn't read documentation. ;-)
So, how do we deal with this? We can't warn about the deprecation without annoying everyone. We can't just remove the empty module without breaking lots of code. We also can't really let the module stick around forever and let people write more code that uselessly imports it.
Any ideas?
6 remaining items
scoder commentedon May 5, 2020
¯\(ツ)/¯
Honestly, let's just keep it. Maybe we can add an invisible PendingDeprecationWarning and wait if some static analysers pick it up. I wouldn't want to invest any more work into it than that.
serhiy-storchaka commentedon May 5, 2020
I do not think that it is right to emit a warning at import time. We should not punish people which do things right: fallback to xml.etree.ElementTree and test with -We.
scoder commentedon Jun 10, 2020
New changeset ec88e1b by Serhiy Storchaka in branch 'master':
bpo-36543: Revert "bpo-36543: Remove the xml.etree.cElementTree module." (GH-20117)
ec88e1b
scoder commentedon Jun 10, 2020
New changeset 3b97d1b by Miss Islington (bot) in branch '3.9':
bpo-36543: Revert "bpo-36543: Remove the xml.etree.cElementTree module." (GH-20117) (GH-20780)
3b97d1b
scoder commentedon Jun 10, 2020
The xml.etree.cElementTree module is restored. We'll see about what to do with it later. It hurts less to keep it around than to delete it.
I think this ticket can be closed.
encukou commentedon Jan 5, 2021
Let me just point out there's an official process for making incompatible changes now: https://www.python.org/dev/peps/pep-0387/#making-incompatible-changes
A warning must be emitted (not just mentioned in docs) for two releases before a feature is removed.
serhiy-storchaka commentedon Jan 5, 2021
This process is not new. The problem is that in this particular case warning will do more harm than removing the module. If we cannot just remove it, we should not introduce warning. It will be kept forever.
encukou commentedon Jan 5, 2021
Should the process be changed?
tiran commentedon Jan 5, 2021
The process is fine. A deprecation warning will give users time to update code before it hits production.
serhiy-storchaka commentedon Jan 5, 2021
Deprecation warning may help users of old lower-quality code at the cost of harming users of old higher-quality code. It does not look right to me.
tiran commentedon Jan 5, 2021
Apparently my defusedxml package is low-quality code...
Anyhow PEP-387 has clear rules how to handle incompatible changes. Every core dev has to follow these rules. You cannot simply ignore the rule, because you don't like it. If you want to change the rules or get an exception for a special case, please open a discussion, lobby for an amendment to PEP-387, and get approval from the SC.
serhiy-storchaka commentedon Jan 5, 2021
I do not understand what this discussion in a long-closed article is about. If you propose to add warning, I am against it. You are against removing the module. Well, the module will not be removed, the warning is no longer needed. The issue is closed.
I do not propose to change the rule. I say that the procedure cannot be executed in this case, because it will cause more harm than benefit. If we cannot follow the procedure for removing module, we cannot remove that module. It is fine to me. If you want to change this, propose changes to the procedure, but it should be discussed not here.
Replace removed getchildren method