Repository navigation
enum.IntFlag regression: missing values cause TypeError #88408
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/ethanfurman'
closed_at = <Date 2021-06-14.19:05:28.255>
created_at = <Date 2021-05-26.19:31:43.243>
labels = ['type-bug', 'library', '3.10', '3.11']
title = 'enum.IntFlag regression: missing values cause TypeError'
updated_at = <Date 2021-06-14.19:05:28.255>
user = 'https://github.com/hroncok'bugs.python.org fields:
activity = <Date 2021-06-14.19:05:28.255>
actor = 'ethan.furman'
assignee = 'ethan.furman'
closed = True
closed_date = <Date 2021-06-14.19:05:28.255>
closer = 'ethan.furman'
components = ['Library (Lib)']
creation = <Date 2021-05-26.19:31:43.243>
creator = 'hroncok'
dependencies = []
files = []
hgrepos = []
issue_num = 44242
keywords = ['patch']
message_count = 27.0
messages = ['394457', '394461', '394464', '394466', '394467', '394468', '394469', '394470', '394471', '394472', '394473', '394475', '394479', '394486', '394487', '394500', '394593', '395134', '395136', '395431', '395539', '395572', '395616', '395628', '395629', '395810', '395838']
nosy_count = 11.0
nosy_names = ['jbelmonte', 'ethan.furman', 'veky', 'tcaswell', 'hroncok', 'pablogsal', 'miss-islington', 'Manjusaka', 'John Belmonte', 'hauntsaninja', 'jacobtylerwalls']
pr_nums = ['26586', '26635', '26669', '26671']
priority = None
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44242'
versions = ['Python 3.10', 'Python 3.11']
Activity
hroncok commentedon May 26, 2021
With the change introduced in https://bugs.python.org/issue38250 7aaeb2a I observe a regression in behavior of enum.IntFlag with missing values.
Consider this code (from pyproj):
This is a valid code in Python 3.9, however produces a TypeError in Python 3.10.0b1:
Traceback (most recent call last): File "intflag.py", line 3, in <module> class GeodIntermediateFlag(IntFlag): File "/usr/lib64/python3.10/enum.py", line 544, in __new__ raise TypeError( TypeError: invalid Flag 'GeodIntermediateFlag' -- missing values: 4, 8, 32, 64, 128, 512, 1024, 2048Since I don't see this behavior mentioned in https://docs.python.org/3.10/library/enum.html or https://docs.python.org/3.10/whatsnew/3.10.html or https://docs.python.org/3.10/whatsnew/changelog.html I believe this is a regression.
pablogsal commentedon May 26, 2021
We are already blocked in Python 3.10 beta 2, Ethan, could you give a look at this so we can introduce the fix when we release the next beta?
ethanfurman commentedon May 26, 2021
That is an intentional change. The cause is that the masks include bits that are not named in the Flag.
The user-side fix is to add a
boundary=KEEPoption to the flag:The enum library fix could be one of two things:
automatically use the KEEP boundary when these conditions arise, and issue a DeprecationWarning; or
lose that particular check.
I'm inclined to go with option 2, since
boundaryis designed to answer the question of what to do when Flag.A | Flag.B does not exist in Flag.belm0 commentedon May 26, 2021
To clarify, it's caused by these mask entries in the enum:
Since the masks are not an aggregation of individual bits defined in the enum, it's an error.
I think that would cause various problems in the API and implementation. For example, without underlying individual bits, repr() may be nonsensical.
belm0 commentedon May 26, 2021
I wonder if CONFORM could tolerate these, since it's ultimately going to discard invalid bits. And then perhaps CONFORM is the default.
ethanfurman commentedon May 26, 2021
Actually, thinking about that a little bit more, KEEP was added for exactly this situation, as some stdlib flags exhibit the same behavior.
So the real question is what should happen with, for example,
?
The idea behind boundary is what should happen when values are created that don't have names in the Enum/Flag? The options for boundary are:
STRICT -> an error is raised (default for Enum)
EJECT -> the integer 0x80 is returned (not a flag)
CONFORM -> unnamed bits are discarded (so the DEFAULT flag would be returned)
KEEP -> an unnamed flag with value 0x80 is returned
So KEEP is currently doing double-duty -- this reinforces my desire to go with option 2 and return KEEP to single-duty status.
ethanfurman commentedon May 26, 2021
Those are good points -- the difficulty is knowing which behavior the user wants. And if the desired run-time behavior doesn't match the boundary flag the user is stuck.
ethanfurman commentedon May 26, 2021
For example, if the default is CONFORM or KEEP, but the user wants an error if 0x80 comes up, they would have to explicitly check for that value since the Flag would happily return it instead of raising.
belm0 commentedon May 26, 2021
Rather than make such masks containing unknown bits, this would be best practice if you want to use STRICT, correct?
Otherwise, if your input may have unknown bits, use CONFORM.
18 remaining items
ethanfurman commentedon Jun 11, 2021
Also changing error reporting to be less susceptible to DOS attacks.
ethanfurman commentedon Jun 11, 2021
New changeset c956734 by Ethan Furman in branch 'main':
bpo-44242: [Enum] improve error messages (GH-26669)
c956734
ethanfurman commentedon Jun 11, 2021
New changeset 0a186b1 by Miss Islington (bot) in branch '3.10':
bpo-44242: [Enum] improve error messages (GH-26669)
0a186b1
jacobtylerwalls commentedon Jun 14, 2021
With the followup patch merged, can this be closed now?
ethanfurman commentedon Jun 14, 2021
Yup, just had to get back from the weekend. :-)