Repository navigation
Unicode-mangled names refer inconsistently to constants #90713
Description
| BPO | 46555 |
|---|---|
| Nosy | @cfbolz, @serhiy-storchaka, @eryksun, @Kodiologist, @jack1142, @snoopjedi |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2022-01-27.21:57:22.495>
labels = ['type-bug', '3.9']
title = 'Unicode-mangled names refer inconsistently to constants'
updated_at = <Date 2022-02-01.00:41:18.436>
user = 'https://github.com/Kodiologist'bugs.python.org fields:
activity = <Date 2022-02-01.00:41:18.436>
actor = 'SnoopJeDi'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2022-01-27.21:57:22.495>
creator = 'Kodiologist'
dependencies = []
files = []
hgrepos = []
issue_num = 46555
keywords = []
message_count = 8.0
messages = ['411930', '412070', '412071', '412150', '412167', '412169', '412170', '412226']
nosy_count = 6.0
nosy_names = ['Carl.Friedrich.Bolz', 'serhiy.storchaka', 'eryksun', 'Kodiologist', 'jack1142', 'SnoopJeDi']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue46555'
versions = ['Python 3.9']
Activity
Kodiologist commentedon Jan 27, 2022
I'm not sure if this is a bug, but it certainly surprised me. Most reserved words, when Unicode-mangled, as in "𝕕𝕖𝕗", act like ordinary identifiers (see e.g. bpo-46520).
True,False, andNoneare weird in that Unicode-mangled versions of them refer to those same constants initially, but can take on their own identity as variables if assigned to:Python 3.9.7 (default, Sep 10 2021, 14:59:43) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> 𝕋𝕣𝕦𝕖 True >>> True = 0 File "<stdin>", line 1 True = 0 ^ SyntaxError: cannot assign to True >>> 𝕋𝕣𝕦𝕖 = 0 >>> True True >>> 𝕋𝕣𝕦𝕖 0I think that
𝕋𝕣𝕦𝕖 = 1should probably be forbidden. The fact that𝕋𝕣𝕦𝕖doesn't always mean the same thing asTrueseems to break the rule in PEP-3131 that "comparison of identifiers is based on NFKC".cfbolz commentedon Jan 29, 2022
hah, this is "great":
The problem is that the lexer assumes that anything that is not ASCII cannot be a keyword and lexes 𝕋𝕣𝕦𝕖 as an identifier.
serhiy-storchaka commentedon Jan 29, 2022
True is a keyword which is compiled to expression whose value is True, 𝕋𝕣𝕦𝕖 is an identifier which refers to the builtin variable "True" which has a value True by default. You can change the value of a builtin variable, but the value of expression True is always True.
I do not see a problem here. Don't use 𝕋𝕣𝕦𝕖 if your intention is not using a variable.
Kodiologist commentedon Jan 30, 2022
Is the existence of this entity, as separate from the constant
True, documented anywhere? constants.rst doesn't seem to acknowledge it. Indeed, is its existence a feature, or is it a CPython quirk?serhiy-storchaka commentedon Jan 30, 2022
https://docs.python.org/3/library/constants.html#built-in-constants
cfbolz commentedon Jan 30, 2022
Ok, I can definitely agree with Serhiy pov: "True" is a keyword that always evaluates to the object that you get when you call bool(1). There is usually no name "True" and directly assigning to it is forbidden. But there are various other ways to assign a name "True". One is eg globals("True") = 5, another one (discussed in this issue) is using identifiers that NFKC-normalize to the string "True".
eryksun commentedon Jan 30, 2022
Why was it decided to not raise a syntax error when the NFKC normalization of a non-ASCII token matches a keyword? I don't see a use for cases such as
𝕚𝕗 = 1and𝕚𝕗 + 1. It seems the cost in terms of confusion far outweighs any potential benefit.snoopjedi commentedon Feb 1, 2022
I'm not sure if such a decision was even ever made, the error happens before normalization is applied. I.e. the parser is doing two things here: (1) validating the syntax against the grammar and (2) building the AST. Normalization happens after (1), and
𝕋𝕣𝕦𝕖 = 0is valid syntax because the grammar is NOT defined in terms of normalized identifiers, it's describing the valid (but confusing!) assignment that Carl described.I agree that this doesn't seem like bug, but it IS my new favorite quirk of identifier normalization.
SnoopJ commentedon Dec 19, 2025
I believe this issue has been addressed by the validation first introduced by #24771, it is no longer possible to assign to
𝕋𝕣𝕦𝕖in Python 3.14:This assignment is forbidden in the 'new' REPL in Python 3.13, but valid in the basic REPL or a source file, so I assume there was some follow-on work that applies this validation more strictly.
Recommend that this issue be closed since the reported behavior has coincidentally been fixed.