Skip to content

PEP 572: TargetScopeError->SyntaxError, other SyntaxError adjustments #1140

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ncoghlan merged 5 commits into from
Aug 14, 2019
Merged

PEP 572: TargetScopeError->SyntaxError, other SyntaxError adjustments #1140

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
cb27015
PEP 572: Update TargetScopeError section based on implementation
ncoghlan Aug 5, 2019
7244266
Add note about dead code, fix markup
ncoghlan Aug 5, 2019
6f7e3e6
PEP 572: address review comments
ncoghlan Aug 9, 2019
2a86898
Add section noting post-acceptance changes
ncoghlan Aug 9, 2019
64f21f7
python-dev reference for TargetScopeError removal
ncoghlan Aug 9, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 74 additions & 11 deletions pep-0572.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,26 @@ Content-Type: text/x-rst
Created: 28-Feb-2018
Python-Version: 3.8
Post-History: 28-Feb-2018, 02-Mar-2018, 23-Mar-2018, 04-Apr-2018, 17-Apr-2018,
25-Apr-2018, 09-Jul-2018
25-Apr-2018, 09-Jul-2018, 05-Aug-2019
Resolution: https://mail.python.org/pipermail/python-dev/2018-July/154601.html


Abstract
========

This is a proposal for creating a way to assign to variables within an
expression using the notation ``NAME := expr``. A new exception,
``TargetScopeError`` is added, and there is one change to evaluation
order.
expression using the notation ``NAME := expr``.

As part of this change, there is also an update to dictionary comprehension
evaluation order to ensure key expressions are executed before value
expressions (allowing the key to be bound to a name and then re-used as part of
calculating the corresponding value).

During discussion of this PEP, the operator became informally known as
"the walrus operator". The construct's formal name is "Assignment Expressions"
(as per the PEP title), but they may also be referred to as "Named Expressions"
(e.g. the CPython reference implementation uses that name internally).


Rationale
=========
Expand Down Expand Up @@ -275,16 +284,50 @@ i`` part establishes that ``i`` is local to the comprehension, but the
``i :=`` part insists that ``i`` is not local to the comprehension.
The same reason makes these examples invalid too::

[[(j := j) for i in range(5)] for j in range(5)]
[i := 0 for i, j in stuff]
[i+1 for i in i := stuff]
[[(j := j) for i in range(5)] for j in range(5)] # INVALID
[i := 0 for i, j in stuff] # INVALID
[i+1 for i in (i := stuff)] # INVALID

While it's technically possible to assign consistent semantics to these cases,
it's difficult to determine whether those semantics actually make *sense* in the
absence of real use cases. Accordingly, the reference implementation will ensure
that such cases raise ``SyntaxError``, rather than executing with implementation
defined behaviour.

This restriction applies even if the assignment expression is never executed::

[False and (i := 0) for i, j in stuff] # INVALID
[i for i, j in stuff if True or (j := 1)] # INVALID

For the comprehension body (the part before the first "for" keyword) and the
filter expression (the part after "if" and before any nested "for"), this
restriction applies solely to target names that are also used as iteration
variables in the comprehension. Lambda expressions appearing in these
positions introduce a new explicit function scope, and hence may use assignment
expressions with no additional restrictions.

Due to design constraints in the reference implementation (the symbol table
analyser cannot easily detect when names are re-used between the leftmost
comprehension iterable expression and the rest of the comprehension), named
expressions are disallowed entirely as part of comprehension iterable
expressions (the part after each "in", and before any subsequent "if" or
"for" keyword)::

[i+1 for i in (j := stuff)] # INVALID
[i+1 for i in range(2) for j in (k := stuff)] # INVALID
[i+1 for i in [j for j in (k := stuff)]] # INVALID
[i+1 for i in (lambda: (j := stuff))()] # INVALID

A further exception applies when an assignment expression occurs in a
comprehension whose containing scope is a class scope. If the rules
above were to result in the target being assigned in that class's
scope, the assignment expression is expressly invalid.
scope, the assignment expression is expressly invalid. This case also raises
``SyntaxError``::

(The reason for the latter exception is the implicit function created
class Example:
[(j := i) for i in range(5)] # INVALID

(The reason for the latter exception is the implicit function scope created
for comprehensions -- there is currently no runtime mechanism for a
function to refer to a variable in the containing class scope, and we
do not want to add such a mechanism. If this issue ever gets resolved
Expand All @@ -295,8 +338,6 @@ variable defined in the class scope from a comprehension.)
See Appendix B for some examples of how the rules for targets in
comprehensions translate to equivalent code.

The two invalid cases listed above raise ``TargetScopeError``, a
new subclass of ``SyntaxError`` (with the same signature).

Relative precedence of ``:=``
-----------------------------
Expand Down Expand Up @@ -413,6 +454,26 @@ found in assignment statements:
total += tax # Equivalent: (total := total + tax)


Specification changes during implementation
===========================================

The following changes have been made based on implementation experience and
additional review after the PEP was first accepted and before Python 3.8 was
released:

* for consistency with other similar exceptions, and to avoid locking in an
exception name that is not necessarily going to improve clarity for end users,
the originally proposed ``TargetScopeError`` subclass of ``SyntaxError`` was
dropped in favour of just raising ``SyntaxError`` directly. [3]_
* due to a limitation in CPython's symbol table analysis process, the reference
implementation raises ``SyntaxError`` for all uses of named expressions inside
comprehension iterable expressions, rather than only raising them when the
named expression target conflicts with one of the iteration variables in the
comprehension. This could be revisited given sufficiently compelling examples,
but the extra complexity needed to implement the more selective restriction
doesn't seem worthwhile for purely hypothetical use cases.


Examples
========

Expand Down Expand Up @@ -1257,6 +1318,8 @@ References
(https://github.com/Rosuav/cpython/tree/assignment-expressions)
.. [2] Pivotal post regarding inline assignment semantics
(https://mail.python.org/pipermail/python-ideas/2018-March/049409.html)
.. [3] Discussion of PEP 572 TargetScopeError
(https://mail.python.org/archives/list/[email protected]/thread/FXVSYCTQOTT7JCFACKPGPXKULBCGEPQY/)
Copyright
Expand Down