Deprecation warnings for the future async and await keywords in Python 3.6

@brettcannon

BPO	26182
Nosy	@brettcannon, @giampaolo, @ned-deily, @1st1, @marco-buttu, @AnishShah
Files	async_await.patch: Patch and related test issue_26182.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/1st1'
closed_at = <Date 2016-09-15.16:51:13.036>
created_at = <Date 2016-01-22.19:06:06.579>
labels = ['interpreter-core', 'type-feature', 'release-blocker', 'expert-asyncio']
title = 'Deprecation warnings for the future async and await keywords in Python 3.6'
updated_at = <Date 2017-11-08.19:58:50.497>
user = 'https://github.com/marco-buttu'

bugs.python.org fields:

activity = <Date 2017-11-08.19:58:50.497>
actor = 'serhiy.storchaka'
assignee = 'yselivanov'
closed = True
closed_date = <Date 2016-09-15.16:51:13.036>
closer = 'yselivanov'
components = ['Interpreter Core', 'asyncio']
creation = <Date 2016-01-22.19:06:06.579>
creator = 'marco.buttu'
dependencies = []
files = ['41908', '44669']
hgrepos = []
issue_num = 26182
keywords = ['patch']
message_count = 24.0
messages = ['258833', '259038', '259394', '259409', '259411', '259412', '259413', '259534', '259887', '259888', '259892', '260097', '260118', '260177', '260225', '276091', '276097', '276118', '276493', '276575', '276576', '276577', '276585', '280355']
nosy_count = 7.0
nosy_names = ['brett.cannon', 'giampaolo.rodola', 'ned.deily', 'python-dev', 'yselivanov', 'marco.buttu', 'anish.shah']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue26182'
versions = ['Python 3.6']

I saw that async and await will become keywords in Python 3.7 :

https://www.python.org/dev/peps/pep-0492/#deprecation-plans

I enabled the deprecation warnings in Python 3.5.1 and Python 3.6 dev, and I noticed that assigning to async or await does not issue any deprecation warning:

$ python -Wd -c "import sys; print(sys.version); async = 33"
3.5.1 (default, Jan 21 2016, 19:59:28)
[GCC 4.8.4]

$ python -Wd -c "import sys; print(sys.version); async = 33"
3.6.0a0 (default:4b434a4770a9, Jan 12 2016, 13:01:29)
[GCC 4.8.4]

If someone wants to try and fix this, I would look at how the warning for the 'with' statement was handled (it will either be in the compiler while generating bytecode or somewhere in the parser, but I'm fairly certain it's the compiler).

The check for the 'with' statement was handled in the parser (parsetok.c), and in Python 2.5 the warnings were enabled by default.
I do not know how to check if the deprecation warning are enabled, otherwise I would have tried to fix the problem.

I'm not quite sure what you mean by "do not know how to check if the deprecation warning are enabled". Do you mean how do you make sure you don't emit a DeprecationWarning if someone hasn't turned warnings on? If that's the case then if you look at some example code and read https://docs.python.org/3/c-api/exceptions.html#issuing-warnings you will notice that when you call the warning-related functions, all you have to do is check if the return value is < 0, then return like there is an exception raised. Otherwise the warnings module handles whether something will be shown.

I would like to work on this, if it is okay with Marco?
I look at the history of parsetok.c file and I think I can solve this issue.

Also, I have a doubt - PEP-492 says that "async and await names will be softly deprecated in CPython 3.5 and 3.6".
What exactly does "softly deprecate" mean? is it just same as throwing a warning?

I don't know what "softly deprecate" means. Hopefully someone involved with the PEP can answer that question.

I don't know what "softly deprecate" means. Hopefully someone involved with the PEP can answer that question.

I actually thought about emitting DeprecationWarnings in 3.6 for async/await NAME tokens. I think it's a reasonable thing to do to prepare for 3.7, where they will become proper keywords.

Thank you Brett, your explanation (msg259409) and the c-api doc are really straightforward. I think I can fix it in a few days, just the time to read the devguide and be confident about the workflow.

Assigning the issue to myself to make sure it won't be forgotten before it's too late. Anish or Marco, feel free to propose a patch.

Can you please mention the python version in the title?

You need to temporarily turn on warnings for it to work. For example:

  with warnings.catch_warnings():
      warnings.simplefilter('always')
      with self.assertWarnsRegex(DeprecationWarning, "reserved keyword"):
        exec('async = 33')

Do notice I used exec() as otherwise the warning will be triggered at import instead of when the test runs.

Thank you Brett, the problem was the missed exec(). With the patch in attachment the tests pass, but it does not seem to me a good solution. Infact, changing the filter at runtime has no effect:

$ cat foo.py 
import warnings

warnings.simplefilter("always")
async = 33
await = 33

$ ./python foo.py 
$

Does this happen because, putting the PyErr_WarnEx() in Python/ast.c, the warning is issued before the runtime?

Furthermore, if I set the filter from the CL, then the warning is properly triggered, but the file name and line number are wrong:

$ ./python -Wd foo.py 
sys:1: DeprecationWarning: 'async' will become a reserved keyword in Python 3.7
sys:1: DeprecationWarning: 'await' will become a reserved keyword in Python 3.7

Because parsing is done before execution you can't flip on warnings during runtime in the file you to be affected.

As for the line number, that's because it's raise in C code that doesn't have a trigger in Python code. Try importing the code and you should get the line number of the import. Otherwise you will have to check if there is some function to specify a syntax warning that lets you set the line number explicitly (I don't think there is).

Ned, would it be OK to commit this patch after b1?

async/await are scheduled to become real keywords in 3.7. Right now they are only keywords in 'async def' blocks, meaning that it's OK to have a class with 'async' or 'await' attributes.

So this will be a backwards compatibility breaking change in 3.7. This patch makes Python to emit a warning each time you use async or await as an attribute/variable/etc.

As long as Brett is also OK with it, it can go in for 360b2.

I'm fine with it being in b2 because IMO the warning really should make it in 3.6 and for stuff like this it's more critical to hit the RC for people's testing than the beta to work out semantic changes.

I had to rewrite the patch to make sure it reports correct position and covers all cases where using async/await should trigger a warning.

Brett, could you please take a look at the patch?

I'm going to commit the patch now (I'm going on vacation tomorrow, and I want to watch the buildbots).

New changeset 82e6017dc841 by Yury Selivanov in branch '3.6':
Issue bpo-26182: Raise DeprecationWarning for improper use of async/await keywords
https://hg.python.org/cpython/rev/82e6017dc841

New changeset 3f8b75173543 by Yury Selivanov in branch 'default':
Merge 3.6 (issue bpo-26182)
https://hg.python.org/cpython/rev/3f8b75173543

Merged.

Sorry I didn't get around to reviewing; I'm sick.

On Thu, Sep 15, 2016, 09:51 Yury Selivanov <[email protected]> wrote:

Yury Selivanov added the comment:

Merged.

----------
resolution: -> fixed
stage: needs patch -> resolved
status: open -> closed

Python tracker <[email protected]>
<http://bugs.python.org/issue26182\>

New changeset 7a996e826f83 by Yury Selivanov in branch '3.6':
Issue bpo-26182: Fix ia refleak in code that raises DeprecationWarning.
https://hg.python.org/cpython/rev/7a996e826f83

New changeset 7b0e79e7f567 by Yury Selivanov in branch 'default':
Merge 3.6 (issue bpo-26182)
https://hg.python.org/cpython/rev/7b0e79e7f567

Navigation Menu

Repository navigation

Deprecation warnings for the future async and await keywords in Python 3.6 #70370

2 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Deprecation warnings for the future async and await keywords in Python 3.6 #70370

Description

Activity

marco-buttu commented on Jan 22, 2016

brettcannon commented on Jan 27, 2016

marco-buttu commented on Feb 2, 2016

brettcannon commented on Feb 2, 2016

AnishShah commented on Feb 2, 2016

brettcannon commented on Feb 2, 2016

1st1 commented on Feb 2, 2016

marco-buttu commented on Feb 4, 2016

1st1 commented on Feb 8, 2016

vstinner commented on Feb 8, 2016

2 remaining items

brettcannon commented on Feb 11, 2016

marco-buttu commented on Feb 12, 2016

brettcannon commented on Feb 13, 2016

1st1 commented on Sep 12, 2016

ned-deily commented on Sep 12, 2016

brettcannon commented on Sep 12, 2016

1st1 commented on Sep 14, 2016

1st1 commented on Sep 15, 2016

python-dev commented on Sep 15, 2016

1st1 commented on Sep 15, 2016

brettcannon commented on Sep 15, 2016

python-dev commented on Nov 8, 2016

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions