Left bracket remains in format string result when '\' preceeds it

@vstinner

BPO	29104
Nosy	@vstinner, @ericvsmith, @serhiy-storchaka, @refi64, @DimitrisJim
PRs	bpo-29104: Fixed parsing backslashes in f-strings. #490 [3.6] bpo-29104: Fixed parsing backslashes in f-strings. (GH-490) #1812
Files	fstring_backslash.patch fstring_backslash_2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ericvsmith'
closed_at = <Date 2017-05-25.11:19:44.542>
created_at = <Date 2016-12-29.11:56:25.482>
labels = ['interpreter-core', 'type-bug', '3.7']
title = "Left bracket remains in format string result when '\\' preceeds it"
updated_at = <Date 2017-05-25.11:19:44.541>
user = 'https://github.com/DimitrisJim'

bugs.python.org fields:

activity = <Date 2017-05-25.11:19:44.541>
actor = 'serhiy.storchaka'
assignee = 'eric.smith'
closed = True
closed_date = <Date 2017-05-25.11:19:44.542>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2016-12-29.11:56:25.482>
creator = 'Jim Fasarakis-Hilliard'
dependencies = []
files = ['46080', '46127']
hgrepos = []
issue_num = 29104
keywords = ['patch']
message_count = 26.0
messages = ['284249', '284256', '284257', '284258', '284259', '284260', '284261', '284262', '284266', '284271', '284558', '284578', '284579', '284580', '285949', '288511', '288979', '289645', '289689', '289690', '289694', '290733', '293604', '293608', '294458', '294469']
nosy_count = 5.0
nosy_names = ['vstinner', 'eric.smith', 'serhiy.storchaka', 'refi64', 'Jim Fasarakis-Hilliard']
pr_nums = ['490', '1812']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue29104'
versions = ['Python 3.6', 'Python 3.7']

In short:

>>> f"\{10}"

yields:

"\\{10"

This is reproducible only when \ precedes the opening bracket, that is:

>>> f"\ {10}"

results in: "\\ 10"

I'm not sure this counts as an error. The backslash means to treat the next character literally, which this does. And since \{ is not a valid escape sequence, it keeps both characters, exactly like:

>>> '\c'
'\\c'

Furthermore, since unknown escape sequences are deprecated, this usage is going to be an error in the future. You can see this now with -Werror:

$ python.exe  -Werror
Python 3.6.0b1+ (3.6:87de1f12c41c+, Sep 16 2016, 07:05:57) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> f'\{10}'
DeprecationWarning: invalid escape sequence '\{'
>>> '\c'
DeprecationWarning: invalid escape sequence '\c'
>>>

Since this works as I expect it to now, and since it will become an error in the future, I don't think any change is warranted.

But I admit that dropping the trailing } seems odd. I think this particular usage should be an error: no bare } allowed, similar to:

>>> f'10}'
  File "<stdin>", line 1
SyntaxError: f-string: single '}' is not allowed

I see, the original "complaint" about this behavior on stack overflow was made due to the discrepancy between the f-string the the format "equivalent": '\{}'.format(10)'.

Do you have a link to the SO question?

Yes, should've attached in my previous message. See http://stackoverflow.com/questions/41330097/why-does-the-symbol-remain-when-f-10-is-evaluated-in-python-3-6

The problem is not that the trailing } is dropped, but that the starting { starts an f-string expression.

>>> f'\{2*5}'
'\\{10'

I expected either '\\10' as in '\{}'.format(2*5), or at least '\\{2*5}'.

There is other f-string parsing error:

>>> f'\\N{2*5}'
'\\N{2*5}'

I expected '\\N10'. '\\N' doesn't start a unicode name escape. This is a legitimate expression.

Yes:

>>> f'\{2*5}'
'\\{10'

I agree '\\10' would make sense.

This problem was no doubt introduced after 3.6a1 when I changed the parsing to disallow backslashes inside {}.

Proposed patch fixes parsing backslashes in f-strings.

P.S. Definitely we should find other name for this term. I hate misleading "f-string".

The PEP-498 should be updated to define the expected behaviour of f'\{5}':
https://www.python.org/dev/peps/pep-0498/#escape-sequences

+ self.assertEqual(f'\{6*7}', '\\42')
+ self.assertEqual(f'\\{6*7}', '\\42')
+ self.assertEqual(fr'\{6*7}', '\\42')
+
+ AMPERSAND = 123
+ self.assertEqual(f'\N{AMPERSAND}', '&')
+ self.assertEqual(f'\\N{AMPERSAND}', '\\N123')
+ self.assertEqual(fr'\N{AMPERSAND}', '\\N123')
+ self.assertEqual(f'\\\N{AMPERSAND}', '\\&')

I'm not sure that I like this behaviour.

f'\\N{AMPERSAND}': reading a local variable looks like a typo or a security vulnerability, rather than a nice feature.

IMHO if you want an anti-slash (\) in the output string, it *must* be written "\\" since we *do* interpret other escape sequences:

f'\n' returns '\n', the newline character U+000A
f'\N{Snowman}' returns the nice snowman character U+2603 (☃)
etc.

What is the issue with having to write "\\{10}" to get "\\10" string? It's less error prone.

Updated patch adds a warning.

Ping.

Serhiy, since review activity has dropped on b.p.o now that the move to github was made, would you like to make this into a pull request to get it reviewed and merged faster?

I am not experienced with git and am waiting until new workflow be described in the devguide.

Serhiy: I plan to find time in the next week or so to look at this. Sorry for the delay. Plus, the new workflow isn't helping me out: I need to get up to speed on it.

Serhiy: if you want, you can give me your email and I'll submit a PR for you. If you want to do it yourself, just:

Download hub: https://hub.github.com/

git clone python/cpython cpython-git
cd cpython-git
git fork
curl https://raw.githubusercontent.com/mozilla/moz-git-tools/master/hg-patch-to-git-patch > hg-patch-to-git-patch
curl https://bugs.python.org/file46127/fstring_backslash_2.patch | python2 hg-patch-to-git-patch > fstring.patch
git checkout -b fstring-fix
git apply fstring.patch
git commit -am 'bpo-29104: Commit message here'
git push -u MY_GITHUB_USERNAME fstring-fix
git pull-request

A PR has been submitted, Ryan. See #490

Thank you Ryan. I already created a PR, but I think your recipe can be helpful for me in future.

Ping again.

Eric?

This is on my list of things to review at PyCon, when I should finally have some free time.

New changeset 0cd7a3f by Serhiy Storchaka in branch 'master':
bpo-29104: Fixed parsing backslashes in f-strings. (#490)
0cd7a3f

New changeset 89a3102 by Serhiy Storchaka in branch '3.6':
[3.6] bpo-29104: Fixed parsing backslashes in f-strings. (GH-490) (bpo-1812)
89a3102

Navigation Menu

Repository navigation

Left bracket remains in format string result when '\' preceeds it #73290

2 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Left bracket remains in format string result when '\' preceeds it #73290

Description

Activity

DimitrisJim commented on Dec 29, 2016

ericvsmith commented on Dec 29, 2016

ericvsmith commented on Dec 29, 2016

DimitrisJim commented on Dec 29, 2016

ericvsmith commented on Dec 29, 2016

DimitrisJim commented on Dec 29, 2016

serhiy-storchaka commented on Dec 29, 2016

ericvsmith commented on Dec 29, 2016

ericvsmith commented on Dec 29, 2016

serhiy-storchaka commented on Dec 29, 2016

vstinner commented on Jan 3, 2017

2 remaining items

serhiy-storchaka commented on Jan 3, 2017

serhiy-storchaka commented on Jan 21, 2017

DimitrisJim commented on Feb 24, 2017

serhiy-storchaka commented on Mar 4, 2017

ericvsmith commented on Mar 15, 2017

refi64 commented on Mar 15, 2017

DimitrisJim commented on Mar 15, 2017

serhiy-storchaka commented on Mar 15, 2017

serhiy-storchaka commented on Mar 28, 2017

serhiy-storchaka commented on May 13, 2017

ericvsmith commented on May 13, 2017

serhiy-storchaka commented on May 25, 2017

serhiy-storchaka commented on May 25, 2017

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions