Skip to content

syntax error in multiline f-string produces ~40k spaces output #81614

New issue
New issue
Closed
Closed
syntax error in multiline f-string produces ~40k spaces output#81614
Labels
3.8 (EOL)end of life3.9 (EOL)end of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)

Description

@asottile
asottile
mannequin
opened on Jun 27, 2019
BPO 37433
Nosy @terryjreedy, @vstinner, @ericvsmith, @asottile, @pablogsal, @asmeurer
PRs
  • [bpo-37433] Fix SyntaxError indicator printing too many spaces for multi-line strings #14433
  • [3.8] bpo-37433: Fix SyntaxError indicator printing too many spaces for multi-line strings (GH-14433) #15001
  • bpo-37433: Add additional test for multi-line SyntaxError #15003
  • [3.8] bpo-37433: Add additional test for multi-line SyntaxError (GH-15003) #15005
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2019-08-21.12:13:50.094>
created_at = <Date 2019-06-27.18:22:41.826>
labels = ['interpreter-core', '3.8', '3.9']
title = 'syntax error in multiline f-string produces ~40k spaces output'
updated_at = <Date 2019-08-21.12:13:50.088>
user = 'https://github.com/asottile'

    bugs.python.org fields:

    activity = <Date 2019-08-21.12:13:50.088>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-08-21.12:13:50.094>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2019-06-27.18:22:41.826>
creator = 'Anthony Sottile'
dependencies = []
files = []
hgrepos = []
issue_num = 37433
keywords = ['patch']
message_count = 12.0
messages = ['346767', '346771', '346773', '346777', '346872', '346873', '346874', '348662', '348664', '348938', '348962', '350074']
nosy_count = 6.0
nosy_names = ['terry.reedy', 'vstinner', 'eric.smith', 'Anthony Sottile', 'pablogsal', 'asmeurer']
pr_nums = ['14433', '15001', '15003', '15005']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue37433'
versions = ['Python 3.8', 'Python 3.9']

    Activity

    1. asottile commented on Jun 27, 2019

      @asottile
      asottilemannequin
      on Jun 27, 2019
      MannequinAuthor

      For instance this file:

      foo = f"""{}
foo"""
      $ python3.8 --version --version
Python 3.8.0b1 (default, Jun  6 2019, 03:44:52) 
[GCC 7.4.0]

$ python3.8 t.py | wc -c
  File "t.py", line 1

      <<snipped, but very very many spaces>>

                                     ^

      SyntaxError: f-string: empty expression not allowed

      $ python3.8 t.py |& wc -c
49134
    2. asottile
      added
      3.8 (EOL)end of life
      3.9 (EOL)end of life
      interpreter-core(Objects, Python, Grammar, and Parser dirs)
      on Jun 27, 2019

    3. asottile commented on Jun 27, 2019

      @asottile
      asottilemannequin
      on Jun 27, 2019
      MannequinAuthor

      I can also trigger this with:

      1+ """\q
"""

      And python -Werror

      so it seems not necessarily related to fstrings

    4. asottile commented on Jun 27, 2019

      @asottile
      asottilemannequin
      on Jun 27, 2019
      MannequinAuthor

      Debugging further, there's a token with a multi_line_start pointing at uninitialized memory -- let's see if I can't track down where that gets set and fix it :)

    5. asmeurer commented on Jun 27, 2019

      @asmeurer
      asmeurermannequin
      on Jun 27, 2019
      Mannequin

      This looks like the same issue I mentioned here https://bugs.python.org/msg344764

    6. terryjreedy commented on Jun 28, 2019

      @terryjreedy
      terryjreedy
      on Jun 28, 2019
      Member

      On Windows 10 with current 3.8 I see correct output, so the bug seems OS specific, even though the simple fix (see PR) is not.

      C:\Users\Terry>py f:/python/a/tem4.py
      File "f:/python/a/tem4.py", line 1
      f'''{}
      ^
      SyntaxError: f-string: empty expression not allowed

    7. terryjreedy
      changed the title [-]syntax error in f-string in multiline string produces ~40k spaces of output[/-] [+]syntax error in multiline f-string produces ~40k spaces output[/+] on Jun 28, 2019

    8. asottile commented on Jun 28, 2019

      @asottile
      asottilemannequin
      on Jun 28, 2019
      MannequinAuthor

      just an f-string doesn't trigger this, there needs to be tokens before it

    9. terryjreedy commented on Jun 28, 2019

      @terryjreedy
      terryjreedy
      on Jun 28, 2019
      Member

      OK, very obnoxious. Fix restores behavior above.

    10. pablogsal commented on Jul 29, 2019

      @pablogsal
      pablogsal
      on Jul 29, 2019
      Member

      Thanks Anthony Sottile for the fix!

      I suggest adding more test cases that do not involve f-strings, what do you think? Can you create another PR for those?

    11. asottile commented on Jul 29, 2019

      @asottile
      asottilemannequin
      on Jul 29, 2019
      MannequinAuthor

      sure! the only other case I could come up with involves multi-line strings and invalid escape sequences + Werror

      let me see if I can fold that in as well

    12. asmeurer commented on Aug 3, 2019

      @asmeurer
      asmeurermannequin
      on Aug 3, 2019
      Mannequin

      This seems related. It's also possible I'm misunderstanding what is supposed to happen here.

      If you create test.py with just the 2 lines:

      """
      a

      and run python test.py from CPython master, you get

      $./python.exe test.py
      File "/Users/aaronmeurer/Documents/cpython/test.py", line 4
      a
      ^
      SyntaxError: EOF while scanning triple-quoted string literal

      Notice that it reports line 4 even though the file only has 2 lines.

      The offset in the syntax error is 6 columns (line numbers and column offsets
      in SyntaxErrors count from 1)

      >>> try:
...     compile('"""\na', '<none>', 'exec')
... except SyntaxError as e:
...     print(repr(e))
...
SyntaxError('EOF while scanning triple-quoted string literal', ('<none>', 2, 6, '"""\na\n'))

    13. asottile commented on Aug 3, 2019

      @asottile
      asottilemannequin
      on Aug 3, 2019
      MannequinAuthor

      seems unrelated and does the same on older versions of python:

      $ python3.5 t.py
  File "t.py", line 4
    
    ^
SyntaxError: EOF while scanning triple-quoted string literal

      I'd suggest a new issue or finding the dupe

    14. vstinner commented on Aug 21, 2019

      @vstinner
      vstinner
      on Aug 21, 2019
      Member

      The initial issue has been fixed in master by the commit 5b94f35:

      commit 5b94f3578c662d5f1ee90c0e6b81481d9ec82d89
Author: Anthony Sottile \<[email protected]\>
Date:   Mon Jul 29 06:59:13 2019 -0700

    Fix `SyntaxError` indicator printing too many spaces for multi-line strings (GH-14433)

Backport to 3.8: commit cf52bd0b9b1c1b7ecdd91e1ebebd15ae5c213a2c.

Thanks Anthony Sottile! I close the issue.

      If there is a different issue in Python 3.7 and you consider that it must be fixed, please file a new issue. Python 3.5 and 3.6 don't accept bugfixes anymore:
      https://devguide.python.org/#status-of-python-branches

    15. vstinner
      closed this as completedon Aug 21, 2019
    16. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.8 (EOL)end of life3.9 (EOL)end of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions