You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm actually not sure how to support this since it is defined in terms of unicode classes, which are not supported by python's regex engine, and there is no efficient way to enumerate the characters in each category (let alone DTRT for the particular version of unicode supported by any given python3 release).
Really seems like more of a misfeature than a feature to me.
Hmm. I haven't actually looked into this in much detail. We should still be able to implement this in an efficient manner for the 99% of code that doesn't use unicode identifiers. It's fine if using unicode identifiers slows down parsing, but it's not okay if this feature slows down parsing everything else.
Maybe something like this:
Have a separate case in the lexer for all non-ascii characters outside string literals and comments (character code >= 128). Fall back to a possibly slow implementation that can do whatever needed to get this working when we hit this, but skip this code path altogether for code that doesn't use any unicode identifiers.
May I ask what's the status of that New Parser? Maybe I can even work on it, if it's Python-only. I was extremely surprised to learn that mypy still doesn't understand Python naming rules. PEP 3131 is almost a decade old. :-o
On Feb 1, 2017 7:48 AM, "Vedran Čačić" ***@***.***> wrote:
May I ask what's the status of that New Parser? Maybe I can even work on
it, if it's Python-only. I was extremely surprised to learn that mypy still
doesn't understand Python naming rules. PEP 3131 is almost a decade old. :-o
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#586 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ACwrMljOZFb_ocMQ5bjaqq7r9vxMImQ2ks5rYKlGgaJpZM4Dj2E->
.
Activity
o11c commentedon Aug 8, 2015
I'm actually not sure how to support this since it is defined in terms of unicode classes, which are not supported by python's regex engine, and there is no efficient way to enumerate the characters in each category (let alone DTRT for the particular version of unicode supported by any given python3 release).
Really seems like more of a misfeature than a feature to me.
JukkaL commentedon Aug 9, 2015
Hmm. I haven't actually looked into this in much detail. We should still be able to implement this in an efficient manner for the 99% of code that doesn't use unicode identifiers. It's fine if using unicode identifiers slows down parsing, but it's not okay if this feature slows down parsing everything else.
Maybe something like this:
Have a separate case in the lexer for all non-ascii characters outside string literals and comments (character code >= 128). Fall back to a possibly slow implementation that can do whatever needed to get this working when we hit this, but skip this code path altogether for code that doesn't use any unicode identifiers.
Also, the
unicodedatamodule could be useful.gvanrossum commentedon Nov 1, 2016
I'm going to close this as a won't fix because the new parser will solve this and we're not going to put any work in the old parser/lexer.
vedgar commentedon Feb 1, 2017
May I ask what's the status of that New Parser? Maybe I can even work on it, if it's Python-only. I was extremely surprised to learn that mypy still doesn't understand Python naming rules. PEP 3131 is almost a decade old. :-o
gvanrossum commentedon Feb 1, 2017