Repository navigation
improve number detection #149
Description
The current regexp for matching numeric literals does not account for:
- An initial (optional) +/- sign used to indicate negative or positive numbers, ex:
-3or+5. Admittedly it's hard or impossible via regexps to reliably differentiate between a negative number vs. a subtraction, exAGE-5vsAGE * -5 - Exponential notation, ex:
-1.2E-3
See https://dev.mysql.com/doc/refman/8.0/en/number-literals.html for the syntax (or at least the syntax for MYSQL).
Activity
scriptsbot commentedon Oct 19, 2023
wkeese commentedon Oct 19, 2023
scriptcoded commentedon Oct 25, 2023
As for exponential notation I don't think that it would be too hard to match. It has a pretty well defined and unique structure.
As for normal signed numbers I think it could be done only using regex, but feels like something that could easily spawn a lot of edge cases. As a proof of concept I threw together this which matches signed numbers only. It has issues with queries that start with a signed number and defines a signed number as a +/- followed by a number and that is not preceded by another number. https://regex101.com/r/bs2Gvb/1
I have briefly been thinking about switching to using something like Ohm for parsing, but it would add to the bundle size, add external dependencies and increase complexity. However if that were be to be implemented this issue would be a breeze to implement.
wkeese commentedon Oct 25, 2023
Agreed.
Yes, agreed. I'm not sure it's possible to solve that problem without using a parser, as I was trying to show in my example
AGE-5vsAGE * -5. But I think a safe enough heuristic is to just assume that a space after a + or - means that it's a binary operator, whereas a + or - followed directly by a number implies that it's a +/- sign.In other words, I was assuming a regexp like
/(?<number>[+-]?\d+(?:\.\d+)?(E[+-]?\d+)?)/(https://regex101.com/r/nQSDHC/2).Agreed.
You could also use a library like https://pegjs.org/, which generates pure javascript, and just release that generated javascript in the bundle. In that way, the external dependency is only at build time, not downloaded to the browser.
But both those approaches would still greatly bloat the code downloaded to the browser.
I think the regexp approach that you have is the best compromise, since a syntax highlighting error is not the end of the world.
scriptcoded commentedon Feb 1, 2024
Same comment as other issues but haven't had a lot of time to work on this library. Next week is vacation so fingers crossed I get some time left over!
WIP fix: improve number detection
fix: improve number detection
fix: improve number detection
scriptcoded commentedon Apr 3, 2024
Fixed in #192. Big thanks to @wkeese! 🥳
fix: improve number detection
fix: improve number detection
fix: improve number detection
fix: improve number detection
chore(release): 5.0.0 [skip ci]
chore(release): 5.0.0 [skip ci]
chore(release): 6.0.0 [skip ci]
scriptsbot commentedon Jul 2, 2024
🎉 This issue has been resolved in version 6.0.0 🎉
The release is available on:
Your semantic-release bot 📦🚀