MDEV-34228: Support underscores in numeric literals (SQL:2023 T662)

[abhishek593]

The main contributions of this PR are:

• Added function get_numeric_token(), which is a wrapper around get_token() and it checks if there's an underscore in the current token.
• Added helper function strip_underscores(), which just erases underscores from the current token.
• Updated sql_lex.cc states (MY_LEX_NUMBER_IDENT, MY_LEX_INT_OR_REAL, MY_LEX_REAL) to permit single underscores between digits in decimal, hexadecimal, and binary literals.
• New Test Suite: Added mysql-test/main/numeric_underscores.test covering all possible cases.

New validation rules:

• Only a single underscore is allowed between digits.
• Consecutive underscores are prohibited (e.g., 1__000 is an error).
• An underscore cannot appear at the end of a numeric literal.
• An underscore cannot appear adjacent to a decimal point . or exponent marker e/E (e.g., 1_.0 and 1_e10 are invalid).
• Underscore is allowed immediately after the base prefix (e.g., 0x_FF, 0b_10).

[gkodinov]

A good start! Thank you for your contribution!
This is a preliminary review. The important part is to measure and document the performance impact.

Please also quote into the worklog not some random blog, but the actual version, clause and ideally the definition of the extension itself.

I'd be especially interested how does the standard define the conversions of such literal to strings. E.g. is cast (1_2_3 AS string) supposed to preserve these?
And vice versa, is cast("123" as INT) supposed to preserve these?

You also need to cover casts to/from strings, e.g.: CAST('1_2.3_4e1_0' AS …)

[gkodinov]

I'd use strchr: maybe the compiler will optimize it better.

[gkodinov]

I would like you to profile and record the following performance comparisons:

1. processing a SQL SELECT statement with 1M integer literals (no underscores): prior to your fix compared to after. Run that 10M times to get a good sample.
2. same as above, but with underscores.

[gkodinov]

this is probably garbled by some editor!