[commit: ghc] master: Implement underscores in numeric literals (NumericUnderscores extension) (4a13c5b)

[git at git.haskell.org]

Repository : ssh://git@git.haskell.org/ghc

On branch  : master
Link       : http://ghc.haskell.org/trac/ghc/changeset/4a13c5b1f4beb53cbf1f3529acdf3ba37528e694/ghc

>---------------------------------------------------------------

commit 4a13c5b1f4beb53cbf1f3529acdf3ba37528e694
Author: Takenobu Tani <takenobu.hs at gmail.com>
Date:   Sun Jan 21 12:08:59 2018 -0500

    Implement underscores in numeric literals (NumericUnderscores extension)
    
    Implement the proposal of underscores in numeric literals.
    Underscores in numeric literals are simply ignored.
    
    The specification of the feature is available here:
    https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/000
    9-numeric-underscores.rst
    
    For a discussion of the various choices:
    https://github.com/ghc-proposals/ghc-proposals/pull/76
    
    Implementation detail:
    
    * Added dynamic flag
      * `NumericUnderscores` extension flag is added for this feature.
    
    * Alex "Regular expression macros" in Lexer.x
      * Add `@numspc` (numeric spacer) macro to represent multiple
        underscores.
      * Modify `@decimal`, `@decimal`, `@binary`, `@octal`, `@hexadecimal`,
        `@exponent`, and `@bin_exponent` macros to include `@numspc`.
    
    * Alex "Rules" in Lexer.x
      * To be simpler, we have only the definitions with underscores.
        And then we have a separate function (`tok_integral` and `tok_frac`)
        that validates the literals.
    
    * Validation functions in Lexer.x
      * `tok_integral` and `tok_frac` functions validate
        whether contain underscores or not.
        If `NumericUnderscores` extensions are not enabled,
        check that there are no underscores.
      * `tok_frac` function is created by merging `strtoken` and
        `init_strtoken`.
      * `init_strtoken` is deleted. Because it is no longer used.
    
    * Remove underscores from target literal string
      * `parseUnsignedInteger`, `readRational__`, and `readHexRational} use
        the customized `span'` function to remove underscores.
    
    * Added Testcase
      * testcase for NumericUnderscores enabled.
          NumericUnderscores0.hs and NumericUnderscores1.hs
      * testcase for NumericUnderscores disabled.
          NoNumericUnderscores0.hs and NoNumericUnderscores1.hs
      * testcase to invalid pattern for NumericUnderscores enabled.
          NumericUnderscoresFail0.hs and NumericUnderscoresFail1.hs
    
    Test Plan: `validate` including the above testcase
    
    Reviewers: goldfire, bgamari
    
    Reviewed By: bgamari
    
    Subscribers: carter, rwbarton, thomie
    
    GHC Trac Issues: #14473
    
    Differential Revision: https://phabricator.haskell.org/D4235


>---------------------------------------------------------------

4a13c5b1f4beb53cbf1f3529acdf3ba37528e694
 compiler/main/DynFlags.hs                          |   1 +
 compiler/parser/Lexer.x                            | 101 +++++++++++++--------
 compiler/utils/StringBuffer.hs                     |   1 +
 compiler/utils/Util.hs                             |  21 ++++-
 docs/users_guide/glasgow_exts.rst                  |  87 ++++++++++++++++++
 .../ghc-boot-th/GHC/LanguageExtensions/Type.hs     |   1 +
 testsuite/tests/driver/T4437.hs                    |   3 +-
 .../parser/should_fail/NoNumericUnderscores0.hs    |  12 +++
 .../should_fail/NoNumericUnderscores0.stderr       |   3 +
 .../parser/should_fail/NoNumericUnderscores1.hs    |  12 +++
 .../should_fail/NoNumericUnderscores1.stderr       |   3 +
 .../parser/should_fail/NumericUnderscoresFail0.hs  |  13 +++
 .../should_fail/NumericUnderscoresFail0.stderr     |   4 +
 .../parser/should_fail/NumericUnderscoresFail1.hs  |  20 ++++
 .../should_fail/NumericUnderscoresFail1.stderr     |   7 ++
 testsuite/tests/parser/should_fail/all.T           |   7 ++
 .../tests/parser/should_run/NumericUnderscores0.hs | 101 +++++++++++++++++++++
 .../parser/should_run/NumericUnderscores0.stdout   |  13 +++
 .../tests/parser/should_run/NumericUnderscores1.hs |  88 ++++++++++++++++++
 .../parser/should_run/NumericUnderscores1.stdout   |  14 +++
 testsuite/tests/parser/should_run/all.T            |   2 +
 21 files changed, 469 insertions(+), 45 deletions(-)

Diff suppressed because of size. To see it, use:

    git diff-tree --root --patch-with-stat --no-color --find-copies-harder --ignore-space-at-eol --cc 4a13c5b1f4beb53cbf1f3529acdf3ba37528e694