[commit: ghc] master: ApiAnnotations: Add SourceText for unicode tokens (fe95463)

[git at git.haskell.org]

Repository : ssh://git@git.haskell.org/ghc

On branch  : master
Link       : http://ghc.haskell.org/trac/ghc/changeset/fe95463bdf42651d53d1edc7723491664257cc5a/ghc

>---------------------------------------------------------------

commit fe95463bdf42651d53d1edc7723491664257cc5a
Author: Alan Zimmerman <alan.zimm at gmail.com>
Date:   Mon Nov 16 19:43:34 2015 +0200

    ApiAnnotations: Add SourceText for unicode tokens
    
    Summary:
    At the moment there is no way to tell if a given token used its unicode
    variant or its normal one, except to look at the length of the token.
    
    This fails for the unicode '*'.
    
    Expose the original source text for unicode variants so that API
    Annotations can capture them specifically.
    
    Test Plan: ./validate
    
    Reviewers: mpickering, austin, bgamari
    
    Reviewed By: bgamari
    
    Subscribers: thomie
    
    Differential Revision: https://phabricator.haskell.org/D1473
    
    GHC Trac Issues: #11018


>---------------------------------------------------------------

fe95463bdf42651d53d1edc7723491664257cc5a
 compiler/parser/ApiAnnotation.hs                   |  40 +++-
 compiler/parser/Lexer.x                            | 100 +++++-----
 compiler/parser/Parser.y                           | 161 +++++++++-------
 .../ghc-api/annotations-literals/literals.stdout   |   8 +-
 testsuite/tests/ghc-api/annotations/Makefile       |   4 +
 testsuite/tests/ghc-api/annotations/T10307.stdout  |   1 +
 testsuite/tests/ghc-api/annotations/T10312.stdout  |   2 +
 testsuite/tests/ghc-api/annotations/T10357.stdout  |   1 +
 testsuite/tests/ghc-api/annotations/T10358.stdout  |   5 +
 testsuite/tests/ghc-api/annotations/T11018.stderr  |  40 ++++
 testsuite/tests/ghc-api/annotations/T11018.stdout  | 203 +++++++++++++++++++++
 testsuite/tests/ghc-api/annotations/Test11018.hs   |  52 ++++++
 testsuite/tests/ghc-api/annotations/all.T          |   1 +
 .../tests/ghc-api/annotations/exampleTest.stdout   |   2 +
 .../tests/ghc-api/annotations/listcomps.stdout     |   2 +
 .../tests/ghc-api/annotations/parseTree.stdout     |   4 +
 16 files changed, 505 insertions(+), 121 deletions(-)

Diff suppressed because of size. To see it, use:

    git diff-tree --root --patch-with-stat --no-color --find-copies-harder --ignore-space-at-eol --cc fe95463bdf42651d53d1edc7723491664257cc5a