[GHC] #7684: cgrun071 segfaults

[GHC]

#7684: cgrun071 segfaults
----------------------------------+----------------------------------
        Reporter:  tibbe          |            Owner:
            Type:  bug            |           Status:  new
        Priority:  normal         |        Milestone:  7.8.1
       Component:  Compiler       |          Version:  7.7
      Resolution:                 |         Keywords:
Operating System:  MacOS X        |     Architecture:  x86_64 (amd64)
 Type of failure:  Runtime crash  |       Difficulty:  Unknown
       Test Case:  cgrun071       |       Blocked By:
        Blocking:                 |  Related Tickets:
----------------------------------+----------------------------------

Comment (by rwbarton):

 Well, which compiler is doing the wrong thing?

 According to the x86_64 ABI, gcc 4.2 is doing the wrong thing, because
 when it generates code for `hs_popcnt8`, it assumes that the high 7 bytes
 of the argument register `%edi` are 0.  The caller is not supposed to have
 to guarantee that, and gcc 4.8 does the right thing by masking off the
 high 7 bytes inside `hs_popcnt8`.

 ghc sees a primop `PopCnt8Op` of type `Word# -> Word#`, and given that, it
 is doing the right thing by taking the whole `Word#` value extracted from
 the argument to `popcnt8` (in `cgrun071`) and putting it in `%edi` before
 calling `hs_popcnt8`.  I don't think ghc looks at the C prototype of
 functions that implement primops, does it?  It just prepares the arguments
 for the call in registers or on the stack according to their Cmm sizes,
 and does a call to the symbol of the right name.

 Now, the combination of ghc passing a whole `Word#` and `hs_popcnt8`
 treating the argument as an `unsigned char` works out according to the
 x86_64 ABI since the intent is to compute the popcnt of the lowest byte.
 It doesn't work out if `hs_popcnt8` expects its argument to be zero-
 extended from 1 byte to 8 bytes like under gcc 4.2.

 Changing the declared argument type of `hs_popcnt8` to `StgWord` will make
 `popcnt.c` agree with the list of primops.  Given that we can modify ghc
 but not gcc 4.2 this seems like the best solution for now.

 If `Word8#` becomes a thing, we can change the signature of `hs_popcnt8`
 back and also change the declared type of `PopCnt8Op` to `Word8# ->
 Word#`.  As I understand things, the key is that those two declarations
 agree.

-- 
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/7684#comment:35>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler