[GHC] #12736: Calling a complex Haskell function (obtained via FFI wrapper function) from MSVC 64-bit C code (passed in as FunPtr) can leave SSE2 registers in the XMM6-XMM15 range modified

[GHC]

#12736: Calling a complex Haskell function (obtained via FFI wrapper function) from
MSVC 64-bit C code (passed in as FunPtr) can leave SSE2 registers in the
XMM6-XMM15 range modified
-------------------------------------+-------------------------------------
        Reporter:  bavism            |                Owner:
            Type:  bug               |               Status:  new
        Priority:  normal            |            Milestone:
       Component:  Compiler (FFI)    |              Version:  7.10.3
      Resolution:                    |             Keywords:
                                     |  ffi,registers,sse2,clobber,xmm
Operating System:  Windows           |         Architecture:  x86_64
                                     |  (amd64)
 Type of failure:  Incorrect result  |            Test Case:
  at runtime                         |  https://github.com/bavis-m/raycast
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by bavism):

 Replying to [comment:1 carter]:
 > Reading the doc
 > {{{
 > XMM6:XMM15, YMM6:YMM15
 > Nonvolatile (XMM), Volatile (upper half of YMM)
 > Must be preserved as needed by callee. YMM registers must be preserved
 as needed by caller.
 >
 > }}}
 >
 > It looks like if you have the callee clobber ymm6-16 you can get the
 caller to handle the save / restore
 > Alternatively, a simple wrapper around the Haskell functions could
 explicitly read xmm6-16 before entering the Haskell call and set the
 values after return.  So that should at least fix it with a simple read
 call and set sequence on the c side
 >
 > That said, sounds like this indeed a bug, though the xmm vs ymm caller
 vs callee stuff is kinda gross and definitely a bug in the wrappers/stubs
 generated for the windows platform.
 >
 > Please share if the near term work around helps.  I realize it adds an
 extra indirection in the Haskell call,  but probably simplest way to fix
 it this week?

 Turns out this was much more difficult than I initially anticipated, as
 MSVC does not allow inline assembly in x64 projects :(. I have pushed up a
 workaround to a new branch in that project, {{{fixasm}}}, that directs
 Visual Studio to build a new file, stub.asm, which exposes a stub function
 for calling a Haskell function pointer. The stub function saves
 XMM6-XMM15. This causes the Haskell function to not clobber the registers,
 and everything works correctly.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/12736#comment:3>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler