new llvm IR callbr instruction useful for proc-point splitting?

Thu Dec 5 17:18:11 UTC 2019

On Wed, 2019-12-04 at 22:16 +0000, Simon Peyton Jones wrote:
> Thanks Kavon.
> 
>  
> 
> What news of CpsCall in LLVM?  That seems crucial.

CPSCall is in essence equivalent to an existing construct in LLVM for GC support
called "statepoints." The difference is that CPSCall expects the front-end to
allocate the stack frame, whereas statepoints are hooked into LLVM's stack-
allocation infrastructure.
While our usage of CPSCall is correct by construction to not require LLVM's
stack-allocation, LLVM's code generator has free-reign to generate stack spills
across a CPSCall for values that one can trivially rematerialize. I don't think
it's sustainable to work around that already deeply-ingrained behavior (however
silly it may be!), so instead the plan is to extend LLVM's statepoints to be
compatible with GHC's runtime system.
How we go about doing this depends on our desires: do we want to also use the
native SP register or not? Some earlier investigation revealed that even on
Windows, one could use a native SP that points to a heap-allocated object,
provided ~512bytes of space is left beyond the SP for the kernel's interrupt
handler. The implementation of Go does this, despite the Windows ABI forbidding
it.
>  
> 
> I didn’t know of an effort to use a different SP register.  I seriously doubt
> that, with GHC’s use of zillions of small, heap-allocated stacks that move
> around during GHC, we’ll be able to leverage profiling or debugging tools in a
> serious
>  way. But perhaps I need education.

Existing debugging and profiling tools (e.g., gdb and Linux perf) can become
more useful for GHC if they understood how to walk the call stack, regardless of
whether the frame moves in memory. One way to do this is to emit call-frame
information (CFI) [1,2] in the output assembly that describes the location and
layout of the frame as it changes. This would work regardless of the SP register
chosen and even if the return address is kept in register. An alternative to
using CFI is to use a the canonical SP register and layout, since the default
"CFI" assumed by these tools is the OS's ABI.
CFI is the most robust solution for GHC, since some functions allocate more than
one frame per invocation to save on code size when applying a curried function
(the Making a Fast Curry paper). This is of course orthogonal to the proc-point
splitting concern.
I believe Ben originally brought up this desire to use a native SP even for the
native backend, because the work of switching to gives us both automatic CFI
support and some branch-predictor performance benefits on x86-64. That's likely
to be a big overhaul though, so I'm personally in favor of extending LLVM's
statepoints to be compatible with GHC and concurrently but separately adding CFI
support for the native backend.

[1] https://sourceware.org/binutils/docs/as/CFI-directives.html[2] 
https://www.imperialviolet.org/2017/01/18/cfi.html

>  
> Simon
>  
> 
> 
> 
> From: ghc-devs <ghc-devs-bounces at haskell.org>
> On Behalf Of Kavon Farvardin
> 
> Sent: 04 December 2019 19:32
> 
> To: Ben Gamari <ben at smart-cactus.org>
> 
> Cc: GHC developers <ghc-devs at haskell.org>
> 
> Subject: Re: new llvm IR callbr instruction useful for proc-point splitting?
> 
> 
>  
> 
> 
> Hi Ben,
> 
> 
>  
> 
> 
> The Wiki now has a link to the Google Doc that Michal and I were working on to
> compile all of the notes into one place.
> 
> 
>  
> 
> 
> The key progress made on this front has been in evaluating the trade-offs of
> the design space being considered by GHC, but tested within Manticore.
> 
> 
>  
> 
> 
> I'm drafting a message today for the LLVM mailing list regarding one of our
> first options: defining a GHC ABI to redefine which stack-pointer register is
> used. 
> 
> 
>  
> 
> 
> Once we have additional answers to the question "how much effort is it to add
> a GHC ABI to LLVM?", we can then then start a discussion on whether GHC wants
> to change instead by moving to a native stack-pointer register (to take
> advantage
>  of existing debugging and profiling tools, etc).
> 
> 
>  
> 
> 
> Ideally, we would then make a decision on which option to take and someone
> (perhaps a GSoC student with mentorship from myself and others) with spare
> time can work on it.
> 
> 
>  
> 
> 
> Cheers,
> 
> 
> Kavon
> 
>  
> 
> 
> On Wed, 2019-12-04 at 11:38 -0500, Ben Gamari wrote:
> 
> > Kavon Farvardin <kavon at farvard.in> writes:
> >  
> > Yes, callbr is part of the discussion on removing proc-point splitting! 
> > Soon there will be an announcement about a new working group dedicated to
> > LLVM
> > issues such as this one on this mailing list. Anyone interested is welcome
> > to
> > join our meetings.
> > We're currently gathering our notes together first, but you can have a look
> > here
> > for a preview: 
> > 
https://gitlab.haskell.org/ghc/ghc/wikis/commentary/compiler/LLVM-Backend-Optimization
> >  
> > Hi Kavon,
> >  
> > What ever happened to this? It came up during the GHC call and we
> > realized that none of us knew what the state of the proposal was. Is
> > this Wiki page still reflective of the current state of play?
> >  
> > Cheers,
> >  
> > - Ben
> >  
> >  
> 
> 
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191205/5c3e5131/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191205/5c3e5131/attachment.sig>