possible solution! Re: llvm calling convention matters
Carter Schonwald
carter.schonwald at gmail.com
Fri Sep 13 02:40:30 UTC 2013
let me know before the weekend starts.... so i can make time to help if
need be (unless Austin gives breathing room on merge window for such a
thing)
On Thu, Sep 12, 2013 at 3:03 PM, Carter Schonwald <
carter.schonwald at gmail.com> wrote:
> emphasis on "very very clear warning"
>
>
> On Thu, Sep 12, 2013 at 3:00 PM, Carter Schonwald <
> carter.schonwald at gmail.com> wrote:
>
>> after a bit more reflection: as long as we provide a clear warning that
>> 7.8 may at some point no longer work with llvm 3.4, i'm down for the
>> change. We just need to make it very very clear, that it may stop working.
>> (and have AVX support via passing on the stack with <= 3.3)
>>
>> before i go and upstream that patch, could we benchmark how multivector
>> perf fairs with patched llvm? i don't have the right hardware for doing
>> the benchmarks you did in your paper...
>>
>> sorry for being a bit over the top yesterday, i'm just juggling a lot
>> right now :)
>>
>> -Carter
>>
>>
>> On Thu, Sep 12, 2013 at 2:47 PM, Carter Schonwald <
>> carter.schonwald at gmail.com> wrote:
>>
>>> oh, i didn't realize you had already done the work! (bah, i'm sorry, i
>>> feel terrible)
>>>
>>> I thought i had communicated ~ a month ago that I was worried about
>>> release engineering interaction with making it impossible to then make a
>>> subsequent changes more thoughtfully because of the LLVM release cycle.
>>> This concern of mine balloned a bit after helping triage a huge number of
>>> problems people were hitting with the Clang transition on mac thats
>>> underway.
>>>
>>> Its actually very easy to package up an llvm with that patch, much
>>> simpler than "build GHC from source". In fact, on OS X, the simplest way to
>>> install LLVM by default essentially does a build from source.
>>>
>>> Geoff, it'd at least be worth running the benchmarks to measure the
>>> work! (and as I said, i'm happy to help)
>>>
>>>
>>> On Thu, Sep 12, 2013 at 2:30 PM, Geoffrey Mainland <mainland at apeiron.net
>>> > wrote:
>>>
>>>> If users have to do a custom llvm build, we might as well ask them to
>>>> build ghc from source too.
>>>>
>>>> Unless I misunderstood ticket #8033, you were originally quite gung-ho
>>>> about changing the LLVM calling conventions to support passing SIMD
>>>> vectors of all widths in registers on both x86-32 and -64, getting these
>>>> patches into LLVM 3.4, and making sure that GHC 7.8 would support all
>>>> this. I spent several days making sure this could happen from the GHC
>>>> side. Now that the plan has changed, I will back out that work, and 7.8
>>>> will only support passing 128-bit SIMD vectors in registers on x86-64.
>>>> Other vectors sizes, and all vectors on x86-32, will be passed on the
>>>> stack.
>>>>
>>>> Geoff
>>>>
>>>> On 9/12/13 1:32 PM, Carter Schonwald wrote:
>>>> > to repeat:
>>>> >
>>>> > I think no one would have object to having a clearly marked,
>>>> > experimental -fllvmExpermentalAVX flag that requires building LLVM
>>>> > with a specified patch, as a way to showcase your multivector work!
>>>> >
>>>> > that would evade all of my objections (provided avx is still exposed
>>>> > with normal -fllvm, but spilled to stack rather than registers), and
>>>> > i'd actually argue in favor of such.
>>>> >
>>>> > Especially since it would not impose any release cycle constraints on
>>>> > a subsequent, systematic exploration for using XMM / YMM / ZMM in the
>>>> > calling convention going forward.
>>>> >
>>>> > @Geoff, Simons, Johan, and others: does anyone object to that
>>>> approach?
>>>> >
>>>> > applying such a calling convention patch to llvm is really quite
>>>> > straightforward, and the build process is pretty zippy after that too.
>>>> >
>>>> > cheers
>>>> > -Carter
>>>> >
>>>> >
>>>> > On Thu, Sep 12, 2013 at 2:34 AM, Carter Schonwald
>>>> > <carter.schonwald at gmail.com <mailto:carter.schonwald at gmail.com>>
>>>> wrote:
>>>> >
>>>> > that said it does occur to me that there is an alternative
>>>> > solution that may be acceptable for everyone!
>>>> >
>>>> > what about providing a pseudo compatible way called
>>>> > -fllvm-experimentalAVX (or something), and simply require that for
>>>> > it to be used, the user has an llvm Patched with the YMM simd in
>>>> > register fun call support? internally that could just be an llvm
>>>> > way that trips the logic that puts the first few AVX values in
>>>> > those YMM1-6 slots if they are the first args, so only the stack
>>>> > spilling logic needs be changed?
>>>> >
>>>> > (ie it wouldn't be tied to an llvm version, but rather this pseduo
>>>> > way flag)
>>>> >
>>>> > does that make sense?
>>>> >
>>>> > either way, i'd really like having avx even if its always spilled
>>>> > to stack at funcalls with standard LLVMs!
>>>> >
>>>> > cheers
>>>> > -carter
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > On Thu, Sep 12, 2013 at 2:28 AM, Carter Schonwald
>>>> > <carter.schonwald at gmail.com <mailto:carter.schonwald at gmail.com>>
>>>> > wrote:
>>>> >
>>>> > Geoff,
>>>> >
>>>> > a prosaic reason why there *might* be a fundamentally breaking
>>>> > change would be the following idea nathan howell suggested to
>>>> > me this afternoon: change the Sp and SPLim register so that
>>>> > the X86/x86_64 target can use the CPU's Push and (maybe) Pop
>>>> > instructions for the stack manipulations, rather than MOV and
>>>> > fam. see http://ghc.haskell.org/trac/ghc/ticket/8272 (which
>>>> > is just what i've said). Thats one change thats pretty simple
>>>> > but deep, but likely worth exploring.
>>>> >
>>>> >
>>>> > i'm saying any ABI change for GHC 7.10, would likely entail
>>>> > patching LLVM 3.4, because thats the only LLVM version likely
>>>> > to come out between now and whenever we get 7.10 out (assuming
>>>> > 7.10 lands within the next 8-12 months, which is reasonable
>>>> > since we've got noticeably more (amazing) people helping out
>>>> > lately). Thus, any change there entails either asking the llvm
>>>> > folks to support >1 GHC convention per architecture, or
>>>> > replace the current one! I'd rather do the latter than the
>>>> > former, when it comes to asking other people to maintain it :)
>>>> > (and llvm engineers do in fact help out maintaining that code)
>>>> >
>>>> >
>>>> > have you run a Nofib, or even benchmarks restricted to your
>>>> > multivector code, for the current calling convention
>>>> > (including the spilling AVX vectors to the stack thats the
>>>> > current plan i gather) VS passing in registers with an LLVM
>>>> > built using the patches i worked out ~ 2 months ago? it'd be
>>>> > really easy to build that custom llvm, then run the
>>>> > benchmarks! (i'm happy to help, and ultimately, benchmarks
>>>> > will reveal if its worth while or not! And if the main goal is
>>>> > for your talk, its still valid even if its not in the merge
>>>> > window over the next 4 days).
>>>> >
>>>> > I really think its not obvious what the "best" abi
>>>> > change would be! It really will require coming up with a list
>>>> > of variants, implementing them, and running nofib with each
>>>> > variant, which i lack the compute/human time resources to do
>>>> > this week. Modern hardware is complex enough that for
>>>> > something like an ABI change, the only healthy attitude can be
>>>> > "lets benchmark it!".
>>>> >
>>>> > i'd really like any change in calling convention to also
>>>> > improve perf on codes that aren't explicitly simd! (and a
>>>> > conservative simd only change, blocks/conflicts with that
>>>> > augmentation going forward, and not just for the stack pointer
>>>> > example i mention early)
>>>> >
>>>> > Not just scalar floats in simd registers , but perhaps also
>>>> > words/ints !
>>>> >
>>>> > (though that latter bit might be pretty ambitious and subtle,
>>>> > i'll need to investigate that a bit to see how feasible it may
>>>> > be).
>>>> > SIMD has great support for ints/words, and any partial abi
>>>> > change on the llvm backend now would make it hard to support
>>>> > that later well (or at least, thats what it looks like to me).
>>>> > actually effectively using simd for scalar ints and words
>>>> > should be doable, but might force us to be a bit more
>>>> > thoughtful on how GHC internally distinguishes ints used for
>>>> > address arithmetic, vs ints used as data. (interestingly, i'm
>>>> > not sure if any current extent x86 calling convention does
>>>> that!)
>>>> >
>>>> >
>>>> > That single change would make 7.10 require a completely
>>>> > different llvm and native code gen convention from our current
>>>> > one, plus touch all of the code gen on x86 architectures.
>>>> >
>>>> >
>>>> > basically: we're lucky that everyone builds haskell code from
>>>> > source, so ABI compat across GHC versions is a non issue. BUT,
>>>> > any ABI changes should be backed by benchmarks (at least when
>>>> > the change is performance motivated). Likewise, because we use
>>>> > LLVM as an external dep for the -fllvm backend, we really need
>>>> > to keep how their release cycle interacts with our release
>>>> > cycle, because people use haskell and ghc! which as many like
>>>> > to say, is both a boon and a pain ;).
>>>> >
>>>> > Having people hit ghc acting broken with an llvm that was
>>>> > "supported before" is risky support problem to deal with.
>>>> > having an LLVM head variant support a modified ABI, and then
>>>> > later needing to break it for 7.10 (for one of the possible
>>>> > exploratory reasons above) would lead to a support headache I
>>>> > don't wish on anyone.
>>>> >
>>>> > pardon the verbose answer, but thats my offhand take
>>>> >
>>>> > cheers
>>>> > -Carter
>>>> >
>>>> >
>>>> > On Wed, Sep 11, 2013 at 10:10 PM, Geoffrey Mainland
>>>> > <mainland at apeiron.net <mailto:mainland at apeiron.net>> wrote:
>>>> >
>>>> > We support compiling some code with -fllvm and some not in
>>>> > the same
>>>> > executable. Otherwise how could users of the Haskell
>>>> > Platform link their
>>>> > -fllvm-compiled code with native-codegen-compiled
>>>> > libraries like base, etc.?
>>>> >
>>>> > In other words, the LLVM and native back ends use the same
>>>> > calling
>>>> > convention. With my SIMD work, they still use the same
>>>> calling
>>>> > conventions, but the native codegen can never generate
>>>> > code that uses
>>>> > SIMD instructions.
>>>> >
>>>> > Geoff
>>>> >
>>>> > On 09/11/2013 10:03 PM, Johan Tibell wrote:
>>>> > > OK. But that doesn't create a problem for the code we
>>>> > output with the
>>>> > > LLVM backend, no? Or do we support compiling some code
>>>> > with -fllvm and
>>>> > > some not in the same executable?
>>>> > >
>>>> > >
>>>> > > On Wed, Sep 11, 2013 at 6:56 PM, Geoffrey Mainland
>>>> > > <mainland at apeiron.net <mailto:mainland at apeiron.net>
>>>> > <mailto:mainland at apeiron.net
>>>> > <mailto:mainland at apeiron.net>>> wrote:
>>>> > >
>>>> > > We definitely have interop between the native
>>>> > codegen and the LLVM
>>>> > > back
>>>> > > end now. Otherwise anyone who wanted to use the LLVM
>>>> > back end
>>>> > > would have
>>>> > > to build GHC themselves. Interop means that users
>>>> > can install the
>>>> > > Haskell Platform and still use -fllvm when it makes
>>>> > a performance
>>>> > > difference.
>>>> > >
>>>> > > Geoff
>>>> > >
>>>> > > On 09/11/2013 07:59 PM, Johan Tibell wrote:
>>>> > > > Do nothing different than you're doing for 7.8, we
>>>> > can sort it out
>>>> > > > later. Just put a comment on the primops saying
>>>> > they're
>>>> > > LLVM-only. See
>>>> > > > e.g.
>>>> > > >
>>>> > > >
>>>> > > >
>>>> > >
>>>> >
>>>> https://github.com/ghc/ghc/blob/master/compiler/prelude/primops.txt.pp#L181
>>>> > > >
>>>> > > > for an example how to add docs to primops.
>>>> > > >
>>>> > > > I don't think we need interop between the native
>>>> > and the LLVM
>>>> > > > backends. We don't have that now do we (i.e. they
>>>> > use different
>>>> > > > calling conventions).
>>>> > > >
>>>> > > >
>>>> > > >
>>>> > > > On Wed, Sep 11, 2013 at 4:51 PM, Geoffrey Mainland
>>>> > > > <mainland at apeiron.net
>>>> > <mailto:mainland at apeiron.net> <mailto:
>>>> mainland at apeiron.net
>>>> > <mailto:mainland at apeiron.net>>
>>>> > > <mailto:mainland at apeiron.net
>>>> > <mailto:mainland at apeiron.net> <mailto:
>>>> mainland at apeiron.net
>>>> > <mailto:mainland at apeiron.net>>>> wrote:
>>>> > > >
>>>> > > > On 09/11/2013 07:44 PM, Johan Tibell wrote:
>>>> > > > > On Wed, Sep 11, 2013 at 4:40 PM, Geoffrey
>>>> > Mainland
>>>> > > > <mainland at apeiron.net
>>>> > <mailto:mainland at apeiron.net> <mailto:
>>>> mainland at apeiron.net
>>>> > <mailto:mainland at apeiron.net>>
>>>> > > <mailto:mainland at apeiron.net
>>>> > <mailto:mainland at apeiron.net> <mailto:
>>>> mainland at apeiron.net
>>>> > <mailto:mainland at apeiron.net>>>> wrote:
>>>> > > > > > Do you mean we need a reasonable emulation
>>>> > of the SIMD
>>>> > > primops for
>>>> > > > > > the native codegen?
>>>> > > > >
>>>> > > > > Yes. Reasonable in the sense that it
>>>> > computes the right
>>>> > > result.
>>>> > > > I can
>>>> > > > > see that some code might still want to
>>>> > #ifdef (if the
>>>> > > fallback isn't
>>>> > > > > fast enough).
>>>> > > >
>>>> > > > Two implications of this requirement:
>>>> > > >
>>>> > > > 1) There will not be SIMD in 7.8. I just don't
>>>> > have the
>>>> > > time. In fact,
>>>> > > > what SIMD support is there already will have
>>>> > to be removed if we
>>>> > > > cannot
>>>> > > > live with LLVM-only SIMD primops.
>>>> > > >
>>>> > > > 2) If we also require interop between the LLVM
>>>> > back-end and
>>>> > > the native
>>>> > > > codegen, then we cannot pass any SIMD vectors
>>>> in
>>>> > > registers---they all
>>>> > > > must be passed on the stack.
>>>> > > >
>>>> > > > My plan, as discussed with Simon PJ, is to not
>>>> > support SIMD
>>>> > > primops at
>>>> > > > all with the native codegen. If there is a
>>>> > strong feeling that
>>>> > > > this *is
>>>> > > > not* the way to go, the I need to know ASAP.
>>>> > > >
>>>> > > > Geoff
>>>> > > >
>>>> > > >
>>>> > > >
>>>> > >
>>>> > >
>>>> >
>>>> >
>>>> >
>>>> >
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/ghc-devs/attachments/20130912/ff153077/attachment.htm>
More information about the ghc-devs
mailing list