From simonpj at microsoft.com Wed Aug 1 11:59:41 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 1 Aug 2018 11:59:41 +0000 Subject: [commit: ghc] master: Revert "Don't inline functions with RULES too early" (1df50a0) In-Reply-To: <20180801105454.EF0F63A8E4@ghc.haskell.org> References: <20180801105454.EF0F63A8E4@ghc.haskell.org> Message-ID: Bother. Sorry, I did validate, but maybe not clean enough... | -----Original Message----- | From: ghc-commits On Behalf Of | git at git.haskell.org | Sent: 01 August 2018 11:55 | To: ghc-commits at haskell.org | Subject: [commit: ghc] master: Revert "Don't inline functions with RULES too | early" (1df50a0) | | Repository : ssh://git at git.haskell.org/ghc | | On branch : master | Link : | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fghc.haskell. | org%2Ftrac%2Fghc%2Fchangeset%2F1df50a0f61f320428f2e6dd07b3c9ce49c4acd31%2Fgh | c&data=02%7C01%7Csimonpj%40microsoft.com%7Cf0f94d0215cc45d3c05f08d5f79d3 | bcf%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636687177045114421&sdat | a=KJfA0OCKfpsAKB0Zz8Z%2FS9mnqr3D3gLhn6AYd7ddzE4%3D&reserved=0 | | >--------------------------------------------------------------- | | commit 1df50a0f61f320428f2e6dd07b3c9ce49c4acd31 | Author: Ben Gamari | Date: Wed Aug 1 06:42:19 2018 -0400 | | Revert "Don't inline functions with RULES too early" | | This commit causes significant performance regressions: | ``` | bytes allocated value is too high: | Expected T9872d(normal) bytes allocated: 578498120 +/-5% | Lower bound T9872d(normal) bytes allocated: 549573214 | Upper bound T9872d(normal) bytes allocated: 607423026 | Actual T9872d(normal) bytes allocated: 677179968 | Deviation T9872d(normal) bytes allocated: 17.1 % | bytes allocated value is too high: | Expected T9872c(normal) bytes allocated: 3096670112 +/-5% | Lower bound T9872c(normal) bytes allocated: 2941836606 | Upper bound T9872c(normal) bytes allocated: 3251503618 | Actual T9872c(normal) bytes allocated: 3601872536 | Deviation T9872c(normal) bytes allocated: 16.3 % | bytes allocated value is too high: | Expected T9872b(normal) bytes allocated: 3730686224 +/-5% | Lower bound T9872b(normal) bytes allocated: 3544151912 | Upper bound T9872b(normal) bytes allocated: 3917220536 | Actual T9872b(normal) bytes allocated: 4374298272 | Deviation T9872b(normal) bytes allocated: 17.3 % | bytes allocated value is too high: | Expected T9872a(normal) bytes allocated: 2729927408 +/-5% | Lower bound T9872a(normal) bytes allocated: 2593431037 | Upper bound T9872a(normal) bytes allocated: 2866423779 | Actual T9872a(normal) bytes allocated: 3225788896 | Deviation T9872a(normal) bytes allocated: 18.2 % | ``` | It's not clear that this was intentional so I'm going to revert for now. | | This reverts commit 2110738b280543698407924a16ac92b6d804dc36. | | | >--------------------------------------------------------------- | | 1df50a0f61f320428f2e6dd07b3c9ce49c4acd31 | compiler/basicTypes/BasicTypes.hs | 10 ----- | compiler/basicTypes/MkId.hs | 2 +- | compiler/specialise/Rules.hs | 49 ++++--------------- | --- | compiler/stranal/WorkWrap.hs | 4 +- | testsuite/tests/simplCore/should_compile/T15445.hs | 8 ---- | .../tests/simplCore/should_compile/T15445.stderr | 13 ------ | .../tests/simplCore/should_compile/T15445a.hs | 10 ----- | testsuite/tests/simplCore/should_compile/all.T | 1 - | 8 files changed, 10 insertions(+), 87 deletions(-) | | Diff suppressed because of size. To see it, use: | | git diff-tree --root --patch-with-stat --no-color --find-copies-harder - | -ignore-space-at-eol --cc 1df50a0f61f320428f2e6dd07b3c9ce49c4acd31 | _______________________________________________ | ghc-commits mailing list | ghc-commits at haskell.org | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell | .org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- | commits&data=02%7C01%7Csimonpj%40microsoft.com%7Cf0f94d0215cc45d3c05f08d | 5f79d3bcf%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636687177045114421&am | p;sdata=7D%2F6r1pP1Pizyev6tFZzHW9pJEsRxwd1r7AnX44KSdc%3D&reserved=0 From me at ara.io Wed Aug 1 14:19:31 2018 From: me at ara.io (Ara Adkins) Date: Wed, 1 Aug 2018 15:19:31 +0100 Subject: Failure to Build In-Reply-To: <87sh3zz9dx.fsf@smart-cactus.org> References: <871sbj1m2g.fsf@smart-cactus.org> <87sh3zz9dx.fsf@smart-cactus.org> Message-ID: Hi Ben, That was spot on! I guess there's something wrong with the stack install of happy - using the one installed from the system package manager worked perfectly. Best, _ara On Tue, 31 Jul 2018 at 16:56, Ben Gamari wrote: > Ara Adkins writes: > > > Thanks all for the insight so far! > > > > Doing that has got me further along in the build, but it is still > failing. > > Amongst the output of `make` it seems that the offending error is as > > follows: > > ``` > > happy: > > > /home/ara/.stack/snapshots/x86_64-linux-tinfo6-nopie/nightly-2018-02-20/8.2.2/share/x86_64-linux-ghc-8.2.2/happy-1.19.9/HappyTemplate-arrays-coerce: > > openFile: does not exist (No such file or directory) > > > It looks like your happy installation is incomplete. Is it possible you > deleted that file? If not, it sounds like a stack issue. Perhaps try > installing happy with cabal or your system's package manager. > > Cheers, > > - Ben > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal.terepeta at gmail.com Wed Aug 1 14:45:46 2018 From: michal.terepeta at gmail.com (Michal Terepeta) Date: Wed, 1 Aug 2018 16:45:46 +0200 Subject: understanding assertions, part deux :) Re: whither built in unlifted Word32# / Word64# etc? In-Reply-To: References: Message-ID: I've rebased the diff and relaxed the assertion - do take a look if that looks reasonable to you :) https://phabricator.haskell.org/D4475 Cheers! - Michal On Wed, Jul 25, 2018 at 9:03 PM Michal Terepeta wrote: > Hi Carter, > > I didn't write this assertion. I only validated locally (IIRC at the time > I uploaded the diff, harbormaster was failing for some other reasons). > > I'll try to have a look at it this weekend. > > Cheers! > > - Michal > > On Wed, Jul 25, 2018 at 2:16 AM Carter Schonwald < > carter.schonwald at gmail.com> wrote: > >> Michal: did you write this Assert about width? and if so could you >> explain it so we can understand? >> >> hrmm... that code is in the procedure for generating C calls for 64bit >> intel systems >> >> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2541 >> is the top of that routine >> >> and width is defined right below the spot in question >> >> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2764-L2774 >> >> it seems like the code/assertion likely predates michal's work? (eg, a >> trick to make sure that genCCall64 doesn't get invoked by 32bit platforms?) >> >> On Mon, Jul 23, 2018 at 8:54 PM Abhiroop Sarkar >> wrote: >> >>> Hi Michal, >>> >>> In the tests that you have added to D4475, are all the tests running >>> fine? >>> >>> On my machine, I was running the FFI tests( >>> https://github.com/michalt/ghc/blob/int8/testsuite/tests/ffi/should_run/PrimFFIInt8.hs) >>> and they seem to fail at a particular assert statement in the code >>> generator. >>> >>> To be precise this one: >>> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2764 >>> >>> Upon commenting that assert the tests run fine. Am I missing something >>> or is the failure expected? >>> >>> Thanks, >>> Abhiroop >>> >>> On Mon, Jul 9, 2018 at 8:31 PM Michal Terepeta < >>> michal.terepeta at gmail.com> wrote: >>> >>>> Just for the record, I've uploaded the changes to binary: >>>> https://github.com/michalt/packages-binary/tree/int8 >>>> >>>> - Michal >>>> >>>> On Wed, Jul 4, 2018 at 11:07 AM Michal Terepeta < >>>> michal.terepeta at gmail.com> wrote: >>>> >>>>> Yeah, if you look at the linked diff, there are a few tiny changes to >>>>> binary that are necessary. >>>>> >>>>> I'll try to upload them to github later this week. >>>>> >>>>> Ben, is something blocking the review of the diff? I think I addressed >>>>> all comments so far. >>>>> >>>>> - Michal >>>>> >>>>> On Wed, Jul 4, 2018 at 1:38 AM Abhiroop Sarkar >>>>> wrote: >>>>> >>>>>> Hello Michal, >>>>>> >>>>>> I was looking at your diff https://phabricator.haskell.org/D4475 >>>>>> and there seems to be some changes that you perhaps made in the binary >>>>>> package ( >>>>>> https://phabricator.haskell.org/differential/changeset/?ref=199152&whitespace=ignore-most). >>>>>> I could not find your version of binary on your github repos list. Is it >>>>>> possible for you to upload that so I can pull those changes? >>>>>> >>>>>> Thanks >>>>>> >>>>>> Abhiroop >>>>>> >>>>>> On Mon, May 28, 2018 at 10:45 PM Carter Schonwald < >>>>>> carter.schonwald at gmail.com> wrote: >>>>>> >>>>>>> Abhiroop has the gist, though the size of word args for simd is more >>>>>>> something we want to be consistent between 32/64 bit modes aka ghc >>>>>>> targeting 32 or 64 bit intel with simd support :). It would be best if the >>>>>>> unpack and pack operations that map to and from unboxed tuples where >>>>>>> consistent and precise type wise. >>>>>>> >>>>>>> >>>>>>> >>>>>>> -Carter >>>>>>> >>>>>>> On May 28, 2018, at 5:02 PM, Abhiroop Sarkar >>>>>>> wrote: >>>>>>> >>>>>>> Hello Michal, >>>>>>> >>>>>>> My understanding of the issues are much lesser compared to Carter. >>>>>>> However, I will state whatever I understood from discussions with him. Be >>>>>>> warned my understanding might be wrong and Carter might be asking this for >>>>>>> some completely different reason. >>>>>>> >>>>>>> > Out of curiosity, why do you need Word64#/Word32# story to be >>>>>>> fixed for SIMD? >>>>>>> >>>>>>> One of the issues we are dealing with is multiple >>>>>>> microarchitectures(SSE, AVX, AVX2 etc). As a result different >>>>>>> microarchitectures has different ways of handling an 8 bit or 16 bit or 32 >>>>>>> bit unsigned integer. An example I can provide is the vector multiply >>>>>>> instruction which has different semantics of multiplying and storing the >>>>>>> first 16 bits of a 32 bit unsigned integer compared to the lower 16 bits >>>>>>> for each of the elements in its SIMD registers. There will be some other >>>>>>> intricacies around handling a byte sized integer or a 64 bit integer which >>>>>>> will be different from the 32 bit version. >>>>>>> >>>>>>> Basically a 128 bit vector instruction will make some minor >>>>>>> differences when dealing with 2 64 bit integers or 4 32 bit integer or 8 16 >>>>>>> bit integers. >>>>>>> >>>>>>> So I think at the higher level we would want to be as precise as >>>>>>> possible when specifying the datatypes and want things like Word8#, >>>>>>> Word16#, Word32#, Word64# rather than a single ambiguous type like Word. >>>>>>> >>>>>>> One more example is this vector operation like : broadcastWord64X2# >>>>>>> :: Word# >>>>>>> >>>>>>> -> Word64X2# >>>>>>> which >>>>>>> should rather be broadcastWord64X2# :: Word64# >>>>>>> >>>>>>> -> Word64X2# >>>>>>> >>>>>>> >>>>>>> Another reason could be improving the space efficiency of packing >>>>>>> various datatypes. This was explained in some detail in this comment: >>>>>>> https://github.com/ghc-proposals/ghc-proposals/pull/74#issuecomment-333951559 >>>>>>> >>>>>>> Thanks, >>>>>>> Abhiroop >>>>>>> >>>>>>> On Mon, May 28, 2018 at 6:06 PM, Michal Terepeta < >>>>>>> michal.terepeta at gmail.com> wrote: >>>>>>> >>>>>>>> Hey Carter, >>>>>>>> >>>>>>>> Yeah, I'm totally snowed under with personal stuff at the moment, >>>>>>>> so not much time to hack on >>>>>>>> anything. I've managed to make some progress on >>>>>>>> https://phabricator.haskell.org/D4475 (prototype >>>>>>>> that adds Int8#/Word8#; waiting for another round of review). Sadly >>>>>>>> June still looks a bit crazy for >>>>>>>> me, so I can't promise anything for Word64#/Word32# (and >>>>>>>> Int64#/Int32#). I hope to have some more >>>>>>>> time in July. >>>>>>>> >>>>>>>> If this is blocking you, then please don't wait for me and go ahead >>>>>>>> (my diff for Int8#/Word8# might >>>>>>>> be a decent starting point to see which places might need changes). >>>>>>>> >>>>>>>> Out of curiosity, why do you need Word64#/Word32# story to be fixed >>>>>>>> for SIMD? Cmm already has >>>>>>>> separate types for vector types (see TyCon.hs and VecRep >>>>>>>> constructor of PrimRep). >>>>>>>> >>>>>>>> Cheers! >>>>>>>> >>>>>>>> - Michal >>>>>>>> >>>>>>>> PS. Are you coming to ZuriHac by any chance? >>>>>>>> >>>>>>>> On Mon, May 28, 2018 at 12:37 AM Carter Schonwald < >>>>>>>> carter.schonwald at gmail.com> wrote: >>>>>>>> >>>>>>>>> Hey Michal! >>>>>>>>> So i'm mentoring Abhiroop around getting SIMD all nice and >>>>>>>>> awesome, and I realized we still dont quite yet have the nice Word32# and >>>>>>>>> Word64# etc story in ghc as yet. >>>>>>>>> >>>>>>>>> Whats the status on that and or is there a way we can help get >>>>>>>>> that over the hump for ghc? These different sized in register ints and >>>>>>>>> friends have a tight partnership with any nice SIMD iterating over the >>>>>>>>> summer, and hopefully helps inform the design >>>>>>>>> >>>>>>>>> -Carter >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Kloona - Coming Soon! >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> Kloona - Coming Soon! >>>>>> >>>>> >>> >>> -- >>> Kloona - Coming Soon! >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From asiamgenius at gmail.com Wed Aug 1 16:36:38 2018 From: asiamgenius at gmail.com (Abhiroop Sarkar) Date: Wed, 1 Aug 2018 17:36:38 +0100 Subject: understanding assertions, part deux :) Re: whither built in unlifted Word32# / Word64# etc? In-Reply-To: References: Message-ID: Hello, I would appreciate some help in debugging a Cmm Lint error, I have been stuck on for quite a while. Basically I am adding support for Int32# on top of the In8#(D4475) and Int16#(D5006) patches. The Cmm being generated for the test programs are incorrect. Taking a sample test like this ( https://github.com/Abhiroop/ghc-1/blob/int8/testsuite/tests/ffi/should_run/PrimFFIInt32.hs ) The crux of the linting error is the type mismatch between LHS and RHS of the CmmAssign statement. For example in the Int8 patch, the Cmm generated looks like this: ``` _s1MC::I8 = I8[Sp]; // CmmAssign _s1MD::I8 = I8[Sp + 8]; // CmmAssign _s1ME::I8 = I8[Sp + 16]; // CmmAssign _s1MF::I8 = I8[Sp + 24]; // CmmAssign _s1MG::I8 = I8[Sp + 32]; // CmmAssign _s1MH::I8 = I8[Sp + 40]; // CmmAssign _s1MI::I8 = I8[Sp + 48]; // CmmAssign _s1MJ::I8 = I8[Sp + 56]; // CmmAssign _s1MK::I8 = I8[Sp + 64]; // CmmAssign _s1ML::I8 = I8[Sp + 72]; // CmmAssign R6 = %MO_XX_Conv_W8_W64(_s1MG::I8); // CmmAssign R5 = %MO_XX_Conv_W8_W64(_s1MF::I8); // CmmAssign R4 = %MO_XX_Conv_W8_W64(_s1ME::I8); // CmmAssign R3 = %MO_XX_Conv_W8_W64(_s1MD::I8); // CmmAssign R2 = %MO_XX_Conv_W8_W64(_s1MC::I8); // CmmAssign ``` or ``` _c1Oq::I8 = %MO_SS_Conv_W64_W8(9); // CmmAssign _s1N0::I8 = _c1Oq::I8; // CmmAssign _c1Ou::I8 = %MO_SS_Conv_W64_W8(8); // CmmAssign _s1MZ::I8 = _c1Ou::I8; // CmmAssign _c1Ox::I8 = %MO_SS_Conv_W64_W8(7); // CmmAssign _s1MY::I8 = _c1Ox::I8; // CmmAssign _c1OA::I8 = %MO_SS_Conv_W64_W8(6); // CmmAssign _s1MX::I8 = _c1OA::I8; // CmmAssign _c1OD::I8 = %MO_SS_Conv_W64_W8(5); // CmmAssign _s1MW::I8 = _c1OD::I8; // CmmAssign _c1OG::I8 = %MO_SS_Conv_W64_W8(4); // CmmAssign _s1MV::I8 = _c1OG::I8; // CmmAssign _c1OJ::I8 = %MO_SS_Conv_W64_W8(3); // CmmAssign _s1MU::I8 = _c1OJ::I8; // CmmAssign ``` Basically the registers on the left hand side have type `I8`, whereas in my case the registers on the LHS always have the width equal to the word size: ``` _c1Ok::I64 = %MO_SS_Conv_W64_W32(9); // CmmAssign _s1N0::I64 = _c1Ok::I64; // CmmAssign _c1Oo::I64 = %MO_SS_Conv_W64_W32(8); // CmmAssign _s1MZ::I64 = _c1Oo::I64; // CmmAssign _c1Or::I64 = %MO_SS_Conv_W64_W32(7); // CmmAssign _s1MY::I64 = _c1Or::I64; // CmmAssign _c1Ou::I64 = %MO_SS_Conv_W64_W32(6); // CmmAssign _s1MX::I64 = _c1Ou::I64; // CmmAssign _c1Ox::I64 = %MO_SS_Conv_W64_W32(5); // CmmAssign _s1MW::I64 = _c1Ox::I64; // CmmAssign _c1OA::I64 = %MO_SS_Conv_W64_W32(4); // CmmAssign _s1MV::I64 = _c1OA::I64; // CmmAssign _c1OD::I64 = %MO_SS_Conv_W64_W32(3); // CmmAssign _s1MU::I64 = _c1OD::I64; // CmmAssign _c1OG::I64 = %MO_SS_Conv_W64_W32(2); // CmmAssign _s1MT::I64 = _c1OG::I64; // CmmAssign ``` I would ideally want to have the datatype of the LHS be `I32`. Michal, I thought this type is picked up using the `primRepCmmType` function https://github.com/Abhiroop/ghc-1/blob/int8/compiler/cmm/CmmUtils.hs#L104-L105 which I modified but I am not sure why the LHS types of the CmmAssign statement allocate `I64` instead of `I32`. Is there any other change on your patch(D4475) which I might have overlooked. I have uploaded my Int32# patch on phab as well for reference( https://phabricator.haskell.org/D5032) Thanks, Abhiroop On Wed, Aug 1, 2018 at 3:45 PM Michal Terepeta wrote: > I've rebased the diff and relaxed the assertion - do take a look if that > looks reasonable to you :) > https://phabricator.haskell.org/D4475 > > Cheers! > > - Michal > > On Wed, Jul 25, 2018 at 9:03 PM Michal Terepeta > wrote: > >> Hi Carter, >> >> I didn't write this assertion. I only validated locally (IIRC at the time >> I uploaded the diff, harbormaster was failing for some other reasons). >> >> I'll try to have a look at it this weekend. >> >> Cheers! >> >> - Michal >> >> On Wed, Jul 25, 2018 at 2:16 AM Carter Schonwald < >> carter.schonwald at gmail.com> wrote: >> >>> Michal: did you write this Assert about width? and if so could you >>> explain it so we can understand? >>> >>> hrmm... that code is in the procedure for generating C calls for 64bit >>> intel systems >>> >>> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2541 >>> is the top of that routine >>> >>> and width is defined right below the spot in question >>> >>> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2764-L2774 >>> >>> it seems like the code/assertion likely predates michal's work? (eg, a >>> trick to make sure that genCCall64 doesn't get invoked by 32bit platforms?) >>> >>> On Mon, Jul 23, 2018 at 8:54 PM Abhiroop Sarkar >>> wrote: >>> >>>> Hi Michal, >>>> >>>> In the tests that you have added to D4475, are all the tests running >>>> fine? >>>> >>>> On my machine, I was running the FFI tests( >>>> https://github.com/michalt/ghc/blob/int8/testsuite/tests/ffi/should_run/PrimFFIInt8.hs) >>>> and they seem to fail at a particular assert statement in the code >>>> generator. >>>> >>>> To be precise this one: >>>> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2764 >>>> >>>> Upon commenting that assert the tests run fine. Am I missing something >>>> or is the failure expected? >>>> >>>> Thanks, >>>> Abhiroop >>>> >>>> On Mon, Jul 9, 2018 at 8:31 PM Michal Terepeta < >>>> michal.terepeta at gmail.com> wrote: >>>> >>>>> Just for the record, I've uploaded the changes to binary: >>>>> https://github.com/michalt/packages-binary/tree/int8 >>>>> >>>>> - Michal >>>>> >>>>> On Wed, Jul 4, 2018 at 11:07 AM Michal Terepeta < >>>>> michal.terepeta at gmail.com> wrote: >>>>> >>>>>> Yeah, if you look at the linked diff, there are a few tiny changes to >>>>>> binary that are necessary. >>>>>> >>>>>> I'll try to upload them to github later this week. >>>>>> >>>>>> Ben, is something blocking the review of the diff? I think I >>>>>> addressed all comments so far. >>>>>> >>>>>> - Michal >>>>>> >>>>>> On Wed, Jul 4, 2018 at 1:38 AM Abhiroop Sarkar >>>>>> wrote: >>>>>> >>>>>>> Hello Michal, >>>>>>> >>>>>>> I was looking at your diff https://phabricator.haskell.org/D4475 >>>>>>> and there seems to be some changes that you perhaps made in the binary >>>>>>> package ( >>>>>>> https://phabricator.haskell.org/differential/changeset/?ref=199152&whitespace=ignore-most). >>>>>>> I could not find your version of binary on your github repos list. Is it >>>>>>> possible for you to upload that so I can pull those changes? >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> Abhiroop >>>>>>> >>>>>>> On Mon, May 28, 2018 at 10:45 PM Carter Schonwald < >>>>>>> carter.schonwald at gmail.com> wrote: >>>>>>> >>>>>>>> Abhiroop has the gist, though the size of word args for simd is >>>>>>>> more something we want to be consistent between 32/64 bit modes aka ghc >>>>>>>> targeting 32 or 64 bit intel with simd support :). It would be best if the >>>>>>>> unpack and pack operations that map to and from unboxed tuples where >>>>>>>> consistent and precise type wise. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -Carter >>>>>>>> >>>>>>>> On May 28, 2018, at 5:02 PM, Abhiroop Sarkar >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hello Michal, >>>>>>>> >>>>>>>> My understanding of the issues are much lesser compared to Carter. >>>>>>>> However, I will state whatever I understood from discussions with him. Be >>>>>>>> warned my understanding might be wrong and Carter might be asking this for >>>>>>>> some completely different reason. >>>>>>>> >>>>>>>> > Out of curiosity, why do you need Word64#/Word32# story to be >>>>>>>> fixed for SIMD? >>>>>>>> >>>>>>>> One of the issues we are dealing with is multiple >>>>>>>> microarchitectures(SSE, AVX, AVX2 etc). As a result different >>>>>>>> microarchitectures has different ways of handling an 8 bit or 16 bit or 32 >>>>>>>> bit unsigned integer. An example I can provide is the vector multiply >>>>>>>> instruction which has different semantics of multiplying and storing the >>>>>>>> first 16 bits of a 32 bit unsigned integer compared to the lower 16 bits >>>>>>>> for each of the elements in its SIMD registers. There will be some other >>>>>>>> intricacies around handling a byte sized integer or a 64 bit integer which >>>>>>>> will be different from the 32 bit version. >>>>>>>> >>>>>>>> Basically a 128 bit vector instruction will make some minor >>>>>>>> differences when dealing with 2 64 bit integers or 4 32 bit integer or 8 16 >>>>>>>> bit integers. >>>>>>>> >>>>>>>> So I think at the higher level we would want to be as precise as >>>>>>>> possible when specifying the datatypes and want things like Word8#, >>>>>>>> Word16#, Word32#, Word64# rather than a single ambiguous type like Word. >>>>>>>> >>>>>>>> One more example is this vector operation like : broadcastWord64X2# >>>>>>>> :: Word# >>>>>>>> >>>>>>>> -> Word64X2# >>>>>>>> which >>>>>>>> should rather be broadcastWord64X2# :: Word64# >>>>>>>> >>>>>>>> -> Word64X2# >>>>>>>> >>>>>>>> >>>>>>>> Another reason could be improving the space efficiency of packing >>>>>>>> various datatypes. This was explained in some detail in this comment: >>>>>>>> https://github.com/ghc-proposals/ghc-proposals/pull/74#issuecomment-333951559 >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Abhiroop >>>>>>>> >>>>>>>> On Mon, May 28, 2018 at 6:06 PM, Michal Terepeta < >>>>>>>> michal.terepeta at gmail.com> wrote: >>>>>>>> >>>>>>>>> Hey Carter, >>>>>>>>> >>>>>>>>> Yeah, I'm totally snowed under with personal stuff at the moment, >>>>>>>>> so not much time to hack on >>>>>>>>> anything. I've managed to make some progress on >>>>>>>>> https://phabricator.haskell.org/D4475 (prototype >>>>>>>>> that adds Int8#/Word8#; waiting for another round of review). >>>>>>>>> Sadly June still looks a bit crazy for >>>>>>>>> me, so I can't promise anything for Word64#/Word32# (and >>>>>>>>> Int64#/Int32#). I hope to have some more >>>>>>>>> time in July. >>>>>>>>> >>>>>>>>> If this is blocking you, then please don't wait for me and go >>>>>>>>> ahead (my diff for Int8#/Word8# might >>>>>>>>> be a decent starting point to see which places might need changes). >>>>>>>>> >>>>>>>>> Out of curiosity, why do you need Word64#/Word32# story to be >>>>>>>>> fixed for SIMD? Cmm already has >>>>>>>>> separate types for vector types (see TyCon.hs and VecRep >>>>>>>>> constructor of PrimRep). >>>>>>>>> >>>>>>>>> Cheers! >>>>>>>>> >>>>>>>>> - Michal >>>>>>>>> >>>>>>>>> PS. Are you coming to ZuriHac by any chance? >>>>>>>>> >>>>>>>>> On Mon, May 28, 2018 at 12:37 AM Carter Schonwald < >>>>>>>>> carter.schonwald at gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hey Michal! >>>>>>>>>> So i'm mentoring Abhiroop around getting SIMD all nice and >>>>>>>>>> awesome, and I realized we still dont quite yet have the nice Word32# and >>>>>>>>>> Word64# etc story in ghc as yet. >>>>>>>>>> >>>>>>>>>> Whats the status on that and or is there a way we can help get >>>>>>>>>> that over the hump for ghc? These different sized in register ints and >>>>>>>>>> friends have a tight partnership with any nice SIMD iterating over the >>>>>>>>>> summer, and hopefully helps inform the design >>>>>>>>>> >>>>>>>>>> -Carter >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Kloona - Coming Soon! >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Kloona - Coming Soon! >>>>>>> >>>>>> >>>> >>>> -- >>>> Kloona - Coming Soon! >>>> >>> -- Kloona - Coming Soon! -------------- next part -------------- An HTML attachment was scrubbed... URL: From george.colpitts at gmail.com Wed Aug 1 19:13:03 2018 From: george.colpitts at gmail.com (George Colpitts) Date: Wed, 1 Aug 2018 13:13:03 -0600 Subject: accuracy of asinh and atanh In-Reply-To: References: Message-ID: Hello Mat Just curious, why the preferred solution isn't to call the system math library? As it says in the README you reference below, - One good solution would be to always call the system math library for these functions. Hope this is isn't a stupid question. Thanks George On Sat, Jun 2, 2018 at 2:23 AM Matt Peddie wrote: > Hi devs, > > I tried to use asinh :: Double -> Double and discovered that it's > inaccurate compared to my system library (GNU libm), even returning > -Infinity in place of finite values in the neighborhood of -22 for > large negative arguments. `atanh` is also inaccurate compared to the > system library. I wrote up a more detailed description of the problem > including plots in the README file at > https://github.com/peddie/ghc-inverse-hyperbolic -- this repository is > package that can help you examine the error for yourself or generate > the plots, and it also contains accurate pure-Haskell translations of > the system library's implementation for these functions. What's the > next step to fixing this in GHC? > > Cheers > > Matt Peddie > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mpeddie at gmail.com Wed Aug 1 23:59:16 2018 From: mpeddie at gmail.com (Matt Peddie) Date: Thu, 2 Aug 2018 09:59:16 +1000 Subject: accuracy of asinh and atanh In-Reply-To: References: Message-ID: Hi George, Not a stupid question. I don't have a single source at hand, but I think I read in a few places on the wiki that calling out to the system math library is not an option due to the variety of system math libraries on the platforms GHC supports. It'd be great if I got the wrong impression and this could just be a call to C. Can anyone set me straight on this point? Matt On Thu, Aug 2, 2018 at 5:13 AM, George Colpitts wrote: > Hello Mat > > Just curious, why the preferred solution isn't to call the system math > library? As it says in the README you reference below, > > One good solution would be to always call the system math library for these > functions. > > Hope this is isn't a stupid question. > > Thanks > > George > > > > > > On Sat, Jun 2, 2018 at 2:23 AM Matt Peddie wrote: >> >> Hi devs, >> >> I tried to use asinh :: Double -> Double and discovered that it's >> inaccurate compared to my system library (GNU libm), even returning >> -Infinity in place of finite values in the neighborhood of -22 for >> large negative arguments. `atanh` is also inaccurate compared to the >> system library. I wrote up a more detailed description of the problem >> including plots in the README file at >> https://github.com/peddie/ghc-inverse-hyperbolic -- this repository is >> package that can help you examine the error for yourself or generate >> the plots, and it also contains accurate pure-Haskell translations of >> the system library's implementation for these functions. What's the >> next step to fixing this in GHC? >> >> Cheers >> >> Matt Peddie >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From ben at smart-cactus.org Thu Aug 2 01:33:21 2018 From: ben at smart-cactus.org (Ben Gamari) Date: Wed, 01 Aug 2018 21:33:21 -0400 Subject: accuracy of asinh and atanh In-Reply-To: References: Message-ID: <87y3dpy2lg.fsf@smart-cactus.org> Matt Peddie writes: > Hi George, > > Not a stupid question. I don't have a single source at hand, but I > think I read in a few places on the wiki that calling out to the > system math library is not an option due to the variety of system math > libraries on the platforms GHC supports. It'd be great if I got the > wrong impression and this could just be a call to C. Can anyone set > me straight on this point? > Indeed it's not a stupid question at all. Indeed this is precisely what we do for the simpler transcendentals (e.g. sin, asin, log). We very well could move in this direction in the case of asinh/atanh as well. I believe the reason we don't currently is that atanh was only standardized in C99, which we only started requiring a few releases ago. Perhaps this is ultimately the right direction. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From mpeddie at gmail.com Thu Aug 2 01:37:53 2018 From: mpeddie at gmail.com (Matt Peddie) Date: Thu, 2 Aug 2018 11:37:53 +1000 Subject: accuracy of asinh and atanh In-Reply-To: <87y3dpy2lg.fsf@smart-cactus.org> References: <87y3dpy2lg.fsf@smart-cactus.org> Message-ID: Thanks, Ben, for chiming in. I think calling out to C for these functions is the way to go if it's now feasible. (Calling out to libm is the workaround I'm using in the application that led me to discover the inaccuracy.) Matt On Thu, Aug 2, 2018 at 11:33 AM, Ben Gamari wrote: > Matt Peddie writes: > >> Hi George, >> >> Not a stupid question. I don't have a single source at hand, but I >> think I read in a few places on the wiki that calling out to the >> system math library is not an option due to the variety of system math >> libraries on the platforms GHC supports. It'd be great if I got the >> wrong impression and this could just be a call to C. Can anyone set >> me straight on this point? >> > Indeed it's not a stupid question at all. Indeed this is precisely what > we do for the simpler transcendentals (e.g. sin, asin, log). We very > well could move in this direction in the case of asinh/atanh as well. I > believe the reason we don't currently is that atanh was only > standardized in C99, which we only started requiring a few releases ago. > Perhaps this is ultimately the right direction. > > Cheers, > > - Ben From asiamgenius at gmail.com Thu Aug 2 03:25:10 2018 From: asiamgenius at gmail.com (Abhiroop Sarkar) Date: Thu, 2 Aug 2018 04:25:10 +0100 Subject: understanding assertions, part deux :) Re: whither built in unlifted Word32# / Word64# etc? In-Reply-To: References: Message-ID: Never mind I found the issue and fixed it. It was the definition of the `Int32` type constructor: int32PrimTyCon = pcPrimTyCon0 int32PrimTyConName IntRep which had to be fixed to int32PrimTyCon = pcPrimTyCon0 int32PrimTyConName Int32Rep Thanks anyways :) Abhiroop On Wed, Aug 1, 2018 at 5:36 PM Abhiroop Sarkar wrote: > Hello, > > I would appreciate some help in debugging a Cmm Lint error, I have been > stuck on for quite a while. > > Basically I am adding support for Int32# on top of the In8#(D4475) and > Int16#(D5006) patches. > > The Cmm being generated for the test programs are incorrect. > > Taking a sample test like this ( > https://github.com/Abhiroop/ghc-1/blob/int8/testsuite/tests/ffi/should_run/PrimFFIInt32.hs > ) > > The crux of the linting error is the type mismatch between LHS and RHS of > the CmmAssign statement. > > For example in the Int8 patch, the Cmm generated looks like this: > > ``` > _s1MC::I8 = I8[Sp]; // CmmAssign > _s1MD::I8 = I8[Sp + 8]; // CmmAssign > _s1ME::I8 = I8[Sp + 16]; // CmmAssign > _s1MF::I8 = I8[Sp + 24]; // CmmAssign > _s1MG::I8 = I8[Sp + 32]; // CmmAssign > _s1MH::I8 = I8[Sp + 40]; // CmmAssign > _s1MI::I8 = I8[Sp + 48]; // CmmAssign > _s1MJ::I8 = I8[Sp + 56]; // CmmAssign > _s1MK::I8 = I8[Sp + 64]; // CmmAssign > _s1ML::I8 = I8[Sp + 72]; // CmmAssign > R6 = %MO_XX_Conv_W8_W64(_s1MG::I8); // CmmAssign > R5 = %MO_XX_Conv_W8_W64(_s1MF::I8); // CmmAssign > R4 = %MO_XX_Conv_W8_W64(_s1ME::I8); // CmmAssign > R3 = %MO_XX_Conv_W8_W64(_s1MD::I8); // CmmAssign > R2 = %MO_XX_Conv_W8_W64(_s1MC::I8); // CmmAssign > ``` > > or > > ``` > _c1Oq::I8 = %MO_SS_Conv_W64_W8(9); // CmmAssign > _s1N0::I8 = _c1Oq::I8; // CmmAssign > _c1Ou::I8 = %MO_SS_Conv_W64_W8(8); // CmmAssign > _s1MZ::I8 = _c1Ou::I8; // CmmAssign > _c1Ox::I8 = %MO_SS_Conv_W64_W8(7); // CmmAssign > _s1MY::I8 = _c1Ox::I8; // CmmAssign > _c1OA::I8 = %MO_SS_Conv_W64_W8(6); // CmmAssign > _s1MX::I8 = _c1OA::I8; // CmmAssign > _c1OD::I8 = %MO_SS_Conv_W64_W8(5); // CmmAssign > _s1MW::I8 = _c1OD::I8; // CmmAssign > _c1OG::I8 = %MO_SS_Conv_W64_W8(4); // CmmAssign > _s1MV::I8 = _c1OG::I8; // CmmAssign > _c1OJ::I8 = %MO_SS_Conv_W64_W8(3); // CmmAssign > _s1MU::I8 = _c1OJ::I8; // CmmAssign > ``` > > Basically the registers on the left hand side have type `I8`, whereas in > my case the registers on the LHS always have the width equal to the word > size: > > ``` > _c1Ok::I64 = %MO_SS_Conv_W64_W32(9); // CmmAssign > _s1N0::I64 = _c1Ok::I64; // CmmAssign > _c1Oo::I64 = %MO_SS_Conv_W64_W32(8); // CmmAssign > _s1MZ::I64 = _c1Oo::I64; // CmmAssign > _c1Or::I64 = %MO_SS_Conv_W64_W32(7); // CmmAssign > _s1MY::I64 = _c1Or::I64; // CmmAssign > _c1Ou::I64 = %MO_SS_Conv_W64_W32(6); // CmmAssign > _s1MX::I64 = _c1Ou::I64; // CmmAssign > _c1Ox::I64 = %MO_SS_Conv_W64_W32(5); // CmmAssign > _s1MW::I64 = _c1Ox::I64; // CmmAssign > _c1OA::I64 = %MO_SS_Conv_W64_W32(4); // CmmAssign > _s1MV::I64 = _c1OA::I64; // CmmAssign > _c1OD::I64 = %MO_SS_Conv_W64_W32(3); // CmmAssign > _s1MU::I64 = _c1OD::I64; // CmmAssign > _c1OG::I64 = %MO_SS_Conv_W64_W32(2); // CmmAssign > _s1MT::I64 = _c1OG::I64; // CmmAssign > ``` > I would ideally want to have the datatype of the LHS be `I32`. > > Michal, I thought this type is picked up using the `primRepCmmType` > function > https://github.com/Abhiroop/ghc-1/blob/int8/compiler/cmm/CmmUtils.hs#L104-L105 which > I modified but I am not sure why the LHS types of the CmmAssign statement > allocate `I64` instead of `I32`. Is there any other change on your > patch(D4475) which I might have overlooked. > > I have uploaded my Int32# patch on phab as well for reference( > https://phabricator.haskell.org/D5032) > > Thanks, > Abhiroop > > > > On Wed, Aug 1, 2018 at 3:45 PM Michal Terepeta > wrote: > >> I've rebased the diff and relaxed the assertion - do take a look if that >> looks reasonable to you :) >> https://phabricator.haskell.org/D4475 >> >> Cheers! >> >> - Michal >> >> On Wed, Jul 25, 2018 at 9:03 PM Michal Terepeta < >> michal.terepeta at gmail.com> wrote: >> >>> Hi Carter, >>> >>> I didn't write this assertion. I only validated locally (IIRC at the >>> time I uploaded the diff, harbormaster was failing for some other reasons). >>> >>> I'll try to have a look at it this weekend. >>> >>> Cheers! >>> >>> - Michal >>> >>> On Wed, Jul 25, 2018 at 2:16 AM Carter Schonwald < >>> carter.schonwald at gmail.com> wrote: >>> >>>> Michal: did you write this Assert about width? and if so could you >>>> explain it so we can understand? >>>> >>>> hrmm... that code is in the procedure for generating C calls for 64bit >>>> intel systems >>>> >>>> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2541 >>>> is the top of that routine >>>> >>>> and width is defined right below the spot in question >>>> >>>> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2764-L2774 >>>> >>>> it seems like the code/assertion likely predates michal's work? (eg, a >>>> trick to make sure that genCCall64 doesn't get invoked by 32bit platforms?) >>>> >>>> On Mon, Jul 23, 2018 at 8:54 PM Abhiroop Sarkar >>>> wrote: >>>> >>>>> Hi Michal, >>>>> >>>>> In the tests that you have added to D4475, are all the tests running >>>>> fine? >>>>> >>>>> On my machine, I was running the FFI tests( >>>>> https://github.com/michalt/ghc/blob/int8/testsuite/tests/ffi/should_run/PrimFFIInt8.hs) >>>>> and they seem to fail at a particular assert statement in the code >>>>> generator. >>>>> >>>>> To be precise this one: >>>>> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2764 >>>>> >>>>> Upon commenting that assert the tests run fine. Am I missing something >>>>> or is the failure expected? >>>>> >>>>> Thanks, >>>>> Abhiroop >>>>> >>>>> On Mon, Jul 9, 2018 at 8:31 PM Michal Terepeta < >>>>> michal.terepeta at gmail.com> wrote: >>>>> >>>>>> Just for the record, I've uploaded the changes to binary: >>>>>> https://github.com/michalt/packages-binary/tree/int8 >>>>>> >>>>>> - Michal >>>>>> >>>>>> On Wed, Jul 4, 2018 at 11:07 AM Michal Terepeta < >>>>>> michal.terepeta at gmail.com> wrote: >>>>>> >>>>>>> Yeah, if you look at the linked diff, there are a few tiny changes >>>>>>> to binary that are necessary. >>>>>>> >>>>>>> I'll try to upload them to github later this week. >>>>>>> >>>>>>> Ben, is something blocking the review of the diff? I think I >>>>>>> addressed all comments so far. >>>>>>> >>>>>>> - Michal >>>>>>> >>>>>>> On Wed, Jul 4, 2018 at 1:38 AM Abhiroop Sarkar < >>>>>>> asiamgenius at gmail.com> wrote: >>>>>>> >>>>>>>> Hello Michal, >>>>>>>> >>>>>>>> I was looking at your diff https://phabricator.haskell.org/D4475 >>>>>>>> and there seems to be some changes that you perhaps made in the binary >>>>>>>> package ( >>>>>>>> https://phabricator.haskell.org/differential/changeset/?ref=199152&whitespace=ignore-most). >>>>>>>> I could not find your version of binary on your github repos list. Is it >>>>>>>> possible for you to upload that so I can pull those changes? >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> Abhiroop >>>>>>>> >>>>>>>> On Mon, May 28, 2018 at 10:45 PM Carter Schonwald < >>>>>>>> carter.schonwald at gmail.com> wrote: >>>>>>>> >>>>>>>>> Abhiroop has the gist, though the size of word args for simd is >>>>>>>>> more something we want to be consistent between 32/64 bit modes aka ghc >>>>>>>>> targeting 32 or 64 bit intel with simd support :). It would be best if the >>>>>>>>> unpack and pack operations that map to and from unboxed tuples where >>>>>>>>> consistent and precise type wise. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -Carter >>>>>>>>> >>>>>>>>> On May 28, 2018, at 5:02 PM, Abhiroop Sarkar < >>>>>>>>> asiamgenius at gmail.com> wrote: >>>>>>>>> >>>>>>>>> Hello Michal, >>>>>>>>> >>>>>>>>> My understanding of the issues are much lesser compared to Carter. >>>>>>>>> However, I will state whatever I understood from discussions with him. Be >>>>>>>>> warned my understanding might be wrong and Carter might be asking this for >>>>>>>>> some completely different reason. >>>>>>>>> >>>>>>>>> > Out of curiosity, why do you need Word64#/Word32# story to be >>>>>>>>> fixed for SIMD? >>>>>>>>> >>>>>>>>> One of the issues we are dealing with is multiple >>>>>>>>> microarchitectures(SSE, AVX, AVX2 etc). As a result different >>>>>>>>> microarchitectures has different ways of handling an 8 bit or 16 bit or 32 >>>>>>>>> bit unsigned integer. An example I can provide is the vector multiply >>>>>>>>> instruction which has different semantics of multiplying and storing the >>>>>>>>> first 16 bits of a 32 bit unsigned integer compared to the lower 16 bits >>>>>>>>> for each of the elements in its SIMD registers. There will be some other >>>>>>>>> intricacies around handling a byte sized integer or a 64 bit integer which >>>>>>>>> will be different from the 32 bit version. >>>>>>>>> >>>>>>>>> Basically a 128 bit vector instruction will make some minor >>>>>>>>> differences when dealing with 2 64 bit integers or 4 32 bit integer or 8 16 >>>>>>>>> bit integers. >>>>>>>>> >>>>>>>>> So I think at the higher level we would want to be as precise as >>>>>>>>> possible when specifying the datatypes and want things like Word8#, >>>>>>>>> Word16#, Word32#, Word64# rather than a single ambiguous type like Word. >>>>>>>>> >>>>>>>>> One more example is this vector operation like : >>>>>>>>> broadcastWord64X2# :: Word# >>>>>>>>> >>>>>>>>> -> Word64X2# >>>>>>>>> which >>>>>>>>> should rather be broadcastWord64X2# :: Word64# >>>>>>>>> >>>>>>>>> -> Word64X2# >>>>>>>>> >>>>>>>>> >>>>>>>>> Another reason could be improving the space efficiency of packing >>>>>>>>> various datatypes. This was explained in some detail in this comment: >>>>>>>>> https://github.com/ghc-proposals/ghc-proposals/pull/74#issuecomment-333951559 >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Abhiroop >>>>>>>>> >>>>>>>>> On Mon, May 28, 2018 at 6:06 PM, Michal Terepeta < >>>>>>>>> michal.terepeta at gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hey Carter, >>>>>>>>>> >>>>>>>>>> Yeah, I'm totally snowed under with personal stuff at the moment, >>>>>>>>>> so not much time to hack on >>>>>>>>>> anything. I've managed to make some progress on >>>>>>>>>> https://phabricator.haskell.org/D4475 (prototype >>>>>>>>>> that adds Int8#/Word8#; waiting for another round of review). >>>>>>>>>> Sadly June still looks a bit crazy for >>>>>>>>>> me, so I can't promise anything for Word64#/Word32# (and >>>>>>>>>> Int64#/Int32#). I hope to have some more >>>>>>>>>> time in July. >>>>>>>>>> >>>>>>>>>> If this is blocking you, then please don't wait for me and go >>>>>>>>>> ahead (my diff for Int8#/Word8# might >>>>>>>>>> be a decent starting point to see which places might need >>>>>>>>>> changes). >>>>>>>>>> >>>>>>>>>> Out of curiosity, why do you need Word64#/Word32# story to be >>>>>>>>>> fixed for SIMD? Cmm already has >>>>>>>>>> separate types for vector types (see TyCon.hs and VecRep >>>>>>>>>> constructor of PrimRep). >>>>>>>>>> >>>>>>>>>> Cheers! >>>>>>>>>> >>>>>>>>>> - Michal >>>>>>>>>> >>>>>>>>>> PS. Are you coming to ZuriHac by any chance? >>>>>>>>>> >>>>>>>>>> On Mon, May 28, 2018 at 12:37 AM Carter Schonwald < >>>>>>>>>> carter.schonwald at gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hey Michal! >>>>>>>>>>> So i'm mentoring Abhiroop around getting SIMD all nice and >>>>>>>>>>> awesome, and I realized we still dont quite yet have the nice Word32# and >>>>>>>>>>> Word64# etc story in ghc as yet. >>>>>>>>>>> >>>>>>>>>>> Whats the status on that and or is there a way we can help get >>>>>>>>>>> that over the hump for ghc? These different sized in register ints and >>>>>>>>>>> friends have a tight partnership with any nice SIMD iterating over the >>>>>>>>>>> summer, and hopefully helps inform the design >>>>>>>>>>> >>>>>>>>>>> -Carter >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Kloona - Coming Soon! >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Kloona - Coming Soon! >>>>>>>> >>>>>>> >>>>> >>>>> -- >>>>> Kloona - Coming Soon! >>>>> >>>> > > -- > Kloona - Coming Soon! > -- Kloona - Coming Soon! -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.pelenitsyn at gmail.com Thu Aug 2 03:26:41 2018 From: a.pelenitsyn at gmail.com (Artem Pelenitsyn) Date: Thu, 2 Aug 2018 06:26:41 +0300 Subject: accuracy of asinh and atanh In-Reply-To: References: <87y3dpy2lg.fsf@smart-cactus.org> Message-ID: I'd be willing to do this. -- Best wishes, Artem On Thu, 2 Aug 2018, 04:38 Matt Peddie, wrote: > Thanks, Ben, for chiming in. I think calling out to C for these > functions is the way to go if it's now feasible. (Calling out to libm > is the workaround I'm using in the application that led me to discover > the inaccuracy.) > > Matt > > On Thu, Aug 2, 2018 at 11:33 AM, Ben Gamari wrote: > > Matt Peddie writes: > > > >> Hi George, > >> > >> Not a stupid question. I don't have a single source at hand, but I > >> think I read in a few places on the wiki that calling out to the > >> system math library is not an option due to the variety of system math > >> libraries on the platforms GHC supports. It'd be great if I got the > >> wrong impression and this could just be a call to C. Can anyone set > >> me straight on this point? > >> > > Indeed it's not a stupid question at all. Indeed this is precisely what > > we do for the simpler transcendentals (e.g. sin, asin, log). We very > > well could move in this direction in the case of asinh/atanh as well. I > > believe the reason we don't currently is that atanh was only > > standardized in C99, which we only started requiring a few releases ago. > > Perhaps this is ultimately the right direction. > > > > Cheers, > > > > - Ben > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carter.schonwald at gmail.com Thu Aug 2 03:37:49 2018 From: carter.schonwald at gmail.com (Carter Schonwald) Date: Wed, 1 Aug 2018 23:37:49 -0400 Subject: understanding assertions, part deux :) Re: whither built in unlifted Word32# / Word64# etc? In-Reply-To: References: Message-ID: So the issue was core lint type error rather than a cmm lint error? Glad you figured it out ! But you didn’t communicate what the error you got from lint was core rather than cmm ... :( On Wed, Aug 1, 2018 at 11:25 PM Abhiroop Sarkar wrote: > Never mind I found the issue and fixed it. > > It was the definition of the `Int32` type constructor: > > int32PrimTyCon = pcPrimTyCon0 int32PrimTyConName IntRep > > which had to be fixed to > > int32PrimTyCon = pcPrimTyCon0 int32PrimTyConName Int32Rep > > Thanks anyways :) > > Abhiroop > > On Wed, Aug 1, 2018 at 5:36 PM Abhiroop Sarkar > wrote: > >> Hello, >> >> I would appreciate some help in debugging a Cmm Lint error, I have been >> stuck on for quite a while. >> >> Basically I am adding support for Int32# on top of the In8#(D4475) and >> Int16#(D5006) patches. >> >> The Cmm being generated for the test programs are incorrect. >> >> Taking a sample test like this ( >> https://github.com/Abhiroop/ghc-1/blob/int8/testsuite/tests/ffi/should_run/PrimFFIInt32.hs >> ) >> >> The crux of the linting error is the type mismatch between LHS and RHS of >> the CmmAssign statement. >> >> For example in the Int8 patch, the Cmm generated looks like this: >> >> ``` >> _s1MC::I8 = I8[Sp]; // CmmAssign >> _s1MD::I8 = I8[Sp + 8]; // CmmAssign >> _s1ME::I8 = I8[Sp + 16]; // CmmAssign >> _s1MF::I8 = I8[Sp + 24]; // CmmAssign >> _s1MG::I8 = I8[Sp + 32]; // CmmAssign >> _s1MH::I8 = I8[Sp + 40]; // CmmAssign >> _s1MI::I8 = I8[Sp + 48]; // CmmAssign >> _s1MJ::I8 = I8[Sp + 56]; // CmmAssign >> _s1MK::I8 = I8[Sp + 64]; // CmmAssign >> _s1ML::I8 = I8[Sp + 72]; // CmmAssign >> R6 = %MO_XX_Conv_W8_W64(_s1MG::I8); // CmmAssign >> R5 = %MO_XX_Conv_W8_W64(_s1MF::I8); // CmmAssign >> R4 = %MO_XX_Conv_W8_W64(_s1ME::I8); // CmmAssign >> R3 = %MO_XX_Conv_W8_W64(_s1MD::I8); // CmmAssign >> R2 = %MO_XX_Conv_W8_W64(_s1MC::I8); // CmmAssign >> ``` >> >> or >> >> ``` >> _c1Oq::I8 = %MO_SS_Conv_W64_W8(9); // CmmAssign >> _s1N0::I8 = _c1Oq::I8; // CmmAssign >> _c1Ou::I8 = %MO_SS_Conv_W64_W8(8); // CmmAssign >> _s1MZ::I8 = _c1Ou::I8; // CmmAssign >> _c1Ox::I8 = %MO_SS_Conv_W64_W8(7); // CmmAssign >> _s1MY::I8 = _c1Ox::I8; // CmmAssign >> _c1OA::I8 = %MO_SS_Conv_W64_W8(6); // CmmAssign >> _s1MX::I8 = _c1OA::I8; // CmmAssign >> _c1OD::I8 = %MO_SS_Conv_W64_W8(5); // CmmAssign >> _s1MW::I8 = _c1OD::I8; // CmmAssign >> _c1OG::I8 = %MO_SS_Conv_W64_W8(4); // CmmAssign >> _s1MV::I8 = _c1OG::I8; // CmmAssign >> _c1OJ::I8 = %MO_SS_Conv_W64_W8(3); // CmmAssign >> _s1MU::I8 = _c1OJ::I8; // CmmAssign >> ``` >> >> Basically the registers on the left hand side have type `I8`, whereas in >> my case the registers on the LHS always have the width equal to the word >> size: >> >> ``` >> _c1Ok::I64 = %MO_SS_Conv_W64_W32(9); // CmmAssign >> _s1N0::I64 = _c1Ok::I64; // CmmAssign >> _c1Oo::I64 = %MO_SS_Conv_W64_W32(8); // CmmAssign >> _s1MZ::I64 = _c1Oo::I64; // CmmAssign >> _c1Or::I64 = %MO_SS_Conv_W64_W32(7); // CmmAssign >> _s1MY::I64 = _c1Or::I64; // CmmAssign >> _c1Ou::I64 = %MO_SS_Conv_W64_W32(6); // CmmAssign >> _s1MX::I64 = _c1Ou::I64; // CmmAssign >> _c1Ox::I64 = %MO_SS_Conv_W64_W32(5); // CmmAssign >> _s1MW::I64 = _c1Ox::I64; // CmmAssign >> _c1OA::I64 = %MO_SS_Conv_W64_W32(4); // CmmAssign >> _s1MV::I64 = _c1OA::I64; // CmmAssign >> _c1OD::I64 = %MO_SS_Conv_W64_W32(3); // CmmAssign >> _s1MU::I64 = _c1OD::I64; // CmmAssign >> _c1OG::I64 = %MO_SS_Conv_W64_W32(2); // CmmAssign >> _s1MT::I64 = _c1OG::I64; // CmmAssign >> ``` >> I would ideally want to have the datatype of the LHS be `I32`. >> >> Michal, I thought this type is picked up using the `primRepCmmType` >> function >> https://github.com/Abhiroop/ghc-1/blob/int8/compiler/cmm/CmmUtils.hs#L104-L105 which >> I modified but I am not sure why the LHS types of the CmmAssign statement >> allocate `I64` instead of `I32`. Is there any other change on your >> patch(D4475) which I might have overlooked. >> >> I have uploaded my Int32# patch on phab as well for reference( >> https://phabricator.haskell.org/D5032) >> >> Thanks, >> Abhiroop >> >> >> >> On Wed, Aug 1, 2018 at 3:45 PM Michal Terepeta >> wrote: >> >>> I've rebased the diff and relaxed the assertion - do take a look if that >>> looks reasonable to you :) >>> https://phabricator.haskell.org/D4475 >>> >>> Cheers! >>> >>> - Michal >>> >>> On Wed, Jul 25, 2018 at 9:03 PM Michal Terepeta < >>> michal.terepeta at gmail.com> wrote: >>> >>>> Hi Carter, >>>> >>>> I didn't write this assertion. I only validated locally (IIRC at the >>>> time I uploaded the diff, harbormaster was failing for some other reasons). >>>> >>>> I'll try to have a look at it this weekend. >>>> >>>> Cheers! >>>> >>>> - Michal >>>> >>>> On Wed, Jul 25, 2018 at 2:16 AM Carter Schonwald < >>>> carter.schonwald at gmail.com> wrote: >>>> >>>>> Michal: did you write this Assert about width? and if so could you >>>>> explain it so we can understand? >>>>> >>>>> hrmm... that code is in the procedure for generating C calls for >>>>> 64bit intel systems >>>>> >>>>> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2541 >>>>> is the top of that routine >>>>> >>>>> and width is defined right below the spot in question >>>>> >>>>> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2764-L2774 >>>>> >>>>> it seems like the code/assertion likely predates michal's work? (eg, a >>>>> trick to make sure that genCCall64 doesn't get invoked by 32bit platforms?) >>>>> >>>>> On Mon, Jul 23, 2018 at 8:54 PM Abhiroop Sarkar >>>>> wrote: >>>>> >>>>>> Hi Michal, >>>>>> >>>>>> In the tests that you have added to D4475, are all the tests running >>>>>> fine? >>>>>> >>>>>> On my machine, I was running the FFI tests( >>>>>> https://github.com/michalt/ghc/blob/int8/testsuite/tests/ffi/should_run/PrimFFIInt8.hs) >>>>>> and they seem to fail at a particular assert statement in the code >>>>>> generator. >>>>>> >>>>>> To be precise this one: >>>>>> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2764 >>>>>> >>>>>> Upon commenting that assert the tests run fine. Am I missing >>>>>> something or is the failure expected? >>>>>> >>>>>> Thanks, >>>>>> Abhiroop >>>>>> >>>>>> On Mon, Jul 9, 2018 at 8:31 PM Michal Terepeta < >>>>>> michal.terepeta at gmail.com> wrote: >>>>>> >>>>>>> Just for the record, I've uploaded the changes to binary: >>>>>>> https://github.com/michalt/packages-binary/tree/int8 >>>>>>> >>>>>>> - Michal >>>>>>> >>>>>>> On Wed, Jul 4, 2018 at 11:07 AM Michal Terepeta < >>>>>>> michal.terepeta at gmail.com> wrote: >>>>>>> >>>>>>>> Yeah, if you look at the linked diff, there are a few tiny changes >>>>>>>> to binary that are necessary. >>>>>>>> >>>>>>>> I'll try to upload them to github later this week. >>>>>>>> >>>>>>>> Ben, is something blocking the review of the diff? I think I >>>>>>>> addressed all comments so far. >>>>>>>> >>>>>>>> - Michal >>>>>>>> >>>>>>>> On Wed, Jul 4, 2018 at 1:38 AM Abhiroop Sarkar < >>>>>>>> asiamgenius at gmail.com> wrote: >>>>>>>> >>>>>>>>> Hello Michal, >>>>>>>>> >>>>>>>>> I was looking at your diff https://phabricator.haskell.org/D4475 >>>>>>>>> and there seems to be some changes that you perhaps made in the binary >>>>>>>>> package ( >>>>>>>>> https://phabricator.haskell.org/differential/changeset/?ref=199152&whitespace=ignore-most). >>>>>>>>> I could not find your version of binary on your github repos list. Is it >>>>>>>>> possible for you to upload that so I can pull those changes? >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> >>>>>>>>> Abhiroop >>>>>>>>> >>>>>>>>> On Mon, May 28, 2018 at 10:45 PM Carter Schonwald < >>>>>>>>> carter.schonwald at gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Abhiroop has the gist, though the size of word args for simd is >>>>>>>>>> more something we want to be consistent between 32/64 bit modes aka ghc >>>>>>>>>> targeting 32 or 64 bit intel with simd support :). It would be best if the >>>>>>>>>> unpack and pack operations that map to and from unboxed tuples where >>>>>>>>>> consistent and precise type wise. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -Carter >>>>>>>>>> >>>>>>>>>> On May 28, 2018, at 5:02 PM, Abhiroop Sarkar < >>>>>>>>>> asiamgenius at gmail.com> wrote: >>>>>>>>>> >>>>>>>>>> Hello Michal, >>>>>>>>>> >>>>>>>>>> My understanding of the issues are much lesser compared to >>>>>>>>>> Carter. However, I will state whatever I understood from discussions with >>>>>>>>>> him. Be warned my understanding might be wrong and Carter might be asking >>>>>>>>>> this for some completely different reason. >>>>>>>>>> >>>>>>>>>> > Out of curiosity, why do you need Word64#/Word32# story to be >>>>>>>>>> fixed for SIMD? >>>>>>>>>> >>>>>>>>>> One of the issues we are dealing with is multiple >>>>>>>>>> microarchitectures(SSE, AVX, AVX2 etc). As a result different >>>>>>>>>> microarchitectures has different ways of handling an 8 bit or 16 bit or 32 >>>>>>>>>> bit unsigned integer. An example I can provide is the vector multiply >>>>>>>>>> instruction which has different semantics of multiplying and storing the >>>>>>>>>> first 16 bits of a 32 bit unsigned integer compared to the lower 16 bits >>>>>>>>>> for each of the elements in its SIMD registers. There will be some other >>>>>>>>>> intricacies around handling a byte sized integer or a 64 bit integer which >>>>>>>>>> will be different from the 32 bit version. >>>>>>>>>> >>>>>>>>>> Basically a 128 bit vector instruction will make some minor >>>>>>>>>> differences when dealing with 2 64 bit integers or 4 32 bit integer or 8 16 >>>>>>>>>> bit integers. >>>>>>>>>> >>>>>>>>>> So I think at the higher level we would want to be as precise as >>>>>>>>>> possible when specifying the datatypes and want things like Word8#, >>>>>>>>>> Word16#, Word32#, Word64# rather than a single ambiguous type like Word. >>>>>>>>>> >>>>>>>>>> One more example is this vector operation like : >>>>>>>>>> broadcastWord64X2# :: Word# >>>>>>>>>> >>>>>>>>>> -> Word64X2# >>>>>>>>>> which >>>>>>>>>> should rather be broadcastWord64X2# :: Word64# >>>>>>>>>> >>>>>>>>>> -> Word64X2# >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Another reason could be improving the space efficiency of packing >>>>>>>>>> various datatypes. This was explained in some detail in this comment: >>>>>>>>>> https://github.com/ghc-proposals/ghc-proposals/pull/74#issuecomment-333951559 >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Abhiroop >>>>>>>>>> >>>>>>>>>> On Mon, May 28, 2018 at 6:06 PM, Michal Terepeta < >>>>>>>>>> michal.terepeta at gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hey Carter, >>>>>>>>>>> >>>>>>>>>>> Yeah, I'm totally snowed under with personal stuff at the >>>>>>>>>>> moment, so not much time to hack on >>>>>>>>>>> anything. I've managed to make some progress on >>>>>>>>>>> https://phabricator.haskell.org/D4475 (prototype >>>>>>>>>>> that adds Int8#/Word8#; waiting for another round of review). >>>>>>>>>>> Sadly June still looks a bit crazy for >>>>>>>>>>> me, so I can't promise anything for Word64#/Word32# (and >>>>>>>>>>> Int64#/Int32#). I hope to have some more >>>>>>>>>>> time in July. >>>>>>>>>>> >>>>>>>>>>> If this is blocking you, then please don't wait for me and go >>>>>>>>>>> ahead (my diff for Int8#/Word8# might >>>>>>>>>>> be a decent starting point to see which places might need >>>>>>>>>>> changes). >>>>>>>>>>> >>>>>>>>>>> Out of curiosity, why do you need Word64#/Word32# story to be >>>>>>>>>>> fixed for SIMD? Cmm already has >>>>>>>>>>> separate types for vector types (see TyCon.hs and VecRep >>>>>>>>>>> constructor of PrimRep). >>>>>>>>>>> >>>>>>>>>>> Cheers! >>>>>>>>>>> >>>>>>>>>>> - Michal >>>>>>>>>>> >>>>>>>>>>> PS. Are you coming to ZuriHac by any chance? >>>>>>>>>>> >>>>>>>>>>> On Mon, May 28, 2018 at 12:37 AM Carter Schonwald < >>>>>>>>>>> carter.schonwald at gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hey Michal! >>>>>>>>>>>> So i'm mentoring Abhiroop around getting SIMD all nice and >>>>>>>>>>>> awesome, and I realized we still dont quite yet have the nice Word32# and >>>>>>>>>>>> Word64# etc story in ghc as yet. >>>>>>>>>>>> >>>>>>>>>>>> Whats the status on that and or is there a way we can help get >>>>>>>>>>>> that over the hump for ghc? These different sized in register ints and >>>>>>>>>>>> friends have a tight partnership with any nice SIMD iterating over the >>>>>>>>>>>> summer, and hopefully helps inform the design >>>>>>>>>>>> >>>>>>>>>>>> -Carter >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Kloona - Coming Soon! >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Kloona - Coming Soon! >>>>>>>>> >>>>>>>> >>>>>> >>>>>> -- >>>>>> Kloona - Coming Soon! >>>>>> >>>>> >> >> -- >> Kloona - Coming Soon! >> > > > -- > Kloona - Coming Soon! > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carter.schonwald at gmail.com Thu Aug 2 03:44:13 2018 From: carter.schonwald at gmail.com (Carter Schonwald) Date: Wed, 1 Aug 2018 23:44:13 -0400 Subject: [Haskell-cafe] Access violation when stack haddock haskell-src-exts since LTS 12.0 In-Reply-To: References: Message-ID: The next version of cabal will have some improvements that folks may appreciate : Namely haddock errors or failures are treated as warnings rather than fatal errors when building packages. This has been merged into master cabal And I’ve been happily using an earlier version of the patch which I helped prototype for at least half a year now. Cheers On Thu, Jul 26, 2018 at 10:53 AM Saurabh Nanda wrote: > Hi, > > How does confirm that this bug is being hit. `stack haddock > haskell-src-exts` on LTS-12.1 fails with the following cryptic error: > > -- While building custom Setup.hs for package haskell-src-exts-1.20.2 > using: > > /home/vl/.stack/setup-exe-cache/x86_64-linux/Cabal-simple_mPHDZzAJ_2.2.0.1_ghc-8.4.3 > --builddir=.stack-work/dist/x86_64-linux/Cabal-2.2.0.1 haddock --html > --hoogle --html-location=../$pkg-$version/ > --haddock-option=--hyperlinked-source > Process exited with code: ExitFailure (-11) > Logs have been written to: > /home/vl/vacationlabs/haskell/.stack-work/logs/haskell-src-exts-1.20.2.log > > The log file doesn't contain an error. > > How does one upgrade the haddock version and get stack to use it? > > -- Saurabh. > > > On Mon, Jul 23, 2018 at 11:58 AM Sven Panne wrote: > >> Am Mo., 23. Juli 2018 um 05:49 Uhr schrieb Yuji Yamamoto < >> whosekiteneverfly at gmail.com>: >> >>> Thank you very much! >>> >>> I confirmed the replaced haddock executable can successfully generate >>> the docs! >>> >> >> Yesterday I had quite some trouble because of the Haddock problem, too, >> and I guess I'm not alone: haskell-src-exts has 165 direct reverse >> dependencies, so probably hundreds of Hackage packages are affected by >> this. The workaround is simple (don't use --haddock with stack), but far >> from satisfying and not very obvious. >> >> Given the fact that this affects a very central piece of the Haskell >> infrastructure in its latest stable incarnation (GHC 8.4.3): Can we have an >> 8.4.4 with a fixed Haddock? >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > > > -- > http://www.saurabhnanda.com > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From asiamgenius at gmail.com Thu Aug 2 03:44:51 2018 From: asiamgenius at gmail.com (Abhiroop Sarkar) Date: Thu, 2 Aug 2018 04:44:51 +0100 Subject: understanding assertions, part deux :) Re: whither built in unlifted Word32# / Word64# etc? In-Reply-To: References: Message-ID: Oh no! it was a Cmm lint error. The register width don't come into the picture before Cmm. The register width in the Cmm is decided based on `primRepCmmType` function(which I mentioned in the previous mail) which in turn uses `Type` and some other info, this is where the error was hiding. It was kind of hard to spot :) On Thu, Aug 2, 2018 at 4:38 AM Carter Schonwald wrote: > So the issue was core lint type error rather than a cmm lint error? Glad > you figured it out ! > > But you didn’t communicate what the error you got from lint was core > rather than cmm ... :( > > On Wed, Aug 1, 2018 at 11:25 PM Abhiroop Sarkar > wrote: > >> Never mind I found the issue and fixed it. >> >> It was the definition of the `Int32` type constructor: >> >> int32PrimTyCon = pcPrimTyCon0 int32PrimTyConName IntRep >> >> which had to be fixed to >> >> int32PrimTyCon = pcPrimTyCon0 int32PrimTyConName Int32Rep >> >> Thanks anyways :) >> >> Abhiroop >> >> On Wed, Aug 1, 2018 at 5:36 PM Abhiroop Sarkar >> wrote: >> >>> Hello, >>> >>> I would appreciate some help in debugging a Cmm Lint error, I have been >>> stuck on for quite a while. >>> >>> Basically I am adding support for Int32# on top of the In8#(D4475) and >>> Int16#(D5006) patches. >>> >>> The Cmm being generated for the test programs are incorrect. >>> >>> Taking a sample test like this ( >>> https://github.com/Abhiroop/ghc-1/blob/int8/testsuite/tests/ffi/should_run/PrimFFIInt32.hs >>> ) >>> >>> The crux of the linting error is the type mismatch between LHS and RHS >>> of the CmmAssign statement. >>> >>> For example in the Int8 patch, the Cmm generated looks like this: >>> >>> ``` >>> _s1MC::I8 = I8[Sp]; // CmmAssign >>> _s1MD::I8 = I8[Sp + 8]; // CmmAssign >>> _s1ME::I8 = I8[Sp + 16]; // CmmAssign >>> _s1MF::I8 = I8[Sp + 24]; // CmmAssign >>> _s1MG::I8 = I8[Sp + 32]; // CmmAssign >>> _s1MH::I8 = I8[Sp + 40]; // CmmAssign >>> _s1MI::I8 = I8[Sp + 48]; // CmmAssign >>> _s1MJ::I8 = I8[Sp + 56]; // CmmAssign >>> _s1MK::I8 = I8[Sp + 64]; // CmmAssign >>> _s1ML::I8 = I8[Sp + 72]; // CmmAssign >>> R6 = %MO_XX_Conv_W8_W64(_s1MG::I8); // CmmAssign >>> R5 = %MO_XX_Conv_W8_W64(_s1MF::I8); // CmmAssign >>> R4 = %MO_XX_Conv_W8_W64(_s1ME::I8); // CmmAssign >>> R3 = %MO_XX_Conv_W8_W64(_s1MD::I8); // CmmAssign >>> R2 = %MO_XX_Conv_W8_W64(_s1MC::I8); // CmmAssign >>> ``` >>> >>> or >>> >>> ``` >>> _c1Oq::I8 = %MO_SS_Conv_W64_W8(9); // CmmAssign >>> _s1N0::I8 = _c1Oq::I8; // CmmAssign >>> _c1Ou::I8 = %MO_SS_Conv_W64_W8(8); // CmmAssign >>> _s1MZ::I8 = _c1Ou::I8; // CmmAssign >>> _c1Ox::I8 = %MO_SS_Conv_W64_W8(7); // CmmAssign >>> _s1MY::I8 = _c1Ox::I8; // CmmAssign >>> _c1OA::I8 = %MO_SS_Conv_W64_W8(6); // CmmAssign >>> _s1MX::I8 = _c1OA::I8; // CmmAssign >>> _c1OD::I8 = %MO_SS_Conv_W64_W8(5); // CmmAssign >>> _s1MW::I8 = _c1OD::I8; // CmmAssign >>> _c1OG::I8 = %MO_SS_Conv_W64_W8(4); // CmmAssign >>> _s1MV::I8 = _c1OG::I8; // CmmAssign >>> _c1OJ::I8 = %MO_SS_Conv_W64_W8(3); // CmmAssign >>> _s1MU::I8 = _c1OJ::I8; // CmmAssign >>> ``` >>> >>> Basically the registers on the left hand side have type `I8`, whereas in >>> my case the registers on the LHS always have the width equal to the word >>> size: >>> >>> ``` >>> _c1Ok::I64 = %MO_SS_Conv_W64_W32(9); // CmmAssign >>> _s1N0::I64 = _c1Ok::I64; // CmmAssign >>> _c1Oo::I64 = %MO_SS_Conv_W64_W32(8); // CmmAssign >>> _s1MZ::I64 = _c1Oo::I64; // CmmAssign >>> _c1Or::I64 = %MO_SS_Conv_W64_W32(7); // CmmAssign >>> _s1MY::I64 = _c1Or::I64; // CmmAssign >>> _c1Ou::I64 = %MO_SS_Conv_W64_W32(6); // CmmAssign >>> _s1MX::I64 = _c1Ou::I64; // CmmAssign >>> _c1Ox::I64 = %MO_SS_Conv_W64_W32(5); // CmmAssign >>> _s1MW::I64 = _c1Ox::I64; // CmmAssign >>> _c1OA::I64 = %MO_SS_Conv_W64_W32(4); // CmmAssign >>> _s1MV::I64 = _c1OA::I64; // CmmAssign >>> _c1OD::I64 = %MO_SS_Conv_W64_W32(3); // CmmAssign >>> _s1MU::I64 = _c1OD::I64; // CmmAssign >>> _c1OG::I64 = %MO_SS_Conv_W64_W32(2); // CmmAssign >>> _s1MT::I64 = _c1OG::I64; // CmmAssign >>> ``` >>> I would ideally want to have the datatype of the LHS be `I32`. >>> >>> Michal, I thought this type is picked up using the `primRepCmmType` >>> function >>> https://github.com/Abhiroop/ghc-1/blob/int8/compiler/cmm/CmmUtils.hs#L104-L105 which >>> I modified but I am not sure why the LHS types of the CmmAssign statement >>> allocate `I64` instead of `I32`. Is there any other change on your >>> patch(D4475) which I might have overlooked. >>> >>> I have uploaded my Int32# patch on phab as well for reference( >>> https://phabricator.haskell.org/D5032) >>> >>> Thanks, >>> Abhiroop >>> >>> >>> >>> On Wed, Aug 1, 2018 at 3:45 PM Michal Terepeta < >>> michal.terepeta at gmail.com> wrote: >>> >>>> I've rebased the diff and relaxed the assertion - do take a look if >>>> that looks reasonable to you :) >>>> https://phabricator.haskell.org/D4475 >>>> >>>> Cheers! >>>> >>>> - Michal >>>> >>>> On Wed, Jul 25, 2018 at 9:03 PM Michal Terepeta < >>>> michal.terepeta at gmail.com> wrote: >>>> >>>>> Hi Carter, >>>>> >>>>> I didn't write this assertion. I only validated locally (IIRC at the >>>>> time I uploaded the diff, harbormaster was failing for some other reasons). >>>>> >>>>> I'll try to have a look at it this weekend. >>>>> >>>>> Cheers! >>>>> >>>>> - Michal >>>>> >>>>> On Wed, Jul 25, 2018 at 2:16 AM Carter Schonwald < >>>>> carter.schonwald at gmail.com> wrote: >>>>> >>>>>> Michal: did you write this Assert about width? and if so could you >>>>>> explain it so we can understand? >>>>>> >>>>>> hrmm... that code is in the procedure for generating C calls for >>>>>> 64bit intel systems >>>>>> >>>>>> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2541 >>>>>> is the top of that routine >>>>>> >>>>>> and width is defined right below the spot in question >>>>>> >>>>>> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2764-L2774 >>>>>> >>>>>> it seems like the code/assertion likely predates michal's work? (eg, >>>>>> a trick to make sure that genCCall64 doesn't get invoked by 32bit >>>>>> platforms?) >>>>>> >>>>>> On Mon, Jul 23, 2018 at 8:54 PM Abhiroop Sarkar < >>>>>> asiamgenius at gmail.com> wrote: >>>>>> >>>>>>> Hi Michal, >>>>>>> >>>>>>> In the tests that you have added to D4475, are all the tests running >>>>>>> fine? >>>>>>> >>>>>>> On my machine, I was running the FFI tests( >>>>>>> https://github.com/michalt/ghc/blob/int8/testsuite/tests/ffi/should_run/PrimFFIInt8.hs) >>>>>>> and they seem to fail at a particular assert statement in the code >>>>>>> generator. >>>>>>> >>>>>>> To be precise this one: >>>>>>> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2764 >>>>>>> >>>>>>> Upon commenting that assert the tests run fine. Am I missing >>>>>>> something or is the failure expected? >>>>>>> >>>>>>> Thanks, >>>>>>> Abhiroop >>>>>>> >>>>>>> On Mon, Jul 9, 2018 at 8:31 PM Michal Terepeta < >>>>>>> michal.terepeta at gmail.com> wrote: >>>>>>> >>>>>>>> Just for the record, I've uploaded the changes to binary: >>>>>>>> https://github.com/michalt/packages-binary/tree/int8 >>>>>>>> >>>>>>>> - Michal >>>>>>>> >>>>>>>> On Wed, Jul 4, 2018 at 11:07 AM Michal Terepeta < >>>>>>>> michal.terepeta at gmail.com> wrote: >>>>>>>> >>>>>>>>> Yeah, if you look at the linked diff, there are a few tiny changes >>>>>>>>> to binary that are necessary. >>>>>>>>> >>>>>>>>> I'll try to upload them to github later this week. >>>>>>>>> >>>>>>>>> Ben, is something blocking the review of the diff? I think I >>>>>>>>> addressed all comments so far. >>>>>>>>> >>>>>>>>> - Michal >>>>>>>>> >>>>>>>>> On Wed, Jul 4, 2018 at 1:38 AM Abhiroop Sarkar < >>>>>>>>> asiamgenius at gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hello Michal, >>>>>>>>>> >>>>>>>>>> I was looking at your diff https://phabricator.haskell.org/D4475 >>>>>>>>>> and there seems to be some changes that you perhaps made in the binary >>>>>>>>>> package ( >>>>>>>>>> https://phabricator.haskell.org/differential/changeset/?ref=199152&whitespace=ignore-most). >>>>>>>>>> I could not find your version of binary on your github repos list. Is it >>>>>>>>>> possible for you to upload that so I can pull those changes? >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> Abhiroop >>>>>>>>>> >>>>>>>>>> On Mon, May 28, 2018 at 10:45 PM Carter Schonwald < >>>>>>>>>> carter.schonwald at gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Abhiroop has the gist, though the size of word args for simd is >>>>>>>>>>> more something we want to be consistent between 32/64 bit modes aka ghc >>>>>>>>>>> targeting 32 or 64 bit intel with simd support :). It would be best if the >>>>>>>>>>> unpack and pack operations that map to and from unboxed tuples where >>>>>>>>>>> consistent and precise type wise. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -Carter >>>>>>>>>>> >>>>>>>>>>> On May 28, 2018, at 5:02 PM, Abhiroop Sarkar < >>>>>>>>>>> asiamgenius at gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>> Hello Michal, >>>>>>>>>>> >>>>>>>>>>> My understanding of the issues are much lesser compared to >>>>>>>>>>> Carter. However, I will state whatever I understood from discussions with >>>>>>>>>>> him. Be warned my understanding might be wrong and Carter might be asking >>>>>>>>>>> this for some completely different reason. >>>>>>>>>>> >>>>>>>>>>> > Out of curiosity, why do you need Word64#/Word32# story to be >>>>>>>>>>> fixed for SIMD? >>>>>>>>>>> >>>>>>>>>>> One of the issues we are dealing with is multiple >>>>>>>>>>> microarchitectures(SSE, AVX, AVX2 etc). As a result different >>>>>>>>>>> microarchitectures has different ways of handling an 8 bit or 16 bit or 32 >>>>>>>>>>> bit unsigned integer. An example I can provide is the vector multiply >>>>>>>>>>> instruction which has different semantics of multiplying and storing the >>>>>>>>>>> first 16 bits of a 32 bit unsigned integer compared to the lower 16 bits >>>>>>>>>>> for each of the elements in its SIMD registers. There will be some other >>>>>>>>>>> intricacies around handling a byte sized integer or a 64 bit integer which >>>>>>>>>>> will be different from the 32 bit version. >>>>>>>>>>> >>>>>>>>>>> Basically a 128 bit vector instruction will make some minor >>>>>>>>>>> differences when dealing with 2 64 bit integers or 4 32 bit integer or 8 16 >>>>>>>>>>> bit integers. >>>>>>>>>>> >>>>>>>>>>> So I think at the higher level we would want to be as precise as >>>>>>>>>>> possible when specifying the datatypes and want things like Word8#, >>>>>>>>>>> Word16#, Word32#, Word64# rather than a single ambiguous type like Word. >>>>>>>>>>> >>>>>>>>>>> One more example is this vector operation like : >>>>>>>>>>> broadcastWord64X2# :: Word# >>>>>>>>>>> >>>>>>>>>>> -> Word64X2# >>>>>>>>>>> which >>>>>>>>>>> should rather be broadcastWord64X2# :: Word64# >>>>>>>>>>> >>>>>>>>>>> -> Word64X2# >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Another reason could be improving the space efficiency of >>>>>>>>>>> packing various datatypes. This was explained in some detail in this >>>>>>>>>>> comment: >>>>>>>>>>> https://github.com/ghc-proposals/ghc-proposals/pull/74#issuecomment-333951559 >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Abhiroop >>>>>>>>>>> >>>>>>>>>>> On Mon, May 28, 2018 at 6:06 PM, Michal Terepeta < >>>>>>>>>>> michal.terepeta at gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hey Carter, >>>>>>>>>>>> >>>>>>>>>>>> Yeah, I'm totally snowed under with personal stuff at the >>>>>>>>>>>> moment, so not much time to hack on >>>>>>>>>>>> anything. I've managed to make some progress on >>>>>>>>>>>> https://phabricator.haskell.org/D4475 (prototype >>>>>>>>>>>> that adds Int8#/Word8#; waiting for another round of review). >>>>>>>>>>>> Sadly June still looks a bit crazy for >>>>>>>>>>>> me, so I can't promise anything for Word64#/Word32# (and >>>>>>>>>>>> Int64#/Int32#). I hope to have some more >>>>>>>>>>>> time in July. >>>>>>>>>>>> >>>>>>>>>>>> If this is blocking you, then please don't wait for me and go >>>>>>>>>>>> ahead (my diff for Int8#/Word8# might >>>>>>>>>>>> be a decent starting point to see which places might need >>>>>>>>>>>> changes). >>>>>>>>>>>> >>>>>>>>>>>> Out of curiosity, why do you need Word64#/Word32# story to be >>>>>>>>>>>> fixed for SIMD? Cmm already has >>>>>>>>>>>> separate types for vector types (see TyCon.hs and VecRep >>>>>>>>>>>> constructor of PrimRep). >>>>>>>>>>>> >>>>>>>>>>>> Cheers! >>>>>>>>>>>> >>>>>>>>>>>> - Michal >>>>>>>>>>>> >>>>>>>>>>>> PS. Are you coming to ZuriHac by any chance? >>>>>>>>>>>> >>>>>>>>>>>> On Mon, May 28, 2018 at 12:37 AM Carter Schonwald < >>>>>>>>>>>> carter.schonwald at gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hey Michal! >>>>>>>>>>>>> So i'm mentoring Abhiroop around getting SIMD all nice and >>>>>>>>>>>>> awesome, and I realized we still dont quite yet have the nice Word32# and >>>>>>>>>>>>> Word64# etc story in ghc as yet. >>>>>>>>>>>>> >>>>>>>>>>>>> Whats the status on that and or is there a way we can help get >>>>>>>>>>>>> that over the hump for ghc? These different sized in register ints and >>>>>>>>>>>>> friends have a tight partnership with any nice SIMD iterating over the >>>>>>>>>>>>> summer, and hopefully helps inform the design >>>>>>>>>>>>> >>>>>>>>>>>>> -Carter >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Kloona - Coming Soon! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Kloona - Coming Soon! >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Kloona - Coming Soon! >>>>>>> >>>>>> >>> >>> -- >>> Kloona - Coming Soon! >>> >> >> >> -- >> Kloona - Coming Soon! >> > -- Kloona - Coming Soon! -------------- next part -------------- An HTML attachment was scrubbed... URL: From carter.schonwald at gmail.com Thu Aug 2 04:36:28 2018 From: carter.schonwald at gmail.com (Carter Schonwald) Date: Thu, 2 Aug 2018 00:36:28 -0400 Subject: understanding assertions, part deux :) Re: whither built in unlifted Word32# / Word64# etc? In-Reply-To: References: Message-ID: Oh ok. The fix sounded like it was about core. My mistake on reading your explanation earlier ! Thanks for clarifying! On Wed, Aug 1, 2018 at 11:45 PM Abhiroop Sarkar wrote: > Oh no! it was a Cmm lint error. The register width don't come into the > picture before Cmm. The register width in the Cmm is decided based on > `primRepCmmType` function(which I mentioned in the previous mail) which in > turn uses `Type` and some other info, this is where the error was hiding. > It was kind of hard to spot :) > > On Thu, Aug 2, 2018 at 4:38 AM Carter Schonwald < > carter.schonwald at gmail.com> wrote: > >> So the issue was core lint type error rather than a cmm lint error? Glad >> you figured it out ! >> >> But you didn’t communicate what the error you got from lint was core >> rather than cmm ... :( >> >> On Wed, Aug 1, 2018 at 11:25 PM Abhiroop Sarkar >> wrote: >> >>> Never mind I found the issue and fixed it. >>> >>> It was the definition of the `Int32` type constructor: >>> >>> int32PrimTyCon = pcPrimTyCon0 int32PrimTyConName IntRep >>> >>> which had to be fixed to >>> >>> int32PrimTyCon = pcPrimTyCon0 int32PrimTyConName Int32Rep >>> >>> Thanks anyways :) >>> >>> Abhiroop >>> >>> On Wed, Aug 1, 2018 at 5:36 PM Abhiroop Sarkar >>> wrote: >>> >>>> Hello, >>>> >>>> I would appreciate some help in debugging a Cmm Lint error, I have been >>>> stuck on for quite a while. >>>> >>>> Basically I am adding support for Int32# on top of the In8#(D4475) and >>>> Int16#(D5006) patches. >>>> >>>> The Cmm being generated for the test programs are incorrect. >>>> >>>> Taking a sample test like this ( >>>> https://github.com/Abhiroop/ghc-1/blob/int8/testsuite/tests/ffi/should_run/PrimFFIInt32.hs >>>> ) >>>> >>>> The crux of the linting error is the type mismatch between LHS and RHS >>>> of the CmmAssign statement. >>>> >>>> For example in the Int8 patch, the Cmm generated looks like this: >>>> >>>> ``` >>>> _s1MC::I8 = I8[Sp]; // CmmAssign >>>> _s1MD::I8 = I8[Sp + 8]; // CmmAssign >>>> _s1ME::I8 = I8[Sp + 16]; // CmmAssign >>>> _s1MF::I8 = I8[Sp + 24]; // CmmAssign >>>> _s1MG::I8 = I8[Sp + 32]; // CmmAssign >>>> _s1MH::I8 = I8[Sp + 40]; // CmmAssign >>>> _s1MI::I8 = I8[Sp + 48]; // CmmAssign >>>> _s1MJ::I8 = I8[Sp + 56]; // CmmAssign >>>> _s1MK::I8 = I8[Sp + 64]; // CmmAssign >>>> _s1ML::I8 = I8[Sp + 72]; // CmmAssign >>>> R6 = %MO_XX_Conv_W8_W64(_s1MG::I8); // CmmAssign >>>> R5 = %MO_XX_Conv_W8_W64(_s1MF::I8); // CmmAssign >>>> R4 = %MO_XX_Conv_W8_W64(_s1ME::I8); // CmmAssign >>>> R3 = %MO_XX_Conv_W8_W64(_s1MD::I8); // CmmAssign >>>> R2 = %MO_XX_Conv_W8_W64(_s1MC::I8); // CmmAssign >>>> ``` >>>> >>>> or >>>> >>>> ``` >>>> _c1Oq::I8 = %MO_SS_Conv_W64_W8(9); // CmmAssign >>>> _s1N0::I8 = _c1Oq::I8; // CmmAssign >>>> _c1Ou::I8 = %MO_SS_Conv_W64_W8(8); // CmmAssign >>>> _s1MZ::I8 = _c1Ou::I8; // CmmAssign >>>> _c1Ox::I8 = %MO_SS_Conv_W64_W8(7); // CmmAssign >>>> _s1MY::I8 = _c1Ox::I8; // CmmAssign >>>> _c1OA::I8 = %MO_SS_Conv_W64_W8(6); // CmmAssign >>>> _s1MX::I8 = _c1OA::I8; // CmmAssign >>>> _c1OD::I8 = %MO_SS_Conv_W64_W8(5); // CmmAssign >>>> _s1MW::I8 = _c1OD::I8; // CmmAssign >>>> _c1OG::I8 = %MO_SS_Conv_W64_W8(4); // CmmAssign >>>> _s1MV::I8 = _c1OG::I8; // CmmAssign >>>> _c1OJ::I8 = %MO_SS_Conv_W64_W8(3); // CmmAssign >>>> _s1MU::I8 = _c1OJ::I8; // CmmAssign >>>> ``` >>>> >>>> Basically the registers on the left hand side have type `I8`, whereas >>>> in my case the registers on the LHS always have the width equal to the word >>>> size: >>>> >>>> ``` >>>> _c1Ok::I64 = %MO_SS_Conv_W64_W32(9); // CmmAssign >>>> _s1N0::I64 = _c1Ok::I64; // CmmAssign >>>> _c1Oo::I64 = %MO_SS_Conv_W64_W32(8); // CmmAssign >>>> _s1MZ::I64 = _c1Oo::I64; // CmmAssign >>>> _c1Or::I64 = %MO_SS_Conv_W64_W32(7); // CmmAssign >>>> _s1MY::I64 = _c1Or::I64; // CmmAssign >>>> _c1Ou::I64 = %MO_SS_Conv_W64_W32(6); // CmmAssign >>>> _s1MX::I64 = _c1Ou::I64; // CmmAssign >>>> _c1Ox::I64 = %MO_SS_Conv_W64_W32(5); // CmmAssign >>>> _s1MW::I64 = _c1Ox::I64; // CmmAssign >>>> _c1OA::I64 = %MO_SS_Conv_W64_W32(4); // CmmAssign >>>> _s1MV::I64 = _c1OA::I64; // CmmAssign >>>> _c1OD::I64 = %MO_SS_Conv_W64_W32(3); // CmmAssign >>>> _s1MU::I64 = _c1OD::I64; // CmmAssign >>>> _c1OG::I64 = %MO_SS_Conv_W64_W32(2); // CmmAssign >>>> _s1MT::I64 = _c1OG::I64; // CmmAssign >>>> ``` >>>> I would ideally want to have the datatype of the LHS be `I32`. >>>> >>>> Michal, I thought this type is picked up using the `primRepCmmType` >>>> function >>>> https://github.com/Abhiroop/ghc-1/blob/int8/compiler/cmm/CmmUtils.hs#L104-L105 which >>>> I modified but I am not sure why the LHS types of the CmmAssign statement >>>> allocate `I64` instead of `I32`. Is there any other change on your >>>> patch(D4475) which I might have overlooked. >>>> >>>> I have uploaded my Int32# patch on phab as well for reference( >>>> https://phabricator.haskell.org/D5032) >>>> >>>> Thanks, >>>> Abhiroop >>>> >>>> >>>> >>>> On Wed, Aug 1, 2018 at 3:45 PM Michal Terepeta < >>>> michal.terepeta at gmail.com> wrote: >>>> >>>>> I've rebased the diff and relaxed the assertion - do take a look if >>>>> that looks reasonable to you :) >>>>> https://phabricator.haskell.org/D4475 >>>>> >>>>> Cheers! >>>>> >>>>> - Michal >>>>> >>>>> On Wed, Jul 25, 2018 at 9:03 PM Michal Terepeta < >>>>> michal.terepeta at gmail.com> wrote: >>>>> >>>>>> Hi Carter, >>>>>> >>>>>> I didn't write this assertion. I only validated locally (IIRC at the >>>>>> time I uploaded the diff, harbormaster was failing for some other reasons). >>>>>> >>>>>> I'll try to have a look at it this weekend. >>>>>> >>>>>> Cheers! >>>>>> >>>>>> - Michal >>>>>> >>>>>> On Wed, Jul 25, 2018 at 2:16 AM Carter Schonwald < >>>>>> carter.schonwald at gmail.com> wrote: >>>>>> >>>>>>> Michal: did you write this Assert about width? and if so could you >>>>>>> explain it so we can understand? >>>>>>> >>>>>>> hrmm... that code is in the procedure for generating C calls for >>>>>>> 64bit intel systems >>>>>>> >>>>>>> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2541 >>>>>>> is the top of that routine >>>>>>> >>>>>>> and width is defined right below the spot in question >>>>>>> >>>>>>> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2764-L2774 >>>>>>> >>>>>>> it seems like the code/assertion likely predates michal's work? (eg, >>>>>>> a trick to make sure that genCCall64 doesn't get invoked by 32bit >>>>>>> platforms?) >>>>>>> >>>>>>> On Mon, Jul 23, 2018 at 8:54 PM Abhiroop Sarkar < >>>>>>> asiamgenius at gmail.com> wrote: >>>>>>> >>>>>>>> Hi Michal, >>>>>>>> >>>>>>>> In the tests that you have added to D4475, are all the tests >>>>>>>> running fine? >>>>>>>> >>>>>>>> On my machine, I was running the FFI tests( >>>>>>>> https://github.com/michalt/ghc/blob/int8/testsuite/tests/ffi/should_run/PrimFFIInt8.hs) >>>>>>>> and they seem to fail at a particular assert statement in the code >>>>>>>> generator. >>>>>>>> >>>>>>>> To be precise this one: >>>>>>>> https://github.com/michalt/ghc/blob/int8/compiler/nativeGen/X86/CodeGen.hs#L2764 >>>>>>>> >>>>>>>> Upon commenting that assert the tests run fine. Am I missing >>>>>>>> something or is the failure expected? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Abhiroop >>>>>>>> >>>>>>>> On Mon, Jul 9, 2018 at 8:31 PM Michal Terepeta < >>>>>>>> michal.terepeta at gmail.com> wrote: >>>>>>>> >>>>>>>>> Just for the record, I've uploaded the changes to binary: >>>>>>>>> https://github.com/michalt/packages-binary/tree/int8 >>>>>>>>> >>>>>>>>> - Michal >>>>>>>>> >>>>>>>>> On Wed, Jul 4, 2018 at 11:07 AM Michal Terepeta < >>>>>>>>> michal.terepeta at gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Yeah, if you look at the linked diff, there are a few tiny >>>>>>>>>> changes to binary that are necessary. >>>>>>>>>> >>>>>>>>>> I'll try to upload them to github later this week. >>>>>>>>>> >>>>>>>>>> Ben, is something blocking the review of the diff? I think I >>>>>>>>>> addressed all comments so far. >>>>>>>>>> >>>>>>>>>> - Michal >>>>>>>>>> >>>>>>>>>> On Wed, Jul 4, 2018 at 1:38 AM Abhiroop Sarkar < >>>>>>>>>> asiamgenius at gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hello Michal, >>>>>>>>>>> >>>>>>>>>>> I was looking at your diff https://phabricator.haskell.org/D4475 >>>>>>>>>>> and there seems to be some changes that you perhaps made in the binary >>>>>>>>>>> package ( >>>>>>>>>>> https://phabricator.haskell.org/differential/changeset/?ref=199152&whitespace=ignore-most). >>>>>>>>>>> I could not find your version of binary on your github repos list. Is it >>>>>>>>>>> possible for you to upload that so I can pull those changes? >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> >>>>>>>>>>> Abhiroop >>>>>>>>>>> >>>>>>>>>>> On Mon, May 28, 2018 at 10:45 PM Carter Schonwald < >>>>>>>>>>> carter.schonwald at gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Abhiroop has the gist, though the size of word args for simd is >>>>>>>>>>>> more something we want to be consistent between 32/64 bit modes aka ghc >>>>>>>>>>>> targeting 32 or 64 bit intel with simd support :). It would be best if the >>>>>>>>>>>> unpack and pack operations that map to and from unboxed tuples where >>>>>>>>>>>> consistent and precise type wise. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -Carter >>>>>>>>>>>> >>>>>>>>>>>> On May 28, 2018, at 5:02 PM, Abhiroop Sarkar < >>>>>>>>>>>> asiamgenius at gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hello Michal, >>>>>>>>>>>> >>>>>>>>>>>> My understanding of the issues are much lesser compared to >>>>>>>>>>>> Carter. However, I will state whatever I understood from discussions with >>>>>>>>>>>> him. Be warned my understanding might be wrong and Carter might be asking >>>>>>>>>>>> this for some completely different reason. >>>>>>>>>>>> >>>>>>>>>>>> > Out of curiosity, why do you need Word64#/Word32# story to be >>>>>>>>>>>> fixed for SIMD? >>>>>>>>>>>> >>>>>>>>>>>> One of the issues we are dealing with is multiple >>>>>>>>>>>> microarchitectures(SSE, AVX, AVX2 etc). As a result different >>>>>>>>>>>> microarchitectures has different ways of handling an 8 bit or 16 bit or 32 >>>>>>>>>>>> bit unsigned integer. An example I can provide is the vector multiply >>>>>>>>>>>> instruction which has different semantics of multiplying and storing the >>>>>>>>>>>> first 16 bits of a 32 bit unsigned integer compared to the lower 16 bits >>>>>>>>>>>> for each of the elements in its SIMD registers. There will be some other >>>>>>>>>>>> intricacies around handling a byte sized integer or a 64 bit integer which >>>>>>>>>>>> will be different from the 32 bit version. >>>>>>>>>>>> >>>>>>>>>>>> Basically a 128 bit vector instruction will make some minor >>>>>>>>>>>> differences when dealing with 2 64 bit integers or 4 32 bit integer or 8 16 >>>>>>>>>>>> bit integers. >>>>>>>>>>>> >>>>>>>>>>>> So I think at the higher level we would want to be as precise >>>>>>>>>>>> as possible when specifying the datatypes and want things like Word8#, >>>>>>>>>>>> Word16#, Word32#, Word64# rather than a single ambiguous type like Word. >>>>>>>>>>>> >>>>>>>>>>>> One more example is this vector operation like : >>>>>>>>>>>> broadcastWord64X2# :: Word# >>>>>>>>>>>> >>>>>>>>>>>> -> Word64X2# >>>>>>>>>>>> which >>>>>>>>>>>> should rather be broadcastWord64X2# :: Word64# >>>>>>>>>>>> >>>>>>>>>>>> -> Word64X2# >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Another reason could be improving the space efficiency of >>>>>>>>>>>> packing various datatypes. This was explained in some detail in this >>>>>>>>>>>> comment: >>>>>>>>>>>> https://github.com/ghc-proposals/ghc-proposals/pull/74#issuecomment-333951559 >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Abhiroop >>>>>>>>>>>> >>>>>>>>>>>> On Mon, May 28, 2018 at 6:06 PM, Michal Terepeta < >>>>>>>>>>>> michal.terepeta at gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hey Carter, >>>>>>>>>>>>> >>>>>>>>>>>>> Yeah, I'm totally snowed under with personal stuff at the >>>>>>>>>>>>> moment, so not much time to hack on >>>>>>>>>>>>> anything. I've managed to make some progress on >>>>>>>>>>>>> https://phabricator.haskell.org/D4475 (prototype >>>>>>>>>>>>> that adds Int8#/Word8#; waiting for another round of review). >>>>>>>>>>>>> Sadly June still looks a bit crazy for >>>>>>>>>>>>> me, so I can't promise anything for Word64#/Word32# (and >>>>>>>>>>>>> Int64#/Int32#). I hope to have some more >>>>>>>>>>>>> time in July. >>>>>>>>>>>>> >>>>>>>>>>>>> If this is blocking you, then please don't wait for me and go >>>>>>>>>>>>> ahead (my diff for Int8#/Word8# might >>>>>>>>>>>>> be a decent starting point to see which places might need >>>>>>>>>>>>> changes). >>>>>>>>>>>>> >>>>>>>>>>>>> Out of curiosity, why do you need Word64#/Word32# story to be >>>>>>>>>>>>> fixed for SIMD? Cmm already has >>>>>>>>>>>>> separate types for vector types (see TyCon.hs and VecRep >>>>>>>>>>>>> constructor of PrimRep). >>>>>>>>>>>>> >>>>>>>>>>>>> Cheers! >>>>>>>>>>>>> >>>>>>>>>>>>> - Michal >>>>>>>>>>>>> >>>>>>>>>>>>> PS. Are you coming to ZuriHac by any chance? >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, May 28, 2018 at 12:37 AM Carter Schonwald < >>>>>>>>>>>>> carter.schonwald at gmail.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hey Michal! >>>>>>>>>>>>>> So i'm mentoring Abhiroop around getting SIMD all nice and >>>>>>>>>>>>>> awesome, and I realized we still dont quite yet have the nice Word32# and >>>>>>>>>>>>>> Word64# etc story in ghc as yet. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Whats the status on that and or is there a way we can help >>>>>>>>>>>>>> get that over the hump for ghc? These different sized in register ints and >>>>>>>>>>>>>> friends have a tight partnership with any nice SIMD iterating over the >>>>>>>>>>>>>> summer, and hopefully helps inform the design >>>>>>>>>>>>>> >>>>>>>>>>>>>> -Carter >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Kloona - Coming Soon! >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Kloona - Coming Soon! >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Kloona - Coming Soon! >>>>>>>> >>>>>>> >>>> >>>> -- >>>> Kloona - Coming Soon! >>>> >>> >>> >>> -- >>> Kloona - Coming Soon! >>> >> > > -- > Kloona - Coming Soon! > -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.pelenitsyn at gmail.com Thu Aug 2 05:16:10 2018 From: a.pelenitsyn at gmail.com (Artem Pelenitsyn) Date: Thu, 2 Aug 2018 08:16:10 +0300 Subject: accuracy of asinh and atanh In-Reply-To: References: <87y3dpy2lg.fsf@smart-cactus.org> Message-ID: Here is the patch: https://phabricator.haskell.org/D5034 -- Best, Artem On Thu, 2 Aug 2018 at 06:26 Artem Pelenitsyn wrote: > I'd be willing to do this. > > -- > Best wishes, > Artem > > > On Thu, 2 Aug 2018, 04:38 Matt Peddie, wrote: > >> Thanks, Ben, for chiming in. I think calling out to C for these >> functions is the way to go if it's now feasible. (Calling out to libm >> is the workaround I'm using in the application that led me to discover >> the inaccuracy.) >> >> Matt >> >> On Thu, Aug 2, 2018 at 11:33 AM, Ben Gamari wrote: >> > Matt Peddie writes: >> > >> >> Hi George, >> >> >> >> Not a stupid question. I don't have a single source at hand, but I >> >> think I read in a few places on the wiki that calling out to the >> >> system math library is not an option due to the variety of system math >> >> libraries on the platforms GHC supports. It'd be great if I got the >> >> wrong impression and this could just be a call to C. Can anyone set >> >> me straight on this point? >> >> >> > Indeed it's not a stupid question at all. Indeed this is precisely what >> > we do for the simpler transcendentals (e.g. sin, asin, log). We very >> > well could move in this direction in the case of asinh/atanh as well. I >> > believe the reason we don't currently is that atanh was only >> > standardized in C99, which we only started requiring a few releases ago. >> > Perhaps this is ultimately the right direction. >> > >> > Cheers, >> > >> > - Ben >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at well-typed.com Thu Aug 2 20:19:28 2018 From: david at well-typed.com (David Feuer) Date: Thu, 02 Aug 2018 16:19:28 -0400 Subject: accuracy of asinh and atanh In-Reply-To: References: Message-ID: <1891634.5OVUaOt0BE@squirrel> Wasn't there a very recent commit to improve these functions, by leftaroundabout? On Thursday, August 2, 2018 8:16:10 AM EDT Artem Pelenitsyn wrote: > Here is the patch: https://phabricator.haskell.org/D5034 > > -- > Best, Artem > > On Thu, 2 Aug 2018 at 06:26 Artem Pelenitsyn wrote: > > > I'd be willing to do this. > > > > -- > > Best wishes, > > Artem > > > > > > On Thu, 2 Aug 2018, 04:38 Matt Peddie, wrote: > > > >> Thanks, Ben, for chiming in. I think calling out to C for these > >> functions is the way to go if it's now feasible. (Calling out to libm > >> is the workaround I'm using in the application that led me to discover > >> the inaccuracy.) > >> > >> Matt > >> > >> On Thu, Aug 2, 2018 at 11:33 AM, Ben Gamari wrote: > >> > Matt Peddie writes: > >> > > >> >> Hi George, > >> >> > >> >> Not a stupid question. I don't have a single source at hand, but I > >> >> think I read in a few places on the wiki that calling out to the > >> >> system math library is not an option due to the variety of system math > >> >> libraries on the platforms GHC supports. It'd be great if I got the > >> >> wrong impression and this could just be a call to C. Can anyone set > >> >> me straight on this point? > >> >> > >> > Indeed it's not a stupid question at all. Indeed this is precisely what > >> > we do for the simpler transcendentals (e.g. sin, asin, log). We very > >> > well could move in this direction in the case of asinh/atanh as well. I > >> > believe the reason we don't currently is that atanh was only > >> > standardized in C99, which we only started requiring a few releases ago. > >> > Perhaps this is ultimately the right direction. > >> > > >> > Cheers, > >> > > >> > - Ben > >> _______________________________________________ > >> ghc-devs mailing list > >> ghc-devs at haskell.org > >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > >> > > > -- David Feuer Well-Typed From a.pelenitsyn at gmail.com Thu Aug 2 20:48:18 2018 From: a.pelenitsyn at gmail.com (Artem Pelenitsyn) Date: Thu, 2 Aug 2018 23:48:18 +0300 Subject: accuracy of asinh and atanh In-Reply-To: <1891634.5OVUaOt0BE@squirrel> References: <1891634.5OVUaOt0BE@squirrel> Message-ID: Thanks David! Indeed, here is the commit and ticket: https://github.com/ghc/ghc/commit/3ea33411d7cbf32c20940cc72ca07df6830eeed7 https://ghc.haskell.org/trac/ghc/ticket/14927 This concerns only `asinh` though. The implementation is closer to what Matt proposes in his package but simpler. Nevertheless, the original issue about `Infinity` on large negative numbers seems to be fixed with this. So, I guess, feel free to kill the patch. -- Best, Artem On Thu, 2 Aug 2018 at 23:19 David Feuer wrote: > Wasn't there a very recent commit to improve these functions, by > leftaroundabout? > > On Thursday, August 2, 2018 8:16:10 AM EDT Artem Pelenitsyn wrote: > > Here is the patch: https://phabricator.haskell.org/D5034 > > > > -- > > Best, Artem > > > > On Thu, 2 Aug 2018 at 06:26 Artem Pelenitsyn > wrote: > > > > > I'd be willing to do this. > > > > > > -- > > > Best wishes, > > > Artem > > > > > > > > > On Thu, 2 Aug 2018, 04:38 Matt Peddie, wrote: > > > > > >> Thanks, Ben, for chiming in. I think calling out to C for these > > >> functions is the way to go if it's now feasible. (Calling out to libm > > >> is the workaround I'm using in the application that led me to discover > > >> the inaccuracy.) > > >> > > >> Matt > > >> > > >> On Thu, Aug 2, 2018 at 11:33 AM, Ben Gamari > wrote: > > >> > Matt Peddie writes: > > >> > > > >> >> Hi George, > > >> >> > > >> >> Not a stupid question. I don't have a single source at hand, but I > > >> >> think I read in a few places on the wiki that calling out to the > > >> >> system math library is not an option due to the variety of system > math > > >> >> libraries on the platforms GHC supports. It'd be great if I got > the > > >> >> wrong impression and this could just be a call to C. Can anyone > set > > >> >> me straight on this point? > > >> >> > > >> > Indeed it's not a stupid question at all. Indeed this is precisely > what > > >> > we do for the simpler transcendentals (e.g. sin, asin, log). We very > > >> > well could move in this direction in the case of asinh/atanh as > well. I > > >> > believe the reason we don't currently is that atanh was only > > >> > standardized in C99, which we only started requiring a few releases > ago. > > >> > Perhaps this is ultimately the right direction. > > >> > > > >> > Cheers, > > >> > > > >> > - Ben > > >> _______________________________________________ > > >> ghc-devs mailing list > > >> ghc-devs at haskell.org > > >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > >> > > > > > > > > -- > David Feuer > Well-Typed > -------------- next part -------------- An HTML attachment was scrubbed... URL: From george.colpitts at gmail.com Fri Aug 3 15:24:31 2018 From: george.colpitts at gmail.com (George Colpitts) Date: Fri, 3 Aug 2018 12:24:31 -0300 Subject: accuracy of asinh and atanh In-Reply-To: <1891634.5OVUaOt0BE@squirrel> References: <1891634.5OVUaOt0BE@squirrel> Message-ID: I believe there was but IMHO calling the libm library function is the better solution, as discussed in the previous emails below. Less maintenance, less testing, and possibly better performance, e.g. as explained here : the C compiler used to produce libm may generate appropriate 128 and 256-bit Intel AVX VEX-encoded instructions, generating multiple, processor-specific, auto-dispatched code paths when there is a performance benefit. The most appropriate code would be executed at run time. Cheers George On Thu, Aug 2, 2018 at 5:19 PM David Feuer wrote: > Wasn't there a very recent commit to improve these functions, by > leftaroundabout? > > On Thursday, August 2, 2018 8:16:10 AM EDT Artem Pelenitsyn wrote: > > Here is the patch: https://phabricator.haskell.org/D5034 > > > > -- > > Best, Artem > > > > On Thu, 2 Aug 2018 at 06:26 Artem Pelenitsyn > wrote: > > > > > I'd be willing to do this. > > > > > > -- > > > Best wishes, > > > Artem > > > > > > > > > On Thu, 2 Aug 2018, 04:38 Matt Peddie, wrote: > > > > > >> Thanks, Ben, for chiming in. I think calling out to C for these > > >> functions is the way to go if it's now feasible. (Calling out to libm > > >> is the workaround I'm using in the application that led me to discover > > >> the inaccuracy.) > > >> > > >> Matt > > >> > > >> On Thu, Aug 2, 2018 at 11:33 AM, Ben Gamari > wrote: > > >> > Matt Peddie writes: > > >> > > > >> >> Hi George, > > >> >> > > >> >> Not a stupid question. I don't have a single source at hand, but I > > >> >> think I read in a few places on the wiki that calling out to the > > >> >> system math library is not an option due to the variety of system > math > > >> >> libraries on the platforms GHC supports. It'd be great if I got > the > > >> >> wrong impression and this could just be a call to C. Can anyone > set > > >> >> me straight on this point? > > >> >> > > >> > Indeed it's not a stupid question at all. Indeed this is precisely > what > > >> > we do for the simpler transcendentals (e.g. sin, asin, log). We very > > >> > well could move in this direction in the case of asinh/atanh as > well. I > > >> > believe the reason we don't currently is that atanh was only > > >> > standardized in C99, which we only started requiring a few releases > ago. > > >> > Perhaps this is ultimately the right direction. > > >> > > > >> > Cheers, > > >> > > > >> > - Ben > > >> _______________________________________________ > > >> ghc-devs mailing list > > >> ghc-devs at haskell.org > > >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > >> > > > > > > > > -- > David Feuer > Well-Typed > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.pelenitsyn at gmail.com Sat Aug 4 17:51:24 2018 From: a.pelenitsyn at gmail.com (Artem Pelenitsyn) Date: Sat, 4 Aug 2018 20:51:24 +0300 Subject: How to test master after breaking changes Message-ID: Hello devs, Wiki page on testing says that in order to run all tests you have to install additional packages: https://ghc.haskell.org/trac/ghc/wiki/Building/RunningTests/Running#AdditionalPackages and kindly provides a command to do this: cabal install --with-compiler=`pwd`/inplace/bin/ghc-stage2 --package-db=`pwd`/inplace/lib/package.conf.d mtl parallel parsec primitive QuickCheck random regex-compat syb stm utf8-string vector After the af9b744bb one of the packages, primitive, does not build anymore. At least, its last released version on Hackage. I see that the problem has been fixed on primitive's master (a2af610 ). But what should I do to actually test master branch of GHC now? -- Best wishes, Artem -------------- next part -------------- An HTML attachment was scrubbed... URL: From klebinger.andreas at gmx.at Sat Aug 4 22:57:36 2018 From: klebinger.andreas at gmx.at (Andreas Klebinger) Date: Sun, 05 Aug 2018 00:57:36 +0200 Subject: PSA: You likely want to use -O2 for the stage1 compiler. Message-ID: <5B662F60.5030306@gmx.at> I've wondered for a good while if using O2 on stage1 might be worth it. So I did some measurements and it should be worth it for most cases. For a single "quick" flavour build they are more or less on equal footing. If you rebuild stage2 multiple times reusing stage1 it will be faster. If you build stage2 with optimizations/profiling it will be faster. Below are the timings using "time make -j9" for a quick build. I forgot to write down the seconds as I didn't expect them to be so close. But it is what it is. Timings stage1 options O1 vs O2, quick build after make clean: stage1 opt | time (wall) | time (user) -O1 | 13m | 53m -O2 | 13m | 51m I've also run the numbers for a optimized stage2 compiler a while ago, where stage1 with O2 was faster. But I no longer have these numbers around. So it seems safe to say one should use O2 if either: * stage2 is built with optimizations * you freeze stage1 and reuse it while working on stage2 -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.pelenitsyn at gmail.com Sat Aug 4 23:27:19 2018 From: a.pelenitsyn at gmail.com (Artem Pelenitsyn) Date: Sun, 5 Aug 2018 02:27:19 +0300 Subject: PSA: You likely want to use -O2 for the stage1 compiler. In-Reply-To: <5B662F60.5030306@gmx.at> References: <5B662F60.5030306@gmx.at> Message-ID: Thanks, Andreas! I believe, the wiki modestly tries to suggest this here: https://ghc.haskell.org/trac/ghc/wiki/Building/Using#Workingonanoptimizedstage2compiler. Maybe, it should be more insistent, e.g., by adding these numbers or moving the advice to a more visible place. Not sure. -- Best, Artem On Sun, 5 Aug 2018 at 01:58 Andreas Klebinger wrote: > I've wondered for a good while if using O2 on stage1 might be worth it. > > So I did some measurements and it should be worth it for most cases. > > For a single "quick" flavour build they are more or less on equal footing. > If you rebuild stage2 multiple times reusing stage1 it will be faster. > If you build stage2 with optimizations/profiling it will be faster. > > Below are the timings using "time make -j9" for a quick build. > I forgot to write down the seconds as I didn't expect them to be so close. > But it is what it is. > > Timings stage1 options O1 vs O2, quick build after make clean: > > stage1 opt | time (wall) | time (user) > -O1 | 13m | 53m > -O2 | 13m | 51m > > I've also run the numbers for a optimized stage2 compiler a while ago, > where stage1 with O2 was faster. > But I no longer have these numbers around. > > So it seems safe to say one should use O2 if either: > * stage2 is built with optimizations > * you freeze stage1 and reuse it while working on stage2 > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From omeragacan at gmail.com Mon Aug 6 07:40:41 2018 From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=) Date: Mon, 6 Aug 2018 10:40:41 +0300 Subject: Any ways to test a GHC build against large set of packages (including test suites)? Message-ID: Hi, I'd like to test some GHC builds + some compile and runtime flag combinations against a large set of packages by building them and running test suites. For this I need - A set of packages that are known to work with latest GHC - A way to build them and run their test suites (if I could specify compile and runtime flags that'd be even better) I think stackage can serve as (1) but I don't know how to do (2). Can anyone point me to the right direction? I vaguely remember some nix-based solution for this that was being discussed on the IRC channel, but can't recall any details. Thanks, Ömer From a.pelenitsyn at gmail.com Mon Aug 6 07:55:43 2018 From: a.pelenitsyn at gmail.com (Artem Pelenitsyn) Date: Mon, 6 Aug 2018 10:55:43 +0300 Subject: Scripts merged for easily loading and running GHC in GHCi, for fast development iterations In-Reply-To: References: Message-ID: I added this to the wiki. It would be nice to have a wrapper similar to `run.sh` but for starting ghcid, I believe. -- Best wishes, Artem On Sun, 29 Jul 2018 at 10:41 Matthew Pickering wrote: > Can you please update the wiki page? > > https://ghc.haskell.org/trac/ghc/wiki/Building/InGhci > > Matt > > On Sun, Jul 29, 2018 at 6:18 AM, Michael Sloan wrote: > > Hello GHC developers! > > > > Now that D4904 and D4986 are merged, if you have recently built GHC > > then you can use "./utils/ghc-in-ghci/run.sh -fobject-code" to load > > GHC into GHCi (throwing on a "-j8" helps too). For me, this makes > > development far more pleasant, as often I can make a change and try it > > out in only 15 or 20 seconds. Thanks to Csongor Kiss for writing the > > initial ghci script, and Matthew Pickering for posting about it to the > > mailinglist, as otherwise I probably wouldn't have known about it. > > > > I wrote a blog post about the topic on my new blog: > > http://www.mgsloan.com/posts/ghcinception/ > > > > Once D5015 is merged, the explicit "-fobject-code" won't be needed. > > Once https://ghc.haskell.org/trac/ghc/ticket/15454 is addressed, it > > might not be needed at all, leading to even faster loading ;) > > > > Happy ghci-ing! > > -Michael > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mgsloan at gmail.com Mon Aug 6 08:03:38 2018 From: mgsloan at gmail.com (Michael Sloan) Date: Mon, 6 Aug 2018 01:03:38 -0700 Subject: Scripts merged for easily loading and running GHC in GHCi, for fast development iterations In-Reply-To: References: Message-ID: Thanks a bunch for updating the wiki page, Artem! It was on my todo list, but it's been a rather busy week. I've added an update mentioning that it will be possible to run "ghcid" directly without args once D5105 is merged. On Mon, Aug 6, 2018 at 12:55 AM, Artem Pelenitsyn wrote: > I added this to the wiki. It would be nice to have a wrapper similar to > `run.sh` but for starting ghcid, I believe. > > -- > Best wishes, > Artem > > On Sun, 29 Jul 2018 at 10:41 Matthew Pickering > wrote: >> >> Can you please update the wiki page? >> >> https://ghc.haskell.org/trac/ghc/wiki/Building/InGhci >> >> Matt >> >> On Sun, Jul 29, 2018 at 6:18 AM, Michael Sloan wrote: >> > Hello GHC developers! >> > >> > Now that D4904 and D4986 are merged, if you have recently built GHC >> > then you can use "./utils/ghc-in-ghci/run.sh -fobject-code" to load >> > GHC into GHCi (throwing on a "-j8" helps too). For me, this makes >> > development far more pleasant, as often I can make a change and try it >> > out in only 15 or 20 seconds. Thanks to Csongor Kiss for writing the >> > initial ghci script, and Matthew Pickering for posting about it to the >> > mailinglist, as otherwise I probably wouldn't have known about it. >> > >> > I wrote a blog post about the topic on my new blog: >> > http://www.mgsloan.com/posts/ghcinception/ >> > >> > Once D5015 is merged, the explicit "-fobject-code" won't be needed. >> > Once https://ghc.haskell.org/trac/ghc/ticket/15454 is addressed, it >> > might not be needed at all, leading to even faster loading ;) >> > >> > Happy ghci-ing! >> > -Michael >> > _______________________________________________ >> > ghc-devs mailing list >> > ghc-devs at haskell.org >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From ben at smart-cactus.org Mon Aug 6 13:44:30 2018 From: ben at smart-cactus.org (Ben Gamari) Date: Mon, 06 Aug 2018 09:44:30 -0400 Subject: Any ways to test a GHC build against large set of packages (including test suites)? In-Reply-To: References: Message-ID: <87effby5hi.fsf@smart-cactus.org> Ömer Sinan Ağacan writes: > Hi, > > I'd like to test some GHC builds + some compile and runtime flag combinations > against a large set of packages by building them and running test suites. For > this I need > > - A set of packages that are known to work with latest GHC > - A way to build them and run their test suites (if I could specify compile and > runtime flags that'd be even better) > > I think stackage can serve as (1) but I don't know how to do (2). Can anyone > point me to the right direction? I vaguely remember some nix-based solution for > this that was being discussed on the IRC channel, but can't recall any details. > You are probably remembering head.hackage [1], which has both nix and non-nix tooling. A few of us have been trying to bring it up-to-date with GHC 8.6. I suspect the changes in HEAD are small enough that it will work with newer compilers as well. Cheers, - Ben [1] https://github.com/hvr/head.hackage -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From moritz.angermann at gmail.com Tue Aug 7 02:39:48 2018 From: moritz.angermann at gmail.com (Moritz Angermann) Date: Tue, 7 Aug 2018 10:39:48 +0800 Subject: Non-Reinstallable packages In-Reply-To: References: Message-ID: <3DAA7CA7-D6C2-4280-AC66-DF7352357C4A@gmail.com> Dear friends, we have a set of non-reinstallable packages with GHC, these include iirc template-haskell, and some other. I've got a few questions concerning those: - do we have a complete up-to-date list of those? - why can't we reinstall them (let's assume we use the identical version for now; and don't upgrade) - does this also hold if we essentially build a stage3 compiler with packages? Our usual build process is: 1. take a boost-strap compiler, which doesn't need to have the same version as the final compiler. 2. build the libraries necessary to build the stage1 compiler while ensuring we build some extra libraries as well, so we don't have to rely on those shipped with the boot-strap compiler. 3. use the stage1 compiler to build all libraries we want to ship with the stage2 compiler; and build the stage2 compiler. Now I do understand that the stage1 compiler could potentially be tainted by the boot-strap compiler and as such yield different libraries compared to what the stage2 compiler would yield. Shouldn't rebuilding any library with the stage1 compiler yield the same libraries these days? If the boot-strap compiler is the same version as the one we build, shouldn't the stage2 compiler be capable of building good enough libraries as well so that we can reinstall them? What I ideally would like to have is a minimal compiler: ghc + rts; than keep building all the lirbaries from ground up. A potential problem I see is that if we use dynamic libraries and get into TH, we could run into issues where we want to link libraries that are different to the ones that the ghc binary links against. Would this also hold if we used `-fexternal-interpreter` only? Cheers, Moritz -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: Message signed with OpenPGP URL: From cheng.shao at tweag.io Tue Aug 7 02:51:15 2018 From: cheng.shao at tweag.io (Shao, Cheng) Date: Tue, 7 Aug 2018 10:51:15 +0800 Subject: Non-Reinstallable packages In-Reply-To: <3DAA7CA7-D6C2-4280-AC66-DF7352357C4A@gmail.com> References: <3DAA7CA7-D6C2-4280-AC66-DF7352357C4A@gmail.com> Message-ID: Hi, IIRC those packages can be "reinstalled", just build & register into a fresh package database and add it to the pkgdb stack, ghc can shadow the ones in the global pkgdb. Regards, Shao Cheng On Tue, Aug 7, 2018 at 10:39 AM, Moritz Angermann wrote: > Dear friends, > > we have a set of non-reinstallable packages with GHC, these > include iirc template-haskell, and some other. I've got > a few questions concerning those: > > - do we have a complete up-to-date list of those? > - why can't we reinstall them (let's assume we use the > identical version for now; and don't upgrade) > - does this also hold if we essentially build a stage3 > compiler with packages? > > Our usual build process is: > 1. take a boost-strap compiler, which doesn't need to have > the same version as the final compiler. > 2. build the libraries necessary to build the stage1 compiler > while ensuring we build some extra libraries as well, > so we don't have to rely on those shipped with the boot-strap > compiler. > 3. use the stage1 compiler to build all libraries we want to ship > with the stage2 compiler; and build the stage2 compiler. > > Now I do understand that the stage1 compiler could potentially be > tainted by the boot-strap compiler and as such yield different > libraries compared to what the stage2 compiler would yield. > > Shouldn't rebuilding any library with the stage1 compiler yield the > same libraries these days? > > If the boot-strap compiler is the same version as the one we build, > shouldn't the stage2 compiler be capable of building good enough > libraries as well so that we can reinstall them? > > What I ideally would like to have is a minimal compiler: > ghc + rts; than keep building all the lirbaries from ground up. > > A potential problem I see is that if we use dynamic libraries and > get into TH, we could run into issues where we want to link libraries > that are different to the ones that the ghc binary links against. > Would this also hold if we used `-fexternal-interpreter` only? > > Cheers, > Moritz > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > From chak at justtesting.org Tue Aug 7 15:15:04 2018 From: chak at justtesting.org (Manuel M T Chakravarty) Date: Tue, 7 Aug 2018 17:15:04 +0200 Subject: Any ways to test a GHC build against large set of packages (including test suites)? In-Reply-To: References: Message-ID: Hi Ömer, This is exactly the motivation for the Stackage HEAD works that we have pushed at Tweag I/O in the context of the GHC DevOps group. Have a look at https://github.com/tweag/stackage-head and also the blog post from when the first version went live: https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html Cheers, Manuel > Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan : > > Hi, > > I'd like to test some GHC builds + some compile and runtime flag combinations > against a large set of packages by building them and running test suites. For > this I need > > - A set of packages that are known to work with latest GHC > - A way to build them and run their test suites (if I could specify compile and > runtime flags that'd be even better) > > I think stackage can serve as (1) but I don't know how to do (2). Can anyone > point me to the right direction? I vaguely remember some nix-based solution for > this that was being discussed on the IRC channel, but can't recall any details. > > Thanks, > > Ömer > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 874 bytes Desc: Message signed with OpenPGP URL: From omeragacan at gmail.com Tue Aug 7 20:28:00 2018 From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=) Date: Tue, 7 Aug 2018 23:28:00 +0300 Subject: Any ways to test a GHC build against large set of packages (including test suites)? In-Reply-To: References: Message-ID: Thanks for both suggestions. I'll try both and see which one works better. Ömer Manuel M T Chakravarty , 7 Ağu 2018 Sal, 18:15 tarihinde şunu yazdı: > > Hi Ömer, > > This is exactly the motivation for the Stackage HEAD works that we have pushed at Tweag I/O in the context of the GHC DevOps group. Have a look at > > https://github.com/tweag/stackage-head > > and also the blog post from when the first version went live: > > https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html > > Cheers, > Manuel > > > Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan : > > > > Hi, > > > > I'd like to test some GHC builds + some compile and runtime flag combinations > > against a large set of packages by building them and running test suites. For > > this I need > > > > - A set of packages that are known to work with latest GHC > > - A way to build them and run their test suites (if I could specify compile and > > runtime flags that'd be even better) > > > > I think stackage can serve as (1) but I don't know how to do (2). Can anyone > > point me to the right direction? I vaguely remember some nix-based solution for > > this that was being discussed on the IRC channel, but can't recall any details. > > > > Thanks, > > > > Ömer > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > From mgsloan at gmail.com Wed Aug 8 22:12:01 2018 From: mgsloan at gmail.com (Michael Sloan) Date: Wed, 8 Aug 2018 15:12:01 -0700 Subject: Scripts merged for easily loading and running GHC in GHCi, for fast development iterations In-Reply-To: References: Message-ID: Now that [D5015] is merged (thanks, monoidal!), -fobject-code no longer needs to be provided manually. This also means that you can run [ghcid] directly in the root of the repo without any arguments. I have updated https://ghc.haskell.org/trac/ghc/wiki/Building/InGhci to use the simpler invocations. [D5015]: https://phabricator.haskell.org/D5015 [ghcid]: https://github.com/ndmitchell/ghcid On Mon, Aug 6, 2018 at 1:03 AM, Michael Sloan wrote: > Thanks a bunch for updating the wiki page, Artem! It was on my todo > list, but it's been a rather busy week. I've added an update > mentioning that it will be possible to run "ghcid" directly without > args once D5105 is merged. > > On Mon, Aug 6, 2018 at 12:55 AM, Artem Pelenitsyn > wrote: >> I added this to the wiki. It would be nice to have a wrapper similar to >> `run.sh` but for starting ghcid, I believe. >> >> -- >> Best wishes, >> Artem >> >> On Sun, 29 Jul 2018 at 10:41 Matthew Pickering >> wrote: >>> >>> Can you please update the wiki page? >>> >>> https://ghc.haskell.org/trac/ghc/wiki/Building/InGhci >>> >>> Matt >>> >>> On Sun, Jul 29, 2018 at 6:18 AM, Michael Sloan wrote: >>> > Hello GHC developers! >>> > >>> > Now that D4904 and D4986 are merged, if you have recently built GHC >>> > then you can use "./utils/ghc-in-ghci/run.sh -fobject-code" to load >>> > GHC into GHCi (throwing on a "-j8" helps too). For me, this makes >>> > development far more pleasant, as often I can make a change and try it >>> > out in only 15 or 20 seconds. Thanks to Csongor Kiss for writing the >>> > initial ghci script, and Matthew Pickering for posting about it to the >>> > mailinglist, as otherwise I probably wouldn't have known about it. >>> > >>> > I wrote a blog post about the topic on my new blog: >>> > http://www.mgsloan.com/posts/ghcinception/ >>> > >>> > Once D5015 is merged, the explicit "-fobject-code" won't be needed. >>> > Once https://ghc.haskell.org/trac/ghc/ticket/15454 is addressed, it >>> > might not be needed at all, leading to even faster loading ;) >>> > >>> > Happy ghci-ing! >>> > -Michael >>> > _______________________________________________ >>> > ghc-devs mailing list >>> > ghc-devs at haskell.org >>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >>> _______________________________________________ >>> ghc-devs mailing list >>> ghc-devs at haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From matthewtpickering at gmail.com Wed Aug 8 22:31:31 2018 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Wed, 8 Aug 2018 23:31:31 +0100 Subject: Scripts merged for easily loading and running GHC in GHCi, for fast development iterations In-Reply-To: References: Message-ID: In practice, how much memory does this use? I find that with 4gb of memory that it is unusable and hard to kill. Perhaps we should add some sort of warning to the page about this. Cheers Matt On Wed, Aug 8, 2018 at 11:12 PM, Michael Sloan wrote: > Now that [D5015] is merged (thanks, monoidal!), -fobject-code no > longer needs to be provided manually. This also means that you can > run [ghcid] directly in the root of the repo without any arguments. I > have updated https://ghc.haskell.org/trac/ghc/wiki/Building/InGhci to > use the simpler invocations. > > [D5015]: https://phabricator.haskell.org/D5015 > [ghcid]: https://github.com/ndmitchell/ghcid > > On Mon, Aug 6, 2018 at 1:03 AM, Michael Sloan wrote: >> Thanks a bunch for updating the wiki page, Artem! It was on my todo >> list, but it's been a rather busy week. I've added an update >> mentioning that it will be possible to run "ghcid" directly without >> args once D5105 is merged. >> >> On Mon, Aug 6, 2018 at 12:55 AM, Artem Pelenitsyn >> wrote: >>> I added this to the wiki. It would be nice to have a wrapper similar to >>> `run.sh` but for starting ghcid, I believe. >>> >>> -- >>> Best wishes, >>> Artem >>> >>> On Sun, 29 Jul 2018 at 10:41 Matthew Pickering >>> wrote: >>>> >>>> Can you please update the wiki page? >>>> >>>> https://ghc.haskell.org/trac/ghc/wiki/Building/InGhci >>>> >>>> Matt >>>> >>>> On Sun, Jul 29, 2018 at 6:18 AM, Michael Sloan wrote: >>>> > Hello GHC developers! >>>> > >>>> > Now that D4904 and D4986 are merged, if you have recently built GHC >>>> > then you can use "./utils/ghc-in-ghci/run.sh -fobject-code" to load >>>> > GHC into GHCi (throwing on a "-j8" helps too). For me, this makes >>>> > development far more pleasant, as often I can make a change and try it >>>> > out in only 15 or 20 seconds. Thanks to Csongor Kiss for writing the >>>> > initial ghci script, and Matthew Pickering for posting about it to the >>>> > mailinglist, as otherwise I probably wouldn't have known about it. >>>> > >>>> > I wrote a blog post about the topic on my new blog: >>>> > http://www.mgsloan.com/posts/ghcinception/ >>>> > >>>> > Once D5015 is merged, the explicit "-fobject-code" won't be needed. >>>> > Once https://ghc.haskell.org/trac/ghc/ticket/15454 is addressed, it >>>> > might not be needed at all, leading to even faster loading ;) >>>> > >>>> > Happy ghci-ing! >>>> > -Michael >>>> > _______________________________________________ >>>> > ghc-devs mailing list >>>> > ghc-devs at haskell.org >>>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >>>> _______________________________________________ >>>> ghc-devs mailing list >>>> ghc-devs at haskell.org >>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From mgsloan at gmail.com Wed Aug 8 22:53:31 2018 From: mgsloan at gmail.com (Michael Sloan) Date: Wed, 8 Aug 2018 15:53:31 -0700 Subject: Scripts merged for easily loading and running GHC in GHCi, for fast development iterations In-Reply-To: References: Message-ID: Good point, I have added a note about it to the wiki page. Surprising that it would be hard to kill. Do you have swap setup? For me it takes ~2.7GB initially and 3.5GB after the first reload. In the past I've seen it get all the way up to 9GB so there may be some memory leakage. I have 40GB of ram in my laptop, so I don't notice this much. I highly recommend investing in more memory if it's within your financial means. The productivity boon of being able to use GHCi on large projects, or run multiple concurrent docker containers or VMs, is huge. A bit off topic, but sometimes folks are surprised that I spend hundreds of dollars on keyboards like the kinesis advantage 2 (which is fantastic, btw) or the keyboard.io (really nice too!). My answer is that if I'm using some equipment many hours per day, the cost amortizes out. I ask myself "Would I pay 3 cents to use this keyboard for an hour?", in the case of those keyboards, my answer is an emphatic YES! I apply similar reasoning to my main computer hardware, though people are used to that being $$$ vs the surprise of $350 keyboards. -Michael On Wed, Aug 8, 2018 at 3:31 PM, Matthew Pickering wrote: > In practice, how much memory does this use? > > I find that with 4gb of memory that it is unusable and hard to kill. > Perhaps we should add some sort of warning to the page about this. > > Cheers > > Matt > > On Wed, Aug 8, 2018 at 11:12 PM, Michael Sloan wrote: >> Now that [D5015] is merged (thanks, monoidal!), -fobject-code no >> longer needs to be provided manually. This also means that you can >> run [ghcid] directly in the root of the repo without any arguments. I >> have updated https://ghc.haskell.org/trac/ghc/wiki/Building/InGhci to >> use the simpler invocations. >> >> [D5015]: https://phabricator.haskell.org/D5015 >> [ghcid]: https://github.com/ndmitchell/ghcid >> >> On Mon, Aug 6, 2018 at 1:03 AM, Michael Sloan wrote: >>> Thanks a bunch for updating the wiki page, Artem! It was on my todo >>> list, but it's been a rather busy week. I've added an update >>> mentioning that it will be possible to run "ghcid" directly without >>> args once D5105 is merged. >>> >>> On Mon, Aug 6, 2018 at 12:55 AM, Artem Pelenitsyn >>> wrote: >>>> I added this to the wiki. It would be nice to have a wrapper similar to >>>> `run.sh` but for starting ghcid, I believe. >>>> >>>> -- >>>> Best wishes, >>>> Artem >>>> >>>> On Sun, 29 Jul 2018 at 10:41 Matthew Pickering >>>> wrote: >>>>> >>>>> Can you please update the wiki page? >>>>> >>>>> https://ghc.haskell.org/trac/ghc/wiki/Building/InGhci >>>>> >>>>> Matt >>>>> >>>>> On Sun, Jul 29, 2018 at 6:18 AM, Michael Sloan wrote: >>>>> > Hello GHC developers! >>>>> > >>>>> > Now that D4904 and D4986 are merged, if you have recently built GHC >>>>> > then you can use "./utils/ghc-in-ghci/run.sh -fobject-code" to load >>>>> > GHC into GHCi (throwing on a "-j8" helps too). For me, this makes >>>>> > development far more pleasant, as often I can make a change and try it >>>>> > out in only 15 or 20 seconds. Thanks to Csongor Kiss for writing the >>>>> > initial ghci script, and Matthew Pickering for posting about it to the >>>>> > mailinglist, as otherwise I probably wouldn't have known about it. >>>>> > >>>>> > I wrote a blog post about the topic on my new blog: >>>>> > http://www.mgsloan.com/posts/ghcinception/ >>>>> > >>>>> > Once D5015 is merged, the explicit "-fobject-code" won't be needed. >>>>> > Once https://ghc.haskell.org/trac/ghc/ticket/15454 is addressed, it >>>>> > might not be needed at all, leading to even faster loading ;) >>>>> > >>>>> > Happy ghci-ing! >>>>> > -Michael >>>>> > _______________________________________________ >>>>> > ghc-devs mailing list >>>>> > ghc-devs at haskell.org >>>>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >>>>> _______________________________________________ >>>>> ghc-devs mailing list >>>>> ghc-devs at haskell.org >>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From allbery.b at gmail.com Wed Aug 8 22:56:24 2018 From: allbery.b at gmail.com (Brandon Allbery) Date: Wed, 8 Aug 2018 18:56:24 -0400 Subject: Scripts merged for easily loading and running GHC in GHCi, for fast development iterations In-Reply-To: References: Message-ID: If you're working on e.g. ARM, you may not have a lot of say in how much RAM you can put into it. On Wed, Aug 8, 2018 at 6:54 PM Michael Sloan wrote: > Good point, I have added a note about it to the wiki page. Surprising > that it would be hard to kill. Do you have swap setup? > > For me it takes ~2.7GB initially and 3.5GB after the first reload. In > the past I've seen it get all the way up to 9GB so there may be some > memory leakage. > > I have 40GB of ram in my laptop, so I don't notice this much. I > highly recommend investing in more memory if it's within your > financial means. The productivity boon of being able to use GHCi on > large projects, or run multiple concurrent docker containers or VMs, > is huge. > > A bit off topic, but sometimes folks are surprised that I spend > hundreds of dollars on keyboards like the kinesis advantage 2 (which > is fantastic, btw) or the keyboard.io (really nice too!). My answer > is that if I'm using some equipment many hours per day, the cost > amortizes out. I ask myself "Would I pay 3 cents to use this keyboard > for an hour?", in the case of those keyboards, my answer is an > emphatic YES! I apply similar reasoning to my main computer hardware, > though people are used to that being $$$ vs the surprise of $350 > keyboards. > > -Michael > > On Wed, Aug 8, 2018 at 3:31 PM, Matthew Pickering > wrote: > > In practice, how much memory does this use? > > > > I find that with 4gb of memory that it is unusable and hard to kill. > > Perhaps we should add some sort of warning to the page about this. > > > > Cheers > > > > Matt > > > > On Wed, Aug 8, 2018 at 11:12 PM, Michael Sloan > wrote: > >> Now that [D5015] is merged (thanks, monoidal!), -fobject-code no > >> longer needs to be provided manually. This also means that you can > >> run [ghcid] directly in the root of the repo without any arguments. I > >> have updated https://ghc.haskell.org/trac/ghc/wiki/Building/InGhci to > >> use the simpler invocations. > >> > >> [D5015]: https://phabricator.haskell.org/D5015 > >> [ghcid]: https://github.com/ndmitchell/ghcid > >> > >> On Mon, Aug 6, 2018 at 1:03 AM, Michael Sloan > wrote: > >>> Thanks a bunch for updating the wiki page, Artem! It was on my todo > >>> list, but it's been a rather busy week. I've added an update > >>> mentioning that it will be possible to run "ghcid" directly without > >>> args once D5105 is merged. > >>> > >>> On Mon, Aug 6, 2018 at 12:55 AM, Artem Pelenitsyn > >>> wrote: > >>>> I added this to the wiki. It would be nice to have a wrapper similar > to > >>>> `run.sh` but for starting ghcid, I believe. > >>>> > >>>> -- > >>>> Best wishes, > >>>> Artem > >>>> > >>>> On Sun, 29 Jul 2018 at 10:41 Matthew Pickering < > matthewtpickering at gmail.com> > >>>> wrote: > >>>>> > >>>>> Can you please update the wiki page? > >>>>> > >>>>> https://ghc.haskell.org/trac/ghc/wiki/Building/InGhci > >>>>> > >>>>> Matt > >>>>> > >>>>> On Sun, Jul 29, 2018 at 6:18 AM, Michael Sloan > wrote: > >>>>> > Hello GHC developers! > >>>>> > > >>>>> > Now that D4904 and D4986 are merged, if you have recently built GHC > >>>>> > then you can use "./utils/ghc-in-ghci/run.sh -fobject-code" to load > >>>>> > GHC into GHCi (throwing on a "-j8" helps too). For me, this makes > >>>>> > development far more pleasant, as often I can make a change and > try it > >>>>> > out in only 15 or 20 seconds. Thanks to Csongor Kiss for writing > the > >>>>> > initial ghci script, and Matthew Pickering for posting about it to > the > >>>>> > mailinglist, as otherwise I probably wouldn't have known about it. > >>>>> > > >>>>> > I wrote a blog post about the topic on my new blog: > >>>>> > http://www.mgsloan.com/posts/ghcinception/ > >>>>> > > >>>>> > Once D5015 is merged, the explicit "-fobject-code" won't be needed. > >>>>> > Once https://ghc.haskell.org/trac/ghc/ticket/15454 is addressed, > it > >>>>> > might not be needed at all, leading to even faster loading ;) > >>>>> > > >>>>> > Happy ghci-ing! > >>>>> > -Michael > >>>>> > _______________________________________________ > >>>>> > ghc-devs mailing list > >>>>> > ghc-devs at haskell.org > >>>>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > >>>>> _______________________________________________ > >>>>> ghc-devs mailing list > >>>>> ghc-devs at haskell.org > >>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -- brandon s allbery kf8nh allbery.b at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From omeragacan at gmail.com Thu Aug 9 10:20:29 2018 From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=) Date: Thu, 9 Aug 2018 13:20:29 +0300 Subject: Any ways to test a GHC build against large set of packages (including test suites)? In-Reply-To: References: Message-ID: Hi Manuel, I'm trying stackage-head. I'm following the steps for the scheduled build in .circleci/config.yml. So far steps I took: - Installed ghc-head (from [1]) to ~/ghc-head - Installed stackage-build-plan, stackage-curator and stackage-head (with -fdev) from git repos, using stack. - export BUILD_PLAN=nightly-2018-07-30 (from config.yml) - curl https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/metadata.json --output metadata.json - curl https://raw.githubusercontent.com/fpco/stackage-nightly/master/$BUILD_PLAN.yaml --output $BUILD_PLAN.yaml Now I'm doing - ./.local/bin/stackage-head already-seen --target $BUILD_PLAN --ghc-metadata metadata.json --outdir build-reports but it's failing with The combination of target and commit is new to me Any ideas what I'm doing wrong? Thanks [1]: https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/bindist.tar.xz Ömer Ömer Sinan Ağacan , 7 Ağu 2018 Sal, 23:28 tarihinde şunu yazdı: > > Thanks for both suggestions. I'll try both and see which one works better. > > Ömer > > Manuel M T Chakravarty , 7 Ağu 2018 Sal, 18:15 > tarihinde şunu yazdı: > > > > Hi Ömer, > > > > This is exactly the motivation for the Stackage HEAD works that we have pushed at Tweag I/O in the context of the GHC DevOps group. Have a look at > > > > https://github.com/tweag/stackage-head > > > > and also the blog post from when the first version went live: > > > > https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html > > > > Cheers, > > Manuel > > > > > Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan : > > > > > > Hi, > > > > > > I'd like to test some GHC builds + some compile and runtime flag combinations > > > against a large set of packages by building them and running test suites. For > > > this I need > > > > > > - A set of packages that are known to work with latest GHC > > > - A way to build them and run their test suites (if I could specify compile and > > > runtime flags that'd be even better) > > > > > > I think stackage can serve as (1) but I don't know how to do (2). Can anyone > > > point me to the right direction? I vaguely remember some nix-based solution for > > > this that was being discussed on the IRC channel, but can't recall any details. > > > > > > Thanks, > > > > > > Ömer > > > _______________________________________________ > > > ghc-devs mailing list > > > ghc-devs at haskell.org > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > From omeragacan at gmail.com Thu Aug 9 11:45:00 2018 From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=) Date: Thu, 9 Aug 2018 14:45:00 +0300 Subject: Any ways to test a GHC build against large set of packages (including test suites)? In-Reply-To: References: Message-ID: Ah, I now realize that that command is supposed to print that output. I'll continue following the steps and keep you updated if I get stuck again. Ömer Ömer Sinan Ağacan , 9 Ağu 2018 Per, 13:20 tarihinde şunu yazdı: > > Hi Manuel, > > I'm trying stackage-head. I'm following the steps for the scheduled build in > .circleci/config.yml. So far steps I took: > > - Installed ghc-head (from [1]) to ~/ghc-head > - Installed stackage-build-plan, stackage-curator and stackage-head (with > -fdev) from git repos, using stack. > - export BUILD_PLAN=nightly-2018-07-30 (from config.yml) > - curl https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/metadata.json > --output metadata.json > - curl https://raw.githubusercontent.com/fpco/stackage-nightly/master/$BUILD_PLAN.yaml > --output $BUILD_PLAN.yaml > > Now I'm doing > > - ./.local/bin/stackage-head already-seen --target $BUILD_PLAN > --ghc-metadata metadata.json --outdir build-reports > > but it's failing with > > The combination of target and commit is new to me > > Any ideas what I'm doing wrong? > > Thanks > > [1]: https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/bindist.tar.xz > > Ömer > > Ömer Sinan Ağacan , 7 Ağu 2018 Sal, 23:28 > tarihinde şunu yazdı: > > > > Thanks for both suggestions. I'll try both and see which one works better. > > > > Ömer > > > > Manuel M T Chakravarty , 7 Ağu 2018 Sal, 18:15 > > tarihinde şunu yazdı: > > > > > > Hi Ömer, > > > > > > This is exactly the motivation for the Stackage HEAD works that we have pushed at Tweag I/O in the context of the GHC DevOps group. Have a look at > > > > > > https://github.com/tweag/stackage-head > > > > > > and also the blog post from when the first version went live: > > > > > > https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html > > > > > > Cheers, > > > Manuel > > > > > > > Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan : > > > > > > > > Hi, > > > > > > > > I'd like to test some GHC builds + some compile and runtime flag combinations > > > > against a large set of packages by building them and running test suites. For > > > > this I need > > > > > > > > - A set of packages that are known to work with latest GHC > > > > - A way to build them and run their test suites (if I could specify compile and > > > > runtime flags that'd be even better) > > > > > > > > I think stackage can serve as (1) but I don't know how to do (2). Can anyone > > > > point me to the right direction? I vaguely remember some nix-based solution for > > > > this that was being discussed on the IRC channel, but can't recall any details. > > > > > > > > Thanks, > > > > > > > > Ömer > > > > _______________________________________________ > > > > ghc-devs mailing list > > > > ghc-devs at haskell.org > > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > From omeragacan at gmail.com Fri Aug 10 08:39:46 2018 From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=) Date: Fri, 10 Aug 2018 11:39:46 +0300 Subject: Any ways to test a GHC build against large set of packages (including test suites)? In-Reply-To: References: Message-ID: Hi, This is working great, I just generated my first report. One problem is stm-2.4 doesn't compile with GHC HEAD, we need stm-2.5.0.0. But that's not published on Hackage yet, and latest nightly still uses stm-2.4.5.0. I wonder if there's anything that can be done about this. Apparently stm blocks 82 packages (I don't know if that's counting transitively or just packages that are directly blocked by stm). Any ideas about this? Ömer Ömer Sinan Ağacan , 9 Ağu 2018 Per, 14:45 tarihinde şunu yazdı: > > Ah, I now realize that that command is supposed to print that output. I'll > continue following the steps and keep you updated if I get stuck again. > > Ömer > > Ömer Sinan Ağacan , 9 Ağu 2018 Per, 13:20 > tarihinde şunu yazdı: > > > > Hi Manuel, > > > > I'm trying stackage-head. I'm following the steps for the scheduled build in > > .circleci/config.yml. So far steps I took: > > > > - Installed ghc-head (from [1]) to ~/ghc-head > > - Installed stackage-build-plan, stackage-curator and stackage-head (with > > -fdev) from git repos, using stack. > > - export BUILD_PLAN=nightly-2018-07-30 (from config.yml) > > - curl https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/metadata.json > > --output metadata.json > > - curl https://raw.githubusercontent.com/fpco/stackage-nightly/master/$BUILD_PLAN.yaml > > --output $BUILD_PLAN.yaml > > > > Now I'm doing > > > > - ./.local/bin/stackage-head already-seen --target $BUILD_PLAN > > --ghc-metadata metadata.json --outdir build-reports > > > > but it's failing with > > > > The combination of target and commit is new to me > > > > Any ideas what I'm doing wrong? > > > > Thanks > > > > [1]: https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/bindist.tar.xz > > > > Ömer > > > > Ömer Sinan Ağacan , 7 Ağu 2018 Sal, 23:28 > > tarihinde şunu yazdı: > > > > > > Thanks for both suggestions. I'll try both and see which one works better. > > > > > > Ömer > > > > > > Manuel M T Chakravarty , 7 Ağu 2018 Sal, 18:15 > > > tarihinde şunu yazdı: > > > > > > > > Hi Ömer, > > > > > > > > This is exactly the motivation for the Stackage HEAD works that we have pushed at Tweag I/O in the context of the GHC DevOps group. Have a look at > > > > > > > > https://github.com/tweag/stackage-head > > > > > > > > and also the blog post from when the first version went live: > > > > > > > > https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html > > > > > > > > Cheers, > > > > Manuel > > > > > > > > > Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan : > > > > > > > > > > Hi, > > > > > > > > > > I'd like to test some GHC builds + some compile and runtime flag combinations > > > > > against a large set of packages by building them and running test suites. For > > > > > this I need > > > > > > > > > > - A set of packages that are known to work with latest GHC > > > > > - A way to build them and run their test suites (if I could specify compile and > > > > > runtime flags that'd be even better) > > > > > > > > > > I think stackage can serve as (1) but I don't know how to do (2). Can anyone > > > > > point me to the right direction? I vaguely remember some nix-based solution for > > > > > this that was being discussed on the IRC channel, but can't recall any details. > > > > > > > > > > Thanks, > > > > > > > > > > Ömer > > > > > _______________________________________________ > > > > > ghc-devs mailing list > > > > > ghc-devs at haskell.org > > > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > > From a.pelenitsyn at gmail.com Fri Aug 10 09:05:04 2018 From: a.pelenitsyn at gmail.com (Artem Pelenitsyn) Date: Fri, 10 Aug 2018 12:05:04 +0300 Subject: Any ways to test a GHC build against large set of packages (including test suites)? In-Reply-To: References: Message-ID: Hello Ömer, Just a week ago I asked very similar question: how to install the test suite dependencies after breaking changes in GHC: https://mail.haskell.org/pipermail/ghc-devs/2018-August/016075.html But no one replied :( The task seems to be not solvable even if an affected package (stm in your case and primitive in mine) has already adopted in its master the breaking change but has no corresponding release on Hackage (which will always be the case, because how would you release depending on base which hasn't been released yet). Some package managers in other languages (Julia's Pkg, go install) allow you to depend directly on master branch of a package if you wish to. I wonder if there is a road here for cabal to take in order to support this kind of unsafe installs. -- Best wishes, Artem On Fri, 10 Aug 2018, 11:40 Ömer Sinan Ağacan, wrote: > Hi, > > This is working great, I just generated my first report. One problem is > stm-2.4 > doesn't compile with GHC HEAD, we need stm-2.5.0.0. But that's not > published on > Hackage yet, and latest nightly still uses stm-2.4.5.0. I wonder if there's > anything that can be done about this. Apparently stm blocks 82 packages (I > don't know if that's counting transitively or just packages that are > directly > blocked by stm). Any ideas about this? > > Ömer > > Ömer Sinan Ağacan , 9 Ağu 2018 Per, 14:45 > tarihinde şunu yazdı: > > > > Ah, I now realize that that command is supposed to print that output. > I'll > > continue following the steps and keep you updated if I get stuck again. > > > > Ömer > > > > Ömer Sinan Ağacan , 9 Ağu 2018 Per, 13:20 > > tarihinde şunu yazdı: > > > > > > Hi Manuel, > > > > > > I'm trying stackage-head. I'm following the steps for the scheduled > build in > > > .circleci/config.yml. So far steps I took: > > > > > > - Installed ghc-head (from [1]) to ~/ghc-head > > > - Installed stackage-build-plan, stackage-curator and stackage-head > (with > > > -fdev) from git repos, using stack. > > > - export BUILD_PLAN=nightly-2018-07-30 (from config.yml) > > > - curl > https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/metadata.json > > > --output metadata.json > > > - curl > https://raw.githubusercontent.com/fpco/stackage-nightly/master/$BUILD_PLAN.yaml > > > --output $BUILD_PLAN.yaml > > > > > > Now I'm doing > > > > > > - ./.local/bin/stackage-head already-seen --target $BUILD_PLAN > > > --ghc-metadata metadata.json --outdir build-reports > > > > > > but it's failing with > > > > > > The combination of target and commit is new to me > > > > > > Any ideas what I'm doing wrong? > > > > > > Thanks > > > > > > [1]: > https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/bindist.tar.xz > > > > > > Ömer > > > > > > Ömer Sinan Ağacan , 7 Ağu 2018 Sal, 23:28 > > > tarihinde şunu yazdı: > > > > > > > > Thanks for both suggestions. I'll try both and see which one works > better. > > > > > > > > Ömer > > > > > > > > Manuel M T Chakravarty , 7 Ağu 2018 Sal, 18:15 > > > > tarihinde şunu yazdı: > > > > > > > > > > Hi Ömer, > > > > > > > > > > This is exactly the motivation for the Stackage HEAD works that we > have pushed at Tweag I/O in the context of the GHC DevOps group. Have a > look at > > > > > > > > > > https://github.com/tweag/stackage-head > > > > > > > > > > and also the blog post from when the first version went live: > > > > > > > > > > https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html > > > > > > > > > > Cheers, > > > > > Manuel > > > > > > > > > > > Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan < > omeragacan at gmail.com>: > > > > > > > > > > > > Hi, > > > > > > > > > > > > I'd like to test some GHC builds + some compile and runtime flag > combinations > > > > > > against a large set of packages by building them and running > test suites. For > > > > > > this I need > > > > > > > > > > > > - A set of packages that are known to work with latest GHC > > > > > > - A way to build them and run their test suites (if I could > specify compile and > > > > > > runtime flags that'd be even better) > > > > > > > > > > > > I think stackage can serve as (1) but I don't know how to do > (2). Can anyone > > > > > > point me to the right direction? I vaguely remember some > nix-based solution for > > > > > > this that was being discussed on the IRC channel, but can't > recall any details. > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Ömer > > > > > > _______________________________________________ > > > > > > ghc-devs mailing list > > > > > > ghc-devs at haskell.org > > > > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hvriedel at gmail.com Fri Aug 10 09:29:24 2018 From: hvriedel at gmail.com (Herbert Valerio Riedel) Date: Fri, 10 Aug 2018 11:29:24 +0200 Subject: Any ways to test a GHC build against large set of packages (including test suites)? In-Reply-To: References: Message-ID: Hi Artem, On Fri, Aug 10, 2018 at 11:05 AM Artem Pelenitsyn wrote: > The task seems to be not solvable even if an affected package (stm in your case and primitive in mine) has already adopted in its master the breaking change but has no corresponding release on Hackage (which will always be the case, because how would you release depending on base which hasn't been released yet). > > Some package managers in other languages (Julia's Pkg, go install) allow you to depend directly on master branch of a package if you wish to. I wonder if there is a road here for cabal to take in order to support this kind of unsafe installs. As a matter of fact we have a couple of options here: - Add the http://cand.hackage.haskell.org/ package index to your config and get access to the stm-2.5.0.0 package release candidate (i.e. http://hackage.haskell.org/package/stm-2.5.0.0/candidate/stm-2.5.0.0.tar.gz) - We could have added `stm-2.5.0.0` to the head.hackage overlay (so far the "overlay" has only modified existing packages from the primary hackage index) - Starting with cabal-2.4, we support adding package tarballs to `cabal.project`; and this should work also for specifiying remote tarball locations such as http://hackage.haskell.org/package/stm-2.5.0.0/candidate/stm-2.5.0.0.tar.gz - Starting with cabal 2.4, we do support remote git repo deps in `cabal.project`, see e.g. https://github.com/hvr/cardano-sl/blob/wip/cabal-project/cabal.project#L41-L156 for an extensive real-world example. There's work planned to refine/improve these options and make them more convenient/expressive. Help is appreciated if somebody wants to help us get there sooner. -- hvr From omeragacan at gmail.com Fri Aug 10 09:50:58 2018 From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=) Date: Fri, 10 Aug 2018 12:50:58 +0300 Subject: How to test master after breaking changes In-Reply-To: References: Message-ID: Hi Artem, I think currently the best you could do is to clone primitive's git repo locally and install it from there, using `cd primitive; cabal install --with-ghc=...`. Note that you can run the test suite without these dependencies. The driver skips the test if a dependency is not found. See also #15137. (I wonder what CI is doing about this, I'm guessing it doesn't install dependencies so some of the tests are not run on CI) Ömer Artem Pelenitsyn , 4 Ağu 2018 Cmt, 20:51 tarihinde şunu yazdı: > > Hello devs, > > Wiki page on testing says that in order to run all tests you have to install additional packages: > > https://ghc.haskell.org/trac/ghc/wiki/Building/RunningTests/Running#AdditionalPackages > > and kindly provides a command to do this: > > cabal install --with-compiler=`pwd`/inplace/bin/ghc-stage2 --package-db=`pwd`/inplace/lib/package.conf.d mtl parallel parsec primitive QuickCheck random regex-compat syb stm utf8-string vector > > After the af9b744bb one of the packages, primitive, does not build anymore. At least, its last released version on Hackage. I see that the problem has been fixed on primitive's master (a2af610). But what should I do to actually test master branch of GHC now? > > -- > Best wishes, > Artem > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From omeragacan at gmail.com Fri Aug 10 11:55:38 2018 From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=) Date: Fri, 10 Aug 2018 14:55:38 +0300 Subject: Any ways to test a GHC build against large set of packages (including test suites)? In-Reply-To: References: Message-ID: I also briefly looked at hackage.head. As far as I understand it doesn't out-of-the-box provide a way to build a large set of packages, right? It'd be useful if I had a package that I want to test against GHC HEAD but currently it doesn't help me, unless I'm missing something. Ömer Ömer Sinan Ağacan , 10 Ağu 2018 Cum, 11:39 tarihinde şunu yazdı: > > Hi, > > This is working great, I just generated my first report. One problem is stm-2.4 > doesn't compile with GHC HEAD, we need stm-2.5.0.0. But that's not published on > Hackage yet, and latest nightly still uses stm-2.4.5.0. I wonder if there's > anything that can be done about this. Apparently stm blocks 82 packages (I > don't know if that's counting transitively or just packages that are directly > blocked by stm). Any ideas about this? > > Ömer > > Ömer Sinan Ağacan , 9 Ağu 2018 Per, 14:45 > tarihinde şunu yazdı: > > > > Ah, I now realize that that command is supposed to print that output. I'll > > continue following the steps and keep you updated if I get stuck again. > > > > Ömer > > > > Ömer Sinan Ağacan , 9 Ağu 2018 Per, 13:20 > > tarihinde şunu yazdı: > > > > > > Hi Manuel, > > > > > > I'm trying stackage-head. I'm following the steps for the scheduled build in > > > .circleci/config.yml. So far steps I took: > > > > > > - Installed ghc-head (from [1]) to ~/ghc-head > > > - Installed stackage-build-plan, stackage-curator and stackage-head (with > > > -fdev) from git repos, using stack. > > > - export BUILD_PLAN=nightly-2018-07-30 (from config.yml) > > > - curl https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/metadata.json > > > --output metadata.json > > > - curl https://raw.githubusercontent.com/fpco/stackage-nightly/master/$BUILD_PLAN.yaml > > > --output $BUILD_PLAN.yaml > > > > > > Now I'm doing > > > > > > - ./.local/bin/stackage-head already-seen --target $BUILD_PLAN > > > --ghc-metadata metadata.json --outdir build-reports > > > > > > but it's failing with > > > > > > The combination of target and commit is new to me > > > > > > Any ideas what I'm doing wrong? > > > > > > Thanks > > > > > > [1]: https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/bindist.tar.xz > > > > > > Ömer > > > > > > Ömer Sinan Ağacan , 7 Ağu 2018 Sal, 23:28 > > > tarihinde şunu yazdı: > > > > > > > > Thanks for both suggestions. I'll try both and see which one works better. > > > > > > > > Ömer > > > > > > > > Manuel M T Chakravarty , 7 Ağu 2018 Sal, 18:15 > > > > tarihinde şunu yazdı: > > > > > > > > > > Hi Ömer, > > > > > > > > > > This is exactly the motivation for the Stackage HEAD works that we have pushed at Tweag I/O in the context of the GHC DevOps group. Have a look at > > > > > > > > > > https://github.com/tweag/stackage-head > > > > > > > > > > and also the blog post from when the first version went live: > > > > > > > > > > https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html > > > > > > > > > > Cheers, > > > > > Manuel > > > > > > > > > > > Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan : > > > > > > > > > > > > Hi, > > > > > > > > > > > > I'd like to test some GHC builds + some compile and runtime flag combinations > > > > > > against a large set of packages by building them and running test suites. For > > > > > > this I need > > > > > > > > > > > > - A set of packages that are known to work with latest GHC > > > > > > - A way to build them and run their test suites (if I could specify compile and > > > > > > runtime flags that'd be even better) > > > > > > > > > > > > I think stackage can serve as (1) but I don't know how to do (2). Can anyone > > > > > > point me to the right direction? I vaguely remember some nix-based solution for > > > > > > this that was being discussed on the IRC channel, but can't recall any details. > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Ömer > > > > > > _______________________________________________ > > > > > > ghc-devs mailing list > > > > > > ghc-devs at haskell.org > > > > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > > > From ben at smart-cactus.org Sat Aug 11 00:06:33 2018 From: ben at smart-cactus.org (Ben Gamari) Date: Fri, 10 Aug 2018 20:06:33 -0400 Subject: Any ways to test a GHC build against large set of packages (including test suites)? In-Reply-To: References: Message-ID: <5E77C90C-E846-49DE-8862-2387CCB92D03@smart-cactus.org> On August 10, 2018 7:55:38 AM EDT, "Ömer Sinan Ağacan" wrote: >I also briefly looked at hackage.head. As far as I understand it >doesn't >out-of-the-box provide a way to build a large set of packages, right? >It'd be >useful if I had a package that I want to test against GHC HEAD but >currently it >doesn't help me, unless I'm missing something. > >Ömer > >Ömer Sinan Ağacan , 10 Ağu 2018 Cum, 11:39 >tarihinde şunu yazdı: >> >> Hi, >> >> This is working great, I just generated my first report. One problem >is stm-2.4 >> doesn't compile with GHC HEAD, we need stm-2.5.0.0. But that's not >published on >> Hackage yet, and latest nightly still uses stm-2.4.5.0. I wonder if >there's >> anything that can be done about this. Apparently stm blocks 82 >packages (I >> don't know if that's counting transitively or just packages that are >directly >> blocked by stm). Any ideas about this? >> >> Ömer >> >> Ömer Sinan Ağacan , 9 Ağu 2018 Per, 14:45 >> tarihinde şunu yazdı: >> > >> > Ah, I now realize that that command is supposed to print that >output. I'll >> > continue following the steps and keep you updated if I get stuck >again. >> > >> > Ömer >> > >> > Ömer Sinan Ağacan , 9 Ağu 2018 Per, 13:20 >> > tarihinde şunu yazdı: >> > > >> > > Hi Manuel, >> > > >> > > I'm trying stackage-head. I'm following the steps for the >scheduled build in >> > > .circleci/config.yml. So far steps I took: >> > > >> > > - Installed ghc-head (from [1]) to ~/ghc-head >> > > - Installed stackage-build-plan, stackage-curator and >stackage-head (with >> > > -fdev) from git repos, using stack. >> > > - export BUILD_PLAN=nightly-2018-07-30 (from config.yml) >> > > - curl >https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/metadata.json >> > > --output metadata.json >> > > - curl >https://raw.githubusercontent.com/fpco/stackage-nightly/master/$BUILD_PLAN.yaml >> > > --output $BUILD_PLAN.yaml >> > > >> > > Now I'm doing >> > > >> > > - ./.local/bin/stackage-head already-seen --target $BUILD_PLAN >> > > --ghc-metadata metadata.json --outdir build-reports >> > > >> > > but it's failing with >> > > >> > > The combination of target and commit is new to me >> > > >> > > Any ideas what I'm doing wrong? >> > > >> > > Thanks >> > > >> > > [1]: >https://ghc-artifacts.s3.amazonaws.com/nightly/validate-x86_64-linux/latest/bindist.tar.xz >> > > >> > > Ömer >> > > >> > > Ömer Sinan Ağacan , 7 Ağu 2018 Sal, 23:28 >> > > tarihinde şunu yazdı: >> > > > >> > > > Thanks for both suggestions. I'll try both and see which one >works better. >> > > > >> > > > Ömer >> > > > >> > > > Manuel M T Chakravarty , 7 Ağu 2018 Sal, >18:15 >> > > > tarihinde şunu yazdı: >> > > > > >> > > > > Hi Ömer, >> > > > > >> > > > > This is exactly the motivation for the Stackage HEAD works >that we have pushed at Tweag I/O in the context of the GHC DevOps >group. Have a look at >> > > > > >> > > > > https://github.com/tweag/stackage-head >> > > > > >> > > > > and also the blog post from when the first version went live: >> > > > > >> > > > > >https://www.tweag.io/posts/2018-04-17-stackage-head-is-live.html >> > > > > >> > > > > Cheers, >> > > > > Manuel >> > > > > >> > > > > > Am 06.08.2018 um 09:40 schrieb Ömer Sinan Ağacan >: >> > > > > > >> > > > > > Hi, >> > > > > > >> > > > > > I'd like to test some GHC builds + some compile and runtime >flag combinations >> > > > > > against a large set of packages by building them and >running test suites. For >> > > > > > this I need >> > > > > > >> > > > > > - A set of packages that are known to work with latest GHC >> > > > > > - A way to build them and run their test suites (if I could >specify compile and >> > > > > > runtime flags that'd be even better) >> > > > > > >> > > > > > I think stackage can serve as (1) but I don't know how to >do (2). Can anyone >> > > > > > point me to the right direction? I vaguely remember some >nix-based solution for >> > > > > > this that was being discussed on the IRC channel, but can't >recall any details. >> > > > > > >> > > > > > Thanks, >> > > > > > >> > > > > > Ömer >> > > > > > _______________________________________________ >> > > > > > ghc-devs mailing list >> > > > > > ghc-devs at haskell.org >> > > > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > > > > >_______________________________________________ >ghc-devs mailing list >ghc-devs at haskell.org >http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs Head.hackage doesn't have an out of the box package set but it is quite straightforward to construct such a set. While I now tend to use nix, in the past I generally just constructed a dummy cabal package listing the packages of interest as dependencies. There are two approaches to choosing a set of packages: extract the packages from Stackage's build-constraints.yaml or just additively build up a set from the top of your head. I find the latter is often more realistic; stackage is now large enough that even getting a fraction of it to build with a prerelease compiler can be a significant undertaking. From ben at well-typed.com Sat Aug 11 02:31:04 2018 From: ben at well-typed.com (Ben Gamari) Date: Fri, 10 Aug 2018 22:31:04 -0400 Subject: [ANNOUNCE] GHC 8.6.1-beta1 available Message-ID: <87y3ddprbw.fsf@smart-cactus.org> Hello everyone, The GHC development team is very pleased to announce the first beta leading up to GHC 8.6.1 release. The usual release artifacts are available from https://downloads.haskell.org/~ghc/8.6.1-beta1 This beta fixes most of the bugs reported in the first two alphas and brings all of the core libraries up to their final release versions. The 8.6 release fixes over 300 bugs from the 8.4 series and introduces a number of exciting features. These most notably include: * Significantly better handling of macOS linker command size limits, avoiding linker errors while linking large projects * A new deriving mechanism, `deriving via`, providing a convenient way for users to extend Haskell's typeclass deriving mechanism * Quantified constraints, allowing forall quantification in contexts * An early version of the GHCi `:doc` command * The `ghc-heap-view` package, allowing introspection into the structure of GHC's heap * Valid hole fit hints, helping the user to find terms to fill typed holes in their programs * The BlockArguments extension, allowing the `$` operator to be omitted in some unambiguous contexts * The next phase of the MonadFail proposal, enabling -XMonadFailDesugaring by default A full list of the changes in this release can be found in the release notes: https://downloads.haskell.org/~ghc/8.6.1-beta1/docs/html/users_guide/8.6.1-notes.html This will very likely be the last release before the final 8.6.1 so do give it a thorough testing and, as always, report any issues you encounter. Thanks for your help! Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From lonetiger at gmail.com Sat Aug 11 09:23:02 2018 From: lonetiger at gmail.com (Phyx) Date: Sat, 11 Aug 2018 10:23:02 +0100 Subject: [ANNOUNCE] GHC 8.6.1-beta1 available In-Reply-To: <87y3ddprbw.fsf@smart-cactus.org> References: <87y3ddprbw.fsf@smart-cactus.org> Message-ID: Hi Ben, I forgot to update the changelog at the time I think, It should probably be stated that this release fixes the 32 bit Windows segfaults. Cheers, Tamar On Sat, Aug 11, 2018 at 3:31 AM, Ben Gamari wrote: > > Hello everyone, > > The GHC development team is very pleased to announce the first beta > leading up to GHC 8.6.1 release. The usual release artifacts are > available from > > https://downloads.haskell.org/~ghc/8.6.1-beta1 > > This beta fixes most of the bugs reported in the first two alphas and > brings all of the core libraries up to their final release versions. > > The 8.6 release fixes over 300 bugs from the 8.4 series and introduces a > number of exciting features. These most notably include: > > * Significantly better handling of macOS linker command size limits, > avoiding linker errors while linking large projects > > * A new deriving mechanism, `deriving via`, providing a convenient way > for users to extend Haskell's typeclass deriving mechanism > > * Quantified constraints, allowing forall quantification in contexts > > * An early version of the GHCi `:doc` command > > * The `ghc-heap-view` package, allowing introspection into the > structure of GHC's heap > > * Valid hole fit hints, helping the user to find terms to fill typed > holes in their programs > > * The BlockArguments extension, allowing the `$` operator to be omitted > in some unambiguous contexts > > * The next phase of the MonadFail proposal, enabling > -XMonadFailDesugaring by default > > A full list of the changes in this release can be found in the > release notes: > > https://downloads.haskell.org/~ghc/8.6.1-beta1/docs/html/ > users_guide/8.6.1-notes.html > > This will very likely be the last release before the final 8.6.1 so do > give it a thorough testing and, as always, report any issues you > encounter. Thanks for your help! > > Cheers, > > - Ben > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mail at joachim-breitner.de Sat Aug 11 17:48:45 2018 From: mail at joachim-breitner.de (Joachim Breitner) Date: Sat, 11 Aug 2018 13:48:45 -0400 Subject: Any ways to test a GHC build against large set of packages (including test suites)? In-Reply-To: <5E77C90C-E846-49DE-8862-2387CCB92D03@smart-cactus.org> References: <5E77C90C-E846-49DE-8862-2387CCB92D03@smart-cactus.org> Message-ID: <336687e2db4e2a34a96e1fb069bcb8338a36c80e.camel@joachim-breitner.de> Hi, Am Freitag, den 10.08.2018, 20:06 -0400 schrieb Ben Gamari: > Head.hackage doesn't have an out of the box package set but it is > quite straightforward to construct such a set. While I now tend to > use nix, in the past I generally just constructed a dummy cabal > package listing the packages of interest as dependencies. > > There are two approaches to choosing a set of packages: extract the > packages from Stackage's build-constraints.yaml or just additively > build up a set from the top of your head. I find the latter is often > more realistic; stackage is now large enough that even getting a > fraction of it to build with a prerelease compiler can be a > significant undertaking. here is another package set worth considering, if you want a representative collection of packages with (somewhat) significance: The packages distributed by Debian: https://salsa.debian.org/haskell-team/package-plan/raw/master/packages.txt Cheers, Joachim -- Joachim Breitner mail at joachim-breitner.de http://www.joachim-breitner.de/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From rae at cs.brynmawr.edu Sun Aug 12 18:14:12 2018 From: rae at cs.brynmawr.edu (Richard Eisenberg) Date: Sun, 12 Aug 2018 14:14:12 -0400 Subject: [ANNOUNCE] GHC 8.6.1-beta1 available In-Reply-To: References: <87y3ddprbw.fsf@smart-cactus.org> Message-ID: > On Aug 11, 2018, at 11:27 AM, Vassil Ognyanov Keremidchiev wrote: > > What are the new features there toward Dependent Typed Haskell? Ben's link to https://downloads.haskell.org/~ghc/8.6.1-beta1/docs/html/users_guide/8.6.1-notes.html includes several items, pasted here: * A new StarIsType language extension has been added which controls whether * is parsed as Data.Kind.Type or a regular type operator. StarIsType is enabled by default. * CUSKs now require all kind variables to be explicitly quantified. This was already the case with TypeInType , but now PolyKinds also exhibits this behavior. * Functionality of TypeInType has been subsumed by PolyKinds , and it is now merely a shorthand for PolyKinds , DataKinds , and NoStarIsType . The users are advised to avoid TypeInType due to its misleading name: the Type :: Type axiom holds regardless of whether it is enabled. These are small steps, to be sure, but there's quite a bit going on behind the scenes. For example see the "Coercion Quantification" and "Type-level visible type applications" to take place at HIW (https://icfp18.sigplan.org/track/hiw-2018-papers#event-overview ). There are also a great many proposals in play at https://github.com/ghc-proposals/ghc-proposals More to come in the future, of course! Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From rmansorokin at gmail.com Mon Aug 13 13:57:41 2018 From: rmansorokin at gmail.com (Andrew Rmnsky) Date: Mon, 13 Aug 2018 16:57:41 +0300 Subject: Starting hacking on GHC Message-ID: Hello everyone! I would like to contribute to GHC, but I don't know where to start (I have built it already from the source). I'd be happy if someone gave me a piece of advice on what task I should pick for the beginning. A few words about my background: CS student, 4-th grade, have some experience with Haskell (I had a course on Haskell in my university where I studied some theory like Monoids, Functors, Applicatives Monads, Monad Transformers, Lens, basics of parallel and concurrent programming in Haskell and I have solved some algorithmic problems. I also practiced developing small web-apps with Haskell, using warp, aeson, postgresql-simple and some other libraries. Looking forward for your replies. Thank you in advance! Andrew Romanovsky. -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at ara.io Mon Aug 13 14:08:18 2018 From: me at ara.io (Ara Adkins) Date: Mon, 13 Aug 2018 15:08:18 +0100 Subject: Starting hacking on GHC In-Reply-To: References: Message-ID: Hi Andrew, Welcome! Before I go suggesting particular tickets, I just wanted to make sure that you've taken a look at this section https://ghc.haskell.org/trac/ghc/wiki/Newcomers#Findingaticket of the Newcomers page! It lists a decent set of tickets deemed workable for newcomers. This doesn't, of course, mean that you have to go it alone. In my experience people are always happy to help! Best, _ara On Mon, 13 Aug 2018 at 14:57, Andrew Rmnsky wrote: > > Hello everyone! I would like to contribute to GHC, but I don't know where > to start (I have built it already from the source). I'd be happy if someone > gave me a piece of advice on what task I should pick for the beginning. > > A few words about my background: > CS student, 4-th grade, have some experience with Haskell (I had a course > on Haskell in my university where I studied some theory like Monoids, > Functors, Applicatives Monads, Monad Transformers, Lens, basics of parallel > and concurrent programming in Haskell and I have solved some algorithmic > problems. I also practiced developing small web-apps with Haskell, using > warp, aeson, postgresql-simple and some other libraries. > > Looking forward for your replies. Thank you in advance! > > Andrew Romanovsky. > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Mon Aug 13 14:17:57 2018 From: ben at well-typed.com (Ben Gamari) Date: Mon, 13 Aug 2018 10:17:57 -0400 Subject: Starting hacking on GHC In-Reply-To: References: Message-ID: <87va8epcz3.fsf@smart-cactus.org> Andrew Rmnsky writes: > Hello everyone! I would like to contribute to GHC, but I don't know where > to start (I have built it already from the source). I'd be happy if someone > gave me a piece of advice on what task I should pick for the beginning. > Hi Andrew, Welcome! Have you seen the Newcomers page [1]? There you will find a list of tickets which various people have deemed good entry points into GHC development. Which you would like to pick up very much depends upon your background, interests, and goals. For instance, if you are familiar (or willing to learn) about stream fusion then #14037 would likely be an intriguing starting point. If you want to improve generics and do some simplifier sleuthing then perhaps #11068 is a better fit. If you are interested in working on the runtime then perhaps #14069 is of interest. Documentation contributions are also always appreciated (e.g. #14099, #14023). Cheers, - Ben [1] https://ghc.haskell.org/trac/ghc/wiki/Newcomers -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From fuuzetsu at fuuzetsu.co.uk Tue Aug 14 06:18:56 2018 From: fuuzetsu at fuuzetsu.co.uk (Mateusz Kowalczyk) Date: Tue, 14 Aug 2018 07:18:56 +0100 Subject: Can't build 8.6.1-beta1 with debugging. Message-ID: <1534227536.2373356.1473346560.2DABEABE@webmail.messagingengine.com> Hi all, I wanted to try our codebase with 8.6. I happened to already have 8.6.0.20180714 ready so I started with that. Compilation went well but I got a segfault when running a benchmark we have with profiling on. GDB told me the segfault was in stg_ap_ppppp_info in AutoApply.cmm which as I understand is generated. Strange but OK... I decided to try unprofiled with LLVM to see if some LLVM issue we had with current version (7.10.x) has gone away. Sadly I encountered a segfault during regular compilation (clean build, LLVM 6.0). I had no debugging symbols. Next I decided to try the beta1 version. I copied mk/build.mk.sample into mk/build.mk and added the following: GhcDebugged = YES GhcStage1HcOpts = -DDEBUG GhcStage2HcOpts = -DDEBUG I checked out the 8.6.1-beta1 tag then ran following. fuuzetsu at rubin:~/programming/ghc$ echo $PREFIX /usr/local/ghc/ghc-8.6.1-beta1 fuuzetsu at rubin:~/programming/ghc$ ./configure --prefix=$PREFIX && make -j4 && make install After some time I was met with: /usr/bin/ld.gold: error: cannot find -lHSrts_thr_debug_p Also a long chain of messages about undefined refs you can find at [1]. Considering I haven't touched any settings beyond adding debug flags, it surprised me a little that I can't build successfully. Please advise and feel free to ask for any specs. uname -a: Linux rubin.tsuru.it 4.15.0-30-generic #32~16.04.1-Ubuntu SMP Thu Jul 26 20:25:39 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux [1]: https://gist.github.com/Fuuzetsu/7a65a963dd625386d13938ba1e22af5c -- Mateusz K. From omeragacan at gmail.com Tue Aug 14 08:08:56 2018 From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=) Date: Tue, 14 Aug 2018 11:08:56 +0300 Subject: Can't build 8.6.1-beta1 with debugging. In-Reply-To: <1534227536.2373356.1473346560.2DABEABE@webmail.messagingengine.com> References: <1534227536.2373356.1473346560.2DABEABE@webmail.messagingengine.com> Message-ID: Hi Mateusz, > /usr/bin/ld.gold: error: cannot find -lHSrts_thr_debug_p We currently don't ship GHC with profiling + debug + threaded runtime. See my previous email on this: https://mail.haskell.org/pipermail/ghc-devs/2018-July/015982.html I show a way to enable these runtimes in https://ghc.haskell.org/trac/ghc/ticket/15508 Hope this helps, Ömer Mateusz Kowalczyk , 14 Ağu 2018 Sal, 09:19 tarihinde şunu yazdı: > > Hi all, > > I wanted to try our codebase with 8.6. I happened to already have 8.6.0.20180714 ready so I started with that. > > Compilation went well but I got a segfault when running a benchmark we have with profiling on. GDB told me the segfault was in stg_ap_ppppp_info in AutoApply.cmm which as I understand is generated. Strange but OK... I decided to try unprofiled with LLVM to see if some LLVM issue we had with current version (7.10.x) has gone away. Sadly I encountered a segfault during regular compilation (clean build, LLVM 6.0). I had no debugging symbols. > > Next I decided to try the beta1 version. I copied mk/build.mk.sample into mk/build.mk and added the following: > > GhcDebugged = YES > GhcStage1HcOpts = -DDEBUG > GhcStage2HcOpts = -DDEBUG > > I checked out the 8.6.1-beta1 tag then ran following. > > fuuzetsu at rubin:~/programming/ghc$ echo $PREFIX > /usr/local/ghc/ghc-8.6.1-beta1 > fuuzetsu at rubin:~/programming/ghc$ ./configure --prefix=$PREFIX && make -j4 && make install > > After some time I was met with: > > /usr/bin/ld.gold: error: cannot find -lHSrts_thr_debug_p > > Also a long chain of messages about undefined refs you can find at [1]. > > Considering I haven't touched any settings beyond adding debug flags, it surprised me a little that I can't build successfully. Please advise and feel free to ask for any specs. > > uname -a: > Linux rubin.tsuru.it 4.15.0-30-generic #32~16.04.1-Ubuntu SMP Thu Jul 26 20:25:39 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux > > > [1]: https://gist.github.com/Fuuzetsu/7a65a963dd625386d13938ba1e22af5c > > -- > Mateusz K. > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From simon.jakobi at googlemail.com Tue Aug 14 15:13:29 2018 From: simon.jakobi at googlemail.com (Simon Jakobi) Date: Tue, 14 Aug 2018 17:13:29 +0200 Subject: Status of the Hi Haddock project Message-ID: Hi! I have summarized the status of my GSoC project at https://sjakobi.github.io/blog/2018/08/14/hi-haddock-3/ My WIP patches for GHC and haddock are at https://phabricator.haskell.org/D5067 and https://github.com/haskell/haddock/pull/906. Cheers, Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.pelenitsyn at gmail.com Tue Aug 14 16:01:59 2018 From: a.pelenitsyn at gmail.com (Artem Pelenitsyn) Date: Tue, 14 Aug 2018 19:01:59 +0300 Subject: CI constanly fails on clonning Message-ID: Hello, devs, Could someone point me where I go wrong in the following? My Differentials constantly fail to CI with the messages like this one: exception 'PhabricatorWorkerPermanentFailureException' with message 'Lease "PHID-DRYL-sydepw7hjxlnim325sdu" never activated.' in /opt/phabricator/src/applications/harbormaster/step/HarbormasterLeaseWorkingCopyBuildStepImplementation.php:91 Stack trace: #0 /opt/phabricator/src/applications/harbormaster/worker/HarbormasterTargetWorker.php(70): HarbormasterLeaseWorkingCopyBuildStepImplementation->execute(Object(HarbormasterBuild), Object(HarbormasterBuildTarget)) #1 /opt/phabricator/src/infrastructure/daemon/workers/PhabricatorWorker.php(123): HarbormasterTargetWorker->doWork() #2 /opt/phabricator/src/infrastructure/daemon/workers/storage/PhabricatorWorkerActiveTask.php(171): PhabricatorWorker->executeTask() #3 /opt/phabricator/src/infrastructure/daemon/workers/PhabricatorTaskmasterDaemon.php(22): PhabricatorWorkerActiveTask->executeTask() #4 /opt/libphutil/src/daemon/PhutilDaemon.php(219): PhabricatorTaskmasterDaemon->run() #5 /opt/libphutil/scripts/daemon/exec/exec_daemon.php(131): PhutilDaemon->execute() #6 {main} Examples: https://phabricator.haskell.org/harbormaster/build/50735/ https://phabricator.haskell.org/harbormaster/build/51066/ https://phabricator.haskell.org/harbormaster/build/51616/ I googled this and found a similar topic on Phab's bug tracker: https://secure.phabricator.com/T10982 Not particularly helpful 'cause I don't really understand the internals of Phab. This started to happen roughly when I moved my ghc hacking setup to a new hardware this summer. Maybe, that's connected. Also, what suspicious is that `arc diff` tries to push changes in master right away, so I have to pass --skip-staging. This wasn't the case before, I believe. -- Best, Artem Best, Artem -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewtpickering at gmail.com Tue Aug 14 16:21:41 2018 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Tue, 14 Aug 2018 17:21:41 +0100 Subject: CI constanly fails on clonning In-Reply-To: References: Message-ID: It fails because you are using `--skip-staging`. It sounds like you haven't set up your SSH keys properly on your new hardware? Matt On Tue, Aug 14, 2018 at 5:01 PM, Artem Pelenitsyn wrote: > Hello, devs, > > Could someone point me where I go wrong in the following? My Differentials > constantly fail to CI with the messages like this one: > > exception 'PhabricatorWorkerPermanentFailureException' with message 'Lease > "PHID-DRYL-sydepw7hjxlnim325sdu" never activated.' in > /opt/phabricator/src/applications/harbormaster/step/HarbormasterLeaseWorkingCopyBuildStepImplementation.php:91 > Stack trace: > #0 > /opt/phabricator/src/applications/harbormaster/worker/HarbormasterTargetWorker.php(70): > HarbormasterLeaseWorkingCopyBuildStepImplementation->execute(Object(HarbormasterBuild), > Object(HarbormasterBuildTarget)) > #1 > /opt/phabricator/src/infrastructure/daemon/workers/PhabricatorWorker.php(123): > HarbormasterTargetWorker->doWork() > #2 > /opt/phabricator/src/infrastructure/daemon/workers/storage/PhabricatorWorkerActiveTask.php(171): > PhabricatorWorker->executeTask() > #3 > /opt/phabricator/src/infrastructure/daemon/workers/PhabricatorTaskmasterDaemon.php(22): > PhabricatorWorkerActiveTask->executeTask() > #4 /opt/libphutil/src/daemon/PhutilDaemon.php(219): > PhabricatorTaskmasterDaemon->run() > #5 /opt/libphutil/scripts/daemon/exec/exec_daemon.php(131): > PhutilDaemon->execute() > #6 {main} > > Examples: > https://phabricator.haskell.org/harbormaster/build/50735/ > https://phabricator.haskell.org/harbormaster/build/51066/ > https://phabricator.haskell.org/harbormaster/build/51616/ > > I googled this and found a similar topic on Phab's bug tracker: > https://secure.phabricator.com/T10982 > Not particularly helpful 'cause I don't really understand the internals of > Phab. > > This started to happen roughly when I moved my ghc hacking setup to a new > hardware this summer. Maybe, that's connected. Also, what suspicious is that > `arc diff` tries to push changes in master right away, so I have to pass > --skip-staging. This wasn't the case before, I believe. > > -- > Best, Artem > Best, Artem > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > From a.pelenitsyn at gmail.com Tue Aug 14 16:38:41 2018 From: a.pelenitsyn at gmail.com (Artem Pelenitsyn) Date: Tue, 14 Aug 2018 19:38:41 +0300 Subject: CI constanly fails on clonning In-Reply-To: References: Message-ID: Ouch, indeed! Thank you, Matt! -- Best, Artem On Tue, 14 Aug 2018 at 19:21 Matthew Pickering wrote: > It fails because you are using `--skip-staging`. It sounds like you > haven't set up your SSH keys properly on your new hardware? > > Matt > > On Tue, Aug 14, 2018 at 5:01 PM, Artem Pelenitsyn > wrote: > > Hello, devs, > > > > Could someone point me where I go wrong in the following? My > Differentials > > constantly fail to CI with the messages like this one: > > > > exception 'PhabricatorWorkerPermanentFailureException' with message > 'Lease > > "PHID-DRYL-sydepw7hjxlnim325sdu" never activated.' in > > > /opt/phabricator/src/applications/harbormaster/step/HarbormasterLeaseWorkingCopyBuildStepImplementation.php:91 > > Stack trace: > > #0 > > > /opt/phabricator/src/applications/harbormaster/worker/HarbormasterTargetWorker.php(70): > > > HarbormasterLeaseWorkingCopyBuildStepImplementation->execute(Object(HarbormasterBuild), > > Object(HarbormasterBuildTarget)) > > #1 > > > /opt/phabricator/src/infrastructure/daemon/workers/PhabricatorWorker.php(123): > > HarbormasterTargetWorker->doWork() > > #2 > > > /opt/phabricator/src/infrastructure/daemon/workers/storage/PhabricatorWorkerActiveTask.php(171): > > PhabricatorWorker->executeTask() > > #3 > > > /opt/phabricator/src/infrastructure/daemon/workers/PhabricatorTaskmasterDaemon.php(22): > > PhabricatorWorkerActiveTask->executeTask() > > #4 /opt/libphutil/src/daemon/PhutilDaemon.php(219): > > PhabricatorTaskmasterDaemon->run() > > #5 /opt/libphutil/scripts/daemon/exec/exec_daemon.php(131): > > PhutilDaemon->execute() > > #6 {main} > > > > Examples: > > https://phabricator.haskell.org/harbormaster/build/50735/ > > https://phabricator.haskell.org/harbormaster/build/51066/ > > https://phabricator.haskell.org/harbormaster/build/51616/ > > > > I googled this and found a similar topic on Phab's bug tracker: > > https://secure.phabricator.com/T10982 > > Not particularly helpful 'cause I don't really understand the internals > of > > Phab. > > > > This started to happen roughly when I moved my ghc hacking setup to a new > > hardware this summer. Maybe, that's connected. Also, what suspicious is > that > > `arc diff` tries to push changes in master right away, so I have to pass > > --skip-staging. This wasn't the case before, I believe. > > > > -- > > Best, Artem > > Best, Artem > > > > > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuuzetsu at fuuzetsu.co.uk Wed Aug 15 02:18:32 2018 From: fuuzetsu at fuuzetsu.co.uk (Mateusz Kowalczyk) Date: Wed, 15 Aug 2018 03:18:32 +0100 Subject: Can't build 8.6.1-beta1 with debugging. In-Reply-To: References: <1534227536.2373356.1473346560.2DABEABE@webmail.messagingengine.com> Message-ID: <1534299512.3587805.1474484952.469626E6@webmail.messagingengine.com> Hi, I was able to get GHC to compile with the patch in your ticket. Thanks! -- Mateusz K. On Tue, 14 Aug 2018, at 9:08 AM, Ömer Sinan Ağacan wrote: > Hi Mateusz, > > > /usr/bin/ld.gold: error: cannot find -lHSrts_thr_debug_p > > We currently don't ship GHC with profiling + debug + threaded runtime. See my > previous email on this: > https://mail.haskell.org/pipermail/ghc-devs/2018-July/015982.html > > I show a way to enable these runtimes in > https://ghc.haskell.org/trac/ghc/ticket/15508 > > Hope this helps, > > Ömer > > Mateusz Kowalczyk , 14 Ağu 2018 Sal, 09:19 > tarihinde şunu yazdı: > > > > Hi all, > > > > I wanted to try our codebase with 8.6. I happened to already have 8.6.0.20180714 ready so I started with that. > > > > Compilation went well but I got a segfault when running a benchmark we have with profiling on. GDB told me the segfault was in stg_ap_ppppp_info in AutoApply.cmm which as I understand is generated. Strange but OK... I decided to try unprofiled with LLVM to see if some LLVM issue we had with current version (7.10.x) has gone away. Sadly I encountered a segfault during regular compilation (clean build, LLVM 6.0). I had no debugging symbols. > > > > Next I decided to try the beta1 version. I copied mk/build.mk.sample into mk/build.mk and added the following: > > > > GhcDebugged = YES > > GhcStage1HcOpts = -DDEBUG > > GhcStage2HcOpts = -DDEBUG > > > > I checked out the 8.6.1-beta1 tag then ran following. > > > > fuuzetsu at rubin:~/programming/ghc$ echo $PREFIX > > /usr/local/ghc/ghc-8.6.1-beta1 > > fuuzetsu at rubin:~/programming/ghc$ ./configure --prefix=$PREFIX && make -j4 && make install > > > > After some time I was met with: > > > > /usr/bin/ld.gold: error: cannot find -lHSrts_thr_debug_p > > > > Also a long chain of messages about undefined refs you can find at [1]. > > > > Considering I haven't touched any settings beyond adding debug flags, it surprised me a little that I can't build successfully. Please advise and feel free to ask for any specs. > > > > uname -a: > > Linux rubin.tsuru.it 4.15.0-30-generic #32~16.04.1-Ubuntu SMP Thu Jul 26 20:25:39 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux > > > > > > [1]: https://gist.github.com/Fuuzetsu/7a65a963dd625386d13938ba1e22af5c > > > > -- > > Mateusz K. > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From david.feuer at gmail.com Sun Aug 19 17:53:18 2018 From: david.feuer at gmail.com (David Feuer) Date: Sun, 19 Aug 2018 13:53:18 -0400 Subject: Stable name equality Message-ID: eqStableName# is not currently defined as pointer equality. Is there some reason for this? My understanding is that there is a one-to-one correspondence between stable name objects and active stable name table entries, so pointer equality should be sufficient. Am I missing something? -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.feuer at gmail.com Sun Aug 19 18:07:07 2018 From: david.feuer at gmail.com (David Feuer) Date: Sun, 19 Aug 2018 14:07:07 -0400 Subject: Unlifted primop types Message-ID: I'm trying to implement a single primop to replace sameMutVar#, sameMutableArray#, etc. The primop should have type unliftedPtrEquality# :: forall (a :: TYPE 'UnliftedRep). a -> a -> Int# Unfortunately, I don't see a way to express this type in primops.pp.txt. Is it possible? If not, what's the right way to give the primop the right type? -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.feuer at gmail.com Sun Aug 19 22:41:07 2018 From: david.feuer at gmail.com (David Feuer) Date: Sun, 19 Aug 2018 18:41:07 -0400 Subject: Should StablePtr# have its own kind? Message-ID: Currently, StablePtr# :: Type -> TYPE 'AddrRep The fact that a stable pointer table index has the same size as a pointer seems somewhat coincidental. Would it make sense to add a `StablePtrRep` constructor to `RuntimeRep` and use that? That should allow StablePtr# to change its representation with less trouble, if that ever proves desirable. From simonpj at microsoft.com Mon Aug 20 11:56:44 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Mon, 20 Aug 2018 11:56:44 +0000 Subject: Stable name equality In-Reply-To: References: Message-ID: I defer to SimonM but IIRC each stable name corresponds to an entry in the stable-name table; the index in the table is the “stable” thing in a stable name. So I bet that comparison compares these indices. If two stable names are pointer-equal, they presumably contain the same index into the table. But is the reverse true? That is, can we be sure that two pointer-distinct stable name objects contain different indices? Simon From: ghc-devs On Behalf Of David Feuer Sent: 19 August 2018 18:53 To: ghc-devs Subject: Stable name equality eqStableName# is not currently defined as pointer equality. Is there some reason for this? My understanding is that there is a one-to-one correspondence between stable name objects and active stable name table entries, so pointer equality should be sufficient. Am I missing something? -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Mon Aug 20 12:11:14 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Mon, 20 Aug 2018 12:11:14 +0000 Subject: Unlifted primop types In-Reply-To: References: Message-ID: I don’t quite know how primops.txt.pp is processed, but perhaps by utils/genprimopcode. You may need to update the syntax a bit? Simon From: ghc-devs On Behalf Of David Feuer Sent: 19 August 2018 19:07 To: ghc-devs Subject: Unlifted primop types I'm trying to implement a single primop to replace sameMutVar#, sameMutableArray#, etc. The primop should have type unliftedPtrEquality# :: forall (a :: TYPE 'UnliftedRep). a -> a -> Int# Unfortunately, I don't see a way to express this type in primops.pp.txt. Is it possible? If not, what's the right way to give the primop the right type? -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Mon Aug 20 13:34:56 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Mon, 20 Aug 2018 13:34:56 +0000 Subject: Trac #13064: push Message-ID: Ben I've pushed fixes for Trac #13064/#15393, and also for #15487. The fixes are on wip/T13064 They concern error reporting for unused imports etc. I have not pushed to master because the fixes produce warnings in at least * containers * bytestring * text about unused imports. And indeed they are unused! But I'm not sure I'll the protocol right about updating those package, doing the right submodule updates etc. Also I have not compiled Haddock. Could you possibly close the loop on this - do the necessary changes and commit to master? Thanks! Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Mon Aug 20 13:44:46 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Mon, 20 Aug 2018 13:44:46 +0000 Subject: Sphinx faliing Message-ID: I have the following crash when building docs on Linux. This is on a clean, up-to-date HEAD build. Does anyone have any ideas? The "full traceback" is here # Sphinx version: 1.3.6 # Python version: 2.7.12 (CPython) # Docutils version: 0.12 release # Jinja2 version: 2.8 # Last messages: # Running Sphinx v1.3.6 # loading pickled environment... # not yet created # building [mo]: targets for 0 po files that are out of date # building [man]: all manpages # updating environment: # 40 added, 0 changed, 0 removed # reading sources... [ 2%] 8.2.1-notes # reading sources... [ 5%] 8.4.1-notes # Loaded extensions: # ghc_packages (1.0) from /5playpen/simonpj/HEAD-4/docs/users_guide/ghc_packages.pyc # sphinx.ext.extlinks (1.3.6) from /usr/lib/python2.7/dist-packages/sphinx/ext/extlinks.pyc # flags (1.0) from /5playpen/simonpj/HEAD-4/docs/users_guide/flags.pyc # sphinx.ext.mathjax (1.3.6) from /usr/lib/python2.7/dist-packages/sphinx/ext/mathjax.pyc # alabaster (0.7.7) from /usr/lib/python2.7/dist-packages/alabaster/__init__.pyc Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/sphinx/cmdline.py", line 244, in main app.build(opts.force_all, filenames) File "/usr/lib/python2.7/dist-packages/sphinx/application.py", line 267, in build self.builder.build_update() File "/usr/lib/python2.7/dist-packages/sphinx/builders/__init__.py", line 246, in build_update self.build(['__all__'], to_build) File "/usr/lib/python2.7/dist-packages/sphinx/builders/__init__.py", line 265, in build self.doctreedir, self.app)) File "/usr/lib/python2.7/dist-packages/sphinx/environment.py", line 547, in update self._read_serial(docnames, app) File "/usr/lib/python2.7/dist-packages/sphinx/environment.py", line 567, in _read_serial self.read_doc(docname, app) File "/usr/lib/python2.7/dist-packages/sphinx/environment.py", line 716, in read_doc encoding=self.config.source_encoding) File "/usr/lib/python2.7/dist-packages/sphinx/io.py", line 104, in __init__ FileInput.__init__(self, *args, **kwds) File "/usr/lib/python2.7/dist-packages/docutils/io.py", line 238, in __init__ raise InputError(error.errno, error.strerror, source_path) InputError: [Errno 2] No such file or directory: u'/5playpen/simonpj/HEAD-4/docs/users_guide/8.4.1-notes.rst' Simon Running Sphinx v1.3.6 loading pickled environment... not yet created building [mo]: targets for 0 po files that are out of date building [man]: all manpages updating environment: 40 added, 0 changed, 0 removed reading sources... [ 2%] 8.2.1-notes loading pickled environment... not yet created building [mo]: targets for 0 po files that are out of date building [html]: targets for 40 source files that are out of date updating environment: 40 added, 0 changed, 0 removed reading sources... [ 2%] 8.2.1-notes reading sources... [ 5%] 8.4.1-notes Exception occurred: File "/usr/lib/python2.7/dist-packages/docutils/io.py", line 238, in __init__ raise InputError(error.errno, error.strerror, source_path) InputError: [Errno 2] No such file or directory: u'/5playpen/simonpj/HEAD-4/docs/users_guide/8.4.1-notes.rst' The full traceback has been saved in /tmp/sphinx-err-XYpRbc.log, if you want to report the issue to the developers. Please also report this if it was a user error, so that a better error message can be provided next time. A bug report can be filed in the tracker at . Thanks! docs/users_guide/ghc.mk:28: recipe for target 'docs/users_guide/build-man/ghc.1' failed make[1]: *** [docs/users_guide/build-man/ghc.1] Error 1 make[1]: *** Waiting for unfinished jobs.... reading sources... [ 5%] 8.4.1-notes Exception occurred: File "/usr/lib/python2.7/dist-packages/docutils/io.py", line 238, in __init__ raise InputError(error.errno, error.strerror, source_path) InputError: [Errno 2] No such file or directory: u'/5playpen/simonpj/HEAD-4/docs/users_guide/8.4.1-notes.rst' The full traceback has been saved in /tmp/sphinx-err-2RrOTD.log, if you want to report the issue to the developers. Please also report this if it was a user error, so that a better error message can be provided next time. A bug report can be filed in the tracker at . Thanks! docs/users_guide/ghc.mk:16: recipe for target 'docs/users_guide/build-html/users_guide/index.html' failed make[1]: *** [docs/users_guide/build-html/users_guide/index.html] Error 1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Mon Aug 20 16:39:20 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Mon, 20 Aug 2018 16:39:20 +0000 Subject: Sphinx faliing In-Reply-To: References: Message-ID: Yes, blowing away the build tree and starting again sorted it. Simon From: Simon Peyton Jones Sent: 20 August 2018 14:45 To: ghc-devs Subject: Sphinx faliing I have the following crash when building docs on Linux. This is on a clean, up-to-date HEAD build. Does anyone have any ideas? The "full traceback" is here # Sphinx version: 1.3.6 # Python version: 2.7.12 (CPython) # Docutils version: 0.12 release # Jinja2 version: 2.8 # Last messages: # Running Sphinx v1.3.6 # loading pickled environment... # not yet created # building [mo]: targets for 0 po files that are out of date # building [man]: all manpages # updating environment: # 40 added, 0 changed, 0 removed # reading sources... [ 2%] 8.2.1-notes # reading sources... [ 5%] 8.4.1-notes # Loaded extensions: # ghc_packages (1.0) from /5playpen/simonpj/HEAD-4/docs/users_guide/ghc_packages.pyc # sphinx.ext.extlinks (1.3.6) from /usr/lib/python2.7/dist-packages/sphinx/ext/extlinks.pyc # flags (1.0) from /5playpen/simonpj/HEAD-4/docs/users_guide/flags.pyc # sphinx.ext.mathjax (1.3.6) from /usr/lib/python2.7/dist-packages/sphinx/ext/mathjax.pyc # alabaster (0.7.7) from /usr/lib/python2.7/dist-packages/alabaster/__init__.pyc Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/sphinx/cmdline.py", line 244, in main app.build(opts.force_all, filenames) File "/usr/lib/python2.7/dist-packages/sphinx/application.py", line 267, in build self.builder.build_update() File "/usr/lib/python2.7/dist-packages/sphinx/builders/__init__.py", line 246, in build_update self.build(['__all__'], to_build) File "/usr/lib/python2.7/dist-packages/sphinx/builders/__init__.py", line 265, in build self.doctreedir, self.app)) File "/usr/lib/python2.7/dist-packages/sphinx/environment.py", line 547, in update self._read_serial(docnames, app) File "/usr/lib/python2.7/dist-packages/sphinx/environment.py", line 567, in _read_serial self.read_doc(docname, app) File "/usr/lib/python2.7/dist-packages/sphinx/environment.py", line 716, in read_doc encoding=self.config.source_encoding) File "/usr/lib/python2.7/dist-packages/sphinx/io.py", line 104, in __init__ FileInput.__init__(self, *args, **kwds) File "/usr/lib/python2.7/dist-packages/docutils/io.py", line 238, in __init__ raise InputError(error.errno, error.strerror, source_path) InputError: [Errno 2] No such file or directory: u'/5playpen/simonpj/HEAD-4/docs/users_guide/8.4.1-notes.rst' Simon Running Sphinx v1.3.6 loading pickled environment... not yet created building [mo]: targets for 0 po files that are out of date building [man]: all manpages updating environment: 40 added, 0 changed, 0 removed reading sources... [ 2%] 8.2.1-notes loading pickled environment... not yet created building [mo]: targets for 0 po files that are out of date building [html]: targets for 40 source files that are out of date updating environment: 40 added, 0 changed, 0 removed reading sources... [ 2%] 8.2.1-notes reading sources... [ 5%] 8.4.1-notes Exception occurred: File "/usr/lib/python2.7/dist-packages/docutils/io.py", line 238, in __init__ raise InputError(error.errno, error.strerror, source_path) InputError: [Errno 2] No such file or directory: u'/5playpen/simonpj/HEAD-4/docs/users_guide/8.4.1-notes.rst' The full traceback has been saved in /tmp/sphinx-err-XYpRbc.log, if you want to report the issue to the developers. Please also report this if it was a user error, so that a better error message can be provided next time. A bug report can be filed in the tracker at . Thanks! docs/users_guide/ghc.mk:28: recipe for target 'docs/users_guide/build-man/ghc.1' failed make[1]: *** [docs/users_guide/build-man/ghc.1] Error 1 make[1]: *** Waiting for unfinished jobs.... reading sources... [ 5%] 8.4.1-notes Exception occurred: File "/usr/lib/python2.7/dist-packages/docutils/io.py", line 238, in __init__ raise InputError(error.errno, error.strerror, source_path) InputError: [Errno 2] No such file or directory: u'/5playpen/simonpj/HEAD-4/docs/users_guide/8.4.1-notes.rst' The full traceback has been saved in /tmp/sphinx-err-2RrOTD.log, if you want to report the issue to the developers. Please also report this if it was a user error, so that a better error message can be provided next time. A bug report can be filed in the tracker at . Thanks! docs/users_guide/ghc.mk:16: recipe for target 'docs/users_guide/build-html/users_guide/index.html' failed make[1]: *** [docs/users_guide/build-html/users_guide/index.html] Error 1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Mon Aug 20 18:37:19 2018 From: ben at well-typed.com (Ben Gamari) Date: Mon, 20 Aug 2018 14:37:19 -0400 Subject: Goals for GHC 8.8 Message-ID: <87wosknaub.fsf@smart-cactus.org> Hi everyone, Since GHC 8.6 is almost released it's time to start thinking about our goals for 8.8, which will branch in November. I have added the items that I know are in-flight to the 8.8 status page[1]. If you have a project that you would like to see present in 8.8 then please do add it as well. If you are uncertain of whether you can have a patch up for review by late October then please do speak to me and we can discuss options. Cheers, - Ben [1] https://ghc.haskell.org/trac/ghc/wiki/Status/GHC-8.8.1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Mon Aug 20 21:33:27 2018 From: ben at well-typed.com (Ben Gamari) Date: Mon, 20 Aug 2018 17:33:27 -0400 Subject: GHC 8.6.1 release status Message-ID: <87o9dwn2ot.fsf@smart-cactus.org> Hi everyone, I was getting ready to prepare the final 8.6.1 release tarballs when Herbert pointed out that the testsuite of his cryptohash-sha256 package currently segfaults. This appears to be a rather serious, non-deterministic regression. I've opened #15544 to track this and will delay the final 8.6.1 release until this is resolved. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From matthewtpickering at gmail.com Mon Aug 20 21:40:28 2018 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Mon, 20 Aug 2018 22:40:28 +0100 Subject: GHC 8.6.1 release status In-Reply-To: <87o9dwn2ot.fsf@smart-cactus.org> References: <87o9dwn2ot.fsf@smart-cactus.org> Message-ID: I think the release should include this fix to haddock. Will that happen? https://github.com/haskell/haddock/pull/905 Matt On Mon, Aug 20, 2018 at 10:33 PM Ben Gamari wrote: > > Hi everyone, > > I was getting ready to prepare the final 8.6.1 release tarballs when > Herbert pointed out that the testsuite of his cryptohash-sha256 package > currently segfaults. This appears to be a rather serious, non-deterministic > regression. I've opened #15544 to track this and will delay the final > 8.6.1 release until this is resolved. > > Cheers, > > - Ben > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From david at well-typed.com Mon Aug 20 22:36:02 2018 From: david at well-typed.com (David Feuer) Date: Mon, 20 Aug 2018 18:36:02 -0400 Subject: Stable name equality In-Reply-To: References: Message-ID: <3084967.pfA7JnK5J5@squirrel> On Monday, August 20, 2018 11:56:44 AM EDT Simon Peyton Jones via ghc-devs wrote: > I defer to SimonM but IIRC each stable name corresponds to an entry in the stable-name table; the index in the table is the “stable” thing in a stable name. So I bet that comparison compares these indices. > > If two stable names are pointer-equal, they presumably contain the same index into the table. > > But is the reverse true? That is, can we be sure that two pointer-distinct stable name objects contain different indices? I believe so, yes. Each stable name table entry has a pointer to the linked stable name object. Calling makeStableName# checks whether the passed pointer already has a stable name, and, if so, returns the linked stable name object. The design seems a bit surprising to me, but it looks like that's actually how it works, at least for now. Each call locks the stable name table, so it shouldn't be possible to miss entry creation. From ben at well-typed.com Mon Aug 20 23:30:47 2018 From: ben at well-typed.com (Ben Gamari) Date: Mon, 20 Aug 2018 19:30:47 -0400 Subject: GHC 8.6.1 release status In-Reply-To: References: <87o9dwn2ot.fsf@smart-cactus.org> Message-ID: <87k1okmx99.fsf@smart-cactus.org> Matthew Pickering writes: > I think the release should include this fix to haddock. Will that happen? > > https://github.com/haskell/haddock/pull/905 > Yes, we can make that happen. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From david at well-typed.com Tue Aug 21 00:42:38 2018 From: david at well-typed.com (David Feuer) Date: Mon, 20 Aug 2018 20:42:38 -0400 Subject: Unlifted primop types In-Reply-To: References: Message-ID: <1763136.o1izTAbIkI@squirrel> On Monday, August 20, 2018 12:11:14 PM EDT Simon Peyton Jones via ghc-devs wrote: > I don’t quite know how primops.txt.pp is processed, but perhaps by utils/genprimopcode. You may need to update the syntax a bit? The whole thing is a mystery to me. Whatever it's doing seems to generate code that puts things in TysPrim.hs together, and that module also looks rather murky to the uninitiated. From svenpanne at gmail.com Tue Aug 21 06:52:29 2018 From: svenpanne at gmail.com (Sven Panne) Date: Tue, 21 Aug 2018 08:52:29 +0200 Subject: GHC 8.6.1 release status In-Reply-To: References: <87o9dwn2ot.fsf@smart-cactus.org> Message-ID: Am Mo., 20. Aug. 2018 um 23:40 Uhr schrieb Matthew Pickering < matthewtpickering at gmail.com>: > I think the release should include this fix to haddock. Will that happen? > > https://github.com/haskell/haddock/pull/905 Will the release contain the fix https://github.com/haskell/haddock/pull/893 for the issue https://github.com/haskell/haddock/issues/462 too? Without that fix, Haddock is basically unusable for lots of packages under the memory restrictions of Travis CI. Cheers, S. -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.feuer at gmail.com Tue Aug 21 07:09:16 2018 From: david.feuer at gmail.com (David Feuer) Date: Tue, 21 Aug 2018 03:09:16 -0400 Subject: Stable name equality In-Reply-To: <3084967.pfA7JnK5J5@squirrel> References: <3084967.pfA7JnK5J5@squirrel> Message-ID: I had another thought. If we want, I believe we can make the stable name mechanism considerably more compact by giving up on a flat array representation for the stable name table. The flat representation means that enlarging the table moves all its entries. Suppose we instead choose some shallow tree representation that can grow without moving (an array of arrays comes to mind, where each array is null or twice the size of the last; index calculations should be pretty simple). I believe we can then play some nice tricks: 1. Lay out each entry in the stable name table like a heap object. 2. Make each StableName# a pointer directly to its stable name table entry. So instead of a stable name object and a stable name table entry that points to it, we'd just have the stable name entry. I believe we could run the free list through the "heap object" headers. -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Tue Aug 21 07:38:56 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Tue, 21 Aug 2018 07:38:56 +0000 Subject: Stable name equality In-Reply-To: <3084967.pfA7JnK5J5@squirrel> References: <3084967.pfA7JnK5J5@squirrel> Message-ID: | > But is the reverse true? That is, can we be sure that two pointer- | distinct stable name objects contain different indices? | | I believe so, yes. Each stable name table entry has a pointer to the | linked stable name object. Calling makeStableName# checks whether the | passed pointer already has a stable name, and, if so, returns the linked | stable name object. Actually I'm confused. Currently we have data StableName a = StableName (StableName# a) I believe (but there is no documentation to state) that the (StableName# a) * Has kind (TYPE UnliftedRep) * Is the index of the entry in the Stable Name Table But if it's the index, why isn't it an IntRep? UnliftedRep is for pointers? Moreover eqStableName# :: StableName# a -> StableName# b -> Bool is directly implemented in the code generator (StgCmmPrim) by an equality comparison. If these things are correct, it would be great to write them down in a Note. And if they are right, I'm now lost about what you question is. Equality is /already/ implemented by direct equality comparison, no? Simon | -----Original Message----- | From: David Feuer | Sent: 20 August 2018 23:36 | To: ghc-devs at haskell.org; Simon Peyton Jones | Cc: David Feuer ; marlowsd at gmail.com | Subject: Re: Stable name equality | | On Monday, August 20, 2018 11:56:44 AM EDT Simon Peyton Jones via ghc- | devs wrote: | > I defer to SimonM but IIRC each stable name corresponds to an entry in | the stable-name table; the index in the table is the “stable” thing in a | stable name. So I bet that comparison compares these indices. | > | > If two stable names are pointer-equal, they presumably contain the same | index into the table. | > | > But is the reverse true? That is, can we be sure that two pointer- | distinct stable name objects contain different indices? | | I believe so, yes. Each stable name table entry has a pointer to the | linked stable name object. Calling makeStableName# checks whether the | passed pointer already has a stable name, and, if so, returns the linked | stable name object. The design seems a bit surprising to me, but it looks | like that's actually how it works, at least for now. Each call locks the | stable name table, so it shouldn't be possible to miss entry creation. From david.feuer at gmail.com Tue Aug 21 08:24:45 2018 From: david.feuer at gmail.com (David Feuer) Date: Tue, 21 Aug 2018 04:24:45 -0400 Subject: Stable name equality In-Reply-To: References: <3084967.pfA7JnK5J5@squirrel> Message-ID: On Tue, Aug 21, 2018, 3:39 AM Simon Peyton Jones wrote: > > Actually I'm confused. Currently we have > > data StableName a = StableName (StableName# a) > > I believe (but there is no documentation to state) that the (StableName# > a) > * Has kind (TYPE UnliftedRep) > * Is the index of the entry in the Stable Name Table > > But if it's the index, why isn't it an IntRep? UnliftedRep is for > pointers? > That's explained in the paper. A StableName# is a pointer to a stable name object in the heap that *contains* an index into the stable name table. Basically, the garbage collector needs to know whether a stable name is alive or not, so it can work out when to clear it from the table. > Moreover eqStableName# :: StableName# a -> StableName# b -> Bool > is directly implemented in the code generator (StgCmmPrim) by an equality > comparison. > > If these things are correct, it would be great to write them down in a > Note. > > And if they are right, I'm now lost about what you question is. Equality > is > /already/ implemented by direct equality comparison, no? > It's currently implemented by an equality test on the indices stored in the stable name objects, rather than the pointers to those objects, if I'm not very much mistaken. -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Tue Aug 21 08:32:11 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Tue, 21 Aug 2018 08:32:11 +0000 Subject: Stable name equality In-Reply-To: References: <3084967.pfA7JnK5J5@squirrel> Message-ID: That's explained in the paper. A StableName# is a pointer to a stable name object in the heap that *contains* an index into the stable name table. Basically, the garbage collector needs to know whether a stable name is alive or not, so it can work out when to clear it from the table. Very good. But could it be explained in a Note too? The paper is from a long time ago, contains lots of surrounding explanation, and might well be out of date (even if it in fact is not out of date). So the entry in the table /also/ points to the same, heap-allocated StableName#? Doesn’t that keep it alive? Or is this akin to the treatment of weak pointers? (Which is part of the same paper.) Do we anywhere keep a pointer to the object that this is a stable name of? Simon From: David Feuer Sent: 21 August 2018 09:25 To: Simon Peyton Jones Cc: David Feuer ; ghc-devs ; marlowsd at gmail.com Subject: Re: Stable name equality On Tue, Aug 21, 2018, 3:39 AM Simon Peyton Jones > wrote: Actually I'm confused. Currently we have data StableName a = StableName (StableName# a) I believe (but there is no documentation to state) that the (StableName# a) * Has kind (TYPE UnliftedRep) * Is the index of the entry in the Stable Name Table But if it's the index, why isn't it an IntRep? UnliftedRep is for pointers? That's explained in the paper. A StableName# is a pointer to a stable name object in the heap that *contains* an index into the stable name table. Basically, the garbage collector needs to know whether a stable name is alive or not, so it can work out when to clear it from the table. Moreover eqStableName# :: StableName# a -> StableName# b -> Bool is directly implemented in the code generator (StgCmmPrim) by an equality comparison. If these things are correct, it would be great to write them down in a Note. And if they are right, I'm now lost about what you question is. Equality is /already/ implemented by direct equality comparison, no? It's currently implemented by an equality test on the indices stored in the stable name objects, rather than the pointers to those objects, if I'm not very much mistaken. -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.feuer at gmail.com Tue Aug 21 09:21:02 2018 From: david.feuer at gmail.com (David Feuer) Date: Tue, 21 Aug 2018 05:21:02 -0400 Subject: Stable name equality In-Reply-To: References: <3084967.pfA7JnK5J5@squirrel> Message-ID: On Tue, Aug 21, 2018, 4:32 AM Simon Peyton Jones wrote: > That's explained in the paper. A StableName# is a pointer to a stable name > object in the heap that *contains* an index into the stable name table. > Basically, the garbage collector needs to know whether a stable name is > alive or not, so it can work out when to clear it from the table. > > Very good. But could it be explained in a Note too? The paper is from a > long time ago, contains lots of surrounding explanation, and might well be > out of date (even if it in fact is not out of date). > Certainly there should be a note, but as I mentioned in this thread, I think we can probably actually do better than we presently do. > > So the entry in the table /also/ points to the same, heap-allocated > StableName#? Doesn’t that keep it alive? Or is this akin to the treatment > of weak pointers? (Which is part of the same paper.) > The stable name table is not in the root set. All its references are weak. > > Do we anywhere keep a pointer to the object that this is a stable name of? > Yes. That's in the stable name table entry. It's also the key for the stable name hash table. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wolfgang-it at jeltsch.info Tue Aug 21 13:21:32 2018 From: wolfgang-it at jeltsch.info (Wolfgang Jeltsch) Date: Tue, 21 Aug 2018 16:21:32 +0300 Subject: Goals for GHC 8.8 In-Reply-To: <87wosknaub.fsf@smart-cactus.org> References: <87wosknaub.fsf@smart-cactus.org> Message-ID: <1534857692.2717.56.camel@jeltsch.info> Hi! What happened to LinearTypes? Will it be in GHC 8.8? All the best, Wolfgang Am Montag, den 20.08.2018, 14:37 -0400 schrieb Ben Gamari: > Hi everyone, > > Since GHC 8.6 is almost released it's time to start thinking about our > goals for 8.8, which will branch in November. I have added the items > that I know are in-flight to the 8.8 status page[1]. If you have a > project that you would like to see present in 8.8 then please do add > it > as well. > > If you are uncertain of whether you can have a patch up for review by > late October then please do speak to me and we can discuss options. > > Cheers, > > - Ben > > > [1] https://ghc.haskell.org/trac/ghc/wiki/Status/GHC-8.8.1 > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From ben at well-typed.com Tue Aug 21 14:44:26 2018 From: ben at well-typed.com (Ben Gamari) Date: Tue, 21 Aug 2018 10:44:26 -0400 Subject: Goals for GHC 8.8 In-Reply-To: <1534857692.2717.56.camel@jeltsch.info> References: <87wosknaub.fsf@smart-cactus.org> <1534857692.2717.56.camel@jeltsch.info> Message-ID: <87h8jnn5iy.fsf@smart-cactus.org> Wolfgang Jeltsch writes: > Hi! > > What happened to LinearTypes? Will it be in GHC 8.8? > It's unclear; the proposal authors (Arnaud is CC'd) will need to comment here. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From m at tweag.io Tue Aug 21 14:47:29 2018 From: m at tweag.io (Mathieu Boespflug) Date: Tue, 21 Aug 2018 16:47:29 +0200 Subject: Goals for GHC 8.8 In-Reply-To: <87h8jnn5iy.fsf@smart-cactus.org> References: <87wosknaub.fsf@smart-cactus.org> <1534857692.2717.56.camel@jeltsch.info> <87h8jnn5iy.fsf@smart-cactus.org> Message-ID: The proposal would need to be accepted by the GHC Steering Committee first before that happens. Best, -- Mathieu Boespflug Founder at http://tweag.io. On Tue, 21 Aug 2018 at 16:44, Ben Gamari wrote: > Wolfgang Jeltsch writes: > > > Hi! > > > > What happened to LinearTypes? Will it be in GHC 8.8? > > > It's unclear; the proposal authors (Arnaud is CC'd) will need to comment > here. > > Cheers, > > - Ben > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at smart-cactus.org Tue Aug 21 15:07:55 2018 From: ben at smart-cactus.org (Ben Gamari) Date: Tue, 21 Aug 2018 11:07:55 -0400 Subject: Unlifted primop types In-Reply-To: <1763136.o1izTAbIkI@squirrel> References: <1763136.o1izTAbIkI@squirrel> Message-ID: <87efern4fs.fsf@smart-cactus.org> David Feuer writes: > On Monday, August 20, 2018 12:11:14 PM EDT Simon Peyton Jones via ghc-devs wrote: >> I don’t quite know how primops.txt.pp is processed, but perhaps by utils/genprimopcode. You may need to update the syntax a bit? > > The whole thing is a mystery to me. Whatever it's doing seems to > generate code that puts things in TysPrim.hs together, and that module > also looks rather murky to the uninitiated. primops.txt.pp is processed by genprimopcode, as suggested by Simon. genprimopcode has several modes, each of which produces a different output. These are listed in compiler/ghc.mk; grep for preprocessCompilerFiles. They broadly fall into three categories: * A set of headers which define the `PrimOp` type and functions defining primops' various properties (e.g. out-of-line-ness) * The GHC.PrimopWrappers module of `ghc-prim`, which defines functions wrapping each of the primops; these are used by GHCi (see Note [Primop Wrappers] * The `GHC.Prim`module of `ghc-prim`, which has no code but rather merely serves as a documented source file to be used by Haddock. genprimopcode's notion of non-Type kinded tyvars is very limited. It provides a few tyvars of kind Type (namely a, b, and c) and an "open-kinded" tyvar of any runtime rep (namely o). I believe you would need to add an additional binder to the genopcode parser to get what you want. I suspect this would only require a few lines, however. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at smart-cactus.org Tue Aug 21 15:09:47 2018 From: ben at smart-cactus.org (Ben Gamari) Date: Tue, 21 Aug 2018 11:09:47 -0400 Subject: GHC 8.6.1 release status In-Reply-To: References: <87o9dwn2ot.fsf@smart-cactus.org> Message-ID: <87bm9vn4cn.fsf@smart-cactus.org> Sven Panne writes: > Am Mo., 20. Aug. 2018 um 23:40 Uhr schrieb Matthew Pickering < > matthewtpickering at gmail.com>: > >> I think the release should include this fix to haddock. Will that happen? >> >> https://github.com/haskell/haddock/pull/905 > > > Will the release contain the fix > > https://github.com/haskell/haddock/pull/893 > > for the issue > > https://github.com/haskell/haddock/issues/462 > > too? Without that fix, Haddock is basically unusable for lots of packages > under the memory restrictions of Travis CI. > Alex, do you have any objection to merging these? Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From simon.jakobi at googlemail.com Tue Aug 21 15:13:37 2018 From: simon.jakobi at googlemail.com (Simon Jakobi) Date: Tue, 21 Aug 2018 17:13:37 +0200 Subject: GHC 8.6.1 release status In-Reply-To: <87bm9vn4cn.fsf@smart-cactus.org> References: <87o9dwn2ot.fsf@smart-cactus.org> <87bm9vn4cn.fsf@smart-cactus.org> Message-ID: Hi Ben, I talked to Alex about these today: * PR #905 has been merged. * PR #893 requires the merge of https://phabricator.haskell.org/D5003. Apart from that there were no objections. Cheers, Simon Am Di., 21. Aug. 2018 um 17:09 Uhr schrieb Ben Gamari : > Sven Panne writes: > > > Am Mo., 20. Aug. 2018 um 23:40 Uhr schrieb Matthew Pickering < > > matthewtpickering at gmail.com>: > > > >> I think the release should include this fix to haddock. Will that > happen? > >> > >> https://github.com/haskell/haddock/pull/905 > > > > > > Will the release contain the fix > > > > https://github.com/haskell/haddock/pull/893 > > > > for the issue > > > > https://github.com/haskell/haddock/issues/462 > > > > too? Without that fix, Haddock is basically unusable for lots of packages > > under the memory restrictions of Travis CI. > > > Alex, do you have any objection to merging these? > > Cheers, > > - Ben > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.feuer at gmail.com Tue Aug 21 15:56:07 2018 From: david.feuer at gmail.com (David Feuer) Date: Tue, 21 Aug 2018 11:56:07 -0400 Subject: Unlifted primop types In-Reply-To: <87efern4fs.fsf@smart-cactus.org> References: <1763136.o1izTAbIkI@squirrel> <87efern4fs.fsf@smart-cactus.org> Message-ID: But how do I have to build something like "openAlphaTyVar" for TYPE 'UnliftedRep in the primitive space? I don't understand the toolkit. On Tue, Aug 21, 2018, 11:07 AM Ben Gamari wrote: > David Feuer writes: > > > On Monday, August 20, 2018 12:11:14 PM EDT Simon Peyton Jones via > ghc-devs wrote: > >> I don’t quite know how primops.txt.pp is processed, but perhaps by > utils/genprimopcode. You may need to update the syntax a bit? > > > > The whole thing is a mystery to me. Whatever it's doing seems to > > generate code that puts things in TysPrim.hs together, and that module > > also looks rather murky to the uninitiated. > > primops.txt.pp is processed by genprimopcode, as suggested by Simon. > genprimopcode has several modes, each of which produces a different > output. These are listed in compiler/ghc.mk; grep for > preprocessCompilerFiles. They broadly fall into three categories: > > * A set of headers which define the `PrimOp` type and functions defining > primops' various properties (e.g. out-of-line-ness) > > * The GHC.PrimopWrappers module of `ghc-prim`, which defines functions > wrapping each of the primops; these are used by GHCi (see Note > [Primop Wrappers] > > * The `GHC.Prim`module of `ghc-prim`, which has no code but rather > merely serves as a documented source file to be used by Haddock. > > genprimopcode's notion of non-Type kinded tyvars is very limited. It > provides a few tyvars of kind Type (namely a, b, and c) and an > "open-kinded" tyvar of any runtime rep (namely o). I believe you would > need to add an additional binder to the genopcode parser to get what you > want. I suspect this would only require a few lines, however. > > Cheers, > > - Ben > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Tue Aug 21 19:34:42 2018 From: ben at well-typed.com (Ben Gamari) Date: Tue, 21 Aug 2018 15:34:42 -0400 Subject: Goals for GHC 8.8 In-Reply-To: References: <87wosknaub.fsf@smart-cactus.org> <1534857692.2717.56.camel@jeltsch.info> <87h8jnn5iy.fsf@smart-cactus.org> Message-ID: <8736v7ms34.fsf@smart-cactus.org> Mathieu Boespflug writes: > The proposal would need to be accepted by the GHC Steering Committee first > before that happens. > Absolutely; I just wasn't sure whether you were considering pushing for merge in the event that it was accepted. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From wolfgang-it at jeltsch.info Tue Aug 21 20:14:32 2018 From: wolfgang-it at jeltsch.info (Wolfgang Jeltsch) Date: Tue, 21 Aug 2018 23:14:32 +0300 Subject: Goals for GHC 8.8 In-Reply-To: References: <87wosknaub.fsf@smart-cactus.org> <1534857692.2717.56.camel@jeltsch.info> <87h8jnn5iy.fsf@smart-cactus.org> Message-ID: <1534882472.2717.67.camel@jeltsch.info> Right. My understanding from looking at https://github.com/ghc-proposals /ghc-proposals/pull/111 is that the discussion about this proposal is generally moving forward and doesn’t have died. Is this understanding correct? All the best, Wolfgang Am Dienstag, den 21.08.2018, 16:47 +0200 schrieb Mathieu Boespflug: > The proposal would need to be accepted by the GHC Steering Committee > first before that happens. > > Best, > > -- > Mathieu Boespflug > Founder at http://tweag.io. > > > On Tue, 21 Aug 2018 at 16:44, Ben Gamari wrote: > > Wolfgang Jeltsch writes: > > > > > Hi! > > > > > > What happened to LinearTypes? Will it be in GHC 8.8? > > > > > It's unclear; the proposal authors (Arnaud is CC'd) will need to > > comment > > here. > > > > Cheers, > > > > - Ben > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at smart-cactus.org Tue Aug 21 20:14:35 2018 From: ben at smart-cactus.org (Ben Gamari) Date: Tue, 21 Aug 2018 16:14:35 -0400 Subject: Unlifted primop types In-Reply-To: References: <1763136.o1izTAbIkI@squirrel> <87efern4fs.fsf@smart-cactus.org> Message-ID: <87tvnnlboc.fsf@smart-cactus.org> David Feuer writes: > But how do I have to build something like "openAlphaTyVar" for TYPE > 'UnliftedRep in the primitive space? I don't understand the toolkit. > I'm not sure I understand the question. openAlphaTyVar is defined in TysPrim. Perhaps looking there will clear up the confusion. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From david.feuer at gmail.com Tue Aug 21 20:18:39 2018 From: david.feuer at gmail.com (David Feuer) Date: Tue, 21 Aug 2018 16:18:39 -0400 Subject: Unlifted primop types In-Reply-To: <87tvnnlboc.fsf@smart-cactus.org> References: <1763136.o1izTAbIkI@squirrel> <87efern4fs.fsf@smart-cactus.org> <87tvnnlboc.fsf@smart-cactus.org> Message-ID: I see the definition, but I don't understand it, so I don't know how to adapt it to the less-polymorphic version I need. On Tue, Aug 21, 2018, 4:14 PM Ben Gamari wrote: > David Feuer writes: > > > But how do I have to build something like "openAlphaTyVar" for TYPE > > 'UnliftedRep in the primitive space? I don't understand the toolkit. > > > I'm not sure I understand the question. openAlphaTyVar is defined in > TysPrim. Perhaps looking there will clear up the confusion. > > Cheers, > > - Ben > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at well-typed.com Tue Aug 21 23:25:32 2018 From: david at well-typed.com (David Feuer) Date: Tue, 21 Aug 2018 19:25:32 -0400 Subject: Unlifted primop types In-Reply-To: References: <87tvnnlboc.fsf@smart-cactus.org> Message-ID: <1588913.T5q9MkzbpF@squirrel> Huh! It looks like what we currently do for some primops is just use a totally bogus kind. For example, mkWeak# will happily accept an Int# as its first argument. So we *could* follow that precedent and generalize reallyUnsafePtrEquality#, makeStableName#, etc., to accept anything, whether it makes sense or not. Or we can work out how to do what I was trying to do and then duplicate those primitives as appropriate (mkLiftedWeak#, mkUnliftedWeak#, makeLiftedStableName#, makeUnliftedStableName#, etc.). From simonpj at microsoft.com Wed Aug 22 09:44:52 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 22 Aug 2018 09:44:52 +0000 Subject: Unlifted primop types In-Reply-To: <1588913.T5q9MkzbpF@squirrel> References: <87tvnnlboc.fsf@smart-cactus.org> <1588913.T5q9MkzbpF@squirrel> Message-ID: | Huh! It looks like what we currently do for some primops is just use a | totally bogus kind. For example, mkWeak# will happily accept an Int# as | its first argument. Well, I see primop MkWeakOp "mkWeak#" GenPrimOp o -> b -> (State# RealWorld -> (# State# RealWorld, c #)) and I believe (from Ben's message) that the "o" means "open type variable", which is the old terminology for what we now call levity-polymorphic. The type from primops.txt.pp is processed into various Haskell source files including compiler/stage1/build/primop-primop-info.hs-incl which includes primOpInfo MkWeakOp = mkGenPrimOp (fsLit "mkWeak#") [runtimeRep1TyVar, openAlphaTyVar, betaTyVar, gammaTyVar] [openAlphaTy, betaTy, (mkFunTy (mkStatePrimTy realWorldTy) ((mkTupleTy Unboxed [ mkStatePrimTy realWorldTy, gammaTy]))) , mkStatePrimTy realWorldTy] ((mkTupleTy Unboxed [mkStatePrimTy realWorldTy, mkWeakPrimTy betaTy])) So it looks as if (rightly or wrongly) mkWeak# is deliberately levity-polymorphic. It would be good to write this stuff down. A good starting point is https://ghc.haskell.org/trac/ghc/wiki/Commentary/PrimOps Simon So we *could* follow that precedent and generalize | reallyUnsafePtrEquality#, makeStableName#, etc., to accept anything, | whether it makes sense or not. Or we can work out how to do what I was | trying to do and then duplicate those primitives as appropriate | (mkLiftedWeak#, mkUnliftedWeak#, makeLiftedStableName#, | makeUnliftedStableName#, etc.). From simonpj at microsoft.com Wed Aug 22 10:11:43 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 22 Aug 2018 10:11:43 +0000 Subject: Build entirely broken Message-ID: >From a clean build I'm getting this ld: cannot find libraries/ghc-prim/dist-install/build/GHC/CString.o: No such file or directory ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Classes.o: No such file or directory ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Debug.o: No such file or directory ld: cannot find libraries/ghc-prim/dist-install/build/GHC/IntWord64.o: No such file or directory ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Magic.o: No such file or directory ld: cannot find libraries/ghc-prim/dist-install/build/GHC/PrimopWrappers.o: No such file or directory ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Tuple.o: No such file or directory ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Types.o: No such file or directory And indeed those files don't exist. In earlier builds they were there. The command line for eg CString was "inplace/bin/ghc-stage1" -hisuf hi -osuf o -hcsuf hc -static -O0 -H64m -Wall -fllvm-fill-undef-with-garbage -Werror -this-unit-id ghc-prim-0.5.3 -hide-all-packages -i -ilibraries/ghc-prim/. -ilibraries/ghc-prim/dist-install/build -Ilibraries/ghc-prim/dist-install/build -ilibraries/ghc-prim/dist-install/build/./autogen -Ilibraries/ghc-prim/dist-install/build/./autogen -Ilibraries/ghc-prim/. -optP-include -optPlibraries/ghc-prim/dist-install/build/./autogen/cabal_macros.h -package-id rts -this-unit-id ghc-prim -XHaskell2010 -O -dcore-lint -dno-debug-output -ticky -no-user-package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags -Wnoncanonical-monad-instances -odir libraries/ghc-prim/dist-install/build -hidir libraries/ghc-prim/dist-install/build -stubdir libraries/ghc-prim/dist-install/build -split-objs -dynamic-too -c libraries/ghc-prim/./GHC/CString.hs -o libraries/ghc-prim/dist-install/build/GHC/CString.o -dyno libraries/ghc-prim/dist-install/build/GHC/CString.dyn_o which looks right. What's going on? This has me totally stalled. Simonb -------------- next part -------------- An HTML attachment was scrubbed... URL: From omeragacan at gmail.com Wed Aug 22 10:36:43 2018 From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=) Date: Wed, 22 Aug 2018 13:36:43 +0300 Subject: Build entirely broken In-Reply-To: References: Message-ID: I wonder if this could be a problem with your tree? I just did git pull git submodule update --init make distclean ./boot ./configure make and it worked. Note that I tried with "quick" build flavor. Ömer Simon Peyton Jones via ghc-devs , 22 Ağu 2018 Çar, 13:12 tarihinde şunu yazdı: > > From a clean build I’m getting this > > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/CString.o: No such file or directory > > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Classes.o: No such file or directory > > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Debug.o: No such file or directory > > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/IntWord64.o: No such file or directory > > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Magic.o: No such file or directory > > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/PrimopWrappers.o: No such file or directory > > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Tuple.o: No such file or directory > > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Types.o: No such file or directory > > > > And indeed those files don’t exist. In earlier builds they were there. > > The command line for eg CString was > > "inplace/bin/ghc-stage1" -hisuf hi -osuf o -hcsuf hc -static -O0 -H64m -Wall -fllvm-fill-undef-with-garbage -Werror -this-unit-id ghc-prim-0.5.3 -hide-all-packages -i -ilibraries/ghc-prim/. -ilibraries/ghc-prim/dist-install/build -Ilibraries/ghc-prim/dist-install/build -ilibraries/ghc-prim/dist-install/build/./autogen -Ilibraries/ghc-prim/dist-install/build/./autogen -Ilibraries/ghc-prim/. -optP-include -optPlibraries/ghc-prim/dist-install/build/./autogen/cabal_macros.h -package-id rts -this-unit-id ghc-prim -XHaskell2010 -O -dcore-lint -dno-debug-output -ticky -no-user-package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags -Wnoncanonical-monad-instances -odir libraries/ghc-prim/dist-install/build -hidir libraries/ghc-prim/dist-install/build -stubdir libraries/ghc-prim/dist-install/build -split-objs -dynamic-too -c libraries/ghc-prim/./GHC/CString.hs -o libraries/ghc-prim/dist-install/build/GHC/CString.o -dyno libraries/ghc-prim/dist-install/build/GHC/CString.dyn_o > > which looks right. > > What’s going on? > > This has me totally stalled. > > Simonb > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From simonpj at microsoft.com Wed Aug 22 10:41:39 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 22 Aug 2018 10:41:39 +0000 Subject: Build entirely broken In-Reply-To: References: Message-ID: Yes, submodules are up to date. I was doing 'sh validate --fast' which should clean the tree. I've just removed the entire build tree and will try from scratch. Simon | -----Original Message----- | From: Ömer Sinan Ağacan | Sent: 22 August 2018 11:37 | To: Simon Peyton Jones | Cc: ghc-devs | Subject: Re: Build entirely broken | | I wonder if this could be a problem with your tree? I just did | | git pull | git submodule update --init | make distclean | ./boot | ./configure | make | | and it worked. Note that I tried with "quick" build flavor. | | Ömer | | Simon Peyton Jones via ghc-devs , 22 Ağu 2018 Çar, | 13:12 tarihinde şunu yazdı: | > | > From a clean build I’m getting this | > | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/CString.o: | > No such file or directory | > | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Classes.o: | > No such file or directory | > | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Debug.o: No | > such file or directory | > | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/IntWord64.o: | > No such file or directory | > | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Magic.o: No | > such file or directory | > | > ld: cannot find | > libraries/ghc-prim/dist-install/build/GHC/PrimopWrappers.o: No such | > file or directory | > | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Tuple.o: No | > such file or directory | > | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Types.o: No | > such file or directory | > | > | > | > And indeed those files don’t exist. In earlier builds they were there. | > | > The command line for eg CString was | > | > "inplace/bin/ghc-stage1" -hisuf hi -osuf o -hcsuf hc -static -O0 - | H64m -Wall -fllvm-fill-undef-with-garbage -Werror -this-unit-id | ghc-prim-0.5.3 -hide-all-packages -i -ilibraries/ghc-prim/. - | ilibraries/ghc-prim/dist-install/build -Ilibraries/ghc-prim/dist- | install/build -ilibraries/ghc-prim/dist-install/build/./autogen - | Ilibraries/ghc-prim/dist-install/build/./autogen -Ilibraries/ghc-prim/. | -optP-include -optPlibraries/ghc-prim/dist- | install/build/./autogen/cabal_macros.h -package-id rts -this-unit-id ghc- | prim -XHaskell2010 -O -dcore-lint -dno-debug-output -ticky -no-user- | package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags - | Wnoncanonical-monad-instances -odir libraries/ghc-prim/dist- | install/build -hidir libraries/ghc-prim/dist-install/build -stubdir | libraries/ghc-prim/dist-install/build -split-objs -dynamic-too -c | libraries/ghc-prim/./GHC/CString.hs -o libraries/ghc-prim/dist- | install/build/GHC/CString.o -dyno libraries/ghc-prim/dist- | install/build/GHC/CString.dyn_o | > | > which looks right. | > | > What’s going on? | > | > This has me totally stalled. | > | > Simonb | > | > _______________________________________________ | > ghc-devs mailing list | > ghc-devs at haskell.org | > https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h | > askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=02%7C01% | > 7Csimonpj%40microsoft.com%7Cd431698f245d4a5027b808d6081b3d6f%7C72f988b | > f86f141af91ab2d7cd011db47%7C1%7C0%7C636705310415122065&sdata=B3njA | > GGeNNhNxfIkKrXOf7C2ujZzqgA5VN%2BynWL2wZ4%3D&reserved=0 From simonpj at microsoft.com Wed Aug 22 11:05:38 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 22 Aug 2018 11:05:38 +0000 Subject: Build entirely broken In-Reply-To: References: Message-ID: Panic over. Removing the entire build tree and starting again solved it. I have no idea what the original problem was. Sorry for false alarm Simon | -----Original Message----- | From: Ömer Sinan Ağacan | Sent: 22 August 2018 11:37 | To: Simon Peyton Jones | Cc: ghc-devs | Subject: Re: Build entirely broken | | I wonder if this could be a problem with your tree? I just did | | git pull | git submodule update --init | make distclean | ./boot | ./configure | make | | and it worked. Note that I tried with "quick" build flavor. | | Ömer | | Simon Peyton Jones via ghc-devs , 22 Ağu 2018 Çar, | 13:12 tarihinde şunu yazdı: | > | > From a clean build I’m getting this | > | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/CString.o: | > No such file or directory | > | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Classes.o: | > No such file or directory | > | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Debug.o: No | > such file or directory | > | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/IntWord64.o: | > No such file or directory | > | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Magic.o: No | > such file or directory | > | > ld: cannot find | > libraries/ghc-prim/dist-install/build/GHC/PrimopWrappers.o: No such | > file or directory | > | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Tuple.o: No | > such file or directory | > | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Types.o: No | > such file or directory | > | > | > | > And indeed those files don’t exist. In earlier builds they were there. | > | > The command line for eg CString was | > | > "inplace/bin/ghc-stage1" -hisuf hi -osuf o -hcsuf hc -static -O0 - | H64m -Wall -fllvm-fill-undef-with-garbage -Werror -this-unit-id | ghc-prim-0.5.3 -hide-all-packages -i -ilibraries/ghc-prim/. - | ilibraries/ghc-prim/dist-install/build -Ilibraries/ghc-prim/dist- | install/build -ilibraries/ghc-prim/dist-install/build/./autogen - | Ilibraries/ghc-prim/dist-install/build/./autogen -Ilibraries/ghc-prim/. | -optP-include -optPlibraries/ghc-prim/dist- | install/build/./autogen/cabal_macros.h -package-id rts -this-unit-id ghc- | prim -XHaskell2010 -O -dcore-lint -dno-debug-output -ticky -no-user- | package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags - | Wnoncanonical-monad-instances -odir libraries/ghc-prim/dist- | install/build -hidir libraries/ghc-prim/dist-install/build -stubdir | libraries/ghc-prim/dist-install/build -split-objs -dynamic-too -c | libraries/ghc-prim/./GHC/CString.hs -o libraries/ghc-prim/dist- | install/build/GHC/CString.o -dyno libraries/ghc-prim/dist- | install/build/GHC/CString.dyn_o | > | > which looks right. | > | > What’s going on? | > | > This has me totally stalled. | > | > Simonb | > | > _______________________________________________ | > ghc-devs mailing list | > ghc-devs at haskell.org | > https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h | > askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=02%7C01% | > 7Csimonpj%40microsoft.com%7Cd431698f245d4a5027b808d6081b3d6f%7C72f988b | > f86f141af91ab2d7cd011db47%7C1%7C0%7C636705310415122065&sdata=B3njA | > GGeNNhNxfIkKrXOf7C2ujZzqgA5VN%2BynWL2wZ4%3D&reserved=0 From simonpj at microsoft.com Wed Aug 22 11:16:56 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 22 Aug 2018 11:16:56 +0000 Subject: Build entirely broken In-Reply-To: References: Message-ID: Sigh. I spoke too soon. Exactly the same thing has happened starting from a clean build. I'm utterly stuck. Why is CString.o not being built? S | -----Original Message----- | From: ghc-devs On Behalf Of Simon Peyton | Jones via ghc-devs | Sent: 22 August 2018 12:06 | To: Ömer Sinan Ağacan | Cc: ghc-devs | Subject: RE: Build entirely broken | | Panic over. Removing the entire build tree and starting again solved it. | I have no idea what the original problem was. | | Sorry for false alarm | | Simon | | | -----Original Message----- | | From: Ömer Sinan Ağacan | | Sent: 22 August 2018 11:37 | | To: Simon Peyton Jones | | Cc: ghc-devs | | Subject: Re: Build entirely broken | | | | I wonder if this could be a problem with your tree? I just did | | | | git pull | | git submodule update --init | | make distclean | | ./boot | | ./configure | | make | | | | and it worked. Note that I tried with "quick" build flavor. | | | | Ömer | | | | Simon Peyton Jones via ghc-devs , 22 Ağu 2018 | | Çar, | | 13:12 tarihinde şunu yazdı: | | > | | > From a clean build I’m getting this > > ld: cannot find | | libraries/ghc-prim/dist-install/build/GHC/CString.o: | | > No such file or directory | | > | | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Classes.o: | | > No such file or directory | | > | | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Debug.o: | | No > such file or directory > > ld: cannot find | | libraries/ghc-prim/dist-install/build/GHC/IntWord64.o: | | > No such file or directory | | > | | > ld: cannot find libraries/ghc-prim/dist-install/build/GHC/Magic.o: | | No > such file or directory > > ld: cannot find > | | libraries/ghc-prim/dist-install/build/GHC/PrimopWrappers.o: No such > | | file or directory > > ld: cannot find | | libraries/ghc-prim/dist-install/build/GHC/Tuple.o: No > such file or | | directory > > ld: cannot find | | libraries/ghc-prim/dist-install/build/GHC/Types.o: No > such file or | | directory > > > > And indeed those files don’t exist. In earlier | | builds they were there. | | > | | > The command line for eg CString was > > "inplace/bin/ghc-stage1" | | -hisuf hi -osuf o -hcsuf hc -static -O0 - | | H64m -Wall -fllvm-fill-undef-with-garbage -Werror -this-unit-id | | ghc-prim-0.5.3 -hide-all-packages -i -ilibraries/ghc-prim/. - | | ilibraries/ghc-prim/dist-install/build -Ilibraries/ghc-prim/dist- | | install/build -ilibraries/ghc-prim/dist-install/build/./autogen - | | Ilibraries/ghc-prim/dist-install/build/./autogen -Ilibraries/ghc-prim/. | | -optP-include -optPlibraries/ghc-prim/dist- | | install/build/./autogen/cabal_macros.h -package-id rts -this-unit-id | | ghc- prim -XHaskell2010 -O -dcore-lint -dno-debug-output -ticky -no- | user- | | package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags - | | Wnoncanonical-monad-instances -odir libraries/ghc-prim/dist- | | install/build -hidir libraries/ghc-prim/dist-install/build -stubdir | | libraries/ghc-prim/dist-install/build -split-objs -dynamic-too -c | | libraries/ghc-prim/./GHC/CString.hs -o libraries/ghc-prim/dist- | | install/build/GHC/CString.o -dyno libraries/ghc-prim/dist- | | install/build/GHC/CString.dyn_o > > which looks right. | | > | | > What’s going on? | | > | | > This has me totally stalled. | | > | | > Simonb | | > | | > _______________________________________________ | | > ghc-devs mailing list | | > ghc-devs at haskell.org | | > | | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h | | > | | askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=02%7C01% | | > | | 7Csimonpj%40microsoft.com%7Cd431698f245d4a5027b808d6081b3d6f%7C72f988b | | > | | f86f141af91ab2d7cd011db47%7C1%7C0%7C636705310415122065&sdata=B3njA | | > GGeNNhNxfIkKrXOf7C2ujZzqgA5VN%2BynWL2wZ4%3D&reserved=0 | _______________________________________________ | ghc-devs mailing list | ghc-devs at haskell.org | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask | ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- | devs&data=02%7C01%7Csimonpj%40microsoft.com%7C776e01d283e64a772dbf08d | 6081f3cd0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636705327589276520 | &sdata=Q0OxveXyo8vJZwVIuShj%2Fhur1YOjTfa4EYViHLEUPSA%3D&reserved= | 0 From simonpj at microsoft.com Wed Aug 22 11:24:16 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 22 Aug 2018 11:24:16 +0000 Subject: Build entirely broken In-Reply-To: References: Message-ID: If I add -v3 to the compile line for CString I see .. *** Linker: gcc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -Wl,--no-as-needed -nostdlib -Wl,-r -no-pie -nodefaultlibs -Wl,-no-relax '-Wl,--build-id=none' -o libraries/ghc-prim/dist-install/build/GHC/CString.o /tmp/ghc17344_0/ghc_4.ldscript and no CString.o is created. But previously we had gcc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-fuse-ld=gold' -Wl,--no-as-needed -nostdlib -Wl,-r -no-pie -nodefaultlibs '-Wl,--build-id=none' -o libraries/ghc-prim/dist-install/build/GHC/CString.o /tmp/ghc57765_0/ghc_4.ldscript So maybe it’s that ‘no-relax’ business. S | -----Original Message----- | From: Simon Peyton Jones | Sent: 22 August 2018 12:17 | To: Simon Peyton Jones ; ghc-devs | Subject: RE: Build entirely broken | | Sigh. I spoke too soon. | | Exactly the same thing has happened starting from a clean build. | | I'm utterly stuck. Why is CString.o not being built? | | S | | | -----Original Message----- | | From: ghc-devs > On Behalf Of Simon | | Peyton Jones via ghc-devs | | Sent: 22 August 2018 12:06 | | To: Ömer Sinan Ağacan > | | Cc: ghc-devs > | | Subject: RE: Build entirely broken | | | | Panic over. Removing the entire build tree and starting again solved | it. | | I have no idea what the original problem was. | | | | Sorry for false alarm | | | | Simon | | | | | -----Original Message----- | | | From: Ömer Sinan Ağacan > | Sent: 22 August | | 2018 11:37 | To: Simon Peyton Jones > | Cc: | | ghc-devs > | Subject: Re: Build entirely broken | | | | I wonder if this could be a problem with your tree? I just did | | | | | | git pull | | | git submodule update --init | | | make distclean | | | ./boot | | | ./configure | | | make | | | | | | and it worked. Note that I tried with "quick" build flavor. | | | | | | Ömer | | | | | | Simon Peyton Jones via ghc-devs >, 22 Ağu | | 2018 | Çar, | 13:12 tarihinde şunu yazdı: | | | > | | | > From a clean build I’m getting this > > ld: cannot find | | | libraries/ghc-prim/dist-install/build/GHC/CString.o: | | | > No such file or directory | | | > | | | > ld: cannot find libraries/ghc-prim/dist- | install/build/GHC/Classes.o: | | | > No such file or directory | | | > | | | > ld: cannot find libraries/ghc-prim/dist- | install/build/GHC/Debug.o: | | | No > such file or directory > > ld: cannot find | | | libraries/ghc-prim/dist-install/build/GHC/IntWord64.o: | | | > No such file or directory | | | > | | | > ld: cannot find libraries/ghc-prim/dist- | install/build/GHC/Magic.o: | | | No > such file or directory > > ld: cannot find > | | | libraries/ghc-prim/dist-install/build/GHC/PrimopWrappers.o: No such > | | | file or directory > > ld: cannot find | | | libraries/ghc-prim/dist-install/build/GHC/Tuple.o: No > such file or | | | directory > > ld: cannot find | | | libraries/ghc-prim/dist-install/build/GHC/Types.o: No > such file or | | | directory > > > > And indeed those files don’t exist. In | | earlier | builds they were there. | | | > | | | > The command line for eg CString was > > "inplace/bin/ghc- | stage1" | | | -hisuf hi -osuf o -hcsuf hc -static -O0 - | | | H64m -Wall -fllvm-fill-undef-with-garbage -Werror -this-unit- | id | | | ghc-prim-0.5.3 -hide-all-packages -i -ilibraries/ghc-prim/. - | | | ilibraries/ghc-prim/dist-install/build -Ilibraries/ghc-prim/dist- | | | install/build -ilibraries/ghc-prim/dist-install/build/./autogen - | | | Ilibraries/ghc-prim/dist-install/build/./autogen -Ilibraries/ghc-prim/. | | | -optP-include -optPlibraries/ghc-prim/dist- | | | install/build/./autogen/cabal_macros.h -package-id rts -this-unit-id | | | ghc- prim -XHaskell2010 -O -dcore-lint -dno-debug-output -ticky | | -no- | | user- | | | package-db -rtsopts -Wno-trustworthy-safe -Wno-deprecated-flags | - | | | Wnoncanonical-monad-instances -odir libraries/ghc-prim/dist- | | | install/build -hidir libraries/ghc-prim/dist-install/build -stubdir | | | libraries/ghc-prim/dist-install/build -split-objs -dynamic-too -c | | | libraries/ghc-prim/./GHC/CString.hs -o libraries/ghc-prim/dist- | | | install/build/GHC/CString.o -dyno libraries/ghc-prim/dist- | | | install/build/GHC/CString.dyn_o > > which looks right. | | | > | | | > What’s going on? | | | > | | | > This has me totally stalled. | | | > | | | > Simonb | | | > | | | > _______________________________________________ | | | > ghc-devs mailing list | | | > ghc-devs at haskell.org | | | > | | | | | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h | | | > | | | | | askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=02%7C01% | | | > | | | | | 7Csimonpj%40microsoft.com%7Cd431698f245d4a5027b808d6081b3d6f%7C72f988b | | | > | | | | | f86f141af91ab2d7cd011db47%7C1%7C0%7C636705310415122065&sdata=B3njA | | | > GGeNNhNxfIkKrXOf7C2ujZzqgA5VN%2BynWL2wZ4%3D&reserved=0 | | _______________________________________________ | | ghc-devs mailing list | | ghc-devs at haskell.org | | | | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h | | ask | | ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- | | | | devs&data=02%7C01%7Csimonpj%40microsoft.com%7C776e01d283e64a772dbf | | 08d | | | | 6081f3cd0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636705327589276 | | 520 | | &sdata=Q0OxveXyo8vJZwVIuShj%2Fhur1YOjTfa4EYViHLEUPSA%3D&reserv | | ed= | | 0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at smart-cactus.org Wed Aug 22 11:43:21 2018 From: ben at smart-cactus.org (Ben Gamari) Date: Wed, 22 Aug 2018 07:43:21 -0400 Subject: Build entirely broken In-Reply-To: References: Message-ID: <87k1oilj8o.fsf@smart-cactus.org> Simon Peyton Jones via ghc-devs writes: > Sigh. I spoke too soon. > > Exactly the same thing has happened starting from a clean build. > > I'm utterly stuck. Why is CString.o not being built? > Can you try reverting 1cc9061fce42? Perhaps this is a linker compatibility issue? Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at smart-cactus.org Wed Aug 22 11:45:26 2018 From: ben at smart-cactus.org (Ben Gamari) Date: Wed, 22 Aug 2018 07:45:26 -0400 Subject: Unlifted primop types In-Reply-To: References: <87tvnnlboc.fsf@smart-cactus.org> <1588913.T5q9MkzbpF@squirrel> Message-ID: <87h8jmlj58.fsf@smart-cactus.org> Simon Peyton Jones writes: > | Huh! It looks like what we currently do for some primops is just use a > | totally bogus kind. For example, mkWeak# will happily accept an Int# as > | its first argument. > > Well, I see > primop MkWeakOp "mkWeak#" GenPrimOp > o -> b -> (State# RealWorld -> (# State# RealWorld, c #)) > > and I believe (from Ben's message) that the "o" means "open type variable", > which is the old terminology for what we now call levity-polymorphic. > Right; currently (largely for historical reasons) we use `o` to accommodate cases that accept both lifted and unlifted pointers. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From david.feuer at gmail.com Wed Aug 22 13:59:06 2018 From: david.feuer at gmail.com (David Feuer) Date: Wed, 22 Aug 2018 09:59:06 -0400 Subject: Unlifted primop types In-Reply-To: <87h8jmlj58.fsf@smart-cactus.org> References: <87tvnnlboc.fsf@smart-cactus.org> <1588913.T5q9MkzbpF@squirrel> <87h8jmlj58.fsf@smart-cactus.org> Message-ID: The problem is that this also accepts things that aren't pointers at all! We could fix that *for primops* by changing RuntimeRep to something like data RuntimeRep = PtrRep Liftedness | ... But that would only work for primops (at least for now) so it may not be worth the breakage. On Wed, Aug 22, 2018, 7:45 AM Ben Gamari wrote: > Simon Peyton Jones writes: > > > | Huh! It looks like what we currently do for some primops is just use a > > | totally bogus kind. For example, mkWeak# will happily accept an Int# > as > > | its first argument. > > > > Well, I see > > primop MkWeakOp "mkWeak#" GenPrimOp > > o -> b -> (State# RealWorld -> (# State# RealWorld, c #)) > > > > and I believe (from Ben's message) that the "o" means "open type > variable", > > which is the old terminology for what we now call levity-polymorphic. > > > Right; currently (largely for historical reasons) we use `o` to > accommodate cases that accept both lifted and unlifted pointers. > > Cheers, > > - Ben > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Wed Aug 22 15:27:28 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 22 Aug 2018 15:27:28 +0000 Subject: Build entirely broken In-Reply-To: <87k1oilj8o.fsf@smart-cactus.org> References: <87k1oilj8o.fsf@smart-cactus.org> Message-ID: Yes I tried that, following my previous email. And indeed that was it. Gah! Time wasted. But at least it's solved now. Simon | -----Original Message----- | From: Ben Gamari | Sent: 22 August 2018 12:43 | To: Simon Peyton Jones via ghc-devs ; Simon Peyton | Jones ; ghc-devs | Subject: RE: Build entirely broken | | Simon Peyton Jones via ghc-devs writes: | | > Sigh. I spoke too soon. | > | > Exactly the same thing has happened starting from a clean build. | > | > I'm utterly stuck. Why is CString.o not being built? | > | Can you try reverting 1cc9061fce42? Perhaps this is a linker | compatibility issue? | | Cheers, | | - Ben From simonpj at microsoft.com Wed Aug 22 20:43:11 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 22 Aug 2018 20:43:11 +0000 Subject: Thank you Message-ID: Ryan I just wanted to thank you for the incredible job you are doing in finding and fixing GHC bugs. From deriving, to pattern-match overlap checking, to Template Haskell, to arcane corners of the type inference engine... you are a bug-fixing machine! Plus you often bisect to find the patch that caused a bug and/or distil a much smaller test case. Both are extremely helpful. Thank you. Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From svenpanne at gmail.com Thu Aug 23 09:07:04 2018 From: svenpanne at gmail.com (Sven Panne) Date: Thu, 23 Aug 2018 11:07:04 +0200 Subject: GHC 8.6.1 release status In-Reply-To: References: <87o9dwn2ot.fsf@smart-cactus.org> <87bm9vn4cn.fsf@smart-cactus.org> Message-ID: Am Di., 21. Aug. 2018 um 17:14 Uhr schrieb Simon Jakobi via ghc-devs < ghc-devs at haskell.org>: > I talked to Alex about these today: > > * PR #905 has been merged. > * PR #893 requires the merge of https://phabricator.haskell.org/D5003. > Apart from that there were no objections. > Just another Haddock-related question: Is the fix in https://github.com/haskell/haddock/issues/837 included in 8.6.1, too? Currently I'm a bit confused about exactly which Haddock release/snapshot will be shipped with 8.6.1... -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon.jakobi at googlemail.com Thu Aug 23 13:46:38 2018 From: simon.jakobi at googlemail.com (Simon Jakobi) Date: Thu, 23 Aug 2018 15:46:38 +0200 Subject: GHC 8.6.1 release status In-Reply-To: References: <87o9dwn2ot.fsf@smart-cactus.org> <87bm9vn4cn.fsf@smart-cactus.org> Message-ID: > > Is the fix in https://github.com/haskell/haddock/issues/837 included in > 8.6.1, too? The fix is both on ghc's ghc-8.6 branch and on haddock's ghc-8.6 branch. Currently I'm a bit confused about exactly which Haddock release/snapshot > will be shipped with 8.6.1... I'm not sure either. CC'ing Herbert. Am Do., 23. Aug. 2018 um 11:07 Uhr schrieb Sven Panne : > Am Di., 21. Aug. 2018 um 17:14 Uhr schrieb Simon Jakobi via ghc-devs < > ghc-devs at haskell.org>: > >> I talked to Alex about these today: >> >> * PR #905 has been merged. >> * PR #893 requires the merge of https://phabricator.haskell.org/D5003. >> Apart from that there were no objections. >> > > Just another Haddock-related question: Is the fix in > https://github.com/haskell/haddock/issues/837 included in 8.6.1, too? > Currently I'm a bit confused about exactly which Haddock release/snapshot > will be shipped with 8.6.1... > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alec.theriault at gmail.com Thu Aug 23 14:29:58 2018 From: alec.theriault at gmail.com (Alec Theriault) Date: Thu, 23 Aug 2018 07:29:58 -0700 Subject: GHC 8.6.1 release status In-Reply-To: References: <87o9dwn2ot.fsf@smart-cactus.org> <87bm9vn4cn.fsf@smart-cactus.org> Message-ID: As far as I can tell, these changes aren’t yet in GHC 8.6: * https://github.com/haskell/haddock/pull/905 is on Haddock’s 8.6 branch, but GHC 8.6 hasn’t yet bumped the submodule. * https://github.com/haskell/haddock/pull/893 is not even yet on Haddock’s 8.6 branch. Note that this one requires matching GHC-side changes to be cherry-picked too. I think the idea is loosely for GHC 8.6 to ship whatever is in the Haddock `ghc-8.6` branch. > On Aug 23, 2018, at 6:46 AM, Simon Jakobi via ghc-devs wrote: > > Is the fix in https://github.com/haskell/haddock/issues/837 included in 8.6.1, too? > > The fix is both on ghc's ghc-8.6 branch and on haddock's ghc-8.6 branch. > > Currently I'm a bit confused about exactly which Haddock release/snapshot will be shipped with 8.6.1... > > I'm not sure either. CC'ing Herbert. > > Am Do., 23. Aug. 2018 um 11:07 Uhr schrieb Sven Panne >: > Am Di., 21. Aug. 2018 um 17:14 Uhr schrieb Simon Jakobi via ghc-devs >: > I talked to Alex about these today: > > * PR #905 has been merged. > * PR #893 requires the merge of https://phabricator.haskell.org/D5003 . Apart from that there were no objections. > > Just another Haddock-related question: Is the fix in https://github.com/haskell/haddock/issues/837 included in 8.6.1, too? Currently I'm a bit confused about exactly which Haddock release/snapshot will be shipped with 8.6.1... > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Thu Aug 23 21:43:11 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Thu, 23 Aug 2018 21:43:11 +0000 Subject: Unlifted primop types In-Reply-To: References: <87tvnnlboc.fsf@smart-cactus.org> <1588913.T5q9MkzbpF@squirrel> <87h8jmlj58.fsf@smart-cactus.org> Message-ID: Gah. We have no way to be polymorphic over all pointers (both lifted and unlifted) but not over Int# etc. As you say, this is too much of a special case to make an invasive change. I’m quite dubious about making weak poitners to unlifted heap-allocated objects. I can’t say it’s wrong but it feels dodgy to me. Simon From: David Feuer Sent: 22 August 2018 14:59 To: Ben Gamari Cc: Simon Peyton Jones ; David Feuer ; ghc-devs Subject: Re: Unlifted primop types The problem is that this also accepts things that aren't pointers at all! We could fix that *for primops* by changing RuntimeRep to something like data RuntimeRep = PtrRep Liftedness | ... But that would only work for primops (at least for now) so it may not be worth the breakage. On Wed, Aug 22, 2018, 7:45 AM Ben Gamari > wrote: Simon Peyton Jones > writes: > | Huh! It looks like what we currently do for some primops is just use a > | totally bogus kind. For example, mkWeak# will happily accept an Int# as > | its first argument. > > Well, I see > primop MkWeakOp "mkWeak#" GenPrimOp > o -> b -> (State# RealWorld -> (# State# RealWorld, c #)) > > and I believe (from Ben's message) that the "o" means "open type variable", > which is the old terminology for what we now call levity-polymorphic. > Right; currently (largely for historical reasons) we use `o` to accommodate cases that accept both lifted and unlifted pointers. Cheers, - Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.feuer at gmail.com Thu Aug 23 21:52:01 2018 From: david.feuer at gmail.com (David Feuer) Date: Thu, 23 Aug 2018 17:52:01 -0400 Subject: Unlifted primop types In-Reply-To: References: <87tvnnlboc.fsf@smart-cactus.org> <1588913.T5q9MkzbpF@squirrel> <87h8jmlj58.fsf@smart-cactus.org> Message-ID: Weak pointers to unlifted heap objects are the opposite of dodgy! We have base library functions (e.g., Data.IORef.mkWeakIORef) for creating weak references to lifted objects that *actually* create references to the underlying unlifted objects instead. This is much more reliable, because the wrapper (e.g., the STRef constructor) could be stripped away by worker-wrapper, while the underlying object (e.g., the MutVar#) won't be. On Thu, Aug 23, 2018, 5:43 PM Simon Peyton Jones wrote: > Gah. We have no way to be polymorphic over all pointers (both lifted and > unlifted) but not over Int# etc. > > > > As you say, this is too much of a special case to make an invasive change. > > > > I’m quite dubious about making weak poitners to unlifted heap-allocated > objects. I can’t say it’s wrong but it feels dodgy to me. > > > > Simon > > > > *From:* David Feuer > *Sent:* 22 August 2018 14:59 > *To:* Ben Gamari > *Cc:* Simon Peyton Jones ; David Feuer < > david at well-typed.com>; ghc-devs > *Subject:* Re: Unlifted primop types > > > > The problem is that this also accepts things that aren't pointers at all! > We could fix that *for primops* by changing RuntimeRep to something like > > > > data RuntimeRep > > = PtrRep Liftedness > > | ... > > > > But that would only work for primops (at least for now) so it may not be > worth the breakage. > > > > On Wed, Aug 22, 2018, 7:45 AM Ben Gamari wrote: > > Simon Peyton Jones writes: > > > | Huh! It looks like what we currently do for some primops is just use a > > | totally bogus kind. For example, mkWeak# will happily accept an Int# > as > > | its first argument. > > > > Well, I see > > primop MkWeakOp "mkWeak#" GenPrimOp > > o -> b -> (State# RealWorld -> (# State# RealWorld, c #)) > > > > and I believe (from Ben's message) that the "o" means "open type > variable", > > which is the old terminology for what we now call levity-polymorphic. > > > Right; currently (largely for historical reasons) we use `o` to > accommodate cases that accept both lifted and unlifted pointers. > > Cheers, > > - Ben > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.feuer at gmail.com Thu Aug 23 22:02:12 2018 From: david.feuer at gmail.com (David Feuer) Date: Thu, 23 Aug 2018 18:02:12 -0400 Subject: Unlifted primop types In-Reply-To: References: <87tvnnlboc.fsf@smart-cactus.org> <1588913.T5q9MkzbpF@squirrel> <87h8jmlj58.fsf@smart-cactus.org> Message-ID: Don't pay *too* much attention to mkWeakIORef in particular; for some reason it only makes a weak reference from an IORef to itself. It *should* have been written mkWeakIORef :: IORef a -> b -> IO () -> IO (Weak (IORef a)) mkWeakIORef r@(IORef (STRef r#)) b (IO finalizer) = IO $ \s -> case mkWeak# r# b finalizer s of (# s1, w #) -> (# s1, Weak w #) and the current version should've been mkSimpleWeakIORef r fin = mkWeakIORef r r fin On Thu, Aug 23, 2018, 5:52 PM David Feuer wrote: > Weak pointers to unlifted heap objects are the opposite of dodgy! We have > base library functions (e.g., Data.IORef.mkWeakIORef) for creating weak > references to lifted objects that *actually* create references to the > underlying unlifted objects instead. This is much more reliable, because > the wrapper (e.g., the STRef constructor) could be stripped away by > worker-wrapper, while the underlying object (e.g., the MutVar#) won't be. > > On Thu, Aug 23, 2018, 5:43 PM Simon Peyton Jones > wrote: > >> Gah. We have no way to be polymorphic over all pointers (both lifted and >> unlifted) but not over Int# etc. >> >> >> >> As you say, this is too much of a special case to make an invasive change. >> >> >> >> I’m quite dubious about making weak poitners to unlifted heap-allocated >> objects. I can’t say it’s wrong but it feels dodgy to me. >> >> >> >> Simon >> >> >> >> *From:* David Feuer >> *Sent:* 22 August 2018 14:59 >> *To:* Ben Gamari >> *Cc:* Simon Peyton Jones ; David Feuer < >> david at well-typed.com>; ghc-devs >> *Subject:* Re: Unlifted primop types >> >> >> >> The problem is that this also accepts things that aren't pointers at all! >> We could fix that *for primops* by changing RuntimeRep to something like >> >> >> >> data RuntimeRep >> >> = PtrRep Liftedness >> >> | ... >> >> >> >> But that would only work for primops (at least for now) so it may not be >> worth the breakage. >> >> >> >> On Wed, Aug 22, 2018, 7:45 AM Ben Gamari wrote: >> >> Simon Peyton Jones writes: >> >> > | Huh! It looks like what we currently do for some primops is just use >> a >> > | totally bogus kind. For example, mkWeak# will happily accept an Int# >> as >> > | its first argument. >> > >> > Well, I see >> > primop MkWeakOp "mkWeak#" GenPrimOp >> > o -> b -> (State# RealWorld -> (# State# RealWorld, c #)) >> > >> > and I believe (from Ben's message) that the "o" means "open type >> variable", >> > which is the old terminology for what we now call levity-polymorphic. >> > >> Right; currently (largely for historical reasons) we use `o` to >> accommodate cases that accept both lifted and unlifted pointers. >> >> Cheers, >> >> - Ben >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at smart-cactus.org Thu Aug 23 22:45:05 2018 From: ben at smart-cactus.org (Ben Gamari) Date: Thu, 23 Aug 2018 18:45:05 -0400 Subject: Unlifted primop types In-Reply-To: References: <87tvnnlboc.fsf@smart-cactus.org> <1588913.T5q9MkzbpF@squirrel> <87h8jmlj58.fsf@smart-cactus.org> Message-ID: <874lfkln2t.fsf@smart-cactus.org> David Feuer writes: > The problem is that this also accepts things that aren't pointers at all! > We could fix that *for primops* by changing RuntimeRep to something like > > data RuntimeRep > = PtrRep Liftedness > | ... > > But that would only work for primops (at least for now) so it may not be > worth the breakage. > The runtime rep design was precisely this but IIRC this was changed since the additional polymorphism ended up costing quite a bit in type-checking effort. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From david at well-typed.com Fri Aug 24 18:22:43 2018 From: david at well-typed.com (David Feuer) Date: Fri, 24 Aug 2018 14:22:43 -0400 Subject: Bump base version Message-ID: <2443875.f1NWrEJ5Fh@squirrel> We need to bump the base version to 4.12.1 to accommodate the StableName update. -- David Feuer Well-Typed From facundo.dominguez at tweag.io Fri Aug 24 20:05:07 2018 From: facundo.dominguez at tweag.io (=?UTF-8?Q?Facundo_Dom=C3=ADnguez?=) Date: Fri, 24 Aug 2018 17:05:07 -0300 Subject: GHC 8.6.1 release status In-Reply-To: References: <87o9dwn2ot.fsf@smart-cactus.org> <87bm9vn4cn.fsf@smart-cactus.org> Message-ID: Hello! I'm having some problem related to plugin recompilation: https://ghc.haskell.org/trac/ghc/ticket/15564 This prevents inline-java from building with ghc-6.8.1. Perhaps it is worth checking before the release. Best, Facundo On Thu, Aug 23, 2018 at 11:29 AM Alec Theriault wrote: > > As far as I can tell, these changes aren’t yet in GHC 8.6: > > * https://github.com/haskell/haddock/pull/905 is on Haddock’s 8.6 branch, but GHC 8.6 hasn’t yet bumped the submodule. > * https://github.com/haskell/haddock/pull/893 is not even yet on Haddock’s 8.6 branch. Note that this one requires matching GHC-side changes to be cherry-picked too. > > I think the idea is loosely for GHC 8.6 to ship whatever is in the Haddock `ghc-8.6` branch. > > On Aug 23, 2018, at 6:46 AM, Simon Jakobi via ghc-devs wrote: > >> Is the fix in https://github.com/haskell/haddock/issues/837 included in 8.6.1, too? > > > The fix is both on ghc's ghc-8.6 branch and on haddock's ghc-8.6 branch. > >> Currently I'm a bit confused about exactly which Haddock release/snapshot will be shipped with 8.6.1... > > > I'm not sure either. CC'ing Herbert. > > Am Do., 23. Aug. 2018 um 11:07 Uhr schrieb Sven Panne : >> >> Am Di., 21. Aug. 2018 um 17:14 Uhr schrieb Simon Jakobi via ghc-devs : >>> >>> I talked to Alex about these today: >>> >>> * PR #905 has been merged. >>> * PR #893 requires the merge of https://phabricator.haskell.org/D5003. Apart from that there were no objections. >> >> >> Just another Haddock-related question: Is the fix in https://github.com/haskell/haddock/issues/837 included in 8.6.1, too? Currently I'm a bit confused about exactly which Haddock release/snapshot will be shipped with 8.6.1... > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From matthewtpickering at gmail.com Fri Aug 24 20:06:35 2018 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Fri, 24 Aug 2018 21:06:35 +0100 Subject: GHC 8.6.1 release status In-Reply-To: References: <87o9dwn2ot.fsf@smart-cactus.org> <87bm9vn4cn.fsf@smart-cactus.org> Message-ID: I just replied on the ticket. I think the issue is fixed by D5048. Matt On Fri, Aug 24, 2018 at 9:05 PM Facundo Domínguez wrote: > > Hello! > > I'm having some problem related to plugin recompilation: > https://ghc.haskell.org/trac/ghc/ticket/15564 > > This prevents inline-java from building with ghc-6.8.1. Perhaps it is > worth checking before the release. > > Best, > Facundo > > > On Thu, Aug 23, 2018 at 11:29 AM Alec Theriault > wrote: > > > > As far as I can tell, these changes aren’t yet in GHC 8.6: > > > > * https://github.com/haskell/haddock/pull/905 is on Haddock’s 8.6 branch, but GHC 8.6 hasn’t yet bumped the submodule. > > * https://github.com/haskell/haddock/pull/893 is not even yet on Haddock’s 8.6 branch. Note that this one requires matching GHC-side changes to be cherry-picked too. > > > > I think the idea is loosely for GHC 8.6 to ship whatever is in the Haddock `ghc-8.6` branch. > > > > On Aug 23, 2018, at 6:46 AM, Simon Jakobi via ghc-devs wrote: > > > >> Is the fix in https://github.com/haskell/haddock/issues/837 included in 8.6.1, too? > > > > > > The fix is both on ghc's ghc-8.6 branch and on haddock's ghc-8.6 branch. > > > >> Currently I'm a bit confused about exactly which Haddock release/snapshot will be shipped with 8.6.1... > > > > > > I'm not sure either. CC'ing Herbert. > > > > Am Do., 23. Aug. 2018 um 11:07 Uhr schrieb Sven Panne : > >> > >> Am Di., 21. Aug. 2018 um 17:14 Uhr schrieb Simon Jakobi via ghc-devs : > >>> > >>> I talked to Alex about these today: > >>> > >>> * PR #905 has been merged. > >>> * PR #893 requires the merge of https://phabricator.haskell.org/D5003. Apart from that there were no objections. > >> > >> > >> Just another Haddock-related question: Is the fix in https://github.com/haskell/haddock/issues/837 included in 8.6.1, too? Currently I'm a bit confused about exactly which Haddock release/snapshot will be shipped with 8.6.1... > > > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > > > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From hexagoxel at hexagoxel.de Sat Aug 25 14:31:03 2018 From: hexagoxel at hexagoxel.de (lennart spitzner) Date: Sat, 25 Aug 2018 16:31:03 +0200 Subject: Regarding Package Environment Files In-Reply-To: References: <87o9dwn2ot.fsf@smart-cactus.org> <87bm9vn4cn.fsf@smart-cactus.org> Message-ID: greetings, this topic was discussed on mailing list before[1], but I don't think anything came of it. It was also recently discussed on the cabal issue tracker [2] but alas the cabal maintainers have so far not shown interest in resolving the issue. And while cabal-install caused this problem to surface for many users, the fact that package environment files unnecessarily add state to ghc behaviour holds regardless of cabal behaviour, and must be addressed on ghc's end. I expand on this reasoning in a blog post [3]. Thanks for considering. -- lennart [1] https://mail.haskell.org/pipermail/haskell-cafe/2018-May/129076.html [2] https://github.com/haskell/cabal/issues/4542 [3] https://hexagoxel.de/postsforpublish/posts/2018-08-25-ghc-pkg-env-misfeature.html From ryan.gl.scott at gmail.com Sun Aug 26 13:54:09 2018 From: ryan.gl.scott at gmail.com (Ryan Scott) Date: Sun, 26 Aug 2018 09:54:09 -0400 Subject: Bump base version Message-ID: I'm confused. base-4.12.0.0 hasn't been released yet, so can't we just make this StableName change a part of 4.12.0.0? (Regardless of what we choose, we should make sure to advertise this change in base's changelog—currently it's unmentioned.) Ryan S. -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at well-typed.com Mon Aug 27 03:04:02 2018 From: david at well-typed.com (David Feuer) Date: Sun, 26 Aug 2018 23:04:02 -0400 Subject: Bump base version In-Reply-To: References: Message-ID: <2357823.pgQhrJQmVY@squirrel> On Sunday, August 26, 2018 9:54:09 AM EDT Ryan Scott wrote: > I'm confused. base-4.12.0.0 hasn't been released yet, so can't we just make > this StableName change a part of 4.12.0.0? > > (Regardless of what we choose, we should make sure to advertise this change > in base's changelog—currently it's unmentioned.) > > Ryan S. I don't know what the policy is, to be honest! Is an alpha version a "real" version, whose base library has its own version number? Or is it a version that is expected to disappear as soon as the beta or full release comes out? -- David Feuer Well-Typed From ryan.gl.scott at gmail.com Mon Aug 27 12:55:00 2018 From: ryan.gl.scott at gmail.com (Ryan Scott) Date: Mon, 27 Aug 2018 08:55:00 -0400 Subject: Bump base version In-Reply-To: <2357823.pgQhrJQmVY@squirrel> References: <2357823.pgQhrJQmVY@squirrel> Message-ID: Everything that ships with a pre-release version of GHC is ultimately subject to change before its final release, and that includes the libraries that it ships with, too. In the case of the base library, the version that GHC 8.6.1-beta ships with (4.12.0.0) doesn't yet correspond to any version number released on Hackage, so one can think of the base-4.12.0.0 shipped with GHC 8.6.1-beta being itself a form of beta release. Importantly, that means we can change it if necessary before the final base-4.12.0.0 Hackage release (which usually coincides with the final GHC 8.6.1 release). Ryan S. On Sun, Aug 26, 2018 at 11:04 PM, David Feuer wrote: > On Sunday, August 26, 2018 9:54:09 AM EDT Ryan Scott wrote: > > I'm confused. base-4.12.0.0 hasn't been released yet, so can't we just > make > > this StableName change a part of 4.12.0.0? > > > > (Regardless of what we choose, we should make sure to advertise this > change > > in base's changelog—currently it's unmentioned.) > > > > Ryan S. > > I don't know what the policy is, to be honest! Is an alpha version a > "real" version, > whose base library has its own version number? Or is it a version that is > expected > to disappear as soon as the beta or full release comes out? > > -- > David Feuer > Well-Typed > -------------- next part -------------- An HTML attachment was scrubbed... URL: From olga.kenobi at gmail.com Mon Aug 27 14:44:54 2018 From: olga.kenobi at gmail.com (=?UTF-8?B?0J7Qu9GM0LPQsCDQpNC40LvQuNC/0L/RgdC60LDRjw==?=) Date: Mon, 27 Aug 2018 17:44:54 +0300 Subject: Haskell DLL (using FFI) causes memory leak. Message-ID: Hello. *Summary: *memory is consumed without releasing when a Haskell DLL (that uses FFI) is loaded and unloaded in an endless loop. *Details*: I'm working on a C++ project relying on a DLL that uses Haskell code. The DLL was built with GHC 8.0.1 x64, OS is Windows 7. (Newer versions of GHC - 8.2.1 and later - do not allow you to build such DLLs due to some bugs: ticket #1 , ticker #2 ). The C++ project was built with MSVC compiler 2015. We noticed that memory is not released when the library is unloaded. I've constructed a baseline example to reproduce the bug. It has three files: the first one is *HaskellSources.hs*(see attachments). It contains one foreign export function foo. This foreign export function is necessary to reproduce the bug. The second file is *CWrapper.cpp*. I intentionally didn't include any Haskell function export because they make no difference for this example. The last file is the code of the main program (see *main.cpp*) that loads the library and unloads it at once in an endless loop. There are two cases: whether HaskellExports.o is included into the library or not. 1. HaskellExports.o is not included into the library (see attached build script* build_no_hs.sh*). Everything works just fine (i.e. amount of memory consumed by the app is the same before DLL loading and right after unloading.) 2. HaskellExports.o is included into the library (see *build_w_hs.sh*). Memory is not released after unloading the DLL, after repeated load/unload cycles memory consumption keeps growing until finally the application crashes. Is this a known problem? Is it tracked in your bugtracker? (Or maybe it is not a problem at all and I'm doing it wrong). Thank you for your time. -- *Olga Philippskaya.* -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- #define DLLExport extern "C" __declspec(dllexport) DLLExport void* c_smth (const int num) { return 0; } -------------- next part -------------- A non-text attachment was scrubbed... Name: build_w_hs.sh Type: text/x-sh Size: 121 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: build_no_hs.sh Type: text/x-sh Size: 104 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: HaskellExports.hs Type: application/octet-stream Size: 223 bytes Desc: not available URL: -------------- next part -------------- #include int main() { for (;;) { HINSTANCE module = ::LoadLibrary(L"HaskellExports.dll"); ::FreeLibrary(module); } return 0; } From vanessa.mchale at iohk.io Mon Aug 27 15:00:13 2018 From: vanessa.mchale at iohk.io (Vanessa McHale) Date: Mon, 27 Aug 2018 10:00:13 -0500 Subject: Haskell DLL (using FFI) causes memory leak. In-Reply-To: References: Message-ID: <678aa9b5-5546-87f9-2acb-c472e5d5272c@iohk.io> I don't know about C++, but I do know that when calling Haskell from C code one must initialize the runtime and then exit it once finished. I also know that one must only initialize the runtime once over the course of the program's run. As far as I can tell, this does not occur in your example. On 08/27/2018 09:44 AM, Ольга Филиппская wrote: > Hello. > > /Summary: /memory is consumed without releasing when a Haskell DLL > (that uses FFI) is loaded and unloaded in an endless loop. > > /Details/: I'm working on a C++ project relying on a DLL that uses > Haskell code. The DLL was built with GHC 8.0.1 x64, OS is Windows 7. > (Newer versions of GHC - 8.2.1 and later - do not allow you to build > such DLLs due to some bugs: ticket #1 > , ticker #2 > ). The C++ project > was built with MSVC compiler 2015. > > We noticed that memory is not released when the library is unloaded. > I've constructed a baseline example to reproduce the bug. It has three > files: the first one is *HaskellSources.hs*(see attachments). It > contains one foreign export function foo. This foreign export function > is necessary to reproduce the bug. The second file is *CWrapper.cpp*. > I intentionally didn't include any Haskell function export because > they make no difference for this example. The last file is the code of > the main program (see *main.cpp*) that loads the library and unloads > it at once in an endless loop. > > There are two cases: whether HaskellExports.o is included into the > library or not. > 1. HaskellExports.o is not included into the library (see attached > build script/ build_no_hs.sh/). Everything works just fine (i.e. > amount of memory consumed by the app is the same before DLL loading > and right after unloading.) > 2. HaskellExports.o is included into the library > (see /build_w_hs.sh/). Memory is not released after unloading the DLL, > after repeated load/unload cycles memory consumption keeps growing > until finally the application crashes. > > Is this a known problem? Is it tracked in your bugtracker? (Or maybe > it is not a problem at all and I'm doing it wrong). > > Thank you for your time. > > -- > / > Olga Philippskaya > ./ > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: OpenPGP digital signature URL: From olga.kenobi at gmail.com Mon Aug 27 15:32:39 2018 From: olga.kenobi at gmail.com (=?UTF-8?B?0J7Qu9GM0LPQsCDQpNC40LvQuNC/0L/RgdC60LDRjw==?=) Date: Mon, 27 Aug 2018 18:32:39 +0300 Subject: Haskell DLL (using FFI) causes memory leak. In-Reply-To: <678aa9b5-5546-87f9-2acb-c472e5d5272c@iohk.io> References: <678aa9b5-5546-87f9-2acb-c472e5d5272c@iohk.io> Message-ID: It's true that one must initialize the runtime when calling Haskell from C/C++. We do this in the real project and just a pair of calls hs_init/hs_exit causes memory to be leaked even faster. But I tried to construct the minimal example to describe the bug, so I reduced all the extra-code. пн, 27 авг. 2018 г. в 18:00, Vanessa McHale : > I don't know about C++, but I do know that when calling Haskell from C > code one must initialize the runtime and then exit it once finished. I also > know that one must only initialize the runtime once over the course of the > program's run. As far as I can tell, this does not occur in your example. > > On 08/27/2018 09:44 AM, Ольга Филиппская wrote: > > Hello. > > *Summary: *memory is consumed without releasing when a Haskell DLL (that > uses FFI) is loaded and unloaded in an endless loop. > > *Details*: I'm working on a C++ project relying on a DLL that uses > Haskell code. The DLL was built with GHC 8.0.1 x64, OS is Windows 7. (Newer > versions of GHC - 8.2.1 and later - do not allow you to build such DLLs due > to some bugs: ticket #1 > , ticker #2 > ). The C++ project was > built with MSVC compiler 2015. > > We noticed that memory is not released when the library is unloaded. I've > constructed a baseline example to reproduce the bug. It has three files: > the first one is *HaskellSources.hs*(see attachments). It contains one > foreign export function foo. This foreign export function is necessary to > reproduce the bug. The second file is *CWrapper.cpp*. I intentionally > didn't include any Haskell function export because they make no difference > for this example. The last file is the code of the main program (see > *main.cpp*) that loads the library and unloads it at once in an endless > loop. > > There are two cases: whether HaskellExports.o is included into the library > or not. > 1. HaskellExports.o is not included into the library (see attached build > script* build_no_hs.sh*). Everything works just fine (i.e. amount of > memory consumed by the app is the same before DLL loading and right after > unloading.) > 2. HaskellExports.o is included into the library (see *build_w_hs.sh*). > Memory is not released after unloading the DLL, after repeated load/unload > cycles memory consumption keeps growing until finally the application > crashes. > > Is this a known problem? Is it tracked in your bugtracker? (Or maybe it is > not a problem at all and I'm doing it wrong). > > Thank you for your time. > > -- > * Olga Philippskaya .* > > > _______________________________________________ > ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -- *Филиппская Ольга.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From lonetiger at gmail.com Mon Aug 27 16:46:05 2018 From: lonetiger at gmail.com (Phyx) Date: Mon, 27 Aug 2018 17:46:05 +0100 Subject: Haskell DLL (using FFI) causes memory leak. In-Reply-To: References: <678aa9b5-5546-87f9-2acb-c472e5d5272c@iohk.io> Message-ID: Hi, You're likely just hitting a memory leak, Haskell DLLs and the RTS aren't designed to be unload-able, which is why you can't call hs_init again after you stop. The leak happens only when you do a foreign export because foreign exports create C wrappers to initializers for each function you export. https://github.com/ghc/ghc/blob/21f0f56164f50844c2150c62f950983b2376f8b6/compiler/deSugar/DsForeign.hs#L668 These are marked as static initializers, and as such the CRT will initialize them on DLL_PROCESS_ATTACH time. At DLL_PROCESS_DETACH time the destructors would be run, but we don't have destructors for these initializers, so whatever they did will always persist. Your use case is fairly uncommon, but file a bug report against the leaks anyway. Why do you need to keep loading and unloading the dll? Note that FreeLibrary like dlclose on unix does not guarantee that the library is freed, it just decrements the reference count. and when it reaches 0 it will *eventually* unload it. Notice that if you use UnmapViewOfFile instead of FreeLibrary the leak doesn't happen as that *actually* immediately unmaps the image. But this will get you in all sorts of trouble later on when terminating (you'll likely segfault at that point). Things will get worse when you actually do call hs_init, because hs_init will call initLinker which will load libraries of it's own. Unloading the top level does not decrease the reference counts of the dependencies, and we do no extra work to ensure this. If you really need to load/reload libraries, then I fear your only choice is some kind of process isolation and communicating back using share memory/pipes or some sort of IPC. Regards, Tamar On Mon, Aug 27, 2018 at 4:33 PM Ольга Филиппская wrote: > It's true that one must initialize the runtime when calling Haskell from > C/C++. We do this in the real project and just a pair of calls > hs_init/hs_exit causes memory to be leaked even faster. But I tried to > construct the minimal example to describe the bug, so I reduced all the > extra-code. > > пн, 27 авг. 2018 г. в 18:00, Vanessa McHale : > >> I don't know about C++, but I do know that when calling Haskell from C >> code one must initialize the runtime and then exit it once finished. I also >> know that one must only initialize the runtime once over the course of the >> program's run. As far as I can tell, this does not occur in your example. >> >> On 08/27/2018 09:44 AM, Ольга Филиппская wrote: >> >> Hello. >> >> *Summary: *memory is consumed without releasing when a Haskell DLL (that >> uses FFI) is loaded and unloaded in an endless loop. >> >> *Details*: I'm working on a C++ project relying on a DLL that uses >> Haskell code. The DLL was built with GHC 8.0.1 x64, OS is Windows 7. (Newer >> versions of GHC - 8.2.1 and later - do not allow you to build such DLLs due >> to some bugs: ticket #1 >> , ticker #2 >> ). The C++ project >> was built with MSVC compiler 2015. >> >> We noticed that memory is not released when the library is unloaded. I've >> constructed a baseline example to reproduce the bug. It has three files: >> the first one is *HaskellSources.hs*(see attachments). It contains one >> foreign export function foo. This foreign export function is necessary to >> reproduce the bug. The second file is *CWrapper.cpp*. I intentionally >> didn't include any Haskell function export because they make no difference >> for this example. The last file is the code of the main program (see >> *main.cpp*) that loads the library and unloads it at once in an endless >> loop. >> >> There are two cases: whether HaskellExports.o is included into the >> library or not. >> 1. HaskellExports.o is not included into the library (see attached build >> script* build_no_hs.sh*). Everything works just fine (i.e. amount of >> memory consumed by the app is the same before DLL loading and right after >> unloading.) >> 2. HaskellExports.o is included into the library (see *build_w_hs.sh*). >> Memory is not released after unloading the DLL, after repeated load/unload >> cycles memory consumption keeps growing until finally the application >> crashes. >> >> Is this a known problem? Is it tracked in your bugtracker? (Or maybe it >> is not a problem at all and I'm doing it wrong). >> >> Thank you for your time. >> >> -- >> * Olga Philippskaya .* >> >> >> _______________________________________________ >> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > > > -- > *Филиппская Ольга.* > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lonetiger at gmail.com Mon Aug 27 16:58:01 2018 From: lonetiger at gmail.com (Phyx) Date: Mon, 27 Aug 2018 17:58:01 +0100 Subject: Haskell DLL (using FFI) causes memory leak. In-Reply-To: References: <678aa9b5-5546-87f9-2acb-c472e5d5272c@iohk.io> Message-ID: Oh and also, neither of those tickets you linked to should prevent you from using a newer GHC to make shared libraries. the Rts.h headers are nothing but convenience functions and you can just create the prototypes you need in your own headers. On Mon, Aug 27, 2018 at 5:46 PM Phyx wrote: > Hi, > > You're likely just hitting a memory leak, Haskell DLLs and the RTS aren't > designed to be unload-able, which is why you can't call hs_init again after > you stop. > > The leak happens only when you do a foreign export because foreign exports > create C wrappers to initializers for each function you export. > > https://github.com/ghc/ghc/blob/21f0f56164f50844c2150c62f950983b2376f8b6/compiler/deSugar/DsForeign.hs#L668 > > These are marked as static initializers, and as such the CRT will > initialize them on DLL_PROCESS_ATTACH time. At DLL_PROCESS_DETACH > time the destructors would be run, but we don't have destructors for these > initializers, so whatever they did will always persist. > > Your use case is fairly uncommon, but file a bug report against the leaks > anyway. Why do you need to keep loading and unloading the dll? > > Note that FreeLibrary like dlclose on unix does not guarantee that the > library is freed, it just decrements the reference count. and when it > reaches 0 it will *eventually* unload it. Notice that if you use > UnmapViewOfFile instead of FreeLibrary the leak doesn't happen as that > *actually* immediately unmaps the image. But this will get you in all > sorts of trouble later on when terminating (you'll likely segfault at that > point). > > Things will get worse when you actually do call hs_init, because hs_init > will call initLinker which will load libraries of it's own. Unloading the > top level > does not decrease the reference counts of the dependencies, and we do no > extra work to ensure this. > > If you really need to load/reload libraries, then I fear your only choice > is some kind of process isolation and communicating back using share > memory/pipes > or some sort of IPC. > > Regards, > Tamar > > On Mon, Aug 27, 2018 at 4:33 PM Ольга Филиппская > wrote: > >> It's true that one must initialize the runtime when calling Haskell from >> C/C++. We do this in the real project and just a pair of calls >> hs_init/hs_exit causes memory to be leaked even faster. But I tried to >> construct the minimal example to describe the bug, so I reduced all the >> extra-code. >> >> пн, 27 авг. 2018 г. в 18:00, Vanessa McHale : >> >>> I don't know about C++, but I do know that when calling Haskell from C >>> code one must initialize the runtime and then exit it once finished. I also >>> know that one must only initialize the runtime once over the course of the >>> program's run. As far as I can tell, this does not occur in your example. >>> >>> On 08/27/2018 09:44 AM, Ольга Филиппская wrote: >>> >>> Hello. >>> >>> *Summary: *memory is consumed without releasing when a Haskell DLL >>> (that uses FFI) is loaded and unloaded in an endless loop. >>> >>> *Details*: I'm working on a C++ project relying on a DLL that uses >>> Haskell code. The DLL was built with GHC 8.0.1 x64, OS is Windows 7. (Newer >>> versions of GHC - 8.2.1 and later - do not allow you to build such DLLs due >>> to some bugs: ticket #1 >>> , ticker #2 >>> ). The C++ project >>> was built with MSVC compiler 2015. >>> >>> We noticed that memory is not released when the library is unloaded. >>> I've constructed a baseline example to reproduce the bug. It has three >>> files: the first one is *HaskellSources.hs*(see attachments). It >>> contains one foreign export function foo. This foreign export function is >>> necessary to reproduce the bug. The second file is *CWrapper.cpp*. I >>> intentionally didn't include any Haskell function export because they make >>> no difference for this example. The last file is the code of the main >>> program (see *main.cpp*) that loads the library and unloads it at once >>> in an endless loop. >>> >>> There are two cases: whether HaskellExports.o is included into the >>> library or not. >>> 1. HaskellExports.o is not included into the library (see attached build >>> script* build_no_hs.sh*). Everything works just fine (i.e. amount of >>> memory consumed by the app is the same before DLL loading and right after >>> unloading.) >>> 2. HaskellExports.o is included into the library (see *build_w_hs.sh*). >>> Memory is not released after unloading the DLL, after repeated load/unload >>> cycles memory consumption keeps growing until finally the application >>> crashes. >>> >>> Is this a known problem? Is it tracked in your bugtracker? (Or maybe it >>> is not a problem at all and I'm doing it wrong). >>> >>> Thank you for your time. >>> >>> -- >>> * Olga Philippskaya .* >>> >>> >>> _______________________________________________ >>> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >>> >>> _______________________________________________ >>> ghc-devs mailing list >>> ghc-devs at haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >>> >> >> >> -- >> *Филиппская Ольга.* >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at smart-cactus.org Mon Aug 27 18:42:05 2018 From: ben at smart-cactus.org (Ben Gamari) Date: Mon, 27 Aug 2018 14:42:05 -0400 Subject: Haskell DLL (using FFI) causes memory leak. In-Reply-To: References: <678aa9b5-5546-87f9-2acb-c472e5d5272c@iohk.io> Message-ID: <87r2ijk5xi.fsf@smart-cactus.org> Phyx writes: > Oh and also, neither of those tickets you linked to should prevent you from > using a newer GHC to make shared libraries. > > the Rts.h headers are nothing but convenience functions and you can just > create the prototypes you need in your own headers. > That being said, I think it would be much better if GHC's headers would accommodate C++ compilers, as Herbert suggested in #14784. I suspect we aren't using much C99 in the headers anyways. Perhaps we should just start by trying a conservative C++11 guard and tighten it later if we find earlier language versions would work. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From olga.kenobi at gmail.com Tue Aug 28 11:35:10 2018 From: olga.kenobi at gmail.com (=?UTF-8?B?0J7Qu9GM0LPQsCDQpNC40LvQuNC/0L/RgdC60LDRjw==?=) Date: Tue, 28 Aug 2018 14:35:10 +0300 Subject: Haskell DLL (using FFI) causes memory leak. In-Reply-To: References: <678aa9b5-5546-87f9-2acb-c472e5d5272c@iohk.io> Message-ID: пн, 27 авг. 2018 г. в 19:46, Phyx : > Hi, > > You're likely just hitting a memory leak, Haskell DLLs and the RTS aren't > designed to be unload-able, which is why you can't call hs_init again after > you stop. > > The leak happens only when you do a foreign export because foreign exports > create C wrappers to initializers for each function you export. > > https://github.com/ghc/ghc/blob/21f0f56164f50844c2150c62f950983b2376f8b6/compiler/deSugar/DsForeign.hs#L668 > > These are marked as static initializers, and as such the CRT will > initialize them on DLL_PROCESS_ATTACH time. At DLL_PROCESS_DETACH > time the destructors would be run, but we don't have destructors for these > initializers, so whatever they did will always persist. > Hello! Thank you very much for such a detailed explanation! Now I gain an insight into what's happening. > > Your use case is fairly uncommon, but file a bug report against the leaks > anyway. Why do you need to keep loading and unloading the dll? > Our goal was to understand whether we're cooking it in a wrong way or that's the implementation details of the Haskell RTS. And your answer just suits our goal perfectly. Anyway I will submit a bug report as you suggested. Kind regards, -- *Olga Philippskaya.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Thu Aug 30 11:12:19 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Thu, 30 Aug 2018 11:12:19 +0000 Subject: Hadrian Message-ID: Alp, Andrey The old build system printed out every command line; and I often copy-paste that info to build single modules. Eg currently, when trying to understand #15570 I see a suspicious GHC.Real.hi. So I want to manually recompile GHC.Real (from base), adding some debug flags. How can I get the right command line to do that from the build log? Where is the "how to use Hadrian" wiki page? I know you've been writing one. Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Thu Aug 30 11:19:29 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Thu, 30 Aug 2018 11:19:29 +0000 Subject: Hadrian In-Reply-To: References: Message-ID: Sigh. As an inconvenient workaround, I tried adding {-# OPTIONS_GHC -dverbose-core2core #-} to GHC.Real, and then doing cabal new-run hadrian -- -c -j4 --flavour=quick --directory=".." in hadrian/ That did recompile GHC.Real - but all the debug output disappeared! I tried adding {-# OPTIONS_GHC -ddebug-output #-} as well, but that didn't work. I'm stuck - any ideas? Simon From: ghc-devs On Behalf Of Simon Peyton Jones via ghc-devs Sent: 30 August 2018 12:12 To: Alp Mestanogullari ; Andrey Mokhov Cc: ghc-devs Subject: Hadrian Alp, Andrey The old build system printed out every command line; and I often copy-paste that info to build single modules. Eg currently, when trying to understand #15570 I see a suspicious GHC.Real.hi. So I want to manually recompile GHC.Real (from base), adding some debug flags. How can I get the right command line to do that from the build log? Where is the "how to use Hadrian" wiki page? I know you've been writing one. Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrey.mokhov at newcastle.ac.uk Thu Aug 30 11:25:34 2018 From: andrey.mokhov at newcastle.ac.uk (Andrey Mokhov) Date: Thu, 30 Aug 2018 11:25:34 +0000 Subject: Hadrian In-Reply-To: References: Message-ID: Simon, If you want to see all command lines, you can pass '--verbose' or '-V' flag to Hadrian and it will then print out everything it does. But you can also choose which particular command lines to print in UserSettings, see: https://github.com/snowleopard/hadrian/blob/master/doc/user-settings.md#verbose-command-lines So, you can do: verboseCommand = input "//GHC/Real.hs" Or, alternatively, verboseCommand = output "//GHC/Real.hi" Both should produce the same result (in theory). In general, we have the following documents on Hadrian: The README: https://github.com/snowleopard/hadrian/blob/master/README.md How to use UserSettings: https://github.com/snowleopard/hadrian/blob/master/doc/user-settings.md An overview of build flavours: https://github.com/snowleopard/hadrian/blob/master/doc/flavours.md I hope the more Hadrian gets used, the more complete the documentation will become. Cheers, Andrey From: Simon Peyton Jones [mailto:simonpj at microsoft.com] Sent: 30 August 2018 12:19 To: Simon Peyton Jones ; Alp Mestanogullari ; Andrey Mokhov Cc: ghc-devs Subject: RE: Hadrian Sigh.  As an inconvenient workaround, I tried adding {-# OPTIONS_GHC -dverbose-core2core #-} to GHC.Real, and then doing         cabal new-run hadrian -- -c -j4 --flavour=quick --directory=".." in hadrian/ That did recompile GHC.Real - but all the debug output disappeared! I tried adding {-# OPTIONS_GHC -ddebug-output #-} as well, but that didn't work. I'm stuck - any ideas? Simon From: ghc-devs On Behalf Of Simon Peyton Jones via ghc-devs Sent: 30 August 2018 12:12 To: Alp Mestanogullari ; Andrey Mokhov Cc: ghc-devs Subject: Hadrian Alp, Andrey The old build system printed out every command line; and I often copy-paste that info to build single modules. Eg currently, when trying to understand #15570 I see a suspicious GHC.Real.hi.  So I want to manually recompile GHC.Real (from base), adding some debug flags.  How can I get the right command line to do that from the build log? Where is the "how to use Hadrian" wiki page?  I know you've been writing one. Simon From a.pelenitsyn at gmail.com Thu Aug 30 11:43:07 2018 From: a.pelenitsyn at gmail.com (Artem Pelenitsyn) Date: Thu, 30 Aug 2018 07:43:07 -0400 Subject: Hadrian In-Reply-To: References: Message-ID: Hello Simon, Andrey, For “How to use Hadrian wikipage”, maybe Simon wanted this one from the GHC Wiki: https://ghc.haskell.org/trac/ghc/wiki/Building/Hadrian/QuickStart It does mention the verbose flag, but does not really explain that you can get what Simon wants, I believe. -- Best wishes, Artem чт, 30 авг. 2018 г. в 7:25, Andrey Mokhov : > Simon, > > If you want to see all command lines, you can pass '--verbose' or '-V' > flag to Hadrian and it will then print out everything it does. > > But you can also choose which particular command lines to print in > UserSettings, see: > > > https://github.com/snowleopard/hadrian/blob/master/doc/user-settings.md#verbose-command-lines > > So, you can do: > > verboseCommand = input "//GHC/Real.hs" > > Or, alternatively, > > verboseCommand = output "//GHC/Real.hi" > > Both should produce the same result (in theory). > > In general, we have the following documents on Hadrian: > > The README: https://github.com/snowleopard/hadrian/blob/master/README.md > How to use UserSettings: > https://github.com/snowleopard/hadrian/blob/master/doc/user-settings.md > An overview of build flavours: > https://github.com/snowleopard/hadrian/blob/master/doc/flavours.md > > I hope the more Hadrian gets used, the more complete the documentation > will become. > > Cheers, > Andrey > > From: Simon Peyton Jones [mailto:simonpj at microsoft.com] > Sent: 30 August 2018 12:19 > To: Simon Peyton Jones ; Alp Mestanogullari < > alp at well-typed.com>; Andrey Mokhov > Cc: ghc-devs > Subject: RE: Hadrian > > Sigh. As an inconvenient workaround, I tried adding {-# OPTIONS_GHC > -dverbose-core2core #-} to GHC.Real, and then doing > cabal new-run hadrian -- -c -j4 --flavour=quick --directory=".." > in hadrian/ > That did recompile GHC.Real - but all the debug output disappeared! > I tried adding {-# OPTIONS_GHC -ddebug-output #-} as well, but that didn't > work. > I'm stuck - any ideas? > Simon > > From: ghc-devs On Behalf Of Simon Peyton > Jones via ghc-devs > Sent: 30 August 2018 12:12 > To: Alp Mestanogullari ; Andrey Mokhov < > andrey.mokhov at newcastle.ac.uk> > Cc: ghc-devs > Subject: Hadrian > > Alp, Andrey > The old build system printed out every command line; and I often > copy-paste that info to build single modules. > Eg currently, when trying to understand #15570 I see a suspicious > GHC.Real.hi. So I want to manually recompile GHC.Real (from base), adding > some debug flags. How can I get the right command line to do that from the > build log? > Where is the "how to use Hadrian" wiki page? I know you've been writing > one. > Simon > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Thu Aug 30 12:00:01 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Thu, 30 Aug 2018 12:00:01 +0000 Subject: Hadrian In-Reply-To: References: Message-ID: | If you want to see all command lines, you can pass '--verbose' or '-V' | flag to Hadrian and it will then print out everything it does. But I am not invoking Hadrian. I am saying (on your instructions) cabal new-run hadrian -- -c -j4 --flavour=quick --directory=".." How do I "pass --verbose to Hadrian"? | -----Original Message----- | From: Andrey Mokhov | Sent: 30 August 2018 12:26 | To: Simon Peyton Jones ; Alp Mestanogullari | | Cc: ghc-devs | Subject: RE: Hadrian | | Simon, | | If you want to see all command lines, you can pass '--verbose' or '-V' | flag to Hadrian and it will then print out everything it does. | | But you can also choose which particular command lines to print in | UserSettings, see: | | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c | om%2Fsnowleopard%2Fhadrian%2Fblob%2Fmaster%2Fdoc%2Fuser- | settings.md%23verbose-command- | lines&data=02%7C01%7Csimonpj%40microsoft.com%7C0c6118a504f74269efd508 | d60e6b4fe7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63671225140342623 | 1&sdata=tMYhmIGMeVbBYjAtxWQTY5%2F4Kyfc4m981N4OTowGkbQ%3D&reserved | =0 | | So, you can do: | | verboseCommand = input "//GHC/Real.hs" | | Or, alternatively, | | verboseCommand = output "//GHC/Real.hi" | | Both should produce the same result (in theory). | | In general, we have the following documents on Hadrian: | | The README: | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c | om%2Fsnowleopard%2Fhadrian%2Fblob%2Fmaster%2FREADME.md&data=02%7C01%7 | Csimonpj%40microsoft.com%7C0c6118a504f74269efd508d60e6b4fe7%7C72f988bf86f | 141af91ab2d7cd011db47%7C1%7C0%7C636712251403426231&sdata=7WigmQG7lbDd | eyhYCSaKOQEb0eeZcn0iXAbAKdwZ5A4%3D&reserved=0 | How to use UserSettings: | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c | om%2Fsnowleopard%2Fhadrian%2Fblob%2Fmaster%2Fdoc%2Fuser- | settings.md&data=02%7C01%7Csimonpj%40microsoft.com%7C0c6118a504f74269 | efd508d60e6b4fe7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63671225140 | 3426231&sdata=tRu2lO%2F6TNV3ao5tzNKO%2FAHJMWsxKX%2BqExHn78nuCvU%3D&am | p;reserved=0 | An overview of build flavours: | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c | om%2Fsnowleopard%2Fhadrian%2Fblob%2Fmaster%2Fdoc%2Fflavours.md&data=0 | 2%7C01%7Csimonpj%40microsoft.com%7C0c6118a504f74269efd508d60e6b4fe7%7C72f | 988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636712251403426231&sdata=Esye | DdAKLV9DC3PUwkqsuQ4zGoOYKcaJeEov1BAH8lU%3D&reserved=0 | | I hope the more Hadrian gets used, the more complete the documentation | will become. | | Cheers, | Andrey | | From: Simon Peyton Jones [mailto:simonpj at microsoft.com] | Sent: 30 August 2018 12:19 | To: Simon Peyton Jones ; Alp Mestanogullari | ; Andrey Mokhov | Cc: ghc-devs | Subject: RE: Hadrian | | Sigh.  As an inconvenient workaround, I tried adding {-# OPTIONS_GHC - | dverbose-core2core #-} to GHC.Real, and then doing |         cabal new-run hadrian -- -c -j4 --flavour=quick --directory=".." | in hadrian/ | That did recompile GHC.Real - but all the debug output disappeared! | I tried adding {-# OPTIONS_GHC -ddebug-output #-} as well, but that | didn't work. | I'm stuck - any ideas? | Simon | | From: ghc-devs On Behalf Of Simon Peyton | Jones via ghc-devs | Sent: 30 August 2018 12:12 | To: Alp Mestanogullari ; Andrey Mokhov | | Cc: ghc-devs | Subject: Hadrian | | Alp, Andrey | The old build system printed out every command line; and I often copy- | paste that info to build single modules. | Eg currently, when trying to understand #15570 I see a suspicious | GHC.Real.hi.  So I want to manually recompile GHC.Real (from base), | adding some debug flags.  How can I get the right command line to do that | from the build log? | Where is the "how to use Hadrian" wiki page?  I know you've been writing | one. | Simon From andrey.mokhov at newcastle.ac.uk Thu Aug 30 12:08:07 2018 From: andrey.mokhov at newcastle.ac.uk (Andrey Mokhov) Date: Thu, 30 Aug 2018 12:08:07 +0000 Subject: Hadrian In-Reply-To: References: Message-ID: > How do I "pass --verbose to Hadrian"? Like this: cabal new-run hadrian -- -c -j4 --flavour=quick --directory=".." -V Everything after "--" in this command line are flags passed to Hadrian. Everything before are flags passed to Cabal. (I hope we'll fix the "build.sh" script for you soon, so you don't need to go via Cabal.) I just tried this with {-# OPTIONS_GHC -dverbose-core2core #-} and this did also print out a lot of Core. And a lot more stuff (somehow this seems to cause further recompilation). > https://ghc.haskell.org/trac/ghc/wiki/Building/Hadrian/QuickStart Artem -- thanks! I forgot to mention this wiki page, which indeed looks Cheers, Andrey -----Original Message----- From: Simon Peyton Jones [mailto:simonpj at microsoft.com] Sent: 30 August 2018 13:00 To: Andrey Mokhov ; Alp Mestanogullari Cc: ghc-devs Subject: RE: Hadrian | If you want to see all command lines, you can pass '--verbose' or '-V' | flag to Hadrian and it will then print out everything it does. But I am not invoking Hadrian. I am saying (on your instructions) cabal new-run hadrian -- -c -j4 --flavour=quick --directory=".." How do I "pass --verbose to Hadrian"? | -----Original Message----- | From: Andrey Mokhov | Sent: 30 August 2018 12:26 | To: Simon Peyton Jones ; Alp Mestanogullari | | Cc: ghc-devs | Subject: RE: Hadrian | | Simon, | | If you want to see all command lines, you can pass '--verbose' or '-V' | flag to Hadrian and it will then print out everything it does. | | But you can also choose which particular command lines to print in | UserSettings, see: | | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c | om%2Fsnowleopard%2Fhadrian%2Fblob%2Fmaster%2Fdoc%2Fuser- | settings.md%23verbose-command- | lines&data=02%7C01%7Csimonpj%40microsoft.com%7C0c6118a504f74269efd508 | d60e6b4fe7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63671225140342623 | 1&sdata=tMYhmIGMeVbBYjAtxWQTY5%2F4Kyfc4m981N4OTowGkbQ%3D&reserved | =0 | | So, you can do: | | verboseCommand = input "//GHC/Real.hs" | | Or, alternatively, | | verboseCommand = output "//GHC/Real.hi" | | Both should produce the same result (in theory). | | In general, we have the following documents on Hadrian: | | The README: | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c | om%2Fsnowleopard%2Fhadrian%2Fblob%2Fmaster%2FREADME.md&data=02%7C01%7 | Csimonpj%40microsoft.com%7C0c6118a504f74269efd508d60e6b4fe7%7C72f988bf86f | 141af91ab2d7cd011db47%7C1%7C0%7C636712251403426231&sdata=7WigmQG7lbDd | eyhYCSaKOQEb0eeZcn0iXAbAKdwZ5A4%3D&reserved=0 | How to use UserSettings: | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c | om%2Fsnowleopard%2Fhadrian%2Fblob%2Fmaster%2Fdoc%2Fuser- | settings.md&data=02%7C01%7Csimonpj%40microsoft.com%7C0c6118a504f74269 | efd508d60e6b4fe7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63671225140 | 3426231&sdata=tRu2lO%2F6TNV3ao5tzNKO%2FAHJMWsxKX%2BqExHn78nuCvU%3D&am | p;reserved=0 | An overview of build flavours: | https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c | om%2Fsnowleopard%2Fhadrian%2Fblob%2Fmaster%2Fdoc%2Fflavours.md&data=0 | 2%7C01%7Csimonpj%40microsoft.com%7C0c6118a504f74269efd508d60e6b4fe7%7C72f | 988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636712251403426231&sdata=Esye | DdAKLV9DC3PUwkqsuQ4zGoOYKcaJeEov1BAH8lU%3D&reserved=0 | | I hope the more Hadrian gets used, the more complete the documentation | will become. | | Cheers, | Andrey | | From: Simon Peyton Jones [mailto:simonpj at microsoft.com] | Sent: 30 August 2018 12:19 | To: Simon Peyton Jones ; Alp Mestanogullari | ; Andrey Mokhov | Cc: ghc-devs | Subject: RE: Hadrian | | Sigh.  As an inconvenient workaround, I tried adding {-# OPTIONS_GHC - | dverbose-core2core #-} to GHC.Real, and then doing |         cabal new-run hadrian -- -c -j4 --flavour=quick --directory=".." | in hadrian/ | That did recompile GHC.Real - but all the debug output disappeared! | I tried adding {-# OPTIONS_GHC -ddebug-output #-} as well, but that | didn't work. | I'm stuck - any ideas? | Simon | | From: ghc-devs On Behalf Of Simon Peyton | Jones via ghc-devs | Sent: 30 August 2018 12:12 | To: Alp Mestanogullari ; Andrey Mokhov | | Cc: ghc-devs | Subject: Hadrian | | Alp, Andrey | The old build system printed out every command line; and I often copy- | paste that info to build single modules. | Eg currently, when trying to understand #15570 I see a suspicious | GHC.Real.hi.  So I want to manually recompile GHC.Real (from base), | adding some debug flags.  How can I get the right command line to do that | from the build log? | Where is the "how to use Hadrian" wiki page?  I know you've been writing | one. | Simon From klebinger.andreas at gmx.at Thu Aug 30 17:08:25 2018 From: klebinger.andreas at gmx.at (Andreas Klebinger) Date: Thu, 30 Aug 2018 19:08:25 +0200 Subject: Compile times: Average vs worst case runtime Message-ID: <5B882489.3060305@gmx.at> Hello Devs, I developed a patch improving on GHC's code layout during GSoC: https://ghc.haskell.org/trac/ghc/ticket/15124 The gains are pretty nice with most library benchmarks showing improvments in the 1-2% range or more. Even compile times went down comparing head vs stage2 built with, and using my patch! Making compilation of nofib overall faster by 0.5-1% depending on the exact setup. Now the bad news: In some cases the algorithm has bad big O performance, in practice this seems to be code with things like cases with 200+ alternatives. Where these cases also represent most of the code compiled. The worst case I saw was doubled compile time in a Module which only consisted of a 500 deep if/else chain only selecting an value. While there are some small optimizations possible to improve on this I don't expect to make these edge cases much faster overall. However it will also be possible to fall back to the old behavior to avoid the large compile times for modules known to trigger this edge case. Which brings me to my main question: When should we use the new code layout. 1. Always since on average both compile and runtime is faster. 2. -O1/O2 since users often use this when performance matters. 3. -O2 since users expect compilation times to blow up with it? 4. Never as users experience compile time slowdowns when they hit these edge cases. I'm would prefer 2. 3. 1. 4. in that order but I wonder what the wider community is thinking. Cheers Andreas From simonpj at microsoft.com Thu Aug 30 21:07:12 2018 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Thu, 30 Aug 2018 21:07:12 +0000 Subject: Compile times: Average vs worst case runtime In-Reply-To: <5B882489.3060305@gmx.at> References: <5B882489.3060305@gmx.at> Message-ID: Good work. | However it will also be possible to fall back to the old behavior to | avoid the large compile times for modules known to trigger this | edge case. Not for specific modules, I hope; rather fall back when the number of cases becomes large, or something like that? That'd be fine. The only other potential worry is how tricky/understandable is your patch. Hopefully it's not hard. | 1. Always since on average both compile and runtime is faster. | 2. -O1/O2 since users often use this when performance matters. | 3. -O2 since users expect compilation times to blow up with it? | 4. Never as users experience compile time slowdowns when they hit these | edge cases. Well, if you can robustly fall back in edge cases, you'll never hit the blow-ups. So then (1) would be fine would it not? Simon | -----Original Message----- | From: ghc-devs On Behalf Of Andreas | Klebinger | Sent: 30 August 2018 18:08 | To: ghc-devs at haskell.org | Subject: Compile times: Average vs worst case runtime | | Hello Devs, | | I developed a patch improving on GHC's code layout during GSoC: | https://ghc.haskell.org/trac/ghc/ticket/15124 | The gains are pretty nice with most library benchmarks showing | improvments in the 1-2% range or more. | | Even compile times went down comparing head vs stage2 built with, and | using my patch! | Making compilation of nofib overall faster by 0.5-1% depending on the | exact setup. | | Now the bad news: | In some cases the algorithm has bad big O performance, in practice this | seems to be code with | things like cases with 200+ alternatives. Where these cases also | represent most of the code compiled. | | The worst case I saw was doubled compile time in a Module which only | consisted of a 500 deep if/else chain only selecting | an value. | | While there are some small optimizations possible to improve on this I | don't expect to make these edge cases much faster overall. | However it will also be possible to fall back to the old behavior to | avoid the large compile times for modules known to trigger this | edge case. | | Which brings me to my main question: When should we use the new code | layout. | 1. Always since on average both compile and runtime is faster. | 2. -O1/O2 since users often use this when performance matters. | 3. -O2 since users expect compilation times to blow up with it? | 4. Never as users experience compile time slowdowns when they hit these | edge cases. | | I'm would prefer 2. 3. 1. 4. in that order but I wonder what the wider | community is thinking. | | Cheers | Andreas | _______________________________________________ | ghc-devs mailing list | ghc-devs at haskell.org | http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From klebinger.andreas at gmx.at Thu Aug 30 22:02:09 2018 From: klebinger.andreas at gmx.at (Andreas Klebinger) Date: Fri, 31 Aug 2018 00:02:09 +0200 Subject: Compile times: Average vs worst case runtime In-Reply-To: References: <5B882489.3060305@gmx.at> Message-ID: <5B886961.3030603@gmx.at> Simon Peyton Jones schrieb: > Good work. > > | However it will also be possible to fall back to the old behavior to > | avoid the large compile times for modules known to trigger this > | edge case. > > Not for specific modules, I hope; rather fall back when the number of cases becomes large, or something like that? That'd be fine. As it stands it's a regular flag and hence per Module (just like worker wrapper or similar) and user controlled. In the code example I have in my head it was a large case that get's desugared to a larger chain of if/elseif/elseif/..../else. So it's not strictly large cases but usually it is. I actually didn't consider doing a "dynamic" fall back so far. That's a great idea! If we are lucky we can use the ratio of edges to blocks, and fall back to the old one if we are above a certain function size and ratio. With the threshold in turn depending on the optimization level. But not sure if that's good design as we then end up with cases where performance suddenly gets 5% worse if people add a constructor or code get's slightly larger for some reason. > > The only other potential worry is how tricky/understandable is your patch. Hopefully it's not hard. I hope so, the algorithm itself isn't that complicated to some of the stuff in GHC and I tried to document it well. But it's also far from being trivial code. > > | 1. Always since on average both compile and runtime is faster. > | 2. -O1/O2 since users often use this when performance matters. > | 3. -O2 since users expect compilation times to blow up with it? > | 4. Never as users experience compile time slowdowns when they hit these > | edge cases. > > Well, if you can robustly fall back in edge cases, you'll never hit the blow-ups. So then (1) would be fine would it not? Guess I will have to see how well the edge/block ratio correlates with compile time blowups. If we can use that to rule out the really bad cases then (1) should be fine. If not I will have to come back to the question. Cheers Andreas > > Simon > > > > > | -----Original Message----- > | From: ghc-devs On Behalf Of Andreas > | Klebinger > | Sent: 30 August 2018 18:08 > | To: ghc-devs at haskell.org > | Subject: Compile times: Average vs worst case runtime > | > | Hello Devs, > | > | I developed a patch improving on GHC's code layout during GSoC: > | https://ghc.haskell.org/trac/ghc/ticket/15124 > | The gains are pretty nice with most library benchmarks showing > | improvments in the 1-2% range or more. > | > | Even compile times went down comparing head vs stage2 built with, and > | using my patch! > | Making compilation of nofib overall faster by 0.5-1% depending on the > | exact setup. > | > | Now the bad news: > | In some cases the algorithm has bad big O performance, in practice this > | seems to be code with > | things like cases with 200+ alternatives. Where these cases also > | represent most of the code compiled. > | > | The worst case I saw was doubled compile time in a Module which only > | consisted of a 500 deep if/else chain only selecting > | an value. > | > | While there are some small optimizations possible to improve on this I > | don't expect to make these edge cases much faster overall. > | However it will also be possible to fall back to the old behavior to > | avoid the large compile times for modules known to trigger this > | edge case. > | > | Which brings me to my main question: When should we use the new code > | layout. > | 1. Always since on average both compile and runtime is faster. > | 2. -O1/O2 since users often use this when performance matters. > | 3. -O2 since users expect compilation times to blow up with it? > | 4. Never as users experience compile time slowdowns when they hit these > | edge cases. > | > | I'm would prefer 2. 3. 1. 4. in that order but I wonder what the wider > | community is thinking. > | > | Cheers > | Andreas > | _______________________________________________ > | ghc-devs mailing list > | ghc-devs at haskell.org > | http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs