From takenobu.hs at gmail.com  Tue Dec  3 12:48:15 2019
From: takenobu.hs at gmail.com (Takenobu Tani)
Date: Tue, 3 Dec 2019 21:48:15 +0900
Subject: users-guide page for master branch
In-Reply-To: <CAPB_NvzTRzkpqepOfSKRtcoXfNqPKZpOAzhK69GHuq+0X=1qtQ@mail.gmail.com>
References: <CAPB_NvwZwY6PmHciF-ndwk_S6VtoLBvbY21EdKxCJEqGmU5XWA@mail.gmail.com>
 <13191C16-3659-47CD-AC59-E8F57897AD97@well-typed.com>
 <CAPB_NvzTRzkpqepOfSKRtcoXfNqPKZpOAzhK69GHuq+0X=1qtQ@mail.gmail.com>
Message-ID: <CAPB_NvwcyRZtFJGJ4WL_KbFi+CHJojN7MHM64swDVHMYyZyM-Q@mail.gmail.com>

Thanks for resolving this.
The recent master document is displayed. It's useful :)

https://ghc.gitlab.haskell.org/ghc/doc/users_guide/

Regards,
Takenobu


On Wed, Oct 30, 2019 at 10:57 PM Takenobu Tani <takenobu.hs at gmail.com> wrote:
>
> Thanks for your explanation.
> I'll check again after the Windows issue is fixed.
>
> Thanks,
> Takenobu
>
> On Wed, Oct 30, 2019 at 10:27 PM Ben Gamari <ben at well-typed.com> wrote:
> >
> > Indeed, this is because of the fact that the Windows build is currently failing due to hsc2hs issues. I have been ignoring the documentation build until the Windows issue has been resolved.
> >
> > Thanks for checking though. Don't hesitate to mention this again if you notice the build is still failing after the Windows issue is fixed.
> >
> > Cheers,
> >
> > - Ben

From ben at smart-cactus.org  Wed Dec  4 16:38:41 2019
From: ben at smart-cactus.org (Ben Gamari)
Date: Wed, 04 Dec 2019 11:38:41 -0500
Subject: new llvm IR callbr instruction useful for proc-point splitting?
In-Reply-To: <a876f857876430b7b089ef6a0241aaeee776c45a.camel@farvard.in>
References: <874l4qzlny.fsf@smart-cactus.org>
 <CAB-d4A6QyumkF3LLc8rk=NQ4jTVDOqXmo5AWOOQ2=qZr6E8JPQ@mail.gmail.com>
 <877e9dy89j.fsf@smart-cactus.org>
 <CAB-d4A5S=Og2-1N07LqYbg64UEX1FWMWXKn_GeCQguZ2Gin6AA@mail.gmail.com>
 <a876f857876430b7b089ef6a0241aaeee776c45a.camel@farvard.in>
Message-ID: <87h82g82r8.fsf@smart-cactus.org>

Kavon Farvardin <kavon at farvard.in> writes:

> Yes, callbr is part of the discussion on removing proc-point splitting! 
> Soon there will be an announcement about a new working group dedicated to LLVM
> issues such as this one on this mailing list. Anyone interested is welcome to
> join our meetings.
> We're currently gathering our notes together first, but you can have a look here
> for a preview: 
> https://gitlab.haskell.org/ghc/ghc/wikis/commentary/compiler/LLVM-Backend-Optimization

Hi Kavon,

What ever happened to this? It came up during the GHC call and we
realized that none of us knew what the state of the proposal was. Is
this Wiki page still reflective of the current state of play?

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191204/2d3428d1/attachment.sig>

From kavon at farvard.in  Wed Dec  4 19:31:46 2019
From: kavon at farvard.in (Kavon Farvardin)
Date: Wed, 04 Dec 2019 13:31:46 -0600
Subject: new llvm IR callbr instruction useful for proc-point splitting?
In-Reply-To: <87h82g82r8.fsf@smart-cactus.org>
References: <874l4qzlny.fsf@smart-cactus.org>
 <CAB-d4A6QyumkF3LLc8rk=NQ4jTVDOqXmo5AWOOQ2=qZr6E8JPQ@mail.gmail.com>
 <877e9dy89j.fsf@smart-cactus.org>
 <CAB-d4A5S=Og2-1N07LqYbg64UEX1FWMWXKn_GeCQguZ2Gin6AA@mail.gmail.com>
 <a876f857876430b7b089ef6a0241aaeee776c45a.camel@farvard.in>
 <87h82g82r8.fsf@smart-cactus.org>
Message-ID: <5a27d1a33100163c0ba11285c2022105b9929cf6.camel@farvard.in>

Hi Ben,
The Wiki now has a link to the Google Doc that Michal and I were working on to
compile all of the notes into one place.
The key progress made on this front has been in evaluating the trade-offs of the
design space being considered by GHC, but tested within Manticore.
I'm drafting a message today for the LLVM mailing list regarding one of our
first options: defining a GHC ABI to redefine which stack-pointer register is
used. 
Once we have additional answers to the question "how much effort is it to add a
GHC ABI to LLVM?", we can then then start a discussion on whether GHC wants to
change instead by moving to a native stack-pointer register (to take advantage
of existing debugging and profiling tools, etc).
Ideally, we would then make a decision on which option to take and someone
(perhaps a GSoC student with mentorship from myself and others) with spare time
can work on it.
Cheers,Kavon
On Wed, 2019-12-04 at 11:38 -0500, Ben Gamari wrote:
> Kavon Farvardin <kavon at farvard.in> writes:
> Yes, callbr is part of the discussion on removing proc-point splitting! Soon
> there will be an announcement about a new working group dedicated to
> LLVMissues such as this one on this mailing list. Anyone interested is welcome
> tojoin our meetings.We're currently gathering our notes together first, but
> you can have a look herefor a preview: 
> https://gitlab.haskell.org/ghc/ghc/wikis/commentary/compiler/LLVM-Backend-Optimization
> Hi Kavon,
> What ever happened to this? It came up during the GHC call and werealized that
> none of us knew what the state of the proposal was. Isthis Wiki page still
> reflective of the current state of play?
> Cheers,
> - Ben
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191204/b5b5ca8d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191204/b5b5ca8d/attachment.sig>

From simonpj at microsoft.com  Wed Dec  4 22:16:10 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Wed, 4 Dec 2019 22:16:10 +0000
Subject: new llvm IR callbr instruction useful for proc-point splitting?
In-Reply-To: <5a27d1a33100163c0ba11285c2022105b9929cf6.camel@farvard.in>
References: <874l4qzlny.fsf@smart-cactus.org>
 <CAB-d4A6QyumkF3LLc8rk=NQ4jTVDOqXmo5AWOOQ2=qZr6E8JPQ@mail.gmail.com>
 <877e9dy89j.fsf@smart-cactus.org>
 <CAB-d4A5S=Og2-1N07LqYbg64UEX1FWMWXKn_GeCQguZ2Gin6AA@mail.gmail.com>
 <a876f857876430b7b089ef6a0241aaeee776c45a.camel@farvard.in>
 <87h82g82r8.fsf@smart-cactus.org>
 <5a27d1a33100163c0ba11285c2022105b9929cf6.camel@farvard.in>
Message-ID: <VI1PR8303MB009307E88147C7160DF78FF3AD5D0@VI1PR8303MB0093.EURPRD83.prod.outlook.com>

Thanks Kavon.

What news of CpsCall in LLVM?  That seems crucial.

I didn’t know of an effort to use a different SP register.  I seriously doubt that, with GHC’s use of zillions of small, heap-allocated stacks that move around during GHC, we’ll be able to leverage profiling or debugging tools in a serious way. But perhaps I need education.

Simon

From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Kavon Farvardin
Sent: 04 December 2019 19:32
To: Ben Gamari <ben at smart-cactus.org>
Cc: GHC developers <ghc-devs at haskell.org>
Subject: Re: new llvm IR callbr instruction useful for proc-point splitting?

Hi Ben,

The Wiki now has a link to the Google Doc that Michal and I were working on to compile all of the notes into one place.

The key progress made on this front has been in evaluating the trade-offs of the design space being considered by GHC, but tested within Manticore.

I'm drafting a message today for the LLVM mailing list regarding one of our first options: defining a GHC ABI to redefine which stack-pointer register is used.

Once we have additional answers to the question "how much effort is it to add a GHC ABI to LLVM?", we can then then start a discussion on whether GHC wants to change instead by moving to a native stack-pointer register (to take advantage of existing debugging and profiling tools, etc).

Ideally, we would then make a decision on which option to take and someone (perhaps a GSoC student with mentorship from myself and others) with spare time can work on it.

Cheers,
Kavon

On Wed, 2019-12-04 at 11:38 -0500, Ben Gamari wrote:

Kavon Farvardin <kavon at farvard.in<mailto:kavon at farvard.in>> writes:



Yes, callbr is part of the discussion on removing proc-point splitting!

Soon there will be an announcement about a new working group dedicated to LLVM

issues such as this one on this mailing list. Anyone interested is welcome to

join our meetings.

We're currently gathering our notes together first, but you can have a look here

for a preview:

https://gitlab.haskell.org/ghc/ghc/wikis/commentary/compiler/LLVM-Backend-Optimization



Hi Kavon,



What ever happened to this? It came up during the GHC call and we

realized that none of us knew what the state of the proposal was. Is

this Wiki page still reflective of the current state of play?



Cheers,



- Ben




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191204/84baf62e/attachment.html>

From george.colpitts at gmail.com  Wed Dec  4 23:02:39 2019
From: george.colpitts at gmail.com (George Colpitts)
Date: Wed, 4 Dec 2019 19:02:39 -0400
Subject: new llvm IR callbr instruction useful for proc-point splitting?
In-Reply-To: <VI1PR8303MB009307E88147C7160DF78FF3AD5D0@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
References: <874l4qzlny.fsf@smart-cactus.org>
 <CAB-d4A6QyumkF3LLc8rk=NQ4jTVDOqXmo5AWOOQ2=qZr6E8JPQ@mail.gmail.com>
 <877e9dy89j.fsf@smart-cactus.org>
 <CAB-d4A5S=Og2-1N07LqYbg64UEX1FWMWXKn_GeCQguZ2Gin6AA@mail.gmail.com>
 <a876f857876430b7b089ef6a0241aaeee776c45a.camel@farvard.in>
 <87h82g82r8.fsf@smart-cactus.org>
 <5a27d1a33100163c0ba11285c2022105b9929cf6.camel@farvard.in>
 <VI1PR8303MB009307E88147C7160DF78FF3AD5D0@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
Message-ID: <CAB-d4A4R9e6NioZovSB_A69VVZ00qQdjK=b6PG=8reT3M-MG+A@mail.gmail.com>

Kavin is definitely the right guy to answer this but as he mentioned the
Wiki now has a link to the Google Doc, i.e.
https://docs.google.com/document/d/1Pn2_TDBJwSBKkjFlT-blwygH10z37TPxIqcKTOOf2mg/edit#.
Simon, have you had a chance to read that?

If I understand that doc correctly the original hope of a CPS
call intrinsic in LLVM has been abandoned. Excerpts from the doc:

First approach -- “CPS call” intrinsic in LLVM
...
Current ideas
...

CPS call

LLVM recently added callbr instruction that looks similar to what we need
to get CPS calls to work (the instruction allows to reify the return
address as a parameter to a call and specify the successor blocks where the
call might jump back to).




On Wed, Dec 4, 2019 at 6:16 PM Simon Peyton Jones via ghc-devs <
ghc-devs at haskell.org> wrote:

> Thanks Kavon.
>
>
>
> What news of CpsCall in LLVM?  That seems crucial.
>
>
>
> I didn’t know of an effort to use a different SP register.  I seriously
> doubt that, with GHC’s use of zillions of small, heap-allocated stacks that
> move around during GHC, we’ll be able to leverage profiling or debugging
> tools in a serious way. But perhaps I need education.
>
>
>
> Simon
>
>
>
> *From:* ghc-devs <ghc-devs-bounces at haskell.org> *On Behalf Of *Kavon
> Farvardin
> *Sent:* 04 December 2019 19:32
> *To:* Ben Gamari <ben at smart-cactus.org>
> *Cc:* GHC developers <ghc-devs at haskell.org>
> *Subject:* Re: new llvm IR callbr instruction useful for proc-point
> splitting?
>
>
>
> Hi Ben,
>
>
>
> The Wiki now has a link to the Google Doc that Michal and I were working
> on to compile all of the notes into one place.
>
>
>
> The key progress made on this front has been in evaluating the trade-offs
> of the design space being considered by GHC, but tested within Manticore.
>
>
>
> I'm drafting a message today for the LLVM mailing list regarding one of
> our first options: defining a GHC ABI to redefine which stack-pointer
> register is used.
>
>
>
> Once we have additional answers to the question "how much effort is it to
> add a GHC ABI to LLVM?", we can then then start a discussion on whether GHC
> wants to change instead by moving to a native stack-pointer register (to
> take advantage of existing debugging and profiling tools, etc).
>
>
>
> Ideally, we would then make a decision on which option to take and someone
> (perhaps a GSoC student with mentorship from myself and others) with spare
> time can work on it.
>
>
>
> Cheers,
>
> Kavon
>
>
>
> On Wed, 2019-12-04 at 11:38 -0500, Ben Gamari wrote:
>
> Kavon Farvardin <kavon at farvard.in> writes:
>
>
>
> Yes, callbr is part of the discussion on removing proc-point splitting!
>
> Soon there will be an announcement about a new working group dedicated to LLVM
>
> issues such as this one on this mailing list. Anyone interested is welcome to
>
> join our meetings.
>
> We're currently gathering our notes together first, but you can have a look here
>
> for a preview:
>
> https://gitlab.haskell.org/ghc/ghc/wikis/commentary/compiler/LLVM-Backend-Optimization
>
>
>
> Hi Kavon,
>
>
>
> What ever happened to this? It came up during the GHC call and we
>
> realized that none of us knew what the state of the proposal was. Is
>
> this Wiki page still reflective of the current state of play?
>
>
>
> Cheers,
>
>
>
> - Ben
>
>
>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191204/7246ac3f/attachment.html>

From rae at richarde.dev  Thu Dec  5 09:53:37 2019
From: rae at richarde.dev (Richard Eisenberg)
Date: Thu, 5 Dec 2019 09:53:37 +0000
Subject: re-engineering overloading and rebindable syntax
Message-ID: <AB2DAEAF-FF9D-49B2-B67E-15B996BBBAE9@richarde.dev>

Hi devs,

I've recently discovered the `overloaded` package, which allows us to do very clever things around overloading in a source plugin. In my over-excitement, I wondered about removing GHC's capability to do overloading and rebindable syntax, leaving those features to be implemented by source plugins. To first approximation, this would work well. But there are many drawbacks: poorer error messages, worries around the trustworthiness of non-GHC source plugins, cross compilation, etc. The idea is not viable right now.

Yet: I wonder if we couldn't take the general idea of doing this as source plugins, without actually implementing it that way. Currently, there is a good deal of code scattered throughout GHC to deal with overloaded constructs (numbers, strings, lists) and rebindable syntax. These make their presence known in a number of places in HsExpr and friends, and various machinations in the renamer and type-checker are needed to deal with them. The type-checker for rebindable syntax is buggy (#14963 and friends); the fix will involve likely *more* code devoted to rebindable syntax (albeit simpler code than what we have today).

If, say, the renamer implemented all overloading and rebindable syntax just by a straightforward syntactic transformation, this would all be much simpler. (Actually, it could also be done in the parser, but I think the renamer is a better home.)

Pros:
 - much, much simpler implementation
 - localized implementation: nothing in the type checker at all nor in HsSyn

Con:
 - worse error messages, which would now refer to the desugared code instead of the user-written code.

I can't think of any other downside.

How can we mitigate this? By expanding the possibilities of a SrcSpan. A SrcSpan is really a description of the provenance of a piece of AST. Right now, it considers two possibilities: that the code came from a specific stretch of rows and columns in an input file, or that the code came from elsewhere. Instead, we can expand (and perhaps rename) SrcSpan to include more possibilities. In support of my idea above, we could now have a SrcSpan that says some AST came from overloading. Such code is suppressed by default when pretty-printing. Thus, `fromString "blah"` could render as just `"blah"` (what the user wrote), if fromString was inserted by the translation pass described above. We can have a separate new SrcSpan that says that AST was written by translation from some original AST. That way, `ifThenElse a b c` can be printed as `if a then b else c`, if the former were translated from the latter. Though it's beyond my use-case above, we can also imagine a new SrcSpans that refer to a Template Haskell splice or a quasiquote.

What do we think? Is this a worthwhile direction of travel? I think the end result would be both a cleaner implementation of overloading and rebindable syntax *and* more informative and useful source-code provenances.

Richard

From simonpj at microsoft.com  Thu Dec  5 10:09:37 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Thu, 5 Dec 2019 10:09:37 +0000
Subject: re-engineering overloading and rebindable syntax
In-Reply-To: <AB2DAEAF-FF9D-49B2-B67E-15B996BBBAE9@richarde.dev>
References: <AB2DAEAF-FF9D-49B2-B67E-15B996BBBAE9@richarde.dev>
Message-ID: <VI1PR8303MB0093D94EAD6DDEC254B6A189AD5C0@VI1PR8303MB0093.EURPRD83.prod.outlook.com>

Sounds attractive.  By all means open a ticket, giving details.  I'm hazy about those details.  I think you are implying that every single piece of rebindable syntax can be typechecked by instead typechecking some (invisible) function application. Including when you are not using rebindable syntax: just use a built-in function.  

I think that's the way rebindable syntax mostly works today, but I'm not 100% sure that the vanilla (no -XRebindableSyntax) case is always done that way.   One way to get at this would be to write a table of all the cases where rebindable syntax is used, and what the equivalent function application or typing judgement is.  That would be helpful regardless!

Examples of where the error message gets worse, and why, would help to focus the conversation.

Simon 

|  -----Original Message-----
|  From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Richard
|  Eisenberg
|  Sent: 05 December 2019 09:54
|  To: Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org>
|  Subject: re-engineering overloading and rebindable syntax
|  
|  Hi devs,
|  
|  I've recently discovered the `overloaded` package, which allows us to do
|  very clever things around overloading in a source plugin. In my over-
|  excitement, I wondered about removing GHC's capability to do overloading
|  and rebindable syntax, leaving those features to be implemented by source
|  plugins. To first approximation, this would work well. But there are many
|  drawbacks: poorer error messages, worries around the trustworthiness of
|  non-GHC source plugins, cross compilation, etc. The idea is not viable
|  right now.
|  
|  Yet: I wonder if we couldn't take the general idea of doing this as source
|  plugins, without actually implementing it that way. Currently, there is a
|  good deal of code scattered throughout GHC to deal with overloaded
|  constructs (numbers, strings, lists) and rebindable syntax. These make
|  their presence known in a number of places in HsExpr and friends, and
|  various machinations in the renamer and type-checker are needed to deal
|  with them. The type-checker for rebindable syntax is buggy (#14963 and
|  friends); the fix will involve likely *more* code devoted to rebindable
|  syntax (albeit simpler code than what we have today).
|  
|  If, say, the renamer implemented all overloading and rebindable syntax
|  just by a straightforward syntactic transformation, this would all be much
|  simpler. (Actually, it could also be done in the parser, but I think the
|  renamer is a better home.)
|  
|  Pros:
|   - much, much simpler implementation
|   - localized implementation: nothing in the type checker at all nor in
|  HsSyn
|  
|  Con:
|   - worse error messages, which would now refer to the desugared code
|  instead of the user-written code.
|  
|  I can't think of any other downside.
|  
|  How can we mitigate this? By expanding the possibilities of a SrcSpan. A
|  SrcSpan is really a description of the provenance of a piece of AST. Right
|  now, it considers two possibilities: that the code came from a specific
|  stretch of rows and columns in an input file, or that the code came from
|  elsewhere. Instead, we can expand (and perhaps rename) SrcSpan to include
|  more possibilities. In support of my idea above, we could now have a
|  SrcSpan that says some AST came from overloading. Such code is suppressed
|  by default when pretty-printing. Thus, `fromString "blah"` could render as
|  just `"blah"` (what the user wrote), if fromString was inserted by the
|  translation pass described above. We can have a separate new SrcSpan that
|  says that AST was written by translation from some original AST. That way,
|  `ifThenElse a b c` can be printed as `if a then b else c`, if the former
|  were translated from the latter. Though it's beyond my use-case above, we
|  can also imagine a new SrcSpans that refer to a Template Haskell splice or
|  a quasiquote.
|  
|  What do we think? Is this a worthwhile direction of travel? I think the
|  end result would be both a cleaner implementation of overloading and
|  rebindable syntax *and* more informative and useful source-code
|  provenances.
|  
|  Richard
|  _______________________________________________
|  ghc-devs mailing list
|  ghc-devs at haskell.org
|  http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From vladislav at serokell.io  Thu Dec  5 10:41:13 2019
From: vladislav at serokell.io (Vladislav Zavialov)
Date: Thu, 5 Dec 2019 13:41:13 +0300
Subject: re-engineering overloading and rebindable syntax
In-Reply-To: <AB2DAEAF-FF9D-49B2-B67E-15B996BBBAE9@richarde.dev>
References: <AB2DAEAF-FF9D-49B2-B67E-15B996BBBAE9@richarde.dev>
Message-ID: <EA94C898-AB63-496C-BE93-DCB3C2D3D494@serokell.io>

I find this idea attractive. Could we desugar `do` in the same manner using this SrcSpan trick? Could we desugar infix operators `a + b` to `(+) a b`? We just need to store in theh SrcSpan that the (+) was actually infix. Implementing as much syntactic sugar as possible this way would let us push complexity out of the type checker. 

- Vlad

> On 5 Dec 2019, at 12:53, Richard Eisenberg <rae at richarde.dev> wrote:
> 
> Hi devs,
> 
> How can we mitigate this? By expanding the possibilities of a SrcSpan. A SrcSpan is really a description of the provenance of a piece of AST. Right now, it considers two possibilities: that the code came from a specific stretch of rows and columns in an input file, or that the code came from elsewhere. Instead, we can expand (and perhaps rename) SrcSpan to include more possibilities. In support of my idea above, we could now have a SrcSpan that says some AST came from overloading. Such code is suppressed by default when pretty-printing. Thus, `fromString "blah"` could render as just `"blah"` (what the user wrote), if fromString was inserted by the translation pass described above. We can have a separate new SrcSpan that says that AST was written by translation from some original AST. That way, `ifThenElse a b c` can be printed as `if a then b else c`, if the former were translated from the latter. Though it's beyond my use-case above, we can also imagine a new SrcSpans that refer to a Template Haskell splice or a quasiquote.
> 
> What do we think? Is this a worthwhile direction of travel? I think the end result would be both a cleaner implementation of overloading and rebindable syntax *and* more informative and useful source-code provenances.
> 
> Richard
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From kavon at farvard.in  Thu Dec  5 17:18:11 2019
From: kavon at farvard.in (Kavon Farvardin)
Date: Thu, 05 Dec 2019 11:18:11 -0600
Subject: new llvm IR callbr instruction useful for proc-point splitting?
In-Reply-To: <VI1PR8303MB009307E88147C7160DF78FF3AD5D0@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
References: <874l4qzlny.fsf@smart-cactus.org>
 <CAB-d4A6QyumkF3LLc8rk=NQ4jTVDOqXmo5AWOOQ2=qZr6E8JPQ@mail.gmail.com>
 <877e9dy89j.fsf@smart-cactus.org>
 <CAB-d4A5S=Og2-1N07LqYbg64UEX1FWMWXKn_GeCQguZ2Gin6AA@mail.gmail.com>
 <a876f857876430b7b089ef6a0241aaeee776c45a.camel@farvard.in>
 <87h82g82r8.fsf@smart-cactus.org>
 <5a27d1a33100163c0ba11285c2022105b9929cf6.camel@farvard.in>
 <VI1PR8303MB009307E88147C7160DF78FF3AD5D0@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
Message-ID: <2a7d8dff22d3b27a025c690e013ba39921360102.camel@farvard.in>



On Wed, 2019-12-04 at 22:16 +0000, Simon Peyton Jones wrote:
> Thanks Kavon.
> 
>  
> 
> What news of CpsCall in LLVM?  That seems crucial.

CPSCall is in essence equivalent to an existing construct in LLVM for GC support
called "statepoints." The difference is that CPSCall expects the front-end to
allocate the stack frame, whereas statepoints are hooked into LLVM's stack-
allocation infrastructure.
While our usage of CPSCall is correct by construction to not require LLVM's
stack-allocation, LLVM's code generator has free-reign to generate stack spills
across a CPSCall for values that one can trivially rematerialize. I don't think
it's sustainable to work around that already deeply-ingrained behavior (however
silly it may be!), so instead the plan is to extend LLVM's statepoints to be
compatible with GHC's runtime system.
How we go about doing this depends on our desires: do we want to also use the
native SP register or not? Some earlier investigation revealed that even on
Windows, one could use a native SP that points to a heap-allocated object,
provided ~512bytes of space is left beyond the SP for the kernel's interrupt
handler. The implementation of Go does this, despite the Windows ABI forbidding
it.
>  
> 
> I didn’t know of an effort to use a different SP register.  I seriously doubt
> that, with GHC’s use of zillions of small, heap-allocated stacks that move
> around during GHC, we’ll be able to leverage profiling or debugging tools in a
> serious
>  way. But perhaps I need education.

Existing debugging and profiling tools (e.g., gdb and Linux perf) can become
more useful for GHC if they understood how to walk the call stack, regardless of
whether the frame moves in memory. One way to do this is to emit call-frame
information (CFI) [1,2] in the output assembly that describes the location and
layout of the frame as it changes. This would work regardless of the SP register
chosen and even if the return address is kept in register. An alternative to
using CFI is to use a the canonical SP register and layout, since the default
"CFI" assumed by these tools is the OS's ABI.
CFI is the most robust solution for GHC, since some functions allocate more than
one frame per invocation to save on code size when applying a curried function
(the Making a Fast Curry paper). This is of course orthogonal to the proc-point
splitting concern.
I believe Ben originally brought up this desire to use a native SP even for the
native backend, because the work of switching to gives us both automatic CFI
support and some branch-predictor performance benefits on x86-64. That's likely
to be a big overhaul though, so I'm personally in favor of extending LLVM's
statepoints to be compatible with GHC and concurrently but separately adding CFI
support for the native backend.

[1] https://sourceware.org/binutils/docs/as/CFI-directives.html[2] 
https://www.imperialviolet.org/2017/01/18/cfi.html

>  
> Simon
>  
> 
> 
> 
> From: ghc-devs <ghc-devs-bounces at haskell.org>
> On Behalf Of Kavon Farvardin
> 
> Sent: 04 December 2019 19:32
> 
> To: Ben Gamari <ben at smart-cactus.org>
> 
> Cc: GHC developers <ghc-devs at haskell.org>
> 
> Subject: Re: new llvm IR callbr instruction useful for proc-point splitting?
> 
> 
>  
> 
> 
> Hi Ben,
> 
> 
>  
> 
> 
> The Wiki now has a link to the Google Doc that Michal and I were working on to
> compile all of the notes into one place.
> 
> 
>  
> 
> 
> The key progress made on this front has been in evaluating the trade-offs of
> the design space being considered by GHC, but tested within Manticore.
> 
> 
>  
> 
> 
> I'm drafting a message today for the LLVM mailing list regarding one of our
> first options: defining a GHC ABI to redefine which stack-pointer register is
> used. 
> 
> 
>  
> 
> 
> Once we have additional answers to the question "how much effort is it to add
> a GHC ABI to LLVM?", we can then then start a discussion on whether GHC wants
> to change instead by moving to a native stack-pointer register (to take
> advantage
>  of existing debugging and profiling tools, etc).
> 
> 
>  
> 
> 
> Ideally, we would then make a decision on which option to take and someone
> (perhaps a GSoC student with mentorship from myself and others) with spare
> time can work on it.
> 
> 
>  
> 
> 
> Cheers,
> 
> 
> Kavon
> 
>  
> 
> 
> On Wed, 2019-12-04 at 11:38 -0500, Ben Gamari wrote:
> 
> > Kavon Farvardin <kavon at farvard.in> writes:
> >  
> > Yes, callbr is part of the discussion on removing proc-point splitting! 
> > Soon there will be an announcement about a new working group dedicated to
> > LLVM
> > issues such as this one on this mailing list. Anyone interested is welcome
> > to
> > join our meetings.
> > We're currently gathering our notes together first, but you can have a look
> > here
> > for a preview: 
> > 
https://gitlab.haskell.org/ghc/ghc/wikis/commentary/compiler/LLVM-Backend-Optimization
> >  
> > Hi Kavon,
> >  
> > What ever happened to this? It came up during the GHC call and we
> > realized that none of us knew what the state of the proposal was. Is
> > this Wiki page still reflective of the current state of play?
> >  
> > Cheers,
> >  
> > - Ben
> >  
> >  
> 
> 
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191205/5c3e5131/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191205/5c3e5131/attachment.sig>

From monkleyon at gmail.com  Thu Dec  5 20:47:25 2019
From: monkleyon at gmail.com (MarLinn)
Date: Thu, 5 Dec 2019 21:47:25 +0100
Subject: re-engineering overloading and rebindable syntax
In-Reply-To: <AB2DAEAF-FF9D-49B2-B67E-15B996BBBAE9@richarde.dev>
References: <AB2DAEAF-FF9D-49B2-B67E-15B996BBBAE9@richarde.dev>
Message-ID: <915a4fe1-cf5e-83ac-6d3a-28c4d7c29ae6@gmail.com>

Hi,

On 05/12/2019 10.53, Richard Eisenberg wrote:
> Con:
>   - worse error messages, which would now refer to the desugared code instead of the user-written code.

I can think of several major parts that seem useful to solve this. 
(Please excuse the very abstract and imprecise descriptions.)

1. Most "plugins" will probably still need a way to annotate the AST in 
some way to inform their error message (i.e. SrcSpan?). These 
annotations would be expand-only for all other plugins, the type 
checker, and all other possible transformations. Their only job is to 
inform their originator about what it did in case of an error. Ideally 
this would be done in an expandable way.

2. Some stages might want to create suggested ASTs to help the user 
understand the error, so this option should be part of an error 
structure bubbling back up through the "plugins". But: the originators 
of said ASTs should not need to know about annotations other than their 
own, so they can not be expected to faithfully add them.

3. A "plugin" needs the ability to analyse an erroneous AST inside an 
error structure and discover how it might have been constructed by said 
plugin. Crucially, it might want to analyse the suggested ASTs with 
potentially missing or misleading annotations.

This last part has me curious… if a plugin could do that well, it could 
also quite possibly re-sugar any random piece of AST. In other words 
there might come a day where GHC can tell a user what their code might 
look like with different language extensions applied. And that's just 
the tip of an iceberg of tooling possibilities.

So this might be a pipe dream for now. But practically speaking, I 
expect most programmers to lean in one of two directions: to rely on 
annotations heavily and ignore any suggested ASTs, or to go all-in on 
the analysing part and annotate as little as possible. So if even one or 
two implementers choose this second route, the pipe dream might become a 
partial reality in the not-so-long term.


From simonpj at microsoft.com  Mon Dec  9 09:10:16 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 9 Dec 2019 09:10:16 +0000
Subject: Running test with the stage-1 compiler
Message-ID: <VI1PR8303MB009360C4415E2872D56327E3AD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>

Apparently 'sh validate' now uses Hadrian.
This has broken one of my workflows:

  *   build the stage1 compiler
  *   cd testsuite/tests
  *   make stage=1
That is, run the testsuite with the stage-1 compile.  (This often produces a small test case for a compiler crash, much easier than debugging GHC compiling itself.)
How do I do this?
Thanks
Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191209/2ee6c843/attachment.html>

From omeragacan at gmail.com  Mon Dec  9 09:12:14 2019
From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=)
Date: Mon, 9 Dec 2019 12:12:14 +0300
Subject: Running test with the stage-1 compiler
In-Reply-To: <VI1PR8303MB009360C4415E2872D56327E3AD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
References: <VI1PR8303MB009360C4415E2872D56327E3AD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
Message-ID: <CAMQQO3nC=G2Ym4w1bw-5GuxrCDc3XHaNjnhoEYvvuSUNtCiUGA@mail.gmail.com>

You can use `sh validate --legacy` to use the old build system.

Ömer

Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org>, 9 Ara 2019
Pzt, 12:10 tarihinde şunu yazdı:
>
> Apparently ‘sh validate’ now uses Hadrian.
>
> This has broken one of my workflows:
>
> build the stage1 compiler
> cd testsuite/tests
> make stage=1
>
> That is, run the testsuite with the stage-1 compile.  (This often produces a small test case for a compiler crash, much easier than debugging GHC compiling itself.)
>
> How do I do this?
>
> Thanks
>
> Simon
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From simonpj at microsoft.com  Mon Dec  9 09:29:01 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 9 Dec 2019 09:29:01 +0000
Subject: Total failure
Message-ID: <VI1PR8303MB009308906619379C3E296DEBAD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>

Bad things are happening.   I can't build GHC at all!
I'm trying 'sh validate -fast' on a clean build of a master checkout
HEAD detached at 3a96a0b6db
(this is the master checkout from which !2276 branches off)
But alas I get this

Exit code: 1

Stderr:

compiler/main/DynFlags.hs:323:0: error:

     error: "GHC_STAGE" is not defined, evaluates to 0 [-Werror=undef]

     #if GHC_STAGE >= 2



    |

323 | #if GHC_STAGE >= 2

    | ^



compiler/main/DynFlags.hs:5750:0: error:

     error: "GHC_STAGE" is not defined, evaluates to 0 [-Werror=undef]

     #if GHC_STAGE < 2



     |

5750 | #if GHC_STAGE < 2

     | ^

cc1: all warnings being treated as errors

What should I do?
Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191209/042a315d/attachment.html>

From simonpj at microsoft.com  Mon Dec  9 09:51:07 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 9 Dec 2019 09:51:07 +0000
Subject: More failure
Message-ID: <VI1PR8303MB00939658692A6946094FB87BAD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>

More pain.  I said
sh validate --fast --no-clean
and got this output

Error when running Shake build system:

  at action, called at src/Rules.hs:71:19 in main:Rules

  at need, called at src/Rules.hs:93:5 in main:Rules

* Depends on: _validatebuild/stage1/lib/package.conf.d/base-4.14.0.0.conf

  at need, called at src/Rules/Register.hs:113:5 in main:Rules.Register

* Depends on: _validatebuild/stage1/libraries/base/build/libHSbase-4.14.0.0-ghc8.11.0.20191201.so

  at need, called at src/Rules/Library.hs:146:5 in main:Rules.Library

* Depends on: _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.dyn_o

  at &%>, called at src/Rules/Compile.hs:54:7 in main:Rules.Compile

* Depends on: _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.dyn_o _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.dyn_hi

  at error, called at src/Development/Shake/Internal/Rules/Files.hs:245:13 in shake-0.18.3-593067565aafb558d09b4352b8abc327d8911a39a0e9abab2804b002b1ae536e:Development.Shake.Internal.Rules.Files

* Raised the exception:

Error, &%> rule failed to produce 1 file (out of 2)

  _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.dyn_o

  _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.dyn_hi - MISSING
I think I'm going to have to revert to 'sh validate -legacy'.  Please let me know when I should try again with Hadrian.
Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191209/d458befb/attachment.html>

From ben at smart-cactus.org  Mon Dec  9 17:20:27 2019
From: ben at smart-cactus.org (Ben Gamari)
Date: Mon, 09 Dec 2019 12:20:27 -0500
Subject: Total failure
In-Reply-To: <VI1PR8303MB009308906619379C3E296DEBAD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
References: <VI1PR8303MB009308906619379C3E296DEBAD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
Message-ID: <87a7818lgm.fsf@smart-cactus.org>

Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:

> Bad things are happening.   I can't build GHC at all!
> I'm trying 'sh validate -fast' on a clean build of a master checkout
> HEAD detached at 3a96a0b6db
> (this is the master checkout from which !2276 branches off)
> But alas I get this
>
I think I see what is going on here. If you run `make WERROR=""` for a
while to get past DynFlags does the build finish?

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191209/58094917/attachment.sig>

From ben at smart-cactus.org  Mon Dec  9 22:23:38 2019
From: ben at smart-cactus.org (Ben Gamari)
Date: Mon, 09 Dec 2019 17:23:38 -0500
Subject: Running test with the stage-1 compiler
In-Reply-To: <VI1PR8303MB009360C4415E2872D56327E3AD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
References: <VI1PR8303MB009360C4415E2872D56327E3AD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
Message-ID: <87lfrl87fj.fsf@smart-cactus.org>

Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:

> Apparently 'sh validate' now uses Hadrian.
> This has broken one of my workflows:
>
>   *   build the stage1 compiler
>   *   cd testsuite/tests
>   *   make stage=1
>
> That is, run the testsuite with the stage-1 compile.  (This often produces a small test case for a compiler crash, much easier than debugging GHC compiling itself.)
> How do I do this?

I suspect (but have not tested) that you can accomplish this with

    hadrian/build.cabal.sh --flavor=validate --build-root=_buildvalidate  --test-compiler=_buildvalidate/stage0/bin/ghc

Although I'll admit  this is far from convenient. If you weren't using
the `validate` script this would just be

    hadrian/build.cabal.sh --test-compiler=_build/stage0/bin/ghc

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191209/a13bfed7/attachment.sig>

From ben at smart-cactus.org  Mon Dec  9 22:27:50 2019
From: ben at smart-cactus.org (Ben Gamari)
Date: Mon, 09 Dec 2019 17:27:50 -0500
Subject: More failure
In-Reply-To: <VI1PR8303MB00939658692A6946094FB87BAD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
References: <VI1PR8303MB00939658692A6946094FB87BAD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
Message-ID: <87immp878b.fsf@smart-cactus.org>

Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:

> More pain.  I said
> sh validate --fast --no-clean
> and got this output
>
> Error when running Shake build system:
>
>   at action, called at src/Rules.hs:71:19 in main:Rules
>
>   at need, called at src/Rules.hs:93:5 in main:Rules
>
> * Depends on: _validatebuild/stage1/lib/package.conf.d/base-4.14.0.0.conf
>
>   at need, called at src/Rules/Register.hs:113:5 in main:Rules.Register
>
> * Depends on: _validatebuild/stage1/libraries/base/build/libHSbase-4.14.0.0-ghc8.11.0.20191201.so
>
>   at need, called at src/Rules/Library.hs:146:5 in main:Rules.Library
>
> * Depends on: _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.dyn_o
>
>   at &%>, called at src/Rules/Compile.hs:54:7 in main:Rules.Compile
>
> * Depends on: _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.dyn_o _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.dyn_hi
>
>   at error, called at src/Development/Shake/Internal/Rules/Files.hs:245:13 in shake-0.18.3-593067565aafb558d09b4352b8abc327d8911a39a0e9abab2804b002b1ae536e:Development.Shake.Internal.Rules.Files
>
> * Raised the exception:
>
> Error, &%> rule failed to produce 1 file (out of 2)
>
>   _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.dyn_o
>
>   _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.dyn_hi - MISSING

I suspect that this is due to a previously-failed build and #17534.
The workaround is to remove the traces of the half-finished build:

    rm _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.*

and rerun the build; this may require multiple iterations.

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191209/ac936c66/attachment.sig>

From simonpj at microsoft.com  Mon Dec  9 23:06:17 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 9 Dec 2019 23:06:17 +0000
Subject: More failure
In-Reply-To: <87immp878b.fsf@smart-cactus.org>
References: <VI1PR8303MB00939658692A6946094FB87BAD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
 <87immp878b.fsf@smart-cactus.org>
Message-ID: <VI1PR8303MB0093CB96833562ECA1593B69AD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>

| I suspect that this is due to a previously-failed build and #17534.
| The workaround is to remove the traces of the half-finished build:
| 
|     rm
| _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.*
| 
| and rerun the build; this may require multiple iterations.

Ugh.  That's not a very happy state of affairs, is it?  It didn't happen with 'make'.  Is it a fundamental problem, or just not yet fixed?

Simon

| -----Original Message-----
| From: Ben Gamari <ben at smart-cactus.org>
| Sent: 09 December 2019 22:28
| To: Simon Peyton Jones <simonpj at microsoft.com>; ghc-devs at haskell.org
| Subject: Re: More failure
| 
| Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:
| 
| > More pain.  I said
| > sh validate --fast --no-clean
| > and got this output
| >
| > Error when running Shake build system:
| >
| >   at action, called at src/Rules.hs:71:19 in main:Rules
| >
| >   at need, called at src/Rules.hs:93:5 in main:Rules
| >
| > * Depends on: _validatebuild/stage1/lib/package.conf.d/base-
| 4.14.0.0.conf
| >
| >   at need, called at src/Rules/Register.hs:113:5 in main:Rules.Register
| >
| > * Depends on: _validatebuild/stage1/libraries/base/build/libHSbase-
| 4.14.0.0-ghc8.11.0.20191201.so
| >
| >   at need, called at src/Rules/Library.hs:146:5 in main:Rules.Library
| >
| > * Depends on:
| _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.dyn_
| o
| >
| >   at &%>, called at src/Rules/Compile.hs:54:7 in main:Rules.Compile
| >
| > * Depends on:
| _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.dyn_
| o
| _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.dyn_
| hi
| >
| >   at error, called at
| src/Development/Shake/Internal/Rules/Files.hs:245:13 in shake-0.18.3-
| 593067565aafb558d09b4352b8abc327d8911a39a0e9abab2804b002b1ae536e:Developm
| ent.Shake.Internal.Rules.Files
| >
| > * Raised the exception:
| >
| > Error, &%> rule failed to produce 1 file (out of 2)
| >
| >
| _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.dyn_
| o
| >
| >
| _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.dyn_
| hi - MISSING
| 
| I suspect that this is due to a previously-failed build and #17534.
| The workaround is to remove the traces of the half-finished build:
| 
|     rm
| _validatebuild/stage1/libraries/base/build/GHC/IO/Handle/Lock/Common.*
| 
| and rerun the build; this may require multiple iterations.
| 
| Cheers,
| 
| - Ben

From simonpj at microsoft.com  Mon Dec  9 23:08:57 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 9 Dec 2019 23:08:57 +0000
Subject: Running test with the stage-1 compiler
In-Reply-To: <87lfrl87fj.fsf@smart-cactus.org>
References: <VI1PR8303MB009360C4415E2872D56327E3AD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
 <87lfrl87fj.fsf@smart-cactus.org>
Message-ID: <VI1PR8303MB0093B847EDD27601263571C2AD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>

|     hadrian/build.cabal.sh --flavor=validate --build-root=_buildvalidate
| --test-compiler=_buildvalidate/stage0/bin/ghc
| 
| Although I'll admit  this is far from convenient. 

I often want to do this in a single testsuite directory, or perhaps just on a single test, like say T15043 in testsuite/tests/typecheck/should_compile.  I'm not validating at all ... I just want to run a test, or a directory-full of tests, with the stage1 compiler.

How can I do that?

Thanks

Simon

| -----Original Message-----
| From: Ben Gamari <ben at smart-cactus.org>
| Sent: 09 December 2019 22:24
| To: Simon Peyton Jones <simonpj at microsoft.com>; ghc-devs at haskell.org
| Subject: Re: Running test with the stage-1 compiler
| 
| Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:
| 
| > Apparently 'sh validate' now uses Hadrian.
| > This has broken one of my workflows:
| >
| >   *   build the stage1 compiler
| >   *   cd testsuite/tests
| >   *   make stage=1
| >
| > That is, run the testsuite with the stage-1 compile.  (This often
| > produces a small test case for a compiler crash, much easier than
| debugging GHC compiling itself.) How do I do this?
| 
| I suspect (but have not tested) that you can accomplish this with
| 
|     hadrian/build.cabal.sh --flavor=validate --build-root=_buildvalidate
| --test-compiler=_buildvalidate/stage0/bin/ghc
| 
| Although I'll admit  this is far from convenient. If you weren't using
| the `validate` script this would just be
| 
|     hadrian/build.cabal.sh --test-compiler=_build/stage0/bin/ghc
| 
| Cheers,
| 
| - Ben

From andrey.mokhov at newcastle.ac.uk  Tue Dec 10 00:57:37 2019
From: andrey.mokhov at newcastle.ac.uk (Andrey Mokhov)
Date: Tue, 10 Dec 2019 00:57:37 +0000
Subject: More failure
Message-ID: <DB7PR07MB5930C7F2E0690A788E40C47DA55B0@DB7PR07MB5930.eurprd07.prod.outlook.com>

Hi Simon,

(Re-sending from the email address that's allowed on the mailing list.)

> Ugh.  That's not a very happy state of affairs, is it?  It didn't happen with 'make'.
> Is it a fundamental problem, or just not yet fixed?

I think this is not a fundamental problem, but the problem of getting dependencies right.

In this case, the complexity comes from the fact that a single invocation of GHC produces a set of files, and which set depends on the command line flags, which are in turn determined dynamically by reading environment settings (specifically, `platformSupportsSharedLibs`).

Such rules are hard to describe precisely, because build systems are tuned to the typical case where we statically know, for every output file, which rule produces it -- recall the Tasks = k -> Maybe Task function from our paper. In this case, we deal with something like k -> f (Maybe Task) instead, i.e. with `f` around the Maybe.

The Make build system happens to do the right thing, somehow. I believe we should be able to express the same logic in Shake, but it's not easy.

(I never really had a chance to look at dynamic builds, since they are not supported on Windows. I guess I should finally find a Linux box for Hadrian.)

Cheers,
Andrey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191210/a7ced6fa/attachment.html>

From lonetiger at gmail.com  Tue Dec 10 07:47:33 2019
From: lonetiger at gmail.com (Phyx)
Date: Tue, 10 Dec 2019 07:47:33 +0000
Subject: More failure
In-Reply-To: <DB7PR07MB5930C7F2E0690A788E40C47DA55B0@DB7PR07MB5930.eurprd07.prod.outlook.com>
References: <DB7PR07MB5930C7F2E0690A788E40C47DA55B0@DB7PR07MB5930.eurprd07.prod.outlook.com>
Message-ID: <CABHktDXXfuSkD1edy7w4p_F1L7suAhVipqQH+o5py08OA-xNkA@mail.gmail.com>

Hi Andrey,

I'm not sure what the original issue here is (should probably find the
original message) but

> The Make build system happens to do the right thing, somehow. I believe
we should be able to express the same logic in Shake, but it's not easy.

I believe this typically works because GCC and GHC support dumping the
dependencies that a command would have caused to a file. So your dynamic
dependencies don't matter as their static to the build system after this
invocation.

See
https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/separate_compilation.html#dependency-generation

These Compilers are able to dump out make rules which enabled better
dependency handling..

Kind regards,
Tamar

Sent from my Mobile

On Tue, Dec 10, 2019, 00:58 Andrey Mokhov <andrey.mokhov at newcastle.ac.uk>
wrote:

> Hi Simon,
>
>
>
> (Re-sending from the email address that’s allowed on the mailing list.)
>
>
>
> > Ugh.  That's not a very happy state of affairs, is it?  It didn't happen
> with 'make'.
>
> > Is it a fundamental problem, or just not yet fixed?
>
>
>
> I think this is not a fundamental problem, but the problem of getting
> dependencies right.
>
>
>
> In this case, the complexity comes from the fact that a single invocation
> of GHC produces a set of files, and which set depends on the command line
> flags, which are in turn determined dynamically by reading environment
> settings (specifically, `platformSupportsSharedLibs`).
>
>
>
> Such rules are hard to describe precisely, because build systems are tuned
> to the typical case where we statically know, for every output file, which
> rule produces it -- recall the Tasks = k -> Maybe Task function from our
> paper. In this case, we deal with something like k -> f (Maybe Task)
> instead, i.e. with `f` around the Maybe.
>
>
>
> The Make build system happens to do the right thing, somehow. I believe we
> should be able to express the same logic in Shake, but it's not easy.
>
>
>
> (I never really had a chance to look at dynamic builds, since they are not
> supported on Windows. I guess I should finally find a Linux box for
> Hadrian.)
>
>
>
> Cheers,
>
> Andrey
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191210/288b7e80/attachment.html>

From alp at well-typed.com  Tue Dec 10 08:54:16 2019
From: alp at well-typed.com (Alp Mestanogullari)
Date: Tue, 10 Dec 2019 09:54:16 +0100
Subject: Running test with the stage-1 compiler
In-Reply-To: <87lfrl87fj.fsf@smart-cactus.org>
References: <VI1PR8303MB009360C4415E2872D56327E3AD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
 <87lfrl87fj.fsf@smart-cactus.org>
Message-ID: <93f00854-704b-e937-52ac-2dbb6e0bed94@well-typed.com>

For what it's worth, there's a "shortcut" to save a few keystrokes: 
--test-compiler=stage1

But this broke at some point in the past few months, as documented in 
#17528. I am however working on a patch to make this work.

On 09/12/2019 23:23, Ben Gamari wrote:
> Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:
>
>> Apparently 'sh validate' now uses Hadrian.
>> This has broken one of my workflows:
>>
>>    *   build the stage1 compiler
>>    *   cd testsuite/tests
>>    *   make stage=1
>>
>> That is, run the testsuite with the stage-1 compile.  (This often produces a small test case for a compiler crash, much easier than debugging GHC compiling itself.)
>> How do I do this?
> I suspect (but have not tested) that you can accomplish this with
>
>      hadrian/build.cabal.sh --flavor=validate --build-root=_buildvalidate  --test-compiler=_buildvalidate/stage0/bin/ghc
>
> Although I'll admit  this is far from convenient. If you weren't using
> the `validate` script this would just be
>
>      hadrian/build.cabal.sh --test-compiler=_build/stage0/bin/ghc
>
> Cheers,
>
> - Ben
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-- 
Alp Mestanogullari, Haskell Consultant
Well-Typed LLP, https://www.well-typed.com/

Registered in England and Wales, OC335890
118 Wymering Mansions, Wymering Road, London, W9 2NF, England

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191210/da655bd1/attachment.html>

From andrey.mokhov at newcastle.ac.uk  Tue Dec 10 10:04:44 2019
From: andrey.mokhov at newcastle.ac.uk (Andrey Mokhov)
Date: Tue, 10 Dec 2019 10:04:44 +0000
Subject: More failure
In-Reply-To: <CABHktDXXfuSkD1edy7w4p_F1L7suAhVipqQH+o5py08OA-xNkA@mail.gmail.com>
References: <DB7PR07MB5930C7F2E0690A788E40C47DA55B0@DB7PR07MB5930.eurprd07.prod.outlook.com>,
 <CABHktDXXfuSkD1edy7w4p_F1L7suAhVipqQH+o5py08OA-xNkA@mail.gmail.com>
Message-ID: <DB7PR07MB5930E3C1D9934D7E2F096871A55B0@DB7PR07MB5930.eurprd07.prod.outlook.com>

Hi Tamar,

I think the difficulty here is more with dynamic *outputs* rather than dynamic inputs/dependencies.

We do not statically know which of the following three alternatives holds:

  *   `*.dyn_o/hi` files are not built at all.
  *   `*.dyn_o/hi` files are built via a separate execution of GHC.
  *   `*.dyn_o/hi` files are built together with `*.o/hi` files, in a single execution of GHC with `-dynamic-too`.

Here is the current implementation:

https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/src/Rules/Compile.hs#L51-69

I believe the last person looking into this was James Foster, so CC-ing to him in case he has any insights.

Cheers,
Andrey

________________________________
From: Phyx <lonetiger at gmail.com>
Sent: 10 December 2019 07:47
To: Andrey Mokhov <andrey.mokhov at newcastle.ac.uk>
Cc: Simon Peyton-Jones (simonpj at microsoft.com) <simonpj at microsoft.com>; Ben Gamari <ben at well-typed.com>; ghc-devs at haskell.org <ghc-devs at haskell.org>
Subject: Re: More failure

Hi Andrey,

I'm not sure what the original issue here is (should probably find the original message) but

> The Make build system happens to do the right thing, somehow. I believe we should be able to express the same logic in Shake, but it's not easy.

I believe this typically works because GCC and GHC support dumping the dependencies that a command would have caused to a file. So your dynamic dependencies don't matter as their static to the build system after this invocation.

See
https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/separate_compilation.html#dependency-generation

These Compilers are able to dump out make rules which enabled better dependency handling..

Kind regards,
Tamar

Sent from my Mobile

On Tue, Dec 10, 2019, 00:58 Andrey Mokhov <andrey.mokhov at newcastle.ac.uk<mailto:andrey.mokhov at newcastle.ac.uk>> wrote:

Hi Simon,



(Re-sending from the email address that’s allowed on the mailing list.)



> Ugh.  That's not a very happy state of affairs, is it?  It didn't happen with 'make'.

> Is it a fundamental problem, or just not yet fixed?



I think this is not a fundamental problem, but the problem of getting dependencies right.



In this case, the complexity comes from the fact that a single invocation of GHC produces a set of files, and which set depends on the command line flags, which are in turn determined dynamically by reading environment settings (specifically, `platformSupportsSharedLibs`).



Such rules are hard to describe precisely, because build systems are tuned to the typical case where we statically know, for every output file, which rule produces it -- recall the Tasks = k -> Maybe Task function from our paper. In this case, we deal with something like k -> f (Maybe Task) instead, i.e. with `f` around the Maybe.



The Make build system happens to do the right thing, somehow. I believe we should be able to express the same logic in Shake, but it's not easy.



(I never really had a chance to look at dynamic builds, since they are not supported on Windows. I guess I should finally find a Linux box for Hadrian.)



Cheers,

Andrey

_______________________________________________
ghc-devs mailing list
ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191210/0f242bf4/attachment.html>

From simonpj at microsoft.com  Tue Dec 10 11:12:40 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Tue, 10 Dec 2019 11:12:40 +0000
Subject: Total failure
In-Reply-To: <87a7818lgm.fsf@smart-cactus.org>
References: <VI1PR8303MB009308906619379C3E296DEBAD580@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
 <87a7818lgm.fsf@smart-cactus.org>
Message-ID: <VI1PR8303MB0093A583866F72132815A3B6AD5B0@VI1PR8303MB0093.EURPRD83.prod.outlook.com>

|  I think I see what is going on here. If you run `make WERROR=""` for a
|  while to get past DynFlags does the build finish?

Strange.  This would switch from a failed *Hadrian* build to using *make*. Is that really what you intend?

S

|  -----Original Message-----
|  From: Ben Gamari <ben at smart-cactus.org>
|  Sent: 09 December 2019 17:20
|  To: Simon Peyton Jones <simonpj at microsoft.com>; ghc-devs at haskell.org
|  Subject: Re: Total failure
|  
|  Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:
|  
|  > Bad things are happening.   I can't build GHC at all!
|  > I'm trying 'sh validate -fast' on a clean build of a master checkout
|  > HEAD detached at 3a96a0b6db
|  > (this is the master checkout from which !2276 branches off)
|  > But alas I get this
|  >
|  I think I see what is going on here. If you run `make WERROR=""` for a
|  while to get past DynFlags does the build finish?
|  
|  Cheers,
|  
|  - Ben

From rae at richarde.dev  Tue Dec 10 17:14:13 2019
From: rae at richarde.dev (Richard Eisenberg)
Date: Tue, 10 Dec 2019 17:14:13 +0000
Subject: GitLab # trick
Message-ID: <A34CE7F2-4C7C-4678-BE7F-4F4E7B18E452@richarde.dev>

Today I learned that if you type # in an input field on GitLab, you can start typing the description of a ticket, and GitLab will find the number of the (open) ticket for you. For example, I typed #zap and got #16775, just what I was looking for. (That I would get this was very clear in the UI -- no surprises here.)

Maybe others already knew this, but this was just so snazzy and ergonomic, I had to write.

Richard

From mail at joachim-breitner.de  Wed Dec 11 10:10:10 2019
From: mail at joachim-breitner.de (Joachim Breitner)
Date: Wed, 11 Dec 2019 11:10:10 +0100
Subject: Request for Nominations to the GHC Steering Committee
Message-ID: <c5ff9334a703bbad7c68860798686ca27c4921cb.camel@joachim-breitner.de>

Dear Haskell community,

the GHC Steering committee is seeking nominations for one new member.

The committee scrutinizes, nitpicks, improves, weights and eventually
accepts or rejects proposals that extend or change the language
supported by GHC and other (public-facing) aspects of GHC.
Our processes are described at
   https://github.com/ghc-proposals/ghc-proposals
which is also the GitHub repository where proposals are proposed.

We are looking for a member who has the ability 
 * to understand such language extension proposals,
 * to find holes and missing corner cases in the specifications,
 * foresee the interaction with other language features and 
   specifications,
 * comment constructively and improve the proposals,
 * judge the cost/benefit ratio and
 * finally come to a justifiable conclusion.

We look for committee members who have some of these properties:
 * have substantial experience in writing Haskell applications or
   libraries, which they can use to inform judgements about the
   utility or otherwise of proposed features,
 * have made active contributions to the Haskell community, for
   some time,
 * have expertise in language design and implementation, in either
   Haskell or related languages, which they can share with us.

The committee’s work requires a small, but non-trivial amount of time,
especially when you are assigned a proposal for shepherding. We
estimate the workload to be around 2 hours per week, and our process
works best if members usually respond to technical emails within 1-2
weeks (within days is even better). Please keep that in mind if your
email inbox is already overflowing.

The GHC developers themselves are already well represented already.  We
seek Haskell _users_ more than GHC hackers.

There is no shortage of people who are eager to get fancy new
features into the language, both in the committee and the wider
community. But each new feature imposes a cost, to implement, to learn,
(particularly) through its unexpected interaction with other features.
We need to strike a balance, one that encourages innovation (as GHC
always has) while still making Haskell attractive for real-world
production use and for teaching. We therefore explicitly invite
“conservative” members of the community to join the committee.

To make a nomination, please send an email to me (as the committee
secretary) at mail at joachim-breitner.de until January 2nd. I will
distribute the nominations among the committee, and we will keep the
nominations and our deliberations private.

We explicitly encourage self-nominations. You can nominate others, but
please obtain their explicit consent to do so. (We don’t want to choose
someone who turns out to be unable to serve.)

On behalf of the committee,
Joachim Breitner

-- 
Joachim Breitner
  mail at joachim-breitner.de
  http://www.joachim-breitner.de/


From ben at well-typed.com  Wed Dec 11 18:27:50 2019
From: ben at well-typed.com (Ben Gamari)
Date: Wed, 11 Dec 2019 13:27:50 -0500
Subject: [ANNOUNCE] Glasgow Haskell Compiler 8.10.1-alpha2 released
Message-ID: <8736dq90pr.fsf@smart-cactus.org>

Hello all,

The GHC team is happy to announce the availability of the first alpha
release in the GHC 8.10 series. Source and binary distributions are
available at the usual place:

    https://downloads.haskell.org/ghc/8.10.1-alpha2/

GHC 8.10.1 will bring a number of new features including:

 * The new UnliftedNewtypes extension allowing newtypes around unlifted
   types.

 * The new StandaloneKindSignatures extension allows users to give
   top-level kind signatures to type, type family, and class
   declarations.

 * A new warning, -Wderiving-defaults, to draw attention to ambiguous
   deriving clauses

 * A number of improvements in code generation, including changes

 * A new GHCi command, :instances, for listing the class instances
   available for a type.

 * An upgraded Windows toolchain lifting the MAX_PATH limitation

 * A new, low-latency garbage collector.

 * Improved support profiling, including support for sending profiler
   samples to the eventlog, allowing correlation between the profile and
   other program events

This release is the second alpha of the 8.10 pre-release cycle. The
release will be a release candidate in roughly three weeks. If all goes
well the final release will be cut roughly two weeks after the
candidate, in mid-January.

This being an alpha release, there are a few issues that are still
outstanding:

 * The new Alpine Linux binary distribution is not present due to an
   apparent correctness issue [1]; any help Alpine users can offer here
   would be greatly appreciated.

 * We have yet to sort out compliance with Apple's notarization
   requirements [2] which will be likely be necessary for users of
   macOS Catalina.

   However, until this has been sorted out Catalina users can exempt the
   current macOS binary distribution from the notarization requirement
   themselves by running `xattr -cr .` on the unpacked tree before
   running `make install`.

 * There is one remaining non-regression correctness issue which we plan
   to fix for the final 8.10.1 but that is not fixed in this release.

Please do test this release and let us know if you encounter any other
issues.

Cheers,

- Ben


[1] https://gitlab.haskell.org/ghc/ghc/issues/17508
[2] https://gitlab.haskell.org/ghc/ghc/issues/17418
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191211/2914c232/attachment.sig>

From takenobu.hs at gmail.com  Thu Dec 12 12:23:33 2019
From: takenobu.hs at gmail.com (Takenobu Tani)
Date: Thu, 12 Dec 2019 21:23:33 +0900
Subject: [ANNOUNCE] Glasgow Haskell Compiler 8.10.1-alpha2 released
In-Reply-To: <8736dq90pr.fsf@smart-cactus.org>
References: <8736dq90pr.fsf@smart-cactus.org>
Message-ID: <CAPB_Nvy1p1DwUMxjeKCfCAx19JR1H3FHyy35pURJ=jjDOZx2vw@mail.gmail.com>

Thanks for the release.

Will deb9-linux binary be released in the future?
Or deb10-linux version only?

Perhaps ubuntu18.04LTS users need deb9-linux binary.

Regards,
Takenobu

On Thu, Dec 12, 2019 at 3:28 AM Ben Gamari <ben at well-typed.com> wrote:
>
> Hello all,
>
> The GHC team is happy to announce the availability of the first alpha
> release in the GHC 8.10 series. Source and binary distributions are
> available at the usual place:
>
>     https://downloads.haskell.org/ghc/8.10.1-alpha2/
>
> GHC 8.10.1 will bring a number of new features including:
>
>  * The new UnliftedNewtypes extension allowing newtypes around unlifted
>    types.
>
>  * The new StandaloneKindSignatures extension allows users to give
>    top-level kind signatures to type, type family, and class
>    declarations.
>
>  * A new warning, -Wderiving-defaults, to draw attention to ambiguous
>    deriving clauses
>
>  * A number of improvements in code generation, including changes
>
>  * A new GHCi command, :instances, for listing the class instances
>    available for a type.
>
>  * An upgraded Windows toolchain lifting the MAX_PATH limitation
>
>  * A new, low-latency garbage collector.
>
>  * Improved support profiling, including support for sending profiler
>    samples to the eventlog, allowing correlation between the profile and
>    other program events
>
> This release is the second alpha of the 8.10 pre-release cycle. The
> release will be a release candidate in roughly three weeks. If all goes
> well the final release will be cut roughly two weeks after the
> candidate, in mid-January.
>
> This being an alpha release, there are a few issues that are still
> outstanding:
>
>  * The new Alpine Linux binary distribution is not present due to an
>    apparent correctness issue [1]; any help Alpine users can offer here
>    would be greatly appreciated.
>
>  * We have yet to sort out compliance with Apple's notarization
>    requirements [2] which will be likely be necessary for users of
>    macOS Catalina.
>
>    However, until this has been sorted out Catalina users can exempt the
>    current macOS binary distribution from the notarization requirement
>    themselves by running `xattr -cr .` on the unpacked tree before
>    running `make install`.
>
>  * There is one remaining non-regression correctness issue which we plan
>    to fix for the final 8.10.1 but that is not fixed in this release.
>
> Please do test this release and let us know if you encounter any other
> issues.
>
> Cheers,
>
> - Ben
>
>
> [1] https://gitlab.haskell.org/ghc/ghc/issues/17508
> [2] https://gitlab.haskell.org/ghc/ghc/issues/17418
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From ben at well-typed.com  Thu Dec 12 19:04:54 2019
From: ben at well-typed.com (Ben Gamari)
Date: Thu, 12 Dec 2019 14:04:54 -0500
Subject: new llvm IR callbr instruction useful for proc-point splitting?
In-Reply-To: <VI1PR8303MB009307E88147C7160DF78FF3AD5D0@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
References: <874l4qzlny.fsf@smart-cactus.org>
 <CAB-d4A6QyumkF3LLc8rk=NQ4jTVDOqXmo5AWOOQ2=qZr6E8JPQ@mail.gmail.com>
 <877e9dy89j.fsf@smart-cactus.org>
 <CAB-d4A5S=Og2-1N07LqYbg64UEX1FWMWXKn_GeCQguZ2Gin6AA@mail.gmail.com>
 <a876f857876430b7b089ef6a0241aaeee776c45a.camel@farvard.in>
 <87h82g82r8.fsf@smart-cactus.org>
 <5a27d1a33100163c0ba11285c2022105b9929cf6.camel@farvard.in>
 <VI1PR8303MB009307E88147C7160DF78FF3AD5D0@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
Message-ID: <87tv6574by.fsf@smart-cactus.org>

Simon Peyton Jones <simonpj at microsoft.com> writes:

> Thanks Kavon.
>
> What news of CpsCall in LLVM?  That seems crucial.
>
> I didn’t know of an effort to use a different SP register. I seriously
> doubt that, with GHC’s use of zillions of small, heap-allocated stacks
> that move around during GHC, we’ll be able to leverage profiling or
> debugging tools in a serious way. But perhaps I need education.
>
I don't see there being a problem here:

 * DWARF debugging information is expressive enough to capture GHC's
   underflow frame concept, so unwinding across stack
   chunks shouldn't be problematic.

 * The fact that the stacks move shouldn't be a problem since
   either:

    * You are using gdb, which must pause the process to look at the stack
      anyways

    * You are using statistical sampling profiling (e.g. perf on Linux)
      which pauses the thread being sampled while capturing the stack
      state.

      If you sample during a GC then you won't capture any Haskell stack
      chunks so the fact that stacks might move isn't problematic; if
      you sample during mutator execution then the stack is guaranteed to
      be stationary.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191212/f315290c/attachment.sig>

From ben at well-typed.com  Thu Dec 12 20:11:14 2019
From: ben at well-typed.com (Ben Gamari)
Date: Thu, 12 Dec 2019 15:11:14 -0500
Subject: [ANNOUNCE] GHC 8.8.2-rc1 is now available
Message-ID: <87lfrh719c.fsf@smart-cactus.org>


Hello everyone,

The GHC team is proud to announce the first release candidate of GHC
8.8.2. The source distribution, binary distributions, and documentation
are available at

    https://downloads.haskell.org/~ghc/8.8.2-rc1

Release notes are also available [1].

This release fixes a handful of issues affecting 8.8.1:

 - A bug (#17088) in the compacting garbage collector resulting in
   segmentations faults under specific circumstances. Note that this may
   affect user programs even if they did not explicitly request the
   compacting GC (using the -c RTS flag) since GHC may fallback to
   compacting collection during times of high memory pressure.
   
 - A code generation bug (#17334) resulting in GHC panics has been
   fixed.

 - A bug in the `process` library causing builds using `hsc2hs` to fail
   non-deterministically on Windows has been fixed (Trac #17480)

 - A typechecker bug (#12088) resulting in programs being unexpectedly
   rejected has been fixed.

 - A bug in the implementation of compact normal forms resulting in
   segmentation faults in some uses (#17044) has been fixed.

 - A bug causing GHC to incorrectly complain about incompatible LLVM
   versions when using LLVM 7.0.1 has been fixed (#16990).

As always, if anything looks amiss do let us know.

Happy compiling!

Cheers,

- Ben


[1] https://downloads.haskell.org/ghc/8.8.2-rc1/docs/html/users_guide/8.8.2-notes.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191212/8dfe7020/attachment.sig>

From simon.jakobi at googlemail.com  Thu Dec 12 23:20:49 2019
From: simon.jakobi at googlemail.com (Simon Jakobi)
Date: Fri, 13 Dec 2019 00:20:49 +0100
Subject: [ANNOUNCE] GHC 8.8.2-rc1 is now available
In-Reply-To: <87lfrh719c.fsf@smart-cactus.org>
References: <87lfrh719c.fsf@smart-cactus.org>
Message-ID: <CAGtp2Sg_xg034gjUVjCnQ+N9v97O=3C=HdvN47R8n8wefvShBQ@mail.gmail.com>

Thanks for all the bug fixes! :)

>  - A typechecker bug (#12088) resulting in programs being unexpectedly
>    rejected has been fixed.

https://gitlab.haskell.org/ghc/ghc/issues/12088 is still open and
doesn't appear to have seen that much recent activity. Is this the
right ticket?

Cheers,
Simon

From csaba.hruska at gmail.com  Sun Dec 15 12:15:49 2019
From: csaba.hruska at gmail.com (Csaba Hruska)
Date: Sun, 15 Dec 2019 13:15:49 +0100
Subject: Typo in closure name in rts/Prelude.h
Message-ID: <CANjChEpTmVE0rh4nHa5CxgJX5Jm-oecaXNgm6axLzjvuAN=__A@mail.gmail.com>

Hello,

Could you confirm if *base_GHCziWeak_rrts/win32/libHSbase.def_closure*
<https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Prelude.h#L30> is a
typo in rts/Prelude.h?
I guess it is a typo because *base_GHCziWeak_runFinalizerBatch_closure*
does exists in base/GHC/Weak.hs.
<https://gitlab.haskell.org/ghc/ghc/blob/master/libraries/base/GHC/Weak.hs#L144>

The following files have the wrong name:

   - rts/package.conf.in
   - rts/rts.cabal.in
   - rts/Prelude.h
   - rts/win32/libHSbase.def

Regards,
Csaba
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191215/b3c602a2/attachment.html>

From csaba.hruska at gmail.com  Sun Dec 15 12:17:49 2019
From: csaba.hruska at gmail.com (Csaba Hruska)
Date: Sun, 15 Dec 2019 13:17:49 +0100
Subject: Typo in closure name in rts/Prelude.h
In-Reply-To: <CANjChEpTmVE0rh4nHa5CxgJX5Jm-oecaXNgm6axLzjvuAN=__A@mail.gmail.com>
References: <CANjChEpTmVE0rh4nHa5CxgJX5Jm-oecaXNgm6axLzjvuAN=__A@mail.gmail.com>
Message-ID: <CANjChErffS=PTA9bwDO2LDrs-Sy7tQtpxV95b6tZSPkZzSnNmA@mail.gmail.com>

Sorry, I messed up the first link. It meant to be
*base_GHCziWeak_runFinalizzerBatch_closure*
<https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Prelude.h#L30>

On Sun, Dec 15, 2019 at 1:15 PM Csaba Hruska <csaba.hruska at gmail.com> wrote:

> Hello,
>
> Could you confirm if *base_GHCziWeak_rrts/win32/libHSbase.def_closure*
> <https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Prelude.h#L30> is a
> typo in rts/Prelude.h?
> I guess it is a typo because *base_GHCziWeak_runFinalizerBatch_closure*
> does exists in base/GHC/Weak.hs.
> <https://gitlab.haskell.org/ghc/ghc/blob/master/libraries/base/GHC/Weak.hs#L144>
>
> The following files have the wrong name:
>
>    - rts/package.conf.in
>    - rts/rts.cabal.in
>    - rts/Prelude.h
>    - rts/win32/libHSbase.def
>
> Regards,
> Csaba
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191215/9f0059e2/attachment.html>

From ggreif at gmail.com  Sun Dec 15 13:49:35 2019
From: ggreif at gmail.com (Gabor Greif)
Date: Sun, 15 Dec 2019 14:49:35 +0100
Subject: Typo in closure name in rts/Prelude.h
In-Reply-To: <CANjChErffS=PTA9bwDO2LDrs-Sy7tQtpxV95b6tZSPkZzSnNmA@mail.gmail.com>
References: <CANjChEpTmVE0rh4nHa5CxgJX5Jm-oecaXNgm6axLzjvuAN=__A@mail.gmail.com>
 <CANjChErffS=PTA9bwDO2LDrs-Sy7tQtpxV95b6tZSPkZzSnNmA@mail.gmail.com>
Message-ID: <CAK-hX0mAbe+RNePX0O0VaCpR6qBKcgr-qTddUUL4Mm0qu=WANw@mail.gmail.com>

No, this is not a typo, it it "z-encoding".
(http://hackage.haskell.org/package/zenc-0.1.1/docs/Text-Encoding-Z.html)

Cheers,

    Gabor

On 12/15/19, Csaba Hruska <csaba.hruska at gmail.com> wrote:
> Sorry, I messed up the first link. It meant to be
> *base_GHCziWeak_runFinalizzerBatch_closure*
> <https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Prelude.h#L30>
>
> On Sun, Dec 15, 2019 at 1:15 PM Csaba Hruska <csaba.hruska at gmail.com>
> wrote:
>
>> Hello,
>>
>> Could you confirm if *base_GHCziWeak_rrts/win32/libHSbase.def_closure*
>> <https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Prelude.h#L30> is a
>> typo in rts/Prelude.h?
>> I guess it is a typo because *base_GHCziWeak_runFinalizerBatch_closure*
>> does exists in base/GHC/Weak.hs.
>> <https://gitlab.haskell.org/ghc/ghc/blob/master/libraries/base/GHC/Weak.hs#L144>
>>
>> The following files have the wrong name:
>>
>>    - rts/package.conf.in
>>    - rts/rts.cabal.in
>>    - rts/Prelude.h
>>    - rts/win32/libHSbase.def
>>
>> Regards,
>> Csaba
>>
>

From csaba.hruska at gmail.com  Sun Dec 15 14:10:17 2019
From: csaba.hruska at gmail.com (Csaba Hruska)
Date: Sun, 15 Dec 2019 15:10:17 +0100
Subject: Typo in closure name in rts/Prelude.h
In-Reply-To: <CAK-hX0mAbe+RNePX0O0VaCpR6qBKcgr-qTddUUL4Mm0qu=WANw@mail.gmail.com>
References: <CANjChEpTmVE0rh4nHa5CxgJX5Jm-oecaXNgm6axLzjvuAN=__A@mail.gmail.com>
 <CANjChErffS=PTA9bwDO2LDrs-Sy7tQtpxV95b6tZSPkZzSnNmA@mail.gmail.com>
 <CAK-hX0mAbe+RNePX0O0VaCpR6qBKcgr-qTddUUL4Mm0qu=WANw@mail.gmail.com>
Message-ID: <CANjChEpFZwJ3Eh=ERV5AHOtFpRgWB20RJ=G2-Yc+CZkorwjr_Q@mail.gmail.com>

Thanks! Totally makes sense.
Now I may look like a noob. :D

On Sun, Dec 15, 2019 at 2:49 PM Gabor Greif <ggreif at gmail.com> wrote:

> No, this is not a typo, it it "z-encoding".
> (http://hackage.haskell.org/package/zenc-0.1.1/docs/Text-Encoding-Z.html)
>
> Cheers,
>
>     Gabor
>
> On 12/15/19, Csaba Hruska <csaba.hruska at gmail.com> wrote:
> > Sorry, I messed up the first link. It meant to be
> > *base_GHCziWeak_runFinalizzerBatch_closure*
> > <https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Prelude.h#L30>
> >
> > On Sun, Dec 15, 2019 at 1:15 PM Csaba Hruska <csaba.hruska at gmail.com>
> > wrote:
> >
> >> Hello,
> >>
> >> Could you confirm if *base_GHCziWeak_rrts/win32/libHSbase.def_closure*
> >> <https://gitlab.haskell.org/ghc/ghc/blob/master/rts/Prelude.h#L30> is a
> >> typo in rts/Prelude.h?
> >> I guess it is a typo because *base_GHCziWeak_runFinalizerBatch_closure*
> >> does exists in base/GHC/Weak.hs.
> >> <
> https://gitlab.haskell.org/ghc/ghc/blob/master/libraries/base/GHC/Weak.hs#L144
> >
> >>
> >> The following files have the wrong name:
> >>
> >>    - rts/package.conf.in
> >>    - rts/rts.cabal.in
> >>    - rts/Prelude.h
> >>    - rts/win32/libHSbase.def
> >>
> >> Regards,
> >> Csaba
> >>
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191215/313b18c0/attachment.html>

From simonpj at microsoft.com  Thu Dec 19 14:30:55 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Thu, 19 Dec 2019 14:30:55 +0000
Subject: Perf metrics
Message-ID: <VI1PR8303MB0093F52C2A4E12C6D26925BEAD520@VI1PR8303MB0093.EURPRD83.prod.outlook.com>

When I run `sh validate -legacy`, towards the end I see stuff like this

Performance Metrics (test environment: local):



Conversions(normal)       runtime/bytes allocated                        107696.000

DeriveNull(normal)        runtime/bytes allocated                     112050960.000

InlineArrayAlloc(normal)  runtime/bytes allocated                    1600041088.000

InlineByteArrayAlloc(normal)  runtime/bytes allocated                    1440041088.000

InlineCloneArrayAlloc(normal)  runtime/bytes allocated                    1600041248.000

ManyAlternatives(normal)  compile_time/bytes allocated                840985728.000

ManyConstructors(normal)  compile_time/bytes allocated               4540766560.000

MethSharing(normal)       runtime/peak_megabytes_allocated                    2.000

MethSharing(normal)       runtime/bytes allocated                     480098136.000

MultiLayerModules(normal)  compile_time/bytes allocated               5856970504.000

...
It is intermingled with other apparently unrelated output.
What should I conclude form this?  Is it good or bad?  By what amount have these figures changed, and relative to what?
How can I run a single perf test?
Thanks
Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191219/b5597409/attachment.html>

From matthewtpickering at gmail.com  Thu Dec 19 16:11:43 2019
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Thu, 19 Dec 2019 16:11:43 +0000
Subject: Invitation to GHC Week
In-Reply-To: <CALuQ0m-8EzVN=cu=gEZw-5HHPwufc2ZmfMJ6aLXLsd36mWYBRQ@mail.gmail.com>
References: <CALuQ0m-8EzVN=cu=gEZw-5HHPwufc2ZmfMJ6aLXLsd36mWYBRQ@mail.gmail.com>
Message-ID: <CALuQ0m9rDidsW2259mhu2uBQrmt5yJbuG5yNhffCKZbsT5aMXw@mail.gmail.com>

Hi everyone,

I put up a list of confirmed participants on the website:
https://andreaspk.github.io/ghc-week

Looking forward to seeing everyone in spring.

Cheers,

Matt

----------------

Alexandre Baldé
Alina Banerjee
Joachim Breitner
Zubin Duggal
John Ericson
Ben Gamari
Krzysztof Gogolewski
Sebastian Graf
Daniel Gröber
Sylvain HENRY
Csaba Hruska
Csongor Kiss
Andreas Klebinger
Luke Lau
Andres Löh
Leif Metcalf
Artem Ohanjanyan
Matthew Pickering
Péter Dávid Podlovics
Bryan Richter
Vaibhav Sagar
Hannes Siebenhandl
Luite Stegeman
Sven Tennie
David van Balen
Jan van Brügge
Jamie Willis
Ningning Xie

On Wed, Oct 16, 2019 at 3:13 PM Matthew Pickering
<matthewtpickering at gmail.com> wrote:
>
> Dear all,
>
> I am happy to announce GHC Week, a developers meeting focusing on GHC,
> organized by myself and AndreasK. We are renting a big holiday cottage
> in rural Belgium during March so that participants can hack on GHC
> uninterrupted for a whole week. The place has a large number of beds
> and a nice open living area for collaboration and relaxation.
>
> Dates: 13th - 20th March 2020
> Location: Annevoie, Belgium
>
> If you are interested in attending GHC Week then please consult the
> linked webpage which contains information about how to attend. There
> are only a limited number of spaces and we already have a number of
> confirmed attendees. Hopefully no one will be disappointed but if all
> 700 subscribers to ghc-devs want to attend there will not be enough
> space!
>
> We would particularly like to encourage undergraduate students who are
> interested in working on GHC to attend GHC Week. Thanks to our
> sponsors, Haskell.org and Well-Typed we are in a position to not only
> make the event itself very affordable, but also to cover travel and
> accommodation costs for those in need.
>
> This is an exceptional opportunity to learn about GHC from experienced
> developers. So if you can, make sure you don’t miss it!
>
> For more details please consult the following page:
>
> https://andreaspk.github.io/ghc-week
>
> Cheers,
>
> Matt & Andreas

From simon.jakobi at googlemail.com  Thu Dec 19 16:20:28 2019
From: simon.jakobi at googlemail.com (Simon Jakobi)
Date: Thu, 19 Dec 2019 17:20:28 +0100
Subject: haskell/bytestring stalled?
In-Reply-To: <CAOFsArtTQzKj1tBV2v7Ph+hmAGRsczM3uautw9QuOJFx2jQyLw@mail.gmail.com>
References: <CAATCwU=btA_sYrFkMkuzy1gq87bc1nfddfN3T3LLG1f2c4afxQ@mail.gmail.com>
 <CAOFsArvBG2ej6AJZNhq8NOqVcPw701vaBExmBnJUf28f=cTOvA@mail.gmail.com>
 <CAGtp2Sg9cJO=P4q-8buDDaQTyU0iCk2WiGR-OjYsz8-dqdehwg@mail.gmail.com>
 <CAOFsArtTQzKj1tBV2v7Ph+hmAGRsczM3uautw9QuOJFx2jQyLw@mail.gmail.com>
Message-ID: <CAGtp2Shs=FCmTypY9J1Xnq50_-uBDyKA81Z_f68SabFzNVB8Gg@mail.gmail.com>

Hi Herbert,

I apologize for the overly harsh and accusatory tone in my last email.
After looking through the existing PRs and issues, I was admittedly
somewhat disappointed over the current situation with bytestring
issues and PRs, but I shouldn't have expressed my feelings in this
way.

Also, thanks a lot for responding so quickly today to my many comments
on issues and PRs.

To be clear, I'm not interested in putting blame on you or Duncan or
anyone else. I do however strongly believe that bytestring needs a
more responsive maintainer:

1. Contributors IMO deserve to get a response within weeks or ideally days.

2. bytestring users (and that transitively includes all GHC users)
could benefit massively if some of the current PRs were finally
merged. For example in [#175], Sylvain Henry reports improvements of
**11%** for some compilation tasks when he used the PR to build GHC.

And I admittedly don't want to ask you, Herbert, to lead the effort of
clearing up the maintenance backlog in bytestring. Of course, _any_
Haskell project you're involved in benefits from your massive
experience! However, I feel that since your involvement is so crucial
in so many other essential projects in the Haskell ecosystem, if you
would shift your focus towards bytestring, these other projects would
suffer! I also believe that projects like cabal or Hackage need your
domain knowledge more dearly than bytestring does. So to ask you to
allocate more time to bytestring, would feel a bit like a zero-sum
game, or possibly a negative-sum game.

A second, more personal reason, why I would prefer to see a different
maintainer for bytestring, is your preference for coordinating things
in private communication. The many PRs and issues in the bytestring
project, and this email thread demonstrate that there are a lot of
people who would like to be involved in improving bytestring. Private
coordination would exclude most of them, and make it more difficult to
make use of all the help that this project currently needs. I
understand that your preference for private communication is related
to your experiences with previous discussions over Haskell projects,
and I am sorry that it has come so far, but I believe that a more open
and transparent style of communication is more healthy.

So, while I'm very grateful for your work on bytestring, and while I
hope that you stay involved in the project, I hope that that someone
else can take over official maintainer duties.

Now I am not sure who could take on that job. Clearly we need someone
whom Duncan trusts. That's why I wonder whether someone involved in
the performance work on GHC could help. Ultimately I don't think that
their time involvement would need to be very high – good judgment and
responsiveness seem more important to me. I'm CC'ing the ghc-devs
list, hoping that someone with the right experience and availability
would come forward.

Thanks,
Simon

[#175] https://github.com/haskell/bytestring/pull/175#issuecomment-567233984

From david.feuer at gmail.com  Thu Dec 19 16:35:47 2019
From: david.feuer at gmail.com (David Feuer)
Date: Thu, 19 Dec 2019 11:35:47 -0500
Subject: haskell/bytestring stalled?
In-Reply-To: <CAGtp2Shs=FCmTypY9J1Xnq50_-uBDyKA81Z_f68SabFzNVB8Gg@mail.gmail.com>
References: <CAATCwU=btA_sYrFkMkuzy1gq87bc1nfddfN3T3LLG1f2c4afxQ@mail.gmail.com>
 <CAOFsArvBG2ej6AJZNhq8NOqVcPw701vaBExmBnJUf28f=cTOvA@mail.gmail.com>
 <CAGtp2Sg9cJO=P4q-8buDDaQTyU0iCk2WiGR-OjYsz8-dqdehwg@mail.gmail.com>
 <CAOFsArtTQzKj1tBV2v7Ph+hmAGRsczM3uautw9QuOJFx2jQyLw@mail.gmail.com>
 <CAGtp2Shs=FCmTypY9J1Xnq50_-uBDyKA81Z_f68SabFzNVB8Gg@mail.gmail.com>
Message-ID: <CAMgWh9vxcjzV6-nVjQi4-+pJeFZ+FtkDxdKxv5F=Do7BH6VrMA@mail.gmail.com>

One thing that would help a bit, I imagine, would be to improve the
treatment of unsafeDupablePerformIO in GHC so that bytestring can drop
the fragile accursedUnutterablePerformIO without significant
performance loss. There's been some discussion about how to do so, but
no resolution has been reached.

On Thu, Dec 19, 2019 at 11:21 AM Simon Jakobi via Libraries
<libraries at haskell.org> wrote:
>
> Hi Herbert,
>
> I apologize for the overly harsh and accusatory tone in my last email.
> After looking through the existing PRs and issues, I was admittedly
> somewhat disappointed over the current situation with bytestring
> issues and PRs, but I shouldn't have expressed my feelings in this
> way.
>
> Also, thanks a lot for responding so quickly today to my many comments
> on issues and PRs.
>
> To be clear, I'm not interested in putting blame on you or Duncan or
> anyone else. I do however strongly believe that bytestring needs a
> more responsive maintainer:
>
> 1. Contributors IMO deserve to get a response within weeks or ideally days.
>
> 2. bytestring users (and that transitively includes all GHC users)
> could benefit massively if some of the current PRs were finally
> merged. For example in [#175], Sylvain Henry reports improvements of
> **11%** for some compilation tasks when he used the PR to build GHC.
>
> And I admittedly don't want to ask you, Herbert, to lead the effort of
> clearing up the maintenance backlog in bytestring. Of course, _any_
> Haskell project you're involved in benefits from your massive
> experience! However, I feel that since your involvement is so crucial
> in so many other essential projects in the Haskell ecosystem, if you
> would shift your focus towards bytestring, these other projects would
> suffer! I also believe that projects like cabal or Hackage need your
> domain knowledge more dearly than bytestring does. So to ask you to
> allocate more time to bytestring, would feel a bit like a zero-sum
> game, or possibly a negative-sum game.
>
> A second, more personal reason, why I would prefer to see a different
> maintainer for bytestring, is your preference for coordinating things
> in private communication. The many PRs and issues in the bytestring
> project, and this email thread demonstrate that there are a lot of
> people who would like to be involved in improving bytestring. Private
> coordination would exclude most of them, and make it more difficult to
> make use of all the help that this project currently needs. I
> understand that your preference for private communication is related
> to your experiences with previous discussions over Haskell projects,
> and I am sorry that it has come so far, but I believe that a more open
> and transparent style of communication is more healthy.
>
> So, while I'm very grateful for your work on bytestring, and while I
> hope that you stay involved in the project, I hope that that someone
> else can take over official maintainer duties.
>
> Now I am not sure who could take on that job. Clearly we need someone
> whom Duncan trusts. That's why I wonder whether someone involved in
> the performance work on GHC could help. Ultimately I don't think that
> their time involvement would need to be very high – good judgment and
> responsiveness seem more important to me. I'm CC'ing the ghc-devs
> list, hoping that someone with the right experience and availability
> would come forward.
>
> Thanks,
> Simon
>
> [#175] https://github.com/haskell/bytestring/pull/175#issuecomment-567233984
> _______________________________________________
> Libraries mailing list
> Libraries at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries

From simon.jakobi at googlemail.com  Thu Dec 19 16:58:17 2019
From: simon.jakobi at googlemail.com (Simon Jakobi)
Date: Thu, 19 Dec 2019 17:58:17 +0100
Subject: haskell/bytestring stalled?
In-Reply-To: <CAGtp2Shs=FCmTypY9J1Xnq50_-uBDyKA81Z_f68SabFzNVB8Gg@mail.gmail.com>
References: <CAATCwU=btA_sYrFkMkuzy1gq87bc1nfddfN3T3LLG1f2c4afxQ@mail.gmail.com>
 <CAOFsArvBG2ej6AJZNhq8NOqVcPw701vaBExmBnJUf28f=cTOvA@mail.gmail.com>
 <CAGtp2Sg9cJO=P4q-8buDDaQTyU0iCk2WiGR-OjYsz8-dqdehwg@mail.gmail.com>
 <CAOFsArtTQzKj1tBV2v7Ph+hmAGRsczM3uautw9QuOJFx2jQyLw@mail.gmail.com>
 <CAGtp2Shs=FCmTypY9J1Xnq50_-uBDyKA81Z_f68SabFzNVB8Gg@mail.gmail.com>
Message-ID: <CAGtp2Sg8=cz70oj--8Cq6Q7ZSdyuma0M_LVtgAzjSr3dRUJhkg@mail.gmail.com>

I'm also CC'ing the core libraries committee – sorry for not realizing
earlier that they would be interested in this discussion!

Am Do., 19. Dez. 2019 um 17:20 Uhr schrieb Simon Jakobi
<simon.jakobi at googlemail.com>:
>
> Hi Herbert,
>
> I apologize for the overly harsh and accusatory tone in my last email.
> After looking through the existing PRs and issues, I was admittedly
> somewhat disappointed over the current situation with bytestring
> issues and PRs, but I shouldn't have expressed my feelings in this
> way.
>
> Also, thanks a lot for responding so quickly today to my many comments
> on issues and PRs.
>
> To be clear, I'm not interested in putting blame on you or Duncan or
> anyone else. I do however strongly believe that bytestring needs a
> more responsive maintainer:
>
> 1. Contributors IMO deserve to get a response within weeks or ideally days.
>
> 2. bytestring users (and that transitively includes all GHC users)
> could benefit massively if some of the current PRs were finally
> merged. For example in [#175], Sylvain Henry reports improvements of
> **11%** for some compilation tasks when he used the PR to build GHC.
>
> And I admittedly don't want to ask you, Herbert, to lead the effort of
> clearing up the maintenance backlog in bytestring. Of course, _any_
> Haskell project you're involved in benefits from your massive
> experience! However, I feel that since your involvement is so crucial
> in so many other essential projects in the Haskell ecosystem, if you
> would shift your focus towards bytestring, these other projects would
> suffer! I also believe that projects like cabal or Hackage need your
> domain knowledge more dearly than bytestring does. So to ask you to
> allocate more time to bytestring, would feel a bit like a zero-sum
> game, or possibly a negative-sum game.
>
> A second, more personal reason, why I would prefer to see a different
> maintainer for bytestring, is your preference for coordinating things
> in private communication. The many PRs and issues in the bytestring
> project, and this email thread demonstrate that there are a lot of
> people who would like to be involved in improving bytestring. Private
> coordination would exclude most of them, and make it more difficult to
> make use of all the help that this project currently needs. I
> understand that your preference for private communication is related
> to your experiences with previous discussions over Haskell projects,
> and I am sorry that it has come so far, but I believe that a more open
> and transparent style of communication is more healthy.
>
> So, while I'm very grateful for your work on bytestring, and while I
> hope that you stay involved in the project, I hope that that someone
> else can take over official maintainer duties.
>
> Now I am not sure who could take on that job. Clearly we need someone
> whom Duncan trusts. That's why I wonder whether someone involved in
> the performance work on GHC could help. Ultimately I don't think that
> their time involvement would need to be very high – good judgment and
> responsiveness seem more important to me. I'm CC'ing the ghc-devs
> list, hoping that someone with the right experience and availability
> would come forward.
>
> Thanks,
> Simon
>
> [#175] https://github.com/haskell/bytestring/pull/175#issuecomment-567233984

From ben at well-typed.com  Thu Dec 19 17:54:23 2019
From: ben at well-typed.com (Ben Gamari)
Date: Thu, 19 Dec 2019 12:54:23 -0500
Subject: Perf metrics
In-Reply-To: <VI1PR8303MB0093F52C2A4E12C6D26925BEAD520@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
References: <VI1PR8303MB0093F52C2A4E12C6D26925BEAD520@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
Message-ID: <87v9qc5hh1.fsf@smart-cactus.org>

Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:

> When I run `sh validate -legacy`, towards the end I see stuff like this
>
> Performance Metrics (test environment: local):
>
>
>
> Conversions(normal)       runtime/bytes allocated                        107696.000
>
> DeriveNull(normal)        runtime/bytes allocated                     112050960.000
>
> InlineArrayAlloc(normal)  runtime/bytes allocated                    1600041088.000
>
> InlineByteArrayAlloc(normal)  runtime/bytes allocated                    1440041088.000
>
> InlineCloneArrayAlloc(normal)  runtime/bytes allocated                    1600041248.000
>
> ManyAlternatives(normal)  compile_time/bytes allocated                840985728.000
>
> ManyConstructors(normal)  compile_time/bytes allocated               4540766560.000
>
> MethSharing(normal)       runtime/peak_megabytes_allocated                    2.000
>
> MethSharing(normal)       runtime/bytes allocated                     480098136.000
>
> MultiLayerModules(normal)  compile_time/bytes allocated               5856970504.000
>
> ...
> It is intermingled with other apparently unrelated output.

Hmm, it really should be a distinct block of output. Are you saying that
you are seeing lines of unrelated output interspersed in the performance
metrics table?

> What should I conclude form this? Is it good or bad? By what amount
> have these figures changed, and relative to what?

Below the performance metrics table you should see a blurb of text like
the following:


    Missing Baseline Metrics these metrics trivially pass because a
    baseline (expected value) cannot be recovered from previous git
    commits. This may be due to HEAD having new tests or having expected
    changes, the presence of expected changes since the last run of the
    tests, and/or the latest test run being too old.

          MultiLayerModules
          ...

    If the tests exist on the previous commit (And are configured to run
    with the same ways), then check out that commit and run the tests to
    generate the missing metrics. Alternatively, a baseline may be
    recovered from ci results once fetched:

          git fetch https://gitlab.haskell.org/ghc/ghc-performance-notes.git refs/notes/perf:refs/notes/ci/perf

The suggested command will fetch up-to-date performance metrics from the
metrics repository (populated by CI). If you then run the testsuite
again you will see output comparing each test's output to the baseline
from CI. For instance,

    Performance Metrics (test environment: local):

    MultiLayerModules(normal)  compile_time/bytes allocated               5710260920.000
                               (baseline @ HEAD~1)                        5726848340.000  [unchanged 0.3%]

The "unchanged" here refers to the fact that the +0.3% observed change
is within the indicated acceptance window of the test.


> How can I run a single perf test?

Perf tests are treated like any other test. A single test can be run
under Hadrian with the following:

    ./hadrian/build.cabal.sh test --build-root=_validatebuild \
      --flavour=Validate --only=MultiLayerModules

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191219/4acdf8a1/attachment.sig>

From simonpj at microsoft.com  Fri Dec 20 09:10:32 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Fri, 20 Dec 2019 09:10:32 +0000
Subject: Perf metrics
In-Reply-To: <87v9qc5hh1.fsf@smart-cactus.org>
References: <VI1PR8303MB0093F52C2A4E12C6D26925BEAD520@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
 <87v9qc5hh1.fsf@smart-cactus.org>
Message-ID: <VI1PR8303MB0093353B318063620CFCFAD1AD2D0@VI1PR8303MB0093.EURPRD83.prod.outlook.com>

|  Hmm, it really should be a distinct block of output. Are you saying
|  that
|  you are seeing lines of unrelated output interspersed in the
|  performance
|  metrics table?

Yes, just so.  I attach the tail of the validate run.

|  The suggested command will fetch up-to-date performance metrics from
|  the
|  metrics repository (populated by CI). If you then run the testsuite
|  again you will see output comparing each test's output to the baseline
|  from CI. For instance,

Aha, I'll try that. I missed those words.  But actually the words don't say "run validate again to see the differences".  They say "a baseline may be recovered from ci results once fetched" which is a deeply cryptic utterance.

Also, does it make sense to spit out all these numbers if they are useless? Better, perhaps in the overall SUMMARY (at the end of validate) to 

 - Have a section for perf tests, along with "Unexpected passes" 
   and "Unexpected failure" we can have "Unexpected perf failures"
 - In that section, if there is no baseline data, put the words that
   explain how to get it.

Simon

|  -----Original Message-----
|  From: Ben Gamari <ben at well-typed.com>
|  Sent: 19 December 2019 17:54
|  To: Simon Peyton Jones <simonpj at microsoft.com>; ghc-devs at haskell.org
|  Subject: Re: Perf metrics
|  
|  Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:
|  
|  > When I run `sh validate -legacy`, towards the end I see stuff like
|  this
|  >
|  > Performance Metrics (test environment: local):
|  >
|  >
|  >
|  > Conversions(normal)       runtime/bytes allocated
|  107696.000
|  >
|  > DeriveNull(normal)        runtime/bytes allocated
|  112050960.000
|  >
|  > InlineArrayAlloc(normal)  runtime/bytes allocated
|  1600041088.000
|  >
|  > InlineByteArrayAlloc(normal)  runtime/bytes allocated
|  1440041088.000
|  >
|  > InlineCloneArrayAlloc(normal)  runtime/bytes allocated
|  1600041248.000
|  >
|  > ManyAlternatives(normal)  compile_time/bytes allocated
|  840985728.000
|  >
|  > ManyConstructors(normal)  compile_time/bytes allocated
|  4540766560.000
|  >
|  > MethSharing(normal)       runtime/peak_megabytes_allocated
|  2.000
|  >
|  > MethSharing(normal)       runtime/bytes allocated
|  480098136.000
|  >
|  > MultiLayerModules(normal)  compile_time/bytes allocated
|  5856970504.000
|  >
|  > ...
|  > It is intermingled with other apparently unrelated output.
|  
|  Hmm, it really should be a distinct block of output. Are you saying
|  that
|  you are seeing lines of unrelated output interspersed in the
|  performance
|  metrics table?
|  
|  > What should I conclude form this? Is it good or bad? By what amount
|  > have these figures changed, and relative to what?
|  
|  Below the performance metrics table you should see a blurb of text
|  like
|  the following:
|  
|  
|      Missing Baseline Metrics these metrics trivially pass because a
|      baseline (expected value) cannot be recovered from previous git
|      commits. This may be due to HEAD having new tests or having
|  expected
|      changes, the presence of expected changes since the last run of
|  the
|      tests, and/or the latest test run being too old.
|  
|            MultiLayerModules
|            ...
|  
|      If the tests exist on the previous commit (And are configured to
|  run
|      with the same ways), then check out that commit and run the tests
|  to
|      generate the missing metrics. Alternatively, a baseline may be
|      recovered from ci results once fetched:
|  
|            git fetch https://gitlab.haskell.org/ghc/ghc-performance-
|  notes.git refs/notes/perf:refs/notes/ci/perf
|  
|  The suggested command will fetch up-to-date performance metrics from
|  the
|  metrics repository (populated by CI). If you then run the testsuite
|  again you will see output comparing each test's output to the baseline
|  from CI. For instance,
|  
|      Performance Metrics (test environment: local):
|  
|      MultiLayerModules(normal)  compile_time/bytes allocated
|  5710260920.000
|                                 (baseline @ HEAD~1)
|  5726848340.000  [unchanged 0.3%]
|  
|  The "unchanged" here refers to the fact that the +0.3% observed change
|  is within the indicated acceptance window of the test.
|  
|  
|  > How can I run a single perf test?
|  
|  Perf tests are treated like any other test. A single test can be run
|  under Hadrian with the following:
|  
|      ./hadrian/build.cabal.sh test --build-root=_validatebuild \
|        --flavour=Validate --only=MultiLayerModules
|  
|  Cheers,
|  
|  - Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: val-tail
Type: application/octet-stream
Size: 56596 bytes
Desc: val-tail
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191220/d58c7940/attachment-0001.obj>

From ben at well-typed.com  Fri Dec 20 16:38:28 2019
From: ben at well-typed.com (Ben Gamari)
Date: Fri, 20 Dec 2019 11:38:28 -0500
Subject: Perf metrics
In-Reply-To: <VI1PR8303MB0093353B318063620CFCFAD1AD2D0@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
References: <VI1PR8303MB0093F52C2A4E12C6D26925BEAD520@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
 <87v9qc5hh1.fsf@smart-cactus.org>
 <VI1PR8303MB0093353B318063620CFCFAD1AD2D0@VI1PR8303MB0093.EURPRD83.prod.outlook.com>
Message-ID: <87r20z54w1.fsf@smart-cactus.org>

Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:

> |  Hmm, it really should be a distinct block of output. Are you saying
> |  that
> |  you are seeing lines of unrelated output interspersed in the
> |  performance
> |  metrics table?
>
> Yes, just so.  I attach the tail of the validate run.
>
Ahh, I see what is happening here. This is due to interleaving of stdout
(where the performance metrics are printed) and stderr (where exceptions
are printed). This is indeed quite unfortunate. I'm not entirely sure

> |  The suggested command will fetch up-to-date performance metrics from
> |  the
> |  metrics repository (populated by CI). If you then run the testsuite
> |  again you will see output comparing each test's output to the baseline
> |  from CI. For instance,
>
> Aha, I'll try that. I missed those words. But actually the words don't
> say "run validate again to see the differences". They say "a baseline
> may be recovered from ci results once fetched" which is a deeply
> cryptic utterance.
>
> Also, does it make sense to spit out all these numbers if they are
> useless? Better, perhaps in the overall SUMMARY (at the end of
> validate) to
>
We started emitting these metrics at the end of the log because they can
be quite handy when diagnosing performance changes in CI. However we
have recently started dumping the metrics to a file which is uploaded as
an artifact from the CI job, so perhaps this is no longer necessary.

>  - Have a section for perf tests, along with "Unexpected passes" 
>    and "Unexpected failure" we can have "Unexpected perf failures"

>  - In that section, if there is no baseline data, put the words that
>    explain how to get it.
>
A fair suggestion. I will try to get to this soon.

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191220/61e744d5/attachment.sig>

From ggreif at gmail.com  Mon Dec 23 15:23:02 2019
From: ggreif at gmail.com (Gabor Greif)
Date: Mon, 23 Dec 2019 16:23:02 +0100
Subject: HEAD bootstrap error macOS
Message-ID: <CAK-hX0mcO+x+EeG0cQ3_ggQumLpQjZ2NEiOTv=fPo6ez1eb4Jg@mail.gmail.com>

Recently I started getting following error:
```
[nix-shell:~/ghc]$ "inplace/bin/ghc-stage1" -optc-Wall -optc-Wall
-optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes
-optc-Wmissing-declarations -optc-Winline -optc-Wpointer-arith
-optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls
-optc-Wno-aggregate-return -optc-Iincludes -optc-Iincludes/dist
-optc-Iincludes/dist-derivedconstants/header
-optc-Iincludes/dist-ghcconstants/header
-optc-Iincludes/dist-install/build -optc-Irts -optc-Irts/dist/build
-optc-DCOMPILING_RTS -optc-DFS_NAMESPACE=rts -optc-fno-strict-aliasing
-optc-fno-common -optc-DDTRACE -optc-Irts/dist/build/./autogen
-optc-Wno-unknown-pragmas -optc-O2 -optc-fomit-frame-pointer -optc-g
-optc-DRtsWay=\"rts_thr\" -static -optc-DTHREADED_RTS  -O0 -H64m -Wall
  -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header
-Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
-Irts -Irts/dist/build -DCOMPILING_RTS -DFS_NAMESPACE=rts
-this-unit-id rts -dcmm-lint  -DDTRACE     -i -irts -irts/dist/build
-Irts/dist/build -irts/dist/build/./autogen -Irts/dist/build/./autogen
           -O2 -Wcpp-undef    -Wnoncanonical-monad-instances  -c
rts/sm/Scav_thr.c -o rts/dist/build/sm/Scav_thr.thr_o -v
Glasgow Haskell Compiler, Version 8.11.0.20191222, stage 1 booted by
GHC version 8.6.5
*** initializing package database:
Using binary package database:
/Users/ggreif/ghc/inplace/lib/package.conf.d/package.cache
package flags []
loading package database /Users/ggreif/ghc/inplace/lib/package.conf.d
wired-in package ghc-prim mapped to ghc-prim-0.6.1
wired-in package integer-wired-in mapped to integer-gmp-1.0.2.0
wired-in package base mapped to base-4.14.0.0
wired-in package rts mapped to rts
wired-in package template-haskell mapped to template-haskell-2.16.0.0
wired-in package ghc mapped to ghc-8.11.0.20191222
!!! initializing package database: finished in 8.51 milliseconds,
allocated 8.770 megabytes
Created temporary directory:
/var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0
*** systool:cc:
*** C Compiler:
/nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc
-x c rts/sm/Scav_thr.c -o
/var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_1.s
-fno-common -U__PIC__ -D__PIC__ -Wimplicit -S -O2 -include
/Users/ggreif/ghc/includes/ghcversion.h -Iincludes -Iincludes/dist
-Iincludes/dist-derivedconstants/header
-Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
-Irts -Irts/dist/build -Irts/dist/build -Irts/dist/build/./autogen
-I/Users/ggreif/ghc/libraries/base/include
-I/Users/ggreif/ghc/libraries/base/dist-install/build/include
-I/Users/ggreif/ghc/libraries/base/dist-install/build/dist-install/build/include
-I/Users/ggreif/ghc/libraries/integer-gmp/include
-I/Users/ggreif/ghc/libraries/integer-gmp/dist-install/build/include
-I/Users/ggreif/ghc/libraries/integer-gmp/dist-install/build/dist-install/build/include
-I/Users/ggreif/ghc/rts/dist/build -I/Users/ggreif/ghc/includes
-I/Users/ggreif/ghc/includes/dist-derivedconstants/header
-I/Users/ggreif/ghc/includes/dist-install/build -Xpreprocessor
-DCOMPILING_RTS -Xpreprocessor '-DFS_NAMESPACE=rts' -Xpreprocessor
-DDTRACE -Wall -Wall -Wextra -Wstrict-prototypes -Wmissing-prototypes
-Wmissing-declarations -Winline -Wpointer-arith -Wmissing-noreturn
-Wnested-externs -Wredundant-decls -Wno-aggregate-return -Iincludes
-Iincludes/dist -Iincludes/dist-derivedconstants/header
-Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
-Irts -Irts/dist/build -DCOMPILING_RTS '-DFS_NAMESPACE=rts'
-fno-strict-aliasing -fno-common -DDTRACE -Irts/dist/build/./autogen
-Wno-unknown-pragmas -O2 -fomit-frame-pointer -g '-DRtsWay="rts_thr"'
-DTHREADED_RTS

In file included from rts/sm/Scav_thr.c:3:0: error:

In file included from rts/sm/Scav.c:51:0: error:

In file included from rts/sm/GCUtils.h:18:0: error:

rts/sm/GCTDecl.h:113:1: error:
     error: register '%r13' unsuitable for global register variables
on this target
    |
113 | GCT_REG_DECL(gc_thread*, gct, REG_Base);
    | ^
GCT_REG_DECL(gc_thread*, gct, REG_Base);
^

rts/sm/GCTDecl.h:37:56: error:
     note: expanded from macro 'GCT_REG_DECL'
   |
37 | #define GCT_REG_DECL(type,name,reg) register type name REG(reg);
   |                                                        ^
#define GCT_REG_DECL(type,name,reg) register type name REG(reg);
                                                       ^

includes/stg/MachRegs.h:160:24: error:
     note: expanded from macro 'REG'
    |
160 | #define REG(x) __asm__("%" #x)
    |                        ^
#define REG(x) __asm__("%" #x)
                       ^
1 error generated.
*** Deleting temp files:
Deleting: /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_1.s
/var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_2.rsp
Warning: deleting non-existent
/var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_1.s
*** Deleting temp dirs:
Deleting: /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0
`cc' failed in phase `C Compiler'. (Exit code: 1)
```
My $CC is clang-9 (but I also tried others).

Has anybody seen something similar?

Cheers nd thanks,

     Gabor

From ben at smart-cactus.org  Mon Dec 23 18:09:16 2019
From: ben at smart-cactus.org (Ben Gamari)
Date: Mon, 23 Dec 2019 13:09:16 -0500
Subject: HEAD bootstrap error macOS
In-Reply-To: <CAK-hX0mcO+x+EeG0cQ3_ggQumLpQjZ2NEiOTv=fPo6ez1eb4Jg@mail.gmail.com>
References: <CAK-hX0mcO+x+EeG0cQ3_ggQumLpQjZ2NEiOTv=fPo6ez1eb4Jg@mail.gmail.com>
Message-ID: <87k16m6hiy.fsf@smart-cactus.org>

Gabor Greif <ggreif at gmail.com> writes:

> Recently I started getting following error:
> ```
> [nix-shell:~/ghc]$ "inplace/bin/ghc-stage1" -optc-Wall -optc-Wall
> -optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes
> -optc-Wmissing-declarations -optc-Winline -optc-Wpointer-arith
> -optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls
> -optc-Wno-aggregate-return -optc-Iincludes -optc-Iincludes/dist
> -optc-Iincludes/dist-derivedconstants/header
> -optc-Iincludes/dist-ghcconstants/header
> -optc-Iincludes/dist-install/build -optc-Irts -optc-Irts/dist/build
> -optc-DCOMPILING_RTS -optc-DFS_NAMESPACE=rts -optc-fno-strict-aliasing
> -optc-fno-common -optc-DDTRACE -optc-Irts/dist/build/./autogen
> -optc-Wno-unknown-pragmas -optc-O2 -optc-fomit-frame-pointer -optc-g
> -optc-DRtsWay=\"rts_thr\" -static -optc-DTHREADED_RTS  -O0 -H64m -Wall
>   -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header
> -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
> -Irts -Irts/dist/build -DCOMPILING_RTS -DFS_NAMESPACE=rts
> -this-unit-id rts -dcmm-lint  -DDTRACE     -i -irts -irts/dist/build
> -Irts/dist/build -irts/dist/build/./autogen -Irts/dist/build/./autogen
>            -O2 -Wcpp-undef    -Wnoncanonical-monad-instances  -c
> rts/sm/Scav_thr.c -o rts/dist/build/sm/Scav_thr.thr_o -v
> Glasgow Haskell Compiler, Version 8.11.0.20191222, stage 1 booted by
> GHC version 8.6.5
> *** initializing package database:
> Using binary package database:
> /Users/ggreif/ghc/inplace/lib/package.conf.d/package.cache
> package flags []
> loading package database /Users/ggreif/ghc/inplace/lib/package.conf.d
> wired-in package ghc-prim mapped to ghc-prim-0.6.1
> wired-in package integer-wired-in mapped to integer-gmp-1.0.2.0
> wired-in package base mapped to base-4.14.0.0
> wired-in package rts mapped to rts
> wired-in package template-haskell mapped to template-haskell-2.16.0.0
> wired-in package ghc mapped to ghc-8.11.0.20191222
> !!! initializing package database: finished in 8.51 milliseconds,
> allocated 8.770 megabytes
> Created temporary directory:
> /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0
> *** systool:cc:
> *** C Compiler:
> /nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc
> -x c rts/sm/Scav_thr.c -o
> /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_1.s
> -fno-common -U__PIC__ -D__PIC__ -Wimplicit -S -O2 -include
> /Users/ggreif/ghc/includes/ghcversion.h -Iincludes -Iincludes/dist
> -Iincludes/dist-derivedconstants/header
> -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
> -Irts -Irts/dist/build -Irts/dist/build -Irts/dist/build/./autogen
> -I/Users/ggreif/ghc/libraries/base/include
> -I/Users/ggreif/ghc/libraries/base/dist-install/build/include
> -I/Users/ggreif/ghc/libraries/base/dist-install/build/dist-install/build/include
> -I/Users/ggreif/ghc/libraries/integer-gmp/include
> -I/Users/ggreif/ghc/libraries/integer-gmp/dist-install/build/include
> -I/Users/ggreif/ghc/libraries/integer-gmp/dist-install/build/dist-install/build/include
> -I/Users/ggreif/ghc/rts/dist/build -I/Users/ggreif/ghc/includes
> -I/Users/ggreif/ghc/includes/dist-derivedconstants/header
> -I/Users/ggreif/ghc/includes/dist-install/build -Xpreprocessor
> -DCOMPILING_RTS -Xpreprocessor '-DFS_NAMESPACE=rts' -Xpreprocessor
> -DDTRACE -Wall -Wall -Wextra -Wstrict-prototypes -Wmissing-prototypes
> -Wmissing-declarations -Winline -Wpointer-arith -Wmissing-noreturn
> -Wnested-externs -Wredundant-decls -Wno-aggregate-return -Iincludes
> -Iincludes/dist -Iincludes/dist-derivedconstants/header
> -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
> -Irts -Irts/dist/build -DCOMPILING_RTS '-DFS_NAMESPACE=rts'
> -fno-strict-aliasing -fno-common -DDTRACE -Irts/dist/build/./autogen
> -Wno-unknown-pragmas -O2 -fomit-frame-pointer -g '-DRtsWay="rts_thr"'
> -DTHREADED_RTS
>
> In file included from rts/sm/Scav_thr.c:3:0: error:
>
> In file included from rts/sm/Scav.c:51:0: error:
>
> In file included from rts/sm/GCUtils.h:18:0: error:
>
> rts/sm/GCTDecl.h:113:1: error:
>      error: register '%r13' unsuitable for global register variables
> on this target
>     |
> 113 | GCT_REG_DECL(gc_thread*, gct, REG_Base);
>     | ^
> GCT_REG_DECL(gc_thread*, gct, REG_Base);
> ^
Hmmm, this is quite odd. Is this a clean tree? Have you pulled without
re-./booting and ./configure'ing recently? Are you using Apple's Clang?

In short, we shouldn't be using a register variable at all on OS X
(since Apple's Clang doesn't support register variables, as this error
suggests). Rather, we use TLS. This is implemented here [1]. Perhaps
check what value configure computed for CC_LLVM_BACKEND?

Cheers,

- Ben


[1] https://gitlab.haskell.org/ghc/ghc/blob/master/rts/sm/GCTDecl.h#L69
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191223/64928999/attachment.sig>

From ggreif at gmail.com  Mon Dec 23 22:38:02 2019
From: ggreif at gmail.com (Gabor Greif)
Date: Mon, 23 Dec 2019 23:38:02 +0100
Subject: HEAD bootstrap error macOS
In-Reply-To: <87k16m6hiy.fsf@smart-cactus.org>
References: <CAK-hX0mcO+x+EeG0cQ3_ggQumLpQjZ2NEiOTv=fPo6ez1eb4Jg@mail.gmail.com>
 <87k16m6hiy.fsf@smart-cactus.org>
Message-ID: <CAK-hX0nzN+YdfLNsd03R5cOXD20w3_1Vbfw2vY=6Tu35Z-nJfw@mail.gmail.com>

This is a clean tree. Modulo
```
$ diff -u mk/build.mk.sample mk/build.mk
--- mk/build.mk.sample	2019-12-07 20:44:48.850209944 +0100
+++ mk/build.mk	2019-11-27 14:18:57.052582066 +0100
@@ -20,7 +20,7 @@
 #BuildFlavour = perf-cross-ncg

 # Fast build with optimised libraries, no profiling (RECOMMENDED):
-#BuildFlavour = quick
+BuildFlavour = quick
```

I see
```
$ grep CC_LLVM_BACKEND config*
config.log:| #define CC_LLVM_BACKEND 1
```

Yes this is with clang-9, but it also happened with clang-7.

I'll try to track this down. The TLS tip is good.

Thanks,

     Gabor

On 12/23/19, Ben Gamari <ben at smart-cactus.org> wrote:
> Gabor Greif <ggreif at gmail.com> writes:
>
>> Recently I started getting following error:
>> ```
>> [nix-shell:~/ghc]$ "inplace/bin/ghc-stage1" -optc-Wall -optc-Wall
>> -optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes
>> -optc-Wmissing-declarations -optc-Winline -optc-Wpointer-arith
>> -optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls
>> -optc-Wno-aggregate-return -optc-Iincludes -optc-Iincludes/dist
>> -optc-Iincludes/dist-derivedconstants/header
>> -optc-Iincludes/dist-ghcconstants/header
>> -optc-Iincludes/dist-install/build -optc-Irts -optc-Irts/dist/build
>> -optc-DCOMPILING_RTS -optc-DFS_NAMESPACE=rts -optc-fno-strict-aliasing
>> -optc-fno-common -optc-DDTRACE -optc-Irts/dist/build/./autogen
>> -optc-Wno-unknown-pragmas -optc-O2 -optc-fomit-frame-pointer -optc-g
>> -optc-DRtsWay=\"rts_thr\" -static -optc-DTHREADED_RTS  -O0 -H64m -Wall
>>   -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header
>> -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
>> -Irts -Irts/dist/build -DCOMPILING_RTS -DFS_NAMESPACE=rts
>> -this-unit-id rts -dcmm-lint  -DDTRACE     -i -irts -irts/dist/build
>> -Irts/dist/build -irts/dist/build/./autogen -Irts/dist/build/./autogen
>>            -O2 -Wcpp-undef    -Wnoncanonical-monad-instances  -c
>> rts/sm/Scav_thr.c -o rts/dist/build/sm/Scav_thr.thr_o -v
>> Glasgow Haskell Compiler, Version 8.11.0.20191222, stage 1 booted by
>> GHC version 8.6.5
>> *** initializing package database:
>> Using binary package database:
>> /Users/ggreif/ghc/inplace/lib/package.conf.d/package.cache
>> package flags []
>> loading package database /Users/ggreif/ghc/inplace/lib/package.conf.d
>> wired-in package ghc-prim mapped to ghc-prim-0.6.1
>> wired-in package integer-wired-in mapped to integer-gmp-1.0.2.0
>> wired-in package base mapped to base-4.14.0.0
>> wired-in package rts mapped to rts
>> wired-in package template-haskell mapped to template-haskell-2.16.0.0
>> wired-in package ghc mapped to ghc-8.11.0.20191222
>> !!! initializing package database: finished in 8.51 milliseconds,
>> allocated 8.770 megabytes
>> Created temporary directory:
>> /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0
>> *** systool:cc:
>> *** C Compiler:
>> /nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc
>> -x c rts/sm/Scav_thr.c -o
>> /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_1.s
>> -fno-common -U__PIC__ -D__PIC__ -Wimplicit -S -O2 -include
>> /Users/ggreif/ghc/includes/ghcversion.h -Iincludes -Iincludes/dist
>> -Iincludes/dist-derivedconstants/header
>> -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
>> -Irts -Irts/dist/build -Irts/dist/build -Irts/dist/build/./autogen
>> -I/Users/ggreif/ghc/libraries/base/include
>> -I/Users/ggreif/ghc/libraries/base/dist-install/build/include
>> -I/Users/ggreif/ghc/libraries/base/dist-install/build/dist-install/build/include
>> -I/Users/ggreif/ghc/libraries/integer-gmp/include
>> -I/Users/ggreif/ghc/libraries/integer-gmp/dist-install/build/include
>> -I/Users/ggreif/ghc/libraries/integer-gmp/dist-install/build/dist-install/build/include
>> -I/Users/ggreif/ghc/rts/dist/build -I/Users/ggreif/ghc/includes
>> -I/Users/ggreif/ghc/includes/dist-derivedconstants/header
>> -I/Users/ggreif/ghc/includes/dist-install/build -Xpreprocessor
>> -DCOMPILING_RTS -Xpreprocessor '-DFS_NAMESPACE=rts' -Xpreprocessor
>> -DDTRACE -Wall -Wall -Wextra -Wstrict-prototypes -Wmissing-prototypes
>> -Wmissing-declarations -Winline -Wpointer-arith -Wmissing-noreturn
>> -Wnested-externs -Wredundant-decls -Wno-aggregate-return -Iincludes
>> -Iincludes/dist -Iincludes/dist-derivedconstants/header
>> -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
>> -Irts -Irts/dist/build -DCOMPILING_RTS '-DFS_NAMESPACE=rts'
>> -fno-strict-aliasing -fno-common -DDTRACE -Irts/dist/build/./autogen
>> -Wno-unknown-pragmas -O2 -fomit-frame-pointer -g '-DRtsWay="rts_thr"'
>> -DTHREADED_RTS
>>
>> In file included from rts/sm/Scav_thr.c:3:0: error:
>>
>> In file included from rts/sm/Scav.c:51:0: error:
>>
>> In file included from rts/sm/GCUtils.h:18:0: error:
>>
>> rts/sm/GCTDecl.h:113:1: error:
>>      error: register '%r13' unsuitable for global register variables
>> on this target
>>     |
>> 113 | GCT_REG_DECL(gc_thread*, gct, REG_Base);
>>     | ^
>> GCT_REG_DECL(gc_thread*, gct, REG_Base);
>> ^
> Hmmm, this is quite odd. Is this a clean tree? Have you pulled without
> re-./booting and ./configure'ing recently? Are you using Apple's Clang?
>
> In short, we shouldn't be using a register variable at all on OS X
> (since Apple's Clang doesn't support register variables, as this error
> suggests). Rather, we use TLS. This is implemented here [1]. Perhaps
> check what value configure computed for CC_LLVM_BACKEND?
>
> Cheers,
>
> - Ben
>
>
> [1] https://gitlab.haskell.org/ghc/ghc/blob/master/rts/sm/GCTDecl.h#L69
>

From ggreif at gmail.com  Mon Dec 23 22:52:28 2019
From: ggreif at gmail.com (Gabor Greif)
Date: Mon, 23 Dec 2019 23:52:28 +0100
Subject: HEAD bootstrap error macOS
In-Reply-To: <CAK-hX0nzN+YdfLNsd03R5cOXD20w3_1Vbfw2vY=6Tu35Z-nJfw@mail.gmail.com>
References: <CAK-hX0mcO+x+EeG0cQ3_ggQumLpQjZ2NEiOTv=fPo6ez1eb4Jg@mail.gmail.com>
 <87k16m6hiy.fsf@smart-cactus.org>
 <CAK-hX0nzN+YdfLNsd03R5cOXD20w3_1Vbfw2vY=6Tu35Z-nJfw@mail.gmail.com>
Message-ID: <CAK-hX0=hAFxv4pMAWi9cBy9_Rum03LXWd+erE-Af61ov9kYrNA@mail.gmail.com>

I see some conflict potential from
```
Configure completed successfully.

   Building GHC version  : 8.11.0.20191222
          Git commit id  : 1c302c6289a6eddc92b48815dd420bd58eb2f286

   Build platform        : x86_64-apple-darwin
   Host platform         : x86_64-apple-darwin
   Target platform       : x86_64-apple-darwin

   Bootstrapping using   :
/nix/store/m7nigmssmx89j5v22r1a83w6fb8cs6qb-ghc-8.6.5/bin/ghc
      which is version   : 8.6.5

>>>   Using (for bootstrapping) : /nix/store/a76n8l1caynyyngdpnyihlrgyr9k9ilg-clang-wrapper-7.1.0/bin/cc
>>>   Using gcc                 : /nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc
      which is version       : 9.0.0
   Building a cross compiler : NO
   Unregisterised            : NO
   TablesNextToCode          : YES
   hs-cpp       :
/nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc
   hs-cpp-flags : -E -undef -traditional -Wno-invalid-pp-token
-Wno-unicode -Wno-trigraphs
```

It is strange that
`/nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc`
is identified as `gcc`. It is `clang`:
```
[nix-shell:~/ghc]$
/nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc
--version
clang version 9.0.0 (tags/RELEASE_900/final)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /nix/store/mg8gzayayl9sbgdy5hq2km6lxgi9x6dp-clang-9.0.0/bin
```

Cheers,

    Gabor



On 12/23/19, Gabor Greif <ggreif at gmail.com> wrote:
> This is a clean tree. Modulo
> ```
> $ diff -u mk/build.mk.sample mk/build.mk
> --- mk/build.mk.sample	2019-12-07 20:44:48.850209944 +0100
> +++ mk/build.mk	2019-11-27 14:18:57.052582066 +0100
> @@ -20,7 +20,7 @@
>  #BuildFlavour = perf-cross-ncg
>
>  # Fast build with optimised libraries, no profiling (RECOMMENDED):
> -#BuildFlavour = quick
> +BuildFlavour = quick
> ```
>
> I see
> ```
> $ grep CC_LLVM_BACKEND config*
> config.log:| #define CC_LLVM_BACKEND 1
> ```
>
> Yes this is with clang-9, but it also happened with clang-7.
>
> I'll try to track this down. The TLS tip is good.
>
> Thanks,
>
>      Gabor
>
> On 12/23/19, Ben Gamari <ben at smart-cactus.org> wrote:
>> Gabor Greif <ggreif at gmail.com> writes:
>>
>>> Recently I started getting following error:
>>> ```
>>> [nix-shell:~/ghc]$ "inplace/bin/ghc-stage1" -optc-Wall -optc-Wall
>>> -optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes
>>> -optc-Wmissing-declarations -optc-Winline -optc-Wpointer-arith
>>> -optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls
>>> -optc-Wno-aggregate-return -optc-Iincludes -optc-Iincludes/dist
>>> -optc-Iincludes/dist-derivedconstants/header
>>> -optc-Iincludes/dist-ghcconstants/header
>>> -optc-Iincludes/dist-install/build -optc-Irts -optc-Irts/dist/build
>>> -optc-DCOMPILING_RTS -optc-DFS_NAMESPACE=rts -optc-fno-strict-aliasing
>>> -optc-fno-common -optc-DDTRACE -optc-Irts/dist/build/./autogen
>>> -optc-Wno-unknown-pragmas -optc-O2 -optc-fomit-frame-pointer -optc-g
>>> -optc-DRtsWay=\"rts_thr\" -static -optc-DTHREADED_RTS  -O0 -H64m -Wall
>>>   -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header
>>> -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
>>> -Irts -Irts/dist/build -DCOMPILING_RTS -DFS_NAMESPACE=rts
>>> -this-unit-id rts -dcmm-lint  -DDTRACE     -i -irts -irts/dist/build
>>> -Irts/dist/build -irts/dist/build/./autogen -Irts/dist/build/./autogen
>>>            -O2 -Wcpp-undef    -Wnoncanonical-monad-instances  -c
>>> rts/sm/Scav_thr.c -o rts/dist/build/sm/Scav_thr.thr_o -v
>>> Glasgow Haskell Compiler, Version 8.11.0.20191222, stage 1 booted by
>>> GHC version 8.6.5
>>> *** initializing package database:
>>> Using binary package database:
>>> /Users/ggreif/ghc/inplace/lib/package.conf.d/package.cache
>>> package flags []
>>> loading package database /Users/ggreif/ghc/inplace/lib/package.conf.d
>>> wired-in package ghc-prim mapped to ghc-prim-0.6.1
>>> wired-in package integer-wired-in mapped to integer-gmp-1.0.2.0
>>> wired-in package base mapped to base-4.14.0.0
>>> wired-in package rts mapped to rts
>>> wired-in package template-haskell mapped to template-haskell-2.16.0.0
>>> wired-in package ghc mapped to ghc-8.11.0.20191222
>>> !!! initializing package database: finished in 8.51 milliseconds,
>>> allocated 8.770 megabytes
>>> Created temporary directory:
>>> /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0
>>> *** systool:cc:
>>> *** C Compiler:
>>> /nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc
>>> -x c rts/sm/Scav_thr.c -o
>>> /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_1.s
>>> -fno-common -U__PIC__ -D__PIC__ -Wimplicit -S -O2 -include
>>> /Users/ggreif/ghc/includes/ghcversion.h -Iincludes -Iincludes/dist
>>> -Iincludes/dist-derivedconstants/header
>>> -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
>>> -Irts -Irts/dist/build -Irts/dist/build -Irts/dist/build/./autogen
>>> -I/Users/ggreif/ghc/libraries/base/include
>>> -I/Users/ggreif/ghc/libraries/base/dist-install/build/include
>>> -I/Users/ggreif/ghc/libraries/base/dist-install/build/dist-install/build/include
>>> -I/Users/ggreif/ghc/libraries/integer-gmp/include
>>> -I/Users/ggreif/ghc/libraries/integer-gmp/dist-install/build/include
>>> -I/Users/ggreif/ghc/libraries/integer-gmp/dist-install/build/dist-install/build/include
>>> -I/Users/ggreif/ghc/rts/dist/build -I/Users/ggreif/ghc/includes
>>> -I/Users/ggreif/ghc/includes/dist-derivedconstants/header
>>> -I/Users/ggreif/ghc/includes/dist-install/build -Xpreprocessor
>>> -DCOMPILING_RTS -Xpreprocessor '-DFS_NAMESPACE=rts' -Xpreprocessor
>>> -DDTRACE -Wall -Wall -Wextra -Wstrict-prototypes -Wmissing-prototypes
>>> -Wmissing-declarations -Winline -Wpointer-arith -Wmissing-noreturn
>>> -Wnested-externs -Wredundant-decls -Wno-aggregate-return -Iincludes
>>> -Iincludes/dist -Iincludes/dist-derivedconstants/header
>>> -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
>>> -Irts -Irts/dist/build -DCOMPILING_RTS '-DFS_NAMESPACE=rts'
>>> -fno-strict-aliasing -fno-common -DDTRACE -Irts/dist/build/./autogen
>>> -Wno-unknown-pragmas -O2 -fomit-frame-pointer -g '-DRtsWay="rts_thr"'
>>> -DTHREADED_RTS
>>>
>>> In file included from rts/sm/Scav_thr.c:3:0: error:
>>>
>>> In file included from rts/sm/Scav.c:51:0: error:
>>>
>>> In file included from rts/sm/GCUtils.h:18:0: error:
>>>
>>> rts/sm/GCTDecl.h:113:1: error:
>>>      error: register '%r13' unsuitable for global register variables
>>> on this target
>>>     |
>>> 113 | GCT_REG_DECL(gc_thread*, gct, REG_Base);
>>>     | ^
>>> GCT_REG_DECL(gc_thread*, gct, REG_Base);
>>> ^
>> Hmmm, this is quite odd. Is this a clean tree? Have you pulled without
>> re-./booting and ./configure'ing recently? Are you using Apple's Clang?
>>
>> In short, we shouldn't be using a register variable at all on OS X
>> (since Apple's Clang doesn't support register variables, as this error
>> suggests). Rather, we use TLS. This is implemented here [1]. Perhaps
>> check what value configure computed for CC_LLVM_BACKEND?
>>
>> Cheers,
>>
>> - Ben
>>
>>
>> [1] https://gitlab.haskell.org/ghc/ghc/blob/master/rts/sm/GCTDecl.h#L69
>>
>

From ggreif at gmail.com  Tue Dec 24 00:15:56 2019
From: ggreif at gmail.com (Gabor Greif)
Date: Tue, 24 Dec 2019 01:15:56 +0100
Subject: HEAD bootstrap error macOS
In-Reply-To: <CAK-hX0=hAFxv4pMAWi9cBy9_Rum03LXWd+erE-Af61ov9kYrNA@mail.gmail.com>
References: <CAK-hX0mcO+x+EeG0cQ3_ggQumLpQjZ2NEiOTv=fPo6ez1eb4Jg@mail.gmail.com>
 <87k16m6hiy.fsf@smart-cactus.org>
 <CAK-hX0nzN+YdfLNsd03R5cOXD20w3_1Vbfw2vY=6Tu35Z-nJfw@mail.gmail.com>
 <CAK-hX0=hAFxv4pMAWi9cBy9_Rum03LXWd+erE-Af61ov9kYrNA@mail.gmail.com>
Message-ID: <CAK-hX0nHeuzDx5m-tc5SUav6ERzowndTABdut3popzhR83=wxA@mail.gmail.com>

Comparing the above to the (successfully bootstrapping) Darwin CI
job's configure output:
```
   Using (for bootstrapping) : gcc
   Using gcc                 : cc
      which is version       : 11.0.0
```


On 12/23/19, Gabor Greif <ggreif at gmail.com> wrote:
> I see some conflict potential from
> ```
> Configure completed successfully.
>
>    Building GHC version  : 8.11.0.20191222
>           Git commit id  : 1c302c6289a6eddc92b48815dd420bd58eb2f286
>
>    Build platform        : x86_64-apple-darwin
>    Host platform         : x86_64-apple-darwin
>    Target platform       : x86_64-apple-darwin
>
>    Bootstrapping using   :
> /nix/store/m7nigmssmx89j5v22r1a83w6fb8cs6qb-ghc-8.6.5/bin/ghc
>       which is version   : 8.6.5
>
>>>>   Using (for bootstrapping) :
>>>> /nix/store/a76n8l1caynyyngdpnyihlrgyr9k9ilg-clang-wrapper-7.1.0/bin/cc
>>>>   Using gcc                 :
>>>> /nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc
>       which is version       : 9.0.0
>    Building a cross compiler : NO
>    Unregisterised            : NO
>    TablesNextToCode          : YES
>    hs-cpp       :
> /nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc
>    hs-cpp-flags : -E -undef -traditional -Wno-invalid-pp-token
> -Wno-unicode -Wno-trigraphs
> ```
>
> It is strange that
> `/nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc`
> is identified as `gcc`. It is `clang`:
> ```
> [nix-shell:~/ghc]$
> /nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc
> --version
> clang version 9.0.0 (tags/RELEASE_900/final)
> Target: x86_64-apple-darwin17.7.0
> Thread model: posix
> InstalledDir: /nix/store/mg8gzayayl9sbgdy5hq2km6lxgi9x6dp-clang-9.0.0/bin
> ```
>
> Cheers,
>
>     Gabor
>
>
>
> On 12/23/19, Gabor Greif <ggreif at gmail.com> wrote:
>> This is a clean tree. Modulo
>> ```
>> $ diff -u mk/build.mk.sample mk/build.mk
>> --- mk/build.mk.sample	2019-12-07 20:44:48.850209944 +0100
>> +++ mk/build.mk	2019-11-27 14:18:57.052582066 +0100
>> @@ -20,7 +20,7 @@
>>  #BuildFlavour = perf-cross-ncg
>>
>>  # Fast build with optimised libraries, no profiling (RECOMMENDED):
>> -#BuildFlavour = quick
>> +BuildFlavour = quick
>> ```
>>
>> I see
>> ```
>> $ grep CC_LLVM_BACKEND config*
>> config.log:| #define CC_LLVM_BACKEND 1
>> ```
>>
>> Yes this is with clang-9, but it also happened with clang-7.
>>
>> I'll try to track this down. The TLS tip is good.
>>
>> Thanks,
>>
>>      Gabor
>>
>> On 12/23/19, Ben Gamari <ben at smart-cactus.org> wrote:
>>> Gabor Greif <ggreif at gmail.com> writes:
>>>
>>>> Recently I started getting following error:
>>>> ```
>>>> [nix-shell:~/ghc]$ "inplace/bin/ghc-stage1" -optc-Wall -optc-Wall
>>>> -optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes
>>>> -optc-Wmissing-declarations -optc-Winline -optc-Wpointer-arith
>>>> -optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls
>>>> -optc-Wno-aggregate-return -optc-Iincludes -optc-Iincludes/dist
>>>> -optc-Iincludes/dist-derivedconstants/header
>>>> -optc-Iincludes/dist-ghcconstants/header
>>>> -optc-Iincludes/dist-install/build -optc-Irts -optc-Irts/dist/build
>>>> -optc-DCOMPILING_RTS -optc-DFS_NAMESPACE=rts -optc-fno-strict-aliasing
>>>> -optc-fno-common -optc-DDTRACE -optc-Irts/dist/build/./autogen
>>>> -optc-Wno-unknown-pragmas -optc-O2 -optc-fomit-frame-pointer -optc-g
>>>> -optc-DRtsWay=\"rts_thr\" -static -optc-DTHREADED_RTS  -O0 -H64m -Wall
>>>>   -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header
>>>> -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
>>>> -Irts -Irts/dist/build -DCOMPILING_RTS -DFS_NAMESPACE=rts
>>>> -this-unit-id rts -dcmm-lint  -DDTRACE     -i -irts -irts/dist/build
>>>> -Irts/dist/build -irts/dist/build/./autogen -Irts/dist/build/./autogen
>>>>            -O2 -Wcpp-undef    -Wnoncanonical-monad-instances  -c
>>>> rts/sm/Scav_thr.c -o rts/dist/build/sm/Scav_thr.thr_o -v
>>>> Glasgow Haskell Compiler, Version 8.11.0.20191222, stage 1 booted by
>>>> GHC version 8.6.5
>>>> *** initializing package database:
>>>> Using binary package database:
>>>> /Users/ggreif/ghc/inplace/lib/package.conf.d/package.cache
>>>> package flags []
>>>> loading package database /Users/ggreif/ghc/inplace/lib/package.conf.d
>>>> wired-in package ghc-prim mapped to ghc-prim-0.6.1
>>>> wired-in package integer-wired-in mapped to integer-gmp-1.0.2.0
>>>> wired-in package base mapped to base-4.14.0.0
>>>> wired-in package rts mapped to rts
>>>> wired-in package template-haskell mapped to template-haskell-2.16.0.0
>>>> wired-in package ghc mapped to ghc-8.11.0.20191222
>>>> !!! initializing package database: finished in 8.51 milliseconds,
>>>> allocated 8.770 megabytes
>>>> Created temporary directory:
>>>> /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0
>>>> *** systool:cc:
>>>> *** C Compiler:
>>>> /nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc
>>>> -x c rts/sm/Scav_thr.c -o
>>>> /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_1.s
>>>> -fno-common -U__PIC__ -D__PIC__ -Wimplicit -S -O2 -include
>>>> /Users/ggreif/ghc/includes/ghcversion.h -Iincludes -Iincludes/dist
>>>> -Iincludes/dist-derivedconstants/header
>>>> -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
>>>> -Irts -Irts/dist/build -Irts/dist/build -Irts/dist/build/./autogen
>>>> -I/Users/ggreif/ghc/libraries/base/include
>>>> -I/Users/ggreif/ghc/libraries/base/dist-install/build/include
>>>> -I/Users/ggreif/ghc/libraries/base/dist-install/build/dist-install/build/include
>>>> -I/Users/ggreif/ghc/libraries/integer-gmp/include
>>>> -I/Users/ggreif/ghc/libraries/integer-gmp/dist-install/build/include
>>>> -I/Users/ggreif/ghc/libraries/integer-gmp/dist-install/build/dist-install/build/include
>>>> -I/Users/ggreif/ghc/rts/dist/build -I/Users/ggreif/ghc/includes
>>>> -I/Users/ggreif/ghc/includes/dist-derivedconstants/header
>>>> -I/Users/ggreif/ghc/includes/dist-install/build -Xpreprocessor
>>>> -DCOMPILING_RTS -Xpreprocessor '-DFS_NAMESPACE=rts' -Xpreprocessor
>>>> -DDTRACE -Wall -Wall -Wextra -Wstrict-prototypes -Wmissing-prototypes
>>>> -Wmissing-declarations -Winline -Wpointer-arith -Wmissing-noreturn
>>>> -Wnested-externs -Wredundant-decls -Wno-aggregate-return -Iincludes
>>>> -Iincludes/dist -Iincludes/dist-derivedconstants/header
>>>> -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
>>>> -Irts -Irts/dist/build -DCOMPILING_RTS '-DFS_NAMESPACE=rts'
>>>> -fno-strict-aliasing -fno-common -DDTRACE -Irts/dist/build/./autogen
>>>> -Wno-unknown-pragmas -O2 -fomit-frame-pointer -g '-DRtsWay="rts_thr"'
>>>> -DTHREADED_RTS
>>>>
>>>> In file included from rts/sm/Scav_thr.c:3:0: error:
>>>>
>>>> In file included from rts/sm/Scav.c:51:0: error:
>>>>
>>>> In file included from rts/sm/GCUtils.h:18:0: error:
>>>>
>>>> rts/sm/GCTDecl.h:113:1: error:
>>>>      error: register '%r13' unsuitable for global register variables
>>>> on this target
>>>>     |
>>>> 113 | GCT_REG_DECL(gc_thread*, gct, REG_Base);
>>>>     | ^
>>>> GCT_REG_DECL(gc_thread*, gct, REG_Base);
>>>> ^
>>> Hmmm, this is quite odd. Is this a clean tree? Have you pulled without
>>> re-./booting and ./configure'ing recently? Are you using Apple's Clang?
>>>
>>> In short, we shouldn't be using a register variable at all on OS X
>>> (since Apple's Clang doesn't support register variables, as this error
>>> suggests). Rather, we use TLS. This is implemented here [1]. Perhaps
>>> check what value configure computed for CC_LLVM_BACKEND?
>>>
>>> Cheers,
>>>
>>> - Ben
>>>
>>>
>>> [1] https://gitlab.haskell.org/ghc/ghc/blob/master/rts/sm/GCTDecl.h#L69
>>>
>>
>

From ggreif at gmail.com  Wed Dec 25 23:04:08 2019
From: ggreif at gmail.com (Gabor Greif)
Date: Thu, 26 Dec 2019 00:04:08 +0100
Subject: HEAD bootstrap error macOS
In-Reply-To: <CAK-hX0nHeuzDx5m-tc5SUav6ERzowndTABdut3popzhR83=wxA@mail.gmail.com>
References: <CAK-hX0mcO+x+EeG0cQ3_ggQumLpQjZ2NEiOTv=fPo6ez1eb4Jg@mail.gmail.com>
 <87k16m6hiy.fsf@smart-cactus.org>
 <CAK-hX0nzN+YdfLNsd03R5cOXD20w3_1Vbfw2vY=6Tu35Z-nJfw@mail.gmail.com>
 <CAK-hX0=hAFxv4pMAWi9cBy9_Rum03LXWd+erE-Af61ov9kYrNA@mail.gmail.com>
 <CAK-hX0nHeuzDx5m-tc5SUav6ERzowndTABdut3popzhR83=wxA@mail.gmail.com>
Message-ID: <CAK-hX0m20SvKBFUgb1_UVdgq_kqkXiog09SxyAMVv_MnSRi1gA@mail.gmail.com>

This is now resolved. There were some incompatible

includes/ghcautoconf.h
includes/ghcplatform.h
includes/ghcversion.h

files hanging around (probably from a previous Hadrian build)
These never caused trouble, because all the right symbols were defined.
But recently with c2290596f10 some of the symbols changed.

Sorry for the noise.

    Gabor

On 12/24/19, Gabor Greif <ggreif at gmail.com> wrote:
> Comparing the above to the (successfully bootstrapping) Darwin CI
> job's configure output:
> ```
>    Using (for bootstrapping) : gcc
>    Using gcc                 : cc
>       which is version       : 11.0.0
> ```
>
>
> On 12/23/19, Gabor Greif <ggreif at gmail.com> wrote:
>> I see some conflict potential from
>> ```
>> Configure completed successfully.
>>
>>    Building GHC version  : 8.11.0.20191222
>>           Git commit id  : 1c302c6289a6eddc92b48815dd420bd58eb2f286
>>
>>    Build platform        : x86_64-apple-darwin
>>    Host platform         : x86_64-apple-darwin
>>    Target platform       : x86_64-apple-darwin
>>
>>    Bootstrapping using   :
>> /nix/store/m7nigmssmx89j5v22r1a83w6fb8cs6qb-ghc-8.6.5/bin/ghc
>>       which is version   : 8.6.5
>>
>>>>>   Using (for bootstrapping) :
>>>>> /nix/store/a76n8l1caynyyngdpnyihlrgyr9k9ilg-clang-wrapper-7.1.0/bin/cc
>>>>>   Using gcc                 :
>>>>> /nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc
>>       which is version       : 9.0.0
>>    Building a cross compiler : NO
>>    Unregisterised            : NO
>>    TablesNextToCode          : YES
>>    hs-cpp       :
>> /nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc
>>    hs-cpp-flags : -E -undef -traditional -Wno-invalid-pp-token
>> -Wno-unicode -Wno-trigraphs
>> ```
>>
>> It is strange that
>> `/nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc`
>> is identified as `gcc`. It is `clang`:
>> ```
>> [nix-shell:~/ghc]$
>> /nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc
>> --version
>> clang version 9.0.0 (tags/RELEASE_900/final)
>> Target: x86_64-apple-darwin17.7.0
>> Thread model: posix
>> InstalledDir: /nix/store/mg8gzayayl9sbgdy5hq2km6lxgi9x6dp-clang-9.0.0/bin
>> ```
>>
>> Cheers,
>>
>>     Gabor
>>
>>
>>
>> On 12/23/19, Gabor Greif <ggreif at gmail.com> wrote:
>>> This is a clean tree. Modulo
>>> ```
>>> $ diff -u mk/build.mk.sample mk/build.mk
>>> --- mk/build.mk.sample	2019-12-07 20:44:48.850209944 +0100
>>> +++ mk/build.mk	2019-11-27 14:18:57.052582066 +0100
>>> @@ -20,7 +20,7 @@
>>>  #BuildFlavour = perf-cross-ncg
>>>
>>>  # Fast build with optimised libraries, no profiling (RECOMMENDED):
>>> -#BuildFlavour = quick
>>> +BuildFlavour = quick
>>> ```
>>>
>>> I see
>>> ```
>>> $ grep CC_LLVM_BACKEND config*
>>> config.log:| #define CC_LLVM_BACKEND 1
>>> ```
>>>
>>> Yes this is with clang-9, but it also happened with clang-7.
>>>
>>> I'll try to track this down. The TLS tip is good.
>>>
>>> Thanks,
>>>
>>>      Gabor
>>>
>>> On 12/23/19, Ben Gamari <ben at smart-cactus.org> wrote:
>>>> Gabor Greif <ggreif at gmail.com> writes:
>>>>
>>>>> Recently I started getting following error:
>>>>> ```
>>>>> [nix-shell:~/ghc]$ "inplace/bin/ghc-stage1" -optc-Wall -optc-Wall
>>>>> -optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes
>>>>> -optc-Wmissing-declarations -optc-Winline -optc-Wpointer-arith
>>>>> -optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls
>>>>> -optc-Wno-aggregate-return -optc-Iincludes -optc-Iincludes/dist
>>>>> -optc-Iincludes/dist-derivedconstants/header
>>>>> -optc-Iincludes/dist-ghcconstants/header
>>>>> -optc-Iincludes/dist-install/build -optc-Irts -optc-Irts/dist/build
>>>>> -optc-DCOMPILING_RTS -optc-DFS_NAMESPACE=rts -optc-fno-strict-aliasing
>>>>> -optc-fno-common -optc-DDTRACE -optc-Irts/dist/build/./autogen
>>>>> -optc-Wno-unknown-pragmas -optc-O2 -optc-fomit-frame-pointer -optc-g
>>>>> -optc-DRtsWay=\"rts_thr\" -static -optc-DTHREADED_RTS  -O0 -H64m -Wall
>>>>>   -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header
>>>>> -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
>>>>> -Irts -Irts/dist/build -DCOMPILING_RTS -DFS_NAMESPACE=rts
>>>>> -this-unit-id rts -dcmm-lint  -DDTRACE     -i -irts -irts/dist/build
>>>>> -Irts/dist/build -irts/dist/build/./autogen -Irts/dist/build/./autogen
>>>>>            -O2 -Wcpp-undef    -Wnoncanonical-monad-instances  -c
>>>>> rts/sm/Scav_thr.c -o rts/dist/build/sm/Scav_thr.thr_o -v
>>>>> Glasgow Haskell Compiler, Version 8.11.0.20191222, stage 1 booted by
>>>>> GHC version 8.6.5
>>>>> *** initializing package database:
>>>>> Using binary package database:
>>>>> /Users/ggreif/ghc/inplace/lib/package.conf.d/package.cache
>>>>> package flags []
>>>>> loading package database /Users/ggreif/ghc/inplace/lib/package.conf.d
>>>>> wired-in package ghc-prim mapped to ghc-prim-0.6.1
>>>>> wired-in package integer-wired-in mapped to integer-gmp-1.0.2.0
>>>>> wired-in package base mapped to base-4.14.0.0
>>>>> wired-in package rts mapped to rts
>>>>> wired-in package template-haskell mapped to template-haskell-2.16.0.0
>>>>> wired-in package ghc mapped to ghc-8.11.0.20191222
>>>>> !!! initializing package database: finished in 8.51 milliseconds,
>>>>> allocated 8.770 megabytes
>>>>> Created temporary directory:
>>>>> /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0
>>>>> *** systool:cc:
>>>>> *** C Compiler:
>>>>> /nix/store/mq8dqgqf2qqkp77pzf3jwc05px8rkbny-clang-wrapper-9.0.0/bin/cc
>>>>> -x c rts/sm/Scav_thr.c -o
>>>>> /var/folders/39/lr50t0q96tx5qp0mzht34r240000gn/T/ghc55616_0/ghc_1.s
>>>>> -fno-common -U__PIC__ -D__PIC__ -Wimplicit -S -O2 -include
>>>>> /Users/ggreif/ghc/includes/ghcversion.h -Iincludes -Iincludes/dist
>>>>> -Iincludes/dist-derivedconstants/header
>>>>> -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
>>>>> -Irts -Irts/dist/build -Irts/dist/build -Irts/dist/build/./autogen
>>>>> -I/Users/ggreif/ghc/libraries/base/include
>>>>> -I/Users/ggreif/ghc/libraries/base/dist-install/build/include
>>>>> -I/Users/ggreif/ghc/libraries/base/dist-install/build/dist-install/build/include
>>>>> -I/Users/ggreif/ghc/libraries/integer-gmp/include
>>>>> -I/Users/ggreif/ghc/libraries/integer-gmp/dist-install/build/include
>>>>> -I/Users/ggreif/ghc/libraries/integer-gmp/dist-install/build/dist-install/build/include
>>>>> -I/Users/ggreif/ghc/rts/dist/build -I/Users/ggreif/ghc/includes
>>>>> -I/Users/ggreif/ghc/includes/dist-derivedconstants/header
>>>>> -I/Users/ggreif/ghc/includes/dist-install/build -Xpreprocessor
>>>>> -DCOMPILING_RTS -Xpreprocessor '-DFS_NAMESPACE=rts' -Xpreprocessor
>>>>> -DDTRACE -Wall -Wall -Wextra -Wstrict-prototypes -Wmissing-prototypes
>>>>> -Wmissing-declarations -Winline -Wpointer-arith -Wmissing-noreturn
>>>>> -Wnested-externs -Wredundant-decls -Wno-aggregate-return -Iincludes
>>>>> -Iincludes/dist -Iincludes/dist-derivedconstants/header
>>>>> -Iincludes/dist-ghcconstants/header -Iincludes/dist-install/build
>>>>> -Irts -Irts/dist/build -DCOMPILING_RTS '-DFS_NAMESPACE=rts'
>>>>> -fno-strict-aliasing -fno-common -DDTRACE -Irts/dist/build/./autogen
>>>>> -Wno-unknown-pragmas -O2 -fomit-frame-pointer -g '-DRtsWay="rts_thr"'
>>>>> -DTHREADED_RTS
>>>>>
>>>>> In file included from rts/sm/Scav_thr.c:3:0: error:
>>>>>
>>>>> In file included from rts/sm/Scav.c:51:0: error:
>>>>>
>>>>> In file included from rts/sm/GCUtils.h:18:0: error:
>>>>>
>>>>> rts/sm/GCTDecl.h:113:1: error:
>>>>>      error: register '%r13' unsuitable for global register variables
>>>>> on this target
>>>>>     |
>>>>> 113 | GCT_REG_DECL(gc_thread*, gct, REG_Base);
>>>>>     | ^
>>>>> GCT_REG_DECL(gc_thread*, gct, REG_Base);
>>>>> ^
>>>> Hmmm, this is quite odd. Is this a clean tree? Have you pulled without
>>>> re-./booting and ./configure'ing recently? Are you using Apple's Clang?
>>>>
>>>> In short, we shouldn't be using a register variable at all on OS X
>>>> (since Apple's Clang doesn't support register variables, as this error
>>>> suggests). Rather, we use TLS. This is implemented here [1]. Perhaps
>>>> check what value configure computed for CC_LLVM_BACKEND?
>>>>
>>>> Cheers,
>>>>
>>>> - Ben
>>>>
>>>>
>>>> [1] https://gitlab.haskell.org/ghc/ghc/blob/master/rts/sm/GCTDecl.h#L69
>>>>
>>>
>>
>

From lexi.lambda at gmail.com  Sat Dec 28 00:16:31 2019
From: lexi.lambda at gmail.com (Alexis King)
Date: Fri, 27 Dec 2019 18:16:31 -0600
Subject: Superclasses of type families returning constraints?
Message-ID: <461E582E-D366-46F8-8E2B-2B1429111803@gmail.com>

Hello all,

I recently noticed that GHC rejects the following program:

    type family F a b :: Constraint where
      F a a = ()

    eq :: F a b => a :~: b
    eq = Refl

This is certainly not shocking, but it is a little unsatisfying: as far as I can tell, accepting this program would be entirely sound. That is, `a ~ b` is morally a “superclass” of `F a b`. In this example the type family is admittedly rather pointless, as `a ~ b` could be used instead, but it is possible to construct more sophisticated examples that cannot be so straightforwardly expressed in other ways.

I am therefore curious: has this kind of scenario ever been discussed before? If yes, is there a paper/GitLab issue/email thread somewhere that discusses it? And if no, is there any fundamental reason that GHC does not propagate such information (i.e. it’s incompatible with some aspect of the type system or constraint solver), or is it simply something that has not been explored? (Maybe you think the above program is horrible and *shouldn’t* be accepted even if it were possible, but that is a different question entirely. :))

Thanks,
Alexis

From rae at richarde.dev  Mon Dec 30 03:02:32 2019
From: rae at richarde.dev (Richard Eisenberg)
Date: Sun, 29 Dec 2019 22:02:32 -0500
Subject: Superclasses of type families returning constraints?
In-Reply-To: <461E582E-D366-46F8-8E2B-2B1429111803@gmail.com>
References: <461E582E-D366-46F8-8E2B-2B1429111803@gmail.com>
Message-ID: <1E0156EB-6241-4AF8-B509-041EC8DD0239@richarde.dev>

Hi Alexis,

I'm not aware of any work in this direction. It's an interesting idea to think about. A few such disconnected thoughts:

- You could refactor the type family equation to be `F a b = a ~ b`. Then your program would be accepted. And -- assuming the more sophisticated example is also of kind constraint -- any non-linear pattern could be refactored similarly, so perhaps this observation would carry over.

- If we had constrained type families (paper: https://repository.brynmawr.edu/cgi/viewcontent.cgi?article=1075&context=compsci_pubs <https://repository.brynmawr.edu/cgi/viewcontent.cgi?article=1075&context=compsci_pubs>, proposal:https://github.com/ghc-proposals/ghc-proposals/pull/177 <https://github.com/ghc-proposals/ghc-proposals/pull/177>), you could express a superclass constraint on the enclosing type family, which would likely work very much in the way you would want.

Richard

> On Dec 27, 2019, at 7:16 PM, Alexis King <lexi.lambda at gmail.com> wrote:
> 
> Hello all,
> 
> I recently noticed that GHC rejects the following program:
> 
>    type family F a b :: Constraint where
>      F a a = ()
> 
>    eq :: F a b => a :~: b
>    eq = Refl
> 
> This is certainly not shocking, but it is a little unsatisfying: as far as I can tell, accepting this program would be entirely sound. That is, `a ~ b` is morally a “superclass” of `F a b`. In this example the type family is admittedly rather pointless, as `a ~ b` could be used instead, but it is possible to construct more sophisticated examples that cannot be so straightforwardly expressed in other ways.
> 
> I am therefore curious: has this kind of scenario ever been discussed before? If yes, is there a paper/GitLab issue/email thread somewhere that discusses it? And if no, is there any fundamental reason that GHC does not propagate such information (i.e. it’s incompatible with some aspect of the type system or constraint solver), or is it simply something that has not been explored? (Maybe you think the above program is horrible and *shouldn’t* be accepted even if it were possible, but that is a different question entirely. :))
> 
> Thanks,
> Alexis
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191229/32e46185/attachment.html>

From goemansrowan at gmail.com  Mon Dec 30 10:53:46 2019
From: goemansrowan at gmail.com (Rowan Goemans)
Date: Mon, 30 Dec 2019 11:53:46 +0100
Subject: Implementation of Type level warnings
Message-ID: <d6e4bfb4-af40-0dd4-b130-7c3fa43b2ae4@gmail.com>

Hello everyone,

A couple of days ago I tried to implement type level warnings [1]. I 
have got something
now that sort of works but issues the same warning multiple times and 
not on the correct
location. It's currently a WIP merge request [2]. I would like some 
help/pointers
to continue the implementation

The approach I have taken is to issue the warning once TypeWarning is 
rewritten.
However,this has severe problems since now a warning is issued multiple 
times and
at different locations due to therewritinghappening multiple times. I 
don't know
what the best location is to do this check and issue the warning. Prior to
constraint solving isn't a good place since the TypeWarning could be nested
deeply in some constraint. Afterwards also isn't doable since then the
TypeWarning constraint is gone.

An alternative could be to not implement any instance in for TypeWarning
similar to TypeError and special case it somehow.

Any help is appreciated!

[1] https://gitlab.haskell.org/ghc/ghc/issues/17027
[2] https://gitlab.haskell.org/ghc/ghc/merge_requests/2361

Best regards,

RowanGoemans
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191230/b766f339/attachment.html>

From lexi.lambda at gmail.com  Mon Dec 30 16:16:31 2019
From: lexi.lambda at gmail.com (Alexis King)
Date: Mon, 30 Dec 2019 10:16:31 -0600
Subject: Superclasses of type families returning constraints?
In-Reply-To: <1E0156EB-6241-4AF8-B509-041EC8DD0239@richarde.dev>
References: <461E582E-D366-46F8-8E2B-2B1429111803@gmail.com>
 <1E0156EB-6241-4AF8-B509-041EC8DD0239@richarde.dev>
Message-ID: <3A25348F-837E-4B2D-B4B6-E6A8981C5F96@gmail.com>

Hi Richard,

Thanks for the pointer to constrained type families—that’s helpful to clarify some of my thoughts about this, and it is indeed relevant! One example I had in my real code seems difficult to express even with constrained type families, however; here is a stripped-down version of it:

    type family F a b :: Constraint where
      F '[]        (A _)   = ()
      F (B a ': b) (C c d) = (a ~ c, F b d)

With the above type family, if we have the given `F (B a ': b) c`, we’d really like GHC to be able to derive `c ~ C t1 t2` so that the type family can continue to reduce, yielding the givens `a ~ t1` and `F b t2`. But we can’t use your trick of moving the equality into the RHS because we need to procure two fresh unification variables, something we are unable to do. The family is simply stuck, and there isn’t much we can do to cajole GHC into believing otherwise.

In the above example, what the “superclasses” of F are is much less clear. We end up with a few different implications:

    F '[] a         :-  a ~ A t1
    F a (A b)       :-  a ~ '[]
    F (B a ': b) c  :-  c ~ C t1 t2
    F a (C b c)     :-  a ~ (B t1 ': t2)

I don’t know if there is a way to encode those using the constraint language available in source Haskell today. If I understand correctly, I don’t think they can be expressed using quantified constraints. Maybe it’s more trouble than it’s worth, but I find it a little interesting to think about.

Anyway, thank you for the response; I think my original question has been answered. :)

Alexis

> On Dec 29, 2019, at 21:02, Richard Eisenberg <rae at richarde.dev> wrote:
> 
> Hi Alexis,
> 
> I'm not aware of any work in this direction. It's an interesting idea to think about. A few such disconnected thoughts:
> 
> - You could refactor the type family equation to be `F a b = a ~ b`. Then your program would be accepted. And -- assuming the more sophisticated example is also of kind constraint -- any non-linear pattern could be refactored similarly, so perhaps this observation would carry over.
> 
> - If we had constrained type families (paper: https://repository.brynmawr.edu/cgi/viewcontent.cgi?article=1075&context=compsci_pubs, proposal:https://github.com/ghc-proposals/ghc-proposals/pull/177), you could express a superclass constraint on the enclosing type family, which would likely work very much in the way you would want.


From rae at richarde.dev  Mon Dec 30 18:51:21 2019
From: rae at richarde.dev (Richard Eisenberg)
Date: Mon, 30 Dec 2019 13:51:21 -0500
Subject: Implementation of Type level warnings
In-Reply-To: <d6e4bfb4-af40-0dd4-b130-7c3fa43b2ae4@gmail.com>
References: <d6e4bfb4-af40-0dd4-b130-7c3fa43b2ae4@gmail.com>
Message-ID: <5C97931A-BC92-407A-92F5-5E18F0C50F2E@richarde.dev>

I've responded on the MR. Thanks for stepping up to do this!

Richard

> On Dec 30, 2019, at 5:53 AM, Rowan Goemans <goemansrowan at gmail.com> wrote:
> 
> Hello everyone,
> 
> A couple of days ago I tried to implement type level warnings [1]. I have got something
> now that sort of works but issues the same warning multiple times and not on the correct
> location. It's currently a WIP merge request [2]. I would like some help/pointers
> to continue the implementation
> 
> The approach I have taken is to issue the warning once TypeWarning is rewritten.
> However, this has severe problems since now a warning is issued multiple times and
> at different locations due to the rewriting happening multiple times. I don't know
> what the best location is to do this check and issue the warning. Prior to
> constraint solving isn't a good place since the TypeWarning could be nested
> deeply in some constraint. Afterwards also isn't doable since then the
> TypeWarning constraint is gone.
> 
> An alternative could be to not implement any instance in for TypeWarning
> similar to TypeError and special case it somehow.
> 
> Any help is appreciated!
> 
> [1] https://gitlab.haskell.org/ghc/ghc/issues/17027 <https://gitlab.haskell.org/ghc/ghc/issues/17027>
> [2] https://gitlab.haskell.org/ghc/ghc/merge_requests/2361 <https://gitlab.haskell.org/ghc/ghc/merge_requests/2361>
> 
> Best regards,
> 
> Rowan Goemans
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191230/8df1563d/attachment.html>