From rae at cs.brynmawr.edu  Thu Jun  1 02:25:09 2017
From: rae at cs.brynmawr.edu (Richard Eisenberg)
Date: Wed, 31 May 2017 22:25:09 -0400
Subject: Hunting down a compilation performance regression involving type
 families
In-Reply-To: <CADdQpRByQ+qaXC9gCcUkW-ZEB1+TwDqw84xzjTOF8spUp196FA@mail.gmail.com>
References: <CADdQpRByQ+qaXC9gCcUkW-ZEB1+TwDqw84xzjTOF8spUp196FA@mail.gmail.com>
Message-ID: <B829D6C4-896B-4A20-BC8F-D6398B24FBC8@cs.brynmawr.edu>


> On May 31, 2017, at 5:21 PM, Ryan Scott <ryan.gl.scott at gmail.com> wrote:
> Does you know what might be going on here?

I think so, but I don't know how to fix it.

The commit you found (thank you!) makes simple_opt_expr (the "simple optimizer", run directly after desugaring, even with -O0) a little more selective in what `case` expressions it throws away. Previous to that commit, the optimizer would throw away a `case error "deferred type error" of _ -> ...` which is terrible. It seems that you have discovered that we are now too timid in throwing away unhelpful cases. It would be interesting to know what the newly-retained cases look like, so that we might throw them away.

But there remains a mystery: Why do we need this code at all? Reading Note [Getting the map/coerce RULE to work] closely, it seems we need to simplify

  forall a b (co :: a ~R# b).
    let dict = MkCoercible @* @a @b co in
    case Coercible_SCSel @* @a @b dict of
      _ [Dead] -> map @a @b (\(x :: a) -> case dict of
         MkCoercible (co :: a ~R# b) -> x |> co) = let dict = ... in ...

to

  forall a b (co :: a ~R# b).
    map @a @b (\(x :: a) -> x |> co) = \(x :: [a]) -> x |> [co]

Part of doing so is to drop the `case Coercible_SCSel ...`, which gets in the way. The mystery is why this needs special code -- shouldn't the eliminate-case-of-known-constructor do the trick? This would require unfolding Coercible_SCSel. Does that happen? It would seem not... but maybe it should, which would remove the special-case code that I changed in that commit, and quite likely would simplify much more code besides.

So: Is Coercible_SCSel unfolded during simple_opt? If not, what wonderful or terrible things happen if we do? If so, why does case-of-known-constructor not work here? My guess is that answering these questions may solve the original problem, but this guess could be wrong.

Richard


From simonpj at microsoft.com  Thu Jun  1 21:47:12 2017
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Thu, 1 Jun 2017 21:47:12 +0000
Subject: Trees that Grow in the hsSyn AST
In-Reply-To: <1955891.vsXtF6osvv@squirrel>
References: <38e18954-f13a-4a6c-8a81-8f76750cd40a@BL2NAM06FT006.Eop-nam06.prod.protection.outlook.com>
 <DM2PR21MB00285BDE68A3F95936BD3388ADFC0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <1955891.vsXtF6osvv@squirrel>
Message-ID: <DM2PR21MB0028103D9AF9679944AE7642ADF60@DM2PR21MB0028.namprd21.prod.outlook.com>

| just wondering what led to the decision to start with HsSyn. Are you
| suggesting that Core wouldn't benefit from these ideas? I surely don't

Core, unlike HsSyn, is heavily optimised and transformed.  It's hard to see how the transformations could soundly maintain arbitrary auxiliary information.  Also unlike HsSyn, Core is a very small language, so it's no big deal to transform it into something else.


| All right; I figured it wouldn't hurt to ask. May I ask what sorts of
| scaling problems you mean?

Simply maintaining a finite map from binders to another binder, or arbitrary other info (eg strictness) can be challenging.  Try it!

Simon

| -----Original Message-----
| From: David Feuer [mailto:david at well-typed.com]
| Sent: 30 May 2017 22:06
| To: Simon Peyton Jones <simonpj at microsoft.com>
| Cc: Alan & Kim Zimmerman <alan.zimm at gmail.com>; ghc-devs at haskell.org
| Subject: Re: Trees that Grow in the hsSyn AST
| 
| On Friday, May 26, 2017 9:03:15 AM EDT Simon Peyton Jones wrote:
| > 1. Which is better to start with: HsSyn or Core? Intuition suggests
| this sort of thing could be very helpful for making zapping more reliable
| and ensuring its efficiency, but there may be better reasons to start
| with HsSyn.
| >
| > Definitely HsSyn.  It’s big, riddled with extra info, and has many
| purposes for different people.  Core is small, tight, nailed down.  I
| don’t want to mess with it.
| 
| Don't get me wrong. I wasn't suggesting that Core should come first; I
| have absolutely no basis to make any suggestion in that regard. I was
| just wondering what led to the decision to start with HsSyn. Are you
| suggesting that Core wouldn't benefit from these ideas? I surely don't
| see why not. Information about arity and strictness, at least, is
| introduced in specific compiler phases. I believe that some information
| needed for join points is only valid/available between certain phases.
| Making such things explicit in the types seems like it can only help.
| 
| > 2. If we're making intrusive changes to representations, would now be a
| sensible era to consider switching to a different variable representation
| (unbound, bound, abt, etc)?
| >
| > I don’t think so.  The issues are quite orthogonal, and no one (to my
| knowledge) has proposed any vaguely plausible change to variable bindings
| that would scale to what GHC does.   In contrast, this stuff is “just”
| re-engineering.
| 
| All right; I figured it wouldn't hurt to ask. May I ask what sorts of
| scaling problems you mean?
| 
| David

From siddu.druid at gmail.com  Sat Jun  3 11:14:51 2017
From: siddu.druid at gmail.com (Siddharth Bhat)
Date: Sat, 03 Jun 2017 11:14:51 +0000
Subject: Interested to help with error messages
Message-ID: <CAPipc=1_AhuutjXJsU+iqibQKgCOXDgp-7Qj4yxjVSrsDBrT8A@mail.gmail.com>

Hello all,

Thanks for all the work that's put into GHC :)

I've tried to get into GHC development before, but I was unsuccessful,
mostly because I didn't dedicate enough time to understanding the problem
at hand & exploring the codebase.

I'd like to give it another shot. This time, I think I have a clear vision
of what I want to help with: Have haskell's error messages be easier to
read and understand.

1. Colors and layout to highlight important parts of the error messages
2. Clear formatting & naming of errors, so they're easily googleable,
stack-overflow able, etc.
3. better hints with error messages, and perhaps integrated lints(?).
4. I don't know if this is already possible, but allowing GHC errors to be
shipped off as JSON or something to interested tooling.

I saw this ticket on trac: https://ghc.haskell.org/trac/ghc/ticket/8809
I would like to take this up, but I'd like help / pointers and stuff. I
have GHC setup, I know how to use phabricator, but.. where do I start? :)

Thanks,
S~iddharth

-- 
Sending this from my phone, please excuse any typos!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170603/c60c476e/attachment.html>

From ben at smart-cactus.org  Sat Jun  3 15:50:34 2017
From: ben at smart-cactus.org (Ben Gamari)
Date: Sat, 03 Jun 2017 11:50:34 -0400
Subject: Interested to help with error messages
In-Reply-To: <CAPipc=1_AhuutjXJsU+iqibQKgCOXDgp-7Qj4yxjVSrsDBrT8A@mail.gmail.com>
References: <CAPipc=1_AhuutjXJsU+iqibQKgCOXDgp-7Qj4yxjVSrsDBrT8A@mail.gmail.com>
Message-ID: <8760gdm5o5.fsf@ben-laptop.smart-cactus.org>


CCing,
   * Alfredo Di Napoli for his on-going work in this area
   * Shivansh Rai for his interest in contributing
   * David Luposchainsky for his recent pretty-printer library
   * Richard Eisenberg due to his participation in #8809
   * Bartosz for his participation in #10735
   * Alan Zimmerman for his interest in Haskell tooling

My apologies for the tome that follows. I have been thinking about this
problem recently and think an overview of where we stand would be helpful.


Siddharth Bhat <siddu.druid at gmail.com> writes:

> Hello all,
>
> Thanks for all the work that's put into GHC :)
>
Thanks for your interest in helping!

> I've tried to get into GHC development before, but I was unsuccessful,
> mostly because I didn't dedicate enough time to understanding the problem
> at hand & exploring the codebase.
>
> I'd like to give it another shot. This time, I think I have a clear vision
> of what I want to help with: Have haskell's error messages be easier to
> read and understand.
>
> 1. Colors and layout to highlight important parts of the error messages

As I say below, I think #8809 will provide a good foundation for
improvements here. More on this below.

> 2. Clear formatting & naming of errors, so they're easily googleable,
> stack-overflow able, etc.

Indeed this is a great goal. Do you have a list of error messages that
you think are particularly egregious in this respect? Are you advocating
that we give error classes unique identifiers (e.g. as rustc does IIRC)
or are you merely suggesting that we improve the wording of the existing
messages?

> 3. better hints with error messages, and perhaps integrated lints(?).

This sounds like a noble goal, but it's a bit unclear how you get there.
We currently do try to give hints where possible, but of course we could
always offer more. It would be helpful to have a set of concrete
examples to discuss.

> 4. I don't know if this is already possible, but allowing GHC errors to be
> shipped off as JSON or something to interested tooling.
>
Indeed, this would be great. Thanks to Matthew Pickering we already
offer some limited form of this in 8.2 [1], but I think having more
structured error documents as suggested in #8809 would make this even
nicer.

[1] https://downloads.haskell.org/~ghc/master/users-guide//debugging.html?highlight=json#ghc-flag--ddump-json



The State of #8809
==================

> I saw this ticket on trac: https://ghc.haskell.org/trac/ghc/ticket/8809
> I would like to take this up, but I'd like help / pointers and stuff. I
> have GHC setup, I know how to use phabricator, but.. where do I start? :)
>
This ticket has recently seen quite a bit of activity and I've been
meaning to write down some thoughts on it. Here it goes:

Currently Alfredo Di Napoli is working [2] on the `pretty` library to
both improve performance and allow us to drop GHC's fork (see #10735),
perhaps to use annotated pretty-printer documents. Meanwhile, David
Luposchainsky, has recently released [3] his `prettyprinter` library
which may serve as a drop-in replacement to `pretty` and handles all of
the cases that Alfredo is working on. Moreover, Shivansh Rai has also
recently expressed interest in helping out with this effort.

All of this is great news: I have been hoping we'd get Idris-style
errors for quite some time. However, given how many hands we have in
this area, we should be careful not to step on each toes. Below I'll
describe the various facets of the task as I see them.

[2] https://github.com/haskell/pretty/pull/43
[3] https://www.reddit.com/r/haskell/comments/6e62i5/ann_prettyprinter_10_ending_the_wadlerleijen_zoo/


# Choice of pretty printer

It seems like we first need to resolve the question of whether switching
from (our fork of) `pretty` to the `prettyprinter` library is
worthwhile. The argument for moving to `prettyprinter` is its support
for optimized infinite-band-width layout, which is one of the things
holding us back from moving back to `pretty`.

However, there are two impediments to switching,

 * `prettyprinter` depends upon the `text` package while GHC does not.
   Making GHC dependent on `text` is an option, but we should be
   careful. Adding a dependency has a non-trivial cost (GHC build times
   rise, GHC API users are stuck using whatever dependency versions GHC
   uses, release engineering is a bit more complicated).

   Currently GHC has its own abstractions for working with text
   efficiently, LitString and FastString. FastString is used throughout
   the compiler, including the pretty-printer, and represents a
   dense UTF-8 buffer (and a hash for quick comparison). It's not clear that we
   would want to move it to `text` as this would introduce UTF-8/UTF-16
   conversion.

 * `prettyprinter` doesn't support rendering to `String`. This
   essentially means that we either use `Text` or fork the package.
   However, if we have already decided on depending on `text`, then
   perhaps the former isn't so bad.

It's unclear to me exactly how difficult switching would be compared to
finishing up the work Alfredo has started on `pretty`. Alfredo, what is
your opinion?

If we decide against moving to `prettyprinter`, then we will need to
finish up something like Alfredo's `pretty` patches to rid GHC of its
fork.


# Representing rich error messages in GHC

In my opinion we should avoid baking more stylistic decisions (e.g. printing
types in red, terms in blue) into the modules like TcErrors which produce
error messages. This is why I propose that we use annotated
pretty-printer documents in #8809 (see comment 3). This would allow us
to represent the typical things seen in GHC error messages (e.g. types,
terms, spans, binders, etc.) in structured form, allowing the error
message consumer (e.g. GHC itself, a GHC API user, or some JSON error
sink) to make decisions about how to present these elements to the user.

I think this approach give us a much better story for dealing with the
problems currently solved by flags like -fprint-runtime-reps,
-fprint-explicit-kinds, etc., especially for users using an IDE.

As far as I can recall, there was still a bit of disagreement
surrounding whether the values carried by the error message should be
statically or dynamically typed. In particular, Richard Eisenberg
advocated that error message documents look like,

    -- A dynamically typed value embedded in an error message
    data ErrItem = forall a. (Outputable a, Typeable a). ErrItem a

    type ErrDoc = Doc ErrItem

Whereas I argue that this would quickly become unmaintainable,
especially when one considers GHC API users. Rather, I say that we
should encode the "vocabulary" of things that may appear in an error
message explicitly,

    data ErrItem = ErrType Type
                 | ErrSpan Span
                 | ErrTerm HsExpr
                 | ErrInstance ClsInst
                 | ErrVar  Var
                 | ...

While there are good arguments for both options, although I think that
in balance an explicit approach will be better for consumers. Anyways,
this is a question that will need to be answered.

Once there is consensus I think it shouldn't be too difficult to move
things forward. The change can be made incrementally and for the most
part should only touch a few modules (with the bulk in TcErrors).


## What do we represent?

There is also the question of what the vocabulary of embeddable items
should consist of. I think the above are pretty non-controversial but I
can think of a variety of items which would more precisely capture
some common patterns,

    data ErrItem = ...
                 | ErrExpectedActual Type Type
                   -- ^ e.g. "Expected type: ty1, Actual type: ty2"
                 | ErrContext Type
                   -- ^ Like ErrType but specifically captures a context
                 | ErrPotentialInstances [ClsInst]
                   -- ^ A list of potentially matching instances
                 | ...

Exactly how far we want to go is something that would need to be
decided. I think we would want to start with the minimal set initially
proposed and then introduce additional items as we gain experience with
the scheme.


# Using rich error messages

Once we have GHC producing rich error documents we can teach GHC's
command line driver to prettify them. We can also teach haskell-mode,
ghc-mod, and friends to preserve their structure to give the user an
Idris-like experience.

Exactly how many stylistic decisions we want GHC to make is a tricky
question; this is prime territory for bike-shedding and people tend to
have rather strong aesthetic beliefs; keeping things simple while
satisfying all tastes may be a challenge.


# Summary

Above I discussed several tasks and a few questions,

 * We need to decide on whether David's `prettyprinter` library is right
   for GHC; having a prototype patch introducing it to the tree would
   help in evaluating this. Alfredo, what is your opinion here?

 * If not we need to drop our fork of `pretty` in favor of upstream

 * We need consensus on whether Idris-style annotated pretty-printer
   documents are the right approach for GHC (I think we are close to
   this)

 * If we want annotated documents, should the items be statically or
   dynamically typed?

 * Once these questions are resolved we can start introducing
   annotations into GHC's error documents (this shouldn't be hard)

 * Then we can teach GHC and associated tooling to pretty-print these
   rich messages prettily

There is certainly a fair bit of work here although it's not
obvious how to parallelize it across all of the interested
parties. Regardless, I would be happy to advise on any bit of this.
 
Cheers,

- Ben


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170603/55d29c4d/attachment.sig>

From alan.zimm at gmail.com  Sat Jun  3 15:56:58 2017
From: alan.zimm at gmail.com (Alan & Kim Zimmerman)
Date: Sat, 3 Jun 2017 17:56:58 +0200
Subject: Interested to help with error messages
In-Reply-To: <8760gdm5o5.fsf@ben-laptop.smart-cactus.org>
References: <CAPipc=1_AhuutjXJsU+iqibQKgCOXDgp-7Qj4yxjVSrsDBrT8A@mail.gmail.com>
 <8760gdm5o5.fsf@ben-laptop.smart-cactus.org>
Message-ID: <CANma=H9-he=gZContASum0RpQLvn+51_4N4+2f=rAjMZzn1wmg@mail.gmail.com>

Thanks Ben, a great summary.  Is there a Wiki page for this? It feels like
it should be on one, so we can easily comment/update the individual points.

In terms of the pretty-printer and its string type. Perhaps we could
backpackify it to use http://next.hackage.haskell.org:8080/package/str-sig,
and then specialise the GHC version to FastString etc.

Alan

On 3 June 2017 at 17:50, Ben Gamari <ben at smart-cactus.org> wrote:

>
> CCing,
>    * Alfredo Di Napoli for his on-going work in this area
>    * Shivansh Rai for his interest in contributing
>    * David Luposchainsky for his recent pretty-printer library
>    * Richard Eisenberg due to his participation in #8809
>    * Bartosz for his participation in #10735
>    * Alan Zimmerman for his interest in Haskell tooling
>
> My apologies for the tome that follows. I have been thinking about this
> problem recently and think an overview of where we stand would be helpful.
>
>
> Siddharth Bhat <siddu.druid at gmail.com> writes:
>
> > Hello all,
> >
> > Thanks for all the work that's put into GHC :)
> >
> Thanks for your interest in helping!
>
> > I've tried to get into GHC development before, but I was unsuccessful,
> > mostly because I didn't dedicate enough time to understanding the problem
> > at hand & exploring the codebase.
> >
> > I'd like to give it another shot. This time, I think I have a clear
> vision
> > of what I want to help with: Have haskell's error messages be easier to
> > read and understand.
> >
> > 1. Colors and layout to highlight important parts of the error messages
>
> As I say below, I think #8809 will provide a good foundation for
> improvements here. More on this below.
>
> > 2. Clear formatting & naming of errors, so they're easily googleable,
> > stack-overflow able, etc.
>
> Indeed this is a great goal. Do you have a list of error messages that
> you think are particularly egregious in this respect? Are you advocating
> that we give error classes unique identifiers (e.g. as rustc does IIRC)
> or are you merely suggesting that we improve the wording of the existing
> messages?
>
> > 3. better hints with error messages, and perhaps integrated lints(?).
>
> This sounds like a noble goal, but it's a bit unclear how you get there.
> We currently do try to give hints where possible, but of course we could
> always offer more. It would be helpful to have a set of concrete
> examples to discuss.
>
> > 4. I don't know if this is already possible, but allowing GHC errors to
> be
> > shipped off as JSON or something to interested tooling.
> >
> Indeed, this would be great. Thanks to Matthew Pickering we already
> offer some limited form of this in 8.2 [1], but I think having more
> structured error documents as suggested in #8809 would make this even
> nicer.
>
> [1] https://downloads.haskell.org/~ghc/master/users-guide//
> debugging.html?highlight=json#ghc-flag--ddump-json
>
>
>
> The State of #8809
> ==================
>
> > I saw this ticket on trac: https://ghc.haskell.org/trac/ghc/ticket/8809
> > I would like to take this up, but I'd like help / pointers and stuff. I
> > have GHC setup, I know how to use phabricator, but.. where do I start? :)
> >
> This ticket has recently seen quite a bit of activity and I've been
> meaning to write down some thoughts on it. Here it goes:
>
> Currently Alfredo Di Napoli is working [2] on the `pretty` library to
> both improve performance and allow us to drop GHC's fork (see #10735),
> perhaps to use annotated pretty-printer documents. Meanwhile, David
> Luposchainsky, has recently released [3] his `prettyprinter` library
> which may serve as a drop-in replacement to `pretty` and handles all of
> the cases that Alfredo is working on. Moreover, Shivansh Rai has also
> recently expressed interest in helping out with this effort.
>
> All of this is great news: I have been hoping we'd get Idris-style
> errors for quite some time. However, given how many hands we have in
> this area, we should be careful not to step on each toes. Below I'll
> describe the various facets of the task as I see them.
>
> [2] https://github.com/haskell/pretty/pull/43
> [3] https://www.reddit.com/r/haskell/comments/6e62i5/ann_
> prettyprinter_10_ending_the_wadlerleijen_zoo/
>
>
> # Choice of pretty printer
>
> It seems like we first need to resolve the question of whether switching
> from (our fork of) `pretty` to the `prettyprinter` library is
> worthwhile. The argument for moving to `prettyprinter` is its support
> for optimized infinite-band-width layout, which is one of the things
> holding us back from moving back to `pretty`.
>
> However, there are two impediments to switching,
>
>  * `prettyprinter` depends upon the `text` package while GHC does not.
>    Making GHC dependent on `text` is an option, but we should be
>    careful. Adding a dependency has a non-trivial cost (GHC build times
>    rise, GHC API users are stuck using whatever dependency versions GHC
>    uses, release engineering is a bit more complicated).
>
>    Currently GHC has its own abstractions for working with text
>    efficiently, LitString and FastString. FastString is used throughout
>    the compiler, including the pretty-printer, and represents a
>    dense UTF-8 buffer (and a hash for quick comparison). It's not clear
> that we
>    would want to move it to `text` as this would introduce UTF-8/UTF-16
>    conversion.
>
>  * `prettyprinter` doesn't support rendering to `String`. This
>    essentially means that we either use `Text` or fork the package.
>    However, if we have already decided on depending on `text`, then
>    perhaps the former isn't so bad.
>
> It's unclear to me exactly how difficult switching would be compared to
> finishing up the work Alfredo has started on `pretty`. Alfredo, what is
> your opinion?
>
> If we decide against moving to `prettyprinter`, then we will need to
> finish up something like Alfredo's `pretty` patches to rid GHC of its
> fork.
>
>
> # Representing rich error messages in GHC
>
> In my opinion we should avoid baking more stylistic decisions (e.g.
> printing
> types in red, terms in blue) into the modules like TcErrors which produce
> error messages. This is why I propose that we use annotated
> pretty-printer documents in #8809 (see comment 3). This would allow us
> to represent the typical things seen in GHC error messages (e.g. types,
> terms, spans, binders, etc.) in structured form, allowing the error
> message consumer (e.g. GHC itself, a GHC API user, or some JSON error
> sink) to make decisions about how to present these elements to the user.
>
> I think this approach give us a much better story for dealing with the
> problems currently solved by flags like -fprint-runtime-reps,
> -fprint-explicit-kinds, etc., especially for users using an IDE.
>
> As far as I can recall, there was still a bit of disagreement
> surrounding whether the values carried by the error message should be
> statically or dynamically typed. In particular, Richard Eisenberg
> advocated that error message documents look like,
>
>     -- A dynamically typed value embedded in an error message
>     data ErrItem = forall a. (Outputable a, Typeable a). ErrItem a
>
>     type ErrDoc = Doc ErrItem
>
> Whereas I argue that this would quickly become unmaintainable,
> especially when one considers GHC API users. Rather, I say that we
> should encode the "vocabulary" of things that may appear in an error
> message explicitly,
>
>     data ErrItem = ErrType Type
>                  | ErrSpan Span
>                  | ErrTerm HsExpr
>                  | ErrInstance ClsInst
>                  | ErrVar  Var
>                  | ...
>
> While there are good arguments for both options, although I think that
> in balance an explicit approach will be better for consumers. Anyways,
> this is a question that will need to be answered.
>
> Once there is consensus I think it shouldn't be too difficult to move
> things forward. The change can be made incrementally and for the most
> part should only touch a few modules (with the bulk in TcErrors).
>
>
> ## What do we represent?
>
> There is also the question of what the vocabulary of embeddable items
> should consist of. I think the above are pretty non-controversial but I
> can think of a variety of items which would more precisely capture
> some common patterns,
>
>     data ErrItem = ...
>                  | ErrExpectedActual Type Type
>                    -- ^ e.g. "Expected type: ty1, Actual type: ty2"
>                  | ErrContext Type
>                    -- ^ Like ErrType but specifically captures a context
>                  | ErrPotentialInstances [ClsInst]
>                    -- ^ A list of potentially matching instances
>                  | ...
>
> Exactly how far we want to go is something that would need to be
> decided. I think we would want to start with the minimal set initially
> proposed and then introduce additional items as we gain experience with
> the scheme.
>
>
> # Using rich error messages
>
> Once we have GHC producing rich error documents we can teach GHC's
> command line driver to prettify them. We can also teach haskell-mode,
> ghc-mod, and friends to preserve their structure to give the user an
> Idris-like experience.
>
> Exactly how many stylistic decisions we want GHC to make is a tricky
> question; this is prime territory for bike-shedding and people tend to
> have rather strong aesthetic beliefs; keeping things simple while
> satisfying all tastes may be a challenge.
>
>
> # Summary
>
> Above I discussed several tasks and a few questions,
>
>  * We need to decide on whether David's `prettyprinter` library is right
>    for GHC; having a prototype patch introducing it to the tree would
>    help in evaluating this. Alfredo, what is your opinion here?
>
>  * If not we need to drop our fork of `pretty` in favor of upstream
>
>  * We need consensus on whether Idris-style annotated pretty-printer
>    documents are the right approach for GHC (I think we are close to
>    this)
>
>  * If we want annotated documents, should the items be statically or
>    dynamically typed?
>
>  * Once these questions are resolved we can start introducing
>    annotations into GHC's error documents (this shouldn't be hard)
>
>  * Then we can teach GHC and associated tooling to pretty-print these
>    rich messages prettily
>
> There is certainly a fair bit of work here although it's not
> obvious how to parallelize it across all of the interested
> parties. Regardless, I would be happy to advise on any bit of this.
>
> Cheers,
>
> - Ben
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170603/fcbfeefb/attachment-0001.html>

From ben at smart-cactus.org  Sat Jun  3 16:22:06 2017
From: ben at smart-cactus.org (Ben Gamari)
Date: Sat, 03 Jun 2017 12:22:06 -0400
Subject: Interested to help with error messages
In-Reply-To: <CANma=H9-he=gZContASum0RpQLvn+51_4N4+2f=rAjMZzn1wmg@mail.gmail.com>
References: <CAPipc=1_AhuutjXJsU+iqibQKgCOXDgp-7Qj4yxjVSrsDBrT8A@mail.gmail.com>
 <8760gdm5o5.fsf@ben-laptop.smart-cactus.org>
 <CANma=H9-he=gZContASum0RpQLvn+51_4N4+2f=rAjMZzn1wmg@mail.gmail.com>
Message-ID: <87wp8tkpn5.fsf@ben-laptop.smart-cactus.org>

Alan & Kim Zimmerman <alan.zimm at gmail.com> writes:

> Thanks Ben, a great summary.  Is there a Wiki page for this? It feels like
> it should be on one, so we can easily comment/update the individual points.
>
Here you are: https://ghc.haskell.org/trac/ghc/wiki/PrettyErrors

In the interest of time I essentially just pasted my response. Feel free
to hack it up to your heart's content. 

> In terms of the pretty-printer and its string type. Perhaps we could
> backpackify it to use http://next.hackage.haskell.org:8080/package/str-sig,
> and then specialise the GHC version to FastString etc.
>
Sounds plausible.

Relatedly, I always found it a bit odd that we used
FastString for Doc literals. Namely we have to pay the cost of
hashing the string, despite the fact that we will never use the hash.
It's likely a tiny effect (hashing is quite quick), but it still seems
odd.

Cheers,
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170603/71c68f21/attachment.sig>

From gracjanpolak at gmail.com  Sun Jun  4 18:08:34 2017
From: gracjanpolak at gmail.com (Gracjan Polak)
Date: Sun, 4 Jun 2017 11:08:34 -0700
Subject: Interested to help with error messages
In-Reply-To: <87wp8tkpn5.fsf@ben-laptop.smart-cactus.org>
References: <CAPipc=1_AhuutjXJsU+iqibQKgCOXDgp-7Qj4yxjVSrsDBrT8A@mail.gmail.com>
 <8760gdm5o5.fsf@ben-laptop.smart-cactus.org>
 <CANma=H9-he=gZContASum0RpQLvn+51_4N4+2f=rAjMZzn1wmg@mail.gmail.com>
 <87wp8tkpn5.fsf@ben-laptop.smart-cactus.org>
Message-ID: <CAChsM3QZKKMzmtemwE8mXbUtVyUAnGPFVR6Kt43VP1h6+tgtUA@mail.gmail.com>

Hi,

Thanks for all the work on error messages, this is important.

Whatever you do please remember that not only humans are recipients of
these messages. Recently GHC changed 'Warning' into 'warning' and even that
caused some issues:

https://github.com/haskell/haskell-mode/issues/1513


2017-06-03 9:22 GMT-07:00 Ben Gamari <ben at smart-cactus.org>:

> Alan & Kim Zimmerman <alan.zimm at gmail.com> writes:
>
> > Thanks Ben, a great summary.  Is there a Wiki page for this? It feels
> like
> > it should be on one, so we can easily comment/update the individual
> points.
> >
> Here you are: https://ghc.haskell.org/trac/ghc/wiki/PrettyErrors
>
> In the interest of time I essentially just pasted my response. Feel free
> to hack it up to your heart's content.
>
> > In terms of the pretty-printer and its string type. Perhaps we could
> > backpackify it to use http://next.hackage.haskell.
> org:8080/package/str-sig,
> > and then specialise the GHC version to FastString etc.
> >
> Sounds plausible.
>
> Relatedly, I always found it a bit odd that we used
> FastString for Doc literals. Namely we have to pay the cost of
> hashing the string, despite the fact that we will never use the hash.
> It's likely a tiny effect (hashing is quite quick), but it still seems
> odd.
>
> Cheers,
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170604/c415f4df/attachment.html>

From rae at cs.brynmawr.edu  Mon Jun  5 03:20:22 2017
From: rae at cs.brynmawr.edu (Richard Eisenberg)
Date: Sun, 4 Jun 2017 23:20:22 -0400
Subject: Interested to help with error messages
In-Reply-To: <8760gdm5o5.fsf@ben-laptop.smart-cactus.org>
References: <CAPipc=1_AhuutjXJsU+iqibQKgCOXDgp-7Qj4yxjVSrsDBrT8A@mail.gmail.com>
 <8760gdm5o5.fsf@ben-laptop.smart-cactus.org>
Message-ID: <92967AF8-8C01-4AB3-83A3-E7C79E1E8D26@cs.brynmawr.edu>


> On Jun 3, 2017, at 11:50 AM, Ben Gamari <ben at smart-cactus.org> wrote:
> 
> In particular, Richard Eisenberg
> advocated that error message documents look like,
> 
>    -- A dynamically typed value embedded in an error message
>    data ErrItem = forall a. (Outputable a, Typeable a). ErrItem a
> 
>    type ErrDoc = Doc ErrItem

I retract this argument.

Otherwise, I contribute only a hearty thanks to whoever is working on this, which likely has a larger impact on the adoption of Haskell than anything I've done. :)

Richard

From alan.zimm at gmail.com  Mon Jun  5 18:30:08 2017
From: alan.zimm at gmail.com (Alan & Kim Zimmerman)
Date: Mon, 5 Jun 2017 20:30:08 +0200
Subject: Trees that Grow / D3609
Message-ID: <CANma=H-cdevMFvcyCRVqchp5pCVvfUH7a8W21OX1Q4=Ex5tfZQ@mail.gmail.com>

This is a heads up that D3609[1], the first step toward implementing Trees
that Grow[2] for the hsSyn AST is about to land in master.

The key change is that it is replacing the HsSyn type parameter with one
that is an index into a set of type families, enabling the extension points
(for a future diff).

Any current work in progress branches will need a one-time search and
replace of all type parameters as follows

RdrName  -> GhcPs
Name  -> GhcRs
Var -> GhcTc
Id -> GhcTcId


Any code that makes use of the GHC API and needs to compile with earlier
versions too can do something like

      #if MIN_VERSION_ghc(8,3,0)
      type ParseI     = GhcPs
      type RenameI    = GhcRn
      type TypecheckI = GhcTc
      #else
      type ParseI     = RdrName
      type RenameI    = Name
      type TypecheckI = Var
      #endif

and then replace the RdrName / Name / Var type params with ParseI / RenameI
/ TypecheckI.

A real world example is in ghc-exactprint[3]

Regards
  Alan

[1] https://phabricator.haskell.org/D3609
[2] https://ghc.haskell.org/trac/ghc/wiki/ImplementingTreesThatGrow
[3]
https://github.com/alanz/ghc-exactprint/commit/6a6e3cdeb40f9623e9a7fc6f1be90da981cb921b
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170605/3982a045/attachment.html>

From qdunkan at gmail.com  Mon Jun  5 20:55:00 2017
From: qdunkan at gmail.com (Evan Laforge)
Date: Mon, 5 Jun 2017 13:55:00 -0700
Subject: Interested to help with error messages
In-Reply-To: <92967AF8-8C01-4AB3-83A3-E7C79E1E8D26@cs.brynmawr.edu>
References: <CAPipc=1_AhuutjXJsU+iqibQKgCOXDgp-7Qj4yxjVSrsDBrT8A@mail.gmail.com>
 <8760gdm5o5.fsf@ben-laptop.smart-cactus.org>
 <92967AF8-8C01-4AB3-83A3-E7C79E1E8D26@cs.brynmawr.edu>
Message-ID: <CACbaDy4cFDQSQpfWKuKNZOS2Wy-f7sqs_zc4hOBBm9Kg7nHPgw@mail.gmail.com>

This is a bit off topic, but is there a collection of not-so-great
error messages along with opinions about what they should be?  Like a
wiki page or something?

I just stumbled across one and was going to complain, but didn't know
the most productive way to do that, aside from try to fix it myself.

Specifically, I'm working in a small EDSL that uses lots of small
expressions composed together with a repurposed (.) operator.  The
pretty printer layout causes the context section of each type error to
become huge, for instance:

Derive/Solkattu/MridangamScore.hs:110:60:
    Couldn't match type ‘Derive.Solkattu.Sequence.Note
                           (Derive.Solkattu.Solkattu.Note
                              Derive.Solkattu.MridangamDsl.Stroke)’
                   with ‘[Derive.Solkattu.Sequence.Note
                            (Derive.Solkattu.Solkattu.Note
                               Derive.Solkattu.MridangamDsl.Stroke)]’
    Expected type: [Sequence]
      Actual type: [Derive.Solkattu.Sequence.Note
                      (Derive.Solkattu.Solkattu.Note
                         Derive.Solkattu.MridangamDsl.Stroke)]
    In the second argument of ‘($)’, namely
      ‘nadai 7
       $ repeat 2 kook
         . od
           . __
             . k
               . d
                 . __
                   . k
         [ keeps walking to the right for many many lines ]
    In the second argument of ‘($)’, namely
      ‘korvai adi
       $ nadai 7
         $ repeat 2 kook
           . od
             [ again ]
    In the second argument of ‘($)’, namely
       [ and again ]


You can probably easily reproduce this by making a lot of little
expressions and putting an error in someplace.  What I think it should
be is put more things on a horizontal line, or maybe cap the
expression at a certain number of lines.  Or maybe the "zooming out"
contexts could replace the previous expression with {- previous
expression -} to emphasize the difference.


Also, the other thing to think about before changing type errors is
that tons of ghc tests rely on (almost) exact error output.  Probably
the first task is to make an abstract output, then change the tests to
rely on that (or at least a legacy formatting mode), so that every
change doesn't break a million tests.

On Sun, Jun 4, 2017 at 8:20 PM, Richard Eisenberg <rae at cs.brynmawr.edu> wrote:
>
>> On Jun 3, 2017, at 11:50 AM, Ben Gamari <ben at smart-cactus.org> wrote:
>>
>> In particular, Richard Eisenberg
>> advocated that error message documents look like,
>>
>>    -- A dynamically typed value embedded in an error message
>>    data ErrItem = forall a. (Outputable a, Typeable a). ErrItem a
>>
>>    type ErrDoc = Doc ErrItem
>
> I retract this argument.
>
> Otherwise, I contribute only a hearty thanks to whoever is working on this, which likely has a larger impact on the adoption of Haskell than anything I've done. :)
>
> Richard
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From siddu.druid at gmail.com  Tue Jun  6 07:48:17 2017
From: siddu.druid at gmail.com (Siddharth Bhat)
Date: Tue, 06 Jun 2017 07:48:17 +0000
Subject: Interested to help with error messages
In-Reply-To: <CACbaDy4cFDQSQpfWKuKNZOS2Wy-f7sqs_zc4hOBBm9Kg7nHPgw@mail.gmail.com>
References: <CAPipc=1_AhuutjXJsU+iqibQKgCOXDgp-7Qj4yxjVSrsDBrT8A@mail.gmail.com>
 <8760gdm5o5.fsf@ben-laptop.smart-cactus.org>
 <92967AF8-8C01-4AB3-83A3-E7C79E1E8D26@cs.brynmawr.edu>
 <CACbaDy4cFDQSQpfWKuKNZOS2Wy-f7sqs_zc4hOBBm9Kg7nHPgw@mail.gmail.com>
Message-ID: <CAPipc=2oE316vAJ8H4Xtv+HvMM2LJT4d9GbHtDnS7P5sUOnnbQ@mail.gmail.com>

Elm has a interesting take on this: a collection of all error messages (
https://github.com/elm-lang/error-message-catalog).

I created an empty repo (
https://github.com/bollu/hask-error-messages-catalog) and shot an email to
haskell-cafe asking for examples where GHC generates unintuitive error
messages. Hopefully, I can start collecting these so there's context the
next time this comes up.

(PRs welcome :)

Thanks,

~Siddharth

On Mon 5 Jun, 2017, 22:56 Evan Laforge, <qdunkan at gmail.com> wrote:

> This is a bit off topic, but is there a collection of not-so-great
> error messages along with opinions about what they should be?  Like a
> wiki page or something?
>
> I just stumbled across one and was going to complain, but didn't know
> the most productive way to do that, aside from try to fix it myself.
>
> Specifically, I'm working in a small EDSL that uses lots of small
> expressions composed together with a repurposed (.) operator.  The
> pretty printer layout causes the context section of each type error to
> become huge, for instance:
>
> Derive/Solkattu/MridangamScore.hs:110:60:
>     Couldn't match type ‘Derive.Solkattu.Sequence.Note
>                            (Derive.Solkattu.Solkattu.Note
>                               Derive.Solkattu.MridangamDsl.Stroke)’
>                    with ‘[Derive.Solkattu.Sequence.Note
>                             (Derive.Solkattu.Solkattu.Note
>                                Derive.Solkattu.MridangamDsl.Stroke)]’
>     Expected type: [Sequence]
>       Actual type: [Derive.Solkattu.Sequence.Note
>                       (Derive.Solkattu.Solkattu.Note
>                          Derive.Solkattu.MridangamDsl.Stroke)]
>     In the second argument of ‘($)’, namely
>       ‘nadai 7
>        $ repeat 2 kook
>          . od
>            . __
>              . k
>                . d
>                  . __
>                    . k
>          [ keeps walking to the right for many many lines ]
>     In the second argument of ‘($)’, namely
>       ‘korvai adi
>        $ nadai 7
>          $ repeat 2 kook
>            . od
>              [ again ]
>     In the second argument of ‘($)’, namely
>        [ and again ]
>
>
> You can probably easily reproduce this by making a lot of little
> expressions and putting an error in someplace.  What I think it should
> be is put more things on a horizontal line, or maybe cap the
> expression at a certain number of lines.  Or maybe the "zooming out"
> contexts could replace the previous expression with {- previous
> expression -} to emphasize the difference.
>
>
> Also, the other thing to think about before changing type errors is
> that tons of ghc tests rely on (almost) exact error output.  Probably
> the first task is to make an abstract output, then change the tests to
> rely on that (or at least a legacy formatting mode), so that every
> change doesn't break a million tests.
>
> On Sun, Jun 4, 2017 at 8:20 PM, Richard Eisenberg <rae at cs.brynmawr.edu>
> wrote:
> >
> >> On Jun 3, 2017, at 11:50 AM, Ben Gamari <ben at smart-cactus.org> wrote:
> >>
> >> In particular, Richard Eisenberg
> >> advocated that error message documents look like,
> >>
> >>    -- A dynamically typed value embedded in an error message
> >>    data ErrItem = forall a. (Outputable a, Typeable a). ErrItem a
> >>
> >>    type ErrDoc = Doc ErrItem
> >
> > I retract this argument.
> >
> > Otherwise, I contribute only a hearty thanks to whoever is working on
> this, which likely has a larger impact on the adoption of Haskell than
> anything I've done. :)
> >
> > Richard
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-- 
Sending this from my phone, please excuse any typos!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170606/bc9aaa2e/attachment.html>

From ryan.gl.scott at gmail.com  Tue Jun  6 17:41:01 2017
From: ryan.gl.scott at gmail.com (Ryan Scott)
Date: Tue, 6 Jun 2017 10:41:01 -0700
Subject: Hunting down a compilation performance regression involving type
 families
In-Reply-To: <B829D6C4-896B-4A20-BC8F-D6398B24FBC8@cs.brynmawr.edu>
References: <CADdQpRByQ+qaXC9gCcUkW-ZEB1+TwDqw84xzjTOF8spUp196FA@mail.gmail.com>
 <B829D6C4-896B-4A20-BC8F-D6398B24FBC8@cs.brynmawr.edu>
Message-ID: <CADdQpRCLowV8wCUFkhox+L8-OeZG=q8Sd0bo3cg24qUdg=Sz2g@mail.gmail.com>

Hrm. It's a shame that supporting this map/coerce RULE causes such pain.

This makes me wonder: can we get rid of this RULE? Eric Mertens pointed out
a trick [1] that's used in the profunctors library to make mapping coerce
over certain Profunctors more efficient. To adapt this trick for Functor,
we'd need to add another class method:

    class Functor f where
        fmap :: (a -> b) -> f a -> f b
        (<#>) :: Coercible a b => (a -> b) -> f a -> f b
        (<#>) = \f -> \p -> p `seq` fmap f p

Now, when implementing Functor instances, if we are working with a datatype
whose role is representational or phantom, we can make (<#>) really fast:

    data List a = Nil | Cons a (List a)
    instance Functor List where
        fmap = ...
        (<#>) = coerce

Now, instead of relying on (map MkNewtype Nil) to rewrite to Nil, we can
just use (MkNewtype <#> Nil)! No map/coerce RULE necessary :)

OK, I realize that suggesting that we remove the RULE is perhaps a touch
too far. But it does sting that we have to pay hefty compilation penalties
because of its existence...

Ryan S.
-----
[1]
http://hackage.haskell.org/package/profunctors-5.2/docs/Data-Profunctor-Unsafe.html#v:-35-
.

On Wed, May 31, 2017 at 7:25 PM, Richard Eisenberg <rae at cs.brynmawr.edu>
wrote:

>
> > On May 31, 2017, at 5:21 PM, Ryan Scott <ryan.gl.scott at gmail.com> wrote:
> > Does you know what might be going on here?
>
> I think so, but I don't know how to fix it.
>
> The commit you found (thank you!) makes simple_opt_expr (the "simple
> optimizer", run directly after desugaring, even with -O0) a little more
> selective in what `case` expressions it throws away. Previous to that
> commit, the optimizer would throw away a `case error "deferred type error"
> of _ -> ...` which is terrible. It seems that you have discovered that we
> are now too timid in throwing away unhelpful cases. It would be interesting
> to know what the newly-retained cases look like, so that we might throw
> them away.
>
> But there remains a mystery: Why do we need this code at all? Reading Note
> [Getting the map/coerce RULE to work] closely, it seems we need to simplify
>
>   forall a b (co :: a ~R# b).
>     let dict = MkCoercible @* @a @b co in
>     case Coercible_SCSel @* @a @b dict of
>       _ [Dead] -> map @a @b (\(x :: a) -> case dict of
>          MkCoercible (co :: a ~R# b) -> x |> co) = let dict = ... in ...
>
> to
>
>   forall a b (co :: a ~R# b).
>     map @a @b (\(x :: a) -> x |> co) = \(x :: [a]) -> x |> [co]
>
> Part of doing so is to drop the `case Coercible_SCSel ...`, which gets in
> the way. The mystery is why this needs special code -- shouldn't the
> eliminate-case-of-known-constructor do the trick? This would require
> unfolding Coercible_SCSel. Does that happen? It would seem not... but maybe
> it should, which would remove the special-case code that I changed in that
> commit, and quite likely would simplify much more code besides.
>
> So: Is Coercible_SCSel unfolded during simple_opt? If not, what wonderful
> or terrible things happen if we do? If so, why does
> case-of-known-constructor not work here? My guess is that answering these
> questions may solve the original problem, but this guess could be wrong.
>
> Richard
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170606/a620c174/attachment.html>

From david at well-typed.com  Tue Jun  6 18:01:30 2017
From: david at well-typed.com (David Feuer)
Date: Tue, 06 Jun 2017 14:01:30 -0400
Subject: Hunting down a compilation performance regression involving
 type families
Message-ID: <20170606173816.E2571BC8B3@haskell.org>

Edward Kmett has explained that this isn't sufficient when things go higher order. His suggested improvement is
    liftCoercion :: Maybe (Coercion a b -> Coercion (f a) (f b))


David FeuerWell-Typed, LLP
-------- Original message --------From: Ryan Scott <ryan.gl.scott at gmail.com> Date: 6/6/17  1:41 PM  (GMT-05:00) To: Richard Eisenberg <rae at cs.brynmawr.edu> Cc: GHC developers <ghc-devs at haskell.org>, Eric Mertens <emertens at gmail.com> Subject: Re: Hunting down a compilation performance regression involving type families 
Hrm. It's a shame that supporting this map/coerce RULE causes such pain.

This makes me wonder: can we get rid of this RULE? Eric Mertens pointed out
a trick [1] that's used in the profunctors library to make mapping coerce
over certain Profunctors more efficient. To adapt this trick for Functor,
we'd need to add another class method:

    class Functor f where
        fmap :: (a -> b) -> f a -> f b
        (<#>) :: Coercible a b => (a -> b) -> f a -> f b
        (<#>) = \f -> \p -> p `seq` fmap f p

Now, when implementing Functor instances, if we are working with a datatype
whose role is representational or phantom, we can make (<#>) really fast:

    data List a = Nil | Cons a (List a)
    instance Functor List where
        fmap = ...
        (<#>) = coerce

Now, instead of relying on (map MkNewtype Nil) to rewrite to Nil, we can
just use (MkNewtype <#> Nil)! No map/coerce RULE necessary :)

OK, I realize that suggesting that we remove the RULE is perhaps a touch
too far. But it does sting that we have to pay hefty compilation penalties
because of its existence...

Ryan S.
-----
[1]
http://hackage.haskell.org/package/profunctors-5.2/docs/Data-Profunctor-Unsafe.html#v:-35-
.

On Wed, May 31, 2017 at 7:25 PM, Richard Eisenberg <rae at cs.brynmawr.edu>
wrote:

>
> > On May 31, 2017, at 5:21 PM, Ryan Scott <ryan.gl.scott at gmail.com> wrote:
> > Does you know what might be going on here?
>
> I think so, but I don't know how to fix it.
>
> The commit you found (thank you!) makes simple_opt_expr (the "simple
> optimizer", run directly after desugaring, even with -O0) a little more
> selective in what `case` expressions it throws away. Previous to that
> commit, the optimizer would throw away a `case error "deferred type error"
> of _ -> ...` which is terrible. It seems that you have discovered that we
> are now too timid in throwing away unhelpful cases. It would be interesting
> to know what the newly-retained cases look like, so that we might throw
> them away.
>
> But there remains a mystery: Why do we need this code at all? Reading Note
> [Getting the map/coerce RULE to work] closely, it seems we need to simplify
>
>   forall a b (co :: a ~R# b).
>     let dict = MkCoercible @* @a @b co in
>     case Coercible_SCSel @* @a @b dict of
>       _ [Dead] -> map @a @b (\(x :: a) -> case dict of
>          MkCoercible (co :: a ~R# b) -> x |> co) = let dict = ... in ...
>
> to
>
>   forall a b (co :: a ~R# b).
>     map @a @b (\(x :: a) -> x |> co) = \(x :: [a]) -> x |> [co]
>
> Part of doing so is to drop the `case Coercible_SCSel ...`, which gets in
> the way. The mystery is why this needs special code -- shouldn't the
> eliminate-case-of-known-constructor do the trick? This would require
> unfolding Coercible_SCSel. Does that happen? It would seem not... but maybe
> it should, which would remove the special-case code that I changed in that
> commit, and quite likely would simplify much more code besides.
>
> So: Is Coercible_SCSel unfolded during simple_opt? If not, what wonderful
> or terrible things happen if we do? If so, why does
> case-of-known-constructor not work here? My guess is that answering these
> questions may solve the original problem, but this guess could be wrong.
>
> Richard
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170606/5dbed680/attachment.html>

From baldurpet at gmail.com  Tue Jun  6 18:07:31 2017
From: baldurpet at gmail.com (=?UTF-8?Q?Baldur_Bl=C3=B6ndal?=)
Date: Tue, 6 Jun 2017 19:07:31 +0100
Subject: Hunting down a compilation performance regression involving type
 families
In-Reply-To: <20170606173816.E2571BC8B3@haskell.org>
References: <20170606173816.E2571BC8B3@haskell.org>
Message-ID: <CAK9DwL_MfSLNLyV=+k9VAALjnxmKp2kOhrVU+x89A_zuQ0Ji2A@mail.gmail.com>

This a use case for ImplicationConstraints, or what

On Jun 6, 2017 19:02, "David Feuer" <david at well-typed.com> wrote:

> Edward Kmett has explained that this isn't sufficient when things go
> higher order. His suggested improvement is
>
>     liftCoercion :: Maybe (Coercion a b -> Coercion (f a) (f b))
>
>
>
> David Feuer
> Well-Typed, LLP
>
> -------- Original message --------
> From: Ryan Scott <ryan.gl.scott at gmail.com>
> Date: 6/6/17 1:41 PM (GMT-05:00)
> To: Richard Eisenberg <rae at cs.brynmawr.edu>
> Cc: GHC developers <ghc-devs at haskell.org>, Eric Mertens <
> emertens at gmail.com>
> Subject: Re: Hunting down a compilation performance regression involving
> type families
>
> Hrm. It's a shame that supporting this map/coerce RULE causes such pain.
>
> This makes me wonder: can we get rid of this RULE? Eric Mertens pointed out
> a trick [1] that's used in the profunctors library to make mapping coerce
> over certain Profunctors more efficient. To adapt this trick for Functor,
> we'd need to add another class method:
>
>     class Functor f where
>         fmap :: (a -> b) -> f a -> f b
>         (<#>) :: Coercible a b => (a -> b) -> f a -> f b
>         (<#>) = \f -> \p -> p `seq` fmap f p
>
> Now, when implementing Functor instances, if we are working with a datatype
> whose role is representational or phantom, we can make (<#>) really fast:
>
>     data List a = Nil | Cons a (List a)
>     instance Functor List where
>         fmap = ...
>         (<#>) = coerce
>
> Now, instead of relying on (map MkNewtype Nil) to rewrite to Nil, we can
> just use (MkNewtype <#> Nil)! No map/coerce RULE necessary :)
>
> OK, I realize that suggesting that we remove the RULE is perhaps a touch
> too far. But it does sting that we have to pay hefty compilation penalties
> because of its existence...
>
> Ryan S.
> -----
> [1]
> http://hackage.haskell.org/package/profunctors-5.2/docs/
> Data-Profunctor-Unsafe.html#v:-35-
> .
>
> On Wed, May 31, 2017 at 7:25 PM, Richard Eisenberg <rae at cs.brynmawr.edu>
> wrote:
>
> >
> > > On May 31, 2017, at 5:21 PM, Ryan Scott <ryan.gl.scott at gmail.com>
> wrote:
> > > Does you know what might be going on here?
> >
> > I think so, but I don't know how to fix it.
> >
> > The commit you found (thank you!) makes simple_opt_expr (the "simple
> > optimizer", run directly after desugaring, even with -O0) a little more
> > selective in what `case` expressions it throws away. Previous to that
> > commit, the optimizer would throw away a `case error "deferred type
> error"
> > of _ -> ...` which is terrible. It seems that you have discovered that we
> > are now too timid in throwing away unhelpful cases. It would be
> interesting
> > to know what the newly-retained cases look like, so that we might throw
> > them away.
> >
> > But there remains a mystery: Why do we need this code at all? Reading
> Note
> > [Getting the map/coerce RULE to work] closely, it seems we need to
> simplify
> >
> >   forall a b (co :: a ~R# b).
> >     let dict = MkCoercible @* @a @b co in
> >     case Coercible_SCSel @* @a @b dict of
> >       _ [Dead] -> map @a @b (\(x :: a) -> case dict of
> >          MkCoercible (co :: a ~R# b) -> x |> co) = let dict = ... in ...
> >
> > to
> >
> >   forall a b (co :: a ~R# b).
> >     map @a @b (\(x :: a) -> x |> co) = \(x :: [a]) -> x |> [co]
> >
> > Part of doing so is to drop the `case Coercible_SCSel ...`, which gets in
> > the way. The mystery is why this needs special code -- shouldn't the
> > eliminate-case-of-known-constructor do the trick? This would require
> > unfolding Coercible_SCSel. Does that happen? It would seem not... but
> maybe
> > it should, which would remove the special-case code that I changed in
> that
> > commit, and quite likely would simplify much more code besides.
> >
> > So: Is Coercible_SCSel unfolded during simple_opt? If not, what wonderful
> > or terrible things happen if we do? If so, why does
> > case-of-known-constructor not work here? My guess is that answering these
> > questions may solve the original problem, but this guess could be wrong.
> >
> > Richard
> >
> >
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170606/5bd04f84/attachment-0001.html>

From ryan.gl.scott at gmail.com  Tue Jun  6 18:21:11 2017
From: ryan.gl.scott at gmail.com (Ryan Scott)
Date: Tue, 6 Jun 2017 11:21:11 -0700
Subject: Hunting down a compilation performance regression involving type
 families
In-Reply-To: <5936ee01.0dd2190a.a13b3.94b4SMTPIN_ADDED_MISSING@mx.google.com>
References: <5936ee01.0dd2190a.a13b3.94b4SMTPIN_ADDED_MISSING@mx.google.com>
Message-ID: <CADdQpRCM08DbCEnbfs1qv7SjJV_ma_Ve6FYhv7-5gYtg6MRcxA@mail.gmail.com>

Ah yes, I believe you're referring to https://ghc.haskell.org/
trac/ghc/ticket/8767 (which I wasn't aware of). And yes, I do believe we'd
need some combination of QuantifiedContexts/ImplicationConstraints to fully
support everything.

But in any case, don't let this discussion side-track too much from the
matter at hand (the type family compilation slowdown). My suggestion _was_
somewhat tongue-in-cheek, after all ;)

Ryan S.

On Tue, Jun 6, 2017 at 11:01 AM, David Feuer <david at well-typed.com> wrote:

> Edward Kmett has explained that this isn't sufficient when things go
> higher order. His suggested improvement is
>
>     liftCoercion :: Maybe (Coercion a b -> Coercion (f a) (f b))
>
>
>
> David Feuer
> Well-Typed, LLP
>
> -------- Original message --------
> From: Ryan Scott <ryan.gl.scott at gmail.com>
> Date: 6/6/17 1:41 PM (GMT-05:00)
> To: Richard Eisenberg <rae at cs.brynmawr.edu>
> Cc: GHC developers <ghc-devs at haskell.org>, Eric Mertens <
> emertens at gmail.com>
> Subject: Re: Hunting down a compilation performance regression involving
> type families
>
> Hrm. It's a shame that supporting this map/coerce RULE causes such pain.
>
> This makes me wonder: can we get rid of this RULE? Eric Mertens pointed out
> a trick [1] that's used in the profunctors library to make mapping coerce
> over certain Profunctors more efficient. To adapt this trick for Functor,
> we'd need to add another class method:
>
>     class Functor f where
>         fmap :: (a -> b) -> f a -> f b
>         (<#>) :: Coercible a b => (a -> b) -> f a -> f b
>         (<#>) = \f -> \p -> p `seq` fmap f p
>
> Now, when implementing Functor instances, if we are working with a datatype
> whose role is representational or phantom, we can make (<#>) really fast:
>
>     data List a = Nil | Cons a (List a)
>     instance Functor List where
>         fmap = ...
>         (<#>) = coerce
>
> Now, instead of relying on (map MkNewtype Nil) to rewrite to Nil, we can
> just use (MkNewtype <#> Nil)! No map/coerce RULE necessary :)
>
> OK, I realize that suggesting that we remove the RULE is perhaps a touch
> too far. But it does sting that we have to pay hefty compilation penalties
> because of its existence...
>
> Ryan S.
> -----
> [1]
> http://hackage.haskell.org/package/profunctors-5.2/docs/
> Data-Profunctor-Unsafe.html#v:-35-
> .
>
> On Wed, May 31, 2017 at 7:25 PM, Richard Eisenberg <rae at cs.brynmawr.edu>
> wrote:
>
> >
> > > On May 31, 2017, at 5:21 PM, Ryan Scott <ryan.gl.scott at gmail.com>
> wrote:
> > > Does you know what might be going on here?
> >
> > I think so, but I don't know how to fix it.
> >
> > The commit you found (thank you!) makes simple_opt_expr (the "simple
> > optimizer", run directly after desugaring, even with -O0) a little more
> > selective in what `case` expressions it throws away. Previous to that
> > commit, the optimizer would throw away a `case error "deferred type
> error"
> > of _ -> ...` which is terrible. It seems that you have discovered that we
> > are now too timid in throwing away unhelpful cases. It would be
> interesting
> > to know what the newly-retained cases look like, so that we might throw
> > them away.
> >
> > But there remains a mystery: Why do we need this code at all? Reading
> Note
> > [Getting the map/coerce RULE to work] closely, it seems we need to
> simplify
> >
> >   forall a b (co :: a ~R# b).
> >     let dict = MkCoercible @* @a @b co in
> >     case Coercible_SCSel @* @a @b dict of
> >       _ [Dead] -> map @a @b (\(x :: a) -> case dict of
> >          MkCoercible (co :: a ~R# b) -> x |> co) = let dict = ... in ...
> >
> > to
> >
> >   forall a b (co :: a ~R# b).
> >     map @a @b (\(x :: a) -> x |> co) = \(x :: [a]) -> x |> [co]
> >
> > Part of doing so is to drop the `case Coercible_SCSel ...`, which gets in
> > the way. The mystery is why this needs special code -- shouldn't the
> > eliminate-case-of-known-constructor do the trick? This would require
> > unfolding Coercible_SCSel. Does that happen? It would seem not... but
> maybe
> > it should, which would remove the special-case code that I changed in
> that
> > commit, and quite likely would simplify much more code besides.
> >
> > So: Is Coercible_SCSel unfolded during simple_opt? If not, what wonderful
> > or terrible things happen if we do? If so, why does
> > case-of-known-constructor not work here? My guess is that answering these
> > questions may solve the original problem, but this guess could be wrong.
> >
> > Richard
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170606/f6ebcbcf/attachment.html>

From simonpj at microsoft.com  Wed Jun  7 17:05:38 2017
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Wed, 7 Jun 2017 17:05:38 +0000
Subject: Removing Hoopl dependency?
In-Reply-To: <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
References: <CAEN7k7m7sq4QffdkYiZRVQxbqgW6xy+2xkGYGUG=55Nw250Xtw@mail.gmail.com>
 <DM2PR21MB0028EB04078B385149F7C7AAADF20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
Message-ID: <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>

Michael

Sorry to be slow.

Note that what I’m actually advocating is to *finish* forking Hoopl. The
fork really started in ~2012 when the “new Cmm backend” was being
finished.

Yes, I know.  But what I’m suggesting is to revisit the reasons for that fork, and re-join if possible.  Eg if Hoopl is too slow, can’t we make it faster?  Why is GHC’s version faster?

apart from the performance
(as noted above), there’s the issue of Hoopl’s interface. IMHO the
node-oriented approach taken by Hoopl is both not flexible enough and it
makes it harder to optimize it. That’s why I’ve already changed GHC’s
`Hoopl.Dataflow` module to operate “block-at-a-time”

Well that sounds like an argument to re-engineer Hoopl’s API, rather an argument to fork it.  If it’s a better API, can’t we make it better for everyone?  I don’t yet understand what the “block-oriented” API is, or how it differs, but let’s have the conversation.

When you say
that we should “just fix Hoopl”, it sounds to me that we’d really need
to rewrite it from scratch. And it’s much easier to do that if we can
just experiment within GHC without worrying about breaking other
existing Hoopl users

Fine.  But then let’s call it hoopl2, make it a separate package (perhaps with GHC as its only client for now), and declare that it’s intended to supersede hoopl.

But do we even need to do that much?  After all, a major version bump on a package is allowed to introduce breaking changes to the API.  Anyone who wants the old API can use the old package.

I wonder if you could start a wiki page somewhere (eg on the GHC wiki) listing all the changes you’d like to make in a “rewrite from scratch” story?   That would help to “ground”  the conversation.

Thanks

Simon


From: Michal Terepeta [mailto:michal.terepeta at gmail.com]
Sent: 29 May 2017 12:53
To: Simon Peyton Jones <simonpj at microsoft.com>; ghc-devs <ghc-devs at haskell.org>
Subject: Re: Removing Hoopl dependency?

On Sun, May 28, 2017 at 11:30 PM Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
Is there really a compelling case for forking Hoopl?  I was talking to Kavon last week about doing exactly the opposite: using Hoopl more wholeheartedly!

Before going ahead with this, let’s remember the downsides

•        If we fork Hoopl, improvements in one place will not be seen in the other.  GHC originally used its own containers library but now uses ‘containers’, most of which is irrelevant to GHC, just to pick up the work that has been done to make ‘containers’ fast.  Similarly, GHC has a clone of ‘pretty’, but someone is working (I think) to make GHC use ‘pretty’.

•        It’s not clear to me why GHC has a clone of parts of Hoopl.  Would it not be better just to make Hoopl faster?

If anything I ‘d like to use Hoopl more in Cmm optimisation passes in GHC, so we may want to use more of Hoopl’s facilities.

The main reason you suggest for forking is that there are some awkward name clashes.  Surely we could resolve these? e.g we could change CLabel in GHC; or agree with Hoopl maintainers that BlockId would be more helpful than Label.

You mention that Hoopl uses Unique set/map.  Why not use ‘containers’ for that?  (Like GHC!)

Let’s discuss this a bit more before executing

I’m also interested to know:

•        who is actively working on Hoopl (Michael, Sophie, …)?

•        how are you using it (within GHC, or somewhere else)?

It’d be good to review and update https://ghc.haskell.org/trac/ghc/wiki/Hoopl/Cleanup.  Are there any other improvements planned?
Simon

Hi Simon,

Thanks for chiming in! Let me try to clarify the current situation and
the motivation for my changes.

1) Initial fork of Hoopl

Note that what I’m actually advocating is to *finish* forking Hoopl. The
fork really started in ~2012 when the “new Cmm backend” was being
finished.
IIRC the main reason was the unacceptable performance and it seems that
even Simon Marlow had trouble making it run fast enough:
https://plus.google.com/107890464054636586545/posts/dBbewpRfw6R<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplus.google.com%2F107890464054636586545%2Fposts%2FdBbewpRfw6R&data=02%7C01%7Csimonpj%40microsoft.com%7C4fd225e63df14788371508d4a6893eac%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636316555796004376&sdata=DwwDF8h7lCAaQSzQuEJtaSGgUbvOHjrYEZoonp3BJPA%3D&reserved=0>
https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/HooplPerformance
The end result is pretty sad: GHC has its own forked/specialized
`Hoopl.Dataflow` module and is using Hoopl only for definitions of
`Block`/`Graph` and maps/sets (if you look at my commit, it’s pretty
clear what I’m copying). In particular it’s not using *any* of dataflow
analysis or rewriting capabilities of the Hoopl package.

2) Reasons to finish forking

The reasons I listed in my previous email already assumed the we have
the forked `Hoopl.Dataflow` module in GHC. But if we want to discuss
what are reasons for forking in general, then apart from the performance
(as noted above), there’s the issue of Hoopl’s interface. IMHO the
node-oriented approach taken by Hoopl is both not flexible enough and it
makes it harder to optimize it. That’s why I’ve already changed GHC’s
`Hoopl.Dataflow` module to operate “block-at-a-time”
(https://github.com/ghc/ghc/commit/679ccd1c8860f1ef4b589c9593b74d04c97ae836<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fghc%2Fghc%2Fcommit%2F679ccd1c8860f1ef4b589c9593b74d04c97ae836&data=02%7C01%7Csimonpj%40microsoft.com%7C4fd225e63df14788371508d4a6893eac%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636316555796004376&sdata=85q7qNZ9gpUFxJKEBNo6sd4qnoRV9R6ob0XJWNHuhVM%3D&reserved=0>)
Some concrete examples:
- For proc-point analysis it was necessary to introduce a hack to GHC’s
  `Dataflow` module to expose a separate analysis function that
  *ignores* the middle nodes (since for proc-points they’re irrelevant).
  My change to go “block-at-a-time” allowed us to remove that hack.
- I’m trying to fix non-linearity of `CmmLayoutStack` in
  (https://phabricator.haskell.org/D3586) and again the block-oriented
  interface is useful - I want to do different rewrites based on
  which block is being considered (whether it’s a proc-point or not).
  This is not easily possible if I don’t know which block I’m in (which
  is the case for the node-oriented interface).

I also don’t think that name clashes and the tension between Hoopl’s
interface and GHC are easy to solve. Hoopl is a public, stand-alone
package, so we can’t just change things without considering
compatibility. For instance, we can’t use GHC’s `Unique` in Hoopl. But
should we switch all of GHC to use Hoopl’s? Also having closely related
concepts spread around GHC and Hoopl is not helping when trying to
understand what’s happening. Finally, any changes to both GHC & Hoopl
have much higher overhead than just changing GHC.

In general, it really seems to me that Hoopl has been released simply
too early, with not enough real-world usage and testing. When you say
that we should “just fix Hoopl”, it sounds to me that we’d really need
to rewrite it from scratch. And it’s much easier to do that if we can
just experiment within GHC without worrying about breaking other
existing Hoopl users. Only once we’re happy with the result, we should
be considering separating it into a stand-alone package.

3) Difference between pretty/containers and Hoopl

I also think that the situation with pretty/containers is quite
different than Hoopl. They are much more general-purpose libraries,
*far* more widely used and with more contributors. Take containers - the
package is still very actively developed and constantly improved.
Whereas Hoopl hasn’t really seen much activity in the last 5 years. So
the benefit-cost ratio is much better - yes there is some cost in having
containers as a dependency, but the benefits from the regular stream of
improvements easily outweigh it. I don’t think that’s the case for
Hoopl.

Does this help understand my motivation? Let me know if anything is
still unclear!

Thanks,
Michal

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170607/04fc1324/attachment-0001.html>

From david at well-typed.com  Wed Jun  7 23:46:36 2017
From: david at well-typed.com (David Feuer)
Date: Wed, 07 Jun 2017 19:46:36 -0400
Subject: More TRAC ticket statuses?
Message-ID: <2432148.WGjr1an9mr@squirrel>

There are (at least) two situations that I don't think we currently have a good way to track:

1. A new bug has been verified, but we do not yet have an expect_broken test case.
2. A bug has been fixed, but we are waiting for a test case.

Lacking (1) means that we have to manually dig through the ticket database and try things out in order to knock down tickets that should have already been closed. Lacking (2) means that a ticket could potentially get stuck in "new" status after someone's already done practically all the work required to fix them. What should we do about this?

David

From ben at smart-cactus.org  Thu Jun  8 01:25:44 2017
From: ben at smart-cactus.org (Ben Gamari)
Date: Wed, 07 Jun 2017 21:25:44 -0400
Subject: More TRAC ticket statuses?
In-Reply-To: <2432148.WGjr1an9mr@squirrel>
References: <2432148.WGjr1an9mr@squirrel>
Message-ID: <0CD26909-F24E-494D-8FE5-30176428A36A@smart-cactus.org>

I think ideally both of these are addressed by the existing "testcase" field. However, this admittedly isn't used very consistently. Perhaps we should be more diligent in this regard.


On June 7, 2017 7:46:36 PM EDT, David Feuer <david at well-typed.com> wrote:
>There are (at least) two situations that I don't think we currently
>have a good way to track:
>
>1. A new bug has been verified, but we do not yet have an expect_broken
>test case.
>2. A bug has been fixed, but we are waiting for a test case.
>
>Lacking (1) means that we have to manually dig through the ticket
>database and try things out in order to knock down tickets that should
>have already been closed. Lacking (2) means that a ticket could
>potentially get stuck in "new" status after someone's already done
>practically all the work required to fix them. What should we do about
>this?
>
>David
>_______________________________________________
>ghc-devs mailing list
>ghc-devs at haskell.org
>http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170607/ffb8e4b2/attachment.html>

From michal.terepeta at gmail.com  Thu Jun  8 18:58:46 2017
From: michal.terepeta at gmail.com (Michal Terepeta)
Date: Thu, 08 Jun 2017 18:58:46 +0000
Subject: Removing Hoopl dependency?
In-Reply-To: <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>
References: <CAEN7k7m7sq4QffdkYiZRVQxbqgW6xy+2xkGYGUG=55Nw250Xtw@mail.gmail.com>
 <DM2PR21MB0028EB04078B385149F7C7AAADF20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
 <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>
Message-ID: <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>

> On Wed, Jun 7, 2017 at 7:05 PM Simon Peyton Jones <simonpj at microsoft.com>
wrote:
> Michael
>
> Sorry to be slow.
>
> > Note that what I’m actually advocating is to *finish* forking Hoopl. The
> > fork really started in ~2012 when the “new Cmm backend” was being
> > finished.
>
> Yes, I know.  But what I’m suggesting is to revisit the reasons for that
fork, and re-join if possible.  Eg if Hoopl is too slow, can’t we make it
faster?  Why is GHC’s version faster?
>
> > apart from the performance
> > (as noted above), there’s the issue of Hoopl’s interface. IMHO the
> > node-oriented approach taken by Hoopl is both not flexible enough and it
> > makes it harder to optimize it. That’s why I’ve already changed GHC’s
> > `Hoopl.Dataflow` module to operate “block-at-a-time”
>
> Well that sounds like an argument to re-engineer Hoopl’s API, rather an
argument to fork it.  If it’s a better API, can’t we make it better for
everyone?  I don’t yet understand what the “block-oriented” API is, or how
it differs, but let’s have the conversation.

Sure, but re-engineering the API of a publicly use package has significant
cost for everyone involved:
- GHC: we might need to wait longer for any improvements and spend
  more time discussing various options (and compromises - what makes
  sense for GHC might not make sense for other people)
- Hoopl users: will need to migrate to the new APIs potentially
  multiple times
- Hoopl maintainers: might need to maintain more than one branches of
  Hoopl for a while

And note that just bumping a version number might not be enough.  IIRC
Stackage only allows one version of each package and since Hoopl is a
boot package for GHC, the new version will move to Stackage along with
GHC. So any users of Hoopl that want to use the old package, will not
be able to use that version of Stackage.

> > When you say
> > that we should “just fix Hoopl”, it sounds to me that we’d really need
> > to rewrite it from scratch. And it’s much easier to do that if we can
> > just experiment within GHC without worrying about breaking other
> > existing Hoopl users
>
> Fine.  But then let’s call it hoopl2, make it a separate package (perhaps
with GHC as its only client for now), and declare that it’s intended to
supersede hoopl.

Maybe this is the core of our disagreement - why is it a good idea to
have Hoopl as a separate package in the first place?

I've pointed multiple reasons why I think it has a significant cost.
But I don't really see any major benefits. Looking at the commit
history of Hoopl there hasn't been much development on it since 2012
when Simon M was trying to get the new GHC backend working (since
then, it's mostly maintenance patches to keep up with changes in
`base`, etc).
Extracting a core part of any project to a shared library has some
real costs, so there should be equally real benefits that outweigh
that cost. (If I proposed extracting parts of Core optimizer to a
separate package, wouldn't you expect some really good reasons for
doing this?)
I also do think this is quite different than a dependency on, say,
`binary`, `containers` or `pretty`, where the API of the library is
smaller (at least conceptually), much better understood and
established.

Cheers,
Michal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170608/3fdc2a28/attachment.html>

From ben at smart-cactus.org  Thu Jun  8 19:23:56 2017
From: ben at smart-cactus.org (Ben Gamari)
Date: Thu, 08 Jun 2017 15:23:56 -0400
Subject: Removing Hoopl dependency?
In-Reply-To: <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>
References: <CAEN7k7m7sq4QffdkYiZRVQxbqgW6xy+2xkGYGUG=55Nw250Xtw@mail.gmail.com>
 <DM2PR21MB0028EB04078B385149F7C7AAADF20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
 <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>
Message-ID: <87a85igu5v.fsf@ben-laptop.smart-cactus.org>

Michal Terepeta <michal.terepeta at gmail.com> writes:

> Maybe this is the core of our disagreement - why is it a good idea to
> have Hoopl as a separate package in the first place?
>
> I've pointed multiple reasons why I think it has a significant cost.
> But I don't really see any major benefits. Looking at the commit
> history of Hoopl there hasn't been much development on it since 2012
> when Simon M was trying to get the new GHC backend working (since
> then, it's mostly maintenance patches to keep up with changes in
> `base`, etc).
> Extracting a core part of any project to a shared library has some
> real costs, so there should be equally real benefits that outweigh
> that cost. (If I proposed extracting parts of Core optimizer to a
> separate package, wouldn't you expect some really good reasons for
> doing this?)

One way forward here would be to ask those who would be affected by a
API rework whether they would be open to change. I don't believe there
are too many hoopl users at the moment but I recall that previous
efforts to change the library's interface were met with some resistance.

However, even if we found that hoopl's current user-base is agreeable to
change we would still need to account for the fact that advancing GHC
in lockstep with an out-of-tree hoopl will take more effort than
advancing it under Michal's merge proposal. Admittedly, with submodules
this additional effort isn't too large, but it's still more than having
hoopl and GHC under one tree.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170608/cedd6734/attachment.sig>

From simonpj at microsoft.com  Fri Jun  9 07:50:51 2017
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Fri, 9 Jun 2017 07:50:51 +0000
Subject: Removing Hoopl dependency?
In-Reply-To: <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>
References: <CAEN7k7m7sq4QffdkYiZRVQxbqgW6xy+2xkGYGUG=55Nw250Xtw@mail.gmail.com>
 <DM2PR21MB0028EB04078B385149F7C7AAADF20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
 <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>
Message-ID: <DM2PR21MB00283CF990508580016E7321ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>

Maybe this is the core of our disagreement - why is it a good idea to have Hoopl as a separate package in the first place?

One reason only: because it makes Hoopl usable by compilers other than GHC.  And, dually, efforts by others to improve Hoopl will benefit GHC.

If I proposed extracting parts of Core optimizer to a separate package, wouldn't you expect some really good reasons for doing this?

A re-usable library should be

a)      a significant chunk of code,

b)      that can plausibly be re-purposed by others

c)      and that has an explicable API

I think the Core optimiser is so big, and so GHC specific, that (b) and (c) are unlikely to hold.  But we carefully designed Hoopl from the ground up so that it was agnostic about the node types, and so can be re-used for control flow graphs of many kinds.  It’s designed to be re-usable.  Whether it is actually re-used is another matter, of course.  But if it’s part of GHC, it can’t be.

Stackage only allows one version of each package

I didn’t know that, but I can see it makes sense.  That makes a strong case for re-doing it as a new package hoopl2, if the API needs to change substantially (something we have yet to discuss).

I've pointed multiple reasons why I think it has a significant cost.
Can you just summarise them again briefly for me?  If we are free to choose nomenclature and API for hoopl2, I’m not yet seeing why making it a separate package is harder than not doing so. E.g. template-haskell is a separate package.

Thanks!

Simon



From: Michal Terepeta [mailto:michal.terepeta at gmail.com]
Sent: 08 June 2017 19:59
To: Simon Peyton Jones <simonpj at microsoft.com>; ghc-devs <ghc-devs at haskell.org>
Cc: Kavon Farvardin <kavon at cs.uchicago.edu>
Subject: Re: Removing Hoopl dependency?

> On Wed, Jun 7, 2017 at 7:05 PM Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
> Michael
>
> Sorry to be slow.
>
> > Note that what I’m actually advocating is to *finish* forking Hoopl. The
> > fork really started in ~2012 when the “new Cmm backend” was being
> > finished.
>
> Yes, I know.  But what I’m suggesting is to revisit the reasons for that fork, and re-join if possible.  Eg if Hoopl is too slow, can’t we make it faster?  Why is GHC’s version faster?
>
> > apart from the performance
> > (as noted above), there’s the issue of Hoopl’s interface. IMHO the
> > node-oriented approach taken by Hoopl is both not flexible enough and it
> > makes it harder to optimize it. That’s why I’ve already changed GHC’s
> > `Hoopl.Dataflow` module to operate “block-at-a-time”
>
> Well that sounds like an argument to re-engineer Hoopl’s API, rather an argument to fork it.  If it’s a better API, can’t we make it better for everyone?  I don’t yet understand what the “block-oriented” API is, or how it differs, but let’s have the conversation.

Sure, but re-engineering the API of a publicly use package has significant
cost for everyone involved:
- GHC: we might need to wait longer for any improvements and spend
  more time discussing various options (and compromises - what makes
  sense for GHC might not make sense for other people)
- Hoopl users: will need to migrate to the new APIs potentially
  multiple times
- Hoopl maintainers: might need to maintain more than one branches of
  Hoopl for a while

And note that just bumping a version number might not be enough.  IIRC
Stackage only allows one version of each package and since Hoopl is a
boot package for GHC, the new version will move to Stackage along with
GHC. So any users of Hoopl that want to use the old package, will not
be able to use that version of Stackage.

> > When you say
> > that we should “just fix Hoopl”, it sounds to me that we’d really need
> > to rewrite it from scratch. And it’s much easier to do that if we can
> > just experiment within GHC without worrying about breaking other
> > existing Hoopl users
>
> Fine.  But then let’s call it hoopl2, make it a separate package (perhaps with GHC as its only client for now), and declare that it’s intended to supersede hoopl.

Maybe this is the core of our disagreement - why is it a good idea to
have Hoopl as a separate package in the first place?

I've pointed multiple reasons why I think it has a significant cost.
But I don't really see any major benefits. Looking at the commit
history of Hoopl there hasn't been much development on it since 2012
when Simon M was trying to get the new GHC backend working (since
then, it's mostly maintenance patches to keep up with changes in
`base`, etc).
Extracting a core part of any project to a shared library has some
real costs, so there should be equally real benefits that outweigh
that cost. (If I proposed extracting parts of Core optimizer to a
separate package, wouldn't you expect some really good reasons for
doing this?)
I also do think this is quite different than a dependency on, say,
`binary`, `containers` or `pretty`, where the API of the library is
smaller (at least conceptually), much better understood and
established.

Cheers,
Michal

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170609/454bdf01/attachment-0001.html>

From merijn at inconsistent.nl  Fri Jun  9 08:31:44 2017
From: merijn at inconsistent.nl (Merijn Verstraaten)
Date: Fri, 9 Jun 2017 10:31:44 +0200
Subject: Removing Hoopl dependency?
In-Reply-To: <DM2PR21MB00283CF990508580016E7321ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
References: <CAEN7k7m7sq4QffdkYiZRVQxbqgW6xy+2xkGYGUG=55Nw250Xtw@mail.gmail.com>
 <DM2PR21MB0028EB04078B385149F7C7AAADF20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
 <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>
 <DM2PR21MB00283CF990508580016E7321ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
Message-ID: <BFB2F0A9-A2CC-40D9-9DAE-14652ACC1D7E@inconsistent.nl>

Lemme toss in my 2 cents as an outsider who likes to dabble in programming language and compilers: I would *love* to be able just drop in (parts) of GHC's optimisation into my toy compilers. Optimisation is complicated, lots of work, and not really the part I care about when toying with languages. I wasn't really aware of Hoopl before this thread, so now that I do I'm kinda sad by the idea of this reusable infrastructure being tossed out. I don't really have any vested interest/opinion on how to deal with the current Hoopl situation, so if it's decided to write a Hoopl2.0 instead, without backwards compatibility, I would still consider that a win.

Cheers,
Merijn

> On 9 Jun 2017, at 9:50, Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> wrote:
> 
> Maybe this is the core of our disagreement - why is it a good idea to have Hoopl as a separate package in the first place?
> 
>  
> One reason only: because it makes Hoopl usable by compilers other than GHC.  And, dually, efforts by others to improve Hoopl will benefit GHC.
>  
> If I proposed extracting parts of Core optimizer to a separate package, wouldn't you expect some really good reasons for doing this?
> 
>  
> A re-usable library should be
> a)      a significant chunk of code,
> b)      that can plausibly be re-purposed by others
> c)      and that has an explicable API
>  
> I think the Core optimiser is so big, and so GHC specific, that (b) and (c) are unlikely to hold.  But we carefully designed Hoopl from the ground up so that it was agnostic about the node types, and so can be re-used for control flow graphs of many kinds.  It’s designed to be re-usable.  Whether it is actually re-used is another matter, of course.  But if it’s part of GHC, it can’t be.
>  
> Stackage only allows one version of each package
>  
> I didn’t know that, but I can see it makes sense.  That makes a strong case for re-doing it as a new package hoopl2, if the API needs to change substantially (something we have yet to discuss).
>  
> I've pointed multiple reasons why I think it has a significant cost.
> 
> Can you just summarise them again briefly for me?  If we are free to choose nomenclature and API for hoopl2, I’m not yet seeing why making it a separate package is harder than not doing so. E.g. template-haskell is a separate package.
>  
> Thanks!
>  
> Simon
>  
>  
>  
> From: Michal Terepeta [mailto:michal.terepeta at gmail.com] 
> Sent: 08 June 2017 19:59
> To: Simon Peyton Jones <simonpj at microsoft.com>; ghc-devs <ghc-devs at haskell.org>
> Cc: Kavon Farvardin <kavon at cs.uchicago.edu>
> Subject: Re: Removing Hoopl dependency?
>  
> > On Wed, Jun 7, 2017 at 7:05 PM Simon Peyton Jones <simonpj at microsoft.com> wrote:
> 
> > Michael
> 
> >  
> 
> > Sorry to be slow.
> 
> >  
> 
> > > Note that what I’m actually advocating is to *finish* forking Hoopl. The
> 
> > > fork really started in ~2012 when the “new Cmm backend” was being
> 
> > > finished.
> 
> >  
> 
> > Yes, I know.  But what I’m suggesting is to revisit the reasons for that fork, and re-join if possible.  Eg if Hoopl is too slow, can’t we make it faster?  Why is GHC’s version faster?
> 
> >  
> 
> > > apart from the performance
> 
> > > (as noted above), there’s the issue of Hoopl’s interface. IMHO the
> 
> > > node-oriented approach taken by Hoopl is both not flexible enough and it
> 
> > > makes it harder to optimize it. That’s why I’ve already changed GHC’s
> 
> > > `Hoopl.Dataflow` module to operate “block-at-a-time”
> 
> >  
> 
> > Well that sounds like an argument to re-engineer Hoopl’s API, rather an argument to fork it.  If it’s a better API, can’t we make it better for everyone?  I don’t yet understand what the “block-oriented” API is, or how it differs, but let’s have the conversation.
> 
>  
> 
> Sure, but re-engineering the API of a publicly use package has significant
> 
> cost for everyone involved:
> 
> - GHC: we might need to wait longer for any improvements and spend
> 
>   more time discussing various options (and compromises - what makes
> 
>   sense for GHC might not make sense for other people)
> 
> - Hoopl users: will need to migrate to the new APIs potentially
> 
>   multiple times
> 
> - Hoopl maintainers: might need to maintain more than one branches of
> 
>   Hoopl for a while
> 
>  
> 
> And note that just bumping a version number might not be enough.  IIRC
> 
> Stackage only allows one version of each package and since Hoopl is a
> 
> boot package for GHC, the new version will move to Stackage along with
> 
> GHC. So any users of Hoopl that want to use the old package, will not
> 
> be able to use that version of Stackage.
> 
>  
> 
> > > When you say
> 
> > > that we should “just fix Hoopl”, it sounds to me that we’d really need
> 
> > > to rewrite it from scratch. And it’s much easier to do that if we can
> 
> > > just experiment within GHC without worrying about breaking other
> 
> > > existing Hoopl users
> 
> >  
> 
> > Fine.  But then let’s call it hoopl2, make it a separate package (perhaps with GHC as its only client for now), and declare that it’s intended to supersede hoopl.
> 
>  
> 
> Maybe this is the core of our disagreement - why is it a good idea to
> 
> have Hoopl as a separate package in the first place?
> 
>  
> 
> I've pointed multiple reasons why I think it has a significant cost.
> 
> But I don't really see any major benefits. Looking at the commit
> 
> history of Hoopl there hasn't been much development on it since 2012
> 
> when Simon M was trying to get the new GHC backend working (since
> 
> then, it's mostly maintenance patches to keep up with changes in
> 
> `base`, etc).
> 
> Extracting a core part of any project to a shared library has some
> 
> real costs, so there should be equally real benefits that outweigh
> 
> that cost. (If I proposed extracting parts of Core optimizer to a
> 
> separate package, wouldn't you expect some really good reasons for
> 
> doing this?)
> 
> I also do think this is quite different than a dependency on, say,
> 
> `binary`, `containers` or `pretty`, where the API of the library is
> 
> smaller (at least conceptually), much better understood and
> 
> established.
> 
>  
> 
> Cheers,
> 
> Michal
> 
>  
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From hvriedel at gmail.com  Fri Jun  9 09:16:14 2017
From: hvriedel at gmail.com (Herbert Valerio Riedel)
Date: Fri, 09 Jun 2017 11:16:14 +0200
Subject: Removing Hoopl dependency?
In-Reply-To: <DM2PR21MB00283CF990508580016E7321ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
 (Simon Peyton Jones via ghc-devs's message of "Fri, 9 Jun 2017
 07:50:51 +0000")
References: <CAEN7k7m7sq4QffdkYiZRVQxbqgW6xy+2xkGYGUG=55Nw250Xtw@mail.gmail.com>
 <DM2PR21MB0028EB04078B385149F7C7AAADF20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
 <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>
 <DM2PR21MB00283CF990508580016E7321ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
Message-ID: <87tw3ppllt.fsf@gmail.com>

Hi Simon,

On 2017-06-09 at 09:50:51 +0200, Simon Peyton Jones via ghc-devs wrote:

[...]

>> Stackage only allows one version of each package
>
> I didn’t know that, but I can see it makes sense.  That makes a strong
> case for re-doing it as a new package hoopl2

The limitations of Stackage's design shouldn't drive nor limit
library design. Cabal has been moving to finally allow us to have
multiple versions and even multiple configurations/instances of the same
version of a package registered in the package db at the same time, and
subjecting ourselves to Stackage's limitations after all the work done
(and more in that direction is being considered to push the boundaries
even further) to that effect *now* seems quite backward to me.

If we push the idea to its conclusion, that we shall rather publish a
new package rather than release a new major version of a package to
workaround Stackage, you'd see a proliferation of number-suffixed
packages on Hackage.  Moreover, packages which can easily support
multiple major versions of a package would have to use conditional logic
boilerplate in their .cabal files (which again would be incompatible
with Stackage's inherent limitations, as it allows only *one
configuration* of a given package version).

We should build upon the facilities we already have in place; and major
versions are here to encode the epoch/generation of an API; moreover, as
a big advantage over classic SemVer, we also have this 2-component major
version which gives us more flexibility for versioning during developing
two or more epochs of an API in parallel. So hoopl-1.* and hoopl-2.*
could keep evolving independently, each branch being able to perform
major version increments in their respective version namespace.

Cheers,
  HVR

From alan.zimm at gmail.com  Fri Jun  9 09:26:16 2017
From: alan.zimm at gmail.com (Alan & Kim Zimmerman)
Date: Fri, 9 Jun 2017 11:26:16 +0200
Subject: Removing Hoopl dependency?
In-Reply-To: <87tw3ppllt.fsf@gmail.com>
References: <CAEN7k7m7sq4QffdkYiZRVQxbqgW6xy+2xkGYGUG=55Nw250Xtw@mail.gmail.com>
 <DM2PR21MB0028EB04078B385149F7C7AAADF20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
 <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>
 <DM2PR21MB00283CF990508580016E7321ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <87tw3ppllt.fsf@gmail.com>
Message-ID: <CANma=H-_rXJdor6Er7jbSkw=KSdOY9joU_vQK4TBbn3y+CQT+Q@mail.gmail.com>

But equally, stackage is a major part of the haskell ecosystem.

As such, implications and paths forward need to be considered.

Alan

On 9 June 2017 at 11:16, Herbert Valerio Riedel <hvriedel at gmail.com> wrote:

> Hi Simon,
>
> On 2017-06-09 at 09:50:51 +0200, Simon Peyton Jones via ghc-devs wrote:
>
> [...]
>
> >> Stackage only allows one version of each package
> >
> > I didn’t know that, but I can see it makes sense.  That makes a strong
> > case for re-doing it as a new package hoopl2
>
> The limitations of Stackage's design shouldn't drive nor limit
> library design. Cabal has been moving to finally allow us to have
> multiple versions and even multiple configurations/instances of the same
> version of a package registered in the package db at the same time, and
> subjecting ourselves to Stackage's limitations after all the work done
> (and more in that direction is being considered to push the boundaries
> even further) to that effect *now* seems quite backward to me.
>
> If we push the idea to its conclusion, that we shall rather publish a
> new package rather than release a new major version of a package to
> workaround Stackage, you'd see a proliferation of number-suffixed
> packages on Hackage.  Moreover, packages which can easily support
> multiple major versions of a package would have to use conditional logic
> boilerplate in their .cabal files (which again would be incompatible
> with Stackage's inherent limitations, as it allows only *one
> configuration* of a given package version).
>
> We should build upon the facilities we already have in place; and major
> versions are here to encode the epoch/generation of an API; moreover, as
> a big advantage over classic SemVer, we also have this 2-component major
> version which gives us more flexibility for versioning during developing
> two or more epochs of an API in parallel. So hoopl-1.* and hoopl-2.*
> could keep evolving independently, each branch being able to perform
> major version increments in their respective version namespace.
>
> Cheers,
>   HVR
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170609/dbf1c943/attachment.html>

From michal.terepeta at gmail.com  Fri Jun  9 12:26:33 2017
From: michal.terepeta at gmail.com (Michal Terepeta)
Date: Fri, 09 Jun 2017 12:26:33 +0000
Subject: Removing Hoopl dependency?
In-Reply-To: <DM2PR21MB00283CF990508580016E7321ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
References: <CAEN7k7m7sq4QffdkYiZRVQxbqgW6xy+2xkGYGUG=55Nw250Xtw@mail.gmail.com>
 <DM2PR21MB0028EB04078B385149F7C7AAADF20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
 <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>
 <DM2PR21MB00283CF990508580016E7321ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
Message-ID: <CAEN7k7=QsogQSmf9R=-dcWy8+8asXnJqGCgHgzhF883QfqB1fw@mail.gmail.com>

> On Fri, Jun 9, 2017 at 9:50 AM Simon Peyton Jones <simonpj at microsoft.com>
wrote:
> > Maybe this is the core of our disagreement - why is it a good idea to
have Hoopl as a separate package in the first place?
>
>
> One reason only: because it makes Hoopl usable by compilers other than
GHC.  And, dually, efforts by others to improve Hoopl will benefit GHC.
>
> > If I proposed extracting parts of Core optimizer to a separate package,
wouldn't you expect some really good reasons for doing this?
>
>
> A re-usable library should be
> a)      a significant chunk of code,
> b)      that can plausibly be re-purposed by others
> c)      and that has an explicable API
>
> I think the Core optimiser is so big, and so GHC specific, that (b) and
(c) are unlikely to hold.  But we carefully designed Hoopl from the ground
up so that it was agnostic about the node types, and so can be re-used for
control flow graphs of many kinds.  It’s designed to be re-usable.  Whether
it is actually re-used is another matter, of course.  But if it’s part of
GHC, it can’t be.

I agree with your characterization of a re-usable library and that
Core optimizer would not be a good fit. But I do think that Hoopl also
has some problems with b) and c) (although smaller):
- Using an optimizer-as-a-library is not really common (I'm not aware
  of any compilers doing this, LLVM is to some degree close but it
  exposes the whole language as the interface so it's closer to the
  idea of extracting the whole Cmm backend). So I don't think the API
  for such a project is well understood.
- The API is pretty wide and does put serious constraints on the IR
  (after all it defines blocks and graphs), making reusability
  potentially more tricky.

So I think I understand your argument and we just disagree on whether
this is worth the effort of having a separate package.

>
> [...]
>
> > I've pointed multiple reasons why I think it has a significant cost.
>
> Can you just summarise them again briefly for me?  If we are free to
choose nomenclature and API for hoopl2, I’m not yet seeing why making it a
separate package is harder than not doing so. E.g. template-haskell is a
separate package.

Having even Hoopl2 as a separate package would still entail
additional work:
- Hoopl2 would still need to duplicate some concepts (eg, `Unique`,
  etc. since it needs to be standalone)
- Understanding code (esp. by newcommers) would be harder: the Cmm
  backend would be split between GHC and Hoopl2, with the latter
  necessarily being far more general/polymorphic than needed by GHC.
- Getting the right performance in the presence of all this additional
  generality/polymorphism will likely require fair amount of
  additional work.
- If Hoopl2 is used by other compilers, then we need to be more
  careful changing anything in incompatible ways, this will require
  more discussions & release coordination.

Considering that Hoopl was never actually picked up by other
compilers, I'm not convinced that this cost is justified. But I
understand that other people might have a different opinion.
So how about a compromise:
- decouple GHC from the current Hoopl (ie, go ahead with my diff),
- keep everything Hoopl related only in `compiler/cmm/Hoopl` with the
  long-term intention of creating a separate package,
- experiment with and improve the code,
- once (if?) we're happy with the results, discuss what/how to
  extract to a separate package.
That gives us the freedom to try things out and see what works well
(I simply don't have ready solutions for anything, being able to
experiment is IMHO quite important). And once we reach the right
performance/representation/abstraction/API we can work on extracting
that.

What do you think?

Cheers,
Michal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170609/fe150112/attachment.html>

From simonpj at microsoft.com  Fri Jun  9 16:27:24 2017
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Fri, 9 Jun 2017 16:27:24 +0000
Subject: 8.2.1-rc2 upgrade report
In-Reply-To: <CA+F8DE6nO_K4bYb_EWJ5h8QS+E6rpCotOC7t=GR-5dqbeVjgSQ@mail.gmail.com>
References: <CA+F8DE4jy7N+hZOnTGYYE5OAXm0X2eM64VsmW-JixSSqyBA0_Q@mail.gmail.com>
 <DM2PR21MB0028A2C527F9E83C2DE4E28CADCB0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CA+F8DE6nO_K4bYb_EWJ5h8QS+E6rpCotOC7t=GR-5dqbeVjgSQ@mail.gmail.com>
Message-ID: <DM2PR21MB00281EE94C211F10CB51EEB3ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>

Great.   Could you put all this into a Trac ticket?

Thanks!

Simon

From: Alberto Valverde [mailto:alberto at toscat.net]
Sent: 08 June 2017 13:57
To: Simon Peyton Jones <simonpj at microsoft.com>
Cc: GHC users <glasgow-haskell-users at haskell.org>
Subject: Re: 8.2.1-rc2 upgrade report

Hi Simon,

Thanks for the pointer. I re-did both builds with -dshow-passes and made a small script to plot the results of the lines which summarize the elapsed time and allocated memory per phase and module. I've uploaded the raw logs, a plot of the results and the script I wrote to generate it to https://gist.githubusercontent.com/albertov/145ac5c01bfbadc5c9ff55e9c5c2e50e<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgist.githubusercontent.com%2Falbertov%2F145ac5c01bfbadc5c9ff55e9c5c2e50e&data=02%7C01%7Csimonpj%40microsoft.com%7C3fffbe72f46e458381e308d4ae6de2bc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636325234362588423&sdata=OK11SsW2MmKkNyNHQTzLTxuR8zzEDwg5gddkxPfLiVk%3D&reserved=0>.

The plotted results live here https://gist.githubusercontent.com/albertov/145ac5c01bfbadc5c9ff55e9c5c2e50e/raw/8996644707fc5c18c1d42ad43ee31b1817509384/bench.png<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgist.githubusercontent.com%2Falbertov%2F145ac5c01bfbadc5c9ff55e9c5c2e50e%2Fraw%2F8996644707fc5c18c1d42ad43ee31b1817509384%2Fbench.png&data=02%7C01%7Csimonpj%40microsoft.com%7C3fffbe72f46e458381e308d4ae6de2bc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636325234362588423&sdata=LZR2%2Fdp9s%2FI9xmSkAiqZuCS9PdEZ1DxZWjp8SGt8TRc%3D&reserved=0>

Apparently, the biggest slowdown in respect to 8.0.2 seems to occur in the SpecConstr and Simplifier passes in the Propag (where the "main" function is) and the Sigym4.Propag.Engine (where the main algorithm lives) modules.

Any other tests that would be helpful for me to run? I'm not sure where to start to create a reproducible case but I'll see if I can come up with something soon...

Alberto

On Tue, Jun 6, 2017 at 1:58 PM, Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
Thanks for the report.

Going from 67G to 56G allocation is a very worthwhile improvement in runtime!  Hurrah.

However, trebling compile time is very bad.  It is (I think) far from typical: generally 8.2 is faster at compiling than 8.0 so you must be hitting something weird.  Anything you can do to make a reproducible case would be helpful.  -dshow-passes shows the size of each intermediate form, which at least sometimes shows where the big changes are.

Simon

From: Glasgow-haskell-users [mailto:glasgow-haskell-users-bounces at haskell.org<mailto:glasgow-haskell-users-bounces at haskell.org>] On Behalf Of Alberto Valverde
Sent: 06 June 2017 12:39
To: GHC users <glasgow-haskell-users at haskell.org<mailto:glasgow-haskell-users at haskell.org>>
Subject: 8.2.1-rc2 upgrade report

Hi,

I've finally managed to upgrade all the dependencies of the proprietary app I mentioned some days ago in this list and there are good and bad differences I've noticed between 8.0.2 that I'd like to share.

The bad
-----------

* An optimized cold build (-O2)  is about 3 times slower (~53s vs. ~2m55s) and consumes more memory (~2Gb vs. ~7Gb) at it's peak.

The good
-------------

* An un-optimized cold build (-O0) takes about the same time (~21s, phew! :) It's maybe even slightly faster with 8.2 (too few and badly taken measurements to really know, though)
* The optimized executable is slightly faster and allocates less memory. For this app it makes up for the performance regression of the optimized build (which is almost always done by CI), IMHO.

I did only a couple of runs and only wrote down [1] the last run results (which were similar to the previous results) so take these observations with a grain of salt (except maybe the optimized build slowdown, which doesn't have much margin for variance to be skewing the results). I also measured the peak memory usage by observing "top".

In case gives a clue: The app is a multi-threaded 2D spread simulator which deals with many mmapped Storable mutable vectors and has been pretty optimized for countless hours (I mean by this that it has (too) many INLINE pragmas. Mostly on polymorphic functions to aid in their specialization). I think some of this information can be deduced from the results I'm linking at the footer. I believe the INLINEs are playing a big part of the slowdown since the slowest modules to compile are the "Main" ones which put everything together, along with the typical lens-th-heavy "Types" ones.

I'd like to help by producing a reproducible and isolated benchmark or a better analysis or ... so someone more knowledgeable than me on GHC internals can someday hopefully attack the regression. Any pointers on what would help and where can I learn to do it?

Thanks!


[1] https://gist.github.com/albertov/46fbb13d940f67a569f9a25c1cb8154c<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgist.github.com%2Falbertov%2F46fbb13d940f67a569f9a25c1cb8154c&data=02%7C01%7Csimonpj%40microsoft.com%7Cce8170b01de84bacdab308d4acd0aef3%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636323459695695689&sdata=iRWChz2fzNFuXBAhiYABO5OepSXYBLxvvIQ8CAiABYg%3D&reserved=0>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170609/4d6af233/attachment-0001.html>

From michal.terepeta at gmail.com  Sun Jun 11 12:03:10 2017
From: michal.terepeta at gmail.com (Michal Terepeta)
Date: Sun, 11 Jun 2017 12:03:10 +0000
Subject: WordX/IntX wrap Word#/Int#?
Message-ID: <CAEN7k7=JirvhTO-U+5xYXSZTpyH3Q61OGLskz37a3svcP_+bVg@mail.gmail.com>

Hi all,

I've just noticed that all `WordX` (and `IntX`) data types are
actually implemented as wrappers around `Word#` (and `Int#`). This
probably doesn't matter much if it's stored on the heap (due to
pointer indirection and heap alignment), but it also means that:
```
data Foo = Foo {-# UNPACK #-} !Word8 {-# UNPACK #-} !Int8
```
will actually take *a lot* of space: on 64 bit we'd need 8 bytes for
header, 8 bytes for `Word8`, 8 bytes for `Int8`.

Is there any reason for this? The only thing I can see is that this
avoids having to add things like `Word8#` primitives into the
compiler. (also the codegen would need to emit zero-extend moves when
loading from memory, like `movzb{l,q}`)

If we had things like `Word8#` we could also consider changing `Bool`
to just wrap it (with the obvious encoding). Which would allow to both
UNPACK `Bool` *and* save the size within the struct. (alternatively
one could imagine a `Bool#` that would be just a byte)

I couldn't find any discussion about this, so any pointers would be
welcome. :)

Thanks,
Michal

PS.  I've had a look at it after reading about the recent
implementation of struct field reordering optimization in rustc:
http://camlorn.net/posts/April%202017/rust-struct-field-reordering.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170611/444e3f83/attachment.html>

From sophie at traumapony.org  Sun Jun 11 13:09:23 2017
From: sophie at traumapony.org (Sophie Taylor)
Date: Sun, 11 Jun 2017 13:09:23 +0000
Subject: Removing Hoopl dependency?
In-Reply-To: <CAEN7k7=QsogQSmf9R=-dcWy8+8asXnJqGCgHgzhF883QfqB1fw@mail.gmail.com>
References: <CAEN7k7m7sq4QffdkYiZRVQxbqgW6xy+2xkGYGUG=55Nw250Xtw@mail.gmail.com>
 <DM2PR21MB0028EB04078B385149F7C7AAADF20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
 <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>
 <DM2PR21MB00283CF990508580016E7321ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7=QsogQSmf9R=-dcWy8+8asXnJqGCgHgzhF883QfqB1fw@mail.gmail.com>
Message-ID: <CABLCELPDfnO5mu50J4J8ixQG-cqy0btdwW5t1CER94-DnjjPQA@mail.gmail.com>

Hello, fellow workers!

So, I'll pop in here with my thoughts.

I'm writing an independent intermediate language library for functional
languages, and I looked at using Hoopl. I would use it, but there are
several reasons why I'm not currently doing so:

1) Combining facts from different domains through fancy lattice algorithms.
This is fairly straightforward to add to Hoopl with minimal extra API
change.

2) I wanted to write my data facts as a type-level list, `freer-effects`
style, in order to be more explicit in my types about dependencies between
analyses. This would require significantly altering the API.

3) Its own custom graph code. This is the biggest reason why I decided not
to. Some problems:
  * It seems impossible to change the topology of the graph in a rewriting
step.
  * I wanted to use term hypergraphs/hyperjungles due to some pretty nifty
properties
  * The intermediate language I'm implementing, a derivative of Graph
Reduction Intermediate Notation, aka GRIN from UHC, is, as its name
implies, intrinsically graph-based. Thus, graph manipulation has to be
pretty easy to do.

So instead, I've decided to optimise another hypergraph library
(`graph-rewriting` - I'm going to be rewriting it to use an inductive
representation a la FGL)  and implement a generic, Hoopl-esque analysis
library on top of that. (Or more accurately, that is my plan for the next
six months - I've been sidetracked getting parsing to work nice with an
effect-based stack!)

So, if Hoopl2 does become a thing, I'd be very keen on working on it, but
if I were to actually use it myself, it'd probably require a complete
rewrite. Fortunately, it's a pretty small library; and for GHC, its current
usage is a pretty straightforward usecase which shouldn't be affected too
much. That being said, if GHC were to better use Hoopl (e.g. moving some of
the optimisations on Core to be Hoopl-based passes) then it would be a
different story.

So I guess I'm volunteering to do the rewrite for a potential Hoopl2 if
it's wanted, as I'm about to do pretty much that anyway.

Cheers,
Sophie



On Fri, 9 Jun 2017 at 22:31 Michal Terepeta <michal.terepeta at gmail.com>
wrote:

> > On Fri, Jun 9, 2017 at 9:50 AM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
> > > Maybe this is the core of our disagreement - why is it a good idea to
> have Hoopl as a separate package in the first place?
> >
> >
> > One reason only: because it makes Hoopl usable by compilers other than
> GHC.  And, dually, efforts by others to improve Hoopl will benefit GHC.
> >
> > > If I proposed extracting parts of Core optimizer to a separate
> package, wouldn't you expect some really good reasons for doing this?
> >
> >
> > A re-usable library should be
> > a)      a significant chunk of code,
> > b)      that can plausibly be re-purposed by others
> > c)      and that has an explicable API
> >
> > I think the Core optimiser is so big, and so GHC specific, that (b) and
> (c) are unlikely to hold.  But we carefully designed Hoopl from the ground
> up so that it was agnostic about the node types, and so can be re-used for
> control flow graphs of many kinds.  It’s designed to be re-usable.  Whether
> it is actually re-used is another matter, of course.  But if it’s part of
> GHC, it can’t be.
>
> I agree with your characterization of a re-usable library and that
> Core optimizer would not be a good fit. But I do think that Hoopl also
> has some problems with b) and c) (although smaller):
> - Using an optimizer-as-a-library is not really common (I'm not aware
>   of any compilers doing this, LLVM is to some degree close but it
>   exposes the whole language as the interface so it's closer to the
>   idea of extracting the whole Cmm backend). So I don't think the API
>   for such a project is well understood.
> - The API is pretty wide and does put serious constraints on the IR
>   (after all it defines blocks and graphs), making reusability
>   potentially more tricky.
>
> So I think I understand your argument and we just disagree on whether
> this is worth the effort of having a separate package.
>
> >
> > [...]
> >
> > > I've pointed multiple reasons why I think it has a significant cost.
> >
> > Can you just summarise them again briefly for me?  If we are free to
> choose nomenclature and API for hoopl2, I’m not yet seeing why making it a
> separate package is harder than not doing so. E.g. template-haskell is a
> separate package.
>
> Having even Hoopl2 as a separate package would still entail
> additional work:
> - Hoopl2 would still need to duplicate some concepts (eg, `Unique`,
>   etc. since it needs to be standalone)
> - Understanding code (esp. by newcommers) would be harder: the Cmm
>   backend would be split between GHC and Hoopl2, with the latter
>   necessarily being far more general/polymorphic than needed by GHC.
> - Getting the right performance in the presence of all this additional
>   generality/polymorphism will likely require fair amount of
>   additional work.
> - If Hoopl2 is used by other compilers, then we need to be more
>   careful changing anything in incompatible ways, this will require
>   more discussions & release coordination.
>
> Considering that Hoopl was never actually picked up by other
> compilers, I'm not convinced that this cost is justified. But I
> understand that other people might have a different opinion.
> So how about a compromise:
> - decouple GHC from the current Hoopl (ie, go ahead with my diff),
> - keep everything Hoopl related only in `compiler/cmm/Hoopl` with the
>   long-term intention of creating a separate package,
> - experiment with and improve the code,
> - once (if?) we're happy with the results, discuss what/how to
>   extract to a separate package.
> That gives us the freedom to try things out and see what works well
> (I simply don't have ready solutions for anything, being able to
> experiment is IMHO quite important). And once we reach the right
> performance/representation/abstraction/API we can work on extracting
> that.
>
> What do you think?
>
> Cheers,
> Michal
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170611/7b741828/attachment.html>

From ben at smart-cactus.org  Sun Jun 11 14:44:47 2017
From: ben at smart-cactus.org (Ben Gamari)
Date: Sun, 11 Jun 2017 10:44:47 -0400
Subject: WordX/IntX wrap Word#/Int#?
In-Reply-To: <CAEN7k7=JirvhTO-U+5xYXSZTpyH3Q61OGLskz37a3svcP_+bVg@mail.gmail.com>
References: <CAEN7k7=JirvhTO-U+5xYXSZTpyH3Q61OGLskz37a3svcP_+bVg@mail.gmail.com>
Message-ID: <82569CFF-956D-4DA2-A04D-933195D5B6A6@smart-cactus.org>



On June 11, 2017 8:03:10 AM EDT, Michal Terepeta <michal.terepeta at gmail.com> wrote:
>Hi all,
>
>I've just noticed that all `WordX` (and `IntX`) data types are
>actually implemented as wrappers around `Word#` (and `Int#`). This
>probably doesn't matter much if it's stored on the heap (due to
>pointer indirection and heap alignment), but it also means that:
>```
>data Foo = Foo {-# UNPACK #-} !Word8 {-# UNPACK #-} !Int8
>```
>will actually take *a lot* of space: on 64 bit we'd need 8 bytes for
>header, 8 bytes for `Word8`, 8 bytes for `Int8`.
>
>Is there any reason for this? The only thing I can see is that this
>avoids having to add things like `Word8#` primitives into the
>compiler. (also the codegen would need to emit zero-extend moves when
>loading from memory, like `movzb{l,q}`)
>
This is certainly one consideration. Another is that you would also need to teach the garbage collector to understand closures with sub-word-size fields. Currently we can encode whether each field of a closure is a pointer or not with a simple bitmap. If we naively allowed smaller fields we would need to increase the granularity of this representation to encode bytes.

Of course, one way to work around this would be to impose an invariant that guarantees that pointers are always word-aligned. Then we would probably want to shuffle sub-word sized fields, allowing two Word16s to inhabit a single word.

As you mention, this would no doubt require a bit of engineering. In particular, while x86 has robust support for sub-word-size operations, I don't believe all the platforms we support do. I these cases we would need to perform, for instance, aligned word-sized loads and stores and mask as appropriate. I may be wrong, however.

Another consideration is that the byte code interpreter would need to learn to understand these closures.

Regardless, Simon Marlow began some work in this direction a few years ago. There is a mostly complete patch in D38. All it needs is rebasing, fixing of the byte code interpreter, and then perhaps introduction of Word8# and friends. I think it would be great if we could make our heap representation a bit more space-conscious. Perhaps you could open a ticket so we collect these tidbits?

Another somewhat related issue that would be good think about in parallel to this issue is the treatment of the word-sized dependence of Word. See #11953.

Cheers,

- Ben


-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

From mail at joachim-breitner.de  Sun Jun 11 21:44:13 2017
From: mail at joachim-breitner.de (Joachim Breitner)
Date: Sun, 11 Jun 2017 23:44:13 +0200
Subject: WordX/IntX wrap Word#/Int#?
In-Reply-To: <82569CFF-956D-4DA2-A04D-933195D5B6A6@smart-cactus.org>
References: <CAEN7k7=JirvhTO-U+5xYXSZTpyH3Q61OGLskz37a3svcP_+bVg@mail.gmail.com>
 <82569CFF-956D-4DA2-A04D-933195D5B6A6@smart-cactus.org>
Message-ID: <1497217453.14706.1.camel@joachim-breitner.de>

Hi,

Am Sonntag, den 11.06.2017, 10:44 -0400 schrieb Ben Gamari:
> This is certainly one consideration. Another is that you would also
> need to teach the garbage collector to understand closures with sub-
> word-size fields. Currently we can encode whether each field of a
> closure is a pointer or not with a simple bitmap. If we naively
> allowed smaller fields we would need to increase the granularity of
> this representation to encode bytes.
> 
> Of course, one way to work around this would be to impose an
> invariant that guarantees that pointers are always word-aligned. Then
> we would probably want to shuffle sub-word sized fields, allowing two
> Word16s to inhabit a single word.

that is not an issue; we already sort field into pointers first, and
non-pointers later. So all pointers are at the beginning and nicely
aligned, and all the non-pointer data can follow in whatever weird
format. The GC only needs to know how many words in total are used by
the non-pointer data.


Greetings,
Joachim
-- 
Joachim “nomeata” Breitner
  mail at joachim-breitner.de • https://www.joachim-breitner.de/
  XMPP: nomeata at joachim-breitner.de • OpenPGP-Key: 0xF0FBF51F
  Debian Developer: nomeata at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170611/efb3f443/attachment.sig>

From ben at smart-cactus.org  Mon Jun 12 01:43:16 2017
From: ben at smart-cactus.org (Ben Gamari)
Date: Sun, 11 Jun 2017 21:43:16 -0400
Subject: WordX/IntX wrap Word#/Int#?
In-Reply-To: <1497217453.14706.1.camel@joachim-breitner.de>
References: <CAEN7k7=JirvhTO-U+5xYXSZTpyH3Q61OGLskz37a3svcP_+bVg@mail.gmail.com>
 <82569CFF-956D-4DA2-A04D-933195D5B6A6@smart-cactus.org>
 <1497217453.14706.1.camel@joachim-breitner.de>
Message-ID: <87r2yqf0az.fsf@ben-laptop.smart-cactus.org>

Joachim Breitner <mail at joachim-breitner.de> writes:

> Hi,
>
> Am Sonntag, den 11.06.2017, 10:44 -0400 schrieb Ben Gamari:
>> This is certainly one consideration. Another is that you would also
>> need to teach the garbage collector to understand closures with sub-
>> word-size fields. Currently we can encode whether each field of a
>> closure is a pointer or not with a simple bitmap. If we naively
>> allowed smaller fields we would need to increase the granularity of
>> this representation to encode bytes.
>> 
>> Of course, one way to work around this would be to impose an
>> invariant that guarantees that pointers are always word-aligned. Then
>> we would probably want to shuffle sub-word sized fields, allowing two
>> Word16s to inhabit a single word.
>
> that is not an issue; we already sort field into pointers first, and
> non-pointers later. So all pointers are at the beginning and nicely
> aligned, and all the non-pointer data can follow in whatever weird
> format. The GC only needs to know how many words in total are used by
> the non-pointer data.
>
Ahh, great point. I stand corrected.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170611/1b174073/attachment-0001.sig>

From simonpj at microsoft.com  Mon Jun 12 07:06:58 2017
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 12 Jun 2017 07:06:58 +0000
Subject: Removing Hoopl dependency?
In-Reply-To: <CABLCELPDfnO5mu50J4J8ixQG-cqy0btdwW5t1CER94-DnjjPQA@mail.gmail.com>
References: <CAEN7k7m7sq4QffdkYiZRVQxbqgW6xy+2xkGYGUG=55Nw250Xtw@mail.gmail.com>
 <DM2PR21MB0028EB04078B385149F7C7AAADF20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
 <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>
 <DM2PR21MB00283CF990508580016E7321ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7=QsogQSmf9R=-dcWy8+8asXnJqGCgHgzhF883QfqB1fw@mail.gmail.com>
 <CABLCELPDfnO5mu50J4J8ixQG-cqy0btdwW5t1CER94-DnjjPQA@mail.gmail.com>
Message-ID: <DM2PR21MB0028A4F30EF31322F5793BAAADCD0@DM2PR21MB0028.namprd21.prod.outlook.com>

Interesting!

Maybe there are a couple of different alternatives:


A.      A rewrite of Hoopl, with all the same basic ideas and data structures, but with a better API (I’m not sure exactly in what way, but Michael has some idea, as does Sophie), and a more efficient implementation.

B.      A more radical change to use hypergraphs, type-level lists etc.   This sounds interesting, but it’s a more substantial change and before using it for GHC we’d need to discuss the new proposed API in some detail

There’s no reason we couldn’t do (A) and (B) in parallel.

Michael is suggesting doing (A) in GHC’s tree, but with a clearly-declared intent to bring it out as a separate library.   (I’d advocate making it a separate library in GHC’s tree; we already have a number of those.

That would leave Sophie free to do (B) free of the constraints of GHC depending on it; but we could always use it later.

Does that sound plausible?  Do we know of any other Hoopl users?

Simon

From: Sophie Taylor [mailto:sophie at traumapony.org]
Sent: 11 June 2017 14:09
To: Michal Terepeta <michal.terepeta at gmail.com>; Simon Peyton Jones <simonpj at microsoft.com>; ghc-devs <ghc-devs at haskell.org>
Cc: Kavon Farvardin <kavon at cs.uchicago.edu>
Subject: Re: Removing Hoopl dependency?

Hello, fellow workers!

So, I'll pop in here with my thoughts.

I'm writing an independent intermediate language library for functional languages, and I looked at using Hoopl. I would use it, but there are several reasons why I'm not currently doing so:

1) Combining facts from different domains through fancy lattice algorithms. This is fairly straightforward to add to Hoopl with minimal extra API change.

2) I wanted to write my data facts as a type-level list, `freer-effects` style, in order to be more explicit in my types about dependencies between analyses. This would require significantly altering the API.

3) Its own custom graph code. This is the biggest reason why I decided not to. Some problems:
  * It seems impossible to change the topology of the graph in a rewriting step.
  * I wanted to use term hypergraphs/hyperjungles due to some pretty nifty properties
  * The intermediate language I'm implementing, a derivative of Graph Reduction Intermediate Notation, aka GRIN from UHC, is, as its name implies, intrinsically graph-based. Thus, graph manipulation has to be pretty easy to do.

So instead, I've decided to optimise another hypergraph library (`graph-rewriting` - I'm going to be rewriting it to use an inductive representation a la FGL)  and implement a generic, Hoopl-esque analysis library on top of that. (Or more accurately, that is my plan for the next six months - I've been sidetracked getting parsing to work nice with an effect-based stack!)

So, if Hoopl2 does become a thing, I'd be very keen on working on it, but if I were to actually use it myself, it'd probably require a complete rewrite. Fortunately, it's a pretty small library; and for GHC, its current usage is a pretty straightforward usecase which shouldn't be affected too much. That being said, if GHC were to better use Hoopl (e.g. moving some of the optimisations on Core to be Hoopl-based passes) then it would be a different story.

So I guess I'm volunteering to do the rewrite for a potential Hoopl2 if it's wanted, as I'm about to do pretty much that anyway.

Cheers,
Sophie



On Fri, 9 Jun 2017 at 22:31 Michal Terepeta <michal.terepeta at gmail.com<mailto:michal.terepeta at gmail.com>> wrote:
> On Fri, Jun 9, 2017 at 9:50 AM Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
> > Maybe this is the core of our disagreement - why is it a good idea to have Hoopl as a separate package in the first place?
>
>
> One reason only: because it makes Hoopl usable by compilers other than GHC.  And, dually, efforts by others to improve Hoopl will benefit GHC.
>
> > If I proposed extracting parts of Core optimizer to a separate package, wouldn't you expect some really good reasons for doing this?
>
>
> A re-usable library should be
> a)      a significant chunk of code,
> b)      that can plausibly be re-purposed by others
> c)      and that has an explicable API
>
> I think the Core optimiser is so big, and so GHC specific, that (b) and (c) are unlikely to hold.  But we carefully designed Hoopl from the ground up so that it was agnostic about the node types, and so can be re-used for control flow graphs of many kinds.  It’s designed to be re-usable.  Whether it is actually re-used is another matter, of course.  But if it’s part of GHC, it can’t be.

I agree with your characterization of a re-usable library and that
Core optimizer would not be a good fit. But I do think that Hoopl also
has some problems with b) and c) (although smaller):
- Using an optimizer-as-a-library is not really common (I'm not aware
  of any compilers doing this, LLVM is to some degree close but it
  exposes the whole language as the interface so it's closer to the
  idea of extracting the whole Cmm backend). So I don't think the API
  for such a project is well understood.
- The API is pretty wide and does put serious constraints on the IR
  (after all it defines blocks and graphs), making reusability
  potentially more tricky.

So I think I understand your argument and we just disagree on whether
this is worth the effort of having a separate package.

>
> [...]
>
> > I've pointed multiple reasons why I think it has a significant cost.
>
> Can you just summarise them again briefly for me?  If we are free to choose nomenclature and API for hoopl2, I’m not yet seeing why making it a separate package is harder than not doing so. E.g. template-haskell is a separate package.

Having even Hoopl2 as a separate package would still entail
additional work:
- Hoopl2 would still need to duplicate some concepts (eg, `Unique`,
  etc. since it needs to be standalone)
- Understanding code (esp. by newcommers) would be harder: the Cmm
  backend would be split between GHC and Hoopl2, with the latter
  necessarily being far more general/polymorphic than needed by GHC.
- Getting the right performance in the presence of all this additional
  generality/polymorphism will likely require fair amount of
  additional work.
- If Hoopl2 is used by other compilers, then we need to be more
  careful changing anything in incompatible ways, this will require
  more discussions & release coordination.

Considering that Hoopl was never actually picked up by other
compilers, I'm not convinced that this cost is justified. But I
understand that other people might have a different opinion.
So how about a compromise:
- decouple GHC from the current Hoopl (ie, go ahead with my diff),
- keep everything Hoopl related only in `compiler/cmm/Hoopl` with the
  long-term intention of creating a separate package,
- experiment with and improve the code,
- once (if?) we're happy with the results, discuss what/how to
  extract to a separate package.
That gives us the freedom to try things out and see what works well
(I simply don't have ready solutions for anything, being able to
experiment is IMHO quite important). And once we reach the right
performance/representation/abstraction/API we can work on extracting
that.

What do you think?

Cheers,
Michal

_______________________________________________
ghc-devs mailing list
ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=02%7C01%7Csimonpj%40microsoft.com%7Cd747eec3caa74856abe408d4b0cb1b80%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636327833778402907&sdata=XF%2FDDgrIvni6kMJQg0ubJXtVtfXUp1HLifUBz2RTxJ4%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170612/cf798f7b/attachment-0001.html>

From sophie at traumapony.org  Mon Jun 12 08:50:09 2017
From: sophie at traumapony.org (Sophie Taylor)
Date: Mon, 12 Jun 2017 08:50:09 +0000
Subject: Removing Hoopl dependency?
In-Reply-To: <DM2PR21MB0028A4F30EF31322F5793BAAADCD0@DM2PR21MB0028.namprd21.prod.outlook.com>
References: <CAEN7k7m7sq4QffdkYiZRVQxbqgW6xy+2xkGYGUG=55Nw250Xtw@mail.gmail.com>
 <DM2PR21MB0028EB04078B385149F7C7AAADF20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
 <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>
 <DM2PR21MB00283CF990508580016E7321ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7=QsogQSmf9R=-dcWy8+8asXnJqGCgHgzhF883QfqB1fw@mail.gmail.com>
 <CABLCELPDfnO5mu50J4J8ixQG-cqy0btdwW5t1CER94-DnjjPQA@mail.gmail.com>
 <DM2PR21MB0028A4F30EF31322F5793BAAADCD0@DM2PR21MB0028.namprd21.prod.outlook.com>
Message-ID: <CABLCELPxXEYtCMwQU3Rdnk9DFWBSE-i_hnJB52pKJW1wJESrrQ@mail.gmail.com>

I don't see why not, other than possible duplication of effort when it
comes to some of the basic algorithms.

Speaking of which, what policies are there on bringing in new dependencies
to GHC, both compile-time and run-time (e.g. possible SMT solver support)?



On Mon, 12 Jun 2017 at 17:07 Simon Peyton Jones <simonpj at microsoft.com>
wrote:

> Interesting!
>
>
>
> Maybe there are a couple of different alternatives:
>
>
>
> A.      A rewrite of Hoopl, with all the same basic ideas and data
> structures, but with a better API (I’m not sure exactly in what way, but
> Michael has some idea, as does Sophie), and a more efficient implementation.
>
> B.      A more radical change to use hypergraphs, type-level lists etc.
> This sounds interesting, but it’s a more substantial change and before
> using it for GHC we’d need to discuss the new proposed API in some detail
>
>
>
> There’s no reason we couldn’t do (A) and (B) in parallel.
>
>
>
> Michael is suggesting doing (A) in GHC’s tree, but with a clearly-declared
> intent to bring it out as a separate library.   (I’d advocate *making* it
> a separate library in GHC’s tree; we already have a number of those.
>
>
>
> That would leave Sophie free to do (B) free of the constraints of GHC
> depending on it; but we could always use it later.
>
>
>
> Does that sound plausible?  Do we know of any other Hoopl users?
>
>
>
> Simon
>
>
>
> *From:* Sophie Taylor [mailto:sophie at traumapony.org]
> *Sent:* 11 June 2017 14:09
> *To:* Michal Terepeta <michal.terepeta at gmail.com>; Simon Peyton Jones <
> simonpj at microsoft.com>; ghc-devs <ghc-devs at haskell.org>
>
>
> *Cc:* Kavon Farvardin <kavon at cs.uchicago.edu>
> *Subject:* Re: Removing Hoopl dependency?
>
>
>
> Hello, fellow workers!
>
>
>
> So, I'll pop in here with my thoughts.
>
>
>
> I'm writing an independent intermediate language library for functional
> languages, and I looked at using Hoopl. I would use it, but there are
> several reasons why I'm not currently doing so:
>
>
>
> 1) Combining facts from different domains through fancy lattice
> algorithms. This is fairly straightforward to add to Hoopl with minimal
> extra API change.
>
>
>
> 2) I wanted to write my data facts as a type-level list, `freer-effects`
> style, in order to be more explicit in my types about dependencies between
> analyses. This would require significantly altering the API.
>
>
>
> 3) Its own custom graph code. This is the biggest reason why I decided not
> to. Some problems:
>
>   * It seems impossible to change the topology of the graph in a rewriting
> step.
>
>   * I wanted to use term hypergraphs/hyperjungles due to some pretty nifty
> properties
>
>   * The intermediate language I'm implementing, a derivative of Graph
> Reduction Intermediate Notation, aka GRIN from UHC, is, as its name
> implies, intrinsically graph-based. Thus, graph manipulation has to be
> pretty easy to do.
>
>
>
> So instead, I've decided to optimise another hypergraph library
> (`graph-rewriting` - I'm going to be rewriting it to use an inductive
> representation a la FGL)  and implement a generic, Hoopl-esque analysis
> library on top of that. (Or more accurately, that is my plan for the next
> six months - I've been sidetracked getting parsing to work nice with an
> effect-based stack!)
>
>
>
> So, if Hoopl2 does become a thing, I'd be very keen on working on it, but
> if I were to actually use it myself, it'd probably require a complete
> rewrite. Fortunately, it's a pretty small library; and for GHC, its current
> usage is a pretty straightforward usecase which shouldn't be affected too
> much. That being said, if GHC were to better use Hoopl (e.g. moving some of
> the optimisations on Core to be Hoopl-based passes) then it would be a
> different story.
>
>
>
> So I guess I'm volunteering to do the rewrite for a potential Hoopl2 if
> it's wanted, as I'm about to do pretty much that anyway.
>
>
>
> Cheers,
>
> Sophie
>
>
>
>
>
>
>
> On Fri, 9 Jun 2017 at 22:31 Michal Terepeta <michal.terepeta at gmail.com>
> wrote:
>
> > On Fri, Jun 9, 2017 at 9:50 AM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> > > Maybe this is the core of our disagreement - why is it a good idea to
> have Hoopl as a separate package in the first place?
>
> >
>
> >
>
> > One reason only: because it makes Hoopl usable by compilers other than
> GHC.  And, dually, efforts by others to improve Hoopl will benefit GHC.
>
> >
>
> > > If I proposed extracting parts of Core optimizer to a separate
> package, wouldn't you expect some really good reasons for doing this?
>
> >
>
> >
>
> > A re-usable library should be
>
> > a)      a significant chunk of code,
>
> > b)      that can plausibly be re-purposed by others
>
> > c)      and that has an explicable API
>
> >
>
> > I think the Core optimiser is so big, and so GHC specific, that (b) and
> (c) are unlikely to hold.  But we carefully designed Hoopl from the ground
> up so that it was agnostic about the node types, and so can be re-used for
> control flow graphs of many kinds.  It’s designed to be re-usable.  Whether
> it is actually re-used is another matter, of course.  But if it’s part of
> GHC, it can’t be.
>
>
>
> I agree with your characterization of a re-usable library and that
>
> Core optimizer would not be a good fit. But I do think that Hoopl also
>
> has some problems with b) and c) (although smaller):
>
> - Using an optimizer-as-a-library is not really common (I'm not aware
>
>   of any compilers doing this, LLVM is to some degree close but it
>
>   exposes the whole language as the interface so it's closer to the
>
>   idea of extracting the whole Cmm backend). So I don't think the API
>
>   for such a project is well understood.
>
> - The API is pretty wide and does put serious constraints on the IR
>
>   (after all it defines blocks and graphs), making reusability
>
>   potentially more tricky.
>
>
>
> So I think I understand your argument and we just disagree on whether
>
> this is worth the effort of having a separate package.
>
>
>
> >
>
> > [...]
>
> >
>
> > > I've pointed multiple reasons why I think it has a significant cost.
>
> >
>
> > Can you just summarise them again briefly for me?  If we are free to
> choose nomenclature and API for hoopl2, I’m not yet seeing why making it a
> separate package is harder than not doing so. E.g. template-haskell is a
> separate package.
>
>
>
> Having even Hoopl2 as a separate package would still entail
>
> additional work:
>
> - Hoopl2 would still need to duplicate some concepts (eg, `Unique`,
>
>   etc. since it needs to be standalone)
>
> - Understanding code (esp. by newcommers) would be harder: the Cmm
>
>   backend would be split between GHC and Hoopl2, with the latter
>
>   necessarily being far more general/polymorphic than needed by GHC.
>
> - Getting the right performance in the presence of all this additional
>
>   generality/polymorphism will likely require fair amount of
>
>   additional work.
>
> - If Hoopl2 is used by other compilers, then we need to be more
>
>   careful changing anything in incompatible ways, this will require
>
>   more discussions & release coordination.
>
>
>
> Considering that Hoopl was never actually picked up by other
>
> compilers, I'm not convinced that this cost is justified. But I
>
> understand that other people might have a different opinion.
>
> So how about a compromise:
>
> - decouple GHC from the current Hoopl (ie, go ahead with my diff),
>
> - keep everything Hoopl related only in `compiler/cmm/Hoopl` with the
>
>   long-term intention of creating a separate package,
>
> - experiment with and improve the code,
>
> - once (if?) we're happy with the results, discuss what/how to
>
>   extract to a separate package.
>
> That gives us the freedom to try things out and see what works well
>
> (I simply don't have ready solutions for anything, being able to
>
> experiment is IMHO quite important). And once we reach the right
>
> performance/representation/abstraction/API we can work on extracting
>
> that.
>
>
>
> What do you think?
>
>
>
> Cheers,
>
> Michal
>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=02%7C01%7Csimonpj%40microsoft.com%7Cd747eec3caa74856abe408d4b0cb1b80%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636327833778402907&sdata=XF%2FDDgrIvni6kMJQg0ubJXtVtfXUp1HLifUBz2RTxJ4%3D&reserved=0>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170612/13c01c19/attachment-0001.html>

From ben at smart-cactus.org  Mon Jun 12 18:05:51 2017
From: ben at smart-cactus.org (Ben Gamari)
Date: Mon, 12 Jun 2017 14:05:51 -0400
Subject: Removing Hoopl dependency?
In-Reply-To: <DM2PR21MB0028A4F30EF31322F5793BAAADCD0@DM2PR21MB0028.namprd21.prod.outlook.com>
References: <CAEN7k7m7sq4QffdkYiZRVQxbqgW6xy+2xkGYGUG=55Nw250Xtw@mail.gmail.com>
 <DM2PR21MB0028EB04078B385149F7C7AAADF20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
 <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>
 <DM2PR21MB00283CF990508580016E7321ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7=QsogQSmf9R=-dcWy8+8asXnJqGCgHgzhF883QfqB1fw@mail.gmail.com>
 <CABLCELPDfnO5mu50J4J8ixQG-cqy0btdwW5t1CER94-DnjjPQA@mail.gmail.com>
 <DM2PR21MB0028A4F30EF31322F5793BAAADCD0@DM2PR21MB0028.namprd21.prod.outlook.com>
Message-ID: <87h8zlf5ds.fsf@ben-laptop.smart-cactus.org>

Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:

Snip
>
> That would leave Sophie free to do (B) free of the constraints of GHC
> depending on it; but we could always use it later.
>
> Does that sound plausible?  Do we know of any other Hoopl users?

CCing Ning, who is currently maintaining hoopl and I believe has some
projects using it.

Ning, you may want to have a look through this thread if you haven't
already seen it. You can find the previous messages in the list archive [1].

Cheers,

- Ben


[1] May messages: https://mail.haskell.org/pipermail/ghc-devs/2017-May/014255.html
    June messages: https://mail.haskell.org/pipermail/ghc-devs/2017-June/014293.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170612/a2c5d61a/attachment.sig>

From michal.terepeta at gmail.com  Mon Jun 12 18:12:52 2017
From: michal.terepeta at gmail.com (Michal Terepeta)
Date: Mon, 12 Jun 2017 18:12:52 +0000
Subject: Removing Hoopl dependency?
In-Reply-To: <87h8zlf5ds.fsf@ben-laptop.smart-cactus.org>
References: <CAEN7k7m7sq4QffdkYiZRVQxbqgW6xy+2xkGYGUG=55Nw250Xtw@mail.gmail.com>
 <DM2PR21MB0028EB04078B385149F7C7AAADF20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
 <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>
 <DM2PR21MB00283CF990508580016E7321ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7=QsogQSmf9R=-dcWy8+8asXnJqGCgHgzhF883QfqB1fw@mail.gmail.com>
 <CABLCELPDfnO5mu50J4J8ixQG-cqy0btdwW5t1CER94-DnjjPQA@mail.gmail.com>
 <DM2PR21MB0028A4F30EF31322F5793BAAADCD0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <87h8zlf5ds.fsf@ben-laptop.smart-cactus.org>
Message-ID: <CAEN7k7kXPVeR=4pxVK2T68t3rgUje9vv0=2i2JPvQZ=qC_WcAA@mail.gmail.com>

> On Mon, Jun 12, 2017 at 8:05 PM Ben Gamari <ben at smart-cactus.org> wrote:
> Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:
>
> Snip
> >
> > That would leave Sophie free to do (B) free of the constraints of GHC
> > depending on it; but we could always use it later.
> >
> > Does that sound plausible?  Do we know of any other Hoopl users?
>
> CCing Ning, who is currently maintaining hoopl and I believe has some
> projects using it.
>
> Ning, you may want to have a look through this thread if you haven't
> already seen it. You can find the previous messages in the list archive
[1].
>
> Cheers,
>
> - Ben

Based on [1] there are four public packages:
- ethereum-analyzer,
- linearscan-hoopl,
- llvm-analysis,
- text-show-instances

But there might be more that are not open-source/uploaded to
hackage/stackage.

Cheers,
Michal

[1] https://www.stackage.org/lts-8.18/package/hoopl-3.10.2.1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170612/ee69ad02/attachment.html>

From ben at smart-cactus.org  Mon Jun 12 18:19:59 2017
From: ben at smart-cactus.org (Ben Gamari)
Date: Mon, 12 Jun 2017 14:19:59 -0400
Subject: Removing Hoopl dependency?
In-Reply-To: <CABLCELPxXEYtCMwQU3Rdnk9DFWBSE-i_hnJB52pKJW1wJESrrQ@mail.gmail.com>
References: <CAEN7k7m7sq4QffdkYiZRVQxbqgW6xy+2xkGYGUG=55Nw250Xtw@mail.gmail.com>
 <DM2PR21MB0028EB04078B385149F7C7AAADF20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
 <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>
 <DM2PR21MB00283CF990508580016E7321ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7=QsogQSmf9R=-dcWy8+8asXnJqGCgHgzhF883QfqB1fw@mail.gmail.com>
 <CABLCELPDfnO5mu50J4J8ixQG-cqy0btdwW5t1CER94-DnjjPQA@mail.gmail.com>
 <DM2PR21MB0028A4F30EF31322F5793BAAADCD0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CABLCELPxXEYtCMwQU3Rdnk9DFWBSE-i_hnJB52pKJW1wJESrrQ@mail.gmail.com>
Message-ID: <87efupf4q8.fsf@ben-laptop.smart-cactus.org>

Sophie Taylor <sophie at traumapony.org> writes:

> I don't see why not, other than possible duplication of effort when it
> comes to some of the basic algorithms.
>
> Speaking of which, what policies are there on bringing in new dependencies
> to GHC, both compile-time and run-time (e.g. possible SMT solver support)?
>
We are generally fairly conservative with adding new dependencies of
either type. There are a variety of reasons for this:

In the case of runtime dependencies the associated costs are fairly
clear: it would either be a) harder for users to use GHC (in the case of
mandatory dependencies) or, b) make it harder to follow the behavior of
the compiler (in the case of optional dependencies discovered at
runtime).

There are also costs in the case of compile-time dependencies, although
they may not be as easy to see. First, in order to 
maintain a reproducible revision history GHC includes all dependent
libraries as submodules and ships them with source distributions. These
submodules carry a small but non-negligible cost to developers due to
idiosyncracies in how they are handled by both git and Phabricator.
Moreover, we need to periodically bump these submodules, which
inevitably brings integration issues which require coordination with
upstream to fix.

Also, there is a significant synchronization overhead associated with
getting upstream maintainers to release new library versions prior to a
GHC release. While this generally only affects the release manager, for
that person it is indeed a significant cost and does tend to slow down
the release cycle.

Finally, dependencies of the `ghc` library affects users of tooling which
links to it (e.g. ghc-mod). Specifically, since we can only link against
a single version of a given package at a time, such tooling packages are
forced to link against whatever version `ghc` depends upon. This means
that users won't get bugfixes and can constrain install plans, sometimes
to the point where no plan is possible.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170612/378fbfac/attachment.sig>

From michal.terepeta at gmail.com  Mon Jun 12 18:45:34 2017
From: michal.terepeta at gmail.com (Michal Terepeta)
Date: Mon, 12 Jun 2017 18:45:34 +0000
Subject: WordX/IntX wrap Word#/Int#?
In-Reply-To: <87r2yqf0az.fsf@ben-laptop.smart-cactus.org>
References: <CAEN7k7=JirvhTO-U+5xYXSZTpyH3Q61OGLskz37a3svcP_+bVg@mail.gmail.com>
 <82569CFF-956D-4DA2-A04D-933195D5B6A6@smart-cactus.org>
 <1497217453.14706.1.camel@joachim-breitner.de>
 <87r2yqf0az.fsf@ben-laptop.smart-cactus.org>
Message-ID: <CAEN7k7mXSNF-LkQKd+h8cVwm4jDwgAEfDsHaP6KuTf-aPTBUxA@mail.gmail.com>

Thanks a lot for the replies & links!

I'll try to finish Simon's diff (and probably ask silly questions if I get
stuck ;)

Cheers,
Michal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170612/4411728a/attachment.html>

From ben at well-typed.com  Mon Jun 12 19:03:18 2017
From: ben at well-typed.com (Ben Gamari)
Date: Mon, 12 Jun 2017 15:03:18 -0400
Subject: 8.2.1 Status
Message-ID: <87a85df2q1.fsf@ben-laptop.smart-cactus.org>

Hello everyone,

GHC 8.2.1 release candidate 2 has been out for a few weeks now and,
while it turned up a number of issues, it seems that the release is
(slowly) solidifying. Currently the ghc-8.2 branch has nearly 50 commits
since the -rc2 tag, fixing many of the issues that have been reported.
While we were hoping that 8.2 would require only two release candidates,
it seems pretty clear that a third is in order. 

At this point, there are still a few more tickets in need of 
resolution before -rc3 release candidate can be cut. Hopefully these
will be sorted out in the coming week, which means that we can cut a new
source release by mid-next week, with the binary releases coming a week
later.

Unfortunately this all means that we'll realistically be looking at a
8.2.1 release date sometime near the beginning of July. As always, help
is greatly appreciated; drop by #ghc if you would like to help squash
some bugs.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170612/f8448af2/attachment.sig>

From simonpj at microsoft.com  Mon Jun 12 21:48:01 2017
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 12 Jun 2017 21:48:01 +0000
Subject: Removing Hoopl dependency?
In-Reply-To: <CABLCELPxXEYtCMwQU3Rdnk9DFWBSE-i_hnJB52pKJW1wJESrrQ@mail.gmail.com>
References: <CAEN7k7m7sq4QffdkYiZRVQxbqgW6xy+2xkGYGUG=55Nw250Xtw@mail.gmail.com>
 <DM2PR21MB0028EB04078B385149F7C7AAADF20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
 <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>
 <DM2PR21MB00283CF990508580016E7321ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7=QsogQSmf9R=-dcWy8+8asXnJqGCgHgzhF883QfqB1fw@mail.gmail.com>
 <CABLCELPDfnO5mu50J4J8ixQG-cqy0btdwW5t1CER94-DnjjPQA@mail.gmail.com>
 <DM2PR21MB0028A4F30EF31322F5793BAAADCD0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CABLCELPxXEYtCMwQU3Rdnk9DFWBSE-i_hnJB52pKJW1wJESrrQ@mail.gmail.com>
Message-ID: <BY2PR21MB0020FD04E309821509675AA9ADCD0@BY2PR21MB0020.namprd21.prod.outlook.com>

Speaking of which, what policies are there on bringing in new dependencies to GHC, both compile-time and run-time (e.g. possible SMT solver support)?

We don’t have a formal policy, but we are generally reluctant to take on new dependencies.   For SMT solvers, Iavor is using one via a typechecker plugin.

Simon

From: Sophie Taylor [mailto:sophie at traumapony.org]
Sent: 12 June 2017 09:50
To: Simon Peyton Jones <simonpj at microsoft.com>; Michal Terepeta <michal.terepeta at gmail.com>; ghc-devs <ghc-devs at haskell.org>
Cc: Kavon Farvardin <kavon at cs.uchicago.edu>
Subject: Re: Removing Hoopl dependency?

I don't see why not, other than possible duplication of effort when it comes to some of the basic algorithms.

Speaking of which, what policies are there on bringing in new dependencies to GHC, both compile-time and run-time (e.g. possible SMT solver support)?



On Mon, 12 Jun 2017 at 17:07 Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
Interesting!

Maybe there are a couple of different alternatives:


A.      A rewrite of Hoopl, with all the same basic ideas and data structures, but with a better API (I’m not sure exactly in what way, but Michael has some idea, as does Sophie), and a more efficient implementation.

B.      A more radical change to use hypergraphs, type-level lists etc.   This sounds interesting, but it’s a more substantial change and before using it for GHC we’d need to discuss the new proposed API in some detail

There’s no reason we couldn’t do (A) and (B) in parallel.

Michael is suggesting doing (A) in GHC’s tree, but with a clearly-declared intent to bring it out as a separate library.   (I’d advocate making it a separate library in GHC’s tree; we already have a number of those.

That would leave Sophie free to do (B) free of the constraints of GHC depending on it; but we could always use it later.

Does that sound plausible?  Do we know of any other Hoopl users?

Simon

From: Sophie Taylor [mailto:sophie at traumapony.org<mailto:sophie at traumapony.org>]
Sent: 11 June 2017 14:09
To: Michal Terepeta <michal.terepeta at gmail.com<mailto:michal.terepeta at gmail.com>>; Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>; ghc-devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>

Cc: Kavon Farvardin <kavon at cs.uchicago.edu<mailto:kavon at cs.uchicago.edu>>
Subject: Re: Removing Hoopl dependency?

Hello, fellow workers!

So, I'll pop in here with my thoughts.

I'm writing an independent intermediate language library for functional languages, and I looked at using Hoopl. I would use it, but there are several reasons why I'm not currently doing so:

1) Combining facts from different domains through fancy lattice algorithms. This is fairly straightforward to add to Hoopl with minimal extra API change.

2) I wanted to write my data facts as a type-level list, `freer-effects` style, in order to be more explicit in my types about dependencies between analyses. This would require significantly altering the API.

3) Its own custom graph code. This is the biggest reason why I decided not to. Some problems:
  * It seems impossible to change the topology of the graph in a rewriting step.
  * I wanted to use term hypergraphs/hyperjungles due to some pretty nifty properties
  * The intermediate language I'm implementing, a derivative of Graph Reduction Intermediate Notation, aka GRIN from UHC, is, as its name implies, intrinsically graph-based. Thus, graph manipulation has to be pretty easy to do.

So instead, I've decided to optimise another hypergraph library (`graph-rewriting` - I'm going to be rewriting it to use an inductive representation a la FGL)  and implement a generic, Hoopl-esque analysis library on top of that. (Or more accurately, that is my plan for the next six months - I've been sidetracked getting parsing to work nice with an effect-based stack!)

So, if Hoopl2 does become a thing, I'd be very keen on working on it, but if I were to actually use it myself, it'd probably require a complete rewrite. Fortunately, it's a pretty small library; and for GHC, its current usage is a pretty straightforward usecase which shouldn't be affected too much. That being said, if GHC were to better use Hoopl (e.g. moving some of the optimisations on Core to be Hoopl-based passes) then it would be a different story.

So I guess I'm volunteering to do the rewrite for a potential Hoopl2 if it's wanted, as I'm about to do pretty much that anyway.

Cheers,
Sophie



On Fri, 9 Jun 2017 at 22:31 Michal Terepeta <michal.terepeta at gmail.com<mailto:michal.terepeta at gmail.com>> wrote:
> On Fri, Jun 9, 2017 at 9:50 AM Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
> > Maybe this is the core of our disagreement - why is it a good idea to have Hoopl as a separate package in the first place?
>
>
> One reason only: because it makes Hoopl usable by compilers other than GHC.  And, dually, efforts by others to improve Hoopl will benefit GHC.
>
> > If I proposed extracting parts of Core optimizer to a separate package, wouldn't you expect some really good reasons for doing this?
>
>
> A re-usable library should be
> a)      a significant chunk of code,
> b)      that can plausibly be re-purposed by others
> c)      and that has an explicable API
>
> I think the Core optimiser is so big, and so GHC specific, that (b) and (c) are unlikely to hold.  But we carefully designed Hoopl from the ground up so that it was agnostic about the node types, and so can be re-used for control flow graphs of many kinds.  It’s designed to be re-usable.  Whether it is actually re-used is another matter, of course.  But if it’s part of GHC, it can’t be.

I agree with your characterization of a re-usable library and that
Core optimizer would not be a good fit. But I do think that Hoopl also
has some problems with b) and c) (although smaller):
- Using an optimizer-as-a-library is not really common (I'm not aware
  of any compilers doing this, LLVM is to some degree close but it
  exposes the whole language as the interface so it's closer to the
  idea of extracting the whole Cmm backend). So I don't think the API
  for such a project is well understood.
- The API is pretty wide and does put serious constraints on the IR
  (after all it defines blocks and graphs), making reusability
  potentially more tricky.

So I think I understand your argument and we just disagree on whether
this is worth the effort of having a separate package.

>
> [...]
>
> > I've pointed multiple reasons why I think it has a significant cost.
>
> Can you just summarise them again briefly for me?  If we are free to choose nomenclature and API for hoopl2, I’m not yet seeing why making it a separate package is harder than not doing so. E.g. template-haskell is a separate package.

Having even Hoopl2 as a separate package would still entail
additional work:
- Hoopl2 would still need to duplicate some concepts (eg, `Unique`,
  etc. since it needs to be standalone)
- Understanding code (esp. by newcommers) would be harder: the Cmm
  backend would be split between GHC and Hoopl2, with the latter
  necessarily being far more general/polymorphic than needed by GHC.
- Getting the right performance in the presence of all this additional
  generality/polymorphism will likely require fair amount of
  additional work.
- If Hoopl2 is used by other compilers, then we need to be more
  careful changing anything in incompatible ways, this will require
  more discussions & release coordination.

Considering that Hoopl was never actually picked up by other
compilers, I'm not convinced that this cost is justified. But I
understand that other people might have a different opinion.
So how about a compromise:
- decouple GHC from the current Hoopl (ie, go ahead with my diff),
- keep everything Hoopl related only in `compiler/cmm/Hoopl` with the
  long-term intention of creating a separate package,
- experiment with and improve the code,
- once (if?) we're happy with the results, discuss what/how to
  extract to a separate package.
That gives us the freedom to try things out and see what works well
(I simply don't have ready solutions for anything, being able to
experiment is IMHO quite important). And once we reach the right
performance/representation/abstraction/API we can work on extracting
that.

What do you think?

Cheers,
Michal

_______________________________________________
ghc-devs mailing list
ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=02%7C01%7Csimonpj%40microsoft.com%7Cd747eec3caa74856abe408d4b0cb1b80%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636327833778402907&sdata=XF%2FDDgrIvni6kMJQg0ubJXtVtfXUp1HLifUBz2RTxJ4%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170612/1fed675d/attachment-0001.html>

From sophie at traumapony.org  Mon Jun 12 23:58:04 2017
From: sophie at traumapony.org (Sophie Taylor)
Date: Mon, 12 Jun 2017 23:58:04 +0000
Subject: Removing Hoopl dependency?
In-Reply-To: <BY2PR21MB0020FD04E309821509675AA9ADCD0@BY2PR21MB0020.namprd21.prod.outlook.com>
References: <CAEN7k7m7sq4QffdkYiZRVQxbqgW6xy+2xkGYGUG=55Nw250Xtw@mail.gmail.com>
 <DM2PR21MB0028EB04078B385149F7C7AAADF20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7k_hA-U-t9-+OaJ1XjecAz2+6yAB3TXrfJHJAxNp_V=Zw@mail.gmail.com>
 <DM2PR21MB002899055C9344CF75173464ADC80@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7mX118FoOVcBAQcUDSAPk+2MTKLGg6eK-bLLDJ+=rS+Pw@mail.gmail.com>
 <DM2PR21MB00283CF990508580016E7321ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CAEN7k7=QsogQSmf9R=-dcWy8+8asXnJqGCgHgzhF883QfqB1fw@mail.gmail.com>
 <CABLCELPDfnO5mu50J4J8ixQG-cqy0btdwW5t1CER94-DnjjPQA@mail.gmail.com>
 <DM2PR21MB0028A4F30EF31322F5793BAAADCD0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CABLCELPxXEYtCMwQU3Rdnk9DFWBSE-i_hnJB52pKJW1wJESrrQ@mail.gmail.com>
 <BY2PR21MB0020FD04E309821509675AA9ADCD0@BY2PR21MB0020.namprd21.prod.outlook.com>
Message-ID: <CABLCELNVb8Jz=dqJmWHk8P2U5KcbQV7LU=PaGToNMt=+ifxaVA@mail.gmail.com>

Ben, Simon,

Thanks, that's good to know!

On Tue, 13 Jun 2017 at 07:48 Simon Peyton Jones <simonpj at microsoft.com>
wrote:

> Speaking of which, what policies are there on bringing in new dependencies
> to GHC, both compile-time and run-time (e.g. possible SMT solver support)?
>
>
>
> We don’t have a formal policy, but we are generally reluctant to take on
> new dependencies.   For SMT solvers, Iavor is using one via a typechecker
> plugin.
>
>
>
> Simon
>
>
>
> *From:* Sophie Taylor [mailto:sophie at traumapony.org]
> *Sent:* 12 June 2017 09:50
> *To:* Simon Peyton Jones <simonpj at microsoft.com>; Michal Terepeta <
> michal.terepeta at gmail.com>; ghc-devs <ghc-devs at haskell.org>
>
>
> *Cc:* Kavon Farvardin <kavon at cs.uchicago.edu>
> *Subject:* Re: Removing Hoopl dependency?
>
>
>
> I don't see why not, other than possible duplication of effort when it
> comes to some of the basic algorithms.
>
>
>
> Speaking of which, what policies are there on bringing in new dependencies
> to GHC, both compile-time and run-time (e.g. possible SMT solver support)?
>
>
>
>
>
>
>
> On Mon, 12 Jun 2017 at 17:07 Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> Interesting!
>
>
>
> Maybe there are a couple of different alternatives:
>
>
>
> A.      A rewrite of Hoopl, with all the same basic ideas and data
> structures, but with a better API (I’m not sure exactly in what way, but
> Michael has some idea, as does Sophie), and a more efficient implementation.
>
> B.      A more radical change to use hypergraphs, type-level lists etc.
> This sounds interesting, but it’s a more substantial change and before
> using it for GHC we’d need to discuss the new proposed API in some detail
>
>
>
> There’s no reason we couldn’t do (A) and (B) in parallel.
>
>
>
> Michael is suggesting doing (A) in GHC’s tree, but with a clearly-declared
> intent to bring it out as a separate library.   (I’d advocate *making* it
> a separate library in GHC’s tree; we already have a number of those.
>
>
>
> That would leave Sophie free to do (B) free of the constraints of GHC
> depending on it; but we could always use it later.
>
>
>
> Does that sound plausible?  Do we know of any other Hoopl users?
>
>
>
> Simon
>
>
>
> *From:* Sophie Taylor [mailto:sophie at traumapony.org]
> *Sent:* 11 June 2017 14:09
> *To:* Michal Terepeta <michal.terepeta at gmail.com>; Simon Peyton Jones <
> simonpj at microsoft.com>; ghc-devs <ghc-devs at haskell.org>
>
>
> *Cc:* Kavon Farvardin <kavon at cs.uchicago.edu>
> *Subject:* Re: Removing Hoopl dependency?
>
>
>
> Hello, fellow workers!
>
>
>
> So, I'll pop in here with my thoughts.
>
>
>
> I'm writing an independent intermediate language library for functional
> languages, and I looked at using Hoopl. I would use it, but there are
> several reasons why I'm not currently doing so:
>
>
>
> 1) Combining facts from different domains through fancy lattice
> algorithms. This is fairly straightforward to add to Hoopl with minimal
> extra API change.
>
>
>
> 2) I wanted to write my data facts as a type-level list, `freer-effects`
> style, in order to be more explicit in my types about dependencies between
> analyses. This would require significantly altering the API.
>
>
>
> 3) Its own custom graph code. This is the biggest reason why I decided not
> to. Some problems:
>
>   * It seems impossible to change the topology of the graph in a rewriting
> step.
>
>   * I wanted to use term hypergraphs/hyperjungles due to some pretty nifty
> properties
>
>   * The intermediate language I'm implementing, a derivative of Graph
> Reduction Intermediate Notation, aka GRIN from UHC, is, as its name
> implies, intrinsically graph-based. Thus, graph manipulation has to be
> pretty easy to do.
>
>
>
> So instead, I've decided to optimise another hypergraph library
> (`graph-rewriting` - I'm going to be rewriting it to use an inductive
> representation a la FGL)  and implement a generic, Hoopl-esque analysis
> library on top of that. (Or more accurately, that is my plan for the next
> six months - I've been sidetracked getting parsing to work nice with an
> effect-based stack!)
>
>
>
> So, if Hoopl2 does become a thing, I'd be very keen on working on it, but
> if I were to actually use it myself, it'd probably require a complete
> rewrite. Fortunately, it's a pretty small library; and for GHC, its current
> usage is a pretty straightforward usecase which shouldn't be affected too
> much. That being said, if GHC were to better use Hoopl (e.g. moving some of
> the optimisations on Core to be Hoopl-based passes) then it would be a
> different story.
>
>
>
> So I guess I'm volunteering to do the rewrite for a potential Hoopl2 if
> it's wanted, as I'm about to do pretty much that anyway.
>
>
>
> Cheers,
>
> Sophie
>
>
>
>
>
>
>
> On Fri, 9 Jun 2017 at 22:31 Michal Terepeta <michal.terepeta at gmail.com>
> wrote:
>
> > On Fri, Jun 9, 2017 at 9:50 AM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> > > Maybe this is the core of our disagreement - why is it a good idea to
> have Hoopl as a separate package in the first place?
>
> >
>
> >
>
> > One reason only: because it makes Hoopl usable by compilers other than
> GHC.  And, dually, efforts by others to improve Hoopl will benefit GHC.
>
> >
>
> > > If I proposed extracting parts of Core optimizer to a separate
> package, wouldn't you expect some really good reasons for doing this?
>
> >
>
> >
>
> > A re-usable library should be
>
> > a)      a significant chunk of code,
>
> > b)      that can plausibly be re-purposed by others
>
> > c)      and that has an explicable API
>
> >
>
> > I think the Core optimiser is so big, and so GHC specific, that (b) and
> (c) are unlikely to hold.  But we carefully designed Hoopl from the ground
> up so that it was agnostic about the node types, and so can be re-used for
> control flow graphs of many kinds.  It’s designed to be re-usable.  Whether
> it is actually re-used is another matter, of course.  But if it’s part of
> GHC, it can’t be.
>
>
>
> I agree with your characterization of a re-usable library and that
>
> Core optimizer would not be a good fit. But I do think that Hoopl also
>
> has some problems with b) and c) (although smaller):
>
> - Using an optimizer-as-a-library is not really common (I'm not aware
>
>   of any compilers doing this, LLVM is to some degree close but it
>
>   exposes the whole language as the interface so it's closer to the
>
>   idea of extracting the whole Cmm backend). So I don't think the API
>
>   for such a project is well understood.
>
> - The API is pretty wide and does put serious constraints on the IR
>
>   (after all it defines blocks and graphs), making reusability
>
>   potentially more tricky.
>
>
>
> So I think I understand your argument and we just disagree on whether
>
> this is worth the effort of having a separate package.
>
>
>
> >
>
> > [...]
>
> >
>
> > > I've pointed multiple reasons why I think it has a significant cost.
>
> >
>
> > Can you just summarise them again briefly for me?  If we are free to
> choose nomenclature and API for hoopl2, I’m not yet seeing why making it a
> separate package is harder than not doing so. E.g. template-haskell is a
> separate package.
>
>
>
> Having even Hoopl2 as a separate package would still entail
>
> additional work:
>
> - Hoopl2 would still need to duplicate some concepts (eg, `Unique`,
>
>   etc. since it needs to be standalone)
>
> - Understanding code (esp. by newcommers) would be harder: the Cmm
>
>   backend would be split between GHC and Hoopl2, with the latter
>
>   necessarily being far more general/polymorphic than needed by GHC.
>
> - Getting the right performance in the presence of all this additional
>
>   generality/polymorphism will likely require fair amount of
>
>   additional work.
>
> - If Hoopl2 is used by other compilers, then we need to be more
>
>   careful changing anything in incompatible ways, this will require
>
>   more discussions & release coordination.
>
>
>
> Considering that Hoopl was never actually picked up by other
>
> compilers, I'm not convinced that this cost is justified. But I
>
> understand that other people might have a different opinion.
>
> So how about a compromise:
>
> - decouple GHC from the current Hoopl (ie, go ahead with my diff),
>
> - keep everything Hoopl related only in `compiler/cmm/Hoopl` with the
>
>   long-term intention of creating a separate package,
>
> - experiment with and improve the code,
>
> - once (if?) we're happy with the results, discuss what/how to
>
>   extract to a separate package.
>
> That gives us the freedom to try things out and see what works well
>
> (I simply don't have ready solutions for anything, being able to
>
> experiment is IMHO quite important). And once we reach the right
>
> performance/representation/abstraction/API we can work on extracting
>
> that.
>
>
>
> What do you think?
>
>
>
> Cheers,
>
> Michal
>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=02%7C01%7Csimonpj%40microsoft.com%7Cd747eec3caa74856abe408d4b0cb1b80%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636327833778402907&sdata=XF%2FDDgrIvni6kMJQg0ubJXtVtfXUp1HLifUBz2RTxJ4%3D&reserved=0>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170612/d08db666/attachment-0001.html>

From carter.schonwald at gmail.com  Tue Jun 13 19:34:37 2017
From: carter.schonwald at gmail.com (Carter Schonwald)
Date: Tue, 13 Jun 2017 15:34:37 -0400
Subject: 8.2.1-rc2 upgrade report
In-Reply-To: <DM2PR21MB00281EE94C211F10CB51EEB3ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
References: <CA+F8DE4jy7N+hZOnTGYYE5OAXm0X2eM64VsmW-JixSSqyBA0_Q@mail.gmail.com>
 <DM2PR21MB0028A2C527F9E83C2DE4E28CADCB0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <CA+F8DE6nO_K4bYb_EWJ5h8QS+E6rpCotOC7t=GR-5dqbeVjgSQ@mail.gmail.com>
 <DM2PR21MB00281EE94C211F10CB51EEB3ADCE0@DM2PR21MB0028.namprd21.prod.outlook.com>
Message-ID: <CAHYVw0wpxOfJXYqZHAeKOk9O5oDFtfagkyL-FgqVZENdYOBOjQ@mail.gmail.com>

8.2 has much more aggressive specialization of polymorphic functions at
their use sites,
is it possible that the build performance would improve and the runtime
improvements would remain, if you experimentally removed all the inline
pragmas?

On Fri, Jun 9, 2017 at 12:27 PM, Simon Peyton Jones via ghc-devs <
ghc-devs at haskell.org> wrote:

> Great.   Could you put all this into a Trac ticket?
>
>
>
> Thanks!
>
>
>
> Simon
>
>
>
> *From:* Alberto Valverde [mailto:alberto at toscat.net]
> *Sent:* 08 June 2017 13:57
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Cc:* GHC users <glasgow-haskell-users at haskell.org>
> *Subject:* Re: 8.2.1-rc2 upgrade report
>
>
>
> Hi Simon,
>
>
>
> Thanks for the pointer. I re-did both builds with -dshow-passes and made a
> small script to plot the results of the lines which summarize the elapsed
> time and allocated memory per phase and module. I've uploaded the raw logs,
> a plot of the results and the script I wrote to generate it to
> https://gist.githubusercontent.com/albertov/145ac5c01bfbadc5c9ff55e9c5c2e5
> 0e
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgist.githubusercontent.com%2Falbertov%2F145ac5c01bfbadc5c9ff55e9c5c2e50e&data=02%7C01%7Csimonpj%40microsoft.com%7C3fffbe72f46e458381e308d4ae6de2bc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636325234362588423&sdata=OK11SsW2MmKkNyNHQTzLTxuR8zzEDwg5gddkxPfLiVk%3D&reserved=0>
> .
>
>
>
> The plotted results live here https://gist.githubusercontent.com/albertov/
> 145ac5c01bfbadc5c9ff55e9c5c2e50e/raw/8996644707fc5c18c1d42ad43ee31b
> 1817509384/bench.png
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgist.githubusercontent.com%2Falbertov%2F145ac5c01bfbadc5c9ff55e9c5c2e50e%2Fraw%2F8996644707fc5c18c1d42ad43ee31b1817509384%2Fbench.png&data=02%7C01%7Csimonpj%40microsoft.com%7C3fffbe72f46e458381e308d4ae6de2bc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636325234362588423&sdata=LZR2%2Fdp9s%2FI9xmSkAiqZuCS9PdEZ1DxZWjp8SGt8TRc%3D&reserved=0>
>
>
>
> Apparently, the biggest slowdown in respect to 8.0.2 seems to occur in the
> SpecConstr and Simplifier passes in the Propag (where the "main" function
> is) and the Sigym4.Propag.Engine (where the main algorithm lives) modules.
>
>
>
> Any other tests that would be helpful for me to run? I'm not sure where to
> start to create a reproducible case but I'll see if I can come up with
> something soon...
>
>
>
> Alberto
>
>
>
> On Tue, Jun 6, 2017 at 1:58 PM, Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> Thanks for the report.
>
>
>
> Going from 67G to 56G allocation is a very worthwhile improvement in
> runtime!  Hurrah.
>
>
>
> However, trebling compile time is very bad.  It is (I think) far from
> typical: generally 8.2 is *faster* at compiling than 8.0 so you must be
> hitting something weird.  Anything you can do to make a reproducible case
> would be helpful.  -dshow-passes shows the size of each intermediate form,
> which at least sometimes shows where the big changes are.
>
>
>
> Simon
>
>
>
> *From:* Glasgow-haskell-users [mailto:glasgow-haskell-users-
> bounces at haskell.org] *On Behalf Of *Alberto Valverde
> *Sent:* 06 June 2017 12:39
> *To:* GHC users <glasgow-haskell-users at haskell.org>
> *Subject:* 8.2.1-rc2 upgrade report
>
>
>
> Hi,
>
>
>
> I've finally managed to upgrade all the dependencies of the proprietary
> app I mentioned some days ago in this list and there are good and bad
> differences I've noticed between 8.0.2 that I'd like to share.
>
>
>
> The bad
>
> -----------
>
>
>
> * An optimized cold build (-O2)  is about 3 times slower (~53s vs. ~2m55s)
> and consumes more memory (~2Gb vs. ~7Gb) at it's peak.
>
>
>
> The good
>
> -------------
>
>
>
> * An un-optimized cold build (-O0) takes about the same time (~21s, phew!
> :) It's maybe even slightly faster with 8.2 (too few and badly taken
> measurements to really know, though)
>
> * The optimized executable is slightly faster and allocates less memory.
> For this app it makes up for the performance regression of the optimized
> build (which is almost always done by CI), IMHO.
>
>
>
> I did only a couple of runs and only wrote down [1] the last run results
> (which were similar to the previous results) so take these observations
> with a grain of salt (except maybe the optimized build slowdown, which
> doesn't have much margin for variance to be skewing the results). I also
> measured the peak memory usage by observing "top".
>
>
>
> In case gives a clue: The app is a multi-threaded 2D spread simulator
> which deals with many mmapped Storable mutable vectors and has been pretty
> optimized for countless hours (I mean by this that it has (too) many INLINE
> pragmas. Mostly on polymorphic functions to aid in their specialization). I
> think some of this information can be deduced from the results I'm linking
> at the footer. I believe the INLINEs are playing a big part of the slowdown
> since the slowest modules to compile are the "Main" ones which put
> everything together, along with the typical lens-th-heavy "Types" ones.
>
>
>
> I'd like to help by producing a reproducible and isolated benchmark or a
> better analysis or ... so someone more knowledgeable than me on GHC
> internals can someday hopefully attack the regression. Any pointers on what
> would help and where can I learn to do it?
>
>
>
> Thanks!
>
>
>
>
>
> [1] https://gist.github.com/albertov/46fbb13d940f67a569f9a25c1cb8154c
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgist.github.com%2Falbertov%2F46fbb13d940f67a569f9a25c1cb8154c&data=02%7C01%7Csimonpj%40microsoft.com%7Cce8170b01de84bacdab308d4acd0aef3%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636323459695695689&sdata=iRWChz2fzNFuXBAhiYABO5OepSXYBLxvvIQ8CAiABYg%3D&reserved=0>
>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170613/f5a3bab9/attachment.html>

From michal.terepeta at gmail.com  Tue Jun 13 19:38:08 2017
From: michal.terepeta at gmail.com (Michal Terepeta)
Date: Tue, 13 Jun 2017 19:38:08 +0000
Subject: WordX/IntX wrap Word#/Int#?
In-Reply-To: <CAEN7k7mXSNF-LkQKd+h8cVwm4jDwgAEfDsHaP6KuTf-aPTBUxA@mail.gmail.com>
References: <CAEN7k7=JirvhTO-U+5xYXSZTpyH3Q61OGLskz37a3svcP_+bVg@mail.gmail.com>
 <82569CFF-956D-4DA2-A04D-933195D5B6A6@smart-cactus.org>
 <1497217453.14706.1.camel@joachim-breitner.de>
 <87r2yqf0az.fsf@ben-laptop.smart-cactus.org>
 <CAEN7k7mXSNF-LkQKd+h8cVwm4jDwgAEfDsHaP6KuTf-aPTBUxA@mail.gmail.com>
Message-ID: <CAEN7k7kcS69G6ryXsFN9q8c_XJcUkgGpXCMdptp69H-ZoOj51g@mail.gmail.com>

Just for the record, I've opened:
https://ghc.haskell.org/trac/ghc/ticket/13825
to track this.

Cheers,
Michal

On Mon, Jun 12, 2017 at 8:45 PM Michal Terepeta <michal.terepeta at gmail.com>
wrote:

> Thanks a lot for the replies & links!
>
> I'll try to finish Simon's diff (and probably ask silly questions if I get
> stuck ;)
>
> Cheers,
> Michal
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170613/2dab59b2/attachment.html>

From email at ningwang.org  Wed Jun 14 01:24:36 2017
From: email at ningwang.org (ning w)
Date: Tue, 13 Jun 2017 18:24:36 -0700
Subject: Removing Hoopl dependency?
Message-ID: <CAKVcRNLZdoZZoxn+c-N5MHLNc_mXW+r64Be2ozJ0pq6_9Mt_Wg@mail.gmail.com>

Thanks Ben.


Both A and B (mentioned in Simon's reply) are good alternatives as long as
existing Hoopl users are NOT forced to upgrade when they upgrade Cable or
Stackage.  Otherwise, we will see more forks of Hoopl.   Once Cable and
Stackage gain the multi-version capability, A and B can be merged back to
Hoopl as a major release.

In my experience, Hoopl based optimizations/program analyses tend to use a
lot of memory, but they are also easy to verify. So it's still a useful
tool in some special use cases.  If the performance of Hoopl can be
improved, it will certainly be more useful.

Cheers,
Ning


*From:* Ben Gamari <ben at smart-cactus.org>
*Date:* June 12, 2017 at 11:05:51 AM PDT
*To:* Simon Peyton Jones <simonpj at microsoft.com>, Sophie Taylor <
sophie at traumapony.org>, Michal Terepeta <michal.terepeta at gmail.com>,
ghc-devs <ghc-devs at haskell.org>, Ning Wang <email at ningwang.org>
*Cc:* Kavon Farvardin <kavon at cs.uchicago.edu>
*Subject:* *RE: Removing Hoopl dependency?*

Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:

Snip


That would leave Sophie free to do (B) free of the constraints of GHC

depending on it; but we could always use it later.


Does that sound plausible?  Do we know of any other Hoopl users?


CCing Ning, who is currently maintaining hoopl and I believe has some
projects using it.

Ning, you may want to have a look through this thread if you haven't
already seen it. You can find the previous messages in the list archive [1].

Cheers,

- Ben


[1] May messages: https://mail.haskell.org/piper
mail/ghc-devs/2017-May/014255.html
   June messages: https://mail.haskell.org/piper
mail/ghc-devs/2017-June/014293.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170613/9cb9bc0d/attachment-0001.html>

From rae at cs.brynmawr.edu  Wed Jun 14 13:55:14 2017
From: rae at cs.brynmawr.edu (Richard Eisenberg)
Date: Wed, 14 Jun 2017 09:55:14 -0400
Subject: external interpreter failing on Mac
Message-ID: <435097A7-F889-4DD7-AF2A-13A1C46BC665@cs.brynmawr.edu>

Hi devs,

It seems every test run through the external interpreter is failing for me on a Mac:

=====> TH_mkName(ext-interp) 1 of 1 [0, 0, 0]
cd "./TH_mkName.run" &&  "/Users/rae/ghc/ghc/inplace/test   spaces/ghc-stage2" -c TH_mkName.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -dno-debug-output -XTemplateHaskell -package template-haskell -fexternal-interpreter -v0
Compile failed (exit code 1) errors were:
ghc-iserv.bin: 
lookupSymbol failed in relocateSection (RELOC_GOT)
/Users/rae/ghc/ghc/libraries/integer-gmp/dist-install/build/HSinteger-gmp-1.0.0.1.o: unknown symbol `___gmp_rands'
ghc-stage2: unable to load package `integer-gmp-1.0.0.1'

*** unexpected failure for TH_mkName(ext-interp)

Any advice? My most recent commit from master is ef07010cf4f480d9f595a71cf5b009884522a75e from Wed Jun 7.

I'm on IRC today if you want to iterate that way.

Thanks!
Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170614/df9267b5/attachment.html>

From simonpj at microsoft.com  Thu Jun 15 07:41:40 2017
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Thu, 15 Jun 2017 07:41:40 +0000
Subject: GHC's module hierarchy
Message-ID: <SN1PR21MB00317365F828C8E62B9932D8ADC00@SN1PR21MB0031.namprd21.prod.outlook.com>

Dear ghc-devs
hsyl20 proposes a radical overhaul of the module structure of GHC itself.  He or she suggested it six months ago in
             https://ghc.haskell.org/trac/ghc/ticket/13009
and has now offered a monster patch
               https://phabricator.haskell.org/D3647
It's clearly the result of a lot of work, but I was the only one who responded on the original ticket, and it'll affect all of your lives in a very immediate way.
So, would you like to

*        consider the idea

*        look at the actual re-mapping of modules hsyl20 proposals

*        express an opinion about whether to go ahead
Probably the ticket, rather than Phab, is the best place to comment on the general idea.
I'd like to thank hsyl20.  GHC's rather flat module structure has grown incrementally over years.
But still, there are pros and cons.
Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170615/a7800378/attachment.html>

From marlowsd at gmail.com  Thu Jun 15 08:47:41 2017
From: marlowsd at gmail.com (Simon Marlow)
Date: Thu, 15 Jun 2017 09:47:41 +0100
Subject: external interpreter failing on Mac
In-Reply-To: <435097A7-F889-4DD7-AF2A-13A1C46BC665@cs.brynmawr.edu>
References: <435097A7-F889-4DD7-AF2A-13A1C46BC665@cs.brynmawr.edu>
Message-ID: <CANsrgPLnRd2AYbs2C5M-2Ly0SaQWi2zkJSqNmONYkPXkXm=ENg@mail.gmail.com>

No idea what this is, but just to point out that the continuous build
succeeded on OS X at the same revision that you have:
https://phabricator.haskell.org/B16186, so this is at least not a universal
failure.  Something specific to the version of GMP, or some other external
tool/library?

On 14 June 2017 at 14:55, Richard Eisenberg <rae at cs.brynmawr.edu> wrote:

> Hi devs,
>
> It seems every test run through the external interpreter is failing for me
> on a Mac:
>
> =====> TH_mkName(ext-interp) 1 of 1 [0, 0, 0]
> cd "./TH_mkName.run" &&  "/Users/rae/ghc/ghc/inplace/test
> spaces/ghc-stage2" -c TH_mkName.hs -dcore-lint -dcmm-lint
> -no-user-package-db -rtsopts -fno-warn-missed-specialisations
> -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret
> -dno-debug-output -XTemplateHaskell -package template-haskell
> -fexternal-interpreter -v0
> Compile failed (exit code 1) errors were:
> ghc-iserv.bin:
> lookupSymbol failed in relocateSection (RELOC_GOT)
> /Users/rae/ghc/ghc/libraries/integer-gmp/dist-install/build/HSinteger-gmp-1.0.0.1.o:
> unknown symbol `___gmp_rands'
> ghc-stage2: unable to load package `integer-gmp-1.0.0.1'
>
> *** unexpected failure for TH_mkName(ext-interp)
>
> Any advice? My most recent commit from master is
> ef07010cf4f480d9f595a71cf5b009884522a75e from Wed Jun 7.
>
> I'm on IRC today if you want to iterate that way.
>
> Thanks!
> Richard
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170615/890720cc/attachment-0001.html>

From marlowsd at gmail.com  Thu Jun 15 09:25:23 2017
From: marlowsd at gmail.com (Simon Marlow)
Date: Thu, 15 Jun 2017 10:25:23 +0100
Subject: WordX/IntX wrap Word#/Int#?
In-Reply-To: <1497217453.14706.1.camel@joachim-breitner.de>
References: <CAEN7k7=JirvhTO-U+5xYXSZTpyH3Q61OGLskz37a3svcP_+bVg@mail.gmail.com>
 <82569CFF-956D-4DA2-A04D-933195D5B6A6@smart-cactus.org>
 <1497217453.14706.1.camel@joachim-breitner.de>
Message-ID: <CANsrgPKG_rhziakabZMFLMy_oHEiYjkQ6wBt3_xutN_8wFdcbw@mail.gmail.com>

On 11 June 2017 at 22:44, Joachim Breitner <mail at joachim-breitner.de> wrote:

> Hi,
>
> Am Sonntag, den 11.06.2017, 10:44 -0400 schrieb Ben Gamari:
> > This is certainly one consideration. Another is that you would also
> > need to teach the garbage collector to understand closures with sub-
> > word-size fields. Currently we can encode whether each field of a
> > closure is a pointer or not with a simple bitmap. If we naively
> > allowed smaller fields we would need to increase the granularity of
> > this representation to encode bytes.
> >
> > Of course, one way to work around this would be to impose an
> > invariant that guarantees that pointers are always word-aligned. Then
> > we would probably want to shuffle sub-word sized fields, allowing two
> > Word16s to inhabit a single word.
>
> that is not an issue; we already sort field into pointers first, and
> non-pointers later. So all pointers are at the beginning and nicely
> aligned, and all the non-pointer data can follow in whatever weird
> format. The GC only needs to know how many words in total are used by
> the non-pointer data.
>

But the compiler has no support for sub-word-sized fields yet.  I made a
partial patch to support it a while ago: https://phabricator.haskell.org/D38


Cheers
Simon


Greetings,
> Joachim
> --
> Joachim “nomeata” Breitner
>   mail at joachim-breitner.de • https://www.joachim-breitner.de/
>   XMPP: nomeata at joachim-breitner.de • OpenPGP-Key: 0xF0FBF51F
>   Debian Developer: nomeata at debian.org
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170615/35a50c2f/attachment.html>

From rae at cs.brynmawr.edu  Thu Jun 15 13:32:09 2017
From: rae at cs.brynmawr.edu (Richard Eisenberg)
Date: Thu, 15 Jun 2017 09:32:09 -0400
Subject: external interpreter failing on Mac
In-Reply-To: <CANsrgPLnRd2AYbs2C5M-2Ly0SaQWi2zkJSqNmONYkPXkXm=ENg@mail.gmail.com>
References: <435097A7-F889-4DD7-AF2A-13A1C46BC665@cs.brynmawr.edu>
 <CANsrgPLnRd2AYbs2C5M-2Ly0SaQWi2zkJSqNmONYkPXkXm=ENg@mail.gmail.com>
Message-ID: <B8DE6A5F-3E1F-4038-841D-19B9EED6B3C0@cs.brynmawr.edu>

With deep gratitude to Ben and Moritz, who helped me on IRC, I’ve figured this out. Somehow my gmp install was broken -- a `brew install gmp` fixes it. What’s very bizarre is that the brokenness happened on two different computers, so it’s not terribly hard to do. Of course, both computers are maintained by me, so I may have done the same strange step twice.

Posting a ticket seems unlikely to be helpful, so I’ll park this issue for now, having solved it. But if anyone ever sees this again, the answer is `brew install gmp`.

Thanks,
Richard

> On Jun 15, 2017, at 4:47 AM, Simon Marlow <marlowsd at gmail.com> wrote:
> 
> No idea what this is, but just to point out that the continuous build succeeded on OS X at the same revision that you have: https://phabricator.haskell.org/B16186 <https://phabricator.haskell.org/B16186>, so this is at least not a universal failure.  Something specific to the version of GMP, or some other external tool/library?
> 
> On 14 June 2017 at 14:55, Richard Eisenberg <rae at cs.brynmawr.edu <mailto:rae at cs.brynmawr.edu>> wrote:
> Hi devs,
> 
> It seems every test run through the external interpreter is failing for me on a Mac:
> 
> =====> TH_mkName(ext-interp) 1 of 1 [0, 0, 0]
> cd "./TH_mkName.run" &&  "/Users/rae/ghc/ghc/inplace/test   spaces/ghc-stage2" -c TH_mkName.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -dno-debug-output -XTemplateHaskell -package template-haskell -fexternal-interpreter -v0
> Compile failed (exit code 1) errors were:
> ghc-iserv.bin: 
> lookupSymbol failed in relocateSection (RELOC_GOT)
> /Users/rae/ghc/ghc/libraries/integer-gmp/dist-install/build/HSinteger-gmp-1.0.0.1.o: unknown symbol `___gmp_rands'
> ghc-stage2: unable to load package `integer-gmp-1.0.0.1'
> 
> *** unexpected failure for TH_mkName(ext-interp)
> 
> Any advice? My most recent commit from master is ef07010cf4f480d9f595a71cf5b009884522a75e from Wed Jun 7.
> 
> I'm on IRC today if you want to iterate that way.
> 
> Thanks!
> Richard
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs <http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs>
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170615/1dabb80b/attachment.html>

From simonpj at microsoft.com  Fri Jun 16 21:22:06 2017
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Fri, 16 Jun 2017 21:22:06 +0000
Subject: #13804
Message-ID: <DM2PR21MB0028596DFA79AFF0B3FB157DADC10@DM2PR21MB0028.namprd21.prod.outlook.com>

Ben
I've just pushed a fix to Trac #13804.  It may belong on the 8.2 branch too because it fixes a regression wrt 8.0.
I can't test on my Unix box, because I'm away from it for a few days, but I've validated on windows.  Can you just check it's ok, and fix up or revert if not?
Thanks
SImon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170616/f753bba8/attachment.html>

From ben at well-typed.com  Fri Jun 16 21:35:45 2017
From: ben at well-typed.com (Ben Gamari)
Date: Fri, 16 Jun 2017 17:35:45 -0400
Subject: #13804
In-Reply-To: <DM2PR21MB0028596DFA79AFF0B3FB157DADC10@DM2PR21MB0028.namprd21.prod.outlook.com>
References: <DM2PR21MB0028596DFA79AFF0B3FB157DADC10@DM2PR21MB0028.namprd21.prod.outlook.com>
Message-ID: <878tkrehu6.fsf@ben-laptop.smart-cactus.org>

Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:

> Ben
> I've just pushed a fix to Trac #13804.  It may belong on the 8.2 branch too because it fixes a regression wrt 8.0.
> I can't test on my Unix box, because I'm away from it for a few days, but I've validated on windows.  Can you just check it's ok, and fix up or revert if not?

Of course.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170616/3bebf575/attachment.sig>

From carter.schonwald at gmail.com  Mon Jun 19 18:45:19 2017
From: carter.schonwald at gmail.com (Carter Schonwald)
Date: Mon, 19 Jun 2017 18:45:19 +0000
Subject: external interpreter failing on Mac
In-Reply-To: <B8DE6A5F-3E1F-4038-841D-19B9EED6B3C0@cs.brynmawr.edu>
References: <435097A7-F889-4DD7-AF2A-13A1C46BC665@cs.brynmawr.edu>
 <CANsrgPLnRd2AYbs2C5M-2Ly0SaQWi2zkJSqNmONYkPXkXm=ENg@mail.gmail.com>
 <B8DE6A5F-3E1F-4038-841D-19B9EED6B3C0@cs.brynmawr.edu>
Message-ID: <CAHYVw0xJTuG8yC1zTEJc-4bP+Dm+LoxmXMeJ5JVEuSCokt-2Gw@mail.gmail.com>

If you build ghc on your Mac, unless you enable intree gmp in the build.mk
file, it will select a system install of gmp. One way to avoid this is to
build your ghc with intree enabled.  Also makes it easier to share binaries
across Mac users :)

On Thu, Jun 15, 2017 at 9:32 AM Richard Eisenberg <rae at cs.brynmawr.edu>
wrote:

> With deep gratitude to Ben and Moritz, who helped me on IRC, I’ve figured
> this out. Somehow my gmp install was broken -- a `brew install gmp` fixes
> it. What’s very bizarre is that the brokenness happened on two different
> computers, so it’s not terribly hard to do. Of course, both computers are
> maintained by me, so I may have done the same strange step twice.
>
> Posting a ticket seems unlikely to be helpful, so I’ll park this issue for
> now, having solved it. But if anyone ever sees this again, the answer is
> `brew install gmp`.
>
> Thanks,
> Richard
>
> On Jun 15, 2017, at 4:47 AM, Simon Marlow <marlowsd at gmail.com> wrote:
>
> No idea what this is, but just to point out that the continuous build
> succeeded on OS X at the same revision that you have:
> https://phabricator.haskell.org/B16186, so this is at least not a
> universal failure.  Something specific to the version of GMP, or some other
> external tool/library?
>
> On 14 June 2017 at 14:55, Richard Eisenberg <rae at cs.brynmawr.edu> wrote:
>
>> Hi devs,
>>
>> It seems every test run through the external interpreter is failing for
>> me on a Mac:
>>
>> =====> TH_mkName(ext-interp) 1 of 1 [0, 0, 0]
>> cd "./TH_mkName.run" &&  "/Users/rae/ghc/ghc/inplace/test
>> spaces/ghc-stage2" -c TH_mkName.hs -dcore-lint -dcmm-lint
>> -no-user-package-db -rtsopts -fno-warn-missed-specialisations
>> -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret
>> -dno-debug-output -XTemplateHaskell -package template-haskell
>> -fexternal-interpreter -v0
>> Compile failed (exit code 1) errors were:
>> ghc-iserv.bin:
>> lookupSymbol failed in relocateSection (RELOC_GOT)
>> /Users/rae/ghc/ghc/libraries/integer-gmp/dist-install/build/HSinteger-gmp-1.0.0.1.o:
>> unknown symbol `___gmp_rands'
>> ghc-stage2: unable to load package `integer-gmp-1.0.0.1'
>>
>> *** unexpected failure for TH_mkName(ext-interp)
>>
>> Any advice? My most recent commit from master
>> is ef07010cf4f480d9f595a71cf5b009884522a75e from Wed Jun 7.
>>
>> I'm on IRC today if you want to iterate that way.
>>
>> Thanks!
>> Richard
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170619/8282985b/attachment.html>

From simonpj at microsoft.com  Mon Jun 26 21:01:51 2017
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 26 Jun 2017 21:01:51 +0000
Subject: hoopl dependency
Message-ID: <CY1PR21MB002272C41980103A4586740AADDF0@CY1PR21MB0022.namprd21.prod.outlook.com>

Devs,
Matko (an intern here at MSR) says
Matko Bosnjak 19:02:
Hi Simon,
There's a 'bug' in some of the latest commits of ghc on git. When running
./build

I get:

Error: libraries/hoopl/LICENSE doesn't exist.
Maybe you haven't done 'git submodule update --init'? at ./boot line 112, <PACKAGES> line 54.

Someone removed the dependency on hoopl library, but hasn't removed it from the 'packages' file.

What should he do?
Simon

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170626/4222ee58/attachment.html>

From ben at smart-cactus.org  Mon Jun 26 21:45:32 2017
From: ben at smart-cactus.org (Ben Gamari)
Date: Mon, 26 Jun 2017 17:45:32 -0400
Subject: hoopl dependency
In-Reply-To: <CY1PR21MB002272C41980103A4586740AADDF0@CY1PR21MB0022.namprd21.prod.outlook.com>
References: <CY1PR21MB002272C41980103A4586740AADDF0@CY1PR21MB0022.namprd21.prod.outlook.com>
Message-ID: <871sq6beyr.fsf@ben-laptop.smart-cactus.org>

Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:

> Devs,
> Matko (an intern here at MSR) says
> Matko Bosnjak 19:02:
> Hi Simon,
> There's a 'bug' in some of the latest commits of ghc on git. When running
> ./build
>
> I get:
>
> Error: libraries/hoopl/LICENSE doesn't exist.
> Maybe you haven't done 'git submodule update --init'? at ./boot line 112, <PACKAGES> line 54.
>
> Someone removed the dependency on hoopl library, but hasn't removed it from the 'packages' file.
>
I don't think that is the issue; hoopl was indeed removed from packages.
What commit is he on? Has he run `git submodule update --init`?

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170626/6e32e718/attachment.sig>

From simonpj at microsoft.com  Wed Jun 28 22:19:37 2017
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Wed, 28 Jun 2017 22:19:37 +0000
Subject: Thank you
Message-ID: <DM2PR21MB0028A02CF08F1640F2AC8B64ADDD0@DM2PR21MB0028.namprd21.prod.outlook.com>

Ryan
I wanted to thank you for all the work you are putting into GHC at the moment.

  *   You often produce nice small repro cases from other people's bug reports
  *   you often bisect to the patch caused the bug;
  *   your analysis is usually spot on;
  *   you are actively maintaining the generics stuff
  *   you contribute fixes
In short, terrific.  Thank you!
Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170628/67ae247d/attachment.html>

From simonpj at microsoft.com  Wed Jun 28 23:34:15 2017
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Wed, 28 Jun 2017 23:34:15 +0000
Subject: Windows broken
Message-ID: <DM2PR21MB0028C11BFC316D1941C231ABADDD0@DM2PR21MB0028.namprd21.prod.outlook.com>

Help please!
Suddenly my Windows build has stopped working.   See below.  What should I do?  I'm totally stuck.
This has never happened before.
I get this:

GHC path canonicalised to: c:/fp/ghc-8.0.2/bin/ghc

checking build system type... x86_64-pc-msys

checking host system type... x86_64-pc-msys

checking target system type... x86_64-pc-msys

Build platform inferred as: x86_64-unknown-mingw32

Host platform inferred as: x86_64-unknown-mingw32

Target platform inferred as: x86_64-unknown-mingw32

GHC build  : x86_64-unknown-mingw32

GHC host   : x86_64-unknown-mingw32

GHC target : x86_64-unknown-mingw32

LLVM target: x86_64-unknown-mingw32

checking for path to top of build tree... C:/code/HEAD

configure: Checking for Windows toolchain tarballs...

configure: Extracting Windows toolchain from archives (may take a while)...

configure: In-tree MingW-w64 tree created

configure: Making in-tree perl tree

configure: In-tree perl tree created

checking whether the C compiler works... no

configure: error: in `/c/code/HEAD':

configure: error: C compiler cannot create executables

See `config.log' for more details
Something wrong with the gcc.exe?  Config.log has this

gcc version 6.3.0 (Rev2, Built by MSYS2 project)

configure:4950: $? = 0

configure:4939: C:/code/HEAD/inplace/mingw/bin/gcc.exe -V >&5

gcc.exe: error: unrecognized command line option '-V'

gcc.exe: fatal error: no input files

compilation terminated.

configure:4950: $? = 1

configure:4939: C:/code/HEAD/inplace/mingw/bin/gcc.exe -qversion >&5

gcc.exe: error: unrecognized command line option '-qversion'; did you mean '--version'?

gcc.exe: fatal error: no input files

compilation terminated.

configure:4950: $? = 1

configure:4970: checking whether the C compiler works

configure:4992: C:/code/HEAD/inplace/mingw/bin/gcc.exe    conftest.c  >&5

configure:4996: $? = 1

configure:5034: result: no

configure: failed program was:

| /* confdefs.h */

| #define PACKAGE_NAME "The Glorious Glasgow Haskell Compilation System"

| #define PACKAGE_TARNAME "ghc-8.3"

| #define PACKAGE_VERSION "8.3"

| #define PACKAGE_STRING "The Glorious Glasgow Haskell Compilation System 8.3"

| #define PACKAGE_BUGREPORT "glasgow-haskell-bugs at haskell.org"

| #define PACKAGE_URL ""

| /* end confdefs.h.  */

|

| int

| main ()

| {

|

|   ;

|   return 0;

| }

configure:5039: error: in `/c/code/HEAD':

configure:5041: error: C compiler cannot create executables
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170628/50c63641/attachment-0001.html>

From lonetiger at gmail.com  Thu Jun 29 01:38:22 2017
From: lonetiger at gmail.com (lonetiger at gmail.com)
Date: Thu, 29 Jun 2017 02:38:22 +0100
Subject: Windows broken
In-Reply-To: <DM2PR21MB0028C11BFC316D1941C231ABADDD0@DM2PR21MB0028.namprd21.prod.outlook.com>
References: <DM2PR21MB0028C11BFC316D1941C231ABADDD0@DM2PR21MB0028.namprd21.prod.outlook.com>
Message-ID: <59545a0e.8e2d1c0a.890bd.dfbe@mx.google.com>

Hi Simon,

What happens when you try to compile that simple test with the bundled gcc?

Could you also run it through strace, this will force msys2 not to swallow errors from the OS.

Should you get an exception dialog then rm -rf inplace/mingw and rerun configure (I assume you’re on a recent revision).

If none of the above give a clue, you could try ./configure --enable-distro-toolchain, this will have it skip the bundled GCC
And use any installed by msys2. If this work then try rm’ing the GHC-tarballs folder and the inplace/mingw folder and try again
With –enable-tarballs-autodownloads.

Tamar.

From: Simon Peyton Jones via ghc-devs
Sent: Thursday, June 29, 2017 00:35
To: ghc-devs
Subject: Windows broken

Help please!
Suddenly my Windows build has stopped working.   See below.  What should I do?  I’m totally stuck.
This has never happened before.
I get this:
GHC path canonicalised to: c:/fp/ghc-8.0.2/bin/ghc
checking build system type... x86_64-pc-msys
checking host system type... x86_64-pc-msys
checking target system type... x86_64-pc-msys
Build platform inferred as: x86_64-unknown-mingw32
Host platform inferred as: x86_64-unknown-mingw32
Target platform inferred as: x86_64-unknown-mingw32
GHC build  : x86_64-unknown-mingw32
GHC host   : x86_64-unknown-mingw32
GHC target : x86_64-unknown-mingw32
LLVM target: x86_64-unknown-mingw32
checking for path to top of build tree... C:/code/HEAD
configure: Checking for Windows toolchain tarballs...
configure: Extracting Windows toolchain from archives (may take a while)...
configure: In-tree MingW-w64 tree created
configure: Making in-tree perl tree
configure: In-tree perl tree created
checking whether the C compiler works... no
configure: error: in `/c/code/HEAD':
configure: error: C compiler cannot create executables
See `config.log' for more details
Something wrong with the gcc.exe?  Config.log has this
gcc version 6.3.0 (Rev2, Built by MSYS2 project) 
configure:4950: $? = 0
configure:4939: C:/code/HEAD/inplace/mingw/bin/gcc.exe -V >&5
gcc.exe: error: unrecognized command line option '-V'
gcc.exe: fatal error: no input files
compilation terminated.
configure:4950: $? = 1
configure:4939: C:/code/HEAD/inplace/mingw/bin/gcc.exe -qversion >&5
gcc.exe: error: unrecognized command line option '-qversion'; did you mean '--version'?
gcc.exe: fatal error: no input files
compilation terminated.
configure:4950: $? = 1
configure:4970: checking whether the C compiler works
configure:4992: C:/code/HEAD/inplace/mingw/bin/gcc.exe    conftest.c  >&5
configure:4996: $? = 1
configure:5034: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "The Glorious Glasgow Haskell Compilation System"
| #define PACKAGE_TARNAME "ghc-8.3"
| #define PACKAGE_VERSION "8.3"
| #define PACKAGE_STRING "The Glorious Glasgow Haskell Compilation System 8.3"
| #define PACKAGE_BUGREPORT "glasgow-haskell-bugs at haskell.org"
| #define PACKAGE_URL ""
| /* end confdefs.h.  */
| 
| int
| main ()
| {
| 
|   ;
|   return 0;
| }
configure:5039: error: in `/c/code/HEAD':
configure:5041: error: C compiler cannot create executables

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170629/53a2c5d7/attachment.html>

From simonpj at microsoft.com  Thu Jun 29 08:10:05 2017
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Thu, 29 Jun 2017 08:10:05 +0000
Subject: Windows broken
In-Reply-To: <59545a0e.8e2d1c0a.890bd.dfbe@mx.google.com>
References: <DM2PR21MB0028C11BFC316D1941C231ABADDD0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <59545a0e.8e2d1c0a.890bd.dfbe@mx.google.com>
Message-ID: <DM2PR21MB0028D818E64C6F7FF29E99C0ADD20@DM2PR21MB0028.namprd21.prod.outlook.com>

If I compile a test foo.c I get this

C:/code/HEAD/inplace/mingw/bin/gcc.exe -v foo.c

Using built-in specs.

COLLECT_GCC=C:\code\HEAD\inplace\mingw\bin\gcc.exe

COLLECT_LTO_WRAPPER=C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/lto-wrapper.exe

Target: x86_64-w64-mingw32

Configured with: ../gcc-6.3.0/configure --prefix=/mingw64 --with-local-prefix=/mingw64/local --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --with-native-system-header-dir=/mingw64/x86_64-w64-mingw32/include --libexecdir=/mingw64/lib --enable-bootstrap --with-arch=x86-64 --with-tune=generic --enable-languages=c,lto,c++,objc,obj-c++,fortran,ada --enable-shared --enable-static --enable-libatomic --enable-threads=posix --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-time=yes --disable-libstdcxx-pch --disable-libstdcxx-debug --disable-isl-version-check --enable-lto --enable-libgomp --disable-multilib --enable-checking=release --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-libiconv --with-system-zlib --with-gmp=/mingw64 --with-mpfr=/mingw64 --with-mpc=/mingw64 --with-isl=/mingw64 --with-pkgversion='Rev2, Built by MSYS2 project' --with-bugurl=https://sourceforge.net/projects/msys2 --with-gnu-as --with-gnu-ld

Thread model: posix

gcc version 6.3.0 (Rev2, Built by MSYS2 project)

COLLECT_GCC_OPTIONS='-v' '-mtune=generic' '-march=x86-64'

C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/cc1.exe -quiet -v -iprefix C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/ -D_REENTRANT foo.c -quiet -dumpbase foo.c -mtune=generic -march=x86-64 -auxbase foo -version -o C:\Users\simonpj\AppData\Local\Temp\ccHvaSrL.s

/c/tmp$

So apparently the last line fails.  Running that by itself I get

C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/cc1.exe  -v -iprefix C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/ -D_REENTRANT foo.c -quiet -dumpbase foo.c -mtune=generic -march=x86-64 -auxbase foo -version -o foo.s

C:/code/HEAD/inplace/mingw/lib/gcc/x86_64-w64-mingw32/6.3.0/cc1.exe: error while loading shared libraries: libwinpthread-1.dll: cannot open shared object file: No such file or directory

/c/tmp$
So it can’t find that DLL.  But it exists all right

/c/code/HEAD$ find inplace/mingw -name 'libwinpthread-1.dll'

inplace/mingw/bin/libwinpthread-1.dll

The same thing happens, but with a popup alert window, if I use ‘strace gcc …’.

What now?

Simon

From: lonetiger at gmail.com [mailto:lonetiger at gmail.com]
Sent: 29 June 2017 02:38
To: Simon Peyton Jones <simonpj at microsoft.com>; ghc-devs <ghc-devs at haskell.org>
Subject: RE: Windows broken

Hi Simon,

What happens when you try to compile that simple test with the bundled gcc?

Could you also run it through strace, this will force msys2 not to swallow errors from the OS.

Should you get an exception dialog then rm -rf inplace/mingw and rerun configure (I assume you’re on a recent revision).

If none of the above give a clue, you could try ./configure --enable-distro-toolchain, this will have it skip the bundled GCC
And use any installed by msys2. If this work then try rm’ing the GHC-tarballs folder and the inplace/mingw folder and try again
With –enable-tarballs-autodownloads.

Tamar.

From: Simon Peyton Jones via ghc-devs<mailto:ghc-devs at haskell.org>
Sent: Thursday, June 29, 2017 00:35
To: ghc-devs<mailto:ghc-devs at haskell.org>
Subject: Windows broken

Help please!
Suddenly my Windows build has stopped working.   See below.  What should I do?  I’m totally stuck.
This has never happened before.
I get this:

GHC path canonicalised to: c:/fp/ghc-8.0.2/bin/ghc

checking build system type... x86_64-pc-msys

checking host system type... x86_64-pc-msys

checking target system type... x86_64-pc-msys

Build platform inferred as: x86_64-unknown-mingw32

Host platform inferred as: x86_64-unknown-mingw32

Target platform inferred as: x86_64-unknown-mingw32

GHC build  : x86_64-unknown-mingw32

GHC host   : x86_64-unknown-mingw32

GHC target : x86_64-unknown-mingw32

LLVM target: x86_64-unknown-mingw32

checking for path to top of build tree... C:/code/HEAD

configure: Checking for Windows toolchain tarballs...

configure: Extracting Windows toolchain from archives (may take a while)...

configure: In-tree MingW-w64 tree created

configure: Making in-tree perl tree

configure: In-tree perl tree created

checking whether the C compiler works... no

configure: error: in `/c/code/HEAD':

configure: error: C compiler cannot create executables

See `config.log' for more details
Something wrong with the gcc.exe?  Config.log has this

gcc version 6.3.0 (Rev2, Built by MSYS2 project)

configure:4950: $? = 0

configure:4939: C:/code/HEAD/inplace/mingw/bin/gcc.exe -V >&5

gcc.exe: error: unrecognized command line option '-V'

gcc.exe: fatal error: no input files

compilation terminated.

configure:4950: $? = 1

configure:4939: C:/code/HEAD/inplace/mingw/bin/gcc.exe -qversion >&5

gcc.exe: error: unrecognized command line option '-qversion'; did you mean '--version'?

gcc.exe: fatal error: no input files

compilation terminated.

configure:4950: $? = 1

configure:4970: checking whether the C compiler works

configure:4992: C:/code/HEAD/inplace/mingw/bin/gcc.exe    conftest.c  >&5

configure:4996: $? = 1

configure:5034: result: no

configure: failed program was:

| /* confdefs.h */

| #define PACKAGE_NAME "The Glorious Glasgow Haskell Compilation System"

| #define PACKAGE_TARNAME "ghc-8.3"

| #define PACKAGE_VERSION "8.3"

| #define PACKAGE_STRING "The Glorious Glasgow Haskell Compilation System 8.3"

| #define PACKAGE_BUGREPORT "glasgow-haskell-bugs at haskell.org<mailto:glasgow-haskell-bugs at haskell.org>"

| #define PACKAGE_URL ""

| /* end confdefs.h.  */

|

| int

| main ()

| {

|

|   ;

|   return 0;

| }

configure:5039: error: in `/c/code/HEAD':

configure:5041: error: C compiler cannot create executables

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170629/82c86426/attachment-0001.html>

From simonpj at microsoft.com  Thu Jun 29 08:10:08 2017
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Thu, 29 Jun 2017 08:10:08 +0000
Subject: Windows broken
In-Reply-To: <59545a0e.8e2d1c0a.890bd.dfbe@mx.google.com>
References: <DM2PR21MB0028C11BFC316D1941C231ABADDD0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <59545a0e.8e2d1c0a.890bd.dfbe@mx.google.com>
Message-ID: <DM2PR21MB0028B86BED598A21019799F2ADD20@DM2PR21MB0028.namprd21.prod.outlook.com>

Tamar

Aha!  It’s you!   Reverting this commit fixes it

git revert d6cecde585b0980ed8e0050c5a1d315789fb6356

[master af9f0f4] Revert "Remove the Windows GCC driver."

5 files changed, 88 insertions(+), 30 deletions(-)

create mode 100644 driver/gcc/gcc.c

If you can see an easy fix I can try that.  Else I’ll probably just revert and let you investigate.

I appear not to have Cygwin, and no other gcc is not in my path.  I just rely on the one that comes with GHC.

After reverting, here’s what happens when I compile foo.c (trace below).   The invocation of cc1.exe looks rather different

C:/code/HEAD/inplace/mingw/bin/gcc.exe -v foo.c

Using built-in specs.

COLLECT_GCC=C:\\code\\HEAD\\inplace\\mingw\\bin/realgcc.exe

COLLECT_LTO_WRAPPER=C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/lto-wrapper.exe

Target: x86_64-w64-mingw32

Configured with: ../gcc-6.3.0/configure --prefix=/mingw64 --with-local-prefix=/mingw64/local --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --with-native-system-header-dir=/mingw64/x86_64-w64-mingw32/include --libexecdir=/mingw64/lib --enable-bootstrap --with-arch=x86-64 --with-tune=generic --enable-languages=c,lto,c++,objc,obj-c++,fortran,ada --enable-shared --enable-static --enable-libatomic --enable-threads=posix --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-time=yes --disable-libstdcxx-pch --disable-libstdcxx-debug --disable-isl-version-check --enable-lto --enable-libgomp --disable-multilib --enable-checking=release --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-libiconv --with-system-zlib --with-gmp=/mingw64 --with-mpfr=/mingw64 --with-mpc=/mingw64 --with-isl=/mingw64 --with-pkgversion='Rev2, Built by MSYS2 project' --with-bugurl=https://sourceforge.net/projects/msys2 --with-gnu-as --with-gnu-ld

Thread model: posix

gcc version 6.3.0 (Rev2, Built by MSYS2 project)

COLLECT_GCC_OPTIONS='-B' 'C:\\code\\HEAD\\inplace\\mingw\\bin' '-B' 'C:\\code\\HEAD\\inplace\\mingw\\bin/../lib' '-B' 'C:\\code\\HEAD\\inplace\\mingw\\bin/../lib/gcc/x86_64-w64-mingw32/6.3.0' '-B' 'C:\\code\\HEAD\\inplace\\mingw\\bin/../libexec/gcc/x86_64-w64-mingw32/6.3.0' '-v' '-mtune=generic' '-march=x86-64'

C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/cc1.exe -quiet -v -iprefix C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/ -isystem C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include -isystem C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include-fixed -D_REENTRANT foo.c -quiet -dumpbase foo.c -mtune=generic -march=x86-64 -auxbase foo -version -o C:\Users\simonpj\AppData\Local\Temp\ccGa7vqE.s

GNU C11 (Rev2, Built by MSYS2 project) version 6.3.0 (x86_64-w64-mingw32)

      compiled by GNU C version 6.3.0, GMP version 6.1.2, MPFR version 3.1.5-p2, MPC version 1.0.3, isl version 0.15

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072

ignoring duplicate directory "C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include"

ignoring duplicate directory "C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include-fixed"

ignoring duplicate directory "C:/code/HEAD/inplace/mingw/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/6.3.0/include"

ignoring nonexistent directory "C:/building/msys64/mingw64/include"

ignoring nonexistent directory "/mingw64/include"

ignoring duplicate directory "C:/code/HEAD/inplace/mingw/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/6.3.0/include-fixed"

ignoring duplicate directory "C:/code/HEAD/inplace/mingw/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/6.3.0/../../../../x86_64-w64-mingw32/include"

ignoring nonexistent directory "C:/building/msys64/mingw64/x86_64-w64-mingw32/include"

#include "..." search starts here:

#include <...> search starts here:

C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include

C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include-fixed

C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/../../../../include

C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/../../../../x86_64-w64-mingw32/include

End of search list.

GNU C11 (Rev2, Built by MSYS2 project) version 6.3.0 (x86_64-w64-mingw32)

      compiled by GNU C version 6.3.0, GMP version 6.1.2, MPFR version 3.1.5-p2, MPC version 1.0.3, isl version 0.15

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072

Compiler executable checksum: bdbc00c2d895b1ff93d4c2cb4c58b5b1


From: lonetiger at gmail.com [mailto:lonetiger at gmail.com]
Sent: 29 June 2017 02:38
To: Simon Peyton Jones <simonpj at microsoft.com>; ghc-devs <ghc-devs at haskell.org>
Subject: RE: Windows broken

Hi Simon,

What happens when you try to compile that simple test with the bundled gcc?

Could you also run it through strace, this will force msys2 not to swallow errors from the OS.

Should you get an exception dialog then rm -rf inplace/mingw and rerun configure (I assume you’re on a recent revision).

If none of the above give a clue, you could try ./configure --enable-distro-toolchain, this will have it skip the bundled GCC
And use any installed by msys2. If this work then try rm’ing the GHC-tarballs folder and the inplace/mingw folder and try again
With –enable-tarballs-autodownloads.

Tamar.

From: Simon Peyton Jones via ghc-devs<mailto:ghc-devs at haskell.org>
Sent: Thursday, June 29, 2017 00:35
To: ghc-devs<mailto:ghc-devs at haskell.org>
Subject: Windows broken

Help please!
Suddenly my Windows build has stopped working.   See below.  What should I do?  I’m totally stuck.
This has never happened before.
I get this:

GHC path canonicalised to: c:/fp/ghc-8.0.2/bin/ghc

checking build system type... x86_64-pc-msys

checking host system type... x86_64-pc-msys

checking target system type... x86_64-pc-msys

Build platform inferred as: x86_64-unknown-mingw32

Host platform inferred as: x86_64-unknown-mingw32

Target platform inferred as: x86_64-unknown-mingw32

GHC build  : x86_64-unknown-mingw32

GHC host   : x86_64-unknown-mingw32

GHC target : x86_64-unknown-mingw32

LLVM target: x86_64-unknown-mingw32

checking for path to top of build tree... C:/code/HEAD

configure: Checking for Windows toolchain tarballs...

configure: Extracting Windows toolchain from archives (may take a while)...

configure: In-tree MingW-w64 tree created

configure: Making in-tree perl tree

configure: In-tree perl tree created

checking whether the C compiler works... no

configure: error: in `/c/code/HEAD':

configure: error: C compiler cannot create executables

See `config.log' for more details
Something wrong with the gcc.exe?  Config.log has this

gcc version 6.3.0 (Rev2, Built by MSYS2 project)

configure:4950: $? = 0

configure:4939: C:/code/HEAD/inplace/mingw/bin/gcc.exe -V >&5

gcc.exe: error: unrecognized command line option '-V'

gcc.exe: fatal error: no input files

compilation terminated.

configure:4950: $? = 1

configure:4939: C:/code/HEAD/inplace/mingw/bin/gcc.exe -qversion >&5

gcc.exe: error: unrecognized command line option '-qversion'; did you mean '--version'?

gcc.exe: fatal error: no input files

compilation terminated.

configure:4950: $? = 1

configure:4970: checking whether the C compiler works

configure:4992: C:/code/HEAD/inplace/mingw/bin/gcc.exe    conftest.c  >&5

configure:4996: $? = 1

configure:5034: result: no

configure: failed program was:

| /* confdefs.h */

| #define PACKAGE_NAME "The Glorious Glasgow Haskell Compilation System"

| #define PACKAGE_TARNAME "ghc-8.3"

| #define PACKAGE_VERSION "8.3"

| #define PACKAGE_STRING "The Glorious Glasgow Haskell Compilation System 8.3"

| #define PACKAGE_BUGREPORT "glasgow-haskell-bugs at haskell.org<mailto:glasgow-haskell-bugs at haskell.org>"

| #define PACKAGE_URL ""

| /* end confdefs.h.  */

|

| int

| main ()

| {

|

|   ;

|   return 0;

| }

configure:5039: error: in `/c/code/HEAD':

configure:5041: error: C compiler cannot create executables

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170629/051ef193/attachment-0001.html>

From lonetiger at gmail.com  Thu Jun 29 08:37:23 2017
From: lonetiger at gmail.com (lonetiger at gmail.com)
Date: Thu, 29 Jun 2017 09:37:23 +0100
Subject: Windows broken
In-Reply-To: <DM2PR21MB0028B86BED598A21019799F2ADD20@DM2PR21MB0028.namprd21.prod.outlook.com>
References: <DM2PR21MB0028C11BFC316D1941C231ABADDD0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <59545a0e.8e2d1c0a.890bd.dfbe@mx.google.com>
 <DM2PR21MB0028B86BED598A21019799F2ADD20@DM2PR21MB0028.namprd21.prod.outlook.com>
Message-ID: <5954bc43.6c8fdf0a.e736c.0c76@mx.google.com>


Oops, sorry, didn’t notice it because both mine and harbormaster’s msys2 have separate GCCs installed as well.

You can just leave it reverted, could you also push it? I don’t see an easy fix that would also work for end user
Configure based cabal installs. So I think I’ll have to go back to the drawing board for this one.

Thanks,
Tamar
From: Simon Peyton Jones
Sent: Thursday, June 29, 2017 09:10
To: lonetiger at gmail.com; ghc-devs
Subject: RE: Windows broken

Tamar

Aha!  It’s you!   Reverting this commit fixes it
git revert d6cecde585b0980ed8e0050c5a1d315789fb6356
[master af9f0f4] Revert "Remove the Windows GCC driver."
5 files changed, 88 insertions(+), 30 deletions(-)
create mode 100644 driver/gcc/gcc.c

If you can see an easy fix I can try that.  Else I’ll probably just revert and let you investigate.

I appear not to have Cygwin, and no other gcc is not in my path.  I just rely on the one that comes with GHC.

After reverting, here’s what happens when I compile foo.c (trace below).   The invocation of cc1.exe looks rather different
C:/code/HEAD/inplace/mingw/bin/gcc.exe -v foo.c
Using built-in specs.
COLLECT_GCC=C:\\code\\HEAD\\inplace\\mingw\\bin/realgcc.exe
COLLECT_LTO_WRAPPER=C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../gcc-6.3.0/configure --prefix=/mingw64 --with-local-prefix=/mingw64/local --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --with-native-system-header-dir=/mingw64/x86_64-w64-mingw32/include --libexecdir=/mingw64/lib --enable-bootstrap --with-arch=x86-64 --with-tune=generic --enable-languages=c,lto,c++,objc,obj-c++,fortran,ada --enable-shared --enable-static --enable-libatomic --enable-threads=posix --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-time=yes --disable-libstdcxx-pch --disable-libstdcxx-debug --disable-isl-version-check --enable-lto --enable-libgomp --disable-multilib --enable-checking=release --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-libiconv --with-system-zlib --with-gmp=/mingw64 --with-mpfr=/mingw64 --with-mpc=/mingw64 --with-isl=/mingw64 --with-pkgversion='Rev2, Built by MSYS2 project' --with-bugurl=https://sourceforge.net/projects/msys2 --with-gnu-as --with-gnu-ld
Thread model: posix
gcc version 6.3.0 (Rev2, Built by MSYS2 project) 
COLLECT_GCC_OPTIONS='-B' 'C:\\code\\HEAD\\inplace\\mingw\\bin' '-B' 'C:\\code\\HEAD\\inplace\\mingw\\bin/../lib' '-B' 'C:\\code\\HEAD\\inplace\\mingw\\bin/../lib/gcc/x86_64-w64-mingw32/6.3.0' '-B' 'C:\\code\\HEAD\\inplace\\mingw\\bin/../libexec/gcc/x86_64-w64-mingw32/6.3.0' '-v' '-mtune=generic' '-march=x86-64'
C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/cc1.exe -quiet -v -iprefix C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/ -isystem C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include -isystem C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include-fixed -D_REENTRANT foo.c -quiet -dumpbase foo.c -mtune=generic -march=x86-64 -auxbase foo -version -o C:\Users\simonpj\AppData\Local\Temp\ccGa7vqE.s
GNU C11 (Rev2, Built by MSYS2 project) version 6.3.0 (x86_64-w64-mingw32)
      compiled by GNU C version 6.3.0, GMP version 6.1.2, MPFR version 3.1.5-p2, MPC version 1.0.3, isl version 0.15
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring duplicate directory "C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include"
ignoring duplicate directory "C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include-fixed"
ignoring duplicate directory "C:/code/HEAD/inplace/mingw/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/6.3.0/include"
ignoring nonexistent directory "C:/building/msys64/mingw64/include"
ignoring nonexistent directory "/mingw64/include"
ignoring duplicate directory "C:/code/HEAD/inplace/mingw/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/6.3.0/include-fixed"
ignoring duplicate directory "C:/code/HEAD/inplace/mingw/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/6.3.0/../../../../x86_64-w64-mingw32/include"
ignoring nonexistent directory "C:/building/msys64/mingw64/x86_64-w64-mingw32/include"
#include "..." search starts here:
#include <...> search starts here:
C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include
C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include-fixed
C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/../../../../include
C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/../../../../x86_64-w64-mingw32/include
End of search list.
GNU C11 (Rev2, Built by MSYS2 project) version 6.3.0 (x86_64-w64-mingw32)
      compiled by GNU C version 6.3.0, GMP version 6.1.2, MPFR version 3.1.5-p2, MPC version 1.0.3, isl version 0.15
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: bdbc00c2d895b1ff93d4c2cb4c58b5b1


From: lonetiger at gmail.com [mailto:lonetiger at gmail.com] 
Sent: 29 June 2017 02:38
To: Simon Peyton Jones <simonpj at microsoft.com>; ghc-devs <ghc-devs at haskell.org>
Subject: RE: Windows broken

Hi Simon,

What happens when you try to compile that simple test with the bundled gcc?

Could you also run it through strace, this will force msys2 not to swallow errors from the OS.

Should you get an exception dialog then rm -rf inplace/mingw and rerun configure (I assume you’re on a recent revision).

If none of the above give a clue, you could try ./configure --enable-distro-toolchain, this will have it skip the bundled GCC
And use any installed by msys2. If this work then try rm’ing the GHC-tarballs folder and the inplace/mingw folder and try again
With –enable-tarballs-autodownloads.

Tamar.

From: Simon Peyton Jones via ghc-devs
Sent: Thursday, June 29, 2017 00:35
To: ghc-devs
Subject: Windows broken

Help please!
Suddenly my Windows build has stopped working.   See below.  What should I do?  I’m totally stuck.
This has never happened before.
I get this:
GHC path canonicalised to: c:/fp/ghc-8.0.2/bin/ghc
checking build system type... x86_64-pc-msys
checking host system type... x86_64-pc-msys
checking target system type... x86_64-pc-msys
Build platform inferred as: x86_64-unknown-mingw32
Host platform inferred as: x86_64-unknown-mingw32
Target platform inferred as: x86_64-unknown-mingw32
GHC build  : x86_64-unknown-mingw32
GHC host   : x86_64-unknown-mingw32
GHC target : x86_64-unknown-mingw32
LLVM target: x86_64-unknown-mingw32
checking for path to top of build tree... C:/code/HEAD
configure: Checking for Windows toolchain tarballs...
configure: Extracting Windows toolchain from archives (may take a while)...
configure: In-tree MingW-w64 tree created
configure: Making in-tree perl tree
configure: In-tree perl tree created
checking whether the C compiler works... no
configure: error: in `/c/code/HEAD':
configure: error: C compiler cannot create executables
See `config.log' for more details
Something wrong with the gcc.exe?  Config.log has this
gcc version 6.3.0 (Rev2, Built by MSYS2 project) 
configure:4950: $? = 0
configure:4939: C:/code/HEAD/inplace/mingw/bin/gcc.exe -V >&5
gcc.exe: error: unrecognized command line option '-V'
gcc.exe: fatal error: no input files
compilation terminated.
configure:4950: $? = 1
configure:4939: C:/code/HEAD/inplace/mingw/bin/gcc.exe -qversion >&5
gcc.exe: error: unrecognized command line option '-qversion'; did you mean '--version'?
gcc.exe: fatal error: no input files
compilation terminated.
configure:4950: $? = 1
configure:4970: checking whether the C compiler works
configure:4992: C:/code/HEAD/inplace/mingw/bin/gcc.exe    conftest.c  >&5
configure:4996: $? = 1
configure:5034: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "The Glorious Glasgow Haskell Compilation System"
| #define PACKAGE_TARNAME "ghc-8.3"
| #define PACKAGE_VERSION "8.3"
| #define PACKAGE_STRING "The Glorious Glasgow Haskell Compilation System 8.3"
| #define PACKAGE_BUGREPORT "glasgow-haskell-bugs at haskell.org"
| #define PACKAGE_URL ""
| /* end confdefs.h.  */
| 
| int
| main ()
| {
| 
|   ;
|   return 0;
| }
configure:5039: error: in `/c/code/HEAD':
configure:5041: error: C compiler cannot create executables


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170629/7d5a77c8/attachment-0001.html>

From simonpj at microsoft.com  Thu Jun 29 12:45:37 2017
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Thu, 29 Jun 2017 12:45:37 +0000
Subject: Windows broken
In-Reply-To: <5954bc43.6c8fdf0a.e736c.0c76@mx.google.com>
References: <DM2PR21MB0028C11BFC316D1941C231ABADDD0@DM2PR21MB0028.namprd21.prod.outlook.com>
 <59545a0e.8e2d1c0a.890bd.dfbe@mx.google.com>
 <DM2PR21MB0028B86BED598A21019799F2ADD20@DM2PR21MB0028.namprd21.prod.outlook.com>
 <5954bc43.6c8fdf0a.e736c.0c76@mx.google.com>
Message-ID: <DM2PR21MB0028FEB996A270F663398BCFADD20@DM2PR21MB0028.namprd21.prod.outlook.com>

You can just leave it reverted, could you also push it?

Yes ok. I’m sorry to push you back to the drawing board ☹.

Simon

From: lonetiger at gmail.com [mailto:lonetiger at gmail.com]
Sent: 29 June 2017 09:37
To: Simon Peyton Jones <simonpj at microsoft.com>; ghc-devs <ghc-devs at haskell.org>
Subject: RE: Windows broken


Oops, sorry, didn’t notice it because both mine and harbormaster’s msys2 have separate GCCs installed as well.

You can just leave it reverted, could you also push it? I don’t see an easy fix that would also work for end user
Configure based cabal installs. So I think I’ll have to go back to the drawing board for this one.

Thanks,
Tamar
From: Simon Peyton Jones<mailto:simonpj at microsoft.com>
Sent: Thursday, June 29, 2017 09:10
To: lonetiger at gmail.com<mailto:lonetiger at gmail.com>; ghc-devs<mailto:ghc-devs at haskell.org>
Subject: RE: Windows broken

Tamar

Aha!  It’s you!   Reverting this commit fixes it

git revert d6cecde585b0980ed8e0050c5a1d315789fb6356

[master af9f0f4] Revert "Remove the Windows GCC driver."

5 files changed, 88 insertions(+), 30 deletions(-)

create mode 100644 driver/gcc/gcc.c

If you can see an easy fix I can try that.  Else I’ll probably just revert and let you investigate.

I appear not to have Cygwin, and no other gcc is not in my path.  I just rely on the one that comes with GHC.

After reverting, here’s what happens when I compile foo.c (trace below).   The invocation of cc1.exe looks rather different

C:/code/HEAD/inplace/mingw/bin/gcc.exe -v foo.c

Using built-in specs.

COLLECT_GCC=C:\\code\\HEAD\\inplace\\mingw\\bin/realgcc.exe

COLLECT_LTO_WRAPPER=C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/lto-wrapper.exe

Target: x86_64-w64-mingw32

Configured with: ../gcc-6.3.0/configure --prefix=/mingw64 --with-local-prefix=/mingw64/local --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --with-native-system-header-dir=/mingw64/x86_64-w64-mingw32/include --libexecdir=/mingw64/lib --enable-bootstrap --with-arch=x86-64 --with-tune=generic --enable-languages=c,lto,c++,objc,obj-c++,fortran,ada --enable-shared --enable-static --enable-libatomic --enable-threads=posix --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-time=yes --disable-libstdcxx-pch --disable-libstdcxx-debug --disable-isl-version-check --enable-lto --enable-libgomp --disable-multilib --enable-checking=release --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-libiconv --with-system-zlib --with-gmp=/mingw64 --with-mpfr=/mingw64 --with-mpc=/mingw64 --with-isl=/mingw64 --with-pkgversion='Rev2, Built by MSYS2 project' --with-bugurl=https://sourceforge.net/projects/msys2 --with-gnu-as --with-gnu-ld

Thread model: posix

gcc version 6.3.0 (Rev2, Built by MSYS2 project)

COLLECT_GCC_OPTIONS='-B' 'C:\\code\\HEAD\\inplace\\mingw\\bin' '-B' 'C:\\code\\HEAD\\inplace\\mingw\\bin/../lib' '-B' 'C:\\code\\HEAD\\inplace\\mingw\\bin/../lib/gcc/x86_64-w64-mingw32/6.3.0' '-B' 'C:\\code\\HEAD\\inplace\\mingw\\bin/../libexec/gcc/x86_64-w64-mingw32/6.3.0' '-v' '-mtune=generic' '-march=x86-64'

C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/cc1.exe -quiet -v -iprefix C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/ -isystem C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include -isystem C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include-fixed -D_REENTRANT foo.c -quiet -dumpbase foo.c -mtune=generic -march=x86-64 -auxbase foo -version -o C:\Users\simonpj\AppData\Local\Temp\ccGa7vqE.s

GNU C11 (Rev2, Built by MSYS2 project) version 6.3.0 (x86_64-w64-mingw32)

      compiled by GNU C version 6.3.0, GMP version 6.1.2, MPFR version 3.1.5-p2, MPC version 1.0.3, isl version 0.15

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072

ignoring duplicate directory "C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include"

ignoring duplicate directory "C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include-fixed"

ignoring duplicate directory "C:/code/HEAD/inplace/mingw/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/6.3.0/include"

ignoring nonexistent directory "C:/building/msys64/mingw64/include"

ignoring nonexistent directory "/mingw64/include"

ignoring duplicate directory "C:/code/HEAD/inplace/mingw/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/6.3.0/include-fixed"

ignoring duplicate directory "C:/code/HEAD/inplace/mingw/lib/gcc/../../lib/gcc/x86_64-w64-mingw32/6.3.0/../../../../x86_64-w64-mingw32/include"

ignoring nonexistent directory "C:/building/msys64/mingw64/x86_64-w64-mingw32/include"

#include "..." search starts here:

#include <...> search starts here:

C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include

C://code//HEAD//inplace//mingw//bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/include-fixed

C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/../../../../include

C:/code/HEAD/inplace/mingw/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/../../../../x86_64-w64-mingw32/include

End of search list.

GNU C11 (Rev2, Built by MSYS2 project) version 6.3.0 (x86_64-w64-mingw32)

      compiled by GNU C version 6.3.0, GMP version 6.1.2, MPFR version 3.1.5-p2, MPC version 1.0.3, isl version 0.15

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072

Compiler executable checksum: bdbc00c2d895b1ff93d4c2cb4c58b5b1


From: lonetiger at gmail.com<mailto:lonetiger at gmail.com> [mailto:lonetiger at gmail.com]
Sent: 29 June 2017 02:38
To: Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>; ghc-devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: RE: Windows broken

Hi Simon,

What happens when you try to compile that simple test with the bundled gcc?

Could you also run it through strace, this will force msys2 not to swallow errors from the OS.

Should you get an exception dialog then rm -rf inplace/mingw and rerun configure (I assume you’re on a recent revision).

If none of the above give a clue, you could try ./configure --enable-distro-toolchain, this will have it skip the bundled GCC
And use any installed by msys2. If this work then try rm’ing the GHC-tarballs folder and the inplace/mingw folder and try again
With –enable-tarballs-autodownloads.

Tamar.

From: Simon Peyton Jones via ghc-devs<mailto:ghc-devs at haskell.org>
Sent: Thursday, June 29, 2017 00:35
To: ghc-devs<mailto:ghc-devs at haskell.org>
Subject: Windows broken

Help please!
Suddenly my Windows build has stopped working.   See below.  What should I do?  I’m totally stuck.
This has never happened before.
I get this:

GHC path canonicalised to: c:/fp/ghc-8.0.2/bin/ghc

checking build system type... x86_64-pc-msys

checking host system type... x86_64-pc-msys

checking target system type... x86_64-pc-msys

Build platform inferred as: x86_64-unknown-mingw32

Host platform inferred as: x86_64-unknown-mingw32

Target platform inferred as: x86_64-unknown-mingw32

GHC build  : x86_64-unknown-mingw32

GHC host   : x86_64-unknown-mingw32

GHC target : x86_64-unknown-mingw32

LLVM target: x86_64-unknown-mingw32

checking for path to top of build tree... C:/code/HEAD

configure: Checking for Windows toolchain tarballs...

configure: Extracting Windows toolchain from archives (may take a while)...

configure: In-tree MingW-w64 tree created

configure: Making in-tree perl tree

configure: In-tree perl tree created

checking whether the C compiler works... no

configure: error: in `/c/code/HEAD':

configure: error: C compiler cannot create executables

See `config.log' for more details
Something wrong with the gcc.exe?  Config.log has this

gcc version 6.3.0 (Rev2, Built by MSYS2 project)

configure:4950: $? = 0

configure:4939: C:/code/HEAD/inplace/mingw/bin/gcc.exe -V >&5

gcc.exe: error: unrecognized command line option '-V'

gcc.exe: fatal error: no input files

compilation terminated.

configure:4950: $? = 1

configure:4939: C:/code/HEAD/inplace/mingw/bin/gcc.exe -qversion >&5

gcc.exe: error: unrecognized command line option '-qversion'; did you mean '--version'?

gcc.exe: fatal error: no input files

compilation terminated.

configure:4950: $? = 1

configure:4970: checking whether the C compiler works

configure:4992: C:/code/HEAD/inplace/mingw/bin/gcc.exe    conftest.c  >&5

configure:4996: $? = 1

configure:5034: result: no

configure: failed program was:

| /* confdefs.h */

| #define PACKAGE_NAME "The Glorious Glasgow Haskell Compilation System"

| #define PACKAGE_TARNAME "ghc-8.3"

| #define PACKAGE_VERSION "8.3"

| #define PACKAGE_STRING "The Glorious Glasgow Haskell Compilation System 8.3"

| #define PACKAGE_BUGREPORT "glasgow-haskell-bugs at haskell.org<mailto:glasgow-haskell-bugs at haskell.org>"

| #define PACKAGE_URL ""

| /* end confdefs.h.  */

|

| int

| main ()

| {

|

|   ;

|   return 0;

| }

configure:5039: error: in `/c/code/HEAD':

configure:5041: error: C compiler cannot create executables


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170629/ba18db24/attachment-0001.html>

From palotai.robin at gmail.com  Fri Jun 30 07:55:21 2017
From: palotai.robin at gmail.com (Robin Palotai)
Date: Fri, 30 Jun 2017 09:55:21 +0200
Subject: Crossreferenced GHC 8.0.2
Message-ID: <CANiLjA26K_KKBdH-AQy2wxVfBYEC6P451CM=+OgROZVOprEpcg@mail.gmail.com>

Hello GHC devs,

I ran haskell-indexer [1] on the GHC 8.0.2 tarball, partly because I find
myself reading GHC source from time to time while working on the indexer,
and partly since it's fun.

First, here you can click around [2] and find where beloved functions are
called from:
http://stuff.codereview.me/#ghc/compiler/hsSyn/HsBinds.hs?corpus&signature
(scroll down a bit, imports are not linked yet).

Second, the way I indexed was pretty simple. I took the output of `make`,
replaced the ghc used with the ghc_kythe_wrapper, and filtered the lines
which included '-c', since I noticed that those duplicate previous large
compile lines. This only indexes the stage1 compilation AFAIU.

Feel free to suggest a better way to tap into the compilations to get
everything properly indexed (and possibly only once).

Any comments welcome!
Robin

[1]: https://github.com/google/haskell-indexer

[2]: TLDR UI quirks:
  - Click the :: in top-left to navigate file tree
  - Ctrl-Click (on linux) to go directly to definition (otherwise click
stuff from bottom pane)
  - Bottom pane often hides content, close it if stuck.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170630/db05a9cf/attachment.html>

From mail at nh2.me  Fri Jun 30 09:19:20 2017
From: mail at nh2.me (=?UTF-8?Q?Niklas_Hamb=c3=bcchen?=)
Date: Fri, 30 Jun 2017 11:19:20 +0200
Subject: Crossreferenced GHC 8.0.2
In-Reply-To: <CANiLjA26K_KKBdH-AQy2wxVfBYEC6P451CM=+OgROZVOprEpcg@mail.gmail.com>
References: <CANiLjA26K_KKBdH-AQy2wxVfBYEC6P451CM=+OgROZVOprEpcg@mail.gmail.com>
Message-ID: <b2b0e542-35fe-6825-4594-55d8b2791b8d@nh2.me>

Hey Robin,

I find that super useful, thanks!

I hope some day we'll get to the stage where for any Haskell code I can
easily discover all inputs, like the Java world has in their IDEs for
decades already.

Niklas

On 30/06/17 09:55, Robin Palotai wrote:
> First, here you can click around [2] and find where beloved functions
> are called from

From matthewtpickering at gmail.com  Fri Jun 30 09:57:09 2017
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Fri, 30 Jun 2017 10:57:09 +0100
Subject: Crossreferenced GHC 8.0.2
In-Reply-To: <CANiLjA26K_KKBdH-AQy2wxVfBYEC6P451CM=+OgROZVOprEpcg@mail.gmail.com>
References: <CANiLjA26K_KKBdH-AQy2wxVfBYEC6P451CM=+OgROZVOprEpcg@mail.gmail.com>
Message-ID: <CALuQ0m9DV4Rfg+2hR-nD8w5qrb7rRU50vRsYH3N_E55jXizXJw@mail.gmail.com>

Hi Robin,

This looks really useful for developers.

1. Would it be possible to provide a script which allows developers to
build this index for themselves easily?
2. Is it possible to use this tool to detect dead code? Functions
which are not used anywhere in the compiler.
3. How are you pretty printing the output whilst retaining the source
formatting? I had a quick look at the source but I couldn't see where
the output was being produced.

Cheers,

Matt

On Fri, Jun 30, 2017 at 8:55 AM, Robin Palotai <palotai.robin at gmail.com> wrote:
> Hello GHC devs,
>
> I ran haskell-indexer [1] on the GHC 8.0.2 tarball, partly because I find
> myself reading GHC source from time to time while working on the indexer,
> and partly since it's fun.
>
> First, here you can click around [2] and find where beloved functions are
> called from:
> http://stuff.codereview.me/#ghc/compiler/hsSyn/HsBinds.hs?corpus&signature
> (scroll down a bit, imports are not linked yet).
>
> Second, the way I indexed was pretty simple. I took the output of `make`,
> replaced the ghc used with the ghc_kythe_wrapper, and filtered the lines
> which included '-c', since I noticed that those duplicate previous large
> compile lines. This only indexes the stage1 compilation AFAIU.
>
> Feel free to suggest a better way to tap into the compilations to get
> everything properly indexed (and possibly only once).
>
> Any comments welcome!
> Robin
>
> [1]: https://github.com/google/haskell-indexer
>
> [2]: TLDR UI quirks:
>   - Click the :: in top-left to navigate file tree
>   - Ctrl-Click (on linux) to go directly to definition (otherwise click
> stuff from bottom pane)
>   - Bottom pane often hides content, close it if stuck.
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>

From wolfgang-it at jeltsch.info  Fri Jun 30 13:49:01 2017
From: wolfgang-it at jeltsch.info (Wolfgang Jeltsch)
Date: Fri, 30 Jun 2017 16:49:01 +0300
Subject: Which stable GHC release is expected to have support for linear types?
Message-ID: <1498830541.24308.5.camel@jeltsch.info>

Hi!

The blog post on http://blog.tweag.io/posts/2017-03-13-linear-types.html
says, “We are targeting a merge by the time of the 8.4 release of GHC.”
Does this mean that GHC 8.4 is expected to have support for linear
types, or does it mean that the merge should happen after a GHC 8.4
branch has been forked off from the HEAD branch, so that only GHC 8.6
will have support for linear types?

All the best,
Wolfgang

From m at tweag.io  Fri Jun 30 14:02:19 2017
From: m at tweag.io (Boespflug, Mathieu)
Date: Fri, 30 Jun 2017 16:02:19 +0200
Subject: Which stable GHC release is expected to have support for linear
 types?
In-Reply-To: <1498830541.24308.5.camel@jeltsch.info>
References: <1498830541.24308.5.camel@jeltsch.info>
Message-ID: <CAJ0eudpY3-CFAoFM-c25T-bejm0O43RYBPfrOa0ai7FxZvPWMg@mail.gmail.com>

Hi Wolfgang,

8.4 is the next release after the (by now imminent) 8.2 release. But
no support for linear types has of yet been merged into the master
branch that will eventually form the basis of the 8.4 release. This
support is under active development at the moment on a side branch.
Some of us are giving ourselves the 8.4 release as a *tentative*
target, so don't count on it. It may well end up happening much
further in the future. It's a research effort at this point and the
design is likely to change a bunch.

In the meantime, you can play around with this Docker image:

https://hub.docker.com/r/tweag/linear-types/

It contains a pre-built fork of GHC with the current state of the
linear types implementation. You can see the development as it's
happening here:

https://github.com/tweag/ghc/tree/linear-types

Best,
--
Mathieu Boespflug
Founder at http://tweag.io.


On 30 June 2017 at 15:49, Wolfgang Jeltsch <wolfgang-it at jeltsch.info> wrote:
> Hi!
>
> The blog post on http://blog.tweag.io/posts/2017-03-13-linear-types.html
> says, “We are targeting a merge by the time of the 8.4 release of GHC.”
> Does this mean that GHC 8.4 is expected to have support for linear
> types, or does it mean that the merge should happen after a GHC 8.4
> branch has been forked off from the HEAD branch, so that only GHC 8.6
> will have support for linear types?
>
> All the best,
> Wolfgang
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From wolfgang-it at jeltsch.info  Fri Jun 30 14:09:26 2017
From: wolfgang-it at jeltsch.info (Wolfgang Jeltsch)
Date: Fri, 30 Jun 2017 17:09:26 +0300
Subject: Which stable GHC release is expected to have support for linear
 types?
In-Reply-To: <CAJ0eudpY3-CFAoFM-c25T-bejm0O43RYBPfrOa0ai7FxZvPWMg@mail.gmail.com>
References: <1498830541.24308.5.camel@jeltsch.info>
 <CAJ0eudpY3-CFAoFM-c25T-bejm0O43RYBPfrOa0ai7FxZvPWMg@mail.gmail.com>
Message-ID: <1498831766.24308.14.camel@jeltsch.info>

Hi, Mathieu!

Thanks a lot for the quick reply.

I am just cloning the Git repository at https://github.com/tweag/ghc.git
and want to build GHC from it. Is this still the right repository? Does
it contain the same as the Docker image?

All the best,
Wolfgang

Am Freitag, den 30.06.2017, 16:02 +0200 schrieb Boespflug, Mathieu:
> Hi Wolfgang,
> 
> 8.4 is the next release after the (by now imminent) 8.2 release. But
> no support for linear types has of yet been merged into the master
> branch that will eventually form the basis of the 8.4 release. This
> support is under active development at the moment on a side branch.
> Some of us are giving ourselves the 8.4 release as a *tentative*
> target, so don't count on it. It may well end up happening much
> further in the future. It's a research effort at this point and the
> design is likely to change a bunch.
> 
> In the meantime, you can play around with this Docker image:
> 
> https://hub.docker.com/r/tweag/linear-types/
> 
> It contains a pre-built fork of GHC with the current state of the
> linear types implementation. You can see the development as it's
> happening here:
> 
> https://github.com/tweag/ghc/tree/linear-types
> 
> Best,
> --
> Mathieu Boespflug
> Founder at http://tweag.io.
> 
> 
> On 30 June 2017 at 15:49, Wolfgang Jeltsch <wolfgang-it at jeltsch.info>
> wrote:
> > 
> > Hi!
> > 
> > The blog post on http://blog.tweag.io/posts/2017-03-13-linear-types.
> > html
> > says, “We are targeting a merge by the time of the 8.4 release of
> > GHC.”
> > Does this mean that GHC 8.4 is expected to have support for linear
> > types, or does it mean that the merge should happen after a GHC 8.4
> > branch has been forked off from the HEAD branch, so that only GHC
> > 8.6
> > will have support for linear types?
> > 
> > All the best,
> > Wolfgang
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From m at tweag.io  Fri Jun 30 14:11:36 2017
From: m at tweag.io (Boespflug, Mathieu)
Date: Fri, 30 Jun 2017 16:11:36 +0200
Subject: Which stable GHC release is expected to have support for linear
 types?
In-Reply-To: <1498831766.24308.14.camel@jeltsch.info>
References: <1498830541.24308.5.camel@jeltsch.info>
 <CAJ0eudpY3-CFAoFM-c25T-bejm0O43RYBPfrOa0ai7FxZvPWMg@mail.gmail.com>
 <1498831766.24308.14.camel@jeltsch.info>
Message-ID: <CAJ0eudpK0630ceHmv4dpD0roriQsiKjwE==BUT_eG=Bzk8b1hg@mail.gmail.com>

Yes, just make sure to checkout the linear-types branch, not the master branch.
--
Mathieu Boespflug
Founder at http://tweag.io.


On 30 June 2017 at 16:09, Wolfgang Jeltsch <wolfgang-it at jeltsch.info> wrote:
> Hi, Mathieu!
>
> Thanks a lot for the quick reply.
>
> I am just cloning the Git repository at https://github.com/tweag/ghc.git
> and want to build GHC from it. Is this still the right repository? Does
> it contain the same as the Docker image?
>
> All the best,
> Wolfgang
>
> Am Freitag, den 30.06.2017, 16:02 +0200 schrieb Boespflug, Mathieu:
>> Hi Wolfgang,
>>
>> 8.4 is the next release after the (by now imminent) 8.2 release. But
>> no support for linear types has of yet been merged into the master
>> branch that will eventually form the basis of the 8.4 release. This
>> support is under active development at the moment on a side branch.
>> Some of us are giving ourselves the 8.4 release as a *tentative*
>> target, so don't count on it. It may well end up happening much
>> further in the future. It's a research effort at this point and the
>> design is likely to change a bunch.
>>
>> In the meantime, you can play around with this Docker image:
>>
>> https://hub.docker.com/r/tweag/linear-types/
>>
>> It contains a pre-built fork of GHC with the current state of the
>> linear types implementation. You can see the development as it's
>> happening here:
>>
>> https://github.com/tweag/ghc/tree/linear-types
>>
>> Best,
>> --
>> Mathieu Boespflug
>> Founder at http://tweag.io.
>>
>>
>> On 30 June 2017 at 15:49, Wolfgang Jeltsch <wolfgang-it at jeltsch.info>
>> wrote:
>> >
>> > Hi!
>> >
>> > The blog post on http://blog.tweag.io/posts/2017-03-13-linear-types.
>> > html
>> > says, “We are targeting a merge by the time of the 8.4 release of
>> > GHC.”
>> > Does this mean that GHC 8.4 is expected to have support for linear
>> > types, or does it mean that the merge should happen after a GHC 8.4
>> > branch has been forked off from the HEAD branch, so that only GHC
>> > 8.6
>> > will have support for linear types?
>> >
>> > All the best,
>> > Wolfgang
>> > _______________________________________________
>> > ghc-devs mailing list
>> > ghc-devs at haskell.org
>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From wolfgang-it at jeltsch.info  Fri Jun 30 14:15:23 2017
From: wolfgang-it at jeltsch.info (Wolfgang Jeltsch)
Date: Fri, 30 Jun 2017 17:15:23 +0300
Subject: Which stable GHC release is expected to have support for linear
 types?
In-Reply-To: <CAJ0eudpK0630ceHmv4dpD0roriQsiKjwE==BUT_eG=Bzk8b1hg@mail.gmail.com>
References: <1498830541.24308.5.camel@jeltsch.info>
 <CAJ0eudpY3-CFAoFM-c25T-bejm0O43RYBPfrOa0ai7FxZvPWMg@mail.gmail.com>
 <1498831766.24308.14.camel@jeltsch.info>
 <CAJ0eudpK0630ceHmv4dpD0roriQsiKjwE==BUT_eG=Bzk8b1hg@mail.gmail.com>
Message-ID: <1498832123.24308.16.camel@jeltsch.info>

Hi!

I am just reading the instructions for cloning GHC from Git under

    https://ghc.haskell.org/trac/ghc/wiki/Building/GettingTheSources#CloningfromGitHub

Do these instructions also apply in an analogous way to cloning your GHC
repository?

All the best,
Wolfgang

Am Freitag, den 30.06.2017, 16:11 +0200 schrieb Boespflug, Mathieu:
> Yes, just make sure to checkout the linear-types branch, not the
> master branch.
> --
> Mathieu Boespflug
> Founder at http://tweag.io.
> 
> 
> On 30 June 2017 at 16:09, Wolfgang Jeltsch <wolfgang-it at jeltsch.info>
> wrote:
> > 
> > Hi, Mathieu!
> > 
> > Thanks a lot for the quick reply.
> > 
> > I am just cloning the Git repository at https://github.com/tweag/ghc
> > .git
> > and want to build GHC from it. Is this still the right repository?
> > Does
> > it contain the same as the Docker image?
> > 
> > All the best,
> > Wolfgang
> > 
> > Am Freitag, den 30.06.2017, 16:02 +0200 schrieb Boespflug, Mathieu:
> > > 
> > > Hi Wolfgang,
> > > 
> > > 8.4 is the next release after the (by now imminent) 8.2 release.
> > > But
> > > no support for linear types has of yet been merged into the master
> > > branch that will eventually form the basis of the 8.4 release.
> > > This
> > > support is under active development at the moment on a side
> > > branch.
> > > Some of us are giving ourselves the 8.4 release as a *tentative*
> > > target, so don't count on it. It may well end up happening much
> > > further in the future. It's a research effort at this point and
> > > the
> > > design is likely to change a bunch.
> > > 
> > > In the meantime, you can play around with this Docker image:
> > > 
> > > https://hub.docker.com/r/tweag/linear-types/
> > > 
> > > It contains a pre-built fork of GHC with the current state of the
> > > linear types implementation. You can see the development as it's
> > > happening here:
> > > 
> > > https://github.com/tweag/ghc/tree/linear-types
> > > 
> > > Best,
> > > --
> > > Mathieu Boespflug
> > > Founder at http://tweag.io.
> > > 
> > > 
> > > On 30 June 2017 at 15:49, Wolfgang Jeltsch <wolfgang-it at jeltsch.in
> > > fo>
> > > wrote:
> > > > 
> > > > 
> > > > Hi!
> > > > 
> > > > The blog post on http://blog.tweag.io/posts/2017-03-13-linear-ty
> > > > pes.
> > > > html
> > > > says, “We are targeting a merge by the time of the 8.4 release
> > > > of
> > > > GHC.”
> > > > Does this mean that GHC 8.4 is expected to have support for
> > > > linear
> > > > types, or does it mean that the merge should happen after a GHC
> > > > 8.4
> > > > branch has been forked off from the HEAD branch, so that only
> > > > GHC
> > > > 8.6
> > > > will have support for linear types?
> > > > 
> > > > All the best,
> > > > Wolfgang
> > > > _______________________________________________
> > > > ghc-devs mailing list
> > > > ghc-devs at haskell.org
> > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From wolfgang-it at jeltsch.info  Fri Jun 30 14:23:12 2017
From: wolfgang-it at jeltsch.info (Wolfgang Jeltsch)
Date: Fri, 30 Jun 2017 17:23:12 +0300
Subject: Which stable GHC release is expected to have support for linear
 types?
In-Reply-To: <1498832123.24308.16.camel@jeltsch.info>
References: <1498830541.24308.5.camel@jeltsch.info>
 <CAJ0eudpY3-CFAoFM-c25T-bejm0O43RYBPfrOa0ai7FxZvPWMg@mail.gmail.com>
 <1498831766.24308.14.camel@jeltsch.info>
 <CAJ0eudpK0630ceHmv4dpD0roriQsiKjwE==BUT_eG=Bzk8b1hg@mail.gmail.com>
 <1498832123.24308.16.camel@jeltsch.info>
Message-ID: <1498832592.24308.17.camel@jeltsch.info>

Hmm, after entering various submodules into the configuration, Git asks
me for a username for https://github.com. Why?

Am Freitag, den 30.06.2017, 17:15 +0300 schrieb Wolfgang Jeltsch:
> Hi!
> 
> I am just reading the instructions for cloning GHC from Git under
> 
>     https://ghc.haskell.org/trac/ghc/wiki/Building/GettingTheSources#C
> loningfromGitHub
> 
> Do these instructions also apply in an analogous way to cloning your
> GHC repository?
> 
> All the best,
> Wolfgang
> 
> Am Freitag, den 30.06.2017, 16:11 +0200 schrieb Boespflug, Mathieu:
> > 
> > Yes, just make sure to checkout the linear-types branch, not the
> > master branch.
> > --
> > Mathieu Boespflug
> > Founder at http://tweag.io.
> > 
> > 
> > On 30 June 2017 at 16:09, Wolfgang Jeltsch <wolfgang-it at jeltsch.info
> > >
> > wrote:
> > > 
> > > 
> > > Hi, Mathieu!
> > > 
> > > Thanks a lot for the quick reply.
> > > 
> > > I am just cloning the Git repository at https://github.com/tweag/g
> > > hc
> > > .git
> > > and want to build GHC from it. Is this still the right repository?
> > > Does
> > > it contain the same as the Docker image?
> > > 
> > > All the best,
> > > Wolfgang
> > > 
> > > Am Freitag, den 30.06.2017, 16:02 +0200 schrieb Boespflug,
> > > Mathieu:
> > > > 
> > > > 
> > > > Hi Wolfgang,
> > > > 
> > > > 8.4 is the next release after the (by now imminent) 8.2 release.
> > > > But
> > > > no support for linear types has of yet been merged into the
> > > > master
> > > > branch that will eventually form the basis of the 8.4 release.
> > > > This
> > > > support is under active development at the moment on a side
> > > > branch.
> > > > Some of us are giving ourselves the 8.4 release as a *tentative*
> > > > target, so don't count on it. It may well end up happening much
> > > > further in the future. It's a research effort at this point and
> > > > the
> > > > design is likely to change a bunch.
> > > > 
> > > > In the meantime, you can play around with this Docker image:
> > > > 
> > > > https://hub.docker.com/r/tweag/linear-types/
> > > > 
> > > > It contains a pre-built fork of GHC with the current state of
> > > > the
> > > > linear types implementation. You can see the development as it's
> > > > happening here:
> > > > 
> > > > https://github.com/tweag/ghc/tree/linear-types
> > > > 
> > > > Best,
> > > > --
> > > > Mathieu Boespflug
> > > > Founder at http://tweag.io.
> > > > 
> > > > 
> > > > On 30 June 2017 at 15:49, Wolfgang Jeltsch <wolfgang-it at jeltsch.
> > > > in
> > > > fo>
> > > > wrote:
> > > > > 
> > > > > 
> > > > > 
> > > > > Hi!
> > > > > 
> > > > > The blog post on http://blog.tweag.io/posts/2017-03-13-linear-
> > > > > ty
> > > > > pes.
> > > > > html
> > > > > says, “We are targeting a merge by the time of the 8.4 release
> > > > > of
> > > > > GHC.”
> > > > > Does this mean that GHC 8.4 is expected to have support for
> > > > > linear
> > > > > types, or does it mean that the merge should happen after a
> > > > > GHC
> > > > > 8.4
> > > > > branch has been forked off from the HEAD branch, so that only
> > > > > GHC
> > > > > 8.6
> > > > > will have support for linear types?
> > > > > 
> > > > > All the best,
> > > > > Wolfgang
> > > > > _______________________________________________
> > > > > ghc-devs mailing list
> > > > > ghc-devs at haskell.org
> > > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> > > _______________________________________________
> > > ghc-devs mailing list
> > > ghc-devs at haskell.org
> > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From simonpj at microsoft.com  Fri Jun 30 14:39:44 2017
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Fri, 30 Jun 2017 14:39:44 +0000
Subject: Which stable GHC release is expected to have support for linear
 types?
In-Reply-To: <CAJ0eudpY3-CFAoFM-c25T-bejm0O43RYBPfrOa0ai7FxZvPWMg@mail.gmail.com>
References: <1498830541.24308.5.camel@jeltsch.info>
 <CAJ0eudpY3-CFAoFM-c25T-bejm0O43RYBPfrOa0ai7FxZvPWMg@mail.gmail.com>
Message-ID: <DM2PR21MB0028650680A23E45027A08D8ADD30@DM2PR21MB0028.namprd21.prod.outlook.com>

And, just to be clear, while I think the design is coming along nicely, we have not yet made a clear proposal to the ghc-devs community, via the ghc-proposals mechanism, for a change to GHC.   Until we have an adopted propsoal we can't say "it'll be in GHC by date X".

Simon

|  -----Original Message-----
|  From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of
|  Boespflug, Mathieu
|  Sent: 30 June 2017 15:02
|  To: Wolfgang Jeltsch <wolfgang-it at jeltsch.info>
|  Cc: Arnaud Spiwack <arnaud.spiwack at tweag.io>; ghc-devs <ghc-
|  devs at haskell.org>
|  Subject: Re: Which stable GHC release is expected to have support for
|  linear types?
|  
|  Hi Wolfgang,
|  
|  8.4 is the next release after the (by now imminent) 8.2 release. But
|  no support for linear types has of yet been merged into the master
|  branch that will eventually form the basis of the 8.4 release. This
|  support is under active development at the moment on a side branch.
|  Some of us are giving ourselves the 8.4 release as a *tentative*
|  target, so don't count on it. It may well end up happening much
|  further in the future. It's a research effort at this point and the
|  design is likely to change a bunch.
|  
|  In the meantime, you can play around with this Docker image:
|  
|  https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhub.d
|  ocker.com%2Fr%2Ftweag%2Flinear-
|  types%2F&data=02%7C01%7Csimonpj%40microsoft.com%7C0f0fa7d2181b498882bd
|  08d4bfc0b5c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636344281789
|  392239&sdata=aWCyABYfJdvkxZAVk4wD5IuErCPTYGvdFO22nbT8gYw%3D&reserved=0
|  
|  It contains a pre-built fork of GHC with the current state of the
|  linear types implementation. You can see the development as it's
|  happening here:
|  
|  https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithu
|  b.com%2Ftweag%2Fghc%2Ftree%2Flinear-
|  types&data=02%7C01%7Csimonpj%40microsoft.com%7C0f0fa7d2181b498882bd08d
|  4bfc0b5c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636344281789392
|  239&sdata=IGDxDD1q%2BxTRK%2FFwS0C%2BsDEpwFzbzHja4sF3twlF55c%3D&reserve
|  d=0
|  
|  Best,
|  --
|  Mathieu Boespflug
|  Founder at
|  https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftweag.
|  io&data=02%7C01%7Csimonpj%40microsoft.com%7C0f0fa7d2181b498882bd08d4bf
|  c0b5c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636344281789392239
|  &sdata=qXdzC81GzESKMHKCykOyM75XrPRbFL8edpsxKKv5VVQ%3D&reserved=0.
|  
|  
|  On 30 June 2017 at 15:49, Wolfgang Jeltsch <wolfgang-it at jeltsch.info>
|  wrote:
|  > Hi!
|  >
|  > The blog post on
|  >
|  https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fblog.t
|  > weag.io%2Fposts%2F2017-03-13-linear-
|  types.html&data=02%7C01%7Csimonpj%
|  >
|  40microsoft.com%7C0f0fa7d2181b498882bd08d4bfc0b5c1%7C72f988bf86f141af9
|  >
|  1ab2d7cd011db47%7C1%7C0%7C636344281789392239&sdata=i8wFqX5an5LyuBKQ5Jk
|  > 2q8jUQdb62l72wOj9OX1OoK0%3D&reserved=0
|  > says, “We are targeting a merge by the time of the 8.4 release of
|  GHC.”
|  > Does this mean that GHC 8.4 is expected to have support for linear
|  > types, or does it mean that the merge should happen after a GHC 8.4
|  > branch has been forked off from the HEAD branch, so that only GHC
|  8.6
|  > will have support for linear types?
|  >
|  > All the best,
|  > Wolfgang
|  > _______________________________________________
|  > ghc-devs mailing list
|  > ghc-devs at haskell.org
|  >
|  https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h
|  > askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&data=02%7C01%7Csi
|  >
|  monpj%40microsoft.com%7C0f0fa7d2181b498882bd08d4bfc0b5c1%7C72f988bf86f
|  >
|  141af91ab2d7cd011db47%7C1%7C0%7C636344281789392239&sdata=5MKuM59nRrrjm
|  > MPMY9%2BPvsR1%2BBAPHDExiCbU%2FlckuGg%3D&reserved=0
|  _______________________________________________
|  ghc-devs mailing list
|  ghc-devs at haskell.org
|  https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h
|  askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&data=02%7C01%7Csimonpj%40microsoft.com%7C0f0fa7d2181b498882bd08d4
|  bfc0b5c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6363442817893922
|  39&sdata=5MKuM59nRrrjmMPMY9%2BPvsR1%2BBAPHDExiCbU%2FlckuGg%3D&reserved
|  =0

From simonpj at microsoft.com  Fri Jun 30 14:40:46 2017
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Fri, 30 Jun 2017 14:40:46 +0000
Subject: Which stable GHC release is expected to have support for linear
 types?
In-Reply-To: <1498830541.24308.5.camel@jeltsch.info>
References: <1498830541.24308.5.camel@jeltsch.info>
Message-ID: <DM2PR21MB00280C859594BC82ACD640A7ADD30@DM2PR21MB0028.namprd21.prod.outlook.com>

Wolfgang

It'd be very interesting to know more about your use-case.  Why do you want linear types?

Simon

|  -----Original Message-----
|  From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of
|  Wolfgang Jeltsch
|  Sent: 30 June 2017 14:49
|  To: ghc-devs at haskell.org
|  Subject: Which stable GHC release is expected to have support for
|  linear types?
|  
|  Hi!
|  
|  The blog post on http://blog.tweag.io/posts/2017-03-13-linear-
|  types.html
|  says, “We are targeting a merge by the time of the 8.4 release of
|  GHC.”
|  Does this mean that GHC 8.4 is expected to have support for linear
|  types, or does it mean that the merge should happen after a GHC 8.4
|  branch has been forked off from the HEAD branch, so that only GHC 8.6
|  will have support for linear types?
|  
|  All the best,
|  Wolfgang
|  _______________________________________________
|  ghc-devs mailing list
|  ghc-devs at haskell.org
|  http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From wolfgang-it at jeltsch.info  Fri Jun 30 15:29:31 2017
From: wolfgang-it at jeltsch.info (Wolfgang Jeltsch)
Date: Fri, 30 Jun 2017 18:29:31 +0300
Subject: Which stable GHC release is expected to have support for linear
 types?
In-Reply-To: <1498832592.24308.17.camel@jeltsch.info>
References: <1498830541.24308.5.camel@jeltsch.info>
 <CAJ0eudpY3-CFAoFM-c25T-bejm0O43RYBPfrOa0ai7FxZvPWMg@mail.gmail.com>
 <1498831766.24308.14.camel@jeltsch.info>
 <CAJ0eudpK0630ceHmv4dpD0roriQsiKjwE==BUT_eG=Bzk8b1hg@mail.gmail.com>
 <1498832123.24308.16.camel@jeltsch.info>
 <1498832592.24308.17.camel@jeltsch.info>
Message-ID: <1498836571.24308.25.camel@jeltsch.info>

After entering my Git username and password, Git showed me the following
error:

> fatal: repository 'https://github.com/tweag/arcanist-external-json-linter.git/' not found

What exact command do I have to use to clone the repository? So far, I
have used the following:

> git clone --recursive https://github.com/tweag/ghc.git

All the best,
Wolfgang

Am Freitag, den 30.06.2017, 17:23 +0300 schrieb Wolfgang Jeltsch:
> Hmm, after entering various submodules into the configuration, Git
> asks me for a username for https://github.com. Why?
> 
> Am Freitag, den 30.06.2017, 17:15 +0300 schrieb Wolfgang Jeltsch:
> > 
> > Hi!
> > 
> > I am just reading the instructions for cloning GHC from Git under
> > 
> >     https://ghc.haskell.org/trac/ghc/wiki/Building/GettingTheSources
> > #C
> > loningfromGitHub
> > 
> > Do these instructions also apply in an analogous way to cloning your
> > GHC repository?
> > 
> > All the best,
> > Wolfgang
> > 
> > Am Freitag, den 30.06.2017, 16:11 +0200 schrieb Boespflug, Mathieu:
> > > 
> > > 
> > > Yes, just make sure to checkout the linear-types branch, not the
> > > master branch.
> > > --
> > > Mathieu Boespflug
> > > Founder at http://tweag.io.
> > > 
> > > 
> > > On 30 June 2017 at 16:09, Wolfgang Jeltsch <wolfgang-it at jeltsch.in
> > > fo
> > > > 
> > > > 
> > > wrote:
> > > > 
> > > > 
> > > > 
> > > > Hi, Mathieu!
> > > > 
> > > > Thanks a lot for the quick reply.
> > > > 
> > > > I am just cloning the Git repository at https://github.com/tweag
> > > > /g
> > > > hc
> > > > .git
> > > > and want to build GHC from it. Is this still the right
> > > > repository?
> > > > Does
> > > > it contain the same as the Docker image?
> > > > 
> > > > All the best,
> > > > Wolfgang
> > > > 
> > > > Am Freitag, den 30.06.2017, 16:02 +0200 schrieb Boespflug,
> > > > Mathieu:
> > > > > 
> > > > > 
> > > > > 
> > > > > Hi Wolfgang,
> > > > > 
> > > > > 8.4 is the next release after the (by now imminent) 8.2
> > > > > release.
> > > > > But
> > > > > no support for linear types has of yet been merged into the
> > > > > master
> > > > > branch that will eventually form the basis of the 8.4 release.
> > > > > This
> > > > > support is under active development at the moment on a side
> > > > > branch.
> > > > > Some of us are giving ourselves the 8.4 release as a
> > > > > *tentative*
> > > > > target, so don't count on it. It may well end up happening
> > > > > much
> > > > > further in the future. It's a research effort at this point
> > > > > and
> > > > > the
> > > > > design is likely to change a bunch.
> > > > > 
> > > > > In the meantime, you can play around with this Docker image:
> > > > > 
> > > > > https://hub.docker.com/r/tweag/linear-types/
> > > > > 
> > > > > It contains a pre-built fork of GHC with the current state of
> > > > > the
> > > > > linear types implementation. You can see the development as
> > > > > it's
> > > > > happening here:
> > > > > 
> > > > > https://github.com/tweag/ghc/tree/linear-types
> > > > > 
> > > > > Best,
> > > > > --
> > > > > Mathieu Boespflug
> > > > > Founder at http://tweag.io.
> > > > > 
> > > > > 
> > > > > On 30 June 2017 at 15:49, Wolfgang Jeltsch <wolfgang-it at jeltsc
> > > > > h.
> > > > > in
> > > > > fo>
> > > > > wrote:
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > Hi!
> > > > > > 
> > > > > > The blog post on http://blog.tweag.io/posts/2017-03-13-linea
> > > > > > r-
> > > > > > ty
> > > > > > pes.
> > > > > > html
> > > > > > says, “We are targeting a merge by the time of the 8.4
> > > > > > release
> > > > > > of
> > > > > > GHC.”
> > > > > > Does this mean that GHC 8.4 is expected to have support for
> > > > > > linear
> > > > > > types, or does it mean that the merge should happen after a
> > > > > > GHC
> > > > > > 8.4
> > > > > > branch has been forked off from the HEAD branch, so that
> > > > > > only
> > > > > > GHC
> > > > > > 8.6
> > > > > > will have support for linear types?
> > > > > > 
> > > > > > All the best,
> > > > > > Wolfgang
> > > > > > _______________________________________________
> > > > > > ghc-devs mailing list
> > > > > > ghc-devs at haskell.org
> > > > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> > > > _______________________________________________
> > > > ghc-devs mailing list
> > > > ghc-devs at haskell.org
> > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From palotai.robin at gmail.com  Fri Jun 30 20:41:18 2017
From: palotai.robin at gmail.com (Robin Palotai)
Date: Fri, 30 Jun 2017 22:41:18 +0200
Subject: Crossreferenced GHC 8.0.2
In-Reply-To: <CALuQ0m9DV4Rfg+2hR-nD8w5qrb7rRU50vRsYH3N_E55jXizXJw@mail.gmail.com>
References: <CANiLjA26K_KKBdH-AQy2wxVfBYEC6P451CM=+OgROZVOprEpcg@mail.gmail.com>
 <CALuQ0m9DV4Rfg+2hR-nD8w5qrb7rRU50vRsYH3N_E55jXizXJw@mail.gmail.com>
Message-ID: <CANiLjA0LK7JEx_Zf0X-kaCKqufzhp=btk0q-sz1RQNGwqKPvVQ@mail.gmail.com>

Hello Matthew,

Please see inline

2017-06-30 11:57 GMT+02:00 Matthew Pickering <matthewtpickering at gmail.com>:

> Hi Robin,
>
> This looks really useful for developers.
>
> 1. Would it be possible to provide a script which allows developers to
> build this index for themselves easily?
>

First, build and install the `ghc_kythe_wrapper` (instructions at
https://github.com/google/haskell-indexer).

Then build GHC with `make`, and capture the build log. Here's the hacky
script I used:
https://gist.github.com/robinp/222cf3a39cc19178ec8691522056d7fe

It filters the log and replaces GHC calls to call the wrapper, which emits
Kythe entries.

Finally run `serve.sh` of the repo to postprocess and serve the entries
through HTTP.

This is all pretty new, so feedback or questions welcome. If the method
distills, could try to write a more formal guide.


> 2. Is it possible to use this tool to detect dead code? Functions
> which are not used anywhere in the compiler.
>

We'll get there eventually, but for now the emitted data is not
fine-grained enough. The main missing piece is recording what are exported
entities of a module (Kythe schema discussion in progress). Without this,
unused locals (which anyway surface with -Wall) would be presented and
noisy.

Also, one would probably need to postprocess the data a bit for this, like
loading into a graph database or other ways.

Did you see https://github.com/ndmitchell/weeder by the way? Might work.


> 3. How are you pretty printing the output whilst retaining the source
> formatting? I had a quick look at the source but I couldn't see where
> the output was being produced.
>
>
The haskell-indexer-frontend-kythe emits Kythe (http://kythe.io) schema
data. We just export the source offsets, and it's the Kythe postprocessing
/ serving pipeline that does all the formatting.


> Cheers,
>
> Matt
>
> On Fri, Jun 30, 2017 at 8:55 AM, Robin Palotai <palotai.robin at gmail.com>
> wrote:
> > Hello GHC devs,
> >
> > I ran haskell-indexer [1] on the GHC 8.0.2 tarball, partly because I find
> > myself reading GHC source from time to time while working on the indexer,
> > and partly since it's fun.
> >
> > First, here you can click around [2] and find where beloved functions are
> > called from:
> > http://stuff.codereview.me/#ghc/compiler/hsSyn/HsBinds.hs?
> corpus&signature
> > (scroll down a bit, imports are not linked yet).
> >
> > Second, the way I indexed was pretty simple. I took the output of `make`,
> > replaced the ghc used with the ghc_kythe_wrapper, and filtered the lines
> > which included '-c', since I noticed that those duplicate previous large
> > compile lines. This only indexes the stage1 compilation AFAIU.
> >
> > Feel free to suggest a better way to tap into the compilations to get
> > everything properly indexed (and possibly only once).
> >
> > Any comments welcome!
> > Robin
> >
> > [1]: https://github.com/google/haskell-indexer
> >
> > [2]: TLDR UI quirks:
> >   - Click the :: in top-left to navigate file tree
> >   - Ctrl-Click (on linux) to go directly to definition (otherwise click
> > stuff from bottom pane)
> >   - Bottom pane often hides content, close it if stuck.
> >
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20170630/ad09200d/attachment.html>