From moritz.angermann at gmail.com  Wed Sep  1 07:19:34 2021
From: moritz.angermann at gmail.com (Moritz Angermann)
Date: Wed, 1 Sep 2021 09:19:34 +0200
Subject: GHC plugin authors?
In-Reply-To: <010f017b9db32a02-ec72e9c8-4c81-46c6-ac00-bf936ef2647c-000000@us-east-2.amazonses.com>
References: <010f017b9db32a02-ec72e9c8-4c81-46c6-ac00-bf936ef2647c-000000@us-east-2.amazonses.com>
Message-ID: <CAKfdd-wLz10mcLEyRaA23BfjX4azmZn4hdwMd-aaZUopoNj4JA@mail.gmail.com>

Hi Richard,

I believe that this is mostly due to plugin development happening to
satisfy a plugin need.  I doubt there is a grand unified vision for
plugins.  And I don't have one either.  I've dabbled with codegen plugins a
long time ago, these days I'm primarily concerned with plugins having a
chance to work in cross compilation settings, and even that is still a very
uncharted area, but Luite has come up with a hack and Sylvain is making
progress :-) We still don't have the cabal side fixes, where we'd need some
`plugin-depends` stanza, but all that only makes sense, once we have the
fundamentals for plugins disentangled in ghc.

I agree that a discussion on discourse might help.  But we won't know
without trying.

On Tue, Aug 31, 2021 at 9:34 PM Richard Eisenberg <lists at richarde.dev>
wrote:

> Hi all,
>
> I have seen a few posts from Sam Derbyshire here asking for feedback about
> plugin API design, and the responses have been minimal. This poses a design
> challenge, because the GHC folk who design the interface are sometimes
> distinct from the people who use the interface. We're trying to be good,
> seeking feedback from real, live clients. Is there a better way to do so
> than this mailing list? Example: we could create a Category on
> discourse.haskell.org, if that would reach the audience better. Or we
> could make a repo with issue trackers somewhere simply to track plugin
> design. What would work?
>
> (I recognize that I'm asking in a perhaps-ineffective channel for advice,
> but I really don't have a better idea right now. Maybe some of you plugin
> authors are here and will point us in a better direction.)
>
> Thanks,
> Richard
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210901/ba165ce8/attachment.html>

From christiaan.baaij at gmail.com  Wed Sep  1 08:45:58 2021
From: christiaan.baaij at gmail.com (Christiaan Baaij)
Date: Wed, 1 Sep 2021 10:45:58 +0200
Subject: GHC plugin authors?
In-Reply-To: <CAKfdd-wLz10mcLEyRaA23BfjX4azmZn4hdwMd-aaZUopoNj4JA@mail.gmail.com>
References: <010f017b9db32a02-ec72e9c8-4c81-46c6-ac00-bf936ef2647c-000000@us-east-2.amazonses.com>
 <CAKfdd-wLz10mcLEyRaA23BfjX4azmZn4hdwMd-aaZUopoNj4JA@mail.gmail.com>
Message-ID: <CABzw7bUFC8wRRwtigTK=xiMsVunFyo6NUMBkSjLrCcvkMNGVEg@mail.gmail.com>

Hi Richard,

So there's multiple reasons/aspects as to why I haven't responded
1. Sam's question/seeking feedback on changing the return type for
constraint solver plugins is something Sam and I discussed at HIW using
ICFPs airmeet instance, as in, it was something I suggested.

And the TL;DR of next three reasons: I can't find enough time / need to get
better at time management / need to hand over work
2. While I've had to maintain the
ghc-typelits-{natnormalize;knownnat;extra} constraint solver plugins as
part of maintaining my (and our company's) livelihood, I usually only try
the next version of the GHC module collection (API) when there's a binary
dist of like an alpha or RC.
3. I guess I've become conditioned to just do the regular impedance
matching for every new major GHC release, as can be witnessed by the
following CPP extravaganza:
https://github.com/clash-lang/ghc-typelits-natnormalise/blob/3f3ae60a796061b7a496f64ba71f4d57dedd01db/src/GHC/TypeLits/Normalise.hs#L181-L316
4. I'm a day-time-only programmer, and with our company growing I've had
less time for coding and maintaining software.

With regards to point 2. if I understand correctly, the gitlab CI creates
bindists as build artifacts, is that correct?
I guess it would be helpful if that were advertised more prominently, so
it's easier to test a new branch without having to build GHC.

With regards to point 3. The "blessed" GHC API, i.e. the module called
"GHC", "Plugins", "TcPlugins" (I forgot what their new names are in the
post ghc 9.0 era), were never enough to get everything done that needed to
be done.
That meant one had to reach out to the GHC module collection, which, for
good reasons (the type and constraint system is always under heavy
development), is changing with every major GHC release.
I guess (constraint solving) plugin creators whose day-job or hobby doesn't
include maintaining plugins got burned out by having to keep up with these
changes.
So I wonder how many constraint-solving plugins currently compile against
GHC 9.0, let alone GHC 9.2; and as a consequence I wonder how many people
are interested in these API changes (since they stopped maintaining their
plugins).

Perhaps they are interested though! But simply not registered to this
mailing list. So asking for feedback on both the haskell discourse and the
haskell sub-reddit certainly wouldn't hurt!

Finally, with regards to point 4. Constraint-solving plugins certainly
required some time to get into.
So often my colleagues look to me whenever there's a new alpha or RC out to
upgrade the plugins.
But Sam's changes for solving type family constraints certainly give the
impression that things will easier going forward!
So now is probably the best time to hand over the baton to one of my
colleagues and hopefully they can provide the asked-for feedback.

On Wed, 1 Sept 2021 at 09:21, Moritz Angermann <moritz.angermann at gmail.com>
wrote:

> Hi Richard,
>
> I believe that this is mostly due to plugin development happening to
> satisfy a plugin need.  I doubt there is a grand unified vision for
> plugins.  And I don't have one either.  I've dabbled with codegen plugins a
> long time ago, these days I'm primarily concerned with plugins having a
> chance to work in cross compilation settings, and even that is still a very
> uncharted area, but Luite has come up with a hack and Sylvain is making
> progress :-) We still don't have the cabal side fixes, where we'd need some
> `plugin-depends` stanza, but all that only makes sense, once we have the
> fundamentals for plugins disentangled in ghc.
>
> I agree that a discussion on discourse might help.  But we won't know
> without trying.
>
> On Tue, Aug 31, 2021 at 9:34 PM Richard Eisenberg <lists at richarde.dev>
> wrote:
>
>> Hi all,
>>
>> I have seen a few posts from Sam Derbyshire here asking for feedback
>> about plugin API design, and the responses have been minimal. This poses a
>> design challenge, because the GHC folk who design the interface are
>> sometimes distinct from the people who use the interface. We're trying to
>> be good, seeking feedback from real, live clients. Is there a better way to
>> do so than this mailing list? Example: we could create a Category on
>> discourse.haskell.org, if that would reach the audience better. Or we
>> could make a repo with issue trackers somewhere simply to track plugin
>> design. What would work?
>>
>> (I recognize that I'm asking in a perhaps-ineffective channel for advice,
>> but I really don't have a better idea right now. Maybe some of you plugin
>> authors are here and will point us in a better direction.)
>>
>> Thanks,
>> Richard
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210901/0e06d5ad/attachment.html>

From Gergo.Erdi at sc.com  Wed Sep  1 08:53:06 2021
From: Gergo.Erdi at sc.com (Erdi, Gergo)
Date: Wed, 1 Sep 2021 08:53:06 +0000
Subject: Inter-module inlining doesn't work, Id unfolding seems to be empty --
 why?
Message-ID: <HK0PR03MB48677B2143A0E878679567B9E9CD9@HK0PR03MB4867.apcprd03.prod.outlook.com>

PUBLIC

Hi,

The attached program uses the GHC 9.0.1 API to typecheck and compile a single module that contains a single definition marked as INLINE:

module Inline where

data A = A1 | A2
data B = B1 | B2

{-# INLINE f #-}
f :: A ->B
f A1 = B1
f A2 = B2

In my bigger program, I noticed that usages of `f` in another module aren't inlined, so I made the attached experiment: I compile this to desugared, simplified, tidied and prep'd Core, and then use updateModDetailsIdInfos to fill in the unfolding of `f`. At least, that's what I thought I do. However, if I print the unfolding of `f` after this process, I get `OtherCon []` instead of what I expected to be a CoreUnfolding. What am I doing wrong?

Thanks,
            Gergo

This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations

Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm

Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer.

Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. 

Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same.

Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210901/76cf530f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Main.hs
Type: application/octet-stream
Size: 5224 bytes
Desc: Main.hs
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210901/76cf530f/attachment.obj>

From Gergo.Erdi at sc.com  Wed Sep  1 09:35:45 2021
From: Gergo.Erdi at sc.com (Erdi, Gergo)
Date: Wed, 1 Sep 2021 09:35:45 +0000
Subject: Inter-module inlining doesn't work, Id unfolding seems to be
 empty -- why?
In-Reply-To: <HK0PR03MB48677B2143A0E878679567B9E9CD9@HK0PR03MB4867.apcprd03.prod.outlook.com>
References: <HK0PR03MB48677B2143A0E878679567B9E9CD9@HK0PR03MB4867.apcprd03.prod.outlook.com>
Message-ID: <HK0PR03MB48674E943CBA5186891B7AD3E9CD9@HK0PR03MB4867.apcprd03.prod.outlook.com>

PUBLIC

Never mind, turns out this was because OmitInterfacePragmas is turned on by default... (why?). After unsetting OmitInterfacePragmas in the dflags, I get the expected unfolding, and inter-module inlining works.

Unf{Src=InlineStable, TopLvl=True, Value=True, ConLike=True,
    WorkFree=True, Expandable=True,
    Guidance=ALWAYS_IF(arity=1,unsat_ok=False,boring_ok=False)
    Tmpl= \ (ds [Occ=Once1!] :: A) ->
            case ds of {
              A1 -> B1;
              A2 -> B2
            }}


From: Erdi, Gergo
Sent: Wednesday, September 1, 2021 4:53 PM
To: 'GHC' <ghc-devs at haskell.org>
Cc: Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>; 'gergo at erdi.hu' <gergo at erdi.hu>
Subject: Inter-module inlining doesn't work, Id unfolding seems to be empty -- why?


PUBLIC

Hi,

The attached program uses the GHC 9.0.1 API to typecheck and compile a single module that contains a single definition marked as INLINE:

module Inline where

data A = A1 | A2
data B = B1 | B2

{-# INLINE f #-}
f :: A ->B
f A1 = B1
f A2 = B2

In my bigger program, I noticed that usages of `f` in another module aren't inlined, so I made the attached experiment: I compile this to desugared, simplified, tidied and prep'd Core, and then use updateModDetailsIdInfos to fill in the unfolding of `f`. At least, that's what I thought I do. However, if I print the unfolding of `f` after this process, I get `OtherCon []` instead of what I expected to be a CoreUnfolding. What am I doing wrong?

Thanks,
            Gergo

This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations

Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm

Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer.

Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. 

Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same.

Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210901/46e768de/attachment-0001.html>

From moritz.angermann at gmail.com  Wed Sep  1 09:37:44 2021
From: moritz.angermann at gmail.com (Moritz Angermann)
Date: Wed, 1 Sep 2021 11:37:44 +0200
Subject: GHC plugin authors?
In-Reply-To: <CABzw7bUFC8wRRwtigTK=xiMsVunFyo6NUMBkSjLrCcvkMNGVEg@mail.gmail.com>
References: <010f017b9db32a02-ec72e9c8-4c81-46c6-ac00-bf936ef2647c-000000@us-east-2.amazonses.com>
 <CAKfdd-wLz10mcLEyRaA23BfjX4azmZn4hdwMd-aaZUopoNj4JA@mail.gmail.com>
 <CABzw7bUFC8wRRwtigTK=xiMsVunFyo6NUMBkSjLrCcvkMNGVEg@mail.gmail.com>
Message-ID: <CAKfdd-wF+X4Zpp_949EN8vaC0D6LgrVLMVOJKNYYCJ_1GVKKmQ@mail.gmail.com>

> With regards to point 2. if I understand correctly, the gitlab CI creates
> bindists as build artifacts, is that correct?
> I guess it would be helpful if that were advertised more prominently, so
> it's easier to test a new branch without having to build GHC.


Yes, they are. Let's take this MR for example (I just picked a random green
one)
https://gitlab.haskell.org/ghc/ghc/-/merge_requests/6452
If you look at the associated CI pipeline:
https://gitlab.haskell.org/ghc/ghc/-/pipelines/40166
You'll find the x86_64-deb-debug validate build here:
https://gitlab.haskell.org/ghc/ghc/-/jobs/774465
Which on the right side has "Job artifacts", clicking "browse" there leads
us to: https://gitlab.haskell.org/ghc/ghc/-/jobs/774465/artifacts/browse
from where we can obtain
https://gitlab.haskell.org/ghc/ghc/-/jobs/774465/artifacts/file/ghc-x86_64-deb9-linux-debug.tar.xz

Hope this helps, it's not very trivial, but at least this should give
directions on how to get the artifacts.
There are also nightly builds which have the artifacts attached as well.

On Wed, Sep 1, 2021 at 10:46 AM Christiaan Baaij <christiaan.baaij at gmail.com>
wrote:

> Hi Richard,
>
> So there's multiple reasons/aspects as to why I haven't responded
> 1. Sam's question/seeking feedback on changing the return type for
> constraint solver plugins is something Sam and I discussed at HIW using
> ICFPs airmeet instance, as in, it was something I suggested.
>
> And the TL;DR of next three reasons: I can't find enough time / need to
> get better at time management / need to hand over work
> 2. While I've had to maintain the
> ghc-typelits-{natnormalize;knownnat;extra} constraint solver plugins as
> part of maintaining my (and our company's) livelihood, I usually only try
> the next version of the GHC module collection (API) when there's a binary
> dist of like an alpha or RC.
> 3. I guess I've become conditioned to just do the regular impedance
> matching for every new major GHC release, as can be witnessed by the
> following CPP extravaganza:
> https://github.com/clash-lang/ghc-typelits-natnormalise/blob/3f3ae60a796061b7a496f64ba71f4d57dedd01db/src/GHC/TypeLits/Normalise.hs#L181-L316
> 4. I'm a day-time-only programmer, and with our company growing I've had
> less time for coding and maintaining software.
>
> With regards to point 2. if I understand correctly, the gitlab CI creates
> bindists as build artifacts, is that correct?
> I guess it would be helpful if that were advertised more prominently, so
> it's easier to test a new branch without having to build GHC.
>
> With regards to point 3. The "blessed" GHC API, i.e. the module called
> "GHC", "Plugins", "TcPlugins" (I forgot what their new names are in the
> post ghc 9.0 era), were never enough to get everything done that needed to
> be done.
> That meant one had to reach out to the GHC module collection, which, for
> good reasons (the type and constraint system is always under heavy
> development), is changing with every major GHC release.
> I guess (constraint solving) plugin creators whose day-job or hobby
> doesn't include maintaining plugins got burned out by having to keep up
> with these changes.
> So I wonder how many constraint-solving plugins currently compile against
> GHC 9.0, let alone GHC 9.2; and as a consequence I wonder how many people
> are interested in these API changes (since they stopped maintaining their
> plugins).
>
> Perhaps they are interested though! But simply not registered to this
> mailing list. So asking for feedback on both the haskell discourse and the
> haskell sub-reddit certainly wouldn't hurt!
>
> Finally, with regards to point 4. Constraint-solving plugins certainly
> required some time to get into.
> So often my colleagues look to me whenever there's a new alpha or RC out
> to upgrade the plugins.
> But Sam's changes for solving type family constraints certainly give the
> impression that things will easier going forward!
> So now is probably the best time to hand over the baton to one of my
> colleagues and hopefully they can provide the asked-for feedback.
>
> On Wed, 1 Sept 2021 at 09:21, Moritz Angermann <moritz.angermann at gmail.com>
> wrote:
>
>> Hi Richard,
>>
>> I believe that this is mostly due to plugin development happening to
>> satisfy a plugin need.  I doubt there is a grand unified vision for
>> plugins.  And I don't have one either.  I've dabbled with codegen plugins a
>> long time ago, these days I'm primarily concerned with plugins having a
>> chance to work in cross compilation settings, and even that is still a very
>> uncharted area, but Luite has come up with a hack and Sylvain is making
>> progress :-) We still don't have the cabal side fixes, where we'd need some
>> `plugin-depends` stanza, but all that only makes sense, once we have the
>> fundamentals for plugins disentangled in ghc.
>>
>> I agree that a discussion on discourse might help.  But we won't know
>> without trying.
>>
>> On Tue, Aug 31, 2021 at 9:34 PM Richard Eisenberg <lists at richarde.dev>
>> wrote:
>>
>>> Hi all,
>>>
>>> I have seen a few posts from Sam Derbyshire here asking for feedback
>>> about plugin API design, and the responses have been minimal. This poses a
>>> design challenge, because the GHC folk who design the interface are
>>> sometimes distinct from the people who use the interface. We're trying to
>>> be good, seeking feedback from real, live clients. Is there a better way to
>>> do so than this mailing list? Example: we could create a Category on
>>> discourse.haskell.org, if that would reach the audience better. Or we
>>> could make a repo with issue trackers somewhere simply to track plugin
>>> design. What would work?
>>>
>>> (I recognize that I'm asking in a perhaps-ineffective channel for
>>> advice, but I really don't have a better idea right now. Maybe some of you
>>> plugin authors are here and will point us in a better direction.)
>>>
>>> Thanks,
>>> Richard
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210901/f9218cb5/attachment.html>

From alexis.praga at gmail.com  Wed Sep  1 22:14:49 2021
From: alexis.praga at gmail.com (Alexis Praga)
Date: Thu, 2 Sep 2021 00:14:49 +0200
Subject: Trying to create a regression test for #20261
In-Reply-To: <8ade7367-e5cf-a771-a0a1-5d3ada8d75f2@well-typed.com>
References: <86a6l0i4n5.fsf@ecchi.mail-host-address-is-not-set>
 <8ade7367-e5cf-a771-a0a1-5d3ada8d75f2@well-typed.com>
Message-ID: <CAH3o9gx09tqtoJhFePfGCkECEbP6uwVK-dXjpAWw0TS-_VcrVQ@mail.gmail.com>

Thanks Adam for pointing me in the righ direction. I've attempted a MR with
some questions in them [1].

However, I have no clue about what this test [2] really does (why are there
unused functions called `unusedXX` ?).
If someone could give me a quick overview, that would be great.


Cheers,

[1] MR 6457
[2] https://github.com/zachjs/ghc-issue-20261/blob/main/Parse.y

On Sun, Aug 29, 2021 at 9:56 PM Adam Gundry <adam at well-typed.com> wrote:

> Hi Alexis,
>
>
> On 29/08/2021 17:42, Alexis Praga wrote:
> > As a intermediate beginner in Haskell, I want to try to tackle a small
> > issue for contributing to GHC.
>
> Welcome! Thanks for trying your hand at contributing to GHC.
>
>
> > I've looked into https://gitlab.haskell.org/ghc/ghc/-/issues/20261, but
> > it's not clear how to integrate a running time in a regression test.
> >
> > From what I understand, regression tests should be compared against
> > sucess/failure or a given output.
> > Should there be a hard-coded limit ? If that's the case, how could I
> > create such a test ?  I could not find a similar test in the source
> > code.
>
> Have a look in testsuite/tests/perf/compiler. That directory contains a
> number of compiler performance tests, which work by having the test
> runner compare metrics about the compilation run for the current commit
> vs. previous commits. The all.T file lists the tests to run and the
> metrics to collect.
>
> There's more details on the wiki:
>
> https://gitlab.haskell.org/ghc/ghc/-/wikis/building/running-tests/adding#performance-tests
>
> Hope this helps,
>
> Adam
>
>
> --
> Adam Gundry, Haskell Consultant
> Well-Typed LLP, https://www.well-typed.com/
>
> Registered in England & Wales, OC335890
> 118 Wymering Mansions, Wymering Road, London W9 2NF, England
>


-- 

   Alexis Praga
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210902/15dce790/attachment.html>

From omeragacan at gmail.com  Thu Sep  2 15:47:08 2021
From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=)
Date: Thu, 2 Sep 2021 18:47:08 +0300
Subject: Potential improvement in compacting GC
In-Reply-To: <CAMQQO3nHFzL5aVFSq-RkSVcoS6g_WhjHJrsDF_MboowMwXCnAA@mail.gmail.com>
References: <CAMQQO3=7haokj1uUUt9KoSQ2YLOd7Y3Ue20atXhW2HwL=Z+2nQ@mail.gmail.com>
 <CAMQQO3nHFzL5aVFSq-RkSVcoS6g_WhjHJrsDF_MboowMwXCnAA@mail.gmail.com>
Message-ID: <CAMQQO3m4hCVerYnbyoakj6QFdL1ZXzjzu54JK857+e0FByMg2Q@mail.gmail.com>

Here's another improvement that fixes a very old issue in GHC's compacting GC
[1].

To summarize the problem: when untreading an object we update references to the
object that we've seen so far to the object's new location. But to get the
object's new location we need to know the object's size, because depending on
the size we may need to move the object to a new block (when the current block
does not have enough space for the object).

For this we currently walk the thread twice, once to get the info table (used
to get the size), then again to update references to the object. Ideally we
want to do just one pass when unthreading.

The solution is explained in The GC Handbook, section 3.4. Instead of using one
bit to mark an object, we use two bits: one for the first word of the object,
one for the last word. Using two bits is not a problem in GHC because heap
objects are at least 2 words. For example, an object with two words is marked
with `11`, 3 words is marked with `101` and so on.

Now we can scan the bitmap to find object size, and unthread it without having
to find the info table first.

Ömer

[1]: https://github.com/ghc/ghc/blob/922c6bc8dd8d089cfe4b90ec2120cb48959ba2b5/rts/sm/Compact.c#L844-L848

Ömer Sinan Ağacan <omeragacan at gmail.com>, 14 Tem 2021 Çar, 09:27
tarihinde şunu yazdı:
>
> Two other ideas that should improve GHC's compacting GC much more
> significantly. I've implemented both of these in another project and the
> results were great. In a GC benchmark (mutator does almost no work other than
> allocating data using a bump allocator), first one reduced Wasm instructions
> executed by 14%, second one 19.8%.
>
> Both of these ideas require pushing object headers to the mark stack with the
> objects, which means larger mark stacks. This is the only downside of these
> algorithms.
>
> - Instead of marking and then threading in the next pass, mark phase threads
>   all fields when pushing the fields to the mark stack. We still need two other
>   passes: one to unthread headers, another to move the objects. (we can't do
>   both in one pass, let me know if you're curious and I can elaborate)
>
>   This has the same number of passes as the current implementation, but it only
>   visits object fields once. Currently, we visit fields once when marking, to
>   mark fields, then again in `update_fwd`. This implementation does both in one
>   pass over the fields. `update_fwd` does not visit fields.
>
>   This reduced Wasm instructions executed by 14% in my benchmark.
>
> - Marking phase threads backwards pointers (ignores forwards pointers). Then we
>   do one pass instead of two: for a marked object, unthread it (update
>   forwards pointers to the object's new location), move it to its new location,
>   then thread its forwards pointers. This completely eliminates one of the 3
>   passes, but fields need to be visited twice as before (unlike the algorithm
>   above).
>
>   I think this one is originally described in "An Efficient Garbage Compaction
>   Algorithm", but I found that paper to be difficult to follow. A short
>   description of the same algorithm exists in "High-Performance Garbage
>   Collection for Memory-Constrained Environments" in section 5.1.2.
>
>   This reduced Wasm instructions executed by 19.8% in my benchmark.
>
>   In this algorithm, fields that won't be moved can be threaded any time before
>   the second pass (pointed objects need to be marked and pushed to the mark
>   stack with headers before threading a field). For example, in GHC, mut list
>   entries can be threaded before or after marking (but before the second pass)
>   as IIRC mut lists are not moved. Same for fields of large objects.
>
> As far as I can see, mark-compact GC is still the default when max heap size is
> specified and the oldest generation size is (by default) more than 30% of the
> max heap size. I'm not sure if max heap size is specified often (it's off by
> default), so not sure what would be the impact of these improvements be, but if
> anyone would be interested in funding me to implement these ideas (second
> algorithm above, and the bitmap iteration in the previous email) I could try to
> allocate one or two days a week to finish in a few months.
>
> Normally these are simple changes, but it's difficult to test and debug GHC's
> RTS as we don't have a test suite readily available and the code is not easily
> testable. In my previous implementations of these algorithms I had unit tests
> for the GC where I could easily generate arbitrary graphs (with cycles,
> backwards ptrs, forwards ptrs, ptrs from/to roots etc.) and test GC in
> isolation. Implementation of (2) took less than a day, and I didn't have to
> debug it more once the tests passed. It's really unfortunate that GHC's RTS
> makes this kind of thing difficult..
>
> Ömer
>
> Ömer Sinan Ağacan <omeragacan at gmail.com>, 7 Oca 2021 Per, 20:42
> tarihinde şunu yazdı:
> >
> > Hello,
> >
> > I recently implemented the algorithm used by GHC's compacting GC in another
> > project. The algorithm (after marking) makes two passes over the heap
> > /generation. In GHC, these passes are implemented in [1] and in the next
> > function.
> >
> > In my implementation I tried 3 ways of implementing these passes, one of which
> > is the same as GHC's code, and benchmarked each version. I found that the
> > fastest implementation is not what's used in GHC, but it could easily be used.
> >
> > I should mention that my code targets Wasm, and I benchmarked Wasm instructions
> > executed. Not CPU cycles, CPU instructions, or anything like that. It's very
> > likely that the results will be different when benchmarking code running on
> > actual hardware.
> >
> > Secondly, in my case the heap is mostly dead (residency is low). In GHC,
> > compaction for the oldest generation is enabled when residency passes a
> > threshold, so the assumption is the heap is mostly live. I'm guessing this
> > should also make some difference.
> >
> > Anyway, the first implementation I tried was similar to GHC's scan, but I did
> >
> >     scan += object_size(scan);
> >
> > instead of bumping scan by one, as GHC does in [2]. This was the slowest
> > version.
> >
> > Second implementation did the same as GHC (bumped scan by one). This was
> > faster, but still slower than the next version.
> >
> > What I found to be the best is scanning the bitmap, not the heap. The bitmap
> > iterator reads one word at a time. In each iteration it checks if the bitmap
> > word is 0. In GHC, in the best case this can skip 16 words on heap on 32-bit
> > systems, and 32 words on 64-bit. Reminder: we need two bits per object in the
> > bitmap, see [3]. (this is not the case in my implementation so the payoff is
> > better)
> >
> > When the bitmap word is not 0 I use "count trailing zeros" (or "count leading
> > zeros" depending on the bitmap implementation) to get the number of words to
> > skip. This is a single instruction on Wasm and x86 (TZCNT or LZCNT, available
> > via __builtin_ctzl and __builtin_clzl in gcc).
> >
> > So instead of skipping one word at a time, this can potentially skip 16 words
> > (or 32 on 64-bit architectures). When that's not possible, it can still skip
> > multiple words by using ctz/clz.
> >
> > Ömer
> >
> > [1]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rts/sm/Compact.c#L824-L879
> > [2]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rts/sm/Compact.c#L838
> > [3]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rts/sm/Compact.h#L18-L55

From rpglover64 at gmail.com  Thu Sep  2 16:10:12 2021
From: rpglover64 at gmail.com (Alex Rozenshteyn)
Date: Thu, 2 Sep 2021 12:10:12 -0400
Subject: New implementation for `ImpredicativeTypes`
In-Reply-To: <DB7PR83MB0313E44B53429012C3BE69A6ADB70@DB7PR83MB0313.EURPRD83.prod.outlook.com>
References: <CAHnFXOumYR+nKkBv6B6_zHVA2D=b9kZVyZ3vTTu87R07h90YuA@mail.gmail.com>
 <CALm==BVhBZ4Gwd+011zmy-qT266JSU+BHH=fjOac_P1bmANTyw@mail.gmail.com>
 <VI1PR83MB0319241FB05C0F793E35C7AEADBA0@VI1PR83MB0319.EURPRD83.prod.outlook.com>
 <CALm==BWMAJgs+t=1pF4H=-WH3wRo1A5ZPfyhOsY5d1y4XL9xCA@mail.gmail.com>
 <DB7PR83MB0313E44B53429012C3BE69A6ADB70@DB7PR83MB0313.EURPRD83.prod.outlook.com>
Message-ID: <CALm==BWCtjvEZ296usfNzP7mj5Cne4t5vMAivSd0mYRrrdeF_Q@mail.gmail.com>

If I understand correctly, the recent ICFP paper "An Existential Crisis
Resolved <https://dl.acm.org/doi/pdf/10.1145/3473569>" finally enables
this; is that right?

On Mon, Sep 9, 2019 at 12:00 PM Simon Peyton Jones <simonpj at microsoft.com>
wrote:

> Suppose Haskell *did* have existentials;
>
>
>
> Yes, I think that’s an interesting thing to work on!  I’m not sure what
> the implications would be.  At very least we’d need to extend System FC
> (GHC’s intermediate language) with existential types and the corresponding
> pack and unpack syntactic forms.
>
>
>
> I don’t know of any work studying that question specifically, but others
> may have pointers.
>
>
>
> simon
>
>
>
> *From:* Alex Rozenshteyn <rpglover64 at gmail.com>
> *Sent:* 06 September 2019 15:21
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Cc:* Alejandro Serrano Mena <trupill at gmail.com>; GHC developers <
> ghc-devs at haskell.org>
> *Subject:* Re: New implementation for `ImpredicativeTypes`
>
>
>
> Hi Simon,
>
>
>
> You're exactly right, of course. My example is confusing, so let me see if
> I can clarify.
>
>
>
> What I want in the ideal is map show [1, 'a', "b"]. That is, minimal
> syntactic overhead to mapping a function over multiple values of distinct
> types that results in a homogeneous list. As the reddit thread points out,
> there are workarounds involving TH or wrapping each element in a
> constructor or using bespoke operators, but when it comes down to it, none
> of them actually allows me to say what I *mean*; the TH one is closest,
> but I reach for TH only in times of desperation.
>
>
>
> I had thought that one of the things preventing this was lack of
> impredicative instantiation, but now I'm not sure. Suppose Haskell *did*
> have existentials; would map show @(exists a. Show a => a) [1, 'a', "b"]
> work in current Haskell and/or in quick-look?
>
>
>
> Tangentially, do you have a reference for what difficulties arise in
> adding existentials to Haskell? I have a feeling that it would make working
> with GADTs more ergonomic.
>
>
>
> On Fri, Sep 6, 2019 at 12:33 AM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> I’m confused.   Char does not have the type (forall a. Show a =>a), so our
> example is iill-typed in System F, never mind about type inference.
> Perhaps there’s a typo?   I think you may have ment
>
>                exists a. Show a => a
>
> which doesn’t exist in Haskell.  You can write existentials with a data
> type
>
>
>
> data Showable where
>
>    S :: forall a. Show a => a -> Showable
>
>
>
> Then
>
>                map show [S 1, S ‘a’, S “b”]
>
> works fine today (without our new stuff), provided you say
>
>
>
>                instance Show Showable where
>
>                  show (S x) = show x
>
>
>
> Our new system can only type programs that can be written in System F.
> (The tricky bit is inferring the impredicative instantiations.)
>
>
>
> Simon
>
>
>
> *From:* ghc-devs <ghc-devs-bounces at haskell.org> *On Behalf Of *Alex
> Rozenshteyn
> *Sent:* 06 September 2019 03:31
> *To:* Alejandro Serrano Mena <trupill at gmail.com>
> *Cc:* GHC developers <ghc-devs at haskell.org>
> *Subject:* Re: New implementation for `ImpredicativeTypes`
>
>
>
> I didn't say anything when you were requesting use cases, so I have no
> right to complain, but I'm still a little disappointed that this doesn't
> fix my (admittedly very minor) issue:
> https://www.reddit.com/r/haskell/comments/3am0qa/existentials_and_the_heterogenous_list_fallacy/csdwlp2/?context=8&depth=9
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.reddit.com%2Fr%2Fhaskell%2Fcomments%2F3am0qa%2Fexistentials_and_the_heterogenous_list_fallacy%2Fcsdwlp2%2F%3Fcontext%3D8%26depth%3D9&data=02%7C01%7Csimonpj%40microsoft.com%7C911c05e1dc5041e685bd08d732d57261%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637033764634397513&sdata=IeOaMmtbwIFEXOBwvLkElsqqS84q50QjFD7MFW0hpK8%3D&reserved=0>
>
>
>
> For those who don't want to click on the reddit link: I would like to be
> able to write something like map show ([1, 'a', "b"] :: [forall a. Show a
> => a]), and have it work.
>
>
>
> On Wed, Sep 4, 2019 at 8:13 AM Alejandro Serrano Mena <trupill at gmail.com>
> wrote:
>
> Hi all,
>
> As I mentioned some time ago, we have been busy working on a new
> implementation of `ImpredicativeTypes` for GHC. I am very thankful to
> everybody who back then sent us examples of impredicativity which would be
> nice to support, as far as we know this branch supports all of them! :)
>
>
>
> If you want to try it, at
> https://gitlab.haskell.org/trupill/ghc/commit/a3f95a0fe0f647702fd7225fa719a8062a4cc0a5/pipelines?ref=quick-look-build
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Ftrupill%2Fghc%2Fcommit%2Fa3f95a0fe0f647702fd7225fa719a8062a4cc0a5%2Fpipelines%3Fref%3Dquick-look-build&data=02%7C01%7Csimonpj%40microsoft.com%7C911c05e1dc5041e685bd08d732d57261%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637033764634397513&sdata=fWmUVHUC5hFG%2FBqNwXl1N4par46XLdHQAUxs2DstpvQ%3D&reserved=0>
> you can find the result of the pipeline, which includes builds for several
> platforms (click on the "Artifacts" button, the one which looks like a
> cloud, to get them). The code is being developed at
> https://gitlab.haskell.org/trupill/ghc
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Ftrupill%2Fghc&data=02%7C01%7Csimonpj%40microsoft.com%7C911c05e1dc5041e685bd08d732d57261%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637033764634407517&sdata=OMEyEff9CzH3iNG6DHUL%2B5TRSCgYLmEXikD9M7BBdQE%3D&reserved=0>
> .
>
>
>
> Any code should run *unchanged* except for some eta-expansion required for
> some specific usage patterns of higher-rank types. Please don't hesitate to
> ask any questions or clarifications about it. A merge request for tracking
> this can be found at
> https://gitlab.haskell.org/ghc/ghc/merge_requests/1659
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2Fmerge_requests%2F1659&data=02%7C01%7Csimonpj%40microsoft.com%7C911c05e1dc5041e685bd08d732d57261%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637033764634407517&sdata=lkLoNtsgThjpOO%2BEujkwYk3%2BujMrcymJJtrJ7w9FDbY%3D&reserved=0>
>
>
>
> Kind regards,
>
> Alejandro
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> <https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=02%7C01%7Csimonpj%40microsoft.com%7C911c05e1dc5041e685bd08d732d57261%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637033764634417525&sdata=2ayQ0%2Fz0YHIxyCVNGybOigFIxCmsd2lEFeQYSbSrvv4%3D&reserved=0>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210902/138fe6c4/attachment.html>

From simonpj at microsoft.com  Thu Sep  2 16:42:05 2021
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Thu, 2 Sep 2021 16:42:05 +0000
Subject: New implementation for `ImpredicativeTypes`
In-Reply-To: <CALm==BWCtjvEZ296usfNzP7mj5Cne4t5vMAivSd0mYRrrdeF_Q@mail.gmail.com>
References: <CAHnFXOumYR+nKkBv6B6_zHVA2D=b9kZVyZ3vTTu87R07h90YuA@mail.gmail.com>
 <CALm==BVhBZ4Gwd+011zmy-qT266JSU+BHH=fjOac_P1bmANTyw@mail.gmail.com>
 <VI1PR83MB0319241FB05C0F793E35C7AEADBA0@VI1PR83MB0319.EURPRD83.prod.outlook.com>
 <CALm==BWMAJgs+t=1pF4H=-WH3wRo1A5ZPfyhOsY5d1y4XL9xCA@mail.gmail.com>
 <DB7PR83MB0313E44B53429012C3BE69A6ADB70@DB7PR83MB0313.EURPRD83.prod.outlook.com>
 <CALm==BWCtjvEZ296usfNzP7mj5Cne4t5vMAivSd0mYRrrdeF_Q@mail.gmail.com>
Message-ID: <VI1PR83MB026923BF6143E9EF17840C5EADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>

If I understand correctly, the recent ICFP paper "An Existential Crisis Resolved<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fdoi%2Fpdf%2F10.1145%2F3473569&data=04%7C01%7Csimonpj%40microsoft.com%7Caee2e0a922204ae3611208d96e2c2c57%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661959634445057%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wsN3Z7t8jCiExzl38%2F2IwNNsPH3Pq5CHJ7dQca5R2Y4%3D&reserved=0>" finally enables this; is that right?
It describes one way to include existentials in GHC's intermediate language, which is a real contribution. But it is not a small change.  So it's not just a question of saying "just add that paper to GHC and voila job done".

Simon

From: Alex Rozenshteyn <rpglover64 at gmail.com>
Sent: 02 September 2021 17:10
To: Simon Peyton Jones <simonpj at microsoft.com>
Cc: GHC developers <ghc-devs at haskell.org>
Subject: Re: New implementation for `ImpredicativeTypes`

If I understand correctly, the recent ICFP paper "An Existential Crisis Resolved<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fdoi%2Fpdf%2F10.1145%2F3473569&data=04%7C01%7Csimonpj%40microsoft.com%7Caee2e0a922204ae3611208d96e2c2c57%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661959634445057%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wsN3Z7t8jCiExzl38%2F2IwNNsPH3Pq5CHJ7dQca5R2Y4%3D&reserved=0>" finally enables this; is that right?

On Mon, Sep 9, 2019 at 12:00 PM Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
Suppose Haskell did have existentials;

Yes, I think that's an interesting thing to work on!  I'm not sure what the implications would be.  At very least we'd need to extend System FC (GHC's intermediate language) with existential types and the corresponding pack and unpack syntactic forms.

I don't know of any work studying that question specifically, but others may have pointers.

simon

From: Alex Rozenshteyn <rpglover64 at gmail.com<mailto:rpglover64 at gmail.com>>
Sent: 06 September 2019 15:21
To: Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
Cc: Alejandro Serrano Mena <trupill at gmail.com<mailto:trupill at gmail.com>>; GHC developers <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: Re: New implementation for `ImpredicativeTypes`

Hi Simon,

You're exactly right, of course. My example is confusing, so let me see if I can clarify.

What I want in the ideal is map show [1, 'a', "b"]. That is, minimal syntactic overhead to mapping a function over multiple values of distinct types that results in a homogeneous list. As the reddit thread points out, there are workarounds involving TH or wrapping each element in a constructor or using bespoke operators, but when it comes down to it, none of them actually allows me to say what I mean; the TH one is closest, but I reach for TH only in times of desperation.

I had thought that one of the things preventing this was lack of impredicative instantiation, but now I'm not sure. Suppose Haskell did have existentials; would map show @(exists a. Show a => a) [1, 'a', "b"] work in current Haskell and/or in quick-look?

Tangentially, do you have a reference for what difficulties arise in adding existentials to Haskell? I have a feeling that it would make working with GADTs more ergonomic.

On Fri, Sep 6, 2019 at 12:33 AM Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
I'm confused.   Char does not have the type (forall a. Show a =>a), so our example is iill-typed in System F, never mind about type inference.  Perhaps there's a typo?   I think you may have ment
               exists a. Show a => a
which doesn't exist in Haskell.  You can write existentials with a data type

data Showable where
   S :: forall a. Show a => a -> Showable

Then
               map show [S 1, S 'a', S "b"]
works fine today (without our new stuff), provided you say

               instance Show Showable where
                 show (S x) = show x

Our new system can only type programs that can be written in System F.   (The tricky bit is inferring the impredicative instantiations.)

Simon

From: ghc-devs <ghc-devs-bounces at haskell.org<mailto:ghc-devs-bounces at haskell.org>> On Behalf Of Alex Rozenshteyn
Sent: 06 September 2019 03:31
To: Alejandro Serrano Mena <trupill at gmail.com<mailto:trupill at gmail.com>>
Cc: GHC developers <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: Re: New implementation for `ImpredicativeTypes`

I didn't say anything when you were requesting use cases, so I have no right to complain, but I'm still a little disappointed that this doesn't fix my (admittedly very minor) issue: https://www.reddit.com/r/haskell/comments/3am0qa/existentials_and_the_heterogenous_list_fallacy/csdwlp2/?context=8&depth=9<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.reddit.com%2Fr%2Fhaskell%2Fcomments%2F3am0qa%2Fexistentials_and_the_heterogenous_list_fallacy%2Fcsdwlp2%2F%3Fcontext%3D8%26depth%3D9&data=04%7C01%7Csimonpj%40microsoft.com%7Caee2e0a922204ae3611208d96e2c2c57%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661959634445057%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=HEn92VgTK3lOrYywm7gml4kn%2BYRreZXxtCKvLib805g%3D&reserved=0>

For those who don't want to click on the reddit link: I would like to be able to write something like map show ([1, 'a', "b"] :: [forall a. Show a => a]), and have it work.

On Wed, Sep 4, 2019 at 8:13 AM Alejandro Serrano Mena <trupill at gmail.com<mailto:trupill at gmail.com>> wrote:
Hi all,
As I mentioned some time ago, we have been busy working on a new implementation of `ImpredicativeTypes` for GHC. I am very thankful to everybody who back then sent us examples of impredicativity which would be nice to support, as far as we know this branch supports all of them! :)

If you want to try it, at https://gitlab.haskell.org/trupill/ghc/commit/a3f95a0fe0f647702fd7225fa719a8062a4cc0a5/pipelines?ref=quick-look-build<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Ftrupill%2Fghc%2Fcommit%2Fa3f95a0fe0f647702fd7225fa719a8062a4cc0a5%2Fpipelines%3Fref%3Dquick-look-build&data=04%7C01%7Csimonpj%40microsoft.com%7Caee2e0a922204ae3611208d96e2c2c57%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661959634455057%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=v5%2BPdW%2Fcx2OXNiIrQ8Dsa45rUgh1CRr6ERNzxbihxoA%3D&reserved=0> you can find the result of the pipeline, which includes builds for several platforms (click on the "Artifacts" button, the one which looks like a cloud, to get them). The code is being developed at https://gitlab.haskell.org/trupill/ghc<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Ftrupill%2Fghc&data=04%7C01%7Csimonpj%40microsoft.com%7Caee2e0a922204ae3611208d96e2c2c57%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661959634465051%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=pFQzrPCiZXF0Y3KqmxjlhrDX1EVddJqII%2B%2BPTJb65h8%3D&reserved=0>.

Any code should run *unchanged* except for some eta-expansion required for some specific usage patterns of higher-rank types. Please don't hesitate to ask any questions or clarifications about it. A merge request for tracking this can be found at https://gitlab.haskell.org/ghc/ghc/merge_requests/1659<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2Fmerge_requests%2F1659&data=04%7C01%7Csimonpj%40microsoft.com%7Caee2e0a922204ae3611208d96e2c2c57%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661959634475049%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=j0JcM0Q%2Bj7JSSb6lV%2F6Aj4lAZWjAysg%2FHg5cb4x00%2FY%3D&reserved=0>

Kind regards,
Alejandro
_______________________________________________
ghc-devs mailing list
ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=04%7C01%7Csimonpj%40microsoft.com%7Caee2e0a922204ae3611208d96e2c2c57%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661959634485038%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=9i55zUcM%2BX%2FN7ngvTARBOvfm962IJ9SYfxfa46f7wm8%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210902/d0ad6dbb/attachment.html>

From rpglover64 at gmail.com  Thu Sep  2 17:13:00 2021
From: rpglover64 at gmail.com (Alex Rozenshteyn)
Date: Thu, 2 Sep 2021 13:13:00 -0400
Subject: New implementation for `ImpredicativeTypes`
In-Reply-To: <VI1PR83MB026923BF6143E9EF17840C5EADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
References: <CAHnFXOumYR+nKkBv6B6_zHVA2D=b9kZVyZ3vTTu87R07h90YuA@mail.gmail.com>
 <CALm==BVhBZ4Gwd+011zmy-qT266JSU+BHH=fjOac_P1bmANTyw@mail.gmail.com>
 <VI1PR83MB0319241FB05C0F793E35C7AEADBA0@VI1PR83MB0319.EURPRD83.prod.outlook.com>
 <CALm==BWMAJgs+t=1pF4H=-WH3wRo1A5ZPfyhOsY5d1y4XL9xCA@mail.gmail.com>
 <DB7PR83MB0313E44B53429012C3BE69A6ADB70@DB7PR83MB0313.EURPRD83.prod.outlook.com>
 <CALm==BWCtjvEZ296usfNzP7mj5Cne4t5vMAivSd0mYRrrdeF_Q@mail.gmail.com>
 <VI1PR83MB026923BF6143E9EF17840C5EADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
Message-ID: <CALm==BV8iGBN5bU57AMPYb-0pdSNc2qAcGP-uTbFe+3MO8yHfA@mail.gmail.com>

>
> So it’s not just a question of saying “just add that paper to GHC and
> voila job done”.


Of course not. The same was true for QuickLook, though, wasn't it?

On Thu, Sep 2, 2021 at 12:42 PM Simon Peyton Jones <simonpj at microsoft.com>
wrote:

> If I understand correctly, the recent ICFP paper "An Existential Crisis
> Resolved
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fdoi%2Fpdf%2F10.1145%2F3473569&data=04%7C01%7Csimonpj%40microsoft.com%7Caee2e0a922204ae3611208d96e2c2c57%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661959634445057%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wsN3Z7t8jCiExzl38%2F2IwNNsPH3Pq5CHJ7dQca5R2Y4%3D&reserved=0>"
> finally enables this; is that right?
>
> It describes one way to include existentials in GHC’s intermediate
> language, which is a real contribution. *But it is not a small change*.
> So it’s not just a question of saying “just add that paper to GHC and voila
> job done”.
>
>
>
> Simon
>
>
>
> *From:* Alex Rozenshteyn <rpglover64 at gmail.com>
> *Sent:* 02 September 2021 17:10
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Cc:* GHC developers <ghc-devs at haskell.org>
> *Subject:* Re: New implementation for `ImpredicativeTypes`
>
>
>
> If I understand correctly, the recent ICFP paper "An Existential Crisis
> Resolved
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fdoi%2Fpdf%2F10.1145%2F3473569&data=04%7C01%7Csimonpj%40microsoft.com%7Caee2e0a922204ae3611208d96e2c2c57%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661959634445057%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wsN3Z7t8jCiExzl38%2F2IwNNsPH3Pq5CHJ7dQca5R2Y4%3D&reserved=0>"
> finally enables this; is that right?
>
>
>
> On Mon, Sep 9, 2019 at 12:00 PM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> Suppose Haskell *did* have existentials;
>
>
>
> Yes, I think that’s an interesting thing to work on!  I’m not sure what
> the implications would be.  At very least we’d need to extend System FC
> (GHC’s intermediate language) with existential types and the corresponding
> pack and unpack syntactic forms.
>
>
>
> I don’t know of any work studying that question specifically, but others
> may have pointers.
>
>
>
> simon
>
>
>
> *From:* Alex Rozenshteyn <rpglover64 at gmail.com>
> *Sent:* 06 September 2019 15:21
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Cc:* Alejandro Serrano Mena <trupill at gmail.com>; GHC developers <
> ghc-devs at haskell.org>
> *Subject:* Re: New implementation for `ImpredicativeTypes`
>
>
>
> Hi Simon,
>
>
>
> You're exactly right, of course. My example is confusing, so let me see if
> I can clarify.
>
>
>
> What I want in the ideal is map show [1, 'a', "b"]. That is, minimal
> syntactic overhead to mapping a function over multiple values of distinct
> types that results in a homogeneous list. As the reddit thread points out,
> there are workarounds involving TH or wrapping each element in a
> constructor or using bespoke operators, but when it comes down to it, none
> of them actually allows me to say what I *mean*; the TH one is closest,
> but I reach for TH only in times of desperation.
>
>
>
> I had thought that one of the things preventing this was lack of
> impredicative instantiation, but now I'm not sure. Suppose Haskell *did*
> have existentials; would map show @(exists a. Show a => a) [1, 'a', "b"]
> work in current Haskell and/or in quick-look?
>
>
>
> Tangentially, do you have a reference for what difficulties arise in
> adding existentials to Haskell? I have a feeling that it would make working
> with GADTs more ergonomic.
>
>
>
> On Fri, Sep 6, 2019 at 12:33 AM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> I’m confused.   Char does not have the type (forall a. Show a =>a), so our
> example is iill-typed in System F, never mind about type inference.
> Perhaps there’s a typo?   I think you may have ment
>
>                exists a. Show a => a
>
> which doesn’t exist in Haskell.  You can write existentials with a data
> type
>
>
>
> data Showable where
>
>    S :: forall a. Show a => a -> Showable
>
>
>
> Then
>
>                map show [S 1, S ‘a’, S “b”]
>
> works fine today (without our new stuff), provided you say
>
>
>
>                instance Show Showable where
>
>                  show (S x) = show x
>
>
>
> Our new system can only type programs that can be written in System F.
> (The tricky bit is inferring the impredicative instantiations.)
>
>
>
> Simon
>
>
>
> *From:* ghc-devs <ghc-devs-bounces at haskell.org> *On Behalf Of *Alex
> Rozenshteyn
> *Sent:* 06 September 2019 03:31
> *To:* Alejandro Serrano Mena <trupill at gmail.com>
> *Cc:* GHC developers <ghc-devs at haskell.org>
> *Subject:* Re: New implementation for `ImpredicativeTypes`
>
>
>
> I didn't say anything when you were requesting use cases, so I have no
> right to complain, but I'm still a little disappointed that this doesn't
> fix my (admittedly very minor) issue:
> https://www.reddit.com/r/haskell/comments/3am0qa/existentials_and_the_heterogenous_list_fallacy/csdwlp2/?context=8&depth=9
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.reddit.com%2Fr%2Fhaskell%2Fcomments%2F3am0qa%2Fexistentials_and_the_heterogenous_list_fallacy%2Fcsdwlp2%2F%3Fcontext%3D8%26depth%3D9&data=04%7C01%7Csimonpj%40microsoft.com%7Caee2e0a922204ae3611208d96e2c2c57%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661959634445057%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=HEn92VgTK3lOrYywm7gml4kn%2BYRreZXxtCKvLib805g%3D&reserved=0>
>
>
>
> For those who don't want to click on the reddit link: I would like to be
> able to write something like map show ([1, 'a', "b"] :: [forall a. Show a
> => a]), and have it work.
>
>
>
> On Wed, Sep 4, 2019 at 8:13 AM Alejandro Serrano Mena <trupill at gmail.com>
> wrote:
>
> Hi all,
>
> As I mentioned some time ago, we have been busy working on a new
> implementation of `ImpredicativeTypes` for GHC. I am very thankful to
> everybody who back then sent us examples of impredicativity which would be
> nice to support, as far as we know this branch supports all of them! :)
>
>
>
> If you want to try it, at
> https://gitlab.haskell.org/trupill/ghc/commit/a3f95a0fe0f647702fd7225fa719a8062a4cc0a5/pipelines?ref=quick-look-build
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Ftrupill%2Fghc%2Fcommit%2Fa3f95a0fe0f647702fd7225fa719a8062a4cc0a5%2Fpipelines%3Fref%3Dquick-look-build&data=04%7C01%7Csimonpj%40microsoft.com%7Caee2e0a922204ae3611208d96e2c2c57%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661959634455057%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=v5%2BPdW%2Fcx2OXNiIrQ8Dsa45rUgh1CRr6ERNzxbihxoA%3D&reserved=0>
> you can find the result of the pipeline, which includes builds for several
> platforms (click on the "Artifacts" button, the one which looks like a
> cloud, to get them). The code is being developed at
> https://gitlab.haskell.org/trupill/ghc
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Ftrupill%2Fghc&data=04%7C01%7Csimonpj%40microsoft.com%7Caee2e0a922204ae3611208d96e2c2c57%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661959634465051%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=pFQzrPCiZXF0Y3KqmxjlhrDX1EVddJqII%2B%2BPTJb65h8%3D&reserved=0>
> .
>
>
>
> Any code should run *unchanged* except for some eta-expansion required for
> some specific usage patterns of higher-rank types. Please don't hesitate to
> ask any questions or clarifications about it. A merge request for tracking
> this can be found at
> https://gitlab.haskell.org/ghc/ghc/merge_requests/1659
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2Fmerge_requests%2F1659&data=04%7C01%7Csimonpj%40microsoft.com%7Caee2e0a922204ae3611208d96e2c2c57%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661959634475049%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=j0JcM0Q%2Bj7JSSb6lV%2F6Aj4lAZWjAysg%2FHg5cb4x00%2FY%3D&reserved=0>
>
>
>
> Kind regards,
>
> Alejandro
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> <https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=04%7C01%7Csimonpj%40microsoft.com%7Caee2e0a922204ae3611208d96e2c2c57%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661959634485038%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=9i55zUcM%2BX%2FN7ngvTARBOvfm962IJ9SYfxfa46f7wm8%3D&reserved=0>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210902/18bff05b/attachment-0001.html>

From simonpj at microsoft.com  Thu Sep  2 17:44:15 2021
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Thu, 2 Sep 2021 17:44:15 +0000
Subject: New implementation for `ImpredicativeTypes`
In-Reply-To: <CALm==BV8iGBN5bU57AMPYb-0pdSNc2qAcGP-uTbFe+3MO8yHfA@mail.gmail.com>
References: <CAHnFXOumYR+nKkBv6B6_zHVA2D=b9kZVyZ3vTTu87R07h90YuA@mail.gmail.com>
 <CALm==BVhBZ4Gwd+011zmy-qT266JSU+BHH=fjOac_P1bmANTyw@mail.gmail.com>
 <VI1PR83MB0319241FB05C0F793E35C7AEADBA0@VI1PR83MB0319.EURPRD83.prod.outlook.com>
 <CALm==BWMAJgs+t=1pF4H=-WH3wRo1A5ZPfyhOsY5d1y4XL9xCA@mail.gmail.com>
 <DB7PR83MB0313E44B53429012C3BE69A6ADB70@DB7PR83MB0313.EURPRD83.prod.outlook.com>
 <CALm==BWCtjvEZ296usfNzP7mj5Cne4t5vMAivSd0mYRrrdeF_Q@mail.gmail.com>
 <VI1PR83MB026923BF6143E9EF17840C5EADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <CALm==BV8iGBN5bU57AMPYb-0pdSNc2qAcGP-uTbFe+3MO8yHfA@mail.gmail.com>
Message-ID: <VI1PR83MB02698AD55C54B8C1B237F253ADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>

Of course not. The same was true for QuickLook, though, wasn't it?
No, not at all.   QuickLook required zero changes to GHC's intermediate language - it impacted only the type inference system.   Adding existentials will entail a substantial change to the intermediate language, affecting every optimisation pass.

Simon

From: Alex Rozenshteyn <rpglover64 at gmail.com>
Sent: 02 September 2021 18:13
To: Simon Peyton Jones <simonpj at microsoft.com>
Cc: GHC developers <ghc-devs at haskell.org>
Subject: Re: New implementation for `ImpredicativeTypes`

So it's not just a question of saying "just add that paper to GHC and voila job done".

Of course not. The same was true for QuickLook, though, wasn't it?

On Thu, Sep 2, 2021 at 12:42 PM Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
If I understand correctly, the recent ICFP paper "An Existential Crisis Resolved<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fdoi%2Fpdf%2F10.1145%2F3473569&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328916809%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=kioGsomPn8BuvBLQx6KEr5glMTPvw6m93DK%2BGA0lM08%3D&reserved=0>" finally enables this; is that right?
It describes one way to include existentials in GHC's intermediate language, which is a real contribution. But it is not a small change.  So it's not just a question of saying "just add that paper to GHC and voila job done".

Simon

From: Alex Rozenshteyn <rpglover64 at gmail.com<mailto:rpglover64 at gmail.com>>
Sent: 02 September 2021 17:10
To: Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
Cc: GHC developers <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: Re: New implementation for `ImpredicativeTypes`

If I understand correctly, the recent ICFP paper "An Existential Crisis Resolved<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fdoi%2Fpdf%2F10.1145%2F3473569&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328926803%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=DtcIgtgrp3zJygorYY32WBKhF40rYtAOhMOOmPWf6pc%3D&reserved=0>" finally enables this; is that right?

On Mon, Sep 9, 2019 at 12:00 PM Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
Suppose Haskell did have existentials;

Yes, I think that's an interesting thing to work on!  I'm not sure what the implications would be.  At very least we'd need to extend System FC (GHC's intermediate language) with existential types and the corresponding pack and unpack syntactic forms.

I don't know of any work studying that question specifically, but others may have pointers.

simon

From: Alex Rozenshteyn <rpglover64 at gmail.com<mailto:rpglover64 at gmail.com>>
Sent: 06 September 2019 15:21
To: Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
Cc: Alejandro Serrano Mena <trupill at gmail.com<mailto:trupill at gmail.com>>; GHC developers <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: Re: New implementation for `ImpredicativeTypes`

Hi Simon,

You're exactly right, of course. My example is confusing, so let me see if I can clarify.

What I want in the ideal is map show [1, 'a', "b"]. That is, minimal syntactic overhead to mapping a function over multiple values of distinct types that results in a homogeneous list. As the reddit thread points out, there are workarounds involving TH or wrapping each element in a constructor or using bespoke operators, but when it comes down to it, none of them actually allows me to say what I mean; the TH one is closest, but I reach for TH only in times of desperation.

I had thought that one of the things preventing this was lack of impredicative instantiation, but now I'm not sure. Suppose Haskell did have existentials; would map show @(exists a. Show a => a) [1, 'a', "b"] work in current Haskell and/or in quick-look?

Tangentially, do you have a reference for what difficulties arise in adding existentials to Haskell? I have a feeling that it would make working with GADTs more ergonomic.

On Fri, Sep 6, 2019 at 12:33 AM Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
I'm confused.   Char does not have the type (forall a. Show a =>a), so our example is iill-typed in System F, never mind about type inference.  Perhaps there's a typo?   I think you may have ment
               exists a. Show a => a
which doesn't exist in Haskell.  You can write existentials with a data type

data Showable where
   S :: forall a. Show a => a -> Showable

Then
               map show [S 1, S 'a', S "b"]
works fine today (without our new stuff), provided you say

               instance Show Showable where
                 show (S x) = show x

Our new system can only type programs that can be written in System F.   (The tricky bit is inferring the impredicative instantiations.)

Simon

From: ghc-devs <ghc-devs-bounces at haskell.org<mailto:ghc-devs-bounces at haskell.org>> On Behalf Of Alex Rozenshteyn
Sent: 06 September 2019 03:31
To: Alejandro Serrano Mena <trupill at gmail.com<mailto:trupill at gmail.com>>
Cc: GHC developers <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: Re: New implementation for `ImpredicativeTypes`

I didn't say anything when you were requesting use cases, so I have no right to complain, but I'm still a little disappointed that this doesn't fix my (admittedly very minor) issue: https://www.reddit.com/r/haskell/comments/3am0qa/existentials_and_the_heterogenous_list_fallacy/csdwlp2/?context=8&depth=9<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.reddit.com%2Fr%2Fhaskell%2Fcomments%2F3am0qa%2Fexistentials_and_the_heterogenous_list_fallacy%2Fcsdwlp2%2F%3Fcontext%3D8%26depth%3D9&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328936796%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=IFOf9nl11qhDksPnK9jf2HxcHm7QMhVUGJZX%2F%2FOVim4%3D&reserved=0>

For those who don't want to click on the reddit link: I would like to be able to write something like map show ([1, 'a', "b"] :: [forall a. Show a => a]), and have it work.

On Wed, Sep 4, 2019 at 8:13 AM Alejandro Serrano Mena <trupill at gmail.com<mailto:trupill at gmail.com>> wrote:
Hi all,
As I mentioned some time ago, we have been busy working on a new implementation of `ImpredicativeTypes` for GHC. I am very thankful to everybody who back then sent us examples of impredicativity which would be nice to support, as far as we know this branch supports all of them! :)

If you want to try it, at https://gitlab.haskell.org/trupill/ghc/commit/a3f95a0fe0f647702fd7225fa719a8062a4cc0a5/pipelines?ref=quick-look-build<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Ftrupill%2Fghc%2Fcommit%2Fa3f95a0fe0f647702fd7225fa719a8062a4cc0a5%2Fpipelines%3Fref%3Dquick-look-build&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328936796%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=eR1k8ntH2aGT5nTFOQxd86hGr5OxDSjmkxFpy09fGlg%3D&reserved=0> you can find the result of the pipeline, which includes builds for several platforms (click on the "Artifacts" button, the one which looks like a cloud, to get them). The code is being developed at https://gitlab.haskell.org/trupill/ghc<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Ftrupill%2Fghc&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328946794%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RdfPCIqvBF8VyeuGKmtOBJsnwcPl%2Bya%2B4KLi%2BWfrT3E%3D&reserved=0>.

Any code should run *unchanged* except for some eta-expansion required for some specific usage patterns of higher-rank types. Please don't hesitate to ask any questions or clarifications about it. A merge request for tracking this can be found at https://gitlab.haskell.org/ghc/ghc/merge_requests/1659<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2Fmerge_requests%2F1659&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328956787%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=aR6Lq4teuUmPe4o%2FOHp878Zm8OBfXVxoMrn0WqcN4fM%3D&reserved=0>

Kind regards,
Alejandro
_______________________________________________
ghc-devs mailing list
ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328956787%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=IUsQu1gDuNetafA6ljvQwNBEYq%2F%2B2NFYqwT5Ytpal24%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210902/4d266bab/attachment.html>

From sgraf1337 at gmail.com  Thu Sep  2 18:09:38 2021
From: sgraf1337 at gmail.com (Sebastian Graf)
Date: Thu, 2 Sep 2021 18:09:38 +0000
Subject: Potential improvement in compacting GC
In-Reply-To: <CAMQQO3m4hCVerYnbyoakj6QFdL1ZXzjzu54JK857+e0FByMg2Q@mail.gmail.com>
References: <CAMQQO3=7haokj1uUUt9KoSQ2YLOd7Y3Ue20atXhW2HwL=Z+2nQ@mail.gmail.com>
 <CAMQQO3nHFzL5aVFSq-RkSVcoS6g_WhjHJrsDF_MboowMwXCnAA@mail.gmail.com>
 <CAMQQO3m4hCVerYnbyoakj6QFdL1ZXzjzu54JK857+e0FByMg2Q@mail.gmail.com>
Message-ID: <PA4PR03MB75183E995784C19B0E0AF254ACCE9@PA4PR03MB7518.eurprd03.prod.outlook.com>

Hey Ömer,

Just in case you are wondering whether you are talking into the void: you aren't! Keep the good suggestions coming, someone will eventually be able to get around to implementing them!
Thanks for your write-ups.

Cheers,
Sebastian
________________________________
Von: ghc-devs <ghc-devs-bounces at haskell.org> im Auftrag von Ömer Sinan Ağacan <omeragacan at gmail.com>
Gesendet: Thursday, September 2, 2021 5:47:08 PM
An: ghc-devs <ghc-devs at haskell.org>
Betreff: Re: Potential improvement in compacting GC

Here's another improvement that fixes a very old issue in GHC's compacting GC
[1].

To summarize the problem: when untreading an object we update references to the
object that we've seen so far to the object's new location. But to get the
object's new location we need to know the object's size, because depending on
the size we may need to move the object to a new block (when the current block
does not have enough space for the object).

For this we currently walk the thread twice, once to get the info table (used
to get the size), then again to update references to the object. Ideally we
want to do just one pass when unthreading.

The solution is explained in The GC Handbook, section 3.4. Instead of using one
bit to mark an object, we use two bits: one for the first word of the object,
one for the last word. Using two bits is not a problem in GHC because heap
objects are at least 2 words. For example, an object with two words is marked
with `11`, 3 words is marked with `101` and so on.

Now we can scan the bitmap to find object size, and unthread it without having
to find the info table first.

Ömer

[1]: https://github.com/ghc/ghc/blob/922c6bc8dd8d089cfe4b90ec2120cb48959ba2b5/rts/sm/Compact.c#L844-L848

Ömer Sinan Ağacan <omeragacan at gmail.com>, 14 Tem 2021 Çar, 09:27
tarihinde şunu yazdı:
>
> Two other ideas that should improve GHC's compacting GC much more
> significantly. I've implemented both of these in another project and the
> results were great. In a GC benchmark (mutator does almost no work other than
> allocating data using a bump allocator), first one reduced Wasm instructions
> executed by 14%, second one 19.8%.
>
> Both of these ideas require pushing object headers to the mark stack with the
> objects, which means larger mark stacks. This is the only downside of these
> algorithms.
>
> - Instead of marking and then threading in the next pass, mark phase threads
>   all fields when pushing the fields to the mark stack. We still need two other
>   passes: one to unthread headers, another to move the objects. (we can't do
>   both in one pass, let me know if you're curious and I can elaborate)
>
>   This has the same number of passes as the current implementation, but it only
>   visits object fields once. Currently, we visit fields once when marking, to
>   mark fields, then again in `update_fwd`. This implementation does both in one
>   pass over the fields. `update_fwd` does not visit fields.
>
>   This reduced Wasm instructions executed by 14% in my benchmark.
>
> - Marking phase threads backwards pointers (ignores forwards pointers). Then we
>   do one pass instead of two: for a marked object, unthread it (update
>   forwards pointers to the object's new location), move it to its new location,
>   then thread its forwards pointers. This completely eliminates one of the 3
>   passes, but fields need to be visited twice as before (unlike the algorithm
>   above).
>
>   I think this one is originally described in "An Efficient Garbage Compaction
>   Algorithm", but I found that paper to be difficult to follow. A short
>   description of the same algorithm exists in "High-Performance Garbage
>   Collection for Memory-Constrained Environments" in section 5.1.2.
>
>   This reduced Wasm instructions executed by 19.8% in my benchmark.
>
>   In this algorithm, fields that won't be moved can be threaded any time before
>   the second pass (pointed objects need to be marked and pushed to the mark
>   stack with headers before threading a field). For example, in GHC, mut list
>   entries can be threaded before or after marking (but before the second pass)
>   as IIRC mut lists are not moved. Same for fields of large objects.
>
> As far as I can see, mark-compact GC is still the default when max heap size is
> specified and the oldest generation size is (by default) more than 30% of the
> max heap size. I'm not sure if max heap size is specified often (it's off by
> default), so not sure what would be the impact of these improvements be, but if
> anyone would be interested in funding me to implement these ideas (second
> algorithm above, and the bitmap iteration in the previous email) I could try to
> allocate one or two days a week to finish in a few months.
>
> Normally these are simple changes, but it's difficult to test and debug GHC's
> RTS as we don't have a test suite readily available and the code is not easily
> testable. In my previous implementations of these algorithms I had unit tests
> for the GC where I could easily generate arbitrary graphs (with cycles,
> backwards ptrs, forwards ptrs, ptrs from/to roots etc.) and test GC in
> isolation. Implementation of (2) took less than a day, and I didn't have to
> debug it more once the tests passed. It's really unfortunate that GHC's RTS
> makes this kind of thing difficult..
>
> Ömer
>
> Ömer Sinan Ağacan <omeragacan at gmail.com>, 7 Oca 2021 Per, 20:42
> tarihinde şunu yazdı:
> >
> > Hello,
> >
> > I recently implemented the algorithm used by GHC's compacting GC in another
> > project. The algorithm (after marking) makes two passes over the heap
> > /generation. In GHC, these passes are implemented in [1] and in the next
> > function.
> >
> > In my implementation I tried 3 ways of implementing these passes, one of which
> > is the same as GHC's code, and benchmarked each version. I found that the
> > fastest implementation is not what's used in GHC, but it could easily be used.
> >
> > I should mention that my code targets Wasm, and I benchmarked Wasm instructions
> > executed. Not CPU cycles, CPU instructions, or anything like that. It's very
> > likely that the results will be different when benchmarking code running on
> > actual hardware.
> >
> > Secondly, in my case the heap is mostly dead (residency is low). In GHC,
> > compaction for the oldest generation is enabled when residency passes a
> > threshold, so the assumption is the heap is mostly live. I'm guessing this
> > should also make some difference.
> >
> > Anyway, the first implementation I tried was similar to GHC's scan, but I did
> >
> >     scan += object_size(scan);
> >
> > instead of bumping scan by one, as GHC does in [2]. This was the slowest
> > version.
> >
> > Second implementation did the same as GHC (bumped scan by one). This was
> > faster, but still slower than the next version.
> >
> > What I found to be the best is scanning the bitmap, not the heap. The bitmap
> > iterator reads one word at a time. In each iteration it checks if the bitmap
> > word is 0. In GHC, in the best case this can skip 16 words on heap on 32-bit
> > systems, and 32 words on 64-bit. Reminder: we need two bits per object in the
> > bitmap, see [3]. (this is not the case in my implementation so the payoff is
> > better)
> >
> > When the bitmap word is not 0 I use "count trailing zeros" (or "count leading
> > zeros" depending on the bitmap implementation) to get the number of words to
> > skip. This is a single instruction on Wasm and x86 (TZCNT or LZCNT, available
> > via __builtin_ctzl and __builtin_clzl in gcc).
> >
> > So instead of skipping one word at a time, this can potentially skip 16 words
> > (or 32 on 64-bit architectures). When that's not possible, it can still skip
> > multiple words by using ctz/clz.
> >
> > Ömer
> >
> > [1]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rts/sm/Compact.c#L824-L879
> > [2]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rts/sm/Compact.c#L838
> > [3]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rts/sm/Compact.h#L18-L55
_______________________________________________
ghc-devs mailing list
ghc-devs at haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210902/2d6fc6c9/attachment.html>

From rpglover64 at gmail.com  Thu Sep  2 18:10:34 2021
From: rpglover64 at gmail.com (Alex Rozenshteyn)
Date: Thu, 2 Sep 2021 14:10:34 -0400
Subject: New implementation for `ImpredicativeTypes`
In-Reply-To: <VI1PR83MB02698AD55C54B8C1B237F253ADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
References: <CAHnFXOumYR+nKkBv6B6_zHVA2D=b9kZVyZ3vTTu87R07h90YuA@mail.gmail.com>
 <CALm==BVhBZ4Gwd+011zmy-qT266JSU+BHH=fjOac_P1bmANTyw@mail.gmail.com>
 <VI1PR83MB0319241FB05C0F793E35C7AEADBA0@VI1PR83MB0319.EURPRD83.prod.outlook.com>
 <CALm==BWMAJgs+t=1pF4H=-WH3wRo1A5ZPfyhOsY5d1y4XL9xCA@mail.gmail.com>
 <DB7PR83MB0313E44B53429012C3BE69A6ADB70@DB7PR83MB0313.EURPRD83.prod.outlook.com>
 <CALm==BWCtjvEZ296usfNzP7mj5Cne4t5vMAivSd0mYRrrdeF_Q@mail.gmail.com>
 <VI1PR83MB026923BF6143E9EF17840C5EADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <CALm==BV8iGBN5bU57AMPYb-0pdSNc2qAcGP-uTbFe+3MO8yHfA@mail.gmail.com>
 <VI1PR83MB02698AD55C54B8C1B237F253ADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
Message-ID: <CALm==BVCTZKUp_-JYqZo01aC3C85Kk+RBkJmDDh8fa_ydBYLLg@mail.gmail.com>

Oh, I see. That's because this would need to introduce `pack ... as ...`
and `open ...` into the core term language, right?

My sense is that it shouldn't negatively affect runtime performance of
programs without existentials even if implemented naively; does that seem
accurate? Not that implementing it, even naively, is a small task.

On Thu, Sep 2, 2021 at 1:44 PM Simon Peyton Jones <simonpj at microsoft.com>
wrote:

> Of course not. The same was true for QuickLook, though, wasn't it?
>
> No, not at all.   QuickLook required zero changes to GHC’s intermediate
> language – it impacted only the type inference system.   Adding
> existentials will entail a substantial change to the intermediate language,
> affecting every optimisation pass.
>
>
>
> Simon
>
>
>
> *From:* Alex Rozenshteyn <rpglover64 at gmail.com>
> *Sent:* 02 September 2021 18:13
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Cc:* GHC developers <ghc-devs at haskell.org>
> *Subject:* Re: New implementation for `ImpredicativeTypes`
>
>
>
> So it’s not just a question of saying “just add that paper to GHC and
> voila job done”.
>
>
>
> Of course not. The same was true for QuickLook, though, wasn't it?
>
>
>
> On Thu, Sep 2, 2021 at 12:42 PM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> If I understand correctly, the recent ICFP paper "An Existential Crisis
> Resolved
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fdoi%2Fpdf%2F10.1145%2F3473569&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328916809%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=kioGsomPn8BuvBLQx6KEr5glMTPvw6m93DK%2BGA0lM08%3D&reserved=0>"
> finally enables this; is that right?
>
> It describes one way to include existentials in GHC’s intermediate
> language, which is a real contribution. *But it is not a small change*.
> So it’s not just a question of saying “just add that paper to GHC and voila
> job done”.
>
>
>
> Simon
>
>
>
> *From:* Alex Rozenshteyn <rpglover64 at gmail.com>
> *Sent:* 02 September 2021 17:10
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Cc:* GHC developers <ghc-devs at haskell.org>
> *Subject:* Re: New implementation for `ImpredicativeTypes`
>
>
>
> If I understand correctly, the recent ICFP paper "An Existential Crisis
> Resolved
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fdoi%2Fpdf%2F10.1145%2F3473569&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328926803%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=DtcIgtgrp3zJygorYY32WBKhF40rYtAOhMOOmPWf6pc%3D&reserved=0>"
> finally enables this; is that right?
>
>
>
> On Mon, Sep 9, 2019 at 12:00 PM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> Suppose Haskell *did* have existentials;
>
>
>
> Yes, I think that’s an interesting thing to work on!  I’m not sure what
> the implications would be.  At very least we’d need to extend System FC
> (GHC’s intermediate language) with existential types and the corresponding
> pack and unpack syntactic forms.
>
>
>
> I don’t know of any work studying that question specifically, but others
> may have pointers.
>
>
>
> simon
>
>
>
> *From:* Alex Rozenshteyn <rpglover64 at gmail.com>
> *Sent:* 06 September 2019 15:21
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Cc:* Alejandro Serrano Mena <trupill at gmail.com>; GHC developers <
> ghc-devs at haskell.org>
> *Subject:* Re: New implementation for `ImpredicativeTypes`
>
>
>
> Hi Simon,
>
>
>
> You're exactly right, of course. My example is confusing, so let me see if
> I can clarify.
>
>
>
> What I want in the ideal is map show [1, 'a', "b"]. That is, minimal
> syntactic overhead to mapping a function over multiple values of distinct
> types that results in a homogeneous list. As the reddit thread points out,
> there are workarounds involving TH or wrapping each element in a
> constructor or using bespoke operators, but when it comes down to it, none
> of them actually allows me to say what I *mean*; the TH one is closest,
> but I reach for TH only in times of desperation.
>
>
>
> I had thought that one of the things preventing this was lack of
> impredicative instantiation, but now I'm not sure. Suppose Haskell *did*
> have existentials; would map show @(exists a. Show a => a) [1, 'a', "b"]
> work in current Haskell and/or in quick-look?
>
>
>
> Tangentially, do you have a reference for what difficulties arise in
> adding existentials to Haskell? I have a feeling that it would make working
> with GADTs more ergonomic.
>
>
>
> On Fri, Sep 6, 2019 at 12:33 AM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> I’m confused.   Char does not have the type (forall a. Show a =>a), so our
> example is iill-typed in System F, never mind about type inference.
> Perhaps there’s a typo?   I think you may have ment
>
>                exists a. Show a => a
>
> which doesn’t exist in Haskell.  You can write existentials with a data
> type
>
>
>
> data Showable where
>
>    S :: forall a. Show a => a -> Showable
>
>
>
> Then
>
>                map show [S 1, S ‘a’, S “b”]
>
> works fine today (without our new stuff), provided you say
>
>
>
>                instance Show Showable where
>
>                  show (S x) = show x
>
>
>
> Our new system can only type programs that can be written in System F.
> (The tricky bit is inferring the impredicative instantiations.)
>
>
>
> Simon
>
>
>
> *From:* ghc-devs <ghc-devs-bounces at haskell.org> *On Behalf Of *Alex
> Rozenshteyn
> *Sent:* 06 September 2019 03:31
> *To:* Alejandro Serrano Mena <trupill at gmail.com>
> *Cc:* GHC developers <ghc-devs at haskell.org>
> *Subject:* Re: New implementation for `ImpredicativeTypes`
>
>
>
> I didn't say anything when you were requesting use cases, so I have no
> right to complain, but I'm still a little disappointed that this doesn't
> fix my (admittedly very minor) issue:
> https://www.reddit.com/r/haskell/comments/3am0qa/existentials_and_the_heterogenous_list_fallacy/csdwlp2/?context=8&depth=9
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.reddit.com%2Fr%2Fhaskell%2Fcomments%2F3am0qa%2Fexistentials_and_the_heterogenous_list_fallacy%2Fcsdwlp2%2F%3Fcontext%3D8%26depth%3D9&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328936796%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=IFOf9nl11qhDksPnK9jf2HxcHm7QMhVUGJZX%2F%2FOVim4%3D&reserved=0>
>
>
>
> For those who don't want to click on the reddit link: I would like to be
> able to write something like map show ([1, 'a', "b"] :: [forall a. Show a
> => a]), and have it work.
>
>
>
> On Wed, Sep 4, 2019 at 8:13 AM Alejandro Serrano Mena <trupill at gmail.com>
> wrote:
>
> Hi all,
>
> As I mentioned some time ago, we have been busy working on a new
> implementation of `ImpredicativeTypes` for GHC. I am very thankful to
> everybody who back then sent us examples of impredicativity which would be
> nice to support, as far as we know this branch supports all of them! :)
>
>
>
> If you want to try it, at
> https://gitlab.haskell.org/trupill/ghc/commit/a3f95a0fe0f647702fd7225fa719a8062a4cc0a5/pipelines?ref=quick-look-build
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Ftrupill%2Fghc%2Fcommit%2Fa3f95a0fe0f647702fd7225fa719a8062a4cc0a5%2Fpipelines%3Fref%3Dquick-look-build&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328936796%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=eR1k8ntH2aGT5nTFOQxd86hGr5OxDSjmkxFpy09fGlg%3D&reserved=0>
> you can find the result of the pipeline, which includes builds for several
> platforms (click on the "Artifacts" button, the one which looks like a
> cloud, to get them). The code is being developed at
> https://gitlab.haskell.org/trupill/ghc
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Ftrupill%2Fghc&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328946794%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RdfPCIqvBF8VyeuGKmtOBJsnwcPl%2Bya%2B4KLi%2BWfrT3E%3D&reserved=0>
> .
>
>
>
> Any code should run *unchanged* except for some eta-expansion required for
> some specific usage patterns of higher-rank types. Please don't hesitate to
> ask any questions or clarifications about it. A merge request for tracking
> this can be found at
> https://gitlab.haskell.org/ghc/ghc/merge_requests/1659
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2Fmerge_requests%2F1659&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328956787%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=aR6Lq4teuUmPe4o%2FOHp878Zm8OBfXVxoMrn0WqcN4fM%3D&reserved=0>
>
>
>
> Kind regards,
>
> Alejandro
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> <https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328956787%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=IUsQu1gDuNetafA6ljvQwNBEYq%2F%2B2NFYqwT5Ytpal24%3D&reserved=0>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210902/5b4b8473/attachment-0001.html>

From john.ericson at obsidian.systems  Thu Sep  2 18:21:20 2021
From: john.ericson at obsidian.systems (john.ericson)
Date: Thu, 02 Sep 2021 14:21:20 -0400
Subject: New implementation for `ImpredicativeTypes`
In-Reply-To: <CALm==BVCTZKUp_-JYqZo01aC3C85Kk+RBkJmDDh8fa_ydBYLLg@mail.gmail.com>
References: <CAHnFXOumYR+nKkBv6B6_zHVA2D=b9kZVyZ3vTTu87R07h90YuA@mail.gmail.com>
 <CALm==BVhBZ4Gwd+011zmy-qT266JSU+BHH=fjOac_P1bmANTyw@mail.gmail.com>
 <VI1PR83MB0319241FB05C0F793E35C7AEADBA0@VI1PR83MB0319.EURPRD83.prod.outlook.com>
 <CALm==BWMAJgs+t=1pF4H=-WH3wRo1A5ZPfyhOsY5d1y4XL9xCA@mail.gmail.com>
 <DB7PR83MB0313E44B53429012C3BE69A6ADB70@DB7PR83MB0313.EURPRD83.prod.outlook.com>
 <CALm==BWCtjvEZ296usfNzP7mj5Cne4t5vMAivSd0mYRrrdeF_Q@mail.gmail.com>
 <VI1PR83MB026923BF6143E9EF17840C5EADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <CALm==BV8iGBN5bU57AMPYb-0pdSNc2qAcGP-uTbFe+3MO8yHfA@mail.gmail.com>
 <VI1PR83MB02698AD55C54B8C1B237F253ADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <CALm==BVCTZKUp_-JYqZo01aC3C85Kk+RBkJmDDh8fa_ydBYLLg@mail.gmail.com>
Message-ID: <17ba7be6d79.c4bd9caa88419.6741373830234513753@obsidian.systems>

This reminds me...can we do newtype GADTs in certain situations as a stepping stone? I would think that would be purely easier — more nominal, no nice projections but only `case` and skolems which cannot escape.Newtype GADTs we're long deemed impossible IIRC, but surely the paper demonstrates that at least some cases should work? ---- On Thu, 02 Sep 2021 14:10:34 -0400  rpglover64 at gmail.com  wrote ----Oh, I see. That's because this would need to introduce `pack ... as ...` and `open ...` into the core term language, right?My sense is that it shouldn't negatively affect runtime performance of programs without existentials even if implemented naively; does that seem accurate? Not that implementing it, even naively, is a small task. On Thu, Sep 2, 2021 at 1:44 PM Simon Peyton Jones <simonpj at microsoft.com> wrote:








Of course not. The same was true for QuickLook, though, wasn't it?
No, not at all.   QuickLook required zero changes to GHC’s intermediate language – it impacted only the type inference system.   Adding existentials will entail a substantial change to the intermediate
 language, affecting every optimisation pass.
 
Simon
 



From: Alex Rozenshteyn <rpglover64 at gmail.com>

Sent: 02 September 2021 18:13
To: Simon Peyton Jones <simonpj at microsoft.com>
Cc: GHC developers <ghc-devs at haskell.org>
Subject: Re: New implementation for `ImpredicativeTypes`


 



So it’s not just a question of saying “just add that paper to GHC and voila job done”. 



 



Of course not. The same was true for QuickLook, though, wasn't it?



 



On Thu, Sep 2, 2021 at 12:42 PM Simon Peyton Jones <simonpj at microsoft.com> wrote:





If I understand correctly, the recent ICFP paper "An
 Existential Crisis Resolved" finally enables this; is that right?
It describes one way to include existentials in GHC’s intermediate language, which is a real contribution.
But it is not a small change.  So it’s not just a question of saying “just add that paper to GHC and voila job done”.
 
Simon
 



From: Alex Rozenshteyn <rpglover64 at gmail.com>

Sent: 02 September 2021 17:10
To: Simon Peyton Jones <simonpj at microsoft.com>
Cc: GHC developers <ghc-devs at haskell.org>
Subject: Re: New implementation for `ImpredicativeTypes`


 

If I understand correctly, the recent ICFP paper "An
 Existential Crisis Resolved" finally enables this; is that right?

 


On Mon, Sep 9, 2019 at 12:00 PM Simon Peyton Jones <simonpj at microsoft.com> wrote:





Suppose Haskell did have existentials;
 
Yes, I think that’s an interesting thing to work on!  I’m not sure what the implications would be.  At very least we’d need to extend System FC (GHC’s intermediate language) with
 existential types and the corresponding pack and unpack syntactic forms.
 
I don’t know of any work studying that question specifically, but others may have pointers.
 
simon
 



From: Alex Rozenshteyn <rpglover64 at gmail.com>

Sent: 06 September 2019 15:21
To: Simon Peyton Jones <simonpj at microsoft.com>
Cc: Alejandro Serrano Mena <trupill at gmail.com>; GHC developers <ghc-devs at haskell.org>
Subject: Re: New implementation for `ImpredicativeTypes`


 


Hi Simon,


 


You're exactly right, of course. My example is confusing, so let me see if I can clarify.


 


What I want in the ideal is
map show [1, 'a', "b"]. That is, minimal syntactic overhead to mapping a function over multiple values of distinct types that results in a homogeneous list. As the reddit thread points out, there are workarounds
 involving TH or wrapping each element in a constructor or using bespoke operators, but when it comes down to it, none of them actually allows me to say what I
mean; the TH one is closest, but I reach for TH only in times of desperation.


 


I had thought that one of the things preventing this was lack of impredicative instantiation, but now I'm not sure. Suppose Haskell
did have existentials; would map show @(exists a. Show a => a) [1, 'a', "b"] work in current Haskell and/or in quick-look?


 


Tangentially, do you have a reference for what difficulties arise in adding existentials to Haskell? I have a feeling that it would make working with GADTs more ergonomic.


 


On Fri, Sep 6, 2019 at 12:33 AM Simon Peyton Jones <simonpj at microsoft.com> wrote:




I’m confused.   Char does not have the type (forall a. Show a =>a), so our example is iill-typed in System F, never mind about type inference.  Perhaps there’s a typo?   I think
 you may have ment
               exists a. Show a => a
which doesn’t exist in Haskell.  You can write existentials with a data type
 
data Showable where
   S :: forall a. Show a => a -> Showable
 
Then
               map show [S 1, S ‘a’, S “b”]
works fine today (without our new stuff), provided you say
 
               instance Show Showable where
                 show (S x) = show x
 
Our new system can only type programs that can be written in System F.   (The tricky bit is inferring the impredicative instantiations.)
 
Simon
 



From: ghc-devs <ghc-devs-bounces at haskell.org>
On Behalf Of Alex Rozenshteyn
Sent: 06 September 2019 03:31
To: Alejandro Serrano Mena <trupill at gmail.com>
Cc: GHC developers <ghc-devs at haskell.org>
Subject: Re: New implementation for `ImpredicativeTypes`


 


I didn't say anything when you were requesting use cases, so I have no right to complain, but I'm still a little disappointed that this doesn't fix my (admittedly very minor) issue:

https://www.reddit.com/r/haskell/comments/3am0qa/existentials_and_the_heterogenous_list_fallacy/csdwlp2/?context=8&depth=9


 


For those who don't want to click on the reddit link: I would like to be able to write something like map show ([1, 'a', "b"] :: [forall a. Show a
 => a]), and have it work.


 


On Wed, Sep 4, 2019 at 8:13 AM Alejandro Serrano Mena <trupill at gmail.com> wrote:




Hi all,


As I mentioned some time ago, we have been busy working on a new implementation of `ImpredicativeTypes` for GHC. I am very thankful to everybody who back then sent us examples of impredicativity
 which would be nice to support, as far as we know this branch supports all of them! :)


 


If you want to try it, at

https://gitlab.haskell.org/trupill/ghc/commit/a3f95a0fe0f647702fd7225fa719a8062a4cc0a5/pipelines?ref=quick-look-build you can find the result of the pipeline, which includes builds for several platforms (click on the "Artifacts" button, the one which looks
 like a cloud, to get them). The code is being developed at 
https://gitlab.haskell.org/trupill/ghc.


 


Any code should run *unchanged* except for some eta-expansion required for some specific usage patterns of higher-rank types. Please don't hesitate to ask any questions or clarifications
 about it. A merge request for tracking this can be found at 
https://gitlab.haskell.org/ghc/ghc/merge_requests/1659


 


Kind regards,


Alejandro


_______________________________________________
ghc-devs mailing list
ghc-devs at haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs






















_______________________________________________ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210902/eafa31bb/attachment.html>

From lists at richarde.dev  Thu Sep  2 18:33:40 2021
From: lists at richarde.dev (Richard Eisenberg)
Date: Thu, 2 Sep 2021 18:33:40 +0000
Subject: New implementation for `ImpredicativeTypes`
In-Reply-To: <CALm==BVCTZKUp_-JYqZo01aC3C85Kk+RBkJmDDh8fa_ydBYLLg@mail.gmail.com>
References: <CAHnFXOumYR+nKkBv6B6_zHVA2D=b9kZVyZ3vTTu87R07h90YuA@mail.gmail.com>
 <CALm==BVhBZ4Gwd+011zmy-qT266JSU+BHH=fjOac_P1bmANTyw@mail.gmail.com>
 <VI1PR83MB0319241FB05C0F793E35C7AEADBA0@VI1PR83MB0319.EURPRD83.prod.outlook.com>
 <CALm==BWMAJgs+t=1pF4H=-WH3wRo1A5ZPfyhOsY5d1y4XL9xCA@mail.gmail.com>
 <DB7PR83MB0313E44B53429012C3BE69A6ADB70@DB7PR83MB0313.EURPRD83.prod.outlook.com>
 <CALm==BWCtjvEZ296usfNzP7mj5Cne4t5vMAivSd0mYRrrdeF_Q@mail.gmail.com>
 <VI1PR83MB026923BF6143E9EF17840C5EADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <CALm==BV8iGBN5bU57AMPYb-0pdSNc2qAcGP-uTbFe+3MO8yHfA@mail.gmail.com>
 <VI1PR83MB02698AD55C54B8C1B237F253ADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <CALm==BVCTZKUp_-JYqZo01aC3C85Kk+RBkJmDDh8fa_ydBYLLg@mail.gmail.com>
Message-ID: <010f017ba7c9b8ff-b15dfb92-5bc3-4cb9-9fa6-9e4e518788ee-000000@us-east-2.amazonses.com>



> On Sep 2, 2021, at 2:10 PM, Alex Rozenshteyn <rpglover64 at gmail.com> wrote:
> 
> Oh, I see. That's because this would need to introduce `pack ... as ...` and `open ...` into the core term language, right?

Exactly, yes.

> 
> My sense is that it shouldn't negatively affect runtime performance of programs without existentials even if implemented naively; does that seem accurate? Not that implementing it, even naively, is a small task. 

I would agree with this, too.

> On Sep 2, 2021, at 2:21 PM, john.ericson <john.ericson at obsidian.systems> wrote:
> 
> This reminds me...can we do newtype GADTs in certain situations as a stepping stone? I would think that would be purely easier — more nominal, no nice projections but only `case` and skolems which cannot escape.
> 
> Newtype GADTs we're long deemed impossible IIRC, but surely the paper demonstrates that at least some cases should work?

I don't quite see how this relates to existentials. Note that the paper does not address e.g. packing equalities in existentials, which would be needed for interacting with GADTs.

Glad folks are enjoying the paper! :)

Richard

> 
> 
> ---- On Thu, 02 Sep 2021 14:10:34 -0400 rpglover64 at gmail.com wrote ----
> 
> Oh, I see. That's because this would need to introduce `pack ... as ...` and `open ...` into the core term language, right?
> 
> My sense is that it shouldn't negatively affect runtime performance of programs without existentials even if implemented naively; does that seem accurate? Not that implementing it, even naively, is a small task. 
> 
> On Thu, Sep 2, 2021 at 1:44 PM Simon Peyton Jones <simonpj at microsoft.com <mailto:simonpj at microsoft.com>> wrote:
> Of course not. The same was true for QuickLook, though, wasn't it?
> 
> No, not at all.   QuickLook required zero changes to GHC’s intermediate language – it impacted only the type inference system.   Adding existentials will entail a substantial change to the intermediate language, affecting every optimisation pass.
> 
>  
> 
> Simon
> 
>  
> 
> From: Alex Rozenshteyn <rpglover64 at gmail.com <mailto:rpglover64 at gmail.com>> 
> Sent: 02 September 2021 18:13
> To: Simon Peyton Jones <simonpj at microsoft.com <mailto:simonpj at microsoft.com>>
> Cc: GHC developers <ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>>
> Subject: Re: New implementation for `ImpredicativeTypes`
> 
>  
> 
> So it’s not just a question of saying “just add that paper to GHC and voila job done”. 
> 
>  
> 
> Of course not. The same was true for QuickLook, though, wasn't it?
> 
>  
> 
> On Thu, Sep 2, 2021 at 12:42 PM Simon Peyton Jones <simonpj at microsoft.com <mailto:simonpj at microsoft.com>> wrote:
> 
> If I understand correctly, the recent ICFP paper "An Existential Crisis Resolved <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fdoi%2Fpdf%2F10.1145%2F3473569&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328916809%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=kioGsomPn8BuvBLQx6KEr5glMTPvw6m93DK%2BGA0lM08%3D&reserved=0>" finally enables this; is that right?
> 
> It describes one way to include existentials in GHC’s intermediate language, which is a real contribution. But it is not a small change.  So it’s not just a question of saying “just add that paper to GHC and voila job done”.
> 
>  
> 
> Simon
> 
>  
> 
> From: Alex Rozenshteyn <rpglover64 at gmail.com <mailto:rpglover64 at gmail.com>> 
> Sent: 02 September 2021 17:10
> To: Simon Peyton Jones <simonpj at microsoft.com <mailto:simonpj at microsoft.com>>
> Cc: GHC developers <ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>>
> Subject: Re: New implementation for `ImpredicativeTypes`
> 
>  
> 
> If I understand correctly, the recent ICFP paper "An Existential Crisis Resolved <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fdoi%2Fpdf%2F10.1145%2F3473569&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328926803%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=DtcIgtgrp3zJygorYY32WBKhF40rYtAOhMOOmPWf6pc%3D&reserved=0>" finally enables this; is that right?
> 
>  
> 
> On Mon, Sep 9, 2019 at 12:00 PM Simon Peyton Jones <simonpj at microsoft.com <mailto:simonpj at microsoft.com>> wrote:
> 
> Suppose Haskell did have existentials;
> 
>  
> 
> Yes, I think that’s an interesting thing to work on!  I’m not sure what the implications would be.  At very least we’d need to extend System FC (GHC’s intermediate language) with existential types and the corresponding pack and unpack syntactic forms.
> 
>  
> 
> I don’t know of any work studying that question specifically, but others may have pointers.
> 
>  
> 
> simon
> 
>  
> 
> From: Alex Rozenshteyn <rpglover64 at gmail.com <mailto:rpglover64 at gmail.com>> 
> Sent: 06 September 2019 15:21
> To: Simon Peyton Jones <simonpj at microsoft.com <mailto:simonpj at microsoft.com>>
> Cc: Alejandro Serrano Mena <trupill at gmail.com <mailto:trupill at gmail.com>>; GHC developers <ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>>
> Subject: Re: New implementation for `ImpredicativeTypes`
> 
>  
> 
> Hi Simon,
> 
>  
> 
> You're exactly right, of course. My example is confusing, so let me see if I can clarify.
> 
>  
> 
> What I want in the ideal is map show [1, 'a', "b"]. That is, minimal syntactic overhead to mapping a function over multiple values of distinct types that results in a homogeneous list. As the reddit thread points out, there are workarounds involving TH or wrapping each element in a constructor or using bespoke operators, but when it comes down to it, none of them actually allows me to say what I mean; the TH one is closest, but I reach for TH only in times of desperation.
> 
>  
> 
> I had thought that one of the things preventing this was lack of impredicative instantiation, but now I'm not sure. Suppose Haskell did have existentials; would map show @(exists a. Show a => a) [1, 'a', "b"] work in current Haskell and/or in quick-look?
> 
>  
> 
> Tangentially, do you have a reference for what difficulties arise in adding existentials to Haskell? I have a feeling that it would make working with GADTs more ergonomic.
> 
>  
> 
> On Fri, Sep 6, 2019 at 12:33 AM Simon Peyton Jones <simonpj at microsoft.com <mailto:simonpj at microsoft.com>> wrote:
> 
> I’m confused.   Char does not have the type (forall a. Show a =>a), so our example is iill-typed in System F, never mind about type inference.  Perhaps there’s a typo?   I think you may have ment
> 
>                exists a. Show a => a
> 
> which doesn’t exist in Haskell.  You can write existentials with a data type
> 
>  
> 
> data Showable where
> 
>    S :: forall a. Show a => a -> Showable
> 
>  
> 
> Then
> 
>                map show [S 1, S ‘a’, S “b”]
> 
> works fine today (without our new stuff), provided you say
> 
>  
> 
>                instance Show Showable where
> 
>                  show (S x) = show x
> 
>  
> 
> Our new system can only type programs that can be written in System F.   (The tricky bit is inferring the impredicative instantiations.)
> 
>  
> 
> Simon
> 
>  
> 
> From: ghc-devs <ghc-devs-bounces at haskell.org <mailto:ghc-devs-bounces at haskell.org>> On Behalf Of Alex Rozenshteyn
> Sent: 06 September 2019 03:31
> To: Alejandro Serrano Mena <trupill at gmail.com <mailto:trupill at gmail.com>>
> Cc: GHC developers <ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>>
> Subject: Re: New implementation for `ImpredicativeTypes`
> 
>  
> 
> I didn't say anything when you were requesting use cases, so I have no right to complain, but I'm still a little disappointed that this doesn't fix my (admittedly very minor) issue:https://www.reddit.com/r/haskell/comments/3am0qa/existentials_and_the_heterogenous_list_fallacy/csdwlp2/?context=8&depth=9 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.reddit.com%2Fr%2Fhaskell%2Fcomments%2F3am0qa%2Fexistentials_and_the_heterogenous_list_fallacy%2Fcsdwlp2%2F%3Fcontext%3D8%26depth%3D9&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328936796%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=IFOf9nl11qhDksPnK9jf2HxcHm7QMhVUGJZX%2F%2FOVim4%3D&reserved=0>
>  
> 
> For those who don't want to click on the reddit link: I would like to be able to write something like map show ([1, 'a', "b"] :: [forall a. Show a => a]), and have it work.
> 
>  
> 
> On Wed, Sep 4, 2019 at 8:13 AM Alejandro Serrano Mena <trupill at gmail.com <mailto:trupill at gmail.com>> wrote:
> 
> Hi all,
> 
> As I mentioned some time ago, we have been busy working on a new implementation of `ImpredicativeTypes` for GHC. I am very thankful to everybody who back then sent us examples of impredicativity which would be nice to support, as far as we know this branch supports all of them! :)
> 
>  
> 
> If you want to try it, athttps://gitlab.haskell.org/trupill/ghc/commit/a3f95a0fe0f647702fd7225fa719a8062a4cc0a5/pipelines?ref=quick-look-build <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Ftrupill%2Fghc%2Fcommit%2Fa3f95a0fe0f647702fd7225fa719a8062a4cc0a5%2Fpipelines%3Fref%3Dquick-look-build&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328936796%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=eR1k8ntH2aGT5nTFOQxd86hGr5OxDSjmkxFpy09fGlg%3D&reserved=0> you can find the result of the pipeline, which includes builds for several platforms (click on the "Artifacts" button, the one which looks like a cloud, to get them). The code is being developed at https://gitlab.haskell.org/trupill/ghc <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Ftrupill%2Fghc&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328946794%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RdfPCIqvBF8VyeuGKmtOBJsnwcPl%2Bya%2B4KLi%2BWfrT3E%3D&reserved=0>.
> 
>  
> 
> Any code should run *unchanged* except for some eta-expansion required for some specific usage patterns of higher-rank types. Please don't hesitate to ask any questions or clarifications about it. A merge request for tracking this can be found at https://gitlab.haskell.org/ghc/ghc/merge_requests/1659 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2Fmerge_requests%2F1659&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328956787%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=aR6Lq4teuUmPe4o%2FOHp878Zm8OBfXVxoMrn0WqcN4fM%3D&reserved=0>
>  
> 
> Kind regards,
> 
> Alejandro
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs <https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328956787%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=IUsQu1gDuNetafA6ljvQwNBEYq%2F%2B2NFYqwT5Ytpal24%3D&reserved=0>_______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs <http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs>
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210902/a366532c/attachment.html>

From john.ericson at obsidian.systems  Thu Sep  2 18:56:40 2021
From: john.ericson at obsidian.systems (john.ericson)
Date: Thu, 02 Sep 2021 14:56:40 -0400
Subject: New implementation for `ImpredicativeTypes`
In-Reply-To: <010f017ba7c9b8ff-b15dfb92-5bc3-4cb9-9fa6-9e4e518788ee-000000@us-east-2.amazonses.com>
References: <CAHnFXOumYR+nKkBv6B6_zHVA2D=b9kZVyZ3vTTu87R07h90YuA@mail.gmail.com>
 <CALm==BVhBZ4Gwd+011zmy-qT266JSU+BHH=fjOac_P1bmANTyw@mail.gmail.com>
 <VI1PR83MB0319241FB05C0F793E35C7AEADBA0@VI1PR83MB0319.EURPRD83.prod.outlook.com>
 <CALm==BWMAJgs+t=1pF4H=-WH3wRo1A5ZPfyhOsY5d1y4XL9xCA@mail.gmail.com>
 <DB7PR83MB0313E44B53429012C3BE69A6ADB70@DB7PR83MB0313.EURPRD83.prod.outlook.com>
 <CALm==BWCtjvEZ296usfNzP7mj5Cne4t5vMAivSd0mYRrrdeF_Q@mail.gmail.com>
 <VI1PR83MB026923BF6143E9EF17840C5EADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <CALm==BV8iGBN5bU57AMPYb-0pdSNc2qAcGP-uTbFe+3MO8yHfA@mail.gmail.com>
 <VI1PR83MB02698AD55C54B8C1B237F253ADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <CALm==BVCTZKUp_-JYqZo01aC3C85Kk+RBkJmDDh8fa_ydBYLLg@mail.gmail.com>
 <010f017ba7c9b8ff-b15dfb92-5bc3-4cb9-9fa6-9e4e518788ee-000000@us-east-2.amazonses.com>
Message-ID: <17ba7dec680.126cbbde189926.7935009804775750820@obsidian.systems>

Does the most basic e.g.newtype Some f where  MkSome :: forall a. f a -> Some fHave one of those problematic equalities? ---- On Thu, 02 Sep 2021 14:33:40 -0400  lists at richarde.dev  wrote ----On Sep 2, 2021, at 2:10 PM, Alex Rozenshteyn <rpglover64 at gmail.com> wrote:Oh, I see. That's because this would need to introduce `pack ... as ...` and `open ...` into the core term language, right?Exactly, yes.My sense is that it shouldn't negatively affect runtime performance of programs without existentials even if implemented naively; does that seem accurate? Not that implementing it, even naively, is a small task. I would agree with this, too.On Sep 2, 2021, at 2:21 PM, john.ericson <john.ericson at obsidian.systems> wrote:This reminds me...can we do newtype GADTs in certain situations as a stepping stone? I would think that would be purely easier — more nominal, no nice projections but only `case` and skolems which cannot escape.Newtype GADTs we're long deemed impossible IIRC, but surely the paper demonstrates that at least some cases should work?I don't quite see how this relates to existentials. Note that the paper does not address e.g. packing equalities in existentials, which would be needed for interacting with GADTs.Glad folks are enjoying the paper! :)Richard---- On Thu, 02 Sep 2021 14:10:34 -0400 rpglover64 at gmail.com wrote ----Oh, I see. That's because this would need to introduce `pack ... as ...` and `open ...` into the core term language, right?My sense is that it shouldn't negatively affect runtime performance of programs without existentials even if implemented naively; does that seem accurate? Not that implementing it, even naively, is a small task. On Thu, Sep 2, 2021 at 1:44 PM Simon Peyton Jones <simonpj at microsoft.com> wrote:Of course not. The same was true for QuickLook, though, wasn't it?No, not at all.   QuickLook required zero changes to GHC’s intermediate language – it impacted only the type inference system.   Adding existentials will entail a substantial change to the intermediate language, affecting every optimisation pass. Simon From: Alex Rozenshteyn <rpglover64 at gmail.com> Sent: 02 September 2021 18:13To: Simon Peyton Jones <simonpj at microsoft.com>Cc: GHC developers <ghc-devs at haskell.org>Subject: Re: New implementation for `ImpredicativeTypes` So it’s not just a question of saying “just add that paper to GHC and voila job done”.  Of course not. The same was true for QuickLook, though, wasn't it? On Thu, Sep 2, 2021 at 12:42 PM Simon Peyton Jones <simonpj at microsoft.com> wrote:If I understand correctly, the recent ICFP paper "An Existential Crisis Resolved" finally enables this; is that right?It describes one way to include existentials in GHC’s intermediate language, which is a real contribution. But it is not a small change.  So it’s not just a question of saying “just add that paper to GHC and voila job done”. Simon From: Alex Rozenshteyn <rpglover64 at gmail.com> Sent: 02 September 2021 17:10To: Simon Peyton Jones <simonpj at microsoft.com>Cc: GHC developers <ghc-devs at haskell.org>Subject: Re: New implementation for `ImpredicativeTypes` If I understand correctly, the recent ICFP paper "An Existential Crisis Resolved" finally enables this; is that right? On Mon, Sep 9, 2019 at 12:00 PM Simon Peyton Jones <simonpj at microsoft.com> wrote:Suppose Haskell did have existentials; Yes, I think that’s an interesting thing to work on!  I’m not sure what the implications would be.  At very least we’d need to extend System FC (GHC’s intermediate language) with existential types and the corresponding pack and unpack syntactic forms. I don’t know of any work studying that question specifically, but others may have pointers. simon From: Alex Rozenshteyn <rpglover64 at gmail.com> Sent: 06 September 2019 15:21To: Simon Peyton Jones <simonpj at microsoft.com>Cc: Alejandro Serrano Mena <trupill at gmail.com>; GHC developers <ghc-devs at haskell.org>Subject: Re: New implementation for `ImpredicativeTypes` Hi Simon, You're exactly right, of course. My example is confusing, so let me see if I can clarify. What I want in the ideal is map show [1, 'a', "b"]. That is, minimal syntactic overhead to mapping a function over multiple values of distinct types that results in a homogeneous list. As the reddit thread points out, there are workarounds involving TH or wrapping each element in a constructor or using bespoke operators, but when it comes down to it, none of them actually allows me to say what I mean; the TH one is closest, but I reach for TH only in times of desperation. I had thought that one of the things preventing this was lack of impredicative instantiation, but now I'm not sure. Suppose Haskell did have existentials; would map show @(exists a. Show a => a) [1, 'a', "b"] work in current Haskell and/or in quick-look? Tangentially, do you have a reference for what difficulties arise in adding existentials to Haskell? I have a feeling that it would make working with GADTs more ergonomic. On Fri, Sep 6, 2019 at 12:33 AM Simon Peyton Jones <simonpj at microsoft.com> wrote:I’m confused.   Char does not have the type (forall a. Show a =>a), so our example is iill-typed in System F, never mind about type inference.  Perhaps there’s a typo?   I think you may have ment               exists a. Show a => awhich doesn’t exist in Haskell.  You can write existentials with a data type data Showable where   S :: forall a. Show a => a -> Showable Then               map show [S 1, S ‘a’, S “b”]works fine today (without our new stuff), provided you say                instance Show Showable where                 show (S x) = show x Our new system can only type programs that can be written in System F.   (The tricky bit is inferring the impredicative instantiations.) Simon From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Alex RozenshteynSent: 06 September 2019 03:31To: Alejandro Serrano Mena <trupill at gmail.com>Cc: GHC developers <ghc-devs at haskell.org>Subject: Re: New implementation for `ImpredicativeTypes` I didn't say anything when you were requesting use cases, so I have no right to complain, but I'm still a little disappointed that this doesn't fix my (admittedly very minor) issue:https://www.reddit.com/r/haskell/comments/3am0qa/existentials_and_the_heterogenous_list_fallacy/csdwlp2/?context=8&depth=9 For those who don't want to click on the reddit link: I would like to be able to write something like map show ([1, 'a', "b"] :: [forall a. Show a => a]), and have it work. On Wed, Sep 4, 2019 at 8:13 AM Alejandro Serrano Mena <trupill at gmail.com> wrote:Hi all,As I mentioned some time ago, we have been busy working on a new implementation of `ImpredicativeTypes` for GHC. I am very thankful to everybody who back then sent us examples of impredicativity which would be nice to support, as far as we know this branch supports all of them! :) If you want to try it, athttps://gitlab.haskell.org/trupill/ghc/commit/a3f95a0fe0f647702fd7225fa719a8062a4cc0a5/pipelines?ref=quick-look-build you can find the result of the pipeline, which includes builds for several platforms (click on the "Artifacts" button, the one which looks like a cloud, to get them). The code is being developed at https://gitlab.haskell.org/trupill/ghc. Any code should run *unchanged* except for some eta-expansion required for some specific usage patterns of higher-rank types. Please don't hesitate to ask any questions or clarifications about it. A merge request for tracking this can be found at https://gitlab.haskell.org/ghc/ghc/merge_requests/1659 Kind regards,Alejandro_______________________________________________ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs_______________________________________________ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs_______________________________________________ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs_______________________________________________ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210902/d38b8297/attachment-0001.html>

From ben at smart-cactus.org  Thu Sep  2 19:00:50 2021
From: ben at smart-cactus.org (Ben Gamari)
Date: Thu, 02 Sep 2021 15:00:50 -0400
Subject: Potential improvement in compacting GC
In-Reply-To: <CAMQQO3m4hCVerYnbyoakj6QFdL1ZXzjzu54JK857+e0FByMg2Q@mail.gmail.com>
References: <CAMQQO3=7haokj1uUUt9KoSQ2YLOd7Y3Ue20atXhW2HwL=Z+2nQ@mail.gmail.com>
 <CAMQQO3nHFzL5aVFSq-RkSVcoS6g_WhjHJrsDF_MboowMwXCnAA@mail.gmail.com>
 <CAMQQO3m4hCVerYnbyoakj6QFdL1ZXzjzu54JK857+e0FByMg2Q@mail.gmail.com>
Message-ID: <87a6kun6pi.fsf@smart-cactus.org>

Ömer Sinan Ağacan <omeragacan at gmail.com> writes:

> Here's another improvement that fixes a very old issue in GHC's compacting GC
> [1].
>
> To summarize the problem: when untreading an object we update references to the
> object that we've seen so far to the object's new location. But to get the
> object's new location we need to know the object's size, because depending on
> the size we may need to move the object to a new block (when the current block
> does not have enough space for the object).
>
> For this we currently walk the thread twice, once to get the info table (used
> to get the size), then again to update references to the object. Ideally we
> want to do just one pass when unthreading.
>
> The solution is explained in The GC Handbook, section 3.4. Instead of using one
> bit to mark an object, we use two bits: one for the first word of the object,
> one for the last word. Using two bits is not a problem in GHC because heap
> objects are at least 2 words. For example, an object with two words is marked
> with `11`, 3 words is marked with `101` and so on.
>
> Now we can scan the bitmap to find object size, and unthread it without having
> to find the info table first.
>
Thanks Ömer! I have plopped all of these ideas into a ticket, #20328.
Hopefully someone will come along to implement.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210902/ffb737f4/attachment.sig>

From lists at richarde.dev  Fri Sep  3 03:04:22 2021
From: lists at richarde.dev (Richard Eisenberg)
Date: Fri, 3 Sep 2021 03:04:22 +0000
Subject: New implementation for `ImpredicativeTypes`
In-Reply-To: <17ba7dec680.126cbbde189926.7935009804775750820@obsidian.systems>
References: <CAHnFXOumYR+nKkBv6B6_zHVA2D=b9kZVyZ3vTTu87R07h90YuA@mail.gmail.com>
 <CALm==BVhBZ4Gwd+011zmy-qT266JSU+BHH=fjOac_P1bmANTyw@mail.gmail.com>
 <VI1PR83MB0319241FB05C0F793E35C7AEADBA0@VI1PR83MB0319.EURPRD83.prod.outlook.com>
 <CALm==BWMAJgs+t=1pF4H=-WH3wRo1A5ZPfyhOsY5d1y4XL9xCA@mail.gmail.com>
 <DB7PR83MB0313E44B53429012C3BE69A6ADB70@DB7PR83MB0313.EURPRD83.prod.outlook.com>
 <CALm==BWCtjvEZ296usfNzP7mj5Cne4t5vMAivSd0mYRrrdeF_Q@mail.gmail.com>
 <VI1PR83MB026923BF6143E9EF17840C5EADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <CALm==BV8iGBN5bU57AMPYb-0pdSNc2qAcGP-uTbFe+3MO8yHfA@mail.gmail.com>
 <VI1PR83MB02698AD55C54B8C1B237F253ADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <CALm==BVCTZKUp_-JYqZo01aC3C85Kk+RBkJmDDh8fa_ydBYLLg@mail.gmail.com>
 <010f017ba7c9b8ff-b15dfb92-5bc3-4cb9-9fa6-9e4e518788ee-000000@us-east-2.amazonses.com>
 <17ba7dec680.126cbbde189926.7935009804775750820@obsidian.systems>
Message-ID: <010f017ba99d4730-68884819-acc3-427b-93ef-7c26e3cd0b53-000000@us-east-2.amazonses.com>



> On Sep 2, 2021, at 2:56 PM, john.ericson <john.ericson at obsidian.systems> wrote:
> 
> Does the most basic e.g.
> 
> newtype Some f where
>   MkSome :: forall a. f a -> Some f
> 
> Have one of those problematic equalities?

No. That's not a GADT -- the constructor doesn't restrict anything about `f`.

I think you're after newtype existentials. I think these should indeed be possible, because what you propose appears to be the same as

newtype Some f = MkSome (exists a. f a)

We can probably support the syntax you wrote, too, but I don't want to commit to that right now.

Richard

> 
> 
> ---- On Thu, 02 Sep 2021 14:33:40 -0400 lists at richarde.dev wrote ----
> 
> 
> 
> On Sep 2, 2021, at 2:10 PM, Alex Rozenshteyn <rpglover64 at gmail.com <mailto:rpglover64 at gmail.com>> wrote:
> 
> Oh, I see. That's because this would need to introduce `pack ... as ...` and `open ...` into the core term language, right?
> 
> Exactly, yes.
> 
> 
> My sense is that it shouldn't negatively affect runtime performance of programs without existentials even if implemented naively; does that seem accurate? Not that implementing it, even naively, is a small task. 
> 
> I would agree with this, too.
> 
> On Sep 2, 2021, at 2:21 PM, john.ericson <john.ericson at obsidian.systems <mailto:john.ericson at obsidian.systems>> wrote:
> 
> This reminds me...can we do newtype GADTs in certain situations as a stepping stone? I would think that would be purely easier — more nominal, no nice projections but only `case` and skolems which cannot escape.
> 
> Newtype GADTs we're long deemed impossible IIRC, but surely the paper demonstrates that at least some cases should work?
> 
> I don't quite see how this relates to existentials. Note that the paper does not address e.g. packing equalities in existentials, which would be needed for interacting with GADTs.
> 
> Glad folks are enjoying the paper! :)
> 
> Richard
> 
> 
> 
> ---- On Thu, 02 Sep 2021 14:10:34 -0400 rpglover64 at gmail.com <mailto:rpglover64 at gmail.com> wrote ----
> 
> Oh, I see. That's because this would need to introduce `pack ... as ...` and `open ...` into the core term language, right?
> 
> My sense is that it shouldn't negatively affect runtime performance of programs without existentials even if implemented naively; does that seem accurate? Not that implementing it, even naively, is a small task. 
> 
> On Thu, Sep 2, 2021 at 1:44 PM Simon Peyton Jones <simonpj at microsoft.com <mailto:simonpj at microsoft.com>> wrote:
> Of course not. The same was true for QuickLook, though, wasn't it?
> 
> No, not at all.   QuickLook required zero changes to GHC’s intermediate language – it impacted only the type inference system.   Adding existentials will entail a substantial change to the intermediate language, affecting every optimisation pass.
> 
>  
> 
> Simon
> 
>  
> 
> From: Alex Rozenshteyn <rpglover64 at gmail.com <mailto:rpglover64 at gmail.com>> 
> Sent: 02 September 2021 18:13
> To: Simon Peyton Jones <simonpj at microsoft.com <mailto:simonpj at microsoft.com>>
> Cc: GHC developers <ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>>
> Subject: Re: New implementation for `ImpredicativeTypes`
> 
>  
> 
> So it’s not just a question of saying “just add that paper to GHC and voila job done”. 
> 
>  
> 
> Of course not. The same was true for QuickLook, though, wasn't it?
> 
>  
> 
> On Thu, Sep 2, 2021 at 12:42 PM Simon Peyton Jones <simonpj at microsoft.com <mailto:simonpj at microsoft.com>> wrote:
> 
> If I understand correctly, the recent ICFP paper "An Existential Crisis Resolved <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fdoi%2Fpdf%2F10.1145%2F3473569&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328916809%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=kioGsomPn8BuvBLQx6KEr5glMTPvw6m93DK%2BGA0lM08%3D&reserved=0>" finally enables this; is that right?
> 
> It describes one way to include existentials in GHC’s intermediate language, which is a real contribution. But it is not a small change.  So it’s not just a question of saying “just add that paper to GHC and voila job done”.
> 
>  
> 
> Simon
> 
>  
> 
> From: Alex Rozenshteyn <rpglover64 at gmail.com <mailto:rpglover64 at gmail.com>> 
> Sent: 02 September 2021 17:10
> To: Simon Peyton Jones <simonpj at microsoft.com <mailto:simonpj at microsoft.com>>
> Cc: GHC developers <ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>>
> Subject: Re: New implementation for `ImpredicativeTypes`
> 
>  
> 
> If I understand correctly, the recent ICFP paper "An Existential Crisis Resolved <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fdoi%2Fpdf%2F10.1145%2F3473569&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328926803%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=DtcIgtgrp3zJygorYY32WBKhF40rYtAOhMOOmPWf6pc%3D&reserved=0>" finally enables this; is that right?
> 
>  
> 
> On Mon, Sep 9, 2019 at 12:00 PM Simon Peyton Jones <simonpj at microsoft.com <mailto:simonpj at microsoft.com>> wrote:
> 
> Suppose Haskell did have existentials;
> 
>  
> 
> Yes, I think that’s an interesting thing to work on!  I’m not sure what the implications would be.  At very least we’d need to extend System FC (GHC’s intermediate language) with existential types and the corresponding pack and unpack syntactic forms.
> 
>  
> 
> I don’t know of any work studying that question specifically, but others may have pointers.
> 
>  
> 
> simon
> 
>  
> 
> From: Alex Rozenshteyn <rpglover64 at gmail.com <mailto:rpglover64 at gmail.com>> 
> Sent: 06 September 2019 15:21
> To: Simon Peyton Jones <simonpj at microsoft.com <mailto:simonpj at microsoft.com>>
> Cc: Alejandro Serrano Mena <trupill at gmail.com <mailto:trupill at gmail.com>>; GHC developers <ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>>
> Subject: Re: New implementation for `ImpredicativeTypes`
> 
>  
> 
> Hi Simon,
> 
>  
> 
> You're exactly right, of course. My example is confusing, so let me see if I can clarify.
> 
>  
> 
> What I want in the ideal is map show [1, 'a', "b"]. That is, minimal syntactic overhead to mapping a function over multiple values of distinct types that results in a homogeneous list. As the reddit thread points out, there are workarounds involving TH or wrapping each element in a constructor or using bespoke operators, but when it comes down to it, none of them actually allows me to say what I mean; the TH one is closest, but I reach for TH only in times of desperation.
> 
>  
> 
> I had thought that one of the things preventing this was lack of impredicative instantiation, but now I'm not sure. Suppose Haskell did have existentials; would map show @(exists a. Show a => a) [1, 'a', "b"] work in current Haskell and/or in quick-look?
> 
>  
> 
> Tangentially, do you have a reference for what difficulties arise in adding existentials to Haskell? I have a feeling that it would make working with GADTs more ergonomic.
> 
>  
> 
> On Fri, Sep 6, 2019 at 12:33 AM Simon Peyton Jones <simonpj at microsoft.com <mailto:simonpj at microsoft.com>> wrote:
> 
> I’m confused.   Char does not have the type (forall a. Show a =>a), so our example is iill-typed in System F, never mind about type inference.  Perhaps there’s a typo?   I think you may have ment
> 
>                exists a. Show a => a
> 
> which doesn’t exist in Haskell.  You can write existentials with a data type
> 
>  
> 
> data Showable where
> 
>    S :: forall a. Show a => a -> Showable
> 
>  
> 
> Then
> 
>                map show [S 1, S ‘a’, S “b”]
> 
> works fine today (without our new stuff), provided you say
> 
>  
> 
>                instance Show Showable where
> 
>                  show (S x) = show x
> 
>  
> 
> Our new system can only type programs that can be written in System F.   (The tricky bit is inferring the impredicative instantiations.)
> 
>  
> 
> Simon
> 
>  
> 
> From: ghc-devs <ghc-devs-bounces at haskell.org <mailto:ghc-devs-bounces at haskell.org>> On Behalf Of Alex Rozenshteyn
> Sent: 06 September 2019 03:31
> To: Alejandro Serrano Mena <trupill at gmail.com <mailto:trupill at gmail.com>>
> Cc: GHC developers <ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>>
> Subject: Re: New implementation for `ImpredicativeTypes`
> 
>  
> 
> I didn't say anything when you were requesting use cases, so I have no right to complain, but I'm still a little disappointed that this doesn't fix my (admittedly very minor) issue:https://www.reddit.com/r/haskell/comments/3am0qa/existentials_and_the_heterogenous_list_fallacy/csdwlp2/?context=8&depth=9 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.reddit.com%2Fr%2Fhaskell%2Fcomments%2F3am0qa%2Fexistentials_and_the_heterogenous_list_fallacy%2Fcsdwlp2%2F%3Fcontext%3D8%26depth%3D9&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328936796%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=IFOf9nl11qhDksPnK9jf2HxcHm7QMhVUGJZX%2F%2FOVim4%3D&reserved=0>
>  
> 
> For those who don't want to click on the reddit link: I would like to be able to write something like map show ([1, 'a', "b"] :: [forall a. Show a => a]), and have it work.
> 
>  
> 
> On Wed, Sep 4, 2019 at 8:13 AM Alejandro Serrano Mena <trupill at gmail.com <mailto:trupill at gmail.com>> wrote:
> 
> Hi all,
> 
> As I mentioned some time ago, we have been busy working on a new implementation of `ImpredicativeTypes` for GHC. I am very thankful to everybody who back then sent us examples of impredicativity which would be nice to support, as far as we know this branch supports all of them! :)
> 
>  
> 
> If you want to try it, athttps://gitlab.haskell.org/trupill/ghc/commit/a3f95a0fe0f647702fd7225fa719a8062a4cc0a5/pipelines?ref=quick-look-build <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Ftrupill%2Fghc%2Fcommit%2Fa3f95a0fe0f647702fd7225fa719a8062a4cc0a5%2Fpipelines%3Fref%3Dquick-look-build&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328936796%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=eR1k8ntH2aGT5nTFOQxd86hGr5OxDSjmkxFpy09fGlg%3D&reserved=0> you can find the result of the pipeline, which includes builds for several platforms (click on the "Artifacts" button, the one which looks like a cloud, to get them). The code is being developed at https://gitlab.haskell.org/trupill/ghc <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Ftrupill%2Fghc&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328946794%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RdfPCIqvBF8VyeuGKmtOBJsnwcPl%2Bya%2B4KLi%2BWfrT3E%3D&reserved=0>.
> 
>  
> 
> Any code should run *unchanged* except for some eta-expansion required for some specific usage patterns of higher-rank types. Please don't hesitate to ask any questions or clarifications about it. A merge request for tracking this can be found at https://gitlab.haskell.org/ghc/ghc/merge_requests/1659 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2Fmerge_requests%2F1659&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328956787%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=aR6Lq4teuUmPe4o%2FOHp878Zm8OBfXVxoMrn0WqcN4fM%3D&reserved=0>
>  
> 
> Kind regards,
> 
> Alejandro
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs <https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=04%7C01%7Csimonpj%40microsoft.com%7Cc0f0a66e10f94a6f17d508d96e34f2ae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637661997328956787%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=IUsQu1gDuNetafA6ljvQwNBEYq%2F%2B2NFYqwT5Ytpal24%3D&reserved=0>_______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs <http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs>
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs <http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs>
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210903/937bfd9d/attachment.html>

From harendra.kumar at gmail.com  Mon Sep  6 13:11:23 2021
From: harendra.kumar at gmail.com (Harendra Kumar)
Date: Mon, 6 Sep 2021 18:41:23 +0530
Subject: Question about specialization
Message-ID: <CAPW+kkarYvt0bic=4MN_Y=chgNZziAaQGhUKS7zVcP+shOKFSQ@mail.gmail.com>

Hi GHC devs,

I have a simple program using the streamly library, as follows, the whole
code is in the same module:

{-# INLINE iterateState #-}
{-# SPECIALIZE iterateState :: Int -> SerialT (StateT Int IO) Int #-}
iterateState :: MonadState Int m => Int -> SerialT m Int
iterateState n = do
    x <- get
    if x > n
    then do
        put (x - 1)
        iterateState n
    else return x

main :: IO ()
main = do
    State.evalStateT (S.drain (iterateState 0)) 100000

Earlier the SPECIALIZE pragma was not required on iterateState, but after
some refactoring in the library (the monad bind of SerialT changed a bit),
this program now requires a SPECIALIZE on iterateState to trigger
specialization, just INLINE also does not help.

My question is whether this may be expected in some conditions or is this
something which can be considered a bug in the compiler? I am also curious
what specifically could have made the compiler not specialize this anymore,
is it the size of the function or some other threshold?

Thanks,
Harendra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210906/372669e0/attachment.html>

From simonpj at microsoft.com  Mon Sep  6 14:47:40 2021
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 6 Sep 2021 14:47:40 +0000
Subject: Question about specialization
In-Reply-To: <CAPW+kkarYvt0bic=4MN_Y=chgNZziAaQGhUKS7zVcP+shOKFSQ@mail.gmail.com>
References: <CAPW+kkarYvt0bic=4MN_Y=chgNZziAaQGhUKS7zVcP+shOKFSQ@mail.gmail.com>
Message-ID: <VI1PR83MB0269F9462E1FD8045951FF6CADD29@VI1PR83MB0269.EURPRD83.prod.outlook.com>

Harendra

That comes as a surprise to me.  Could you possibly make a repo case, and say what version of the compiler does, and does not, specialise the function?

File it as a ticket … to me it looks like a bug.

Thanks

Simon

From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Harendra Kumar
Sent: 06 September 2021 14:11
To: ghc-devs at haskell.org
Subject: Question about specialization

Hi GHC devs,

I have a simple program using the streamly library, as follows, the whole code is in the same module:

{-# INLINE iterateState #-}
{-# SPECIALIZE iterateState :: Int -> SerialT (StateT Int IO) Int #-}
iterateState :: MonadState Int m => Int -> SerialT m Int
iterateState n = do
    x <- get
    if x > n
    then do
        put (x - 1)
        iterateState n
    else return x

main :: IO ()
main = do
    State.evalStateT (S.drain (iterateState 0)) 100000

Earlier the SPECIALIZE pragma was not required on iterateState, but after some refactoring in the library (the monad bind of SerialT changed a bit), this program now requires a SPECIALIZE on iterateState to trigger specialization, just INLINE also does not help.

My question is whether this may be expected in some conditions or is this something which can be considered a bug in the compiler? I am also curious what specifically could have made the compiler not specialize this anymore, is it the size of the function or some other threshold?

Thanks,
Harendra


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210906/a6032ecb/attachment.html>

From john.ericson at obsidian.systems  Mon Sep  6 15:21:53 2021
From: john.ericson at obsidian.systems (John Ericson)
Date: Mon, 6 Sep 2021 11:21:53 -0400
Subject: New implementation for `ImpredicativeTypes`
In-Reply-To: <010f017ba99d4730-68884819-acc3-427b-93ef-7c26e3cd0b53-000000@us-east-2.amazonses.com>
References: <CAHnFXOumYR+nKkBv6B6_zHVA2D=b9kZVyZ3vTTu87R07h90YuA@mail.gmail.com>
 <CALm==BVhBZ4Gwd+011zmy-qT266JSU+BHH=fjOac_P1bmANTyw@mail.gmail.com>
 <VI1PR83MB0319241FB05C0F793E35C7AEADBA0@VI1PR83MB0319.EURPRD83.prod.outlook.com>
 <CALm==BWMAJgs+t=1pF4H=-WH3wRo1A5ZPfyhOsY5d1y4XL9xCA@mail.gmail.com>
 <DB7PR83MB0313E44B53429012C3BE69A6ADB70@DB7PR83MB0313.EURPRD83.prod.outlook.com>
 <CALm==BWCtjvEZ296usfNzP7mj5Cne4t5vMAivSd0mYRrrdeF_Q@mail.gmail.com>
 <VI1PR83MB026923BF6143E9EF17840C5EADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <CALm==BV8iGBN5bU57AMPYb-0pdSNc2qAcGP-uTbFe+3MO8yHfA@mail.gmail.com>
 <VI1PR83MB02698AD55C54B8C1B237F253ADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <CALm==BVCTZKUp_-JYqZo01aC3C85Kk+RBkJmDDh8fa_ydBYLLg@mail.gmail.com>
 <010f017ba7c9b8ff-b15dfb92-5bc3-4cb9-9fa6-9e4e518788ee-000000@us-east-2.amazonses.com>
 <17ba7dec680.126cbbde189926.7935009804775750820@obsidian.systems>
 <010f017ba99d4730-68884819-acc3-427b-93ef-7c26e3cd0b53-000000@us-east-2.amazonses.com>
Message-ID: <ee279076-aa4e-7e69-7c83-73a0f401759a@obsidian.systems>

On 9/2/21 11:04 PM, Richard Eisenberg wrote:

> On Sep 2, 2021, at 2:56 PM, john.ericson 
> <john.ericson at obsidian.systems <mailto:john.ericson at obsidian.systems>> 
> wrote:
>>
>> Does the most basic e.g.
>>
>> newtype Some f where
>>   MkSome :: forall a. f a -> Some f
>>
>> Have one of those problematic equalities?
>
> No. That's not a GADT -- the constructor doesn't restrict anything 
> about `f`.

Morally, sure, but GHC doesn't know about this.

I tried, and -XGADTSyntax + -XExistenialTypes = -XGADTs it seems.

>
> I think you're after newtype existentials. I think these should indeed 
> be possible, because what you propose appears to be the same as
>
> newtype Some f = MkSome (exists a. f a)
>
> We can probably support the syntax you wrote, too, but I don't want to 
> commit to that right now.

The syntax I wrote is already basically valid?

data Some f = forall a. Some (f a)

data Some f where MkSome :: forall a f. f a -> Some f

Is accepted

newtype Some f = forall a. Some (f a)

newtype Some f where MkSome :: forall a f. f a -> Some f

Is not with "A newtype constructor cannot have existential type variables"

I propose we teach GHC how these "GADTs" in fact merely have existential 
variables, and not the FC constraints that require the extra evaluation 
for soundness. Than we can get the operational/runtime benefits of what 
you propose for cheap. Don't get me wrong -- the other aspects in the 
paper this doesn't address are still quite valuable, but I think this is 
a useful stepping stone / removal of artificial restrictions we should 
do first.

This sort of thing is brought up in #1965, where it is alleged this is 
in fact more difficult than it sounds. All more reason it is a good 
stepping stone, I say!

John

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210906/44d3b766/attachment.html>

From harendra.kumar at gmail.com  Tue Sep  7 04:32:15 2021
From: harendra.kumar at gmail.com (Harendra Kumar)
Date: Tue, 7 Sep 2021 10:02:15 +0530
Subject: Question about specialization
In-Reply-To: <VI1PR83MB0269F9462E1FD8045951FF6CADD29@VI1PR83MB0269.EURPRD83.prod.outlook.com>
References: <CAPW+kkarYvt0bic=4MN_Y=chgNZziAaQGhUKS7zVcP+shOKFSQ@mail.gmail.com>
 <VI1PR83MB0269F9462E1FD8045951FF6CADD29@VI1PR83MB0269.EURPRD83.prod.outlook.com>
Message-ID: <CAPW+kkYSmd+-wn5+YVoGJLdM6uuwNXQzxtGDHYqE=xO8+93ghg@mail.gmail.com>

I am in the middle of a refactor. I will file a ticket once I am done.

On Mon, 6 Sept 2021 at 20:17, Simon Peyton Jones <simonpj at microsoft.com>
wrote:

> Harendra
>
>
>
> That comes as a surprise to me.  Could you possibly make a repo case, and
> say what version of the compiler does, and does not, specialise the
> function?
>
>
>
> File it as a ticket … to me it looks like a bug.
>
>
>
> Thanks
>
>
>
> Simon
>
>
>
> *From:* ghc-devs <ghc-devs-bounces at haskell.org> *On Behalf Of *Harendra
> Kumar
> *Sent:* 06 September 2021 14:11
> *To:* ghc-devs at haskell.org
> *Subject:* Question about specialization
>
>
>
> Hi GHC devs,
>
>
>
> I have a simple program using the streamly library, as follows, the whole
> code is in the same module:
>
>
>
> {-# INLINE iterateState #-}
> {-# SPECIALIZE iterateState :: Int -> SerialT (StateT Int IO) Int #-}
> iterateState :: MonadState Int m => Int -> SerialT m Int
> iterateState n = do
>     x <- get
>     if x > n
>     then do
>         put (x - 1)
>         iterateState n
>     else return x
>
>
>
> main :: IO ()
> main = do
>     State.evalStateT (S.drain (iterateState 0)) 100000
>
>
>
> Earlier the SPECIALIZE pragma was not required on iterateState, but after
> some refactoring in the library (the monad bind of SerialT changed a bit),
> this program now requires a SPECIALIZE on iterateState to trigger
> specialization, just INLINE also does not help.
>
>
>
> My question is whether this may be expected in some conditions or is this
> something which can be considered a bug in the compiler? I am also curious
> what specifically could have made the compiler not specialize this anymore,
> is it the size of the function or some other threshold?
>
>
>
> Thanks,
>
> Harendra
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210907/1f371fbb/attachment.html>

From lists at richarde.dev  Tue Sep  7 18:49:47 2021
From: lists at richarde.dev (Richard Eisenberg)
Date: Tue, 7 Sep 2021 18:49:47 +0000
Subject: new label: diagnostic quality
Message-ID: <010f017bc1984523-d9153075-2f0b-47e8-9bd8-b144650848d1-000000@us-east-2.amazonses.com>

Hi devs,

I've just created a new label in GitLab, called "diagnostic quality". (Diagnostics = errors ∪ warnings.) I intend for this label to be used when the report is about the quality of an error message or warning. That is, the existing message is not wrong, per se, but it's somehow not as helpful as it can be to users. I think it's helpful to separate out such issues from those describing error messages that are just wrong.

In particular, if you spot a ticket arising from https://github.com/haskell/error-messages, you may want to give it the "diagnostic quality" label.

If you disagree with the choice to make this label, please push back!

Thanks,
Richard

From lists at richarde.dev  Wed Sep  8 00:41:13 2021
From: lists at richarde.dev (Richard Eisenberg)
Date: Wed, 8 Sep 2021 00:41:13 +0000
Subject: New implementation for `ImpredicativeTypes`
In-Reply-To: <ee279076-aa4e-7e69-7c83-73a0f401759a@obsidian.systems>
References: <CAHnFXOumYR+nKkBv6B6_zHVA2D=b9kZVyZ3vTTu87R07h90YuA@mail.gmail.com>
 <CALm==BVhBZ4Gwd+011zmy-qT266JSU+BHH=fjOac_P1bmANTyw@mail.gmail.com>
 <VI1PR83MB0319241FB05C0F793E35C7AEADBA0@VI1PR83MB0319.EURPRD83.prod.outlook.com>
 <CALm==BWMAJgs+t=1pF4H=-WH3wRo1A5ZPfyhOsY5d1y4XL9xCA@mail.gmail.com>
 <DB7PR83MB0313E44B53429012C3BE69A6ADB70@DB7PR83MB0313.EURPRD83.prod.outlook.com>
 <CALm==BWCtjvEZ296usfNzP7mj5Cne4t5vMAivSd0mYRrrdeF_Q@mail.gmail.com>
 <VI1PR83MB026923BF6143E9EF17840C5EADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <CALm==BV8iGBN5bU57AMPYb-0pdSNc2qAcGP-uTbFe+3MO8yHfA@mail.gmail.com>
 <VI1PR83MB02698AD55C54B8C1B237F253ADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <CALm==BVCTZKUp_-JYqZo01aC3C85Kk+RBkJmDDh8fa_ydBYLLg@mail.gmail.com>
 <010f017ba7c9b8ff-b15dfb92-5bc3-4cb9-9fa6-9e4e518788ee-000000@us-east-2.amazonses.com>
 <17ba7dec680.126cbbde189926.7935009804775750820@obsidian.systems>
 <010f017ba99d4730-68884819-acc3-427b-93ef-7c26e3cd0b53-000000@us-east-2.amazonses.com>
 <ee279076-aa4e-7e69-7c83-73a0f401759a@obsidian.systems>
Message-ID: <010f017bc2da0727-00662b22-2002-46db-8989-ae5874af25f5-000000@us-east-2.amazonses.com>



> On Sep 6, 2021, at 11:21 AM, John Ericson <john.ericson at obsidian.systems> wrote:
> 
> On 9/2/21 11:04 PM, Richard Eisenberg wrote:
> 
>> On Sep 2, 2021, at 2:56 PM, john.ericson <john.ericson at obsidian.systems <mailto:john.ericson at obsidian.systems>> wrote:
>>> 
>>> Does the most basic e.g.
>>> 
>>> newtype Some f where
>>>   MkSome :: forall a. f a -> Some f
>>> 
>>> Have one of those problematic equalities?
>> 
>> No. That's not a GADT -- the constructor doesn't restrict anything about `f`.
> Morally, sure, but GHC doesn't know about this.
> 

Sure it does -- GHC doesn't include an equality constraint as one of the fields of MkSome. This isn't about extensions -- it's about the way the data constructor is interpreted.
>> 
>> I think you're after newtype existentials. I think these should indeed be possible, because what you propose appears to be the same as
>> 
>> newtype Some f = MkSome (exists a. f a)
>> 
>> We can probably support the syntax you wrote, too, but I don't want to commit to that right now.
> The syntax I wrote is already basically valid?
> 
> data Some f = forall a. Some (f a)
> data Some f where MkSome :: forall a f. f a -> Some f
> Is accepted
> 
> newtype Some f = forall a. Some (f a)
> newtype Some f where MkSome :: forall a f. f a -> Some f
> Is not with "A newtype constructor cannot have existential type variables"
> 
> I propose we teach GHC how these "GADTs" in fact merely have existential variables, and not the FC constraints that require the extra evaluation for soundness. Than we can get the operational/runtime benefits of what you propose for cheap. Don't get me wrong -- the other aspects in the paper this doesn't address are still quite valuable, but I think this is a useful stepping stone / removal of artificial restrictions we should do first.
> 
> This sort of thing is brought up in #1965, where it is alleged this is in fact more difficult than it sounds. All more reason it is a good stepping stone, I say!
> 

This is more difficult than it sounds. :) Newtypes are implemented via coercions in Core, and coercions are inherently bidirectional. The appearance of an existential in this way requires one-way conversions, which are currently not supported. So, to get what you want, we'd have to modify the core language as in the existentials paper, along with some extra work to automatically add `pack` and `open` -- rather similar to the type inference described in the existentials paper. The bottom line for me is that this seems just as hard as implementing the whole thing, so I see no value in having the stepping stone. If we always wrote out the newtype constructor, then maybe we could use its appearance to guide the `pack` and `open`, but we don't: sometimes, we just use `coerce`. So I really don't think this is any easier than implementing the paper as written. Once that's done, we can come back and add this new feature relatively easily (I think).

Richard

> John
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210908/cf25aec8/attachment.html>

From cdsmith at gmail.com  Wed Sep  8 01:03:19 2021
From: cdsmith at gmail.com (Chris Smith)
Date: Tue, 7 Sep 2021 21:03:19 -0400
Subject: new label: diagnostic quality
In-Reply-To: <010f017bc1984523-d9153075-2f0b-47e8-9bd8-b144650848d1-000000@us-east-2.amazonses.com>
References: <010f017bc1984523-d9153075-2f0b-47e8-9bd8-b144650848d1-000000@us-east-2.amazonses.com>
Message-ID: <CAPq5PvLo2d9t3Q5Ma_aiDyQ0KcvC-YgAmu7jFLCRZVfr4m6s-g@mail.gmail.com>

Hi Richard,

One thing that I've done with CodeWorld for years now was to integrate
reporting poor error messages into the core workflow by adding a report
button on the compiler output directly in the tool.  This integration files
github issues with a special tag for error messages.  Because CodeWorld is
targeted at younger users and already rewrites error messages, most of the
reports I get from CodeWorld itself are not directly applicable to GHC.
Here's a list of cases where someone clicked that button at
code.world/haskell, but I usually close them with no action unless there's
something egregious because I want code.world/haskell to be an
authentic GHC experience.  In any case, I strongly suggest looking into
similar tooling for more standard Haskell environments with editors like
VSCode or Emacs.  I don't know if this could be done in HLS, or would need
to be separately implemented for each editor.  I'll happily offer to
implement a CodeWorld endpoint for filing GitLab error message reports
similar to this which such integrations could post to.  The lower you can
make the friction to report a misleading message, the more likely you'll
get authentic reports from users rather than idle issue-theorizing.

These will probably need some kind of a triage process to turn them into
actionable issues, and perhaps in my experience to just close 75% of them
where it's unclear what was meant or the user just wanted to gripe.  (That
may have something to do with the fact that my users have often
procrastinated on their homework, though...)  But that triage process is
something that you could easily ask for help on.  I would be willing to
comb through reports now and then, for example, and look for opportunities
to do better.

On Tue, Sep 7, 2021 at 2:51 PM Richard Eisenberg <lists at richarde.dev> wrote:

> Hi devs,
>
> I've just created a new label in GitLab, called "diagnostic quality".
> (Diagnostics = errors ∪ warnings.) I intend for this label to be used when
> the report is about the quality of an error message or warning. That is,
> the existing message is not wrong, per se, but it's somehow not as helpful
> as it can be to users. I think it's helpful to separate out such issues
> from those describing error messages that are just wrong.
>
> In particular, if you spot a ticket arising from
> https://github.com/haskell/error-messages, you may want to give it the
> "diagnostic quality" label.
>
> If you disagree with the choice to make this label, please push back!
>
> Thanks,
> Richard
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210907/dcad7f51/attachment.html>

From cdsmith at gmail.com  Wed Sep  8 01:04:06 2021
From: cdsmith at gmail.com (Chris Smith)
Date: Tue, 7 Sep 2021 21:04:06 -0400
Subject: new label: diagnostic quality
In-Reply-To: <CAPq5PvLo2d9t3Q5Ma_aiDyQ0KcvC-YgAmu7jFLCRZVfr4m6s-g@mail.gmail.com>
References: <010f017bc1984523-d9153075-2f0b-47e8-9bd8-b144650848d1-000000@us-east-2.amazonses.com>
 <CAPq5PvLo2d9t3Q5Ma_aiDyQ0KcvC-YgAmu7jFLCRZVfr4m6s-g@mail.gmail.com>
Message-ID: <CAPq5PvJ7zCAPba4iTvnajFzugwDS_Xjsc-OtQyHwiMt+Q-bY=w@mail.gmail.com>

Oops, I meant to include this URL:
https://github.com/google/codeworld/issues?q=label%3Aerror-message+%22code.world%2Fhaskell%22
is the full list (including closed issues) of cases where someone has
reported a misleading error message at code.world/haskell

On Tue, Sep 7, 2021 at 9:03 PM Chris Smith <cdsmith at gmail.com> wrote:

> Hi Richard,
>
> One thing that I've done with CodeWorld for years now was to integrate
> reporting poor error messages into the core workflow by adding a report
> button on the compiler output directly in the tool.  This integration files
> github issues with a special tag for error messages.  Because CodeWorld is
> targeted at younger users and already rewrites error messages, most of the
> reports I get from CodeWorld itself are not directly applicable to GHC.
> Here's a list of cases where someone clicked that button at
> code.world/haskell, but I usually close them with no action unless there's
> something egregious because I want code.world/haskell to be an
> authentic GHC experience.  In any case, I strongly suggest looking into
> similar tooling for more standard Haskell environments with editors like
> VSCode or Emacs.  I don't know if this could be done in HLS, or would need
> to be separately implemented for each editor.  I'll happily offer to
> implement a CodeWorld endpoint for filing GitLab error message reports
> similar to this which such integrations could post to.  The lower you can
> make the friction to report a misleading message, the more likely you'll
> get authentic reports from users rather than idle issue-theorizing.
>
> These will probably need some kind of a triage process to turn them into
> actionable issues, and perhaps in my experience to just close 75% of them
> where it's unclear what was meant or the user just wanted to gripe.  (That
> may have something to do with the fact that my users have often
> procrastinated on their homework, though...)  But that triage process is
> something that you could easily ask for help on.  I would be willing to
> comb through reports now and then, for example, and look for opportunities
> to do better.
>
> On Tue, Sep 7, 2021 at 2:51 PM Richard Eisenberg <lists at richarde.dev>
> wrote:
>
>> Hi devs,
>>
>> I've just created a new label in GitLab, called "diagnostic quality".
>> (Diagnostics = errors ∪ warnings.) I intend for this label to be used when
>> the report is about the quality of an error message or warning. That is,
>> the existing message is not wrong, per se, but it's somehow not as helpful
>> as it can be to users. I think it's helpful to separate out such issues
>> from those describing error messages that are just wrong.
>>
>> In particular, if you spot a ticket arising from
>> https://github.com/haskell/error-messages, you may want to give it the
>> "diagnostic quality" label.
>>
>> If you disagree with the choice to make this label, please push back!
>>
>> Thanks,
>> Richard
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210907/cd1aedfb/attachment.html>

From john.ericson at obsidian.systems  Wed Sep  8 04:48:31 2021
From: john.ericson at obsidian.systems (John Ericson)
Date: Wed, 8 Sep 2021 00:48:31 -0400
Subject: New implementation for `ImpredicativeTypes`
In-Reply-To: <010f017bc2da0727-00662b22-2002-46db-8989-ae5874af25f5-000000@us-east-2.amazonses.com>
References: <CAHnFXOumYR+nKkBv6B6_zHVA2D=b9kZVyZ3vTTu87R07h90YuA@mail.gmail.com>
 <CALm==BWMAJgs+t=1pF4H=-WH3wRo1A5ZPfyhOsY5d1y4XL9xCA@mail.gmail.com>
 <DB7PR83MB0313E44B53429012C3BE69A6ADB70@DB7PR83MB0313.EURPRD83.prod.outlook.com>
 <CALm==BWCtjvEZ296usfNzP7mj5Cne4t5vMAivSd0mYRrrdeF_Q@mail.gmail.com>
 <VI1PR83MB026923BF6143E9EF17840C5EADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <CALm==BV8iGBN5bU57AMPYb-0pdSNc2qAcGP-uTbFe+3MO8yHfA@mail.gmail.com>
 <VI1PR83MB02698AD55C54B8C1B237F253ADCE9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <CALm==BVCTZKUp_-JYqZo01aC3C85Kk+RBkJmDDh8fa_ydBYLLg@mail.gmail.com>
 <010f017ba7c9b8ff-b15dfb92-5bc3-4cb9-9fa6-9e4e518788ee-000000@us-east-2.amazonses.com>
 <17ba7dec680.126cbbde189926.7935009804775750820@obsidian.systems>
 <010f017ba99d4730-68884819-acc3-427b-93ef-7c26e3cd0b53-000000@us-east-2.amazonses.com>
 <ee279076-aa4e-7e69-7c83-73a0f401759a@obsidian.systems>
 <010f017bc2da0727-00662b22-2002-46db-8989-ae5874af25f5-000000@us-east-2.amazonses.com>
Message-ID: <726db007-e57a-5369-ce12-f2c4f59ec6e3@obsidian.systems>


On 9/7/21 8:41 PM, Richard Eisenberg wrote:
>
>
>> On Sep 6, 2021, at 11:21 AM, John Ericson 
>> <john.ericson at obsidian.systems 
>> <mailto:john.ericson at obsidian.systems>> wrote:
>>
>> On 9/2/21 11:04 PM, Richard Eisenberg wrote:
>>
>>> On Sep 2, 2021, at 2:56 PM, john.ericson 
>>> <john.ericson at obsidian.systems 
>>> <mailto:john.ericson at obsidian.systems>> wrote:
>>>>
>>>> Does the most basic e.g.
>>>>
>>>> newtype Some f where
>>>>   MkSome :: forall a. f a -> Some f
>>>>
>>>> Have one of those problematic equalities?
>>>
>>> No. That's not a GADT -- the constructor doesn't restrict anything 
>>> about `f`.
>>
>> Morally, sure, but GHC doesn't know about this.
>>
>
> Sure it does -- GHC doesn't include an equality constraint as one of 
> the fields of MkSome. This isn't about extensions -- it's about the 
> way the data constructor is interpreted.

Oops, agreed. I guess meant extensions didn't seem to check for this in 
a really syntax-driven way. But yes deciding the cases apart is not hard 
once the data definition is compiled.

>>>
>>> I think you're after newtype existentials. I think these should 
>>> indeed be possible, because what you propose appears to be the same as
>>>
>>> newtype Some f = MkSome (exists a. f a)
>>>
>>> We can probably support the syntax you wrote, too, but I don't want 
>>> to commit to that right now.
>>
>> The syntax I wrote is already basically valid?
>>
>> data Some f = forall a. Some (f a)
>> data Some f where MkSome :: forall a f. f a -> Some f
>>
>> Is accepted
>>
>> newtype Some f = forall a. Some (f a)
>> newtype Some f where MkSome :: forall a f. f a -> Some f
>>
>> Is not with "A newtype constructor cannot have existential type 
>> variables"
>>
>> I propose we teach GHC how these "GADTs" in fact merely have 
>> existential variables, and not the FC constraints that require the 
>> extra evaluation for soundness. Than we can get the 
>> operational/runtime benefits of what you propose for cheap. Don't get 
>> me wrong -- the other aspects in the paper this doesn't address are 
>> still quite valuable, but I think this is a useful stepping stone / 
>> removal of artificial restrictions we should do first.
>>
>> This sort of thing is brought up in #1965, where it is alleged this 
>> is in fact more difficult than it sounds. All more reason it is a 
>> good stepping stone, I say!
>>
>
> This is more difficult than it sounds. :) Newtypes are implemented via 
> coercions in Core, and coercions are inherently bidirectional. The 
> appearance of an existential in this way requires one-way conversions, 
> which are currently not supported. So, to get what you want, we'd have 
> to modify the core language as in the existentials paper, along with 
> some extra work to automatically add `pack` and `open` -- rather 
> similar to the type inference described in the existentials paper. The 
> bottom line for me is that this seems just as hard as implementing the 
> whole thing, so I see no value in having the stepping stone. If we 
> always wrote out the newtype constructor, then maybe we could use its 
> appearance to guide the `pack` and `open` but we don't: sometimes, we 
> just use `coerce`. So I really don't think this is any easier than 
> implementing the paper as written. Once that's done, we can come back 
> and add this new feature relatively easily (I think).
Makes sense. That route is indeed harder than I was thinking.

There was a time before coercions when I gather newtypes were mostly 
implemented with a special-case lowering to STG. I imagine it would be 
quite easy to implement this that way, but that door is now mostly shut.

I had a few more thoughts, but I am not sure if I should trouble the 
list with them as they are still a bit rough around the edges. Suffice 
to say, I hope we could sort of "smooth out" the feature space a bit so 
that these and GADTs are not completely different, e.g. I think all 
existential variables can be projected like this (I opened #20337 for 
this), and perhaps connect GADTs to the existentials with evidence you 
mention in the "future work" section, so "true" GADTs also have 
structural counterparts in a similar way.

Happy to just continue mulling on it until a proposal or something, 
however. :)

John

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210908/44be3355/attachment.html>

From Gergo.Erdi at sc.com  Wed Sep  8 09:54:31 2021
From: Gergo.Erdi at sc.com (Erdi, Gergo)
Date: Wed, 8 Sep 2021 09:54:31 +0000
Subject: Registering orphan instances and filling the `deps` field of
 ModIface [RE: Where (else) do I need to register instances from loaded
 modules?]
Message-ID: <HK0PR03MB48678DE1485CD0707502F131E9D49@HK0PR03MB4867.apcprd03.prod.outlook.com>

PUBLIC

Hi,

Any other ideas I could try?

Gergo

-----Original Message-----
From: Erdi, Gergo 
Sent: Friday, August 27, 2021 2:42 PM
To: 'Matthew Pickering' <matthewtpickering at gmail.com>
Cc: 'GHC' <ghc-devs at haskell.org>; 'gergo at erdi.hu' <gergo at erdi.hu>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>
Subject: Registering orphan instances and filling the `deps` field of ModIface [RE: Where (else) do I need to register instances from loaded modules?]

PUBLIC

Re: #1., I've simplified the code by removing all the Unit shuffling: everything is now in mainUnit. Unfortunately, this doesn't change anything other than cutting a cool 100 lines from the MWE code; I'm attaching the new smaller & simpler version. It is based on GHC 9.0.1 with an extra patch to export the first half of mkIfaceTc, just before calling mkFullIface. With this change, it really should be filling everything in ModIface the same way as the real GHC (so e.g. `deps` should be properly filled), modulo the fingerprints. 

Unfortunately, the `deps` field of the `ModIface`s created this way still doesn't contain any orphan instance modules: `Class.src`'s deps are empty (which is what I'd expect), `Instance.src` has a dependency in dep_mods (but not in dep_orphs) on `Class {-# SORUCE #-}` (I'm not sure why it's marked as a source import when `Class` module was already loaded into the module providers map before typechecking `Instance`...), and of course `Use.src` still fails so I can't get to its dependencies.

It is confusing to me that the problem should be because of the dep_orphs, because `Instance.src` does NOT use any orphan instances from `Class.src; merely, it provides a new orphan instance for other modules to depend on. Can someone explain this part to me?

Thanks,
	Gergo

-----Original Message-----
From: Erdi, Gergo 
Sent: Wednesday, August 25, 2021 10:37 AM
To: Matthew Pickering <matthewtpickering at gmail.com>
Cc: GHC <ghc-devs at haskell.org>; gergo at erdi.hu; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>
Subject: Re: Where (else) do I need to register instances from loaded modules?

Hi,

Thanks Matt! Unfortunately, I couldn't get it working even with these suggestions.

1. I changed registerModule so that it only changes the HPT when toUnitId (moduleUnit mod) == homeUnitId (hsc_dflags env). But I'm pretty sure this will be vacuously true, since the unit of the module comes from prepareModule setting the home unit before the call to `downsweep`.

TBH I don't understand the whole idea behind the home package, and maybe all my pain comes from misusing packages in the first place. As you can see in the original code I've sent, I'm loading modules from different units; in my real use case, I'd even like to do things like load module1 from unit1, then module2 from unit2, then module3 from unit1 again -- basically just using the units as a cheap way to classify modules so that later downstream, I can make decisions based on the unit of the module of the source of a given definition. Should I give up on this idea, is this going to lead to problems later on?

2. I also changed mkModIface to fill in the `deps` and while I'm at it, the `usages` field, by copy-pasting more code from mkIfaceTc. I'm really starting to think that I'd be better off just changing mkIfaceTc to return a PartialModIface... Anyway, the new definitions of `deps` and `usages` are:

    let pluginModules = map lpModule (cachedPlugins (hsc_dflags hsc_env))
        unit_id = homeUnitId (hsc_dflags hsc_env)
    deps <- mkDependencies
        unit_id (map mi_module pluginModules) tcg

    dep_files <- readIORef dependent_files
    let used_names = mkUsedNames tcg
    usages <- mkUsageInfo hsc_env this_mod (imp_mods imports) used_names
              dep_files merged pluginModules

With these changes applied, I am still seeing the same error. Printing shows that `dep_orphs deps` is empty for Instance.src; further tracing, in turn, shows that `imp_orphs . tcg_imports $ tcg` is also empty!

I have also found the wording in the documentation regarding dep_orphs confusing: it sounds like, in my case, this would mean that Use has a dependency on Instance, because it is using an orphan instance from there. But my problems are way before I am making a ModIface for Use, since it fails already during typechecking. But you saying the problem is the incomplete deps in ModIface must mean that dep_orphs needs to be filled in the ModIface for Instance (or, oh god I hope not, Class), right?

If I sound somewhat confused, rest assured that in reality I am even MORE confused!

Any further advice welcome,
	Gergo

-----Original Message-----
From: Matthew Pickering <matthewtpickering at gmail.com> 
Sent: Tuesday, August 24, 2021 5:16 PM
To: Erdi, Gergo <Gergo.Erdi at sc.com>
Cc: GHC <ghc-devs at haskell.org>; gergo at erdi.hu; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>
Subject: [External] Re: Where (else) do I need to register instances from loaded modules?


Hi Gergo,

Some things I noticed:

* The HPT only contains modules from the current home unit. It seems that registerModule ignores this so you break this invariant.
* It seems that `Instance` contains an orphan, but you don't fill in the `deps` field when making a `ModIface`, which should contain all the orphan modules below the module you are compiling.

I think that if you fill in `deps` correctly then things may work out.
`Dependencies` eventually becomes `ImportAvail`, which gets consulted by `tcVisibleOrphanMods` which is called in `tcGetInstEnvs` which is called by `matchInstEnv`.

Matt

This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations

Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm

Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer.

Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. 

Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same.

Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Main.hs
Type: application/octet-stream
Size: 8094 bytes
Desc: Main.hs
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210908/183b95d5/attachment.obj>

From lists at richarde.dev  Thu Sep  9 17:49:11 2021
From: lists at richarde.dev (Richard Eisenberg)
Date: Thu, 9 Sep 2021 17:49:11 +0000
Subject: new label: diagnostic quality
In-Reply-To: <CAPq5PvJ7zCAPba4iTvnajFzugwDS_Xjsc-OtQyHwiMt+Q-bY=w@mail.gmail.com>
References: <010f017bc1984523-d9153075-2f0b-47e8-9bd8-b144650848d1-000000@us-east-2.amazonses.com>
 <CAPq5PvLo2d9t3Q5Ma_aiDyQ0KcvC-YgAmu7jFLCRZVfr4m6s-g@mail.gmail.com>
 <CAPq5PvJ7zCAPba4iTvnajFzugwDS_Xjsc-OtQyHwiMt+Q-bY=w@mail.gmail.com>
Message-ID: <010f017bcbad81c0-bbacb1e9-9adc-43fa-8fc4-6c471bff18c6-000000@us-east-2.amazonses.com>

Great idea! See https://github.com/haskellfoundation/tech-proposals/issues/12#issuecomment-916310039, which will eventually work its way into a more proper proposal.

Thanks,
Richard

> On Sep 7, 2021, at 9:04 PM, Chris Smith <cdsmith at gmail.com> wrote:
> 
> Oops, I meant to include this URL: https://github.com/google/codeworld/issues?q=label%3Aerror-message+%22code.world%2Fhaskell%22 <https://github.com/google/codeworld/issues?q=label%3Aerror-message+%22code.world%2Fhaskell%22> is the full list (including closed issues) of cases where someone has reported a misleading error message at code.world/haskell
> 
> On Tue, Sep 7, 2021 at 9:03 PM Chris Smith <cdsmith at gmail.com <mailto:cdsmith at gmail.com>> wrote:
> Hi Richard,
> 
> One thing that I've done with CodeWorld for years now was to integrate reporting poor error messages into the core workflow by adding a report button on the compiler output directly in the tool.  This integration files github issues with a special tag for error messages.  Because CodeWorld is targeted at younger users and already rewrites error messages, most of the reports I get from CodeWorld itself are not directly applicable to GHC.  Here's a list of cases where someone clicked that button at code.world/haskell, but I usually close them with no action unless there's something egregious because I want code.world/haskell to be an authentic GHC experience.  In any case, I strongly suggest looking into similar tooling for more standard Haskell environments with editors like VSCode or Emacs.  I don't know if this could be done in HLS, or would need to be separately implemented for each editor.  I'll happily offer to implement a CodeWorld endpoint for filing GitLab error message reports similar to this which such integrations could post to.  The lower you can make the friction to report a misleading message, the more likely you'll get authentic reports from users rather than idle issue-theorizing.
> 
> These will probably need some kind of a triage process to turn them into actionable issues, and perhaps in my experience to just close 75% of them where it's unclear what was meant or the user just wanted to gripe.  (That may have something to do with the fact that my users have often procrastinated on their homework, though...)  But that triage process is something that you could easily ask for help on.  I would be willing to comb through reports now and then, for example, and look for opportunities to do better.
> 
> On Tue, Sep 7, 2021 at 2:51 PM Richard Eisenberg <lists at richarde.dev <mailto:lists at richarde.dev>> wrote:
> Hi devs,
> 
> I've just created a new label in GitLab, called "diagnostic quality". (Diagnostics = errors ∪ warnings.) I intend for this label to be used when the report is about the quality of an error message or warning. That is, the existing message is not wrong, per se, but it's somehow not as helpful as it can be to users. I think it's helpful to separate out such issues from those describing error messages that are just wrong.
> 
> In particular, if you spot a ticket arising from https://github.com/haskell/error-messages <https://github.com/haskell/error-messages>, you may want to give it the "diagnostic quality" label.
> 
> If you disagree with the choice to make this label, please push back!
> 
> Thanks,
> Richard
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs <http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210909/7badd704/attachment.html>

From liuyiyun at terpmail.umd.edu  Fri Sep 10 19:43:50 2021
From: liuyiyun at terpmail.umd.edu (Yiyun Liu)
Date: Fri, 10 Sep 2021 15:43:50 -0400
Subject: How to set idScope properly when converting LhsExpr GhcPs to
 CoreExpr?
In-Reply-To: <010f017b9dac8553-3728260b-a26f-4ecf-a5c2-0de232b711c1-000000@us-east-2.amazonses.com>
References: <d022261b-cb2b-2ca2-cee2-27fa6b237d97@terpmail.umd.edu>
 <010f017b9dac8553-3728260b-a26f-4ecf-a5c2-0de232b711c1-000000@us-east-2.amazonses.com>
Message-ID: <79474202-b784-9b6f-4b0c-4cf609cca2d8@terpmail.umd.edu>

Hi Richard,

Thanks for the reply. The problem has something to do with how LH 
currently handles symbols. Internally, LH uses the idScope field to 
identify whether a symbol is exported and handles them differently (e.g. 
erase the unique id only when the symbol is exported). Ideally, we want 
LH to be aware of the case where an exported Id is labeled as local but 
it's probably easier to let GHC set the idScope as if we were accessing 
from a different module.

We tried using tidyExpr to turn the LocalIds into GlobalIds 
(https://github.com/jprider63/ghc-elaboration-test/blob/49b33d42bdefb597ac8fab49bb4cea4969c929b3/src/Test/Plugin.hs#L101) 
but the output expression still shows lid for QQQ. Are there any flags 
we need to set to "trick" GHC into setting those ids as globals?

Thanks,

-Yiyun

On 8/31/21 3:25 PM, Richard Eisenberg wrote:
> I took a look at this today. I found
>
>> Note [GlobalId/LocalId]
>> ~~~~~~~~~~~~~~~~~~~~~~~
>> A GlobalId is
>>    * always a constant (top-level)
>>    * imported, or data constructor, or primop, or record selector
>>    * has a Unique that is globally unique across the whole
>>      GHC invocation (a single invocation may compile multiple modules)
>>    * never treated as a candidate by the free-variable finder;
>>          it's a constant!
>>
>> A LocalId is
>>    * bound within an expression (lambda, case, local let(rec))
>>    * or defined at top level in the module being compiled
>>    * always treated as a candidate by the free-variable finder
>>
>> After CoreTidy, top-level LocalIds are turned into GlobalIds
> This suggests that the output you're seeing -- with "lid" in QQQ and "gid" in PPP -- is correct. What has drawn your attention to this problem? Is there something else that's misbehaving?
>
> Thanks,
> Richard
>
>> On Aug 25, 2021, at 4:16 PM, Yiyun Liu <liuyiyun at terpmail.umd.edu> wrote:
>>
>> Hi all,
>>
>> We have a function in Liquidhaskell's code base for converting
>> refinement expressions into GHC's core expressions using the GHC API.
>> Its implementation is based on tcRnExpr
>> <https://gitlab.haskell.org/ghc/ghc/-/blob/ghc-8.10.6-release/compiler/typecheck/TcRnDriver.hs#L2358>except
>> we keep the typechecked expression and desugar it before returning.
>>
>> However, our function fails to set the idScope information properly when
>> LiquidHaskell is invoked as a plugin (installed as a
>> typeCheckResultAction). This repo
>> <https://github.com/yiyunliu/ghc-elaboration-test>contains a minimal
>> example that demonstrates the issue. In the CoreExpr returned by
>> elabRnExpr
>> <https://github.com/yiyunliu/ghc-elaboration-test/blob/cb7397884b7949e6b7fb6617ded10e86ee5dcad2/src/Test/Plugin.hs#L71>,
>> every symbol from the current module that's being compiled is labeled as
>> local. The README.md file shows how the exported symbol testPlus is
>> marked as local in QQQ.hs, the module where it's defined.
>>
>> Is there a function we should call to set the idScope information properly?
>>
>> Thanks,
>>
>> -Yiyun
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From simonpj at microsoft.com  Mon Sep 13 11:31:29 2021
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 13 Sep 2021 11:31:29 +0000
Subject: Resuming normal service
Message-ID: <VI1PR83MB02693A2F69EB8691CA78FE93ADD99@VI1PR83MB0269.EURPRD83.prod.outlook.com>

Friends
What with the summer holidays and leaving MSR, I've been rather preoccupied and have done almost nothing on GHC for the last six weeks.
I'm getting back in action now - I hope.  But my GHC backlog is huge.  If you are waiting on some response from me, please feel free to ping me.
Simon

PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com<mailto:simonpj at microsoft.com> will cease to work.  Use simon.peytonjones at gmail.com<mailto:simon.peytonjones at gmail.com> instead.  (For now, it just forwards to simonpj at microsoft.com.)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210913/953b430b/attachment.html>

From simonpj at microsoft.com  Tue Sep 14 11:56:48 2021
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Tue, 14 Sep 2021 11:56:48 +0000
Subject: Documenting GHC: blogs, wiki pages, Notes, Haddocks, etc
Message-ID: <VI1PR83MB02695FEC9ED49208D5D9F6FAADDA9@VI1PR83MB0269.EURPRD83.prod.outlook.com>

Alfredo writes (below for full thread)

That is a deceptively simple question you ask there :-) I don't have a strong view myself, but I can offer the perspective of somebody who was been for a long time on the "other side of the trenches" (i.e. working Haskell programmer, not necessarily working GHC programmer):

* Blog post: yes, it's true that is a snapshot, and it's true that is not under GHC's gitlab umbrella, so I wouldn't treat it as a reliable source of documentation (for the reasons you also explain) but it's surely a good testament that "at this point in time, for this particular GHC commit, things were this way);

* The wiki page: in the past, when I wanted to learn more about some GHC feature, Google would point me to the relevant Wiki page on the GHC repo describing such a feature, but I have to say I have almost always dismissed it, because everybody knows Wikis are seldomly up-to-date :) In order for a Wiki page to work we would have to at least add a banner at the top that states this can be trusted as a reliable source of information, and offer in the main section the current, up-to-date design. We can still offer the historical breakdown of the designs in later sections, as it's still valuable info to keep;

* GHC notes: I have always considered GHC notes a double-edge sword -- from one side they are immensely useful when navigating the source code, but these won't be rendered in the Hackage's haddocks, and this is not helpful for GHC-the-library users willing to understand how to use (or which is the semantic of) a particular type (sure, one can click "Show Source" on Hackage but it's an annoying extra step to do just to hunt for notes). We already have Notes for this work in strategic places -- even better, we have proper Haddock comments for things like "Severity vs DiagnosticReason" , e.g. https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255320972%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WU2dKu2Q%2FFdwntJ2h%2F6zO1Ic01c9o0VhZc5JrE0AurY%3D&reserved=0> .


Yes Haddock doesn't understand Notes but that's a deficiency in Haddock!  There so much in GHC that simply does not fit well with the Haddocks attached to a particular data decl or function.  We need Notes to explain how all the moving parts fit together, and to point to them.

Even better, we have proper Haddock comments for things like "Severity vs DiagnosticReason"

But I don't think this is better - I think it is significantly worse!   In the case you cite, the Haddock is about DiagnosticReason, and mentions Severity only incidentally.  I bet that the Haddock for Severity doesn't refer to this.   Nor is there a clear "Note [Severity vs DiagnosticReason]" title that bits of code across GHC can refer to by saying "See Note [Severity vs DiagnosticReason]".   It's far less satisfactory (to me) than a single Note that

  *   covers just one topic (the difference between Severity and DiagnosticReason, rather than fully describing either
  *   can be pointed to symmetrically from both Severity and DiagnosticReason
  *   can be pointed to by many other bits of code

The way it is better is that today's Haddock doesn't understand Notes.  But we could fix that if we were minded to.

Returning to how to document the error-message architecture, if you'd prefer to use a Note than a wiki page, that's fine.  But please write that Overview Note that explains all the pieces, points to them one by one.  And then copiously refer to that Note from all those places, so people will update it.

Hopefully as the time goes by the new design will "spread" across all the different peers working on GHC, and it will become "second nature".

I really don't think that will happen unless there is a Note that explains what the new design is!  Lacking this explicit design, everyone will infer their own mental model of how it all works from sundry scattered clues - and those mental models will differ.   So instead of one thing "spreading"  a dozen subtly different things will spread.  And then the next one, confused by these slightly different clues, will be even less coherent.

Let's have one, fully-explicit version of The Plan that we constantly refer to.

cc'ing ghc-devs because we must constantly question and refine how we describe and document GHC.

Simon

PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com<mailto:simonpj at microsoft.com> will cease to work.  Use simon.peytonjones at gmail.com<mailto:simon.peytonjones at gmail.com> instead.  (For now, it just forwards to simonpj at microsoft.com.)

From: Alfredo Di Napoli <alfredo.dinapoli at gmail.com>
Sent: 26 August 2021 07:25
To: Simon Peyton Jones <simonpj at microsoft.com>
Cc: rae at richarde.dev
Subject: Re: [Haskell Community] [Links] [Well-Typed Blog] The new GHC diagnostic infrastructure

Hello Simon!

On Wed, 25 Aug 2021 at 13:36, Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
Alfredo

Thanks for all the work you are doing on GHC's error message infrastructure.  Your blog post gives a great overview.

Thanks, and I am glad you enjoyed it :)


As you know I'm very keen for GHC to have a Note or wiki page that gives a solid, up-to-date overview of all the moving parts.  (NOT the design alternatives, nor the time sequence; just the outcome.)  This is incredibly useful for our future selves; and it helps ensure that people understand (say) the difference between Severity and DiagnosticReason, and use them correctly.

So the question is: where is the canonical overview?  It could be

  *   Your blog post below. But that is a snapshot... you aren't going to go back to edit it as the design evolves.  And it's not in the repo.
  *   The wiki page: https://gitlab.haskell.org/ghc/ghc/-/wikis/Errors-as-(structured)-values<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fwikis%2FErrors-as-(structured)-values&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255310976%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=A%2FyWqfqPWPYUk3EpaorYP29JvLIhgcdSdcYceFIKvhc%3D&reserved=0>.  But it's hard to keep up to date (it was last edited 3 months ago).
  *   Note(s) in the code.  We seem to use this increasingly, and it has the great merit of being part of the source code itself.  But then we need clear pointer to the canonical overview Notes, and need to make sure they are up to date.

I'm not advocating any particular path here... just wanting to be sure that we end up with a good overview somewhere! What is your view?


TL;DR Probably a combo of a well-written (and up-to-date Wiki) plus some carefully added Notes (and Haddock comments) in GHC might do the trick.

That is a deceptively simple question you ask there :-) I don't have a strong view myself, but I can offer the perspective of somebody who was been for a long time on the "other side of the trenches" (i.e. working Haskell programmer, not necessarily working GHC programmer):

* Blog post: yes, it's true that is a snapshot, and it's true that is not under GHC's gitlab umbrella, so I wouldn't treat it as a reliable source of documentation (for the reasons you also explain) but it's surely a good testament that "at this point in time, for this particular GHC commit, things were this way);

* The wiki page: in the past, when I wanted to learn more about some GHC feature, Google would point me to the relevant Wiki page on the GHC repo describing such a feature, but I have to say I have almost always dismissed it, because everybody knows Wikis are seldomly up-to-date :) In order for a Wiki page to work we would have to at least add a banner at the top that states this can be trusted as a reliable source of information, and offer in the main section the current, up-to-date design. We can still offer the historical breakdown of the designs in later sections, as it's still valuable info to keep;

* GHC notes: I have always considered GHC notes a double-edge sword -- from one side they are immensely useful when navigating the source code, but these won't be rendered in the Hackage's haddocks, and this is not helpful for GHC-the-library users willing to understand how to use (or which is the semantic of) a particular type (sure, one can click "Show Source" on Hackage but it's an annoying extra step to do just to hunt for notes). We already have Notes for this work in strategic places -- even better, we have proper Haddock comments for things like "Severity vs DiagnosticReason" , e.g. https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255320972%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WU2dKu2Q%2FFdwntJ2h%2F6zO1Ic01c9o0VhZc5JrE0AurY%3D&reserved=0> .

So, in practical terms, I suggest we (I) give the Wiki a little overhaul to add at the top the current design (or anything not captured directly in GHC's source code) and I will keep an eye on the GHC notes and Haddock comments to see if there is anything worth adding. Hopefully as the time goes by the new design will "spread" across all the different peers working on GHC, and it will become "second nature".

Hope it helps, and sorry for the long ramble!

Alfredo


Thanks

Simon

From: Alfredo Di Napoli via Haskell Community <discourse at haskell.org<mailto:discourse at haskell.org>>
Sent: 23 August 2021 11:26
To: Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
Subject: [Haskell Community] [Links] [Well-Typed Blog] The new GHC diagnostic infrastructure

[Image removed by sender.]
adinapoli<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Fu%2Fadinapoli&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255330973%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PgX4crGMBTWVwI2UMcq%2BIFDZ0dDr%2FRWNYZdV%2Fqi8mX8%3D&reserved=0>
August 23
[Image removed by sender.]well-typed.com<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255340965%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X51rdrGoKmUBPB8upLVL69LyInf%2BsYYQqM%2Fd4PnLnGQ%3D&reserved=0>
Error! Filename not specified.
The new GHC diagnostic infrastructure - Well-Typed: The Haskell Consultants<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255340965%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X51rdrGoKmUBPB8upLVL69LyInf%2BsYYQqM%2Fd4PnLnGQ%3D&reserved=0>
________________________________

Visit Topic<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Ft%2Fwell-typed-blog-the-new-ghc-diagnostic-infrastructure%2F2918%2F1&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255350960%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tOfAGO5BbhanwBDgMA6eqpgKCLcLTtkum8QOuMsROdc%3D&reserved=0> or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, click here<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Femail%2Funsubscribe%2F962dfad7651b2ce3d7e30ba9267bdb857c77298d6fdec12626b65e014aaeee33&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255360954%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4616hEpSSUcOZ5zQYZMmEbF6mTJcIVKx2nlgA8ENsHM%3D&reserved=0>.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/220e2b28/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 823 bytes
Desc: image001.jpg
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/220e2b28/attachment.jpg>

From hecate at glitchbra.in  Tue Sep 14 12:29:42 2021
From: hecate at glitchbra.in (=?UTF-8?Q?H=c3=a9cate?=)
Date: Tue, 14 Sep 2021 14:29:42 +0200
Subject: Documenting GHC: blogs, wiki pages, Notes, Haddocks, etc
In-Reply-To: <VI1PR83MB02695FEC9ED49208D5D9F6FAADDA9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
References: <VI1PR83MB02695FEC9ED49208D5D9F6FAADDA9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
Message-ID: <080562bf-45f7-b77c-9ea7-5120c4f46fbe@glitchbra.in>

 > today’s Haddock doesn’t understand Notes.  But we could fix that if 
we were minded to.

I may have missed an episode or two here but what prevents us from 
writing Notes as Named Chunks¹, write them where Haddock expects you to 
put documentation, and refer to them from the relevant spot in the code?
Viktor (in CC) has done a wonderful work at producing nice layouts for 
Haddocks in base, and we could learn a couple of lessons from his MRs.

---

Now, on the matter of improving Haddock to understand GHC's notes, I'd 
like to remind everyone that Haddock is currently understaffed in terms 
of feature development, and I would like to call to everyone with 
experience dealing with its codebase to give a hand in refactoring, 
dusting off and improving the code so that its maintainability is not 
jeopardised by people simply going elsewhere.
Our bus factor (or as I like to call it, circus factor), is quite 
terrifying considering the importance of the tool in our ecosystem.


¹ https://haskell-haddock.readthedocs.io/en/latest/markup.html#named-chunks

Le 14/09/2021 à 13:56, Simon Peyton Jones via ghc-devs a écrit :
>
> Alfredo writes (below for full thread)
>
> That is a deceptively simple question you ask there :-) I don't have a 
> strong view myself, but I can offer the perspective of somebody who 
> was been for a long time on the "other side of the trenches" (i.e. 
> working Haskell programmer, not necessarily working GHC programmer):
>
> * Blog post: yes, it's true that is a snapshot, and it's true that is 
> not under GHC's gitlab umbrella, so I wouldn't treat it as a reliable 
> source of documentation (for the reasons you also explain) but it's 
> surely a good testament that "at this point in time, for this 
> particular GHC commit, things were this way);
>
> * The wiki page: in the past, when I wanted to learn more about some 
> GHC feature, Google would point me to the relevant Wiki page on the 
> GHC repo describing such a feature, but I have to say I have almost 
> always dismissed it, because everybody knows Wikis are seldomly 
> up-to-date :) In order for a Wiki page to work we would have to at 
> least add a banner at the top that states this can be trusted as a 
> reliable source of information, and offer in the main section the 
> current, up-to-date design. We can still offer the historical 
> breakdown of the designs in later sections, as it's still valuable 
> info to keep;
>
> * GHC notes: I have always considered GHC notes a double-edge sword -- 
> from one side they are immensely useful when navigating the source 
> code, but these won't be rendered in the Hackage's haddocks, and this 
> is not helpful for GHC-the-library users willing to understand how to 
> use (or which is the semantic of) a particular type (sure, one can 
> click "Show Source" on Hackage but it's an annoying extra step to do 
> just to hunt for notes). We already have Notes for this work in 
> strategic places -- even better, we have proper Haddock comments for 
> things like "Severity vs DiagnosticReason" , e.g. 
> https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279 
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255320972%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WU2dKu2Q%2FFdwntJ2h%2F6zO1Ic01c9o0VhZc5JrE0AurY%3D&reserved=0> 
> .
>
> Yes Haddock doesn’t understand Notes but that’s a deficiency in 
> Haddock!  There so much in GHC that simply does not fit well with the 
> Haddocks attached to a particular data decl or function.  We need 
> Notes to explain how all the moving parts fit together, and to point 
> to them.
>
> Even better, we have proper Haddock comments for things like "Severity 
> vs DiagnosticReason"
>
> But I don’t think this is better – I think it is significantly 
> worse!   In the case you cite, the Haddock is about DiagnosticReason, 
> and mentions Severity only incidentally. I bet that the Haddock for 
> Severity doesn’t refer to this. Nor is there a clear “Note [Severity 
> vs DiagnosticReason]” title that bits of code across GHC can refer to 
> by saying “See Note [Severity vs DiagnosticReason]”.   It’s far less 
> satisfactory (to me) than a single Note that
>
>   * covers just *one topic* (the difference between Severity and
>     DiagnosticReason, rather than fully describing either
>   * can be *pointed to* symmetrically from both Severity and
>     DiagnosticReason
>   * can be *pointed to* by many other bits of code
>
> The way it is better is that today’s Haddock doesn’t understand 
> Notes.  But we could fix that if we were minded to.
>
>
> Returning to how to document the error-message architecture, if you’d 
> prefer to use a Note than a wiki page, that’s fine.  But please write 
> that Overview Note that explains all the pieces, points to them one by 
> one.  And then copiously refer to that Note from all those places, so 
> people will update it.
>
> _Hopefully as the time goes by the new design will "spread" across all 
> the different peers working on GHC, and it will become "second nature"._
>
> I really don’t think that will happen unless there is a Note that 
> explains what the new design is!  Lacking this explicit design, 
> everyone will infer their own mental model of how it all works from 
> sundry scattered clues – and those mental models will differ.   So 
> instead of one thing “spreading”  a dozen subtly different things will 
> spread.  And then the next one, confused by these slightly different 
> clues, will be even less coherent.
>
> Let’s have one, fully-explicit version of The Plan that we constantly 
> refer to.
>
> cc’ing ghc-devs because we must constantly question and refine how we 
> describe and document GHC.
>
> Simon
>
> PS: I am leaving Microsoft at the end of November 2021, at which point 
> simonpj at microsoft.com <mailto:simonpj at microsoft.com> will cease to 
> work.  Use simon.peytonjones at gmail.com 
> <mailto:simon.peytonjones at gmail.com> instead.  (For now, it just 
> forwards to simonpj at microsoft.com.)
>
> *From:*Alfredo Di Napoli <alfredo.dinapoli at gmail.com>
> *Sent:* 26 August 2021 07:25
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Cc:* rae at richarde.dev
> *Subject:* Re: [Haskell Community] [Links] [Well-Typed Blog] The new 
> GHC diagnostic infrastructure
>
> Hello Simon!
>
> On Wed, 25 Aug 2021 at 13:36, Simon Peyton Jones 
> <simonpj at microsoft.com> wrote:
>
>     Alfredo
>
>     Thanks for all the work you are doing on GHC’s error message
>     infrastructure.  Your blog post gives a great overview.
>
> Thanks, and I am glad you enjoyed it :)
>
>     As you know I’m very keen for GHC to have a Note or wiki page that
>     gives a solid, up-to-date overview of all the moving parts.  (NOT
>     the design alternatives, nor the time sequence; just the
>     outcome.)  This is incredibly useful for our future selves; and it
>     helps ensure that people understand (say) the difference between
>     Severity and DiagnosticReason, and use them correctly.
>
>     So the question is: where is the canonical overview?  It could be
>
>       * Your blog post below. But that is a snapshot… you aren’t going
>         to go back to edit it as the design evolves.  And it’s not in
>         the repo.
>       * The wiki page:
>         https://gitlab.haskell.org/ghc/ghc/-/wikis/Errors-as-(structured)-values
>         <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fwikis%2FErrors-as-(structured)-values&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255310976%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=A%2FyWqfqPWPYUk3EpaorYP29JvLIhgcdSdcYceFIKvhc%3D&reserved=0>.
>         But it’s hard to keep up to date (it was last edited 3 months
>         ago).
>       * Note(s) in the code.  We seem to use this increasingly, and it
>         has the great merit of being part of the source code itself. 
>         But then we need clear pointer to the canonical overview
>         Notes, and need to make sure they are up to date.
>
>     I’m not advocating any particular path here… just wanting to be
>     sure that we end up with a good overview somewhere! What is your view?
>
> _TL;DR Probably a combo of a well-written (and up-to-date Wiki) plus 
> some carefully added Notes (and Haddock comments) in GHC might do the 
> trick._
>
> That is a deceptively simple question you ask there :-) I don't have a 
> strong view myself, but I can offer the perspective of somebody who 
> was been for a long time on the "other side of the trenches" (i.e. 
> working Haskell programmer, not necessarily working GHC programmer):
>
> * Blog post: yes, it's true that is a snapshot, and it's true that is 
> not under GHC's gitlab umbrella, so I wouldn't treat it as a reliable 
> source of documentation (for the reasons you also explain) but it's 
> surely a good testament that "at this point in time, for this 
> particular GHC commit, things were this way);
>
> * The wiki page: in the past, when I wanted to learn more about some 
> GHC feature, Google would point me to the relevant Wiki page on the 
> GHC repo describing such a feature, but I have to say I have almost 
> always dismissed it, because everybody knows Wikis are seldomly 
> up-to-date :) In order for a Wiki page to work we would have to at 
> least add a banner at the top that states this can be trusted as a 
> reliable source of information, and offer in the main section the 
> current, up-to-date design. We can still offer the historical 
> breakdown of the designs in later sections, as it's still valuable 
> info to keep;
>
> * GHC notes: I have always considered GHC notes a double-edge sword -- 
> from one side they are immensely useful when navigating the source 
> code, but these won't be rendered in the Hackage's haddocks, and this 
> is not helpful for GHC-the-library users willing to understand how to 
> use (or which is the semantic of) a particular type (sure, one can 
> click "Show Source" on Hackage but it's an annoying extra step to do 
> just to hunt for notes). We already have Notes for this work in 
> strategic places -- even better, we have proper Haddock comments for 
> things like "Severity vs DiagnosticReason" , e.g. 
> https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279 
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255320972%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WU2dKu2Q%2FFdwntJ2h%2F6zO1Ic01c9o0VhZc5JrE0AurY%3D&reserved=0> 
> .
>
> _So, in practical terms, I suggest we (I) give the Wiki a little 
> overhaul to add at the top the current design (or anything not 
> captured directly in GHC's source code) and I will keep an eye on the 
> GHC notes and Haddock comments to see if there is anything worth 
> adding. Hopefully as the time goes by the new design will "spread" 
> across all the different peers working on GHC, and it will become 
> "second nature"._
>
> Hope it helps, and sorry for the long ramble!
>
> Alfredo
>
>     Thanks
>
>     Simon
>
>     *From:*Alfredo Di Napoli via Haskell Community
>     <discourse at haskell.org>
>     *Sent:* 23 August 2021 11:26
>     *To:* Simon Peyton Jones <simonpj at microsoft.com>
>     *Subject:* [Haskell Community] [Links] [Well-Typed Blog] The new
>     GHC diagnostic infrastructure
>
>     Image removed by sender.
>
>     	
>
>     *adinapoli*
>     <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Fu%2Fadinapoli&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255330973%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PgX4crGMBTWVwI2UMcq%2BIFDZ0dDr%2FRWNYZdV%2Fqi8mX8%3D&reserved=0>
>
>     August 23
>
>     Image removed by sender.*well-typed.com*
>     <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255340965%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X51rdrGoKmUBPB8upLVL69LyInf%2BsYYQqM%2Fd4PnLnGQ%3D&reserved=0>
>
>
>     *Error! Filename not specified.*
>
>
>           The new GHC diagnostic infrastructure - Well-Typed: The
>           Haskell Consultants
>           <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255340965%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X51rdrGoKmUBPB8upLVL69LyInf%2BsYYQqM%2Fd4PnLnGQ%3D&reserved=0>
>
>     ------------------------------------------------------------------------
>
>     *Visit Topic*
>     <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Ft%2Fwell-typed-blog-the-new-ghc-diagnostic-infrastructure%2F2918%2F1&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255350960%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tOfAGO5BbhanwBDgMA6eqpgKCLcLTtkum8QOuMsROdc%3D&reserved=0>
>     or reply to this email to respond.
>
>     You are receiving this because you enabled mailing list mode.
>
>     To unsubscribe from these emails, *click here*
>     <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Femail%2Funsubscribe%2F962dfad7651b2ce3d7e30ba9267bdb857c77298d6fdec12626b65e014aaeee33&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255360954%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4616hEpSSUcOZ5zQYZMmEbF6mTJcIVKx2nlgA8ENsHM%3D&reserved=0>.
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-- 
Hécate ✨
🐦: @TechnoEmpress
IRC: Hecate
WWW:https://glitchbra.in
RUN: BSD
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/02c9daf8/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 823 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/02c9daf8/attachment.jpg>

From arifordsham at gmail.com  Tue Sep 14 12:38:30 2021
From: arifordsham at gmail.com (Ari Fordsham)
Date: Tue, 14 Sep 2021 13:38:30 +0100
Subject: More type safety in Core?
Message-ID: <CAPQL5pK=to19tuJ5SMMvzWdxgR8JhFmh1r0qBzmf=r_GjXXZnQ@mail.gmail.com>

I don't know if this is the right forum for this, I apologise if I'm
intruding...

Are there any plans to use the type system to enforce safety in Core, via
e.g. GADTs? This would replace much of core-lint with static checking.

Conal Eliottt has done something similar in a blog post (
http://conal.net/blog/posts/overloading-lambda#:~:text=Haskell%20source%20language.-,I,-originally%20intended%20to)
and it seems relatively straightforward.

This would be especially beneficial to those working at the cutting edge of
GHC features, statically ensuring their Core manipulations are correct. I
would be surprised if existing compiler bugs wouldn't be found while
implementing this.

What would the performance impact be? would using GADTs incur extra
overhead? I'd assume you'd save something by lugging around less type
information in Core.

Ari Fordsham
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/a329d410/attachment.html>

From arifordsham at gmail.com  Tue Sep 14 12:49:55 2021
From: arifordsham at gmail.com (Ari Fordsham)
Date: Tue, 14 Sep 2021 13:49:55 +0100
Subject: More type safety in Core?
In-Reply-To: <CAPQL5pK=to19tuJ5SMMvzWdxgR8JhFmh1r0qBzmf=r_GjXXZnQ@mail.gmail.com>
References: <CAPQL5pK=to19tuJ5SMMvzWdxgR8JhFmh1r0qBzmf=r_GjXXZnQ@mail.gmail.com>
Message-ID: <CAPQL5pJWNgH28CSEPKWxTDyY+C1EmasZ1sPzfmv_KVtjCc11jg@mail.gmail.com>

Source for Eliott:
https://github.com/conal/lambda-ccc/blob/master/src/LambdaCCC/Lambda.hs

AF


On Tue, 14 Sept 2021 at 13:38, Ari Fordsham <arifordsham at gmail.com> wrote:

> I don't know if this is the right forum for this, I apologise if I'm
> intruding...
>
> Are there any plans to use the type system to enforce safety in Core, via
> e.g. GADTs? This would replace much of core-lint with static checking.
>
> Conal Eliottt has done something similar in a blog post (
> http://conal.net/blog/posts/overloading-lambda#:~:text=Haskell%20source%20language.-,I,-originally%20intended%20to)
> and it seems relatively straightforward.
>
> This would be especially beneficial to those working at the cutting edge
> of GHC features, statically ensuring their Core manipulations are correct.
> I would be surprised if existing compiler bugs wouldn't be found while
> implementing this.
>
> What would the performance impact be? would using GADTs incur extra
> overhead? I'd assume you'd save something by lugging around less type
> information in Core.
>
> Ari Fordsham
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/b73daf87/attachment.html>

From sgraf1337 at gmail.com  Tue Sep 14 14:00:23 2021
From: sgraf1337 at gmail.com (Sebastian Graf)
Date: Tue, 14 Sep 2021 16:00:23 +0200
Subject: Documenting GHC: blogs, wiki pages, Notes, Haddocks, etc
In-Reply-To: <080562bf-45f7-b77c-9ea7-5120c4f46fbe@glitchbra.in>
References: <VI1PR83MB02695FEC9ED49208D5D9F6FAADDA9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <080562bf-45f7-b77c-9ea7-5120c4f46fbe@glitchbra.in>
Message-ID: <CAAS+=P-iW-T58w2XH7Z3BBDLqxfRmpxjfAeRCswLd3capA8fFg@mail.gmail.com>

Hi,

I've been using Haddock's named chunks feature too when writing the docs
for selective lambda lifting.
This is the result:
https://hackage.haskell.org/package/ghc-8.10.2/docs/StgLiftLams.html, and
this is how the source code looks:
https://hackage.haskell.org/package/ghc-8.10.2/docs/src/StgLiftLams.html

I quite like it. As you can see, I enabled both the existing Notes workflow
and Haddock to work with it. It takes a bit of annoying extra work, though.
Ideally, Haddock would simply recognise the Note syntax directly or provide
a similar alternative.

And as far as linking is concerned: Sure, haddocks don't have a title to
refer to. But you can always link to them by linking to the syntactic
entity! For example, if I want to link to DiagnosticReason from Severity, I
can simply do so by saying "Also see 'Severity'".
I do admit this might not be enough info at the reference site to determine
whether the haddock linked to is relevant to the particular goal I want to
achieve. Also as Simon points out, there are Notes that don't have a clear
"owner".

Heck, even writing an unused binding `_Late_lambda_lifting_in_STG` and put
the haddocks there would work, I suppose. We could simply link to it with
'_Late_lambda_lifting_in_STG' from other haddocks.

My point is: If we managed to have something quite like named chunks, but
with a title and one place it gets rendered and then linked to (I don't
like that named chunks are inlined into every use site), we could probably
agree on using that.
Also I'd like to see the Notes rendered *regardless* of whether the thing
it is attached to is exported. That would make Notes a lot more accessible.

Sebastian


Am Di., 14. Sept. 2021 um 14:32 Uhr schrieb Hécate <hecate at glitchbra.in>:

> > today’s Haddock doesn’t understand Notes.  But we could fix that if we
> were minded to.
>
> I may have missed an episode or two here but what prevents us from writing
> Notes as Named Chunks¹, write them where Haddock expects you to put
> documentation, and refer to them from the relevant spot in the code?
> Viktor (in CC) has done a wonderful work at producing nice layouts for
> Haddocks in base, and we could learn a couple of lessons from his MRs.
>
> ---
>
> Now, on the matter of improving Haddock to understand GHC's notes, I'd
> like to remind everyone that Haddock is currently understaffed in terms of
> feature development, and I would like to call to everyone with experience
> dealing with its codebase to give a hand in refactoring, dusting off and
> improving the code so that its maintainability is not jeopardised by people
> simply going elsewhere.
> Our bus factor (or as I like to call it, circus factor), is quite
> terrifying considering the importance of the tool in our ecosystem.
>
>
> ¹
> https://haskell-haddock.readthedocs.io/en/latest/markup.html#named-chunks
>
> Le 14/09/2021 à 13:56, Simon Peyton Jones via ghc-devs a écrit :
>
> Alfredo writes (below for full thread)
>
>
>
> That is a deceptively simple question you ask there :-) I don't have a
> strong view myself, but I can offer the perspective of somebody who was
> been for a long time on the "other side of the trenches" (i.e. working
> Haskell programmer, not necessarily working GHC programmer):
>
>
>
> * Blog post: yes, it's true that is a snapshot, and it's true that is not
> under GHC's gitlab umbrella, so I wouldn't treat it as a reliable source of
> documentation (for the reasons you also explain) but it's surely a good
> testament that "at this point in time, for this particular GHC commit,
> things were this way);
>
>
>
> * The wiki page: in the past, when I wanted to learn more about some GHC
> feature, Google would point me to the relevant Wiki page on the GHC repo
> describing such a feature, but I have to say I have almost always dismissed
> it, because everybody knows Wikis are seldomly up-to-date :) In order for a
> Wiki page to work we would have to at least add a banner at the top that
> states this can be trusted as a reliable source of information, and offer
> in the main section the current, up-to-date design. We can still offer the
> historical breakdown of the designs in later sections, as it's still
> valuable info to keep;
>
>
>
> * GHC notes: I have always considered GHC notes a double-edge sword --
> from one side they are immensely useful when navigating the source code,
> but these won't be rendered in the Hackage's haddocks, and this is not
> helpful for GHC-the-library users willing to understand how to use (or
> which is the semantic of) a particular type (sure, one can click "Show
> Source" on Hackage but it's an annoying extra step to do just to hunt for
> notes). We already have Notes for this work in strategic places -- even
> better, we have proper Haddock comments for things like "Severity vs
> DiagnosticReason" , e.g.
> https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255320972%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WU2dKu2Q%2FFdwntJ2h%2F6zO1Ic01c9o0VhZc5JrE0AurY%3D&reserved=0>
> .
>
>
>
>
>
> Yes Haddock doesn’t understand Notes but that’s a deficiency in Haddock!
> There so much in GHC that simply does not fit well with the Haddocks
> attached to a particular data decl or function.  We need Notes to explain
> how all the moving parts fit together, and to point to them.
>
>
>
> Even better, we have proper Haddock comments for things like "Severity vs
> DiagnosticReason"
>
>
>
> But I don’t think this is better – I think it is significantly worse!   In
> the case you cite, the Haddock is about DiagnosticReason, and mentions
> Severity only incidentally.  I bet that the Haddock for Severity doesn’t
> refer to this.   Nor is there a clear “Note [Severity vs DiagnosticReason]”
> title that bits of code across GHC can refer to by saying “See Note
> [Severity vs DiagnosticReason]”.   It’s far less satisfactory (to me) than
> a single Note that
>
>    - covers just *one topic* (the difference between Severity and
>    DiagnosticReason, rather than fully describing either
>    - can be *pointed to* symmetrically from both Severity and
>    DiagnosticReason
>    - can be *pointed to* by many other bits of code
>
>
>
> The way it is better is that today’s Haddock doesn’t understand Notes.
> But we could fix that if we were minded to.
>
>
> Returning to how to document the error-message architecture, if you’d
> prefer to use a Note than a wiki page, that’s fine.  But please write that
> Overview Note that explains all the pieces, points to them one by one.  And
> then copiously refer to that Note from all those places, so people will
> update it.
>
>
>
> *Hopefully as the time goes by the new design will "spread" across all the
> different peers working on GHC, and it will become "second nature".*
>
>
>
> I really don’t think that will happen unless there is a Note that explains
> what the new design is!  Lacking this explicit design, everyone will infer
> their own mental model of how it all works from sundry scattered clues –
> and those mental models will differ.   So instead of one thing “spreading”
> a dozen subtly different things will spread.  And then the next one,
> confused by these slightly different clues, will be even less coherent.
>
>
>
> Let’s have one, fully-explicit version of The Plan that we constantly
> refer to.
>
>
>
> cc’ing ghc-devs because we must constantly question and refine how we
> describe and document GHC.
>
>
>
> Simon
>
>
>
> PS: I am leaving Microsoft at the end of November 2021, at which point
> simonpj at microsoft.com will cease to work.  Use simon.peytonjones at gmail.com
> instead.  (For now, it just forwards to simonpj at microsoft.com.)
>
>
>
> *From:* Alfredo Di Napoli <alfredo.dinapoli at gmail.com>
> <alfredo.dinapoli at gmail.com>
> *Sent:* 26 August 2021 07:25
> *To:* Simon Peyton Jones <simonpj at microsoft.com> <simonpj at microsoft.com>
> *Cc:* rae at richarde.dev
> *Subject:* Re: [Haskell Community] [Links] [Well-Typed Blog] The new GHC
> diagnostic infrastructure
>
>
>
> Hello Simon!
>
>
>
> On Wed, 25 Aug 2021 at 13:36, Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> Alfredo
>
>
>
> Thanks for all the work you are doing on GHC’s error message
> infrastructure.  Your blog post gives a great overview.
>
>
>
> Thanks, and I am glad you enjoyed it :)
>
>
>
>
>
> As you know I’m very keen for GHC to have a Note or wiki page that gives a
> solid, up-to-date overview of all the moving parts.  (NOT the design
> alternatives, nor the time sequence; just the outcome.)  This is incredibly
> useful for our future selves; and it helps ensure that people understand
> (say) the difference between Severity and DiagnosticReason, and use them
> correctly.
>
>
>
> So the question is: where is the canonical overview?  It could be
>
>    - Your blog post below. But that is a snapshot… you aren’t going to go
>    back to edit it as the design evolves.  And it’s not in the repo.
>    - The wiki page:
>    https://gitlab.haskell.org/ghc/ghc/-/wikis/Errors-as-(structured)-values
>    <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fwikis%2FErrors-as-(structured)-values&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255310976%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=A%2FyWqfqPWPYUk3EpaorYP29JvLIhgcdSdcYceFIKvhc%3D&reserved=0>.
>    But it’s hard to keep up to date (it was last edited 3 months ago).
>    - Note(s) in the code.  We seem to use this increasingly, and it has
>    the great merit of being part of the source code itself.  But then we need
>    clear pointer to the canonical overview Notes, and need to make sure they
>    are up to date.
>
>
>
> I’m not advocating any particular path here… just wanting to be sure that
> we end up with a good overview somewhere! What is your view?
>
>
>
>
>
> *TL;DR Probably a combo of a well-written (and up-to-date Wiki) plus some
> carefully added Notes (and Haddock comments) in GHC might do the trick.*
>
>
>
> That is a deceptively simple question you ask there :-) I don't have a
> strong view myself, but I can offer the perspective of somebody who was
> been for a long time on the "other side of the trenches" (i.e. working
> Haskell programmer, not necessarily working GHC programmer):
>
>
>
> * Blog post: yes, it's true that is a snapshot, and it's true that is not
> under GHC's gitlab umbrella, so I wouldn't treat it as a reliable source of
> documentation (for the reasons you also explain) but it's surely a good
> testament that "at this point in time, for this particular GHC commit,
> things were this way);
>
>
>
> * The wiki page: in the past, when I wanted to learn more about some GHC
> feature, Google would point me to the relevant Wiki page on the GHC repo
> describing such a feature, but I have to say I have almost always dismissed
> it, because everybody knows Wikis are seldomly up-to-date :) In order for a
> Wiki page to work we would have to at least add a banner at the top that
> states this can be trusted as a reliable source of information, and offer
> in the main section the current, up-to-date design. We can still offer the
> historical breakdown of the designs in later sections, as it's still
> valuable info to keep;
>
>
>
> * GHC notes: I have always considered GHC notes a double-edge sword --
> from one side they are immensely useful when navigating the source code,
> but these won't be rendered in the Hackage's haddocks, and this is not
> helpful for GHC-the-library users willing to understand how to use (or
> which is the semantic of) a particular type (sure, one can click "Show
> Source" on Hackage but it's an annoying extra step to do just to hunt for
> notes). We already have Notes for this work in strategic places -- even
> better, we have proper Haddock comments for things like "Severity vs
> DiagnosticReason" , e.g.
> https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255320972%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WU2dKu2Q%2FFdwntJ2h%2F6zO1Ic01c9o0VhZc5JrE0AurY%3D&reserved=0>
> .
>
>
>
> *So, in practical terms, I suggest we (I) give the Wiki a little overhaul
> to add at the top the current design (or anything not captured directly in
> GHC's source code) and I will keep an eye on the GHC notes and Haddock
> comments to see if there is anything worth adding. Hopefully as the time
> goes by the new design will "spread" across all the different peers working
> on GHC, and it will become "second nature".*
>
>
>
> Hope it helps, and sorry for the long ramble!
>
>
>
> Alfredo
>
>
>
>
>
> Thanks
>
>
>
> Simon
>
>
>
> *From:* Alfredo Di Napoli via Haskell Community <discourse at haskell.org>
> *Sent:* 23 August 2021 11:26
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Subject:* [Haskell Community] [Links] [Well-Typed Blog] The new GHC
> diagnostic infrastructure
>
>
>
> [image: Image removed by sender.]
>
> *adinapoli*
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Fu%2Fadinapoli&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255330973%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PgX4crGMBTWVwI2UMcq%2BIFDZ0dDr%2FRWNYZdV%2Fqi8mX8%3D&reserved=0>
> August 23
>
> [image: Image removed by sender.]*well-typed.com*
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255340965%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X51rdrGoKmUBPB8upLVL69LyInf%2BsYYQqM%2Fd4PnLnGQ%3D&reserved=0>
>
> *Error! Filename not specified.*
> The new GHC diagnostic infrastructure - Well-Typed: The Haskell Consultants
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255340965%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X51rdrGoKmUBPB8upLVL69LyInf%2BsYYQqM%2Fd4PnLnGQ%3D&reserved=0>
> ------------------------------
>
> *Visit Topic*
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Ft%2Fwell-typed-blog-the-new-ghc-diagnostic-infrastructure%2F2918%2F1&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255350960%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tOfAGO5BbhanwBDgMA6eqpgKCLcLTtkum8QOuMsROdc%3D&reserved=0>
> or reply to this email to respond.
>
> You are receiving this because you enabled mailing list mode.
>
> To unsubscribe from these emails, *click here*
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Femail%2Funsubscribe%2F962dfad7651b2ce3d7e30ba9267bdb857c77298d6fdec12626b65e014aaeee33&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255360954%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4616hEpSSUcOZ5zQYZMmEbF6mTJcIVKx2nlgA8ENsHM%3D&reserved=0>
> .
>
>
>
>
> _______________________________________________
> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> --
> Hécate ✨
> 🐦: @TechnoEmpress
> IRC: Hecate
> WWW: https://glitchbra.in
> RUN: BSD
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/8fd1b6c9/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 823 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/8fd1b6c9/attachment-0001.jpg>

From hecate at glitchbra.in  Tue Sep 14 14:25:05 2021
From: hecate at glitchbra.in (=?UTF-8?Q?H=c3=a9cate?=)
Date: Tue, 14 Sep 2021 16:25:05 +0200
Subject: Documenting GHC: blogs, wiki pages, Notes, Haddocks, etc
In-Reply-To: <CAAS+=P-iW-T58w2XH7Z3BBDLqxfRmpxjfAeRCswLd3capA8fFg@mail.gmail.com>
References: <VI1PR83MB02695FEC9ED49208D5D9F6FAADDA9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <080562bf-45f7-b77c-9ea7-5120c4f46fbe@glitchbra.in>
 <CAAS+=P-iW-T58w2XH7Z3BBDLqxfRmpxjfAeRCswLd3capA8fFg@mail.gmail.com>
Message-ID: <5735914a-f322-ac17-1b4e-23efd86b8943@glitchbra.in>

Hi,

The named chunks can be positioned through the use of the export list 
syntax:

module Foo
   ( main
   -- * Section name that will appear
   --
   -- $chunk
   )

This should produce a free section that is not linked to any exported item.
I see you're already using them though, so maybe I am understanding 
something else?

Le 14/09/2021 à 16:00, Sebastian Graf a écrit :
> Hi,
>
> I've been using Haddock's named chunks feature too when writing the 
> docs for selective lambda lifting.
> This is the result: 
> https://hackage.haskell.org/package/ghc-8.10.2/docs/StgLiftLams.html, 
> and this is how the source code looks: 
> https://hackage.haskell.org/package/ghc-8.10.2/docs/src/StgLiftLams.html
>
> I quite like it. As you can see, I enabled both the existing Notes 
> workflow and Haddock to work with it. It takes a bit of annoying extra 
> work, though. Ideally, Haddock would simply recognise the Note syntax 
> directly or provide a similar alternative.
>
> And as far as linking is concerned: Sure, haddocks don't have a title 
> to refer to. But you can always link to them by linking to the 
> syntactic entity! For example, if I want to link to DiagnosticReason 
> from Severity, I can simply do so by saying "Also see 'Severity'".
> I do admit this might not be enough info at the reference site to 
> determine whether the haddock linked to is relevant to the particular 
> goal I want to achieve. Also as Simon points out, there are Notes that 
> don't have a clear "owner".
>
> Heck, even writing an unused binding `_Late_lambda_lifting_in_STG` and 
> put the haddocks there would work, I suppose. We could simply link to 
> it with '_Late_lambda_lifting_in_STG' from other haddocks.
>
> My point is: If we managed to have something quite like named chunks, 
> but with a title and one place it gets rendered and then linked to (I 
> don't like that named chunks are inlined into every use site), we 
> could probably agree on using that.
> Also I'd like to see the Notes rendered *regardless* of whether the 
> thing it is attached to is exported. That would make Notes a lot more 
> accessible.
>
> Sebastian
>
>
> Am Di., 14. Sept. 2021 um 14:32 Uhr schrieb Hécate <hecate at glitchbra.in>:
>
>     > today’s Haddock doesn’t understand Notes.  But we could fix that
>     if we were minded to.
>
>     I may have missed an episode or two here but what prevents us from
>     writing Notes as Named Chunks¹, write them where Haddock expects
>     you to put documentation, and refer to them from the relevant spot
>     in the code?
>     Viktor (in CC) has done a wonderful work at producing nice layouts
>     for Haddocks in base, and we could learn a couple of lessons from
>     his MRs.
>
>     ---
>
>     Now, on the matter of improving Haddock to understand GHC's notes,
>     I'd like to remind everyone that Haddock is currently understaffed
>     in terms of feature development, and I would like to call to
>     everyone with experience dealing with its codebase to give a hand
>     in refactoring, dusting off and improving the code so that its
>     maintainability is not jeopardised by people simply going elsewhere.
>     Our bus factor (or as I like to call it, circus factor), is quite
>     terrifying considering the importance of the tool in our ecosystem.
>
>
>     ¹
>     https://haskell-haddock.readthedocs.io/en/latest/markup.html#named-chunks
>
>     Le 14/09/2021 à 13:56, Simon Peyton Jones via ghc-devs a écrit :
>>
>>     Alfredo writes (below for full thread)
>>
>>     That is a deceptively simple question you ask there :-) I don't
>>     have a strong view myself, but I can offer the perspective of
>>     somebody who was been for a long time on the "other side of the
>>     trenches" (i.e. working Haskell programmer, not necessarily
>>     working GHC programmer):
>>
>>     * Blog post: yes, it's true that is a snapshot, and it's true
>>     that is not under GHC's gitlab umbrella, so I wouldn't treat it
>>     as a reliable source of documentation (for the reasons you also
>>     explain) but it's surely a good testament that "at this point in
>>     time, for this particular GHC commit, things were this way);
>>
>>     * The wiki page: in the past, when I wanted to learn more about
>>     some GHC feature, Google would point me to the relevant Wiki page
>>     on the GHC repo describing such a feature, but I have to say I
>>     have almost always dismissed it, because everybody knows
>>     Wikis are seldomly up-to-date :) In order for a Wiki page to work
>>     we would have to at least add a banner at the top that states
>>     this can be trusted as a reliable source of information, and
>>     offer in the main section the current, up-to-date design. We can
>>     still offer the historical breakdown of the designs in later
>>     sections, as it's still valuable info to keep;
>>
>>     * GHC notes: I have always considered GHC notes a double-edge
>>     sword -- from one side they are immensely useful when navigating
>>     the source code, but these won't be rendered in the Hackage's
>>     haddocks, and this is not helpful for GHC-the-library users
>>     willing to understand how to use (or which is the semantic of) a
>>     particular type (sure, one can click "Show Source" on Hackage but
>>     it's an annoying extra step to do just to hunt for notes). We
>>     already have Notes for this work in strategic places -- even
>>     better, we have proper Haddock comments for things like "Severity
>>     vs DiagnosticReason" , e.g.
>>     https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279
>>     <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255320972%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WU2dKu2Q%2FFdwntJ2h%2F6zO1Ic01c9o0VhZc5JrE0AurY%3D&reserved=0>
>>     .
>>
>>     Yes Haddock doesn’t understand Notes but that’s a deficiency in
>>     Haddock!  There so much in GHC that simply does not fit well with
>>     the Haddocks attached to a particular data decl or function.  We
>>     need Notes to explain how all the moving parts fit together, and
>>     to point to them.
>>
>>     Even better, we have proper Haddock comments for things like
>>     "Severity vs DiagnosticReason"
>>
>>     But I don’t think this is better – I think it is significantly
>>     worse!   In the case you cite, the Haddock is about
>>     DiagnosticReason, and mentions Severity only incidentally.  I bet
>>     that the Haddock for Severity doesn’t refer to this.   Nor is
>>     there a clear “Note [Severity vs DiagnosticReason]” title that
>>     bits of code across GHC can refer to by saying “See Note
>>     [Severity vs DiagnosticReason]”.   It’s far less satisfactory (to
>>     me) than a single Note that
>>
>>       * covers just *one topic* (the difference between Severity and
>>         DiagnosticReason, rather than fully describing either
>>       * can be *pointed to* symmetrically from both Severity and
>>         DiagnosticReason
>>       * can be *pointed to* by many other bits of code
>>
>>     The way it is better is that today’s Haddock doesn’t understand
>>     Notes.  But we could fix that if we were minded to.
>>
>>
>>     Returning to how to document the error-message architecture, if
>>     you’d prefer to use a Note than a wiki page, that’s fine.  But
>>     please write that Overview Note that explains all the pieces,
>>     points to them one by one.  And then copiously refer to that Note
>>     from all those places, so people will update it.
>>
>>     _Hopefully as the time goes by the new design will "spread"
>>     across all the different peers working on GHC, and it will become
>>     "second nature"._
>>
>>     I really don’t think that will happen unless there is a Note that
>>     explains what the new design is!  Lacking this explicit design,
>>     everyone will infer their own mental model of how it all works
>>     from sundry scattered clues – and those mental models will
>>     differ.   So instead of one thing “spreading”  a dozen subtly
>>     different things will spread.  And then the next one, confused by
>>     these slightly different clues, will be even less coherent.
>>
>>     Let’s have one, fully-explicit version of The Plan that we
>>     constantly refer to.
>>
>>     cc’ing ghc-devs because we must constantly question and refine
>>     how we describe and document GHC.
>>
>>     Simon
>>
>>     PS: I am leaving Microsoft at the end of November 2021, at which
>>     point simonpj at microsoft.com <mailto:simonpj at microsoft.com> will
>>     cease to work.  Use simon.peytonjones at gmail.com
>>     <mailto:simon.peytonjones at gmail.com> instead.  (For now, it just
>>     forwards to simonpj at microsoft.com.)
>>
>>     *From:*Alfredo Di Napoli <alfredo.dinapoli at gmail.com>
>>     <mailto:alfredo.dinapoli at gmail.com>
>>     *Sent:* 26 August 2021 07:25
>>     *To:* Simon Peyton Jones <simonpj at microsoft.com>
>>     <mailto:simonpj at microsoft.com>
>>     *Cc:* rae at richarde.dev
>>     *Subject:* Re: [Haskell Community] [Links] [Well-Typed Blog] The
>>     new GHC diagnostic infrastructure
>>
>>     Hello Simon!
>>
>>     On Wed, 25 Aug 2021 at 13:36, Simon Peyton Jones
>>     <simonpj at microsoft.com> wrote:
>>
>>         Alfredo
>>
>>         Thanks for all the work you are doing on GHC’s error message
>>         infrastructure.  Your blog post gives a great overview.
>>
>>     Thanks, and I am glad you enjoyed it :)
>>
>>         As you know I’m very keen for GHC to have a Note or wiki page
>>         that gives a solid, up-to-date overview of all the moving
>>         parts.  (NOT the design alternatives, nor the time sequence;
>>         just the outcome.)  This is incredibly useful for our future
>>         selves; and it helps ensure that people understand (say) the
>>         difference between Severity and DiagnosticReason, and use
>>         them correctly.
>>
>>         So the question is: where is the canonical overview?  It could be
>>
>>           * Your blog post below. But that is a snapshot… you aren’t
>>             going to go back to edit it as the design evolves.  And
>>             it’s not in the repo.
>>           * The wiki page:
>>             https://gitlab.haskell.org/ghc/ghc/-/wikis/Errors-as-(structured)-values
>>             <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fwikis%2FErrors-as-(structured)-values&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255310976%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=A%2FyWqfqPWPYUk3EpaorYP29JvLIhgcdSdcYceFIKvhc%3D&reserved=0>.
>>             But it’s hard to keep up to date (it was last edited 3
>>             months ago).
>>           * Note(s) in the code.  We seem to use this increasingly,
>>             and it has the great merit of being part of the source
>>             code itself.  But then we need clear pointer to the
>>             canonical overview Notes, and need to make sure they are
>>             up to date.
>>
>>         I’m not advocating any particular path here… just wanting to
>>         be sure that we end up with a good overview somewhere! What
>>         is your view?
>>
>>     _TL;DR Probably a combo of a well-written (and up-to-date Wiki)
>>     plus some carefully added Notes (and Haddock comments) in GHC
>>     might do the trick._
>>
>>     That is a deceptively simple question you ask there :-) I don't
>>     have a strong view myself, but I can offer the perspective of
>>     somebody who was been for a long time on the "other side of the
>>     trenches" (i.e. working Haskell programmer, not necessarily
>>     working GHC programmer):
>>
>>     * Blog post: yes, it's true that is a snapshot, and it's true
>>     that is not under GHC's gitlab umbrella, so I wouldn't treat it
>>     as a reliable source of documentation (for the reasons you also
>>     explain) but it's surely a good testament that "at this point in
>>     time, for this particular GHC commit, things were this way);
>>
>>     * The wiki page: in the past, when I wanted to learn more about
>>     some GHC feature, Google would point me to the relevant Wiki page
>>     on the GHC repo describing such a feature, but I have to say I
>>     have almost always dismissed it, because everybody knows
>>     Wikis are seldomly up-to-date :) In order for a Wiki page to work
>>     we would have to at least add a banner at the top that states
>>     this can be trusted as a reliable source of information, and
>>     offer in the main section the current, up-to-date design. We can
>>     still offer the historical breakdown of the designs in later
>>     sections, as it's still valuable info to keep;
>>
>>     * GHC notes: I have always considered GHC notes a double-edge
>>     sword -- from one side they are immensely useful when navigating
>>     the source code, but these won't be rendered in the Hackage's
>>     haddocks, and this is not helpful for GHC-the-library users
>>     willing to understand how to use (or which is the semantic of) a
>>     particular type (sure, one can click "Show Source" on Hackage but
>>     it's an annoying extra step to do just to hunt for notes). We
>>     already have Notes for this work in strategic places -- even
>>     better, we have proper Haddock comments for things like "Severity
>>     vs DiagnosticReason" , e.g.
>>     https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279
>>     <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255320972%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WU2dKu2Q%2FFdwntJ2h%2F6zO1Ic01c9o0VhZc5JrE0AurY%3D&reserved=0>
>>     .
>>
>>     _So, in practical terms, I suggest we (I) give the Wiki a little
>>     overhaul to add at the top the current design (or anything not
>>     captured directly in GHC's source code) and I will keep an eye on
>>     the GHC notes and Haddock comments to see if there is anything
>>     worth adding. Hopefully as the time goes by the new design will
>>     "spread" across all the different peers working on GHC, and it
>>     will become "second nature"._
>>
>>     Hope it helps, and sorry for the long ramble!
>>
>>     Alfredo
>>
>>         Thanks
>>
>>         Simon
>>
>>         *From:*Alfredo Di Napoli via Haskell Community
>>         <discourse at haskell.org>
>>         *Sent:* 23 August 2021 11:26
>>         *To:* Simon Peyton Jones <simonpj at microsoft.com>
>>         *Subject:* [Haskell Community] [Links] [Well-Typed Blog] The
>>         new GHC diagnostic infrastructure
>>
>>         Image removed by sender.
>>
>>         	
>>
>>         *adinapoli*
>>         <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Fu%2Fadinapoli&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255330973%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PgX4crGMBTWVwI2UMcq%2BIFDZ0dDr%2FRWNYZdV%2Fqi8mX8%3D&reserved=0>
>>
>>         August 23
>>
>>         Image removed by sender.*well-typed.com*
>>         <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255340965%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X51rdrGoKmUBPB8upLVL69LyInf%2BsYYQqM%2Fd4PnLnGQ%3D&reserved=0>
>>
>>
>>         *Error! Filename not specified.*
>>
>>
>>               The new GHC diagnostic infrastructure - Well-Typed: The
>>               Haskell Consultants
>>               <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255340965%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X51rdrGoKmUBPB8upLVL69LyInf%2BsYYQqM%2Fd4PnLnGQ%3D&reserved=0>
>>
>>         ------------------------------------------------------------------------
>>
>>         *Visit Topic*
>>         <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Ft%2Fwell-typed-blog-the-new-ghc-diagnostic-infrastructure%2F2918%2F1&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255350960%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tOfAGO5BbhanwBDgMA6eqpgKCLcLTtkum8QOuMsROdc%3D&reserved=0>
>>         or reply to this email to respond.
>>
>>         You are receiving this because you enabled mailing list mode.
>>
>>         To unsubscribe from these emails, *click here*
>>         <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Femail%2Funsubscribe%2F962dfad7651b2ce3d7e30ba9267bdb857c77298d6fdec12626b65e014aaeee33&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255360954%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4616hEpSSUcOZ5zQYZMmEbF6mTJcIVKx2nlgA8ENsHM%3D&reserved=0>.
>>
>>
>>     _______________________________________________
>>     ghc-devs mailing list
>>     ghc-devs at haskell.org
>>     http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>     -- 
>     Hécate ✨
>     🐦: @TechnoEmpress
>     IRC: Hecate
>     WWW:https://glitchbra.in
>     RUN: BSD
>
>     _______________________________________________
>     ghc-devs mailing list
>     ghc-devs at haskell.org
>     http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-- 
Hécate ✨
🐦: @TechnoEmpress
IRC: Hecate
WWW:https://glitchbra.in
RUN: BSD
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/0156bef1/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 823 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/0156bef1/attachment.jpg>

From sgraf1337 at gmail.com  Tue Sep 14 14:37:16 2021
From: sgraf1337 at gmail.com (Sebastian Graf)
Date: Tue, 14 Sep 2021 16:37:16 +0200
Subject: Documenting GHC: blogs, wiki pages, Notes, Haddocks, etc
In-Reply-To: <5735914a-f322-ac17-1b4e-23efd86b8943@glitchbra.in>
References: <VI1PR83MB02695FEC9ED49208D5D9F6FAADDA9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <080562bf-45f7-b77c-9ea7-5120c4f46fbe@glitchbra.in>
 <CAAS+=P-iW-T58w2XH7Z3BBDLqxfRmpxjfAeRCswLd3capA8fFg@mail.gmail.com>
 <5735914a-f322-ac17-1b4e-23efd86b8943@glitchbra.in>
Message-ID: <CAAS+=P90RmZ1iHEF0Bn8iUAnFteFnT5BVNSSUYG+sCVwOj_3OA@mail.gmail.com>

What I don't like:

   - I have to give a section name to an otherwise unused $chunk
   - I can't refer to $chunk from other modules
   - Everywhere I reference $chunk in a haddock, it gets inlined instead of
   appearing with its title or inserting a link with the section name it is
   currently decoupled from.


Am Di., 14. Sept. 2021 um 16:25 Uhr schrieb Hécate <hecate at glitchbra.in>:

> Hi,
>
> The named chunks can be positioned through the use of the export list
> syntax:
>
> module Foo
>   ( main
>   -- * Section name that will appear
>   --
>   -- $chunk
>   )
>
> This should produce a free section that is not linked to any exported item.
> I see you're already using them though, so maybe I am understanding
> something else?
> Le 14/09/2021 à 16:00, Sebastian Graf a écrit :
>
> Hi,
>
> I've been using Haddock's named chunks feature too when writing the docs
> for selective lambda lifting.
> This is the result:
> https://hackage.haskell.org/package/ghc-8.10.2/docs/StgLiftLams.html, and
> this is how the source code looks:
> https://hackage.haskell.org/package/ghc-8.10.2/docs/src/StgLiftLams.html
>
> I quite like it. As you can see, I enabled both the existing Notes
> workflow and Haddock to work with it. It takes a bit of annoying extra
> work, though. Ideally, Haddock would simply recognise the Note syntax
> directly or provide a similar alternative.
>
> And as far as linking is concerned: Sure, haddocks don't have a title to
> refer to. But you can always link to them by linking to the syntactic
> entity! For example, if I want to link to DiagnosticReason from Severity, I
> can simply do so by saying "Also see 'Severity'".
> I do admit this might not be enough info at the reference site to
> determine whether the haddock linked to is relevant to the particular goal
> I want to achieve. Also as Simon points out, there are Notes that don't
> have a clear "owner".
>
> Heck, even writing an unused binding `_Late_lambda_lifting_in_STG` and put
> the haddocks there would work, I suppose. We could simply link to it with
> '_Late_lambda_lifting_in_STG' from other haddocks.
>
> My point is: If we managed to have something quite like named chunks, but
> with a title and one place it gets rendered and then linked to (I don't
> like that named chunks are inlined into every use site), we could probably
> agree on using that.
> Also I'd like to see the Notes rendered *regardless* of whether the thing
> it is attached to is exported. That would make Notes a lot more accessible.
>
> Sebastian
>
>
> Am Di., 14. Sept. 2021 um 14:32 Uhr schrieb Hécate <hecate at glitchbra.in>:
>
>> > today’s Haddock doesn’t understand Notes.  But we could fix that if we
>> were minded to.
>>
>> I may have missed an episode or two here but what prevents us from
>> writing Notes as Named Chunks¹, write them where Haddock expects you to put
>> documentation, and refer to them from the relevant spot in the code?
>> Viktor (in CC) has done a wonderful work at producing nice layouts for
>> Haddocks in base, and we could learn a couple of lessons from his MRs.
>>
>> ---
>>
>> Now, on the matter of improving Haddock to understand GHC's notes, I'd
>> like to remind everyone that Haddock is currently understaffed in terms of
>> feature development, and I would like to call to everyone with experience
>> dealing with its codebase to give a hand in refactoring, dusting off and
>> improving the code so that its maintainability is not jeopardised by people
>> simply going elsewhere.
>> Our bus factor (or as I like to call it, circus factor), is quite
>> terrifying considering the importance of the tool in our ecosystem.
>>
>>
>> ¹
>> https://haskell-haddock.readthedocs.io/en/latest/markup.html#named-chunks
>>
>> Le 14/09/2021 à 13:56, Simon Peyton Jones via ghc-devs a écrit :
>>
>> Alfredo writes (below for full thread)
>>
>>
>>
>> That is a deceptively simple question you ask there :-) I don't have a
>> strong view myself, but I can offer the perspective of somebody who was
>> been for a long time on the "other side of the trenches" (i.e. working
>> Haskell programmer, not necessarily working GHC programmer):
>>
>>
>>
>> * Blog post: yes, it's true that is a snapshot, and it's true that is not
>> under GHC's gitlab umbrella, so I wouldn't treat it as a reliable source of
>> documentation (for the reasons you also explain) but it's surely a good
>> testament that "at this point in time, for this particular GHC commit,
>> things were this way);
>>
>>
>>
>> * The wiki page: in the past, when I wanted to learn more about some GHC
>> feature, Google would point me to the relevant Wiki page on the GHC repo
>> describing such a feature, but I have to say I have almost always dismissed
>> it, because everybody knows Wikis are seldomly up-to-date :) In order for a
>> Wiki page to work we would have to at least add a banner at the top that
>> states this can be trusted as a reliable source of information, and offer
>> in the main section the current, up-to-date design. We can still offer the
>> historical breakdown of the designs in later sections, as it's still
>> valuable info to keep;
>>
>>
>>
>> * GHC notes: I have always considered GHC notes a double-edge sword --
>> from one side they are immensely useful when navigating the source code,
>> but these won't be rendered in the Hackage's haddocks, and this is not
>> helpful for GHC-the-library users willing to understand how to use (or
>> which is the semantic of) a particular type (sure, one can click "Show
>> Source" on Hackage but it's an annoying extra step to do just to hunt for
>> notes). We already have Notes for this work in strategic places -- even
>> better, we have proper Haddock comments for things like "Severity vs
>> DiagnosticReason" , e.g.
>> https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255320972%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WU2dKu2Q%2FFdwntJ2h%2F6zO1Ic01c9o0VhZc5JrE0AurY%3D&reserved=0>
>> .
>>
>>
>>
>>
>>
>> Yes Haddock doesn’t understand Notes but that’s a deficiency in Haddock!
>> There so much in GHC that simply does not fit well with the Haddocks
>> attached to a particular data decl or function.  We need Notes to explain
>> how all the moving parts fit together, and to point to them.
>>
>>
>>
>> Even better, we have proper Haddock comments for things like "Severity vs
>> DiagnosticReason"
>>
>>
>>
>> But I don’t think this is better – I think it is significantly worse!
>> In the case you cite, the Haddock is about DiagnosticReason, and mentions
>> Severity only incidentally.  I bet that the Haddock for Severity doesn’t
>> refer to this.   Nor is there a clear “Note [Severity vs DiagnosticReason]”
>> title that bits of code across GHC can refer to by saying “See Note
>> [Severity vs DiagnosticReason]”.   It’s far less satisfactory (to me) than
>> a single Note that
>>
>>    - covers just *one topic* (the difference between Severity and
>>    DiagnosticReason, rather than fully describing either
>>    - can be *pointed to* symmetrically from both Severity and
>>    DiagnosticReason
>>    - can be *pointed to* by many other bits of code
>>
>>
>>
>> The way it is better is that today’s Haddock doesn’t understand Notes.
>> But we could fix that if we were minded to.
>>
>>
>> Returning to how to document the error-message architecture, if you’d
>> prefer to use a Note than a wiki page, that’s fine.  But please write that
>> Overview Note that explains all the pieces, points to them one by one.  And
>> then copiously refer to that Note from all those places, so people will
>> update it.
>>
>>
>>
>> *Hopefully as the time goes by the new design will "spread" across all
>> the different peers working on GHC, and it will become "second nature".*
>>
>>
>>
>> I really don’t think that will happen unless there is a Note that
>> explains what the new design is!  Lacking this explicit design, everyone
>> will infer their own mental model of how it all works from sundry scattered
>> clues – and those mental models will differ.   So instead of one thing
>> “spreading”  a dozen subtly different things will spread.  And then the
>> next one, confused by these slightly different clues, will be even less
>> coherent.
>>
>>
>>
>> Let’s have one, fully-explicit version of The Plan that we constantly
>> refer to.
>>
>>
>>
>> cc’ing ghc-devs because we must constantly question and refine how we
>> describe and document GHC.
>>
>>
>>
>> Simon
>>
>>
>>
>> PS: I am leaving Microsoft at the end of November 2021, at which point
>> simonpj at microsoft.com will cease to work.  Use
>> simon.peytonjones at gmail.com instead.  (For now, it just forwards to
>> simonpj at microsoft.com.)
>>
>>
>>
>> *From:* Alfredo Di Napoli <alfredo.dinapoli at gmail.com>
>> <alfredo.dinapoli at gmail.com>
>> *Sent:* 26 August 2021 07:25
>> *To:* Simon Peyton Jones <simonpj at microsoft.com> <simonpj at microsoft.com>
>> *Cc:* rae at richarde.dev
>> *Subject:* Re: [Haskell Community] [Links] [Well-Typed Blog] The new GHC
>> diagnostic infrastructure
>>
>>
>>
>> Hello Simon!
>>
>>
>>
>> On Wed, 25 Aug 2021 at 13:36, Simon Peyton Jones <simonpj at microsoft.com>
>> wrote:
>>
>> Alfredo
>>
>>
>>
>> Thanks for all the work you are doing on GHC’s error message
>> infrastructure.  Your blog post gives a great overview.
>>
>>
>>
>> Thanks, and I am glad you enjoyed it :)
>>
>>
>>
>>
>>
>> As you know I’m very keen for GHC to have a Note or wiki page that gives
>> a solid, up-to-date overview of all the moving parts.  (NOT the design
>> alternatives, nor the time sequence; just the outcome.)  This is incredibly
>> useful for our future selves; and it helps ensure that people understand
>> (say) the difference between Severity and DiagnosticReason, and use them
>> correctly.
>>
>>
>>
>> So the question is: where is the canonical overview?  It could be
>>
>>    - Your blog post below. But that is a snapshot… you aren’t going to
>>    go back to edit it as the design evolves.  And it’s not in the repo.
>>    - The wiki page:
>>    https://gitlab.haskell.org/ghc/ghc/-/wikis/Errors-as-(structured)-values
>>    <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fwikis%2FErrors-as-(structured)-values&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255310976%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=A%2FyWqfqPWPYUk3EpaorYP29JvLIhgcdSdcYceFIKvhc%3D&reserved=0>.
>>    But it’s hard to keep up to date (it was last edited 3 months ago).
>>    - Note(s) in the code.  We seem to use this increasingly, and it has
>>    the great merit of being part of the source code itself.  But then we need
>>    clear pointer to the canonical overview Notes, and need to make sure they
>>    are up to date.
>>
>>
>>
>> I’m not advocating any particular path here… just wanting to be sure that
>> we end up with a good overview somewhere! What is your view?
>>
>>
>>
>>
>>
>> *TL;DR Probably a combo of a well-written (and up-to-date Wiki) plus some
>> carefully added Notes (and Haddock comments) in GHC might do the trick.*
>>
>>
>>
>> That is a deceptively simple question you ask there :-) I don't have a
>> strong view myself, but I can offer the perspective of somebody who was
>> been for a long time on the "other side of the trenches" (i.e. working
>> Haskell programmer, not necessarily working GHC programmer):
>>
>>
>>
>> * Blog post: yes, it's true that is a snapshot, and it's true that is not
>> under GHC's gitlab umbrella, so I wouldn't treat it as a reliable source of
>> documentation (for the reasons you also explain) but it's surely a good
>> testament that "at this point in time, for this particular GHC commit,
>> things were this way);
>>
>>
>>
>> * The wiki page: in the past, when I wanted to learn more about some GHC
>> feature, Google would point me to the relevant Wiki page on the GHC repo
>> describing such a feature, but I have to say I have almost always dismissed
>> it, because everybody knows Wikis are seldomly up-to-date :) In order for a
>> Wiki page to work we would have to at least add a banner at the top that
>> states this can be trusted as a reliable source of information, and offer
>> in the main section the current, up-to-date design. We can still offer the
>> historical breakdown of the designs in later sections, as it's still
>> valuable info to keep;
>>
>>
>>
>> * GHC notes: I have always considered GHC notes a double-edge sword --
>> from one side they are immensely useful when navigating the source code,
>> but these won't be rendered in the Hackage's haddocks, and this is not
>> helpful for GHC-the-library users willing to understand how to use (or
>> which is the semantic of) a particular type (sure, one can click "Show
>> Source" on Hackage but it's an annoying extra step to do just to hunt for
>> notes). We already have Notes for this work in strategic places -- even
>> better, we have proper Haddock comments for things like "Severity vs
>> DiagnosticReason" , e.g.
>> https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255320972%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WU2dKu2Q%2FFdwntJ2h%2F6zO1Ic01c9o0VhZc5JrE0AurY%3D&reserved=0>
>> .
>>
>>
>>
>> *So, in practical terms, I suggest we (I) give the Wiki a little overhaul
>> to add at the top the current design (or anything not captured directly in
>> GHC's source code) and I will keep an eye on the GHC notes and Haddock
>> comments to see if there is anything worth adding. Hopefully as the time
>> goes by the new design will "spread" across all the different peers working
>> on GHC, and it will become "second nature".*
>>
>>
>>
>> Hope it helps, and sorry for the long ramble!
>>
>>
>>
>> Alfredo
>>
>>
>>
>>
>>
>> Thanks
>>
>>
>>
>> Simon
>>
>>
>>
>> *From:* Alfredo Di Napoli via Haskell Community <discourse at haskell.org>
>> *Sent:* 23 August 2021 11:26
>> *To:* Simon Peyton Jones <simonpj at microsoft.com>
>> *Subject:* [Haskell Community] [Links] [Well-Typed Blog] The new GHC
>> diagnostic infrastructure
>>
>>
>>
>> [image: Image removed by sender.]
>>
>> *adinapoli*
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Fu%2Fadinapoli&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255330973%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PgX4crGMBTWVwI2UMcq%2BIFDZ0dDr%2FRWNYZdV%2Fqi8mX8%3D&reserved=0>
>> August 23
>>
>> [image: Image removed by sender.]*well-typed.com*
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255340965%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X51rdrGoKmUBPB8upLVL69LyInf%2BsYYQqM%2Fd4PnLnGQ%3D&reserved=0>
>>
>> *Error! Filename not specified.*
>> The new GHC diagnostic infrastructure - Well-Typed: The Haskell
>> Consultants
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255340965%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X51rdrGoKmUBPB8upLVL69LyInf%2BsYYQqM%2Fd4PnLnGQ%3D&reserved=0>
>> ------------------------------
>>
>> *Visit Topic*
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Ft%2Fwell-typed-blog-the-new-ghc-diagnostic-infrastructure%2F2918%2F1&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255350960%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tOfAGO5BbhanwBDgMA6eqpgKCLcLTtkum8QOuMsROdc%3D&reserved=0>
>> or reply to this email to respond.
>>
>> You are receiving this because you enabled mailing list mode.
>>
>> To unsubscribe from these emails, *click here*
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Femail%2Funsubscribe%2F962dfad7651b2ce3d7e30ba9267bdb857c77298d6fdec12626b65e014aaeee33&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255360954%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4616hEpSSUcOZ5zQYZMmEbF6mTJcIVKx2nlgA8ENsHM%3D&reserved=0>
>> .
>>
>>
>>
>>
>> _______________________________________________
>> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>> --
>> Hécate ✨
>> 🐦: @TechnoEmpress
>> IRC: Hecate
>> WWW: https://glitchbra.in
>> RUN: BSD
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
> --
> Hécate ✨
> 🐦: @TechnoEmpress
> IRC: Hecate
> WWW: https://glitchbra.in
> RUN: BSD
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/664b5c3b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 823 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/664b5c3b/attachment-0001.jpg>

From hecate at glitchbra.in  Tue Sep 14 14:38:23 2021
From: hecate at glitchbra.in (=?UTF-8?Q?H=c3=a9cate?=)
Date: Tue, 14 Sep 2021 16:38:23 +0200
Subject: Documenting GHC: blogs, wiki pages, Notes, Haddocks, etc
In-Reply-To: <CAAS+=P90RmZ1iHEF0Bn8iUAnFteFnT5BVNSSUYG+sCVwOj_3OA@mail.gmail.com>
References: <VI1PR83MB02695FEC9ED49208D5D9F6FAADDA9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <080562bf-45f7-b77c-9ea7-5120c4f46fbe@glitchbra.in>
 <CAAS+=P-iW-T58w2XH7Z3BBDLqxfRmpxjfAeRCswLd3capA8fFg@mail.gmail.com>
 <5735914a-f322-ac17-1b4e-23efd86b8943@glitchbra.in>
 <CAAS+=P90RmZ1iHEF0Bn8iUAnFteFnT5BVNSSUYG+sCVwOj_3OA@mail.gmail.com>
Message-ID: <25b7c295-3386-8a6e-afa0-c86d2202ac2b@glitchbra.in>

Okay I see what you mean, thank you. :)

Le 14/09/2021 à 16:37, Sebastian Graf a écrit :
> What I don't like:
>
>   * I have to give a section name to an otherwise unused $chunk
>   * I can't refer to $chunk from other modules
>   * Everywhere I reference $chunk in a haddock, it gets inlined
>     instead of appearing with its title or inserting a link with the
>     section name it is currently decoupled from.
>
>
> Am Di., 14. Sept. 2021 um 16:25 Uhr schrieb Hécate <hecate at glitchbra.in>:
>
>     Hi,
>
>     The named chunks can be positioned through the use of the export
>     list syntax:
>
>     module Foo
>       ( main
>       -- * Section name that will appear
>       --
>       -- $chunk
>       )
>
>     This should produce a free section that is not linked to any
>     exported item.
>     I see you're already using them though, so maybe I am
>     understanding something else?
>
>     Le 14/09/2021 à 16:00, Sebastian Graf a écrit :
>>     Hi,
>>
>>     I've been using Haddock's named chunks feature too when writing
>>     the docs for selective lambda lifting.
>>     This is the result:
>>     https://hackage.haskell.org/package/ghc-8.10.2/docs/StgLiftLams.html,
>>     and this is how the source code looks:
>>     https://hackage.haskell.org/package/ghc-8.10.2/docs/src/StgLiftLams.html
>>
>>     I quite like it. As you can see, I enabled both the existing
>>     Notes workflow and Haddock to work with it. It takes a bit of
>>     annoying extra work, though. Ideally, Haddock would simply
>>     recognise the Note syntax directly or provide a similar alternative.
>>
>>     And as far as linking is concerned: Sure, haddocks don't have a
>>     title to refer to. But you can always link to them by linking to
>>     the syntactic entity! For example, if I want to link to
>>     DiagnosticReason from Severity, I can simply do so by saying
>>     "Also see 'Severity'".
>>     I do admit this might not be enough info at the reference site to
>>     determine whether the haddock linked to is relevant to the
>>     particular goal I want to achieve. Also as Simon points out,
>>     there are Notes that don't have a clear "owner".
>>
>>     Heck, even writing an unused binding
>>     `_Late_lambda_lifting_in_STG` and put the haddocks there would
>>     work, I suppose. We could simply link to it with
>>     '_Late_lambda_lifting_in_STG' from other haddocks.
>>
>>     My point is: If we managed to have something quite like named
>>     chunks, but with a title and one place it gets rendered and then
>>     linked to (I don't like that named chunks are inlined into every
>>     use site), we could probably agree on using that.
>>     Also I'd like to see the Notes rendered *regardless* of whether
>>     the thing it is attached to is exported. That would make Notes a
>>     lot more accessible.
>>
>>     Sebastian
>>
>>
>>     Am Di., 14. Sept. 2021 um 14:32 Uhr schrieb Hécate
>>     <hecate at glitchbra.in>:
>>
>>         > today’s Haddock doesn’t understand Notes. But we could fix
>>         that if we were minded to.
>>
>>         I may have missed an episode or two here but what prevents us
>>         from writing Notes as Named Chunks¹, write them where Haddock
>>         expects you to put documentation, and refer to them from the
>>         relevant spot in the code?
>>         Viktor (in CC) has done a wonderful work at producing nice
>>         layouts for Haddocks in base, and we could learn a couple of
>>         lessons from his MRs.
>>
>>         ---
>>
>>         Now, on the matter of improving Haddock to understand GHC's
>>         notes, I'd like to remind everyone that Haddock is currently
>>         understaffed in terms of feature development, and I would
>>         like to call to everyone with experience dealing with its
>>         codebase to give a hand in refactoring, dusting off and
>>         improving the code so that its maintainability is not
>>         jeopardised by people simply going elsewhere.
>>         Our bus factor (or as I like to call it, circus factor), is
>>         quite terrifying considering the importance of the tool in
>>         our ecosystem.
>>
>>
>>         ¹
>>         https://haskell-haddock.readthedocs.io/en/latest/markup.html#named-chunks
>>
>>         Le 14/09/2021 à 13:56, Simon Peyton Jones via ghc-devs a écrit :
>>>
>>>         Alfredo writes (below for full thread)
>>>
>>>         That is a deceptively simple question you ask there :-) I
>>>         don't have a strong view myself, but I can offer the
>>>         perspective of somebody who was been for a long time on the
>>>         "other side of the trenches" (i.e. working Haskell
>>>         programmer, not necessarily working GHC programmer):
>>>
>>>         * Blog post: yes, it's true that is a snapshot, and it's
>>>         true that is not under GHC's gitlab umbrella, so I wouldn't
>>>         treat it as a reliable source of documentation (for the
>>>         reasons you also explain) but it's surely a good testament
>>>         that "at this point in time, for this particular GHC commit,
>>>         things were this way);
>>>
>>>         * The wiki page: in the past, when I wanted to learn more
>>>         about some GHC feature, Google would point me to the
>>>         relevant Wiki page on the GHC repo describing such a
>>>         feature, but I have to say I have almost always dismissed
>>>         it, because everybody knows Wikis are seldomly up-to-date :)
>>>         In order for a Wiki page to work we would have to at least
>>>         add a banner at the top that states this can be trusted as a
>>>         reliable source of information, and offer in the main
>>>         section the current, up-to-date design. We can still offer
>>>         the historical breakdown of the designs in later sections,
>>>         as it's still valuable info to keep;
>>>
>>>         * GHC notes: I have always considered GHC notes a
>>>         double-edge sword -- from one side they are immensely useful
>>>         when navigating the source code, but these won't be rendered
>>>         in the Hackage's haddocks, and this is not helpful for
>>>         GHC-the-library users willing to understand how to use (or
>>>         which is the semantic of) a particular type (sure, one can
>>>         click "Show Source" on Hackage but it's an annoying extra
>>>         step to do just to hunt for notes). We already have Notes
>>>         for this work in strategic places -- even better, we have
>>>         proper Haddock comments for things like "Severity vs
>>>         DiagnosticReason" , e.g.
>>>         https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279
>>>         <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255320972%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WU2dKu2Q%2FFdwntJ2h%2F6zO1Ic01c9o0VhZc5JrE0AurY%3D&reserved=0>
>>>         .
>>>
>>>         Yes Haddock doesn’t understand Notes but that’s a deficiency
>>>         in Haddock!  There so much in GHC that simply does not fit
>>>         well with the Haddocks attached to a particular data decl or
>>>         function.  We need Notes to explain how all the moving parts
>>>         fit together, and to point to them.
>>>
>>>         Even better, we have proper Haddock comments for things like
>>>         "Severity vs DiagnosticReason"
>>>
>>>         But I don’t think this is better – I think it is
>>>         significantly worse!   In the case you cite, the Haddock is
>>>         about DiagnosticReason, and mentions Severity only
>>>         incidentally.  I bet that the Haddock for Severity doesn’t
>>>         refer to this.   Nor is there a clear “Note [Severity vs
>>>         DiagnosticReason]” title that bits of code across GHC can
>>>         refer to by saying “See Note [Severity vs
>>>         DiagnosticReason]”.   It’s far less satisfactory (to me)
>>>         than a single Note that
>>>
>>>           * covers just *one topic* (the difference between Severity
>>>             and DiagnosticReason, rather than fully describing either
>>>           * can be *pointed to* symmetrically from both Severity and
>>>             DiagnosticReason
>>>           * can be *pointed to* by many other bits of code
>>>
>>>         The way it is better is that today’s Haddock doesn’t
>>>         understand Notes.  But we could fix that if we were minded to.
>>>
>>>
>>>         Returning to how to document the error-message architecture,
>>>         if you’d prefer to use a Note than a wiki page, that’s
>>>         fine.  But please write that Overview Note that explains all
>>>         the pieces, points to them one by one.  And then copiously
>>>         refer to that Note from all those places, so people will
>>>         update it.
>>>
>>>         _Hopefully as the time goes by the new design will "spread"
>>>         across all the different peers working on GHC, and it will
>>>         become "second nature"._
>>>
>>>         I really don’t think that will happen unless there is a Note
>>>         that explains what the new design is!  Lacking this explicit
>>>         design, everyone will infer their own mental model of how it
>>>         all works from sundry scattered clues – and those mental
>>>         models will differ.   So instead of one thing “spreading”  a
>>>         dozen subtly different things will spread.  And then the
>>>         next one, confused by these slightly different clues, will
>>>         be even less coherent.
>>>
>>>         Let’s have one, fully-explicit version of The Plan that we
>>>         constantly refer to.
>>>
>>>         cc’ing ghc-devs because we must constantly question and
>>>         refine how we describe and document GHC.
>>>
>>>         Simon
>>>
>>>         PS: I am leaving Microsoft at the end of November 2021, at
>>>         which point simonpj at microsoft.com
>>>         <mailto:simonpj at microsoft.com> will cease to work.  Use
>>>         simon.peytonjones at gmail.com
>>>         <mailto:simon.peytonjones at gmail.com> instead.  (For now, it
>>>         just forwards to simonpj at microsoft.com.)
>>>
>>>         *From:*Alfredo Di Napoli <alfredo.dinapoli at gmail.com>
>>>         <mailto:alfredo.dinapoli at gmail.com>
>>>         *Sent:* 26 August 2021 07:25
>>>         *To:* Simon Peyton Jones <simonpj at microsoft.com>
>>>         <mailto:simonpj at microsoft.com>
>>>         *Cc:* rae at richarde.dev
>>>         *Subject:* Re: [Haskell Community] [Links] [Well-Typed Blog]
>>>         The new GHC diagnostic infrastructure
>>>
>>>         Hello Simon!
>>>
>>>         On Wed, 25 Aug 2021 at 13:36, Simon Peyton Jones
>>>         <simonpj at microsoft.com> wrote:
>>>
>>>             Alfredo
>>>
>>>             Thanks for all the work you are doing on GHC’s error
>>>             message infrastructure.  Your blog post gives a great
>>>             overview.
>>>
>>>         Thanks, and I am glad you enjoyed it :)
>>>
>>>             As you know I’m very keen for GHC to have a Note or wiki
>>>             page that gives a solid, up-to-date overview of all the
>>>             moving parts.  (NOT the design alternatives, nor the
>>>             time sequence; just the outcome.)  This is incredibly
>>>             useful for our future selves; and it helps ensure that
>>>             people understand (say) the difference between Severity
>>>             and DiagnosticReason, and use them correctly.
>>>
>>>             So the question is: where is the canonical overview?  It
>>>             could be
>>>
>>>               * Your blog post below. But that is a snapshot… you
>>>                 aren’t going to go back to edit it as the design
>>>                 evolves.  And it’s not in the repo.
>>>               * The wiki page:
>>>                 https://gitlab.haskell.org/ghc/ghc/-/wikis/Errors-as-(structured)-values
>>>                 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fwikis%2FErrors-as-(structured)-values&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255310976%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=A%2FyWqfqPWPYUk3EpaorYP29JvLIhgcdSdcYceFIKvhc%3D&reserved=0>.
>>>                 But it’s hard to keep up to date (it was last edited
>>>                 3 months ago).
>>>               * Note(s) in the code.  We seem to use this
>>>                 increasingly, and it has the great merit of being
>>>                 part of the source code itself.  But then we need
>>>                 clear pointer to the canonical overview Notes, and
>>>                 need to make sure they are up to date.
>>>
>>>             I’m not advocating any particular path here… just
>>>             wanting to be sure that we end up with a good overview
>>>             somewhere! What is your view?
>>>
>>>         _TL;DR Probably a combo of a well-written (and up-to-date
>>>         Wiki) plus some carefully added Notes (and Haddock comments)
>>>         in GHC might do the trick._
>>>
>>>         That is a deceptively simple question you ask there :-) I
>>>         don't have a strong view myself, but I can offer the
>>>         perspective of somebody who was been for a long time on the
>>>         "other side of the trenches" (i.e. working Haskell
>>>         programmer, not necessarily working GHC programmer):
>>>
>>>         * Blog post: yes, it's true that is a snapshot, and it's
>>>         true that is not under GHC's gitlab umbrella, so I wouldn't
>>>         treat it as a reliable source of documentation (for the
>>>         reasons you also explain) but it's surely a good testament
>>>         that "at this point in time, for this particular GHC commit,
>>>         things were this way);
>>>
>>>         * The wiki page: in the past, when I wanted to learn more
>>>         about some GHC feature, Google would point me to the
>>>         relevant Wiki page on the GHC repo describing such a
>>>         feature, but I have to say I have almost always dismissed
>>>         it, because everybody knows Wikis are seldomly up-to-date :)
>>>         In order for a Wiki page to work we would have to at least
>>>         add a banner at the top that states this can be trusted as a
>>>         reliable source of information, and offer in the main
>>>         section the current, up-to-date design. We can still offer
>>>         the historical breakdown of the designs in later sections,
>>>         as it's still valuable info to keep;
>>>
>>>         * GHC notes: I have always considered GHC notes a
>>>         double-edge sword -- from one side they are immensely useful
>>>         when navigating the source code, but these won't be rendered
>>>         in the Hackage's haddocks, and this is not helpful for
>>>         GHC-the-library users willing to understand how to use (or
>>>         which is the semantic of) a particular type (sure, one can
>>>         click "Show Source" on Hackage but it's an annoying extra
>>>         step to do just to hunt for notes). We already have Notes
>>>         for this work in strategic places -- even better, we have
>>>         proper Haddock comments for things like "Severity vs
>>>         DiagnosticReason" , e.g.
>>>         https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279
>>>         <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255320972%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WU2dKu2Q%2FFdwntJ2h%2F6zO1Ic01c9o0VhZc5JrE0AurY%3D&reserved=0>
>>>         .
>>>
>>>         _So, in practical terms, I suggest we (I) give the Wiki a
>>>         little overhaul to add at the top the current design (or
>>>         anything not captured directly in GHC's source code) and I
>>>         will keep an eye on the GHC notes and Haddock comments to
>>>         see if there is anything worth adding. Hopefully as the time
>>>         goes by the new design will "spread" across all the
>>>         different peers working on GHC, and it will become "second
>>>         nature"._
>>>
>>>         Hope it helps, and sorry for the long ramble!
>>>
>>>         Alfredo
>>>
>>>             Thanks
>>>
>>>             Simon
>>>
>>>             *From:*Alfredo Di Napoli via Haskell Community
>>>             <discourse at haskell.org>
>>>             *Sent:* 23 August 2021 11:26
>>>             *To:* Simon Peyton Jones <simonpj at microsoft.com>
>>>             *Subject:* [Haskell Community] [Links] [Well-Typed Blog]
>>>             The new GHC diagnostic infrastructure
>>>
>>>             Image removed by sender.
>>>
>>>             	
>>>
>>>             *adinapoli*
>>>             <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Fu%2Fadinapoli&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255330973%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PgX4crGMBTWVwI2UMcq%2BIFDZ0dDr%2FRWNYZdV%2Fqi8mX8%3D&reserved=0>
>>>
>>>             August 23
>>>
>>>             Image removed by sender.*well-typed.com*
>>>             <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255340965%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X51rdrGoKmUBPB8upLVL69LyInf%2BsYYQqM%2Fd4PnLnGQ%3D&reserved=0>
>>>
>>>
>>>             *Error! Filename not specified.*
>>>
>>>
>>>                   The new GHC diagnostic infrastructure -
>>>                   Well-Typed: The Haskell Consultants
>>>                   <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255340965%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X51rdrGoKmUBPB8upLVL69LyInf%2BsYYQqM%2Fd4PnLnGQ%3D&reserved=0>
>>>
>>>             ------------------------------------------------------------------------
>>>
>>>             *Visit Topic*
>>>             <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Ft%2Fwell-typed-blog-the-new-ghc-diagnostic-infrastructure%2F2918%2F1&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255350960%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tOfAGO5BbhanwBDgMA6eqpgKCLcLTtkum8QOuMsROdc%3D&reserved=0>
>>>             or reply to this email to respond.
>>>
>>>             You are receiving this because you enabled mailing list
>>>             mode.
>>>
>>>             To unsubscribe from these emails, *click here*
>>>             <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Femail%2Funsubscribe%2F962dfad7651b2ce3d7e30ba9267bdb857c77298d6fdec12626b65e014aaeee33&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255360954%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4616hEpSSUcOZ5zQYZMmEbF6mTJcIVKx2nlgA8ENsHM%3D&reserved=0>.
>>>
>>>
>>>         _______________________________________________
>>>         ghc-devs mailing list
>>>         ghc-devs at haskell.org
>>>         http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>>         -- 
>>         Hécate ✨
>>         🐦: @TechnoEmpress
>>         IRC: Hecate
>>         WWW:https://glitchbra.in
>>         RUN: BSD
>>
>>         _______________________________________________
>>         ghc-devs mailing list
>>         ghc-devs at haskell.org
>>         http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>     -- 
>     Hécate ✨
>     🐦: @TechnoEmpress
>     IRC: Hecate
>     WWW:https://glitchbra.in
>     RUN: BSD
>
-- 
Hécate ✨
🐦: @TechnoEmpress
IRC: Hecate
WWW:https://glitchbra.in
RUN: BSD
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/ec9cbac6/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 823 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/ec9cbac6/attachment.jpg>

From ben at smart-cactus.org  Tue Sep 14 16:24:27 2021
From: ben at smart-cactus.org (Ben Gamari)
Date: Tue, 14 Sep 2021 12:24:27 -0400
Subject: More type safety in Core?
In-Reply-To: <CAPQL5pK=to19tuJ5SMMvzWdxgR8JhFmh1r0qBzmf=r_GjXXZnQ@mail.gmail.com>
References: <CAPQL5pK=to19tuJ5SMMvzWdxgR8JhFmh1r0qBzmf=r_GjXXZnQ@mail.gmail.com>
Message-ID: <878rzzktw8.fsf@smart-cactus.org>

Ari Fordsham <arifordsham at gmail.com> writes:

> I don't know if this is the right forum for this, I apologise if I'm
> intruding...
>
Indeed it is! No reason to apologise.

> Are there any plans to use the type system to enforce safety in Core, via
> e.g. GADTs? This would replace much of core-lint with static checking.
>
I am unaware of any plans currently. In general changing GHC's Core
language (or even just its Haskell representation) is something that
doesn't happen very often and for good reason.

To me, giving the Core AST more precise types would be a very
significant undertaking (touching a good fraction of the compiler) and
it is not clear to me that it wouldn't have compiler performance
implications. Afterall, expression types are still runtime-relevant to
the compiler and therefore can't be dropped. Moreover, the lifted
equality constraints that your GADTs constructors would no doubt carry
do have a runtime representation.

It's also not clear that catching ill-formed Core statically would pay
its way given the inevitable bookkeeping that the more elaborate types
would involve; this is especially true given that typing expressions
won't eliminate the need for dynamic Core Linting, which would still be
necessary to check things like well-scoped-ness.

These are my high-level thoughts; to say anything more concrete we would
need to consider a concrete proposal. It is certainly an interesting
area to explore and while I may be skeptical I would love to be proven
wrong.

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/066463a5/attachment.sig>

From simonpj at microsoft.com  Tue Sep 14 16:43:29 2021
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Tue, 14 Sep 2021 16:43:29 +0000
Subject: More type safety in Core?
In-Reply-To: <CAPQL5pK=to19tuJ5SMMvzWdxgR8JhFmh1r0qBzmf=r_GjXXZnQ@mail.gmail.com>
References: <CAPQL5pK=to19tuJ5SMMvzWdxgR8JhFmh1r0qBzmf=r_GjXXZnQ@mail.gmail.com>
Message-ID: <VI1PR83MB0269C80F19FF2D01CF29EB0EADDA9@VI1PR83MB0269.EURPRD83.prod.outlook.com>

One difficulty is that I think that writing Core-to-Core passes might become a lot more challenging.   It gets gnarly writing code that satisfies the type checker, depending of course on how strong the invariants are.

I think Typesafe runtime code generation<https://www.researchgate.net/publication/292674440_Type-safe_Runtime_Code_Generation_Accelerate_to_LLVM> has some material on this.

TL;DR: by all means give it a try.  I'm not terribly optimistic... but progress is made when we find that things we thought weren't possible are possible after all.  So I'd be happy to be proved wrong.

Simon

PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com<mailto:simonpj at microsoft.com> will cease to work.  Use simon.peytonjones at gmail.com<mailto:simon.peytonjones at gmail.com> instead.  (For now, it just forwards to simonpj at microsoft.com.)

From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Ari Fordsham
Sent: 14 September 2021 13:39
To: ghc-devs at haskell.org
Subject: More type safety in Core?

You don't often get email from arifordsham at gmail.com<mailto:arifordsham at gmail.com>. Learn why this is important<http://aka.ms/LearnAboutSenderIdentification>
I don't know if this is the right forum for this, I apologise if I'm intruding...

Are there any plans to use the type system to enforce safety in Core, via e.g. GADTs? This would replace much of core-lint with static checking.

Conal Eliottt has done something similar in a blog post (http://conal.net/blog/posts/overloading-lambda#:~:text=Haskell%20source%20language.-,I,-originally%20intended%20to<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fconal.net%2Fblog%2Fposts%2Foverloading-lambda%23%3A~%3Atext%3DHaskell%2520source%2520language.-%2CI%2C-originally%2520intended%2520to&data=04%7C01%7Csimonpj%40microsoft.com%7C3f5f8d3d0bc141e5a78408d9777ca375%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672217369668112%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=6IB9AcOy%2BzMmbVU51i9IenjPVA6usmeQE87j4Z1tXlk%3D&reserved=0>) and it seems relatively straightforward.

This would be especially beneficial to those working at the cutting edge of GHC features, statically ensuring their Core manipulations are correct. I would be surprised if existing compiler bugs wouldn't be found while implementing this.

What would the performance impact be? would using GADTs incur extra overhead? I'd assume you'd save something by lugging around less type information in Core.

Ari Fordsham
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/eee7e0b0/attachment-0001.html>

From simonpj at microsoft.com  Tue Sep 14 16:54:27 2021
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Tue, 14 Sep 2021 16:54:27 +0000
Subject: Documenting GHC: blogs, wiki pages, Notes, Haddocks, etc
In-Reply-To: <CAAS+=P-iW-T58w2XH7Z3BBDLqxfRmpxjfAeRCswLd3capA8fFg@mail.gmail.com>
References: <VI1PR83MB02695FEC9ED49208D5D9F6FAADDA9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <080562bf-45f7-b77c-9ea7-5120c4f46fbe@glitchbra.in>
 <CAAS+=P-iW-T58w2XH7Z3BBDLqxfRmpxjfAeRCswLd3capA8fFg@mail.gmail.com>
Message-ID: <VI1PR83MB0269216EEECE248CCB79BFD9ADDA9@VI1PR83MB0269.EURPRD83.prod.outlook.com>

And as far as linking is concerned: Sure, haddocks don't have a title to refer to. But you can always link to them by linking to the syntactic entity! For example, if I want to link to DiagnosticReason from Severity, I can simply do so by saying "Also see 'Severity'".
I don’t like this at all!   The most important and interesting Notes are about the relationships between multiple distinct entities.  I don’t want to say “See Severity” because

  *   It’s not about Severity; it’s about the relationship between DiagnosticReason and Severity
  *   There might be a lot else about Severity that I want to say with Severity, that is nothing to do with DiagnosticReason.  I want to refer specifically to the note about the relationship.

Also, many, many Notes are referred to from some part of the source code of a function body (e.g. a guard, accompanied with “See Note [Special case for local ids]”).    Here there is no entity to act as the anchor.  But the Note still might have many pointers to it.

Ideally, Haddock would simply recognise the Note syntax directly or provide a similar alternative

Yes, exactly. Notes have evolved into such a compellingly useful part of GHC that I really think they deserve Haddock support.

But here’s another tension: the Haddock for a function should be about the user interface; its specification.  But many Notes are specifically about the details of an implementation.

I have more questions than answers.   But even though Notes are incredibly low tech, I read and write them constantly.  I think they are a surprisingly powerful, albeit extremely modest, idea that GHC has given to the world.

Simon

PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com<mailto:simonpj at microsoft.com> will cease to work.  Use simon.peytonjones at gmail.com<mailto:simon.peytonjones at gmail.com> instead.  (For now, it just forwards to simonpj at microsoft.com.)

From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Sebastian Graf
Sent: 14 September 2021 15:00
To: Hécate <hecate at glitchbra.in>
Cc: ghc-devs <ghc-devs at haskell.org>
Subject: Re: Documenting GHC: blogs, wiki pages, Notes, Haddocks, etc

Hi,

I've been using Haddock's named chunks feature too when writing the docs for selective lambda lifting.
This is the result: https://hackage.haskell.org/package/ghc-8.10.2/docs/StgLiftLams.html<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhackage.haskell.org%2Fpackage%2Fghc-8.10.2%2Fdocs%2FStgLiftLams.html&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247560036%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=sDKPBAiSSJYWjQysWkod%2BB1ewyzv4ubiwt2%2BfxEOjdI%3D&reserved=0>, and this is how the source code looks: https://hackage.haskell.org/package/ghc-8.10.2/docs/src/StgLiftLams.html<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhackage.haskell.org%2Fpackage%2Fghc-8.10.2%2Fdocs%2Fsrc%2FStgLiftLams.html&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247570032%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=GpJji7mVYmEi3KLJeCiRFDwQRfgMXN1boZmZuynykVI%3D&reserved=0>

I quite like it. As you can see, I enabled both the existing Notes workflow and Haddock to work with it. It takes a bit of annoying extra work, though. Ideally, Haddock would simply recognise the Note syntax directly or provide a similar alternative.

And as far as linking is concerned: Sure, haddocks don't have a title to refer to. But you can always link to them by linking to the syntactic entity! For example, if I want to link to DiagnosticReason from Severity, I can simply do so by saying "Also see 'Severity'".
I do admit this might not be enough info at the reference site to determine whether the haddock linked to is relevant to the particular goal I want to achieve. Also as Simon points out, there are Notes that don't have a clear "owner".

Heck, even writing an unused binding `_Late_lambda_lifting_in_STG` and put the haddocks there would work, I suppose. We could simply link to it with '_Late_lambda_lifting_in_STG' from other haddocks.

My point is: If we managed to have something quite like named chunks, but with a title and one place it gets rendered and then linked to (I don't like that named chunks are inlined into every use site), we could probably agree on using that.
Also I'd like to see the Notes rendered *regardless* of whether the thing it is attached to is exported. That would make Notes a lot more accessible.

Sebastian


Am Di., 14. Sept. 2021 um 14:32 Uhr schrieb Hécate <hecate at glitchbra.in<mailto:hecate at glitchbra.in>>:

> today’s Haddock doesn’t understand Notes.  But we could fix that if we were minded to.

I may have missed an episode or two here but what prevents us from writing Notes as Named Chunks¹, write them where Haddock expects you to put documentation, and refer to them from the relevant spot in the code?
Viktor (in CC) has done a wonderful work at producing nice layouts for Haddocks in base, and we could learn a couple of lessons from his MRs.

---

Now, on the matter of improving Haddock to understand GHC's notes, I'd like to remind everyone that Haddock is currently understaffed in terms of feature development, and I would like to call to everyone with experience dealing with its codebase to give a hand in refactoring, dusting off and improving the code so that its maintainability is not jeopardised by people simply going elsewhere.
Our bus factor (or as I like to call it, circus factor), is quite terrifying considering the importance of the tool in our ecosystem.


¹ https://haskell-haddock.readthedocs.io/en/latest/markup.html#named-chunks<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhaskell-haddock.readthedocs.io%2Fen%2Flatest%2Fmarkup.html%23named-chunks&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247580027%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=34wGCjxKe4A0u9hiIHHG82PN82gxV1MM%2B7w2Kl2RFT0%3D&reserved=0>
Le 14/09/2021 à 13:56, Simon Peyton Jones via ghc-devs a écrit :
Alfredo writes (below for full thread)

That is a deceptively simple question you ask there :-) I don't have a strong view myself, but I can offer the perspective of somebody who was been for a long time on the "other side of the trenches" (i.e. working Haskell programmer, not necessarily working GHC programmer):

* Blog post: yes, it's true that is a snapshot, and it's true that is not under GHC's gitlab umbrella, so I wouldn't treat it as a reliable source of documentation (for the reasons you also explain) but it's surely a good testament that "at this point in time, for this particular GHC commit, things were this way);

* The wiki page: in the past, when I wanted to learn more about some GHC feature, Google would point me to the relevant Wiki page on the GHC repo describing such a feature, but I have to say I have almost always dismissed it, because everybody knows Wikis are seldomly up-to-date :) In order for a Wiki page to work we would have to at least add a banner at the top that states this can be trusted as a reliable source of information, and offer in the main section the current, up-to-date design. We can still offer the historical breakdown of the designs in later sections, as it's still valuable info to keep;

* GHC notes: I have always considered GHC notes a double-edge sword -- from one side they are immensely useful when navigating the source code, but these won't be rendered in the Hackage's haddocks, and this is not helpful for GHC-the-library users willing to understand how to use (or which is the semantic of) a particular type (sure, one can click "Show Source" on Hackage but it's an annoying extra step to do just to hunt for notes). We already have Notes for this work in strategic places -- even better, we have proper Haddock comments for things like "Severity vs DiagnosticReason" , e.g. https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247590020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wnMtmaZ3aQKLqKgsEXBuq3RiOt5%2FkYcpwrzmNb6Asdw%3D&reserved=0> .


Yes Haddock doesn’t understand Notes but that’s a deficiency in Haddock!  There so much in GHC that simply does not fit well with the Haddocks attached to a particular data decl or function.  We need Notes to explain how all the moving parts fit together, and to point to them.

Even better, we have proper Haddock comments for things like "Severity vs DiagnosticReason"

But I don’t think this is better – I think it is significantly worse!   In the case you cite, the Haddock is about DiagnosticReason, and mentions Severity only incidentally.  I bet that the Haddock for Severity doesn’t refer to this.   Nor is there a clear “Note [Severity vs DiagnosticReason]” title that bits of code across GHC can refer to by saying “See Note [Severity vs DiagnosticReason]”.   It’s far less satisfactory (to me) than a single Note that
·         covers just one topic (the difference between Severity and DiagnosticReason, rather than fully describing either
·         can be pointed to symmetrically from both Severity and DiagnosticReason
·         can be pointed to by many other bits of code

The way it is better is that today’s Haddock doesn’t understand Notes.  But we could fix that if we were minded to.

Returning to how to document the error-message architecture, if you’d prefer to use a Note than a wiki page, that’s fine.  But please write that Overview Note that explains all the pieces, points to them one by one.  And then copiously refer to that Note from all those places, so people will update it.

Hopefully as the time goes by the new design will "spread" across all the different peers working on GHC, and it will become "second nature".

I really don’t think that will happen unless there is a Note that explains what the new design is!  Lacking this explicit design, everyone will infer their own mental model of how it all works from sundry scattered clues – and those mental models will differ.   So instead of one thing “spreading”  a dozen subtly different things will spread.  And then the next one, confused by these slightly different clues, will be even less coherent.

Let’s have one, fully-explicit version of The Plan that we constantly refer to.

cc’ing ghc-devs because we must constantly question and refine how we describe and document GHC.

Simon

PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com<mailto:simonpj at microsoft.com> will cease to work.  Use simon.peytonjones at gmail.com<mailto:simon.peytonjones at gmail.com> instead.  (For now, it just forwards to simonpj at microsoft.com<mailto:simonpj at microsoft.com>.)

From: Alfredo Di Napoli <alfredo.dinapoli at gmail.com><mailto:alfredo.dinapoli at gmail.com>
Sent: 26 August 2021 07:25
To: Simon Peyton Jones <simonpj at microsoft.com><mailto:simonpj at microsoft.com>
Cc: rae at richarde.dev<mailto:rae at richarde.dev>
Subject: Re: [Haskell Community] [Links] [Well-Typed Blog] The new GHC diagnostic infrastructure

Hello Simon!

On Wed, 25 Aug 2021 at 13:36, Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
Alfredo

Thanks for all the work you are doing on GHC’s error message infrastructure.  Your blog post gives a great overview.

Thanks, and I am glad you enjoyed it :)


As you know I’m very keen for GHC to have a Note or wiki page that gives a solid, up-to-date overview of all the moving parts.  (NOT the design alternatives, nor the time sequence; just the outcome.)  This is incredibly useful for our future selves; and it helps ensure that people understand (say) the difference between Severity and DiagnosticReason, and use them correctly.

So the question is: where is the canonical overview?  It could be

  *   Your blog post below. But that is a snapshot… you aren’t going to go back to edit it as the design evolves.  And it’s not in the repo.
  *   The wiki page: https://gitlab.haskell.org/ghc/ghc/-/wikis/Errors-as-(structured)-values<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fwikis%2FErrors-as-(structured)-values&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247600018%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Fms1dGIEYyuGHJvtI9vBO93zk53PJe8gc7usPXCDboM%3D&reserved=0>.  But it’s hard to keep up to date (it was last edited 3 months ago).
  *   Note(s) in the code.  We seem to use this increasingly, and it has the great merit of being part of the source code itself.  But then we need clear pointer to the canonical overview Notes, and need to make sure they are up to date.

I’m not advocating any particular path here… just wanting to be sure that we end up with a good overview somewhere! What is your view?


TL;DR Probably a combo of a well-written (and up-to-date Wiki) plus some carefully added Notes (and Haddock comments) in GHC might do the trick.

That is a deceptively simple question you ask there :-) I don't have a strong view myself, but I can offer the perspective of somebody who was been for a long time on the "other side of the trenches" (i.e. working Haskell programmer, not necessarily working GHC programmer):

* Blog post: yes, it's true that is a snapshot, and it's true that is not under GHC's gitlab umbrella, so I wouldn't treat it as a reliable source of documentation (for the reasons you also explain) but it's surely a good testament that "at this point in time, for this particular GHC commit, things were this way);

* The wiki page: in the past, when I wanted to learn more about some GHC feature, Google would point me to the relevant Wiki page on the GHC repo describing such a feature, but I have to say I have almost always dismissed it, because everybody knows Wikis are seldomly up-to-date :) In order for a Wiki page to work we would have to at least add a banner at the top that states this can be trusted as a reliable source of information, and offer in the main section the current, up-to-date design. We can still offer the historical breakdown of the designs in later sections, as it's still valuable info to keep;

* GHC notes: I have always considered GHC notes a double-edge sword -- from one side they are immensely useful when navigating the source code, but these won't be rendered in the Hackage's haddocks, and this is not helpful for GHC-the-library users willing to understand how to use (or which is the semantic of) a particular type (sure, one can click "Show Source" on Hackage but it's an annoying extra step to do just to hunt for notes). We already have Notes for this work in strategic places -- even better, we have proper Haddock comments for things like "Severity vs DiagnosticReason" , e.g. https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247610014%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=4AVRsMc0ZbG7%2Bd4TasEQ2Oz9kcjnwqDcjq1vSoaRaVo%3D&reserved=0> .

So, in practical terms, I suggest we (I) give the Wiki a little overhaul to add at the top the current design (or anything not captured directly in GHC's source code) and I will keep an eye on the GHC notes and Haddock comments to see if there is anything worth adding. Hopefully as the time goes by the new design will "spread" across all the different peers working on GHC, and it will become "second nature".

Hope it helps, and sorry for the long ramble!

Alfredo


Thanks

Simon

From: Alfredo Di Napoli via Haskell Community <discourse at haskell.org<mailto:discourse at haskell.org>>
Sent: 23 August 2021 11:26
To: Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
Subject: [Haskell Community] [Links] [Well-Typed Blog] The new GHC diagnostic infrastructure

[Image removed by                                              sender.]
adinapoli<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Fu%2Fadinapoli&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247610014%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=JrCzoU83GcVYoYKHRSpjOO7U%2FpYsMlsYySVym2U8kRo%3D&reserved=0>
August 23
[Image removed by                                            sender.]well-typed.com<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247620006%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=LqnTNbl049Et3ePE1B5j4ZkTPBF%2BJRHs5JJO7yTLU7o%3D&reserved=0>
Error! Filename not specified.
The new GHC diagnostic infrastructure - Well-Typed: The Haskell Consultants<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247630000%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=duBX20BWM0JoqCtXUIR0oWYibSVn7g1J1yg4QqPWpVs%3D&reserved=0>
________________________________

Visit Topic<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Ft%2Fwell-typed-blog-the-new-ghc-diagnostic-infrastructure%2F2918%2F1&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247639995%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=JJlsMnZ2qhoIg3G0iwcJNNDJYzbhNH6KH0vvOul8yXE%3D&reserved=0> or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, click here<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Femail%2Funsubscribe%2F962dfad7651b2ce3d7e30ba9267bdb857c77298d6fdec12626b65e014aaeee33&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247649991%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=MjpvoWdGcZFyBEW4jyzGByq47OFLWIq7622XnT8TWm8%3D&reserved=0>.



_______________________________________________

ghc-devs mailing list

ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>

http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247649991%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=PQ2QQLTfuy%2FF3USu3mpdmqczBuShQVHj6pph7BDdtJI%3D&reserved=0>

--

Hécate ✨

🐦: @TechnoEmpress

IRC: Hecate

WWW: https://glitchbra.in<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fglitchbra.in%2F&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247659987%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=hoRmDtamUR9fRowlNR1USylyLViCNBALD%2Fl75351bXA%3D&reserved=0>

RUN: BSD
_______________________________________________
ghc-devs mailing list
ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247669986%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=LRZo55%2BDrGc76Ku%2FTnZ7C1cL5mfiw%2Bvg0KS0Sn7FSu0%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/842c8d9b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 823 bytes
Desc: image001.jpg
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/842c8d9b/attachment-0001.jpg>

From ben at smart-cactus.org  Tue Sep 14 17:04:03 2021
From: ben at smart-cactus.org (Ben Gamari)
Date: Tue, 14 Sep 2021 13:04:03 -0400
Subject: Documenting GHC: blogs, wiki pages, Notes, Haddocks, etc
In-Reply-To: <CAAS+=P90RmZ1iHEF0Bn8iUAnFteFnT5BVNSSUYG+sCVwOj_3OA@mail.gmail.com>
References: <VI1PR83MB02695FEC9ED49208D5D9F6FAADDA9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <080562bf-45f7-b77c-9ea7-5120c4f46fbe@glitchbra.in>
 <CAAS+=P-iW-T58w2XH7Z3BBDLqxfRmpxjfAeRCswLd3capA8fFg@mail.gmail.com>
 <5735914a-f322-ac17-1b4e-23efd86b8943@glitchbra.in>
 <CAAS+=P90RmZ1iHEF0Bn8iUAnFteFnT5BVNSSUYG+sCVwOj_3OA@mail.gmail.com>
Message-ID: <87v933jdhs.fsf@smart-cactus.org>

Sebastian Graf <sgraf1337 at gmail.com> writes:

> What I don't like:
>
>    - I can't refer to $chunk from other modules
>    - Everywhere I reference $chunk in a haddock, it gets inlined instead of
>    appearing with its title or inserting a link with the section name it is
>    currently decoupled from.
>
It seems like we could easily fix both of these with a simple change to
Haddock: don't inline chunks except in references in the export list
(which determine where the chunk will be placed in the output) and
ensure that the anchor name of the chunk is predictable (e.g. based on
the chunk name). Then references from other modules (e.g. using
$MyModule.my_chunk syntax) could simply produce a link to the anchor.

Regardless, I too have long wanted to make Notes consumable from Haddock
documentation. It would be great to make progress on this.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/f7b0c9c2/attachment.sig>

From lists at richarde.dev  Tue Sep 14 19:43:23 2021
From: lists at richarde.dev (Richard Eisenberg)
Date: Tue, 14 Sep 2021 19:43:23 +0000
Subject: More type safety in Core?
In-Reply-To: <CAPQL5pK=to19tuJ5SMMvzWdxgR8JhFmh1r0qBzmf=r_GjXXZnQ@mail.gmail.com>
References: <CAPQL5pK=to19tuJ5SMMvzWdxgR8JhFmh1r0qBzmf=r_GjXXZnQ@mail.gmail.com>
Message-ID: <010f017be5d5dd9d-231549a7-502a-47d7-b3b1-280c9ae42607-000000@us-east-2.amazonses.com>

Hi Ari,

This is a fine idea in theory, but (at present) a poor one in practice, for at least the following reasons:

* GHC's internal language is based on System F, allowing polymorphism. Modeling polymorphism in the way you describe is hard, even for a language that supports full dependent types -- so much so that successfully doing it (in Agda) is the subject of a recent peer-reviewed publication: https://iohk.io/en/research/library/papers/system-f-in-agdafor-fun-and-profit/   There is some work on encoding System F in this way in Haskell, but it's rough: https://www.cis.upenn.edu/~plclub/blog/2020-06-26-Strongly-typed-System-F/   Note that GHC Core is significantly more complex than either of these more modest languages.

* Core is manipulated a *lot*. Having intrinsic typing means, essentially, that every optimization would have to be proved sound, in the compiler. This is another thing that would be wonderful in theory, but we're just very far away from being able to achieve this in practice.

* I don't think we'd save very much at all: any information used to make runtime decisions must be present at runtime, and types are erased. So if we did this, we'd still need to carry (likely via class constraints) lots of information around to runtime. The difference would be that it would be passed implicitly instead of explicitly, but doing this won't speed GHC up.

* I actually tried something like this while on holiday a few years ago: I wanted to label Coercions with their role. This is a tempting subset of the challenge you describe, because roles are very first-order (there are only 3 of them!) and yet hard to get right. My work ran into no dead ends, exactly, but it quickly required lots and lots of fancy support structures. (For example, we would need a finite map where both keys and values are indexed by some role. And we'd need existentials. Lots of them.) If I had more time, I might have finished this, but there are bigger fish to fry.

So: I'd be very happy with being able to do this as a long-term goal, but I'd say we are years away from it -- and the best way toward it is simply adding support for dependent types.

Richard

> On Sep 14, 2021, at 8:38 AM, Ari Fordsham <arifordsham at gmail.com> wrote:
> 
> I don't know if this is the right forum for this, I apologise if I'm intruding...
> 
> Are there any plans to use the type system to enforce safety in Core, via e.g. GADTs? This would replace much of core-lint with static checking.
> 
> Conal Eliottt has done something similar in a blog post (http://conal.net/blog/posts/overloading-lambda#:~:text=Haskell%20source%20language.-,I,-originally%20intended%20to <http://conal.net/blog/posts/overloading-lambda#:~:text=Haskell%20source%20language.-,I,-originally%20intended%20to>) and it seems relatively straightforward.
> 
> This would be especially beneficial to those working at the cutting edge of GHC features, statically ensuring their Core manipulations are correct. I would be surprised if existing compiler bugs wouldn't be found while implementing this.
> 
> What would the performance impact be? would using GADTs incur extra overhead? I'd assume you'd save something by lugging around less type information in Core.
> 
> Ari Fordsham
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/236ecfff/attachment.html>

From ietf-dane at dukhovni.org  Tue Sep 14 20:31:47 2021
From: ietf-dane at dukhovni.org (Viktor Dukhovni)
Date: Tue, 14 Sep 2021 16:31:47 -0400
Subject: Documenting GHC: blogs, wiki pages, Notes, Haddocks, etc
In-Reply-To: <080562bf-45f7-b77c-9ea7-5120c4f46fbe@glitchbra.in>
References: <VI1PR83MB02695FEC9ED49208D5D9F6FAADDA9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <080562bf-45f7-b77c-9ea7-5120c4f46fbe@glitchbra.in>
Message-ID: <B4926D56-2E1B-4C8D-A805-969A0622120E@dukhovni.org>

> On 14 Sep 2021, at 8:29 am, Hécate <hecate at glitchbra.in> wrote:
> 
> I may have missed an episode or two here but what prevents us from writing Notes as Named Chunks¹, write them where Haddock expects you to put documentation, and refer to them from the relevant spot in the code?
> Viktor (in CC) has done a wonderful work at producing nice layouts for Haddocks in base, and we could learn a couple of lessons from his MRs.

Thanks for the callout.  My contribution to the documentation has thus far been
limited to just Data.Foldable and Data.Traversable, though I was hoping that
the approach might catch on if others find it a step in the right direction.

Specific content aside, in terms of haddock techniques, the main thing I
did was to append a more expansive prose overview of a library module below
the function synopses.  This did not require anything fancy, just `$section`
references from the module header.

However I also needed to occasionally create hyperlinks within the overview,
and here I ran into a limitation.  Haddock renders a hyperlink to a particular
anchor section of a module as:

	<a href="....">Module.Name</a>

with no syntax to customise the user-friendly text.  This means that one
is forced into some linguistic contortions to create natural sentences
with the desired hyperlinks.

This is particularly tricky when the hyperlink will appear not only in
the prose of a module's overview, but also in the synopsis of a function
or a class that may be re-exported by another module (e.g. the Prelude).

It would ideally be possible to render the hyperlink differently in its
"home" module than in a re-exporting module.

Otherwise, I found anchors and hyperlinks to be largely usable...

-- 
	Viktor.


From callan.mcgill at gmail.com  Tue Sep 14 21:43:37 2021
From: callan.mcgill at gmail.com (Callan McGill)
Date: Tue, 14 Sep 2021 17:43:37 -0400
Subject: More type safety in Core?
In-Reply-To: <010f017be5d5dd9d-231549a7-502a-47d7-b3b1-280c9ae42607-000000@us-east-2.amazonses.com>
References: <CAPQL5pK=to19tuJ5SMMvzWdxgR8JhFmh1r0qBzmf=r_GjXXZnQ@mail.gmail.com>
 <010f017be5d5dd9d-231549a7-502a-47d7-b3b1-280c9ae42607-000000@us-east-2.amazonses.com>
Message-ID: <CAOf_dZjJqVEs7eXrA6khsOLqhw8ZXCGvGpbUd4FxTnk+vqO4Qw@mail.gmail.com>

Following on from Richard mentioning this paper and video there was an
extremely nice version of this done recently by Sam Derbyshire using type
checker plugins that is well worth a look to see what is involved:
https://github.com/sheaf/ghc-tcplugin-api/tree/main/examples/SystemF

Best,
Callan

On Tue, 14 Sept 2021 at 15:44, Richard Eisenberg <lists at richarde.dev> wrote:

> Hi Ari,
>
> This is a fine idea in theory, but (at present) a poor one in practice,
> for at least the following reasons:
>
> * GHC's internal language is based on System F, allowing polymorphism.
> Modeling polymorphism in the way you describe is hard, even for a language
> that supports full dependent types -- so much so that successfully doing it
> (in Agda) is the subject of a recent peer-reviewed publication:
> https://iohk.io/en/research/library/papers/system-f-in-agdafor-fun-and-profit/
>   There is some work on encoding System F in this way in Haskell, but it's
> rough:
> https://www.cis.upenn.edu/~plclub/blog/2020-06-26-Strongly-typed-System-F/
>   Note that GHC Core is significantly more complex than either of these
> more modest languages.
>
> * Core is manipulated a *lot*. Having intrinsic typing means, essentially,
> that every optimization would have to be proved sound, in the compiler.
> This is another thing that would be wonderful in theory, but we're just
> very far away from being able to achieve this in practice.
>
> * I don't think we'd save very much at all: any information used to make
> runtime decisions must be present at runtime, and types are erased. So if
> we did this, we'd still need to carry (likely via class constraints) lots
> of information around to runtime. The difference would be that it would be
> passed implicitly instead of explicitly, but doing this won't speed GHC up.
>
> * I actually tried something like this while on holiday a few years ago: I
> wanted to label Coercions with their role. This is a tempting subset of the
> challenge you describe, because roles are very first-order (there are only
> 3 of them!) and yet hard to get right. My work ran into no dead ends,
> exactly, but it quickly required lots and lots of fancy support structures.
> (For example, we would need a finite map where both keys and values are
> indexed by some role. And we'd need existentials. Lots of them.) If I had
> more time, I might have finished this, but there are bigger fish to fry.
>
> So: I'd be very happy with being able to do this as a long-term goal, but
> I'd say we are years away from it -- and the best way toward it is simply
> adding support for dependent types.
>
> Richard
>
> On Sep 14, 2021, at 8:38 AM, Ari Fordsham <arifordsham at gmail.com> wrote:
>
> I don't know if this is the right forum for this, I apologise if I'm
> intruding...
>
> Are there any plans to use the type system to enforce safety in Core, via
> e.g. GADTs? This would replace much of core-lint with static checking.
>
> Conal Eliottt has done something similar in a blog post (
> http://conal.net/blog/posts/overloading-lambda#:~:text=Haskell%20source%20language.-,I,-originally%20intended%20to)
> and it seems relatively straightforward.
>
> This would be especially beneficial to those working at the cutting edge
> of GHC features, statically ensuring their Core manipulations are correct.
> I would be surprised if existing compiler bugs wouldn't be found while
> implementing this.
>
> What would the performance impact be? would using GADTs incur extra
> overhead? I'd assume you'd save something by lugging around less type
> information in Core.
>
> Ari Fordsham
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/ed5e08ca/attachment.html>

From lists at richarde.dev  Wed Sep 15 01:15:05 2021
From: lists at richarde.dev (Richard Eisenberg)
Date: Wed, 15 Sep 2021 01:15:05 +0000
Subject: More type safety in Core?
In-Reply-To: <CAOf_dZjJqVEs7eXrA6khsOLqhw8ZXCGvGpbUd4FxTnk+vqO4Qw@mail.gmail.com>
References: <CAPQL5pK=to19tuJ5SMMvzWdxgR8JhFmh1r0qBzmf=r_GjXXZnQ@mail.gmail.com>
 <010f017be5d5dd9d-231549a7-502a-47d7-b3b1-280c9ae42607-000000@us-east-2.amazonses.com>
 <CAOf_dZjJqVEs7eXrA6khsOLqhw8ZXCGvGpbUd4FxTnk+vqO4Qw@mail.gmail.com>
Message-ID: <010f017be7058a92-9cb348ba-2683-4b63-8c4e-991dca788ae0-000000@us-east-2.amazonses.com>



> On Sep 14, 2021, at 5:43 PM, Callan McGill <callan.mcgill at gmail.com> wrote:
> 
> Following on from Richard mentioning this paper and video there was an extremely nice version of this done recently by Sam Derbyshire using type checker plugins that is well worth a look to see what is involved: https://github.com/sheaf/ghc-tcplugin-api/tree/main/examples/SystemF <https://github.com/sheaf/ghc-tcplugin-api/tree/main/examples/SystemF>

I just want to amplify that Callan above in promoting Sam's work -- an unfortunate oversight on my part. Very cool how it works with the new plugins architecture!

Richard

> 
> Best,
> Callan
> 
> On Tue, 14 Sept 2021 at 15:44, Richard Eisenberg <lists at richarde.dev <mailto:lists at richarde.dev>> wrote:
> Hi Ari,
> 
> This is a fine idea in theory, but (at present) a poor one in practice, for at least the following reasons:
> 
> * GHC's internal language is based on System F, allowing polymorphism. Modeling polymorphism in the way you describe is hard, even for a language that supports full dependent types -- so much so that successfully doing it (in Agda) is the subject of a recent peer-reviewed publication: https://iohk.io/en/research/library/papers/system-f-in-agdafor-fun-and-profit/ <https://iohk.io/en/research/library/papers/system-f-in-agdafor-fun-and-profit/>   There is some work on encoding System F in this way in Haskell, but it's rough: https://www.cis.upenn.edu/~plclub/blog/2020-06-26-Strongly-typed-System-F/ <https://www.cis.upenn.edu/~plclub/blog/2020-06-26-Strongly-typed-System-F/>   Note that GHC Core is significantly more complex than either of these more modest languages.
> 
> * Core is manipulated a *lot*. Having intrinsic typing means, essentially, that every optimization would have to be proved sound, in the compiler. This is another thing that would be wonderful in theory, but we're just very far away from being able to achieve this in practice.
> 
> * I don't think we'd save very much at all: any information used to make runtime decisions must be present at runtime, and types are erased. So if we did this, we'd still need to carry (likely via class constraints) lots of information around to runtime. The difference would be that it would be passed implicitly instead of explicitly, but doing this won't speed GHC up.
> 
> * I actually tried something like this while on holiday a few years ago: I wanted to label Coercions with their role. This is a tempting subset of the challenge you describe, because roles are very first-order (there are only 3 of them!) and yet hard to get right. My work ran into no dead ends, exactly, but it quickly required lots and lots of fancy support structures. (For example, we would need a finite map where both keys and values are indexed by some role. And we'd need existentials. Lots of them.) If I had more time, I might have finished this, but there are bigger fish to fry.
> 
> So: I'd be very happy with being able to do this as a long-term goal, but I'd say we are years away from it -- and the best way toward it is simply adding support for dependent types.
> 
> Richard
> 
>> On Sep 14, 2021, at 8:38 AM, Ari Fordsham <arifordsham at gmail.com <mailto:arifordsham at gmail.com>> wrote:
>> 
>> I don't know if this is the right forum for this, I apologise if I'm intruding...
>> 
>> Are there any plans to use the type system to enforce safety in Core, via e.g. GADTs? This would replace much of core-lint with static checking.
>> 
>> Conal Eliottt has done something similar in a blog post (http://conal.net/blog/posts/overloading-lambda#:~:text=Haskell%20source%20language.-,I,-originally%20intended%20to <http://conal.net/blog/posts/overloading-lambda#:~:text=Haskell%20source%20language.-,I,-originally%20intended%20to>) and it seems relatively straightforward.
>> 
>> This would be especially beneficial to those working at the cutting edge of GHC features, statically ensuring their Core manipulations are correct. I would be surprised if existing compiler bugs wouldn't be found while implementing this.
>> 
>> What would the performance impact be? would using GADTs incur extra overhead? I'd assume you'd save something by lugging around less type information in Core.
>> 
>> Ari Fordsham
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs <http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs>
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs <http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210915/38b4fd0f/attachment.html>

From zubin at well-typed.com  Wed Sep 15 07:27:11 2021
From: zubin at well-typed.com (Zubin Duggal)
Date: Wed, 15 Sep 2021 12:57:11 +0530
Subject: Documenting GHC: blogs, wiki pages, Notes, Haddocks, etc
In-Reply-To: <VI1PR83MB0269216EEECE248CCB79BFD9ADDA9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
References: <VI1PR83MB02695FEC9ED49208D5D9F6FAADDA9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
 <080562bf-45f7-b77c-9ea7-5120c4f46fbe@glitchbra.in>
 <CAAS+=P-iW-T58w2XH7Z3BBDLqxfRmpxjfAeRCswLd3capA8fFg@mail.gmail.com>
 <VI1PR83MB0269216EEECE248CCB79BFD9ADDA9@VI1PR83MB0269.EURPRD83.prod.outlook.com>
Message-ID: <20210915072711.qxtc3tigwdr2fmca@zubin-msi>

On 21/09/14 16:54, Simon Peyton Jones via ghc-devs wrote:

>Yes, exactly. Notes have evolved into such a compellingly useful part of GHC that I really think they deserve Haddock support.
>
>But here’s another tension: the Haddock for a function should be about the user interface; its specification.  But many Notes are specifically about the details of an implementation.

We could have haddock recognise and link the Note syntax in the
hyperlinked source only, not the exported documentation. Haddock could
process all comments, not just documentation comments for declarations
('-- |' '-- ^' etc.) and provide a way to identify and link to other
comments and declarations. These links could show up in the hyperlinked
source instead of the haddock page detailing the API.

From carter.schonwald at gmail.com  Thu Sep 16 18:21:58 2021
From: carter.schonwald at gmail.com (Carter Schonwald)
Date: Thu, 16 Sep 2021 14:21:58 -0400
Subject: [Haskell-cafe] Bundle patterns with type aliases
In-Reply-To: <CAMgWh9t8P79_KqiUNWG=7H3y0mUnbQmnGL5ELnbQ7+dTk-6kGQ@mail.gmail.com>
References: <0EC9C321-12FF-405B-B9BC-10E8DE9D165E@gmail.com>
 <CAMgWh9t8P79_KqiUNWG=7H3y0mUnbQmnGL5ELnbQ7+dTk-6kGQ@mail.gmail.com>
Message-ID: <CAHYVw0w33GC7mmaCY8k=74y0P8s4nbn_An6bZcj=SGbqiCznhw@mail.gmail.com>

These are great ideas! Could you please create a ghc tracker ticket with a
tiny examples or two?

There may be specific technical reasons we might not be able to do so for
type synonyms in ghc, but I don’t see any obvious barriers in the case of
David’s excellent idea, I’ve def seen lots of great code out there where
you’d really want either associated pattern synonyms or to bundle pattern
synonyms with the exported public interface for a type class.

I’m sure there’s some devil in the details but these sound lovely.  Step -1
is making up 1-2 toy examples and explaining what and why you want it on a
ghc ticket!

On Wed, Sep 8, 2021 at 1:25 PM David Feuer <david.feuer at gmail.com> wrote:

> I would like that, along with the ability to bundle patterns with classes.
>
> On Wed, Sep 8, 2021, 1:13 PM Keith <keith.wygant at gmail.com> wrote:
>
>> Is there currently a way to 'bundle' a pattern with a type alias? And if
>> not, could that capability be added to the PatternSynonyms GHC extension?
>> (Is this the right place to ask, or should I be asking a GHC list?)
>>
>> --Keith
>> Sent from my phone with K-9 Mail.
>> _______________________________________________
>> Haskell-Cafe mailing list
>> To (un)subscribe, modify options or view archives go to:
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>> Only members subscribed via the mailman list are allowed to post.
>
> _______________________________________________
> Haskell-Cafe mailing list
> To (un)subscribe, modify options or view archives go to:
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
> Only members subscribed via the mailman list are allowed to post.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210916/ef761378/attachment.html>

From david.feuer at gmail.com  Thu Sep 16 19:08:45 2021
From: david.feuer at gmail.com (David Feuer)
Date: Thu, 16 Sep 2021 15:08:45 -0400
Subject: [Haskell-cafe] Bundle patterns with type aliases
In-Reply-To: <CAHYVw0w33GC7mmaCY8k=74y0P8s4nbn_An6bZcj=SGbqiCznhw@mail.gmail.com>
References: <0EC9C321-12FF-405B-B9BC-10E8DE9D165E@gmail.com>
 <CAMgWh9t8P79_KqiUNWG=7H3y0mUnbQmnGL5ELnbQ7+dTk-6kGQ@mail.gmail.com>
 <CAHYVw0w33GC7mmaCY8k=74y0P8s4nbn_An6bZcj=SGbqiCznhw@mail.gmail.com>
Message-ID: <CAMgWh9vSfQwQHkE3tg7d5NJ4CtAVLP2p=hsah3a5St3q0t7zeQ@mail.gmail.com>

Here's an example:

pattern State :: (s -> (a, s)) -> State s a
pattern State f <- (coerce . runStateT -> f) where
  State = state

This would be very nice to bundle with the State type synonym.

On Thu, Sep 16, 2021, 2:22 PM Carter Schonwald <carter.schonwald at gmail.com>
wrote:

> These are great ideas! Could you please create a ghc tracker ticket with a
> tiny examples or two?
>
> There may be specific technical reasons we might not be able to do so for
> type synonyms in ghc, but I don’t see any obvious barriers in the case of
> David’s excellent idea, I’ve def seen lots of great code out there where
> you’d really want either associated pattern synonyms or to bundle pattern
> synonyms with the exported public interface for a type class.
>
> I’m sure there’s some devil in the details but these sound lovely.  Step
> -1 is making up 1-2 toy examples and explaining what and why you want it on
> a ghc ticket!
>
> On Wed, Sep 8, 2021 at 1:25 PM David Feuer <david.feuer at gmail.com> wrote:
>
>> I would like that, along with the ability to bundle patterns with classes.
>>
>> On Wed, Sep 8, 2021, 1:13 PM Keith <keith.wygant at gmail.com> wrote:
>>
>>> Is there currently a way to 'bundle' a pattern with a type alias? And if
>>> not, could that capability be added to the PatternSynonyms GHC extension?
>>> (Is this the right place to ask, or should I be asking a GHC list?)
>>>
>>> --Keith
>>> Sent from my phone with K-9 Mail.
>>> _______________________________________________
>>> Haskell-Cafe mailing list
>>> To (un)subscribe, modify options or view archives go to:
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>>> Only members subscribed via the mailman list are allowed to post.
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> To (un)subscribe, modify options or view archives go to:
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>> Only members subscribed via the mailman list are allowed to post.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210916/82745d58/attachment.html>

From david.feuer at gmail.com  Thu Sep 16 19:12:57 2021
From: david.feuer at gmail.com (David Feuer)
Date: Thu, 16 Sep 2021 15:12:57 -0400
Subject: [Haskell-cafe] Bundle patterns with type aliases
In-Reply-To: <CAHYVw0w33GC7mmaCY8k=74y0P8s4nbn_An6bZcj=SGbqiCznhw@mail.gmail.com>
References: <0EC9C321-12FF-405B-B9BC-10E8DE9D165E@gmail.com>
 <CAMgWh9t8P79_KqiUNWG=7H3y0mUnbQmnGL5ELnbQ7+dTk-6kGQ@mail.gmail.com>
 <CAHYVw0w33GC7mmaCY8k=74y0P8s4nbn_An6bZcj=SGbqiCznhw@mail.gmail.com>
Message-ID: <CAMgWh9scvLkAeZpGV+FrLWReR55=gOBVLYBHOrwVDwbpY_G6rg@mail.gmail.com>

Here's a class example:

class (MFoldable t, Monoid t) => Sequence t where
  singleton :: Elem t -> t
   ....

One might wish to write pattern synonyms for viewing the ends of a
sequence, like the ones in Data.Sequence, and bundle them with this class.

On Thu, Sep 16, 2021, 2:22 PM Carter Schonwald <carter.schonwald at gmail.com>
wrote:

> These are great ideas! Could you please create a ghc tracker ticket with a
> tiny examples or two?
>
> There may be specific technical reasons we might not be able to do so for
> type synonyms in ghc, but I don’t see any obvious barriers in the case of
> David’s excellent idea, I’ve def seen lots of great code out there where
> you’d really want either associated pattern synonyms or to bundle pattern
> synonyms with the exported public interface for a type class.
>
> I’m sure there’s some devil in the details but these sound lovely.  Step
> -1 is making up 1-2 toy examples and explaining what and why you want it on
> a ghc ticket!
>
> On Wed, Sep 8, 2021 at 1:25 PM David Feuer <david.feuer at gmail.com> wrote:
>
>> I would like that, along with the ability to bundle patterns with classes.
>>
>> On Wed, Sep 8, 2021, 1:13 PM Keith <keith.wygant at gmail.com> wrote:
>>
>>> Is there currently a way to 'bundle' a pattern with a type alias? And if
>>> not, could that capability be added to the PatternSynonyms GHC extension?
>>> (Is this the right place to ask, or should I be asking a GHC list?)
>>>
>>> --Keith
>>> Sent from my phone with K-9 Mail.
>>> _______________________________________________
>>> Haskell-Cafe mailing list
>>> To (un)subscribe, modify options or view archives go to:
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>>> Only members subscribed via the mailman list are allowed to post.
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> To (un)subscribe, modify options or view archives go to:
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>> Only members subscribed via the mailman list are allowed to post.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210916/1569e2b4/attachment.html>

From mail at nh2.me  Fri Sep 17 23:26:14 2021
From: mail at nh2.me (=?UTF-8?Q?Niklas_Hamb=c3=bcchen?=)
Date: Sat, 18 Sep 2021 01:26:14 +0200
Subject: The future of Phabricator
In-Reply-To: <87zhuw2fw2.fsf@smart-cactus.org>
References: <87zhuw2fw2.fsf@smart-cactus.org>
Message-ID: <665a82c0-5223-5cff-e990-684a09fc52f2@nh2.me>

For those interested:

Three years later, Phabricator shut down.

May 29, 2021:
https://admin.phacility.com/phame/post/view/11/phacility_is_winding_down_operations/

On 10/30/18 5:54 AM, Ben Gamari wrote:
> For one, at this point we have no options for support in the event that
> something goes wrong as the company responsible for Phabricator,
> Phacility, has closed their support channels to non-paying customers.
> Furthermore, in the past year or two Phacility has been placing their
> development resources in the parts their customers pay them for, which
> appear to be much different that the parts that we actively use. For
> this reason, some parts that we rely on seem oddly half-finished.

From carter.schonwald at gmail.com  Sat Sep 18 16:23:13 2021
From: carter.schonwald at gmail.com (Carter Schonwald)
Date: Sat, 18 Sep 2021 12:23:13 -0400
Subject: The future of Phabricator
In-Reply-To: <665a82c0-5223-5cff-e990-684a09fc52f2@nh2.me>
References: <87zhuw2fw2.fsf@smart-cactus.org>
 <665a82c0-5223-5cff-e990-684a09fc52f2@nh2.me>
Message-ID: <CAHYVw0xU08mGM3M8UaPRk8i3L=QGpmK=AwBqYdOv3gxVhJT=pg@mail.gmail.com>

In one sense our migration was prescient, on the other hand i really wish
gitlab had a more performant code review ux.

Are there any performant for large code review tools? I find the reload
rendering latency and auto collapse defaults in gitlab pretty painful and
I’m pretty ignorant of the land scape these days.

On Fri, Sep 17, 2021 at 7:27 PM Niklas Hambüchen via ghc-devs <
ghc-devs at haskell.org> wrote:

> For those interested:
>
> Three years later, Phabricator shut down.
>
> May 29, 2021:
>
> https://admin.phacility.com/phame/post/view/11/phacility_is_winding_down_operations/
>
> On 10/30/18 5:54 AM, Ben Gamari wrote:
> > For one, at this point we have no options for support in the event that
> > something goes wrong as the company responsible for Phabricator,
> > Phacility, has closed their support channels to non-paying customers.
> > Furthermore, in the past year or two Phacility has been placing their
> > development resources in the parts their customers pay them for, which
> > appear to be much different that the parts that we actively use. For
> > this reason, some parts that we rely on seem oddly half-finished.
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210918/d35fe788/attachment.html>

From ssbothwell at gmail.com  Mon Sep 20 06:50:12 2021
From: ssbothwell at gmail.com (Solomon Bothwell)
Date: Sun, 19 Sep 2021 23:50:12 -0700
Subject: Problem building GHC
Message-ID: <CAJX8NmKFL74RhgOEf6TV_2qw-zt+W50kQCN846DLQz0dE_B4NQ@mail.gmail.com>

Hi. I'm a Haskell developer and I have recently started exploring and
building GHC, mainly with the hope of writing some small MRs for base.

I use nixos and ghc.nix to setup a build environment. Last week I was able
to build `ghc-8.10.7-release` with no trouble. I'm not sure what may have
happened but I am unable to build either `ghc-8-10.7-release` or `master`
this week. I tried deleting my fork and creating a new one but am getting
the same results.

For `master` I get this error:
https://gist.github.com/ssbothwell/9693a919c521decf52503e9152b879b6
For `ghc-8.10.7-release` I get this error:
https://gist.github.com/ssbothwell/fb5c418b4ee509cf0ad95d4337381e9e

I hope this is the right venue to be asking for help on this. Would it be
better for me to write an issue on the ghc.nix github page?

Thanks,
Solomon.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210919/df65760d/attachment.html>

From moritz.angermann at gmail.com  Mon Sep 20 07:40:39 2021
From: moritz.angermann at gmail.com (Moritz Angermann)
Date: Mon, 20 Sep 2021 09:40:39 +0200
Subject: Problem building GHC
In-Reply-To: <CAJX8NmKFL74RhgOEf6TV_2qw-zt+W50kQCN846DLQz0dE_B4NQ@mail.gmail.com>
References: <CAJX8NmKFL74RhgOEf6TV_2qw-zt+W50kQCN846DLQz0dE_B4NQ@mail.gmail.com>
Message-ID: <CAKfdd-wwvQxjuegRYNwsW-XRXhW6QKX5QRwAhAG+PrLijCCfZQ@mail.gmail.com>

Those errors in both logs seem fairly odd. The first one appears to be
mostly a missing libgmp, the second one looks like hsc2hs's template file
is missing, however hadrian should have a rule for that.

You can try to clean your tree and rebuild.

git clean -xfd
git submodule foreach git clean -xfd
git submodule update --init --recursive

./boot && ./configure
cabal v2-update
hadrian/build -j --flavour=Quick/boot

In general the instructions on https://ghc.dev are pretty good.

Cheers,
 Moritz

On Mon, Sep 20, 2021 at 8:50 AM Solomon Bothwell <ssbothwell at gmail.com>
wrote:

> Hi. I'm a Haskell developer and I have recently started exploring and
> building GHC, mainly with the hope of writing some small MRs for base.
>
> I use nixos and ghc.nix to setup a build environment. Last week I was able
> to build `ghc-8.10.7-release` with no trouble. I'm not sure what may have
> happened but I am unable to build either `ghc-8-10.7-release` or `master`
> this week. I tried deleting my fork and creating a new one but am getting
> the same results.
>
> For `master` I get this error:
> https://gist.github.com/ssbothwell/9693a919c521decf52503e9152b879b6
> For `ghc-8.10.7-release` I get this error:
> https://gist.github.com/ssbothwell/fb5c418b4ee509cf0ad95d4337381e9e
>
> I hope this is the right venue to be asking for help on this. Would it be
> better for me to write an issue on the ghc.nix github page?
>
> Thanks,
> Solomon.
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210920/2dc107b0/attachment.html>

From lists at richarde.dev  Mon Sep 20 15:28:16 2021
From: lists at richarde.dev (Richard Eisenberg)
Date: Mon, 20 Sep 2021 15:28:16 +0000
Subject: GHC development with VSCode
Message-ID: <010f017c03d275a0-12317e0d-96fa-468b-a96d-b161ba39943a-000000@us-east-2.amazonses.com>

Hi devs,

I have migrated to use VSCode instead of emacs. There are the usual switchover pains, but I'm mostly pleased. One particular point of pleasure was that I had to do nothing, at all, to get VSCode working within the GHC code base. (Well, I had to switch to Hadrian, but perhaps that's for the best.)

My problem: VSCode over GHC pins my processor at 100% if I edit anything. Any advice for fixing this?

A little more detail: When VSCode starts up, it spends a while "processing" and "indexing" (no idea what these mean). OK. I can pay that one-time cost. But as I start editing, etc., it needs to process and index a lot more. Somewhat continuously. This slows my computer down generally, and -- more annoyingly -- slows down my builds (run in a separate terminal).

I understand why VSCode wants to do this: it's checking my code for errors, etc. But is there a way to say "not now, please"? More specifically: I'd like to stop VSCode from detecting errors in my code while I'm actively editing. It's just too slow. On the other hand, it would be brilliant if VSCode could continue to allow me to, say, jump to definitions and gather references, using its latest knowledge of the code. As I edit, I understand this "latest knowledge" may become stale, if I'm stopping VSCode from reprocessing and reindexing. So, it would be nice to be able to tell VSCode to refresh, when I want that.

Any pointers here? https://gitlab.haskell.org/ghc/ghc/-/wikis/Visual-Studio-Code <https://gitlab.haskell.org/ghc/ghc/-/wikis/Visual-Studio-Code> would be a good place to put them!

Thanks,
Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210920/3c552d70/attachment.html>

From maurer.benjamin at gmail.com  Mon Sep 20 16:33:51 2021
From: maurer.benjamin at gmail.com (Benjamin Maurer)
Date: Mon, 20 Sep 2021 18:33:51 +0200
Subject: GHC development with VSCode
In-Reply-To: <010f017c03d275a0-12317e0d-96fa-468b-a96d-b161ba39943a-000000@us-east-2.amazonses.com>
References: <010f017c03d275a0-12317e0d-96fa-468b-a96d-b161ba39943a-000000@us-east-2.amazonses.com>
Message-ID: <CAOZ2gfu9GmUAZCaZkbkMPDMNba+WBh+UQkDopbnQytRsRpNtmg@mail.gmail.com>

Hi Richard,

great to hear, I've also seen a tweet about new videos from you using VS
code (had no time to watch yet).
Hope that more adoption improves things!

So, I've been using VS Code for almost a year now and I have seen great
improvements - almost all of that is the Haskell LSP.

"processing" and "indexing" may be (at least the former?) the LSP, I guess?
I have head single instances where it went haywire and I had to do a hard
reset of my machine,
but generally I can work with VS code and LSP doing it's thing (Laptop with
Ryzen 4700U & 16 GB RAM).

You can hit Ctrl+Shift+P for the command HUD and type "Haskell" - this
gives you the options to stop and (re-)start the LSP.
The latter can help when it becomes sluggish.

In File -> Preferences -> Settings search for "Haskell", there you have the
option to turn off things, e.g., "Haskell: Diagnostics on Change" (turning
off will create diagnostics only when you save).

Hope that helps.

Cheers,
Ben

Am Mo., 20. Sept. 2021 um 17:29 Uhr schrieb Richard Eisenberg <
lists at richarde.dev>:

> Hi devs,
>
> I have migrated to use VSCode instead of emacs. There are the usual
> switchover pains, but I'm mostly pleased. One particular point of pleasure
> was that I had to do nothing, at all, to get VSCode working within the GHC
> code base. (Well, I had to switch to Hadrian, but perhaps that's for the
> best.)
>
> My problem: VSCode over GHC pins my processor at 100% if I edit anything.
> Any advice for fixing this?
>
> A little more detail: When VSCode starts up, it spends a while
> "processing" and "indexing" (no idea what these mean). OK. I can pay that
> one-time cost. But as I start editing, etc., it needs to process and index
> a lot more. Somewhat continuously. This slows my computer down generally,
> and -- more annoyingly -- slows down my builds (run in a separate terminal).
>
> I understand why VSCode wants to do this: it's checking my code for
> errors, etc. But is there a way to say "not now, please"? More
> specifically: I'd like to stop VSCode from detecting errors in my code
> while I'm actively editing. It's just too slow. On the other hand, it would
> be brilliant if VSCode could continue to allow me to, say, jump to
> definitions and gather references, using its latest knowledge of the code.
> As I edit, I understand this "latest knowledge" may become stale, if I'm
> stopping VSCode from reprocessing and reindexing. So, it would be nice to
> be able to tell VSCode to refresh, when I want that.
>
> Any pointers here?
> https://gitlab.haskell.org/ghc/ghc/-/wikis/Visual-Studio-Code would be a
> good place to put them!
>
> Thanks,
> Richard
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210920/dd712a49/attachment.html>

From carter.schonwald at gmail.com  Mon Sep 20 16:45:37 2021
From: carter.schonwald at gmail.com (Carter Schonwald)
Date: Mon, 20 Sep 2021 12:45:37 -0400
Subject: GHC development with VSCode
In-Reply-To: <010f017c03d275a0-12317e0d-96fa-468b-a96d-b161ba39943a-000000@us-east-2.amazonses.com>
References: <010f017c03d275a0-12317e0d-96fa-468b-a96d-b161ba39943a-000000@us-east-2.amazonses.com>
Message-ID: <CAHYVw0zA2GNTapzFokCEGnxvNA8qnHydEKm7BTfp4tSgLc8Ojg@mail.gmail.com>

For ghc dev I generally disable tools / plugins that do indexing.
Especially if I’m on a laptop. The non cabalized builds tend to create a
huge build/indexing overhead and some of these integrations have a blocking
semantics. Plus underdocumented caching so there’s no “prebuilt index” as a
warm start for a local indexing process.



On Mon, Sep 20, 2021 at 11:29 AM Richard Eisenberg <lists at richarde.dev>
wrote:

> Hi devs,
>
> I have migrated to use VSCode instead of emacs. There are the usual
> switchover pains, but I'm mostly pleased. One particular point of pleasure
> was that I had to do nothing, at all, to get VSCode working within the GHC
> code base. (Well, I had to switch to Hadrian, but perhaps that's for the
> best.)
>
> My problem: VSCode over GHC pins my processor at 100% if I edit anything.
> Any advice for fixing this?
>
> A little more detail: When VSCode starts up, it spends a while
> "processing" and "indexing" (no idea what these mean). OK. I can pay that
> one-time cost. But as I start editing, etc., it needs to process and index
> a lot more. Somewhat continuously. This slows my computer down generally,
> and -- more annoyingly -- slows down my builds (run in a separate terminal).
>
> I understand why VSCode wants to do this: it's checking my code for
> errors, etc. But is there a way to say "not now, please"? More
> specifically: I'd like to stop VSCode from detecting errors in my code
> while I'm actively editing. It's just too slow. On the other hand, it would
> be brilliant if VSCode could continue to allow me to, say, jump to
> definitions and gather references, using its latest knowledge of the code.
> As I edit, I understand this "latest knowledge" may become stale, if I'm
> stopping VSCode from reprocessing and reindexing. So, it would be nice to
> be able to tell VSCode to refresh, when I want that.
>
> Any pointers here?
> https://gitlab.haskell.org/ghc/ghc/-/wikis/Visual-Studio-Code would be a
> good place to put them!
>
> Thanks,
> Richard
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210920/1e60cc87/attachment.html>

From ben at smart-cactus.org  Mon Sep 20 17:03:27 2021
From: ben at smart-cactus.org (Ben Gamari)
Date: Mon, 20 Sep 2021 13:03:27 -0400
Subject: GHC development with VSCode
In-Reply-To: <010f017c03d275a0-12317e0d-96fa-468b-a96d-b161ba39943a-000000@us-east-2.amazonses.com>
References: <010f017c03d275a0-12317e0d-96fa-468b-a96d-b161ba39943a-000000@us-east-2.amazonses.com>
Message-ID: <87h7efji2h.fsf@smart-cactus.org>

Richard Eisenberg <lists at richarde.dev> writes:

> Hi devs,
>
> I have migrated to use VSCode instead of emacs. There are the usual switchover pains, but I'm mostly pleased. One particular point of pleasure was that I had to do nothing, at all, to get VSCode working within the GHC code base. (Well, I had to switch to Hadrian, but perhaps that's for the best.)
>
> My problem: VSCode over GHC pins my processor at 100% if I edit anything. Any advice for fixing this?
>
> A little more detail: When VSCode starts up, it spends a while "processing" and "indexing" (no idea what these mean). OK. I can pay that one-time cost. But as I start editing, etc., it needs to process and index a lot more. Somewhat continuously. This slows my computer down generally, and -- more annoyingly -- slows down my builds (run in a separate terminal).
>
I suspect you are using not just VS Code (which generally performs
fairly well on files of "reasonable" size) but also Haskell Language
Server; is this so?

Indeed it is sadly true that LSP is currently a bit sluggish on GHC.
This is something that Matt Pickering has done a bit of work on the
past; he may have some concrete guidance for how to improve matters.
In the past he has suggested to me that disabling some language server
functionality helps matters.

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210920/984c9094/attachment.sig>

From nr at cs.tufts.edu  Wed Sep 22 13:25:34 2021
From: nr at cs.tufts.edu (Norman Ramsey)
Date: Wed, 22 Sep 2021 09:25:34 -0400
Subject: Two questions about building GHC from sources
Message-ID: <20210922132534.12DBC2C3331@homedog.cs.tufts.edu>

I've got two questions about building:

 1. If my bootstrap compiler changes (e.g., from 8.10.5 to 9.0.1), is
    Hadrian smart enough to rebuild everything?  If not, how do I
    force it to start over?

 2. At the moment I'm not installing my GHC; I'm using elements
    directly from the build tree.  If I build GHC and then move the
    whole tree to the whole location, will everything be happy?
    Or do I need to do something?  (Again, using Hadrian.)


Norman

From moritz.angermann at gmail.com  Wed Sep 22 13:47:10 2021
From: moritz.angermann at gmail.com (Moritz Angermann)
Date: Wed, 22 Sep 2021 15:47:10 +0200
Subject: Two questions about building GHC from sources
In-Reply-To: <20210922132534.12DBC2C3331@homedog.cs.tufts.edu>
References: <20210922132534.12DBC2C3331@homedog.cs.tufts.edu>
Message-ID: <CAKfdd-wo3nGQKR_h3b7o5X4CfznOXtcoM5cD56OP--DNdEm2yA@mail.gmail.com>

(1), I would hope so. If not, that's a bug.
(2), hadrian supports --build-root=, such that you can have different build
product trees based from one source. One of the nice features of hadrian
is, that it finally is able to separate build and source directories, there
are still the inplace files, which I believe should go. Yet, anything under
the --build-root, which defaults to _build, should be hermetic in there.
Whether or not you can just *move* those directories around is a bit
complicated, primarily because there are facilities in Cabal (the Paths
module), as well as potentially use of non ${pkgroot} prefixed package
config files, that hardcode paths. Again the Paths module from Cabal should
just be removed at some point. And missing ${pkgroot} values could be
fixed. My guess is, that we've done quite a lot of making it relocatable,
so just try to move it somewhere else and see if things break when calling
it somewhere else, they might, but if they do, it's a bug.


On Wed, Sep 22, 2021 at 3:27 PM Norman Ramsey <nr at cs.tufts.edu> wrote:

> I've got two questions about building:
>
>  1. If my bootstrap compiler changes (e.g., from 8.10.5 to 9.0.1), is
>     Hadrian smart enough to rebuild everything?  If not, how do I
>     force it to start over?
>
>  2. At the moment I'm not installing my GHC; I'm using elements
>     directly from the build tree.  If I build GHC and then move the
>     whole tree to the whole location, will everything be happy?
>     Or do I need to do something?  (Again, using Hadrian.)
>
>
> Norman
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210922/38c7c404/attachment.html>

From pepeiborra at gmail.com  Thu Sep 23 11:37:54 2021
From: pepeiborra at gmail.com (Pepe Iborra)
Date: Thu, 23 Sep 2021 13:37:54 +0200
Subject: GHC development with VSCode
In-Reply-To: <87h7efji2h.fsf@smart-cactus.org>
References: <010f017c03d275a0-12317e0d-96fa-468b-a96d-b161ba39943a-000000@us-east-2.amazonses.com>
 <87h7efji2h.fsf@smart-cactus.org>
Message-ID: <CAAkbwWKruA_rOvLmrOH+HXkhv7fVodb75R2cNDks+2hr5QuusQ@mail.gmail.com>

I have worked on making ghcide scale better to large projects, in the
context of Facebook Sigma, bringing down the complexity of "edits" from
O(transitive imports) to O(transitive modules).
The changes have not yet made it into an HLS release, but they are very
much ready for use if you are willing to build your own HLS binary:

https://github.com/haskell/haskell-language-server/pull/2060

On Mon, 20 Sept 2021 at 19:05, Ben Gamari <ben at smart-cactus.org> wrote:

> Richard Eisenberg <lists at richarde.dev> writes:
>
> > Hi devs,
> >
> > I have migrated to use VSCode instead of emacs. There are the usual
> switchover pains, but I'm mostly pleased. One particular point of pleasure
> was that I had to do nothing, at all, to get VSCode working within the GHC
> code base. (Well, I had to switch to Hadrian, but perhaps that's for the
> best.)
> >
> > My problem: VSCode over GHC pins my processor at 100% if I edit
> anything. Any advice for fixing this?
> >
> > A little more detail: When VSCode starts up, it spends a while
> "processing" and "indexing" (no idea what these mean). OK. I can pay that
> one-time cost. But as I start editing, etc., it needs to process and index
> a lot more. Somewhat continuously. This slows my computer down generally,
> and -- more annoyingly -- slows down my builds (run in a separate terminal).
> >
> I suspect you are using not just VS Code (which generally performs
> fairly well on files of "reasonable" size) but also Haskell Language
> Server; is this so?
>
> Indeed it is sadly true that LSP is currently a bit sluggish on GHC.
> This is something that Matt Pickering has done a bit of work on the
> past; he may have some concrete guidance for how to improve matters.
> In the past he has suggested to me that disabling some language server
> functionality helps matters.
>
> Cheers,
>
> - Ben
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210923/afdb4911/attachment.html>

From matthewtpickering at gmail.com  Thu Sep 23 12:06:12 2021
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Thu, 23 Sep 2021 13:06:12 +0100
Subject: GHC development with VSCode
In-Reply-To: <CAAkbwWKruA_rOvLmrOH+HXkhv7fVodb75R2cNDks+2hr5QuusQ@mail.gmail.com>
References: <010f017c03d275a0-12317e0d-96fa-468b-a96d-b161ba39943a-000000@us-east-2.amazonses.com>
 <87h7efji2h.fsf@smart-cactus.org>
 <CAAkbwWKruA_rOvLmrOH+HXkhv7fVodb75R2cNDks+2hr5QuusQ@mail.gmail.com>
Message-ID: <CALuQ0m-9zdNh8aAEc-Ng33Noe+QopHt66WzK0NKTVU60vET=FQ@mail.gmail.com>

I have been using Pepe's branch, it is stable and faster than the release.

Matt

On Thu, Sep 23, 2021 at 12:38 PM Pepe Iborra <pepeiborra at gmail.com> wrote:
>
> I have worked on making ghcide scale better to large projects, in the context of Facebook Sigma, bringing down the complexity of "edits" from O(transitive imports) to O(transitive modules).
> The changes have not yet made it into an HLS release, but they are very much ready for use if you are willing to build your own HLS binary:
>
> https://github.com/haskell/haskell-language-server/pull/2060
>
> On Mon, 20 Sept 2021 at 19:05, Ben Gamari <ben at smart-cactus.org> wrote:
>>
>> Richard Eisenberg <lists at richarde.dev> writes:
>>
>> > Hi devs,
>> >
>> > I have migrated to use VSCode instead of emacs. There are the usual switchover pains, but I'm mostly pleased. One particular point of pleasure was that I had to do nothing, at all, to get VSCode working within the GHC code base. (Well, I had to switch to Hadrian, but perhaps that's for the best.)
>> >
>> > My problem: VSCode over GHC pins my processor at 100% if I edit anything. Any advice for fixing this?
>> >
>> > A little more detail: When VSCode starts up, it spends a while "processing" and "indexing" (no idea what these mean). OK. I can pay that one-time cost. But as I start editing, etc., it needs to process and index a lot more. Somewhat continuously. This slows my computer down generally, and -- more annoyingly -- slows down my builds (run in a separate terminal).
>> >
>> I suspect you are using not just VS Code (which generally performs
>> fairly well on files of "reasonable" size) but also Haskell Language
>> Server; is this so?
>>
>> Indeed it is sadly true that LSP is currently a bit sluggish on GHC.
>> This is something that Matt Pickering has done a bit of work on the
>> past; he may have some concrete guidance for how to improve matters.
>> In the past he has suggested to me that disabling some language server
>> functionality helps matters.
>>
>> Cheers,
>>
>> - Ben
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From nr at cs.tufts.edu  Thu Sep 23 18:30:16 2021
From: nr at cs.tufts.edu (Norman Ramsey)
Date: Thu, 23 Sep 2021 14:30:16 -0400
Subject: help needed configuring GHC API client
Message-ID: <20210923183016.BBA612C2F53@homedog.cs.tufts.edu>

I'm writing client code against the GHC API in HEAD (version 9.3),
using 9.0.1 as my bootstrap compiler.  To make it possible to build
this code, I've set up cabal using

   cabal v1-configure \
      --package-db clear \
      --package-db $STAGE0/lib/package.conf.d/      # stage0 libraries

In my Haskell code I'm invoking `runGhc (Just thelibdir)` where

   thelibir = "/home/nr/asterius/ghc/_build/stage0/lib"

which is my `$STAGE0/lib`.

Unfortunuately, when I launch my app, `setSessionDynFlags` panics.
The output, along with some diagnostic information about some dflags
that seemed relevant, looks like this:

  libdir = /home/nr/asterius/ghc/_build/stage0/lib
  includePaths = IncludeSpecs {includePathsQuote = [], includePathsGlobal = [], includePathsQuoteImplicit = []}
  libraryPaths = []
  packageDBFlags = []
  packageEnv = Nothing
  panic! (the 'impossible' happened)
    GHC version 9.3.20210918:
          GHC couldn't find the RTS constants (#define HS_CONSTANTS ") in /home/nr/.ghcup/ghc/9.0.1/lib/ghc-9.0.1/include/DerivedConstants.h: the RTS package you are trying to use is perhaps for another GHC version(e.g. you are using the wrong package database) or the package database is broken.

  CallStack (from HasCallStack):
    error, called at _build/stage0/compiler/build/GHC/Platform/Constants.hs:143:20 in ghc:GHC.Platform.Constants

  Please report this as a GHC bug:  https://www.haskell.org/ghc/reportabug


I'm a little suprprised that my app is hunting for 9.3 information in
the tree that belongs to the bootstrap compiler.

I could start tinkering with `packageEnv` or other dflags, but I'm on
thin ice here, and I thought it made more sense to ask for advice.

What is going on in my configuration, and how can I set up my app so
it is using the library in my `libdir` rather than the library that
belongs to the bootstrap compiler?


Norman

From ben at smart-cactus.org  Fri Sep 24 01:36:45 2021
From: ben at smart-cactus.org (Ben Gamari)
Date: Thu, 23 Sep 2021 21:36:45 -0400
Subject: help needed configuring GHC API client
In-Reply-To: <20210923183016.BBA612C2F53@homedog.cs.tufts.edu>
References: <20210923183016.BBA612C2F53@homedog.cs.tufts.edu>
Message-ID: <87bl4ikb53.fsf@smart-cactus.org>

Norman Ramsey <nr at cs.tufts.edu> writes:

> I'm writing client code against the GHC API in HEAD (version 9.3),
> using 9.0.1 as my bootstrap compiler.  To make it possible to build
> this code, I've set up cabal using
>
>    cabal v1-configure \
>       --package-db clear \
>       --package-db $STAGE0/lib/package.conf.d/      # stage0 libraries
>
> In my Haskell code I'm invoking `runGhc (Just thelibdir)` where
>
>    thelibir = "/home/nr/asterius/ghc/_build/stage0/lib"
>
> which is my `$STAGE0/lib`.
>
> Unfortunuately, when I launch my app, `setSessionDynFlags` panics.
> The output, along with some diagnostic information about some dflags
> that seemed relevant, looks like this:
>
>   libdir = /home/nr/asterius/ghc/_build/stage0/lib
>   includePaths = IncludeSpecs {includePathsQuote = [], includePathsGlobal = [], includePathsQuoteImplicit = []}
>   libraryPaths = []
>   packageDBFlags = []
>   packageEnv = Nothing
>   panic! (the 'impossible' happened)
>     GHC version 9.3.20210918:
>           GHC couldn't find the RTS constants (#define HS_CONSTANTS ") in /home/nr/.ghcup/ghc/9.0.1/lib/ghc-9.0.1/include/DerivedConstants.h: the RTS package you are trying to use is perhaps for another GHC version(e.g. you are using the wrong package database) or the package database is broken.
>
>   CallStack (from HasCallStack):
>     error, called at _build/stage0/compiler/build/GHC/Platform/Constants.hs:143:20 in ghc:GHC.Platform.Constants
>
>   Please report this as a GHC bug:  https://www.haskell.org/ghc/reportabug
>
> I'm a little suprprised that my app is hunting for 9.3 information in
> the tree that belongs to the bootstrap compiler.
>
I suspect that the stages are getting mixed up here. Would it be
possible to post a full reproducer? I'd be happy to investigate further,
but without being able to reproduce locally it's a bit hard to say
anything useful. My recollection is that we look for DerivedConstants.h
in the usual include paths, so there are many ways in which things could
go wrong.

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210923/045eed55/attachment.sig>

From alfredo.dinapoli at gmail.com  Fri Sep 24 06:24:39 2021
From: alfredo.dinapoli at gmail.com (Alfredo Di Napoli)
Date: Fri, 24 Sep 2021 08:24:39 +0200
Subject: help needed configuring GHC API client
In-Reply-To: <87bl4ikb53.fsf@smart-cactus.org>
References: <20210923183016.BBA612C2F53@homedog.cs.tufts.edu>
 <87bl4ikb53.fsf@smart-cactus.org>
Message-ID: <CAFqA6+XuMFDZz+M0JzRo-d0HqHsXqfj8PQ-4XznDTM2vqkU6Gg@mail.gmail.com>

Hi Norman,

in addition to what Ben already said: is there any particular reason you
wanted to use the stage0 compiler? I have written a small program against
HEAD which used the GHC API fairly recently, and I have simply used the
`stage1` compiler. An excerpt from my little code snippet:

```
-- This is just the output of '_build/stage1/bin/ghc --print-libdir'
myGhcLibDir :: FilePath
myGhcLibDir = "./_build/stage1/lib"

playground :: FilePath -> IO ()
playground fn = do
  res <- runGhc (Just myGhcLibDir) $ do
  ...
```

As you can see that's exactly what you are doing, modulo the fact I was
using the `stage1` (where _build is the default directory for Hadrian
builds).

Hope this helps in some way!

A.




On Fri, 24 Sept 2021 at 03:37, Ben Gamari <ben at smart-cactus.org> wrote:

> Norman Ramsey <nr at cs.tufts.edu> writes:
>
> > I'm writing client code against the GHC API in HEAD (version 9.3),
> > using 9.0.1 as my bootstrap compiler.  To make it possible to build
> > this code, I've set up cabal using
> >
> >    cabal v1-configure \
> >       --package-db clear \
> >       --package-db $STAGE0/lib/package.conf.d/      # stage0 libraries
> >
> > In my Haskell code I'm invoking `runGhc (Just thelibdir)` where
> >
> >    thelibir = "/home/nr/asterius/ghc/_build/stage0/lib"
> >
> > which is my `$STAGE0/lib`.
> >
> > Unfortunuately, when I launch my app, `setSessionDynFlags` panics.
> > The output, along with some diagnostic information about some dflags
> > that seemed relevant, looks like this:
> >
> >   libdir = /home/nr/asterius/ghc/_build/stage0/lib
> >   includePaths = IncludeSpecs {includePathsQuote = [],
> includePathsGlobal = [], includePathsQuoteImplicit = []}
> >   libraryPaths = []
> >   packageDBFlags = []
> >   packageEnv = Nothing
> >   panic! (the 'impossible' happened)
> >     GHC version 9.3.20210918:
> >           GHC couldn't find the RTS constants (#define HS_CONSTANTS ")
> in /home/nr/.ghcup/ghc/9.0.1/lib/ghc-9.0.1/include/DerivedConstants.h: the
> RTS package you are trying to use is perhaps for another GHC version(e.g.
> you are using the wrong package database) or the package database is broken.
> >
> >   CallStack (from HasCallStack):
> >     error, called at
> _build/stage0/compiler/build/GHC/Platform/Constants.hs:143:20 in
> ghc:GHC.Platform.Constants
> >
> >   Please report this as a GHC bug:
> https://www.haskell.org/ghc/reportabug
> >
> > I'm a little suprprised that my app is hunting for 9.3 information in
> > the tree that belongs to the bootstrap compiler.
> >
> I suspect that the stages are getting mixed up here. Would it be
> possible to post a full reproducer? I'd be happy to investigate further,
> but without being able to reproduce locally it's a bit hard to say
> anything useful. My recollection is that we look for DerivedConstants.h
> in the usual include paths, so there are many ways in which things could
> go wrong.
>
> Cheers,
>
> - Ben
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210924/6a2241d3/attachment.html>

From matthewtpickering at gmail.com  Fri Sep 24 08:46:50 2021
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Fri, 24 Sep 2021 09:46:50 +0100
Subject: help needed configuring GHC API client
In-Reply-To: <20210923183016.BBA612C2F53@homedog.cs.tufts.edu>
References: <20210923183016.BBA612C2F53@homedog.cs.tufts.edu>
Message-ID: <CALuQ0m-Z-4ZvCbSmBmZT6r=a-R3nNeGftNJYQFauBFhtps_1VQ@mail.gmail.com>

Hi Norman,

I think you are overcomplicating things quite a bit.

1. Build a HEAD (9.3) compiler
2. Setup a normal cabal project to depend on the `ghc` library
3. Use the cabal -w option to point to your HEAD compiler to build the
project (`cabal build -w /path/to/head`)

That will work reliably rather than messing around with package databases.

Matt

On Thu, Sep 23, 2021 at 7:31 PM Norman Ramsey <nr at cs.tufts.edu> wrote:
>
> I'm writing client code against the GHC API in HEAD (version 9.3),
> using 9.0.1 as my bootstrap compiler.  To make it possible to build
> this code, I've set up cabal using
>
>    cabal v1-configure \
>       --package-db clear \
>       --package-db $STAGE0/lib/package.conf.d/      # stage0 libraries
>
> In my Haskell code I'm invoking `runGhc (Just thelibdir)` where
>
>    thelibir = "/home/nr/asterius/ghc/_build/stage0/lib"
>
> which is my `$STAGE0/lib`.
>
> Unfortunuately, when I launch my app, `setSessionDynFlags` panics.
> The output, along with some diagnostic information about some dflags
> that seemed relevant, looks like this:
>
>   libdir = /home/nr/asterius/ghc/_build/stage0/lib
>   includePaths = IncludeSpecs {includePathsQuote = [], includePathsGlobal = [], includePathsQuoteImplicit = []}
>   libraryPaths = []
>   packageDBFlags = []
>   packageEnv = Nothing
>   panic! (the 'impossible' happened)
>     GHC version 9.3.20210918:
>           GHC couldn't find the RTS constants (#define HS_CONSTANTS ") in /home/nr/.ghcup/ghc/9.0.1/lib/ghc-9.0.1/include/DerivedConstants.h: the RTS package you are trying to use is perhaps for another GHC version(e.g. you are using the wrong package database) or the package database is broken.
>
>   CallStack (from HasCallStack):
>     error, called at _build/stage0/compiler/build/GHC/Platform/Constants.hs:143:20 in ghc:GHC.Platform.Constants
>
>   Please report this as a GHC bug:  https://www.haskell.org/ghc/reportabug
>
>
> I'm a little suprprised that my app is hunting for 9.3 information in
> the tree that belongs to the bootstrap compiler.
>
> I could start tinkering with `packageEnv` or other dflags, but I'm on
> thin ice here, and I thought it made more sense to ask for advice.
>
> What is going on in my configuration, and how can I set up my app so
> it is using the library in my `libdir` rather than the library that
> belongs to the bootstrap compiler?
>
>
> Norman
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From nr at cs.tufts.edu  Fri Sep 24 14:29:28 2021
From: nr at cs.tufts.edu (Norman Ramsey)
Date: Fri, 24 Sep 2021 10:29:28 -0400
Subject: help needed configuring GHC API client
In-Reply-To: <CALuQ0m-Z-4ZvCbSmBmZT6r=a-R3nNeGftNJYQFauBFhtps_1VQ@mail.gmail.com>
 (sfid-H-20210924-092048-+92.67-1@multi.osbf.lua)
References: <20210923183016.BBA612C2F53@homedog.cs.tufts.edu>
 <sfid-H-20210924-092048-+92.67-1@multi.osbf.lua>
 <CALuQ0m-Z-4ZvCbSmBmZT6r=a-R3nNeGftNJYQFauBFhtps_1VQ@mail.gmail.com>
 (sfid-H-20210924-092048-+92.67-1@multi.osbf.lua)
Message-ID: <20210924142928.77D862C2836@homedog.cs.tufts.edu>

 > Hi Norman,
 > 
 > I think you are overcomplicating things quite a bit.
 > 
 > 1. Build a HEAD (9.3) compiler
 > 2. Setup a normal cabal project to depend on the `ghc` library
 > 3. Use the cabal -w option to point to your HEAD compiler to build the
 > project (`cabal build -w /path/to/head`)
 > 
 > That will work reliably rather than messing around with package databases.

My understanding is that if I do things this way, I give up the
Haskell Language Server.  I am in the middle of learning a large,
unfamiliar API, and the HLS is a lifeline to the documentation.
Am I wrong?  Is there another fast path to the documentation?


Norman

From nr at cs.tufts.edu  Fri Sep 24 14:32:25 2021
From: nr at cs.tufts.edu (Norman Ramsey)
Date: Fri, 24 Sep 2021 10:32:25 -0400
Subject: help needed configuring GHC API client
In-Reply-To: <CAFqA6+XuMFDZz+M0JzRo-d0HqHsXqfj8PQ-4XznDTM2vqkU6Gg@mail.gmail.com>
 (sfid-H-20210924-092047-+98.21-1@multi.osbf.lua)
References: <20210923183016.BBA612C2F53@homedog.cs.tufts.edu>
 <87bl4ikb53.fsf@smart-cactus.org>
 <sfid-H-20210924-092047-+98.21-1@multi.osbf.lua>
 <CAFqA6+XuMFDZz+M0JzRo-d0HqHsXqfj8PQ-4XznDTM2vqkU6Gg@mail.gmail.com>
 (sfid-H-20210924-092047-+98.21-1@multi.osbf.lua)
Message-ID: <20210924143225.99E6E2C2836@homedog.cs.tufts.edu>

 > in addition to what Ben already said: is there any particular reason you
 > wanted to use the stage0 compiler? 

Yes!  I want to use the Haskell Language Server with my code.  I am
given to understand that HLS does not work with GHC HEAD, so if I want
to use HLS with my own code, it needs to be compiled with the
bootstrap compiler, which *does* support HLS.  Therefore I am using
stage0 so that my own code and the GHC library are compiled with the
same compiler.

I will, however, repeat my experiment using stage1.  It would be good
to confirm your experience.



Norman


From nr at cs.tufts.edu  Fri Sep 24 14:33:45 2021
From: nr at cs.tufts.edu (Norman Ramsey)
Date: Fri, 24 Sep 2021 10:33:45 -0400
Subject: help needed configuring GHC API client
In-Reply-To: <87bl4ikb53.fsf@smart-cactus.org>
 (sfid-H-20210924-092042-+71.05-1@multi.osbf.lua)
References: <20210923183016.BBA612C2F53@homedog.cs.tufts.edu>
 <sfid-H-20210924-092042-+71.05-1@multi.osbf.lua>
 <87bl4ikb53.fsf@smart-cactus.org>
 (sfid-H-20210924-092042-+71.05-1@multi.osbf.lua)
Message-ID: <20210924143346.00CEE2C2836@homedog.cs.tufts.edu>

 > I suspect that the stages are getting mixed up here. Would it be
 > possible to post a full reproducer?

I think so.  I'm thinking a source tree for my client, plus step-by-step
instructions.  Would that do the job?


Norman

From carter.schonwald at gmail.com  Fri Sep 24 17:04:09 2021
From: carter.schonwald at gmail.com (Carter Schonwald)
Date: Fri, 24 Sep 2021 13:04:09 -0400
Subject: [Haskell-cafe] Numerics (was: Re: Trouble with asinh)
In-Reply-To: <AS8PR04MB85938ABDC936F235C90B8CEFB6DE9@AS8PR04MB8593.eurprd04.prod.outlook.com>
References: <CANa01BJaFZjpNdph5P17=tAaQWoPzsTOF8JKnSbgsTnRkTTroA@mail.gmail.com>
 <CAHYVw0w6yc7q9MO5L5dQqy4+nG0cx_82rVDHtcrGDFO4ERy8Cg@mail.gmail.com>
 <CANa01BJq=7EzKXV26N-fXP3ptNzE5oDXOoa22fbf2ZZov8cYpA@mail.gmail.com>
 <AS8PR04MB85938ABDC936F235C90B8CEFB6DE9@AS8PR04MB8593.eurprd04.prod.outlook.com>
Message-ID: <CAHYVw0yin+WVFUA-zR7zBEewVNiqQu5aMzvGmeQgXT2vbwgyUQ@mail.gmail.com>

Hey David,
Create a tracking ticket on the ghc for the set of changes / issues you
wanna address and how, and @ me on it  (I’ll then make sure to at other
applicable Dolan )and I’ll try to help you get oriented and make sure it
has the right visibility so we can help you out!

On Sat, Sep 18, 2021 at 12:00 PM David James <davjam at live.com> wrote:

> Hi thanks for the comments. I actually have draft rewrites of the Haskell
> complex functions with (I think) the correct behaviour, including branch
> cuts. But I discovered the errors in the underlying real functions while
> testing them. I’d like to try fixing all of these, but will probably need
> some help.
>
> Are there instructions somewhere on the process to fix a bug (presumably
> forking in GitHub, fixing code, adding test cases, running some CI, both
> for Linux and Windows, etc)? (And is there something similar for fixing
> mingle-w64 bugs?)
>
> I’ve been developing more test cases (esp for infinities +/-0, etc), but
> if there are any ideas/references on how to thoroughly test real or complex
> functions, that would also be useful.
>
> I’m currently away (and struggling to type on a phone) but will send more
> details (and pictures based on those in Common Lisp The Language 2nd
> edition) when I’m back.
>
> Thanks! David.
>
>
>
> > On 17 Sep 2021, at 22:06, Barak A. Pearlmutter <barak at pearlmutter.net>
> wrote:
> >
> > I suspect that most implementations of Common Lisp just call the C
> > standard library catan(3) etc, which are well tuned.
> >
> > $ clisp
> > Welcome to GNU CLISP 2.49.92 (2018-02-18) <http://clisp.org/>
> > [1]> (atan #c(0 1d-40))
> > #C(0 1.0d-40)
> >
> > In this particular case, the problem is that the Haskell Data.Complex
> > code has its own implementation of atan, which uses a log(1 + x) in
> > calculating the imaginary part. A foreign function call to the
> > appropriate libm routine would robustly address this, but that would
> > be difficult because it's trying to be generic over RealFloat a =>
> > Complex a, instead of special casing Complex Float / Complex Double.
> > Anyway, the Standard Prelude code for this is naïve: it should call
> > log1p, at the very least—which it actually goes to the trouble of
> > defining correctly, but not exporting.
> > _______________________________________________
> > Haskell-Cafe mailing list
> > To (un)subscribe, modify options or view archives go to:
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
> > Only members subscribed via the mailman list are allowed to post.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210924/b9bdd8fe/attachment.html>

From nr at cs.tufts.edu  Fri Sep 24 17:48:18 2021
From: nr at cs.tufts.edu (Norman Ramsey)
Date: Fri, 24 Sep 2021 13:48:18 -0400
Subject: help needed configuring GHC API client
In-Reply-To: <CAFqA6+XuMFDZz+M0JzRo-d0HqHsXqfj8PQ-4XznDTM2vqkU6Gg@mail.gmail.com>
 (sfid-H-20210924-092047-+98.21-1@multi.osbf.lua)
References: <20210923183016.BBA612C2F53@homedog.cs.tufts.edu>
 <87bl4ikb53.fsf@smart-cactus.org>
 <sfid-H-20210924-092047-+98.21-1@multi.osbf.lua>
 <CAFqA6+XuMFDZz+M0JzRo-d0HqHsXqfj8PQ-4XznDTM2vqkU6Gg@mail.gmail.com>
 (sfid-H-20210924-092047-+98.21-1@multi.osbf.lua)
Message-ID: <20210924174818.5F5902C2EEC@homedog.cs.tufts.edu>

Alfredo,

I've confirmed that I can use the stage1 compiler and it doesn't
panic.

It occurs to me that I might be able to *compile* my code with the
stage1 compiler while using the bootstrap compiler and Haskell
Language Server.   I'll give that a shot.


Norman


 > Hi Norman,
 > 
 > in addition to what Ben already said: is there any particular reason you
 > wanted to use the stage0 compiler? I have written a small program against
 > HEAD which used the GHC API fairly recently, and I have simply used the
 > `stage1` compiler. An excerpt from my little code snippet:
 > 
 > ```
 > -- This is just the output of '_build/stage1/bin/ghc --print-libdir'
 > myGhcLibDir :: FilePath
 > myGhcLibDir = "./_build/stage1/lib"
 > 
 > playground :: FilePath -> IO ()
 > playground fn = do
 >   res <- runGhc (Just myGhcLibDir) $ do
 >   ...
 > ```
 > 
 > As you can see that's exactly what you are doing, modulo the fact I was
 > using the `stage1` (where _build is the default directory for Hadrian
 > builds).
 > 
 > Hope this helps in some way!
 > 
 > A.
 > 
 > 
 > 
 > 
 > On Fri, 24 Sept 2021 at 03:37, Ben Gamari <ben at smart-cactus.org> wrote:
 > 
 > > Norman Ramsey <nr at cs.tufts.edu> writes:
 > >
 > > > I'm writing client code against the GHC API in HEAD (version 9.3),
 > > > using 9.0.1 as my bootstrap compiler.  To make it possible to build
 > > > this code, I've set up cabal using
 > > >
 > > >    cabal v1-configure \
 > > >       --package-db clear \
 > > >       --package-db $STAGE0/lib/package.conf.d/      # stage0 libraries
 > > >
 > > > In my Haskell code I'm invoking `runGhc (Just thelibdir)` where
 > > >
 > > >    thelibir = "/home/nr/asterius/ghc/_build/stage0/lib"
 > > >
 > > > which is my `$STAGE0/lib`.
 > > >
 > > > Unfortunuately, when I launch my app, `setSessionDynFlags` panics.
 > > > The output, along with some diagnostic information about some dflags
 > > > that seemed relevant, looks like this:
 > > >
 > > >   libdir = /home/nr/asterius/ghc/_build/stage0/lib
 > > >   includePaths = IncludeSpecs {includePathsQuote = [],
 > > includePathsGlobal = [], includePathsQuoteImplicit = []}
 > > >   libraryPaths = []
 > > >   packageDBFlags = []
 > > >   packageEnv = Nothing
 > > >   panic! (the 'impossible' happened)
 > > >     GHC version 9.3.20210918:
 > > >           GHC couldn't find the RTS constants (#define HS_CONSTANTS ")
 > > in /home/nr/.ghcup/ghc/9.0.1/lib/ghc-9.0.1/include/DerivedConstants.h: the
 > > RTS package you are trying to use is perhaps for another GHC version(e.g.
 > > you are using the wrong package database) or the package database is
 > > broken.
 > > >
 > > >   CallStack (from HasCallStack):
 > > >     error, called at
 > > _build/stage0/compiler/build/GHC/Platform/Constants.hs:143:20 in
 > > ghc:GHC.Platform.Constants
 > > >
 > > >   Please report this as a GHC bug:
 > > https://www.haskell.org/ghc/reportabug
 > > >
 > > > I'm a little suprprised that my app is hunting for 9.3 information in
 > > > the tree that belongs to the bootstrap compiler.
 > > >
 > > I suspect that the stages are getting mixed up here. Would it be
 > > possible to post a full reproducer? I'd be happy to investigate further,
 > > but without being able to reproduce locally it's a bit hard to say
 > > anything useful. My recollection is that we look for DerivedConstants.h
 > > in the usual include paths, so there are many ways in which things could
 > > go wrong.
 > >
 > > Cheers,
 > >
 > > - Ben
 > > _______________________________________________
 > > ghc-devs mailing list
 > > ghc-devs at haskell.org
 > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
 > >
 > 

From nr at cs.tufts.edu  Fri Sep 24 19:38:51 2021
From: nr at cs.tufts.edu (Norman Ramsey)
Date: Fri, 24 Sep 2021 15:38:51 -0400
Subject: GHC 9.3 API panics with "No home unit"; need advice on what next
Message-ID: <20210924193851.980B32C2EEC@homedog.cs.tufts.edu>

All,

I tried Alfredo's suggestion of building with the stage1 compiler.
(Sticking with the bootstrap compiler for HLS support, which seems
to be working OK.)  I've gotten a bit further.

I have a small app that loads .hs files, translates to STG,
and dumps the results.  It also dumps the `ModSummary` for each
module in the graph.

The code worked in the 9.0 API, but using the 9.3 API causes a panic:

  nr at homedog ~/a/sandbox> cabal v1-build
  Resolving dependencies...
  Configuring sandbox-0.1.0.0...
  Preprocessing executable 'sandbox' for sandbox-0.1.0.0..
  Building executable 'sandbox' for sandbox-0.1.0.0..
  [1 of 1] Compiling Main             ( app/Main.hs, dist/build/sandbox/sandbox-tmp/Main.o ) [Source file changed]
  Linking dist/build/sandbox/sandbox ...
  nr at homedog ~/a/sandbox> ./dist/build/sandbox/sandbox programs/Church.hs
  libdir == /home/nr/asterius/ghc/_build/stage1/lib
  ModSummary {
     ms_hs_hash = 8ed607a72dd3968f1e65123b865b8572
     ms_mod = Church,
     ms_textual_imps = [(Nothing, Prelude)]
     ms_srcimps = []
  }
  ..............
  sandbox: panic! (the 'impossible' happened)
    GHC version 9.3.20210918:
          unsafeGetHomeUnit: No home unit

  Please report this as a GHC bug:  https://www.haskell.org/ghc/reportabug

I poked around, but I expected the home unit to be set by
`setSessionDynFlags`, and I don't know why it isn't.

Everything needed to reproduce the problem can be found (I hope!) at

  https://github.com/nrnrnr/ghc-api-sandbox 

I'm a bit reluctant to simply report a bug as I'm afraid I might be
doing something wacky.

What should I look at next?


Norman

From nr at cs.tufts.edu  Fri Sep 24 20:24:02 2021
From: nr at cs.tufts.edu (Norman Ramsey)
Date: Fri, 24 Sep 2021 16:24:02 -0400
Subject: GHC 9.3 API panics with "No home unit"; need advice on what next
In-Reply-To: <20210924193851.980B32C2EEC@homedog.cs.tufts.edu>
 (sfid-H-20210924-153853-+64.77-1@multi.osbf.lua)
References: <sfid-H-20210924-153853-+64.77-1@multi.osbf.lua>
 <20210924193851.980B32C2EEC@homedog.cs.tufts.edu>
 (sfid-H-20210924-153853-+64.77-1@multi.osbf.lua)
Message-ID: <20210924202402.1D7C92C283B@homedog.cs.tufts.edu>

Followup: I managed to get things configured to work with `cabal v2-build`,
and with this build, the app dumps STG code as expected.  I am
unblocked!


Norman


 >   nr at homedog ~/a/sandbox> cabal v1-build
 >   Resolving dependencies...
 >   Configuring sandbox-0.1.0.0...
 >   Preprocessing executable 'sandbox' for sandbox-0.1.0.0..
 >   Building executable 'sandbox' for sandbox-0.1.0.0..
 >   [1 of 1] Compiling Main             ( app/Main.hs,
 >   dist/build/sandbox/sandbox-tmp/Main.o ) [Source file changed]
 >   Linking dist/build/sandbox/sandbox ...
 >   nr at homedog ~/a/sandbox> ./dist/build/sandbox/sandbox programs/Church.hs
 >   libdir == /home/nr/asterius/ghc/_build/stage1/lib
 >   ModSummary {
 >      ms_hs_hash = 8ed607a72dd3968f1e65123b865b8572
 >      ms_mod = Church,
 >      ms_textual_imps = [(Nothing, Prelude)]
 >      ms_srcimps = []
 >   }
 >   ..............
 >   sandbox: panic! (the 'impossible' happened)
 >     GHC version 9.3.20210918:
 >           unsafeGetHomeUnit: No home unit
 > 
 >   Please report this as a GHC bug:  https://www.haskell.org/ghc/reportabug

From nr at cs.tufts.edu  Mon Sep 27 20:50:42 2021
From: nr at cs.tufts.edu (Norman Ramsey)
Date: Mon, 27 Sep 2021 16:50:42 -0400
Subject: please help with ghc package-db flags
Message-ID: <20210927205042.133412C307E@homedog.cs.tufts.edu>

I've traced some troubles to a problem with GHC's response
to the -clear-package-db and -package-db flags.  I would very much
like to know if others can duplicate this issue.

All that is needed is for you to try the following commands, or
whatever variations may be appropriate for whatever ghc versions you
have installed on your system:

  ghc-pkg init /tmp/empty-package-db
  ghc        -clear-package-db -package-db /tmp/empty-package-db/ -v
  ghc-9.0.1  -clear-package-db -package-db /tmp/empty-package-db/ -v
  ghc-8.10.7 -clear-package-db -package-db /tmp/empty-package-db/ -v

On my system at this present moment, ghc-8.10.7 respects the commands,
but ghc-9.0.1 does not.  The output from the `-v` options show what's
happening: if GHC finds any packages that are *not* "wired-in," then
something is broken.  As examples, I attach the sample outputs from my
own system.

I think the issue is being caused by something mysterious in my filesystem.
Last Friday, ghc 9.0.1 was respecting those flags.  But today it is not.
Before I try to figure out what is going on, I would *very* much
appreciate learning if anyone else can duplicate the issue.

Please try running GHC with an empty package database and let me know
what happens.


Norman
-------------- next part --------------
nr at homedog ~> ghc-8.10.7 -v -clear-package-db -package-db $STAGE1/lib/package.conf.d -v
Glasgow Haskell Compiler, Version 8.10.7, stage 2 booted by GHC version 8.10.4
*** initializing package database:
Using binary package database: /home/nr/asterius/ghc/_build/stage1/lib/package.conf.d/package.cache
package flags []
loading package database /home/nr/asterius/ghc/_build/stage1/lib/package.conf.d
wired-in package ghc-prim mapped to ghc-prim-0.8.0
wired-in package integer-wired-in mapped to integer-gmp-1.1
wired-in package base mapped to base-4.16.0.0
wired-in package rts mapped to rts-1.0.2
wired-in package template-haskell mapped to template-haskell-2.18.0.0
wired-in package ghc mapped to ghc-9.3
!!! initializing package database: finished in 7.82 milliseconds, allocated 8.501 megabytes
*** Deleting temp files:
Deleting: 
*** Deleting temp dirs:
Deleting: 
ghc: no input files
Usage: For basic information, try the `--help' option.
nr at homedog ~ [1]> ghc-9.0.1 -v -clear-package-db -package-db $STAGE1/lib/package.conf.d -v
Loaded package environment from /home/nr/.ghc/x86_64-linux-9.0.1/environments/default
Glasgow Haskell Compiler, Version 9.0.1, stage 2 booted by GHC version 8.8.3
*** initializing unit database:
Using binary package database: /home/nr/.ghcup/ghc/9.0.1/lib/ghc-9.0.1/package.conf.d/package.cache
Using binary package database: /home/nr/.cabal/store/ghc-9.0.1/package.db/package.cache
package flags [-package-id ghc-9.0.1{unit ghc-9.0.1 True ([])},
               -package-id bytestring-0.10.12.1{unit bytestring-0.10.12.1 True ([])},
               -package-id unix-2.7.2.2{unit unix-2.7.2.2 True ([])},
               -package-id base-4.15.0.0{unit base-4.15.0.0 True ([])},
               -package-id time-1.9.3{unit time-1.9.3 True ([])},
               -package-id hpc-0.6.1.0{unit hpc-0.6.1.0 True ([])},
               -package-id filepath-1.4.2.1{unit filepath-1.4.2.1 True ([])},
               -package-id process-1.6.11.0{unit process-1.6.11.0 True ([])},
               -package-id array-0.5.4.0{unit array-0.5.4.0 True ([])},
               -package-id integer-gmp-1.1{unit integer-gmp-1.1 True ([])},
               -package-id containers-0.6.4.1{unit containers-0.6.4.1 True ([])},
               -package-id ghc-boot-9.0.1{unit ghc-boot-9.0.1 True ([])},
               -package-id binary-0.8.8.0{unit binary-0.8.8.0 True ([])},
               -package-id ghc-prim-0.7.0{unit ghc-prim-0.7.0 True ([])},
               -package-id ghci-9.0.1{unit ghci-9.0.1 True ([])},
               -package-id rts{unit rts True ([])},
               -package-id terminfo-0.4.1.4{unit terminfo-0.4.1.4 True ([])},
               -package-id transformers-0.5.6.2{unit transformers-0.5.6.2 True ([])},
               -package-id deepseq-1.4.5.0{unit deepseq-1.4.5.0 True ([])},
               -package-id ghc-boot-th-9.0.1{unit ghc-boot-th-9.0.1 True ([])},
               -package-id pretty-1.1.3.6{unit pretty-1.1.3.6 True ([])},
               -package-id template-haskell-2.17.0.0{unit template-haskell-2.17.0.0 True ([])},
               -package-id directory-1.3.6.1{unit directory-1.3.6.1 True ([])},
               -package-id text-1.2.4.1{unit text-1.2.4.1 True ([])},
               -package-id ghc-paths-0.1.0.12-3f4623015bdeee15e2d2cf4307a5331bbd2d0c0ee8ac137a5857a6db6f44b0cc{unit ghc-paths-0.1.0.12-3f4623015bdeee15e2d2cf4307a5331bbd2d0c0ee8ac137a5857a6db6f44b0cc True ([])}]
loading package database /home/nr/.ghcup/ghc/9.0.1/lib/ghc-9.0.1/package.conf.d
loading package database /home/nr/.cabal/store/ghc-9.0.1/package.db
wired-in package ghc-prim mapped to ghc-prim-0.7.0
wired-in package ghc-bignum mapped to ghc-bignum-1.0
wired-in package base mapped to base-4.15.0.0
wired-in package rts mapped to rts
wired-in package template-haskell mapped to template-haskell-2.17.0.0
wired-in package ghc mapped to ghc-9.0.1
!!! initializing unit database: finished in 15.03 milliseconds, allocated 12.331 megabytes
*** Deleting temp files:
Deleting: 
*** Deleting temp dirs:
Deleting: 
ghc: no input files
Usage: For basic information, try the `--help' option.

From allbery.b at gmail.com  Mon Sep 27 20:53:09 2021
From: allbery.b at gmail.com (Brandon Allbery)
Date: Mon, 27 Sep 2021 16:53:09 -0400
Subject: please help with ghc package-db flags
In-Reply-To: <20210927205042.133412C307E@homedog.cs.tufts.edu>
References: <20210927205042.133412C307E@homedog.cs.tufts.edu>
Message-ID: <CAKFCL4UFa5=4UOhKmqL_3UhHFZQQBvb1u6huLh=u_ekGhC=fpA@mail.gmail.com>

Not necessarily filesystem; also check GHC_PACKAGE_PATH in the environment.

On Mon, Sep 27, 2021 at 4:51 PM Norman Ramsey <nr at cs.tufts.edu> wrote:

> I've traced some troubles to a problem with GHC's response
> to the -clear-package-db and -package-db flags.  I would very much
> like to know if others can duplicate this issue.
>
> All that is needed is for you to try the following commands, or
> whatever variations may be appropriate for whatever ghc versions you
> have installed on your system:
>
>   ghc-pkg init /tmp/empty-package-db
>   ghc        -clear-package-db -package-db /tmp/empty-package-db/ -v
>   ghc-9.0.1  -clear-package-db -package-db /tmp/empty-package-db/ -v
>   ghc-8.10.7 -clear-package-db -package-db /tmp/empty-package-db/ -v
>
> On my system at this present moment, ghc-8.10.7 respects the commands,
> but ghc-9.0.1 does not.  The output from the `-v` options show what's
> happening: if GHC finds any packages that are *not* "wired-in," then
> something is broken.  As examples, I attach the sample outputs from my
> own system.
>
> I think the issue is being caused by something mysterious in my filesystem.
> Last Friday, ghc 9.0.1 was respecting those flags.  But today it is not.
> Before I try to figure out what is going on, I would *very* much
> appreciate learning if anyone else can duplicate the issue.
>
> Please try running GHC with an empty package database and let me know
> what happens.
>
>
> Norman
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>


-- 
brandon s allbery kf8nh
allbery.b at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210927/9b2ea96d/attachment.html>

From Gergo.Erdi at sc.com  Tue Sep 28 02:21:27 2021
From: Gergo.Erdi at sc.com (Erdi, Gergo)
Date: Tue, 28 Sep 2021 02:21:27 +0000
Subject: SOLVED: Registering orphan instances and filling the `deps` field of
 ModIface [RE: Where (else) do I need to register instances from loaded
 modules?]
Message-ID: <HK0PR03MB4867D911E9FEB7179C5A4C17E9A89@HK0PR03MB4867.apcprd03.prod.outlook.com>

PUBLIC

So these orphans got less loving attention on the mailing list than Oliver Twist, but I ended up solving this so just in case anyone else runs into this, here's what was going wrong.

It turns out the key to everything was this:

> 2. I also changed mkModIface [...] by copy-pasting more code from mkIfaceTc. I'm really starting to think that I'd be better off just changing mkIfaceTc to return a PartialModIface...

As you may recall from an earlier email from me (attached), the reason I couldn't use mkIfaceTc wholesale was because the fingerprint calculation code panicked with `lookupVers1`. Frustratingly, I never got any reply on that. Since I am feeding GHC with module sources instead of letting GHC itself find them on the file system, and I don't generate `.hi` files at all, I thought the fingerprinting simply isn't going to work in this setup. Consequently, I just worked around this by splitting `mkIfaceTc` into a `PartilaModIface`-creating part and never calling `addFingerprints`.

It turns out `addFingerprints` does more than just filling in fingerprints -- it also fills in a field that tells if the given module contains any orphan instances at all! So of course importing a module with orphan instances didn't work for me -- the "this module contains orphan instances" flag was never set... At this point I could have just copied some logic from `addFingerprints` to my code, but instead I decided to figure out why/how exactly `addFingerprints` panics.

It all comes down to this part of my code:

```
prepareModule unit modName = do
    -- ...

    (errs, mss) <- do
        env <- setHomeUnit (toUn
TheitId env) <$> getSession
        liftIO $ partitionEithers <$> downsweep env [] exclude False
    reportErrors errs

    let menv = mkModuleEnv [(ms_mod ms, ms) | ms <- mss]

    let Just ms = lookupModuleEnv menv $ mkModule unit mod
    return ms

prepareSource ms = do
    pmod <- parseModule ms
    tmod <- typecheckModule pmod

    -- ...
```

So I set the home unit to whatever unit we want to load the module into, before calling `downsweep`. This returns a `ModGraph` from which I retrieve the `ModSummary` I want. However, when I then pass that `ModSummary` to `parseModule >=> typecheckModule` , this choice of home unit is discarded. I expected it to be stored in the `ModSummary` by `downsweep`; isn't this exactly the kind of situation that `ms_hspp_opts` is for?! So anyway, because of this, during typechecking, GHC got confused about where to look for the interface of an already loaded module (since it was compiled in home unit = mainUnit, but then loaded into myUnit) and that is what lead to the panic during fingerprinting.

IN SUMMARY (or, "tl;dr"), the solution for my problem was to fix the handling of the home unit by setting it in the global session before calling `preareSource`, and then using full `mkIfaceTc` which correctly fills in the "this module has orphans" flag. This has been a very awkward experience because:
* `addFingerprints` doesn't just do what it says on the tin.
* "does this module has orphans" is not, prima facie, something that couldn't be in `PartialModIface`.
* The `lookupVers1` panic message is gibberish (even now I don't know what the "vers" stands for!) and doesn't give any hints to what is actually going wrong.
* Asking about the specific GHC panic on this mailing list yielded absolutely zero replies.
* `ModSummary` contains its own `DynFlags` but that `DynFlags` isn't used for everything in `parseModule` &co., so it doesn't really bake in the choices made during the preparation of the source file.

-----Original Message-----
From: Erdi, Gergo 
Sent: Wednesday, September 8, 2021 5:55 PM
To: 'Matthew Pickering' <matthewtpickering at gmail.com>; 'GHC' <ghc-devs at haskell.org>
Cc: 'gergo at erdi.hu' <gergo at erdi.hu>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>
Subject: Re: Registering orphan instances and filling the `deps` field of ModIface [RE: Where (else) do I need to register instances from loaded modules?]

PUBLIC

Hi,

Any other ideas I could try?

Gergo

-----Original Message-----
From: Erdi, Gergo 
Sent: Friday, August 27, 2021 2:42 PM
To: 'Matthew Pickering' <matthewtpickering at gmail.com>
Cc: 'GHC' <ghc-devs at haskell.org>; 'gergo at erdi.hu' <gergo at erdi.hu>; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>
Subject: Registering orphan instances and filling the `deps` field of ModIface [RE: Where (else) do I need to register instances from loaded modules?]

PUBLIC

Re: #1., I've simplified the code by removing all the Unit shuffling: everything is now in mainUnit. Unfortunately, this doesn't change anything other than cutting a cool 100 lines from the MWE code; I'm attaching the new smaller & simpler version. It is based on GHC 9.0.1 with an extra patch to export the first half of mkIfaceTc, just before calling mkFullIface. With this change, it really should be filling everything in ModIface the same way as the real GHC (so e.g. `deps` should be properly filled), modulo the fingerprints. 

Unfortunately, the `deps` field of the `ModIface`s created this way still doesn't contain any orphan instance modules: `Class.src`'s deps are empty (which is what I'd expect), `Instance.src` has a dependency in dep_mods (but not in dep_orphs) on `Class {-# SORUCE #-}` (I'm not sure why it's marked as a source import when `Class` module was already loaded into the module providers map before typechecking `Instance`...), and of course `Use.src` still fails so I can't get to its dependencies.

It is confusing to me that the problem should be because of the dep_orphs, because `Instance.src` does NOT use any orphan instances from `Class.src; merely, it provides a new orphan instance for other modules to depend on. Can someone explain this part to me?

Thanks,
	Gergo

-----Original Message-----
From: Erdi, Gergo 
Sent: Wednesday, August 25, 2021 10:37 AM
To: Matthew Pickering <matthewtpickering at gmail.com>
Cc: GHC <ghc-devs at haskell.org>; gergo at erdi.hu; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>
Subject: Re: Where (else) do I need to register instances from loaded modules?

Hi,

Thanks Matt! Unfortunately, I couldn't get it working even with these suggestions.

1. I changed registerModule so that it only changes the HPT when toUnitId (moduleUnit mod) == homeUnitId (hsc_dflags env). But I'm pretty sure this will be vacuously true, since the unit of the module comes from prepareModule setting the home unit before the call to `downsweep`.

TBH I don't understand the whole idea behind the home package, and maybe all my pain comes from misusing packages in the first place. As you can see in the original code I've sent, I'm loading modules from different units; in my real use case, I'd even like to do things like load module1 from unit1, then module2 from unit2, then module3 from unit1 again -- basically just using the units as a cheap way to classify modules so that later downstream, I can make decisions based on the unit of the module of the source of a given definition. Should I give up on this idea, is this going to lead to problems later on?

2. I also changed mkModIface to fill in the `deps` and while I'm at it, the `usages` field, by copy-pasting more code from mkIfaceTc. I'm really starting to think that I'd be better off just changing mkIfaceTc to return a PartialModIface... Anyway, the new definitions of `deps` and `usages` are:

    let pluginModules = map lpModule (cachedPlugins (hsc_dflags hsc_env))
        unit_id = homeUnitId (hsc_dflags hsc_env)
    deps <- mkDependencies
        unit_id (map mi_module pluginModules) tcg

    dep_files <- readIORef dependent_files
    let used_names = mkUsedNames tcg
    usages <- mkUsageInfo hsc_env this_mod (imp_mods imports) used_names
              dep_files merged pluginModules

With these changes applied, I am still seeing the same error. Printing shows that `dep_orphs deps` is empty for Instance.src; further tracing, in turn, shows that `imp_orphs . tcg_imports $ tcg` is also empty!

I have also found the wording in the documentation regarding dep_orphs confusing: it sounds like, in my case, this would mean that Use has a dependency on Instance, because it is using an orphan instance from there. But my problems are way before I am making a ModIface for Use, since it fails already during typechecking. But you saying the problem is the incomplete deps in ModIface must mean that dep_orphs needs to be filled in the ModIface for Instance (or, oh god I hope not, Class), right?

If I sound somewhat confused, rest assured that in reality I am even MORE confused!

Any further advice welcome,
	Gergo

-----Original Message-----
From: Matthew Pickering <matthewtpickering at gmail.com> 
Sent: Tuesday, August 24, 2021 5:16 PM
To: Erdi, Gergo <Gergo.Erdi at sc.com>
Cc: GHC <ghc-devs at haskell.org>; gergo at erdi.hu; Montelatici, Raphael Laurent <Raphael.Montelatici at sc.com>
Subject: [External] Re: Where (else) do I need to register instances from loaded modules?


Hi Gergo,

Some things I noticed:

* The HPT only contains modules from the current home unit. It seems that registerModule ignores this so you break this invariant.
* It seems that `Instance` contains an orphan, but you don't fill in the `deps` field when making a `ModIface`, which should contain all the orphan modules below the module you are compiling.

I think that if you fill in `deps` correctly then things may work out.
`Dependencies` eventually becomes `ImportAvail`, which gets consulted by `tcVisibleOrphanMods` which is called in `tcGetInstEnvs` which is called by `matchInstEnv`.

Matt

This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations

Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm

Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer.

Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. 

Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same.

Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.
-------------- next part --------------
An embedded message was scrubbed...
From: "Erdi, Gergo" <Gergo.Erdi at sc.com>
Subject: mkIfaceTc panics with "lookupVers1"
Date: Tue, 13 Jul 2021 08:02:09 +0000
Size: 23597
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210928/593e6044/attachment.mht>

From sylvain at haskus.fr  Tue Sep 28 08:28:42 2021
From: sylvain at haskus.fr (Sylvain Henry)
Date: Tue, 28 Sep 2021 10:28:42 +0200
Subject: please help with ghc package-db flags
In-Reply-To: <20210927205042.133412C307E@homedog.cs.tufts.edu>
References: <20210927205042.133412C307E@homedog.cs.tufts.edu>
Message-ID: <9517f599-a701-fabb-fe32-0a134f8870b5@haskus.fr>

Hi,

Could you try with `-package-env -` to disable package environment 
misfeature. Especially with Asterius as it may mix packages for the host 
with target packages that use WebAssembly... (I have been bitten by this 
2 years ago IIRC).

Sylvain


On 27/09/2021 22:50, Norman Ramsey wrote:
> I've traced some troubles to a problem with GHC's response
> to the -clear-package-db and -package-db flags.  I would very much
> like to know if others can duplicate this issue.
>
> All that is needed is for you to try the following commands, or
> whatever variations may be appropriate for whatever ghc versions you
> have installed on your system:
>
>    ghc-pkg init /tmp/empty-package-db
>    ghc        -clear-package-db -package-db /tmp/empty-package-db/ -v
>    ghc-9.0.1  -clear-package-db -package-db /tmp/empty-package-db/ -v
>    ghc-8.10.7 -clear-package-db -package-db /tmp/empty-package-db/ -v
>
> On my system at this present moment, ghc-8.10.7 respects the commands,
> but ghc-9.0.1 does not.  The output from the `-v` options show what's
> happening: if GHC finds any packages that are *not* "wired-in," then
> something is broken.  As examples, I attach the sample outputs from my
> own system.
>
> I think the issue is being caused by something mysterious in my filesystem.
> Last Friday, ghc 9.0.1 was respecting those flags.  But today it is not.
> Before I try to figure out what is going on, I would *very* much
> appreciate learning if anyone else can duplicate the issue.
>
> Please try running GHC with an empty package database and let me know
> what happens.
>
>
> Norman
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210928/4c7b0da7/attachment.html>

From nr at cs.tufts.edu  Tue Sep 28 13:36:08 2021
From: nr at cs.tufts.edu (Norman Ramsey)
Date: Tue, 28 Sep 2021 09:36:08 -0400
Subject: Two questions about building GHC from sources
In-Reply-To: <CAKfdd-wo3nGQKR_h3b7o5X4CfznOXtcoM5cD56OP--DNdEm2yA@mail.gmail.com>
 (sfid-H-20210922-094739-+84.39-1@multi.osbf.lua)
References: <20210922132534.12DBC2C3331@homedog.cs.tufts.edu>
 <sfid-H-20210922-094739-+84.39-1@multi.osbf.lua>
 <CAKfdd-wo3nGQKR_h3b7o5X4CfznOXtcoM5cD56OP--DNdEm2yA@mail.gmail.com>
 (sfid-H-20210922-094739-+84.39-1@multi.osbf.lua)
Message-ID: <20210928133608.9BEA62C281F@homedog.cs.tufts.edu>

 > >  1. If my bootstrap compiler changes (e.g., from 8.10.5 to 9.0.1), is
 > >     Hadrian smart enough to rebuild everything?  If not, how do I
 > >     force it to start over?
 > 
 > (1), I would hope so. If not, that's a bug.

I have confirmed that when the bootstrap compiler changes, Hadrian
cannot successfully rebuild GHC.  Reproducing the fault takes 11
steps, one of which is a full GHC build, so it's a bit time-consuming.
The second build fails with an uncaught exception in Shake.

I've reported the bug as issue #20429.


Norman

From nr at cs.tufts.edu  Tue Sep 28 23:36:28 2021
From: nr at cs.tufts.edu (Norman Ramsey)
Date: Tue, 28 Sep 2021 19:36:28 -0400
Subject: SOLVED: GHC 9.3 API panics with "No home unit"
In-Reply-To: <20210924193851.980B32C2EEC@homedog.cs.tufts.edu>
 (sfid-H-20210924-153853-+64.77-1@multi.osbf.lua)
References: <sfid-H-20210924-153853-+64.77-1@multi.osbf.lua>
 <20210924193851.980B32C2EEC@homedog.cs.tufts.edu>
 (sfid-H-20210924-153853-+64.77-1@multi.osbf.lua)
Message-ID: <20210928233628.B16662C2812@homedog.cs.tufts.edu>

 > The code worked in the 9.0 API, but using the 9.3 API causes a panic:
 > 
 >   sandbox: panic! (the 'impossible' happened)
 >     GHC version 9.3.20210918:
 >           unsafeGetHomeUnit: No home unit

SOLVED: I was getting an `HscEnv` by using `newHscEnv` with a `DynFlags`.
Turns out I'm not supposed to do that; using `getSession` instead
solved the problem.  Thanks to Cheng Shao for the diagnosis!

If there is something I can read to help me avoid similar mistakes in
the future, please point me in its direction!


Norman

From simonpj at microsoft.com  Wed Sep 29 08:06:16 2021
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Wed, 29 Sep 2021 08:06:16 +0000
Subject: SOLVED: GHC 9.3 API panics with "No home unit"
In-Reply-To: <20210928233628.B16662C2812@homedog.cs.tufts.edu>
References: <sfid-H-20210924-153853-+64.77-1@multi.osbf.lua>
 <20210924193851.980B32C2EEC@homedog.cs.tufts.edu>
 (sfid-H-20210924-153853-+64.77-1@multi.osbf.lua)
 <20210928233628.B16662C2812@homedog.cs.tufts.edu>
Message-ID: <VI1PR83MB0269D07C4AFBE7377AD88465ADA99@VI1PR83MB0269.EURPRD83.prod.outlook.com>

Norman,

I'm glad you are unstuck.

We badly need a bird's eye view of the GHC API, with chapters about specific bits. It shouldn't be a question of trial an error.  But it evolved rather than being designed, and since then everyone's shortest path to completion is simply to find a way through and execute on it.

If you felt able to write down what you have learned, we could make it part of GHC's user manual, and improve it incrementally.   I know that's not on your shortest path to completion either, but you have particularly superior writing skills, and I think you might find that others join in if you took the lead.

Also then you could say "but why do you need to do getSession not newHscEnv, and we might be able to fill in some of the rationale rather than just write down cookbook recipes.  And it might lead to design changes too, I hope.

Posting here in the hope that others say "oh yes, I'd love to help with that"! 

Simon


PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com will cease to work.  Use simon.peytonjones at gmail.com instead.  (For now, it just forwards to simonpj at microsoft.com.)

|  -----Original Message-----
|  From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Norman
|  Ramsey
|  Sent: 29 September 2021 00:36
|  Cc: ghc-devs at haskell.org
|  Subject: SOLVED: GHC 9.3 API panics with "No home unit"
|  
|   > The code worked in the 9.0 API, but using the 9.3 API causes a
|  panic:
|   >
|   >   sandbox: panic! (the 'impossible' happened)
|   >     GHC version 9.3.20210918:
|   >           unsafeGetHomeUnit: No home unit
|  
|  SOLVED: I was getting an `HscEnv` by using `newHscEnv` with a
|  `DynFlags`.
|  Turns out I'm not supposed to do that; using `getSession` instead
|  solved the problem.  Thanks to Cheng Shao for the diagnosis!
|  
|  If there is something I can read to help me avoid similar mistakes in
|  the future, please point me in its direction!
|  
|  
|  Norman
|  _______________________________________________
|  ghc-devs mailing list
|  ghc-devs at haskell.org
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.
|  haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&amp;data=04%7C01%7Csimonpj%40microsoft.com%7C96b747b71fd6484be28c
|  08d982d8f526%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637684691082
|  936473%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJ
|  BTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=A4Mw547%2FzPS1w6WZ%2BkR11
|  f%2F5GMxccIUOqcCPjvIRXVk%3D&amp;reserved=0