From lexi.lambda at gmail.com  Tue Oct  1 22:46:23 2019
From: lexi.lambda at gmail.com (Alexis King)
Date: Tue, 1 Oct 2019 17:46:23 -0500
Subject: Compiling projects with dependencies using GHC HEAD
In-Reply-To: <CAA8dsac7ZbbXr5K8KQaVVJvBfM6cPROow3MTZ2f8VVpsUCassw@mail.gmail.com>
References: <CAA8dsac7ZbbXr5K8KQaVVJvBfM6cPROow3MTZ2f8VVpsUCassw@mail.gmail.com>
Message-ID: <CAA8dsae09WReyY3gi1+NkLSuvK3Dk14ZHGLYXfkrEehgwcrPrA@mail.gmail.com>

Hi all,

I have a project that I’d like to try to build with GHC HEAD, but it’s a
large project with a lot of dependencies, and while there’s a lot of
information on building GHC itself, I haven’t been able to find very many
resources on what workflow people use to actually build projects in the
wild using it. I found this wiki page, “Installing packages in your test
compiler,” which has a little guidance, but the details are sparse:

https://gitlab.haskell.org/ghc/ghc/wikis/debugging/installing-packages-inplace


However, I’ve been struggling to actually get my project to build using
cabal-install paired with a stage 2 compiler. I realize some of this is
just fundamental—various libraries need to be updated for new versions of
GHC—but I’m willing to go to the effort to manually resolve those conflicts
so long as I know the right way to go about doing it.

One simple thing I did try (and failed at) is building an updated version
of cabal-install itself using my updated GHC. From inside my checkout of
the GHC repo, I tried the following:

$ cd libraries/Cabal/cabal-install
$ cabal --version
cabal --version
cabal-install version 2.4.1.0
compiled using version 2.4.1.0 of the Cabal library
$ cabal --with-compiler=<path to ghc> --package-db=<path to package db> \
    new-build cabal-install


However, this failed to compile due to out-of-bounds versions of certain
packages, so I tried again with --allow-newer:

$ cabal --with-compiler=<path to ghc> --package-db=<path to package db> \
    new-build cabal-install --allow-newer=base,ghc-prim,template-haskell


This time, it managed to produce a build plan, but for reasons I don’t
understand, both Cabal-3.1.0.0 and Cabal-2.4.1.0 are in the build plan, and
the latter fails to compile against newer versions of base (due to the
MonadFail changes). I tried to figure out what was causing Cabal-2.4.1.0 to
end up in the build plan, but running new-build with the -v3 option didn’t
seem to give me any more explanation.

Apologies if this is really a request for support with cabal-install, and
it doesn’t belong on this list, but I figured I’d start here, since I
imagine the people on this list are best-equipped to answer questions
involving using GHC HEAD. I’d appreciate any pointers or resources anyone
has!

Thanks,
Alexis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191001/8c929769/attachment.html>

From ben at well-typed.com  Tue Oct  1 23:19:59 2019
From: ben at well-typed.com (Ben Gamari)
Date: Tue, 01 Oct 2019 19:19:59 -0400
Subject: Compiling projects with dependencies using GHC HEAD
In-Reply-To: <CAA8dsae09WReyY3gi1+NkLSuvK3Dk14ZHGLYXfkrEehgwcrPrA@mail.gmail.com>
References: <CAA8dsac7ZbbXr5K8KQaVVJvBfM6cPROow3MTZ2f8VVpsUCassw@mail.gmail.com>
 <CAA8dsae09WReyY3gi1+NkLSuvK3Dk14ZHGLYXfkrEehgwcrPrA@mail.gmail.com>
Message-ID: <87sgochx47.fsf@smart-cactus.org>

Alexis King <lexi.lambda at gmail.com> writes:

> Hi all,
>
> I have a project that I’d like to try to build with GHC HEAD, but it’s a
> large project with a lot of dependencies, and while there’s a lot of
> information on building GHC itself, I haven’t been able to find very many
> resources on what workflow people use to actually build projects in the
> wild using it. I found this wiki page, “Installing packages in your test
> compiler,” which has a little guidance, but the details are sparse:
>
...
>
> Apologies if this is really a request for support with cabal-install, and
> it doesn’t belong on this list, but I figured I’d start here, since I
> imagine the people on this list are best-equipped to answer questions
> involving using GHC HEAD. I’d appreciate any pointers or resources anyone
> has!
>
Hi Alexis,

Your question is a very good one which, until quite recently, there was
not a particularly good answer to. However, in the past months we have
been working on infrastructure (in the form of head.hackage [0]) to make
GHC more testable in pre-release form.

I have a draft blog post explaining the currently state-of-play
here [1,2]. Unfortunately between ICFP, release things, vacation, and bug
fixing I've lacked the time to finish these off (the tutorial in particular).

Also, note that this is still just infrastructure. While head.hackage
does include a reasonable set of patches, it would be quite surprising
if it has everything your project requires. Merge requests contributing
new patches are highly encouraged.

Cheers,

- Ben


[0] https://gitlab.haskell.org/ghc/head.hackage/merge_requests
[1] https://gitlab.haskell.org/ghc/homepage/merge_requests/16
[2] https://gitlab.haskell.org/ghc/homepage/merge_requests/29

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191001/c218f498/attachment.sig>

From lexi.lambda at gmail.com  Tue Oct  1 23:45:05 2019
From: lexi.lambda at gmail.com (Alexis King)
Date: Tue, 1 Oct 2019 18:45:05 -0500
Subject: Compiling projects with dependencies using GHC HEAD
In-Reply-To: <87sgochx47.fsf@smart-cactus.org>
References: <CAA8dsac7ZbbXr5K8KQaVVJvBfM6cPROow3MTZ2f8VVpsUCassw@mail.gmail.com>
 <CAA8dsae09WReyY3gi1+NkLSuvK3Dk14ZHGLYXfkrEehgwcrPrA@mail.gmail.com>
 <87sgochx47.fsf@smart-cactus.org>
Message-ID: <FF1D20FD-FB3A-4230-A706-D6B53BFFFD2C@gmail.com>

> On Oct 1, 2019, at 18:19, Ben Gamari <ben at well-typed.com> wrote:
> 
> Your question is a very good one which, until quite recently, there was
> not a particularly good answer to. However, in the past months we have
> been working on infrastructure (in the form of head.hackage [0]) to make
> GHC more testable in pre-release form.
> 
> I have a draft blog post explaining the currently state-of-play
> here [1,2]. Unfortunately between ICFP, release things, vacation, and bug
> fixing I've lacked the time to finish these off (the tutorial in particular).

Hi Ben,

Many thanks for your response—this infrastructure looks incredibly helpful! I’ll give it a try. One question I do have just from giving it a look involves the cabal.constraints file available here:

https://ghc.gitlab.haskell.org/head.hackage/cabal.constraints <https://ghc.gitlab.haskell.org/head.hackage/cabal.constraints>

A number of packages appear to be listed twice, and some of them (but not all of them) are pinned to different versions. Is that intentional? Or is it just a part of still being a bit of a work in progress?

In any case, this gets me quite a bit further, and though I’m not sure yet if my project will actually build, I can now at least construct a valid build plan. Thanks again!

Alexis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191001/786cd850/attachment.html>

From ben at well-typed.com  Tue Oct  1 23:56:11 2019
From: ben at well-typed.com (Ben Gamari)
Date: Tue, 01 Oct 2019 19:56:11 -0400
Subject: Compiling projects with dependencies using GHC HEAD
In-Reply-To: <FF1D20FD-FB3A-4230-A706-D6B53BFFFD2C@gmail.com>
References: <CAA8dsac7ZbbXr5K8KQaVVJvBfM6cPROow3MTZ2f8VVpsUCassw@mail.gmail.com>
 <CAA8dsae09WReyY3gi1+NkLSuvK3Dk14ZHGLYXfkrEehgwcrPrA@mail.gmail.com>
 <87sgochx47.fsf@smart-cactus.org>
 <FF1D20FD-FB3A-4230-A706-D6B53BFFFD2C@gmail.com>
Message-ID: <87pnjghvfu.fsf@smart-cactus.org>

Alexis King <lexi.lambda at gmail.com> writes:

>> On Oct 1, 2019, at 18:19, Ben Gamari <ben at well-typed.com> wrote:
>> 
>> Your question is a very good one which, until quite recently, there was
>> not a particularly good answer to. However, in the past months we have
>> been working on infrastructure (in the form of head.hackage [0]) to make
>> GHC more testable in pre-release form.
>> 
>> I have a draft blog post explaining the currently state-of-play
>> here [1,2]. Unfortunately between ICFP, release things, vacation, and bug
>> fixing I've lacked the time to finish these off (the tutorial in particular).
>
> Hi Ben,
>
> Many thanks for your response—this infrastructure looks incredibly helpful! I’ll give it a try. One question I do have just from giving it a look involves the cabal.constraints file available here:
>
> https://ghc.gitlab.haskell.org/head.hackage/cabal.constraints <https://ghc.gitlab.haskell.org/head.hackage/cabal.constraints>
>
> A number of packages appear to be listed twice, and some of them (but
> not all of them) are pinned to different versions. Is that
> intentional? Or is it just a part of still being a bit of a work in
> progress?
>
Indeed it's just a function of this being a bit work in progress; we
only started allowing multiple patch versions relatively recently and it
looks like the constraint file generation hasn't been updated to account
for this yet. I'll try to fix that tonight.

> In any case, this gets me quite a bit further, and though I’m not sure
> yet if my project will actually build, I can now at least construct a
> valid build plan. Thanks again!
>
Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191001/0c1a0994/attachment.sig>

From ben at well-typed.com  Wed Oct  2 15:34:10 2019
From: ben at well-typed.com (Ben Gamari)
Date: Wed, 02 Oct 2019 11:34:10 -0400
Subject: GitLab upgrade soon
Message-ID: <87k19ni2kx.fsf@smart-cactus.org>

Hi everyone,

In about 30 minutes I will start an upgrade of gitlab.haskell.org. I
expect it shouldn't take long but there will be a few minutes of outage.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191002/74d3f4b4/attachment.sig>

From ben at well-typed.com  Wed Oct  2 16:01:29 2019
From: ben at well-typed.com (Ben Gamari)
Date: Wed, 02 Oct 2019 12:01:29 -0400
Subject: GitLab upgrade soon
In-Reply-To: <87k19ni2kx.fsf@smart-cactus.org>
References: <87k19ni2kx.fsf@smart-cactus.org>
Message-ID: <87h84ri1bk.fsf@smart-cactus.org>

Ben Gamari <ben at well-typed.com> writes:

> Hi everyone,
>
> In about 30 minutes I will start an upgrade of gitlab.haskell.org. I
> expect it shouldn't take long but there will be a few minutes of outage.
>
Upgrade started.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191002/eb07d429/attachment.sig>

From sylvain at haskus.fr  Wed Oct  2 16:42:25 2019
From: sylvain at haskus.fr (Sylvain Henry)
Date: Wed, 2 Oct 2019 18:42:25 +0200
Subject: GHC's module hierarchy
In-Reply-To: <SN1PR21MB00317365F828C8E62B9932D8ADC00@SN1PR21MB0031.namprd21.prod.outlook.com>
References: <SN1PR21MB00317365F828C8E62B9932D8ADC00@SN1PR21MB0031.namprd21.prod.outlook.com>
Message-ID: <de145375-6dd0-44a7-018a-75b94f10454a@haskus.fr>

Hi all,

We are back at considering an overhaul of the module structure of GHC. 
Ticket #13009 <https://gitlab.haskell.org/ghc/ghc/issues/13009> [1] is 
the place where the discussion takes place: this is a call for 
participation to this discussion!

Thanks,
Sylvain

PS: this work was supposed to be step 1 of a larger effort to make GHC 
more modular. See the wiki page 
<https://gitlab.haskell.org/ghc/ghc/wikis/Make-GHC-codebase-more-modular> 
[2] for more details and don't hesitate to give some feedback.

[1] https://gitlab.haskell.org/ghc/ghc/issues/13009
[2] https://gitlab.haskell.org/ghc/ghc/wikis/Make-GHC-codebase-more-modular


On 15/06/2017 09:41, Simon Peyton Jones via ghc-devs wrote:
>
> Dear ghc-devs
>
> hsyl20 proposes a radical overhaul of the module structure of GHC 
> itself.  He or she suggested it six months ago in
>
> https://ghc.haskell.org/trac/ghc/ticket/13009 
> <https://ghc.haskell.org/trac/ghc/ticket/13009>
>
> and has now offered a monster patch
>
> https://phabricator.haskell.org/D3647 
> <https://phabricator.haskell.org/D3647>
>
> It’s clearly the result of a lot of work, but I was the only one who 
> responded on the original ticket, and it’ll affect all of your lives 
> in a very immediate way.
>
> So, would you like to
>
> ·consider the idea
>
> ·look at the actual re-mapping of modules hsyl20 proposals
>
> ·express an opinion about whether to go ahead
>
> Probably the ticket, rather than Phab, is the best place to comment on 
> the general idea.
>
> I’d like to thank hsyl20.  GHC’s rather flat module structure has 
> grown incrementally over years.
>
> But still, there are pros and cons.
>
> Simon
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191002/aed00494/attachment.html>

From ben at well-typed.com  Wed Oct  2 19:33:14 2019
From: ben at well-typed.com (Ben Gamari)
Date: Wed, 02 Oct 2019 15:33:14 -0400
Subject: GitLab upgrade soon
In-Reply-To: <87h84ri1bk.fsf@smart-cactus.org>
References: <87k19ni2kx.fsf@smart-cactus.org> <87h84ri1bk.fsf@smart-cactus.org>
Message-ID: <87eezvhrir.fsf@smart-cactus.org>

Ben Gamari <ben at well-typed.com> writes:

> Ben Gamari <ben at well-typed.com> writes:
>
>> Hi everyone,
>>
>> In about 30 minutes I will start an upgrade of gitlab.haskell.org. I
>> expect it shouldn't take long but there will be a few minutes of outage.
>>
> Upgrade started.

Well, that was significantly harder than expected but it's now done and
things seem stable.

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191002/eb20fa62/attachment.sig>

From takenobu.hs at gmail.com  Thu Oct  3 13:01:36 2019
From: takenobu.hs at gmail.com (Takenobu Tani)
Date: Thu, 3 Oct 2019 22:01:36 +0900
Subject: SPAM snippets on GitLab
Message-ID: <CAPB_NvzJvrg0fgBVrOm9VGitD64_-Ny=p15nT=SLOMRtci-WCQ@mail.gmail.com>

Hi Ben, devs,

It looks like GHC gitlab snipeets [1] are exposed to spam.
Could we control them?

[1] https://gitlab.haskell.org/explore/snippets

Regards,
Takenobu

From ben at smart-cactus.org  Thu Oct  3 13:37:54 2019
From: ben at smart-cactus.org (Ben Gamari)
Date: Thu, 03 Oct 2019 09:37:54 -0400
Subject: SPAM snippets on GitLab
In-Reply-To: <CAPB_NvzJvrg0fgBVrOm9VGitD64_-Ny=p15nT=SLOMRtci-WCQ@mail.gmail.com>
References: <CAPB_NvzJvrg0fgBVrOm9VGitD64_-Ny=p15nT=SLOMRtci-WCQ@mail.gmail.com>
Message-ID: <87bluyhrv9.fsf@smart-cactus.org>

Takenobu Tani <takenobu.hs at gmail.com> writes:

> Hi Ben, devs,
>
> It looks like GHC gitlab snipeets [1] are exposed to spam.
> Could we control them?
>
> [1] https://gitlab.haskell.org/explore/snippets
>
Thanks Takenobu!

In general feel free to use the "Report Spam" button in the top right
corner of the page when you see something like this. I do periodically
look through the spam report queue and deleting things as necessary
(although admittedly the last time was a few months ago).

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191003/517d6675/attachment.sig>

From takenobu.hs at gmail.com  Thu Oct  3 13:49:37 2019
From: takenobu.hs at gmail.com (Takenobu Tani)
Date: Thu, 3 Oct 2019 22:49:37 +0900
Subject: SPAM snippets on GitLab
In-Reply-To: <87bluyhrv9.fsf@smart-cactus.org>
References: <CAPB_NvzJvrg0fgBVrOm9VGitD64_-Ny=p15nT=SLOMRtci-WCQ@mail.gmail.com>
 <87bluyhrv9.fsf@smart-cactus.org>
Message-ID: <CAPB_NvyoVyfGdSCri8DYCBjNYcM+KpqF0W4AZrN9Wq11Qf0vgA@mail.gmail.com>

Thanks for explain me.
Next time I'll do it :)

Regards,
Takenobu


On Thu, Oct 3, 2019 at 10:37 PM Ben Gamari <ben at smart-cactus.org> wrote:
>
> Takenobu Tani <takenobu.hs at gmail.com> writes:
>
> > Hi Ben, devs,
> >
> > It looks like GHC gitlab snipeets [1] are exposed to spam.
> > Could we control them?
> >
> > [1] https://gitlab.haskell.org/explore/snippets
> >
> Thanks Takenobu!
>
> In general feel free to use the "Report Spam" button in the top right
> corner of the page when you see something like this. I do periodically
> look through the spam report queue and deleting things as necessary
> (although admittedly the last time was a few months ago).
>
> Cheers,
>
> - Ben

From matthewtpickering at gmail.com  Fri Oct  4 15:57:08 2019
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Fri, 4 Oct 2019 16:57:08 +0100
Subject: How to work out why a data constructor is allocated using gdb?
In-Reply-To: <CALuQ0m8++i2hYMOxWXeUAyR4iG+BCEtaTsR_oNi5h5H9ew_B9g@mail.gmail.com>
References: <CALuQ0m8++i2hYMOxWXeUAyR4iG+BCEtaTsR_oNi5h5H9ew_B9g@mail.gmail.com>
Message-ID: <CALuQ0m-RxR-G+xDE-A44nYFnE9ybe3KT-6X4W_UB8qUO_R5iaw@mail.gmail.com>

It occurred to me today that in order to work out where a constructor
is allocated from you can look in the profiling header.

The profiling header contains information about the cost centre stack
at the precise point of allocation. You should be able
to look at this to see what the call stack was and the source position
of the cost centre at the bottom of the stack.

Cheers,

Matt

On Tue, Aug 13, 2019 at 8:29 AM Matthew Pickering
<matthewtpickering at gmail.com> wrote:
>
> Hi,
>
> I am trying to work out questions such as
>
> * Why are there thousands of Module data constructors allocated when
> building something with GHC
> * What is allocating all a lot of strings when building GHC?
>
> In order to do this I can use gdb in order to find some of the
> Module/String closures but then I'm a bit stuck about what to do.
>
> gdb displays a list of all the Module closures for example and then
> you can usually try to find the retainer for Module by using findPtr.
>
> If the retainer is a THUNK closure, it would be easy, as THUNK
> closures have DWARF information which maps straight to a particular
> line. However if the retainer is just some other
> data constructor, for example, the Module is stored in a Map, it's
> data constructors all the way up and none of them have DWARF info. I
> need to fall back to domain specific knowledge
> to work out where such a sequence of constructors might appear in my program.
>
> * Is there anything better I can do to map a constructor allocation to
> a more precise source location?
>
> The string closures were causing me some particular issues as
> `findPtr` was not showing any retainers so it's hard to work out why
> they are not GCd.
>
> * What situations can an object be retained but show no retainer when
> using findPtr?
>
> Cheers,
>
> Matt

From matthewtpickering at gmail.com  Fri Oct  4 16:06:21 2019
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Fri, 4 Oct 2019 17:06:21 +0100
Subject: How to work out why a data constructor is allocated using gdb?
In-Reply-To: <CALuQ0m-RxR-G+xDE-A44nYFnE9ybe3KT-6X4W_UB8qUO_R5iaw@mail.gmail.com>
References: <CALuQ0m8++i2hYMOxWXeUAyR4iG+BCEtaTsR_oNi5h5H9ew_B9g@mail.gmail.com>
 <CALuQ0m-RxR-G+xDE-A44nYFnE9ybe3KT-6X4W_UB8qUO_R5iaw@mail.gmail.com>
Message-ID: <CALuQ0m-m1njrKUNdu3BjHaoGHRcqQzGfWBFvQeTTzL=fv+siJQ@mail.gmail.com>

In the unlikely situation that anyone cares about this or attempts to
try to do it.  If you combine together `-prof` and `-debug` invoking
any heap profiling mode will take about `10x` longer than usual.

So you can still use `-prof` in order to populate the profiling
headers but don't also try to perform a heap census.

Cheers,

Matt


On Fri, Oct 4, 2019 at 4:57 PM Matthew Pickering
<matthewtpickering at gmail.com> wrote:
>
> It occurred to me today that in order to work out where a constructor
> is allocated from you can look in the profiling header.
>
> The profiling header contains information about the cost centre stack
> at the precise point of allocation. You should be able
> to look at this to see what the call stack was and the source position
> of the cost centre at the bottom of the stack.
>
> Cheers,
>
> Matt
>
> On Tue, Aug 13, 2019 at 8:29 AM Matthew Pickering
> <matthewtpickering at gmail.com> wrote:
> >
> > Hi,
> >
> > I am trying to work out questions such as
> >
> > * Why are there thousands of Module data constructors allocated when
> > building something with GHC
> > * What is allocating all a lot of strings when building GHC?
> >
> > In order to do this I can use gdb in order to find some of the
> > Module/String closures but then I'm a bit stuck about what to do.
> >
> > gdb displays a list of all the Module closures for example and then
> > you can usually try to find the retainer for Module by using findPtr.
> >
> > If the retainer is a THUNK closure, it would be easy, as THUNK
> > closures have DWARF information which maps straight to a particular
> > line. However if the retainer is just some other
> > data constructor, for example, the Module is stored in a Map, it's
> > data constructors all the way up and none of them have DWARF info. I
> > need to fall back to domain specific knowledge
> > to work out where such a sequence of constructors might appear in my program.
> >
> > * Is there anything better I can do to map a constructor allocation to
> > a more precise source location?
> >
> > The string closures were causing me some particular issues as
> > `findPtr` was not showing any retainers so it's hard to work out why
> > they are not GCd.
> >
> > * What situations can an object be retained but show no retainer when
> > using findPtr?
> >
> > Cheers,
> >
> > Matt

From omeragacan at gmail.com  Fri Oct  4 20:01:14 2019
From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=)
Date: Fri, 4 Oct 2019 23:01:14 +0300
Subject: How to work out why a data constructor is allocated using gdb?
In-Reply-To: <CALuQ0m-m1njrKUNdu3BjHaoGHRcqQzGfWBFvQeTTzL=fv+siJQ@mail.gmail.com>
References: <CALuQ0m8++i2hYMOxWXeUAyR4iG+BCEtaTsR_oNi5h5H9ew_B9g@mail.gmail.com>
 <CALuQ0m-RxR-G+xDE-A44nYFnE9ybe3KT-6X4W_UB8qUO_R5iaw@mail.gmail.com>
 <CALuQ0m-m1njrKUNdu3BjHaoGHRcqQzGfWBFvQeTTzL=fv+siJQ@mail.gmail.com>
Message-ID: <CAMQQO3mQY-A4tUjYUbSNXrJSR2BxjWRE1Py+T_TjHySyt9or3g@mail.gmail.com>

When I'm interested in only one specific object I add a watchpoint to the
object's header and then do reverse execution. If it stops at mutator code then
I inspect Haskell stack to figure out the Haskell code that it's currently
executing (this is currently quite hard, !1654 helps a lot but it can't be done
on the whole compiler and libraries, the compiler should be booted without it
and then programs can be built with !1654).

If it stops at GC code then I add a conditional breakpoint after line `q =
UNTAG_CLOSURE(q);` in evacuate() to stop if `q` is the object I'm looking for.
That gives me the previous location of the object, and I repeat the process (add
watchpoint to the header, reverse-continue ...).

It's a painful process that can be automated to some extent, for example I once
added a print statement to copy_tag, and wrote a script that maps all locations
of an object during its lifetime using the print statement.

Ömer

Matthew Pickering <matthewtpickering at gmail.com>, 4 Eki 2019 Cum, 19:06
tarihinde şunu yazdı:
>
> In the unlikely situation that anyone cares about this or attempts to
> try to do it.  If you combine together `-prof` and `-debug` invoking
> any heap profiling mode will take about `10x` longer than usual.
>
> So you can still use `-prof` in order to populate the profiling
> headers but don't also try to perform a heap census.
>
> Cheers,
>
> Matt
>
>
> On Fri, Oct 4, 2019 at 4:57 PM Matthew Pickering
> <matthewtpickering at gmail.com> wrote:
> >
> > It occurred to me today that in order to work out where a constructor
> > is allocated from you can look in the profiling header.
> >
> > The profiling header contains information about the cost centre stack
> > at the precise point of allocation. You should be able
> > to look at this to see what the call stack was and the source position
> > of the cost centre at the bottom of the stack.
> >
> > Cheers,
> >
> > Matt
> >
> > On Tue, Aug 13, 2019 at 8:29 AM Matthew Pickering
> > <matthewtpickering at gmail.com> wrote:
> > >
> > > Hi,
> > >
> > > I am trying to work out questions such as
> > >
> > > * Why are there thousands of Module data constructors allocated when
> > > building something with GHC
> > > * What is allocating all a lot of strings when building GHC?
> > >
> > > In order to do this I can use gdb in order to find some of the
> > > Module/String closures but then I'm a bit stuck about what to do.
> > >
> > > gdb displays a list of all the Module closures for example and then
> > > you can usually try to find the retainer for Module by using findPtr.
> > >
> > > If the retainer is a THUNK closure, it would be easy, as THUNK
> > > closures have DWARF information which maps straight to a particular
> > > line. However if the retainer is just some other
> > > data constructor, for example, the Module is stored in a Map, it's
> > > data constructors all the way up and none of them have DWARF info. I
> > > need to fall back to domain specific knowledge
> > > to work out where such a sequence of constructors might appear in my program.
> > >
> > > * Is there anything better I can do to map a constructor allocation to
> > > a more precise source location?
> > >
> > > The string closures were causing me some particular issues as
> > > `findPtr` was not showing any retainers so it's hard to work out why
> > > they are not GCd.
> > >
> > > * What situations can an object be retained but show no retainer when
> > > using findPtr?
> > >
> > > Cheers,
> > >
> > > Matt
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From lexi.lambda at gmail.com  Sat Oct  5 00:21:59 2019
From: lexi.lambda at gmail.com (Alexis King)
Date: Fri, 4 Oct 2019 19:21:59 -0500
Subject: Compiling projects with dependencies using GHC HEAD
In-Reply-To: <87pnjghvfu.fsf@smart-cactus.org>
References: <CAA8dsac7ZbbXr5K8KQaVVJvBfM6cPROow3MTZ2f8VVpsUCassw@mail.gmail.com>
 <CAA8dsae09WReyY3gi1+NkLSuvK3Dk14ZHGLYXfkrEehgwcrPrA@mail.gmail.com>
 <87sgochx47.fsf@smart-cactus.org>
 <FF1D20FD-FB3A-4230-A706-D6B53BFFFD2C@gmail.com>
 <87pnjghvfu.fsf@smart-cactus.org>
Message-ID: <1B5D03B7-DD4E-4394-B3EE-0BD0C1E58A7B@gmail.com>

> On Oct 1, 2019, at 18:56, Ben Gamari <ben at well-typed.com> wrote:
> 
> Indeed it's just a function of this being a bit work in progress; we
> only started allowing multiple patch versions relatively recently and it
> looks like the constraint file generation hasn't been updated to account
> for this yet. I'll try to fix that tonight.

Thanks for the clarification. I just wanted to follow up and say that this was indeed enough to get me off the ground, and I was eventually able to successfully compile my project with GHC HEAD after updating a few other dependencies for MonadFail compatibility. Thanks again for your efforts!

Alexis

From michal.terepeta at gmail.com  Sat Oct  5 14:16:58 2019
From: michal.terepeta at gmail.com (Michal Terepeta)
Date: Sat, 5 Oct 2019 16:16:58 +0200
Subject: Coercible questions
Message-ID: <CAEN7k7=4vRpeqP4d71gqgiq4=CF46hP34QJ7H9jrhr3Gsd8x1g@mail.gmail.com>

Hi,

I've started looking into using `Data.Coerce` (and the `Coercible`
type-class) for a personal project and was wondering why coercing between
`Int` and `Word` is not allowed? I don't see any fundamental reason why
this shouldn't work...

Perhaps, it's just a matter of GHC's implementation details leaking out?
IIRC internally GHC has separate `RuntimeRep`/`PrimRep` for a `Word#` and
for an `Int#`. If that's the case, would it make sense to unify these?
Their actual runtime representation should be the same and I'd expect most
(all?) of their differences should be attached to `PrimOp`s.

And that leads me to another question--what exactly goes wrong here:
```
data Foo = Foo Int#
data Bar = Bar Int#

test :: Bar
test = coerce (Foo 42#)
```
Which fails with: "Couldn't match representation of type ‘Foo’ with that of
‘Bar’ arising from a use of ‘coerce’"

Perhaps I'm just misunderstanding exactly how `Coercible` works?

Thanks in advance!

- Michal

PS. The ability to coerce through things like lists is amazing :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191005/f6b2b899/attachment.html>

From michal.terepeta at gmail.com  Sat Oct  5 16:29:55 2019
From: michal.terepeta at gmail.com (Michal Terepeta)
Date: Sat, 5 Oct 2019 18:29:55 +0200
Subject: Coercible questions
In-Reply-To: <CADRtjwagU4YNMRfB5yCQuqzS1nJxr=Fp-_dzOmkSoxu01Tji8A@mail.gmail.com>
References: <CAEN7k7=4vRpeqP4d71gqgiq4=CF46hP34QJ7H9jrhr3Gsd8x1g@mail.gmail.com>
 <CADRtjwagU4YNMRfB5yCQuqzS1nJxr=Fp-_dzOmkSoxu01Tji8A@mail.gmail.com>
Message-ID: <CAEN7k7kWYha7TzZLSwpYqp2O-cQJA=hcJoimn+Qp=+ipryk0yA@mail.gmail.com>

Adding +ghc-devs <ghc-devs at haskell.org> to continue the thread

Hi Sandy,

Thanks for the answer! Do you think there is some fundamental reason for
this? Or just a matter of implementing this in GHC? It seems to me that
this should work just fine as long as the runtime representation is the
same.

And a related question--is it safe to `unsafeCoerce` an `Int` to a `Word`?
The only reason for why this could be problematic that comes to my mind is
that there could be an assumption that different `data`s do not alias each
other (although `newtype`s can due to `Coercible` functionality). But I'm
not sure this is ever used by GHC? Are there any other reasons why this
could be problematic?

Thanks!

- Michal



On Sat, Oct 5, 2019 at 5:27 PM Sandy Maguire <sandy at sandymaguire.me> wrote:

> Hi Michal,
>
> Datas aren't coercible, only newtypes. This is why you can't coerce Ints
> and Words, and why Foo and Bar don't work.
>
> Sandy
>
> On Sat, Oct 5, 2019 at 4:17 PM Michal Terepeta <michal.terepeta at gmail.com>
> wrote:
>
>> Hi,
>>
>> I've started looking into using `Data.Coerce` (and the `Coercible`
>> type-class) for a personal project and was wondering why coercing between
>> `Int` and `Word` is not allowed? I don't see any fundamental reason why
>> this shouldn't work...
>>
>> Perhaps, it's just a matter of GHC's implementation details leaking out?
>> IIRC internally GHC has separate `RuntimeRep`/`PrimRep` for a `Word#` and
>> for an `Int#`. If that's the case, would it make sense to unify these?
>> Their actual runtime representation should be the same and I'd expect most
>> (all?) of their differences should be attached to `PrimOp`s.
>>
>> And that leads me to another question--what exactly goes wrong here:
>> ```
>> data Foo = Foo Int#
>> data Bar = Bar Int#
>>
>> test :: Bar
>> test = coerce (Foo 42#)
>> ```
>> Which fails with: "Couldn't match representation of type ‘Foo’ with that
>> of ‘Bar’ arising from a use of ‘coerce’"
>>
>> Perhaps I'm just misunderstanding exactly how `Coercible` works?
>>
>> Thanks in advance!
>>
>> - Michal
>>
>> PS. The ability to coerce through things like lists is amazing :)
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
>
> --
> I'm currently travelling the world, sleeping on people's couches and doing
> full-time collaboration on Haskell projects. If this seems interesting to
> you, please consider signing up as a host!
> https://isovector.github.io/erdos/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191005/319147f2/attachment.html>

From allbery.b at gmail.com  Sat Oct  5 16:51:34 2019
From: allbery.b at gmail.com (Brandon Allbery)
Date: Sat, 5 Oct 2019 12:51:34 -0400
Subject: Coercible questions
In-Reply-To: <CAEN7k7kWYha7TzZLSwpYqp2O-cQJA=hcJoimn+Qp=+ipryk0yA@mail.gmail.com>
References: <CAEN7k7=4vRpeqP4d71gqgiq4=CF46hP34QJ7H9jrhr3Gsd8x1g@mail.gmail.com>
 <CADRtjwagU4YNMRfB5yCQuqzS1nJxr=Fp-_dzOmkSoxu01Tji8A@mail.gmail.com>
 <CAEN7k7kWYha7TzZLSwpYqp2O-cQJA=hcJoimn+Qp=+ipryk0yA@mail.gmail.com>
Message-ID: <CAKFCL4W6kg_YcMdYEyWi1gAZydfOeAr78ZTx4HH74P3+s8eDbw@mail.gmail.com>

I think there's some work going on to expose the representations, which
would enable some ability to coerce. But possibly not this much, as they're
separate RuntimeReps so you don't combine signed and unsigned numbers
inadvertently; currently that's a little magical inside ghc iirc, with the
RuntimeRep the only way to distinguish at all and that vanishing
post-typechecking.

On Sat, Oct 5, 2019 at 12:30 PM Michal Terepeta <michal.terepeta at gmail.com>
wrote:

> Adding +ghc-devs <ghc-devs at haskell.org> to continue the thread
>
> Hi Sandy,
>
> Thanks for the answer! Do you think there is some fundamental reason for
> this? Or just a matter of implementing this in GHC? It seems to me that
> this should work just fine as long as the runtime representation is the
> same.
>
> And a related question--is it safe to `unsafeCoerce` an `Int` to a `Word`?
> The only reason for why this could be problematic that comes to my mind is
> that there could be an assumption that different `data`s do not alias each
> other (although `newtype`s can due to `Coercible` functionality). But I'm
> not sure this is ever used by GHC? Are there any other reasons why this
> could be problematic?
>
> Thanks!
>
> - Michal
>
>
>
> On Sat, Oct 5, 2019 at 5:27 PM Sandy Maguire <sandy at sandymaguire.me>
> wrote:
>
>> Hi Michal,
>>
>> Datas aren't coercible, only newtypes. This is why you can't coerce Ints
>> and Words, and why Foo and Bar don't work.
>>
>> Sandy
>>
>> On Sat, Oct 5, 2019 at 4:17 PM Michal Terepeta <michal.terepeta at gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I've started looking into using `Data.Coerce` (and the `Coercible`
>>> type-class) for a personal project and was wondering why coercing between
>>> `Int` and `Word` is not allowed? I don't see any fundamental reason why
>>> this shouldn't work...
>>>
>>> Perhaps, it's just a matter of GHC's implementation details leaking out?
>>> IIRC internally GHC has separate `RuntimeRep`/`PrimRep` for a `Word#` and
>>> for an `Int#`. If that's the case, would it make sense to unify these?
>>> Their actual runtime representation should be the same and I'd expect most
>>> (all?) of their differences should be attached to `PrimOp`s.
>>>
>>> And that leads me to another question--what exactly goes wrong here:
>>> ```
>>> data Foo = Foo Int#
>>> data Bar = Bar Int#
>>>
>>> test :: Bar
>>> test = coerce (Foo 42#)
>>> ```
>>> Which fails with: "Couldn't match representation of type ‘Foo’ with
>>> that of ‘Bar’ arising from a use of ‘coerce’"
>>>
>>> Perhaps I'm just misunderstanding exactly how `Coercible` works?
>>>
>>> Thanks in advance!
>>>
>>> - Michal
>>>
>>> PS. The ability to coerce through things like lists is amazing :)
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>>
>>
>> --
>> I'm currently travelling the world, sleeping on people's couches and
>> doing full-time collaboration on Haskell projects. If this seems
>> interesting to you, please consider signing up as a host!
>> https://isovector.github.io/erdos/
>>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>


-- 
brandon s allbery kf8nh
allbery.b at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191005/dcbcea31/attachment.html>

From b at chreekat.net  Sat Oct  5 18:30:08 2019
From: b at chreekat.net (Bryan Richter)
Date: Sat, 5 Oct 2019 21:30:08 +0300
Subject: Hadrian failure after worktree update
Message-ID: <8a1441db-4a26-f5eb-d0cf-255fea3eb9d7@chreekat.net>

Below is an error message I have seen a number of times after
rebasing my git worktree, i.e. this is how to "reproduce":

    git fetch origin
    git rebase origin/master
    git submodule update
    ./hadrian/build.sh

(Obviously some details are elided, and this is not a complete reproduction.)

I don't have many copies of the error, but I do remember seeing the
complaint about "No generator for .../PmOracle.hs" on other occasions.

Any idea what this comes from?

-Bryan

--- 8< ---

Error when running Shake build system:
  at action, called at src/Rules.hs:71:19 in main:Rules
  at need, called at src/Rules.hs:93:5 in main:Rules
* Depends on: _build/stage0/lib/package.conf.d/ghc-8.9.0.20190929.conf
  at need, called at src/Rules/Register.hs:113:5 in main:Rules.Register
* Depends on: _build/stage0/compiler/build/libHSghc-8.9.0.20190929.a
* Depends on: _build/stage0/compiler/build/GHCi.o
* Depends on: _build/stage0/compiler/build/GHCi.o _build/stage0/compiler/build/GHCi.hi
* Depends on: OracleQ (KeyValues ("_build/stage0/compiler/.dependencies","_build/stage0/compiler/build/GHCi.o"))
* Depends on: _build/stage0/compiler/.dependencies
* Depends on: _build/stage0/compiler/.dependencies.mk
  at need, called at src/Rules/Dependencies.hs:26:9 in main:Rules.Dependencies
* Depends on: _build/stage0/compiler/build/PmOracle.hs
  at error, called at src/Rules/Generate.hs:105:34 in main:Rules.Generate
* Raised the exception:
No generator for _build/stage0/compiler/build/PmOracle.hs.


From artem.pyanykh at gmail.com  Sat Oct  5 18:37:51 2019
From: artem.pyanykh at gmail.com (Artem Pyanykh)
Date: Sat, 5 Oct 2019 19:37:51 +0100
Subject: Hadrian failure after worktree update
In-Reply-To: <8a1441db-4a26-f5eb-d0cf-255fea3eb9d7@chreekat.net>
References: <8a1441db-4a26-f5eb-d0cf-255fea3eb9d7@chreekat.net>
Message-ID: <0EDE5E95-975D-4F23-B476-B2ED8B0DCE90@gmail.com>

Hi Bryan,

Could you try to run ./configure before hadrian/build.sh?

Best regards,
Artem Pyanykh

> On 5 Oct 2019, at 19:30, Bryan Richter <b at chreekat.net> wrote:
> 
> Below is an error message I have seen a number of times after
> rebasing my git worktree, i.e. this is how to "reproduce":
> 
>    git fetch origin
>    git rebase origin/master
>    git submodule update
>    ./hadrian/build.sh
> 
> (Obviously some details are elided, and this is not a complete reproduction.)
> 
> I don't have many copies of the error, but I do remember seeing the
> complaint about "No generator for .../PmOracle.hs" on other occasions.
> 
> Any idea what this comes from?
> 
> -Bryan
> 
> --- 8< ---
> 
> Error when running Shake build system:
>  at action, called at src/Rules.hs:71:19 in main:Rules
>  at need, called at src/Rules.hs:93:5 in main:Rules
> * Depends on: _build/stage0/lib/package.conf.d/ghc-8.9.0.20190929.conf
>  at need, called at src/Rules/Register.hs:113:5 in main:Rules.Register
> * Depends on: _build/stage0/compiler/build/libHSghc-8.9.0.20190929.a
> * Depends on: _build/stage0/compiler/build/GHCi.o
> * Depends on: _build/stage0/compiler/build/GHCi.o _build/stage0/compiler/build/GHCi.hi
> * Depends on: OracleQ (KeyValues ("_build/stage0/compiler/.dependencies","_build/stage0/compiler/build/GHCi.o"))
> * Depends on: _build/stage0/compiler/.dependencies
> * Depends on: _build/stage0/compiler/.dependencies.mk
>  at need, called at src/Rules/Dependencies.hs:26:9 in main:Rules.Dependencies
> * Depends on: _build/stage0/compiler/build/PmOracle.hs
>  at error, called at src/Rules/Generate.hs:105:34 in main:Rules.Generate
> * Raised the exception:
> No generator for _build/stage0/compiler/build/PmOracle.hs.
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From omeragacan at gmail.com  Sun Oct  6 09:27:46 2019
From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=)
Date: Sun, 6 Oct 2019 12:27:46 +0300
Subject: Should coercion binders (arguments or binders in patterns) be TyVars?
Message-ID: <CAMQQO3nExWLD+h5s=7q56O744ymQD_d_hDEb=3g3Hqru637Ejw@mail.gmail.com>

Hi,

I just realized that coercion binders are currently Ids and not TyVars (unlike
other type arguments). This means that we don't drop coercion binders in
CoreToStg. Example:

    {-# LANGUAGE ScopedTypeVariables, TypeOperators, PolyKinds, GADTs,
       TypeApplications, MagicHash #-}

    module UnsafeCoerce where

    import Data.Type.Equality ((:~:)(..))
    import GHC.Prim
    import GHC.Types

    unsafeEqualityProof :: forall k (a :: k) (b :: k) . a :~: b
    unsafeEqualityProof = error "unsafeEqualityProof evaluated"

    unsafeCoerce :: forall a b . a -> b
    unsafeCoerce x = case unsafeEqualityProof @_ @a @b of Refl -> x

If I build this with -ddump-stg this is what I get for `unsafeCoerce`:

    UnsafeCoerce.unsafeCoerce :: forall a b. a -> b
    [GblId, Arity=1, Unf=OtherCon []] =
        [] \r [x_s2jn]
            case UnsafeCoerce.unsafeEqualityProof of {
              Data.Type.Equality.Refl co_a2fd -> x_s2jn;
            };

See the binder in `Refl` pattern.

Unarise drops this binder because it's a "void" argument (doesn't have a runtime
representation), but still it's a bit weird that we drop types but not coercions
in CoreToStg.

Is this intentional?

Ömer

From sgraf1337 at gmail.com  Sun Oct  6 09:50:37 2019
From: sgraf1337 at gmail.com (Sebastian Graf)
Date: Sun, 6 Oct 2019 10:50:37 +0100
Subject: Should coercion binders (arguments or binders in patterns) be
 TyVars?
In-Reply-To: <CAMQQO3nExWLD+h5s=7q56O744ymQD_d_hDEb=3g3Hqru637Ejw@mail.gmail.com>
References: <CAMQQO3nExWLD+h5s=7q56O744ymQD_d_hDEb=3g3Hqru637Ejw@mail.gmail.com>
Message-ID: <CAAS+=P9_6+poiUJvYApXRD6L4n6sUEOMpO0bpGAd+yq1riJZiA@mail.gmail.com>

Hi Ömer,

I'm not sure if there's a case in GHC (yet, because newtype coercions are
zero-cost), but coercions in general (as introduced for example in Types
and Programming Languages) can carry computational content and thus can't
be erased.

Think of a hypothetical coercion `co :: Int ~ Double`; applying that
coercion as in `x |> co` to `x :: Int` would need to `fild` (load the
integer in a floating point register) at run-time, so you can't erase it.
The fact that we can for newtypes is because `coerce` is basically just the
`id` function at runtime.

Cheers,
Sebastian

Am So., 6. Okt. 2019 um 10:28 Uhr schrieb Ömer Sinan Ağacan <
omeragacan at gmail.com>:

> Hi,
>
> I just realized that coercion binders are currently Ids and not TyVars
> (unlike
> other type arguments). This means that we don't drop coercion binders in
> CoreToStg. Example:
>
>     {-# LANGUAGE ScopedTypeVariables, TypeOperators, PolyKinds, GADTs,
>        TypeApplications, MagicHash #-}
>
>     module UnsafeCoerce where
>
>     import Data.Type.Equality ((:~:)(..))
>     import GHC.Prim
>     import GHC.Types
>
>     unsafeEqualityProof :: forall k (a :: k) (b :: k) . a :~: b
>     unsafeEqualityProof = error "unsafeEqualityProof evaluated"
>
>     unsafeCoerce :: forall a b . a -> b
>     unsafeCoerce x = case unsafeEqualityProof @_ @a @b of Refl -> x
>
> If I build this with -ddump-stg this is what I get for `unsafeCoerce`:
>
>     UnsafeCoerce.unsafeCoerce :: forall a b. a -> b
>     [GblId, Arity=1, Unf=OtherCon []] =
>         [] \r [x_s2jn]
>             case UnsafeCoerce.unsafeEqualityProof of {
>               Data.Type.Equality.Refl co_a2fd -> x_s2jn;
>             };
>
> See the binder in `Refl` pattern.
>
> Unarise drops this binder because it's a "void" argument (doesn't have a
> runtime
> representation), but still it's a bit weird that we drop types but not
> coercions
> in CoreToStg.
>
> Is this intentional?
>
> Ömer
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191006/49da9ae3/attachment.html>

From omeragacan at gmail.com  Sun Oct  6 09:55:21 2019
From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=)
Date: Sun, 6 Oct 2019 12:55:21 +0300
Subject: Should coercion binders (arguments or binders in patterns) be
 TyVars?
In-Reply-To: <CAAS+=P9_6+poiUJvYApXRD6L4n6sUEOMpO0bpGAd+yq1riJZiA@mail.gmail.com>
References: <CAMQQO3nExWLD+h5s=7q56O744ymQD_d_hDEb=3g3Hqru637Ejw@mail.gmail.com>
 <CAAS+=P9_6+poiUJvYApXRD6L4n6sUEOMpO0bpGAd+yq1riJZiA@mail.gmail.com>
Message-ID: <CAMQQO3=9=urfcKsc7m1zUBL=moh_c1KvxkNWvJBTG1-0i1NZ7w@mail.gmail.com>

> I'm not sure if there's a case in GHC (yet, because newtype coercions are
> zero-cost), but coercions in general (as introduced for example in Types and
> Programming Languages) can carry computational content and thus can't be
> erased.

But they're already erased! They don't have a runtime representation and the
unarise pass eliminates coercion values and binders. See the last paragraph in
my original email.

> Think of a hypothetical coercion `co :: Int ~ Double`; applying that coercion
> as in `x |> co` to `x :: Int` would need to `fild` (load the integer in a
> floating point register) at run-time, so you can't erase it. The fact that we
> can for newtypes is because `coerce` is basically just the `id` function at
> runtime.

Do we even support such coercions? I think a `co :: Int ~ Double` can only be an
unsafe coercion, which are basically no-op in runtime, so again they can be
erased.

Let me know if I'm mistaken.

Ömer

Sebastian Graf <sgraf1337 at gmail.com>, 6 Eki 2019 Paz, 12:50 tarihinde
şunu yazdı:
>
> Hi Ömer,
>
> I'm not sure if there's a case in GHC (yet, because newtype coercions are zero-cost), but coercions in general (as introduced for example in Types and Programming Languages) can carry computational content and thus can't be erased.
>
> Think of a hypothetical coercion `co :: Int ~ Double`; applying that coercion as in `x |> co` to `x :: Int` would need to `fild` (load the integer in a floating point register) at run-time, so you can't erase it. The fact that we can for newtypes is because `coerce` is basically just the `id` function at runtime.
>
> Cheers,
> Sebastian
>
> Am So., 6. Okt. 2019 um 10:28 Uhr schrieb Ömer Sinan Ağacan <omeragacan at gmail.com>:
>>
>> Hi,
>>
>> I just realized that coercion binders are currently Ids and not TyVars (unlike
>> other type arguments). This means that we don't drop coercion binders in
>> CoreToStg. Example:
>>
>>     {-# LANGUAGE ScopedTypeVariables, TypeOperators, PolyKinds, GADTs,
>>        TypeApplications, MagicHash #-}
>>
>>     module UnsafeCoerce where
>>
>>     import Data.Type.Equality ((:~:)(..))
>>     import GHC.Prim
>>     import GHC.Types
>>
>>     unsafeEqualityProof :: forall k (a :: k) (b :: k) . a :~: b
>>     unsafeEqualityProof = error "unsafeEqualityProof evaluated"
>>
>>     unsafeCoerce :: forall a b . a -> b
>>     unsafeCoerce x = case unsafeEqualityProof @_ @a @b of Refl -> x
>>
>> If I build this with -ddump-stg this is what I get for `unsafeCoerce`:
>>
>>     UnsafeCoerce.unsafeCoerce :: forall a b. a -> b
>>     [GblId, Arity=1, Unf=OtherCon []] =
>>         [] \r [x_s2jn]
>>             case UnsafeCoerce.unsafeEqualityProof of {
>>               Data.Type.Equality.Refl co_a2fd -> x_s2jn;
>>             };
>>
>> See the binder in `Refl` pattern.
>>
>> Unarise drops this binder because it's a "void" argument (doesn't have a runtime
>> representation), but still it's a bit weird that we drop types but not coercions
>> in CoreToStg.
>>
>> Is this intentional?
>>
>> Ömer
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From omeragacan at gmail.com  Sun Oct  6 10:21:32 2019
From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=)
Date: Sun, 6 Oct 2019 13:21:32 +0300
Subject: Should coercion binders (arguments or binders in patterns) be
 TyVars?
In-Reply-To: <CAMQQO3=9=urfcKsc7m1zUBL=moh_c1KvxkNWvJBTG1-0i1NZ7w@mail.gmail.com>
References: <CAMQQO3nExWLD+h5s=7q56O744ymQD_d_hDEb=3g3Hqru637Ejw@mail.gmail.com>
 <CAAS+=P9_6+poiUJvYApXRD6L4n6sUEOMpO0bpGAd+yq1riJZiA@mail.gmail.com>
 <CAMQQO3=9=urfcKsc7m1zUBL=moh_c1KvxkNWvJBTG1-0i1NZ7w@mail.gmail.com>
Message-ID: <CAMQQO3nqoT0rgd1V61YD+pxT9E90E-nwVTc9hN0uEABJFGjm+w@mail.gmail.com>

> > Think of a hypothetical coercion `co :: Int ~ Double`; applying that coercion
> > as in `x |> co` to `x :: Int` would need to `fild` (load the integer in a
> > floating point register) at run-time, so you can't erase it. The fact that we
> > can for newtypes is because `coerce` is basically just the `id` function at
> > runtime.
>
> Do we even support such coercions? I think a `co :: Int ~ Double` can only be an
> unsafe coercion, which are basically no-op in runtime, so again they can be
> erased.

To elaborate, for coercions that have a meaning in runtime I think we use
primops, type coercions are not used for this purpose. E.g. for Int to Double
coercion, if you really mean boxed Ints and Doubles then you can simply treat
Int as Double in runtime (and enjoy the segfaults or worse), if you mean Int# to
Double# then you use a primop like int2Double# which does `fild` if necessary.

Ömer

Ömer Sinan Ağacan <omeragacan at gmail.com>, 6 Eki 2019 Paz, 12:55
tarihinde şunu yazdı:
>
> > I'm not sure if there's a case in GHC (yet, because newtype coercions are
> > zero-cost), but coercions in general (as introduced for example in Types and
> > Programming Languages) can carry computational content and thus can't be
> > erased.
>
> But they're already erased! They don't have a runtime representation and the
> unarise pass eliminates coercion values and binders. See the last paragraph in
> my original email.
>
> > Think of a hypothetical coercion `co :: Int ~ Double`; applying that coercion
> > as in `x |> co` to `x :: Int` would need to `fild` (load the integer in a
> > floating point register) at run-time, so you can't erase it. The fact that we
> > can for newtypes is because `coerce` is basically just the `id` function at
> > runtime.
>
> Do we even support such coercions? I think a `co :: Int ~ Double` can only be an
> unsafe coercion, which are basically no-op in runtime, so again they can be
> erased.
>
> Let me know if I'm mistaken.
>
> Ömer
>
> Sebastian Graf <sgraf1337 at gmail.com>, 6 Eki 2019 Paz, 12:50 tarihinde
> şunu yazdı:
> >
> > Hi Ömer,
> >
> > I'm not sure if there's a case in GHC (yet, because newtype coercions are zero-cost), but coercions in general (as introduced for example in Types and Programming Languages) can carry computational content and thus can't be erased.
> >
> > Think of a hypothetical coercion `co :: Int ~ Double`; applying that coercion as in `x |> co` to `x :: Int` would need to `fild` (load the integer in a floating point register) at run-time, so you can't erase it. The fact that we can for newtypes is because `coerce` is basically just the `id` function at runtime.
> >
> > Cheers,
> > Sebastian
> >
> > Am So., 6. Okt. 2019 um 10:28 Uhr schrieb Ömer Sinan Ağacan <omeragacan at gmail.com>:
> >>
> >> Hi,
> >>
> >> I just realized that coercion binders are currently Ids and not TyVars (unlike
> >> other type arguments). This means that we don't drop coercion binders in
> >> CoreToStg. Example:
> >>
> >>     {-# LANGUAGE ScopedTypeVariables, TypeOperators, PolyKinds, GADTs,
> >>        TypeApplications, MagicHash #-}
> >>
> >>     module UnsafeCoerce where
> >>
> >>     import Data.Type.Equality ((:~:)(..))
> >>     import GHC.Prim
> >>     import GHC.Types
> >>
> >>     unsafeEqualityProof :: forall k (a :: k) (b :: k) . a :~: b
> >>     unsafeEqualityProof = error "unsafeEqualityProof evaluated"
> >>
> >>     unsafeCoerce :: forall a b . a -> b
> >>     unsafeCoerce x = case unsafeEqualityProof @_ @a @b of Refl -> x
> >>
> >> If I build this with -ddump-stg this is what I get for `unsafeCoerce`:
> >>
> >>     UnsafeCoerce.unsafeCoerce :: forall a b. a -> b
> >>     [GblId, Arity=1, Unf=OtherCon []] =
> >>         [] \r [x_s2jn]
> >>             case UnsafeCoerce.unsafeEqualityProof of {
> >>               Data.Type.Equality.Refl co_a2fd -> x_s2jn;
> >>             };
> >>
> >> See the binder in `Refl` pattern.
> >>
> >> Unarise drops this binder because it's a "void" argument (doesn't have a runtime
> >> representation), but still it's a bit weird that we drop types but not coercions
> >> in CoreToStg.
> >>
> >> Is this intentional?
> >>
> >> Ömer
> >> _______________________________________________
> >> ghc-devs mailing list
> >> ghc-devs at haskell.org
> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From simonpj at microsoft.com  Sun Oct  6 21:38:20 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Sun, 6 Oct 2019 21:38:20 +0000
Subject: Should coercion binders (arguments or binders in patterns) be
 TyVars?
In-Reply-To: <CAMQQO3nExWLD+h5s=7q56O744ymQD_d_hDEb=3g3Hqru637Ejw@mail.gmail.com>
References: <CAMQQO3nExWLD+h5s=7q56O744ymQD_d_hDEb=3g3Hqru637Ejw@mail.gmail.com>
Message-ID: <VI1PR83MB0319AFD8EDDBFB17CE849E05AD980@VI1PR83MB0319.EURPRD83.prod.outlook.com>

| Is this intentional?

Yes, it's absolutely intentional.

Consider this Core defn
   f :: forall a b. (a ~# b) -> Int
   f = /\a b. \(g :: a ~# b). error "urk"

Now is `seq f True` equal to True, or to (error "urk").  Definitely the former.

So we cannot and must not discard coercion lambda.  However, a coercion is represented by a zero-bit value, and takes no space.  

Simon

| -----Original Message-----
| From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Ömer Sinan
| Agacan
| Sent: 06 October 2019 10:28
| To: ghc-devs <ghc-devs at haskell.org>
| Subject: Should coercion binders (arguments or binders in patterns) be
| TyVars?
| 
| Hi,
| 
| I just realized that coercion binders are currently Ids and not TyVars
| (unlike other type arguments). This means that we don't drop coercion
| binders in CoreToStg. Example:
| 
|     {-# LANGUAGE ScopedTypeVariables, TypeOperators, PolyKinds, GADTs,
|        TypeApplications, MagicHash #-}
| 
|     module UnsafeCoerce where
| 
|     import Data.Type.Equality ((:~:)(..))
|     import GHC.Prim
|     import GHC.Types
| 
|     unsafeEqualityProof :: forall k (a :: k) (b :: k) . a :~: b
|     unsafeEqualityProof = error "unsafeEqualityProof evaluated"
| 
|     unsafeCoerce :: forall a b . a -> b
|     unsafeCoerce x = case unsafeEqualityProof @_ @a @b of Refl -> x
| 
| If I build this with -ddump-stg this is what I get for `unsafeCoerce`:
| 
|     UnsafeCoerce.unsafeCoerce :: forall a b. a -> b
|     [GblId, Arity=1, Unf=OtherCon []] =
|         [] \r [x_s2jn]
|             case UnsafeCoerce.unsafeEqualityProof of {
|               Data.Type.Equality.Refl co_a2fd -> x_s2jn;
|             };
| 
| See the binder in `Refl` pattern.
| 
| Unarise drops this binder because it's a "void" argument (doesn't have a
| runtime representation), but still it's a bit weird that we drop types
| but not coercions in CoreToStg.
| 
| Is this intentional?
| 
| Ömer
| _______________________________________________
| ghc-devs mailing list
| ghc-devs at haskell.org
| https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.has
| kell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
| devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7Cf7f3feaf26b4452c1e8e08d
| 74a3f92f6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637059509215789548
| &amp;sdata=T9iolaftOD1ncWw1JICW1UJU1X8pyCivNyB7enTS4X8%3D&amp;reserved=0

From rae at richarde.dev  Mon Oct  7 09:14:37 2019
From: rae at richarde.dev (Richard Eisenberg)
Date: Mon, 7 Oct 2019 10:14:37 +0100
Subject: Should coercion binders (arguments or binders in patterns) be
 TyVars?
In-Reply-To: <VI1PR83MB0319AFD8EDDBFB17CE849E05AD980@VI1PR83MB0319.EURPRD83.prod.outlook.com>
References: <CAMQQO3nExWLD+h5s=7q56O744ymQD_d_hDEb=3g3Hqru637Ejw@mail.gmail.com>
 <VI1PR83MB0319AFD8EDDBFB17CE849E05AD980@VI1PR83MB0319.EURPRD83.prod.outlook.com>
Message-ID: <199E2749-38EF-41DF-BD24-D2CF9F4CA056@richarde.dev>

And just to close the loop: the more general coercions that Sebastian has talked about would have computational content, but these are well beyond what GHC supports at the moment (and there are no plans to move in that direction).

Richard

> On Oct 6, 2019, at 10:38 PM, Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> wrote:
> 
> | Is this intentional?
> 
> Yes, it's absolutely intentional.
> 
> Consider this Core defn
>   f :: forall a b. (a ~# b) -> Int
>   f = /\a b. \(g :: a ~# b). error "urk"
> 
> Now is `seq f True` equal to True, or to (error "urk").  Definitely the former.
> 
> So we cannot and must not discard coercion lambda.  However, a coercion is represented by a zero-bit value, and takes no space.  
> 
> Simon
> 
> | -----Original Message-----
> | From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Ömer Sinan
> | Agacan
> | Sent: 06 October 2019 10:28
> | To: ghc-devs <ghc-devs at haskell.org>
> | Subject: Should coercion binders (arguments or binders in patterns) be
> | TyVars?
> | 
> | Hi,
> | 
> | I just realized that coercion binders are currently Ids and not TyVars
> | (unlike other type arguments). This means that we don't drop coercion
> | binders in CoreToStg. Example:
> | 
> |     {-# LANGUAGE ScopedTypeVariables, TypeOperators, PolyKinds, GADTs,
> |        TypeApplications, MagicHash #-}
> | 
> |     module UnsafeCoerce where
> | 
> |     import Data.Type.Equality ((:~:)(..))
> |     import GHC.Prim
> |     import GHC.Types
> | 
> |     unsafeEqualityProof :: forall k (a :: k) (b :: k) . a :~: b
> |     unsafeEqualityProof = error "unsafeEqualityProof evaluated"
> | 
> |     unsafeCoerce :: forall a b . a -> b
> |     unsafeCoerce x = case unsafeEqualityProof @_ @a @b of Refl -> x
> | 
> | If I build this with -ddump-stg this is what I get for `unsafeCoerce`:
> | 
> |     UnsafeCoerce.unsafeCoerce :: forall a b. a -> b
> |     [GblId, Arity=1, Unf=OtherCon []] =
> |         [] \r [x_s2jn]
> |             case UnsafeCoerce.unsafeEqualityProof of {
> |               Data.Type.Equality.Refl co_a2fd -> x_s2jn;
> |             };
> | 
> | See the binder in `Refl` pattern.
> | 
> | Unarise drops this binder because it's a "void" argument (doesn't have a
> | runtime representation), but still it's a bit weird that we drop types
> | but not coercions in CoreToStg.
> | 
> | Is this intentional?
> | 
> | Ömer
> | _______________________________________________
> | ghc-devs mailing list
> | ghc-devs at haskell.org
> | https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.has
> | kell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
> | devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7Cf7f3feaf26b4452c1e8e08d
> | 74a3f92f6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637059509215789548
> | &amp;sdata=T9iolaftOD1ncWw1JICW1UJU1X8pyCivNyB7enTS4X8%3D&amp;reserved=0
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From rae at richarde.dev  Mon Oct  7 10:45:25 2019
From: rae at richarde.dev (Richard Eisenberg)
Date: Mon, 7 Oct 2019 11:45:25 +0100
Subject: Coercible questions
In-Reply-To: <CAKFCL4W6kg_YcMdYEyWi1gAZydfOeAr78ZTx4HH74P3+s8eDbw@mail.gmail.com>
References: <CAEN7k7=4vRpeqP4d71gqgiq4=CF46hP34QJ7H9jrhr3Gsd8x1g@mail.gmail.com>
 <CADRtjwagU4YNMRfB5yCQuqzS1nJxr=Fp-_dzOmkSoxu01Tji8A@mail.gmail.com>
 <CAEN7k7kWYha7TzZLSwpYqp2O-cQJA=hcJoimn+Qp=+ipryk0yA@mail.gmail.com>
 <CAKFCL4W6kg_YcMdYEyWi1gAZydfOeAr78ZTx4HH74P3+s8eDbw@mail.gmail.com>
Message-ID: <81349010-86E0-46E2-A3CF-1BD8C819BD05@richarde.dev>

Hi devs,

In a perfect world, Coercible would relate any two types with the same representation. But the world is not perfect! So we fall well short of this goal.

Specifically, Coercible allows conversions between a newtype and its representation, and additionally between datatypes where the roles work out. Section 2 of the JFP paper (https://repository.brynmawr.edu/cgi/viewcontent.cgi?referer=&httpsredir=1&article=1010&context=compsci_pubs <https://repository.brynmawr.edu/cgi/viewcontent.cgi?referer=&httpsredir=1&article=1010&context=compsci_pubs>) is an accessible description of how this works.

In the specific cases of Int and Word: there is no guarantee (that I know of) that the tags on the sole constructors of these types are the same. So we don't know that they have the same representation. Even if we did know that the tags are the same, there is no guarantee that Int# and Word# have the same representation. I *think* GHC is general enough that it could cope on a system that used, say, different registers to store signed and unsigned machine integers.

More generally, it might be nice if we could coerce between

> data Booly = Bad | Good

and Bool, but we can't. It would additionally be good to coerce between A and B, where:

> data A = A1 | A2 | A3
> data B = B1 | B2 !C
> data C = C1 | C2

Both A and B have precisely 3 possibilities. But we don't do that yet either. Maybe someday.

In contrast to Brandon, I don't know of anyone working on this. That's not to say that it's no good to work on it, of course!

Richard

> On Oct 5, 2019, at 5:51 PM, Brandon Allbery <allbery.b at gmail.com> wrote:
> 
> I think there's some work going on to expose the representations, which would enable some ability to coerce. But possibly not this much, as they're separate RuntimeReps so you don't combine signed and unsigned numbers inadvertently; currently that's a little magical inside ghc iirc, with the RuntimeRep the only way to distinguish at all and that vanishing post-typechecking.
> 
> On Sat, Oct 5, 2019 at 12:30 PM Michal Terepeta <michal.terepeta at gmail.com <mailto:michal.terepeta at gmail.com>> wrote:
> Adding +ghc-devs <mailto:ghc-devs at haskell.org> to continue the thread
> 
> Hi Sandy,
> 
> Thanks for the answer! Do you think there is some fundamental reason for this? Or just a matter of implementing this in GHC? It seems to me that this should work just fine as long as the runtime representation is the same.
> 
> And a related question--is it safe to `unsafeCoerce` an `Int` to a `Word`? The only reason for why this could be problematic that comes to my mind is that there could be an assumption that different `data`s do not alias each other (although `newtype`s can due to `Coercible` functionality). But I'm not sure this is ever used by GHC? Are there any other reasons why this could be problematic?
> 
> Thanks!
> 
> - Michal
> 
> 
> 
> On Sat, Oct 5, 2019 at 5:27 PM Sandy Maguire <sandy at sandymaguire.me <mailto:sandy at sandymaguire.me>> wrote:
> Hi Michal,
> 
> Datas aren't coercible, only newtypes. This is why you can't coerce Ints and Words, and why Foo and Bar don't work.
> 
> Sandy
> 
> On Sat, Oct 5, 2019 at 4:17 PM Michal Terepeta <michal.terepeta at gmail.com <mailto:michal.terepeta at gmail.com>> wrote:
> Hi,
> 
> I've started looking into using `Data.Coerce` (and the `Coercible` type-class) for a personal project and was wondering why coercing between `Int` and `Word` is not allowed? I don't see any fundamental reason why this shouldn't work...
> 
> Perhaps, it's just a matter of GHC's implementation details leaking out? IIRC internally GHC has separate `RuntimeRep`/`PrimRep` for a `Word#` and for an `Int#`. If that's the case, would it make sense to unify these? Their actual runtime representation should be the same and I'd expect most (all?) of their differences should be attached to `PrimOp`s.
> 
> And that leads me to another question--what exactly goes wrong here:
> ```
> data Foo = Foo Int#
> data Bar = Bar Int#
> 
> test :: Bar
> test = coerce (Foo 42#)
> ```
> Which fails with: "Couldn't match representation of type ‘Foo’ with that of ‘Bar’ arising from a use of ‘coerce’"
> 
> Perhaps I'm just misunderstanding exactly how `Coercible` works?
> 
> Thanks in advance!
> 
> - Michal
> 
> PS. The ability to coerce through things like lists is amazing :)
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs <http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs>
> 
> 
> -- 
> I'm currently travelling the world, sleeping on people's couches and doing full-time collaboration on Haskell projects. If this seems interesting to you, please consider signing up as a host! https://isovector.github.io/erdos/ <https://isovector.github.io/erdos/>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs <http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs>
> 
> 
> -- 
> brandon s allbery kf8nh
> allbery.b at gmail.com <mailto:allbery.b at gmail.com>_______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191007/1e89043a/attachment.html>

From george.colpitts at gmail.com  Tue Oct  8 00:44:43 2019
From: george.colpitts at gmail.com (George Colpitts)
Date: Mon, 7 Oct 2019 21:44:43 -0300
Subject: GHC 8.8.1 and cabal-install 3.0 not compatible with ghc-pkg
In-Reply-To: <CAB-d4A5-n7uz7qptOe4=OT5kM1_JTJZVFqueUz-wZdL09LrqAQ@mail.gmail.com>
References: <87zhjwl1mx.fsf@smart-cactus.org>
 <CAB-d4A742d6w1_3g03JiC_0Ud3YTOnvP+w+Ok-WRi79DgPZz3Q@mail.gmail.com>
 <CAHYVw0zQeprxercecps0y-AEGJ5Oc0q=ETK_m8uowd2yT5V3ZQ@mail.gmail.com>
 <CAKe9RRGBm=W83MtSY=500b-RjP9f2TxRV5Np5SauqkQrP2vJ+Q@mail.gmail.com>
 <CAB-d4A5-n7uz7qptOe4=OT5kM1_JTJZVFqueUz-wZdL09LrqAQ@mail.gmail.com>
Message-ID: <CAB-d4A51atytAvrYhgRcc2XbnX6b8OHOE8A77dbCdOaUg1Zj_Q@mail.gmail.com>

Unfortunately ghc 8.8.1 and cabal-install 3.0 are not compatible with
ghc-pkg as documented in
https://github.com/haskell/cabal/issues/6262#issuecomment-538850477. As
Daniel Grober wrote there:

I think this is expected behaviour, at least from the cabal side of things.
Version 3.0.0.0 switched to using v2-build by default so installed
libraries are now registered into the "default" package environment file in
.ghc/*/environments/default as well as a package-db in
.cabal/store/ghc-*/package.db which GHC doesn't know about by itself.

AFAICS ghc-pkg simply doesn't have support for listing package environments
which sort of kind of makes sense but also breaks this workflow. I think
this should be reported as a GHC bug but I'm not sure it would even be a
good idea to add pkg-env support to ghc-pkg.



I thought this might be a ghc issue but since nobody else was complaining I
assumed I must have done something wrong. Should I file a ghc bug ?

Thanks
George



On Sat, Sep 7, 2019 at 6:20 PM George Colpitts <george.colpitts at gmail.com>
wrote:

> Thanks for everybody's responses. I figured out that the following
>
> cabal-install users should note that cabal-install-3.0 or later is
> required for use with GHC 8.8.
>
>
> means  I should have cabal-install-3.0 before installing 8.8.1. Once I did
> that everything is fine.
>
> Maybe configure should give an error if the user does not
> have cabal-install-3.0?
>
> Thanks
> George
>
>
> On Fri, Sep 6, 2019 at 1:48 PM Shayne Fletcher <shayne.fletcher at daml.com>
> wrote:
>
>> I got there by doing,
>> ```
>>   cabal v2-install --installdir=~/.cabal/bin alex
>>   cabal v2-install --installdir=~/.cabal/bin happy
>> ```
>> and things seemed to be going smoothly enough after that.
>>
>> On Fri, Sep 6, 2019 at 12:08 PM Carter Schonwald <
>> carter.schonwald at gmail.com> wrote:
>>
>>> V1 or v2 install?
>>>
>>> On Mon, Sep 2, 2019 at 11:42 AM George Colpitts <
>>> george.colpitts at gmail.com> wrote:
>>>
>>>> https://www.haskell.org/ghc/blog/20190825-ghc-8.8.1-released.html says
>>>>
>>>> cabal-install users should note that cabal-install-3.0 or later is
>>>> required for use with GHC 8.8.
>>>>
>>>> but this seems wrong or have I done something wrong?
>>>>
>>>> $ cabal install cabal-install
>>>> cabal install cabal-install
>>>> ...
>>>>
>>>> Resolving dependencies...
>>>> cabal: Could not resolve dependencies:
>>>> [__0] trying: cabal-install-3.0.0.0 (user goal)
>>>> [__1] next goal: time (dependency of cabal-install)
>>>> [__1] rejecting: time-1.9.3/installed-1.9... (conflict: cabal-install =>
>>>> base>=4.8 && <4.13, time => base==4.13.0.0/installed-4.1...)
>>>> [__1] trying: time-1.9.3
>>>> [__2] next goal: stm (dependency of cabal-install)
>>>> [__2] rejecting: stm-2.5.0.0/installed-2.5... (conflict: cabal-install
>>>> =>
>>>> base>=4.8 && <4.13, stm => base==4.13.0.0/installed-4.1...)
>>>> [__2] trying: stm-2.5.0.0
>>>> [__3] next goal: process (dependency of cabal-install)
>>>> [__3] rejecting: process-1.6.5.1/installed-1.6... (conflict:
>>>> cabal-install =>
>>>> base>=4.8 && <4.13, process => base==4.13.0.0/installed-4.1...)
>>>> [__3] trying: process-1.6.5.1
>>>> [__4] next goal: pretty (dependency of cabal-install)
>>>> [__4] rejecting: pretty-1.1.3.6/installed-1.1... (conflict:
>>>> cabal-install =>
>>>> base>=4.8 && <4.13, pretty => base==4.13.0.0/installed-4.1...)
>>>> [__4] trying: pretty-1.1.3.6
>>>> [__5] next goal: network (dependency of cabal-install)
>>>> [__5] rejecting: network-3.1.0.1/installed-CeX... (conflict:
>>>> cabal-install =>
>>>> base>=4.8 && <4.13, network => base==4.13.0.0/installed-4.1...)
>>>> [__5] trying: network-3.1.0.1
>>>> [__6] trying: hackage-security-0.5.3.0 (dependency of cabal-install)
>>>> [__7] next goal: template-haskell (dependency of hackage-security)
>>>> [__7] rejecting: template-haskell-2.15.0.0/installed-2.1... (conflict:
>>>> cabal-install => base>=4.8 && <4.13, template-haskell =>
>>>> base==4.13.0.0/installed-4.1...)
>>>> [__7] rejecting: template-haskell-2.15.0.0, template-haskell-2.14.0.0,
>>>> template-haskell-2.13.0.0, template-haskell-2.12.0.0,
>>>> template-haskell-2.11.1.0, template-haskell-2.11.0.0,
>>>> template-haskell-2.10.0.0, template-haskell-2.9.0.0,
>>>> template-haskell-2.8.0.0,
>>>> template-haskell-2.7.0.0, template-haskell-2.6.0.0,
>>>> template-haskell-2.5.0.0,
>>>> template-haskell-2.4.0.1, template-haskell-2.4.0.0,
>>>> template-haskell-2.3.0.1,
>>>> template-haskell-2.3.0.0, template-haskell-2.2.0.0 (constraint from
>>>> non-upgradeable package requires installed instance)
>>>> [__7] fail (backjumping, conflict set: cabal-install, hackage-security,
>>>> template-haskell)
>>>> After searching the rest of the dependency tree exhaustively, these
>>>> were the
>>>> goals I've had most trouble fulfilling: base, cabal-install, directory,
>>>> template-haskell, process, time, network, pretty, hackage-security,
>>>> deepseq,
>>>> HTTP, stm, cabal-install:lib
>>>>
>>>>
>>>> On Mon, Aug 26, 2019 at 6:25 AM Ben Gamari <ben at well-typed.com> wrote:
>>>>
>>>>>
>>>>> Hello everyone,
>>>>>
>>>>> The GHC team is pleased to announce the release candidate for GHC
>>>>> 8.8.1.
>>>>> The source distribution, binary distributions, and documentation are
>>>>> available at
>>>>>
>>>>>     https://downloads.haskell.org/ghc/8.8.1
>>>>>
>>>>> This release is the culmination of over 3000 commits by over one
>>>>> hundred
>>>>> contributors and has several new features and numerous bug fixes
>>>>> relative to GHC 8.6:
>>>>>
>>>>>  * Visible kind applications are now supported (GHC Proposal #15)
>>>>>
>>>>>  * Profiling now works correctly on 64-bit Windows (although still may
>>>>>    be problematic on 32-bit Windows due to platform limitations; see
>>>>>    #15934)
>>>>>
>>>>>  * A new code layout algorithm for amd64's native code generator
>>>>>    significantly improving the runtime performance of some kernels
>>>>>
>>>>>  * The introduction of a late lambda-lifting pass which may reduce
>>>>>    allocations significantly for some programs.
>>>>>
>>>>>  * Further work on Trees That Grow, enabling improved code re-use of
>>>>> the
>>>>>    Haskell AST in tooling
>>>>>
>>>>>  * Users can write `forall` in more contexts (GHC Proposal #7)
>>>>>
>>>>>  * The pattern-match checker is now more precise in the presence of
>>>>>    strict fields with uninhabited types.
>>>>>
>>>>>  * A comprehensive audit of GHC's memory ordering barriers has been
>>>>>    performed, resulting in a number of fixes that should significantly
>>>>>    improve the reliability of programs on architectures with
>>>>>    weakly-ordered memory models (e.g. PowerPC, many ARM and AArch64
>>>>>    implementations).
>>>>>
>>>>>  * A long-standing linker limitation rendering GHCi unusable with
>>>>>    projects with cyclic symbol dependencies has been fixed (#13786)
>>>>>
>>>>>  * Further work on the Hadrian build system
>>>>>
>>>>>  * Countless miscellaneous bug-fixes
>>>>>
>>>>> Unfortunately, due to a build issue (#17108) found late in the release
>>>>> process
>>>>> i386 Windows builds are currently unavailable. These will be provided
>>>>> in
>>>>> the coming weeks.
>>>>>
>>>>> As always, if anything looks amiss do let us know.
>>>>>
>>>>> Happy compiling!
>>>>>
>>>>> Cheers,
>>>>>
>>>>> - Ben
>>>>>
>>>>>
>>>>> [1]
>>>>> https://downloads.haskell.org/ghc/8.8.1/docs/html/users_guide/8.8.1-notes.html
>>>>> _______________________________________________
>>>>> Glasgow-haskell-users mailing list
>>>>> Glasgow-haskell-users at haskell.org
>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
>>>>>
>>>> _______________________________________________
>>>> Glasgow-haskell-users mailing list
>>>> Glasgow-haskell-users at haskell.org
>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
>>>>
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>>
>>
>> --
>> *Shayne Fletcher*
>> Language Engineer */* +1 917 699 7663
>> *Digital Asset* <https://digitalasset.com/>, creators of *DAML
>> <https://daml.com/>*
>>
>> This message, and any attachments, is for the intended recipient(s) only,
>> may contain information that is privileged, confidential and/or proprietary
>> and subject to important terms and conditions available at
>> http://www.digitalasset.com/emaildisclaimer.html. If you are not the
>> intended recipient, please delete this message.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191007/c70d739f/attachment.html>

From sam.halliday at gmail.com  Tue Oct  8 14:14:56 2019
From: sam.halliday at gmail.com (Sam Halliday)
Date: Tue, 08 Oct 2019 15:14:56 +0100
Subject: slow load and typecheck
Message-ID: <87mueb493z.fsf@gmail.com>

Hello all,

I am writing an interactive tool using the ghc api. It is able to load
and typecheck a source file in a user's package.

I obtain the flags that cabal uses to compile the user's package via the
hie-bios trick, and I `parseDynamicFlagsCmdLine' them inside my tool,
then I `setTargets' all the home modules (with targetAllowObjCode=True).

I use HscNothing and NoLink because I only want access to the trees, I
don't want to produce any output files.

For the file that I wish to inspect, I `removeTarget' the module and
`addTarget` it again but this time providing the full path to the file
and don't allow object code.

Then I LoadUpTo and typecheck. For the sake of simplicity, let's assume
that the file under inspection only has a module definiton and no
imports or top levels.

Functionally, my code is working great and I am able to do what I want
with the typechecked tree.

However, load is very slow (~10 seconds user time) on large projects.
Here is a cpu time trace of my program (milliseconds):

  main              1
  parse flags      93
  load          20436
  typecheck     20437

I can enable a bit more ghc timing info via -Rghc-timings and I see

  !!! Chasing dependencies: finished in 157.20 milliseconds, allocated
      528.112 megabytes

This seems fine, anything sub-second is ok.

But then I see a bunch of home modules in CodeGen that I was not expecting:

   !!! CodeGen [My.Module.Dependency]:
       finished in 3335.62 milliseconds, allocated 270.615 megabytes

So it looks like the targetAllowObjCode is being ignored... is there any
way to force it? Actually I'd prefer to fail fast than to ever compile
or codegen a dependency module.


I know that it should be possible to load the module a lot faster
because if I make a small change in the file under inspection and ask
cabal to recompile the module it is super fast (less than a second).

Could somebody who understands how incremental/partial compiles work
please help me out?


PS: If this textual description is confusing, I could put together a
minimal reproduction and example project but it will take me some time
to do that.

-- 
Best regards,
Sam
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/df0a2161/attachment.sig>

From allbery.b at gmail.com  Tue Oct  8 14:29:30 2019
From: allbery.b at gmail.com (Brandon Allbery)
Date: Tue, 8 Oct 2019 10:29:30 -0400
Subject: slow load and typecheck
In-Reply-To: <87mueb493z.fsf@gmail.com>
References: <87mueb493z.fsf@gmail.com>
Message-ID: <CAKFCL4Vr6UihK0vYZptgq=Hw2VZD-2S6iRerDfDC0kyDvYaVRw@mail.gmail.com>

I think the only path for loading a dependency that doesn't involve loading
object code of some kind is the {-# SOURCE #-} hack as part of .hs-boot
files, which isn't general enough to be reused here as I understand it. A
decent chunk of the compiler would need to be duplicated to avoid this, and
it might use a fair amount of memory and end up generating at least part of
the object into memory.

Also recall that if any TH or quasiquotation is involved, it'll need to
load object code in support of that; and it might well need to prepare for
this in the general case rather than again having to duplicate a bunch of
code to support different no-TH and TH paths.

Cabal will build all that stuff the first time and then reuse it the next,
so it's not quite the same thing. Since you told ghc no object code, it
discards what it generates here and may not use existing compiled modules;
or you may have specified settings incompatible with any it did find.

In short, you may want to rethink this; ghc is a compiler, not an IDE, and
doesn't quite work the way you had hoped.

On Tue, Oct 8, 2019 at 10:15 AM Sam Halliday <sam.halliday at gmail.com> wrote:

> Hello all,
>
> I am writing an interactive tool using the ghc api. It is able to load
> and typecheck a source file in a user's package.
>
> I obtain the flags that cabal uses to compile the user's package via the
> hie-bios trick, and I `parseDynamicFlagsCmdLine' them inside my tool,
> then I `setTargets' all the home modules (with targetAllowObjCode=True).
>
> I use HscNothing and NoLink because I only want access to the trees, I
> don't want to produce any output files.
>
> For the file that I wish to inspect, I `removeTarget' the module and
> `addTarget` it again but this time providing the full path to the file
> and don't allow object code.
>
> Then I LoadUpTo and typecheck. For the sake of simplicity, let's assume
> that the file under inspection only has a module definiton and no
> imports or top levels.
>
> Functionally, my code is working great and I am able to do what I want
> with the typechecked tree.
>
> However, load is very slow (~10 seconds user time) on large projects.
> Here is a cpu time trace of my program (milliseconds):
>
>   main              1
>   parse flags      93
>   load          20436
>   typecheck     20437
>
> I can enable a bit more ghc timing info via -Rghc-timings and I see
>
>   !!! Chasing dependencies: finished in 157.20 milliseconds, allocated
>       528.112 megabytes
>
> This seems fine, anything sub-second is ok.
>
> But then I see a bunch of home modules in CodeGen that I was not expecting:
>
>    !!! CodeGen [My.Module.Dependency]:
>        finished in 3335.62 milliseconds, allocated 270.615 megabytes
>
> So it looks like the targetAllowObjCode is being ignored... is there any
> way to force it? Actually I'd prefer to fail fast than to ever compile
> or codegen a dependency module.
>
>
> I know that it should be possible to load the module a lot faster
> because if I make a small change in the file under inspection and ask
> cabal to recompile the module it is super fast (less than a second).
>
> Could somebody who understands how incremental/partial compiles work
> please help me out?
>
>
> PS: If this textual description is confusing, I could put together a
> minimal reproduction and example project but it will take me some time
> to do that.
>
> --
> Best regards,
> Sam
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>


-- 
brandon s allbery kf8nh
allbery.b at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/5daa5ebb/attachment.html>

From sam.halliday at gmail.com  Tue Oct  8 14:30:26 2019
From: sam.halliday at gmail.com (Sam Halliday)
Date: Tue, 08 Oct 2019 15:30:26 +0100
Subject: slow load and typecheck
In-Reply-To: <87mueb493z.fsf@gmail.com>
References: <87mueb493z.fsf@gmail.com>
Message-ID: <87k19f48e5.fsf@gmail.com>

A quick follow-up to this, Rahul Muttinieni gave me some advice to try
out

  HscInterpreted / LinkInMemory

instead of

  HscNothing / NoLink

and now I am no longer seeing home modules being compiled, and
everything is a lot faster. Woohoo!


But I have no idea why this speeds things up... my code isn't using
TemplateHaskell so HscNothing should really mean "don't do any codegen".
Something is causing the HscNothing to be ignored. I'd still really like
to get to the bottom of this so if anybody knows how the batch compiler
is able to avoid recompiling home modules then please let me know... I
would like to continue using HscNothing instead of HscInterpreted.


Sam Halliday <sam.halliday at gmail.com> writes:

> Hello all,
>
> I am writing an interactive tool using the ghc api. It is able to load
> and typecheck a source file in a user's package.
>
> I obtain the flags that cabal uses to compile the user's package via the
> hie-bios trick, and I `parseDynamicFlagsCmdLine' them inside my tool,
> then I `setTargets' all the home modules (with targetAllowObjCode=True).
>
> I use HscNothing and NoLink because I only want access to the trees, I
> don't want to produce any output files.
>
> For the file that I wish to inspect, I `removeTarget' the module and
> `addTarget` it again but this time providing the full path to the file
> and don't allow object code.
>
> Then I LoadUpTo and typecheck. For the sake of simplicity, let's assume
> that the file under inspection only has a module definiton and no
> imports or top levels.
>
> Functionally, my code is working great and I am able to do what I want
> with the typechecked tree.
>
> However, load is very slow (~10 seconds user time) on large projects.
> Here is a cpu time trace of my program (milliseconds):
>
>   main              1
>   parse flags      93
>   load          20436
>   typecheck     20437
>
> I can enable a bit more ghc timing info via -Rghc-timings and I see
>
>   !!! Chasing dependencies: finished in 157.20 milliseconds, allocated
>       528.112 megabytes
>
> This seems fine, anything sub-second is ok.
>
> But then I see a bunch of home modules in CodeGen that I was not expecting:
>
>    !!! CodeGen [My.Module.Dependency]:
>        finished in 3335.62 milliseconds, allocated 270.615 megabytes
>
> So it looks like the targetAllowObjCode is being ignored... is there any
> way to force it? Actually I'd prefer to fail fast than to ever compile
> or codegen a dependency module.
>
>
> I know that it should be possible to load the module a lot faster
> because if I make a small change in the file under inspection and ask
> cabal to recompile the module it is super fast (less than a second).
>
> Could somebody who understands how incremental/partial compiles work
> please help me out?
>
>
> PS: If this textual description is confusing, I could put together a
> minimal reproduction and example project but it will take me some time
> to do that.
>
> -- 
> Best regards,
> Sam

-- 
Best regards,
Sam
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/2064b059/attachment.sig>

From sam.halliday at gmail.com  Tue Oct  8 14:37:34 2019
From: sam.halliday at gmail.com (Sam Halliday)
Date: Tue, 08 Oct 2019 15:37:34 +0100
Subject: slow load and typecheck
In-Reply-To: <CAKFCL4Vr6UihK0vYZptgq=Hw2VZD-2S6iRerDfDC0kyDvYaVRw@mail.gmail.com>
References: <87mueb493z.fsf@gmail.com>
 <CAKFCL4Vr6UihK0vYZptgq=Hw2VZD-2S6iRerDfDC0kyDvYaVRw@mail.gmail.com>
Message-ID: <87ftk34829.fsf@gmail.com>

Thanks Brandon,

Brandon Allbery <allbery.b at gmail.com> writes:
> Cabal will build all that stuff the first time and then reuse it the next,
> so it's not quite the same thing. Since you told ghc no object code,

Sorry, I meant that I used targetAllowObjCode=True for everything,
except the file under inspection. Do you mean that if I used
targetAllowObjCode=False for just one module it will invalidate the
object code for everything it depends on? That is unexpected.


> In short, you may want to rethink this; ghc is a compiler, not an IDE, and
> doesn't quite work the way you had hoped.

How would you suggest rethinking it? Bare in mind that the api is
working exactly the way I want from a functional point of view (just
slow) with HscNothing... and seems to work exactly the way I want with
HscInterpreted (but with all the ghci caveats like unboxed tuples etc).

-- 
Best regards,
Sam
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/f3225b25/attachment.sig>

From allbery.b at gmail.com  Tue Oct  8 14:40:47 2019
From: allbery.b at gmail.com (Brandon Allbery)
Date: Tue, 8 Oct 2019 10:40:47 -0400
Subject: slow load and typecheck
In-Reply-To: <87ftk34829.fsf@gmail.com>
References: <87mueb493z.fsf@gmail.com>
 <CAKFCL4Vr6UihK0vYZptgq=Hw2VZD-2S6iRerDfDC0kyDvYaVRw@mail.gmail.com>
 <87ftk34829.fsf@gmail.com>
Message-ID: <CAKFCL4W0DFjzmVZOGLMVrwZMf24Cr3vBSTiUyLRsPAz8_+fhfw@mail.gmail.com>

It's doing what you — but not ghc — consider "extra work", though. ghc
expects to be compiling code, and doesn't have a separate code path for
"load symbols from an external module by parsing its source code" instead
of "load symbols from an external module by loading its .hsc file and
object code", aside from HscInterpreted.

On Tue, Oct 8, 2019 at 10:37 AM Sam Halliday <sam.halliday at gmail.com> wrote:

> Thanks Brandon,
>
> Brandon Allbery <allbery.b at gmail.com> writes:
> > Cabal will build all that stuff the first time and then reuse it the
> next,
> > so it's not quite the same thing. Since you told ghc no object code,
>
> Sorry, I meant that I used targetAllowObjCode=True for everything,
> except the file under inspection. Do you mean that if I used
> targetAllowObjCode=False for just one module it will invalidate the
> object code for everything it depends on? That is unexpected.
>
>
> > In short, you may want to rethink this; ghc is a compiler, not an IDE,
> and
> > doesn't quite work the way you had hoped.
>
> How would you suggest rethinking it? Bare in mind that the api is
> working exactly the way I want from a functional point of view (just
> slow) with HscNothing... and seems to work exactly the way I want with
> HscInterpreted (but with all the ghci caveats like unboxed tuples etc).
>
> --
> Best regards,
> Sam
>


-- 
brandon s allbery kf8nh
allbery.b at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/4ab5767c/attachment.html>

From matthewtpickering at gmail.com  Tue Oct  8 14:47:24 2019
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Tue, 8 Oct 2019 15:47:24 +0100
Subject: slow load and typecheck
In-Reply-To: <87k19f48e5.fsf@gmail.com>
References: <87mueb493z.fsf@gmail.com> <87k19f48e5.fsf@gmail.com>
Message-ID: <CALuQ0m-7NA01S8nH6DiAxDN0krHqYuuQ-O352Yx2dhEXVLCk6A@mail.gmail.com>

Are you writing interface files (-fwrite-interface)? It makes no sense
for HscInterpreted to be faster than HscNothing.

Cheers,

Matt

On Tue, Oct 8, 2019 at 3:30 PM Sam Halliday <sam.halliday at gmail.com> wrote:
>
> A quick follow-up to this, Rahul Muttinieni gave me some advice to try
> out
>
>   HscInterpreted / LinkInMemory
>
> instead of
>
>   HscNothing / NoLink
>
> and now I am no longer seeing home modules being compiled, and
> everything is a lot faster. Woohoo!
>
>
> But I have no idea why this speeds things up... my code isn't using
> TemplateHaskell so HscNothing should really mean "don't do any codegen".
> Something is causing the HscNothing to be ignored. I'd still really like
> to get to the bottom of this so if anybody knows how the batch compiler
> is able to avoid recompiling home modules then please let me know... I
> would like to continue using HscNothing instead of HscInterpreted.
>
>
> Sam Halliday <sam.halliday at gmail.com> writes:
>
> > Hello all,
> >
> > I am writing an interactive tool using the ghc api. It is able to load
> > and typecheck a source file in a user's package.
> >
> > I obtain the flags that cabal uses to compile the user's package via the
> > hie-bios trick, and I `parseDynamicFlagsCmdLine' them inside my tool,
> > then I `setTargets' all the home modules (with targetAllowObjCode=True).
> >
> > I use HscNothing and NoLink because I only want access to the trees, I
> > don't want to produce any output files.
> >
> > For the file that I wish to inspect, I `removeTarget' the module and
> > `addTarget` it again but this time providing the full path to the file
> > and don't allow object code.
> >
> > Then I LoadUpTo and typecheck. For the sake of simplicity, let's assume
> > that the file under inspection only has a module definiton and no
> > imports or top levels.
> >
> > Functionally, my code is working great and I am able to do what I want
> > with the typechecked tree.
> >
> > However, load is very slow (~10 seconds user time) on large projects.
> > Here is a cpu time trace of my program (milliseconds):
> >
> >   main              1
> >   parse flags      93
> >   load          20436
> >   typecheck     20437
> >
> > I can enable a bit more ghc timing info via -Rghc-timings and I see
> >
> >   !!! Chasing dependencies: finished in 157.20 milliseconds, allocated
> >       528.112 megabytes
> >
> > This seems fine, anything sub-second is ok.
> >
> > But then I see a bunch of home modules in CodeGen that I was not expecting:
> >
> >    !!! CodeGen [My.Module.Dependency]:
> >        finished in 3335.62 milliseconds, allocated 270.615 megabytes
> >
> > So it looks like the targetAllowObjCode is being ignored... is there any
> > way to force it? Actually I'd prefer to fail fast than to ever compile
> > or codegen a dependency module.
> >
> >
> > I know that it should be possible to load the module a lot faster
> > because if I make a small change in the file under inspection and ask
> > cabal to recompile the module it is super fast (less than a second).
> >
> > Could somebody who understands how incremental/partial compiles work
> > please help me out?
> >
> >
> > PS: If this textual description is confusing, I could put together a
> > minimal reproduction and example project but it will take me some time
> > to do that.
> >
> > --
> > Best regards,
> > Sam
>
> --
> Best regards,
> Sam
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From sam.halliday at gmail.com  Tue Oct  8 14:50:59 2019
From: sam.halliday at gmail.com (Sam Halliday)
Date: Tue, 08 Oct 2019 15:50:59 +0100
Subject: slow load and typecheck
In-Reply-To: <CALuQ0m-7NA01S8nH6DiAxDN0krHqYuuQ-O352Yx2dhEXVLCk6A@mail.gmail.com>
References: <87mueb493z.fsf@gmail.com> <87k19f48e5.fsf@gmail.com>
 <CALuQ0m-7NA01S8nH6DiAxDN0krHqYuuQ-O352Yx2dhEXVLCk6A@mail.gmail.com>
Message-ID: <87d0f747fw.fsf@gmail.com>

Matthew Pickering <matthewtpickering at gmail.com> writes:

> Are you writing interface files (-fwrite-interface)? It makes no sense
> for HscInterpreted to be faster than HscNothing.

Nope, not writing anything like that (I just checked the ghc flags from
hie-bios to confirm)... and I agree that this makes no sense.


-- 
Best regards,
Sam
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/1ca31c2a/attachment.sig>

From allbery.b at gmail.com  Tue Oct  8 14:53:53 2019
From: allbery.b at gmail.com (Brandon Allbery)
Date: Tue, 8 Oct 2019 10:53:53 -0400
Subject: slow load and typecheck
In-Reply-To: <87d0f747fw.fsf@gmail.com>
References: <87mueb493z.fsf@gmail.com> <87k19f48e5.fsf@gmail.com>
 <CALuQ0m-7NA01S8nH6DiAxDN0krHqYuuQ-O352Yx2dhEXVLCk6A@mail.gmail.com>
 <87d0f747fw.fsf@gmail.com>
Message-ID: <CAKFCL4WJCWbNNFBCNfEeJYkws=HP-Ris8s43S-=5YHVpWqzc4Q@mail.gmail.com>

I already mentioned needing .hi (I may have said hsc, whoops; Haskell
Interface files) from dependencies; you really want to turn that part on,
at least. And possibly ensure your other options are compatible with
existing .hi files, so they can be loaded directly. I think the .o isn't
used until link time, which should be irrelevant for you; but you really do
want those .hi files, otherwise it must compile the dependency module to
generate one.

On Tue, Oct 8, 2019 at 10:51 AM Sam Halliday <sam.halliday at gmail.com> wrote:

> Matthew Pickering <matthewtpickering at gmail.com> writes:
>
> > Are you writing interface files (-fwrite-interface)? It makes no sense
> > for HscInterpreted to be faster than HscNothing.
>
> Nope, not writing anything like that (I just checked the ghc flags from
> hie-bios to confirm)... and I agree that this makes no sense.
>
>
> --
> Best regards,
> Sam
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>


-- 
brandon s allbery kf8nh
allbery.b at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/f48e2064/attachment.html>

From sam.halliday at gmail.com  Tue Oct  8 14:57:14 2019
From: sam.halliday at gmail.com (Sam Halliday)
Date: Tue, 08 Oct 2019 15:57:14 +0100
Subject: slow load and typecheck
In-Reply-To: <CAKFCL4W0DFjzmVZOGLMVrwZMf24Cr3vBSTiUyLRsPAz8_+fhfw@mail.gmail.com>
References: <87mueb493z.fsf@gmail.com>
 <CAKFCL4Vr6UihK0vYZptgq=Hw2VZD-2S6iRerDfDC0kyDvYaVRw@mail.gmail.com>
 <87ftk34829.fsf@gmail.com>
 <CAKFCL4W0DFjzmVZOGLMVrwZMf24Cr3vBSTiUyLRsPAz8_+fhfw@mail.gmail.com>
Message-ID: <87a7ab475h.fsf@gmail.com>

Brandon Allbery <allbery.b at gmail.com> writes:

> It's doing what you — but not ghc — consider "extra work", though. ghc
> expects to be compiling code, and doesn't have a separate code path for
> "load symbols from an external module by parsing its source code" instead
> of "load symbols from an external module by loading its .hsc file and
> object code", aside from HscInterpreted.


I'm confused: it sounds like you saying that only HscInterpreted can
load symbols of dependencies from object code. Then how does cabal+ghc
do this when I make a change to one file in my project and do a
recompile of the package?

BTW, I am seeing modules going through CodeGen that are not part of the
file's dependency graph... LoadUpTo is behaving more like LoadAll.

-- 
Best regards,
Sam
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/f0645bcb/attachment.sig>

From sam.halliday at gmail.com  Tue Oct  8 14:59:06 2019
From: sam.halliday at gmail.com (Sam Halliday)
Date: Tue, 08 Oct 2019 15:59:06 +0100
Subject: slow load and typecheck
In-Reply-To: <CAKFCL4WJCWbNNFBCNfEeJYkws=HP-Ris8s43S-=5YHVpWqzc4Q@mail.gmail.com>
References: <87mueb493z.fsf@gmail.com> <87k19f48e5.fsf@gmail.com>
 <CALuQ0m-7NA01S8nH6DiAxDN0krHqYuuQ-O352Yx2dhEXVLCk6A@mail.gmail.com>
 <87d0f747fw.fsf@gmail.com>
 <CAKFCL4WJCWbNNFBCNfEeJYkws=HP-Ris8s43S-=5YHVpWqzc4Q@mail.gmail.com>
Message-ID: <877e5f472d.fsf@gmail.com>

Brandon Allbery <allbery.b at gmail.com> writes:

> you really do want those .hi files, otherwise it must compile the
> dependency module to generate one.

Right, exactly! But I thought that's what targetAllowObjCode=True was
doing, is it not? Is there another setting that I'm missing?

Should I use that for all my modules and not just the dependencies?

-- 
Best regards,
Sam
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/31e7550d/attachment.sig>

From allbery.b at gmail.com  Tue Oct  8 15:05:34 2019
From: allbery.b at gmail.com (Brandon Allbery)
Date: Tue, 8 Oct 2019 11:05:34 -0400
Subject: slow load and typecheck
In-Reply-To: <87a7ab475h.fsf@gmail.com>
References: <87mueb493z.fsf@gmail.com>
 <CAKFCL4Vr6UihK0vYZptgq=Hw2VZD-2S6iRerDfDC0kyDvYaVRw@mail.gmail.com>
 <87ftk34829.fsf@gmail.com>
 <CAKFCL4W0DFjzmVZOGLMVrwZMf24Cr3vBSTiUyLRsPAz8_+fhfw@mail.gmail.com>
 <87a7ab475h.fsf@gmail.com>
Message-ID: <CAKFCL4WrGpNB24nw1mM_Qa-xuxdnkcpGxYOORbJnTsx03caUeA@mail.gmail.com>

It reuses the .hi files already built for other modules. Those aren't in
the source directory but under a build directory. If they don't exist
there, it will build the dependencies to create them.

On Tue, Oct 8, 2019 at 10:57 AM Sam Halliday <sam.halliday at gmail.com> wrote:

> Brandon Allbery <allbery.b at gmail.com> writes:
>
> > It's doing what you — but not ghc — consider "extra work", though. ghc
> > expects to be compiling code, and doesn't have a separate code path for
> > "load symbols from an external module by parsing its source code" instead
> > of "load symbols from an external module by loading its .hsc file and
> > object code", aside from HscInterpreted.
>
>
> I'm confused: it sounds like you saying that only HscInterpreted can
> load symbols of dependencies from object code. Then how does cabal+ghc
> do this when I make a change to one file in my project and do a
> recompile of the package?
>
> BTW, I am seeing modules going through CodeGen that are not part of the
> file's dependency graph... LoadUpTo is behaving more like LoadAll.
>
> --
> Best regards,
> Sam
>


-- 
brandon s allbery kf8nh
allbery.b at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/f2256283/attachment.html>

From allbery.b at gmail.com  Tue Oct  8 15:06:17 2019
From: allbery.b at gmail.com (Brandon Allbery)
Date: Tue, 8 Oct 2019 11:06:17 -0400
Subject: slow load and typecheck
In-Reply-To: <877e5f472d.fsf@gmail.com>
References: <87mueb493z.fsf@gmail.com> <87k19f48e5.fsf@gmail.com>
 <CALuQ0m-7NA01S8nH6DiAxDN0krHqYuuQ-O352Yx2dhEXVLCk6A@mail.gmail.com>
 <87d0f747fw.fsf@gmail.com>
 <CAKFCL4WJCWbNNFBCNfEeJYkws=HP-Ris8s43S-=5YHVpWqzc4Q@mail.gmail.com>
 <877e5f472d.fsf@gmail.com>
Message-ID: <CAKFCL4UsfpXcYriOJXPOYL2Q_gL_QuK_8VaaYAnmAxFS38C5wQ@mail.gmail.com>

If they are loading each other, they likewise need .hi files. .o files are
optional if you aren't linking them.

On Tue, Oct 8, 2019 at 10:59 AM Sam Halliday <sam.halliday at gmail.com> wrote:

> Brandon Allbery <allbery.b at gmail.com> writes:
>
> > you really do want those .hi files, otherwise it must compile the
> > dependency module to generate one.
>
> Right, exactly! But I thought that's what targetAllowObjCode=True was
> doing, is it not? Is there another setting that I'm missing?
>
> Should I use that for all my modules and not just the dependencies?
>
> --
> Best regards,
> Sam
>


-- 
brandon s allbery kf8nh
allbery.b at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/5a6d4e38/attachment.html>

From sam.halliday at gmail.com  Tue Oct  8 15:53:06 2019
From: sam.halliday at gmail.com (Sam Halliday)
Date: Tue, 08 Oct 2019 16:53:06 +0100
Subject: slow load and typecheck
In-Reply-To: <CAKFCL4WrGpNB24nw1mM_Qa-xuxdnkcpGxYOORbJnTsx03caUeA@mail.gmail.com>
References: <87mueb493z.fsf@gmail.com>
 <CAKFCL4Vr6UihK0vYZptgq=Hw2VZD-2S6iRerDfDC0kyDvYaVRw@mail.gmail.com>
 <87ftk34829.fsf@gmail.com>
 <CAKFCL4W0DFjzmVZOGLMVrwZMf24Cr3vBSTiUyLRsPAz8_+fhfw@mail.gmail.com>
 <87a7ab475h.fsf@gmail.com>
 <CAKFCL4WrGpNB24nw1mM_Qa-xuxdnkcpGxYOORbJnTsx03caUeA@mail.gmail.com>
Message-ID: <874l0j44kd.fsf@gmail.com>

Brandon Allbery <allbery.b at gmail.com> writes:
> It reuses the .hi files already built for other modules. Those aren't in
> the source directory but under a build directory. If they don't exist
> there, it will build the dependencies to create them.

The .hi files exist in the target directory and my tool has informed the
ghc api about that location, but it's not using them and I don't know
why... I guess I'm asking "how can I make ghc use the .hi files instead
of compiling the .hs files?". It seems to work fine when I use
HscInterpreted instead of HscNothing.

BTW I tried using targetAllowObjCode=True for everything, but it makes
no difference.

-- 
Best regards,
Sam
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/13596b85/attachment.sig>

From ben at well-typed.com  Tue Oct  8 18:16:59 2019
From: ben at well-typed.com (Ben Gamari)
Date: Tue, 08 Oct 2019 14:16:59 -0400
Subject: Proposed changes to merge request workflow
Message-ID: <87blurje5l.fsf@smart-cactus.org>

tl;dr. I would like feedback on a few proposed changes [1] to our merge
       request workflow.


Hello everyone,

Over the past six months I have been monitoring the operation of our
merge request workflow, which arose rather organically in the wake of
the initial move to GitLab. While it works reasonably well, there is
clearly room for improvement:

  * we have no formal way to track the status of in-flight merge
    requests (e.g. for authors to mark an MR as ready for review or
    reviewers to mark work as ready for merge)

  * merge requests still at times languish without review

  * the backport protocol is somewhat error prone and requires a great
    deal of attention to ensure that patches don't slip through the
    cracks

  * there is no technical mechanism to prevent that under-reviewed
    patches from being merged (either intentionally or otherwise) to
    `master`

To address this I propose [1] a few changes to our workflow:

  1. Define explicit phases of the merge request lifecycle,
     systematically identified with labels. This will help to make it
     clear who is responsible for a merge request at every stage of its
     lifecycle.

  2. Make it clear that it is the contributor's responsibility to
     identify reviewers for their merge requests.

  3. Institute a final pre-merge sanity check to ensure that
     patches are adequately reviewed, documented, tested, and have had
     their ticket and MR metadata updated.

Note that this is merely a proposal; I am actively seeking input from
the developer community. Do let me know what you think.

Cheers,

- Ben


[1] https://gitlab.haskell.org/ghc/ghc/wikis/proposals/merge-request-workflow
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/a7f6dbe6/attachment.sig>

From shayne.fletcher at daml.com  Tue Oct  8 18:19:23 2019
From: shayne.fletcher at daml.com (Shayne Fletcher)
Date: Tue, 8 Oct 2019 14:19:23 -0400
Subject: Proposed changes to merge request workflow
In-Reply-To: <87blurje5l.fsf@smart-cactus.org>
References: <87blurje5l.fsf@smart-cactus.org>
Message-ID: <CAKe9RREutFFEGn_Vj-GdrNM7oEiC2LF9kdNeKiFnauzzH8Dr1w@mail.gmail.com>

All sounds very sensible to me.

On Tue, Oct 8, 2019 at 2:17 PM Ben Gamari <ben at well-typed.com> wrote:

> tl;dr. I would like feedback on a few proposed changes [1] to our merge
>        request workflow.
>
>
> Hello everyone,
>
> Over the past six months I have been monitoring the operation of our
> merge request workflow, which arose rather organically in the wake of
> the initial move to GitLab. While it works reasonably well, there is
> clearly room for improvement:
>
>   * we have no formal way to track the status of in-flight merge
>     requests (e.g. for authors to mark an MR as ready for review or
>     reviewers to mark work as ready for merge)
>
>   * merge requests still at times languish without review
>
>   * the backport protocol is somewhat error prone and requires a great
>     deal of attention to ensure that patches don't slip through the
>     cracks
>
>   * there is no technical mechanism to prevent that under-reviewed
>     patches from being merged (either intentionally or otherwise) to
>     `master`
>
> To address this I propose [1] a few changes to our workflow:
>
>   1. Define explicit phases of the merge request lifecycle,
>      systematically identified with labels. This will help to make it
>      clear who is responsible for a merge request at every stage of its
>      lifecycle.
>
>   2. Make it clear that it is the contributor's responsibility to
>      identify reviewers for their merge requests.
>
>   3. Institute a final pre-merge sanity check to ensure that
>      patches are adequately reviewed, documented, tested, and have had
>      their ticket and MR metadata updated.
>
> Note that this is merely a proposal; I am actively seeking input from
> the developer community. Do let me know what you think.
>
> Cheers,
>
> - Ben
>
>
> [1]
> https://gitlab.haskell.org/ghc/ghc/wikis/proposals/merge-request-workflow
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>


-- 
*Shayne Fletcher*
Language Engineer */* +1 917 699 7663
*Digital Asset* <https://digitalasset.com/>, creators of *DAML
<https://daml.com/>*

-- 
This message, and any attachments, is for the intended recipient(s) only, 
may contain information that is privileged, confidential and/or proprietary 
and subject to important terms and conditions available at 
http://www.digitalasset.com/emaildisclaimer.html 
<http://www.digitalasset.com/emaildisclaimer.html>. If you are not the 
intended recipient, please delete this message.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/da37f1d4/attachment.html>

From rae at richarde.dev  Tue Oct  8 21:28:29 2019
From: rae at richarde.dev (Richard Eisenberg)
Date: Tue, 8 Oct 2019 22:28:29 +0100
Subject: Proposed changes to merge request workflow
In-Reply-To: <CAKe9RREutFFEGn_Vj-GdrNM7oEiC2LF9kdNeKiFnauzzH8Dr1w@mail.gmail.com>
References: <87blurje5l.fsf@smart-cactus.org>
 <CAKe9RREutFFEGn_Vj-GdrNM7oEiC2LF9kdNeKiFnauzzH8Dr1w@mail.gmail.com>
Message-ID: <453CAFFF-03C1-4453-9FBC-4D299442FD23@richarde.dev>

+1 from me.

> On Oct 8, 2019, at 7:19 PM, Shayne Fletcher via ghc-devs <ghc-devs at haskell.org> wrote:
> 
> All sounds very sensible to me.
> 
> On Tue, Oct 8, 2019 at 2:17 PM Ben Gamari <ben at well-typed.com <mailto:ben at well-typed.com>> wrote:
> tl;dr. I would like feedback on a few proposed changes [1] to our merge
>        request workflow.
> 
> 
> Hello everyone,
> 
> Over the past six months I have been monitoring the operation of our
> merge request workflow, which arose rather organically in the wake of
> the initial move to GitLab. While it works reasonably well, there is
> clearly room for improvement:
> 
>   * we have no formal way to track the status of in-flight merge
>     requests (e.g. for authors to mark an MR as ready for review or
>     reviewers to mark work as ready for merge)
> 
>   * merge requests still at times languish without review
> 
>   * the backport protocol is somewhat error prone and requires a great
>     deal of attention to ensure that patches don't slip through the
>     cracks
> 
>   * there is no technical mechanism to prevent that under-reviewed
>     patches from being merged (either intentionally or otherwise) to
>     `master`
> 
> To address this I propose [1] a few changes to our workflow:
> 
>   1. Define explicit phases of the merge request lifecycle,
>      systematically identified with labels. This will help to make it
>      clear who is responsible for a merge request at every stage of its
>      lifecycle.
> 
>   2. Make it clear that it is the contributor's responsibility to
>      identify reviewers for their merge requests.
> 
>   3. Institute a final pre-merge sanity check to ensure that
>      patches are adequately reviewed, documented, tested, and have had
>      their ticket and MR metadata updated.
> 
> Note that this is merely a proposal; I am actively seeking input from
> the developer community. Do let me know what you think.
> 
> Cheers,
> 
> - Ben
> 
> 
> [1] https://gitlab.haskell.org/ghc/ghc/wikis/proposals/merge-request-workflow <https://gitlab.haskell.org/ghc/ghc/wikis/proposals/merge-request-workflow>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs <http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs>
> 
> 
> -- 
> Shayne Fletcher
> Language Engineer / +1 917 699 7663
> Digital Asset <https://digitalasset.com/>, creators of DAML <https://daml.com/>
> 
> This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.digitalasset.com/emaildisclaimer.html <http://www.digitalasset.com/emaildisclaimer.html>. If you are not the intended recipient, please delete this message._______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191008/de5c3513/attachment-0001.html>

From qdunkan at gmail.com  Wed Oct  9 04:16:49 2019
From: qdunkan at gmail.com (Evan Laforge)
Date: Tue, 8 Oct 2019 21:16:49 -0700
Subject: slow load and typecheck
In-Reply-To: <87mueb493z.fsf@gmail.com>
References: <87mueb493z.fsf@gmail.com>
Message-ID: <CACbaDy4Sm4SwhJFHDcX04xsV9UmvnhC12DRvj+Qu8u2zds0Tpw@mail.gmail.com>

I'm not sure if I'm doing the same thing as you, but I use a GHC repl
for my program.  It loads a 200-300 modules in under a second, and is
able to reload changed ones dynamically, just like ghci.

The source is https://github.com/elaforge/karya/blob/work/Cmd/ReplGhc.hs,
see 'parse_flags' and its call in 'interpreter'.

The main thing is getting ghc to load the .o files, but if ghci will
do it, then the ghc API will do it.  You just have to get the flags to
be the same, and ghc is pretty opaque about why it doesn't want to
load.  There is a -ddump-something flag but it doesn't say what flags
actually changed.  I actually wound up patching ghc to add that
feature.

On Tue, Oct 8, 2019 at 7:15 AM Sam Halliday <sam.halliday at gmail.com> wrote:
>
> Hello all,
>
> I am writing an interactive tool using the ghc api. It is able to load
> and typecheck a source file in a user's package.
>
> I obtain the flags that cabal uses to compile the user's package via the
> hie-bios trick, and I `parseDynamicFlagsCmdLine' them inside my tool,
> then I `setTargets' all the home modules (with targetAllowObjCode=True).
>
> I use HscNothing and NoLink because I only want access to the trees, I
> don't want to produce any output files.
>
> For the file that I wish to inspect, I `removeTarget' the module and
> `addTarget` it again but this time providing the full path to the file
> and don't allow object code.
>
> Then I LoadUpTo and typecheck. For the sake of simplicity, let's assume
> that the file under inspection only has a module definiton and no
> imports or top levels.
>
> Functionally, my code is working great and I am able to do what I want
> with the typechecked tree.
>
> However, load is very slow (~10 seconds user time) on large projects.
> Here is a cpu time trace of my program (milliseconds):
>
>   main              1
>   parse flags      93
>   load          20436
>   typecheck     20437
>
> I can enable a bit more ghc timing info via -Rghc-timings and I see
>
>   !!! Chasing dependencies: finished in 157.20 milliseconds, allocated
>       528.112 megabytes
>
> This seems fine, anything sub-second is ok.
>
> But then I see a bunch of home modules in CodeGen that I was not expecting:
>
>    !!! CodeGen [My.Module.Dependency]:
>        finished in 3335.62 milliseconds, allocated 270.615 megabytes
>
> So it looks like the targetAllowObjCode is being ignored... is there any
> way to force it? Actually I'd prefer to fail fast than to ever compile
> or codegen a dependency module.
>
>
> I know that it should be possible to load the module a lot faster
> because if I make a small change in the file under inspection and ask
> cabal to recompile the module it is super fast (less than a second).
>
> Could somebody who understands how incremental/partial compiles work
> please help me out?
>
>
> PS: If this textual description is confusing, I could put together a
> minimal reproduction and example project but it will take me some time
> to do that.
>
> --
> Best regards,
> Sam
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From sam.halliday at gmail.com  Wed Oct  9 06:14:45 2019
From: sam.halliday at gmail.com (Sam Halliday)
Date: Wed, 09 Oct 2019 07:14:45 +0100
Subject: slow load and typecheck
In-Reply-To: <CACbaDy4Sm4SwhJFHDcX04xsV9UmvnhC12DRvj+Qu8u2zds0Tpw@mail.gmail.com>
References: <87mueb493z.fsf@gmail.com>
 <CACbaDy4Sm4SwhJFHDcX04xsV9UmvnhC12DRvj+Qu8u2zds0Tpw@mail.gmail.com>
Message-ID: <87d0f6qwbu.fsf@gmail.com>

Thanks Evan,

Evan Laforge writes:
> https://github.com/elaforge/karya/blob/work/Cmd/ReplGhc.hs

Yes what you're doing is very similar. I'm also adding args_left as
modules because they tend to be RTS and home modules. But it looks like
you always use False for object code. You're also using ghcMode so I'll
investigate if I need to force a mode.


> ghc is pretty opaque about why it doesn't want to load.

:-D

-- 
Best regards,
Sam
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191009/cb69269b/attachment.sig>

From rae at richarde.dev  Wed Oct  9 08:38:43 2019
From: rae at richarde.dev (Richard Eisenberg)
Date: Wed, 9 Oct 2019 09:38:43 +0100
Subject: validate-x86_64-darwin failing
Message-ID: <35FB4B62-1C19-4EF5-94C4-8DECCD981B8C@richarde.dev>

Hi devs,

validate-x86_64-darwin seems to be failing. I've restarted it when I came across the failure, but several small patches last night tripped on this.

Richard

From rae at richarde.dev  Wed Oct  9 08:45:57 2019
From: rae at richarde.dev (Richard Eisenberg)
Date: Wed, 9 Oct 2019 09:45:57 +0100
Subject: T13615 fragile?
Message-ID: <55F3B613-D669-46EC-83F2-D4F3928C78AC@richarde.dev>

Hi devs,

https://gitlab.haskell.org/rae/ghc/-/jobs/173504 <https://gitlab.haskell.org/rae/ghc/-/jobs/173504> tells me that T13615 failed -- but only in DEBUG mode. My patch is nowhere near this code. Is this a new fragile test?

The number of fragile tests seems to be growing quickly. And once a test is labeled fragile, we're essentially no longer tracking the test, if I understand correctly. So that's a bit unfortunate.

Thanks,
Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191009/79053e3f/attachment.html>

From alp at well-typed.com  Wed Oct  9 08:53:12 2019
From: alp at well-typed.com (Alp Mestanogullari)
Date: Wed, 9 Oct 2019 10:53:12 +0200
Subject: T13615 fragile?
In-Reply-To: <55F3B613-D669-46EC-83F2-D4F3928C78AC@richarde.dev>
References: <55F3B613-D669-46EC-83F2-D4F3928C78AC@richarde.dev>
Message-ID: <1e5b1a7d-b9e0-b79e-81d5-0f4e22312e19@well-typed.com>

For what it's worth, I found two recent tickets documenting failures for 
that very same test: #16897 and #17269.

On 09/10/2019 10:45, Richard Eisenberg wrote:
> Hi devs,
>
> https://gitlab.haskell.org/rae/ghc/-/jobs/173504 tells me that T13615 
> failed -- but only in DEBUG mode. My patch is nowhere near this code. 
> Is this a new fragile test?
>
> The number of fragile tests seems to be growing quickly. And once a 
> test is labeled fragile, we're essentially no longer tracking the 
> test, if I understand correctly. So that's a bit unfortunate.
>
> Thanks,
> Richard
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-- 
Alp Mestanogullari, Haskell Consultant
Well-Typed LLP, https://www.well-typed.com/

Registered in England and Wales, OC335890
118 Wymering Mansions, Wymering Road, London, W9 2NF, England

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191009/ee10c51d/attachment.html>

From matthewtpickering at gmail.com  Wed Oct  9 10:18:02 2019
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Wed, 9 Oct 2019 11:18:02 +0100
Subject: Proposed changes to merge request workflow
In-Reply-To: <87blurje5l.fsf@smart-cactus.org>
References: <87blurje5l.fsf@smart-cactus.org>
Message-ID: <CALuQ0m-GcVMAaQMw8KjJ--mLCOAgWOKAKEMqymDVcLoAuAhYzA@mail.gmail.com>

Sounds good in principal but I object to

>  Make it clear that it is the contributor's responsibility to identify reviewers for their merge requests.

Asking for reviews is one of the most frustrating parts of
contributing patches, even if you know who to ask! So I think the
maintainer's should be responsible for finding suitable and willing
reviewers.

Cheers,

Matt

On Tue, Oct 8, 2019 at 7:17 PM Ben Gamari <ben at well-typed.com> wrote:
>
> tl;dr. I would like feedback on a few proposed changes [1] to our merge
>        request workflow.
>
>
> Hello everyone,
>
> Over the past six months I have been monitoring the operation of our
> merge request workflow, which arose rather organically in the wake of
> the initial move to GitLab. While it works reasonably well, there is
> clearly room for improvement:
>
>   * we have no formal way to track the status of in-flight merge
>     requests (e.g. for authors to mark an MR as ready for review or
>     reviewers to mark work as ready for merge)
>
>   * merge requests still at times languish without review
>
>   * the backport protocol is somewhat error prone and requires a great
>     deal of attention to ensure that patches don't slip through the
>     cracks
>
>   * there is no technical mechanism to prevent that under-reviewed
>     patches from being merged (either intentionally or otherwise) to
>     `master`
>
> To address this I propose [1] a few changes to our workflow:
>
>   1. Define explicit phases of the merge request lifecycle,
>      systematically identified with labels. This will help to make it
>      clear who is responsible for a merge request at every stage of its
>      lifecycle.
>
>   2. Make it clear that it is the contributor's responsibility to
>      identify reviewers for their merge requests.
>
>   3. Institute a final pre-merge sanity check to ensure that
>      patches are adequately reviewed, documented, tested, and have had
>      their ticket and MR metadata updated.
>
> Note that this is merely a proposal; I am actively seeking input from
> the developer community. Do let me know what you think.
>
> Cheers,
>
> - Ben
>
>
> [1] https://gitlab.haskell.org/ghc/ghc/wikis/proposals/merge-request-workflow
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From oleg.grenrus at iki.fi  Wed Oct  9 10:26:56 2019
From: oleg.grenrus at iki.fi (Oleg Grenrus)
Date: Wed, 9 Oct 2019 13:26:56 +0300
Subject: Proposed changes to merge request workflow
In-Reply-To: <CALuQ0m-GcVMAaQMw8KjJ--mLCOAgWOKAKEMqymDVcLoAuAhYzA@mail.gmail.com>
References: <87blurje5l.fsf@smart-cactus.org>
 <CALuQ0m-GcVMAaQMw8KjJ--mLCOAgWOKAKEMqymDVcLoAuAhYzA@mail.gmail.com>
Message-ID: <f3a5ed93-15e2-d32f-a4cb-9a05a1d6af7f@iki.fi>

+1

On 9.10.2019 13.18, Matthew Pickering wrote:
> Sounds good in principal but I object to
>
>>   Make it clear that it is the contributor's responsibility to identify reviewers for their merge requests.
> Asking for reviews is one of the most frustrating parts of
> contributing patches, even if you know who to ask! So I think the
> maintainer's should be responsible for finding suitable and willing
> reviewers.
>
> Cheers,
>
> Matt
>
> On Tue, Oct 8, 2019 at 7:17 PM Ben Gamari <ben at well-typed.com> wrote:
>> tl;dr. I would like feedback on a few proposed changes [1] to our merge
>>         request workflow.
>>
>>
>> Hello everyone,
>>
>> Over the past six months I have been monitoring the operation of our
>> merge request workflow, which arose rather organically in the wake of
>> the initial move to GitLab. While it works reasonably well, there is
>> clearly room for improvement:
>>
>>    * we have no formal way to track the status of in-flight merge
>>      requests (e.g. for authors to mark an MR as ready for review or
>>      reviewers to mark work as ready for merge)
>>
>>    * merge requests still at times languish without review
>>
>>    * the backport protocol is somewhat error prone and requires a great
>>      deal of attention to ensure that patches don't slip through the
>>      cracks
>>
>>    * there is no technical mechanism to prevent that under-reviewed
>>      patches from being merged (either intentionally or otherwise) to
>>      `master`
>>
>> To address this I propose [1] a few changes to our workflow:
>>
>>    1. Define explicit phases of the merge request lifecycle,
>>       systematically identified with labels. This will help to make it
>>       clear who is responsible for a merge request at every stage of its
>>       lifecycle.
>>
>>    2. Make it clear that it is the contributor's responsibility to
>>       identify reviewers for their merge requests.
>>
>>    3. Institute a final pre-merge sanity check to ensure that
>>       patches are adequately reviewed, documented, tested, and have had
>>       their ticket and MR metadata updated.
>>
>> Note that this is merely a proposal; I am actively seeking input from
>> the developer community. Do let me know what you think.
>>
>> Cheers,
>>
>> - Ben
>>
>>
>> [1] https://gitlab.haskell.org/ghc/ghc/wikis/proposals/merge-request-workflow
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From simonpj at microsoft.com  Wed Oct  9 10:31:34 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Wed, 9 Oct 2019 10:31:34 +0000
Subject: Proposed changes to merge request workflow
In-Reply-To: <CALuQ0m-GcVMAaQMw8KjJ--mLCOAgWOKAKEMqymDVcLoAuAhYzA@mail.gmail.com>
References: <87blurje5l.fsf@smart-cactus.org>
 <CALuQ0m-GcVMAaQMw8KjJ--mLCOAgWOKAKEMqymDVcLoAuAhYzA@mail.gmail.com>
Message-ID: <AM6PR83MB0279F280D5505B453A5A2BF0AD950@AM6PR83MB0279.EURPRD83.prod.outlook.com>

|  >  Make it clear that it is the contributor's responsibility to identify
|  reviewers for their merge requests.
|  
|  Asking for reviews is one of the most frustrating parts of
|  contributing patches, even if you know who to ask! So I think the
|  maintainer's should be responsible for finding suitable and willing
|  reviewers.

It is true that it's hard to find reviewers. But if it's hard for the author it is also hard for the maintainers.  A patch is a service that an author is offering, which is great.   But every patch is owed, as a matter of right, suitable and willing reviewers, the patch is /also/ a blank cheque that any author can write, but it's up to someone else to pay.   That's not good either.  No author has an unlimited call on the time of other volunteers, and I don't think any author truly expects that.

It's an informal gift economy.  I review your patches (a) because I have learned that you have good judgement and write good code (b) because I want the bug that you are fixing to be fixed and (c) because you give me all sorts of helpful feedback about my patches, or otherwise contribute to the community in constructive ways.

That may make it hard for /new/ authors to get started.  Being an assiduous reviewer is an excellent plan, because it gets you into GHC's code base, guided by someone else's work; and it earns you all those good-contributor points.  But even then it may be hard.  So I think it's absolutely reasonable for authors to ask for help in finding reviewers.

But simply saying that it's "the maintainers" responsibility to find reviewers goes much too far in the other direction, IMHO.

Perhaps we should articulate some of this thinking.

Simon

|  -----Original Message-----
|  From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Matthew
|  Pickering
|  Sent: 09 October 2019 11:18
|  To: Ben Gamari <ben at well-typed.com>
|  Cc: ghc-devs at haskell.org
|  Subject: Re: Proposed changes to merge request workflow
|  
|  Sounds good in principal but I object to
|  
|  >  Make it clear that it is the contributor's responsibility to identify
|  reviewers for their merge requests.
|  
|  Asking for reviews is one of the most frustrating parts of
|  contributing patches, even if you know who to ask! So I think the
|  maintainer's should be responsible for finding suitable and willing
|  reviewers.
|  
|  Cheers,
|  
|  Matt
|  
|  On Tue, Oct 8, 2019 at 7:17 PM Ben Gamari <ben at well-typed.com> wrote:
|  >
|  > tl;dr. I would like feedback on a few proposed changes [1] to our merge
|  >        request workflow.
|  >
|  >
|  > Hello everyone,
|  >
|  > Over the past six months I have been monitoring the operation of our
|  > merge request workflow, which arose rather organically in the wake of
|  > the initial move to GitLab. While it works reasonably well, there is
|  > clearly room for improvement:
|  >
|  >   * we have no formal way to track the status of in-flight merge
|  >     requests (e.g. for authors to mark an MR as ready for review or
|  >     reviewers to mark work as ready for merge)
|  >
|  >   * merge requests still at times languish without review
|  >
|  >   * the backport protocol is somewhat error prone and requires a great
|  >     deal of attention to ensure that patches don't slip through the
|  >     cracks
|  >
|  >   * there is no technical mechanism to prevent that under-reviewed
|  >     patches from being merged (either intentionally or otherwise) to
|  >     `master`
|  >
|  > To address this I propose [1] a few changes to our workflow:
|  >
|  >   1. Define explicit phases of the merge request lifecycle,
|  >      systematically identified with labels. This will help to make it
|  >      clear who is responsible for a merge request at every stage of its
|  >      lifecycle.
|  >
|  >   2. Make it clear that it is the contributor's responsibility to
|  >      identify reviewers for their merge requests.
|  >
|  >   3. Institute a final pre-merge sanity check to ensure that
|  >      patches are adequately reviewed, documented, tested, and have had
|  >      their ticket and MR metadata updated.
|  >
|  > Note that this is merely a proposal; I am actively seeking input from
|  > the developer community. Do let me know what you think.
|  >
|  > Cheers,
|  >
|  > - Ben
|  >
|  >
|  > [1]
|  https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.h
|  askell.org%2Fghc%2Fghc%2Fwikis%2Fproposals%2Fmerge-request-
|  workflow&amp;data=02%7C01%7Csimonpj%40microsoft.com%7Cd1199fd308b442cf744f
|  08d74ca2074b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6370621311033130
|  52&amp;sdata=SxBADAuF%2FvGzduaytetUzIxGr8lC%2BjTX2eCLNEoOCkQ%3D&amp;reserv
|  ed=0
|  > _______________________________________________
|  > ghc-devs mailing list
|  > ghc-devs at haskell.org
|  >
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
|  ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7Cd1199fd308b442cf744f08d7
|  4ca2074b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637062131103313052&a
|  mp;sdata=T%2FyLoRH9BTIVPxMzF0%2BAa3c20qCBkhvQrp53FtROz40%3D&amp;reserved=0
|  _______________________________________________
|  ghc-devs mailing list
|  ghc-devs at haskell.org
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
|  ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7Cd1199fd308b442cf744f08d7
|  4ca2074b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637062131103323047&a
|  mp;sdata=IwsIP3P6W5qtsLxfePbYOWTXdPLttNMLHWXkuTtVWgI%3D&amp;reserved=0

From oleg.grenrus at iki.fi  Wed Oct  9 11:24:36 2019
From: oleg.grenrus at iki.fi (Oleg Grenrus)
Date: Wed, 9 Oct 2019 14:24:36 +0300
Subject: Proposed changes to merge request workflow
In-Reply-To: <AM6PR83MB0279F280D5505B453A5A2BF0AD950@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <87blurje5l.fsf@smart-cactus.org>
 <CALuQ0m-GcVMAaQMw8KjJ--mLCOAgWOKAKEMqymDVcLoAuAhYzA@mail.gmail.com>
 <AM6PR83MB0279F280D5505B453A5A2BF0AD950@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <842ad569-4bdc-751b-3abb-5bbf14d16187@iki.fi>

Becoming a recognized reviewer before starting writing code feels 
perverse for me.

Linux kernel writes in 
https://www.kernel.org/doc/html/v4.17/process/submitting-patches.html#select-the-recipients-for-your-patch 
as follows. (Note that patches are submitted by sending emails, read 
accordingly)

 > You should always copy the appropriate subsystem maintainer(s) on any 
patch to code that they maintain; look through the MAINTAINERS file and 
the source code revision history to see who those maintainers are. The 
script scripts/get_maintainer.pl can be very useful at this step. If you 
cannot find a maintainer for the subsystem you are working on, Andrew 
Morton (akpm at linux-foundation.org) serves as a maintainer of last resort.

GHC already has CODEOWNERS file, and there could be a person of last 
resort. There should be more names there though, e.g. /libraries/base/ 
should have the whole CLC, not only HVR; template-haskell could have 
Ryan Scott, etc. As the first commit message of CODEOWNERS says:

 > GitLab uses this file to suggest reviewers based upon the files that 
a Merge Request touches.

Kernel guidelines also have a section about trivial patches. Thing to 
learn, there is a light way to get trivial patches in, but what's a 
trivial patch is objectively defined. An actual trivial patch monkey is 
a real person, but to my understanding it's a circulating role.

 > For small patches you may want to CC the Trivial Patch Monkey 
trivial at kernel.org which collects “trivial” patches. Have a look into 
the MAINTAINERS file for its current manager.
 >
 > Trivial patches must qualify for one of the following rules:
 >
 > - Spelling fixes in documentation
 > - Spelling fixes for errors which could break grep(1)
 > - Warning fixes (cluttering with useless warnings is bad)
 > - Compilation fixes (only if they are actually correct)
 > - Runtime fixes (only if they actually fix things)
 > - Removing use of deprecated functions/macros
 > - Contact detail and documentation fixes
 > - Non-portable code replaced by portable code (even in arch-specific, 
since people copy, as long as it’s trivial)
 > - Any fix by the author/maintainer of the file (ie. patch monkey in 
re-transmission mode)

- Oleg


On 9.10.2019 13.31, Simon Peyton Jones via ghc-devs wrote:
> |  >  Make it clear that it is the contributor's responsibility to identify
> |  reviewers for their merge requests.
> |
> |  Asking for reviews is one of the most frustrating parts of
> |  contributing patches, even if you know who to ask! So I think the
> |  maintainer's should be responsible for finding suitable and willing
> |  reviewers.
>
> It is true that it's hard to find reviewers. But if it's hard for the author it is also hard for the maintainers.  A patch is a service that an author is offering, which is great.   But every patch is owed, as a matter of right, suitable and willing reviewers, the patch is /also/ a blank cheque that any author can write, but it's up to someone else to pay.   That's not good either.  No author has an unlimited call on the time of other volunteers, and I don't think any author truly expects that.
>
> It's an informal gift economy.  I review your patches (a) because I have learned that you have good judgement and write good code (b) because I want the bug that you are fixing to be fixed and (c) because you give me all sorts of helpful feedback about my patches, or otherwise contribute to the community in constructive ways.
>
> That may make it hard for /new/ authors to get started.  Being an assiduous reviewer is an excellent plan, because it gets you into GHC's code base, guided by someone else's work; and it earns you all those good-contributor points.  But even then it may be hard.  So I think it's absolutely reasonable for authors to ask for help in finding reviewers.
>
> But simply saying that it's "the maintainers" responsibility to find reviewers goes much too far in the other direction, IMHO.
>
> Perhaps we should articulate some of this thinking.
>
> Simon
>
> |  -----Original Message-----
> |  From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Matthew
> |  Pickering
> |  Sent: 09 October 2019 11:18
> |  To: Ben Gamari <ben at well-typed.com>
> |  Cc: ghc-devs at haskell.org
> |  Subject: Re: Proposed changes to merge request workflow
> |
> |  Sounds good in principal but I object to
> |
> |  >  Make it clear that it is the contributor's responsibility to identify
> |  reviewers for their merge requests.
> |
> |  Asking for reviews is one of the most frustrating parts of
> |  contributing patches, even if you know who to ask! So I think the
> |  maintainer's should be responsible for finding suitable and willing
> |  reviewers.
> |
> |  Cheers,
> |
> |  Matt
> |
> |  On Tue, Oct 8, 2019 at 7:17 PM Ben Gamari <ben at well-typed.com> wrote:
> |  >
> |  > tl;dr. I would like feedback on a few proposed changes [1] to our merge
> |  >        request workflow.
> |  >
> |  >
> |  > Hello everyone,
> |  >
> |  > Over the past six months I have been monitoring the operation of our
> |  > merge request workflow, which arose rather organically in the wake of
> |  > the initial move to GitLab. While it works reasonably well, there is
> |  > clearly room for improvement:
> |  >
> |  >   * we have no formal way to track the status of in-flight merge
> |  >     requests (e.g. for authors to mark an MR as ready for review or
> |  >     reviewers to mark work as ready for merge)
> |  >
> |  >   * merge requests still at times languish without review
> |  >
> |  >   * the backport protocol is somewhat error prone and requires a great
> |  >     deal of attention to ensure that patches don't slip through the
> |  >     cracks
> |  >
> |  >   * there is no technical mechanism to prevent that under-reviewed
> |  >     patches from being merged (either intentionally or otherwise) to
> |  >     `master`
> |  >
> |  > To address this I propose [1] a few changes to our workflow:
> |  >
> |  >   1. Define explicit phases of the merge request lifecycle,
> |  >      systematically identified with labels. This will help to make it
> |  >      clear who is responsible for a merge request at every stage of its
> |  >      lifecycle.
> |  >
> |  >   2. Make it clear that it is the contributor's responsibility to
> |  >      identify reviewers for their merge requests.
> |  >
> |  >   3. Institute a final pre-merge sanity check to ensure that
> |  >      patches are adequately reviewed, documented, tested, and have had
> |  >      their ticket and MR metadata updated.
> |  >
> |  > Note that this is merely a proposal; I am actively seeking input from
> |  > the developer community. Do let me know what you think.
> |  >
> |  > Cheers,
> |  >
> |  > - Ben
> |  >
> |  >
> |  > [1]
> |  https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.h
> |  askell.org%2Fghc%2Fghc%2Fwikis%2Fproposals%2Fmerge-request-
> |  workflow&amp;data=02%7C01%7Csimonpj%40microsoft.com%7Cd1199fd308b442cf744f
> |  08d74ca2074b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6370621311033130
> |  52&amp;sdata=SxBADAuF%2FvGzduaytetUzIxGr8lC%2BjTX2eCLNEoOCkQ%3D&amp;reserv
> |  ed=0
> |  > _______________________________________________
> |  > ghc-devs mailing list
> |  > ghc-devs at haskell.org
> |  >
> |  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
> |  ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
> |  devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7Cd1199fd308b442cf744f08d7
> |  4ca2074b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637062131103313052&a
> |  mp;sdata=T%2FyLoRH9BTIVPxMzF0%2BAa3c20qCBkhvQrp53FtROz40%3D&amp;reserved=0
> |  _______________________________________________
> |  ghc-devs mailing list
> |  ghc-devs at haskell.org
> |  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
> |  ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
> |  devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7Cd1199fd308b442cf744f08d7
> |  4ca2074b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637062131103323047&a
> |  mp;sdata=IwsIP3P6W5qtsLxfePbYOWTXdPLttNMLHWXkuTtVWgI%3D&amp;reserved=0
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From ben at well-typed.com  Wed Oct  9 11:39:10 2019
From: ben at well-typed.com (Ben Gamari)
Date: Wed, 09 Oct 2019 07:39:10 -0400
Subject: T13615 fragile?
In-Reply-To: <55F3B613-D669-46EC-83F2-D4F3928C78AC@richarde.dev>
References: <55F3B613-D669-46EC-83F2-D4F3928C78AC@richarde.dev>
Message-ID: <85E1BA4F-F45F-431A-A3AA-D39E46902A9B@well-typed.com>

Unfortunately T13615 is indeed known to be fragile. This is tracked as #17269, ss you discovered. This is especially scary given that the test tests some rather subtle RTS logic. It's been on my list of things to investigate but unfortunately I've not been able to reproduce the failure locally so progress has been slow. 

Given that it seems to fail fairly reliably on CI in some cases we probably ought to mark it as fragile. 

Note that it's not quite true that fragile test outcomes are no longer tracked. We maintain a database of test outcomes from CI runs which includes both passing and failing ribs of fragile tests. I've written about this infrastructure here [1]. 

Cheers, 

- Ben 


[1] https://www.haskell.org/ghc/blog/20190708-testsuite-work.html




On October 9, 2019 4:45:57 AM EDT, Richard Eisenberg <rae at richarde.dev> wrote:
>Hi devs,
>
>https://gitlab.haskell.org/rae/ghc/-/jobs/173504
><https://gitlab.haskell.org/rae/ghc/-/jobs/173504> tells me that T13615
>failed -- but only in DEBUG mode. My patch is nowhere near this code.
>Is this a new fragile test?
>
>The number of fragile tests seems to be growing quickly. And once a
>test is labeled fragile, we're essentially no longer tracking the test,
>if I understand correctly. So that's a bit unfortunate.
>
>Thanks,
>Richard

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191009/371d0936/attachment.html>

From ben at smart-cactus.org  Wed Oct  9 12:21:11 2019
From: ben at smart-cactus.org (Ben Gamari)
Date: Wed, 09 Oct 2019 08:21:11 -0400
Subject: validate-x86_64-darwin failing
In-Reply-To: <35FB4B62-1C19-4EF5-94C4-8DECCD981B8C@richarde.dev>
References: <35FB4B62-1C19-4EF5-94C4-8DECCD981B8C@richarde.dev>
Message-ID: <878spujej1.fsf@smart-cactus.org>

Richard Eisenberg <rae at richarde.dev> writes:

> Hi devs,
>
> validate-x86_64-darwin seems to be failing. I've restarted it when I
> came across the failure, but several small patches last night tripped
> on this.
>
Which jobs in particular failed? I looked through the recent job list
[1] but all Darwin jobs in the last 10 pages of jobs were either skipped
(due to another job failing) or passing.

Cheers,

- Ben

[1] https://gitlab.haskell.org/ghc/ghc/-/jobs
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191009/c52be851/attachment.sig>

From rae at richarde.dev  Wed Oct  9 14:19:35 2019
From: rae at richarde.dev (Richard Eisenberg)
Date: Wed, 9 Oct 2019 15:19:35 +0100
Subject: validate-x86_64-darwin failing
In-Reply-To: <878spujej1.fsf@smart-cactus.org>
References: <35FB4B62-1C19-4EF5-94C4-8DECCD981B8C@richarde.dev>
 <878spujej1.fsf@smart-cactus.org>
Message-ID: <928A805D-866A-45CE-9CD1-D494D5D63183@richarde.dev>

Here's one: https://gitlab.haskell.org/rae/ghc/-/jobs/173464 <https://gitlab.haskell.org/rae/ghc/-/jobs/173464>  I don't know if you can get more information...

> On Oct 9, 2019, at 1:21 PM, Ben Gamari <ben at smart-cactus.org> wrote:
> 
> Richard Eisenberg <rae at richarde.dev> writes:
> 
>> Hi devs,
>> 
>> validate-x86_64-darwin seems to be failing. I've restarted it when I
>> came across the failure, but several small patches last night tripped
>> on this.
>> 
> Which jobs in particular failed? I looked through the recent job list
> [1] but all Darwin jobs in the last 10 pages of jobs were either skipped
> (due to another job failing) or passing.
> 
> Cheers,
> 
> - Ben
> 
> [1] https://gitlab.haskell.org/ghc/ghc/-/jobs

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191009/44dd7e4c/attachment.html>

From shayne.fletcher at daml.com  Thu Oct 10 11:41:47 2019
From: shayne.fletcher at daml.com (Shayne Fletcher)
Date: Thu, 10 Oct 2019 07:41:47 -0400
Subject: GHC_STAGE
Message-ID: <CAKe9RRGCuJZuGr-VAHUsXSgkvrAgKVxaoX7hDvP-WLKPifHKxw@mail.gmail.com>

Hi Ben,

I observed warnings of this kind in a couple of places building head
yesterday:
```
compiler/utils/FastString.hs:336:5: error:
     warning: 'GHC_STAGE' is not defined, evaluates to 0 [-Wundef]
    |
336 | #if GHC_STAGE < 2
    |     ^
#if GHC_STAGE < 2
    ^
```

```
compiler/main/DynFlags.hs:323:5: error:
     warning: 'GHC_STAGE' is not defined, evaluates to 0 [-Wundef]
    |
323 | #if GHC_STAGE >= 2
    |     ^
#if GHC_STAGE >= 2
    ^
```
I was using the `hadrian/build.stack.sh` approach. Figured you'd want to
know!

-- 
*Shayne Fletcher*
Language Engineer */* +1 917 699 7663
*Digital Asset* <https://digitalasset.com/>, creators of *DAML
<https://daml.com/>*

-- 
This message, and any attachments, is for the intended recipient(s) only, 
may contain information that is privileged, confidential and/or proprietary 
and subject to important terms and conditions available at 
http://www.digitalasset.com/emaildisclaimer.html 
<http://www.digitalasset.com/emaildisclaimer.html>. If you are not the 
intended recipient, please delete this message.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191010/ec1d0ecb/attachment.html>

From klebinger.andreas at gmx.at  Thu Oct 10 13:53:57 2019
From: klebinger.andreas at gmx.at (Andreas Klebinger)
Date: Thu, 10 Oct 2019 15:53:57 +0200
Subject: Unsquashed merge requests
Message-ID: <ab70e4b4-b457-df6a-5fc9-6e7438eeb805@gmx.at>

Hello devs,

I've recently seen a few MR's make it into master with commits
which rather obviously were meant to be squashed.

Please look out for this when assigning MRs to marge.

Cheers
Andreas

From ml at stefansf.de  Thu Oct 10 16:46:35 2019
From: ml at stefansf.de (Stefan Schulze Frielinghaus)
Date: Thu, 10 Oct 2019 18:46:35 +0200
Subject: GHC_STAGE
In-Reply-To: <CAKe9RRGCuJZuGr-VAHUsXSgkvrAgKVxaoX7hDvP-WLKPifHKxw@mail.gmail.com>
References: <CAKe9RRGCuJZuGr-VAHUsXSgkvrAgKVxaoX7hDvP-WLKPifHKxw@mail.gmail.com>
Message-ID: <20191010164635.GA16766@localhost.localdomain>

Hi Shayne,

does commit 35cc5eff fix your problem?

Cheers,
Stefan

On Thu, Oct 10, 2019 at 07:41:47AM -0400, Shayne Fletcher via ghc-devs wrote:
> Hi Ben,
> 
> I observed warnings of this kind in a couple of places building head
> yesterday:
> ```
> compiler/utils/FastString.hs:336:5: error:
>      warning: 'GHC_STAGE' is not defined, evaluates to 0 [-Wundef]
>     |
> 336 | #if GHC_STAGE < 2
>     |     ^
> #if GHC_STAGE < 2
>     ^
> ```
> 
> ```
> compiler/main/DynFlags.hs:323:5: error:
>      warning: 'GHC_STAGE' is not defined, evaluates to 0 [-Wundef]
>     |
> 323 | #if GHC_STAGE >= 2
>     |     ^
> #if GHC_STAGE >= 2
>     ^
> ```
> I was using the `hadrian/build.stack.sh` approach. Figured you'd want to
> know!
> 
> -- 
> *Shayne Fletcher*
> Language Engineer */* +1 917 699 7663
> *Digital Asset* <https://digitalasset.com/>, creators of *DAML
> <https://daml.com/>*
> 
> -- 
> This message, and any attachments, is for the intended recipient(s) only, 
> may contain information that is privileged, confidential and/or proprietary 
> and subject to important terms and conditions available at 
> http://www.digitalasset.com/emaildisclaimer.html 
> <http://www.digitalasset.com/emaildisclaimer.html>. If you are not the 
> intended recipient, please delete this message.

> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From shayne.fletcher at daml.com  Thu Oct 10 17:24:10 2019
From: shayne.fletcher at daml.com (Shayne Fletcher)
Date: Thu, 10 Oct 2019 13:24:10 -0400
Subject: GHC_STAGE
In-Reply-To: <20191010164635.GA16766@localhost.localdomain>
References: <CAKe9RRGCuJZuGr-VAHUsXSgkvrAgKVxaoX7hDvP-WLKPifHKxw@mail.gmail.com>
 <20191010164635.GA16766@localhost.localdomain>
Message-ID: <CAKe9RRFqGn+PkD1REhH0znXTDwqTmTgD+dKp_T6u-FUTuRabbA@mail.gmail.com>

Well, I thought I had but must have got into a funny state since I can't
reproduce today. Sorry for the noise!

On Thu, Oct 10, 2019 at 12:46 PM Stefan Schulze Frielinghaus <ml at stefansf.de>
wrote:

> Hi Shayne,
>
> does commit 35cc5eff fix your problem?
>
> Cheers,
> Stefan
>
> On Thu, Oct 10, 2019 at 07:41:47AM -0400, Shayne Fletcher via ghc-devs
> wrote:
> > Hi Ben,
> >
> > I observed warnings of this kind in a couple of places building head
> > yesterday:
> > ```
> > compiler/utils/FastString.hs:336:5: error:
> >      warning: 'GHC_STAGE' is not defined, evaluates to 0 [-Wundef]
> >     |
> > 336 | #if GHC_STAGE < 2
> >     |     ^
> > #if GHC_STAGE < 2
> >     ^
> > ```
> >
> > ```
> > compiler/main/DynFlags.hs:323:5: error:
> >      warning: 'GHC_STAGE' is not defined, evaluates to 0 [-Wundef]
> >     |
> > 323 | #if GHC_STAGE >= 2
> >     |     ^
> > #if GHC_STAGE >= 2
> >     ^
> > ```
> > I was using the `hadrian/build.stack.sh` approach. Figured you'd want to
> > know!
> >
> > --
> > *Shayne Fletcher*
> > Language Engineer */* +1 917 699 7663
> > *Digital Asset* <https://digitalasset.com/>, creators of *DAML
> > <https://daml.com/>*
> >
> > --
> > This message, and any attachments, is for the intended recipient(s)
> only,
> > may contain information that is privileged, confidential and/or
> proprietary
> > and subject to important terms and conditions available at
> > http://www.digitalasset.com/emaildisclaimer.html
> > <http://www.digitalasset.com/emaildisclaimer.html>. If you are not the
> > intended recipient, please delete this message.
>
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>

-- 
*Shayne Fletcher*
Language Engineer */* +1 917 699 7663
*Digital Asset* <https://digitalasset.com/>, creators of *DAML
<https://daml.com/>*

-- 
This message, and any attachments, is for the intended recipient(s) only, 
may contain information that is privileged, confidential and/or proprietary 
and subject to important terms and conditions available at 
http://www.digitalasset.com/emaildisclaimer.html 
<http://www.digitalasset.com/emaildisclaimer.html>. If you are not the 
intended recipient, please delete this message.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191010/edc4f28c/attachment.html>

From cheng.shao at tweag.io  Thu Oct 10 19:15:27 2019
From: cheng.shao at tweag.io (Shao, Cheng)
Date: Fri, 11 Oct 2019 03:15:27 +0800
Subject: ByteArray# as a foreign import argument?
Message-ID: <CALHrtFnPXpxCN0JG0g8eEOnut6TRt8Xr4=E55veg7Eq+dhX2Jw@mail.gmail.com>

Hello devs,

I've been trying to figure out how to pass lifted types as foreign
types, then encountered the following code in the `DsCCall` module
(https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/deSugar/DsCCall.hs#L172):

```
  -- Byte-arrays, both mutable and otherwise; hack warning
  -- We're looking for values of type ByteArray, MutableByteArray
  --    data ByteArray          ix = ByteArray        ix ix ByteArray#
  --    data MutableByteArray s ix = MutableByteArray ix ix
(MutableByteArray# s)
  | is_product_type &&
    data_con_arity == 3 &&
    isJust maybe_arg3_tycon &&
    (arg3_tycon ==  byteArrayPrimTyCon ||
     arg3_tycon ==  mutableByteArrayPrimTyCon)
  = do case_bndr <- newSysLocalDs arg_ty
       vars@[_l_var, _r_var, arr_cts_var] <- newSysLocalsDs data_con_arg_tys
       return (Var arr_cts_var,
               \ body -> Case arg case_bndr (exprType body) [(DataAlt
data_con,vars,body)]
              )
```

It seems we allow a "ByteArray" type as a foreign import argument, if
the third field of the datacon is a ByteArray# or MutableByteArray#.
But I can't find such a ByteArray type definition in today's common
packages. What's the rationale for this piece of code?

Cheers,
Cheng

From csaba.hruska at gmail.com  Thu Oct 10 19:54:44 2019
From: csaba.hruska at gmail.com (Csaba Hruska)
Date: Thu, 10 Oct 2019 20:54:44 +0100
Subject: ByteArray# as a foreign import argument?
In-Reply-To: <CALHrtFnPXpxCN0JG0g8eEOnut6TRt8Xr4=E55veg7Eq+dhX2Jw@mail.gmail.com>
References: <CALHrtFnPXpxCN0JG0g8eEOnut6TRt8Xr4=E55veg7Eq+dhX2Jw@mail.gmail.com>
Message-ID: <CANjChEp7bowBc6Fm6gBuRFVn9EwqUho+rz_fgJcadu9TpqaAOg@mail.gmail.com>

It's a primitive type.
https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/prelude/primops.txt.pp#L1388
https://gitlab.haskell.org/ghc/ghc/wikis/commentary/prim-ops

Cheers,
Csaba

On Thu, Oct 10, 2019 at 8:16 PM Shao, Cheng <cheng.shao at tweag.io> wrote:

> Hello devs,
>
> I've been trying to figure out how to pass lifted types as foreign
> types, then encountered the following code in the `DsCCall` module
> (
> https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/deSugar/DsCCall.hs#L172
> ):
>
> ```
>   -- Byte-arrays, both mutable and otherwise; hack warning
>   -- We're looking for values of type ByteArray, MutableByteArray
>   --    data ByteArray          ix = ByteArray        ix ix ByteArray#
>   --    data MutableByteArray s ix = MutableByteArray ix ix
> (MutableByteArray# s)
>   | is_product_type &&
>     data_con_arity == 3 &&
>     isJust maybe_arg3_tycon &&
>     (arg3_tycon ==  byteArrayPrimTyCon ||
>      arg3_tycon ==  mutableByteArrayPrimTyCon)
>   = do case_bndr <- newSysLocalDs arg_ty
>        vars@[_l_var, _r_var, arr_cts_var] <- newSysLocalsDs
> data_con_arg_tys
>        return (Var arr_cts_var,
>                \ body -> Case arg case_bndr (exprType body) [(DataAlt
> data_con,vars,body)]
>               )
> ```
>
> It seems we allow a "ByteArray" type as a foreign import argument, if
> the third field of the datacon is a ByteArray# or MutableByteArray#.
> But I can't find such a ByteArray type definition in today's common
> packages. What's the rationale for this piece of code?
>
> Cheers,
> Cheng
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191010/a1d5ac52/attachment.html>

From csaba.hruska at gmail.com  Thu Oct 10 20:00:01 2019
From: csaba.hruska at gmail.com (Csaba Hruska)
Date: Thu, 10 Oct 2019 21:00:01 +0100
Subject: ByteArray# as a foreign import argument?
In-Reply-To: <CANjChEp7bowBc6Fm6gBuRFVn9EwqUho+rz_fgJcadu9TpqaAOg@mail.gmail.com>
References: <CALHrtFnPXpxCN0JG0g8eEOnut6TRt8Xr4=E55veg7Eq+dhX2Jw@mail.gmail.com>
 <CANjChEp7bowBc6Fm6gBuRFVn9EwqUho+rz_fgJcadu9TpqaAOg@mail.gmail.com>
Message-ID: <CANjChEoCsif0A2X+ve=qj6+d5TnqD=YA+EVOk4SsHLLTarzqGA@mail.gmail.com>

Sorry, maybe I got it wrong. Are you asking if is there any package that
passes ByteArrays via FFI?

On Thu, Oct 10, 2019 at 8:54 PM Csaba Hruska <csaba.hruska at gmail.com> wrote:

> It's a primitive type.
>
> https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/prelude/primops.txt.pp#L1388
> https://gitlab.haskell.org/ghc/ghc/wikis/commentary/prim-ops
>
> Cheers,
> Csaba
>
> On Thu, Oct 10, 2019 at 8:16 PM Shao, Cheng <cheng.shao at tweag.io> wrote:
>
>> Hello devs,
>>
>> I've been trying to figure out how to pass lifted types as foreign
>> types, then encountered the following code in the `DsCCall` module
>> (
>> https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/deSugar/DsCCall.hs#L172
>> ):
>>
>> ```
>>   -- Byte-arrays, both mutable and otherwise; hack warning
>>   -- We're looking for values of type ByteArray, MutableByteArray
>>   --    data ByteArray          ix = ByteArray        ix ix ByteArray#
>>   --    data MutableByteArray s ix = MutableByteArray ix ix
>> (MutableByteArray# s)
>>   | is_product_type &&
>>     data_con_arity == 3 &&
>>     isJust maybe_arg3_tycon &&
>>     (arg3_tycon ==  byteArrayPrimTyCon ||
>>      arg3_tycon ==  mutableByteArrayPrimTyCon)
>>   = do case_bndr <- newSysLocalDs arg_ty
>>        vars@[_l_var, _r_var, arr_cts_var] <- newSysLocalsDs
>> data_con_arg_tys
>>        return (Var arr_cts_var,
>>                \ body -> Case arg case_bndr (exprType body) [(DataAlt
>> data_con,vars,body)]
>>               )
>> ```
>>
>> It seems we allow a "ByteArray" type as a foreign import argument, if
>> the third field of the datacon is a ByteArray# or MutableByteArray#.
>> But I can't find such a ByteArray type definition in today's common
>> packages. What's the rationale for this piece of code?
>>
>> Cheers,
>> Cheng
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191010/7460c372/attachment.html>

From cheng.shao at tweag.io  Thu Oct 10 20:09:42 2019
From: cheng.shao at tweag.io (Shao, Cheng)
Date: Fri, 11 Oct 2019 04:09:42 +0800
Subject: ByteArray# as a foreign import argument?
In-Reply-To: <CANjChEoCsif0A2X+ve=qj6+d5TnqD=YA+EVOk4SsHLLTarzqGA@mail.gmail.com>
References: <CALHrtFnPXpxCN0JG0g8eEOnut6TRt8Xr4=E55veg7Eq+dhX2Jw@mail.gmail.com>
 <CANjChEp7bowBc6Fm6gBuRFVn9EwqUho+rz_fgJcadu9TpqaAOg@mail.gmail.com>
 <CANjChEoCsif0A2X+ve=qj6+d5TnqD=YA+EVOk4SsHLLTarzqGA@mail.gmail.com>
Message-ID: <CALHrtF=3Nb3dEhVonZOqnP_W=wK8kLH1=ZAwdWN69JhqXQt0MA@mail.gmail.com>

Yes, and specifically, the lifted ByteArray type as described in the
comments, not the unlifted ByteArray# type.

On Fri, Oct 11, 2019 at 4:00 AM Csaba Hruska <csaba.hruska at gmail.com> wrote:
>
> Sorry, maybe I got it wrong. Are you asking if is there any package that passes ByteArrays via FFI?
>
> On Thu, Oct 10, 2019 at 8:54 PM Csaba Hruska <csaba.hruska at gmail.com> wrote:
>>
>> It's a primitive type.
>> https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/prelude/primops.txt.pp#L1388
>> https://gitlab.haskell.org/ghc/ghc/wikis/commentary/prim-ops
>>
>> Cheers,
>> Csaba
>>
>> On Thu, Oct 10, 2019 at 8:16 PM Shao, Cheng <cheng.shao at tweag.io> wrote:
>>>
>>> Hello devs,
>>>
>>> I've been trying to figure out how to pass lifted types as foreign
>>> types, then encountered the following code in the `DsCCall` module
>>> (https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/deSugar/DsCCall.hs#L172):
>>>
>>> ```
>>>   -- Byte-arrays, both mutable and otherwise; hack warning
>>>   -- We're looking for values of type ByteArray, MutableByteArray
>>>   --    data ByteArray          ix = ByteArray        ix ix ByteArray#
>>>   --    data MutableByteArray s ix = MutableByteArray ix ix
>>> (MutableByteArray# s)
>>>   | is_product_type &&
>>>     data_con_arity == 3 &&
>>>     isJust maybe_arg3_tycon &&
>>>     (arg3_tycon ==  byteArrayPrimTyCon ||
>>>      arg3_tycon ==  mutableByteArrayPrimTyCon)
>>>   = do case_bndr <- newSysLocalDs arg_ty
>>>        vars@[_l_var, _r_var, arr_cts_var] <- newSysLocalsDs data_con_arg_tys
>>>        return (Var arr_cts_var,
>>>                \ body -> Case arg case_bndr (exprType body) [(DataAlt
>>> data_con,vars,body)]
>>>               )
>>> ```
>>>
>>> It seems we allow a "ByteArray" type as a foreign import argument, if
>>> the third field of the datacon is a ByteArray# or MutableByteArray#.
>>> But I can't find such a ByteArray type definition in today's common
>>> packages. What's the rationale for this piece of code?
>>>
>>> Cheers,
>>> Cheng
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From chessai1996 at gmail.com  Thu Oct 10 21:28:11 2019
From: chessai1996 at gmail.com (chessai .)
Date: Thu, 10 Oct 2019 17:28:11 -0400
Subject: ByteArray# as a foreign import argument?
In-Reply-To: <CALHrtF=3Nb3dEhVonZOqnP_W=wK8kLH1=ZAwdWN69JhqXQt0MA@mail.gmail.com>
References: <CALHrtFnPXpxCN0JG0g8eEOnut6TRt8Xr4=E55veg7Eq+dhX2Jw@mail.gmail.com>
 <CANjChEp7bowBc6Fm6gBuRFVn9EwqUho+rz_fgJcadu9TpqaAOg@mail.gmail.com>
 <CANjChEoCsif0A2X+ve=qj6+d5TnqD=YA+EVOk4SsHLLTarzqGA@mail.gmail.com>
 <CALHrtF=3Nb3dEhVonZOqnP_W=wK8kLH1=ZAwdWN69JhqXQt0MA@mail.gmail.com>
Message-ID: <CAD34_kJdcqq5y3MO5Bxo6hoCo+MXyFQ85dBSwEzd8GHxwTsyAA@mail.gmail.com>

Can you do that? I know with the UnliftedFFI you can use ByteArray# and if
you have something like uint8_t*, it will make sure that the pointer lines
up with the payload of the ByteArray#. You could always just wrap it in
Data.Primitive.ByteArray.ByteArray, no? That is to say, I don't think GHC
can do what you're asking.

On Thu, Oct 10, 2019, 4:10 PM Shao, Cheng <cheng.shao at tweag.io> wrote:

> Yes, and specifically, the lifted ByteArray type as described in the
> comments, not the unlifted ByteArray# type.
>
> On Fri, Oct 11, 2019 at 4:00 AM Csaba Hruska <csaba.hruska at gmail.com>
> wrote:
> >
> > Sorry, maybe I got it wrong. Are you asking if is there any package that
> passes ByteArrays via FFI?
> >
> > On Thu, Oct 10, 2019 at 8:54 PM Csaba Hruska <csaba.hruska at gmail.com>
> wrote:
> >>
> >> It's a primitive type.
> >>
> https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/prelude/primops.txt.pp#L1388
> >> https://gitlab.haskell.org/ghc/ghc/wikis/commentary/prim-ops
> >>
> >> Cheers,
> >> Csaba
> >>
> >> On Thu, Oct 10, 2019 at 8:16 PM Shao, Cheng <cheng.shao at tweag.io>
> wrote:
> >>>
> >>> Hello devs,
> >>>
> >>> I've been trying to figure out how to pass lifted types as foreign
> >>> types, then encountered the following code in the `DsCCall` module
> >>> (
> https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/deSugar/DsCCall.hs#L172
> ):
> >>>
> >>> ```
> >>>   -- Byte-arrays, both mutable and otherwise; hack warning
> >>>   -- We're looking for values of type ByteArray, MutableByteArray
> >>>   --    data ByteArray          ix = ByteArray        ix ix ByteArray#
> >>>   --    data MutableByteArray s ix = MutableByteArray ix ix
> >>> (MutableByteArray# s)
> >>>   | is_product_type &&
> >>>     data_con_arity == 3 &&
> >>>     isJust maybe_arg3_tycon &&
> >>>     (arg3_tycon ==  byteArrayPrimTyCon ||
> >>>      arg3_tycon ==  mutableByteArrayPrimTyCon)
> >>>   = do case_bndr <- newSysLocalDs arg_ty
> >>>        vars@[_l_var, _r_var, arr_cts_var] <- newSysLocalsDs
> data_con_arg_tys
> >>>        return (Var arr_cts_var,
> >>>                \ body -> Case arg case_bndr (exprType body) [(DataAlt
> >>> data_con,vars,body)]
> >>>               )
> >>> ```
> >>>
> >>> It seems we allow a "ByteArray" type as a foreign import argument, if
> >>> the third field of the datacon is a ByteArray# or MutableByteArray#.
> >>> But I can't find such a ByteArray type definition in today's common
> >>> packages. What's the rationale for this piece of code?
> >>>
> >>> Cheers,
> >>> Cheng
> >>> _______________________________________________
> >>> ghc-devs mailing list
> >>> ghc-devs at haskell.org
> >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191010/d3418bc9/attachment.html>

From chessai1996 at gmail.com  Thu Oct 10 21:31:59 2019
From: chessai1996 at gmail.com (chessai .)
Date: Thu, 10 Oct 2019 17:31:59 -0400
Subject: ByteArray# as a foreign import argument?
In-Reply-To: <CAD34_kJdcqq5y3MO5Bxo6hoCo+MXyFQ85dBSwEzd8GHxwTsyAA@mail.gmail.com>
References: <CALHrtFnPXpxCN0JG0g8eEOnut6TRt8Xr4=E55veg7Eq+dhX2Jw@mail.gmail.com>
 <CANjChEp7bowBc6Fm6gBuRFVn9EwqUho+rz_fgJcadu9TpqaAOg@mail.gmail.com>
 <CANjChEoCsif0A2X+ve=qj6+d5TnqD=YA+EVOk4SsHLLTarzqGA@mail.gmail.com>
 <CALHrtF=3Nb3dEhVonZOqnP_W=wK8kLH1=ZAwdWN69JhqXQt0MA@mail.gmail.com>
 <CAD34_kJdcqq5y3MO5Bxo6hoCo+MXyFQ85dBSwEzd8GHxwTsyAA@mail.gmail.com>
Message-ID: <CAD34_kJ_c7dfK=c73AFk62PGR0cC08fP6rX7GWxYBs06u4U_HA@mail.gmail.com>

I just tried this and got an error message: "Unacceptable argument type in
foreign declaration: 'MutableByteArray s' cannot be marshalled in a foreign
call"

I got a similar error message when switching ByteArray# to ByteArray.

On Thu, Oct 10, 2019, 5:28 PM chessai . <chessai1996 at gmail.com> wrote:

> Can you do that? I know with the UnliftedFFI you can use ByteArray# and if
> you have something like uint8_t*, it will make sure that the pointer lines
> up with the payload of the ByteArray#. You could always just wrap it in
> Data.Primitive.ByteArray.ByteArray, no? That is to say, I don't think GHC
> can do what you're asking.
>
> On Thu, Oct 10, 2019, 4:10 PM Shao, Cheng <cheng.shao at tweag.io> wrote:
>
>> Yes, and specifically, the lifted ByteArray type as described in the
>> comments, not the unlifted ByteArray# type.
>>
>> On Fri, Oct 11, 2019 at 4:00 AM Csaba Hruska <csaba.hruska at gmail.com>
>> wrote:
>> >
>> > Sorry, maybe I got it wrong. Are you asking if is there any package
>> that passes ByteArrays via FFI?
>> >
>> > On Thu, Oct 10, 2019 at 8:54 PM Csaba Hruska <csaba.hruska at gmail.com>
>> wrote:
>> >>
>> >> It's a primitive type.
>> >>
>> https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/prelude/primops.txt.pp#L1388
>> >> https://gitlab.haskell.org/ghc/ghc/wikis/commentary/prim-ops
>> >>
>> >> Cheers,
>> >> Csaba
>> >>
>> >> On Thu, Oct 10, 2019 at 8:16 PM Shao, Cheng <cheng.shao at tweag.io>
>> wrote:
>> >>>
>> >>> Hello devs,
>> >>>
>> >>> I've been trying to figure out how to pass lifted types as foreign
>> >>> types, then encountered the following code in the `DsCCall` module
>> >>> (
>> https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/deSugar/DsCCall.hs#L172
>> ):
>> >>>
>> >>> ```
>> >>>   -- Byte-arrays, both mutable and otherwise; hack warning
>> >>>   -- We're looking for values of type ByteArray, MutableByteArray
>> >>>   --    data ByteArray          ix = ByteArray        ix ix ByteArray#
>> >>>   --    data MutableByteArray s ix = MutableByteArray ix ix
>> >>> (MutableByteArray# s)
>> >>>   | is_product_type &&
>> >>>     data_con_arity == 3 &&
>> >>>     isJust maybe_arg3_tycon &&
>> >>>     (arg3_tycon ==  byteArrayPrimTyCon ||
>> >>>      arg3_tycon ==  mutableByteArrayPrimTyCon)
>> >>>   = do case_bndr <- newSysLocalDs arg_ty
>> >>>        vars@[_l_var, _r_var, arr_cts_var] <- newSysLocalsDs
>> data_con_arg_tys
>> >>>        return (Var arr_cts_var,
>> >>>                \ body -> Case arg case_bndr (exprType body) [(DataAlt
>> >>> data_con,vars,body)]
>> >>>               )
>> >>> ```
>> >>>
>> >>> It seems we allow a "ByteArray" type as a foreign import argument, if
>> >>> the third field of the datacon is a ByteArray# or MutableByteArray#.
>> >>> But I can't find such a ByteArray type definition in today's common
>> >>> packages. What's the rationale for this piece of code?
>> >>>
>> >>> Cheers,
>> >>> Cheng
>> >>> _______________________________________________
>> >>> ghc-devs mailing list
>> >>> ghc-devs at haskell.org
>> >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191010/b2f7489a/attachment.html>

From cheng.shao at tweag.io  Thu Oct 10 21:32:22 2019
From: cheng.shao at tweag.io (Shao, Cheng)
Date: Fri, 11 Oct 2019 05:32:22 +0800
Subject: ByteArray# as a foreign import argument?
In-Reply-To: <CAD34_kJdcqq5y3MO5Bxo6hoCo+MXyFQ85dBSwEzd8GHxwTsyAA@mail.gmail.com>
References: <CALHrtFnPXpxCN0JG0g8eEOnut6TRt8Xr4=E55veg7Eq+dhX2Jw@mail.gmail.com>
 <CANjChEp7bowBc6Fm6gBuRFVn9EwqUho+rz_fgJcadu9TpqaAOg@mail.gmail.com>
 <CANjChEoCsif0A2X+ve=qj6+d5TnqD=YA+EVOk4SsHLLTarzqGA@mail.gmail.com>
 <CALHrtF=3Nb3dEhVonZOqnP_W=wK8kLH1=ZAwdWN69JhqXQt0MA@mail.gmail.com>
 <CAD34_kJdcqq5y3MO5Bxo6hoCo+MXyFQ85dBSwEzd8GHxwTsyAA@mail.gmail.com>
Message-ID: <CALHrtFkncf5KMDc+5EgW=dU4ezP1RwHxKUj8b5WaG7sG9fDyRQ@mail.gmail.com>

I've just manually checked with a `data ByteArray = ByteArray Int Int
ByteArray#` example, indeed GHC doesn't support marshalling it. That
being said, I'm still curious about the meaning of that blob of code
I've just pasted.

On Fri, Oct 11, 2019 at 5:28 AM chessai . <chessai1996 at gmail.com> wrote:
>
> Can you do that? I know with the UnliftedFFI you can use ByteArray# and if you have something like uint8_t*, it will make sure that the pointer lines up with the payload of the ByteArray#. You could always just wrap it in Data.Primitive.ByteArray.ByteArray, no? That is to say, I don't think GHC can do what you're asking.
>
> On Thu, Oct 10, 2019, 4:10 PM Shao, Cheng <cheng.shao at tweag.io> wrote:
>>
>> Yes, and specifically, the lifted ByteArray type as described in the
>> comments, not the unlifted ByteArray# type.
>>
>> On Fri, Oct 11, 2019 at 4:00 AM Csaba Hruska <csaba.hruska at gmail.com> wrote:
>> >
>> > Sorry, maybe I got it wrong. Are you asking if is there any package that passes ByteArrays via FFI?
>> >
>> > On Thu, Oct 10, 2019 at 8:54 PM Csaba Hruska <csaba.hruska at gmail.com> wrote:
>> >>
>> >> It's a primitive type.
>> >> https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/prelude/primops.txt.pp#L1388
>> >> https://gitlab.haskell.org/ghc/ghc/wikis/commentary/prim-ops
>> >>
>> >> Cheers,
>> >> Csaba
>> >>
>> >> On Thu, Oct 10, 2019 at 8:16 PM Shao, Cheng <cheng.shao at tweag.io> wrote:
>> >>>
>> >>> Hello devs,
>> >>>
>> >>> I've been trying to figure out how to pass lifted types as foreign
>> >>> types, then encountered the following code in the `DsCCall` module
>> >>> (https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/deSugar/DsCCall.hs#L172):
>> >>>
>> >>> ```
>> >>>   -- Byte-arrays, both mutable and otherwise; hack warning
>> >>>   -- We're looking for values of type ByteArray, MutableByteArray
>> >>>   --    data ByteArray          ix = ByteArray        ix ix ByteArray#
>> >>>   --    data MutableByteArray s ix = MutableByteArray ix ix
>> >>> (MutableByteArray# s)
>> >>>   | is_product_type &&
>> >>>     data_con_arity == 3 &&
>> >>>     isJust maybe_arg3_tycon &&
>> >>>     (arg3_tycon ==  byteArrayPrimTyCon ||
>> >>>      arg3_tycon ==  mutableByteArrayPrimTyCon)
>> >>>   = do case_bndr <- newSysLocalDs arg_ty
>> >>>        vars@[_l_var, _r_var, arr_cts_var] <- newSysLocalsDs data_con_arg_tys
>> >>>        return (Var arr_cts_var,
>> >>>                \ body -> Case arg case_bndr (exprType body) [(DataAlt
>> >>> data_con,vars,body)]
>> >>>               )
>> >>> ```
>> >>>
>> >>> It seems we allow a "ByteArray" type as a foreign import argument, if
>> >>> the third field of the datacon is a ByteArray# or MutableByteArray#.
>> >>> But I can't find such a ByteArray type definition in today's common
>> >>> packages. What's the rationale for this piece of code?
>> >>>
>> >>> Cheers,
>> >>> Cheng
>> >>> _______________________________________________
>> >>> ghc-devs mailing list
>> >>> ghc-devs at haskell.org
>> >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From sylvain at haskus.fr  Fri Oct 11 08:32:24 2019
From: sylvain at haskus.fr (Sylvain Henry)
Date: Fri, 11 Oct 2019 10:32:24 +0200
Subject: ByteArray# as a foreign import argument?
In-Reply-To: <CALHrtFnPXpxCN0JG0g8eEOnut6TRt8Xr4=E55veg7Eq+dhX2Jw@mail.gmail.com>
References: <CALHrtFnPXpxCN0JG0g8eEOnut6TRt8Xr4=E55veg7Eq+dhX2Jw@mail.gmail.com>
Message-ID: <fa48d7d8-f97a-c78e-5ea7-2ffb3c646602@haskus.fr>

 > But I can't find such a ByteArray type definition in today's common 
packages. What's the rationale for this piece of code?

Doing some archeology they seem to have been removed from 
ghc/lib/std/PrelArr.lhs in e921b2e307532e0f30eefa88b11a124be592bde4 (1999):

  data Ix ix => Array ix elt        = Array            ix ix (Array# elt)
-data Ix ix => ByteArray ix          = ByteArray ix ix ByteArray#
  data Ix ix => MutableArray     s ix elt = MutableArray     ix ix 
(MutableArray# s elt)
-data Ix ix => MutableByteArray s ix     = MutableByteArray ix ix 
(MutableByteArray# s)

So it's probably dead code since then.

Cheers,
Sylvain


On 10/10/2019 21:15, Shao, Cheng wrote:
> Hello devs,
>
> I've been trying to figure out how to pass lifted types as foreign
> types, then encountered the following code in the `DsCCall` module
> (https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/deSugar/DsCCall.hs#L172):
>
> ```
>    -- Byte-arrays, both mutable and otherwise; hack warning
>    -- We're looking for values of type ByteArray, MutableByteArray
>    --    data ByteArray          ix = ByteArray        ix ix ByteArray#
>    --    data MutableByteArray s ix = MutableByteArray ix ix
> (MutableByteArray# s)
>    | is_product_type &&
>      data_con_arity == 3 &&
>      isJust maybe_arg3_tycon &&
>      (arg3_tycon ==  byteArrayPrimTyCon ||
>       arg3_tycon ==  mutableByteArrayPrimTyCon)
>    = do case_bndr <- newSysLocalDs arg_ty
>         vars@[_l_var, _r_var, arr_cts_var] <- newSysLocalsDs data_con_arg_tys
>         return (Var arr_cts_var,
>                 \ body -> Case arg case_bndr (exprType body) [(DataAlt
> data_con,vars,body)]
>                )
> ```
>
> It seems we allow a "ByteArray" type as a foreign import argument, if
> the third field of the datacon is a ByteArray# or MutableByteArray#.
> But I can't find such a ByteArray type definition in today's common
> packages. What's the rationale for this piece of code?
>
> Cheers,
> Cheng
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From sylvain at haskus.fr  Fri Oct 11 08:43:00 2019
From: sylvain at haskus.fr (Sylvain Henry)
Date: Fri, 11 Oct 2019 10:43:00 +0200
Subject: ByteArray# as a foreign import argument?
In-Reply-To: <fa48d7d8-f97a-c78e-5ea7-2ffb3c646602@haskus.fr>
References: <CALHrtFnPXpxCN0JG0g8eEOnut6TRt8Xr4=E55veg7Eq+dhX2Jw@mail.gmail.com>
 <fa48d7d8-f97a-c78e-5ea7-2ffb3c646602@haskus.fr>
Message-ID: <b438ffe0-33d3-ec07-c232-99a6be66ae92@haskus.fr>

Or better 98668305453ea1158c97c8a2c1a90c108aa3585a (2001):

 From the commit message:

     - finally, remove the last vestiges of ByteArray and MutableByteArray
       from the core libraries.  Deprecated implementations will be 
available
       in the lang compatibility package.


On 11/10/2019 10:32, Sylvain Henry wrote:
> > But I can't find such a ByteArray type definition in today's common 
> packages. What's the rationale for this piece of code?
>
> Doing some archeology they seem to have been removed from 
> ghc/lib/std/PrelArr.lhs in e921b2e307532e0f30eefa88b11a124be592bde4 
> (1999):
>
>  data Ix ix => Array ix elt        = Array            ix ix (Array# elt)
> -data Ix ix => ByteArray ix          = ByteArray ix ix ByteArray#
>  data Ix ix => MutableArray     s ix elt = MutableArray     ix ix 
> (MutableArray# s elt)
> -data Ix ix => MutableByteArray s ix     = MutableByteArray ix ix 
> (MutableByteArray# s)
>
> So it's probably dead code since then.
>
> Cheers,
> Sylvain
>
>
> On 10/10/2019 21:15, Shao, Cheng wrote:
>> Hello devs,
>>
>> I've been trying to figure out how to pass lifted types as foreign
>> types, then encountered the following code in the `DsCCall` module
>> (https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/deSugar/DsCCall.hs#L172): 
>>
>>
>> ```
>>    -- Byte-arrays, both mutable and otherwise; hack warning
>>    -- We're looking for values of type ByteArray, MutableByteArray
>>    --    data ByteArray          ix = ByteArray        ix ix ByteArray#
>>    --    data MutableByteArray s ix = MutableByteArray ix ix
>> (MutableByteArray# s)
>>    | is_product_type &&
>>      data_con_arity == 3 &&
>>      isJust maybe_arg3_tycon &&
>>      (arg3_tycon ==  byteArrayPrimTyCon ||
>>       arg3_tycon ==  mutableByteArrayPrimTyCon)
>>    = do case_bndr <- newSysLocalDs arg_ty
>>         vars@[_l_var, _r_var, arr_cts_var] <- newSysLocalsDs 
>> data_con_arg_tys
>>         return (Var arr_cts_var,
>>                 \ body -> Case arg case_bndr (exprType body) [(DataAlt
>> data_con,vars,body)]
>>                )
>> ```
>>
>> It seems we allow a "ByteArray" type as a foreign import argument, if
>> the third field of the datacon is a ByteArray# or MutableByteArray#.
>> But I can't find such a ByteArray type definition in today's common
>> packages. What's the rationale for this piece of code?
>>
>> Cheers,
>> Cheng
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From simonpj at microsoft.com  Fri Oct 11 15:08:22 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Fri, 11 Oct 2019 15:08:22 +0000
Subject: atomicModifyMutVar2
Message-ID: <AM6PR83MB027941FDD4F28B4E66E452EDAD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>

David
I'm deeply puzzled atomicModifyMutVar2#.  I have read the proposal<https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0149-atomicModifyMutVar.rst>, and the comments in primops.txt.pp (reproduced below).
Question 1
I think the "real" type of atomicModifyMutVar2 is

atomicModifyMutVar2# :: MutVar# s a

                     -> (a -> (a,b))

                     -> State# s

                     -> (# State# s, a, (a, b) #)
Nowhere is this explicitly stated, but I believe that the intended semantics of a call

case (atomicModifyMutVar2# mv f s) of (# s', x, r #) -> blah
Then, suppose the old value of the MutVar was 'old'

  *   The primop builds a thunk  t = f old
  *   The new value of the mutable variable is (fst t)
  *   The result r is t
  *   The result x is old
Question: is that correct?   We should state it explicitly.
Question 2
Next question: Why does f have to return a pair?  So far as I can tell, it's only so that a client can force it.   The 'b' part never seems to play a useful role.   So we could equally well have had

atomicModifyMutVar2# :: MutVar# s a

                     -> (a -> Box a)

                     -> State# s

                     -> (# State# s, a, Unit a #)
where Unit is defined in Data.Tuple

    data Unit a = Unit a
Now you can force the result of (f old), just as with a pair.  But the 'b' would no longer complicate matters.
Question: is the 'b' in the pair significant?   Or could we use Unit?
Question 3
In the comments below you say "but we don't know about pairs here".   Are you sure?  What stops you importing Data.Tuple into GHC.Prim?   This fancy footwork is one more complication, if it could be avoided.

Thanks
Simon


primop  AtomicModifyMutVar2Op "atomicModifyMutVar2#" GenPrimOp

   MutVar# s a -> (a -> c) -> State# s -> (# State# s, a, c #)

   { Modify the contents of a {\tt MutVar\#}, returning the previous

     contents and the result of applying the given function to the

     previous contents. Note that this isn't strictly

     speaking the correct type for this function; it should really be

     {\tt MutVar\# s a -> (a -> (a,b)) -> State\# s -> (\# State\# s, a, (a, b) \#)},

     but we don't know about pairs here. }

   with

   out_of_line = True

   has_side_effects = True

   can_fail         = True
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191011/27e1d96b/attachment.html>

From david.feuer at gmail.com  Fri Oct 11 16:03:26 2019
From: david.feuer at gmail.com (David Feuer)
Date: Fri, 11 Oct 2019 12:03:26 -0400
Subject: atomicModifyMutVar2
In-Reply-To: <AM6PR83MB027941FDD4F28B4E66E452EDAD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <AM6PR83MB027941FDD4F28B4E66E452EDAD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <CAMgWh9ueifTZmSW=rf9s3-S0-iw_Rn4+RKzRB69q00oZU_1etg@mail.gmail.com>

On Fri, Oct 11, 2019, 11:08 AM Simon Peyton Jones <simonpj at microsoft.com>
wrote:

> David
>
> I’m deeply puzzled atomicModifyMutVar2#.  I have read the proposal
> <https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0149-atomicModifyMutVar.rst>,
> and the comments in primops.txt.pp (reproduced below).
> Question 1
>
> I think the “real” type of atomicModifyMutVar2 is
>
> atomicModifyMutVar2# :: MutVar# s a
>
>                      -> (a -> (a,b))
>
>                      -> State# s
>
>                      -> (# State# s, a, (a, b) #)
>

Close, but not quite. The result doesn't have to be a pair. It can be a
tuple of any size at all. Indeed, it can even be an arbitrary record type
whose first pointer field has the appropriate type.

Nowhere is this explicitly stated, but I believe that the intended
> semantics of a call
>
> case (atomicModifyMutVar2# mv f s) of (# s’, x, r #) -> blah
>
> Then, suppose the old value of the MutVar was ‘*old’*
>
>    - The primop builds a thunk  *t *= *f old*
>    - The new value of the mutable variable is *(fst t)*
>    - The result *r* is t
>    - The result *x* is *old*
>
> Question: is that correct?   We should state it explicitly.
>
Yes, that sounds right.

> Question 2
>
> Next question: Why does f have to return a pair?  So far as I can tell,
> it’s only so that a client can force it.   The ‘b’ part never seems to play
> a useful role.   So we could equally well have had
>
> atomicModifyMutVar2# :: MutVar# s a
>
>                      -> (a -> Box a)
>
>                      -> State# s
>
>                      -> (# State# s, a, Unit a #)
>
> where Unit is defined in Data.Tuple
>
>     data Unit a = Unit a
>
> Now you can force the result of (f old), just as with a pair.  But the ‘b’
> would no longer complicate matters.
>
> Question: is the ‘b’ in the pair significant?   Or could we use Unit?
>
Yes, it's somewhat significant. You actually can use Unit with the new
primop (it's a tuple of arity 1), so that option is free. But using a pair
gets you a bit more: you can build a thunk that's *shared* between the
value installed in the MutVar and the one returned to the outside. Consider

atomicModifyMutVar2# mv $ \a ->

  let foo = expensive_computation a

  in ([3,foo], foo)

> Question 3
>
> In the comments below you say "but we don't know about pairs here”.   Are
> you sure?  What stops you importing Data.Tuple into GHC.Prim?   This fancy
> footwork is one more complication, if it could be avoided.
>
That whole regime came before my time, but since we win a bit by *not*
fixing it, o wouldn't jump on it too quick.

>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191011/b54069d0/attachment.html>

From simonpj at microsoft.com  Fri Oct 11 16:56:20 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Fri, 11 Oct 2019 16:56:20 +0000
Subject: atomicModifyMutVar2
In-Reply-To: <CAMgWh9ueifTZmSW=rf9s3-S0-iw_Rn4+RKzRB69q00oZU_1etg@mail.gmail.com>
References: <AM6PR83MB027941FDD4F28B4E66E452EDAD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9ueifTZmSW=rf9s3-S0-iw_Rn4+RKzRB69q00oZU_1etg@mail.gmail.com>
Message-ID: <AM6PR83MB0279B1C6C105C77C3EC98476AD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>

The result doesn't have to be a pair. It can be a tuple of any size at all. Indeed, it can even be an arbitrary record type whose first pointer field has the appropriate type.

I think that is 100.0% undocumented, in the code, or in the proposal.  Are you sure this is a settled consensus among the interested parties?

Adopting it would impose new invariants on the representation of values in GHC that I am deeply reluctant to impose.  I would much much prefer to stick with the pair that is (somewhat) documented.

About pair vs Unit, yes, I can see (just) your point about why a pair might be useful.  Here’s a better example:

Suppose mv :: MutVar# Int


atomicModifyMutVar2# mv $ \a ->

  let foo = f a

  in (g foo, foo)

Now, if f is expensive, and g is not invertible, then sharing foo might be useful.  It’s hard to think of a credible example, though.  Regardless, we should document it.

Simon

From: David Feuer <david.feuer at gmail.com>
Sent: 11 October 2019 17:03
To: Simon Peyton Jones <simonpj at microsoft.com>
Cc: ghc-devs <ghc-devs at haskell.org>
Subject: Re: atomicModifyMutVar2

On Fri, Oct 11, 2019, 11:08 AM Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
David
I’m deeply puzzled atomicModifyMutVar2#.  I have read the proposal<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fghc-proposals%2Fghc-proposals%2Fblob%2Fmaster%2Fproposals%2F0149-atomicModifyMutVar.rst&data=02%7C01%7Csimonpj%40microsoft.com%7C10950014fe9d46a6682508d74e649537%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637064066214164213&sdata=2phccb9r%2Fcg4xs7beKByn%2F1Ivxz7kvN5xkejhOpDfg8%3D&reserved=0>, and the comments in primops.txt.pp (reproduced below).
Question 1
I think the “real” type of atomicModifyMutVar2 is

atomicModifyMutVar2# :: MutVar# s a

                     -> (a -> (a,b))

                     -> State# s

                     -> (# State# s, a, (a, b) #)

Close, but not quite. The result doesn't have to be a pair. It can be a tuple of any size at all. Indeed, it can even be an arbitrary record type whose first pointer field has the appropriate type.

Nowhere is this explicitly stated, but I believe that the intended semantics of a call

case (atomicModifyMutVar2# mv f s) of (# s’, x, r #) -> blah
Then, suppose the old value of the MutVar was ‘old’

  *   The primop builds a thunk  t = f old
  *   The new value of the mutable variable is (fst t)
  *   The result r is t
  *   The result x is old
Question: is that correct?   We should state it explicitly.
Yes, that sounds right.
Question 2
Next question: Why does f have to return a pair?  So far as I can tell, it’s only so that a client can force it.   The ‘b’ part never seems to play a useful role.   So we could equally well have had

atomicModifyMutVar2# :: MutVar# s a

                     -> (a -> Box a)

                     -> State# s

                     -> (# State# s, a, Unit a #)
where Unit is defined in Data.Tuple

    data Unit a = Unit a
Now you can force the result of (f old), just as with a pair.  But the ‘b’ would no longer complicate matters.
Question: is the ‘b’ in the pair significant?   Or could we use Unit?
Yes, it's somewhat significant. You actually can use Unit with the new primop (it's a tuple of arity 1), so that option is free. But using a pair gets you a bit more: you can build a thunk that's *shared* between the value installed in the MutVar and the one returned to the outside. Consider


atomicModifyMutVar2# mv $ \a ->

  let foo = expensive_computation a

  in ([3,foo], foo)

Question 3
In the comments below you say "but we don't know about pairs here”.   Are you sure?  What stops you importing Data.Tuple into GHC.Prim?   This fancy footwork is one more complication, if it could be avoided.
That whole regime came before my time, but since we win a bit by *not* fixing it, o wouldn't jump on it too quick.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191011/3feb5468/attachment.html>

From carter.schonwald at gmail.com  Fri Oct 11 21:02:44 2019
From: carter.schonwald at gmail.com (Carter Schonwald)
Date: Fri, 11 Oct 2019 17:02:44 -0400
Subject: atomicModifyMutVar2
In-Reply-To: <AM6PR83MB0279B1C6C105C77C3EC98476AD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <AM6PR83MB027941FDD4F28B4E66E452EDAD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9ueifTZmSW=rf9s3-S0-iw_Rn4+RKzRB69q00oZU_1etg@mail.gmail.com>
 <AM6PR83MB0279B1C6C105C77C3EC98476AD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <CAHYVw0xsaHdqj35S-2qJ+=z9vZ16Ed8BU7CHg0uPNokgqPO8BA@mail.gmail.com>

that additional representation invariant requirement is why i ultimately
chose not to include the user space version of davids work in primitive

https://github.com/haskell/primitive/pull/194

theres some really clever ideas, but i couldnt tease apart a natural
example of where i'd want to use it

On Fri, Oct 11, 2019 at 12:56 PM Simon Peyton Jones via ghc-devs <
ghc-devs at haskell.org> wrote:

> The result doesn't have to be a pair. It can be a tuple of any size at
> all. Indeed, it can even be an arbitrary record type whose first pointer
> field has the appropriate type.
>
>
>
> I think that is 100.0% undocumented, in the code, or in the proposal.  Are
> you sure this is a settled consensus among the interested parties?
>
>
>
> Adopting it would impose new invariants on the representation of values in
> GHC that I am deeply reluctant to impose.  I would much much prefer to
> stick with the pair that is (somewhat) documented.
>
>
>
> About pair vs Unit, yes, I can see (just) your point about why a pair
> might be useful.  Here’s a better example:
>
>
>
> Suppose mv :: MutVar# Int
>
>
>
> atomicModifyMutVar2# mv $ \a ->
>
>   let foo = f a
>
>   in (g foo, foo)
>
>
>
> Now, if f is expensive, *and g is not invertible*, then sharing foo might
> be useful.  It’s hard to think of a credible example, though.  Regardless,
> we should document it.
>
>
>
> Simon
>
>
>
> *From:* David Feuer <david.feuer at gmail.com>
> *Sent:* 11 October 2019 17:03
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Cc:* ghc-devs <ghc-devs at haskell.org>
> *Subject:* Re: atomicModifyMutVar2
>
>
>
> On Fri, Oct 11, 2019, 11:08 AM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> David
>
> I’m deeply puzzled atomicModifyMutVar2#.  I have read the proposal
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fghc-proposals%2Fghc-proposals%2Fblob%2Fmaster%2Fproposals%2F0149-atomicModifyMutVar.rst&data=02%7C01%7Csimonpj%40microsoft.com%7C10950014fe9d46a6682508d74e649537%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637064066214164213&sdata=2phccb9r%2Fcg4xs7beKByn%2F1Ivxz7kvN5xkejhOpDfg8%3D&reserved=0>,
> and the comments in primops.txt.pp (reproduced below).
> Question 1
>
> I think the “real” type of atomicModifyMutVar2 is
>
> atomicModifyMutVar2# :: MutVar# s a
>
>                      -> (a -> (a,b))
>
>                      -> State# s
>
>                      -> (# State# s, a, (a, b) #)
>
>
>
> Close, but not quite. The result doesn't have to be a pair. It can be a
> tuple of any size at all. Indeed, it can even be an arbitrary record type
> whose first pointer field has the appropriate type.
>
>
>
> Nowhere is this explicitly stated, but I believe that the intended
> semantics of a call
>
> case (atomicModifyMutVar2# mv f s) of (# s’, x, r #) -> blah
>
> Then, suppose the old value of the MutVar was ‘*old’*
>
>    - The primop builds a thunk  *t *= *f old*
>    - The new value of the mutable variable is *(fst t)*
>    - The result *r* is t
>    - The result *x* is *old*
>
> Question: is that correct?   We should state it explicitly.
>
> Yes, that sounds right.
>
> Question 2
>
> Next question: Why does f have to return a pair?  So far as I can tell,
> it’s only so that a client can force it.   The ‘b’ part never seems to play
> a useful role.   So we could equally well have had
>
> atomicModifyMutVar2# :: MutVar# s a
>
>                      -> (a -> Box a)
>
>                      -> State# s
>
>                      -> (# State# s, a, Unit a #)
>
> where Unit is defined in Data.Tuple
>
>     data Unit a = Unit a
>
> Now you can force the result of (f old), just as with a pair.  But the ‘b’
> would no longer complicate matters.
>
> Question: is the ‘b’ in the pair significant?   Or could we use Unit?
>
> Yes, it's somewhat significant. You actually can use Unit with the new
> primop (it's a tuple of arity 1), so that option is free. But using a pair
> gets you a bit more: you can build a thunk that's *shared* between the
> value installed in the MutVar and the one returned to the outside. Consider
>
>
>
> atomicModifyMutVar2# mv $ \a ->
>
>   let foo = expensive_computation a
>
>   in ([3,foo], foo)
>
> Question 3
>
> In the comments below you say "but we don't know about pairs here”.   Are
> you sure?  What stops you importing Data.Tuple into GHC.Prim?   This fancy
> footwork is one more complication, if it could be avoided.
>
> That whole regime came before my time, but since we win a bit by *not*
> fixing it, o wouldn't jump on it too quick.
>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191011/539921ae/attachment.html>

From simonpj at microsoft.com  Fri Oct 11 22:44:15 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Fri, 11 Oct 2019 22:44:15 +0000
Subject: atomicModifyMutVar2
In-Reply-To: <AM6PR83MB0279B1C6C105C77C3EC98476AD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <AM6PR83MB027941FDD4F28B4E66E452EDAD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9ueifTZmSW=rf9s3-S0-iw_Rn4+RKzRB69q00oZU_1etg@mail.gmail.com>
 <AM6PR83MB0279B1C6C105C77C3EC98476AD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <AM6PR83MB02793E23F42EC3D438158F1AAD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>

David,

Wait!  It gets worse!

Question 4

As I understand it, the idea in the proposal is that you can force the pair that comes back from the primop, and that helps you cure a space leak.  Thus


case atomicModifyMutVar2# mv f s of

   (# s’, old, pr #) -> pr `seq`  (# s’, () #)


But it’s extremely easy to write calls that complete defeat such a strategy.  Your examples and mine below both have this property. Suppose f is

(\x. let v = expensive x in (v,v))
Well, forcing that pair will do nothing at all!  It certainly won’t force v!   You should probably write

(\x. let v = expensive x in v `seq` (v,v))
or something like that.

Is this right?   At least this should be documented super-clearly.

Simon

From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Simon Peyton Jones via ghc-devs
Sent: 11 October 2019 17:56
To: David Feuer <david.feuer at gmail.com>
Cc: ghc-devs <ghc-devs at haskell.org>
Subject: RE: atomicModifyMutVar2

The result doesn't have to be a pair. It can be a tuple of any size at all. Indeed, it can even be an arbitrary record type whose first pointer field has the appropriate type.

I think that is 100.0% undocumented, in the code, or in the proposal.  Are you sure this is a settled consensus among the interested parties?

Adopting it would impose new invariants on the representation of values in GHC that I am deeply reluctant to impose.  I would much much prefer to stick with the pair that is (somewhat) documented.

About pair vs Unit, yes, I can see (just) your point about why a pair might be useful.  Here’s a better example:

Suppose mv :: MutVar# Int


atomicModifyMutVar2# mv $ \a ->

  let foo = f a

  in (g foo, foo)

Now, if f is expensive, and g is not invertible, then sharing foo might be useful.  It’s hard to think of a credible example, though.  Regardless, we should document it.

Simon

From: David Feuer <david.feuer at gmail.com<mailto:david.feuer at gmail.com>>
Sent: 11 October 2019 17:03
To: Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
Cc: ghc-devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: Re: atomicModifyMutVar2

On Fri, Oct 11, 2019, 11:08 AM Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
David
I’m deeply puzzled atomicModifyMutVar2#.  I have read the proposal<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fghc-proposals%2Fghc-proposals%2Fblob%2Fmaster%2Fproposals%2F0149-atomicModifyMutVar.rst&data=02%7C01%7Csimonpj%40microsoft.com%7C6f3e8178c339489f52ab08d74e6bfbf6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637064098002911775&sdata=Acc5vYlorYg%2Bct6Vq2sddv%2B3Q%2BAuPU6yrOKDQLNDZv8%3D&reserved=0>, and the comments in primops.txt.pp (reproduced below).
Question 1
I think the “real” type of atomicModifyMutVar2 is

atomicModifyMutVar2# :: MutVar# s a

                     -> (a -> (a,b))

                     -> State# s

                     -> (# State# s, a, (a, b) #)

Close, but not quite. The result doesn't have to be a pair. It can be a tuple of any size at all. Indeed, it can even be an arbitrary record type whose first pointer field has the appropriate type.

Nowhere is this explicitly stated, but I believe that the intended semantics of a call

case (atomicModifyMutVar2# mv f s) of (# s’, x, r #) -> blah
Then, suppose the old value of the MutVar was ‘old’

  *   The primop builds a thunk  t = f old
  *   The new value of the mutable variable is (fst t)
  *   The result r is t
  *   The result x is old
Question: is that correct?   We should state it explicitly.
Yes, that sounds right.
Question 2
Next question: Why does f have to return a pair?  So far as I can tell, it’s only so that a client can force it.   The ‘b’ part never seems to play a useful role.   So we could equally well have had

atomicModifyMutVar2# :: MutVar# s a

                     -> (a -> Box a)

                     -> State# s

                     -> (# State# s, a, Unit a #)
where Unit is defined in Data.Tuple

    data Unit a = Unit a
Now you can force the result of (f old), just as with a pair.  But the ‘b’ would no longer complicate matters.
Question: is the ‘b’ in the pair significant?   Or could we use Unit?
Yes, it's somewhat significant. You actually can use Unit with the new primop (it's a tuple of arity 1), so that option is free. But using a pair gets you a bit more: you can build a thunk that's *shared* between the value installed in the MutVar and the one returned to the outside. Consider


atomicModifyMutVar2# mv $ \a ->

  let foo = expensive_computation a

  in ([3,foo], foo)

Question 3
In the comments below you say "but we don't know about pairs here”.   Are you sure?  What stops you importing Data.Tuple into GHC.Prim?   This fancy footwork is one more complication, if it could be avoided.
That whole regime came before my time, but since we win a bit by *not* fixing it, o wouldn't jump on it too quick.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191011/5c403214/attachment-0001.html>

From david.feuer at gmail.com  Fri Oct 11 22:59:50 2019
From: david.feuer at gmail.com (David Feuer)
Date: Fri, 11 Oct 2019 18:59:50 -0400
Subject: atomicModifyMutVar2
In-Reply-To: <AM6PR83MB0279B1C6C105C77C3EC98476AD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <AM6PR83MB027941FDD4F28B4E66E452EDAD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9ueifTZmSW=rf9s3-S0-iw_Rn4+RKzRB69q00oZU_1etg@mail.gmail.com>
 <AM6PR83MB0279B1C6C105C77C3EC98476AD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <CAMgWh9tReq55DzSiccW1w3i7pyLAx9htrEpRphO3V0tdk_8yvQ@mail.gmail.com>

I don't remember what documentation, if any, it has. You're right that
taking advantage of it is potentially risky. Here's what I think we really
want:

atomicModifyMutVarQ# :: MutVar# s a -> (q -> a) -> (a -> q)  -> State# s ->
(# State# s, a, q #)

where there's a special rule that (q -> a) is "obviously" a selector that
selects a pointer.

On Fri, Oct 11, 2019, 12:56 PM Simon Peyton Jones <simonpj at microsoft.com>
wrote:

> The result doesn't have to be a pair. It can be a tuple of any size at
> all. Indeed, it can even be an arbitrary record type whose first pointer
> field has the appropriate type.
>
>
>
> I think that is 100.0% undocumented, in the code, or in the proposal.  Are
> you sure this is a settled consensus among the interested parties?
>
>
>
> Adopting it would impose new invariants on the representation of values in
> GHC that I am deeply reluctant to impose.  I would much much prefer to
> stick with the pair that is (somewhat) documented.
>
>
>
> About pair vs Unit, yes, I can see (just) your point about why a pair
> might be useful.  Here’s a better example:
>
>
>
> Suppose mv :: MutVar# Int
>
>
>
> atomicModifyMutVar2# mv $ \a ->
>
>   let foo = f a
>
>   in (g foo, foo)
>
>
>
> Now, if f is expensive, *and g is not invertible*, then sharing foo might
> be useful.  It’s hard to think of a credible example, though.  Regardless,
> we should document it.
>
>
>
> Simon
>
>
>
> *From:* David Feuer <david.feuer at gmail.com>
> *Sent:* 11 October 2019 17:03
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Cc:* ghc-devs <ghc-devs at haskell.org>
> *Subject:* Re: atomicModifyMutVar2
>
>
>
> On Fri, Oct 11, 2019, 11:08 AM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> David
>
> I’m deeply puzzled atomicModifyMutVar2#.  I have read the proposal
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fghc-proposals%2Fghc-proposals%2Fblob%2Fmaster%2Fproposals%2F0149-atomicModifyMutVar.rst&data=02%7C01%7Csimonpj%40microsoft.com%7C10950014fe9d46a6682508d74e649537%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637064066214164213&sdata=2phccb9r%2Fcg4xs7beKByn%2F1Ivxz7kvN5xkejhOpDfg8%3D&reserved=0>,
> and the comments in primops.txt.pp (reproduced below).
> Question 1
>
> I think the “real” type of atomicModifyMutVar2 is
>
> atomicModifyMutVar2# :: MutVar# s a
>
>                      -> (a -> (a,b))
>
>                      -> State# s
>
>                      -> (# State# s, a, (a, b) #)
>
>
>
> Close, but not quite. The result doesn't have to be a pair. It can be a
> tuple of any size at all. Indeed, it can even be an arbitrary record type
> whose first pointer field has the appropriate type.
>
>
>
> Nowhere is this explicitly stated, but I believe that the intended
> semantics of a call
>
> case (atomicModifyMutVar2# mv f s) of (# s’, x, r #) -> blah
>
> Then, suppose the old value of the MutVar was ‘*old’*
>
>    - The primop builds a thunk  *t *= *f old*
>    - The new value of the mutable variable is *(fst t)*
>    - The result *r* is t
>    - The result *x* is *old*
>
> Question: is that correct?   We should state it explicitly.
>
> Yes, that sounds right.
>
> Question 2
>
> Next question: Why does f have to return a pair?  So far as I can tell,
> it’s only so that a client can force it.   The ‘b’ part never seems to play
> a useful role.   So we could equally well have had
>
> atomicModifyMutVar2# :: MutVar# s a
>
>                      -> (a -> Box a)
>
>                      -> State# s
>
>                      -> (# State# s, a, Unit a #)
>
> where Unit is defined in Data.Tuple
>
>     data Unit a = Unit a
>
> Now you can force the result of (f old), just as with a pair.  But the ‘b’
> would no longer complicate matters.
>
> Question: is the ‘b’ in the pair significant?   Or could we use Unit?
>
> Yes, it's somewhat significant. You actually can use Unit with the new
> primop (it's a tuple of arity 1), so that option is free. But using a pair
> gets you a bit more: you can build a thunk that's *shared* between the
> value installed in the MutVar and the one returned to the outside. Consider
>
>
>
> atomicModifyMutVar2# mv $ \a ->
>
>   let foo = expensive_computation a
>
>   in ([3,foo], foo)
>
> Question 3
>
> In the comments below you say "but we don't know about pairs here”.   Are
> you sure?  What stops you importing Data.Tuple into GHC.Prim?   This fancy
> footwork is one more complication, if it could be avoided.
>
> That whole regime came before my time, but since we win a bit by *not*
> fixing it, o wouldn't jump on it too quick.
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191011/298a606a/attachment.html>

From david.feuer at gmail.com  Fri Oct 11 23:42:00 2019
From: david.feuer at gmail.com (David Feuer)
Date: Fri, 11 Oct 2019 19:42:00 -0400
Subject: atomicModifyMutVar2
In-Reply-To: <CAMgWh9tReq55DzSiccW1w3i7pyLAx9htrEpRphO3V0tdk_8yvQ@mail.gmail.com>
References: <AM6PR83MB027941FDD4F28B4E66E452EDAD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9ueifTZmSW=rf9s3-S0-iw_Rn4+RKzRB69q00oZU_1etg@mail.gmail.com>
 <AM6PR83MB0279B1C6C105C77C3EC98476AD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9tReq55DzSiccW1w3i7pyLAx9htrEpRphO3V0tdk_8yvQ@mail.gmail.com>
Message-ID: <CAMgWh9u1nvL5PKZ7i+n_t2m=okv0uGkXOXnxrNUnoxhrECk7dw@mail.gmail.com>

Actually, maybe we can do better! We don't inherently need the function to
be a selector. But to make it more general, we'll need to take some care to
make sure to produce good code when it *is* a selector.

On Fri, Oct 11, 2019, 6:59 PM David Feuer <david.feuer at gmail.com> wrote:

> I don't remember what documentation, if any, it has. You're right that
> taking advantage of it is potentially risky. Here's what I think we really
> want:
>
> atomicModifyMutVarQ# :: MutVar# s a -> (q -> a) -> (a -> q)  -> State# s ->
> (# State# s, a, q #)
>
> where there's a special rule that (q -> a) is "obviously" a selector that
> selects a pointer.
>
> On Fri, Oct 11, 2019, 12:56 PM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
>> The result doesn't have to be a pair. It can be a tuple of any size at
>> all. Indeed, it can even be an arbitrary record type whose first pointer
>> field has the appropriate type.
>>
>>
>>
>> I think that is 100.0% undocumented, in the code, or in the proposal.
>> Are you sure this is a settled consensus among the interested parties?
>>
>>
>>
>> Adopting it would impose new invariants on the representation of values
>> in GHC that I am deeply reluctant to impose.  I would much much prefer to
>> stick with the pair that is (somewhat) documented.
>>
>>
>>
>> About pair vs Unit, yes, I can see (just) your point about why a pair
>> might be useful.  Here’s a better example:
>>
>>
>>
>> Suppose mv :: MutVar# Int
>>
>>
>>
>> atomicModifyMutVar2# mv $ \a ->
>>
>>   let foo = f a
>>
>>   in (g foo, foo)
>>
>>
>>
>> Now, if f is expensive, *and g is not invertible*, then sharing foo
>> might be useful.  It’s hard to think of a credible example, though.
>> Regardless, we should document it.
>>
>>
>>
>> Simon
>>
>>
>>
>> *From:* David Feuer <david.feuer at gmail.com>
>> *Sent:* 11 October 2019 17:03
>> *To:* Simon Peyton Jones <simonpj at microsoft.com>
>> *Cc:* ghc-devs <ghc-devs at haskell.org>
>> *Subject:* Re: atomicModifyMutVar2
>>
>>
>>
>> On Fri, Oct 11, 2019, 11:08 AM Simon Peyton Jones <simonpj at microsoft.com>
>> wrote:
>>
>> David
>>
>> I’m deeply puzzled atomicModifyMutVar2#.  I have read the proposal
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fghc-proposals%2Fghc-proposals%2Fblob%2Fmaster%2Fproposals%2F0149-atomicModifyMutVar.rst&data=02%7C01%7Csimonpj%40microsoft.com%7C10950014fe9d46a6682508d74e649537%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637064066214164213&sdata=2phccb9r%2Fcg4xs7beKByn%2F1Ivxz7kvN5xkejhOpDfg8%3D&reserved=0>,
>> and the comments in primops.txt.pp (reproduced below).
>> Question 1
>>
>> I think the “real” type of atomicModifyMutVar2 is
>>
>> atomicModifyMutVar2# :: MutVar# s a
>>
>>                      -> (a -> (a,b))
>>
>>                      -> State# s
>>
>>                      -> (# State# s, a, (a, b) #)
>>
>>
>>
>> Close, but not quite. The result doesn't have to be a pair. It can be a
>> tuple of any size at all. Indeed, it can even be an arbitrary record type
>> whose first pointer field has the appropriate type.
>>
>>
>>
>> Nowhere is this explicitly stated, but I believe that the intended
>> semantics of a call
>>
>> case (atomicModifyMutVar2# mv f s) of (# s’, x, r #) -> blah
>>
>> Then, suppose the old value of the MutVar was ‘*old’*
>>
>>    - The primop builds a thunk  *t *= *f old*
>>    - The new value of the mutable variable is *(fst t)*
>>    - The result *r* is t
>>    - The result *x* is *old*
>>
>> Question: is that correct?   We should state it explicitly.
>>
>> Yes, that sounds right.
>>
>> Question 2
>>
>> Next question: Why does f have to return a pair?  So far as I can tell,
>> it’s only so that a client can force it.   The ‘b’ part never seems to play
>> a useful role.   So we could equally well have had
>>
>> atomicModifyMutVar2# :: MutVar# s a
>>
>>                      -> (a -> Box a)
>>
>>                      -> State# s
>>
>>                      -> (# State# s, a, Unit a #)
>>
>> where Unit is defined in Data.Tuple
>>
>>     data Unit a = Unit a
>>
>> Now you can force the result of (f old), just as with a pair.  But the
>> ‘b’ would no longer complicate matters.
>>
>> Question: is the ‘b’ in the pair significant?   Or could we use Unit?
>>
>> Yes, it's somewhat significant. You actually can use Unit with the new
>> primop (it's a tuple of arity 1), so that option is free. But using a pair
>> gets you a bit more: you can build a thunk that's *shared* between the
>> value installed in the MutVar and the one returned to the outside. Consider
>>
>>
>>
>> atomicModifyMutVar2# mv $ \a ->
>>
>>   let foo = expensive_computation a
>>
>>   in ([3,foo], foo)
>>
>> Question 3
>>
>> In the comments below you say "but we don't know about pairs here”.   Are
>> you sure?  What stops you importing Data.Tuple into GHC.Prim?   This fancy
>> footwork is one more complication, if it could be avoided.
>>
>> That whole regime came before my time, but since we win a bit by *not*
>> fixing it, o wouldn't jump on it too quick.
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191011/3d1270ea/attachment.html>

From rsx at bluewin.ch  Sun Oct 13 15:07:46 2019
From: rsx at bluewin.ch (Roland Senn)
Date: Sun, 13 Oct 2019 17:07:46 +0200
Subject: CI fails in git submodule update --init --recursive for 386-linux-deb9
Message-ID: <1570979266.9333.4.camel@bluewin.ch>

Hi 

After pushing some commits, while running a CI pipeline, I persistently
get the following error for the validate-i386-linux-deb9 step:

$ git submodule update --init --recursive
fatal: Unable to create
'/builds/RolandSenn/ghc/.git/modules/libraries/Cabal/index.lock': File
exists.

Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.
Unable to checkout '63331c95ed15cc7e3d83850d308dc3a86a8c3c76' in
submodule path 'libraries/Cabal'

See https://gitlab.haskell.org/RolandSenn/ghc/pipelines/11311 .

How can I fix this? I have no access to this 386 linux machine, so I'm
unable to delete the file.

Roland
  

From ben at smart-cactus.org  Sun Oct 13 17:38:50 2019
From: ben at smart-cactus.org (Ben Gamari)
Date: Sun, 13 Oct 2019 13:38:50 -0400
Subject: validate-x86_64-darwin failing
In-Reply-To: <928A805D-866A-45CE-9CD1-D494D5D63183@richarde.dev>
References: <35FB4B62-1C19-4EF5-94C4-8DECCD981B8C@richarde.dev>
 <878spujej1.fsf@smart-cactus.org>
 <928A805D-866A-45CE-9CD1-D494D5D63183@richarde.dev>
Message-ID: <87zhi4ilzu.fsf@smart-cactus.org>

Richard Eisenberg <rae at richarde.dev> writes:

> Here's one: https://gitlab.haskell.org/rae/ghc/-/jobs/173464
> <https://gitlab.haskell.org/rae/ghc/-/jobs/173464> I don't know if you
> can get more information...
>
Right, this particular job simply timed out (e.g. waited 10 hours)
before a Darwin builder became available. It's not an indication of a
failure, merely that we are underprovisioned on the Darwin front.

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191013/3ffe6310/attachment.sig>

From rae at richarde.dev  Mon Oct 14 09:55:39 2019
From: rae at richarde.dev (Richard Eisenberg)
Date: Mon, 14 Oct 2019 10:55:39 +0100
Subject: validate-x86_64-darwin failing
In-Reply-To: <87zhi4ilzu.fsf@smart-cactus.org>
References: <35FB4B62-1C19-4EF5-94C4-8DECCD981B8C@richarde.dev>
 <878spujej1.fsf@smart-cactus.org>
 <928A805D-866A-45CE-9CD1-D494D5D63183@richarde.dev>
 <87zhi4ilzu.fsf@smart-cactus.org>
Message-ID: <88A1B0F3-FA3B-47F0-9EAD-18A76682E9E5@richarde.dev>



> On Oct 13, 2019, at 6:38 PM, Ben Gamari <ben at smart-cactus.org> wrote:
> 
> Richard Eisenberg <rae at richarde.dev> writes:
> 
>> Here's one: https://gitlab.haskell.org/rae/ghc/-/jobs/173464
>> <https://gitlab.haskell.org/rae/ghc/-/jobs/173464> I don't know if you
>> can get more information...
>> 
> Right, this particular job simply timed out (e.g. waited 10 hours)
> before a Darwin builder became available. It's not an indication of a
> failure, merely that we are underprovisioned on the Darwin front.

That was my guess. But how often does this happen? Does this failure mean we should make the darwin build somehow optional? Increase the timeout? Any fragility in our CI infrastructure is damaging: it slows the acceptance of otherwise-ready patches and contributes to a "broken windows" effect encouraging developers to ignore CI failures. Conversely, how expensive would it be to improve our darwin provisioning?

Thanks,
Richard

> 
> Cheers,
> 
> - Ben


From allbery.b at gmail.com  Mon Oct 14 12:40:42 2019
From: allbery.b at gmail.com (Brandon Allbery)
Date: Mon, 14 Oct 2019 08:40:42 -0400
Subject: validate-x86_64-darwin failing
In-Reply-To: <88A1B0F3-FA3B-47F0-9EAD-18A76682E9E5@richarde.dev>
References: <35FB4B62-1C19-4EF5-94C4-8DECCD981B8C@richarde.dev>
 <878spujej1.fsf@smart-cactus.org>
 <928A805D-866A-45CE-9CD1-D494D5D63183@richarde.dev>
 <87zhi4ilzu.fsf@smart-cactus.org>
 <88A1B0F3-FA3B-47F0-9EAD-18A76682E9E5@richarde.dev>
Message-ID: <CAKFCL4WdEjHMu0Ns49H5WdgkCROQSWrN--q8hp8xiy8QqwJ7gQ@mail.gmail.com>

Apple makes that annoyingly difficult; someone has to in effect donate a
Mac to the cause, preferably one with enough memory and fast CPU. (Not
literally: one can keep the Mac physically, but would more or less lose use
of it for any other purpose.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191014/a5cde8df/attachment.html>

From carter.schonwald at gmail.com  Mon Oct 14 14:45:06 2019
From: carter.schonwald at gmail.com (Carter Schonwald)
Date: Mon, 14 Oct 2019 10:45:06 -0400
Subject: validate-x86_64-darwin failing
In-Reply-To: <CAKFCL4WdEjHMu0Ns49H5WdgkCROQSWrN--q8hp8xiy8QqwJ7gQ@mail.gmail.com>
References: <35FB4B62-1C19-4EF5-94C4-8DECCD981B8C@richarde.dev>
 <878spujej1.fsf@smart-cactus.org>
 <928A805D-866A-45CE-9CD1-D494D5D63183@richarde.dev>
 <87zhi4ilzu.fsf@smart-cactus.org>
 <88A1B0F3-FA3B-47F0-9EAD-18A76682E9E5@richarde.dev>
 <CAKFCL4WdEjHMu0Ns49H5WdgkCROQSWrN--q8hp8xiy8QqwJ7gQ@mail.gmail.com>
Message-ID: <CAHYVw0zbZitQszRpUvpgtTfMVyodqveq9hqdk+vcDx=MnJPQwg@mail.gmail.com>

We are in slow process of reprovisioning 1-2 new builders.  One of our
active builders had pretty bad hardware issues so for near term we’re down
in mac builder capacity

On Mon, Oct 14, 2019 at 8:41 AM Brandon Allbery <allbery.b at gmail.com> wrote:

> Apple makes that annoyingly difficult; someone has to in effect donate a
> Mac to the cause, preferably one with enough memory and fast CPU. (Not
> literally: one can keep the Mac physically, but would more or less lose use
> of it for any other purpose.)
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191014/31454166/attachment.html>

From chessai1996 at gmail.com  Mon Oct 14 15:05:59 2019
From: chessai1996 at gmail.com (chessai .)
Date: Mon, 14 Oct 2019 11:05:59 -0400
Subject: Switch/Case optimisations
Message-ID: <CAD34_kJyxQYxCF67p+utSbkOnENax6cWf7j=+5TNieLrqhA_Gg@mail.gmail.com>

I thought Andreas Klebinger in particular might find this interesting.

The talk is here: https://youtu.be/IAdLwUXRUvg?t=1867

(timestamp is 31:07)

basically, if you have an interpreter consisting of a switch statement
inside an infinite loop, there's a mechanical transformation using GOTOs
that causes a 15% speedup
in the switch statement blob, there's n + 1 basic blocks, each of which has
a jmp back to the first basic block with probability 1, and the first basic
block has a jmp that will go to one of the other n basic blocks with
probability 1 / n
whereas in the goto statement setup, there's n basic blocks, each of which
has a jmp that will go to one of the other n - 1 basic blocks with
probability 1 / (n - 1)
so if your sequence of instructions is produced by a markov decision
process, then the branch predictor will efficiently predict in the latter
situation but not necessarily the former
in the goto setup the callgraph looks like a complete graph on n vertices
whereas in the switch setup it looks like a bipartite graph on (1, n)
vertices (where the edges represent both "is called by" and "calls")
and because of all those extra edges (n^2 rather than 2n) there's more
spots for the branch predictor to build up data

At 39:13 he explains that you get per-address jmp predictions. If you only
share a single relative jmp, the cpu only knows how common every opcode is.
But, if you have a jmp from each opcode, to the next, the cpu can determine
how common an opcode A is followed an opcode B.

He references godbolt (https://godbolt.org/), which seems like a useful
tool. and mentions that ruby, jvm (on android only?), and some arm
compilers implement this.

There are some tradeoffs - code size does increase. According to a GCC
developer in the audience, GCC is not currently able to do this. I have no
idea about Clang.

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191014/b09a53e5/attachment.html>

From omeragacan at gmail.com  Mon Oct 14 19:31:12 2019
From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=)
Date: Mon, 14 Oct 2019 22:31:12 +0300
Subject: Help with debugging a batch mode (or type checker) bug
Message-ID: <CAMQQO3nEdp6-1KequuPvyvqYrb-+M2a_Gr20ciCpHM4M9emQJA@mail.gmail.com>

Hi,

In !1304 I'm currently having a bug where I get correct IdInfos for imported Ids
in one-shot mode (-c), but not when I use batch mode (--make). If I add a few
prints I can see that in hscIncrementalCompile right before
hscIncrementalFrontend the hsc_HPT has the correct IdInfos for the imported Ids,
but somehow in the desugarer output the IdInfo is lost. Can anyone point me to
the right places in the typechecker or the desugarer for me to look at?

Thanks,

Ömer

From simonpj at microsoft.com  Mon Oct 14 21:35:07 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 14 Oct 2019 21:35:07 +0000
Subject: Help with debugging a batch mode (or type checker) bug
In-Reply-To: <CAMQQO3nEdp6-1KequuPvyvqYrb-+M2a_Gr20ciCpHM4M9emQJA@mail.gmail.com>
References: <CAMQQO3nEdp6-1KequuPvyvqYrb-+M2a_Gr20ciCpHM4M9emQJA@mail.gmail.com>
Message-ID: <AM6PR83MB027971669E2F04DE458E1388AD900@AM6PR83MB0279.EURPRD83.prod.outlook.com>

In --make mode, after compiling module M, rather than serialising M.hi and the deserialising and typechecking it, we simply use the type environment generated by compiling M and put it into M's entry of the Home Package Table (HPT).

But we have to take care to ensure that the IdInfo in that type-env matches the IdInfo that would be obtained by reading M.hi.  This is the tidy_type_env returned by TidyPgm.tidyPgm.

Now, in your cafInfo work you are working out the CAFFy-ness of Ids later, and popping this info into M's ModIface.  But of course, in --make mode, you must _also_ put that same info into M's md_types type envt in the ModDetails returned by tidyPgm.

I bet that omitting that step is the problem.

Simon

| -----Original Message-----
| From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Ömer Sinan
| Agacan
| Sent: 14 October 2019 20:31
| To: ghc-devs <ghc-devs at haskell.org>
| Subject: Help with debugging a batch mode (or type checker) bug
| 
| Hi,
| 
| In !1304 I'm currently having a bug where I get correct IdInfos for
| imported Ids in one-shot mode (-c), but not when I use batch mode (--
| make). If I add a few prints I can see that in hscIncrementalCompile
| right before hscIncrementalFrontend the hsc_HPT has the correct IdInfos
| for the imported Ids, but somehow in the desugarer output the IdInfo is
| lost. Can anyone point me to the right places in the typechecker or the
| desugarer for me to look at?
| 
| Thanks,
| 
| Ömer
| _______________________________________________
| ghc-devs mailing list
| ghc-devs at haskell.org
| https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.has
| kell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
| devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C82e14cca4ab94038a05208d
| 750dd31e1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637066783278881153
| &amp;sdata=grXCcgzashiOYGOLafr%2F32Q%2BYr5X1tonSavQ5pVlx0M%3D&amp;reserve
| d=0

From simonpj at microsoft.com  Mon Oct 14 22:35:24 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 14 Oct 2019 22:35:24 +0000
Subject: atomicModifyMutVar2
In-Reply-To: <CAMgWh9tReq55DzSiccW1w3i7pyLAx9htrEpRphO3V0tdk_8yvQ@mail.gmail.com>
References: <AM6PR83MB027941FDD4F28B4E66E452EDAD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9ueifTZmSW=rf9s3-S0-iw_Rn4+RKzRB69q00oZU_1etg@mail.gmail.com>
 <AM6PR83MB0279B1C6C105C77C3EC98476AD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9tReq55DzSiccW1w3i7pyLAx9htrEpRphO3V0tdk_8yvQ@mail.gmail.com>
Message-ID: <AM6PR83MB0279925C28BF51C096DDB917AD900@AM6PR83MB0279.EURPRD83.prod.outlook.com>

OK.   I propose:


  *   To give atomicModieyMutVarOf# its proper type, with a  pair, as in the proposal.
  *   To do that I’ll fiddle with genprimopcode, to allow it to parse tuples as well as unboxed tuples; not hard.
  *   This would disallow all this stuff about “any type that has a first field looking like a”, restricting to pairs alone.  This didn’t form part of the proposal, and was never documented.
  *   Add a bit more clarity to the documentation, so it’d clear what must be forced.

Any objections?

Simon

From: David Feuer <david.feuer at gmail.com>
Sent: 12 October 2019 00:00
To: Simon Peyton Jones <simonpj at microsoft.com>
Cc: ghc-devs <ghc-devs at haskell.org>
Subject: Re: atomicModifyMutVar2

I don't remember what documentation, if any, it has. You're right that taking advantage of it is potentially risky. Here's what I think we really want:

atomicModifyMutVarQ# :: MutVar# s a -> (q -> a) -> (a -> q)  -> State# s -> (# State# s, a, q #)

where there's a special rule that (q -> a) is "obviously" a selector that selects a pointer.
On Fri, Oct 11, 2019, 12:56 PM Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
The result doesn't have to be a pair. It can be a tuple of any size at all. Indeed, it can even be an arbitrary record type whose first pointer field has the appropriate type.

I think that is 100.0% undocumented, in the code, or in the proposal.  Are you sure this is a settled consensus among the interested parties?

Adopting it would impose new invariants on the representation of values in GHC that I am deeply reluctant to impose.  I would much much prefer to stick with the pair that is (somewhat) documented.

About pair vs Unit, yes, I can see (just) your point about why a pair might be useful.  Here’s a better example:

Suppose mv :: MutVar# Int


atomicModifyMutVar2# mv $ \a ->

  let foo = f a

  in (g foo, foo)

Now, if f is expensive, and g is not invertible, then sharing foo might be useful.  It’s hard to think of a credible example, though.  Regardless, we should document it.

Simon

From: David Feuer <david.feuer at gmail.com<mailto:david.feuer at gmail.com>>
Sent: 11 October 2019 17:03
To: Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
Cc: ghc-devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: Re: atomicModifyMutVar2

On Fri, Oct 11, 2019, 11:08 AM Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
David
I’m deeply puzzled atomicModifyMutVar2#.  I have read the proposal<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fghc-proposals%2Fghc-proposals%2Fblob%2Fmaster%2Fproposals%2F0149-atomicModifyMutVar.rst&data=02%7C01%7Csimonpj%40microsoft.com%7C0b9a79c810d7453ce16008d74e9ebfc1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637064316038353079&sdata=LGyYiZC9b9g3lZpzTdvQgyEwKCaEsIzFDBcacVE2N5M%3D&reserved=0>, and the comments in primops.txt.pp (reproduced below).
Question 1
I think the “real” type of atomicModifyMutVar2 is

atomicModifyMutVar2# :: MutVar# s a

                     -> (a -> (a,b))

                     -> State# s

                     -> (# State# s, a, (a, b) #)

Close, but not quite. The result doesn't have to be a pair. It can be a tuple of any size at all. Indeed, it can even be an arbitrary record type whose first pointer field has the appropriate type.

Nowhere is this explicitly stated, but I believe that the intended semantics of a call

case (atomicModifyMutVar2# mv f s) of (# s’, x, r #) -> blah
Then, suppose the old value of the MutVar was ‘old’

  *   The primop builds a thunk  t = f old
  *   The new value of the mutable variable is (fst t)
  *   The result r is t
  *   The result x is old
Question: is that correct?   We should state it explicitly.
Yes, that sounds right.
Question 2
Next question: Why does f have to return a pair?  So far as I can tell, it’s only so that a client can force it.   The ‘b’ part never seems to play a useful role.   So we could equally well have had

atomicModifyMutVar2# :: MutVar# s a

                     -> (a -> Box a)

                     -> State# s

                     -> (# State# s, a, Unit a #)
where Unit is defined in Data.Tuple

    data Unit a = Unit a
Now you can force the result of (f old), just as with a pair.  But the ‘b’ would no longer complicate matters.
Question: is the ‘b’ in the pair significant?   Or could we use Unit?
Yes, it's somewhat significant. You actually can use Unit with the new primop (it's a tuple of arity 1), so that option is free. But using a pair gets you a bit more: you can build a thunk that's *shared* between the value installed in the MutVar and the one returned to the outside. Consider


atomicModifyMutVar2# mv $ \a ->

  let foo = expensive_computation a

  in ([3,foo], foo)

Question 3
In the comments below you say "but we don't know about pairs here”.   Are you sure?  What stops you importing Data.Tuple into GHC.Prim?   This fancy footwork is one more complication, if it could be avoided.
That whole regime came before my time, but since we win a bit by *not* fixing it, o wouldn't jump on it too quick.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191014/d2443226/attachment.html>

From omeragacan at gmail.com  Tue Oct 15 07:04:45 2019
From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=)
Date: Tue, 15 Oct 2019 10:04:45 +0300
Subject: Help with debugging a batch mode (or type checker) bug
In-Reply-To: <AM6PR83MB027971669E2F04DE458E1388AD900@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <CAMQQO3nEdp6-1KequuPvyvqYrb-+M2a_Gr20ciCpHM4M9emQJA@mail.gmail.com>
 <AM6PR83MB027971669E2F04DE458E1388AD900@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <CAMQQO3k1h5ah29JEUjw3fnQ2ckmLyV3U2uV6tnZFHTLs6GeQNQ@mail.gmail.com>

Thanks Simon,

I can't confirm just yet, because it's impossible to print the Id details in the
TypeEnv (probably because of some know-tying somewhere?), but looking at the
code I can't see where we update ModDetails returned by hscIncrementalCompile in
compileOne' (used by batch mode but not by one-shot mode), so I think you're
right about this being the problem.

Ömer

Simon Peyton Jones <simonpj at microsoft.com>, 15 Eki 2019 Sal, 00:35
tarihinde şunu yazdı:
>
> In --make mode, after compiling module M, rather than serialising M.hi and the deserialising and typechecking it, we simply use the type environment generated by compiling M and put it into M's entry of the Home Package Table (HPT).
>
> But we have to take care to ensure that the IdInfo in that type-env matches the IdInfo that would be obtained by reading M.hi.  This is the tidy_type_env returned by TidyPgm.tidyPgm.
>
> Now, in your cafInfo work you are working out the CAFFy-ness of Ids later, and popping this info into M's ModIface.  But of course, in --make mode, you must _also_ put that same info into M's md_types type envt in the ModDetails returned by tidyPgm.
>
> I bet that omitting that step is the problem.
>
> Simon
>
> | -----Original Message-----
> | From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Ömer Sinan
> | Agacan
> | Sent: 14 October 2019 20:31
> | To: ghc-devs <ghc-devs at haskell.org>
> | Subject: Help with debugging a batch mode (or type checker) bug
> |
> | Hi,
> |
> | In !1304 I'm currently having a bug where I get correct IdInfos for
> | imported Ids in one-shot mode (-c), but not when I use batch mode (--
> | make). If I add a few prints I can see that in hscIncrementalCompile
> | right before hscIncrementalFrontend the hsc_HPT has the correct IdInfos
> | for the imported Ids, but somehow in the desugarer output the IdInfo is
> | lost. Can anyone point me to the right places in the typechecker or the
> | desugarer for me to look at?
> |
> | Thanks,
> |
> | Ömer
> | _______________________________________________
> | ghc-devs mailing list
> | ghc-devs at haskell.org
> | https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.has
> | kell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
> | devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C82e14cca4ab94038a05208d
> | 750dd31e1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637066783278881153
> | &amp;sdata=grXCcgzashiOYGOLafr%2F32Q%2BYr5X1tonSavQ5pVlx0M%3D&amp;reserve
> | d=0

From omeragacan at gmail.com  Tue Oct 15 07:30:58 2019
From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=)
Date: Tue, 15 Oct 2019 10:30:58 +0300
Subject: Help with debugging a batch mode (or type checker) bug
In-Reply-To: <CAMQQO3k1h5ah29JEUjw3fnQ2ckmLyV3U2uV6tnZFHTLs6GeQNQ@mail.gmail.com>
References: <CAMQQO3nEdp6-1KequuPvyvqYrb-+M2a_Gr20ciCpHM4M9emQJA@mail.gmail.com>
 <AM6PR83MB027971669E2F04DE458E1388AD900@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMQQO3k1h5ah29JEUjw3fnQ2ckmLyV3U2uV6tnZFHTLs6GeQNQ@mail.gmail.com>
Message-ID: <CAMQQO3kUX27uEpxXUdQwaBj+y3vQ26D0covVj2W+TR42OZwfVg@mail.gmail.com>

Just confirmed that this is the problem -- thanks again.

Ömer


Ömer Sinan Ağacan <omeragacan at gmail.com>, 15 Eki 2019 Sal, 10:04
tarihinde şunu yazdı:
>
> Thanks Simon,
>
> I can't confirm just yet, because it's impossible to print the Id details in the
> TypeEnv (probably because of some know-tying somewhere?), but looking at the
> code I can't see where we update ModDetails returned by hscIncrementalCompile in
> compileOne' (used by batch mode but not by one-shot mode), so I think you're
> right about this being the problem.
>
> Ömer
>
> Simon Peyton Jones <simonpj at microsoft.com>, 15 Eki 2019 Sal, 00:35
> tarihinde şunu yazdı:
> >
> > In --make mode, after compiling module M, rather than serialising M.hi and the deserialising and typechecking it, we simply use the type environment generated by compiling M and put it into M's entry of the Home Package Table (HPT).
> >
> > But we have to take care to ensure that the IdInfo in that type-env matches the IdInfo that would be obtained by reading M.hi.  This is the tidy_type_env returned by TidyPgm.tidyPgm.
> >
> > Now, in your cafInfo work you are working out the CAFFy-ness of Ids later, and popping this info into M's ModIface.  But of course, in --make mode, you must _also_ put that same info into M's md_types type envt in the ModDetails returned by tidyPgm.
> >
> > I bet that omitting that step is the problem.
> >
> > Simon
> >
> > | -----Original Message-----
> > | From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Ömer Sinan
> > | Agacan
> > | Sent: 14 October 2019 20:31
> > | To: ghc-devs <ghc-devs at haskell.org>
> > | Subject: Help with debugging a batch mode (or type checker) bug
> > |
> > | Hi,
> > |
> > | In !1304 I'm currently having a bug where I get correct IdInfos for
> > | imported Ids in one-shot mode (-c), but not when I use batch mode (--
> > | make). If I add a few prints I can see that in hscIncrementalCompile
> > | right before hscIncrementalFrontend the hsc_HPT has the correct IdInfos
> > | for the imported Ids, but somehow in the desugarer output the IdInfo is
> > | lost. Can anyone point me to the right places in the typechecker or the
> > | desugarer for me to look at?
> > |
> > | Thanks,
> > |
> > | Ömer
> > | _______________________________________________
> > | ghc-devs mailing list
> > | ghc-devs at haskell.org
> > | https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.has
> > | kell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
> > | devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C82e14cca4ab94038a05208d
> > | 750dd31e1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637066783278881153
> > | &amp;sdata=grXCcgzashiOYGOLafr%2F32Q%2BYr5X1tonSavQ5pVlx0M%3D&amp;reserve
> > | d=0

From simonpj at microsoft.com  Tue Oct 15 07:38:58 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Tue, 15 Oct 2019 07:38:58 +0000
Subject: Help with debugging a batch mode (or type checker) bug
In-Reply-To: <CAMQQO3kUX27uEpxXUdQwaBj+y3vQ26D0covVj2W+TR42OZwfVg@mail.gmail.com>
References: <CAMQQO3nEdp6-1KequuPvyvqYrb-+M2a_Gr20ciCpHM4M9emQJA@mail.gmail.com>
 <AM6PR83MB027971669E2F04DE458E1388AD900@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMQQO3k1h5ah29JEUjw3fnQ2ckmLyV3U2uV6tnZFHTLs6GeQNQ@mail.gmail.com>
 <CAMQQO3kUX27uEpxXUdQwaBj+y3vQ26D0covVj2W+TR42OZwfVg@mail.gmail.com>
Message-ID: <AM6PR83MB02797A1C47E1CD5ACAACA7AAAD930@AM6PR83MB0279.EURPRD83.prod.outlook.com>

Great.  Before you are done with this patch, let's make sure there is an overview Note to explain the big picture.

Simon

|  -----Original Message-----
|  From: Ömer Sinan Ağacan <omeragacan at gmail.com>
|  Sent: 15 October 2019 08:31
|  To: Simon Peyton Jones <simonpj at microsoft.com>
|  Cc: ghc-devs <ghc-devs at haskell.org>
|  Subject: Re: Help with debugging a batch mode (or type checker) bug
|  
|  Just confirmed that this is the problem -- thanks again.
|  
|  Ömer
|  
|  
|  Ömer Sinan Ağacan <omeragacan at gmail.com>, 15 Eki 2019 Sal, 10:04
|  tarihinde şunu yazdı:
|  >
|  > Thanks Simon,
|  >
|  > I can't confirm just yet, because it's impossible to print the Id
|  details in the
|  > TypeEnv (probably because of some know-tying somewhere?), but looking at
|  the
|  > code I can't see where we update ModDetails returned by
|  hscIncrementalCompile in
|  > compileOne' (used by batch mode but not by one-shot mode), so I think
|  you're
|  > right about this being the problem.
|  >
|  > Ömer
|  >
|  > Simon Peyton Jones <simonpj at microsoft.com>, 15 Eki 2019 Sal, 00:35
|  > tarihinde şunu yazdı:
|  > >
|  > > In --make mode, after compiling module M, rather than serialising M.hi
|  and the deserialising and typechecking it, we simply use the type
|  environment generated by compiling M and put it into M's entry of the Home
|  Package Table (HPT).
|  > >
|  > > But we have to take care to ensure that the IdInfo in that type-env
|  matches the IdInfo that would be obtained by reading M.hi.  This is the
|  tidy_type_env returned by TidyPgm.tidyPgm.
|  > >
|  > > Now, in your cafInfo work you are working out the CAFFy-ness of Ids
|  later, and popping this info into M's ModIface.  But of course, in --make
|  mode, you must _also_ put that same info into M's md_types type envt in
|  the ModDetails returned by tidyPgm.
|  > >
|  > > I bet that omitting that step is the problem.
|  > >
|  > > Simon
|  > >
|  > > | -----Original Message-----
|  > > | From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Ömer
|  Sinan
|  > > | Agacan
|  > > | Sent: 14 October 2019 20:31
|  > > | To: ghc-devs <ghc-devs at haskell.org>
|  > > | Subject: Help with debugging a batch mode (or type checker) bug
|  > > |
|  > > | Hi,
|  > > |
|  > > | In !1304 I'm currently having a bug where I get correct IdInfos for
|  > > | imported Ids in one-shot mode (-c), but not when I use batch mode (-
|  -
|  > > | make). If I add a few prints I can see that in hscIncrementalCompile
|  > > | right before hscIncrementalFrontend the hsc_HPT has the correct
|  IdInfos
|  > > | for the imported Ids, but somehow in the desugarer output the IdInfo
|  is
|  > > | lost. Can anyone point me to the right places in the typechecker or
|  the
|  > > | desugarer for me to look at?
|  > > |
|  > > | Thanks,
|  > > |
|  > > | Ömer
|  > > | _______________________________________________
|  > > | ghc-devs mailing list
|  > > | ghc-devs at haskell.org
|  > > |
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.has
|  > > | kell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  > > |
|  devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C82e14cca4ab94038a05208d
|  > > |
|  750dd31e1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637066783278881153
|  > > |
|  &amp;sdata=grXCcgzashiOYGOLafr%2F32Q%2BYr5X1tonSavQ5pVlx0M%3D&amp;reserve
|  > > | d=0

From ben at well-typed.com  Tue Oct 15 16:59:12 2019
From: ben at well-typed.com (Ben Gamari)
Date: Tue, 15 Oct 2019 12:59:12 -0400
Subject: [Reminder] Merge window for 8.10 closing
Message-ID: <87imoqhrmt.fsf@smart-cactus.org>

Hello everyone,

As described on the GHC status page [1] the freeze for GHC 8.10 is coming up
shortly. If you have work that you would like to get into 8.10 and
haven't yet opened an MR you should either do so today or be in touch
with me.

After this freeze begins there will be approximately a week of reduced
merge rate to let the master branch settle before the branch, which
could happen as early as early next week.

Cheers,

- Ben


[1] https://gitlab.haskell.org/ghc/ghc/-/milestones/54?title=8.10.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191015/8b298507/attachment.sig>

From matthewtpickering at gmail.com  Wed Oct 16 10:11:20 2019
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Wed, 16 Oct 2019 11:11:20 +0100
Subject: [Reminder] Merge window for 8.10 closing
In-Reply-To: <87imoqhrmt.fsf@smart-cactus.org>
References: <87imoqhrmt.fsf@smart-cactus.org>
Message-ID: <CALuQ0m_+f_8CzRz9mVcwtT0ccQYJg5eCeLmUh8XX8CHJh5JEwA@mail.gmail.com>

I have two patches which need to be merged.

1. ghc-debug patch - currently blocked on concurrent GC getting
merged, is that going to happen?
https://gitlab.haskell.org/ghc/ghc/merge_requests/1435
2. Time profiling eventlog events (finished & unblocked) -
https://gitlab.haskell.org/ghc/ghc/merge_requests/1927

Cheers,

Matt

On Tue, Oct 15, 2019 at 5:59 PM Ben Gamari <ben at well-typed.com> wrote:
>
> Hello everyone,
>
> As described on the GHC status page [1] the freeze for GHC 8.10 is coming up
> shortly. If you have work that you would like to get into 8.10 and
> haven't yet opened an MR you should either do so today or be in touch
> with me.
>
> After this freeze begins there will be approximately a week of reduced
> merge rate to let the master branch settle before the branch, which
> could happen as early as early next week.
>
> Cheers,
>
> - Ben
>
>
> [1] https://gitlab.haskell.org/ghc/ghc/-/milestones/54?title=8.10.1
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From ryan.gl.scott at gmail.com  Wed Oct 16 13:07:15 2019
From: ryan.gl.scott at gmail.com (Ryan Scott)
Date: Wed, 16 Oct 2019 09:07:15 -0400
Subject: [Reminder] Merge window for 8.10 closing
Message-ID: <CADdQpRAieUjX7B8zTzgEePu33rEv3Z7XVJt7P+_JpjUfQA8-iA@mail.gmail.com>

I've been working on a fix for #13154 [1] that I'd like to get into GHC
8.10. My patch [2] is mostly done; I just need to find the time to add
comments and perform some slight code cleanup. (Hopefully this week!)

Ryan S.
-----
[1] https://gitlab.haskell.org/ghc/ghc/issues/13154
[2] https://gitlab.haskell.org/RyanGlScott/ghc/tree/wip/T13154-take-two
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191016/983659c8/attachment.html>

From matthewtpickering at gmail.com  Wed Oct 16 14:13:53 2019
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Wed, 16 Oct 2019 15:13:53 +0100
Subject: Invitation to GHC Week
Message-ID: <CALuQ0m-8EzVN=cu=gEZw-5HHPwufc2ZmfMJ6aLXLsd36mWYBRQ@mail.gmail.com>

Dear all,

I am happy to announce GHC Week, a developers meeting focusing on GHC,
organized by myself and AndreasK. We are renting a big holiday cottage
in rural Belgium during March so that participants can hack on GHC
uninterrupted for a whole week. The place has a large number of beds
and a nice open living area for collaboration and relaxation.

Dates: 13th - 20th March 2020
Location: Annevoie, Belgium

If you are interested in attending GHC Week then please consult the
linked webpage which contains information about how to attend. There
are only a limited number of spaces and we already have a number of
confirmed attendees. Hopefully no one will be disappointed but if all
700 subscribers to ghc-devs want to attend there will not be enough
space!

We would particularly like to encourage undergraduate students who are
interested in working on GHC to attend GHC Week. Thanks to our
sponsors, Haskell.org and Well-Typed we are in a position to not only
make the event itself very affordable, but also to cover travel and
accommodation costs for those in need.

This is an exceptional opportunity to learn about GHC from experienced
developers. So if you can, make sure you don’t miss it!

For more details please consult the following page:

https://andreaspk.github.io/ghc-week

Cheers,

Matt & Andreas

From carter.schonwald at gmail.com  Wed Oct 16 17:23:29 2019
From: carter.schonwald at gmail.com (Carter Schonwald)
Date: Wed, 16 Oct 2019 13:23:29 -0400
Subject: atomicModifyMutVar2
In-Reply-To: <AM6PR83MB0279925C28BF51C096DDB917AD900@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <AM6PR83MB027941FDD4F28B4E66E452EDAD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9ueifTZmSW=rf9s3-S0-iw_Rn4+RKzRB69q00oZU_1etg@mail.gmail.com>
 <AM6PR83MB0279B1C6C105C77C3EC98476AD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9tReq55DzSiccW1w3i7pyLAx9htrEpRphO3V0tdk_8yvQ@mail.gmail.com>
 <AM6PR83MB0279925C28BF51C096DDB917AD900@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <CAHYVw0woA53wNGBcxtZGg7WPE2-DH-MUj5BOTMYVJzbG8a1Mpw@mail.gmail.com>

sounds lovely! (the current type is certainly strange and not very sound :)
)

On Mon, Oct 14, 2019 at 6:35 PM Simon Peyton Jones via ghc-devs <
ghc-devs at haskell.org> wrote:

> OK.   I propose:
>
>
>
>    - To give atomicModieyMutVarOf# its proper type, with a  pair, as in
>    the proposal.
>    - To do that I’ll fiddle with genprimopcode, to allow it to parse
>    tuples as well as unboxed tuples; not hard.
>    - This would disallow all this stuff about “any type that has a first
>    field looking like a”, restricting to pairs alone.  This didn’t form part
>    of the proposal, and was never documented.
>    - Add a bit more clarity to the documentation, so it’d clear what must
>    be forced.
>
>
>
> Any objections?
>
>
>
> Simon
>
>
>
> *From:* David Feuer <david.feuer at gmail.com>
> *Sent:* 12 October 2019 00:00
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Cc:* ghc-devs <ghc-devs at haskell.org>
> *Subject:* Re: atomicModifyMutVar2
>
>
>
> I don't remember what documentation, if any, it has. You're right that
> taking advantage of it is potentially risky. Here's what I think we really
> want:
>
>
>
> atomicModifyMutVarQ# :: MutVar# s a -> (q -> a) -> (a -> q)  -> State#
> s -> (# State# s, a, q #)
>
>
>
> where there's a special rule that (q -> a) is "obviously" a selector that
> selects a pointer.
>
> On Fri, Oct 11, 2019, 12:56 PM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> The result doesn't have to be a pair. It can be a tuple of any size at
> all. Indeed, it can even be an arbitrary record type whose first pointer
> field has the appropriate type.
>
>
>
> I think that is 100.0% undocumented, in the code, or in the proposal.  Are
> you sure this is a settled consensus among the interested parties?
>
>
>
> Adopting it would impose new invariants on the representation of values in
> GHC that I am deeply reluctant to impose.  I would much much prefer to
> stick with the pair that is (somewhat) documented.
>
>
>
> About pair vs Unit, yes, I can see (just) your point about why a pair
> might be useful.  Here’s a better example:
>
>
>
> Suppose mv :: MutVar# Int
>
>
>
> atomicModifyMutVar2# mv $ \a ->
>
>   let foo = f a
>
>   in (g foo, foo)
>
>
>
> Now, if f is expensive, *and g is not invertible*, then sharing foo might
> be useful.  It’s hard to think of a credible example, though.  Regardless,
> we should document it.
>
>
>
> Simon
>
>
>
> *From:* David Feuer <david.feuer at gmail.com>
> *Sent:* 11 October 2019 17:03
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Cc:* ghc-devs <ghc-devs at haskell.org>
> *Subject:* Re: atomicModifyMutVar2
>
>
>
> On Fri, Oct 11, 2019, 11:08 AM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> David
>
> I’m deeply puzzled atomicModifyMutVar2#.  I have read the proposal
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fghc-proposals%2Fghc-proposals%2Fblob%2Fmaster%2Fproposals%2F0149-atomicModifyMutVar.rst&data=02%7C01%7Csimonpj%40microsoft.com%7C0b9a79c810d7453ce16008d74e9ebfc1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637064316038353079&sdata=LGyYiZC9b9g3lZpzTdvQgyEwKCaEsIzFDBcacVE2N5M%3D&reserved=0>,
> and the comments in primops.txt.pp (reproduced below).
> Question 1
>
> I think the “real” type of atomicModifyMutVar2 is
>
> atomicModifyMutVar2# :: MutVar# s a
>
>                      -> (a -> (a,b))
>
>                      -> State# s
>
>                      -> (# State# s, a, (a, b) #)
>
>
>
> Close, but not quite. The result doesn't have to be a pair. It can be a
> tuple of any size at all. Indeed, it can even be an arbitrary record type
> whose first pointer field has the appropriate type.
>
>
>
> Nowhere is this explicitly stated, but I believe that the intended
> semantics of a call
>
> case (atomicModifyMutVar2# mv f s) of (# s’, x, r #) -> blah
>
> Then, suppose the old value of the MutVar was ‘*old’*
>
>    - The primop builds a thunk  *t *= *f old*
>    - The new value of the mutable variable is *(fst t)*
>    - The result *r* is t
>    - The result *x* is *old*
>
> Question: is that correct?   We should state it explicitly.
>
> Yes, that sounds right.
>
> Question 2
>
> Next question: Why does f have to return a pair?  So far as I can tell,
> it’s only so that a client can force it.   The ‘b’ part never seems to play
> a useful role.   So we could equally well have had
>
> atomicModifyMutVar2# :: MutVar# s a
>
>                      -> (a -> Box a)
>
>                      -> State# s
>
>                      -> (# State# s, a, Unit a #)
>
> where Unit is defined in Data.Tuple
>
>     data Unit a = Unit a
>
> Now you can force the result of (f old), just as with a pair.  But the ‘b’
> would no longer complicate matters.
>
> Question: is the ‘b’ in the pair significant?   Or could we use Unit?
>
> Yes, it's somewhat significant. You actually can use Unit with the new
> primop (it's a tuple of arity 1), so that option is free. But using a pair
> gets you a bit more: you can build a thunk that's *shared* between the
> value installed in the MutVar and the one returned to the outside. Consider
>
>
>
> atomicModifyMutVar2# mv $ \a ->
>
>   let foo = expensive_computation a
>
>   in ([3,foo], foo)
>
> Question 3
>
> In the comments below you say "but we don't know about pairs here”.   Are
> you sure?  What stops you importing Data.Tuple into GHC.Prim?   This fancy
> footwork is one more complication, if it could be avoided.
>
> That whole regime came before my time, but since we win a bit by *not*
> fixing it, o wouldn't jump on it too quick.
>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191016/b31b9b10/attachment.html>

From david.feuer at gmail.com  Wed Oct 16 17:50:40 2019
From: david.feuer at gmail.com (David Feuer)
Date: Wed, 16 Oct 2019 13:50:40 -0400
Subject: atomicModifyMutVar2
In-Reply-To: <AM6PR83MB0279925C28BF51C096DDB917AD900@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <AM6PR83MB027941FDD4F28B4E66E452EDAD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9ueifTZmSW=rf9s3-S0-iw_Rn4+RKzRB69q00oZU_1etg@mail.gmail.com>
 <AM6PR83MB0279B1C6C105C77C3EC98476AD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9tReq55DzSiccW1w3i7pyLAx9htrEpRphO3V0tdk_8yvQ@mail.gmail.com>
 <AM6PR83MB0279925C28BF51C096DDB917AD900@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <CAMgWh9snu7qA78iF7N-CU0YNgshzHU==Of+k5wtmdtYS=AajhQ@mail.gmail.com>

What do you think about the version that takes a function as an argument,
and is optimized for the case where that's a selector? That really feels
like the platonic ideal here.

On Mon, Oct 14, 2019, 6:35 PM Simon Peyton Jones <simonpj at microsoft.com>
wrote:

> OK.   I propose:
>
>
>
>    - To give atomicModieyMutVarOf# its proper type, with a  pair, as in
>    the proposal.
>    - To do that I’ll fiddle with genprimopcode, to allow it to parse
>    tuples as well as unboxed tuples; not hard.
>    - This would disallow all this stuff about “any type that has a first
>    field looking like a”, restricting to pairs alone.  This didn’t form part
>    of the proposal, and was never documented.
>    - Add a bit more clarity to the documentation, so it’d clear what must
>    be forced.
>
>
>
> Any objections?
>
>
>
> Simon
>
>
>
> *From:* David Feuer <david.feuer at gmail.com>
> *Sent:* 12 October 2019 00:00
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Cc:* ghc-devs <ghc-devs at haskell.org>
> *Subject:* Re: atomicModifyMutVar2
>
>
>
> I don't remember what documentation, if any, it has. You're right that
> taking advantage of it is potentially risky. Here's what I think we really
> want:
>
>
>
> atomicModifyMutVarQ# :: MutVar# s a -> (q -> a) -> (a -> q)  -> State#
> s -> (# State# s, a, q #)
>
>
>
> where there's a special rule that (q -> a) is "obviously" a selector that
> selects a pointer.
>
> On Fri, Oct 11, 2019, 12:56 PM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> The result doesn't have to be a pair. It can be a tuple of any size at
> all. Indeed, it can even be an arbitrary record type whose first pointer
> field has the appropriate type.
>
>
>
> I think that is 100.0% undocumented, in the code, or in the proposal.  Are
> you sure this is a settled consensus among the interested parties?
>
>
>
> Adopting it would impose new invariants on the representation of values in
> GHC that I am deeply reluctant to impose.  I would much much prefer to
> stick with the pair that is (somewhat) documented.
>
>
>
> About pair vs Unit, yes, I can see (just) your point about why a pair
> might be useful.  Here’s a better example:
>
>
>
> Suppose mv :: MutVar# Int
>
>
>
> atomicModifyMutVar2# mv $ \a ->
>
>   let foo = f a
>
>   in (g foo, foo)
>
>
>
> Now, if f is expensive, *and g is not invertible*, then sharing foo might
> be useful.  It’s hard to think of a credible example, though.  Regardless,
> we should document it.
>
>
>
> Simon
>
>
>
> *From:* David Feuer <david.feuer at gmail.com>
> *Sent:* 11 October 2019 17:03
> *To:* Simon Peyton Jones <simonpj at microsoft.com>
> *Cc:* ghc-devs <ghc-devs at haskell.org>
> *Subject:* Re: atomicModifyMutVar2
>
>
>
> On Fri, Oct 11, 2019, 11:08 AM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
> David
>
> I’m deeply puzzled atomicModifyMutVar2#.  I have read the proposal
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fghc-proposals%2Fghc-proposals%2Fblob%2Fmaster%2Fproposals%2F0149-atomicModifyMutVar.rst&data=02%7C01%7Csimonpj%40microsoft.com%7C0b9a79c810d7453ce16008d74e9ebfc1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637064316038353079&sdata=LGyYiZC9b9g3lZpzTdvQgyEwKCaEsIzFDBcacVE2N5M%3D&reserved=0>,
> and the comments in primops.txt.pp (reproduced below).
> Question 1
>
> I think the “real” type of atomicModifyMutVar2 is
>
> atomicModifyMutVar2# :: MutVar# s a
>
>                      -> (a -> (a,b))
>
>                      -> State# s
>
>                      -> (# State# s, a, (a, b) #)
>
>
>
> Close, but not quite. The result doesn't have to be a pair. It can be a
> tuple of any size at all. Indeed, it can even be an arbitrary record type
> whose first pointer field has the appropriate type.
>
>
>
> Nowhere is this explicitly stated, but I believe that the intended
> semantics of a call
>
> case (atomicModifyMutVar2# mv f s) of (# s’, x, r #) -> blah
>
> Then, suppose the old value of the MutVar was ‘*old’*
>
>    - The primop builds a thunk  *t *= *f old*
>    - The new value of the mutable variable is *(fst t)*
>    - The result *r* is t
>    - The result *x* is *old*
>
> Question: is that correct?   We should state it explicitly.
>
> Yes, that sounds right.
>
> Question 2
>
> Next question: Why does f have to return a pair?  So far as I can tell,
> it’s only so that a client can force it.   The ‘b’ part never seems to play
> a useful role.   So we could equally well have had
>
> atomicModifyMutVar2# :: MutVar# s a
>
>                      -> (a -> Box a)
>
>                      -> State# s
>
>                      -> (# State# s, a, Unit a #)
>
> where Unit is defined in Data.Tuple
>
>     data Unit a = Unit a
>
> Now you can force the result of (f old), just as with a pair.  But the ‘b’
> would no longer complicate matters.
>
> Question: is the ‘b’ in the pair significant?   Or could we use Unit?
>
> Yes, it's somewhat significant. You actually can use Unit with the new
> primop (it's a tuple of arity 1), so that option is free. But using a pair
> gets you a bit more: you can build a thunk that's *shared* between the
> value installed in the MutVar and the one returned to the outside. Consider
>
>
>
> atomicModifyMutVar2# mv $ \a ->
>
>   let foo = expensive_computation a
>
>   in ([3,foo], foo)
>
> Question 3
>
> In the comments below you say "but we don't know about pairs here”.   Are
> you sure?  What stops you importing Data.Tuple into GHC.Prim?   This fancy
> footwork is one more complication, if it could be avoided.
>
> That whole regime came before my time, but since we win a bit by *not*
> fixing it, o wouldn't jump on it too quick.
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191016/a4b96ecb/attachment.html>

From carter.schonwald at gmail.com  Thu Oct 17 01:07:34 2019
From: carter.schonwald at gmail.com (Carter Schonwald)
Date: Wed, 16 Oct 2019 21:07:34 -0400
Subject: atomicModifyMutVar2
In-Reply-To: <CAMgWh9snu7qA78iF7N-CU0YNgshzHU==Of+k5wtmdtYS=AajhQ@mail.gmail.com>
References: <AM6PR83MB027941FDD4F28B4E66E452EDAD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9ueifTZmSW=rf9s3-S0-iw_Rn4+RKzRB69q00oZU_1etg@mail.gmail.com>
 <AM6PR83MB0279B1C6C105C77C3EC98476AD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9tReq55DzSiccW1w3i7pyLAx9htrEpRphO3V0tdk_8yvQ@mail.gmail.com>
 <AM6PR83MB0279925C28BF51C096DDB917AD900@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9snu7qA78iF7N-CU0YNgshzHU==Of+k5wtmdtYS=AajhQ@mail.gmail.com>
Message-ID: <CAHYVw0zC3qNRg8PxHMqY2Eg8X6O2Liz1Ud9dDE-S99DestDRsQ@mail.gmail.com>

 how do you type check it / what type would you plausibly given it thats
wired into very innards of ghc type system implementation? (by which i mean
that most / all primops currently are pretty conservative in terms of the
range of ghc type system features they use)


On Wed, Oct 16, 2019 at 1:51 PM David Feuer <david.feuer at gmail.com> wrote:

> What do you think about the version that takes a function as an argument,
> and is optimized for the case where that's a selector? That really feels
> like the platonic ideal here.
>
> On Mon, Oct 14, 2019, 6:35 PM Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
>
>> OK.   I propose:
>>
>>
>>
>>    - To give atomicModieyMutVarOf# its proper type, with a  pair, as in
>>    the proposal.
>>    - To do that I’ll fiddle with genprimopcode, to allow it to parse
>>    tuples as well as unboxed tuples; not hard.
>>    - This would disallow all this stuff about “any type that has a first
>>    field looking like a”, restricting to pairs alone.  This didn’t form part
>>    of the proposal, and was never documented.
>>    - Add a bit more clarity to the documentation, so it’d clear what
>>    must be forced.
>>
>>
>>
>> Any objections?
>>
>>
>>
>> Simon
>>
>>
>>
>> *From:* David Feuer <david.feuer at gmail.com>
>> *Sent:* 12 October 2019 00:00
>> *To:* Simon Peyton Jones <simonpj at microsoft.com>
>> *Cc:* ghc-devs <ghc-devs at haskell.org>
>> *Subject:* Re: atomicModifyMutVar2
>>
>>
>>
>> I don't remember what documentation, if any, it has. You're right that
>> taking advantage of it is potentially risky. Here's what I think we really
>> want:
>>
>>
>>
>> atomicModifyMutVarQ# :: MutVar# s a -> (q -> a) -> (a -> q)  -> State#
>> s -> (# State# s, a, q #)
>>
>>
>>
>> where there's a special rule that (q -> a) is "obviously" a selector that
>> selects a pointer.
>>
>> On Fri, Oct 11, 2019, 12:56 PM Simon Peyton Jones <simonpj at microsoft.com>
>> wrote:
>>
>> The result doesn't have to be a pair. It can be a tuple of any size at
>> all. Indeed, it can even be an arbitrary record type whose first pointer
>> field has the appropriate type.
>>
>>
>>
>> I think that is 100.0% undocumented, in the code, or in the proposal.
>> Are you sure this is a settled consensus among the interested parties?
>>
>>
>>
>> Adopting it would impose new invariants on the representation of values
>> in GHC that I am deeply reluctant to impose.  I would much much prefer to
>> stick with the pair that is (somewhat) documented.
>>
>>
>>
>> About pair vs Unit, yes, I can see (just) your point about why a pair
>> might be useful.  Here’s a better example:
>>
>>
>>
>> Suppose mv :: MutVar# Int
>>
>>
>>
>> atomicModifyMutVar2# mv $ \a ->
>>
>>   let foo = f a
>>
>>   in (g foo, foo)
>>
>>
>>
>> Now, if f is expensive, *and g is not invertible*, then sharing foo
>> might be useful.  It’s hard to think of a credible example, though.
>> Regardless, we should document it.
>>
>>
>>
>> Simon
>>
>>
>>
>> *From:* David Feuer <david.feuer at gmail.com>
>> *Sent:* 11 October 2019 17:03
>> *To:* Simon Peyton Jones <simonpj at microsoft.com>
>> *Cc:* ghc-devs <ghc-devs at haskell.org>
>> *Subject:* Re: atomicModifyMutVar2
>>
>>
>>
>> On Fri, Oct 11, 2019, 11:08 AM Simon Peyton Jones <simonpj at microsoft.com>
>> wrote:
>>
>> David
>>
>> I’m deeply puzzled atomicModifyMutVar2#.  I have read the proposal
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fghc-proposals%2Fghc-proposals%2Fblob%2Fmaster%2Fproposals%2F0149-atomicModifyMutVar.rst&data=02%7C01%7Csimonpj%40microsoft.com%7C0b9a79c810d7453ce16008d74e9ebfc1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637064316038353079&sdata=LGyYiZC9b9g3lZpzTdvQgyEwKCaEsIzFDBcacVE2N5M%3D&reserved=0>,
>> and the comments in primops.txt.pp (reproduced below).
>> Question 1
>>
>> I think the “real” type of atomicModifyMutVar2 is
>>
>> atomicModifyMutVar2# :: MutVar# s a
>>
>>                      -> (a -> (a,b))
>>
>>                      -> State# s
>>
>>                      -> (# State# s, a, (a, b) #)
>>
>>
>>
>> Close, but not quite. The result doesn't have to be a pair. It can be a
>> tuple of any size at all. Indeed, it can even be an arbitrary record type
>> whose first pointer field has the appropriate type.
>>
>>
>>
>> Nowhere is this explicitly stated, but I believe that the intended
>> semantics of a call
>>
>> case (atomicModifyMutVar2# mv f s) of (# s’, x, r #) -> blah
>>
>> Then, suppose the old value of the MutVar was ‘*old’*
>>
>>    - The primop builds a thunk  *t *= *f old*
>>    - The new value of the mutable variable is *(fst t)*
>>    - The result *r* is t
>>    - The result *x* is *old*
>>
>> Question: is that correct?   We should state it explicitly.
>>
>> Yes, that sounds right.
>>
>> Question 2
>>
>> Next question: Why does f have to return a pair?  So far as I can tell,
>> it’s only so that a client can force it.   The ‘b’ part never seems to play
>> a useful role.   So we could equally well have had
>>
>> atomicModifyMutVar2# :: MutVar# s a
>>
>>                      -> (a -> Box a)
>>
>>                      -> State# s
>>
>>                      -> (# State# s, a, Unit a #)
>>
>> where Unit is defined in Data.Tuple
>>
>>     data Unit a = Unit a
>>
>> Now you can force the result of (f old), just as with a pair.  But the
>> ‘b’ would no longer complicate matters.
>>
>> Question: is the ‘b’ in the pair significant?   Or could we use Unit?
>>
>> Yes, it's somewhat significant. You actually can use Unit with the new
>> primop (it's a tuple of arity 1), so that option is free. But using a pair
>> gets you a bit more: you can build a thunk that's *shared* between the
>> value installed in the MutVar and the one returned to the outside. Consider
>>
>>
>>
>> atomicModifyMutVar2# mv $ \a ->
>>
>>   let foo = expensive_computation a
>>
>>   in ([3,foo], foo)
>>
>> Question 3
>>
>> In the comments below you say "but we don't know about pairs here”.   Are
>> you sure?  What stops you importing Data.Tuple into GHC.Prim?   This fancy
>> footwork is one more complication, if it could be avoided.
>>
>> That whole regime came before my time, but since we win a bit by *not*
>> fixing it, o wouldn't jump on it too quick.
>>
>>
>>
>> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191016/1367d39d/attachment.html>

From david.feuer at gmail.com  Thu Oct 17 01:31:14 2019
From: david.feuer at gmail.com (David Feuer)
Date: Wed, 16 Oct 2019 21:31:14 -0400
Subject: atomicModifyMutVar2
In-Reply-To: <CAHYVw0zC3qNRg8PxHMqY2Eg8X6O2Liz1Ud9dDE-S99DestDRsQ@mail.gmail.com>
References: <AM6PR83MB027941FDD4F28B4E66E452EDAD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9ueifTZmSW=rf9s3-S0-iw_Rn4+RKzRB69q00oZU_1etg@mail.gmail.com>
 <AM6PR83MB0279B1C6C105C77C3EC98476AD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9tReq55DzSiccW1w3i7pyLAx9htrEpRphO3V0tdk_8yvQ@mail.gmail.com>
 <AM6PR83MB0279925C28BF51C096DDB917AD900@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9snu7qA78iF7N-CU0YNgshzHU==Of+k5wtmdtYS=AajhQ@mail.gmail.com>
 <CAHYVw0zC3qNRg8PxHMqY2Eg8X6O2Liz1Ud9dDE-S99DestDRsQ@mail.gmail.com>
Message-ID: <CAMgWh9uRFNpjCQw8YXuiNEhHQU0JSAakh-hXFtGC1Vprvo-Bbg@mail.gmail.com>

Well, you basically need two primops. In approximate terms,

atomGen :: (a -> q) -> (q -> a) -> IORef a -> IO (a, q)
atomSel :: (a -> q) -> Int -> IORef a -> IO (a, q)

atomGen is rewritten to atomSel whenever the (q -> a) function is actually
a selector function, in which case it is represented by the field position.
Only atomGen would be considered "public", as the true type of atomSel
can't be expressed.

On Wed, Oct 16, 2019, 9:07 PM Carter Schonwald <carter.schonwald at gmail.com>
wrote:

>  how do you type check it / what type would you plausibly given it thats
> wired into very innards of ghc type system implementation? (by which i mean
> that most / all primops currently are pretty conservative in terms of the
> range of ghc type system features they use)
>
>
> On Wed, Oct 16, 2019 at 1:51 PM David Feuer <david.feuer at gmail.com> wrote:
>
>> What do you think about the version that takes a function as an argument,
>> and is optimized for the case where that's a selector? That really feels
>> like the platonic ideal here.
>>
>> On Mon, Oct 14, 2019, 6:35 PM Simon Peyton Jones <simonpj at microsoft.com>
>> wrote:
>>
>>> OK.   I propose:
>>>
>>>
>>>
>>>    - To give atomicModieyMutVarOf# its proper type, with a  pair, as in
>>>    the proposal.
>>>    - To do that I’ll fiddle with genprimopcode, to allow it to parse
>>>    tuples as well as unboxed tuples; not hard.
>>>    - This would disallow all this stuff about “any type that has a
>>>    first field looking like a”, restricting to pairs alone.  This didn’t form
>>>    part of the proposal, and was never documented.
>>>    - Add a bit more clarity to the documentation, so it’d clear what
>>>    must be forced.
>>>
>>>
>>>
>>> Any objections?
>>>
>>>
>>>
>>> Simon
>>>
>>>
>>>
>>> *From:* David Feuer <david.feuer at gmail.com>
>>> *Sent:* 12 October 2019 00:00
>>> *To:* Simon Peyton Jones <simonpj at microsoft.com>
>>> *Cc:* ghc-devs <ghc-devs at haskell.org>
>>> *Subject:* Re: atomicModifyMutVar2
>>>
>>>
>>>
>>> I don't remember what documentation, if any, it has. You're right that
>>> taking advantage of it is potentially risky. Here's what I think we really
>>> want:
>>>
>>>
>>>
>>> atomicModifyMutVarQ# :: MutVar# s a -> (q -> a) -> (a -> q)  -> State#
>>> s -> (# State# s, a, q #)
>>>
>>>
>>>
>>> where there's a special rule that (q -> a) is "obviously" a selector
>>> that selects a pointer.
>>>
>>> On Fri, Oct 11, 2019, 12:56 PM Simon Peyton Jones <simonpj at microsoft.com>
>>> wrote:
>>>
>>> The result doesn't have to be a pair. It can be a tuple of any size at
>>> all. Indeed, it can even be an arbitrary record type whose first pointer
>>> field has the appropriate type.
>>>
>>>
>>>
>>> I think that is 100.0% undocumented, in the code, or in the proposal.
>>> Are you sure this is a settled consensus among the interested parties?
>>>
>>>
>>>
>>> Adopting it would impose new invariants on the representation of values
>>> in GHC that I am deeply reluctant to impose.  I would much much prefer to
>>> stick with the pair that is (somewhat) documented.
>>>
>>>
>>>
>>> About pair vs Unit, yes, I can see (just) your point about why a pair
>>> might be useful.  Here’s a better example:
>>>
>>>
>>>
>>> Suppose mv :: MutVar# Int
>>>
>>>
>>>
>>> atomicModifyMutVar2# mv $ \a ->
>>>
>>>   let foo = f a
>>>
>>>   in (g foo, foo)
>>>
>>>
>>>
>>> Now, if f is expensive, *and g is not invertible*, then sharing foo
>>> might be useful.  It’s hard to think of a credible example, though.
>>> Regardless, we should document it.
>>>
>>>
>>>
>>> Simon
>>>
>>>
>>>
>>> *From:* David Feuer <david.feuer at gmail.com>
>>> *Sent:* 11 October 2019 17:03
>>> *To:* Simon Peyton Jones <simonpj at microsoft.com>
>>> *Cc:* ghc-devs <ghc-devs at haskell.org>
>>> *Subject:* Re: atomicModifyMutVar2
>>>
>>>
>>>
>>> On Fri, Oct 11, 2019, 11:08 AM Simon Peyton Jones <simonpj at microsoft.com>
>>> wrote:
>>>
>>> David
>>>
>>> I’m deeply puzzled atomicModifyMutVar2#.  I have read the proposal
>>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fghc-proposals%2Fghc-proposals%2Fblob%2Fmaster%2Fproposals%2F0149-atomicModifyMutVar.rst&data=02%7C01%7Csimonpj%40microsoft.com%7C0b9a79c810d7453ce16008d74e9ebfc1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637064316038353079&sdata=LGyYiZC9b9g3lZpzTdvQgyEwKCaEsIzFDBcacVE2N5M%3D&reserved=0>,
>>> and the comments in primops.txt.pp (reproduced below).
>>> Question 1
>>>
>>> I think the “real” type of atomicModifyMutVar2 is
>>>
>>> atomicModifyMutVar2# :: MutVar# s a
>>>
>>>                      -> (a -> (a,b))
>>>
>>>                      -> State# s
>>>
>>>                      -> (# State# s, a, (a, b) #)
>>>
>>>
>>>
>>> Close, but not quite. The result doesn't have to be a pair. It can be a
>>> tuple of any size at all. Indeed, it can even be an arbitrary record type
>>> whose first pointer field has the appropriate type.
>>>
>>>
>>>
>>> Nowhere is this explicitly stated, but I believe that the intended
>>> semantics of a call
>>>
>>> case (atomicModifyMutVar2# mv f s) of (# s’, x, r #) -> blah
>>>
>>> Then, suppose the old value of the MutVar was ‘*old’*
>>>
>>>    - The primop builds a thunk  *t *= *f old*
>>>    - The new value of the mutable variable is *(fst t)*
>>>    - The result *r* is t
>>>    - The result *x* is *old*
>>>
>>> Question: is that correct?   We should state it explicitly.
>>>
>>> Yes, that sounds right.
>>>
>>> Question 2
>>>
>>> Next question: Why does f have to return a pair?  So far as I can tell,
>>> it’s only so that a client can force it.   The ‘b’ part never seems to play
>>> a useful role.   So we could equally well have had
>>>
>>> atomicModifyMutVar2# :: MutVar# s a
>>>
>>>                      -> (a -> Box a)
>>>
>>>                      -> State# s
>>>
>>>                      -> (# State# s, a, Unit a #)
>>>
>>> where Unit is defined in Data.Tuple
>>>
>>>     data Unit a = Unit a
>>>
>>> Now you can force the result of (f old), just as with a pair.  But the
>>> ‘b’ would no longer complicate matters.
>>>
>>> Question: is the ‘b’ in the pair significant?   Or could we use Unit?
>>>
>>> Yes, it's somewhat significant. You actually can use Unit with the new
>>> primop (it's a tuple of arity 1), so that option is free. But using a pair
>>> gets you a bit more: you can build a thunk that's *shared* between the
>>> value installed in the MutVar and the one returned to the outside. Consider
>>>
>>>
>>>
>>> atomicModifyMutVar2# mv $ \a ->
>>>
>>>   let foo = expensive_computation a
>>>
>>>   in ([3,foo], foo)
>>>
>>> Question 3
>>>
>>> In the comments below you say "but we don't know about pairs here”.
>>> Are you sure?  What stops you importing Data.Tuple into GHC.Prim?   This
>>> fancy footwork is one more complication, if it could be avoided.
>>>
>>> That whole regime came before my time, but since we win a bit by *not*
>>> fixing it, o wouldn't jump on it too quick.
>>>
>>>
>>>
>>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191016/e8284cc5/attachment-0001.html>

From simonpj at microsoft.com  Thu Oct 17 07:35:26 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Thu, 17 Oct 2019 07:35:26 +0000
Subject: atomicModifyMutVar2
In-Reply-To: <CAHYVw0zC3qNRg8PxHMqY2Eg8X6O2Liz1Ud9dDE-S99DestDRsQ@mail.gmail.com>
References: <AM6PR83MB027941FDD4F28B4E66E452EDAD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9ueifTZmSW=rf9s3-S0-iw_Rn4+RKzRB69q00oZU_1etg@mail.gmail.com>
 <AM6PR83MB0279B1C6C105C77C3EC98476AD970@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9tReq55DzSiccW1w3i7pyLAx9htrEpRphO3V0tdk_8yvQ@mail.gmail.com>
 <AM6PR83MB0279925C28BF51C096DDB917AD900@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAMgWh9snu7qA78iF7N-CU0YNgshzHU==Of+k5wtmdtYS=AajhQ@mail.gmail.com>
 <CAHYVw0zC3qNRg8PxHMqY2Eg8X6O2Liz1Ud9dDE-S99DestDRsQ@mail.gmail.com>
Message-ID: <AM6PR83MB0279B6FC7025EF6EA641FE7BAD6D0@AM6PR83MB0279.EURPRD83.prod.outlook.com>

What do you think about the version that takes a function as an argument, and is optimized for the case where that's a selector?

In any case that would change the API etc and I think that’s a bridge too far for a tidy-up.  Make a proposal if you like!

Simon

From: Carter Schonwald <carter.schonwald at gmail.com>
Sent: 17 October 2019 02:08
To: David Feuer <david.feuer at gmail.com>
Cc: Simon Peyton Jones <simonpj at microsoft.com>; ghc-devs <ghc-devs at haskell.org>
Subject: Re: atomicModifyMutVar2

 how do you type check it / what type would you plausibly given it thats wired into very innards of ghc type system implementation? (by which i mean that most / all primops currently are pretty conservative in terms of the range of ghc type system features they use)


On Wed, Oct 16, 2019 at 1:51 PM David Feuer <david.feuer at gmail.com<mailto:david.feuer at gmail.com>> wrote:
What do you think about the version that takes a function as an argument, and is optimized for the case where that's a selector? That really feels like the platonic ideal here.

On Mon, Oct 14, 2019, 6:35 PM Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
OK.   I propose:


  *   To give atomicModieyMutVarOf# its proper type, with a  pair, as in the proposal.
  *   To do that I’ll fiddle with genprimopcode, to allow it to parse tuples as well as unboxed tuples; not hard.
  *   This would disallow all this stuff about “any type that has a first field looking like a”, restricting to pairs alone.  This didn’t form part of the proposal, and was never documented.
  *   Add a bit more clarity to the documentation, so it’d clear what must be forced.

Any objections?

Simon

From: David Feuer <david.feuer at gmail.com<mailto:david.feuer at gmail.com>>
Sent: 12 October 2019 00:00
To: Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
Cc: ghc-devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: Re: atomicModifyMutVar2

I don't remember what documentation, if any, it has. You're right that taking advantage of it is potentially risky. Here's what I think we really want:

atomicModifyMutVarQ# :: MutVar# s a -> (q -> a) -> (a -> q)  -> State# s -> (# State# s, a, q #)

where there's a special rule that (q -> a) is "obviously" a selector that selects a pointer.
On Fri, Oct 11, 2019, 12:56 PM Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
The result doesn't have to be a pair. It can be a tuple of any size at all. Indeed, it can even be an arbitrary record type whose first pointer field has the appropriate type.

I think that is 100.0% undocumented, in the code, or in the proposal.  Are you sure this is a settled consensus among the interested parties?

Adopting it would impose new invariants on the representation of values in GHC that I am deeply reluctant to impose.  I would much much prefer to stick with the pair that is (somewhat) documented.

About pair vs Unit, yes, I can see (just) your point about why a pair might be useful.  Here’s a better example:

Suppose mv :: MutVar# Int


atomicModifyMutVar2# mv $ \a ->

  let foo = f a

  in (g foo, foo)

Now, if f is expensive, and g is not invertible, then sharing foo might be useful.  It’s hard to think of a credible example, though.  Regardless, we should document it.

Simon

From: David Feuer <david.feuer at gmail.com<mailto:david.feuer at gmail.com>>
Sent: 11 October 2019 17:03
To: Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
Cc: ghc-devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: Re: atomicModifyMutVar2

On Fri, Oct 11, 2019, 11:08 AM Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
David
I’m deeply puzzled atomicModifyMutVar2#.  I have read the proposal<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fghc-proposals%2Fghc-proposals%2Fblob%2Fmaster%2Fproposals%2F0149-atomicModifyMutVar.rst&data=02%7C01%7Csimonpj%40microsoft.com%7Cfc36241d51074e135aa908d7529e6fb5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637068712746481714&sdata=IsK47SbO26e4lGf8Mvhwaz6gVHWWa07Twwk2WHnjEg8%3D&reserved=0>, and the comments in primops.txt.pp (reproduced below).
Question 1
I think the “real” type of atomicModifyMutVar2 is

atomicModifyMutVar2# :: MutVar# s a

                     -> (a -> (a,b))

                     -> State# s

                     -> (# State# s, a, (a, b) #)

Close, but not quite. The result doesn't have to be a pair. It can be a tuple of any size at all. Indeed, it can even be an arbitrary record type whose first pointer field has the appropriate type.

Nowhere is this explicitly stated, but I believe that the intended semantics of a call

case (atomicModifyMutVar2# mv f s) of (# s’, x, r #) -> blah
Then, suppose the old value of the MutVar was ‘old’

  *   The primop builds a thunk  t = f old
  *   The new value of the mutable variable is (fst t)
  *   The result r is t
  *   The result x is old
Question: is that correct?   We should state it explicitly.
Yes, that sounds right.
Question 2
Next question: Why does f have to return a pair?  So far as I can tell, it’s only so that a client can force it.   The ‘b’ part never seems to play a useful role.   So we could equally well have had

atomicModifyMutVar2# :: MutVar# s a

                     -> (a -> Box a)

                     -> State# s

                     -> (# State# s, a, Unit a #)
where Unit is defined in Data.Tuple

    data Unit a = Unit a
Now you can force the result of (f old), just as with a pair.  But the ‘b’ would no longer complicate matters.
Question: is the ‘b’ in the pair significant?   Or could we use Unit?
Yes, it's somewhat significant. You actually can use Unit with the new primop (it's a tuple of arity 1), so that option is free. But using a pair gets you a bit more: you can build a thunk that's *shared* between the value installed in the MutVar and the one returned to the outside. Consider


atomicModifyMutVar2# mv $ \a ->

  let foo = expensive_computation a

  in ([3,foo], foo)

Question 3
In the comments below you say "but we don't know about pairs here”.   Are you sure?  What stops you importing Data.Tuple into GHC.Prim?   This fancy footwork is one more complication, if it could be avoided.
That whole regime came before my time, but since we win a bit by *not* fixing it, o wouldn't jump on it too quick.

_______________________________________________
ghc-devs mailing list
ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=02%7C01%7Csimonpj%40microsoft.com%7Cfc36241d51074e135aa908d7529e6fb5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637068712746481714&sdata=Ew%2BUkvEMWFrhTM7ZemjCOISid7n6iqdJeJKCi9RX1IA%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191017/0122b36a/attachment.html>

From omeragacan at gmail.com  Fri Oct 18 11:23:48 2019
From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=)
Date: Fri, 18 Oct 2019 14:23:48 +0300
Subject: Why is libraries/ghci built with stage 0 compiler instead of stage 1?
Message-ID: <CAMQQO3kwXHHoWcoWkGK4=yBFoC85szXrdyonC=CsKJFpSC0jzg@mail.gmail.com>

Stage 1 compiler doesn't have interpreter, and doesn't run plugins or TH, so I
think GHCi stuff should not be used by stage 1 compiler, but for some reason the
"ghci" library (libraries/ghci) is built with stage 0 compiler instead of stage
1. Anyone know what this is?

Thanks,

Ömer

From moritz.angermann at gmail.com  Fri Oct 18 11:27:00 2019
From: moritz.angermann at gmail.com (Moritz Angermann)
Date: Fri, 18 Oct 2019 19:27:00 +0800
Subject: Why is libraries/ghci built with stage 0 compiler instead of
 stage 1?
In-Reply-To: <CAMQQO3kwXHHoWcoWkGK4=yBFoC85szXrdyonC=CsKJFpSC0jzg@mail.gmail.com>
References: <CAMQQO3kwXHHoWcoWkGK4=yBFoC85szXrdyonC=CsKJFpSC0jzg@mail.gmail.com>
Message-ID: <CAKfdd-zjNKEPxzMoOqHNFrAr+AmLf62px3Qq0ZfxetA6-q7Z=g@mail.gmail.com>

It can run TH using iserv; which depends on lib:ghci iirc.

On Fri, 18 Oct 2019 at 7:24 PM, Ömer Sinan Ağacan <omeragacan at gmail.com>
wrote:

> Stage 1 compiler doesn't have interpreter, and doesn't run plugins or TH,
> so I
> think GHCi stuff should not be used by stage 1 compiler, but for some
> reason the
> "ghci" library (libraries/ghci) is built with stage 0 compiler instead of
> stage
> 1. Anyone know what this is?
>
> Thanks,
>
> Ömer
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191018/df99f3ce/attachment.html>

From atreyu.bbb at gmail.com  Wed Oct 23 06:52:32 2019
From: atreyu.bbb at gmail.com (Javier Neira Sanchez)
Date: Wed, 23 Oct 2019 08:52:32 +0200
Subject: Status of the rich errors proposal
Message-ID: <CA+Nwh6d+9XufHw_=HkzRs6d-Y_-fQA_5=9sOdNZGuM692qokaQ@mail.gmail.com>

Hi, lately i am being collaborated in the haskell-ide-engine (hie) repo and
one of the issues i get to fix was related with the parsing of ghc errors.
It turns out that delimiters of terms in errors are different in windows
(`term') and *nix (‘term’) systems and that drive to parse errors.
Quite code actions and diagnostics are based in parsing them and they were
broken in windows.
I am afraid that the code is very brittle and close to human readable error
messages. the actual code look like this:

-- | Extract a term from a compiler message.

-- It looks for terms delimited between '‘' and '’' falling back to '`' and '\''

-- (the used ones in Windows systems).

extractTerm :: T.Text -> T.Text

extractTerm txt =

  case extract '‘' '’' txt of

    ""  -> extract '`' '\'' txt -- Needed for windows

    term -> term

  where extract b e = T.dropWhile (== b)

                    . T.dropWhileEnd (== e)

                    . T.dropAround (\c -> c /= b && c /= e)


or

extractImportableTerm :: T.Text -> Maybe (T.Text, SymbolImport SymbolType)

extractImportableTerm dirtyMsg = do

  (n, s) <- extractedTerm

  let n' = T.strip n

  return (n', s)

  where

    importMsg = S.headMay

      -- Get rid of the rename suggestion parts

      $ T.splitOn "Perhaps you meant "

      $ T.replace "\n" " "

      -- Get rid of trailing/leading whitespace on each individual line

      $ T.unlines

      $ map T.strip

      $ T.lines

      $ T.replace "* " "" -- Needed for Windows

      $ T.replace "• " "" dirtyMsg

    extractTerm prefix symTy =

      importMsg

          >>= T.stripPrefix prefix

          >>= \name -> Just (name, Import symTy)

    extractType b =

      extractTerm ("Not in scope: type constructor or class " <> b) Type

    extractedTerm = asum

      [ extractTerm "Variable not in scope: " Symbol

      , extractType "‘"

      , extractType "`" -- Needed for windows

      , extractTerm "Data constructor not in scope: " Constructor]


It is clearly unsatisfactory but hard to improve without changing the
messages to make it more structured.
Moreover any legitimate change on errors to make it better will likely
break it.

After exposing my worries in the hie irc channel, @mpickering pointed out
that it is already a proposal to improve error messages:

https://github.com/bgamari/ghc-proposals/blob/rich-errors-proposal/proposals/0000-rich-errors-proposal.rst

that nicely will improve the state of things.

Otoh there are already a way to output ghc errors as json (see
https://gitlab.haskell.org/ghc/ghc/issues/13190). It contains valuable info
about the error in specific fields but the message itself is in plain text.
So merging both features will let tools to handle compiler errors without
use the ghc api directly if needed.

what is the status of the proposal? hie an other tooling developers will
welcome it very heartly.

Thanks in advance!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191023/7436b9f0/attachment.html>

From matthewtpickering at gmail.com  Wed Oct 23 10:52:26 2019
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Wed, 23 Oct 2019 11:52:26 +0100
Subject: How to navigate around the source tree?
Message-ID: <CALuQ0m_xgHRUJzBRrOWFOJQPd17akc-Rmpe9PimqiQg=-SPvpQ@mail.gmail.com>

Hi,

The module rework has broken my workflow.

Now my tags file is useless for jumping for modules as there are
multiple "Utils" and "Types" modules. Invariable I am jumping to the
wrong one. What do other people do to avoid this?

Can we either revert these changes or give these modules unique names
to facilitate that only reliable way of navigating the code base.
Perhaps `HsUtils` and `StgUtils` would be appropriate to disambiguate
`Hs/Utils` and `StgToCmm/Utils`.

Cheers,

Matt

From sylvain at haskus.fr  Wed Oct 23 11:08:19 2019
From: sylvain at haskus.fr (Sylvain Henry)
Date: Wed, 23 Oct 2019 13:08:19 +0200
Subject: How to navigate around the source tree?
In-Reply-To: <CALuQ0m_xgHRUJzBRrOWFOJQPd17akc-Rmpe9PimqiQg=-SPvpQ@mail.gmail.com>
References: <CALuQ0m_xgHRUJzBRrOWFOJQPd17akc-Rmpe9PimqiQg=-SPvpQ@mail.gmail.com>
Message-ID: <929a4765-5e5a-afe4-c360-e482c94bc276@haskus.fr>

Hi,

How do you generate your tags file? It seems to be a shortcoming of the 
generator to not take into account the location of the definition file.

 > Perhaps `HsUtils` and `StgUtils` would be appropriate to 
disambiguate`Hs/Utils` and `StgToCmm/Utils`.

We are promoting the module prefixes (`Hs`, `Stg`, `Tc`, etc.) into 
proper module layers (e.g. `HsUtils` becomes `GHC.Hs.Utils`) so it would 
be redundant to add the prefixes back. :/

Cheers,
Sylvain

On 23/10/2019 12:52, Matthew Pickering wrote:
> Hi,
>
> The module rework has broken my workflow.
>
> Now my tags file is useless for jumping for modules as there are
> multiple "Utils" and "Types" modules. Invariable I am jumping to the
> wrong one. What do other people do to avoid this?
>
> Can we either revert these changes or give these modules unique names
> to facilitate that only reliable way of navigating the code base.
> Perhaps `HsUtils` and `StgUtils` would be appropriate to disambiguate
> `Hs/Utils` and `StgToCmm/Utils`.
>
> Cheers,
>
> Matt
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From matthewtpickering at gmail.com  Wed Oct 23 13:26:47 2019
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Wed, 23 Oct 2019 14:26:47 +0100
Subject: How to navigate around the source tree?
In-Reply-To: <929a4765-5e5a-afe4-c360-e482c94bc276@haskus.fr>
References: <CALuQ0m_xgHRUJzBRrOWFOJQPd17akc-Rmpe9PimqiQg=-SPvpQ@mail.gmail.com>
 <929a4765-5e5a-afe4-c360-e482c94bc276@haskus.fr>
Message-ID: <CALuQ0m-N0A1=jjsFRLt_Q=Mgn9mkLTUFNxLOWXNbFxfOxkBe3g@mail.gmail.com>

I use `fast-tags` which doesn't look at the hierarchy at all and I'm
not sure what the improvement would be as the names of the modules
would still clash.

If there is some other recommended way to jump to a module then that
would also work for me.

Matt


On Wed, Oct 23, 2019 at 12:08 PM Sylvain Henry <sylvain at haskus.fr> wrote:
>
> Hi,
>
> How do you generate your tags file? It seems to be a shortcoming of the
> generator to not take into account the location of the definition file.
>
>  > Perhaps `HsUtils` and `StgUtils` would be appropriate to
> disambiguate`Hs/Utils` and `StgToCmm/Utils`.
>
> We are promoting the module prefixes (`Hs`, `Stg`, `Tc`, etc.) into
> proper module layers (e.g. `HsUtils` becomes `GHC.Hs.Utils`) so it would
> be redundant to add the prefixes back. :/
>
> Cheers,
> Sylvain
>
> On 23/10/2019 12:52, Matthew Pickering wrote:
> > Hi,
> >
> > The module rework has broken my workflow.
> >
> > Now my tags file is useless for jumping for modules as there are
> > multiple "Utils" and "Types" modules. Invariable I am jumping to the
> > wrong one. What do other people do to avoid this?
> >
> > Can we either revert these changes or give these modules unique names
> > to facilitate that only reliable way of navigating the code base.
> > Perhaps `HsUtils` and `StgUtils` would be appropriate to disambiguate
> > `Hs/Utils` and `StgToCmm/Utils`.
> >
> > Cheers,
> >
> > Matt
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From sgraf1337 at gmail.com  Wed Oct 23 13:48:29 2019
From: sgraf1337 at gmail.com (Sebastian Graf)
Date: Wed, 23 Oct 2019 14:48:29 +0100
Subject: How to navigate around the source tree?
In-Reply-To: <CALuQ0m-N0A1=jjsFRLt_Q=Mgn9mkLTUFNxLOWXNbFxfOxkBe3g@mail.gmail.com>
References: <CALuQ0m_xgHRUJzBRrOWFOJQPd17akc-Rmpe9PimqiQg=-SPvpQ@mail.gmail.com>
 <929a4765-5e5a-afe4-c360-e482c94bc276@haskus.fr>
 <CALuQ0m-N0A1=jjsFRLt_Q=Mgn9mkLTUFNxLOWXNbFxfOxkBe3g@mail.gmail.com>
Message-ID: <CAAS+=P_phjciJc_O+6LWUNY2556HfXk=X80QHA+EShWLXgSmWA@mail.gmail.com>

FWIW, I'm using VSCode's fuzzy file search with Ctrl+P (and vim's
equivalent) rather successfully. Just tried it for Hs/Utils.hs by typing
'hsutils.hs'. It didn't turn up as the first result in VSCode, but it in
vim.

Am Mi., 23. Okt. 2019 um 14:27 Uhr schrieb Matthew Pickering <
matthewtpickering at gmail.com>:

> I use `fast-tags` which doesn't look at the hierarchy at all and I'm
> not sure what the improvement would be as the names of the modules
> would still clash.
>
> If there is some other recommended way to jump to a module then that
> would also work for me.
>
> Matt
>
>
> On Wed, Oct 23, 2019 at 12:08 PM Sylvain Henry <sylvain at haskus.fr> wrote:
> >
> > Hi,
> >
> > How do you generate your tags file? It seems to be a shortcoming of the
> > generator to not take into account the location of the definition file.
> >
> >  > Perhaps `HsUtils` and `StgUtils` would be appropriate to
> > disambiguate`Hs/Utils` and `StgToCmm/Utils`.
> >
> > We are promoting the module prefixes (`Hs`, `Stg`, `Tc`, etc.) into
> > proper module layers (e.g. `HsUtils` becomes `GHC.Hs.Utils`) so it would
> > be redundant to add the prefixes back. :/
> >
> > Cheers,
> > Sylvain
> >
> > On 23/10/2019 12:52, Matthew Pickering wrote:
> > > Hi,
> > >
> > > The module rework has broken my workflow.
> > >
> > > Now my tags file is useless for jumping for modules as there are
> > > multiple "Utils" and "Types" modules. Invariable I am jumping to the
> > > wrong one. What do other people do to avoid this?
> > >
> > > Can we either revert these changes or give these modules unique names
> > > to facilitate that only reliable way of navigating the code base.
> > > Perhaps `HsUtils` and `StgUtils` would be appropriate to disambiguate
> > > `Hs/Utils` and `StgToCmm/Utils`.
> > >
> > > Cheers,
> > >
> > > Matt
> > > _______________________________________________
> > > ghc-devs mailing list
> > > ghc-devs at haskell.org
> > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191023/12ecfe0c/attachment.html>

From sylvain at haskus.fr  Wed Oct 23 13:54:49 2019
From: sylvain at haskus.fr (Sylvain Henry)
Date: Wed, 23 Oct 2019 15:54:49 +0200
Subject: How to navigate around the source tree?
In-Reply-To: <CALuQ0m-N0A1=jjsFRLt_Q=Mgn9mkLTUFNxLOWXNbFxfOxkBe3g@mail.gmail.com>
References: <CALuQ0m_xgHRUJzBRrOWFOJQPd17akc-Rmpe9PimqiQg=-SPvpQ@mail.gmail.com>
 <929a4765-5e5a-afe4-c360-e482c94bc276@haskus.fr>
 <CALuQ0m-N0A1=jjsFRLt_Q=Mgn9mkLTUFNxLOWXNbFxfOxkBe3g@mail.gmail.com>
Message-ID: <2bd1f930-ca84-1ab5-a423-14667b77d29a@haskus.fr>

With `--fully-qualified` fast-tags also generates qualified tags:

```
GHC.Hs    GHC/Hs.hs    21;"    m
GHC.Hs.Binds    GHC/Hs/Binds.hs    20;"    m
GHC.Hs.Binds.ABE    GHC/Hs/Binds.hs    349;"    C
...
```

If your code editor can search for qualified tags, I guess it should 
work. There is a script for Vim 
(https://github.com/elaforge/fast-tags/blob/master/tools/qualified_tag.py) 
for example.

Sylvain


On 23/10/2019 15:26, Matthew Pickering wrote:
> I use `fast-tags` which doesn't look at the hierarchy at all and I'm
> not sure what the improvement would be as the names of the modules
> would still clash.
>
> If there is some other recommended way to jump to a module then that
> would also work for me.
>
> Matt
>
>
> On Wed, Oct 23, 2019 at 12:08 PM Sylvain Henry <sylvain at haskus.fr> wrote:
>> Hi,
>>
>> How do you generate your tags file? It seems to be a shortcoming of the
>> generator to not take into account the location of the definition file.
>>
>>   > Perhaps `HsUtils` and `StgUtils` would be appropriate to
>> disambiguate`Hs/Utils` and `StgToCmm/Utils`.
>>
>> We are promoting the module prefixes (`Hs`, `Stg`, `Tc`, etc.) into
>> proper module layers (e.g. `HsUtils` becomes `GHC.Hs.Utils`) so it would
>> be redundant to add the prefixes back. :/
>>
>> Cheers,
>> Sylvain
>>
>> On 23/10/2019 12:52, Matthew Pickering wrote:
>>> Hi,
>>>
>>> The module rework has broken my workflow.
>>>
>>> Now my tags file is useless for jumping for modules as there are
>>> multiple "Utils" and "Types" modules. Invariable I am jumping to the
>>> wrong one. What do other people do to avoid this?
>>>
>>> Can we either revert these changes or give these modules unique names
>>> to facilitate that only reliable way of navigating the code base.
>>> Perhaps `HsUtils` and `StgUtils` would be appropriate to disambiguate
>>> `Hs/Utils` and `StgToCmm/Utils`.
>>>
>>> Cheers,
>>>
>>> Matt
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From omeragacan at gmail.com  Wed Oct 23 13:54:17 2019
From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=)
Date: Wed, 23 Oct 2019 16:54:17 +0300
Subject: How to navigate around the source tree?
In-Reply-To: <CAAS+=P_phjciJc_O+6LWUNY2556HfXk=X80QHA+EShWLXgSmWA@mail.gmail.com>
References: <CALuQ0m_xgHRUJzBRrOWFOJQPd17akc-Rmpe9PimqiQg=-SPvpQ@mail.gmail.com>
 <929a4765-5e5a-afe4-c360-e482c94bc276@haskus.fr>
 <CALuQ0m-N0A1=jjsFRLt_Q=Mgn9mkLTUFNxLOWXNbFxfOxkBe3g@mail.gmail.com>
 <CAAS+=P_phjciJc_O+6LWUNY2556HfXk=X80QHA+EShWLXgSmWA@mail.gmail.com>
Message-ID: <CAMQQO3=p__dWw+ewfLZkvnrnXqeaCCiqFKyKBKQBSFHwPNW3Cg@mail.gmail.com>

I use a file finder (fzf) for jumping to files. Because module names follow file
paths to jump to e.g. StgToCmmUtils.Utils I usually type `<C-p>stgcmmutils` and
fzf finds the correct file `compiler/GHC/StgToCmm/Utils.hs`.

When generating tags I omit module names for this reason, it's easy with a good
file finder to jump to modules already, no need to generate tags for the
modules.

fast-tags commands I use:

- When working on the compiler:

  $ fast-tags --no-module-tags driver ghc compiler

- When working on the RTS:

  $ fast-tags --no-module-tags driver ghc compiler
  $ ctags --append -R rts/**/*.c rts/**/*.h includes/**/*.h

- When working on the libraries:

  $ fast-tags --no-module-tags driver ghc compiler libraries

Ömer

Sebastian Graf <sgraf1337 at gmail.com>, 23 Eki 2019 Çar, 16:49 tarihinde
şunu yazdı:
>
> FWIW, I'm using VSCode's fuzzy file search with Ctrl+P (and vim's equivalent) rather successfully. Just tried it for Hs/Utils.hs by typing 'hsutils.hs'. It didn't turn up as the first result in VSCode, but it in vim.
>
> Am Mi., 23. Okt. 2019 um 14:27 Uhr schrieb Matthew Pickering <matthewtpickering at gmail.com>:
>>
>> I use `fast-tags` which doesn't look at the hierarchy at all and I'm
>> not sure what the improvement would be as the names of the modules
>> would still clash.
>>
>> If there is some other recommended way to jump to a module then that
>> would also work for me.
>>
>> Matt
>>
>>
>> On Wed, Oct 23, 2019 at 12:08 PM Sylvain Henry <sylvain at haskus.fr> wrote:
>> >
>> > Hi,
>> >
>> > How do you generate your tags file? It seems to be a shortcoming of the
>> > generator to not take into account the location of the definition file.
>> >
>> >  > Perhaps `HsUtils` and `StgUtils` would be appropriate to
>> > disambiguate`Hs/Utils` and `StgToCmm/Utils`.
>> >
>> > We are promoting the module prefixes (`Hs`, `Stg`, `Tc`, etc.) into
>> > proper module layers (e.g. `HsUtils` becomes `GHC.Hs.Utils`) so it would
>> > be redundant to add the prefixes back. :/
>> >
>> > Cheers,
>> > Sylvain
>> >
>> > On 23/10/2019 12:52, Matthew Pickering wrote:
>> > > Hi,
>> > >
>> > > The module rework has broken my workflow.
>> > >
>> > > Now my tags file is useless for jumping for modules as there are
>> > > multiple "Utils" and "Types" modules. Invariable I am jumping to the
>> > > wrong one. What do other people do to avoid this?
>> > >
>> > > Can we either revert these changes or give these modules unique names
>> > > to facilitate that only reliable way of navigating the code base.
>> > > Perhaps `HsUtils` and `StgUtils` would be appropriate to disambiguate
>> > > `Hs/Utils` and `StgToCmm/Utils`.
>> > >
>> > > Cheers,
>> > >
>> > > Matt
>> > > _______________________________________________
>> > > ghc-devs mailing list
>> > > ghc-devs at haskell.org
>> > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> > _______________________________________________
>> > ghc-devs mailing list
>> > ghc-devs at haskell.org
>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From matthewtpickering at gmail.com  Wed Oct 23 15:48:45 2019
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Wed, 23 Oct 2019 16:48:45 +0100
Subject: How to navigate around the source tree?
In-Reply-To: <CAMQQO3=p__dWw+ewfLZkvnrnXqeaCCiqFKyKBKQBSFHwPNW3Cg@mail.gmail.com>
References: <CALuQ0m_xgHRUJzBRrOWFOJQPd17akc-Rmpe9PimqiQg=-SPvpQ@mail.gmail.com>
 <929a4765-5e5a-afe4-c360-e482c94bc276@haskus.fr>
 <CALuQ0m-N0A1=jjsFRLt_Q=Mgn9mkLTUFNxLOWXNbFxfOxkBe3g@mail.gmail.com>
 <CAAS+=P_phjciJc_O+6LWUNY2556HfXk=X80QHA+EShWLXgSmWA@mail.gmail.com>
 <CAMQQO3=p__dWw+ewfLZkvnrnXqeaCCiqFKyKBKQBSFHwPNW3Cg@mail.gmail.com>
Message-ID: <CALuQ0m8YSePBgndc1V=i-GLHEZuP6O4YL2_+U6q1Vk6mzDmqSA@mail.gmail.com>

Thanks Omer, Sylvain and Sebastian

.

I just configured my editor to use fzf and now I can use the `:GFiles`
command to perform fuzzy search on files which is probably better than
tags. If anyone else is using NixOS, all I had to do was add the
`fzf-vim` plugin to the vim configuration.

Cheers,

Matt

On Wed, Oct 23, 2019 at 2:54 PM Ömer Sinan Ağacan <omeragacan at gmail.com> wrote:
>
> I use a file finder (fzf) for jumping to files. Because module names follow file
> paths to jump to e.g. StgToCmmUtils.Utils I usually type `<C-p>stgcmmutils` and
> fzf finds the correct file `compiler/GHC/StgToCmm/Utils.hs`.
>
> When generating tags I omit module names for this reason, it's easy with a good
> file finder to jump to modules already, no need to generate tags for the
> modules.
>
> fast-tags commands I use:
>
> - When working on the compiler:
>
>   $ fast-tags --no-module-tags driver ghc compiler
>
> - When working on the RTS:
>
>   $ fast-tags --no-module-tags driver ghc compiler
>   $ ctags --append -R rts/**/*.c rts/**/*.h includes/**/*.h
>
> - When working on the libraries:
>
>   $ fast-tags --no-module-tags driver ghc compiler libraries
>
> Ömer
>
> Sebastian Graf <sgraf1337 at gmail.com>, 23 Eki 2019 Çar, 16:49 tarihinde
> şunu yazdı:
> >
> > FWIW, I'm using VSCode's fuzzy file search with Ctrl+P (and vim's equivalent) rather successfully. Just tried it for Hs/Utils.hs by typing 'hsutils.hs'. It didn't turn up as the first result in VSCode, but it in vim.
> >
> > Am Mi., 23. Okt. 2019 um 14:27 Uhr schrieb Matthew Pickering <matthewtpickering at gmail.com>:
> >>
> >> I use `fast-tags` which doesn't look at the hierarchy at all and I'm
> >> not sure what the improvement would be as the names of the modules
> >> would still clash.
> >>
> >> If there is some other recommended way to jump to a module then that
> >> would also work for me.
> >>
> >> Matt
> >>
> >>
> >> On Wed, Oct 23, 2019 at 12:08 PM Sylvain Henry <sylvain at haskus.fr> wrote:
> >> >
> >> > Hi,
> >> >
> >> > How do you generate your tags file? It seems to be a shortcoming of the
> >> > generator to not take into account the location of the definition file.
> >> >
> >> >  > Perhaps `HsUtils` and `StgUtils` would be appropriate to
> >> > disambiguate`Hs/Utils` and `StgToCmm/Utils`.
> >> >
> >> > We are promoting the module prefixes (`Hs`, `Stg`, `Tc`, etc.) into
> >> > proper module layers (e.g. `HsUtils` becomes `GHC.Hs.Utils`) so it would
> >> > be redundant to add the prefixes back. :/
> >> >
> >> > Cheers,
> >> > Sylvain
> >> >
> >> > On 23/10/2019 12:52, Matthew Pickering wrote:
> >> > > Hi,
> >> > >
> >> > > The module rework has broken my workflow.
> >> > >
> >> > > Now my tags file is useless for jumping for modules as there are
> >> > > multiple "Utils" and "Types" modules. Invariable I am jumping to the
> >> > > wrong one. What do other people do to avoid this?
> >> > >
> >> > > Can we either revert these changes or give these modules unique names
> >> > > to facilitate that only reliable way of navigating the code base.
> >> > > Perhaps `HsUtils` and `StgUtils` would be appropriate to disambiguate
> >> > > `Hs/Utils` and `StgToCmm/Utils`.
> >> > >
> >> > > Cheers,
> >> > >
> >> > > Matt
> >> > > _______________________________________________
> >> > > ghc-devs mailing list
> >> > > ghc-devs at haskell.org
> >> > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >> > _______________________________________________
> >> > ghc-devs mailing list
> >> > ghc-devs at haskell.org
> >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >> _______________________________________________
> >> ghc-devs mailing list
> >> ghc-devs at haskell.org
> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From b at chreekat.net  Wed Oct 23 16:00:52 2019
From: b at chreekat.net (Bryan Richter)
Date: Wed, 23 Oct 2019 18:00:52 +0200
Subject: How to navigate around the source tree?
In-Reply-To: <CALuQ0m8YSePBgndc1V=i-GLHEZuP6O4YL2_+U6q1Vk6mzDmqSA@mail.gmail.com>
References: <CALuQ0m_xgHRUJzBRrOWFOJQPd17akc-Rmpe9PimqiQg=-SPvpQ@mail.gmail.com>
 <929a4765-5e5a-afe4-c360-e482c94bc276@haskus.fr>
 <CALuQ0m-N0A1=jjsFRLt_Q=Mgn9mkLTUFNxLOWXNbFxfOxkBe3g@mail.gmail.com>
 <CAAS+=P_phjciJc_O+6LWUNY2556HfXk=X80QHA+EShWLXgSmWA@mail.gmail.com>
 <CAMQQO3=p__dWw+ewfLZkvnrnXqeaCCiqFKyKBKQBSFHwPNW3Cg@mail.gmail.com>
 <CALuQ0m8YSePBgndc1V=i-GLHEZuP6O4YL2_+U6q1Vk6mzDmqSA@mail.gmail.com>
Message-ID: <CAH9BU=zXro4sU4-jX1VHiH6oekH=ALQ5MsHEHf-TvPorgCfWTQ@mail.gmail.com>

Duplicate record fields is going to make this a bigger problem. Vim does
support duplicate tags (:tselect and :tjump and related bindings), but
hopefully haskell-ide-engine will one day provide us with semantic tags and
solve this problem once and for all!

On Wed, 23 Oct 2019, 17.49 Matthew Pickering, <matthewtpickering at gmail.com>
wrote:

> Thanks Omer, Sylvain and Sebastian
>
> .
>
> I just configured my editor to use fzf and now I can use the `:GFiles`
> command to perform fuzzy search on files which is probably better than
> tags. If anyone else is using NixOS, all I had to do was add the
> `fzf-vim` plugin to the vim configuration.
>
> Cheers,
>
> Matt
>
> On Wed, Oct 23, 2019 at 2:54 PM Ömer Sinan Ağacan <omeragacan at gmail.com>
> wrote:
> >
> > I use a file finder (fzf) for jumping to files. Because module names
> follow file
> > paths to jump to e.g. StgToCmmUtils.Utils I usually type
> `<C-p>stgcmmutils` and
> > fzf finds the correct file `compiler/GHC/StgToCmm/Utils.hs`.
> >
> > When generating tags I omit module names for this reason, it's easy with
> a good
> > file finder to jump to modules already, no need to generate tags for the
> > modules.
> >
> > fast-tags commands I use:
> >
> > - When working on the compiler:
> >
> >   $ fast-tags --no-module-tags driver ghc compiler
> >
> > - When working on the RTS:
> >
> >   $ fast-tags --no-module-tags driver ghc compiler
> >   $ ctags --append -R rts/**/*.c rts/**/*.h includes/**/*.h
> >
> > - When working on the libraries:
> >
> >   $ fast-tags --no-module-tags driver ghc compiler libraries
> >
> > Ömer
> >
> > Sebastian Graf <sgraf1337 at gmail.com>, 23 Eki 2019 Çar, 16:49 tarihinde
> > şunu yazdı:
> > >
> > > FWIW, I'm using VSCode's fuzzy file search with Ctrl+P (and vim's
> equivalent) rather successfully. Just tried it for Hs/Utils.hs by typing
> 'hsutils.hs'. It didn't turn up as the first result in VSCode, but it in
> vim.
> > >
> > > Am Mi., 23. Okt. 2019 um 14:27 Uhr schrieb Matthew Pickering <
> matthewtpickering at gmail.com>:
> > >>
> > >> I use `fast-tags` which doesn't look at the hierarchy at all and I'm
> > >> not sure what the improvement would be as the names of the modules
> > >> would still clash.
> > >>
> > >> If there is some other recommended way to jump to a module then that
> > >> would also work for me.
> > >>
> > >> Matt
> > >>
> > >>
> > >> On Wed, Oct 23, 2019 at 12:08 PM Sylvain Henry <sylvain at haskus.fr>
> wrote:
> > >> >
> > >> > Hi,
> > >> >
> > >> > How do you generate your tags file? It seems to be a shortcoming of
> the
> > >> > generator to not take into account the location of the definition
> file.
> > >> >
> > >> >  > Perhaps `HsUtils` and `StgUtils` would be appropriate to
> > >> > disambiguate`Hs/Utils` and `StgToCmm/Utils`.
> > >> >
> > >> > We are promoting the module prefixes (`Hs`, `Stg`, `Tc`, etc.) into
> > >> > proper module layers (e.g. `HsUtils` becomes `GHC.Hs.Utils`) so it
> would
> > >> > be redundant to add the prefixes back. :/
> > >> >
> > >> > Cheers,
> > >> > Sylvain
> > >> >
> > >> > On 23/10/2019 12:52, Matthew Pickering wrote:
> > >> > > Hi,
> > >> > >
> > >> > > The module rework has broken my workflow.
> > >> > >
> > >> > > Now my tags file is useless for jumping for modules as there are
> > >> > > multiple "Utils" and "Types" modules. Invariable I am jumping to
> the
> > >> > > wrong one. What do other people do to avoid this?
> > >> > >
> > >> > > Can we either revert these changes or give these modules unique
> names
> > >> > > to facilitate that only reliable way of navigating the code base.
> > >> > > Perhaps `HsUtils` and `StgUtils` would be appropriate to
> disambiguate
> > >> > > `Hs/Utils` and `StgToCmm/Utils`.
> > >> > >
> > >> > > Cheers,
> > >> > >
> > >> > > Matt
> > >> > > _______________________________________________
> > >> > > ghc-devs mailing list
> > >> > > ghc-devs at haskell.org
> > >> > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> > >> > _______________________________________________
> > >> > ghc-devs mailing list
> > >> > ghc-devs at haskell.org
> > >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> > >> _______________________________________________
> > >> ghc-devs mailing list
> > >> ghc-devs at haskell.org
> > >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> > >
> > > _______________________________________________
> > > ghc-devs mailing list
> > > ghc-devs at haskell.org
> > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191023/8904a1b8/attachment.html>

From simonpj at microsoft.com  Wed Oct 23 16:21:57 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Wed, 23 Oct 2019 16:21:57 +0000
Subject: Urgent: git problem
Message-ID: <AM6PR83MB02793C8CD7E9E9C67E553D33AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>

Aieee!   All my GHC repos are failing with this.  As a result I can't pull.  What should I do?   Thanks!
Simon


git pull

error: cannot lock ref 'refs/remotes/origin/wip/rae/remove-tc-dep': 'refs/remotes/origin/wip/rae' exists; cannot create 'refs/remotes/origin/wip/rae/remove-tc-dep'

>From gitlab.haskell.org:ghc/ghc

! [new branch]            wip/rae/remove-tc-dep    -> origin/wip/rae/remove-tc-dep  (unable to update local ref)

error: cannot lock ref 'refs/remotes/origin/wip/rae/split-up-modules': 'refs/remotes/origin/wip/rae' exists; cannot create 'refs/remotes/origin/wip/rae/split-up-modules'

! [new branch]            wip/rae/split-up-modules -> origin/wip/rae/split-up-modules  (unable to update local ref)

simonpj at MSRC-3645512:~/code/HEAD-2$
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191023/e7b0669f/attachment.html>

From simonpj at microsoft.com  Wed Oct 23 16:28:55 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Wed, 23 Oct 2019 16:28:55 +0000
Subject: Urgent: git problem
In-Reply-To: <AM6PR83MB02793C8CD7E9E9C67E553D33AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <AM6PR83MB02793C8CD7E9E9C67E553D33AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <AM6PR83MB0279633AA3CF6715A506B1B8AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>

a completely fresh clone seems ok.  but that's a bit brutal

From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Simon Peyton Jones via ghc-devs
Sent: 23 October 2019 17:22
To: ghc-devs <ghc-devs at haskell.org>
Subject: Urgent: git problem

Aieee!   All my GHC repos are failing with this.  As a result I can't pull.  What should I do?   Thanks!
Simon


git pull

error: cannot lock ref 'refs/remotes/origin/wip/rae/remove-tc-dep': 'refs/remotes/origin/wip/rae' exists; cannot create 'refs/remotes/origin/wip/rae/remove-tc-dep'

>From gitlab.haskell.org:ghc/ghc

! [new branch]            wip/rae/remove-tc-dep    -> origin/wip/rae/remove-tc-dep  (unable to update local ref)

error: cannot lock ref 'refs/remotes/origin/wip/rae/split-up-modules': 'refs/remotes/origin/wip/rae' exists; cannot create 'refs/remotes/origin/wip/rae/split-up-modules'

! [new branch]            wip/rae/split-up-modules -> origin/wip/rae/split-up-modules  (unable to update local ref)

simonpj at MSRC-3645512:~/code/HEAD-2$
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191023/dcef9ef2/attachment.html>

From sgraf1337 at gmail.com  Wed Oct 23 16:29:20 2019
From: sgraf1337 at gmail.com (Sebastian Graf)
Date: Wed, 23 Oct 2019 17:29:20 +0100
Subject: Urgent: git problem
In-Reply-To: <AM6PR83MB02793C8CD7E9E9C67E553D33AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <AM6PR83MB02793C8CD7E9E9C67E553D33AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <CAAS+=P_yYMKLyU3ym-+tzTzfAMasZrHNED1e1DhbpMiMcCG8vA@mail.gmail.com>

Hi,

Some googling turned up this SO thread
https://stackoverflow.com/a/43253320/388010
Does that help?

Cheers
Sebastian

Am Mi., 23. Okt. 2019 um 17:22 Uhr schrieb Simon Peyton Jones via ghc-devs <
ghc-devs at haskell.org>:

> Aieee!   All my GHC repos are failing with this.  As a result I can’t
> pull.  What should I do?   Thanks!
>
> Simon
>
>
>
> git pull
>
> error: cannot lock ref 'refs/remotes/origin/wip/rae/remove-tc-dep': 'refs/remotes/origin/wip/rae'
> exists; cannot create 'refs/remotes/origin/wip/rae/remove-tc-dep'
>
> From gitlab.haskell.org:ghc/ghc
>
> ! [new branch]            wip/rae/remove-tc-dep    ->
> origin/wip/rae/remove-tc-dep  (unable to update local ref)
>
> error: cannot lock ref 'refs/remotes/origin/wip/rae/split-up-modules':
> 'refs/remotes/origin/wip/rae' exists; cannot create
> 'refs/remotes/origin/wip/rae/split-up-modules'
>
> ! [new branch]            wip/rae/split-up-modules ->
> origin/wip/rae/split-up-modules  (unable to update local ref)
>
> simonpj at MSRC-3645512:~/code/HEAD-2$
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191023/998e240c/attachment.html>

From alan.zimm at gmail.com  Wed Oct 23 16:29:50 2019
From: alan.zimm at gmail.com (Alan & Kim Zimmerman)
Date: Wed, 23 Oct 2019 17:29:50 +0100
Subject: Urgent: git problem
In-Reply-To: <AM6PR83MB0279633AA3CF6715A506B1B8AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <AM6PR83MB02793C8CD7E9E9C67E553D33AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <AM6PR83MB0279633AA3CF6715A506B1B8AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <CANma=H8Di8ant=rQZxHhBY0omYw+RRkExSbsMZ7WDM98Zdp6QQ@mail.gmail.com>

I do

git pull origin master

And it then works

Alan

On Wed, 23 Oct 2019, 17:29 Simon Peyton Jones via ghc-devs, <
ghc-devs at haskell.org> wrote:

> a completely fresh clone seems ok.  but that’s a bit brutal
>
>
>
> *From:* ghc-devs <ghc-devs-bounces at haskell.org> *On Behalf Of *Simon
> Peyton Jones via ghc-devs
> *Sent:* 23 October 2019 17:22
> *To:* ghc-devs <ghc-devs at haskell.org>
> *Subject:* Urgent: git problem
>
>
>
> Aieee!   All my GHC repos are failing with this.  As a result I can’t
> pull.  What should I do?   Thanks!
>
> Simon
>
>
>
> git pull
>
> error: cannot lock ref 'refs/remotes/origin/wip/rae/remove-tc-dep': 'refs/remotes/origin/wip/rae'
> exists; cannot create 'refs/remotes/origin/wip/rae/remove-tc-dep'
>
> From gitlab.haskell.org:ghc/ghc
>
> ! [new branch]            wip/rae/remove-tc-dep    ->
> origin/wip/rae/remove-tc-dep  (unable to update local ref)
>
> error: cannot lock ref 'refs/remotes/origin/wip/rae/split-up-modules':
> 'refs/remotes/origin/wip/rae' exists; cannot create
> 'refs/remotes/origin/wip/rae/split-up-modules'
>
> ! [new branch]            wip/rae/split-up-modules ->
> origin/wip/rae/split-up-modules  (unable to update local ref)
>
> simonpj at MSRC-3645512:~/code/HEAD-2$
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191023/cde2e849/attachment.html>

From ollie at ocharles.org.uk  Wed Oct 23 16:29:56 2019
From: ollie at ocharles.org.uk (Oliver Charles)
Date: Wed, 23 Oct 2019 17:29:56 +0100
Subject: Urgent: git problem
In-Reply-To: <AM6PR83MB02793C8CD7E9E9C67E553D33AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <AM6PR83MB02793C8CD7E9E9C67E553D33AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <CAGRp5Rn8i_fyB3v0eWh=g_7O7SqvfUUrukUwcLPN6NOMb99jxQ@mail.gmail.com>

I usually encounter this when there has once been a branch called
"wip/rae" (so .git would have a *file* wip/rae), but now there's a
branch called "wip/rae/remove-tc-dep", so "wip/rae" should be a
directory, but it's already in .git as a file! I think "git remote
prune" fixes this, but I can't entirely recall.

On Wed, Oct 23, 2019 at 5:22 PM Simon Peyton Jones via ghc-devs
<ghc-devs at haskell.org> wrote:
>
> Aieee!   All my GHC repos are failing with this.  As a result I can’t pull.  What should I do?   Thanks!
>
> Simon
>
>
>
> git pull
>
> error: cannot lock ref 'refs/remotes/origin/wip/rae/remove-tc-dep': 'refs/remotes/origin/wip/rae' exists; cannot create 'refs/remotes/origin/wip/rae/remove-tc-dep'
>
> From gitlab.haskell.org:ghc/ghc
>
> ! [new branch]            wip/rae/remove-tc-dep    -> origin/wip/rae/remove-tc-dep  (unable to update local ref)
>
> error: cannot lock ref 'refs/remotes/origin/wip/rae/split-up-modules': 'refs/remotes/origin/wip/rae' exists; cannot create 'refs/remotes/origin/wip/rae/split-up-modules'
>
> ! [new branch]            wip/rae/split-up-modules -> origin/wip/rae/split-up-modules  (unable to update local ref)
>
> simonpj at MSRC-3645512:~/code/HEAD-2$
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From simonpj at microsoft.com  Wed Oct 23 16:35:24 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Wed, 23 Oct 2019 16:35:24 +0000
Subject: Urgent: git problem
In-Reply-To: <CAGRp5Rn8i_fyB3v0eWh=g_7O7SqvfUUrukUwcLPN6NOMb99jxQ@mail.gmail.com>
References: <AM6PR83MB02793C8CD7E9E9C67E553D33AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAGRp5Rn8i_fyB3v0eWh=g_7O7SqvfUUrukUwcLPN6NOMb99jxQ@mail.gmail.com>
Message-ID: <AM6PR83MB0279462F16B7D025757D1A86AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>

Thanks.  Yes 'git remote prune origin' did the trick!

Simon

|  -----Original Message-----
|  From: Oliver Charles <ollie at ocharles.org.uk>
|  Sent: 23 October 2019 17:30
|  To: Simon Peyton Jones <simonpj at microsoft.com>
|  Cc: ghc-devs <ghc-devs at haskell.org>
|  Subject: Re: Urgent: git problem
|  
|  I usually encounter this when there has once been a branch called
|  "wip/rae" (so .git would have a *file* wip/rae), but now there's a
|  branch called "wip/rae/remove-tc-dep", so "wip/rae" should be a
|  directory, but it's already in .git as a file! I think "git remote
|  prune" fixes this, but I can't entirely recall.
|  
|  On Wed, Oct 23, 2019 at 5:22 PM Simon Peyton Jones via ghc-devs
|  <ghc-devs at haskell.org> wrote:
|  >
|  > Aieee!   All my GHC repos are failing with this.  As a result I can’t
|  pull.  What should I do?   Thanks!
|  >
|  > Simon
|  >
|  >
|  >
|  > git pull
|  >
|  > error: cannot lock ref 'refs/remotes/origin/wip/rae/remove-tc-dep':
|  'refs/remotes/origin/wip/rae' exists; cannot create
|  'refs/remotes/origin/wip/rae/remove-tc-dep'
|  >
|  > From gitlab.haskell.org:ghc/ghc
|  >
|  > ! [new branch]            wip/rae/remove-tc-dep    ->
|  origin/wip/rae/remove-tc-dep  (unable to update local ref)
|  >
|  > error: cannot lock ref 'refs/remotes/origin/wip/rae/split-up-modules':
|  'refs/remotes/origin/wip/rae' exists; cannot create
|  'refs/remotes/origin/wip/rae/split-up-modules'
|  >
|  > ! [new branch]            wip/rae/split-up-modules ->
|  origin/wip/rae/split-up-modules  (unable to update local ref)
|  >
|  > simonpj at MSRC-3645512:~/code/HEAD-2$
|  >
|  > _______________________________________________
|  > ghc-devs mailing list
|  > ghc-devs at haskell.org
|  >
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
|  ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7Ca21353034c8540fe2e1208d7
|  57d644a6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637074450093374131&a
|  mp;sdata=jMgjQRgGI93G4HbZPhfnmEEOyRcgK%2BBOhdcQYmH2m1M%3D&amp;reserved=0

From b at chreekat.net  Wed Oct 23 16:36:03 2019
From: b at chreekat.net (Bryan Richter)
Date: Wed, 23 Oct 2019 18:36:03 +0200
Subject: Urgent: git problem
In-Reply-To: <AM6PR83MB0279633AA3CF6715A506B1B8AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <AM6PR83MB02793C8CD7E9E9C67E553D33AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <AM6PR83MB0279633AA3CF6715A506B1B8AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <CAH9BU=yktY==PRx1n4ZZkgadH_+MwA+iZb+8_wz5qHS_Ji7+kQ@mail.gmail.com>

I suspect a git fetch --prune might help. It looks like git's mapping of
remote branches to a directory tree has caused a conflict between new and
stale branches: ../rae/... can't be a directory if it's already a file!

If that still doesn't work (if it tries to record new branches before
deleting stale ones), you'd have to muck around in .git/refs/remotes and
remove the files manually.

On Wed, 23 Oct 2019, 18.29 Simon Peyton Jones via ghc-devs, <
ghc-devs at haskell.org> wrote:

> a completely fresh clone seems ok.  but that’s a bit brutal
>
>
>
> *From:* ghc-devs <ghc-devs-bounces at haskell.org> *On Behalf Of *Simon
> Peyton Jones via ghc-devs
> *Sent:* 23 October 2019 17:22
> *To:* ghc-devs <ghc-devs at haskell.org>
> *Subject:* Urgent: git problem
>
>
>
> Aieee!   All my GHC repos are failing with this.  As a result I can’t
> pull.  What should I do?   Thanks!
>
> Simon
>
>
>
> git pull
>
> error: cannot lock ref 'refs/remotes/origin/wip/rae/remove-tc-dep': 'refs/remotes/origin/wip/rae'
> exists; cannot create 'refs/remotes/origin/wip/rae/remove-tc-dep'
>
> From gitlab.haskell.org:ghc/ghc
>
> ! [new branch]            wip/rae/remove-tc-dep    ->
> origin/wip/rae/remove-tc-dep  (unable to update local ref)
>
> error: cannot lock ref 'refs/remotes/origin/wip/rae/split-up-modules':
> 'refs/remotes/origin/wip/rae' exists; cannot create
> 'refs/remotes/origin/wip/rae/split-up-modules'
>
> ! [new branch]            wip/rae/split-up-modules ->
> origin/wip/rae/split-up-modules  (unable to update local ref)
>
> simonpj at MSRC-3645512:~/code/HEAD-2$
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191023/216dc48c/attachment.html>

From b at chreekat.net  Wed Oct 23 16:37:47 2019
From: b at chreekat.net (Bryan Richter)
Date: Wed, 23 Oct 2019 18:37:47 +0200
Subject: Urgent: git problem
In-Reply-To: <CAGRp5Rn8i_fyB3v0eWh=g_7O7SqvfUUrukUwcLPN6NOMb99jxQ@mail.gmail.com>
References: <AM6PR83MB02793C8CD7E9E9C67E553D33AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAGRp5Rn8i_fyB3v0eWh=g_7O7SqvfUUrukUwcLPN6NOMb99jxQ@mail.gmail.com>
Message-ID: <CAH9BU=zCpVMk-tb_ePJAQM4hY1eXEUwubsY+3NNFWdSvmrj7sg@mail.gmail.com>

Ah, good to know there's a command *just* for pruning! I'd only ever seen
the --prune flag to fetch/pull.

On Wed, 23 Oct 2019, 18.30 Oliver Charles, <ollie at ocharles.org.uk> wrote:

> I usually encounter this when there has once been a branch called
> "wip/rae" (so .git would have a *file* wip/rae), but now there's a
> branch called "wip/rae/remove-tc-dep", so "wip/rae" should be a
> directory, but it's already in .git as a file! I think "git remote
> prune" fixes this, but I can't entirely recall.
>
> On Wed, Oct 23, 2019 at 5:22 PM Simon Peyton Jones via ghc-devs
> <ghc-devs at haskell.org> wrote:
> >
> > Aieee!   All my GHC repos are failing with this.  As a result I can’t
> pull.  What should I do?   Thanks!
> >
> > Simon
> >
> >
> >
> > git pull
> >
> > error: cannot lock ref 'refs/remotes/origin/wip/rae/remove-tc-dep':
> 'refs/remotes/origin/wip/rae' exists; cannot create
> 'refs/remotes/origin/wip/rae/remove-tc-dep'
> >
> > From gitlab.haskell.org:ghc/ghc
> >
> > ! [new branch]            wip/rae/remove-tc-dep    ->
> origin/wip/rae/remove-tc-dep  (unable to update local ref)
> >
> > error: cannot lock ref 'refs/remotes/origin/wip/rae/split-up-modules':
> 'refs/remotes/origin/wip/rae' exists; cannot create
> 'refs/remotes/origin/wip/rae/split-up-modules'
> >
> > ! [new branch]            wip/rae/split-up-modules ->
> origin/wip/rae/split-up-modules  (unable to update local ref)
> >
> > simonpj at MSRC-3645512:~/code/HEAD-2$
> >
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191023/8a0951dc/attachment.html>

From simonpj at microsoft.com  Wed Oct 23 16:42:32 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Wed, 23 Oct 2019 16:42:32 +0000
Subject: Urgent: git problem
In-Reply-To: <CAH9BU=yktY==PRx1n4ZZkgadH_+MwA+iZb+8_wz5qHS_Ji7+kQ@mail.gmail.com>
References: <AM6PR83MB02793C8CD7E9E9C67E553D33AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <AM6PR83MB0279633AA3CF6715A506B1B8AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <CAH9BU=yktY==PRx1n4ZZkgadH_+MwA+iZb+8_wz5qHS_Ji7+kQ@mail.gmail.com>
Message-ID: <AM6PR83MB02793ABE8DA77D3F7A36BD95AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>

Thanks everyone!

From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Bryan Richter
Sent: 23 October 2019 17:36
To: GHC developers <ghc-devs at haskell.org>
Subject: Re: Urgent: git problem

I suspect a git fetch --prune might help. It looks like git's mapping of remote branches to a directory tree has caused a conflict between new and stale branches: ../rae/... can't be a directory if it's already a file!

If that still doesn't work (if it tries to record new branches before deleting stale ones), you'd have to muck around in .git/refs/remotes and remove the files manually.
On Wed, 23 Oct 2019, 18.29 Simon Peyton Jones via ghc-devs, <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>> wrote:
a completely fresh clone seems ok.  but that’s a bit brutal

From: ghc-devs <ghc-devs-bounces at haskell.org<mailto:ghc-devs-bounces at haskell.org>> On Behalf Of Simon Peyton Jones via ghc-devs
Sent: 23 October 2019 17:22
To: ghc-devs <ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>>
Subject: Urgent: git problem

Aieee!   All my GHC repos are failing with this.  As a result I can’t pull.  What should I do?   Thanks!
Simon


git pull

error: cannot lock ref 'refs/remotes/origin/wip/rae/remove-tc-dep': 'refs/remotes/origin/wip/rae' exists; cannot create 'refs/remotes/origin/wip/rae/remove-tc-dep'

From gitlab.haskell.org:ghc/ghc

! [new branch]            wip/rae/remove-tc-dep    -> origin/wip/rae/remove-tc-dep  (unable to update local ref)

error: cannot lock ref 'refs/remotes/origin/wip/rae/split-up-modules': 'refs/remotes/origin/wip/rae' exists; cannot create 'refs/remotes/origin/wip/rae/split-up-modules'

! [new branch]            wip/rae/split-up-modules -> origin/wip/rae/split-up-modules  (unable to update local ref)

simonpj at MSRC-3645512:~/code/HEAD-2$
_______________________________________________
ghc-devs mailing list
ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191023/a53272bb/attachment.html>

From gergo at erdi.hu  Wed Oct 23 16:36:15 2019
From: gergo at erdi.hu (Dr. ERDI Gergo)
Date: Wed, 23 Oct 2019 18:36:15 +0200 (CEST)
Subject: Urgent: git problem
In-Reply-To: <AM6PR83MB02793C8CD7E9E9C67E553D33AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <AM6PR83MB02793C8CD7E9E9C67E553D33AD6B0@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <alpine.DEB.2.21.1910231835460.27104@galaxy>

On Wed, 23 Oct 2019, Simon Peyton Jones via ghc-devs wrote:

> 
> Aieee!   All my GHC repos are failing with this.  As a result I can’t pull.  What
> should I do?   Thanks!

git remote prune origin

From klebinger.andreas at gmx.at  Thu Oct 24 11:48:29 2019
From: klebinger.andreas at gmx.at (Andreas Klebinger)
Date: Thu, 24 Oct 2019 13:48:29 +0200
Subject: How to navigate around the source tree?
In-Reply-To: <CAH9BU=zXro4sU4-jX1VHiH6oekH=ALQ5MsHEHf-TvPorgCfWTQ@mail.gmail.com>
References: <CALuQ0m_xgHRUJzBRrOWFOJQPd17akc-Rmpe9PimqiQg=-SPvpQ@mail.gmail.com>
 <929a4765-5e5a-afe4-c360-e482c94bc276@haskus.fr>
 <CALuQ0m-N0A1=jjsFRLt_Q=Mgn9mkLTUFNxLOWXNbFxfOxkBe3g@mail.gmail.com>
 <CAAS+=P_phjciJc_O+6LWUNY2556HfXk=X80QHA+EShWLXgSmWA@mail.gmail.com>
 <CAMQQO3=p__dWw+ewfLZkvnrnXqeaCCiqFKyKBKQBSFHwPNW3Cg@mail.gmail.com>
 <CALuQ0m8YSePBgndc1V=i-GLHEZuP6O4YL2_+U6q1Vk6mzDmqSA@mail.gmail.com>
 <CAH9BU=zXro4sU4-jX1VHiH6oekH=ALQ5MsHEHf-TvPorgCfWTQ@mail.gmail.com>
Message-ID: <137fbe4b-72b4-dd05-9da2-f29281f600fb@gmx.at>

Hello devs,

I also often jump to files. In my case usually using VS Code using Ctr+P
as well which searches for files by name.
While I can check which folder a file is in in the case of duplicates it
is a overhead which this refactor forces onto me.

While there are workarounds, both for my case as for Matts. It's worth
asking if requiring these workarounds is better
than just accepting redundant prefixes on module names.

Personally I would prefer unique file names even at the cost of redundancy.
I rarely add import statements/full module names, but I *very* often
jump to files.

Cheers
Andreas

Bryan Richter schrieb am 23.10.2019 um 18:00:
> Duplicate record fields is going to make this a bigger problem. Vim
> does support duplicate tags (:tselect and :tjump and related
> bindings), but hopefully haskell-ide-engine will one day provide us
> with semantic tags and solve this problem once and for all!
>
> On Wed, 23 Oct 2019, 17.49 Matthew Pickering,
> <matthewtpickering at gmail.com <mailto:matthewtpickering at gmail.com>> wrote:
>
>     Thanks Omer, Sylvain and Sebastian
>
>     .
>
>     I just configured my editor to use fzf and now I can use the `:GFiles`
>     command to perform fuzzy search on files which is probably better than
>     tags. If anyone else is using NixOS, all I had to do was add the
>     `fzf-vim` plugin to the vim configuration.
>
>     Cheers,
>
>     Matt
>
>     On Wed, Oct 23, 2019 at 2:54 PM Ömer Sinan Ağacan
>     <omeragacan at gmail.com <mailto:omeragacan at gmail.com>> wrote:
>     >
>     > I use a file finder (fzf) for jumping to files. Because module
>     names follow file
>     > paths to jump to e.g. StgToCmmUtils.Utils I usually type
>     `<C-p>stgcmmutils` and
>     > fzf finds the correct file `compiler/GHC/StgToCmm/Utils.hs`.
>     >
>     > When generating tags I omit module names for this reason, it's
>     easy with a good
>     > file finder to jump to modules already, no need to generate tags
>     for the
>     > modules.
>     >
>     > fast-tags commands I use:
>     >
>     > - When working on the compiler:
>     >
>     >   $ fast-tags --no-module-tags driver ghc compiler
>     >
>     > - When working on the RTS:
>     >
>     >   $ fast-tags --no-module-tags driver ghc compiler
>     >   $ ctags --append -R rts/**/*.c rts/**/*.h includes/**/*.h
>     >
>     > - When working on the libraries:
>     >
>     >   $ fast-tags --no-module-tags driver ghc compiler libraries
>     >
>     > Ömer
>     >
>     > Sebastian Graf <sgraf1337 at gmail.com
>     <mailto:sgraf1337 at gmail.com>>, 23 Eki 2019 Çar, 16:49 tarihinde
>     > şunu yazdı:
>     > >
>     > > FWIW, I'm using VSCode's fuzzy file search with Ctrl+P (and
>     vim's equivalent) rather successfully. Just tried it for
>     Hs/Utils.hs by typing 'hsutils.hs'. It didn't turn up as the first
>     result in VSCode, but it in vim.
>     > >
>     > > Am Mi., 23. Okt. 2019 um 14:27 Uhr schrieb Matthew Pickering
>     <matthewtpickering at gmail.com <mailto:matthewtpickering at gmail.com>>:
>     > >>
>     > >> I use `fast-tags` which doesn't look at the hierarchy at all
>     and I'm
>     > >> not sure what the improvement would be as the names of the
>     modules
>     > >> would still clash.
>     > >>
>     > >> If there is some other recommended way to jump to a module
>     then that
>     > >> would also work for me.
>     > >>
>     > >> Matt
>     > >>
>     > >>
>     > >> On Wed, Oct 23, 2019 at 12:08 PM Sylvain Henry
>     <sylvain at haskus.fr <mailto:sylvain at haskus.fr>> wrote:
>     > >> >
>     > >> > Hi,
>     > >> >
>     > >> > How do you generate your tags file? It seems to be a
>     shortcoming of the
>     > >> > generator to not take into account the location of the
>     definition file.
>     > >> >
>     > >> >  > Perhaps `HsUtils` and `StgUtils` would be appropriate to
>     > >> > disambiguate`Hs/Utils` and `StgToCmm/Utils`.
>     > >> >
>     > >> > We are promoting the module prefixes (`Hs`, `Stg`, `Tc`,
>     etc.) into
>     > >> > proper module layers (e.g. `HsUtils` becomes
>     `GHC.Hs.Utils`) so it would
>     > >> > be redundant to add the prefixes back. :/
>     > >> >
>     > >> > Cheers,
>     > >> > Sylvain
>     > >> >
>     > >> > On 23/10/2019 12:52, Matthew Pickering wrote:
>     > >> > > Hi,
>     > >> > >
>     > >> > > The module rework has broken my workflow.
>     > >> > >
>     > >> > > Now my tags file is useless for jumping for modules as
>     there are
>     > >> > > multiple "Utils" and "Types" modules. Invariable I am
>     jumping to the
>     > >> > > wrong one. What do other people do to avoid this?
>     > >> > >
>     > >> > > Can we either revert these changes or give these modules
>     unique names
>     > >> > > to facilitate that only reliable way of navigating the
>     code base.
>     > >> > > Perhaps `HsUtils` and `StgUtils` would be appropriate to
>     disambiguate
>     > >> > > `Hs/Utils` and `StgToCmm/Utils`.
>     > >> > >
>     > >> > > Cheers,
>     > >> > >
>     > >> > > Matt
>     > >> > > _______________________________________________
>     > >> > > ghc-devs mailing list
>     > >> > > ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
>     > >> > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>     > >> > _______________________________________________
>     > >> > ghc-devs mailing list
>     > >> > ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
>     > >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>     > >> _______________________________________________
>     > >> ghc-devs mailing list
>     > >> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
>     > >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>     > >
>     > > _______________________________________________
>     > > ghc-devs mailing list
>     > > ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
>     > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>     _______________________________________________
>     ghc-devs mailing list
>     ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
>     http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191024/0691f799/attachment.html>

From lexi.lambda at gmail.com  Thu Oct 24 17:13:14 2019
From: lexi.lambda at gmail.com (Alexis King)
Date: Thu, 24 Oct 2019 12:13:14 -0500
Subject: Debugging specialization
Message-ID: <51F7FBD8-7090-460A-9A61-1AB3FDDD7872@gmail.com>

Hi all,

I am trying to understand why GHC is not specializing an imported INLINABLE instance method, and the information provided by -ddump-spec -Wall-missed-specialisations has not been enough to help me figure it out. Is there some easy/well-trodden way that I could build GHC with some additional instrumentation in place to help me better understand the decisions being made by the specializer, or is -ddump-spec the most granularity available?

I’ve noticed that Specialise.hs has handful of pprTrace calls sprinkled about, but they are all commented out. Is the recommended method to just uncomment these calls and rebuild GHC?

Thanks,
Alexis


From kili at outback.escape.de  Thu Oct 24 19:58:26 2019
From: kili at outback.escape.de (Matthias Kilian)
Date: Thu, 24 Oct 2019 21:58:26 +0200
Subject: ghc, rts/Linker.c, OpenBSD and WXNEEDED and NOPIE
Message-ID: <20191024195826.GA32023@nutty.outback.escape.de>

Hi,

tl;dr: with shared / dynamic libs, is anything from rts/Linker.c
still in use? If so, where and why?

I hope this hasn't already been discussed in the past, and I apolotzie
if all this is no longer relevant for ghc-8.8 or newer (I'm still
at 8.6)...

On OpenBSD, I had to explicitely mark ghc as WXNEEDED (i.e. allow
it to map pages both as writable and executable), which I would
like to get rid of. Likewise, I had to do some dances to explicitely
disable PIE. If my memory doesn't fail completely, most (if not
all) oft his was because of rts/Linker.c mmapping objects to be
loaded at runtime and kind of simulating what dlopen() does for
shared objects.

Recently, thanks to Greg Steuck (who helped me a lot with this),
we updated our ghc port to ghc-8.6 and also enabled shared (or
dynamic, if you prefer this term) libraries.

So my hope was that the code in rts/Linker.c were no longer used
(because dynamic loading as done by ghci or template Haskell) could
be done with dlopen().

However, after building ghc-8.6 with all the WXNNEDED stuff removed,
and running ghci from this build, I still get an abort because pages
are mapped both writable and executable (I've configured my system
to abort processes when a W^X violation happens). Here's a backtrace
in case it's helpfull at all:

#0  _thread_sys___syscall () at -:3
#1  0x00000002010bb179 in _libc_mmap (addr=Variable "addr" is not available.) at /usr/src/lib/libc/sys/mmap.c:47
#2  0x000000021be22374 in mmapForLinker (bytes=Variable "bytes" is not available.) at rts/Linker.c:1029
#3  0x000000021be3e99b in m32_allocator_init () at rts/linker/M32Alloc.c:160
#4  0x00000002fd518be1 in cimc_info () from /usr/local/lib/ghc/bin/../ghci-8.6.4/libHSghci-8.6.4-ghc8.6.4.so
#5  0x0000000000000000 in ?? ()

Any thougts about this?

And, since W|X (and nopie) isn't good on any system: is anyone
working on getting rid of this for other unixes?

Thanks in advance for any help.

Ciao,
	Kili

From alan.zimm at gmail.com  Thu Oct 24 21:42:49 2019
From: alan.zimm at gmail.com (Alan & Kim Zimmerman)
Date: Thu, 24 Oct 2019 22:42:49 +0100
Subject: How to navigate around the source tree?
In-Reply-To: <137fbe4b-72b4-dd05-9da2-f29281f600fb@gmx.at>
References: <CALuQ0m_xgHRUJzBRrOWFOJQPd17akc-Rmpe9PimqiQg=-SPvpQ@mail.gmail.com>
 <929a4765-5e5a-afe4-c360-e482c94bc276@haskus.fr>
 <CALuQ0m-N0A1=jjsFRLt_Q=Mgn9mkLTUFNxLOWXNbFxfOxkBe3g@mail.gmail.com>
 <CAAS+=P_phjciJc_O+6LWUNY2556HfXk=X80QHA+EShWLXgSmWA@mail.gmail.com>
 <CAMQQO3=p__dWw+ewfLZkvnrnXqeaCCiqFKyKBKQBSFHwPNW3Cg@mail.gmail.com>
 <CALuQ0m8YSePBgndc1V=i-GLHEZuP6O4YL2_+U6q1Vk6mzDmqSA@mail.gmail.com>
 <CAH9BU=zXro4sU4-jX1VHiH6oekH=ALQ5MsHEHf-TvPorgCfWTQ@mail.gmail.com>
 <137fbe4b-72b4-dd05-9da2-f29281f600fb@gmx.at>
Message-ID: <CANma=H-rJy4k=ci9p41ph8o0E5NU+t+0ox7q-9wcGNatqptmRA@mail.gmail.com>

I use `hasktags -b .` in the compiler directory, and it generates a `tags`
file with lines in it like

```
tags\021080:mkHsPar ./GHC/Hs/Utils.hs 142
tags\021081:mkHsParPV ./parser/RdrHsSyn.hs 1920
tags\021082:mkHsParPV ./parser/RdrHsSyn.hs 2019
tags\021083:mkHsParPV ./parser/RdrHsSyn.hs 2072
tags\021084:mkHsParPV ./parser/RdrHsSyn.hs 2154
```

This gives the full path, and emacs is happy.
And of course in this example mkHsParPV does in fact have multiple
definitions, it is a class method.

Alan

On Thu, 24 Oct 2019 at 12:48, Andreas Klebinger <klebinger.andreas at gmx.at>
wrote:

> Hello devs,
>
> I also often jump to files. In my case usually using VS Code using Ctr+P
> as well which searches for files by name.
> While I can check which folder a file is in in the case of duplicates it
> is a overhead which this refactor forces onto me.
>
> While there are workarounds, both for my case as for Matts. It's worth
> asking if requiring these workarounds is better
> than just accepting redundant prefixes on module names.
>
> Personally I would prefer unique file names even at the cost of redundancy.
> I rarely add import statements/full module names, but I *very* often jump
> to files.
>
> Cheers
> Andreas
>
> Bryan Richter schrieb am 23.10.2019 um 18:00:
>
> Duplicate record fields is going to make this a bigger problem. Vim does
> support duplicate tags (:tselect and :tjump and related bindings), but
> hopefully haskell-ide-engine will one day provide us with semantic tags and
> solve this problem once and for all!
>
> On Wed, 23 Oct 2019, 17.49 Matthew Pickering, <matthewtpickering at gmail.com>
> wrote:
>
>> Thanks Omer, Sylvain and Sebastian
>>
>> .
>>
>> I just configured my editor to use fzf and now I can use the `:GFiles`
>> command to perform fuzzy search on files which is probably better than
>> tags. If anyone else is using NixOS, all I had to do was add the
>> `fzf-vim` plugin to the vim configuration.
>>
>> Cheers,
>>
>> Matt
>>
>> On Wed, Oct 23, 2019 at 2:54 PM Ömer Sinan Ağacan <omeragacan at gmail.com>
>> wrote:
>> >
>> > I use a file finder (fzf) for jumping to files. Because module names
>> follow file
>> > paths to jump to e.g. StgToCmmUtils.Utils I usually type
>> `<C-p>stgcmmutils` and
>> > fzf finds the correct file `compiler/GHC/StgToCmm/Utils.hs`.
>> >
>> > When generating tags I omit module names for this reason, it's easy
>> with a good
>> > file finder to jump to modules already, no need to generate tags for the
>> > modules.
>> >
>> > fast-tags commands I use:
>> >
>> > - When working on the compiler:
>> >
>> >   $ fast-tags --no-module-tags driver ghc compiler
>> >
>> > - When working on the RTS:
>> >
>> >   $ fast-tags --no-module-tags driver ghc compiler
>> >   $ ctags --append -R rts/**/*.c rts/**/*.h includes/**/*.h
>> >
>> > - When working on the libraries:
>> >
>> >   $ fast-tags --no-module-tags driver ghc compiler libraries
>> >
>> > Ömer
>> >
>> > Sebastian Graf <sgraf1337 at gmail.com>, 23 Eki 2019 Çar, 16:49 tarihinde
>> > şunu yazdı:
>> > >
>> > > FWIW, I'm using VSCode's fuzzy file search with Ctrl+P (and vim's
>> equivalent) rather successfully. Just tried it for Hs/Utils.hs by typing
>> 'hsutils.hs'. It didn't turn up as the first result in VSCode, but it in
>> vim.
>> > >
>> > > Am Mi., 23. Okt. 2019 um 14:27 Uhr schrieb Matthew Pickering <
>> matthewtpickering at gmail.com>:
>> > >>
>> > >> I use `fast-tags` which doesn't look at the hierarchy at all and I'm
>> > >> not sure what the improvement would be as the names of the modules
>> > >> would still clash.
>> > >>
>> > >> If there is some other recommended way to jump to a module then that
>> > >> would also work for me.
>> > >>
>> > >> Matt
>> > >>
>> > >>
>> > >> On Wed, Oct 23, 2019 at 12:08 PM Sylvain Henry <sylvain at haskus.fr>
>> wrote:
>> > >> >
>> > >> > Hi,
>> > >> >
>> > >> > How do you generate your tags file? It seems to be a shortcoming
>> of the
>> > >> > generator to not take into account the location of the definition
>> file.
>> > >> >
>> > >> >  > Perhaps `HsUtils` and `StgUtils` would be appropriate to
>> > >> > disambiguate`Hs/Utils` and `StgToCmm/Utils`.
>> > >> >
>> > >> > We are promoting the module prefixes (`Hs`, `Stg`, `Tc`, etc.) into
>> > >> > proper module layers (e.g. `HsUtils` becomes `GHC.Hs.Utils`) so it
>> would
>> > >> > be redundant to add the prefixes back. :/
>> > >> >
>> > >> > Cheers,
>> > >> > Sylvain
>> > >> >
>> > >> > On 23/10/2019 12:52, Matthew Pickering wrote:
>> > >> > > Hi,
>> > >> > >
>> > >> > > The module rework has broken my workflow.
>> > >> > >
>> > >> > > Now my tags file is useless for jumping for modules as there are
>> > >> > > multiple "Utils" and "Types" modules. Invariable I am jumping to
>> the
>> > >> > > wrong one. What do other people do to avoid this?
>> > >> > >
>> > >> > > Can we either revert these changes or give these modules unique
>> names
>> > >> > > to facilitate that only reliable way of navigating the code base.
>> > >> > > Perhaps `HsUtils` and `StgUtils` would be appropriate to
>> disambiguate
>> > >> > > `Hs/Utils` and `StgToCmm/Utils`.
>> > >> > >
>> > >> > > Cheers,
>> > >> > >
>> > >> > > Matt
>> > >> > > _______________________________________________
>> > >> > > ghc-devs mailing list
>> > >> > > ghc-devs at haskell.org
>> > >> > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> > >> > _______________________________________________
>> > >> > ghc-devs mailing list
>> > >> > ghc-devs at haskell.org
>> > >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> > >> _______________________________________________
>> > >> ghc-devs mailing list
>> > >> ghc-devs at haskell.org
>> > >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> > >
>> > > _______________________________________________
>> > > ghc-devs mailing list
>> > > ghc-devs at haskell.org
>> > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
>
> _______________________________________________
> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191024/d9176285/attachment.html>

From omeragacan at gmail.com  Fri Oct 25 06:43:52 2019
From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=)
Date: Fri, 25 Oct 2019 09:43:52 +0300
Subject: How to navigate around the source tree?
In-Reply-To: <137fbe4b-72b4-dd05-9da2-f29281f600fb@gmx.at>
References: <CALuQ0m_xgHRUJzBRrOWFOJQPd17akc-Rmpe9PimqiQg=-SPvpQ@mail.gmail.com>
 <929a4765-5e5a-afe4-c360-e482c94bc276@haskus.fr>
 <CALuQ0m-N0A1=jjsFRLt_Q=Mgn9mkLTUFNxLOWXNbFxfOxkBe3g@mail.gmail.com>
 <CAAS+=P_phjciJc_O+6LWUNY2556HfXk=X80QHA+EShWLXgSmWA@mail.gmail.com>
 <CAMQQO3=p__dWw+ewfLZkvnrnXqeaCCiqFKyKBKQBSFHwPNW3Cg@mail.gmail.com>
 <CALuQ0m8YSePBgndc1V=i-GLHEZuP6O4YL2_+U6q1Vk6mzDmqSA@mail.gmail.com>
 <CAH9BU=zXro4sU4-jX1VHiH6oekH=ALQ5MsHEHf-TvPorgCfWTQ@mail.gmail.com>
 <137fbe4b-72b4-dd05-9da2-f29281f600fb@gmx.at>
Message-ID: <CAMQQO3kxcAE1HOKJ24z85iMs7QzQCndwm7Kc1icVjgXAuaW6DA@mail.gmail.com>

> While there are workarounds, both for my case as for Matts. It's worth asking
> if requiring these workarounds is better than just accepting redundant
> prefixes on module names.

"Workarounds" are for problems, but I don't understand why duplicate file names
are a problem. Can you elaborate? Perhaps this is problem with your IDE/editor
setup? Many of us (as can be seen in responses) use tools/editors/IDEs that can
handle this just fine.

I worked on projects with duplicate file names in the past too; having tools
that can deal with this helps, and I don't think this is too hard to achieve.
You can't ask devs of every project you join to rename their files because your
editor can't handle them.

(I don't know VS Code enough to help ..)

Ömer

Andreas Klebinger <klebinger.andreas at gmx.at>, 24 Eki 2019 Per, 14:48
tarihinde şunu yazdı:
>
> Hello devs,
>
> I also often jump to files. In my case usually using VS Code using Ctr+P as well which searches for files by name.
> While I can check which folder a file is in in the case of duplicates it is a overhead which this refactor forces onto me.
>
> While there are workarounds, both for my case as for Matts. It's worth asking if requiring these workarounds is better
> than just accepting redundant prefixes on module names.
>
> Personally I would prefer unique file names even at the cost of redundancy.
> I rarely add import statements/full module names, but I *very* often jump to files.
>
> Cheers
> Andreas
>
> Bryan Richter schrieb am 23.10.2019 um 18:00:
>
> Duplicate record fields is going to make this a bigger problem. Vim does support duplicate tags (:tselect and :tjump and related bindings), but hopefully haskell-ide-engine will one day provide us with semantic tags and solve this problem once and for all!
>
> On Wed, 23 Oct 2019, 17.49 Matthew Pickering, <matthewtpickering at gmail.com> wrote:
>>
>> Thanks Omer, Sylvain and Sebastian
>>
>> .
>>
>> I just configured my editor to use fzf and now I can use the `:GFiles`
>> command to perform fuzzy search on files which is probably better than
>> tags. If anyone else is using NixOS, all I had to do was add the
>> `fzf-vim` plugin to the vim configuration.
>>
>> Cheers,
>>
>> Matt
>>
>> On Wed, Oct 23, 2019 at 2:54 PM Ömer Sinan Ağacan <omeragacan at gmail.com> wrote:
>> >
>> > I use a file finder (fzf) for jumping to files. Because module names follow file
>> > paths to jump to e.g. StgToCmmUtils.Utils I usually type `<C-p>stgcmmutils` and
>> > fzf finds the correct file `compiler/GHC/StgToCmm/Utils.hs`.
>> >
>> > When generating tags I omit module names for this reason, it's easy with a good
>> > file finder to jump to modules already, no need to generate tags for the
>> > modules.
>> >
>> > fast-tags commands I use:
>> >
>> > - When working on the compiler:
>> >
>> >   $ fast-tags --no-module-tags driver ghc compiler
>> >
>> > - When working on the RTS:
>> >
>> >   $ fast-tags --no-module-tags driver ghc compiler
>> >   $ ctags --append -R rts/**/*.c rts/**/*.h includes/**/*.h
>> >
>> > - When working on the libraries:
>> >
>> >   $ fast-tags --no-module-tags driver ghc compiler libraries
>> >
>> > Ömer
>> >
>> > Sebastian Graf <sgraf1337 at gmail.com>, 23 Eki 2019 Çar, 16:49 tarihinde
>> > şunu yazdı:
>> > >
>> > > FWIW, I'm using VSCode's fuzzy file search with Ctrl+P (and vim's equivalent) rather successfully. Just tried it for Hs/Utils.hs by typing 'hsutils.hs'. It didn't turn up as the first result in VSCode, but it in vim.
>> > >
>> > > Am Mi., 23. Okt. 2019 um 14:27 Uhr schrieb Matthew Pickering <matthewtpickering at gmail.com>:
>> > >>
>> > >> I use `fast-tags` which doesn't look at the hierarchy at all and I'm
>> > >> not sure what the improvement would be as the names of the modules
>> > >> would still clash.
>> > >>
>> > >> If there is some other recommended way to jump to a module then that
>> > >> would also work for me.
>> > >>
>> > >> Matt
>> > >>
>> > >>
>> > >> On Wed, Oct 23, 2019 at 12:08 PM Sylvain Henry <sylvain at haskus.fr> wrote:
>> > >> >
>> > >> > Hi,
>> > >> >
>> > >> > How do you generate your tags file? It seems to be a shortcoming of the
>> > >> > generator to not take into account the location of the definition file.
>> > >> >
>> > >> >  > Perhaps `HsUtils` and `StgUtils` would be appropriate to
>> > >> > disambiguate`Hs/Utils` and `StgToCmm/Utils`.
>> > >> >
>> > >> > We are promoting the module prefixes (`Hs`, `Stg`, `Tc`, etc.) into
>> > >> > proper module layers (e.g. `HsUtils` becomes `GHC.Hs.Utils`) so it would
>> > >> > be redundant to add the prefixes back. :/
>> > >> >
>> > >> > Cheers,
>> > >> > Sylvain
>> > >> >
>> > >> > On 23/10/2019 12:52, Matthew Pickering wrote:
>> > >> > > Hi,
>> > >> > >
>> > >> > > The module rework has broken my workflow.
>> > >> > >
>> > >> > > Now my tags file is useless for jumping for modules as there are
>> > >> > > multiple "Utils" and "Types" modules. Invariable I am jumping to the
>> > >> > > wrong one. What do other people do to avoid this?
>> > >> > >
>> > >> > > Can we either revert these changes or give these modules unique names
>> > >> > > to facilitate that only reliable way of navigating the code base.
>> > >> > > Perhaps `HsUtils` and `StgUtils` would be appropriate to disambiguate
>> > >> > > `Hs/Utils` and `StgToCmm/Utils`.
>> > >> > >
>> > >> > > Cheers,
>> > >> > >
>> > >> > > Matt
>> > >> > > _______________________________________________
>> > >> > > ghc-devs mailing list
>> > >> > > ghc-devs at haskell.org
>> > >> > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> > >> > _______________________________________________
>> > >> > ghc-devs mailing list
>> > >> > ghc-devs at haskell.org
>> > >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> > >> _______________________________________________
>> > >> ghc-devs mailing list
>> > >> ghc-devs at haskell.org
>> > >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> > >
>> > > _______________________________________________
>> > > ghc-devs mailing list
>> > > ghc-devs at haskell.org
>> > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From simonpj at microsoft.com  Fri Oct 25 07:59:41 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Fri, 25 Oct 2019 07:59:41 +0000
Subject: How to navigate around the source tree?
In-Reply-To: <CAMQQO3kxcAE1HOKJ24z85iMs7QzQCndwm7Kc1icVjgXAuaW6DA@mail.gmail.com>
References: <CALuQ0m_xgHRUJzBRrOWFOJQPd17akc-Rmpe9PimqiQg=-SPvpQ@mail.gmail.com>
 <929a4765-5e5a-afe4-c360-e482c94bc276@haskus.fr>
 <CALuQ0m-N0A1=jjsFRLt_Q=Mgn9mkLTUFNxLOWXNbFxfOxkBe3g@mail.gmail.com>
 <CAAS+=P_phjciJc_O+6LWUNY2556HfXk=X80QHA+EShWLXgSmWA@mail.gmail.com>
 <CAMQQO3=p__dWw+ewfLZkvnrnXqeaCCiqFKyKBKQBSFHwPNW3Cg@mail.gmail.com>
 <CALuQ0m8YSePBgndc1V=i-GLHEZuP6O4YL2_+U6q1Vk6mzDmqSA@mail.gmail.com>
 <CAH9BU=zXro4sU4-jX1VHiH6oekH=ALQ5MsHEHf-TvPorgCfWTQ@mail.gmail.com>
 <137fbe4b-72b4-dd05-9da2-f29281f600fb@gmx.at>
 <CAMQQO3kxcAE1HOKJ24z85iMs7QzQCndwm7Kc1icVjgXAuaW6DA@mail.gmail.com>
Message-ID: <AM6PR83MB027947C1E542FFAA24EAE343AD650@AM6PR83MB0279.EURPRD83.prod.outlook.com>

I think there is at least something to be said for module names that are unique, mainly for the human interface.

For example, I often say (in Notes or Gitlab comments) "In TcType.foo we see ...".   Now I could say "In GHC.Hs.Typecheck.Type we see ..." but it's less convenient.  I could take the shortest unique prefix "In Typecheck.Type we see ...", but then I need to figure out what name clashes there are and add a disambiguating prefix.

This is not a do-or-die issue. But my instinct is to avoid name clashes where we can.

Simon

|  -----Original Message-----
|  From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Ömer Sinan
|  Agacan
|  Sent: 25 October 2019 07:44
|  To: Andreas Klebinger <klebinger.andreas at gmx.at>
|  Cc: GHC developers <ghc-devs at haskell.org>
|  Subject: Re: How to navigate around the source tree?
|  
|  > While there are workarounds, both for my case as for Matts. It's worth
|  asking
|  > if requiring these workarounds is better than just accepting redundant
|  > prefixes on module names.
|  
|  "Workarounds" are for problems, but I don't understand why duplicate file
|  names
|  are a problem. Can you elaborate? Perhaps this is problem with your
|  IDE/editor
|  setup? Many of us (as can be seen in responses) use tools/editors/IDEs
|  that can
|  handle this just fine.
|  
|  I worked on projects with duplicate file names in the past too; having
|  tools
|  that can deal with this helps, and I don't think this is too hard to
|  achieve.
|  You can't ask devs of every project you join to rename their files because
|  your
|  editor can't handle them.
|  
|  (I don't know VS Code enough to help ..)
|  
|  Ömer
|  
|  Andreas Klebinger <klebinger.andreas at gmx.at>, 24 Eki 2019 Per, 14:48
|  tarihinde şunu yazdı:
|  >
|  > Hello devs,
|  >
|  > I also often jump to files. In my case usually using VS Code using Ctr+P
|  as well which searches for files by name.
|  > While I can check which folder a file is in in the case of duplicates it
|  is a overhead which this refactor forces onto me.
|  >
|  > While there are workarounds, both for my case as for Matts. It's worth
|  asking if requiring these workarounds is better
|  > than just accepting redundant prefixes on module names.
|  >
|  > Personally I would prefer unique file names even at the cost of
|  redundancy.
|  > I rarely add import statements/full module names, but I *very* often
|  jump to files.
|  >
|  > Cheers
|  > Andreas
|  >
|  > Bryan Richter schrieb am 23.10.2019 um 18:00:
|  >
|  > Duplicate record fields is going to make this a bigger problem. Vim does
|  support duplicate tags (:tselect and :tjump and related bindings), but
|  hopefully haskell-ide-engine will one day provide us with semantic tags
|  and solve this problem once and for all!
|  >
|  > On Wed, 23 Oct 2019, 17.49 Matthew Pickering,
|  <matthewtpickering at gmail.com> wrote:
|  >>
|  >> Thanks Omer, Sylvain and Sebastian
|  >>
|  >> .
|  >>
|  >> I just configured my editor to use fzf and now I can use the `:GFiles`
|  >> command to perform fuzzy search on files which is probably better than
|  >> tags. If anyone else is using NixOS, all I had to do was add the
|  >> `fzf-vim` plugin to the vim configuration.
|  >>
|  >> Cheers,
|  >>
|  >> Matt
|  >>
|  >> On Wed, Oct 23, 2019 at 2:54 PM Ömer Sinan Ağacan
|  <omeragacan at gmail.com> wrote:
|  >> >
|  >> > I use a file finder (fzf) for jumping to files. Because module names
|  follow file
|  >> > paths to jump to e.g. StgToCmmUtils.Utils I usually type `<C-
|  p>stgcmmutils` and
|  >> > fzf finds the correct file `compiler/GHC/StgToCmm/Utils.hs`.
|  >> >
|  >> > When generating tags I omit module names for this reason, it's easy
|  with a good
|  >> > file finder to jump to modules already, no need to generate tags for
|  the
|  >> > modules.
|  >> >
|  >> > fast-tags commands I use:
|  >> >
|  >> > - When working on the compiler:
|  >> >
|  >> >   $ fast-tags --no-module-tags driver ghc compiler
|  >> >
|  >> > - When working on the RTS:
|  >> >
|  >> >   $ fast-tags --no-module-tags driver ghc compiler
|  >> >   $ ctags --append -R rts/**/*.c rts/**/*.h includes/**/*.h
|  >> >
|  >> > - When working on the libraries:
|  >> >
|  >> >   $ fast-tags --no-module-tags driver ghc compiler libraries
|  >> >
|  >> > Ömer
|  >> >
|  >> > Sebastian Graf <sgraf1337 at gmail.com>, 23 Eki 2019 Çar, 16:49
|  tarihinde
|  >> > şunu yazdı:
|  >> > >
|  >> > > FWIW, I'm using VSCode's fuzzy file search with Ctrl+P (and vim's
|  equivalent) rather successfully. Just tried it for Hs/Utils.hs by typing
|  'hsutils.hs'. It didn't turn up as the first result in VSCode, but it in
|  vim.
|  >> > >
|  >> > > Am Mi., 23. Okt. 2019 um 14:27 Uhr schrieb Matthew Pickering
|  <matthewtpickering at gmail.com>:
|  >> > >>
|  >> > >> I use `fast-tags` which doesn't look at the hierarchy at all and
|  I'm
|  >> > >> not sure what the improvement would be as the names of the modules
|  >> > >> would still clash.
|  >> > >>
|  >> > >> If there is some other recommended way to jump to a module then
|  that
|  >> > >> would also work for me.
|  >> > >>
|  >> > >> Matt
|  >> > >>
|  >> > >>
|  >> > >> On Wed, Oct 23, 2019 at 12:08 PM Sylvain Henry <sylvain at haskus.fr>
|  wrote:
|  >> > >> >
|  >> > >> > Hi,
|  >> > >> >
|  >> > >> > How do you generate your tags file? It seems to be a shortcoming
|  of the
|  >> > >> > generator to not take into account the location of the
|  definition file.
|  >> > >> >
|  >> > >> >  > Perhaps `HsUtils` and `StgUtils` would be appropriate to
|  >> > >> > disambiguate`Hs/Utils` and `StgToCmm/Utils`.
|  >> > >> >
|  >> > >> > We are promoting the module prefixes (`Hs`, `Stg`, `Tc`, etc.)
|  into
|  >> > >> > proper module layers (e.g. `HsUtils` becomes `GHC.Hs.Utils`) so
|  it would
|  >> > >> > be redundant to add the prefixes back. :/
|  >> > >> >
|  >> > >> > Cheers,
|  >> > >> > Sylvain
|  >> > >> >
|  >> > >> > On 23/10/2019 12:52, Matthew Pickering wrote:
|  >> > >> > > Hi,
|  >> > >> > >
|  >> > >> > > The module rework has broken my workflow.
|  >> > >> > >
|  >> > >> > > Now my tags file is useless for jumping for modules as there
|  are
|  >> > >> > > multiple "Utils" and "Types" modules. Invariable I am jumping
|  to the
|  >> > >> > > wrong one. What do other people do to avoid this?
|  >> > >> > >
|  >> > >> > > Can we either revert these changes or give these modules
|  unique names
|  >> > >> > > to facilitate that only reliable way of navigating the code
|  base.
|  >> > >> > > Perhaps `HsUtils` and `StgUtils` would be appropriate to
|  disambiguate
|  >> > >> > > `Hs/Utils` and `StgToCmm/Utils`.
|  >> > >> > >
|  >> > >> > > Cheers,
|  >> > >> > >
|  >> > >> > > Matt
|  >> > >> > > _______________________________________________
|  >> > >> > > ghc-devs mailing list
|  >> > >> > > ghc-devs at haskell.org
|  >> > >> > >
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
|  ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C3ffb77db43b74c34a0af08d7
|  5916d1e4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637075826856171272&a
|  mp;sdata=H%2BCCWwdR%2Fv1rhnYi%2BBuFPvB5pzlFpHkqdsG6S0tVOfE%3D&amp;reserved
|  =0
|  >> > >> > _______________________________________________
|  >> > >> > ghc-devs mailing list
|  >> > >> > ghc-devs at haskell.org
|  >> > >> >
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
|  ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C3ffb77db43b74c34a0af08d7
|  5916d1e4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637075826856171272&a
|  mp;sdata=H%2BCCWwdR%2Fv1rhnYi%2BBuFPvB5pzlFpHkqdsG6S0tVOfE%3D&amp;reserved
|  =0
|  >> > >> _______________________________________________
|  >> > >> ghc-devs mailing list
|  >> > >> ghc-devs at haskell.org
|  >> > >>
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
|  ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C3ffb77db43b74c34a0af08d7
|  5916d1e4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637075826856171272&a
|  mp;sdata=H%2BCCWwdR%2Fv1rhnYi%2BBuFPvB5pzlFpHkqdsG6S0tVOfE%3D&amp;reserved
|  =0
|  >> > >
|  >> > > _______________________________________________
|  >> > > ghc-devs mailing list
|  >> > > ghc-devs at haskell.org
|  >> > >
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
|  ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C3ffb77db43b74c34a0af08d7
|  5916d1e4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637075826856181264&a
|  mp;sdata=qU2fbJithYT0tvxy299nYcjYPk265OxnKzJLRtb%2FpVI%3D&amp;reserved=0
|  >> _______________________________________________
|  >> ghc-devs mailing list
|  >> ghc-devs at haskell.org
|  >>
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
|  ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C3ffb77db43b74c34a0af08d7
|  5916d1e4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637075826856181264&a
|  mp;sdata=qU2fbJithYT0tvxy299nYcjYPk265OxnKzJLRtb%2FpVI%3D&amp;reserved=0
|  >
|  >
|  >
|  > _______________________________________________
|  > ghc-devs mailing list
|  > ghc-devs at haskell.org
|  >
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
|  ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C3ffb77db43b74c34a0af08d7
|  5916d1e4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637075826856181264&a
|  mp;sdata=qU2fbJithYT0tvxy299nYcjYPk265OxnKzJLRtb%2FpVI%3D&amp;reserved=0
|  >
|  >
|  > _______________________________________________
|  > ghc-devs mailing list
|  > ghc-devs at haskell.org
|  >
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
|  ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C3ffb77db43b74c34a0af08d7
|  5916d1e4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637075826856181264&a
|  mp;sdata=qU2fbJithYT0tvxy299nYcjYPk265OxnKzJLRtb%2FpVI%3D&amp;reserved=0
|  _______________________________________________
|  ghc-devs mailing list
|  ghc-devs at haskell.org
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
|  ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C3ffb77db43b74c34a0af08d7
|  5916d1e4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637075826856181264&a
|  mp;sdata=qU2fbJithYT0tvxy299nYcjYPk265OxnKzJLRtb%2FpVI%3D&amp;reserved=0

From klebinger.andreas at gmx.at  Fri Oct 25 08:56:22 2019
From: klebinger.andreas at gmx.at (Andreas Klebinger)
Date: Fri, 25 Oct 2019 10:56:22 +0200
Subject: How to navigate around the source tree?
In-Reply-To: <CAMQQO3kxcAE1HOKJ24z85iMs7QzQCndwm7Kc1icVjgXAuaW6DA@mail.gmail.com>
References: <CALuQ0m_xgHRUJzBRrOWFOJQPd17akc-Rmpe9PimqiQg=-SPvpQ@mail.gmail.com>
 <929a4765-5e5a-afe4-c360-e482c94bc276@haskus.fr>
 <CALuQ0m-N0A1=jjsFRLt_Q=Mgn9mkLTUFNxLOWXNbFxfOxkBe3g@mail.gmail.com>
 <CAAS+=P_phjciJc_O+6LWUNY2556HfXk=X80QHA+EShWLXgSmWA@mail.gmail.com>
 <CAMQQO3=p__dWw+ewfLZkvnrnXqeaCCiqFKyKBKQBSFHwPNW3Cg@mail.gmail.com>
 <CALuQ0m8YSePBgndc1V=i-GLHEZuP6O4YL2_+U6q1Vk6mzDmqSA@mail.gmail.com>
 <CAH9BU=zXro4sU4-jX1VHiH6oekH=ALQ5MsHEHf-TvPorgCfWTQ@mail.gmail.com>
 <137fbe4b-72b4-dd05-9da2-f29281f600fb@gmx.at>
 <CAMQQO3kxcAE1HOKJ24z85iMs7QzQCndwm7Kc1icVjgXAuaW6DA@mail.gmail.com>
Message-ID: <ab4118db-b0f0-1c2b-141a-88127047b954@gmx.at>


> "Workarounds" are for problems, but I don't understand why duplicate file names
> are a problem. Can you elaborate? Perhaps this is problem with your IDE/editor
> setup? Many of us (as can be seen in responses) use tools/editors/IDEs that can
> handle this just fine.
>
> I worked on projects with duplicate file names in the past too; having tools
> that can deal with this helps, and I don't think this is too hard to achieve.
> You can't ask devs of every project you join to rename their files because your
> editor can't handle them.
But I can ask if we really want to create more identical ones :)
GHC already has a few duplicate file names. It's not a tragedy.

File names as the primary identifier crops up all the time.
Be it `find`ind files, jumping to them in the editor or other things.

It's not horrible by any means. VS Code has fuzzy search which usually
works for me in these cases.
Although it might not work so well if we rename all 15 Utils modules in
ghc to Utils.hs.

Overall it just seems easier to work with unique names when we have the
choice to do so.
And clearly in this case we have the choice.

Personally I never ran into a situation where prefixing the file name
was an issue at all.
But cases where non-unique names cause annoyance do happen from time to
time.

Hence why I prefer one over the other.
>
> (I don't know VS Code enough to help ..)
>
> Ömer
>
> Andreas Klebinger <klebinger.andreas at gmx.at>, 24 Eki 2019 Per, 14:48
> tarihinde şunu yazdı:
>> Hello devs,
>>
>> I also often jump to files. In my case usually using VS Code using Ctr+P as well which searches for files by name.
>> While I can check which folder a file is in in the case of duplicates it is a overhead which this refactor forces onto me.
>>
>> While there are workarounds, both for my case as for Matts. It's worth asking if requiring these workarounds is better
>> than just accepting redundant prefixes on module names.
>>
>> Personally I would prefer unique file names even at the cost of redundancy.
>> I rarely add import statements/full module names, but I *very* often jump to files.
>>
>> Cheers
>> Andreas
>>
>> Bryan Richter schrieb am 23.10.2019 um 18:00:
>>
>> Duplicate record fields is going to make this a bigger problem. Vim does support duplicate tags (:tselect and :tjump and related bindings), but hopefully haskell-ide-engine will one day provide us with semantic tags and solve this problem once and for all!
>>
>> On Wed, 23 Oct 2019, 17.49 Matthew Pickering, <matthewtpickering at gmail.com> wrote:
>>> Thanks Omer, Sylvain and Sebastian
>>>
>>> .
>>>
>>> I just configured my editor to use fzf and now I can use the `:GFiles`
>>> command to perform fuzzy search on files which is probably better than
>>> tags. If anyone else is using NixOS, all I had to do was add the
>>> `fzf-vim` plugin to the vim configuration.
>>>
>>> Cheers,
>>>
>>> Matt
>>>
>>> On Wed, Oct 23, 2019 at 2:54 PM Ömer Sinan Ağacan <omeragacan at gmail.com> wrote:
>>>> I use a file finder (fzf) for jumping to files. Because module names follow file
>>>> paths to jump to e.g. StgToCmmUtils.Utils I usually type `<C-p>stgcmmutils` and
>>>> fzf finds the correct file `compiler/GHC/StgToCmm/Utils.hs`.
>>>>
>>>> When generating tags I omit module names for this reason, it's easy with a good
>>>> file finder to jump to modules already, no need to generate tags for the
>>>> modules.
>>>>
>>>> fast-tags commands I use:
>>>>
>>>> - When working on the compiler:
>>>>
>>>>    $ fast-tags --no-module-tags driver ghc compiler
>>>>
>>>> - When working on the RTS:
>>>>
>>>>    $ fast-tags --no-module-tags driver ghc compiler
>>>>    $ ctags --append -R rts/**/*.c rts/**/*.h includes/**/*.h
>>>>
>>>> - When working on the libraries:
>>>>
>>>>    $ fast-tags --no-module-tags driver ghc compiler libraries
>>>>
>>>> Ömer
>>>>
>>>> Sebastian Graf <sgraf1337 at gmail.com>, 23 Eki 2019 Çar, 16:49 tarihinde
>>>> şunu yazdı:
>>>>> FWIW, I'm using VSCode's fuzzy file search with Ctrl+P (and vim's equivalent) rather successfully. Just tried it for Hs/Utils.hs by typing 'hsutils.hs'. It didn't turn up as the first result in VSCode, but it in vim.
>>>>>
>>>>> Am Mi., 23. Okt. 2019 um 14:27 Uhr schrieb Matthew Pickering <matthewtpickering at gmail.com>:
>>>>>> I use `fast-tags` which doesn't look at the hierarchy at all and I'm
>>>>>> not sure what the improvement would be as the names of the modules
>>>>>> would still clash.
>>>>>>
>>>>>> If there is some other recommended way to jump to a module then that
>>>>>> would also work for me.
>>>>>>
>>>>>> Matt
>>>>>>
>>>>>>
>>>>>> On Wed, Oct 23, 2019 at 12:08 PM Sylvain Henry <sylvain at haskus.fr> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> How do you generate your tags file? It seems to be a shortcoming of the
>>>>>>> generator to not take into account the location of the definition file.
>>>>>>>
>>>>>>>   > Perhaps `HsUtils` and `StgUtils` would be appropriate to
>>>>>>> disambiguate`Hs/Utils` and `StgToCmm/Utils`.
>>>>>>>
>>>>>>> We are promoting the module prefixes (`Hs`, `Stg`, `Tc`, etc.) into
>>>>>>> proper module layers (e.g. `HsUtils` becomes `GHC.Hs.Utils`) so it would
>>>>>>> be redundant to add the prefixes back. :/
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Sylvain
>>>>>>>
>>>>>>> On 23/10/2019 12:52, Matthew Pickering wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> The module rework has broken my workflow.
>>>>>>>>
>>>>>>>> Now my tags file is useless for jumping for modules as there are
>>>>>>>> multiple "Utils" and "Types" modules. Invariable I am jumping to the
>>>>>>>> wrong one. What do other people do to avoid this?
>>>>>>>>
>>>>>>>> Can we either revert these changes or give these modules unique names
>>>>>>>> to facilitate that only reliable way of navigating the code base.
>>>>>>>> Perhaps `HsUtils` and `StgUtils` would be appropriate to disambiguate
>>>>>>>> `Hs/Utils` and `StgToCmm/Utils`.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> Matt
>>>>>>>> _______________________________________________
>>>>>>>> ghc-devs mailing list
>>>>>>>> ghc-devs at haskell.org
>>>>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>>>>> _______________________________________________
>>>>>>> ghc-devs mailing list
>>>>>>> ghc-devs at haskell.org
>>>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>>>> _______________________________________________
>>>>>> ghc-devs mailing list
>>>>>> ghc-devs at haskell.org
>>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>>> _______________________________________________
>>>>> ghc-devs mailing list
>>>>> ghc-devs at haskell.org
>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From simonpj at microsoft.com  Fri Oct 25 09:07:04 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Fri, 25 Oct 2019 09:07:04 +0000
Subject: Debugging specialization
In-Reply-To: <51F7FBD8-7090-460A-9A61-1AB3FDDD7872@gmail.com>
References: <51F7FBD8-7090-460A-9A61-1AB3FDDD7872@gmail.com>
Message-ID: <AM6PR83MB0279C61C686C857A6E053E40AD650@AM6PR83MB0279.EURPRD83.prod.outlook.com>

Hi Alexis

|  I am trying to understand why GHC is not specializing an imported
|  INLINABLE instance method, and the information provided by -ddump-spec -
|  Wall-missed-specialisations has not been enough to help me figure it out.
|  Is there some easy/well-trodden way that I could build GHC with some
|  additional instrumentation in place to help me better understand the
|  decisions being made by the specializer, or is -ddump-spec the most
|  granularity available?

I'm afraid those are all the flags we currently have, yes.

|  I’ve noticed that Specialise.hs has handful of pprTrace calls sprinkled
|  about, but they are all commented out. Is the recommended method to just
|  uncomment these calls and rebuild GHC?

That's what I do.

But it would be better to stand back a bit and ask "what information would I really like to see?" and then design some logging mechanism to show it.  I'd be happy to advise.

Simon

|  -----Original Message-----
|  From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Alexis King
|  Sent: 24 October 2019 18:13
|  To: ghc-devs at haskell.org
|  Subject: Debugging specialization
|  
|  Hi all,
|  
|  I am trying to understand why GHC is not specializing an imported
|  INLINABLE instance method, and the information provided by -ddump-spec -
|  Wall-missed-specialisations has not been enough to help me figure it out.
|  Is there some easy/well-trodden way that I could build GHC with some
|  additional instrumentation in place to help me better understand the
|  decisions being made by the specializer, or is -ddump-spec the most
|  granularity available?
|  
|  I’ve noticed that Specialise.hs has handful of pprTrace calls sprinkled
|  about, but they are all commented out. Is the recommended method to just
|  uncomment these calls and rebuild GHC?
|  
|  Thanks,
|  Alexis
|  
|  _______________________________________________
|  ghc-devs mailing list
|  ghc-devs at haskell.org
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
|  ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C000a160e4f6a40bdf56208d7
|  58a57f63%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637075340144181354&a
|  mp;sdata=uPk4mfRrgGE5t6SQpXZyfU0xQ%2F6d9BlM50I3%2FmYcyms%3D&amp;reserved=0

From lexi.lambda at gmail.com  Fri Oct 25 16:34:55 2019
From: lexi.lambda at gmail.com (Alexis King)
Date: Fri, 25 Oct 2019 11:34:55 -0500
Subject: Debugging specialization
In-Reply-To: <AM6PR83MB0279C61C686C857A6E053E40AD650@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <51F7FBD8-7090-460A-9A61-1AB3FDDD7872@gmail.com>
 <AM6PR83MB0279C61C686C857A6E053E40AD650@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <17BC940E-133C-470B-A986-A91F15FFBCB2@gmail.com>

> On Oct 25, 2019, at 04:07, Simon Peyton Jones <simonpj at microsoft.com> wrote:
> 
> |  I’ve noticed that Specialise.hs has handful of pprTrace calls sprinkled
> |  about, but they are all commented out. Is the recommended method to just
> |  uncomment these calls and rebuild GHC?
> 
> That's what I do.

Thanks, I did indeed end up doing this, and it was enough to diagnose my problem.

> But it would be better to stand back a bit and ask "what information would I really like to see?" and then design some logging mechanism to show it.  I'd be happy to advise.

It would be nice to have a -dspecialise-check in the same vein as the existing -dinline-check/-drule-check flags, which seems probably not too hard to add. I unfortunately can’t dedicate the time to add it myself right now, as I’ve solved my immediate problem and probably already sunk more time into this than I ought to, but if it comes up again and I can spare the time I might give it a shot.


From ben at well-typed.com  Mon Oct 28 03:21:23 2019
From: ben at well-typed.com (Ben Gamari)
Date: Sun, 27 Oct 2019 23:21:23 -0400
Subject: ghc, rts/Linker.c, OpenBSD and WXNEEDED and NOPIE
In-Reply-To: <20191024195826.GA32023@nutty.outback.escape.de>
References: <20191024195826.GA32023@nutty.outback.escape.de>
Message-ID: <874kzttv0x.fsf@smart-cactus.org>

Matthias Kilian <kili at outback.escape.de> writes:

> Hi,
>
Hi Matthias,

...
>
> However, after building ghc-8.6 with all the WXNNEDED stuff removed,
> and running ghci from this build, I still get an abort because pages
> are mapped both writable and executable (I've configured my system
> to abort processes when a W^X violation happens). Here's a backtrace
> in case it's helpfull at all:
>
> #0  _thread_sys___syscall () at -:3
> #1  0x00000002010bb179 in _libc_mmap (addr=Variable "addr" is not available.) at /usr/src/lib/libc/sys/mmap.c:47
> #2  0x000000021be22374 in mmapForLinker (bytes=Variable "bytes" is not available.) at rts/Linker.c:1029
> #3  0x000000021be3e99b in m32_allocator_init () at rts/linker/M32Alloc.c:160
> #4  0x00000002fd518be1 in cimc_info () from /usr/local/lib/ghc/bin/../ghci-8.6.4/libHSghci-8.6.4-ghc8.6.4.so
> #5  0x0000000000000000 in ?? ()
>
> Any thougts about this?
>
I suspect what you ran into here is GHCi loading a static archive:


I tried running GHCi compiled with debug information under GDB, set a
breakpoint on m32_allocator_init and found the following:

    >>> bt
    #0  m32_allocator_init () at rts/linker/M32Alloc.c:155
    #1  0x00007ffff2d680b7 in ghcizm8zi9zi0zi20191022_GHCiziRun_run1_info () from _build/stage1/lib/../lib/x86_64-linux-ghc-8.9.0.20191022/libHSghci-8.9.0.20191022-ghc8.9.0.20191022.so                                                                                          
    #2  0x0000000000000000 in ?? ()

While the backtrace doesn't reveal this, the caller appears to be
initLinker_ which I suspect is called by GHCi.ObjLink.initObjLinker
(which is in turn called by GHCi.Run.run).

I have opened #17407 to track this.

> And, since W|X (and nopie) isn't good on any system: is anyone
> working on getting rid of this for other unixes?
>
There has been some work on this on Darwin (see #13624). Unfortunately
no work has been put into ridding other platforms of W|X mappings (e.g.
see #12657).

It would be great if you felt you could contribute some help!

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191027/f3cc4d24/attachment.sig>

From lexi.lambda at gmail.com  Mon Oct 28 06:15:51 2019
From: lexi.lambda at gmail.com (Alexis King)
Date: Mon, 28 Oct 2019 01:15:51 -0500
Subject: Simplifier bug fixed in GHC 8.8.1?
Message-ID: <CCD73C86-2EBE-4C2C-810F-3F73DAB23F3A@gmail.com>

Hi all,

I have an odd question: I’ve bumped into a clear simplifier bug, and although it only happens on GHC 8.6.5, not 8.8.1, I’d like to locate the change that fixed it. My library’s test suite currently fails on GHC 8.6.5 due to the bug, and I’d rather not force all my users to upgrade to 8.8 if I can help it, so I’m hoping to find a workaround.

The minimal test case I’ve found for the bug is this program:

    {-# LANGUAGE GeneralizedNewtypeDeriving, StandaloneDeriving, TypeFamilies #-}

    import Control.Exception
    import Control.Monad.IO.Class
    import Control.Monad.Trans.Identity
    import Control.Monad.Trans.Reader

    class Monad m => MonadFoo m where
      foo :: m a -> m a
    instance MonadFoo IO where
      foo m = onException m (pure ())
    instance MonadFoo m => MonadFoo (ReaderT r m) where
      foo m = ReaderT $ \r -> foo (runReaderT m r)
    deriving instance MonadFoo m => MonadFoo (IdentityT m)

    type family F m where
      F m = IdentityT m

    newtype FT m a = FT { runFT :: F m a }
      deriving (Functor, Applicative, Monad, MonadIO, MonadFoo)

    main :: IO ()
    main = run (foo (liftIO (throwIO (IndexOutOfBounds "bang"))))
      where
        run :: ReaderT () (FT (ReaderT () IO)) a -> IO a
        run = flip runReaderT () . runIdentityT . runFT . flip runReaderT ()

Using GHC 8.6.5 on macOS 10.14.5, compiling this program with optimizations reliably triggers the -fcatch-bottoms sanitization:

    $ ghc -O -fcatch-bottoms weird.hs && ./weird
    [1 of 1] Compiling Main             ( weird.hs, weird.o )
    Linking weird ...
    weird: Bottoming expression returned

What goes wrong? Somehow the generated core for this program includes the following:

    lvl_s47B :: SomeException
    lvl_s47B = $fExceptionArrayException_$ctoException lvl_s483

    m_s47r :: () -> State# RealWorld -> (# State# RealWorld, () #)
    m_s47r
      = \ _ (eta_B1 :: State# RealWorld) -> raiseIO# lvl_s47B eta_B1

    main_s2Ww :: State# RealWorld -> (# State# RealWorld, () #)
    main_s2Ww
      = \ (eta_a2wK :: State# RealWorld) ->
          catch# (case m_s47r `cast` <Co:33> of { }) raiseIO# eta_a2wK

This core is completely bogus: it assumes that m_s47r is bottom, but m_s47r is a top-level function! The program still passes -dcore-lint, unfortunately, as it is still well-typed. (Also, in case it helps: -ddump-simplifier-iterations shows that the buggy transformation occurs in the first iteration of the very first simplifier pass.)

I’ve been trying to figure out what change might have fixed this so that I can assess if it’s possible to work around, but I haven’t found anything obvious. I’ve been slowly `git bisect`ing to look for the commit that introduced the fix, but many of the commits I’ve tested cause unrelated panics on my machine, which has been exacerbating the problem of the slow recompilation times. I’m a little at wits’ end, but opening a bug report hasn’t felt right, since the bug does appear to already be fixed.

Does this issue ring any bells to anyone on this list? Is there a particular patch that landed between GHC 8.6.5 and GHC 8.8.1 that might have fixed this problem? If not, I’ll keep trying with `git bisect`, but I’d appreciate any pointers.

Thanks,
Alexis


From simonpj at microsoft.com  Mon Oct 28 09:31:06 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 28 Oct 2019 09:31:06 +0000
Subject: Handling source locations in HsSyn via TTG
Message-ID: <AM6PR83MB0279697E188BA6A33A14EDE6AD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>

Friends
As you know

  *   We are trying to use "Trees That Grow<https://gitlab.haskell.org/ghc/ghc/wikis/implementing-trees-that-grow>" (TTG) to move HsSyn towards a situation in which GHC is merely a client of a generic HsSyn data type that can be used by clients other than GHC.
  *   One sticking point has been the question of attaching source locations.  We used to have a "ping-pong" style, in which very node is decorated with a source location, but that's a bit more awkward in TTG.
  *   This wiki page<https://gitlab.haskell.org/ghc/ghc/wikis/implementing-trees-that-grow/handling-source-locations> outlines some choices, while ticket #15495<https://gitlab.haskell.org/ghc/ghc/issues/15495> has a lot of discussion.
  *   HEAD embodies Solution A.  But it has the disadvantage that the type system doesn't enforce locations to be present at all.   That has undesirable consequences (eg ticket #17330<https://gitlab.haskell.org/ghc/ghc/issues/17330>)
  *   The proposal is to move to Solution D on that page; you can see how it plays out in MR !1970<https://gitlab.haskell.org/ghc/ghc/merge_requests/1970>.
  *   (I think Solutions B and C are non-starters by comparison.)
If you care, please check out the design and the MR.   We can change later, of course, but doing so changes a lot of code, including client code, so we'd prefer not to.
Let's try to converge by the end of the week.
Thanks
Simon


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191028/f3895afb/attachment.html>

From chrisdone at gmail.com  Mon Oct 28 09:59:31 2019
From: chrisdone at gmail.com (Christopher Done)
Date: Mon, 28 Oct 2019 09:59:31 +0000
Subject: Quick Q: do all FFI (non-primop) calls involve State# and RealWorld?
Message-ID: <CAAJHNPACof--Qdu1h6nXM6+0hjtTYywR9pMsqpMoz0OLRe8F_A@mail.gmail.com>

Hi all,

I tried compiling this file:

{-# LANGUAGE NoImplicitPrelude #-}-- | Demonstrate various use of the
FFI.module Foreign whereimport Foreign.Cforeign import ccall "math.h
sin" sin :: CDouble -> CDoubleit :: CDoubleit = sin 1

And I’ve noticed that the annotated type given for this foreign op in Core,
is (# State# RealWorld, CDouble #), whereas I would have expected e.g.
CDouble.

Meanwhile, the foreign op call is passed a RealWorld argument.

Additionally, code that consumes the result of this foreign call expects a (#
CDouble #) as a return value.

So there are some assumptions I put in my interpreter to test this FFI call
out:

   1. Despite claiming to return the real world in a tuple, it actually
   should just return an unboxed tuple of the value.
   2. It should ignore the RealWorld argument entirely.

I assume, if I were to lift this function into returning IO, that I should
indeed return the RealWorld argument given. So the lesson is:

All FFI functions accept a RealWorld, and may return a 2-tuple of State#
RealWorld *if* it’s impure, else it’ll return a 1-tuple of the value.
Correct?

Can someone confirm that my observations are right? Also, if so, is there
somewhere I can read more about this?

Cheers

Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191028/98b2ce47/attachment.html>

From sgraf1337 at gmail.com  Mon Oct 28 10:06:31 2019
From: sgraf1337 at gmail.com (Sebastian Graf)
Date: Mon, 28 Oct 2019 11:06:31 +0100
Subject: Simplifier bug fixed in GHC 8.8.1?
In-Reply-To: <CCD73C86-2EBE-4C2C-810F-3F73DAB23F3A@gmail.com>
References: <CCD73C86-2EBE-4C2C-810F-3F73DAB23F3A@gmail.com>
Message-ID: <CAAS+=P_zyuB_9oRH839MkdZLEvtu1ND7DZ8ngM04NWQ=cWjLew@mail.gmail.com>

Hi Alexis,

I think the fact that it looks like it's fixed is only a coincidence. See
https://gitlab.haskell.org/ghc/ghc/issues/17409, where I go into a bit more
detail.

Cheers
Sebastian

Am Mo., 28. Okt. 2019 um 07:16 Uhr schrieb Alexis King <
lexi.lambda at gmail.com>:

> Hi all,
>
> I have an odd question: I’ve bumped into a clear simplifier bug, and
> although it only happens on GHC 8.6.5, not 8.8.1, I’d like to locate the
> change that fixed it. My library’s test suite currently fails on GHC 8.6.5
> due to the bug, and I’d rather not force all my users to upgrade to 8.8 if
> I can help it, so I’m hoping to find a workaround.
>
> The minimal test case I’ve found for the bug is this program:
>
>     {-# LANGUAGE GeneralizedNewtypeDeriving, StandaloneDeriving,
> TypeFamilies #-}
>
>     import Control.Exception
>     import Control.Monad.IO.Class
>     import Control.Monad.Trans.Identity
>     import Control.Monad.Trans.Reader
>
>     class Monad m => MonadFoo m where
>       foo :: m a -> m a
>     instance MonadFoo IO where
>       foo m = onException m (pure ())
>     instance MonadFoo m => MonadFoo (ReaderT r m) where
>       foo m = ReaderT $ \r -> foo (runReaderT m r)
>     deriving instance MonadFoo m => MonadFoo (IdentityT m)
>
>     type family F m where
>       F m = IdentityT m
>
>     newtype FT m a = FT { runFT :: F m a }
>       deriving (Functor, Applicative, Monad, MonadIO, MonadFoo)
>
>     main :: IO ()
>     main = run (foo (liftIO (throwIO (IndexOutOfBounds "bang"))))
>       where
>         run :: ReaderT () (FT (ReaderT () IO)) a -> IO a
>         run = flip runReaderT () . runIdentityT . runFT . flip runReaderT
> ()
>
> Using GHC 8.6.5 on macOS 10.14.5, compiling this program with
> optimizations reliably triggers the -fcatch-bottoms sanitization:
>
>     $ ghc -O -fcatch-bottoms weird.hs && ./weird
>     [1 of 1] Compiling Main             ( weird.hs, weird.o )
>     Linking weird ...
>     weird: Bottoming expression returned
>
> What goes wrong? Somehow the generated core for this program includes the
> following:
>
>     lvl_s47B :: SomeException
>     lvl_s47B = $fExceptionArrayException_$ctoException lvl_s483
>
>     m_s47r :: () -> State# RealWorld -> (# State# RealWorld, () #)
>     m_s47r
>       = \ _ (eta_B1 :: State# RealWorld) -> raiseIO# lvl_s47B eta_B1
>
>     main_s2Ww :: State# RealWorld -> (# State# RealWorld, () #)
>     main_s2Ww
>       = \ (eta_a2wK :: State# RealWorld) ->
>           catch# (case m_s47r `cast` <Co:33> of { }) raiseIO# eta_a2wK
>
> This core is completely bogus: it assumes that m_s47r is bottom, but
> m_s47r is a top-level function! The program still passes -dcore-lint,
> unfortunately, as it is still well-typed. (Also, in case it helps:
> -ddump-simplifier-iterations shows that the buggy transformation occurs in
> the first iteration of the very first simplifier pass.)
>
> I’ve been trying to figure out what change might have fixed this so that I
> can assess if it’s possible to work around, but I haven’t found anything
> obvious. I’ve been slowly `git bisect`ing to look for the commit that
> introduced the fix, but many of the commits I’ve tested cause unrelated
> panics on my machine, which has been exacerbating the problem of the slow
> recompilation times. I’m a little at wits’ end, but opening a bug report
> hasn’t felt right, since the bug does appear to already be fixed.
>
> Does this issue ring any bells to anyone on this list? Is there a
> particular patch that landed between GHC 8.6.5 and GHC 8.8.1 that might
> have fixed this problem? If not, I’ll keep trying with `git bisect`, but
> I’d appreciate any pointers.
>
> Thanks,
> Alexis
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191028/d4bb666f/attachment.html>

From vladislav at serokell.io  Mon Oct 28 10:06:41 2019
From: vladislav at serokell.io (Vladislav Zavialov)
Date: Mon, 28 Oct 2019 13:06:41 +0300
Subject: Handling source locations in HsSyn via TTG
In-Reply-To: <AM6PR83MB0279697E188BA6A33A14EDE6AD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <AM6PR83MB0279697E188BA6A33A14EDE6AD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <A8CFA060-0511-4280-A835-DFCE68438ECE@serokell.io>

I care about this, and I maintain my viewpoint described in https://mail.haskell.org/pipermail/ghc-devs/2019-February/017080.html

I’m willing to implement this.

As to merge request !1970, it isn’t good to special-case GhcPass in a closed type family, making other tools second-class citizens. Let’s say I have `MyToolPass`, how would I write an instance of `WrapL` for it?

- Vlad

> On 28 Oct 2019, at 12:31, Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> wrote:
> 
> Friends
> 
> As you know
> 
> 	• We are trying to use “Trees That Grow” (TTG) to move HsSyn towards a situation in which GHC is merely a client of a generic HsSyn data type that can be used by clients other than GHC.
> 	• One sticking point has been the question of attaching source locations.  We used to have a “ping-pong” style, in which very node is decorated with a source location, but that’s a bit more awkward in TTG.
> 	• This wiki page outlines some choices, while ticket #15495 has a lot of discussion.
> 	• HEAD embodies Solution A.  But it has the disadvantage that the type system doesn’t enforce locations to be present at all.   That has undesirable consequences (eg ticket #17330)
> 	• The proposal is to move to Solution D on that page; you can see how it plays out in MR !1970.
> 	• (I think Solutions B and C are non-starters by comparison.)
> If you care, please check out the design and the MR.   We can change later, of course, but doing so changes a lot of code, including client code, so we’d prefer not to.
> 
> Let’s try to converge by the end of the week.
> 
> Thanks
> 
> Simon
> 
>  
> 
>  
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From sgraf1337 at gmail.com  Mon Oct 28 10:13:10 2019
From: sgraf1337 at gmail.com (Sebastian Graf)
Date: Mon, 28 Oct 2019 11:13:10 +0100
Subject: Handling source locations in HsSyn via TTG
In-Reply-To: <A8CFA060-0511-4280-A835-DFCE68438ECE@serokell.io>
References: <AM6PR83MB0279697E188BA6A33A14EDE6AD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <A8CFA060-0511-4280-A835-DFCE68438ECE@serokell.io>
Message-ID: <CAAS+=P_JSDmaebsk8MdUJdv-nynoab5PAmkp1fTE+ZHgXuq+dw@mail.gmail.com>

Hi Vlad,

Note that the MR description is a little misleading and I should update it:
I'm using an open type family, really. See the section for solution D on
the wiki page
<https://gitlab.haskell.org/ghc/ghc/wikis/implementing-trees-that-grow/handling-source-locations#solution-d-example-code>
that shows how to extend the approach to Haddock (which needs SrcLocs, too).
If I understand correctly, you're advocating solution B. If you can think
of any more Pros and Cons (comparing to solution D, in particular), feel
free to edit the wiki page.

Sebastian


Am Mo., 28. Okt. 2019 um 11:07 Uhr schrieb Vladislav Zavialov <
vladislav at serokell.io>:

> I care about this, and I maintain my viewpoint described in
> https://mail.haskell.org/pipermail/ghc-devs/2019-February/017080.html
>
> I’m willing to implement this.
>
> As to merge request !1970, it isn’t good to special-case GhcPass in a
> closed type family, making other tools second-class citizens. Let’s say I
> have `MyToolPass`, how would I write an instance of `WrapL` for it?
>
> - Vlad
>
> > On 28 Oct 2019, at 12:31, Simon Peyton Jones via ghc-devs <
> ghc-devs at haskell.org> wrote:
> >
> > Friends
> >
> > As you know
> >
> >       • We are trying to use “Trees That Grow” (TTG) to move HsSyn
> towards a situation in which GHC is merely a client of a generic HsSyn data
> type that can be used by clients other than GHC.
> >       • One sticking point has been the question of attaching source
> locations.  We used to have a “ping-pong” style, in which very node is
> decorated with a source location, but that’s a bit more awkward in TTG.
> >       • This wiki page outlines some choices, while ticket #15495 has a
> lot of discussion.
> >       • HEAD embodies Solution A.  But it has the disadvantage that the
> type system doesn’t enforce locations to be present at all.   That has
> undesirable consequences (eg ticket #17330)
> >       • The proposal is to move to Solution D on that page; you can see
> how it plays out in MR !1970.
> >       • (I think Solutions B and C are non-starters by comparison.)
> > If you care, please check out the design and the MR.   We can change
> later, of course, but doing so changes a lot of code, including client
> code, so we’d prefer not to.
> >
> > Let’s try to converge by the end of the week.
> >
> > Thanks
> >
> > Simon
> >
> >
> >
> >
> >
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191028/ea16dae9/attachment.html>

From vladislav at serokell.io  Mon Oct 28 10:20:19 2019
From: vladislav at serokell.io (Vladislav Zavialov)
Date: Mon, 28 Oct 2019 13:20:19 +0300
Subject: Handling source locations in HsSyn via TTG
In-Reply-To: <CAAS+=P_JSDmaebsk8MdUJdv-nynoab5PAmkp1fTE+ZHgXuq+dw@mail.gmail.com>
References: <AM6PR83MB0279697E188BA6A33A14EDE6AD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <A8CFA060-0511-4280-A835-DFCE68438ECE@serokell.io>
 <CAAS+=P_JSDmaebsk8MdUJdv-nynoab5PAmkp1fTE+ZHgXuq+dw@mail.gmail.com>
Message-ID: <500A25CA-C719-4F65-A029-BF3A26659607@serokell.io>

> Note that the MR description is a little misleading and I should update it: I'm using an open type family, really. 

Ah, that’s good to know. In this case, I’m in support.

- Vlad

> On 28 Oct 2019, at 13:13, Sebastian Graf <sgraf1337 at gmail.com> wrote:
> 
> Hi Vlad,
> 
> Note that the MR description is a little misleading and I should update it: I'm using an open type family, really. See the section for solution D on the wiki page that shows how to extend the approach to Haddock (which needs SrcLocs, too).
> If I understand correctly, you're advocating solution B. If you can think of any more Pros and Cons (comparing to solution D, in particular), feel free to edit the wiki page.
> 
> Sebastian
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From simonpj at microsoft.com  Mon Oct 28 10:20:27 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 28 Oct 2019 10:20:27 +0000
Subject: Handling source locations in HsSyn via TTG
In-Reply-To: <A8CFA060-0511-4280-A835-DFCE68438ECE@serokell.io>
References: <AM6PR83MB0279697E188BA6A33A14EDE6AD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <A8CFA060-0511-4280-A835-DFCE68438ECE@serokell.io>
Message-ID: <AM6PR83MB0279B5D5F9565648F9877CBEAD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>

Vlad

Are you arguing for Solution D?  Or are you proposing some new solution E?  I can't tell.


| As to merge request !1970, it isn’t good to special-case GhcPass in a
| closed type family, making other tools second-class citizens. Let’s say I
| have `MyToolPass`, how would I write an instance of `WrapL` for it?

Easy

	type instance WrapL ToolPass t = ...

What am I missing?

Simon

| -----Original Message-----
| From: Vladislav Zavialov <vladislav at serokell.io>
| Sent: 28 October 2019 10:07
| To: Simon Peyton Jones <simonpj at microsoft.com>
| Cc: ghc-devs at haskell.org
| Subject: Re: Handling source locations in HsSyn via TTG
| 
| I care about this, and I maintain my viewpoint described in
| https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.has
| kell.org%2Fpipermail%2Fghc-devs%2F2019-
| February%2F017080.html&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C06c859
| d8bc0f48c73cb208d75b8e895c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63
| 7078540055975603&amp;sdata=FhkqfWGXaNX%2Fz4IiCcvoVCyzVsSAlyz6Y1dxEGUjX9I%3
| D&amp;reserved=0
| 
| I’m willing to implement this.
| 
| As to merge request !1970, it isn’t good to special-case GhcPass in a
| closed type family, making other tools second-class citizens. Let’s say I
| have `MyToolPass`, how would I write an instance of `WrapL` for it?
| 
| - Vlad
| 
| > On 28 Oct 2019, at 12:31, Simon Peyton Jones via ghc-devs <ghc-
| devs at haskell.org> wrote:
| >
| > Friends
| >
| > As you know
| >
| > 	• We are trying to use “Trees That Grow” (TTG) to move HsSyn towards
| a situation in which GHC is merely a client of a generic HsSyn data type
| that can be used by clients other than GHC.
| > 	• One sticking point has been the question of attaching source
| locations.  We used to have a “ping-pong” style, in which very node is
| decorated with a source location, but that’s a bit more awkward in TTG.
| > 	• This wiki page outlines some choices, while ticket #15495 has a
| lot of discussion.
| > 	• HEAD embodies Solution A.  But it has the disadvantage that the
| type system doesn’t enforce locations to be present at all.   That has
| undesirable consequences (eg ticket #17330)
| > 	• The proposal is to move to Solution D on that page; you can see
| how it plays out in MR !1970.
| > 	• (I think Solutions B and C are non-starters by comparison.)
| > If you care, please check out the design and the MR.   We can change
| later, of course, but doing so changes a lot of code, including client
| code, so we’d prefer not to.
| >
| > Let’s try to converge by the end of the week.
| >
| > Thanks
| >
| > Simon
| >
| >
| >
| >
| >
| > _______________________________________________
| > ghc-devs mailing list
| > ghc-devs at haskell.org
| >
| https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
| ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
| devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C06c859d8bc0f48c73cb208d7
| 5b8e895c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637078540055975603&a
| mp;sdata=CRYEYmjuAYYIhoAeTPbi%2FctCiWmkIjL6pKUTBgVBwo8%3D&amp;reserved=0


From vladislav at serokell.io  Mon Oct 28 10:30:37 2019
From: vladislav at serokell.io (Vladislav Zavialov)
Date: Mon, 28 Oct 2019 13:30:37 +0300
Subject: Handling source locations in HsSyn via TTG
In-Reply-To: <AM6PR83MB0279B5D5F9565648F9877CBEAD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <AM6PR83MB0279697E188BA6A33A14EDE6AD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <A8CFA060-0511-4280-A835-DFCE68438ECE@serokell.io>
 <AM6PR83MB0279B5D5F9565648F9877CBEAD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <2B448026-9E21-4A0A-82CD-E2D1C4E6EDF0@serokell.io>

> Are you arguing for Solution D?  Or are you proposing some new solution E?  I can't tell.

I suspect that I’m arguing for Solution B, but it’s hard to tell because it’s not described in enough detail in the Wiki.

> Easy
> 
> 	type instance WrapL ToolPass t = ...
> 
> What am I missing?


This assumes that `WrapL` is an open type family. In this case, there’s no problem. The merge request description has the following definition of WrapL:

type instance WrapL p (f :: * -> *) where
  WrapL (GhcPass p) f = Located (f (GhcPass p))
  WrapL p           f =          f p
type LPat p = WrapL p Pat

That wouldn’t be extensible. However, if WrapL is open, then Solution D sounds good to me.

- Vlad 

> On 28 Oct 2019, at 13:20, Simon Peyton Jones <simonpj at microsoft.com> wrote:
> 
> Vlad
> 
> Are you arguing for Solution D?  Or are you proposing some new solution E?  I can't tell.
> 
> 
> | As to merge request !1970, it isn’t good to special-case GhcPass in a
> | closed type family, making other tools second-class citizens. Let’s say I
> | have `MyToolPass`, how would I write an instance of `WrapL` for it?
> 
> Easy
> 
> 	type instance WrapL ToolPass t = ...
> 
> What am I missing?
> 
> Simon
> 
> | -----Original Message-----
> | From: Vladislav Zavialov <vladislav at serokell.io>
> | Sent: 28 October 2019 10:07
> | To: Simon Peyton Jones <simonpj at microsoft.com>
> | Cc: ghc-devs at haskell.org
> | Subject: Re: Handling source locations in HsSyn via TTG
> | 
> | I care about this, and I maintain my viewpoint described in
> | https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.has
> | kell.org%2Fpipermail%2Fghc-devs%2F2019-
> | February%2F017080.html&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C06c859
> | d8bc0f48c73cb208d75b8e895c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63
> | 7078540055975603&amp;sdata=FhkqfWGXaNX%2Fz4IiCcvoVCyzVsSAlyz6Y1dxEGUjX9I%3
> | D&amp;reserved=0
> | 
> | I’m willing to implement this.
> | 
> | As to merge request !1970, it isn’t good to special-case GhcPass in a
> | closed type family, making other tools second-class citizens. Let’s say I
> | have `MyToolPass`, how would I write an instance of `WrapL` for it?
> | 
> | - Vlad
> | 
> | > On 28 Oct 2019, at 12:31, Simon Peyton Jones via ghc-devs <ghc-
> | devs at haskell.org> wrote:
> | >
> | > Friends
> | >
> | > As you know
> | >
> | > 	• We are trying to use “Trees That Grow” (TTG) to move HsSyn towards
> | a situation in which GHC is merely a client of a generic HsSyn data type
> | that can be used by clients other than GHC.
> | > 	• One sticking point has been the question of attaching source
> | locations.  We used to have a “ping-pong” style, in which very node is
> | decorated with a source location, but that’s a bit more awkward in TTG.
> | > 	• This wiki page outlines some choices, while ticket #15495 has a
> | lot of discussion.
> | > 	• HEAD embodies Solution A.  But it has the disadvantage that the
> | type system doesn’t enforce locations to be present at all.   That has
> | undesirable consequences (eg ticket #17330)
> | > 	• The proposal is to move to Solution D on that page; you can see
> | how it plays out in MR !1970.
> | > 	• (I think Solutions B and C are non-starters by comparison.)
> | > If you care, please check out the design and the MR.   We can change
> | later, of course, but doing so changes a lot of code, including client
> | code, so we’d prefer not to.
> | >
> | > Let’s try to converge by the end of the week.
> | >
> | > Thanks
> | >
> | > Simon
> | >
> | >
> | >
> | >
> | >
> | > _______________________________________________
> | > ghc-devs mailing list
> | > ghc-devs at haskell.org
> | >
> | https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
> | ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
> | devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C06c859d8bc0f48c73cb208d7
> | 5b8e895c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637078540055975603&a
> | mp;sdata=CRYEYmjuAYYIhoAeTPbi%2FctCiWmkIjL6pKUTBgVBwo8%3D&amp;reserved=0
> 


From rae at richarde.dev  Mon Oct 28 10:45:55 2019
From: rae at richarde.dev (Richard Eisenberg)
Date: Mon, 28 Oct 2019 10:45:55 +0000
Subject: Handling source locations in HsSyn via TTG
In-Reply-To: <2B448026-9E21-4A0A-82CD-E2D1C4E6EDF0@serokell.io>
References: <AM6PR83MB0279697E188BA6A33A14EDE6AD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <A8CFA060-0511-4280-A835-DFCE68438ECE@serokell.io>
 <AM6PR83MB0279B5D5F9565648F9877CBEAD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <2B448026-9E21-4A0A-82CD-E2D1C4E6EDF0@serokell.io>
Message-ID: <B7F52998-8378-4849-BFD9-9C0144FE056B@richarde.dev>

A nice property of Solution D (and I think this is a new observation) is that we could accommodate both located source and unlocated. For example:

> data GhcPass (c :: Pass)   -- as today
> data Pass = MkPass Phase HasLoc
> data Phase = Parsed | Renamed | Typechecked
> data HasLoc = YesLoc | NoLoc
> 
> type instance WrapL (GhcPass (MkPass p YesLoc)) f = Located (f (GhcPass (MkPass p YesLoc)))
> type instance WrapL (GhcPass (MkPass p NoLoc)) f = f (GhcPass (MkPass p NoLoc))

I don't actually think this is a good idea, as it would mean making a bunch of functions polymorphic in the HasLoc parameter, which is syntactically annoying. But the existence of this approach suggests that the design scales.

Regardless of what you think of this new idea, I'm in favor of Solution D. I like my types to stop me from making errors, and I'm willing to put up with the odd type error asking me to write `unLoc` as I work in order to avoid errors.

Richard

> On Oct 28, 2019, at 10:30 AM, Vladislav Zavialov <vladislav at serokell.io> wrote:
> 
>> Are you arguing for Solution D?  Or are you proposing some new solution E?  I can't tell.
> 
> I suspect that I’m arguing for Solution B, but it’s hard to tell because it’s not described in enough detail in the Wiki.
> 
>> Easy
>> 
>> 	type instance WrapL ToolPass t = ...
>> 
>> What am I missing?
> 
> 
> This assumes that `WrapL` is an open type family. In this case, there’s no problem. The merge request description has the following definition of WrapL:
> 
> type instance WrapL p (f :: * -> *) where
>  WrapL (GhcPass p) f = Located (f (GhcPass p))
>  WrapL p           f =          f p
> type LPat p = WrapL p Pat
> 
> That wouldn’t be extensible. However, if WrapL is open, then Solution D sounds good to me.
> 
> - Vlad 
> 
>> On 28 Oct 2019, at 13:20, Simon Peyton Jones <simonpj at microsoft.com> wrote:
>> 
>> Vlad
>> 
>> Are you arguing for Solution D?  Or are you proposing some new solution E?  I can't tell.
>> 
>> 
>> | As to merge request !1970, it isn’t good to special-case GhcPass in a
>> | closed type family, making other tools second-class citizens. Let’s say I
>> | have `MyToolPass`, how would I write an instance of `WrapL` for it?
>> 
>> Easy
>> 
>> 	type instance WrapL ToolPass t = ...
>> 
>> What am I missing?
>> 
>> Simon
>> 
>> | -----Original Message-----
>> | From: Vladislav Zavialov <vladislav at serokell.io>
>> | Sent: 28 October 2019 10:07
>> | To: Simon Peyton Jones <simonpj at microsoft.com>
>> | Cc: ghc-devs at haskell.org
>> | Subject: Re: Handling source locations in HsSyn via TTG
>> | 
>> | I care about this, and I maintain my viewpoint described in
>> | https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.has
>> | kell.org%2Fpipermail%2Fghc-devs%2F2019-
>> | February%2F017080.html&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C06c859
>> | d8bc0f48c73cb208d75b8e895c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63
>> | 7078540055975603&amp;sdata=FhkqfWGXaNX%2Fz4IiCcvoVCyzVsSAlyz6Y1dxEGUjX9I%3
>> | D&amp;reserved=0
>> | 
>> | I’m willing to implement this.
>> | 
>> | As to merge request !1970, it isn’t good to special-case GhcPass in a
>> | closed type family, making other tools second-class citizens. Let’s say I
>> | have `MyToolPass`, how would I write an instance of `WrapL` for it?
>> | 
>> | - Vlad
>> | 
>> | > On 28 Oct 2019, at 12:31, Simon Peyton Jones via ghc-devs <ghc-
>> | devs at haskell.org> wrote:
>> | >
>> | > Friends
>> | >
>> | > As you know
>> | >
>> | > 	• We are trying to use “Trees That Grow” (TTG) to move HsSyn towards
>> | a situation in which GHC is merely a client of a generic HsSyn data type
>> | that can be used by clients other than GHC.
>> | > 	• One sticking point has been the question of attaching source
>> | locations.  We used to have a “ping-pong” style, in which very node is
>> | decorated with a source location, but that’s a bit more awkward in TTG.
>> | > 	• This wiki page outlines some choices, while ticket #15495 has a
>> | lot of discussion.
>> | > 	• HEAD embodies Solution A.  But it has the disadvantage that the
>> | type system doesn’t enforce locations to be present at all.   That has
>> | undesirable consequences (eg ticket #17330)
>> | > 	• The proposal is to move to Solution D on that page; you can see
>> | how it plays out in MR !1970.
>> | > 	• (I think Solutions B and C are non-starters by comparison.)
>> | > If you care, please check out the design and the MR.   We can change
>> | later, of course, but doing so changes a lot of code, including client
>> | code, so we’d prefer not to.
>> | >
>> | > Let’s try to converge by the end of the week.
>> | >
>> | > Thanks
>> | >
>> | > Simon
>> | >
>> | >
>> | >
>> | >
>> | >
>> | > _______________________________________________
>> | > ghc-devs mailing list
>> | > ghc-devs at haskell.org
>> | >
>> | https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
>> | ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
>> | devs&amp;data=02%7C01%7Csimonpj%40microsoft.com%7C06c859d8bc0f48c73cb208d7
>> | 5b8e895c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637078540055975603&a
>> | mp;sdata=CRYEYmjuAYYIhoAeTPbi%2FctCiWmkIjL6pKUTBgVBwo8%3D&amp;reserved=0
>> 
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From rae at richarde.dev  Mon Oct 28 11:26:30 2019
From: rae at richarde.dev (Richard Eisenberg)
Date: Mon, 28 Oct 2019 11:26:30 +0000
Subject: Debugging specialization
In-Reply-To: <17BC940E-133C-470B-A986-A91F15FFBCB2@gmail.com>
References: <51F7FBD8-7090-460A-9A61-1AB3FDDD7872@gmail.com>
 <AM6PR83MB0279C61C686C857A6E053E40AD650@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <17BC940E-133C-470B-A986-A91F15FFBCB2@gmail.com>
Message-ID: <621A46C6-4341-4A59-BCF5-6A10A1238C0B@richarde.dev>

Inspection Testing by Joachim Breitner may be of interest: https://arxiv.org/abs/1803.07130

> On Oct 25, 2019, at 5:34 PM, Alexis King <lexi.lambda at gmail.com> wrote:
> 
>> On Oct 25, 2019, at 04:07, Simon Peyton Jones <simonpj at microsoft.com> wrote:
>> 
>> |  I’ve noticed that Specialise.hs has handful of pprTrace calls sprinkled
>> |  about, but they are all commented out. Is the recommended method to just
>> |  uncomment these calls and rebuild GHC?
>> 
>> That's what I do.
> 
> Thanks, I did indeed end up doing this, and it was enough to diagnose my problem.
> 
>> But it would be better to stand back a bit and ask "what information would I really like to see?" and then design some logging mechanism to show it.  I'd be happy to advise.
> 
> It would be nice to have a -dspecialise-check in the same vein as the existing -dinline-check/-drule-check flags, which seems probably not too hard to add. I unfortunately can’t dedicate the time to add it myself right now, as I’ve solved my immediate problem and probably already sunk more time into this than I ought to, but if it comes up again and I can spare the time I might give it a shot.
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From lexi.lambda at gmail.com  Mon Oct 28 11:46:24 2019
From: lexi.lambda at gmail.com (Alexis King)
Date: Mon, 28 Oct 2019 06:46:24 -0500
Subject: Simplifier bug fixed in GHC 8.8.1?
In-Reply-To: <CAAS+=P_zyuB_9oRH839MkdZLEvtu1ND7DZ8ngM04NWQ=cWjLew@mail.gmail.com>
References: <CCD73C86-2EBE-4C2C-810F-3F73DAB23F3A@gmail.com>
 <CAAS+=P_zyuB_9oRH839MkdZLEvtu1ND7DZ8ngM04NWQ=cWjLew@mail.gmail.com>
Message-ID: <96B95806-1AE3-47D2-BF8D-E9DFBAE2F0BC@gmail.com>

Wonderful, thank you for the investigation! (Well, maybe not so wonderful if it isn’t actually fixed yet, but your followup comment in the issue gives me hope it is.) I’ll take the discussion over there, then.

> On Oct 28, 2019, at 05:06, Sebastian Graf <sgraf1337 at gmail.com> wrote:
> 
> Hi Alexis,
> 
> I think the fact that it looks like it's fixed is only a coincidence. See https://gitlab.haskell.org/ghc/ghc/issues/17409 <https://gitlab.haskell.org/ghc/ghc/issues/17409>, where I go into a bit more detail.
> 
> Cheers
> Sebastian
> 
> Am Mo., 28. Okt. 2019 um 07:16 Uhr schrieb Alexis King <lexi.lambda at gmail.com <mailto:lexi.lambda at gmail.com>>:
> Hi all,
> 
> I have an odd question: I’ve bumped into a clear simplifier bug, and although it only happens on GHC 8.6.5, not 8.8.1, I’d like to locate the change that fixed it. My library’s test suite currently fails on GHC 8.6.5 due to the bug, and I’d rather not force all my users to upgrade to 8.8 if I can help it, so I’m hoping to find a workaround.
> 
> The minimal test case I’ve found for the bug is this program:
> 
>     {-# LANGUAGE GeneralizedNewtypeDeriving, StandaloneDeriving, TypeFamilies #-}
> 
>     import Control.Exception
>     import Control.Monad.IO.Class
>     import Control.Monad.Trans.Identity
>     import Control.Monad.Trans.Reader
> 
>     class Monad m => MonadFoo m where
>       foo :: m a -> m a
>     instance MonadFoo IO where
>       foo m = onException m (pure ())
>     instance MonadFoo m => MonadFoo (ReaderT r m) where
>       foo m = ReaderT $ \r -> foo (runReaderT m r)
>     deriving instance MonadFoo m => MonadFoo (IdentityT m)
> 
>     type family F m where
>       F m = IdentityT m
> 
>     newtype FT m a = FT { runFT :: F m a }
>       deriving (Functor, Applicative, Monad, MonadIO, MonadFoo)
> 
>     main :: IO ()
>     main = run (foo (liftIO (throwIO (IndexOutOfBounds "bang"))))
>       where
>         run :: ReaderT () (FT (ReaderT () IO)) a -> IO a
>         run = flip runReaderT () . runIdentityT . runFT . flip runReaderT ()
> 
> Using GHC 8.6.5 on macOS 10.14.5, compiling this program with optimizations reliably triggers the -fcatch-bottoms sanitization:
> 
>     $ ghc -O -fcatch-bottoms weird.hs && ./weird
>     [1 of 1] Compiling Main             ( weird.hs, weird.o )
>     Linking weird ...
>     weird: Bottoming expression returned
> 
> What goes wrong? Somehow the generated core for this program includes the following:
> 
>     lvl_s47B :: SomeException
>     lvl_s47B = $fExceptionArrayException_$ctoException lvl_s483
> 
>     m_s47r :: () -> State# RealWorld -> (# State# RealWorld, () #)
>     m_s47r
>       = \ _ (eta_B1 :: State# RealWorld) -> raiseIO# lvl_s47B eta_B1
> 
>     main_s2Ww :: State# RealWorld -> (# State# RealWorld, () #)
>     main_s2Ww
>       = \ (eta_a2wK :: State# RealWorld) ->
>           catch# (case m_s47r `cast` <Co:33> of { }) raiseIO# eta_a2wK
> 
> This core is completely bogus: it assumes that m_s47r is bottom, but m_s47r is a top-level function! The program still passes -dcore-lint, unfortunately, as it is still well-typed. (Also, in case it helps: -ddump-simplifier-iterations shows that the buggy transformation occurs in the first iteration of the very first simplifier pass.)
> 
> I’ve been trying to figure out what change might have fixed this so that I can assess if it’s possible to work around, but I haven’t found anything obvious. I’ve been slowly `git bisect`ing to look for the commit that introduced the fix, but many of the commits I’ve tested cause unrelated panics on my machine, which has been exacerbating the problem of the slow recompilation times. I’m a little at wits’ end, but opening a bug report hasn’t felt right, since the bug does appear to already be fixed.
> 
> Does this issue ring any bells to anyone on this list? Is there a particular patch that landed between GHC 8.6.5 and GHC 8.8.1 that might have fixed this problem? If not, I’ll keep trying with `git bisect`, but I’d appreciate any pointers.
> 
> Thanks,
> Alexis
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs <http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191028/5de6d338/attachment.html>

From rae at richarde.dev  Mon Oct 28 21:37:24 2019
From: rae at richarde.dev (Richard Eisenberg)
Date: Mon, 28 Oct 2019 21:37:24 +0000
Subject: Status of the rich errors proposal
In-Reply-To: <CA+Nwh6d+9XufHw_=HkzRs6d-Y_-fQA_5=9sOdNZGuM692qokaQ@mail.gmail.com>
References: <CA+Nwh6d+9XufHw_=HkzRs6d-Y_-fQA_5=9sOdNZGuM692qokaQ@mail.gmail.com>
Message-ID: <8085E565-9B0D-46B2-8A16-0BE748633099@richarde.dev>

Ben Gamari is, I believe, leading the effort with that proposal. It continues to get attention but has not yet blossomed to something we're ready to debate widely. This is a hard nut to crack, and we'd like to get it right. So I guess all there is to say right now is that the effort is ongoing, but we don't have anything concrete to report at the moment.

Richard

> On Oct 23, 2019, at 7:52 AM, Javier Neira Sanchez <atreyu.bbb at gmail.com> wrote:
> 
> Hi, lately i am being collaborated in the haskell-ide-engine (hie) repo and one of the issues i get to fix was related with the parsing of ghc errors.
> It turns out that delimiters of terms in errors are different in windows (`term') and *nix (‘term’) systems and that drive to parse errors.
> Quite code actions and diagnostics are based in parsing them and they were broken in windows.
> I am afraid that the code is very brittle and close to human readable error messages. the actual code look like this:
> 
> -- | Extract a term from a compiler message.
> -- It looks for terms delimited between '‘' and '’' falling back to '`' and '\''
> -- (the used ones in Windows systems).
> extractTerm :: T.Text -> T.Text
> extractTerm txt =
>   case extract '‘' '’' txt of
>     ""  -> extract '`' '\'' txt -- Needed for windows
>     term -> term
>   where extract b e = T.dropWhile (== b)
>                     . T.dropWhileEnd (== e)
>                     . T.dropAround (\c -> c /= b && c /= e)
> 
> 
> or
> 
> extractImportableTerm :: T.Text -> Maybe (T.Text, SymbolImport SymbolType)
> extractImportableTerm dirtyMsg = do
>   (n, s) <- extractedTerm
>   let n' = T.strip n
>   return (n', s)
>   where
>     importMsg = S.headMay
>       -- Get rid of the rename suggestion parts
>       $ T.splitOn "Perhaps you meant "
>       $ T.replace "\n" " "
>       -- Get rid of trailing/leading whitespace on each individual line
>       $ T.unlines
>       $ map T.strip
>       $ T.lines
>       $ T.replace "* " "" -- Needed for Windows
>       $ T.replace "• " "" dirtyMsg
>     extractTerm prefix symTy =
>       importMsg
>           >>= T.stripPrefix prefix
>           >>= \name -> Just (name, Import symTy)
>     extractType b =
>       extractTerm ("Not in scope: type constructor or class " <> b) Type
>     extractedTerm = asum
>       [ extractTerm "Variable not in scope: " Symbol
>       , extractType "‘"
>       , extractType "`" -- Needed for windows
>       , extractTerm "Data constructor not in scope: " Constructor]
> 
> It is clearly unsatisfactory but hard to improve without changing the messages to make it more structured.
> Moreover any legitimate change on errors to make it better will likely break it.
> 
> After exposing my worries in the hie irc channel, @mpickering pointed out that it is already a proposal to improve error messages:
> 
> https://github.com/bgamari/ghc-proposals/blob/rich-errors-proposal/proposals/0000-rich-errors-proposal.rst <https://github.com/bgamari/ghc-proposals/blob/rich-errors-proposal/proposals/0000-rich-errors-proposal.rst>
> 
> that nicely will improve the state of things.
> 
> Otoh there are already a way to output ghc errors as json (see https://gitlab.haskell.org/ghc/ghc/issues/13190 <https://gitlab.haskell.org/ghc/ghc/issues/13190>). It contains valuable info about the error in specific fields but the message itself is in plain text.
> So merging both features will let tools to handle compiler errors without use the ghc api directly if needed.
> 
> what is the status of the proposal? hie an other tooling developers will welcome it very heartly.
> 
> Thanks in advance!
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191028/48be4099/attachment.html>

From alp at well-typed.com  Tue Oct 29 14:09:31 2019
From: alp at well-typed.com (Alp Mestanogullari)
Date: Tue, 29 Oct 2019 15:09:31 +0100
Subject: Status of the rich errors proposal
In-Reply-To: <CA+Nwh6d+9XufHw_=HkzRs6d-Y_-fQA_5=9sOdNZGuM692qokaQ@mail.gmail.com>
References: <CA+Nwh6d+9XufHw_=HkzRs6d-Y_-fQA_5=9sOdNZGuM692qokaQ@mail.gmail.com>
Message-ID: <b5a41989-deb7-e1e8-1d7b-e8c40f60e4c0@well-typed.com>

Hello Javier,

Ben and I have indeed been thinking about how to improve the situation, 
and have been talking to a few fellow Haskellers in the tooling/IDE 
space to get a first round of feedback on the ideas that we came up 
with. I will soon be writing down a proposal for one of those ideas, 
that is likely to work well for everyone (GHC devs, tooling authors and 
more generally GHC API consumers).

Once that document is ready and up on github, I wil make sure to send 
the link here on ghc-devs at .

On 23/10/2019 08:52, Javier Neira Sanchez wrote:
> Hi, lately i am being collaborated in the haskell-ide-engine (hie) 
> repo and one of the issues i get to fix was related with the parsing 
> of ghc errors.
> It turns out that delimiters of terms in errors are different in 
> windows (`term') and *nix (‘term’) systems and that drive to parse errors.
> Quite code actions and diagnostics are based in parsing them and they 
> were broken in windows.
> I am afraid that the code is very brittle and close to human readable 
> error messages. the actual code look like this:
>
> -- | Extract a term from a compiler message.
> -- It looks for terms delimited between '‘' and '’' falling back to 
> '`' and '\''
> -- (the used ones in Windows systems).
> extractTerm :: T.Text -> T.Text
> extractTerm txt =
>   case extract '‘' '’' txt of
>     ""  -> extract '`' '\'' txt -- Needed for windows
>     term -> term
>   where extract b e = T.dropWhile (== b)
>                     . T.dropWhileEnd (== e)
>                     . T.dropAround (\c -> c /= b && c /= e)
> or
>
> extractImportableTerm :: T.Text -> Maybe (T.Text, SymbolImport SymbolType)
> extractImportableTerm dirtyMsg = do
>   (n, s) <- extractedTerm
>   let n' = T.strip n
>   return (n', s)
>   where
>     importMsg = S.headMay
>       -- Get rid of the rename suggestion parts
>       $ T.splitOn "Perhaps you meant "
>       $ T.replace "\n" " "
>       -- Get rid of trailing/leading whitespace on each individual line
>       $ T.unlines
>       $ map T.strip
>       $ T.lines
>       $ T.replace "* " "" -- Needed for Windows
>       $ T.replace "• " "" dirtyMsg
>     extractTerm prefix symTy =
>       importMsg
>           >>= T.stripPrefix prefix
>           >>= \name -> Just (name, Import symTy)
>     extractType b =
>       extractTerm ("Not in scope: type constructor or class " <> b) Type
>     extractedTerm = asum
>       [ extractTerm "Variable not in scope: " Symbol
>       , extractType "‘"
>       , extractType "`" -- Needed for windows
>       , extractTerm "Data constructor not in scope: " Constructor]
>
> It is clearly unsatisfactory but hard to improve without changing the 
> messages to make it more structured.
> Moreover any legitimate change on errors to make it better will likely 
> break it.
>
> After exposing my worries in the hie irc channel, @mpickering pointed 
> out that it is already a proposal to improve error messages:
>
> https://github.com/bgamari/ghc-proposals/blob/rich-errors-proposal/proposals/0000-rich-errors-proposal.rst
>
> that nicely will improve the state of things.
>
> Otoh there are already a way to output ghc errors as json (see 
> https://gitlab.haskell.org/ghc/ghc/issues/13190). It contains valuable 
> info about the error in specific fields but the message itself is in 
> plain text.
> So merging both features will let tools to handle compiler errors 
> without use the ghc api directly if needed.
>
> what is the status of the proposal? hie an other tooling developers 
> will welcome it very heartly.
>
> Thanks in advance!
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-- 
Alp Mestanogullari, Haskell Consultant
Well-Typed LLP, https://www.well-typed.com/

Registered in England and Wales, OC335890
118 Wymering Mansions, Wymering Road, London, W9 2NF, England

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191029/40e26d5e/attachment.html>

From simonpj at microsoft.com  Wed Oct 30 08:25:19 2019
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Wed, 30 Oct 2019 08:25:19 +0000
Subject: Linking problem
In-Reply-To: <AM6PR83MB0279B04A9A7E2CE4E799CC6FAD610@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <AM6PR83MB0279B04A9A7E2CE4E799CC6FAD610@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <AM6PR83MB0279909AFCCF5F32C97F312EAD600@AM6PR83MB0279.EURPRD83.prod.outlook.com>

Devs
On the branch wip/inline-compulsory, I'm getting a strange link error:

simonpj at MSRC-3645512:~/tmp$ ~/code/HEAD-2/inplace/bin/ghc-stage1 Hello.hs

[1 of 1] Compiling Main             ( Hello.hs, Hello.o )

Linking Hello ...

/home/simonpj/code/HEAD-2/rts/dist/build/libHSrts.a(Updates.o):stg_bh_upd_frame_info: error: undefined reference to 'stg_marked_upd_frame_ret'

/home/simonpj/code/HEAD-2/rts/dist/build/libHSrts.a(StgMiscClosures.o):stg_CAF_BLACKHOLE_info: error: undefined reference to 'stg_BLACKHOLE_entry'

/home/simonpj/code/HEAD-2/rts/dist/build/libHSrts.a(StgMiscClosures.o):__stg_EAGER_BLACKHOLE_info: error: undefined reference to 'stg_BLACKHOLE_entry'

/home/simonpj/code/HEAD-2/rts/dist/build/libHSrts.a(Apply.o):stg_AP_STACK_info: error: undefined reference to 'stg_WHITEHOLE_entry'

/home/simonpj/code/HEAD-2/rts/dist/build/libHSrts.a(Exception.o):function stg_raisezh: error: undefined reference to 'stg_ap_n_ret'

/home/simonpj/code/HEAD-2/rts/dist/build/libHSrts.a(Exception.o):function stg_raisezh: error: undefined reference to 'stg_ap_pv_ret'

/home/simonpj/code/HEAD-2/rts/dist/build/libHSrts.a(AutoApply.o):stg_ap_v_info: error: undefined reference to 'stg_BCO_entry'

/home/simonpj/code/HEAD-2/rts/dist/build/libHSrts.a(AutoApply.o):stg_ap_f_info: error: undefined reference to 'stg_BCO_entry'

/home/simonpj/code/HEAD-2/rts/dist/build/libHSrts.a(AutoApply.o):stg_ap_d_info: error: undefined reference to 'stg_BCO_entry'

/home/simonpj/code/HEAD-2/rts/dist/build/libHSrts.a(AutoApply.o):stg_ap_l_info: error: undefined reference to 'stg_BCO_entry'

               ... etc...


But I have done nothing in the RTS!
If I try 'nm' I get

bash$ nm rts/dist/build/libHSrts.a | grep stg_ap_p_fast

                 U stg_ap_p_fast

                 U stg_ap_p_fast

0000000000003f58 T stg_ap_p_fast
So somehow the same symbol is defined and unresolved.  Really strange.
Can you help?  Thanks
Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191030/866475b9/attachment.html>

From takenobu.hs at gmail.com  Wed Oct 30 12:32:49 2019
From: takenobu.hs at gmail.com (Takenobu Tani)
Date: Wed, 30 Oct 2019 21:32:49 +0900
Subject: users-guide page for master branch
Message-ID: <CAPB_NvwZwY6PmHciF-ndwk_S6VtoLBvbY21EdKxCJEqGmU5XWA@mail.gmail.com>

Hi Ben, devs,

This is not in hurry.

It looks like the users-guide page for master branch [1] has not been
updated recently.

[1]: https://ghc.gitlab.haskell.org/ghc/doc/users_guide

Regards,
Takenobu

From arnaud.spiwack at tweag.io  Wed Oct 30 13:05:57 2019
From: arnaud.spiwack at tweag.io (Spiwack, Arnaud)
Date: Wed, 30 Oct 2019 14:05:57 +0100
Subject: Handling source locations in HsSyn via TTG
In-Reply-To: <B7F52998-8378-4849-BFD9-9C0144FE056B@richarde.dev>
References: <AM6PR83MB0279697E188BA6A33A14EDE6AD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <A8CFA060-0511-4280-A835-DFCE68438ECE@serokell.io>
 <AM6PR83MB0279B5D5F9565648F9877CBEAD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <2B448026-9E21-4A0A-82CD-E2D1C4E6EDF0@serokell.io>
 <B7F52998-8378-4849-BFD9-9C0144FE056B@richarde.dev>
Message-ID: <CAH6fHQmimJzABtdQr_Q7ke8yS=BEaw7juk1JCg70Z45ibWQGGA@mail.gmail.com>

I would like to submit a solution E, which is just a variant of D (or a
meshing together of D and B), but may have different pros and cons.

Keep the ping-pong style. But! Let Located take the pass as an argument.
Now, Located would be

data Located p a = L (XLoc p) a

We could define XLoc p to be a source span in GHC passes, and () in
Template Haskell.
------------------------------

I’m not arguing in favour of E, at this point: just submitting an
alternative. I don’t like A though: I’m assuming that if we are free to add
the L constructor or not, it will be forgotten often. This is the sort of
mistake which will be hard to catch before releases. It sounds like
unnecessary risk.

PS: Maybe a friendlier version of Located, in this solution E, could be

data Located a p = L (XLoc p) (a p)


On Mon, Oct 28, 2019 at 11:46 AM Richard Eisenberg <rae at richarde.dev> wrote:

> A nice property of Solution D (and I think this is a new observation) is
> that we could accommodate both located source and unlocated. For example:
>
> > data GhcPass (c :: Pass)   -- as today
> > data Pass = MkPass Phase HasLoc
> > data Phase = Parsed | Renamed | Typechecked
> > data HasLoc = YesLoc | NoLoc
> >
> > type instance WrapL (GhcPass (MkPass p YesLoc)) f = Located (f (GhcPass
> (MkPass p YesLoc)))
> > type instance WrapL (GhcPass (MkPass p NoLoc)) f = f (GhcPass (MkPass p
> NoLoc))
>
> I don't actually think this is a good idea, as it would mean making a
> bunch of functions polymorphic in the HasLoc parameter, which is
> syntactically annoying. But the existence of this approach suggests that
> the design scales.
>
> Regardless of what you think of this new idea, I'm in favor of Solution D.
> I like my types to stop me from making errors, and I'm willing to put up
> with the odd type error asking me to write `unLoc` as I work in order to
> avoid errors.
>
> Richard
>
> > On Oct 28, 2019, at 10:30 AM, Vladislav Zavialov <vladislav at serokell.io>
> wrote:
> >
> >> Are you arguing for Solution D?  Or are you proposing some new solution
> E?  I can't tell.
> >
> > I suspect that I’m arguing for Solution B, but it’s hard to tell because
> it’s not described in enough detail in the Wiki.
> >
> >> Easy
> >>
> >>      type instance WrapL ToolPass t = ...
> >>
> >> What am I missing?
> >
> >
> > This assumes that `WrapL` is an open type family. In this case, there’s
> no problem. The merge request description has the following definition of
> WrapL:
> >
> > type instance WrapL p (f :: * -> *) where
> >  WrapL (GhcPass p) f = Located (f (GhcPass p))
> >  WrapL p           f =          f p
> > type LPat p = WrapL p Pat
> >
> > That wouldn’t be extensible. However, if WrapL is open, then Solution D
> sounds good to me.
> >
> > - Vlad
> >
> >> On 28 Oct 2019, at 13:20, Simon Peyton Jones <simonpj at microsoft.com>
> wrote:
> >>
> >> Vlad
> >>
> >> Are you arguing for Solution D?  Or are you proposing some new solution
> E?  I can't tell.
> >>
> >>
> >> | As to merge request !1970, it isn’t good to special-case GhcPass in a
> >> | closed type family, making other tools second-class citizens. Let’s
> say I
> >> | have `MyToolPass`, how would I write an instance of `WrapL` for it?
> >>
> >> Easy
> >>
> >>      type instance WrapL ToolPass t = ...
> >>
> >> What am I missing?
> >>
> >> Simon
> >>
> >> | -----Original Message-----
> >> | From: Vladislav Zavialov <vladislav at serokell.io>
> >> | Sent: 28 October 2019 10:07
> >> | To: Simon Peyton Jones <simonpj at microsoft.com>
> >> | Cc: ghc-devs at haskell.org
> >> | Subject: Re: Handling source locations in HsSyn via TTG
> >> |
> >> | I care about this, and I maintain my viewpoint described in
> >> |
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.has
> >> | kell.org%2Fpipermail%2Fghc-devs%2F2019-
> >> | February%2F017080.html&amp;data=02%7C01%7Csimonpj%40microsoft.com
> %7C06c859
> >> |
> d8bc0f48c73cb208d75b8e895c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63
> >> |
> 7078540055975603&amp;sdata=FhkqfWGXaNX%2Fz4IiCcvoVCyzVsSAlyz6Y1dxEGUjX9I%3
> >> | D&amp;reserved=0
> >> |
> >> | I’m willing to implement this.
> >> |
> >> | As to merge request !1970, it isn’t good to special-case GhcPass in a
> >> | closed type family, making other tools second-class citizens. Let’s
> say I
> >> | have `MyToolPass`, how would I write an instance of `WrapL` for it?
> >> |
> >> | - Vlad
> >> |
> >> | > On 28 Oct 2019, at 12:31, Simon Peyton Jones via ghc-devs <ghc-
> >> | devs at haskell.org> wrote:
> >> | >
> >> | > Friends
> >> | >
> >> | > As you know
> >> | >
> >> | >  • We are trying to use “Trees That Grow” (TTG) to move HsSyn
> towards
> >> | a situation in which GHC is merely a client of a generic HsSyn data
> type
> >> | that can be used by clients other than GHC.
> >> | >  • One sticking point has been the question of attaching source
> >> | locations.  We used to have a “ping-pong” style, in which very node is
> >> | decorated with a source location, but that’s a bit more awkward in
> TTG.
> >> | >  • This wiki page outlines some choices, while ticket #15495 has a
> >> | lot of discussion.
> >> | >  • HEAD embodies Solution A.  But it has the disadvantage that the
> >> | type system doesn’t enforce locations to be present at all.   That has
> >> | undesirable consequences (eg ticket #17330)
> >> | >  • The proposal is to move to Solution D on that page; you can see
> >> | how it plays out in MR !1970.
> >> | >  • (I think Solutions B and C are non-starters by comparison.)
> >> | > If you care, please check out the design and the MR.   We can change
> >> | later, of course, but doing so changes a lot of code, including client
> >> | code, so we’d prefer not to.
> >> | >
> >> | > Let’s try to converge by the end of the week.
> >> | >
> >> | > Thanks
> >> | >
> >> | > Simon
> >> | >
> >> | >
> >> | >
> >> | >
> >> | >
> >> | > _______________________________________________
> >> | > ghc-devs mailing list
> >> | > ghc-devs at haskell.org
> >> | >
> >> |
> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
> >> | ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
> >> | devs&amp;data=02%7C01%7Csimonpj%40microsoft.com
> %7C06c859d8bc0f48c73cb208d7
> >> |
> 5b8e895c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637078540055975603&a
> >> |
> mp;sdata=CRYEYmjuAYYIhoAeTPbi%2FctCiWmkIjL6pKUTBgVBwo8%3D&amp;reserved=0
> >>
> >
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191030/a6b54203/attachment.html>

From sgraf1337 at gmail.com  Wed Oct 30 13:12:36 2019
From: sgraf1337 at gmail.com (Sebastian Graf)
Date: Wed, 30 Oct 2019 14:12:36 +0100
Subject: Handling source locations in HsSyn via TTG
In-Reply-To: <CAH6fHQmimJzABtdQr_Q7ke8yS=BEaw7juk1JCg70Z45ibWQGGA@mail.gmail.com>
References: <AM6PR83MB0279697E188BA6A33A14EDE6AD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <A8CFA060-0511-4280-A835-DFCE68438ECE@serokell.io>
 <AM6PR83MB0279B5D5F9565648F9877CBEAD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <2B448026-9E21-4A0A-82CD-E2D1C4E6EDF0@serokell.io>
 <B7F52998-8378-4849-BFD9-9C0144FE056B@richarde.dev>
 <CAH6fHQmimJzABtdQr_Q7ke8yS=BEaw7juk1JCg70Z45ibWQGGA@mail.gmail.com>
Message-ID: <CAAS+=P_nzKO+S5aja3P9r8ynnZGKo0ffF=cv=MYNCApgWrXv9w@mail.gmail.com>

> I would like to submit a solution E, which is just a variant of D (or a
meshing together of D and B), but may have different pros and cons.

I like this conceptually: No `WrapL`/`WrapX`/`XWrap`/`XRec` (the trouble to
find a fitting name already suggests that it's maybe a little too general a
concept), but rather a nicely targeted `XLoc`.

But I'm afraid that the unconditional ping-pong incurs an indirection even
if you instantiate `XLoc` to `()` all the time. I think that's a no-go,
according to the wiki page: Think of TH, which would pay needlessly.
Also it's not much better than just the old ping-pong style, where we could
always pass `noSrcLoc` instead of `()` for basically the same heap layout.

Am Mi., 30. Okt. 2019 um 14:06 Uhr schrieb Spiwack, Arnaud <
arnaud.spiwack at tweag.io>:

> I would like to submit a solution E, which is just a variant of D (or a
> meshing together of D and B), but may have different pros and cons.
>
> Keep the ping-pong style. But! Let Located take the pass as an argument.
> Now, Located would be
>
> data Located p a = L (XLoc p) a
>
> We could define XLoc p to be a source span in GHC passes, and () in
> Template Haskell.
> ------------------------------
>
> I’m not arguing in favour of E, at this point: just submitting an
> alternative. I don’t like A though: I’m assuming that if we are free to add
> the L constructor or not, it will be forgotten often. This is the sort of
> mistake which will be hard to catch before releases. It sounds like
> unnecessary risk.
>
> PS: Maybe a friendlier version of Located, in this solution E, could be
>
> data Located a p = L (XLoc p) (a p)
>
>
> On Mon, Oct 28, 2019 at 11:46 AM Richard Eisenberg <rae at richarde.dev>
> wrote:
>
>> A nice property of Solution D (and I think this is a new observation) is
>> that we could accommodate both located source and unlocated. For example:
>>
>> > data GhcPass (c :: Pass)   -- as today
>> > data Pass = MkPass Phase HasLoc
>> > data Phase = Parsed | Renamed | Typechecked
>> > data HasLoc = YesLoc | NoLoc
>> >
>> > type instance WrapL (GhcPass (MkPass p YesLoc)) f = Located (f (GhcPass
>> (MkPass p YesLoc)))
>> > type instance WrapL (GhcPass (MkPass p NoLoc)) f = f (GhcPass (MkPass p
>> NoLoc))
>>
>> I don't actually think this is a good idea, as it would mean making a
>> bunch of functions polymorphic in the HasLoc parameter, which is
>> syntactically annoying. But the existence of this approach suggests that
>> the design scales.
>>
>> Regardless of what you think of this new idea, I'm in favor of Solution
>> D. I like my types to stop me from making errors, and I'm willing to put up
>> with the odd type error asking me to write `unLoc` as I work in order to
>> avoid errors.
>>
>> Richard
>>
>> > On Oct 28, 2019, at 10:30 AM, Vladislav Zavialov <vladislav at serokell.io>
>> wrote:
>> >
>> >> Are you arguing for Solution D?  Or are you proposing some new
>> solution E?  I can't tell.
>> >
>> > I suspect that I’m arguing for Solution B, but it’s hard to tell
>> because it’s not described in enough detail in the Wiki.
>> >
>> >> Easy
>> >>
>> >>      type instance WrapL ToolPass t = ...
>> >>
>> >> What am I missing?
>> >
>> >
>> > This assumes that `WrapL` is an open type family. In this case, there’s
>> no problem. The merge request description has the following definition of
>> WrapL:
>> >
>> > type instance WrapL p (f :: * -> *) where
>> >  WrapL (GhcPass p) f = Located (f (GhcPass p))
>> >  WrapL p           f =          f p
>> > type LPat p = WrapL p Pat
>> >
>> > That wouldn’t be extensible. However, if WrapL is open, then Solution D
>> sounds good to me.
>> >
>> > - Vlad
>> >
>> >> On 28 Oct 2019, at 13:20, Simon Peyton Jones <simonpj at microsoft.com>
>> wrote:
>> >>
>> >> Vlad
>> >>
>> >> Are you arguing for Solution D?  Or are you proposing some new
>> solution E?  I can't tell.
>> >>
>> >>
>> >> | As to merge request !1970, it isn’t good to special-case GhcPass in a
>> >> | closed type family, making other tools second-class citizens. Let’s
>> say I
>> >> | have `MyToolPass`, how would I write an instance of `WrapL` for it?
>> >>
>> >> Easy
>> >>
>> >>      type instance WrapL ToolPass t = ...
>> >>
>> >> What am I missing?
>> >>
>> >> Simon
>> >>
>> >> | -----Original Message-----
>> >> | From: Vladislav Zavialov <vladislav at serokell.io>
>> >> | Sent: 28 October 2019 10:07
>> >> | To: Simon Peyton Jones <simonpj at microsoft.com>
>> >> | Cc: ghc-devs at haskell.org
>> >> | Subject: Re: Handling source locations in HsSyn via TTG
>> >> |
>> >> | I care about this, and I maintain my viewpoint described in
>> >> |
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.has
>> >> | kell.org%2Fpipermail%2Fghc-devs%2F2019-
>> >> | February%2F017080.html&amp;data=02%7C01%7Csimonpj%40microsoft.com
>> %7C06c859
>> >> |
>> d8bc0f48c73cb208d75b8e895c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63
>> >> |
>> 7078540055975603&amp;sdata=FhkqfWGXaNX%2Fz4IiCcvoVCyzVsSAlyz6Y1dxEGUjX9I%3
>> >> | D&amp;reserved=0
>> >> |
>> >> | I’m willing to implement this.
>> >> |
>> >> | As to merge request !1970, it isn’t good to special-case GhcPass in a
>> >> | closed type family, making other tools second-class citizens. Let’s
>> say I
>> >> | have `MyToolPass`, how would I write an instance of `WrapL` for it?
>> >> |
>> >> | - Vlad
>> >> |
>> >> | > On 28 Oct 2019, at 12:31, Simon Peyton Jones via ghc-devs <ghc-
>> >> | devs at haskell.org> wrote:
>> >> | >
>> >> | > Friends
>> >> | >
>> >> | > As you know
>> >> | >
>> >> | >  • We are trying to use “Trees That Grow” (TTG) to move HsSyn
>> towards
>> >> | a situation in which GHC is merely a client of a generic HsSyn data
>> type
>> >> | that can be used by clients other than GHC.
>> >> | >  • One sticking point has been the question of attaching source
>> >> | locations.  We used to have a “ping-pong” style, in which very node
>> is
>> >> | decorated with a source location, but that’s a bit more awkward in
>> TTG.
>> >> | >  • This wiki page outlines some choices, while ticket #15495 has a
>> >> | lot of discussion.
>> >> | >  • HEAD embodies Solution A.  But it has the disadvantage that the
>> >> | type system doesn’t enforce locations to be present at all.   That
>> has
>> >> | undesirable consequences (eg ticket #17330)
>> >> | >  • The proposal is to move to Solution D on that page; you can see
>> >> | how it plays out in MR !1970.
>> >> | >  • (I think Solutions B and C are non-starters by comparison.)
>> >> | > If you care, please check out the design and the MR.   We can
>> change
>> >> | later, of course, but doing so changes a lot of code, including
>> client
>> >> | code, so we’d prefer not to.
>> >> | >
>> >> | > Let’s try to converge by the end of the week.
>> >> | >
>> >> | > Thanks
>> >> | >
>> >> | > Simon
>> >> | >
>> >> | >
>> >> | >
>> >> | >
>> >> | >
>> >> | > _______________________________________________
>> >> | > ghc-devs mailing list
>> >> | > ghc-devs at haskell.org
>> >> | >
>> >> |
>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
>> >> | ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
>> >> | devs&amp;data=02%7C01%7Csimonpj%40microsoft.com
>> %7C06c859d8bc0f48c73cb208d7
>> >> |
>> 5b8e895c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637078540055975603&a
>> >> |
>> mp;sdata=CRYEYmjuAYYIhoAeTPbi%2FctCiWmkIjL6pKUTBgVBwo8%3D&amp;reserved=0
>> >>
>> >
>> > _______________________________________________
>> > ghc-devs mailing list
>> > ghc-devs at haskell.org
>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191030/ec367020/attachment.html>

From ben at well-typed.com  Wed Oct 30 13:27:16 2019
From: ben at well-typed.com (Ben Gamari)
Date: Wed, 30 Oct 2019 09:27:16 -0400
Subject: users-guide page for master branch
In-Reply-To: <CAPB_NvwZwY6PmHciF-ndwk_S6VtoLBvbY21EdKxCJEqGmU5XWA@mail.gmail.com>
References: <CAPB_NvwZwY6PmHciF-ndwk_S6VtoLBvbY21EdKxCJEqGmU5XWA@mail.gmail.com>
Message-ID: <13191C16-3659-47CD-AC59-E8F57897AD97@well-typed.com>

Indeed, this is because of the fact that the Windows build is currently failing due to hsc2hs issues. I have been ignoring the documentation build until the Windows issue has been resolved.

Thanks for checking though. Don't hesitate to mention this again if you notice the build is still failing after the Windows issue is fixed. 

Cheers,

- Ben 

On October 30, 2019 8:32:49 AM EDT, Takenobu Tani <takenobu.hs at gmail.com> wrote:
>Hi Ben, devs,
>
>This is not in hurry.
>
>It looks like the users-guide page for master branch [1] has not been
>updated recently.
>
>[1]: https://ghc.gitlab.haskell.org/ghc/doc/users_guide
>
>Regards,
>Takenobu
>_______________________________________________
>ghc-devs mailing list
>ghc-devs at haskell.org
>http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191030/516ae935/attachment.html>

From ben at smart-cactus.org  Wed Oct 30 13:47:50 2019
From: ben at smart-cactus.org (Ben Gamari)
Date: Wed, 30 Oct 2019 09:47:50 -0400
Subject: Linking problem
In-Reply-To: <AM6PR83MB0279909AFCCF5F32C97F312EAD600@AM6PR83MB0279.EURPRD83.prod.outlook.com>
References: <AM6PR83MB0279B04A9A7E2CE4E799CC6FAD610@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <AM6PR83MB0279909AFCCF5F32C97F312EAD600@AM6PR83MB0279.EURPRD83.prod.outlook.com>
Message-ID: <874kzqs5to.fsf@smart-cactus.org>

Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:

> Devs
> On the branch wip/inline-compulsory, I'm getting a strange link error:
>
I have a build running now to try to reproduce the issue.

The fact that stg_marked_upt_frame_ret isn't defined in Updates.o is
bizarre as the definition site of that symbol lives in that file.

I assume that this is a clean build?

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191030/fcc4ee00/attachment.sig>

From takenobu.hs at gmail.com  Wed Oct 30 13:57:03 2019
From: takenobu.hs at gmail.com (Takenobu Tani)
Date: Wed, 30 Oct 2019 22:57:03 +0900
Subject: users-guide page for master branch
In-Reply-To: <13191C16-3659-47CD-AC59-E8F57897AD97@well-typed.com>
References: <CAPB_NvwZwY6PmHciF-ndwk_S6VtoLBvbY21EdKxCJEqGmU5XWA@mail.gmail.com>
 <13191C16-3659-47CD-AC59-E8F57897AD97@well-typed.com>
Message-ID: <CAPB_NvzTRzkpqepOfSKRtcoXfNqPKZpOAzhK69GHuq+0X=1qtQ@mail.gmail.com>

Thanks for your explanation.
I'll check again after the Windows issue is fixed.

Thanks,
Takenobu

On Wed, Oct 30, 2019 at 10:27 PM Ben Gamari <ben at well-typed.com> wrote:
>
> Indeed, this is because of the fact that the Windows build is currently failing due to hsc2hs issues. I have been ignoring the documentation build until the Windows issue has been resolved.
>
> Thanks for checking though. Don't hesitate to mention this again if you notice the build is still failing after the Windows issue is fixed.
>
> Cheers,
>
> - Ben

From arnaud.spiwack at tweag.io  Wed Oct 30 14:11:38 2019
From: arnaud.spiwack at tweag.io (Spiwack, Arnaud)
Date: Wed, 30 Oct 2019 15:11:38 +0100
Subject: Handling source locations in HsSyn via TTG
In-Reply-To: <CAAS+=P_nzKO+S5aja3P9r8ynnZGKo0ffF=cv=MYNCApgWrXv9w@mail.gmail.com>
References: <AM6PR83MB0279697E188BA6A33A14EDE6AD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <A8CFA060-0511-4280-A835-DFCE68438ECE@serokell.io>
 <AM6PR83MB0279B5D5F9565648F9877CBEAD660@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <2B448026-9E21-4A0A-82CD-E2D1C4E6EDF0@serokell.io>
 <B7F52998-8378-4849-BFD9-9C0144FE056B@richarde.dev>
 <CAH6fHQmimJzABtdQr_Q7ke8yS=BEaw7juk1JCg70Z45ibWQGGA@mail.gmail.com>
 <CAAS+=P_nzKO+S5aja3P9r8ynnZGKo0ffF=cv=MYNCApgWrXv9w@mail.gmail.com>
Message-ID: <CAH6fHQndQ3JMwqydmtA=E+g1x7ykiNTxK4GJ7_dECbX90P0B3Q@mail.gmail.com>

Agreed: it doesn't affect the heap layout, just the types. My reading of
the wiki page was not that the heap indirection was the main motivation for
this little enquiry, though. Rather some type annoyance things. I don't
know that E is the best solution, or even a good solution: I just think it
deserves to be part of this conversation.

On Wed, Oct 30, 2019 at 2:12 PM Sebastian Graf <sgraf1337 at gmail.com> wrote:

> > I would like to submit a solution E, which is just a variant of D (or a
> meshing together of D and B), but may have different pros and cons.
>
> I like this conceptually: No `WrapL`/`WrapX`/`XWrap`/`XRec` (the trouble
> to find a fitting name already suggests that it's maybe a little too
> general a concept), but rather a nicely targeted `XLoc`.
>
> But I'm afraid that the unconditional ping-pong incurs an indirection even
> if you instantiate `XLoc` to `()` all the time. I think that's a no-go,
> according to the wiki page: Think of TH, which would pay needlessly.
> Also it's not much better than just the old ping-pong style, where we
> could always pass `noSrcLoc` instead of `()` for basically the same heap
> layout.
>
> Am Mi., 30. Okt. 2019 um 14:06 Uhr schrieb Spiwack, Arnaud <
> arnaud.spiwack at tweag.io>:
>
>> I would like to submit a solution E, which is just a variant of D (or a
>> meshing together of D and B), but may have different pros and cons.
>>
>> Keep the ping-pong style. But! Let Located take the pass as an argument.
>> Now, Located would be
>>
>> data Located p a = L (XLoc p) a
>>
>> We could define XLoc p to be a source span in GHC passes, and () in
>> Template Haskell.
>> ------------------------------
>>
>> I’m not arguing in favour of E, at this point: just submitting an
>> alternative. I don’t like A though: I’m assuming that if we are free to add
>> the L constructor or not, it will be forgotten often. This is the sort
>> of mistake which will be hard to catch before releases. It sounds like
>> unnecessary risk.
>>
>> PS: Maybe a friendlier version of Located, in this solution E, could be
>>
>> data Located a p = L (XLoc p) (a p)
>>
>>
>> On Mon, Oct 28, 2019 at 11:46 AM Richard Eisenberg <rae at richarde.dev>
>> wrote:
>>
>>> A nice property of Solution D (and I think this is a new observation) is
>>> that we could accommodate both located source and unlocated. For example:
>>>
>>> > data GhcPass (c :: Pass)   -- as today
>>> > data Pass = MkPass Phase HasLoc
>>> > data Phase = Parsed | Renamed | Typechecked
>>> > data HasLoc = YesLoc | NoLoc
>>> >
>>> > type instance WrapL (GhcPass (MkPass p YesLoc)) f = Located (f
>>> (GhcPass (MkPass p YesLoc)))
>>> > type instance WrapL (GhcPass (MkPass p NoLoc)) f = f (GhcPass (MkPass
>>> p NoLoc))
>>>
>>> I don't actually think this is a good idea, as it would mean making a
>>> bunch of functions polymorphic in the HasLoc parameter, which is
>>> syntactically annoying. But the existence of this approach suggests that
>>> the design scales.
>>>
>>> Regardless of what you think of this new idea, I'm in favor of Solution
>>> D. I like my types to stop me from making errors, and I'm willing to put up
>>> with the odd type error asking me to write `unLoc` as I work in order to
>>> avoid errors.
>>>
>>> Richard
>>>
>>> > On Oct 28, 2019, at 10:30 AM, Vladislav Zavialov <
>>> vladislav at serokell.io> wrote:
>>> >
>>> >> Are you arguing for Solution D?  Or are you proposing some new
>>> solution E?  I can't tell.
>>> >
>>> > I suspect that I’m arguing for Solution B, but it’s hard to tell
>>> because it’s not described in enough detail in the Wiki.
>>> >
>>> >> Easy
>>> >>
>>> >>      type instance WrapL ToolPass t = ...
>>> >>
>>> >> What am I missing?
>>> >
>>> >
>>> > This assumes that `WrapL` is an open type family. In this case,
>>> there’s no problem. The merge request description has the following
>>> definition of WrapL:
>>> >
>>> > type instance WrapL p (f :: * -> *) where
>>> >  WrapL (GhcPass p) f = Located (f (GhcPass p))
>>> >  WrapL p           f =          f p
>>> > type LPat p = WrapL p Pat
>>> >
>>> > That wouldn’t be extensible. However, if WrapL is open, then Solution
>>> D sounds good to me.
>>> >
>>> > - Vlad
>>> >
>>> >> On 28 Oct 2019, at 13:20, Simon Peyton Jones <simonpj at microsoft.com>
>>> wrote:
>>> >>
>>> >> Vlad
>>> >>
>>> >> Are you arguing for Solution D?  Or are you proposing some new
>>> solution E?  I can't tell.
>>> >>
>>> >>
>>> >> | As to merge request !1970, it isn’t good to special-case GhcPass in
>>> a
>>> >> | closed type family, making other tools second-class citizens. Let’s
>>> say I
>>> >> | have `MyToolPass`, how would I write an instance of `WrapL` for it?
>>> >>
>>> >> Easy
>>> >>
>>> >>      type instance WrapL ToolPass t = ...
>>> >>
>>> >> What am I missing?
>>> >>
>>> >> Simon
>>> >>
>>> >> | -----Original Message-----
>>> >> | From: Vladislav Zavialov <vladislav at serokell.io>
>>> >> | Sent: 28 October 2019 10:07
>>> >> | To: Simon Peyton Jones <simonpj at microsoft.com>
>>> >> | Cc: ghc-devs at haskell.org
>>> >> | Subject: Re: Handling source locations in HsSyn via TTG
>>> >> |
>>> >> | I care about this, and I maintain my viewpoint described in
>>> >> |
>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.has
>>> >> | kell.org%2Fpipermail%2Fghc-devs%2F2019-
>>> >> | February%2F017080.html&amp;data=02%7C01%7Csimonpj%40microsoft.com
>>> %7C06c859
>>> >> |
>>> d8bc0f48c73cb208d75b8e895c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63
>>> >> |
>>> 7078540055975603&amp;sdata=FhkqfWGXaNX%2Fz4IiCcvoVCyzVsSAlyz6Y1dxEGUjX9I%3
>>> >> | D&amp;reserved=0
>>> >> |
>>> >> | I’m willing to implement this.
>>> >> |
>>> >> | As to merge request !1970, it isn’t good to special-case GhcPass in
>>> a
>>> >> | closed type family, making other tools second-class citizens. Let’s
>>> say I
>>> >> | have `MyToolPass`, how would I write an instance of `WrapL` for it?
>>> >> |
>>> >> | - Vlad
>>> >> |
>>> >> | > On 28 Oct 2019, at 12:31, Simon Peyton Jones via ghc-devs <ghc-
>>> >> | devs at haskell.org> wrote:
>>> >> | >
>>> >> | > Friends
>>> >> | >
>>> >> | > As you know
>>> >> | >
>>> >> | >  • We are trying to use “Trees That Grow” (TTG) to move HsSyn
>>> towards
>>> >> | a situation in which GHC is merely a client of a generic HsSyn data
>>> type
>>> >> | that can be used by clients other than GHC.
>>> >> | >  • One sticking point has been the question of attaching source
>>> >> | locations.  We used to have a “ping-pong” style, in which very node
>>> is
>>> >> | decorated with a source location, but that’s a bit more awkward in
>>> TTG.
>>> >> | >  • This wiki page outlines some choices, while ticket #15495 has a
>>> >> | lot of discussion.
>>> >> | >  • HEAD embodies Solution A.  But it has the disadvantage that the
>>> >> | type system doesn’t enforce locations to be present at all.   That
>>> has
>>> >> | undesirable consequences (eg ticket #17330)
>>> >> | >  • The proposal is to move to Solution D on that page; you can see
>>> >> | how it plays out in MR !1970.
>>> >> | >  • (I think Solutions B and C are non-starters by comparison.)
>>> >> | > If you care, please check out the design and the MR.   We can
>>> change
>>> >> | later, of course, but doing so changes a lot of code, including
>>> client
>>> >> | code, so we’d prefer not to.
>>> >> | >
>>> >> | > Let’s try to converge by the end of the week.
>>> >> | >
>>> >> | > Thanks
>>> >> | >
>>> >> | > Simon
>>> >> | >
>>> >> | >
>>> >> | >
>>> >> | >
>>> >> | >
>>> >> | > _______________________________________________
>>> >> | > ghc-devs mailing list
>>> >> | > ghc-devs at haskell.org
>>> >> | >
>>> >> |
>>> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
>>> >> | ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
>>> >> | devs&amp;data=02%7C01%7Csimonpj%40microsoft.com
>>> %7C06c859d8bc0f48c73cb208d7
>>> >> |
>>> 5b8e895c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637078540055975603&a
>>> >> |
>>> mp;sdata=CRYEYmjuAYYIhoAeTPbi%2FctCiWmkIjL6pKUTBgVBwo8%3D&amp;reserved=0
>>> >>
>>> >
>>> > _______________________________________________
>>> > ghc-devs mailing list
>>> > ghc-devs at haskell.org
>>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191030/4657c40c/attachment.html>

From ben at smart-cactus.org  Wed Oct 30 14:34:32 2019
From: ben at smart-cactus.org (Ben Gamari)
Date: Wed, 30 Oct 2019 10:34:32 -0400
Subject: Linking problem
In-Reply-To: <874kzqs5to.fsf@smart-cactus.org>
References: <AM6PR83MB0279B04A9A7E2CE4E799CC6FAD610@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <AM6PR83MB0279909AFCCF5F32C97F312EAD600@AM6PR83MB0279.EURPRD83.prod.outlook.com>
 <874kzqs5to.fsf@smart-cactus.org>
Message-ID: <87y2x2qp3e.fsf@smart-cactus.org>

Ben Gamari <ben at smart-cactus.org> writes:

> Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:
>
>> Devs
>> On the branch wip/inline-compulsory, I'm getting a strange link error:
>>
> I have a build running now to try to reproduce the issue.
>
> The fact that stg_marked_upt_frame_ret isn't defined in Updates.o is
> bizarre as the definition site of that symbol lives in that file.
>
> I assume that this is a clean build?
>
Unfortunately I'm having a hard time reproducing this locally. I noticed
that you are using ghc-stage1. Which compiler are you using to
bootstrap?

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20191030/b3b2df74/attachment.sig>

From omeragacan at gmail.com  Thu Oct 31 18:45:09 2019
From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=)
Date: Thu, 31 Oct 2019 21:45:09 +0300
Subject: DriverPipeline/HscMain DynFlags mystery
Message-ID: <CAMQQO3=baFe2PHBtMzQ5wvcQV+NyKS1s9SR0=GvcPC=mpyA7qg@mail.gmail.com>

Hi,

We recently did some refactoring in HscMain and DriverPipeline to generate
interfaces after final Cmms are generated (previously interfaces would be
generated after the tidying pass). It's mostly done but there's one thing that I
couldn't figure out after two full days of debugging (I also asked a few people
about it on IRC), so I wanted to ask here in case someone here knows.

Previously the interface value (ModIface) would be generated and written to disk
in `HscMain.finish`. The DynFlags we use to generate the ModIface and to write
it to disk would be the one passed to `HscMain.hscIncrementalCompile`.

In the new implementation part of the interface is still generated in
`HscMain.hscIncrementalCompile` (see mkPartialIface), using the same DynFlags as
before. But more stuff is added after the final Cmms are generated (see
mkFullIface calls in DriverPipeline) using DynFlags in `compileOne'` or
`runPhase` (called by `runPipeline`). It turns out these DynFlags are different
enough from the one passed to `HscMain.hscIncrementalCompile` that some tests
fail (I remember a backpack test, but there may be more).

("Full" interfaces are written to disk right after generation)

See [1] for the hack I added as a workaround. Basically I keep the DynFlags
passed to hscIncrementalCompile so that I can generate the final interfaces
correctly.

The question is what's changing in DynFlags that makes things go wrong. I tried
looking at the fields used by mkFullIface and hscMaybeWriteIface, but as far as
I can see none of the fields used by these functions are different from the
DynFlags passed to hscIncrementalCompile.

If anyone knows what's going on any help would be appreciated.

Thanks,

Ömer

[1]: https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/main/HscTypes.hs#L255-259