From matthewtpickering at gmail.com  Fri Jul  1 11:05:59 2022
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Fri, 1 Jul 2022 12:05:59 +0100
Subject: Guide for testing hackage documentation upload
Message-ID: <CALuQ0m9mXMsxbSSLm_jc0KJxcwp=-p2mR6yjHCkyd71p+WT3BA@mail.gmail.com>

Hi all,

In case anyone needs this in the future or is interested then I wrote
a little guide about how to test the upload of hackage documentation
locally.

https://gitlab.haskell.org/ghc/ghc/-/wikis/testing-hackage-documentation

Cheers,

Matt

From matthewtpickering at gmail.com  Fri Jul  1 11:14:56 2022
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Fri, 1 Jul 2022 12:14:56 +0100
Subject: RST Documentation Trick for reference links
Message-ID: <CALuQ0m_2ccy7iBsOqtpvw2028+pqLEfMSwiFCeE=1idCffjH-w@mail.gmail.com>

Hello,

I noticed myself writing things like:

```
and enable the eventlog using :rts-flag:`-l ⟨flags⟩`.
```

When in fact what I wanted was to create a link the eventlog options
but render the link as just -l rather than -l <flags>.. but I thought
I had to write the whole thing to get the linter to not complain.

It turns out can achieve this goal using the following syntax:

```
and enable the eventlog using :rts-flag:`-l <-l ⟨flags⟩>`.
```

ie, include the thing you actually want to reference in angular
brackets < > and the text you want to render before that.

Yet another thing which has changed my life for good.

Cheers,

Matt

From maurer.benjamin at gmail.com  Sun Jul  3 18:02:12 2022
From: maurer.benjamin at gmail.com (Benjamin Maurer)
Date: Sun, 3 Jul 2022 20:02:12 +0200
Subject: Howto build optimized with stack traces/9.2.3 build broken?
Message-ID: <CAOZ2gfu36Lvx062GZStLEzimYAWxUhDi9fHEVXJ_sO=+qxLp3Q@mail.gmail.com>

Hey all,

I'm trying to debug a segfault caused by my optimization. GHC only
segfaults when compiled with -O2 and my flag, i.e., I used the "perf"
flavour:

> ./hadrian/build -j "stage1.*.ghc.hs.opts += -fregs-ssa"  --flavour=perf

However, I can't get a stack trace.
[1] suggests I need a profiling build for that. So I tried:
> ./hadrian/build -j "stage1.*.ghc.hs.opts += -fregs-ssa"
 --flavour=perf+debug_info+profiled_ghc+no_dynamic_ghc

Unfortunately, I'm using GHC 9.2.1 from ghcup to bootstrap and this version
doesn't come with profiling enabled base [2].

8.10.7, which I have installed, can't compile ghc anymore (it seems). So I
tried 9.2.3 from ghcup and that gives me the strangest errors. Basically,
the compiler seems completely bricked.
I installed 9.2.3, ran "hadrian/build clean", "./configure && ./boot" and
the same line as above with both "default" and "perf" flavours.

When I try to compile nofib/spectral/simple, I get:
> /home/ben/Projekte/haskell/ghc/_build/stage1/bin/ghc -O2 Main.hs
> [1 of 2] Compiling Main             ( Main.hs, Main.o )

> Main.hs:1125:1: error: lexical error (UTF-8 decoding error)

(Main.hs only has 1124 lines though, file was not modified, compiles fine
with any other ghc version)

But most programs I tried give me:
> /home/ben/Projekte/haskell/ghc/_build/stage1/bin/ghc -O2 Main.hs
> ghc: Unable to commit 283468890112 bytes of memory
> ghc: Exiting. The system might be out of memory.

Oh kay. I got 16GB RAM, 10 gig available. GHC 9.2.3 works fine.

Is my flavour combination bad, or what is broken here?
Any other (better) way to get an optimized build with all the debugging
aids I can get?
I heard GHC 9.2.2 is a broken release, or does anyone know whether it works
on Linux x86_64 (and comes with profiling libs from ghcup) - maybe I can
get a working version using that?

I'd try, but creating a new build takes almost an hour and I have spent
most of the day trying to solve this. I am puzzled and desearate to fix my
bug and get a working compiler.

Appreciate any help!

Best Regards,
Benjamin


[1]
https://gitlab.haskell.org/ghc/ghc/-/blob/master/hadrian/doc/debugging.md
[2] https://gitlab.haskell.org/haskell/ghcup-hs/-/issues/296
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220703/3bda9732/attachment.html>

From matthewtpickering at gmail.com  Tue Jul  5 09:39:21 2022
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Tue, 5 Jul 2022 10:39:21 +0100
Subject: 9.4.1 Final Call
Message-ID: <CALuQ0m-oiN3V+3xg5E+iFLS=Y+BXGy04pF51w5tnL6aNv5rEBw@mail.gmail.com>

Hello all,

This is the final call for patches which need to be included in 9.4.1.

The status of which I do not know the answer:

* GHC 9.4 infinite loop in typechecker - #21530 (@simonpj, @rae) ?

To my knowledge the remaining unfinished work list is:

* Resolve static linking issues with text-2.0 - #21787 (Unresolved)
(Matt/Ben/Zubin/Tamar/Andreww)
* CAF reachability bug with -fprof-late - !7797 (WIP) (Andreas/Ben)
* XMonad segfault - #21708 (WIP) (Ben/Andreas)
* HIE File Fixes - !7888 (landing) (Zubin)
* template-haskell-2.19.0.0 depends on filepath - !8516 (landing) (Matt)
* Module loops and multiple home units leads to a panic - !8573
(nearly ready to land) (Matt)
* extendMG leaks memory - #21816 (Zubin/Matt)

To my knowledge, the following are not making it into 9.4.1.. but are
not critical to revert:

* Tidy up withDict and friends - #21568

Looking like a great release.

Matt

From alan.zimm at gmail.com  Tue Jul  5 10:37:25 2022
From: alan.zimm at gmail.com (Alan & Kim Zimmerman)
Date: Tue, 5 Jul 2022 11:37:25 +0100
Subject: 9.4.1 Final Call
In-Reply-To: <CALuQ0m-oiN3V+3xg5E+iFLS=Y+BXGy04pF51w5tnL6aNv5rEBw@mail.gmail.com>
References: <CALuQ0m-oiN3V+3xg5E+iFLS=Y+BXGy04pF51w5tnL6aNv5rEBw@mail.gmail.com>
Message-ID: <CANma=H9hziasDxy0CZdTeU2zv-8_H2DYT38Le3JxBAtfxZ-LBw@mail.gmail.com>

I have a last exact print annotation diff, picked up from hackage
roundtripping: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/8562

Alan

On Tue, 05 Jul 2022, 10:39 Matthew Pickering, <matthewtpickering at gmail.com>
wrote:

> Hello all,
>
> This is the final call for patches which need to be included in 9.4.1.
>
> The status of which I do not know the answer:
>
> * GHC 9.4 infinite loop in typechecker - #21530 (@simonpj, @rae) ?
>
> To my knowledge the remaining unfinished work list is:
>
> * Resolve static linking issues with text-2.0 - #21787 (Unresolved)
> (Matt/Ben/Zubin/Tamar/Andreww)
> * CAF reachability bug with -fprof-late - !7797 (WIP) (Andreas/Ben)
> * XMonad segfault - #21708 (WIP) (Ben/Andreas)
> * HIE File Fixes - !7888 (landing) (Zubin)
> * template-haskell-2.19.0.0 depends on filepath - !8516 (landing) (Matt)
> * Module loops and multiple home units leads to a panic - !8573
> (nearly ready to land) (Matt)
> * extendMG leaks memory - #21816 (Zubin/Matt)
>
> To my knowledge, the following are not making it into 9.4.1.. but are
> not critical to revert:
>
> * Tidy up withDict and friends - #21568
>
> Looking like a great release.
>
> Matt
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220705/9f2b672c/attachment.html>

From ryan.gl.scott at gmail.com  Tue Jul  5 12:55:31 2022
From: ryan.gl.scott at gmail.com (Ryan Scott)
Date: Tue, 5 Jul 2022 08:55:31 -0400
Subject: 9.4.1 Final Call
Message-ID: <CADdQpRB442hXjXv74k8R9=_a_Z6PEOk1h6oibv2TYbKzhRZ=rw@mail.gmail.com>

One thing that wasn't included in the list but arguably should be is #21618
(Loading libc++ in ghci results in segfaults). As far as I am aware, this
renders GHC 9.4 unusable on Windows for libraries that link against C++.
(Or, at the very least, I haven't figured out a way to make it work.)

Best,

Ryan S.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220705/edb0b7e0/attachment.html>

From simon.peytonjones at gmail.com  Mon Jul 11 15:08:14 2022
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Mon, 11 Jul 2022 16:08:14 +0100
Subject: Hadrian
Message-ID: <CAJKmMz__kZWABigB_MwjAzfLRpS_WfDwPv1f4kH+j0MxAnwqBA@mail.gmail.com>

I am working on a branch of GHC, actually on !8210.  I have rebased on
master.  Then I say
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220711/d0847b11/attachment.html>

From simon.peytonjones at gmail.com  Mon Jul 11 15:09:56 2022
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Mon, 11 Jul 2022 16:09:56 +0100
Subject: Hadrian
In-Reply-To: <CAJKmMz__kZWABigB_MwjAzfLRpS_WfDwPv1f4kH+j0MxAnwqBA@mail.gmail.com>
References: <CAJKmMz__kZWABigB_MwjAzfLRpS_WfDwPv1f4kH+j0MxAnwqBA@mail.gmail.com>
Message-ID: <CAJKmMz_ifwt_2J3jL5=H6dD1=eGdXDhmtD8iYDecJO4zRJrFjA@mail.gmail.com>

 (apols for premature send)

I am working on a branch of GHC, actually on !8210.  I have rebased on
master.  Then I say
hadrian/build
and I get the log below.  It falls over saying
No generator for _build/stage0/compiler/build/GHC/Unit/Module/Name.hs.

Surely that should not happen?

I'll try make clean; but isn't this a bug?

Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220711/3353b161/attachment.html>

From sylvain at haskus.fr  Mon Jul 11 15:18:41 2022
From: sylvain at haskus.fr (Sylvain Henry)
Date: Mon, 11 Jul 2022 17:18:41 +0200
Subject: Hadrian
In-Reply-To: <CAJKmMz_ifwt_2J3jL5=H6dD1=eGdXDhmtD8iYDecJO4zRJrFjA@mail.gmail.com>
References: <CAJKmMz__kZWABigB_MwjAzfLRpS_WfDwPv1f4kH+j0MxAnwqBA@mail.gmail.com>
 <CAJKmMz_ifwt_2J3jL5=H6dD1=eGdXDhmtD8iYDecJO4zRJrFjA@mail.gmail.com>
Message-ID: <0af1f131-95e0-2ae7-c4ee-d743405c4d8c@haskus.fr>

Hi Simon,

You have to re-run `./configure` in cases like this. It's because 
`compiler/ghc.cabal` is generated from `compiler/ghc.cabal.in` by 
`./configure`. This isn't tracked by Hadrian.

 >Surely that should not happen?I'll try make clean; but isn't this a bug?

Hopefully when `make` build system will be removed it should be easy to 
make Hadrian (instead of `./configure`) generate and track this file. In 
fact I already did this in a MR more than a year ago but it was blocked 
on make-removal.

Sylvain


On 11/07/2022 17:09, Simon Peyton Jones wrote:
> (apols for premature send)
>
> I am working on a branch of GHC, actually on !8210.  I have rebased on 
> master.  Then I say
> hadrian/build
> and I get the log below.  It falls over saying
> No generator for _build/stage0/compiler/build/GHC/Unit/Module/Name.hs.
>
> Surely that should not happen?
>
> I'll try make clean; but isn't this a bug?
>
> Simon
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220711/e228d306/attachment.html>

From zubin at well-typed.com  Mon Jul 11 15:21:14 2022
From: zubin at well-typed.com (Zubin Duggal)
Date: Mon, 11 Jul 2022 20:51:14 +0530
Subject: Hadrian
In-Reply-To: <CAJKmMz_ifwt_2J3jL5=H6dD1=eGdXDhmtD8iYDecJO4zRJrFjA@mail.gmail.com>
References: <CAJKmMz__kZWABigB_MwjAzfLRpS_WfDwPv1f4kH+j0MxAnwqBA@mail.gmail.com>
 <CAJKmMz_ifwt_2J3jL5=H6dD1=eGdXDhmtD8iYDecJO4zRJrFjA@mail.gmail.com>
Message-ID: <20220711152114.tswpdpsu27upm3vp@zubin-msi>

It seems like GHC.Unit.Module.Name was recently removed/renamed in
commit c43dbac08b0d56406fe13de1e9b49c944f478b4a (Refactor ModuleName to
L.H.S.Module.Name).

What is probably going on is that as that file no longer exists in the
compiler/ source directory, but presumably some references to it still
exist in your hadrian _build directory, hadrian believes it a generated
file (like GHC.Settings.Config or 

All you need to fix this is to delete the _build directory, which is
the hadrian build root you are using. It is advisable to do this after
rebasing, especially if the rebase involves changes to the module
structure or build system.

On 22/07/11 16:09, Simon Peyton Jones wrote:
> (apols for premature send)
>
>I am working on a branch of GHC, actually on !8210.  I have rebased on
>master.  Then I say
>hadrian/build
>and I get the log below.  It falls over saying
>No generator for _build/stage0/compiler/build/GHC/Unit/Module/Name.hs.
>
>Surely that should not happen?
>
>I'll try make clean; but isn't this a bug?
>
>Simon

>_______________________________________________
>ghc-devs mailing list
>ghc-devs at haskell.org
>http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From simon.peytonjones at gmail.com  Mon Jul 11 15:25:05 2022
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Mon, 11 Jul 2022 16:25:05 +0100
Subject: Hadrian
In-Reply-To: <0af1f131-95e0-2ae7-c4ee-d743405c4d8c@haskus.fr>
References: <CAJKmMz__kZWABigB_MwjAzfLRpS_WfDwPv1f4kH+j0MxAnwqBA@mail.gmail.com>
 <CAJKmMz_ifwt_2J3jL5=H6dD1=eGdXDhmtD8iYDecJO4zRJrFjA@mail.gmail.com>
 <0af1f131-95e0-2ae7-c4ee-d743405c4d8c@haskus.fr>
Message-ID: <CAJKmMz_-gZGY8ZaFq+xSwAaTUahfkhwUgHZQjoBGZhFLsZ-sKg@mail.gmail.com>

>
> It is advisable to do this after
> rebasing, especially if the rebase involves changes to the module
> structure or build system.


But *any* commit could involve such a change.  Isn't it a goal of Hadrian
to respond to, well, changes?   For example, it is clever enough to rebuild
Hadrian itself when it changes.

It's a bit unsatisfying to say "it'll usually work, but if anything strange
happens, just make clean and try again".  Understandable perhaps, but not
satisfying.

Simon

On Mon, 11 Jul 2022 at 16:19, Sylvain Henry <sylvain at haskus.fr> wrote:

> Hi Simon,
>
> You have to re-run `./configure` in cases like this. It's because
> `compiler/ghc.cabal` is generated from `compiler/ghc.cabal.in` by
> `./configure`. This isn't tracked by Hadrian.
>
> >Surely that should not happen? I'll try make clean; but isn't this a bug?
>
> Hopefully when `make` build system will be removed it should be easy to
> make Hadrian (instead of `./configure`) generate and track this file. In
> fact I already did this in a MR more than a year ago but it was blocked on
> make-removal.
>
> Sylvain
>
>
> On 11/07/2022 17:09, Simon Peyton Jones wrote:
>
> (apols for premature send)
>
> I am working on a branch of GHC, actually on !8210.  I have rebased on
> master.  Then I say
> hadrian/build
> and I get the log below.  It falls over saying
> No generator for _build/stage0/compiler/build/GHC/Unit/Module/Name.hs.
>
> Surely that should not happen?
>
> I'll try make clean; but isn't this a bug?
>
> Simon
>
> _______________________________________________
> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220711/bd4d41cc/attachment.html>

From zubin at well-typed.com  Mon Jul 11 15:26:18 2022
From: zubin at well-typed.com (Zubin Duggal)
Date: Mon, 11 Jul 2022 20:56:18 +0530
Subject: Hadrian
In-Reply-To: <20220711152114.tswpdpsu27upm3vp@zubin-msi>
References: <CAJKmMz__kZWABigB_MwjAzfLRpS_WfDwPv1f4kH+j0MxAnwqBA@mail.gmail.com>
 <CAJKmMz_ifwt_2J3jL5=H6dD1=eGdXDhmtD8iYDecJO4zRJrFjA@mail.gmail.com>
 <20220711152114.tswpdpsu27upm3vp@zubin-msi>
Message-ID: <20220711152618.jy7ltamfsjw3ezus@zubin-msi>

And like Sylvain says, you also do need to reconfigure.

On 22/07/11 20:51, Zubin Duggal wrote:
>It seems like GHC.Unit.Module.Name was recently removed/renamed in
>commit c43dbac08b0d56406fe13de1e9b49c944f478b4a (Refactor ModuleName to
>L.H.S.Module.Name).
>
>What is probably going on is that as that file no longer exists in the
>compiler/ source directory, but presumably some references to it still
>exist in your hadrian _build directory, hadrian believes it a generated
>file (like GHC.Settings.Config or
>
>All you need to fix this is to delete the _build directory, which is
>the hadrian build root you are using. It is advisable to do this after
>rebasing, especially if the rebase involves changes to the module
>structure or build system.
>
>On 22/07/11 16:09, Simon Peyton Jones wrote:
>>(apols for premature send)
>>
>>I am working on a branch of GHC, actually on !8210.  I have rebased on
>>master.  Then I say
>>hadrian/build
>>and I get the log below.  It falls over saying
>>No generator for _build/stage0/compiler/build/GHC/Unit/Module/Name.hs.
>>
>>Surely that should not happen?
>>
>>I'll try make clean; but isn't this a bug?
>>
>>Simon
>
>>_______________________________________________
>>ghc-devs mailing list
>>ghc-devs at haskell.org
>>http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>

From zubin at well-typed.com  Mon Jul 11 15:36:20 2022
From: zubin at well-typed.com (Zubin Duggal)
Date: Mon, 11 Jul 2022 21:06:20 +0530
Subject: Hadrian
In-Reply-To: <CAJKmMz_-gZGY8ZaFq+xSwAaTUahfkhwUgHZQjoBGZhFLsZ-sKg@mail.gmail.com>
References: <CAJKmMz__kZWABigB_MwjAzfLRpS_WfDwPv1f4kH+j0MxAnwqBA@mail.gmail.com>
 <CAJKmMz_ifwt_2J3jL5=H6dD1=eGdXDhmtD8iYDecJO4zRJrFjA@mail.gmail.com>
 <0af1f131-95e0-2ae7-c4ee-d743405c4d8c@haskus.fr>
 <CAJKmMz_-gZGY8ZaFq+xSwAaTUahfkhwUgHZQjoBGZhFLsZ-sKg@mail.gmail.com>
Message-ID: <20220711153620.fytbglgt7ifam3lz@zubin-msi>

Yes, but as Sylvain noted, the configure step (which generates ghc.cabal
and so is particularly sensitive to changes in module structure) is not
tracked by hadrian at the moment. We could in principle fix this, and
as also noted by Sylvain this will become much easier once Make no longer
exists.

On 22/07/11 16:25, Simon Peyton Jones wrote:
>>
>> It is advisable to do this after
>> rebasing, especially if the rebase involves changes to the module
>> structure or build system.
>
>
>But *any* commit could involve such a change.  Isn't it a goal of Hadrian
>to respond to, well, changes?   For example, it is clever enough to rebuild
>Hadrian itself when it changes.
>
>It's a bit unsatisfying to say "it'll usually work, but if anything strange
>happens, just make clean and try again".  Understandable perhaps, but not
>satisfying.
>
>Simon
>
>On Mon, 11 Jul 2022 at 16:19, Sylvain Henry <sylvain at haskus.fr> wrote:
>
>> Hi Simon,
>>
>> You have to re-run `./configure` in cases like this. It's because
>> `compiler/ghc.cabal` is generated from `compiler/ghc.cabal.in` by
>> `./configure`. This isn't tracked by Hadrian.
>>
>> >Surely that should not happen? I'll try make clean; but isn't this a bug?
>>
>> Hopefully when `make` build system will be removed it should be easy to
>> make Hadrian (instead of `./configure`) generate and track this file. In
>> fact I already did this in a MR more than a year ago but it was blocked on
>> make-removal.
>>
>> Sylvain
>>
>>
>> On 11/07/2022 17:09, Simon Peyton Jones wrote:
>>
>> (apols for premature send)
>>
>> I am working on a branch of GHC, actually on !8210.  I have rebased on
>> master.  Then I say
>> hadrian/build
>> and I get the log below.  It falls over saying
>> No generator for _build/stage0/compiler/build/GHC/Unit/Module/Name.hs.
>>
>> Surely that should not happen?
>>
>> I'll try make clean; but isn't this a bug?
>>
>> Simon
>>
>> _______________________________________________
>> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>

>_______________________________________________
>ghc-devs mailing list
>ghc-devs at haskell.org
>http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From simon.peytonjones at gmail.com  Mon Jul 11 15:39:47 2022
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Mon, 11 Jul 2022 16:39:47 +0100
Subject: Hadrian
In-Reply-To: <20220711153620.fytbglgt7ifam3lz@zubin-msi>
References: <CAJKmMz__kZWABigB_MwjAzfLRpS_WfDwPv1f4kH+j0MxAnwqBA@mail.gmail.com>
 <CAJKmMz_ifwt_2J3jL5=H6dD1=eGdXDhmtD8iYDecJO4zRJrFjA@mail.gmail.com>
 <0af1f131-95e0-2ae7-c4ee-d743405c4d8c@haskus.fr>
 <CAJKmMz_-gZGY8ZaFq+xSwAaTUahfkhwUgHZQjoBGZhFLsZ-sKg@mail.gmail.com>
 <20220711153620.fytbglgt7ifam3lz@zubin-msi>
Message-ID: <CAJKmMz-=obo9qp7mtu1wp_DRgWddw8zGNSr+1uJuc7heroRtzA@mail.gmail.com>

OK.  So it's a bug.  Do we have an open Hadrian ticket to track it?  If
not, might you open one?

Make *already *no longer exists, incidentally.   Nowadays we need ghc-9.2
to build HEAD, and make doesn't work with ghc-9.2 for some reason.   So I
believe that make is dead for HEAD at least.

Simon

On Mon, 11 Jul 2022 at 16:36, Zubin Duggal <zubin at well-typed.com> wrote:

> Yes, but as Sylvain noted, the configure step (which generates ghc.cabal
> and so is particularly sensitive to changes in module structure) is not
> tracked by hadrian at the moment. We could in principle fix this, and
> as also noted by Sylvain this will become much easier once Make no longer
> exists.
>
> On 22/07/11 16:25, Simon Peyton Jones wrote:
> >>
> >> It is advisable to do this after
> >> rebasing, especially if the rebase involves changes to the module
> >> structure or build system.
> >
> >
> >But *any* commit could involve such a change.  Isn't it a goal of Hadrian
> >to respond to, well, changes?   For example, it is clever enough to
> rebuild
> >Hadrian itself when it changes.
> >
> >It's a bit unsatisfying to say "it'll usually work, but if anything
> strange
> >happens, just make clean and try again".  Understandable perhaps, but not
> >satisfying.
> >
> >Simon
> >
> >On Mon, 11 Jul 2022 at 16:19, Sylvain Henry <sylvain at haskus.fr> wrote:
> >
> >> Hi Simon,
> >>
> >> You have to re-run `./configure` in cases like this. It's because
> >> `compiler/ghc.cabal` is generated from `compiler/ghc.cabal.in` by
> >> `./configure`. This isn't tracked by Hadrian.
> >>
> >> >Surely that should not happen? I'll try make clean; but isn't this a
> bug?
> >>
> >> Hopefully when `make` build system will be removed it should be easy to
> >> make Hadrian (instead of `./configure`) generate and track this file. In
> >> fact I already did this in a MR more than a year ago but it was blocked
> on
> >> make-removal.
> >>
> >> Sylvain
> >>
> >>
> >> On 11/07/2022 17:09, Simon Peyton Jones wrote:
> >>
> >> (apols for premature send)
> >>
> >> I am working on a branch of GHC, actually on !8210.  I have rebased on
> >> master.  Then I say
> >> hadrian/build
> >> and I get the log below.  It falls over saying
> >> No generator for _build/stage0/compiler/build/GHC/Unit/Module/Name.hs.
> >>
> >> Surely that should not happen?
> >>
> >> I'll try make clean; but isn't this a bug?
> >>
> >> Simon
> >>
> >> _______________________________________________
> >> ghc-devs mailing listghc-devs at haskell.orghttp://
> mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >>
> >> _______________________________________________
> >> ghc-devs mailing list
> >> ghc-devs at haskell.org
> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >>
>
> >_______________________________________________
> >ghc-devs mailing list
> >ghc-devs at haskell.org
> >http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220711/9d6e94e8/attachment.html>

From bryan at haskell.foundation  Tue Jul 12 11:03:29 2022
From: bryan at haskell.foundation (Bryan Richter)
Date: Tue, 12 Jul 2022 14:03:29 +0300
Subject: Tracking intermittently failing CI jobs
In-Reply-To: <czzKVKXJAL4F1htJPSgwSH5ISqY8AtP8bFDHwoyYEwJx5k5AYviwVUE7uzymbikuf4w03da1wAHG6O0MS6EXqCt9GQtWeKR4If4hD8WhmxY=@chreekat.net>
References: <oNDoXkEp8XlDrJeARWq_yXLb4iae_jjUo4MH287A43KpOLQH0bc46v_29wMK66wZMbWzJta1BIRr32-LvdO3jQ==@protonmail.internalid>
 <czzKVKXJAL4F1htJPSgwSH5ISqY8AtP8bFDHwoyYEwJx5k5AYviwVUE7uzymbikuf4w03da1wAHG6O0MS6EXqCt9GQtWeKR4If4hD8WhmxY=@chreekat.net>
Message-ID: <9364d047-07ff-dffe-98ff-0438931670a8@haskell.foundation>

Hello again,

Thanks to everyone who pointed out spurious failures over the last few 
weeks. Here's the current state of affairs and some discussion on next 
steps.

*
*

*Dashboard
***

I made a dashboard for tracking spurious failures:

https://grafana.gitlab.haskell.org/d/167r9v6nk/ci-spurious-failures?orgId=2

I created this for three reasons:

 1. Keep tabs on new occurrences of spurious failures
 2. Understand which problems are causing the most issues
 3. Measure the effectiveness of any intervention

The dashboard still needs development, but it can already be used to 
show that the number of "Cannot connect to Docker daemon" failures has 
been reduced.

*
*

*Characterizing and Fixing Failures*

I have preliminary results on a few failure types. For instance, I used 
the "docker" type of failure to bootstrap the dashboard. Along with 
"Killed with signal 9", it seems to indicate a problem with the CI 
runner, itself.

To look more deeply into these types of runner-system failures, *I will 
need more access*. If you are responsible for some runners and you're 
comfortable giving me shell access, you can find my public ssh key at 
https://gitlab.haskell.org/-/snippets/5546. (Posted as a snippet so at 
least you know the key comes from somebody who can access my GitLab 
account. Other secure means of communication are listed at 
https://keybase.io/chreekat.) Send me a message if you do so.

Besides runner problems, there are spurious failures that may have more 
to do with the CI code, itself. They include some problem with 
environment variables and (probably) some issue with console buffering. 
Neither of these are being tracked on the dashboard yet. Many other 
problems are yet to be explored at all.


*Next Steps*

The theme for the next steps is finalizing the dashboard and 
characterizing more failures.

  * Track more failure types on the dashboard
  * Improve the process of backfilling failure data on the dashboard
  * Include more metadata (like project id!) on the dashboard so it's
    easier to zoom on failures
  * Document the dashboard and the processes that populate it for posterity
  * Diagnose runner-system failures (if accessible)
  * Continue exploring other failure types
  * Fix failures omg!?

The list of next steps is currently heavy on finalizing the dashboard 
and light on fixing spurious failures. I know that might be frustrating. 
My justification is that CI is a complex hardware/software/human system 
under continuous operation where most the low-hanging fruit have already 
been plucked. It's time to get serious. :) My goal is to make spurious 
failures surprising rather than commonplace. This is the best way I know 
to achieve that.

Thanks again for helping me with this goal. :)


-Bryan

P.S. If you're interested, I've been posting updates like this one on 
Discourse:

https://discourse.haskell.org/search?q=DevOps%20Weekly%20Log%20%23haskell-foundation%20order%3Alatest_topic


On 18/05/2022 13:25, Bryan wrote:
> Hi all,
>
> I'd like to get some data on weird CI failures. Before clicking 
> "retry" on a spurious failure, please paste the url for your job into 
> the spreadsheet you'll find linked at 
> https://gitlab.haskell.org/ghc/ghc/-/issues/21591.
>
> Sorry for the slight misdirection. I wanted the spreadsheet to be 
> world-writable, which means I don't want its url floating around in 
> too many places. Maybe you can bookmark it if CI is causing you too 
> much trouble. :)
>
> -Bryan
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220712/813b44ab/attachment.html>

From simon.peytonjones at gmail.com  Tue Jul 12 12:24:49 2022
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Tue, 12 Jul 2022 13:24:49 +0100
Subject: Hadrian problem
Message-ID: <CAJKmMz_vmKtVGGMTgUqoKAuPOdAiVk=i9Ba2L9FJOBVzEr73ww@mail.gmail.com>

I'm in a GHC tree, built with Hadrian, I'm getting this red problem.  But
compilation has got way past compiling base.

why is it looking in my .ghc/... directory?   It should be looking in my
build tree.

Simon

bash$ ~/code/HEAD-1/_build/ghc-stage1 -c Foo.hs
Loaded package environment from
/home/simonpj/.ghc/x86_64-linux-9.5.20220628/environments/default
<command line>: cannot satisfy -package-id base-4.17.0.0
    (use -v for more information)

bash$ cat ~/code/HEAD-1/_build/ghc-stage1
"/home/simonpj/code/HEAD-1/_build/stage0/bin/ghc" "-no-global-package-db"
"-package-db /home/simonpj/code/HEAD-1/_build/stage1/lib/package.conf.d"
"$@"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220712/bc1a6f3f/attachment.html>

From sylvain at haskus.fr  Tue Jul 12 13:00:12 2022
From: sylvain at haskus.fr (Sylvain Henry)
Date: Tue, 12 Jul 2022 15:00:12 +0200
Subject: Hadrian problem
In-Reply-To: <CAJKmMz_vmKtVGGMTgUqoKAuPOdAiVk=i9Ba2L9FJOBVzEr73ww@mail.gmail.com>
References: <CAJKmMz_vmKtVGGMTgUqoKAuPOdAiVk=i9Ba2L9FJOBVzEr73ww@mail.gmail.com>
Message-ID: <8533d33e-05f1-5fbc-97a7-ba1cfc38a347@haskus.fr>

Hi Simon,

Matt should have fixed it with 
https://gitlab.haskell.org/ghc/ghc/-/merge_requests/8556

Sylvain


On 12/07/2022 14:24, Simon Peyton Jones wrote:
> I'm in a GHC tree, built with Hadrian, I'm getting this red problem.  
> But compilation has got way past compiling base.
>
> why is it looking in my .ghc/... directory?   It should be looking in 
> my build tree.
>
> Simon
>
> bash$ ~/code/HEAD-1/_build/ghc-stage1 -c Foo.hs
> Loaded package environment from 
> /home/simonpj/.ghc/x86_64-linux-9.5.20220628/environments/default
> <command line>: cannot satisfy -package-id base-4.17.0.0
>     (use -v for more information)
>
> bash$ cat ~/code/HEAD-1/_build/ghc-stage1
> "/home/simonpj/code/HEAD-1/_build/stage0/bin/ghc" 
> "-no-global-package-db" "-package-db 
> /home/simonpj/code/HEAD-1/_build/stage1/lib/package.conf.d" "$@"
>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220712/9f324037/attachment.html>

From douglas.wilson at gmail.com  Tue Jul 12 13:00:54 2022
From: douglas.wilson at gmail.com (Douglas Wilson)
Date: Tue, 12 Jul 2022 14:00:54 +0100
Subject: Hadrian problem
In-Reply-To: <CAJKmMz_vmKtVGGMTgUqoKAuPOdAiVk=i9Ba2L9FJOBVzEr73ww@mail.gmail.com>
References: <CAJKmMz_vmKtVGGMTgUqoKAuPOdAiVk=i9Ba2L9FJOBVzEr73ww@mail.gmail.com>
Message-ID: <CALFBYq_0x9sQaZEtQrx4=ZbOzCNT5t3ZsZBU_k3t5+aoOtKN8g@mail.gmail.com>

DHi Simon,

It seems that GHC is "helpfully" adding flags to your invocation see
https://ghc.gitlab.haskell.org/ghc/doc/users_guide/packages.html#package-environments

You can see exactly what is being added by inspecting the environment file
/home/simonpj/.ghc/x86_64-linux-9.5.20220628/environments/default.

You can safely delete .ghc/x86_64-linux-9.5.20220628/environments/default.
Although without knowing how it got there, it may re-appear.

You can set `export GHC_ENVIRONMENT=-` or pass `-package-env -` on your
command line to disable the reading of environment files.

Regards,
Douglas Wilson


On Tue, Jul 12, 2022 at 1:24 PM Simon Peyton Jones <
simon.peytonjones at gmail.com> wrote:

> I'm in a GHC tree, built with Hadrian, I'm getting this red problem.  But
> compilation has got way past compiling base.
>
> why is it looking in my .ghc/... directory?   It should be looking in my
> build tree.
>
> Simon
>
> bash$ ~/code/HEAD-1/_build/ghc-stage1 -c Foo.hs
> Loaded package environment from
> /home/simonpj/.ghc/x86_64-linux-9.5.20220628/environments/default
> <command line>: cannot satisfy -package-id base-4.17.0.0
>     (use -v for more information)
>
> bash$ cat ~/code/HEAD-1/_build/ghc-stage1
> "/home/simonpj/code/HEAD-1/_build/stage0/bin/ghc" "-no-global-package-db"
> "-package-db /home/simonpj/code/HEAD-1/_build/stage1/lib/package.conf.d"
> "$@"
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220712/3c7627cb/attachment.html>

From hecate at glitchbra.in  Tue Jul 19 16:20:44 2022
From: hecate at glitchbra.in (=?UTF-8?Q?H=c3=a9cate?=)
Date: Tue, 19 Jul 2022 18:20:44 +0200
Subject: GHC development asks too much of the host system
Message-ID: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>

Hello ghc-devs,

I hadn't made significant contributions to the GHC code base in a while, 
until a few days ago, where I discovered that my computer wasn't able to 
sustain running the test suite, nor handle HLS well.

Whether it is my OS automatically killing the process due to oom-killer 
or just the fact that I don't have a war machine, I find it too bad and 
I'm frankly discouraged.
This is not the first time such feedback emerges, as the documentation 
task force for the base library was unable to properly onboard some 
people from third-world countries who do not have access to hardware 
we'd consider "standard" in western Europe or some parts of North 
America. Or at least "standard" until even my standard stuff didn't cut 
it anymore.

So yeah, I'll stay around but I'm afraid I'm going to have to focus on 
projects for which the feedback loop is not on the scale of hours , as 
this is a hobby project.

Hope this will open some eyes.

Cheers,
Hécate

-- 
Hécate ✨
🐦: @TechnoEmpress
IRC: Hecate
WWW: https://glitchbra.in
RUN: BSD


From simon.peytonjones at gmail.com  Tue Jul 19 16:29:33 2022
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Tue, 19 Jul 2022 17:29:33 +0100
Subject: GHC development asks too much of the host system
In-Reply-To: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
References: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
Message-ID: <CAJKmMz-VjrOKskORcYh_tqVgOkBiFCxDGpam+u+jvF-enu+=mg@mail.gmail.com>

That's bad Hecate. We need GHC to be fun to work with, not a pain.

Can you be (much) more specific?  The more concrete the problem, the more
likely we can address it.

e.g. What if you don't use HLS?  Or maybe Hadrian is building much more
than you need? It would be super helpful to have more information.  There
may be things we can't reasonably address (e.g. make a small, light,
non-optimising compiler instead, throwing away most of the code base) but I
bet that sheer size isn't the only factor.

Thanks!

Simon

On Tue, 19 Jul 2022 at 17:21, Hécate <hecate at glitchbra.in> wrote:

> Hello ghc-devs,
>
> I hadn't made significant contributions to the GHC code base in a while,
> until a few days ago, where I discovered that my computer wasn't able to
> sustain running the test suite, nor handle HLS well.
>
> Whether it is my OS automatically killing the process due to oom-killer
> or just the fact that I don't have a war machine, I find it too bad and
> I'm frankly discouraged.
> This is not the first time such feedback emerges, as the documentation
> task force for the base library was unable to properly onboard some
> people from third-world countries who do not have access to hardware
> we'd consider "standard" in western Europe or some parts of North
> America. Or at least "standard" until even my standard stuff didn't cut
> it anymore.
>
> So yeah, I'll stay around but I'm afraid I'm going to have to focus on
> projects for which the feedback loop is not on the scale of hours , as
> this is a hobby project.
>
> Hope this will open some eyes.
>
> Cheers,
> Hécate
>
> --
> Hécate ✨
> 🐦: @TechnoEmpress
> IRC: Hecate
> WWW: https://glitchbra.in
> RUN: BSD
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220719/701d2a32/attachment.html>

From moritz.angermann at gmail.com  Tue Jul 19 16:34:01 2022
From: moritz.angermann at gmail.com (Moritz Angermann)
Date: Tue, 19 Jul 2022 18:34:01 +0200
Subject: GHC development asks too much of the host system
In-Reply-To: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
References: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
Message-ID: <CAKfdd-y_0cYUHcfbXdgsJ1pFUXroVDT44BwuAK_X1nq9Qm-fcQ@mail.gmail.com>

Hi Hecate,

I don't think this is entirely fair in either direction.  So sharing my
personal experience
might shed some light.  I've often worked on GHC on fairly weak machines.
However
the ability to use HLS on GHC or even the ability to load GHC into GHCi are
fairly
recent additions.

I don't run the full test-suite either much.

The general development experience has more been closer to this:
- pick an issue I want to work on
- checkout the relevant branch (or master)
- kick off a ghc build (hadrian)
- start looking for the relevant code in GHC to address this.
- build a tiny reproducer (if possible, or run the relevant test from the
test-suite if available) -- once the initial ghc is build.
- hack on the codebase; rebuild (subsequent rebuilds are fairly fast)
- retry the reproducer, iterate until done.

Most of my development has been without much codelevel help and at most a
syntax highlighter. This is decidedly different from the experience you can
have
working on haskell libraries with the availability of ghcid, hls, ... would
it be nice
if ghc development would be that nice as well? I'd assume so, I've just
never
even tried.

Cheers,
 Moritz


On Tue, 19 Jul 2022 at 18:21, Hécate <hecate at glitchbra.in> wrote:

> Hello ghc-devs,
>
> I hadn't made significant contributions to the GHC code base in a while,
> until a few days ago, where I discovered that my computer wasn't able to
> sustain running the test suite, nor handle HLS well.
>
> Whether it is my OS automatically killing the process due to oom-killer
> or just the fact that I don't have a war machine, I find it too bad and
> I'm frankly discouraged.
> This is not the first time such feedback emerges, as the documentation
> task force for the base library was unable to properly onboard some
> people from third-world countries who do not have access to hardware
> we'd consider "standard" in western Europe or some parts of North
> America. Or at least "standard" until even my standard stuff didn't cut
> it anymore.
>
> So yeah, I'll stay around but I'm afraid I'm going to have to focus on
> projects for which the feedback loop is not on the scale of hours , as
> this is a hobby project.
>
> Hope this will open some eyes.
>
> Cheers,
> Hécate
>
> --
> Hécate ✨
> 🐦: @TechnoEmpress
> IRC: Hecate
> WWW: https://glitchbra.in
> RUN: BSD
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220719/9f646daa/attachment.html>

From hecate at glitchbra.in  Tue Jul 19 17:55:49 2022
From: hecate at glitchbra.in (=?UTF-8?Q?H=c3=a9cate?=)
Date: Tue, 19 Jul 2022 19:55:49 +0200
Subject: GHC development asks too much of the host system
In-Reply-To: <CAJKmMz-VjrOKskORcYh_tqVgOkBiFCxDGpam+u+jvF-enu+=mg@mail.gmail.com>
References: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
 <CAJKmMz-VjrOKskORcYh_tqVgOkBiFCxDGpam+u+jvF-enu+=mg@mail.gmail.com>
Message-ID: <633d4086-42f8-7d5b-e19d-70f3bcdc8f42@glitchbra.in>

Hi Simon,

Regarding the compiler compilation:
I often have to re-run ./configure and ./boot between development phases 
(usually spaced by one or two weeks), which prevents me from benefiting 
from `--freeze1` from the get-go.
The flavour I use is `Quick`, as it is advertised on https://ghc.dev.
The technique I use now is to restrict the `-j` parameter to 1 or 2. I 
can go above for night builds, but if I want to keep my system usable, 
this is how it goes.
This takes 47m07s according to Hadrian.

Moreover, modern computers have a BIOS setting that shuts down the 
computer when the temperature is too high (between 70°C and 100°C I 
believe). I ended up upping this limit in my BIOS some time ago, but I 
wish I didn't need to.
(and yes I do frequently remove the dust ;)

Regarding documentation contribution,
The time it takes to run `hadrian/build -j --flavour=Quick` *plus* 
`hadrian/build -j2 --freeze1 --flavour=Quick docs --docs=no-sphinx-pdfs 
` in order to render the haddocks is way too high for occasional 
contributors.

I started a process while writing this email, in order to get fresh and 
accurate numbers, and here is what I got in return:

juil. 19 19:37:21 elatha systemd[1759]: 
app-kitty-dedaa3d5a1ee43dd8b2552b7afd53cd5.scope: systemd-oomd killed 36 
process(es) in this unit.
juil. 19 19:37:21 elatha systemd-oomd[682429]: Killed 
/user.slice/user-1000.slice/user at 1000.service/app.slice/app-kitty-dedaa3d5a1ee43dd8b2552b7afd53cd5.scope 
due to memory used (16192806912) / total (16358338560) and swap used 
(8102125568) / total (8589930496) being more than 90.00%


Now regarding HLS, I remember that a point of marketing for it is that 
it supports GHC. Considering the very skilled people who hold the wheel 
of HLS development, my first (nor second) reflex isn't to doubt them. So 
yes I believe I must disable HLS when working on GHC, but then how do we 
spread this kind of information / work-around while at the same time 
saying "but HLS is still reliable, pinky swear, it's just that it 
doesn't scale".

I do not doubt that HLS does some heavy stuff and I'm truly grateful for 
all the times where it detects unused imports, unused extensions, and 
even inserts import lines for me! But I find it hard to promote HLS 
whilst also putting an asterisk that says "Provided that you can afford 
it".

This is pretty much all I can observe. I'd be more than happy to have 
better observability into the build process, though!
I'm not too desperate for the future, because chatting with Bryan gave 
me hope for progress in that area.

Le 19/07/2022 à 18:29, Simon Peyton Jones a écrit :
> That's bad Hecate. We need GHC to be fun to work with, not a pain.
>
> Can you be (much) more specific?  The more concrete the problem, the 
> more likely we can address it.
>
> e.g. What if you don't use HLS?  Or maybe Hadrian is building much 
> more than you need? It would be super helpful to have more 
> information.  There may be things we can't reasonably address (e.g. 
> make a small, light, non-optimising compiler instead, throwing away 
> most of the code base) but I bet that sheer size isn't the only factor.
>
> Thanks!
>
> Simon
>
> On Tue, 19 Jul 2022 at 17:21, Hécate <hecate at glitchbra.in> wrote:
>
>     Hello ghc-devs,
>
>     I hadn't made significant contributions to the GHC code base in a
>     while,
>     until a few days ago, where I discovered that my computer wasn't
>     able to
>     sustain running the test suite, nor handle HLS well.
>
>     Whether it is my OS automatically killing the process due to
>     oom-killer
>     or just the fact that I don't have a war machine, I find it too
>     bad and
>     I'm frankly discouraged.
>     This is not the first time such feedback emerges, as the
>     documentation
>     task force for the base library was unable to properly onboard some
>     people from third-world countries who do not have access to hardware
>     we'd consider "standard" in western Europe or some parts of North
>     America. Or at least "standard" until even my standard stuff
>     didn't cut
>     it anymore.
>
>     So yeah, I'll stay around but I'm afraid I'm going to have to
>     focus on
>     projects for which the feedback loop is not on the scale of hours
>     , as
>     this is a hobby project.
>
>     Hope this will open some eyes.
>
>     Cheers,
>     Hécate
>
>     -- 
>     Hécate ✨
>     🐦: @TechnoEmpress
>     IRC: Hecate
>     WWW: https://glitchbra.in
>     RUN: BSD
>
>     _______________________________________________
>     ghc-devs mailing list
>     ghc-devs at haskell.org
>     http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-- 
Hécate ✨
🐦: @TechnoEmpress
IRC: Hecate
WWW:https://glitchbra.in
RUN: BSD
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220719/0887942e/attachment.html>

From hecate at glitchbra.in  Tue Jul 19 18:00:24 2022
From: hecate at glitchbra.in (=?UTF-8?Q?H=c3=a9cate?=)
Date: Tue, 19 Jul 2022 20:00:24 +0200
Subject: GHC development asks too much of the host system
In-Reply-To: <CAJKmMz-VjrOKskORcYh_tqVgOkBiFCxDGpam+u+jvF-enu+=mg@mail.gmail.com>
References: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
 <CAJKmMz-VjrOKskORcYh_tqVgOkBiFCxDGpam+u+jvF-enu+=mg@mail.gmail.com>
Message-ID: <995f3c57-87f3-77a3-389f-1a778fd1ba6d@glitchbra.in>

(re-sending with replay-all this time)

Hi Simon, thank you for your concern.

Regarding the compiler compilation:
I often have to re-run ./configure and ./boot between development phases 
(usually spaced by one or two weeks), which prevents me from benefiting 
from `--freeze1` from the get-go.
The flavour I use is `Quick`, as it is advertised on https://ghc.dev.
The technique I use now is to restrict the `-j` parameter to 1 or 2. I 
can go above for night builds, but if I want to keep my system usable, 
this is how it goes.
This takes 47m07s according to Hadrian.

Moreover, modern computers have a BIOS setting that shuts down the 
computer when the temperature is too high (between 70°C and 100°C I 
believe). I ended up upping this limit in my BIOS some time ago, but I 
wish I didn't need to.
(and yes I do frequently remove the dust ;)

Regarding documentation contribution,
The time it takes to run `hadrian/build -j --flavour=Quick` *plus* 
`hadrian/build -j2 --freeze1 --flavour=Quick docs --docs=no-sphinx-pdfs 
` in order to render the haddocks is way too high for occasional 
contributors.

I started a process while writing this email, in order to get fresh and 
accurate numbers, and here is what I got in return:

juil. 19 19:37:21 elatha systemd[1759]: 
app-kitty-dedaa3d5a1ee43dd8b2552b7afd53cd5.scope: systemd-oomd killed 36 
process(es) in this unit.
juil. 19 19:37:21 elatha systemd-oomd[682429]: Killed 
/user.slice/user-1000.slice/user at 1000.service/app.slice/app-kitty-dedaa3d5a1ee43dd8b2552b7afd53cd5.scope 
due to memory used (16192806912) / total (16358338560) and swap used 
(8102125568) / total (8589930496) being more than 90.00%


Now regarding HLS, I remember that a point of marketing for it is that 
it supports GHC. Considering the very skilled people who hold the wheel 
of HLS development, my first (nor second) reflex isn't to doubt them. So 
yes I believe I must disable HLS when working on GHC, but then how do we 
spread this kind of information / work-around while at the same time 
saying "but HLS is still reliable, pinky swear, it's just that it 
doesn't scale".

I do not doubt that HLS does some heavy stuff and I'm truly grateful for 
all the times where it detects unused imports, unused extensions, and 
even inserts import lines for me! But I find it hard to promote HLS 
whilst also putting an asterisk that says "Provided that you can afford 
it".

This is pretty much all I can observe. I'd be more than happy to have 
better observability into the build process, though!
I'm not too desperate for the future, because chatting with Bryan gave 
me hope for progress in that area.
Le 19/07/2022 à 18:29, Simon Peyton Jones a écrit :
> That's bad Hecate. We need GHC to be fun to work with, not a pain.
>
> Can you be (much) more specific?  The more concrete the problem, the 
> more likely we can address it.
>
> e.g. What if you don't use HLS?  Or maybe Hadrian is building much 
> more than you need? It would be super helpful to have more 
> information.  There may be things we can't reasonably address (e.g. 
> make a small, light, non-optimising compiler instead, throwing away 
> most of the code base) but I bet that sheer size isn't the only factor.
>
> Thanks!
>
> Simon
>
> On Tue, 19 Jul 2022 at 17:21, Hécate <hecate at glitchbra.in> wrote:
>
>     Hello ghc-devs,
>
>     I hadn't made significant contributions to the GHC code base in a
>     while,
>     until a few days ago, where I discovered that my computer wasn't
>     able to
>     sustain running the test suite, nor handle HLS well.
>
>     Whether it is my OS automatically killing the process due to
>     oom-killer
>     or just the fact that I don't have a war machine, I find it too
>     bad and
>     I'm frankly discouraged.
>     This is not the first time such feedback emerges, as the
>     documentation
>     task force for the base library was unable to properly onboard some
>     people from third-world countries who do not have access to hardware
>     we'd consider "standard" in western Europe or some parts of North
>     America. Or at least "standard" until even my standard stuff
>     didn't cut
>     it anymore.
>
>     So yeah, I'll stay around but I'm afraid I'm going to have to
>     focus on
>     projects for which the feedback loop is not on the scale of hours
>     , as
>     this is a hobby project.
>
>     Hope this will open some eyes.
>
>     Cheers,
>     Hécate
>
>     -- 
>     Hécate ✨
>     🐦: @TechnoEmpress
>     IRC: Hecate
>     WWW: https://glitchbra.in
>     RUN: BSD
>
>     _______________________________________________
>     ghc-devs mailing list
>     ghc-devs at haskell.org
>     http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-- 
Hécate ✨
🐦: @TechnoEmpress
IRC: Hecate
WWW:https://glitchbra.in
RUN: BSD
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220719/4f8c8c2c/attachment.html>

From hecate at glitchbra.in  Tue Jul 19 18:01:33 2022
From: hecate at glitchbra.in (=?UTF-8?Q?H=c3=a9cate?=)
Date: Tue, 19 Jul 2022 20:01:33 +0200
Subject: GHC development asks too much of the host system
In-Reply-To: <CAKfdd-y_0cYUHcfbXdgsJ1pFUXroVDT44BwuAK_X1nq9Qm-fcQ@mail.gmail.com>
References: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
 <CAKfdd-y_0cYUHcfbXdgsJ1pFUXroVDT44BwuAK_X1nq9Qm-fcQ@mail.gmail.com>
Message-ID: <dbd87704-c7bf-7ed3-7256-cbfa5576df49@glitchbra.in>

Hi Moritz, thank you very much for this bullet list. No doubt my own 
development workflow could be improved. :)

Le 19/07/2022 à 18:34, Moritz Angermann a écrit :
> Hi Hecate,
>
> I don't think this is entirely fair in either direction. So sharing my 
> personal experience
> might shed some light.  I've often worked on GHC on fairly weak 
> machines.  However
> the ability to use HLS on GHC or even the ability to load GHC into 
> GHCi are fairly
> recent additions.
>
> I don't run the full test-suite either much.
>
> The general development experience has more been closer to this:
> - pick an issue I want to work on
> - checkout the relevant branch (or master)
> - kick off a ghc build (hadrian)
> - start looking for the relevant code in GHC to address this.
> - build a tiny reproducer (if possible, or run the relevant test from 
> the test-suite if available) -- once the initial ghc is build.
> - hack on the codebase; rebuild (subsequent rebuilds are fairly fast)
> - retry the reproducer, iterate until done.
>
> Most of my development has been without much codelevel help and at most a
> syntax highlighter. This is decidedly different from the experience 
> you can have
> working on haskell libraries with the availability of ghcid, hls, ... 
> would it be nice
> if ghc development would be that nice as well? I'd assume so, I've 
> just never
> even tried.
>
> Cheers,
>  Moritz
>
>
> On Tue, 19 Jul 2022 at 18:21, Hécate <hecate at glitchbra.in> wrote:
>
>     Hello ghc-devs,
>
>     I hadn't made significant contributions to the GHC code base in a
>     while,
>     until a few days ago, where I discovered that my computer wasn't
>     able to
>     sustain running the test suite, nor handle HLS well.
>
>     Whether it is my OS automatically killing the process due to
>     oom-killer
>     or just the fact that I don't have a war machine, I find it too
>     bad and
>     I'm frankly discouraged.
>     This is not the first time such feedback emerges, as the
>     documentation
>     task force for the base library was unable to properly onboard some
>     people from third-world countries who do not have access to hardware
>     we'd consider "standard" in western Europe or some parts of North
>     America. Or at least "standard" until even my standard stuff
>     didn't cut
>     it anymore.
>
>     So yeah, I'll stay around but I'm afraid I'm going to have to
>     focus on
>     projects for which the feedback loop is not on the scale of hours
>     , as
>     this is a hobby project.
>
>     Hope this will open some eyes.
>
>     Cheers,
>     Hécate
>
>     -- 
>     Hécate ✨
>     🐦: @TechnoEmpress
>     IRC: Hecate
>     WWW: https://glitchbra.in
>     RUN: BSD
>
>     _______________________________________________
>     ghc-devs mailing list
>     ghc-devs at haskell.org
>     http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-- 
Hécate ✨
🐦: @TechnoEmpress
IRC: Hecate
WWW:https://glitchbra.in
RUN: BSD
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220719/86b674a3/attachment.html>

From ben at smart-cactus.org  Tue Jul 19 19:10:35 2022
From: ben at smart-cactus.org (Ben Gamari)
Date: Tue, 19 Jul 2022 15:10:35 -0400
Subject: GHC development asks too much of the host system
In-Reply-To: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
References: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
Message-ID: <877d482a21.fsf@smart-cactus.org>

Hécate <hecate at glitchbra.in> writes:

> Hello ghc-devs,
>
> I hadn't made significant contributions to the GHC code base in a while, 
> until a few days ago, where I discovered that my computer wasn't able to 
> sustain running the test suite, nor handle HLS well.
>
> Whether it is my OS automatically killing the process due to oom-killer 
> or just the fact that I don't have a war machine, I find it too bad and 
> I'm frankly discouraged.

Do you know which process was being killed? There is one testsuite tests
that I know of which does have quite a considerable memory footprint
(T16992) due to its nature; otherwise I would expect a reasonably recent
machine to pass the testsuite without much trouble. It's particularly
concerning if this is a new regression; is this the first time you have
observed this particular failure?

> This is not the first time such feedback emerges, as the documentation 
> task force for the base library was unable to properly onboard some 
> people from third-world countries who do not have access to hardware 
> we'd consider "standard" in western Europe or some parts of North 
> America. Or at least "standard" until even my standard stuff didn't cut 
> it anymore.
>
> So yeah, I'll stay around but I'm afraid I'm going to have to focus on 
> projects for which the feedback loop is not on the scale of hours , as 
> this is a hobby project.
>
> Hope this will open some eyes.
>
Hi Hécate,

I would reiterate that the more specific feedback you can offer, the
better.

To share my some of my own experience: I have access to a variety of hardware,
some of which is quite powerful. However, I find that I end up doing
much of my development on my laptop which, while certainly not a slouch
(being a Ryzen 4750U), is also not a monster. In particular, while a
fresh build takes nearly twice as long on my laptop than some of the
other hardware I have, I nevertheless find ways to make it worthwhile
(due to the ease of iteration compared to ssh). If you routinely have
multi-hour iteration times then something isn't right.

In particular, I think there are a few tricks which make life far
easier:


 * Be careful about doing things that would incur
   significant amounts of rebuilding. This includes:

    * After modifying, e.g., `compiler/ghc.cabal.in` (e.g. to add a new
      module to GHC), modify `compiler/ghc.cabal` manually instead of
      rerunning `configure`.

    * Be careful about pulling/rebase. I generally pick a base commit to
      build off of and rebase sparingly: Having to stop what I'm doing to
      wait for full rebuild is an easy way to lose momentum.

    * Avoid switching branches; I generally have a GHC tree per on-going
      project.

 * Take advantage of Hadrian's `--freeze1` flag

 * Use `hadrian/ghci` to typecheck changes

 * Use the stage1 compiler instead of stage2 to smoke-test changes when
   possible. (specifically, using the script generated by Hadrian's
   `_build/ghc-stage1` target)

 * Use the right build flavour for the task at hand: If I don't need a
   performant compiler and am confident that I can get by without
   thorough testsuite validation, I use `quick`. Otherwise, plan ahead
   for what you need (e.g. `default+assertions+debug_info` or
   `validate`)

 * Run the fraction of the testsuite that is relevant to your change.
   Hadrian's `--test-way` and `--only` flags are your friends.

 * Take advantage of CI. At the moment we have a fair amount of CI
   capacity. If you think that your change is close to working, you can
   open an MR and start a build locally. If it fails, iterate on just the
   failing testcases locally.

 * Task-level parallelism. Admittedly, this is harder when you are
   working as a hobby, but I often have two or three projects on-going
   at a time. While one tree is building I try to make progress on
   another.

I don't use HLS so I may be insulated from some of the pain in this
regard. However, I do know that Matt is a regular user and he
disables most plugins.

I would also say that, sadly, GHC is comparable to other similarly-size
compilers in its build time: A build of LLVM (not even clang) takes ~50
minutes on my 8-core desktop; impressively, rustc takes ~7 minutes
although it is a considerably smaller compiler (being just a front-end).
By contrast, GHC takes around 20 minutes. I know that this doesn't
make the cost any easier to bear and I would love to bring this number
down, but ultimately there are only so many hours in the day.

I think one underexplored approach to addressing the build-time problem
is to look not at the full-build time but rather look for common tasks
where we could *avoid* doing a full build (e.g. updating documentation,
typechecking `base`, running a "good enough" subset of the testsuite)
and find ways to make those workflows more efficient.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220719/29bf4af9/attachment.sig>

From a.pelenitsyn at gmail.com  Tue Jul 19 20:40:27 2022
From: a.pelenitsyn at gmail.com (Artem Pelenitsyn)
Date: Tue, 19 Jul 2022 16:40:27 -0400
Subject: GHC development asks too much of the host system
In-Reply-To: <877d482a21.fsf@smart-cactus.org>
References: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
 <877d482a21.fsf@smart-cactus.org>
Message-ID: <CAPKAvQX3hckQ_HHdwc=vA+smHA_zhfTvYAt+BKxhOp6mx_7yLQ@mail.gmail.com>

Hey everyone! I'm not a frequent contributor but I observed similar
challenges
as Hécate. I notice couple points.

### HLS and other editor integrations

I've never tried HLS for GHC development but it absolutely chokes on Cabal
for me (a $2K laptop), so I'm not surprised it's having troubles with GHC.
I've never tried to dig into it, but I heard before that disabling plugins
is a
good start.

Ghcid (after the introduction of ghc-in-ghci) was interesting but still on
a slow side.

I once tried to generate ETAGS and use them from Emacs (with plain
haskell-mode): this
was quite nice. As Moritz, I didn't use much above syntax coloring, but
ETAGS
allowed jumping to definitions, which is important. Maintaining tags wasn't
fun,
on the other hand.

In all fairness, I think that's an issue with HLS more than with GHC.

### Build Times

I have been using a dedicated server for this, but this still was painful at
times (even git clone takes non-negligible amount of time, and I never got
used
to git worktree because of a hoop you have to jump over, which I already
forgot
but I know it can be looked up in Andreas Herrmann's presentation on
developing
GHC). I'm surprised no one seems to try to challenge the status quo.

Hadrian is a Shake application. How is Cloud Shake doing? In the era of Nix
and
Bazel you start assuming niceties like remote caching. It'd be great to
improve
on this front as it just feels very wrong rebuilding master again and again
on every
contributor's computer. Especially after so much effort put into GHC
modularity,
which, I believe, should make it easier to cache.

It's sad that GHC still needs ./boot && ./configure: this can preclude any
remote caching technology that I can imagine. At one point it seemed like
configure could go into Hadrian, but it didn't really happen.

In the bright future I see Hadrian doing the whole job and using remote
cache
on GitLab CI. Not sure how big of a task that would be.

--
Cheers, Artem


On Tue, 19 Jul 2022 at 15:11, Ben Gamari <ben at smart-cactus.org> wrote:

> Hécate <hecate at glitchbra.in> writes:
>
> > Hello ghc-devs,
> >
> > I hadn't made significant contributions to the GHC code base in a while,
> > until a few days ago, where I discovered that my computer wasn't able to
> > sustain running the test suite, nor handle HLS well.
> >
> > Whether it is my OS automatically killing the process due to oom-killer
> > or just the fact that I don't have a war machine, I find it too bad and
> > I'm frankly discouraged.
>
> Do you know which process was being killed? There is one testsuite tests
> that I know of which does have quite a considerable memory footprint
> (T16992) due to its nature; otherwise I would expect a reasonably recent
> machine to pass the testsuite without much trouble. It's particularly
> concerning if this is a new regression; is this the first time you have
> observed this particular failure?
>
> > This is not the first time such feedback emerges, as the documentation
> > task force for the base library was unable to properly onboard some
> > people from third-world countries who do not have access to hardware
> > we'd consider "standard" in western Europe or some parts of North
> > America. Or at least "standard" until even my standard stuff didn't cut
> > it anymore.
> >
> > So yeah, I'll stay around but I'm afraid I'm going to have to focus on
> > projects for which the feedback loop is not on the scale of hours , as
> > this is a hobby project.
> >
> > Hope this will open some eyes.
> >
> Hi Hécate,
>
> I would reiterate that the more specific feedback you can offer, the
> better.
>
> To share my some of my own experience: I have access to a variety of
> hardware,
> some of which is quite powerful. However, I find that I end up doing
> much of my development on my laptop which, while certainly not a slouch
> (being a Ryzen 4750U), is also not a monster. In particular, while a
> fresh build takes nearly twice as long on my laptop than some of the
> other hardware I have, I nevertheless find ways to make it worthwhile
> (due to the ease of iteration compared to ssh). If you routinely have
> multi-hour iteration times then something isn't right.
>
> In particular, I think there are a few tricks which make life far
> easier:
>
>
>  * Be careful about doing things that would incur
>    significant amounts of rebuilding. This includes:
>
>     * After modifying, e.g., `compiler/ghc.cabal.in` (e.g. to add a new
>       module to GHC), modify `compiler/ghc.cabal` manually instead of
>       rerunning `configure`.
>
>     * Be careful about pulling/rebase. I generally pick a base commit to
>       build off of and rebase sparingly: Having to stop what I'm doing to
>       wait for full rebuild is an easy way to lose momentum.
>
>     * Avoid switching branches; I generally have a GHC tree per on-going
>       project.
>
>  * Take advantage of Hadrian's `--freeze1` flag
>
>  * Use `hadrian/ghci` to typecheck changes
>
>  * Use the stage1 compiler instead of stage2 to smoke-test changes when
>    possible. (specifically, using the script generated by Hadrian's
>    `_build/ghc-stage1` target)
>
>  * Use the right build flavour for the task at hand: If I don't need a
>    performant compiler and am confident that I can get by without
>    thorough testsuite validation, I use `quick`. Otherwise, plan ahead
>    for what you need (e.g. `default+assertions+debug_info` or
>    `validate`)
>
>  * Run the fraction of the testsuite that is relevant to your change.
>    Hadrian's `--test-way` and `--only` flags are your friends.
>
>  * Take advantage of CI. At the moment we have a fair amount of CI
>    capacity. If you think that your change is close to working, you can
>    open an MR and start a build locally. If it fails, iterate on just the
>    failing testcases locally.
>
>  * Task-level parallelism. Admittedly, this is harder when you are
>    working as a hobby, but I often have two or three projects on-going
>    at a time. While one tree is building I try to make progress on
>    another.
>
> I don't use HLS so I may be insulated from some of the pain in this
> regard. However, I do know that Matt is a regular user and he
> disables most plugins.
>
> I would also say that, sadly, GHC is comparable to other similarly-size
> compilers in its build time: A build of LLVM (not even clang) takes ~50
> minutes on my 8-core desktop; impressively, rustc takes ~7 minutes
> although it is a considerably smaller compiler (being just a front-end).
> By contrast, GHC takes around 20 minutes. I know that this doesn't
> make the cost any easier to bear and I would love to bring this number
> down, but ultimately there are only so many hours in the day.
>
> I think one underexplored approach to addressing the build-time problem
> is to look not at the full-build time but rather look for common tasks
> where we could *avoid* doing a full build (e.g. updating documentation,
> typechecking `base`, running a "good enough" subset of the testsuite)
> and find ways to make those workflows more efficient.
>
> Cheers,
>
> - Ben
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220719/ba35bf04/attachment.html>

From mikolaj at well-typed.com  Tue Jul 19 21:31:47 2022
From: mikolaj at well-typed.com (Mikolaj Konarski)
Date: Tue, 19 Jul 2022 23:31:47 +0200
Subject: GHC development asks too much of the host system
In-Reply-To: <CAPKAvQX3hckQ_HHdwc=vA+smHA_zhfTvYAt+BKxhOp6mx_7yLQ@mail.gmail.com>
References: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
 <877d482a21.fsf@smart-cactus.org>
 <CAPKAvQX3hckQ_HHdwc=vA+smHA_zhfTvYAt+BKxhOp6mx_7yLQ@mail.gmail.com>
Message-ID: <CAHFo2urL4L=GyL04HEtVsHnsYGFFq76JdkvTboaWG=GMZN0Bxw@mail.gmail.com>

> I once tried to generate ETAGS and use them from Emacs (with plain haskell-mode): this
> was quite nice. As Moritz, I didn't use much above syntax coloring, but ETAGS
> allowed jumping to definitions, which is important. Maintaining tags wasn't fun,
> on the other hand.

Package https://hackage.haskell.org/package/hasktags does it all
for you automatically at buffer save IIRC. I don't even remember
how it works, because I've never had a problem in years.
OTOH, I use the same haskell-mode as 10 years ago, having
decided after a few botched upgrades that it's good enough.

From ben at smart-cactus.org  Tue Jul 19 22:03:13 2022
From: ben at smart-cactus.org (Ben Gamari)
Date: Tue, 19 Jul 2022 18:03:13 -0400
Subject: GHC development asks too much of the host system
In-Reply-To: <CAPKAvQX3hckQ_HHdwc=vA+smHA_zhfTvYAt+BKxhOp6mx_7yLQ@mail.gmail.com>
References: <CAPKAvQX3hckQ_HHdwc=vA+smHA_zhfTvYAt+BKxhOp6mx_7yLQ@mail.gmail.com>
Message-ID: <874jzc222a.fsf@smart-cactus.org>

Artem Pelenitsyn <a.pelenitsyn at gmail.com> writes:

> Hey everyone! I'm not a frequent contributor but I observed similar
> challenges as Hécate. I notice couple points.
>
> ### HLS and other editor integrations
>
> I've never tried HLS for GHC development but it absolutely chokes on
> Cabal for me (a $2K laptop), so I'm not surprised it's having troubles
> with GHC. I've never tried to dig into it, but I heard before that
> disabling plugins is a good start.
>
> Ghcid (after the introduction of ghc-in-ghci) was interesting but still on
> a slow side.
>
> I once tried to generate ETAGS and use them from Emacs (with plain
> haskell-mode): this was quite nice. As Moritz, I didn't use much above
> syntax coloring, but ETAGS allowed jumping to definitions, which is
> important. Maintaining tags wasn't fun, on the other hand.
>
> In all fairness, I think that's an issue with HLS more than with GHC.
>
> ### Build Times
>
> I have been using a dedicated server for this, but this still was
> painful at times (even git clone takes non-negligible amount of time,
> and I never got used to git worktree because of a hoop you have to
> jump over, which I already forgot but I know it can be looked up in
> Andreas Herrmann's presentation on developing GHC). I'm surprised no
> one seems to try to challenge the status quo.
>
IMHO, `git worktree` is indispensible. Not only does it make cloning
cheaper but it makes it trivial to share commits between work trees,
which is incredibly helpful when cleaning up branch history,
backporting, and other common tasks. I just wish it also worked
transparently for submodules.

> Hadrian is a Shake application. How is Cloud Shake doing? In the era
> of Nix and Bazel you start assuming niceties like remote caching. It'd
> be great to improve on this front as it just feels very wrong
> rebuilding master again and again on every contributor's computer.
> Especially after so much effort put into GHC modularity, which, I
> believe, should make it easier to cache.

Sadly using Cloud Shake in Hadrian ran into some rather fundamental
difficulties:

 * GHC has native dependencies (namely, the native toolchain, ncurses,
   gmp, and possible libdw and libnuma). If everyone were to use, e.g.,
   ghc.nix this could be largely mitigated, but this isn't the world in
   which we live.

 * GHC is a bootstrapped compiler. Consequently, most changes will
   invalidate well over half of the build cache (that is, everything
   built by stage 1). This significantly limits the benefit that one
   could gain from Cloud Shake, especially since in the typical
   development workflow the stage 1 build (which is where most of the
   caching benefit would be seen) is a rather small cost (IIRC it takes
   around 5 minutes to make it to the stage 2 build on my machine).

   One might think that we could simply "freeze" the stage 1 compiler,
   but in general this is not safe. For instance, it would break subtly
   on any change to known keys, the interface file format, the ABI, or
   primop definitions.

>
> It's sad that GHC still needs ./boot && ./configure: this can preclude any
> remote caching technology that I can imagine. At one point it seemed like
> configure could go into Hadrian, but it didn't really happen.
>
I don't see us moving away from `configure` (or something like it) as
long as GHC has native dependencies. Having a clear separation between
"configuration" and "building" is very much necessary to maintain sanity.

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220719/431ff14e/attachment.sig>

From rodrigo.m.mesquita at gmail.com  Tue Jul 19 22:26:36 2022
From: rodrigo.m.mesquita at gmail.com (Rodrigo Mesquita)
Date: Wed, 20 Jul 2022 00:26:36 +0200
Subject: GHC development asks too much of the host system
In-Reply-To: <877d482a21.fsf@smart-cactus.org>
References: <877d482a21.fsf@smart-cactus.org>
Message-ID: <D47D25EF-3991-4E25-8B22-11636324F0F5@gmail.com>

Dear Ben,

The list of tips you put together is quite nice.

I suggest we add it to hadrian’s wiki page under a “Tips for making your life easier” section (as is, it is already useful! at least I learned something new).

Cheers
Rodrigo

> On 19 Jul 2022, at 21:11, Ben Gamari <ben at smart-cactus.org> wrote:
> 
> Hécate <hecate at glitchbra.in> writes:
> 
>> Hello ghc-devs,
>> 
>> I hadn't made significant contributions to the GHC code base in a while, 
>> until a few days ago, where I discovered that my computer wasn't able to 
>> sustain running the test suite, nor handle HLS well.
>> 
>> Whether it is my OS automatically killing the process due to oom-killer 
>> or just the fact that I don't have a war machine, I find it too bad and 
>> I'm frankly discouraged.
> 
> Do you know which process was being killed? There is one testsuite tests
> that I know of which does have quite a considerable memory footprint
> (T16992) due to its nature; otherwise I would expect a reasonably recent
> machine to pass the testsuite without much trouble. It's particularly
> concerning if this is a new regression; is this the first time you have
> observed this particular failure?
> 
>> This is not the first time such feedback emerges, as the documentation 
>> task force for the base library was unable to properly onboard some 
>> people from third-world countries who do not have access to hardware 
>> we'd consider "standard" in western Europe or some parts of North 
>> America. Or at least "standard" until even my standard stuff didn't cut 
>> it anymore.
>> 
>> So yeah, I'll stay around but I'm afraid I'm going to have to focus on 
>> projects for which the feedback loop is not on the scale of hours , as 
>> this is a hobby project.
>> 
>> Hope this will open some eyes.
>> 
> Hi Hécate,
> 
> I would reiterate that the more specific feedback you can offer, the
> better.
> 
> To share my some of my own experience: I have access to a variety of hardware,
> some of which is quite powerful. However, I find that I end up doing
> much of my development on my laptop which, while certainly not a slouch
> (being a Ryzen 4750U), is also not a monster. In particular, while a
> fresh build takes nearly twice as long on my laptop than some of the
> other hardware I have, I nevertheless find ways to make it worthwhile
> (due to the ease of iteration compared to ssh). If you routinely have
> multi-hour iteration times then something isn't right.
> 
> In particular, I think there are a few tricks which make life far
> easier:
> 
> 
> * Be careful about doing things that would incur
>   significant amounts of rebuilding. This includes:
> 
>    * After modifying, e.g., `compiler/ghc.cabal.in` (e.g. to add a new
>      module to GHC), modify `compiler/ghc.cabal` manually instead of
>      rerunning `configure`.
> 
>    * Be careful about pulling/rebase. I generally pick a base commit to
>      build off of and rebase sparingly: Having to stop what I'm doing to
>      wait for full rebuild is an easy way to lose momentum.
> 
>    * Avoid switching branches; I generally have a GHC tree per on-going
>      project.
> 
> * Take advantage of Hadrian's `--freeze1` flag
> 
> * Use `hadrian/ghci` to typecheck changes
> 
> * Use the stage1 compiler instead of stage2 to smoke-test changes when
>   possible. (specifically, using the script generated by Hadrian's
>   `_build/ghc-stage1` target)
> 
> * Use the right build flavour for the task at hand: If I don't need a
>   performant compiler and am confident that I can get by without
>   thorough testsuite validation, I use `quick`. Otherwise, plan ahead
>   for what you need (e.g. `default+assertions+debug_info` or
>   `validate`)
> 
> * Run the fraction of the testsuite that is relevant to your change.
>   Hadrian's `--test-way` and `--only` flags are your friends.
> 
> * Take advantage of CI. At the moment we have a fair amount of CI
>   capacity. If you think that your change is close to working, you can
>   open an MR and start a build locally. If it fails, iterate on just the
>   failing testcases locally.
> 
> * Task-level parallelism. Admittedly, this is harder when you are
>   working as a hobby, but I often have two or three projects on-going
>   at a time. While one tree is building I try to make progress on
>   another.
> 
> I don't use HLS so I may be insulated from some of the pain in this
> regard. However, I do know that Matt is a regular user and he
> disables most plugins.
> 
> I would also say that, sadly, GHC is comparable to other similarly-size
> compilers in its build time: A build of LLVM (not even clang) takes ~50
> minutes on my 8-core desktop; impressively, rustc takes ~7 minutes
> although it is a considerably smaller compiler (being just a front-end).
> By contrast, GHC takes around 20 minutes. I know that this doesn't
> make the cost any easier to bear and I would love to bring this number
> down, but ultimately there are only so many hours in the day.
> 
> I think one underexplored approach to addressing the build-time problem
> is to look not at the full-build time but rather look for common tasks
> where we could *avoid* doing a full build (e.g. updating documentation,
> typechecking `base`, running a "good enough" subset of the testsuite)
> and find ways to make those workflows more efficient.
> 
> Cheers,
> 
> - Ben
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/octet-stream
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220720/25bc6910/attachment-0001.obj>

From a.pelenitsyn at gmail.com  Wed Jul 20 02:42:36 2022
From: a.pelenitsyn at gmail.com (Artem Pelenitsyn)
Date: Tue, 19 Jul 2022 22:42:36 -0400
Subject: GHC development asks too much of the host system
In-Reply-To: <874jzc222a.fsf@smart-cactus.org>
References: <CAPKAvQX3hckQ_HHdwc=vA+smHA_zhfTvYAt+BKxhOp6mx_7yLQ@mail.gmail.com>
 <874jzc222a.fsf@smart-cactus.org>
Message-ID: <CAPKAvQXuHzvYkrovsPmVXRNLFT0fuhj1ta6+7Xu7TSR4xd4d4Q@mail.gmail.com>

Thanks Ben, very interesting, especially the cloud Shake stuff.

> If everyone were to use, e.g.,
>   ghc.nix this could be largely mitigated, but this isn't the world in
>   which we live.

I don't know why you'd want everyone to use it necessarily to improve
things.
If there was a clear statement that you may elect to use ghc.nix and get X%
speedup,
that would be a good start. And then people can decide, based on their setup
and their aversion to tools like Nix, etc. in general.

The real issue is that currently you don't benefit much from ghc.nix
because the main
performance sink is the GHC tree itself. The way out is to use a cloud
build system, most likely
Cloud Shake, which, as you describe, has a couple of issues in this case:

1) Native dependencies. This should be possible to solve via Nix, but
unfortunately,
this is not quite there yet because Shake doesn't know about Nix (afaik). I
think to actually get
there, you'd need some sort of integration between Nix and Shake akin to
what Tweag
built for Nix and Bazel (cf. rules_nixpkgs [1]). Their moto is: Nix for
"external" or "system"
components, and Bazel for "internal" or "local" ones.

2) Bootstrapping aspect. Maybe this is a challenge for rebuilds after
modification,
but I think people on this thread were quoting the "time to first build"
more.
I don't see how avoiding to build master locally after a fresh (worktree)
checkout by
downloading build results from somewhere, connects to bootstrapping. I
think it doesn't.

As for rebuilds. People are already using --freeze1 (you suggested it
earlier in this very thread!),
so I don't see how saying "freezing stage 1 is dangerous even if faster"
connects to practise of
GHC development. Of course, you may not find a remote cache with relevant
artefacts after
local updates, but that's not the point. The point is to not have to build
`master`, not
`feaure-branch-t12345`. Rebuilds should be rather pain-free in comparison.

--
Best, Artem

[1]: https://github.com/tweag/rules_nixpkgs


On Tue, 19 Jul 2022 at 18:03, Ben Gamari <ben at smart-cactus.org> wrote:

> Artem Pelenitsyn <a.pelenitsyn at gmail.com> writes:
>
> > Hey everyone! I'm not a frequent contributor but I observed similar
> > challenges as Hécate. I notice couple points.
> >
> > ### HLS and other editor integrations
> >
> > I've never tried HLS for GHC development but it absolutely chokes on
> > Cabal for me (a $2K laptop), so I'm not surprised it's having troubles
> > with GHC. I've never tried to dig into it, but I heard before that
> > disabling plugins is a good start.
> >
> > Ghcid (after the introduction of ghc-in-ghci) was interesting but still
> on
> > a slow side.
> >
> > I once tried to generate ETAGS and use them from Emacs (with plain
> > haskell-mode): this was quite nice. As Moritz, I didn't use much above
> > syntax coloring, but ETAGS allowed jumping to definitions, which is
> > important. Maintaining tags wasn't fun, on the other hand.
> >
> > In all fairness, I think that's an issue with HLS more than with GHC.
> >
> > ### Build Times
> >
> > I have been using a dedicated server for this, but this still was
> > painful at times (even git clone takes non-negligible amount of time,
> > and I never got used to git worktree because of a hoop you have to
> > jump over, which I already forgot but I know it can be looked up in
> > Andreas Herrmann's presentation on developing GHC). I'm surprised no
> > one seems to try to challenge the status quo.
> >
> IMHO, `git worktree` is indispensible. Not only does it make cloning
> cheaper but it makes it trivial to share commits between work trees,
> which is incredibly helpful when cleaning up branch history,
> backporting, and other common tasks. I just wish it also worked
> transparently for submodules.
>
> > Hadrian is a Shake application. How is Cloud Shake doing? In the era
> > of Nix and Bazel you start assuming niceties like remote caching. It'd
> > be great to improve on this front as it just feels very wrong
> > rebuilding master again and again on every contributor's computer.
> > Especially after so much effort put into GHC modularity, which, I
> > believe, should make it easier to cache.
>
> Sadly using Cloud Shake in Hadrian ran into some rather fundamental
> difficulties:
>
>  * GHC has native dependencies (namely, the native toolchain, ncurses,
>    gmp, and possible libdw and libnuma). If everyone were to use, e.g.,
>    ghc.nix this could be largely mitigated, but this isn't the world in
>    which we live.
>
>  * GHC is a bootstrapped compiler. Consequently, most changes will
>    invalidate well over half of the build cache (that is, everything
>    built by stage 1). This significantly limits the benefit that one
>    could gain from Cloud Shake, especially since in the typical
>    development workflow the stage 1 build (which is where most of the
>    caching benefit would be seen) is a rather small cost (IIRC it takes
>    around 5 minutes to make it to the stage 2 build on my machine).
>
>    One might think that we could simply "freeze" the stage 1 compiler,
>    but in general this is not safe. For instance, it would break subtly
>    on any change to known keys, the interface file format, the ABI, or
>    primop definitions.
>
> >
> > It's sad that GHC still needs ./boot && ./configure: this can preclude
> any
> > remote caching technology that I can imagine. At one point it seemed like
> > configure could go into Hadrian, but it didn't really happen.
> >
> I don't see us moving away from `configure` (or something like it) as
> long as GHC has native dependencies. Having a clear separation between
> "configuration" and "building" is very much necessary to maintain sanity.
>
> Cheers,
>
> - Ben
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220719/a3dbb38a/attachment.html>

From simon.peytonjones at gmail.com  Wed Jul 20 07:59:44 2022
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Wed, 20 Jul 2022 08:59:44 +0100
Subject: GHC development asks too much of the host system
In-Reply-To: <877d482a21.fsf@smart-cactus.org>
References: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
 <877d482a21.fsf@smart-cactus.org>
Message-ID: <CAJKmMz8ko1mGcZ166X6DZjzA=x-5d0Av-w03=SVB3ftBqW_oBA@mail.gmail.com>

This is a great thread.  I would love it to conclude with some concrete
actions, rather than just petering out.

I think one underexplored approach to addressing the build-time problem
> is to look not at the full-build time but rather look for common tasks
> where we could *avoid* doing a full build (e.g. updating documentation,
> typechecking `base`, running a "good enough" subset of the testsuite)
> and find ways to make those workflows more efficient.
>

This is a great example of a concrete step.

   - Identify common tasks
   - Write done the procedure for doing that task

I suspect that much waiting time is just because we are building too much.
e.g. if all you want to do is typeset Haddock docs, you probably don't need
the RTS build in threaded-debug way (or even at all).

As to the testsuite, 99% of them run fast.  If there are some slow culprits
that are causing genuine pain, let's identify them.

Simon

On Tue, 19 Jul 2022 at 20:11, Ben Gamari <ben at smart-cactus.org> wrote:

> Hécate <hecate at glitchbra.in> writes:
>
> > Hello ghc-devs,
> >
> > I hadn't made significant contributions to the GHC code base in a while,
> > until a few days ago, where I discovered that my computer wasn't able to
> > sustain running the test suite, nor handle HLS well.
> >
> > Whether it is my OS automatically killing the process due to oom-killer
> > or just the fact that I don't have a war machine, I find it too bad and
> > I'm frankly discouraged.
>
> Do you know which process was being killed? There is one testsuite tests
> that I know of which does have quite a considerable memory footprint
> (T16992) due to its nature; otherwise I would expect a reasonably recent
> machine to pass the testsuite without much trouble. It's particularly
> concerning if this is a new regression; is this the first time you have
> observed this particular failure?
>
> > This is not the first time such feedback emerges, as the documentation
> > task force for the base library was unable to properly onboard some
> > people from third-world countries who do not have access to hardware
> > we'd consider "standard" in western Europe or some parts of North
> > America. Or at least "standard" until even my standard stuff didn't cut
> > it anymore.
> >
> > So yeah, I'll stay around but I'm afraid I'm going to have to focus on
> > projects for which the feedback loop is not on the scale of hours , as
> > this is a hobby project.
> >
> > Hope this will open some eyes.
> >
> Hi Hécate,
>
> I would reiterate that the more specific feedback you can offer, the
> better.
>
> To share my some of my own experience: I have access to a variety of
> hardware,
> some of which is quite powerful. However, I find that I end up doing
> much of my development on my laptop which, while certainly not a slouch
> (being a Ryzen 4750U), is also not a monster. In particular, while a
> fresh build takes nearly twice as long on my laptop than some of the
> other hardware I have, I nevertheless find ways to make it worthwhile
> (due to the ease of iteration compared to ssh). If you routinely have
> multi-hour iteration times then something isn't right.
>
> In particular, I think there are a few tricks which make life far
> easier:
>
>
>  * Be careful about doing things that would incur
>    significant amounts of rebuilding. This includes:
>
>     * After modifying, e.g., `compiler/ghc.cabal.in` (e.g. to add a new
>       module to GHC), modify `compiler/ghc.cabal` manually instead of
>       rerunning `configure`.
>
>     * Be careful about pulling/rebase. I generally pick a base commit to
>       build off of and rebase sparingly: Having to stop what I'm doing to
>       wait for full rebuild is an easy way to lose momentum.
>
>     * Avoid switching branches; I generally have a GHC tree per on-going
>       project.
>
>  * Take advantage of Hadrian's `--freeze1` flag
>
>  * Use `hadrian/ghci` to typecheck changes
>
>  * Use the stage1 compiler instead of stage2 to smoke-test changes when
>    possible. (specifically, using the script generated by Hadrian's
>    `_build/ghc-stage1` target)
>
>  * Use the right build flavour for the task at hand: If I don't need a
>    performant compiler and am confident that I can get by without
>    thorough testsuite validation, I use `quick`. Otherwise, plan ahead
>    for what you need (e.g. `default+assertions+debug_info` or
>    `validate`)
>
>  * Run the fraction of the testsuite that is relevant to your change.
>    Hadrian's `--test-way` and `--only` flags are your friends.
>
>  * Take advantage of CI. At the moment we have a fair amount of CI
>    capacity. If you think that your change is close to working, you can
>    open an MR and start a build locally. If it fails, iterate on just the
>    failing testcases locally.
>
>  * Task-level parallelism. Admittedly, this is harder when you are
>    working as a hobby, but I often have two or three projects on-going
>    at a time. While one tree is building I try to make progress on
>    another.
>
> I don't use HLS so I may be insulated from some of the pain in this
> regard. However, I do know that Matt is a regular user and he
> disables most plugins.
>
> I would also say that, sadly, GHC is comparable to other similarly-size
> compilers in its build time: A build of LLVM (not even clang) takes ~50
> minutes on my 8-core desktop; impressively, rustc takes ~7 minutes
> although it is a considerably smaller compiler (being just a front-end).
> By contrast, GHC takes around 20 minutes. I know that this doesn't
> make the cost any easier to bear and I would love to bring this number
> down, but ultimately there are only so many hours in the day.
>
> I think one underexplored approach to addressing the build-time problem
> is to look not at the full-build time but rather look for common tasks
> where we could *avoid* doing a full build (e.g. updating documentation,
> typechecking `base`, running a "good enough" subset of the testsuite)
> and find ways to make those workflows more efficient.
>
> Cheers,
>
> - Ben
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220720/00daf45f/attachment.html>

From ben at smart-cactus.org  Wed Jul 20 16:01:12 2022
From: ben at smart-cactus.org (Ben Gamari)
Date: Wed, 20 Jul 2022 12:01:12 -0400
Subject: GHC development asks too much of the host system
In-Reply-To: <D47D25EF-3991-4E25-8B22-11636324F0F5@gmail.com>
References: <877d482a21.fsf@smart-cactus.org>
 <D47D25EF-3991-4E25-8B22-11636324F0F5@gmail.com>
Message-ID: <87zgh3zsct.fsf@smart-cactus.org>

Rodrigo Mesquita <rodrigo.m.mesquita at gmail.com> writes:

> Dear Ben,
>
> The list of tips you put together is quite nice.
>
> I suggest we add it to hadrian’s wiki page under a “Tips for making
> your life easier” section (as is, it is already useful! at least I
> learned something new).
>
Hi Rodrigo,

I'm happy to hear that it was useful! Indeed I always struggle to know
where to put this information; our developer documentation is quite
scattered and it's hard to know where new contributors will enter it.
Nevertheless, the page that you mention seems like an obvious place for
these tips.

Incidentally, I have long felt that migrating the Wiki documentation
into a comprehensive developer's guide (see #18853) in the repository
(much like the Rustc Book [1]) would be a useful direction of travel. It
would be both prompt us review (and, in some cases, retire) existing and
give us the opportunity to structure the documentation in a more
coherent manner. Moreover, the fact that it would be part of the
repository would both give us the benefit of code review and make it
easier to ensure that the documentation remains consistent with the
code. This is obviously a longer-term goal, but one which it would be
great to have help in moving towards.

Cheers,

- Ben


[1] https://doc.rust-lang.org/rustc/contributing.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220720/b1f6459d/attachment.sig>

From george.colpitts at gmail.com  Wed Jul 20 16:11:35 2022
From: george.colpitts at gmail.com (George Colpitts)
Date: Wed, 20 Jul 2022 13:11:35 -0300
Subject: GHC development asks too much of the host system
In-Reply-To: <87zgh3zsct.fsf@smart-cactus.org>
References: <877d482a21.fsf@smart-cactus.org>
 <D47D25EF-3991-4E25-8B22-11636324F0F5@gmail.com>
 <87zgh3zsct.fsf@smart-cactus.org>
Message-ID: <CAB-d4A58oPYXd6PJt6HQaWnUuE-sbJFy39X2a5Vb5HFb1j4Cmw@mail.gmail.com>

Great idea, Ben. It would also make getting started less intimidating
particularly if it had sections on changing the doc and adding test cases.
The latter areas would be a good way for non experts to help move ghc
forward. Of course those processes would ideally not involve building ghc.
Less expert software developers are experts on what needs more
explanation for less sophisticated developers  in e.g. the  hackage doc.
Similarly many people who submit bugs can't fix them but they could write a
test case.

On Wed, Jul 20, 2022 at 1:01 PM Ben Gamari <ben at smart-cactus.org> wrote:

> Rodrigo Mesquita <rodrigo.m.mesquita at gmail.com> writes:
>
> > Dear Ben,
> >
> > The list of tips you put together is quite nice.
> >
> > I suggest we add it to hadrian’s wiki page under a “Tips for making
> > your life easier” section (as is, it is already useful! at least I
> > learned something new).
> >
> Hi Rodrigo,
>
> I'm happy to hear that it was useful! Indeed I always struggle to know
> where to put this information; our developer documentation is quite
> scattered and it's hard to know where new contributors will enter it.
> Nevertheless, the page that you mention seems like an obvious place for
> these tips.
>
> Incidentally, I have long felt that migrating the Wiki documentation
> into a comprehensive developer's guide (see #18853) in the repository
> (much like the Rustc Book [1]) would be a useful direction of travel. It
> would be both prompt us review (and, in some cases, retire) existing and
> give us the opportunity to structure the documentation in a more
> coherent manner. Moreover, the fact that it would be part of the
> repository would both give us the benefit of code review and make it
> easier to ensure that the documentation remains consistent with the
> code. This is obviously a longer-term goal, but one which it would be
> great to have help in moving towards.
>
> Cheers,
>
> - Ben
>
>
> [1] https://doc.rust-lang.org/rustc/contributing.html
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220720/a6cb486e/attachment.html>

From ben at well-typed.com  Wed Jul 20 16:34:19 2022
From: ben at well-typed.com (Ben Gamari)
Date: Wed, 20 Jul 2022 12:34:19 -0400
Subject: GHC development asks too much of the host system
In-Reply-To: <CAPKAvQXuHzvYkrovsPmVXRNLFT0fuhj1ta6+7Xu7TSR4xd4d4Q@mail.gmail.com>
References: <CAPKAvQX3hckQ_HHdwc=vA+smHA_zhfTvYAt+BKxhOp6mx_7yLQ@mail.gmail.com>
 <874jzc222a.fsf@smart-cactus.org>
 <CAPKAvQXuHzvYkrovsPmVXRNLFT0fuhj1ta6+7Xu7TSR4xd4d4Q@mail.gmail.com>
Message-ID: <87v8rrzqtl.fsf@smart-cactus.org>

Artem Pelenitsyn <a.pelenitsyn at gmail.com> writes:

> Thanks Ben, very interesting, especially the cloud Shake stuff.
>
>> If everyone were to use, e.g.,
>>   ghc.nix this could be largely mitigated, but this isn't the world in
>>   which we live.
>
> I don't know why you'd want everyone to use it necessarily to improve
> things. If there was a clear statement that you may elect to use
> ghc.nix and get X% speedup, that would be a good start. And then
> people can decide, based on their setup and their aversion to tools
> like Nix, etc. in general.
>
> The real issue is that currently you don't benefit much from ghc.nix
> because the main performance sink is the GHC tree itself. The way out
> is to use a cloud build system, most likely Cloud Shake, which, as you
> describe, has a couple of issues in this case:
>
> 1) Native dependencies. This should be possible to solve via Nix, but
>    unfortunately, this is not quite there yet because Shake doesn't
>    know about Nix (afaik). I think to actually get
>    there, you'd need some sort of integration between Nix and Shake akin to
>    what Tweag built for Nix and Bazel (cf. rules_nixpkgs [1]). Their
>    moto is: Nix for "external" or "system" components, and Bazel for
>    "internal" or "local" ones.

I disagree. Caching is quite feasible while maintaining a clear division
between configuration and the build system.

Today, the only channel of communication between `configure` and Hadrian
is `hadrian/cfg/system.config`. If `ghc.nix` is doing its job correctly
then two invocations of `./configure` in nix-shell on two different
machines should end up with the same `hadrian/cfg/system.config`.
Further, if two trees have the same contents in that file, then they
can share build artifacts [1].


However, to reiterate, the real problem here is the one below:


> 2) Bootstrapping aspect. Maybe this is a challenge for rebuilds after
>    modification, but I think people on this thread were quoting the
>    "time to first build" more. I don't see how avoiding to build
>    master locally after a fresh (worktree) checkout by downloading
>    build results from somewhere, connects to bootstrapping. I think it
>    doesn't.

If you merely want to build `master` then indeed caching would work
fine. However, in that case you could have also just downloaded a binary
distribution from GitLab. The problem is that usually the reason that
you want to build `master` is that you then want to *modify* it.
In general, a modification of `master` will require rebuilding some
subset of the stage1 ghc, which will then require a full build of
stage2 (which includes GHC, as well as all of the boot libraries,
`base`, etc.). The latter would see zero cache hits since one of the
inputs, the stage 1 GHC, has changed. Unfortunately, the latter is also
well over half of the build effort.

> As for rebuilds. People are already using --freeze1 (you suggested it
> earlier in this very thread!),
>
Yes, but they are doing so explicitly after having already built their
branch to produce a consistent stage1 compiler. If you checkout
`master`, build stage 1, switch to some arbitrary branch, and attempt to
build stage2 with --freeze1, chances are you will end up with a broken
compiler. In the best case this will manifest as a build failure.
However, there is a non-negigible possibility that the outcome is far
more sinister (e.g. segmentation faults).

> so I don't see how saying "freezing stage 1 is dangerous even if
> faster" connects to practise of GHC development. Of course, you may
> not find a remote cache with relevant artefacts after local updates,
> but that's not the point. The point is to not have to build `master`,
> not `feaure-branch-t12345`. Rebuilds should be rather pain-free in
> comparison.
>
For safe caching we must be certain that the build graph is accurate
and complete. However, we know with certainty that it currently is not and
fixing this requires real implementation effort (David Eichmann
spent a few months on this problem in 2019; see #16926 and related
tickets). Consequently, we must weigh the benefit of caching against the
development cost. Currently, my sense is that the benefit would be some
subset of the stage 1 build could be shared some of the time. This
strikes me as a rather small benefit compared to the cost.

Of course, we would love to have help in addressing #16926. In principle
having build caching would be nice; however, at the moment we just don't
believe that putting precious GHC team resources towards that goal is
the most effective way to serve users. If someone were to come along and
start chipping away at the #16926, we would be happy to advise and
assist.

Cheers,

 - Ben


[1] Strictly speaking, I don't believe this is quite true today since
    the absolute path of the working tree almost certainly leaks into
    the build artifacts. However, in principle this could be fixed.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220720/6693c803/attachment.sig>

From ben at smart-cactus.org  Wed Jul 20 16:46:14 2022
From: ben at smart-cactus.org (Ben Gamari)
Date: Wed, 20 Jul 2022 12:46:14 -0400
Subject: GHC development asks too much of the host system
In-Reply-To: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
References: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
Message-ID: <87r12fzq9p.fsf@smart-cactus.org>

Hécate <hecate at glitchbra.in> writes:

> Hello ghc-devs,
>
> I hadn't made significant contributions to the GHC code base in a while, 
> until a few days ago, where I discovered that my computer wasn't able to 
> sustain running the test suite, nor handle HLS well.
>
> Whether it is my OS automatically killing the process due to oom-killer 
> or just the fact that I don't have a war machine, I find it too bad and 
> I'm frankly discouraged.

Hi Hécate,

I have opened #21892 to track the issue of large testsuite tests. If you
are able, do leave some references on that ticket regarding which tests
were giving you trouble. I'm going to try to do some measurements on my
end as well, but it would be good to know that I am seeing similar
things to what you are seeing.

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220720/47f41c99/attachment.sig>

From steven at steshaw.org  Thu Jul 21 00:58:25 2022
From: steven at steshaw.org (Steven Shaw)
Date: Thu, 21 Jul 2022 10:58:25 +1000
Subject: GHC development asks too much of the host system
In-Reply-To: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
References: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
Message-ID: <CAMP6nJxtZC=UPy-1-ggSjN=SLzbkUBe_5_bPCpeE3c-M4DFxjg@mail.gmail.com>

Hi Hécate,

What would help here is a Gitpod or GitHub Codespace for working on GHC.
I've been meaning to set up a Gitpod following the recent article by Cheng
Shao:

https://dev.to/terrorjack/my-ghc-dev-environment-with-vscode-remote-docker-3ek4

I'll see if I can find time this weekend, but don't let that stop anyone
from giving it a go.

Cheers,
Steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220721/abf663d0/attachment.html>

From steven at steshaw.org  Thu Jul 21 01:02:23 2022
From: steven at steshaw.org (Steven Shaw)
Date: Thu, 21 Jul 2022 11:02:23 +1000
Subject: GHC development asks too much of the host system
In-Reply-To: <CAMP6nJxtZC=UPy-1-ggSjN=SLzbkUBe_5_bPCpeE3c-M4DFxjg@mail.gmail.com>
References: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
 <CAMP6nJxtZC=UPy-1-ggSjN=SLzbkUBe_5_bPCpeE3c-M4DFxjg@mail.gmail.com>
Message-ID: <CAMP6nJyY2cR-On6RfAmPiyz4U3W94yWw40tCfMqQJ=RP+hFvfg@mail.gmail.com>

In the meantime, you could try this Gitpod setup that Andrea Bedini has
been working on:

https://github.com/andreabedini/ghc/tree/andrea/gitpod

Cheers,
Steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220721/1b9893df/attachment.html>

From steven at steshaw.org  Thu Jul 21 01:19:27 2022
From: steven at steshaw.org (Steven Shaw)
Date: Thu, 21 Jul 2022 11:19:27 +1000
Subject: GHC development asks too much of the host system
In-Reply-To: <CAMP6nJyY2cR-On6RfAmPiyz4U3W94yWw40tCfMqQJ=RP+hFvfg@mail.gmail.com>
References: <91241c72-3df2-e691-ef31-7b3a512d1a20@glitchbra.in>
 <CAMP6nJxtZC=UPy-1-ggSjN=SLzbkUBe_5_bPCpeE3c-M4DFxjg@mail.gmail.com>
 <CAMP6nJyY2cR-On6RfAmPiyz4U3W94yWw40tCfMqQJ=RP+hFvfg@mail.gmail.com>
Message-ID: <CAMP6nJwzE0+uZR=YsMq=ogQnOxVAnVw5QsCW8Vcwb8gOaAXigg@mail.gmail.com>

Oh my, I just discovered that Cheng Shao has recently put something
together for working on GHC with Gitpod:

https://github.com/TerrorJack/gitpod-ghc

However, it looks like it isn't using the Debian images from the GHC CI.

Sorry for the noise.

Cheers,
Steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220721/af6eaaa5/attachment.html>

From gergo at erdi.hu  Fri Jul 22 06:23:18 2022
From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=)
Date: Fri, 22 Jul 2022 14:23:18 +0800 (+08)
Subject: Wildcards in type synonyms
Message-ID: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>

Hi,

I'd like to implement type synonyms containing wildcards. The idea is
that if you have `type MySyn a = MyType a _ Int`, then during
typechecking, every occurrence of `MySyn T` would be expanded into
`MyType T w123 Int`, with a fresh type (meta)variable `w123`.

One worrying thing I noticed in my initial exploration of the GHC
codebase is that the Core representation of `Type`s can still contain
type synonym occurrences. And this doesn't seem like just an artefact
of sharing the `Type` representation with `TcType`, since the
`coreView` function also has code to look through type synonyms.

What is the reason for this? I would have expected type synonyms to be
only relevant during typechecking, and then fully resolved in the
elaborated Core output. If that were the case, then a new version of
`expand_syn` could live in `TcM` and take care of making these fresh
metavariables.

Beside this concrete question, taking a step back, I would also like
to hear from people who know their way around this part of GHC, what
they think about this and how they'd approach implementing it.

Thanks,
         Gergo

From iavor.diatchki at gmail.com  Fri Jul 22 06:44:32 2022
From: iavor.diatchki at gmail.com (Iavor Diatchki)
Date: Fri, 22 Jul 2022 09:44:32 +0300
Subject: Wildcards in type synonyms
In-Reply-To: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
Message-ID: <CAGK9nupnrnEyMQ6gNAbLdekFjU9Lti7ZV+ThyBwcvsa0gVkFNQ@mail.gmail.com>

Hello,

I've implemented such a feature in Cryptol, not GHC, so it is quite doable,
but I think the implementation would be easier if you decided on the
overall design of the feature first.

Some things to consider:
  * these kind of "existential" variable make sense in other type
signatures, not just type synonyms
   * there might be some contexts where you may want to disallow such
wildcards (e. g., in a data declaration)
   * you have to be careful with the scoping of type variables.  For
example, you should not unify an existential/wildcard variable with a type
that refers to variables that are not in scope of the wildcard.  I don't
remember if GHC already has a system for such things, but in Cryptol we
implanted this by having each unification variable keep track of the
quantified variables that it may depend on.

Hope this helps,
Iavor

On Fri, Jul 22, 2022, 09:30 ÉRDI Gergő <gergo at erdi.hu> wrote:

> Hi,
>
> I'd like to implement type synonyms containing wildcards. The idea is
> that if you have `type MySyn a = MyType a _ Int`, then during
> typechecking, every occurrence of `MySyn T` would be expanded into
> `MyType T w123 Int`, with a fresh type (meta)variable `w123`.
>
> One worrying thing I noticed in my initial exploration of the GHC
> codebase is that the Core representation of `Type`s can still contain
> type synonym occurrences. And this doesn't seem like just an artefact
> of sharing the `Type` representation with `TcType`, since the
> `coreView` function also has code to look through type synonyms.
>
> What is the reason for this? I would have expected type synonyms to be
> only relevant during typechecking, and then fully resolved in the
> elaborated Core output. If that were the case, then a new version of
> `expand_syn` could live in `TcM` and take care of making these fresh
> metavariables.
>
> Beside this concrete question, taking a step back, I would also like
> to hear from people who know their way around this part of GHC, what
> they think about this and how they'd approach implementing it.
>
> Thanks,
>          Gergo
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220722/64b7f9b7/attachment.html>

From gergo at erdi.hu  Fri Jul 22 06:59:13 2022
From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=)
Date: Fri, 22 Jul 2022 14:59:13 +0800 (+08)
Subject: Wildcards in type synonyms
In-Reply-To: <CAGK9nupnrnEyMQ6gNAbLdekFjU9Lti7ZV+ThyBwcvsa0gVkFNQ@mail.gmail.com>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
 <CAGK9nupnrnEyMQ6gNAbLdekFjU9Lti7ZV+ThyBwcvsa0gVkFNQ@mail.gmail.com>
Message-ID: <alpine.DEB.2.22.394.2207221456570.4398@galaxy>

On Fri, 22 Jul 2022, Iavor Diatchki wrote:

> I've implemented such a feature in Cryptol, not GHC, so it is quite doable, but I
> think the implementation would be easier if you decided on the overall design of the
> feature first.

I'm hoping the details pretty much fall out from what it would mean if I 
used the existing PartialTypeSignatures extension to write 'MyData T _ Int`
at those places.

> Some things to consider:
>   * these kind of "existential" variable make sense in other type signatures, not just
> type synonyms
>    * there might be some contexts where you may want to disallow such wildcards (e.
> g., in a data declaration)
>    * you have to be careful with the scoping of type variables.  For example, you
> should not unify an existential/wildcard variable with a type that refers to variables
> that are not in scope of the wildcard.  I don't remember if GHC already has a system
> for such things, but in Cryptol we implanted this by having each unification variable
> keep track of the quantified variables that it may depend on.

From simon.peytonjones at gmail.com  Fri Jul 22 08:14:03 2022
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Fri, 22 Jul 2022 09:14:03 +0100
Subject: Wildcards in type synonyms
In-Reply-To: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
Message-ID: <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>

Hi Gergo

I'd like to implement type synonyms containing wildcards. The idea is
> that if you have `type MySyn a = MyType a _ Int`, then during
> typechecking, every occurrence of `MySyn T` would be expanded into
> `MyType T w123 Int`, with a fresh type (meta)variable `w123`.
>

I imagine you mean that if you then write
f :: MySyn a -> a -> Int
then it's exactly as if you wrote (using PartialTypeSignatures)
f :: MyType a _ Int -> a -> Int
today.   So if I understand it right, you intend that these type synonyms
are second-class citizens: *they can occur precisely (and only) where
wildcards are allowed to occur today*.  For example, as Iavor suggests,
you'd reject
data T a = MkT (MySyn a)

If you want to do this in a fork of GHC, that's obviously fine. If you want
to offer it as a language extension, the best thing would be to write a GHC
Proposal.  Also you'd get a lot of useful design feedback that way, which
might save you implementation cycles.

What is the reason for this? I would have expected type synonyms to be
> only relevant during typechecking, and then fully resolved in the
> elaborated Core output.
>

In GHC *an Id has only one type*.   It does not have a "source type" and a
"Core type".  So we allow Core types to contain synonyms so that when we
export that Id the importing scope (e.g. GHCi, and type error messages) can
see it.  Synonyms can also allow types to take less space. E.g.  we have
Type, where (if we fully expanded) we'd have to have (TYPE LiftedRep).  One
could imagine a different design.

I would expect that, by the time typechecking is over, all your wildcard
synonyms are gone.  They really are second class.

Just to mention too that the entire "wildcards in type signatures" story is
(I think) jolly useful, but it also turned out to be pretty tricky to
implement.  If you just macro-expand your new synonyms, you won't disturb
the wildcard story, but I just wanted to advertise that it's a tricky area.

Simon

On Fri, 22 Jul 2022 at 07:30, ÉRDI Gergő <gergo at erdi.hu> wrote:

> Hi,
>
> I'd like to implement type synonyms containing wildcards. The idea is
> that if you have `type MySyn a = MyType a _ Int`, then during
> typechecking, every occurrence of `MySyn T` would be expanded into
> `MyType T w123 Int`, with a fresh type (meta)variable `w123`.
>
> One worrying thing I noticed in my initial exploration of the GHC
> codebase is that the Core representation of `Type`s can still contain
> type synonym occurrences. And this doesn't seem like just an artefact
> of sharing the `Type` representation with `TcType`, since the
> `coreView` function also has code to look through type synonyms.
>
> What is the reason for this? I would have expected type synonyms to be
> only relevant during typechecking, and then fully resolved in the
> elaborated Core output. If that were the case, then a new version of
> `expand_syn` could live in `TcM` and take care of making these fresh
> metavariables.
>
> Beside this concrete question, taking a step back, I would also like
> to hear from people who know their way around this part of GHC, what
> they think about this and how they'd approach implementing it.
>
> Thanks,
>          Gergo
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220722/2d3c726b/attachment.html>

From gergo at erdi.hu  Fri Jul 22 08:50:45 2022
From: gergo at erdi.hu (=?UTF-8?B?R2VyZ8WRIMOJcmRp?=)
Date: Fri, 22 Jul 2022 16:50:45 +0800
Subject: Wildcards in type synonyms
In-Reply-To: <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
 <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>
Message-ID: <CAO7EwMrJGL9YVKrY+h_2uwSfA_TZ_0VGJSExtkbwTPYGCcAAQA@mail.gmail.com>

Thanks Simon, these are very useful hints. My plan is to just push
ahead on a separate fork, with this "macro semantics", and maybe if it
comes out nicer than I'd hope, I'll propose it.

So it seems that instead of shoehorning it into the existing type
synonyms, a better bet would be to branch off to a separate path quite
early (maybe as soon as during renaming), expand them during
typechecking, and *not* touch how/when existing "normal" type synonyms
are resolved.

On Fri, Jul 22, 2022 at 4:14 PM Simon Peyton Jones
<simon.peytonjones at gmail.com> wrote:
>
> Hi Gergo
>
>> I'd like to implement type synonyms containing wildcards. The idea is
>> that if you have `type MySyn a = MyType a _ Int`, then during
>> typechecking, every occurrence of `MySyn T` would be expanded into
>> `MyType T w123 Int`, with a fresh type (meta)variable `w123`.
>
>
> I imagine you mean that if you then write
> f :: MySyn a -> a -> Int
> then it's exactly as if you wrote (using PartialTypeSignatures)
> f :: MyType a _ Int -> a -> Int
> today.   So if I understand it right, you intend that these type synonyms are second-class citizens: they can occur precisely (and only) where wildcards are allowed to occur today.  For example, as Iavor suggests, you'd reject
> data T a = MkT (MySyn a)
>
> If you want to do this in a fork of GHC, that's obviously fine. If you want to offer it as a language extension, the best thing would be to write a GHC Proposal.  Also you'd get a lot of useful design feedback that way, which might save you implementation cycles.
>
>> What is the reason for this? I would have expected type synonyms to be
>> only relevant during typechecking, and then fully resolved in the
>> elaborated Core output.
>
>
> In GHC an Id has only one type.   It does not have a "source type" and a "Core type".  So we allow Core types to contain synonyms so that when we export that Id the importing scope (e.g. GHCi, and type error messages) can see it.  Synonyms can also allow types to take less space. E.g.  we have Type, where (if we fully expanded) we'd have to have (TYPE LiftedRep).  One could imagine a different design.
>
> I would expect that, by the time typechecking is over, all your wildcard synonyms are gone.  They really are second class.
>
> Just to mention too that the entire "wildcards in type signatures" story is (I think) jolly useful, but it also turned out to be pretty tricky to implement.  If you just macro-expand your new synonyms, you won't disturb the wildcard story, but I just wanted to advertise that it's a tricky area.
>
> Simon
>
> On Fri, 22 Jul 2022 at 07:30, ÉRDI Gergő <gergo at erdi.hu> wrote:
>>
>> Hi,
>>
>> I'd like to implement type synonyms containing wildcards. The idea is
>> that if you have `type MySyn a = MyType a _ Int`, then during
>> typechecking, every occurrence of `MySyn T` would be expanded into
>> `MyType T w123 Int`, with a fresh type (meta)variable `w123`.
>>
>> One worrying thing I noticed in my initial exploration of the GHC
>> codebase is that the Core representation of `Type`s can still contain
>> type synonym occurrences. And this doesn't seem like just an artefact
>> of sharing the `Type` representation with `TcType`, since the
>> `coreView` function also has code to look through type synonyms.
>>
>> What is the reason for this? I would have expected type synonyms to be
>> only relevant during typechecking, and then fully resolved in the
>> elaborated Core output. If that were the case, then a new version of
>> `expand_syn` could live in `TcM` and take care of making these fresh
>> metavariables.
>>
>> Beside this concrete question, taking a step back, I would also like
>> to hear from people who know their way around this part of GHC, what
>> they think about this and how they'd approach implementing it.
>>
>> Thanks,
>>          Gergo
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From simon.peytonjones at gmail.com  Fri Jul 22 08:53:44 2022
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Fri, 22 Jul 2022 09:53:44 +0100
Subject: Wildcards in type synonyms
In-Reply-To: <CAO7EwMrJGL9YVKrY+h_2uwSfA_TZ_0VGJSExtkbwTPYGCcAAQA@mail.gmail.com>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
 <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>
 <CAO7EwMrJGL9YVKrY+h_2uwSfA_TZ_0VGJSExtkbwTPYGCcAAQA@mail.gmail.com>
Message-ID: <CAJKmMz9qcSpmVyDDyK5UABM-hx=jeiyxq9dN6m+3mooLQkoOcw@mail.gmail.com>

>
> So it seems that instead of shoehorning it into the existing type
> synonyms, a better bet would be to branch off to a separate path quite
> early (maybe as soon as during renaming), expand them during
> typechecking, and *not* touch how/when existing "normal" type synonyms
> are resolved.
>

That sounds plausible, yes.

Simon

On Fri, 22 Jul 2022 at 09:50, Gergő Érdi <gergo at erdi.hu> wrote:

> Thanks Simon, these are very useful hints. My plan is to just push
> ahead on a separate fork, with this "macro semantics", and maybe if it
> comes out nicer than I'd hope, I'll propose it.
>
> So it seems that instead of shoehorning it into the existing type
> synonyms, a better bet would be to branch off to a separate path quite
> early (maybe as soon as during renaming), expand them during
> typechecking, and *not* touch how/when existing "normal" type synonyms
> are resolved.
>
> On Fri, Jul 22, 2022 at 4:14 PM Simon Peyton Jones
> <simon.peytonjones at gmail.com> wrote:
> >
> > Hi Gergo
> >
> >> I'd like to implement type synonyms containing wildcards. The idea is
> >> that if you have `type MySyn a = MyType a _ Int`, then during
> >> typechecking, every occurrence of `MySyn T` would be expanded into
> >> `MyType T w123 Int`, with a fresh type (meta)variable `w123`.
> >
> >
> > I imagine you mean that if you then write
> > f :: MySyn a -> a -> Int
> > then it's exactly as if you wrote (using PartialTypeSignatures)
> > f :: MyType a _ Int -> a -> Int
> > today.   So if I understand it right, you intend that these type
> synonyms are second-class citizens: they can occur precisely (and only)
> where wildcards are allowed to occur today.  For example, as Iavor
> suggests, you'd reject
> > data T a = MkT (MySyn a)
> >
> > If you want to do this in a fork of GHC, that's obviously fine. If you
> want to offer it as a language extension, the best thing would be to write
> a GHC Proposal.  Also you'd get a lot of useful design feedback that way,
> which might save you implementation cycles.
> >
> >> What is the reason for this? I would have expected type synonyms to be
> >> only relevant during typechecking, and then fully resolved in the
> >> elaborated Core output.
> >
> >
> > In GHC an Id has only one type.   It does not have a "source type" and a
> "Core type".  So we allow Core types to contain synonyms so that when we
> export that Id the importing scope (e.g. GHCi, and type error messages) can
> see it.  Synonyms can also allow types to take less space. E.g.  we have
> Type, where (if we fully expanded) we'd have to have (TYPE LiftedRep).  One
> could imagine a different design.
> >
> > I would expect that, by the time typechecking is over, all your wildcard
> synonyms are gone.  They really are second class.
> >
> > Just to mention too that the entire "wildcards in type signatures" story
> is (I think) jolly useful, but it also turned out to be pretty tricky to
> implement.  If you just macro-expand your new synonyms, you won't disturb
> the wildcard story, but I just wanted to advertise that it's a tricky area.
> >
> > Simon
> >
> > On Fri, 22 Jul 2022 at 07:30, ÉRDI Gergő <gergo at erdi.hu> wrote:
> >>
> >> Hi,
> >>
> >> I'd like to implement type synonyms containing wildcards. The idea is
> >> that if you have `type MySyn a = MyType a _ Int`, then during
> >> typechecking, every occurrence of `MySyn T` would be expanded into
> >> `MyType T w123 Int`, with a fresh type (meta)variable `w123`.
> >>
> >> One worrying thing I noticed in my initial exploration of the GHC
> >> codebase is that the Core representation of `Type`s can still contain
> >> type synonym occurrences. And this doesn't seem like just an artefact
> >> of sharing the `Type` representation with `TcType`, since the
> >> `coreView` function also has code to look through type synonyms.
> >>
> >> What is the reason for this? I would have expected type synonyms to be
> >> only relevant during typechecking, and then fully resolved in the
> >> elaborated Core output. If that were the case, then a new version of
> >> `expand_syn` could live in `TcM` and take care of making these fresh
> >> metavariables.
> >>
> >> Beside this concrete question, taking a step back, I would also like
> >> to hear from people who know their way around this part of GHC, what
> >> they think about this and how they'd approach implementing it.
> >>
> >> Thanks,
> >>          Gergo
> >> _______________________________________________
> >> ghc-devs mailing list
> >> ghc-devs at haskell.org
> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220722/7e7a7229/attachment.html>

From ben at well-typed.com  Fri Jul 22 13:05:15 2022
From: ben at well-typed.com (Ben Gamari)
Date: Fri, 22 Jul 2022 09:05:15 -0400
Subject: [ANNOUNCE] GHC 9.4.1-rc1 is now available
Message-ID: <87o7xhz4ax.fsf@smart-cactus.org>


The GHC developers are happy to announce the availability of the first
(and likely last) release candidate of GHC 9.4.1. Binary distributions, source
distributions, and documentation are available at [downloads.haskell.org].

This major release will include:

 - A new profiling mode, `-fprof-late`, which adds automatic cost-center
   annotations to all top-level functions *after* Core optimisation has
   run. This provides informative profiles while interfering
   significantly less with GHC's aggressive optimisations, making it
   easier to understand the performance of programs which depend upon
   simplification..

 - A variety of plugin improvements including the introduction of a new
   plugin type, *defaulting plugins*, and the ability for typechecking
   plugins to rewrite type-families.

 - An improved constructed product result analysis, allowing unboxing of
   nested structures, and a new boxity analysis, leading to less reboxing.

 - Introduction of a tag-check elision optimisation, bringing
   significant performance improvements in strict programs.

 - Generalisation of a variety of primitive types to be levity
   polymorphic. Consequently, the `ArrayArray#` type can at long last be
   retired, replaced by standard `Array#`.

 - Introduction of the `\cases` syntax from [GHC proposal 0302]

 - A complete overhaul of GHC's Windows support. This includes a
   migration to a fully Clang-based C toolchain, a deep refactoring of
   the linker, and many fixes in WinIO.

 - Support for multiple home packages, significantly improving support
   in IDEs and other tools for multi-package projects.

 - A refactoring of GHC's error message infrastructure, allowing GHC to
   provide diagnostic information to downstream consumers as structured
   data, greatly easing IDE support.

 - Significant compile-time improvements to runtime and memory consumption.

 - On overhaul of our packaging infrastructure, allowing full
   traceability of release artifacts and more reliable binary
   distributions.

 - Reintroduction of deep subsumption (which was previously dropped with the
   *simplified subsumption* change) as a language extension.

 - ... and much more. See the [release notes] for a full accounting.

Note that, as 9.4.1 is the first release for which the released artifacts will
all be generated by our Hadrian build system, it is possible that there will be
packaging issues. If you enounter trouble while using a binary distribution,
please open a [ticket]. Likewise, if you are a downstream packager, do consider
migrating to [Hadrian] to run your build; the Hadrian build system can be built
using `cabal-install`, `stack`, or the in-tree [bootstrap script]. We will be
publishing a blog post describing the migration process to Hadrian in the
coming weeks.

We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk
stake pool, Tweag I/O, Serokell, Equinix, SimSpace, the Haskell
Foundation, and other anonymous contributors whose on-going financial
and in-kind support has facilitated GHC maintenance and release
management over the years. Finally, this release would not have been
possible without the hundreds of open-source contributors whose work
comprise this release.

As always, do give this release a try and open a [ticket] if you see
anything amiss.

Happy testing,

- Ben

[downloads.haskell.org]: https://downloads.haskell.org/ghc/9.4.1-rc1/
[GHC proposal 0302]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0302-cases.rst 
[ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new
[bootstrap script]: https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d11c07d35/hadrian/bootstrap/README.md
[Hadrian]: https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d11c07d35/hadrian
[release notes]: https://downloads.haskell.org/~ghc/9.4.1-rc1/docs/users_guide/9.4.1-notes.html

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220722/11ec042d/attachment.sig>

From lists at richarde.dev  Fri Jul 22 16:53:44 2022
From: lists at richarde.dev (Richard Eisenberg)
Date: Fri, 22 Jul 2022 16:53:44 +0000
Subject: Wildcards in type synonyms
In-Reply-To: <CAJKmMz9qcSpmVyDDyK5UABM-hx=jeiyxq9dN6m+3mooLQkoOcw@mail.gmail.com>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
 <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>
 <CAO7EwMrJGL9YVKrY+h_2uwSfA_TZ_0VGJSExtkbwTPYGCcAAQA@mail.gmail.com>
 <CAJKmMz9qcSpmVyDDyK5UABM-hx=jeiyxq9dN6m+3mooLQkoOcw@mail.gmail.com>
Message-ID: <010f018226d44e31-e6c9db0d-e1d8-42d3-be05-411b6f5d2bbc-000000@us-east-2.amazonses.com>



> On Jul 22, 2022, at 4:53 AM, Simon Peyton Jones <simon.peytonjones at gmail.com> wrote:
> 
> expand them during
> typechecking,

Just to expand on this point (haha): your new type macros (distinct from type synonyms) would have to be eagerly expanded during type checking. You say this (quoted above), but I wanted to highlight that this is in opposition to the way today's type synonyms work, which are expanded only when necessary. (Rationale: programmers probably want to retain the very clever synonym name they came up with, which is hopefully easier to reason about.)

Interestingly, type macros may solve another problem that has come up recently: Gershom proposed (a quick search couldn't find where this was, but it was around restoring deep-subsumption behavior) a change to the way polytypes work in type synonyms. Specifically, he wondered about, e.g.

type T a = forall b. b -> Either a b

meaning to take the `forall b` and lift it to the top of whatever type T appears in. So that

f :: [a] -> T a

would really mean

f :: forall a b. [a] -> b -> Either a b

and not

f :: forall a. [a] -> forall b. b -> Either a b

as it does today. With deep subsumption, you can spot the difference between these types only with type applications, but they are incomparable types with simple subsumption.

At the time, I didn't understand what the semantics of Gershon's new type synonyms were, but in the context of Gergo's type macros, they make sense. To wit, we could imagine

type T a = b -> Either a b

Note: b is unbound. This is actually a type *macro*, not a type synonym, and it expands to a form that mentions a free variable b. (Presumably, this b would not capture a b in scope at the usage site.) This macro behavior delivers what Gershom was looking for.

I'm not saying Gergo should necessarily implement this new aspect of type macros (that don't mention wildcards), but if this ever does come to a proposal, I think these kind of variables are a new motivator for such a proposal. I'd probably favor some explicit notation to introduce a macro (e.g. `type macro T a = Either _ a`) instead of using a syntactic marker like the presence of a _ or an unbound variable, but we can debate that later.

Good luck with the implementation, Gergo!
Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220722/49fb1f73/attachment.html>

From george.colpitts at gmail.com  Fri Jul 22 17:43:25 2022
From: george.colpitts at gmail.com (George Colpitts)
Date: Fri, 22 Jul 2022 14:43:25 -0300
Subject: [ANNOUNCE] GHC 9.4.1-rc1 is now available
In-Reply-To: <87o7xhz4ax.fsf@smart-cactus.org>
References: <87o7xhz4ax.fsf@smart-cactus.org>
Message-ID: <CAB-d4A4EgmQT6-GWbyi_sYWpf1cFk9Nughq7mL50qyK=XGkDqA@mail.gmail.com>

Hi Ben,

I expected https://gitlab.haskell.org/ghc/ghc/-/issues/21506 (ghc-9.4.1-alpha1
does not install on macos: ghc-pkg-9.4.0-20220501 cannot be opened because
the developer cannot be verified) to be fixed in rc1 but it is not. Are my
expectations wrong? What is the ETA for fixing it?

Thanks
George


On Fri, Jul 22, 2022 at 10:05 AM Ben Gamari <ben at well-typed.com> wrote:

>
> The GHC developers are happy to announce the availability of the first
> (and likely last) release candidate of GHC 9.4.1. Binary distributions,
> source
> distributions, and documentation are available at [downloads.haskell.org].
>
> This major release will include:
>
>  - A new profiling mode, `-fprof-late`, which adds automatic cost-center
>    annotations to all top-level functions *after* Core optimisation has
>    run. This provides informative profiles while interfering
>    significantly less with GHC's aggressive optimisations, making it
>    easier to understand the performance of programs which depend upon
>    simplification..
>
>  - A variety of plugin improvements including the introduction of a new
>    plugin type, *defaulting plugins*, and the ability for typechecking
>    plugins to rewrite type-families.
>
>  - An improved constructed product result analysis, allowing unboxing of
>    nested structures, and a new boxity analysis, leading to less reboxing.
>
>  - Introduction of a tag-check elision optimisation, bringing
>    significant performance improvements in strict programs.
>
>  - Generalisation of a variety of primitive types to be levity
>    polymorphic. Consequently, the `ArrayArray#` type can at long last be
>    retired, replaced by standard `Array#`.
>
>  - Introduction of the `\cases` syntax from [GHC proposal 0302]
>
>  - A complete overhaul of GHC's Windows support. This includes a
>    migration to a fully Clang-based C toolchain, a deep refactoring of
>    the linker, and many fixes in WinIO.
>
>  - Support for multiple home packages, significantly improving support
>    in IDEs and other tools for multi-package projects.
>
>  - A refactoring of GHC's error message infrastructure, allowing GHC to
>    provide diagnostic information to downstream consumers as structured
>    data, greatly easing IDE support.
>
>  - Significant compile-time improvements to runtime and memory consumption.
>
>  - On overhaul of our packaging infrastructure, allowing full
>    traceability of release artifacts and more reliable binary
>    distributions.
>
>  - Reintroduction of deep subsumption (which was previously dropped with
> the
>    *simplified subsumption* change) as a language extension.
>
>  - ... and much more. See the [release notes] for a full accounting.
>
> Note that, as 9.4.1 is the first release for which the released artifacts
> will
> all be generated by our Hadrian build system, it is possible that there
> will be
> packaging issues. If you enounter trouble while using a binary
> distribution,
> please open a [ticket]. Likewise, if you are a downstream packager, do
> consider
> migrating to [Hadrian] to run your build; the Hadrian build system can be
> built
> using `cabal-install`, `stack`, or the in-tree [bootstrap script]. We will
> be
> publishing a blog post describing the migration process to Hadrian in the
> coming weeks.
>
> We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk
> stake pool, Tweag I/O, Serokell, Equinix, SimSpace, the Haskell
> Foundation, and other anonymous contributors whose on-going financial
> and in-kind support has facilitated GHC maintenance and release
> management over the years. Finally, this release would not have been
> possible without the hundreds of open-source contributors whose work
> comprise this release.
>
> As always, do give this release a try and open a [ticket] if you see
> anything amiss.
>
> Happy testing,
>
> - Ben
>
> [downloads.haskell.org]: https://downloads.haskell.org/ghc/9.4.1-rc1/
> [GHC proposal 0302]:
> https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0302-cases.rst
> [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new
> [bootstrap script]:
> https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d11c07d35/hadrian/bootstrap/README.md
> [Hadrian]:
> https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d11c07d35/hadrian
> [release notes]:
> https://downloads.haskell.org/~ghc/9.4.1-rc1/docs/users_guide/9.4.1-notes.html
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220722/74f1bb12/attachment.html>

From ben at well-typed.com  Sat Jul 23 03:30:28 2022
From: ben at well-typed.com (Ben Gamari)
Date: Fri, 22 Jul 2022 23:30:28 -0400
Subject: [ANNOUNCE] GHC 9.4.1-rc1 is now available
In-Reply-To: <CAB-d4A4EgmQT6-GWbyi_sYWpf1cFk9Nughq7mL50qyK=XGkDqA@mail.gmail.com>
References: <87o7xhz4ax.fsf@smart-cactus.org>
 <CAB-d4A4EgmQT6-GWbyi_sYWpf1cFk9Nughq7mL50qyK=XGkDqA@mail.gmail.com>
Message-ID: <87edyczeti.fsf@smart-cactus.org>

George Colpitts <george.colpitts at gmail.com> writes:

> Hi Ben,
>
> I expected https://gitlab.haskell.org/ghc/ghc/-/issues/21506 (ghc-9.4.1-alpha1
> does not install on macos: ghc-pkg-9.4.0-20220501 cannot be opened because
> the developer cannot be verified) to be fixed in rc1 but it is not. Are my
> expectations wrong? What is the ETA for fixing it?
>
Thanks for letting us know, George. The fix that we have [1] is present
in 9.4.1-rc1. If that commit doesn't resolve the issue then there is
something that we don't understand. Does `/usr/bin/xattr` exist? Running
`xattr -rc` manually on the binary distribution allow you to run the
compiler?

Cheers,

- Ben


[1] https://gitlab.haskell.org/ghc/ghc/-/commit/641972d65b476aac11424bde6c3bcfda1c65aef5

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220722/036440c4/attachment.sig>

From george.colpitts at gmail.com  Sat Jul 23 11:20:17 2022
From: george.colpitts at gmail.com (George Colpitts)
Date: Sat, 23 Jul 2022 08:20:17 -0300
Subject: [ANNOUNCE] GHC 9.4.1-rc1 is now available
In-Reply-To: <87edyczeti.fsf@smart-cactus.org>
References: <87o7xhz4ax.fsf@smart-cactus.org>
 <CAB-d4A4EgmQT6-GWbyi_sYWpf1cFk9Nughq7mL50qyK=XGkDqA@mail.gmail.com>
 <87edyczeti.fsf@smart-cactus.org>
Message-ID: <CAB-d4A4LwLEawOyK1oDhHKb-q3B2Dj=dE-bKqTwCwna1D3PiCw@mail.gmail.com>

Hi Ben

/ust/bin/xattr exists on my machine. Running "xattr -rc ." manually does
not fix the bug as noted at the start of 21506. It was sufficient in the
past but no longer fixes this error. As noted farther down in 21506

the workaround given in #17418 </ghc/ghc/-/issues/17418> no longer works.
It did not work in 9.2.2 either. The current workaround is similar to what
Kazu explained in
https://twitter.com/kazu_yamamoto/status/1500643489985761282

sudo xattr -rc .

sudo spctl --global-disable

./configure

sudo make install

sudo spctl --global-enable

It seems there are files created during sudo make install that have an
issue as xattr -rc . was never run on them. Perhaps this is related to
using Hadrian. Is it possible that the fix that was made was never tested?

Thanks
George


On Sat, Jul 23, 2022 at 12:30 AM Ben Gamari <ben at well-typed.com> wrote:

> George Colpitts <george.colpitts at gmail.com> writes:
>
> > Hi Ben,
> >
> > I expected https://gitlab.haskell.org/ghc/ghc/-/issues/21506
> (ghc-9.4.1-alpha1
> > does not install on macos: ghc-pkg-9.4.0-20220501 cannot be opened
> because
> > the developer cannot be verified) to be fixed in rc1 but it is not. Are
> my
> > expectations wrong? What is the ETA for fixing it?
> >
> Thanks for letting us know, George. The fix that we have [1] is present
> in 9.4.1-rc1. If that commit doesn't resolve the issue then there is
> something that we don't understand. Does `/usr/bin/xattr` exist? Running
> `xattr -rc` manually on the binary distribution allow you to run the
> compiler?
>
> Cheers,
>
> - Ben
>
>
> [1]
> https://gitlab.haskell.org/ghc/ghc/-/commit/641972d65b476aac11424bde6c3bcfda1c65aef5
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220723/46befac2/attachment.html>

From ben at well-typed.com  Sat Jul 23 13:03:53 2022
From: ben at well-typed.com (Ben Gamari)
Date: Sat, 23 Jul 2022 09:03:53 -0400
Subject: [ANNOUNCE] GHC 9.4.1-rc1 is now available
In-Reply-To: <CAB-d4A4LwLEawOyK1oDhHKb-q3B2Dj=dE-bKqTwCwna1D3PiCw@mail.gmail.com>
References: <87o7xhz4ax.fsf@smart-cactus.org>
 <CAB-d4A4EgmQT6-GWbyi_sYWpf1cFk9Nughq7mL50qyK=XGkDqA@mail.gmail.com>
 <87edyczeti.fsf@smart-cactus.org>
 <CAB-d4A4LwLEawOyK1oDhHKb-q3B2Dj=dE-bKqTwCwna1D3PiCw@mail.gmail.com>
Message-ID: <87bktgyo9s.fsf@smart-cactus.org>

George Colpitts <george.colpitts at gmail.com> writes:

> Hi Ben
>
> /ust/bin/xattr exists on my machine. Running "xattr -rc ." manually does
> not fix the bug as noted at the start of 21506. It was sufficient in the
> past but no longer fixes this error. As noted farther down in 21506
>
> the workaround given in #17418 </ghc/ghc/-/issues/17418> no longer works.
> It did not work in 9.2.2 either. The current workaround is similar to what
> Kazu explained in
> https://twitter.com/kazu_yamamoto/status/1500643489985761282
>
> sudo xattr -rc .
>
> sudo spctl --global-disable
>
> ./configure
>
> sudo make install
>
> sudo spctl --global-enable
>
> It seems there are files created during sudo make install that have an
> issue as xattr -rc . was never run on them. Perhaps this is related to
> using Hadrian. Is it possible that the fix that was made was never tested?
>
I tested the change both manually and via CI on the hardware that I have
access to; in both cases installing the binary distribution resulted in
a functional compiler. However, given how the effects of SIP are
essentially undocumented, it is very hard to know what variables may be
at play here. Running spctl --status on the machine on which I tested
claims:

    > spctl --status
    objc[48908]: Class SPExecutionPolicy is implemented in both /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
    objc[48908]: Class AppWrapper is implemented in both /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
    objc[48908]: Class AppWrapperPolicyResult is implemented in both /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
    objc[48908]: Class AppWrapperPolicy is implemented in both /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
    objc[48908]: Class SPLog is implemented in both /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
    objc[48908]: Class MIS is implemented in both /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
    objc[48908]: Class SPExecutionHistoryItem is implemented in both /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
    objc[48908]: Class SPExecutionPolicyItem is implemented in both /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
    objc[48908]: Class SPDeveloperPolicy is implemented in both /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
    objc[48908]: Class GKScanResult is implemented in both /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
    assessments enabled

Which to me suggests that SIP (which, I imagine, is responsible for
#21506) is enabled. However, the lack of comprehensive documentation
here makes it very hard to say with certainty what might differ between
your machine and mine. Without more information I don't know how to
proceed here. Perhaps someone (Moritz or Simon, perhaps) with more
familiarity with macOS has some insight?

Thanks for your help in testing, George!

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220723/85bc1acc/attachment.sig>

From george.colpitts at gmail.com  Sat Jul 23 14:51:52 2022
From: george.colpitts at gmail.com (George Colpitts)
Date: Sat, 23 Jul 2022 11:51:52 -0300
Subject: [ANNOUNCE] GHC 9.4.1-rc1 is now available
In-Reply-To: <87bktgyo9s.fsf@smart-cactus.org>
References: <87o7xhz4ax.fsf@smart-cactus.org>
 <CAB-d4A4EgmQT6-GWbyi_sYWpf1cFk9Nughq7mL50qyK=XGkDqA@mail.gmail.com>
 <87edyczeti.fsf@smart-cactus.org>
 <CAB-d4A4LwLEawOyK1oDhHKb-q3B2Dj=dE-bKqTwCwna1D3PiCw@mail.gmail.com>
 <87bktgyo9s.fsf@smart-cactus.org>
Message-ID: <CAB-d4A54B-z9vJT0PFWr7y2GYVWSdyKxLRCWanZVc12HQj9Y+g@mail.gmail.com>

+Kazu Yamamoto <kazu at iij.ad.jp>

Hi Ben

My 2 machines also have:

$ spctl --status
assessments enabled

I've duplicated the issue on both of my machines. It would be good to know
if anybody else is seeing it. Kazu, I know you have seen this in the past.
Do you get the same error installing rc1?
When I run sudo make install I get a popup that says:

*“libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib” cannot be opened because
the developer cannot be verified.*

When I cancel out of that I get another popup with the same error. When I
hit cancel on that the script ends with the output:

# We finally replace the original file.
mv
'/usr/local/lib/ghc-9.4.0.20220721/lib/package.conf.d/process-1.6.14.0.conf.copy.copy'
'/usr/local/lib/ghc-9.4.0.20220721/lib/package.conf.d/process-1.6.14.0.conf'
'/usr/local/lib/ghc-9.4.0.20220721/bin/ghc-pkg' --global-package-db
"/usr/local/lib/ghc-9.4.0.20220721/lib/package.conf.d" recache
dyld[32239]: Library not loaded:
'@rpath/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib'
  Referenced from:
'/usr/local/lib/ghc-9.4.0.20220721/bin/ghc-pkg-9.4.0.20220721'
  Reason: tried:*
'/usr/local/lib/ghc-9.4.0.20220721/bin/../lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib'
(code signature in <CA45AED4-4028-3B3F-9AA6-91EDE1078A7E>
'/usr/local/lib/ghc-9.4.0.20220721/lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib'
not valid for use in process: library load disallowed by system policy),
'$ORIGIN/../../../lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib'
(no such file),
'/usr/local/lib/ghc-9.4.0.20220721/bin/../lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib'
(code signature in <CA45AED4-4028-3B3F-9AA6-91EDE1078A7E>
'/usr/local/lib/ghc-9.4.0.20220721/lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib'
not valid for use in process: library load disallowed by system policy),*
'$ORIGIN/../../../lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib'
(no such file),
'/usr/local/lib/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib' (no such
file), '/usr/lib/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib' (no such
file)
make: *** [update_package_db] Abort trap: 6
gcolpitts at macMini-2018 ghc-9.4.0.20220721-x86_64-apple-darwin %

 Hope this helps.

Speculations:

/usr/local/lib/ghc-9.4.0.20220721/bin/../lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib
is created after xattr -rc . was run so it doesn't have the necessary
attributes. Is it possible that ghc developers and/or the test machines
have this file on another of the paths in the error message and that is why
it works for them?

I hope I didn't offend you by asking if the fix had been tested; I assume
it had been but I thought it was important to rule that out.

More than happy to test. I really appreciate all the work you and others
have put into GHC !

Cheers
George

On Sat, Jul 23, 2022 at 10:03 AM Ben Gamari <ben at well-typed.com> wrote:

> George Colpitts <george.colpitts at gmail.com> writes:
>
> > Hi Ben
> >
> > /ust/bin/xattr exists on my machine. Running "xattr -rc ." manually does
> > not fix the bug as noted at the start of 21506. It was sufficient in the
> > past but no longer fixes this error. As noted farther down in 21506
> >
> > the workaround given in #17418 </ghc/ghc/-/issues/17418> no longer works.
> > It did not work in 9.2.2 either. The current workaround is similar to
> what
> > Kazu explained in
> > https://twitter.com/kazu_yamamoto/status/1500643489985761282
> >
> > sudo xattr -rc .
> >
> > sudo spctl --global-disable
> >
> > ./configure
> >
> > sudo make install
> >
> > sudo spctl --global-enable
> >
> > It seems there are files created during sudo make install that have an
> > issue as xattr -rc . was never run on them. Perhaps this is related to
> > using Hadrian. Is it possible that the fix that was made was never
> tested?
> >
> I tested the change both manually and via CI on the hardware that I have
> access to; in both cases installing the binary distribution resulted in
> a functional compiler. However, given how the effects of SIP are
> essentially undocumented, it is very hard to know what variables may be
> at play here. Running spctl --status on the machine on which I tested
> claims:
>
>     > spctl --status
>     objc[48908]: Class SPExecutionPolicy is implemented in both
> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>     objc[48908]: Class AppWrapper is implemented in both
> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>     objc[48908]: Class AppWrapperPolicyResult is implemented in both
> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>     objc[48908]: Class AppWrapperPolicy is implemented in both
> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>     objc[48908]: Class SPLog is implemented in both
> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>     objc[48908]: Class MIS is implemented in both
> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>     objc[48908]: Class SPExecutionHistoryItem is implemented in both
> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>     objc[48908]: Class SPExecutionPolicyItem is implemented in both
> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>     objc[48908]: Class SPDeveloperPolicy is implemented in both
> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>     objc[48908]: Class GKScanResult is implemented in both
> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>     assessments enabled
>
> Which to me suggests that SIP (which, I imagine, is responsible for
> #21506) is enabled. However, the lack of comprehensive documentation
> here makes it very hard to say with certainty what might differ between
> your machine and mine. Without more information I don't know how to
> proceed here. Perhaps someone (Moritz or Simon, perhaps) with more
> familiarity with macOS has some insight?
>
> Thanks for your help in testing, George!
>
> Cheers,
>
> - Ben
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220723/7dc19079/attachment.html>

From george.colpitts at gmail.com  Sun Jul 24 11:43:54 2022
From: george.colpitts at gmail.com (George Colpitts)
Date: Sun, 24 Jul 2022 08:43:54 -0300
Subject: [ANNOUNCE] GHC 9.4.1-rc1 is now available
In-Reply-To: <CAB-d4A54B-z9vJT0PFWr7y2GYVWSdyKxLRCWanZVc12HQj9Y+g@mail.gmail.com>
References: <87o7xhz4ax.fsf@smart-cactus.org>
 <CAB-d4A4EgmQT6-GWbyi_sYWpf1cFk9Nughq7mL50qyK=XGkDqA@mail.gmail.com>
 <87edyczeti.fsf@smart-cactus.org>
 <CAB-d4A4LwLEawOyK1oDhHKb-q3B2Dj=dE-bKqTwCwna1D3PiCw@mail.gmail.com>
 <87bktgyo9s.fsf@smart-cactus.org>
 <CAB-d4A54B-z9vJT0PFWr7y2GYVWSdyKxLRCWanZVc12HQj9Y+g@mail.gmail.com>
Message-ID: <CAB-d4A7HiyehD3P8-QpvWXECH7U=L1W_0nM4j+M3aACdyhwNWQ@mail.gmail.com>

+ address that hopefully doesn't bounce for Moritz

On Sat, Jul 23, 2022 at 11:51 AM George Colpitts <george.colpitts at gmail.com>
wrote:

> +Kazu Yamamoto <kazu at iij.ad.jp>
>
> Hi Ben
>
> My 2 machines also have:
>
> $ spctl --status
> assessments enabled
>
> I've duplicated the issue on both of my machines. It would be good to know
> if anybody else is seeing it. Kazu, I know you have seen this in the past.
> Do you get the same error installing rc1?
> When I run sudo make install I get a popup that says:
>
> *“libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib” cannot be opened because
> the developer cannot be verified.*
>
> When I cancel out of that I get another popup with the same error. When I
> hit cancel on that the script ends with the output:
>
> # We finally replace the original file.
> mv
> '/usr/local/lib/ghc-9.4.0.20220721/lib/package.conf.d/process-1.6.14.0.conf.copy.copy'
> '/usr/local/lib/ghc-9.4.0.20220721/lib/package.conf.d/process-1.6.14.0.conf'
> '/usr/local/lib/ghc-9.4.0.20220721/bin/ghc-pkg' --global-package-db
> "/usr/local/lib/ghc-9.4.0.20220721/lib/package.conf.d" recache
> dyld[32239]: Library not loaded:
> '@rpath/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib'
>   Referenced from:
> '/usr/local/lib/ghc-9.4.0.20220721/bin/ghc-pkg-9.4.0.20220721'
>   Reason: tried:*
> '/usr/local/lib/ghc-9.4.0.20220721/bin/../lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib'
> (code signature in <CA45AED4-4028-3B3F-9AA6-91EDE1078A7E>
> '/usr/local/lib/ghc-9.4.0.20220721/lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib'
> not valid for use in process: library load disallowed by system policy),
> '$ORIGIN/../../../lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib'
> (no such file),
> '/usr/local/lib/ghc-9.4.0.20220721/bin/../lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib'
> (code signature in <CA45AED4-4028-3B3F-9AA6-91EDE1078A7E>
> '/usr/local/lib/ghc-9.4.0.20220721/lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib'
> not valid for use in process: library load disallowed by system policy),*
> '$ORIGIN/../../../lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib'
> (no such file),
> '/usr/local/lib/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib' (no such
> file), '/usr/lib/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib' (no such
> file)
> make: *** [update_package_db] Abort trap: 6
> gcolpitts at macMini-2018 ghc-9.4.0.20220721-x86_64-apple-darwin %
>
>  Hope this helps.
>
> Speculations:
>
> /usr/local/lib/ghc-9.4.0.20220721/bin/../lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib
> is created after xattr -rc . was run so it doesn't have the necessary
> attributes. Is it possible that ghc developers and/or the test machines
> have this file on another of the paths in the error message and that is why
> it works for them?
>
> I hope I didn't offend you by asking if the fix had been tested; I assume
> it had been but I thought it was important to rule that out.
>
> More than happy to test. I really appreciate all the work you and others
> have put into GHC !
>
> Cheers
> George
>
> On Sat, Jul 23, 2022 at 10:03 AM Ben Gamari <ben at well-typed.com> wrote:
>
>> George Colpitts <george.colpitts at gmail.com> writes:
>>
>> > Hi Ben
>> >
>> > /ust/bin/xattr exists on my machine. Running "xattr -rc ." manually does
>> > not fix the bug as noted at the start of 21506. It was sufficient in the
>> > past but no longer fixes this error. As noted farther down in 21506
>> >
>> > the workaround given in #17418 </ghc/ghc/-/issues/17418> no longer
>> works.
>> > It did not work in 9.2.2 either. The current workaround is similar to
>> what
>> > Kazu explained in
>> > https://twitter.com/kazu_yamamoto/status/1500643489985761282
>> >
>> > sudo xattr -rc .
>> >
>> > sudo spctl --global-disable
>> >
>> > ./configure
>> >
>> > sudo make install
>> >
>> > sudo spctl --global-enable
>> >
>> > It seems there are files created during sudo make install that have an
>> > issue as xattr -rc . was never run on them. Perhaps this is related to
>> > using Hadrian. Is it possible that the fix that was made was never
>> tested?
>> >
>> I tested the change both manually and via CI on the hardware that I have
>> access to; in both cases installing the binary distribution resulted in
>> a functional compiler. However, given how the effects of SIP are
>> essentially undocumented, it is very hard to know what variables may be
>> at play here. Running spctl --status on the machine on which I tested
>> claims:
>>
>>     > spctl --status
>>     objc[48908]: Class SPExecutionPolicy is implemented in both
>> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
>> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>>     objc[48908]: Class AppWrapper is implemented in both
>> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
>> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>>     objc[48908]: Class AppWrapperPolicyResult is implemented in both
>> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
>> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>>     objc[48908]: Class AppWrapperPolicy is implemented in both
>> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
>> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>>     objc[48908]: Class SPLog is implemented in both
>> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
>> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>>     objc[48908]: Class MIS is implemented in both
>> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
>> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>>     objc[48908]: Class SPExecutionHistoryItem is implemented in both
>> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
>> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>>     objc[48908]: Class SPExecutionPolicyItem is implemented in both
>> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
>> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>>     objc[48908]: Class SPDeveloperPolicy is implemented in both
>> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
>> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>>     objc[48908]: Class GKScanResult is implemented in both
>> /System/Library/PrivateFrameworks/SystemPolicy.framework/Versions/A/SystemPolicy
>> and /usr/sbin/spctl. One of the two will be used. Which one is undefined.
>>     assessments enabled
>>
>> Which to me suggests that SIP (which, I imagine, is responsible for
>> #21506) is enabled. However, the lack of comprehensive documentation
>> here makes it very hard to say with certainty what might differ between
>> your machine and mine. Without more information I don't know how to
>> proceed here. Perhaps someone (Moritz or Simon, perhaps) with more
>> familiarity with macOS has some insight?
>>
>> Thanks for your help in testing, George!
>>
>> Cheers,
>>
>> - Ben
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220724/4921b6f1/attachment.html>

From ben at well-typed.com  Sun Jul 24 14:33:29 2022
From: ben at well-typed.com (Ben Gamari)
Date: Sun, 24 Jul 2022 10:33:29 -0400
Subject: [ANNOUNCE] GHC 9.4.1-rc1 is now available
In-Reply-To: <CAB-d4A54B-z9vJT0PFWr7y2GYVWSdyKxLRCWanZVc12HQj9Y+g@mail.gmail.com>
References: <87o7xhz4ax.fsf@smart-cactus.org>
 <CAB-d4A4EgmQT6-GWbyi_sYWpf1cFk9Nughq7mL50qyK=XGkDqA@mail.gmail.com>
 <87edyczeti.fsf@smart-cactus.org>
 <CAB-d4A4LwLEawOyK1oDhHKb-q3B2Dj=dE-bKqTwCwna1D3PiCw@mail.gmail.com>
 <87bktgyo9s.fsf@smart-cactus.org>
 <CAB-d4A54B-z9vJT0PFWr7y2GYVWSdyKxLRCWanZVc12HQj9Y+g@mail.gmail.com>
Message-ID: <878roizilc.fsf@smart-cactus.org>

George Colpitts <george.colpitts at gmail.com> writes:

> +Kazu Yamamoto <kazu at iij.ad.jp>
>
> Hi Ben
>
> My 2 machines also have:
>
> $ spctl --status
> assessments enabled
>
Hmm, interesting. Then I am truly perplexed.

> Speculations:
>
> /usr/local/lib/ghc-9.4.0.20220721/bin/../lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib
> is created after xattr -rc . was run so it doesn't have the necessary
> attributes. Is it possible that ghc developers and/or the test machines
> have this file on another of the paths in the error message and that is why
> it works for them?
>
I'm not sure where this would be but at this point anything is possible. What happens
if you try to install to do something like (in the extracted binary distribution),

    $ ./configure --prefix=`pwd`/tmp
    $ make install            # this will fail
    $ xattr -rc .
    $ make install            # perhaps this will finish successfully?
    # tmp/bin/ghc --version   # GHC should be usable

> I hope I didn't offend you by asking if the fix had been tested; I assume
> it had been but I thought it was important to rule that out.
>
Not to worry; it's a very reasonable question to ask given the circumstances.

> More than happy to test. I really appreciate all the work you and others
> have put into GHC !
>
Ultimately I think we may just need to bite the bullet and start
properly notarising/codesigning releases (resolving #17418). At this point we have
spent more time trying to avoid the notarisation requirement than
it would likely take to satisfy it. Unfortunately, this will require
that I find an Apple device somewhere which may take a few weeks.

I'm afraid I am on holiday next week but I would quite grateful if we
could arrange for a chat after I return such that we can debug this in
realtime.

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220724/43d1997d/attachment.sig>

From george.colpitts at gmail.com  Sun Jul 24 16:58:41 2022
From: george.colpitts at gmail.com (George Colpitts)
Date: Sun, 24 Jul 2022 13:58:41 -0300
Subject: success on a slightly modified version of your suggestion for 21506
In-Reply-To: <878roizilc.fsf@smart-cactus.org>
References: <87o7xhz4ax.fsf@smart-cactus.org>
 <CAB-d4A4EgmQT6-GWbyi_sYWpf1cFk9Nughq7mL50qyK=XGkDqA@mail.gmail.com>
 <87edyczeti.fsf@smart-cactus.org>
 <CAB-d4A4LwLEawOyK1oDhHKb-q3B2Dj=dE-bKqTwCwna1D3PiCw@mail.gmail.com>
 <87bktgyo9s.fsf@smart-cactus.org>
 <CAB-d4A54B-z9vJT0PFWr7y2GYVWSdyKxLRCWanZVc12HQj9Y+g@mail.gmail.com>
 <878roizilc.fsf@smart-cactus.org>
Message-ID: <CAB-d4A548yUaqFERYX8O_VuCHmOoPVOY+73ksLz7kN6PZKcsZg@mail.gmail.com>

Hi Ben

Thanks for the quick responses particularly on the weekend before your
vacation.

you wrote:

What happens
if you try to install to do something like (in the extracted binary
distribution),


    $ ./configure --prefix=`pwd`/tmp
    $ make install            # this will fail
    $ xattr -rc .
    $ make install            # perhaps this will finish successfully?
    # tmp/bin/ghc --version   # GHC should be usable


success on both my machines using a slightly modified version of your
suggestion above for 21506:

 ./configure --prefix=`pwd`/tmp    # specifying ./tmp seems to be critical

xattr -rc .                                        # seems to be necessary
also

sudo make install                           # seems to works , output ends
with "recache"

./tmp/bin/ghc --version                  # success , although admittedly
the smallest smoke test possible

You wrote:

>
Ultimately I think we may just need to bite the bullet and start
properly notarising/codesigning releases (resolving #17418). At this point
we have
spent more time trying to avoid the notarisation requirement than
it would likely take to satisfy it. Unfortunately, this will require
that I find an Apple device somewhere which may take a few weeks.


I'm afraid I am on holiday next week but I would quite grateful if we
could arrange for a chat after I return such that we can debug this in
realtime.


Sure, we can chat when you return from your vacation.

Not sure if it is worth trying to fix the release on the basis of what I
write above. My opinion is: it is if we can get reports of at least one
other person having this issue. I am fine with not doing this for 9.4.1

I agree that we should raise the priority of "notarising/codesigning
releases (resolving #17418)". My opinion is that it is not worth delaying
9.4.1 for this.

Have a great vacation.

Cheers
George






On Sun, Jul 24, 2022 at 11:33 AM Ben Gamari <ben at well-typed.com> wrote:

> George Colpitts <george.colpitts at gmail.com> writes:
>
> > +Kazu Yamamoto <kazu at iij.ad.jp>
> >
> > Hi Ben
> >
> > My 2 machines also have:
> >
> > $ spctl --status
> > assessments enabled
> >
> Hmm, interesting. Then I am truly perplexed.
>
> > Speculations:
> >
> >
> /usr/local/lib/ghc-9.4.0.20220721/bin/../lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib
> > is created after xattr -rc . was run so it doesn't have the necessary
> > attributes. Is it possible that ghc developers and/or the test machines
> > have this file on another of the paths in the error message and that is
> why
> > it works for them?
> >
> I'm not sure where this would be but at this point anything is possible.
> What happens
> if you try to install to do something like (in the extracted binary
> distribution),
>
>     $ ./configure --prefix=`pwd`/tmp
>     $ make install            # this will fail
>     $ xattr -rc .
>     $ make install            # perhaps this will finish successfully?
>     # tmp/bin/ghc --version   # GHC should be usable
>
> > I hope I didn't offend you by asking if the fix had been tested; I assume
> > it had been but I thought it was important to rule that out.
> >
> Not to worry; it's a very reasonable question to ask given the
> circumstances.
>
> > More than happy to test. I really appreciate all the work you and others
> > have put into GHC !
> >
> Ultimately I think we may just need to bite the bullet and start
> properly notarising/codesigning releases (resolving #17418). At this point
> we have
> spent more time trying to avoid the notarisation requirement than
> it would likely take to satisfy it. Unfortunately, this will require
> that I find an Apple device somewhere which may take a few weeks.
>
> I'm afraid I am on holiday next week but I would quite grateful if we
> could arrange for a chat after I return such that we can debug this in
> realtime.
>
> Cheers,
>
> - Ben
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220724/8f83e567/attachment.html>

From kazu at iij.ad.jp  Mon Jul 25 04:23:17 2022
From: kazu at iij.ad.jp (Kazu Yamamoto (=?iso-2022-jp?B?GyRCOzNLXE9CSScbKEI=?=))
Date: Mon, 25 Jul 2022 13:23:17 +0900 (JST)
Subject: [ANNOUNCE] GHC 9.4.1-rc1 is now available
In-Reply-To: <CAB-d4A54B-z9vJT0PFWr7y2GYVWSdyKxLRCWanZVc12HQj9Y+g@mail.gmail.com>
References: <CAB-d4A4LwLEawOyK1oDhHKb-q3B2Dj=dE-bKqTwCwna1D3PiCw@mail.gmail.com>
 <87bktgyo9s.fsf@smart-cactus.org>
 <CAB-d4A54B-z9vJT0PFWr7y2GYVWSdyKxLRCWanZVc12HQj9Y+g@mail.gmail.com>
Message-ID: <20220725.132317.759628945463268605.kazu@iij.ad.jp>

Hi George,

> I've duplicated the issue on both of my machines. It would be good to know
> if anybody else is seeing it. Kazu, I know you have seen this in the past.
> Do you get the same error installing rc1?
> When I run sudo make install I get a popup that says:

I had no problem on 9.4.1-rc1. 
"xattr -rc ." and "make install" worked perfectly.

macOS Monterey v12.4
Xcode 13.4.1

--Kazu



From gergo at erdi.hu  Mon Jul 25 09:18:14 2022
From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=)
Date: Mon, 25 Jul 2022 17:18:14 +0800 (+08)
Subject: Wildcards in type synonyms
In-Reply-To: <CAJKmMz9qcSpmVyDDyK5UABM-hx=jeiyxq9dN6m+3mooLQkoOcw@mail.gmail.com>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
 <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>
 <CAO7EwMrJGL9YVKrY+h_2uwSfA_TZ_0VGJSExtkbwTPYGCcAAQA@mail.gmail.com>
 <CAJKmMz9qcSpmVyDDyK5UABM-hx=jeiyxq9dN6m+3mooLQkoOcw@mail.gmail.com>
Message-ID: <alpine.DEB.2.22.394.2207251717290.4398@galaxy>

On Fri, 22 Jul 2022, Simon Peyton Jones wrote:

>       So it seems that instead of shoehorning it into the existing type
>       synonyms, a better bet would be to branch off to a separate path quite
>       early (maybe as soon as during renaming), expand them during
>       typechecking, and *not* touch how/when existing "normal" type synonyms
>       are resolved.
> 
> 
> That sounds plausible, yes.

Do we have an existing way of substituting types over type variables, *in 
HsType instead of Core Type*?

From simon.peytonjones at gmail.com  Mon Jul 25 09:25:34 2022
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Mon, 25 Jul 2022 10:25:34 +0100
Subject: Wildcards in type synonyms
In-Reply-To: <alpine.DEB.2.22.394.2207251717290.4398@galaxy>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
 <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>
 <CAO7EwMrJGL9YVKrY+h_2uwSfA_TZ_0VGJSExtkbwTPYGCcAAQA@mail.gmail.com>
 <CAJKmMz9qcSpmVyDDyK5UABM-hx=jeiyxq9dN6m+3mooLQkoOcw@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251717290.4398@galaxy>
Message-ID: <CAJKmMz_DgrXavKWZmdOss8Y18Lq-ha9Md-HNTDfJn09iWgm7yA@mail.gmail.com>

>
> Do we have an existing way of substituting types over type variables, *in
> HsType instead of Core Type*?
>

I'm afraid not. Currently HsType is not processed much -- just renamed and
typechecked into a Type.

The nearest we have is Note [Handling overloaded and rebindable
constructs], in the renamer.  That doesn't do what you want, but the
HsExpansion idea is close

S

On Mon, 25 Jul 2022 at 10:18, ÉRDI Gergő <gergo at erdi.hu> wrote:

> On Fri, 22 Jul 2022, Simon Peyton Jones wrote:
>
> >       So it seems that instead of shoehorning it into the existing type
> >       synonyms, a better bet would be to branch off to a separate path
> quite
> >       early (maybe as soon as during renaming), expand them during
> >       typechecking, and *not* touch how/when existing "normal" type
> synonyms
> >       are resolved.
> >
> >
> > That sounds plausible, yes.
>
> Do we have an existing way of substituting types over type variables, *in
> HsType instead of Core Type*?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220725/c9f0d034/attachment.html>

From gergo at erdi.hu  Mon Jul 25 10:04:38 2022
From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=)
Date: Mon, 25 Jul 2022 18:04:38 +0800 (+08)
Subject: Wildcards in type synonyms
In-Reply-To: <CAJKmMz_DgrXavKWZmdOss8Y18Lq-ha9Md-HNTDfJn09iWgm7yA@mail.gmail.com>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
 <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>
 <CAO7EwMrJGL9YVKrY+h_2uwSfA_TZ_0VGJSExtkbwTPYGCcAAQA@mail.gmail.com>
 <CAJKmMz9qcSpmVyDDyK5UABM-hx=jeiyxq9dN6m+3mooLQkoOcw@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251717290.4398@galaxy>
 <CAJKmMz_DgrXavKWZmdOss8Y18Lq-ha9Md-HNTDfJn09iWgm7yA@mail.gmail.com>
Message-ID: <alpine.DEB.2.22.394.2207251802380.4398@galaxy>

On Mon, 25 Jul 2022, Simon Peyton Jones wrote:

>       Do we have an existing way of substituting types over type variables, *in
>       HsType instead of Core Type*?
> 
> 
> I'm afraid not. Currently HsType is not processed much -- just renamed and typechecked
> into a Type.

I wonder if, instead, I could expand the rhs, typecheck it "abstractly" 
(i.e. in the context of the synonym's binders), and THEN do the 
substitution. If I typecheck the rhs for every occurrence, I should get 
fresh metavars for each wildcard, which is pretty much what I want. I just
have to make sure I don't zonk before the substitution.

Does this make sense?

From lists at richarde.dev  Mon Jul 25 12:57:49 2022
From: lists at richarde.dev (Richard Eisenberg)
Date: Mon, 25 Jul 2022 12:57:49 +0000
Subject: Wildcards in type synonyms
In-Reply-To: <alpine.DEB.2.22.394.2207251802380.4398@galaxy>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
 <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>
 <CAO7EwMrJGL9YVKrY+h_2uwSfA_TZ_0VGJSExtkbwTPYGCcAAQA@mail.gmail.com>
 <CAJKmMz9qcSpmVyDDyK5UABM-hx=jeiyxq9dN6m+3mooLQkoOcw@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251717290.4398@galaxy>
 <CAJKmMz_DgrXavKWZmdOss8Y18Lq-ha9Md-HNTDfJn09iWgm7yA@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251802380.4398@galaxy>
Message-ID: <010f0182356f669c-baee0be8-066e-46f5-a303-b857140b6dd3-000000@us-east-2.amazonses.com>



> On Jul 25, 2022, at 6:04 AM, ÉRDI Gergő <gergo at erdi.hu> wrote:
> 
> On Mon, 25 Jul 2022, Simon Peyton Jones wrote:
> 
>>      Do we have an existing way of substituting types over type variables, *in
>>      HsType instead of Core Type*?
>> I'm afraid not. Currently HsType is not processed much -- just renamed and typechecked
>> into a Type.
> 
> I wonder if, instead, I could expand the rhs, typecheck it "abstractly" (i.e. in the context of the synonym's binders), and THEN do the substitution.

Why type-check the RHS at all? Presumably, to give nice error messages. But it looks like this aspect of the plan is inessential. To be clear, I *do* think you should type-check the RHS, but I'm also checking my understanding here. If type-checking the RHS is indeed inessential, then the result of that type-checking (a desugared `Type`) should be discarded.

> If I typecheck the rhs for every occurrence, I should get fresh metavars for each wildcard, which is pretty much what I want. I just
> have to make sure I don't zonk before the substitution.
> 

I see this substitution as happening before any type-checking, so zonking shouldn't be an issue. That is, I would expect a 

substHsTy :: UniqFM Name (HsType GhcRn) -> HsType GhcRn -> HsType GhcRn

to do the work, entirely before type-checking.

(Presumably, you don't want the macro-like behavior to extend to fixity resolution. That is, if we have

type macro T a = a + b

and then write `f :: T Int * Double`, we want `f :: (Int + b) * Double`, not `f :: Int + (b * Double)`. If you indeed want the latter (strange days!), then you'd need to be careful to do the substitution before fixity resolution, just after renaming.)

Richard

> Does this make sense?
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From gergo at erdi.hu  Mon Jul 25 13:21:39 2022
From: gergo at erdi.hu (=?UTF-8?B?R2VyZ8WRIMOJcmRp?=)
Date: Mon, 25 Jul 2022 21:21:39 +0800
Subject: Wildcards in type synonyms
In-Reply-To: <010f0182356f669c-baee0be8-066e-46f5-a303-b857140b6dd3-000000@us-east-2.amazonses.com>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
 <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>
 <CAO7EwMrJGL9YVKrY+h_2uwSfA_TZ_0VGJSExtkbwTPYGCcAAQA@mail.gmail.com>
 <CAJKmMz9qcSpmVyDDyK5UABM-hx=jeiyxq9dN6m+3mooLQkoOcw@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251717290.4398@galaxy>
 <CAJKmMz_DgrXavKWZmdOss8Y18Lq-ha9Md-HNTDfJn09iWgm7yA@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251802380.4398@galaxy>
 <010f0182356f669c-baee0be8-066e-46f5-a303-b857140b6dd3-000000@us-east-2.amazonses.com>
Message-ID: <CAO7EwMo-TmGukS7SKriCqTivL1XfGkEJWzshnv3u1o-0tGXLag@mail.gmail.com>

So my idea from my last email is to avoid the need for `substHsTy`:
When `tc_hs_type` encounters a macro occurrence,  I would `tcLHsType`
the rhs then and there (thereby getting fresh metas for each
wildcard), use `substTy` to instantiate with the given type arguments.
Then `tc_hs_type` returns that.

Note that this is NOT about typechecking the rhs *for the definition*,
but rather, using `tcLHsType` as the function that creates fresh metas
for each wildcard.

On Mon, Jul 25, 2022 at 8:58 PM Richard Eisenberg <lists at richarde.dev> wrote:
>
>
>
> > On Jul 25, 2022, at 6:04 AM, ÉRDI Gergő <gergo at erdi.hu> wrote:
> >
> > On Mon, 25 Jul 2022, Simon Peyton Jones wrote:
> >
> >>      Do we have an existing way of substituting types over type variables, *in
> >>      HsType instead of Core Type*?
> >> I'm afraid not. Currently HsType is not processed much -- just renamed and typechecked
> >> into a Type.
> >
> > I wonder if, instead, I could expand the rhs, typecheck it "abstractly" (i.e. in the context of the synonym's binders), and THEN do the substitution.
>
> Why type-check the RHS at all? Presumably, to give nice error messages. But it looks like this aspect of the plan is inessential. To be clear, I *do* think you should type-check the RHS, but I'm also checking my understanding here. If type-checking the RHS is indeed inessential, then the result of that type-checking (a desugared `Type`) should be discarded.
>
> > If I typecheck the rhs for every occurrence, I should get fresh metavars for each wildcard, which is pretty much what I want. I just
> > have to make sure I don't zonk before the substitution.
> >
>
> I see this substitution as happening before any type-checking, so zonking shouldn't be an issue. That is, I would expect a
>
> substHsTy :: UniqFM Name (HsType GhcRn) -> HsType GhcRn -> HsType GhcRn
>
> to do the work, entirely before type-checking.
>
> (Presumably, you don't want the macro-like behavior to extend to fixity resolution. That is, if we have
>
> type macro T a = a + b
>
> and then write `f :: T Int * Double`, we want `f :: (Int + b) * Double`, not `f :: Int + (b * Double)`. If you indeed want the latter (strange days!), then you'd need to be careful to do the substitution before fixity resolution, just after renaming.)
>
> Richard
>
> > Does this make sense?
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>

From gergo at erdi.hu  Tue Jul 26 03:14:27 2022
From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=)
Date: Tue, 26 Jul 2022 11:14:27 +0800 (+08)
Subject: Wildcards in type synonyms
In-Reply-To: <010f0182356f669c-baee0be8-066e-46f5-a303-b857140b6dd3-000000@us-east-2.amazonses.com>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
 <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>
 <CAO7EwMrJGL9YVKrY+h_2uwSfA_TZ_0VGJSExtkbwTPYGCcAAQA@mail.gmail.com>
 <CAJKmMz9qcSpmVyDDyK5UABM-hx=jeiyxq9dN6m+3mooLQkoOcw@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251717290.4398@galaxy>
 <CAJKmMz_DgrXavKWZmdOss8Y18Lq-ha9Md-HNTDfJn09iWgm7yA@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251802380.4398@galaxy>
 <010f0182356f669c-baee0be8-066e-46f5-a303-b857140b6dd3-000000@us-east-2.amazonses.com>
Message-ID: <alpine.DEB.2.22.394.2207261103160.4398@galaxy>

On Mon, 25 Jul 2022, Richard Eisenberg wrote:

> I see this substitution as happening before any type-checking, so zonking shouldn't be an issue. That is, I would expect a
>
> substHsTy :: UniqFM Name (HsType GhcRn) -> HsType GhcRn -> HsType GhcRn
>
> to do the work, entirely before type-checking.

If I were to do this fully during renaming, how do I even find out that a 
`Name` refers to a type macro? During typechecking I can look up the `TyThing`,
see if it's a `MacroTyCon`, and get the rhs from that. But I don't think we store
anything about already-renamed declarations during renaming the same way 
we store info about already-typechecked things during typechecking in 
`tcg_type_env`.

It sounds awfully special-case to store these type macros in a new field 
of `TcGblEnv` instead of as just another kind of `TyCon`.

From simon.peytonjones at gmail.com  Tue Jul 26 08:03:48 2022
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Tue, 26 Jul 2022 09:03:48 +0100
Subject: Wildcards in type synonyms
In-Reply-To: <alpine.DEB.2.22.394.2207261103160.4398@galaxy>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
 <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>
 <CAO7EwMrJGL9YVKrY+h_2uwSfA_TZ_0VGJSExtkbwTPYGCcAAQA@mail.gmail.com>
 <CAJKmMz9qcSpmVyDDyK5UABM-hx=jeiyxq9dN6m+3mooLQkoOcw@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251717290.4398@galaxy>
 <CAJKmMz_DgrXavKWZmdOss8Y18Lq-ha9Md-HNTDfJn09iWgm7yA@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251802380.4398@galaxy>
 <010f0182356f669c-baee0be8-066e-46f5-a303-b857140b6dd3-000000@us-east-2.amazonses.com>
 <alpine.DEB.2.22.394.2207261103160.4398@galaxy>
Message-ID: <CAJKmMz_SPdjki_fHiBA=iA8KtMpzoYE7irXMetcwYiZpUbpVQQ@mail.gmail.com>

>
> It sounds awfully special-case to store these type macros in a new field
> of `TcGblEnv` instead of as just another kind of `TyCon`.
>

But all this is just in your personal fork of GHC, so you can be as
special-case as you like.  I'd just plough ahead and do the easiest thing.
If you make a proposal, and persuade the committee, we can look again.
Until then it's probably not worth us spending a lot of time making the
implementation beautiful.

Simon

On Tue, 26 Jul 2022 at 04:14, ÉRDI Gergő <gergo at erdi.hu> wrote:

> On Mon, 25 Jul 2022, Richard Eisenberg wrote:
>
> > I see this substitution as happening before any type-checking, so
> zonking shouldn't be an issue. That is, I would expect a
> >
> > substHsTy :: UniqFM Name (HsType GhcRn) -> HsType GhcRn -> HsType GhcRn
> >
> > to do the work, entirely before type-checking.
>
> If I were to do this fully during renaming, how do I even find out that a
> `Name` refers to a type macro? During typechecking I can look up the
> `TyThing`,
> see if it's a `MacroTyCon`, and get the rhs from that. But I don't think
> we store
> anything about already-renamed declarations during renaming the same way
> we store info about already-typechecked things during typechecking in
> `tcg_type_env`.
>
> It sounds awfully special-case to store these type macros in a new field
> of `TcGblEnv` instead of as just another kind of `TyCon`.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220726/c62334ce/attachment.html>

From lists at richarde.dev  Tue Jul 26 19:35:27 2022
From: lists at richarde.dev (Richard Eisenberg)
Date: Tue, 26 Jul 2022 19:35:27 +0000
Subject: Wildcards in type synonyms
In-Reply-To: <CAO7EwMo-TmGukS7SKriCqTivL1XfGkEJWzshnv3u1o-0tGXLag@mail.gmail.com>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
 <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>
 <CAO7EwMrJGL9YVKrY+h_2uwSfA_TZ_0VGJSExtkbwTPYGCcAAQA@mail.gmail.com>
 <CAJKmMz9qcSpmVyDDyK5UABM-hx=jeiyxq9dN6m+3mooLQkoOcw@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251717290.4398@galaxy>
 <CAJKmMz_DgrXavKWZmdOss8Y18Lq-ha9Md-HNTDfJn09iWgm7yA@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251802380.4398@galaxy>
 <010f0182356f669c-baee0be8-066e-46f5-a303-b857140b6dd3-000000@us-east-2.amazonses.com>
 <CAO7EwMo-TmGukS7SKriCqTivL1XfGkEJWzshnv3u1o-0tGXLag@mail.gmail.com>
Message-ID: <010f01823c01cc6e-a3e2b46a-4ee3-43b5-aa7a-257e9e781e55-000000@us-east-2.amazonses.com>

Yes, your approach makes sense, and you're right that my during-the-renamer approach would struggle a bit. Go with your idea. :)

Richard

> On Jul 25, 2022, at 9:21 AM, Gergő Érdi <gergo at erdi.hu> wrote:
> 
> So my idea from my last email is to avoid the need for `substHsTy`:
> When `tc_hs_type` encounters a macro occurrence,  I would `tcLHsType`
> the rhs then and there (thereby getting fresh metas for each
> wildcard), use `substTy` to instantiate with the given type arguments.
> Then `tc_hs_type` returns that.
> 
> Note that this is NOT about typechecking the rhs *for the definition*,
> but rather, using `tcLHsType` as the function that creates fresh metas
> for each wildcard.
> 
> On Mon, Jul 25, 2022 at 8:58 PM Richard Eisenberg <lists at richarde.dev> wrote:
>> 
>> 
>> 
>>> On Jul 25, 2022, at 6:04 AM, ÉRDI Gergő <gergo at erdi.hu> wrote:
>>> 
>>> On Mon, 25 Jul 2022, Simon Peyton Jones wrote:
>>> 
>>>>     Do we have an existing way of substituting types over type variables, *in
>>>>     HsType instead of Core Type*?
>>>> I'm afraid not. Currently HsType is not processed much -- just renamed and typechecked
>>>> into a Type.
>>> 
>>> I wonder if, instead, I could expand the rhs, typecheck it "abstractly" (i.e. in the context of the synonym's binders), and THEN do the substitution.
>> 
>> Why type-check the RHS at all? Presumably, to give nice error messages. But it looks like this aspect of the plan is inessential. To be clear, I *do* think you should type-check the RHS, but I'm also checking my understanding here. If type-checking the RHS is indeed inessential, then the result of that type-checking (a desugared `Type`) should be discarded.
>> 
>>> If I typecheck the rhs for every occurrence, I should get fresh metavars for each wildcard, which is pretty much what I want. I just
>>> have to make sure I don't zonk before the substitution.
>>> 
>> 
>> I see this substitution as happening before any type-checking, so zonking shouldn't be an issue. That is, I would expect a
>> 
>> substHsTy :: UniqFM Name (HsType GhcRn) -> HsType GhcRn -> HsType GhcRn
>> 
>> to do the work, entirely before type-checking.
>> 
>> (Presumably, you don't want the macro-like behavior to extend to fixity resolution. That is, if we have
>> 
>> type macro T a = a + b
>> 
>> and then write `f :: T Int * Double`, we want `f :: (Int + b) * Double`, not `f :: Int + (b * Double)`. If you indeed want the latter (strange days!), then you'd need to be careful to do the substitution before fixity resolution, just after renaming.)
>> 
>> Richard
>> 
>>> Does this make sense?
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> 


From carter.schonwald at gmail.com  Wed Jul 27 13:25:39 2022
From: carter.schonwald at gmail.com (Carter Schonwald)
Date: Wed, 27 Jul 2022 09:25:39 -0400
Subject: success on a slightly modified version of your suggestion for
 21506
In-Reply-To: <CAB-d4A548yUaqFERYX8O_VuCHmOoPVOY+73ksLz7kN6PZKcsZg@mail.gmail.com>
References: <87o7xhz4ax.fsf@smart-cactus.org>
 <CAB-d4A4EgmQT6-GWbyi_sYWpf1cFk9Nughq7mL50qyK=XGkDqA@mail.gmail.com>
 <87edyczeti.fsf@smart-cactus.org>
 <CAB-d4A4LwLEawOyK1oDhHKb-q3B2Dj=dE-bKqTwCwna1D3PiCw@mail.gmail.com>
 <87bktgyo9s.fsf@smart-cactus.org>
 <CAB-d4A54B-z9vJT0PFWr7y2GYVWSdyKxLRCWanZVc12HQj9Y+g@mail.gmail.com>
 <878roizilc.fsf@smart-cactus.org>
 <CAB-d4A548yUaqFERYX8O_VuCHmOoPVOY+73ksLz7kN6PZKcsZg@mail.gmail.com>
Message-ID: <CAHYVw0z8+TUz0UbLQhF5wtizXMZzAe=onKO3msbXtVGEYUp+OQ@mail.gmail.com>

that matches my experience, namely that i've successfully installed ghc
9.2.2 on osx 12.4 aka monterey a few times

On Sun, Jul 24, 2022 at 12:59 PM George Colpitts <george.colpitts at gmail.com>
wrote:

> Hi Ben
>
> Thanks for the quick responses particularly on the weekend before your
> vacation.
>
> you wrote:
>
> What happens
> if you try to install to do something like (in the extracted binary
> distribution),
>
>
>     $ ./configure --prefix=`pwd`/tmp
>     $ make install            # this will fail
>     $ xattr -rc .
>     $ make install            # perhaps this will finish successfully?
>     # tmp/bin/ghc --version   # GHC should be usable
>
>
> success on both my machines using a slightly modified version of your
> suggestion above for 21506:
>
>  ./configure --prefix=`pwd`/tmp    # specifying ./tmp seems to be critical
>
> xattr -rc .                                        # seems to be necessary
> also
>
> sudo make install                           # seems to works , output ends
> with "recache"
>
> ./tmp/bin/ghc --version                  # success , although admittedly
> the smallest smoke test possible
>
> You wrote:
>
> >
> Ultimately I think we may just need to bite the bullet and start
> properly notarising/codesigning releases (resolving #17418). At this point
> we have
> spent more time trying to avoid the notarisation requirement than
> it would likely take to satisfy it. Unfortunately, this will require
> that I find an Apple device somewhere which may take a few weeks.
>
>
> I'm afraid I am on holiday next week but I would quite grateful if we
> could arrange for a chat after I return such that we can debug this in
> realtime.
>
>
> Sure, we can chat when you return from your vacation.
>
> Not sure if it is worth trying to fix the release on the basis of what I
> write above. My opinion is: it is if we can get reports of at least one
> other person having this issue. I am fine with not doing this for 9.4.1
>
> I agree that we should raise the priority of "notarising/codesigning
> releases (resolving #17418)". My opinion is that it is not worth delaying
> 9.4.1 for this.
>
> Have a great vacation.
>
> Cheers
> George
>
>
>
>
>
>
> On Sun, Jul 24, 2022 at 11:33 AM Ben Gamari <ben at well-typed.com> wrote:
>
>> George Colpitts <george.colpitts at gmail.com> writes:
>>
>> > +Kazu Yamamoto <kazu at iij.ad.jp>
>> >
>> > Hi Ben
>> >
>> > My 2 machines also have:
>> >
>> > $ spctl --status
>> > assessments enabled
>> >
>> Hmm, interesting. Then I am truly perplexed.
>>
>> > Speculations:
>> >
>> >
>> /usr/local/lib/ghc-9.4.0.20220721/bin/../lib/x86_64-osx-ghc-9.4.0.20220721/libHSterminfo-0.4.1.5-ghc9.4.0.20220721.dylib
>> > is created after xattr -rc . was run so it doesn't have the necessary
>> > attributes. Is it possible that ghc developers and/or the test machines
>> > have this file on another of the paths in the error message and that is
>> why
>> > it works for them?
>> >
>> I'm not sure where this would be but at this point anything is possible.
>> What happens
>> if you try to install to do something like (in the extracted binary
>> distribution),
>>
>>     $ ./configure --prefix=`pwd`/tmp
>>     $ make install            # this will fail
>>     $ xattr -rc .
>>     $ make install            # perhaps this will finish successfully?
>>     # tmp/bin/ghc --version   # GHC should be usable
>>
>> > I hope I didn't offend you by asking if the fix had been tested; I
>> assume
>> > it had been but I thought it was important to rule that out.
>> >
>> Not to worry; it's a very reasonable question to ask given the
>> circumstances.
>>
>> > More than happy to test. I really appreciate all the work you and others
>> > have put into GHC !
>> >
>> Ultimately I think we may just need to bite the bullet and start
>> properly notarising/codesigning releases (resolving #17418). At this
>> point we have
>> spent more time trying to avoid the notarisation requirement than
>> it would likely take to satisfy it. Unfortunately, this will require
>> that I find an Apple device somewhere which may take a few weeks.
>>
>> I'm afraid I am on holiday next week but I would quite grateful if we
>> could arrange for a chat after I return such that we can debug this in
>> realtime.
>>
>> Cheers,
>>
>> - Ben
>>
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220727/bdd1d312/attachment.html>

From gergo at erdi.hu  Thu Jul 28 05:53:21 2022
From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=)
Date: Thu, 28 Jul 2022 13:53:21 +0800 (+08)
Subject: Wildcards in type synonyms
In-Reply-To: <CAJKmMz9uagunA90AZ-iYjc3EwF9OpkLeo6B3HJvYcNi1ng4rzQ@mail.gmail.com>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
 <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>
 <CAO7EwMrJGL9YVKrY+h_2uwSfA_TZ_0VGJSExtkbwTPYGCcAAQA@mail.gmail.com>
 <CAJKmMz9qcSpmVyDDyK5UABM-hx=jeiyxq9dN6m+3mooLQkoOcw@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251717290.4398@galaxy>
 <CAJKmMz_DgrXavKWZmdOss8Y18Lq-ha9Md-HNTDfJn09iWgm7yA@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251802380.4398@galaxy>
 <CAJKmMz9uagunA90AZ-iYjc3EwF9OpkLeo6B3HJvYcNi1ng4rzQ@mail.gmail.com>
Message-ID: <alpine.DEB.2.22.394.2207281239580.4398@galaxy>

(TL;DR: `newMetaTyVarX` gives me type metavars that behave weirdly and I 
don't understand why. What shoudl I use instead?)

OK so I have two half-done implementations now:

   * Doing `HsType`-level substitution in the middle of `tc_infer_hs_type`
     (see my exchange with Richard on why this needs to happen in
     `tc_infer_hs_type` instead of `rnHsTyKi`)

   * Doing Core `Type`-level substitution in the middle of `tc_infer_hs_type`

The advantage of the first one is that it works :) The disadvantage is 
that it involves storing a `HsType` in a `TyCon`, which in turn means 
making it work inter-module will require an `Iface` representation for 
`HsType`s.

Hence the second attempt. I think that would be a more principled solution 
anyway. This approach is based on typechecking the macro's right-hand side 
into a core `Type`, and storing that, and the list of wildcard-originating 
`TyVar`s, in the `TyCon`. At every occurrence site, I take this core 
`Type` and apply a substitution on it that is the composition of the 
following two:

   * A substitution from macro type synonym type parameters to the type 
arguments
   * An instantiation of each wildcard variable into a fresh metavariable

Unfortunately, it is this second step that is tripping me up. If I use 
`newMetaTyVarX` to make these "refreshing" metavars, then while the 
substitution looks OK when eyeballing it, the resulting 
*type* metavariables seem to be handled by GHC as if they were *kind* 
metavariables?!

Here's an example. The source input is:

```
{-# LANGUAGE NoPolyKinds, NoStarIsType #-} -- Makes it easier to see how it goes wrong

data MyData a b c = MkMyData a b c
type MySyn a = MyData a _ Int

f1 :: MyData a b c -> b
f1 (MkMyData _ x _) = x

f2 :: MySyn a -> Double
f2 = f1
```

I start with the following "macro type template" (using `-dppr-debug` format):

```
   TySynWildcard.MyData{tc r3}
     (a{tv auq} Nothing [sk:1] :: GHC.Types.Type{(w) tc 32Q})
     ((w_awX{tv} Nothing [tau:0] :: (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q}))
        |> {(co_awY{v} Just 'GHC.Types.Many{(w) d 65I} [lid[CoVarId]] :: GHC.Prim.~#{(w) tc 31I}
                                                                           GHC.Types.Type{(w) tc 32Q}
                                                                           GHC.Types.Type{(w) tc 32Q}
                                                                           (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q})
                                                                           GHC.Types.Type{(w) tc 32Q})})
     GHC.Types.Int{(w) tc 3u}
```

The substitution applied:

```
   [TCvSubst
      In scope: InScope {a{tv auu} k_awW{tv} w_axc{tv}}
      Type env: [auq :-> (a{tv auu} Nothing [sk:2] :: (k_ax9{tv} Nothing [tau:2] :: GHC.Types.Type{(w) tc 32Q})),
                 awX :-> (w_axc{tv} Nothing [tau:2] :: (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q}))]
      Co env: []]
```

Note that the second type substitution, (w_awX :: k_awW) :-> (w_axc :: k_awW) is the 
one that should take care of instantiating the wildcard metavariable. And 
the result of applying this substitution still looks OK:

```
   TySynWildcard.MyData{tc r3}
     (a{tv auu} Nothing [sk:2] :: (k_ax9{tv} Nothing [tau:2] :: GHC.Types.Type{(w) tc 32Q}))
     ((w_axc{tv} Nothing [tau:2] :: (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q}))
        |> {(co_awY{v} Just 'GHC.Types.Many{(w) d 65I} [lid[CoVarId]] :: GHC.Prim.~#{(w) tc 31I}
                                                                           GHC.Types.Type{(w) tc 32Q}
                                                                           GHC.Types.Type{(w) tc 32Q}
                                                                           (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q})
                                                                           GHC.Types.Type{(w) tc 32Q})})
     GHC.Types.Int{(w) tc 3u}
```

But soon after, typechecking fails:

```
     • Couldn't match type ‘Type’ with ‘Double’
       Expected: MyData a Type Int -> Double
         Actual: MyData a Type Int -> Type
     • In the expression: f1
       In an equation for ‘f2’: f2 = f1
```

So this is weird. Instead of unification solving `w_axc ~ Double`, it 
seems `w_axc` is left unrestricted, and then `NoPolyKinds` picks it up as 
a kind variable (why?) and defaults it to `Type`.

As an experiment, I have also tried *not* refreshing `w_awX`, only 
substituting in the type arguments. Now, of course, this can't possibly 
work as soon as I have more than one occurrence of `MySyn` due to the
interference between the wildcard metavars, but if I only have one, then
the program typechecks. So to me this suggests I'm doing things mostly 
right, except that the metavar returned by `newMetaTyVarX` is not fit for 
my use case.

What should I use instead of `newMetaTyVarX` to instantiate / "refresh" 
the (wildcard-originating) type metavariables in my "macro type template"?

Thanks,
 	Gergo


On Mon, 25 Jul 2022, Simon Peyton Jones wrote:

> I'm afraid I don't understand, but it sounds delicate.  By all means try!
> 
> Simon
> 
> On Mon, 25 Jul 2022 at 11:04, ÉRDI Gergő <gergo at erdi.hu> wrote:
>       On Mon, 25 Jul 2022, Simon Peyton Jones wrote:
>
>       >       Do we have an existing way of substituting types over type
>       variables, *in
>       >       HsType instead of Core Type*?
>       >
>       >
>       > I'm afraid not. Currently HsType is not processed much -- just renamed
>       and typechecked
>       > into a Type.
>
>       I wonder if, instead, I could expand the rhs, typecheck it "abstractly"
>       (i.e. in the context of the synonym's binders), and THEN do the
>       substitution. If I typecheck the rhs for every occurrence, I should get
>       fresh metavars for each wildcard, which is pretty much what I want. I just
>       have to make sure I don't zonk before the substitution.
>
>       Does this make sense?
> 
> 
>

From simon.peytonjones at gmail.com  Thu Jul 28 07:41:20 2022
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Thu, 28 Jul 2022 08:41:20 +0100
Subject: Wildcards in type synonyms
In-Reply-To: <alpine.DEB.2.22.394.2207281239580.4398@galaxy>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
 <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>
 <CAO7EwMrJGL9YVKrY+h_2uwSfA_TZ_0VGJSExtkbwTPYGCcAAQA@mail.gmail.com>
 <CAJKmMz9qcSpmVyDDyK5UABM-hx=jeiyxq9dN6m+3mooLQkoOcw@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251717290.4398@galaxy>
 <CAJKmMz_DgrXavKWZmdOss8Y18Lq-ha9Md-HNTDfJn09iWgm7yA@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251802380.4398@galaxy>
 <CAJKmMz9uagunA90AZ-iYjc3EwF9OpkLeo6B3HJvYcNi1ng4rzQ@mail.gmail.com>
 <alpine.DEB.2.22.394.2207281239580.4398@galaxy>
Message-ID: <CAJKmMz-xdC2n6u8yLvUQjo1t=NyNihJP+tA_VRWVKkw3pqaHcg@mail.gmail.com>

Wild guess: you aren't instantiating the kinds of the meta-tyvars to
something sensible, so kinds don't line up.  Eg. where are k_ax9 and k_awW
bound?

You need to be super-careful about the *level* of wildcards.  That is a
tricky bit about the whole wildcard implementation.

I'm still unconvinced whether this is a good use of your valuable time --
but of course that is up to you.

Simon

On Thu, 28 Jul 2022 at 06:53, ÉRDI Gergő <gergo at erdi.hu> wrote:

> (TL;DR: `newMetaTyVarX` gives me type metavars that behave weirdly and I
> don't understand why. What shoudl I use instead?)
>
> OK so I have two half-done implementations now:
>
>    * Doing `HsType`-level substitution in the middle of `tc_infer_hs_type`
>      (see my exchange with Richard on why this needs to happen in
>      `tc_infer_hs_type` instead of `rnHsTyKi`)
>
>    * Doing Core `Type`-level substitution in the middle of
> `tc_infer_hs_type`
>
> The advantage of the first one is that it works :) The disadvantage is
> that it involves storing a `HsType` in a `TyCon`, which in turn means
> making it work inter-module will require an `Iface` representation for
> `HsType`s.
>
> Hence the second attempt. I think that would be a more principled solution
> anyway. This approach is based on typechecking the macro's right-hand side
> into a core `Type`, and storing that, and the list of wildcard-originating
> `TyVar`s, in the `TyCon`. At every occurrence site, I take this core
> `Type` and apply a substitution on it that is the composition of the
> following two:
>
>    * A substitution from macro type synonym type parameters to the type
> arguments
>    * An instantiation of each wildcard variable into a fresh metavariable
>
> Unfortunately, it is this second step that is tripping me up. If I use
> `newMetaTyVarX` to make these "refreshing" metavars, then while the
> substitution looks OK when eyeballing it, the resulting
> *type* metavariables seem to be handled by GHC as if they were *kind*
> metavariables?!
>
> Here's an example. The source input is:
>
> ```
> {-# LANGUAGE NoPolyKinds, NoStarIsType #-} -- Makes it easier to see how
> it goes wrong
>
> data MyData a b c = MkMyData a b c
> type MySyn a = MyData a _ Int
>
> f1 :: MyData a b c -> b
> f1 (MkMyData _ x _) = x
>
> f2 :: MySyn a -> Double
> f2 = f1
> ```
>
> I start with the following "macro type template" (using `-dppr-debug`
> format):
>
> ```
>    TySynWildcard.MyData{tc r3}
>      (a{tv auq} Nothing [sk:1] :: GHC.Types.Type{(w) tc 32Q})
>      ((w_awX{tv} Nothing [tau:0] :: (k_awW{tv} Nothing [tau:0] ::
> GHC.Types.Type{(w) tc 32Q}))
>         |> {(co_awY{v} Just 'GHC.Types.Many{(w) d 65I} [lid[CoVarId]] ::
> GHC.Prim.~#{(w) tc 31I}
>
>  GHC.Types.Type{(w) tc 32Q}
>
>  GHC.Types.Type{(w) tc 32Q}
>
>  (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q})
>
>  GHC.Types.Type{(w) tc 32Q})})
>      GHC.Types.Int{(w) tc 3u}
> ```
>
> The substitution applied:
>
> ```
>    [TCvSubst
>       In scope: InScope {a{tv auu} k_awW{tv} w_axc{tv}}
>       Type env: [auq :-> (a{tv auu} Nothing [sk:2] :: (k_ax9{tv} Nothing
> [tau:2] :: GHC.Types.Type{(w) tc 32Q})),
>                  awX :-> (w_axc{tv} Nothing [tau:2] :: (k_awW{tv} Nothing
> [tau:0] :: GHC.Types.Type{(w) tc 32Q}))]
>       Co env: []]
> ```
>
> Note that the second type substitution, (w_awX :: k_awW) :-> (w_axc ::
> k_awW) is the
> one that should take care of instantiating the wildcard metavariable. And
> the result of applying this substitution still looks OK:
>
> ```
>    TySynWildcard.MyData{tc r3}
>      (a{tv auu} Nothing [sk:2] :: (k_ax9{tv} Nothing [tau:2] ::
> GHC.Types.Type{(w) tc 32Q}))
>      ((w_axc{tv} Nothing [tau:2] :: (k_awW{tv} Nothing [tau:0] ::
> GHC.Types.Type{(w) tc 32Q}))
>         |> {(co_awY{v} Just 'GHC.Types.Many{(w) d 65I} [lid[CoVarId]] ::
> GHC.Prim.~#{(w) tc 31I}
>
>  GHC.Types.Type{(w) tc 32Q}
>
>  GHC.Types.Type{(w) tc 32Q}
>
>  (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q})
>
>  GHC.Types.Type{(w) tc 32Q})})
>      GHC.Types.Int{(w) tc 3u}
> ```
>
> But soon after, typechecking fails:
>
> ```
>      • Couldn't match type ‘Type’ with ‘Double’
>        Expected: MyData a Type Int -> Double
>          Actual: MyData a Type Int -> Type
>      • In the expression: f1
>        In an equation for ‘f2’: f2 = f1
> ```
>
> So this is weird. Instead of unification solving `w_axc ~ Double`, it
> seems `w_axc` is left unrestricted, and then `NoPolyKinds` picks it up as
> a kind variable (why?) and defaults it to `Type`.
>
> As an experiment, I have also tried *not* refreshing `w_awX`, only
> substituting in the type arguments. Now, of course, this can't possibly
> work as soon as I have more than one occurrence of `MySyn` due to the
> interference between the wildcard metavars, but if I only have one, then
> the program typechecks. So to me this suggests I'm doing things mostly
> right, except that the metavar returned by `newMetaTyVarX` is not fit for
> my use case.
>
> What should I use instead of `newMetaTyVarX` to instantiate / "refresh"
> the (wildcard-originating) type metavariables in my "macro type template"?
>
> Thanks,
>         Gergo
>
>
> On Mon, 25 Jul 2022, Simon Peyton Jones wrote:
>
> > I'm afraid I don't understand, but it sounds delicate.  By all means try!
> >
> > Simon
> >
> > On Mon, 25 Jul 2022 at 11:04, ÉRDI Gergő <gergo at erdi.hu> wrote:
> >       On Mon, 25 Jul 2022, Simon Peyton Jones wrote:
> >
> >       >       Do we have an existing way of substituting types over type
> >       variables, *in
> >       >       HsType instead of Core Type*?
> >       >
> >       >
> >       > I'm afraid not. Currently HsType is not processed much -- just
> renamed
> >       and typechecked
> >       > into a Type.
> >
> >       I wonder if, instead, I could expand the rhs, typecheck it
> "abstractly"
> >       (i.e. in the context of the synonym's binders), and THEN do the
> >       substitution. If I typecheck the rhs for every occurrence, I
> should get
> >       fresh metavars for each wildcard, which is pretty much what I
> want. I just
> >       have to make sure I don't zonk before the substitution.
> >
> >       Does this make sense?
> >
> >
> >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220728/38270d59/attachment.html>

From gergo at erdi.hu  Thu Jul 28 08:26:10 2022
From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=)
Date: Thu, 28 Jul 2022 16:26:10 +0800 (+08)
Subject: Wildcards in type synonyms
In-Reply-To: <CAJKmMz-xdC2n6u8yLvUQjo1t=NyNihJP+tA_VRWVKkw3pqaHcg@mail.gmail.com>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
 <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>
 <CAO7EwMrJGL9YVKrY+h_2uwSfA_TZ_0VGJSExtkbwTPYGCcAAQA@mail.gmail.com>
 <CAJKmMz9qcSpmVyDDyK5UABM-hx=jeiyxq9dN6m+3mooLQkoOcw@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251717290.4398@galaxy>
 <CAJKmMz_DgrXavKWZmdOss8Y18Lq-ha9Md-HNTDfJn09iWgm7yA@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251802380.4398@galaxy>
 <CAJKmMz9uagunA90AZ-iYjc3EwF9OpkLeo6B3HJvYcNi1ng4rzQ@mail.gmail.com>
 <alpine.DEB.2.22.394.2207281239580.4398@galaxy>
 <CAJKmMz-xdC2n6u8yLvUQjo1t=NyNihJP+tA_VRWVKkw3pqaHcg@mail.gmail.com>
Message-ID: <alpine.DEB.2.22.394.2207281551300.4398@galaxy>

On Thu, 28 Jul 2022, Simon Peyton Jones wrote:

> Wild guess: you aren't instantiating the kinds of the meta-tyvars to something
> sensible, so kinds don't line up.  Eg. where are k_ax9 and k_awW bound?

`a_auu :: k_ax9` is the result of typechecking my type argument (the `a` 
in the source type `MySyn a`), so I would expect its kind to work because 
it is computed by parts of GHC that I am not changing.

`w_awX :: k_awW` is the metavariable created during the typechecking of 
the right-hand side of my type macro `type MySyn a = MyData a _ Int`. This
kind is then kept for the fresh metavariable that I try to replace `w_awX` 
with, since I have `w_axc :: k_awW`.

The part that confuses me the most is that the right-hand side seems to 
work just fine on its own:

>       As an experiment, I have also tried *not* refreshing `w_awX`, only
>       substituting in the type arguments. Now, of course, this can't possibly
>       work as soon as I have more than one occurrence of `MySyn` due to the
>       interference between the wildcard metavars, but if I only have one, then
>       the program typechecks.

So I have two versions of my code returning two types from 
`tc_infer_hs_type` that only differ in containing either `w_awX :: k_awW` 
or `w_axc :: k_awW`, and one of them typechecks while the other causes a 
type error.

Typechecks (as long as only used once of course):

```
   TySynWildcard.MyData{tc r3}
     (a{tv auu} Nothing [sk:2] :: (k_ax9{tv} Nothing [tau:2] :: GHC.Types.Type{(w) tc 32Q}))
     ((w_awX{tv} Nothing [tau:0] :: (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q}))
        |> {(co_awY{v} Just 'GHC.Types.Many{(w) d 65I} [lid[CoVarId]] :: GHC.Prim.~#{(w) tc 31I}
                                                                           GHC.Types.Type{(w) tc 32Q}
                                                                           GHC.Types.Type{(w) tc 32Q}
                                                                           (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q})
                                                                           GHC.Types.Type{(w) tc 32Q})})
     GHC.Types.Int{(w) tc 3u}
```

Doesn't typecheck:

```
   TySynWildcard.MyData{tc r3}
     (a{tv auu} Nothing [sk:2] :: (k_ax9{tv} Nothing [tau:2] :: GHC.Types.Type{(w) tc 32Q}))
     ((w_axc{tv} Nothing [tau:2] :: (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q}))
        |> {(co_awY{v} Just 'GHC.Types.Many{(w) d 65I} [lid[CoVarId]] :: GHC.Prim.~#{(w) tc 31I}
                                                                           GHC.Types.Type{(w) tc 32Q}
                                                                           GHC.Types.Type{(w) tc 32Q}
                                                                           (k_awW{tv} Nothing [tau:0] :: GHC.Types.Type{(w) tc 32Q})
                                                                           GHC.Types.Type{(w) tc 32Q})})
     GHC.Types.Int{(w) tc 3u}
```

What is the difference?

From simon.peytonjones at gmail.com  Thu Jul 28 09:51:40 2022
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Thu, 28 Jul 2022 10:51:40 +0100
Subject: Gitlab labels
Message-ID: <CAJKmMz_kc+HGeZ=yVSNGmN-GTpXYhOzs2U_miZ+WqnaHgyqYSw@mail.gmail.com>

Whenever I try to add a label to a ticket I get "an error occurred while
updating labels".  See below.

When I repeat the exact same sequence, it works.

If I add another label, again the error; repeating again works.

This is tiresome and peculiar.  Does anyone have any ideas?

Simon

[image: image.png]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220728/1e6f880b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 19732 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220728/1e6f880b/attachment.png>

From zubin at well-typed.com  Thu Jul 28 09:55:11 2022
From: zubin at well-typed.com (Zubin Duggal)
Date: Thu, 28 Jul 2022 15:25:11 +0530
Subject: Gitlab labels
In-Reply-To: <CAJKmMz_kc+HGeZ=yVSNGmN-GTpXYhOzs2U_miZ+WqnaHgyqYSw@mail.gmail.com>
References: <CAJKmMz_kc+HGeZ=yVSNGmN-GTpXYhOzs2U_miZ+WqnaHgyqYSw@mail.gmail.com>
Message-ID: <20220728095511.e55p2rfdbn7x73tn@zubin-msi>

The error is bogus, if you refresh the page you will see that the label
has been updated.

This seems to be a bug in Gitlab, and I've also been seeing it a lot
more often recently. I think Bryan attempted to investigate at a certain
point, but I don't know what came of it.

On 22/07/28 10:51, Simon Peyton Jones wrote:
>Whenever I try to add a label to a ticket I get "an error occurred while
>updating labels".  See below.
>
>When I repeat the exact same sequence, it works.
>
>If I add another label, again the error; repeating again works.
>
>This is tiresome and peculiar.  Does anyone have any ideas?
>
>Simon
>
>[image: image.png]



>_______________________________________________
>ghc-devs mailing list
>ghc-devs at haskell.org
>http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From bryan at haskell.foundation  Thu Jul 28 10:31:58 2022
From: bryan at haskell.foundation (Bryan Richter)
Date: Thu, 28 Jul 2022 13:31:58 +0300
Subject: Gitlab labels
In-Reply-To: <20220728095511.e55p2rfdbn7x73tn@zubin-msi>
References: <CAJKmMz_kc+HGeZ=yVSNGmN-GTpXYhOzs2U_miZ+WqnaHgyqYSw@mail.gmail.com>
 <20220728095511.e55p2rfdbn7x73tn@zubin-msi>
Message-ID: <59c4441f-06a5-ce61-fd87-d5d358734840@haskell.foundation>

I haven't looked into it yet.

As I haven't seen this problem on any other instance of GitLab, I have a 
hunch it's unique to gitlab.haskell.org. I'd like to look into it 
eventually.

-Bryan

On 28/07/2022 12:55, Zubin Duggal wrote:
> The error is bogus, if you refresh the page you will see that the label
> has been updated.
>
> This seems to be a bug in Gitlab, and I've also been seeing it a lot
> more often recently. I think Bryan attempted to investigate at a certain
> point, but I don't know what came of it.
>
> On 22/07/28 10:51, Simon Peyton Jones wrote:
>> Whenever I try to add a label to a ticket I get "an error occurred while
>> updating labels".  See below.
>>
>> When I repeat the exact same sequence, it works.
>>
>> If I add another label, again the error; repeating again works.
>>
>> This is tiresome and peculiar.  Does anyone have any ideas?
>>
>> Simon
>>
>> [image: image.png]
>
>
>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From zubin at well-typed.com  Thu Jul 28 11:17:13 2022
From: zubin at well-typed.com (Zubin Duggal)
Date: Thu, 28 Jul 2022 16:47:13 +0530
Subject: [Haskell] [ANNOUNCE] GHC 9.2.4 released
Message-ID: <20220728111713.owpnsu43zwq2klhn@zubin-msi>

The GHC developers are very happy to at announce the availability of GHC
9.2.4. Binary distributions, source distributions, and documentation are
available at [`downloads.haskell.org`](https://downloads.haskell.org/ghc/9.2.4).

Download Page: https://www.haskell.org/ghc/download_ghc_9_2_4.html
Blog Post: https://www.haskell.org/ghc/blog/20220728-ghc-9.2.4-released.html

This release will include:

  - The new `DeepSubsumption` language extension which reverses the
    effects of the [Simplified Subsumption Proposal] introduced in GHC 9.0. This
    is an attempt to make GHC 9.2.4 more backwards compatible with GHC 8.10 and
    eases migration for users who depended on this feature.

    This extension is enabled by default with the `Haskell2010`
    and `Haskell98` languages but disabled with the `GHC2021`
    language originally introduced in GHC 9.2.1.

    See the [Deep Subsumption Proposal] for more details.

  - Fixes for segfaults that may arise due to a bug in the implementation of the
    `keepAlive#` primop. This may regress performance for certain programs which
    use this primop or functions which use the primop, such as `withForeignPtr`.
    These regressions are mostly small, but can be larger in certain edge cases.
    Judicious use of `unsafeWithForeignPtr` when its argument is known not to
    statically diverge can mitigate these in many cases. It is our judgment that
    the critical correctness issues justify the regression in performance and that
    it is important to get a release out with the fix while we work on a better
    approach which will improve performance for future releases (#21708).

    We have a [wiki page](https://gitlab.haskell.org/ghc/ghc/-/wikis/proposal/with-combinator)
    that tracks possible solutions to this problem, and Ben wrote a
    [blog post](https://www.haskell.org/ghc/blog/20210607-the-keepAlive-story.html)
    detailing the introduction of the `keepAlive#` primop and its history.

  - Fixes for a number of miscompilations on AArch64 and other platforms (#21624,
    #21773, #20735, #21685).

  - Fixes for segfaults due to bugs in the RTS and GC (#21708, #21880, #21885).

  - Fixing the behaviour of Ctrl-C with GHCi on Windows (#21889).

   - ... and much more. See the [release notes] for a full accounting.

As some of the fixed issues do affect correctness users are encouraged to
upgrade promptly.

We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk
stake pool, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation, and other anonymous
contributors whose on-going financial and in-kind support has
facilitated GHC maintenance and release management over the years.
Finally, this release would not have been possible without the hundreds
of open-source contributors whose work comprise this release.

As always, do give this release a try and open a [ticket] if you see
anything amiss.

Happy compiling,

- Zubin

[ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new
[release notes]: https://downloads.haskell.org/~ghc/9.2.4/docs/users_guide/9.2.4-notes.html
[Simplified Subsumption Proposal]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0287-simplify-subsumption.rst
[Deep Subsumption Proposal]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0511-deep-subsumption.rst
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220728/77ff490b/attachment-0001.sig>

From gergo at erdi.hu  Thu Jul 28 13:16:55 2022
From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=)
Date: Thu, 28 Jul 2022 21:16:55 +0800 (+08)
Subject: Wildcards in type synonyms
In-Reply-To: <CAJKmMz-xdC2n6u8yLvUQjo1t=NyNihJP+tA_VRWVKkw3pqaHcg@mail.gmail.com>
References: <alpine.DEB.2.22.394.2207221418020.4398@galaxy>
 <CAJKmMz_XV7GV4X+WbaT4gYTCnvHA8QXpz0_wwREOJyuNMrPxqQ@mail.gmail.com>
 <CAO7EwMrJGL9YVKrY+h_2uwSfA_TZ_0VGJSExtkbwTPYGCcAAQA@mail.gmail.com>
 <CAJKmMz9qcSpmVyDDyK5UABM-hx=jeiyxq9dN6m+3mooLQkoOcw@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251717290.4398@galaxy>
 <CAJKmMz_DgrXavKWZmdOss8Y18Lq-ha9Md-HNTDfJn09iWgm7yA@mail.gmail.com>
 <alpine.DEB.2.22.394.2207251802380.4398@galaxy>
 <CAJKmMz9uagunA90AZ-iYjc3EwF9OpkLeo6B3HJvYcNi1ng4rzQ@mail.gmail.com>
 <alpine.DEB.2.22.394.2207281239580.4398@galaxy>
 <CAJKmMz-xdC2n6u8yLvUQjo1t=NyNihJP+tA_VRWVKkw3pqaHcg@mail.gmail.com>
Message-ID: <alpine.DEB.2.22.394.2207282115040.4398@galaxy>

On Thu, 28 Jul 2022, Simon Peyton Jones wrote:

> You need to be super-careful about the *level* of wildcards.  That is a tricky bit
> about the whole wildcard implementation.

Indeed! Ensuring the fresh metavars are at `topTcLevel` seems to have 
fixed my problem. My goal doesn't include support higher-rank types at all, 
anyway, so this should be good enough for now.

Thanks, this was the key to all my troubles.

From simon.peytonjones at gmail.com  Thu Jul 28 13:47:17 2022
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Thu, 28 Jul 2022 14:47:17 +0100
Subject: Test suite
Message-ID: <CAJKmMz_14cHR1eA7DDd+yo6-pitqeNmqMwnCuFnpE9JDyVFjxQ@mail.gmail.com>

Doing *hadrian/build test*
takes absolutely ages.   It seems that it is building haddock, running
haddock, building check_exact, and other things.  Eg right now it is doing
Run Haddock BuildPackage: libraries/parsec/src/Text/Parsec.hs (and 24 more)
=> _build/doc/html/libraries/parsec-3.1.15.0/parsec.haddock

But I didn't ask to do Haddock. I just wanted to run the testsuite.   How
can I do that?

I would prefer never to build haddock.

Thanks

Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220728/016f9186/attachment.html>

From cheng.shao at tweag.io  Thu Jul 28 14:15:08 2022
From: cheng.shao at tweag.io (Cheng Shao)
Date: Thu, 28 Jul 2022 16:15:08 +0200
Subject: Test suite
In-Reply-To: <CAJKmMz_14cHR1eA7DDd+yo6-pitqeNmqMwnCuFnpE9JDyVFjxQ@mail.gmail.com>
References: <CAJKmMz_14cHR1eA7DDd+yo6-pitqeNmqMwnCuFnpE9JDyVFjxQ@mail.gmail.com>
Message-ID: <CALHrtFnCBKKAhjrgswH8dOKhYe5PnDch70dAYxZmerpgS+Vo9Q@mail.gmail.com>

Simon,

I believe you can use `--docs=none`, at least it'll prevent running haddock.

On Thu, Jul 28, 2022 at 3:46 PM Simon Peyton Jones
<simon.peytonjones at gmail.com> wrote:
>
> Doing hadrian/build test
> takes absolutely ages.   It seems that it is building haddock, running haddock, building check_exact, and other things.  Eg right now it is doing
> Run Haddock BuildPackage: libraries/parsec/src/Text/Parsec.hs (and 24 more) => _build/doc/html/libraries/parsec-3.1.15.0/parsec.haddock
>
> But I didn't ask to do Haddock. I just wanted to run the testsuite.   How can I do that?
>
> I would prefer never to build haddock.
>
> Thanks
>
> Simon
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From zubin at well-typed.com  Thu Jul 28 14:23:11 2022
From: zubin at well-typed.com (Zubin Duggal)
Date: Thu, 28 Jul 2022 19:53:11 +0530
Subject: Test suite
In-Reply-To: <CAJKmMz_14cHR1eA7DDd+yo6-pitqeNmqMwnCuFnpE9JDyVFjxQ@mail.gmail.com>
References: <CAJKmMz_14cHR1eA7DDd+yo6-pitqeNmqMwnCuFnpE9JDyVFjxQ@mail.gmail.com>
Message-ID: <20220728142311.7zqwk6cnxhvtrhvy@zubin-msi>

There are tests that depend upon haddock in the testsuite, so we need to
build them to run those tests. Like Cheng says, passing `--docs=none`
will avoid those tests.

On 22/07/28 14:47, Simon Peyton Jones wrote:
>Doing *hadrian/build test*
>takes absolutely ages.   It seems that it is building haddock, running
>haddock, building check_exact, and other things.  Eg right now it is doing
>Run Haddock BuildPackage: libraries/parsec/src/Text/Parsec.hs (and 24 more)
>=> _build/doc/html/libraries/parsec-3.1.15.0/parsec.haddock
>
>But I didn't ask to do Haddock. I just wanted to run the testsuite.   How
>can I do that?
>
>I would prefer never to build haddock.
>
>Thanks
>
>Simon

>_______________________________________________
>ghc-devs mailing list
>ghc-devs at haskell.org
>http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From simon.peytonjones at gmail.com  Thu Jul 28 15:06:21 2022
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Thu, 28 Jul 2022 16:06:21 +0100
Subject: Test suite
In-Reply-To: <20220728142311.7zqwk6cnxhvtrhvy@zubin-msi>
References: <CAJKmMz_14cHR1eA7DDd+yo6-pitqeNmqMwnCuFnpE9JDyVFjxQ@mail.gmail.com>
 <20220728142311.7zqwk6cnxhvtrhvy@zubin-msi>
Message-ID: <CAJKmMz_fNDfxZdGkYbP0KCFb5ZEYk35CNx4EOYb5-JGOP7KWQg@mail.gmail.com>

>
> There are tests that depend upon haddock in the testsuite, so we need to
> build them to run those tests. Like Cheng says, passing `--docs=none`
> will avoid those tests.
>

But looking at `ps` I see an invocation of haddock that was running for 20
minutes!  This can't be right.

Adding --docs=none apparently didn't work.  I get zillions of failures.

But then it turned out that this failed because I also added -VV, in the
hope of seeing what is going on.  So this works
hadrian/build test --docs=none --only=T13199
But this does not (log below)
hadrian/build test --docs=none --only=T13199 -VV

I thought -VV controlled verbosity only!

Simon

=====> T13199(normal) 1 of 1 [0, 0, 0]
cd "/tmp/ghctest-6yc58m02/test   spaces/testsuite/tests/printer/T13199.run"
&& $MAKE --no-print-directory T13199   <
]0;T13199(normal) 1 of 1 [0, 0, 0] Actual stdout output differs from
expected:
diff -uw "/dev/null" "/tmp/ghctest-6yc58m02/test
spaces/testsuite/tests/printer/T13199.run/T13199.run.stdout.normalised"<
--- /dev/null 2022-07-28 08:52:58.419999998 +0100
+++ "/tmp/ghctest-6yc58m02/test
spaces/testsuite/tests/printer/T13199.run/T13199.run.stdout.normalised"
2022-07-28 16:04:06.479419465 +0100
@@ -0,0 +1,2 @@
+/home/simonpj/code/HEAD-1/_build/test/bin/check-ppr
"`'/home/simonpj/code/HEAD-1/_build/stage1/bin/ghc' -dcore-lint -dstg-lint
-dcmm-lint -fno-dump-with-ways -no-user-package-db -rtsopts
 -fno-warn-missed-specialisations -fshow-warning-groups
-fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat
-dno-debug-output --print-libdir | tr -d '\r'`" T13199.hs
+/home/simonpj/code/HEAD-1/_build/test/bin/check-exact
"`'/home/simonpj/code/HEAD-1/_build/stage1/bin/ghc' -dcore-lint -dstg-lint
-dcmm-lint -fno-dump-with-ways -no-user-package-db -rtsopts
 -fno-warn-missed-specialisations -fshow-warning-groups
-fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat
-dno-debug-output --print-libdir | tr -d '\r'`" T13199.hs
*** unexpected failure for T13199(normal)

Performance Metrics (test environment: local):


None collected.


Unexpected results from:
TEST="T13199"

SUMMARY for test run started at Thu Jul 28 16:04:05 2022
0:00:00.845518 spent to go through
       1 total tests, which gave rise to
       7 test cases, of which
       6 were skipped
       0 had missing libraries

       0 expected passes
       0 expected failures

       0 caused framework failures
       0 caused framework warnings
       0 unexpected passes
       1 unexpected failures
       0 unexpected stat failures
       0 fragile tests

Unexpected failures:
   /tmp/ghctest-6yc58m02/test   spaces/testsuite/tests/printer/T13199.run
 T13199 [bad stdout] (normal)

On Thu, 28 Jul 2022 at 15:23, Zubin Duggal <zubin at well-typed.com> wrote:

> There are tests that depend upon haddock in the testsuite, so we need to
> build them to run those tests. Like Cheng says, passing `--docs=none`
> will avoid those tests.
>
> On 22/07/28 14:47, Simon Peyton Jones wrote:
> >Doing *hadrian/build test*
> >takes absolutely ages.   It seems that it is building haddock, running
> >haddock, building check_exact, and other things.  Eg right now it is doing
> >Run Haddock BuildPackage: libraries/parsec/src/Text/Parsec.hs (and 24
> more)
> >=> _build/doc/html/libraries/parsec-3.1.15.0/parsec.haddock
> >
> >But I didn't ask to do Haddock. I just wanted to run the testsuite.   How
> >can I do that?
> >
> >I would prefer never to build haddock.
> >
> >Thanks
> >
> >Simon
>
> >_______________________________________________
> >ghc-devs mailing list
> >ghc-devs at haskell.org
> >http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220728/5b0ff39e/attachment.html>

From george.colpitts at gmail.com  Thu Jul 28 15:52:01 2022
From: george.colpitts at gmail.com (George Colpitts)
Date: Thu, 28 Jul 2022 12:52:01 -0300
Subject: [Haskell] [ANNOUNCE] GHC 9.2.4 released
In-Reply-To: <20220728111713.owpnsu43zwq2klhn@zubin-msi>
References: <20220728111713.owpnsu43zwq2klhn@zubin-msi>
Message-ID: <CAB-d4A4oqJ9xNhQSapq8gcnAfX8VsU7M=ppJXeoOX2D+WrzKWQ@mail.gmail.com>

Thanks Zubin, this is good news.

On a Mac when do ./configure I see

checking Xcode version... xcode-select: error: tool 'xcodebuild' requires
Xcode, but active developer directory '/Library/Developer/CommandLineTools'
is a command line tools instance

not found (too old?)

I assume I can ignore that but FWIW it's weird as I do have that
directory and it is not old:


$ ls -ld /Library/Developer/CommandLineTools
drwxr-xr-x@ 5 root  wheel  160 27 May 20:54
/Library/Developer/CommandLineTools

$ xcode-select --version
xcode-select version 2395.


On a Mac it is still necessary to do

xattr -rc .

before doing

sudo make install


this is also true for 9.4.1-rc1 which I noted in 21506
<https://gitlab.haskell.org/ghc/ghc/-/issues/21506>.

Thanks
George





On Thu, Jul 28, 2022 at 8:17 AM Zubin Duggal <zubin at well-typed.com> wrote:

> The GHC developers are very happy to at announce the availability of GHC
> 9.2.4. Binary distributions, source distributions, and documentation are
> available at [`downloads.haskell.org`](
> https://downloads.haskell.org/ghc/9.2.4).
>
> Download Page: https://www.haskell.org/ghc/download_ghc_9_2_4.html
> Blog Post:
> https://www.haskell.org/ghc/blog/20220728-ghc-9.2.4-released.html
>
> This release will include:
>
>   - The new `DeepSubsumption` language extension which reverses the
>     effects of the [Simplified Subsumption Proposal] introduced in GHC
> 9.0. This
>     is an attempt to make GHC 9.2.4 more backwards compatible with GHC
> 8.10 and
>     eases migration for users who depended on this feature.
>
>     This extension is enabled by default with the `Haskell2010`
>     and `Haskell98` languages but disabled with the `GHC2021`
>     language originally introduced in GHC 9.2.1.
>
>     See the [Deep Subsumption Proposal] for more details.
>
>   - Fixes for segfaults that may arise due to a bug in the implementation
> of the
>     `keepAlive#` primop. This may regress performance for certain programs
> which
>     use this primop or functions which use the primop, such as
> `withForeignPtr`.
>     These regressions are mostly small, but can be larger in certain edge
> cases.
>     Judicious use of `unsafeWithForeignPtr` when its argument is known not
> to
>     statically diverge can mitigate these in many cases. It is our
> judgment that
>     the critical correctness issues justify the regression in performance
> and that
>     it is important to get a release out with the fix while we work on a
> better
>     approach which will improve performance for future releases (#21708).
>
>     We have a [wiki page](
> https://gitlab.haskell.org/ghc/ghc/-/wikis/proposal/with-combinator)
>     that tracks possible solutions to this problem, and Ben wrote a
>     [blog post](
> https://www.haskell.org/ghc/blog/20210607-the-keepAlive-story.html)
>     detailing the introduction of the `keepAlive#` primop and its history.
>
>   - Fixes for a number of miscompilations on AArch64 and other platforms
> (#21624,
>     #21773, #20735, #21685).
>
>   - Fixes for segfaults due to bugs in the RTS and GC (#21708, #21880,
> #21885).
>
>   - Fixing the behaviour of Ctrl-C with GHCi on Windows (#21889).
>
>    - ... and much more. See the [release notes] for a full accounting.
>
> As some of the fixed issues do affect correctness users are encouraged to
> upgrade promptly.
>
> We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk
> stake pool, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation,
> and other anonymous
> contributors whose on-going financial and in-kind support has
> facilitated GHC maintenance and release management over the years.
> Finally, this release would not have been possible without the hundreds
> of open-source contributors whose work comprise this release.
>
> As always, do give this release a try and open a [ticket] if you see
> anything amiss.
>
> Happy compiling,
>
> - Zubin
>
> [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new
> [release notes]:
> https://downloads.haskell.org/~ghc/9.2.4/docs/users_guide/9.2.4-notes.html
> [Simplified Subsumption Proposal]:
> https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0287-simplify-subsumption.rst
> [Deep Subsumption Proposal]:
> https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0511-deep-subsumption.rst
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220728/2820337b/attachment.html>

From kazu at iij.ad.jp  Fri Jul 29 02:10:54 2022
From: kazu at iij.ad.jp (Kazu Yamamoto (=?iso-2022-jp?B?GyRCOzNLXE9CSScbKEI=?=))
Date: Fri, 29 Jul 2022 11:10:54 +0900 (JST)
Subject: [Haskell] [ANNOUNCE] GHC 9.2.4 released
In-Reply-To: <CAB-d4A4oqJ9xNhQSapq8gcnAfX8VsU7M=ppJXeoOX2D+WrzKWQ@mail.gmail.com>
References: <20220728111713.owpnsu43zwq2klhn@zubin-msi>
 <CAB-d4A4oqJ9xNhQSapq8gcnAfX8VsU7M=ppJXeoOX2D+WrzKWQ@mail.gmail.com>
Message-ID: <20220729.111054.1976367299003263525.kazu@iij.ad.jp>

Hi,

> On a Mac it is still necessary to do
> 
> xattr -rc .
> 
> before doing
> 
> sudo make install

For 9.2.4, "xattr -rc ." is not good enough on my Mac (upgraded to
v12.5 today).  "sudo spctl --global-disable" is necessary, sigh.

--Kazu



From benjamin.redelings at gmail.com  Fri Jul 29 16:07:25 2022
From: benjamin.redelings at gmail.com (Benjamin Redelings)
Date: Fri, 29 Jul 2022 17:07:25 +0100
Subject: Advice for implementing GADTs?
In-Reply-To: <0251fb0b-b4b2-d91b-ca34-55e3b812e456@gmail.com>
References: <ea6cf444-b311-c9aa-2ccb-6e60ef6f5edd@gmail.com>
 <0251fb0b-b4b2-d91b-ca34-55e3b812e456@gmail.com>
Message-ID: <a2618533-3315-2140-bd36-aa08da9cdafa@gmail.com>

Hi,

I've been working on implementing the Haskell type system for my 
specialized Haskell interpreter.  I have now constructed a system that 
can type-check and run Haskell code that contains explicit type 
signatures, type constraints, and arbitrary-rank types.

I'm now thinking that I may need to implement GADTs -- i.e. constructors 
that introduce local constraints, including equality constraints.  I'm 
looking at the paper "OutsideIn(X): Modular type inference with local 
assumptions" from 2011.  I have three questions about implementing GADTs 
-- I'd be grateful for answers to any of them.


QUESTION 1: Are there any obviously important resources that I've 
overlooked?

The 2011 OutsideIn paper mentions several previous papers that seem 
quite helpful:

* Peyton Jones el at 2006. Simple Unification-based type inference for GADTs

* Schrijvers etal 2007. Towards open type functions for Haskell

* Peyton Jones et al 2004. Wobbly Types: etc.

* Schrijvers et al 2008. Type checking with open type functions.

* Shrijvers et al 2009. Complete and decidable type inference for GADTs

* Vytiniotis et al 2010. Let should not be generalized.

And of course the GHC source code.  (I'm not looking at coercions at the 
present time, because my type-checker translates to the plain lambda 
calculus without type annotations, not system F or F_C.  Hopefully I can 
remedy this later...)


QUESTION 2: if my quick scan is correct, none of the papers mention the 
GHC technique of determining untouchability by assigning "levels" to 
type variables.  Is there any written paper (outside the GHC sources) 
that discusses type levels?


QUESTION 3: My impression is that:

(a) type variable levels were introduced in order to clarify which 
MetaTyVars are "untouchable", but

(b) levels now also check that type variables do not escape their 
quantification scope.

(c) levels can also be used to figure out which variables are free in 
the type environment, and therefore should not be generalized over.

Does this sound right?  I suspect that I might be wrong about the last 
one...


Thanks again, and sorry for the long e-mail.

-BenRI


On 1/18/22 8:55 PM, Benjamin Redelings wrote:
>
> Hi,
>
> 1. I think I have clarified my problem a bit.  It is actually not 
> related to pattern bindings. Here's an example:
>
> h = let f c   i = if i > 10 then c else g c 'b'
>          g 'a' w = f 'b' 10
>          g z   w = z
>      in (f 'a' (1::Int), f 'a' (1.0::Double))
>
> If I am understanding the Haskell type system correctly,
>
> * the definitions of f and g form a recursive group
>
> * the monomorphism restriction is not invoked
>
> * the inner binding (to f and g) leads to a local value environment (LVE):
>
> { f :: Char -> a -> Char; g :: Char -> Char -> Char }
>
> with predicates (Num a, Ord a)
>
> 2. In this situation, Typing Haskell in Haskell suggests that we 
> should NOT apply the predicates to the environment because the type 
> for g does not contain 'a', and would become ambiguous (section 
> 11.6.2).  Instead, we should only apply predicates to the environment 
> if their type variables are present in ALL types in the current 
> declaration group.
>
> Since the predicates (Num a, and Ord a) are not retained, then we 
> cannot quantify over a.
>
> It seems like this should make `f` monomorphic in a, and thus we 
> should not be able apply (f 'a') to both (1::Int) and (1::Double).
>
> Does that make any sense?
>
> 3. GHC, however, compiles this just fine.  However, if I add "default 
> ()", then it no longer compiles.
>
> 4. On further reflection, Typing Haskell in Haskell applies defaulting 
> rules when evaluating each binding, and not just at the top level.  So 
> this might be part of where my code is going wrong.
>
> -BenRI
>
>
> On 1/15/22 11:09 AM, Benjamin Redelings wrote:
>> Hi,
>>
>> 1. I'm reading "A Static semantics for Haskell" and trying to code it 
>> up.  I came across some odd behavior with pattern bindings, and I was 
>> wondering if someone could explain or point me in the right direction.
>>
>> Suppose you have the declaration
>>
>>     (x,y) = ('a',2)
>>
>> My current code is yielding:
>>
>>     x :: Num a => Char
>>
>>     y :: Num a => a
>>
>> However, I notice that ghci gives x the type Char, with no 
>> constraint, which is what I would expect.  It also gives y the type 
>> 'Num b => b', so I don't think it is defaulting a to Int here.
>>
>> The weird results from my code stem from rule BIND-PRED in Figure 15 
>> of 
>> https://homepages.inf.ed.ac.uk/wadler/papers/staticsemantics/static-semantics.ps
>>
>>     E  |-  bind ~~> \dicts : theta -> monobinds in bind : 
>> (LIE_{encl}, theta => LVE)
>>
>> Here theta = ( Num a ) and LVE = { x :: Char, y :: a }.  So, theta => 
>> LVE is
>>
>>     { x :: Num a => Char, y :: Num a => a }
>>
>> The obvious thing to do is avoid changing a type T to Num a => T if T 
>> does not contain a.  Also I'm not totally sure if that trick gets to 
>> the bottom of the issue.  However, the paper doesn't mention define 
>> theta => LVE that way.  Is there something else I should read on this?
>>
>> 2. If we just chop out predicates which mention variables not in the 
>> type ( == ambiguous predicates?) then I'm not totally sure how to 
>> create code for this.
>>
>> I would imagine that we have something like
>>
>>     tup dn = ('a', fromInteger dn 2)
>>
>>     x = case (tup dn) of (x,y) -> x
>>
>>     y dn case (tup dn) of (x,y) -> y
>>
>> In this case its not clear where to get the `dn` argument of `tup` 
>> from, in the definition of `x`.  Can we pass in `undefined`?  Should 
>> we do something else?
>>
>> If anyone can shed light on this, I would be grateful :-)
>>
>> -BenRI
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20220729/96e1b386/attachment.html>