From ben at smart-cactus.org  Sun Jan  3 14:45:27 2021
From: ben at smart-cactus.org (Ben Gamari)
Date: Sun, 03 Jan 2021 09:45:27 -0500
Subject: ghcs-nix: An archive of GHC derivations
Message-ID: <87v9ceqdtr.fsf@smart-cactus.org>

Hi all,

As a NixOS user, I have in the past found that I need an older GHC
release that is no longer available (or not yet available) in nixpkgs.

While mpickering's old-ghc-nix [1] conveniently allows standard GHC
binary distributions to be adapted for use under NixOS, I have found
that it is less than reliable when building later projects [2].

For this reason, I have started collecting a set of nix expressions for
building GHC versions all the way back to GHC 8.4.1 and extending up to
GHC 9.0.1-rc1. In addition, there is a binary cache with pre-built
binaries (complements of Cachix) here [3]. I intend on continuing to add
new releases to this as needed.

I hope others also find this helpful.

Cheers,

- Ben


[1] https://github.com/mpickering/old-ghc-nix
[2] https://gitlab.haskell.org/ghc/ghc/-/issues/19144
[3] https://app.cachix.org/cache/ghc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210103/e06d353c/attachment.sig>

From ben at smart-cactus.org  Sun Jan  3 16:07:20 2021
From: ben at smart-cactus.org (Ben Gamari)
Date: Sun, 03 Jan 2021 11:07:20 -0500
Subject: ghcs-nix: An archive of GHC derivations
In-Reply-To: <87v9ceqdtr.fsf@smart-cactus.org>
References: <87v9ceqdtr.fsf@smart-cactus.org>
Message-ID: <87sg7iqa16.fsf@smart-cactus.org>

Ben Gamari <ben at smart-cactus.org> writes:

> Hi all,
>
Naturally, I forgot to include the most important link:

ghcs-nix can be found at

    https://gitlab.haskell.org/bgamari/ghcs-nix/

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210103/d8d929ac/attachment.sig>

From alexbiehl at gmail.com  Sun Jan  3 19:18:57 2021
From: alexbiehl at gmail.com (Alex Biehl)
Date: Sun, 3 Jan 2021 20:18:57 +0100
Subject: [ANNOUNCE] Glasgow Haskell Compiler 9.0.1-rc1 released
In-Reply-To: <878s9fs4s2.fsf@smart-cactus.org>
References: <878s9fs4s2.fsf@smart-cactus.org>
Message-ID: <CAJgVECsQxTW9PqNT__9GA=WqCs7EENZ1XZGKrnxP2hgMGGGJdA@mail.gmail.com>

Ben,

Thank you for your effort. It is greatly appreciated.

One nit though: I noticed the Haddocks for ghc and the libraries don't come
with hyperlinked source but instead with the old, deprecated HsColour
sources.
Grepping on GitLab showed some commits around switching
--hyperlinked-sources on unfortinately it doesn't



Am Mi., 30. Dez. 2020 um 16:17 Uhr schrieb Ben Gamari <ben at well-typed.com>:

>
> Hello all,
>
> The GHC team is very pleased to announce the availability of the first
> release candidate of GHC 9.0.1 series. Source and binary distributions are
> available at the usual place:
>
>     https://downloads.haskell.org/ghc/9.0.1-rc1/
>
> This release candidate comes quite a bit later than expected after
> difficulty finding a performance neutral fix for a critical soundness bug,
> #17760. See [1] for details on the solution, particularly
> if you are a library maintainer currently using the touch# primop or
> withForeignPtr.
>
> Nevertheless, this release has nevertheless seen a considerable amount of
> testing and consequently we hope that this should be the last
> pre-release before the final release.
>
> In addition to numerous bug fixes, GHC 9.0.1 will bring a number of new
> features:
>
>  * A first cut of the new LinearTypes language extension [2], allowing
>    use of linear function syntax and linear record fields.
>
>  * A new bignum library (ghc-bignum), allowing GHC to be more easily
>    used with integer libraries other than GMP.
>
>  * Improvements in code generation, resulting in considerable
>    performance improvements in some programs.
>
>  * Improvements in pattern-match checking, allowing more precise
>    detection of redundant cases and reduced compilation time.
>
>  * Implementation of the "simplified subsumption" proposal [3]
>    simplifying the type system and paving the way for QuickLook
>    impredicativity in GHC 9.2.
>
>  * Implementation of the QualifiedDo extension [4], allowing more
>    convenient overloading of `do` syntax.
>
>  * Improvements in compilation time.
>
> And many more. See the release notes [5] for a full accounting of the
> changes in this release.
>
> As always, do test this release and open tickets for whatever issues you
> encounter.
>
> Cheers,
>
> - Ben
>
> [1] https://gitlab.haskell.org/ghc/ghc/-/wikis/migration/9.0#ghc-prim-07
> [2]
> https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0111-linear-types.rst
> [3]
> https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0287-simplify-subsumption.rst
> [4]
> https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0216-qualified-do.rst
> [5]
> https://downloads.haskell.org/ghc/9.0.1-rc1/docs/html/users_guide/9.0.1-notes.html
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210103/8403bdef/attachment.html>

From david.feuer at gmail.com  Mon Jan  4 04:08:36 2021
From: david.feuer at gmail.com (David Feuer)
Date: Sun, 3 Jan 2021 23:08:36 -0500
Subject: Interruptible foreign calls and uninterruptibleMask
Message-ID: <CAMgWh9tG5_Asu9mu12UGLHVFqBDjaQRqjQ0y-6rWbuKDt2WhZA@mail.gmail.com>

As I understand it, `interruptible` foreign calls aren't affected by
the masking state at all. It seems to me that they really should
respect `uninterruptibleMask`. In particular, within the scope of
`uninterruptibleMask`, if an exception arrives and successfully
interrupts the foreign call, then the `allowInterrupt` call will not
actually deliver the exception, and the failure/EINTR loop (e.g.,
throwIfMinus1Retry) will just perform the interrupted call again. How
I think it should work is that an `interruptible` foreign call should
act exactly like a `safe` foreign call in the scope of
`uninterruptibleMask`. Does this make sense? If so, does it need a GHC
proposal?

From david.feuer at gmail.com  Mon Jan  4 06:22:54 2021
From: david.feuer at gmail.com (David Feuer)
Date: Mon, 4 Jan 2021 01:22:54 -0500
Subject: Interruptible foreign calls, part 2
Message-ID: <CAMgWh9v-9x35hPcxxaK3Z4BzGJBPvXkm06zvJsXPsRZgWX=5Dw@mail.gmail.com>

Another issue is that (as the recently expanded documentation shows),
an `interruptible` foreign call is not actually an "interruptible
operation" in the sense used in Control.Exception. To make it so, the
caller has to manually call `allowInterrupt`. Or, in the
single-threaded runtime, has to call `interruptibleYield`. So ... they
have to make a foreign call to `rts_isThreaded` and then use the
result to figure out how to deliver a potential exception. What a
terrible user experience! Why not bake the "deliver exceptions" part
into the FFI code for `interruptible` calls? This stuff has been
documented so very recently (not in any released GHC) that I don't
imagine much will break if we make these changes.

From david.feuer at gmail.com  Mon Jan  4 07:12:22 2021
From: david.feuer at gmail.com (David Feuer)
Date: Mon, 4 Jan 2021 02:12:22 -0500
Subject: Interruptible foreign calls, part 2
In-Reply-To: <CAMgWh9v-9x35hPcxxaK3Z4BzGJBPvXkm06zvJsXPsRZgWX=5Dw@mail.gmail.com>
References: <CAMgWh9v-9x35hPcxxaK3Z4BzGJBPvXkm06zvJsXPsRZgWX=5Dw@mail.gmail.com>
Message-ID: <CAMgWh9u3nGPOBnNiAJ=Vkb6RCMFy=NogEYZt9C2BW3YHLPpz6Q@mail.gmail.com>

Whoops, I meant `interruptible yield`. Sorry for the confusion.

On Mon, Jan 4, 2021, 1:22 AM David Feuer <david.feuer at gmail.com> wrote:

> Another issue is that (as the recently expanded documentation shows),
> an `interruptible` foreign call is not actually an "interruptible
> operation" in the sense used in Control.Exception. To make it so, the
> caller has to manually call `allowInterrupt`. Or, in the
> single-threaded runtime, has to call `interruptibleYield`. So ... they
> have to make a foreign call to `rts_isThreaded` and then use the
> result to figure out how to deliver a potential exception. What a
> terrible user experience! Why not bake the "deliver exceptions" part
> into the FFI code for `interruptible` calls? This stuff has been
> documented so very recently (not in any released GHC) that I don't
> imagine much will break if we make these changes.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210104/6c259fe9/attachment.html>

From lonetiger at gmail.com  Mon Jan  4 07:21:29 2021
From: lonetiger at gmail.com (Phyx)
Date: Mon, 4 Jan 2021 07:21:29 +0000
Subject: Debugging annotation machinery
Message-ID: <CABHktDURQLHSayprKDOj2FpZDvhM65qohpd=fNj8F-jo8AqOCA@mail.gmail.com>

Hi,

I am changing the hashing algorithm used to create Fingerprints but somehow
the annrun01.hs test has started failing and I don't quite understand why.

>From what I can tell the failure happens during deserialization but the
module itself is still found.
The calls findGlobalAnns deserializeWithData target seem to return nothing.

Calling ~/ghc/inplace/bin/ghc-stage2.exe --show-iface Annrun01_Help.hi
shows that GHC itself understands the module fine though.

Does someone have an idea why deserialization might be failing? but also
how does one debug this? Almost none of the structures have a Show or
Outputable instance...

Thanks,
Tamar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210104/381a5026/attachment.html>

From simonpj at microsoft.com  Mon Jan  4 21:18:00 2021
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 4 Jan 2021 21:18:00 +0000
Subject: Nominations close Jan 11th for the Haskell Foundation Board
Message-ID: <HE1PR83MB02171DA0672D096CE2BF5D35ADD29@HE1PR83MB0217.EURPRD83.prod.outlook.com>

Friends
Happy new year!
The closing date for self-nominations for membership of the Board of the Haskell Foundation<https://haskell.foundation/> is in just under a week:
Monday January 11th 2021

The Haskell Foundation is a new non-profit organisation that seeks to articulate the benefits of functional programming to a broader audience, to erase barriers to entry, and to support Haskell as a solidly reliable basis for mission-critical applications.

The Board provides the strategic leadership for the Foundation, including its goals, governance, finances, and staff.  Membership of the Board is a key leadership role, not an honorary appointment.

The Call for Nominations<https://haskell.foundation/board-nominations/> gives more details.  Please do consider nominating yourself, or encouraging a suitable (but perhaps modest) colleague to do so.  The HF needs a strong Board, and that means strong nominations.

Thanks!  Please do forward this message on social media or elsewhere.

Simon Peyton Jones



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210104/83fdb3c0/attachment.html>

From carter.schonwald at gmail.com  Mon Jan  4 23:32:36 2021
From: carter.schonwald at gmail.com (Carter Schonwald)
Date: Mon, 4 Jan 2021 18:32:36 -0500
Subject: Debugging annotation machinery
In-Reply-To: <CABHktDURQLHSayprKDOj2FpZDvhM65qohpd=fNj8F-jo8AqOCA@mail.gmail.com>
References: <CABHktDURQLHSayprKDOj2FpZDvhM65qohpd=fNj8F-jo8AqOCA@mail.gmail.com>
Message-ID: <CAHYVw0yL1_4xMJCw4MS3gysvA6g176xizG0oZcEuwYAw+UR2Fg@mail.gmail.com>

I don’t have good debugging wisdom, but I’d love to know what hash
functions you’re considering!

On Mon, Jan 4, 2021 at 2:22 AM Phyx <lonetiger at gmail.com> wrote:

> Hi,
>
> I am changing the hashing algorithm used to create Fingerprints but
> somehow the annrun01.hs test has started failing and I don't quite
> understand why.
>
> From what I can tell the failure happens during deserialization but the
> module itself is still found.
> The calls findGlobalAnns deserializeWithData target seem to return nothing.
>
> Calling ~/ghc/inplace/bin/ghc-stage2.exe --show-iface Annrun01_Help.hi
> shows that GHC itself understands the module fine though.
>
> Does someone have an idea why deserialization might be failing? but also
> how does one debug this? Almost none of the structures have a Show or
> Outputable instance...
>
> Thanks,
> Tamar
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210104/235b6646/attachment.html>

From ben at smart-cactus.org  Mon Jan  4 23:46:22 2021
From: ben at smart-cactus.org (Ben Gamari)
Date: Mon, 04 Jan 2021 18:46:22 -0500
Subject: Debugging annotation machinery
In-Reply-To: <CAHYVw0yL1_4xMJCw4MS3gysvA6g176xizG0oZcEuwYAw+UR2Fg@mail.gmail.com>
References: <CABHktDURQLHSayprKDOj2FpZDvhM65qohpd=fNj8F-jo8AqOCA@mail.gmail.com>
 <CAHYVw0yL1_4xMJCw4MS3gysvA6g176xizG0oZcEuwYAw+UR2Fg@mail.gmail.com>
Message-ID: <87mtxoqn91.fsf@smart-cactus.org>

Carter Schonwald <carter.schonwald at gmail.com> writes:

> I don’t have good debugging wisdom, but I’d love to know what hash
> functions you’re considering!
>
The relevant MR is https://gitlab.haskell.org/ghc/ghc/-/merge_requests/4248.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210104/8497b10a/attachment.sig>

From simonpj at microsoft.com  Wed Jan  6 09:34:33 2021
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Wed, 6 Jan 2021 09:34:33 +0000
Subject: Title index
Message-ID: <HE1PR83MB02170130A25C133575AA045EADD09@HE1PR83MB0217.EURPRD83.prod.outlook.com>

Ben
On GHC's wiki home page<https://gitlab.haskell.org/ghc/ghc/-/wikis/home> we have Title Index<https://gitlab.haskell.org/ghc/ghc/wikis/index> over in the right margin.
I thought the title index was auto-generated.  But it doesn't contain this page<https://gitlab.haskell.org/ghc/ghc/-/wikis/Functional-dependencies-in-GHC>.
What's up?
Thanks
Simon

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210106/b35b9969/attachment.html>

From ben at well-typed.com  Thu Jan  7 02:08:44 2021
From: ben at well-typed.com (Ben Gamari)
Date: Wed, 06 Jan 2021 21:08:44 -0500
Subject: Title index
In-Reply-To: <HE1PR83MB02170130A25C133575AA045EADD09@HE1PR83MB0217.EURPRD83.prod.outlook.com>
References: <HE1PR83MB02170130A25C133575AA045EADD09@HE1PR83MB0217.EURPRD83.prod.outlook.com>
Message-ID: <87im89qz12.fsf@smart-cactus.org>

Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> writes:

> Ben
> On GHC's wiki home
> page<https://gitlab.haskell.org/ghc/ghc/-/wikis/home> we have Title
> Index<https://gitlab.haskell.org/ghc/ghc/wikis/index> over in the
> right margin.
> I thought the title index was auto-generated. But it doesn't contain
> this
> page<https://gitlab.haskell.org/ghc/ghc/-/wikis/Functional-dependencies-in-GHC>.
> What's up?

Sigh, it looks like the cron job has been failing since... apparently
forever.

I've manually triggered a rebuild and will try to sort it out tomorrow.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210106/c71c492d/attachment.sig>

From compl.yue at icloud.com  Thu Jan  7 14:48:49 2021
From: compl.yue at icloud.com (YueCompl)
Date: Thu, 7 Jan 2021 22:48:49 +0800
Subject: How to get dwarf stack trace of all live threads (HECs) on demand
 (e.g. SIGQUIT)?
Message-ID: <F455FFAF-5E5A-4C7D-A11D-19E6DA8CDCEC@icloud.com>

Dear GHC devs,

I recently encountered 100% ~ 400% CPU hogging situations to trouble-shoot, I managed to build and use a dwarf enabled GHC-8.10.3 on Ubuntu 18.04 and 20.04 for that, then I find it seemingly dumps only the main thread's stack trace when I press Ctrl+\, but my program is a concurrent language server that will fork a new thread to handle each LSP request, so I really need to see what is going on those threads when CPU get hogged, as the situation is not trivially reproducible, only occurs occasionally after some time of active IDE activity.

And I can't figure out how to do that from the user guide, so help please!

Thanks & best regards,
Compl


From omeragacan at gmail.com  Thu Jan  7 17:42:08 2021
From: omeragacan at gmail.com (=?UTF-8?Q?=C3=96mer_Sinan_A=C4=9Facan?=)
Date: Thu, 7 Jan 2021 20:42:08 +0300
Subject: Potential improvement in compacting GC
Message-ID: <CAMQQO3=7haokj1uUUt9KoSQ2YLOd7Y3Ue20atXhW2HwL=Z+2nQ@mail.gmail.com>

Hello,

I recently implemented the algorithm used by GHC's compacting GC in another
project. The algorithm (after marking) makes two passes over the heap
/generation. In GHC, these passes are implemented in [1] and in the next
function.

In my implementation I tried 3 ways of implementing these passes, one of which
is the same as GHC's code, and benchmarked each version. I found that the
fastest implementation is not what's used in GHC, but it could easily be used.

I should mention that my code targets Wasm, and I benchmarked Wasm instructions
executed. Not CPU cycles, CPU instructions, or anything like that. It's very
likely that the results will be different when benchmarking code running on
actual hardware.

Secondly, in my case the heap is mostly dead (residency is low). In GHC,
compaction for the oldest generation is enabled when residency passes a
threshold, so the assumption is the heap is mostly live. I'm guessing this
should also make some difference.

Anyway, the first implementation I tried was similar to GHC's scan, but I did

    scan += object_size(scan);

instead of bumping scan by one, as GHC does in [2]. This was the slowest
version.

Second implementation did the same as GHC (bumped scan by one). This was
faster, but still slower than the next version.

What I found to be the best is scanning the bitmap, not the heap. The bitmap
iterator reads one word at a time. In each iteration it checks if the bitmap
word is 0. In GHC, in the best case this can skip 16 words on heap on 32-bit
systems, and 32 words on 64-bit. Reminder: we need two bits per object in the
bitmap, see [3]. (this is not the case in my implementation so the payoff is
better)

When the bitmap word is not 0 I use "count trailing zeros" (or "count leading
zeros" depending on the bitmap implementation) to get the number of words to
skip. This is a single instruction on Wasm and x86 (TZCNT or LZCNT, available
via __builtin_ctzl and __builtin_clzl in gcc).

So instead of skipping one word at a time, this can potentially skip 16 words
(or 32 on 64-bit architectures). When that's not possible, it can still skip
multiple words by using ctz/clz.

Ömer

[1]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rts/sm/Compact.c#L824-L879
[2]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rts/sm/Compact.c#L838
[3]: https://github.com/ghc/ghc/blob/bd877edd9499a351db947cd51ed583872b2facdf/rts/sm/Compact.h#L18-L55

From csaba.hruska at gmail.com  Sat Jan  9 23:30:39 2021
From: csaba.hruska at gmail.com (Csaba Hruska)
Date: Sun, 10 Jan 2021 00:30:39 +0100
Subject: presentation: Next-gen Haskell Compilation Techniques
Message-ID: <CANjChEqhbasLZk86n2RTsM=YbLQ5nAK3jF=OOX84seUYygygDA@mail.gmail.com>

Hello,
I did an online presentation about Haskell related (futuristic) compilation
techniques.
The application of these methods is also the main motivation of my work
with the grin compiler project and ghc-wpc.

video: https://www.youtube.com/watch?v=jyaR8E325ok
slides:
https://docs.google.com/presentation/d/1g_-bHgeD7lV4AYybnvjgkWa9GKuP6QFUyd26zpqXssQ/edit?usp=sharing

Regards,
Csaba
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210110/a5beca71/attachment.html>

From sgraf1337 at gmail.com  Sun Jan 10 12:58:01 2021
From: sgraf1337 at gmail.com (Sebastian Graf)
Date: Sun, 10 Jan 2021 13:58:01 +0100
Subject: presentation: Next-gen Haskell Compilation Techniques
In-Reply-To: <CANjChEqhbasLZk86n2RTsM=YbLQ5nAK3jF=OOX84seUYygygDA@mail.gmail.com>
References: <CANjChEqhbasLZk86n2RTsM=YbLQ5nAK3jF=OOX84seUYygygDA@mail.gmail.com>
Message-ID: <CAAS+=P91R_r0uqmYF7Yk9gwJiWcWQDnPxQiVBzCdh5gc3U0+ZQ@mail.gmail.com>

Hi Csaba,

Thanks for your presentation, that's a nice high-level overview of what
you're up to.

A few thoughts:

   - Whole-program optimization sounds great, but also very ambitious,
   given the amount of code GHC generates today. I'd be amazed to see advances
   in that area, though, and your >100-module CFA performance incites hope!
   - I wonder if going through GRIN results in a more efficient mapping to
   hardware. I recently found that the code GHC generates is dominated by
   administrative traffic from and to the heap [1]. I suspect that you can
   have big wins here if you manage to convey better call stack, heap and
   alias information to LLVM.
   - The Control Analysis+specialisation approach sounds pretty similar to
   doing Constructor Specialisation [2] for Lambdas (cf. 6.2) if you also
   inline the function for which you specialise afterwards. I sunk many hours
   into making that work reliably, fast and without code bloat in the past, to
   no avail. Frankly, if you can do it in GRIN, I don't see why we couldn't do
   it in Core. But maybe we can learn from the GRIN implementation afterwards
   and maybe rethink SpecConstr. Maybe the key is not to inline the function
   for which we specialise? But then you don't gain that much...
   - I follow the Counting Immutable Beans [3] stuff quite closely
   (Sebastian is a colleague of mine) and hope that it is applicable to
   Haskell some day. But I think using Perceus, like any purely RC-based
   memory management scheme, means that you can't have cycles in your heap, so
   no loopy thunks (such as constant-space `ones = 1:ones`) and mutability. I
   think that makes a pretty huge difference for many use cases. Sebastian
   also told me that they have to adapt their solutions to the cycle
   restriction from time to time, so far always successfully. But it comes at
   a cost: You have to adapt the code you want to write into a form that works.

I only read the slides, apologies if some of my points were invalidated by
something you said.

Keep up the good work!
Cheers,
Sebastian

[1] https://gitlab.haskell.org/ghc/ghc/-/issues/19113
[2]
https://www.microsoft.com/en-us/research/wp-content/uploads/2016/07/spec-constr.pdf
[3] https://arxiv.org/abs/1908.05647

Am So., 10. Jan. 2021 um 00:31 Uhr schrieb Csaba Hruska <
csaba.hruska at gmail.com>:

> Hello,
> I did an online presentation about Haskell related (futuristic)
> compilation techniques.
> The application of these methods is also the main motivation of my work
> with the grin compiler project and ghc-wpc.
>
> video: https://www.youtube.com/watch?v=jyaR8E325ok
> slides:
> https://docs.google.com/presentation/d/1g_-bHgeD7lV4AYybnvjgkWa9GKuP6QFUyd26zpqXssQ/edit?usp=sharing
>
> Regards,
> Csaba
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210110/9b80c70d/attachment.html>

From csaba.hruska at gmail.com  Mon Jan 11 12:18:37 2021
From: csaba.hruska at gmail.com (Csaba Hruska)
Date: Mon, 11 Jan 2021 13:18:37 +0100
Subject: presentation: Next-gen Haskell Compilation Techniques
In-Reply-To: <CAAS+=P91R_r0uqmYF7Yk9gwJiWcWQDnPxQiVBzCdh5gc3U0+ZQ@mail.gmail.com>
References: <CANjChEqhbasLZk86n2RTsM=YbLQ5nAK3jF=OOX84seUYygygDA@mail.gmail.com>
 <CAAS+=P91R_r0uqmYF7Yk9gwJiWcWQDnPxQiVBzCdh5gc3U0+ZQ@mail.gmail.com>
Message-ID: <CANjChEoBEaHnvL=Qf8_YwgkQQZ8j_37M5b0H_xW-Nj9L6-F-Rg@mail.gmail.com>

Hi Sebastian,

Thanks for your feedback.
I know that CIB and Perceus have issues with cycles, but these systems are
still in development so who knows what will be the conclusion.
I may not emphasize in the talk, but the goal of the grin compiler project
is to build a compiler pipeline that allows easy experimentation of
different compilation techniques. Anything between whole program
compilation to per module incremental codegen. So the whole program
compilation is not really a requirement but an option.

Cheers,
Csaba

On Sun, Jan 10, 2021 at 1:58 PM Sebastian Graf <sgraf1337 at gmail.com> wrote:

> Hi Csaba,
>
> Thanks for your presentation, that's a nice high-level overview of what
> you're up to.
>
> A few thoughts:
>
>    - Whole-program optimization sounds great, but also very ambitious,
>    given the amount of code GHC generates today. I'd be amazed to see advances
>    in that area, though, and your >100-module CFA performance incites hope!
>    - I wonder if going through GRIN results in a more efficient mapping
>    to hardware. I recently found that the code GHC generates is dominated by
>    administrative traffic from and to the heap [1]. I suspect that you can
>    have big wins here if you manage to convey better call stack, heap and
>    alias information to LLVM.
>    - The Control Analysis+specialisation approach sounds pretty similar
>    to doing Constructor Specialisation [2] for Lambdas (cf. 6.2) if you also
>    inline the function for which you specialise afterwards. I sunk many hours
>    into making that work reliably, fast and without code bloat in the past, to
>    no avail. Frankly, if you can do it in GRIN, I don't see why we couldn't do
>    it in Core. But maybe we can learn from the GRIN implementation afterwards
>    and maybe rethink SpecConstr. Maybe the key is not to inline the function
>    for which we specialise? But then you don't gain that much...
>    - I follow the Counting Immutable Beans [3] stuff quite closely
>    (Sebastian is a colleague of mine) and hope that it is applicable to
>    Haskell some day. But I think using Perceus, like any purely RC-based
>    memory management scheme, means that you can't have cycles in your heap, so
>    no loopy thunks (such as constant-space `ones = 1:ones`) and mutability. I
>    think that makes a pretty huge difference for many use cases. Sebastian
>    also told me that they have to adapt their solutions to the cycle
>    restriction from time to time, so far always successfully. But it comes at
>    a cost: You have to adapt the code you want to write into a form that works.
>
> I only read the slides, apologies if some of my points were invalidated by
> something you said.
>
> Keep up the good work!
> Cheers,
> Sebastian
>
> [1] https://gitlab.haskell.org/ghc/ghc/-/issues/19113
> [2]
> https://www.microsoft.com/en-us/research/wp-content/uploads/2016/07/spec-constr.pdf
> [3] https://arxiv.org/abs/1908.05647
>
> Am So., 10. Jan. 2021 um 00:31 Uhr schrieb Csaba Hruska <
> csaba.hruska at gmail.com>:
>
>> Hello,
>> I did an online presentation about Haskell related (futuristic)
>> compilation techniques.
>> The application of these methods is also the main motivation of my work
>> with the grin compiler project and ghc-wpc.
>>
>> video: https://www.youtube.com/watch?v=jyaR8E325ok
>> slides:
>> https://docs.google.com/presentation/d/1g_-bHgeD7lV4AYybnvjgkWa9GKuP6QFUyd26zpqXssQ/edit?usp=sharing
>>
>> Regards,
>> Csaba
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210111/f1eee803/attachment.html>

From simonpj at microsoft.com  Mon Jan 11 12:51:41 2021
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 11 Jan 2021 12:51:41 +0000
Subject: presentation: Next-gen Haskell Compilation Techniques
In-Reply-To: <CANjChEoBEaHnvL=Qf8_YwgkQQZ8j_37M5b0H_xW-Nj9L6-F-Rg@mail.gmail.com>
References: <CANjChEqhbasLZk86n2RTsM=YbLQ5nAK3jF=OOX84seUYygygDA@mail.gmail.com>
 <CAAS+=P91R_r0uqmYF7Yk9gwJiWcWQDnPxQiVBzCdh5gc3U0+ZQ@mail.gmail.com>
 <CANjChEoBEaHnvL=Qf8_YwgkQQZ8j_37M5b0H_xW-Nj9L6-F-Rg@mail.gmail.com>
Message-ID: <VI1PR83MB0222F8C34FDF5E78CB033F2FADAB9@VI1PR83MB0222.EURPRD83.prod.outlook.com>

I may not emphasize in the talk, but the goal of the grin compiler project is to build a compiler pipeline that allows easy experimentation of different compilation techniques. Anything between whole program compilation to per module incremental codegen. So the whole program compilation is not really a requirement but an option.
Right - but some optimisations absolutely require whole-program analysis, don't they?  I'm thinking of flow analyses that support defunctionalisation, when you must know all the lambdas that could be bound to `f` in the definition of `map` for example.

Such optimisations are powerful, but brittle because they are simply inapplicable without whole-program analysis.  Or maybe you can find ways to make them more resilient.

Simon

From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Csaba Hruska
Sent: 11 January 2021 12:19
To: Sebastian Graf <sgraf1337 at gmail.com>
Cc: GHC developers <ghc-devs at haskell.org>
Subject: Re: presentation: Next-gen Haskell Compilation Techniques

Hi Sebastian,

Thanks for your feedback.
I know that CIB and Perceus have issues with cycles, but these systems are still in development so who knows what will be the conclusion.
I may not emphasize in the talk, but the goal of the grin compiler project is to build a compiler pipeline that allows easy experimentation of different compilation techniques. Anything between whole program compilation to per module incremental codegen. So the whole program compilation is not really a requirement but an option.

Cheers,
Csaba

On Sun, Jan 10, 2021 at 1:58 PM Sebastian Graf <sgraf1337 at gmail.com<mailto:sgraf1337 at gmail.com>> wrote:
Hi Csaba,

Thanks for your presentation, that's a nice high-level overview of what you're up to.

A few thoughts:

  *   Whole-program optimization sounds great, but also very ambitious, given the amount of code GHC generates today. I'd be amazed to see advances in that area, though, and your >100-module CFA performance incites hope!
  *   I wonder if going through GRIN results in a more efficient mapping to hardware. I recently found that the code GHC generates is dominated by administrative traffic from and to the heap [1]. I suspect that you can have big wins here if you manage to convey better call stack, heap and alias information to LLVM.
  *   The Control Analysis+specialisation approach sounds pretty similar to doing Constructor Specialisation [2] for Lambdas (cf. 6.2) if you also inline the function for which you specialise afterwards. I sunk many hours into making that work reliably, fast and without code bloat in the past, to no avail. Frankly, if you can do it in GRIN, I don't see why we couldn't do it in Core. But maybe we can learn from the GRIN implementation afterwards and maybe rethink SpecConstr. Maybe the key is not to inline the function for which we specialise? But then you don't gain that much...
  *   I follow the Counting Immutable Beans [3] stuff quite closely (Sebastian is a colleague of mine) and hope that it is applicable to Haskell some day. But I think using Perceus, like any purely RC-based memory management scheme, means that you can't have cycles in your heap, so no loopy thunks (such as constant-space `ones = 1:ones`) and mutability. I think that makes a pretty huge difference for many use cases. Sebastian also told me that they have to adapt their solutions to the cycle restriction from time to time, so far always successfully. But it comes at a cost: You have to adapt the code you want to write into a form that works.
I only read the slides, apologies if some of my points were invalidated by something you said.

Keep up the good work!
Cheers,
Sebastian

[1] https://gitlab.haskell.org/ghc/ghc/-/issues/19113<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fissues%2F19113&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503167010%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0W9ejmKCDKpuBT0mEArvIwAmHDUS4QI9kc5j%2BhGUX5I%3D&reserved=0>
[2] https://www.microsoft.com/en-us/research/wp-content/uploads/2016/07/spec-constr.pdf<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.microsoft.com%2Fen-us%2Fresearch%2Fwp-content%2Fuploads%2F2016%2F07%2Fspec-constr.pdf&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503177005%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=8wJJXOpNuaQpjOWBTwbQ0upeOj1LLXSUD86cn8TbKI8%3D&reserved=0>
[3] https://arxiv.org/abs/1908.05647<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Farxiv.org%2Fabs%2F1908.05647&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503186998%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=LCFO8R5SZT0KveoB9GyPcMpwbhU9DTLOnYhxD%2FZNXxU%3D&reserved=0>

Am So., 10. Jan. 2021 um 00:31 Uhr schrieb Csaba Hruska <csaba.hruska at gmail.com<mailto:csaba.hruska at gmail.com>>:
Hello,
I did an online presentation about Haskell related (futuristic) compilation techniques.
The application of these methods is also the main motivation of my work with the grin compiler project and ghc-wpc.

video: https://www.youtube.com/watch?v=jyaR8E325ok<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DjyaR8E325ok&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503196994%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=dbxjJKAKqZJZ2jdYE2aR6ymQIp9awCmOHwsBAHLz9AM%3D&reserved=0>
slides: https://docs.google.com/presentation/d/1g_-bHgeD7lV4AYybnvjgkWa9GKuP6QFUyd26zpqXssQ/edit?usp=sharing<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fpresentation%2Fd%2F1g_-bHgeD7lV4AYybnvjgkWa9GKuP6QFUyd26zpqXssQ%2Fedit%3Fusp%3Dsharing&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503196994%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=UHHOI6Nr80zuDrFDPVUz6wsNXKlxY06%2B5tG%2BCxf847I%3D&reserved=0>

Regards,
Csaba
_______________________________________________
ghc-devs mailing list
ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503206993%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2BAdAJUowngbDQt7BMX43JegOeUNMfh%2B00VYYBVyTPN0%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210111/4b665615/attachment.html>

From csaba.hruska at gmail.com  Mon Jan 11 13:17:05 2021
From: csaba.hruska at gmail.com (Csaba Hruska)
Date: Mon, 11 Jan 2021 14:17:05 +0100
Subject: presentation: Next-gen Haskell Compilation Techniques
In-Reply-To: <VI1PR83MB0222F8C34FDF5E78CB033F2FADAB9@VI1PR83MB0222.EURPRD83.prod.outlook.com>
References: <CANjChEqhbasLZk86n2RTsM=YbLQ5nAK3jF=OOX84seUYygygDA@mail.gmail.com>
 <CAAS+=P91R_r0uqmYF7Yk9gwJiWcWQDnPxQiVBzCdh5gc3U0+ZQ@mail.gmail.com>
 <CANjChEoBEaHnvL=Qf8_YwgkQQZ8j_37M5b0H_xW-Nj9L6-F-Rg@mail.gmail.com>
 <VI1PR83MB0222F8C34FDF5E78CB033F2FADAB9@VI1PR83MB0222.EURPRD83.prod.outlook.com>
Message-ID: <CANjChErkOyFs0TbcH3k7rT6rzUzZhu-uZpmqyHwkYohj=jMArA@mail.gmail.com>

Sure, some require whole-program analysis. But I really do not worry about
it, because what I'd like to build is an engineering vehicle. Where a
single optimization idea could be built in several ways with different
tradeoffs. Then the sweet spot could be found after an in depth
investigation of the problem domain.
I.e. removing all indirect calls surely require whole program
defunctionalization, but a significant reduction of indirect calls could be
achieved with other techniques that does not require whole program
analysis. But it is totally valuable to compare the two approaches just to
know the tradeoffs even if only one of them is applicable in practice.

Csaba

On Mon, Jan 11, 2021 at 1:51 PM Simon Peyton Jones <simonpj at microsoft.com>
wrote:

> I may not emphasize in the talk, but the goal of the grin compiler project
> is to build a compiler pipeline that allows easy experimentation of
> different compilation techniques. Anything between whole program
> compilation to per module incremental codegen. So the whole program
> compilation is not really a requirement but an option.
>
> Right – but *some* optimisations absolutely require whole-program
> analysis, don’t they?  I’m thinking of flow analyses that support
> defunctionalisation, when you must know all the lambdas that could be bound
> to `f` in the definition of `map` for example.
>
>
>
> Such optimisations are powerful, but brittle because they are simply
> inapplicable without whole-program analysis.  Or maybe you can find ways to
> make them more resilient.
>
>
>
> Simon
>
>
>
> *From:* ghc-devs <ghc-devs-bounces at haskell.org> *On Behalf Of *Csaba
> Hruska
> *Sent:* 11 January 2021 12:19
> *To:* Sebastian Graf <sgraf1337 at gmail.com>
> *Cc:* GHC developers <ghc-devs at haskell.org>
> *Subject:* Re: presentation: Next-gen Haskell Compilation Techniques
>
>
>
> Hi Sebastian,
>
>
>
> Thanks for your feedback.
>
> I know that CIB and Perceus have issues with cycles, but these systems are
> still in development so who knows what will be the conclusion.
>
> I may not emphasize in the talk, but the goal of the grin compiler project
> is to build a compiler pipeline that allows easy experimentation of
> different compilation techniques. Anything between whole program
> compilation to per module incremental codegen. So the whole program
> compilation is not really a requirement but an option.
>
>
>
> Cheers,
>
> Csaba
>
>
>
> On Sun, Jan 10, 2021 at 1:58 PM Sebastian Graf <sgraf1337 at gmail.com>
> wrote:
>
> Hi Csaba,
>
>
>
> Thanks for your presentation, that's a nice high-level overview of what
> you're up to.
>
>
>
> A few thoughts:
>
>    - Whole-program optimization sounds great, but also very ambitious,
>    given the amount of code GHC generates today. I'd be amazed to see advances
>    in that area, though, and your >100-module CFA performance incites hope!
>    - I wonder if going through GRIN results in a more efficient mapping
>    to hardware. I recently found that the code GHC generates is dominated by
>    administrative traffic from and to the heap [1]. I suspect that you can
>    have big wins here if you manage to convey better call stack, heap and
>    alias information to LLVM.
>    - The Control Analysis+specialisation approach sounds pretty similar
>    to doing Constructor Specialisation [2] for Lambdas (cf. 6.2) if you also
>    inline the function for which you specialise afterwards. I sunk many hours
>    into making that work reliably, fast and without code bloat in the past, to
>    no avail. Frankly, if you can do it in GRIN, I don't see why we couldn't do
>    it in Core. But maybe we can learn from the GRIN implementation afterwards
>    and maybe rethink SpecConstr. Maybe the key is not to inline the function
>    for which we specialise? But then you don't gain that much...
>    - I follow the Counting Immutable Beans [3] stuff quite closely
>    (Sebastian is a colleague of mine) and hope that it is applicable to
>    Haskell some day. But I think using Perceus, like any purely RC-based
>    memory management scheme, means that you can't have cycles in your heap, so
>    no loopy thunks (such as constant-space `ones = 1:ones`) and mutability. I
>    think that makes a pretty huge difference for many use cases. Sebastian
>    also told me that they have to adapt their solutions to the cycle
>    restriction from time to time, so far always successfully. But it comes at
>    a cost: You have to adapt the code you want to write into a form that works.
>
> I only read the slides, apologies if some of my points were invalidated by
> something you said.
>
>
>
> Keep up the good work!
>
> Cheers,
>
> Sebastian
>
>
>
> [1] https://gitlab.haskell.org/ghc/ghc/-/issues/19113
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fissues%2F19113&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503167010%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0W9ejmKCDKpuBT0mEArvIwAmHDUS4QI9kc5j%2BhGUX5I%3D&reserved=0>
>
> [2]
> https://www.microsoft.com/en-us/research/wp-content/uploads/2016/07/spec-constr.pdf
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.microsoft.com%2Fen-us%2Fresearch%2Fwp-content%2Fuploads%2F2016%2F07%2Fspec-constr.pdf&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503177005%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=8wJJXOpNuaQpjOWBTwbQ0upeOj1LLXSUD86cn8TbKI8%3D&reserved=0>
>
> [3] https://arxiv.org/abs/1908.05647
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Farxiv.org%2Fabs%2F1908.05647&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503186998%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=LCFO8R5SZT0KveoB9GyPcMpwbhU9DTLOnYhxD%2FZNXxU%3D&reserved=0>
>
>
>
> Am So., 10. Jan. 2021 um 00:31 Uhr schrieb Csaba Hruska <
> csaba.hruska at gmail.com>:
>
> Hello,
>
> I did an online presentation about Haskell related (futuristic)
> compilation techniques.
>
> The application of these methods is also the main motivation of my work
> with the grin compiler project and ghc-wpc.
>
>
>
> video: https://www.youtube.com/watch?v=jyaR8E325ok
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DjyaR8E325ok&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503196994%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=dbxjJKAKqZJZ2jdYE2aR6ymQIp9awCmOHwsBAHLz9AM%3D&reserved=0>
>
> slides:
> https://docs.google.com/presentation/d/1g_-bHgeD7lV4AYybnvjgkWa9GKuP6QFUyd26zpqXssQ/edit?usp=sharing
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fpresentation%2Fd%2F1g_-bHgeD7lV4AYybnvjgkWa9GKuP6QFUyd26zpqXssQ%2Fedit%3Fusp%3Dsharing&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503196994%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=UHHOI6Nr80zuDrFDPVUz6wsNXKlxY06%2B5tG%2BCxf847I%3D&reserved=0>
>
>
>
> Regards,
>
> Csaba
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> <https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503206993%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2BAdAJUowngbDQt7BMX43JegOeUNMfh%2B00VYYBVyTPN0%3D&reserved=0>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210111/2aafaa0c/attachment.html>

From john.ericson at obsidian.systems  Mon Jan 11 14:09:03 2021
From: john.ericson at obsidian.systems (John Ericson)
Date: Mon, 11 Jan 2021 09:09:03 -0500
Subject: presentation: Next-gen Haskell Compilation Techniques
In-Reply-To: <CANjChErkOyFs0TbcH3k7rT6rzUzZhu-uZpmqyHwkYohj=jMArA@mail.gmail.com>
References: <CANjChEqhbasLZk86n2RTsM=YbLQ5nAK3jF=OOX84seUYygygDA@mail.gmail.com>
 <CAAS+=P91R_r0uqmYF7Yk9gwJiWcWQDnPxQiVBzCdh5gc3U0+ZQ@mail.gmail.com>
 <CANjChEoBEaHnvL=Qf8_YwgkQQZ8j_37M5b0H_xW-Nj9L6-F-Rg@mail.gmail.com>
 <VI1PR83MB0222F8C34FDF5E78CB033F2FADAB9@VI1PR83MB0222.EURPRD83.prod.outlook.com>
 <CANjChErkOyFs0TbcH3k7rT6rzUzZhu-uZpmqyHwkYohj=jMArA@mail.gmail.com>
Message-ID: <1e885c94-a712-754b-aa40-f1c3e9b6b348@obsidian.systems>

Great presentation, Csaba!

I definitely strongly agree in broad terms that these are the overlooked 
big questions we should be asking.

On the is last point, not only can we find a middle ground we like best, 
we can also do the full spectrum. I know something that many people I've 
talked to would like is a "compilation server" of sorts that just keeps 
on optimizing, building up a bigger database of knowledge of spitting 
out better binaries the longer you keep it running. If I understand 
correctly, a datalog-based method with very good properties re 
incrementality and monotonicity should be the perfect architecture for this.

Cheers,

John

On 1/11/21 8:17 AM, Csaba Hruska wrote:
> Sure, some require whole-program analysis. But I really do not worry 
> about it, because what I'd like to build is an engineering vehicle. 
> Where a single optimization idea could be built in several ways with 
> different tradeoffs. Then the sweet spot could be found after an in 
> depth investigation of the problem domain.
> I.e. removing all indirect calls surely require whole program 
> defunctionalization, but a significant reduction of indirect calls 
> could be achieved with other techniques that does not require whole 
> program analysis. But it is totally valuable to compare the two 
> approaches just to know the tradeoffs even if only one of them is 
> applicable in practice.
>
> Csaba
>
> On Mon, Jan 11, 2021 at 1:51 PM Simon Peyton Jones 
> <simonpj at microsoft.com <mailto:simonpj at microsoft.com>> wrote:
>
>     I may not emphasize in the talk, but the goal of the grin compiler
>     project is to build a compiler pipeline that allows easy
>     experimentation of different compilation techniques. Anything
>     between whole program compilation to per module incremental
>     codegen. So the whole program compilation is not really a
>     requirement but an option.
>
>     Right – but /some/ optimisations absolutely require whole-program
>     analysis, don’t they?  I’m thinking of flow analyses that support
>     defunctionalisation, when you must know all the lambdas that could
>     be bound to `f` in the definition of `map` for example.
>
>     Such optimisations are powerful, but brittle because they are
>     simply inapplicable without whole-program analysis.  Or maybe you
>     can find ways to make them more resilient.
>
>     Simon
>
>     *From:*ghc-devs <ghc-devs-bounces at haskell.org
>     <mailto:ghc-devs-bounces at haskell.org>> *On Behalf Of *Csaba Hruska
>     *Sent:* 11 January 2021 12:19
>     *To:* Sebastian Graf <sgraf1337 at gmail.com
>     <mailto:sgraf1337 at gmail.com>>
>     *Cc:* GHC developers <ghc-devs at haskell.org
>     <mailto:ghc-devs at haskell.org>>
>     *Subject:* Re: presentation: Next-gen Haskell Compilation Techniques
>
>     Hi Sebastian,
>
>     Thanks for your feedback.
>
>     I know that CIB and Perceus have issues with cycles, but these
>     systems are still in development so who knows what will be the
>     conclusion.
>
>     I may not emphasize in the talk, but the goal of the grin compiler
>     project is to build a compiler pipeline that allows easy
>     experimentation of different compilation techniques. Anything
>     between whole program compilation to per module incremental
>     codegen. So the whole program compilation is not really a
>     requirement but an option.
>
>     Cheers,
>
>     Csaba
>
>     On Sun, Jan 10, 2021 at 1:58 PM Sebastian Graf
>     <sgraf1337 at gmail.com <mailto:sgraf1337 at gmail.com>> wrote:
>
>         Hi Csaba,
>
>         Thanks for your presentation, that's a nice high-level
>         overview of what you're up to.
>
>         A few thoughts:
>
>           * Whole-program optimization sounds great, but also very
>             ambitious, given the amount of code GHC generates today.
>             I'd be amazed to see advances in that area, though, and
>             your >100-module CFA performance incites hope!
>           * I wonder if going through GRIN results in a more efficient
>             mapping to hardware. I recently found that the code GHC
>             generates is dominated by administrative traffic from and
>             to the heap [1]. I suspect that you can have big wins here
>             if you manage to convey better call stack, heap and alias
>             information to LLVM.
>           * The Control Analysis+specialisation approach sounds pretty
>             similar to doing Constructor Specialisation [2] for
>             Lambdas (cf. 6.2) if you also inline the function for
>             which you specialise afterwards. I sunk many hours into
>             making that work reliably, fast and without code bloat in
>             the past, to no avail. Frankly, if you can do it in GRIN,
>             I don't see why we couldn't do it in Core. But maybe we
>             can learn from the GRIN implementation afterwards and
>             maybe rethink SpecConstr. Maybe the key is not to inline
>             the function for which we specialise? But then you don't
>             gain that much...
>           * I follow the Counting Immutable Beans [3] stuff quite
>             closely (Sebastian is a colleague of mine) and hope that
>             it is applicable to Haskell some day. But I think using
>             Perceus, like any purely RC-based memory management
>             scheme, means that you can't have cycles in your heap, so
>             no loopy thunks (such as constant-space `ones = 1:ones`)
>             and mutability. I think that makes a pretty huge
>             difference for many use cases. Sebastian also told me that
>             they have to adapt their solutions to the cycle
>             restriction from time to time, so far always successfully.
>             But it comes at a cost: You have to adapt the code you
>             want to write into a form that works.
>
>         I only read the slides, apologies if some of my points were
>         invalidated by something you said.
>
>         Keep up the good work!
>
>         Cheers,
>
>         Sebastian
>
>         [1] https://gitlab.haskell.org/ghc/ghc/-/issues/19113
>         <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fissues%2F19113&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503167010%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0W9ejmKCDKpuBT0mEArvIwAmHDUS4QI9kc5j%2BhGUX5I%3D&reserved=0>
>
>         [2]
>         https://www.microsoft.com/en-us/research/wp-content/uploads/2016/07/spec-constr.pdf
>         <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.microsoft.com%2Fen-us%2Fresearch%2Fwp-content%2Fuploads%2F2016%2F07%2Fspec-constr.pdf&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503177005%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=8wJJXOpNuaQpjOWBTwbQ0upeOj1LLXSUD86cn8TbKI8%3D&reserved=0>
>
>         [3] https://arxiv.org/abs/1908.05647
>         <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Farxiv.org%2Fabs%2F1908.05647&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503186998%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=LCFO8R5SZT0KveoB9GyPcMpwbhU9DTLOnYhxD%2FZNXxU%3D&reserved=0>
>
>         Am So., 10. Jan. 2021 um 00:31 Uhr schrieb Csaba Hruska
>         <csaba.hruska at gmail.com <mailto:csaba.hruska at gmail.com>>:
>
>             Hello,
>
>             I did an online presentation about Haskell related
>             (futuristic) compilation techniques.
>
>             The application of these methods is also the main
>             motivation of my work with the grin compiler project and
>             ghc-wpc.
>
>             video: https://www.youtube.com/watch?v=jyaR8E325ok
>             <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DjyaR8E325ok&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503196994%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=dbxjJKAKqZJZ2jdYE2aR6ymQIp9awCmOHwsBAHLz9AM%3D&reserved=0>
>
>             slides:
>             https://docs.google.com/presentation/d/1g_-bHgeD7lV4AYybnvjgkWa9GKuP6QFUyd26zpqXssQ/edit?usp=sharing
>             <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fpresentation%2Fd%2F1g_-bHgeD7lV4AYybnvjgkWa9GKuP6QFUyd26zpqXssQ%2Fedit%3Fusp%3Dsharing&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503196994%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=UHHOI6Nr80zuDrFDPVUz6wsNXKlxY06%2B5tG%2BCxf847I%3D&reserved=0>
>
>             Regards,
>
>             Csaba
>
>             _______________________________________________
>             ghc-devs mailing list
>             ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
>             http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>             <https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=04%7C01%7Csimonpj%40microsoft.com%7C0a889669aa404e4b938008d8b62b23f0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637459644503206993%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2BAdAJUowngbDQt7BMX43JegOeUNMfh%2B00VYYBVyTPN0%3D&reserved=0>
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210111/20bdb0bf/attachment-0001.html>

From ben at well-typed.com  Tue Jan 19 15:43:55 2021
From: ben at well-typed.com (Ben Gamari)
Date: Tue, 19 Jan 2021 10:43:55 -0500
Subject: Large Haddock submodule merge landing soon
Message-ID: <87pn210w5n.fsf@smart-cactus.org>


Hi all,

I have a very large Haddock change (merging Haddock's development
branch, `ghc-8.10`, into the `ghc-head` branch) pending in !4819. I
will try merge it as soon as Marge's next batch finishes.

If you have a Haddock change outstanding you will need to perform a
rather significant rebase after this change goes in. However, I am
available to help with rebase if needed; just send me the MR number of
your merge request and I can take of the rest.

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210119/1263227e/attachment.sig>

From stefansf at linux.ibm.com  Tue Jan 19 16:34:03 2021
From: stefansf at linux.ibm.com (Stefan Schulze Frielinghaus)
Date: Tue, 19 Jan 2021 17:34:03 +0100
Subject: Storage layout of integral types
Message-ID: <20210119163403.GA256775@localhost.localdomain>

Hi all,

I'm wondering what the supposed storage layout of integral types is.  In
particular for integral types with size less than the size of a word.  For
example, on a 64bit machine is a 32bit integer supposed to be written as a
whole word and therefore as 64 bits or just as 32bits in the payload of a
closure?

I'm asking because since commit be5d74ca I see differently aligned integers in
the payload of a closure on a 64bit big-endian machine.  For example, in the
following code an Int32 object is created which contains the actual integer in
the high part of the payload (the snippet comes from the add operator
GHC.Int.$fNumInt32_$c+_entry):

    Hp = Hp + 16;
    ...
    I64[Hp - 8] = GHC.Int.I32#_con_info;
    I32[Hp] = _scz7::I32;

whereas e.g. in function rts_getInt32 the opposite is assumed and the actual
integer is expected in the low part of the payload:

    HsInt32
    rts_getInt32 (HaskellObj p)
    {
        // See comment above:
        // ASSERT(p->header.info == I32zh_con_info ||
        //        p->header.info == I32zh_static_info);
        return (HsInt32)(HsInt)(UNTAG_CLOSURE(p)->payload[0]);
    }

The same seems to be the case for the interpreter and foreign calls (case
bci_CCALL) where integral arguments are passed in the low part of a whole word.

Currently, my intuition is that the payload of a closure for an integral type
with size smaller than WordSize is written as a whole word where the subword is
aligned according to the machines endianness.  Can someone confirm this?  If
that is indeed true, then rts_getInt32 seems to be correct but not the former.
Otherwise the converse seems to be the case.

Cheers,
Stefan

From alan.zimm at gmail.com  Tue Jan 19 21:35:09 2021
From: alan.zimm at gmail.com (Alan & Kim Zimmerman)
Date: Tue, 19 Jan 2021 21:35:09 +0000
Subject: Lexing / Parsing and final token
Message-ID: <CANma=H9Bc+Rva4gnO8gbm52zP0D5inD5jfuM2i_+D0ftUaLFvA@mail.gmail.com>

I am (still) working on !2418 to bring the API Annotations into the GHC
ParsedSource, and making good progress.

I am currently making a rough port of ghc-exactprint, to ensure I can get
all the tests around modifying the AST to work.

One of the last pieces is being able to capture the spacing from the last
token in the file to the EOF.  I guess technically it is the second last
token.

Empirically (calling getTokenStream), it seems this is always ITsemi.  I am
not sure how this comes about, as the `module` parsing rule in Parser.y
ends with body or body2, and those both finish with an actual or virtual
'}'.

Can I rely on the token before ITEof always being ITsemi?

Alan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210119/b297cc99/attachment.html>

From rae at richarde.dev  Tue Jan 19 21:50:17 2021
From: rae at richarde.dev (Richard Eisenberg)
Date: Tue, 19 Jan 2021 21:50:17 +0000
Subject: Lexing / Parsing and final token
In-Reply-To: <CANma=H9Bc+Rva4gnO8gbm52zP0D5inD5jfuM2i_+D0ftUaLFvA@mail.gmail.com>
References: <CANma=H9Bc+Rva4gnO8gbm52zP0D5inD5jfuM2i_+D0ftUaLFvA@mail.gmail.com>
Message-ID: <010f01771ca08201-d7d809f8-0c25-448c-9085-863ee9ef24a0-000000@us-east-2.amazonses.com>

That's bizarre. Does it still happen with explicit braces?

Just to test, I tried

module Bug where {
x = 5;
y = 6;
};

and GHC rejected because of the trailing ;.

Richard

> On Jan 19, 2021, at 4:35 PM, Alan & Kim Zimmerman <alan.zimm at gmail.com> wrote:
> 
> I am (still) working on !2418 to bring the API Annotations into the GHC ParsedSource, and making good progress.
> 
> I am currently making a rough port of ghc-exactprint, to ensure I can get all the tests around modifying the AST to work.
> 
> One of the last pieces is being able to capture the spacing from the last token in the file to the EOF.  I guess technically it is the second last token.
> 
> Empirically (calling getTokenStream), it seems this is always ITsemi.  I am not sure how this comes about, as the `module` parsing rule in Parser.y ends with body or body2, and those both finish with an actual or virtual '}'.
> 
> Can I rely on the token before ITEof always being ITsemi?
> 
> Alan
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From alan.zimm at gmail.com  Tue Jan 19 21:58:01 2021
From: alan.zimm at gmail.com (Alan & Kim Zimmerman)
Date: Tue, 19 Jan 2021 21:58:01 +0000
Subject: Lexing / Parsing and final token
In-Reply-To: <010f01771ca08201-d7d809f8-0c25-448c-9085-863ee9ef24a0-000000@us-east-2.amazonses.com>
References: <CANma=H9Bc+Rva4gnO8gbm52zP0D5inD5jfuM2i_+D0ftUaLFvA@mail.gmail.com>
 <010f01771ca08201-d7d809f8-0c25-448c-9085-863ee9ef24a0-000000@us-east-2.amazonses.com>
Message-ID: <CANma=H_=3HxMken849tvWBkV_8foxQChN5wwnCWp60ODePeFEg@mail.gmail.com>

Changing it to remove the final ';' gives a last token of ITccurly.

Changing it to

module Bug where
x = 5
y = 6

Gives a last token of ITsemi.

Alan

On Tue, 19 Jan 2021 at 21:50, Richard Eisenberg <rae at richarde.dev> wrote:

> That's bizarre. Does it still happen with explicit braces?
>
> Just to test, I tried
>
> module Bug where {
> x = 5;
> y = 6;
> };
>
> and GHC rejected because of the trailing ;.
>
> Richard
>
> > On Jan 19, 2021, at 4:35 PM, Alan & Kim Zimmerman <alan.zimm at gmail.com>
> wrote:
> >
> > I am (still) working on !2418 to bring the API Annotations into the GHC
> ParsedSource, and making good progress.
> >
> > I am currently making a rough port of ghc-exactprint, to ensure I can
> get all the tests around modifying the AST to work.
> >
> > One of the last pieces is being able to capture the spacing from the
> last token in the file to the EOF.  I guess technically it is the second
> last token.
> >
> > Empirically (calling getTokenStream), it seems this is always ITsemi.  I
> am not sure how this comes about, as the `module` parsing rule in Parser.y
> ends with body or body2, and those both finish with an actual or virtual
> '}'.
> >
> > Can I rely on the token before ITEof always being ITsemi?
> >
> > Alan
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210119/2caa9d28/attachment.html>

From rae at richarde.dev  Tue Jan 19 21:59:32 2021
From: rae at richarde.dev (Richard Eisenberg)
Date: Tue, 19 Jan 2021 21:59:32 +0000
Subject: Lexing / Parsing and final token
In-Reply-To: <CANma=H_=3HxMken849tvWBkV_8foxQChN5wwnCWp60ODePeFEg@mail.gmail.com>
References: <CANma=H9Bc+Rva4gnO8gbm52zP0D5inD5jfuM2i_+D0ftUaLFvA@mail.gmail.com>
 <010f01771ca08201-d7d809f8-0c25-448c-9085-863ee9ef24a0-000000@us-east-2.amazonses.com>
 <CANma=H_=3HxMken849tvWBkV_8foxQChN5wwnCWp60ODePeFEg@mail.gmail.com>
Message-ID: <010f01771ca8fa86-6134b133-2aa2-4bf4-aefd-542cbe03c04c-000000@us-east-2.amazonses.com>

So, I think there's your answer: the last token might be ITccurly, not ITsemi. It seems that the "insert invisible curlies and semis" is taken more literally for semis than for curlies.

Richard

> On Jan 19, 2021, at 4:58 PM, Alan & Kim Zimmerman <alan.zimm at gmail.com> wrote:
> 
> Changing it to remove the final ';' gives a last token of ITccurly.
> 
> Changing it to
> 
> module Bug where
> x = 5
> y = 6
> 
> Gives a last token of ITsemi.
> 
> Alan
> 
> On Tue, 19 Jan 2021 at 21:50, Richard Eisenberg <rae at richarde.dev <mailto:rae at richarde.dev>> wrote:
> That's bizarre. Does it still happen with explicit braces?
> 
> Just to test, I tried
> 
> module Bug where {
> x = 5;
> y = 6;
> };
> 
> and GHC rejected because of the trailing ;.
> 
> Richard
> 
> > On Jan 19, 2021, at 4:35 PM, Alan & Kim Zimmerman <alan.zimm at gmail.com <mailto:alan.zimm at gmail.com>> wrote:
> > 
> > I am (still) working on !2418 to bring the API Annotations into the GHC ParsedSource, and making good progress.
> > 
> > I am currently making a rough port of ghc-exactprint, to ensure I can get all the tests around modifying the AST to work.
> > 
> > One of the last pieces is being able to capture the spacing from the last token in the file to the EOF.  I guess technically it is the second last token.
> > 
> > Empirically (calling getTokenStream), it seems this is always ITsemi.  I am not sure how this comes about, as the `module` parsing rule in Parser.y ends with body or body2, and those both finish with an actual or virtual '}'.
> > 
> > Can I rely on the token before ITEof always being ITsemi?
> > 
> > Alan
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org <mailto:ghc-devs at haskell.org>
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs <http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs>
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210119/727bd490/attachment.html>

From alan.zimm at gmail.com  Tue Jan 19 22:04:21 2021
From: alan.zimm at gmail.com (Alan & Kim Zimmerman)
Date: Tue, 19 Jan 2021 22:04:21 +0000
Subject: Lexing / Parsing and final token
In-Reply-To: <010f01771ca8fa86-6134b133-2aa2-4bf4-aefd-542cbe03c04c-000000@us-east-2.amazonses.com>
References: <CANma=H9Bc+Rva4gnO8gbm52zP0D5inD5jfuM2i_+D0ftUaLFvA@mail.gmail.com>
 <010f01771ca08201-d7d809f8-0c25-448c-9085-863ee9ef24a0-000000@us-east-2.amazonses.com>
 <CANma=H_=3HxMken849tvWBkV_8foxQChN5wwnCWp60ODePeFEg@mail.gmail.com>
 <010f01771ca8fa86-6134b133-2aa2-4bf4-aefd-542cbe03c04c-000000@us-east-2.amazonses.com>
Message-ID: <CANma=H8LcKApdB4ahfwiLZmh9jfstF7NdpnPofwwTTa-7ZfNQQ@mail.gmail.com>

And if there is a comment after the '}' and then more blank lines, the last
token is a comment.

If no curlies, it is a ITsemi for the last location, after the comment.

So my hacky scheme of using ITsemi as the means to track the last gap is
not viable.

And I don't want to put extra housekeeping on every token to track two
tokens back, not just one. Back to the drawing board.

Thanks
  Alan


On Tue, 19 Jan 2021 at 21:59, Richard Eisenberg <rae at richarde.dev> wrote:

> So, I think there's your answer: the last token might be ITccurly, not
> ITsemi. It seems that the "insert invisible curlies and semis" is taken
> more literally for semis than for curlies.
>
> Richard
>
> On Jan 19, 2021, at 4:58 PM, Alan & Kim Zimmerman <alan.zimm at gmail.com>
> wrote:
>
> Changing it to remove the final ';' gives a last token of ITccurly.
>
> Changing it to
>
> module Bug where
> x = 5
> y = 6
>
> Gives a last token of ITsemi.
>
> Alan
>
> On Tue, 19 Jan 2021 at 21:50, Richard Eisenberg <rae at richarde.dev> wrote:
>
>> That's bizarre. Does it still happen with explicit braces?
>>
>> Just to test, I tried
>>
>> module Bug where {
>> x = 5;
>> y = 6;
>> };
>>
>> and GHC rejected because of the trailing ;.
>>
>> Richard
>>
>> > On Jan 19, 2021, at 4:35 PM, Alan & Kim Zimmerman <alan.zimm at gmail.com>
>> wrote:
>> >
>> > I am (still) working on !2418 to bring the API Annotations into the GHC
>> ParsedSource, and making good progress.
>> >
>> > I am currently making a rough port of ghc-exactprint, to ensure I can
>> get all the tests around modifying the AST to work.
>> >
>> > One of the last pieces is being able to capture the spacing from the
>> last token in the file to the EOF.  I guess technically it is the second
>> last token.
>> >
>> > Empirically (calling getTokenStream), it seems this is always ITsemi.
>> I am not sure how this comes about, as the `module` parsing rule in
>> Parser.y ends with body or body2, and those both finish with an actual or
>> virtual '}'.
>> >
>> > Can I rely on the token before ITEof always being ITsemi?
>> >
>> > Alan
>> > _______________________________________________
>> > ghc-devs mailing list
>> > ghc-devs at haskell.org
>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210119/f2a22082/attachment.html>

From alan.zimm at gmail.com  Tue Jan 19 23:03:18 2021
From: alan.zimm at gmail.com (Alan & Kim Zimmerman)
Date: Tue, 19 Jan 2021 23:03:18 +0000
Subject: Lexing / Parsing and final token
In-Reply-To: <CANma=H8LcKApdB4ahfwiLZmh9jfstF7NdpnPofwwTTa-7ZfNQQ@mail.gmail.com>
References: <CANma=H9Bc+Rva4gnO8gbm52zP0D5inD5jfuM2i_+D0ftUaLFvA@mail.gmail.com>
 <010f01771ca08201-d7d809f8-0c25-448c-9085-863ee9ef24a0-000000@us-east-2.amazonses.com>
 <CANma=H_=3HxMken849tvWBkV_8foxQChN5wwnCWp60ODePeFEg@mail.gmail.com>
 <010f01771ca8fa86-6134b133-2aa2-4bf4-aefd-542cbe03c04c-000000@us-east-2.amazonses.com>
 <CANma=H8LcKApdB4ahfwiLZmh9jfstF7NdpnPofwwTTa-7ZfNQQ@mail.gmail.com>
Message-ID: <CANma=H9wWHMrbKs457H3qEJ6Y2xChydHZkoycew-LuJFHaTnvw@mail.gmail.com>

FYI I did the horrible thing for now, optimisations welcome.

The change is at [1]

Alan

[1]
https://gitlab.haskell.org/ghc/ghc/-/commit/742273a94c187f51e3b143f9c206c42024486ecf?merge_request_iid=2418

On Tue, 19 Jan 2021 at 22:04, Alan & Kim Zimmerman <alan.zimm at gmail.com>
wrote:

> And if there is a comment after the '}' and then more blank lines, the
> last token is a comment.
>
> If no curlies, it is a ITsemi for the last location, after the comment.
>
> So my hacky scheme of using ITsemi as the means to track the last gap is
> not viable.
>
> And I don't want to put extra housekeeping on every token to track two
> tokens back, not just one. Back to the drawing board.
>
> Thanks
>   Alan
>
>
> On Tue, 19 Jan 2021 at 21:59, Richard Eisenberg <rae at richarde.dev> wrote:
>
>> So, I think there's your answer: the last token might be ITccurly, not
>> ITsemi. It seems that the "insert invisible curlies and semis" is taken
>> more literally for semis than for curlies.
>>
>> Richard
>>
>> On Jan 19, 2021, at 4:58 PM, Alan & Kim Zimmerman <alan.zimm at gmail.com>
>> wrote:
>>
>> Changing it to remove the final ';' gives a last token of ITccurly.
>>
>> Changing it to
>>
>> module Bug where
>> x = 5
>> y = 6
>>
>> Gives a last token of ITsemi.
>>
>> Alan
>>
>> On Tue, 19 Jan 2021 at 21:50, Richard Eisenberg <rae at richarde.dev> wrote:
>>
>>> That's bizarre. Does it still happen with explicit braces?
>>>
>>> Just to test, I tried
>>>
>>> module Bug where {
>>> x = 5;
>>> y = 6;
>>> };
>>>
>>> and GHC rejected because of the trailing ;.
>>>
>>> Richard
>>>
>>> > On Jan 19, 2021, at 4:35 PM, Alan & Kim Zimmerman <alan.zimm at gmail.com>
>>> wrote:
>>> >
>>> > I am (still) working on !2418 to bring the API Annotations into the
>>> GHC ParsedSource, and making good progress.
>>> >
>>> > I am currently making a rough port of ghc-exactprint, to ensure I can
>>> get all the tests around modifying the AST to work.
>>> >
>>> > One of the last pieces is being able to capture the spacing from the
>>> last token in the file to the EOF.  I guess technically it is the second
>>> last token.
>>> >
>>> > Empirically (calling getTokenStream), it seems this is always ITsemi.
>>> I am not sure how this comes about, as the `module` parsing rule in
>>> Parser.y ends with body or body2, and those both finish with an actual or
>>> virtual '}'.
>>> >
>>> > Can I rely on the token before ITEof always being ITsemi?
>>> >
>>> > Alan
>>> > _______________________________________________
>>> > ghc-devs mailing list
>>> > ghc-devs at haskell.org
>>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210119/e126bb32/attachment.html>

From klebinger.andreas at gmx.at  Wed Jan 20 14:27:59 2021
From: klebinger.andreas at gmx.at (Andreas Klebinger)
Date: Wed, 20 Jan 2021 15:27:59 +0100
Subject: Benchmarking experiences: Cabal test vs compiling
 nofib/spectral/simple/Main.hs
Message-ID: <3301124e-2c4d-3c4a-cc68-7ffe7c337b80@gmx.at>

Hello Devs,

When I started to work on GHC a few years back the Wiki recommended
using nofib/spectral/simple/Main.hs as
a test case for compiler performance changes. I've been using this ever
since.

"Recently" the cabal-test (compiling cabal-the-library) has become sort
of a default benchmark for GHC performance.
I've used the Cabal test as well and it's probably a better test case
than nofib/spectral/simple/Main.hs.
I've started using both usually using spectral/simple to benchmark
intermediate changes and then looking
at the cabal test for the final patch at the end. So far I have rarely
seen a large
difference between using cabal or spectral/simple. Sometimes the
magnitude of the effect was different
between the two, but I've never seen one regress/improve while the other
didn't.

Since the topic came up recently in a discussion I wonder if others use
similar means to quickly bench ghc changes
and what your experiences were in terms of simpler benchmarks being
representative compared to the cabal test.

Cheers,
Andreas

From sgraf1337 at gmail.com  Sat Jan 23 11:51:51 2021
From: sgraf1337 at gmail.com (Sebastian Graf)
Date: Sat, 23 Jan 2021 12:51:51 +0100
Subject: Benchmarking experiences: Cabal test vs compiling
 nofib/spectral/simple/Main.hs
In-Reply-To: <3301124e-2c4d-3c4a-cc68-7ffe7c337b80@gmx.at>
References: <3301124e-2c4d-3c4a-cc68-7ffe7c337b80@gmx.at>
Message-ID: <CAAS+=P9HsqQ92a848bLbKUGYonHQPe5ASqC0cJSewbJBDyUhhA@mail.gmail.com>

Hi Andreas,

I similarly benchmark compiler performance by compiling Cabal, but only
occasionally. I mostly trust ghc/alloc metrics in CI and check Cabal when I
think there's something afoot and/or want to measure runtime, not only
allocations.

I'm inclined to think that for my purposes (testing the impact of
optimisations) the GHC codebase offers sufficient variety to turn up
fundamental regressions, but maybe it makes sense to build some packages
from head.hackage to detect regressions like
https://gitlab.haskell.org/ghc/ghc/-/issues/19203 earlier. It's all a bit
open-ended and I frankly think I wouldn't get done anything if all my
patches would have to get to the bottom of all regressions and improvements
on the entire head.hackage set. I somewhat trust that users will complain
eventually and file a bug report and that our CI efforts mean that compiler
performance will improve in the mean.

Although it's probably more of a tooling problem: I simply don't know how
to collect the compiler performance metrics for arbitrary cabal packages.
If these metrics would be collected as part of CI, maybe as a nightly or
weekly job, it would be easier to get to the bottom of a regression before
it manifests in a released GHC version. But it all depends on how easy that
would be to set up and how many CI cycles it would burn, and I certainly
don't feel like I'm in a position to answer either question.

Cheers,
Sebastian


Am Mi., 20. Jan. 2021 um 15:28 Uhr schrieb Andreas Klebinger <
klebinger.andreas at gmx.at>:

> Hello Devs,
>
> When I started to work on GHC a few years back the Wiki recommended
> using nofib/spectral/simple/Main.hs as
> a test case for compiler performance changes. I've been using this ever
> since.
>
> "Recently" the cabal-test (compiling cabal-the-library) has become sort
> of a default benchmark for GHC performance.
> I've used the Cabal test as well and it's probably a better test case
> than nofib/spectral/simple/Main.hs.
> I've started using both usually using spectral/simple to benchmark
> intermediate changes and then looking
> at the cabal test for the final patch at the end. So far I have rarely
> seen a large
> difference between using cabal or spectral/simple. Sometimes the
> magnitude of the effect was different
> between the two, but I've never seen one regress/improve while the other
> didn't.
>
> Since the topic came up recently in a discussion I wonder if others use
> similar means to quickly bench ghc changes
> and what your experiences were in terms of simpler benchmarks being
> representative compared to the cabal test.
>
> Cheers,
> Andreas
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210123/e26831cb/attachment.html>

From ben at smart-cactus.org  Sat Jan 23 16:36:34 2021
From: ben at smart-cactus.org (Ben Gamari)
Date: Sat, 23 Jan 2021 11:36:34 -0500
Subject: Benchmarking experiences: Cabal test vs compiling
 nofib/spectral/simple/Main.hs
In-Reply-To: <CAAS+=P9HsqQ92a848bLbKUGYonHQPe5ASqC0cJSewbJBDyUhhA@mail.gmail.com>
References: <3301124e-2c4d-3c4a-cc68-7ffe7c337b80@gmx.at>
 <CAAS+=P9HsqQ92a848bLbKUGYonHQPe5ASqC0cJSewbJBDyUhhA@mail.gmail.com>
Message-ID: <87mtwz1ugg.fsf@smart-cactus.org>

Sebastian Graf <sgraf1337 at gmail.com> writes:

> Hi Andreas,
>
> I similarly benchmark compiler performance by compiling Cabal, but only
> occasionally. I mostly trust ghc/alloc metrics in CI and check Cabal when I
> think there's something afoot and/or want to measure runtime, not only
> allocations.
>
I think this is a very reasonable strategy. When working explicitly on
compiler performance I generally default to the Cabal test as

 1. I find the 20 or 90 seconds (depending upon optimisation level) that
    it takes is small relative to the time it took to actually find the
    issue I am trying to fix, and

 2. I want to be certain I am not sacrificing compiler performance in
    one case in exchange for improvements elsewhere; the nofib tests are so
    small that I find it hard to convince myself that this is the case.

> I'm inclined to think that for my purposes (testing the impact of
> optimisations) the GHC codebase offers sufficient variety to turn up
> fundamental regressions, but maybe it makes sense to build some packages
> from head.hackage to detect regressions like
> https://gitlab.haskell.org/ghc/ghc/-/issues/19203 earlier. It's all a bit
> open-ended and I frankly think I wouldn't get done anything if all my
> patches would have to get to the bottom of all regressions and improvements
> on the entire head.hackage set. I somewhat trust that users will complain
> eventually and file a bug report and that our CI efforts mean that compiler
> performance will improve in the mean.
>
> Although it's probably more of a tooling problem: I simply don't know how
> to collect the compiler performance metrics for arbitrary cabal packages.
> If these metrics would be collected as part of CI, maybe as a nightly or
> weekly job, it would be easier to get to the bottom of a regression before
> it manifests in a released GHC version. But it all depends on how easy that
> would be to set up and how many CI cycles it would burn, and I certainly
> don't feel like I'm in a position to answer either question.
>
We actually already do this in head.hackage: every GHC commit on
`master` runs `head.hackage` with -ddump-timings. The compiler metrics
that result are then dumped into a database, which can be queried via
Postgrest. IIRC, I described this in an email to ghc-devs a few months
ago.

Unfortunately, Ryan and I have thusfar found it very difficult to
keep head.hackage and the associated infrastructure building reliably
enough to make this a useful long-term metric. I do hope we can do
better in the future; I suspect we will want to be better about marking
MRs that may break user code with ~"user facing", allowing us to ensure
that head.hackage is updated *before* the change makes it into `master`.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210123/1a4e5ef9/attachment.sig>

From shayne.fletcher.50 at gmail.com  Sun Jan 24 00:55:56 2021
From: shayne.fletcher.50 at gmail.com (Shayne Fletcher)
Date: Sat, 23 Jan 2021 19:55:56 -0500
Subject: [hadrian] HEAD link failure
Message-ID: <CAMsAzy8PzWD037egSZf__zwOpvmEqAw7+M9Z5WQ8=ktNK28+kg@mail.gmail.com>

Building `HEAD` just now with `hadrian/build-stack --flavour=quickest -j`,
I get a link failure for stage1 `bin/ghc`:
```
ld: library not found for -lHSrts-1.0_thr_l
```
I'll try another flavour or something but in the meantime is this ringing
any bells?

-- 
Shayne Fletcher
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210123/997e1f0c/attachment.html>

From viluon at seznam.cz  Mon Jan 25 11:05:46 2021
From: viluon at seznam.cz (Andrew Kvapil)
Date: Mon, 25 Jan 2021 12:05:46 +0100
Subject: Inspecting function arguments in GHCi
Message-ID: <0bd42dc9-87b7-7d8f-2c22-7b60599cc55e@seznam.cz>

Hello,

I'm interested in inspecting the strictness of functions at runtime
and the depth of thunks "in the wild." For this reason I'm modifying
GHC 8.10.2, essentially to add additional information to breakpoints.
I'd like to reuse the logic behind GHCi's :print command
(pprintClosureCommand, obtainTermFromId, ...) for which I suppose I
need Id's. Those however don't exist for destructuring patterns, such
as those in the following equations:

     last [x] = x
     last (_:xs) = last xs

So I'm wondering where would be a good place in the pipeline to
transform patterns like these into at-patterns, to give them Id's.
However, the breakpoint logic only looks at the free variables of the
right-hand sides and not transitively, which means that e.g. in the
following example neither ':print arg1' nor ':print as' works when the
interpreter hits a breakpoint in the top level expression on the RHS:

     qsort arg1@(a:as) = qsort left ++ [a] ++ qsort right
       where (left, right) = (filter (<=a) as, filter (>a) as)

Thus I'd also like to know how to extend the free var logic for
Tickish that eventually leads to CgBreakInfo and :print's ability to
inspect these bindings at runtime. My goal would be to determine to what
extent was a thunk evaluated during function application.

Any advice would be greatly appreciated!

Regards,
Andrew Kvapil

From simonpj at microsoft.com  Mon Jan 25 12:22:47 2021
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Mon, 25 Jan 2021 12:22:47 +0000
Subject: Inspecting function arguments in GHCi
In-Reply-To: <0bd42dc9-87b7-7d8f-2c22-7b60599cc55e@seznam.cz>
References: <0bd42dc9-87b7-7d8f-2c22-7b60599cc55e@seznam.cz>
Message-ID: <VI1PR83MB02222A44A69E318CAEB272B5ADBD9@VI1PR83MB0222.EURPRD83.prod.outlook.com>

Andrew

We have very poor documentation of the inner workings of the entire breakpoint
and debugging mechanism.   And very few (zero?) people who truly understand it.

You could do a great service by starting a Note or a wiki page or something
that lays out the moving parts.  You may not feel that you are well equipped to
do so, but you'd almost certainly improve matters!

Has anyone else been working in this space.  Matthew P perhaps?

|  So I'm wondering where would be a good place in the pipeline to
|  transform patterns like these into at-patterns, to give them Id's.

I'm not sure what you have in mind. Could you say more about (a) what you'd like
the user experience to be, and (b) how you are considering implementing it.

|  However, the breakpoint logic only looks at the free variables of the
|  right-hand sides and not transitively, which means that e.g. in the
|  following example neither ':print arg1' nor ':print as' works when the
|  interpreter hits a breakpoint in the top level expression on the RHS:
 
Perhaps you are suggesting that each breakpoint should capture bindings for
*all in-scope variables* rather than *all free variable of the sub-expression".
If so, that sounds pretty feasible.  It might risk keeping variables alive
that would otherwise have been garbage-collected, but maybe that's a price
worth paying.

Simon


|  -----Original Message-----
|  From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Andrew
|  Kvapil
|  Sent: 25 January 2021 11:06
|  To: ghc-devs at haskell.org
|  Subject: Inspecting function arguments in GHCi
|  
|  Hello,
|  
|  I'm interested in inspecting the strictness of functions at runtime
|  and the depth of thunks "in the wild." For this reason I'm modifying
|  GHC 8.10.2, essentially to add additional information to breakpoints.
|  I'd like to reuse the logic behind GHCi's :print command
|  (pprintClosureCommand, obtainTermFromId, ...) for which I suppose I
|  need Id's. Those however don't exist for destructuring patterns, such
|  as those in the following equations:
|  
|       last [x] = x
|       last (_:xs) = last xs
|  
|  So I'm wondering where would be a good place in the pipeline to
|  transform patterns like these into at-patterns, to give them Id's.
|  However, the breakpoint logic only looks at the free variables of the
|  right-hand sides and not transitively, which means that e.g. in the
|  following example neither ':print arg1' nor ':print as' works when the
|  interpreter hits a breakpoint in the top level expression on the RHS:
|  
|       qsort arg1@(a:as) = qsort left ++ [a] ++ qsort right
|         where (left, right) = (filter (<=a) as, filter (>a) as)
|  
|  Thus I'd also like to know how to extend the free var logic for
|  Tickish that eventually leads to CgBreakInfo and :print's ability to
|  inspect these bindings at runtime. My goal would be to determine to
|  what extent was a thunk evaluated during function application.
|  
|  Any advice would be greatly appreciated!
|  
|  Regards,
|  Andrew Kvapil
|  _______________________________________________
|  ghc-devs mailing list
|  ghc-devs at haskell.org
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.
|  haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devs&amp;data=04%7C01%7Csimonpj%40microsoft.com%7C329b12ba7bb74a2657d2
|  08d8c1213bcc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637471695782
|  207814%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJ
|  BTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=uAi4NmjQLj3QG3B7ton5GeFDy
|  IWJecxtXoXiTIP11tE%3D&amp;reserved=0

From viluon at seznam.cz  Mon Jan 25 13:18:13 2021
From: viluon at seznam.cz (Andrew Kvapil)
Date: Mon, 25 Jan 2021 14:18:13 +0100
Subject: Inspecting function arguments in GHCi
In-Reply-To: <VI1PR83MB02222A44A69E318CAEB272B5ADBD9@VI1PR83MB0222.EURPRD83.prod.outlook.com>
References: <0bd42dc9-87b7-7d8f-2c22-7b60599cc55e@seznam.cz>
 <VI1PR83MB02222A44A69E318CAEB272B5ADBD9@VI1PR83MB0222.EURPRD83.prod.outlook.com>
Message-ID: <6427fc7e-ca63-841f-fb5d-86fa2050c2f4@seznam.cz>

Dear Simon,

Thank you for the swift response. I did notice the lack of
comprehensive documentation on the details of the debugger. I keep
notes of my findings as I explore the codebase and expect to publish
them once I get them into a more presentable form. This work is a part
of my bachelor's thesis, so a fairly detailed explanation should come
out of it.

 > I'm not sure what you have in mind. Could you say more about (a) what 
you'd like
 > the user experience to be, and (b) how you are considering 
implementing it.

To be clear, I don't intend to affect the user experience of
breakpoints themselves. What I'm after is the information about thunks
collected at runtime without user intervention, I added what is
essentially a recreation of breakpoints called "tracepoints" for this
purpose. The user would simply invoke a traced program and information
about the strictness of various function applications would be logged
to a file. The result of the execution would be ultimately useful for
statistical analysis of the use of laziness in Haskell, although at
this point that is a distant and uncertain goal.

At the moment, the implementation of tracepoints suspends the GHCi
evaluation thread and immediately resumes it (by literally queueing
":continue" in the UI). To get any useful information and start
logging it at runtime, I need to capture more information than just
the free variables of an expression at bytecode generation time (or
sooner, since from what I gather this is in part also a responsibility
of the Tickish logic).

Perhaps the approach I've taken thus far is nonsensical or
significantly inferior to some well-known alternative I'm unaware of.
If that seems to be the case, I will happily listen to advice
suggesting a change of course. This is the first time I'm hacking on
GHC and I have to say I find the scale and complexity of the codebase
somewhat daunting, to say the least.

 > Perhaps you are suggesting that each breakpoint should capture 
bindings for
 > *all in-scope variables* rather than *all free variable of the 
sub-expression".
 > If so, that sounds pretty feasible.  It might risk keeping variables 
alive
 > that would otherwise have been garbage-collected, but maybe that's a 
price
 > worth paying.

Yes, not (necessarily) for breakpoints, but for tracepoints I think
that's closer to the desired behaviour. Perhaps there's even a simpler
approach that lets tracepoints only capture the "top-level" function
arguments, i.e. not the bindings that come from destructuring, as in
the shared qsort example. The top-level function arguments are what I
had in mind when suggesting the conversion to at-patterns.

Regards,
Andrew

On 25/01/2021 13:22, Simon Peyton Jones wrote:
 > Andrew
 >
 > We have very poor documentation of the inner workings of the entire 
breakpoint
 > and debugging mechanism.   And very few (zero?) people who truly 
understand it.
 >
 > You could do a great service by starting a Note or a wiki page or 
something
 > that lays out the moving parts.  You may not feel that you are well 
equipped to
 > do so, but you'd almost certainly improve matters!
 >
 > Has anyone else been working in this space.  Matthew P perhaps?
 >
 > |  So I'm wondering where would be a good place in the pipeline to
 > |  transform patterns like these into at-patterns, to give them Id's.
 >
 > I'm not sure what you have in mind. Could you say more about (a) what 
you'd like
 > the user experience to be, and (b) how you are considering 
implementing it.
 >
 > |  However, the breakpoint logic only looks at the free variables of the
 > |  right-hand sides and not transitively, which means that e.g. in the
 > |  following example neither ':print arg1' nor ':print as' works when the
 > |  interpreter hits a breakpoint in the top level expression on the RHS:
 >
 > Perhaps you are suggesting that each breakpoint should capture 
bindings for
 > *all in-scope variables* rather than *all free variable of the 
sub-expression".
 > If so, that sounds pretty feasible.  It might risk keeping variables 
alive
 > that would otherwise have been garbage-collected, but maybe that's a 
price
 > worth paying.
 >
 > Simon
 >
 >
 > |  -----Original Message-----
 > |  From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Andrew
 > |  Kvapil
 > |  Sent: 25 January 2021 11:06
 > |  To: ghc-devs at haskell.org
 > |  Subject: Inspecting function arguments in GHCi
 > |
 > |  Hello,
 > |
 > |  I'm interested in inspecting the strictness of functions at runtime
 > |  and the depth of thunks "in the wild." For this reason I'm modifying
 > |  GHC 8.10.2, essentially to add additional information to breakpoints.
 > |  I'd like to reuse the logic behind GHCi's :print command
 > |  (pprintClosureCommand, obtainTermFromId, ...) for which I suppose I
 > |  need Id's. Those however don't exist for destructuring patterns, such
 > |  as those in the following equations:
 > |
 > |       last [x] = x
 > |       last (_:xs) = last xs
 > |
 > |  So I'm wondering where would be a good place in the pipeline to
 > |  transform patterns like these into at-patterns, to give them Id's.
 > |  However, the breakpoint logic only looks at the free variables of the
 > |  right-hand sides and not transitively, which means that e.g. in the
 > |  following example neither ':print arg1' nor ':print as' works when the
 > |  interpreter hits a breakpoint in the top level expression on the RHS:
 > |
 > |       qsort arg1@(a:as) = qsort left ++ [a] ++ qsort right
 > |         where (left, right) = (filter (<=a) as, filter (>a) as)
 > |
 > |  Thus I'd also like to know how to extend the free var logic for
 > |  Tickish that eventually leads to CgBreakInfo and :print's ability to
 > |  inspect these bindings at runtime. My goal would be to determine to
 > |  what extent was a thunk evaluated during function application.
 > |
 > |  Any advice would be greatly appreciated!
 > |
 > |  Regards,
 > |  Andrew Kvapil
 > |  _______________________________________________
 > |  ghc-devs mailing list
 > |  ghc-devs at haskell.org
 > |  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.
 > |  haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
 > |  devs&amp;data=04%7C01%7Csimonpj%40microsoft.com%7C329b12ba7bb74a2657d2
 > |  08d8c1213bcc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637471695782
 > |  207814%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJ
 > |  BTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=uAi4NmjQLj3QG3B7ton5GeFDy
 > |  IWJecxtXoXiTIP11tE%3D&amp;reserved=0
 >

From ben at smart-cactus.org  Mon Jan 25 16:06:59 2021
From: ben at smart-cactus.org (Ben Gamari)
Date: Mon, 25 Jan 2021 11:06:59 -0500
Subject: [hadrian] HEAD link failure
In-Reply-To: <CAMsAzy8PzWD037egSZf__zwOpvmEqAw7+M9Z5WQ8=ktNK28+kg@mail.gmail.com>
References: <CAMsAzy8PzWD037egSZf__zwOpvmEqAw7+M9Z5WQ8=ktNK28+kg@mail.gmail.com>
Message-ID: <87k0s10zmn.fsf@smart-cactus.org>

Shayne Fletcher <shayne.fletcher.50 at gmail.com> writes:

> Building `HEAD` just now with `hadrian/build-stack --flavour=quickest -j`,
> I get a link failure for stage1 `bin/ghc`:
> ```
> ld: library not found for -lHSrts-1.0_thr_l
> ```
> I'll try another flavour or something but in the meantime is this ringing
> any bells?
>
For the record this is being tracked as #19253 (thanks for opening this, Shayne!)

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210125/a6ef5c28/attachment.sig>

From juhpetersen at gmail.com  Tue Jan 26 10:33:01 2021
From: juhpetersen at gmail.com (Jens Petersen)
Date: Tue, 26 Jan 2021 18:33:01 +0800
Subject: [ANNOUNCE] Glasgow Haskell Compiler 9.0.1-rc1 released
In-Reply-To: <878s9fs4s2.fsf@smart-cactus.org>
References: <878s9fs4s2.fsf@smart-cactus.org>
Message-ID: <CAEDGBqK=XVzpDn8ffP91fEwo9G9MAR6ZPTn_Y_gpvTvJnwhAyg@mail.gmail.com>

Thanks - I forgot to follow up earlier but on Fedora you can test 9.0.1 RC1
now with:

sudo dnf module install ghc:9.0/default

Jens
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210126/c2779644/attachment.html>

From ben at smart-cactus.org  Tue Jan 26 15:28:20 2021
From: ben at smart-cactus.org (Ben Gamari)
Date: Tue, 26 Jan 2021 10:28:20 -0500
Subject: Inspecting function arguments in GHCi
In-Reply-To: <0bd42dc9-87b7-7d8f-2c22-7b60599cc55e@seznam.cz>
References: <0bd42dc9-87b7-7d8f-2c22-7b60599cc55e@seznam.cz>
Message-ID: <87a6sv1zw0.fsf@smart-cactus.org>

Andrew Kvapil <viluon at seznam.cz> writes:

> Hello,
>
> I'm interested in inspecting the strictness of functions at runtime
> and the depth of thunks "in the wild."

Hi Andrew,

Interesting. When I first read your introduction my first thought was to
rather walk the "normal" heap using ghc-heap or, perhaps, the relatively
new ghc-debug library [1] rather than introduce this feature in GHCi.
It's hard to know whether non-GHCi-based approach is viable without
knowing more about what you are doing, but it potentially brings the
benefit of generality: your analysis is not limited to programs with can
be run under GHCi.

Of course, it also brings some challenges: You would need to find a way
to associate info table symbols with whatever information you need of
from the Core program and in the presence of simplification tying your
results back to the source program may not be possible at all.

Cheers,

- Ben


[1] https://gitlab.haskell.org/ghc/ghc-debug

> For this reason I'm modifying
> GHC 8.10.2, essentially to add additional information to breakpoints.
> I'd like to reuse the logic behind GHCi's :print command
> (pprintClosureCommand, obtainTermFromId, ...) for which I suppose I
> need Id's. Those however don't exist for destructuring patterns, such
> as those in the following equations:
>
>      last [x] = x
>      last (_:xs) = last xs
>
> So I'm wondering where would be a good place in the pipeline to
> transform patterns like these into at-patterns, to give them Id's.
> However, the breakpoint logic only looks at the free variables of the
> right-hand sides and not transitively, which means that e.g. in the
> following example neither ':print arg1' nor ':print as' works when the
> interpreter hits a breakpoint in the top level expression on the RHS:
>
>      qsort arg1@(a:as) = qsort left ++ [a] ++ qsort right
>        where (left, right) = (filter (<=a) as, filter (>a) as)
>
> Thus I'd also like to know how to extend the free var logic for
> Tickish that eventually leads to CgBreakInfo and :print's ability to
> inspect these bindings at runtime. My goal would be to determine to what
> extent was a thunk evaluated during function application.
>
Note that Luite's recent work on refactoring the bytecode generator to
produce code from STG is quite relevant here. In particular, you will
likely want to look at !4589 [1], which does the work of refactoring
Tickish to follow the Trees That Grow pattern. You would likely want to
do the same to capture your free variable information.

Cheers,

- Ben


[1] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/4589
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210126/68e7165b/attachment.sig>

From rae at richarde.dev  Wed Jan 27 20:56:45 2021
From: rae at richarde.dev (Richard Eisenberg)
Date: Wed, 27 Jan 2021 20:56:45 +0000
Subject: safe to upgrade my Mac?
Message-ID: <010f017745a260ee-fff66bae-0629-4920-a288-65e9bacc8fa1-000000@us-east-2.amazonses.com>

Hi devs,

I'm currently running MacOS High Sierra, 10.13.6. Things are fine, but I'd like to upgrade to get Dark Mode and (hopefully) to speed up Mail. (That is, upgrading is nice, but not at all necessary.)

If I upgrade, will GHC hate me? That is, will GHC 8.10 continue to work? Will I continue to be able to compile GHC? Will it be as performant? (A few years ago, I had a strange issue on a secondary computer where any binary built by GHC was horribly slow; we never got to the bottom of it.)

All advice is welcome.

Thanks!
Richard

From merijn at inconsistent.nl  Wed Jan 27 21:28:36 2021
From: merijn at inconsistent.nl (Merijn Verstraaten)
Date: Wed, 27 Jan 2021 22:28:36 +0100
Subject: safe to upgrade my Mac?
In-Reply-To: <010f017745a260ee-fff66bae-0629-4920-a288-65e9bacc8fa1-000000@us-east-2.amazonses.com>
References: <010f017745a260ee-fff66bae-0629-4920-a288-65e9bacc8fa1-000000@us-east-2.amazonses.com>
Message-ID: <22C492E2-3601-4082-8E01-1A556614B9A5@inconsistent.nl>


> On 27 Jan 2021, at 21:56, Richard Eisenberg <rae at richarde.dev> wrote:
> I'm currently running MacOS High Sierra, 10.13.6. Things are fine, but I'd like to upgrade to get Dark Mode and (hopefully) to speed up Mail. (That is, upgrading is nice, but not at all necessary.)

I just (finally) upgraded to Catalina last week and the 8.10.2 bindist works just fine (you have to fiddle with xattr after unzipping, there's an issue on gitlab that has the right command in there). If you're going to Mojave then there's 0 issues. Big Sur I haven't tried yet.

> If I upgrade, will GHC hate me? That is, will GHC 8.10 continue to work? Will I continue to be able to compile GHC? Will it be as performant? (A few years ago, I had a strange issue on a secondary computer where any binary built by GHC was horribly slow; we never got to the bottom of it.)

I didn't notice any performance regressions, but I also haven't paid close attention.

- Merijn
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210127/005eb943/attachment.sig>

From viluon at seznam.cz  Wed Jan 27 22:15:08 2021
From: viluon at seznam.cz (Andrew Kvapil)
Date: Wed, 27 Jan 2021 23:15:08 +0100
Subject: Inspecting function arguments in GHCi
In-Reply-To: <87a6sv1zw0.fsf@smart-cactus.org>
References: <0bd42dc9-87b7-7d8f-2c22-7b60599cc55e@seznam.cz>
 <87a6sv1zw0.fsf@smart-cactus.org>
Message-ID: <615845bc-571d-15aa-80c8-fce4b03b3bb5@seznam.cz>

Hello Ben,

Thanks for your suggestions. The decision to adapt GHCi came out of a
discussion with my supervisor and his colleagues. At this point the
entire set of desired capabilities of the work is still unknown, however
we do consider the GHCi-compatible programs to represent a large enough
set for future analysis, and the ease of mapping breakpoints back to
source code is a significant benefit.

I do plan on using the ghc-heap-view (I assume that's what you meant by
ghc-heap, or is there another library I don't know about?) logic in the
project, although I'm currently more focused on implementing the proper
hook mechanism. I expect that events of deeply nested thunks being
forced will be quite important. The possibility of tracking control
flow via breakpoints/tracepoints also seems appealing. I'm not aware
of any existing solutions which would allow dynamic tracing, although
it's very well possible I didn't look hard enough.

Regarding ghc-debug, I'm not sure what kinds of trade-offs it offers
compared to the approach I'm currently taking. It looks like it's a
fairly newborn project, do you think it's mature enough for the
proposed use cases? I couldn't find docs online, although I did come
across [0] which Discourse[1] says is related. I've yet to watch the
introduction video. Support for unboxed tuples and other features not
supported by GHCi would of course be nice, although performance is not
a concern. Keeping the relationship between source code spans and
heap objects in the infotables is an intriguing idea.

 > Note that Luite's recent work on refactoring the bytecode generator to

 > produce code from STG is quite relevant here. In particular, you will

 > likely want to look at !4589 [1], which does the work of refactoring

 > Tickish to follow the Trees That Grow pattern. You would likely want to

 > do the same to capture your free variable information.


Excellent, I was not aware of this. Thank you!


Regards,
Andrew



[0]: https://well-typed.com/blog/2021/01/first-look-at-hi-profiling-mode/
[1]: 
https://discourse.haskell.org/t/an-introduction-to-ghc-debug-precise-memory-analysis-for-haskell-programs/1771


On 26/01/2021 16:28, Ben Gamari wrote:
> Andrew Kvapil <viluon at seznam.cz> writes:
> 
>> Hello,
>>
>> I'm interested in inspecting the strictness of functions at runtime
>> and the depth of thunks "in the wild."
> 
> Hi Andrew,
> 
> Interesting. When I first read your introduction my first thought was to
> rather walk the "normal" heap using ghc-heap or, perhaps, the relatively
> new ghc-debug library [1] rather than introduce this feature in GHCi.
> It's hard to know whether non-GHCi-based approach is viable without
> knowing more about what you are doing, but it potentially brings the
> benefit of generality: your analysis is not limited to programs with can
> be run under GHCi.
> 
> Of course, it also brings some challenges: You would need to find a way
> to associate info table symbols with whatever information you need of
> from the Core program and in the presence of simplification tying your
> results back to the source program may not be possible at all.
> 
> Cheers,
> 
> - Ben
> 
> 
> [1] https://gitlab.haskell.org/ghc/ghc-debug
> 
>> For this reason I'm modifying
>> GHC 8.10.2, essentially to add additional information to breakpoints.
>> I'd like to reuse the logic behind GHCi's :print command
>> (pprintClosureCommand, obtainTermFromId, ...) for which I suppose I
>> need Id's. Those however don't exist for destructuring patterns, such
>> as those in the following equations:
>>
>>       last [x] = x
>>       last (_:xs) = last xs
>>
>> So I'm wondering where would be a good place in the pipeline to
>> transform patterns like these into at-patterns, to give them Id's.
>> However, the breakpoint logic only looks at the free variables of the
>> right-hand sides and not transitively, which means that e.g. in the
>> following example neither ':print arg1' nor ':print as' works when the
>> interpreter hits a breakpoint in the top level expression on the RHS:
>>
>>       qsort arg1@(a:as) = qsort left ++ [a] ++ qsort right
>>         where (left, right) = (filter (<=a) as, filter (>a) as)
>>
>> Thus I'd also like to know how to extend the free var logic for
>> Tickish that eventually leads to CgBreakInfo and :print's ability to
>> inspect these bindings at runtime. My goal would be to determine to what
>> extent was a thunk evaluated during function application.
>>
> Note that Luite's recent work on refactoring the bytecode generator to
> produce code from STG is quite relevant here. In particular, you will
> likely want to look at !4589 [1], which does the work of refactoring
> Tickish to follow the Trees That Grow pattern. You would likely want to
> do the same to capture your free variable information.
> 
> Cheers,
> 
> - Ben
> 
> 
> [1] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/4589
> 

From alfredo.dinapoli at gmail.com  Thu Jan 28 07:32:22 2021
From: alfredo.dinapoli at gmail.com (Alfredo Di Napoli)
Date: Thu, 28 Jan 2021 08:32:22 +0100
Subject: safe to upgrade my Mac?
In-Reply-To: <22C492E2-3601-4082-8E01-1A556614B9A5@inconsistent.nl>
References: <010f017745a260ee-fff66bae-0629-4920-a288-65e9bacc8fa1-000000@us-east-2.amazonses.com>
 <22C492E2-3601-4082-8E01-1A556614B9A5@inconsistent.nl>
Message-ID: <CAFqA6+UF66k9e1xUjk_gMbHSEeNfq3R8MA0ZZU9uAF=pWLMqhA@mail.gmail.com>

Hi Richard,

I am running 10.15.7 and I think I upgraded to Catalina months ago (but I
don't exactly recall when). From my experience, everything works just fine,
at least for my current workflow (clients' projects, GHC development etc).
At work I have successfully used GHC 8.6.5 and 8.10.2 for various projects,
both with Cabal and Stack and didn't experience any major problems.

In terms of performance, I cannot judge. If it got slower, I certainly
didn't notice.

I hope it helps! :)

A.

On Wed, 27 Jan 2021 at 22:29, Merijn Verstraaten <merijn at inconsistent.nl>
wrote:

>
> > On 27 Jan 2021, at 21:56, Richard Eisenberg <rae at richarde.dev> wrote:
> > I'm currently running MacOS High Sierra, 10.13.6. Things are fine, but
> I'd like to upgrade to get Dark Mode and (hopefully) to speed up Mail.
> (That is, upgrading is nice, but not at all necessary.)
>
> I just (finally) upgraded to Catalina last week and the 8.10.2 bindist
> works just fine (you have to fiddle with xattr after unzipping, there's an
> issue on gitlab that has the right command in there). If you're going to
> Mojave then there's 0 issues. Big Sur I haven't tried yet.
>
> > If I upgrade, will GHC hate me? That is, will GHC 8.10 continue to work?
> Will I continue to be able to compile GHC? Will it be as performant? (A few
> years ago, I had a strange issue on a secondary computer where any binary
> built by GHC was horribly slow; we never got to the bottom of it.)
>
> I didn't notice any performance regressions, but I also haven't paid close
> attention.
>
> - Merijn
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210128/7ce91945/attachment.html>

From carter.schonwald at gmail.com  Thu Jan 28 20:00:03 2021
From: carter.schonwald at gmail.com (Carter Schonwald)
Date: Thu, 28 Jan 2021 15:00:03 -0500
Subject: safe to upgrade my Mac?
In-Reply-To: <CAFqA6+UF66k9e1xUjk_gMbHSEeNfq3R8MA0ZZU9uAF=pWLMqhA@mail.gmail.com>
References: <010f017745a260ee-fff66bae-0629-4920-a288-65e9bacc8fa1-000000@us-east-2.amazonses.com>
 <22C492E2-3601-4082-8E01-1A556614B9A5@inconsistent.nl>
 <CAFqA6+UF66k9e1xUjk_gMbHSEeNfq3R8MA0ZZU9uAF=pWLMqhA@mail.gmail.com>
Message-ID: <CAHYVw0xanjRk0-PpJU2Q0zQkJP9TXXJgdU-j61FVnUub31HAdA@mail.gmail.com>

8.10.3 works on every more recent OS X.

That said, I’m waiting for cabal 3.4 Release, which 8.10 can build, before
I do the Big Sur update.

On Thu, Jan 28, 2021 at 2:35 AM Alfredo Di Napoli <
alfredo.dinapoli at gmail.com> wrote:

> Hi Richard,
>
> I am running 10.15.7 and I think I upgraded to Catalina months ago (but I
> don't exactly recall when). From my experience, everything works just fine,
> at least for my current workflow (clients' projects, GHC development etc).
> At work I have successfully used GHC 8.6.5 and 8.10.2 for various projects,
> both with Cabal and Stack and didn't experience any major problems.
>
> In terms of performance, I cannot judge. If it got slower, I certainly
> didn't notice.
>
> I hope it helps! :)
>
> A.
>
> On Wed, 27 Jan 2021 at 22:29, Merijn Verstraaten <merijn at inconsistent.nl>
> wrote:
>
>>
>> > On 27 Jan 2021, at 21:56, Richard Eisenberg <rae at richarde.dev> wrote:
>> > I'm currently running MacOS High Sierra, 10.13.6. Things are fine, but
>> I'd like to upgrade to get Dark Mode and (hopefully) to speed up Mail.
>> (That is, upgrading is nice, but not at all necessary.)
>>
>> I just (finally) upgraded to Catalina last week and the 8.10.2 bindist
>> works just fine (you have to fiddle with xattr after unzipping, there's an
>> issue on gitlab that has the right command in there). If you're going to
>> Mojave then there's 0 issues. Big Sur I haven't tried yet.
>>
>> > If I upgrade, will GHC hate me? That is, will GHC 8.10 continue to
>> work? Will I continue to be able to compile GHC? Will it be as performant?
>> (A few years ago, I had a strange issue on a secondary computer where any
>> binary built by GHC was horribly slow; we never got to the bottom of it.)
>>
>> I didn't notice any performance regressions, but I also haven't paid
>> close attention.
>>
>> - Merijn
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210128/f0eeec35/attachment.html>

From rae at richarde.dev  Thu Jan 28 20:20:54 2021
From: rae at richarde.dev (Richard Eisenberg)
Date: Thu, 28 Jan 2021 20:20:54 +0000
Subject: -ddump-json
Message-ID: <010f01774aa7ebc9-44ce5848-5bb4-4fc5-bb9a-9725f30aab25-000000@us-east-2.amazonses.com>

Hi devs,

In my work with Alfredo at revising our error message infrastructure, we ran across some code that renders error messages as JSON. Given that our data structures are changing, it seems natural to change the JSON output, too, but it's unclear whether that's wise. The manual currently lists -ddump-json in the chapter on "Debugging the compiler", suggesting that a change is fine, but I'm not yet convinced.

So:
 - Is there someone in charge of -ddump-json? Matthew Pickering authored -ddump-json in https://gitlab.haskell.org/ghc/ghc/-/commit/91691117fc194c525f58ccd5b266dd1d10493e5a <https://gitlab.haskell.org/ghc/ghc/-/commit/91691117fc194c525f58ccd5b266dd1d10493e5a>.
 - Are there clients of -ddump-json?
 - Is there a specification of -ddump-json?
 - There will likely be a desire to evolve this feature. What is the process for doing so?

Thanks!
Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210128/fbb30a0c/attachment.html>

From matthewtpickering at gmail.com  Fri Jan 29 08:57:16 2021
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Fri, 29 Jan 2021 08:57:16 +0000
Subject: -ddump-json
In-Reply-To: <010f01774aa7ebc9-44ce5848-5bb4-4fc5-bb9a-9725f30aab25-000000@us-east-2.amazonses.com>
References: <010f01774aa7ebc9-44ce5848-5bb4-4fc5-bb9a-9725f30aab25-000000@us-east-2.amazonses.com>
Message-ID: <CALuQ0m9uwr6uV_59zhE3bEkRNtR0GpAuBkTk8RsTpzRLpHGBOQ@mail.gmail.com>

Just grepping Github there appears to be a few users of the flag but
no more than a handful. I think you can probably change it how you
like.

Cheers,

Matt

On Thu, Jan 28, 2021 at 8:20 PM Richard Eisenberg <rae at richarde.dev> wrote:
>
> Hi devs,
>
> In my work with Alfredo at revising our error message infrastructure, we ran across some code that renders error messages as JSON. Given that our data structures are changing, it seems natural to change the JSON output, too, but it's unclear whether that's wise. The manual currently lists -ddump-json in the chapter on "Debugging the compiler", suggesting that a change is fine, but I'm not yet convinced.
>
> So:
>  - Is there someone in charge of -ddump-json? Matthew Pickering authored -ddump-json in https://gitlab.haskell.org/ghc/ghc/-/commit/91691117fc194c525f58ccd5b266dd1d10493e5a.
>  - Are there clients of -ddump-json?
>  - Is there a specification of -ddump-json?
>  - There will likely be a desire to evolve this feature. What is the process for doing so?
>
> Thanks!
> Richard

From hecate at glitchbra.in  Fri Jan 29 09:02:32 2021
From: hecate at glitchbra.in (=?UTF-8?Q?H=c3=a9cate?=)
Date: Fri, 29 Jan 2021 10:02:32 +0100
Subject: -ddump-json
In-Reply-To: <CALuQ0m9uwr6uV_59zhE3bEkRNtR0GpAuBkTk8RsTpzRLpHGBOQ@mail.gmail.com>
References: <010f01774aa7ebc9-44ce5848-5bb4-4fc5-bb9a-9725f30aab25-000000@us-east-2.amazonses.com>
 <CALuQ0m9uwr6uV_59zhE3bEkRNtR0GpAuBkTk8RsTpzRLpHGBOQ@mail.gmail.com>
Message-ID: <46f56540-4009-7d8f-c771-6224fc8f19af@glitchbra.in>

Do we provide any kind of versioning or data schema for the JSON dumps?

On 29/01/2021 09:57, Matthew Pickering wrote:
> Just grepping Github there appears to be a few users of the flag but
> no more than a handful. I think you can probably change it how you
> like.
>
> Cheers,
>
> Matt
>
> On Thu, Jan 28, 2021 at 8:20 PM Richard Eisenberg <rae at richarde.dev> wrote:
>> Hi devs,
>>
>> In my work with Alfredo at revising our error message infrastructure, we ran across some code that renders error messages as JSON. Given that our data structures are changing, it seems natural to change the JSON output, too, but it's unclear whether that's wise. The manual currently lists -ddump-json in the chapter on "Debugging the compiler", suggesting that a change is fine, but I'm not yet convinced.
>>
>> So:
>>   - Is there someone in charge of -ddump-json? Matthew Pickering authored -ddump-json in https://gitlab.haskell.org/ghc/ghc/-/commit/91691117fc194c525f58ccd5b266dd1d10493e5a.
>>   - Are there clients of -ddump-json?
>>   - Is there a specification of -ddump-json?
>>   - There will likely be a desire to evolve this feature. What is the process for doing so?
>>
>> Thanks!
>> Richard
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-- 
Hécate ✨
🐦: @TechnoEmpress
IRC: Uniaika
WWW: https://glitchbra.in
RUN: BSD


From rae at richarde.dev  Fri Jan 29 19:45:30 2021
From: rae at richarde.dev (Richard Eisenberg)
Date: Fri, 29 Jan 2021 19:45:30 +0000
Subject: -ddump-json
In-Reply-To: <46f56540-4009-7d8f-c771-6224fc8f19af@glitchbra.in>
References: <010f01774aa7ebc9-44ce5848-5bb4-4fc5-bb9a-9725f30aab25-000000@us-east-2.amazonses.com>
 <CALuQ0m9uwr6uV_59zhE3bEkRNtR0GpAuBkTk8RsTpzRLpHGBOQ@mail.gmail.com>
 <46f56540-4009-7d8f-c771-6224fc8f19af@glitchbra.in>
Message-ID: <010f01774faddec0-ebb08e7b-0158-4e05-89b5-f70c44c613c4-000000@us-east-2.amazonses.com>

I have filed https://gitlab.haskell.org/ghc/ghc/-/issues/19278 <https://gitlab.haskell.org/ghc/ghc/-/issues/19278> to try to get this sorted. Thanks for the quick responses!

Richard

> On Jan 29, 2021, at 4:02 AM, Hécate <hecate at glitchbra.in> wrote:
> 
> Do we provide any kind of versioning or data schema for the JSON dumps?
> 
> On 29/01/2021 09:57, Matthew Pickering wrote:
>> Just grepping Github there appears to be a few users of the flag but
>> no more than a handful. I think you can probably change it how you
>> like.
>> 
>> Cheers,
>> 
>> Matt
>> 
>> On Thu, Jan 28, 2021 at 8:20 PM Richard Eisenberg <rae at richarde.dev> wrote:
>>> Hi devs,
>>> 
>>> In my work with Alfredo at revising our error message infrastructure, we ran across some code that renders error messages as JSON. Given that our data structures are changing, it seems natural to change the JSON output, too, but it's unclear whether that's wise. The manual currently lists -ddump-json in the chapter on "Debugging the compiler", suggesting that a change is fine, but I'm not yet convinced.
>>> 
>>> So:
>>>  - Is there someone in charge of -ddump-json? Matthew Pickering authored -ddump-json in https://gitlab.haskell.org/ghc/ghc/-/commit/91691117fc194c525f58ccd5b266dd1d10493e5a.
>>>  - Are there clients of -ddump-json?
>>>  - Is there a specification of -ddump-json?
>>>  - There will likely be a desire to evolve this feature. What is the process for doing so?
>>> 
>>> Thanks!
>>> Richard
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> 
> -- 
> Hécate ✨
> 🐦: @TechnoEmpress
> IRC: Uniaika
> WWW: https://glitchbra.in
> RUN: BSD
> 
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210129/53ba2d65/attachment.html>

From ben at well-typed.com  Fri Jan 29 19:50:52 2021
From: ben at well-typed.com (Ben Gamari)
Date: Fri, 29 Jan 2021 14:50:52 -0500
Subject: -ddump-json
In-Reply-To: <010f01774aa7ebc9-44ce5848-5bb4-4fc5-bb9a-9725f30aab25-000000@us-east-2.amazonses.com>
References: <010f01774aa7ebc9-44ce5848-5bb4-4fc5-bb9a-9725f30aab25-000000@us-east-2.amazonses.com>
Message-ID: <871re3ttd5.fsf@smart-cactus.org>

Richard Eisenberg <rae at richarde.dev> writes:

> Hi devs,
>
> In my work with Alfredo at revising our error message infrastructure,
> we ran across some code that renders error messages as JSON. Given
> that our data structures are changing, it seems natural to change the
> JSON output, too, but it's unclear whether that's wise. The manual
> currently lists -ddump-json in the chapter on "Debugging the
> compiler", suggesting that a change is fine, but I'm not yet
> convinced.
>
I think it would be fine to change the output. However, note that there
is a reason why this flag is in the -d flag namespace and the "Debugging
the compiler". The output is quite unstructured and we reserve the right to
change the representation, largely because it was hard to do better
without first fixing #8809.

After we have the new rich errors infrastucture in place I think we will
be in a much better place to discuss a properly-supported flag (via the
proposal process, presumably). However, I think when we do so we should
be careful to constrain the scope of the provided output. GHC is not a
language server and I don't think it would be wise to make it one.

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210129/feab6c43/attachment.sig>

From iavor.diatchki at gmail.com  Fri Jan 29 20:48:06 2021
From: iavor.diatchki at gmail.com (Iavor Diatchki)
Date: Fri, 29 Jan 2021 12:48:06 -0800
Subject: -ddump-json
In-Reply-To: <871re3ttd5.fsf@smart-cactus.org>
References: <010f01774aa7ebc9-44ce5848-5bb4-4fc5-bb9a-9725f30aab25-000000@us-east-2.amazonses.com>
 <871re3ttd5.fsf@smart-cactus.org>
Message-ID: <CAGK9nuqVuO44rkV-HGWhnKffvCDuYz-RTGXS+4g4GHQue3DFRg@mail.gmail.com>

Hello,
it seems pretty reasonable to change it, but we should change the version
number as well (well, and add it if it is not there).  In general, having a
version number is probably good practice for any outward facing machine
readable format.
-Iavor


On Fri, Jan 29, 2021 at 11:51 AM Ben Gamari <ben at well-typed.com> wrote:

> Richard Eisenberg <rae at richarde.dev> writes:
>
> > Hi devs,
> >
> > In my work with Alfredo at revising our error message infrastructure,
> > we ran across some code that renders error messages as JSON. Given
> > that our data structures are changing, it seems natural to change the
> > JSON output, too, but it's unclear whether that's wise. The manual
> > currently lists -ddump-json in the chapter on "Debugging the
> > compiler", suggesting that a change is fine, but I'm not yet
> > convinced.
> >
> I think it would be fine to change the output. However, note that there
> is a reason why this flag is in the -d flag namespace and the "Debugging
> the compiler". The output is quite unstructured and we reserve the right to
> change the representation, largely because it was hard to do better
> without first fixing #8809.
>
> After we have the new rich errors infrastucture in place I think we will
> be in a much better place to discuss a properly-supported flag (via the
> proposal process, presumably). However, I think when we do so we should
> be careful to constrain the scope of the provided output. GHC is not a
> language server and I don't think it would be wise to make it one.
>
> Cheers,
>
> - Ben
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210129/e951e539/attachment.html>