From zubin at well-typed.com  Fri Nov  1 12:11:38 2024
From: zubin at well-typed.com (Zubin Duggal)
Date: Fri, 1 Nov 2024 17:41:38 +0530
Subject: 9.12.1-alpha2 delay
In-Reply-To: <ksv72envoturshwkb2t3c35gmricb3gjprirtba3igtgyqjdvd@ywpc7nhvxjzt>
References: <ksv72envoturshwkb2t3c35gmricb3gjprirtba3igtgyqjdvd@ywpc7nhvxjzt>
Message-ID: <3rrjk4tclmev4sdwifmn3t2qs4a4m32fk7xg65lplpb3jzuehk@ygvlyhxm5fnc>

Hi all,

9.12.1-alpha2 is slightly delayed because we discovered some
pipeline failures on the x86_64-linux.maurer.smart-cactus.org
runner. Some investigation points to memory corruption/hardware
failure as the likely cause of these issues and the runner
has been taken offline. 

While the pipeline is now passing, out of an abundance of
caution we are rerunning any jobs that ran on the affected runner.

Cheers,
Zubin

On 24/10/30 17:30, Zubin Duggal wrote:
>Hi all,
>
>I am currently in the process of preparing 9.12.1-alpha2.
>I expect a release to be published by the end of this week.
>
>We are tracking the current status of the submodules in this
>spreadsheet: https://docs.google.com/spreadsheets/d/19WeWjXPh8Q64qoe3fjs8AswpTHk6uLBD8WG-7R97a1U/edit?gid=1512690813#gid=1512690813
>
>I want to finalise submodule versions by the next alpha in
>2 weeks. If you have any requests for newer versions of
>submodules/boot libraries, please update the spreadsheet and
>drop me a message.
>
>Cheers,
>Zubin


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241101/a8fa7d96/attachment.sig>

From hecate at glitchbra.in  Fri Nov  1 13:59:16 2024
From: hecate at glitchbra.in (=?UTF-8?Q?H=C3=A9cate?=)
Date: Fri, 1 Nov 2024 14:59:16 +0100
Subject: What makes C++ FFI painful?
In-Reply-To: <CAH3XM1kvQbUuqruHxF5JUj1Wu7yeMEdB4evL9Den46PKV2Pw2A@mail.gmail.com>
References: <04322faa-58bb-4102-af5b-d893c9aa8831@glitchbra.in>
 <877c9o2qbr.fsf@smart-cactus.org>
 <CAH3XM1kvQbUuqruHxF5JUj1Wu7yeMEdB4evL9Den46PKV2Pw2A@mail.gmail.com>
Message-ID: <872e29f3-1f4e-4ecf-a5a7-59160bc210e0@glitchbra.in>

Thank you both. From what I understand, some it appears to be linked to 
the fact that C as the lowest common denominator, forces a conscious 
effort on the part of other languages to provide this "simplified" 
interface to their binary artifacts. Maybe crabi¹ will help in the future.
I knew this was a challenge but I understand better now. :)

Cheers,
Hécate

¹ "Interoperability between high-level programming languages that have 
safe data types", 
https://github.com/joshtriplett/rfcs/blob/crabi-v1/text/3470-crabi-v1.md

Le 31/10/2024 à 19:15, Lyle Kopnicky a écrit :
> Not to mention name mangling, since method names do not export as 
> simple C function names. (Not unlike GHC’s Z-encoding.) I think the 
> way that’s usually dealt with is to have a C wrapper and declare it as 
> extern “C”?
>
> On Thu, Oct 31, 2024 at 11:25 AM Ben Gamari <ben at smart-cactus.org> wrote:
>
>     Hécate via ghc-devs <ghc-devs at haskell.org> writes:
>
>     > Hi devs,
>     >
>     > Pardon me for the naïve question. I know that C++ FFI is *hard* in
>     > Haskell. From the perspective of an end-user I have heard that
>     `text`'s
>     > new UTF-8 validation caused problems when released, as it relied
>     on C++
>     > code, but I never had any problems with it personally. From the GHC
>     > perspective, what makes C++ a difficult language to interface with?
>     >
>     There are a few reasons for this. First, there are considerations
>     due to
>     the language itself:
>
>      * C++ has an object system which our binding generators do not
>        currently make any attempt to capture. Consequently, developing
>        bindings to C++ libraries written in an object-oriented style can
>        require a fair amount of work.
>
>      * the prevalence of C++'s template system means that one may need to
>        generate one or more C++ snippets to instantiate an interface
>     before
>        one can even begin thinking about binding to the interface.
>     This can
>        be particularly tricky in libraries where you may want polymorphism
>        in the Haskell binding to be reflected in the C++ instantiation.
>
>      * Dealing with C++'s exception system requires great care when
>        developing bindings.
>
>      * The language itself is otherwise vast in scope, with numerous
>        features that don't play well with others. Thankfully, usually C++
>        library authors who intend for their work to be bound by others
>     often
>        restrict themselves to easily-bound features in their
>     outward-facing
>        interfaces.
>
>     Perhaps more significantly, there are also a variety of practical
>     considerations:
>
>      * there are three implementations of the C++ standard library in
>     common
>        use today (libstdc++, libc++, MSVC). Determining which library
>     should be
>        used on a particular platform, and subsequently *how* to
>     compile/link
>        against it, is quite non-trivial (e.g. see #20010). The
>        `system-cxx-std-lib` meta-package introduced in GHC 9.2 was aimed
>        at improving this situation for Haskell packages by making GHC
>        responsible for determining this configuration in such a way that
>        users can easily depend upon.
>
>      * even once you have compiled your program, linking against the C++
>        standard library introduces a dynamic library dependency on most
>        platforms. This is problematic as C++ standard library availability
>        and installation path is not nearly as consistent as the C standard
>        library. This is particularly problematic on Windows, where dynamic
>        linking is already fraught and non-MSVC runtime dependencies are
>        not widely available (e.g. [1])
>
>      * Code generated by C++ compilers tends to use less-common relocation
>        types, uncovering new and exciting ways for GHC's RTS linker to
>     crash
>        (e.g. see #21618)
>
>      * To make matters worse, C++11 introduced an ABI breakage to optimise
>        the representation of `std::string`. This was in the past a
>     source of
>        linking failures due to linking differently-compiled objects
>     into the
>        same library/executable. Thankfully most people have moved to
>     the new
>        ABI at this point so it's rare to see this manifest in the wild
>        except when linking against ancient proprietary binaries.
>
>     All-in-all, the difficulty is just death by a thousand cuts, often due
>     to platform-dependent toolchain issues, many even entirely independent
>     of Haskell.
>
>     Does this help?
>
>     - Ben
>
>
>     [1] https://github.com/haskell/ghcup-hs/issues/745
>     _______________________________________________
>     ghc-devs mailing list
>     ghc-devs at haskell.org
>     http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-- 
Hécate ✨
🐦: @TechnoEmpress
IRC: Hecate
WWW:https://glitchbra.in
RUN: BSD
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241101/628d69b6/attachment.html>

From julytreee at gmail.com  Sun Nov  3 08:37:09 2024
From: julytreee at gmail.com (Lei Zhu)
Date: Sun, 3 Nov 2024 16:37:09 +0800
Subject: Allow `base < 5` for package `array`
Message-ID: <CAGfc3OtzHGKqN3-6RtwA2Q_iMHMQLZjd5=mQQ80fGSON5R5eYg@mail.gmail.com>

Hi,

I'm preparing the next `array` release for the coming ghc 9.12.
The process seems boring because I just need to loosen the
upper bound of the `base`, and then make a release.

Shall I have a larger range like `base < 5` or are there any
restriction in the `base`, so we have to bump the version
one by one?

Thanks,
Lei Zhu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241103/b27eeffd/attachment.html>

From zubin at well-typed.com  Sun Nov  3 11:40:20 2024
From: zubin at well-typed.com (Zubin Duggal)
Date: Sun, 3 Nov 2024 17:10:20 +0530
Subject: Allow `base < 5` for package `array`
In-Reply-To: <CAGfc3OtzHGKqN3-6RtwA2Q_iMHMQLZjd5=mQQ80fGSON5R5eYg@mail.gmail.com>
References: <CAGfc3OtzHGKqN3-6RtwA2Q_iMHMQLZjd5=mQQ80fGSON5R5eYg@mail.gmail.com>
Message-ID: <6rp5i5anpp5my7tlansagwqfay3jbnhuwq5t5gucn62gcdydoc@uufuj53d3fso>

Hi,

It is best only only allow versions that are known to compile.
Breaking changes in `base` might break the build otherwise.
In this case, as no other changes are needed, we don't need
a new release of `array`. A hackage revision of array-0.5.8.0,
bumping the base bound to allow base-4.21 will be sufficient.

Along with this, I would appreciate if a commit bumping the bound
in the cabal file was also pushed to the repository, with an
appropriate tag like 0.5.8.0-r2 (as there already seems to be
an r1 revision).

Thanks,
Zubin

On 24/11/03 16:37, Lei Zhu wrote:
>Hi,
>
>I'm preparing the next `array` release for the coming ghc 9.12.
>The process seems boring because I just need to loosen the
>upper bound of the `base`, and then make a release.
>
>Shall I have a larger range like `base < 5` or are there any
>restriction in the `base`, so we have to bump the version
>one by one?
>
>Thanks,
>Lei Zhu

>_______________________________________________
>ghc-devs mailing list
>ghc-devs at haskell.org
>http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


From julytreee at gmail.com  Sun Nov  3 12:11:47 2024
From: julytreee at gmail.com (Lei Zhu)
Date: Sun, 3 Nov 2024 20:11:47 +0800
Subject: ghc-devs Digest, Vol 255, Issue 3
In-Reply-To: <mailman.3.1730635201.7038.ghc-devs@haskell.org>
References: <mailman.3.1730635201.7038.ghc-devs@haskell.org>
Message-ID: <CAGfc3Ou4c-DU5H=xB9bfzjAOGTsQB1mkOx4Od+Lr_TmO-S+BEw@mail.gmail.com>

Hi Zubin,

I'll do the r2 release as you advised.

Thanks,
Lei Zhu

On Sun, Nov 3, 2024 at 8:02 PM <ghc-devs-request at haskell.org> wrote:

> Send ghc-devs mailing list submissions to
>         ghc-devs at haskell.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> or, via email, send a message with subject or body 'help' to
>         ghc-devs-request at haskell.org
>
> You can reach the person managing the list at
>         ghc-devs-owner at haskell.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of ghc-devs digest..."
>
>
> Today's Topics:
>
>    1. Allow `base < 5` for package `array` (Lei Zhu)
>    2. Re: Allow `base < 5` for package `array` (Zubin Duggal)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sun, 3 Nov 2024 16:37:09 +0800
> From: Lei Zhu <julytreee at gmail.com>
> To: ghc-devs at haskell.org
> Subject: Allow `base < 5` for package `array`
> Message-ID:
>         <CAGfc3OtzHGKqN3-6RtwA2Q_iMHMQLZjd5=
> mQQ80fGSON5R5eYg at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi,
>
> I'm preparing the next `array` release for the coming ghc 9.12.
> The process seems boring because I just need to loosen the
> upper bound of the `base`, and then make a release.
>
> Shall I have a larger range like `base < 5` or are there any
> restriction in the `base`, so we have to bump the version
> one by one?
>
> Thanks,
> Lei Zhu
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mail.haskell.org/pipermail/ghc-devs/attachments/20241103/b27eeffd/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 2
> Date: Sun, 3 Nov 2024 17:10:20 +0530
> From: Zubin Duggal <zubin at well-typed.com>
> To: Lei Zhu <julytreee at gmail.com>
> Cc: ghc-devs at haskell.org
> Subject: Re: Allow `base < 5` for package `array`
> Message-ID:
>         <6rp5i5anpp5my7tlansagwqfay3jbnhuwq5t5gucn62gcdydoc at uufuj53d3fso>
> Content-Type: text/plain; charset=us-ascii; format=flowed
>
> Hi,
>
> It is best only only allow versions that are known to compile.
> Breaking changes in `base` might break the build otherwise.
> In this case, as no other changes are needed, we don't need
> a new release of `array`. A hackage revision of array-0.5.8.0,
> bumping the base bound to allow base-4.21 will be sufficient.
>
> Along with this, I would appreciate if a commit bumping the bound
> in the cabal file was also pushed to the repository, with an
> appropriate tag like 0.5.8.0-r2 (as there already seems to be
> an r1 revision).
>
> Thanks,
> Zubin
>
> On 24/11/03 16:37, Lei Zhu wrote:
> >Hi,
> >
> >I'm preparing the next `array` release for the coming ghc 9.12.
> >The process seems boring because I just need to loosen the
> >upper bound of the `base`, and then make a release.
> >
> >Shall I have a larger range like `base < 5` or are there any
> >restriction in the `base`, so we have to bump the version
> >one by one?
> >
> >Thanks,
> >Lei Zhu
>
> >_______________________________________________
> >ghc-devs mailing list
> >ghc-devs at haskell.org
> >http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
> ------------------------------
>
> End of ghc-devs Digest, Vol 255, Issue 3
> ****************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241103/21084155/attachment.html>

From davean at xkcd.com  Mon Nov  4 16:56:43 2024
From: davean at xkcd.com (davean)
Date: Mon, 4 Nov 2024 11:56:43 -0500
Subject: Allow `base < 5` for package `array`
In-Reply-To: <6rp5i5anpp5my7tlansagwqfay3jbnhuwq5t5gucn62gcdydoc@uufuj53d3fso>
References: <CAGfc3OtzHGKqN3-6RtwA2Q_iMHMQLZjd5=mQQ80fGSON5R5eYg@mail.gmail.com>
 <6rp5i5anpp5my7tlansagwqfay3jbnhuwq5t5gucn62gcdydoc@uufuj53d3fso>
Message-ID: <CABFs2Vjf+-dfCVXZMzXVUyQwA2_+N7XEO_mhovjtEOPViZtfuA@mail.gmail.com>

Lose upper bounds like that cause quite a bit of issues. It's really best
to use the revision system hackage has, like Zubin says.

-davean


On Sun, Nov 3, 2024 at 6:40 AM Zubin Duggal <zubin at well-typed.com> wrote:

> Hi,
>
> It is best only only allow versions that are known to compile.
> Breaking changes in `base` might break the build otherwise.
> In this case, as no other changes are needed, we don't need
> a new release of `array`. A hackage revision of array-0.5.8.0,
> bumping the base bound to allow base-4.21 will be sufficient.
>
> Along with this, I would appreciate if a commit bumping the bound
> in the cabal file was also pushed to the repository, with an
> appropriate tag like 0.5.8.0-r2 (as there already seems to be
> an r1 revision).
>
> Thanks,
> Zubin
>
> On 24/11/03 16:37, Lei Zhu wrote:
> >Hi,
> >
> >I'm preparing the next `array` release for the coming ghc 9.12.
> >The process seems boring because I just need to loosen the
> >upper bound of the `base`, and then make a release.
> >
> >Shall I have a larger range like `base < 5` or are there any
> >restriction in the `base`, so we have to bump the version
> >one by one?
> >
> >Thanks,
> >Lei Zhu
>
> >_______________________________________________
> >ghc-devs mailing list
> >ghc-devs at haskell.org
> >http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241104/4d05a939/attachment.html>

From zubin at well-typed.com  Tue Nov  5 12:13:24 2024
From: zubin at well-typed.com (Zubin Duggal)
Date: Tue, 5 Nov 2024 17:43:24 +0530
Subject: [ANNOUNCE] GHC 9.12.1-alpha2 is now available
In-Reply-To: <2snpei5s6aapoanssr266h5idtgjcfsrojdiso6unuszkhdeih@7vyl6vfi2wzq>
References: <2snpei5s6aapoanssr266h5idtgjcfsrojdiso6unuszkhdeih@7vyl6vfi2wzq>
Message-ID: <idkawcjbatbqdpogomxlmvyiuvlkxbjxcpjxtdgtluerdruosv@s6wkkxyx7g32>

The GHC developers are very pleased to announce the availability
of the second alpha release of GHC 9.12.1. Binary distributions, source
distributions, and documentation are available at [downloads.haskell.org][].


We hope to have this release available via ghcup shortly.

GHC 9.12 will bring a number of new features and improvements, including:

  * The new language extension [OrPatterns] allowing you to combine multiple
    pattern clauses into one.

  * The [MultilineStrings] language extension to allow you to more easily write
    strings spanning multiple lines in your source code.

  * Improvements to the OverloadedRecordDot extension, allowing the built-in
    `HasField` class to be used for records with fields of non lifted representations.

  * The [NamedDefaults] language extension has been introduced allowing you to
    define defaults for typeclasses other than `Num`.

  * More deterministic object code output, controlled by the `-fobject-determinism`
    flag, which improves determinism of builds a lot (though does not fully do so)
    at the cost of some compiler performance (1-2%). See #12935 for the details

  * GHC now accepts type syntax in expressions as part of [GHC Proposal #281].

  * The WASM backend now has support for TemplateHaskell.

  * ... and many more

A full accounting of changes can be found in the [release notes][].
As always, GHC's release status, including planned future releases, can
be found on the GHC Wiki [status][].

We would like to thank GitHub, IOG, the Zw3rk stake pool,
Well-Typed, Tweag I/O, Serokell, Equinix, SimSpace, the Haskell
Foundation, and other anonymous contributors whose on-going financial
and in-kind support has facilitated GHC maintenance and release
management over the years. Finally, this release would not have been
possible without the hundreds of open-source contributors whose work
comprise this release.

As always, do give this release a try and open a [ticket][] if you see
anything amiss.


[release notes]: https://downloads.haskell.org/ghc/9.12.1-alpha2/docs/users_guide/9.12.1-notes.html
[status]: https://gitlab.haskell.org/ghc/ghc/-/wikis/GHC-status
[downloads.haskell.org]: https://downloads.haskell.org/ghc/9.12.1-alpha2
[ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new
[OrPatterns]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0522-or-patterns.rst
[MultilineStrings]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0569-multiline-strings.rst
[GHC Proposal #281]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0281-visible-forall.rst
[NamedDefaults]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0409-exportable-named-default.rst


Cheers,
Zubin


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241105/bd480099/attachment.sig>

From kazu at iij.ad.jp  Tue Nov  5 23:06:27 2024
From: kazu at iij.ad.jp (Kazu Yamamoto (=?iso-2022-jp?B?GyRCOzNLXE9CSScbKEI=?=))
Date: Wed, 06 Nov 2024 08:06:27 +0900 (JST)
Subject: giving a label to a thread
Message-ID: <20241106.080627.1405807136676457805.kazu@iij.ad.jp>

Hi all,

GHC 9.6 provides listThreads.  This is very useful to check thread
leaks on network servers.  Thanks.

Users cannot label threads if fundamental functions fork threads
without labelling.  A good example is "timeout".  If there are many
threads without label, the debugging gets harder.

So, I would like to ask to label threads in libraries shipped with
GHC.

--Kazu



From sylvain at haskus.fr  Wed Nov  6 08:03:01 2024
From: sylvain at haskus.fr (Sylvain Henry)
Date: Wed, 6 Nov 2024 09:03:01 +0100
Subject: giving a label to a thread
In-Reply-To: <20241106.080627.1405807136676457805.kazu@iij.ad.jp>
References: <20241106.080627.1405807136676457805.kazu@iij.ad.jp>
Message-ID: <c694a582-6bf0-4340-93e1-8c68f35f4840@haskus.fr>

Hi,

Good idea. I've opened https://gitlab.haskell.org/ghc/ghc/-/issues/25452 
to track this.

Sylvain


On 06/11/2024 00:06, Kazu Yamamoto (山本和彦) via ghc-devs wrote:
> Hi all,
>
> GHC 9.6 provides listThreads.  This is very useful to check thread
> leaks on network servers.  Thanks.
>
> Users cannot label threads if fundamental functions fork threads
> without labelling.  A good example is "timeout".  If there are many
> threads without label, the debugging gets harder.
>
> So, I would like to ask to label threads in libraries shipped with
> GHC.
>
> --Kazu
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From simon.peytonjones at gmail.com  Wed Nov  6 08:37:35 2024
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Wed, 6 Nov 2024 08:37:35 +0000
Subject: GHC CI
Message-ID: <CAJKmMz-MxaZs3=vDWrVQ3WQ+ky=kFQGTyk6TR2hiP-m024bUzw@mail.gmail.com>

Dear GHC devs

Is something stuck in CI?

https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13553 has been stalled
for 9 hrs.   I have not seen that before.

Thanks

Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241106/d17859bf/attachment.html>

From klebinger.andreas at gmx.at  Wed Nov  6 10:33:59 2024
From: klebinger.andreas at gmx.at (Andreas Klebinger)
Date: Wed, 6 Nov 2024 11:33:59 +0100
Subject: GHC CI
In-Reply-To: <CAJKmMz-MxaZs3=vDWrVQ3WQ+ky=kFQGTyk6TR2hiP-m024bUzw@mail.gmail.com>
References: <CAJKmMz-MxaZs3=vDWrVQ3WQ+ky=kFQGTyk6TR2hiP-m024bUzw@mail.gmail.com>
Message-ID: <6f08d5e7-2b69-4829-940d-e908419ff114@gmx.at>

Hi Simon,

I took a brief look and it might just be CI being strained from
ghc-9.12.1-alpha2
and related CI jobs.

It seems the number of pending jobs already went down slightly
between when I first looked at it and just now.
I would expect to resolve itself in the next few hours.
But I will take a closer look in a few hours if the issue persists.

Cheers
Andreas


Am 06/11/2024 um 09:37 schrieb Simon Peyton Jones:
> Dear GHC devs
>
> Is something stuck in CI?
>
> https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13553 has been
> stalled for 9 hrs.   I have not seen that before.
>
> Thanks
>
> Simon
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241106/ab08c8bc/attachment.html>

From terrorjack at type.dance  Wed Nov  6 11:55:03 2024
From: terrorjack at type.dance (Cheng Shao)
Date: Wed, 6 Nov 2024 12:55:03 +0100
Subject: GHC CI
In-Reply-To: <6f08d5e7-2b69-4829-940d-e908419ff114@gmx.at>
References: <CAJKmMz-MxaZs3=vDWrVQ3WQ+ky=kFQGTyk6TR2hiP-m024bUzw@mail.gmail.com>
 <6f08d5e7-2b69-4829-940d-e908419ff114@gmx.at>
Message-ID: <CAGegzNDaSrFwZVhqO1_q6tMsPus2=8F=rhew6f8F=+Tsagv7OQ@mail.gmail.com>

When we have a large queue of blocked pipelines, we should cancel the
validation/nightly pipeline on master since they'll consume the runner
resource for too long, and runners should be prioritized for marge batches
and regular MRs.

Cheers,
Cheng

On Wed, Nov 6, 2024 at 11:34 AM Andreas Klebinger via ghc-devs <
ghc-devs at haskell.org> wrote:

> Hi Simon,
>
> I took a brief look and it might just be CI being strained from
> ghc-9.12.1-alpha2
> and related CI jobs.
>
> It seems the number of pending jobs already went down slightly
> between when I first looked at it and just now.
> I would expect to resolve itself in the next few hours.
> But I will take a closer look in a few hours if the issue persists.
>
> Cheers
> Andreas
>
>
> Am 06/11/2024 um 09:37 schrieb Simon Peyton Jones:
>
> Dear GHC devs
>
> Is something stuck in CI?
>
> https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13553 has been
> stalled for 9 hrs.   I have not seen that before.
>
> Thanks
>
> Simon
>
> _______________________________________________
> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241106/c3784227/attachment.html>

From oleg.grenrus at iki.fi  Wed Nov  6 12:43:54 2024
From: oleg.grenrus at iki.fi (Oleg Grenrus)
Date: Wed, 6 Nov 2024 14:43:54 +0200
Subject: GHC CI
In-Reply-To: <CAGegzNDaSrFwZVhqO1_q6tMsPus2=8F=rhew6f8F=+Tsagv7OQ@mail.gmail.com>
References: <CAJKmMz-MxaZs3=vDWrVQ3WQ+ky=kFQGTyk6TR2hiP-m024bUzw@mail.gmail.com>
 <6f08d5e7-2b69-4829-940d-e908419ff114@gmx.at>
 <CAGegzNDaSrFwZVhqO1_q6tMsPus2=8F=rhew6f8F=+Tsagv7OQ@mail.gmail.com>
Message-ID: <8a9d5e45-6322-45ea-97de-c2867f462ddd@iki.fi>

OTOH, if nightly pipeline is cancelled for a week in succession, and 
*then* it fails, we are screwed.

In my experience failing marge jobs is what consumes a lot of CI time.

In summary, until there is are analytics where the CI time is actually 
spent, it's at best guesses, at worst wrong guesses.

- Oleg

On 6.11.2024 13.55, Cheng Shao wrote:
> When we have a large queue of blocked pipelines, we should cancel the 
> validation/nightly pipeline on master since they'll consume the runner 
> resource for too long, and runners should be prioritized for marge 
> batches and regular MRs.
>
> Cheers,
> Cheng
>
> On Wed, Nov 6, 2024 at 11:34 AM Andreas Klebinger via ghc-devs 
> <ghc-devs at haskell.org> wrote:
>
>     Hi Simon,
>
>     I took a brief look and it might just be CI being strained from
>     ghc-9.12.1-alpha2
>     and related CI jobs.
>
>     It seems the number of pending jobs already went down slightly
>     between when I first looked at it and just now.
>     I would expect to resolve itself in the next few hours.
>     But I will take a closer look in a few hours if the issue persists.
>
>     Cheers
>     Andreas
>
>
>     Am 06/11/2024 um 09:37 schrieb Simon Peyton Jones:
>>     Dear GHC devs
>>
>>     Is something stuck in CI?
>>
>>     https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13553 has
>>     been stalled for 9 hrs.   I have not seen that before.
>>
>>     Thanks
>>
>>     Simon
>>
>>     _______________________________________________
>>     ghc-devs mailing list
>>     ghc-devs at haskell.org
>>     http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>     _______________________________________________
>     ghc-devs mailing list
>     ghc-devs at haskell.org
>     http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241106/19f22486/attachment.html>

From dashiellwill at 163.com  Thu Nov  7 01:12:56 2024
From: dashiellwill at 163.com (DashiellWill)
Date: Thu, 7 Nov 2024 09:12:56 +0800 (GMT+08:00)
Subject: GitLab Account Approval
Message-ID: <541f3eb4.7e596.193042e7e29.Coremail.dashiellwill@163.com>

I'm looking to get my gitlab account approved. user name: dashiell
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241107/9bca9ad3/attachment.html>

From dashiellwill at 163.com  Thu Nov  7 01:23:43 2024
From: dashiellwill at 163.com (DashiellWill)
Date: Thu, 7 Nov 2024 09:23:43 +0800 (GMT+08:00)
Subject: Haskeline test failed in GHC 9.4.5
Message-ID: <2e3abb21.7e69e.19304385f5a.Coremail.dashiellwill@163.com>

Hi all,




I'm currently getting an error while testing the Haskeline library via `cabal test`.  (GHC version: 9.4.5, Platform: x86_64)

I get this:

        tests/RunTTY.hs:106

        expected: ["tests/dummy-\206\188\206\177\207\131/\r\n\207\130\206\181\207\129\207\132  bar\r\n0:tests/dummy-\206\188\206\177\207\131/"]

         but got: ["tests/dummy-\206\188\206\177\207\131/\r\nbar   \207\130\206\181\207\129\207\132\r\n0:tests/dummy-\206\188\206\177\207\131/"]

 

The specific test in question is related to tab completion. The test case is defined as follows:

        tabCompletion :: Invocation -> Test  

        tabCompletion i = "tab completion" ~:  

            [ utf8Test i [ utf8 "tests/dummy-μ\t\t" ]  

                [ prompt 0, utf8 "tests/dummy-μασ/"  

                    <> nl <> utf8 "ςερτ bar" <> nl  

                    <> prompt' 0 <> utf8 "tests/dummy-μασ/"  

                ]  

            ]

 

It seems that the functionality of the TabCompletion is correct, but the order of the displayed files

after tab completion does not match the expected output. 

Then, I ran the `haskeline-examples-Test` program directly and manually typed "tests/dummy-μ\t\t".

The results were indeed inconsistent with the expected output:

        0:tests/dummy-μασ/

        bar ςερτ

        0:tests/dummy-μασ/

 

Could anyone shed some light on the potential cause of this issue?

Is there a problem with the expected value? For example,  there are variations in the expected output across different environments?

 

Thanks,

Dashiell
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241107/4d40cd5e/attachment.html>

From ben at smart-cactus.org  Fri Nov  8 16:50:09 2024
From: ben at smart-cactus.org (Ben Gamari)
Date: Fri, 08 Nov 2024 11:50:09 -0500
Subject: GHC CI
In-Reply-To: <CAJKmMz-MxaZs3=vDWrVQ3WQ+ky=kFQGTyk6TR2hiP-m024bUzw@mail.gmail.com>
References: <CAJKmMz-MxaZs3=vDWrVQ3WQ+ky=kFQGTyk6TR2hiP-m024bUzw@mail.gmail.com>
Message-ID: <87h68hy8jl.fsf@smart-cactus.org>

Simon Peyton Jones <simon.peytonjones at gmail.com> writes:

> Dear GHC devs
>
> Is something stuck in CI?
>
> https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13553 has been stalled
> for 9 hrs.   I have not seen that before.
>
Late last week I had to take one of my personal runners down due to a
hardware failure. As it is one of the larger machines in the runner
fleet (in terms of concurrent jobs), I suspect that CI may be struggling
a bit more than usual. I intend on replacing the machine in question in
the near future.

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241108/166bc25f/attachment.sig>

From simon.peytonjones at gmail.com  Fri Nov  8 21:59:45 2024
From: simon.peytonjones at gmail.com (Simon Peyton Jones)
Date: Fri, 8 Nov 2024 21:59:45 +0000
Subject: Lint performance
Message-ID: <CAJKmMz8VnLuf8Yar55qMn1FEODQ9hiJqw2RyJBsQ5RGvrPnPnQ@mail.gmail.com>

Friends

-dcore-lint is switched off in our compile-time tests
testsuite/tests/perf/compiler

But I'm making some changes whose primary effect will be on Lint.   How can
I get a before-vs-after analysis of the tess in perf/compiler, with
-dcore-lint switched on?

Locally would be fine.. does not have to be in CI

Thanks!

Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241108/9768b83d/attachment.html>

From ben at smart-cactus.org  Fri Nov  8 23:26:41 2024
From: ben at smart-cactus.org (Ben Gamari)
Date: Fri, 08 Nov 2024 18:26:41 -0500
Subject: Lint performance
In-Reply-To: <CAJKmMz8VnLuf8Yar55qMn1FEODQ9hiJqw2RyJBsQ5RGvrPnPnQ@mail.gmail.com>
References: <CAJKmMz8VnLuf8Yar55qMn1FEODQ9hiJqw2RyJBsQ5RGvrPnPnQ@mail.gmail.com>
Message-ID: <87cyj5xq6p.fsf@smart-cactus.org>

Simon Peyton Jones <simon.peytonjones at gmail.com> writes:

> Friends
>
> -dcore-lint is switched off in our compile-time tests
> testsuite/tests/perf/compiler
>
> But I'm making some changes whose primary effect will be on Lint.   How can
> I get a before-vs-after analysis of the tess in perf/compiler, with
> -dcore-lint switched on?
>
I'm afraid we don't have a good way of doing that without modifying the
.T file, which explicitly disables linting. The following patch will
drop the appropriate line:

    diff --git a/testsuite/tests/perf/compiler/all.T b/testsuite/tests/perf/compiler/all.T
    index 82390ff27e0..61092d2e1e1 100644
    --- a/testsuite/tests/perf/compiler/all.T
    +++ b/testsuite/tests/perf/compiler/all.T
    @@ -1,7 +1,3 @@
    -# Tests that call 'collect_compiler_stats' are skipped when debugging is on.
    -# See testsuite/driver/testlib.py.
    -setTestOpts(no_lint)
    -
    test('T1969',
          [# expect_broken(12437),
          collect_compiler_residency(20),

With this patch, linting will be enabled for the perf/compiler tests
(since the default testsuite compiler arguments include -dcore-lint; see
Settings.Builders.RunTest.runTestGhcFlags).

Concretely, I would:

 1. run the testsuite as described on [1]
 2. apply the above patch and commit
 3. run the testsuite again as described on [1]
 4. compare the results using the `perf_notes.py` tool as described on [1]

Naturally, I'd be happy to answer any questions that arise.

Cheers,

- Ben


[1] https://gitlab.haskell.org/ghc/ghc/-/wikis/building/running-tests/performance-tests
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241108/110bb606/attachment.sig>

From zubin at well-typed.com  Mon Nov 11 08:50:59 2024
From: zubin at well-typed.com (Zubin Duggal)
Date: Mon, 11 Nov 2024 14:20:59 +0530
Subject: GHC 9.12 release schedule and submodule status
Message-ID: <57y5mkyy6rjdsspbbkzuvwiqpeangoxaz6ugmrcih7l5rc3rhc@34yxw2ea7uld>

Cc
Bcc:
Subject: 9.12 schedule and submodule status

Hi all,

I'm writing to your in your role as a maintainer of one of
the GHC boot packages.

This is an update on the 9.12 release schedule. We plan to
have the release out by the end of the month (ideally on
29/11/24), with an alpha/rc every week from now until the release is
finalised:

| Date              | Event    |
| ----------------- | -------- |
| Nov 15 2024       | alpha 3  |
| Nov 22 2024       | release candidate |
| Nov 29 2024       | final    |

Additionally, this is the final call for any submodule bumps
that may be desired. The current status of the submodules is described in this
spreadsheet, under the GHC 9.12 tab: https://docs.google.com/spreadsheets/d/19WeWjXPh8Q64qoe3fjs8AswpTHk6uLBD8WG-7R97a1U/edit?gid=1512690813#gid=1512690813

If any version that is not listed in the spreadsheet is
desired, please notify me and ensure that a release is
tagged and uploaded to hackage as soon as possible. I would
like to finalise the state of submodules by the first release candidate.

This email is individually addressed to the maintainers of
all the boot libraries, as best as we could collate such a
list. If you are aware of any changes to the list of GHC
boot packages maintainers, please let us know. 

We aim to have a more systematic process for such
communications with library maintainers for releases in the
future. Once we finalise such a process we will let you know.

Thanks for all your work, 
Zubin.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241111/ed9eddac/attachment.sig>

From bryan at haskell.foundation  Tue Nov 12 13:54:40 2024
From: bryan at haskell.foundation (Bryan Richter)
Date: Tue, 12 Nov 2024 15:54:40 +0200
Subject: GHC CI
In-Reply-To: <8a9d5e45-6322-45ea-97de-c2867f462ddd@iki.fi>
References: <CAJKmMz-MxaZs3=vDWrVQ3WQ+ky=kFQGTyk6TR2hiP-m024bUzw@mail.gmail.com>
 <6f08d5e7-2b69-4829-940d-e908419ff114@gmx.at>
 <CAGegzNDaSrFwZVhqO1_q6tMsPus2=8F=rhew6f8F=+Tsagv7OQ@mail.gmail.com>
 <8a9d5e45-6322-45ea-97de-c2867f462ddd@iki.fi>
Message-ID: <CACUTo6xdHhApzCvXuXrDJ2fTiHJ9A=LdLNbTuoy3zp9pSNzD6g@mail.gmail.com>

I have been told by Equinix representatives that if we built something that
dynamically scales up and down the number of machines we use, not only
would it *not* impact their bottom line very much, but it would actually be
*valuable* to Equinix. Their sponsorship of us comes out of their
sales/marketing budget. If we can create cool things they can show off
("look at the cool things you can do on our platform"), they would
appreciate it.

If I had more time to spend on DevOps, this is something I'd be interested
in building.

On Wed, 6 Nov 2024, 14.44 Oleg Grenrus, <oleg.grenrus at iki.fi> wrote:

> OTOH, if nightly pipeline is cancelled for a week in succession, and
> *then* it fails, we are screwed.
>
> In my experience failing marge jobs is what consumes a lot of CI time.
>
> In summary, until there is are analytics where the CI time is actually
> spent, it's at best guesses, at worst wrong guesses.
>
> - Oleg
> On 6.11.2024 13.55, Cheng Shao wrote:
>
> When we have a large queue of blocked pipelines, we should cancel the
> validation/nightly pipeline on master since they'll consume the runner
> resource for too long, and runners should be prioritized for marge batches
> and regular MRs.
>
> Cheers,
> Cheng
>
> On Wed, Nov 6, 2024 at 11:34 AM Andreas Klebinger via ghc-devs <
> ghc-devs at haskell.org> wrote:
>
>> Hi Simon,
>>
>> I took a brief look and it might just be CI being strained from
>> ghc-9.12.1-alpha2
>> and related CI jobs.
>>
>> It seems the number of pending jobs already went down slightly
>> between when I first looked at it and just now.
>> I would expect to resolve itself in the next few hours.
>> But I will take a closer look in a few hours if the issue persists.
>>
>> Cheers
>> Andreas
>>
>>
>> Am 06/11/2024 um 09:37 schrieb Simon Peyton Jones:
>>
>> Dear GHC devs
>>
>> Is something stuck in CI?
>>
>> https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13553 has been
>> stalled for 9 hrs.   I have not seen that before.
>>
>> Thanks
>>
>> Simon
>>
>> _______________________________________________
>> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
> _______________________________________________
> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241112/869b62a3/attachment.html>

From hecate at glitchbra.in  Wed Nov 13 12:39:46 2024
From: hecate at glitchbra.in (=?UTF-8?Q?H=C3=A9cate?=)
Date: Wed, 13 Nov 2024 13:39:46 +0100
Subject: Under which circumstances are the elements of a linked list laid next
 to each-other in memory?
Message-ID: <92b176a1-b8c6-4bac-8021-58bd60e37b86@glitchbra.in>

Hi everyone,

I was asked at work today why I went for Vector instead plain old List 
in order to store an API result that is quite small.

While writing my reasons (pointer indirection, conversion to Aeson's 
internal representation of a JSON array using Vector), I remember that I 
heard that in some cases, GHC would be able to lay down in memory the 
elements of a linked list next to each other. I don't think this gets 
rid of the pointer-chasing, but I'm interested to know more about the 
circumstances in which linked lists are optimised.

Does anyone know if there is a wiki page or something that I could read 
on the topic of linked list optimisation **other than fusion**?

Cheers,
Hécate

-- 
Hécate ✨
🐦: @TechnoEmpress
IRC: Hecate
WWW: https://glitchbra.in
RUN: BSD


From ietf-dane at dukhovni.org  Wed Nov 13 13:47:34 2024
From: ietf-dane at dukhovni.org (Viktor Dukhovni)
Date: Thu, 14 Nov 2024 00:47:34 +1100
Subject: Under which circumstances are the elements of a linked list laid
 next to each-other in memory?
In-Reply-To: <92b176a1-b8c6-4bac-8021-58bd60e37b86@glitchbra.in>
References: <92b176a1-b8c6-4bac-8021-58bd60e37b86@glitchbra.in>
Message-ID: <ZzSt9vy7uvrzMipJ@chardros.imrryr.org>

On Wed, Nov 13, 2024 at 01:39:46PM +0100, Hécate via ghc-devs wrote:

> I was asked at work today why I went for Vector instead plain old List in
> order to store an API result that is quite small.
> 
> While writing my reasons (pointer indirection, conversion to Aeson's
> internal representation of a JSON array using Vector), I remember that I
> heard that in some cases, GHC would be able to lay down in memory the
> elements of a linked list next to each other. I don't think this gets rid of
> the pointer-chasing, but I'm interested to know more about the circumstances
> in which linked lists are optimised.

An intesting question to which I have only a very partial answer.

If the linked list a `String`, and it is actually stored in memory as a
packed UTF-8 array of bytes (modified for C-friendly NUL-termination by
representing \x00 as the denormalised \xc000), then its "list" form is
really just a lazy computation over the byte array, which if not used
repeatedly, but rather consumed just once, would remain a
lazily-evaluation iterator over the stored byte array.

Similar considerations might apply in any situation where the list
elements are "Storable", (perhaps even unboxed) and the list is really
an iterator over the Storable or Unboxed array.

In simple enough situations, where you're not using the fancier features
of the `Vector` API, you might also find that the IArray, MArray,
UArray, STUArray set of types meets your needs.

I'm eager to hear about new contexts in which one might expect a list to
be automatically represented by some densely packed pre-computed array,
other than by explicit prior construction of such an array, for which
the list is an iterator.  At the very list the list elements would have
to be known to all be strictly evaluated, and precomputation to store
the entire list would have to be justified.

-- 
    Viktor.

From klebinger.andreas at gmx.at  Wed Nov 13 15:50:38 2024
From: klebinger.andreas at gmx.at (Andreas Klebinger)
Date: Wed, 13 Nov 2024 16:50:38 +0100
Subject: Under which circumstances are the elements of a linked list laid
 next to each-other in memory?
In-Reply-To: <92b176a1-b8c6-4bac-8021-58bd60e37b86@glitchbra.in>
References: <92b176a1-b8c6-4bac-8021-58bd60e37b86@glitchbra.in>
Message-ID: <21d15b6c-3e36-42ef-9da4-c1dec8e4703c@gmx.at>

I think what you are remembering is "just" a effect of copying GC. As
you said that won't get rid of the pointer chasing nor magically
transform the result into some kind of dense structure internally.

Am 13/11/2024 um 13:39 schrieb Hécate via ghc-devs:
> I remember that I heard that in some cases, GHC would be able to lay
> down in memory the elements of a linked list next to each other. I
> don't think this gets rid of the pointer-chasing, but I'm interested
> to know more about the circumstances in which linked lists are optimised.

From ben at smart-cactus.org  Wed Nov 13 17:27:56 2024
From: ben at smart-cactus.org (Ben Gamari)
Date: Wed, 13 Nov 2024 12:27:56 -0500
Subject: Under which circumstances are the elements of a linked list
 laid next to each-other in memory?
In-Reply-To: <92b176a1-b8c6-4bac-8021-58bd60e37b86@glitchbra.in>
References: <92b176a1-b8c6-4bac-8021-58bd60e37b86@glitchbra.in>
Message-ID: <877c97vyav.fsf@smart-cactus.org>

Hécate via ghc-devs <ghc-devs at haskell.org> writes:

> Hi everyone,
>
> I was asked at work today why I went for Vector instead plain old List 
> in order to store an API result that is quite small.
>
> While writing my reasons (pointer indirection, conversion to Aeson's 
> internal representation of a JSON array using Vector), I remember that I 
> heard that in some cases, GHC would be able to lay down in memory the 
> elements of a linked list next to each other. I don't think this gets 
> rid of the pointer-chasing, but I'm interested to know more about the 
> circumstances in which linked lists are optimised.
>
As Andreas suggests, you are likely thinking of the the fact that
copying GC tends to re-layout the heap such that topologically proximate
closures also tend to be nearby in memory. This result in a considerable
improvement in cache locality over traditionally non-moving allocators
although, as you point out, the pointer chasing cost and representation
size of a cons cell with small payload is still considerable compared to
a dense representation.

I don't think the wiki has any discussion of this but, e.g., Steve
Blackburn's Immix paper [1] does have some representative measurements
of the effect of non-moving (mark-sweep) and compacting (mark-compact
and Immix) allocation schemes on mutator performance.

Cheers,

- Ben


[1] https://www.steveblackburn.org/pubs/papers/immix-pldi-2008.pdf
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241113/1d0419a6/attachment.sig>

From zubin at well-typed.com  Fri Nov 15 11:11:11 2024
From: zubin at well-typed.com (Zubin Duggal)
Date: Fri, 15 Nov 2024 16:41:11 +0530
Subject: [ANNOUNCE] GHC 9.12.1-alpha3 is now available
Message-ID: <wfgw34m5jmmgllla4ddacy52yi73phjfdg7flk5zkgs2h3kmic@sty2qwb4sxvj>

The GHC developers are very pleased to announce the availability
of the third alpha release of GHC 9.12.1. Binary distributions, source
distributions, and documentation are available at [downloads.haskell.org][].


We hope to have this release available via ghcup shortly.

GHC 9.12 will bring a number of new features and improvements, including:

  * The new language extension [OrPatterns] allowing you to combine multiple
    pattern clauses into one.

  * The [MultilineStrings] language extension to allow you to more easily write
    strings spanning multiple lines in your source code.

  * Improvements to the OverloadedRecordDot extension, allowing the built-in
    `HasField` class to be used for records with fields of non lifted representations.

  * The [NamedDefaults] language extension has been introduced allowing you to
    define defaults for typeclasses other than `Num`.

  * More deterministic object code output, controlled by the `-fobject-determinism`
    flag, which improves determinism of builds a lot (though does not fully do so)
    at the cost of some compiler performance (1-2%). See #12935 for the details

  * GHC now accepts type syntax in expressions as part of [GHC Proposal #281].

  * The WASM backend now has support for TemplateHaskell.

  * ... and many more

A full accounting of changes can be found in the [release notes][].
As always, GHC's release status, including planned future releases, can
be found on the GHC Wiki [status][].

We would like to thank GitHub, IOG, the Zw3rk stake pool,
Well-Typed, Tweag I/O, Serokell, Equinix, SimSpace, the Haskell
Foundation, and other anonymous contributors whose on-going financial
and in-kind support has facilitated GHC maintenance and release
management over the years. Finally, this release would not have been
possible without the hundreds of open-source contributors whose work
comprise this release.

As always, do give this release a try and open a [ticket][] if you see
anything amiss.


[release notes]: https://downloads.haskell.org/ghc/9.12.1-alpha3/docs/users_guide/9.12.1-notes.html
[status]: https://gitlab.haskell.org/ghc/ghc/-/wikis/GHC-status
[downloads.haskell.org]: https://downloads.haskell.org/ghc/9.12.1-alpha3
[ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new
[OrPatterns]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0522-or-patterns.rst
[MultilineStrings]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0569-multiline-strings.rst
[GHC Proposal #281]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0281-visible-forall.rst
[NamedDefaults]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0409-exportable-named-default.rst

Cheers,
Zubin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241115/1ffa553b/attachment.sig>

From harendra.kumar at gmail.com  Fri Nov 15 13:15:40 2024
From: harendra.kumar at gmail.com (Harendra Kumar)
Date: Fri, 15 Nov 2024 18:45:40 +0530
Subject: openFile gives "file is locked" error on Linux when creating a
 non-existing file
In-Reply-To: <ZwcLiPPnsM8OfTyh@chardros.imrryr.org>
References: <ZwO9Vj4uPv5axFsY@chardros.imrryr.org>
 <CAPW+kkZ480FyWABTh=ZOjANX59P0e0+2r8-Z_7UgS0Uivv24pg@mail.gmail.com>
 <ZwTPFOdJ9kFCInfr@chardros.imrryr.org>
 <CAPW+kkbcSFzYcQv-FwHDQww8P9CgWWLyDKp10_JcBoEZG4uV4w@mail.gmail.com>
 <ZwUI-EInD3b-569V@chardros.imrryr.org>
 <CAPW+kkZyActMm_C9VphFD-k6ezhLY7MwTRzvTvpO6GCED01KXg@mail.gmail.com>
 <ZwXyp29LdyxQlIqt@chardros.imrryr.org>
 <CAPW+kkYR2pdLUsrc1p5rSouBiMtOqUxtHbBPOyy7X2kH+nCGhQ@mail.gmail.com>
 <ZwYOOwnirs0lqI0-@chardros.imrryr.org>
 <CAPW+kkaspne_dqHTxQTcuvN1sFRYJh+V-P9swaw0vtaDmz7g9Q@mail.gmail.com>
 <ZwcLiPPnsM8OfTyh@chardros.imrryr.org>
Message-ID: <CAPW+kkayL0U4fo3r2HUTipKfBvEcyGJ5KQfxwKbF12aPXUQmxg@mail.gmail.com>

Coming back to this issue after a break. I reviewed the code carefully
and I cannot find anything where we are doing something in the
application code that affects the RTS locking mechanism. Let me walk
through the steps of the test up to failure and what we are doing in
the code. The test output is like this:

Ensuring dir: /tmp/fsevent_common-3725879775a96dfe/watch-root
createFile: Creating file: /tmp/fsevent_common-3725879775a96dfe/watch-root/file1
FileSystem.Event:
/tmp/fsevent_common-3725879775a96dfe/watch-root/file1: openFile:
resource busy (file is locked)

These steps are done before we start the file watching tests. We first
create a "watch-root" directory using createDirectory and then use
openFile to create a file named "file1" in that directory which we
just created, this openFile operation fails with "file is locked"
error. These steps are executed serially. The directory path is
unique, this path is never used before in this test or in any other
tests. This is the first time we are creating this file.

Now let me explain a bit about the custom file opening/closing logic
in the program.  In each test, first we create a watch fd using
inotify_init, and create a Handle from it:

watch_fd <- c_inotify_init

Note that the watch_fd does not belong to an actual file on the file
system, so locking is irrelevant for this fd. FWIW, we avoid the
lockFile code path in this case when creating the Handle from fd.

To watch a file or directory we add that file to the watch using something like:

 watch_descriptor <- c_inotify_add_watch watch_fd path flags

Note that we never create a GHC Handle from watch_descriptor. To
remove a file from the watch we do something like this:

rm = c_inotify_rm_watch watch_fd watch_descriptor

This closes the watch_descriptor returned by inotify_add_watch.

In any of these operations we have not inserted anything in the GHC
RTS locking table. In fact we are not even creating a Handle in any
case other than the watch_fd which does not even represent a file on
the file system and we skip the lockFile code when creating this
Handle.

After adding the file to the watch, we open the file, perform
read/write operations on it using the regular GHC file operations, and
then close the file. I am not sure if closing the watch_descriptor
interferes in any way with GHC's file descriptor which is open for
normal reading/writing. I assume the watch_descriptor returned by
inotify_add_watch operation on the file and the regular fd in the GHC
Handle returned by the "openFile" operation on the file are different.
So our closing the watch_descriptor should not affect the removal of
the lock entry by RTS when the file Handle is closed.

Regarding, how to debug this - I have not been able to reproduce this
on a local Linux machine, even when using ext4 as temp directory's
file system. On the local system I can see that inodes are being
reused all the time on ext4, but still the problem is not reproduced.
I ran the tests for several days and did not get a single instance of
the issue. The only way forward seems to be to create a GHC build
which logs the lock table entry and removal, and then run the CI using
that GHC build. Even in CIs I can reproduce the problem only on lucky
days.

-harendra

On Thu, 10 Oct 2024 at 04:32, Viktor Dukhovni <ietf-dane at dukhovni.org> wrote:
>
> On Wed, Oct 09, 2024 at 12:15:32PM +0530, Harendra Kumar wrote:
> > We do use low level C APIs and GHC APIs to create a Handle in the
> > event watching module. But that is for the watch-root and not for the
> > file that is experiencing this problem. So here is how it works. We
> > have a top level directory which is watched for events using inotify.
> > We first create this directory, this directory is opened using
> > inotify_init which returns a C file descriptor. We then create a
> > Handle from this fd, this Handle is used for watching inotify events.
> > We are then creating a file inside this directory which is being
> > watched while we are reading events from the parent directory. The
> > resource-busy issue occurs when creating a file inside this directory.
> > So we are not creating the Handle for the file in question in a
> > non-standard manner, but the parent directory Handle is being created
> > in that manner. I do not know if that somehow affects anything. Or if
> > the fact that the directory is being watched using inotify makes any
> > difference?
> >
> > The code for creating the watch Handle is here:
> > https://github.com/composewell/streamly/blob/bbac52d9e09fa5ad760ab6ee5572c701e198d4ee/src/Streamly/Internal/FileSystem/Event/Linux.hs#L589
> > . Viktor, you may want to take a quick look at this to see if it can
> > make any difference to the issue at hand.
>
> I don't have the cycles to isolate the problem.  I still suspect that
> your code is somehow directly closing file descriptors associated with a
> Handle.  This then orphans the associated logical reader/writer lock,
> which then gets inherited by the next incarnation of the same (dev, ino)
> pair.  However, if the filesystem underlying "/tmp" were actually "tmpfs",
> inode reuse would be quite unlikely, because tmpfs inodes are assigned
> from a strictly incrementing counter:
>
>     $ for i in {1..10}; do touch /tmp/foobar; ls -i /tmp/foobar; rm
>     /tmp/foobar; done
>     3830 /tmp/foobar
>     3831 /tmp/foobar
>     3832 /tmp/foobar
>     3833 /tmp/foobar
>     3834 /tmp/foobar
>     3835 /tmp/foobar
>     3836 /tmp/foobar
>     3837 /tmp/foobar
>     3838 /tmp/foobar
>     3839 /tmp/foobar
>
> but IIRC you mentioned that on Github "/tmp" is ext4, not "tmpfs"
> (perhaps RAM-backed storage is a more scarce resource), in which
> case indeed inode reuse is quite likely:
>
>     $ for i in {1..10}; do touch /var/tmp/foobar; ls -i /var/tmp/foobar; rm
>     /var/tmp/foobar; done
>     25854141 /var/tmp/foobar
>     25854142 /var/tmp/foobar
>     25854141 /var/tmp/foobar
>     25854142 /var/tmp/foobar
>     25854141 /var/tmp/foobar
>     25854142 /var/tmp/foobar
>     25854141 /var/tmp/foobar
>     25854142 /var/tmp/foobar
>     25854141 /var/tmp/foobar
>     25854142 /var/tmp/foobar
>
> But since normal open/close of Handles acquires the lock after open, and
> releases it before close, the evidence points to a bypass of the normal
> open file lifecycle.
>
> Your codebase contains a bunch of custom file management logic, which
> could be the source the of problem.  To find the problem code path,
> you'd probably need to instrument the RTS lock/unlock code to log its
> activity: (mode, descriptor, dev, ino) tuples being added and removed.
> And strace execution to be able to identify descriptor open and close
> events.  Ideally the problem will be reproducible even with strace.
>
> Good luck.
>
> --
>     Viktor.
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

From ietf-dane at dukhovni.org  Sat Nov 16 07:32:07 2024
From: ietf-dane at dukhovni.org (Viktor Dukhovni)
Date: Sat, 16 Nov 2024 18:32:07 +1100
Subject: openFile gives "file is locked" error on Linux when creating a
 non-existing file
In-Reply-To: <CAPW+kkayL0U4fo3r2HUTipKfBvEcyGJ5KQfxwKbF12aPXUQmxg@mail.gmail.com>
References: <ZwTPFOdJ9kFCInfr@chardros.imrryr.org>
 <CAPW+kkbcSFzYcQv-FwHDQww8P9CgWWLyDKp10_JcBoEZG4uV4w@mail.gmail.com>
 <ZwUI-EInD3b-569V@chardros.imrryr.org>
 <CAPW+kkZyActMm_C9VphFD-k6ezhLY7MwTRzvTvpO6GCED01KXg@mail.gmail.com>
 <ZwXyp29LdyxQlIqt@chardros.imrryr.org>
 <CAPW+kkYR2pdLUsrc1p5rSouBiMtOqUxtHbBPOyy7X2kH+nCGhQ@mail.gmail.com>
 <ZwYOOwnirs0lqI0-@chardros.imrryr.org>
 <CAPW+kkaspne_dqHTxQTcuvN1sFRYJh+V-P9swaw0vtaDmz7g9Q@mail.gmail.com>
 <ZwcLiPPnsM8OfTyh@chardros.imrryr.org>
 <CAPW+kkayL0U4fo3r2HUTipKfBvEcyGJ5KQfxwKbF12aPXUQmxg@mail.gmail.com>
Message-ID: <ZzhKd5ccnk5lP-pn@chardros.imrryr.org>

On Fri, Nov 15, 2024 at 06:45:40PM +0530, Harendra Kumar wrote:

> Coming back to this issue after a break. I reviewed the code carefully
> and I cannot find anything where we are doing something in the
> application code that affects the RTS locking mechanism. Let me walk
> through the steps of the test up to failure and what we are doing in
> the code. The test output is like this:

It is indeed not immediately clear where in your code or in some
dependency (including base, GHC, ...) a descriptor that contributes to
the RTS file reader/writer count (indexed by (dev, ino)) might be closed
without adjusting the count by calling the RTS `unlockFile()` function
(via GHC.IO.FD.release).

It may be worth noting that GHC does not *reliably* prevent simultaneous
open handles for the same underlying file, because handles returned by
hDuplicate do not contribute to the count:

    demo.hs:
        import GHC.IO.Handle (hDuplicate)
        import System.IO

        main :: IO ()
        main = do
            fh1 <- dupOpen "/tmp/foo"
            fh2 <- dupOpen "/tmp/foo"
            writeNow fh1 "abc\n"
            writeNow fh2 "def\n"
            readFile "/tmp/foo" >>= putStr
            hClose fh1
            hClose fh2
         where
            -- Look Mom, no lock!
            dupOpen path = do
                fh <- openFile path WriteMode
                hDuplicate fh <* hClose fh

            writeNow fh s = hPutStr fh s >> hFlush fh

    $ ghc -O demo.hs
    [1 of 2] Compiling Main             ( demo.hs, demo.o )
    [2 of 2] Linking demo
    $ ./demo
    def

I am not sure that Haskell really should be holding the application's
hand in this area, corrupting output files by concurrent writers can
just as easily happen by running two independent processes.  But letting
go of this guardrail would IIRC be a deviation from the Haskell report,
and there are likely applications that depend on this (and don't use
hDupicate or equivalent to break the reader/writer tracking).

-- 
    Viktor.

From zubin at well-typed.com  Mon Nov 11 10:54:17 2024
From: zubin at well-typed.com (Zubin Duggal)
Date: Mon, 11 Nov 2024 16:24:17 +0530
Subject: GHC 9.12 release schedule and submodule status
Message-ID: <ojmjtgj7ggrrve2mmzogbey2p3ocx3y3cnewmqvilpeplcse6r@r3452u6h5xhv>

Hi all,

(This is a resend of an email I sent a short while ago. I
seem to have messed up some email headers that might trip up
some filters so I'm resending it to be safe. You only need
to read one of these if you recieved my previous email)

I'm writing to your in your role as a maintainer of one of
the GHC boot packages.

This is an update on the 9.12 release schedule. We plan to
have the release out by the end of the month (ideally on
29/11/24), with an alpha/rc every week from now until the release is
finalised:

| Date              | Event    |
| ----------------- | -------- |
| Nov 15 2024       | alpha 3  |
| Nov 22 2024       | release candidate |
| Nov 29 2024       | final    |

Additionally, this is the final call for any submodule bumps
that may be desired. The current status of the submodules is described in this
spreadsheet, under the GHC 9.12 tab: https://docs.google.com/spreadsheets/d/19WeWjXPh8Q64qoe3fjs8AswpTHk6uLBD8WG-7R97a1U/edit?gid=1512690813#gid=1512690813

If any version that is not listed in the spreadsheet is
desired, please notify me and ensure that a release is
tagged and uploaded to hackage as soon as possible. I would
like to finalise the state of submodules by the first release candidate.

This email is individually addressed to the maintainers of
all the boot libraries, as best as we could collate such a
list. If you are aware of any changes to the list of GHC
boot packages maintainers, please let us know.

We aim to have a more systematic process for such
communications with library maintainers for releases in the
future. Once we finalise such a process we will let you know.

Thanks for all your work,
Zubin.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241111/837912f5/attachment.sig>

From kazu at iij.ad.jp  Wed Nov 20 01:16:46 2024
From: kazu at iij.ad.jp (Kazu Yamamoto (=?iso-2022-jp?B?GyRCOzNLXE9CSScbKEI=?=))
Date: Wed, 20 Nov 2024 10:16:46 +0900 (JST)
Subject: labeling MVar and STM data structures
Message-ID: <20241120.101646.1569348605073110397.kazu@iij.ad.jp>

Hello,

I'm seeking of the source of the bug that my server based on Warp gets
not-responding. "listThreads" shows that IOManager is blocked on a
MVar permanently:

2 IOManager on cap 0: ThreadBlocked BlockedOnForeignCall
3 IOManager on cap 1: ThreadBlocked BlockedOnMVar
4 TimerManager: ThreadBlocked BlockedOnForeignCall

IOManager should have the BlockedOnForeignCall reason since it's
calling epoll_wait().

It's quit hard to debug this bug since I don't understand which MVar blocks
IOManager of 3. Users can register MVar operations to IOManager while
IOManager itself is using MVars internally.

To make this kind of debugging easier, I would like ask GHC developers
to extend MVar to hold its label and provide "labelMVar" like
"labelThread". The same mechanism should be provided for STM as well.
"BlockReason" should contain the target label.

What do you think?

Background: https://github.com/haskell/network/issues/590

--Kazu



From matthewtpickering at gmail.com  Wed Nov 20 09:05:19 2024
From: matthewtpickering at gmail.com (Matthew Pickering)
Date: Wed, 20 Nov 2024 09:05:19 +0000
Subject: labeling MVar and STM data structures
In-Reply-To: <20241120.101646.1569348605073110397.kazu@iij.ad.jp>
References: <20241120.101646.1569348605073110397.kazu@iij.ad.jp>
Message-ID: <CALuQ0m_wV9P-PFidyJxE_wrhVJFEVfYZsUjyCb2=9WjkhqtnCQ@mail.gmail.com>

Hi Kazu,

This sounds like something you could investigate with ghc-debug. Once the
mvar blocks you can look at the stack using ghc-debug-brick which should be
enlightening.

Matt

On Wed, 20 Nov 2024, 01:17 Kazu Yamamoto (山本和彦) via ghc-devs, <
ghc-devs at haskell.org> wrote:

> Hello,
>
> I'm seeking of the source of the bug that my server based on Warp gets
> not-responding. "listThreads" shows that IOManager is blocked on a
> MVar permanently:
>
> 2 IOManager on cap 0: ThreadBlocked BlockedOnForeignCall
> 3 IOManager on cap 1: ThreadBlocked BlockedOnMVar
> 4 TimerManager: ThreadBlocked BlockedOnForeignCall
>
> IOManager should have the BlockedOnForeignCall reason since it's
> calling epoll_wait().
>
> It's quit hard to debug this bug since I don't understand which MVar blocks
> IOManager of 3. Users can register MVar operations to IOManager while
> IOManager itself is using MVars internally.
>
> To make this kind of debugging easier, I would like ask GHC developers
> to extend MVar to hold its label and provide "labelMVar" like
> "labelThread". The same mechanism should be provided for STM as well.
> "BlockReason" should contain the target label.
>
> What do you think?
>
> Background: https://github.com/haskell/network/issues/590
>
> --Kazu
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241120/e19c5eea/attachment.html>

From ben at smart-cactus.org  Thu Nov 21 04:51:52 2024
From: ben at smart-cactus.org (Ben Gamari)
Date: Wed, 20 Nov 2024 23:51:52 -0500
Subject: labeling MVar and STM data structures
In-Reply-To: <20241120.101646.1569348605073110397.kazu@iij.ad.jp>
References: <20241120.101646.1569348605073110397.kazu@iij.ad.jp>
Message-ID: <8734jlur2y.fsf@smart-cactus.org>

"Kazu Yamamoto \(山本和彦\) via ghc-devs" <ghc-devs at haskell.org> writes:

> Hello,
>
> I'm seeking of the source of the bug that my server based on Warp gets
> not-responding. "listThreads" shows that IOManager is blocked on a
> MVar permanently:
>
> 2 IOManager on cap 0: ThreadBlocked BlockedOnForeignCall
> 3 IOManager on cap 1: ThreadBlocked BlockedOnMVar
> 4 TimerManager: ThreadBlocked BlockedOnForeignCall
>
> IOManager should have the BlockedOnForeignCall reason since it's
> calling epoll_wait().
>
> It's quit hard to debug this bug since I don't understand which MVar blocks
> IOManager of 3. Users can register MVar operations to IOManager while
> IOManager itself is using MVars internally.
>
> To make this kind of debugging easier, I would like ask GHC developers
> to extend MVar to hold its label and provide "labelMVar" like
> "labelThread". The same mechanism should be provided for STM as well.
> "BlockReason" should contain the target label.
>
> What do you think?

This is generally in the direction I proposed in #21877; I agree that it
would be quite useful. I don't think that we necessarily want to extend
all MVars with labels as this would impose a fixed memory cost on all
users. However, I think the approach that I describe in the ticket,
introducing an out-of-band map for labelling heap objects, would be
a reasonable way forward.

As Matt suggests, ghc-debug is also a great tool for investigating
issues like this. However, I still think there is value in having better
support in the core libraries for understanding issues such as this.

Thoughts?

Cheers,

- Ben
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241120/b6a6efaf/attachment.sig>

From kazu at iij.ad.jp  Thu Nov 21 06:23:27 2024
From: kazu at iij.ad.jp (Kazu Yamamoto (=?iso-2022-jp?B?GyRCOzNLXE9CSScbKEI=?=))
Date: Thu, 21 Nov 2024 15:23:27 +0900 (JST)
Subject: labeling MVar and STM data structures
In-Reply-To: <8734jlur2y.fsf@smart-cactus.org>
References: <20241120.101646.1569348605073110397.kazu@iij.ad.jp>
 <8734jlur2y.fsf@smart-cactus.org>
Message-ID: <20241121.152327.480973038859334972.kazu@iij.ad.jp>

Hi Ben,

> This is generally in the direction I proposed in #21877; I agree that it
> would be quite useful. I don't think that we necessarily want to extend
> all MVars with labels as this would impose a fixed memory cost on all
> users. However, I think the approach that I describe in the ticket,
> introducing an out-of-band map for labelling heap objects, would be
> a reasonable way forward.

I support this.
Thanks.

--Kazu



From zubin at well-typed.com  Fri Nov 22 13:37:47 2024
From: zubin at well-typed.com (Zubin Duggal)
Date: Fri, 22 Nov 2024 19:07:47 +0530
Subject: GHC 9.12.1 updates and rc1 status
Message-ID: <euvthzd2yaqkhy4r46qrtkcat5uf4ptrx4ebkv42qh2blkd5sj@5lnfhlzchpfc>

Hi all,

We had planned to release 9.12.1-rc1 today. Unfortunately,
we discovered a few regressions and bugs which need to be fixed
before we can publish a release candidate, including #25382,
#25375 and !13633. I am currently waiting for some of these
fixes to land in master before backporting them to the release
branch.

I will try to have the release candidate out on Monday the 25th.

The final release is currently scheduled for December 6th. We
will inform you if there are any changes to this schedule.

Cheers,
Zubin.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241122/99d2e4d4/attachment.sig>

From ben at well-typed.com  Tue Nov 26 20:14:01 2024
From: ben at well-typed.com (Ben Gamari)
Date: Tue, 26 Nov 2024 15:14:01 -0500
Subject: Feedback on potential change in boot library
Message-ID: <87serdu516.fsf@smart-cactus.org>

tl;dr. We propose that GHC more aggressively bump its boot library
       dependencies.


Hello all,

Historically, GHC minor releases have been quite conservative in bumping
boot library versions, generally defaulting to retaining the versions
used in the GHC-x.y.1 release unless

 * a request is made by a boot library maintainer, or
 * we discover an issue which warrants a bump

The motivation for this policy is both to

 * minimize the potential for regressions in correctness and
   performance, and

 * reduce the potential for end-user breakage due to interface changes
   as even minor releases can result in considerable impact (e.g.
   #25411, #22633)

While this policy has served us well, its conservative nature demands
the vigilence of both GHC and upstream maintainers to ensure that bumps
that are truly necessary do not fall through the cracks.

As the flaws of this process have been more apparent recently (e.g.
#24597), we have done a bit of reflection on how we might improve the
status quo. Concretely, I would like feedback on the adopting the
following policy going forward:

> Unless guidance is provided otherwise by a library maintainer, a GHC
> x.y.z release will attempt to ship the newest minor version of each
> boot libray in the same major series shipped with GHC x.y.1.

I believe this policy would leave less room for human error and open
opportunities for automated checking. On the other hand, the more
aggressive bumping of submodules may mean that users are hit with more
(usually minor) compatibility issues when moving between minor GHC
releases.

How do others feel about this? We are particularly interested to hear
from boot library maintainers but feedback from packagers and users is
also very much welcome.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241126/0948a88e/attachment.sig>

From hecate at glitchbra.in  Tue Nov 26 22:27:50 2024
From: hecate at glitchbra.in (=?UTF-8?Q?H=C3=A9cate?=)
Date: Tue, 26 Nov 2024 23:27:50 +0100
Subject: Feedback on potential change in boot library
In-Reply-To: <87serdu516.fsf@smart-cactus.org>
References: <87serdu516.fsf@smart-cactus.org>
Message-ID: <051942f7-f840-4ed5-84a8-689c5321b7d9@glitchbra.in>

 From the cabal perspective this great news, as this will allow the 
feedback cycle between our two projects to be shorter. Glad to see the 
GHC team is considering this!

Cheers,
Hécate

Le 26/11/2024 à 21:14, Ben Gamari a écrit :
> tl;dr. We propose that GHC more aggressively bump its boot library
>         dependencies.
>
>
> Hello all,
>
> Historically, GHC minor releases have been quite conservative in bumping
> boot library versions, generally defaulting to retaining the versions
> used in the GHC-x.y.1 release unless
>
>   * a request is made by a boot library maintainer, or
>   * we discover an issue which warrants a bump
>
> The motivation for this policy is both to
>
>   * minimize the potential for regressions in correctness and
>     performance, and
>
>   * reduce the potential for end-user breakage due to interface changes
>     as even minor releases can result in considerable impact (e.g.
>     #25411, #22633)
>
> While this policy has served us well, its conservative nature demands
> the vigilence of both GHC and upstream maintainers to ensure that bumps
> that are truly necessary do not fall through the cracks.
>
> As the flaws of this process have been more apparent recently (e.g.
> #24597), we have done a bit of reflection on how we might improve the
> status quo. Concretely, I would like feedback on the adopting the
> following policy going forward:
>
>> Unless guidance is provided otherwise by a library maintainer, a GHC
>> x.y.z release will attempt to ship the newest minor version of each
>> boot libray in the same major series shipped with GHC x.y.1.
> I believe this policy would leave less room for human error and open
> opportunities for automated checking. On the other hand, the more
> aggressive bumping of submodules may mean that users are hit with more
> (usually minor) compatibility issues when moving between minor GHC
> releases.
>
> How do others feel about this? We are particularly interested to hear
> from boot library maintainers but feedback from packagers and users is
> also very much welcome.
>
> Cheers,
>
> - Ben
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-- 
Hécate ✨
🐦: @TechnoEmpress
IRC: Hecate
WWW: https://glitchbra.in
RUN: BSD


From adam at well-typed.com  Wed Nov 27 08:02:10 2024
From: adam at well-typed.com (Adam Gundry)
Date: Wed, 27 Nov 2024 08:02:10 +0000
Subject: Feedback on potential change in boot library
In-Reply-To: <051942f7-f840-4ed5-84a8-689c5321b7d9@glitchbra.in>
References: <87serdu516.fsf@smart-cactus.org>
 <051942f7-f840-4ed5-84a8-689c5321b7d9@glitchbra.in>
Message-ID: <c0cc8858-6e3d-481b-be9f-c13b80e027a2@well-typed.com>

It's not obvious to me that this is a good idea. Users already complain 
that "upgrading GHC broke my code", when in practice this frequently 
means "when upgrading GHC I used new boot library versions with 
incompatible APIs that broke my code". So having a policy of introducing 
more frequent API changes (and making it harder to switch between GHC 
releases in a single release series) needs clear justification IMHO.

(Of course in the long term we should make it easier for users to pin 
boot library APIs while upgrading GHC independently, but we're not there 
yet.)

I think that by default, it makes sense for a new minor GHC release to 
keep the same API versions of boot libraries, but bump to the latest 
super-minor component (i.e. for an A.B.C.D version, keep A.B.C the same 
but bump D). If a boot library fixes a bug that is critical enough to 
require bumping the version distributed with GHC, ideally there would be 
an API-compatible release of the boot library. Of course that might not 
always be feasible, and it is ultimately a judgement call for the GHC 
maintainers as to which boot library versions are best to ship.

Cheers,

Adam



On 26/11/2024 22:27, Hécate via ghc-devs wrote:
>  From the cabal perspective this great news, as this will allow the 
> feedback cycle between our two projects to be shorter. Glad to see the 
> GHC team is considering this!
> 
> Cheers,
> Hécate
> 
> Le 26/11/2024 à 21:14, Ben Gamari a écrit :
>> tl;dr. We propose that GHC more aggressively bump its boot library
>>         dependencies.
>>
>>
>> Hello all,
>>
>> Historically, GHC minor releases have been quite conservative in bumping
>> boot library versions, generally defaulting to retaining the versions
>> used in the GHC-x.y.1 release unless
>>
>>   * a request is made by a boot library maintainer, or
>>   * we discover an issue which warrants a bump
>>
>> The motivation for this policy is both to
>>
>>   * minimize the potential for regressions in correctness and
>>     performance, and
>>
>>   * reduce the potential for end-user breakage due to interface changes
>>     as even minor releases can result in considerable impact (e.g.
>>     #25411, #22633)
>>
>> While this policy has served us well, its conservative nature demands
>> the vigilence of both GHC and upstream maintainers to ensure that bumps
>> that are truly necessary do not fall through the cracks.
>>
>> As the flaws of this process have been more apparent recently (e.g.
>> #24597), we have done a bit of reflection on how we might improve the
>> status quo. Concretely, I would like feedback on the adopting the
>> following policy going forward:
>>
>>> Unless guidance is provided otherwise by a library maintainer, a GHC
>>> x.y.z release will attempt to ship the newest minor version of each
>>> boot libray in the same major series shipped with GHC x.y.1.
>> I believe this policy would leave less room for human error and open
>> opportunities for automated checking. On the other hand, the more
>> aggressive bumping of submodules may mean that users are hit with more
>> (usually minor) compatibility issues when moving between minor GHC
>> releases.
>>
>> How do others feel about this? We are particularly interested to hear
>> from boot library maintainers but feedback from packagers and users is
>> also very much welcome.
>>
>> Cheers,
>>
>> - Ben
>>

-- 
Adam Gundry, Haskell Consultant
Well-Typed LLP, https://www.well-typed.com/

Registered in England & Wales, OC335890
27 Old Gloucester Street, London WC1N 3AX, England


From tom-lists-haskell-cafe-2023 at jaguarpaw.co.uk  Wed Nov 27 09:57:57 2024
From: tom-lists-haskell-cafe-2023 at jaguarpaw.co.uk (Tom Ellis)
Date: Wed, 27 Nov 2024 09:57:57 +0000
Subject: Feedback on potential change in boot library
In-Reply-To: <c0cc8858-6e3d-481b-be9f-c13b80e027a2@well-typed.com>
References: <87serdu516.fsf@smart-cactus.org>
 <051942f7-f840-4ed5-84a8-689c5321b7d9@glitchbra.in>
 <c0cc8858-6e3d-481b-be9f-c13b80e027a2@well-typed.com>
Message-ID: <Z0btJQq6K2Jbe7GQ@cloudinit-builder>

Thanks Adam,

I'm not too familiar with the GHC-side requirements here, but I wonder
if we can approach this issue from a different direction: is there
something we can do to help boot libraries avoid breaking changes?

Tom


On Wed, Nov 27, 2024 at 08:02:10AM +0000, Adam Gundry wrote:
> It's not obvious to me that this is a good idea. Users already complain that
> "upgrading GHC broke my code", when in practice this frequently means "when
> upgrading GHC I used new boot library versions with incompatible APIs that
> broke my code". So having a policy of introducing more frequent API changes
> (and making it harder to switch between GHC releases in a single release
> series) needs clear justification IMHO.
> 
> (Of course in the long term we should make it easier for users to pin boot
> library APIs while upgrading GHC independently, but we're not there yet.)
> 
> I think that by default, it makes sense for a new minor GHC release to keep
> the same API versions of boot libraries, but bump to the latest super-minor
> component (i.e. for an A.B.C.D version, keep A.B.C the same but bump D). If
> a boot library fixes a bug that is critical enough to require bumping the
> version distributed with GHC, ideally there would be an API-compatible
> release of the boot library. Of course that might not always be feasible,
> and it is ultimately a judgement call for the GHC maintainers as to which
> boot library versions are best to ship.
> 
> On 26/11/2024 22:27, Hécate via ghc-devs wrote:
> >  From the cabal perspective this great news, as this will allow the
> > feedback cycle between our two projects to be shorter. Glad to see the
> > GHC team is considering this!
> > 
> > Cheers,
> > Hécate
> > 
> > Le 26/11/2024 à 21:14, Ben Gamari a écrit :
> > > tl;dr. We propose that GHC more aggressively bump its boot library
> > >         dependencies.
> > > 
> > > 
> > > Hello all,
> > > 
> > > Historically, GHC minor releases have been quite conservative in bumping
> > > boot library versions, generally defaulting to retaining the versions
> > > used in the GHC-x.y.1 release unless
> > > 
> > >   * a request is made by a boot library maintainer, or
> > >   * we discover an issue which warrants a bump
> > > 
> > > The motivation for this policy is both to
> > > 
> > >   * minimize the potential for regressions in correctness and
> > >     performance, and
> > > 
> > >   * reduce the potential for end-user breakage due to interface changes
> > >     as even minor releases can result in considerable impact (e.g.
> > >     #25411, #22633)
> > > 
> > > While this policy has served us well, its conservative nature demands
> > > the vigilence of both GHC and upstream maintainers to ensure that bumps
> > > that are truly necessary do not fall through the cracks.
> > > 
> > > As the flaws of this process have been more apparent recently (e.g.
> > > #24597), we have done a bit of reflection on how we might improve the
> > > status quo. Concretely, I would like feedback on the adopting the
> > > following policy going forward:
> > > 
> > > > Unless guidance is provided otherwise by a library maintainer, a GHC
> > > > x.y.z release will attempt to ship the newest minor version of each
> > > > boot libray in the same major series shipped with GHC x.y.1.
> > > I believe this policy would leave less room for human error and open
> > > opportunities for automated checking. On the other hand, the more
> > > aggressive bumping of submodules may mean that users are hit with more
> > > (usually minor) compatibility issues when moving between minor GHC
> > > releases.
> > > 
> > > How do others feel about this? We are particularly interested to hear
> > > from boot library maintainers but feedback from packagers and users is
> > > also very much welcome.

From sgraf1337 at gmail.com  Wed Nov 27 10:15:22 2024
From: sgraf1337 at gmail.com (Sebastian Graf)
Date: Wed, 27 Nov 2024 10:15:22 +0000
Subject: Feedback on potential change in boot library
In-Reply-To: <Z0btJQq6K2Jbe7GQ@cloudinit-builder>
References: <87serdu516.fsf@smart-cactus.org>
 <051942f7-f840-4ed5-84a8-689c5321b7d9@glitchbra.in>
 <c0cc8858-6e3d-481b-be9f-c13b80e027a2@well-typed.com>
 <Z0btJQq6K2Jbe7GQ@cloudinit-builder>
Message-ID: <emd7f9a473-2eb7-47a3-9bed-8ff351a04de0@e233e972.com>

Hi,

Do we have concrete evidence that minor bumps in a boot library have 
caused breakage in the past? (That would of course be a bug in the boot 
library according to the PVP.)

I would think that such breakage is even less of an issue for a new GHC 
minor release (say GHC 9.8.4).
The hackage ecosystem is likely prepared to compile with the GHC 9.8 
series at this point, so it should be possible to detect such breakages 
by compiling stackage/hackage and running testsuites.
That is, I would think it is harder to anticipate unforeseen boot 
library breakage for a new major release of GHC than it is to anticipate 
breakage for a minor GHC release.

Besides, the only reliable way to prevent such breakage is not to bump 
any boot library *at all*.
If a boot library breaks on a minor bump, it is just as likely that it 
breaks on a super-minor bump.

So bumping minor versions of boot library sounds reasonable to me, 
barring concrete evidence that minor bumps have caused problems in the 
past (which again
means there was a bug in a boot library).

Sebastian

------ Originalnachricht ------
Von "Tom Ellis" <tom-lists-haskell-cafe-2023 at jaguarpaw.co.uk>
An ghc-devs at haskell.org
Datum 27.11.2024 10:57:57
Betreff Re: Feedback on potential change in boot library

>Thanks Adam,
>
>I'm not too familiar with the GHC-side requirements here, but I wonder
>if we can approach this issue from a different direction: is there
>something we can do to help boot libraries avoid breaking changes?
>
>Tom
>
>
>On Wed, Nov 27, 2024 at 08:02:10AM +0000, Adam Gundry wrote:
>>  It's not obvious to me that this is a good idea. Users already complain that
>>  "upgrading GHC broke my code", when in practice this frequently means "when
>>  upgrading GHC I used new boot library versions with incompatible APIs that
>>  broke my code". So having a policy of introducing more frequent API changes
>>  (and making it harder to switch between GHC releases in a single release
>>  series) needs clear justification IMHO.
>>
>>  (Of course in the long term we should make it easier for users to pin boot
>>  library APIs while upgrading GHC independently, but we're not there yet.)
>>
>>  I think that by default, it makes sense for a new minor GHC release to keep
>>  the same API versions of boot libraries, but bump to the latest super-minor
>>  component (i.e. for an A.B.C.D version, keep A.B.C the same but bump D). If
>>  a boot library fixes a bug that is critical enough to require bumping the
>>  version distributed with GHC, ideally there would be an API-compatible
>>  release of the boot library. Of course that might not always be feasible,
>>  and it is ultimately a judgement call for the GHC maintainers as to which
>>  boot library versions are best to ship.
>>
>>  On 26/11/2024 22:27, Hécate via ghc-devs wrote:
>>  >  From the cabal perspective this great news, as this will allow the
>>  > feedback cycle between our two projects to be shorter. Glad to see the
>>  > GHC team is considering this!
>>  >
>>  > Cheers,
>>  > Hécate
>>  >
>>  > Le 26/11/2024 à 21:14, Ben Gamari a écrit :
>>  > > tl;dr. We propose that GHC more aggressively bump its boot library
>>  > >         dependencies.
>>  > >
>>  > >
>>  > > Hello all,
>>  > >
>>  > > Historically, GHC minor releases have been quite conservative in bumping
>>  > > boot library versions, generally defaulting to retaining the versions
>>  > > used in the GHC-x.y.1 release unless
>>  > >
>>  > >   * a request is made by a boot library maintainer, or
>>  > >   * we discover an issue which warrants a bump
>>  > >
>>  > > The motivation for this policy is both to
>>  > >
>>  > >   * minimize the potential for regressions in correctness and
>>  > >     performance, and
>>  > >
>>  > >   * reduce the potential for end-user breakage due to interface changes
>>  > >     as even minor releases can result in considerable impact (e.g.
>>  > >     #25411, #22633)
>>  > >
>>  > > While this policy has served us well, its conservative nature demands
>>  > > the vigilence of both GHC and upstream maintainers to ensure that bumps
>>  > > that are truly necessary do not fall through the cracks.
>>  > >
>>  > > As the flaws of this process have been more apparent recently (e.g.
>>  > > #24597), we have done a bit of reflection on how we might improve the
>>  > > status quo. Concretely, I would like feedback on the adopting the
>>  > > following policy going forward:
>>  > >
>>  > > > Unless guidance is provided otherwise by a library maintainer, a GHC
>>  > > > x.y.z release will attempt to ship the newest minor version of each
>>  > > > boot libray in the same major series shipped with GHC x.y.1.
>>  > > I believe this policy would leave less room for human error and open
>>  > > opportunities for automated checking. On the other hand, the more
>>  > > aggressive bumping of submodules may mean that users are hit with more
>>  > > (usually minor) compatibility issues when moving between minor GHC
>>  > > releases.
>>  > >
>>  > > How do others feel about this? We are particularly interested to hear
>>  > > from boot library maintainers but feedback from packagers and users is
>>  > > also very much welcome.
>_______________________________________________
>ghc-devs mailing list
>ghc-devs at haskell.org
>http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241127/691322c6/attachment.html>

From zubin at well-typed.com  Wed Nov 27 13:38:59 2024
From: zubin at well-typed.com (Zubin Duggal)
Date: Wed, 27 Nov 2024 19:08:59 +0530
Subject: GHC 9.12.1 updates and rc1 status
In-Reply-To: <euvthzd2yaqkhy4r46qrtkcat5uf4ptrx4ebkv42qh2blkd5sj@5lnfhlzchpfc>
References: <euvthzd2yaqkhy4r46qrtkcat5uf4ptrx4ebkv42qh2blkd5sj@5lnfhlzchpfc>
Message-ID: <55jt7bwcbebsfl55p25bfzus6fcrgzthjnsvoetxgdoqaza3hc@x3fpsa4n3db5>

Hi,

The patches I was waiting for have been merged, but I am
still waiting on some boot library maintainers to finalise
the commits that will be shipped with the release. I hope
to have the rc1 release out by the end of the week.

Also keeping in mind the wishes of many in the community to
have a longer testing cycle between the release candidates
and the final release, the final release will be pushed back
to two weeks after the date of the final release candidate.

The tentative schedule as of now is:

RC1: 2024-11-29
Final: 2024-12-13

Thanks,
Zubin.

On 24/11/22 19:07, Zubin Duggal wrote:
>Hi all,
>
>We had planned to release 9.12.1-rc1 today. Unfortunately,
>we discovered a few regressions and bugs which need to be fixed
>before we can publish a release candidate, including #25382,
>#25375 and !13633. I am currently waiting for some of these
>fixes to land in master before backporting them to the release
>branch.
>
>I will try to have the release candidate out on Monday the 25th.
>
>The final release is currently scheduled for December 6th. We
>will inform you if there are any changes to this schedule.
>
>Cheers,
>Zubin.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241127/e4f6329b/attachment.sig>

From ben at well-typed.com  Wed Nov 27 18:21:14 2024
From: ben at well-typed.com (Ben Gamari)
Date: Wed, 27 Nov 2024 13:21:14 -0500
Subject: Feedback on potential change in boot library
In-Reply-To: <emd7f9a473-2eb7-47a3-9bed-8ff351a04de0@e233e972.com>
References: <87serdu516.fsf@smart-cactus.org>
 <051942f7-f840-4ed5-84a8-689c5321b7d9@glitchbra.in>
 <c0cc8858-6e3d-481b-be9f-c13b80e027a2@well-typed.com>
 <Z0btJQq6K2Jbe7GQ@cloudinit-builder>
 <emd7f9a473-2eb7-47a3-9bed-8ff351a04de0@e233e972.com>
Message-ID: <87mshktu5l.fsf@smart-cactus.org>

"Sebastian Graf" <sgraf1337 at gmail.com> writes:

> Hi,
>
> Do we have concrete evidence that minor bumps in a boot library have 
> caused breakage in the past? (That would of course be a bug in the boot 
> library according to the PVP.)
>
Yes, I cited two such examples that came to mind in the original
message. These weren't hard to find, despite boot library bumps in GHC
minor releases being historically rather rare.

Contrary to what one might be tempted to conclude, PVP-compliant,
non-buggy minor bumps can cause breakage for downstream users. The
reason for this is typically the introduction of new exports. For
instance, in #25411 we found that the bump of `deepseq` in 9.8.3
introduced a new export (`Unit`) from `Control.DeepSeq`. This is, of
course, a minor change under the PVP but renders previously working
modules unbuildable due to ambiguous occurrences.

Perhaps in an ideal world all `import`s would be either `qualified` or
include an explicit import list but, sadly, this is not the world in
which we live. Consequently, the principle difference between a major
and minor bump is that the breakage from the latter is usually "easy" to
fix. However, it is breakage nevertheless and even easy breakage can
cost significant time when migrating large code-bases.

> I would think that such breakage is even less of an issue for a new GHC 
> minor release (say GHC 9.8.4).
> The hackage ecosystem is likely prepared to compile with the GHC 9.8 
> series at this point, so it should be possible to detect such breakages 
> by compiling stackage/hackage and running testsuites.
>
Indeed head.hackage identified the breakage in 9.8.3. However, we
decided to move ahead regardless since the maintainer requested it.
The policy proposed in the head of thread will make this sort of
breakage considerably more likely to appear in future releases.

> Besides, the only reliable way to prevent such breakage is not to bump 
> any boot library *at all*.
> If a boot library breaks on a minor bump, it is just as likely that it 
> breaks on a super-minor bump.
>
I am not sure I agree. While the PVP is silent on this matter, IMHO
patch-level bumps should not touch external interfaces at all, only
implementation and packaging. This makes it much less likely for
breakage to manifest.

Cheers,

- Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241127/40b8d5f0/attachment.sig>

From oleg.grenrus at iki.fi  Wed Nov 27 18:49:58 2024
From: oleg.grenrus at iki.fi (Oleg Grenrus)
Date: Wed, 27 Nov 2024 20:49:58 +0200
Subject: Feedback on potential change in boot library
In-Reply-To: <87mshktu5l.fsf@smart-cactus.org>
References: <87serdu516.fsf@smart-cactus.org>
 <051942f7-f840-4ed5-84a8-689c5321b7d9@glitchbra.in>
 <c0cc8858-6e3d-481b-be9f-c13b80e027a2@well-typed.com>
 <Z0btJQq6K2Jbe7GQ@cloudinit-builder>
 <emd7f9a473-2eb7-47a3-9bed-8ff351a04de0@e233e972.com>
 <87mshktu5l.fsf@smart-cactus.org>
Message-ID: <abe20ff5-191a-4259-8661-9a0254593300@iki.fi>


On 27.11.2024 20.21, Ben Gamari wrote:
> "Sebastian Graf"<sgraf1337 at gmail.com>  writes:
>
>> Hi,
>>
>> Do we have concrete evidence that minor bumps in a boot library have
>> caused breakage in the past? (That would of course be a bug in the boot
>> library according to the PVP.)
>>
> Yes, I cited two such examples that came to mind in the original
> message. These weren't hard to find, despite boot library bumps in GHC
> minor releases being historically rather rare.
>
> Contrary to what one might be tempted to conclude, PVP-compliant,
> non-buggy minor bumps can cause breakage for downstream users. The
> reason for this is typically the introduction of new exports. For
> instance, in #25411 we found that the bump of `deepseq` in 9.8.3
> introduced a new export (`Unit`) from `Control.DeepSeq`. This is, of
> course, a minor change under the PVP but renders previously working
> modules unbuildable due to ambiguous occurrences.
>
> Perhaps in an ideal world all `import`s would be either `qualified` or
> include an explicit import list but, sadly, this is not the world in

PVP suggest to use qualified or explicit import lists.
https://wiki.haskell.org/Import_modules_properly

Only then "A client MAY specify that they are insensitive to additions 
to the API by allowing a range of C values, e.g. build-depends: base >= 
2.1.1 && < 2.2."

That's right where in the PVP: https://pvp.haskell.org/ point 5.

> which we live. Consequently, the principle difference between a major
> and minor bump is that the breakage from the latter is usually "easy" to
> fix. However, it is breakage nevertheless and even easy breakage can
> cost significant time when migrating large code-bases.

No. PVP clearly says which kind of changes MUST be major bumps, which 
MUST be at least minor bumps etc. There is very little of "interpretation".

But in addition to that, PVP is contract between library authors and 
consumers. It also says how consumer should specify their dependencies.
BOTH have to do their parts, if consumers don't follow PVP, it's 
irrelevant what library authors do.

"The goal of a versioning system is to inform clients of a package of 
changes to that package that might affect them,
and to provide a way for clients to specify a particular version or 
range of versions of a dependency that they are compatible with."

Otherwise we can simply treat all changes as breaking changes and throw 
any semantic* version policy into a garbage bin. Make people read the 
changelogs.

* Semantic as in "assigns semantics to version number changes".

To make this even more concrete: Minor GHC releases couldn't bump minor 
`base` either. If GHC-9.12.1 ships with `base-4.21.0.0`, then will 
GHC-9.12.2 and later too. Significant amount of ecosystem does `base 
<5`, so it's arguable that we cannot introduce any (breaking) change, 
like removing modules from `base`, without going past version 5 (and 
before people start to do `base < 1000` bounds, which I think i have seen).

So to repeat: if clients don't care to specify their compatibility 
correctly, have courage to say it's their fault their code broke.

>> I would think that such breakage is even less of an issue for a new GHC
>> minor release (say GHC 9.8.4).
>> The hackage ecosystem is likely prepared to compile with the GHC 9.8
>> series at this point, so it should be possible to detect such breakages
>> by compiling stackage/hackage and running testsuites.
>>
> Indeed head.hackage identified the breakage in 9.8.3. However, we
> decided to move ahead regardless since the maintainer requested it.
> The policy proposed in the head of thread will make this sort of
> breakage considerably more likely to appear in future releases.
>
>> Besides, the only reliable way to prevent such breakage is not to bump
>> any boot library *at all*.
>> If a boot library breaks on a minor bump, it is just as likely that it
>> breaks on a super-minor bump.
>>
> I am not sure I agree. While the PVP is silent on this matter, IMHO
> patch-level bumps should not touch external interfaces at all, only
> implementation and packaging. This makes it much less likely for
> breakage to manifest.
>
> Cheers,
>
> - Ben
>
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241127/7de84cc4/attachment.html>

From oleg.grenrus at iki.fi  Wed Nov 27 18:59:51 2024
From: oleg.grenrus at iki.fi (Oleg Grenrus)
Date: Wed, 27 Nov 2024 20:59:51 +0200
Subject: Feedback on potential change in boot library
In-Reply-To: <abe20ff5-191a-4259-8661-9a0254593300@iki.fi>
References: <87serdu516.fsf@smart-cactus.org>
 <051942f7-f840-4ed5-84a8-689c5321b7d9@glitchbra.in>
 <c0cc8858-6e3d-481b-be9f-c13b80e027a2@well-typed.com>
 <Z0btJQq6K2Jbe7GQ@cloudinit-builder>
 <emd7f9a473-2eb7-47a3-9bed-8ff351a04de0@e233e972.com>
 <87mshktu5l.fsf@smart-cactus.org>
 <abe20ff5-191a-4259-8661-9a0254593300@iki.fi>
Message-ID: <5cb9d54e-820a-4af1-9c45-4d97e9328ce9@iki.fi>


On 27.11.2024 20.49, Oleg Grenrus wrote:
>
>
> On 27.11.2024 20.21, Ben Gamari wrote:
>> "Sebastian Graf"<sgraf1337 at gmail.com>  writes:
>>
>>> Hi,
>>>
>>> Do we have concrete evidence that minor bumps in a boot library have
>>> caused breakage in the past? (That would of course be a bug in the boot
>>> library according to the PVP.)
>>>
>> Yes, I cited two such examples that came to mind in the original
>> message. These weren't hard to find, despite boot library bumps in GHC
>> minor releases being historically rather rare.
>>
>> Contrary to what one might be tempted to conclude, PVP-compliant,
>> non-buggy minor bumps can cause breakage for downstream users. The
>> reason for this is typically the introduction of new exports. For
>> instance, in #25411 we found that the bump of `deepseq` in 9.8.3
>> introduced a new export (`Unit`) from `Control.DeepSeq`. This is, of
>> course, a minor change under the PVP but renders previously working
>> modules unbuildable due to ambiguous occurrences.
>>
>> Perhaps in an ideal world all `import`s would be either `qualified` or
>> include an explicit import list but, sadly, this is not the world in
>
> PVP suggest to use qualified or explicit import lists.
> https://wiki.haskell.org/Import_modules_properly
>
> Only then "A client MAY specify that they are insensitive to additions 
> to the API by allowing a range of C values, e.g. build-depends: base 
> >= 2.1.1 && < 2.2."
>
> That's right where in the PVP: https://pvp.haskell.org/ point 5.
>
I forgot to say, that if clients are not insensitive to additions to the 
API, i.e. they do use open imports, the PVP dictates to use strict bounds:

4. Client specification. Hence A.B.C uniquely identifies the API. A 
client that wants to specify that they depend on a particular version of 
the API can specify a particular A.B.C and be sure of getting that API 
only. For example, build-depends: mypkg >= 2.1.1 && < 2.1.2.

which no-one does, but that's their fault.

I'd argue that a design mistake in Haskell. We should not have open 
imports, nor implicit Prelude. The default thing (import Foo) does the 
wrong thing. Compare to Rust where "use foo" isn't a wildcard import. 
https://www.lurklurk.org/effective-rust/wildcard.html

>
>> which we live. Consequently, the principle difference between a major
>> and minor bump is that the breakage from the latter is usually "easy" to
>> fix. However, it is breakage nevertheless and even easy breakage can
>> cost significant time when migrating large code-bases.
>
> No. PVP clearly says which kind of changes MUST be major bumps, which 
> MUST be at least minor bumps etc. There is very little of 
> "interpretation".
>
> But in addition to that, PVP is contract between library authors and 
> consumers. It also says how consumer should specify their dependencies.
> BOTH have to do their parts, if consumers don't follow PVP, it's 
> irrelevant what library authors do.
>
> "The goal of a versioning system is to inform clients of a package of 
> changes to that package that might affect them,
> and to provide a way for clients to specify a particular version or 
> range of versions of a dependency that they are compatible with."
>
> Otherwise we can simply treat all changes as breaking changes and 
> throw any semantic* version policy into a garbage bin. Make people 
> read the changelogs.
>
> * Semantic as in "assigns semantics to version number changes".
>
> To make this even more concrete: Minor GHC releases couldn't bump 
> minor `base` either. If GHC-9.12.1 ships with `base-4.21.0.0`, then 
> will GHC-9.12.2 and later too. Significant amount of ecosystem does 
> `base <5`, so it's arguable that we cannot introduce any (breaking) 
> change, like removing modules from `base`, without going past version 
> 5 (and before people start to do `base < 1000` bounds, which I think i 
> have seen).
>
> So to repeat: if clients don't care to specify their compatibility 
> correctly, have courage to say it's their fault their code broke.
>
>>> I would think that such breakage is even less of an issue for a new GHC
>>> minor release (say GHC 9.8.4).
>>> The hackage ecosystem is likely prepared to compile with the GHC 9.8
>>> series at this point, so it should be possible to detect such breakages
>>> by compiling stackage/hackage and running testsuites.
>>>
>> Indeed head.hackage identified the breakage in 9.8.3. However, we
>> decided to move ahead regardless since the maintainer requested it.
>> The policy proposed in the head of thread will make this sort of
>> breakage considerably more likely to appear in future releases.
>>
>>> Besides, the only reliable way to prevent such breakage is not to bump
>>> any boot library *at all*.
>>> If a boot library breaks on a minor bump, it is just as likely that it
>>> breaks on a super-minor bump.
>>>
>> I am not sure I agree. While the PVP is silent on this matter, IMHO
>> patch-level bumps should not touch external interfaces at all, only
>> implementation and packaging. This makes it much less likely for
>> breakage to manifest.
>>
>> Cheers,
>>
>> - Ben
>>
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241127/69c5b646/attachment.html>

From tom-lists-haskell-cafe-2023 at jaguarpaw.co.uk  Wed Nov 27 19:36:58 2024
From: tom-lists-haskell-cafe-2023 at jaguarpaw.co.uk (Tom Ellis)
Date: Wed, 27 Nov 2024 19:36:58 +0000
Subject: Feedback on potential change in boot library
In-Reply-To: <5cb9d54e-820a-4af1-9c45-4d97e9328ce9@iki.fi>
References: <87serdu516.fsf@smart-cactus.org>
 <051942f7-f840-4ed5-84a8-689c5321b7d9@glitchbra.in>
 <c0cc8858-6e3d-481b-be9f-c13b80e027a2@well-typed.com>
 <Z0btJQq6K2Jbe7GQ@cloudinit-builder>
 <emd7f9a473-2eb7-47a3-9bed-8ff351a04de0@e233e972.com>
 <87mshktu5l.fsf@smart-cactus.org>
 <abe20ff5-191a-4259-8661-9a0254593300@iki.fi>
 <5cb9d54e-820a-4af1-9c45-4d97e9328ce9@iki.fi>
Message-ID: <Z0d02nsNtqSb4LF6@cloudinit-builder>

On Wed, Nov 27, 2024 at 08:59:51PM +0200, Oleg Grenrus wrote:
> On 27.11.2024 20.49, Oleg Grenrus wrote:
> > PVP suggest to use qualified or explicit import lists.
> > https://wiki.haskell.org/Import_modules_properly
> > 
> > Only then "A client MAY specify that they are insensitive to additions
> > to the API by allowing a range of C values, e.g. build-depends: base >=
> > 2.1.1 && < 2.2."
> > 
> > That's right where in the PVP: https://pvp.haskell.org/ point 5.
> > 
> I forgot to say, that if clients are not insensitive to additions to the
> API, i.e. they do use open imports, the PVP dictates to use strict bounds:
> 
> 4. Client specification. Hence A.B.C uniquely identifies the API. A client
> that wants to specify that they depend on a particular version of the API
> can specify a particular A.B.C and be sure of getting that API only. For
> example, build-depends: mypkg >= 2.1.1 && < 2.1.2.
> 
> which no-one does, but that's their fault.
> 
> I'd argue that a design mistake in Haskell. We should not have open imports,
> nor implicit Prelude. The default thing (import Foo) does the wrong thing.
> Compare to Rust where "use foo" isn't a wildcard import.
> https://www.lurklurk.org/effective-rust/wildcard.html

I agree with everything you've said in this thread, Oleg.  Digressing
slightly, I think there's scope for a new language extension
("ImportQualifiedDefault"?) that makes

    import Foo
    import Bar unqualifed

mean what

    import Foo qualified
    import Bar

currently means.

Tom

From imantc at gmail.com  Thu Nov 28 07:44:37 2024
From: imantc at gmail.com (Imants Cekusins)
Date: Thu, 28 Nov 2024 09:44:37 +0200
Subject: Feedback on potential change in boot library
In-Reply-To: <Z0btJQq6K2Jbe7GQ@cloudinit-builder>
References: <87serdu516.fsf@smart-cactus.org>
 <051942f7-f840-4ed5-84a8-689c5321b7d9@glitchbra.in>
 <c0cc8858-6e3d-481b-be9f-c13b80e027a2@well-typed.com>
 <Z0btJQq6K2Jbe7GQ@cloudinit-builder>
Message-ID: <CAP1qinaSWeXx-Ttnzn7O3ieNHcUTfZ+WUUZa-EAL7e+uMcpyoQ@mail.gmail.com>

On Wed, 27 Nov 2024 at 11:58, Tom Ellis <
tom-lists-haskell-cafe-2023 at jaguarpaw.co.uk> wrote:

> Thanks Adam,
>
> I'm not too familiar with the GHC-side requirements here, but I wonder
> if we can approach this issue from a different direction: is there
> something we can do to help boot libraries avoid breaking changes?
>
> Tom
>


Breaking changes caused by new exports could be mitigated with a library
package or a GHC plugin with such auto refactoring functionality :

* Uniquely qualify (i.e., leave no different modules similarly qualified in
the same module) each imported module and each imported symbol.

This refactoring should be possible in code which already compiles.
It would not require syntax / grammar changes.

Something similar to smuggler2 package.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241128/4d404dbc/attachment.html>

From adam at well-typed.com  Thu Nov 28 08:05:30 2024
From: adam at well-typed.com (Adam Gundry)
Date: Thu, 28 Nov 2024 08:05:30 +0000
Subject: Feedback on potential change in boot library
In-Reply-To: <5cb9d54e-820a-4af1-9c45-4d97e9328ce9@iki.fi>
References: <87serdu516.fsf@smart-cactus.org>
 <051942f7-f840-4ed5-84a8-689c5321b7d9@glitchbra.in>
 <c0cc8858-6e3d-481b-be9f-c13b80e027a2@well-typed.com>
 <Z0btJQq6K2Jbe7GQ@cloudinit-builder>
 <emd7f9a473-2eb7-47a3-9bed-8ff351a04de0@e233e972.com>
 <87mshktu5l.fsf@smart-cactus.org>
 <abe20ff5-191a-4259-8661-9a0254593300@iki.fi>
 <5cb9d54e-820a-4af1-9c45-4d97e9328ce9@iki.fi>
Message-ID: <a49fb89a-fdf0-4b6c-b620-90d300ff579c@well-typed.com>

On 27/11/2024 18:59, Oleg Grenrus wrote:
>
>> So to repeat: if clients don't care to specify their compatibility 
>> correctly, have courage to say it's their fault their code broke.

In the general case, I agree: if a library introduces new exports and 
correctly bumps the minor version bound per the PVP, then the library 
author is not responsible for their clients having overly lax bounds. 
And perhaps it would be better if the language defaulted to encouraging 
more precise imports.

However for boot libraries specifically, clients may de facto be forced 
to upgrade to a newer library version in order to use a new GHC release 
(at least, if lib:ghc is in the install plan or they are using a 
Stackage-style universal package set). Correctly specified bounds don't 
help, they just result in a solver failure rather than a build failure. 
That's why I think GHC should aim to minimize boot library API changes 
within a single release series, at least until we can relax the coupling 
of boot libraries.

Adam


-- 
Adam Gundry, Haskell Consultant
Well-Typed LLP, https://www.well-typed.com/

Registered in England & Wales, OC335890
27 Old Gloucester Street, London WC1N 3AX, England


From klebinger.andreas at gmx.at  Thu Nov 28 12:51:32 2024
From: klebinger.andreas at gmx.at (Andreas Klebinger)
Date: Thu, 28 Nov 2024 13:51:32 +0100
Subject: Feedback on potential change in boot library
In-Reply-To: <CAP1qinaSWeXx-Ttnzn7O3ieNHcUTfZ+WUUZa-EAL7e+uMcpyoQ@mail.gmail.com>
References: <87serdu516.fsf@smart-cactus.org>
 <051942f7-f840-4ed5-84a8-689c5321b7d9@glitchbra.in>
 <c0cc8858-6e3d-481b-be9f-c13b80e027a2@well-typed.com>
 <Z0btJQq6K2Jbe7GQ@cloudinit-builder>
 <CAP1qinaSWeXx-Ttnzn7O3ieNHcUTfZ+WUUZa-EAL7e+uMcpyoQ@mail.gmail.com>
Message-ID: <db88c569-366c-4c4d-af5b-779e3acc6790@gmx.at>

 > Breaking changes caused by new exports could be mitigated with a
library package or a GHC plugin with such auto refactoring functionality :

All of this is true. But from my side the worry here is less the effort
required to fix such issues.
Rather it's that users need to take any action at all when
upgrading from ghc-9.X.Y to ghc-9.X.Y+1 in order for their projects to
continue to compile.

Our goals here are:
* Ensure bug fix releases of libraries are shipped with GHC without
requiring maintainer attention.
* Try to keep breakage for users minimal.

In concept the idea of only bumping sumperminor versions makes sense, as
those typically don't contain any breaking changes.
But after looking through some of the changelogs of boot libraries it
seems like *many* bug fixes are currently released as minor
releases. As a consequence I don't see applying this policy only to
superminor versions as productive unless the release practices
of boot libraries change significantly which seems unlikely.

Similarly in theory we can disregard concerns about breakage with a
reference to best practices or paint users as responsible for open
imports. But that won't change the fact that it's common practice to use
open imports and as a result there will be breakage from such
changes.

So this leaves us with:

## Always bump minor versions [unless there are objections].

This ensures bug fixes make it into releases reliably with little
maintainer burden, but will result in breakage for some projects
when upgrading from ghc-X.Y.Z to ghc-X.Y.Z+1.

Breakage from such issues tends to be immediately observed as tied to a
ghc upgrade, and as such we tend to more reliably hear of it.

## Never bump minor versions [unless explicitly requested by maintainers
of the library, or deemed necessary by ghc maintainers].

This reduces breakage from added imports, but means ghc might ship at
times with libraries which contain bugs for which a fix already existed.
It is hard to accurately judge the cost/benefit of this approach. As
such bugs typically are discovered once applications are deployed and as
ghc devs we might never hear of issues caused this way.

Some projects are also locked into boot library versions, and might want
to use newer versions for non-bugfix reasons.

-----------------

Personally I think we should bump minor versions by default despite it
occasionally causing breakage. However I think we should also be willing
to avoid a bump if it causes a large amount of known breakage while not
providing any bug fixes.

For example we should probably not upgrade point releases from
text-2.1.1 to text-2.1.2 as the addition of `show` causes breakage and
there seem to be no bug fixes in the release, unless the library authors
request us to do so.






Am 28/11/2024 um 08:44 schrieb Imants Cekusins:
> On Wed, 27 Nov 2024 at 11:58, Tom Ellis
> <tom-lists-haskell-cafe-2023 at jaguarpaw.co.uk> wrote:
>
>     Thanks Adam,
>
>     I'm not too familiar with the GHC-side requirements here, but I wonder
>     if we can approach this issue from a different direction: is there
>     something we can do to help boot libraries avoid breaking changes?
>
>     Tom
>
>
>
> Breaking changes caused by new exports could be mitigated with a
> library package or a GHC plugin with such auto refactoring functionality :
>
> * Uniquely qualify (i.e., leave no different modules similarly
> qualified in the same module) each imported module and each imported
> symbol.
>
> This refactoring should be possible in code which already compiles.
> It would not require syntax / grammar changes.
>
> Something similar to smuggler2 package.
>
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241128/6195815f/attachment.html>

From zubin at well-typed.com  Fri Nov 29 12:33:23 2024
From: zubin at well-typed.com (Zubin Duggal)
Date: Fri, 29 Nov 2024 18:03:23 +0530
Subject: [ANNOUNCE] GHC 9.12.1-rc1 is now available
Message-ID: <dt4m5ldbndxvuc7d72hdrwasq7yrl4dlffo7ggdqu2o7w6m6qi@rivfifncqdyj>

The GHC developers are very pleased to announce the availability
of the release candidate for GHC 9.12.1. Binary distributions, source
distributions, and documentation are available at [downloads.haskell.org][].

We hope to have this release available via ghcup shortly.

GHC 9.12 will bring a number of new features and improvements, including:

  * The new language extension [OrPatterns] allowing you to combine multiple
    pattern clauses into one.

  * The [MultilineStrings] language extension to allow you to more easily write
    strings spanning multiple lines in your source code.

  * Improvements to the OverloadedRecordDot extension, allowing the built-in
    `HasField` class to be used for records with fields of non lifted representations.

  * The [NamedDefaults] language extension has been introduced allowing you to
    define defaults for typeclasses other than `Num`.

  * More deterministic object code output, controlled by the `-fobject-determinism`
    flag, which improves determinism of builds a lot (though does not fully do so)
    at the cost of some compiler performance (1-2%). See #12935 for the details

  * GHC now accepts type syntax in expressions as part of [GHC Proposal #281].

  * The WASM backend now has support for TemplateHaskell.

  * ... and many more

A full accounting of changes can be found in the [release notes][].
As always, GHC's release status, including planned future releases, can
be found on the GHC Wiki [status][].

We would like to thank GitHub, IOG, the Zw3rk stake pool,
Well-Typed, Tweag I/O, Serokell, Equinix, SimSpace, the Haskell
Foundation, and other anonymous contributors whose on-going financial
and in-kind support has facilitated GHC maintenance and release
management over the years. Finally, this release would not have been
possible without the hundreds of open-source contributors whose work
comprise this release.

As always, do give this release a try and open a [ticket][] if you see
anything amiss.

Cheers,
Zubin

[release notes]: https://downloads.haskell.org/ghc/9.12.1-rc1/docs/users_guide/9.12.1-notes.html
[status]: https://gitlab.haskell.org/ghc/ghc/-/wikis/GHC-status
[downloads.haskell.org]: https://downloads.haskell.org/ghc/9.12.1-rc1
[ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new
[OrPatterns]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0522-or-patterns.rst
[MultilineStrings]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0569-multiline-strings.rst
[GHC Proposal #281]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0281-visible-forall.rst
[NamedDefaults]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0409-exportable-named-default.rst

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20241129/14ad3934/attachment.sig>