From jaro.reinders at gmail.com Tue Aug 2 13:31:57 2022 From: jaro.reinders at gmail.com (J. Reinders) Date: Tue, 2 Aug 2022 15:31:57 +0200 Subject: Mixed boxed/unboxed arrays? Message-ID: <8EEA5871-56E1-4A01-B71F-0ED6B430C152@gmail.com> Hi GHC devs, I’ve been investigating fast hash table implementations. In particular hash tables used for counting unique items. For this use case, I believe the most performant hash tables are, in C terms, arrays of structures with a (boxed) pointer to the key, which is the item that we are counting, and an (unboxed) integer which holds the actual count. I already know of the ‘vector-hashtables’ package which uses two separate arrays, for example one boxed to hold the keys and one unboxed to hold the counts. However, I believe it can be quite important to store all the elements in the same array as that can reduce the number of cache misses. Because with random access to two arrays there is a higher chance that there will be two cache misses even if it immediately finds the right key in the hash table. So, I have also been looking at the low level arrays from the ‘primitive’ package and even in GHC.Exts, but I don’t believe it is currently possible to create a single array that contains both boxed and unboxed elements. Have I overlooked something? Or else, would it be possible to support this use case in a future version of GHC? Cheers, Jaro From ietf-dane at dukhovni.org Tue Aug 2 15:06:25 2022 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Tue, 2 Aug 2022 11:06:25 -0400 Subject: Mixed boxed/unboxed arrays? In-Reply-To: <8EEA5871-56E1-4A01-B71F-0ED6B430C152@gmail.com> References: <8EEA5871-56E1-4A01-B71F-0ED6B430C152@gmail.com> Message-ID: On Tue, Aug 02, 2022 at 03:31:57PM +0200, J. Reinders wrote: > I’ve been investigating fast hash table implementations. In particular > hash tables used for counting unique items. For this use case, I > believe the most performant hash tables are, in C terms, arrays of > structures with a (boxed) pointer to the key, which is the item that > we are counting, and an (unboxed) integer which holds the actual > count. > > I already know of the ‘vector-hashtables’ package which uses two > separate arrays, for example one boxed to hold the keys and one > unboxed to hold the counts. However, I believe it can be quite > important to store all the elements in the same array as that can > reduce the number of cache misses. Because with random access to two > arrays there is a higher chance that there will be two cache misses > even if it immediately finds the right key in the hash table. Could you use `StablePtr` for the keys? https://downloads.haskell.org/~ghc/latest/docs/html/libraries/base-4.16.1.0/GHC-Stable.html The corresponding `Ptr` can be stored in an unboxed Storable array along with the count. This comes at the cost of later having to explicitly free each StablePtr. https://downloads.haskell.org/~ghc/latest/docs/html/libraries/base-4.16.1.0/GHC-Stable.html#v:freeStablePtr How does the cost of computing object hashes and comparing colliding objects compare with the potential cache miss cost of using boxed integers or a separate array? Would such an "optimisation" be worth the effort? -- Viktor. From jaro.reinders at gmail.com Tue Aug 2 15:32:58 2022 From: jaro.reinders at gmail.com (J. Reinders) Date: Tue, 2 Aug 2022 17:32:58 +0200 Subject: Mixed boxed/unboxed arrays? In-Reply-To: References: <8EEA5871-56E1-4A01-B71F-0ED6B430C152@gmail.com> Message-ID: <864C4634-6BA6-49FF-8FC8-873FDC4E9254@gmail.com> > Could you use `StablePtr` for the keys? That might be an option, but I have no idea how performant stable pointers are and manual management is obviously not ideal. > How does the cost of computing object hashes and comparing colliding > objects compare with the potential cache miss cost of using boxed > integers or a separate array? Would such an "optimisation" be worth > the effort? Literature on hash tables suggests that cache misses were a very important factor in running time (in 2001): https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.25.4189 I don’t know whether it has become less or more important now, but I have been told there haven’t been that many advances in memory latency. From matthewtpickering at gmail.com Tue Aug 2 16:29:53 2022 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Tue, 2 Aug 2022 17:29:53 +0100 Subject: [ANNOUNCE] GHC 9.4.1-rc1 is now available In-Reply-To: <20220725.132317.759628945463268605.kazu@iij.ad.jp> References: <87bktgyo9s.fsf@smart-cactus.org> <20220725.132317.759628945463268605.kazu@iij.ad.jp> Message-ID: George, Kazu, I also can't reproduce on the mac which I can access over SSH. I downloaded the bindist for 9.2.4 and 9.4.1-rc1 and could install them both and run the binaries. Matt On Mon, Jul 25, 2022 at 5:23 AM Kazu Yamamoto (山本和彦) via ghc-devs wrote: > > Hi George, > > > I've duplicated the issue on both of my machines. It would be good to know > > if anybody else is seeing it. Kazu, I know you have seen this in the past. > > Do you get the same error installing rc1? > > When I run sudo make install I get a popup that says: > > I had no problem on 9.4.1-rc1. > "xattr -rc ." and "make install" worked perfectly. > > macOS Monterey v12.4 > Xcode 13.4.1 > > --Kazu > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From klebinger.andreas at gmx.at Tue Aug 2 18:24:33 2022 From: klebinger.andreas at gmx.at (Andreas Klebinger) Date: Tue, 2 Aug 2022 20:24:33 +0200 Subject: Mixed boxed/unboxed arrays? In-Reply-To: <864C4634-6BA6-49FF-8FC8-873FDC4E9254@gmail.com> References: <8EEA5871-56E1-4A01-B71F-0ED6B430C152@gmail.com> <864C4634-6BA6-49FF-8FC8-873FDC4E9254@gmail.com> Message-ID: I think it's possible to do this *today* using unsafeCoerce#. I was able to come up with this basic example below. In practice one would at the very least want to abstract away the gnarly stuff inside a library. But since it sounds like you want to be the one to write a library that shouldn't be a problem. {-# LANGUAGE MagicHash #-} {-# LANGUAGE UnboxedTuples #-} {-# LANGUAGE UnliftedDatatypes #-} moduleMainwhere importGHC.Exts importGHC.IO importUnsafe.Coerce importData.Kind dataSA= SA (SmallMutableArray# RealWorldAny) mkArray:: Int-> a-> IO(SA) mkArray (I# n) initial = IO $ \s -> caseunsafeCoerce# (newSmallArray# n initial s) of         (# s', arr #) -> (# s', SA arr #) readLifted:: SA-> Int-> IOa readLifted (SA arr) (I# i) = IO (\s ->     unsafeCoerce# (readSmallArray# arr i s)     ) dataUWrap(a:: UnliftedType) = UWrap a -- UWrap is just here because we can't return unlifted types in IO -- If you don't need your result in IO you can eliminate this indirection. readUnlifted:: foralla. SA-> Int-> IO(UWrapa) readUnlifted (SA arr) (I# i) = IO (\s -> caseunsafeCoerce# (readSmallArray# arr i s) of         (# s', a :: a#) -> (# s', UWrap a #)     ) writeLifted:: a-> Int-> SA-> IO() writeLifted x (I# i) (SA arr) = IO $ \s -> casewriteSmallArray# (unsafeCoerce# arr) i x s of         s -> (# s, ()#) writeUnlifted:: (a:: UnliftedType) -> Int-> SA-> IO() writeUnlifted x (I# i) (SA arr) = IO $ \s -> casewriteSmallArray# arr i (unsafeCoerce# x) s of         s -> (# s, ()#) typeUB:: UnliftedType dataUB= UT | UF showU:: UWrapUB-> String showU (UWrap UT) = "UT" showU (UWrap UF) = "UF" main:: IO() main = do     arr <- mkArray 4()     writeLifted True 0arr     writeLifted False 1arr     writeUnlifted UT 2arr     writeUnlifted UT 3arr     (readLifted arr 0:: IOBool) >>= print     (readLifted arr 1:: IOBool) >>= print     (readUnlifted arr 2:: IO(UWrapUB)) >>= (putStrLn . showU)     (readUnlifted arr 3:: IO(UWrapUB)) >>= (putStrLn . showU)     return () Cheers Andreas Am 02/08/2022 um 17:32 schrieb J. Reinders: >> Could you use `StablePtr` for the keys? > That might be an option, but I have no idea how performant stable pointers are and manual management is obviously not ideal. > >> How does the cost of computing object hashes and comparing colliding >> objects compare with the potential cache miss cost of using boxed >> integers or a separate array? Would such an "optimisation" be worth >> the effort? > Literature on hash tables suggests that cache misses were a very important factor in running time (in 2001):https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.25.4189 > > I don’t know whether it has become less or more important now, but I have been told there haven’t been that many advances in memory latency. > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From jaro.reinders at gmail.com Tue Aug 2 18:51:44 2022 From: jaro.reinders at gmail.com (Jaro Reinders) Date: Tue, 2 Aug 2022 20:51:44 +0200 Subject: Mixed boxed/unboxed arrays? In-Reply-To: References: <8EEA5871-56E1-4A01-B71F-0ED6B430C152@gmail.com> <864C4634-6BA6-49FF-8FC8-873FDC4E9254@gmail.com> Message-ID: <77955754-cb22-b273-88cb-ddc1ceb7306b@gmail.com> It seems you have misunderstood me. I want to store *unboxed* Int#s inside the array, not just some unlifted types. Surely in the case of unboxed integers the unsafeCoerce# function can make the garbage collector crash as they might be interpreted as arbitrary pointers. Cheers, Jaro On 02/08/2022 20:24, Andreas Klebinger wrote: > > I think it's possible to do this *today* using unsafeCoerce#. > > I was able to come up with this basic example below. In practice one > would at the very least want to abstract away the gnarly stuff inside a > library. But since it sounds like you want to be the one to write a > library that shouldn't be a problem. > > {-# LANGUAGE MagicHash #-} > {-# LANGUAGE UnboxedTuples #-} > {-# LANGUAGE UnliftedDatatypes #-} > moduleMainwhere > importGHC.Exts > importGHC.IO > importUnsafe.Coerce > importData.Kind > dataSA= SA (SmallMutableArray# RealWorldAny) > mkArray:: Int-> a-> IO(SA) > mkArray (I# n) initial = IO $ \s -> > caseunsafeCoerce# (newSmallArray# n initial s) of >         (# s', arr #) -> (# s', SA arr #) > readLifted:: SA-> Int-> IOa > readLifted (SA arr) (I# i) = IO (\s -> >     unsafeCoerce# (readSmallArray# arr i s) >     ) > dataUWrap(a:: UnliftedType) = UWrap a > -- UWrap is just here because we can't return unlifted types in IO > -- If you don't need your result in IO you can eliminate this indirection. > readUnlifted:: foralla. SA-> Int-> IO(UWrapa) > readUnlifted (SA arr) (I# i) = IO (\s -> > caseunsafeCoerce# (readSmallArray# arr i s) of >         (# s', a :: a#) -> (# s', UWrap a #) >     ) > writeLifted:: a-> Int-> SA-> IO() > writeLifted x (I# i) (SA arr) = IO $ \s -> > casewriteSmallArray# (unsafeCoerce# arr) i x s of >         s -> (# s, ()#) > writeUnlifted:: (a:: UnliftedType) -> Int-> SA-> IO() > writeUnlifted x (I# i) (SA arr) = IO $ \s -> > casewriteSmallArray# arr i (unsafeCoerce# x) s of >         s -> (# s, ()#) > typeUB:: UnliftedType > dataUB= UT | UF > showU:: UWrapUB-> String > showU (UWrap UT) = "UT" > showU (UWrap UF) = "UF" > main:: IO() > main = do >     arr <- mkArray 4() >     writeLifted True 0arr >     writeLifted False 1arr >     writeUnlifted UT 2arr >     writeUnlifted UT 3arr >     (readLifted arr 0:: IOBool) >>= print >     (readLifted arr 1:: IOBool) >>= print >     (readUnlifted arr 2:: IO(UWrapUB)) >>= (putStrLn . showU) >     (readUnlifted arr 3:: IO(UWrapUB)) >>= (putStrLn . showU) >     return () > > Cheers > > Andreas > > Am 02/08/2022 um 17:32 schrieb J. Reinders: >>> Could you use `StablePtr` for the keys? >> That might be an option, but I have no idea how performant stable pointers are and manual management is obviously not ideal. >> >>> How does the cost of computing object hashes and comparing colliding >>> objects compare with the potential cache miss cost of using boxed >>> integers or a separate array? Would such an "optimisation" be worth >>> the effort? >> Literature on hash tables suggests that cache misses were a very important factor in running time (in 2001):https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.25.4189 >> >> I don’t know whether it has become less or more important now, but I have been told there haven’t been that many advances in memory latency. >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From klebinger.andreas at gmx.at Tue Aug 2 19:25:33 2022 From: klebinger.andreas at gmx.at (Andreas Klebinger) Date: Tue, 2 Aug 2022 21:25:33 +0200 Subject: Mixed boxed/unboxed arrays? In-Reply-To: <77955754-cb22-b273-88cb-ddc1ceb7306b@gmail.com> References: <8EEA5871-56E1-4A01-B71F-0ED6B430C152@gmail.com> <864C4634-6BA6-49FF-8FC8-873FDC4E9254@gmail.com> <77955754-cb22-b273-88cb-ddc1ceb7306b@gmail.com> Message-ID: <8f386fdd-ae86-9775-534e-e11cc8cf8094@gmx.at> Indeed I misunderstood. As you already suspected this wouldn't work for Int# (or other unboxed types) sadly as the GC would assume these to be pointers which no doubt would lead to segfaults or worse. Rereading your initial mail I can say the runtime currently doesn't support such a heap object. If I understand you correctly what you would like is basically a something like: Con n P I# P I# P I#  ...        \/ \/\/      Pair1 Pair2 Pair3 ... Where n gives the number of pairs. I can see how it might be feasible to add a heap object like this to GHC but I'm unsure if it would be worth the complexity as it's layout diverges quite a bit from what GHC usually expects. The other option would be to expose to users a way to have an object that consist of a given number of words and a bitmap which indicates to the GHC which fields are pointers. This is more or less the representation that's already used to deal with stack frames iirc so that might not be as far fetched as it seems at first. It might even be possible to implement some sort of prototype for this using hand written Cmm. But there are not any plans to implement anything like this as far as I know. Am 02/08/2022 um 20:51 schrieb Jaro Reinders: > > It seems you have misunderstood me. I want to store *unboxed* Int#s > inside the array, not just some unlifted types. Surely in the case of > unboxed integers the unsafeCoerce# function can make the garbage > collector crash as they might be interpreted as arbitrary pointers. > > Cheers, > > Jaro > > On 02/08/2022 20:24, Andreas Klebinger wrote: >> >> I think it's possible to do this *today* using unsafeCoerce#. >> >> I was able to come up with this basic example below. In practice one >> would at the very least want to abstract away the gnarly stuff inside a >> library. But since it sounds like you want to be the one to write a >> library that shouldn't be a problem. >> >> {-# LANGUAGE MagicHash #-} >> {-# LANGUAGE UnboxedTuples #-} >> {-# LANGUAGE UnliftedDatatypes #-} >> moduleMainwhere >> importGHC.Exts >> importGHC.IO >> importUnsafe.Coerce >> importData.Kind >> dataSA= SA (SmallMutableArray# RealWorldAny) >> mkArray:: Int-> a-> IO(SA) >> mkArray (I# n) initial = IO $ \s -> >> caseunsafeCoerce# (newSmallArray# n initial s) of >>         (# s', arr #) -> (# s', SA arr #) >> readLifted:: SA-> Int-> IOa >> readLifted (SA arr) (I# i) = IO (\s -> >>     unsafeCoerce# (readSmallArray# arr i s) >>     ) >> dataUWrap(a:: UnliftedType) = UWrap a >> -- UWrap is just here because we can't return unlifted types in IO >> -- If you don't need your result in IO you can eliminate this >> indirection. >> readUnlifted:: foralla. SA-> Int-> IO(UWrapa) >> readUnlifted (SA arr) (I# i) = IO (\s -> >> caseunsafeCoerce# (readSmallArray# arr i s) of >>         (# s', a :: a#) -> (# s', UWrap a #) >>     ) >> writeLifted:: a-> Int-> SA-> IO() >> writeLifted x (I# i) (SA arr) = IO $ \s -> >> casewriteSmallArray# (unsafeCoerce# arr) i x s of >>         s -> (# s, ()#) >> writeUnlifted:: (a:: UnliftedType) -> Int-> SA-> IO() >> writeUnlifted x (I# i) (SA arr) = IO $ \s -> >> casewriteSmallArray# arr i (unsafeCoerce# x) s of >>         s -> (# s, ()#) >> typeUB:: UnliftedType >> dataUB= UT | UF >> showU:: UWrapUB-> String >> showU (UWrap UT) = "UT" >> showU (UWrap UF) = "UF" >> main:: IO() >> main = do >>     arr <- mkArray 4() >>     writeLifted True 0arr >>     writeLifted False 1arr >>     writeUnlifted UT 2arr >>     writeUnlifted UT 3arr >>     (readLifted arr 0:: IOBool) >>= print >>     (readLifted arr 1:: IOBool) >>= print >>     (readUnlifted arr 2:: IO(UWrapUB)) >>= (putStrLn . showU) >>     (readUnlifted arr 3:: IO(UWrapUB)) >>= (putStrLn . showU) >>     return () >> >> Cheers >> >> Andreas >> >> Am 02/08/2022 um 17:32 schrieb J. Reinders: >>>> Could you use `StablePtr` for the keys? >>> That might be an option, but I have no idea how performant stable pointers are and manual management is obviously not ideal. >>> >>>> How does the cost of computing object hashes and comparing colliding >>>> objects compare with the potential cache miss cost of using boxed >>>> integers or a separate array? Would such an "optimisation" be worth >>>> the effort? >>> Literature on hash tables suggests that cache misses were a very important factor in running time (in 2001):https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.25.4189 >>> >>> I don’t know whether it has become less or more important now, but I have been told there haven’t been that many advances in memory latency. >>> _______________________________________________ >>> ghc-devs mailing list >>> ghc-devs at haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From ietf-dane at dukhovni.org Wed Aug 3 03:30:04 2022 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Tue, 2 Aug 2022 23:30:04 -0400 Subject: Mixed boxed/unboxed arrays? In-Reply-To: <864C4634-6BA6-49FF-8FC8-873FDC4E9254@gmail.com> References: <8EEA5871-56E1-4A01-B71F-0ED6B430C152@gmail.com> <864C4634-6BA6-49FF-8FC8-873FDC4E9254@gmail.com> Message-ID: On Tue, Aug 02, 2022 at 05:32:58PM +0200, J. Reinders wrote: > > Could you use `StablePtr` for the keys? > > That might be an option, but I have no idea how performant stable > pointers are and manual management is obviously not ideal. If your hash table keys qualify for being stored in a "compact region", you may not need per-key stable pointers, just (carefully) coercing the keys to pointers suffices to produce primitive "handles" that are stable for the lifetime of the "compact region". The inverse (unsafe) coercion recovers the key. This also has the advantage that a key count does not incur a high ongoing GC cost. The keys are of course copied into the compact region. With this you could store "pointer + count" in a primitive cell. The hash table then holds a reference to the compact region and compacts keys on insert. https://hackage.haskell.org/package/compact-0.2.0.0/docs/Data-Compact.html -- Viktor. From gergo at erdi.hu Wed Aug 3 10:22:40 2022 From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=) Date: Wed, 3 Aug 2022 18:22:40 +0800 (+08) Subject: Wildcards in type synonyms In-Reply-To: References: Message-ID: In case anyone's still interested, Atze Dijkstra and I have come up with an alternative approach to implementing this which requires changing much fewer moving parts. The idea is to internally regard `type MySyn a = T[_, a, _]` as `type MySyn w1 w2 a = T[w1, a, w2]`, recording in `SynTyCon` that we'll need to synthesize 2 wildcard argument; then, during typechecking of occurrences, `MySyn` gets expanded into `MySyn _ _`. This is basically a simplified version of what languages with implicit parameters do -- in Agda syntax, we'd have MySyn : {w1 : Type} {w2 : Type} (a : Type) -> Type and the application `MySyn A` is short-hand for `MySyn {w1 = _} {w2 = _} A`. Of course, for a robust implementation, we can't just put all these implicit parameters up front, because they can form a telescope with other parameters; but, because type synonym applications need to be saturated anyway, I think even that could be implemented without complicated impredicativity decisions, simply by storing a bit more than just a single natural number as the extra info in `SynTyCon`. From jaro.reinders at gmail.com Wed Aug 3 20:16:50 2022 From: jaro.reinders at gmail.com (J. Reinders) Date: Wed, 3 Aug 2022 22:16:50 +0200 Subject: Mixed boxed/unboxed arrays? In-Reply-To: References: <8EEA5871-56E1-4A01-B71F-0ED6B430C152@gmail.com> <864C4634-6BA6-49FF-8FC8-873FDC4E9254@gmail.com> Message-ID: <6D754302-80F1-44D9-97C2-949FF154D63C@gmail.com> Thanks for your suggestion. That sounds like a promising technique. I have an implementation that mostly works here: https://github.com/noughtmare/clutter in the src/Counter.hs file. The only problem is that I get segfaults or internal GHC errors if I run it on large files. I’ve adding some tracing and it seems to occur when I try to coerce back pointers from the hash table array to proper Haskell values in the ’toList’ function. I can reproduce the problem by running ‘bash test.sh’. Currently, I’m using the ‘ptrToAny' and ‘anyToPtr' functions to do the coercing, because that sounds like the safest option. Do you know what’s going wrong or do you have a safer design for coercing the pointers? I thought it might be because the compact region gets deallocated before all the pointers are extracted, but even if I add a ’touch c’ (where c contains the compact region) at the end it still gives the same errors. > On 3 Aug 2022, at 05:30, Viktor Dukhovni wrote: > > On Tue, Aug 02, 2022 at 05:32:58PM +0200, J. Reinders wrote: > >>> Could you use `StablePtr` for the keys? >> >> That might be an option, but I have no idea how performant stable >> pointers are and manual management is obviously not ideal. > > If your hash table keys qualify for being stored in a "compact region", > you may not need per-key stable pointers, just (carefully) coercing the > keys to pointers suffices to produce primitive "handles" that are stable > for the lifetime of the "compact region". The inverse (unsafe) coercion > recovers the key. > > This also has the advantage that a key count does not incur a high > ongoing GC cost. The keys are of course copied into the compact region. > > With this you could store "pointer + count" in a primitive cell. The > hash table then holds a reference to the compact region and compacts > keys on insert. > > https://hackage.haskell.org/package/compact-0.2.0.0/docs/Data-Compact.html > > -- > Viktor. > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From jaro.reinders at gmail.com Wed Aug 3 20:35:43 2022 From: jaro.reinders at gmail.com (J. Reinders) Date: Wed, 3 Aug 2022 22:35:43 +0200 Subject: Mixed boxed/unboxed arrays? In-Reply-To: <6D754302-80F1-44D9-97C2-949FF154D63C@gmail.com> References: <8EEA5871-56E1-4A01-B71F-0ED6B430C152@gmail.com> <864C4634-6BA6-49FF-8FC8-873FDC4E9254@gmail.com> <6D754302-80F1-44D9-97C2-949FF154D63C@gmail.com> Message-ID: <9659DF2F-9B1C-446A-B134-E7A96C7309CF@gmail.com> I found the mistake: compactAdd c k p <- anyToPtr k Should be: p <- anyToPtr . getCompact =<< compactAdd c k Otherwise I guess I’m not using the pointer that’s on the compact region. From ietf-dane at dukhovni.org Wed Aug 3 20:50:31 2022 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Wed, 3 Aug 2022 16:50:31 -0400 Subject: Mixed boxed/unboxed arrays? In-Reply-To: <6D754302-80F1-44D9-97C2-949FF154D63C@gmail.com> References: <8EEA5871-56E1-4A01-B71F-0ED6B430C152@gmail.com> <864C4634-6BA6-49FF-8FC8-873FDC4E9254@gmail.com> <6D754302-80F1-44D9-97C2-949FF154D63C@gmail.com> Message-ID: On Wed, Aug 03, 2022 at 10:16:50PM +0200, J. Reinders wrote: > I have an implementation that mostly works here: > https://github.com/noughtmare/clutter > in the src/Counter.hs file. > > The only problem is that I get segfaults or internal GHC errors if I > run it on large files. I’ve adding some tracing and it seems to occur > when I try to coerce back pointers from the hash table array to proper > Haskell values in the ’toList’ function. Yes, this is delicate, requiring detailed knowledge of the internals. > Currently, I’m using the ‘ptrToAny' and ‘anyToPtr' functions to do the > coercing, because that sounds like the safest option. > > Do you know what’s going wrong or do you have a safer design for coercing the pointers? The code at: https://github.com/noughtmare/clutter/blob/main/src/Counter.hs#L50-L52 looks wrong. You're ignoring the return value of `compactAdd`, and coercing the original (non-compact) key to a pointer, but this is liable to be moved by GC. You need something like: p <- addCompact c k >>= getCompact >>= anyToPtr > I thought it might be because the compact region gets deallocated > before all the pointers are extracted, but even if I add a ’touch c’ > (where c contains the compact region) at the end it still gives the > same errors. Given the issue above, it is too early to speculate along these lines. It may also turn out that once the code works, it may be no faster or even much slower than the two-array approach. Compacting new keys has a cost, and perhaps that will dominate any speedup from combining the key and value in the same primitive cell. -- Viktor. From ietf-dane at dukhovni.org Wed Aug 3 20:51:30 2022 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Wed, 3 Aug 2022 16:51:30 -0400 Subject: Mixed boxed/unboxed arrays? In-Reply-To: <9659DF2F-9B1C-446A-B134-E7A96C7309CF@gmail.com> References: <8EEA5871-56E1-4A01-B71F-0ED6B430C152@gmail.com> <864C4634-6BA6-49FF-8FC8-873FDC4E9254@gmail.com> <6D754302-80F1-44D9-97C2-949FF154D63C@gmail.com> <9659DF2F-9B1C-446A-B134-E7A96C7309CF@gmail.com> Message-ID: On Wed, Aug 03, 2022 at 10:35:43PM +0200, J. Reinders wrote: > I found the mistake: > > compactAdd c k > p <- anyToPtr k > > Should be: > > p <- anyToPtr . getCompact =<< compactAdd c k > > Otherwise I guess I’m not using the pointer that’s on the compact region. Correct, I started my reply to your previous message before seeing that you also found the same error. -- Viktor. From simon.peytonjones at gmail.com Thu Aug 4 22:45:58 2022 From: simon.peytonjones at gmail.com (Simon Peyton Jones) Date: Thu, 4 Aug 2022 23:45:58 +0100 Subject: Advice for implementing GADTs? In-Reply-To: References: <0251fb0b-b4b2-d91b-ca34-55e3b812e456@gmail.com> Message-ID: QUESTION 1: Are there any obviously important resources that I've overlooked? That's a good list. Ningning's thesis https://xnning.github.io/ is also good stuff. QUESTION 2: if my quick scan is correct, none of the papers mention the GHC technique of determining untouchability by assigning "levels" to type variables. Is there any written paper (outside the GHC sources) that discusses type levels? It is disgracefully undocumented, I'm afraid. Sorry. Didier Remy used similar ideas, in some INRIA papers I think. QUESTION 3: My impression is that: (a) type variable levels were introduced in order to clarify which MetaTyVars are "untouchable", but (b) levels now also check that type variables do not escape their quantification scope. (c) levels can also be used to figure out which variables are free in the type environment, and therefore should not be generalized over. Does this sound right? I suspect that I might be wrong about the last one... Correct about all three. Except that a unification variable is only untouchable if it comes from an outer level *and* there are some intervening Given equalities. If there are no equalities it's not untouchable. E.b. f = \x -> case x of Just y -> 3::Int Here the (3::Int) can affect the result type of the function because the Just pattern match does not bind any Given equalities (in a GADT like way). I keep meaning to write an updated version of Practical type inference for arbitrary rank types , but failing to get around to it! Simon On Fri, 29 Jul 2022 at 17:08, Benjamin Redelings < benjamin.redelings at gmail.com> wrote: > Hi, > > I've been working on implementing the Haskell type system for my > specialized Haskell interpreter. I have now constructed a system that can > type-check and run Haskell code that contains explicit type signatures, > type constraints, and arbitrary-rank types. > > I'm now thinking that I may need to implement GADTs -- i.e. constructors > that introduce local constraints, including equality constraints. I'm > looking at the paper "OutsideIn(X): Modular type inference with local > assumptions" from 2011. I have three questions about implementing GADTs -- > I'd be grateful for answers to any of them. > > > QUESTION 1: Are there any obviously important resources that I've > overlooked? > > The 2011 OutsideIn paper mentions several previous papers that seem quite > helpful: > > * Peyton Jones el at 2006. Simple Unification-based type inference for > GADTs > > * Schrijvers etal 2007. Towards open type functions for Haskell > > * Peyton Jones et al 2004. Wobbly Types: etc. > > * Schrijvers et al 2008. Type checking with open type functions. > > * Shrijvers et al 2009. Complete and decidable type inference for GADTs > > * Vytiniotis et al 2010. Let should not be generalized. > > And of course the GHC source code. (I'm not looking at coercions at the > present time, because my type-checker translates to the plain lambda > calculus without type annotations, not system F or F_C. Hopefully I can > remedy this later...) > > > QUESTION 2: if my quick scan is correct, none of the papers mention the > GHC technique of determining untouchability by assigning "levels" to type > variables. Is there any written paper (outside the GHC sources) that > discusses type levels? > > > QUESTION 3: My impression is that: > > (a) type variable levels were introduced in order to clarify which > MetaTyVars are "untouchable", but > > (b) levels now also check that type variables do not escape their > quantification scope. > > (c) levels can also be used to figure out which variables are free in the > type environment, and therefore should not be generalized over. > > Does this sound right? I suspect that I might be wrong about the last > one... > > > Thanks again, and sorry for the long e-mail. > -BenRI > > > On 1/18/22 8:55 PM, Benjamin Redelings wrote: > > Hi, > > 1. I think I have clarified my problem a bit. It is actually not related > to pattern bindings. Here's an example: > > h = let f c i = if i > 10 then c else g c 'b' > g 'a' w = f 'b' 10 > g z w = z > in (f 'a' (1::Int), f 'a' (1.0::Double)) > > If I am understanding the Haskell type system correctly, > > * the definitions of f and g form a recursive group > > * the monomorphism restriction is not invoked > > * the inner binding (to f and g) leads to a local value environment (LVE): > > { f :: Char -> a -> Char; g :: Char -> Char -> Char } > > with predicates (Num a, Ord a) > > 2. In this situation, Typing Haskell in Haskell suggests that we should > NOT apply the predicates to the environment because the type for g does not > contain 'a', and would become ambiguous (section 11.6.2). Instead, we > should only apply predicates to the environment if their type variables are > present in ALL types in the current declaration group. > > Since the predicates (Num a, and Ord a) are not retained, then we cannot > quantify over a. > > It seems like this should make `f` monomorphic in a, and thus we should > not be able apply (f 'a') to both (1::Int) and (1::Double). > > Does that make any sense? > > 3. GHC, however, compiles this just fine. However, if I add "default ()", > then it no longer compiles. > > 4. On further reflection, Typing Haskell in Haskell applies defaulting > rules when evaluating each binding, and not just at the top level. So this > might be part of where my code is going wrong. > > -BenRI > > On 1/15/22 11:09 AM, Benjamin Redelings wrote: > > Hi, > > 1. I'm reading "A Static semantics for Haskell" and trying to code it up. > I came across some odd behavior with pattern bindings, and I was wondering > if someone could explain or point me in the right direction. > > Suppose you have the declaration > > (x,y) = ('a',2) > > My current code is yielding: > > x :: Num a => Char > > y :: Num a => a > > However, I notice that ghci gives x the type Char, with no constraint, > which is what I would expect. It also gives y the type 'Num b => b', so I > don't think it is defaulting a to Int here. > > The weird results from my code stem from rule BIND-PRED in Figure 15 of > https://homepages.inf.ed.ac.uk/wadler/papers/staticsemantics/static-semantics.ps > > E |- bind ~~> \dicts : theta -> monobinds in bind : (LIE_{encl}, > theta => LVE) > > Here theta = ( Num a ) and LVE = { x :: Char, y :: a }. So, theta => LVE > is > > { x :: Num a => Char, y :: Num a => a } > > The obvious thing to do is avoid changing a type T to Num a => T if T does > not contain a. Also I'm not totally sure if that trick gets to the bottom > of the issue. However, the paper doesn't mention define theta => LVE that > way. Is there something else I should read on this? > > 2. If we just chop out predicates which mention variables not in the type > ( == ambiguous predicates?) then I'm not totally sure how to create code > for this. > > I would imagine that we have something like > > tup dn = ('a', fromInteger dn 2) > > x = case (tup dn) of (x,y) -> x > > y dn case (tup dn) of (x,y) -> y > > In this case its not clear where to get the `dn` argument of `tup` from, > in the definition of `x`. Can we pass in `undefined`? Should we do > something else? > > If anyone can shed light on this, I would be grateful :-) > > -BenRI > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at haskell.foundation Fri Aug 5 04:22:26 2022 From: david at haskell.foundation (David Christiansen) Date: Fri, 5 Aug 2022 06:22:26 +0200 Subject: Advice for implementing GADTs? In-Reply-To: References: <0251fb0b-b4b2-d91b-ca34-55e3b812e456@gmail.com> Message-ID: > > QUESTION 2: if my quick scan is correct, none of the papers mention the > GHC technique of determining untouchability by assigning "levels" to type > variables. Is there any written paper (outside the GHC sources) that > discusses type levels? > It is disgracefully undocumented, I'm afraid. Sorry. Didier Remy used > similar ideas, in some INRIA papers I think. > (I'm first answering now because I wasn't sure until this comment that it was the same concept of levels that I know of) A couple of resources do exist for learning this idea. There's a quite accessible description of the basic idea in Peter Sestoft's book "Programming Language Concepts" from about ten years ago. A collection of fancier and more efficient versions are laid out by Oleg Kiselyov here: https://okmij.org/ftp/ML/generalization.html . A good source back to Remy is this one: https://hal.inria.fr/inria-00077006/document /David -------------- next part -------------- An HTML attachment was scrubbed... URL: From douglas.wilson at gmail.com Fri Aug 5 08:41:09 2022 From: douglas.wilson at gmail.com (Douglas Wilson) Date: Fri, 5 Aug 2022 09:41:09 +0100 Subject: Gitlab labels In-Reply-To: <59c4441f-06a5-ce61-fd87-d5d358734840@haskell.foundation> References: <20220728095511.e55p2rfdbn7x73tn@zubin-msi> <59c4441f-06a5-ce61-fd87-d5d358734840@haskell.foundation> Message-ID: I agree this is annoying. I have found "quick actions" [1] useful in alleviating the frustruation somewhat. One can issue short commands, into the comment box. e.g /relabel ~HPC ~documentation ~"T::task" ~"P::high" there's a completion interface so it's very straightforward. [1] https://gitlab.haskell.org/help/user/project/quick_actions Regards, Douglas Wilson On 7/28/22, Bryan Richter via ghc-devs wrote: > I haven't looked into it yet. > > As I haven't seen this problem on any other instance of GitLab, I have a > hunch it's unique to gitlab.haskell.org. I'd like to look into it > eventually. > > -Bryan > > On 28/07/2022 12:55, Zubin Duggal wrote: >> The error is bogus, if you refresh the page you will see that the label >> has been updated. >> >> This seems to be a bug in Gitlab, and I've also been seeing it a lot >> more often recently. I think Bryan attempted to investigate at a certain >> point, but I don't know what came of it. >> >> On 22/07/28 10:51, Simon Peyton Jones wrote: >>> Whenever I try to add a label to a ticket I get "an error occurred while >>> updating labels". See below. >>> >>> When I repeat the exact same sequence, it works. >>> >>> If I add another label, again the error; repeating again works. >>> >>> This is tiresome and peculiar. Does anyone have any ideas? >>> >>> Simon >>> >>> [image: image.png] >> >> >> >>> _______________________________________________ >>> ghc-devs mailing list >>> ghc-devs at haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > From gergo at erdi.hu Fri Aug 5 10:17:33 2022 From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=) Date: Fri, 5 Aug 2022 18:17:33 +0800 (+08) Subject: Partial type synonyms -- first-class! Message-ID: As I mentioned at the end of the original thread (but probably no one was interested in this enough to read that far), after a breakthrough idea I have now started working on partial type synonyms that are NOT just defined using macro expansion semantics, and indeed can be a first-class abstraction. I think if I can iron out the wrinkles detailed at the bottom of this message, this would be good for a GHC proposal. 1. Defining partial type synonyms The idea is to typecheck the type synonym declaration type Syn a = MyType _ [a] Int into a type synonym that has some implicit (invisible) type parameters: type Syn {w1} a = MyType w1 [a] Int We don't need to come up with any new design for "what this means": this means exactly the same as type Syn w1 a = MyType w1 [a] Int which is something you can already write today. The only difference is how `w1` is given at use sites. 2. Using partial type synonyms When a partial type synonym is applied, its argument list is expanded by adding wildcard arguments for the invisible parameters. So this use site: Syn Double is typececked into the type application Syn {_} Double Again, we don't need to come up with any new design for "what this means": this is the same as writing `Syn _ Double` into the same position, it's just the `_` is not written in the surface syntax. In particular: * The level of the type variable conjured for the `_` is determined fully by where the `Syn Double` occurs, and not at all by the definition of `Syn` * Using the type synonym `Syn` is only valid where a wildcard occurrence would be valid. In particular, using `Syn` in a type signature causes the signature itself to be regarded as partial (`isCompleteHsSig` returns `False` for it). 3. Implementation I have a proof-of-concept implementation at https://gitlab.haskell.org/cactus/ghc/-/compare/master...cactus%2Fpartial-tysyns?from_project_id=1117 I tried to make it as small a change as I could. Its key points are: * In the renamer, we allow wildcards in `TySynCtx` context * In `tcTySynRhs`, we allow wildcards in the right-hand side, retrieve them from the result of typechecking, and intersperse them with the "normal" type parameters to get a well-scoped telescope. * About a third of the lines changed is just a boring refactoring of `isCompleteHsSig` to be able to look up type synonym `TyCon`s. This is needed because a type signature referring to partial type synonyms will be partial itself once those type synonym applications are elaborated. * When typechecking a type application, implicit arguments get filled with the result of `tcAnonWildCardOcc`. Now, my questions about the problems I've encountered along the way. I hope these questions are concrete enough that I can get concrete answers to them from this list: 1. How do I find the wildcard-originating tyvars robustly? Currently, I am using `candidateQTyVarsWithBinders` but that picks up "regular" out-of-scope tyvars as well. I believe this causes a regression of #17567; furthermore, it means if the user writes `type Syn a = Foo b` it is handled the same as if they wrote `type Syn a = Foo _`, but it would be better to reject it instead. 2. What should the `Role` of these implicit type parameters be? For now, I've went with `Nominal` for lack of a better answer. 3. When typechecking a type application, I need to insert wildcards for these implicit arguments. I am currently using `AnonTCB InvisArg` binders to represent these implicit type parameters, which means by the time I get to `tcInferTyApps`, they would get passed through `instantiate` to `tcInstInvisibleTyBinder`. I currently have a horrible kuldge here to check, before calling `tcInstInivisibleTyBinder`, that the invisible binder's kind doesn't meet the preconditions of it; and if it doesn't, I create a wildcard instead. But of course, it would be nicer if I could put something in the `TcTyConBinder` that identifies these binders properly. Thanks in advance, Gergo -- .--= ULLA! =-----------------. \ http://gergo.erdi.hu \ `---= gergo at erdi.hu =-------' From ben at well-typed.com Sun Aug 7 21:29:03 2022 From: ben at well-typed.com (Ben Gamari) Date: Sun, 07 Aug 2022 17:29:03 -0400 Subject: [ANNOUNCE] GHC 9.4.1 is now available Message-ID: <87v8r3wxo8.fsf@smart-cactus.org> The GHC developers are very pleased to announce the availability of GHC 9.4.1. Binary distributions, source distributions, and documentation are available at downloads.haskell.org: https://downloads.haskell.org/ghc/9.4.1 This release includes: - A new profiling mode, `-fprof-late`, which adds automatic cost-center annotations to all top-level functions *after* Core optimisation has run. This provides informative profiles while interfering significantly less with GHC's aggressive optimisations, making it easier to understand the performance of programs which depend upon simplification.. - A variety of plugin improvements including the introduction of a new plugin type, *defaulting plugins*, and the ability for typechecking plugins to rewrite type-families. - An improved constructed product result analysis, allowing unboxing of nested structures, and a new boxity analysis, leading to less reboxing. - Introduction of a tag-check elision optimisation, bringing significant performance improvements in strict programs. - Generalisation of a variety of primitive types to be levity polymorphic. Consequently, the `ArrayArray#` type can at long last be retired, replaced by standard `Array#`. - Introduction of the `\cases` syntax from [GHC proposal 0302]. - A complete overhaul of GHC's Windows support. This includes a migration to a fully Clang-based C toolchain, a deep refactoring of the linker, and many fixes in WinIO. - Support for multiple home packages, significantly improving support in IDEs and other tools for multi-package projects. - A refactoring of GHC's error message infrastructure, allowing GHC to provide diagnostic information to downstream consumers as structured data, greatly easing IDE support. - Significant compile-time improvements to runtime and memory consumption. - On overhaul of our packaging infrastructure, allowing full traceability of release artifacts and more reliable binary distributions. - Reintroduction of deep subsumption (which was previously dropped with the *simplified subsumption* change) as a language extension. - ... and much more. See the [release notes] for a full accounting. Note that, as 9.4.1 is the first release for which the released artifacts will all be generated by our Hadrian build system, it is possible that there will be packaging issues. If you enounter trouble while using a binary distribution, please open a [ticket]. Likewise, if you are a downstream packager, do consider migrating to [Hadrian] to run your build; the Hadrian build system can be built using `cabal-install`, `stack`, or the in-tree [bootstrap script]. See the accompanying [blog post] for details on migrating packaging to Hadrian. We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, Well-Typed, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation, and other anonymous contributors whose on-going financial and in-kind support has facilitated GHC maintenance and release management over the years. Finally, this release would not have been possible without the hundreds of open-source contributors whose work comprise this release. As always, do give this release a try and open a [ticket] if you see anything amiss. Happy testing, - Ben [GHC proposal 0302]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0302-cases.rst [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new [bootstrap script]: https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d11c07d35/hadrian/bootstrap/README.md [Hadrian]: https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d11c07d35/hadrian [release notes]: https://downloads.haskell.org/~ghc/9.4.1/docs/users_guide/9.4.1-notes.html [blog post]: https://www.haskell.org/ghc/blog/20220805-make-to-hadrian.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ryan.gl.scott at gmail.com Mon Aug 8 11:29:38 2022 From: ryan.gl.scott at gmail.com (Ryan Scott) Date: Mon, 8 Aug 2022 07:29:38 -0400 Subject: A macOS static linking mystery Message-ID: I'm trying to diagnose strange GHC linking behavior which, as far as I can tell, only occurs on macOS. I feel a bit out of my league debugging this, so I'm hoping someone knows what is causing this. When building the Haskell libffi library [1], it will link pass -lffi to GHC. Typically, most installations of libffi will include both a static library (libffi.a) and a shared library (libffi.{so,dylib,dll}). My impression is that when the linker finds both, it will default to the shared library. This is definitely the case on Linux and Windows, at least. An exception to this rule is macOS, however. On macOS, building libffi always appears to default to linking against the static version of libffi, even when a dynamic version is also available. To reproduce this phenomenon, check out libffi [1] and run the following commands: $ brew install libffi # If it is not already installed $ cabal build ctime $ otool -L $(cabal list-bin ctime) The output reveals that the compiled executable does not dynamically link against libffi.dylib: $ otool -L $(cabal list-bin ctime) /Users/rscott/Documents/Hacking/Haskell/libffi/dist-newstyle/build/x86_64-osx/ghc-8.10.7/libffi-examples-0.1/x/ctime/build/ctime/ctime: /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1) /usr/lib/libcharset.1.dylib (compatibility version 2.0.0, current version 2.0.0) This is exceedingly strange, since my Hombrew installation does in fact provide libffi.dylib: $ ls -alh ~/Software/homebrew/Cellar/libffi/3.4.2/lib/ total 232 drwxr-xr-x 6 rscott 1340850540 192B Aug 7 09:38 . drwxr-xr-x 11 rscott 1340850540 352B Aug 7 08:51 .. -rw-r--r-- 1 rscott 1340850540 70K Aug 7 08:51 libffi.8.dylib -r--r--r-- 1 rscott 1340850540 41K Jun 28 2021 libffi.a lrwxr-xr-x 1 rscott 1340850540 14B Jun 28 2021 libffi.dylib -> libffi.8.dylib drwxr-xr-x 3 rscott 1340850540 96B Aug 7 08:51 pkgconfig What's more, this only seems to happen when building the libffi library in particular. If I try building another library that has a C extra-libraries dependency, such as HsOpenSSL, then I can find its dynamic dependencies (libssl.dylib and libcrypto.dylib) using otool: $ otool -L dist-newstyle/build/x86_64-osx/ghc-8.10.7/HsOpenSSL-0.11.7.2/build/test-dsa/test-dsa dist-newstyle/build/x86_64-osx/ghc-8.10.7/HsOpenSSL-0.11.7.2/build/test-dsa/test-dsa: /usr/local/opt/openssl at 1.1/lib/libssl.1.1.dylib (compatibility version 1.1.0, current version 1.1.0) /usr/local/opt/openssl at 1.1/lib/libcrypto.1.1.dylib (compatibility version 1.1.0, current version 1.1.0) /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1) /usr/lib/libcharset.1.dylib (compatibility version 2.0.0, current version 2.0.0) I'm at a bit of a loss trying to figure out why this only happens for libffi, and only on macOS. To make things even stranger, this only seems to happen when using GHC. If I try to compile a simple libffi C program using -lffi, for instance, then otool finds libffi.dylib. Is there something about GHC's linking behavior that would cause this? Ryan S. ----- [1] https://github.com/remiturk/libffi -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Mon Aug 8 13:41:48 2022 From: ben at well-typed.com (Ben Gamari) Date: Mon, 08 Aug 2022 09:41:48 -0400 Subject: [Haskell-cafe] [ANNOUNCE] GHC 9.4.1 is now available In-Reply-To: <9408ff89-2e27-4344-be77-c52e40d4bd99@Canary> References: <9408ff89-2e27-4344-be77-c52e40d4bd99@Canary> Message-ID: <87o7wux38f.fsf@smart-cactus.org> Bruno Damour writes: > Hello, > Thanks for this new release ! > Do you plan to add FreeBSD binaries ? Yes, I have recently been working on the FreeBSD CI infrastructure [1] and hope to have this finished in time for 9.4.2. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/6318 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ietf-dane at dukhovni.org Mon Aug 8 13:59:48 2022 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Mon, 8 Aug 2022 09:59:48 -0400 Subject: A macOS static linking mystery In-Reply-To: References: Message-ID: On Mon, Aug 08, 2022 at 07:29:38AM -0400, Ryan Scott wrote: > An exception to this rule is macOS, however. On macOS, building libffi > always appears to default to linking against the static version of libffi, > even when a dynamic version is also available. To reproduce this > phenomenon, check out libffi [1] and run the following commands: > > $ brew install libffi # If it is not already installed > $ cabal build ctime > $ otool -L $(cabal list-bin ctime) What is the output of $ pkg-config --libs libffi on this system? If "cabal" passes any additional flags to "pkg-config" use those as well. On my MacOS laptop I get: $ /usr/local/bin/pkg-config --libs libffi -lffi which does not use the "brew"-installed libffi. Not surprising, since /usr/local/lib/pkgconfig/ has no symlink to the "libffi.pc" file. > This is exceedingly strange, since my Hombrew installation does in fact > provide libffi.dylib: > > $ ls -alh ~/Software/homebrew/Cellar/libffi/3.4.2/lib/ > [...] > drwxr-xr-x 3 rscott 1340850540 96B Aug 7 08:51 pkgconfig For "pkg-config" to find the HomeBrew "libffi", there would need to be a "libffi.pc" symlink to the one in the "pkgconfig" directory. Perhaps there are additional steps to perform in HomeBrew to activate this "libffi" as a default target for "pkg-config". -- Viktor. From ietf-dane at dukhovni.org Mon Aug 8 14:46:16 2022 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Mon, 8 Aug 2022 10:46:16 -0400 Subject: A macOS static linking mystery In-Reply-To: References: Message-ID: On Mon, Aug 08, 2022 at 09:59:48AM -0400, Viktor Dukhovni wrote: > On my MacOS laptop I get: > > $ /usr/local/bin/pkg-config --libs libffi > -lffi > > which does not use the "brew"-installed libffi. Not surprising, since > /usr/local/lib/pkgconfig/ has no symlink to the "libffi.pc" file. When updating "libffi" HomeBrew reports: ==> libffi libffi is keg-only, which means it was not symlinked into /usr/local, because macOS already provides this software and installing another version in parallel can cause all kinds of trouble. For compilers to find libffi you may need to set: export LDFLAGS="-L/usr/local/opt/libffi/lib" export CPPFLAGS="-I/usr/local/opt/libffi/include" If the MacOS libffi works, it is probably safer to use it rather than the HomeBrew version. -- Viktor. From carter.schonwald at gmail.com Mon Aug 8 18:17:21 2022 From: carter.schonwald at gmail.com (Carter Schonwald) Date: Mon, 8 Aug 2022 14:17:21 -0400 Subject: A macOS static linking mystery In-Reply-To: References: Message-ID: ahh, pkgconfig is what i was overlooking! agreed with viktor, using the system provided one is definitely safer On Mon, Aug 8, 2022 at 10:46 AM Viktor Dukhovni wrote: > On Mon, Aug 08, 2022 at 09:59:48AM -0400, Viktor Dukhovni wrote: > > > On my MacOS laptop I get: > > > > $ /usr/local/bin/pkg-config --libs libffi > > -lffi > > > > which does not use the "brew"-installed libffi. Not surprising, since > > /usr/local/lib/pkgconfig/ has no symlink to the "libffi.pc" file. > > When updating "libffi" HomeBrew reports: > > ==> libffi > libffi is keg-only, which means it was not symlinked into /usr/local, > because macOS already provides this software and installing another > version in > parallel can cause all kinds of trouble. > > For compilers to find libffi you may need to set: > export LDFLAGS="-L/usr/local/opt/libffi/lib" > export CPPFLAGS="-I/usr/local/opt/libffi/include" > > If the MacOS libffi works, it is probably safer to use it rather than > the HomeBrew version. > > -- > Viktor. > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ryan.gl.scott at gmail.com Mon Aug 8 23:33:13 2022 From: ryan.gl.scott at gmail.com (Ryan Scott) Date: Mon, 8 Aug 2022 19:33:13 -0400 Subject: A macOS static linking mystery Message-ID: I should clarify that I'm using a borrowed macOS on which I don't have admin privileges, so I'm unable to install pkg-config. As a result, I'm commenting out the pkgconfig-depends: line in libffi.cabal and manually specifying the extra-lib-dirs and include-dirs via a cabal.project.local file. I've configured the project to use both the macOS system libffi (located at /usr/lib/libffi.dylib) as well as the version that Homebrew provides. Both configurations exhibit the strange, apparently-statically-linked behavior. Even stranger, the macOS system libffi _only_ provides a .dylib file, not an .a file. If I examine the x86 code of the compiled executable using `objdump -d`, I can see various libffi definitions (e.g., _ffi_call), so perhaps these definitions are being inlined through some kind of link-time optimization? I'm really not sure how else to explain what's happening. Ryan -------------- next part -------------- An HTML attachment was scrubbed... URL: From allbery.b at gmail.com Mon Aug 8 23:45:16 2022 From: allbery.b at gmail.com (Brandon Allbery) Date: Mon, 8 Aug 2022 19:45:16 -0400 Subject: A macOS static linking mystery In-Reply-To: References: Message-ID: Any chance this is related to the weird system cache thing for system dylibs that came in with the most recent OS X releases? I don't think those show up in the normal way. On Mon, Aug 8, 2022 at 7:33 PM Ryan Scott wrote: > > I should clarify that I'm using a borrowed macOS on which I don't have admin privileges, so I'm unable to install pkg-config. As a result, I'm commenting out the pkgconfig-depends: line in libffi.cabal and manually specifying the extra-lib-dirs and include-dirs via a cabal.project.local file. > > I've configured the project to use both the macOS system libffi (located at /usr/lib/libffi.dylib) as well as the version that Homebrew provides. Both configurations exhibit the strange, apparently-statically-linked behavior. Even stranger, the macOS system libffi _only_ provides a .dylib file, not an .a file. If I examine the x86 code of the compiled executable using `objdump -d`, I can see various libffi definitions (e.g., _ffi_call), so perhaps these definitions are being inlined through some kind of link-time optimization? I'm really not sure how else to explain what's happening. > > Ryan > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -- brandon s allbery kf8nh allbery.b at gmail.com From ryan.gl.scott at gmail.com Mon Aug 8 23:49:00 2022 From: ryan.gl.scott at gmail.com (Ryan Scott) Date: Mon, 8 Aug 2022 19:49:00 -0400 Subject: A macOS static linking mystery In-Reply-To: References: Message-ID: Possibly? I'm not familiar with the system cache you're referring to, but it's a more solid lead than what I have currently. Do you know how one would confirm if this is the case or not? Ryan -------------- next part -------------- An HTML attachment was scrubbed... URL: From allbery.b at gmail.com Mon Aug 8 23:59:54 2022 From: allbery.b at gmail.com (Brandon Allbery) Date: Mon, 8 Aug 2022 19:59:54 -0400 Subject: A macOS static linking mystery In-Reply-To: References: Message-ID: https://developer.apple.com/forums/thread/692383 is what I'm thinking of, but in this case there would still be library references shown by "otool -L", so I guess that's not what you're seeing. On Mon, Aug 8, 2022 at 7:49 PM Ryan Scott wrote: > > Possibly? I'm not familiar with the system cache you're referring to, but it's a more solid lead than what I have currently. Do you know how one would confirm if this is the case or not? > > Ryan -- brandon s allbery kf8nh allbery.b at gmail.com From gergo at erdi.hu Tue Aug 9 02:08:23 2022 From: gergo at erdi.hu (=?UTF-8?B?R2VyZ8WRIMOJcmRp?=) Date: Tue, 9 Aug 2022 10:08:23 +0800 Subject: Partial type synonyms -- first-class! In-Reply-To: References: Message-ID: I haven't heard back on this so I pushed ahead on the 3 outstanding problems by just applying more violence to the code base, basically. 1. For wildcard-originating type variables, I added a flag to the `TauTv` constructor of `MetaInfo`. The knock-on effects of this change were smaller than I expected: https://gitlab.haskell.org/cactus/ghc/-/commit/d3165c3f37e9455bc7ca88b884cc370ca39d07f8 2. It turns out I misunderstood the `Role` situation and they are actually computed via some weird knot tying from the `TyCon` itself. So on one hand, I don't really need to do anything to "come up with" roles for these implicit type parameters. On the other hand, I think this also means that this breaks the surface syntax for role declarations, since the user would be expected to give roles for type parameters they never explicitly added to their tysyns... 3. Similar to #1, I started just pushing all the way through GHC a change to `AnonArgFlag` that adds a third `ImplArg` flag. It is handled mostly the same way as an `InvisArg`, except `tcInferTyApps` can use it to insert wildcard arguments. I don't have a full commit for this yet. Beside comments on this approach, I would also like to hear from Simon & Richard if they agree that with this new approach, this could now be a real language feature that could get upstreamed once all these implementation wrinkles are smoothed out. Thanks, Gergo On Fri, Aug 5, 2022 at 6:17 PM ÉRDI Gergő wrote: > > As I mentioned at the end of the original thread (but probably no one > was interested in this enough to read that far), after a breakthrough > idea I have now started working on partial type synonyms that are NOT > just defined using macro expansion semantics, and indeed can be a > first-class abstraction. I think if I can iron out the wrinkles > detailed at the bottom of this message, this would be good for a GHC > proposal. > > 1. Defining partial type synonyms > > The idea is to typecheck the type synonym declaration > > type Syn a = MyType _ [a] Int > > into a type synonym that has some implicit (invisible) type parameters: > > type Syn {w1} a = MyType w1 [a] Int > > We don't need to come up with any new design for "what this means": > this means exactly the same as > > type Syn w1 a = MyType w1 [a] Int > > which is something you can already write today. The only difference is > how `w1` is given at use sites. > > 2. Using partial type synonyms > > When a partial type synonym is applied, its argument list is expanded > by adding wildcard arguments for the invisible parameters. So this use > site: > > Syn Double > > is typececked into the type application > > Syn {_} Double > > Again, we don't need to come up with any new design for "what this > means": this is the same as writing `Syn _ Double` into the same > position, it's just the `_` is not written in the surface syntax. > > In particular: > > * The level of the type variable conjured for the `_` is determined > fully by where the `Syn Double` occurs, and not at all by the > definition of `Syn` > > * Using the type synonym `Syn` is only valid where a wildcard > occurrence would be valid. In particular, using `Syn` in a type > signature causes the signature itself to be regarded as partial > (`isCompleteHsSig` returns `False` for it). > > 3. Implementation > > I have a proof-of-concept implementation at > https://gitlab.haskell.org/cactus/ghc/-/compare/master...cactus%2Fpartial-tysyns?from_project_id=1117 > > I tried to make it as small a change as I could. Its key points are: > > * In the renamer, we allow wildcards in `TySynCtx` context > > * In `tcTySynRhs`, we allow wildcards in the right-hand side, > retrieve them from the result of typechecking, and intersperse > them with the "normal" type parameters to get a well-scoped > telescope. > > * About a third of the lines changed is just a boring refactoring of > `isCompleteHsSig` to be able to look up type synonym > `TyCon`s. This is needed because a type signature referring to > partial type synonyms will be partial itself once those type > synonym applications are elaborated. > > * When typechecking a type application, implicit arguments get > filled with the result of `tcAnonWildCardOcc`. > > Now, my questions about the problems I've encountered along the way. I > hope these questions are concrete enough that I can get concrete > answers to them from this list: > > 1. How do I find the wildcard-originating tyvars robustly? Currently, > I am using `candidateQTyVarsWithBinders` but that picks up "regular" > out-of-scope tyvars as well. I believe this causes a regression of > #17567; furthermore, it means if the user writes `type Syn a = Foo b` it > is handled the same as if they wrote `type Syn a = Foo _`, but it would > be better to reject it instead. > > 2. What should the `Role` of these implicit type parameters be? For > now, I've went with `Nominal` for lack of a better answer. > > 3. When typechecking a type application, I need to insert wildcards > for these implicit arguments. I am currently using `AnonTCB InvisArg` > binders to represent these implicit type parameters, which means by > the time I get to `tcInferTyApps`, they would get passed through > `instantiate` to `tcInstInvisibleTyBinder`. I currently have a > horrible kuldge here to check, before calling > `tcInstInivisibleTyBinder`, that the invisible binder's kind doesn't > meet the preconditions of it; and if it doesn't, I create a wildcard > instead. But of course, it would be nicer if I could put something in > the `TcTyConBinder` that identifies these binders properly. > > Thanks in advance, > Gergo > > -- > > .--= ULLA! =-----------------. > \ http://gergo.erdi.hu \ > `---= gergo at erdi.hu =-------' From carter.schonwald at gmail.com Tue Aug 9 13:53:15 2022 From: carter.schonwald at gmail.com (Carter Schonwald) Date: Tue, 9 Aug 2022 09:53:15 -0400 Subject: A macOS static linking mystery In-Reply-To: References: Message-ID: i think brandon is also referring to the Rosetta stuff on Arm Macs, are you using one of those? On Mon, Aug 8, 2022 at 8:00 PM Brandon Allbery wrote: > https://developer.apple.com/forums/thread/692383 is what I'm thinking > of, but in this case there would still be library references shown by > "otool -L", so I guess that's not what you're seeing. > > On Mon, Aug 8, 2022 at 7:49 PM Ryan Scott wrote: > > > > Possibly? I'm not familiar with the system cache you're referring to, > but it's a more solid lead than what I have currently. Do you know how one > would confirm if this is the case or not? > > > > Ryan > > > > -- > brandon s allbery kf8nh > allbery.b at gmail.com > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ryan.gl.scott at gmail.com Tue Aug 9 13:58:29 2022 From: ryan.gl.scott at gmail.com (Ryan Scott) Date: Tue, 9 Aug 2022 09:58:29 -0400 Subject: A macOS static linking mystery In-Reply-To: References: Message-ID: No, I'm using macOS 10.15.7 (BuildVersion 19H2) on an x86_64 machine. Ryan -------------- next part -------------- An HTML attachment was scrubbed... URL: From george.colpitts at gmail.com Tue Aug 9 20:32:58 2022 From: george.colpitts at gmail.com (George Colpitts) Date: Tue, 9 Aug 2022 17:32:58 -0300 Subject: ghc doesn't work after installing 9.4.1 on my Mac Message-ID: Unfortunately ghc doesn't work after installing 9.4.1 on my Mac. Does it work for others? After the install finishes when I do the following: $ ghc --version bash: /usr/local/bin/ghc: Permission denied $ sudo chmod +x /usr/local/bin/ghc $ghc --version /usr/local/bin/ghc: line 1: exec: : not found $ cat /usr/local/bin/ghc exec "$executablename" -B"$libdir" ${1+"$@"} $ cat /usr/local/bin/ghc-9.4.1 #!/bin/sh exedir="/usr/local/lib/ghc-9.4.1/bin" exeprog="ghc-9.4.1" executablename="/usr/local/lib/ghc-9.4.1/bin/ghc-9.4.1" bindir="/usr/local/bin" libdir="/usr/local/lib/ghc-9.4.1/lib" docdir="/usr/local/share/doc/ghc-9.4.1" includedir="/usr/local/include" exec "$executablename" -B"$libdir" ${1+"$@"} Thanks George On Sun, Aug 7, 2022 at 6:30 PM Ben Gamari wrote: > The GHC developers are very pleased to announce the availability of GHC > 9.4.1. Binary distributions, source distributions, and documentation are > available at downloads.haskell.org: > > https://downloads.haskell.org/ghc/9.4.1 > > This release includes: > > - A new profiling mode, `-fprof-late`, which adds automatic cost-center > annotations to all top-level functions *after* Core optimisation has > run. This provides informative profiles while interfering > significantly less with GHC's aggressive optimisations, making it > easier to understand the performance of programs which depend upon > simplification.. > > - A variety of plugin improvements including the introduction of a new > plugin type, *defaulting plugins*, and the ability for typechecking > plugins to rewrite type-families. > > - An improved constructed product result analysis, allowing unboxing of > nested structures, and a new boxity analysis, leading to less reboxing. > > - Introduction of a tag-check elision optimisation, bringing > significant performance improvements in strict programs. > > - Generalisation of a variety of primitive types to be levity > polymorphic. Consequently, the `ArrayArray#` type can at long last be > retired, replaced by standard `Array#`. > > - Introduction of the `\cases` syntax from [GHC proposal 0302]. > > - A complete overhaul of GHC's Windows support. This includes a > migration to a fully Clang-based C toolchain, a deep refactoring of > the linker, and many fixes in WinIO. > > - Support for multiple home packages, significantly improving support > in IDEs and other tools for multi-package projects. > > - A refactoring of GHC's error message infrastructure, allowing GHC to > provide diagnostic information to downstream consumers as structured > data, greatly easing IDE support. > > - Significant compile-time improvements to runtime and memory consumption. > > - On overhaul of our packaging infrastructure, allowing full > traceability of release artifacts and more reliable binary > distributions. > > - Reintroduction of deep subsumption (which was previously dropped with > the > *simplified subsumption* change) as a language extension. > > - ... and much more. See the [release notes] for a full accounting. > > Note that, as 9.4.1 is the first release for which the released artifacts > will > all be generated by our Hadrian build system, it is possible that there > will be > packaging issues. If you enounter trouble while using a binary > distribution, > please open a [ticket]. Likewise, if you are a downstream packager, do > consider > migrating to [Hadrian] to run your build; the Hadrian build system can be > built > using `cabal-install`, `stack`, or the in-tree [bootstrap script]. See the > accompanying > [blog post] for details on migrating packaging to Hadrian. > > We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, > Well-Typed, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation, and > other anonymous contributors whose on-going financial and in-kind support > has > facilitated GHC maintenance and release management over the years. Finally, > this release would not have been possible without the hundreds of > open-source > contributors whose work comprise this release. > > As always, do give this release a try and open a [ticket] if you see > anything amiss. > > Happy testing, > > - Ben > > > [GHC proposal 0302]: > https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0302-cases.rst > [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new > [bootstrap script]: > https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d11c07d35/hadrian/bootstrap/README.md > [Hadrian]: > https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d11c07d35/hadrian > [release notes]: > https://downloads.haskell.org/~ghc/9.4.1/docs/users_guide/9.4.1-notes.html > [blog post]: > https://www.haskell.org/ghc/blog/20220805-make-to-hadrian.html > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shayne.fletcher.50 at gmail.com Tue Aug 9 22:40:48 2022 From: shayne.fletcher.50 at gmail.com (Shayne Fletcher) Date: Tue, 9 Aug 2022 18:40:48 -0400 Subject: ghc doesn't work after installing 9.4.1 on my Mac In-Reply-To: References: Message-ID: Known issue George. See e.g. https://gitlab.haskell.org/ghc/ghc/-/issues/21985 On Tue, Aug 9, 2022, 4:33 PM George Colpitts wrote: > Unfortunately ghc doesn't work after installing 9.4.1 on my Mac. Does it > work for others? > > After the install finishes when I do the following: > > $ ghc --version > bash: /usr/local/bin/ghc: Permission denied > $ sudo chmod +x /usr/local/bin/ghc > $ghc --version > /usr/local/bin/ghc: line 1: exec: : not found > $ cat /usr/local/bin/ghc > exec "$executablename" -B"$libdir" ${1+"$@"} > $ cat /usr/local/bin/ghc-9.4.1 > #!/bin/sh > exedir="/usr/local/lib/ghc-9.4.1/bin" > exeprog="ghc-9.4.1" > executablename="/usr/local/lib/ghc-9.4.1/bin/ghc-9.4.1" > bindir="/usr/local/bin" > libdir="/usr/local/lib/ghc-9.4.1/lib" > docdir="/usr/local/share/doc/ghc-9.4.1" > includedir="/usr/local/include" > > exec "$executablename" -B"$libdir" ${1+"$@"} > > > Thanks > George > > > > On Sun, Aug 7, 2022 at 6:30 PM Ben Gamari wrote: > >> The GHC developers are very pleased to announce the availability of GHC >> 9.4.1. Binary distributions, source distributions, and documentation are >> available at downloads.haskell.org: >> >> https://downloads.haskell.org/ghc/9.4.1 >> >> This release includes: >> >> - A new profiling mode, `-fprof-late`, which adds automatic cost-center >> annotations to all top-level functions *after* Core optimisation has >> run. This provides informative profiles while interfering >> significantly less with GHC's aggressive optimisations, making it >> easier to understand the performance of programs which depend upon >> simplification.. >> >> - A variety of plugin improvements including the introduction of a new >> plugin type, *defaulting plugins*, and the ability for typechecking >> plugins to rewrite type-families. >> >> - An improved constructed product result analysis, allowing unboxing of >> nested structures, and a new boxity analysis, leading to less reboxing. >> >> - Introduction of a tag-check elision optimisation, bringing >> significant performance improvements in strict programs. >> >> - Generalisation of a variety of primitive types to be levity >> polymorphic. Consequently, the `ArrayArray#` type can at long last be >> retired, replaced by standard `Array#`. >> >> - Introduction of the `\cases` syntax from [GHC proposal 0302]. >> >> - A complete overhaul of GHC's Windows support. This includes a >> migration to a fully Clang-based C toolchain, a deep refactoring of >> the linker, and many fixes in WinIO. >> >> - Support for multiple home packages, significantly improving support >> in IDEs and other tools for multi-package projects. >> >> - A refactoring of GHC's error message infrastructure, allowing GHC to >> provide diagnostic information to downstream consumers as structured >> data, greatly easing IDE support. >> >> - Significant compile-time improvements to runtime and memory >> consumption. >> >> - On overhaul of our packaging infrastructure, allowing full >> traceability of release artifacts and more reliable binary >> distributions. >> >> - Reintroduction of deep subsumption (which was previously dropped with >> the >> *simplified subsumption* change) as a language extension. >> >> - ... and much more. See the [release notes] for a full accounting. >> >> Note that, as 9.4.1 is the first release for which the released artifacts >> will >> all be generated by our Hadrian build system, it is possible that there >> will be >> packaging issues. If you enounter trouble while using a binary >> distribution, >> please open a [ticket]. Likewise, if you are a downstream packager, do >> consider >> migrating to [Hadrian] to run your build; the Hadrian build system can be >> built >> using `cabal-install`, `stack`, or the in-tree [bootstrap script]. See >> the accompanying >> [blog post] for details on migrating packaging to Hadrian. >> >> We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, >> Well-Typed, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation, >> and >> other anonymous contributors whose on-going financial and in-kind support >> has >> facilitated GHC maintenance and release management over the years. >> Finally, >> this release would not have been possible without the hundreds of >> open-source >> contributors whose work comprise this release. >> >> As always, do give this release a try and open a [ticket] if you see >> anything amiss. >> >> Happy testing, >> >> - Ben >> >> >> [GHC proposal 0302]: >> https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0302-cases.rst >> [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new >> [bootstrap script]: >> https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d11c07d35/hadrian/bootstrap/README.md >> [Hadrian]: >> https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d11c07d35/hadrian >> [release notes]: >> https://downloads.haskell.org/~ghc/9.4.1/docs/users_guide/9.4.1-notes.html >> [blog post]: >> https://www.haskell.org/ghc/blog/20220805-make-to-hadrian.html >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Tue Aug 9 22:44:49 2022 From: ben at well-typed.com (Ben Gamari) Date: Tue, 09 Aug 2022 18:44:49 -0400 Subject: [ANNOUNCE] GHC 9.4.1 is now available In-Reply-To: <87v8r3wxo8.fsf@smart-cactus.org> References: <87v8r3wxo8.fsf@smart-cactus.org> Message-ID: <87edxpvxyr.fsf@smart-cactus.org> Ben Gamari writes: > The GHC developers are very pleased to announce the availability of GHC > 9.4.1. Binary distributions, source distributions, and documentation are > available at downloads.haskell.org: > > https://downloads.haskell.org/ghc/9.4.1 > Hi all, Due to an unfortunate packaging issue, the macOS binary distributions for 9.4.1 are not usable as uploaded. The problem is described in #21974, which also includes a small patch to mitigate the breakage. We will be releasing a 9.4.2 within the week fixing the issue. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ryan.gl.scott at gmail.com Wed Aug 10 01:59:17 2022 From: ryan.gl.scott at gmail.com (Ryan Scott) Date: Tue, 9 Aug 2022 21:59:17 -0400 Subject: A macOS static linking mystery In-Reply-To: References: Message-ID: I believe I've figured out what is happening here. This will take a bit of explanation, so bear with me. First, it's important to note that this behavior depends on whether your GHC was configured with the `--with-system-libffi` flag or not. If it was configured with this flag, then GHC will always link against your system's version of libffi, and the behavior seen in this thread would not occur. I am using a bindist provided by ghcup which was _not_ configured with `--with-system-libffi`, so GHC bundles its own copy of libffi. Interestingly, there appear to be at least two copies of libffi that are bundled: one dynamic library, simply named libffi.dylib, and one static library, which is named libCffi.a [1]. Remember the name libCffi.a, since it will come up again later. Next, let's take a look at how GHC performs the final linking step when compiling an executable. GHC will call the C compiler to perform this final linking step, which will have roughly the following shape on macOS: gcc ... -L/rts -lCffi -Wl,-dead_strip_dylibs -o Main I'm omitting quite a few details here, but the important bits are that (1) -lCffi comes before any of the C libraries listed in an `extra-libraries` stanza in a .cabal file, and (2) GHC on macOS will pass -Wl,-dead_strip_dylibs, which causes the linker to remove any references to shared libraries that go unused. In the particular case of libffi.cabal, we have `extra-libraries: ffi`, so the C compiler is given `-lCffi -lffi` as arguments, in that order. Recall that -libCffi.a (i.e., -lCffi) is a statically linked copy of libffi, whereas libffi.dylib (i.e., -lffi) is a dynamically linked version. The linker will resolve all of its references to libffi functionality from libCffi.a because it appears first, and since the linker never needs to resolve anything from libffi.dylib, the -Wl,-dead_strip_dylibs will remove the reference to libffi.dylib entirely. The result is that the final executable will statically link against libffi. This is why this phenomenon only occurs with libffi and not with other C libraries, since only libffi is given special treatment within GHC like this. Why, then, does this only happen on macOS and not on, say, Linux? On Linux, the final linking step looks slightly different. Instead of passing -Wl,-dead_strip_dylibs (which is a macOS-specific option), it will pass -Wl,--no-as-needed. Note that -Wl,--no-as-needed has the entirely _opposite_ effect of -Wl,-dead_strip_dylibs: even if a shared library never gets referenced, the linker will still declare a dynamic dependency on it. That is exactly what happens here, as the -lCffi argument will still take precedence over the -lffi argument on Linux, but there will be a useless dynamic dependency on a dynamic libffi library due to -Wl,--no-as-needed. At this point, I realized that one consequence of all this is that if you want to statically link against against libffi, and you are using a GHC configured without ` --with-system-libffi`, then it suffices to simply compile your program without any `extra-libraries` at all. That is because the final link step will pass `-lCffi` no matter what, so the resulting executable will always statically link against libffi by way of libCffi.a. Moreover, libCffi is provided on all platforms, not just macOS, so this trick is reasonably portable. ----- Having explained all this, I am forced to wonder if this is intended behavior of GHC, or if it is a series of coincidences. The fact that macOS and Linux pass -Wl,-dead_strip_dylibs and -Wl,--no-as-needed, two flags with completely opposite behavior, is certainly a little unusual. But these choices appear to be deliberate, as they are explained in Notes in the GHC source code here [2] (for macOS) and here [3] (for Linux). The other possibly unusual thing is that GHC always passes -lCffi before any of the `extra-libraries`. That could very well be just a coincidence—I'm not sure if there's a particular reason for -lCffi to come first. The reason that I'm looking so deeply into this matter is that I'm trying to see what it would take to reliably link against libffi statically in the Haskell bindings [4]. I've discovered a trick above to make it work for most versions of GHC: just rely on `-lCffi`. Does this trick seem like it would continue to work for the foreseeable future, or is this approach liable to break in a future release? Ryan ----- [1] See https://gitlab.haskell.org/ghc/ghc/-/issues/20395 for more on this unusual amount of library duplication. [2] https://gitlab.haskell.org/ghc/ghc/-/blob/d71a20514546e0befe6e238d0658cbaad5a13996/compiler/GHC/Driver/Pipeline.hs#L293-340 [3] https://gitlab.haskell.org/ghc/ghc/-/blob/d71a20514546e0befe6e238d0658cbaad5a13996/compiler/GHC/SysTools/Info.hs#L57-69 [4] https://github.com/remiturk/libffi/issues/6 On Tue, Aug 9, 2022 at 9:58 AM Ryan Scott wrote: > No, I'm using macOS 10.15.7 (BuildVersion 19H2) on an x86_64 machine. > > Ryan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From allbery.b at gmail.com Wed Aug 10 02:09:53 2022 From: allbery.b at gmail.com (Brandon Allbery) Date: Tue, 9 Aug 2022 22:09:53 -0400 Subject: A macOS static linking mystery In-Reply-To: References: Message-ID: One peculiarity with the ordering is that linkers only search static archives for existing undefined references. If the reference to `Cffi` actually comes first, then nothing should be required from it yet and *it shouldn't be linked* (absent options like GNU ld's `--whole-archive`). This said, ghc also passes a bunch of explicitly undefined symbols (`-Wl,-u…`) to ld; if those include symbols from `-lCffi` then it will in fact be linked in. On Tue, Aug 9, 2022 at 9:59 PM Ryan Scott wrote: > > I believe I've figured out what is happening here. This will take a bit of explanation, so bear with me. > > First, it's important to note that this behavior depends on whether your GHC was configured with the `--with-system-libffi` flag or not. If it was configured with this flag, then GHC will always link against your system's version of libffi, and the behavior seen in this thread would not occur. I am using a bindist provided by ghcup which was _not_ configured with `--with-system-libffi`, so GHC bundles its own copy of libffi. Interestingly, there appear to be at least two copies of libffi that are bundled: one dynamic library, simply named libffi.dylib, and one static library, which is named libCffi.a [1]. Remember the name libCffi.a, since it will come up again later. > > Next, let's take a look at how GHC performs the final linking step when compiling an executable. GHC will call the C compiler to perform this final linking step, which will have roughly the following shape on macOS: > > gcc ... -L/rts -lCffi -Wl,-dead_strip_dylibs -o Main > > I'm omitting quite a few details here, but the important bits are that (1) -lCffi comes before any of the C libraries listed in an `extra-libraries` stanza in a .cabal file, and (2) GHC on macOS will pass -Wl,-dead_strip_dylibs, which causes the linker to remove any references to shared libraries that go unused. > > In the particular case of libffi.cabal, we have `extra-libraries: ffi`, so the C compiler is given `-lCffi -lffi` as arguments, in that order. Recall that -libCffi.a (i.e., -lCffi) is a statically linked copy of libffi, whereas libffi.dylib (i.e., -lffi) is a dynamically linked version. The linker will resolve all of its references to libffi functionality from libCffi.a because it appears first, and since the linker never needs to resolve anything from libffi.dylib, the -Wl,-dead_strip_dylibs will remove the reference to libffi.dylib entirely. The result is that the final executable will statically link against libffi. This is why this phenomenon only occurs with libffi and not with other C libraries, since only libffi is given special treatment within GHC like this. > > Why, then, does this only happen on macOS and not on, say, Linux? On Linux, the final linking step looks slightly different. Instead of passing -Wl,-dead_strip_dylibs (which is a macOS-specific option), it will pass -Wl,--no-as-needed. Note that -Wl,--no-as-needed has the entirely _opposite_ effect of -Wl,-dead_strip_dylibs: even if a shared library never gets referenced, the linker will still declare a dynamic dependency on it. That is exactly what happens here, as the -lCffi argument will still take precedence over the -lffi argument on Linux, but there will be a useless dynamic dependency on a dynamic libffi library due to -Wl,--no-as-needed. > > At this point, I realized that one consequence of all this is that if you want to statically link against against libffi, and you are using a GHC configured without ` --with-system-libffi`, then it suffices to simply compile your program without any `extra-libraries` at all. That is because the final link step will pass `-lCffi` no matter what, so the resulting executable will always statically link against libffi by way of libCffi.a. Moreover, libCffi is provided on all platforms, not just macOS, so this trick is reasonably portable. > > ----- > > Having explained all this, I am forced to wonder if this is intended behavior of GHC, or if it is a series of coincidences. The fact that macOS and Linux pass -Wl,-dead_strip_dylibs and -Wl,--no-as-needed, two flags with completely opposite behavior, is certainly a little unusual. But these choices appear to be deliberate, as they are explained in Notes in the GHC source code here [2] (for macOS) and here [3] (for Linux). The other possibly unusual thing is that GHC always passes -lCffi before any of the `extra-libraries`. That could very well be just a coincidence—I'm not sure if there's a particular reason for -lCffi to come first. > > The reason that I'm looking so deeply into this matter is that I'm trying to see what it would take to reliably link against libffi statically in the Haskell bindings [4]. I've discovered a trick above to make it work for most versions of GHC: just rely on `-lCffi`. Does this trick seem like it would continue to work for the foreseeable future, or is this approach liable to break in a future release? > > Ryan > ----- > [1] See https://gitlab.haskell.org/ghc/ghc/-/issues/20395 for more on this unusual amount of library duplication. > [2] https://gitlab.haskell.org/ghc/ghc/-/blob/d71a20514546e0befe6e238d0658cbaad5a13996/compiler/GHC/Driver/Pipeline.hs#L293-340 > [3] https://gitlab.haskell.org/ghc/ghc/-/blob/d71a20514546e0befe6e238d0658cbaad5a13996/compiler/GHC/SysTools/Info.hs#L57-69 > [4] https://github.com/remiturk/libffi/issues/6 > > On Tue, Aug 9, 2022 at 9:58 AM Ryan Scott wrote: >> >> No, I'm using macOS 10.15.7 (BuildVersion 19H2) on an x86_64 machine. >> >> Ryan -- brandon s allbery kf8nh allbery.b at gmail.com From ryan.gl.scott at gmail.com Wed Aug 10 13:25:03 2022 From: ryan.gl.scott at gmail.com (Ryan Scott) Date: Wed, 10 Aug 2022 09:25:03 -0400 Subject: A macOS static linking mystery In-Reply-To: References: Message-ID: > One peculiarity with the ordering is that linkers only search static > archives for existing undefined references. If the reference to `Cffi` > actually comes first, then nothing should be required from it yet and > *it shouldn't be linked* (absent options like GNU ld's > `--whole-archive`). Hm, this doesn't seem to be the case when I try things locally on macOS. If I try compiling this C libffi example [1] like so: $ gcc "-L$(ghc --print-libdir)/rts" -lCffi -lffi -Wl,-dead_strip_dylibs test.c -o test Then `otool -L test` does not indicate a dynamic dependency on libffi.dylib: $ otool -L test test: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1) On the other hand, if I swap the order of -lCffi and -lffi: $ gcc "-L$(ghc --print-libdir)/rts" -lffi -lCffi -Wl,-dead_strip_dylibs test.c -o test Then `otool -L test` _does_ indicate a dynamic dependency on libffi.dylib: $ otool -L test test: @rpath/libffi.dylib (compatibility version 9.0.0, current version 9.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1) So on macOS at least, the order does seem to matter. Ryan ----- [1] https://www.chiark.greenend.org.uk/doc/libffi-dev/html/Simple-Example.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From ietf-dane at dukhovni.org Thu Aug 11 02:06:43 2022 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Wed, 10 Aug 2022 22:06:43 -0400 Subject: GHC 9.4 symlink wrappers installed incorrectly on FreeBSD 12 Message-ID: I just built a GHC 9.4 bindist on FreeBSD 12, and found that the bindist Makefile installed unusable wrappers when the source was a symlink. It seems plausible that the issue may not be FreeBSD-specific, but that's all I've tried so far. The problem wrappers do not have execute permissions, and lack the variable settings at the top of the scripts needed to make them work: $ ls -l ~/.local/ghc-9.4/bin total 98 -rw-r--r-- 1 viktor viktor 45 Aug 10 21:46 ghc -rwxr-xr-x 1 viktor viktor 406 Aug 10 21:46 ghc-9.4.1 -rw-r--r-- 1 viktor viktor 97 Aug 10 21:46 ghc-pkg -rwxr-xr-x 1 viktor viktor 466 Aug 10 21:46 ghc-pkg-9.4.1 -rw-r--r-- 1 viktor viktor 67 Aug 10 21:46 ghci -rwxr-xr-x 1 viktor viktor 430 Aug 10 21:46 ghci-9.4.1 -rw-r--r-- 1 viktor viktor 57 Aug 10 21:46 haddock -rwxr-xr-x 1 viktor viktor 434 Aug 10 21:46 haddock-ghc-9.4.1 -rw-r--r-- 1 viktor viktor 33 Aug 10 21:46 hp2ps -rwxr-xr-x 1 viktor viktor 406 Aug 10 21:46 hp2ps-ghc-9.4.1 -rw-r--r-- 1 viktor viktor 33 Aug 10 21:46 hpc -rwxr-xr-x 1 viktor viktor 402 Aug 10 21:46 hpc-ghc-9.4.1 -rw-r--r-- 1 viktor viktor 734 Aug 10 21:46 hsc2hs -rwxr-xr-x 1 viktor viktor 1109 Aug 10 21:46 hsc2hs-ghc-9.4.1 -rw-r--r-- 1 viktor viktor 50 Aug 10 21:46 runghc -rwxr-xr-x 1 viktor viktor 417 Aug 10 21:46 runghc-9.4.1 -rw-r--r-- 1 viktor viktor 50 Aug 10 21:46 runhaskell -rwxr-xr-x 1 viktor viktor 425 Aug 10 21:46 runhaskell-9.4.1 $ (cd ~/.local/ghc-9.4; grep . $(find bin ! -name hsc2hs ! -perm -001)) bin/ghci:executable="$bindir/ghc-9.4.1" bin/ghci:exec $executable --interactive "$@" bin/ghc:exec "$executablename" -B"$libdir" ${1+"$@"} bin/haddock:exec "$executablename" -B"$libdir" -l"$libdir" ${1+"$@"} bin/hp2ps:exec "$executablename" ${1+"$@"} bin/ghc-pkg:PKGCONF="$libdir/package.conf.d" bin/ghc-pkg:exec "$executablename" --global-package-db "$PKGCONF" ${1+"$@"} bin/runghc:exec "$executablename" -f "$exedir/ghc" ${1+"$@"} bin/hpc:exec "$executablename" ${1+"$@"} bin/runhaskell:exec "$executablename" -f "$exedir/ghc" ${1+"$@"} The same holds for "hsc2hs", but the "meat" of the script is longer so I chose to skip it. -- Viktor. From ietf-dane at dukhovni.org Thu Aug 11 03:34:13 2022 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Wed, 10 Aug 2022 23:34:13 -0400 Subject: [Solved via #21974] GHC 9.4 symlink wrappers installed incorrectly on FreeBSD 12 In-Reply-To: References: Message-ID: On Wed, Aug 10, 2022 at 10:06:43PM -0400, Viktor Dukhovni wrote: > I just built a GHC 9.4 bindist on FreeBSD 12, and found that the bindist > Makefile installed unusable wrappers when the source was a symlink. It > seems plausible that the issue may not be FreeBSD-specific, but that's > all I've tried so far. It seems I neglected to read the post-release announcement: Due to an unfortunate packaging issue, the macOS binary distributions for 9.4.1 are not usable as uploaded. The problem is described in #21974, which also includes a small patch to mitigate the breakage. We will be releasing a 9.4.2 within the week fixing the issue. Indeed #21974 fixes the issue. -- Viktor. From facundo.dominguez at tweag.io Thu Aug 11 11:11:23 2022 From: facundo.dominguez at tweag.io (=?UTF-8?Q?Dom=C3=ADnguez=2C_Facundo?=) Date: Thu, 11 Aug 2022 08:11:23 -0300 Subject: Recompilation avoidance on ABI hash changes Message-ID: Dear devs, rules_haskell [1] recently earned the ability to skip recompiling modules when the ABI hashes of dependencies don't change in interface files. We have noticed since then that sometimes ghc --make does avoid rebuilding even when ABI hashes change. However, so far I've been unable to reproduce this in the small. The wiki page on recompilation avoidance does remark that a changing ABI hash is a necessary but not sufficient condition for recompilation [2]. Does anyone have a small example showing how ghc avoids rebuilding even when the ABI hash of a dependency changes? Thanks in advance! Facundo [1] https://github.com/tweag/rules_haskell/issues/1760#issuecomment-1179102232 [2] https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/recompilation-avoidance#interface-file-invariants -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewtpickering at gmail.com Thu Aug 11 12:20:21 2022 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Thu, 11 Aug 2022 13:20:21 +0100 Subject: Recompilation avoidance on ABI hash changes In-Reply-To: References: Message-ID: Facundo, If the interface in question is from a home package module, then it's required that the ABI of the specifically used function is modified rather than the ABI hash of the whole module. If the interface is from an external package then if the ABI of the entire module changes then that is sufficient to trigger recompilation. Matt On Thu, Aug 11, 2022 at 12:11 PM Domínguez, Facundo wrote: > > Dear devs, > > rules_haskell [1] recently earned the ability to skip recompiling modules when the ABI hashes of dependencies don't change in interface files. > > We have noticed since then that sometimes ghc --make does avoid rebuilding even when ABI hashes change. However, so far I've been unable to reproduce this in the small. The wiki page on recompilation avoidance does remark that a changing ABI hash is a necessary but not sufficient condition for recompilation [2]. > > Does anyone have a small example showing how ghc avoids rebuilding even when the ABI hash of a dependency changes? > > Thanks in advance! > Facundo > > [1] https://github.com/tweag/rules_haskell/issues/1760#issuecomment-1179102232 > [2] https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/recompilation-avoidance#interface-file-invariants > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From facundo.dominguez at tweag.io Thu Aug 11 12:50:30 2022 From: facundo.dominguez at tweag.io (=?UTF-8?Q?Dom=C3=ADnguez=2C_Facundo?=) Date: Thu, 11 Aug 2022 09:50:30 -0300 Subject: Recompilation avoidance on ABI hash changes In-Reply-To: References: Message-ID: Thanks Matthew. I've now found my example, which I share bellow for the record. Cheers! Facundo > module A where > > a :: Bool > a = False > > a2 :: Bool > a2 = True > module B where > > import A > > b :: Bool > b = not a Building I get > $ ghc A.hs B.hs > [1 of 2] Compiling A ( A.hs, A.o ) > [2 of 2] Compiling B ( B.hs, B.o ) [A changed] > > $ ghc --show-iface A.hi | grep "ABI hash" > ABI hash: 811ba0740c0157da5cd2ea747468b19f Changing A.a2 to have a different type causes the ABI hash to change yet GHC doesn't recompile module B: > - a2 :: Bool > - a2 = True > + a2 :: Int > + a2 = 0 > $ ghc A.hs B.hs > [1 of 2] Compiling A ( A.hs, A.o ) > > $ ghc --show-iface A.hi | grep "ABI hash" > ABI hash: 6d7a56ce0b9f5fbca71f48bf08c2db84 I was previously trying to change the value of a2 rather than the type, which doesn't change the ABI hash if one isn't compiling with optimizations :) I also tried doing modifications to the export list of A, but this seems to trigger recompilation of module B in all cases. On Thu, Aug 11, 2022 at 9:20 AM Matthew Pickering < matthewtpickering at gmail.com> wrote: > Facundo, > > If the interface in question is from a home package module, then it's > required that the ABI of the specifically used function is modified > rather than the ABI hash of the whole module. If the interface is from > an external package then if the ABI of the entire module changes then > that is sufficient to trigger recompilation. > > Matt > > On Thu, Aug 11, 2022 at 12:11 PM Domínguez, Facundo > wrote: > > > > Dear devs, > > > > rules_haskell [1] recently earned the ability to skip recompiling > modules when the ABI hashes of dependencies don't change in interface files. > > > > We have noticed since then that sometimes ghc --make does avoid > rebuilding even when ABI hashes change. However, so far I've been unable to > reproduce this in the small. The wiki page on recompilation avoidance does > remark that a changing ABI hash is a necessary but not sufficient condition > for recompilation [2]. > > > > Does anyone have a small example showing how ghc avoids rebuilding even > when the ABI hash of a dependency changes? > > > > Thanks in advance! > > Facundo > > > > [1] > https://github.com/tweag/rules_haskell/issues/1760#issuecomment-1179102232 > > [2] > https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/compiler/recompilation-avoidance#interface-file-invariants > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at richarde.dev Thu Aug 11 17:37:50 2022 From: lists at richarde.dev (Richard Eisenberg) Date: Thu, 11 Aug 2022 17:37:50 +0000 Subject: Partial type synonyms -- first-class! In-Reply-To: References: Message-ID: <010f01828dfbdf4c-6c0e4b37-8768-4c05-b6a0-6ec7c7930109-000000@us-east-2.amazonses.com> > On Aug 5, 2022, at 6:17 AM, ÉRDI Gergő wrote: > > 1. Defining partial type synonyms > This makes some sense to me. You're introducing a new form of invisible (implicit) parameter, adding to the two we already have. Today, we have A. invisible parameters that are filled in via unification. We write these in types as `forall a.` B. invisible parameters that are filled in via looking for a unique inhabitant of a type. We write these in types as `C a =>` You want a third: C. invisible parameters that are filled in with a fresh wildcard. We would need to have some way of writing out the type of such a thing (i.e. what kind would `Syn` have?), but I presume this is possible. > 2. Using partial type synonyms > > This bit also makes sense, but I think users will want more functionality. Specifically, what if a user does not want a wildcard inserted, because they know that the right choice is `Int`? Or maybe they want a *named* wildcard inserted. My experience is that once something can be done implicitly, folks will soon find good reasons to do it explicitly on occasion. > 3. Implementation > > > * When typechecking a type application, implicit arguments get > filled with the result of `tcAnonWildCardOcc`. What about named wildcards? Even if they're not passed in, perhaps someone wants type SomeEndo = _t -> _t where the two types are known to be the same, but we don't know what. > > 1. How do I find the wildcard-originating tyvars robustly? Currently, > I am using `candidateQTyVarsWithBinders` but that picks up "regular" > out-of-scope tyvars as well. I believe this causes a regression of > #17567; furthermore, it means if the user writes `type Syn a = Foo b` it > is handled the same as if they wrote `type Syn a = Foo _`, but it would > be better to reject it instead. Yes, sadly I think that you may have to add a new bit of information to e.g. TauTv to do this. Sad, but I can't think of another way. > > 2. What should the `Role` of these implicit type parameters be? For > now, I've went with `Nominal` for lack of a better answer. Answered by you. > > 3. When typechecking a type application, I need to insert wildcards > for these implicit arguments. I am currently using `AnonTCB InvisArg` > binders to represent these implicit type parameters, which means by > the time I get to `tcInferTyApps`, they would get passed through > `instantiate` to `tcInstInvisibleTyBinder`. I currently have a > horrible kuldge here to check, before calling > `tcInstInivisibleTyBinder`, that the invisible binder's kind doesn't > meet the preconditions of it; and if it doesn't, I create a wildcard > instead. But of course, it would be nicer if I could put something in > the `TcTyConBinder` that identifies these binders properly. Yes, I think AnonTCB InvisArg will get you into trouble, because that's what's used for class constraints. As I observed earlier, you're creating a new form of quantification, and so InvisArg doesn't quite say what you want. Make a new constructor. > On Aug 8, 2022, at 10:08 PM, Gergő Érdi wrote: > > 2. It turns out I misunderstood the `Role` situation and they are > actually computed via some weird knot tying from the `TyCon` itself. > So on one hand, I don't really need to do anything to "come up with" > roles for these implicit type parameters. On the other hand, I think > this also means that this breaks the surface syntax for role > declarations, since the user would be expected to give roles for type > parameters they never explicitly added to their tysyns... Type synonyms do not support role annotations, so we can dodge this bullet. Just let the weird knot tying thing do its work and stay out of the way. :) > > 3. Similar to #1, I started just pushing all the way through GHC a > change to `AnonArgFlag` that adds a third `ImplArg` flag. It is > handled mostly the same way as an `InvisArg`, except `tcInferTyApps` > can use it to insert wildcard arguments. I don't have a full commit > for this yet. I don't love adding a new constructor to AnonArgFlag, because that's used in terms. Instead, it would be great to localize this new extension to tycon binders somehow. > > Beside comments on this approach, I would also like to hear from Simon > & Richard if they agree that with this new approach, this could now be > a real language feature that could get upstreamed once all these > implementation wrinkles are smoothed out. I think the route you're taking is a reasonable route to your destination, but I'm not yet convinced it's a destination I want GHC to travel to. As I hint above, I think the feature would have to be expanded somewhat to cover its likely use cases, and yet I worry that it will not be widely enough used to make its specification and implementation worthwhile. I'm happy to be convinced otherwise, though. Given your design, however, I think the implementation you describe is reasonable. I have not read the code you linked to. I hope this helps! Richard > Thanks, > Gergo > > On Fri, Aug 5, 2022 at 6:17 PM ÉRDI Gergő wrote: >> >> As I mentioned at the end of the original thread (but probably no one >> was interested in this enough to read that far), after a breakthrough >> idea I have now started working on partial type synonyms that are NOT >> just defined using macro expansion semantics, and indeed can be a >> first-class abstraction. I think if I can iron out the wrinkles >> detailed at the bottom of this message, this would be good for a GHC >> proposal. >> >> 1. Defining partial type synonyms >> >> The idea is to typecheck the type synonym declaration >> >> type Syn a = MyType _ [a] Int >> >> into a type synonym that has some implicit (invisible) type parameters: >> >> type Syn {w1} a = MyType w1 [a] Int >> >> We don't need to come up with any new design for "what this means": >> this means exactly the same as >> >> type Syn w1 a = MyType w1 [a] Int >> >> which is something you can already write today. The only difference is >> how `w1` is given at use sites. >> >> 2. Using partial type synonyms >> >> When a partial type synonym is applied, its argument list is expanded >> by adding wildcard arguments for the invisible parameters. So this use >> site: >> >> Syn Double >> >> is typececked into the type application >> >> Syn {_} Double >> >> Again, we don't need to come up with any new design for "what this >> means": this is the same as writing `Syn _ Double` into the same >> position, it's just the `_` is not written in the surface syntax. >> >> In particular: >> >> * The level of the type variable conjured for the `_` is determined >> fully by where the `Syn Double` occurs, and not at all by the >> definition of `Syn` >> >> * Using the type synonym `Syn` is only valid where a wildcard >> occurrence would be valid. In particular, using `Syn` in a type >> signature causes the signature itself to be regarded as partial >> (`isCompleteHsSig` returns `False` for it). >> >> 3. Implementation >> >> I have a proof-of-concept implementation at >> https://gitlab.haskell.org/cactus/ghc/-/compare/master...cactus%2Fpartial-tysyns?from_project_id=1117 >> >> I tried to make it as small a change as I could. Its key points are: >> >> * In the renamer, we allow wildcards in `TySynCtx` context >> >> * In `tcTySynRhs`, we allow wildcards in the right-hand side, >> retrieve them from the result of typechecking, and intersperse >> them with the "normal" type parameters to get a well-scoped >> telescope. >> >> * About a third of the lines changed is just a boring refactoring of >> `isCompleteHsSig` to be able to look up type synonym >> `TyCon`s. This is needed because a type signature referring to >> partial type synonyms will be partial itself once those type >> synonym applications are elaborated. >> >> * When typechecking a type application, implicit arguments get >> filled with the result of `tcAnonWildCardOcc`. >> >> Now, my questions about the problems I've encountered along the way. I >> hope these questions are concrete enough that I can get concrete >> answers to them from this list: >> >> 1. How do I find the wildcard-originating tyvars robustly? Currently, >> I am using `candidateQTyVarsWithBinders` but that picks up "regular" >> out-of-scope tyvars as well. I believe this causes a regression of >> #17567; furthermore, it means if the user writes `type Syn a = Foo b` it >> is handled the same as if they wrote `type Syn a = Foo _`, but it would >> be better to reject it instead. >> >> 2. What should the `Role` of these implicit type parameters be? For >> now, I've went with `Nominal` for lack of a better answer. >> >> 3. When typechecking a type application, I need to insert wildcards >> for these implicit arguments. I am currently using `AnonTCB InvisArg` >> binders to represent these implicit type parameters, which means by >> the time I get to `tcInferTyApps`, they would get passed through >> `instantiate` to `tcInstInvisibleTyBinder`. I currently have a >> horrible kuldge here to check, before calling >> `tcInstInivisibleTyBinder`, that the invisible binder's kind doesn't >> meet the preconditions of it; and if it doesn't, I create a wildcard >> instead. But of course, it would be nicer if I could put something in >> the `TcTyConBinder` that identifies these binders properly. >> >> Thanks in advance, >> Gergo >> >> -- >> >> .--= ULLA! =-----------------. >> \ http://gergo.erdi.hu \ >> `---= gergo at erdi.hu =-------' > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From gergo at erdi.hu Fri Aug 12 02:33:25 2022 From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=) Date: Fri, 12 Aug 2022 10:33:25 +0800 (+08) Subject: Partial type synonyms -- first-class! In-Reply-To: <010f01828dfbdf4c-6c0e4b37-8768-4c05-b6a0-6ec7c7930109-000000@us-east-2.amazonses.com> References: <010f01828dfbdf4c-6c0e4b37-8768-4c05-b6a0-6ec7c7930109-000000@us-east-2.amazonses.com> Message-ID: Hi Richard, Thanks for getting back to me! My replies are inline below. On Thu, 11 Aug 2022, Richard Eisenberg wrote: > You want a third: > > C. invisible parameters that are filled in with a fresh wildcard. > > We would need to have some way of writing out the type of such a thing > (i.e. what kind would `Syn` have?), but I presume this is possible. I think there's a tension between this and your suggestion to only add implicit parameters as a new `TyConBndrVis`, but more on that below. >> 2. Using partial type synonyms >> >> > > This bit also makes sense, but I think users will want more > functionality. Specifically, what if a user does not want a wildcard > inserted, because they know that the right choice is `Int`? Or maybe > they want a *named* wildcard inserted. My experience is that once > something can be done implicitly, folks will soon find good reasons to > do it explicitly on occasion. Good point, but I think we can punt on this and not close any doors ahead. So today, you would only be able to write `Syn T` to mean `Syn {_} T`, and then in the future we can add typechecker support (and new surface syntax!) for `Syn {S} T`, without causing any compatibility problems with any existing code that doesn't give explicit args for implicit params. >> 3. Implementation >> >> >> * When typechecking a type application, implicit arguments get >> filled with the result of `tcAnonWildCardOcc`. > > What about named wildcards? Even if they're not passed in, perhaps someone wants > > type SomeEndo = _t -> _t > > where the two types are known to be the same, but we don't know what. This would be something to support when typechecking the definition, not a given application. Your example would still elaborate to type SomeEndo {t} = t -> t it would just use the same implicitly-bound type parameter `t` twice on the right-hand side. But when you use `SomeEndo`, the usage would still elaborate into a (single) anonymous wildcard argument, i.e. `SomeEndo {_}`. My current implementation doesn't support your example, but I think it's only because the renamer rejects it. I think if I get it through the renamer, it should already work because that `_t` will typecheck into a wildcard `TauTv`. >> 3. Similar to #1, I started just pushing all the way through GHC a >> change to `AnonArgFlag` that adds a third `ImplArg` flag. > > I don't love adding a new constructor to AnonArgFlag, because that's > used in terms. Instead, it would be great to localize this new extension > to tycon binders somehow. OK so while I'd love to get away with only changing `TyConBndrVis`, this is the part of your email that I don't understand how to do :/ First, when a type application is typechecked, we only have the kind of the type constructor, not its binders (and that makes sense, since we could be applying something more complex than directly a defined type constructor). So if I only add a new `TyConBndrVis` constructor, then I have no way of representing this in the `tyConKind` and so no way of finding out that I need to put in implicit args in `tcInferTyApps`. Second, you ask what the kind of `Syn` in e.g. `type Syn a = TC _ a` is. I think (supposing `TC :: K -> L -> M`) its kind should be (stealing syntax from Agda) something like `{K} -> L -> M`, i.e. a function kind with domain `K`, codomain `L -> M`, and a new kind of visibility on the arrow itself. But that means it's not just the binder of the implicit parameter that has a new visibility, but the arrow as well. And isn't that what `AnonArgFlag` is for? > I think the route you're taking is a reasonable route to your > destination, but I'm not yet convinced it's a destination I want GHC to > travel to. As I hint above, I think the feature would have to be > expanded somewhat to cover its likely use cases, and yet I worry that it > will not be widely enough used to make its specification and > implementation worthwhile. I'm happy to be convinced otherwise, though. Fair enough. Although I was hoping that with Dependent Haskell, we would have more situations where unification can give useful solutions, and so we would want the feature of implicit arguments (even for terms!). But if there's no appetite from GHC for partial type synonyms, what would help me a lot in keeping this maintainable / avoiding churn in chasing `master` would be if I can upstream two refactorings that are enablers for my implementation but don't actually change any existing behaviour: * Adding "does it come from a wildcard" flag to `TauTv` (https://gitlab.haskell.org/cactus/ghc/-/commit/e8be2cb2b1093275385467741b1297ce91ef7547) * Changing `isCompleteHsSig` to run in `TcM`, so that its result can depend on type constructor details (https://gitlab.haskell.org/cactus/ghc/-/commit/49f60ef13ad7f63f91519ca88cd8095db0781c92) * Maybe a third one, depending on what we come up with for the representation of these implicit binders What do you think about this? Thanks, Gergo From ekmett at gmail.com Fri Aug 12 02:42:56 2022 From: ekmett at gmail.com (Edward Kmett) Date: Thu, 11 Aug 2022 22:42:56 -0400 Subject: Partial type synonyms -- first-class! In-Reply-To: References: <010f01828dfbdf4c-6c0e4b37-8768-4c05-b6a0-6ec7c7930109-000000@us-east-2.amazonses.com> Message-ID: FWIW, Gergo, I've been following what you've been doing pretty closely, so there's at least two of us tracking it in the background. =) I might have some clever(?) (ab)uses for it in the future in my linear haskell code. -Edward On Thu, Aug 11, 2022 at 10:33 PM ÉRDI Gergő wrote: > Hi Richard, > > Thanks for getting back to me! My replies are inline below. > > On Thu, 11 Aug 2022, Richard Eisenberg wrote: > > > You want a third: > > > > C. invisible parameters that are filled in with a fresh wildcard. > > > > We would need to have some way of writing out the type of such a thing > > (i.e. what kind would `Syn` have?), but I presume this is possible. > > I think there's a tension between this and your suggestion to only add > implicit parameters as a new `TyConBndrVis`, but more on that below. > > >> 2. Using partial type synonyms > >> > >> > > > > This bit also makes sense, but I think users will want more > > functionality. Specifically, what if a user does not want a wildcard > > inserted, because they know that the right choice is `Int`? Or maybe > > they want a *named* wildcard inserted. My experience is that once > > something can be done implicitly, folks will soon find good reasons to > > do it explicitly on occasion. > > Good point, but I think we can punt on this and not close any doors ahead. > So today, you would only be able to write `Syn T` to mean `Syn {_} T`, and > then in the future we can add typechecker support (and new surface > syntax!) for `Syn {S} T`, without causing any compatibility problems with > any existing code that doesn't give explicit args for implicit params. > > >> 3. Implementation > >> > >> > >> * When typechecking a type application, implicit arguments get > >> filled with the result of `tcAnonWildCardOcc`. > > > > What about named wildcards? Even if they're not passed in, perhaps > someone wants > > > > type SomeEndo = _t -> _t > > > > where the two types are known to be the same, but we don't know what. > > This would be something to support when typechecking the definition, not a > given application. Your example would still elaborate to > > type SomeEndo {t} = t -> t > > it would just use the same implicitly-bound type parameter `t` twice on > the right-hand side. But when you use `SomeEndo`, the usage would still > elaborate into a (single) anonymous wildcard argument, i.e. > `SomeEndo {_}`. > > My current implementation doesn't support your example, but I think it's > only because the renamer rejects it. I think if I get it through the > renamer, it should already work because that `_t` will typecheck into a > wildcard `TauTv`. > > >> 3. Similar to #1, I started just pushing all the way through GHC a > >> change to `AnonArgFlag` that adds a third `ImplArg` flag. > > > > I don't love adding a new constructor to AnonArgFlag, because that's > > used in terms. Instead, it would be great to localize this new extension > > to tycon binders somehow. > > OK so while I'd love to get away with only changing `TyConBndrVis`, this > is the part of your email that I don't understand how to do :/ > > First, when a type application is typechecked, we only have the kind of > the type constructor, not its binders (and that makes sense, since we > could be applying something more complex than directly a defined type > constructor). So if I only add a new `TyConBndrVis` constructor, then I > have no way of representing this in the `tyConKind` and so no way of > finding out that I need to put in implicit args in `tcInferTyApps`. > > Second, you ask what the kind of `Syn` in e.g. `type Syn a = TC _ a` is. I > think (supposing `TC :: K -> L -> M`) its kind should be (stealing syntax > from Agda) something like `{K} -> L -> M`, i.e. a function kind with > domain `K`, codomain `L -> M`, and a new kind of visibility on the arrow > itself. But that means it's not just the binder of the implicit parameter > that has a new visibility, but the arrow as well. And isn't that what > `AnonArgFlag` is for? > > > I think the route you're taking is a reasonable route to your > > destination, but I'm not yet convinced it's a destination I want GHC to > > travel to. As I hint above, I think the feature would have to be > > expanded somewhat to cover its likely use cases, and yet I worry that it > > will not be widely enough used to make its specification and > > implementation worthwhile. I'm happy to be convinced otherwise, though. > > Fair enough. Although I was hoping that with Dependent Haskell, we would > have more situations where unification can give useful solutions, and so > we would want the feature of implicit arguments (even for terms!). > > But if there's no appetite from GHC for partial type synonyms, what would > help me a lot in keeping this maintainable / avoiding churn in chasing > `master` would be if I can upstream two refactorings that are enablers > for my implementation but don't actually change any existing behaviour: > > * Adding "does it come from a wildcard" flag to `TauTv` > ( > https://gitlab.haskell.org/cactus/ghc/-/commit/e8be2cb2b1093275385467741b1297ce91ef7547 > ) > > * Changing `isCompleteHsSig` to run in `TcM`, so that its result can > depend on type constructor details > ( > https://gitlab.haskell.org/cactus/ghc/-/commit/49f60ef13ad7f63f91519ca88cd8095db0781c92 > ) > > * Maybe a third one, depending on what we come up with for the > representation of these implicit binders > > What do you think about this? > > Thanks, > Gergo > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gergo at erdi.hu Fri Aug 12 03:05:05 2022 From: gergo at erdi.hu (=?UTF-8?B?R2VyZ8WRIMOJcmRp?=) Date: Fri, 12 Aug 2022 11:05:05 +0800 Subject: Partial type synonyms -- first-class! In-Reply-To: References: <010f01828dfbdf4c-6c0e4b37-8768-4c05-b6a0-6ec7c7930109-000000@us-east-2.amazonses.com> Message-ID: That's great to hear because it looks like this will need all the support it can get to ever be allowed into upstream... On Fri, Aug 12, 2022, 10:43 Edward Kmett wrote: > FWIW, Gergo, I've been following what you've been doing pretty closely, so > there's at least two of us tracking it in the background. =) I might have > some clever(?) (ab)uses for it in the future in my linear haskell code. > > -Edward > > On Thu, Aug 11, 2022 at 10:33 PM ÉRDI Gergő wrote: > >> Hi Richard, >> >> Thanks for getting back to me! My replies are inline below. >> >> On Thu, 11 Aug 2022, Richard Eisenberg wrote: >> >> > You want a third: >> > >> > C. invisible parameters that are filled in with a fresh wildcard. >> > >> > We would need to have some way of writing out the type of such a thing >> > (i.e. what kind would `Syn` have?), but I presume this is possible. >> >> I think there's a tension between this and your suggestion to only add >> implicit parameters as a new `TyConBndrVis`, but more on that below. >> >> >> 2. Using partial type synonyms >> >> >> >> >> > >> > This bit also makes sense, but I think users will want more >> > functionality. Specifically, what if a user does not want a wildcard >> > inserted, because they know that the right choice is `Int`? Or maybe >> > they want a *named* wildcard inserted. My experience is that once >> > something can be done implicitly, folks will soon find good reasons to >> > do it explicitly on occasion. >> >> Good point, but I think we can punt on this and not close any doors >> ahead. >> So today, you would only be able to write `Syn T` to mean `Syn {_} T`, >> and >> then in the future we can add typechecker support (and new surface >> syntax!) for `Syn {S} T`, without causing any compatibility problems with >> any existing code that doesn't give explicit args for implicit params. >> >> >> 3. Implementation >> >> >> >> >> >> * When typechecking a type application, implicit arguments get >> >> filled with the result of `tcAnonWildCardOcc`. >> > >> > What about named wildcards? Even if they're not passed in, perhaps >> someone wants >> > >> > type SomeEndo = _t -> _t >> > >> > where the two types are known to be the same, but we don't know what. >> >> This would be something to support when typechecking the definition, not >> a >> given application. Your example would still elaborate to >> >> type SomeEndo {t} = t -> t >> >> it would just use the same implicitly-bound type parameter `t` twice on >> the right-hand side. But when you use `SomeEndo`, the usage would still >> elaborate into a (single) anonymous wildcard argument, i.e. >> `SomeEndo {_}`. >> >> My current implementation doesn't support your example, but I think it's >> only because the renamer rejects it. I think if I get it through the >> renamer, it should already work because that `_t` will typecheck into a >> wildcard `TauTv`. >> >> >> 3. Similar to #1, I started just pushing all the way through GHC a >> >> change to `AnonArgFlag` that adds a third `ImplArg` flag. >> > >> > I don't love adding a new constructor to AnonArgFlag, because that's >> > used in terms. Instead, it would be great to localize this new >> extension >> > to tycon binders somehow. >> >> OK so while I'd love to get away with only changing `TyConBndrVis`, this >> is the part of your email that I don't understand how to do :/ >> >> First, when a type application is typechecked, we only have the kind of >> the type constructor, not its binders (and that makes sense, since we >> could be applying something more complex than directly a defined type >> constructor). So if I only add a new `TyConBndrVis` constructor, then I >> have no way of representing this in the `tyConKind` and so no way of >> finding out that I need to put in implicit args in `tcInferTyApps`. >> >> Second, you ask what the kind of `Syn` in e.g. `type Syn a = TC _ a` is. >> I >> think (supposing `TC :: K -> L -> M`) its kind should be (stealing syntax >> from Agda) something like `{K} -> L -> M`, i.e. a function kind with >> domain `K`, codomain `L -> M`, and a new kind of visibility on the arrow >> itself. But that means it's not just the binder of the implicit parameter >> that has a new visibility, but the arrow as well. And isn't that what >> `AnonArgFlag` is for? >> >> > I think the route you're taking is a reasonable route to your >> > destination, but I'm not yet convinced it's a destination I want GHC to >> > travel to. As I hint above, I think the feature would have to be >> > expanded somewhat to cover its likely use cases, and yet I worry that >> it >> > will not be widely enough used to make its specification and >> > implementation worthwhile. I'm happy to be convinced otherwise, though. >> >> Fair enough. Although I was hoping that with Dependent Haskell, we would >> have more situations where unification can give useful solutions, and so >> we would want the feature of implicit arguments (even for terms!). >> >> But if there's no appetite from GHC for partial type synonyms, what would >> help me a lot in keeping this maintainable / avoiding churn in chasing >> `master` would be if I can upstream two refactorings that are enablers >> for my implementation but don't actually change any existing behaviour: >> >> * Adding "does it come from a wildcard" flag to `TauTv` >> ( >> https://gitlab.haskell.org/cactus/ghc/-/commit/e8be2cb2b1093275385467741b1297ce91ef7547 >> ) >> >> * Changing `isCompleteHsSig` to run in `TcM`, so that its result can >> depend on type constructor details >> ( >> https://gitlab.haskell.org/cactus/ghc/-/commit/49f60ef13ad7f63f91519ca88cd8095db0781c92 >> ) >> >> * Maybe a third one, depending on what we come up with for the >> representation of these implicit binders >> >> What do you think about this? >> >> Thanks, >> Gergo >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Gergo.Erdi at sc.com Fri Aug 12 09:16:17 2022 From: Gergo.Erdi at sc.com (Erdi, Gergo) Date: Fri, 12 Aug 2022 09:16:17 +0000 Subject: Partial type synonyms -- first-class! Message-ID: PUBLIC For anyone interested, I've added MRs https://gitlab.haskell.org/ghc/ghc/-/merge_requests/8818 and https://gitlab.haskell.org/ghc/ghc/-/merge_requests/8819 to track my progress. From: ghc-devs On Behalf Of Edward Kmett Sent: Friday, August 12, 2022 10:43 AM To: ÉRDI Gergő Cc: GHC Devs Subject: [External] Re: Partial type synonyms -- first-class! FWIW, Gergo, I've been following what you've been doing pretty closely, so there's at least two of us tracking it in the background. =) I might have some clever(?) (ab)uses for it in the future in my linear haskell code. -Edward This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin.redelings at gmail.com Mon Aug 15 19:00:56 2022 From: benjamin.redelings at gmail.com (Benjamin Redelings) Date: Mon, 15 Aug 2022 12:00:56 -0700 Subject: Advice for implementing GADTs? In-Reply-To: References: <0251fb0b-b4b2-d91b-ca34-55e3b812e456@gmail.com> Message-ID: <2789e945-61cb-5c85-d2e0-e6cc59b37213@gmail.com> Thanks a bunch for this! On 8/4/22 3:45 PM, Simon Peyton Jones wrote: > > QUESTION 1: Are there any obviously important resources that I've > overlooked? > > That's a good list.  Ningning's thesis https://xnning.github.io/ is > also good stuff. Thanks! > QUESTION 2: if my quick scan is correct, none of the papers mention > the GHC technique of determining untouchability by assigning "levels" > to type variables.  Is there any written paper (outside the GHC > sources) that discusses type levels? > It is disgracefully undocumented, I'm afraid.  Sorry.  Didier Remy > used similar ideas, in some INRIA papers I think. OK, that was my impression, just checking.  I think I get the basic idea... > QUESTION 3: My impression is that: > > (a) type variable levels were introduced in order to clarify which > MetaTyVars are "untouchable", but > > (b) levels now also check that type variables do not escape their > quantification scope. > > (c) levels can also be used to figure out which variables are free in > the type environment, and therefore should not be generalized over. > > Does this sound right?  I suspect that I might be wrong about the last > one... > > > Correct about all three. > Good to know! > Except that a unification variable is only untouchable if it comes > from an outer level *and* there are some intervening Given > equalities.   If there are no equalities it's not untouchable.  E.b. > >    f = \x -> case x of > Just y -> 3::Int > > Here the (3::Int) can affect the result type of the function because > the Just pattern match does not bind any Given equalities (in a GADT > like way). > 3.1: OK, so if the case branch introduces type-class constraints, but not equality constraints, then the unification variables from an outer level are still touchable?  Presumably because the type-class constraints don't interact with the equality constraints? 3.2: The OutsideIn paper talks about creating implication constraints, which then bubble UP to the level where the solver is applied.  Maybe only at the outermost level? However, it sounds like GHC pushes given constraints from a GADT pattern match DOWN into the case branch.  Implication constraints would only be created if the wanted constraints bubble up to a GADT pattern match, and are not entailed by the givens.  So * implication constraints like (a ~ Int ==> a ~ Int) are never created. * however, implication constraints like (a ~ Int ==> b ~ Int) could be created. I may be assuming that GHC runs the solver for each GADT pattern match before creating implication constraints. Does that sound right? 3.3: Also, is there a standard way to pretty-print implication constraints?  The OutsideIn paper uses LaTeX \supset I think, but there isn't an obvious ASCII character to use to for \supset... > I keep meaning to write an updated version of Practical type inference > for arbitrary rank types > , > but failing to get around to it! > That would be great, if you find the time!  Are you thinking of adding practical steps for handling equality constraints to it? Or, removing the deep-subsumption language?  Or something else? It has already been quite helpful. -BenRI -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin.redelings at gmail.com Mon Aug 15 19:01:58 2022 From: benjamin.redelings at gmail.com (Benjamin Redelings) Date: Mon, 15 Aug 2022 12:01:58 -0700 Subject: Advice for implementing GADTs? In-Reply-To: References: <0251fb0b-b4b2-d91b-ca34-55e3b812e456@gmail.com> Message-ID: <14f8e328-4793-4017-adab-4121d6bbc46c@gmail.com> Thanks for the references!  I will take a look. -BenRI On 8/4/22 9:22 PM, David Christiansen wrote: > > QUESTION 2: if my quick scan is correct, none of the papers > mention the GHC technique of determining untouchability by > assigning "levels" to type variables.  Is there any written paper > (outside the GHC sources) that discusses type levels? > > It is disgracefully undocumented, I'm afraid.  Sorry.  Didier Remy > used similar ideas, in some INRIA papers I think. > > > (I'm first answering now because I wasn't sure until this comment that > it was the same concept of levels that I know of) > > A couple of resources do exist for learning this idea. There's a quite > accessible description of the basic idea in Peter Sestoft's book > "Programming Language Concepts" from about ten years ago. A collection > of fancier and more efficient versions are laid out by Oleg Kiselyov > here: https://okmij.org/ftp/ML/generalization.html . A good source > back to Remy is this one: https://hal.inria.fr/inria-00077006/document > > /David -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewtpickering at gmail.com Mon Aug 15 19:17:21 2022 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Mon, 15 Aug 2022 20:17:21 +0100 Subject: 9.4.2 release plans Message-ID: Hi all, Within the next week we are planning to release 9.4.2 to fix the critical issues found in 9.4.1. The 9.4.2 milestone is up-to-date with all the issues which are planned to be fixed in this release. https://gitlab.haskell.org/ghc/ghc/-/milestones/378#tab-issues Amongst these are: * -x option broken on command line (#22044 ) * Darwin bindists can't be installed (#21974) * Bytecode interpreter incorrect result when inlining constructor wrappers (#22042) * Some fixes to the make build system (#22047) Cheers, Matt From Gergo.Erdi at sc.com Tue Aug 16 06:36:46 2022 From: Gergo.Erdi at sc.com (Erdi, Gergo) Date: Tue, 16 Aug 2022 06:36:46 +0000 Subject: Test failures on `master` Message-ID: PUBLIC Hi, As of dca43a04fb, I am seeing the following test failures locally. Is this due to some configuration problem in my development environment, or did something slip through CI? Thanks, Gergo Unexpected failures: /tmp/ghctest-2v5a6979/test spaces/testsuite/tests/driver/j-space/jspace.run jspace [bad exit code (2)] (normal) /tmp/ghctest-2v5a6979/test spaces/testsuite/tests/plugins/plugins-external.run plugins-external [bad exit code (2)] (normal) /tmp/ghctest-2v5a6979/test spaces/testsuite/tests/quasiquotation/qq005/qq005.run qq005 [exit code non-0] (normal) /tmp/ghctest-2v5a6979/test spaces/testsuite/tests/quasiquotation/qq006/qq006.run qq006 [stderr mismatch] (normal) /tmp/ghctest-2v5a6979/test spaces/libraries/unix/tests/T3816.run T3816 [bad exit code (1)] (normal) /tmp/ghctest-2v5a6979/test spaces/testsuite/tests/plugins/test-echo-in-line.run test-echo-in-line [bad stdout] (normal) /tmp/ghctest-2v5a6979/test spaces/testsuite/tests/plugins/test-echo-in-line-many-args.run test-echo-in-line-many-args [bad stdout] (normal) /tmp/ghctest-2v5a6979/test spaces/testsuite/tests/plugins/test-echo-in-turn.run test-echo-in-turn [bad stdout] (normal) /tmp/ghctest-2v5a6979/test spaces/testsuite/tests/plugins/test-echo-in-turn-many-args.run test-echo-in-turn-many-args [bad stdout] (normal) /tmp/ghctest-2v5a6979/test spaces/libraries/unix/tests/user001.run user001 [bad stdout] (normal) This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer. Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same. Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products. -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon.peytonjones at gmail.com Tue Aug 16 08:47:06 2022 From: simon.peytonjones at gmail.com (Simon Peyton Jones) Date: Tue, 16 Aug 2022 09:47:06 +0100 Subject: Advice for implementing GADTs? In-Reply-To: <2789e945-61cb-5c85-d2e0-e6cc59b37213@gmail.com> References: <0251fb0b-b4b2-d91b-ca34-55e3b812e456@gmail.com> <2789e945-61cb-5c85-d2e0-e6cc59b37213@gmail.com> Message-ID: > > 3.1: OK, so if the case branch introduces type-class constraints, but not > equality constraints, then the unification variables from an outer level > are still touchable? Presumably because the type-class constraints don't > interact with the equality constraints? > Yes, but the check is a bit conservative. Look at `inert_given_eqs` in `InertCans`, and `updateGivenEqs`. * implication constraints like (a ~ Int ==> a ~ Int) are never created. > I may be assuming that GHC runs the solver for each GADT pattern match > before creating implication constraints. > That's not right. They can be created, but then get solved, by solving the Wanted from the Given. 3.3: Also, is there a standard way to pretty-print implication > constraints? The OutsideIn paper uses LaTeX \supset I think, but there > isn't an obvious ASCII character to use to for \supset... > Well there is a `Outputable` instance. It uses `=>`. Simon On Mon, 15 Aug 2022 at 20:00, Benjamin Redelings < benjamin.redelings at gmail.com> wrote: > Thanks a bunch for this! > On 8/4/22 3:45 PM, Simon Peyton Jones wrote: > > QUESTION 1: Are there any obviously important resources that I've > overlooked? > That's a good list. Ningning's thesis https://xnning.github.io/ is also > good stuff. > > Thanks! > > > QUESTION 2: if my quick scan is correct, none of the papers mention the > GHC technique of determining untouchability by assigning "levels" to type > variables. Is there any written paper (outside the GHC sources) that > discusses type levels? > It is disgracefully undocumented, I'm afraid. Sorry. Didier Remy used > similar ideas, in some INRIA papers I think. > > OK, that was my impression, just checking. I think I get the basic idea... > > > QUESTION 3: My impression is that: > > (a) type variable levels were introduced in order to clarify which > MetaTyVars are "untouchable", but > > (b) levels now also check that type variables do not escape their > quantification scope. > > (c) levels can also be used to figure out which variables are free in the > type environment, and therefore should not be generalized over. > > Does this sound right? I suspect that I might be wrong about the last > one... > > > Correct about all three. > > Good to know! > > Except that a unification variable is only untouchable if it comes from an > outer level *and* there are some intervening Given equalities. If there > are no equalities it's not untouchable. E.b. > f = \x -> case x of > Just y -> 3::Int > > Here the (3::Int) can affect the result type of the function because the > Just pattern match does not bind any Given equalities (in a GADT like way). > > 3.1: OK, so if the case branch introduces type-class constraints, but not > equality constraints, then the unification variables from an outer level > are still touchable? Presumably because the type-class constraints don't > interact with the equality constraints? > > 3.2: The OutsideIn paper talks about creating implication constraints, > which then bubble UP to the level where the solver is applied. Maybe only > at the outermost level? > > However, it sounds like GHC pushes given constraints from a GADT pattern > match DOWN into the case branch. Implication constraints would only be > created if the wanted constraints bubble up to a GADT pattern match, and > are not entailed by the givens. So > > * implication constraints like (a ~ Int ==> a ~ Int) are never created. > > * however, implication constraints like (a ~ Int ==> b ~ Int) could be > created. > > I may be assuming that GHC runs the solver for each GADT pattern match > before creating implication constraints. > > Does that sound right? > > 3.3: Also, is there a standard way to pretty-print implication > constraints? The OutsideIn paper uses LaTeX \supset I think, but there > isn't an obvious ASCII character to use to for \supset... > > > I keep meaning to write an updated version of Practical type inference > for arbitrary rank types > , > but failing to get around to it! > > That would be great, if you find the time! Are you thinking of adding > practical steps for handling equality constraints to it? Or, removing the > deep-subsumption language? Or something else? It has already been quite > helpful. > > -BenRI > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon.peytonjones at gmail.com Wed Aug 17 21:53:46 2022 From: simon.peytonjones at gmail.com (Simon Peyton Jones) Date: Wed, 17 Aug 2022 22:53:46 +0100 Subject: Partial type synonyms -- first-class! In-Reply-To: References: <010f01828dfbdf4c-6c0e4b37-8768-4c05-b6a0-6ec7c7930109-000000@us-east-2.amazonses.com> Message-ID: Richard writes I think the route you're taking is a reasonable route to your destination, > but I'm not yet convinced it's a destination I want GHC to travel to. > I'm in the same boat -- not yet convinced. To me it seems like quite a bit of implementation complexity to hit a pretty narrow use-case. But maybe you have convincing use-cases to offer, when you write a proposal. (Maybe Edward can help with some.) In any case, it's not me or Richard you need to convince, it's the GHC Steering Committee via the proposals process. You are doing a great job of working out an implementation before submitting a proposal (which few people do), but that doesn't, in and of itself, make the design more attractive. Yet it must be very attractive to you if you are contemplating maintaining a fork in the medium term. Maybe you can convince everyone! Simon On Fri, 12 Aug 2022 at 04:05, Gergő Érdi wrote: > That's great to hear because it looks like this will need all the support > it can get to ever be allowed into upstream... > > On Fri, Aug 12, 2022, 10:43 Edward Kmett wrote: > >> FWIW, Gergo, I've been following what you've been doing pretty closely, >> so there's at least two of us tracking it in the background. =) I might >> have some clever(?) (ab)uses for it in the future in my linear haskell code. >> >> -Edward >> >> On Thu, Aug 11, 2022 at 10:33 PM ÉRDI Gergő wrote: >> >>> Hi Richard, >>> >>> Thanks for getting back to me! My replies are inline below. >>> >>> On Thu, 11 Aug 2022, Richard Eisenberg wrote: >>> >>> > You want a third: >>> > >>> > C. invisible parameters that are filled in with a fresh wildcard. >>> > >>> > We would need to have some way of writing out the type of such a thing >>> > (i.e. what kind would `Syn` have?), but I presume this is possible. >>> >>> I think there's a tension between this and your suggestion to only add >>> implicit parameters as a new `TyConBndrVis`, but more on that below. >>> >>> >> 2. Using partial type synonyms >>> >> >>> >> >>> > >>> > This bit also makes sense, but I think users will want more >>> > functionality. Specifically, what if a user does not want a wildcard >>> > inserted, because they know that the right choice is `Int`? Or maybe >>> > they want a *named* wildcard inserted. My experience is that once >>> > something can be done implicitly, folks will soon find good reasons to >>> > do it explicitly on occasion. >>> >>> Good point, but I think we can punt on this and not close any doors >>> ahead. >>> So today, you would only be able to write `Syn T` to mean `Syn {_} T`, >>> and >>> then in the future we can add typechecker support (and new surface >>> syntax!) for `Syn {S} T`, without causing any compatibility problems >>> with >>> any existing code that doesn't give explicit args for implicit params. >>> >>> >> 3. Implementation >>> >> >>> >> >>> >> * When typechecking a type application, implicit arguments get >>> >> filled with the result of `tcAnonWildCardOcc`. >>> > >>> > What about named wildcards? Even if they're not passed in, perhaps >>> someone wants >>> > >>> > type SomeEndo = _t -> _t >>> > >>> > where the two types are known to be the same, but we don't know what. >>> >>> This would be something to support when typechecking the definition, not >>> a >>> given application. Your example would still elaborate to >>> >>> type SomeEndo {t} = t -> t >>> >>> it would just use the same implicitly-bound type parameter `t` twice on >>> the right-hand side. But when you use `SomeEndo`, the usage would still >>> elaborate into a (single) anonymous wildcard argument, i.e. >>> `SomeEndo {_}`. >>> >>> My current implementation doesn't support your example, but I think it's >>> only because the renamer rejects it. I think if I get it through the >>> renamer, it should already work because that `_t` will typecheck into a >>> wildcard `TauTv`. >>> >>> >> 3. Similar to #1, I started just pushing all the way through GHC a >>> >> change to `AnonArgFlag` that adds a third `ImplArg` flag. >>> > >>> > I don't love adding a new constructor to AnonArgFlag, because that's >>> > used in terms. Instead, it would be great to localize this new >>> extension >>> > to tycon binders somehow. >>> >>> OK so while I'd love to get away with only changing `TyConBndrVis`, this >>> is the part of your email that I don't understand how to do :/ >>> >>> First, when a type application is typechecked, we only have the kind of >>> the type constructor, not its binders (and that makes sense, since we >>> could be applying something more complex than directly a defined type >>> constructor). So if I only add a new `TyConBndrVis` constructor, then I >>> have no way of representing this in the `tyConKind` and so no way of >>> finding out that I need to put in implicit args in `tcInferTyApps`. >>> >>> Second, you ask what the kind of `Syn` in e.g. `type Syn a = TC _ a` is. >>> I >>> think (supposing `TC :: K -> L -> M`) its kind should be (stealing >>> syntax >>> from Agda) something like `{K} -> L -> M`, i.e. a function kind with >>> domain `K`, codomain `L -> M`, and a new kind of visibility on the arrow >>> itself. But that means it's not just the binder of the implicit >>> parameter >>> that has a new visibility, but the arrow as well. And isn't that what >>> `AnonArgFlag` is for? >>> >>> > I think the route you're taking is a reasonable route to your >>> > destination, but I'm not yet convinced it's a destination I want GHC >>> to >>> > travel to. As I hint above, I think the feature would have to be >>> > expanded somewhat to cover its likely use cases, and yet I worry that >>> it >>> > will not be widely enough used to make its specification and >>> > implementation worthwhile. I'm happy to be convinced otherwise, though. >>> >>> Fair enough. Although I was hoping that with Dependent Haskell, we would >>> have more situations where unification can give useful solutions, and so >>> we would want the feature of implicit arguments (even for terms!). >>> >>> But if there's no appetite from GHC for partial type synonyms, what >>> would >>> help me a lot in keeping this maintainable / avoiding churn in chasing >>> `master` would be if I can upstream two refactorings that are enablers >>> for my implementation but don't actually change any existing behaviour: >>> >>> * Adding "does it come from a wildcard" flag to `TauTv` >>> ( >>> https://gitlab.haskell.org/cactus/ghc/-/commit/e8be2cb2b1093275385467741b1297ce91ef7547 >>> ) >>> >>> * Changing `isCompleteHsSig` to run in `TcM`, so that its result can >>> depend on type constructor details >>> ( >>> https://gitlab.haskell.org/cactus/ghc/-/commit/49f60ef13ad7f63f91519ca88cd8095db0781c92 >>> ) >>> >>> * Maybe a third one, depending on what we come up with for the >>> representation of these implicit binders >>> >>> What do you think about this? >>> >>> Thanks, >>> Gergo >>> _______________________________________________ >>> ghc-devs mailing list >>> ghc-devs at haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >>> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Mon Aug 22 18:55:05 2022 From: ben at well-typed.com (Ben Gamari) Date: Mon, 22 Aug 2022 14:55:05 -0400 Subject: [ANNOUNCE] GHC 9.4.2 is now available Message-ID: <87lergun0v.fsf@smart-cactus.org> The GHC developers are happy to announce the availability of GHC 9.4.2. Binary distributions, source distributions, and documentation are available at downloads.haskell.org: https://downloads.haskell.org/ghc/9.4.2 This release is primarily a bugfix release addressing a few packaging issues found in 9.4.1. See the [release notes] for a full accounting. Note that, as GHC 9.4 is the first release series where the release artifacts are all generated by our new Hadrian build system, it is possible that there will be packaging issues. If you enounter trouble while using a binary distribution, please open a [ticket]. Likewise, if you are a downstream packager, do consider migrating to [Hadrian] to run your build; the Hadrian build system can be built using `cabal-install`, `stack`, or the in-tree [bootstrap script]. See the accompanying [blog post] for details on migrating packaging to Hadrian. We would also like to emphasize that GHC 9.4 must be used in conjunction with Cabal-3.8 or later. This is particularly important for Windows users due to changes in GHC's Windows toolchain. We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, Well-Typed, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation, and other anonymous contributors whose on-going financial and in-kind support has facilitated GHC maintenance and release management over the years. Finally, this release would not have been possible without the hundreds of open-source contributors whose work comprise this release. As always, do give this release a try and open a [ticket] if you see anything amiss. Happy Haskelling, - Ben [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new [bootstrap script]: https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d11c07d35/hadrian/bootstrap/README.md [Hadrian]: https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d11c07d35/hadrian [release notes]: https://downloads.haskell.org/~ghc/9.4.2/docs/users_guide/9.4.2-notes.html [blog post]: https://www.haskell.org/ghc/blog/20220805-make-to-hadrian.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Wed Aug 24 17:09:01 2022 From: ben at well-typed.com (Ben Gamari) Date: Wed, 24 Aug 2022 13:09:01 -0400 Subject: The end of the road for the `make` build system Message-ID: <87pmgplgbr.fsf@smart-cactus.org> Hello all, With the 9.4.2 release being out, we are turning our attention to preparations for 9.6. Among these tasks is the removal of the `make`-based build system, allowing us to finally close #17527. The branch removing `make` can be found in !7094, which we hope to send to Marge by the end of this week. For those of you still using `make`, this is your call to switch. Hadrian's documentation can be found in the tree (`hadrian/doc` [1]), the Wiki [2], and our packagers' migration guide [3]. If you have any remaining issues with Hadrian or its documentation do let us know so that we can address them. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/-/tree/master/hadrian/doc [2] https://gitlab.haskell.org/ghc/ghc/-/wikis/building/hadrian [3] https://www.haskell.org/ghc/blog/20220805-make-to-hadrian.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From harendra.kumar at gmail.com Fri Aug 26 10:45:45 2022 From: harendra.kumar at gmail.com (Harendra Kumar) Date: Fri, 26 Aug 2022 16:15:45 +0530 Subject: Instructions to build GHC Message-ID: Hi, I used to follow the instructions in INSTALL.md and README.md in the root of the repo, and they always worked until recently. I am aware that the make based system has been replaced by Hadrian. But it seems these files are not updated with the new build instructions. 1. Either these files should be removed from the source or they should be updated. 2. Where are the new build instructions? Is it hadrian/README.md? If so can we put a pointer to that in INSTALL.md? Thanks, Harendra From tdecacqu at redhat.com Sun Aug 28 18:54:31 2022 From: tdecacqu at redhat.com (Tristan Cacqueray) Date: Sun, 28 Aug 2022 18:54:31 +0000 Subject: Benchmarking lexical scan during the downsweep for the ImplicitQualifiedImport proposal Message-ID: <87y1v8ci6w.tristanC@fedora> Hello all, In the context of https://github.com/ghc-proposals/ghc-proposals/pull/500, I am looking for a strategy to measure the impact of resolving modules dependencies using a lexical scan during the downsweep phase. For example, in this branch: https://gitlab.haskell.org/TristanCacqueray/ghc/-/tree/make-lexical-analysis I've added a tokenizer pass in the GHC.Driver.Make.getPreprocessedImports function, and using `/usr/bin/time -o /dev/stdout -v _build/stage1/bin/ghc compiler/GHC/Tc/TyCl.hs 2> /dev/null` I measure: LexicalAnalysis found 91 qualified name, and lasted: 150msec Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.72 Maximum resident set size (kbytes): 142396 Using a clean build I get: Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.68 Maximum resident set size (kbytes): 140012 Now my question is, how would you measure the time and space cost of such change. For example, what profiling tool would you recommend and how can I test a modified ghc to build ghc itself? Thanks in advance, -Tristan -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 515 bytes Desc: not available URL: From matthewtpickering at gmail.com Mon Aug 29 09:08:21 2022 From: matthewtpickering at gmail.com (Matthew Pickering) Date: Mon, 29 Aug 2022 10:08:21 +0100 Subject: Benchmarking lexical scan during the downsweep for the ImplicitQualifiedImport proposal In-Reply-To: <87y1v8ci6w.tristanC@fedora> References: <87y1v8ci6w.tristanC@fedora> Message-ID: 1. You want to test this in the context of recompilation, that is where the cost of downsweep is most prominent. Ie, compile all the modules normally once and then test how long it takes to recompile. There is already a test in tree which does this (MultiLayerModulesRecomp). 2. You can build GHC using `--make` mode with an invocation something like.. $GHC -O -hidir $out -odir $out -i$dir -Icompiler -I_perf/stage1/compiler/build/ -i_perf/stage1/compiler/build/ GHC -XNoImplicitPrelude -XNoPolyKinds -O 3. The way to extract timing information is to use the `+RTS -s` option. Matt On Sun, Aug 28, 2022 at 7:55 PM Tristan Cacqueray wrote: > > > Hello all, > > In the context of https://github.com/ghc-proposals/ghc-proposals/pull/500, > I am looking for a strategy to measure the impact of resolving modules > dependencies using a lexical scan during the downsweep phase. > > For example, in this branch: https://gitlab.haskell.org/TristanCacqueray/ghc/-/tree/make-lexical-analysis > I've added a tokenizer pass in the GHC.Driver.Make.getPreprocessedImports function, > and using `/usr/bin/time -o /dev/stdout -v _build/stage1/bin/ghc compiler/GHC/Tc/TyCl.hs 2> /dev/null` > I measure: > > LexicalAnalysis found 91 qualified name, and lasted: 150msec > Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.72 > Maximum resident set size (kbytes): 142396 > > Using a clean build I get: > > Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.68 > Maximum resident set size (kbytes): 140012 > > > Now my question is, how would you measure the time and space cost of > such change. For example, what profiling tool would you recommend and > how can I test a modified ghc to build ghc itself? > > Thanks in advance, > -Tristan > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From tdecacqu at redhat.com Mon Aug 29 15:18:00 2022 From: tdecacqu at redhat.com (Tristan Cacqueray) Date: Mon, 29 Aug 2022 15:18:00 +0000 Subject: Benchmarking lexical scan during the downsweep for the ImplicitQualifiedImport proposal In-Reply-To: References: <87y1v8ci6w.tristanC@fedora> Message-ID: <87fshf6puf.tristanC@fedora> Thank you Matt, that's exactly what I was looking for. The `--make GHC` command failed because of hidden packages, and adding the list of `-package-id` arguments used by hadrian fixed the errors. For the record, here is the full invocation I got working: https://gitlab.haskell.org/TristanCacqueray/ghc/-/blob/make-lexical-analysis/build-ghc.sh -Tristan On Mon, Aug 29, 2022 at 10:08 Matthew Pickering wrote: > 1. You want to test this in the context of recompilation, that is > where the cost of downsweep is most prominent. Ie, compile all the > modules normally once and then test how long it takes to recompile. > There is already a test in tree which does this > (MultiLayerModulesRecomp). > > 2. You can build GHC using `--make` mode with an invocation something like.. > > $GHC -O -hidir $out -odir $out -i$dir -Icompiler > -I_perf/stage1/compiler/build/ -i_perf/stage1/compiler/build/ GHC > -XNoImplicitPrelude -XNoPolyKinds -O > > 3. The way to extract timing information is to use the `+RTS -s` option. > > Matt > > > On Sun, Aug 28, 2022 at 7:55 PM Tristan Cacqueray wrote: >> >> >> Hello all, >> >> In the context of https://github.com/ghc-proposals/ghc-proposals/pull/500, >> I am looking for a strategy to measure the impact of resolving modules >> dependencies using a lexical scan during the downsweep phase. >> >> For example, in this branch: https://gitlab.haskell.org/TristanCacqueray/ghc/-/tree/make-lexical-analysis >> I've added a tokenizer pass in the GHC.Driver.Make.getPreprocessedImports function, >> and using `/usr/bin/time -o /dev/stdout -v _build/stage1/bin/ghc compiler/GHC/Tc/TyCl.hs 2> /dev/null` >> I measure: >> >> LexicalAnalysis found 91 qualified name, and lasted: 150msec >> Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.72 >> Maximum resident set size (kbytes): 142396 >> >> Using a clean build I get: >> >> Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.68 >> Maximum resident set size (kbytes): 140012 >> >> >> Now my question is, how would you measure the time and space cost of >> such change. For example, what profiling tool would you recommend and >> how can I test a modified ghc to build ghc itself? >> >> Thanks in advance, >> -Tristan >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 515 bytes Desc: not available URL: