From nboldi at elte.hu Fri Dec 1 02:49:16 2017 From: nboldi at elte.hu (=?UTF-8?B?TsOpbWV0aCBCb2xkaXpzw6Fy?=) Date: Fri, 1 Dec 2017 11:49:16 +0900 Subject: GHC typecheck API In-Reply-To: References:

Message-ID: Thank you for the suggestions! Setting the -fdefer-type-errors flag is indeed a good way to do it (with also adding Opt_DeferTypedHoles and Opt_DeferOutOfScopeVariables for other kind of errors). Boldizsár 2017.12.01. 4:12 keltezéssel, Christopher Done írta: > I suppose setting -fdefer-type-errors would also be handy in this > scenario! > > On Thu, 30 Nov 2017 at 15:37, Simon Peyton Jones via ghc-devs > > wrote: > > This sounds like a good project! > > For the most part things look good: > > * Most type checker errors arise from *type constraints*. The type > checkder tries to solve these, but returns an elaborated syntax > tree (i.e. typechecked, and annotated with types) even if > constraint solving fails. > > * Some renamer errors are like this, notably out-of-scope > variables. (They just show up as another constraint.) > > However there is historical baggage. Back in the beginning, most > errors were treated by throwing an exception in the typechecker > monad; such exceptions can be caught, so that we can get more than > one error from the file, but no syntax tree is returned. Example > let f = in > If there was an error in we'd throw an exception, > catch it at the 'let', give 'f' the type > f :: forall a. a > and continue to typecheck . > > The trouble with the exception stuff is that you don't get an > elaborated syntax tree. > > So: I think you can get some of the way today, just by returning > the tree anyway even if there is an error to report. But it'd > take a bit more work to make more and more errors into things that > don't throw an exception. (Look for failTc, failRn in thd code.) > > I'm not very familiar with the GHC API for this part, but others > will be. I'm certain it can be improved, so rather than hacking > around what is there already, do propose and implement improvements. > > Simon > > | -----Original Message----- > | From: ghc-devs [mailto:ghc-devs-bounces at haskell.org > ] On Behalf Of Németh > | Boldizsár > | Sent: 30 November 2017 05:42 > | To: ghc-devs at haskell.org > | Subject: GHC typecheck API > | > | Dear GHC developers, > | > | I'm developing a framework for development tools for Haskell. I > use the > | GHC API to parse and typecheck the source files. I recently > started to > | work on a quick-fix (automatic program correction) for Haskell > source > | code (correcting parenthesis problems, like 'putStrLn "xxx" ++ > show a' > | ==> 'putStrLn ("xxx" ++ show a)'). To do that I need to get the > typed > | syntax tree even if the program contains type or rename errors. > I could > | only do this by a nasty hack, adding a new TH module finalizer > | (tcg_th_modfinalizers) to extract the type checker's state before it > | fails. Is there a "correct" way to do this? I would not like this > | refactorings to be unusable if the GHC API changes. > | > | By the way, if you know of any similar attempt please let me know. > | > | Sincerely, > | Boldizsár Németh > | haskelltools.org > | > | _______________________________________________ > | ghc-devs mailing list > | ghc-devs at haskell.org > | > https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask > | ell.org %2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- > | devs&data=02%7C01%7Csimonpj%40microsoft.com > %7C364b8db55e064415650d08d537b > | > 5323f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636476173738772433&sda > | ta=Kf5rZYwht8ZGBtGNNH6Q44wLIeefxzHn2UfDIdrNNMU%3D&reserved=0 > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrisdone at gmail.com Fri Dec 1 12:00:41 2017 From: chrisdone at gmail.com (Christopher Done) Date: Fri, 1 Dec 2017 12:00:41 +0000 Subject: GHC typecheck API In-Reply-To: References:

Message-ID: We're currently experimenting with this for Intero. There does seem to be a significant performance reduction with -fdefer-type-errors: https://github.com/commercialhaskell/intero/pull/495#issuecomment-348474127 It's a classic cost benefit scenario: the benefits are much more type info available, including go to definition and things like that being more immediately up to date. The cost is having to wait more time. I believe if there's a good compromise to this, it'll probably involve sometimes enabling -fdefer-type-errors and most of the time turning it off for immediate feedback. On 1 December 2017 at 02:49, Németh Boldizsár wrote: > Thank you for the suggestions! > > Setting the -fdefer-type-errors flag is indeed a good way to do it (with > also adding Opt_DeferTypedHoles and Opt_DeferOutOfScopeVariables for other > kind of errors). > > Boldizsár > > 2017.12.01. 4:12 keltezéssel, Christopher Done írta: > > I suppose setting -fdefer-type-errors would also be handy in this scenario! > > On Thu, 30 Nov 2017 at 15:37, Simon Peyton Jones via ghc-devs < > ghc-devs at haskell.org> wrote: > >> This sounds like a good project! >> >> For the most part things look good: >> >> * Most type checker errors arise from *type constraints*. The type >> checkder tries to solve these, but returns an elaborated syntax tree (i.e. >> typechecked, and annotated with types) even if constraint solving fails. >> >> * Some renamer errors are like this, notably out-of-scope variables. >> (They just show up as another constraint.) >> >> However there is historical baggage. Back in the beginning, most errors >> were treated by throwing an exception in the typechecker monad; such >> exceptions can be caught, so that we can get more than one error from the >> file, but no syntax tree is returned. Example >> let f = in >> If there was an error in we'd throw an exception, catch it >> at the 'let', give 'f' the type >> f :: forall a. a >> and continue to typecheck . >> >> The trouble with the exception stuff is that you don't get an elaborated >> syntax tree. >> >> So: I think you can get some of the way today, just by returning the tree >> anyway even if there is an error to report. But it'd take a bit more work >> to make more and more errors into things that don't throw an exception. >> (Look for failTc, failRn in thd code.) >> >> I'm not very familiar with the GHC API for this part, but others will >> be. I'm certain it can be improved, so rather than hacking around what is >> there already, do propose and implement improvements. >> >> Simon >> >> | -----Original Message----- >> | From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of >> Németh >> | Boldizsár >> | Sent: 30 November 2017 05:42 >> | To: ghc-devs at haskell.org >> | Subject: GHC typecheck API >> | >> | Dear GHC developers, >> | >> | I'm developing a framework for development tools for Haskell. I use the >> | GHC API to parse and typecheck the source files. I recently started to >> | work on a quick-fix (automatic program correction) for Haskell source >> | code (correcting parenthesis problems, like 'putStrLn "xxx" ++ show a' >> | ==> 'putStrLn ("xxx" ++ show a)'). To do that I need to get the typed >> | syntax tree even if the program contains type or rename errors. I could >> | only do this by a nasty hack, adding a new TH module finalizer >> | (tcg_th_modfinalizers) to extract the type checker's state before it >> | fails. Is there a "correct" way to do this? I would not like this >> | refactorings to be unusable if the GHC API changes. >> | >> | By the way, if you know of any similar attempt please let me know. >> | >> | Sincerely, >> | Boldizsár Németh >> | haskelltools.org >> | >> | _______________________________________________ >> | ghc-devs mailing list >> | ghc-devs at haskell.org >> | https://na01.safelinks.protection.outlook.com/?url= >> http%3A%2F%2Fmail.hask >> | ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- >> | devs&data=02%7C01%7Csimonpj%40microsoft.com% >> 7C364b8db55e064415650d08d537b >> | 5323f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0% >> 7C636476173738772433&sda >> | ta=Kf5rZYwht8ZGBtGNNH6Q44wLIeefxzHn2UfDIdrNNMU%3D&reserved=0 >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Fri Dec 1 12:03:30 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Fri, 1 Dec 2017 12:03:30 +0000 Subject: GHC typecheck API In-Reply-To: References:

Message-ID: There does seem to be a significant performance reduction with -fdefer-type-errors: https://github.com/commercialhaskell/intero/pull/495#issuecomment-348474127 That’s odd. I would expect zero effect if there aren’t any errors, and not much even if ther are. It’s not clear whether the report is to do with compile time or runtime. If you can boil down a test case, maybe a ticket would be good From: Christopher Done [mailto:chrisdone at gmail.com] Sent: 01 December 2017 12:01 To: Németh Boldizsár Cc: Simon Peyton Jones ; ghc-devs at haskell.org Subject: Re: GHC typecheck API We're currently experimenting with this for Intero. There does seem to be a significant performance reduction with -fdefer-type-errors: https://github.com/commercialhaskell/intero/pull/495#issuecomment-348474127 It's a classic cost benefit scenario: the benefits are much more type info available, including go to definition and things like that being more immediately up to date. The cost is having to wait more time. I believe if there's a good compromise to this, it'll probably involve sometimes enabling -fdefer-type-errors and most of the time turning it off for immediate feedback. On 1 December 2017 at 02:49, Németh Boldizsár > wrote: Thank you for the suggestions! Setting the -fdefer-type-errors flag is indeed a good way to do it (with also adding Opt_DeferTypedHoles and Opt_DeferOutOfScopeVariables for other kind of errors). Boldizsár 2017.12.01. 4:12 keltezéssel, Christopher Done írta: I suppose setting -fdefer-type-errors would also be handy in this scenario! On Thu, 30 Nov 2017 at 15:37, Simon Peyton Jones via ghc-devs > wrote: This sounds like a good project! For the most part things look good: * Most type checker errors arise from *type constraints*. The type checkder tries to solve these, but returns an elaborated syntax tree (i.e. typechecked, and annotated with types) even if constraint solving fails. * Some renamer errors are like this, notably out-of-scope variables. (They just show up as another constraint.) However there is historical baggage. Back in the beginning, most errors were treated by throwing an exception in the typechecker monad; such exceptions can be caught, so that we can get more than one error from the file, but no syntax tree is returned. Example let f = in If there was an error in we'd throw an exception, catch it at the 'let', give 'f' the type f :: forall a. a and continue to typecheck . The trouble with the exception stuff is that you don't get an elaborated syntax tree. So: I think you can get some of the way today, just by returning the tree anyway even if there is an error to report. But it'd take a bit more work to make more and more errors into things that don't throw an exception. (Look for failTc, failRn in thd code.) I'm not very familiar with the GHC API for this part, but others will be. I'm certain it can be improved, so rather than hacking around what is there already, do propose and implement improvements. Simon | -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of Németh | Boldizsár | Sent: 30 November 2017 05:42 | To: ghc-devs at haskell.org | Subject: GHC typecheck API | | Dear GHC developers, | | I'm developing a framework for development tools for Haskell. I use the | GHC API to parse and typecheck the source files. I recently started to | work on a quick-fix (automatic program correction) for Haskell source | code (correcting parenthesis problems, like 'putStrLn "xxx" ++ show a' | ==> 'putStrLn ("xxx" ++ show a)'). To do that I need to get the typed | syntax tree even if the program contains type or rename errors. I could | only do this by a nasty hack, adding a new TH module finalizer | (tcg_th_modfinalizers) to extract the type checker's state before it | fails. Is there a "correct" way to do this? I would not like this | refactorings to be unusable if the GHC API changes. | | By the way, if you know of any similar attempt please let me know. | | Sincerely, | Boldizsár Németh | haskelltools.org | | _______________________________________________ | ghc-devs mailing list | ghc-devs at haskell.org | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask | ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- | devs&data=02%7C01%7Csimonpj%40microsoft.com%7C364b8db55e064415650d08d537b | 5323f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636476173738772433&sda | ta=Kf5rZYwht8ZGBtGNNH6Q44wLIeefxzHn2UfDIdrNNMU%3D&reserved=0 _______________________________________________ ghc-devs mailing list ghc-devs at haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrisdone at gmail.com Fri Dec 1 12:10:32 2017 From: chrisdone at gmail.com (Christopher Done) Date: Fri, 1 Dec 2017 12:10:32 +0000 Subject: GHC typecheck API In-Reply-To: References:

Message-ID: The report is timing how long a `:load` takes. Sure, if/when I reproduce this on some open source module I'll make a ticket with a test case! On 1 December 2017 at 12:03, Simon Peyton Jones wrote: > There does seem to be a significant performance reduction with > -fdefer-type-errors: > > https://github.com/commercialhaskell/intero/pull/ > 495#issuecomment-348474127 > > > > > That’s odd. I would expect zero effect if there aren’t any errors, and not > much even if ther are. It’s not clear whether the report is to do with > compile time or runtime. > > > > If you can boil down a test case, maybe a ticket would be good > > > > *From:* Christopher Done [mailto:chrisdone at gmail.com] > *Sent:* 01 December 2017 12:01 > *To:* Németh Boldizsár > *Cc:* Simon Peyton Jones ; ghc-devs at haskell.org > *Subject:* Re: GHC typecheck API > > > > We're currently experimenting with this for Intero. There does seem to be > a significant performance reduction with -fdefer-type-errors: > > > > https://github.com/commercialhaskell/intero/pull/ > 495#issuecomment-348474127 > > > > > It's a classic cost benefit scenario: the benefits are much more type info > available, including go to definition and things like that being more > immediately up to date. The cost is having to wait more time. > > > > I believe if there's a good compromise to this, it'll probably involve > sometimes enabling -fdefer-type-errors and most of the time turning it off > for immediate feedback. > > > > > > > > On 1 December 2017 at 02:49, Németh Boldizsár wrote: > > Thank you for the suggestions! > > Setting the -fdefer-type-errors flag is indeed a good way to do it (with > also adding Opt_DeferTypedHoles and Opt_DeferOutOfScopeVariables for other > kind of errors). > > Boldizsár > > > > 2017.12.01. 4:12 keltezéssel, Christopher Done írta: > > I suppose setting -fdefer-type-errors would also be handy in this scenario! > > > > On Thu, 30 Nov 2017 at 15:37, Simon Peyton Jones via ghc-devs < > ghc-devs at haskell.org> wrote: > > This sounds like a good project! > > For the most part things look good: > > * Most type checker errors arise from *type constraints*. The type > checkder tries to solve these, but returns an elaborated syntax tree (i.e. > typechecked, and annotated with types) even if constraint solving fails. > > * Some renamer errors are like this, notably out-of-scope variables. > (They just show up as another constraint.) > > However there is historical baggage. Back in the beginning, most errors > were treated by throwing an exception in the typechecker monad; such > exceptions can be caught, so that we can get more than one error from the > file, but no syntax tree is returned. Example > let f = in > If there was an error in we'd throw an exception, catch it at > the 'let', give 'f' the type > f :: forall a. a > and continue to typecheck . > > The trouble with the exception stuff is that you don't get an elaborated > syntax tree. > > So: I think you can get some of the way today, just by returning the tree > anyway even if there is an error to report. But it'd take a bit more work > to make more and more errors into things that don't throw an exception. > (Look for failTc, failRn in thd code.) > > I'm not very familiar with the GHC API for this part, but others will be. > I'm certain it can be improved, so rather than hacking around what is there > already, do propose and implement improvements. > > Simon > > | -----Original Message----- > | From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of Németh > | Boldizsár > | Sent: 30 November 2017 05:42 > | To: ghc-devs at haskell.org > | Subject: GHC typecheck API > | > | Dear GHC developers, > | > | I'm developing a framework for development tools for Haskell. I use the > | GHC API to parse and typecheck the source files. I recently started to > | work on a quick-fix (automatic program correction) for Haskell source > | code (correcting parenthesis problems, like 'putStrLn "xxx" ++ show a' > | ==> 'putStrLn ("xxx" ++ show a)'). To do that I need to get the typed > | syntax tree even if the program contains type or rename errors. I could > | only do this by a nasty hack, adding a new TH module finalizer > | (tcg_th_modfinalizers) to extract the type checker's state before it > | fails. Is there a "correct" way to do this? I would not like this > | refactorings to be unusable if the GHC API changes. > | > | By the way, if you know of any similar attempt please let me know. > | > | Sincerely, > | Boldizsár Németh > | haskelltools.org > > | > | _______________________________________________ > | ghc-devs mailing list > | ghc-devs at haskell.org > | https://na01.safelinks.protection.outlook.com/?url= > http%3A%2F%2Fmail.hask > | ell.org > > %2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- > | devs&data=02%7C01%7Csimonpj%40microsoft.com > > %7C364b8db55e064415650d08d537b > | 5323f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0% > 7C636476173738772433&sda > | ta=Kf5rZYwht8ZGBtGNNH6Q44wLIeefxzHn2UfDIdrNNMU%3D&reserved=0 > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From saurabhnanda at gmail.com Sat Dec 2 15:59:17 2017 From: saurabhnanda at gmail.com (Saurabh Nanda) Date: Sat, 2 Dec 2017 21:29:17 +0530 Subject: How to load & parse an HI (interface) file? Message-ID: (GHC newbie alert -- is this the right mailing list for these kind of questions?) I"m writing some code to figure out all the instances of particular type-classes and after exploring a lot of options (hlint, haskell-src-exts, annotations, doctests, etc), I realized that the compiler had already figured it out and written it to disk for me! More digging led me to https://www.stackage.org/haddock/lts-9.0/ghc-8.0.2/LoadIface.html#v:loadSrcInterface after which I got stuck. How does one call this function? Specifically: * What is SDoc and how to construct a reasonable value for this argument? * IsBootInterface would mostly be False, right? * What does `Maybe FastString` represent and how does one construct it? * Finally how does one evaluate the resulting monadic action to get access to the underlying `ModIface`? -- Saurabh. -------------- next part -------------- An HTML attachment was scrubbed... URL: From harendra.kumar at gmail.com Sat Dec 2 20:20:44 2017 From: harendra.kumar at gmail.com (Harendra Kumar) Date: Sun, 3 Dec 2017 01:50:44 +0530 Subject: How to load & parse an HI (interface) file? In-Reply-To: References: Message-ID: GHC has a "--show-iface" option which pretty prints the ".hi" file. Not sure if it works for your use-case but it may be easier to parse the text displayed by this option. -harendra On 2 December 2017 at 21:29, Saurabh Nanda wrote: > (GHC newbie alert -- is this the right mailing list for these kind of > questions?) > > I"m writing some code to figure out all the instances of particular > type-classes and after exploring a lot of options (hlint, haskell-src-exts, > annotations, doctests, etc), I realized that the compiler had already > figured it out and written it to disk for me! > > More digging led me to https://www.stackage.org/haddock/lts-9.0/ghc-8.0.2/ > LoadIface.html#v:loadSrcInterface after which I got stuck. How does one > call this function? Specifically: > > * What is SDoc and how to construct a reasonable value for this argument? > * IsBootInterface would mostly be False, right? > * What does `Maybe FastString` represent and how does one construct it? > * Finally how does one evaluate the resulting monadic action to get access > to the underlying `ModIface`? > > -- Saurabh. > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harendra.kumar at gmail.com Sat Dec 2 20:24:03 2017 From: harendra.kumar at gmail.com (Harendra Kumar) Date: Sun, 3 Dec 2017 01:54:03 +0530 Subject: How to load & parse an HI (interface) file? In-Reply-To: References: Message-ID: Also, "-ddump-hi" dumps the same information at compile time. -harendra On 3 December 2017 at 01:50, Harendra Kumar wrote: > GHC has a "--show-iface" option which pretty prints the ".hi" file. Not > sure if it works for your use-case but it may be easier to parse the text > displayed by this option. > > -harendra > > On 2 December 2017 at 21:29, Saurabh Nanda wrote: > >> (GHC newbie alert -- is this the right mailing list for these kind of >> questions?) >> >> I"m writing some code to figure out all the instances of particular >> type-classes and after exploring a lot of options (hlint, haskell-src-exts, >> annotations, doctests, etc), I realized that the compiler had already >> figured it out and written it to disk for me! >> >> More digging led me to https://www.stackage.org/haddo >> ck/lts-9.0/ghc-8.0.2/LoadIface.html#v:loadSrcInterface after which I got >> stuck. How does one call this function? Specifically: >> >> * What is SDoc and how to construct a reasonable value for this argument? >> * IsBootInterface would mostly be False, right? >> * What does `Maybe FastString` represent and how does one construct it? >> * Finally how does one evaluate the resulting monadic action to get >> access to the underlying `ModIface`? >> >> -- Saurabh. >> >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From allbery.b at gmail.com Sat Dec 2 20:34:27 2017 From: allbery.b at gmail.com (Brandon Allbery) Date: Sat, 2 Dec 2017 15:34:27 -0500 Subject: How to load & parse an HI (interface) file? In-Reply-To: References: Message-ID: I would be cautious about using the ghc-api hi file interfaces; hi files turn out to interact with a lot of low-level parts in complex ways (even to the extent that they're a large part of why ghc can't parallelize builds itself and attempts to change that have mostly failed). But if you must do this, you *really* want to have https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler ready to hand --- and go through it first so you have some idea of how it works; much of it is links to the lower level details (often straight into the source). On Sat, Dec 2, 2017 at 10:59 AM, Saurabh Nanda wrote: > (GHC newbie alert -- is this the right mailing list for these kind of > questions?) > > I"m writing some code to figure out all the instances of particular > type-classes and after exploring a lot of options (hlint, haskell-src-exts, > annotations, doctests, etc), I realized that the compiler had already > figured it out and written it to disk for me! > > More digging led me to https://www.stackage.org/haddock/lts-9.0/ghc-8.0.2/ > LoadIface.html#v:loadSrcInterface after which I got stuck. How does one > call this function? Specifically: > > * What is SDoc and how to construct a reasonable value for this argument? > * IsBootInterface would mostly be False, right? > * What does `Maybe FastString` represent and how does one construct it? > * Finally how does one evaluate the resulting monadic action to get access > to the underlying `ModIface`? > > -- Saurabh. > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > -- brandon s allbery kf8nh sine nomine associates allbery.b at gmail.com ballbery at sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From saurabhnanda at gmail.com Sun Dec 3 02:03:56 2017 From: saurabhnanda at gmail.com (Saurabh Nanda) Date: Sun, 3 Dec 2017 07:33:56 +0530 Subject: How to load & parse an HI (interface) file? In-Reply-To: References: Message-ID: GHC has a "--show-iface" option which pretty prints the ".hi" file. Not > sure if it works for your use-case but it may be easier to parse the text > displayed by this option. > >From https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler/IfaceFiles -- > This textual format is not particularly designed for machine parsing. Doing so might be possible, but if you > want to read GHC interface files you are almost certainly better off using the GHC API to do so. That's the reason why I started investigating the GHC API. -- Saurabh. -------------- next part -------------- An HTML attachment was scrubbed... URL: From saurabhnanda at gmail.com Sun Dec 3 02:11:43 2017 From: saurabhnanda at gmail.com (Saurabh Nanda) Date: Sun, 3 Dec 2017 07:41:43 +0530 Subject: How to load & parse an HI (interface) file? In-Reply-To: References: Message-ID: > I would be cautious about using the ghc-api hi file interfaces; hi files turn out to interact with a > lot of low-level parts in complex ways (even to the extent that they're a large part of why ghc > can't parallelize builds itself and attempts to change that have mostly failed). Are you cautioning against using the GHC API (as opposed to the --show-iface command line interface) or using HI files themselves? -- Saurabh. On Sun, Dec 3, 2017 at 2:04 AM, Brandon Allbery wrote: > I would be cautious about using the ghc-api hi file interfaces; hi files > turn out to interact with a lot of low-level parts in complex ways (even to > the extent that they're a large part of why ghc can't parallelize builds > itself and attempts to change that have mostly failed). > > But if you must do this, you *really* want to have > https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler ready to hand > --- and go through it first so you have some idea of how it works; much of > it is links to the lower level details (often straight into the source). > > On Sat, Dec 2, 2017 at 10:59 AM, Saurabh Nanda > wrote: > >> (GHC newbie alert -- is this the right mailing list for these kind of >> questions?) >> >> I"m writing some code to figure out all the instances of particular >> type-classes and after exploring a lot of options (hlint, haskell-src-exts, >> annotations, doctests, etc), I realized that the compiler had already >> figured it out and written it to disk for me! >> >> More digging led me to https://www.stackage.org/haddo >> ck/lts-9.0/ghc-8.0.2/LoadIface.html#v:loadSrcInterface after which I got >> stuck. How does one call this function? Specifically: >> >> * What is SDoc and how to construct a reasonable value for this argument? >> * IsBootInterface would mostly be False, right? >> * What does `Maybe FastString` represent and how does one construct it? >> * Finally how does one evaluate the resulting monadic action to get >> access to the underlying `ModIface`? >> >> -- Saurabh. >> >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> >> > > > -- > brandon s allbery kf8nh sine nomine > associates > allbery.b at gmail.com > ballbery at sinenomine.net > unix, openafs, kerberos, infrastructure, xmonad > http://sinenomine.net > -- http://www.saurabhnanda.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From saurabhnanda at gmail.com Sun Dec 3 02:37:25 2017 From: saurabhnanda at gmail.com (Saurabh Nanda) Date: Sun, 3 Dec 2017 08:07:25 +0530 Subject: How to load & parse an HI (interface) file? In-Reply-To: References: Message-ID: I _think_ I've found what I need from https://github.com/ghc/ghc/blob/8843a39b3c941b1908a8d839f52bc323f3b45081/compiler/iface/LoadIface.hs#L994-L1002 On Sun, Dec 3, 2017 at 7:41 AM, Saurabh Nanda wrote: > > I would be cautious about using the ghc-api hi file interfaces; hi files > turn out to interact with a > > lot of low-level parts in complex ways (even to the extent that they're > a large part of why ghc > > can't parallelize builds itself and attempts to change that have mostly > failed). > > Are you cautioning against using the GHC API (as opposed to the > --show-iface command line interface) > or using HI files themselves? > > -- Saurabh. > > > On Sun, Dec 3, 2017 at 2:04 AM, Brandon Allbery > wrote: > >> I would be cautious about using the ghc-api hi file interfaces; hi files >> turn out to interact with a lot of low-level parts in complex ways (even to >> the extent that they're a large part of why ghc can't parallelize builds >> itself and attempts to change that have mostly failed). >> >> But if you must do this, you *really* want to have >> https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler ready to hand >> --- and go through it first so you have some idea of how it works; much of >> it is links to the lower level details (often straight into the source). >> >> On Sat, Dec 2, 2017 at 10:59 AM, Saurabh Nanda >> wrote: >> >>> (GHC newbie alert -- is this the right mailing list for these kind of >>> questions?) >>> >>> I"m writing some code to figure out all the instances of particular >>> type-classes and after exploring a lot of options (hlint, haskell-src-exts, >>> annotations, doctests, etc), I realized that the compiler had already >>> figured it out and written it to disk for me! >>> >>> More digging led me to https://www.stackage.org/haddo >>> ck/lts-9.0/ghc-8.0.2/LoadIface.html#v:loadSrcInterface after which I >>> got stuck. How does one call this function? Specifically: >>> >>> * What is SDoc and how to construct a reasonable value for this argument? >>> * IsBootInterface would mostly be False, right? >>> * What does `Maybe FastString` represent and how does one construct it? >>> * Finally how does one evaluate the resulting monadic action to get >>> access to the underlying `ModIface`? >>> >>> -- Saurabh. >>> >>> >>> _______________________________________________ >>> ghc-devs mailing list >>> ghc-devs at haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >>> >>> >> >> >> -- >> brandon s allbery kf8nh sine nomine >> associates >> allbery.b at gmail.com >> ballbery at sinenomine.net >> unix, openafs, kerberos, infrastructure, xmonad >> http://sinenomine.net >> > > > > -- > http://www.saurabhnanda.com > -- http://www.saurabhnanda.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From allbery.b at gmail.com Sun Dec 3 02:43:29 2017 From: allbery.b at gmail.com (Brandon Allbery) Date: Sat, 2 Dec 2017 21:43:29 -0500 Subject: How to load & parse an HI (interface) file? In-Reply-To: References: Message-ID: The problem with the API is it's complex and can break between ghc versions. But --show-iface is even more fragile and prone to break between ghc versions. The history of the plugins package constitutes a record of both kinds of pain. On Sat, Dec 2, 2017 at 9:11 PM, Saurabh Nanda wrote: > > I would be cautious about using the ghc-api hi file interfaces; hi files > turn out to interact with a > > lot of low-level parts in complex ways (even to the extent that they're > a large part of why ghc > > can't parallelize builds itself and attempts to change that have mostly > failed). > > Are you cautioning against using the GHC API (as opposed to the > --show-iface command line interface) > or using HI files themselves? > > -- Saurabh. > > > On Sun, Dec 3, 2017 at 2:04 AM, Brandon Allbery > wrote: > >> I would be cautious about using the ghc-api hi file interfaces; hi files >> turn out to interact with a lot of low-level parts in complex ways (even to >> the extent that they're a large part of why ghc can't parallelize builds >> itself and attempts to change that have mostly failed). >> >> But if you must do this, you *really* want to have >> https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler ready to hand >> --- and go through it first so you have some idea of how it works; much of >> it is links to the lower level details (often straight into the source). >> >> On Sat, Dec 2, 2017 at 10:59 AM, Saurabh Nanda >> wrote: >> >>> (GHC newbie alert -- is this the right mailing list for these kind of >>> questions?) >>> >>> I"m writing some code to figure out all the instances of particular >>> type-classes and after exploring a lot of options (hlint, haskell-src-exts, >>> annotations, doctests, etc), I realized that the compiler had already >>> figured it out and written it to disk for me! >>> >>> More digging led me to https://www.stackage.org/haddo >>> ck/lts-9.0/ghc-8.0.2/LoadIface.html#v:loadSrcInterface after which I >>> got stuck. How does one call this function? Specifically: >>> >>> * What is SDoc and how to construct a reasonable value for this argument? >>> * IsBootInterface would mostly be False, right? >>> * What does `Maybe FastString` represent and how does one construct it? >>> * Finally how does one evaluate the resulting monadic action to get >>> access to the underlying `ModIface`? >>> >>> -- Saurabh. >>> >>> >>> _______________________________________________ >>> ghc-devs mailing list >>> ghc-devs at haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >>> >>> >> >> >> -- >> brandon s allbery kf8nh sine nomine >> associates >> allbery.b at gmail.com >> ballbery at sinenomine.net >> unix, openafs, kerberos, infrastructure, xmonad >> http://sinenomine.net >> > > > > -- > http://www.saurabhnanda.com > -- brandon s allbery kf8nh sine nomine associates allbery.b at gmail.com ballbery at sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgraf1337 at gmail.com Sun Dec 3 14:27:33 2017 From: sgraf1337 at gmail.com (Sebastian Graf) Date: Sun, 3 Dec 2017 15:27:33 +0100 Subject: How to load & parse an HI (interface) file? In-Reply-To: References:

Message-ID: Hey, there's this relatively recent thread on finding instances of a type class: https://mail.haskell.org/pipermail/ghc-devs/2017-May/014217.html I'm sorry, but I couldn't find a better archive for ghc-devs. Enjoy Sebastian On Sun, Dec 3, 2017 at 3:43 AM, Brandon Allbery wrote: > The problem with the API is it's complex and can break between ghc > versions. > But --show-iface is even more fragile and prone to break between ghc > versions. > The history of the plugins package constitutes a record of both kinds of > pain. > > On Sat, Dec 2, 2017 at 9:11 PM, Saurabh Nanda > wrote: > >> > I would be cautious about using the ghc-api hi file interfaces; hi >> files turn out to interact with a >> > lot of low-level parts in complex ways (even to the extent that they're >> a large part of why ghc >> > can't parallelize builds itself and attempts to change that have mostly >> failed). >> >> Are you cautioning against using the GHC API (as opposed to the >> --show-iface command line interface) >> or using HI files themselves? >> >> -- Saurabh. >> >> >> On Sun, Dec 3, 2017 at 2:04 AM, Brandon Allbery >> wrote: >> >>> I would be cautious about using the ghc-api hi file interfaces; hi files >>> turn out to interact with a lot of low-level parts in complex ways (even to >>> the extent that they're a large part of why ghc can't parallelize builds >>> itself and attempts to change that have mostly failed). >>> >>> But if you must do this, you *really* want to have >>> https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler ready to hand >>> --- and go through it first so you have some idea of how it works; much of >>> it is links to the lower level details (often straight into the source). >>> >>> On Sat, Dec 2, 2017 at 10:59 AM, Saurabh Nanda >>> wrote: >>> >>>> (GHC newbie alert -- is this the right mailing list for these kind of >>>> questions?) >>>> >>>> I"m writing some code to figure out all the instances of particular >>>> type-classes and after exploring a lot of options (hlint, haskell-src-exts, >>>> annotations, doctests, etc), I realized that the compiler had already >>>> figured it out and written it to disk for me! >>>> >>>> More digging led me to https://www.stackage.org/haddo >>>> ck/lts-9.0/ghc-8.0.2/LoadIface.html#v:loadSrcInterface after which I >>>> got stuck. How does one call this function? Specifically: >>>> >>>> * What is SDoc and how to construct a reasonable value for this >>>> argument? >>>> * IsBootInterface would mostly be False, right? >>>> * What does `Maybe FastString` represent and how does one construct it? >>>> * Finally how does one evaluate the resulting monadic action to get >>>> access to the underlying `ModIface`? >>>> >>>> -- Saurabh. >>>> >>>> >>>> _______________________________________________ >>>> ghc-devs mailing list >>>> ghc-devs at haskell.org >>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >>>> >>>> >>> >>> >>> -- >>> brandon s allbery kf8nh sine nomine >>> associates >>> allbery.b at gmail.com >>> ballbery at sinenomine.net >>> unix, openafs, kerberos, infrastructure, xmonad >>> http://sinenomine.net >>> >> >> >> >> -- >> http://www.saurabhnanda.com >> > > > > -- > brandon s allbery kf8nh sine nomine > associates > allbery.b at gmail.com > ballbery at sinenomine.net > unix, openafs, kerberos, infrastructure, xmonad > http://sinenomine.net > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cheater00 at gmail.com Tue Dec 5 17:36:50 2017 From: cheater00 at gmail.com (cheater00 cheater00) Date: Tue, 5 Dec 2017 18:36:50 +0100 Subject: Long standing annoying issue in ghci Message-ID: Hi guys, this one seems to have gotten buried under more important things, but maybe someone would like to take a look at it none the less? I've been encountering it literally every day for the last 4 years. I sometimes find it difficult to find read the past commands without color coding the prompt so I can't really turn it off. It seems like a simple arithmetic issue somewhere in the readline implementation. https://ghc.haskell.org/trac/ghc/ticket/9364 Thanks a lot. From allbery.b at gmail.com Tue Dec 5 20:57:23 2017 From: allbery.b at gmail.com (Brandon Allbery) Date: Tue, 5 Dec 2017 15:57:23 -0500 Subject: Long standing annoying issue in ghci In-Reply-To: References: Message-ID: On Tue, Dec 5, 2017 at 12:36 PM, cheater00 cheater00 wrote: > without color coding the prompt so I can't really turn it off. It > seems like a simple arithmetic issue somewhere in the readline > implementation. > It's not arithmetic except in the sense that it's not doing *any* math. Color codes in a terminal are necessarily implemented as character sequences (this is pretty much the definition of a terminal interface), and haskeline makes no effort to recognize them, so it treats them the same as displayed character sequences and skips over them as if they were displayed characters. GNU readline handles this by recognizing the character mode sequences as not taking up character positions (this is more complex than you think given that GNU readline doesn't assume all terminals obey the ANSI standard; as it turns out, neither does haskeline, so it actually gets a bit nasty), and recognizing the special behavior of carriage return, and providing \[ \] escapes to declare the sequence inside as "invisible" to to character positioning (and it's on the person crafting the prompt to insure that it actually is). Beyond that, it'd actually have to implement a 'terminal emulator' internally to get it right in all cases --- and i'd be on the user to ensure their declared terminal type matches the actual one well enough for the 'terminal emulator' to reflect the terminal's actual behavior, so it'd be a potential source of even weirder behavior. So, (a) haskeline issue, not strictly ghc/ghci, and (b) not really fixable, but partially work-around-able for common cases. -- brandon s allbery kf8nh sine nomine associates allbery.b at gmail.com ballbery at sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From qdunkan at gmail.com Tue Dec 5 21:49:22 2017 From: qdunkan at gmail.com (Evan Laforge) Date: Tue, 5 Dec 2017 13:49:22 -0800 Subject: Long standing annoying issue in ghci In-Reply-To: References:

Message-ID: Here's what I use: :set prompt "\ESC[46m\STX%s>\ESC[39;49m\STX " I believe \STX is a signal to haskeline for control sequences. Documentation is here: https://github.com/judah/haskeline/wiki/ControlSequencesInPrompt On Tue, Dec 5, 2017 at 12:57 PM, Brandon Allbery wrote: > On Tue, Dec 5, 2017 at 12:36 PM, cheater00 cheater00 > wrote: >> >> without color coding the prompt so I can't really turn it off. It >> seems like a simple arithmetic issue somewhere in the readline >> implementation. > > > It's not arithmetic except in the sense that it's not doing *any* math. > Color codes in a terminal are necessarily implemented as character sequences > (this is pretty much the definition of a terminal interface), and haskeline > makes no effort to recognize them, so it treats them the same as displayed > character sequences and skips over them as if they were displayed > characters. > > GNU readline handles this by recognizing the character mode sequences as not > taking up character positions (this is more complex than you think given > that GNU readline doesn't assume all terminals obey the ANSI standard; as it > turns out, neither does haskeline, so it actually gets a bit nasty), and > recognizing the special behavior of carriage return, and providing \[ \] > escapes to declare the sequence inside as "invisible" to to character > positioning (and it's on the person crafting the prompt to insure that it > actually is). Beyond that, it'd actually have to implement a 'terminal > emulator' internally to get it right in all cases --- and i'd be on the user > to ensure their declared terminal type matches the actual one well enough > for the 'terminal emulator' to reflect the terminal's actual behavior, so > it'd be a potential source of even weirder behavior. > > So, (a) haskeline issue, not strictly ghc/ghci, and (b) not really fixable, > but partially work-around-able for common cases. > > -- > brandon s allbery kf8nh sine nomine associates > allbery.b at gmail.com ballbery at sinenomine.net > unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > From cheater00 at gmail.com Tue Dec 5 21:57:56 2017 From: cheater00 at gmail.com (cheater00 cheater00) Date: Tue, 05 Dec 2017 21:57:56 +0000 Subject: Long standing annoying issue in ghci In-Reply-To: References:

Message-ID: Thanks Evan, the \STX thing fixed my issue. Much happier now. I'll update the bug and set it to won't fix. On Tue, 5 Dec 2017 22:49 Evan Laforge, wrote: > Here's what I use: > > :set prompt "\ESC[46m\STX%s>\ESC[39;49m\STX " > > I believe \STX is a signal to haskeline for control sequences. > Documentation is here: > https://github.com/judah/haskeline/wiki/ControlSequencesInPrompt > > On Tue, Dec 5, 2017 at 12:57 PM, Brandon Allbery > wrote: > > On Tue, Dec 5, 2017 at 12:36 PM, cheater00 cheater00 < > cheater00 at gmail.com> > > wrote: > >> > >> without color coding the prompt so I can't really turn it off. It > >> seems like a simple arithmetic issue somewhere in the readline > >> implementation. > > > > > > It's not arithmetic except in the sense that it's not doing *any* math. > > Color codes in a terminal are necessarily implemented as character > sequences > > (this is pretty much the definition of a terminal interface), and > haskeline > > makes no effort to recognize them, so it treats them the same as > displayed > > character sequences and skips over them as if they were displayed > > characters. > > > > GNU readline handles this by recognizing the character mode sequences as > not > > taking up character positions (this is more complex than you think given > > that GNU readline doesn't assume all terminals obey the ANSI standard; > as it > > turns out, neither does haskeline, so it actually gets a bit nasty), and > > recognizing the special behavior of carriage return, and providing \[ \] > > escapes to declare the sequence inside as "invisible" to to character > > positioning (and it's on the person crafting the prompt to insure that it > > actually is). Beyond that, it'd actually have to implement a 'terminal > > emulator' internally to get it right in all cases --- and i'd be on the > user > > to ensure their declared terminal type matches the actual one well enough > > for the 'terminal emulator' to reflect the terminal's actual behavior, so > > it'd be a potential source of even weirder behavior. > > > > So, (a) haskeline issue, not strictly ghc/ghci, and (b) not really > fixable, > > but partially work-around-able for common cases. > > > > -- > > brandon s allbery kf8nh sine nomine > associates > > allbery.b at gmail.com > ballbery at sinenomine.net > > unix, openafs, kerberos, infrastructure, xmonad > http://sinenomine.net > > > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From allbery.b at gmail.com Tue Dec 5 21:59:06 2017 From: allbery.b at gmail.com (Brandon Allbery) Date: Tue, 5 Dec 2017 16:59:06 -0500 Subject: Long standing annoying issue in ghci In-Reply-To: References:

Message-ID: On Tue, Dec 5, 2017 at 4:49 PM, Evan Laforge wrote: > I believe \STX is a signal to haskeline for control sequences. > Documentation is here: > https://github.com/judah/haskeline/wiki/ControlSequencesInPrompt Huh, so they did add support for that. With a version constraint, so this will depend also on haskeline version (and therefore ghc version, and what works in newer ghci will therefore not work in older ones). -- brandon s allbery kf8nh sine nomine associates allbery.b at gmail.com ballbery at sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at well-typed.com Tue Dec 5 22:13:21 2017 From: david at well-typed.com (David Feuer) Date: Tue, 05 Dec 2017 17:13:21 -0500 Subject: Long standing annoying issue in ghci Message-ID: <20171205214243.85EBFBC90E@haskell.org> It sounds like this should be added to the GHCi documentation, even if it's not strictly about GHCi. David FeuerWell-Typed, LLP -------- Original message --------From: Evan Laforge Date: 12/5/17 4:49 PM (GMT-05:00) To: Brandon Allbery Cc: ghc-devs at haskell.org Subject: Re: Long standing annoying issue in ghci Here's what I use: :set prompt "\ESC[46m\STX%s>\ESC[39;49m\STX " I believe \STX is a signal to haskeline for control sequences. Documentation is here: https://github.com/judah/haskeline/wiki/ControlSequencesInPrompt On Tue, Dec 5, 2017 at 12:57 PM, Brandon Allbery wrote: > On Tue, Dec 5, 2017 at 12:36 PM, cheater00 cheater00 > wrote: >> >> without color coding the prompt so I can't really turn it off. It >> seems like a simple arithmetic issue somewhere in the readline >> implementation. > > > It's not arithmetic except in the sense that it's not doing *any* math. > Color codes in a terminal are necessarily implemented as character sequences > (this is pretty much the definition of a terminal interface), and haskeline > makes no effort to recognize them, so it treats them the same as displayed > character sequences and skips over them as if they were displayed > characters. > > GNU readline handles this by recognizing the character mode sequences as not > taking up character positions (this is more complex than you think given > that GNU readline doesn't assume all terminals obey the ANSI standard; as it > turns out, neither does haskeline, so it actually gets a bit nasty), and > recognizing the special behavior of carriage return, and providing \[ \] > escapes to declare the sequence inside as "invisible" to to character > positioning (and it's on the person crafting the prompt to insure that it > actually is). Beyond that, it'd actually have to implement a 'terminal > emulator' internally to get it right in all cases --- and i'd be on the user > to ensure their declared terminal type matches the actual one well enough > for the 'terminal emulator' to reflect the terminal's actual behavior, so > it'd be a potential source of even weirder behavior. > > So, (a) haskeline issue, not strictly ghc/ghci, and (b) not really fixable, > but partially work-around-able for common cases. > > -- > brandon s allbery kf8nh sine nomine associates > allbery.b at gmail.com ballbery at sinenomine.net > unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > _______________________________________________ ghc-devs mailing list ghc-devs at haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -------------- next part -------------- An HTML attachment was scrubbed... URL: From allbery.b at gmail.com Tue Dec 5 22:16:37 2017 From: allbery.b at gmail.com (Brandon Allbery) Date: Tue, 5 Dec 2017 17:16:37 -0500 Subject: Long standing annoying issue in ghci In-Reply-To: <5a271a07.b29adf0a.8175a.6438SMTPIN_ADDED_MISSING@mx.google.com> References: <5a271a07.b29adf0a.8175a.6438SMTPIN_ADDED_MISSING@mx.google.com> Message-ID: It's indirectly already there: https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ghci.html#the-ghci-and-haskeline-files Possibly an additional pointer needs to be in the FAQ/things to watch out for section. On Tue, Dec 5, 2017 at 5:13 PM, David Feuer wrote: > It sounds like this should be added to the GHCi documentation, even if > it's not strictly about GHCi. > > > > David Feuer > Well-Typed, LLP > > -------- Original message -------- > From: Evan Laforge > Date: 12/5/17 4:49 PM (GMT-05:00) > To: Brandon Allbery > Cc: ghc-devs at haskell.org > Subject: Re: Long standing annoying issue in ghci > > Here's what I use: > > :set prompt "\ESC[46m\STX%s>\ESC[39;49m\STX " > > I believe \STX is a signal to haskeline for control sequences. > Documentation is here: > https://github.com/judah/haskeline/wiki/ControlSequencesInPrompt > > On Tue, Dec 5, 2017 at 12:57 PM, Brandon Allbery > wrote: > > On Tue, Dec 5, 2017 at 12:36 PM, cheater00 cheater00 < > cheater00 at gmail.com> > > wrote: > >> > >> without color coding the prompt so I can't really turn it off. It > >> seems like a simple arithmetic issue somewhere in the readline > >> implementation. > > > > > > It's not arithmetic except in the sense that it's not doing *any* math. > > Color codes in a terminal are necessarily implemented as character > sequences > > (this is pretty much the definition of a terminal interface), and > haskeline > > makes no effort to recognize them, so it treats them the same as > displayed > > character sequences and skips over them as if they were displayed > > characters. > > > > GNU readline handles this by recognizing the character mode sequences as > not > > taking up character positions (this is more complex than you think given > > that GNU readline doesn't assume all terminals obey the ANSI standard; > as it > > turns out, neither does haskeline, so it actually gets a bit nasty), and > > recognizing the special behavior of carriage return, and providing \[ \] > > escapes to declare the sequence inside as "invisible" to to character > > positioning (and it's on the person crafting the prompt to insure that it > > actually is). Beyond that, it'd actually have to implement a 'terminal > > emulator' internally to get it right in all cases --- and i'd be on the > user > > to ensure their declared terminal type matches the actual one well enough > > for the 'terminal emulator' to reflect the terminal's actual behavior, so > > it'd be a potential source of even weirder behavior. > > > > So, (a) haskeline issue, not strictly ghc/ghci, and (b) not really > fixable, > > but partially work-around-able for common cases. > > > > -- > > brandon s allbery kf8nh sine nomine > associates > > allbery.b at gmail.com > ballbery at sinenomine.net > > unix, openafs, kerberos, infrastructure, xmonad > http://sinenomine.net > > > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -- brandon s allbery kf8nh sine nomine associates allbery.b at gmail.com ballbery at sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From cheater00 at gmail.com Tue Dec 5 22:20:49 2017 From: cheater00 at gmail.com (cheater00 cheater00) Date: Tue, 05 Dec 2017 22:20:49 +0000 Subject: Long standing annoying issue in ghci In-Reply-To: References: <5a271a07.b29adf0a.8175a.6438SMTPIN_ADDED_MISSING@mx.google.com> Message-ID: A simple example of colored prompt would be good enough, so as not to make the docs too huge. People will be able to figure out they need \STX, or just make sure to tell them. On Tue, 5 Dec 2017 23:17 Brandon Allbery, wrote: > It's indirectly already there: > https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ghci.html#the-ghci-and-haskeline-files > Possibly an additional pointer needs to be in the FAQ/things to watch out > for section. > > On Tue, Dec 5, 2017 at 5:13 PM, David Feuer wrote: > >> It sounds like this should be added to the GHCi documentation, even if >> it's not strictly about GHCi. >> >> >> >> David Feuer >> Well-Typed, LLP >> >> -------- Original message -------- >> From: Evan Laforge >> Date: 12/5/17 4:49 PM (GMT-05:00) >> To: Brandon Allbery >> Cc: ghc-devs at haskell.org >> Subject: Re: Long standing annoying issue in ghci >> >> Here's what I use: >> >> :set prompt "\ESC[46m\STX%s>\ESC[39;49m\STX " >> >> I believe \STX is a signal to haskeline for control sequences. >> Documentation is here: >> https://github.com/judah/haskeline/wiki/ControlSequencesInPrompt >> >> On Tue, Dec 5, 2017 at 12:57 PM, Brandon Allbery >> wrote: >> > On Tue, Dec 5, 2017 at 12:36 PM, cheater00 cheater00 < >> cheater00 at gmail.com> >> > wrote: >> >> >> >> without color coding the prompt so I can't really turn it off. It >> >> seems like a simple arithmetic issue somewhere in the readline >> >> implementation. >> > >> > >> > It's not arithmetic except in the sense that it's not doing *any* math. >> > Color codes in a terminal are necessarily implemented as character >> sequences >> > (this is pretty much the definition of a terminal interface), and >> haskeline >> > makes no effort to recognize them, so it treats them the same as >> displayed >> > character sequences and skips over them as if they were displayed >> > characters. >> > >> > GNU readline handles this by recognizing the character mode sequences >> as not >> > taking up character positions (this is more complex than you think given >> > that GNU readline doesn't assume all terminals obey the ANSI standard; >> as it >> > turns out, neither does haskeline, so it actually gets a bit nasty), and >> > recognizing the special behavior of carriage return, and providing \[ \] >> > escapes to declare the sequence inside as "invisible" to to character >> > positioning (and it's on the person crafting the prompt to insure that >> it >> > actually is). Beyond that, it'd actually have to implement a 'terminal >> > emulator' internally to get it right in all cases --- and i'd be on the >> user >> > to ensure their declared terminal type matches the actual one well >> enough >> > for the 'terminal emulator' to reflect the terminal's actual behavior, >> so >> > it'd be a potential source of even weirder behavior. >> > >> > So, (a) haskeline issue, not strictly ghc/ghci, and (b) not really >> fixable, >> > but partially work-around-able for common cases. >> > >> > -- >> > brandon s allbery kf8nh sine nomine >> associates >> > allbery.b at gmail.com >> ballbery at sinenomine.net >> > unix, openafs, kerberos, infrastructure, xmonad >> http://sinenomine.net >> > >> > _______________________________________________ >> > ghc-devs mailing list >> > ghc-devs at haskell.org >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > > > > -- > brandon s allbery kf8nh sine nomine > associates > allbery.b at gmail.com > ballbery at sinenomine.net > unix, openafs, kerberos, infrastructure, xmonad > http://sinenomine.net > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Thu Dec 7 17:32:02 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Thu, 7 Dec 2017 17:32:02 +0000 Subject: Can't push to haddock Message-ID: I'm trying to push a patch that needs a supporting change to haddock. I've pushed the haddock change to the ghc-head branch of ssh://git at github.com/haskell/haddock.git, which is (according to 'packages') the relevant haddock upstream repo. But when I try to push the GHC patch, I get this message bash$ git push Counting objects: 45, done. Delta compression using up to 32 threads. Compressing objects: 100% (45/45), done. Writing objects: 100% (45/45), 27.56 KiB | 0 bytes/s, done. Total 45 (delta 43), reused 0 (delta 0) remote: performing commit message validations... remote: Commit message validation passed! remote: performing submodule-ref update validations... remote: Submodule update(s) detected in fa29df02a1b0b926afb2525a258172dcbf0ea460: remote: utils/haddock => 24841386cff6fdccc11accf9daa815c2c7444d65 remote: *FAIL* commit not found in submodule repo ('../haddock.git') remote: or not reachable from persistent branches remote: hooklet hooks/update.secondary.d/check-submodule-refs failed remote: hooks/update.secondary died remote: error: hook declined to update refs/heads/master To ssh://git at git.haskell.org/ghc.git ! [remote rejected] HEAD -> master (hook declined) error: failed to push some refs to 'ssh://git at git.haskell.org/ghc.git' simonpj at cam-05-unx:~/code/HEAD$ What's up? I have pushed the haddock commit! THanks Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Thu Dec 7 17:53:05 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Thu, 7 Dec 2017 17:53:05 +0000 Subject: Can't push to haddock In-Reply-To: References: Message-ID: But when I try to push the GHC patch, I get this message Ah... it worked after a while. Maybe a mirroring thing? But in pushing to GHC I saw: git push Counting objects: 45, done. Delta compression using up to 32 threads. Compressing objects: 100% (45/45), done. Writing objects: 100% (45/45), 27.56 KiB | 0 bytes/s, done. Total 45 (delta 43), reused 0 (delta 0) remote: performing commit message validations... remote: Commit message validation passed! remote: performing submodule-ref update validations... remote: Submodule update(s) detected in fa29df02a1b0b926afb2525a258172dcbf0ea460: remote: utils/haddock => 24841386cff6fdccc11accf9daa815c2c7444d65 remote: utils/hsc2hs => 9483ad10064fbbb97ab525280623826b1ef63959 remote: OK remote: performing whitespace validations... remote: whitespace validation passed! remote: mirroring ssh://git at git.haskell.org/ghc to ssh://git at github.com/ghc/ghc ... remote: To ssh://git at github.com/ghc/ghc remote: 5f332e1..fa29df0 master -> master remote: running notifier To ssh://git at git.haskell.org/ghc.git 5f332e1..fa29df0 HEAD -> master simonpj at cam-05-unx:~/code/HEAD$ I did not intend to monkey around with hsc2hs. I can't think how that happened, or whether it matter. With many apologies, would a wiser person that me like to see if I've accidentally messed up hsc2hs. Thanks Simon From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of Simon Peyton Jones via ghc-devs Sent: 07 December 2017 17:32 To: ghc-devs at haskell.org Subject: Can't push to haddock I'm trying to push a patch that needs a supporting change to haddock. I've pushed the haddock change to the ghc-head branch of ssh://git at github.com/haskell/haddock.git, which is (according to 'packages') the relevant haddock upstream repo. But when I try to push the GHC patch, I get this message bash$ git push Counting objects: 45, done. Delta compression using up to 32 threads. Compressing objects: 100% (45/45), done. Writing objects: 100% (45/45), 27.56 KiB | 0 bytes/s, done. Total 45 (delta 43), reused 0 (delta 0) remote: performing commit message validations... remote: Commit message validation passed! remote: performing submodule-ref update validations... remote: Submodule update(s) detected in fa29df02a1b0b926afb2525a258172dcbf0ea460: remote: utils/haddock => 24841386cff6fdccc11accf9daa815c2c7444d65 remote: *FAIL* commit not found in submodule repo ('../haddock.git') remote: or not reachable from persistent branches remote: hooklet hooks/update.secondary.d/check-submodule-refs failed remote: hooks/update.secondary died remote: error: hook declined to update refs/heads/master To ssh://git at git.haskell.org/ghc.git ! [remote rejected] HEAD -> master (hook declined) error: failed to push some refs to 'ssh://git at git.haskell.org/ghc.git' simonpj at cam-05-unx:~/code/HEAD$ What's up? I have pushed the haddock commit! THanks Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From monkleyon at gmail.com Thu Dec 7 22:08:46 2017 From: monkleyon at gmail.com (MarLinn) Date: Thu, 7 Dec 2017 23:08:46 +0100 Subject: Long standing annoying issue in ghci In-Reply-To: References:

Message-ID: <814bb618-30cc-6024-28b0-17c7c9f95a45@gmail.com> > Here's what I use: > > :set prompt "\ESC[46m\STX%s>\ESC[39;49m\STX " > > I believe \STX is a signal to haskeline for control sequences. > Documentation is here: > https://github.com/judah/haskeline/wiki/ControlSequencesInPrompt Note: If you're using a multi-line prompt, things may be different again. I don't know what the rules are, but I found that if I put \STX on any but the last line of prompts I get weird characters. The same goes for any \SOH you might want to add for some reason. Cheers, MarLinn From cheater00 at gmail.com Thu Dec 7 22:15:12 2017 From: cheater00 at gmail.com (cheater00 cheater00) Date: Thu, 07 Dec 2017 22:15:12 +0000 Subject: Long standing annoying issue in ghci In-Reply-To: <814bb618-30cc-6024-28b0-17c7c9f95a45@gmail.com> References:

<814bb618-30cc-6024-28b0-17c7c9f95a45@gmail.com> Message-ID: Interesting. Would you mind reopening the issue and providing a buggy example? Amd alerting haskeline maintainers? How does it work on a 1 line prompt that is so long it wraps? On Thu, 7 Dec 2017 23:11 MarLinn, wrote: > > > Here's what I use: > > > > :set prompt "\ESC[46m\STX%s>\ESC[39;49m\STX " > > > > I believe \STX is a signal to haskeline for control sequences. > > Documentation is here: > > https://github.com/judah/haskeline/wiki/ControlSequencesInPrompt > Note: If you're using a multi-line prompt, things may be different > again. I don't know what the rules are, but I found that if I put \STX > on any but the last line of prompts I get weird characters. The same > goes for any \SOH you might want to add for some reason. > > Cheers, > MarLinn > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Fri Dec 8 09:12:48 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Fri, 8 Dec 2017 09:12:48 +0000 Subject: Can't push to haddock In-Reply-To: References: Message-ID: | Yes, the mirroring has a little bit of latency (assuming the mirroring | trigger event notification from github to git.haskell.org didn't get | lost). How much time did you wait between pushing to github and | ghc.git? I didn't allow any time -- I didn't know that time was needed. Perhaps we should add a note to https://ghc.haskell.org/trac/ghc/wiki/Repositories to explain? Under "Updating sub-repos" perhaps. I wonder if it'd be worth us articulating the reason why some submodules live in github, but some live in git.haskell.org -- with only mirroring github. I'm sure there's a rationale but I don't get it yet. Simon | -----Original Message----- | From: Herbert Valerio Riedel [mailto:hvriedel at gmail.com] | Sent: 07 December 2017 17:57 | To: Simon Peyton Jones | Subject: Re: Can't push to haddock | | Hi Simon, | | Yes, the mirroring has a little bit of latency (assuming the mirroring | trigger event notification from github to git.haskell.org didn't get | lost). How much time did you wait between pushing to github and | ghc.git? | | On Thu, Dec 7, 2017 at 6:53 PM, Simon Peyton Jones via ghc-devs wrote: | > But when I try to push the GHC patch, I get this message | > | > Ah… it worked after a while. Maybe a mirroring thing? | > | > But in pushing to GHC I saw: | > | > git push | > | > Counting objects: 45, done. | > | > Delta compression using up to 32 threads. | > | > Compressing objects: 100% (45/45), done. | > | > Writing objects: 100% (45/45), 27.56 KiB | 0 bytes/s, done. | > | > Total 45 (delta 43), reused 0 (delta 0) | > | > remote: performing commit message validations... | > | > remote: Commit message validation passed! | > | > remote: performing submodule-ref update validations... | > | > remote: Submodule update(s) detected in | > fa29df02a1b0b926afb2525a258172dcbf0ea460: | > | > remote: utils/haddock => 24841386cff6fdccc11accf9daa815c2c7444d65 | > | > remote: utils/hsc2hs => 9483ad10064fbbb97ab525280623826b1ef63959 | > | > remote: OK | > | > remote: performing whitespace validations... | > | > remote: whitespace validation passed! | > | > remote: mirroring ssh://git at git.haskell.org/ghc to | > ssh://git at github.com/ghc/ghc ... | > | > remote: To ssh://git at github.com/ghc/ghc | > | > remote: 5f332e1..fa29df0 master -> master | > | > remote: running notifier | > | > To ssh://git at git.haskell.org/ghc.git | > | > 5f332e1..fa29df0 HEAD -> master | > | > simonpj at cam-05-unx:~/code/HEAD$ | > | > I did not intend to monkey around with hsc2hs. I can’t think how | that | > happened, or whether it matter. | > | > With many apologies, would a wiser person that me like to see if | I’ve | > accidentally messed up hsc2hs. | > | > Thanks | > | > Simon | > | > | > | > From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of | > Simon Peyton Jones via ghc-devs | > Sent: 07 December 2017 17:32 | > To: ghc-devs at haskell.org | > Subject: Can't push to haddock | > | > | > | > I’m trying to push a patch that needs a supporting change to | haddock. | > | > I’ve pushed the haddock change to the ghc-head branch of | > ssh://git at github.com/haskell/haddock.git, which is (according to | > ‘packages’) the relevant haddock upstream repo. | > | > But when I try to push the GHC patch, I get this message | > | > bash$ git push | > | > Counting objects: 45, done. | > | > Delta compression using up to 32 threads. | > | > Compressing objects: 100% (45/45), done. | > | > Writing objects: 100% (45/45), 27.56 KiB | 0 bytes/s, done. | > | > Total 45 (delta 43), reused 0 (delta 0) | > | > remote: performing commit message validations... | > | > remote: Commit message validation passed! | > | > remote: performing submodule-ref update validations... | > | > remote: Submodule update(s) detected in | > fa29df02a1b0b926afb2525a258172dcbf0ea460: | > | > remote: utils/haddock => 24841386cff6fdccc11accf9daa815c2c7444d65 | > | > remote: *FAIL* commit not found in submodule repo ('../haddock.git') | > | > remote: or not reachable from persistent branches | > | > remote: hooklet hooks/update.secondary.d/check-submodule-refs failed | > | > remote: hooks/update.secondary died | > | > remote: error: hook declined to update refs/heads/master | > | > To ssh://git at git.haskell.org/ghc.git | > | > ! [remote rejected] HEAD -> master (hook declined) | > | > error: failed to push some refs to | 'ssh://git at git.haskell.org/ghc.git' | > | > simonpj at cam-05-unx:~/code/HEAD$ | > | > | > | > What’s up? I have pushed the haddock commit! | > | > THanks | > | > Simon | > | > | > | > | > _______________________________________________ | > ghc-devs mailing list | > ghc-devs at haskell.org | > | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h | > askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- | devs&data=02%7C01%7Csimonpj%40microsoft.com%7C684b5c6cdac34213317708d5 | 3d9be387%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6364826622509770 | 90&sdata=kMhG2iTALLRxhwyDw%2BzTN8VvMMn%2FqfvnSn9cPm0AK4Q%3D&reserved=0 | > From simonpj at microsoft.com Fri Dec 8 10:24:32 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Fri, 8 Dec 2017 10:24:32 +0000 Subject: unix/tests/user001 Message-ID: On Linux I get this validate failure all the time. What should I do? I've trained myself to ignore it, but that seems wrong Simon Unexpected failures: ../libraries/unix/tests/user001.run user001 [bad stdout] (normal) =====> user001(normal) 1 of 1 [0, 0, 0] cd "./user001.run" && "/5playpen/simonpj/HEAD/inplace/test spaces/ghc-stage2" -o user001 user001.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -dno-debug-output -package unix cd "./user001.run" && ./user001 Actual stdout output differs from expected: diff -uw "./user001.run/user001.stdout.normalised" "./user001.run/user001.run.stdout.normalised" --- ./user001.run/user001.stdout.normalised 2017-12-08 10:22:59.835931843 +0000 +++ ./user001.run/user001.run.stdout.normalised 2017-12-08 10:22:59.835931843 +0000 @@ -4,8 +4,8 @@ getEffectiveGroupID: OK getGroups: OK getEffectiveUserName: OK -getGroupEntryForID: OK -getGroupEntryForName: OK +getGroupEntryForID: ERROR: getGroupEntryForID: does not exist (no such group) +getGroupEntryForName: ERROR: getGroupEntryForID: does not exist (no such group) getAllGroupEntries: OK getUserEntryForID: OK getAllUserEntries: OK *** unexpected failure for user001(normal) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Fri Dec 8 15:13:25 2017 From: ben at well-typed.com (Ben Gamari) Date: Fri, 08 Dec 2017 10:13:25 -0500 Subject: unix/tests/user001 In-Reply-To: References: Message-ID: <87374ll0bx.fsf@ben-laptop.smart-cactus.org> Simon Peyton Jones via ghc-devs writes: > On Linux I get this validate failure all the time. What should I do? I've trained myself to ignore it, but that seems wrong > Simon > > Unexpected failures: > > ../libraries/unix/tests/user001.run user001 [bad stdout] (normal) > It sounds like your machine may be misconfigured. The test attempts to lookup the entry in /etc/group corresponding to the process's group. However, it appears that such an entry doesn't exist. This would imply that your user doesn't have a valid group, which is quite odd. What does the `id` command say? Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From simonpj at microsoft.com Fri Dec 8 15:21:33 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Fri, 8 Dec 2017 15:21:33 +0000 Subject: unix/tests/user001 In-Reply-To: <87374ll0bx.fsf@ben-laptop.smart-cactus.org> References: <87374ll0bx.fsf@ben-laptop.smart-cactus.org> Message-ID: | that your user doesn't have a valid group, which is quite odd. What | does the `id` command say? simonpj at cam-05-unx:~/tmp$ id uid=501(simonpj) gid=1001 groups=1001,27(sudo) | -----Original Message----- | From: Ben Gamari [mailto:ben at well-typed.com] | Sent: 08 December 2017 15:13 | To: Simon Peyton Jones ; ghc-devs at haskell.org | Subject: Re: unix/tests/user001 | | Simon Peyton Jones via ghc-devs writes: | | > On Linux I get this validate failure all the time. What should I | do? | > I've trained myself to ignore it, but that seems wrong Simon | > | > Unexpected failures: | > | > ../libraries/unix/tests/user001.run user001 [bad stdout] | (normal) | > | It sounds like your machine may be misconfigured. The test attempts to | lookup the entry in /etc/group corresponding to the process's group. | However, it appears that such an entry doesn't exist. This would imply | that your user doesn't have a valid group, which is quite odd. What | does the `id` command say? | | Cheers, | | - Ben From ben at well-typed.com Fri Dec 8 16:46:19 2017 From: ben at well-typed.com (Ben Gamari) Date: Fri, 08 Dec 2017 11:46:19 -0500 Subject: unix/tests/user001 In-Reply-To: References: <87374ll0bx.fsf@ben-laptop.smart-cactus.org> Message-ID: <87zi6tjhgp.fsf@ben-laptop.smart-cactus.org> Simon Peyton Jones via ghc-devs writes: > | that your user doesn't have a valid group, which is quite odd. What > | does the `id` command say? > > simonpj at cam-05-unx:~/tmp$ id > uid=501(simonpj) gid=1001 groups=1001,27(sudo) Indeed it looks like your group doesn't have an entry in /etc/group. Quite odd. Perhaps this was due to the upgrade? Is there an entry for group id 501 in /etc/group? Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From monkleyon at gmail.com Fri Dec 8 17:24:23 2017 From: monkleyon at gmail.com (MarLinn) Date: Fri, 8 Dec 2017 18:24:23 +0100 Subject: Long standing annoying issue in ghci In-Reply-To: References:

<814bb618-30cc-6024-28b0-17c7c9f95a45@gmail.com> Message-ID: <572bbe0d-1c65-19ef-bbb1-473d395fc46e@gmail.com> I opened an issue on the Haskeline github (https://github.com/judah/haskeline/issues/72). But it seems to be completely Haskeline-side, so I'm not sure if it's worth re-opening the one for ghci? As missing documentation maybe? (BTW, I found this on the wiki: https://wiki.haskell.org/GHCi_in_colour. Might be a good place to put it, if linked.) If you want to, here are my test cases rewritten as ghci prompts: -- single line, positioning error :set prompt " \ESC[36m%\ESC[0m " -- single line, works :set prompt " \ESC[36m\STX%\ESC[0m\STX " -- multiline, bad output :set prompt "\ESC[32m\STX–––\ESC[0m\STX\n \ESC[36m\STX%\ESC[0m\STX " -- multiline, works but is inconsistent :set prompt "\ESC[32m–––\ESC[0m\n \ESC[36m\STX%\ESC[0m\STX " In my tests, the positioning errors consistently happen if there are any "unclosed" escape-sequences on the last line of the prompt, regardless of its length. Escape sequences on previous lines consistently create "weird characters", but don't influence the positioning. Also regardless of their lengths. That makes sense, as both sets of lines seem to be handled quite differently. Are multiline prompts even used by a lot of people? I like mine because it gives me a both a list of modules and a consistent cursor position. But maybe I'm the exception? Cheers. On 2017-12-07 23:15, cheater00 cheater00 wrote: > > Interesting. Would you mind reopening the issue and providing a buggy > example? Amd alerting haskeline maintainers? How does it work on a 1 > line prompt that is so long it wraps? > > > On Thu, 7 Dec 2017 23:11 MarLinn, > wrote: > > > > Here's what I use: > > > > :set prompt "\ESC[46m\STX%s>\ESC[39;49m\STX " > > > > I believe \STX is a signal to haskeline for control sequences. > > Documentation is here: > > https://github.com/judah/haskeline/wiki/ControlSequencesInPrompt > Note: If you're using a multi-line prompt, things may be different > again. I don't know what the rules are, but I found that if I put \STX > on any but the last line of prompts I get weird characters. The same > goes for any \SOH you might want to add for some reason. > > Cheers, > MarLinn > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Fri Dec 8 18:50:29 2017 From: ben at well-typed.com (Ben Gamari) Date: Fri, 08 Dec 2017 13:50:29 -0500 Subject: Moving hadrian to submodule Message-ID: <87o9n9jbpr.fsf@ben-laptop.smart-cactus.org> Hello everyone, A bit over a month ago we merged hadrian into the ghc tree as a subtree. Unfortunately, those working on Hadrian have found the subtree mechanism to provide a rather poor developer experience. Consequently, today I will be ripping out the subtree and replacing it with a submodule. After pulling the commit performing this change you will likely need to do the following to emplace the new submodule, $ git submodule update --init $ git -C hadrian checkout . Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From me at ara.io Fri Dec 8 19:10:09 2017 From: me at ara.io (Ara Adkins) Date: Fri, 8 Dec 2017 19:10:09 +0000 Subject: Moving hadrian to submodule In-Reply-To: <87o9n9jbpr.fsf@ben-laptop.smart-cactus.org> References: <87o9n9jbpr.fsf@ben-laptop.smart-cactus.org> Message-ID: <5E391C39-1C3E-4560-90DC-4B7EAD17D4CC@ara.io> Sounds good! Hopefully this doesn’t cause a flood of commit messages. _ara > On 8 Dec 2017, at 18:50, Ben Gamari wrote: > > Hello everyone, > > A bit over a month ago we merged hadrian into the ghc tree as a subtree. > Unfortunately, those working on Hadrian have found the subtree mechanism > to provide a rather poor developer experience. Consequently, today I > will be ripping out the subtree and replacing it with a submodule. > > After pulling the commit performing this change you will likely need to > do the following to emplace the new submodule, > > $ git submodule update --init > $ git -C hadrian checkout . > > Cheers, > > - Ben > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From m at tweag.io Fri Dec 8 19:25:51 2017 From: m at tweag.io (Boespflug, Mathieu) Date: Fri, 8 Dec 2017 20:25:51 +0100 Subject: Moving hadrian to submodule In-Reply-To: <5E391C39-1C3E-4560-90DC-4B7EAD17D4CC@ara.io> References: <87o9n9jbpr.fsf@ben-laptop.smart-cactus.org> <5E391C39-1C3E-4560-90DC-4B7EAD17D4CC@ara.io> Message-ID: Great! A submodule makes a lot more sense to me, at least short term. I would hope that medium term the development processes of GHC-as-a-whole and Hadrian(-a-part-of-GHC) can converge, so that a submodule is no longer necessary. Submodules do have their downsides and it would be odd for such a core part as the build system to be kept external to the main repo. -- Mathieu Boespflug Founder at http://tweag.io. On 8 December 2017 at 20:10, Ara Adkins wrote: > Sounds good! Hopefully this doesn’t cause a flood of commit messages. > > _ara > > > On 8 Dec 2017, at 18:50, Ben Gamari wrote: > > > > Hello everyone, > > > > A bit over a month ago we merged hadrian into the ghc tree as a subtree. > > Unfortunately, those working on Hadrian have found the subtree mechanism > > to provide a rather poor developer experience. Consequently, today I > > will be ripping out the subtree and replacing it with a submodule. > > > > After pulling the commit performing this change you will likely need to > > do the following to emplace the new submodule, > > > > $ git submodule update --init > > $ git -C hadrian checkout . > > > > Cheers, > > > > - Ben > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qdunkan at gmail.com Mon Dec 11 20:08:00 2017 From: qdunkan at gmail.com (Evan Laforge) Date: Mon, 11 Dec 2017 12:08:00 -0800 Subject: GHCi recompilation avoidance UI In-Reply-To: References: <7587879.l535S4SiGz@squirrel> <5110217.Pvi1NnV8H5@squirrel> Message-ID: https://ghc.haskell.org/trac/ghc/ticket/13604 was just closed, I look forward to trying it out. Thanks to dfeuer for the fix! On Wed, Nov 22, 2017 at 12:41 AM, Simon Marlow wrote: > David, > > Perhaps it would be good to defer changing the behaviour of :load *M (I > believe you that it's hard, that code is quite convoluted) and for now just > focus on making GHCi able to load compiled object code again, which I think > is a much simpler problem? > > Cheers > Simon > > > On 21 November 2017 at 21:49, David Feuer wrote: >> >> I started digging back into this today, particularly considering Simon >> PJ's view >> that it's a bit odd for optimization flags to imply -fobject-code >> (specifically >> because we could potentially support optimization for the bytecode >> interpreter some day). I'm left even more lost about exactly what we want. >> I believe it's fairly clear that, as Simon M wrote, >> >> > [W]e'll want at least -fignore-optim-changes to be the default, so that >> > GHCi >> > does the expected thing when you have compiled object files. >> >> Based on Simon PJ's comment, I believe we want to *continue* to discard >> optimization flags when -fobject-code is not enabled. As for my suggestion >> in (2), >> I spent the last couple hours attempting to figure out what would be >> necessary >> to allow :load *M to load a module interpreted even when using >> -fobject-code, >> but found myself utterly lost in the module loading logic. I see that the >> IIModule >> constructor is deeply involved in this, but I haven't been able to figure >> out >> where/how that interacts with -fobject-code to determine whether the >> module >> will actually be loaded interpreted or compiled. Can someone give me a >> clue? >> >> On Thursday, November 2, 2017 10:21:07 AM EST Simon Marlow wrote: >> > On 31 October 2017 at 15:42, David Feuer wrote: >> > >> > > Changes in GHC 8.2.1 lead to a lot of recompilation, because GHCi now >> > > refuses to load optimized >> > > code unless -fobject-code (and optimization flags) are enabled. I >> > > propose >> > > the following slight >> > > modification to >> > > https://ghc.haskell.org/trac/ghc/ticket/13604#comment:48 >> > > >> > > 1. Optimization flags (except -O0) imply -fobject-code. This ensures >> > > that >> > > GHC respects optimization flags regardless of --interactive. >> > > >> > > 2. Even when -fobject-code is on, :load *M will load M as bytecode. >> > > This >> > > provides the "escape hatch" from -fobject-code that you need to use >> > > debugging features, etc. >> > > >> > >> > Yes, I think this is probably what we want. I'm not sure how smooth it >> > will >> > be to implement though. >> > >> > >> > > 3. New -fignore-optim-changes and -fignore-hpc-changes (Phab:D4123) >> > > flags should enable users to put together object code and bytecode >> > > with >> > > diverse optimization levels/options and HPC options while still >> > > updating >> > > automatically based on source changes and whether profiling is >> > > enabled. >> > > >> > >> > As I mentioned on the diff, I think we'll want at least >> > -fignore-optim-changes to be the default, so that GHCi does the expected >> > thing when you have compiled object files. >> > >> > Cheers >> > Simon > > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > From cheater00 at gmail.com Tue Dec 12 19:44:56 2017 From: cheater00 at gmail.com (cheater00 cheater00) Date: Tue, 12 Dec 2017 19:44:56 +0000 Subject: Long standing annoying issue in ghci In-Reply-To: <572bbe0d-1c65-19ef-bbb1-473d395fc46e@gmail.com> References:

<814bb618-30cc-6024-28b0-17c7c9f95a45@gmail.com> <572bbe0d-1c65-19ef-bbb1-473d395fc46e@gmail.com> Message-ID: Yes, it is worth doing it, because until Haskeline has been fixed and integrated into ghci, the issue persists and needs to remain filed. On Fri, 8 Dec 2017 18:25 MarLinn, wrote: > I opened an issue on the Haskeline github ( > https://github.com/judah/haskeline/issues/72). > > But it seems to be completely Haskeline-side, so I'm not sure if it's > worth re-opening the one for ghci? As missing documentation maybe? > (BTW, I found this on the wiki: https://wiki.haskell.org/GHCi_in_colour. > Might be a good place to put it, if linked.) > > If you want to, here are my test cases rewritten as ghci prompts: > > -- single line, positioning error > :set prompt " \ESC[36m%\ESC[0m " > -- single line, works > :set prompt " \ESC[36m\STX%\ESC[0m\STX " > -- multiline, bad output > :set prompt "\ESC[32m\STX–––\ESC[0m\STX\n \ESC[36m\STX%\ESC[0m\STX " > -- multiline, works but is inconsistent > :set prompt "\ESC[32m–––\ESC[0m\n \ESC[36m\STX%\ESC[0m\STX " > > In my tests, the positioning errors consistently happen if there are any > "unclosed" escape-sequences on the last line of the prompt, regardless of > its length. Escape sequences on previous lines consistently create "weird > characters", but don't influence the positioning. Also regardless of their > lengths. That makes sense, as both sets of lines seem to be handled quite > differently. > > Are multiline prompts even used by a lot of people? I like mine because it > gives me a both a list of modules and a consistent cursor position. But > maybe I'm the exception? > > Cheers. > > > On 2017-12-07 23:15, cheater00 cheater00 wrote: > > Interesting. Would you mind reopening the issue and providing a buggy > example? Amd alerting haskeline maintainers? How does it work on a 1 line > prompt that is so long it wraps? > > On Thu, 7 Dec 2017 23:11 MarLinn, wrote: > >> >> > Here's what I use: >> > >> > :set prompt "\ESC[46m\STX%s>\ESC[39;49m\STX " >> > >> > I believe \STX is a signal to haskeline for control sequences. >> > Documentation is here: >> > https://github.com/judah/haskeline/wiki/ControlSequencesInPrompt >> Note: If you're using a multi-line prompt, things may be different >> again. I don't know what the rules are, but I found that if I put \STX >> on any but the last line of prompts I get weird characters. The same >> goes for any \SOH you might want to add for some reason. >> >> Cheers, >> MarLinn >> >> > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Wed Dec 13 16:43:55 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Wed, 13 Dec 2017 16:43:55 +0000 Subject: Release policies Message-ID: Dear GHC devops group The conversation on Trac #14558 suggests that we might want to consider reviewing GHC's release policies. This email is to invite your input. The broad questions is this. We want GHC to serve the needs of all its users, including downstream tooling that uses GHC. What release policies will best support that goal? For example, we already ensure that GHC 8.4 can be compiled with 8.2 and 8.0. This imposes a slight tax on GHC development, but it means that users don't need to upgrade quite as often. (If the tempo of releases increases, we might want to increase the window.) Trac #14558 suggests that we might want to ensure the metadata on GHC's built-in libraries is parsable with older Cabals. One possibility would be this: * Ensure that the Cabal metadata of non-reinstallable packages (e.g. integer-gmp) shipped with GHC be parsable by the Cabal versions shipped with the last two major GHC releases [i.e. have a sufficiently old cabal-version field]. That is, in general a new Cabal specification will need to be shipped with two GHC releases before GHC will use start using its features in non-reinstallable packages. * Upholding this policy won't always be possible. There may be cases (as is the case Hadrian for GHC 8.4) where the benefit of quickly introducing incompatible syntax outweighs the need for compatibility. In this (hopefully rare) case we would explicitly advertise the incompatibility in the release documentation, and give as much notice as possible to users to allow downstream tools to adapt. * For reinstallable packages, of which GHC is simply a client (like text or bytestring), we can't reasonably enforce such a policy, because GHC devs have no control over what the maintainers of external core libraries put in their Cabal files. This is just a proposal. The narrow questions are these: * Would this be sufficient to deal with the concerns raised in #14558? * Is it necessary, ow would anything simpler be sufficient? * What costs would the policy impose on GHC development? * There may be matters of detail: e.g. is two releases the right grace period. Would one do? Both the broad question and the narrow ones are appropriate for the Devops group. Thanks! Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From lonetiger at gmail.com Wed Dec 13 16:46:52 2017 From: lonetiger at gmail.com (Phyx) Date: Wed, 13 Dec 2017 16:46:52 +0000 Subject: [GHC] #14537: Do not link to msvcrt.dll In-Reply-To: <061.3987f9ac05db4d2e2767cca51dacc31b@haskell.org> References: <046.1c327e3acbc706888aef5a746e2bc1a4@haskell.org> <061.3987f9ac05db4d2e2767cca51dacc31b@haskell.org> Message-ID: Add a -v3 to ghc to see what's happening. On Wed, Dec 13, 2017, 07:23 GHC wrote: > #14537: Do not link to msvcrt.dll > -------------------------------------+------------------------------------- > Reporter: johndoe | Owner: (none) > Type: feature request | Status: closed > Priority: normal | Milestone: > Component: Compiler | Version: 8.2.1 > (Linking) | > Resolution: invalid | Keywords: > Operating System: Windows | Architecture: > | Unknown/Multiple > Type of failure: Other | Test Case: > Blocked By: | Blocking: > Related Tickets: | Differential Rev(s): > Wiki Page: | > -------------------------------------+------------------------------------- > > Comment (by johndoe): > > Well, I tried to modify `specs` directly, but project failed to build: > I've got messagebox that `setup.exe` crashed. > That was on my local machine, but the same you can see on appveyor: > https://ci.appveyor.com/project/johnd0e/pandoc/build/1.0.79 > > I am not sure, maybe my `specs` file is not good: > https://github.com/johnd0e/pandoc/blob/master/specs > But it worked well with other project (not GHC). > > -- > Ticket URL: > GHC > The Glasgow Haskell Compiler > -------------- next part -------------- An HTML attachment was scrubbed... URL: From m at tweag.io Wed Dec 13 22:12:42 2017 From: m at tweag.io (Boespflug, Mathieu) Date: Wed, 13 Dec 2017 23:12:42 +0100 Subject: Fwd: Release policies In-Reply-To: References: Message-ID: ---------- Forwarded message ---------- From: Boespflug, Mathieu Date: 13 December 2017 at 23:03 Subject: Re: Release policies To: Simon Peyton Jones Cc: ghc-devops-group at haskell.org [replying to ghc-devops-group@, which I assume based on your email's content is the mailing list you intended.] Hi Simon, feedback from downstream consumers of Cabal metadata (e.g. build tool authors) will be particularly useful for the discussion here. Here are my thoughts as a bystander. It's worth trying to identify what problems came up during the integer-gmp incident in Trac #14558: * GHC 8.2.1 shipped with integer-gmp-1.0.1.0 but the release notes said otherwise. * GHC 8.2.1 shipped with Cabal-2.0.0.2, but specifically claimed in the release notes that cabal-install-1.24 (and by implication any other build tool based on Cabal-the-library version 1.24) was supported: "GHC 8.2 only works with cabal-install version 1.24 or later. Please upgrade if you have an older version of cabal-install." * GHC 8.2.2 also claimed Cabal-1.24 support. * GHC 8.2.1 was released in July 2017 with Cabal-2.0.0.2, a brand new major release with breaking changes to the metadata format, without much lead time for downstream tooling authors (like Stack) to adapt. * But actually if we look at their respective release notes, GHC 8.2.1 was relased in July 2017, even though the Cabal website claims that Cabal-2.0.0.2 was released in August 2017 (see https://www.haskell.org/cabal/download.html). So it looks like GHC didn't just not give enough lead time about an upstream dependency it shipped with, it shipped with an unreleased version of Cabal! * Libraries that ship with GHC are usually also uploaded to Hackage, to make the documentation easily accessible, but integer-gmp-1.0.1.0 was not uploaded to Hackage until 4 months after the release. * The metadata for integer-gmp-1.0.1.0 as uploaded to Hackage differed from the metadata that was actually in the source tarball of GHC-8.2.1 and GHC-8.2.2. * The metadata for integer-gmp-1.0.1.0 as uploaded to Hackage included Cabal-2.0 specific syntactic sugar, making the metadata unreadable using any tooling that did not link against the Cabal-2.0.0.2 library (or any later version). * It so happened that one particular version of one particular downstream build tool, Stack, had a bug, compounding the bad effects of the previous point. But a new release has now been made, and in any case that's not a problem for GHC to solve. So let's keep that out of the discussion here. So I suggest we discuss ways to eliminate or reduce the likelihood of any of the above problems from occurring again. Here are some ideas: * GHC should never under any circumstance ship with an unreleased version of any independently maintained dependency. Cabal is one such dependency. This should hold true for anything else. We could just add that policy to the Release Policy. * Stronger still, GHC should not switch to a new major release of a dependency at any time during feature freeze ahead of a release. E.g. if Cabal-3.0.0 ships before feature freeze for GHC-9.6, then maybe it's fair game to include in GHC. But not if Cabal-3.0.0 hasn't shipped yet. * The 3-release backwards compat rule should apply in all circumstances. That means major version bumps of any library GHC ships with, including base, should not imply any breaking change in the API's of any such library. * GHC does have control over reinstallable packages (like text and bytestring): GHC need not ship with the latest versions of these, if indeed they introduce breaking changes that would contravene the 3-release policy. * Note: today, users are effectively tied to whatever version of the packages ships with GHC (i.e. the "reinstallable" bit is problematic today for various technical reasons). That's why a breaking change in bytestring is technically a breaking change in GHC. * The current release policy covers API stability, but what about metadata? In the extreme, we could say a 3-release policy applies to metadata too. Meaning, all metadata shipping with GHC now and in the next 2 releases should be parseable by today's version of Cabal and downstream tooling. Is such a long lead time necessary? That's for build tool authors to say, and a point to negotiate with GHC devs. * Because there are far fewer consumers of metadata than consumers of say base, I think shorter lead time is reasonable. At the other extreme, it could even be just the few months during feature freeze. * The release notes bugs mentioned above and the lack of consistent upload to Hackage are a symptom of lack of release automation, I suspect. That's how to fix it, but we could also spell out in the Release Policy that GHC libraries should all be on Hackage from the day of release. Finally, a question for discussion: * Hackage allows revising the metadata of an uploaded package even without changing the version number. This happens routinely on Hackage today by the Hackage trustees. Should this be permitted for packages whose release is completely tied to that of GHC itself (like integer-gmp)? Best, Mathieu On 13 December 2017 at 17:43, Simon Peyton Jones via ghc-devs < ghc-devs at haskell.org> wrote: > Dear GHC devops group > > The conversation on Trac #14558 > suggests that we might > want to consider reviewing GHC’s release policies > . > This email is to invite your input. > > The broad questions is this. We want GHC to serve the needs of all its > users, including downstream tooling that uses GHC. What release policies > will best support that goal? For example, we already ensure that GHC 8.4 > can be compiled with 8.2 and 8.0. This imposes a slight tax on GHC > development, but it means that users don't need to upgrade quite as often. > (If the tempo of releases increases, we might want to increase the window.) > > Trac #14558 suggests that we might want to ensure the metadata on GHC’s > built-in libraries is parsable with older Cabals. One possibility would be > this: > > - Ensure that the Cabal metadata of non-reinstallable packages (e.g. > integer-gmp) shipped with GHC be parsable by the Cabal versions shipped > with the last two major GHC releases [i.e. have a sufficiently old > cabal-version field]. That is, in general a new Cabal specification will > need to be shipped with two GHC releases before GHC will use start using > its features in non-reinstallable packages. > - Upholding this policy won't always be possible. There may be cases > (as is the case Hadrian for GHC 8.4) where the benefit of quickly > introducing incompatible syntax outweighs the need for compatibility. In > this (hopefully rare) case we would explicitly advertise the > incompatibility in the release documentation, and give as much notice as > possible to users to allow downstream tools to adapt. > - For reinstallable packages, of which GHC is simply a client (like > text or bytestring), we can’t reasonably enforce such a policy, because GHC > devs have no control over what the maintainers of external core libraries > put in their Cabal files. > > This is just a proposal. The narrow questions are these: > > - Would this be sufficient to deal with the concerns raised in #14558? > - Is it necessary, ow would anything simpler be sufficient? > - What costs would the policy impose on GHC development? > - There may be matters of detail: e.g. is two releases the right grace > period. Would one do? > > Both the broad question and the narrow ones are appropriate for the Devops > group. > > Thanks! > > Simon > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gershomb at gmail.com Wed Dec 13 23:19:37 2017 From: gershomb at gmail.com (Gershom B) Date: Wed, 13 Dec 2017 18:19:37 -0500 Subject: Fwd: Release policies Message-ID: Mathieu: I think the points about better tooling for documenting the correct claims in the release process are well taken. Updating the release notes manually leaves way too much room for error. However, I think you are incorrect that GHC 8.2.1 and 8.2.2 did not have cabal-install 1.24 support. They did. it works with them. I used it! The fact that the cabal files for builtin libraries use Cabal-2 syntax does not cause a problem. I think elsewhere Mikhail has corrected your timeline a bit. The general point is that Cabal-the-library and ghc releases do tend to be in tandem. But it is also the case that this happens because they are often developed in a coupled fashion. This in not really different than other builtin libs to GHC. They are necessarily coupled to changes in the compiler, and so end up being released together. Our process for making sure they are actually uploaded is absolutely error-prone as is the process for documenting what occurs in release notes. But I don't think we can simply by fiat decouple this stuff -- rather, this needs to occur on a technical level first, before other policies can really even be considered. Onto the concrete ideas: > * GHC should never under any circumstance ship with an unreleased version > of any independently maintained dependency. Cabal is one such dependency. > This should hold true for anything else. We could just add that policy to > the Release Policy. Yes, this is a good policy. Sometimes in the drive for a release, GHC will run ahead of a dependencies' upload by a bit. (Iirc this happened at one point in the past for bytestring, but this was some years ago). It is always better to hold things by a few days to get the dependency released first. > * Stronger still, GHC should not switch to a new major release of a > dependency at any time during feature freeze ahead of a release. E.g. if > Cabal-3.0.0 ships before feature freeze for GHC-9.6, then maybe it's fair > game to include in GHC. But not if Cabal-3.0.0 hasn't shipped yet. I don't think this works, in terms of coupled dependencies. If/when Cabal-3 is developed it will almost certainly be in tandem with GHC support. > * GHC does have control over reinstallable packages (like text and > bytestring): GHC need not ship with the latest versions of these, if indeed > they introduce breaking changes that would contravene the 3-release policy. Right, but this won't occur, because those libs are already under the control of the libraries committee and abide by the policy. > * Because there are far fewer consumers of metadata than consumers of say > base, I think shorter lead time is reasonable. At the other extreme, it > could even be just the few months during feature freeze. Right. There's not a sufficient quantity of metadata-consuming downstream tooling to motivate a long lead. And furthermore, the examples I can think of -- tracking dependencies for notifications or graphs -- have no need of pulling this data out of ghc-builtin libraries anyway, as they're for exploring the _userland_ portion of the package world regardless. (The other key example -- stack -- can also choose to ignore the metadata of builtin libs without harm). The general motivation of making a "feature freeze" more of a "freeze all the moving parts, really" I do agree with. Having a real freeze is part of a better release process, and it should allow all the downstream consumers of everything more time to really catch up. This is just one instance of that need. > Finally, a question for discussion: > > * Hackage allows revising the metadata of an uploaded package even without > changing the version number. This happens routinely on Hackage today by the > Hackage trustees. Should this be permitted for packages whose release is > completely tied to that of GHC itself (like integer-gmp)? It is rare that this is needed, but the ability just served us well -- editing integer-gmp to remove the new syntax was very useful, as it let us fix up old stack nightlies. Like all revisions, these should be made with some care and thought, but since we've just seen why it was helpful, I'd hate to now say we can't do it again if some other unforseen circumstance crops up. Regards, Gershom > Hi Simon, > > feedback from downstream consumers of Cabal metadata (e.g. build tool > authors) will be particularly useful for the discussion here. Here are my > thoughts as a bystander. > > It's worth trying to identify what problems came up during the integer-gmp > incident in Trac #14558: > > * GHC 8.2.1 shipped with integer-gmp-1.0.1.0 but the release notes said > otherwise. > * GHC 8.2.1 shipped with Cabal-2.0.0.2, but specifically claimed in the > release notes that cabal-install-1.24 (and by implication any other build > tool based on Cabal-the-library version 1.24) was supported: "GHC 8.2 only > works with cabal-install version 1.24 or later. Please upgrade if you have > an older version of cabal-install." > * GHC 8.2.2 also claimed Cabal-1.24 support. > * GHC 8.2.1 was released in July 2017 with Cabal-2.0.0.2, a brand new major > release with breaking changes to the metadata format, without much lead > time for downstream tooling authors (like Stack) to adapt. > * But actually if we look at their respective release notes, GHC 8.2.1 was > relased in July 2017, even though the Cabal website claims that > Cabal-2.0.0.2 was released in August 2017 (see > https://www.haskell.org/cabal/download.html). So it looks like GHC didn't > just not give enough lead time about an upstream dependency it shipped > with, it shipped with an unreleased version of Cabal! > * Libraries that ship with GHC are usually also uploaded to Hackage, to > make the documentation easily accessible, but integer-gmp-1.0.1.0 was not > uploaded to Hackage until 4 months after the release. > * The metadata for integer-gmp-1.0.1.0 as uploaded to Hackage differed from > the metadata that was actually in the source tarball of GHC-8.2.1 and > GHC-8.2.2. > * The metadata for integer-gmp-1.0.1.0 as uploaded to Hackage included > Cabal-2.0 specific syntactic sugar, making the metadata unreadable using > any tooling that did not link against the Cabal-2.0.0.2 library (or any > later version). > * It so happened that one particular version of one particular downstream > build tool, Stack, had a bug, compounding the bad effects of the previous > point. But a new release has now been made, and in any case that's not a > problem for GHC to solve. So let's keep that out of the discussion here. > > So I suggest we discuss ways to eliminate or reduce the likelihood of any > of the above problems from occurring again. Here are some ideas: > > * GHC should never under any circumstance ship with an unreleased version > of any independently maintained dependency. Cabal is one such dependency. > This should hold true for anything else. We could just add that policy to > the Release Policy. > * Stronger still, GHC should not switch to a new major release of a > dependency at any time during feature freeze ahead of a release. E.g. if > Cabal-3.0.0 ships before feature freeze for GHC-9.6, then maybe it's fair > game to include in GHC. But not if Cabal-3.0.0 hasn't shipped yet. > * The 3-release backwards compat rule should apply in all circumstances. > That means major version bumps of any library GHC ships with, including > base, should not imply any breaking change in the API's of any such library. > * GHC does have control over reinstallable packages (like text and > bytestring): GHC need not ship with the latest versions of these, if indeed > they introduce breaking changes that would contravene the 3-release policy. > * Note: today, users are effectively tied to whatever version of the > packages ships with GHC (i.e. the "reinstallable" bit is problematic today > for various technical reasons). That's why a breaking change in bytestring > is technically a breaking change in GHC. > * The current release policy covers API stability, but what about metadata? > In the extreme, we could say a 3-release policy applies to metadata too. > Meaning, all metadata shipping with GHC now and in the next 2 releases > should be parseable by today's version of Cabal and downstream tooling. Is > such a long lead time necessary? That's for build tool authors to say, and > a point to negotiate with GHC devs. > * Because there are far fewer consumers of metadata than consumers of say > base, I think shorter lead time is reasonable. At the other extreme, it > could even be just the few months during feature freeze. > * The release notes bugs mentioned above and the lack of consistent upload > to Hackage are a symptom of lack of release automation, I suspect. That's > how to fix it, but we could also spell out in the Release Policy that GHC > libraries should all be on Hackage from the day of release. > > Finally, a question for discussion: > > * Hackage allows revising the metadata of an uploaded package even without > changing the version number. This happens routinely on Hackage today by the > Hackage trustees. Should this be permitted for packages whose release is > completely tied to that of GHC itself (like integer-gmp)? > > Best, > > Mathieu > > > On 13 December 2017 at 17:43, Simon Peyton Jones via ghc-devs < > ghc-devs at haskell.org> wrote: > >> Dear GHC devops group >> >> The conversation on Trac #14558 >> suggests that we might >> want to consider reviewing GHC’s release policies >> . >> This email is to invite your input. >> >> The broad questions is this. We want GHC to serve the needs of all its >> users, including downstream tooling that uses GHC. What release policies >> will best support that goal? For example, we already ensure that GHC 8.4 >> can be compiled with 8.2 and 8.0. This imposes a slight tax on GHC >> development, but it means that users don't need to upgrade quite as often. >> (If the tempo of releases increases, we might want to increase the window.) >> >> Trac #14558 suggests that we might want to ensure the metadata on GHC’s >> built-in libraries is parsable with older Cabals. One possibility would be >> this: >> >> - Ensure that the Cabal metadata of non-reinstallable packages (e.g. >> integer-gmp) shipped with GHC be parsable by the Cabal versions shipped >> with the last two major GHC releases [i.e. have a sufficiently old >> cabal-version field]. That is, in general a new Cabal specification will >> need to be shipped with two GHC releases before GHC will use start using >> its features in non-reinstallable packages. >> - Upholding this policy won't always be possible. There may be cases >> (as is the case Hadrian for GHC 8.4) where the benefit of quickly >> introducing incompatible syntax outweighs the need for compatibility. In >> this (hopefully rare) case we would explicitly advertise the >> incompatibility in the release documentation, and give as much notice as >> possible to users to allow downstream tools to adapt. >> - For reinstallable packages, of which GHC is simply a client (like >> text or bytestring), we can’t reasonably enforce such a policy, because GHC >> devs have no control over what the maintainers of external core libraries >> put in their Cabal files. >> >> This is just a proposal. The narrow questions are these: >> >> - Would this be sufficient to deal with the concerns raised in #14558? >> - Is it necessary, ow would anything simpler be sufficient? >> - What costs would the policy impose on GHC development? >> - There may be matters of detail: e.g. is two releases the right grace >> period. Would one do? >> >> Both the broad question and the narrow ones are appropriate for the Devops >> group. >> >> Thanks! >> >> Simon >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> >> > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > ------------------------------ > > End of ghc-devs Digest, Vol 172, Issue 15 > ***************************************** From m at tweag.io Thu Dec 14 00:06:10 2017 From: m at tweag.io (Boespflug, Mathieu) Date: Thu, 14 Dec 2017 01:06:10 +0100 Subject: Fwd: Release policies In-Reply-To: References: Message-ID: Hi Gerhom, On 14 December 2017 at 00:19, Gershom B wrote: > > Mathieu: > > I think the points about better tooling for documenting the correct > claims in the release process are well taken. Updating the release > notes manually leaves way too much room for error. > > However, I think you are incorrect that GHC 8.2.1 and 8.2.2 did not > have cabal-install 1.24 support. They did. it works with them. They did, and indeed Stack too worked just fine with them, but that was assuming that integer-gmp-1.0.1.0 really was what was shipped in the tarballs, not what it was on Hackage (until it got recently revised). I don't know which version of integer-gmp-1.0.1.0 was the intended one. They both have the same version number and neither seems more authoritative than the other to me. Had the Hackage one been the one that shipped, then I'm not sure that cabal-install-1.24 would have worked. Stack broke the moment what was on Hackage and what was in GHC bindists did not line up anymore. And with release notes mentioning incorrect version numbers, harder still to tell. But crucially, what *is* the policy around Cabal versions? This comment, https://ghc.haskell.org/trac/ghc/ticket/14558#comment:23 claims "if Stack doesn't support the version of Cabal that ships with a certain version of GHC, it shouldn't claim that it supports that version of GHC. The same applies to cabal-install". Is any build tool linked against Cabal-X by definition "not a supported configuration" by GHC-Z if it ships with Cabal-Y such that X < Y? > > * Stronger still, GHC should not switch to a new major release of a > > dependency at any time during feature freeze ahead of a release. E.g. if > > Cabal-3.0.0 ships before feature freeze for GHC-9.6, then maybe it's fair > > game to include in GHC. But not if Cabal-3.0.0 hasn't shipped yet. > > I don't think this works, in terms of coupled dependencies. If/when > Cabal-3 is developed it will almost certainly be in tandem with GHC > support. Right. But switching from Cabal-2 to Cabal-3 (a hypothetical at this point) sounds like a whole new set of features transitively just made it into the compiler. Is that something we're happy to happen during feature freeze? > The general motivation of making a "feature freeze" more of a "freeze > all the moving parts, really" I do agree with. Having a real freeze is > part of a better release process, and it should allow all the > downstream consumers of everything more time to really catch up. This > is just one instance of that need. Agreed. > > Finally, a question for discussion: > > > > * Hackage allows revising the metadata of an uploaded package even without > > changing the version number. This happens routinely on Hackage today by the > > Hackage trustees. Should this be permitted for packages whose release is > > completely tied to that of GHC itself (like integer-gmp)? > > It is rare that this is needed, but the ability just served us well -- > editing integer-gmp to remove the new syntax was very useful, as it > let us fix up old stack nightlies. Like all revisions, these should be > made with some care and thought, but since we've just seen why it was > helpful, I'd hate to now say we can't do it again if some other > unforseen circumstance crops up. I don't disagree. But then we'd need to abandon any notion that versions of packages on Hackage and versions of packages in the GHC release tarball always match up. Might even be worth calling that out explicitly in the policy. From gershomb at gmail.com Thu Dec 14 00:27:13 2017 From: gershomb at gmail.com (Gershom B) Date: Wed, 13 Dec 2017 19:27:13 -0500 Subject: Fwd: Release policies In-Reply-To: References:

Message-ID: On Wed, Dec 13, 2017 at 7:06 PM, Boespflug, Mathieu wrote: > > But crucially, what *is* the policy around Cabal versions? This > comment, https://ghc.haskell.org/trac/ghc/ticket/14558#comment:23 > claims "if Stack doesn't support the version of Cabal that ships with > a certain version of GHC, it shouldn't claim that it supports that > version of GHC. The same applies to cabal-install". Is any build tool > linked against Cabal-X by definition "not a supported configuration" > by GHC-Z if it ships with Cabal-Y such that X < Y? My understanding is that this is the general thought, yes. In fact, I've been told that even though cabal-install 1.24 did end up working with the GHC 8.2.x series, the release notes, which were not updated properly, actually _were supposed_ to say cabal-install 2.0.0.0 was what was supported there. I believe future cabal-installs will warn when used with a ghc with a newer Cabal-lib than they were built against... > Right. But switching from Cabal-2 to Cabal-3 (a hypothetical at this > point) sounds like a whole new set of features transitively just made > it into the compiler. Is that something we're happy to happen during > feature freeze? Right. After freeze, the compiler itself shouldn't switch from Cabal-2 to Cabal-3. But I would imagine rather that the Cabal-3 tree and the compiler tree would be updated in tandem, and then the "freeze" would sort of apply to both in tandem as well. So there wouldn't be big changes after the freeze, but nor would the compiler be coupled to a _released_ lib. Rather, they would develop together, freeze together, and release together. > I don't disagree. But then we'd need to abandon any notion that > versions of packages on Hackage and versions of packages in the GHC > release tarball always match up. Might even be worth calling that out > explicitly in the policy. Not exactly. The tarball of the package on hackage should match the release tarball. Revisions don't change the tarball. They just add additional metadata to the index as well that cabal-install knows how to use in conjunction with the tarball: https://github.com/haskell-infra/hackage-trustees/blob/master/revisions-information.md#what-are-revisions --Gershom From cheater00 at gmail.com Thu Dec 14 00:28:37 2017 From: cheater00 at gmail.com (cheater00 cheater00) Date: Thu, 14 Dec 2017 01:28:37 +0100 Subject: Long standing annoying issue in ghci In-Reply-To: References:

<814bb618-30cc-6024-28b0-17c7c9f95a45@gmail.com> <572bbe0d-1c65-19ef-bbb1-473d395fc46e@gmail.com> Message-ID: I went ahead and did this for you. On Tue, Dec 12, 2017 at 8:44 PM, cheater00 cheater00 wrote: > Yes, it is worth doing it, because until Haskeline has been fixed and > integrated into ghci, the issue persists and needs to remain filed. > > > On Fri, 8 Dec 2017 18:25 MarLinn, wrote: >> >> I opened an issue on the Haskeline github >> (https://github.com/judah/haskeline/issues/72). >> >> But it seems to be completely Haskeline-side, so I'm not sure if it's >> worth re-opening the one for ghci? As missing documentation maybe? >> (BTW, I found this on the wiki: https://wiki.haskell.org/GHCi_in_colour. >> Might be a good place to put it, if linked.) >> >> If you want to, here are my test cases rewritten as ghci prompts: >> >> -- single line, positioning error >> :set prompt " \ESC[36m%\ESC[0m " >> -- single line, works >> :set prompt " \ESC[36m\STX%\ESC[0m\STX " >> -- multiline, bad output >> :set prompt "\ESC[32m\STX–––\ESC[0m\STX\n \ESC[36m\STX%\ESC[0m\STX " >> -- multiline, works but is inconsistent >> :set prompt "\ESC[32m–––\ESC[0m\n \ESC[36m\STX%\ESC[0m\STX " >> >> In my tests, the positioning errors consistently happen if there are any >> "unclosed" escape-sequences on the last line of the prompt, regardless of >> its length. Escape sequences on previous lines consistently create "weird >> characters", but don't influence the positioning. Also regardless of their >> lengths. That makes sense, as both sets of lines seem to be handled quite >> differently. >> >> Are multiline prompts even used by a lot of people? I like mine because it >> gives me a both a list of modules and a consistent cursor position. But >> maybe I'm the exception? >> >> Cheers. >> >> >> On 2017-12-07 23:15, cheater00 cheater00 wrote: >> >> Interesting. Would you mind reopening the issue and providing a buggy >> example? Amd alerting haskeline maintainers? How does it work on a 1 line >> prompt that is so long it wraps? >> >> >> On Thu, 7 Dec 2017 23:11 MarLinn, wrote: >>> >>> >>> > Here's what I use: >>> > >>> > :set prompt "\ESC[46m\STX%s>\ESC[39;49m\STX " >>> > >>> > I believe \STX is a signal to haskeline for control sequences. >>> > Documentation is here: >>> > https://github.com/judah/haskeline/wiki/ControlSequencesInPrompt >>> Note: If you're using a multi-line prompt, things may be different >>> again. I don't know what the rules are, but I found that if I put \STX >>> on any but the last line of prompts I get weird characters. The same >>> goes for any \SOH you might want to add for some reason. >>> >>> Cheers, >>> MarLinn >>> >> >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs at haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From cheater00 at gmail.com Thu Dec 14 00:42:56 2017 From: cheater00 at gmail.com (cheater00 cheater00) Date: Thu, 14 Dec 2017 01:42:56 +0100 Subject: Cannot unsubscribe Message-ID: I went to http://mail.haskell.org/cgi-bin/mailman/options/ghc-devs logged in and clicked on "unsubscribe" and no unsubscription email shows up. What do i do to unsubscribe? Thanks From cheater00 at gmail.com Thu Dec 14 01:27:17 2017 From: cheater00 at gmail.com (cheater00 cheater00) Date: Thu, 14 Dec 2017 02:27:17 +0100 Subject: Cannot unsubscribe In-Reply-To: References: Message-ID: Got it. Got filed under "promotions". On Thu, Dec 14, 2017 at 1:42 AM, cheater00 cheater00 wrote: > I went to http://mail.haskell.org/cgi-bin/mailman/options/ghc-devs > logged in and clicked on "unsubscribe" and no unsubscription email > shows up. What do i do to unsubscribe? > > Thanks From m at tweag.io Thu Dec 14 09:27:59 2017 From: m at tweag.io (Boespflug, Mathieu) Date: Thu, 14 Dec 2017 10:27:59 +0100 Subject: Fwd: Release policies In-Reply-To: References:

Message-ID: Hi Gershom, thanks for the extra input. So we've confirmed two facts: * GHC (intended to) ship with only Cabal-2.0 support, but there was a mistake in the release notes so this was unclear to downstream tooling authors. * Cabal-2.0 was released anywhere between slightly *after* and *exactly at the same as* GHC, despite GHC itself shipping with Cabal-2.0. I'm not too concerned by the first point: so long as Cabal-X does not introduce breaking changes, the fact that GHC-Y ultimately shipped with Cabal-X shouldn't be a problem. And this kind of bug in the release notes should go away provided more automation. The second one is more interesting. It is, as you point out, a product of GHC and Cabal being intimately linked and co-developed to a large extent. This leads to a simultaneous release that poses a concrete problem: * if new Cabal versions are used immediately in GHC, then that gives no time at all ahead of a GHC release for downstream tooling authors to adapt, because Cabal is, up until the point of the GHC release, a moving target. Three possible solutions: * Provided no API breaking changes in Cabal, if no metadata that ships with GHC uses new Cabal features for some period of time before release, then the problem goes away. * Or something close to what Manuel proposed in another thread: ship in GHC-X+1 the Cabal version that was co-developed during the development cycle of GHC-X. * Or a middle ground: make feature freeze a thing. Meaning that for a couple of months before a major GHC release, the major new Cabal isn't technically released yet, but like GHC itself within this period, it's pretty staid, so not so much a moving target, and something downstream tooling authors can possibly adapt to even without any grace period on new metadata features. This assumes that the 2 months of feature freeze are enough time for downstream tooling. Thoughts from any of those maintainers? On 14 December 2017 at 01:27, Gershom B wrote: > On Wed, Dec 13, 2017 at 7:06 PM, Boespflug, Mathieu wrote: >> >> But crucially, what *is* the policy around Cabal versions? This >> comment, https://ghc.haskell.org/trac/ghc/ticket/14558#comment:23 >> claims "if Stack doesn't support the version of Cabal that ships with >> a certain version of GHC, it shouldn't claim that it supports that >> version of GHC. The same applies to cabal-install". Is any build tool >> linked against Cabal-X by definition "not a supported configuration" >> by GHC-Z if it ships with Cabal-Y such that X < Y? > > My understanding is that this is the general thought, yes. In fact, > I've been told that even though cabal-install 1.24 did end up working > with the GHC 8.2.x series, the release notes, which were not updated > properly, actually _were supposed_ to say cabal-install 2.0.0.0 was > what was supported there. I believe future cabal-installs will warn > when used with a ghc with a newer Cabal-lib than they were built > against... > >> Right. But switching from Cabal-2 to Cabal-3 (a hypothetical at this >> point) sounds like a whole new set of features transitively just made >> it into the compiler. Is that something we're happy to happen during >> feature freeze? > > Right. After freeze, the compiler itself shouldn't switch from Cabal-2 > to Cabal-3. But I would imagine rather that the Cabal-3 tree and the > compiler tree would be updated in tandem, and then the "freeze" would > sort of apply to both in tandem as well. So there wouldn't be big > changes after the freeze, but nor would the compiler be coupled to a > _released_ lib. Rather, they would develop together, freeze together, > and release together. > >> I don't disagree. But then we'd need to abandon any notion that >> versions of packages on Hackage and versions of packages in the GHC >> release tarball always match up. Might even be worth calling that out >> explicitly in the policy. > > Not exactly. The tarball of the package on hackage should match the > release tarball. Revisions don't change the tarball. They just add > additional metadata to the index as well that cabal-install knows how > to use in conjunction with the tarball: > https://github.com/haskell-infra/hackage-trustees/blob/master/revisions-information.md#what-are-revisions > > --Gershom From ben at well-typed.com Thu Dec 14 17:56:54 2017 From: ben at well-typed.com (Ben Gamari) Date: Thu, 14 Dec 2017 12:56:54 -0500 Subject: Fwd: Release policies In-Reply-To: References:

Message-ID: <87ind9fb1g.fsf@ben-laptop.smart-cactus.org> "Boespflug, Mathieu" writes: > Hi Gerhom, > > On 14 December 2017 at 00:19, Gershom B wrote: >> >> Mathieu: >> >> I think the points about better tooling for documenting the correct >> claims in the release process are well taken. Updating the release >> notes manually leaves way too much room for error. >> Indeed, the release notes have historically been a massive headache. Happily, I wrote a bit of the necessary tooling to fix this a few weeks ago [1]. [1] e4dc2cd51902a8cd83476f861cf52996e5adf157 >> However, I think you are incorrect that GHC 8.2.1 and 8.2.2 did not >> have cabal-install 1.24 support. They did. it works with them. > > They did, and indeed Stack too worked just fine with them, but that > was assuming that integer-gmp-1.0.1.0 really was what was shipped in > the tarballs, not what it was on Hackage (until it got recently > revised). I don't know which version of integer-gmp-1.0.1.0 was the > intended one. They both have the same version number and neither seems > more authoritative than the other to me. Had the Hackage one been the > one that shipped, then I'm not sure that cabal-install-1.24 would have > worked. Stack broke the moment what was on Hackage and what was in GHC > bindists did not line up anymore. And with release notes mentioning > incorrect version numbers, harder still to tell. > I agree that the cabal files uploaded to Hackage should match what is released or, if not, there should be a very good reason for divergence. [snip] > >> The general motivation of making a "feature freeze" more of a "freeze >> all the moving parts, really" I do agree with. Having a real freeze is >> part of a better release process, and it should allow all the >> downstream consumers of everything more time to really catch up. This >> is just one instance of that need. > > Agreed. > Also agreed. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From ben at well-typed.com Thu Dec 14 18:28:38 2017 From: ben at well-typed.com (Ben Gamari) Date: Thu, 14 Dec 2017 13:28:38 -0500 Subject: Fwd: Release policies In-Reply-To: References:

Message-ID: <87fu8df9kf.fsf@ben-laptop.smart-cactus.org> "Boespflug, Mathieu" writes: > ---------- Forwarded message ---------- > From: Boespflug, Mathieu > Date: 13 December 2017 at 23:03 > Subject: Re: Release policies > To: Simon Peyton Jones > Cc: ghc-devops-group at haskell.org > > > [replying to ghc-devops-group@, which I assume based on your email's > content is the mailing list you intended.] > > Hi Simon, > > feedback from downstream consumers of Cabal metadata (e.g. build tool > authors) will be particularly useful for the discussion here. Here are my > thoughts as a bystander. > > It's worth trying to identify what problems came up during the integer-gmp > incident in Trac #14558: > > * GHC 8.2.1 shipped with integer-gmp-1.0.1.0 but the release notes said > otherwise. > > * GHC 8.2.1 shipped with Cabal-2.0.0.2, but specifically claimed in the > release notes that cabal-install-1.24 (and by implication any other build > tool based on Cabal-the-library version 1.24) was supported: "GHC 8.2 only > works with cabal-install version 1.24 or later. Please upgrade if you have > an older version of cabal-install." > > * GHC 8.2.2 also claimed Cabal-1.24 support. > > * GHC 8.2.1 was released in July 2017 with Cabal-2.0.0.2, a brand new major > release with breaking changes to the metadata format, without much lead > time for downstream tooling authors (like Stack) to adapt. > > * But actually if we look at their respective release notes, GHC 8.2.1 was > relased in July 2017, even though the Cabal website claims that > Cabal-2.0.0.2 was released in August 2017 (see > https://www.haskell.org/cabal/download.html). So it looks like GHC didn't > just not give enough lead time about an upstream dependency it shipped > with, it shipped with an unreleased version of Cabal! Perhaps this is true and I admit I wasn't happy about releasing the compiler without a Cabal release. However, there was no small amount of pressure to push forward nevertheless as the release was already quite late and the expectation was a Cabal release would be coming shortly after the GHC release. Coordination issues like this are a major reason why I think it would be better if GHC were more decoupled from its dependencies' upstreams. I think the approach that we discussed at ICFP, where library authors must upstream their version bumps before the freeze, just like any library, is perhaps one way forward although I suspect it exceptions will need to be made. > * Libraries that ship with GHC are usually also uploaded to Hackage, to > make the documentation easily accessible, but integer-gmp-1.0.1.0 was not > uploaded to Hackage until 4 months after the release. > > * The metadata for integer-gmp-1.0.1.0 as uploaded to Hackage differed from > the metadata that was actually in the source tarball of GHC-8.2.1 and > GHC-8.2.2. > > * The metadata for integer-gmp-1.0.1.0 as uploaded to Hackage included > Cabal-2.0 specific syntactic sugar, making the metadata unreadable using > any tooling that did not link against the Cabal-2.0.0.2 library (or any > later version). > > * It so happened that one particular version of one particular downstream > build tool, Stack, had a bug, compounding the bad effects of the previous > point. But a new release has now been made, and in any case that's not a > problem for GHC to solve. So let's keep that out of the discussion here. > > So I suggest we discuss ways to eliminate or reduce the likelihood of any > of the above problems from occurring again. Here are some ideas: > > * GHC should never under any circumstance ship with an unreleased version > of any independently maintained dependency. Cabal is one such dependency. > This should hold true for anything else. We could just add that policy to > the Release Policy. > We can adopt this as a policy, but doing so very well may mean that GHC will be subject to schedule slips beyond its control. We can hope that upstream maintainers will be responsive, but there is little we can do when they are not. Of course, if we adopt the policy of disallowing all but essentially core library bumps during the freeze then we may be able to mitigate this. > * Stronger still, GHC should not switch to a new major release of a > dependency at any time during feature freeze ahead of a release. E.g. if > Cabal-3.0.0 ships before feature freeze for GHC-9.6, then maybe it's fair > game to include in GHC. But not if Cabal-3.0.0 hasn't shipped yet. > Yes, this I agree with. I think we can be more accomodating of minor bumps to fix bugs which may come to light during the freeze, but major releases should be avoided. > * The 3-release backwards compat rule should apply in all circumstances. > That means major version bumps of any library GHC ships with, including > base, should not imply any breaking change in the API's of any such library. > I'm not sure I follow what you are suggesting here. > * GHC does have control over reinstallable packages (like text and > bytestring): GHC need not ship with the latest versions of these, if indeed > they introduce breaking changes that would contravene the 3-release policy. > > * Note: today, users are effectively tied to whatever version of the > packages ships with GHC (i.e. the "reinstallable" bit is problematic today > for various technical reasons). That's why a breaking change in bytestring > is technically a breaking change in GHC. > I don't follow: Only a small fraction of packages, namely those that explicitly link against the `ghc` library, are tied. Can you clarify what technical reasons you are referring to here? > * The current release policy covers API stability, but what about metadata? > In the extreme, we could say a 3-release policy applies to metadata too. > Meaning, all metadata shipping with GHC now and in the next 2 releases > should be parseable by today's version of Cabal and downstream tooling. Is > such a long lead time necessary? That's for build tool authors to say, and > a point to negotiate with GHC devs. > > * Because there are far fewer consumers of metadata than consumers of say > base, I think shorter lead time is reasonable. At the other extreme, it > could even be just the few months during feature freeze. Right, I wouldn't be opposed to striving for this in principle although I think we should be aware that breakage is at times necessary and the policy should accomodate this. I think the important thing is that we be aware of when we are breaking metadata compatibility and convey this to our users. > * The release notes bugs mentioned above and the lack of consistent upload > to Hackage are a symptom of lack of release automation, I suspect. That's > how to fix it, but we could also spell out in the Release Policy that GHC > libraries should all be on Hackage from the day of release. > Yes, the hackage uploads have historically been handled manually. I have and AFAIK most release managers coming before me have generally deferred this to Herbert as is quite meticulous. However, I think it would be nice if we could remove the need for human intervention entirely. > Finally, a question for discussion: > > * Hackage allows revising the metadata of an uploaded package even without > changing the version number. This happens routinely on Hackage today by the > Hackage trustees. Should this be permitted for packages whose release is > completely tied to that of GHC itself (like integer-gmp)? > If there is a bug in the metadata then I don't think we should necessarily tie our hands from fixing it. However, we should clearly take great care when making such changes to avoid further breakage. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From m at tweag.io Thu Dec 14 21:19:48 2017 From: m at tweag.io (Boespflug, Mathieu) Date: Thu, 14 Dec 2017 22:19:48 +0100 Subject: Fwd: Release policies In-Reply-To: <87ind9fb1g.fsf@ben-laptop.smart-cactus.org> References:

<87ind9fb1g.fsf@ben-laptop.smart-cactus.org> Message-ID: >> On 14 December 2017 at 00:19, Gershom B wrote: >>> >>> Mathieu: >>> >>> I think the points about better tooling for documenting the correct >>> claims in the release process are well taken. Updating the release >>> notes manually leaves way too much room for error. >>> > Indeed, the release notes have historically been a massive headache. > Happily, I wrote a bit of the necessary tooling to fix this a few weeks > ago [1]. > > [1] e4dc2cd51902a8cd83476f861cf52996e5adf157 Very cool! From m at tweag.io Thu Dec 14 21:59:38 2017 From: m at tweag.io (Boespflug, Mathieu) Date: Thu, 14 Dec 2017 22:59:38 +0100 Subject: Fwd: Release policies In-Reply-To: <87fu8df9kf.fsf@ben-laptop.smart-cactus.org> References:

<87fu8df9kf.fsf@ben-laptop.smart-cactus.org> Message-ID: >> * But actually if we look at their respective release notes, GHC 8.2.1 was >> relased in July 2017, even though the Cabal website claims that >> Cabal-2.0.0.2 was released in August 2017 (see >> https://www.haskell.org/cabal/download.html). So it looks like GHC didn't >> just not give enough lead time about an upstream dependency it shipped >> with, it shipped with an unreleased version of Cabal! > > Perhaps this is true and I admit I wasn't happy about releasing the compiler > without a Cabal release. However, there was no small amount of pressure to > push forward nevertheless as the release was already quite late and the > expectation was a Cabal release would be coming shortly after the GHC > release. Coordination issues like this are a major reason why I think it > would be better if GHC were more decoupled from its dependencies' > upstreams. I have the same sentiment. Do you think this is feasible in the case of Cabal? Even if say something like Backpack shows up all over again? If so, are there concrete changes that could be made to support the following workflow: * upstreams develop their respective libraries independently of GHC using their own testing. * If they want GHC to ship a newer version, they create a Diff. As Manuel proposed in a separate thread, this must be before feature freeze, unless... * ... a critical issue is found in the upstream release, in which case upstream cuts a new release, and submits a Diff again. * GHC always has the option to back out an offending upgrade, and revert to a known good version. In fact it should preemptively do so while waiting for a new release of upstream. * In general, GHC does not track git commits of upstream dependencies in an unknown state of quality, but tracks vetted and tested releases instead. >> * GHC should never under any circumstance ship with an unreleased version >> of any independently maintained dependency. Cabal is one such dependency. >> This should hold true for anything else. We could just add that policy to >> the Release Policy. >> > We can adopt this as a policy, but doing so very well may mean that GHC > will be subject to schedule slips beyond its control. We can hope that > upstream maintainers will be responsive, but there is little we can do > when they are not. Why not? If GHC only ever tracks upstream releases (as I think it should), not git commits in unknown state, then we don't need upstream maintainer responsiveness. Because at any point in time, all GHC dependencies are already released. If GHC should ship with a newer version of a dependency, the onus is on the upstream maintainer to submit a Diff asking GHC to move to the latest version. Are there good reasons for GHC to track patches not upstreamed and released? >> * Stronger still, GHC should not switch to a new major release of a >> dependency at any time during feature freeze ahead of a release. E.g. if >> Cabal-3.0.0 ships before feature freeze for GHC-9.6, then maybe it's fair >> game to include in GHC. But not if Cabal-3.0.0 hasn't shipped yet. >> > Yes, this I agree with. I think we can be more accomodating of minor > bumps to fix bugs which may come to light during the freeze, but major > releases should be avoided. Agreed. >> * The 3-release backwards compat rule should apply in all circumstances. >> That means major version bumps of any library GHC ships with, including >> base, should not imply any breaking change in the API's of any such library. >> > I'm not sure I follow what you are suggesting here. Nothing new: just that the 3-release policy doesn't just apply to base, but also anything else that happens to ship with GHC (including Cabal). Perhaps that already the policy? >> * GHC does have control over reinstallable packages (like text and >> bytestring): GHC need not ship with the latest versions of these, if indeed >> they introduce breaking changes that would contravene the 3-release policy. >> >> * Note: today, users are effectively tied to whatever version of the >> packages ships with GHC (i.e. the "reinstallable" bit is problematic today >> for various technical reasons). That's why a breaking change in bytestring >> is technically a breaking change in GHC. >> > I don't follow: Only a small fraction of packages, namely those that > explicitly link against the `ghc` library, are tied. Can you clarify > what technical reasons you are referring to here? Builds often fail for strange reasons when both bytestring-0.10.2 and bytestring-0.10.1 are in scope. Some libraries in a build plan pick up one version where some pick up another. The situation here might well be better than it used to be, but at this point in time Stackage works hard to ensure that in any given package set, there is *exactly one* version of any package. That's why Stackage aligns versions of core packages to whatever ships with the GHC version the package set is based on. So in this sense, AFAIK a bug in bytestring can't be worked around by reinstalling bytestring (not in Stackage land): it requires waiting for the next GHC version that will ship with a new version of bytestring with that bug fixed. I'm not entirely familiar with all Stackage details so Michael - please step in if this is incorrect. >> * Because there are far fewer consumers of metadata than consumers of say >> base, I think shorter lead time is reasonable. At the other extreme, it >> could even be just the few months during feature freeze. > > Right, I wouldn't be opposed to striving for this in principle although > I think we should be aware that breakage is at times necessary and the > policy should accomodate this. I think the important thing is that we be > aware of when we are breaking metadata compatibility and convey this to > our users. That sounds reasonable. But when have we ever needed to use non backwards compatible metadata ASAP? The integer-gmp example was a case in point: the Cabal-2.0 feature it was using was merely syntactic sugar at this point, since no tool *yet* interprets the new constructs in any special way AFAIK. >> * The release notes bugs mentioned above and the lack of consistent upload >> to Hackage are a symptom of lack of release automation, I suspect. That's >> how to fix it, but we could also spell out in the Release Policy that GHC >> libraries should all be on Hackage from the day of release. >> > Yes, the hackage uploads have historically been handled manually. I have > and AFAIK most release managers coming before me have generally deferred > this to Herbert as is quite meticulous. However, I think it would be > nice if we could remove the need for human intervention entirely. Indeed. Can be part of the deploy step in the continuous integration pipeline. From nboldi at elte.hu Fri Dec 15 05:46:14 2017 From: nboldi at elte.hu (=?UTF-8?B?TsOpbWV0aCBCb2xkaXpzw6Fy?=) Date: Fri, 15 Dec 2017 14:46:14 +0900 Subject: GHC-API unloading from external interpreter Message-ID: Dear GHC Developers, I'm using the GHC API to load Haskell modules and extract their syntax tree. I need to generate code where TH is used (by default I do in-memory linking). I've run into some trouble with external interpreter. When -fexternal-interpreter is NOT set, I use Linker.unload to remove bytecode from the global linker before reloading modules. However it is not applicable when external interpreter is used and I could not find corresponding functionality in the GHCi module (where communication with external interpreter is implemented). I've recently created 2 tickets related to this question: https://ghc.haskell.org/trac/ghc/ticket/14576, https://ghc.haskell.org/trac/ghc/ticket/14577 Can someone help me with this problem? Best Regards, Boldizsár Németh From moritz at lichtzwerge.de Fri Dec 15 06:04:02 2017 From: moritz at lichtzwerge.de (Moritz Angermann) Date: Fri, 15 Dec 2017 14:04:02 +0800 Subject: GHC-API unloading from external interpreter In-Reply-To: References: Message-ID: <82C197A9-8783-4669-BF00-A0BCE834D0C6@lichtzwerge.de> Hi, I believe you are looking for the `Message` GADT in `GHCi.Message` ``` -- ----------------------------------------------------------------------------- -- The RPC protocol between GHC and the interactive server -- | A @Message a@ is a message that returns a value of type @a at . -- These are requests sent from GHC to the server. data Message a where ``` maybe specifically the `UnloadObj` message. Cheers, Moritz > On Dec 15, 2017, at 1:46 PM, Németh Boldizsár wrote: > > Dear GHC Developers, > > I'm using the GHC API to load Haskell modules and extract their syntax tree. I need to generate code where TH is used (by default I do in-memory linking). I've run into some trouble with external interpreter. When -fexternal-interpreter is NOT set, I use Linker.unload to remove bytecode from the global linker before reloading modules. However it is not applicable when external interpreter is used and I could not find corresponding functionality in the GHCi module (where communication with external interpreter is implemented). > > I've recently created 2 tickets related to this question: https://ghc.haskell.org/trac/ghc/ticket/14576, https://ghc.haskell.org/trac/ghc/ticket/14577 > > Can someone help me with this problem? > > Best Regards, > Boldizsár Németh > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs From michael at snoyman.com Fri Dec 15 07:42:55 2017 From: michael at snoyman.com (Michael Snoyman) Date: Fri, 15 Dec 2017 10:42:55 +0300 Subject: Fwd: Release policies In-Reply-To: References:

Message-ID: On Thu, Dec 14, 2017 at 12:27 PM, Boespflug, Mathieu wrote: [snip] * Or a middle ground: make feature freeze a thing. Meaning that for a > couple of months before a major GHC release, the major new Cabal isn't > technically released yet, but like GHC itself within this period, it's > pretty staid, so not so much a moving target, and something downstream > tooling authors can possibly adapt to even without any grace period on > new metadata features. This assumes that the 2 months of feature > freeze are enough time for downstream tooling. Thoughts from any of > those maintainers? > > Short answer: if there's a clear idea in advance of when this feature freeze is going to happen, I think we can coordinate releases of downstream tooling (Stack being the most important, but stackage-curator playing in as well) so that 2 months is sufficient. I'll talk with the rest of the Stack team to see if there are any concerns. Longer answer: Stack intentionally avoids depending on the internals of Cabal wherever possible. Instead of calling library functions directly from within Haskell code to perform builds, for example, it interacts with the Setup.hs files over their command line interface.[1] This has two results: * Stack can usually start using new GHC/Cabal versions without a new Stack release, since it's just shelling out for the actual build * There's not usually very much code churn needed in Stack to upgrade to a newer Cabal release This past release was an exception because of all of the changes that landed, both the new cabal grammar to support the ^>= operator (making the old parser incapable of lossily parsing new files) and API changes (I think mostly around Backpack, though there was some code cleanup as well). In particular, the main interface we need from Cabal—the package description data types and parser—changed significantly enough that it took significant effort to upgrade. There were also new features added (like sub libraries and foreign libraries) that weren't immediately supported by the old Stack version, and had to be manually added in. Tying this up: generally upgrading to a new Cabal release should be fine, and the only concern I'd have is fitting it into a release schedule with Stack. The complications that could slow that down are: * Changes to the command line interface that Stack uses (hopefully those are exceedingly rare) * Major overhauls to the Stack-facing API Michael [1] This allows for more reproducible builds of older snapshots, insuring that the exact same Cabal library is performing the builds -------------- next part -------------- An HTML attachment was scrubbed... URL: From m at tweag.io Fri Dec 15 08:41:36 2017 From: m at tweag.io (Boespflug, Mathieu) Date: Fri, 15 Dec 2017 09:41:36 +0100 Subject: Fwd: Release policies In-Reply-To: References:

Message-ID: Thanks for the feedback, Michael. Manuel, I believe you are also a Cabal-the-library consumer in Haskell For Mac? Michael, you brought up another problem tangentially related to the original integer-gmp issue but that was not in my original list earlier in this thread: * Cabal-2.0.0 had breaking changes in the API. This means that by association GHC itself broke BC, because it shipped with Cabal-2.0, without the usual grace period. Now, there are far fewer users of Cabal than of base. All, Michael in his previous email seems to be okay with breaking changes in Cabal given the conditions he stated (2 months grace period, advance notice of when the 2 months start). And perhaps this points to the lack of a need for the regular grace period applying to Cabal. How many other users of Cabal-the-library are there? In principle, every single Hackage package out there, which all have a Setup.hs script. Most of them are trivial, but how many did break because of these API changes? I for one am pretty happy for Cabal to move fast, but I'm concerned that these breaking changes happened without any kind of advance notice. To Simon's original point - there does not to be a clear policy and a good process surrounding Cabal itself and other GHC dependencies. So far we discussed mostly metadata changes, not API changes. And to be clear, folks did get some (post facto) notice in September: http://coldwa.st/e/blog/2017-09-09-Cabal-2-0.html. That's helpful, but I submit that in the future this really should be part of the GHC release announcement (which happened over a month before that), and in fact a migration guide circulated before the feature freeze, so downstream tooling authors can adapt. If this is not possible, then perhaps it's premature for GHC to include that given Cabal release. Again, GHC should always have the option to stick to the old Cabal version until things get ironed out. On 15 December 2017 at 08:42, Michael Snoyman wrote: > > > On Thu, Dec 14, 2017 at 12:27 PM, Boespflug, Mathieu wrote: > > [snip] > >> * Or a middle ground: make feature freeze a thing. Meaning that for a >> couple of months before a major GHC release, the major new Cabal isn't >> technically released yet, but like GHC itself within this period, it's >> pretty staid, so not so much a moving target, and something downstream >> tooling authors can possibly adapt to even without any grace period on >> new metadata features. This assumes that the 2 months of feature >> freeze are enough time for downstream tooling. Thoughts from any of >> those maintainers? >> > > Short answer: if there's a clear idea in advance of when this feature freeze > is going to happen, I think we can coordinate releases of downstream tooling > (Stack being the most important, but stackage-curator playing in as well) so > that 2 months is sufficient. I'll talk with the rest of the Stack team to > see if there are any concerns. > > Longer answer: Stack intentionally avoids depending on the internals of > Cabal wherever possible. Instead of calling library functions directly from > within Haskell code to perform builds, for example, it interacts with the > Setup.hs files over their command line interface.[1] This has two results: > > * Stack can usually start using new GHC/Cabal versions without a new Stack > release, since it's just shelling out for the actual build > * There's not usually very much code churn needed in Stack to upgrade to a > newer Cabal release > > This past release was an exception because of all of the changes that > landed, both the new cabal grammar to support the ^>= operator (making the > old parser incapable of lossily parsing new files) and API changes (I think > mostly around Backpack, though there was some code cleanup as well). In > particular, the main interface we need from Cabal—the package description > data types and parser—changed significantly enough that it took significant > effort to upgrade. There were also new features added (like sub libraries > and foreign libraries) that weren't immediately supported by the old Stack > version, and had to be manually added in. > > Tying this up: generally upgrading to a new Cabal release should be fine, > and the only concern I'd have is fitting it into a release schedule with > Stack. The complications that could slow that down are: > > * Changes to the command line interface that Stack uses (hopefully those are > exceedingly rare) > * Major overhauls to the Stack-facing API > > Michael > > [1] This allows for more reproducible builds of older snapshots, insuring > that the exact same Cabal library is performing the builds From simonpj at microsoft.com Fri Dec 15 09:10:53 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Fri, 15 Dec 2017 09:10:53 +0000 Subject: Fwd: Release policies In-Reply-To: References:

<87fu8df9kf.fsf@ben-laptop.smart-cactus.org> Message-ID: | at this point in time Stackage works | hard to ensure that in any given package set, there is *exactly one* | version of any package. That's why Stackage aligns versions of core | packages to whatever ships with the GHC version the package set is | based on. Ah. It follows that if Stackage wants to find a set of packages compatible with GHC-X, then it must pick precisely the version of bytestring that GHC-X depends on. (I'm assuming here that GHC-X fixes a particular version, even though bytestring is reinstallable? Certainly, a /distribution/ of GHC-X will do so.) If meanwhile the bytestring author has decided to use a newer version of .cabal file syntax, then GHC-X is stuck with that. Or would have to go back to an earlier version of bytestring, for which there might be material disadvantages. That would make it hard to GHC to guarantee to downstream tools that it doesn't depend on any packages whose .cabal files use new syntax; which is where this thread started. Hmm. I wonder if I have understood this correctly. Perhaps Michael would like to comment? Simon | -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces at haskell.org] On Behalf Of | Boespflug, Mathieu | Sent: 14 December 2017 22:00 | To: Ben Gamari | Cc: ghc-devs | Subject: Re: Fwd: Release policies | | >> * But actually if we look at their respective release notes, GHC | >> 8.2.1 was relased in July 2017, even though the Cabal website | claims | >> that | >> Cabal-2.0.0.2 was released in August 2017 (see | >> https://www.haskell.org/cabal/download.html). So it looks like GHC | >> didn't just not give enough lead time about an upstream dependency | it | >> shipped with, it shipped with an unreleased version of Cabal! | > | > Perhaps this is true and I admit I wasn't happy about releasing the | > compiler without a Cabal release. However, there was no small amount | > of pressure to push forward nevertheless as the release was already | > quite late and the expectation was a Cabal release would be coming | > shortly after the GHC release. Coordination issues like this are a | > major reason why I think it would be better if GHC were more | decoupled from its dependencies' | > upstreams. | | I have the same sentiment. Do you think this is feasible in the case | of Cabal? Even if say something like Backpack shows up all over again? | If so, are there concrete changes that could be made to support the | following workflow: | | * upstreams develop their respective libraries independently of GHC | using their own testing. | * If they want GHC to ship a newer version, they create a Diff. As | Manuel proposed in a separate thread, this must be before feature | freeze, unless... | * ... a critical issue is found in the upstream release, in which case | upstream cuts a new release, and submits a Diff again. | * GHC always has the option to back out an offending upgrade, and | revert to a known good version. In fact it should preemptively do so | while waiting for a new release of upstream. | * In general, GHC does not track git commits of upstream dependencies | in an unknown state of quality, but tracks vetted and tested releases | instead. | | >> * GHC should never under any circumstance ship with an unreleased | >> version of any independently maintained dependency. Cabal is one | such dependency. | >> This should hold true for anything else. We could just add that | >> policy to the Release Policy. | >> | > We can adopt this as a policy, but doing so very well may mean that | > GHC will be subject to schedule slips beyond its control. We can | hope | > that upstream maintainers will be responsive, but there is little we | > can do when they are not. | | Why not? If GHC only ever tracks upstream releases (as I think it | should), not git commits in unknown state, then we don't need upstream | maintainer responsiveness. Because at any point in time, all GHC | dependencies are already released. If GHC should ship with a newer | version of a dependency, the onus is on the upstream maintainer to | submit a Diff asking GHC to move to the latest version. Are there good | reasons for GHC to track patches not upstreamed and released? | | >> * Stronger still, GHC should not switch to a new major release of a | >> dependency at any time during feature freeze ahead of a release. | E.g. | >> if | >> Cabal-3.0.0 ships before feature freeze for GHC-9.6, then maybe | it's | >> fair game to include in GHC. But not if Cabal-3.0.0 hasn't shipped | yet. | >> | > Yes, this I agree with. I think we can be more accomodating of minor | > bumps to fix bugs which may come to light during the freeze, but | major | > releases should be avoided. | | Agreed. | | >> * The 3-release backwards compat rule should apply in all | circumstances. | >> That means major version bumps of any library GHC ships with, | >> including base, should not imply any breaking change in the API's | of any such library. | >> | > I'm not sure I follow what you are suggesting here. | | Nothing new: just that the 3-release policy doesn't just apply to | base, but also anything else that happens to ship with GHC (including | Cabal). Perhaps that already the policy? | | >> * GHC does have control over reinstallable packages (like text and | >> bytestring): GHC need not ship with the latest versions of these, | if | >> indeed they introduce breaking changes that would contravene the 3- | release policy. | >> | >> * Note: today, users are effectively tied to whatever version of | the | >> packages ships with GHC (i.e. the "reinstallable" bit is | problematic | >> today for various technical reasons). That's why a breaking change | in | >> bytestring is technically a breaking change in GHC. | >> | > I don't follow: Only a small fraction of packages, namely those that | > explicitly link against the `ghc` library, are tied. Can you clarify | > what technical reasons you are referring to here? | | Builds often fail for strange reasons when both bytestring-0.10.2 and | bytestring-0.10.1 are in scope. Some libraries in a build plan pick up | one version where some pick up another. The situation here might well | be better than it used to be, but at this point in time Stackage works | hard to ensure that in any given package set, there is *exactly one* | version of any package. That's why Stackage aligns versions of core | packages to whatever ships with the GHC version the package set is | based on. | | So in this sense, AFAIK a bug in bytestring can't be worked around by | reinstalling bytestring (not in Stackage land): it requires waiting | for the next GHC version that will ship with a new version of | bytestring with that bug fixed. I'm not entirely familiar with all | Stackage details so Michael - please step in if this is incorrect. | | >> * Because there are far fewer consumers of metadata than consumers | of | >> say base, I think shorter lead time is reasonable. At the other | >> extreme, it could even be just the few months during feature | freeze. | > | > Right, I wouldn't be opposed to striving for this in principle | > although I think we should be aware that breakage is at times | > necessary and the policy should accomodate this. I think the | important | > thing is that we be aware of when we are breaking metadata | > compatibility and convey this to our users. | | That sounds reasonable. But when have we ever needed to use non | backwards compatible metadata ASAP? The integer-gmp example was a case | in point: the Cabal-2.0 feature it was using was merely syntactic | sugar at this point, since no tool *yet* interprets the new constructs | in any special way AFAIK. | | >> * The release notes bugs mentioned above and the lack of consistent | >> upload to Hackage are a symptom of lack of release automation, I | >> suspect. That's how to fix it, but we could also spell out in the | >> Release Policy that GHC libraries should all be on Hackage from the | day of release. | >> | > Yes, the hackage uploads have historically been handled manually. I | > have and AFAIK most release managers coming before me have generally | > deferred this to Herbert as is quite meticulous. However, I think it | > would be nice if we could remove the need for human intervention | entirely. | | Indeed. Can be part of the deploy step in the continuous integration | pipeline. | _______________________________________________ | ghc-devs mailing list | ghc-devs at haskell.org | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h | askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- | devs&data=04%7C01%7Csimonpj%40microsoft.com%7Cc036e34b7c204915be3708d5 | 433e1bc5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6364888564139091 | 21%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI | 6Ik1haWwifQ%3D%3D%7C- | 1&sdata=JifR49MJxw8tGDXFCmZJcNy8yd4KQyM9upeRQPIL9TI%3D&reserved=0 From mikhail.glushenkov at gmail.com Fri Dec 15 09:19:37 2017 From: mikhail.glushenkov at gmail.com (Mikhail Glushenkov) Date: Fri, 15 Dec 2017 09:19:37 +0000 Subject: [GHC DevOps Group] Fwd: Release policies In-Reply-To: References:

Message-ID: Hi Mathieu, On 15 December 2017 at 08:41, Boespflug, Mathieu wrote: > How many other > users of Cabal-the-library are there? In principle, every single > Hackage package out there, which all have a Setup.hs script. This is not such a big deal now, because build-type: Custom packages can declare the dependencies of the Setup script via the custom-setup stanza. By default (when there's no custom-setup stanza), Cabal < 2 is chosen. From mikhail.glushenkov at gmail.com Fri Dec 15 09:26:28 2017 From: mikhail.glushenkov at gmail.com (Mikhail Glushenkov) Date: Fri, 15 Dec 2017 09:26:28 +0000 Subject: [GHC DevOps Group] Fwd: Release policies In-Reply-To: References:

Message-ID: Hi Mathieu, On 15 December 2017 at 08:41, Boespflug, Mathieu wrote: > In principle, every single > Hackage package out there, which all have a Setup.hs script. Also, the build-type: Simple packages (which are the vast majority on Hackage) are not affected at all, because they all use a default built-in setup script. From michael at snoyman.com Fri Dec 15 09:27:06 2017 From: michael at snoyman.com (Michael Snoyman) Date: Fri, 15 Dec 2017 12:27:06 +0300 Subject: Fwd: Release policies In-Reply-To: References:

<87fu8df9kf.fsf@ben-laptop.smart-cactus.org> Message-ID: On Fri, Dec 15, 2017 at 12:10 PM, Simon Peyton Jones via ghc-devs < ghc-devs at haskell.org> wrote: > | at this point in time Stackage works > | hard to ensure that in any given package set, there is *exactly one* > | version of any package. That's why Stackage aligns versions of core > | packages to whatever ships with the GHC version the package set is > | based on. > > Ah. It follows that if Stackage wants to find a set of packages compatible > with GHC-X, then it must pick precisely the version of bytestring that > GHC-X depends on. (I'm assuming here that GHC-X fixes a particular > version, even though bytestring is reinstallable? Certainly, a > /distribution/ of GHC-X will do so.) > > If meanwhile the bytestring author has decided to use a newer version of > .cabal file syntax, then GHC-X is stuck with that. Or would have to go > back to an earlier version of bytestring, for which there might be material > disadvantages. > > That would make it hard to GHC to guarantee to downstream tools that it > doesn't depend on any packages whose .cabal files use new syntax; which is > where this thread started. > > Hmm. I wonder if I have understood this correctly. Perhaps Michael would > like to comment? > > > Stackage does in fact pin snapshots down to precisely one version of each package. And in the case of non-reinstallable packages, it ensures that those package's transitive dependency set are pinned to the same version that ships with GHC. I know there's work around making more package reinstallable, and the ghc package itself may have crossed that line now, but for the moment Stackage assumes that the ghc package and all its dependencies are non-reinstallable. Therefore bytestring, process, containers, transformers, and many more will be pinned in a Stackage snapshot. Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From simonpj at microsoft.com Fri Dec 15 15:23:29 2017 From: simonpj at microsoft.com (Simon Peyton Jones) Date: Fri, 15 Dec 2017 15:23:29 +0000 Subject: Fwd: Release policies In-Reply-To: References:

<87fu8df9kf.fsf@ben-laptop.smart-cactus.org> Message-ID: Therefore bytestring, process, containers, transformers, and many more will be pinned in a Stackage snapshot. So that would make it significantly harder, even impossible, for GHC releases to make any promises about the .cabal-file format of these packages, wouldn’t it? So even if we made some back-compat promise for non-reinstallable things like integer-gmp or base, we could not do so for bytestring. Does that give you cause for concern? After all, it’s where Trac #14558 started. I don’t see how we can avoid the original problem, since we don’t have control over the .cabal file format used by the authors of the packages on which we depend. Still: GHC can only depend on a package P if the version X of Cabal that GHC is using can parse P.cabal. So if we fix Cabal-X some while in advance and announce that, perhaps that would serve the purpose? Simon From: Michael Snoyman [mailto:michael at snoyman.com] Sent: 15 December 2017 09:27 To: Simon Peyton Jones Cc: Boespflug, Mathieu ; Ben Gamari ; ghc-devs Subject: Re: Fwd: Release policies On Fri, Dec 15, 2017 at 12:10 PM, Simon Peyton Jones via ghc-devs > wrote: | at this point in time Stackage works | hard to ensure that in any given package set, there is *exactly one* | version of any package. That's why Stackage aligns versions of core | packages to whatever ships with the GHC version the package set is | based on. Ah. It follows that if Stackage wants to find a set of packages compatible with GHC-X, then it must pick precisely the version of bytestring that GHC-X depends on. (I'm assuming here that GHC-X fixes a particular version, even though bytestring is reinstallable? Certainly, a /distribution/ of GHC-X will do so.) If meanwhile the bytestring author has decided to use a newer version of .cabal file syntax, then GHC-X is stuck with that. Or would have to go back to an earlier version of bytestring, for which there might be material disadvantages. That would make it hard to GHC to guarantee to downstream tools that it doesn't depend on any packages whose .cabal files use new syntax; which is where this thread started. Hmm. I wonder if I have understood this correctly. Perhaps Michael would like to comment? Stackage does in fact pin snapshots down to precisely one version of each package. And in the case of non-reinstallable packages, it ensures that those package's transitive dependency set are pinned to the same version that ships with GHC. I know there's work around making more package reinstallable, and the ghc package itself may have crossed that line now, but for the moment Stackage assumes that the ghc package and all its dependencies are non-reinstallable. Therefore bytestring, process, containers, transformers, and many more will be pinned in a Stackage snapshot. Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Fri Dec 15 15:32:13 2017 From: ben at well-typed.com (Ben Gamari) Date: Fri, 15 Dec 2017 10:32:13 -0500 Subject: [GHC DevOps Group] Fwd: Release policies In-Reply-To: References:

Message-ID: <87tvwsdn2m.fsf@ben-laptop.smart-cactus.org> "Boespflug, Mathieu" writes: > Thanks for the feedback, Michael. > > Manuel, I believe you are also a Cabal-the-library consumer in Haskell For Mac? > > Michael, you brought up another problem tangentially related to the > original integer-gmp issue but that was not in my original list > earlier in this thread: > > * Cabal-2.0.0 had breaking changes in the API. > > This means that by association GHC itself broke BC, because it shipped > with Cabal-2.0, without the usual grace period. > I'm a bit confused; by "the usual grace period" do you mean the Core Library Committee's three release policy? AFAIK this policy only applies to libraries under CLC control (e.g. those defined in the Report and perhaps template-haskell). The only other compatibility guarantee that GHC provides is the "two release policy", which stipulates that GHC should be bootstrappable with the two most recent major GHC releases. GHC has never, as far as I am aware, considered major version bumps of its dependencies to be part of its interface. We perform a major bump of most libraries with nearly every release [1]. Perhaps I've misunderstood your statement? Cheers, - Ben [1] https://ghc.haskell.org/trac/ghc/wiki/Commentary/Libraries/VersionHistory -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From m at tweag.io Fri Dec 15 17:37:18 2017 From: m at tweag.io (Boespflug, Mathieu) Date: Fri, 15 Dec 2017 18:37:18 +0100 Subject: [GHC DevOps Group] Fwd: Release policies In-Reply-To: <87tvwsdn2m.fsf@ben-laptop.smart-cactus.org> References: