From cabal at stefan-klinger.de Sun Jul 18 14:46:56 2021 From: cabal at stefan-klinger.de (Stefan Klinger) Date: Sun, 18 Jul 2021 16:46:56 +0200 Subject: There are two sorts of auto-generated files Message-ID: Hi, I, too, ran into the requirement to have a compiled Haskell binary print the Git hash it was derived from. There's a package for that (gitrev), which does not work to my satisfaction. Investigation led me to a wider concept. I now think that Cabal should discern two sorts of generated files: 1. Early generated files, that are not stored in the source repository but have to be present in a source distribution created by `cabal sdist`. 2. Late generated files, that are also not in the source distribution but are created during compilation. If that was possible, then the git hash problem would reduce to a special case of the first sort. (if it already is, where would I find it documented?) I've elaborated on this a bit, and provide my current attempt to include a git hash into a binary at https://github.com/s5k6/versionInfo which demonstrates the current issues when trying so implement such an approach using current Cabal. It also demonstrates the issue of the current auto-generated `Paths_…` module depending on the `base` package. What do you think? Cheers Stefan -- Stefan Klinger, Ph.D. -- computer scientist o/X http://stefan-klinger.de /\/ https://github.com/s5k6 \ I prefer receiving plain text messages, not exceeding 32kB. From mikolaj at well-typed.com Mon Jul 19 06:28:19 2021 From: mikolaj at well-typed.com (Mikolaj Konarski) Date: Mon, 19 Jul 2021 08:28:19 +0200 Subject: There are two sorts of auto-generated files In-Reply-To: References: Message-ID: > Hi, Hi Stefan, > I, too, ran into the requirement to have a compiled Haskell binary > print the Git hash it was derived from. There's a package for that > (gitrev), which does not work to my satisfaction. I can see the problem is with `cabal v2-install` breaking gitrev. https://github.com/acfoltzer/gitrev/issues/23 That's unfortunate, but why not use the normal workflow, that is, `cabal build`, possibly with `cabal list-bin`? > Investigation led me to a wider concept. I now think that Cabal > should discern two sorts of generated files: > > 1. Early generated files, that are not stored in the source > repository but have to be present in a source distribution created > by `cabal sdist`. > > 2. Late generated files, that are also not in the source distribution > but are created during compilation. > > If that was possible, then the git hash problem would reduce to a > special case of the first sort. (if it already is, where would I find > it documented?) > > I've elaborated on this a bit, and provide my current attempt to > include a git hash into a binary at > > https://github.com/s5k6/versionInfo > > which demonstrates the current issues when trying so implement such an > approach using current Cabal. This seems a nice workaround for the bug with `cabal v2-install` and `gitrev`. However, this adds complexity, so somebody would need to implement this and then somebody would need to maintain it, preferably a group of interested users, unless this can be shown to solve/simplify other major things in cabal. Would you organize such an interest group, brainstorm and coordinate with cabal maintainers (this list is fine initially, but a ticket would help in the long term)? I guess some of the users or contributors of `gitrev` may be interested. > It also demonstrates the issue of the > current auto-generated `Paths_…` module depending on the `base` > package. Why is this an issue? RIO depends on `base`, too. If the worry is that other modules may be mistake start depending on base, perhaps compile Paths in a separate internal library that depends on `base` while the rest of your package demonstrably doesn't? Kind regards, Mikolaj From cabal at stefan-klinger.de Mon Jul 19 18:26:23 2021 From: cabal at stefan-klinger.de (Stefan Klinger) Date: Mon, 19 Jul 2021 20:26:23 +0200 Subject: There are two sorts of auto-generated files In-Reply-To: References: Message-ID: Hello Mikolaj, thanks for getting back to me so quickly. Mikolaj Konarski (2021-Jul-19, excerpt): > https://github.com/acfoltzer/gitrev/issues/23 > > That's unfortunate, but why not use the normal workflow, > that is, `cabal build`, possibly with `cabal list-bin`? Hmmmm, I'm sorry, I don't understand what you refer to. There might be a normal workflow that I'm not aware of. Are you hinting at not using `cabal install` at all, and just grab he binary? Would that be a smart approach? No judgement here, I honestly don't know, but why have `cabal install` at all then? How would that scale to tools that need installation (maybe for setting up data files)? > > https://github.com/s5k6/versionInfo > > This seems a nice workaround for the bug with `cabal v2-install` and > `gitrev`. However, this adds complexity, Oh yes, indeed. The more I think of it, the more natural the distinction between early and late generated code appears to me. But I do understand that there are others with a much deeper understanding of the build process and the implications of my approach. > Would you organize such an interest group, brainstorm and coordinate > with cabal maintainers That's what I'm trying to do here, get brainstorming from the people who would know. > (this list is fine initially, but a ticket would help in the long > term)? I guess some of the users or contributors of `gitrev` may be > interested. ...so the plan would be to raise an issue on https://github.com/haskell/cabal/issues and then point, e.g., the gitrev maintainers to it? Yes, I can do that. Any requirements against how to set up the ticket? (after checking: I think I can only file bug reports. Is that ok?) > > It also demonstrates the issue of the > > current auto-generated `Paths_…` module depending on the `base` > > package. > > Why is this an issue? RIO depends on `base`, too. I was just thinking that generalising the idea of late generated code might encompass the generation of the `Paths_…` module. Giving control over its generation to the developer would also remove that complexity from Cabal. This could be done via a simple developer-provided template that is used by Cabal to create the module. One could use Template Haskell, but I think even a blunt text-based approach would do, in less than 100 SLOC. > If the worry is that other modules may be mistake start depending on > base, perhaps compile Paths in a separate internal library that > depends on `base` while the rest of your package demonstrably > doesn't? Yeah, that sounds like a very good idea (unfortunately, in my real project I depend on `base` for other reasons, but I'll give it a try next time I see the opportunity). Thanks for the hint! Cheers Stefan -- Stefan Klinger, Ph.D. -- computer scientist o/X http://stefan-klinger.de /\/ https://github.com/s5k6 \ I prefer receiving plain text messages, not exceeding 32kB. From mikolaj at well-typed.com Tue Jul 20 16:26:06 2021 From: mikolaj at well-typed.com (Mikolaj Konarski) Date: Tue, 20 Jul 2021 18:26:06 +0200 Subject: There are two sorts of auto-generated files In-Reply-To: References: Message-ID: Hi, > > That's unfortunate, but why not use the normal workflow, > > that is, `cabal build`, possibly with `cabal list-bin`? > > Hmmmm, I'm sorry, I don't understand what you refer to. There might > be a normal workflow that I'm not aware of. Are you hinting at not > using `cabal install` at all, and just grab he binary? Would that be > a smart approach? Depends. I'm not particularly up to date with how people use cabal nowadays, but I think `cabal build` is the default, `cabal run` is for when needing to run something once and `cabal install` mostly for installing build tools that will be used often, but these should often be installed automatically one way or another. > No judgement here, I honestly don't know, but why > have `cabal install` at all then? How would that scale to tools that > need installation (maybe for setting up data files)? I'm told data files generally don't work (probably too much variety in directory schemes among OSes and even their versions). So either embed your data or set up a fixed known directory with them via other means. > The more I think of it, the more natural the > distinction between early and late generated code appears to me. But > I do understand that there are others with a much deeper understanding > of the build process and the implications of my approach. That's one more reason not to lose the idea but open a ticket. I don't have such a deep understanding at this time. And mailing lists are sadly out of fashion generally. > ...so the plan would be to raise an issue on > > https://github.com/haskell/cabal/issues > > and then point, e.g., the gitrev maintainers to it? Yes, I can do > that. Yes, please. > Any requirements against how to set up the ticket? (after > checking: I think I can only file bug reports. Is that ok?) I guess nobody has the time to work on feature requests (or on updating the ticket template). However, your feature request is welcome, given the thought you've already put into it and the potential of gathering enough interested users to analyze and implement some form of it. > I was just thinking that generalising the idea of late generated code > might encompass the generation of the `Paths_…` module. Giving > control over its generation to the developer would also remove that > complexity from Cabal. This could be done via a simple > developer-provided template that is used by Cabal to create the > module. One could use Template Haskell, but I think even a blunt > text-based approach would do, in less than 100 SLOC. Sounds interesting. I'm sure I wouldn't be able to see obvious problems with that, if there were any, but quite possibly there was just never enough bandwidth nor determined enough users. Cheers, Mikolaj From cabal at stefan-klinger.de Wed Jul 21 17:08:37 2021 From: cabal at stefan-klinger.de (Stefan Klinger) Date: Wed, 21 Jul 2021 19:08:37 +0200 Subject: There are two sorts of auto-generated files In-Reply-To: References: Message-ID: Hello Mikolaj! Mikolaj Konarski (2021-Jul-20, excerpt): > > ...so the plan would be to raise an issue > Yes, please. Done. https://github.com/haskell/cabal/issues/7488 Thanks for the encouragement, let's hope for the best! Kind regards Stefan -- Stefan Klinger, Ph.D. -- computer scientist o/X http://stefan-klinger.de /\/ https://github.com/s5k6 \ I prefer receiving plain text messages, not exceeding 32kB.