From b at chreekat.net Mon Mar 1 06:06:10 2021 From: b at chreekat.net (Bryan Richter) Date: Mon, 1 Mar 2021 08:06:10 +0200 Subject: [RFC] Qualified module renamings In-Reply-To: <1614542926073.30694@mit.edu> References: <1614269634603.68860@mit.edu> <1614286880156.80270@mit.edu> <1614542926073.30694@mit.edu> Message-ID: Thanks. :) Den sön 28 feb. 2021 22:08Edward Z Yang skrev: > ​I added this to the PR! Hopefully it is what you are looking for. > ------------------------------ > *From:* Bryan Richter > *Sent:* Thursday, February 25, 2021 10:44 PM > *To:* Edward Z Yang > *Cc:* cabal-devel at haskell.org; ekmett at gmail.com > *Subject:* Re: [RFC] Qualified module renamings > > Thanks for the context! > > By intent, I meant an example like "Here's what it would like now, and > here's what it would look like with this implemented. For a much longer > example with rationale, see [Kmett's comment]" > > > Den tors 25 feb. 2021 23:01Edward Z Yang skrev: > >> > It took me about five minutes to arrive at the guess that this is >> about the syntax in Cabal files for using backpack - is that right? >> >> >> Oops yes, sorry for omitting this context. >> >> >> > What is the intent of what got implemented, anyway? Are there example >> use cases? >> >> >> I'm not exactly sure what you mean by intent. But a common pattern in >> Backpack is to instantiate a library multiple time with different >> requirements, and if you want them all in scope you have to rename them. >> Right now, this has to be done one-by-one for each provided module, which >> can be a bit annoying. For example, let say you parametrized parsec by >> string type, and you wanted a bytestring version and a text version, it >> would be convenient to be able to unconditionally rename every >> Text.Parsec.* module to Text.Parsec.*.ByteString (for example) >> >> >> Edward Kmett described a concrete motivating use case at >> https://github.com/haskell/cabal/issues/7290#issuecomment-783540208​ >> although his use case is a little difficult to understand. >> >> >> Edward >> >> >> ------------------------------ >> *From:* Bryan Richter >> *Sent:* Thursday, February 25, 2021 10:06 AM >> *To:* Edward Z Yang >> *Cc:* cabal-devel at haskell.org; ekmett at gmail.com >> *Subject:* Re: [RFC] Qualified module renamings >> >> It took me about five minutes to arrive at the guess that this is about >> the syntax in Cabal files for using backpack - is that right? >> >> What is the intent of what got implemented, anyway? Are there example use >> cases? >> >> Den tors 25 feb. 2021 18:14Edward Z Yang skrev: >> >>> Today, using the 'mixins' field you can rename modules that come from >>> other packages by manually expressing a renaming one-by-one. In some >>> Backpack use cases, you may have a lot of modules that you would like to >>> mechanically rename into some subnamespace; today, you have manually list >>> each renaming one by one. >>> >>> >>> https://github.com/haskell/cabal/pull/7303 contains an implementation >>> of one possible way to extend mixin syntax to support qualified renaming; >>> the implementation is very simple. The syntax here is based off of Richard >>> Eisenberg's local modules proposal ( >>> https://github.com/ghc-proposals/ghc-proposals/pull/283) which supports >>> the qualified keyword before module exports/imports which has the same >>> effect (bring the module into scope under a sub-module namespace). However, >>> the PR isn't really meant to be an end all to the discussion: it's just to >>> show that it's pretty simple to implement this functionality. >>> >>> >>> There are two primary axes which I am looking for feedback: >>> >>> >>> * Expressivity. The current PoC implementation only permits >>> unconditionally prefix-ing all modules that would have been brought into >>> scope by the mixin; e.g., transforming module A to Prefix.A. Edward Kmett >>> has expressed that in some cases, he would like it if you could implement >>> the import as a suffix. One could also imagine allowing arbitrary string >>> transformations. Opinions on where to draw the line for expressivity are >>> solicited. >>> >>> >>> * Syntax. The current syntax is "pkgname qualified Prefix" as it is >>> symmetric with "pkgname hiding (A, B)" and it was simple to implement. But >>> I am not particularly attached to this syntax, and am open to other >>> suggestions. If we permit suffixing, a wildcard based syntax like "pkgname >>> (* as *.Suffix)" may be preferable (though modestly more complex to specify >>> and implement; for example, is the glob recursive over dots?). Edward Kmett >>> has offered some other possibilities at >>> https://github.com/haskell/cabal/issues/7290#issue-812744575​ >>> >>> >>> Thanks Oleg for reminding me to send this RFC to this mailing list. >>> >>> >>> Cheers, >>> >>> Edward >>> _______________________________________________ >>> cabal-devel mailing list >>> cabal-devel at haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From fgaz at fgaz.me Mon Mar 1 10:31:17 2021 From: fgaz at fgaz.me (Francesco Gazzetta) Date: Mon, 1 Mar 2021 11:31:17 +0100 Subject: [RFC] Qualified module renamings In-Reply-To: <1614541058608.47518@mit.edu> References: <1614269634603.68860@mit.edu> <1614286880156.80270@mit.edu> <465389bf-96f3-523e-e076-dd9009930299@iki.fi> <1614541058608.47518@mit.edu> Message-ID: <20210301113117.7a6f1429@phi> On Sun, 28 Feb 2021 19:37:38 +0000 Edward Z Yang wrote: > As a more general thought: I noticed in a few other situations where > traditional PVP does a poor job at addressing situations where API > stability depends on packages you yourself depend on. For example, > let's say I wanted to make a metapackage that took a few packages and > reexported their modules. You don't even have to talk about the > module level: at the declaration level, what identifiers are > available from this package will depend on the version choices of its > dependencies. But it's infeasible to publish combinatorially many > versions of the metapackage for each possible version of its > dependency Revisions make this a bit more bearable. But still, backpack aside, I'd say that cross-package module reexports are an antipattern exactly for this reason! From gergo at erdi.hu Wed Mar 3 01:10:28 2021 From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=) Date: Wed, 3 Mar 2021 09:10:28 +0800 (+08) Subject: Interleaving per-component hooks with build Message-ID: Hi, I am looking for a way to do custom preprocessing for each component (of type `UnitId -> IO ()`) but this preprocessing depends on the dependencies of the component being already built. So imagine a Cabal file which defines a library and an exe, with the exe depending on the library. Then, I would need the following sequence of events to happen during `buildHook`: 1. myPreprocessing unitIdOfLibrary 2. standard build of library 3. myPreprocessing unitIdOfExe 4. standard build of exe How can I achieve that without copy-pasting large parts of the internals of `buildHook simpleUserHooks`? Thanks, Gergo From gergo at erdi.hu Sun Mar 14 11:15:00 2021 From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=) Date: Sun, 14 Mar 2021 19:15:00 +0800 (+08) Subject: Interleaving per-component hooks with build In-Reply-To: References: Message-ID: Hi, I've made some progress on this, but I still have some questions. My new idea is to use withAllComponentsInBuildOrder to do the preprocessing before handing it over to buildHook simpleUserHooks. So then the next missing piece of the puzzle is how to arrange for buildHook simpleUserHooks to only build a single component: ``` myBuildHook pkg localInfo userHooks buildFlags = do withAllComponentsInBuildOrder pkg localInfo $ \c clbi -> do localInfo <- return $ restrictLocalInfo c clbi localInfo buildFlags <- return $ restrictBuildFlags c buildFlags bi <- preprocessBuildInfo localInfo buildFlags c pkg <- return $ updateBuildInfo c bi pkg buildHook simpleUserHooks pkg localInfo userHooks buildFlags ``` All the magic is supposed to happen in `restrictLocalInfo` and `restrictBuildFlags` to restrict the `LocalInfo` and the `BuildFlags` so that `buildHook simpleUserHooks` will only build the single given component. My first idea was to edit the `componentEnableSpec` / `componentGraph` / `componentNameMap` to be singleton lists: ``` restrictLocalInfo :: Component -> ComponentLocalBuildInfo -> LocalBuildInfo -> LocalBuildInfo restrictLocalInfo c clbi localInfo = localInfo { componentEnabledSpec = OneComponentRequestedSpec $ componentName c , componentGraph = G.insert clbi G.empty , componentNameMap = M.singleton (componentName c) [clbi] } ``` If I try this, building fails if some components are explicitly requested: setup: Cannot process the executable 'compucolor2-cpu-screen' because this package was configured only to build lib. Re-run configure with the argument executable 'compucolor2-cpu-screen' So next idea was to also delete the `buildArgs`: ``` restrictBuildFlags :: Component -> BuildFlags -> BuildFlags restrictBuildFlags c buildFlags = buildFlags { buildArgs = [] } ``` This gets us over building the lib, but now the exes fail: Preprocessing executable 'compucolor2-cpu-textscreen' for compucolor2-0.1.0.. Building executable 'compucolor2-cpu-textscreen' for compucolor2-0.1.0.. : cannot satisfy -package-id compucolor2-0.1.0-1E5h9nTsLCR9N8r4sKLlg5 (use -v for more information) My other idea is to NOT change the LocalInfo, and instead only edit `buildArgs` to contain only the current component. I haven't managed to get this quite right either: * If I only put the current component's name (via `prettyShow . componentName`) in the `buildArgs` list, then that fails for libraries: `prettyShow . componentName` for the library component is `"lib"`, but it seems that Cabal actually expects `"lib:packagename"` as the argument. * If I hack it by hardcoding the addition of `"lib:mylibname"` to the `buildArgs`, then the library is built and re-built and re-built for each component again and again. Of course, these "builds" finish quickly because nothing has changed, but it's still a nuisance. * The whole point of this exercise is to have access to already built libraries when preprocessing executables. But it seems that that is not as simple as adding `absolutePackaBgeDBPaths $ withPackageDB localInfo` to the package DB path when building, since that directory doesn't have a pkgconf file for the just-built library. So how do I use just-built dependency libraries? Thanks, Gergo On Wed, 3 Mar 2021, ÉRDI Gergő wrote: > I am looking for a way to do custom preprocessing for each component (of type > `UnitId -> IO ()`) but this preprocessing depends on the dependencies of the > component being already built. So imagine a Cabal file which defines a > library and an exe, with the exe depending on the library. Then, I would need > the following sequence of events to happen during `buildHook`: > > 1. myPreprocessing unitIdOfLibrary > 2. standard build of library > 3. myPreprocessing unitIdOfExe > 4. standard build of exe > > How can I achieve that without copy-pasting large parts of the internals of > `buildHook simpleUserHooks`? From gergo at erdi.hu Mon Mar 15 11:25:32 2021 From: gergo at erdi.hu (=?ISO-8859-2?Q?=C9RDI_Gerg=F5?=) Date: Mon, 15 Mar 2021 19:25:32 +0800 (+08) Subject: Interleaving per-component hooks with build In-Reply-To: References: Message-ID: Let's put this in more concrete terms. Here is a simple model of my Setup.hs: https://gist.github.com/gergoerdi/2cdc4d984ddf039d650f6a98f2508a96 Here, I hope to run `justBeforeBuilding` for each `Component` just before building them (the actual building is to be done by `buildHook simpleUserHooks`). I print the package DBs, and their contents, in `justBeforeBuilding`. With a simple package that contains a library and an exe that depends on the library, the output is the following: ``` !!! Processing component CLibName LMainLibName !!! At this point, the package DB paths are: /home/cactus/prog/clash/bugs/cabal-component-hook/.stack-work/install/x86_64-linux-tinfo6/82781a0829e0e0da301c1db4825858dea8980a6a982b679eb178870c6c0ec1ee/8.10.4/pkgdb package.cache.lock package.cache /home/cactus/sdk/stack/snapshots/x86_64-linux-tinfo6/82781a0829e0e0da301c1db4825858dea8980a6a982b679eb178870c6c0ec1ee/8.10.4/pkgdb (snip) Preprocessing library for cabal-component-hook-0.1.0.. Building library for cabal-component-hook-0.1.0.. !!! Processing component CExeName (UnqualComponentName "bar") !!! At this point, the package DB paths are: /home/cactus/prog/clash/bugs/cabal-component-hook/.stack-work/install/x86_64-linux-tinfo6/82781a0829e0e0da301c1db4825858dea8980a6a982b679eb178870c6c0ec1ee/8.10.4/pkgdb package.cache.lock package.cache /home/cactus/sdk/stack/snapshots/x86_64-linux-tinfo6/82781a0829e0e0da301c1db4825858dea8980a6a982b679eb178870c6c0ec1ee/8.10.4/pkgdb package.cache.lock (snip) Preprocessing library for cabal-component-hook-0.1.0.. Building library for cabal-component-hook-0.1.0.. Preprocessing executable 'bar' for cabal-component-hook-0.1.0.. Building executable 'bar' for cabal-component-hook-0.1.0.. ``` After the whole process finishes, I do see my library's file in the package DB: ``` /home/cactus/prog/clash/bugs/cabal-component-hook/.stack-work/install/x86_64-linux-tinfo6/82781a0829e0e0da301c1db4825858dea8980a6a982b679eb178870c6c0ec1ee/8.10.4/pkgdb ├── cabal-component-hook-0.1.0-8kPvy0LMfqKAbvXmH5zWaP.conf ├── package.cache └── package.cache.lock ``` So my problem is that the package DB returned by `absolutePackageDBPaths $ withPackageDB localInfo` doesn't contain the just-build library, even though (supposedly) the library has already been built by that point. And then there is this mysterious second occurrence of "Preprocessing library for cabal-component-hook-0.1.0..Building library for cabal-component-hook-0.1.0.." while building (supposedly) just the exe. What's going on here? How can I arrange for `justBeforeBuilding` to have access to dependency libraries? Thanks, Gergo On Sun, 14 Mar 2021, ÉRDI Gergő wrote: > Hi, > > I've made some progress on this, but I still have some questions. My new idea > is to use withAllComponentsInBuildOrder to do the preprocessing before > handing it over to buildHook simpleUserHooks. So then the next missing piece > of the puzzle is how to arrange for buildHook simpleUserHooks to only build a > single component: > > ``` > myBuildHook pkg localInfo userHooks buildFlags = do > withAllComponentsInBuildOrder pkg localInfo $ \c clbi -> do > localInfo <- return $ restrictLocalInfo c clbi localInfo > buildFlags <- return $ restrictBuildFlags c buildFlags > bi <- preprocessBuildInfo localInfo buildFlags c > pkg <- return $ updateBuildInfo c bi pkg > buildHook simpleUserHooks pkg localInfo userHooks buildFlags > ``` > > All the magic is supposed to happen in `restrictLocalInfo` and > `restrictBuildFlags` to restrict the `LocalInfo` and the `BuildFlags` so that > `buildHook simpleUserHooks` will only build the single given component. > > My first idea was to edit the `componentEnableSpec` / `componentGraph` / > `componentNameMap` to be singleton lists: > > ``` > restrictLocalInfo :: Component -> ComponentLocalBuildInfo -> LocalBuildInfo > -> LocalBuildInfo > restrictLocalInfo c clbi localInfo = localInfo > { componentEnabledSpec = OneComponentRequestedSpec $ componentName c > , componentGraph = G.insert clbi G.empty > , componentNameMap = M.singleton (componentName c) [clbi] > } > ``` > > If I try this, building fails if some components are explicitly requested: > > setup: Cannot process the executable 'compucolor2-cpu-screen' because this > package was configured only to build lib. Re-run configure with the argument > executable 'compucolor2-cpu-screen' > > > So next idea was to also delete the `buildArgs`: > > ``` > restrictBuildFlags :: Component -> BuildFlags -> BuildFlags > restrictBuildFlags c buildFlags = buildFlags > { buildArgs = [] > } > ``` > > This gets us over building the lib, but now the exes fail: > > Preprocessing executable 'compucolor2-cpu-textscreen' for compucolor2-0.1.0.. > Building executable 'compucolor2-cpu-textscreen' for compucolor2-0.1.0.. > : cannot satisfy -package-id > compucolor2-0.1.0-1E5h9nTsLCR9N8r4sKLlg5 > (use -v for more information) > > > My other idea is to NOT change the LocalInfo, and instead only edit > `buildArgs` to contain only the current component. I haven't managed to get > this quite right either: > > * If I only put the current component's name (via `prettyShow . > componentName`) > in the `buildArgs` list, then that fails for libraries: `prettyShow . > componentName` for the library component is `"lib"`, but it seems that Cabal > actually expects `"lib:packagename"` as the argument. > > * If I hack it by hardcoding the addition of `"lib:mylibname"` to the > `buildArgs`, then the library is built and re-built and re-built for each > component again and again. Of course, these "builds" finish quickly because > nothing has changed, but it's still a nuisance. > > * The whole point of this exercise is to have access to already built > libraries when preprocessing executables. But it seems that that is not as > simple as adding `absolutePackaBgeDBPaths $ withPackageDB localInfo` to the > package DB path when building, since that directory doesn't have a pkgconf > file for the just-built library. So how do I use just-built dependency > libraries? > > Thanks, > Gergo > > > > On Wed, 3 Mar 2021, ÉRDI Gergő wrote: > >> I am looking for a way to do custom preprocessing for each component (of >> type `UnitId -> IO ()`) but this preprocessing depends on the dependencies >> of the component being already built. So imagine a Cabal file which defines >> a library and an exe, with the exe depending on the library. Then, I would >> need the following sequence of events to happen during `buildHook`: >> >> 1. myPreprocessing unitIdOfLibrary >> 2. standard build of library >> 3. myPreprocessing unitIdOfExe >> 4. standard build of exe >> >> How can I achieve that without copy-pasting large parts of the internals of >> `buildHook simpleUserHooks`? -- .--= ULLA! =-----------------. \ http://gergo.erdi.hu \ `---= gergo at erdi.hu =-------' Marijuana... because your friends just aren't that funny From nicolas.frisby at gmail.com Fri Mar 19 20:20:24 2021 From: nicolas.frisby at gmail.com (Nicolas Frisby) Date: Fri, 19 Mar 2021 13:20:24 -0700 Subject: plan.json documentation Message-ID: Hi all. Without show-build-info yet and without a cabal-install library, a hobby project relies on parsing plan.json to access eg the list of .cabal files and their flags as configured during the most recent cabal-install invocation. I haven't found much documentation about plan.json. I'm particularly curious what are its current semantics and intended use, and if these are intended to remain stable going forward. Some example specific questions: * Would show-build-info supplant plan.json? If so, would there be a deprecation delay? * The docs say "A JSON serialization of the computed install plan intended for integrating cabal with external tooling." It seems likely that answering my questions will involve explaining what an "install plan" is. (I'm unfortunately unfamiliar with Cabal internals, and ReadTheDocs search doesn't seem to support phrases as queries.) * What determines which packages have entries in the plan.json file after a particular cabal-install invocation? Will it always include all local .cabal files? (🤞) * Is there enough information in the plan.json file in order to map an entry there for source repository package to the corresponding path under the builddir's `src` directory? There seems to be some sort of hash involved in that path and I haven't figured what its inputs are and whether the plan.json file includes them all. Thank you for your time. -Nick PS - My hobby project driving this question is based on wanting to parse all local source files without having to typecheck/build any of them (ideally nor their non-local dependencies). I need each source file path and corresponding language extensions etc in order to invoke the parser. (hie-bios doesn't suffice because its solution to this problem involves building dependencies, possibly including local packages.) If you see a more direct path that doesn't involve plan.json, please say so. Thanks again! -------------- next part -------------- An HTML attachment was scrubbed... URL: From oleg.grenrus at iki.fi Fri Mar 19 20:36:36 2021 From: oleg.grenrus at iki.fi (Oleg Grenrus) Date: Fri, 19 Mar 2021 22:36:36 +0200 Subject: plan.json documentation In-Reply-To: References: Message-ID: A short answer: Look how cabal-docspec is implemented. It does do what you want (i.e. finds all source files) to do its job too. A slightly longer version: - cabal-plan library parses plan.json, its types tell you what is there [1] - plan doesn't include location of .cabal files directly, but it has PkgLoc. From there you can find .cabal files - in cabal-docspec (and cabal-hie) for that matter, I simplify GPD parts using flags information in plan.json to figure out exactly which modules are in play     - this part is too complicated, can probably be done simpler with finalizePD [2] - cabal-docspec uses utilities in cabal-extras repository to figure out GHC information for the previous step. It's not simple, you need to combine parts, but all pieces are there. - Oleg [1]: https://hackage.haskell.org/package/cabal-plan-0.7.2.0/docs/Cabal-Plan.html [2]: https://hackage.haskell.org/package/Cabal-3.4.0.0/docs/Distribution-PackageDescription-Configuration.html#v:finalizePD On 19.3.2021 22.20, Nicolas Frisby wrote: > Hi all. Without show-build-info yet and without a cabal-install > library, a hobby project relies on parsing plan.json to access eg the > list of .cabal files and their flags as configured during the most > recent cabal-install invocation. > > I haven't found much documentation about plan.json. I'm particularly > curious what are its current semantics and intended use, and if these > are intended to remain stable going forward. > > Some example specific questions: > >   * Would show-build-info supplant plan.json? If so, would there be a > deprecation delay? > >   * The docs say "A JSON serialization of the computed install plan > intended for integrating cabal with external tooling." It seems likely > that answering my questions will involve explaining what an "install > plan" is. (I'm unfortunately unfamiliar with Cabal internals, and > ReadTheDocs search doesn't seem to support phrases as queries.) > >   * What determines which packages have entries in the plan.json file > after a particular cabal-install invocation? Will it always include > all local .cabal files? (🤞) > >   * Is there enough information in the plan.json file in order to map > an entry there for source repository package to the corresponding path > under the builddir's `src` directory? There seems to be some sort of > hash involved in that path and I haven't figured what its inputs are > and whether the plan.json file includes them all. > > Thank you for your time. -Nick > > PS - My hobby project driving this question is based on wanting to > parse all local source files without having to typecheck/build any of > them (ideally nor their non-local dependencies). I need each source > file path and corresponding language extensions etc in order to invoke > the parser. (hie-bios doesn't suffice because its solution to this > problem involves building dependencies, possibly including local > packages.) > > If you see a more direct path that doesn't involve plan.json, please > say so. Thanks again! > > _______________________________________________ > cabal-devel mailing list > cabal-devel at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: