<div dir="ltr">Hi Herbert,<div><br></div><div>This sounds like a great idea :) Glad to see ideas from Eta inspiring new ideas in GHC. </div><div><br></div><div>"<span style="font-size:13px">constantly nagging upstream maintainers to release quickfixes</span></div><span style="font-size:13px">for unreleased GHC HEAD snapshots which will likely break again a few</span><br style="font-size:13px"><span style="font-size:13px">weeks later"</span><div><br><div>^ This is what motivated us to start eta-hackage in the first place. In our case, we didn't know whether the patches were completely correct when porting C FFI to Java FFI, at least in the corner cases, so we needed a way to quickly deploy updates to patches without having the end user wait a significant amount of time. Once they stabilise, we do plan on pushing the patches upstream to the respective packages - we've already done that for a few of them.</div><div><br></div><div>Thanks,</div></div><div>Rahul</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Sep 18, 2017 at 8:04 AM, Herbert Valerio Riedel <span dir="ltr"><<a href="mailto:hvriedel@gmail.com" target="_blank">hvriedel@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi GHC devs,<br>
<br>
A long-standing common problem when testing/dogfooding GHC HEAD is that<br>
at some point during the development cycle more and more packages from<br>
Hackage will run into build failures.<br>
<br>
Obviously, constantly nagging upstream maintainers to release quickfixes<br>
for unreleased GHC HEAD snapshots which will likely break again a few<br>
weeks later (as things are generally in flux until freeze comes into<br>
effect) does not scale and only introduces a latency/coordination<br>
bottleneck, and on top of it ultimately results in spamming the primary<br>
Hackage Package index with releases (which has non-negligible negative<br>
impact/costs of its own on the Hackage infrastructure, and thus ought to<br>
be minimised).<br>
<br>
OTOH, we need the ability to easily test, debug, profile, and prototype<br>
changes to GHC HEAD while things are still in motion, and case in point,<br>
if you try to e.g. build `pandoc` with GHC HEAD today, you'll currently<br>
run into a dozen or so of packages not building with GHC HEAD.<br>
<br>
To this end, I've finally found time to work on a side-project (related<br>
to <a href="http://matrix.hackage.haskell.org" rel="noreferrer" target="_blank">matrix.hackage.haskell.org</a>) which implements a scheme tailored to<br>
`cabal new-build`, which is inspired by how Eta copes with a very<br>
related issue (they rely on it for stable versions of the compiler);<br>
i.e., they maintain a set of patches at<br>
<br>
<a href="https://github.com/typelead/eta-hackage/tree/master/patches" rel="noreferrer" target="_blank">https://github.com/typelead/<wbr>eta-hackage/tree/master/<wbr>patches</a><br>
<br>
which fix up existing Hackage packages to work with the Eta compiler.<br>
<br>
<br>
And this gave me the idea to use a similar scheme for GHC HEAD:<br>
<br>
<a href="https://github.com/hvr/head.hackage/tree/master/patches" rel="noreferrer" target="_blank">https://github.com/hvr/head.<wbr>hackage/tree/master/patches</a><br>
<br>
This folder already contains several of patches (which mostly originate<br>
from Ryan, Ben and myself) to packages which I needed to patch in order<br>
to build popular Hackage packages & tools.<br>
<br>
The main difference is how those patches are applied; Eta uses a<br>
modified `cabal` which they renamed to `etlas` which is checks<br>
availability of .patch & .cabal files in the GitHub repo linked above;<br>
<br>
Whereas for GHC HEAD with `cabal new-build` a different scheme makes<br>
more sense: we simply generate an add-on Hackage repo, and use the<br>
existing `cabal` facilities (e.g. multi-repo support or the nix-style<br>
package store which makes sure that unofficially patched packages don't<br>
contaminate "normal" install-plans, etc.) to opt into the opt-in Hackage<br>
repo containing fixed up packages.<br>
<br>
<br>
<br>
I've tried to describe how to use the HEAD.hackage add-on repo in the<br>
README at<br>
<br>
<a href="https://github.com/hvr/head.hackage#how-to-use" rel="noreferrer" target="_blank">https://github.com/hvr/head.<wbr>hackage#how-to-use</a><br>
<br>
<br>
And finally, here's a practical example of how you can use it to build<br>
e.g. the `pandoc` executable with GHC HEAD (can easily be adapted to<br>
build your project of choice; please refer to<br>
<br>
<a href="http://cabal.readthedocs.io/en/latest/nix-local-build-overview.html" rel="noreferrer" target="_blank">http://cabal.readthedocs.io/<wbr>en/latest/nix-local-build-<wbr>overview.html</a><br>
<br>
to learn more about how to describe your project via `cabal.project`<br>
files):<br>
<br>
<br>
0.) This assumes you have a recent cabal 2.1 snapshot built from Git<br>
<br>
1.) create & `cd` into a new work-folder<br>
<br>
2.) invoke `<a href="http://head.hackage.sh" rel="noreferrer" target="_blank">head.hackage.sh</a> update` to update the HEAD.hackage index cache<br>
<br>
3.) invoke `<a href="http://head.hackage.sh" rel="noreferrer" target="_blank">head.hackage.sh</a> init` to create an initial `cabal.project`<br>
configuration which locally activates the HEAD.hackage overlay repo<br>
<br>
4.) If needed, edit the cabal.project file and change where GHC<br>
HEAD can be found (the script currently assumes GHC HEAD is<br>
installed from my Ubuntu PPA), e.g.<br>
<br>
with-compiler: /home/hvr/src/ghc-dev/inplace/<wbr>bin/ghc-stage2<br>
<br>
or you can add something like `optional-packages: deps/*/*.cabal`<br>
to have cabal pick up package source-trees unpacked in the deps/<br>
folder, or you can inject ghc-options, relax upper bounds via<br>
`allow-newer: *:base` etc (please refer to the cabal user guide)<br>
<br>
5.) Create a `dummy.cabal` file (in future we will have `cabal<br>
new-install` or other facilities, but for now we use this<br>
workaround):<br>
<br>
<br>
--8<---------------cut here---------------start------<wbr>------->8---<br>
name: dummy<br>
version: 0<br>
build-type: Simple<br>
cabal-version: >=2.0<br>
<br>
library<br>
default-language: Haskell2010<br>
<br>
-- library components you want cabal to solve & build for<br>
-- and become accessible via .ghc.environment files and/or<br>
-- `cabal new-repl`<br>
build-depends: base, lens<br>
<br>
-- executable components you want cabal to build<br>
build-tool-depends: pandoc:pandoc<br>
<br>
--8<---------------cut here---------------end--------<wbr>------->8---<br>
<br>
6.) invoke `cabal new-build`<br>
<br>
7.) If everything works, you'll find the `pandoc:pandoc` executable<br>
somewhere in your ~/.cabal/store/ghc-8.3.*/ folder<br>
(you can use <a href="http://hackage.haskell.org/package/cabal-plan" rel="noreferrer" target="_blank">http://hackage.haskell.org/<wbr>package/cabal-plan</a><br>
to conveniently list the location via `cabal-plan list-bins`)<br>
<br>
8.) As for libraries, you can either use `cabal new-repl`<br>
or you can leverage GHC's package environment files:<br>
<br>
`cabal new-build` will have generated a file like<br>
<br>
.ghc.environment.x86_64-linux-<wbr>8.3.20170913<br>
<br>
which brings into scope all transitive dependencies of<br>
`build-depends: base, lens`<br>
<br>
Now all you need to do is simply call<br>
<br>
ghc-stage2 --make MyTestProg.hs<br>
<br>
to compile a program against those libs, or start up GHCi via<br>
<br>
ghc-stage2 --interactive<br>
<br>
and you'll be thrown into that package environment.<br>
<br>
<br>
<br>
I hope you find this useful<br>
<br>
Cheers,<br>
Herbert<br>
______________________________<wbr>_________________<br>
ghc-devs mailing list<br>
<a href="mailto:ghc-devs@haskell.org">ghc-devs@haskell.org</a><br>
<a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs" rel="noreferrer" target="_blank">http://mail.haskell.org/cgi-<wbr>bin/mailman/listinfo/ghc-devs</a><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">Rahul Muttineni</div>
</div>