[Haskell-cafe] Problem with publishing project/packages

[Dominik Schrempf]

Hello,

I am not sure if this is the right place to report/discuss the following issue,
but I have been fiddling around with properly publishing a project I am
developing in Haskell for quite some time now. I have encountered various
difficulties that I would like to share and hear your opinion about, and also
your suggestions about what I could improve, and probably also, how the Haskell
ecosystem could be improved.

First, I use Stack+Hpack to manage `.cabal` files. I know, that this combination
has summoned many controversies, but I don't want to spend hours fiddling around
in my `.cabal` files and on the version bounds, so decided for this option. I
use strict PVP bounds (--pvp-bounds both options), so that at least compilation
should be reproducible, albeit only allowing the specific set of library
versions present in the Stack resolver.

My project does heavy numerical calculations, so I was awaiting the support of
LLVM9.0, which will come with GHC 8.10, for a long time. In the meantime, I
tried to compile and publish my project in its current state with the newest
Stack resolver 15.x with GHC 8.8.3. On my local machine, compilation and the
development cycle work reasonably well, but as soon as I try to publish my work,
I have serious trouble.

First, I encountered an issue with auto generated modules of the packages
causing trouble in the `.cabal` files generated by Hpack [1]. I think this is
because the Cabal specification was improved (i.e., has changed), and new
specifications have to be honored. I try to work around the issue by directly
specifying the newest Cabal specification 3.0 and by tinkering around with the
flags, until Hpack produces files compatible with the newest version of Cabal.
Finally, I can create source tarballs with `stack sdist --pvp-bounds both`.
However, I cannot upload these tarballs, because somehow, the created `.cabal`
files in the source tarballs include dependencies of the following form:
`base : {} >= x.y, <= z.a` (or something similar), and Hackage chokes on this
form (is this a new syntax? or is it just a bug?). So I think, maybe this is
caused by an incompatibility of Stack with my Cabal or cabal-install version,
which are newer than the newest ones officially supported by Stack. I had
already lost one day of working time at this point.

I did not give up, but decided to install a Cabal version officially supported
by Stack. Also, I removed all my dependencies on auto-generated modules from the
packages and the `.cabal` files, I reverted other changes I had implemented
because of the Cabal specification change (for example, reverting all license
identifiers). I managed to compile and publish my project on Hackage. However,
Hackage is not able to compile the packages and show the documentation (which is
what I need most). Because I am going the easy route, and use strict PVP bounds,
Hackage cannot compile my packages because they depend on base >= 4.13, and
Cabal version 3.0. It took me another hour or so, to find out that the official
package repository of Haskell does not support the official base library which
has been published over six months ago. From a users point of view, this is
unacceptable. I have found some resources about this issue now, but I was not
aware of this discrepancy before experiencing it.

So my next thought is: "Let's just put it on Stackage, because the best days of
Hackage seem to be over...". However, I cannot do that, because my project needs
to compile on Hackage in order to be put on Stackage...

I guess all this is consequence of using an operating system with up-to-date
packages (Arch Linux), and my inherent demand to use the newest compiler, and so
on. However, I feel that it is not entirely my fault, the new compiler with the
new base libraries should be supported by the official repository, and in a
coherent way by the most widely used build tools Cabal and Stack (which uses
Hpack by default, by the way).

I sincerely apologize for being so harsh, but I am angry at the ecosystem, and
angry at myself, because, obviously, I don't know enough to get things done.

Thanks for your feedback, it is appreciated!

Dominik Schrempf

References:
[1] https://github.com/sol/hpack/issues/384
[2] In case of interest, the project I am developing is a set of
    libraries/binaries to be used in evolutionary biology
    https://github.com/dschrempf/elynx