[Haskell-cafe] [Haskell] JustHub 'Sherkin' Release

[Andres Löh]

Hi Chris.

[Sorry if I'm slow responding, but I'm at a summer school right now
and have relatively little time to follow my email.]

> At issue is whether the JustHub Haskell distribution for Enterprise Linux
> and
> the hub hackage for sandboxing development projects and integrating multiple
> GHC
> tool chains is redundant because all of the functionality is covered by the
> Nix Haskell distribution, allowing as they do multiple Nix Haskell releases
> to
> be deployed simultaneously.]

Let me clarify this in advance: I'm *not* questioning the usefulness
of JustHub at all! It's great that you provide it. I was asking
whether integrating JustHub *into* Nix and/or NixOS would make sense
and would provide any benefit on top of the features already provided
by these systems. As I indicated, I'm not yet convinced of that. I
asked the question in order to learn more about JustHub. I do
completely understand that Nix is not for everyone and that JustHub
may have less dependencies that make it a very attractive choice for a
wide range of applications.

>        All of the standard Haskell tools inter-operate cleanly and
>        transparently with the sandboxes and operate in the normal way
>        outside of them.

I guess this may be the main point. You're saying you can use the
standard tools to work with hubs and don't need to do anything
special, whereas in Nix(OS) you admittedly only get the full benefit
of the system if you're installing packages via Nix rather than via
Cabal.

As Peter has pointed out, you're usually working in Nix(OS) with
several profiles. Each of these profiles can contain an arbitrary set
of packages (not limited to Haskell or GHC). It's possible to switch
between profiles easily. It's possible to use different profiles for
development on different projects. Different profiles can contain
different versions of GHC as well as different sets or variants of
packages.

>        Sandboxed environments (hubs) can be shared between work trees as well
>        as being (re)named, annotated, replicated, swapped, archived, locked
>        and removed. Proper package deletion with the option of garbage
>        collecting orphaned code and documentation is also supported.

I'm not completely sure what each of these mean. Package removal is
certainly supported in Nix(OS), but again, admittedly, not via the
standard tools. You have to invoke nix-specific commands in order to
perform garbage collection.

> I have loaded GHC-7.4.1 platform and the GHC-7.0.4 into Nix and it installs
> all of the ghc drivers
> into a single bin directory in the user's profile. I am guessing that
> running `ghc` will generally
> get you the latest compiler you have installed (7.4.1 in my case); specific
> releases can be
> invoked with ghc-7.0.4, etc.

Installing the latest version into the main user profile is merely the
default functionality. There's absolutely no problem selecting older
versions or installing into separate profiles. If you look at the
files actually installed, you'll find that all files in a profile are
just links, and that each Haskell package in fact lives in isolation
in its own directory in the Nix store.

> This hardly covers all of the above functionality!

It might not. We're having this discussion in order to find out, I think :)

> Quite related to this (in my mind anyway) are the user-level facilities for
> managing the package
> databases that each work tree uses -- the problems that cabal-dev was
> created to solve. What I
> have done is to create a system that manages the environment each source
> work tree uses.
> If you are in a 2012.2.0.0-based project work tree then the ghc-driver will
> detect that and invoke
> the right tools. The 2012.2 platform uses cabal-instal-0.14 and that is what
> you will get when
> you invoke cabal in such a work tree.

You still have to say at some initial point what version you want to
use, I hope? Otherwise, I can't see how it could be detected.

> However in work trees based on earlier version of the compiler (e.g.,
> GHC-7.2.2), cabal-install-0.10.2
> will be used because cabal-install-0.14.0 doesn't interoperate very well
> with cabal-0.10 and earlier
> (see https://github.com/haskell/cabal/issues/932). Also in such a work tree
> you will get all
> of the tools that were shipped with the GHC-7.2.2 and all through issuing
> the usual command
> 'ghc', 'ghci', 'ghc-pkg', etc).

Independent of concrete bugs, who's making these decisions? Can I use
cabal-install-0.14.0 on projects working with older platforms if I
want to?

> Without some system to help the user invoke the right tools in the right
> context, having to invoke
> each version of the compiler explicitly can get awkward to use quite
> quickly.

This is not required in Nix(OS). You switch the profile and then get
the versions you have in your profile. The profile is not tied to
working dirs, you have to manually switch. I think both approaches
have their disadvantages in practice (i.e., either the user shell or
the working dir are "stateful").

> Think about installing a
> package into a 7.0.4-based work tree where 'ghc' runs version 7.4.1. Cabal
> will reach for 7.4.1 unless
> told otherwise.  The only practical way to do this is to build a PATH where
> the right tools get run on
> the default commands ('ghc', 'ghc-pkg', etc),

Yes. This is what you get with Nix(OS).

> or equivalently use
> intelligent drivers that make sure
> the right tools get invoked (the method JustHub uses).

Ok.

> It is also sometimes helpful to be able to start from the minimal collection
> of packages that you get with
> compiler (sans platform packages) and build your collection from there. The
> JustHub system allows
> you to do this, even on installations that are working double time as
> Haskell Platform instantiations.

Not a problem in Nix(OS) either. Indeed, for each compiler version I
have standard "plain" and "platform" profiles installed on my machine,
with just the GHC-provided or just the platform packages. In addition,
I then have project-specific profiles. The packages, as far as they're
contained in several profiles, are installed only once and shared via
the Nix store.

> GHC+cabal do provide an awesome build system but the package management
> mechanisms aren't as
> easy to use as they could be. There is no proper mechanism for erasing
> packages for instance (arguably
> because it is meaningless -- reinforcing my point) and the mechanisms for
> managing package databases
> are inconsistent across the tools (differing between 'cabal' and
> 'ghc'/ghc-pkg') and not very friendly
> (requiring the setting up of environment variables and the use of special
> flags). I have also found it highly
> useful to be able to name, replicate, archive and share environments between
> different work trees

Right. That's why in Nix(OS) we currently don't try to actually patch
cabal-install or other tools to behave differently from normal. So
you're free to use for example cabal-dev on top of a NixOS profile,
but in order to benefit from NixOS fully, you should install Haskell
packages via Nix. We have tools that try to make it easy to add new
packages to the Nixpkgs collection semi-automatically, but admittedly,
that's not quite as simple as typing "cabal install".

> You may dismiss that as pointless or irrelevant but it _is_ a large part of
> what I am trying get right and it did
> feature prominently in the announcement.

Let me say again that I'm curious to learn the differences. Thanks for
taking the time to explain this to me, as I haven't had the time to
work with JustHub yet, and it certainly looks very interesting. I'm
*not* trying to dismiss JustHub as being useless. OTOH, there's
certainly overlap between the two systems. So if we're ever thinking
of integrating one into the other, I'd like to make sure that there
isn't too much redundancy.

> I think the Nix functional
> approach to distributions is definitely going in the right direction in
> giving user better control over which
> packages are installed but, unless I have missed something, I don't think it
> is so definitive that there isn't
> room for other developments.

I never said that.

> Finally a couple of gripes with Nix as a replacement for JustHub.
> [...]

I also never suggested that, at least not in general. If anything, I'd
say that people interested in JustHub might also be interested to look
at Nix(OS).

> Also I could not determine which releases of GHC you are supporting. It
> looks as the distribution is focused
> on the Haskell Platform, in which case only those releases of GHC will be
> available, when they get scheduled
> for inclusion in the platform.

NixOS is a community project that the moment, there's no official
definition of "support". In practice, we have GHC versions 6.10.4,
6.12.1, 6.12.2, 6.12.3, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.2.1, 7.2.2,
7.4.1, 7.4.2 and HEAD (snapshot). There's no restriction to platform
releases.

Cheers,
  Andres