[Haskell-cafe] real haskell difficulties (at least for me)

[Duncan Coutts]

On Tue, 2009-01-13 at 16:33 +0100, Regis Saint-Paul wrote:
> Hi, 
> 
> I’ve seen many times the monad topic coming around on the cafe and plentiful
> tutorials on monads have been published. However, as a complete Haskell
> newbie coming from OOP, I felt monads were not particularly difficult to
> grasp, and very exciting to work with. 
> 
> During my experiments with Haskell so far, the main problems I kept bumping
> into were not related to the language but to its libraries: their
> compilation and installation. Unfortunately, this topic has not received
> nearly as much attention. I was unable to find a comprehensive tutorial on
> how to deal with the variety of problems I get when trying to install
> Hackage packages. This turned out to be (and still is) THE main source of
> wasted time and headaches. And worse, unlike type problems, these are not
> interesting ones to solve.

You're right, we've only been actively working on this for the last
couple years. The tools are not yet as good as we would like them to be.

Unfortunately as you say the problem does not interest a lot of people.
So it is "good enough" for lots of people and we do not have quite
enough people working on the package infrastructure to make it really
great as quickly as we'd like. Of course it's not for lack of ideas or
suggestions, we've got a huge TODO list! :-)

Let me add a plug for our resources for new Cabal hackers:
http://hackage.haskell.org/trac/hackage/
There's a source code guide, an "easy ticket" list, a link to the
development mailing list and instructions on how to get the code.

New hackers are most welcome! :-)

> Thus, as a beginner, the package management is what is really getting in the
> way of switching to Haskell--not the language. Even books like Real World
> Haskell (otherwise excellent) ignore entirely the topic.

In fairness I think it does mention Cabal and cabal-install. But it does
not go into a lot of detail I admit. At the time they were writing that
cabal-install had only just got to the stage of being usable (the
authors had to slightly make a guess as to whether this "cabal-install"
thing was going to become popular). It's still pretty young software.

> Cabal and Cabal-install are clearly wonderful applications that make
> installing most packages very straightforward.

Phew! That's a big improvement. It used to be universally hard to
install packages, and there were many fewer packages around because they
used to be so hard to make.

> Unfortunately, whenever this "standard" method for package
> installation fails (or when it is not available as with, e.g.,
> gtk2hs), I find myself in complete disarray. 

Yes, that is a serious problem.

> Below are some of the questions and issues I faced regarding package
> management: 
> 
> - For a number of packages, cabal-install gets stuck and has to be killed.

Assuming you are using the latest version (0.6.0) this should never
happen. If it does please report bugs with details on how to reproduce
the problem. I'm the maintainer of Cabal so email me or add tickets in
our bug tracker:

http://hackage.haskell.org/trac/hackage/

If it gets stuck building the package then that's a slightly different
issue. Either way, more information would help in diagnosing what is
going on. It is certainly not expected behaviour.

> I assume this is due to some difficulties in solving the dependencies and it
> is fine, not all can be automated and cabal-install is not responsible for
> poor packages. But the question then becomes what to do from there? Is their
> some method to solve dependencies? How should we proceed to "debug" a
> package installation? How do gurus deal with that? (maybe some less known
> command line arguments? Or ways to figure out the problem and work out its
> solution (cabal-install is silent in such case)? In particular, how to know
> why did cabal get stuck in the first place?

cabal install --dry-run -v

This produces quite a bit of information about what the dependency
resolver is doing, especially in the latest version.

I've not had the situation where it sometimes fails to terminate. The
only infinite loop bug it has ever had (as far as I am aware) is when
cabal-install version 0.5.x is used with ghc-6.10. In that case it
always fails to terminate, it does not matter what package. So like I
said, if you are getting non-termination in the solver we need more
details.

What does happen sometimes is that the solver cannont find a solution.
In this case you can look at the error message and perhaps the output of
--dry-run -v and try adding constraints to give it some help on what
versions of packages to pick, for example:

cabal install --dry-run -v 'foo < 2' 'bar == 1.2.0'

> - Some packages on Hackage are reported as not building successfully with
> GHC6.10 (e.g., encoding) while others do not build with 6.8 (e.g., salvia)
> and the later might depend on the former...What is one supposed to do in
> such case?

Note that the hackage build results are not very accurate at the moment.
They can fail for a number of reasons that would not prevent you from
making it work on your own machine.

If the package really does not build with the version of ghc you have
then either try a different version of the package (in the case that the
latest one works with ghv 6.10 but you want to use ghc 6.8). You may of
course find packages that need patching before they will work with 6.10.
In such cases if you cannot patch them yourself then email the
maintainer and tell them that you would like to use it with 6.10. There
are still quite a few of these about.

>  For example, is it an appropriate way to proceed to compile a
> package with one version of GHC and then use the compiled package with
> another version of GHC? Is it safe? What could possibly go wrong?

It will not work. For one thing it would be hard to try and get into
that situation since the package databases are different for each ghc
version. If you managed to hack it anyway then ghc will detect that you
used a different version to build the other package and refuse to
compile against it.

> If it is the right way to go, how should we setup the two GHC
> versions? For instance,should we have a shared package configuration
> file and choose through the path which GHC is used or is there nicer
> way to set this up?

It is perfectly possible to have multiple versions of ghc installed.
I've got 4 versions. Each has an independent set of libraries however.

By default ghc installs into a versioned lib directory and the binaries
are versioned too, so 'ghc' on the $PATH will refer to the ghc version
you installed most recently and use ghc-x.y for specific versions.
 
> - Taking for example the "encoding" package on Hackage. Last time I tried,
> the log was saying it fails to build on GHC 6.10, however, looking inside
> this Hackage log, I could see a successful compilation using "preferred
> versions". So it looks as if the thing can be compiled somehow. What should
> one do with this information?

cabal-install uses it automatically, at least if you're using the latest
version (0.6.0).

> If cabal manages to compile it using this method on Hackage, then
> isn't cabal install just doing it on my disk?

Yes.

> Is it possible through some command line? Is it possible manually
> (without cabal-install) and, if so, how? (I tried to copy-past the
> build instruction as it appeared on the log...that somehow compiled,
> but then, I failed to figure out how to install...)

It sounds like you're using an older version. It works fine for me here
using cabal-install-0.6.0.

> - I’m primarily a windows user and lots of my initial struggles probably
> came from that. After many difficulties, I figured out that installing MinGW
> and MSys was *THE* way to get a bit more of the things working. First, a lot
> of time would be saved by just saying clearly on the GHC download page that
> MinGW and MSys are mandatory installation (or even package that with GHC for
> the windows distribution if license allows, who cares the extra few Mb).
> Even if that is not technically exact, i.e., even if ghci and many trivial
> command line programs can work without, MSys and MinGW turn out to be quiet
> necessary whenever trying to install anything producing side effect.

They should only be needed for packages that use ./configure scripts.
Perhaps that applies to all the ones you were trying.

> Making it plain that these two are necessary would real come has a
> great time savers for newbie like me on windows (personal opinion of
> course). Or, if another path exists to go without these two, I'd be
> very glad to learn. Besides, even these tools basic installation is
> not enough, you need automake and various things of the like. That
> makes me wonder if the most precious skill for programming with
> Haskell would not be a strong C/C++ programming background. 

These should not be necessary unless you are modifying packages that
use ./configure scripts. Just to build them it is not necessary.

To be honest I've never got autoconf or automake to work on MinGW/MSYS.
I always generate the tarballs on linux and build them on windows (eg
for building the gtk2hs windows installers)

> - In face of the difficulties with windows, I switched to Linux. While some
> things worked better, there were still lots of difficulties with package
> compilation. For instance, it is very difficult to figure out which Linux
> packages of a given distribution are needed for compiling this or that
> package. Again, gtk2hs is epitome here: which C development packages are
> needed to compile it is obscure at best (cairo, codeview, etc...). I ended
> up querying the Debian package management with any keyword found after
> running gtk2hs and randomly installing all the dev packages...

That's a reasonable strategy. Or using native gtk2hs packages if they
are available for your system (I think the debian gtk2hs packages are a
bit older than the current gtk2hs release but probably still usable.)

> And when gtk2hs finally compiled, it failed to install anyway. As of
> today, I’ve never been able to compile even the dumbest demo using
> gtk2hs whether on linux or on windows and whether using ghc 6.8.3 or
> 6.10.1.

The released version should work with ghc-6.8.x. Perhaps ask for help on
the gtk2hs-users mailing list.

The last release came out before ghc-6.10 and do not work with it. The
development version works with 6.10 and we're expecting a new release
any time.

> On windows, the automated setup install worked but did not allow me to
> compile with codeview and I still do not know how to add codeview to
> the install packages.

No, the windows installer does not include the sourceview package
because it requires many many more C libraries and we decided it was not
worth it for most users. Again feel free to ask on the gtk2hs-users
list. If people do want it and don't mind the bigger download then it's
doable.

> Trust that I tried hard and read the docs thoroughly. Gtk2hs is just
> on of many examples; I had problems under Linux also with, e.g.,
> Happs, yi, database things, etc. and figured out that the situation
> was roughly identical to windows with MSys and MinGW. So Linux appears
> not to be the right solution here. Maybe it's just that Linux users
> are more experienced with the GNU C/C++ libraries...but it won't help
> a windows user to switch to Linux since this knowledge can't be built
> out of thin air. 

Partly it is familiarity and partly it is the fact that hackage is a
'live' collection of the latest version of everything. It is not a
packaged collection of stuff that has been tested together. So there
really are many packages that will be hard to get to work with whatever
versions of other things you've got.

It's a situation we are trying to improve by improved reporting and
information on the hackage website. We're also building a "haskell
platform" which is a collection of packages that have been tested
together on several different operating systems.

> - Would there be some binary version of cabal targeting various OSs? I
> believe the Haskell platform project is about that.

Yes exactly.

> But without waiting for a fair and objective selection of the packages
> (it seems to be the current status of the project), I’d be happy
> working with some authoritative bundle produced by a Haskell guru and
> would trust his subjective choices (who am I to question these choices
> anyway). Or even an image (e.g., virtual box or Xen) of a fully setup
> development environment since there are so many dependencies involved
> in, e.g., simply compiling GHC... 

It sounds like you're describing the platform project. Hopefully that
will not be too long a wait.

> Now, one might argue that these are not Haskell problems, that they are
> normal when dealing with non-mature packages. So let me explain why I've
> been trying hard to install these packages: 
> 
> As a beginner with no experience with emacs, I tried to find some IDE-like
> environment which would, at least, save me from manually reloading files in
> ghci or help me browse the source files. Following the Haskellwiki advice,
> that led to trying out Yi, Leksah, eclipsefp, or a Visual Studio extension.
> To this date, NOT ANY SINGLE ONE of these worked, be it on Linux or Windows.
> I had to resort to learning emacs which seems the only sensible choice
> available today.  

No, none of them are mature. Most people are using ordinary editors with
Haskell modes and manually reloading files in ghci.

> I am particularly unskilled, no question here. But, would a charitable soul
> take the pain of writing a comprehensive package management tutorial instead
> of a monad one, (s)he would have my deepest gratitude :)

Yes if we could find someone to help us improve the documentation for
using cabal that would be brilliant. We've got a lot of information in
the cabal user guide but I get the feeling it is not very accessible.
Also it does not cover hackage.

Personally I find it very hard to document broken stuff. I prefer to
spend that time fixing things. I realise that's not very helpful. As I
said, we could do with some help in this area.

> Apologies for the long mail.

Thanks for the feedback. It makes it clear how much work we have yet to
do to make this process smoother for new users.

> P.S. People on #haskell are wonderful. They helped me solve many issues.
> Unfortunately, solving specific instances of problem did not contribute much
> to a deeper understanding of the internal working. I find myself randomly
> trying things without knowing which would work or why; Hence this plea for a
> tutorial.