Build system idea

[Simon Marlow]

John Meacham wrote:

> unfortunately the cabal approach doesn't work. note, I am not saying a
> declarative configuration manager won't work. in fact, I have sketched a
> design for one on occasion. but cabal's particular choices are broken.
> It is treading the same waters that made 'imake' fail.
> 
> the ideas of forwards and backwards compatability are _the_ defining
> features of a configuration manager. Think about this, I can take my old
> sunsite CD, burned _ten years_ ago and take the unchanged tarballs off
> that CD and ./configure && make and in general most will work. many were
> written before linux even existed, many were written with non gcc
> compilers, yet they work today. The cabal way wasn't able to handle a
> single release of ghc and keep forwards or backwards compatability.
> 
> That any project ever had to be changed to use the flag 'split-base' is
> a travesty. What about all the projects on burnt cds or that don't have
> someone to update them? 20 years from now when we are all using 'fhc'
> (Fred's Haskell Compiler) will we still have this reference to
> 'split-base' in our cabal files? how many more flags will have
> accumulated by then? Sure it's declarative, but in a language that
> doesn't make sense without the rule-book.  autoconf tests things like
> 'does a library named foo exist and export bar'. 'is char signed or
> unsigned on the target system'. those are declarative statement and
> have a defined meaning through all time. (though, implemented in a
> pretty ugly imperative way) That is what allows autoconfed packages to
> be compiled by compilers on systems that were never dreamed of when the
> packages were written.

The important thing about Cabal's way of specifying dependencies is that 
they can be made sound with not much difficulty.  If I say that my package 
depends on base==3.0 and network==1.0, then I can guarantee that as long as 
those dependencies are present then my package will build.  ("but but 
but..." I hear you say - don't touch that keyboard yet!)

Suppose you used autoconf tests instead.  You might happen to know that 
Network.Socket.blah was added at some point and write a test for that, but 
alas if you didn't also write a test for Network.Socket.foo (which your 
code uses but ends up getting removed in network-1.1) then your code 
breaks.  Autoconf doesn't help you make your configuration sound, and you 
get no prior guarantee that your code will build.

Now, Cabal's dependencies have the well-known problem that they're 
exceptionally brittle, because they either overspecify or underspecify, and 
it's not possible to get it "just right".  On the other hand, autoconf 
configurations tend to underspecify dependencies, because you typically 
only write an autoconf test for something that you know has changed in the 
past - you don't know what's going to change in the future, so you usually 
just hope for the best.  For Cabal I can ask the question "if I modify the 
API of package P, which other packages might be broken as a result?", but I 
can't do that with autoconf.

Both systems are flawed, but neither fundamentally.  For Cabal I think it 
would be interesting to look into using more precise dependencies 
(module.identifier::type, rather than package-version) and have them 
auto-generated.  But this has difficult implications: implementing 
cabal-install's installation plans becomes much harder, for example.

>> So I accept that we do not yet cover the range of configuration choices
>> that are needed by the more complex packages (cf darcs), but I think
>> that we can and that the approach is basically sound. The fact that we
>> can automatically generate distro packages for hundreds of packages is
>> not insignificant. This is just not possible with the autoconf approach.
> 
> This is just utterly untrue. autoconfed packages that generate rpms,
> debs, etc are quite common. The only reason cabal can autogenerate
> distro packages for so many is that many interesting or hard ones just
> _arn't possible with cabal at all_.

Exactly!  Cabal is designed so that a distro packager can write a program 
that takes a Cabal package and generates a distro package for their distro. 
  It has to do distro-specific stuff, but it doesn't typically need to do 
package-specific stuff.

To generate a distro package from an autoconf package either the package 
author has to include support for that distro, or a distro packager has to 
write specific support for that package.  There's no way to do generic 
autoconf->distro package generation, like there is with Cabal.

Yes this means that Cabal is less general than autoconf.  It was quite a 
revelation when we discovered this during the design of Cabal - originally 
we were going to have everything done programmatically in the Setup.hs 
file, but then we realised that having the package configuration available 
*as data* gave us a lot more scope for automation, albeit at the expense of 
some generality.

That's the tradeoff - but there's still nothing stopping you from using 
autoconf and your own build system instead if you need to!

> As for programs written in haskell, I don't want people's first
> impression of haskell being "oh crap, I gotta learn a new way to build
> things just because this program is written in some odd language called
> 'haskell'" I don't care how awesome a language is, I am going to be
> annoyed by having to deal with it when I just want to compile/install a
> program. It will leave a bad taste in my mouth. I would much rather
> peoples first impression be "oh wow, this program is pretty sweet. I
> wonder what it is written in?" hence they all use ./configure && make by
> design rather than necessity.

Python packages don't have ./configure or make...

> I sometimes hear that I just shouldn't use cabal for some projects but,
> when it comes down to it. If cabal is a limited build/configuration
> system in any way, why would I ever choose it when starting a project
> when I know it is either putting a limit on my projects ability to
> innovate or knowing that at some point in the future I am going to have
> to switch build systems?

Because if you *can* use Cabal, you get a lot of value-adds for free 
(distro packages, cabal-install, Haddock, source distributions, Hackage). 
What's more, it's really cheap to use Cabal: a .cabal file is typically 
less than a screenful, so it's no big deal to switch to something else 
later if you need to.

Cheers,
	Simon