Stop untracked dependencies! (was RE: mapping module identifiers to URLs (was RE: [Haskell] URLs in haskellmodule namespace))

[S. Alexander Jacobson]

A major problem in every major language I have ever used is module 
systems that make it all too easy for untracked dependencies to creep 
silently into code.  The result has repeatedly been nasty surprises 
when moving code between machines.  A Haskell example of this problem 
might look something like this:

* Oh, Network.HTTP is non-standard!? Thats wierd.
* Uh Oh. Google reveals at least three different implementations...
* Now, how do I determine which one is installed here?
* Is there a local copy of the package somewhere?
* No? Ok, is it available for download somewhere?
* Uhm, is the version I am using still available for download?
* Oh, we made local changes too!? What were they?
* Ok, we added Network.Socket.SSL. That's standard right? [rinse/repeat]
[A lot of detective work later...]
* Ok, now we've figured out all the packages, how do we ship them?
[shipping strategy devised...]
* But what happens if I don't have root/admin on the target machine? [...]
* Oh, this package conflicts with something already installed? [etc.]

> It is true that Cabal's Build-Depends doesn't do everything you want it
> to, but your proposal also duplicates some of the functionality of
> Build-Depends, and makes it so that there are two places where
> dependencies are stored. We should only have to specify 
> dependencies once.

I agree you should only have to specify dependencies once.  But, 
deferring dependency tracking to the point you are ready to ship (with 
Cabal or otherwise) is a recipe for disaster.  Every code dependency 
should be explicit and verified at compile time. You should be able to 
to look at any block of code at any time and be able to produce the 
list of packages required to make it run.

Strawman proposal 2:

   Modify the import syntax to allow package identifiers:

    import qualified HaXML HaXML.XML.Parse
    import HAppS HAppS.ACID
    import Personal MyLib

   Define a packages file format to resolve package ids to locations:

     <packages>
      <package id="HaXML" src="http://haskell.org/HaXML/HaXML.hkg">
        <remap ref="HUnit" src="http://haskell.org/HUnit2"/>
        <remap ref="Parse" src="#Parse"/>
      </package>
      <package id="Parse" src="http://parser.org/parse.hkg"/>
      <package id="HAppS" src="http://haskell.org/HAppS.hkg" />
      <package id="Personal" src="../MyLibs" />
      <redirect from="http://oldversion" to="http://newversion" />
     </packages>

   Add a -p option to the compiler pointing to a packages file
   and use the content of that file to resolve module ids to
   implementations.  The compiler MUST give an error if the user tries
   to associate the same module id with two different implementations
   [no grafting/module relativity!]

   Remove the -i compiler option.  Any import of a module not
   in the local path must have a package id.  [Get rid of import path
   order/overlap ambiguity!]

   Replace Cabal's Build-Depends field label with a Packages field
   label that takes a path to a packages file as a value.  Add option
   to include a copy of all packages used by the current package.

   Give error if modules from other packages give conflicting module
   implementations.  Allow user to resolve conflict in imported package
   files by using remap and redirect elements in the packages file.

> I don't think it's necessary to specify dependencies with module
> granularity.  This just increases the number of possible error cases,
> without adding functionality.

If I use multiple packages that export the same module identifier, I 
need a way to specify which one I want to use.  Haskell's existing 
packaging model doesn't let met do that easily.

>   - Package dependencies in Build-Depends could be specified using
>     URLs.

But then dependencies aren't checked at compile time and you can't 
specify which modules come from which packages.

>   - Cabal could download, build, and install dependencies without any
>     user intervention.

Download and build? Great!  Install? No thank you!

In a fundamental sense, *installation* of third party libraries is 
exactly what I am opposing here.  Library installation is what allows 
untracked dependencies to creep in.  Users of executables definitely 
want a process for installation of executables in their local 
environment because they need a way to adjudicate potential conflicts 
between them (e.g. file associations).  However, what users of 
libraries want is just a safe and easy way to populate a local cache 
of module-implementations from various sources and that is what I am 
asking for here.

-Alex-

______________________________________________________________
S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com