compiler-independent core libraries infrastructure

[Bulat Ziganshin]

(i crosspost my letter into main haskell list because i hope that
proposed solution is of great interest for (discussion with) many
developers)


Hello Ian,

Friday, September 8, 2006, 5:52:57 AM, you wrote:

>> and last question - i don't like inclusion of unix and win32 in a list
>> of core libs. why they are here? may be it's possible to include small
>> modules with functionality required for compiler itself in GHC.*
>> hierarchy and then move the rest into extra libraries?

> This doesn't work well for any parts shared with the other compilers.
> "core libs" is possibly the wrong name for it - "bootstrapping libs"
> might be more appropriate.

> Note also that unix and win32 haven't been included in anything they
> weren't already, it's just that other libraries (those that are now
> called "extra libs") have been taken out. We can always take more out
> for 6.8 if we want.

thank you - you have cleaned up the situation for me. thanks to Cabal,
now ghc is much more modular than 6.4. but it is not yet the ultimate
solution and i propose to discuss what we can do in future, possibly
even in 6.6.1. i will become devil advocate for a little :)

i am, John de MacLee programmer, never planned to build ghc itself and
i don't need any bootstrapping libs in my download. please sell it as
separate ghc-for-ghc package :)  the core-ghc package should then
contain only libraries that are dependent on GHC compiler internals
(GHC.* part of base, stm and th - and nothing more!)

moreover, i want to be able to upgrade even these libraries without
upgrading compiler proper. or, to be exact, i may need to install
newer versions of these libraries which contains new features and
therefore not 100% compatible with libraries shipped at the moment
when GHC 6.6.1 was released

why this may be impossible? first, because existing libs and programs
may rely on older interfaces. i think that Cabal should eventually
solve this problem so that multiple version of any lib can be
installed on computer (to be exact, on concrete GHC installation) and
proper version of library selected for any project. but that's another
discussion..

what is a 'base' library now? it is the library that implements common set
of operations for latest versions of ghc, hugs and nhc. it contains
low-level implementation for ghc, but relies on separate hugsbase
package for hugs (the same for nhc, afaiu). so, first step is obvious
- separate ghc-base library from the rest. hugsbase, ghc-base and
nhc-base packages should provide common set of low-level operations,
hiding from other libraries implementation details, differences
between compilers, and differences between compiler versions. they
should provide _incremental_ interfaces so that old code will continue
to work with newer compilers. eventually compiler-specific code for
stm and th should also go into these libraries but that is not the
immediate goal

then, a base library may be written against "virtual Haskell compiler",
which provides uniform set of low-level features while 'base' decorates
these features with user-friendly interfaces


even more interesting variant is to allow ghc-base and other
compiler-specific base packages to export non-incremental interfaces
and use 'base' solely to "equalize" all compilers to some common
interface, providing emulation of all missing features (i think that
such emulation will be compiler-independent that means that it's
better to put it into compiler-independent package). then, _all_ other
libs should rely on version of base package version instead of version of
compiler they are use. so:

ghc 6.2
ghc 6.2.2
ghc 6.4.3
hugs 2003
....

all supported in base 1.0 package which expose stable interface
independent on compiler used. all other libraries relies on this
interface and therefore works with any compiler whose support included
in base 1.0. as ghc 6.6 rolls out, we add its support to 'base' library,
rolling out base-1.0.1. and all libraries written against base 1.0,
now will work with ghc 6.6, although they can ignore some features
what was not included in base 1.0 API. at the same time, base-2.0
rolled out which includes new APIs (but don't omit old ones!),
supporting new features of ghc 6.6. but base 2.0 continues to support
existing compilers, providing emulation of new features for old
compilers. those developers that need these new features upgrade their
cabal files to require base 2.0. those users of old compilers that go
to compile these apps download (automatically) and install base-2.0 lib


so, i propose:

- ghc-base/hugsbase/.. libs to implement _subset_ of common low-level API
- base lib to "equalize" several compilers and compiler versions,
  providing _full_ common low-level API. when we need to include new API,
  we roll out new major version of base and work hard to support
  old compilers by providing some emulation of new feature.
  base lib versions should be independent on compiler versions and as
  much backward-compatible as possible
- all other libs to just specify version of base lib they utilize and be happy

and of course we should split base lib into ~10 independent libraries
to simplify development and deployment of new features. if i need,
say, new ByteString features, i would be very unhappy if i need at the
same time to upgrade interface to arrays

last line: i have some experience of writing compiler-independent code
with Haskell and C++ and believe that this plan is realistic



-- 
Best regards,
 Bulat                            mailto:Bulat.Ziganshin at gmail.com