compiler-independent core libraries infrastructure

[Bulat Ziganshin]

Hello Neil,

Thursday, September 14, 2006, 6:14:30 PM, you wrote:

>> then, a base library may be written against "virtual Haskell compiler",
>> which provides uniform set of low-level features while 'base' decorates
>> these features with user-friendly interfaces

> Nice idea. There are a few practical issues - for example does this
> virtual Haskell copmiler support higher rank types? Multi-parameter
> type classes? Bang patterns? It quickly gets a lot more complicated.

i mean by "virtual Haskell compiler" set of _library_ functions, not
the language features, pragmas and other differences. problem of
availability of language features for standard libraries typically
solved in conservative way - i.e. we try to use as less language features
as possible and separate modules that use non-standard features in
separate packages. for example, "modern array libraries" should go
into separate package because they use MPTC and therefore not
available for many compilers. shebang patters, of course, should be no
used - it's a feature for applications, not for writers of standard
packages


>> - ghc-base/hugsbase/.. libs to implement _subset_ of common low-level API
> Sounds like a very good idea.

>> - base lib to "equalize" several compilers and compiler versions,
> Yes, some operations might be implemented in the base library, but
> have more efficient versions in the ghc-base library. For example, map
> for base should be defined the obvious way, for ghc it should be
> defined with foldr. How can you accomodate this?

i'm pragmatic and don't think that base package should hide _all_ the
compiler differences and especially all compiler-specific
optimizations. going this way, we should put to ghc-base byte strings
and many other things

my proposal is to extract from 'base' package all high-level, written
in pure Haskell algorithms and put it in ~10 'application' packages,
say ByteString, Array, DataStructures, FFI... 'map' definitely should
be defined here, even if its definition will have ghc-specific version

'base' package should provide common API to compiler libraries. for
example, it should provide functions integerMult, arrayCreate, type IO
and so on 

*hc-base packages should provide implementations of these functions,
specific for concrete compiler, i.e.

arrayCreate = arrayCreate#
integerMult (S# a) (S# b) = intMult# a è
...
newtype IO = IO (...)

'base' package should then analyze compiler name and version and
import appropriate modules or define operations itself if there is no
implementation:

module ArrayOps where
#if GHC
import GHC.Arr
#elseif Hugs
import Hugs.Arr
#else
type Array a b = [(a,b)]
arrayCreate = []
...
#endif

all functions/types implemented in pure Haskell, all complex algorithms, all
class definitions should go away from this package! it just provides
common set of low-level operations and of no interest for end users.
it's just a tool which provides "virtual Haskell compiler API", which
allows to write all other libraries in rather portable way


>> last line: i have some experience of writing compiler-independent code
>> with Haskell and C++ and believe that this plan is realistic
> The differences between Haskell compilers may well be bigger than
> those between C++ compilers!

> I wish you the best of luck, and think this would be really nice to
> hvae - unfortunately I think its unobtainable - but if we could just
> get some of this goodness that would be fantastic!

you've missed one point - we _already_ have working solution, the
'base' library itself. all that we need is just to split it carefully
to modules which may be independently upgraded, plus add compatibility
with previous compiler version that 'base' currently lacks. so, it's
more moving code around and careful planning task than a real
technological challenge :)

i think you just misunderstood me - i don't plan to make ultimate solution,
just to solve some current meaningless problems - say, that we can't use
old MArray interface with ghc 6.6 or new implementation of HashTable
with ghc 6.2. cabal provided us with all the instruments required -
all that is remain is to refactor base library to make it
compiler-version-independent

-- 
Best regards,
 Bulat                            mailto:Bulat.Ziganshin at gmail.com