HaXml, memory usage and segmentation fault

[Joe English]

Dmitry Astapov wrote:

>  JE> and the Hugs interpreter just isn't designed to work with large
>  JE> inputs.  Try compiling the program instead.
> well, ghc-5.02 seems to dislike something inside XmlLib.hs - it could not
> find interface defs file for modules IOExts .. I plan to look more deeply
> into it though.

I got it to compile with ghc 5.02 using

	ghc --make -package lang translate.hs

The compiled version succeeds, but on a large document it uses
a *lot* of memory and starts paging pretty badly.

>  JE> Try the identity transform 'main = processXmlWith keep' on your sample
>  JE> document and see if that runs out of heap too.  If so, there's not
>  JE> much you can do short of replacing the HaXml parser.

I tried this as well, modifying your program to use an
XML parser I wrote a while ago that has better laziness
properties than the HaXML one.  Alas, my parser also
suffers from a space leak under Hugs, so this only deferred
the problem.  Under ghc/ghci, though, it has modest memory
requirements and runs without paging.


--Joe English

  jenglish@flightlab.com