[Haskell-cafe] Re: Profiling makes memory leak go away? Is Haskell a practical language?

[apfelmus]

Oren Ben-Kiki wrote:
> The code is in http://hpaste.org/1314#a1 if anyone at all is willing
> to take pity on me and explain what I'm doing wrong.

There is an important point to note about streaming, namely that it
conflicts with knowing whether the input is syntactically correct or
not. In other words, evaluating (rResult reply) parses the hole input to
the bitter end. For example, take your input data

  cycle "a\n" = "a\na\na\na\n..."

Can we determine whether (rResult reply) is Nothing or Just after
reading the first 'a'? No, we can't because the hole parse could fail
far to the end resulting in Nothing.

You've probably noticed this already and introduced rToken for that
reason but I'm not sure.

> Chasing down my memory leak I got into a weird situation where adding
> a magic manual SCC section and compiling with -prof makes the leak
> disappear.

That sounds fishy. Note that manual SCC annotations + optimization
currently is a bit buggy in GHC, in the sense that optimization disturbs
correct cost attribution.

Does the program eat the output of yes 'a' without memory explosion
showed by "top"? If so, it could perhaps be a problem of the
optimization phase which maybe expands

  D.append parsed (case result of ...)

to

  case result of {
      Nothing -> D.append parsed D.empty
      Just .. -> D.append parsed (D.singleton...) }

The latter won't stream because of the reason noted above. But it's
highly unlikely that GHC performs a transformation that changes
semantics this radically.

> I can achieve the results I want in very short elegant code...

In my opinion, your streaming parser is not as elegant as it could be
which makes it hard to read the code. With monad transformers, we almost
have

  Parser a ~= StateT State (WriterT (D.DList Token) Maybe) a
           ~= State -> (D.DList Token, Maybe (a,State))

Your Parser is slightly different but I think that the monad instances
behave exactly the same way. The power functional programming is to
assemble big things from small things. Monad transformers supply tons of
small monads that you can reuse to build big monads. In a sense, the
many pattern matches on Maybe have already been written down in general
form by someone else.

Also, I'm really missing type signatures, especially for many. When
reading the code, I expected

  many :: Parser a -> Parser [a]

but had to conclude

  many :: Parser a -> Parser ()

by inspecting its code.


Regards,
apfelmus