[Haskell-cafe] Memory leak from close fields in conduit

[Michael Snoyman]

Hi all,

Hideyuki Tanaka alerted me[1] to a memory leak in conduit. Long story
short: it appears that Pipe composition leads to collection of a large
number of `return ()` actions for unnecessary memory cleanup. We came
up with a possible solution: a Finalize type[2]. In both of our
testing, this eliminates the memory leak in question.

So now I have a few questions:

1. Does anyone have a different idea on how to approach this problem?
2. Are there any refinements to be applied to the solution?
3. Technically, this is an API breakage, but (a) conduit 0.4 was
released very recently, (b) this is a fairly important change, and (c)
the real-world breakage from this is very minimal. Will anyone call
foul if I release such a change as 0.4.1?

Just to stress 3c: I compiled all of Yesod against this modified
version of conduit, and only http-conduit had to be tweaked at all.

Michael

[1] https://github.com/snoyberg/conduit/issues/40
[2] https://github.com/snoyberg/conduit/commit/ebc7a68e0f1f0c1b80c0105916b00060207944dd#L0R37