[Haskell-cafe] Is it possible to have constant-space JSON decoding?

[Herbert Valerio Riedel]

Iustin Pop <iustin at google.com> writes:

[...]

> Let's say we have a simple JSON message: an array of 5 million numbers.
> I would like to parse this in constant space, such that if I only need
> the last element, overall memory usage is low (yes, unrealistic use, but
> please bear with me for a moment).
>
> Using aeson, I thought the following program will be nicely-behaved:

part of the problem is that aeson builds an intermediate JSON parse-tree
which has quite an overhead for representing a list of numbers on the
heap as each numbers requires multiple heap objects (see also [1]). This
is an area where e.g. Python has a significantly smaller footprint
(mostly due to a more efficient heap representation).

[...]

> It seems that the Array constructor holds a vector, and this results in
> too much strictness?

btw, using a list on the other hand would add an overhead of 2 words
(due to the ':' constructor) for representing each JSON array element in
the parse-tree, that's probably why aeson uses vectors instead of lists.

[...]

cheers,
  hvr

 [1]: https://github.com/bos/aeson/issues/22