<div dir="ltr"><div>On Sun, May 29, 2016 at 1:53 PM, Stephen Tetley <span dir="ltr"><<a href="mailto:stephen.tetley@gmail.com" target="_blank">stephen.tetley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">Isn't this a problem of JSON rather than it's parsers?<br></blockquote></div><div><br></div>I can understand that a file with multiple JSONs is not a legal "JSON text". But... isn't that issue separate from whether parsers expect terminated strings, or, conversely, are tolerant of arbitrary text following the JSON expr? Scheme "read" functions from time immemorial would read the first expression off a handle without worrying about what followed it! It doesn't mean the whole file needs to be valid JSON, just that each prefix chewed off the front is valid JSON.<div><br></div><div>Thanks to Nikita for the links to json-stream and json-incremental-decoder. My understanding is that if I use a top-level array to wrap the objects, then these approaches will let me retain a streaming/incremental IO. I'm not sure yet how to use this to stream output from a monadic computation.</div><div><br></div><div>Let me be specific about the scenario I'm trying to handle:</div><div><br></div><div>Criterion loops over benchmarks, and after running each, it writes the report out to disk appending it to a file:</div><div><br></div><div><a href="https://github.com/bos/criterion/blob/fb815c928af2cb089cea9399503304530e27883d/Criterion/Internal.hs#L128">https://github.com/bos/criterion/blob/fb815c928af2cb089cea9399503304530e27883d/Criterion/Internal.hs#L128</a><br></div><div><br></div><div>This way, the report doesn't sit in memory affecting subsequent benchmarks. (I.e. polluting the live set for major GC.) When all benchmarks are complete, the reports are read back from the file.</div><div><br></div><div>There are bugs in the binary serialization used in the linked code. We want to switch it to dump and read back in JSON instead.</div><div><br></div><div>In this case, we can just write an initial "[" to the file, and then serialize one JSON object at a time, interspersed with ",". That's ok... but it's kind of an ugly solution -- it requires that, we, the client of the JSON serialization API, make assumptions about the serialization format and reimplement a tiny tiny fraction of it.</div><div><br></div><div>Cheers,</div><div> -Ryan</div><div><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, May 29, 2016 at 1:53 PM, Stephen Tetley <span dir="ltr"><<a href="mailto:stephen.tetley@gmail.com" target="_blank">stephen.tetley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">Hi Ryan<br>
<br>
Isn't this a problem of JSON rather than it's parsers?<br>
<br>
That's too say I believe (but could easily be wrong...) that a file<br>
with multiple JSON objects would be ill-formed; it would be<br>
well-formed if the multiple objects were in a single top-level array.<br>
<div class=""><div class="h5"><br>
On 29 May 2016 at 18:09, Ryan Newton <<a href="mailto:rrnewton@gmail.com">rrnewton@gmail.com</a>> wrote:<br>
> As someone who spent many years putting data in S-expression format, it<br>
> seems natural to me to write multiple S-expressions (or JSON objects) to a<br>
> file, and expect a reader to be able to read them back one at a time.<br>
><br>
> This seems comparatively uncommon in the JSON world. Accordingly, it looks<br>
> like the most popular JSON parsing lib, Aeson, doesn't directly provide this<br>
> functionality. Functions like decode just return a "Maybe a", not the<br>
> left-over input, meaning that you would need to somehow split up your<br>
> multi-object file before attempting to parse, which is annoying and error<br>
> prone.<br>
><br>
> It looks like maybe you can get Aeson to do what I want by dropping down to<br>
> the attoparsec layer and messing with IResult.<br>
><br>
> But is there a better way to do this? Would this be a good convenience<br>
> routine to add to aeson in a PR? I.e. would anyone else use this?<br>
><br>
> Thanks,<br>
> -Ryan<br>
><br>
><br>
><br>
</div></div><div class=""><div class="h5">> _______________________________________________<br>
> Haskell-Cafe mailing list<br>
> <a href="mailto:Haskell-Cafe@haskell.org">Haskell-Cafe@haskell.org</a><br>
> <a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe" rel="noreferrer" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe</a><br>
><br>
_______________________________________________<br>
Haskell-Cafe mailing list<br>
<a href="mailto:Haskell-Cafe@haskell.org">Haskell-Cafe@haskell.org</a><br>
<a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe" rel="noreferrer" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe</a><br>
</div></div></blockquote></div><br></div></div>