[Haskell-cafe] Re： Yet another binary reader (10x faster thanHackage's binary; )

[Станислав Черничкин]

>These two assumptions are basically the most challenging issues stop
people from getting more performance

I guess so, that is why I've decided to introduce it. Actually this design
was inspired by my job task. I'm working on graph processing library
written in Scala, which stores large amount of data in of-heap memory in
binary format. I want to read this data. Storable is not good enough
because there is a variable-length data types as well. On other hand all
always available loaded in memory and no incremental parsing required.

So, my approach is something between Storable and fully featured serializes
such as binary/cereal.


ср, 20 мар. 2019 г. в 09:20, Dr.Koster <drkoster at qq.com>:

> These two assumptions are basically the most challenging issues stop
> people from getting more performance, e.g. If somehow you want to support
> incremental parsing, a primitive like `ensureN` probably need to do
> something like
>
> ```
> ensureN n = Parser $ \ buf ... k ->
>     if n < length buf then k buf ...
>                             else partial buf ... k
> ```
>
> The k appear twice inside `ensureN` 's  both branch, which will stop
> parsers from getting inline sooner or later since inlining k into both
> branch will produce exponential amount of code, so instead GHC will
> allocate k and make a jump, the best we can get is probably to try make it
> a joint point, so no allocation is needed, but that's the best you can get
> from argument passing buffers.
>
> Regards
> Han Dong
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Joachim Durchholz"<jo at durchholz.org>;
> *发送时间:* 2019年3月20日(星期三) 中午1:28
> *收件人:* "haskell-cafe"<haskell-cafe at haskell.org>;
> *主题:* Re: [Haskell-cafe] Yet another binary reader (10x faster
> thanHackage's binary; )
>
> Am 19.03.19 um 22:58 schrieb Станислав Черничкин:
> > possibly because of inlining of
> > specialization magic.
>
> I have seen people validate this kind of assumption, by looking at the
> various intermediate representations of code.
> If you're after performance, you may want to do that. You may find that
> things work differently than expected, with the potential for becoming a
> bottleneck when stuff is scaled up (more varied datatypes in the input
> stream, or more complex target datatypes, stuff hidden behind "oh I
> don't need GC yet", that kind of finding).
>
> Regards,
> Jo
> _______________________________________________
> Haskell-Cafe mailing list
> To (un)subscribe, modify options or view archives go to:
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
> Only members subscribed via the mailman list are allowed to post.
> _______________________________________________
> Haskell-Cafe mailing list
> To (un)subscribe, modify options or view archives go to:
> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
> Only members subscribed via the mailman list are allowed to post.



-- 
Sincerely, Stanislav Chernichkin.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20190320/cf19ff1a/attachment.html>