I got 4.7s for similar amount of data in 2013.<br>However I was pretty sure that fully inlined implementation could potentially go 5x faster.<br><a href="http://hackage.haskell.org/package/hPDB">http://hackage.haskell.org/package/hPDB</a><br><br>Please check xeno XML parser benchmarks for another example.<br><a href="https://hackage.haskell.org/package/xeno">https://hackage.haskell.org/package/xeno</a><br><div class="gmail_quote"><div dir="ltr">On Fri, 31 Aug 2018 at 14:41, <<a href="mailto:haskell-cafe-request@haskell.org">haskell-cafe-request@haskell.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Send Haskell-Cafe mailing list submissions to<br>
<a href="mailto:haskell-cafe@haskell.org" target="_blank">haskell-cafe@haskell.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe" rel="noreferrer" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:haskell-cafe-request@haskell.org" target="_blank">haskell-cafe-request@haskell.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:haskell-cafe-owner@haskell.org" target="_blank">haskell-cafe-owner@haskell.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of Haskell-Cafe digest..."<br>
Today's Topics:<br>
<br>
1. Re: HDBC packages looking for maintainer (Tobias Dammers)<br>
2. Re: Alternative instance for non-backtracking parsers<br>
(Olaf Klinke)<br>
3. Re: Alternative instance for non-backtracking parsers<br>
(Bardur Arantsson)<br>
<br><br><br>---------- Forwarded message ----------<br>From: Tobias Dammers <<a href="mailto:tdammers@gmail.com" target="_blank">tdammers@gmail.com</a>><br>To: <a href="mailto:haskell-cafe@haskell.org" target="_blank">haskell-cafe@haskell.org</a><br>Cc: <br>Bcc: <br>Date: Thu, 30 Aug 2018 15:24:04 +0200<br>Subject: Re: [Haskell-cafe] HDBC packages looking for maintainer<br>Hi,<br>
<br>
I'd be interested. I've used HDBC on a few projects, and my yeshql<br>
library was originally built with HDBC as the only backend. It would be<br>
a terrible shame to see this bitrot.<br>
<br>
Cheers,<br>
<br>
Tobias (tdammers on github etc.)<br>
<br>
On Mon, Aug 13, 2018 at 12:07:38PM +0200, Erik Hesselink wrote:<br>
> Hi all,<br>
> <br>
> I've been the maintainer for some of the HDBC packages for a while now.<br>
> Sadly, I've mostly neglected them due to lack of time and usage. While the<br>
> packages mostly work, there are occasional pull requests and updates for<br>
> new compiler versions.<br>
> <br>
> Because of this I'm looking for someone who wants to take over HDBC and<br>
> related packages [1]. If you use HDBC and would like to take over<br>
> maintainership, please let me know and we can get things set up.<br>
> <br>
> Regards,<br>
> <br>
> Erik<br>
> <br>
> [1] <a href="https://github.com/hdbc" rel="noreferrer" target="_blank">https://github.com/hdbc</a><br>
<br>
> _______________________________________________<br>
> Haskell-Cafe mailing list<br>
> To (un)subscribe, modify options or view archives go to:<br>
> <a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe" rel="noreferrer" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe</a><br>
> Only members subscribed via the mailman list are allowed to post.<br>
<br>
<br>
-- <br>
Tobias Dammers - <a href="mailto:tdammers@gmail.com" target="_blank">tdammers@gmail.com</a><br>
<br>
<br><br><br>---------- Forwarded message ----------<br>From: Olaf Klinke <<a href="mailto:olf@aatal-apotheke.de" target="_blank">olf@aatal-apotheke.de</a>><br>To: PY <<a href="mailto:aquagnu@gmail.com" target="_blank">aquagnu@gmail.com</a>><br>Cc: haskell-cafe <<a href="mailto:haskell-cafe@haskell.org" target="_blank">haskell-cafe@haskell.org</a>><br>Bcc: <br>Date: Thu, 30 Aug 2018 20:21:07 +0200<br>Subject: Re: [Haskell-cafe] Alternative instance for non-backtracking parsers<br>> Hello, Olaf. I have some distrust of elegant solutions (one of them are<br>
> C.P. libs).<br>
<br>
I have a program that parses several CSV files, one of them 50MB in size, and writes its result as HTML. When I started optimizing, the execution time was 144 seconds. Profiling (thanks to Jasper Van der Jeugt for writing profiteur!) revealed that most of the time was spent parsing and postprocessing the 50MB CSV file. Changing the data structure of the postprocessing stage cut down the execution time to 32 seconds, but still the majority is spent on parsing. <br>
Then I realized that (StateT String Maybe) is a parser which conveniently has all the class instances one needs, most notably its Alternative instance make it a backtracking parser. After defining a few combinators I was able to swap out my megaparsec parser against the new parser, which slashed execution time in half. Now most of the parsing time is dedicated to transforming text to numbers and dates. I doubt that parsing time can be reduced much further [*]. The new parser was identical to the old parser, only the combinators now come from another module. That is the elegant thing about monadic parser libraries. <br>
I will now use the fast parser by default, and if it returns a Nothing, the program will suggest a command line flag that switches to the original megaparsec parser, exactly telling the user where the parse failed and why. <br>
I am not sure whether there is another family of parsers that have interfaces so similar that switching from one package to another is as effortless as monadic parsers. <br>
<br>
Cheers<br>
Olaf<br>
<br>
[*] To the parser experts on this list: How much time should a parser take that processes a 50MB, 130000-line text file, extracting 5 values (String, UTCTime, Int, Double) from each line?<br>
<br><br><br>---------- Forwarded message ----------<br>From: Bardur Arantsson <<a href="mailto:spam@scientician.net" target="_blank">spam@scientician.net</a>><br>To: <a href="mailto:haskell-cafe@haskell.org" target="_blank">haskell-cafe@haskell.org</a><br>Cc: <br>Bcc: <br>Date: Thu, 30 Aug 2018 21:43:55 +0200<br>Subject: Re: [Haskell-cafe] Alternative instance for non-backtracking parsers<br>On 30/08/2018 20.21, Olaf Klinke wrote:<br>
>> Hello, Olaf. I have some distrust of elegant solutions (one of them are<br>
>> C.P. libs).<br>
> <br>
> [*] To the parser experts on this list: How much time should a parser take that processes a 50MB, 130000-line text file, extracting 5 values (String, UTCTime, Int, Double) from each line?<br>
<br>
Not an expert, but for something as (relatively!) standard as CSV, I'd<br>
probably go for a specialized solution like 'cassava', which seems like<br>
it does quite well according to <a href="https://github.com/haskell-perf/csv" rel="noreferrer" target="_blank">https://github.com/haskell-perf/csv</a><br>
<br>
Based purely the lines/second numbers on that page and the number you've<br>
given, I'd guesstimate that your parsing could potentially be as fast as<br>
(3.185ms / 1000 lines) * 130000 lines = 414.05ms = 0.4 s.<br>
<br>
(Of coure that still doesn't account for extracting the Int, Double,<br>
etc., but there are also specialized solutions for that which should be<br>
pretty hard to beat, see e.g. bytestring-lexing.)<br>
<br>
It's also probably a bit less elegant than a generic parsec-like thing,<br>
but that's to be expected for a more special-case solution.<br>
<br>
Regards,<br>
<br>
<br>
_______________________________________________<br>
Haskell-Cafe mailing list<br>
<a href="mailto:Haskell-Cafe@haskell.org" target="_blank">Haskell-Cafe@haskell.org</a><br>
<a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe" rel="noreferrer" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe</a><br>
</blockquote></div>