<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta name=Title content=""><meta name=Keywords content=""><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:Calibri;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:595.0pt 842.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style></head><body bgcolor=white lang=EN-GB link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri;mso-fareast-language:EN-US'>Hi Bram,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri;mso-fareast-language:EN-US'> <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri;mso-fareast-language:EN-US'>Sorry for being a bit late to this -- I have been on the road.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri;mso-fareast-language:EN-US'> <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri;mso-fareast-language:EN-US'>I have switched over you example to pre-compile the REs and use ByteString and can see 13x speedup on scan and a 9x speedup on mapping. Curiously, nearly all of that speedup seems to be gained by lifting the RE compilation out of the loop but I am pretty sure there are gains to be had from re-writing the loops.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri;mso-fareast-language:EN-US'> <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri;mso-fareast-language:EN-US'>Do you have the Python code that was performing 80x better?<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri;mso-fareast-language:EN-US'> <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri;mso-fareast-language:EN-US'>Chris<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri;mso-fareast-language:EN-US'><o:p> </o:p></span></p><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span style='font-family:Calibri;color:black'>From: </span></b><span style='font-family:Calibri;color:black'>Alfredo Di Napoli <alfredo.dinapoli@gmail.com><br><b>Date: </b>Monday, 22 May 2017 at 08:48<br><b>To: </b>Bram Neijt <bneijt@gmail.com><br><b>Cc: </b>Станислав Черничкин <schernichkin@gmail.com>, haskell-cafe <haskell-cafe@haskell.org>, Chris Dornan <chris@chrisdornan.com><br><b>Subject: </b>Re: [Haskell-cafe] Haskell performance when it comes to regex?<o:p></o:p></span></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Hi Bram,<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>you might be interested in the “regex” package from my colleague Chris Dornan:<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal><a href="http://regex.uk/">http://regex.uk/</a><o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>I know some proper performance work still needs to be done, but I would be curious to hear your experience report ;)<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Alfredo<o:p></o:p></p></div></div><div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>On 19 May 2017 at 18:52, Bram Neijt <<a href="mailto:bneijt@gmail.com" target="_blank">bneijt@gmail.com</a>> wrote:<o:p></o:p></p><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm'><p class=MsoNormal>Thank you!<br><br>I already changed to Text instead, but I thought the regex was already<br>memoized by GHC, so that should not be a problem.<br><br>I'm trying regex-applicative now, maybe that will help, but it takes<br>some time to figure out the syntax. I'll also try to see if<br>precompilation helps.<br><br>Greetings,<br><br>Bram<o:p></o:p></p><div><div><p class=MsoNormal><br><br><br>On Fri, May 19, 2017 at 1:17 PM, Станислав Черничкин<br><<a href="mailto:schernichkin@gmail.com">schernichkin@gmail.com</a>> wrote:<br>> Try to use Text or ByteString instead of strings. Try to use compile and<br>> execute methods<br>> (<a href="http://hackage.haskell.org/package/regex-tdfa-1.2.1/docs/Text-Regex-TDFA-ByteString.html" target="_blank">http://hackage.haskell.org/package/regex-tdfa-1.2.1/docs/Text-Regex-TDFA-ByteString.html</a>),<br>> make sure regex get compiled once.<br>><br>> 2017-05-16 12:12 GMT+03:00 Bram Neijt <<a href="mailto:bneijt@gmail.com">bneijt@gmail.com</a>>:<br>>><br>>> Dear reader,<br>>><br>>> I decided to do a little project which is a simple search and replace<br>>> program for large text files.<br>>><br>>> Written in Haskell, it does a few different regex matches on each line<br>>> and stores them in a leveldb key-value store to create a<br>>> consistent/reviewable search-replace index. It should provide for some<br>>> simple/brute-force anonymization of data and therefore I called it<br>>> hanon (sorry, could not think of a better name).<br>>><br>>> <a href="https://github.com/BigDataRepublic/hanon" target="_blank">https://github.com/BigDataRepublic/hanon</a><br>>><br>>> The code works, but I've done some benchmarking to compare it with<br>>> Python and the code is about 80x slower then doing the same thing in<br>>> Python, making it useless for larger data files.<br>>><br>>> I'm obviously doing something wrong.<br>>><br>>> Could you give me tips on improving the performance of this code?<br>>> Probably mainly looking at<br>>><br>>> <a href="https://github.com/BigDataRepublic/hanon/blob/master/src/Mapper.hs" target="_blank">https://github.com/BigDataRepublic/hanon/blob/master/src/Mapper.hs</a><br>>><br>>> where the regex code lives?<br>>><br>>> Greetings,<br>>><br>>> Bram<br>>> _______________________________________________<br>>> Haskell-Cafe mailing list<br>>> To (un)subscribe, modify options or view archives go to:<br>>> <a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe</a><br>>> Only members subscribed via the mailman list are allowed to post.<br>><br>><br>><br>><br>> --<br>> Sincerely, Stanislav Chernichkin.<br>_______________________________________________<br>Haskell-Cafe mailing list<br>To (un)subscribe, modify options or view archives go to:<br><a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe</a><br>Only members subscribed via the mailman list are allowed to post.<o:p></o:p></p></div></div></blockquote></div><p class=MsoNormal><o:p> </o:p></p></div></div></body></html>