<div dir="ltr">I looked at prose and did some tests. It builds and works very well functionally (normalization tests passed) but turns out to be pretty poor on normalization performance (171 times slower than text-icu). I believe it can be improved with some changes to the data structures. Though the performance may or may not matter depending on your use case.<div><br></div><div>Here are the results of a quick normalization benchmarking test that I did using text-icu, unicode-transforms (bindings to the utf8proc C library) and prose:<div><br></div><div>text-icu = 1 sec (224 MB/s on the test machine)</div><div>unicode-transforms = 6 sec (40 MB/s)</div><div>prose = 171 sec (1.3 MB/s)</div></div><div><br></div><div>It looks like icu is the gold standard in performance. Even GNU libunistring's performance seems to be very similar to utf8proc.</div><div><br></div><div>-harendra</div></div><div class="gmail_extra"><br><div class="gmail_quote">On 25 March 2016 at 15:57, Harendra Kumar <span dir="ltr"><<a href="mailto:harendra.kumar@gmail.com" target="_blank">harendra.kumar@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Ah, I created a package for unicode normalization already since I got no responses to my mail:<div><br></div><div><a href="https://github.com/harendra-kumar/unicode-transforms" target="_blank">https://github.com/harendra-kumar/unicode-transforms</a><br></div><div><br></div><div>I will take a look at prose as well since it is native Haskell. It does not seem to be on Hackage yet.</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>-harendra</div><div><br></div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On 25 March 2016 at 05:08, Rob Leslie <span dir="ltr"><<a href="mailto:rob@mars.org" target="_blank">rob@mars.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div>I don’t have a good answer, but I thought I’d mention this project which looks interesting and I’m considering using myself:</div><div><br></div><div> <a href="https://github.com/llelf/prose" target="_blank">https://github.com/llelf/prose</a></div><div><br></div><div><span style="text-align:-webkit-auto">-- </span></div><div><span style="border-collapse:separate;line-height:normal;border-spacing:0px"><div style="word-wrap:break-word">Rob Leslie<br><a href="mailto:rob@mars.org" target="_blank">rob@mars.org</a><br><br></div></span>
</div>
<br><div><blockquote type="cite"><div><div><div>On Mar 17, 2016, at 12:59 AM, Harendra Kumar <<a href="mailto:harendra.kumar@gmail.com" target="_blank">harendra.kumar@gmail.com</a>> wrote:</div><br></div></div><div><div><div><div dir="ltr">I looked around and found only one package, text-icu which provides unicode normalization operations and a lot more. But text-icu depends on the icu library being installed on the system. We would prefer to avoid dependency on the icu library.<div><br></div><div>Is there a lightweight alternative which does not depend on icu? It could be a pure Haskell package or bindings to a lightweight C library where the library is small and shipped with the package itself.<div><br></div><div>I wonder if there is a need for unicode normalization operations in GHC code itself? If so how does it handle that?</div><div><br></div><div>I found a lightweight C library (<a href="https://github.com/JuliaLang/utf8proc" target="_blank">https://github.com/JuliaLang/utf8proc</a>) for normalization and case folding used by the Julia lang project. If there is no other option I am considering creating bindings to this library.</div><div><br></div><div>Any pointers, thoughts?</div><div><br></div><div>Thanks,</div><div>Harendra</div></div></div></div></div>
_______________________________________________<br>Haskell-Cafe mailing list<br><a href="mailto:Haskell-Cafe@haskell.org" target="_blank">Haskell-Cafe@haskell.org</a><br><a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe</a><br></div></blockquote></div><br></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>