[Haskell-cafe] Optimization with Strings ?

[wren ng thornton]

David Menendez wrote:
> Alec Berryman wrote:
>> I don't know of a library to intern strings, but it's not too hard to
>> implement.  I couldn't find the code I wrote to do this, but I looked
>> around a bit and this is about what I remember doing:
>>
>> http://www.haskell.org/pipermail/haskell-cafe/2005-June/010335.html
>>
>> For the application I was using, interning strings did provide a
>> significant reduction in memory, but for whatever reason didn't help
>> with speed.
> 
> I'd use a trie. Edison provides Data.Edison.Assoc.TernaryTrie, and
> there are a few other trie packages at hackage.

One of those:

 
http://hackage.haskell.org/cgi-bin/hackage-scripts/package/bytestring-trie

also uses ByteStrings instead of Strings. This will further reduce your 
memory footprint and will improve performance (because of cache 
consistency, less pointer chasing, and using C's fast string comparison 
function).

I've been meaning to write a ByteString interning library on top of it, 
but haven't found the time just yet.

-- 
Live well,
~wren