[Haskell-cafe] Re: String vs ByteString

[Michael Snoyman]

On Tue, Aug 17, 2010 at 10:08 AM, Ketil Malde <ketil at malde.org> wrote:

> Benedikt Huber <benjovi at gmx.net> writes:
>
> > Despite of all this, I think the performance of the text
> > package is very promising, and hope it will improve further!
>
> I agree, Data.Text is great.  Unfortunately, its internal use of UTF-16
> makes it inefficient for many purposes.
>
> [..]

>From a quick glance, it appears that utf8-string is the most complete
> and well maintained of the crowd, but I could be wrong.  It'd be nice if
> a similar effort as Data.Text has seen could be applied to
> e.g. utf8-string, to produce a similarly efficient and effective library
> and allow the deprecation of the others.  IMO, this could in time
> replace .Char8 as the default ByteString string representation.
> Hackathon, anyone?
>
> Let me ask the question a different way: what are the motivations for
having the text package use UTF-16 internaly? I know that some system APIs
in Windows use it (at least, I think they do), and perhaps it's more
efficient for certain types of processing, but overall do those benefits
outweigh all of the reasons for UTF-8 pointed out in this thread?

Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20100817/fd367e3d/attachment.html