[Haskell] ANNOUNCE: FPS - FastPackedStrings 0.2

[Bulat Ziganshin]

Hello Donald,

Wednesday, April 19, 2006, 1:23:25 PM, you wrote:

> I'm pleased to announce version 0.2 of FPS, the fast, packed string
> library for Haskell.

i want to write some critique about current state of your lib.
something of this may be interesting only for me

1. it don't support Unicode. there are at least two libs (Simon's and from
JHC) that uses UTF-8 to do this. of course, they will be not so
efficient on some operations. i think that it is essential to
general-purpose library and Data.PackedString replacement. Simon's lib
already implements utf-8, latin-1, ucs-2 and ucs-4 encoding. may be
it's possible to join them all together in one lib that uses
prerocessing or some other technique to implement differences between
utf-8 and fixed-width encoding

2. it uses ForeignPtr, that is slow in ghc 6.4 and require more
memory than ByteArray# in any ghc version. in addition, you place two
Ints here. it's question of taste, of course - whether we want to have
better speed or memory requirements. i personally prefer to have
minimal memory overhead that will be accomplished by using ByteArray#
plus one Int to hold string's size. it's possible to write lib so it
will use any unboxed array and this will work both with ForeignPtr
and ByteArray# implementations. It will also allow to implement
general list-like interface for any immutable arrays what should be
useful, in particular for collections framework that is now written by
JP Bernardy. of course, using general Array instead of ForeignPtr will
make some functions (such as mmap) unimplementable for common case

3. i'm also have two libs that is intended to be included in GHC 6.6. one
of them is reworking of Array facilities, another replaces the Handles
functionality. there are some common interests between us here - adding
common ListEmulation interface to Arrays, then implementing
FastStrings on top of it, then implementing really fast I/O for these
FastStrings. we definitely should join our work at some moment to provide
consistent features set for ghc 6.6. currently i can say that i also tried
to implement fast I/O using packed strings from your lib and from
jhc's module and - yes, i found that without prior `findEOL` it's hard
to make it fast with your (ForeignPtr-based) implementation. it seems
that you encounter the same problem

disclaimer: i know that i say about huge changes in the library
structure, so it's just thoughts on this issue. the only really
required thing is to make my streams I/O work efficient with your lib
if both libs will go into 6.6



-- 
Best regards,
 Bulat                            mailto:Bulat.Ziganshin at gmail.com