[web-devel] [WAI] Switching to WAI/Warp...

[Michael Snoyman]

On Sun, Feb 20, 2011 at 11:04 AM, Bardur Arantsson <spam at scientician.net> wrote:
> Hi all,
>
> I've recently switched my "Hums" UPnP server over to using WAI/Warp, but I'm
> seeing quite high CPU usage compared to monadic I/O. (1-7% with WAI as
> opposed to 0-1% for monadic).
>
> (Just to be clear: The server is using a constant amount of memory, so it's
> definitely not leaking or anything like that.)
>
> I think the source of the efficiency problem is that I'm enumerating strict
> ByteStrings from a file, but the WAI response enumerator requires Builder
> chunks, so I'm forced to use "fromByteString" to convert between the strict
> ByteString representation and the Builder representation -- I'm assuming
> this performs a memory copy.
>
> Is there any way to avoid this extra "fromByteString"?

blaze-builder is usually pretty intelligent about this. If I remember
correctly, Simon Meier said that it won't do a memory copy for
ByteStrings larger than 8k. In any event, if you want to force
insertion instead of copying, replace fromByteString with
insertByteString. It *might* be that the CPU overhead is warranted,
however: you may end up seeing increased system call overhead with
this switch, since the average chunk size will be smaller.

> Other than this little issue, WAI seems to be working very well and it seems
> like a great fit if you just want low-level access to the HTTP protocol.

Good to hear, I'm glad it's working out.

> On a slightly related note: Is there an elegant way to map enumerator chunk
> elements from type a -> b using a mapping function? Right now, I've embedded
> the ByteString -> Builder conversion deep in my enumerator, but it should
> really be happening at a higher level.

Yes, Data.Enumerator provides a map enumeratee. You can use the
enumeratee to either modify at enumerator or an enumeratee, eg:

f :: a -> b
e :: Enumerator a IO x
i :: Iteratee b IO x

e `joinE` map f :: Enumerator b IO x
joinI $ map f $$ i :: Iteratee a IO x

I hope that clarifies things.

Michael