Converting things to and from binary

[Glynn Clements]

Tomasz Zielonka wrote:

> > > The general problem to solve is "How to we convert things to and from a
> > > binary format, so they can be efficiently written to and from disk".
> > 
> > Possibly the best way to handle this would be to extend the Storable
> > class, e.g.
> > 
> > class Storable a where
> > 	...
> > 	hPutObj :: Handle -> a -> IO ()
> > 	hGetObj :: Handle -> IO a
> > 	toByteList :: a -> [Word8]
> > 	fromByteList :: [Word8] -> a
> > 	toByteArray :: a -> Array Int Word8
> > 	fromByteArray :: Array Int Word8 -> a
> 
> Hmmm, I thought that Foreign.Storable.Storable is meant for different
> purposes, namely for accessing C data in memory and for marshalling
> between Haskell and C worlds. Are you sure that these functionalities
> must overlap?
> 
> Extending Storable would require to extend its every existing instance.
> It might be better to introduce some subclass, but I'm not sure that
> every type supporting marshalling to and from binary format can
> support operations from Storable.
> 
> It's quite easy to write value of type [[Int]] to disk, but how to
> define alignment and sizeOf methods for this type?

Sorry, I'd overlooked the fact that Storable was limited to fixed-size
types.

In any case, reading/writing data to memory and to disk aren't all
that different; the existence of mmap() relies upon that fact. And,
mmap() aside, on most OSes, all I/O comes down to reading and writing
blocks of memory (i.e. read() and write()); everything else is just
additional layers.

I still think that we need a superclass of Storable, as any instance
of Storable can be read and written with e.g.:

hPutObj :: (Storable a) => Handle -> a -> IO ()
hPutObj h x = do
	alloca $ \ptr -> do
		poke ptr x
		fd <- handleToFd h
		let bptr = castPtr ptr :: Ptr CChar
		let blen = fromIntegral $ sizeOf x
		write fd bptr blen

hGetObj' :: (Storable a) => Handle -> a -> IO a
hGetObj' h dummy = do
	alloca $ \ptr -> do
		let blen = fromIntegral $ sizeOf dummy
		let bptr = castPtr ptr :: Ptr CChar
		fd <- handleToFd h
		read fd bptr blen
		x <- peek ptr
		return x

hGetObj :: (Storable a) => Handle -> IO a
hGetObj h = hGetObj' h undefined

A similar process can be used to convert instances of Storable to or
from a list (or array) of bytes.

-- 
Glynn Clements <glynn.clements@virgin.net>