getting a Binary module into the standard libs

[Malcolm Wallace]

Hal Daume III <hdaume@ISI.EDU> writes:

> It seems that in order to accomplish this, the BinMem constructor needs
> to be augmented with two fields, one Word8 which contains bits which
> have been "put" but haven't yet been written to the array and another
> Word8 which stores the current bit position we are at in this Word8. 
> Then, the work comes down mostly to bit-twiddling in the putWord8 and
> putBit functions (putBit being the simpler of the two).  It seems the
> BinIO constructor would require basically the identical thing, which
> means perhaps this stuff should be added to the BinHandleState variable.

That's basically what we did with the nhc98 implementation, yes, except
that the BinHandle was originally implemented in C rather than Haskell.

> It might be wise also to add a function like:
>   flushByte :: BinHandle -> IO ()

Yep.

> There's a concern that if you only write, say, 2 of the last 8 bits,
> this won't actually get written unless you call flushByte.  I don't know
> how this was handled in the NHC version; perhaps just require the user
> to call flushByte at the end if they're worried about such behavior?

The final-byte flush should be automatic when the BinHandle is closed.

One very tricky issue is what to do with a bitstream that has been
written in a prior program run, with the final byte incomplete, and
is then re-opened in AppendMode.  We used to have a little trick that
added an extra three-bit value to the end of the final byte of the
actual file.  This represented the position of the last "real" bit
in the stream as an offset backwards from the final bit in the file
(allowing for the three-bit tag itself of course).  Later, we realised
that this solution was kind of nasty because it adds meta-data to
the binary stream, which is OK if the only code that ever processes
the bits is in Haskell, but if it is any kind of external format,
e.g. JPEG, then it really screws you up.  I think the real solution is
to forbid AppendMode.

Regards,
    Malcolm