[Yhc] Regarding Yhc bytecode versioning

[Neil Mitchell]

Hi Robert,

> * Compatibility-breaking changes to the bytecode file format.
>
> The second problem is the one I wish to discuss.  Since I began work
> on the Yhc bytecode library in May, there have been at least two
> instances of compatibility-breaking changes.  One relates to Hat
> integration, I believe, and the other has to do with the switch to
> libFFI.

> Both of these changes were made without bumping the version
> number that appears in the file header. I would like to suggest that
> such changes be avoided in the future.

While there is still work ongoing I think its unfortunate but a
definate reality that things will have to be broken in binary file
formats - we want to put the .hi information in .hbc files, that will
require breaking. We want to do a linking pass to merge multiple
.hbc's into one - that will require breaking.

However, you're entirely right, any change that breaks anything from
now on needs a version bump.

> It may also be a good time to think about a ways to future-proof the
> file format so that future additions can be made without breaking
> compatibility.  Right now the format is quite fragile.  Perhaps we
> could take inspiration from the Java classfile format.  The basic
> idea is that there are named blocks of data with a minimal header
> which gives the name of the block and the size of the data payload.
> The name of the block defines the meaning of the data.  Eg, the
> 'CODE' block contains bytecode instructions, etc.  If any block is
> encountered with an unrecognized name, it is ignored.  That way, one
> can have optional blocks, or one can add blocks without breaking
> compatibility.  One can also have optional information (like
> debugging symbols) and things of that nature.

That was always the intention, I'm hoping that once we move to having
Yhc.ByteCode handle everything, we can treat that as an abstraction
over the file format, and then we can work on defining a new .hbc file
format designed to last a very long time without changes. Me and Tom
did a brain storm a while on the "perfect" .hbc file format, but
unfortunately we've never had time to implement or document it...

As a side note, all these issues apply equally to .ycr files, for
which there are now 3 projects making active use of. For that I have
defined a Haskell interface which is the only supported way of getting
at the data (that can't be done for the .hbc files, as the C needs
access to them). I am also very agressively bumping the version number
- the tiniest change gets a new version. I am also ignoring backwards
compatability, at every version bump I just ignore all old files.

Thanks

Neil