[Haskell-community] technical thoughts on stack

[Bardur Arantsson]

On 2016-09-16 02:19, Christopher Allen wrote:
> The YAML parser backing hpack and Stack supports a subset of yaml, so
> the complexity isn't there.

As I understand it, the point of using YAML in this case isn't that
"there's a library for that". It's that it's *standard* such that other
tools can read/write it and know for certain that what gets read/written
is at the very least 'syntactically' correct. As such, using some
nebulous "subset of yaml" defined by an implementation[1] doesn't really
make sense for interop purposes.

(I'm honestly not sure how many *actually* fully conforming YAML parsers
there are out there, but I'd place my bets of "not many" given the
complexity of the spec. See http://yaml.org/spec/1.2/spec.html )

> TOML isn't supported because a well-supported/known-good library
> wasn't to hand, whereas the yaml one did. I don't think people would
> mind changing over, but somebody has to do the work validating that
> the one-still-maintained-of-two-libraries TOML library is suitable.

Even though there are probably fewer implementations, TOML is aiming[2]
to become a standard format, so it's (hopefully going to be) a standard
and a much simpler one than YAML. That'll hopefully lead to more
implementations over time, but who knows?

It's *also* simple enough that implementing a fully conforming
parser/writer is probably a few days' work, if that. That means that any
tools that cannot use an existing library (for whatever reason) actually
have a shot in hell of implementing TOML support themselves. That is not
the case for YAML -- witnessed by the number of yaml libaries that are
actually just bindings for C-based yaml libraries.

(I have other issues with TOML, but that's a different matter. The major
sticking point for me would be the lack of *clean* support for nesting
without the order-dependence, but that may not matter much for Cabal.)

> 
> Keep in mind while discussing all this that Cabal's own solver doesn't
> actually live outside of cabal-install.

FWIW, this seems mostly like a historical accident more than anything
else and at this point it's a *very* non-trivial undertaking to fully
separate it out, though there have been steps in this direction. I don't
see it happening unless the parties who are actually interested in
*using* it outside of cabal-install do the majority of the work.

> Contributing to the Cabal project itself is a gauntlet run, which
> is partly why many of these
> projects spawned outside of it. Further, Cabal itself can't really
> grow any dependencies because of the bootstrap problem.
> 
> Nobody wants to shim / inline all the dependencies that enable the
> devs to work efficiently into Cabal and such a PR would never get
> merged anyway.

You're right that Cabal-the-library is generally a very difficult thing
to change (due to sheer inertia and stability guarantees), but -- as
Patrick Pelletier points out -- implementing a new file format shouldn't
matter for consumers of Cabal as long as Cabal emits the old standard
Cabal configuration data structures.

Meanwhile, it would open up the possibility of non-Cabal-based programs
understanding and writing .cabal files (up to semantics defined by
documentation). I'm not sure how much that benefit that would confer,
but given that Cabal's parser is already a mess[3] with loads of corner
cases I'd say it's probably worth it *just* from a software engineering
perspective to get the basic syntax parsing disconnected from Cabal and
to get rid of the old code.

Btw, Cabal/cabal-install development has opened up and picked up pace
considerably in the last year or two, you might want to reconsider
automatically concluding that such-and-such PR "[...] would never get
merged anyway".

Anyway, this sub-thread is probably veering too off-topic, so I'll stop
there.

Regards,

/b

Notes:

[1] At least, from a quick skim of the available documentation at
https://hackage.haskell.org/package/yaml I wasn't able deduce what
subset of YAML was supposed to be supported. Maybe I just missed it.

[2] As of now the README for the current v0.4.0 spec says: "Be warned,
this spec is still changing a lot. Until it's marked as 1.0, you should
assume that it is unstable and act accordingly.". I guess one can pin to
0.4.0.

[3] Work to at least replace it with a parsec-based implementation has
been happening here, but the amount of work in
https://github.com/haskell/cabal/pull/3602 seems indicative of a *lot*
of complexity. (Granted it's hard to tell exactly how much of that
complexity is due to mapping to Cabal's data structure vs. just the
"parse into a KV-map" bits.)