cabal for ghc 6.8.2, package upgrade warnings, please?

[Claus Reinke]

> We're pretty much fully backwards compatible with old
> .cabal files, the only problem is Setup.hs files.

compatible, as in "can process them", but some things,
like the base split, will lead to plain failures, even though
cabal could put 2 and 2 together when it sees an old
.cabal file and missing dependencies in packages split
from base.

the package maintainer hasn't updated, the package
builder has no idea what goes wrong, the whole thing
breaks because of outside influences (libraries updates),
and cabal could make a good guess, it just doesn't tell.

> This would be great, unfortunately I don't think it's technically
> feasible with the current design. At the moment, the Setup.hs is the
> first entry point and if that doesn't compile then we're stuffed. We
> have no opportunity to offer helpful messages, the user just gets a
> deeply unfriendly type error message.

ah, yes, that is a problem. but even if non-standard Setup.hs
is now discouraged, it is still part of cabal's design, so the same
issue will come up the next time you improve the api.

what about the old idea that there should be a 'cabal' script?
in the past, that seemed somewhat superfluous, as it wouldn't
have done anything but call "runhaskell Setup", but now it
seems that extra level of indirection would allow for helpful
checks and messages before or after compilation of Setup.hs
fails.

that could include format and age checks on .cabal (if
it doesn't mention any new dependencies or isn't precise in
versions, there are some likely issues it will encounter with
a new ghc/cabal version; if it depends on base, but needs
containers, it is pre-base-split, etc.), import/type checks
on Setup.hs (does it expect the old or the new api), or
rough postprocessing of error messages from compiling
Setup.hs (pattern matching on the kind of type errors or
missing import errors that unambiguously imply old api).

> Setup.hs files are allowed to use the entire Cabal api, which includes a
> lot of useful stuff but it completely exposes the inner workings. Since
> we keep improving the inner workings that api keeps changing. Even the
> more public parts place constraints on the implementation that we would
> like to lift in future (esp the UserHooks api).
> ..

ok, i think i've accepted before that cabal's design would
make it difficult to have a compatibility module. and you
say that this design gives you liberties that you wouldn't
want to trade for a more restrictive, more maintenance-
friendly design. okay, but that isn't the only way to help
with maintenance:

- prefixing 'Compat.' to all cabal imports in Setup.hs
    seems out - too much duplication of implementation.
    but perhaps that duplication of implementation would
    have been worth it for a few months, if only to be
    able to issue warnings and give package authors or
    third parties some time to upgrade their packages.

- if the new api can do everything the old api can do,
    is it possible to transform an old Setup.hs into one
    that would very nearly work with the new api?

    using Language.Haskell's parser/ast/pretty-printer
    and one of the no-boilerplater libraries, one could
    search for uses of the old api and replace with
    uses of the new api wherever drop-in replacements
    are available. the bulk of the code would be rules
    encoding old->new replacements.

    when you changed cabal's api, you must have had
    some ideas about how the new api would replace
    uses of the old api?

- even if the transformation cannot be automated
    completely, can the old code be analysed to
    issue warnings about features it uses which are
    no longer available, together with a link to a wiki
    page that explains how to replace those uses of
    old cabal api features with uses of new cabal
    api features? again, give the package builder
    more info than "it doesn't compile/build".

- is there a wiki page that explains the cabal api
    changes in more detail, with rationale, with
    examples of how to replace old uses with new?

    something that cabal and hackage home pages,
    as well as warning issued by cabal, could point to.

> The current approach works about 90%, in that 10% of packages broke in
> the transition from Cabal 1.1.6 to 1.2 (according to my hackage survey).

that survey was useful, but it only covered hackage,
and even there it found 10% breakage not caused
by package authors or builders. 10% breakage
just by time passing and by cabal not wanting to
provide the old implementation side-by-side with
the new, not by package providers making any changes..

as you say, the cabal is going to move on, so this is
an opportunity to see what issues come up and how
things might be changed to handle such issues better
next time.

even simple things help, such as an additional
indirection (a 'cabal' script calling 'runhaskell Setup',
now able to give warnings instead of fail), wiki pages
documenting what people have to do to follow cabal
or library changes, warning messages that direct
package builders to such wiki pages.

is there a hackage mailing list offering to keep all
package maintainers informed about build results,
breakage, issues, api changes, discussion? the email
addresses could be extracted from the .cabal files
being uploaded, but hackage would need to ask
for permission before adding anyone to such list.

> I've added a link from the cabal website and I've reorganised the page
> so it's hopefully rather clearer:
>
> http://haskell.org/haskellwiki/Upgrading_packages

thanks. the part that still needs expansion is the part
1.3 on cabal api changes: any feature that the old
cabal made available which is not unchanged in the
new api should be listed, side-by-side with intended
replacements.

that is, a user-level equivalent to the page that would
explain and summarize the cabal design changes.

claus