[Haskell-cafe] Hackage package "synopsis" sections

Tue Sep 16 15:58:36 UTC 2014

On 09/16/2014 05:13 AM, Michael Snoyman wrote:
> On Tue, Sep 16, 2014 at 2:36 AM, Mateusz Kowalczyk <fuuzetsu at fuuzetsu.co.uk>
> wrote:
> 
>> On 09/15/2014 09:11 PM, Michael Snoyman wrote:
>>> I've had cases where some code I wanted to put in an example could not be
>>> expressed in haddock due to escaping rules.
>>
>> The parser has been improved and the escaping rules have been made
>> saner. You shouldn't have real problems since 2.14 release which came
>> out along with GHC 7.8.1. If you have problems with escaping then I'd
>> like to hear about it on the issue tracker rather than stumbling upon
>> complaints on café.
>>
>>
> I ran into these issues over four years ago, discussed them at the time,
> and the general consensus I got at the time was that these issues weren't
> going to be resolved, so I didn't follow up. I've seen nothing in recent
> release announcements to imply otherwise. 

Really? [1] has quite a few things and I even made a visual guide to
each change which was quite widely circulated. IIRC it was on main list,
cafe, Planet Haskell, reddit, circulated on Twitter, posted in #haskell…
My nginx says the post demonstrating the changes had ~3000 hits. I don't
know what more I can do to make people aware of the releases.

> To test this out, I put together
> a simple demonstration of a Hamlet synopsis in Markdown[1], and then
> converted it via Pandoc to Haddock[2]. I copied that Haddock content into a
> cabal file's description, then indented it, ran cabal haddock, and got:
> 
>     cabal: hamlet.cabal:30: unexpected span: "!"

Cabal problem, no idea why it happens however, I have not looked into it.

> For some reason, the trailing exclamation point is rejected by cabal. I
> tried removing the exclamation point, and it *did* generate content.
> However, all of the content was on the same line, because I needed to add
> periods between each block to deal with cabal's description parsing. 

Also Cabal problem but you raise an important issue. Cabal's synopsis
parsing sucks so if it could instead read from a separate file without
these weird limitations, that'd be a big improvement in itself and a lot
easier to automated if one wants to.

> After
> adding those periods, things *mostly* worked, except that my first code
> sample came out as:
> 
>    <p>Hi, my name is #
> 
> Notice how the {name} portion has been stripped away. So I see a number of
> ways that this current situation for writing synopses is inferior:

Maybe this is to do with Haddock issue #308 which was fixed in June and
made it into 7.8.3 release. In fact, here[2] is the AST the parser
(found in ‘haddock-library’ package which you can trivially install and
try yourself) produces. This means I have to ask for your Haddock
version. If you're on 7.8.2 or 7.8.3 you should be able to upgrade to
2.15.0 (that's right, we had another release since) simply by cabal
install haddock which should demonstrate the correct behaviour. I'm
surprised that you have a version without this fix unless you're stuck
on 7.6.3 or something.

> 1. Far more people are comfortable writing Markdown than writing Haddock,
> even those of us who write a lot of Haddock. Tooling support for editing
> Markdown is also superior.

As usual, I point out that there are tens of different Markdown
flavours. Of course today I will be pointed to the recent common
Markdown creation but as far as I'm concerned, we had 14 standards and
this is the 15th until it gets established. More importantly, Markdown
is not suited for documenting code. We'd have to introduce new
identifiers if we wanted to link to some functions from our synopsis.
16th standard aside, this is quickly starts looking like Haddock.

I'm pretty sure that if you only used Haddock features that Markdown
also has, you could tell someone your Haddock is a flavour of Markdown
and they'd believe you. In any case, I'm not here to wage war against
Markdown people but to find out why your Haddock experience is bad so
let's move on.

Bah, regarding tooling, with the parser now being in its own small
package it's trivial to get out the AST for any snippet of text. If
someone is eager then they can write emacs modes or whatever people use
pretty easily now.

> 2. Having the synopsis in a separate file means that it's easy to give a
> link to the file on Github and get readable docs, which is *not* the case
> with linking to a cabal file.

I am not against this at all, I would quite like to see a separate file,
it would make for much easier editing and you don't have to post-process
anything spit out by tools to conform to additional cabal gotchas.

> 3. I don't have to go through any preprocess step to convert from Markdown
> to Haddock if that's the way I want to edit.

Seems like something cabal could handle for you if it sees pandoc on
your system.

> 4. Putting the synopsis in the cabal file means we have to deal with cabal
> rules *in addition to* Haddock rules, such as the periods between blocks
> and not having an exclamation point at the end of the line[3].

Solved by your point 2.

> 5. On top of all that, Haddock still can't handle the documentation I want
> to write for Hamlet, to demonstrate #{variable} interpolation.

As the AST at [2] demonstrates, it can. Bah, just for you I used your
pandoc-generated Haddock, stuck it in a module and produced the HTML
file at [3]. It looks fine to me. We probably want to teach pandoc that
it can put the anchors on the same line as the headers and that it can
put content on the very next line from the header which would remove
excessive spacing but other than that, it looks fine to me. Notably your
#{name} problem is not present. [4] houses the Haskell file used to
generate the HTML. Generated with Haddock HEAD but it only is 4 commits
ahead of 2.15.0 release with the only changes being some documentation
stuff.

> So again, I really like the idea of doing the same thing for synopses as we
> have already for changelogs.

I also do, I am here merely to investigate why Markdown is so strongly
preferred when Haddock does the job. Yes, the escaping was really bad in
the past. Yes, it had some weird bugs. Yes, it has been fixed. To my
knowledge, we have zero bugs open on the issue tracker[5]. There are
some that are to do with GHC lexer + parser but those are not relevant
at all here. If you know of bugs then please report them.

> Michael
> 
> [1] http://lpaste.net/111103
> [2] http://lpaste.net/111104
> [3] For the record, I was completely unaware of this limitation, and
> stumbled upon it by accident when putting together this example.
> 
> 
>> I think just about the only thing nowadays that might surprise people is
>> that tokens stretch over newlines but this is by design, especially
>> considering one can embed code.
>>
>>> I would much rather be able tom
>>> write a synopsis in markdown, or asciidoc, or HTML, rather than haddock
>>> markup.
>>>
>>
>> As I mention on the other e-mail, you can now use pandoc to achieve
>> this. If people really care then it should not be difficult to
>> co-ordinate with cabal + Hackage and make it parse your favourite format
>> and splice it into synopsis.
>>
>> I'd quite like the complaints about the existing Haddock markup, the
>> rules are pretty damn close to what Markdown uses.
>>
>> --
>> Mateusz K.
>> _______________________________________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe at haskell.org
>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>>
> 

[1]: https://github.com/haskell/haddock/blob/master/CHANGES#L55
[2]: http://lpaste.net/111124
[3]: http://fuuzetsu.co.uk/misc/md/MarkdownToHaddock.html
[4]: http://lpaste.net/111125
[5]: https://github.com/haskell/haddock/issues

-- 
Mateusz K.