[Haskell-cafe] On Markdown in Haddock and why it's not going to happen

Mon Sep 2 22:34:55 CEST 2013

On 02/09/13 19:43, John MacFarlane wrote:
> When the proposal was first being discussed, I suggested that instead of
> adding markdown support to haddock, one might enhance the existing
> haddock markup, making it more expressive, so that it could encode the same
> range of structural features as markdown.  If I'm not mistaken, currently
> haddock doesn't allow list items with multiple paragraphs or other block
> elements, or nested lists, or images, or blockquotes.  
Paragraph level elements are indeed not allowed in lists. The list
contents are currently (that is, on my working branch) allowed to be:
monospace (delimited by @), anchors, identifiers, module names,
pictures, URLs, bold, emphasis, HTML escape codes, regular strings. Note
that if a monospaced string is the only element of the list content, it
gets turned into a code block.

I'm not sure what you mean by block elements. Do you mean bird tracks
and code blocks?

There's a ticket about nested lists at
http://trac.haskell.org/haddock/ticket/27 . While I say that nested
paragraphs are not planned, I'm still playing with the thought. The
issue is having to balance adding new features with not breaking a
significant amount of existing documentation: we do not want to force
everyone to go through all their libraries and change their markup.

While I say on the ticket that it's hard to implement, I think it could
be done with indentation. I would have to experiment. If you would like
to see paragraph nesting, please make a ticket describing the behaviour!
If list nesting is all you would be after then the ticket already
exists. If you're looking for allowing more elements in lists, please
make a ticket.

> This makes it
> impossible to create a pandoc writer for haddock markup (we already have a
> haddock reader, soon to be released).
Yes, indeed you might remember that my original proposal was to take
care of this: improving Haddock to the point where a writer would be
possible and providing the writer. I even mentioned somewhere that my
proposal and the proposal I'm currently working on would eventually end
up with the same result, just in a different order of events.
> 
> With a pandoc writer for haddock, one could write longer documentation sections
> (e.g. tutorials) in markdown and convert them to haddock for inclusion in
> source code.  This should help people who don't like haddock markup or don't
> want to learn it.  With the reader, haddock comments can already be converted
> into other formats.
> 
> I haven't been following your work, but do I understand correctly
> that you've been making haddock markup more expressive and rational?
> Has this gotten to the point where a pandoc writer would be feasible?
I will enumerate some changes. Probably the most important part is the
new parser which makes new markup additions and changes far easier and
testable than working with two separate grammars fed into Alex and
Happy. I also added a fair amount of tests and set up LaTeX backend
testing (I still need to populate it with tests though). I hope to do
the same with Hoogle.

* Long missing bold was added, with double underscore as delimiter.
  "__foo bar baz__". You don't have to escape single underscores as you
would have to in the past: "__fo_o__" is fine. Even "__foo bar\__baz__"
is fine.
* Nested markup is now possible and allowed where sensible.
  "__/foo/__"
* Escaping of delimiters mid-markup is now possible.
  "/fo\/o/" will produce emphasised "fo\o". The old behaviour would
only emphasise "fo\". URLs, anchors and images follow the old rules
(terminate on first delimiter, even if escaped. I'll probably allow
escaping in URL and image syntax though.
* The parser should no longer fail on any strings.
  This means that a line like "This is / a line." will parse as a simple
string. In the past this would fail because there's no closing slash and
you'd be forced to escape it. If you happen to see the old ‘parsing
failed’ message with new release, please file as a major bug!
* Image ‘title’ text is now possible.
  "<<imagePath titleText>>"
* You no longer need a line break between lists of the same type.
  "* foo
   * bar" is fine.

   "* foo
    2. bar" is not fine if two lists are what you're after.
* Headers are now allowed in function documentation:
  "= @Monospaced Header 1@
   == __Bold Header 2__
   === /Emphasised Header 3/
   […]
   ====== Header 6"
   Header levers 3-6 all get rendered at the same size in LaTeX. Also
put it into the Hoogle back-end but of course, it's out of our hands to
decide how that will be rendered. I e-mailed Neil Mitchel and he said
that this should be no problem however. Note that you do NOT need a line
break between a header and a paragraph-level markup. That is things like
  "= Header
   > birdtracks can only be on their own paragraphs"
   are completely fine. I'm still thinking about this behaviour but I
don't see any real problems with it.

Currently implementing per-module enabled extension listing, enabled for
rendering with a Haddock pragma but I need to solve a testing problem
before I push it up. On the side I started to implement a Markdown
implementation that Niklas seems very eager to have, even if that means
a lot less features available when using it. Personally, I don't think
it will be nice to use at all seeing that some Markdown AND Hadock
features will be missing and some of the syntax has to be semantically
altered so it's more of a proof of concept. Hopefully a Pandoc writer
would solve this problem for us: considering how far along we are in the
GSoC timeline, I'd much rather keep adding functionality and fix bugs
than to add implementation and maintenance burden for a rather
incomplete Markdown implementation.

Please, do tell us what you need doing for the writer. Best way to do
this is to file a ticket on Haddock Trac
http://trac.haskell.org/haddock/ . If the ticket exists already, let me
know (here or on Trac) and I'll see whether I could prioritise it a bit
more.
> 
> John
> 
> +++ Mateusz Kowalczyk [Aug 30 13 02:30 ]:
>> Greetings café,
>>
>> Perhaps some saddening news for Markdown fans out there. As you might
>> remember, there was a fair amount of push for having Markdown as an
>> alternate syntax for Haddock.
>>
>> Unfortunately, this is probably not going to happen for reasons listed
>> on the post I just published at [1].
>>
>> This thread is meant to be for discussion about the post as many people,
>> myself included, felt that Markdown would be a nice thing to have.
>>
>> I feel like I covered the topic fairly well on the post but feel free to
>> give suggestions or ask questions.
>>
>> I would also like to remind you that if there's something that you'd
>> like to see in Haddock or something that you feel is broken, a good way
>> express this is to make a ticket on the Haddock Trac[2].
>>
>> I will close the relevant Markdown ticket on the Trac[3] in about 3
>> days, unless someone can come up with a reasonable solution that meets
>> the initial intent of this part of the project: a widely known markup
>> format that could be used as an alternate syntax for Haddock so that
>> it's possible to write the documentation without learning the vanilla
>> syntax itself.
>>
>> [1]:
>> http://fuuzetsu.co.uk/blog/posts/2013-08-30-why-Markdown-in-Haddock-can't-happen.html
>>
>> [2]: http://trac.haskell.org/haddock
>>
>> [3]: http://trac.haskell.org/haddock/ticket/244
>> -- 
>> Mateusz K.
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe at haskell.org
>> http://www.haskell.org/mailman/listinfo/haskell-cafe

-- 
Mateusz K.