Documenting GHC: blogs, wiki pages, Notes, Haddocks, etc

Sebastian Graf sgraf1337 at gmail.com
Tue Sep 14 14:37:16 UTC 2021 

What I don't like:

   - I have to give a section name to an otherwise unused $chunk
   - I can't refer to $chunk from other modules
   - Everywhere I reference $chunk in a haddock, it gets inlined instead of
   appearing with its title or inserting a link with the section name it is
   currently decoupled from.


Am Di., 14. Sept. 2021 um 16:25 Uhr schrieb Hécate <hecate at glitchbra.in>:

> Hi,
>
> The named chunks can be positioned through the use of the export list
> syntax:
>
> module Foo
>   ( main
>   -- * Section name that will appear
>   --
>   -- $chunk
>   )
>
> This should produce a free section that is not linked to any exported item.
> I see you're already using them though, so maybe I am understanding
> something else?
> Le 14/09/2021 à 16:00, Sebastian Graf a écrit :
>
> Hi,
>
> I've been using Haddock's named chunks feature too when writing the docs
> for selective lambda lifting.
> This is the result:
> https://hackage.haskell.org/package/ghc-8.10.2/docs/StgLiftLams.html, and
> this is how the source code looks:
> https://hackage.haskell.org/package/ghc-8.10.2/docs/src/StgLiftLams.html
>
> I quite like it. As you can see, I enabled both the existing Notes
> workflow and Haddock to work with it. It takes a bit of annoying extra
> work, though. Ideally, Haddock would simply recognise the Note syntax
> directly or provide a similar alternative.
>
> And as far as linking is concerned: Sure, haddocks don't have a title to
> refer to. But you can always link to them by linking to the syntactic
> entity! For example, if I want to link to DiagnosticReason from Severity, I
> can simply do so by saying "Also see 'Severity'".
> I do admit this might not be enough info at the reference site to
> determine whether the haddock linked to is relevant to the particular goal
> I want to achieve. Also as Simon points out, there are Notes that don't
> have a clear "owner".
>
> Heck, even writing an unused binding `_Late_lambda_lifting_in_STG` and put
> the haddocks there would work, I suppose. We could simply link to it with
> '_Late_lambda_lifting_in_STG' from other haddocks.
>
> My point is: If we managed to have something quite like named chunks, but
> with a title and one place it gets rendered and then linked to (I don't
> like that named chunks are inlined into every use site), we could probably
> agree on using that.
> Also I'd like to see the Notes rendered *regardless* of whether the thing
> it is attached to is exported. That would make Notes a lot more accessible.
>
> Sebastian
>
>
> Am Di., 14. Sept. 2021 um 14:32 Uhr schrieb Hécate <hecate at glitchbra.in>:
>
>> > today’s Haddock doesn’t understand Notes.  But we could fix that if we
>> were minded to.
>>
>> I may have missed an episode or two here but what prevents us from
>> writing Notes as Named Chunks¹, write them where Haddock expects you to put
>> documentation, and refer to them from the relevant spot in the code?
>> Viktor (in CC) has done a wonderful work at producing nice layouts for
>> Haddocks in base, and we could learn a couple of lessons from his MRs.
>>
>> ---
>>
>> Now, on the matter of improving Haddock to understand GHC's notes, I'd
>> like to remind everyone that Haddock is currently understaffed in terms of
>> feature development, and I would like to call to everyone with experience
>> dealing with its codebase to give a hand in refactoring, dusting off and
>> improving the code so that its maintainability is not jeopardised by people
>> simply going elsewhere.
>> Our bus factor (or as I like to call it, circus factor), is quite
>> terrifying considering the importance of the tool in our ecosystem.
>>
>>
>> ¹
>> https://haskell-haddock.readthedocs.io/en/latest/markup.html#named-chunks
>>
>> Le 14/09/2021 à 13:56, Simon Peyton Jones via ghc-devs a écrit :
>>
>> Alfredo writes (below for full thread)
>>
>>
>>
>> That is a deceptively simple question you ask there :-) I don't have a
>> strong view myself, but I can offer the perspective of somebody who was
>> been for a long time on the "other side of the trenches" (i.e. working
>> Haskell programmer, not necessarily working GHC programmer):
>>
>>
>>
>> * Blog post: yes, it's true that is a snapshot, and it's true that is not
>> under GHC's gitlab umbrella, so I wouldn't treat it as a reliable source of
>> documentation (for the reasons you also explain) but it's surely a good
>> testament that "at this point in time, for this particular GHC commit,
>> things were this way);
>>
>>
>>
>> * The wiki page: in the past, when I wanted to learn more about some GHC
>> feature, Google would point me to the relevant Wiki page on the GHC repo
>> describing such a feature, but I have to say I have almost always dismissed
>> it, because everybody knows Wikis are seldomly up-to-date :) In order for a
>> Wiki page to work we would have to at least add a banner at the top that
>> states this can be trusted as a reliable source of information, and offer
>> in the main section the current, up-to-date design. We can still offer the
>> historical breakdown of the designs in later sections, as it's still
>> valuable info to keep;
>>
>>
>>
>> * GHC notes: I have always considered GHC notes a double-edge sword --
>> from one side they are immensely useful when navigating the source code,
>> but these won't be rendered in the Hackage's haddocks, and this is not
>> helpful for GHC-the-library users willing to understand how to use (or
>> which is the semantic of) a particular type (sure, one can click "Show
>> Source" on Hackage but it's an annoying extra step to do just to hunt for
>> notes). We already have Notes for this work in strategic places -- even
>> better, we have proper Haddock comments for things like "Severity vs
>> DiagnosticReason" , e.g.
>> https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255320972%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WU2dKu2Q%2FFdwntJ2h%2F6zO1Ic01c9o0VhZc5JrE0AurY%3D&reserved=0>
>> .
>>
>>
>>
>>
>>
>> Yes Haddock doesn’t understand Notes but that’s a deficiency in Haddock!
>> There so much in GHC that simply does not fit well with the Haddocks
>> attached to a particular data decl or function.  We need Notes to explain
>> how all the moving parts fit together, and to point to them.
>>
>>
>>
>> Even better, we have proper Haddock comments for things like "Severity vs
>> DiagnosticReason"
>>
>>
>>
>> But I don’t think this is better – I think it is significantly worse!
>> In the case you cite, the Haddock is about DiagnosticReason, and mentions
>> Severity only incidentally.  I bet that the Haddock for Severity doesn’t
>> refer to this.   Nor is there a clear “Note [Severity vs DiagnosticReason]”
>> title that bits of code across GHC can refer to by saying “See Note
>> [Severity vs DiagnosticReason]”.   It’s far less satisfactory (to me) than
>> a single Note that
>>
>>    - covers just *one topic* (the difference between Severity and
>>    DiagnosticReason, rather than fully describing either
>>    - can be *pointed to* symmetrically from both Severity and
>>    DiagnosticReason
>>    - can be *pointed to* by many other bits of code
>>
>>
>>
>> The way it is better is that today’s Haddock doesn’t understand Notes.
>> But we could fix that if we were minded to.
>>
>>
>> Returning to how to document the error-message architecture, if you’d
>> prefer to use a Note than a wiki page, that’s fine.  But please write that
>> Overview Note that explains all the pieces, points to them one by one.  And
>> then copiously refer to that Note from all those places, so people will
>> update it.
>>
>>
>>
>> *Hopefully as the time goes by the new design will "spread" across all
>> the different peers working on GHC, and it will become "second nature".*
>>
>>
>>
>> I really don’t think that will happen unless there is a Note that
>> explains what the new design is!  Lacking this explicit design, everyone
>> will infer their own mental model of how it all works from sundry scattered
>> clues – and those mental models will differ.   So instead of one thing
>> “spreading”  a dozen subtly different things will spread.  And then the
>> next one, confused by these slightly different clues, will be even less
>> coherent.
>>
>>
>>
>> Let’s have one, fully-explicit version of The Plan that we constantly
>> refer to.
>>
>>
>>
>> cc’ing ghc-devs because we must constantly question and refine how we
>> describe and document GHC.
>>
>>
>>
>> Simon
>>
>>
>>
>> PS: I am leaving Microsoft at the end of November 2021, at which point
>> simonpj at microsoft.com will cease to work.  Use
>> simon.peytonjones at gmail.com instead.  (For now, it just forwards to
>> simonpj at microsoft.com.)
>>
>>
>>
>> *From:* Alfredo Di Napoli <alfredo.dinapoli at gmail.com>
>> <alfredo.dinapoli at gmail.com>
>> *Sent:* 26 August 2021 07:25
>> *To:* Simon Peyton Jones <simonpj at microsoft.com> <simonpj at microsoft.com>
>> *Cc:* rae at richarde.dev
>> *Subject:* Re: [Haskell Community] [Links] [Well-Typed Blog] The new GHC
>> diagnostic infrastructure
>>
>>
>>
>> Hello Simon!
>>
>>
>>
>> On Wed, 25 Aug 2021 at 13:36, Simon Peyton Jones <simonpj at microsoft.com>
>> wrote:
>>
>> Alfredo
>>
>>
>>
>> Thanks for all the work you are doing on GHC’s error message
>> infrastructure.  Your blog post gives a great overview.
>>
>>
>>
>> Thanks, and I am glad you enjoyed it :)
>>
>>
>>
>>
>>
>> As you know I’m very keen for GHC to have a Note or wiki page that gives
>> a solid, up-to-date overview of all the moving parts.  (NOT the design
>> alternatives, nor the time sequence; just the outcome.)  This is incredibly
>> useful for our future selves; and it helps ensure that people understand
>> (say) the difference between Severity and DiagnosticReason, and use them
>> correctly.
>>
>>
>>
>> So the question is: where is the canonical overview?  It could be
>>
>>    - Your blog post below. But that is a snapshot… you aren’t going to
>>    go back to edit it as the design evolves.  And it’s not in the repo.
>>    - The wiki page:
>>    https://gitlab.haskell.org/ghc/ghc/-/wikis/Errors-as-(structured)-values
>>    <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fwikis%2FErrors-as-(structured)-values&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255310976%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=A%2FyWqfqPWPYUk3EpaorYP29JvLIhgcdSdcYceFIKvhc%3D&reserved=0>.
>>    But it’s hard to keep up to date (it was last edited 3 months ago).
>>    - Note(s) in the code.  We seem to use this increasingly, and it has
>>    the great merit of being part of the source code itself.  But then we need
>>    clear pointer to the canonical overview Notes, and need to make sure they
>>    are up to date.
>>
>>
>>
>> I’m not advocating any particular path here… just wanting to be sure that
>> we end up with a good overview somewhere! What is your view?
>>
>>
>>
>>
>>
>> *TL;DR Probably a combo of a well-written (and up-to-date Wiki) plus some
>> carefully added Notes (and Haddock comments) in GHC might do the trick.*
>>
>>
>>
>> That is a deceptively simple question you ask there :-) I don't have a
>> strong view myself, but I can offer the perspective of somebody who was
>> been for a long time on the "other side of the trenches" (i.e. working
>> Haskell programmer, not necessarily working GHC programmer):
>>
>>
>>
>> * Blog post: yes, it's true that is a snapshot, and it's true that is not
>> under GHC's gitlab umbrella, so I wouldn't treat it as a reliable source of
>> documentation (for the reasons you also explain) but it's surely a good
>> testament that "at this point in time, for this particular GHC commit,
>> things were this way);
>>
>>
>>
>> * The wiki page: in the past, when I wanted to learn more about some GHC
>> feature, Google would point me to the relevant Wiki page on the GHC repo
>> describing such a feature, but I have to say I have almost always dismissed
>> it, because everybody knows Wikis are seldomly up-to-date :) In order for a
>> Wiki page to work we would have to at least add a banner at the top that
>> states this can be trusted as a reliable source of information, and offer
>> in the main section the current, up-to-date design. We can still offer the
>> historical breakdown of the designs in later sections, as it's still
>> valuable info to keep;
>>
>>
>>
>> * GHC notes: I have always considered GHC notes a double-edge sword --
>> from one side they are immensely useful when navigating the source code,
>> but these won't be rendered in the Hackage's haddocks, and this is not
>> helpful for GHC-the-library users willing to understand how to use (or
>> which is the semantic of) a particular type (sure, one can click "Show
>> Source" on Hackage but it's an annoying extra step to do just to hunt for
>> notes). We already have Notes for this work in strategic places -- even
>> better, we have proper Haddock comments for things like "Severity vs
>> DiagnosticReason" , e.g.
>> https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255320972%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WU2dKu2Q%2FFdwntJ2h%2F6zO1Ic01c9o0VhZc5JrE0AurY%3D&reserved=0>
>> .
>>
>>
>>
>> *So, in practical terms, I suggest we (I) give the Wiki a little overhaul
>> to add at the top the current design (or anything not captured directly in
>> GHC's source code) and I will keep an eye on the GHC notes and Haddock
>> comments to see if there is anything worth adding. Hopefully as the time
>> goes by the new design will "spread" across all the different peers working
>> on GHC, and it will become "second nature".*
>>
>>
>>
>> Hope it helps, and sorry for the long ramble!
>>
>>
>>
>> Alfredo
>>
>>
>>
>>
>>
>> Thanks
>>
>>
>>
>> Simon
>>
>>
>>
>> *From:* Alfredo Di Napoli via Haskell Community <discourse at haskell.org>
>> *Sent:* 23 August 2021 11:26
>> *To:* Simon Peyton Jones <simonpj at microsoft.com>
>> *Subject:* [Haskell Community] [Links] [Well-Typed Blog] The new GHC
>> diagnostic infrastructure
>>
>>
>>
>> [image: Image removed by sender.]
>>
>> *adinapoli*
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Fu%2Fadinapoli&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255330973%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PgX4crGMBTWVwI2UMcq%2BIFDZ0dDr%2FRWNYZdV%2Fqi8mX8%3D&reserved=0>
>> August 23
>>
>> [image: Image removed by sender.]*well-typed.com*
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255340965%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X51rdrGoKmUBPB8upLVL69LyInf%2BsYYQqM%2Fd4PnLnGQ%3D&reserved=0>
>>
>> *Error! Filename not specified.*
>> The new GHC diagnostic infrastructure - Well-Typed: The Haskell
>> Consultants
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255340965%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X51rdrGoKmUBPB8upLVL69LyInf%2BsYYQqM%2Fd4PnLnGQ%3D&reserved=0>
>> ------------------------------
>>
>> *Visit Topic*
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Ft%2Fwell-typed-blog-the-new-ghc-diagnostic-infrastructure%2F2918%2F1&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255350960%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tOfAGO5BbhanwBDgMA6eqpgKCLcLTtkum8QOuMsROdc%3D&reserved=0>
>> or reply to this email to respond.
>>
>> You are receiving this because you enabled mailing list mode.
>>
>> To unsubscribe from these emails, *click here*
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Femail%2Funsubscribe%2F962dfad7651b2ce3d7e30ba9267bdb857c77298d6fdec12626b65e014aaeee33&data=04%7C01%7Csimonpj%40microsoft.com%7Cdb46814133bc4404b6d308d9685a487e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637655559255360954%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4616hEpSSUcOZ5zQYZMmEbF6mTJcIVKx2nlgA8ENsHM%3D&reserved=0>
>> .
>>
>>
>>
>>
>> _______________________________________________
>> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>> --
>> Hécate ✨
>> 🐦: @TechnoEmpress
>> IRC: Hecate
>> WWW: https://glitchbra.in
>> RUN: BSD
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
> --
> Hécate ✨
> 🐦: @TechnoEmpress
> IRC: Hecate
> WWW: https://glitchbra.in
> RUN: BSD
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/664b5c3b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 823 bytes
Desc: not available
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/664b5c3b/attachment-0001.jpg>

More information about the ghc-devs mailing list