Documenting GHC: blogs, wiki pages, Notes, Haddocks, etc
Simon Peyton Jones
simonpj at microsoft.com
Tue Sep 14 16:54:27 UTC 2021
And as far as linking is concerned: Sure, haddocks don't have a title to refer to. But you can always link to them by linking to the syntactic entity! For example, if I want to link to DiagnosticReason from Severity, I can simply do so by saying "Also see 'Severity'".
I don’t like this at all! The most important and interesting Notes are about the relationships between multiple distinct entities. I don’t want to say “See Severity” because
* It’s not about Severity; it’s about the relationship between DiagnosticReason and Severity
* There might be a lot else about Severity that I want to say with Severity, that is nothing to do with DiagnosticReason. I want to refer specifically to the note about the relationship.
Also, many, many Notes are referred to from some part of the source code of a function body (e.g. a guard, accompanied with “See Note [Special case for local ids]”). Here there is no entity to act as the anchor. But the Note still might have many pointers to it.
Ideally, Haddock would simply recognise the Note syntax directly or provide a similar alternative
Yes, exactly. Notes have evolved into such a compellingly useful part of GHC that I really think they deserve Haddock support.
But here’s another tension: the Haddock for a function should be about the user interface; its specification. But many Notes are specifically about the details of an implementation.
I have more questions than answers. But even though Notes are incredibly low tech, I read and write them constantly. I think they are a surprisingly powerful, albeit extremely modest, idea that GHC has given to the world.
Simon
PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com<mailto:simonpj at microsoft.com> will cease to work. Use simon.peytonjones at gmail.com<mailto:simon.peytonjones at gmail.com> instead. (For now, it just forwards to simonpj at microsoft.com.)
From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Sebastian Graf
Sent: 14 September 2021 15:00
To: Hécate <hecate at glitchbra.in>
Cc: ghc-devs <ghc-devs at haskell.org>
Subject: Re: Documenting GHC: blogs, wiki pages, Notes, Haddocks, etc
Hi,
I've been using Haddock's named chunks feature too when writing the docs for selective lambda lifting.
This is the result: https://hackage.haskell.org/package/ghc-8.10.2/docs/StgLiftLams.html<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhackage.haskell.org%2Fpackage%2Fghc-8.10.2%2Fdocs%2FStgLiftLams.html&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247560036%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=sDKPBAiSSJYWjQysWkod%2BB1ewyzv4ubiwt2%2BfxEOjdI%3D&reserved=0>, and this is how the source code looks: https://hackage.haskell.org/package/ghc-8.10.2/docs/src/StgLiftLams.html<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhackage.haskell.org%2Fpackage%2Fghc-8.10.2%2Fdocs%2Fsrc%2FStgLiftLams.html&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247570032%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=GpJji7mVYmEi3KLJeCiRFDwQRfgMXN1boZmZuynykVI%3D&reserved=0>
I quite like it. As you can see, I enabled both the existing Notes workflow and Haddock to work with it. It takes a bit of annoying extra work, though. Ideally, Haddock would simply recognise the Note syntax directly or provide a similar alternative.
And as far as linking is concerned: Sure, haddocks don't have a title to refer to. But you can always link to them by linking to the syntactic entity! For example, if I want to link to DiagnosticReason from Severity, I can simply do so by saying "Also see 'Severity'".
I do admit this might not be enough info at the reference site to determine whether the haddock linked to is relevant to the particular goal I want to achieve. Also as Simon points out, there are Notes that don't have a clear "owner".
Heck, even writing an unused binding `_Late_lambda_lifting_in_STG` and put the haddocks there would work, I suppose. We could simply link to it with '_Late_lambda_lifting_in_STG' from other haddocks.
My point is: If we managed to have something quite like named chunks, but with a title and one place it gets rendered and then linked to (I don't like that named chunks are inlined into every use site), we could probably agree on using that.
Also I'd like to see the Notes rendered *regardless* of whether the thing it is attached to is exported. That would make Notes a lot more accessible.
Sebastian
Am Di., 14. Sept. 2021 um 14:32 Uhr schrieb Hécate <hecate at glitchbra.in<mailto:hecate at glitchbra.in>>:
> today’s Haddock doesn’t understand Notes. But we could fix that if we were minded to.
I may have missed an episode or two here but what prevents us from writing Notes as Named Chunks¹, write them where Haddock expects you to put documentation, and refer to them from the relevant spot in the code?
Viktor (in CC) has done a wonderful work at producing nice layouts for Haddocks in base, and we could learn a couple of lessons from his MRs.
---
Now, on the matter of improving Haddock to understand GHC's notes, I'd like to remind everyone that Haddock is currently understaffed in terms of feature development, and I would like to call to everyone with experience dealing with its codebase to give a hand in refactoring, dusting off and improving the code so that its maintainability is not jeopardised by people simply going elsewhere.
Our bus factor (or as I like to call it, circus factor), is quite terrifying considering the importance of the tool in our ecosystem.
¹ https://haskell-haddock.readthedocs.io/en/latest/markup.html#named-chunks<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhaskell-haddock.readthedocs.io%2Fen%2Flatest%2Fmarkup.html%23named-chunks&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247580027%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=34wGCjxKe4A0u9hiIHHG82PN82gxV1MM%2B7w2Kl2RFT0%3D&reserved=0>
Le 14/09/2021 à 13:56, Simon Peyton Jones via ghc-devs a écrit :
Alfredo writes (below for full thread)
That is a deceptively simple question you ask there :-) I don't have a strong view myself, but I can offer the perspective of somebody who was been for a long time on the "other side of the trenches" (i.e. working Haskell programmer, not necessarily working GHC programmer):
* Blog post: yes, it's true that is a snapshot, and it's true that is not under GHC's gitlab umbrella, so I wouldn't treat it as a reliable source of documentation (for the reasons you also explain) but it's surely a good testament that "at this point in time, for this particular GHC commit, things were this way);
* The wiki page: in the past, when I wanted to learn more about some GHC feature, Google would point me to the relevant Wiki page on the GHC repo describing such a feature, but I have to say I have almost always dismissed it, because everybody knows Wikis are seldomly up-to-date :) In order for a Wiki page to work we would have to at least add a banner at the top that states this can be trusted as a reliable source of information, and offer in the main section the current, up-to-date design. We can still offer the historical breakdown of the designs in later sections, as it's still valuable info to keep;
* GHC notes: I have always considered GHC notes a double-edge sword -- from one side they are immensely useful when navigating the source code, but these won't be rendered in the Hackage's haddocks, and this is not helpful for GHC-the-library users willing to understand how to use (or which is the semantic of) a particular type (sure, one can click "Show Source" on Hackage but it's an annoying extra step to do just to hunt for notes). We already have Notes for this work in strategic places -- even better, we have proper Haddock comments for things like "Severity vs DiagnosticReason" , e.g. https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247590020%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wnMtmaZ3aQKLqKgsEXBuq3RiOt5%2FkYcpwrzmNb6Asdw%3D&reserved=0> .
Yes Haddock doesn’t understand Notes but that’s a deficiency in Haddock! There so much in GHC that simply does not fit well with the Haddocks attached to a particular data decl or function. We need Notes to explain how all the moving parts fit together, and to point to them.
Even better, we have proper Haddock comments for things like "Severity vs DiagnosticReason"
But I don’t think this is better – I think it is significantly worse! In the case you cite, the Haddock is about DiagnosticReason, and mentions Severity only incidentally. I bet that the Haddock for Severity doesn’t refer to this. Nor is there a clear “Note [Severity vs DiagnosticReason]” title that bits of code across GHC can refer to by saying “See Note [Severity vs DiagnosticReason]”. It’s far less satisfactory (to me) than a single Note that
· covers just one topic (the difference between Severity and DiagnosticReason, rather than fully describing either
· can be pointed to symmetrically from both Severity and DiagnosticReason
· can be pointed to by many other bits of code
The way it is better is that today’s Haddock doesn’t understand Notes. But we could fix that if we were minded to.
Returning to how to document the error-message architecture, if you’d prefer to use a Note than a wiki page, that’s fine. But please write that Overview Note that explains all the pieces, points to them one by one. And then copiously refer to that Note from all those places, so people will update it.
Hopefully as the time goes by the new design will "spread" across all the different peers working on GHC, and it will become "second nature".
I really don’t think that will happen unless there is a Note that explains what the new design is! Lacking this explicit design, everyone will infer their own mental model of how it all works from sundry scattered clues – and those mental models will differ. So instead of one thing “spreading” a dozen subtly different things will spread. And then the next one, confused by these slightly different clues, will be even less coherent.
Let’s have one, fully-explicit version of The Plan that we constantly refer to.
cc’ing ghc-devs because we must constantly question and refine how we describe and document GHC.
Simon
PS: I am leaving Microsoft at the end of November 2021, at which point simonpj at microsoft.com<mailto:simonpj at microsoft.com> will cease to work. Use simon.peytonjones at gmail.com<mailto:simon.peytonjones at gmail.com> instead. (For now, it just forwards to simonpj at microsoft.com<mailto:simonpj at microsoft.com>.)
From: Alfredo Di Napoli <alfredo.dinapoli at gmail.com><mailto:alfredo.dinapoli at gmail.com>
Sent: 26 August 2021 07:25
To: Simon Peyton Jones <simonpj at microsoft.com><mailto:simonpj at microsoft.com>
Cc: rae at richarde.dev<mailto:rae at richarde.dev>
Subject: Re: [Haskell Community] [Links] [Well-Typed Blog] The new GHC diagnostic infrastructure
Hello Simon!
On Wed, 25 Aug 2021 at 13:36, Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>> wrote:
Alfredo
Thanks for all the work you are doing on GHC’s error message infrastructure. Your blog post gives a great overview.
Thanks, and I am glad you enjoyed it :)
As you know I’m very keen for GHC to have a Note or wiki page that gives a solid, up-to-date overview of all the moving parts. (NOT the design alternatives, nor the time sequence; just the outcome.) This is incredibly useful for our future selves; and it helps ensure that people understand (say) the difference between Severity and DiagnosticReason, and use them correctly.
So the question is: where is the canonical overview? It could be
* Your blog post below. But that is a snapshot… you aren’t going to go back to edit it as the design evolves. And it’s not in the repo.
* The wiki page: https://gitlab.haskell.org/ghc/ghc/-/wikis/Errors-as-(structured)-values<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fwikis%2FErrors-as-(structured)-values&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247600018%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Fms1dGIEYyuGHJvtI9vBO93zk53PJe8gc7usPXCDboM%3D&reserved=0>. But it’s hard to keep up to date (it was last edited 3 months ago).
* Note(s) in the code. We seem to use this increasingly, and it has the great merit of being part of the source code itself. But then we need clear pointer to the canonical overview Notes, and need to make sure they are up to date.
I’m not advocating any particular path here… just wanting to be sure that we end up with a good overview somewhere! What is your view?
TL;DR Probably a combo of a well-written (and up-to-date Wiki) plus some carefully added Notes (and Haddock comments) in GHC might do the trick.
That is a deceptively simple question you ask there :-) I don't have a strong view myself, but I can offer the perspective of somebody who was been for a long time on the "other side of the trenches" (i.e. working Haskell programmer, not necessarily working GHC programmer):
* Blog post: yes, it's true that is a snapshot, and it's true that is not under GHC's gitlab umbrella, so I wouldn't treat it as a reliable source of documentation (for the reasons you also explain) but it's surely a good testament that "at this point in time, for this particular GHC commit, things were this way);
* The wiki page: in the past, when I wanted to learn more about some GHC feature, Google would point me to the relevant Wiki page on the GHC repo describing such a feature, but I have to say I have almost always dismissed it, because everybody knows Wikis are seldomly up-to-date :) In order for a Wiki page to work we would have to at least add a banner at the top that states this can be trusted as a reliable source of information, and offer in the main section the current, up-to-date design. We can still offer the historical breakdown of the designs in later sections, as it's still valuable info to keep;
* GHC notes: I have always considered GHC notes a double-edge sword -- from one side they are immensely useful when navigating the source code, but these won't be rendered in the Hackage's haddocks, and this is not helpful for GHC-the-library users willing to understand how to use (or which is the semantic of) a particular type (sure, one can click "Show Source" on Hackage but it's an annoying extra step to do just to hunt for notes). We already have Notes for this work in strategic places -- even better, we have proper Haddock comments for things like "Severity vs DiagnosticReason" , e.g. https://gitlab.haskell.org/ghc/ghc/-/blob/master/compiler/GHC/Types/Error.hs#L279<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.haskell.org%2Fghc%2Fghc%2F-%2Fblob%2Fmaster%2Fcompiler%2FGHC%2FTypes%2FError.hs%23L279&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247610014%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=4AVRsMc0ZbG7%2Bd4TasEQ2Oz9kcjnwqDcjq1vSoaRaVo%3D&reserved=0> .
So, in practical terms, I suggest we (I) give the Wiki a little overhaul to add at the top the current design (or anything not captured directly in GHC's source code) and I will keep an eye on the GHC notes and Haddock comments to see if there is anything worth adding. Hopefully as the time goes by the new design will "spread" across all the different peers working on GHC, and it will become "second nature".
Hope it helps, and sorry for the long ramble!
Alfredo
Thanks
Simon
From: Alfredo Di Napoli via Haskell Community <discourse at haskell.org<mailto:discourse at haskell.org>>
Sent: 23 August 2021 11:26
To: Simon Peyton Jones <simonpj at microsoft.com<mailto:simonpj at microsoft.com>>
Subject: [Haskell Community] [Links] [Well-Typed Blog] The new GHC diagnostic infrastructure
[Image removed by sender.]
adinapoli<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Fu%2Fadinapoli&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247610014%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=JrCzoU83GcVYoYKHRSpjOO7U%2FpYsMlsYySVym2U8kRo%3D&reserved=0>
August 23
[Image removed by sender.]well-typed.com<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247620006%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=LqnTNbl049Et3ePE1B5j4ZkTPBF%2BJRHs5JJO7yTLU7o%3D&reserved=0>
Error! Filename not specified.
The new GHC diagnostic infrastructure - Well-Typed: The Haskell Consultants<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwell-typed.com%2Fblog%2F2021%2F08%2Fthe-new-ghc-diagnostic-infrastructure%2F&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247630000%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=duBX20BWM0JoqCtXUIR0oWYibSVn7g1J1yg4QqPWpVs%3D&reserved=0>
________________________________
Visit Topic<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Ft%2Fwell-typed-blog-the-new-ghc-diagnostic-infrastructure%2F2918%2F1&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247639995%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=JJlsMnZ2qhoIg3G0iwcJNNDJYzbhNH6KH0vvOul8yXE%3D&reserved=0> or reply to this email to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, click here<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdiscourse.haskell.org%2Femail%2Funsubscribe%2F962dfad7651b2ce3d7e30ba9267bdb857c77298d6fdec12626b65e014aaeee33&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247649991%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=MjpvoWdGcZFyBEW4jyzGByq47OFLWIq7622XnT8TWm8%3D&reserved=0>.
_______________________________________________
ghc-devs mailing list
ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247649991%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=PQ2QQLTfuy%2FF3USu3mpdmqczBuShQVHj6pph7BDdtJI%3D&reserved=0>
--
Hécate ✨
🐦: @TechnoEmpress
IRC: Hecate
WWW: https://glitchbra.in<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fglitchbra.in%2F&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247659987%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=hoRmDtamUR9fRowlNR1USylyLViCNBALD%2Fl75351bXA%3D&reserved=0>
RUN: BSD
_______________________________________________
ghc-devs mailing list
ghc-devs at haskell.org<mailto:ghc-devs at haskell.org>
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs&data=04%7C01%7Csimonpj%40microsoft.com%7C1bd0b32d10d941aff47008d977883fda%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637672251247669986%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=LRZo55%2BDrGc76Ku%2FTnZ7C1cL5mfiw%2Bvg0KS0Sn7FSu0%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/842c8d9b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 823 bytes
Desc: image001.jpg
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20210914/842c8d9b/attachment-0001.jpg>
More information about the ghc-devs
mailing list