Roadmap to compacting ModIface

Wed Mar 25 10:16:49 UTC 2020

Omer,

In particular the EPS contains the PackageIfaceTable which contains a
map of modules to ModIface for external packages. This can end up
containing a lot of ModIfaces in a big project.

In every `HomeModInfo` there is also a `ModIface` which can be
compacted in order to save GC traversal.

ModIface is also chosen because it's possible serialise/deserialise
and hence more likely to be easily compactable, ModDetails will
contain too much stuff which can't be compacted, just looking now the
`TypeEnv` could certainly not be compated.

Cheers,

Matt

On Wed, Mar 25, 2020 at 10:06 AM Ömer Sinan Ağacan <omeragacan at gmail.com> wrote:
>
> How is ModIface used by IDEs exactly? I'd expect IDEs to use ModDetails, not
> ModIface.
>
> They're basically two different representations of the same thing (a module
> interface), but ModIface is more focused on serialization and deseriazliation
> (the type is designed to make that easy) and ModDetails is what GHC is using to
> e.g. type check an imported module.
>
> For example, In batch mode we add ModDetails to the module graph, not ModIface,
> because that's what we use to compile downstream.
>
> (In one-shot mode GHC makes ModDetails for imported modules after reading the
> interfaces using IfaceToCore.typecheckIface)
>
> Ömer
>
> Matthew Pickering <matthewtpickering at gmail.com>, 25 Mar 2020 Çar,
> 00:31 tarihinde şunu yazdı:
> >
> > The things which can't be compacted are
> >
> > * Pinned objects
> > * Functions
> > * Mutable variables
> >
> > It is only a hypothesis at the moment that compacting a ModIface will
> > help GC times in an IDE, but in order to try it we have to implement
> > this roadmap..
> >
> > It is certainly true that the EPS can get very large for realistic
> > projects with hundreds of dependencies, and not traversing it during
> > GC could be a huge win.
> >
> > Cheers,
> >
> > Matt
> >
> > On Tue, Mar 24, 2020 at 9:27 PM Simon Peyton Jones
> > <simonpj at microsoft.com> wrote:
> > >
> > > Thanks for writing this down Matthew.
> > >
> > > But I look at #17097 and I am baffled.  Why is that the right list of tasks?  Why do we need FastStrings backed by an unpinned ByteArray? (And similarly for each other bullet.)  What will the API look like if this project is successful?  Why do we want ModIfaces in a compact region?  To reduce residency?  Do we have data showing that this is a real issue in practice.
> > >
> > > I feel as if a wiki page to explain the problem and articulate the proposed solution would make it easier for outsiders to contribute.
> > >
> > > Thanks
> > >
> > > Simon
> > >
> > > |  -----Original Message-----
> > > |  From: ghc-devs <ghc-devs-bounces at haskell.org> On Behalf Of Matthew
> > > |  Pickering
> > > |  Sent: 24 March 2020 10:58
> > > |  To: GHC developers <ghc-devs at haskell.org>
> > > |  Subject: Roadmap to compacting ModIface
> > > |
> > > |  Hello all,
> > > |
> > > |  I have written down the remaining steps which need to be taken in
> > > |  order to compact a ModIface, which we hope will be useful for
> > > |  applications such as IDEs to reduce GC time.
> > > |
> > > |  https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.h
> > > |  askell.org%2Fghc%2Fghc%2Fissues%2F17097%23roadmap-to-compacting-a-
> > > |  modiface&data=02%7C01%7Csimonpj%40microsoft.com%7C5614bf7bb16847bf5bab
> > > |  08d7cfe23972%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372064428432732
> > > |  44&sdata=YWpD1VEj%2FF4JVrRRi9cdNzlZ%2BQqgfFeRZ40NXC1kI2o%3D&reserv
> > > |  ed=0
> > > |
> > > |  If there is anyone who wishes to help with this project then please
> > > |  ping me on IRC. So far this is joint work between myself and Daniel G.
> > > |
> > > |  The first step we need to take is to get 1675 merged which replaces
> > > |  the type backing a FastString from a ByteString to a ShortByteString
> > > |  (and hence from a pinned ByteArray to an unpinned ByteArray).
> > > |
> > > |  https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.h
> > > |  askell.org%2Fghc%2Fghc%2F-
> > > |  %2Fmerge_requests%2F1675&data=02%7C01%7Csimonpj%40microsoft.com%7C5614
> > > |  bf7bb16847bf5bab08d7cfe23972%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C
> > > |  637206442843273244&sdata=N7fLEXXhlSESuye9BiCFQo76UmQ4%2B6GSbegaQcef0Lc
> > > |  %3D&reserved=0
> > > |
> > > |  Cheers,
> > > |
> > > |  Matt
> > > |  _______________________________________________
> > > |  ghc-devs mailing list
> > > |  ghc-devs at haskell.org
> > > |  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.hask
> > > |  ell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
> > > |  devs&data=02%7C01%7Csimonpj%40microsoft.com%7C5614bf7bb16847bf5bab08d7
> > > |  cfe23972%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637206442843283238&a
> > > |  mp;sdata=a1gBw6q0tuSxuPaByJgR9Jq0Ksk5%2BsP0kzMhaxeVgzs%3D&reserved=0
> > _______________________________________________
> > ghc-devs mailing list
> > ghc-devs at haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs