Loading a typechecked module and then using it immediately as a package

Tue Jun 29 08:40:32 UTC 2021

PUBLIC

What's weird about it is that if I print the `moduleNameProvidersMap`, I can see `MyLib` inside, and it looks no different than any other module from e.g. `ghc-prim` that I can import into `Test.hs` without any package qualification. Also, why does the error message refer to the file name `input/MyLib.hs`? Why does GHC even know that that is where it would have to be loaded from, if it weren't to be used from an already loaded package?

-----Original Message-----
From: Erdi, Gergo 
Sent: Monday, June 28, 2021 3:04 PM
To: 'Matthew Pickering' <matthewtpickering at gmail.com>
Cc: 'ghc-devs at haskell.org' <ghc-devs at haskell.org>
Subject: RE: [External] Re: Loading a typechecked module and then using it immediately as a package

INTERNAL

Oh, and one more question. This all works if in `Test.hs` I use a package import for `MyLib`. However, if I remove that, I get an error 

	attempting to use module ‘main:MyLib’ (input/MyLib.hs) which is not loaded

So I tried adding an ExposePackage to the DynFlags's packageFlags between loading the library and typechecking `Test.hs`, but I couldn't get that to work:
1. If I just set it in the `hsc_dflags`, it doesn't take.
2. If I use `initUnits` after setting it in `hsc_dflags`, I get a new error:

	cannot satisfy my-pkg

-----Original Message-----
From: Erdi, Gergo
Sent: Monday, June 28, 2021 2:41 PM
To: Matthew Pickering <matthewtpickering at gmail.com>
Cc: ghc-devs at haskell.org
Subject: RE: [External] Re: Loading a typechecked module and then using it immediately as a package

INTERNAL

Thanks, I got this working (after changing it slightly to use the GHC 9.0 API to the UnitState). However, I have some followup questions:

1. In the fakeUnitInfo, what is the relationship between unitLibraries and unitExposedModules? 
2. In the fakeUnitInfo, what is the unitImportDirs used for?
3. What is the difference between a PackageId and a PackageName?

Other than that, the code registers MyLib into the my-pkg package by letting `load` do the actual heavy lifting. For my purposes, I will need to control this loading myself: only `MyLib`'s typechecked interface should be used, with no actual compilation of definitions. So I wrote some code that makes a ModIface from a TcGblEnv. I seemingly got the whole thing working just by setting mi_exports from the TcGblEnv and leaving everything else with trivial values (fingerprint0 for fingerprints, empty lists for instances, etc.). If `MyLib` has no external dependencies and doesn't define any typeclasses, instances, or type families, is this enough? What is the role of `mi_globals`, should I set that from `tcg_rdr_env`, or should I leave it empty?

Thanks,
	Gergo

-----Original Message-----
From: Matthew Pickering <matthewtpickering at gmail.com>
Sent: Friday, June 25, 2021 6:54 PM
To: Erdi, Gergo <Gergo.Erdi at sc.com>
Cc: ghc-devs at haskell.org
Subject: [External] Re: Loading a typechecked module and then using it immediately as a package

ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Always report suspicious emails using the Report As Phishing button in Outlook to protect the Bank and our clients.

Hi Gergo,

Please see a minimal example in this gist.

https://clicktime.symantec.com/3Eb2qXqu8Yp6VtdK9d5pNmL7Vc?u=https%3A%2F%2Fgist.github.com%2Fmpickering%2F5029c7f244c484c91d665bcbc6bc6406

Cheers,

Matt

On Fri, Jun 25, 2021 at 10:20 AM Erdi, Gergo via ghc-devs <ghc-devs at haskell.org> wrote:
>
> PUBLIC
>
>
> Hi,
>
>
>
> I have the following to .hs files:
>
>
>
> MyLib.hs:
>
> module MyLib where
> …
>
> Test.hs:
>
> {-# LANGUAGE PackageImports  #-}
> module Test where
> import “my-pkg” MyLib
> …
>
>
>
> I would like to parse/typecheck/load MyLib.hs into some Unit “my-unit”, then add that to the package “my-pkg”, and then typecheck Test.hs, all in-proc using the GHC API, without putting any other files on disk. How do I do that?
>
>
>
> What I tried is loading MyLib.hs after setting the homeUnitId in the DynFlags to “my-unit”, then editing the packageNameMap in the unitState of the DynFlags to may “my-pkg” to “my-unit”:
>
> setHomeUnit :: (GhcMonad m) => UnitId -> m ()
>
> setHomeUnit unitId = do
>
>     dflags <- getSessionDynFlags
>
>     modifySession $ \h -> h{ hsc_dflags = dflags{ homeUnitId = unitId 
> } }
>
>
>
> registerUnit :: (GhcMonad m) => PackageName -> UnitId -> m ()
>
> registerUnit pkg unitId = modifySession $ \h -> h{ hsc_dflags = 
> addUnit $ hsc_dflags h }
>
>   where
>
>     addUnit dflags = dflags
>
>         { unitState = let us = unitState dflags in us
>
>             { packageNameMap = M.insert pkg (Indefinite unitId
> Nothing) $ packageNameMap us
>
>             }
>
>         }
>
>
>
> pipeline = do
>
>         setHomeUnit myUnit
>
>         loadModule =<< typecheckModule =<< parseModule =<< modSumarryFor “MyLib”
>
>         registerUnit myPkg myUnit
>
>
>
>         setHomeUnit mainUnitId
>
>         typecheckModule =<< parseModule =<< modSumarryFor “Test”
>
>
>
>
>
> Alas, this doesn’t work: the import of `MyLib` from `my-pkg` fails with:
>
>
>
> input/linking/Test.hs:5:1: error:
>
>     Could not find module ‘MyLib’
>
>     It is not a module in the current program, or in any known package.
>
>
>
> TBH I’m not very surprised that it didn’t work – that registerUnit function is doing some pretty deep surgery on the unitState that probably breaks several invariants. Still, I wasn’t able to find a better way – all the functions in GHC.Unit.State seem to be for querying only.
>
>
>
> Thanks,
>
>             Gergo
>
>
> This email and any attachments are confidential and may also be 
> privileged. If you are not the intended recipient, please delete all 
> copies and notify the sender immediately. You may wish to refer to the 
> incorporation details of Standard Chartered PLC, Standard Chartered 
> Bank and their subsidiaries at https: //www.sc.com/en/our-locations
>
> Where you have a Financial Markets relationship with Standard 
> Chartered PLC, Standard Chartered Bank and their subsidiaries (the 
> "Group"), information on the regulatory standards we adhere to and how 
> it may affect you can be found in our Regulatory Compliance Statement 
> at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at
> http: //www.sc.com/rcs/fm
>
> Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer.
>
> Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions.
>
> Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same.
>
> Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> https://clicktime.symantec.com/3A3FModesGCHWyu3Lvr5tp57Vc?u=http%3A%2F
> %2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs

This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https: //www.sc.com/en/our-locations

Where you have a Financial Markets relationship with Standard Chartered PLC, Standard Chartered Bank and their subsidiaries (the "Group"), information on the regulatory standards we adhere to and how it may affect you can be found in our Regulatory Compliance Statement at https: //www.sc.com/rcs/ and Regulatory Compliance Disclosures at http: //www.sc.com/rcs/fm

Insofar as this communication is not sent by the Global Research team and contains any market commentary, the market commentary has been prepared by the sales and/or trading desk of Standard Chartered Bank or its affiliate. It is not and does not constitute research material, independent research, recommendation or financial advice. Any market commentary is for information purpose only and shall not be relied on for any other purpose and is subject to the relevant disclaimers available at https: //www.sc.com/en/regulatory-disclosures/#market-disclaimer.

Insofar as this communication is sent by the Global Research team and contains any research materials prepared by members of the team, the research material is for information purpose only and shall not be relied on for any other purpose, and is subject to the relevant disclaimers available at https: //research.sc.com/research/api/application/static/terms-and-conditions. 

Insofar as this e-mail contains the term sheet for a proposed transaction, by responding affirmatively to this e-mail, you agree that you have understood the terms and conditions in the attached term sheet and evaluated the merits and risks of the transaction. We may at times also request you to sign the term sheet to acknowledge the same.

Please visit https: //www.sc.com/en/regulatory-disclosures/dodd-frank/ for important information with respect to derivative products.