Q Monad State

[Michael Sloan]

On Fri, Mar 10, 2017 at 6:22 PM, Ethan Pailes <ethanpailes at gmail.com> wrote:
> I had not considered the issue about incremental compilation. It certainly
> feels like it should cause problems. I wanted to confirm your suspicious, so
> I threw together an example project which is meant to cause the issues that
> you mentioned, but I was unable to produce any sort of error. I've thrown
> the code up at https://github.com/ethanpailes/th-ioref if you want to take a
> look. Basically there is a Ref module which makes an IORef, then a Quote
> module which reads it and a Lib module which uses the Quote module to add a
> top level value which depends on the contents of the IORef. I tried building
> it, then just modifying Quote and rebuilding. I was expecting exactly the
> sort of error you described, but everything works fine. When I added an
> executable which depends on the library, that also works fine. I feel like
> there should be an error though, so I would appreciate it if you could take
> a look at the code and try to figure out how to produce the sort of error
> that it seems like should occur. If it turns out that this does actually
> work, it would be great to know why because your point really seems like it
> should be right.

I don't see any writing of the IORef in this example.  Indeed, if I add

$(runIO (writeIORef ref 5) >> return [])

To Lib.hs, it still just prints out 4 for both.  You *really* won't be
able to have the IORef transfer info from Lib.hs to Main.hs, since
they are in different cabal components and so will be compiled by
different invocations of ghc.

>> A third approach to consider is to just write out a file for each module,
>> listing the metadata.  When compiling other modules, you can use
>> `reifyModules` to get the imports, so that you know which metadata files to
>> load.  One issue with this (as well as the IORef) is that it won't work
>> across multiple packages.
>
> Do you know if there is a way to learn where the build output is going in
> the Q Monad? It would be much better to write the tmp files out to some
> place in .stack_work rather than in /tmp. That way if /tmp disappears we
> don't loose the cached data.

Could use findCabalFile from
https://hackage.haskell.org/package/th-utilities-0.2.0.1/docs/TH-RelativePaths.html
to find the cabal file, take the dir that file is in, and add
"/.stack-work" to it.

> Ethan
>
> On Fri, Mar 10, 2017 at 5:47 PM, Michael Sloan <mgsloan at gmail.com> wrote:
>>
>> Hi!
>>
>> I don't think the IORef approach will work well, because GHC won't
>> recompile modules unless they've changed (or if a file added via
>> qAddDependentFile changes).  So if you've done a compile of module A, and B
>> depends on it, and then you just change B, only B will recompile and you
>> won't have info from A.
>>
>> Not sure if this applies to your circumstance, but one issue with the top
>> level declaration approach is that TH can't enumerate the members of a
>> module.  One hacky way to write down such metadata in a retrievable way is
>> to use typeclass instances.  For example you might have
>>
>> class TypeInfo (name :: Symbol) (typeInfo :: Symbol)
>>
>> Then, you can reify the typeclass to get all its instances.  As long as
>> you only need to write out this info from declaration quasiquotes, this info
>> ought to be available for typechecking later in the module.
>>
>> A third approach to consider is to just write out a file for each module,
>> listing the metadata.  When compiling other modules, you can use
>> `reifyModules` to get the imports, so that you know which metadata files to
>> load.  One issue with this (as well as the IORef) is that it won't work
>> across multiple packages.
>>
>> -Michael
>>
>> On Fri, Mar 10, 2017 at 12:29 PM Ethan Pailes <ethanpailes at gmail.com>
>> wrote:
>>>
>>> Hi,
>>>
>>> I've been working on PADS with Kathleen Fisher for the past couple of
>>> months, and we have some questions about the Q Monad.
>>>
>>> In working on extending the PADS language, we have encountered a need to
>>> maintain a type environment across invocations of the [pads||] quasi quoter.
>>> This allows PADS types to be defined in one quote block and used in a later
>>> quote block. Unfortunately, the right way to do this is not obvious from the
>>> Q Monad’s interface[1]. The monad provides `qGetQ` and `qPutQ` methods to
>>> provide a place to store state across quotes, but unfortunately this state
>>> is local to each module. It is desirable from a usability perspective to be
>>> able to reuse PADS declarations across modules, but in the particular case
>>> of PADS the problem is actually worse than that. PADS ships with a standard
>>> library module which uses PADS to define several of the core types required
>>> to be productive, so without the ability to do type checking across modules
>>> the use of this standard library becomes impossible. There are two solutions
>>> to the problem that we have considered.
>>>
>>>
>>> We can generate metadata about each type as it is defined and emit a top
>>> level declaration which contains this metadata.
>>>
>>> This clutters the global namespace.
>>>
>>> It lets you examine the metadata for each type easily in `ghci` which
>>> leads to a nice debugging workflow.
>>>
>>> If you want to perform any typechecks or other calculations based on the
>>> metadata, they must happen after generated code is spliced in.
>>>
>>> This means you lose a lot of type safety because everything must be
>>> represented as an `Exp` (even with typed template haskell there is no way to
>>> get a type for an `Exp` of the form `VarE <<name of metadata>>`)
>>>
>>> It becomes impossible to report type errors at compile time. This is a
>>> dealbreaker because otherwise, what is the point?
>>>
>>> Using `unsafePerformIO` to create an IORef and then using the Q Monand’s
>>> `qRunIO` function to access the state cell.
>>>
>>> This works smoothly across modules
>>>
>>> You get type safety back because everything happens in the Q Monad. You
>>> don’t have to generate code in order to get at the values.
>>>
>>> It uses `unsafePerformIO` and `qRunIO` so you have to worry about how
>>> often everything gets run.
>>>
>>> It seems like it should be safe with a {-# NONINLINE #-} pragma for the
>>> code which creates the IORef.
>>>
>>> The IORef is hidden, and no one else should be able to modify it.
>>>
>>> Adding stuff to a `Data.Map.Strict` (the environment implementation we
>>> are using) is idempotent, so the quote code getting run multiple times
>>> should not be an issue.
>>>
>>>
>>> We have two main questions based on this.
>>>
>>> Is the IO (ab)use outlined above really as safe as we think it is?
>>>
>>> Given that environments are such a common requirement in compiling
>>> programming languages, and the goal of quasi quotation is allowing easy
>>> creation of EDSLs for Haskell, does it make sense to provide some state
>>> mechanism for the Q monad which is not restricted by modules.
>>>
>>>
>>>
>>> [1]:
>>> https://hackage.haskell.org/package/template-haskell-2.11.1.0/docs/Language-Haskell-TH-Syntax.html
>>>
>>>
>>> Thanks,
>>>
>>> Ethan Pailes
>>>
>>> _______________________________________________
>>> Libraries mailing list
>>> Libraries at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>
>