Request for feedback: deriving strategies syntax

[Shayan Najd]

Hi all,

Shayan, have you written anything describing how things are going?


Ben,
  thank you for reaching out.

I am not sure about the history and the context of the discussions in this
thread so far, but here is a brief description of what we intend to do and
how far we have come so far.

The idea is simple: allow grand larceny, with a clear conscience!
We are trying to allow metaprograms to directly access and reuse the
"native" machinery inside the compiler. There is no need to have a separate
representation of syntax, and the associated sets of tools, as in Template
Haskell, or even as in third-party libraries like Haskell-Src-Exts (HSE)
and others in Haskell-Suite. GHC has a "native" representation of terms,
with many tools already built on top of that, including the compiler passes
like the renamer, or the typechecker. We would certainly like to unify
these representations and tools as much as possible, and allow metaprograms
to directly access these internal machineries.

The eventual goal is indeed more than sole reuse of the AST in GHC (HsSyn);
metaprograms should also be able to reuse the "infrastructure" like the
different environments and monads used for name resolution or typechecking.
If we are to treat `deriving`, and many other similar constructs in the
language simply as metaprograms, access to types is *sometimes* crucial:
they describe a type-directed elaboration process.
I am a strong advocate of such simplifications for both the front-end
(i.e., how users perceive constructs in the language), and the back-end
(i.e., how the compiler implements them). However, there is a huge, often
underestimated, gap between having these constructs as built-in, and
 having them as yet another metaprogram. The gap is both in theory, e.g.,
the equational and algebraic properties of the constructs, and in practice,
e.g., handling the error messages and maintain the general
macro/metaprogramming system allowing for such definitions. Many
researchers, including me, have worked on these problems; there is still
work to be done. We can discuss some of these related works, if you are
interested.

For now, as the first step, we are focusing on how to reuse GHC AST (HsSyn)
for metaprogramming. The immediate problem with reusing GHC AST is that it
comes with a large set of extra fields and constructors carrying the
information only necessary for the passes inside GHC.
Users (metaprogrammers?) do not want to, and do not need to, deal with
these extra fields and constructors. Moreover, the AST should be,
to some degree, easy to use, and we have no exact metric for the ease of
use.

Our solution is twofold, addressing the two mentioned problems:
  (a) after a rather deep analysis, we have come up with a simple, yet
powerful, encoding of extensible ASTs that allows us to extend a base AST
with new fields and constructors;
  (b) we are updating GHC AST to match HSE AST, hence unifying the two
(based on its popularity, we can say HSE is easy enough to use).

In theory, due to (a), code for GHC passes do not need to change.
In practice, due to (b), we may need to update the code in GHC passes in
some cases.

It is needless to say, that this is all work in progress.
Following Simon's suggestion, I am planning to give a talk on this in
Haskell Implementors' Workshop in Japan. But since then, for further
information, you may be brave enough to read our sketchy notes / pieces of
code:
- An example using the extensible encoding (a bit outdated variant though):

https://github.com/shayan-najd/NativeMetaprogramming/blob/master/HsSyn/Example.hs
- Discussion on why this extensible encoding is suitable:

https://github.com/shayan-najd/NativeMetaprogramming/wiki/Extensions-&-Annotations
- To see progress on unifying HSE AST and GHC AST compare
      the updated GHC AST at

https://github.com/shayan-najd/NativeMetaprogramming/blob/master/HsSyn/SyntaxExtensibleAutoSplitted.hs
      and the updated HSE AST at

https://github.com/shayan-najd/NativeMetaprogramming/blob/master/HSE/SyntaxExtensibleAuto.hs
   They should be line-by-line comparable unless the comments says so
otherwise.

There is much to be done without a doubt, specially to synchronise with and
learn form communities working on similar topics

Yours,
  Shayan

On Fri, Aug 5, 2016 at 11:06 AM, Ben Gamari <ben at smart-cactus.org> wrote:

> Ryan Scott <ryan.gl.scott at gmail.com> writes:
>
> > Sorry for not including the full context on that link. It's part of a
> > Summer of Haskell 2016 project called Native Metaprogramming in
> > Haskell [1] (a.k.a. Introspective Template Haskell [2]), aiming to fix
> > Trac #11081 [3].
> >
> On this note, it would be great to hear a bit about the state of this
> project. Shayan, have you written anything describing how things are
> going? It would be great if you could update the Wiki page [2]
> describing a bit about the approach you have taken and the current state
> of your implementation.
>
> Cheers,
>
> - Ben
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20160805/329116da/attachment.html>