<p dir="ltr">I think Jasmin is really overrated, we can clearly do better in Haskell. </p>
<p dir="ltr">FWIW, the Kuna JVM assembler do support stack map frames, a lot of effort was put into getting this right, it would be a shame to redo this works if someone looks for a pure haskell implementation. <br>
</p>
<div class="gmail_quote">On 7 May 2016 5:32 pm, "Thomas Jakway" <<a href="mailto:tjakway@nyu.edu">tjakway@nyu.edu</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
This is a strange coincidence. I'm definitely no expert GHC hacker
but I started (highly preliminary) work on a JVM backend for GHC a
few weeks ago. It's here:
<a href="https://github.com/tjakway/ghcjvm/tree/jvm/compiler/jvmGen/Jvm" target="_blank">https://github.com/tjakway/ghcjvm/tree/jvm/compiler/jvmGen/Jvm</a><br>
(The memory runtime is here: <a href="https://github.com/tjakway/lljvm" target="_blank">https://github.com/tjakway/lljvm</a>)<br>
<br>
I'm very new to this so pardon my ignorance, but I don't understand
what the benefit is of intercepting STG code and translating that to
bytecode vs. translating Cmm to bytecode (or Jasmin assembly, as I'd
prefer)? It seems like Cmm is designed for backends and the obvious
choice. Or have I got this really mixed up?<br>
<br>
I hope this isn't out of line considering my overall lack of
experience but I think I can give some advice:<br>
<ul>
<li>read the JVM 7 spec cover-to-cover.</li>
<li>I <b><u>highly</u></b> suggest outputting <a href="http://web.mit.edu/javadev/packages/jasmin/doc/" target="_blank">Jasmin</a>
assembly instead of raw bytecode. The classfile format is
complicated and you will have to essentially rewrite Jasmin in
Haskell if you don't want to reuse it. Jasmin is also the de
facto standard assembler and much more thoroughly tested than
any homegrown solution we might make.<br>
</li>
<li>read the LLVM code generator. This project is more like the
LLVM backend than the native code generator.</li>
<li>Don't go for speed. The approach that I've begun is to
emulate a C stack and memory system the RTS can run on top of
(<a href="https://github.com/tjakway/lljvm/blob/master/src/main/java/lljvm/runtime/Memory.java" target="_blank">https://github.com/tjakway/lljvm/blob/master/src/main/java/lljvm/runtime/Memory.java</a>).
This will make getting <i>something</i> working much faster and
also solves the problem of how to deal with
memcpy/memset/memmove on the JVM. This will of course be <u>very</u>
slow (I think) and is not a permanent solution. Can't do
everything at once. Any other approach will probably require
rewriting the entire RTS from the beginning.</li>
<li>I don't think Frege is especially useful to this project,
though I'd love to be proven wrong. Frege's compilation model
is completely different from GHC's: they compile Haskell to Java
and then send that to javac. Porting GHC to the JVM is really
more like writing a Cmm to JVM compiler.</li>
</ul>
<p><br>
I've heard of the LambdaVM project but couldn't find the actual
code anywhere. The site where it was hosted appears to be
offline. I'd certainly like to look at it if anyone knows where
to find it.<br>
</p>
<p>Information on Jasmin:<br>
<a href="http://web.mit.edu/javadev/packages/jasmin/doc/" target="_blank">http://web.mit.edu/javadev/packages/jasmin/doc/</a><br>
<a href="http://web.mit.edu/javadev/packages/jasmin/doc/instructions.html" target="_blank">http://web.mit.edu/javadev/packages/jasmin/doc/instructions.html</a><br>
<a href="http://web.mit.edu/javadev/packages/jasmin/doc/about.html" target="_blank">http://web.mit.edu/javadev/packages/jasmin/doc/about.html</a><br>
</p>
<p>Once you've tried manually dealing with constant pools you'll
appreciate Jonathan Meyer's work!<br>
</p>
<p>I forked <a href="https://github.com/davidar" target="_blank">davidar's</a>
extended version of Jasmin. The differences versus the original
Jasmin are detailed <a href="https://github.com/davidar/jasmin/blob/master/html2x/xt.html" target="_blank">here</a>.
Some nice additions:<br>
</p>
<ul>
<li>supports invokedynamic</li>
<li>supports .annotation, .inner, .attribute, .deprecated
directives</li>
<li>better handling of the ldc_w instruction</li>
<li>multi-line fields</li>
<li>.debug directives</li>
<li>signatures for local variables</li>
<li>.bytecode directive to specify bytecode version</li>
<li>(most importantly, I think): support for the StackMap
attribute. If we eventually want to use new JVM instructions
like invokedynamic, we <b>need</b> stack map frames or the JVM
will reject our bytecode. JVM 7 has options to bypass this (but
it's a hack), but they're deprecated and I believe not optional
going forward. Alternatively we can stick with older bytecode
versions indefinitely and not use the new features.</li>
</ul>
<p>(Just to be clear, I forked it in case it was deleted. I didn't
write those features, the credit belongs to him).<br>
</p>
<p>I think the biggest risk is taking too much on at once. Any one
of these subtasks, writing a bytecode assembler, porting the RTS,
etc. could consume the whole summer if you're not careful.<br>
</p>
<p>I'd love to help out with this project!<br>
</p>
<p>Sincerely,<br>
Thomas Jakway<br>
</p>
<p>-------<br>
</p>
<p>Woops, after scrolling back through the emails it looks like
someone sent out the LambdaVM source. I'll have to take a look at
that.<br>
</p>
<br>
<br>
<div>On 05/02/2016 11:26 AM, Rahul Muttineni
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi GHC Developers,
<div><br>
</div>
<div>I've started working on a JVM backend for GHC [1] and I'd
love to work on it as my Summer of Haskell project. </div>
<div><br>
</div>
<div>Currently, the build system is setup using a mix of Shake
(for the RTS build) and Stack (for the main compiler build)
and I ensure that most commits build successfully. I have
ported the core part of the scheduler and ported over the
fundamental types (Capability, StgTSO, Task, StgClosure, etc.)
taking advantage of OOP in the implementation when I could. </div>
<div><br>
</div>
<div>Additionally, I performed a non-trivial refactor of the
hs-java package adding support for inner classes and fields
which was very cumbersome to do in the original package. On
the frontend, I have tapped into the STG code from the GHC
7.10.3 library and setup a CodeGen monad for generating JVM
bytecode. The main task of generating the actual bytecode,
porting the more critical parts of the RTS, and adding support
for the threaded RTS remain.</div>
<div><br>
</div>
<div>The strategy for compilation is as follows:</div>
<div>- Intercept the STG code in the GHC pipeline</div>
<div>- Convert from STG->JVM bytecode [2] in a similar manner
as STG->Cmm preserving semantics as best as possible [3]</div>
<div>- Port the GHC RTS (normal & threaded) to Java [4]</div>
<div>- Put all the generated class files + RTS into a single jar
to be run directly by the JVM.</div>
<div><br>
</div>
<div>My objectives for the project during the summer are:</div>
<div>- To implement the compilation strategy mentioned above</div>
<div>
<div>- Implement the Java FFI for foreign imports. [5]</div>
</div>
<div>
<div>- Implement the most important [6] PrimOps that GHC
supports. </div>
</div>
<div>- Port the base package replacing the C FFI imports with
equivalent Java FFI imports. [7]</div>
<div><br>
</div>
<div>A little bit about myself: I spent a lot of time studying
functional language implementation by reading SPJ's famous
book and reading research papers on related topics last summer
as self-study. </div>
<div><br>
</div>
<div>I took a break and resumed a couple months ago where I
spent a lot of time plowing through the STG->Cmm code
generator as well as the RTS and going back and forth between
them to get a clear understanding of how everything works. </div>
<div><br>
</div>
<div>Moreover, I compiled simple Haskell programs and observed
the STG, Cmm, and assembly output (by decompiling the final
executable with objdump) to understand bits of the code
generator where the source code wasn't that clear.</div>
<div><br>
</div>
<div>I also spent a great deal of time studying the JVM
internals, reading the JVM spec, looking for any new features
that could facilitate a high performance implementation [8]. </div>
<div><br>
</div>
<div>
<div>It would be great if someone with an understanding of
nuances of the RTS and code generator could mentor me for
this project. It has been a blast so far learning all the
prerequisites and contemplating the design. I'd be very
excited to take this on as a summer project.</div>
</div>
<div><br>
</div>
<div>Also, given that I have hardly 5 days remaining, does
anyone have suggestions on how I can structure the proposal
without getting into too many details? There are still some
parts of the design I haven't figured out, but I know I could
find some solution when I get to it during the porting
process.</div>
<div><br>
</div>
<div>Thanks,<br>
<div>Rahul Muttineni</div>
<div><br>
</div>
<div>[1] <a href="http://github.com/rahulmutt/ghcvm" target="_blank">http://github.com/rahulmutt/ghcvm</a><br clear="all">
<div><br>
</div>
<div>[2] I intend to organically derive an IR at a later
stage to allow for some optimizations by looking at the
final working implementation without an IR and looking for
patterns of repeated sequences of bytecode and assigning
each sequence its own instruction in the IR. </div>
<div><br>
</div>
<div>[3] Obviously, the lack of control of memory layouts
(besides allocating off the JVM heap using
DirectByteBuffers) and lack of general tail calls makes it
tough to match the semantics of Cmm, but there are many
solutions around it, as can be found in the few papers on
translating STG to Java/JVM bytecode. </div>
<div><br>
</div>
<div>[4] This is the GHC RTS without GC and profiling since
the JVM has great support for those already. Also, lots of
care must be taken to ensure that the lock semantics stays
in tact during the port.</div>
<div><br>
</div>
<div>[5] foreign exports will be dealt at a later stage, but
I am taking care of naming the closures nicely so that in
the future you don't have to type long names like the
labels GHC compiles to call a Haskell function in Java.</div>
<div><br>
</div>
<div>[6] Basically all the PrimOps that would be required to
provide plumbing for the Prelude functions that can
compile beginner-level programs found in books such as
Learn You a Haskell for Great Good.</div>
<div>
<div><br>
</div>
<div>[7] I know that it's a lot more complicated than just
replacing FFI calls. I'd have to change around a lot of
the code in base as well.</div>
</div>
<div><br>
</div>
<div>[8] I found that the new "invokedynamic" instruction as
well as the MethodHandle API (something like function
pointers) that were introduced in JDK 7 could fit the
bill. But as of now, I want to get a baseline
implementation that is compatible with Java 5 so I will
not be utilizing these newer features. </div>
<div><br>
</div>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
ghc-devs mailing list
<a href="mailto:ghc-devs@haskell.org" target="_blank">ghc-devs@haskell.org</a>
<a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs</a>
</pre>
</blockquote>
<br>
</div>
<br>_______________________________________________<br>
ghc-devs mailing list<br>
<a href="mailto:ghc-devs@haskell.org">ghc-devs@haskell.org</a><br>
<a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs" rel="noreferrer" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs</a><br>
<br></blockquote></div>