<div dir="ltr">As usual, I could suggest a really crazy alternative: It could be possible to design your code as an EDSL that can emit his own source code, in the same way that web formlets emit HTML rendering. <div><br></div><div>In this case, instead of HTML rendering, the rendering would be the source code of the closure that you want to execute remotely. Then you can compile it at the emitting node or at the receiver. </div><div><br></div><div>The advantage is that you may remotely execute any routine coded using the EDSL. Hiding the mechanism behind a few primitives of the EDSL.</div><div><br></div><div>The disadvantage is that you can not use IO routines with liftIO. You need an special lifting mechanism that produces also the source code of the IO routine.</div><div><br></div><div>I´m doing some research on this mechanism with the Transient monad. </div></div><div class="gmail_extra"><br><div class="gmail_quote">2015-03-16 1:54 GMT+01:00 felipe zapata <span dir="ltr"><<a href="mailto:tifonzafel@gmail.com" target="_blank">tifonzafel@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I haven't considered that idea, but it seems the natural solution.<div><br></div><div>Many thanks</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On 15 March 2015 at 20:31, Ozgun Ataman <span dir="ltr"><<a href="mailto:ozgun.ataman@soostone.com" target="_blank">ozgun.ataman@soostone.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div><span></span></div><div><div>Anecdotal support for this idea: This is exactly how we distribute hadron[1]-based Hadoop MapReduce programs to cluster nodes at work. The compiled executable essentially ships itself to the nodes and recognizes the different environment when executed in that context. <br><br>[1] hadron is a haskell hadoop streaming framework that came out of our work. It's on github and close to being released on hackage once the current dev branch is finalized/merged. In case it's helpful: <a href="https://github.com/soostone/hadron" target="_blank">https://github.com/soostone/hadron</a><br><br></div><div>Oz</div><div><div><div><br>On Mar 15, 2015, at 8:06 PM, Andrew Cowie <<a href="mailto:andrew@operationaldynamics.com" target="_blank">andrew@operationaldynamics.com</a>> wrote:<br><br></div><blockquote type="cite"><div><div dir="ltr"><div><div><div><div>Bit of a whinger from left-field, but rather than deploying a Main script and then using GHCi, have you considered compiling the program and shipping that?<br><br></div>Before you veto the idea out of hand, statically compiled binaries are good for being almost self-contained, and (depending on what you changed) and they rsync well. And if that doesn't appeal, then consider instead building the Haskell program dynamically; Hello World is only a couple kB; serious program only a hundred or so.<br><br></div>Anyway, I know you're just looking to send a code fragment closure, but if you're dealing with the input and output of the program through a stable interface, then the program is the closure.<br><br></div>Just a thought.<br><br></div>AfC<br></div><br><div class="gmail_quote">On Mon, Mar 16, 2015 at 9:53 AM felipe zapata <<a href="mailto:tifonzafel@gmail.com" target="_blank">tifonzafel@gmail.com</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hi all,</div><div>I have posted the following question on stackoverflow, but so far I have not received an answer.</div><a href="http://stackoverflow.com/questions/29039815/distributing-haskell-on-a-cluster" target="_blank">http://stackoverflow.com/questions/29039815/distributing-haskell-on-a-cluster</a><br><div><br></div><div><br></div><div><p style="margin:0px 0px 1em;padding:0px;border:0px;font-size:15px;clear:both;font-family:'Helvetica Neue',Helvetica,Arial,sans-serif;line-height:19.5px">I have a piece of code that process files,</p><pre style="margin-top:0px;padding:5px;border:0px;font-size:13px;overflow:auto;width:auto;max-height:600px;font-family:Consolas,Menlo,Monaco,'Lucida Console','Liberation Mono','DejaVu Sans Mono','Bitstream Vera Sans Mono','Courier New',monospace,sans-serif;color:rgb(57,51,24);word-wrap:normal;background-color:rgb(238,238,238)"><code style="margin:0px;padding:0px;border:0px;font-size:13px;font-family:Consolas,Menlo,Monaco,'Lucida Console','Liberation Mono','DejaVu Sans Mono','Bitstream Vera Sans Mono','Courier New',monospace,sans-serif;white-space:inherit"><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)">processFiles </span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)">::</span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)"> </span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)">[</span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)">FilePath</span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)">]</span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)"> </span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)">-></span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)"> </span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)">(</span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)">FilePath </span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)">-></span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)"> IO </span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)">())</span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)"> </span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)">-></span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)"> IO </span><span style="margin:0px;padding:0px;border:0px;font-size:13px;color:rgb(0,0,0)">()</span></code></pre><p style="margin:0px 0px 1em;padding:0px;border:0px;font-size:15px;clear:both;font-family:'Helvetica Neue',Helvetica,Arial,sans-serif;line-height:19.5px">This function spawns an async process that execute an IO action. This IO action must be submitted to a cluster through a job scheduling system (e.g Slurm).</p><p style="margin:0px 0px 1em;padding:0px;border:0px;font-size:15px;clear:both;font-family:'Helvetica Neue',Helvetica,Arial,sans-serif;line-height:19.5px">Because I must use the job scheduling system, it's not possible to use cloudHaskell to distribute the closure. Instead the program writes a new <em style="margin:0px;padding:0px;border:0px;font-size:15px">Main.hs</em> containing the desired computations, that is copy to the cluster node together with all the modules that main depends on and then it is executed remotely with "runhaskell Main.hs [opts]". Then the async process should ask periodically to the job scheduling system (using <em style="margin:0px;padding:0px;border:0px;font-size:15px">threadDelay</em>) if the job is done.</p><p style="margin:0px 0px 1em;padding:0px;border:0px;font-size:15px;clear:both;font-family:'Helvetica Neue',Helvetica,Arial,sans-serif;line-height:19.5px">Is there a way to avoid creating a new Main? Can I serialize the IO action and execute it somehow in the node?</p><p style="margin:0px 0px 1em;padding:0px;border:0px;font-size:15px;clear:both;font-family:'Helvetica Neue',Helvetica,Arial,sans-serif;line-height:19.5px">Best,</p><p style="margin:0px 0px 1em;padding:0px;border:0px;font-size:15px;clear:both;font-family:'Helvetica Neue',Helvetica,Arial,sans-serif;line-height:19.5px">Felipe</p></div></div>
______________________________<u></u>_________________<br>
Haskell-Cafe mailing list<br>
<a href="mailto:Haskell-Cafe@haskell.org" target="_blank">Haskell-Cafe@haskell.org</a><br>
<a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe" target="_blank">http://mail.haskell.org/cgi-<u></u>bin/mailman/listinfo/haskell-<u></u>cafe</a><br>
</blockquote></div>
</div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>Haskell-Cafe mailing list</span><br><span><a href="mailto:Haskell-Cafe@haskell.org" target="_blank">Haskell-Cafe@haskell.org</a></span><br><span><a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe</a></span><br></div></blockquote></div></div></div></div></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
Haskell-Cafe mailing list<br>
<a href="mailto:Haskell-Cafe@haskell.org">Haskell-Cafe@haskell.org</a><br>
<a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">Alberto.</div>
</div>