<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang=EN-US link=blue vlink="#954F72"><div class=WordSection1><p class=MsoNormal>Hi Ben,</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Thanks for the reply!</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>> </p><p class=MsoNormal>> > Hi *,</p><p class=MsoNormal>> ></p><p class=MsoNormal>> > I’ve been working the past 4 or so months on reviving dynamic linking</p><p class=MsoNormal>> > support for Windows in a way that has the most chance of working.</p><p class=MsoNormal>> ></p><p class=MsoNormal>> > My first patch in the series is up on Phabricator and with this patch</p><p class=MsoNormal>> > dynamic linking work again, but only for the threaded RTS.</p><p class=MsoNormal>> ></p><p class=MsoNormal>> Thanks for all of your work on this, Tamar!</p><p class=MsoNormal>> </p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Home stretch :)</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>> </p><p class=MsoNormal>> > The reason for this is because the core libraries that get distributed</p><p class=MsoNormal>> > with GHC get compiled with -threaded and shared libraries on Windows</p><p class=MsoNormal>> > can’t have dangling symbols.</p><p class=MsoNormal>> ></p><p class=MsoNormal>> Let me make sure we are on the same page here: By "dangling symbols" do</p><p class=MsoNormal>> you just mean symbols that the linker did not find a definition for at</p><p class=MsoNormal>> link time (e.g. as is the case with libHSrts symbols when we link</p><p class=MsoNormal>> libHSbase)?</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Yes, indeed.</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>> </p><p class=MsoNormal>> > In any case, I’m at the point now where I need to be able to delay the</p><p class=MsoNormal>> > selection of the runtime library till the final link. E.g. when the</p><p class=MsoNormal>> > exe or dll is made. The problem however is that when linked normally,</p><p class=MsoNormal>> > dependencies are satisfied by the Windows loader, before the program</p><p class=MsoNormal>> > is run. One way I could do this is with Window’s equivalent to SONAME.</p><p class=MsoNormal>> > Unfortunately this only works with SxS Assemblies, and I’ll need Admin</p><p class=MsoNormal>> > rights to be able to register the shared libraries.</p><p class=MsoNormal>> ></p><p class=MsoNormal>> Hmm, why? I thought recent Windows releases had a notion of "user local"</p><p class=MsoNormal>> installation, no? From what little I have heard it sounds like SxS</p><p class=MsoNormal>> assemblies is the right solution here.</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Yes, so to be clear, SxS absolutely solve this problem. For final installs.</p><p class=MsoNormal>The majority of the issue is that the testsuite won't have the assemblies in</p><p class=MsoNormal>the SxS cache.</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>There *is* a sort of RPATH equivalent for SxS that can be used here, it however</p><p class=MsoNormal>has two problems:</p><p class=MsoNormal>1) Even though the API has no such limit, the implementation in the Windows loader</p><p class=MsoNormal> limits it per application to 5 entries. Obviously this won't be enough. </p><p class=MsoNormal> So this is absolutely another option (maybe even preferable now that I think about it</p><p class=MsoNormal> since it require almost no code change, mostly some build system changes.).</p><p class=MsoNormal> Can do either one of two things:</p><p class=MsoNormal> a) Copy all dll's to the lib folder when they're compiled instead of leaving them in place and</p><p class=MsoNormal> add a single SxS search entry to find them.</p><p class=MsoNormal> b) Turn of SxS in the testsuite for any Assemblies not the RTS, and add the inplace RTS directory to</p><p class=MsoNormal> the SxS search path. Since it's only the RTS that's an issue.</p><p class=MsoNormal> </p><p class=MsoNormal>2) The other problem is that the paths specified have to be relative to the application.</p><p class=MsoNormal> (Of the top of my head) It doesn't support absolute paths. Which means I can't have GHC generate the entry because I have no idea where the testsuite intends to run the binary.</p><p class=MsoNormal> One way around this is to have the testsuite generate the needed config file. That should be do-able.</p><p class=MsoNormal> </p><p class=MsoNormal>I'll investigate this method first. I had discarded it for some reason before but now can't remember...</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>> </p><p class=MsoNormal>> > This is a problem for running tests in the testsuite using the inplace GHC.</p><p class=MsoNormal>> ></p><p class=MsoNormal>> > Typically on Windows the way you would do this is by delay loading the</p><p class=MsoNormal>> > dll. This allows me to write some code on startup and manually load</p><p class=MsoNormal>> > the runtime dll. The Windows loader would then just use the loaded</p><p class=MsoNormal>> > dll. Unfortunately delay loading does not support const extern data.</p><p class=MsoNormal>> > Such as const extern RtsConfig defaultRtsConfig;</p><p class=MsoNormal>> ></p><p class=MsoNormal>> Silly Windows.</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Yeah, unfortunately this is because this isn't done by any OS functions. Lazy loading is purely something</p><p class=MsoNormal>implemented by linkers on Windows and the appropriate runtimes. In essense it just creates a stub that replaces all functions, which first checks if the dll is loaded, if not load it and then call the real function. Which is why it only works for functions.</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>> </p><p class=MsoNormal>> > The RTS/GHC is full of such usage so it won’t be easy to change.</p><p class=MsoNormal>> > Though I’d only have to change those exposed by Rts.h.</p><p class=MsoNormal>> ></p><p class=MsoNormal>> > So I have two options:</p><p class=MsoNormal>> > 1) Replace all const externs with a function call. This would work,</p><p class=MsoNormal>> > but isn’t ideal because it would break if someone in the future</p><p class=MsoNormal>> > adds a new data entry instead of a function. And we have an extra</p><p class=MsoNormal>> > function call everywhere.</p><p class=MsoNormal>> </p><p class=MsoNormal>> Right, I'm really not a fan of this option. Crippling the RTS's use of C</p><p class=MsoNormal>> on account of arbitrary limitations of the Windows dynamic linker</p><p class=MsoNormal>> doesn't seem very appealing.</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I was not a fan of this either. Would imagine I would be chased around with flaming pitchforks were I to do this..</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>> </p><p class=MsoNormal>> > 2) I could do some hacks on the Windows side, e.g. compile the program</p><p class=MsoNormal>> > to a shared library, embed the shared library inside the exe and on</p><p class=MsoNormal>> > startup after loading the propert rts, load the DLL from (mmapped)</p><p class=MsoNormal>> > memory and run the code.</p><p class=MsoNormal>> </p><p class=MsoNormal>> This sounds like it would get the job done, although it certainly adds</p><p class=MsoNormal>> complexity. Do you have any intuition for how much work it would be to</p><p class=MsoNormal>> implement this?</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>We sorta already do 80% of this. For Windows when making a dynamic version of GHC, the ghc-stage2.exe is a </p><p class=MsoNormal>very thin shell, who's only purpose is to load the right libraries and change the search path to include the lib folder. The actual code is in a dll named e.g. ghc-stage2.exe.dll. The change needed would be to embed this dll into the exe (which is trivially done), and then load it from memory. This would require some work but there are enough wrapper code out there with appropriate licenses that we can use to accomplish this. Or at least get a running head start.</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>> </p><p class=MsoNormal>> Cheers,</p><p class=MsoNormal>> </p><p class=MsoNormal>> - Ben</p></div></body></html>