Haskell DLL (using FFI) causes memory leak.

[Phyx]

Oh and also, neither of those tickets you linked to should prevent you from
using a newer GHC to make shared libraries.

the Rts.h headers are nothing but convenience functions and you can just
create the prototypes you need in your own headers.

On Mon, Aug 27, 2018 at 5:46 PM Phyx <lonetiger at gmail.com> wrote:

> Hi,
>
> You're likely just hitting a memory leak, Haskell DLLs and the RTS aren't
> designed to be unload-able, which is why you can't call hs_init again after
> you stop.
>
> The leak happens only when you do a foreign export because foreign exports
> create C wrappers to initializers for each function you export.
>
> https://github.com/ghc/ghc/blob/21f0f56164f50844c2150c62f950983b2376f8b6/compiler/deSugar/DsForeign.hs#L668
>
> These are marked as static initializers, and as such the CRT will
> initialize them on DLL_PROCESS_ATTACH time. At DLL_PROCESS_DETACH
> time the destructors would be run, but we don't have destructors for these
> initializers, so whatever they did will always persist.
>
> Your use case is fairly uncommon, but file a bug report against the leaks
> anyway. Why do you need to keep loading and unloading the dll?
>
> Note that FreeLibrary like dlclose on unix does not guarantee that the
> library is freed, it just decrements the reference count. and when it
> reaches 0 it will *eventually* unload it. Notice that if you use
> UnmapViewOfFile instead of FreeLibrary the leak doesn't happen as that
> *actually* immediately unmaps the image.  But this will get you in all
> sorts of trouble later on when terminating (you'll likely segfault at that
> point).
>
> Things will get worse when you actually do call hs_init, because hs_init
> will call initLinker which will load libraries of it's own. Unloading the
> top level
> does not decrease the reference counts of the dependencies, and we do no
> extra work to ensure this.
>
> If you really need to load/reload libraries, then I fear your only choice
> is some kind of process isolation and communicating back using share
> memory/pipes
> or some sort of IPC.
>
> Regards,
> Tamar
>
> On Mon, Aug 27, 2018 at 4:33 PM Ольга Филиппская <olga.kenobi at gmail.com>
> wrote:
>
>> It's true that one must initialize the runtime when calling Haskell from
>> C/C++. We do this in the real project and just a pair of calls
>> hs_init/hs_exit causes memory to be leaked even faster. But I tried to
>> construct the minimal example to describe the bug, so I reduced all the
>> extra-code.
>>
>> пн, 27 авг. 2018 г. в 18:00, Vanessa McHale <vanessa.mchale at iohk.io>:
>>
>>> I don't know about C++, but I do know that when calling Haskell from C
>>> code one must initialize the runtime and then exit it once finished. I also
>>> know that one must only initialize the runtime once over the course of the
>>> program's run. As far as I can tell, this does not occur in your example.
>>>
>>> On 08/27/2018 09:44 AM, Ольга Филиппская wrote:
>>>
>>> Hello.
>>>
>>> *Summary: *memory is consumed without releasing when a Haskell DLL
>>> (that uses FFI) is loaded and unloaded in an endless loop.
>>>
>>> *Details*: I'm working on a C++ project relying on a DLL that uses
>>> Haskell code. The DLL was built with GHC 8.0.1 x64, OS is Windows 7. (Newer
>>> versions of GHC - 8.2.1 and later - do not allow you to build such DLLs due
>>> to some bugs: ticket #1
>>> <https://ghc.haskell.org/trac/ghc/ticket/14472#no2>, ticker #2
>>> <https://ghc.haskell.org/trac/ghc/ticket/14784#no1>). The C++ project
>>> was built with MSVC compiler 2015.
>>>
>>> We noticed that memory is not released when the library is unloaded.
>>> I've constructed a baseline example to reproduce the bug. It has three
>>> files: the first one is *HaskellSources.hs*(see attachments). It
>>> contains one foreign export function foo. This foreign export function is
>>> necessary to reproduce the bug. The second file is *CWrapper.cpp*. I
>>> intentionally didn't include any Haskell function export because they make
>>> no difference for this example. The last file is the code of the main
>>> program (see *main.cpp*) that loads the library and unloads it at once
>>> in an endless loop.
>>>
>>> There are two cases: whether HaskellExports.o is included into the
>>> library or not.
>>> 1. HaskellExports.o is not included into the library (see attached build
>>> script* build_no_hs.sh*). Everything works just fine (i.e. amount of
>>> memory consumed by the app is the same before DLL loading and right after
>>> unloading.)
>>> 2. HaskellExports.o is included into the library (see *build_w_hs.sh*).
>>> Memory is not released after unloading the DLL, after repeated load/unload
>>> cycles memory consumption keeps growing until finally the application
>>> crashes.
>>>
>>> Is this a known problem? Is it tracked in your bugtracker? (Or maybe it
>>> is not a problem at all and I'm doing it wrong).
>>>
>>> Thank you for your time.
>>>
>>> --
>>> * Olga Philippskaya .*
>>>
>>>
>>> _______________________________________________
>>> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>>
>>
>> --
>> *Филиппская Ольга.*
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20180827/4f0a7343/attachment.html>