Haskell DLL (using FFI) causes memory leak.

[Phyx]

Hi,

You're likely just hitting a memory leak, Haskell DLLs and the RTS aren't
designed to be unload-able, which is why you can't call hs_init again after
you stop.

The leak happens only when you do a foreign export because foreign exports
create C wrappers to initializers for each function you export.
https://github.com/ghc/ghc/blob/21f0f56164f50844c2150c62f950983b2376f8b6/compiler/deSugar/DsForeign.hs#L668

These are marked as static initializers, and as such the CRT will
initialize them on DLL_PROCESS_ATTACH time. At DLL_PROCESS_DETACH
time the destructors would be run, but we don't have destructors for these
initializers, so whatever they did will always persist.

Your use case is fairly uncommon, but file a bug report against the leaks
anyway. Why do you need to keep loading and unloading the dll?

Note that FreeLibrary like dlclose on unix does not guarantee that the
library is freed, it just decrements the reference count. and when it
reaches 0 it will *eventually* unload it. Notice that if you use
UnmapViewOfFile instead of FreeLibrary the leak doesn't happen as that
*actually* immediately unmaps the image.  But this will get you in all
sorts of trouble later on when terminating (you'll likely segfault at that
point).

Things will get worse when you actually do call hs_init, because hs_init
will call initLinker which will load libraries of it's own. Unloading the
top level
does not decrease the reference counts of the dependencies, and we do no
extra work to ensure this.

If you really need to load/reload libraries, then I fear your only choice
is some kind of process isolation and communicating back using share
memory/pipes
or some sort of IPC.

Regards,
Tamar

On Mon, Aug 27, 2018 at 4:33 PM Ольга Филиппская <olga.kenobi at gmail.com>
wrote:

> It's true that one must initialize the runtime when calling Haskell from
> C/C++. We do this in the real project and just a pair of calls
> hs_init/hs_exit causes memory to be leaked even faster. But I tried to
> construct the minimal example to describe the bug, so I reduced all the
> extra-code.
>
> пн, 27 авг. 2018 г. в 18:00, Vanessa McHale <vanessa.mchale at iohk.io>:
>
>> I don't know about C++, but I do know that when calling Haskell from C
>> code one must initialize the runtime and then exit it once finished. I also
>> know that one must only initialize the runtime once over the course of the
>> program's run. As far as I can tell, this does not occur in your example.
>>
>> On 08/27/2018 09:44 AM, Ольга Филиппская wrote:
>>
>> Hello.
>>
>> *Summary: *memory is consumed without releasing when a Haskell DLL (that
>> uses FFI) is loaded and unloaded in an endless loop.
>>
>> *Details*: I'm working on a C++ project relying on a DLL that uses
>> Haskell code. The DLL was built with GHC 8.0.1 x64, OS is Windows 7. (Newer
>> versions of GHC - 8.2.1 and later - do not allow you to build such DLLs due
>> to some bugs: ticket #1
>> <https://ghc.haskell.org/trac/ghc/ticket/14472#no2>, ticker #2
>> <https://ghc.haskell.org/trac/ghc/ticket/14784#no1>). The C++ project
>> was built with MSVC compiler 2015.
>>
>> We noticed that memory is not released when the library is unloaded. I've
>> constructed a baseline example to reproduce the bug. It has three files:
>> the first one is *HaskellSources.hs*(see attachments). It contains one
>> foreign export function foo. This foreign export function is necessary to
>> reproduce the bug. The second file is *CWrapper.cpp*. I intentionally
>> didn't include any Haskell function export because they make no difference
>> for this example. The last file is the code of the main program (see
>> *main.cpp*) that loads the library and unloads it at once in an endless
>> loop.
>>
>> There are two cases: whether HaskellExports.o is included into the
>> library or not.
>> 1. HaskellExports.o is not included into the library (see attached build
>> script* build_no_hs.sh*). Everything works just fine (i.e. amount of
>> memory consumed by the app is the same before DLL loading and right after
>> unloading.)
>> 2. HaskellExports.o is included into the library (see *build_w_hs.sh*).
>> Memory is not released after unloading the DLL, after repeated load/unload
>> cycles memory consumption keeps growing until finally the application
>> crashes.
>>
>> Is this a known problem? Is it tracked in your bugtracker? (Or maybe it
>> is not a problem at all and I'm doing it wrong).
>>
>> Thank you for your time.
>>
>> --
>> * Olga Philippskaya .*
>>
>>
>> _______________________________________________
>> ghc-devs mailing listghc-devs at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
>
> --
> *Филиппская Ольга.*
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20180827/c941eedb/attachment.html>