From ben at well-typed.com Sat Aug 8 19:45:08 2020 From: ben at well-typed.com (Ben Gamari) Date: Sat, 08 Aug 2020 15:45:08 -0400 Subject: [ANNOUNCE] Glasgow Haskell Compiler 8.10.2 released Message-ID: <874kpcexbz.fsf@smart-cactus.org> Hello all, The GHC team is happy to announce the release of GHC 8.10.2. Source and binary distributions are available at the usual place: https://downloads.haskell.org/ghc/8.10.2/ GHC 8.10.2 fixes a number of issues in present in GHC 8.10.1 including: * Fixes a bug in process creation on Windows (#17926). * Fixes a code generation bug resulting in incorrect code generation for foreign calls with complex arguments (#18527) * Fixes a bug causing object merging to fail when the lld linker is in use (#17962) * Introduces a workaround for a Linux kernel bug in the implementation of the timerfd mechanism (#18033). * Fixes a few specialiser regressions (#17810, #18120) as well introduces a variety of miscellaneous specialiser improvements (#16473, #17930, #17966) * Fixes a potential loss of sharing due to left operator sections (#18151). * Fix bootstrapping of GHC with the LLVM backend on x86-64 (#17920). * A few important correctness fixes for the low-latency garbage collector. Users of `--nonmoving-gc` are strongly encouraged to upgrade promptly. Note that at the moment we still require that macOS Catalina users exempt the binary distribution from the notarization requirement by running `xattr -cr .` on the unpacked tree before running `make install`. This situation will hopefully be improved for GHC 9.0.1 with the resolution of #17418 [1]. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/issues/17418 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From post at volker-wysk.de Sun Aug 9 12:50:30 2020 From: post at volker-wysk.de (Volker Wysk) Date: Sun, 09 Aug 2020 14:50:30 +0200 Subject: Linking completely statically Message-ID: <1ae26f0c1cb8fdcefe97eddf1d61789e091f6cb1.camel@volker-wysk.de> Hi! I know of the command line argument "-static". But this only affects the Haskell libraries. I want to link some programs completely statically, no external libraries needed. When just linking with "-static" I still have those dynamically linked things: desktop ~/bin $ ldd sicherung linux-vdso.so.1 (0x00007ffdab53f000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f3633da0000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f3633d95000) libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f3633d90000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f3633d8a000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f3633d67000) libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f3633ce3000) libatomic.so.1 => /lib/x86_64-linux-gnu/libatomic.so.1 (0x00007f3633cd7000) libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 (0x00007f3633ccb000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3633ad9000) /lib64/ld-linux-x86-64.so.2 (0x00007f3633f0c000) Is it possible to link the remaining libraries statically too? Regards, Volker -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: This is a digitally signed message part URL: From allbery.b at gmail.com Sun Aug 9 12:59:24 2020 From: allbery.b at gmail.com (Brandon Allbery) Date: Sun, 9 Aug 2020 08:59:24 -0400 Subject: Linking completely statically In-Reply-To: <1ae26f0c1cb8fdcefe97eddf1d61789e091f6cb1.camel@volker-wysk.de> References: <1ae26f0c1cb8fdcefe97eddf1d61789e091f6cb1.camel@volker-wysk.de> Message-ID: Linux is not friendly to static linking, and you would need to either package or match exact versions of things like the nss and locale libraries that are dynamically loaded at runtime and don't show up in ldd. This limitation comes from glibc and is documented in its manual. On Sun, Aug 9, 2020, 08:50 Volker Wysk wrote: > Hi! > > I know of the command line argument "-static". But this only affects > the Haskell libraries. I want to link some programs completely > statically, no external libraries needed. > > When just linking with "-static" I still have those dynamically linked > things: > > desktop ~/bin $ ldd sicherung > linux-vdso.so.1 (0x00007ffdab53f000) > libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 > (0x00007f3633da0000) > librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 > (0x00007f3633d95000) > libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 > (0x00007f3633d90000) > libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 > (0x00007f3633d8a000) > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 > (0x00007f3633d67000) > libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 > (0x00007f3633ce3000) > libatomic.so.1 => /lib/x86_64-linux-gnu/libatomic.so.1 > (0x00007f3633cd7000) > libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 > (0x00007f3633ccb000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 > (0x00007f3633ad9000) > /lib64/ld-linux-x86-64.so.2 (0x00007f3633f0c000) > > > Is it possible to link the remaining libraries statically too? > > Regards, > Volker > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From post at volker-wysk.de Sun Aug 9 13:11:46 2020 From: post at volker-wysk.de (Volker Wysk) Date: Sun, 09 Aug 2020 15:11:46 +0200 Subject: Linking completely statically In-Reply-To: References: <1ae26f0c1cb8fdcefe97eddf1d61789e091f6cb1.camel@volker-wysk.de> Message-ID: <90ddffaa54fb7d53c634d51b45ca631d66cfa579.camel@volker-wysk.de> Am Sonntag, den 09.08.2020, 08:59 -0400 schrieb Brandon Allbery: > Linux is not friendly to static linking, and you would need to either > package or match exact versions of things like the nss and locale > libraries that are dynamically loaded at runtime and don't show up in > ldd. This limitation comes from glibc and is documented in its > manual. That's bad. But, like you said, it *is* possible. The dar (disk archiver) program comes with a statically linked version "dar_static": % ldd dar_static Das Programm ist nicht dynamisch gelinkt ("The program isn't linked dynamically.") From your answer, I assume that this isn't supported by GHC. Cheers Volker > > On Sun, Aug 9, 2020, 08:50 Volker Wysk wrote: > > Hi! > > > > I know of the command line argument "-static". But this only > > affects > > the Haskell libraries. I want to link some programs completely > > statically, no external libraries needed. > > > > When just linking with "-static" I still have those dynamically > > linked > > things: > > > > desktop ~/bin $ ldd sicherung > > linux-vdso.so.1 (0x00007ffdab53f000) > > libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 > > (0x00007f3633da0000) > > librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 > > (0x00007f3633d95000) > > libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 > > (0x00007f3633d90000) > > libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 > > (0x00007f3633d8a000) > > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 > > (0x00007f3633d67000) > > libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 > > (0x00007f3633ce3000) > > libatomic.so.1 => /lib/x86_64-linux-gnu/libatomic.so.1 > > (0x00007f3633cd7000) > > libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 > > (0x00007f3633ccb000) > > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 > > (0x00007f3633ad9000) > > /lib64/ld-linux-x86-64.so.2 (0x00007f3633f0c000) > > > > > > Is it possible to link the remaining libraries statically too? > > > > Regards, > > Volker > > _______________________________________________ > > Glasgow-haskell-users mailing list > > Glasgow-haskell-users at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: This is a digitally signed message part URL: From post at volker-wysk.de Sun Aug 9 17:12:52 2020 From: post at volker-wysk.de (Volker Wysk) Date: Sun, 09 Aug 2020 19:12:52 +0200 Subject: Linking completely statically In-Reply-To: References: <1ae26f0c1cb8fdcefe97eddf1d61789e091f6cb1.camel@volker-wysk.de> <90ddffaa54fb7d53c634d51b45ca631d66cfa579.camel@volker-wysk.de> Message-ID: Am Sonntag, den 09.08.2020, 09:53 -0400 schrieb Brandon Allbery: > If you are obeying all the necessary restrictions on glibc, or you > are > using a different libc such as musl which is designed for static > linking, then "-Wl,-static" should be sufficient. Doing this with > glibc will likely result in a crash unless the same version of glibc > is available at runtime. Thanks for the tip! But I'll let it be for now. Cheers, Volker > > On 8/9/20, Volker Wysk wrote: > > Am Sonntag, den 09.08.2020, 08:59 -0400 schrieb Brandon Allbery: > > > Linux is not friendly to static linking, and you would need to > > > either > > > package or match exact versions of things like the nss and locale > > > libraries that are dynamically loaded at runtime and don't show > > > up in > > > ldd. This limitation comes from glibc and is documented in its > > > manual. > > > > That's bad. But, like you said, it *is* possible. The dar (disk > > archiver) program comes with a statically linked version > > "dar_static": > > > > % ldd dar_static > > Das Programm ist nicht dynamisch gelinkt > > > > ("The program isn't linked dynamically.") > > > > From your answer, I assume that this isn't supported by GHC. > > > > Cheers > > Volker > > > > > > > On Sun, Aug 9, 2020, 08:50 Volker Wysk > > > wrote: > > > > Hi! > > > > > > > > I know of the command line argument "-static". But this only > > > > affects > > > > the Haskell libraries. I want to link some programs completely > > > > statically, no external libraries needed. > > > > > > > > When just linking with "-static" I still have those dynamically > > > > linked > > > > things: > > > > > > > > desktop ~/bin $ ldd sicherung > > > > linux-vdso.so.1 (0x00007ffdab53f000) > > > > libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 > > > > (0x00007f3633da0000) > > > > librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 > > > > (0x00007f3633d95000) > > > > libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 > > > > (0x00007f3633d90000) > > > > libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 > > > > (0x00007f3633d8a000) > > > > libpthread.so.0 => /lib/x86_64-linux- > > > > gnu/libpthread.so.0 > > > > (0x00007f3633d67000) > > > > libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 > > > > (0x00007f3633ce3000) > > > > libatomic.so.1 => /lib/x86_64-linux-gnu/libatomic.so.1 > > > > (0x00007f3633cd7000) > > > > libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 > > > > (0x00007f3633ccb000) > > > > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 > > > > (0x00007f3633ad9000) > > > > /lib64/ld-linux-x86-64.so.2 (0x00007f3633f0c000) > > > > > > > > > > > > Is it possible to link the remaining libraries statically too? > > > > > > > > Regards, > > > > Volker > > > > _______________________________________________ > > > > Glasgow-haskell-users mailing list > > > > Glasgow-haskell-users at haskell.org > > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: This is a digitally signed message part URL: From spam at scientician.net Mon Aug 10 07:11:17 2020 From: spam at scientician.net (Bardur Arantsson) Date: Mon, 10 Aug 2020 09:11:17 +0200 Subject: Linking completely statically In-Reply-To: <1ae26f0c1cb8fdcefe97eddf1d61789e091f6cb1.camel@volker-wysk.de> References: <1ae26f0c1cb8fdcefe97eddf1d61789e091f6cb1.camel@volker-wysk.de> Message-ID: On 09/08/2020 14.50, Volker Wysk wrote: > Hi! > > I know of the command line argument "-static". But this only affects > the Haskell libraries. I want to link some programs completely > statically, no external libraries needed. > > When just linking with "-static" I still have those dynamically linked > things: > > desktop ~/bin $ ldd sicherung > linux-vdso.so.1 (0x00007ffdab53f000) > libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 > (0x00007f3633da0000) [--snip--] > libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 > (0x00007f3633ce3000) > libatomic.so.1 => /lib/x86_64-linux-gnu/libatomic.so.1 > (0x00007f3633cd7000) > libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 > (0x00007f3633ccb000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 > (0x00007f3633ad9000) > /lib64/ld-linux-x86-64.so.2 (0x00007f3633f0c000) > > > Is it possible to link the remaining libraries statically too? > As Brandon already mentioned, linking glibc statically is not really supported (by glibc!), but what you can do is bundle a few of the .so's together with your binary and use RPATH to help your executable find them at load time (well, dynamic link time). A project of ours where we do this is using Stack, so I can only really provide info on doing it with Stack, but hopefully you can adapt to whatever you're using: We added ld-options: - -Wl,-rpath,$ORIGIN/../lib to the 'executable' portion of the package.yaml. This instructs the dynamic loader to look for libraries in '../lib' relative to the executable's location. You can pick whatever path you want there. Once you have that bit set up, you can copy *most* of the libraries .so's you're using to that folder on the install target and they'll be loaded from there. (Copying libc.so is ill advised, but most of the others will work just fine. I believe ld-linux-... is also never ever loaded from there since that *is* the dynamic loader.) (Of course a possibly more robust version of this is to just bundle everything into a container of some sort, but that may be a bit excessive for your needs.) HTH, From post at volker-wysk.de Mon Aug 10 12:47:55 2020 From: post at volker-wysk.de (Volker Wysk) Date: Mon, 10 Aug 2020 14:47:55 +0200 Subject: Linking completely statically In-Reply-To: References: <1ae26f0c1cb8fdcefe97eddf1d61789e091f6cb1.camel@volker-wysk.de> Message-ID: <15eda32e1f6c3cdb4ec7ba7fc76b035fedb7d056.camel@volker-wysk.de> Am Montag, den 10.08.2020, 09:11 +0200 schrieb Bardur Arantsson: > On 09/08/2020 14.50, Volker Wysk wrote: > > Hi! > > > > I know of the command line argument "-static". But this only > > affects > > the Haskell libraries. I want to link some programs completely > > statically, no external libraries needed. > > > > When just linking with "-static" I still have those dynamically > > linked > > things: > > > > desktop ~/bin $ ldd sicherung > > linux-vdso.so.1 (0x00007ffdab53f000) > > libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 > > (0x00007f3633da0000) > [--snip--] > > libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 > > (0x00007f3633ce3000) > > libatomic.so.1 => /lib/x86_64-linux-gnu/libatomic.so.1 > > (0x00007f3633cd7000) > > libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 > > (0x00007f3633ccb000) > > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 > > (0x00007f3633ad9000) > > /lib64/ld-linux-x86-64.so.2 (0x00007f3633f0c000) > > > > > > Is it possible to link the remaining libraries statically too? > > > > As Brandon already mentioned, linking glibc statically is not really > supported (by glibc!), but what you can do is bundle a few of the > .so's > together with your binary and use RPATH to help your executable find > them at load time (well, dynamic link time). > > A project of ours where we do this is using Stack, so I can only > really > provide info on doing it with Stack, but hopefully you can adapt to > whatever you're using: > > We added > > ld-options: > - -Wl,-rpath,$ORIGIN/../lib > > to the 'executable' portion of the package.yaml. This instructs the > dynamic loader to look for libraries in '../lib' relative to the > executable's location. You can pick whatever path you want there. > > Once you have that bit set up, you can copy *most* of the libraries > .so's you're using to that folder on the install target and they'll > be > loaded from there. (Copying libc.so is ill advised, but most of the > others will work just fine. I believe ld-linux-... is also never ever > loaded from there since that *is* the dynamic loader.) > > (Of course a possibly more robust version of this is to just bundle > everything into a container of some sort, but that may be a bit > excessive for your needs.) Thank you for your elaborate explanation. This would be overkill for what I have in mind. I'm just trying to save a statically linked copy of a program of mine, which is involved in my backup, along with the backup, and also on a maintenance USB stick. I've stowed you answer away in my personal wiki, and may get back to it later. Regards, Volker -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: This is a digitally signed message part URL: From twhitehead at gmail.com Mon Aug 10 21:22:41 2020 From: twhitehead at gmail.com (Tyson Whitehead) Date: Mon, 10 Aug 2020 17:22:41 -0400 Subject: Linking completely statically In-Reply-To: References: <1ae26f0c1cb8fdcefe97eddf1d61789e091f6cb1.camel@volker-wysk.de> Message-ID: On Mon, 10 Aug 2020 at 03:12, Bardur Arantsson wrote: > Once you have that bit set up, you can copy *most* of the libraries > .so's you're using to that folder on the install target and they'll be > loaded from there. (Copying libc.so is ill advised, but most of the > others will work just fine. I believe ld-linux-... is also never ever > loaded from there since that *is* the dynamic loader.) Not 100% certain, but I think the issue with libc may be that it may be somewhat tied to the ld-linux. You can actually copy everything, including ld-linux, though. You then invoke the program via ld-linux (it's actually an executable -- see the ld.so page) ./ld-linux-x86_64.so.2 --library-path ... If you wrap your binaries with a shell script it is actually pretty clean (make sure to exec so you don't windup with the shell sitting between parent and childs that can cause some subtle issues) The only real breakage I've seen from this is when you try and run on too old of a kernel for your libc or the remote system has an unusual nsswitch plugin you didn't provide. -------------- next part -------------- An HTML attachment was scrubbed... URL: From spam at scientician.net Tue Aug 11 04:18:19 2020 From: spam at scientician.net (Bardur Arantsson) Date: Tue, 11 Aug 2020 06:18:19 +0200 Subject: Linking completely statically In-Reply-To: References: <1ae26f0c1cb8fdcefe97eddf1d61789e091f6cb1.camel@volker-wysk.de> Message-ID: On 10/08/2020 23.22, Tyson Whitehead wrote: > On Mon, 10 Aug 2020 at 03:12, Bardur Arantsson > wrote: > > Once you have that bit set up, you can copy *most* of the libraries > ..so's you're using to that folder on the install target and they'll be > loaded from there. (Copying libc.so is ill advised, but most of the > others will work just fine. I believe ld-linux-... is also never ever > loaded from there since that *is* the dynamic loader.) > > > Not 100% certain, but I think the issue with libc may be that it may be > somewhat tied to the ld-linux. > > You can actually copy everything, including ld-linux, though. You then > invoke the program via ld-linux (it's actually an executable -- see the > ld..so page) > > ./ld-linux-x86_64.so.2 --library-path > ... > Hm, yes I was vaguely aware of this ability to invoke the loader directly, but IIRC the most immediate problem with glibc has always been that the nsswitch.conf plugins (which are always dynamically loaded) are strictly tied to the exact gblic version. I guess if you copy them too, it should work, but the issue is that there may be ones on the user's system which aren't present on the CI/build system... > If you wrap your binaries with a shell script it is actually pretty > clean (make sure to exec so you don't windup with the shell sitting > between parent and childs that can cause some subtle issues) Yes, this is also a good way to do it and doesn't require messing with the build setup. > > The only real breakage I've seen from this is when you try and run on > too old of a kernel for your libc or the remote system has an unusual > nsswitch plugin you didn't provide. Ack Regards, From andreas.herrmann at tweag.io Tue Aug 11 08:26:20 2020 From: andreas.herrmann at tweag.io (Herrmann, Andreas) Date: Tue, 11 Aug 2020 10:26:20 +0200 Subject: Linking completely statically Message-ID: Hi Volker, > Is it possible to link the remaining libraries statically too? > Yes, it is possible to generate fully statically linked Haskell binaries. Though it requires a bit of setup. For example the GNU C library glibc is not really intended for fully static linking, but you can use musl as an alternative libc instead. Probably the easiest way is to use static-haskell-nix [1]. Usage instructions are available in the project README. See [2] if you're not familiar with Nix. Recently, the Haskell extension to Bazel, rules_haskell, also gained the ability to generate fully statically linked binaries building on top of Nix, see [3]. Best, Andreas [1]: https://github.com/nh2/static-haskell-nix [2]: https://nixos.org/ [3]: https://rules-haskell.readthedocs.io/en/latest/haskell-use-cases.html#building-fully-statically-linked-binaries -------------- next part -------------- An HTML attachment was scrubbed... URL: From post at volker-wysk.de Tue Aug 11 12:58:58 2020 From: post at volker-wysk.de (Volker Wysk) Date: Tue, 11 Aug 2020 14:58:58 +0200 Subject: Linking completely statically In-Reply-To: References: Message-ID: <918f3b244092f8a6c326898ef2cf261e8aca8d1a.camel@volker-wysk.de> Am Dienstag, den 11.08.2020, 10:26 +0200 schrieb Herrmann, Andreas: > Hi Volker, Hi! > > Is it possible to link the remaining libraries statically too? > > Yes, it is possible to generate fully statically linked Haskell > binaries. Though it requires a bit of setup. For example the GNU C > library glibc is not really intended for fully static linking, but > you can use musl as an alternative libc instead. > > Probably the easiest way is to use static-haskell-nix [1]. Usage > instructions are available in the project README. See [2] if you're > not familiar with Nix. This looks complicated, even though it is the easiest way. I've tried to build it from the git sources, as well as from the latest release, but that failed. I'd have to look into nix, which is new to me. Be it as it may, it isn't that important for me right now. But thank you very much for your tips. Maybe I'll be going back to them later. Cheers, Volker > Recently, the Haskell extension to Bazel, rules_haskell, also gained > the ability to generate fully statically linked binaries building on > top of Nix, see [3]. > > Best, Andreas > > [1]: https://github.com/nh2/static-haskell-nix > [2]: https://nixos.org/ > [3]: > https://rules-haskell.readthedocs.io/en/latest/haskell-use-cases.html#building-fully-statically-linked-binaries -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: This is a digitally signed message part URL: From iricanaycan at gmail.com Tue Aug 11 14:32:13 2020 From: iricanaycan at gmail.com (Aycan iRiCAN) Date: Tue, 11 Aug 2020 07:32:13 -0700 Subject: Linking completely statically In-Reply-To: <918f3b244092f8a6c326898ef2cf261e8aca8d1a.camel@volker-wysk.de> References: <918f3b244092f8a6c326898ef2cf261e8aca8d1a.camel@volker-wysk.de> Message-ID: <4425A2C9-A6D5-47A7-82AA-8073A582BA7F@gmail.com> Hi Volker, You may also want to check out ghc-musl project https://github.com/utdemir/ghc-musl which provides compiled docker images to build atatic executables for various ghc versions. Cheers, -- aycan > On Aug 11, 2020, at 5:59 AM, Volker Wysk wrote: > > Am Dienstag, den 11.08.2020, 10:26 +0200 schrieb Herrmann, Andreas: >> Hi Volker, > > Hi! > >>> Is it possible to link the remaining libraries statically too? >> >> Yes, it is possible to generate fully statically linked Haskell >> binaries. Though it requires a bit of setup. For example the GNU C >> library glibc is not really intended for fully static linking, but >> you can use musl as an alternative libc instead. >> >> Probably the easiest way is to use static-haskell-nix [1]. Usage >> instructions are available in the project README. See [2] if you're >> not familiar with Nix. > > This looks complicated, even though it is the easiest way. I've tried > to build it from the git sources, as well as from the latest release, > but that failed. I'd have to look into nix, which is new to me. > > Be it as it may, it isn't that important for me right now. > > But thank you very much for your tips. Maybe I'll be going back to them > later. > > > Cheers, > Volker > >> Recently, the Haskell extension to Bazel, rules_haskell, also gained >> the ability to generate fully statically linked binaries building on >> top of Nix, see [3]. >> >> Best, Andreas >> >> [1]: https://github.com/nh2/static-haskell-nix >> [2]: https://nixos.org/ >> [3]: >> https://rules-haskell.readthedocs.io/en/latest/haskell-use-cases.html#building-fully-statically-linked-binaries > > > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.dubuisson at gmail.com Tue Aug 11 15:38:20 2020 From: thomas.dubuisson at gmail.com (Thomas DuBuisson) Date: Tue, 11 Aug 2020 08:38:20 -0700 Subject: Linking completely statically In-Reply-To: <1ae26f0c1cb8fdcefe97eddf1d61789e091f6cb1.camel@volker-wysk.de> References: <1ae26f0c1cb8fdcefe97eddf1d61789e091f6cb1.camel@volker-wysk.de> Message-ID: I was able to get static linking working recently using docker alpine images and ghcup to install GHC based on the musl library. The details are in my Stan fork [1]. This borrowed heavily from ShellCheck's static linking release system except it uses cabal v2-build instead of v1. [1] Script: https://github.com/TomMD/stan/blob/feature/basic-json-output/mkRelease.sh Dockerfile: https://github.com/TomMD/stan/blob/feature/basic-json-output/Dockerfile On Sun, Aug 9, 2020 at 5:51 AM Volker Wysk wrote: > Hi! > > I know of the command line argument "-static". But this only affects > the Haskell libraries. I want to link some programs completely > statically, no external libraries needed. > > When just linking with "-static" I still have those dynamically linked > things: > > desktop ~/bin $ ldd sicherung > linux-vdso.so.1 (0x00007ffdab53f000) > libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 > (0x00007f3633da0000) > librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 > (0x00007f3633d95000) > libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 > (0x00007f3633d90000) > libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 > (0x00007f3633d8a000) > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 > (0x00007f3633d67000) > libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 > (0x00007f3633ce3000) > libatomic.so.1 => /lib/x86_64-linux-gnu/libatomic.so.1 > (0x00007f3633cd7000) > libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 > (0x00007f3633ccb000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 > (0x00007f3633ad9000) > /lib64/ld-linux-x86-64.so.2 (0x00007f3633f0c000) > > > Is it possible to link the remaining libraries statically too? > > Regards, > Volker > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From post at volker-wysk.de Wed Aug 12 14:15:46 2020 From: post at volker-wysk.de (Volker Wysk) Date: Wed, 12 Aug 2020 16:15:46 +0200 Subject: Linking completely statically In-Reply-To: <1ae26f0c1cb8fdcefe97eddf1d61789e091f6cb1.camel@volker-wysk.de> References: <1ae26f0c1cb8fdcefe97eddf1d61789e091f6cb1.camel@volker-wysk.de> Message-ID: <0e4071660dc959dcab8e9e0c46187c0ff4ef8a19.camel@volker-wysk.de> Hi Thank you very much to Brandon, Tyson, Bardur, Thomas, Andreas, Aycan for your explanations. I thought that it probably was just another GHC argument or something like that. But it sounds quite complicated. What I have in mind isn't worth the trouble. I just wanted to have a statically linked version for inclusion in my backup and in my maintenance USB stick. Perhaps someone with insight could compile the answers to my post into a page for the Haskell Wiki. When I should really need it, I'll come back to this thread later. Cheers, Volker -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 195 bytes Desc: This is a digitally signed message part URL: From trebla at vex.net Wed Aug 12 16:24:20 2020 From: trebla at vex.net (Albert Y. C. Lai) Date: Wed, 12 Aug 2020 12:24:20 -0400 Subject: Linking completely statically In-Reply-To: <0e4071660dc959dcab8e9e0c46187c0ff4ef8a19.camel@volker-wysk.de> References: <1ae26f0c1cb8fdcefe97eddf1d61789e091f6cb1.camel@volker-wysk.de> <0e4071660dc959dcab8e9e0c46187c0ff4ef8a19.camel@volker-wysk.de> Message-ID: On 2020-08-12 10:15 a.m., Volker Wysk wrote: > But it sounds quite complicated. What I have in mind isn't worth the > trouble. I just wanted to have a statically linked version for > inclusion in my backup and in my maintenance USB stick. I have an exe built by GHC in January 2014 (that's like GHC 7.6), and it is still running just fine on the latest Ubuntu. (In fact I have a daily cron job that runs it.) This is a data point for how long GHC default linking can last IRL. Historically my recollection of the only one breakage event in this regard was a libgmp version bump. And that was a bygone era, clearly before my vintage 2014 exe. From david.feuer at gmail.com Thu Aug 20 19:25:23 2020 From: david.feuer at gmail.com (David Feuer) Date: Thu, 20 Aug 2020 15:25:23 -0400 Subject: What does freezing an array really do? Message-ID: I know that a frozen array doesn't have to be searched for elements in a younger generation, but how does it differ from an unfrozen array that hasn't been mutated since the last collection? David From bertram.felgenhauer at googlemail.com Thu Aug 20 22:12:07 2020 From: bertram.felgenhauer at googlemail.com (Bertram Felgenhauer) Date: Fri, 21 Aug 2020 00:12:07 +0200 Subject: What does freezing an array really do? In-Reply-To: References: Message-ID: <20200820221140.GA37589@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> David Feuer wrote: > I know that a frozen array doesn't have to be searched for elements in > a younger generation, but how does it differ from an unfrozen array > that hasn't been mutated since the last collection? Frozen arrays are not put on the mutable list once they're clean (meaning they have no references to younger generations). Thawed arrays are always on the mutable list. Cheers, Bertram You can crash a program by updating a clean frozen array in the old generation, then doing a minor GC, and then accessing the updated entry: import Data.Array import Data.Array.Base (unsafeFreezeIOArray, unsafeThawIOArray) import Data.Array.IO import Data.Array.Unsafe import System.Mem import Control.Monad (forM_) main = do arr <- newArray (0, 1024) 42 :: IO (IOArray Int Int) -- `arr` points to a MUT_ARR_PTRS closure arr' <- unsafeFreezeIOArray arr :: IO (Array Int Int) -- unsafeFreezeIOArray changes the closure type to -- MUT_ARR_PTRS_FROZEN_DIRTY performMajorGC -- the first major GC changes it to MUT_ARR_PTRS_FROZEN performMajorGC -- the second GC kicks it off the mutable list?? -- unsafeThaw would change the closure type to MUT_ARR_PTRS_DIRTY -- *and* put the array on the mutable list. -- _ <- unsafeThawIOArray arr' -- in contrast, `writeArray` changes the closure type to -- MUT_ARR_PTRS_DIRTY but does not touch the `mutable list` forM_ [0..1023] $ \i -> do writeArray arr i $ f i print $ sum [1..1000000] -- Evaluating the sum produces a lot of garbage, and the minor GCs -- will not scan the array. arr' <- unsafeFreezeIOArray arr :: IO (Array Int Int) print $ arr' ! 0 print $ arr' ! 1 -- so these array entries have a good chance to point to utter garbage. {-# NOINLINE f #-} f x = x*x {- > ghc -O0 Crash.hs && ./Crash [1 of 1] Compiling Main ( Crash.hs, Crash.o ) Linking Crash ... Crash: internal error: evacuate: strange closure type 29041 (GHC version 8.6.5 for x86_64_unknown_linux) Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug Aborted (core dumped) > ghc -O1 Crash.hs && ./Crash [1 of 1] Compiling Main ( Crash.hs, Crash.o ) Linking Crash ... 500000500000 482299980870 Segmentation fault (core dumped) > ghc -O2 Crash.hs && ./Crash [1 of 1] Compiling Main ( Crash.hs, Crash.o ) [Optimisation flags changed] Linking Crash ... 500000500000 482299980870 Segmentation fault (core dumped) -} From david.feuer at gmail.com Thu Aug 20 23:08:06 2020 From: david.feuer at gmail.com (David Feuer) Date: Thu, 20 Aug 2020 19:08:06 -0400 Subject: What does freezing an array really do? In-Reply-To: <20200820221140.GA37589@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> References: <20200820221140.GA37589@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> Message-ID: So I guess this is to avoid having to check the closure type on each mutation to see if the array needs to be added to the mutable list? On Thu, Aug 20, 2020, 6:12 PM Bertram Felgenhauer via Glasgow-haskell-users wrote: > David Feuer wrote: > > I know that a frozen array doesn't have to be searched for elements in > > a younger generation, but how does it differ from an unfrozen array > > that hasn't been mutated since the last collection? > > Frozen arrays are not put on the mutable list once they're clean > (meaning they have no references to younger generations). Thawed > arrays are always on the mutable list. > > Cheers, > > Bertram > > > You can crash a program by updating a clean frozen array in the old > generation, then doing a minor GC, and then accessing the updated > entry: > > import Data.Array > import Data.Array.Base (unsafeFreezeIOArray, unsafeThawIOArray) > import Data.Array.IO > import Data.Array.Unsafe > import System.Mem > import Control.Monad (forM_) > > main = do > arr <- newArray (0, 1024) 42 :: IO (IOArray Int Int) > -- `arr` points to a MUT_ARR_PTRS closure > > arr' <- unsafeFreezeIOArray arr :: IO (Array Int Int) > -- unsafeFreezeIOArray changes the closure type to > -- MUT_ARR_PTRS_FROZEN_DIRTY > > performMajorGC > -- the first major GC changes it to MUT_ARR_PTRS_FROZEN > > performMajorGC > -- the second GC kicks it off the mutable list?? > > -- unsafeThaw would change the closure type to MUT_ARR_PTRS_DIRTY > -- *and* put the array on the mutable list. > -- _ <- unsafeThawIOArray arr' > > -- in contrast, `writeArray` changes the closure type to > -- MUT_ARR_PTRS_DIRTY but does not touch the `mutable list` > forM_ [0..1023] $ \i -> do > writeArray arr i $ f i > > print $ sum [1..1000000] > -- Evaluating the sum produces a lot of garbage, and the minor GCs > -- will not scan the array. > > arr' <- unsafeFreezeIOArray arr :: IO (Array Int Int) > print $ arr' ! 0 > print $ arr' ! 1 > -- so these array entries have a good chance to point to utter garbage. > > {-# NOINLINE f #-} > f x = x*x > > {- > > > ghc -O0 Crash.hs && ./Crash > [1 of 1] Compiling Main ( Crash.hs, Crash.o ) > Linking Crash ... > Crash: internal error: evacuate: strange closure type 29041 > (GHC version 8.6.5 for x86_64_unknown_linux) > Please report this as a GHC bug: > http://www.haskell.org/ghc/reportabug > Aborted (core dumped) > > > ghc -O1 Crash.hs && ./Crash > [1 of 1] Compiling Main ( Crash.hs, Crash.o ) > Linking Crash ... > 500000500000 > 482299980870 > Segmentation fault (core dumped) > > > ghc -O2 Crash.hs && ./Crash > [1 of 1] Compiling Main ( Crash.hs, Crash.o ) [Optimisation > flags changed] > Linking Crash ... > 500000500000 > 482299980870 > Segmentation fault (core dumped) > > -} > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben at well-typed.com Fri Aug 21 01:47:51 2020 From: ben at well-typed.com (Ben Gamari) Date: Thu, 20 Aug 2020 21:47:51 -0400 Subject: What does freezing an array really do? In-Reply-To: References: <20200820221140.GA37589@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> Message-ID: On August 20, 2020 7:08:06 PM EDT, David Feuer wrote: >So I guess this is to avoid having to check the closure type on each >mutation to see if the array needs to be added to the mutable list? > Correct. -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From ben at well-typed.com Mon Aug 24 18:01:17 2020 From: ben at well-typed.com (Ben Gamari) Date: Mon, 24 Aug 2020 14:01:17 -0400 Subject: Haskell Implementors' Workshop lightning talks Message-ID: <87h7srly8p.fsf@smart-cactus.org> Hello everyone, The Haskell Implementors' Workshop will be held this Friday, August 28th, colocated with virtual ICFP. As usual, we will have a number of slots for lightning-talks. These are five-minute-long talks where you can present libraries, language extension ideas, plugins, tools... anything that the Haskell implementor community might find interesting. We are collecting talk submissions via Google Forms [1] and will notify speakers of their slots on August 27th. We are looking forward to seeing you there! Cheers, - Ben [1] https://forms.gle/a8sZQNRRMe9H9YnX8 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 487 bytes Desc: not available URL: From david.feuer at gmail.com Wed Aug 26 17:20:51 2020 From: david.feuer at gmail.com (David Feuer) Date: Wed, 26 Aug 2020 13:20:51 -0400 Subject: Does newByteArray clear? Message-ID: I'm looking to play around with an array-based structure with sub-linear worst-case bounds. Array is pretty awkward in that context because creating a new one takes O(n) time to initialize it. Is that all true of newByteArray, or can I get one with arbitrary garbage in it for cheap? -------------- next part -------------- An HTML attachment was scrubbed... URL: From bertram.felgenhauer at googlemail.com Thu Aug 27 00:56:29 2020 From: bertram.felgenhauer at googlemail.com (Bertram Felgenhauer) Date: Thu, 27 Aug 2020 02:56:29 +0200 Subject: Does newByteArray clear? In-Reply-To: References: Message-ID: <20200827005629.GA24966@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> David Feuer wrote: > I'm looking to play around with an array-based structure with > sub-linear worst-case bounds. Array is pretty awkward in that context > because creating a new one takes O(n) time to initialize it. Is that > all true of newByteArray, or can I get one with arbitrary garbage in it > for cheap? newByteArray# does not actively clear memory. However, for large arrays, I think the memory is likely to be freshly allocated from the OS, and the OS will have cleared it for security reasons. Cheers, Bertram From david.feuer at gmail.com Thu Aug 27 01:02:28 2020 From: david.feuer at gmail.com (David Feuer) Date: Wed, 26 Aug 2020 21:02:28 -0400 Subject: Does newByteArray clear? In-Reply-To: <20200827005629.GA24966@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> References: <20200827005629.GA24966@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> Message-ID: Thanks! I'm still going to feel free to pretend I get arrays for free :-). I'm guessing I'll get some reused ones from the Haskell allocator, and the OS is of course free to do clearing work on another core. It'd be awfully nice to have a way to get "incrementally-cleared" arrays of pointers, but that would require a new heap object type, which would be a lot to ask for. On Wed, Aug 26, 2020, 8:56 PM Bertram Felgenhauer via Glasgow-haskell-users wrote: > David Feuer wrote: > > I'm looking to play around with an array-based structure with > > sub-linear worst-case bounds. Array is pretty awkward in that context > > because creating a new one takes O(n) time to initialize it. Is that > > all true of newByteArray, or can I get one with arbitrary garbage in it > > for cheap? > > newByteArray# does not actively clear memory. > > However, for large arrays, I think the memory is likely to be freshly > allocated from the OS, and the OS will have cleared it for security > reasons. > > Cheers, > > Bertram > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From spam at scientician.net Thu Aug 27 06:45:57 2020 From: spam at scientician.net (Bardur Arantsson) Date: Thu, 27 Aug 2020 08:45:57 +0200 Subject: Does newByteArray clear? In-Reply-To: <20200827005629.GA24966@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> References: <20200827005629.GA24966@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f> Message-ID: On 27/08/2020 02.56, Bertram Felgenhauer via Glasgow-haskell-users wrote: > David Feuer wrote: >> I'm looking to play around with an array-based structure with >> sub-linear worst-case bounds. Array is pretty awkward in that context >> because creating a new one takes O(n) time to initialize it. Is that >> all true of newByteArray, or can I get one with arbitrary garbage in it >> for cheap? > > newByteArray# does not actively clear memory. > > However, for large arrays, I think the memory is likely to be freshly > allocated from the OS, and the OS will have cleared it for security > reasons. > Not sure how common it is in practice, but it's worth noting that OS'es should be able to clear freed memory as a background process, so that they don't have to do it on demand at the precie point of allocation. Of course this depends on memory churn, etc. etc., so it's not exactly guaranteed even if the OS supports it. Regards,