HEAD doesn't build. Totally stalled.

Fri Jul 17 02:45:37 UTC 2020

Well, we actually *do* test for __SSP__ in HEAD:
https://github.com/ghc/ghc/blob/master/rts/RtsSymbols.c#L1170
Which currently lists:
#if !defined(mingw32_HOST_OS) && !defined(DYNAMIC) &&
(defined(_FORTIFY_SOURCE) || defined(__SSP__))

But this seems to still be ill conceived.  And while Simon is the only
one I'm aware of, for whom this breaks we need to find a better
solution. As such, we will revert the commits.

Why do we do all this symbol nonsense in the RTS to begin with?  It
has to do with our static linker we have in GHC. Loading arbitrary
archives, means we need to be able to resolve all kinds of symbols
that objects might refer to. For regular dependencies this will work
if the dependencies are listed in the package configuration file, the
linker will know which dependencies to link. This get a bit annoying
for libraries that the compiler will automagically provide. libgcc,
libssp, librt, ...

The solution so far was simply to have the RTS depend on these
symbols, and keep a list of them around. That way when the linker
built the RTS we'd get it to link all these symbols into the RTS, and
we could refer to them in the linker. Essentially looking them up in
the linked binary (ghc, or iserv).

This is a rather tricky problem, and almost all solutions we came up
with are annoying in one or more dimensions.  After some discussion on
IRC last night, we'll go forward trying the following solution:

We'll keep a file in the lib folder (similar to the settings,
llvm-targets, ...) that is essentially a lookup table of Symbol ->
[Library]. If we encounter an unknown symbol, and we have it in our
lookup table, we will try to load the named libraries, hoping for them
to contain the symbol we are looking for. If everything fails we'll
bail.

For the example symbols that prompted this issue: (which are emitted
when stack smashing protector hardening is enabled, which seems to be
the default on most linux distributions today, which is likely why I
couldn't reproduce this easily.)

[("__stack_chk_guard", ["ssp"])]

would tell the compiler to try to locate (through the usual library
location means) the library called "ssp", if it encounters the symbol
"__stack_chk_guard".

Isn't this what the dynamic linker is supposed to solve? Why do we
have to do all this on our own? Can't we just use the dynamic linker?
Yes, and no. Yes we can use the dynamic linker, and we even do. But
not all platforms have a working, or usable linker. iOS for example
has a working dynamic linker, but user programs can't use it. muslc
reports "Dynamic loading not supported" when calling dlopen on arm.

Thus I'm reluctant to drop the static linker outright for the dynamic linker.

Cheers,
 Moritz

On Fri, Jul 17, 2020 at 2:45 AM Phyx <lonetiger at gmail.com> wrote:
>
> But, where do you actually check for __SSP__
>
> The guard just checks for not windows and not dynamic https://github.com/ghc/ghc/commit/686e72253aed3880268dd6858eadd8c320f09e97#diff-03f5bc5a50fd8ae13e902782c4392c38R1157
>
> shouldn't it just be checking for defined(__SSP__) instead? This check is currently only correct if the distro has turned stack protector on by default.
>
>
> Regards,
> Tamar
>
> On Thu, Jul 16, 2020 at 3:46 PM Moritz Angermann <moritz.angermann at gmail.com> wrote:
>>
>> I’ve tried to reproduce this and it turns out, I fail to. You are somehow building the rts either with _FORTYFY_SOURCE or __SSP__, but then your linker ends up not passing -lssp or the equivalent for your tool chain.
>>
>> At this point I’m tempted to add an additional ARM arch guard. While that would be conceptually wrong, it would reduce the cases where this could go wrong to a rarely used platform. Maybe @Ben Gamari has an idea?
>>
>> On Thu, 16 Jul 2020 at 10:25 PM, Simon Peyton Jones <simonpj at microsoft.com> wrote:
>>>
>>> Moritz
>>>
>>> How’s it going getting this patch committed?
>>>
>>> It’s painful manually applying a fix, but then NOT committing that to master by mistake
>>>
>>>
>>>
>>> Thanks
>>>
>>> s
>>>
>>>
>>>
>>> From: Moritz Angermann <moritz.angermann at gmail.com>
>>> Sent: 14 July 2020 12:14
>>> To: Simon Peyton Jones <simonpj at microsoft.com>
>>> Cc: ghc-devs at haskell.org
>>> Subject: Re: HEAD doesn't build. Totally stalled.
>>>
>>>
>>>
>>> For some reason, you end up in the defined RTS_SSP_SYMBOLS, I believe and then the RTS wants __stack_chk symbols. Which it can’t find when linking.
>>>
>>>
>>>
>>> Replacing
>>>
>>> #if !defined(mingw32_HOST_OS) && !defined(DYNAMIC)
>>>
>>> #define RTS_SSP_SYMBOLS                                \
>>>
>>>       SymI_NeedsProto(__stack_chk_guard)               \
>>>
>>>       SymI_NeedsProto(__stack_chk_fail)
>>>
>>> #else
>>>
>>> #define RTS_SSP_SYMBOLS
>>>
>>> #endif
>>>
>>> With just
>>>
>>>
>>>
>>> #define RTS_SSP_SYMBOLS
>>>
>>>
>>>
>>> Should do. I hope.
>>>
>>>
>>>
>>> Currently only on mobile phone :-/
>>>
>>>
>>>
>>> Cheers,
>>>
>>>  Moritz
>>>
>>>
>>>
>>> On Tue, 14 Jul 2020 at 7:06 PM, Simon Peyton Jones <simonpj at microsoft.com> wrote:
>>>
>>> thanks.  What specifically do I comment out?
>>>
>>>
>>>
>>> From: Moritz Angermann <moritz.angermann at gmail.com>
>>> Sent: 14 July 2020 12:00
>>> To: Simon Peyton Jones <simonpj at microsoft.com>
>>> Cc: ghc-devs at haskell.org
>>> Subject: Re: HEAD doesn't build. Totally stalled.
>>>
>>>
>>>
>>> This was my fault. Not sure why this wasn’t caught in CI.
>>>
>>> It’s due to the addition of the symbols here
>>>
>>>
>>>
>>> https://github.com/ghc/ghc/commit/686e72253aed3880268dd6858eadd8c320f09e97#diff-03f5bc5a50fd8ae13e902782c4392c38R1159
>>>
>>>
>>>
>>> You should be able to just comment them out. I’ll prepare a proper fix.
>>>
>>>
>>>
>>> Cheers,
>>>
>>>  Moritz
>>>
>>>
>>>
>>> On Tue, 14 Jul 2020 at 6:41 PM, Simon Peyton Jones via ghc-devs <ghc-devs at haskell.org> wrote:
>>>
>>> I’m getting this failure in a clean HEAD build. Any ideas?    I’m totally stalled because I can’t build GHC any more.
>>>
>>> I’m using Windows Subsystem for Linux (WSL).
>>>
>>> Help help!
>>>
>>> Thanks
>>>
>>> Simon
>>>
>>> /home/simonpj/code/HEAD-9/rts/dist/build/libHSrts_thr_p.a(RtsSymbols.thr_p_o): RtsSymbols.c:rtsSyms: error: undefined reference to '__stack_chk_guard'
>>>
>>> collect2: error: ld returned 1 exit status
>>>
>>> `cc' failed in phase `Linker'. (Exit code: 1)
>>>
>>> utils/iserv/ghc.mk:105: recipe for target 'utils/iserv/stage2_p/build/tmp/ghc-iserv-prof' failed
>>>
>>> make[1]: *** [utils/iserv/stage2_p/build/tmp/ghc-iserv-prof] Error 1
>>>
>>> make[1]: *** Waiting for unfinished jobs....
>>>
>>> _______________________________________________
>>> ghc-devs mailing list
>>> ghc-devs at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>> _______________________________________________
>> ghc-devs mailing list
>> ghc-devs at haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs