Scavenging SRTs in scavenge_one
Simon Marlow
marlowsd at gmail.com
Thu Jun 21 18:27:52 UTC 2018
When scavenge_one() sees a STACK, it calls scavenge_stack() which traverses
the stack frames, including their SRTs.
So I don't understand what's going wrong for you - how are the SRTs not
being traversed?
Cheers
Simon
On 21 June 2018 at 11:58, Ömer Sinan Ağacan <omeragacan at gmail.com> wrote:
> Here's an example where we allocate a large (4K) stack:
>
> >>> bt
> #0 allocateMightFail (cap=0x7f366808cfc0 <MainCapability>,
> n=4096) at rts/sm/Storage.c:876
> #1 0x00007f3667e4a85d in allocate (cap=0x7f366808cfc0
> <MainCapability>, n=4096) at rts/sm/Storage.c:849
> #2 0x00007f3667e16f46 in threadStackOverflow (cap=0x7f366808cfc0
> <MainCapability>, tso=0x4200152a68) at rts/Threads.c:600
> #3 0x00007f3667e12a64 in schedule
> (initialCapability=0x7f366808cfc0 <MainCapability>, task=0x78c970) at
> rts/Schedule.c:520
> #4 0x00007f3667e1215f in scheduleWaitThread (tso=0x4200105388,
> ret=0x0, pcap=0x7ffef40dce78) at rts/Schedule.c:2533
> #5 0x00007f3667e25685 in rts_evalLazyIO (cap=0x7ffef40dce78,
> p=0x736ef8, ret=0x0) at rts/RtsAPI.c:530
> #6 0x00007f3667e25f7a in hs_main (argc=16, argv=0x7ffef40dd0a8,
> main_closure=0x736ef8, rts_config=...) t rts/RtsMain.c:72
> #7 0x00000000004f738f in main ()
>
> This is based on an old tree so source locations may not be correct, it's
> this
> code in threadStackOverflow():
>
> // Charge the current thread for allocating stack. Stack usage is
> // non-deterministic, because the chunk boundaries might vary from
> // run to run, but accounting for this is better than not
> // accounting for it, since a deep recursion will otherwise not be
> // subject to allocation limits.
> cap->r.rCurrentTSO = tso;
> new_stack = (StgStack*) allocate(cap, chunk_size);
> cap->r.rCurrentTSO = NULL;
>
> SET_HDR(new_stack, &stg_STACK_info, old_stack->header.prof.ccs);
> TICK_ALLOC_STACK(chunk_size);
>
> Ömer
> Ömer Sinan Ağacan <omeragacan at gmail.com>, 21 Haz 2018 Per, 13:42
> tarihinde şunu yazdı:
> >
> > > Large objects can only be primitive objects, like MUT_ARR_PTRS,
> allocated by
> > > the RTS, and none of these have SRTs.
> >
> > Is is not possible to allocate a large STACK? I'm currently observing
> this in
> > gdb:
> >
> > >>> call *Bdescr(0x4200ec9000)
> > $2 = {
> > start = 0x4200ec9000,
> > free = 0x4200ed1000,
> > link = 0x4200100e80,
> > u = {
> > back = 0x4200103980,
> > bitmap = 0x4200103980,
> > scan = 0x4200103980
> > },
> > gen = 0x77b4b8,
> > gen_no = 1,
> > dest_no = 1,
> > node = 0,
> > flags = 1027, <-- BF_LARGE | BF_EVACUTED | ...
> > blocks = 8,
> > _padding = {[0] = 0, [1] = 0, [2] = 0}
> > }
> >
> > >>> call printClosure(0x4200ec9000)
> > 0x4200ec9000: STACK
> >
> > >>> call checkClosure(0x4200ec9000)
> > $3 = 4096 -- makes sense, larger than 3277 bytes
> >
> > So I have a large STACK object, and STACKs can refer to static objects.
> But
> > when we scavenge this object we don't scavenge its SRTs because we use
> > scavenge_one(). This seems wrong to me.
> >
> > Ömer
> >
> > Simon Marlow <marlowsd at gmail.com>, 20 Haz 2018 Çar, 14:32 tarihinde
> şunu yazdı:
> > >
> > > Interesting point. I don't think there are any large objects with
> SRTs, but we should document the invariant because we're relying on it.
> > >
> > > Large objects can only be primitive objects, like MUT_ARR_PTRS,
> allocated by the RTS, and none of these have SRTs.
> > >
> > > We did have plans to allocate memory for large dynamic objects using
> `allocate()` from compiled code, in which case we could have large objects
> that could be THUNK, FUN, etc. and could have an SRT, in which case we
> would need to revisit this. You might want to take a look at Note [big
> objects] in GCUtils.c, which is relevant here.
> > >
> > > Cheers
> > > Simon
> > >
> > >
> > > On 20 June 2018 at 09:20, Ömer Sinan Ağacan <omeragacan at gmail.com>
> wrote:
> > >>
> > >> Hi Simon,
> > >>
> > >> I'm confused about this code again. You said
> > >>
> > >> > scavenge_one() is only used for a non-major collection, where we
> aren't
> > >> > traversing SRTs.
> > >>
> > >> But I think this is not true; scavenge_one() is also used to scavenge
> large
> > >> objects (in scavenge_large()), which are scavenged even in major GCs.
> So it
> > >> seems like we never really scavenge SRTs of large objects. This
> doesn't look
> > >> right to me. Am I missing anything? Can large objects not refer to
> static
> > >> objects?
> > >>
> > >> Thanks
> > >>
> > >> Ömer
> > >>
> > >> Ömer Sinan Ağacan <omeragacan at gmail.com>, 2 May 2018 Çar, 09:03
> > >> tarihinde şunu yazdı:
> > >> >
> > >> > Thanks Simon, this is really helpful.
> > >> >
> > >> > > If you look at scavenge_fun_srt() and co, you'll see that they
> return
> > >> > > immediately if !major_gc.
> > >> >
> > >> > Thanks for pointing this out -- I didn't realize it's returning
> early when
> > >> > !major_gc and this caused a lot of confusion. Now everything makes
> sense.
> > >> >
> > >> > I'll add a note for scavenging SRTs and refer to it in relevant
> code and submit
> > >> > a diff.
> > >> >
> > >> > Ömer
> > >> >
> > >> > 2018-05-01 22:10 GMT+03:00 Simon Marlow <marlowsd at gmail.com>:
> > >> > > Your explanation is basically right. scavenge_one() is only used
> for a
> > >> > > non-major collection, where we aren't traversing SRTs. Admittedly
> this is a
> > >> > > subtle point that could almost certainly be documented better, I
> probably
> > >> > > just overlooked it.
> > >> > >
> > >> > > More inline:
> > >> > >
> > >> > > On 1 May 2018 at 10:26, Ömer Sinan Ağacan <omeragacan at gmail.com>
> wrote:
> > >> > >>
> > >> > >> I have an idea but it doesn't explain everything;
> > >> > >>
> > >> > >> SRTs are used to collect CAFs, and CAFs are always added to the
> oldest
> > >> > >> generation's mut_list when allocated [1].
> > >> > >>
> > >> > >> When we're scavenging a mut_list we know we're not doing a major
> GC, and
> > >> > >> because mut_list of oldest generation has all the newly
> allocated CAFs,
> > >> > >> which
> > >> > >> will be scavenged anyway, no need to scavenge SRTs for those.
> > >> > >>
> > >> > >> Also, static objects are always evacuated to the oldest gen [2],
> so any
> > >> > >> CAFs
> > >> > >> that are alive but not in the mut_list of the oldest gen will
> stay alive
> > >> > >> after
> > >> > >> a non-major GC, again no need to scavenge SRTs to keep these
> alive.
> > >> > >>
> > >> > >> This also explains why it's OK to not collect static objects
> (and not
> > >> > >> treat
> > >> > >> them as roots) in non-major GCs.
> > >> > >>
> > >> > >> However this doesn't explain
> > >> > >>
> > >> > >> - Why it's OK to scavenge large objects with scavenge_one().
> > >> > >
> > >> > >
> > >> > > I don't understand - perhaps you could elaborate on why you think
> it might
> > >> > > not be OK? Large objects are treated exactly the same as small
> objects with
> > >> > > respect to their lifetimes.
> > >> > >
> > >> > >>
> > >> > >> - Why we scavenge SRTs in non-major collections in other places
> (e.g.
> > >> > >> scavenge_block()).
> > >> > >
> > >> > >
> > >> > > If you look at scavenge_fun_srt() and co, you'll see that they
> return
> > >> > > immediately if !major_gc.
> > >> > >
> > >> > >>
> > >> > >> Simon, could you say a few words about this?
> > >> > >
> > >> > >
> > >> > > Was that enough words? I have more if necessary :)
> > >> > >
> > >> > > Cheers
> > >> > > Simon
> > >> > >
> > >> > >
> > >> > >>
> > >> > >>
> > >> > >> [1]: https://github.com/ghc/ghc/blob/master/rts/sm/Storage.c#
> L445-L449
> > >> > >> [2]: https://github.com/ghc/ghc/blob/master/rts/sm/Scav.c#
> L1761-L1763
> > >> > >>
> > >> > >> Ömer
> > >> > >>
> > >> > >> 2018-03-28 17:49 GMT+03:00 Ben Gamari <ben at well-typed.com>:
> > >> > >> > Hi Simon,
> > >> > >> >
> > >> > >> > I'm a bit confused by scavenge_one; namely it doesn't scavenge
> SRTs. It
> > >> > >> > appears that it is primarily used for remembered set entries
> but it's
> > >> > >> > not at all clear why this means that we can safely ignore SRTs
> (e.g. in
> > >> > >> > the FUN and THUNK cases).
> > >> > >> >
> > >> > >> > Can you shed some light on this?
> > >> > >> >
> > >> > >> > Cheers,
> > >> > >> >
> > >> > >> > - Ben
> > >> > >> >
> > >> > >> > _______________________________________________
> > >> > >> > ghc-devs mailing list
> > >> > >> > ghc-devs at haskell.org
> > >> > >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> > >> > >> >
> > >> > >> _______________________________________________
> > >> > >> ghc-devs mailing list
> > >> > >> ghc-devs at haskell.org
> > >> > >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> > >> > >
> > >> > >
> > >
> > >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20180621/fa22d8cb/attachment.html>
More information about the ghc-devs
mailing list