[Git][ghc/ghc][wip/marge_bot_batch_merge_job] 13 commits: withTimings: Emit allocations counter
Marge Bot
gitlab at gitlab.haskell.org
Sun Nov 29 21:05:00 UTC 2020
Marge Bot pushed to branch wip/marge_bot_batch_merge_job at Glasgow Haskell Compiler / GHC
Commits:
1bc104b0 by Ben Gamari at 2020-11-29T15:33:18-05:00
withTimings: Emit allocations counter
This will allow us to back out the allocations per compiler pass from
the eventlog. Note that we dump the allocation counter rather than the
difference since this will allow us to determine how much work is done
*between* `withTiming` blocks.
- - - - -
e992ea84 by GHC GitLab CI at 2020-11-29T15:33:54-05:00
ThreadPaused: Don't zero slop until free vars are pushed
When threadPaused blackholes a thunk it calls `OVERWRITING_CLOSURE` to
zero the slop for the benefit of the sanity checker. Previously this was
done *before* pushing the thunk's free variables to the update
remembered set. Consequently we would pull zero'd pointers to the update
remembered set.
- - - - -
e82cd140 by GHC GitLab CI at 2020-11-29T15:33:54-05:00
nonmoving: Fix regression from TSAN work
The TSAN rework (specifically aad1f803) introduced a subtle regression
in GC.c, swapping `g0` in place of `gen`. Whoops!
Fixes #18997.
- - - - -
35a5207e by GHC GitLab CI at 2020-11-29T15:33:54-05:00
rts/Messages: Add missing write barrier in THROWTO message update
After a THROWTO message has been handle the message closure is
overwritten by a NULL message. We must ensure that the original
closure's pointers continue to be visible to the nonmoving GC.
- - - - -
0120829f by GHC GitLab CI at 2020-11-29T15:33:54-05:00
nonmoving: Add missing write barrier in shrinkSmallByteArray
- - - - -
8a4d8fb6 by GHC GitLab CI at 2020-11-29T15:33:54-05:00
Updates: Don't zero slop until closure has been pushed
Ensure that the the free variables have been pushed to the update
remembered set before we zero the slop.
- - - - -
2793cfdc by GHC GitLab CI at 2020-11-29T15:33:54-05:00
OSThreads: Fix error code checking
pthread_join returns its error code and apparently doesn't set errno.
- - - - -
e391a16f by GHC GitLab CI at 2020-11-29T15:33:54-05:00
nonmoving: Don't join to mark_thread on shutdown
The mark thread is not joinable as we detach from it on creation.
- - - - -
60d088ab by Ben Gamari at 2020-11-29T15:33:54-05:00
nonmoving: Add reference to Ueno 2016
- - - - -
3aa60362 by GHC GitLab CI at 2020-11-29T15:33:54-05:00
nonmoving: Ensure that evacuated large objects are marked
See Note [Non-moving GC: Marking evacuated objects].
- - - - -
a1efdf34 by Ben Gamari at 2020-11-29T16:04:48-05:00
rts: Use CHECK instead of assert
Use the GHC wrappers instead of <assert.h>.
- - - - -
6c2d941c by Ben Gamari at 2020-11-29T16:04:49-05:00
rts/linker: Replace some ASSERTs with CHECK
In the past some people have confused ASSERT, which is for checking
internal invariants, which CHECK, which should be used when checking
things that might fail due to bad input (and therefore should be enabled
even in the release compiler). Change some of these cases in the linker
to use CHECK.
- - - - -
706c924b by Ryan Scott at 2020-11-29T16:04:49-05:00
Allow deploy:pages job to fail
See #18973.
- - - - -
20 changed files:
- .gitlab-ci.yml
- compiler/GHC/Utils/Error.hs
- includes/rts/storage/ClosureMacros.h
- rts/Linker.c
- rts/Messages.c
- rts/Messages.h
- rts/PrimOps.cmm
- rts/RaiseAsync.c
- rts/ThreadPaused.c
- rts/Updates.h
- rts/linker/Elf.c
- rts/linker/MachO.c
- rts/linker/PEi386.c
- rts/linker/elf_got.c
- rts/linker/elf_reloc_aarch64.c
- rts/posix/OSThreads.c
- rts/sm/Evac.c
- rts/sm/GC.c
- rts/sm/NonMoving.c
- rts/win32/veh_excn.c
Changes:
=====================================
.gitlab-ci.yml
=====================================
@@ -1237,6 +1237,8 @@ pages:
dependencies:
- doc-tarball
image: ghcci/x86_64-linux-deb9:0.2
+ # See #18973
+ allow_failure: true
tags:
- x86_64-linux
script:
=====================================
compiler/GHC/Utils/Error.hs
=====================================
@@ -536,14 +536,16 @@ withTiming' dflags what force_result prtimings action
logInfo dflags $ withPprStyle defaultUserStyle $
text "***" <+> what <> colon
let ctx = initDefaultSDocContext dflags
- eventBegins ctx what
alloc0 <- liftIO getAllocationCounter
start <- liftIO getCPUTime
+ eventBegins ctx what
+ recordAllocs alloc0
!r <- action
() <- pure $ force_result r
eventEnds ctx what
end <- liftIO getCPUTime
alloc1 <- liftIO getAllocationCounter
+ recordAllocs alloc1
-- recall that allocation counter counts down
let alloc = alloc0 - alloc1
time = realToFrac (end - start) * 1e-9
@@ -569,12 +571,19 @@ withTiming' dflags what force_result prtimings action
else action
where whenPrintTimings = liftIO . when (prtimings == PrintTimings)
+
+ recordAllocs alloc = do
+ liftIO $ traceMarkerIO $ "GHC:allocs:" ++ show alloc
+
eventBegins ctx w = do
- whenPrintTimings $ traceMarkerIO (eventBeginsDoc ctx w)
- liftIO $ traceEventIO (eventBeginsDoc ctx w)
+ let doc = eventBeginsDoc ctx w
+ whenPrintTimings $ traceMarkerIO doc
+ liftIO $ traceEventIO doc
+
eventEnds ctx w = do
- whenPrintTimings $ traceMarkerIO (eventEndsDoc ctx w)
- liftIO $ traceEventIO (eventEndsDoc ctx w)
+ let doc = eventEndsDoc ctx w
+ whenPrintTimings $ traceMarkerIO doc
+ liftIO $ traceEventIO doc
eventBeginsDoc ctx w = showSDocOneLine ctx $ text "GHC:started:" <+> w
eventEndsDoc ctx w = showSDocOneLine ctx $ text "GHC:finished:" <+> w
=====================================
includes/rts/storage/ClosureMacros.h
=====================================
@@ -520,11 +520,15 @@ INLINE_HEADER StgWord8 *mutArrPtrsCard (StgMutArrPtrs *a, W_ n)
#if defined(PROFILING) || defined(DEBUG)
#define OVERWRITING_CLOSURE(c) \
overwritingClosure(c)
+#define OVERWRITING_CLOSURE_SIZE(c, size) \
+ overwritingClosureSize(c, size)
#define OVERWRITING_CLOSURE_MUTABLE(c, off) \
overwritingMutableClosureOfs(c, off)
#else
#define OVERWRITING_CLOSURE(c) \
do { (void) sizeof(c); } while(0)
+#define OVERWRITING_CLOSURE_SIZE(c, size) \
+ do { (void) sizeof(c); (void) sizeof(size); } while(0)
#define OVERWRITING_CLOSURE_MUTABLE(c, off) \
do { (void) sizeof(c); (void) sizeof(off); } while(0)
#endif
=====================================
rts/Linker.c
=====================================
@@ -49,7 +49,6 @@
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
-#include <assert.h>
#include <fs_rts.h>
#if defined(HAVE_SYS_STAT_H)
@@ -885,12 +884,11 @@ SymbolAddr* lookupDependentSymbol (SymbolName* lbl, ObjectCode *dependent)
*/
IF_DEBUG(linker, debugBelch("lookupSymbol: looking up %s with dlsym\n",
lbl));
- ASSERT(lbl[0] == '_');
+ CHECK(lbl[0] == '_');
return internal_dlsym(lbl + 1);
# else
- ASSERT(false);
- return NULL;
+# error No OBJFORMAT_* macro set
# endif
} else {
if (dependent) {
@@ -2112,7 +2110,7 @@ HsInt unloadNativeObj (void *handle)
n_unloaded_objects += 1;
// dynamic objects have no symbols
- ASSERT(nc->symbols == NULL);
+ CHECK(nc->symbols == NULL);
freeOcStablePtrs(nc);
// Remove object code from root set
=====================================
rts/Messages.c
=====================================
@@ -97,7 +97,7 @@ loop:
case THROWTO_SUCCESS: {
// this message is done
StgTSO *source = t->source;
- doneWithMsgThrowTo(t);
+ doneWithMsgThrowTo(cap, t);
tryWakeupThread(cap, source);
break;
}
=====================================
rts/Messages.h
=====================================
@@ -23,8 +23,16 @@ void sendMessage (Capability *from_cap, Capability *to_cap, Message *msg);
#include "SMPClosureOps.h"
INLINE_HEADER void
-doneWithMsgThrowTo (MessageThrowTo *m)
+doneWithMsgThrowTo (Capability *cap, MessageThrowTo *m)
{
+ // The message better be locked
+ ASSERT(m->header.info == &stg_WHITEHOLE_info);
+ IF_NONMOVING_WRITE_BARRIER_ENABLED {
+ updateRemembSetPushClosure(cap, (StgClosure *) m->link);
+ updateRemembSetPushClosure(cap, (StgClosure *) m->source);
+ updateRemembSetPushClosure(cap, (StgClosure *) m->target);
+ updateRemembSetPushClosure(cap, (StgClosure *) m->exception);
+ }
OVERWRITING_CLOSURE((StgClosure*)m);
unlockClosure((StgClosure*)m, &stg_MSG_NULL_info);
LDV_RECORD_CREATE(m);
=====================================
rts/PrimOps.cmm
=====================================
@@ -227,6 +227,21 @@ stg_shrinkSmallMutableArrayzh ( gcptr mba, W_ new_size )
{
ASSERT(new_size <= StgSmallMutArrPtrs_ptrs(mba));
+ IF_NONMOVING_WRITE_BARRIER_ENABLED {
+ // Ensure that the elements we are about to shrink out of existence
+ // remain visible to the non-moving collector.
+ W_ p, end;
+ p = mba + SIZEOF_StgSmallMutArrPtrs + WDS(new_size);
+ end = mba + SIZEOF_StgSmallMutArrPtrs + WDS(StgSmallMutArrPtrs_ptrs(mba));
+again:
+ ccall updateRemembSetPushClosure_(BaseReg "ptr",
+ W_[p] "ptr");
+ if (p < end) {
+ p = p + SIZEOF_W;
+ goto again;
+ }
+ }
+
OVERWRITING_CLOSURE_MUTABLE(mba, (BYTES_TO_WDS(SIZEOF_StgSmallMutArrPtrs) +
new_size));
StgSmallMutArrPtrs_ptrs(mba) = new_size;
=====================================
rts/RaiseAsync.c
=====================================
@@ -336,7 +336,7 @@ check_target:
}
// nobody else can wake up this TSO after we claim the message
- doneWithMsgThrowTo(m);
+ doneWithMsgThrowTo(cap, m);
raiseAsync(cap, target, msg->exception, false, NULL);
return THROWTO_SUCCESS;
@@ -580,7 +580,7 @@ maybePerformBlockedException (Capability *cap, StgTSO *tso)
throwToSingleThreaded(cap, msg->target, msg->exception);
source = msg->source;
- doneWithMsgThrowTo(msg);
+ doneWithMsgThrowTo(cap, msg);
tryWakeupThread(cap, source);
return 1;
}
@@ -602,7 +602,7 @@ awakenBlockedExceptionQueue (Capability *cap, StgTSO *tso)
i = lockClosure((StgClosure *)msg);
if (i != &stg_MSG_NULL_info) {
source = msg->source;
- doneWithMsgThrowTo(msg);
+ doneWithMsgThrowTo(cap, msg);
tryWakeupThread(cap, source);
} else {
unlockClosure((StgClosure *)msg,i);
@@ -700,7 +700,7 @@ removeFromQueues(Capability *cap, StgTSO *tso)
// ASSERT(m->header.info == &stg_WHITEHOLE_info);
// unlock and revoke it at the same time
- doneWithMsgThrowTo(m);
+ doneWithMsgThrowTo(cap, m);
break;
}
=====================================
rts/ThreadPaused.c
=====================================
@@ -314,10 +314,6 @@ threadPaused(Capability *cap, StgTSO *tso)
continue;
}
- // zero out the slop so that the sanity checker can tell
- // where the next closure is.
- OVERWRITING_CLOSURE(bh);
-
// an EAGER_BLACKHOLE or CAF_BLACKHOLE gets turned into a
// BLACKHOLE here.
#if defined(THREADED_RTS)
@@ -345,11 +341,16 @@ threadPaused(Capability *cap, StgTSO *tso)
// overwrite to the update remembered set.
// N.B. We caught the WHITEHOLE case above.
updateRemembSetPushThunkEager(cap,
- THUNK_INFO_PTR_TO_STRUCT(bh_info),
- (StgThunk *) bh);
+ THUNK_INFO_PTR_TO_STRUCT(bh_info),
+ (StgThunk *) bh);
}
}
+ // zero out the slop so that the sanity checker can tell
+ // where the next closure is. N.B. We mustn't do this until we have
+ // pushed the free variables to the update remembered set above.
+ OVERWRITING_CLOSURE_SIZE(bh, closure_sizeW_(bh, INFO_PTR_TO_STRUCT(bh_info)));
+
// The payload of the BLACKHOLE points to the TSO
RELAXED_STORE(&((StgInd *)bh)->indirectee, (StgClosure *)tso);
SET_INFO_RELEASE(bh,&stg_BLACKHOLE_info);
=====================================
rts/Updates.h
=====================================
@@ -49,7 +49,6 @@
W_ bd; \
\
prim_write_barrier; \
- OVERWRITING_CLOSURE(p1); \
bd = Bdescr(p1); \
if (bdescr_gen_no(bd) != 0 :: bits16) { \
IF_NONMOVING_WRITE_BARRIER_ENABLED { \
@@ -60,6 +59,7 @@
} else { \
TICK_UPD_NEW_IND(); \
} \
+ OVERWRITING_CLOSURE(p1); \
StgInd_indirectee(p1) = p2; \
prim_write_barrier; \
SET_INFO(p1, stg_BLACKHOLE_info); \
=====================================
rts/linker/Elf.c
=====================================
@@ -416,7 +416,7 @@ ocVerifyImage_ELF ( ObjectCode* oc )
"\nSection header table: start %ld, n_entries %d, ent_size %d\n",
(long)ehdr->e_shoff, shnum, ehdr->e_shentsize ));
- ASSERT(ehdr->e_shentsize == sizeof(Elf_Shdr));
+ CHECK(ehdr->e_shentsize == sizeof(Elf_Shdr));
shdr = (Elf_Shdr*) (ehdrC + ehdr->e_shoff);
@@ -537,7 +537,7 @@ ocVerifyImage_ELF ( ObjectCode* oc )
#if defined(SHN_XINDEX)
/* See Note [Many ELF Sections] */
if (secno == SHN_XINDEX) {
- ASSERT(shndxTable);
+ CHECK(shndxTable);
secno = shndxTable[j];
}
#endif
@@ -864,7 +864,7 @@ ocGetNames_ELF ( ObjectCode* oc )
PROT_READ | PROT_WRITE,
MAP_ANON | MAP_PRIVATE,
-1, 0);
- ASSERT(common_mem != NULL);
+ CHECK(common_mem != NULL);
}
//TODO: we ignore local symbols anyway right? So we can use the
@@ -893,7 +893,7 @@ ocGetNames_ELF ( ObjectCode* oc )
secno = shndx;
#if defined(SHN_XINDEX)
if (shndx == SHN_XINDEX) {
- ASSERT(shndxTable);
+ CHECK(shndxTable);
secno = shndxTable[j];
}
#endif
@@ -902,11 +902,11 @@ ocGetNames_ELF ( ObjectCode* oc )
if (shndx == SHN_COMMON) {
isLocal = false;
- ASSERT(common_used < common_size);
- ASSERT(common_mem);
+ CHECK(common_used < common_size);
+ CHECK(common_mem);
symbol->addr = (void*)((uintptr_t)common_mem + common_used);
common_used += symbol->elf_sym->st_size;
- ASSERT(common_used <= common_size);
+ CHECK(common_used <= common_size);
IF_DEBUG(linker,
debugBelch("COMMON symbol, size %ld name %s allocated at %p\n",
@@ -935,7 +935,7 @@ ocGetNames_ELF ( ObjectCode* oc )
)
) {
/* Section 0 is the undefined section, hence > and not >=. */
- ASSERT(secno > 0 && secno < shnum);
+ CHECK(secno > 0 && secno < shnum);
/*
if (shdr[secno].sh_type == SHT_NOBITS) {
debugBelch(" BSS symbol, size %d off %d name %s\n",
@@ -945,7 +945,7 @@ ocGetNames_ELF ( ObjectCode* oc )
symbol->addr = (SymbolAddr*)(
(intptr_t) oc->sections[secno].start +
(intptr_t) symbol->elf_sym->st_value);
- ASSERT(symbol->addr != 0x0);
+ CHECK(symbol->addr != 0x0);
if (ELF_ST_BIND(symbol->elf_sym->st_info) == STB_LOCAL) {
isLocal = true;
isWeak = false;
@@ -962,7 +962,7 @@ ocGetNames_ELF ( ObjectCode* oc )
/* And the decision is ... */
if (symbol->addr != NULL) {
- ASSERT(nm != NULL);
+ CHECK(nm != NULL);
/* Acquire! */
if (!isLocal) {
@@ -1045,7 +1045,7 @@ do_Elf_Rel_relocations ( ObjectCode* oc, char* ehdrC,
break;
}
}
- ASSERT(stab != NULL);
+ CHECK(stab != NULL);
targ = (Elf_Word*)oc->sections[target_shndx].start;
IF_DEBUG(linker,debugBelch(
@@ -1251,7 +1251,7 @@ do_Elf_Rel_relocations ( ObjectCode* oc, char* ehdrC,
result = ((S + A) | T) - P;
result &= ~1; // Clear thumb indicator bit
- ASSERT(isInt(26, result)); /* X in range */
+ CHECK(isInt(26, result)); /* X in range */
}
// Update the branch target
@@ -1426,7 +1426,7 @@ do_Elf_Rel_relocations ( ObjectCode* oc, char* ehdrC,
case COMPAT_R_ARM_GOT_PREL: {
int32_t A = *pP;
void* GOT_S = symbol->got_addr;
- ASSERT(GOT_S);
+ CHECK(GOT_S);
*(uint32_t *)P = (uint32_t) GOT_S + A - P;
break;
}
@@ -1552,21 +1552,21 @@ do_Elf_Rela_relocations ( ObjectCode* oc, char* ehdrC,
case R_SPARC_WDISP30:
w1 = *pP & 0xC0000000;
w2 = (Elf_Word)((value - P) >> 2);
- ASSERT((w2 & 0xC0000000) == 0);
+ CHECK((w2 & 0xC0000000) == 0);
w1 |= w2;
*pP = w1;
break;
case R_SPARC_HI22:
w1 = *pP & 0xFFC00000;
w2 = (Elf_Word)(value >> 10);
- ASSERT((w2 & 0xFFC00000) == 0);
+ CHECK((w2 & 0xFFC00000) == 0);
w1 |= w2;
*pP = w1;
break;
case R_SPARC_LO10:
w1 = *pP & ~0x3FF;
w2 = (Elf_Word)(value & 0x3FF);
- ASSERT((w2 & ~0x3FF) == 0);
+ CHECK((w2 & ~0x3FF) == 0);
w1 |= w2;
*pP = w1;
break;
@@ -1866,13 +1866,13 @@ ocResolve_ELF ( ObjectCode* oc )
Elf_Word secno = symbol->elf_sym->st_shndx;
#if defined(SHN_XINDEX)
if (secno == SHN_XINDEX) {
- ASSERT(shndxTable);
+ CHECK(shndxTable);
secno = shndxTable[i];
}
#endif
- ASSERT(symbol->elf_sym->st_name == 0);
- ASSERT(symbol->elf_sym->st_value == 0);
- ASSERT(0x0 != oc->sections[ secno ].start);
+ CHECK(symbol->elf_sym->st_name == 0);
+ CHECK(symbol->elf_sym->st_value == 0);
+ CHECK(0x0 != oc->sections[ secno ].start);
symbol->addr = oc->sections[ secno ].start;
}
}
@@ -1946,7 +1946,7 @@ int ocRunInit_ELF( ObjectCode *oc )
init_start = (init_t*)init_startC;
init_end = (init_t*)(init_startC + shdr[i].sh_size);
for (init = init_start; init < init_end; init++) {
- ASSERT(0x0 != *init);
+ CHECK(0x0 != *init);
(*init)(argc, argv, envv);
}
}
=====================================
rts/linker/MachO.c
=====================================
@@ -252,7 +252,6 @@ resolveImports(
"%s: unknown symbol `%s'", oc->fileName, symbol->name);
return 0;
}
- ASSERT(addr);
checkProddableBlock(oc,
((void**)(oc->image + sect->offset)) + i,
@@ -847,7 +846,7 @@ relocateSection(ObjectCode* oc, int curSection)
IF_DEBUG(linker, debugBelch(" : value = %p\n", (void *)symbol->nlist->n_value));
if ((symbol->nlist->n_type & N_TYPE) == N_SECT) {
- ASSERT(symbol->addr != NULL);
+ CHECK(symbol->addr != NULL);
value = (uint64_t) symbol->addr;
IF_DEBUG(linker, debugBelch("relocateSection, defined external symbol %s, relocated address %p\n",
nm, (void *)value));
@@ -949,29 +948,29 @@ relocateSection(ObjectCode* oc, int curSection)
{
if((int32_t)(value - baseValue) != (int64_t)(value - baseValue))
{
- ASSERT(reloc->r_extern);
+ CHECK(reloc->r_extern);
value = (uint64_t) &makeSymbolExtra(oc, reloc->r_symbolnum, value)
-> jumpIsland;
}
- ASSERT((int32_t)(value - baseValue) == (int64_t)(value - baseValue));
+ CHECK((int32_t)(value - baseValue) == (int64_t)(value - baseValue));
type = X86_64_RELOC_SIGNED;
}
switch(type)
{
case X86_64_RELOC_UNSIGNED:
- ASSERT(!reloc->r_pcrel);
+ CHECK(!reloc->r_pcrel);
thing += value;
break;
case X86_64_RELOC_SIGNED:
case X86_64_RELOC_SIGNED_1:
case X86_64_RELOC_SIGNED_2:
case X86_64_RELOC_SIGNED_4:
- ASSERT(reloc->r_pcrel);
+ CHECK(reloc->r_pcrel);
thing += value - baseValue;
break;
case X86_64_RELOC_SUBTRACTOR:
- ASSERT(!reloc->r_pcrel);
+ CHECK(!reloc->r_pcrel);
thing -= value;
break;
default:
=====================================
rts/linker/PEi386.c
=====================================
@@ -1594,7 +1594,7 @@ ocGetNames_PEi386 ( ObjectCode* oc )
barf ("Could not allocate any heap memory from private heap.");
}
- ASSERT(section.size == 0 || section.info->virtualSize == 0);
+ CHECK(section.size == 0 || section.info->virtualSize == 0);
sz = section.size;
if (sz < section.info->virtualSize) sz = section.info->virtualSize;
@@ -2032,7 +2032,7 @@ ocRunInit_PEi386 ( ObjectCode *oc )
getProgEnvv(&envc, &envv);
Section section = *oc->info->init;
- ASSERT(SECTIONKIND_INIT_ARRAY == section.kind);
+ CHECK(SECTIONKIND_INIT_ARRAY == section.kind);
uint8_t *init_startC = section.start;
init_t *init_start = (init_t*)init_startC;
=====================================
rts/linker/elf_got.c
=====================================
@@ -136,10 +136,10 @@ verifyGot(ObjectCode * oc) {
for(size_t i=0; i < symTab->n_symbols; i++) {
ElfSymbol * symbol = &symTab->symbols[i];
if(symbol->got_addr) {
- ASSERT((void*)(*(void**)symbol->got_addr)
- == (void*)symbol->addr);
+ CHECK((void*)(*(void**)symbol->got_addr)
+ == (void*)symbol->addr);
}
- ASSERT(0 == ((uintptr_t)symbol->addr & 0xffff000000000000));
+ CHECK(0 == ((uintptr_t)symbol->addr & 0xffff000000000000));
}
}
return EXIT_SUCCESS;
=====================================
rts/linker/elf_reloc_aarch64.c
=====================================
@@ -6,7 +6,6 @@
#include "elf_plt.h"
#include <stdlib.h>
-#include <assert.h>
#if defined(aarch64_HOST_ARCH)
@@ -71,15 +70,15 @@ encodeAddendAarch64(Section * section, Elf_Rel * rel, int64_t addend) {
*(uint64_t*)P = (uint64_t)addend;
break;
case COMPAT_R_AARCH64_ABS32:
- assert(isInt64(32, addend));
+ CHECK(isInt64(32, addend));
case COMPAT_R_AARCH64_PREL32:
- assert(isInt64(32, addend));
+ CHECK(isInt64(32, addend));
*(uint32_t*)P = (uint32_t)addend;
break;
case COMPAT_R_AARCH64_ABS16:
- assert(isInt64(16, addend));
+ CHECK(isInt64(16, addend));
case COMPAT_R_AARCH64_PREL16:
- assert(isInt64(16, addend));
+ CHECK(isInt64(16, addend));
*(uint16_t*)P = (uint16_t)addend;
break;
/* static aarch64 relocations */
@@ -95,8 +94,8 @@ encodeAddendAarch64(Section * section, Elf_Rel * rel, int64_t addend) {
// imm64 = SignExtend(hi:lo:0x000,64)
// Range is 21 bits + the 12 page relative bits
// known to be 0. -2^32 <= X < 2^32
- assert(isInt64(21+12, addend));
- assert((addend & 0xfff) == 0); /* page relative */
+ CHECK(isInt64(21+12, addend));
+ CHECK((addend & 0xfff) == 0); /* page relative */
*(inst_t *)P = (*(inst_t *)P & 0x9f00001f)
| (inst_t) (((uint64_t) addend << 17) & 0x60000000)
@@ -106,7 +105,7 @@ encodeAddendAarch64(Section * section, Elf_Rel * rel, int64_t addend) {
/* - control flow relocations */
case COMPAT_R_AARCH64_JUMP26: /* relocate b ... */
case COMPAT_R_AARCH64_CALL26: { /* relocate bl ... */
- assert(isInt64(26+2, addend)); /* X in range */
+ CHECK(isInt64(26+2, addend)); /* X in range */
*(inst_t *)P = (*(inst_t *)P & 0xfc000000) /* keep upper 6 (32-6)
* bits */
| ((uint32_t)(addend >> 2) & 0x03ffffff);
@@ -114,8 +113,8 @@ encodeAddendAarch64(Section * section, Elf_Rel * rel, int64_t addend) {
}
case COMPAT_R_AARCH64_ADR_GOT_PAGE: {
/* range is -2^32 <= X < 2^32 */
- assert(isInt64(21+12, addend)); /* X in range */
- assert((addend & 0xfff) == 0); /* page relative */
+ CHECK(isInt64(21+12, addend)); /* X in range */
+ CHECK((addend & 0xfff) == 0); /* page relative */
*(inst_t *)P = (*(inst_t *)P & 0x9f00001f)
| (inst_t)(((uint64_t)addend << 17) & 0x60000000) // lo
@@ -149,10 +148,10 @@ encodeAddendAarch64(Section * section, Elf_Rel * rel, int64_t addend) {
FALLTHROUGH;
case COMPAT_R_AARCH64_LD64_GOT_LO12_NC: {
if(exp_shift == -1) {
- assert( (addend & 7) == 0 );
+ CHECK( (addend & 7) == 0 );
exp_shift = 3;
}
- assert((addend & 0xfff) == addend);
+ CHECK((addend & 0xfff) == addend);
int shift = 0;
if(isLoadStore(P)) {
/* bits 31, 30 encode the size. */
@@ -161,7 +160,7 @@ encodeAddendAarch64(Section * section, Elf_Rel * rel, int64_t addend) {
shift = 4;
}
}
- assert(addend == 0 || exp_shift == shift);
+ CHECK(addend == 0 || exp_shift == shift);
*(inst_t *)P = (*(inst_t *)P & 0xffc003ff)
| ((inst_t)(addend >> shift << 10) & 0x003ffc00);
break;
@@ -188,12 +187,12 @@ computeAddend(Section * section, Elf_Rel * rel,
/* Position where something is relocated */
addr_t P = (addr_t)((uint8_t*)section->start + rel->r_offset);
- assert(0x0 != P);
- assert((uint64_t)section->start <= P);
- assert(P <= (uint64_t)section->start + section->size);
+ CHECK(0x0 != P);
+ CHECK((uint64_t)section->start <= P);
+ CHECK(P <= (uint64_t)section->start + section->size);
/* Address of the symbol */
addr_t S = (addr_t) symbol->addr;
- assert(0x0 != S);
+ CHECK(0x0 != S);
/* GOT slot for the symbol */
addr_t GOT_S = (addr_t) symbol->got_addr;
@@ -243,16 +242,16 @@ computeAddend(Section * section, Elf_Rel * rel,
}
}
- assert(0 == (0xffff000000000000 & S));
+ CHECK(0 == (0xffff000000000000 & S));
V = S + A - P;
- assert(isInt64(26+2, V)); /* X in range */
+ CHECK(isInt64(26+2, V)); /* X in range */
}
return V;
}
- case COMPAT_R_AARCH64_LDST128_ABS_LO12_NC: assert(0 == ((S+A) & 0x0f));
- case COMPAT_R_AARCH64_LDST64_ABS_LO12_NC: assert(0 == ((S+A) & 0x07));
- case COMPAT_R_AARCH64_LDST32_ABS_LO12_NC: assert(0 == ((S+A) & 0x03));
- case COMPAT_R_AARCH64_LDST16_ABS_LO12_NC: assert(0 == ((S+A) & 0x01));
+ case COMPAT_R_AARCH64_LDST128_ABS_LO12_NC: CHECK(0 == ((S+A) & 0x0f));
+ case COMPAT_R_AARCH64_LDST64_ABS_LO12_NC: CHECK(0 == ((S+A) & 0x07));
+ case COMPAT_R_AARCH64_LDST32_ABS_LO12_NC: CHECK(0 == ((S+A) & 0x03));
+ case COMPAT_R_AARCH64_LDST16_ABS_LO12_NC: CHECK(0 == ((S+A) & 0x01));
case COMPAT_R_AARCH64_LDST8_ABS_LO12_NC:
/* type: static, class: aarch64, op: S + A */
return (S + A) & 0xfff;
@@ -266,12 +265,12 @@ computeAddend(Section * section, Elf_Rel * rel,
// TODO: fix this story proper, so that the transformation
// makes sense without resorting to: everyone else
// does it like this as well.
- assert(0x0 != GOT_S);
+ CHECK(0x0 != GOT_S);
return Page(GOT_S+A) - Page(P);
}
case COMPAT_R_AARCH64_LD64_GOT_LO12_NC: {
// G(GDAT(S+A))
- assert(0x0 != GOT_S);
+ CHECK(0x0 != GOT_S);
return (GOT_S + A) & 0xfff;
}
default:
@@ -297,7 +296,7 @@ relocateObjectCodeAarch64(ObjectCode * oc) {
relTab->sectionHeader->sh_link,
ELF64_R_SYM((Elf64_Xword)rel->r_info));
- assert(0x0 != symbol);
+ CHECK(0x0 != symbol);
/* decode implicit addend */
int64_t addend = decodeAddendAarch64(targetSection, rel);
@@ -323,8 +322,8 @@ relocateObjectCodeAarch64(ObjectCode * oc) {
relaTab->sectionHeader->sh_link,
ELF64_R_SYM((Elf64_Xword)rel->r_info));
- assert(0x0 != symbol);
- assert(0x0 != symbol->addr);
+ CHECK(0x0 != symbol);
+ CHECK(0x0 != symbol->addr);
/* take explicit addend */
int64_t addend = rel->r_addend;
=====================================
rts/posix/OSThreads.c
=====================================
@@ -401,8 +401,9 @@ interruptOSThread (OSThreadId id)
void
joinOSThread (OSThreadId id)
{
- if (pthread_join(id, NULL) != 0) {
- sysErrorBelch("joinOSThread: error %d", errno);
+ int ret = pthread_join(id, NULL);
+ if (ret != 0) {
+ sysErrorBelch("joinOSThread: error %d", ret);
}
}
=====================================
rts/sm/Evac.c
=====================================
@@ -109,6 +109,8 @@ alloc_for_copy (uint32_t size, uint32_t gen_no)
//
// However, if we are in a deadlock detection GC then we disable aging
// so there is no need.
+ //
+ // See Note [Non-moving GC: Marking evacuated objects].
if (major_gc && !deadlock_detect_gc)
markQueuePushClosureGC(&gct->cap->upd_rem_set.queue, (StgClosure *) to);
return to;
@@ -134,6 +136,52 @@ alloc_for_copy (uint32_t size, uint32_t gen_no)
The evacuate() code
-------------------------------------------------------------------------- */
+/*
+ * Note [Non-moving GC: Marking evacuated objects]
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ *
+ * When the non-moving collector is in use we must be careful to ensure that any
+ * references to objects in the non-moving generation from younger generations
+ * are pushed to the mark queue.
+ *
+ * In particular we need to ensure that we handle newly-promoted objects are
+ * correctly marked. For instance, consider this case:
+ *
+ * generation 0 generation 1
+ * ────────────── ──────────────
+ *
+ * ┌───────┐
+ * ┌───────┐ │ A │
+ * │ B │ ◁────────────────────────── │ │
+ * │ │ ──┬─────────────────┐ └───────┘
+ * └───────┘ ┆ after GC │
+ * ┆ │
+ * ┌───────┐ ┆ before GC │ ┌───────┐
+ * │ C │ ◁┄┘ └─────▷ │ C' │
+ * │ │ │ │
+ * └───────┘ └───────┘
+ *
+ *
+ * In this case object C started off in generation 0 and was evacuated into
+ * generation 1 during the preparatory GC. However, the only reference to C'
+ * is from B, which lives in the generation 0 (via aging); this reference will
+ * not be visible to the concurrent non-moving collector (which can only
+ * traverse the generation 1 heap). Consequently, upon evacuating C we need to
+ * ensure that C' is added to the update remembered set as we know that it will
+ * continue to be reachable via B (which is assumed to be reachable as it lives
+ * in a younger generation).
+ *
+ * Where this happens depends upon the type of the object (e.g. C'):
+ *
+ * - In the case of "normal" small heap-allocated objects this happens in
+ * alloc_for_copy.
+ * - In the case of compact region this happens in evacuate_compact.
+ * - In the case of large objects this happens in evacuate_large.
+ *
+ * See also Note [Aging under the non-moving collector] in NonMoving.c.
+ *
+ */
+
/* size is in words
We want to *always* inline this as often the size of the closure is static,
@@ -356,6 +404,9 @@ evacuate_large(StgPtr p)
__atomic_fetch_or(&bd->flags, BF_EVACUATED, __ATOMIC_ACQ_REL);
if (RTS_UNLIKELY(RtsFlags.GcFlags.useNonmoving && new_gen == oldest_gen)) {
__atomic_fetch_or(&bd->flags, BF_NONMOVING, __ATOMIC_ACQ_REL);
+
+ // See Note [Non-moving GC: Marking evacuated objects].
+ markQueuePushClosureGC(&gct->cap->upd_rem_set.queue, (StgClosure *) p);
}
initBdescr(bd, new_gen, new_gen->to);
@@ -510,6 +561,9 @@ evacuate_compact (StgPtr p)
bd->flags |= BF_EVACUATED;
if (RTS_UNLIKELY(RtsFlags.GcFlags.useNonmoving && new_gen == oldest_gen)) {
__atomic_fetch_or(&bd->flags, BF_NONMOVING, __ATOMIC_RELAXED);
+
+ // See Note [Non-moving GC: Marking evacuated objects].
+ markQueuePushClosureGC(&gct->cap->upd_rem_set.queue, (StgClosure *) str);
}
initBdescr(bd, new_gen, new_gen->to);
@@ -695,13 +749,6 @@ loop:
*/
if (flags & BF_LARGE) {
evacuate_large((P_)q);
-
- // We may have evacuated the block to the nonmoving generation. If so
- // we need to make sure it is added to the mark queue since the only
- // reference to it may be from the moving heap.
- if (major_gc && flags & BF_NONMOVING && !deadlock_detect_gc) {
- markQueuePushClosureGC(&gct->cap->upd_rem_set.queue, q);
- }
return;
}
=====================================
rts/sm/GC.c
=====================================
@@ -1701,13 +1701,8 @@ collect_gct_blocks (void)
static void
collect_pinned_object_blocks (void)
{
- generation *gen;
const bool use_nonmoving = RtsFlags.GcFlags.useNonmoving;
- if (use_nonmoving && major_gc) {
- gen = oldest_gen;
- } else {
- gen = g0;
- }
+ generation *const gen = (use_nonmoving && major_gc) ? oldest_gen : g0;
for (uint32_t n = 0; n < n_capabilities; n++) {
bdescr *last = NULL;
@@ -1732,7 +1727,7 @@ collect_pinned_object_blocks (void)
if (gen->large_objects != NULL) {
gen->large_objects->u.back = last;
}
- g0->large_objects = RELAXED_LOAD(&capabilities[n]->pinned_object_blocks);
+ gen->large_objects = RELAXED_LOAD(&capabilities[n]->pinned_object_blocks);
RELAXED_STORE(&capabilities[n]->pinned_object_blocks, NULL);
}
}
=====================================
rts/sm/NonMoving.c
=====================================
@@ -191,8 +191,8 @@ Mutex concurrent_coll_finished_lock;
* === Other references ===
*
* Apart from the design document in docs/storage/nonmoving-gc and the Ueno
- * 2016 paper (TODO citation) from which it drew inspiration, there are a
- * variety of other relevant Notes scattered throughout the tree:
+ * 2016 paper [ueno 2016] from which it drew inspiration, there are a variety
+ * of other relevant Notes scattered throughout the tree:
*
* - Note [Concurrent non-moving collection] (NonMoving.c) describes
* concurrency control of the nonmoving collector
@@ -204,6 +204,10 @@ Mutex concurrent_coll_finished_lock;
* - Note [Aging under the non-moving collector] (NonMoving.c) describes how
* we accommodate aging
*
+ * - Note [Non-moving GC: Marking evacuated objects] (Evac.c) describes how
+ * non-moving objects reached by evacuate() are marked, which is necessary
+ * due to aging.
+ *
* - Note [Large objects in the non-moving collector] (NonMovingMark.c)
* describes how we track large objects.
*
@@ -232,6 +236,11 @@ Mutex concurrent_coll_finished_lock;
* how we use the DIRTY flags associated with MUT_VARs and TVARs to improve
* barrier efficiency.
*
+ * [ueno 2016]:
+ * Katsuhiro Ueno and Atsushi Ohori. 2016. A fully concurrent garbage
+ * collector for functional programs on multicore processors. SIGPLAN Not. 51,
+ * 9 (September 2016), 421–433. DOI:https://doi.org/10.1145/3022670.2951944
+ *
*
* Note [Concurrent non-moving collection]
* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -312,6 +321,8 @@ Mutex concurrent_coll_finished_lock;
*
* The non-moving collector will come to C in the mark queue and mark it.
*
+ * The implementation details of this are described in Note [Non-moving GC:
+ * Marking evacuated objects] in Evac.c.
*
* Note [Deadlock detection under the non-moving collector]
* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -726,7 +737,6 @@ void nonmovingStop(void)
"waiting for nonmoving collector thread to terminate");
ACQUIRE_LOCK(&concurrent_coll_finished_lock);
waitCondition(&concurrent_coll_finished, &concurrent_coll_finished_lock);
- joinOSThread(mark_thread);
}
#endif
}
=====================================
rts/win32/veh_excn.c
=====================================
@@ -10,7 +10,6 @@
#include "ghcconfig.h"
#include "veh_excn.h"
#include "LinkerInternals.h"
-#include <assert.h>
#include <stdbool.h>
#include <dbghelp.h>
#include <shellapi.h>
@@ -195,7 +194,7 @@ void __register_hs_exception_handler( void )
__hs_handle = AddVectoredContinueHandler(CALL_LAST,
__hs_exception_handler);
// should the handler not be registered this will return a null.
- assert(__hs_handle);
+ CHECK(__hs_handle);
// Register for an exception filter to ensure the continue handler gets
// hit if no one handled the exception.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/1d18e37b595c91f6fd4ab8b50fdd6f747b5fa235...706c924bc15979f349c9cc6671910972b04b251c
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/1d18e37b595c91f6fd4ab8b50fdd6f747b5fa235...706c924bc15979f349c9cc6671910972b04b251c
You're receiving this email because of your account on gitlab.haskell.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20201129/f7cb17d5/attachment-0001.html>
More information about the ghc-commits
mailing list