[Git][ghc/ghc][wip/romes/graph-compact-easy] 44 commits: Don't store boot locations in finder cache

Tue Dec 3 00:44:34 UTC 2024

Rodrigo Mesquita pushed to branch wip/romes/graph-compact-easy at Glasgow Haskell Compiler / GHC

Commits:
44d909a3 by Sjoerd Visscher at 2024-11-19T14:38:24-05:00
Don't store boot locations in finder cache

Partially reverts commit fff55592a7b

Amends add(Home)ModuleToFinder so that locations for boot files are not stored in the finder cache.

Removes InstalledModule field from InstalledFound constructor since it's the same as the key that was searched for.

- - - - -
64c95292 by Sjoerd Visscher at 2024-11-19T14:38:24-05:00
Concentrate boot extension logic in Finder

With new mkHomeModLocation that takes an extra HscSource to add boot extensions if required.

- - - - -
11bad98d by ARATA Mizuki at 2024-11-19T14:39:08-05:00
Better documentation for floating-point min/max and SIMD primitives

See #25350 for floating-point min/max

Co-authored-by: sheaf <sam.derbyshire at gmail.com>

- - - - -
791a47b2 by Arnaud Spiwack at 2024-11-20T14:00:05+00:00
Add test for #25185

- - - - -
374e18e5 by Arnaud Spiwack at 2024-11-20T14:09:30+00:00
Quick look: emit the multiplicity of app heads in tcValArgs

Otherwise it's not scaled properly by the context, allowing unsound
expressions.

Fixes #25185.

- - - - -
1fc02399 by sheaf at 2024-11-20T18:11:03-05:00
x86 NCG: fix regUsageOfInstr for VMOVU & friends

This commit fixes the implementation of 'regUsageOfInstr' for vector
operations that take an 'Operand' as the destination, by ensuring that
when the destination is an address then the address should be *READ*,
and not *WRITTEN*.

Getting this wrong is a disaster, as it means the register allocator
has incorrect information, which can lead to it discard stores to
registers, segfaults ensuing.

Fixes #25486

- - - - -
7bd407a6 by Brandon Chinn at 2024-11-21T14:08:15-05:00
Fix CRLF in multiline strings (#25375)

- - - - -
7575709b by Rodrigo Mesquita at 2024-11-21T14:08:52-05:00
Improve reachability queries on ModuleGraph

Introduces `ReachabilityIndex`, an index constructed from a
`GHC.Data.Graph.Directed` `Graph` that supports fast reachability
queries (in $O(1)$). This abstract data structure is exposed from
`GHC.Data.Graph.Directed.Reachability`.

This index is constructed from the module graph nodes and cached in
`ModuleGraph`, enabling efficient reachability queries on the module
graph. Previously, we'd construct a Map of Set of ModuleGraph nodes
which used a lot of memory (`O(n^2)` in the number of nodes) and cache
that in the `ModuleGraph`. By using the reachability index we get rid of
this space leak in the module graph -- even though the index is still
quadratic in the number of modules, it is much, much more space
efficient due to its representation using an IntMap of IntSet as opposed
to the transitive closure we previously cached.

In a memory profile of MultiLayerModules with 100x100 modules, memory
usage improved from 6GB residency to 2.8GB, out of which roughly 1.8GB
are caused by a second space leak related to ModuleGraph. On the same
program, it brings compile time from 7.5s to 5.5s.

Note how we simplify `checkHomeUnitsClosed` in terms of
`isReachableMany` and by avoiding constructing a second graph with the
full transitive closure -- it suffices to answer the reachability query
on the full graph without collapsing the transitive closure completely
into nodes.

Unfortunately, solving this leak means we have to do a little bit more
work since we can no longer cache the result of turning vertex indices
into nodes. This results in a slight regression in MultiLayerModulesTH_Make,
but results in large performance and memory wins when compiling large
amounts of modules.

-------------------------
Metric Decrease:
    mhu-perf
Metric Increase:
    MultiLayerModulesTH_Make
-------------------------

- - - - -
bcbcdaaf by Cheng Shao at 2024-11-21T14:09:28-05:00
driver: fix hpc undefined symbol issue in TH with -fprefer-byte-code

This commit fixes an undefined symbol error in RTS linker when
attempting to compile home modules with -fhpc and
-fbyte-code-and-object-code/-fprefer-byte-code, see #25510 for
detailed description and analysis of the bug.

Also adds T25510/T25510c regression tests to test make mode/oneshot
mode of the bug.

- - - - -
970ada5a by Ben Gamari at 2024-11-22T23:32:06-05:00
gitlab-ci: Bump ci-images

For introduction of Alpine/i386 image.

Thanks to Julian for the base image.

Co-Authored-By: Julian Ospald <hasufell at hasufell.de>

- - - - -
8115abc2 by Ben Gamari at 2024-11-22T23:32:06-05:00
gitlab-ci: Add release job for i386/Alpine

As requested by Mikolaj and started by Julian.

Co-Authored-By: Julian Ospald <hasufell at hasufell.de>

- - - - -
639f0149 by Ben Gamari at 2024-11-22T23:32:06-05:00
rts/linker/Elf: Resolve _GLOBAL_OFFSET_TABLE_

- - - - -
490d4d0a by Ben Gamari at 2024-11-22T23:32:06-05:00
gitlab-ci: Mark i386 Alpine test breakages

Marks the following tests as broken on i386/Alpine:

 * T22033 due to #25497
 * simd009, T25062_V16, T25169, T22187_run due to #25498

- - - - -
536cdf09 by Cheng Shao at 2024-11-22T23:32:42-05:00
compiler: remove unused GHC.Linker.Loader.loadExpr

This patch removes the unused `GHC.Linker.Loader.loadExpr` function.
It was moved from `GHC.Runtime.Linker.linkExpr` in `ghc-9.0` to
`GHC.Linker.Loader.loadExpr` in `ghc-9.2`, and remain completely
unused and untested ever since. There's also no third party user of
this function to my best knowledge, so let's remove this. Anyone who
wants to write their own GHC API function to load bytecode can consult
the source code in older release branches.

- - - - -
6ee35024 by Drew Fenwick at 2024-11-22T23:33:26-05:00
Fix a non-compiling example in the type abstractions docs

This patch adds a missing Show constraint to a code example in the User Guide's type abstractions docs to fix issue #25422.
- - - - -
d1172e20 by Rodrigo Mesquita at 2024-11-22T23:34:02-05:00
Re-introduce ErrorCallWithLocation with a deprecation pragma

With the removal of the duplicate backtrace, part of CLC proposal #285,
the constructor `ErrorCallWithLocation` was removed from base.

This commit re-introduces it with a deprecation.

- - - - -
1187a60a by Ben Gamari at 2024-11-22T23:34:39-05:00
testsuite: Skip tests requiring Hadrian deps in out-of-tree testsuite runs

Some testsuite tests require specific tools (e.g. `check-ppr` and
`check-exact`) beyond those shipped in the binary distribution. Skip
these tests.

Fixes #13897.

- - - - -
c37d7a2e by Ben Gamari at 2024-11-22T23:34:39-05:00
testsuite: Declare exactprint tests' dependency on check-exact

- - - - -
454ce957 by Ben Gamari at 2024-11-22T23:35:15-05:00
ghc-internal: Fix a few cases of missing Haddock markup

- - - - -
a249649b by Ben Gamari at 2024-11-22T23:35:51-05:00
testsuite/GHCiPrimCall : Add missing Makefile includes

- - - - -
a021a493 by Ben Gamari at 2024-11-22T23:35:51-05:00
testsuite/IpeStats: Use Make rather than shell interpolation

- - - - -
6e1fbda7 by Ben Gamari at 2024-11-25T03:55:44-05:00
hadrian-ghci-multi: Pass -this-package-name in unit response files

As noted in #25509, the `-this-package-name` must be passed for each
package to ensure that GHC can response references to the packages'
exposed modules via package-qualified imports. Fix this.

Closes #25509.

- - - - -
a05e4a9b by Simon Hengel at 2024-11-25T03:56:33-05:00
Refactoring: Use `OnOff` more consistently for `Extension`

- - - - -
4dbe7cfe by Rodrigo Mesquita at 2024-11-25T17:42:26+00:00
mg: Drop unnecessary HasCallStack

This HasCallStack was a debugging artifact from a previous commit.

- - - - -
7c99b9fc by Rodrigo Mesquita at 2024-11-25T17:42:26+00:00
Improve haddock of graphReachabilityCyclic

- - - - -
0bd2b8c3 by Rodrigo Mesquita at 2024-11-27T14:09:00+00:00
Refactor ModuleGraph interface

The 'ModuleGraph' abstraction represents the relationship and strucutre
of the modules being compiled. This structure is meant to be constructed
once at the start of compilation, and never changed again.

However, it's exposed interface was confusing and exposed too many
footguns which led to inneficient usages of the ModuleGraph. This commit
improves significantly the exported interface of ModuleGraph, taking
into consideration the recent improvements around reachability queries.

Since the ModuleGraph graphs and related structures (HPT, EPS) are
performance critical in the sense that somewhat simple mistakes can
cause bad leaks and non-linear memory usage, we want to have proper APIs
that guide efficient usage. This is a good step in that direction.

- - - - -
30d8fd1e by Rodrigo Mesquita at 2024-11-27T14:10:32+00:00
Simplify export list of GHC.Unit.Env

- - - - -
6e9b7fc1 by Rodrigo Mesquita at 2024-11-27T14:10:32+00:00
WIP

- - - - -
7fbac08f by Rodrigo Mesquita at 2024-11-27T14:10:32+00:00
HPT HUG refactor

- - - - -
91695b13 by Rodrigo Mesquita at 2024-11-29T14:07:34+00:00
Tons of changes...

- - - - -
13596614 by Rodrigo Mesquita at 2024-11-29T15:26:41+00:00
MORE

- - - - -
de159005 by Rodrigo Mesquita at 2024-11-29T16:06:49+00:00
MORE

- - - - -
cf9f061e by Rodrigo Mesquita at 2024-12-02T11:21:27+00:00
Now it compiles.

- - - - -
f0fb9ba4 by Rodrigo Mesquita at 2024-12-02T11:52:24+00:00
Compile Again

- - - - -
65f6513f by Rodrigo Mesquita at 2024-12-02T11:52:37+00:00
OHHH ADD THESE TWO MODULES

- - - - -
163e7a7c by Rodrigo Mesquita at 2024-12-02T14:09:22+00:00
Delete HUG internal map representation unneeded functions

- - - - -
8d5dd82c by Rodrigo Mesquita at 2024-12-02T14:15:03+00:00
Finalize HUG module!

- - - - -
ba876dee by Rodrigo Mesquita at 2024-12-02T14:40:38+00:00
More

- - - - -
67e4a9aa by Rodrigo Mesquita at 2024-12-02T14:41:00+00:00
X

- - - - -
8601b7b0 by Rodrigo Mesquita at 2024-12-02T14:44:05+00:00
Remove some unused imports

- - - - -
6a81ab24 by Rodrigo Mesquita at 2024-12-02T14:49:17+00:00
more clean ups

- - - - -
b4fc8fe9 by Rodrigo Mesquita at 2024-12-02T14:49:24+00:00
progress

- - - - -
fa46859e by Rodrigo Mesquita at 2024-12-02T16:26:05+00:00
FINISH V0

- - - - -
00904291 by Rodrigo Mesquita at 2024-12-02T17:06:11+00:00
Finish GHC.hs

- - - - -

30 changed files:

- .gitattributes
- .gitlab-ci.yml
- .gitlab/generate-ci/gen_ci.hs
- .gitlab/jobs.yaml
- compiler/GHC.hs
- compiler/GHC/Builtin/primops.txt.pp
- compiler/GHC/CmmToAsm/Format.hs
- compiler/GHC/CmmToAsm/X86/Instr.hs
- compiler/GHC/Core/Opt/Pipeline.hs
- compiler/GHC/Data/Graph/Directed.hs
- + compiler/GHC/Data/Graph/Directed/Internal.hs
- + compiler/GHC/Data/Graph/Directed/Reachability.hs
- compiler/GHC/Data/IOEnv.hs
- compiler/GHC/Driver/Backpack.hs
- compiler/GHC/Driver/Config/StgToCmm.hs
- compiler/GHC/Driver/DynFlags.hs
- compiler/GHC/Driver/Env.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/Hooks.hs
- compiler/GHC/Driver/Main.hs
- compiler/GHC/Driver/Make.hs
- compiler/GHC/Driver/MakeFile.hs
- compiler/GHC/Driver/Phases.hs
- compiler/GHC/Driver/Pipeline.hs
- compiler/GHC/Driver/Pipeline/Execute.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/HsToCore/Coverage.hs
- compiler/GHC/HsToCore/Monad.hs
- compiler/GHC/HsToCore/Usage.hs
- compiler/GHC/Iface/Errors.hs

The diff was not included because it is too large.

View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/83e84702e2344285f17c5353d236290d243dacab...0090429152cedf89c751ce744a5895b92ba63744

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/83e84702e2344285f17c5353d236290d243dacab...0090429152cedf89c751ce744a5895b92ba63744
You're receiving this email because of your account on gitlab.haskell.org.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20241202/146fd7f8/attachment-0001.html>