[Git][ghc/ghc][wip/T24150] 11 commits: Escape multiple arguments in the settings file

Cheng Shao (@TerrorJack) gitlab at gitlab.haskell.org
Tue Mar 19 20:39:35 UTC 2024 


Cheng Shao pushed to branch wip/T24150 at Glasgow Haskell Compiler / GHC


Commits:
31bf85ee by Fendor at 2024-03-19T14:48:08-04:00
Escape multiple arguments in the settings file

Uses responseFile syntax.

The issue arises when GHC is installed on windows into a location that
has a space, for example the user name is 'Fake User'.
The $topdir will also contain a space, consequentially.
When we resolve the top dir in the string `-I$topdir/mingw/include`,
then `words` will turn this single argument into `-I/C/Users/Fake` and
`User/.../mingw/include` which trips up the flag argument parser of
various tools such as gcc or clang.
We avoid this by escaping the $topdir before replacing it in
`initSettngs`.
Additionally, we allow to escape spaces and quotation marks for
arguments in `settings` file.

Add regression test case to count the number of options after variable
expansion and argument escaping took place.
Additionally, we check that escaped spaces and double quotation marks are
correctly parsed.

- - - - -
f45f700e by Matthew Pickering at 2024-03-19T14:48:44-04:00
Read global package database from settings file

Before this patch, the global package database was always assumed to be
in libdir </> package.conf.d.

This causes issues in GHC's build system because there are sometimes
situations where the package database you need to use is not located in
the same place as the settings file.

* The stage1 compiler needs to use stage1 libraries, so we should set
  "Global Package DB" for the stage1 compiler to the stage1 package
  database.
* Stage 2 cross compilers need to use stage2 libraries, so likewise, we
  should set the package database path to `_build/stage2/lib/`

* The normal situation is where the stage2 compiler uses stage1
  libraries. Then everything lines up.

* When installing we have rearranged everything so that the settings
  file and package database line up properly, so then everything should
  continue to work as before. In this case we set the relative package
  db path to `package.conf.d`, so it resolves the same as before.

* ghc-pkg needs to be modified as well to look in the settings file fo
  the package database rather than assuming the global package database
  location relative to the lib folder.

* Cabal/cabal-install will work correctly because they query the global
  package database using `--print-global-package-db`.

A reasonable question is why not generate the "right" settings files in
the right places in GHC's build system. In order to do this you would
need to engineer wrappers for all executables to point to a specific
libdir. There are also situations where the same package db is used by
two different compilers with two different settings files (think stage2
cross compiler and stage3 compiler).

In short, this 10 line patch allows for some reasonable simplifications
in Hadrian at very little cost to anything else.

Fixes #24502

- - - - -
4c8f1794 by Matthew Pickering at 2024-03-19T14:48:44-04:00
hadrian: Remove stage1 testsuite wrappers logic

Now instead of producing wrappers which pass the global package database
argument to ghc and ghc-pkg, we write the location of the correct
package database into the settings file so you can just use the intree
compiler directly.

- - - - -
da0d8ba5 by Matthew Craven at 2024-03-19T14:49:20-04:00
Remove unused ghc-internal module "GHC.Internal.Constants"

- - - - -
b56d2761 by Matthew Craven at 2024-03-19T14:49:20-04:00
CorePrep: Rework lowering of BigNat# literals

Don't use bigNatFromWord#, because that's terrible:
 * We shouldn't have to traverse a linked list at run-time
   to build a BigNat# literal. That's just silly!
 * The static List object we have to create is much larger
   than the actual BigNat#'s contents, bloating code size.
 * We have to read the corresponding interface file,
   which causes un-tracked implicit dependencies. (#23942)

Instead, encode them into the appropriate platform-dependent
sequence of bytes, and generate code that copies these bytes
at run-time from an Addr# literal into a new ByteArray#.
A ByteArray# literal would be the correct thing to generate,
but these are not yet supported; see also #17747.

Somewhat surprisingly, this change results in a slight
reduction in compiler allocations, averaging around 0.5%
on ghc's compiler performance tests, including when compiling
programs that contain no bignum literals to begin with.
The specific cause of this has not been investigated.

Since this lowering no longer reads the interface file for
GHC.Num.BigNat, the reasoning in Note [Depend on GHC.Num.Integer]
is obsoleted.  But the story of un-tracked built-in dependencies
remains complex, and Note [Tracking dependencies on primitives]
now exists to explain this complexity.

Additionally, many empty imports have been modified to refer to
this new note and comply with its guidance.  Several empty imports
necessary for other reasons have also been given brief explanations.

Metric Decrease:
    MultiLayerModulesTH_OneShot

- - - - -
349ea330 by Fendor at 2024-03-19T14:50:00-04:00
Eliminate thunk in 'IfaceTyCon'

Heap analysis showed that `IfaceTyCon` retains a thunk to
`IfaceTyConInfo`, defeating the sharing of the most common instances of
`IfaceTyConInfo`.
We make sure the indirection is removed by adding bang patterns to
`IfaceTyCon`.

Experimental results on the agda code base, where the `mi_extra_decls`
were read from disk:

Before this change, we observe around 8654045 instances of:

`IfaceTyCon[Name,THUNK_1_0]`

But these thunks almost exclusively point to a shared value!

Forcing the thunk a little bit more, leads to `ghc-debug` reporting:

`IfaceTyCon[Name:Name,IfaceTyConInfo]`

and a noticeable reduction of live bytes (on agda ~10%).

- - - - -
594bee0b by Krzysztof Gogolewski at 2024-03-19T14:50:36-04:00
Minor misc cleanups

- GHC.HsToCore.Foreign.JavaScript: remove dropRuntimeRepArgs;
  boxed tuples don't take RuntimeRep args
- GHC.HsToCore.Foreign.Call: avoid partial pattern matching
- GHC.Stg.Unarise: strengthen the assertion; we can assert that
  non-rubbish literals are unary rather than just non-void
- GHC.Tc.Gen.HsType: make sure the fsLit "literal" rule fires
- users_guide/using-warnings.rst: remove -Wforall-identifier,
  now deprecated and does nothing
- users_guide/using.rst: fix formatting
- andy_cherry/test.T: remove expect_broken_for(23272...), 23272 is fixed

The rest are simple cleanups.

- - - - -
cf55a54b by Ben Gamari at 2024-03-19T14:51:12-04:00
mk/relpath: Fix quoting

Previously there were two instances in this script which lacked proper
quoting. This resulted in `relpath` invocations in the binary
distribution Makefile producing incorrect results on Windows, leading to
confusing failures from `sed` and the production of empty package
registrations.

Fixes #24538.

- - - - -
5ff88389 by Bryan Richter at 2024-03-19T14:51:48-04:00
testsuite: Disable T21336a on wasm

- - - - -
64f3c101 by Cheng Shao at 2024-03-19T20:39:25+00:00
rts: do not prefetch mark_closure bdescr in non-moving gc when ASSERTS_ENABLED

This commit fixes a small an oversight in !12148: the prefetch logic
in non-moving GC may trap in debug RTS because it calls Bdescr() for
mark_closure which may be a static one. It's fine in non-debug RTS
because even invalid bdescr addresses are prefetched, they will not
cause segfaults, so this commit implements the most straightforward
fix: don't prefetch mark_closure bdescr when assertions are enabled.

- - - - -
8d857b78 by Teo Camarasu at 2024-03-19T20:39:25+00:00
rts: Allocate non-moving segments with megablocks

Non-moving segments are 8 blocks long and need to be aligned.
Previously we serviced allocations by grabbing 15 blocks, finding
an aligned 8 block group in it and returning the rest.
This proved to lead to high levels of fragmentation as a de-allocating a segment
caused an 8 block gap to form, and this could not be reused for allocation.

This patch introduces a segment allocator based around using entire
megablocks to service segment allocations in bulk.

When there are no free segments, we grab an entire megablock and fill it
with aligned segments. As the megablock is free, we can easily guarantee
alignment. Any unused segments are placed on a free list.

It only makes sense to free segments in bulk when all of the segments in
a megablock are freeable. After sweeping, we grab the free list, sort it,
and find all groups of segments where they cover the megablock and free
them.
This introduces a period of time when free segments are not available to
the mutator, but the risk that this would lead to excessive allocation
is low. Right after sweep, we should have an abundance of partially full
segments, and this pruning step is relatively quick.

In implementing this we drop the logic that kept NONMOVING_MAX_FREE
segments on the free list.

See Note [Segment allocation strategy]

Resolves #24150

- - - - -


30 changed files:

- compiler/GHC/Builtin/Names.hs
- compiler/GHC/Builtin/PrimOps.hs-boot
- compiler/GHC/Builtin/Types/Prim.hs
- compiler/GHC/Cmm/Dominators.hs
- compiler/GHC/Core/LateCC/OverloadedCalls.hs
- compiler/GHC/Core/Opt/Simplify/Iteration.hs
- compiler/GHC/Core/TyCo/Subst.hs
- compiler/GHC/CoreToStg/Prep.hs
- compiler/GHC/Data/Word64Map/Strict.hs
- compiler/GHC/Driver/CmdLine.hs
- compiler/GHC/Driver/Config/CoreToStg/Prep.hs
- compiler/GHC/Driver/Config/Diagnostic.hs
- compiler/GHC/Driver/Errors/Ppr.hs
- compiler/GHC/Driver/Main.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/HsToCore/Expr.hs
- compiler/GHC/HsToCore/Foreign/Call.hs
- compiler/GHC/HsToCore/Foreign/JavaScript.hs
- compiler/GHC/Iface/Errors/Ppr.hs
- compiler/GHC/Iface/Syntax.hs
- compiler/GHC/Iface/Type.hs
- compiler/GHC/Iface/Type.hs-boot
- compiler/GHC/Parser/PostProcess.hs
- compiler/GHC/Platform.hs
- compiler/GHC/Settings/IO.hs
- compiler/GHC/Stg/Unarise.hs
- compiler/GHC/Stg/Utils.hs
- compiler/GHC/StgToCmm/DataCon.hs
- compiler/GHC/StgToJS/CodeGen.hs
- compiler/GHC/StgToJS/Expr.hs


The diff was not included because it is too large.


View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/1757ae3489776d8178783d869dbdb5e91cbff626...8d857b78cebcd8dbe4b57dc1c879c66f6b1cc496

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/1757ae3489776d8178783d869dbdb5e91cbff626...8d857b78cebcd8dbe4b57dc1c879c66f6b1cc496
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20240319/c23163c1/attachment-0001.html>

More information about the ghc-commits mailing list