[Git][ghc/ghc][wip/winio] 118 commits: Bump Cabal submodule

Ben Gamari gitlab at gitlab.haskell.org
Tue Jul 14 14:29:04 UTC 2020 


Ben Gamari pushed to branch wip/winio at Glasgow Haskell Compiler / GHC


Commits:
7414bb9f by GHC GitLab CI at 2020-07-14T14:16:51+00:00
Bump Cabal submodule

Updates a variety of tests as Cabal is now more strict about Cabal file
form.

- - - - -
4b63e5ae by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Drop Windows Vista support, require Windows 7

- - - - -
9c6b4ea4 by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Update Windows FileSystem wrapper utilities.

- - - - -
88e15f1c by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Use SlimReaderLocks and ConditonalVariables provided by the OS instead of emulated ones

- - - - -
a5beb87b by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Small linker comment and ifdef cleanups

- - - - -
88d0181e by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Flush event logs eagerly.

- - - - -
104c4dd7 by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Refactor Buffer structures to be able to track async operations

- - - - -
4116bd45 by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Implement new Console API

- - - - -
2ed238ba by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Add IOPort synchronization primitive

- - - - -
e15ddec5 by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Add new io-manager cmdline options

- - - - -
8d80cb96 by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Init Windows console Codepage to UTF-8.

- - - - -
f0c1027b by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Add unsafeSplat to GHC.Event.Array

- - - - -
757ced44 by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Add size and iterate to GHC.Event.IntTable.

- - - - -
9f00cf41 by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Switch Testsuite to test winio by default

- - - - -
edac4e53 by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Multiple refactorings and support changes.

- - - - -
37f42e37 by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: core threaded I/O manager

- - - - -
ec55d928 by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: core non-threaded I/O manager

- - - - -
7ed5b3bd by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Fix a scheduler bug with the threaded-runtime.

- - - - -
aee27b18 by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Relaxing some constraints in io-manager.

- - - - -
282207ee by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Fix issues with non-threaded I/O manager after split.

- - - - -
be768949 by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: Remove some barf statements that are a bit strict.

- - - - -
9c1fbf81 by Andreas Klebinger at 2020-07-14T14:17:44+00:00
winio: Expand comments describing non-threaded loop

- - - - -
a7d6477a by Tamar Christina at 2020-07-14T14:17:44+00:00
winio: fix FileSize unstat-able handles

- - - - -
85159715 by Tamar Christina at 2020-07-14T14:17:45+00:00
winio: Implement new tempfile routines for winio

- - - - -
ac14ed16 by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Fix input truncation when reading from handle.

This was caused by not upholding the read buffer invariant
that bufR == bufL == 0 for empty read buffers.

- - - - -
7c1ce17f by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Fix output truncation for writes larger than buffer size

- - - - -
0359fccb by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Rewrite bufWrite.

I think it's far easier to follow the code now.
It's also correct now as I had still missed a spot
where we didn't update the offset.

- - - - -
4bd58460 by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Fix offset set by bufReadEmpty.

bufReadEmpty returns the bytes read *including* content that
was already buffered,
But for calculating the offset we only care about the number
of bytes read into the new buffer.

- - - - -
d19c5d06 by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Clean up code surrounding IOPort primitives.

According to phyx these should only be read and written once per
object. Not neccesarily in that order.

To strengthen that guarantee the primitives will now throw an
exception if we violate this invariant.

As a consequence we can eliminate some code from their primops.
In particular code dealing with multiple queued readers/writers
now simply checks the invariant and throws an exception if it
was violated. That is in contrast to mvars which will do things
like wake up all readers, queue multi writers etc.

- - - - -
48a557a8 by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Fix multi threaded threadDelay and a few other small changes.

Multithreaded threadDelay suffered from a race condition
based on the ioManagerStatus. Since the status isn't needed
for WIO I removed it completely.

This resulted in a light refactoring, as consequence we will always
wake up the IO manager using interruptSystemManager, which uses
`postQueuedCompletionStatus` internally.

I also added a few comments which hopefully makes the code easier to
dive into for the next person diving in.

- - - - -
ac11323c by Andreas Klebinger at 2020-07-14T14:17:45+00:00
wionio: Make IO subsystem check a no-op on non-windows platforms.

- - - - -
98e35ec5 by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Set handle offset when opening files in Append mode.

Otherwise we would truncate the file.

- - - - -
71fdbf8c by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Remove debug event log trace

- - - - -
42f2b188 by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Fix sqrt and openFile009 test cases

- - - - -
96d478d6 by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Allow hp2ps to build with -DDEBUG

- - - - -
7a5961ad by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Update output of T9681 since we now actually run it.

- - - - -
4e535138 by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: A few more improvements to the IOPort primitives.

- - - - -
5f9d13f1 by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Fix expected tempfiles output.

Tempfiles now works properly on windows, as such we can
delete the win32 specific output.

- - - - -
6e9fa407 by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Assign thread labels to IOManager threads.

- - - - -
a4fa84f5 by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Properly check for the tso of an incall to be zero.

- - - - -
c62059e1 by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Mark FD instances as unsupported under WINIO.

- - - - -
8522127e by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Fix threadDelay maxBound invocations.

Instead of letting the ns timer overflow now clamp it at
(maxBound :: Word64) ns. That still gives a few hundred
years.

- - - - -
b1e43de4 by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Add comments/cleanup an import in base

- - - - -
601df27a by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Mark outstanding_service_requests volatile.

As far as I know C(99) gives no guarantees for code like

    bool condition;

    ...

    while(condition)
        sleep();

that condition will be updated if it's changed by another thread.
So we are explicit here and mark it as volatile, this will force
a reload from memory on each iteration.

- - - - -
0385034f by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Make last_event a local variable

- - - - -
5a6aaf7c by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Add comment about thread safety of processCompletion.

- - - - -
a461995e by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: nonthreaded: Create io processing threads in main thread.

We now set a flag in the IO thread. The scheduler when looking for work
will check the flag and create/queue threads accordingly.

We used to create these in the IO thread. This improved performance
but caused frequent segfaults. Thread creation/allocation is only safe to
do if nothing currently accesses the storeagemanager. However without
locks in the non-threaded runtime this can't be guaranteed.

This shouldn't change performance all too much.

In the past we had:
* IO: Create/Queue thread.
* Scheduler: Runs a few times. Eventually picks up IO processing thread.

Now it's:
* IO: Set flag to queue thread.
* Scheduler: Pick up flag, if set create/queue thread. Eventually picks up IO processing thread.

- - - - -
1f11b48a by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Add an exported isHeapAlloced function to the RTS

- - - - -
72f5aa6a by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Queue IO processing threads at the front of the queue.

This will unblock the IO thread sooner hopefully leading to higher
throughput in some situations.

- - - - -
ac87ed52 by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: ThreadDelay001: Use higher resolution timer.

- - - - -
907b4667 by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Update T9681 output, disable T4808 on windows.

T4808 tests functionality of the FD interface which won't be supported
under WINIO.

T9681 just has it's expected output tweaked.

- - - - -
9717c5e9 by Andreas Klebinger at 2020-07-14T14:17:45+00:00
winio: Wake io manager once per registerTimeout.

Which is implicitly done in editTimeouts, so need to wake it
up twice.

- - - - -
3cf285b7 by Andreas Klebinger at 2020-07-14T14:17:46+00:00
winio: Update placeholder comment with actual function name.

- - - - -
a5fa1e36 by Andreas Klebinger at 2020-07-14T14:17:46+00:00
winio: Always lock win32 event queue

- - - - -
7b630395 by Andreas Klebinger at 2020-07-14T14:17:46+00:00
winio: Display thread labels when tracing scheduler events.

- - - - -
9983e9e7 by Andreas Klebinger at 2020-07-14T14:17:46+00:00
winio: Refactor non-threaded runner thread and scheduler interface.

Only use a single communication point (registerAlertableWait) to inform
the C side aobut both timeouts to use as well as outstanding requests.

Also queue a haskell processing thread after each return from alertable
waits. This way there is no risk of us missing a timer event.

- - - - -
87ec98a5 by Andreas Klebinger at 2020-07-14T14:17:46+00:00
winio: Remove outstanding_requests from runner.

We used a variable to keep track of situations where we got
entries from the IO port, but all of them had already been
canceled. While we can avoid some work that way this case
seems quite rare.

So we give up on tracking this and instead always assume at
least one of the returned entries is valid.

If that's not the case no harm is done, we just perform some
additional work. But it makes the runner easier to reason about.

In particular we don't need to care if another thread modifies
oustanding_requests after we return from waiting on the IO Port.

- - - - -
1691c26e by Tamar Christina at 2020-07-14T14:17:46+00:00
winio: Various fixes related to rebase and testdriver

- - - - -
a18f715c by Tamar Christina at 2020-07-14T14:17:46+00:00
winio: Fix rebase artifacts

- - - - -
b48ee522 by Andreas Klebinger at 2020-07-14T14:17:46+00:00
winio: Rename unsafeSplat to unsafeCopyFromBuffer

- - - - -
31a83586 by Andreas Klebinger at 2020-07-14T14:17:46+00:00
winio: Remove unused size/iterate operations from IntTable

- - - - -
bbcde8c3 by Andreas Klebinger at 2020-07-14T14:17:46+00:00
winio: Detect running IO Backend via peeking at RtsConfig

- - - - -
be753e3e by Tamar Christina at 2020-07-14T14:17:46+00:00
winio: update temp path so GCC etc can handle it.

Also fix PIPE support, clean up error casting, fix memory leaks

- - - - -
939d7203 by Ben Gamari at 2020-07-14T14:17:46+00:00
winio: Minor comments/renamings

- - - - -
f6dd6e60 by Andreas Klebinger at 2020-07-14T14:17:46+00:00
winio: Checking if an error code indicates completion is now a function.

- - - - -
a4ab2634 by Andreas Klebinger at 2020-07-14T14:17:46+00:00
winio: Small refactor in withOverlappedEx

- - - - -
784a6576 by Andreas Klebinger at 2020-07-14T14:17:46+00:00
winio: A few comments and commented out dbxIO

- - - - -
974ec46b by Andreas Klebinger at 2020-07-14T14:17:46+00:00
winio: Don't drop buffer offset in byteView/cwcharView

- - - - -
429b041b by Tamar Christina at 2020-07-14T14:17:46+00:00
winio: revert BHandle changes.

- - - - -
b5714218 by Ben Gamari at 2020-07-14T14:17:46+00:00
winio: Fix imports

- - - - -
a2620b47 by Tamar Christina at 2020-07-14T14:17:46+00:00
winio: update ghc-cabal to handle new Cabal submodule bump

- - - - -
423a469e by Ben Gamari at 2020-07-14T14:17:46+00:00
winio: Only compile sources on Windows

- - - - -
df2da8a3 by Andreas Klebinger at 2020-07-14T14:17:46+00:00
winio: Actually return Nothing on EOF for non-blocking read

- - - - -
cbb75c74 by Andreas Klebinger at 2020-07-14T14:17:46+00:00
winio: Deduplicate logic in encodeMultiByte[Raw]IO.

- - - - -
88209a68 by Andreas Klebinger at 2020-07-14T14:17:46+00:00
winio: Deduplicate openFile logic

- - - - -
0d842fed by Tamar Christina at 2020-07-14T14:17:46+00:00
winio: fix -werror issue in encoding file

- - - - -
01ab8d05 by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Don't mention windows specific functions when building on Linux.

- - - - -
d8e15ee7 by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: add a note about file locking in the RTS.

- - - - -
61c785b5 by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Add version to @since annotation

- - - - -
b481f7d0 by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Rename GHC.Conc.IOCP -> GHC.Conc.WinIO

- - - - -
01495032 by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Expand GHC.Conc.POSIX description

It now explains users may not use these functions when
using the old IO manager.

- - - - -
8099d43a by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Fix potential spaceleak in __createUUIDTempFileErrNo

- - - - -
2cf0711b by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Remove redundant -Wno-missing-signatures pragmas

- - - - -
f8645b51 by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Make it explicit that we only create one IO manager

- - - - -
b29e1b98 by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Note why we don't use blocking waits.

- - - - -
9d93ffec by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Remove commented out pragma

- - - - -
0ea48212 by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Remove redundant buffer write in Handle/Text.hs:bufReadEmpty

- - - - -
0594ce08 by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Rename SmartHandles to StdHandles

- - - - -
19cd1ef7 by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: add comment stating failure behaviour for getUniqueFileInfo.

- - - - -
aa95057b by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Update IOPort haddocks.

- - - - -
f6c52683 by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Add a note cross reference

- - - - -
918f0dcf by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Name Haskell/OS I/O Manager explicitly in Note

- - - - -
56446ea4 by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Expand BlockedOnIOCompletion description.

- - - - -
e5bf2284 by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Remove historical todos

- - - - -
e1b3cb4f by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Update note, remove debugging pragma.

- - - - -
bb6f9515 by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: flushCharReadBuffer shouldn't need to adjust offsets.

- - - - -
c738b105 by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Remove obsolete comment about cond. variables

- - - - -
c51e15fe by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: fix initial linux validate build

- - - - -
37d86f01 by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Fix ThreadDelay001 CPP

- - - - -
38de3c4a by Andreas Klebinger at 2020-07-14T14:17:47+00:00
winio: Fix openFile009 merge conflict leftover

- - - - -
890fe21c by Andreas Klebinger at 2020-07-14T14:17:48+00:00
winio: Accept T9681 output.

GHC now reports String instead of [Char].

- - - - -
0b8b2407 by Andreas Klebinger at 2020-07-14T14:17:48+00:00
winio: Fix cabal006 after upgrading cabal submodule

Demand cabal 2.0 syntax instead of >= 1.20 as required by newer cabal versions.

- - - - -
fc765ba5 by Andreas Klebinger at 2020-07-14T14:17:48+00:00
winio: Fix stderr output for ghci/linking/dyn tests.

We used to filter rtsopts, i opted to instead just accept the warning of it having no effect.
This works both for -rtsopts, as well as -with-rtsopts which winio adds.

- - - - -
89e01afc by Andreas Klebinger at 2020-07-14T14:17:48+00:00
winio: Adjust T15261b stdout for --io-manager flag.

- - - - -
74dd99d9 by Andreas Klebinger at 2020-07-14T14:17:48+00:00
winio: Adjust T5435_dyn_asm stderr

The warning about rtsopts having no consequences is expected.
So accept new stderr.

- - - - -
3cb4ea6e by Andreas Klebinger at 2020-07-14T14:17:48+00:00
winio: Also accept T7037 stderr

- - - - -
cb04f2a8 by Andreas Klebinger at 2020-07-14T14:17:48+00:00
winio: fix cabal04 by filtering rts args

- - - - -
b1233d19 by Andreas Klebinger at 2020-07-14T14:17:48+00:00
winio: fix cabal01 by accepting expected stderr

- - - - -
c35d2640 by Andreas Klebinger at 2020-07-14T14:17:48+00:00
winio: fix safePkg01 by accepting expected stderr

- - - - -
f7c41db5 by Andreas Klebinger at 2020-07-14T14:17:48+00:00
winio: fix T5435_dyn_gcc by accepting expected stderr

- - - - -
20e6175b by Andreas Klebinger at 2020-07-14T14:17:48+00:00
winio: fix tempfiles test on linux

- - - - -
7f264f8d by Andreas Klebinger at 2020-07-14T14:17:48+00:00
winio: Accept accepted stderr for T3807

- - - - -
0a7093e4 by Andreas Klebinger at 2020-07-14T14:17:48+00:00
winio: Accept accepted stderr for linker_unload

- - - - -
ea6fd1d7 by Andreas Klebinger at 2020-07-14T14:17:48+00:00
winio: Accept accepted stderr for linker_unload_multiple_objs

- - - - -
63b9ef60 by Tamar Christina at 2020-07-14T14:17:48+00:00
winio: clarify wording on conditional variables.

- - - - -
8c70997d by Tamar Christina at 2020-07-14T14:17:48+00:00
winio: clarify comment on cooked mode.

- - - - -
28968bda by Tamar Christina at 2020-07-14T14:17:48+00:00
winio: update lockfile signature and remove mistaken symbol in rts.

- - - - -
180f07a9 by Ben Gamari at 2020-07-14T14:17:48+00:00
testsuite: Add winio and winio_threaded ways

Reverts many of the testsuite changes

- - - - -


30 changed files:

- Makefile
- compiler/GHC/Builtin/Names.hs
- compiler/GHC/Builtin/Types/Prim.hs
- compiler/GHC/Builtin/primops.txt.pp
- compiler/GHC/Driver/Pipeline.hs
- compiler/GHC/StgToCmm/Prim.hs
- compiler/GHC/SysTools/Info.hs
- configure.ac
- hadrian/src/Settings/Packages.hs
- includes/rts/Constants.h
- includes/rts/FileLock.h
- includes/rts/Flags.h
- includes/rts/IOManager.h
- includes/rts/OSThreads.h
- includes/rts/storage/TSO.h
- includes/stg/MiscClosures.h
- libraries/Cabal
- + libraries/base/Control/Concurrent.hs-boot
- libraries/base/GHC/Conc/IO.hs
- + libraries/base/GHC/Conc/POSIX.hs
- + libraries/base/GHC/Conc/POSIX/Const.hsc
- libraries/base/GHC/Conc/Sync.hs
- + libraries/base/GHC/Conc/Sync.hs-boot
- + libraries/base/GHC/Conc/WinIO.hs
- libraries/base/GHC/Conc/Windows.hs
- libraries/base/GHC/ConsoleHandler.hs → libraries/base/GHC/ConsoleHandler.hsc
- libraries/base/GHC/Event/Array.hs
- libraries/base/GHC/Event/IntTable.hs
- libraries/base/GHC/Event/Internal.hs
- + libraries/base/GHC/Event/Internal/Types.hs


The diff was not included because it is too large.


View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/9514380ca0f445be9ff21fb0d1b7bedb8a22ca11...180f07a91a3b91225c178a7a9faf432fc1c9c976

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/9514380ca0f445be9ff21fb0d1b7bedb8a22ca11...180f07a91a3b91225c178a7a9faf432fc1c9c976
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20200714/13615bb1/attachment-0001.html>

More information about the ghc-commits mailing list