[Git][ghc/ghc][wip/winio] 118 commits: Bump Cabal submodule

Ben Gamari gitlab at gitlab.haskell.org
Wed Jul 15 20:41:15 UTC 2020 


Ben Gamari pushed to branch wip/winio at Glasgow Haskell Compiler / GHC


Commits:
0e3c277a by Ben Gamari at 2020-07-15T16:41:01-04:00
Bump Cabal submodule

Updates a variety of tests as Cabal is now more strict about Cabal file
form.

- - - - -
ceed994a by Tamar Christina at 2020-07-15T16:41:01-04:00
winio: Drop Windows Vista support, require Windows 7

- - - - -
00a23bfd by Tamar Christina at 2020-07-15T16:41:01-04:00
winio: Update Windows FileSystem wrapper utilities.

- - - - -
459e1c5f by Tamar Christina at 2020-07-15T16:41:01-04:00
winio: Use SlimReaderLocks and ConditonalVariables provided by the OS instead of emulated ones

- - - - -
763088fc by Tamar Christina at 2020-07-15T16:41:01-04:00
winio: Small linker comment and ifdef cleanups

- - - - -
1a228ff9 by Tamar Christina at 2020-07-15T16:41:01-04:00
winio: Flush event logs eagerly.

- - - - -
e9e04dda by Tamar Christina at 2020-07-15T16:41:01-04:00
winio: Refactor Buffer structures to be able to track async operations

- - - - -
356dc3fe by Tamar Christina at 2020-07-15T16:41:01-04:00
winio: Implement new Console API

- - - - -
90e69f77 by Tamar Christina at 2020-07-15T16:41:01-04:00
winio: Add IOPort synchronization primitive

- - - - -
71245fcc by Tamar Christina at 2020-07-15T16:41:01-04:00
winio: Add new io-manager cmdline options

- - - - -
d548a3b3 by Tamar Christina at 2020-07-15T16:41:01-04:00
winio: Init Windows console Codepage to UTF-8.

- - - - -
58ef6366 by Tamar Christina at 2020-07-15T16:41:01-04:00
winio: Add unsafeSplat to GHC.Event.Array

- - - - -
d660725e by Tamar Christina at 2020-07-15T16:41:01-04:00
winio: Add size and iterate to GHC.Event.IntTable.

- - - - -
050da6dd by Tamar Christina at 2020-07-15T16:41:01-04:00
winio: Switch Testsuite to test winio by default

- - - - -
4bf542bf by Tamar Christina at 2020-07-15T16:41:01-04:00
winio: Multiple refactorings and support changes.

- - - - -
4489af6b by Tamar Christina at 2020-07-15T16:41:02-04:00
winio: core threaded I/O manager

- - - - -
64d8f2fe by Tamar Christina at 2020-07-15T16:41:02-04:00
winio: core non-threaded I/O manager

- - - - -
8da15a09 by Tamar Christina at 2020-07-15T16:41:02-04:00
winio: Fix a scheduler bug with the threaded-runtime.

- - - - -
84ea3d14 by Tamar Christina at 2020-07-15T16:41:02-04:00
winio: Relaxing some constraints in io-manager.

- - - - -
ccf0d107 by Tamar Christina at 2020-07-15T16:41:02-04:00
winio: Fix issues with non-threaded I/O manager after split.

- - - - -
b492fe6e by Tamar Christina at 2020-07-15T16:41:02-04:00
winio: Remove some barf statements that are a bit strict.

- - - - -
01423fd2 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Expand comments describing non-threaded loop

- - - - -
4b69004f by Tamar Christina at 2020-07-15T16:41:02-04:00
winio: fix FileSize unstat-able handles

- - - - -
9b384270 by Tamar Christina at 2020-07-15T16:41:02-04:00
winio: Implement new tempfile routines for winio

- - - - -
f1e0be82 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Fix input truncation when reading from handle.

This was caused by not upholding the read buffer invariant
that bufR == bufL == 0 for empty read buffers.

- - - - -
e176b625 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Fix output truncation for writes larger than buffer size

- - - - -
a831ce0e by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Rewrite bufWrite.

I think it's far easier to follow the code now.
It's also correct now as I had still missed a spot
where we didn't update the offset.

- - - - -
6aefdf62 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Fix offset set by bufReadEmpty.

bufReadEmpty returns the bytes read *including* content that
was already buffered,
But for calculating the offset we only care about the number
of bytes read into the new buffer.

- - - - -
750ebaee by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Clean up code surrounding IOPort primitives.

According to phyx these should only be read and written once per
object. Not neccesarily in that order.

To strengthen that guarantee the primitives will now throw an
exception if we violate this invariant.

As a consequence we can eliminate some code from their primops.
In particular code dealing with multiple queued readers/writers
now simply checks the invariant and throws an exception if it
was violated. That is in contrast to mvars which will do things
like wake up all readers, queue multi writers etc.

- - - - -
ffd31db9 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Fix multi threaded threadDelay and a few other small changes.

Multithreaded threadDelay suffered from a race condition
based on the ioManagerStatus. Since the status isn't needed
for WIO I removed it completely.

This resulted in a light refactoring, as consequence we will always
wake up the IO manager using interruptSystemManager, which uses
`postQueuedCompletionStatus` internally.

I also added a few comments which hopefully makes the code easier to
dive into for the next person diving in.

- - - - -
6ec26df2 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
wionio: Make IO subsystem check a no-op on non-windows platforms.

- - - - -
29bcd936 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Set handle offset when opening files in Append mode.

Otherwise we would truncate the file.

- - - - -
55c29700 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Remove debug event log trace

- - - - -
9acb9f40 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Fix sqrt and openFile009 test cases

- - - - -
57017cb7 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Allow hp2ps to build with -DDEBUG

- - - - -
b8cd9995 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Update output of T9681 since we now actually run it.

- - - - -
10af5b14 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: A few more improvements to the IOPort primitives.

- - - - -
39afc4a7 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Fix expected tempfiles output.

Tempfiles now works properly on windows, as such we can
delete the win32 specific output.

- - - - -
99db46e0 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Assign thread labels to IOManager threads.

- - - - -
be6af732 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Properly check for the tso of an incall to be zero.

- - - - -
e2c6dac7 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Mark FD instances as unsupported under WINIO.

- - - - -
fd02ceed by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Fix threadDelay maxBound invocations.

Instead of letting the ns timer overflow now clamp it at
(maxBound :: Word64) ns. That still gives a few hundred
years.

- - - - -
bc79f9f1 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Add comments/cleanup an import in base

- - - - -
1d197f4b by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Mark outstanding_service_requests volatile.

As far as I know C(99) gives no guarantees for code like

    bool condition;

    ...

    while(condition)
        sleep();

that condition will be updated if it's changed by another thread.
So we are explicit here and mark it as volatile, this will force
a reload from memory on each iteration.

- - - - -
dc438186 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Make last_event a local variable

- - - - -
2fc957c5 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Add comment about thread safety of processCompletion.

- - - - -
4c026b6c by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: nonthreaded: Create io processing threads in main thread.

We now set a flag in the IO thread. The scheduler when looking for work
will check the flag and create/queue threads accordingly.

We used to create these in the IO thread. This improved performance
but caused frequent segfaults. Thread creation/allocation is only safe to
do if nothing currently accesses the storeagemanager. However without
locks in the non-threaded runtime this can't be guaranteed.

This shouldn't change performance all too much.

In the past we had:
* IO: Create/Queue thread.
* Scheduler: Runs a few times. Eventually picks up IO processing thread.

Now it's:
* IO: Set flag to queue thread.
* Scheduler: Pick up flag, if set create/queue thread. Eventually picks up IO processing thread.

- - - - -
f47c7208 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Add an exported isHeapAlloced function to the RTS

- - - - -
cc5d7bb1 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Queue IO processing threads at the front of the queue.

This will unblock the IO thread sooner hopefully leading to higher
throughput in some situations.

- - - - -
e7630115 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: ThreadDelay001: Use higher resolution timer.

- - - - -
451b5f96 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Update T9681 output, disable T4808 on windows.

T4808 tests functionality of the FD interface which won't be supported
under WINIO.

T9681 just has it's expected output tweaked.

- - - - -
dd06f930 by Andreas Klebinger at 2020-07-15T16:41:02-04:00
winio: Wake io manager once per registerTimeout.

Which is implicitly done in editTimeouts, so need to wake it
up twice.

- - - - -
e87d0bf9 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Update placeholder comment with actual function name.

- - - - -
fc9025db by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Always lock win32 event queue

- - - - -
c24c9a1f by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Display thread labels when tracing scheduler events.

- - - - -
06542b03 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Refactor non-threaded runner thread and scheduler interface.

Only use a single communication point (registerAlertableWait) to inform
the C side aobut both timeouts to use as well as outstanding requests.

Also queue a haskell processing thread after each return from alertable
waits. This way there is no risk of us missing a timer event.

- - - - -
256299b1 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Remove outstanding_requests from runner.

We used a variable to keep track of situations where we got
entries from the IO port, but all of them had already been
canceled. While we can avoid some work that way this case
seems quite rare.

So we give up on tracking this and instead always assume at
least one of the returned entries is valid.

If that's not the case no harm is done, we just perform some
additional work. But it makes the runner easier to reason about.

In particular we don't need to care if another thread modifies
oustanding_requests after we return from waiting on the IO Port.

- - - - -
3ebd8ad9 by Tamar Christina at 2020-07-15T16:41:03-04:00
winio: Various fixes related to rebase and testdriver

- - - - -
6be6bcba by Tamar Christina at 2020-07-15T16:41:03-04:00
winio: Fix rebase artifacts

- - - - -
2c649dc3 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Rename unsafeSplat to unsafeCopyFromBuffer

- - - - -
a18b73f3 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Remove unused size/iterate operations from IntTable

- - - - -
16bab48e by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Detect running IO Backend via peeking at RtsConfig

- - - - -
8b8405a0 by Tamar Christina at 2020-07-15T16:41:03-04:00
winio: update temp path so GCC etc can handle it.

Also fix PIPE support, clean up error casting, fix memory leaks

- - - - -
2092bc54 by Ben Gamari at 2020-07-15T16:41:03-04:00
winio: Minor comments/renamings

- - - - -
a5b5b6c0 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Checking if an error code indicates completion is now a function.

- - - - -
362176fd by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Small refactor in withOverlappedEx

- - - - -
32e20597 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: A few comments and commented out dbxIO

- - - - -
a4bfc1d9 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Don't drop buffer offset in byteView/cwcharView

- - - - -
b3ad2a54 by Tamar Christina at 2020-07-15T16:41:03-04:00
winio: revert BHandle changes.

- - - - -
3dcd87e2 by Ben Gamari at 2020-07-15T16:41:03-04:00
winio: Fix imports

- - - - -
5a371890 by Tamar Christina at 2020-07-15T16:41:03-04:00
winio: update ghc-cabal to handle new Cabal submodule bump

- - - - -
d07ebe0d by Ben Gamari at 2020-07-15T16:41:03-04:00
winio: Only compile sources on Windows

- - - - -
dcb42393 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Actually return Nothing on EOF for non-blocking read

- - - - -
895a3beb by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Deduplicate logic in encodeMultiByte[Raw]IO.

- - - - -
e06e6734 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Deduplicate openFile logic

- - - - -
b59430c0 by Tamar Christina at 2020-07-15T16:41:03-04:00
winio: fix -werror issue in encoding file

- - - - -
f8d39a51 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Don't mention windows specific functions when building on Linux.

- - - - -
6a533d2a by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: add a note about file locking in the RTS.

- - - - -
cf37ce34 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Add version to @since annotation

- - - - -
0fafa2eb by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Rename GHC.Conc.IOCP -> GHC.Conc.WinIO

- - - - -
1854fc23 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Expand GHC.Conc.POSIX description

It now explains users may not use these functions when
using the old IO manager.

- - - - -
fcc7ba41 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Fix potential spaceleak in __createUUIDTempFileErrNo

- - - - -
6b3fd9fa by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Remove redundant -Wno-missing-signatures pragmas

- - - - -
916fc861 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Make it explicit that we only create one IO manager

- - - - -
f260a721 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Note why we don't use blocking waits.

- - - - -
aa0a4bbf by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Remove commented out pragma

- - - - -
d679b544 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Remove redundant buffer write in Handle/Text.hs:bufReadEmpty

- - - - -
d3f94368 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Rename SmartHandles to StdHandles

- - - - -
bd6b8ec1 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: add comment stating failure behaviour for getUniqueFileInfo.

- - - - -
12846b85 by Andreas Klebinger at 2020-07-15T16:41:03-04:00
winio: Update IOPort haddocks.

- - - - -
9f39fb14 by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Add a note cross reference

- - - - -
62dd5a73 by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Name Haskell/OS I/O Manager explicitly in Note

- - - - -
fa807828 by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Expand BlockedOnIOCompletion description.

- - - - -
f0880a1d by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Remove historical todos

- - - - -
8e58e714 by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Update note, remove debugging pragma.

- - - - -
aa4d84d5 by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: flushCharReadBuffer shouldn't need to adjust offsets.

- - - - -
e580893a by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Remove obsolete comment about cond. variables

- - - - -
d54e9d79 by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: fix initial linux validate build

- - - - -
3cd4de46 by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Fix ThreadDelay001 CPP

- - - - -
c88b1b9f by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Fix openFile009 merge conflict leftover

- - - - -
849e8889 by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Accept T9681 output.

GHC now reports String instead of [Char].

- - - - -
e7701818 by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Fix cabal006 after upgrading cabal submodule

Demand cabal 2.0 syntax instead of >= 1.20 as required by newer cabal versions.

- - - - -
a44f0373 by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Fix stderr output for ghci/linking/dyn tests.

We used to filter rtsopts, i opted to instead just accept the warning of it having no effect.
This works both for -rtsopts, as well as -with-rtsopts which winio adds.

- - - - -
515d9896 by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Adjust T15261b stdout for --io-manager flag.

- - - - -
949aaacc by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Adjust T5435_dyn_asm stderr

The warning about rtsopts having no consequences is expected.
So accept new stderr.

- - - - -
7d424e1e by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Also accept T7037 stderr

- - - - -
1f009768 by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: fix cabal04 by filtering rts args

- - - - -
981a9f2e by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: fix cabal01 by accepting expected stderr

- - - - -
b7b0464e by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: fix safePkg01 by accepting expected stderr

- - - - -
32734b29 by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: fix T5435_dyn_gcc by accepting expected stderr

- - - - -
acc5cebf by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: fix tempfiles test on linux

- - - - -
c577b789 by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Accept accepted stderr for T3807

- - - - -
c108c527 by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Accept accepted stderr for linker_unload

- - - - -
2b0b9a08 by Andreas Klebinger at 2020-07-15T16:41:04-04:00
winio: Accept accepted stderr for linker_unload_multiple_objs

- - - - -
67afb03c by Tamar Christina at 2020-07-15T16:41:04-04:00
winio: clarify wording on conditional variables.

- - - - -
3bd41572 by Tamar Christina at 2020-07-15T16:41:04-04:00
winio: clarify comment on cooked mode.

- - - - -
ded58a03 by Tamar Christina at 2020-07-15T16:41:04-04:00
winio: update lockfile signature and remove mistaken symbol in rts.

- - - - -
2143c492 by Ben Gamari at 2020-07-15T16:41:04-04:00
testsuite: Add winio and winio_threaded ways

Reverts many of the testsuite changes

- - - - -


30 changed files:

- Makefile
- compiler/GHC/Builtin/Names.hs
- compiler/GHC/Builtin/Types/Prim.hs
- compiler/GHC/Builtin/primops.txt.pp
- compiler/GHC/Driver/Pipeline.hs
- compiler/GHC/StgToCmm/Prim.hs
- compiler/GHC/SysTools/Info.hs
- configure.ac
- hadrian/src/Settings/Packages.hs
- includes/rts/Constants.h
- includes/rts/FileLock.h
- includes/rts/Flags.h
- includes/rts/IOManager.h
- includes/rts/OSThreads.h
- includes/rts/storage/TSO.h
- includes/stg/MiscClosures.h
- libraries/Cabal
- + libraries/base/Control/Concurrent.hs-boot
- libraries/base/GHC/Conc/IO.hs
- + libraries/base/GHC/Conc/POSIX.hs
- + libraries/base/GHC/Conc/POSIX/Const.hsc
- libraries/base/GHC/Conc/Sync.hs
- + libraries/base/GHC/Conc/Sync.hs-boot
- + libraries/base/GHC/Conc/WinIO.hs
- libraries/base/GHC/Conc/Windows.hs
- libraries/base/GHC/ConsoleHandler.hs → libraries/base/GHC/ConsoleHandler.hsc
- libraries/base/GHC/Event/Array.hs
- libraries/base/GHC/Event/IntTable.hs
- libraries/base/GHC/Event/Internal.hs
- + libraries/base/GHC/Event/Internal/Types.hs


The diff was not included because it is too large.


View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/867843e0d2d7edcaa1906c8b7fffb89a0a4e92eb...2143c49273d7d87ee2f3ef1211856d60b1427af1

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/867843e0d2d7edcaa1906c8b7fffb89a0a4e92eb...2143c49273d7d87ee2f3ef1211856d60b1427af1
You're receiving this email because of your account on gitlab.haskell.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-commits/attachments/20200715/f55abd12/attachment-0001.html>

More information about the ghc-commits mailing list