[GHC] #9775: "Failed to remove" errors during Windows build from hsc2hs

[GHC]

#9775: "Failed to remove" errors during Windows build from hsc2hs
---------------------------------+----------------------------------------
        Reporter:  gintas        |                Owner:  Phyx-
            Type:  bug           |               Status:  new
        Priority:  normal        |            Milestone:  8.2.1
       Component:  hsc2hs        |              Version:  7.8.3
      Resolution:                |             Keywords:
Operating System:  Windows       |         Architecture:  Unknown/Multiple
 Type of failure:  None/Unknown  |            Test Case:
      Blocked By:                |             Blocking:
 Related Tickets:                |  Differential Rev(s):
       Wiki Page:                |
---------------------------------+----------------------------------------

Comment (by Tamar Christina <tamar@…>):

 In [changeset:"a3704409acc3bd237d3e872f640686918fb51f5f/ghc"
 a3704409/ghc]:
 {{{
 #!CommitTicketReference repository="ghc"
 revision="a3704409acc3bd237d3e872f640686918fb51f5f"
 Fix various issues with testsuite code on Windows

 Summary:
 Previously we would make direct calls to `diff` using `os.system`.
 On Windows `os.system` is implemented using the standard
 idiom `CreateProcess .. WaitForSingleObject ..`.

 This again runs afoul with the `_exec` behaviour on Windows. So we ran
 into some trouble where sometimes `diff` would return before it's done.

 On tests which run multiple ways, such as `8086` what happens is that
 we think the diff is done and continue. The next way tries to set things
 up again by removing any previous directory. This would then fail with
 and error saying the directory can't be removed. Which is true, because
 the previous diff code/child is still running.

 We shouldn't make any external calls to anything using `os.system`.
 Instead just use `runCmd` which uses `timeout`. This also ensures that if
 we hit the cygwin bug where diff or any other utility hangs, we kill it
 and
 continue and not hang the entire test and leave hanging processes.

 Further more we also:
 Ignore error lines from `removeFile` from tools in the testsuite. This is
 a rather large
 hammer to work around the fact that `hsc2hs` often tries to remove it's
 own file too early.
 When this is patched the workaround can be removed. See Trac #9775

 We mark `prog003` as skip. Since this test randomly fails and passes. For
 stability it's disabled
 but it is a genuine bug which we should find. It's something with
 interface files being
 overwritten. See Trac #11317

 when `rmtree` hits a readonly file, the `onerror` handler is raised
 afterwards but not
 during the tree walk. It doesn't allow you to recover and continue as we
 thought.
 Instead you have to explicitly start again. This is why sometimes even
 though we
 call `cleanup` before `os.mkdirs`, it would sometimes fail with an error
 that the
 folder already exists. So we now do a second walk.

 A new verbosity level (4) will strip the silent flags from `MAKE`
 invocations so you can actually
 see what's going on.

 Test Plan: ./validate on build bots.

 Reviewers: bgamari, austin

 Reviewed By: bgamari

 Subscribers: mpickering, thomie, #ghc_windows_task_force

 Differential Revision: https://phabricator.haskell.org/D2894

 GHC Trac Issues: #12661, #11317, #9775
 }}}

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9775#comment:9>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler