[GHC] #8591: Concurrent executions of ghc-pkg can cause inconstant package.cache files

[GHC]

#8591: Concurrent executions of ghc-pkg can cause inconstant package.cache files
-------------------------------------+-------------------------------------
              Reporter:  janm        |            Owner:  pgj
                  Type:  bug         |           Status:  new
              Priority:  normal      |        Milestone:
             Component:  Package     |          Version:  7.6.3
  system                             |         Keywords:  ghc-pkg race
            Resolution:              |     Architecture:  Unknown/Multiple
      Operating System:  FreeBSD     |       Difficulty:  Unknown
       Type of failure:  Other       |       Blocked By:
             Test Case:              |  Related Tickets:
              Blocking:              |
Differential Revisions:              |
-------------------------------------+-------------------------------------

Comment (by janm):

 I am running a test  build now but code inspection shows that the same
 race is present. I expect it to fail.

 We resolved the issue by single-threading all package installation in the
 build process. We are running on 9.3-p5 at the moment -- It was probably
 9.2-RELEASE or 9-STABLE at the time of the original bug report.

 I also see that the FreeBSD port for lang/ghc limits concurrency to 4
 jobs, added in SVN rev 348842. Assuming you're also pgj at freebsd you
 committed the change, so probably know! This was in response to FreeBSD
 bug ports/186829 (see
 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=186829 ).

 The same problem seems to be the underlying cause. The error trace in the
 bug report shows cabal failing to install a package, just like I saw in my
 system build processes. Resolving the problem in ghc-pkg should resolve
 this problem and allow lang/ghc builds to run with full parallelism.

 In principle, the withFileAtomic function should (in order):

 1. take a lock on the target file (creating it if it is not present)
 2. create the temporary file
 3. write to the temporary file
 4. close the temporary file
 5. rename the temporary file on top of the locked target file
 6. close the original target file which is now no longer present in the
 file system.

 Concurrent execution will now be serialised around the lock on the target
 file.

 This doesn't resolve the "big hairy race condition" on Windows, but it
 should resolve the big hairy race condition I'm hitting that isn't
 mentioned code comments.

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/8591#comment:5>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler