patch applied (cabal): make rawSystemStdout put its temp files in the temp dir rather than cwd

[Duncan Coutts]

On Fri, 2007-12-28 at 09:54 -0500, Gregory Wright wrote:
> Hi Duncan,
> 
> (Cross-posting to ghc-users since some of the issues were brought up  
> there.)
> 
> On Dec 2, 2007, at 5:04 PM, Duncan Coutts wrote:
> 
> > Sun Dec  2 14:06:20 PST 2007  Duncan Coutts <duncan at haskell.org>
> >  * make rawSystemStdout put its temp files in the temp dir rather  
> > than cwd
> >  Should fixe reported wierdness with finding program version numbers
> >
> >    M ./Distribution/Simple/Utils.hs -2 +4
> >
> 
> I saw this problem when trying to build 6.8.1 on a mac (Leopard/Intel)
> using 6.4 to build compiler.  I edited rawSystemStdout to put the tmp
> files directory in "/tmp" instead of using getTemporaryDirectory. On
> this combination of compilers, moving the temporary file did not fix
> the problem of finding the program version number.  The underlying
> error was that the temp file was reported to still be locked.

I've no idea what the problem could be but I would not expect changing
the directory where the temp files are put to be related.

What can cause file locked errors? I thought the OS did not enforce any
locking and it was only the RTS that did so, and only within a process,
not between processes using OS voluntary locking apis.

As far as I know, only Windows enforces these kinds of file locks (like
preventing deleting of open files).

> Building 6.8.2 with another 6.8.2 eliminated the program version  
> problem, but I have seen two other intermittent errors which seem to
> be related.  (Reported in trac as issue #1992.)  Building 6.8.2 with
> 6.8.2 using the MacPorts infrastructure, I get this error about half
> the time:

> 	../compiler/stage1/ghc-inplace -M -optdep-f -optdep.depend-BASE  - 
> osuf o -I../includes   -H16m -O -I/opt/local/include -I/usr/include -L/ 
[snip]
> .depend-BASE: openFile: resource busy (file is locked)

Cabal never directly calls ghc to generate Makefile style dependencies,
though ghc's build system does use Cabal to generate Makefiles which
call ghc to generate .depend files.

> When I build in an eshell running under Aquamacs, I've consistently  
> had the build fail with:
> 
> ../utils/mkdependC/mkdependC -f .depend-BASE -D__GLASGOW_HASKELL__=608  
> -I../includes -Iutils -IbasicTypes -Itypes -IhsSyn -Iprelude -Irename - 
> Itypecheck -IdeSugar -IcoreSyn -Ivectorise -Ispecialise -IsimplCore - 
> Istranal -IstgSyn -IsimplStg -IcodeGen -Imain -Iprofiling -Iparser - 
> IcprAnalysis -IndpFlatten -Iiface -Icmm -InativeGen -Ighci -I../ 
> includes    -- -O -I/opt/local/include -Iparser -I. -O    -- parser/ 
> cutils.c parser/hschooks.c
> /opt/local/bin/perl -pe 'binmode(stdin); binmode(stdout);  
> s@(\S*[._]o)@stage2/$1 at g; s@(\S*[._]hi)@stage2/$1 at g; s@^.*/compat.* 
> $@@g;' <.depend-BASE >.depend-2
> /usr/bin/make -C compiler stage=2
> ../compiler/stage1/ghc-inplace -cpp  stage2/ghc-inplace.c -o stage2/ 
> ghc-inplace
> ld: in /Users/gwright/Desktop/ghc-6.8.2/libraries/haskell98/dist/build/ 
> libHShaskell98-1.0.1.0.a, archive has no table of contents
> collect2: ld returned 1 exit status
> make[2]: *** [stage2/ghc-inplace] Error 1
> make[1]: *** [stage2] Error 2
> make: *** [bootstrap2] Error 2

Again, this is running from a Makefile, not from Cabal directly.

> In this last case, it looks as if libHShaskell98-1.0.1.0.a is being  
> used before ranlib has been run on it.

The interesting thing is why this happens in your environment and not
more generally. Admittedly, looking at the Makefile that cabal generates
it does not mention ranlib but then if that were the problem we would
expect all builds on OSX to fail always, which is clearly not the case.

So again, I don't know what's going on :-)

> The common theme of these errors is that they all come out of Cabal,

Actually all of these are Makefiles, though possibly Makefiles generated
by Cabal.

> and seem to involve using the results of external processes before
> they are ready.  In the original case of the temporary file used to
> record the version number, runProcess/waitForProcess are used.  In the
> last case (running ranlib), runProcessPosix/waitForProcess are used.

runProcessPosix?

> Since the bug seems to be a race condition, it's not surprising that  
> changing the code can cause it to disappear for a while.  I will
> continue to look at what runProcess and waitForProcess are doing.  Do
> you know of anything in Cabal that might confuse the handling of
> external processes?

No. It uses rawSystem for most purposes which waits for process
termination or for the rawSystemStdout it uses runProcess and
waitForProcess. However, as I said, the above examples were from
Makefiles not from code in Cabal.

> Were the original reports of weirdness in detecting the version number
> on OS X, or another operating system?

They were OS independent, it was just due to using a dir that was not
writable rather than the proper tmp directory. It was not related to
locking.

The only other similar bug I've seen was on Windows where cabal-install
sometimes fails to delete a file in a build directory due to it being
locked, even though the last process to use that file has terminated.

So all in all, I'm confused.

Duncan