Windows test failures

Tue Dec 4 11:52:33 UTC 2018

Hi Tamar,
WINDOWS=======On Windows I did the following changes before running the
'plugin09' test: 1.) In the compiler (HscMain.hs), just before calling
the plugin function 'parsedPlugin', I set BufferMode of the file stdout
to LineBuffering. 2.) At the same place, I write a message to stdout
with the text "COMPILER About to call plugin parse" and the result of
the buffer-mode query. 3.) In the plugin, in the function parsedPlugin,
I query and print (to stdout) the buffer mode. 4.) I added the heading
PLUGIN to the normal parse message issued by the parsedPlugin function
5.) In the compiler (HscMain.hs) just after returning from the plugin,
I print the line "COMPILER Returning from plugin parse" to stdout. 6.)
In the  plugin function interfaceLoadPlugin' that is called much later,
I flush stdout, and add the heading "PLUGIN".
This gives the following interesting result:
 COMPILER About to call plugin parse: LineBuffering COMPILER Returning
from plugin parse PLUGIN Buffermode: BlockBuffering Nothing PLUGIN
parsePlugin(a,b) PLUGIN interfacePlugin: Prelude ...
The output lines do not appear in the sequence they were produced!!The
plugin doesn't see/inherit the BlockBuffer mode (LineBuffering) set by
the compiler!!
This is a strong indication, that there are two different buffers for
stdout. One in the compiler and another one in the plugin.At the end of
the processing, the buffer in the compiler is automatically flushed,
however the buffer in the plugin never gets flushed!
LINUX=====I did a similar test in Linux, however, here I set the buffer
mode to 'Blockmode Nothing' and I didn't do a manual flush in the
plugin. I got the following result:
 COMPILER About to call plugin parse: Buffering mode: BlockBuffering
Nothing PLUGIN Buffering: BlockBuffering Nothing PLUGIN
parsePlugin(a,b) COMPILER Returning from plugin parse PLUGIN
interfacePlugin: Prelude ...
Here the lines are in the same order as they were produced.The setting
of the Buffering mode is inherited by the plugin.
I think, on Linux the compiler and the plugin share the same buffer.
To fix the issue on Windows, the compiler and the plugin should use the
same buffer for stdout.However I don't know whether this is possible /
difficult / easy?What's your opinion?
Many thanks and kind regards   Roland
Here are my changes for Windows in code:
Change in HscMain the line "import System.IO (fixIO)" to "import
System.IO"
Last lines of function HscMain.hs:hscParse'
            -- apply parse transformation of plugins            let
applyPluginAction p opts                  = parsedResultAction p opts
mod_summary            liftIO $ hSetBuffering stdout
LineBuffering            mode <- liftIO $ hGetBuffering
stdout            liftIO $ putStrLn ("COMPILER About to call plugin
parse: " ++ show mode)            rsxresult <- withPlugins dflags
applyPluginAction res            liftIO $ putStrLn "COMPILER Returning
from plugin parse"            return rsxresult
New code for function SourcePlugin.hs:parsedPlugin
parsedPlugin opts _ pm  = do       mode <- liftIO $ hGetBuffering
stdout       liftIO $ putStrLn $ "PLUGIN Buffermode: " ++ show
mode       liftIO $ putStrLn $ "PLUGIN parsePlugin(" ++ intercalate ","
opts ++ ")"       return pm
New code for function SourcePlugin.hs:interfaceLoadPlugin'
interfaceLoadPlugin' :: [CommandLineOption] -> ModIface -> IfM lcl
ModIfaceinterfaceLoadPlugin' _ iface  = do liftIO $ putStrLn $ "PLUGIN
interfacePlugin: "                              ++ (showSDocUnsafe $
ppr $ mi_module iface)       liftIO $ hFlush stdout       return iface


Am Dienstag, den 04.12.2018, 00:02 +0000 schrieb Phyx:
> Hi Roland,
> 
> Thanks for looking into these.
> 
> > 
> I looked into the testcases 'plugins09', 'plugins10' and 'plugins11'
> and
>  found the following: GHC-Windows uses BufferMode 'BlockBuffering 
> Nothing', however, GHC-Linux uses 'LineBuffering'. 
> 
> Ah, yes, this isn't technically a Linux vs Windows thing, GHC will
> always default to LineBuffering for terminals and BlockBuffering for
> anything else. The issue is that msys2 terminals have std handles
> that are backed by files instead of pipes. This is an artifact of how
> they try to emulate posix shells, See https://github.com/msys2/msys2/
> wiki/Porting#standard-streams-in-mintty.  
> This means that when GHC runs inside an msys2 program such as bash
> it'll always default to BlockBuffering.
> 
> 
> 
> > 
> I don't know anything about the "Why" and "Where" in the GHC IO
> module on Windows, so I'm unable to come up with a patch. 
> 
> The above said the handles should be getting flushed when the
> finalizers are run, but these aren't 100% guaranteed so if the tests
> rely on this then your solution (to force buffer mode to
> LineBuffering) sounds like perfectly adequate.  Could you put a patch
> up with that?
> 
> My new I/O manager takes a different approach to determine the buffer
> mode, but I still have some kinks to work out before posting it.
> 
> > 
> PS: I can't say anything about the tests 'plugin-recomp-pure' and 
> 'plugin-recomp-impure' as these tests run successfully on my (slow) 
> Windows box. 
> 
> These don't fail for me or harbormaster either, so if Simon is able
> to consistently reproduce these then I'll have to ask him for a core
> dump so I can take a look.
> 
> Thanks again,
> I appreciate the help!
> 
> Regards,
> Tamar
> 
> 
> On Mon, Dec 3, 2018 at 3:34 PM Roland Senn <rsx at bluewin.ch> wrote:
> > Hi Tamar,
> > 
> > I looked into the testcases 'plugins09', 'plugins10' and
> > 'plugins11' and found the following: GHC-Windows uses BufferMode
> > 'BlockBuffering Nothing', however, GHC-Linux uses 'LineBuffering'.
> > 
> > If I add an 'import System.IO' and the line 'liftIO $ hSetBuffering
> > stdout LineBuffer' as first line in the do block of the function
> > '.../testuite/tests/plugins/simple-
> > plugin/Simple/SourcePlugin.hs:parsedPlugin' then all 3 tests pass
> > successfully on Windows!
> > 
> > I don't know anything about the "Why" and "Where" in the GHC IO
> > module on Windows, so I'm unable to come up with a patch.
> > 
> > Regards
> >    Roland
> > 
> > 
> > PS: I can't say anything about the tests 'plugin-recomp-pure' and
> > 'plugin-recomp-impure' as these tests run successfully on my (slow)
> > Windows box.
> > 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/ghc-devs/attachments/20181204/63d58ac3/attachment.html>