<html><head></head><body><div>Hi Tamar,</div><div><br></div><div>WINDOWS</div><div>=======</div><div>On Windows I did the following changes before running the 'plugin09' test:</div><div> 1.) In the compiler (HscMain.hs), just before calling the plugin function 'parsedPlugin', I set BufferMode of the file stdout to LineBuffering.</div><div> 2.) At the same place, I write a message to stdout with the text "COMPILER About to call plugin parse" and the result of the buffer-mode query.</div><div> 3.) In the plugin, in the function parsedPlugin, I query and print (to stdout) the buffer mode.</div><div> 4.) I added the heading PLUGIN to the normal parse message issued by the parsedPlugin function</div><div> 5.) In the compiler (HscMain.hs) just after returning from the plugin, I print the line "COMPILER Returning from plugin parse" to stdout.</div><div> 6.) In the plugin function interfaceLoadPlugin' that is called much later, I flush stdout, and add the heading "PLUGIN".</div><div><br></div><div>This gives the following interesting result:</div><div><br></div><div><font face="monospace" size="3"> COMPILER About to call plugin parse: LineBuffering</font></div><div><font face="monospace" size="3"> COMPILER Returning from plugin parse</font></div><div><font face="monospace" size="3"> PLUGIN Buffermode: BlockBuffering Nothing</font></div><div><font face="monospace" size="3"> PLUGIN parsePlugin(a,b)</font></div><div><font face="monospace" size="3"> PLUGIN interfacePlugin: Prelude</font></div><div> <font face="monospace" size="3">...</font></div><div><br></div><div>The output lines do not appear in the sequence they were produced!!</div><div>The plugin doesn't see/inherit the BlockBuffer mode (LineBuffering) set by the compiler!!</div><div><br></div><div>This is a strong indication, that <b>there are two different buffers for stdout</b>. One in the compiler and another one in the plugin.</div><div>At the end of the processing, the buffer in the compiler is automatically flushed, however the buffer in the plugin never gets flushed!</div><div><br></div><div>LINUX</div><div>=====</div><div>I did a similar test in Linux, however, here I set the buffer mode to 'Blockmode Nothing' and I didn't do a manual flush in the plugin. I got the following result:</div><div><br></div><div><font face="monospace" size="3"> COMPILER About to call plugin parse: Buffering mode: BlockBuffering Nothing</font></div><div><font face="monospace" size="3"> PLUGIN Buffering: BlockBuffering Nothing</font></div><div><font face="monospace" size="3"> PLUGIN parsePlugin(a,b)</font></div><div><font face="monospace" size="3"> COMPILER Returning from plugin parse</font></div><div><font face="monospace" size="3"> PLUGIN interfacePlugin: Prelude</font></div><div><font face="monospace" size="3"></font> <font face="monospace" size="3">...</font></div><div><br></div><div>Here the lines are in the same order as they were produced.</div><div>The setting of the Buffering mode is inherited by the plugin.</div><div><br></div><div>I think, on Linux the compiler and the plugin share the same buffer.</div><div><br></div><div>To fix the issue on Windows, the compiler and the plugin should use the same buffer for stdout.</div><div>However I don't know whether this is possible / difficult / easy?</div><div>What's your opinion?</div><div><br></div><div>Many thanks and kind regards</div><div> Roland</div><div><br></div><div>Here are my changes for Windows in code:</div><div><br></div><div>Change in HscMain the line "<font face="monospace" size="3">import System.IO (fixIO)</font>" to "<font face="monospace" size="3">import System.IO</font>"</div><div><br></div><div>Last lines of function HscMain.hs:hscParse'</div><div><br></div><div><font face="monospace" size="3"> -- apply parse transformation of plugins</font></div><div><font face="monospace" size="3"> let applyPluginAction p opts</font></div><div><font face="monospace" size="3"> = parsedResultAction p opts mod_summary</font></div><div><font face="monospace" size="3"> liftIO $ hSetBuffering stdout LineBuffering</font></div><div><font face="monospace" size="3"> mode <- liftIO $ hGetBuffering stdout</font></div><div><font face="monospace" size="3"> liftIO $ putStrLn ("COMPILER About to call plugin parse: " ++ show mode)</font></div><div><font face="monospace" size="3"> rsxresult <- withPlugins dflags applyPluginAction res</font></div><div><font face="monospace" size="3"> liftIO $ putStrLn "COMPILER Returning from plugin parse"</font></div><div><font face="monospace" size="3"> return rsxresult</font></div><div><font face="monospace" size="3"></font><br></div><div>New code for function SourcePlugin.hs:parsedPlugin</div><div><br></div><div><font face="monospace" size="3">parsedPlugin opts _ pm</font></div><div><font face="monospace" size="3"> = do</font></div><div><font face="monospace" size="3"> mode <- liftIO $ hGetBuffering stdout</font></div><div><font face="monospace" size="3"> liftIO $ putStrLn $ "PLUGIN Buffermode: " ++ show mode</font></div><div><font face="monospace" size="3"> liftIO $ putStrLn $ "PLUGIN parsePlugin(" ++ intercalate "," opts ++ ")"</font></div><div><font face="monospace" size="3"> return pm</font></div><div><font face="monospace" size="3"></font><br></div><div>New code for function SourcePlugin.hs:interfaceLoadPlugin'</div><div><br></div><div><font face="monospace" size="3">interfaceLoadPlugin' :: [CommandLineOption] -> ModIface -> IfM lcl ModIface</font></div><div><font face="monospace" size="3">interfaceLoadPlugin' _ iface</font></div><div><font face="monospace" size="3"> = do liftIO $ putStrLn $ "PLUGIN interfacePlugin: "</font></div><div><font face="monospace" size="3"> ++ (showSDocUnsafe $ ppr $ mi_module iface)</font></div><div><font face="monospace" size="3"> liftIO $ hFlush stdout</font></div><div><font face="monospace" size="3"> return iface</font></div><div><font face="monospace" size="3"></font></div><div><font face="monospace" size="3"></font><br></div><div><br></div><div><br></div><div>Am Dienstag, den 04.12.2018, 00:02 +0000 schrieb Phyx:</div><blockquote type="cite"><div dir="ltr"><div dir="ltr"><div>Hi Roland,</div><div><br></div><div>Thanks for looking into these.</div><div><br></div><div>>
I looked into the testcases 'plugins09', 'plugins10' and 'plugins11' and
found the following: GHC-Windows uses BufferMode 'BlockBuffering
Nothing', however, GHC-Linux uses 'LineBuffering'. <br></div><div><br></div><div>Ah, yes, this isn't technically a Linux vs Windows thing, GHC will always default to LineBuffering for terminals and BlockBuffering for anything else. The issue is that msys2 terminals have std handles that are backed by files instead of pipes. This is an artifact of how they try to emulate posix shells, See <a href="https://github.com/msys2/msys2/wiki/Porting#standard-streams-in-mintty">https://github.com/msys2/msys2/wiki/Porting#standard-streams-in-mintty</a>.
This means that when GHC runs inside an msys2 program such as bash it'll always default to BlockBuffering.
</div><div><br></div><div>>
I don't know anything about the "Why" and "Where" in the GHC IO module on Windows, so I'm unable to come up with a patch. <br></div><div><br></div><div>The above said the handles should be getting flushed when the finalizers are run, but these aren't 100% guaranteed so if the tests rely on this then your solution (to force buffer mode to LineBuffering) sounds like perfectly adequate. Could you put a patch up with that?</div><div><br></div><div>My new I/O manager takes a different approach to determine the buffer mode, but I still have some kinks to work out before posting it.</div><div><br></div><div>>
PS: I can't say anything about the tests 'plugin-recomp-pure' and
'plugin-recomp-impure' as these tests run successfully on my (slow)
Windows box. <br></div><div><br></div><div>These don't fail for me or harbormaster either, so if Simon is able to consistently reproduce these then I'll have to ask him for a core dump so I can take a look.</div><div><br></div><div>Thanks again,</div><div>I appreciate the help!</div><div><br></div><div>Regards,</div><div>Tamar<br></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Dec 3, 2018 at 3:34 PM Roland Senn <<a href="mailto:rsx@bluewin.ch">rsx@bluewin.ch</a>> wrote:<br></div><blockquote type="cite"><div><div>Hi Tamar,</div><div><br></div><div>I looked into the testcases 'plugins09', 'plugins10' and 'plugins11' and found the following: GHC-Windows uses BufferMode 'BlockBuffering Nothing', however, GHC-Linux uses 'LineBuffering'.</div><div><br></div><div>If I add an 'import System.IO' and the line 'liftIO $ hSetBuffering stdout LineBuffer' as first line in the do block of the function '.../testuite/tests/plugins/simple-plugin/Simple/SourcePlugin.hs:parsedPlugin' then all 3 tests pass successfully on Windows!</div><div><br></div><div>I don't know anything about the "Why" and "Where" in the GHC IO module on Windows, so I'm unable to come up with a patch.</div><div><br></div><div>Regards</div><div> Roland</div><div></div><div><br></div><div>PS: I can't say anything about the tests 'plugin-recomp-pure' and 'plugin-recomp-impure' as these tests run successfully on my (slow) Windows box.</div><div><br></div></div></blockquote><div><br></div></div></blockquote></body></html>