<div dir="ltr"><div dir="ltr">Another approach would be to use the Data.Text.IO.hGetContents function on a file handle that explicitly sets its character encoding to UTF-8. This is what we do in rio:<div><br></div><div><a href="https://www.stackage.org/haddock/lts-13.10/rio-0.1.8.0/src/RIO.Prelude.IO.html#readFileUtf8">https://www.stackage.org/haddock/lts-13.10/rio-0.1.8.0/src/RIO.Prelude.IO.html#readFileUtf8</a><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Mar 4, 2019 at 3:36 PM David Fox <<a href="mailto:dsf@seereason.com">dsf@seereason.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-size:small">This fixes it by forcing the evaluation of the decode where it can be caught:</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default">return $ Right $! TE.decodeUtf8 bufferStrict</div><div class="gmail_default"><br></div><div class="gmail_default">or </div><div class="gmail_default"><br></div><div class="gmail_default">Right <$> evaluate (TE.decodeUtf8 bufferStrict)<br></div><div class="gmail_default"><br></div><div class="gmail_default"><br></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Mar 4, 2019 at 3:50 AM Kees Bleijenberg <<a href="mailto:K.Bleijenberg@lijbrandt.nl" target="_blank">K.Bleijenberg@lijbrandt.nl</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div lang="NL"><div class="gmail-m_152412307694788344gmail-m_4240515626836095125WordSection1"><p class="MsoNormal"><span lang="EN-US">Hi all,<u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-US">The program reads lots of small Text files. readCDFile handles the encoding. Below is the simplest version of readCDFile.<u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-US">If I call readCDFile "/home/kees/freeDB/inputError/" "blah" (the file blah does not exist) I get:<u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-US">Left "MyError: /home/kees/freeDB/inputError/blah: openBinaryFile: does not exist (No such file or directory)". The exception is caught by exceptionHandler<u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-US">If I call readCDFile "/home/kees/freeDB/inputError/" "67129209" I get freeDB: Cannot decode byte '\xa0': Data.Text.Internal.Encoding.decodeUtf8: Invalid UTF-8 stream. The exception is not caught by exceptionHandler (No “MyError: ” in front). The file 67129209 is indeed bad encoded.<u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-US">I’am using SomeException. Still, this ‘bad encoding exception’ is not caught. Why?<u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-US">Kees<u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">import</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">qualified</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> Data.Text </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">as</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> T<u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">import</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> System.FilePath.Posix<u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">import</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">qualified</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> Data.Text.Encoding </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">as</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> TE<u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">import</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">qualified</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> Data.ByteString.Lazy </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">as</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> B<u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">import</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> Prelude </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">hiding</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> (</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(220,220,170)">catch</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)">)<u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">import</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> Control.Exception<u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"><u></u> <u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(220,220,170)">main</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">::</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">IO</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> ()<u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)">main = </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(197,134,192)">do</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"><u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> res <- readCDFile </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(206,145,120)">"/home/kees/freeDB/inputError/"</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(206,145,120)">"67129209"</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> <u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> print res<u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"><u></u> <u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(220,220,170)">readCDFile</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">::</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">FilePath</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">-></span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">FilePath</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">-></span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">IO</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> (</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">Either</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">String</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">T</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)">.</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">Text</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)">)<u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)">readCDFile baseDir fn = </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(197,134,192)">do</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"><u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> catch ( </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(197,134,192)">do</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> <u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> buffer <- B.readFile (combine baseDir fn)<u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">let</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> bufferStrict = B.toStrict buffer<u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> return $ Right $ TE.decodeUtf8 bufferStrict<u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> ) exceptionHandler<u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"><u></u> <u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(220,220,170)">exceptionHandler</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">::</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">SomeException</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">-></span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">IO</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> (</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">Either</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">String</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">T</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)">.</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">Text</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)">)<u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)">exceptionHandler e = </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(197,134,192)">do</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(86,156,214)">let</span><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> err = show e<u></u><u></u></span></p><p class="MsoNormal" style="line-height:14.25pt;background:rgb(30,30,30)"><span lang="EN-US" style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> </span><span style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)">return $ Left $ </span><span style="font-size:10.5pt;font-family:Consolas;color:rgb(206,145,120)">"MyError: "</span><span style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"> ++ err<u></u><u></u></span></p><p class="MsoNormal" style="margin-bottom:12pt;line-height:14.25pt;background:rgb(30,30,30)"><span style="font-size:10.5pt;font-family:Consolas;color:rgb(212,212,212)"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p></div><div id="gmail-m_152412307694788344gmail-m_4240515626836095125DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2"><br> <table style="border-top:1px solid rgb(211,212,222)">
<tbody><tr>
<td style="width:55px;padding-top:18px"><a href="https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient" target="_blank"><img src="https://ipmcdn.avast.com/images/icons/icon-envelope-tick-round-orange-animated-no-repeat-v1.gif" alt="" width="46" height="29" style="width: 46px; height: 29px;"></a></td>
<td style="width:470px;padding-top:17px;color:rgb(65,66,78);font-size:13px;font-family:Arial,Helvetica,sans-serif;line-height:18px">Virusvrij. <a href="https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient" style="color:rgb(68,83,234)" target="_blank">www.avast.com</a> </td>
</tr>
</tbody></table>
<a href="#m_152412307694788344_m_4240515626836095125_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2" width="1" height="1"> </a></div></div>_______________________________________________<br>
Haskell-Cafe mailing list<br>
To (un)subscribe, modify options or view archives go to:<br>
<a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe" rel="noreferrer" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe</a><br>
Only members subscribed via the mailman list are allowed to post.</blockquote></div>
_______________________________________________<br>
Haskell-Cafe mailing list<br>
To (un)subscribe, modify options or view archives go to:<br>
<a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe" rel="noreferrer" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe</a><br>
Only members subscribed via the mailman list are allowed to post.</blockquote></div>