[Haskell-cafe] Streaming bytes and performance

[Don Stewart]

This isn't a valid entry -- it uses strict IO (so allocates O(n) space) and
reads from standard input, which pretty much swamps the interesting
constant factors with buffered IO overhead.

Compare your program (made lazy) on lazy bytestrings using file IO:

import Prelude hiding ( readFile, foldl )
import Data.ByteString.Lazy.Char8

countSpace :: Int -> Char -> Int
countSpace i c | c == ' ' || c == '\n' = i + 1
               | otherwise             = i

main :: IO ()
main = readFile "test.txt" >>= print . foldl countSpace 0


Against my earlier optimized one (that manually specializes and does other
tricks).


$ time ./C
4166680
./C  1.49s user 0.42s system 82% cpu 2.326 total

$ time ./fast
4166680
./fast  1.05s user 0.11s system 96% cpu 1.201 total


The optimized one is twice as fast. You can write the same program on lists
, and it also runs in constant space but completes 32s instead  of 1.3

Constant factors matter.

On Tue, Mar 19, 2013 at 9:03 PM, Peter Simons <simons at cryp.to> wrote:

> Don Stewart <dons00 at gmail.com> writes:
>
>  > Here's the final program: [...]
>
> Here is a version of the program that is just as fast:
>
>   import Prelude hiding ( getContents, foldl )
>   import Data.ByteString.Char8
>
>   countSpace :: Int -> Char -> Int
>   countSpace i c | c == ' ' || c == '\n' = i + 1
>                  | otherwise             = i
>
>   main :: IO ()
>   main = getContents >>= print . foldl countSpace 0
>
> Generally speaking, I/O performance is not about fancy low-level system
> features, it's about having a proper evaluation order:
>
>  | $ ghc --make -O2 -funbox-strict-fields test1 && time ./test1
>  | 37627064
>  |
>  | real 0m0.381s
>  | user 0m0.356s
>  | sys  0m0.023s
>
> Versus:
>
>  | $ ghc --make -O2 -funbox-strict-fields test2 && time ./test2 <test.txt
>  | Linking test2 ...
>  | 37627064
>  |
>  | real 0m0.383s
>  | user 0m0.316s
>  | sys  0m0.065s
>
> Using this input file stored in /dev/shm:
>
>  | $ ls -l test.txt
>  | -rw-r--r-- 1 simons users 208745650 Mar 19 21:40 test.txt
>
> Take care,
> Peter
>
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe at haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/haskell-cafe/attachments/20130319/a103980f/attachment.htm>