[Haskell-cafe] How to optimize a directory scanning?

[Neil Mayhew]

It would be possible to avoid that TCGETS ioctl since the immediately 
preceding fstat shows that the file is a regular file and not a device. 
However, I'm not sure how easy it would be for the library to make that 
optimization.

The Haskell implementation actually makes less syscalls than the Rust 
one, because Rust reads the file in very small chunks (32,32,64,128,64) 
whereas Haskell reads one big chunk (8192) which is sufficient to 
contain the entire file. I think it's unlikely that the extra ioctl 
outweighs the multiple extra reads. However, if you use the -r option 
with strace to include timestamps in the output, you'll be able to see 
just how long each syscall is taking. On my system, they all take about 
the same amount of time.

It would also be worth using time on the program, to see how much of the 
CPU time is in user space vs kernel.

On 2019-05-10 9:35 AM, Brandon Allbery wrote:
> The ioctl is standard, including in C unless you are using open() 
> directly: it checks to see if the opened file is a terminal, to 
> determine whether to set block or line buffering.
>
> On Fri, May 10, 2019 at 11:09 AM Magicloud Magiclouds 
> <magicloud.magiclouds at gmail.com 
> <mailto:magicloud.magiclouds at gmail.com>> wrote:
>
>     So this is what I got. Seems like both calls two stat(stat/newfstatat)
>     for dir checking and uid checking. But when open file for reading,
>     there is an ioctl call (maybe from System.IO.Strict) which seems
>     failed, for Haskell. I want to test the case without System.IO.Strict.
>     But have no idea how to get exception catching works with lazy
>     readFIle.
>
>     For Haskell implenmentation,
>     ```
>     stat("/proc/230", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
>     stat("/proc/230", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
>     openat(AT_FDCWD, "/proc/230/stat", O_RDONLY|O_NOCTTY|O_NONBLOCK) = 23
>     fstat(23, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
>     ioctl(23, TCGETS, 0x7ffe88c18090)       = -1 ENOTTY (Inappropriate
>     ioctl for device)
>     read(23, "230 (scsi_eh_5) S 2 0 0 0 -1 212"..., 8192) = 155
>     read(23, "", 8192)                      = 0
>     close(23)
>     ```
>     For Rust implenmentation,
>     ```
>     newfstatat(3, "1121", {st_mode=S_IFDIR|0555, st_size=0, ...},
>     AT_SYMLINK_NOFOLLOW) = 0
>     stat("/proc/1121", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
>     open("/proc/1121/stat", O_RDONLY|O_CLOEXEC) = 4
>     fcntl(4, F_SETFD, FD_CLOEXEC)           = 0
>     fstat(4, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
>     read(4, "1121 (ibus-engine-sim) S 1077 10", 32) = 32
>     read(4, "77 1077 0 -1 4194304 425 0 1 0 5", 32) = 32
>     read(4, "866 1689 0 0 20 0 3 0 3490 16454"..., 64) = 64
>     read(4, "4885264596992 94885264603013 140"..., 128) = 128
>     read(4, "0724521155542 140724521155575 14"..., 256) = 64
>     read(4, "", 192)                        = 0
>     close(4)
>     ```
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/haskell-cafe/attachments/20190510/c073062e/attachment.html>