[web-devel] Content-Length on sendFile

[Michael Snoyman]

On Wed, Jun 15, 2011 at 5:57 AM, Greg Weber <greg at gregweber.info> wrote:
>
>
> On Tue, Jun 14, 2011 at 7:29 PM, Michael Snoyman <michael at snoyman.com>
> wrote:
>>
>> Let me point out one other distinction: sendFile versus yesod-static.
>> The former is a function you would call from a normal handler, while
>> yesod-static is the "magical" package which would actually know all of
>> the stuff about your files at compile time. For the sendFile case, the
>> only options for getting the file size are (1) the programmer manually
>> adding the header and (2) Yesod automatically doing a system call to
>> get it.
>>
>> As for the behavior of Warp...  while I agree Kazu that we should
>> reduce system call overhead, it might make sense for Warp to perform
>> the system call to get file size *if* no content-length header is
>> present.
>>
>> And I'm still very uncomfortable setting the content-length header for
>> static files based on compile-time information. In this case, having
>> the wrong value (e.g., someone modified a CSS file after compiling)
>> will completely break things and corrupt an open HTTP connection.
>
> Then it should probably not be the default. Lets add a big scary warning to
> such a setting and tell users the files should be set to read-only
> permission.
> The most efficient technique for file serving static files that could be
> changed would actually be to setup a file notifier (that uses an efficient
> OS listener, like inotify on linux) that listens to static assets and knows
> when they are changed or a new one is added and would stat them just when
> changed. But there would be a race condition if you had to wait for the
> notification, so you would actually have to have the old file revision on
> hand- perhaps having a symlink convention for adding new files, but meaning
> the system could still get screwed up by someone careless.

I'm really starting to think that the correct solution for this use
case really is embedded files[1]. It avoids any kind of system call
overhead, doesn't touch the hard drive at all, and doesn't allow for
modification of the files after compile time. It would be interesting
to see if it ends up faster than sendfile.

Michael

[1] Using Template Haskell to store the entire static file inside the
executable during compilation.

>>
>> Michael
>>
>> On Wed, Jun 15, 2011 at 5:22 AM, Kazu Yamamoto <kazu at iij.ad.jp> wrote:
>> > Greg,
>> >
>> >> I apologize for the confusing terminology. I am not differentiating
>> >> between
>> >> sending a static file with sendfile and a streaming response. I
>> >> am differentiating between 2 different use cases for sending static
>> >> files
>> >> (with sendfile). For all of my web applications, I know what all the
>> >> static
>> >> files are and they will never change until I deploy another web
>> >> application.
>> >> That means I can stat the files once when the application is deployed
>> >> and keep
>> >> that information in memory. So I already have the file length
>> >> information to
>> >> include in the header, even though I don't do a file stat when the file
>> >> is
>> >> requested. wai-app-static and yesod-static supports these techniques.
>> >
>> > Thanks. I think I understand. :)
>> >
>> > So, do you support to *not* change the API (apps should add CL: by
>> > themselves)?
>> >
>> > --Kazu
>> >
>> > _______________________________________________
>> > web-devel mailing list
>> > web-devel at haskell.org
>> > http://www.haskell.org/mailman/listinfo/web-devel
>> >
>>
>> _______________________________________________
>> web-devel mailing list
>> web-devel at haskell.org
>> http://www.haskell.org/mailman/listinfo/web-devel
>
>