[Haskell-cafe] I/O interface

[Marcin 'Qrczak' Kowalczyk]

Ben Rudiak-Gould <Benjamin.Rudiak-Gould at cl.cam.ac.uk> writes:

> Yes, this is a problem. In my original proposal InputStream and
> OutputStream were types, but I enthusiastically embraced Simon M's
> idea of turning them into classes. As you say, it's not without its
> disadvantages.

This is my greatest single complaint about Haskell: that it doesn't
support embedding either OO-style abstract supertypes, or dynamnic
typing with the ability to use polymorphic operations on objects that
we don't know the exact type.

The Dynamic type doesn't count for the latter because you must guess
the concrete type before using the object. You can't say "it should be
something implementing class Foo, I don't care what, and I only want
to use Foo's methods with it".

Haskell provides only:
- algebraic types (must specify all "subtypes" in one place),
- classes (requires foralls which limits applicability:
  no heterogeneous lists, I guess no implicit parameters),
- classes wrapped in existentials, or records of functions
  (these two approaches don't support controlled downcasting,
  i.e. "if this is a regular file, do something, otherwise do
  something else").

The problem manifests itself more when we add more kinds of streams:
transparent compression/decompression, character recoding, newline
conversion, buffering, userspace /dev/null, concatenation of several
input streams, making a copy of data as it's passed, automatic
flushing of a related output stream when an input stream is read, etc.

A case similar to streams which would benefit from this is DB
interface. Should it use separate types for separate backends? Awkward
to write code which works with multiple backends. Should it use a
record of functions? Then we must decide at the beginning the complete
set of supported operations, and if one backend provides something
that another doesn't, it's impossible to write code which requires
the first backend and uses the capability (unless we decide at the
beginning about all possible extensions and make stubs which throw
exceptions in cases it's not supported). I would like to mix these
two approaches: if some code uses only operations supported by all
backends, then it's fully polymorphic, and when it starts using
specific operations, it becomes limited. Without two completely
different designs for these cases. I don't know how to fit it into
Haskell's type system. This has led me to exploring dynamic typing.

> Again, to try to avoid confusion, what you call a "seekable file" the
> library calls a "file", and what you call a "file" I would call a
> "Posix filehandle".

So the incompleteness problem can be rephrased: the interface doesn't
provide the functionality of open() with returns an arbitrary POSIX
filehandle.

> By the same token, stdin is never a file, but the data which appears
> through stdin may ultimately be coming from a file, and it's sometimes
> useful, in that case, to bypass stdin and access the file directly.
> The way to handle this is to have a separate stdinFile :: Maybe File.

And a third stdin, as POSIX filehandle, to be used e.g. for I/O
redirection for a process.

> As for openFile: in the context of a certain filesystem at a certain
> time, a certain pathname may refer to
>
>   * Nothing
>   * A directory
>   * A file (in the library sense); this might include things like
> /dev/hda and /dev/kmem
>   * Both ends of a (named) pipe
>   * A data source and a data sink which are related in some
> qualitative way (for example, keyboard and screen, or stdin and stdout)
>   * A data source only
>   * A data sink only
>   * ...
>
> How to provide an interface to this zoo?

In such cases I tend to just expose the OS interface, without trying
to be smart. This way I can be sure I don't make anything worse than
it already is.

Yes, it probably makes portability harder. Suitability of this
approach depends on our goals: either we want to provide a nice and
portable abstraction over the basic functionality of all systems,
or we want to make everything implementable in C also implementable
in Haskell, including a Unix shell.

Perhaps Haskell is in the first group. Maybe its goal is to invent
an ideal interface to the computer's world, even if this means doing
things differently than everyone else. It's hard to predict beforehand
how far in being different we can go without alienating users.

For my language I'm trying to do the second thing. I currently
concentrate on Unix because there are enough Windows-inspired
interfaces in .NET, while only Perl and Python seem to care about
providing a rich access to Unix API from a different language than C.

I try to separate interfaces which should be portable from interfaces
to Unix-specific things. Unfortunately I have never programmed for
Windows and I can make mistakes about which things are common to
various systems and which are not. Time will tell and will fix this.

Obviously I'm not copying the Unix interface literally. A file is
distinguished from an integer, and an integer is distinguished from a
Unix signal, even though my language is dynamically typed. But when
Unix makes some objects interchangeable, I retain this.

So I'm using a single RAW_FILE type, which wraps an arbitrary POSIX
file handle. Then there are various kinds of streams: mostly wrappers
for other streams which perform transparent conversion, buffering etc.
RAW_FILE is a stream itself. I distinguish input streams from output
streams (RAW_FILE is both) and byte streams from character streams.

Functions {Text,Binary}{Reader,Writer} put a default stack of stream
converters, with default or provided parameters like the encoding.
The last part in all of them is a buffering layer; only buffered
streams provide reading by line. Input buffers provide unlimited
lookahead and putback, output buffers provide automatic flushing
(after every operation or after full lines).

One controversial design decision is that read/write operations on
blocks *move* data between the stream and the buffer (flexible array).
That is, WriteBlock appends at the end, while ReadBlock cuts from the
beginning. It makes passing data between streams less error-prone,
at the cost of unnecessary copying of memory if we want to retain in
memory what we have just written.

> The Haskell approach is, I guess, to use an algebraic datatype, e.g.
>
>     data FilesystemObject
>       = Directory Directory
>       | File File
>       | InputOutput PosixInputStream PosixOutputStream
>       | Input PosixInputStream
>       | Output PosixOutputStream

If openFile was to try to infer what kind of object it just opened,
I'm worried that the kinds are not disjoint and that determining this
is often unnecessary. For example if I intend to only read from a file
sequentially, it doesn't matter whether it is really InputOutput (the
path named a character device), Input (Haskell somehow determined that
only reading will succeed), or a File on which an input stream can be
wrapped. A Haskell runtime tries to determine what it is, even though
it could just blindly use read()/write() and let the OS dispatch these
calls to the appropriate implementation.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/