Synchronous I/O. Synchronous I/O. Synchronous I/O

This module defines the Rust interface for synchronous I/O. It models byte-oriented input and output with the Reader and Writer traits. Types that implement both Reader and Writer and called 'streams', and automatically implement trait Stream. Implementations are provided for common I/O streams like file, TCP, UDP, Unix domain sockets. Readers and Writers may be composed to add capabilities like string parsing, encoding, and compression.

This will likely live in std::io, not std::rt::io.

Examples

Some examples of obvious things you might want to do

• Read lines from stdin

  for stdin().each_line |line| { println(line) }

• Read a complete file to a string, (converting newlines?)

  let contents = File::open("message.txt").read_to_str(); // read_to_str??

• Write a line to a file

  let file = File::open("message.txt", Create, Write); file.write_line("hello, file!");

• Iterate over the lines of a file

  do File::open("message.txt").each_line |line| { println(line) }

• Pull the lines of a file into a vector of strings

  let lines = File::open("message.txt").line_iter().to_vec();

• Make an simple HTTP request

  let socket = TcpStream::open("localhost:8080"); socket.write_line("GET / HTTP/1.0"); socket.write_line(""); let response = socket.read_to_end();

• Connect based on URL? Requires thinking about where the URL type lives and how to make protocol handlers extensible, e.g. the "tcp" protocol yields a TcpStream.

  connect("tcp://localhost:8080");

Terms

• Reader - An I/O source, reads bytes into a buffer
• Writer - An I/O sink, writes bytes from a buffer
• Stream - Typical I/O sources like files and sockets are both Readers and Writers, and are collectively referred to a streams.
• Decorator - A Reader or Writer that composes with others to add additional capabilities such as encoding or decoding

Blocking and synchrony

When discussing I/O you often hear the terms 'synchronous' and 'asynchronous', along with 'blocking' and 'non-blocking' compared and contrasted. A synchronous I/O interface performs each I/O operation to completion before proceeding to the next. Synchronous interfaces are usually used in imperative style as a sequence of commands. An asynchronous interface allows multiple I/O requests to be issued simultaneously, without waiting for each to complete before proceeding to the next.

Asynchronous interfaces are used to achieve 'non-blocking' I/O. In traditional single-threaded systems, performing a synchronous I/O operation means that the program stops all activity (it 'blocks') until the I/O is complete. Blocking is bad for performance when there are other computations that could be done.

Asynchronous interfaces are most often associated with the callback (continuation-passing) style popularised by node.js. Such systems rely on all computations being run inside an event loop which maintains a list of all pending I/O events; when one completes the registered callback is run and the code that made the I/O request continues. Such interfaces achieve non-blocking at the expense of being more difficult to reason about.

Rust's I/O interface is synchronous - easy to read - and non-blocking by default.

Remember that Rust tasks are 'green threads', lightweight threads that are multiplexed onto a single operating system thread. If that system thread blocks then no other task may proceed. Rust tasks are relatively cheap to create, so as long as other tasks are free to execute then non-blocking code may be written by simply creating a new task.

When discussing blocking in regards to Rust's I/O model, we are concerned with whether performing I/O blocks other Rust tasks from proceeding. In other words, when a task calls read, it must then wait (or 'sleep', or 'block') until the call to read is complete. During this time, other tasks may or may not be executed, depending on how read is implemented.

Rust's default I/O implementation is non-blocking; by cooperating directly with the task scheduler it arranges to never block progress of other tasks. Under the hood, Rust uses asynchronous I/O via a per-scheduler (and hence per-thread) event loop. Synchronous I/O requests are implemented by descheduling the running task and performing an asynchronous request; the task is only resumed once the asynchronous request completes.

For blocking (but possibly more efficient) implementations, look in the io::native module.

Error Handling

I/O is an area where nearly every operation can result in unexpected errors. It should allow errors to be handled efficiently. It needs to be convenient to use I/O when you don't care about dealing with specific errors.

Rust's I/O employs a combination of techniques to reduce boilerplate while still providing feedback about errors. The basic strategy:

• Errors are fatal by default, resulting in task failure
• Errors raise the io_error condition which provides an opportunity to inspect an IoError object containing details.
• Return values must have a sensible null or zero value which is returned if a condition is handled successfully. This may be an Option, an empty vector, or other designated error value.
• Common traits are implemented for Option, e.g. impl<R: Reader> Reader for Option<R>, so that nullable values do not have to be 'unwrapped' before use.

These features combine in the API to allow for expressions like File::new("diary.txt").write_line("met a girl") without having to worry about whether "diary.txt" exists or whether the write succeeds. As written, if either new or write_line encounters an error the task will fail.

If you wanted to handle the error though you might write

let mut error = None;
do io_error::cond(|e: IoError| {
    error = Some(e);
}).in {
    File::new("diary.txt").write_line("met a girl");
}

if error.is_some() {
    println("failed to write my diary");
}

XXX: Need better condition handling syntax

In this case the condition handler will have the opportunity to inspect the IoError raised by either the call to new or the call to write_line, but then execution will continue.

So what actually happens if new encounters an error? To understand that it's important to know that what new returns is not a File but an Option<File>. If the file does not open, and the condition is handled, then new will simply return None. Because there is an implementation of Writer (the trait required ultimately required for types to implement write_line) there is no need to inspect or unwrap the Option<File> and we simply call write_line on it. If new returned a None then the followup call to write_line will also raise an error.

Concerns about this strategy

This structure will encourage a programming style that is prone to errors similar to null pointer dereferences. In particular code written to ignore errors and expect conditions to be unhandled will start passing around null or zero objects when wrapped in a condition handler.

• XXX: How should we use condition handlers that return values?
• XXX: Should EOF raise default conditions when EOF is not an error?

Issues with i/o scheduler affinity, work stealing, task pinning

Resource management

• close vs. RAII

Paths, URLs and overloaded constructors

Scope

In scope for core

• Url?

Some I/O things don't belong in core

• url
• net - fn connect
  • http
• flate

Out of scope

• Async I/O. We'll probably want it eventually

XXX Questions and issues

• Should default constructors take Path or &str? Path makes simple cases verbose. Overloading would be nice.
• Add overloading for Path and &str and Url &str
• stdin/err/out
• print, println, etc.
• fsync
• relationship with filesystem querying, Directory, File types etc.
• Rename Reader/Writer to ByteReader/Writer, make Reader/Writer generic?
• Can Port and Chan be implementations of a generic Reader/Writer?
• Trait for things that are both readers and writers, Stream?
• How to handle newline conversion
• String conversion
• File vs. FileStream? File is shorter but could also be used for getting file info
  • maybe File is for general file querying and also has a static open method
• open vs. connect for generic stream opening
• Do we need close at all? dtors might be good enough
• How does I/O relate to the Iterator trait?
• std::base64 filters
• Using conditions is a big unknown since we don't have much experience with them
• Too many uses of OtherIoError