Download Reference Manual
The Developer's Library for D
About Wiki Forums Source Search Contact

Asynchronous IO

Discussions and specifications for such functionality in Tango.

Early plans (no progress)

Further on, we're building an asynchronous I/O library based on Tango's IO abstractions with notifications sent on completion of I/O events. The plan for the first stage of development is to have an API capable of delivering I/O, timer and (possibly) Unix signal events to applications through delegates. It will be able to efficiently handle large numbers ( i.e. thousands) of active file descriptors/handles (sockets, pipes, etc.) on all the platforms that Tango supports. Initially the library will work both on Windows (using I/O completion ports) and on Linux (using epoll); we will then provide a Mac OS X and FreeBSD implementation (based on kqueue), and other platforms if there is enough interest from the community.

During the second stage of development we will build a framework on top of the asynchronous I/O library that will be able to multiplex I/O jobs using Tango Fibers (i.e. lightweight or userspace threads). Each fiber waiting for I/O events will be suspended until the event is received, helping to avoid consuming excessive resources. The load from each fiber will be distributed among a pool of threads.

The idea behind both libraries is to be able to efficiently implement network protocols that are either synchronous (HTTP, SMTP, etc.) and asynchronous (XMPP, etc.) in nature in both client and server applications.

User discussions / thoughts

Async IO key/value pair database

I have need of a generic DB similar to berkdb. My first though has an interface like this:

single threaded:

auto db = new DB("file.db");

auto dbwrite = db.put("myitem", "this is a long value item");
assert(db.exists("myitem")); // undefined
// do something here
dbwrite.wait();
assert(db.exists("myitem")); // true

auto dbread = db.read("myitem");
// do something here
string value = cast(string)dbread.wait();

multithreaded:

to be added

The concept of the database will be to be as fast as possible while using as little memory and CPU to do the job. Items are stored as key value pairs. Because the core of the library is centered around asyncronous IO, I have a few opportunities for doing such a thing. I need comments. My current plan, will be to implement a wait function for single thread (see above) and for multithread, the second param could be a callback, which can do anything you want -- including waking the thread up

Comments