Skip to main content

Async

Partly implemented

This page is a design sketch, and only partly implemented. std/async has the effect, tasks and an executor that runs them in one thread: Tasks and the executor says what it does. Waiting for the outside world, like fetch, and cancelling aren't implemented. The syntax of effect types is that of the effects design.

Code that waits, for a network request or a timer, is usually marked with async, and its results are taken out of a future with await. With algebraic effects, neither keyword is needed: code that can wait has the Async effect in its type, and the compiler infers it like any other effect. The reactive UI design builds on this.

Code that waits​

A function that waits says so in its type, and returns its result directly:

fn fetch(url: str) -> Result[Response, HttpError] does Async;

fn is_up(url: str) -> bool does Async = match fetch(url) {
Ok(_) => true,
Err(_) => false,
};

There's no await: fetch returns the response once it's there. Callers don't write async either. A signature like is_up's says does Async, as it says its result type, and closures and functions without a written result type get Async by inference.

The effect itself has three operations, none of them generic:

effect Async {
// Let the other tasks that are ready run first.
fn pause();
// Run `task` next to the code that calls fork.
fn fork(task: mut fn() does Async);
// Stop until the waker that `start` is given is called.
fn suspend(start: mut fn(mut fn()));
}

What's typed, the result of a task, is in Task[T], which ordinary functions work with:

// Run `f` next to the code that calls spawn.
fn spawn[T](f: mut fn() -> T does Async) -> mut Task[T] does Async;
// Wait until a task is done, and get its result.
fn wait[T](task: mut Task[T]) -> T does Async;

suspend connects Bolt to the outside world: fetch calls the fetch of the browser, and gives the promise it returns a callback that keeps the response and calls the waker. Most code never calls suspend or fork itself.

Why the operations aren't generic​

An earlier version of this design had spawn[T] and suspend[T] as operations. Then the T of spawn and that of wait are two type parameters, each of its own operation, and a handler can't hand the result of a task from one to the other: nothing says they're the same. They're also any in the compiled operations, so each value of them would be converted on its way through (see Conduit).

Tokio splits it the same way: its scheduler runs untyped tasks, and the JoinHandle<T> that spawn gives back points at the task's typed result. In Bolt, Task[T] holds the result and the wakers of the tasks waiting for it, spawn forks a closure that sets it, and wait suspends until it's set. They're compiled for each T, like any generic function.

No function colours​

In JavaScript and Rust, async spreads up the call chain, and a function like map needs a second version for async callbacks. With effect polymorphism, one map is enough: the effects of its callback become its own.

fn map[A, B, E](xs: [A], f: fn(A) -> B does E) -> [B] does E;

map([1, 2, 3], |x| x * 2); // no effects
map(urls, is_up); // does Async

An effect variable like E stands for a whole row of effects; the effects design has the details.

Running things at the same time​

Without futures, code runs one step after the other:

let a = fetch(x); // waits for x
let b = fetch(y); // and only then asks for y

To ask for both at once, spawn one of them, or use a function built on it:

fn both[A, B](a: fn() -> A does Async, b: fn() -> B does Async) -> (A, B) does Async {
let task = spawn(a);
let result = b();
(wait(task), result)
}

let (a, b) = both(|| fetch(x), || fetch(y));

So the common case reads like code that doesn't wait, and running things at the same time is explicit. Task[T] is the only kind of future, for the places where it's needed.

Tasks and the executor​

block_on is the executor: the handler of Async that runs a program's tasks, one at a time, in one thread.

import "std/async" *;

block_on(|| {
let a = spawn(|| { pause(); 40 });
let b = spawn(|| 2);
print(show(wait(a) + wait(b))); // 42
});

Like tokio's, it's a loop over a queue of what's ready to run. Its handler's arms don't run the next task themselves: they put the rest of the current one on the queue and return to the loop. So switching tasks doesn't make the stack deeper, however often it happens. A handler whose arms resume the next task directly gets deeper with each switch, since nothing returns until the end.

  • pause puts the task at the end of the queue.
  • fork puts the new task, then the rest of the current one, on the queue.
  • suspend gives start a waker, and puts nothing on the queue. The first call of the waker puts the task back on it; more calls do nothing, since a task can only go on once.

Tasks don't outlive their scope​

block_on returns once every task is done, so no task outlives it. Tokio lets a task go on when its JoinHandle is dropped, which makes it easy to lose track of tasks; its JoinSet, which cancels its tasks when it's dropped, came later to fix that. Bolt can't do the former anyway: values are shared, so nothing knows when a Task is dropped, as for use. A handler is already a scope, so tasks are scoped to the handler that runs them. A scope for a part of a program, which waits for its tasks, or cancels them, needs cancelling first.

Fair turns​

A task only gives the others a turn when it waits. Tokio also makes a task that's always ready give a turn after a while, with a budget of operations. block_on doesn't need one so far: suspend always puts the task at the end of the queue, even when the waker is called right away. A handler that lets a task go on at once when the waker is called during start would need a budget.

One effect, many handlers​

What waiting means depends on the handler that runs the code:

WhereWhat the Async handler does
An event handlerruns it as a task of its own, next to the page
A componentshows a fallback, and draws again once the data is there
Rendering on a serverwaits for every request, then sends the page
A testruns tasks one by one in a fixed order, with a fake clock

The code is the same in all of them. Tests in particular get easier: a handler with a fake clock runs timers and timeouts right away, and always in the same order.

Cancelling​

A task that isn't needed anymore should stop: a component that left the page, or a search whose text changed. The handler of Async cancels it.

The problem​

A handler could cancel a task by never resuming it. With effects that's allowed, since a continuation can be resumed once or not at all. But then the rest of the task never runs, cleanup included:

fn download(url: String) -> Bytes does Async {
let conn = open(url);
let data = conn.read_all(); // cancelled while it waits here
conn.close(); // never runs: the connection stays open
data
}

Unwinding a task​

So the handler doesn't drop a task. It resumes it once more, with "you're cancelled" instead of a value. The suspend the task waits in then stops, like an error would, and the task unwinds from there up to the handler, running its cleanup on the way. Nothing in the task runs after that.

A task is only cancelled while it waits, at a suspend. Code between two waits always runs to the end, so this can't be cut off halfway:

balance -= amount;
history.push(amount);

That's much easier to reason about than threads, which can be stopped anywhere.

defer​

defer runs code when the block it's in ends, however it ends: normally, by return or break, by an error, or by a cancel.

fn download(url: String) -> Bytes does Async {
let conn = open(url);
defer conn.close();
conn.read_all()
}

The defers of a block run in the opposite order of their statements, so what was opened last is closed first. defer keeps the cleanup next to what it cleans up, which a finally doesn't, and Bolt has no try blocks for one anyway. Go, Swift and Zig have defer too. It's tracked in #348.

use​

A type that knows how to clean up can say so once, with a class:

class Close[T] {
fn close(self: mut T);
}

Then use is a let with the defer built in:

fn download(url: String) -> Bytes does Async {
use conn = open(url); // let conn = open(url); defer conn.close();
conn.read_all()
}

This is C#'s using var (with IDisposable) and Python's with. It's like Rust's Drop in that the type writes the cleanup, but it runs where use says, at the end of the block. Rust can drop values on its own because every value has one owner. Bolt values can be shared, let b = a gives the same connection a second name, so the end of a's block can't know that nobody uses it anymore. With use, the programmer says so. The language server can warn about a value of a Close type that's bound with let, as C#'s tools do for IDisposable. It's tracked in #349.

Still open​

  • Catching a cancel (#351). Code that could catch "you're cancelled" could refuse to stop. Most designs let code see a cancel but not stop it.
  • Waiting in cleanup (#352). Closing a connection politely may need a network call, so a defer may want to wait, and then it can hang. C# has a separate IAsyncDisposable for this. Bolt could allow waiting in a defer, but with a time limit.
  • Blocks that can't be cancelled (#353), for steps that must finish together even though they wait, like a database commit.
  • Not losing what a task got. Tokio's select! cancels the futures that lose, and one that had taken a value, like a message from a channel, loses it. A task should only be cancelled while it waits, and not between a waker giving it a value and the task taking it.
  • Conduit (#350). Unwinding needs a way to resume a continuation with an unwind instead of values, which OCaml 5 calls discontinue. Conduit only has resume so far. Wasm stack switching has resume_throw, which does this.

Compiling it​

Async is the one effect that truly suspends, so code that performs it is compiled differently: as a state machine or with a CPS transform, or with the stack switching of wasm. async tells a compiler which functions those are; effect types tell it the same, so nothing is lost.

Effect-polymorphic functions like map are harder: they may or may not suspend, depending on their callback. They can be compiled twice, once for each case, or always use stack switching. In the browser, JSPI lets wasm wait for a promise of JavaScript, which is enough for suspend around fetch.

The keywords​

This design doesn't need async, await and yield, so they're no longer keywords, and can be names (#263). The parser could still recognize async fn and await x, and say what to write instead: does Async after the parameters or the result type, and the call without await.

Seeing where code waits​

A call that can wait looks like any other call. That's the point, but sometimes you want to know. The language server can mark these calls, with a highlight or an inlay hint, like it can mark inferred reactive state.

Still to design​

  • What's still open about cancelling.
  • Errors. A task that fails, with Fail[E], could fail the code that waits for it, as tokio's JoinHandle gives a JoinError: wait with does Fail[TaskError[E]], or a try_wait that gives a Result.
  • Timeouts and racing, like first(a, b), which need cancelling.
  • Tasks with other effects. fork takes a task that only does Async; taking one that does Async + E needs effects whose parameters are rows (see Conduit).
  • Threads. Tokio needs Send + 'static for a task that can move to another thread. In Bolt, an imm fn() captures only what's imm all the way down, so it could run on another thread as it is.