Afternoon Rusting

Turning functions into first-class citizens in our programming languages is one of the major changes of the decade. Well, kind of. The concept, also known as lambda, is far from new. Functional programming languages had it from the very start, during the late ’50s. Even some of the object-oriented languages like Python had it quite early, back in 1994. However it became an official part of C++ only in 2011, and Java brought it even later in 2014. And with those two languages and many others, it became the norm, even for non-functional programming. As first-class citizens, functions can be saved as variables or transfer as arguments to functions easily.

Other Post in the Series:

• Series Introduction.
• Project Management.
• Enum & Pattern Matching - Part 1.
• Enum & Pattern Matching - Part 2.

Syntax

I’m going to cover only the tip of the iceberg. For the full picture read this section of the Rust Book. The basic syntax of the closure is:

    let bigger_fn = |a : i32, b: i32| -> bool {
        a > b
    };

The code above demonstrates how to define a closure and save it to a variable. The arguments to the closures are placed between the pipes (||), while the return value is coming after the arrow (->). Similar to traditional functions in Rust, the value of the last expression is returned. To use the function write: bigger_fn(1,2). Closures can use any variable from the surrounding scopes, more on this later.

This closure can be written more concisely as follows |a, b| a < b. Rust is capable of deducing argument and return types of closures. Therefore typing those is entirely optional. In case a closure contains only one statement, the enclosing braces {} are also optional.

Now let us see how one receives a closure as an argument:

fn winner<T, F> (a : T, b : T, bigger_fn : F) -> bool
    where F : FnOnce(T, T) -> bool {
    bigger_fn(a,b)
}

Closures do not have a concrete type, so it is a necessity to use a generic parameter in order to obtain them as an argument. By itself, it isn’t enough though. While we haven’t covered the Rust trait system just yet, we can think about them as constraints on the generic type. FnOnce(T, T) -> bool is one of the possible traits for receiving a function or a closure. Each of those traits has two parts. The first is one of the following: Fn, FnMut, FnOnce, we will cover their meaning soon. The second part is the signature of the functions. A simple case would be (i32, i32) -> bool, which constrain the function to accept two integers and return a bool. Our case is a little more complicated: (T, T) -> bool constrains the closure to receive two arguments of the same generic type T, and return a bool. By using generic type for the arguments, we allow a little more freedom to the user.

Let’s understand the second part: Fn, FnMut, FnOnce. It is directly related to Rust ownership, so be sure you are well familiar with it first, read about it here.

• Fn - Closures which either don’t use variables from the surrounding scope or use them but don’t mutate or take ownership.
• FnMut - Similar to Fn but remove the requirement not to mutate variables from the enclosing scope.
• FnOnce - Similar to FnMute but remove the last constraint not to take ownership of variables from the surrounding scope.

In our example, bigger_fn met all the requirements mentioned above. Therefore it implements all the three traits. In general, any closure that implements Fn also implements FnMut, and in the same way any closure which implements FnMut also implements FnOnce. We prefer receiving FnOnce, as it allows handling any function, comparable to the way you want to use &self over self for instance. The easiest way to think about it is to observe the resemblance to regular ownership: FnOnce <=> &type, FnMut <=> &mut type, Fn <=> type. The major implication is as follows: FnOnce, as its names suggest, can only be called once. FnMut can be referenced only once at a time, similar to a mutable reference.

The last syntax detail is the move closures, this closure force the closure to take ownership of any captured variable, even if it isn’t required otherwise. It is particularly helpful and widespread when handling concurrency and multi-threading. See the following example, which also demonstrates the ease of capturing a variable:

let string = String::from("test");

if winner(String::from("test"), String::from("test2"), move |a,b| {
    println!("{}", string); // Doesn't require ownership on the string
    true
}) {
    //println!("{}", string);
    // The line above will only compile if we removes the move keyword
}

Closure - Future Proof API

Closures allow us to create APIs which tend to survive the test of time. We will start with an example illustrating the idea, and will complete the discussion afterward:

fn winner<T, F> (a : T, b : T, bigger_fn : F) -> bool
    where F : FnOnce(T, T) -> bool {
    bigger_fn(a,b)
}

#[test]
fn closures_test() {
    let bigger_fn = |a : i32, b: i32| -> bool {
        a > b
    };

    if winner(1, 7,  bigger_fn) {
        println!("One is the winner");
    } else {
        println!("Seven is the winner");
    }

    // Note the short syntax:
    if winner(&1, &7, |a, b| a < b) {
        println!("Now one is the winner");
    } else {
        println!("Now seven is the winner");
    }

    let helper : i32 = 7;
    if winner(1, 7, |a, b| a + helper > b) {
        println!("Helper made one a winner");
    } else {
        println!("Helper wasn't enough, seven is still the winner");
    }
}

This code segment will print the following:

Seven is the winner
Now one is the winner
Helper made one a winner

In the last code sample, we saw three methods of transferring arguments to the winner function. We have begun with the somewhat naive one. The winner is the one with the bigger number. Of course, if it was our only use case, using a closure is an overkill. Instead, we should have performed the direct compression inside the function.

The second example manifests the first case where accepting closures can help. By accepting a closure, we can customize the logic of deciding the winner. We chose the winner as the one with the lower value, not the higher. I’ve also demonstrated the power of combining closures with generic. The arguments are not regular integers, but a reference to an integer. While entirely unnecessary, this technique is valuable when operating with complex types. In Rust, Being able to be agnostic to the ownership of the type is tremendous. It will save you a lot of design and refactoring work.

Now for the last example, it demonstrates the flexibility of the API. When we designed the winner function, we could only imagine these two contenders. However, we failed to recognize in some use-cases, others might be involved. For instance, as we saw, one of the contenders got some help. Without taking a closure, we would have required to add an argument to the winner function. Not only it requires a code refactoring. The final result would be weird for any use-case not obliging this helper.

Closure - Delayed & Conditional Execution

Another major use case for closure is delayed and conditional execution. We are used to writing our programs in a linear fashion. This behavior has strong roots with the days we had only one CPU running our code, and threading support was lacking. It is no longer the circumstance for a very long time. As a result, people are trying to experiment with different approaches for writing code. In many cases, it is convenient to define the code in one place while running it in a completely different location. This technique is widely used in async programming, a method of writing code which saw a gradual rise in popularity since the early 2000s. However, in the last few years, it literally exploded. It is turning to the defacto way of writing quality software.

In the last post, about enums, I’ve introduced a way to create an if statement as an enum. If you have read it, you can skip the next code sample, and the paragraph immediately following it. Here is the relevant part for our needs:

fn do_a() -> bool {
    // Complex operation which might fail
    true
}

fn do_very_expensive_a() -> bool {
    // Complex operation which might fail
    true
}

enum TypedIf {
    Then,
    Else
}

impl TypedIf {
    fn do_if(test: bool) -> TypedIf {
        if test {
            TypedIf::Then
        } else {
            TypedIf::Else
        }
    }

    fn and_then(self, test : bool) -> TypedIf {
        match self {
            TypedIf::Then => TypedIf::do_if(test),
            TypedIf::Else => TypedIf::Else
        }
    }
}

The first two functions (do_a & do_very_expensive_a) will be used later on to demonstrate various aspects related to our new type. The enum itself defined as TypedIf, and contain a way to model the two branches of the if statement. We see we have two functions working with this enum. The first do_if used to perform the if statement itself, and the second and_then is a function used to attach a second if statement in case the first one succeeded. As you can see it receives a result from a previous run of do_if, and conditionally running a new one.

Let’s Look at a usage example of the TypedIf and discuss the problem with it:

#[test]
fn closures_test() {
    let type_id = TypedIf::do_if(do_a()).and_then(do_very_expensive_a());
}

Both do_if and and_then suffer from the same problem, they accept a bool directly. It means the calculation resulting it, is being run when the if itself is defined. The above example shows one of the unfortunate events which could happen. We run do_very_expensive_a which takes a long time, no matter if do_if succeeded or not. In practice, we need to run it, only if do_if did succeed. This behavior is very wasteful, even worse, it is not too hard to come up with an example which would be dead simple wrong. So let’s see how to fix one of our functions, and_then:

fn better_and_then<F>(self, test : F) -> TypedIf
    where F: FnOnce() -> bool {
    match self {
        TypedIf::Then => TypedIf::do_if(test()),
        TypedIf::Else => TypedIf::Else
    }
}

#[test]
fn closures_test() {
    let type_id = TypedIf::do_if(do_a()).better_and_then(do_very_expensive_a);
    let type_id = TypedIf::do_if(do_a()).better_and_then(|| { do_very_expensive_a() });
}

The change is not very big. The argument now is a closure returning a bool, instead of receiving the bool directly. It has one fundamental difference, though. We would run the closure only if the previous test succeeded. You can also see how minor the change it requires in the call site. You have two options, either transferring the function directly or wrap it in a closure. Personally, I prefer the second option, although more verbose, it is very apparent a function is involved. As we just saw, code involving functions tends to behave a little differently, and I like being explicit about it.

Closure - Composability

The last capability we will discuss today is composability. While not limited to Rust, it is definitely the place where closures shine the brightest in Rust. Closures are being used to enable composability extensively in Rust, and they are vital to avoid countless awkward situations. As we’ve already seen, throwing generics parameter into the mix, allows your code to be agnostic to ownership. A composable system which is agnostic to ownership is friendly to use, especially among beginners. Let’s see how closure allows our TypedIf enum to be composable.

We will add two functions, compose_do_if and compose_better_and_then (yes funny name I know, don’t really name your function like this in real code). The changes for both is similar:

1. Accept additional closure called compose, receiving TypedIf and returning generic parameter U
2. Return the generic parameter U instead of TypedIf
3. Calling compose where we previously returned a TypedIf

impl TypedIf {
    // ...

    fn compose_do_if<F, U>(test:bool, compose : F) -> U
        where F: FnOnce(TypedIf) -> U {
        if test {
            compose(TypedIf::Then)
        } else {
            compose(TypedIf::Else)
        }
    }

    fn compose_better_and_then<P, F, U>(self, test : P, compose : F) -> U
        where P: FnOnce() -> bool,
                F: FnOnce(TypedIf) -> U {
        match self {
            TypedIf::Then => TypedIf::compose_do_if(test(), compose),
            TypedIf::Else => compose(TypedIf::Else)
        }
    }
}

The final result allows the user to pass additional closure. This closure receives the result as a TypedIf, and returns whatever the user prefers. Let see how we can use it:

#[test]
fn closures_test() {
    let bool_result = TypedIf::do_if(do_a())
        .compose_better_and_then(|| {do_very_expensive_a()}, |result| {
            match result {
                TypedIf::Then => true,
                TypedIf::Else => false
            }
        });
}

We used it to translate the final result from the TypedIf to a regular bool. While a type like TypedIf can be awesome, some of the code you interact with is unaware of it. Being capable of efficiently converting it to a more common type like bool, is a great advantage. I admit it might be an overkill for this simple example. But in many cases, especially where converting between owned types to reference or the other way around, this technique is instrumental. We are going to generalize this concept even further in the post about Option & Result.

With this last example, I conclude this post. You can check out the code accompanying this post here

We are getting close to the end of what I personally refer to “Chapter One” of the series. In fact, we have one more topic to cover, “Option & Result” and the related issue of combinators. We will discuss those next time!