1. 1. Introduction
  2. 2. Getting Started
  3. 3. Tutorial: Guessing Game
  4. 4. Syntax and Semantics
    1. 4.1. Variable Bindings
    2. 4.2. Functions
    3. 4.3. Primitive Types
    4. 4.4. Comments
    5. 4.5. if
    6. 4.6. Loops
    7. 4.7. Vectors
    8. 4.8. Ownership
    9. 4.9. References and Borrowing
    10. 4.10. Lifetimes
    11. 4.11. Mutability
    12. 4.12. Structs
    13. 4.13. Enums
    14. 4.14. Match
    15. 4.15. Patterns
    16. 4.16. Method Syntax
    17. 4.17. Strings
    18. 4.18. Generics
    19. 4.19. Traits
    20. 4.20. Drop
    21. 4.21. if let
    22. 4.22. Trait Objects
    23. 4.23. Closures
    24. 4.24. Universal Function Call Syntax
    25. 4.25. Crates and Modules
    26. 4.26. `const` and `static`
    27. 4.27. Attributes
    28. 4.28. `type` aliases
    29. 4.29. Casting between types
    30. 4.30. Associated Types
    31. 4.31. Unsized Types
    32. 4.32. Operators and Overloading
    33. 4.33. Deref coercions
    34. 4.34. Macros
    35. 4.35. Raw Pointers
    36. 4.36. `unsafe`
  5. 5. Effective Rust
    1. 5.1. The Stack and the Heap
    2. 5.2. Testing
    3. 5.3. Conditional Compilation
    4. 5.4. Documentation
    5. 5.5. Iterators
    6. 5.6. Concurrency
    7. 5.7. Error Handling
    8. 5.8. Choosing your Guarantees
    9. 5.9. FFI
    10. 5.10. Borrow and AsRef
    11. 5.11. Release Channels
    12. 5.12. Using Rust without the standard library
    13. 5.13. Procedural Macros (and custom derive)
  6. 6. Nightly Rust
    1. 6.1. Compiler Plugins
    2. 6.2. Inline Assembly
    3. 6.3. No stdlib
    4. 6.4. Intrinsics
    5. 6.5. Lang items
    6. 6.6. Advanced linking
    7. 6.7. Benchmark Tests
    8. 6.8. Box Syntax and Patterns
    9. 6.9. Slice Patterns
    10. 6.10. Associated Constants
    11. 6.11. Custom Allocators
  7. 7. Glossary
  8. 8. Syntax Index
  9. 9. Bibliography

Benchmark tests

Rust supports benchmark tests, which can test the performance of your code. Let's make our src/lib.rs look like this (comments elided):

#![feature(test)]

extern crate test;

pub fn add_two(a: i32) -> i32 {
    a + 2
}

#[cfg(test)]
mod tests {
    use super::*;
    use test::Bencher;

    #[test]
    fn it_works() {
        assert_eq!(4, add_two(2));
    }

    #[bench]
    fn bench_add_two(b: &mut Bencher) {
        b.iter(|| add_two(2));
    }
}Run

Note the test feature gate, which enables this unstable feature.

We've imported the test crate, which contains our benchmarking support. We have a new function as well, with the bench attribute. Unlike regular tests, which take no arguments, benchmark tests take a &mut Bencher. This Bencher provides an iter method, which takes a closure. This closure contains the code we'd like to benchmark.

We can run benchmark tests with cargo bench:

$ cargo bench
   Compiling adder v0.0.1 (file:///home/steve/tmp/adder)
     Running target/release/adder-91b3e234d4ed382a

running 2 tests
test tests::it_works ... ignored
test tests::bench_add_two ... bench:         1 ns/iter (+/- 0)

test result: ok. 0 passed; 0 failed; 1 ignored; 1 measured

Our non-benchmark test was ignored. You may have noticed that cargo bench takes a bit longer than cargo test. This is because Rust runs our benchmark a number of times, and then takes the average. Because we're doing so little work in this example, we have a 1 ns/iter (+/- 0), but this would show the variance if there was one.

Advice on writing benchmarks:

Gotcha: optimizations

There's another tricky part to writing benchmarks: benchmarks compiled with optimizations activated can be dramatically changed by the optimizer so that the benchmark is no longer benchmarking what one expects. For example, the compiler might recognize that some calculation has no external effects and remove it entirely.

#![feature(test)]

extern crate test;
use test::Bencher;

#[bench]
fn bench_xor_1000_ints(b: &mut Bencher) {
    b.iter(|| {
        (0..1000).fold(0, |old, new| old ^ new);
    });
}Run

gives the following results

running 1 test
test bench_xor_1000_ints ... bench:         0 ns/iter (+/- 0)

test result: ok. 0 passed; 0 failed; 0 ignored; 1 measured

The benchmarking runner offers two ways to avoid this. Either, the closure that the iter method receives can return an arbitrary value which forces the optimizer to consider the result used and ensures it cannot remove the computation entirely. This could be done for the example above by adjusting the b.iter call to

b.iter(|| {
    // Note lack of `;` (could also use an explicit `return`).
    (0..1000).fold(0, |old, new| old ^ new)
});Run

Or, the other option is to call the generic test::black_box function, which is an opaque "black box" to the optimizer and so forces it to consider any argument as used.

#![feature(test)]

extern crate test;

b.iter(|| {
    let n = test::black_box(1000);

    (0..n).fold(0, |a, b| a ^ b)
})Run

Neither of these read or modify the value, and are very cheap for small values. Larger values can be passed indirectly to reduce overhead (e.g. black_box(&huge_struct)).

Performing either of the above changes gives the following benchmarking results

running 1 test
test bench_xor_1000_ints ... bench:       131 ns/iter (+/- 3)

test result: ok. 0 passed; 0 failed; 0 ignored; 1 measured

However, the optimizer can still modify a testcase in an undesirable manner even when using either of the above.