japaric/utest

utest standard testing for Rust

Make sure your not using this for testing no_std code as it relies on the unstable branch

(2021-04-11) Hey, I don't recommend this approach for testing no_std code as it relies on unstable details about the Rust compiler and the standard library so it's prone to breakage. For a stable approach you can use a custom Cargo runner and a procedural macro as it's done in the defmt-test crate.

If you'd like to know more about testing embedded Rust firmware in general I recommend this series of blog posts.


μtest

Unit #[test]ing for microcontrollers and other no_std systems

WARNING This crate relies on #[test] / rustc implementation details and could break at any time.

Table of Contents

  • Features

  • Limitations

  • Testing on an emulated Cortex-M processor

  • Testing on a real Cortex-M microcontroller

    • Requirements
    • Steps
  • Building a custom test runner

    • Hooks
      • __test_start
      • __test_ignored
      • __test_before_run
      • __test_failed
      • __test_success
      • __test_summary
      • __test_panic_fmt
  • How does this work without unwinding?

  • License

    • Contribution

Features

  • Doesn't depend on std.

  • Fully configurable, through hooks.

Limitations

  • Tests are executed sequentially. This is required to support bare metal systems where threads may not be implemented.

  • All tests will print to stdout / stderr as they progress.

  • panic!s outside the crate under test will NOT mark the unit test as failed; those panics will likely abort the test runner but this is implementation defined. (more about this later)

  • #[bench] is not supported.

  • No colorized output

Testing on an emulated Cortex-M processor

Using the utest-cortex-m-qemu test runner.

This uses QEMU user emulation to emulate a Cortex-M processor that has access to the host Linux kernel thus you can do stuff like using the WRITE system call to print to the host console.

The downside of this approach is that the QEMU user emulation doesn't emulate the peripherals of a Cortex-M microcontroller so this is mainly useful to test pure functions / functions that don't do embedded I/O (by embedded I/O, I mean I2C, Serial, PWM, etc.).

  1. Start with a no_std library crate.

  2. Append this to your crate's Cargo.toml

NOTE Change thumbv7m-linux-eabi as necessary. The other options are thumbv6m-linux-eabi, thumbv7em-linux-eabi and thumbv7em-linux-eabihf. (Yes, linux not none)

  1. Add this to your src/lib.rs

  2. Create the target specification file.

Start with this target specification

And perform these modifications

  1. Built the test runner

  2. Execute the test runner using QEMU

Testing on a real Cortex-M microcontroller

Using the utest-cortex-m-semihosting test runner.

Requirements

  • Your target crate must support vanilla fn main(). This means that the start lang item must be defined somewhere in your crate dependency graph.

  • The panic_fmt lang item must be defined in your crate dependency graph. Hitting panic_fmt while running the test suite is considered a fatal error so it doesn't matter how you have implemented it.

These two requirements can be fulfilled if your crate is based on the cortex-m-template.

  • You should be able to use GDB to run / debug a Rust program. GDB is required for semihosting.

Steps

  1. Start with a crate that meets the requirements and some unit tests.

  2. Append this to your Cargo.toml

  3. Add this to you your integration test file (tests/foo.rs as per our example)

  4. If required (this is required for cortex-m-template based crates), define how exceptions and interrupts are handled. In our example, add this to tests/foo.rs.

  5. Build the test runner

  6. Flash the test runner and execute the program using GDB.

NOTE These steps assume OpenOCD support.

If testing a crate based on the cortex-m-template, you'll only have to launch OpenOCD.

and then launch GDB.

You should see this in the OpenOCD output

If you are not using a cortex-m-template based crate, then make sure you enable semihosting from the GDB command line.

Building a custom test runner

You can create a custom test runner that, for example, doesn't require executing the test runner under GDB and that instead reports the tests results via Serial port or ITM.

The best way to implement a custom test runner is to base your implementation on the implementation of the two tests runners shown above.

But in a nutshell you'll have to define all these "hook" functions:

Hooks

Hooks are just vanilla functions with predefined symbol names that configure the behavior of the test runner.

__test_start

Runs before the unit tests are executed.

Signature:

__test_ignored

Runs when a test if marked as #[ignore]d.

Signature:

__test_before_run

Runs right before an unit test gets executed.

Signature:

__test_failed

Runs if the unit test failed

Signature:

__test_success

Runs if the unit test succeeded

Signature:

__test_summary

Runs after all the unit tests have been executed.

Signature:

__test_panic_fmt

Runs when utest-macros's panic! macro is called.

Signature:

How does this work without unwinding?

std unit tests rely heavily on unwinding. Each unit test is run inside a catch_unwind block and if the unit test panics then the panic is caught and the test is marked as failed (or as passed if the unit test was marked with #[should_panic]).

We attempt to emulate this behavior by overriding the panic! macro to mark the test failed and then early return instead of unwind. Of course, this emulation breaks down if the panic! originates from outside the crate under test, because panic is not overridden in that scope. So this setup is certainly not perfect.

License

Licensed under either of

  • Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)

  • MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Issues

Collection of the latest Issues

MathiasKoch

MathiasKoch

Comment Icon1

Hi.

Is there a way of using this crate for unit testing (either emulated or on real hardware, preferably both) without the use of xargo?

I am using RTFM on a stm32l4 processor, and would very much like to be able to run unit tests in my no_std environment.

Information - Updated Aug 26, 2022

Stars: 126
Forks: 6
Issues: 3

Repositories & Extras

K9 - Rust Testing Library

Snapshot testing + better assertions

K9 - Rust Testing Library

Spec "it" for Rust testing

The test output is like the following

Spec "it" for Rust testing

Shuttle is a library for testing concurrent Rust code

It is an implementation of a number of

Shuttle is a library for testing concurrent Rust code

Let's make a web service and client in Rust

So I'm working on this project Rust regression testing

Let's make a web service and client in Rust

Simple golden file testing for Rust

Add the following to your Cargo manifest

Simple golden file testing for Rust
Http

329

HTTP mocking to test Rust applications

wiremock provides HTTP mocking to perform black-box testing of Rust applications that

HTTP mocking to test Rust applications

just testing rust

cli + advanced cli features

just testing rust

Testing Rust Code

This repo contains examples of many common features and approaches for testing here

Testing Rust Code
Facebook Instagram Twitter GitHub Dribbble
Privacy