cargo-all-features

Cargo subcommands that build and test all feature flag combinations for a crate

cargo-all-features

Cargo subcommands that build and test all feature flag combinations for a crate.

Install

Usage

The following commands can be run within a Cargo package or at the root of a Cargo workspace.

Build crate with all feature flag combinations:

Check crate with all feature flag combinations:

Test crate with all feature flag combinations:

Why?

If you have a crate that utilizes Rust feature flags, it’s common to set up a test matrix in your continuous integration tooling to individually test all feature flags. This setup can be difficult to maintain and easy to forget to update as feature flags come and go. It’s also not exhaustive, as it’s possible enabling combinations of feature flags could result in a compilation error that should be fixed. This utility was built to address these concerns.

Options

You can add the following options to your Cargo.toml file to configure the behavior of cargo-all-features under the heading [package.metadata.cargo-all-features]:

License

Licensed under either of

  • Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
  • MIT license (LICENSE-MIT or #404)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Issues

Collection of the latest Issues

Shadow53

Shadow53

0

Suppose I have a workspace with crates foo and bar:

If I want to run cargo test-all-features on crate foo, I expect to be able to run this command from the root of the workspace:

and have it run tests only for foo. Instead, I get output like this:

As far as I can tell, test-all-features is running cargo test for all crates in the workspace and passing all arguments, including --manifest-path, to cargo test.

I need this to consider --manifest-path beforehand, though, as I am using actions-rs/cargo to test individual crates depending on which files got changed. actions-rs/cargo does not support any sort of working-directory input, so to work on a specific crate I have to use --manifest-path.

Is this a use case you'd be willing to support?

jjl

jjl

0

Hi. I was about to write this utility and then I found yours :)

It would be nice to have miri test support. It would be exactly like test-all-features except it would run cargo miri test instead of cargo test. miri-test-all-features seems like a good name.

Thoughts?

tyt2y3

tyt2y3

0

My crate currently has 10 features, it will then explode to 2^10 = 1024 combinations. It'd be useful to be able to test some meaningful subsets of all the possible combinations.

I'd propose some modes:

one-by-one: enable each feature one-by-one all-features: all features enabled pseudo-random: pseudo-randomly pick N combinations random: true-randomly pick N combinations mix-and-match: divide features into M groups, and randomly pick R features from each group

For example, if we divide 10 features into 2 groups of 5, and picking up to 3 from each group, then:

((5 choose 3) + (5 choose 2) + (5 choose 1)) ^ 2 = 625 combinations.

After that, we can still randomly pick N from these possible combinations.

It would be a more reasonable set of combinations than picking purely randomly, as I observe that crates with many features usually divide their feature flags by function categories. As an analogy people can choose different wheels and engines when building a car.

What do you think about this?

echeran

echeran

enhancement
2

Could there be a way to just print out the commands instead of executing them?

For our continuous integration setup in Github, we're noticing that the wall clock times of our simple build & test jobs have gone up in our pending switch to cargo-all-features. I suspect that it's due to all of the different combinations of feature sets being run serially.

In our CI, we have the ability to run jobs concurrently in order to save on wall clock time. In fact, our build & test job is set up that way using a "job matrix" in which we indicate which variables take on which enumerated sets of values, and the CI system creates the combinatorial number of jobs according to the Cartesian product of the enum sets. We currently run on 3 different OSes, and our enumerated set of feature sets is ["", "--all-features"], where the empty string actually refers to the default set.

So if there was a way to obtain the series of commands used to test each unique combination of features, then it would be possible to cut down on our wall clock time by a significant factor.

Does cargo-all-features work well with specifying parallel jobs using the -j option in cargo? Our CI system uses 2-core CPUs, which is something.

Independent of that, however, is the fact that the max distributed job concurrency allowed by the CI is 5 jobs. So having the sequence of commands used per unique combination of features could enable that improvement, too.

@sffc

woshilapin

woshilapin

enhancement
1

As I'm reading the documentation, only build-all-features and test-all-features are available. But it might be useful to support other commands too (I'm thinking of clippy but there might be others).

I'm wondering if an API like the following (a bit more generic) would be possible instead.

cuviper

cuviper

enhancement
1

In crates where I'm supporting older Rust versions, it's frequently the case that some of the features will only work on more recent Rust. So in order to use cargo all-features, I would need a way to list which features are actually expected to work for the Rust that I'm currently testing.

Another case is that many crates have features like "unstable" that would only be expected to work when using a nightly toolchain. That would need to be excluded when testing all stable features.

Information - Updated Nov 15, 2021

Stars: 63
Forks: 6
Issues: 6

netsim - A Rust library for network simulation and testing (currently linux-only)

netsim is a crate for simulating networks for the sake of testing network-oriented Rust

netsim - A Rust library for network simulation and testing (currently linux-only)

utest standard testing for Rust

Make sure your not using this for testing no_std code as it relies on the unstable branch

utest standard testing for Rust

K9 - Rust Testing Library

Snapshot testing + better assertions

K9 - Rust Testing Library

Spec "it" for Rust testing

The test output is like the following

Spec "it" for Rust testing

Shuttle is a library for testing concurrent Rust code

It is an implementation of a number of

Shuttle is a library for testing concurrent Rust code

A contract testing tool built in Rust using the filmReel format

0 or greater for making gRPC requests

A contract testing tool built in Rust using the filmReel format

Let's make a web service and client in Rust

So I'm working on this project Rust regression testing

Let's make a web service and client in Rust

Simple golden file testing for Rust

Add the following to your Cargo manifest

Simple golden file testing for Rust

cargo-mutants is a mutation testing tool for Rust

Decartes mutation-testing tool for Java

cargo-mutants is a mutation testing tool for Rust
Http

329

HTTP mocking to test Rust applications

wiremock provides HTTP mocking to perform black-box testing of Rust applications that

HTTP mocking to test Rust applications

just testing rust

cli + advanced cli features

just testing rust
Facebook Instagram Twitter GitHub Dribbble
Privacy