grcov collects and aggregates code coverage information for multiple source files

gcda files which can be generated from llvm/clang or gcc

grcov

grcov collects and aggregates code coverage information for multiple source files. grcov processes .profraw and .gcda files which can be generated from llvm/clang or gcc. grcov also processes lcov files (for JS coverage) and JaCoCo files (for Java coverage). Linux, macOS and Windows are supported.

This is a project initiated by Mozilla to gather code coverage results on Firefox.

Table of Contents

  • man grcov
  • How to get grcov
  • Usage
    • Example: How to generate source-based coverage for a Rust project
    • Example: How to generate .gcda files for C/C++
    • Example: How to generate .gcda files for a Rust project
    • Generate a coverage report from coverage artifacts
      • LCOV output
      • Coveralls/Codecov output
      • grcov with Travis
    • Alternative reports
    • Hosting HTML reports and using coverage badges
      • Example
    • Enabling symlinks on Windows
  • Auto-formatting
  • Build & Test
  • Minimum requirements
  • License

man grcov

How to get grcov

Grcov can be downloaded from releases or, if you have Rust installed, you can run cargo install grcov.

Usage

Nightly Rust is required to use grcov for Rust coverage. Alternatively, you can export RUSTC_BOOTSTRAP=1, which basically turns your stable rustc into a Nightly one.

Example: How to generate source-based coverage for a Rust project

  1. Install the llvm-tools or llvm-tools-preview component:

  2. Ensure that the following environment variable is set up:

  3. Build your code:

    cargo build

  4. Ensure each test runs gets its own profile information by defining the LLVM_PROFILE_FILE environment variable (%p will be replaced by the process ID, and %m by the binary signature):

  5. Run your tests:

    cargo test

In the CWD, you will see a .profraw file has been generated. This contains the profiling information that grcov will parse, alongside with your binaries.

Example: How to generate .gcda files for C/C++

Pass --coverage to clang or gcc (or for older gcc versions pass -ftest-coverage and -fprofile-arcs options (see gcc docs).

Example: How to generate .gcda files for a Rust project

  1. Ensure that the following environment variables are set up:

    These will ensure that things like dead code elimination do not skew the coverage.

  2. Build your code:

    cargo build

    If you look in target/debug/deps dir you will see .gcno files have appeared. These are the locations that could be covered.

  3. Run your tests:

    cargo test

    In the target/debug/deps/ dir you will now also see .gcda files. These contain the hit counts on which of those locations have been reached. Both sets of files are used as inputs to grcov.

Generate a coverage report from coverage artifacts

Generate a html coverage report like this:

N.B.: The --binary-path argument is only necessary for source-based coverage.

You can see the report in target/debug/coverage/index.html.

(or alternatively with -t lcov grcov will output a lcov compatible coverage report that you could then feed into lcov's genhtml command).

LCOV output

By passing -t lcov you could generate an lcov.info file and pass it to genhtml:

LCOV output should be used when uploading to Codecov, with the --branch argument for branch coverage support.

Coveralls output

Coverage can also be generated in coveralls format:

grcov with Travis

Here is an example of .travis.yml file for source-based coverage:

Here is an example of .travis.yml file:

Alternative reports

grcov provides the following output types:

Output Type -t Description
lcov (default) lcov's INFO format that is compatible with the linux coverage project.
ade ActiveData-ETL format. Only useful for Mozilla projects.
coveralls Generates coverage in Coveralls format.
coveralls+ Like coveralls but with function level information.
files Output a file list of covered or uncovered source files.
covdir Provides coverage in a recursive JSON format.
html Output a HTML coverage report, including coverage badges for your README.

Hosting HTML reports and using coverage badges

The HTML report can be hosted on static website providers like GitHub Pages, Netlify and others. It is common to provide a coverage badge in a project's readme to show the current percentage of covered code.

To still allow adding the badge when using a static site host, grcov generates coverage badges and a JSON file with coverage information that can be used with https://shields.io to dynamically generate badges.

The coverage data for htttps://shields.io can be found at /coverage.json and the generated bagdes are available as SVGs at /badges/*svg.

The design of generated badges is taken from shields.io but may not be updated immediately if there is any change. Using their endpoint method is recommended if other badges from their service are used already.

Enabling symlinks on Windows

grcov uses symbolic links to avoid copying files, when processing directories of coverage data. On Windows, by default, creating symbolic links to files requires Administrator privileges. (The reason is to avoid security attacks in applications that were designed before Windows added support for symbolic links.)

When running on Windows grcov will attempt to create a symbolic link. If that fails then grcov will fall back to copying the file. Copying is less efficient but at least allows users to run grcov. grcov will also print a warning when it falls back to copying a file, advising the user either to enable the privilege for their account or to run as Administrator.

You can enable the "Create Symbolic Links" privilege for your account so that you do not need to run as Administrator to use grcov.

  1. Click Start, then select "Local Group Policy Editor". Or just run gpedit.msc to open it directly.
  2. In the navigation tree, select "Computer Configuration", "Windows Settings", "Security Settings", "Local Policies".
  3. In the pane on the right, select "Create symbolic links" and double-click it.
  4. Click "Add User or Group", and add your account.
  5. Log out and then log back in.

Example

Let's consider we have a project at with username sample and project awesome that is hosted with GitHub Pages at https://sample.github.io/awesome.

By using the the shields.io endpoint we can create a Markdown badge like so:

If we want to avoid using shields.io as well, we can use the generated badges as follows (note the different URL for the image):

Auto-formatting

This project is using pre-commit. Please run pre-commit install to install the git pre-commit hooks on your clone. Instructions on how to install pre-commit can be found here.

Every time you will try to commit, pre-commit will run checks on your files to make sure they follow our style standards and they aren't affected by some simple issues. If the checks fail, pre-commit won't let you commit.

Build & Test

Build with:

To run unit tests:

To run integration tests, it is suggested to use the Docker image defined in tests/Dockerfile. Simply build the image to run them:

Otherwise, if you don't want to use Docker, the only prerequisite is to install GCC 7, setting the GCC_CXX environment variable to g++-7 and the GCOV environment variable to gcov-7. Then run the tests with:

Minimum requirements

  • GCC 4.9 or higher is required (if parsing coverage artifacts generated by GCC).
  • Rust 1.52

License

Published under the MPL 2.0 license.

Issues

Collection of the latest Issues

AntBlo

AntBlo

0

If all tests pass, the generated html shows everything in green.
But if one test fails, the other fails too, as can be seen below: image

I've tried the commands in grcov's README.md, and attempted some variants like adding --no-fail-fast to the cargo test command, but neither outputs a profraw file.

However, for some reason, reloading vscode generates a default.profraw file.
I suspect it's due to the Rust Analyzer extension. The result of this default.profraw file can be seen in the above image, where one of the tests should be passing while the other should be green.

Here's a repo to hopefully be able to reproduce the issue: https://github.com/AntBlo/grcov-test

I realize grcov isn't stable yet, and that this may just haven't been implemented yet. Not sure if it's an edge case on const functions or something, but I've seen it give reports on failing and succeeding tests in other parts of my main repo.

Furthermore, I'm using Windows. But this seems to happen on my Linux desktop too (Ubuntu 21.04) I think.

rhysd

rhysd

1

Since -C instrument-coverage was stabilized at Rust 1.60, I tried it with grcov and found an issue on Windows.

Repro

I created a demo repository for this issue. It has a single tiny file src/lib.rs. It measures the coverage on GitHub Actions Windows runner and reports it to Codecov.

https://github.com/rhysd/grcov-windows-bug

  1. Fork the repository, enable GitHub Actions and push an empty commit
  2. Wait CI job finishes
  3. Check Codecov results

Expected behavior

Since the unit test covers all lines, coverage should be 100%.

Actual behavior

Coverage is 36% though all lines are covered. Though the file contains only 9 LoC, the lcov output reports it contains 25 LoC.

スクリーンショット 2022-04-09 21 00 15

Notes

  • This does not occur on macOS or Linux. The output file lcov.info is far larger on Windows than on macOS or Linux. It is 35.7KB only for one function with 3 lines. Here is the generated lcov output: artifact.zip
  • This does not occur when the project depends on no crate. So I added anyhow to dependency as example, but I don't think this issue is not specific for the crate.
  • This does not occur when the output format is HTML (-t html).

Links

Versions

  • Rust: 1.60.0
  • grcov: v0.8.7 or v0.8.9
giovannitangredi

giovannitangredi

2

I was testing grcov on some large repository and while try to getting the coveralls json for the cargo repository(https://github.com/rust-lang/cargo), grcov panics while running the following comand: grcov . --binary-path ./target/debug/ -t coveralls -s . --token YOUR_COVERALLS_TOKEN > coveralls.json these are all the executed steps:

  • rustup component add llvm-tools-preview
  • export RUSTFLAGS="-Zinstrument-coverage" && export LLVM_PROFILE_FILE="your_name-%p-%m.profraw"
  • cargo test
  • grcov . --binary-path ./target/debug/ -t coveralls -s . --token YOUR_COVERALLS_TOKEN > coveralls.json
TomPridham

TomPridham

0

i am running grcov locally against a closed source project for my company and am getting duplicate results for files, one result under the relative path and another under the absolute path. that wouldn't be too much of an issue except that they have different results. i'm currently working around it by just preferring the relative results when interpreting the results, but it would be nice to not have duplicate results test command i'm running: CARGO_INCREMENTAL=0; RUSTFLAGS="-Z instrument-coverage -C codegen-units=1 -C opt-level=0 -C link-dead-code -C overflow-checks=off"; LLVM_PROFILE_FILE="coverage/rover_test-%p-%m.profraw"; cargo test +nightly coverage command i'm running: grcov . --branch -b ./target/debug/build --ignore *github.com* --ignore *libcore* --ignore *rustc* --ignore *liballoc* -t lcov -o ./coverage/coverage.info relative path result: https://gist.github.com/TomPridham/be7b08ab6cacfc68c57a25bad1458e19 absolute path result: https://gist.github.com/TomPridham/1a241dba1e4b634dff273fd319c5a9a6

federicomenaquintero

federicomenaquintero

1

Let's say you have this source tree:

If you then get an HTML report with something like

what happens is that the initial html::gen_index() gets passed output=./html, and so it creates ./html/index.html with the overall reports for src/ and src/some_dir/. This fine.

However, the presence of build.rs in the toplevel causes the subsequent html::gen_dir_index() to get passed output=./html but dir_name="", and so it overwrites the ./html/index.html that had the directory index with the report for a directory with a single file, namely build.rs.

One ends up with html/index.html that only lists build.rs, but no overall list of directories scanned.

If one runs grcov with --ignore build.rs, it works as expected — since there are no source files to be scanned in the toplevel, html::gen_dir_index() never gets called with dir_name being an empty string. However, I'd really like to get a coverage report for build.rs as well! :smiley:

Maybe the the toplevel index.html should list the directories, but also any source files in the toplevel source directory?

trinity-1686a

trinity-1686a

2

Hi, I'm trying to use grcov to get code coverage on a project hosted on Gitlab. I got blocked because Gitlab won't process coverage reports of more than 10MB (comment) and the report I'm generating is about 43MB.

After searching for a way to reduce the size of the report, I noticed it seemed to contain some duplicated data. It turns out it's actually a substantial part of the report. There are 55812 <method ..> reported, but only 11009 unique ones. 2462 blocks are never duplicated, and 8547 are duplicated, from 2 to 68 times.

how to reproduce :

Inside with_coverage.sh, grcov invocation looks like the following:

Edit: after testing with lcov, this does not seems related to the output format, which gives exactly the same result. Looking inside what's grcov is doing, it looks like it have multiple names (with different mangling) for the same function. This is probably the source of the problem

refaelsh

refaelsh

0

I am seeing wrong/strange paths in Cobertura report. Here is what I do:

And here is and example of a strange/wrong path in the resulting coverage.xml file:

Please help :-)

UPDATE: I've noticed that if I add this --ignore "$HOME/.cargo/*" to the grcov command mentioned above, the problem goes away. Now, the question is how did the $HOME/.cargo/* folder got added in the first place?

lu-zero

lu-zero

0

rust-code-analysis can provide information regarding how complex a function is according to some metrics.

The code coverage information provides a good estimation on how well the code is tested, but does that w/out any knowledge of what the code actually is.

Using the complexity information to give different weight on the line of code may help focusing first on testing the code that may contain more bugs since it is objectively harder to understand according to the metrics at hand.

This issue is written to track the idea and the experiment we can do to see if it is sound in practice.

seanpianka

seanpianka

2

Hello! :wave: The command shown below works locally on MacOS, but fails in a CI job running alpine3.13 with musl and ldd (configured in ~/.cargo/config as the linker):

grcov recently started to overflow its stack during code coverage processing. Any tips for debugging this issue? Thanks!

Versions

Find the latest versions by id

v0.8.7 - Feb 11, 2022

v0.8.6 - Feb 01, 2022

v0.8.5 - Feb 01, 2022

v0.8.2 - Jul 30, 2021

v0.8.1 - Jul 29, 2021

v0.8.0 - Apr 26, 2021

v0.7.1 - Jan 21, 2021

v0.7.0 - Jan 21, 2021

v0.6.1 - Nov 27, 2020

v0.6.0 - Nov 24, 2020

v0.5.15 - May 13, 2020

v0.5.14 - May 11, 2020

v0.5.13 - Mar 30, 2020

v0.5.12 - Mar 26, 2020

v0.5.11 - Mar 26, 2020

v0.5.10 - Mar 25, 2020

v0.5.9 - Jan 09, 2020

v0.5.8 - Jan 07, 2020

v0.5.7 - Dec 19, 2019

v0.5.6 - Dec 18, 2019

v0.5.5 - Oct 07, 2019

v0.5.4 - Oct 04, 2019

v0.5.3 - Aug 08, 2019

v0.5.2 - Aug 08, 2019

v0.5.1 - Jun 03, 2019

v0.5.0 - May 14, 2019

v0.4.3 - Mar 25, 2019

v0.4.2 - Mar 25, 2019

v0.4.1 - Jan 15, 2019

v0.4.0 - Jan 14, 2019

Information - Updated May 06, 2022

Stars: 714
Forks: 105
Issues: 91

Repositories & Extras

serde-json for no_std programs

MIT license (LICENSE-MIT or

serde-json for no_std programs

macOS/iOS Security framework for Rust

MIT license (LICENSE-MIT or

macOS/iOS Security framework for Rust

OpenGliderNetwork client for Rust based on actix

MIT license (LICENSE-MIT or

OpenGliderNetwork client for Rust based on actix

A video game for programmers set in space

MIT license (LICENSE-MIT or

A video game for programmers set in space

Simple SQL migration manager for your project

MIT license (LICENSE-MIT or

Simple SQL migration manager for your project

TODO_README_DESCRIPTION

(LICENSE-MIT or Semantic Versioning 2

TODO_README_DESCRIPTION

CBOR Event library

MIT license (LICENSE-MIT or

CBOR Event library

HAL for the STM32WB55 family of microcontrollers

MIT license (LICENSE-MIT or

HAL for the STM32WB55 family of microcontrollers

A simple easy to use wrapper around Ctrl-C signal

MIT license (LICENSE-MIT or

A simple easy to use wrapper around Ctrl-C signal
Facebook Instagram Twitter GitHub Dribbble
Privacy