Merge rust-bitcoin/rust-bitcoin#1495: Introduce mutation testing

8ce928b8e7 Add testing section to readme (Tobin C. Harding)
2e79a0bdc4 Introduce mutation testing (Tobin C. Harding)

Pull request description:

  Introduce mutation testing by way of mutagen [0] (see #1484 for context).

  - Conditionally add the dev-dependency `mutagen` (using `do_mutate` flag)

      This flag is not very well named but `mutagen` and `mutate` are already taken?

  - Mutate all methods of the `U256` struct that do not require additional unit tests.

      Uses `cfg(all(test, do_mutate), mutate)` - I cannot workout why we need to check on `test` as well i.e., I don't understand why we cannot use `cfg(do_mutate, mutate)`?

  With this applied test can be run as usual with a stable toolchain. To run mutagen we use `RUSTFLAGS='--cfg=do_mutate' cargo +nightly mutagen` (doing so runs 29 mutants).

  [0] https://github.com/llogiq/mutagen

ACKs for top commit:
  Kixunil:
    ACK 8ce928b8e7
  apoelstra:
    ACK 8ce928b8e7

Tree-SHA512: 024ba5d2dc983f7cd0444e09ba13280771157204999d2a44502e07a1d6436f571b75003c7cb543c943f17949b848d4070eda4e194bda774a3e41443ff79af0af
This commit is contained in:
Andrew Poelstra 2022-12-24 05:53:55 +00:00
commit 615759a8c2
No known key found for this signature in database
GPG Key ID: C588D63CE41B97C1
3 changed files with 42 additions and 1 deletions

View File

@ -114,11 +114,39 @@ shell alias to check your documentation changes build correctly.
alias build-docs='RUSTDOCFLAGS="--cfg docsrs" cargo +nightly rustdoc --features="$FEATURES" -- -D rustdoc::broken-intra-doc-links'
```
### Running benchmarks
## Testing
Unit and integration tests are available for those interested, along with benchmarks. For project
developers, especially new contributors looking for something to work on, we do:
- Fuzz testing with [`Hongfuzz`](https://github.com/rust-fuzz/honggfuzz-rs)
- Mutation testing with [`Mutagen`](https://github.com/llogiq/mutagen)
- Code verification with [`Kani`](https://github.com/model-checking/kani)
There are always more tests to write and more bugs to find, contributions to our testing efforts
extremely welcomed. Please consider testing code a first class citizen, we definitely do take PRs
improving and cleaning up test code.
### Unit/Integration tests
Run as for any other Rust project `cargo test --all-features`.
### Benchmarks
We use a custom Rust compiler configuration conditional to guard the bench mark code. To run the
bench marks use: `RUSTFLAGS='--cfg=bench' cargo +nightly bench`.
### Mutation tests
We have started doing mutation testing with [mutagen](https://github.com/llogiq/mutagen). To run
these tests first install the latest dev version with `cargo +nightly install --git https://github.com/llogiq/mutagen`
then run with `RUSTFLAGS='--cfg=mutate' cargo +nightly mutagen`.
### Code verification
We have started using [kani](https://github.com/model-checking/kani), install with `cargo install
--locked kani-verifier` (no need to run `cargo kani setup`). Run the tests with `cargo kani`.
## Pull Requests
Every PR needs at least two reviews to get merged. During the review phase

View File

@ -54,6 +54,9 @@ serde_derive = "1.0.103"
secp256k1 = { version = "0.25.0", features = [ "recovery", "rand-std" ] }
bincode = "1.3.1"
[target.'cfg(mutate)'.dev-dependencies]
mutagen = { git = "https://github.com/llogiq/mutagen" }
[[example]]
name = "bip32"

View File

@ -10,6 +10,9 @@
use core::fmt::{self, LowerHex, UpperHex};
use core::ops::{Add, Div, Mul, Not, Rem, Shl, Shr, Sub};
#[cfg(all(test, mutate))]
use mutagen::mutate;
use crate::consensus::encode::{self, Decodable, Encodable};
#[cfg(doc)]
use crate::consensus::Params;
@ -322,6 +325,7 @@ impl U256 {
const ONE: U256 = U256(0, 1);
/// Creates [`U256`] from a big-endian array of `u8`s.
#[cfg_attr(all(test, mutate), mutate)]
fn from_be_bytes(a: [u8; 32]) -> U256 {
let (high, low) = split_in_half(a);
let big = u128::from_be_bytes(high);
@ -330,6 +334,7 @@ impl U256 {
}
/// Creates a [`U256`] from a little-endian array of `u8`s.
#[cfg_attr(all(test, mutate), mutate)]
fn from_le_bytes(a: [u8; 32]) -> U256 {
let (high, low) = split_in_half(a);
let little = u128::from_le_bytes(high);
@ -338,6 +343,7 @@ impl U256 {
}
/// Converts `Self` to a big-endian array of `u8`s.
#[cfg_attr(all(test, mutate), mutate)]
fn to_be_bytes(self) -> [u8; 32] {
let mut out = [0; 32];
out[..16].copy_from_slice(&self.0.to_be_bytes());
@ -346,6 +352,7 @@ impl U256 {
}
/// Converts `Self` to a little-endian array of `u8`s.
#[cfg_attr(all(test, mutate), mutate)]
fn to_le_bytes(self) -> [u8; 32] {
let mut out = [0; 32];
out[..16].copy_from_slice(&self.1.to_le_bytes());
@ -377,8 +384,10 @@ impl U256 {
ret.wrapping_inc()
}
#[cfg_attr(all(test, mutate), mutate)]
fn is_zero(&self) -> bool { self.0 == 0 && self.1 == 0 }
#[cfg_attr(all(test, mutate), mutate)]
fn is_one(&self) -> bool { self.0 == 0 && self.1 == 1 }
fn is_max(&self) -> bool { self.0 == u128::max_value() && self.1 == u128::max_value() }
@ -571,6 +580,7 @@ impl U256 {
/// Returns `self` incremented by 1 wrapping around at the boundary of the type.
#[must_use = "this returns the result of the increment, without modifying the original"]
#[cfg_attr(all(test, mutate), mutate)]
fn wrapping_inc(&self) -> U256 {
let mut ret = U256::ZERO;