Commit Graph

3181 Commits

Author SHA1 Message Date
yancy 9536a9947c Add base_size test 2023-09-04 15:26:44 +02:00
Tobin C. Harding 4222e4d817
Add one-ack carve outside
In an effort to reduce review burden add so strict rules defining 3
times where we think it is acceptable for code to be merged with a
single ACK. Feels a bit like we only just added the refactor carve out
and now we are adding more carve outs, so whats next? We should take
these rules seriously, if we are to be taken seriously.
2023-09-04 08:17:22 +10:00
Tobin C. Harding b229fd5555
Use correct form for noun: carve-out
One is said to "carve out" something, not "carve output" - woops.
However, when used as a noun its written "carve-out".
2023-09-04 08:10:24 +10:00
Andrew Poelstra 8e573f4b57
Merge rust-bitcoin/rust-bitcoin#2018: Introduce a basic justfile
eccd3fe57b Introduce a basic justfile (Tobin C. Harding)

Pull request description:

  Introduce usage of `just` by adding a basic `justfile`.

  If this goes in we can add various script invocations to it plus other useful things that often get red CI runs (eg, checking no-std).

ACKs for top commit:
  RCasatta:
    ACK eccd3fe57b
  apoelstra:
    ACK eccd3fe57b

Tree-SHA512: c10514e0ee623f366904c9103bc961a1316b199c09083180767fb9b5864e94df122c9db1a3ea084c8790f0b849e89c68b304f87247b70763fb93c28a33642d28
2023-09-03 15:06:27 +00:00
Andrew Poelstra 1cb5cb7547
Merge rust-bitcoin/rust-bitcoin#2032: Update base64 usage to 0.21.3
18e2854a42 Update base64 usage to 0.21.3 (junderw)

Pull request description:

  Closes #2031

  The imports are a bit bike-sheddy... but I think this is fine.

ACKs for top commit:
  tcharding:
    ACK 18e2854a42
  apoelstra:
    ACK 18e2854a42

Tree-SHA512: 5707adc2066ef33a30e81eb74ddcb938350e5853438152278767c824a45909a0b4761d036e4e5f21fccd61102c4734936b5fd570dbd87140cd3b679dc9c7eec4
2023-09-02 14:01:57 +00:00
junderw 18e2854a42
Update base64 usage to 0.21.3 2023-08-30 22:47:56 -07:00
Tobin C. Harding bc398204bf
Remove redundant segwit version from function names
A P2TR output does not need to be clarified with version 1, it is
implicit. As with p2wpkh/p2wsh and version 0.

Remove redundant version identifiers from function names, deprecating
the originals.
2023-08-31 13:23:46 +10:00
Tobin C. Harding f17bb0d18f
Remove unnecessary reference
`T` is a generic that implements`AsRef<PushBytes>`, it should not be a
reference. This is inline with other usages of `AsRef<PushBytes>` for
example in `Builder::push_slice`.
2023-08-31 13:21:06 +10:00
Tobin C. Harding eccd3fe57b
Introduce a basic justfile
Introduce usage of `just` by adding a basic `justfile`.
2023-08-31 13:19:49 +10:00
Tobin C. Harding 4f43965ade
Make Encodable/Decodable usage uniform
One encodes to a writer and decodes from a reader, most of the time in
the consensus `Encodable`/`Decodable` traits we use generic `R`/`W` and
variable `r`/`w` but there are other places that use other characters.

While touching these lines note also that there are a bunch of unneeded
`mut`s, I'm not sure why since usually between the compiler and the
linter `mut` is handled correctly.

Make implementations of `Encodable` and `Decodable` uniform by:
- Use R/W and r/w for trait and variable name
- Remove unneeded mut
2023-08-31 13:14:02 +10:00
Tobin C. Harding 7309c7749a
Split witness version errors up
Done as part of the push to have small specific errors instead of large
general ones.

Split the `witness_version::Error` up into small specific errors.
2023-08-31 13:12:35 +10:00
Tobin C. Harding 40db2f5ed6
witness_version: Remove rustdocs from TryFrom imlps
These docs do not add much value. Done in preparation for splitting up
the `witness_version::Error` type into specific errors.
2023-08-31 13:12:35 +10:00
Tobin C. Harding 3397ff9910
witness_version: Use Self in error From impl
We can use self in error `From` impls with no loss of clarity and it is
more maintainable.
2023-08-31 13:12:35 +10:00
Tobin C. Harding de95bf52cb
Use checked_sub
Recently we "if" guarded subtraction manually using `> 0`, we can better
convey the meaning by using `checked_sub` and pattern match on the
option.

Refactor only, no logic changes.
2023-08-31 13:11:11 +10:00
Tobin C. Harding ccdcffe69c
Re-export Opcode
We recently rename `opcodes::All` to `Opcode` but did not re-export it
from the crate root. Since it now has a nice clear name we can do so.
2023-08-31 13:10:19 +10:00
Tobin C. Harding a0a3d4728a
Fix deprecation notice
Recently we deprecated the `segwit_signature_hash` function but during
development the deprecation notice got stale.

Fix deprecation notice to use the actual function names.
2023-08-31 13:06:03 +10:00
sanket1729 c06c9beb01
Merge rust-bitcoin/rust-bitcoin#2036: CI: Fx pinning
adcc01c0bd CI: Fix pinning (Tobin C. Harding)

Pull request description:

  More crates broke our MSRV by using edition 2021 without doing a major release, pin them in the CI script.

  Other open PRs need this to get past CI.

ACKs for top commit:
  sanket1729:
    ACK adcc01c0bd

Tree-SHA512: 3fbaaf2677791c4761fb0543a09f855441a977aec8d277c01c12c3389ce64158833b6a5ea89a3cf4f2182b51f480849deaf8e33127a14f818024a1af95346baa
2023-08-30 19:58:14 -07:00
Riccardo Casatta feafac3c65
Merge rust-bitcoin/rust-bitcoin#1999: Fix witness display bug
84614d9997 Unit test debug print of witness with empty instruction (Tobin C. Harding)
e96be5ee6e Fix Witness debug display bug (Tobin C. Harding)

Pull request description:

  When we introduce a custom `Debug` implementation for the `Witness` we introduced a bug that causes code to panic if the witness contains an empty instruction.

  The bug can be verified by putting patch 2 first or by running `cargo run --example sighash` on master.

ACKs for top commit:
  apoelstra:
    ACK 84614d9997
  RCasatta:
    ACK 84614d9997

Tree-SHA512: d51891206ab15f74dda07eb29ff3f6c69dc3f983a5a5abb55685688548481a19f7c1d33aa1183a89c553ff2bc86cf41057c2bae33d75e8a7f3b801056775bf9e
2023-08-30 11:48:27 +02:00
Tobin C. Harding adcc01c0bd
CI: Fix pinning
More crates broke our MSRV by using edition 2021 without doing a major
release, pin them in the CI script.

diff --git a/contrib/test.sh b/contrib/test.sh
index 74f8ffb3..79932ad5 100755
--- a/contrib/test.sh
+++ b/contrib/test.sh
@@ -13,6 +13,12 @@ if cargo --version | grep ${MSRV}; then
     cargo update -p quote --precise 1.0.30
     cargo update -p proc-macro2 --precise 1.0.63
     cargo update -p serde_test --precise 1.0.175
+    # Have to pin this so we can pin `schemars_derive`
+    cargo update -p schemars --precise 0.8.12
+    # schemars_derive 0.8.13 uses edition 2021
+    cargo update -p schemars_derive --precise 0.8.12
+    # memcrh 2.6.0 uses edition 2021
+    cargo update -p memchr --precise 2.5.0

     cargo update -p bitcoin:0.30.1 --precise 0.30.0
2023-08-29 12:33:19 +10:00
Andrew Poelstra 7fd9b89e82
Merge rust-bitcoin/rust-bitcoin#2010: Use weight type for stripped_size
55e94b5dea Remove test from test names for Weight type (yancy)
142dde64c3 Use Weight type for stripped_size (yancy)
cb76f3ec43 Add scale_by_witness_factor to Weight type (yancy)
38c9e9947e Add witness scale factor to the Weight type (yancy)
77552987ab Add from_wu_usize to Weight type (yancy)
1a88c887f5 Rename strippedsize to stripped_size (yancy)
3369257c75 Fix grammar (yancy)

Pull request description:

  Return Weight type for the strippedize function.

ACKs for top commit:
  apoelstra:
    ACK 55e94b5dea
  tcharding:
    ACK 55e94b5dea

Tree-SHA512: ad3e4bc29380f22e20a6302c1b24c201c772be759c655c62ba4717840a01fcaa36f0f8442c9a3ba71c6400d6af47a9a815e6d90877b5f14c6883fb950b9669fd
2023-08-26 16:12:43 +00:00
Andrew Poelstra eef5cc3b92
Merge rust-bitcoin/rust-bitcoin#1978: Wrap the `bitcoinconsensus` error
29a4f9b114 Wrap the bitcoinconsensus error (Tobin C. Harding)

Pull request description:

  Currently the `bitcoinconsensus` error is part of the public API. This hinders maintainability because changes to the verison of `bitcoinconsensus` force a re-release in `rust-bitcoin`. This is an unnecessary maintenance burden, we can wrap the error instead.

ACKs for top commit:
  apoelstra:
    ACK 29a4f9b114
  sanket1729:
    utACK 29a4f9b114

Tree-SHA512: 36bc1b0ad5f5675d79eea2409844a839d862997c256e301c53c5f1af547edc9a0b83e586bd70e1b8853722cd7ef279e7515e09fbe942660f8049090d1be39d3a
2023-08-25 15:06:42 +00:00
Andrew Poelstra 4dc71b0de7
Merge rust-bitcoin/rust-bitcoin#2020: Add version bytes consts
f18f684ad2 Add version bytes consts (Tobin C. Harding)

Pull request description:

  BIP-32 defines 4 4-byte consts used as version bytes; currently we are hardcoding the version bytes in multiple places.

  Add BIP-32 version bytes consts and use them throughout the module.

ACKs for top commit:
  apoelstra:
    ACK f18f684ad2
  RCasatta:
    utACK f18f684ad2

Tree-SHA512: 50bf2d26f0f8e3528642ffcc621c03b82f536994deb808a6c84225676b4b8849db8e0d16e46f3819e0810296a422b31cf90d0595739910afdb92fb768ef7696e
2023-08-25 14:57:59 +00:00
Andrew Poelstra a458461a6b
Merge rust-bitcoin/rust-bitcoin#2004: psbt: Add IndexOutOfBounds error
66d5800ac0 psbt: Add IndexOutOfBounds error (Tobin C. Harding)

Pull request description:

  We currently have a bunch of functions that are infallible if the `index` argument is within-bounds however we return a `SignError`, this obfuscates the code.

  Add an `IndexOutOfBoundsError`. While we are at it make it an enum so users can differentiate between which vector the out of bounds access was attempted against.

ACKs for top commit:
  sanket1729:
    utACK 66d5800ac0. This is a clean improvement over existing code.
  apoelstra:
    ACK 66d5800ac0

Tree-SHA512: fa8a24990d1dcdab0c9b019fb2387b5a518b02d0a65715f0ab62519894b19c0c74750d3dcdc928626fa68b146038b907d79de3ba9712c9287db8fa64693ebc11
2023-08-25 13:58:06 +00:00
Andrew Poelstra 082bd03120
Merge rust-bitcoin/rust-bitcoin#2019: Rename xpub and xpriv types
be05f9d852 Rename xpub and xpriv types (Tobin C. Harding)

Pull request description:

  The BIP-32 extended public key and extended private key exist in the Bitcoin vernacular as xpub and xpriv. We can use these terms with no loss of clarity.

  Rename our current BIP-32 types

  - `ExtendedPubKey` to `Xpub`
  - `ExtendedPrivKey` to `Xpriv`

  This patch is a mechanical search-and-replace, followed by running the formatter, no other manual changes.

ACKs for top commit:
  apoelstra:
    ACK be05f9d852
  sanket1729:
    ACK be05f9d852

Tree-SHA512: 49925688783c3f37a9b92a9767a0df095323a3fa51f3d672a0b5dd1d8bca86f7facbcc33921274bc147b369de09042c4850b08c31e63f71110903435daa6c00c
2023-08-25 13:41:50 +00:00
Andrew Poelstra bd05e3f309
Merge rust-bitcoin/rust-bitcoin#2029: Fix clippy warnings
724be17394 Remove useless usage of vec! macro (Tobin C. Harding)
e84ca292d9 Clear incorrect implementation of clone warning (Tobin C. Harding)

Pull request description:

  A new version `clippy 0.1.72 (5680fa1 2023-08-23)` just came out and we get a two new warnings. Fix them up.

ACKs for top commit:
  apoelstra:
    ACK 724be17394

Tree-SHA512: f82f026773f8738d8d89710b36b979850e0e33c4d1afb4f78d2d4e957b37dd850ef9e83c9e25197dac981c7b49c448e49078777261749567303f1aac282b3d33
2023-08-25 13:27:12 +00:00
Andrew Poelstra 9dbefc6f85
Merge rust-bitcoin/rust-bitcoin#1988: Export all hash types
7bbdd9b2af Export all hash types (Tobin C. Harding)

Pull request description:

  During the 0.30.0 release we removed the re-exports of hash types. This upset some folk and since the aim of our re-exports is not exactly clean as well as the fact that the public API surface is not yet fixed just re-export all the hash types at the crate root again.

  This is #1792 but does not use a wildcard and also grabs the other hashes that we recently moved.

ACKs for top commit:
  apoelstra:
    ACK 7bbdd9b2af
  sanket1729:
    ACK 7bbdd9b2af

Tree-SHA512: addfb617fae2fce12eb9d198ffeb1b0c99792dfd757222c21c62bd4b408432de5dc93c42da7d886b43c29b2c34bd318573e813f567a29080fc264ee7beba0f70
2023-08-25 13:02:19 +00:00
yancy 55e94b5dea Remove test from test names for Weight type 2023-08-25 10:28:34 +02:00
yancy 142dde64c3 Use Weight type for stripped_size 2023-08-25 10:28:34 +02:00
yancy cb76f3ec43 Add scale_by_witness_factor to Weight type 2023-08-25 10:28:34 +02:00
yancy 38c9e9947e Add witness scale factor to the Weight type 2023-08-25 10:28:05 +02:00
yancy 77552987ab Add from_wu_usize to Weight type 2023-08-25 10:03:59 +02:00
yancy 1a88c887f5 Rename strippedsize to stripped_size 2023-08-25 10:03:39 +02:00
Tobin C. Harding 724be17394
Remove useless usage of vec! macro
Clippy emits a bunch of warnings of form:

  warning: useless use of `vec!`

As suggested, remove the vec and just use an array.
2023-08-25 12:30:04 +10:00
Tobin C. Harding e84ca292d9
Clear incorrect implementation of clone warning
Clippy emits:

  error: incorrect implementation of `clone` on a `Copy` type

As suggested use `*self` instead of each individual field.
2023-08-25 12:08:56 +10:00
Andrew Poelstra cdbd4be4b1
Merge rust-bitcoin/rust-bitcoin#1917: hashes: Release `v0.13.0`
53f68383b7 hashes: Bump version to 0.13.0 (Tobin C. Harding)

Pull request description:

  In preparation for `hashes` release, bump the version. Depend on new version in `rust-bitcoin`.

  Note the `bitcoin-private` addition to the lock files is because we temporarily have two `bitcoin_hashes` dependencies and the secp one (v.0.12.0) depends on `bitcoin-private`.

ACKs for top commit:
  sanket1729:
    utACK 53f68383b7 . I am not sure about the exact nature of release dance between various crates in rust-bitcoin. This code and changelog entries looks good.
  apoelstra:
    ACK 53f68383b7

Tree-SHA512: a1933bcda1fa9a06c96a4c7079ff49f531e4f366373e98700446cecdf1eed5a104d4d4aa737202ec426cb1f1edf88eff5eee80df9f0cc3a9779c051a55f847b5
2023-08-24 17:22:10 +00:00
Andrew Poelstra da1836edf5
Merge rust-bitcoin/rust-bitcoin#2012: Add a script for updating lock files
8b84227aec Add a script for updating lock files (Tobin C. Harding)

Pull request description:

  Every time one of the dependencies is explicitly changed we have to update the minimal/recent lock files. Add a script to do so.

ACKs for top commit:
  apoelstra:
    ACK 8b84227aec
  sanket1729:
    ACK 8b84227aec

Tree-SHA512: f165308131077c55712d9ae5b85a4dc5f0f24c660dd4c196c48a53f64411b25714b2f32d3a6538f040b3d8ef5240df7d16b7b66561e0d8bb49245b44eb8f522c
2023-08-24 16:50:46 +00:00
Andrew Poelstra 407dec0bce
Merge rust-bitcoin/rust-bitcoin#2024: Add VarInt from implementations by way of macro
0419fa278b Add VarInt from implementations by way of macro (Tobin C. Harding)

Pull request description:

  Throughout the codebase we cast values to `u64` when constructing a `VarInt`. We can make the code marginally cleaner by adding `From<T>` impls for all unsigned integer types less than or equal to 64 bits. Also allows us to (possibly unnecessarily) comment the cast in a single place.

ACKs for top commit:
  sanket1729:
    utACK 0419fa278b
  apoelstra:
    ACK 0419fa278b

Tree-SHA512: 0cbcc7e9ec6a1a102693cb13685c348672fb13b098cbecd0a36bed0331165adb008f149f87f7b0c64f131974cfe513adbc12f508bc4853906adb2a65c0c647ee
2023-08-24 15:50:29 +00:00
Tobin C. Harding 53f68383b7
hashes: Bump version to 0.13.0
Add a changelog entry and bump the version to 0.13.0

Does not include changes to `bitcoin` to depend on the new version.
2023-08-24 12:25:42 +10:00
Tobin C. Harding 66d5800ac0
psbt: Add IndexOutOfBounds error
We currently have a bunch of functions that are infallible if the
`index` argument is within-bounds however we return a `SignError`, this
obfuscates the code.

Add an `IndexOutOfBoundsErorr`. While we are at it make it an enum so
users can differentiate between which vector the out of bounds access
was attempted against.
2023-08-24 12:12:04 +10:00
Tobin C. Harding 8b84227aec
Add a script for updating lock files
Every time one of the dependencies is explicitly changed we have to
update the minimal/recent lock files. Add a script to do so.
2023-08-24 11:03:06 +10:00
Tobin C. Harding 0419fa278b
Add VarInt from implementations by way of macro
Throughout the codebase we cast values to `u64` when constructing a
`VarInt`. We can make the code marginally cleaner by adding `From<T>`
impls for all unsigned integer types less than or equal to 64 bits.
Also allows us to (possibly unnecessarily) comment the cast in a single
place.
2023-08-24 10:37:53 +10:00
Andrew Poelstra 1991b7af40
Merge rust-bitcoin/rust-bitcoin#1998: Remove usage of ThirtyTwoByteHash
d9533523ac Remove usage of ThirtyTwoByteHash (Tobin C. Harding)

Pull request description:

  The `ThirtyTwoByteHash` trait is defined in `secp256k1` and used in `hashes` as well as `bitcoin`. This means that we must use the same version of `hashes` in both `bitcoin` and `secp256k1`. This makes doing release difficult.

  Remove usage of `ThirtyTwoByteHash` and use `Message::from_slice`. Include TODO above each usage because as soon as we release the new version of secp we can use the new `Message::from_digest`.

  This is step backwards as far as type safety goes and it makes the code more ugly as well because it uses `expect` but thems the breaks.

  For context see #1985

ACKs for top commit:
  sanket1729:
    utACK d9533523ac
  apoelstra:
    ACK d9533523ac

Tree-SHA512: 0dc6f7895ba6e1d2de978d45152e6e12b9f81b3fbe9f3ba89c090005b6c8d2e1221e0a04a3ac38c7e7669f6ce62edaa21739ae58cc1d2cad63f608a36231718e
2023-08-23 14:07:11 +00:00
Tobin C. Harding d9533523ac
Remove usage of ThirtyTwoByteHash
The `ThirtyTwoByteHash` trait is defined in `secp256k1` and used in
`hashes` as well as `bitcoin`. This means that we must use the same
version of `hashes` in both `bitcoin` and `secp256k1`. This makes doing
release difficult.

Remove usage of `ThirtyTwoByteHash` and use `Message::from_slice`.
Include TODO above each usage because as soon as we release the new
version of secp we can use the new `Message::from_digest`.

This is step backwards as far as type safety goes and it makes the code
more ugly as well because it uses `expect` but thems the breaks.
2023-08-23 12:21:26 +10:00
Tobin C. Harding f18f684ad2
Add version bytes consts
BIP-32 defines 4 4-byte consts used as version bytes; currently we are
hardcoding the version bytes in multiple places.

Add BIP-32 version bytes consts and use them throughout the module.
2023-08-22 13:49:46 +10:00
Tobin C. Harding be05f9d852
Rename xpub and xpriv types
The BIP-32 extended public key and extended private key exist in the
Bitcoin vernacular as xpub and xpriv. We can use these terms with no
loss of clarity.

Rename our current BIP-32 types

- `ExtendedPubKey` to `Xpub`
- `ExtendedPrivKey` to `Xpriv`

This patch is a mechanical search-and-replace, followed by running the
formatter, no other manual changes.
2023-08-22 13:47:35 +10:00
Andrew Poelstra 5bf2117dc4
Merge rust-bitcoin/rust-bitcoin#1995: Improve segwit signature hash API
4300cf2210 Add p2wpkh and p2wsh signature hash functions (Tobin C. Harding)

Pull request description:

  The word "segwit" refers to segwit v0 and taproot but these functions are version specific. Add `v0` into the function names.

  This is similar to #1994, both based on recent post of mine to bitcoin dev mailing list.

ACKs for top commit:
  stevenroose:
    ACK 4300cf2210
  apoelstra:
    ACK 4300cf2210

Tree-SHA512: 723fc302954514da0fa57a3890b9f62e9d8d1b25289b8db00611d8bc34c5000b9e54943f57b8e94befcaf72633ac078b2ff66a1da0c5bb483cfaa584e3cb6014
2023-08-21 14:53:04 +00:00
yancy 3369257c75 Fix grammar 2023-08-21 15:51:58 +02:00
Andrew Poelstra 672656515e
Merge rust-bitcoin/rust-bitcoin#2001: Add txin base weight
7ec33d29eb refactor: developer doc first (yancy)
5496feb5c1 Add base weight const to TxIn (yancy)

Pull request description:

  Add a base weight const to TxIn.  I also used this const in strippedsize() and scaledsize().  As a different PR, I think strippedsize and scaledsize could return Weight instead of usize.  Also added a small commit to re-arrange commit messages.

ACKs for top commit:
  apoelstra:
    ACK 7ec33d29eb
  tcharding:
    ACK 7ec33d29eb

Tree-SHA512: b20f95605ed664b88df0a5a178d48f15f27d90eb404c9707aef010c4504d7ffd4a3565c217710b9289f87ed2a0724fd8f7cc78a79a58547fe3ee87339c0d74c1
2023-08-18 14:28:38 +00:00
Andrew Poelstra 1b3a9d3580
Merge rust-bitcoin/rust-bitcoin#1990: Introduce the `small-hash` feature for `bitcoin_hashes`
f2c5f19557 Introduce the `small-hash` feature for `bitcoin_hashes` (Alekos Filini)

Pull request description:

  When enabled this feature swaps the hash implementation of sha512, sha256 and ripemd160 for a smaller (but also slower) one.

  On embedded processors (Cortex-M4) it can lead to up to a 52% size reduction, from around 37KiB for just the `process_block` methods of the three hash functions to 17.8KiB.

  The following numbers were collected on `aarch64-unknown-linux-gnu` with `cargo 1.72.0-nightly`.

  ## Original

  ```
  RUSTFLAGS='--cfg=bench -C opt-level=z' cargo bench
  ```

  ```
  test hash160::benches::hash160_10                 ... bench:          33 ns/iter (+/- 1) = 303 MB/s
  test hash160::benches::hash160_1k                 ... bench:       2,953 ns/iter (+/- 187) = 346 MB/s
  test hash160::benches::hash160_64k                ... bench:     188,480 ns/iter (+/- 11,595) = 347 MB/s
  test hmac::benches::hmac_sha256_10                ... bench:          33 ns/iter (+/- 2) = 303 MB/s
  test hmac::benches::hmac_sha256_1k                ... bench:       2,957 ns/iter (+/- 104) = 346 MB/s
  test hmac::benches::hmac_sha256_64k               ... bench:     192,022 ns/iter (+/- 6,407) = 341 MB/s
  test ripemd160::benches::ripemd160_10             ... bench:          25 ns/iter (+/- 1) = 400 MB/s
  test ripemd160::benches::ripemd160_1k             ... bench:       2,288 ns/iter (+/- 93) = 447 MB/s
  test ripemd160::benches::ripemd160_64k            ... bench:     146,823 ns/iter (+/- 1,102) = 446 MB/s
  test sha1::benches::sha1_10                       ... bench:          41 ns/iter (+/- 0) = 243 MB/s
  test sha1::benches::sha1_1k                       ... bench:       3,844 ns/iter (+/- 70) = 266 MB/s
  test sha1::benches::sha1_64k                      ... bench:     245,854 ns/iter (+/- 10,158) = 266 MB/s
  test sha256::benches::sha256_10                   ... bench:          35 ns/iter (+/- 0) = 285 MB/s
  test sha256::benches::sha256_1k                   ... bench:       3,063 ns/iter (+/- 15) = 334 MB/s
  test sha256::benches::sha256_64k                  ... bench:     195,729 ns/iter (+/- 2,880) = 334 MB/s
  test sha256d::benches::sha256d_10                 ... bench:          34 ns/iter (+/- 1) = 294 MB/s
  test sha256d::benches::sha256d_1k                 ... bench:       3,071 ns/iter (+/- 107) = 333 MB/s
  test sha256d::benches::sha256d_64k                ... bench:     188,614 ns/iter (+/- 8,101) = 347 MB/s
  test sha512::benches::sha512_10                   ... bench:          21 ns/iter (+/- 0) = 476 MB/s
  test sha512::benches::sha512_1k                   ... bench:       1,714 ns/iter (+/- 36) = 597 MB/s
  test sha512::benches::sha512_64k                  ... bench:     110,084 ns/iter (+/- 3,637) = 595 MB/s
  test sha512_256::benches::sha512_256_10           ... bench:          22 ns/iter (+/- 1) = 454 MB/s
  test sha512_256::benches::sha512_256_1k           ... bench:       1,822 ns/iter (+/- 70) = 562 MB/s
  test sha512_256::benches::sha512_256_64k          ... bench:     116,231 ns/iter (+/- 4,745) = 563 MB/s
  test siphash24::benches::siphash24_1ki            ... bench:       1,072 ns/iter (+/- 41) = 955 MB/s
  test siphash24::benches::siphash24_1ki_hash       ... bench:       1,102 ns/iter (+/- 42) = 929 MB/s
  test siphash24::benches::siphash24_1ki_hash_u64   ... bench:       1,064 ns/iter (+/- 41) = 962 MB/s
  test siphash24::benches::siphash24_64ki           ... bench:      69,957 ns/iter (+/- 2,712) = 936 MB/
  ```

  ```
  0000000000005872 t _ZN84_$LT$bitcoin_hashes..ripemd160..HashEngine$u20$as$u20$bitcoin_hashes..HashEngine$GT$5input17hc4800746a9da7ff4E
  0000000000007956 t _ZN81_$LT$bitcoin_hashes..sha256..HashEngine$u20$as$u20$bitcoin_hashes..HashEngine$GT$5input17hf49345f65130ce9bE
  0000000000008024 t _ZN14bitcoin_hashes6sha2568Midstate10const_hash17h57317bc8012004b4E.llvm.441255102889972912
  0000000000010528 t _ZN81_$LT$bitcoin_hashes..sha512..HashEngine$u20$as$u20$bitcoin_hashes..HashEngine$GT$5input17h9bc868d4392bd9acE
  ```

  Total size: 32380 bytes

  ## With `small-hash` enabled

  ```
  RUSTFLAGS='--cfg=bench -C opt-level=z' cargo bench --features small-hash
  ```

  ```
  test hash160::benches::hash160_10                 ... bench:          52 ns/iter (+/- 3) = 192 MB/s
  test hash160::benches::hash160_1k                 ... bench:       4,817 ns/iter (+/- 286) = 212 MB/s
  test hash160::benches::hash160_64k                ... bench:     319,572 ns/iter (+/- 11,031) = 205 MB/s
  test hmac::benches::hmac_sha256_10                ... bench:          54 ns/iter (+/- 2) = 185 MB/s
  test hmac::benches::hmac_sha256_1k                ... bench:       4,846 ns/iter (+/- 204) = 211 MB/s
  test hmac::benches::hmac_sha256_64k               ... bench:     319,114 ns/iter (+/- 4,451) = 205 MB/s
  test ripemd160::benches::ripemd160_10             ... bench:          27 ns/iter (+/- 0) = 370 MB/s
  test ripemd160::benches::ripemd160_1k             ... bench:       2,358 ns/iter (+/- 150) = 434 MB/s
  test ripemd160::benches::ripemd160_64k            ... bench:     154,573 ns/iter (+/- 3,954) = 423 MB/s
  test sha1::benches::sha1_10                       ... bench:          41 ns/iter (+/- 1) = 243 MB/s
  test sha1::benches::sha1_1k                       ... bench:       3,700 ns/iter (+/- 243) = 276 MB/s
  test sha1::benches::sha1_64k                      ... bench:     231,039 ns/iter (+/- 13,989) = 283 MB/s
  test sha256::benches::sha256_10                   ... bench:          51 ns/iter (+/- 3) = 196 MB/s
  test sha256::benches::sha256_1k                   ... bench:       4,823 ns/iter (+/- 182) = 212 MB/s
  test sha256::benches::sha256_64k                  ... bench:     299,960 ns/iter (+/- 17,545) = 218 MB/s
  test sha256d::benches::sha256d_10                 ... bench:          52 ns/iter (+/- 2) = 192 MB/s
  test sha256d::benches::sha256d_1k                 ... bench:       4,827 ns/iter (+/- 323) = 212 MB/s
  test sha256d::benches::sha256d_64k                ... bench:     302,844 ns/iter (+/- 15,796) = 216 MB/s
  test sha512::benches::sha512_10                   ... bench:          34 ns/iter (+/- 1) = 294 MB/s
  test sha512::benches::sha512_1k                   ... bench:       3,002 ns/iter (+/- 123) = 341 MB/s
  test sha512::benches::sha512_64k                  ... bench:     189,767 ns/iter (+/- 10,396) = 345 MB/s
  test sha512_256::benches::sha512_256_10           ... bench:          34 ns/iter (+/- 1) = 294 MB/s
  test sha512_256::benches::sha512_256_1k           ... bench:       2,996 ns/iter (+/- 198) = 341 MB/s
  test sha512_256::benches::sha512_256_64k          ... bench:     192,024 ns/iter (+/- 8,181) = 341 MB/s
  test siphash24::benches::siphash24_1ki            ... bench:       1,081 ns/iter (+/- 65) = 947 MB/s
  test siphash24::benches::siphash24_1ki_hash       ... bench:       1,083 ns/iter (+/- 63) = 945 MB/s
  test siphash24::benches::siphash24_1ki_hash_u64   ... bench:       1,084 ns/iter (+/- 63) = 944 MB/s
  test siphash24::benches::siphash24_64ki           ... bench:      67,237 ns/iter (+/- 4,185) = 974 MB/s
  ```

  ```
  0000000000005384 t _ZN81_$LT$bitcoin_hashes..sha256..HashEngine$u20$as$u20$bitcoin_hashes..HashEngine$GT$5input17hae341658cf9b880bE
  0000000000005608 t _ZN14bitcoin_hashes9ripemd16010HashEngine13process_block17h3276b13f1e9feef8E.llvm.13618235596061801146
  0000000000005616 t _ZN14bitcoin_hashes6sha2568Midstate10const_hash17h3e6fbef64c15ee00E.llvm.7326223909590351031
  0000000000005944 t _ZN81_$LT$bitcoin_hashes..sha512..HashEngine$u20$as$u20$bitcoin_hashes..HashEngine$GT$5input17h321a237bfbe5c0bbE
  ```

  Total size: 22552 bytes

  ## Conclusion

  On `aarch64` there's overall a ~30% improvement in size, although ripemd160 doesn't really shrink that much (and its performance also aren't impacted much with only a 6% slowdown). sha512 and sha256 instead are almost 40% slower with `small-hash` enabled.

  I don't have performance numbers for other architectures, but in terms of size there was an even larger improvements on `thumbv7em-none-eabihf`, with a 52% size reduction overall:

  ```
     Size          Crate Name
  25.3KiB bitcoin_hashes <bitcoin_hashes[fe467ef2aa3a1470]::sha512::HashEngine as bitcoin_hashes[fe467ef2aa3a1470]::HashEngine>::input
   6.9KiB bitcoin_hashes <bitcoin_hashes[fe467ef2aa3a1470]::sha256::HashEngine as bitcoin_hashes[fe467ef2aa3a1470]::HashEngine>::input
   4.8KiB bitcoin_hashes <bitcoin_hashes[fe467ef2aa3a1470]::ripemd160::HashEngine as bitcoin_hashes[fe467ef2aa3a1470]::HashEngine>::input
  ```

  vs

  ```
    Size          Crate Name
  9.5KiB bitcoin_hashes <bitcoin_hashes[974bb476ef905797]::sha512::HashEngine as bitcoin_hashes[974bb476ef905797]::HashEngine>::input
  4.5KiB bitcoin_hashes <bitcoin_hashes[974bb476ef905797]::ripemd160::HashEngine>::process_block
  3.8KiB bitcoin_hashes <bitcoin_hashes[974bb476ef905797]::sha256::HashEngine as bitcoin_hashes[974bb476ef905797]::HashEngine>::input
  ```

  I'm assuming this is because on more limited architectures the compiler needs to use more instructions to move data in and out of registers (especially for sha512 which ideally would benefit from 64-bit registers), so reusing the code by moving it into functions saves a lot of those instructions.

  Also note that the `const_hash` method on `sha256` causes the compiler to emit two independent implementations. I haven't looked into the code yet, maybe there's a way to merge them so that the non-const `process_block` calls into the const fn.

  -----

  Note: commits are unverified right now because I don't have the keys available, I will sign them after addressing the review comments.

ACKs for top commit:
  apoelstra:
    ACK f2c5f19557
  tcharding:
    ACK f2c5f19557

Tree-SHA512: 1d5eb56324c458660e2571e8cf59895dc31dae9c5427c7ed36f8a0e81ca2e9a0f39026f56b6803df03635cc8b66aee3bf5182d51ab8972d169d56bcfec33771c
2023-08-17 16:34:10 +00:00
Andrew Poelstra af85528428
Merge rust-bitcoin/rust-bitcoin#1962: Add simd sha256 intrinsics for x86 machines
546c0122d7 Add simd sha256 intrinsics for x86 machines (sanket1729)

Pull request description:

  This is my first time dabbling into architecture specific code and simd. The algorithm is a word to word translation of the C code from  4899efc81d/sha256-x86.c .

  Some benchmarks:

  With simd
  ```
  test sha256::benches::sha256_10                   ... bench:          11 ns/iter (+/- 0) = 909 MB/s
  test sha256::benches::sha256_1k                   ... bench:         712 ns/iter (+/- 2) = 1438 MB/s
  test sha256::benches::sha256_64k                  ... bench:      45,597 ns/iter (+/- 189) = 1437 MB/s
  ```
  Without simd
  ```
  test sha256::benches::sha256_10                   ... bench:          47 ns/iter (+/- 0) = 212 MB/s
  test sha256::benches::sha256_1k                   ... bench:       4,243 ns/iter (+/- 17) = 241 MB/s
  test sha256::benches::sha256_64k                  ... bench:     271,263 ns/iter (+/- 1,610) = 241 MB/s
  ```

ACKs for top commit:
  apoelstra:
    ACK 546c0122d7
  tcharding:
    ACK 546c0122d7

Tree-SHA512: 7167c900b77e63cf38135a3960cf9ac2615f73b2ef7020a12b5cc3f4c047910063ba9045217b9ecfa70f7de1eb0f02f2674f291bd023a853bad2b9162fae831e
2023-08-17 16:22:46 +00:00