53f68383b7 hashes: Bump version to 0.13.0 (Tobin C. Harding)
Pull request description:
In preparation for `hashes` release, bump the version. Depend on new version in `rust-bitcoin`.
Note the `bitcoin-private` addition to the lock files is because we temporarily have two `bitcoin_hashes` dependencies and the secp one (v.0.12.0) depends on `bitcoin-private`.
ACKs for top commit:
sanket1729:
utACK 53f68383b7 . I am not sure about the exact nature of release dance between various crates in rust-bitcoin. This code and changelog entries looks good.
apoelstra:
ACK 53f68383b7
Tree-SHA512: a1933bcda1fa9a06c96a4c7079ff49f531e4f366373e98700446cecdf1eed5a104d4d4aa737202ec426cb1f1edf88eff5eee80df9f0cc3a9779c051a55f847b5
8b84227aec Add a script for updating lock files (Tobin C. Harding)
Pull request description:
Every time one of the dependencies is explicitly changed we have to update the minimal/recent lock files. Add a script to do so.
ACKs for top commit:
apoelstra:
ACK 8b84227aec
sanket1729:
ACK 8b84227aec
Tree-SHA512: f165308131077c55712d9ae5b85a4dc5f0f24c660dd4c196c48a53f64411b25714b2f32d3a6538f040b3d8ef5240df7d16b7b66561e0d8bb49245b44eb8f522c
0419fa278b Add VarInt from implementations by way of macro (Tobin C. Harding)
Pull request description:
Throughout the codebase we cast values to `u64` when constructing a `VarInt`. We can make the code marginally cleaner by adding `From<T>` impls for all unsigned integer types less than or equal to 64 bits. Also allows us to (possibly unnecessarily) comment the cast in a single place.
ACKs for top commit:
sanket1729:
utACK 0419fa278b
apoelstra:
ACK 0419fa278b
Tree-SHA512: 0cbcc7e9ec6a1a102693cb13685c348672fb13b098cbecd0a36bed0331165adb008f149f87f7b0c64f131974cfe513adbc12f508bc4853906adb2a65c0c647ee
We currently have a bunch of functions that are infallible if the
`index` argument is within-bounds however we return a `SignError`, this
obfuscates the code.
Add an `IndexOutOfBoundsErorr`. While we are at it make it an enum so
users can differentiate between which vector the out of bounds access
was attempted against.
Throughout the codebase we cast values to `u64` when constructing a
`VarInt`. We can make the code marginally cleaner by adding `From<T>`
impls for all unsigned integer types less than or equal to 64 bits.
Also allows us to (possibly unnecessarily) comment the cast in a single
place.
d9533523ac Remove usage of ThirtyTwoByteHash (Tobin C. Harding)
Pull request description:
The `ThirtyTwoByteHash` trait is defined in `secp256k1` and used in `hashes` as well as `bitcoin`. This means that we must use the same version of `hashes` in both `bitcoin` and `secp256k1`. This makes doing release difficult.
Remove usage of `ThirtyTwoByteHash` and use `Message::from_slice`. Include TODO above each usage because as soon as we release the new version of secp we can use the new `Message::from_digest`.
This is step backwards as far as type safety goes and it makes the code more ugly as well because it uses `expect` but thems the breaks.
For context see #1985
ACKs for top commit:
sanket1729:
utACK d9533523ac
apoelstra:
ACK d9533523ac
Tree-SHA512: 0dc6f7895ba6e1d2de978d45152e6e12b9f81b3fbe9f3ba89c090005b6c8d2e1221e0a04a3ac38c7e7669f6ce62edaa21739ae58cc1d2cad63f608a36231718e
The `ThirtyTwoByteHash` trait is defined in `secp256k1` and used in
`hashes` as well as `bitcoin`. This means that we must use the same
version of `hashes` in both `bitcoin` and `secp256k1`. This makes doing
release difficult.
Remove usage of `ThirtyTwoByteHash` and use `Message::from_slice`.
Include TODO above each usage because as soon as we release the new
version of secp we can use the new `Message::from_digest`.
This is step backwards as far as type safety goes and it makes the code
more ugly as well because it uses `expect` but thems the breaks.
BIP-32 defines 4 4-byte consts used as version bytes; currently we are
hardcoding the version bytes in multiple places.
Add BIP-32 version bytes consts and use them throughout the module.
The BIP-32 extended public key and extended private key exist in the
Bitcoin vernacular as xpub and xpriv. We can use these terms with no
loss of clarity.
Rename our current BIP-32 types
- `ExtendedPubKey` to `Xpub`
- `ExtendedPrivKey` to `Xpriv`
This patch is a mechanical search-and-replace, followed by running the
formatter, no other manual changes.
4300cf2210 Add p2wpkh and p2wsh signature hash functions (Tobin C. Harding)
Pull request description:
The word "segwit" refers to segwit v0 and taproot but these functions are version specific. Add `v0` into the function names.
This is similar to #1994, both based on recent post of mine to bitcoin dev mailing list.
ACKs for top commit:
stevenroose:
ACK 4300cf2210
apoelstra:
ACK 4300cf2210
Tree-SHA512: 723fc302954514da0fa57a3890b9f62e9d8d1b25289b8db00611d8bc34c5000b9e54943f57b8e94befcaf72633ac078b2ff66a1da0c5bb483cfaa584e3cb6014
7ec33d29eb refactor: developer doc first (yancy)
5496feb5c1 Add base weight const to TxIn (yancy)
Pull request description:
Add a base weight const to TxIn. I also used this const in strippedsize() and scaledsize(). As a different PR, I think strippedsize and scaledsize could return Weight instead of usize. Also added a small commit to re-arrange commit messages.
ACKs for top commit:
apoelstra:
ACK 7ec33d29eb
tcharding:
ACK 7ec33d29eb
Tree-SHA512: b20f95605ed664b88df0a5a178d48f15f27d90eb404c9707aef010c4504d7ffd4a3565c217710b9289f87ed2a0724fd8f7cc78a79a58547fe3ee87339c0d74c1
f2c5f19557 Introduce the `small-hash` feature for `bitcoin_hashes` (Alekos Filini)
Pull request description:
When enabled this feature swaps the hash implementation of sha512, sha256 and ripemd160 for a smaller (but also slower) one.
On embedded processors (Cortex-M4) it can lead to up to a 52% size reduction, from around 37KiB for just the `process_block` methods of the three hash functions to 17.8KiB.
The following numbers were collected on `aarch64-unknown-linux-gnu` with `cargo 1.72.0-nightly`.
## Original
```
RUSTFLAGS='--cfg=bench -C opt-level=z' cargo bench
```
```
test hash160::benches::hash160_10 ... bench: 33 ns/iter (+/- 1) = 303 MB/s
test hash160::benches::hash160_1k ... bench: 2,953 ns/iter (+/- 187) = 346 MB/s
test hash160::benches::hash160_64k ... bench: 188,480 ns/iter (+/- 11,595) = 347 MB/s
test hmac::benches::hmac_sha256_10 ... bench: 33 ns/iter (+/- 2) = 303 MB/s
test hmac::benches::hmac_sha256_1k ... bench: 2,957 ns/iter (+/- 104) = 346 MB/s
test hmac::benches::hmac_sha256_64k ... bench: 192,022 ns/iter (+/- 6,407) = 341 MB/s
test ripemd160::benches::ripemd160_10 ... bench: 25 ns/iter (+/- 1) = 400 MB/s
test ripemd160::benches::ripemd160_1k ... bench: 2,288 ns/iter (+/- 93) = 447 MB/s
test ripemd160::benches::ripemd160_64k ... bench: 146,823 ns/iter (+/- 1,102) = 446 MB/s
test sha1::benches::sha1_10 ... bench: 41 ns/iter (+/- 0) = 243 MB/s
test sha1::benches::sha1_1k ... bench: 3,844 ns/iter (+/- 70) = 266 MB/s
test sha1::benches::sha1_64k ... bench: 245,854 ns/iter (+/- 10,158) = 266 MB/s
test sha256::benches::sha256_10 ... bench: 35 ns/iter (+/- 0) = 285 MB/s
test sha256::benches::sha256_1k ... bench: 3,063 ns/iter (+/- 15) = 334 MB/s
test sha256::benches::sha256_64k ... bench: 195,729 ns/iter (+/- 2,880) = 334 MB/s
test sha256d::benches::sha256d_10 ... bench: 34 ns/iter (+/- 1) = 294 MB/s
test sha256d::benches::sha256d_1k ... bench: 3,071 ns/iter (+/- 107) = 333 MB/s
test sha256d::benches::sha256d_64k ... bench: 188,614 ns/iter (+/- 8,101) = 347 MB/s
test sha512::benches::sha512_10 ... bench: 21 ns/iter (+/- 0) = 476 MB/s
test sha512::benches::sha512_1k ... bench: 1,714 ns/iter (+/- 36) = 597 MB/s
test sha512::benches::sha512_64k ... bench: 110,084 ns/iter (+/- 3,637) = 595 MB/s
test sha512_256::benches::sha512_256_10 ... bench: 22 ns/iter (+/- 1) = 454 MB/s
test sha512_256::benches::sha512_256_1k ... bench: 1,822 ns/iter (+/- 70) = 562 MB/s
test sha512_256::benches::sha512_256_64k ... bench: 116,231 ns/iter (+/- 4,745) = 563 MB/s
test siphash24::benches::siphash24_1ki ... bench: 1,072 ns/iter (+/- 41) = 955 MB/s
test siphash24::benches::siphash24_1ki_hash ... bench: 1,102 ns/iter (+/- 42) = 929 MB/s
test siphash24::benches::siphash24_1ki_hash_u64 ... bench: 1,064 ns/iter (+/- 41) = 962 MB/s
test siphash24::benches::siphash24_64ki ... bench: 69,957 ns/iter (+/- 2,712) = 936 MB/
```
```
0000000000005872 t _ZN84_$LT$bitcoin_hashes..ripemd160..HashEngine$u20$as$u20$bitcoin_hashes..HashEngine$GT$5input17hc4800746a9da7ff4E
0000000000007956 t _ZN81_$LT$bitcoin_hashes..sha256..HashEngine$u20$as$u20$bitcoin_hashes..HashEngine$GT$5input17hf49345f65130ce9bE
0000000000008024 t _ZN14bitcoin_hashes6sha2568Midstate10const_hash17h57317bc8012004b4E.llvm.441255102889972912
0000000000010528 t _ZN81_$LT$bitcoin_hashes..sha512..HashEngine$u20$as$u20$bitcoin_hashes..HashEngine$GT$5input17h9bc868d4392bd9acE
```
Total size: 32380 bytes
## With `small-hash` enabled
```
RUSTFLAGS='--cfg=bench -C opt-level=z' cargo bench --features small-hash
```
```
test hash160::benches::hash160_10 ... bench: 52 ns/iter (+/- 3) = 192 MB/s
test hash160::benches::hash160_1k ... bench: 4,817 ns/iter (+/- 286) = 212 MB/s
test hash160::benches::hash160_64k ... bench: 319,572 ns/iter (+/- 11,031) = 205 MB/s
test hmac::benches::hmac_sha256_10 ... bench: 54 ns/iter (+/- 2) = 185 MB/s
test hmac::benches::hmac_sha256_1k ... bench: 4,846 ns/iter (+/- 204) = 211 MB/s
test hmac::benches::hmac_sha256_64k ... bench: 319,114 ns/iter (+/- 4,451) = 205 MB/s
test ripemd160::benches::ripemd160_10 ... bench: 27 ns/iter (+/- 0) = 370 MB/s
test ripemd160::benches::ripemd160_1k ... bench: 2,358 ns/iter (+/- 150) = 434 MB/s
test ripemd160::benches::ripemd160_64k ... bench: 154,573 ns/iter (+/- 3,954) = 423 MB/s
test sha1::benches::sha1_10 ... bench: 41 ns/iter (+/- 1) = 243 MB/s
test sha1::benches::sha1_1k ... bench: 3,700 ns/iter (+/- 243) = 276 MB/s
test sha1::benches::sha1_64k ... bench: 231,039 ns/iter (+/- 13,989) = 283 MB/s
test sha256::benches::sha256_10 ... bench: 51 ns/iter (+/- 3) = 196 MB/s
test sha256::benches::sha256_1k ... bench: 4,823 ns/iter (+/- 182) = 212 MB/s
test sha256::benches::sha256_64k ... bench: 299,960 ns/iter (+/- 17,545) = 218 MB/s
test sha256d::benches::sha256d_10 ... bench: 52 ns/iter (+/- 2) = 192 MB/s
test sha256d::benches::sha256d_1k ... bench: 4,827 ns/iter (+/- 323) = 212 MB/s
test sha256d::benches::sha256d_64k ... bench: 302,844 ns/iter (+/- 15,796) = 216 MB/s
test sha512::benches::sha512_10 ... bench: 34 ns/iter (+/- 1) = 294 MB/s
test sha512::benches::sha512_1k ... bench: 3,002 ns/iter (+/- 123) = 341 MB/s
test sha512::benches::sha512_64k ... bench: 189,767 ns/iter (+/- 10,396) = 345 MB/s
test sha512_256::benches::sha512_256_10 ... bench: 34 ns/iter (+/- 1) = 294 MB/s
test sha512_256::benches::sha512_256_1k ... bench: 2,996 ns/iter (+/- 198) = 341 MB/s
test sha512_256::benches::sha512_256_64k ... bench: 192,024 ns/iter (+/- 8,181) = 341 MB/s
test siphash24::benches::siphash24_1ki ... bench: 1,081 ns/iter (+/- 65) = 947 MB/s
test siphash24::benches::siphash24_1ki_hash ... bench: 1,083 ns/iter (+/- 63) = 945 MB/s
test siphash24::benches::siphash24_1ki_hash_u64 ... bench: 1,084 ns/iter (+/- 63) = 944 MB/s
test siphash24::benches::siphash24_64ki ... bench: 67,237 ns/iter (+/- 4,185) = 974 MB/s
```
```
0000000000005384 t _ZN81_$LT$bitcoin_hashes..sha256..HashEngine$u20$as$u20$bitcoin_hashes..HashEngine$GT$5input17hae341658cf9b880bE
0000000000005608 t _ZN14bitcoin_hashes9ripemd16010HashEngine13process_block17h3276b13f1e9feef8E.llvm.13618235596061801146
0000000000005616 t _ZN14bitcoin_hashes6sha2568Midstate10const_hash17h3e6fbef64c15ee00E.llvm.7326223909590351031
0000000000005944 t _ZN81_$LT$bitcoin_hashes..sha512..HashEngine$u20$as$u20$bitcoin_hashes..HashEngine$GT$5input17h321a237bfbe5c0bbE
```
Total size: 22552 bytes
## Conclusion
On `aarch64` there's overall a ~30% improvement in size, although ripemd160 doesn't really shrink that much (and its performance also aren't impacted much with only a 6% slowdown). sha512 and sha256 instead are almost 40% slower with `small-hash` enabled.
I don't have performance numbers for other architectures, but in terms of size there was an even larger improvements on `thumbv7em-none-eabihf`, with a 52% size reduction overall:
```
Size Crate Name
25.3KiB bitcoin_hashes <bitcoin_hashes[fe467ef2aa3a1470]::sha512::HashEngine as bitcoin_hashes[fe467ef2aa3a1470]::HashEngine>::input
6.9KiB bitcoin_hashes <bitcoin_hashes[fe467ef2aa3a1470]::sha256::HashEngine as bitcoin_hashes[fe467ef2aa3a1470]::HashEngine>::input
4.8KiB bitcoin_hashes <bitcoin_hashes[fe467ef2aa3a1470]::ripemd160::HashEngine as bitcoin_hashes[fe467ef2aa3a1470]::HashEngine>::input
```
vs
```
Size Crate Name
9.5KiB bitcoin_hashes <bitcoin_hashes[974bb476ef905797]::sha512::HashEngine as bitcoin_hashes[974bb476ef905797]::HashEngine>::input
4.5KiB bitcoin_hashes <bitcoin_hashes[974bb476ef905797]::ripemd160::HashEngine>::process_block
3.8KiB bitcoin_hashes <bitcoin_hashes[974bb476ef905797]::sha256::HashEngine as bitcoin_hashes[974bb476ef905797]::HashEngine>::input
```
I'm assuming this is because on more limited architectures the compiler needs to use more instructions to move data in and out of registers (especially for sha512 which ideally would benefit from 64-bit registers), so reusing the code by moving it into functions saves a lot of those instructions.
Also note that the `const_hash` method on `sha256` causes the compiler to emit two independent implementations. I haven't looked into the code yet, maybe there's a way to merge them so that the non-const `process_block` calls into the const fn.
-----
Note: commits are unverified right now because I don't have the keys available, I will sign them after addressing the review comments.
ACKs for top commit:
apoelstra:
ACK f2c5f19557
tcharding:
ACK f2c5f19557
Tree-SHA512: 1d5eb56324c458660e2571e8cf59895dc31dae9c5427c7ed36f8a0e81ca2e9a0f39026f56b6803df03635cc8b66aee3bf5182d51ab8972d169d56bcfec33771c
546c0122d7 Add simd sha256 intrinsics for x86 machines (sanket1729)
Pull request description:
This is my first time dabbling into architecture specific code and simd. The algorithm is a word to word translation of the C code from 4899efc81d/sha256-x86.c .
Some benchmarks:
With simd
```
test sha256::benches::sha256_10 ... bench: 11 ns/iter (+/- 0) = 909 MB/s
test sha256::benches::sha256_1k ... bench: 712 ns/iter (+/- 2) = 1438 MB/s
test sha256::benches::sha256_64k ... bench: 45,597 ns/iter (+/- 189) = 1437 MB/s
```
Without simd
```
test sha256::benches::sha256_10 ... bench: 47 ns/iter (+/- 0) = 212 MB/s
test sha256::benches::sha256_1k ... bench: 4,243 ns/iter (+/- 17) = 241 MB/s
test sha256::benches::sha256_64k ... bench: 271,263 ns/iter (+/- 1,610) = 241 MB/s
```
ACKs for top commit:
apoelstra:
ACK 546c0122d7
tcharding:
ACK 546c0122d7
Tree-SHA512: 7167c900b77e63cf38135a3960cf9ac2615f73b2ef7020a12b5cc3f4c047910063ba9045217b9ecfa70f7de1eb0f02f2674f291bd023a853bad2b9162fae831e
Currently if the witness has zero elements or any of the individual
witnesses is empty we panic. Panic is caused by subtracting 1 from a
zero length.
Check the length is non-zero before subtracting 1, print `[]` if empty.
When enabled this feature swaps the hash implementation of sha512,
sha256 and ripemd160 for a smaller (but also slower) one.
On embedded processors (Cortex-M4) it can lead to up to a 52% size
reduction, from around 37KiB for just the `process_block` methods of the
three hash functions to 17.8KiB.
During the 0.30.0 release we removed the re-exports of hash types. This
upset some folk and since the aim of our re-exports is not exactly clean
as well as the fact that the public API surface is not yet fixed just
re-export all the hash types at the crate root again.
This is 1792 but does not use a wildcard and also grabs the other
hashes that we recently moved.
The word "segwit" refers to segwit v0 and taproot but currently we have
`segwit_signature_hash` that is version specific (segwit v0).
- Rename `segwit_encode_signing_data_to` to
`segwit_v0_encode_signing_data_to`
- Add `p2wpkh_signature_hash` and `p2wsh_signature_hash` functions
We keep the single encode function because the error handling is better
that way.
While we are at it test the bip-143 test vectors against all the
sighash types of wrapped p2wsh.
02890d3fba Use spaces instead of tabs (yancy)
a640f0f034 Refer to the location where the deps are pinned (yancy)
Pull request description:
A rollup of two simple commits. 1) add a line to point developers to the CI test where the deps are tested and 2) replace tabs with spaces (spaces are used everywhere else) in `contrib/test.sh`.
ACKs for top commit:
apoelstra:
ACK 02890d3fba
tcharding:
ACK 02890d3fba
Tree-SHA512: 29b02e6d91a169429fe931a97ff7f86ac99016c6356f85d887a84141b3eec57527f85a43aeb3905a4acc73730e33a505c265b6a22ffbca1b102b63adf8599e4f
fa10668a35 Eliminate a heap allocation from PartialMerkleTree encoding & decoding (Steven Roose)
Pull request description:
Just came across this and felt like doing this.
ACKs for top commit:
apoelstra:
ACK fa10668a35
tcharding:
ACK fa10668a35
Tree-SHA512: 7167c63077851c4c461b33292948d9b09fe21eb45d52ed278ecf884ce15bc8b21c14040fa1eb5f50bfe51c5cde10abc133d0c59be502de139408c0d107ffa7eb
63d0fa0164 Rename All to Opcode (Tobin C. Harding)
881d1b1a9d Move opcode aliases to the all module (Tobin C. Harding)
f3e8dbc392 Remove floating code comment (Tobin C. Harding)
dfd9384d14 opcodes: Fix whitespace (Tobin C. Harding)
Pull request description:
I may be missing something about the `All` type but it seems it can be re-named to `Opcode` with no loss of clarity.
The first few patches do some other clean ups, the last patch is the meat and potatoes.
ACKs for top commit:
apoelstra:
ACK 63d0fa0164
Tree-SHA512: c1b8be9e0c090d015c2bbfcb7ed6abe998ccada5a1eb0f6966c2a7dc462f8bbc45e6b99bf1387c736d2fca21a993a2c7fd5844c18b97a2e37ec43da517566ab1
894aa130b6 Add CI test for pinned versions using MSRV (yancy)
Pull request description:
Add a test for pinned version with current MSRV. I'm not sure how to test this in CI before opening this PR (will probably fail).
ACKs for top commit:
apoelstra:
ACK 894aa130b6
tcharding:
ACK 894aa130b6
Tree-SHA512: ab6a68628d486f8a3aa47ad6921fd90a60df62f99f3bf017d4aac092510b226b3acb8d1ee3b80e9884006bfc746f4ec0d7f150df104ec30a2e0da67886ebd3cd
Currently the `bitcoinconsensus` error is part of the public API. This
hinders maintainability because changes to the verison of
`bitcoinconsensus` force a re-release in `rust-bitcoin`. This is
an unnecessary maintenance burden, we can wrap the error instead.
yancy
Tobin C. Harding
Andrew Poelstra
Alekos Filini
sanket1729
Steven Roose