db9ec3bed8 Remove From<newtype> for $hash (Tobin C. Harding)
6b2b89c2f7 Remove From<hash> for not-general-hash types (Tobin C. Harding)
200ff47327 Use compute_merkle_root (Tobin C. Harding)
Pull request description:
The `hash_newtype` macro is explicitly designed to produce a hash that is not a general purpose hash type to try and prevent users hashing arbitrary stuff with it. E.g., `Txid` isn't meant to be just hash arbitrary data. However we provide a `From` impl that will convert any instance of the inner hash type into the new type. This kind of defeats the purpose. We provide `from_byte_array` and `to_byte_array` to allow folk to 'cast' from one hash type to another if they really want to and its ugly on purpose.
Also, it is becoming apparent that we may be able to remove the `hashes` crate from the public API of `primitives` allowing us to stabalise `primitives` without stabalising `hashes`.
For both these reasons remove the `From` impl from the `hash_newtype` macro. Note that deprecating doesn't seem to work so we just delete it.
ACKs for top commit:
Kixunil:
ACK db9ec3bed8
apoelstra:
ACK db9ec3bed8d6164a0345ba8db1e2162626db7cc5; successfully ran local tests
Tree-SHA512: 90bc325821cd2d72bbaef5b3cfef2d299192d1e7999cd4f96b6b69b8872e419964e431e91674c59bfdd2e9a5959dbc13ee89d5f87d03e96785044c616db19d72
The `hash_newtype` macro is explicitly designed to produce a hash that
is not a general purpose hash type to try and prevent users hashing
arbitrary stuff with it. E.g., `Txid` isn't meant to be just hash
arbitrary data. However we provide a `From` impl that will convert any
instance of the inner hash type into the new type. This kind of defeats
the purpose. We provide `from_byte_array` and `to_byte_array` to allow
folk to 'cast' from one hash type to another if they really want to and
its ugly on purpose.
Also, it is becoming apparent that we may be able to remove the `hashes`
crate from the public API of `primitives` allowing us to stabalise
`primitives` without stabalising `hashes`.
For both these reasons remove the `From` impl from the `hash_newtype`
macro. Note that deprecating doesn't seem to work so we just delete it.
These are defined in the BIP as invalid. The previous commit fixed a bug
where invalid key was parsed as valid and this bug can be caught by
these vectors. Therefore, if this commit is ordered before the last one
the test will fail.
Previously we've used `try_into().expect()` because const generics were
unavailable. Then they became available but we didn't realize we could
already convert a bunch of code to not use panicking conversions. But we
can (and could for a while).
This adds an extension trait for arrays to provide basic non-panicking
operations returning arrays, so they can be composed with other
functions accepting arrays without any conversions. It also refactors a
bunch of code to use the non-panicking constructs but it's certainly not
all of it. That could be done later. This just aims at removing the
ugliest offenders and demonstrate the usefulness of this approach.
Aside from this, to avoid a bunch of duplicated work, this refactors
BIP32 key parsing to use a common method where xpub and xpriv are
encoded the same. Not doing this already led to a mistake where xpriv
implemented some additional checks that were missing in xpub. Thus this
change also indirectly fixes that bug.
9ea2e9262f Don't use references to `TaprootMerkleBranchBuf` (Martin Habovstiak)
c528f52894 Change `Deref::Target` of `TaprootMerkleBranchBuf` (Martin Habovstiak)
04a4efbe63 Introduce unsized `TaprootMerkleBranch` (Martin Habovstiak)
370c2597c6 Add `as_mut_slice` to `TaprootMerkleBranchBuf` (Martin Habovstiak)
33d75659da Push `merkle_branch` module one level deeper. (Martin Habovstiak)
277045bad7 Add `Buf` suffix to `TaprootMerkleBranch` (Martin Habovstiak)
Pull request description:
This implements a bunch of changes needed to make `ControlBlock` alloc-free. In particular, this allows constructing `Witness` without the intermediate allocation. It is also a step towards having `P2TrSpend` public.
Closes#1614
This also intentionally does **not** address decoding of `ControlBlock` from `Witness` since I'm not sure about the API.
Rationale for doing the `Buf` rename: while doing it with `Script` was very painful it shouldn't be here since it's not used that often and also we can just backport the first commit with deprecated type alias. I was thinking of having `TaprootMerkleBr` but it'd be inconsistent and the name is silly.
(Also if anyone is wondering why I did this: I was too exhausted to do more important stuff but felt like doing something nice and easy like this.)
ACKs for top commit:
tcharding:
ACK 9ea2e9262f
apoelstra:
ACK 9ea2e9262fbc04ea6fad33047de0fc1ead999dc7; successfully ran local tests
Tree-SHA512: c5e3ea61d10fbe0cbce5e900943e3cef77a175a62043c500b3ff6df57a96f00692d80fb1c4dd75bca9a704201baab6ddfcc430b12c7ecabc43968198466fed9d
We have a ton of calls to `from_sat_unchecked` for small constants which
were clearly in range, e.g. in fee.rs. Add a new constfn for these
cases. Don't bother making a generic Into<u32>/Into<u16> variant because
there isn't an obvious name for it.
There are 7 instances where we're using this method with values that are
out of range, which we leave as from_sat_unchecked for now.
The new unsized type is more flexible and so are the references to it.
Just like we pass around `&str` instead of `&String` we should be
passing `&TaprootMerkleBranch` instead of `&TaprootMerkleBranchBuf`.
`TaprootMerkleBranchBuf` previously derefed to a slice which lost the
information about length being valid. This commit changes the type
which, while API-breaking, is not disruptive because the type has API
very similar to slice.
`TaprootMerkleBranchBuf` being a vec introduced intermediate allocation
when creating or decoding `Witness`. However the representation on the
wire is the same as in-memory (aside from `#[repr(transparent)]`) so
this allocation wasn't really needed.
This commit introduces `TaprootMerkleBranch` type which is unsized and
can be used in place of `TaprootMerkleBranchBuf` within `ControlBlock`.
Aside from removing the intermediate allocation, this improves the API a
bit: the conversion from array to other type is no longer needed because
it's performed by `ControlBlock` in its methods. Thus, consumers who
have an array can simply set it as `merkle_branch` field and then encode
the `ControlBlock` into witness. A convenience method is also provided
to push the `ControlBlock` along with other parts at the end of the
`Witness`.
f4f79f88eb Enable getting the network kind from an address (Tobin C. Harding)
Pull request description:
Users may wish to ask of an address 'what kind of address is this?' We have the `NetworkKind` struct that abstracts over the answer but currently no API to ask the question.
The address may have been parsed or constructed and weather the network has been checked already is immaterial. Hence we add the function for both `NetworkChecked` and `NetworkUnchecked` addresses.
Fix: #4247
ACKs for top commit:
apoelstra:
ACK f4f79f88eb2c6c80c46c95c69fcc43b17d306be2; successfully ran local tests
Kixunil:
ACK f4f79f88eb
Tree-SHA512: 57bdf7a0f2ae8bf599b3830d10201af3f6312a802ab72c0d86e346af660cbc4f430954e46d6698032a062514ec3ee1ee7edc732beff79af99a84ce718a519afa
Users may wish to ask of an address 'what kind of address is this?' We
have the `NetworkKind` struct that abstracts over the answer but
currently no API to ask the question.
The address may have been parsed or constructed and weather the network
has been checked already is immaterial. Hence we add the function for
both `NetworkChecked` and `NetworkUnchecked` addresses.
Fix: #4247
ab4ea7c13d Enforce the MAX_MONEY invariant in amount types (Tobin C. Harding)
Pull request description:
Enforcing the `MAX_MONEY` invariant is quite involved because it means multiple things:
- Constructing amounts is now fallible
- Converting from unsigned to signed is now infallible
- Taking the absolute value is now infallible
- Integer overflow is eliminated in various places
Details:
- Update `from_sat` to check the invariant
- Fix all docs including examples
- Use the unchecked constructor in test code
- Comment any other use of the unchecked constructor
- Deprecate `unchecked_abs`
- Fail serde (using the horrible string error variant)
- Try not to use the unchecked constructor in rustdocs, no need to encourage unsuspecting users to use it.
- Use `?` in rustdoc examples (required by Rust API guidlines)
- Remove `TryFrom<Amount> for SignedAmount` because the conversion is now infallible. Add a `From` impl.
- Fix the arbitrary impls
- Maintain correct formatting
- Remove private `check_max` function as its no longer needed
Close#620
ACKs for top commit:
apoelstra:
ACK ab4ea7c13d08411044bd5f9c17457e926c80ed4d; successfully ran local tests
Tree-SHA512: bec963d8ea69e202f399cd19bca864b06f3e86323d376c2d2126d74093598f8bbbf19792b2327dba0862ef6f0201202778014a2be7a14991f02917d8ca312afb
c707b959b7 Rename timestamp module to time (Tobin C. Harding)
e2dee4900f Re-name Timestamp to BlockTime (Tobin C. Harding)
Pull request description:
Done in two patches so we can bikeshed the name of the type and separately the name of the module.
- Rename type: `Timestamp` to `BlockTime`
- Rename module: `timestamp` to `time`
ACKs for top commit:
apoelstra:
ACK c707b959b72dd89ca6df581a6102f32daedb8368; successfully ran local tests
Tree-SHA512: de3855b38445a58b6767a6081919eecb81c6c12aee3f6699f3bfa10efaf5770b54fb412da23991a9ee734e14dfb642af670f0218d1886cdc8c8d3f393ef65d7e
c0e20dbf2e test: add coverage for ServiceFlags::P2P_V2 (Bruno Garcia)
Pull request description:
Include `P2P_V2` on `service_flags_test`
ACKs for top commit:
tcharding:
ACK c0e20dbf2e
apoelstra:
ACK c0e20dbf2e1406a3f3df5538d1397b233078cd7c; successfully ran local tests
Tree-SHA512: 6ebb2f493bcc8fc6643cec67945a81692f03ceb976d75e10ce6052e775901846b9795f31817439b4484ef2d1eaae122d7da270cac2b59d7bb41cb2015593d1f5
Enforcing the MAX_MONEY invariant is quite involved because it means
multiple things:
- Constructing amounts is now fallible
- Converting from unsigned to signed is now infallible
- Taking the absolute value is now infallible
- Integer overflow is illuminated in various places
Details:
- Update from_sat to check the invariant
- Fix all docs including examples
- Use the unchecked constructor in test code
- Comment any other use of the unchecked constructor
- Deprecate unchecked_abs
- Fail serde (using the horrible string error variant)
- Try not to use the unchecked constructor in rustdocs, no need to encourage unsuspecting users to use it.
- Use ? in rustdoc examples (required by Rust API guidlines)
- Remove TryFrom<Amount> for SignedAmount because the conversion is now infallible. Add a From impl.
- Fix the arbitrary impls
- Maintain correct formatting
- Remove private check_max function as its no longer needed
8f74b823ab Add validation for private key format and master key constraints (Erick Cestari)
Pull request description:
This PR addresses issue #4195 by adding proper validation when decoding extended private keys:
### Changes
- Add validation to ensure byte 45 is zero as required by BIP-32 specification for private keys
- For master keys (depth=0), add validation to ensure parent fingerprint is zero
- For master keys (depth=0), add validation to ensure child number is zero
- Add corresponding error types to handle these validation failures
- Add unit tests to verify each validation rule
### Validation Rationale
These checks improve security by rejecting malformed extended keys that could potentially lead to unexpected behavior. As noted in the issue discussion, these validations are explicitly required by the BIP-32 specification.
### Testing
Added three new unit tests to verify each validation rule:
- test_reject_xpriv_with_non_zero_byte_at_index_45
- test_reject_xpriv_with_zero_depth_and_non_zero_index
- test_reject_xpriv_with_zero_depth_and_non_zero_parent_fingerprint
Fixes#4195
ACKs for top commit:
jrakibi:
ACK 8f74b823ab
tcharding:
ACK 8f74b823ab
apoelstra:
ACK 8f74b823ab8ef44bde7d003f8ba43fbe44dbef3e; successfully ran local tests
Tree-SHA512: 6a013e4917f83cfd7e39a2a18f7491853d791ab1d981a99eeea6204e1dab723fed7a168ff2a89e8850d512c3c381bfa1afef7fa32e5a0d246d949a46b01a3023
e4513bf925 feat: add MAX_BLOCK_SERIALIZED_SIZE existing in core (ChrisCho-H)
Pull request description:
fad0d9ea2d1e807806fa141238e279fddea6ae99: add `MAX_BLOCK_SERIALIZED_SIZE` as constant, which also exists in [bitcoin-core](59ff17e5af/src/consensus/consensus.h (L13)).
I originally thought it would be better to use this value for checking limit of push_bytes [here](0870cd1660/bitcoin/src/blockdata/script/push_bytes.rs (L31)), as it's the actual limit(`OP_PUSHDATA4` semantic says it could allow up to 4GB though). However, I'm not sure whether there might be need to push_bytes larger than `MAX_BLOCK_SERIALIZED_SIZE`, so just let developer use this constant to check the actual limit rather than enforcing it.
ACKs for top commit:
tcharding:
ACK e4513bf925
apoelstra:
ACK e4513bf9250799bc18a10728af184d6c86a561a4; successfully ran local tests
Tree-SHA512: 44c5a4882666ad286c1e1c40b9738929e2a8ad4bb44aaf48865fc395291185ae5aae351d26ac9334671e47a11e844bd037bd251a921b6b028a116d1b442b9183
5d851f1c3e Remove deprecated amount methods (Tobin C. Harding)
76a2d70b28 Make mul weight by fee return NumOpResult (Tobin C. Harding)
f9eb307953 Remove panic in dust value functions (Tobin C. Harding)
13595fbe7d Fix amount whole bitcoin constructors (Tobin C. Harding)
ac71680202 Pick one - MAX or MAX_MONEY (Tobin C. Harding)
6d70c77cf9 Enforce newtype sanity rules for amount types (Tobin C. Harding)
e6f7b26d80 Use _unchecked in amount const types (Tobin C. Harding)
ef0af8d62e Use sat/ssat constructors throughout tests (Andrew Poelstra)
8ecdc7c275 Use den_ prefix for local Denomination variable (Tobin C. Harding)
938461cc65 psbt: Use Amount::ZERO in unit test (Tobin C. Harding)
Pull request description:
We want to start enforcing MAX_MONEY as an invariant in the amount types. There are a few more steps we can do first to make that change easier to review.
ACKs for top commit:
jamillambert:
ACK 5d851f1c3e
apoelstra:
ACK 5d851f1c3e98d7d426e5897b2d734b77a299ccfb; successfully ran local tests
Tree-SHA512: 9e28b273d41fc143656e3a84736b6abe477fae5721b02bce7436551bd489cc235dc7e9fc68ffafa98f75a61065470ac514570a42bea94e90fedbb31f3cd61031
199f57849a Remove references to cfg(mutate) from lint allow - no longer allowed (AM)
a65d1d8b95 docs: Update README to replace use of mutagen with cargo-mutants (AM)
Pull request description:
Hey there!
I am just getting up to speed with the project and in following the README discovered that there are still references to the previous mutation testing tool `mutagen`. I updated the README to refer to the new tool, `cargo-mutation`.
I'm suggesting the user use the same command, `cargo mutants --in-place --no-shuffle`, as is run in the weekly CI workflow.
I noticed that there are still references to the old `mutate` attribute in the following files. I removed these as well as per [feedback](https://github.com/rust-bitcoin/rust-bitcoin/pull/4228#issuecomment-2709407253).
`primitives/Cargo.toml`:
```
[lints.rust]
unexpected_cfgs = { level = "deny", check-cfg = ['cfg(mutate)'] }
```
and
`bitcoin/Cargo.toml`:
```
unexpected_cfgs = { level = "deny", check-cfg = ['cfg(bench)', 'cfg(fuzzing)', 'cfg(kani)', 'cfg(mutate)'] }
```
Update to PR: removed incorrect understanding of logic in PR description as per [feedback](https://github.com/rust-bitcoin/rust-bitcoin/pull/4228#issuecomment-2709408598) and removed `cfg(mutate)` from above 2 files.
ACKs for top commit:
tcharding:
ACK 199f57849a
apoelstra:
ACK 199f57849acd9845902a8090ad6490a61ee03d24; successfully ran local tests
Tree-SHA512: e154c504aa5283f1da05d0120ea8dda97d1159389e692b0d57d7d864032ecb2b48c496054ede5500477367bc732dc34b0870f2709b8bd6e7b3f5c18a10f7a29e
Calculating the minimum non-dust fee currently panics if either the
script is really big or the dust fee rate is really big.
Harden the API by returning an `Option` instead of panicing.
This commit adds additional validation checks when decoding extended private keys:
1. Verifies that byte 45 is zero as required by BIP-32 specification
2. For master keys (depth=0), ensures parent fingerprint is zero
3. For master keys (depth=0), ensures child number is zero
These checks improve security by rejecting malformed keys that could
potentially lead to unexpected behavior. Added corresponding error types
and unit tests to verify each validation rule.
2aac5a1f81 Fix some comments (NinaLua)
Pull request description:
I fixed some typos in the comments, please review it.
ACKs for top commit:
Kixunil:
ACK 2aac5a1f81
apoelstra:
ACK 2aac5a1f81a9bb217c4dfb7e45b96188ea60e35b; successfully ran local tests
Tree-SHA512: 50a55451b166189e8ca3d2725ed7bb8ff95a8f1ebef0296c0003414871f1b211e6ffcc3b7225302dd3d6760bfc3f65cf8ed730327ceab60cd55b868ccb0cea9a
d1c758f5a4 Add fee_rate::serde re-export (Tobin C. Harding)
Pull request description:
When we added the `fee_rate::serde` module we forgot to re-export it. This is needed so downstream can do specify serde attributes on struct fields.
```rust
#[serde(with = "bitcoin::fee_rate::serde::as_sat_per_kwu")]
rate: FeeRate,
```
ACKs for top commit:
Kixunil:
ACK d1c758f5a4
apoelstra:
ACK d1c758f5a472a4a67cf9c7afa9ef9c0d793a2e16; successfully ran local tests
Tree-SHA512: 6e6f7879d8a0dab59d79f0e41dd5f9f791b72dfb5a1583d0c87ec04216c0a9c0e5c4fb328b93f5298af47b56d898f48717b1641f51295314423e6a569b4677fe
a013700527 Replace uses of `chunks_exact` with `as_chunks` (Martin Habovstiak)
Pull request description:
This is now ready for review.
In the past we've been using `chunks_exact` because const generics were unstable but then, when they were stabilized we didn't use `as_chunks` (or `array_chunks`) since they were unstable. But the instability was only because Rust devs don't know how to handle `0` being passed in. The function is perfectly implementable on stable. (With a tiny, easy-to-understand `unsafe` block.) `core` doesn't want to make a decision for all other crates yet but we can make it for our own crates because we know that we simply never pass zero. (And even if we did, we could just change the decision.)
It also turns out there's a hack to simulate `const {}` block in our MSRV, so we can make compilation fail early.
This commit adds an extension trait to internals to provide the methods, so we no longer have to use `chunks_exact`. It also cleans up the code quite nicely.
Previous unresolved question, leaving for reference:
> One issue with this change is that the names collide which could lead to hard error in future Rust versions. How do we solve it?
> * ignore and just backport the fix once that actually happens
> * rename the methods to something reasonable (e.g. `as_array_chunks`) - this risks that they'll rename the methods to the same thing by accident and it'll break anyway
> * rename the methods to something silly (`bitcoin_as_chunks`) - yeah, the risk above is not there but then we have silly-looking code.
We've decide to just rename the methods to something that won't possibly collide.
ACKs for top commit:
tcharding:
ACK a013700527
apoelstra:
ACK a01370052715b6733f07011f28944105493bda63; successfully ran local tests; nice!
Tree-SHA512: cc3359518f97e510da5ee9a33495e26c338bfc3e4162aaffcc72ed9c7daad0daf5e9ca3d23bce50877b0d3881792e98e28d21174a4426bb01281f12285ce08d1
ae0ba6c135 Take spent closure by value in count_witness_sigops and count_p2sh_sigops (jrakibi)
Pull request description:
This fixes#4141
Changed `count_witness_sigops` to take the `spent` closure by value instead of `&mut`
This removes the need for `&mut` when calling the function while still allowing mutable closure to be passed when needed
ACKs for top commit:
Kixunil:
ACK ae0ba6c135
tcharding:
ACK ae0ba6c135
apoelstra:
ACK ae0ba6c1356505697fc5e841741ac488538e3407; successfully ran local tests
Tree-SHA512: 76c5c98994b00412d0d371c07e3e83538f21754129a67889c66e1299e0453defaecb82bd4305297f772d65b042045d3579eaac14f8ea59419bf26b8b0d2ac84f
We just re-named `Timestamp` to `BlockTime`. We have a `units::block`
module but it currently holds abstractions (`BlockHeight` and
`BlockInterval`) that are not onchain abstractions and therefore
somewhat different from the `BlockTime`. Instead of making `block` a
block 'utils' module instead re-name the `timestamp` module to `time`.
We just added a `Timestamp` type without knowing that there was a push
by OpenTimestamps to also create a timestamp and that our new type may
lead to confusion. Our timestamp is explicitly for the `time` field in a
block so we can call it `BlockTime`. This name change makes the module
name stale but we will change that in a following patch to ease review.
merge-script
Tobin C. Harding
Martin Habovstiak
planetBoy
healthyyyoung
Fmt Bot
kevincatty
Bruno Garcia
ChrisCho-H
Erick Cestari
AM
wgyt
yancy