ab4ea7c13d Enforce the MAX_MONEY invariant in amount types (Tobin C. Harding)
Pull request description:
Enforcing the `MAX_MONEY` invariant is quite involved because it means multiple things:
- Constructing amounts is now fallible
- Converting from unsigned to signed is now infallible
- Taking the absolute value is now infallible
- Integer overflow is eliminated in various places
Details:
- Update `from_sat` to check the invariant
- Fix all docs including examples
- Use the unchecked constructor in test code
- Comment any other use of the unchecked constructor
- Deprecate `unchecked_abs`
- Fail serde (using the horrible string error variant)
- Try not to use the unchecked constructor in rustdocs, no need to encourage unsuspecting users to use it.
- Use `?` in rustdoc examples (required by Rust API guidlines)
- Remove `TryFrom<Amount> for SignedAmount` because the conversion is now infallible. Add a `From` impl.
- Fix the arbitrary impls
- Maintain correct formatting
- Remove private `check_max` function as its no longer needed
Close#620
ACKs for top commit:
apoelstra:
ACK ab4ea7c13d08411044bd5f9c17457e926c80ed4d; successfully ran local tests
Tree-SHA512: bec963d8ea69e202f399cd19bca864b06f3e86323d376c2d2126d74093598f8bbbf19792b2327dba0862ef6f0201202778014a2be7a14991f02917d8ca312afb
c707b959b7 Rename timestamp module to time (Tobin C. Harding)
e2dee4900f Re-name Timestamp to BlockTime (Tobin C. Harding)
Pull request description:
Done in two patches so we can bikeshed the name of the type and separately the name of the module.
- Rename type: `Timestamp` to `BlockTime`
- Rename module: `timestamp` to `time`
ACKs for top commit:
apoelstra:
ACK c707b959b72dd89ca6df581a6102f32daedb8368; successfully ran local tests
Tree-SHA512: de3855b38445a58b6767a6081919eecb81c6c12aee3f6699f3bfa10efaf5770b54fb412da23991a9ee734e14dfb642af670f0218d1886cdc8c8d3f393ef65d7e
c0e20dbf2e test: add coverage for ServiceFlags::P2P_V2 (Bruno Garcia)
Pull request description:
Include `P2P_V2` on `service_flags_test`
ACKs for top commit:
tcharding:
ACK c0e20dbf2e
apoelstra:
ACK c0e20dbf2e1406a3f3df5538d1397b233078cd7c; successfully ran local tests
Tree-SHA512: 6ebb2f493bcc8fc6643cec67945a81692f03ceb976d75e10ce6052e775901846b9795f31817439b4484ef2d1eaae122d7da270cac2b59d7bb41cb2015593d1f5
Enforcing the MAX_MONEY invariant is quite involved because it means
multiple things:
- Constructing amounts is now fallible
- Converting from unsigned to signed is now infallible
- Taking the absolute value is now infallible
- Integer overflow is illuminated in various places
Details:
- Update from_sat to check the invariant
- Fix all docs including examples
- Use the unchecked constructor in test code
- Comment any other use of the unchecked constructor
- Deprecate unchecked_abs
- Fail serde (using the horrible string error variant)
- Try not to use the unchecked constructor in rustdocs, no need to encourage unsuspecting users to use it.
- Use ? in rustdoc examples (required by Rust API guidlines)
- Remove TryFrom<Amount> for SignedAmount because the conversion is now infallible. Add a From impl.
- Fix the arbitrary impls
- Maintain correct formatting
- Remove private check_max function as its no longer needed
8f74b823ab Add validation for private key format and master key constraints (Erick Cestari)
Pull request description:
This PR addresses issue #4195 by adding proper validation when decoding extended private keys:
### Changes
- Add validation to ensure byte 45 is zero as required by BIP-32 specification for private keys
- For master keys (depth=0), add validation to ensure parent fingerprint is zero
- For master keys (depth=0), add validation to ensure child number is zero
- Add corresponding error types to handle these validation failures
- Add unit tests to verify each validation rule
### Validation Rationale
These checks improve security by rejecting malformed extended keys that could potentially lead to unexpected behavior. As noted in the issue discussion, these validations are explicitly required by the BIP-32 specification.
### Testing
Added three new unit tests to verify each validation rule:
- test_reject_xpriv_with_non_zero_byte_at_index_45
- test_reject_xpriv_with_zero_depth_and_non_zero_index
- test_reject_xpriv_with_zero_depth_and_non_zero_parent_fingerprint
Fixes#4195
ACKs for top commit:
jrakibi:
ACK 8f74b823ab
tcharding:
ACK 8f74b823ab
apoelstra:
ACK 8f74b823ab8ef44bde7d003f8ba43fbe44dbef3e; successfully ran local tests
Tree-SHA512: 6a013e4917f83cfd7e39a2a18f7491853d791ab1d981a99eeea6204e1dab723fed7a168ff2a89e8850d512c3c381bfa1afef7fa32e5a0d246d949a46b01a3023
e4513bf925 feat: add MAX_BLOCK_SERIALIZED_SIZE existing in core (ChrisCho-H)
Pull request description:
fad0d9ea2d1e807806fa141238e279fddea6ae99: add `MAX_BLOCK_SERIALIZED_SIZE` as constant, which also exists in [bitcoin-core](59ff17e5af/src/consensus/consensus.h (L13)).
I originally thought it would be better to use this value for checking limit of push_bytes [here](0870cd1660/bitcoin/src/blockdata/script/push_bytes.rs (L31)), as it's the actual limit(`OP_PUSHDATA4` semantic says it could allow up to 4GB though). However, I'm not sure whether there might be need to push_bytes larger than `MAX_BLOCK_SERIALIZED_SIZE`, so just let developer use this constant to check the actual limit rather than enforcing it.
ACKs for top commit:
tcharding:
ACK e4513bf925
apoelstra:
ACK e4513bf9250799bc18a10728af184d6c86a561a4; successfully ran local tests
Tree-SHA512: 44c5a4882666ad286c1e1c40b9738929e2a8ad4bb44aaf48865fc395291185ae5aae351d26ac9334671e47a11e844bd037bd251a921b6b028a116d1b442b9183
5d851f1c3e Remove deprecated amount methods (Tobin C. Harding)
76a2d70b28 Make mul weight by fee return NumOpResult (Tobin C. Harding)
f9eb307953 Remove panic in dust value functions (Tobin C. Harding)
13595fbe7d Fix amount whole bitcoin constructors (Tobin C. Harding)
ac71680202 Pick one - MAX or MAX_MONEY (Tobin C. Harding)
6d70c77cf9 Enforce newtype sanity rules for amount types (Tobin C. Harding)
e6f7b26d80 Use _unchecked in amount const types (Tobin C. Harding)
ef0af8d62e Use sat/ssat constructors throughout tests (Andrew Poelstra)
8ecdc7c275 Use den_ prefix for local Denomination variable (Tobin C. Harding)
938461cc65 psbt: Use Amount::ZERO in unit test (Tobin C. Harding)
Pull request description:
We want to start enforcing MAX_MONEY as an invariant in the amount types. There are a few more steps we can do first to make that change easier to review.
ACKs for top commit:
jamillambert:
ACK 5d851f1c3e
apoelstra:
ACK 5d851f1c3e98d7d426e5897b2d734b77a299ccfb; successfully ran local tests
Tree-SHA512: 9e28b273d41fc143656e3a84736b6abe477fae5721b02bce7436551bd489cc235dc7e9fc68ffafa98f75a61065470ac514570a42bea94e90fedbb31f3cd61031
199f57849a Remove references to cfg(mutate) from lint allow - no longer allowed (AM)
a65d1d8b95 docs: Update README to replace use of mutagen with cargo-mutants (AM)
Pull request description:
Hey there!
I am just getting up to speed with the project and in following the README discovered that there are still references to the previous mutation testing tool `mutagen`. I updated the README to refer to the new tool, `cargo-mutation`.
I'm suggesting the user use the same command, `cargo mutants --in-place --no-shuffle`, as is run in the weekly CI workflow.
I noticed that there are still references to the old `mutate` attribute in the following files. I removed these as well as per [feedback](https://github.com/rust-bitcoin/rust-bitcoin/pull/4228#issuecomment-2709407253).
`primitives/Cargo.toml`:
```
[lints.rust]
unexpected_cfgs = { level = "deny", check-cfg = ['cfg(mutate)'] }
```
and
`bitcoin/Cargo.toml`:
```
unexpected_cfgs = { level = "deny", check-cfg = ['cfg(bench)', 'cfg(fuzzing)', 'cfg(kani)', 'cfg(mutate)'] }
```
Update to PR: removed incorrect understanding of logic in PR description as per [feedback](https://github.com/rust-bitcoin/rust-bitcoin/pull/4228#issuecomment-2709408598) and removed `cfg(mutate)` from above 2 files.
ACKs for top commit:
tcharding:
ACK 199f57849a
apoelstra:
ACK 199f57849acd9845902a8090ad6490a61ee03d24; successfully ran local tests
Tree-SHA512: e154c504aa5283f1da05d0120ea8dda97d1159389e692b0d57d7d864032ecb2b48c496054ede5500477367bc732dc34b0870f2709b8bd6e7b3f5c18a10f7a29e
Calculating the minimum non-dust fee currently panics if either the
script is really big or the dust fee rate is really big.
Harden the API by returning an `Option` instead of panicing.
This commit adds additional validation checks when decoding extended private keys:
1. Verifies that byte 45 is zero as required by BIP-32 specification
2. For master keys (depth=0), ensures parent fingerprint is zero
3. For master keys (depth=0), ensures child number is zero
These checks improve security by rejecting malformed keys that could
potentially lead to unexpected behavior. Added corresponding error types
and unit tests to verify each validation rule.
2aac5a1f81 Fix some comments (NinaLua)
Pull request description:
I fixed some typos in the comments, please review it.
ACKs for top commit:
Kixunil:
ACK 2aac5a1f81
apoelstra:
ACK 2aac5a1f81a9bb217c4dfb7e45b96188ea60e35b; successfully ran local tests
Tree-SHA512: 50a55451b166189e8ca3d2725ed7bb8ff95a8f1ebef0296c0003414871f1b211e6ffcc3b7225302dd3d6760bfc3f65cf8ed730327ceab60cd55b868ccb0cea9a
d1c758f5a4 Add fee_rate::serde re-export (Tobin C. Harding)
Pull request description:
When we added the `fee_rate::serde` module we forgot to re-export it. This is needed so downstream can do specify serde attributes on struct fields.
```rust
#[serde(with = "bitcoin::fee_rate::serde::as_sat_per_kwu")]
rate: FeeRate,
```
ACKs for top commit:
Kixunil:
ACK d1c758f5a4
apoelstra:
ACK d1c758f5a472a4a67cf9c7afa9ef9c0d793a2e16; successfully ran local tests
Tree-SHA512: 6e6f7879d8a0dab59d79f0e41dd5f9f791b72dfb5a1583d0c87ec04216c0a9c0e5c4fb328b93f5298af47b56d898f48717b1641f51295314423e6a569b4677fe
a013700527 Replace uses of `chunks_exact` with `as_chunks` (Martin Habovstiak)
Pull request description:
This is now ready for review.
In the past we've been using `chunks_exact` because const generics were unstable but then, when they were stabilized we didn't use `as_chunks` (or `array_chunks`) since they were unstable. But the instability was only because Rust devs don't know how to handle `0` being passed in. The function is perfectly implementable on stable. (With a tiny, easy-to-understand `unsafe` block.) `core` doesn't want to make a decision for all other crates yet but we can make it for our own crates because we know that we simply never pass zero. (And even if we did, we could just change the decision.)
It also turns out there's a hack to simulate `const {}` block in our MSRV, so we can make compilation fail early.
This commit adds an extension trait to internals to provide the methods, so we no longer have to use `chunks_exact`. It also cleans up the code quite nicely.
Previous unresolved question, leaving for reference:
> One issue with this change is that the names collide which could lead to hard error in future Rust versions. How do we solve it?
> * ignore and just backport the fix once that actually happens
> * rename the methods to something reasonable (e.g. `as_array_chunks`) - this risks that they'll rename the methods to the same thing by accident and it'll break anyway
> * rename the methods to something silly (`bitcoin_as_chunks`) - yeah, the risk above is not there but then we have silly-looking code.
We've decide to just rename the methods to something that won't possibly collide.
ACKs for top commit:
tcharding:
ACK a013700527
apoelstra:
ACK a01370052715b6733f07011f28944105493bda63; successfully ran local tests; nice!
Tree-SHA512: cc3359518f97e510da5ee9a33495e26c338bfc3e4162aaffcc72ed9c7daad0daf5e9ca3d23bce50877b0d3881792e98e28d21174a4426bb01281f12285ce08d1
ae0ba6c135 Take spent closure by value in count_witness_sigops and count_p2sh_sigops (jrakibi)
Pull request description:
This fixes#4141
Changed `count_witness_sigops` to take the `spent` closure by value instead of `&mut`
This removes the need for `&mut` when calling the function while still allowing mutable closure to be passed when needed
ACKs for top commit:
Kixunil:
ACK ae0ba6c135
tcharding:
ACK ae0ba6c135
apoelstra:
ACK ae0ba6c1356505697fc5e841741ac488538e3407; successfully ran local tests
Tree-SHA512: 76c5c98994b00412d0d371c07e3e83538f21754129a67889c66e1299e0453defaecb82bd4305297f772d65b042045d3579eaac14f8ea59419bf26b8b0d2ac84f
We just re-named `Timestamp` to `BlockTime`. We have a `units::block`
module but it currently holds abstractions (`BlockHeight` and
`BlockInterval`) that are not onchain abstractions and therefore
somewhat different from the `BlockTime`. Instead of making `block` a
block 'utils' module instead re-name the `timestamp` module to `time`.
We just added a `Timestamp` type without knowing that there was a push
by OpenTimestamps to also create a timestamp and that our new type may
lead to confusion. Our timestamp is explicitly for the `time` field in a
block so we can call it `BlockTime`. This name change makes the module
name stale but we will change that in a following patch to ease review.
95ad91cdb6 hashes: Remove the GeneralHash trait (Tobin C. Harding)
6426e59c63 Remove unused trait import (Tobin C. Harding)
791501eabc io: Use function in place of GeneralHashExt (Tobin C. Harding)
2b6ef31469 hashes: Add hash_byte_chunks function to modules (Tobin C. Harding)
d3846895d7 hashes: Add hash function to modules (Tobin C. Harding)
e1bac7da55 Bound HmacEngine on HashEngine (Tobin C. Harding)
ab63b7a0ff Add Hash type and finalize method to HashEngine (Tobin C. Harding)
84623ffaf9 Add hash_again regression test (Tobin C. Harding)
Pull request description:
This is the done as part of #4051.
Requires some surgery on the `Hmac` and `Hkdf` types as well as a few other patches to maintain the logic that is currently provided by the trait. Final patch is a pure red diff - enjoy.
ACKs for top commit:
Kixunil:
ACK 95ad91cdb6
apoelstra:
ACK 95ad91cdb64c8870d3eb992090bab7a70d1369b9; successfully ran local tests
Tree-SHA512: bfd215de51c115144c6f5b3430b17dad5d770a8c876fe3775af2828ec620a1f4e4155e63bb18dac244a82c3840413a615d55a0fef91b6949d3de319aa5bb8c2f
When we added the `fee_rate::serde` module we forgot to re-export it.
This is needed so downstream can do specify serde attributes on struct
fields.
```rust
#[serde(with = "bitcoin::fee_rate::serde::as_sat_per_kwu")]
rate: FeeRate,
```
5da506b506 Add additional re-exports (Tobin C. Harding)
Pull request description:
As we do for other types add two new alias' at the crate root of `primitives` and mirror it in `bitcoin`:
- `BlockVersion`
- `TransactionVersion`
ACKs for top commit:
apoelstra:
ACK 5da506b506aa41b88aa7e8ecdffdcc0478ec72b6; successfully ran local tests
Kixunil:
ACK 5da506b506
Tree-SHA512: 5f91e3aae87b1128256b528d20d9aab562bf054131ed8028e0db789e2b487863ad4c89aaca080d0fbcf760dd160815a1843046c5fab0ed1483230fdedc66402e
b3f122b399 Add Timestamp newtype (Tobin C. Harding)
Pull request description:
Bitcoin block headers have a timestamp. Currently we are using a `u32`. While this functions correctly it gives the compiler no chance to enforce type safety.
Add a `Timestamp` newtype that is a thin wrapper around a `u32`. Document it and test the API surface in `api.rs`.
ACKs for top commit:
apoelstra:
ACK b3f122b3996c1a73479be2f95b7f2ae642c9c56f; successfully ran local tests
Kixunil:
ACK b3f122b399
Tree-SHA512: 6f4a4a588bc836243ae28f3d36be6c0ae264cb2b7a0061277910b107d05e5ca0e679497d2890208f5d8ec148f37bf263bcd0b0410f9e5e6370d8e763ff30b78a
Pull request description:
Enhance Witness struct element access methods:
- Rename `nth()` to `get()` for clearer slice-like element retrieval
- Introduce `get_back()` method for flexible reverse indexing
- Remove redundant `second_to_last()` and `third_to_last()` methods
- Add `#[track_caller]` to index implementation for better error tracking
- Update all references to use new method names
- Improve documentation with usage examples
The changes provide a more intuitive and consistent approach to accessing witness elements.
Close#4098
ACKs for top commit:
Kixunil:
ACK 3ca3218c23
tcharding:
ACK 3ca3218c23
apoelstra:
ACK 3ca3218c236c63a9b006047524e2b47e310f07d9; successfully ran local tests
Tree-SHA512: 163e7457f3fe5141373e27a6df5fe1da6f2f05f02e877ef96243510d030d832c0fa86ade781e015a3c392f004651170b60438a83d330f1059457e5ade6478af7
In the past we've been using `chunks_exact` because const generics were
unstable but then, when they were stabilized we didn't use `as_chunks`
(or `array_chunks`) since they were unstable. But the instability was
only because Rust devs don't know how to handle `0` being passed in. The
function is perfectly implementable on stable. (With a tiny,
easy-to-understand `unsafe` block.) `core` doesn't want to make a
decision for all other crates yet but we can make it for our own crates
because we know that we simply never pass zero. (And even if we did, we
could just change the decision.)
It also turns out there's a hack to simulate `const {}` block in our
MSRV, so we can make compilation fail early.
This commit adds an extension trait to internals to provide the methods,
so we no longer have to use `chunks_exact`. It also cleans up the code
quite nicely.
d1d483491f Make `hex` in `internals` optional (Martin Habovstiak)
Pull request description:
The `hex` crate is not always desirable - e.g. when the consumer wants to work with raw data only. We already had this optional in `hashes` but if `hashes` is going to depend on `internals` it would break this property.
This change makes `hash` optional, since it's easy: there's just one struct that depends on it.
ACKs for top commit:
tcharding:
ACK d1d483491f
apoelstra:
ACK d1d483491f5f0181e60778e557fadd99b44b5d30; successfully ran local tests; nice
Tree-SHA512: c20091e6febb49b9114273c280580ddcdafc91893f3c73288b374f51990f09f035a044806dd26777148f2e4341ad082ab05f1b49f8ceb3bcd24eb210ffa5fd5f
This fixes issue #4141
Change count_witness_sigops and count_p2sh_sigops to take the spent
closure by value instead of &mut
- Changed both functions to accept S as a value (FnMut) instead of &mut S
- Removes need to annotate &mut when calling the function
Add a standalone `hash` function that is a drop in replacement for
`GeneralHash::hash`. Do not add it to `hmac` - this is in parity with
the current code because `Hmac` does not implement `GeneralHash::hash`.
Use the new function in `bitcoin` removing all occurrences of
`GeneralHash` from `bitcoin`.
In `hashes` replace usage of `GeneralHash::hash` with the new `hash`
function.
We would like to do away with the `GeneralHash` trait. Currently we
bound `Hmac` and `HmacEngine` on it but this is unnecessary now that we
have added `HashEngine::finalize` and `HashEngine::Hash`.
Bound the `HmacEngine` on `HashEngine` (which has an associated `Hash`
type returned by `finilalize`).
Bound `Hmac` type on `T::Hash` where `T` is `HashEngine`.
Includes some minor shortening of local variable names around hmac
engine usage.
Note this means that `Hmac` no longer implements `GeneralHash`.
ecc5791930 Add test checking `XOnlyPublicKey::deserialize` (Martin Habovstiak)
bbe87eccf2 Fix bug in PSBT `Deserialize` for `XOnlyPublicKey` (Martin Habovstiak)
8efacd4dda Deprecate `PrivateKey::from_slice` method (Martin Habovstiak)
0d5cd7af43 Add `from_byte_array` to `PrivateKey`. (Martin Habovstiak)
1778fea66e Add a test checking `PrivateKey::from_slice` (Martin Habovstiak)
b87ddc0043 Don't panic in `PrivateKey::from_slice` (Martin Habovstiak)
Pull request description:
This fixes the bug introduced in #4154 and deprecates `from_slice` method in favor of `from_byte_array` (see commits).
Closes#4191
ACKs for top commit:
jamillambert:
ACK ecc5791930
apoelstra:
ACK ecc5791930b88581fcbc8f2417b0221486bd1031; successfully ran local tests
Tree-SHA512: 05117c68db7b46605dba6104ee7696220416f4efaef1fff01843a910037e4c96bfebc45fcdd16f875e5e800bb33af17193c4aa9b0b1593807df5153e7e935c22
During upgrade of `secp256k1` a number of function calls needed to be
rewritten to not use `from_slice` but `from_byte_array`. Unfortunately,
the conversion wasn't correct and caused panics on invalid inputs
rather than errors.
This fixes it simply by calling `try_into` on the entire slice and
converting the error.
852bcf6017 bitcoin: Remove hash type re-exports (Tobin C. Harding)
Pull request description:
The `{W}PubkeyHash` and `{W}ScriptHash` types are not likely to be used directly by consumers of the library because we have other function that return them and are more ergonomic to use. There is therefor no good reason to re-export them from the crate root.
ACKs for top commit:
apoelstra:
ACK 852bcf60178c9491ebcbf13243cf79857592ccec; successfully ran local tests; will one-ACK merge, still does not affect any public API
Kixunil:
ACK 852bcf6017
Tree-SHA512: fbc4a7e3e116e962dc4b65b8681343e6ecd485a2289c5798979ce8e6ead4c0b6bda04ae33af1bdd711e852f95896107f9fa3c63d52fd36a6f802de4a7b677073
a74393324b Move opcodes back to bitcoin (Tobin C. Harding)
Pull request description:
Duplicate `opcodes` in `bitcoin` and hide it in `primitives` so we do not have to commit to the API.
We use opcodes in `impl fmt::Display for Script`.
Close: #4144
ACKs for top commit:
apoelstra:
ACK a74393324bd47f89fd47281d567ab15ab6bcb2ba; successfully ran local tests; sure
Kixunil:
ACK a74393324b
Tree-SHA512: 738685b9cd2288a581daa6219e3b21bd48bb4845ea627bf6b8085e0e48f5649ac5ec616a3421d10cd37543f76b66d31f94fd55bf94effc2fb8f91d1ecf5c8611
a4b9c196b1 Manually update nightly version (Tobin C. Harding)
Pull request description:
While trying to use the `macro_use_imports` lint I found that there is a bug in last weeks nightly. It has been fixed already so lets update.
Update to todays nightly compiler. Doing so causes some new linter warnings, for now we just allow them.
ACKs for top commit:
apoelstra:
ACK a4b9c196b13a2029ef0de198114a29c71a192d03; successfully ran local tests
Tree-SHA512: a982ba05713c214af3b9375ecf37b66aea07ec0330392c30d2dbe3c9b40e1219b587ee43222f2355292b9b374a6a1d8a19d18b559bb47d33e544ed78509a51b8
Since arrays better convey the intention than slices when parsing
fixed-sized bytes we're migrating to them. This deprecates the
`from_slice` method similarly to how we do it elsewhere.
Private keys have statically-known length of 32 bytes and we are
migrating types with known lenths to use `from_byte_array` methods. This
adds the method to `PrivateKey` as well and uses it to implement
`from_slice`.
During upgrade of `secp256k1` a number of function calls needed to be
rewritten to not use `from_slice` but `from_byte_array`. Unfortunately,
the conversions wasn't correct and caused panics on invalid inputs
rather than errors.
This fixes it simply by calling `try_into` on the entire slice and
converting the error.
Fmt Bot
merge-script
Tobin C. Harding
kevincatty
Bruno Garcia
planetBoy
ChrisCho-H
Erick Cestari
AM
wgyt
yancy
NinaLua
Martin Habovstiak
jrakibi