bb8bd16302 internals: Remove hex module (Tobin C. Harding)
2268b44911 Depend on hex-conservative (Tobin C. Harding)
db50509cd3 Add usage docs to the "core2" feature (Tobin C. Harding)
Pull request description:
Use the newly released `hex-conservative` crate, by doing the following:
- Depend on `hex-conservative` in `bitcoin` and `hashes`
- Re-export `hex-conservative` as `hex` from both crate roots.
- Remove all the old hex code from `hashes`
- Remove all the old hex code from `internals`
- Remove the now unused `internals::prelude`
- Fix all the import statements (makes up the bulk of the lines changes in this patch)
ACKs for top commit:
apoelstra:
ACK bb8bd16302
sanket1729:
utACK bb8bd16302
Tree-SHA512: ec83b3941cae6f32272471779f28461bb04959a3f6a126a68bbf2c748d83ff9518ff8932d9e937a6f389c10028bf3eb58c6b6d71ea066924dd7a34faaec7a087
This type was defined in the `transaction` module because it was
originally used in a function that had been deprecated in favour of
moving the logic to the `sighash` module.
We just removed the deprecated code so we can now move this type to the
`sighash` module where it is used.
We only keep deprecated code around for one release so we can now remove
code deprecated in v0.30.0
Done in preparation as we gear up for v0.31.0 release.
We have just released the `hex-conservative` crate, we can now use it.
Do the following:
- Depend on `hex-conservative` in `bitcoin` and `hashes`
- Re-export `hex-conservative` as `hex` from both crate roots.
- Remove all the old hex code from `hashes`
- Fix all the import statements (makes up the bulk of the lines changed
in this patch)
Pull all the code that depends on `bitcoinconsensus` out into a separate
module `consensus::validation`.
Leave transaction testing of bitcoinconsensus code in the transaction
module.
There is not need to return the general `script::Error` from the
transaction verify functions. We can better describe the error path by
returning a custom error.
Currently the test `hex` macro is only available when the `test`
compiler configuration option is set but we are using it in benches
code, this works for use because `cargo bench` sets `test` for the
current crate, however it breaks downstream crates.
Fix: #1830
These constants had an error that they had `script_size` set to 0 which
was incorrect because it's not length of the script but serialized size.
Rather than just bumping the value this uses the `from_slice` method
which is less error-prone.
This also deletes a useless test of the constants.
Closes#1834
dff757d7db Comment predict_weight (yancy)
Pull request description:
I've been reading over the `predict_weight` function since it is one of the biggest challenges for coin-selection. IE choosing inputs and constructing an optimal selection strategy requires predicting the weight to get the best selection. It's great this work has been done but there are some things I don't understand well enough to comment.
1) why are we looking at the size of VarInt struct here
> let script_size = script_len + VarInt(script_len as u64).len()
2) [predict_weight_internal](36500b4451/bitcoin/src/blockdata/transaction.rs (L1245)) has a bunch of magic numbers. I'd like to be able to comment this as well but I don't fully understand that function.
Also, `Transaction.rs` is a big file and it seems like all of the prediction stuff could be moved to a separate module or maybe a separate crate?
ACKs for top commit:
tcharding:
ACK dff757d7db
Kixunil:
ACK dff757d7db
Tree-SHA512: 8ffa16d500075d691528ce1819b9352a148af431889bebbd7cddcf470bd4e3048ec53a5e778bc3659e33d8c25b68422a93dac1d46b9489ff56f41d88d7f05433
Our previous MSRV did not support MIN/MAX associated consts so we had
methods min/max_value. Now that our MSRV is Rust 1.48.0 we can use the
consts.
Deprecate min/max_value methods in favor of MIN/MAX associated conts.
We currently use the functions `min_value` and `max_value` because the
consts were not available in Rust 1.41.1, however we recently bumped the
MSRV so we can use the consts now.
Currently we have a mishmash of attribution lines accompanying the SPDX
identifier. These lines are basically meaningless because:
- The date is often wrong
- The original author attributed is not the only contributor to a file
- The term "rust bitcoin developers" is basically just noise
Just remove all the attribution lines and be done with it. While we are
at it add an SPDX line to the few files missing it, whether this license
nonsense is even needed is left as an argument for another day.
dd4ad9444e Hardcode expected weight in txin_txout_weight_tests (Peter Todd)
Pull request description:
Rational: the expected weight is fixed so this both ensures we don't accidentally change it somehow, and makes it easier to re-use these test cases in other codebases (eg python-bitcoinlib).
ACKs for top commit:
apoelstra:
ACK dd4ad9444e
tcharding:
ACK dd4ad9444e
Kixunil:
ACK dd4ad9444e
Tree-SHA512: 4769a4bb8695f4f4c95e258bb5f06a232090b14c3d9159d6d5de2d09d7fc934a1b920b90cc09677a88fc0cf37ac21ed27794692dff2c73df4252c9551dc10fc2
Rational: the expected weight is fixed so this both ensures we don't
accidentally change it somehow, and makes it easier to re-use these test
cases in other codebases (eg python-bitcoinlib).
a54e1ceab1 Apply rustfmt (The rustfmt Tyranny)
38d11ce3da ci: Make release CI search for NEXT.RELEASE instead (Steven Roose)
dad3abd20f transaction: Rename is_coin_base to is_coinbase (Steven Roose)
Pull request description:
Alternative to https://github.com/rust-bitcoin/rust-bitcoin/pull/1795.
Keep the old method as deprecated and add doc alias. Also change internal usage of the method.
ACKs for top commit:
tcharding:
ACK a54e1ceab1
apoelstra:
ACK a54e1ceab1
Tree-SHA512: 52d9729bf83da164556d960f8867cb836ff57a0f619da3dd3620efffb28a974aac23b8085863ab0e072a4bdb2f13ac576efa43ad2eec9a271ad044227f4d00a4
fabcde036f Use package in manifest and shorten import (Tobin C. Harding)
Pull request description:
We can use `package` to rename `bitcoin_hashes` to `hashes` and `bitcoin_internals` to `internals`. This makes imports more terse with no loss of meaning.
ACKs for top commit:
apoelstra:
ACK fabcde036f
Kixunil:
ACK fabcde036f
Tree-SHA512: bc5bff6f7f6bf3b68ba1e0644a83da014081d8c6c9d578c21cb54fdd56a018f68733dd1135d05b590ba193ed9efd12fa9019182c1fed347e604d8548f6ef9103
If we use `#![cfg_attr(docsrs, feature(doc_auto_cfg))]` instead of
`#![cfg_attr(docsrs, feature(doc_cfg))]` we no longer need to manually
mark types with `#[cfg_attr(docsrs, doc(cfg(feature = "std")))]`.
Sweeeeeet.
We can use `package` to rename `bitcoin_hashes` to `hashes` and
`bitcoin_internals` to `internals`. This makes imports more terse with
no loss of meaning.
122188f7dd Use shorter import statements (Tobin C. Harding)
Pull request description:
Just patch 2, patch 1 is #1728
From the commit log of patch 2
Use shorter import statements
As per discussion [0] use the shorter form for importing crates that we
re-export (`hashes` and `secp256k1`).
[0] https://github.com/rust-bitcoin/rust-bitcoin/discussions/1661
ACKs for top commit:
apoelstra:
ACK 122188f7dd
sanket1729:
utACK 122188f7dd
Tree-SHA512: 3f540464d38c72ba9d68f8ceda8600540bd0c3eef0ba67531c87fa1e0e4f757af7035cf80a1a5f17aa05604a17fdd9ef59bb6bece6b4145d540dac1e5362fc01
Some smart contracts or simplified wallets statically know the sizes of
transactions or inputs. The possible approaches to handling them so far
were re-computing the values (and hoping the optimizer will const fold
them) or using a simple constant which may be harder to understand and
get right. It's much nicer to just use a `const` but our code didn't
support it until now.
This change adds methods that can compute the prediction in `const`
context for Rust versions >= 1.46.0 which allow use of loops (and
conditions but those could be workaround anyway).
As a side effect of this, the change also adds `const` to `VarInt::len`
in Rust 1.46+. While this one could be made unconditional using array
trick it's probably not worth it because of the planned MSRV bump.
Note: this commit is intentionally unformatted to make diff easier to
understand. Formatting will be done in future commit.
There are several common spends in Bitcoin that have known input weight
predictions. It can be useful to have these as constants, so this change
adds them. However, this only adds native segwit ones as the others are
slowly fading away and might clutter the API.
We created the `crypto` crate as a container for cryptography modules
with the idea that it may be split out into a separate crate. There is
no reason for users of the lib to know about this module. Also, we have
two `taproot` modules, one in `crypto` and one at the crate root, this
makes for un-ergonomic usage of the lib.
Improve the public API by doing:
- Make the `crypto` module private (`pub(crate)`).
- Re-export `crypto::taproot::Signature` (and `Error`) from
`crate::taproot`
090dad770f Improve string parsing (Tobin C. Harding)
Pull request description:
Currently we implement string parsing for height/time from the `absolute` module but not the `relative` module.
Improve the macros used to implement string parsing and use the new versions to implement string parsing for the height and time types in `relative`.
Done while reviewing data structures in relation to `serde`.
ACKs for top commit:
apoelstra:
ACK 090dad770f
Kixunil:
ACK 090dad770f
Tree-SHA512: bfa88efbaf5dc35755eb46df373a08e223f112860e8a65f58db9fdd77e2c01dc9377da735b33ef58940004fe5fe11690ac09be19591fded2c9fd04cd7d2bdf73
Currently we have `TapSighash` that is used for taproot sighashes but
for non-taproot sighashes we use `hash_types::Sighash`. We can improve
the API by creating a `LegacySighash`, and `SegwitV0Sighash`.
Copy the original `Sighash` macro calls to create the two new types in
the `sighash` module.
While we are at it, put the `TapSighash` and `TapSighashTag` into the
`sighash` module also.
ae2aaaa436 Add `script_pubkey_lens` method (Martin Habovstiak)
cf068d16b0 Implement transaction weight prediction (Martin Habovstiak)
Pull request description:
When creating a transaction one must know the the fee beforehand to set
appropriate amounts for outputs and to know the fee, weight is required.
So far we only had a method on an already-constructed transaction. This
method clearly wasn't helpful when constructing the transaction except
for hacks like temporarily adding an all-zeroes signature.
This change adds a function that can compute the transaction weight
without knowing individual bytes of the scripts, witnesses and other
elements. It only needs to know their sizes.
To make the API less error-prone a special, trivial, type is also added
for computing the lengths of witnesses.
Based on #1627
ACKs for top commit:
apoelstra:
ACK ae2aaaa436
tcharding:
ACK ae2aaaa436
Tree-SHA512: 55376601c2c2826bb0909cc25ff5b65816f0b1a2d57fb2cd8831f3db5382de0f4a364d518b312f0528bb5f44c30f3f74f8d254145eed2bfd65e2332b7c4d7c8b
In some cases people construct the transaction with a dummy fee output
value before calculating the weight. A method to create the iterator
over `script_pubkey` lengths is useful in such cases.
In some scenarios it's useful to create outputs with minimal relayable
value. E.g. outputs designated for fee bumping using CPFP. A method for
this is useful.
This implements a constructor of `TxOut` that computes the minimal
non-dust value from the passed script.
Closes#1459
When creating a transaction one must know the the fee beforehand to set
appropriate amounts for outputs and to know the fee, weight is required.
So far we only had a method on an already-constructed transaction. This
method clearly wasn't helpful when constructing the transaction except
for hacks like temporarily adding an all-zeroes signature.
This change adds a function that can compute the transaction weight
without knowing individual bytes of the scripts, witnesses and other
elements. It only needs to know their sizes.
To make the API less error-prone a special, trivial, type is also added
for computing the lengths of witnesses.
Currently we implement string parsing for height/time from the
`absolute` module but not the `relative` module.
Improve the macros used to implement string parsing and use the new
versions to implement string parsing for the height and time types in
`relative`.
Use of general-purpose integers is often error-prone and annoying. We're
working towards improving it by introducing newtypes.
This adds newtypes for weight and fee rate to make fee computation
easier and more readable. Note however that this dosn't change the type
for individual parts of the transaction since computing the total weight
is not as simple as summing them up and we want to avoid such confusion.
Part of #630
Currently we use a wildcard to export all the hash types in
`hash_types`. We are moving to a world were we only export
normal/standard types from the crate root.
Remove the reexport of the following hash types:
- `FilterHash`
- `FilterHeader`
- `TxMerkleNode`
- `WitnessCommitment`
- `WitnessMerkleNode`
- `XpubIdentifier`
- `Sighash`
Fix: #1541
Remove `FromHex` from hash and script types
- Remove the `FromHex` implementation from hash types and `ScriptBuf`
- Remove the `FromStr` implementation from `ScriptBuf` because it does not
roundtrip with `Display`.
- Implement a method `from_hex` on `ScriptBuf`.
- Implement `FromStr` on hash types using a fixed size array.
This leaves `FromHex` implementations only on `Vec` and fixed size arrays.
The `max_value` and `min_value` functions only exist to be
compatible/uniform with Rust 1.41.1 they will never change and they just
return a constant value. They can therefore be made const functions.
Some of the lock time structs (`Height`, `Time` ect.) are missing
standard constants for min, max ect.
Add standard constants taking into consideration the various locktime
corner cases.
Add `max_value` and `min_value` to be consistent with Rust 1.41.1 (incl.
`Sequence`).
Fix: #1451
The term "final" is an archaic Bitcoin term however it is well used, it
exists in Bitcoin Core code as well as in various bips. To help folks
new to Bitcoin add documentation to the `is_final` method including
historical notes.
In preparation for deprecating the `is_final` method; move the
`enables_absolute_lock_time` method to be directly above the `is_final`
method.
Refactor only, no logic changes.
This makes the code less noisy and is a preparation for changing it to
`const`-based literal. Because of the preparation, places that used
variables to store the hex string were changed to constants.
There are still some instances of `Vec::from_hex` left - where they
won't be changeable to `const` and where `hex!` is unavailable
(integration tests). These may be dealt with later.
See also #1189
02c1cd6291 add some documentation clarifying the locktime ordering shenanigans in #1330 (Andrew Poelstra)
Pull request description:
Updates the CHANGELOG and also the doccomment on `Transaction`.
ACKs for top commit:
tcharding:
ACK 02c1cd6291
Kixunil:
ACK 02c1cd6291
sanket1729:
ACK 02c1cd6291
Tree-SHA512: e2d23a90fb1e53758449fe49a3db7ae1497a260ce7efcade4b50265fa70840db273609019590d9d0a69e1272607a6bcf37924b805b4f09909487eb0c3b91a3cd
This renames `Script` to `ScriptBuf` and adds unsized `Script` modeled
after `PathBuf`/`Path`. The change cleans up the API a bit, especially
all functions that previously accepted `&Script` now accept truly
borrowed version. Some functions that perviously accepted `&[u8]` can
now accept `&Script` because constructing it is no loger costly.
This can be replicated by deleting the `type PackedLockTime = LockTime'
line, and then running
find . -type f | xargs sed -i 's/PackedLockTime/LockTime/g
at the root of the repo.
e00dfa9806 impl FromHexStr for structs with single u32 member (connormullett)
Pull request description:
Closes: #1112
- Adds new trait `FromStrHex` with 2 methods: `from_hex_str` and `from_hex_str_no_prefix`
- Impl new trait on each tuple struct with single u32 member. eg `Time(u32)`
As stated in the issue, grep through codebase with `\(u32\)` and `\(pub u32\)` to see all implementations and verify none were missed.
NonStandardSighashType is an error type and should never be constructed from a hex string. Therefore, it has been omitted from this change.
Tests are somewhat redundant, but cover 4 cases each. 2 happy paths, 1 for each function. 1 case for malformed/invalid hex input, and 1 for calling no_prefix without a prefix
ACKs for top commit:
Kixunil:
ACK e00dfa9806
apoelstra:
ACK e00dfa9806
Tree-SHA512: 221faef7fc1fa8fdb4cba79cfae317a0b63984937c345c6ca2287123a078f38911cdc07db7589a88b7bc6fbecf389e9bcff47952728410510ffcfc1857e0f91f
Adds new module `string` to be later converted to its own
crate. The module currently contains the FromHexStr trait and an error
type to be used for implementing hex parsing on types. This change
also adds implementations of FromHexStr for types with a single u32
member such as `Sequence(pub u32)`. All structs that match the
following regex have been given this implementation
`\(u32\)` and `\(pub u32\)`. All implementations have associated
unit tests matching all possible cases. NonStandardSighashType has
been ommitted from this change as it is an error and should not be
constructed using the methods added in this change.
Adds parse::hex_u32 for future use to be made generic to allow
different sizes of integers to be parsed from hex strings.
The error type FromHexError implements required traits such as
Display and std::error::Error
Recently we (tcharding) do some mechanical improvements to the rustdocs
in the `blockdata` module without considering the content. On review a
bunch of improvements were suggested.
Improve the content of various rustdoc comments in the `blockdata`
module.
Suggested content came from reviewers, all mistakes are my own :)
In some protocols it is preferred to serialize consensus-encodable types
using consensus encoding. E.g. serialize `Transaction` as hex-encoded
string in Json in Bitcoin Core RPC protocol. This change provides
adapter to make this easier.
The adapter allows providing custom byte-to-string encoder for more
exotic cases and provides a hex implementation which should be useful in
majority of the cases.
Should help with #765
1050fe9cae Remove unnecessary borrow (Tobin C. Harding)
3966709336 Use is_none() (Tobin C. Harding)
d192052519 Remove unnecessary dereference (Tobin C. Harding)
624cda07b3 Remove unnecessary casts (Tobin C. Harding)
Pull request description:
Clippy has been updated and new warnings are being triggered in our codebase. This PR does all warnings using nightly since they all looked like reasonable things to fix.
Needed for CI to pass in other open PRs.
ACKs for top commit:
Kixunil:
ACK 1050fe9cae
sanket1729:
ACK 1050fe9cae.
Tree-SHA512: 7dcfb6a72a0aae51b49b417bb94cbe1becb1095d1bf0011921b1834a10f792cfcdeee37993ab9b103bd2dfcc9cd3c26cd7f1bb80b06b0d1aa4aaa454bfb0b3f0
This transaction broke past versions of `rust-bitcoin` and LND so this
adds a test to avoid reintroducing the problem in the future.
See also https://github.com/romanz/electrs/issues/783
Done as part of the effort to flatten the `util` module.
The `sighash` module can stand alone in the crate root, it provides a
discreet set of functionality - the `SighashCache` and associated types.
7d851b42ee Move serde_string_* macros to the serde_utils module (Tobin C. Harding)
53b681b838 Move const_assert to bitcoin_internals (Tobin C. Harding)
5a8a5ff6c9 Move debug_from_display to bitcoin_internals (Tobin C. Harding)
a2f08f2bc6 Improve docs on impl_array_newtype macro (Tobin C. Harding)
771cdde282 Move impl_array_newtype to bitcoin_internals (Tobin C. Harding)
Pull request description:
Move macros out of `internal_macros`, done in an effort to work towards removing the `internal_macros` module since we have `bitcoin_internals` now.
ACKs for top commit:
apoelstra:
ACK 7d851b42ee
Kixunil:
ACK 7d851b42ee
Tree-SHA512: b31b3a5b4d18a2dbe3f358bff62ae6ca4041d432c755e9c45b0241d48903e02c95e79ec72a7478b9d2a53486ce9eef19bfe3b8905aba19036e59c0719f193ce7
In preparation for emptying the `internal_macros` module move the
`serde_string_impl` and `serde_struct_human_string_imp` macros to the
`serde_utils` module.
Rationale: `internal_macros` stuff can go over in the `internals` crate
now that we have one. The serde macros could go over there but we have a
`serde_utils` module that holds code for implementing serde traits,
these two macros are exactly that.
Recently we moved a few types from `transaction` to `sighash`, while
doing so I erroneously annotated code with the `deprecated` attribute
hoping it would give downstream users a gentle upgrade experience. It
turns out `deprecated` only works on functions.
During that same work, we re-exported from the crate root a bunch of
types from the `sighash` module that probably should not have been
re-exported. We are currently trying to create a nice clean API surface,
in an effort to move in the right direction we should remove the
re-exports and just re-export the `sighash` module.
Try to clean up the sighash export mess by doing:
- Remove the re-exports from the `transaction` module
- Remove crate level re-exports of `sighash` module types
- Re-export `sighash` module
Note, this patch is a breaking API change, justified by the fact that
there is no good way to gently lead downstream when moving types since
types cannot be deprecated with the `deprecated` attribute.
Add a new crate `bitcoin-internals` to be used for internal code needed
by multiple soon-to-be-created crates.
Add the `write_err` macro to `bitcoin-internals`, nothing else.
This patch uses a `path` dependency which means `rust-bitcoin` cannot be
released in its current state, will need to be changed once we release
the `bitcoin-internals` crate on `crates.io`.
Create a directory `bitcoin` and move into it the following as is with
no code changes:
- src
- Cargo.toml
- contrib
- test_data
- examples
Then do:
- Add a workspace to the repository root directory.
- Add the newly created `bitcoin` crate to the workspace.
- Exclude `fuzz` and `embedded` crates from the workspace.
- Add a contrib/test.sh script that runs contrib/test.sh in each
sub-crate
- Fix the bitcoin/contrib/test.sh script
Andrew Poelstra
Tobin C. Harding
yancy
TATHAGATA ROY
Martin Habovstiak
Peter Todd
The rustfmt Tyranny
Steven Roose
Harshil Jani
Daniela Brozzoni
connormullett
Noah Lanson
sanket1729
Noah