d6296cd3d1 Remove usage of hex::test_hex_unwrap (Tobin C. Harding)
37035e20e8 Simplify and improve transaction benchmarks (Tobin C. Harding)
Pull request description:
We have the `hex_lit` dependency for converting a hex string literal to an array.
Currently we have a `test_hex_unwrap` macro in the `hex v0.3.0` release but not on either `master`
or the upcoming `v1.0.0-alpha.0` release. This is making PRs around releasing and depending on the
release more noisy than required.
Introduce a `test_hex_unwrap` macro in internals for usage when the input is not a string literal.
Use `hex_lit::hex` where possible (often needing an additional call to
ACKs for top commit:
apoelstra:
ACK d6296cd3d1989cf28d67a5329ad60da4f814ba92; successfully ran local tests
Kixunil:
ACK d6296cd3d1
Tree-SHA512: eab3573f6b7fee408ae11821b77e56cbaddf7cc4540bdc31ed7ef9eb3f25987f50e484f1553aaaa9709367e614eb77ed36250875d0faf5a51ab3fe709d4d4054
We have the `hex_lit` dependency for converting a hex string literal
to an array.
Currently we have a `test_hex_unwrap` macro in the `hex v0.3.0` release
but not on either `master` or the upcoming `v1.0.0-alpha.0` release.
This is making PRs around releasing and depending on the release more
noisy than required.
Use `hex_lit::hex` where possible (often needing an additional call to
`to_vec()`) and where not possible use `Vec::from_hex`.
492073f288 Strengthen the type of `taproot_control_block()` (Martin Habovstiak)
e8a42d5851 Unify/reduce usage of `unsafe` (Martin Habovstiak)
d42364bd9d Swap around the fields in `Address` (Martin Habovstiak)
7a115e3cf1 Make `Address` obey sanity rules (Martin Habovstiak)
bc6da1fe07 Swap around the fields in `sha256t::Hash` (Martin Habovstiak)
8ee088df74 Make `sha256t` obey sanity rules (Martin Habovstiak)
Pull request description:
Well, I thought this PR will be just the last commit... 😅
Anyway, this implements a bunch of changes to allow returning `ControlBlock` from `Witness` method(s). One cool side effect is that this PR also reduces the number of `unsafe` blocks.
ACKs for top commit:
apoelstra:
ACK 492073f28876406f8fe5a07a8a2495c8e0ba1fb3; successfully ran local tests
Tree-SHA512: 11979517cc310abf25644fc93a75deccacae66af8ba2d9b4011fdc3f414b15fac7e748399c7eef492ca850c11b7aacc3f24ec46fccf95e6d57a400212979637e
The type returned by `Witness::taproot_control_block()` was just `&[u8]`
which wasn't very nice since users then had to manually decode it which
so far also required allocation. Thanks to previous improvements to
`ControlBlock` it is now possible to return a `ControlBlock` type
directly.
To avoid expensive checks, this change adds a new type
`SerializedXOnlyPublicKey` which is a wrapper around `[u8; 32]` that is
used in `ControlBlock` if complete checking is undesirable. It is then
used in the `ControlBlock` returned from
`Witness::taproot_control_block`. Users can still conveniently validate
the key using `to_validated` method.
It then uses this type in the recently-added `P2TrSpend` type. As a side
effect this checks more properties of `Witness` when calling unrelated
methods on `Witness`. From correctness perspective this should be OK: a
witness obtained from a verified source will be correct anyway and, if
these checks were done by the caller, they can be removed.
From performance perspective, if the `Witness` was obtained from a
verified source (e.g. using Bitcoin Core RPC) these checks are wasted
CPU time. But they shouldn't be too expensive, we already avoid
`secp256k1` overhead and, given that they always succeed in such case,
they should be easy to branch-predict.
The `Witness`-related tests were constructing `Witness` in
over-complicated way by serializing `Vec<Vec<u8>>` and then
deserializing `Witness` even though they were not supposed to test
serialization but Taproot accessor methods. This was difficult to
understand and maintain.
This change simplifies them to just construct the `Witness` from array
of `Vec<u8>`s using the recently-added constructors. Note that we
already have serialization tests written separately so we're not losing
meaningful coverage here.
`TaprootMerkleBranchBuf` being a vec introduced intermediate allocation
when creating or decoding `Witness`. However the representation on the
wire is the same as in-memory (aside from `#[repr(transparent)]`) so
this allocation wasn't really needed.
This commit introduces `TaprootMerkleBranch` type which is unsized and
can be used in place of `TaprootMerkleBranchBuf` within `ControlBlock`.
Aside from removing the intermediate allocation, this improves the API a
bit: the conversion from array to other type is no longer needed because
it's performed by `ControlBlock` in its methods. Thus, consumers who
have an array can simply set it as `merkle_branch` field and then encode
the `ControlBlock` into witness. A convenience method is also provided
to push the `ControlBlock` along with other parts at the end of the
`Witness`.
Enhance Witness struct element access methods:
- Rename `nth()` to `get()` for clearer slice-like element retrieval
- Introduce `get_back()` method for flexible reverse indexing
- Remove redundant `second_to_last()` and `third_to_last()` methods
- Add `#[track_caller]` to index implementation for better error tracking
- Update all references to use new method names
- Improve documentation with usage examples
The changes provide a more intuitive and consistent approach to
accessing witness elements.
We already have `tapscript` method on `Witness` which is broken because
it doesn't check that the leaf script is a tapscript, however that
behavior might have been intended by some consumers who want to inspect
the script independent of the version. To resolve the confusion, we're
going to add a new method that returns both the leaf script and, to
avoid forgetting version check, also the leaf version.
This doesn't touch the `tapscript` method yet to make backporting of
this commit easier. It's also worth noting that leaf script is often
used together with version. To make passing them around easier it'd be
helpful to use a separate type. Thus this also adds a public POD type
containing the script and the version. In anticipation of if being
usable in different APIs it's also generic over the script type.
Similarly to the `tapscript` method, this also only adds the type and
doesn't change other functions to use it yet. Only the newly added
`taproot_leaf_script` method uses it now.
This is a part of #4073
The previous commit fixed a bug when `taproot_control_block` returned
`Some` on key-spends. This adds a test case for it which succeeds when
applied after the previous commit and fails if applied before it.
The `taproot_control_block` did not properly detect whether it deals
with script spend or key spend. As a result, if key spend with annex was
used it'd return the first element (the signature) as if it was a
control block.
Further, the conditions identifying which kind of spend it was were
repeated multiple times but behaved subtly differently making only
`taproot_control_block` buggy but the other places confusing.
To resolve these issues this change adds a `P2TrSpend` enum that
represents a parsed witness and has a single method doing all the
parsing. The other methods can then be trivially implemented by matching
on that type. This way only one place needs to be verified and the
parsing code is more readable since it uses one big `match` to handle
all possibilities.
The downside of this is a potential perf impact if the parsing code
doesn't get inlined since the common parsing code has to shuffle around
data that the caller is not intersted in. I don't think this will be a
problem but if it will I suppose it will be solvable (e.g. by using
`#[inline(always)]`).
The enum also looks somewhat nice and perhaps downstream consumers could
make use of it. This change does not expose it yet but is written such
that after exposing it the API would be (mostly) idiomatic.
Closes#4097
There is a loose convention in Rust to not use `test_` prefix. The
reason being that `cargo test` outputs 'test <test name>' using the
prefix makes the output stutter.
This patch smells a bit like code-churn but having the prefix in some
places and not others is confusing to new contributors and is leading me
to explain this many times now. Lets just fix it.
Remove the prefix unless doing so breaks the code.
Add a unit test that verifies we correctly remove the annex when getting
the tapscript from the witness stack.
Unit test written by Casey, pulled out of #3599.
Co-developed-by: Casey Rodarmor <casey@rodarmor.com>
In #2646 we introduced a bug in the taproot witness stack getter
functions, of which we have three:
- `tapscript`
- `taproot_control_block`
- `taproot_annex`
Each returns `Some` if a possible bytes slice is found (with no other
guarantees).
Use `taproot_annex` combined with getters from `primitives` to implement
the other two getters. This simplifies the code and fixes the bug.
Add an additional getter to `primitives` `Witness::third_from_last`.
Fix: #3598
There is a range of different wordings used in the docs of constructor
type functions.
Change all to start with `Constructs a new` or `Constructs an empty`.
In functions that act like constructors there is a mixture of the usage
of `creates` and `constructs`.
Replace all occurrences of `creates` with `constructs` in the first line
of docs of constructor like functions.
This has been fixed and we use nightly to lint so we have access to the
merged fix.
Removing the attribute uncovers a bunch of real lint warnings, fix
them while we are at it.
During this release cycle we deprecated `to_vec` in favour of
`to_bytes`, we have since reversed our position on the name.
Remove the deprecation of `to_bytes` from the three types that had it
and use `to_vec`.
Up until recently we were using wildcard re-exports for types moved to
`units` and `primitives`. We have decided against doing so in favour of
explicit re-exports.
Audit `units` and `primitives` using `git grep 'pub enum'` (and
`struct`) and explicitly re-export all types.
Remove all wildcards except for the re-exports from `opcodes`, there are
too many opcodes, explicitly re-exporting them does not aid clarity.
bbffa3db43 Remove the IO error from DecodeError (Tobin C. Harding)
713196be0d Return DeserError from encode::deserialize (Tobin C. Harding)
33566ac58c Split encode::Error into two parts (Tobin C. Harding)
b04142c745 Add encode::Error::MissingData variant (Tobin C. Harding)
5a42ef2850 Do not manually map IO error (Tobin C. Harding)
efd7f9f06c Add error constructor parse_failed_error (Tobin C. Harding)
ebfef3f114 Return generic error as Some (Tobin C. Harding)
a6254212dc Move consensus error code to submodule (Tobin C. Harding)
Pull request description:
The `consensus::deserialize` and `consensus::deserilaize_partial` functions should not return an I/O error. Doing so causes various other error types to include an `io::Error` and the `io::Error` is an annoying type to work with.
This PR is a bunch of steps, and it took me a good while with quite a bit of backtracking to get here. As such you may want to review the final state before looking at each patch.
The `consensus` errors can be further cleaned up but I'd prefer not to spend more time on this unless it has some chance of merging.
ACKs for top commit:
apoelstra:
ACK bbffa3db43802b30d23259c0372f16a877a0ef8b; successfully ran local tests
Tree-SHA512: 522fdd29638a214cb7fcee29dd3b9f5c846f041fba087a56a91b83e6d85f033cbed95f659dc4321cd4596943ff233bdd184cdfbfcc787fe89172bb93aa4ab186
The `consensus::encode::Error` contains an IO error but reading from a
buffer only ever errors for EOF. We converted all instances of EOF to
`MissingData` already so now we can split the IO error apart from the
actual encoding errors variants.
The extension traits are temporary just while we try to stabalize
`primitives`, they are not intended to be implemented by downstream.
Seal the extension traits so that downstream crates cannot implement
them.
Fix: #3231
Move the `Witness` over to `primitives` leaving behind any method that
takes or returns a `Script` or a signature.
Includes addition of a feature gate to unit test.
In preparation for moving the `Witness` to `primitives` we need a way to
construct the `Witness` when decoding. In order to maintain the current
performance and not introduce additional allocations we need to be able
to construct a `Witness` from the content buffer, this leaks the
implementation details of `Witness`.
Add a clearly marked unstable constructor to create a `Witness` from
parts. This introduces the concept of `foo__unstable` function names;
add a section to the README describing unstable functions and semver
guarantees.
Recently we deprecated `to_vec` in favour of `to_bytes` however we
continued to use `to_vec` in a few places. This wasn't noticed because
of our usage of `TBD` in the `deprecated` attribute.
Use `to_bytes` instead of `to_vec`.
We would like to move the `Witness` to `primitives` however in the
`Encodable` implementation we are currently accessing the private
`content` field.
Instead of accessing `content` we can iterate over the witness elements
and write each individually, this has the same result but does a bunch
of additional calls to `Write::write_all` (via `emit_slice`).
This patch effects performance negatively but makes no changes to the
encoding.
The `Witness::push_slice` function is called by `Witness::push` after
calling `as_ref`, hence is equivalent for all types that implement
`AsRef<[u8]>`. Also, `push_slice` is a private method on `Witness`.
In preparation for moving `Witness` over to `primitives` stop using
`push_slice` in favour of `push`.
Internal change only.
In preparation for moving the `Witness` oven to `primitives` use the
`len` function instead of accessing the `witness_elements` field.
No logic change, `Witness::len()` returns `witness_elements`.
5fab6b178f Rename iter len unit test (Tobin C. Harding)
f6a74ef4af Refactor the serde Witness unit tests (Tobin C. Harding)
9860453b5b Improve Witness consensus encode unit test (Tobin C. Harding)
7e2899d310 Improve Witness::push unit test (Tobin C. Harding)
fe967279e5 Improve witness unit tests for single empty element (Tobin C. Harding)
Pull request description:
In preparation for moving the `Witness` type over to `primitives` refactor and improve all the unit tests that will be moved, do not touch the ones that will stay behind.
The first five patches are from #3406, the last is just a re-name of the test function I tried to refactor in ac6fe3a881
ACKs for top commit:
apoelstra:
ACK 5fab6b178f successfully ran local tests
Tree-SHA512: bc00f81e3c5cc92ae58dd2fc876d368a487ae6c08cc0735d7227c3a89287e321dbfb5b571b951d0616af0ec7cf9a0ea2d0e724645b1c419933a212ece80a0fbf
Rust convention is to not use `test_` prefix on unit tests. Also this
unit test is testing that the `ExactSizedIterator` trait is implemented
and working.
Re-name unit test to `exact_sized_iterator`.
In preparation for moving unit tests to `primitives` give the serde
tests some love by doing:
- Split them up to do one thing only
- Round trip arbitrary witness
- Use better names
Make an effort to clean up the encoding unit test, by doing:
- Remove element accessor assertions (tested already above)
- Add roundtrip encoding assertion
The current unit test is incorrect, the indices field of a witness with
a single element starts at 1 because 0 is encode as a single
byte (compact encoded integer).
Fix the debug test and add a test that pushes an empty slice.
At some stage we named the compact encoding `VarInt` (which makes sense
because the compact size encoding is a variable length integer encoding).
However it turns out the term "varint" is used in Core for a different
encoding so this may lead to confusion.
While we fix this naming thing observe also that the `VarInt` type is
unnecessarily complicated, all we need to be able to do is encode and
decode integers in compact form as specified by Core. We can do this
simply by extending our `WriteExt` and `ReadExt` traits.
Add `emit_compact_size` and `read_compact_size` to emit and read compact
endcodings respectively.
Includes addition of `internals::compact_size::encoded_size_const`.
Patch originally written by Steven, Tobin cherry-picked and did a bunch
of impovements after the varint vs compact_size thing (#1016).
ref: https://en.bitcoin.it/wiki/Protocol_documentation#Variable_length_integer
Co-developed-by: Tobin C. Harding <me@tobin.cc>
We would like to add a `emit_varint` function, however doing so requires
that we can get access to the length of a slice when we are encoding it
so we can use `emit_slice` to implement `emit_varint`. It would be
easier to do so if `emit_slice` returned the length of the slice.
In preparation for adding `emit_varint` (and removing the `VarInt` type)
return the encoded length of a slice from `WriteExt::emit_slice`.
(Patch originally written by Steven, cherry-pick and patch description
written by Tobin.)
Co-developed-by: Tobin C. Harding <me@tobin.cc>
d65de7c7de Introduce and use new compact_size module (Tobin C. Harding)
Pull request description:
We would like to move the witness module to `primitives` but there is a bunch of usage of `VarInt`.
Introduce a module that does the encoding and decoding instead, note that while the functionality is internal decoding returns an error which may one day end up in the public API. So put the module in `primitives` and make it public.
Adds the module to `primitives`, adds a public `MAX_ENCODABLE_SIZE` variable that is commented with an issue link.
https://github.com/rust-bitcoin/rust-bitcoin/issues/3264
ACKs for top commit:
apoelstra:
ACK d65de7c7de successfully ran local tests
Kixunil:
ACK d65de7c7de
Tree-SHA512: d9483c29b2b324e27460564a23f4639dde4037e6e773f4356216b02ebdea893a6361c342002b8e93a54de47b71ac69369431554f8cd0a2522fc451bf8493c81c
We would like to move the witness module to `primitives` but there is
a bunch of usage of `VarInt`.
Introduce a module that does the encoding and decoding instead, this
code is internal so put it in `internals`.
Note we add an unused public `MAX_ENCODABLE_SIZE` variable that is
commented with an issue link. Done like this because its quite
important that we see to it and it makes it clear that we are not and
we know about it.
https://github.com/rust-bitcoin/rust-bitcoin/issues/3264
The `Witness` struct is non-trivial, in particular it is not immediately
obvious where and when the compact size encode value for each witness
element is stored.
Make an effort to improve the docs on `Witness` in relation to the
compact size encoded length of each witness element.
Fmt Bot
merge-script
Jamil Lambert
Tobin C. Harding
Martin Habovstiak
Erick Cestari
Steven Roose
Shing Him Ng