a9108d3939 Refactor script module (Tobin C. Harding)
Pull request description:
The `script` module is large and unwieldy.
Refactor the `script` module, splitting it up into a tree of modules. Here are a few of the changes and their stated benefits
- Split the two script types out into separate files: Readers of the methods can then tell immediately from the file name which type they are reading.
- Put all the impls for the two script types together: Makes parsing the API easier because one can more quickly see which traits are implemented on what i.e., all the `AsRef` imlps are grouped together.
- Put the impls for the two script types in order, first `Script` then `ScriptBuf`: Makes it easier for us to see if we missed something.
- Put the `Builder` and `Instruction` (and associated) types in their own modules: Some devs find long files hard to navigate, so far there hasn't been too much push back against short files.
- Put tests in a separate file: This idea was recently discussed.
This is only moving code and fixing import statements etc. No other changes to the code.
## Note to reviewers
This PR is impossible to review from the diff because it moves so much code. Perhaps better to look at the resulting `src/blockdata/script/` directory and see if you like it.
#### Motivation
While adding script tagging I was having difficulty navigating the script module.
ACKs for top commit:
apoelstra:
ACK a9108d3939
Kixunil:
ACK a9108d3939
Tree-SHA512: 19123c8cfbdce6c42b322fa75a74073a0114b0ed21bd06ca5727981b3573b74cf05075723b774b92ae2b497e20644fca6e2fac14e30cc44f2802dde5aa567f66
32d2d62e0f Rename from_slice methods to decode (Tobin C. Harding)
Pull request description:
The `TaprootMerkleBranch` and `ControlBlock` both have methods on them called `from_slice` but these methods do more that just basic copy from a slice. `decode` is a more descriptive name.
Deprecate the `from_slice` methods and implement `decode`, on other changes to the logic.
cc sanket1729
ACKs for top commit:
apoelstra:
ACK 32d2d62e0f
Kixunil:
ACK 32d2d62e0f
Tree-SHA512: e8c089545411a214ef9393f65d3990be46983000bd045182cc27dd70b62273bf48ac97adaf89d1e7fc807c72964a01eef176c7685684e8f87a01c219746d6d3d
86f372774b Add '_ back into the BitStreamWriter (Tobin C. Harding)
Pull request description:
Recently we merged `commit 53d4fe66b57c255086def2b5e47afaddee776b75` to fix CI even though a better approach is to use `'_` because it assists reading the code (shows that the bit stream writer is not writing from a reference since its writing a `Copy` type `n`).
Add back in the `'_` (I forget what its called).
ACKs for top commit:
apoelstra:
ACK 86f372774b
Kixunil:
ACK 86f372774b
Tree-SHA512: 2a9989164562dbe7bf133e3aeb090fbff7831bfeefb0ac8431e75b17d57184c4d60ac206578c6ebbcff903a3832502a162027ed9f37e5ed87e42a6bf61efa594
The `script` module is large and unwieldy.
Refactor the `script` module, splitting it up into a tree of modules.
Here are a few of the changes and their stated benefits
- Split the two script types out into separate files: Readers of the
methods can then tell immediately from the file name which type they are
reading.
- Put all the impls for the two script types together: Makes parsing the
API easier because one can more quickly see which traits are implemented
on what i.e., all the `AsRef` imlps are grouped together.
- Put the impls for the two script types in order, first `Script` then
`ScriptBuf`: Makes it easier for us to see if we missed something.
- Put the `Builder` and `Instruction` (and associated) types in their
own modules: Some devs find long files hard to navigate, so far there
hasn't been too much push back against short files.
- Put tests in a separate file: This idea was recently discussed.
This is only moving code and fixing import statements etc. No other
changes to the code.
The `TaprootMerkleBranch` and `ControlBlock` both have methods on them
called `from_slice` but these methods do more that just basic copy from
a slice. `decode` is a more descriptive name.
Deprecate the `from_slice` methods and implement `decode`, on other
changes to the logic.
75b266a129 Improve `sighash` module documentation (Martin Habovstiak)
Pull request description:
"Sighash" is a technical term that newbies in Bitcoin may not know and could get lost when trying to find how to sign a transaction. This change attempts to make it more obvious that this module is needed for signing.
Closes#1463
ACKs for top commit:
tcharding:
ACK 75b266a129
apoelstra:
ACK 75b266a129
Tree-SHA512: 7157566c1639c63ce0fba2832e8e5e846e689d89e24077ed7769b721c5db4613cd7fd8d91464992eb78de74b42912ca877e7182a9c3c9c8848bf94d89767b8cc
b3188bbac3 Add `Transaction` accessors to `SighashCache` (Martin Habovstiak)
7c6854fe02 Use `Borrow` instead of `Deref` in `SighashCache` (Martin Habovstiak)
Pull request description:
This changes the bound from `Deref<Target = Transaction>` to `Borrow<Transaction>` (with respective `mut` changes) and adds accessors.
Closes#1423 (PSBT stuff will be separate issue).
ACKs for top commit:
tcharding:
ACK b3188bbac3
apoelstra:
ACK b3188bbac3
Tree-SHA512: 9db2c5890b26e9eefd483d697b42e84b1d7d3b8676fc39b4f39075c149e12697aa538828a757f9187578a958d72a592bb913f8f5788c93feb273db5370979d99
118a593c89 Implement from arrays for TaprootMerkleBranch (Tobin C. Harding)
Pull request description:
The `TaprootMerkleBranch` contains a vector of `TapNodeHash`s, as such it can trivially be constructed from an array of the same type.
Implement `From` for all array sizes 1 - 128 inclusive.
Fix: #1469
ACKs for top commit:
Kixunil:
ACK 118a593c89
apoelstra:
ACK 118a593c89
Tree-SHA512: dd497abd9143ea8b43485133beaccac9049fb915a95a3422d41c1f99961b59ec95df93efe759aa02f62ba1cf3e1afc4597671f1202ff0fa78eeee8b305d21305
The `TaprootMerkleBranch` contains a vector of `TapNodeHash`s, as such
it can trivially be constructed from an array of the same type.
Implement `From` for all array sizes 1 - 128 inclusive.
"Sighash" is a technical term that newbies in Bitcoin may not know and
could get lost when trying to find how to sign a transaction. This
change attempts to make it more obvious that this module is needed for
signing.
Closes#1463
277e8e96bd Add KeyPair import to rustdoc example (Tobin C. Harding)
Pull request description:
Recently, and bizarrely, a PR merged that broke `cargo test --doc`.
Add an import for `KeyPair` to the `schnorr` rustdoc example.
ACKs for top commit:
apoelstra:
ACK 277e8e96bd
Kixunil:
ACK 277e8e96bd
Tree-SHA512: ad214b668827b35848cc7b260cbd2104a916a82a5a6d242bdc498c62edc9a0e864f4bdb4abcade42924dbaf951223ae80feacbe68d8a4ccb4562d8ead50b23a9
bb612fdafa Set rustv_1_53 in build script (Tobin C. Harding)
Pull request description:
The rust version is supposed to be set by the build script so that users automagically get features matching the toolchain in use. Currently we have a feature in the manifest for `rustv_1_53` instead setting a compiler conditional configuration option in the build script. This causes `cargo +1.41.1 --all-features check` to fail.
## Note
I don't see `rustv_1_46` used anywhere, do we need that still?
ACKs for top commit:
apoelstra:
ACK bb612fdafa
Kixunil:
ACK bb612fdafa
Tree-SHA512: f74195d4ee5a5bc5f209e99d30789df3552cef10aee5ea8b61a5a701b753999c34d04be9fe0321ccee7a8ec14fa5a05e0b454b9dc5f8deddd7b5b8d4f3d7e744
It may be useful to access the transaction stored in `SighashCache`
during signing or afterwards, especially when the transaction is stored
without indirection (to enable long-lived storage).
This change adds the appropriate accessors.
The requirement for a type dereferencing to `Transaction` prevented
storing the cache in long-lived data without resorting to silly
wrappers. Since `Borrow` is implemented both for `T` and for smart
pointers it's a more flexible bound which this change implements.
While this is technically breaking, all usual non-generic code will
continue to work beause smart pointers generally have `Borrow`
implemented.
Currently we use a wildcard to export all the hash types in
`hash_types`. We are moving to a world were we only export
normal/standard types from the crate root.
Remove the reexport of the following hash types:
- `FilterHash`
- `FilterHeader`
- `TxMerkleNode`
- `WitnessCommitment`
- `WitnessMerkleNode`
- `XpubIdentifier`
- `Sighash`
Fix: #1541
f0e4e38844 Add newline in rustdoc (Tobin C. Harding)
Pull request description:
Docs created with the `sha256t_hash_newtype` macro are missing a newline between the doc heading and doc main section.
Note that the strings used span multiple lines and therefor the subsequent lines must be aligned with the start of the line (not indented).
Fix: #1540
ACKs for top commit:
Kixunil:
ACK f0e4e38844
apoelstra:
ACK f0e4e38844
Tree-SHA512: 240c68864da63688c400498903d5cc345bee224dcd3235df0127dcf391c66ee08c487d31fe59f890009c674574810b689d9a53628d07d8cdd46b79bc0ac3eb2b
Docs created with the `sha256t_hash_newtype` macro are missing a newline
between the doc heading and doc main section.
Note that the strings used span multiple lines and therefor the
subsequent lines must be aligned with the start of the line (not
indented).
Fix: #1540
The rust version is supposed to be set by the build script so that users
automagically get features matching the toolchain in use. Currently we
have a feature in the manifest for `rustv_1_53` instead setting a
compiler conditional configuration option in the build script. This
causes `cargo +1.41.1 --all-features check` to fail.
32ca6cc320 Remove hex_from_slice and display Sighash forwards (Tobin C. Harding)
a308e1e2ea Remove FromHex for all types except Vec and array (Tobin C. Harding)
3e70c01826 Manually format a bunch of vecs (Tobin C. Harding)
83e1c40c4d Remove script:: prefix from unambiguous types (Tobin C. Harding)
5ab5c264d2 Use fully qualified path in macro (Tobin C. Harding)
7e85452cd9 hashes: Implement std::error::Error (Tobin C. Harding)
5e3abc5e11 Fix feature gating on unit tests (Tobin C. Harding)
3344cf6be2 Favour $reverse instead of $reversed (Tobin C. Harding)
Pull request description:
This work started out, as the branch name suggests, as an effort to use the `hex_lit` crate. But once I got to this stage it seems that the `hex!` macro we have provides different, useful, functionality than the `hex_lit::hex!` macro (it allows usage with non-consts). So I'm unsure if we want to remove it now.
- Patches 1 - 6 are preparatory clean ups
- Patch 7 reduces usage of `FromHex`, please see git log for full description
- Patch 8 removes `hex_from_slice` and fixes a bug in how we display `Sighash`
ACKs for top commit:
apoelstra:
ACK 32ca6cc320
Kixunil:
ACK 32ca6cc320
Tree-SHA512: 11b45b39ec2fc0f837d7395b5fb86de7cc44641fd51cf7e93394a635e6a8fb1c7ac441a6070d5516dae60e084c04cc6e8b605a5167093f964679e445ef60c271
facaefc49c Add conversions for TweakedKeyPair -> TweakedPublicKey (Tobin C. Harding)
2407f241e4 Remove sep256k1 path from Parity (Tobin C. Harding)
Pull request description:
It is trivially possible to get `TweakedPublicKey` from a `TweakedKeyPair`, add conversion methods for doing so.
Patch 1 is preparatory cleanup. Please note `From` is not implemented because the conversion returns the `Parity` also.
Fix: #1452
ACKs for top commit:
apoelstra:
ACK facaefc49c
Kixunil:
ACK facaefc49c
Tree-SHA512: 597026c481fe2622a625cbeb381cac345af6f49f4a115418b69817345fc3c2140bbdbc5208eae1149d7d171f94c776365d302ffe1f9c01d944e738807db28a89
`Sighash` should be displayed forwards according to BIP143. Currently we
are displaying it backwards (as we do for double SHA256). This is
working because parse using `Vec::from_hex`.
We have the means to parse hex strings directly for hashes, we no longer
need `hex_from_slice`.
BIP143 test vectors display double SHA256 forwards but we display
backwards, this is acceptable because there is no fixed display in the
ecosystem for double SHA256 hashes. In order to overcome this we parse
test vector hex strings with into `Vec` when needed.
Remove `FromHex` from hash and script types
- Remove the `FromHex` implementation from hash types and `ScriptBuf`
- Remove the `FromStr` implementation from `ScriptBuf` because it does not
roundtrip with `Display`.
- Implement a method `from_hex` on `ScriptBuf`.
- Implement `FromStr` on hash types using a fixed size array.
This leaves `FromHex` implementations only on `Vec` and fixed size arrays.
In preparation for modifying some unit test data structures, manually
format the code so it is uniform.
Move elements added to a vec with `vec!` onto a new line so they all
line up and one can better see what fields go where.
Refactor only, no logic changes.
ed6f6d11dd Implement fmt traits for ScriptBuf (Tobin C. Harding)
Pull request description:
We can improve ergonomics of the `script` module by implementing the `fmt` traits on `ScriptBuf`, trivial because we can call through to the `Script` implementations.
Fix: #1585
ACKs for top commit:
Kixunil:
ACK ed6f6d11dd
apoelstra:
ACK ed6f6d11dd
Tree-SHA512: 878a1522af4ed1e10d1d8d60d150e6571008c008b5e5c662c67462f9e09075b4f1fe4e399ed50e98cd7253b6815937c6732cd1ce02b74a5be017d5b8fcdbbd2f
We can improve ergonomics of the `script` module by implementing the
`fmt` traits on `ScriptBuf`, trivial because we can call through to the
`Script` implementations.
The `max_value` and `min_value` functions only exist to be
compatible/uniform with Rust 1.41.1 they will never change and they just
return a constant value. They can therefore be made const functions.
Recently we merged `commit 53d4fe66b57c255086def2b5e47afaddee776b75` to
fix CI even though a better approach is to use `'_` because it assists
reading the code (shows that the bit stream writer is not writing from a
reference since its writing a `Copy` type `n`).
Add back in the `'_` (I forget what its called).
877f9af364 Add new hex parse error variant (Tobin C. Harding)
Pull request description:
Recently we used an error type that holds only one expected hex string length when parsing but for `PublicKey`s we have two (66 and 130). Add a new error variant to express the error. Requires adding a variant to `bip32` for the same thing.
Fix: #1281
ACKs for top commit:
Kixunil:
ACK 877f9af364
apoelstra:
ACK 877f9af364
Tree-SHA512: c1ca493ee30418bd82bc326b35c18731260e4217c371f37301a73c64f9a6631163801acc217c6c2c7b14f632a2ad5043174266c1b4fdce127698e68ab8494f20
3c0598b399 Add standard constants to lock times (Tobin C. Harding)
Pull request description:
Some of the lock time structs (`Height`, `Time` ect.) are missing standard constants for min, max ect.
Add standard constants taking into consideration the various locktime corner cases.
Add `max_value` and `min_value` to be consistent with Rust 1.41.1 (incl. `Sequence`).
Fix: #1451
This PR is not complex in itself but **locktimes are notoriously complex, please wait for 3 acks before merging** - and ack'ing makes no guarantee that reviewer got all corner cases :)
There is no rush on this one, apoelstra, Kixunil, sanket1729 please just review when your brain is fresh.
ACKs for top commit:
apoelstra:
ACK 3c0598b399
Kixunil:
ACK 3c0598b399
Tree-SHA512: aa3d112db83b4785edb0a7a517cc335ded59f5967eb39b8979a6d68f9bba4644a27e5ca400fcabf368a1f8c0eecdef0b87b1011933ac7fd96b467b8501533203
Recently we used an error type that holds only one expected hex string
length when parsing but for `PublicKey`s we have two (66 and 130). Add a
new error variant to express the error. Requires adding a variant to
`bip32` for the same thing.
Fix: #1281
Some of the lock time structs (`Height`, `Time` ect.) are missing
standard constants for min, max ect.
Add standard constants taking into consideration the various locktime
corner cases.
Add `max_value` and `min_value` to be consistent with Rust 1.41.1 (incl.
`Sequence`).
Fix: #1451
a762a89b48 Add documentation to Sequence::is_final (Tobin C. Harding)
b1490a26ea Move enables_absolute_lock_time method (Tobin C. Harding)
Pull request description:
The term "final" is an archaic Bitcoin term however it is well used, it exists in Bitcoin Core code as well as in various bips. To help folks new to Bitcoin add documentation to the `is_final` method including historical notes.
Note, this does _not_ deprecate `is_final` - while writing the notes I found the term "final" in enough official places that I think its fair game to keep the term, some things people just have to learn, we can definitely help with that learning though.
Fix: #1198
ACKs for top commit:
Kixunil:
ACK a762a89b48
apoelstra:
ACK a762a89b48
Tree-SHA512: 895fbdce90223d90c0a68fb1e3d6b7aada4a3606d1294ea4df1f4194681a79d970b0434e7bb078f6d5cbf413b3550e72560d6d5cf811a5a959adf53f7f778ab2
8c0e5213d3 Delegate debug for ScriptBuf to Script (Tobin C. Harding)
Pull request description:
Currently the derived implementation of `Debug` for `ScriptBuf` prints the inner vector of u8s as integers, this is ugly and hard to read. The `Script` implementation of `Debug` prints the script opcodes and data as hex, we can just delegate to it.
With this applied we get debug output of form:
Script(OP_DUP OP_HASH160 OP_PUSHBYTES_20 3bde42dbee7e4dbe6a21b2d50ce2f0167faa8159 OP_EQUALVERIFY OP_CHECKSIG)
Fix: #1516
ACKs for top commit:
Kixunil:
ACK 8c0e5213d3
apoelstra:
ACK 8c0e5213d3
Tree-SHA512: ca07d9fb191f4e0379cbd96b2944e6881094a8334d39b97209b6bf452a3c15d4aede53b9c88176b9b7667b7a539d47897940bc561dc9f8cd83ce1990a08047e1
1d3d5a9c5b Take Into<secp256k1::PublicKey> in PublicKey constructors (Tobin C. Harding)
b13a76407b keys: Clean up test imports (Tobin C. Harding)
Pull request description:
We can make the API more ergonomic by taking a generic argument that implements `Into<secp256k1::PublicKey>` in the `bitcoin::PublicKey` constructors.
The only thing than this is useful for is passing in `KeyPair` and the `From` implementation already exists. Add a unit test to verify.
Fix: #1453
## Note
As per the discussion in #1453 I checked secp and bitcoin for all keys that can be converted using `From` and it turns out its only `KeyPair` which already has `From` impls - good rust-bitcoin devs :)
ACKs for top commit:
Kixunil:
ACK 1d3d5a9c5b
apoelstra:
ACK 1d3d5a9c5b
Tree-SHA512: b5e5272561de15cdcfb15913aa5d42ddc96bf2fd5835068a5a9aa0274074ffa698ec9e81707f102b7d1b244f1abd0fdbd0eb4b6b505c84c3d5719dcb01d46efb
49e8b8da32 Use write_all for sighash encoding (Tobin C. Harding)
Pull request description:
From BIP143:
> If sighash type is SINGLE and the input index is smaller than the number of outputs, hashOutputs is the double SHA256 of the output amount with scriptPubKey of the same index as the input;
Currently we are using a `Sighash` which wraps double sha256 so while technically correct this means we are relying on `Sighash` to implement `Encodable`. We can remove this requirement by directly using the `sha256d::Hash` type to hash the outputs data.
Fix: #1549
ACKs for top commit:
Kixunil:
ACK 49e8b8da32
apoelstra:
ACK 49e8b8da32
Tree-SHA512: 8dd0037245a7cf180ba8a6eceeadad912d4adc14fc3f49df9008856de262624666d7d575195eea4868b2a5252dc565590e6be78471053b5e6367f3d2363310e8
e7bbfd3913 Improve Psbt error handling (DanGould)
Pull request description:
## Separate `encode::Error` and `psbt::Error` recursive dependency
This initial work attempts to fix#837's first 2 points
> - The current psbt::serialize::Deserialize has an error type of consensus::encode::Error. I think we should cleanly separate consensus encoding errors from application-level encoding errors like psbt.
> - There is a recursive dependence between encode::Error and psbt::Error which would need to be cleanly dissected and separated so that there is no dependence or only one-way dependence.
## Better `ParseError(String)` types
arturomf94 how compatible do your #1310 changes look to address #837's third point with this design?
> - There are a lot ParseError(String) messages that could use a better type to downflow the information.
I think your prior art would completely address this issue now.
## On handling `io::Error` with an associated error
`encode::Error` has an `Io` variant. now that `Psbt::deserialize` returns `psbt::Error` and produces an `io::Error`, we need an `Io` variant on `psbt::Error`. Except that doing so breaks `#[derive(Eq)]` and lots of tests for `psbt::Error`.
Kixunil, I'm trying to understand your feedback regarding a solution to this problem.
> I believe that the best error untangling would be to make decodable error associated.
> I meant having associated `Error` type at `Decodable` trait. Encoding should only fail if the writer fails so we should have `io::Error` there (at least until we have something like `genio`).
>
> > [it] is a problem to instantiate consensus::encode::Error in [the psbt] module for `io::Error`?
>
> It certainly does look strange. Maybe we should have this shared type:
>
> ```rust
> /// Error used when reading or decoding fails.
> pub enum ReadError<Io, Decode> {
> /// Reading failed
> Io(Io),
> /// Decoding failed
> Decode(Decode), // consensus and PSBT error here
> }
> ```
>
> However this one will be annoying to use with `?` :( We could have `ResultExt` to provide `decode()` and `io()` methods to make it easier.
>
> If that's not acceptable then I think deduplicated IO error is better.
Kixunil didn't we just get rid of Psbt as `Decodable`? Would this make more sense to have as an error associated with `Deserialize`? Or did we do the opposite of what we should have by making Psbt only `Serialize`/`Deserialize` because of #934, where only consensus objects are allowed to be `Decodable`? I wonder if we prioritized that strict categorization and are stuck with worth machinery because of it. My goal with #988 was to get to a point where we could address #837 and ultimately implement PSBTv2.
ACKs for top commit:
tcharding:
ACK e7bbfd3913
apoelstra:
ACK e7bbfd3913
Tree-SHA512: 32975594fde42727ea9030f46570a1403ae1a108570ab115519ebeddc28938f141e2134b04d6b29ce94817ed776c13815dea5647c463e4a13b47ba55f4e7858a
44d3ec487d Rename Payload::as_bytes to inner_prog_as_bytes (sanket1729)
a446df583c Make Payload non-exhaustive (sanket1729)
6ebc9de252 Introduce WitnessProgram struct and cleanup Address validity invariants (sanket1729)
41652caf05 Introduce is_spend_standard method (sanket1729)
Pull request description:
Fixes#1561.
Highlights:
- Segwitv0 programs with lengths apart from 20 or 32 are invalid `Address` struct. Such Addresses are useless and we should not parse/create them.
- Renamed `is_standard` to `is_spend_standard`.
ACKs for top commit:
apoelstra:
ACK 44d3ec487d
tcharding:
ACK 44d3ec487d
Tree-SHA512: 1ee36f7ea25c65619ddf7d643d025690096876843dbe6fbdf877ce23e88049d79b0bbd78cee6cf4b415bca028b3634bb70c9f52d1098bd90558e6ba7f8731332
Tobin C. Harding
Andrew Poelstra
Martin Habovstiak
DanGould
Arturo Marquez
sanket1729