96e0e720fd feat(bip158): compute canonical filter hash (Rob N)
Pull request description:
From [BIP-157](https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki#filter-headers)
> The canonical hash of a block filter is the double-SHA256 of the serialized filter.
If a user forgets the "double" in double-SHA256 they will be computing a nonsensical filter hash when this is easily handled by the API.
ACKs for top commit:
Kixunil:
ACK 96e0e720fd
tcharding:
ACK 96e0e720fd
Tree-SHA512: 5fc0b1632e2327adacbd0ab56b4cd7edce0774f8be2f819782519c51b9691a748f4b3c01887f3bf1146dee49a35f9dfac833f53ae69ee7a53858bd2cedcec01a
c97389596b Remove stale docs from sha256t_hash_newtype (Tobin C. Harding)
39f7dcb816 Reduce API surface of tagged wrapped hash types (Tobin C. Harding)
Pull request description:
Recently we made it so that wrapper types created with `hash_newtype` were not general purpose hash types i.e., one could not easily hash arbitrary data into them. We would like to do the same for tagged wrapped hash types.
In `hashes` do:
- Create a new macro `sha256_tag` that does just the tag/engine stuff out of the `sha256t_hash_newtype` macro.
- Deprecate the `sha256t_hash_newtype` macro.
In `bitcoin` do:
- Use a combination of `sha256_tag` and `hash_newtype` to create tagged wrapped hash types.
Note that we do not add private helper functions `engine` and `from_engine` to the tagged wrapper types as we do for legacy/segwit in `sighash`. Can be done later if wanted/needed.
Fix: #3135
ACKs for top commit:
Kixunil:
ACK c97389596b
apoelstra:
ACK c97389596b successfully ran local tests
Tree-SHA512: d937a8eac1a77298231f946f9dfbc2f7739af8da00f2075b0b54803b4111c0cec810bc6564515153769193056cf102a9c954e216664f055b249d4a6153b14bca
Recently we made it so that wrapper types created with `hash_newtype`
were not general purpose hash types i.e., one could not easily hash
arbitrary data into them. We would like to do the same for tagged
wrapped hash types.
In `hashes` do:
- Create a new macro `sha256t_tag` that does just the tag/engine stuff
out of the `sha256t_hash_newtype` macro.
- Deprecate the `sha256t_hash_newtype` macro.
In `bitcoin` do:
- Use a combination of `sha256t_tag` and `hash_newtype` to create tagged
wrapped hash types.
Note that we do not add private helper functions `engine` and
`from_engine` to the tagged wrapper types as we do for legacy/segwit in
`sighash`. Can be done later if wanted/needed.
c9053511b2 Remove misleading version metadata (Martin Habovstiak)
Pull request description:
The metadata in dependency specification was misleading because the version was not guaranteed to be the same anyway this was correctly linted but nobody so far cared to fix it. This change fixes it and adds a hint how to get the real version since some people seem still confused about how these things work.
ACKs for top commit:
apoelstra:
ACK c9053511b2 successfully ran local tests
tcharding:
ACK c9053511b2
Tree-SHA512: 99d849b1c1b202a429bc7eb25394ff84ba720b3ca6e73bce615f767e0924f5ef63a87553214f0f4fadd3b68ea84246ab8284d75df56cbbe794da7dea0d169dd9
2bb90b8203 Introduce two extensions traits for ScriptBuf (Tobin C. Harding)
ae0a5bd64a Run cargo fmt (Tobin C. Harding)
3fdc574851 Add temporary script buf modules (Tobin C. Harding)
4ff5d6886b Add private ScriptBufAsVec type (Tobin C. Harding)
c81fb93359 Make push_slice_no_opt pub(crate) (Tobin C. Harding)
1001a33f19 Add second ScriptBuf impl block (Tobin C. Harding)
3625d74e8b Make pub in crate functions pub crate (Tobin C. Harding)
b368384317 Separate ScriptBuf POD methods (Tobin C. Harding)
Pull request description:
Similar to #3155 but for `ScriptBuf`, however it is a little more involved.
Note:
- the change to use `impl` syntax (and addition of #3179)
- mad trickery of `ScriptBufAsVec` (props to Kix)
- widening of scope of private functions
Onward and upward!
ACKs for top commit:
Kixunil:
ACK 2bb90b8203
apoelstra:
ACK 2bb90b8203 successfully ran local tests
Tree-SHA512: 7209d8dc436e52b23e1dbfd9db8432df225ebdb701f465e4d1b55328e22988c98a0f28efdf2a8b3edbafc754354d718ab36bd2f5b1621d12e061b2dadaf49a05
The metadata in dependency specification was misleading because the
version was not guaranteed to be the same anyway this was correctly
linted but nobody so far cared to fix it. This change fixes it and adds
a hint how to get the real version for people who are mistakenly
investigating wrong file.
ad34a98c61 Refactor Rust version checking (Martin Habovstiak)
7d5ce89dad Fix type ambiguity in IO tests (Martin Habovstiak)
Pull request description:
Conditional compilation depending on Rust version using `cfg` had the disadvantage that we had to write the same code multiple times, compile it multiple times, execute it multiple times, update it multiple times... Apart from obvious maintenance issues the build script wasn't generating the list of allowed `cfg`s so those had to be maintained manually in `Cargo.toml`. This was fixable by printing an appropriate line but it's best to do it together with the other changes.
Because we cannot export `cfg` flags from a crate to different crates we take a completely different approach: we define a macro called `rust_version` that takes a very naturally looking condition such as `if >= 1.70 {}`. This macro is auto-generated so that it produces different results based on the compiler version - it either expands to first block or the second block (after `else`).
This way, the other crates can simply call the macro when needed.
Unfortunately some minimal maintenance is still needed: to update the max version number when a newer version is used. (Note that code will still work with higher versions, it only limits which conditions can be used in downstream code.) This can be automated with the pin update script or we could just put the pin file into the `internals` directory and read the value from there. Not automating isn't terrible either since anyone adding a cfg with higher version will see a nice error about unknown version of Rust and can update it manually.
Because this changes syntax to a more naturally looking version number, as a side effect the `cond_const` macro could be also updated to use the new macro under the hood, providing much nicer experience - it is no longer needed to provide human-readable version of the version string to put in the note about `const`ness requiring a newer version. As such the note is now always there using a single source of truth.
It's also a great moment to introduce this change right now since there's currently no conditional compilation used in `bitcoin` crate making the changes minimal. However it is not yet added to `bitcoin-io` since `bitcoin-io` is not depending on `internals`. It might be a reason to start depending on it but that's for later discussion.
ACKs for top commit:
apoelstra:
ACK ad34a98c61 successfully ran local tests
tcharding:
ACK ad34a98c61
Tree-SHA512: 7a82017fc51ba1b473ca638c7bdbf5c893da0a78c70ea8f1a0241049e7874592fad328abd60b3e09a00439b771e7cfc5ebad5e874314d15a48271ec6cb2d7bb7
In preparation for moving the `ScritpBuf` type to `primitives` add a
public and private extension trait for the functions we want to leave
here in `bitcoin`.
Note, includes a change to the `difine_extension_trait` metavariable
used on `$gent` from `ident` to `path` to support the generic
`AsRef<PushBytes>`.
Conditional compilation depending on Rust version using `cfg` had the
disadvantage that we had to write the same code multiple times, compile
it multiple times, execute it multiple times, update it multiple
times... Apart from obvious maintenance issues the build script wasn't
generating the list of allowed `cfg`s so those had to be maintained
manually in `Cargo.toml`. This was fixable by printing an appropriate
line but it's best to do it together with the other changes.
Because we cannot export `cfg` flags from a crate to different crates we
take a completely different approach: we define a macro called
`rust_version` that takes a very naturally looking condition such as
`if >= 1.70 {}`. This macro is auto-generated so that it produces
different results based on the compiler version - it either expands to
first block or the second block (after `else`).
This way, the other crates can simply call the macro when needed.
Unfortunately some minimal maintenance is still needed: to update the
max version number when a newer version is used. (Note that code will
still work with higher versions, it only limits which conditions can be
used in downstream code.) This can be automated with the pin update
script or we could just put the pin file into the `internals` directory
and read the value from there. Not automating isn't terrible either
since anyone adding a cfg with higher version will see a nice error
about unknown version of Rust and can update it manually.
Because this changes syntax to a more naturally looking version number,
as a side effect the `cond_const` macro could be also updated to use the
new macro under the hood, providing much nicer experience - it is no
longer needed to provide human-readable version of the version string to
put in the note about `const`ness requiring a newer version. As such the
note is now always there using a single source of truth.
It's also a great moment to introduce this change right now since
there's currently no conditional compilation used in `bitcoin` crate
making the changes minimal.
Add a private type that allows us to mutate the inner vector of a
`ScriptBuf` only using public functions and never touching the inner
field.
Done in preparation for moving the `ScriptBuf` to `primitives`.
Mad hackery by Kix!
In preparation for adding script buf extension make the
`push_slice_no_opt` have the same scope as the other private functions,
this will be the scope of the private extension trait.
In preparation for adding a private extension trait change the scope to
`pub(crate)` because the more specific `pub(in ...)` is not currently
supported by our `define_extension_trait` macro.
In preparation for moving the `ScriptBuf` as a plain old datatype to
`primitives`; separate the POD methods into their own impl block.
Refactor only, no logic changes.
b8067da934 Parse MSRV minor version number from env (yancy)
Pull request description:
Parse MSRV minor version number from env
replaces: https://github.com/rust-bitcoin/rust-bitcoin/pull/3145
ACKs for top commit:
Kixunil:
ACK b8067da934
tcharding:
ACK b8067da934
apoelstra:
ACK b8067da934 successfully ran local tests
Tree-SHA512: 60b3898c9b2739ca4858218b21d5c136b91b6ef45b1dc8f798d97aac861c42d046f114efcefdfa79f514d6da392f715a715ee6b3f230eb459f34e554b48ccf0e
0857697665 Replace impl blocks with extension traits (Martin Habovstiak)
b99bdcfdd6 Format `Script` blocks (Martin Habovstiak)
b027edffe7 Wrap `Script` impl blocks in temporary modules (Martin Habovstiak)
5a461545c7 Separate private `Script` methods (Martin Habovstiak)
27adc09e9f Generalize fn params in `define_extension_trait` (Martin Habovstiak)
fcc3cb03f0 Support non-doc attrs in extension trait macro (Martin Habovstiak)
ca1735f24c Separate POD methods (Tobin C. Harding)
Pull request description:
This moves methods from `Script` to extension traits in steps that should be easy to follow.
Moving to `primitives` requires doing the same with `ScriptBuf` so I'm holding off until this approach gets concept ACK (or alternatively someone else can do it :))
Closes#3161
ACKs for top commit:
tcharding:
ACK 0857697665
apoelstra:
ACK 0857697665 successfully ran local tests
Tree-SHA512: 3768d879e36139cf971c1921d3236141cbe87d707fd4bab7852f6ed8857b7867fa4146dfe720bd54e3d8cc50ecdc93886a10254cf9a82246358253f0312ffb47
2ec901fd63 Move the CompactTarget type to primitives (Tobin C. Harding)
a00bd7cc4d Introduce CompactTargetExt trait (Tobin C. Harding)
100ce03643 Run cargo +nightly fmt (Tobin C. Harding)
9c4a629659 Wrap CompactTarget impl block in temporary module (Tobin C. Harding)
578143c09e Separate CompactTarget impl blocks (Tobin C. Harding)
22d5646f7b Stop using CompactTarget inner field (Tobin C. Harding)
244d7dbe6c Remove generic test impl (Tobin C. Harding)
3d85ee3a02 primitives: Fix alloc feature (Tobin C. Harding)
Pull request description:
Done in preparation for moving `BlockHash` and `block::Header` to `primitives`.
- Patch 1 introduces an extension trait using `define_extension_trait!`
- Patch 2 is the trivial copy and past to move the type to `primitives`
This one shouldn't be to arduous to review, thanks.
ACKs for top commit:
Kixunil:
ACK 2ec901fd63
apoelstra:
ACK 2ec901fd63 successfully ran local tests
Tree-SHA512: b0e4f1af0b268e249a056cae71d7cafd1b025c4a079e5393ce80cd0b9c9bb6d2c6306531dc6786d986ff8a094b61866a86285b20d54037ef1395d127876bfd9c
In preparation to move script types to `primitives` we replace impl
block with extension traits by replacing the temporary modules with
`define_extension_trait`.
`rustfmt` is unable to format macro calls so instead we wrap the impl
blocks in modules to enable formatting in the next commit. We need to
change the visibility of the methods but that's OK since they're
internal.
The macro was trying to "parse" the parameters of functions defined in
extension trait. This was not needed and it was causing problems around
the `self` parameter. In this commit we change the macro to just pass
the parameters through.
Potentially the whole `pow` module will move to `primitives` but this
is not possible easily right now. However, we would like to be able to
move the `BlockHash` and `block::Header` types over to `primitives`
and doing so requires the `CompactTarget` to be there.
Move the `CompactTarget` type to `primitives` and re-export it from the
`primitives` crate root.
Note also, we re-export the type publicly from `bitcoin::pow`.
In preparation for moving the `CompactTarget` type to `primitives` stop
using the inner field in code that will stay behind in the
`bitcoin::pow` module.
In preparation for moving the `CompactTarget` to `primitives` remove the
generic `Into` impl and explicitly implement for just the `From` impls
that the `pow` unit tests use.
Test code only.
The `define_extension_trait` macro originally didn't support `#[inline]`
or other attributes for simplicity. We still want them so this commit
adds basic support for it. It adds the `doc` attributes to trait
*definition* only and adds all other attributes to the *implementation*
only. This should support `#[inline]` and other attributes. The downside
is it doesn't support adding non-doc attributes to trait *definition*
but I can't think of any relevant ones that we would want and we can
find a solution later if we do.
34e8212594 Replace &self with self: &Self (Tobin C. Harding)
Pull request description:
`foo(&self)` is syntax sugar for `foo(self: &Self)`.
The `define_extension_trait` is currently large, ugly, and not that expressive. If we use `self: &Self` then the macro is greatly simplified.
(Also consuming version `self: Self`)
De-sugar only, no logic changes.
ACKs for top commit:
apoelstra:
ACK 34e8212594 successfully ran local tests; lol this looks so much better
Kixunil:
ACK 34e8212594
Tree-SHA512: 7ec81bee99ede328d73a661c9e683a774ba14404ceb89ecb06765bedddf04dc9721672775b9ad3a9e3356d510c76681848f24ce4392a59d53216d23e6a27d9ad
579b76b7cb Introduce ToU64 conversion trait (Tobin C. Harding)
Pull request description:
The idea for this was pulled out of Steven's work in #2133
We already explicitly do not support 16 bit machines.
Also, because Rust supports `u182`s one cannot infallibly convert from a `usize` to a `u64`. This is unergonomic and results in a ton of casts.
We can instead limit our code to running only on machines where `usize` is less that or equal to 64 bits then the infallible conversion is possible.
Since 128 bit machines are not a thing yet this does not in reality introduce any limitations on the library.
Add a "private" trait to the `internals` crate to do infallible conversion to a `u64` from `usize`.
Implement it for all unsigned integers smaller than `u64` as well so we have the option to use the trait instead of `u32::from(foo)`.
ACKs for top commit:
Kixunil:
ACK 579b76b7cb
apoelstra:
ACK 579b76b7cb successfully ran local tests
Tree-SHA512: 2eaddfff995987a346e052386c6dfef3510e4732e674e3a2cfab60ee391b4cce1bf7ba4fb2dfd4926f8203d7251eea2198ccb61f0b40332e624c88fda4fa7f48
191897f9ea Manually format (Tobin C. Harding)
Pull request description:
Run `rustfmt` and manually fix the places where comments are moved to the wrong place.
ACKs for top commit:
Kixunil:
ACK 191897f9ea
apoelstra:
ACK 191897f9ea successfully ran local tests
Tree-SHA512: f977ff373d1d410012734208c090bfcd8f9dbda414d0b19400acf8f552df481b4a2bc20d77c61538895a6fb66197be13cbdadf74956d67fd4d055b99ba8ab356
We already explicitly do not support 16 bit machines.
Also, because Rust supports `u182`s one cannot infallibly convert from a
`usize` to a `u64`. This is unergonomic and results in a ton of casts.
We can instead limit our code to running only on machines where `usize`
is less that or equal to 64 bits then the infallible conversion is
possible.
Since 128 bit machines are not a thing yet this does not in reality
introduce any limitations on the library.
Add a "private" trait to the `internals` crate to do infallible
conversion to a `u64` from `usize`.
Implement it for all unsigned integers smaller than `u64` as well so
we have the option to use the trait instead of `u32::from(foo)`.
`foo(&self)` is syntax sugar for `foo(self: &Self)`.
The `define_extension_trait` is currently large, ugly, and not that
expressive. If we use `self: &Self` then the macro is greatly
simplified.
De-sugar only, no logic changes.
6836de9ee6 Remove catch all pattern (Tobin C. Harding)
Pull request description:
The `PushBytes` type enforces len is less than 0x100000000 so we do not need to panic in a catch all pattern after explicitly matching against less than 0x100000000.
Refactor only because of the invariant on `PushBytes` - no logic changes.
ACKs for top commit:
apoelstra:
ACK 6836de9ee6 successfully ran local tests
Kixunil:
ACK 6836de9ee6
Tree-SHA512: a7cdb31683a8c00eecbdd0879886bec48133f9029f899b5279f1f5294ef40320592db196bfcafdeef4507636fc785a7ab87879b25b6d1b4905ae573b545f1ff4
The `PushBytes` type enforces len is less than 0x100000000 so we do
not need to panic in a catch all pattern after explicitly matching
against less than 0x100000000.
Refactor only because of the invariant on `PushBytes` - no logic
changes.
84df3438ca Fix markdown list items (Tobin C. Harding)
0a45c68cf8 Introduce helper function name policy (Tobin C. Harding)
Pull request description:
As much as it hurts the C hacker inside me we have settled on using `_internal` to mark private function names that clash with a public function of the same name.
Introduce a policy section and rename one instance, I did not grep the codebase looking for other violations.
This came up because I had to look at what `_inner` implied when reading the function name somewhere else.
ACKs for top commit:
storopoli:
ACK 84df3438ca
apoelstra:
ACK 84df3438ca successfully ran local tests; sgtm. should probably update rust-miniscript which uses a `real_` prefix
Tree-SHA512: aece73fac54c4f34bdba72c08721586eb6d76dc9191442bbca43301f2af3d3b3e3217383f1ad20752fa7654e7f5d09927d1a4b72602aa064e76d550b5d6e81bd
yancy
merge-script
Rob N
Tobin C. Harding
Martin Habovstiak
leichak
Antoni Spaanderman