Add an integration test the verifies we have serde traits implemented
for all the main crate types (excl. errors and helper structs).
This also acts as a regression test.
We need to do a quick release because it turns out I was wrong in
thinking that making `hex` an optional dependency is not a breaking
change.
```
the package `bitcoin` depends on `bitcoin_hashes`, with features: `hex`
but `bitcoin_hashes` does not have these features. It has a required
dependency with that name, but only optional dependencies can be used as
features.
```
Add a changelog, bump the version, depend on the new version repo wide,
and update the lock files - like its our job.
Out with the old in with the new, update to the latest minor version of
arbitrary.
No real reason to do this except that I wanted to add the minor version
instead of using just `1` so that we don't accidentally pull a new minor
version in during a patch release (after we 1.0).
Needs release notes but there are 20 pages of them, that feels like too
many to go through manually.
Includes a missing changelog entry in `units` for an already released
change that is included in this `bitcoin` release.
In preparation for releasing `primitives v0.101.0` bump the version
number, add a changelog entry, update the lock files, and depend on the
new version in all crates that depend on `primitives`.
c1eccfde25 Move Witness to primitives (Tobin C. Harding)
6ce76cd7c8 Add unstable constructor to Witness (Tobin C. Harding)
Pull request description:
Patch 1 introduces a new policy to the codebase, we use `foo__unstable` for public unstable functions and there are zero semver guarantees if you call these functions.
Patch 2 does the move.
Close#3406
ACKs for top commit:
apoelstra:
ACK c1eccfde25fd4c2b19e7ec6759352b46ac246113; successfully ran local tests
Tree-SHA512: 2388066be2b6bb2cf3d6757c8f6334beeda6115ef1ce7c537955d32aa5e466add5162d0d2adee27f416fe622fc93c4e94bd848326463ee55e08d1c0f4e03719c
Move the `Witness` over to `primitives` leaving behind any method that
takes or returns a `Script` or a signature.
Includes addition of a feature gate to unit test.
f37b573290 bitcoin: Set version to 0.33.0-alpha (Tobin C. Harding)
Pull request description:
We would like to create branches in other repos/crates that track master in this repo for testing purposes. In order to do so we need a non-0.32 version otherwise `cargo` pulls from crates.io.
Set the version to `0.33.0-alpha` - try not to get too excited, this release is a looong way away.
ACKs for top commit:
apoelstra:
ACK f37b5732901133eff0b8b11744f743c71533c240; successfully ran local tests; though I think if you try you can make cargo behave better
Tree-SHA512: 08d9f6e658ac575e4d42681ffe3fff2116ee35e31c1150b0f21228cb5ef9fc326ea604a9cfc56378a7da60815c2e6e9add4dd4d96969fea2dda2005d1c9af34f
a5f5c795f5 Add the ChaCha20Poly1305 AEAD crate (Nick Johnson)
Pull request description:
Hopefully this isn't a little too (or grossly) ambitious, but figured I'd open a PR for feedback and guidance.
BIP324 depends on the ChaCha20 stream cipher and the Poly1305 authenticator. We coded these up for our [BIP324 library](https://github.com/rust-bitcoin/bip324/tree/main/protocol/src/chacha20poly1305) (this is mostly copy/paste from there), but I believe we want these lower-level protocols to live in rust-bitcoin. This follows with how Core has its implementation and also lets others in the ecosystem depend on it. For example, the Payjoin project is looking to specifically use this AEAD algorithm.
While I don't believe this is a case of "rolling your own crypto" since the intention was to match the RFCs, and all test vectors are coded up and passing, I'll admit to being a bit naive on next steps. I tossed this into `hashes` just going with the momentum following the HKDF code that went in first.
## SIMD
This patch was originally just a copy/pasta from our BIP324 library, but modifications have been made to give compilers the best shot at leveraging SIMD instructions. This includes the new `U32x4` type and updates to the cipher's `chacha_block` and `quarter_round` functions to make them easier on the compiler to vector-ize.
Analyzing the assembly produced with the `opt-level=3` and `target-cpu=native` flags shows that the code is about as good as the LDK version, but extensive benchmarks have not been run. With a recent `rustc` version the quarter_round function is fully vector-ized. The project's MSRV `1.63.0` version is also able to vector-ize some parts of the cipher, but not as well as the tested `1.80.1` version.
More of the digging and raw asm output can be found in this [report](https://hackmd.io/@yonson/B1Ot7P0j0).
ACKs for top commit:
apoelstra:
ACK a5f5c795f5e7861d584fc325dcb01dd7d6966494; successfully ran local tests
Tree-SHA512: 692ce27d93f19fccea5a6d322a3eef5f12ed014a80e06738734a5e19fd354044f846718209ca1ed283aab8102f72d1da909315db335e5cf0d6bf58a4a0ba9c78
We would like to create branches in other repos/crates that track master
in this repo for testing purposes. In order to do so we need a non-0.32
version otherwise `cargo` pulls from crates.io.
Set the version to `0.33.0-alpha` - try not to get too excited, this
release is a looong way away.
adf2fa3113 Update lock files and downgrade minimum arbitrary (Tobin C. Harding)
Pull request description:
Recently we added a use of `Arbitrary` for `[u8; 64]` (schnorr sig). I hit a build failure locally and discovered that
- `arbitrary 1.0.0` does not implement `Arbitrary` for 64 byte array
- We use `arbitrary = 1` in the `bitcoin` manifest
- We have `arbitrary 1.3.2` in the `Cargo-minimal.lock` file
Fix this all up by doing:
- Use `1.0.1` in the `bitcoin` manifest because that is a hard minimum version required to build
- Downgrade the `Cargo-minimal.lock` file to use `arbitrary v1.0.1`
- Upgrade the `Cargo-recent.lock` file while we are at it for good measure.
ACKs for top commit:
shinghim:
ACK adf2fa3113
apoelstra:
ACK adf2fa3113 successfully ran local tests
Tree-SHA512: 7be48d2086523322ca4b5f83a344d5096c46cfaff6bd2deb829d30b439feb045ba86b51e9ab2eac4ad5693b0f54a56ca4b86aadafc17a7e4a15cda4a349d0b18
3b7ba4f977 Remove the SliceIndex implementation from hash types (Tobin C. Harding)
Pull request description:
If folk really want to index into a hash they can us `as_byte_array` then index that.
Includes a bump to the version number of `hashes` to `v0.15.0` - this is because otherwise `secp` won't build since we are breaking an API that is used in the current release of secp.
Fix: #3115
ACKs for top commit:
apoelstra:
ACK 3b7ba4f977 successfully ran local tests
Tree-SHA512: 0ba93268cd8133fe683183c5e39ab8b3bf25c15bfa5767d2934d67a5f6a0d2f65f6c9304952315fe8a33abfce488d810a8d28400a28facfb658879ed06acca63
If folk really want to index into a hash they can us `as_byte_array`
then index that.
Includes a bump to the version number of `hashes` to `v0.15.0` - this
is because otherwise `secp` won't build since we are breaking an API
that is used in the current release of secp.
Fix: #3115
In preparation for releasing `io v0.2.0` bump the version number,
add a changelog entry, update the lock files, and depend on the new
version in all crates that depend on `io`.
07a529a132 Bump version of bitcoin-units to 0.2.0 (Tobin C. Harding)
148711a4c6 units: Use double ## in changelog entries (Tobin C. Harding)
80e600ba0c units: Copy 0.1.2 release notes (Tobin C. Harding)
Pull request description:
In preparation for releasing `units v0.2.0` bump the version number, add a changelog entry, update the lock files, and depend on the new version in all crates that depend on `units`.
Close: #3095
ACKs for top commit:
apoelstra:
ACK 07a529a132 successfully ran local tests
Tree-SHA512: 98a75d485ded6225551a5fc4b4a14d8efecc76911a720f959044cdd62781024a8787f258f171ed297705f5ab470f9a88a81ad5d255c9e03c1e22857615ad2e6d
Recently we added a use of `Arbitrary` for `[u8; 64]` (schnorr sig).
I hit a build failure locally and discovered that
- `arbitrary 1.0.0` does not implement `Arbitrary` for 64 byte array
- We use `arbitrary = 1` in the `bitcoin` manifest
- We have `arbitrary 1.3.2` in the `Cargo-minimal.lock` file
Fix this all up by doing:
- Use `1.0.1` in the `bitcoin` manifest because that is a hard minimum
version required to build
- Downgrade the `Cargo-minimal.lock` file to use `arbitrary v1.0.1`
- Upgrade the `Cargo-recent.lock` file while we are at it for good
measure.
Move the `OutPoint` type and associated code over to `primitives`. Take
the opportunity to re-order the code so the file reads like a story,
things are implemented below where they are called.
In preparation for releasing `units v0.2.0` bump the version number,
add a changelog entry, update the lock files, and depend on the new
version in all crates that depend on `units`.
58704c2eff Remove schemars all together (Tobin C. Harding)
Pull request description:
We introduced schemars as a personal favor to a user, and it broke our CI repeatedly but eventually it seemed like it was stable (mainly, our MSRV caught up with its MSRV) so we just let it slide. In the end having schemars on hashes but nowhere else in the rust-bitcoin ecosystem did not prove that useful.
Remove schemars all together.
Fix: #3393
ACKs for top commit:
apoelstra:
ACK 58704c2eff successfully ran local tests
Tree-SHA512: 11c136797f28903c7d6b5199ad55d86bc4bc29ee8dd6f0d575e029f4dbebebabed57ebce6cf773b286297ea84f18d0b6cc58e150299e99457e048226478b49cc
18110a51f2 Bump version of internals to 0.4.0 (Tobin C. Harding)
Pull request description:
In preparation for releasing `internals v0.4.0` bump the version number, add a changelog entry, update the lock files, and depend on the new version in all crates that depend on `internals`.
ACKs for top commit:
apoelstra:
ACK 18110a51f2 successfully ran local tests; lots of nice stuff here
Tree-SHA512: a4d3d5279b7d7fa993cbc3b7b34fc6dc4024dd54c0bfa1ecd0f0d5f09b984871f156c3695092a1f6c44b7571f8b2051699040f5f77636d44d4cae6c972ab597f
We introduced schemars as a personal favor to a user, and it broke our
CI repeatedly but eventually it seemed like it was stable (mainly, our
MSRV caught up with its MSRV) so we just let it slide. In the end having
schemars on hashes but nowhere else in the rust-bitcoin ecosystem did
not prove that useful.
Remove schemars all together.
Fix: #3393
In preparation for releasing `internals v0.4.0` bump the version number,
add a changelog entry, update the lock files, and depend on the new
version in all crates that depend on `internals`.
Conditional compilation depending on Rust version using `cfg` had the
disadvantage that we had to write the same code multiple times, compile
it multiple times, execute it multiple times, update it multiple
times... Apart from obvious maintenance issues the build script wasn't
generating the list of allowed `cfg`s so those had to be maintained
manually in `Cargo.toml`. This was fixable by printing an appropriate
line but it's best to do it together with the other changes.
Because we cannot export `cfg` flags from a crate to different crates we
take a completely different approach: we define a macro called
`rust_version` that takes a very naturally looking condition such as
`if >= 1.70 {}`. This macro is auto-generated so that it produces
different results based on the compiler version - it either expands to
first block or the second block (after `else`).
This way, the other crates can simply call the macro when needed.
Unfortunately some minimal maintenance is still needed: to update the
max version number when a newer version is used. (Note that code will
still work with higher versions, it only limits which conditions can be
used in downstream code.) This can be automated with the pin update
script or we could just put the pin file into the `internals` directory
and read the value from there. Not automating isn't terrible either
since anyone adding a cfg with higher version will see a nice error
about unknown version of Rust and can update it manually.
Because this changes syntax to a more naturally looking version number,
as a side effect the `cond_const` macro could be also updated to use the
new macro under the hood, providing much nicer experience - it is no
longer needed to provide human-readable version of the version string to
put in the note about `const`ness requiring a newer version. As such the
note is now always there using a single source of truth.
It's also a great moment to introduce this change right now since
there's currently no conditional compilation used in `bitcoin` crate
making the changes minimal.
The `absolute` and `relative` locktimes as well as the `Sequence` are
all primitive bitcoin types.
Move the `Sequence`, and `locktime` stuff over to `primitives`.
There is nothing surprising here, the consensus encoding stuff stays in
`bitcoin` and we re-export everything from `blockdata`.
Move the `opcodes` module to the new `primitives` crate. This is pretty
straight forward, some things to note:
- Are we ok with the public wildcard re-export from `blockdata`? I think
so because the whole `blockdata` module should, IMO, be deleted after
everything in it is moved to `primitives`.
- `decode_pushnum` becomes public.
Includes addition of a `patch` section for `primitives` in the
`bitcoin/embedded` crate.
7fa53440dc Move serde_round_trip macro to internals (Tobin C. Harding)
Pull request description:
We currently duplicate the serde_round_trip macro in `units` and `bitcoin`, this is unnecessary since it is a private test macro we can just throw it in `internals`.
While we are at it lets improve the macro by testing a binary encoding also, elect to use the `bincode` crate because we already have it in our dependency graph.
Add `test-serde` feature to `internals` to feature gate the macro and its usage (preventing the transient dependency on `bincode` and `serde_json`).
ACKs for top commit:
Kixunil:
ACK 7fa53440dc
apoelstra:
ACK 7fa53440dc
Tree-SHA512: f40c78bf2539940b7836ed413d5fe96ce4e9ce59bad7b3f86d831971320d1c2effcd23d0da5c785d6c372a2c6962bf720080ec4351248fbbdc0f2cfb4ffd602c
We currently duplicate the serde_round_trip macro in `units` and
`bitcoin`, this is unnecessary since it is a private test macro we can
just throw it in `internals`.
While we are at it lets improve the macro by testing a binary encoding
also, elect to use the `bincode` crate because we already have it in
our dependency graph.
Add `test-serde` feature to `internals` to feature gate the macro and
its usage (preventing the transient dependency on `bincode` and
`serde_json`).
Introduce an empty `bitcoin-primitives` crate.
We were give the name on crates.io and previously a version
`v0.1.16-alpha` was released so we use `v0.100.0`.
We intend on splitting the address types and logic out into a separate
crate. In preparation for doing so, and so that we can grab the name on
crates.io, add an empty crate `bitcoin-addresses`.
Tie it in to the CI infrastructure.
Updated bitcoinconsensus version to 0.106.0+26.0.
The new version supports taproot and has a new parameter for spent outputs in the `verify()` and `verify_with_flags()` functions.
The validation module was changed to keep the existing functionality by adding `None` as the `spent_outputs` and the flag `VERIFY_ALL_PRE_TAPROOT`.
830a6e1b0c fuzz: delete CBOR test (Andrew Poelstra)
91eb50b2db fuzz: add lint to generate-files.sh (Andrew Poelstra)
Pull request description:
We were using an outdated CBOR crate for MSRV reasons. But this old crate is causing suprious test failures. So delete it. (Sadly, updating the crate doesn't fix the issue, replacing it with ciborium breaks our MSRV tests because it needs a more recent `half` dependency, and replacing it with `minicbor` doesn't work because minicbor is not based on serde. So we don't really have any options.)
In general, I am suspicious of this decode-then-reencode test. CBOR has some ambiguity in integer encoding. Empirically it has seemed to work for a long time, but this seems more like an indictment of our test than a positive result.
Also, round-trip testing serde encoding of a byte vector is probably not a great use of our fuzz resources. I don't believe we have ever had a problem with this.
Fixes#2801
ACKs for top commit:
tcharding:
ACK 830a6e1b0c
Tree-SHA512: f207b68da2f0910542cd8b6a35bb2364462030bdf08ac1e954fd9dcdbef47b2035ac85f964adb9590078dfc2151e8fc7fe2ed41ec0919ff937723c5954612a47
We were using an outdated CBOR crate for MSRV reasons. But this old
crate is causing suprious test failures. So delete it. (Sadly, updating
the crate doesn't fix the issue, replacing it with ciborium breaks our
MSRV tests because it needs a more recent `half` dependency, and
replacing it with `minicbor` doesn't work because minicbor is not based
on serde. So we don't really have any options.)
In general, I am suspicious of this decode-then-reencode test. CBOR has
some ambiguity in integer encoding. Empirically it has seemed to
work for a long time, but this seems more like an indictment of our test
than a positive result.
Also, round-trip testing serde encoding of a byte vector is probably not
a great use of our fuzz resources. I don't believe we have ever had a
problem with this.
Fixes#2801
Currently we have a dependency on `hex_lit` and it is used in exactly
one place outside of test code, if we instead use a hardcoded array
instead we can move the `hex_lit` dependency to `dev-dependencies`.
Hard code the genesis block script bytes as an array of hex digits, link
to the blockstream explorer for those interested and comment the bytes
liberally since it took me a while to work out what they were.
Move the `hex_lit` dependency and update the lock files.
46e0ce59a3 bitcoin: Set version number (Tobin C. Harding)
Pull request description:
During the recent release cycle we left `bitcoin` on the last rc version.
Set the version number to `v0.33.0-unreleased` to make it obvious what it is.
Close: #2724
ACKs for top commit:
apoelstra:
ACK 46e0ce59a3
sanket1729:
ACK 46e0ce59a3
Tree-SHA512: 8a1fa062cf0240c5a7e1c05f0378097464423c29827f14068d01ac562d1be571bec2d267a5a12d0828c5a2809ef743f03ef29ccea9a31a5c9e3eefce66b3d30d
During the recent release cycle we left `bitcoin` on the last rc
version.
Set the version number to `v0.33.0-unreleased` to make it obvious what
it is.
Close: #2724
In preparation for dropping the first release candidate bump the version
and add a changelog.
Please not I went to much more effort that usual with the changelog,
open to review on the overall form - not promising I'll change it but
definitely would like to keep iterating and improving.
If this changelog is appreciated then FWIW I don't think we should
bother automating it, a machine does not have all the context required
to create it.
We just did a minor bug fix to error code in the `amounts` module. This
change did not effect the public API but improved the display of two
error types from that module.
In preparation for doing a point release bump the version number and add
a changelog entry.
Tobin C. Harding
Nick Johnson
Leo Nash
merge-script
Shing Him Ng
yancy
Martin Habovstiak
Jamil Lambert, PhD