Moves all of the content from `bitcoin/p2p` into `p2p`.
`TryFrom<Network>` must be implemented for `Network -> Magic` now that
`Network` is a foreign, non-exhaustive type. Ser/de test is updated
accordingly, as well as the `Magic::from_network` constructor, which I
have opted to return an `Option<Self>`. The `TryFrom` implementation
uses a new `UnknownNetworkError(Network)` that mirrors the `Magic ->
Network` error.
The example handshake does not generate a random nonce for the version
message, as there is no `rand` dependency in this crate and the program
only makes a single, user-defined connection.
It appears we can do better than copying and pasting the consensus
encoding macros and functions from `bitcoin` without doing weird
cross-crate macros that require some special knowledge of the crate to
know when they will compile.
Peer to peer messages change on occasion, and we would like to add
messages as they are release from Bitcoin Core. Add an empty crate to
take the crate name.
Not all the fields within `block::Header` implement `Display` however
a block header can reasonably be displayed by using 160 hex characters.
Implement `Display` for `block::Header` by printing the header in hex in
the same layout as we hash it in `block_hash`.
Make the `hex` dependency optional. This means not implementing
`Display` for some types if `hex` is not enabled and only implementing
`Debug`.
Also without `hex` enabled:
- We loose the ability to parse an `OutPoint` from string because we
can no longer parse `Txid`.
- We loose the hex formatting of witness elements.
Note also that `primitives` builds with the `serde` feature even if
`hex?/serde` is excluded from the `serde` feature. I found this
surprising.
In the past we've been using `chunks_exact` because const generics were
unstable but then, when they were stabilized we didn't use `as_chunks`
(or `array_chunks`) since they were unstable. But the instability was
only because Rust devs don't know how to handle `0` being passed in. The
function is perfectly implementable on stable. (With a tiny,
easy-to-understand `unsafe` block.) `core` doesn't want to make a
decision for all other crates yet but we can make it for our own crates
because we know that we simply never pass zero. (And even if we did, we
could just change the decision.)
It also turns out there's a hack to simulate `const {}` block in our
MSRV, so we can make compilation fail early.
This commit adds an extension trait to internals to provide the methods,
so we no longer have to use `chunks_exact`. It also cleans up the code
quite nicely.
It has turned out that the `rust-ordered` crate and it's
`ArbitraryOrd` trait are only useful for locktimes and only marginally
useful for them at best.
Remove the `ArbitraryOrd` impls and the `rust-ordered` dependency.
This topic was discussed in various places including:
- #2500
- #4002
- #3881Close: #4029
Currently in order to release `hashes v1.0` we need to 1.0 `io` as well.
For multiple reasons, many out of our control, the `io` crate may not
stabalise any time soon.
Instead we can invert the dependency between the two crates.
This is an ingenious idea, props to Kixunil for coming up with it.
Notes
- `io` does not currently re-export the `hashes` crate.
- This work highlights that we cannot call `hash_reader` on a siphash.
- The `Hmac::hash_reader` uses the default key which may not be obvious.
Signed-off-by: Tobin C. Harding <me@tobin.cc>
Currently the `primitives` crate does not use the `io` dependency. I
don't know if this is just a mistake of if it used to and the manifest
is just stale.
Back in 2022 we elected to use `mutagen` for mutation testing. Since
then `cargo mutants` has progressed to a point where we would now like
to use it instead.
Remove all the `mutagen` stuff and update the lock files.
Close: #2829
f5eb8f4747 api: Run just check-api (Tobin C. Harding)
472b1d3ff3 units: Add serde regression test (Tobin C. Harding)
dedae8acf2 Implement custom serde modules for FeeRate (Tobin C. Harding)
d94e5f03e6 Move fee_rate.rs to module (Tobin C. Harding)
c3c1f6f82d Add missing license comment to test file (Tobin C. Harding)
Pull request description:
Implement and enforce explicit unit when serializing. This is as we do for `Amount` (see #3672 for similar).
To test it, and as part of the 1.0 effort; add regression tests for `serde` stuff in `units`.
With this applied one must use attributes to serialize `FeeRate`.
```rust
#[derive(Serialize, Deserialize)]
pub struct Foo {
#[serde(with = "bitcoin_units::fee_rate::serde::as_sat_per_kwu")]
pub fee_rate: FeeRate,
}
```
ACKs for top commit:
apoelstra:
ACK f5eb8f4747a7cd303cad2b7f8f442bb31862c52a; successfully ran local tests; great idea!
Tree-SHA512: 0968ead568b1e3142efd4c0e856192ddde0f441de84215cbb0950b60a56922f1abaf6d4ccfe243b722a6883c0a927d26bcfba979acf3ca84c4f21baba73af764
I just went to town on the `rust-ordered` crate to get it ready for
releasing a `1.0` version. None of the changes effect our usage here in
`rust-bitcoin`.
Upgrade to the latest version of `rust-ordered` - `v0.3.0`.
Add an integration test the verifies we have serde traits implemented
for all the main crate types (excl. errors and helper structs).
This also acts as a regression test.
We need to do a quick release because it turns out I was wrong in
thinking that making `hex` an optional dependency is not a breaking
change.
```
the package `bitcoin` depends on `bitcoin_hashes`, with features: `hex`
but `bitcoin_hashes` does not have these features. It has a required
dependency with that name, but only optional dependencies can be used as
features.
```
Add a changelog, bump the version, depend on the new version repo wide,
and update the lock files - like its our job.
Out with the old in with the new, update to the latest minor version of
arbitrary.
No real reason to do this except that I wanted to add the minor version
instead of using just `1` so that we don't accidentally pull a new minor
version in during a patch release (after we 1.0).
Needs release notes but there are 20 pages of them, that feels like too
many to go through manually.
Includes a missing changelog entry in `units` for an already released
change that is included in this `bitcoin` release.
In preparation for releasing `primitives v0.101.0` bump the version
number, add a changelog entry, update the lock files, and depend on the
new version in all crates that depend on `primitives`.
c1eccfde25 Move Witness to primitives (Tobin C. Harding)
6ce76cd7c8 Add unstable constructor to Witness (Tobin C. Harding)
Pull request description:
Patch 1 introduces a new policy to the codebase, we use `foo__unstable` for public unstable functions and there are zero semver guarantees if you call these functions.
Patch 2 does the move.
Close#3406
ACKs for top commit:
apoelstra:
ACK c1eccfde25fd4c2b19e7ec6759352b46ac246113; successfully ran local tests
Tree-SHA512: 2388066be2b6bb2cf3d6757c8f6334beeda6115ef1ce7c537955d32aa5e466add5162d0d2adee27f416fe622fc93c4e94bd848326463ee55e08d1c0f4e03719c
Move the `Witness` over to `primitives` leaving behind any method that
takes or returns a `Script` or a signature.
Includes addition of a feature gate to unit test.
f37b573290 bitcoin: Set version to 0.33.0-alpha (Tobin C. Harding)
Pull request description:
We would like to create branches in other repos/crates that track master in this repo for testing purposes. In order to do so we need a non-0.32 version otherwise `cargo` pulls from crates.io.
Set the version to `0.33.0-alpha` - try not to get too excited, this release is a looong way away.
ACKs for top commit:
apoelstra:
ACK f37b5732901133eff0b8b11744f743c71533c240; successfully ran local tests; though I think if you try you can make cargo behave better
Tree-SHA512: 08d9f6e658ac575e4d42681ffe3fff2116ee35e31c1150b0f21228cb5ef9fc326ea604a9cfc56378a7da60815c2e6e9add4dd4d96969fea2dda2005d1c9af34f
a5f5c795f5 Add the ChaCha20Poly1305 AEAD crate (Nick Johnson)
Pull request description:
Hopefully this isn't a little too (or grossly) ambitious, but figured I'd open a PR for feedback and guidance.
BIP324 depends on the ChaCha20 stream cipher and the Poly1305 authenticator. We coded these up for our [BIP324 library](https://github.com/rust-bitcoin/bip324/tree/main/protocol/src/chacha20poly1305) (this is mostly copy/paste from there), but I believe we want these lower-level protocols to live in rust-bitcoin. This follows with how Core has its implementation and also lets others in the ecosystem depend on it. For example, the Payjoin project is looking to specifically use this AEAD algorithm.
While I don't believe this is a case of "rolling your own crypto" since the intention was to match the RFCs, and all test vectors are coded up and passing, I'll admit to being a bit naive on next steps. I tossed this into `hashes` just going with the momentum following the HKDF code that went in first.
## SIMD
This patch was originally just a copy/pasta from our BIP324 library, but modifications have been made to give compilers the best shot at leveraging SIMD instructions. This includes the new `U32x4` type and updates to the cipher's `chacha_block` and `quarter_round` functions to make them easier on the compiler to vector-ize.
Analyzing the assembly produced with the `opt-level=3` and `target-cpu=native` flags shows that the code is about as good as the LDK version, but extensive benchmarks have not been run. With a recent `rustc` version the quarter_round function is fully vector-ized. The project's MSRV `1.63.0` version is also able to vector-ize some parts of the cipher, but not as well as the tested `1.80.1` version.
More of the digging and raw asm output can be found in this [report](https://hackmd.io/@yonson/B1Ot7P0j0).
ACKs for top commit:
apoelstra:
ACK a5f5c795f5e7861d584fc325dcb01dd7d6966494; successfully ran local tests
Tree-SHA512: 692ce27d93f19fccea5a6d322a3eef5f12ed014a80e06738734a5e19fd354044f846718209ca1ed283aab8102f72d1da909315db335e5cf0d6bf58a4a0ba9c78
We would like to create branches in other repos/crates that track master
in this repo for testing purposes. In order to do so we need a non-0.32
version otherwise `cargo` pulls from crates.io.
Set the version to `0.33.0-alpha` - try not to get too excited, this
release is a looong way away.
adf2fa3113 Update lock files and downgrade minimum arbitrary (Tobin C. Harding)
Pull request description:
Recently we added a use of `Arbitrary` for `[u8; 64]` (schnorr sig). I hit a build failure locally and discovered that
- `arbitrary 1.0.0` does not implement `Arbitrary` for 64 byte array
- We use `arbitrary = 1` in the `bitcoin` manifest
- We have `arbitrary 1.3.2` in the `Cargo-minimal.lock` file
Fix this all up by doing:
- Use `1.0.1` in the `bitcoin` manifest because that is a hard minimum version required to build
- Downgrade the `Cargo-minimal.lock` file to use `arbitrary v1.0.1`
- Upgrade the `Cargo-recent.lock` file while we are at it for good measure.
ACKs for top commit:
shinghim:
ACK adf2fa3113
apoelstra:
ACK adf2fa3113 successfully ran local tests
Tree-SHA512: 7be48d2086523322ca4b5f83a344d5096c46cfaff6bd2deb829d30b439feb045ba86b51e9ab2eac4ad5693b0f54a56ca4b86aadafc17a7e4a15cda4a349d0b18
3b7ba4f977 Remove the SliceIndex implementation from hash types (Tobin C. Harding)
Pull request description:
If folk really want to index into a hash they can us `as_byte_array` then index that.
Includes a bump to the version number of `hashes` to `v0.15.0` - this is because otherwise `secp` won't build since we are breaking an API that is used in the current release of secp.
Fix: #3115
ACKs for top commit:
apoelstra:
ACK 3b7ba4f977 successfully ran local tests
Tree-SHA512: 0ba93268cd8133fe683183c5e39ab8b3bf25c15bfa5767d2934d67a5f6a0d2f65f6c9304952315fe8a33abfce488d810a8d28400a28facfb658879ed06acca63
If folk really want to index into a hash they can us `as_byte_array`
then index that.
Includes a bump to the version number of `hashes` to `v0.15.0` - this
is because otherwise `secp` won't build since we are breaking an API
that is used in the current release of secp.
Fix: #3115
In preparation for releasing `io v0.2.0` bump the version number,
add a changelog entry, update the lock files, and depend on the new
version in all crates that depend on `io`.
07a529a132 Bump version of bitcoin-units to 0.2.0 (Tobin C. Harding)
148711a4c6 units: Use double ## in changelog entries (Tobin C. Harding)
80e600ba0c units: Copy 0.1.2 release notes (Tobin C. Harding)
Pull request description:
In preparation for releasing `units v0.2.0` bump the version number, add a changelog entry, update the lock files, and depend on the new version in all crates that depend on `units`.
Close: #3095
ACKs for top commit:
apoelstra:
ACK 07a529a132 successfully ran local tests
Tree-SHA512: 98a75d485ded6225551a5fc4b4a14d8efecc76911a720f959044cdd62781024a8787f258f171ed297705f5ab470f9a88a81ad5d255c9e03c1e22857615ad2e6d
Recently we added a use of `Arbitrary` for `[u8; 64]` (schnorr sig).
I hit a build failure locally and discovered that
- `arbitrary 1.0.0` does not implement `Arbitrary` for 64 byte array
- We use `arbitrary = 1` in the `bitcoin` manifest
- We have `arbitrary 1.3.2` in the `Cargo-minimal.lock` file
Fix this all up by doing:
- Use `1.0.1` in the `bitcoin` manifest because that is a hard minimum
version required to build
- Downgrade the `Cargo-minimal.lock` file to use `arbitrary v1.0.1`
- Upgrade the `Cargo-recent.lock` file while we are at it for good
measure.
Move the `OutPoint` type and associated code over to `primitives`. Take
the opportunity to re-order the code so the file reads like a story,
things are implemented below where they are called.
In preparation for releasing `units v0.2.0` bump the version number,
add a changelog entry, update the lock files, and depend on the new
version in all crates that depend on `units`.
58704c2eff Remove schemars all together (Tobin C. Harding)
Pull request description:
We introduced schemars as a personal favor to a user, and it broke our CI repeatedly but eventually it seemed like it was stable (mainly, our MSRV caught up with its MSRV) so we just let it slide. In the end having schemars on hashes but nowhere else in the rust-bitcoin ecosystem did not prove that useful.
Remove schemars all together.
Fix: #3393
ACKs for top commit:
apoelstra:
ACK 58704c2eff successfully ran local tests
Tree-SHA512: 11c136797f28903c7d6b5199ad55d86bc4bc29ee8dd6f0d575e029f4dbebebabed57ebce6cf773b286297ea84f18d0b6cc58e150299e99457e048226478b49cc
18110a51f2 Bump version of internals to 0.4.0 (Tobin C. Harding)
Pull request description:
In preparation for releasing `internals v0.4.0` bump the version number, add a changelog entry, update the lock files, and depend on the new version in all crates that depend on `internals`.
ACKs for top commit:
apoelstra:
ACK 18110a51f2 successfully ran local tests; lots of nice stuff here
Tree-SHA512: a4d3d5279b7d7fa993cbc3b7b34fc6dc4024dd54c0bfa1ecd0f0d5f09b984871f156c3695092a1f6c44b7571f8b2051699040f5f77636d44d4cae6c972ab597f
We introduced schemars as a personal favor to a user, and it broke our
CI repeatedly but eventually it seemed like it was stable (mainly, our
MSRV caught up with its MSRV) so we just let it slide. In the end having
schemars on hashes but nowhere else in the rust-bitcoin ecosystem did
not prove that useful.
Remove schemars all together.
Fix: #3393
In preparation for releasing `internals v0.4.0` bump the version number,
add a changelog entry, update the lock files, and depend on the new
version in all crates that depend on `internals`.
Conditional compilation depending on Rust version using `cfg` had the
disadvantage that we had to write the same code multiple times, compile
it multiple times, execute it multiple times, update it multiple
times... Apart from obvious maintenance issues the build script wasn't
generating the list of allowed `cfg`s so those had to be maintained
manually in `Cargo.toml`. This was fixable by printing an appropriate
line but it's best to do it together with the other changes.
Because we cannot export `cfg` flags from a crate to different crates we
take a completely different approach: we define a macro called
`rust_version` that takes a very naturally looking condition such as
`if >= 1.70 {}`. This macro is auto-generated so that it produces
different results based on the compiler version - it either expands to
first block or the second block (after `else`).
This way, the other crates can simply call the macro when needed.
Unfortunately some minimal maintenance is still needed: to update the
max version number when a newer version is used. (Note that code will
still work with higher versions, it only limits which conditions can be
used in downstream code.) This can be automated with the pin update
script or we could just put the pin file into the `internals` directory
and read the value from there. Not automating isn't terrible either
since anyone adding a cfg with higher version will see a nice error
about unknown version of Rust and can update it manually.
Because this changes syntax to a more naturally looking version number,
as a side effect the `cond_const` macro could be also updated to use the
new macro under the hood, providing much nicer experience - it is no
longer needed to provide human-readable version of the version string to
put in the note about `const`ness requiring a newer version. As such the
note is now always there using a single source of truth.
It's also a great moment to introduce this change right now since
there's currently no conditional compilation used in `bitcoin` crate
making the changes minimal.
The `absolute` and `relative` locktimes as well as the `Sequence` are
all primitive bitcoin types.
Move the `Sequence`, and `locktime` stuff over to `primitives`.
There is nothing surprising here, the consensus encoding stuff stays in
`bitcoin` and we re-export everything from `blockdata`.
Move the `opcodes` module to the new `primitives` crate. This is pretty
straight forward, some things to note:
- Are we ok with the public wildcard re-export from `blockdata`? I think
so because the whole `blockdata` module should, IMO, be deleted after
everything in it is moved to `primitives`.
- `decode_pushnum` becomes public.
Includes addition of a `patch` section for `primitives` in the
`bitcoin/embedded` crate.
7fa53440dc Move serde_round_trip macro to internals (Tobin C. Harding)
Pull request description:
We currently duplicate the serde_round_trip macro in `units` and `bitcoin`, this is unnecessary since it is a private test macro we can just throw it in `internals`.
While we are at it lets improve the macro by testing a binary encoding also, elect to use the `bincode` crate because we already have it in our dependency graph.
Add `test-serde` feature to `internals` to feature gate the macro and its usage (preventing the transient dependency on `bincode` and `serde_json`).
ACKs for top commit:
Kixunil:
ACK 7fa53440dc
apoelstra:
ACK 7fa53440dc
Tree-SHA512: f40c78bf2539940b7836ed413d5fe96ce4e9ce59bad7b3f86d831971320d1c2effcd23d0da5c785d6c372a2c6962bf720080ec4351248fbbdc0f2cfb4ffd602c
rustaceanrob
Nick Johnson
Tobin C. Harding
lfg2
Martin Habovstiak
19年梦醒
merge-script
Leo Nash
Shing Him Ng
yancy