6e5bd473a6 Improve siphash's `as_u64` -> `to_u64` rename (Martin Habovstiak)
1a91492204 Clean up the siphash mess (Martin Habovstiak)
Pull request description:
Never break SSOT. Never break SSOT. Never break SSOT. Never break SSOT. Never break SSOT. Never break SSOT. Never break SSOT. ...
Previously we had removed `Default` impl on `siphash24::HashEngine` by
reimplementing the type manually. This was a really bad idea as it
inevitably led to API differences that broke the build which we didn't
notice because of unrelated bug. It should've just split the macro from
the start as was suggested but it was claimed to be difficult, I don't
think was the case as can be seen by this PR.
This commit does what the previous one should've done: it renames the
macro to have `_no_default` suffix, creates another one of the original
name that calls into `_no_default` one and moves anything related to
`Default`. This cleanly ensures all previous hashes stay the same while
siphash gets `Default` removed. This also removes all now-conflicting
impls from `siphash24` module which makes the module almost identical to
what it looked like before the change. The only differences are removed
`Default`/`new`, fixes in tests and recent rename of `as_u64` to
`to_u64`.
FTR both of our recent bugs were caused by breaking SSOT. It confirms again my life experience that breaking it will almost certainly bite one in the ass. If there is the most important principle in programming it's quite likely this one.
I was trying to be more "friendly" towards contributors breaking it but I no longer feel that this is a good choice, especially for complicated PRs. Thus I plan to be more aggressive here and NACK pretty much all breaks of SSOT that aren't provably unsolvable.
ACKs for top commit:
apoelstra:
ACK 6e5bd473a6 successfully ran local tests
Tree-SHA512: 03f413f302a41d07b52ac4645552231bb136aad9eb15010758fbad935ac2a686287d1adab7637372dd2629c37434b6ff9a085fba6792b8b6ec6c2a357e99538e
96e0e720fd feat(bip158): compute canonical filter hash (Rob N)
Pull request description:
From [BIP-157](https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki#filter-headers)
> The canonical hash of a block filter is the double-SHA256 of the serialized filter.
If a user forgets the "double" in double-SHA256 they will be computing a nonsensical filter hash when this is easily handled by the API.
ACKs for top commit:
Kixunil:
ACK 96e0e720fd
tcharding:
ACK 96e0e720fd
Tree-SHA512: 5fc0b1632e2327adacbd0ab56b4cd7edce0774f8be2f819782519c51b9691a748f4b3c01887f3bf1146dee49a35f9dfac833f53ae69ee7a53858bd2cedcec01a
The previous change was just a dumb rename with no deprecation and it
also kept the `self` type which should be taken by value since the hash
is `Copy`. This improves on it by adding a deprecated method of the
original name and changing the type to be `self` instead of `&self`.
Previously we had removed `Default` impl on `siphash24::HashEngine` by
reimplementing the type manually. This was a really bad idea as it
inevitably led to API differences that broke the build which we didn't
notice because of unrelated bug. It should've just split the macro from
the start as was suggested but it was claimed to be difficult, I don't
think was the case as can be seen by this PR.
This commit does what the previous one should've done: it renames the
macro to have `_no_default` suffix, creates another one of the original
name that calls into `_no_default` one and moves anything related to
`Default`. This cleanly ensures all previous hashes stay the same while
siphash gets `Default` removed. This also removes all now-conflicting
impls from `siphash24` module which makes the module almost identical to
what it looked like before the change. The only differences are removed
`Default`/`new`, fixes in tests and recent rename of `as_u64` to
`to_u64`.
d03e5456a3 ci: fix `rev` to be `ref` when pinning bitcoin-maintainer-tools (Andrew Poelstra)
d04b6aabe5 bitcoin: add a couple missing prelude imports (Andrew Poelstra)
30f6bd43a4 Add primitives to the CRATES list (Andrew Poelstra)
e1e0230827 primitives: Enable alloc from serde (Tobin C. Harding)
ce36a37d68 primitives: Test ordered feature (Tobin C. Harding)
Pull request description:
This is a rebase of #3195 on master (since #3200 was merged) and adds one commit which fixes a few missing imports, which it appears was missed in the other "fix CI" PRs.
ACKs for top commit:
Kixunil:
ACK d03e5456a3
tcharding:
ACK d03e5456a3
Tree-SHA512: 6f5dc5c2cbb42716e83e46e70dbf4d644f9e80f6b3d2e75a276e37d33d461f4a86e67eb5c32444dbd23bb9767abedc88bfe958b1beba39852c382cf73736711f
Apparently the `checkout` action will just grab random shit in the case
that you configure it with an unknown key. There is a warning in the
middle of the CI output so ok, I guess there's that.
This commit does not change the pinned version of
rust-bitcoin-maintainer-tools, though we do want to update the pin,
because I want to make sure that the pin is actually working.
This should have been done when we introduced the `primitives`
crate - epic fail.
Intentionally put it before `fuzz` because of the known bug in the CI
script where nothing after `fuzz` gets tested.
Currently the `serde` feature requires an allocator, this is a hang over
from code being written for the `bitcoin` crate.
This was not found because we are not testing `primitives` correctly in
CI (there is the `fuzz` bug and also `primitives` is not even in the
CRATES list).
60b3cabb41 Panic in const context (Tobin C. Harding)
Pull request description:
Now that our MSRV is past 1.57 we can panic in const contexts.
Fix: #2427
ACKs for top commit:
Kixunil:
ACK 60b3cabb41
apoelstra:
ACK 60b3cabb41 successfully ran local tests
Tree-SHA512: 705a8b7d52a11826e6084684706cb7e01dfaa554e4e369739e64e64263537b0c8c0e518b04e96249ecdeea1f22b534594ffd2a89e17ebba85b369d896e820239
c97389596b Remove stale docs from sha256t_hash_newtype (Tobin C. Harding)
39f7dcb816 Reduce API surface of tagged wrapped hash types (Tobin C. Harding)
Pull request description:
Recently we made it so that wrapper types created with `hash_newtype` were not general purpose hash types i.e., one could not easily hash arbitrary data into them. We would like to do the same for tagged wrapped hash types.
In `hashes` do:
- Create a new macro `sha256_tag` that does just the tag/engine stuff out of the `sha256t_hash_newtype` macro.
- Deprecate the `sha256t_hash_newtype` macro.
In `bitcoin` do:
- Use a combination of `sha256_tag` and `hash_newtype` to create tagged wrapped hash types.
Note that we do not add private helper functions `engine` and `from_engine` to the tagged wrapper types as we do for legacy/segwit in `sighash`. Can be done later if wanted/needed.
Fix: #3135
ACKs for top commit:
Kixunil:
ACK c97389596b
apoelstra:
ACK c97389596b successfully ran local tests
Tree-SHA512: d937a8eac1a77298231f946f9dfbc2f7739af8da00f2075b0b54803b4111c0cec810bc6564515153769193056cf102a9c954e216664f055b249d4a6153b14bca
39f10339ad Document the new build script magic (Tobin C. Harding)
Pull request description:
This is a follow up to #3182 which introduced a new way of conditionally including code based on the compiler version.
When originally reviewing I missed the fact that the two loops were controlled by the current compiler version (`minor`) so the created macro is different dependent on the compiler used to build the code.
To help the next guy notice, add a comment.
ACKs for top commit:
Kixunil:
ACK 39f10339ad
apoelstra:
ACK 39f10339ad successfully ran local tests
Tree-SHA512: 9a223718e2691a0f3b6a72c6d666ef4c238764b3ed967e15779311625f6535aea41c2e10795c5a98e69d0d50f23fc928edd72c01ed4a0074cf320f7c1a032b6a
4ead0adcb5 Add blanket impl of io traits for `&mut T` (Martin Habovstiak)
Pull request description:
The impl wasn't previously available because we thought we can do a blanket impl for `std` traits. We've removed the `std` blanket impl when we realized it's broken but forgot to add the `&mut T` impls. This adds them.
Note: this is in part an experiment to see if this is API breaking. I suspect it might be.
ACKs for top commit:
apoelstra:
ACK 4ead0adcb5 successfully ran local tests
tcharding:
ACK 4ead0adcb5
Tree-SHA512: 1e4411fdf019f3793e6366eda7e8b46246e3263bd7e41802877c20cc5e8ea451a79d89b4c59204ea9c1eb590054975de52a791a9d17a1d180a5cfac316efa959
009e747323 CI: Put fuzz last in CRATES list (Tobin C. Harding)
aed61bd2d4 Implement FromStr and serde impls for siphash (Tobin C. Harding)
f8846101ae siphash: Make functions inherent (Tobin C. Harding)
321d82ca53 hashes: Pin in extra_test (Tobin C. Harding)
42c7617a46 Fix shchemars test (Tobin C. Harding)
b6fda6517c Implement JsonSchema for siphash::Hash (Tobin C. Harding)
1af6ff4394 hashes: Feature gate hash_reader unit test (Tobin C. Harding)
5230d3309c Remove hash_reader from sha256t_hash_newtype (Tobin C. Harding)
Pull request description:
This is just the bug fixes pulled out of #2861
The root problem here is the bug described in https://github.com/rust-bitcoin/rust-bitcoin-maintainer-tools/issues/10
This PR works around the bug by putting `fuzz` last in the list.
ACKs for top commit:
apoelstra:
ACK 009e74732321f9f3e6d09e158dd577a7d9ebca35; confirmed that local CI STILL does not pass after this; needs #3195 as well; but it is improved
Kixunil:
ACK 009e747323 but I'd much rather see the siphash problems addressed by splitting up the macro. I can make a PR for that (but just that) later.
Tree-SHA512: f724570003b862e994787192b720c4698e6b904ececa7188770ee507dd9e12382ae64db3363fa15ab3f347341c99ed9a75938f4da0438ce3a0fa88717497a233
4b10490521 Automated update to Github CI to rustc nightly-2024-08-21 (Update Nightly Rustc Bot)
Pull request description:
Automated update to Github CI workflow `rust.yml` by [create-pull-request](https://github.com/peter-evans/create-pull-request) GitHub action
ACKs for top commit:
tcharding:
ACK 4b10490521
Tree-SHA512: 445a50d3c11b8ef6b18130244f4a79c96ad034fa58b79baff463caca1bb3badf6ab829a99e56529dddc1a67d5bd488591af668684135d10cba6264be2915096c
Recently we made it so that wrapper types created with `hash_newtype`
were not general purpose hash types i.e., one could not easily hash
arbitrary data into them. We would like to do the same for tagged
wrapped hash types.
In `hashes` do:
- Create a new macro `sha256t_tag` that does just the tag/engine stuff
out of the `sha256t_hash_newtype` macro.
- Deprecate the `sha256t_hash_newtype` macro.
In `bitcoin` do:
- Use a combination of `sha256t_tag` and `hash_newtype` to create tagged
wrapped hash types.
Note that we do not add private helper functions `engine` and
`from_engine` to the tagged wrapper types as we do for legacy/segwit in
`sighash`. Can be done later if wanted/needed.
This is a follow up to #3182 which introduced a new way of
conditionally including code based on the compiler version.
When originally reviewing I missed the fact that the two loops were
controlled by the current compiler version (`minor`) so the created
macro is different dependent on the compiler used to build the code.
To help the next guy notice, add a comment.
c9053511b2 Remove misleading version metadata (Martin Habovstiak)
Pull request description:
The metadata in dependency specification was misleading because the version was not guaranteed to be the same anyway this was correctly linted but nobody so far cared to fix it. This change fixes it and adds a hint how to get the real version since some people seem still confused about how these things work.
ACKs for top commit:
apoelstra:
ACK c9053511b2 successfully ran local tests
tcharding:
ACK c9053511b2
Tree-SHA512: 99d849b1c1b202a429bc7eb25394ff84ba720b3ca6e73bce615f767e0924f5ef63a87553214f0f4fadd3b68ea84246ab8284d75df56cbbe794da7dea0d169dd9
In recent work making functions on hash types inherent it seems we
missed `siphash`. Add inherent getters/setters to the `siphash::Hash`
type and call through to them from the `Hash` trait impl.
Note:
- The `extra_test.sh` script runs for all toolchains.
- The `schemars` crate does not use the repo lock files.
- We need to pin some deps when building the `schemars` test.
Pin in the `extra_test.sh` script, and mention it in the docs so the
docs don't go stale next MSRV update.
This was previously unnoticed because of the `run_task` bug.
ref: rust-bitcoin/rust-bitcoin-maintainer-tools#10
2bb90b8203 Introduce two extensions traits for ScriptBuf (Tobin C. Harding)
ae0a5bd64a Run cargo fmt (Tobin C. Harding)
3fdc574851 Add temporary script buf modules (Tobin C. Harding)
4ff5d6886b Add private ScriptBufAsVec type (Tobin C. Harding)
c81fb93359 Make push_slice_no_opt pub(crate) (Tobin C. Harding)
1001a33f19 Add second ScriptBuf impl block (Tobin C. Harding)
3625d74e8b Make pub in crate functions pub crate (Tobin C. Harding)
b368384317 Separate ScriptBuf POD methods (Tobin C. Harding)
Pull request description:
Similar to #3155 but for `ScriptBuf`, however it is a little more involved.
Note:
- the change to use `impl` syntax (and addition of #3179)
- mad trickery of `ScriptBufAsVec` (props to Kix)
- widening of scope of private functions
Onward and upward!
ACKs for top commit:
Kixunil:
ACK 2bb90b8203
apoelstra:
ACK 2bb90b8203 successfully ran local tests
Tree-SHA512: 7209d8dc436e52b23e1dbfd9db8432df225ebdb701f465e4d1b55328e22988c98a0f28efdf2a8b3edbafc754354d718ab36bd2f5b1621d12e061b2dadaf49a05
The metadata in dependency specification was misleading because the
version was not guaranteed to be the same anyway this was correctly
linted but nobody so far cared to fix it. This change fixes it and adds
a hint how to get the real version for people who are mistakenly
investigating wrong file.
ad34a98c61 Refactor Rust version checking (Martin Habovstiak)
7d5ce89dad Fix type ambiguity in IO tests (Martin Habovstiak)
Pull request description:
Conditional compilation depending on Rust version using `cfg` had the disadvantage that we had to write the same code multiple times, compile it multiple times, execute it multiple times, update it multiple times... Apart from obvious maintenance issues the build script wasn't generating the list of allowed `cfg`s so those had to be maintained manually in `Cargo.toml`. This was fixable by printing an appropriate line but it's best to do it together with the other changes.
Because we cannot export `cfg` flags from a crate to different crates we take a completely different approach: we define a macro called `rust_version` that takes a very naturally looking condition such as `if >= 1.70 {}`. This macro is auto-generated so that it produces different results based on the compiler version - it either expands to first block or the second block (after `else`).
This way, the other crates can simply call the macro when needed.
Unfortunately some minimal maintenance is still needed: to update the max version number when a newer version is used. (Note that code will still work with higher versions, it only limits which conditions can be used in downstream code.) This can be automated with the pin update script or we could just put the pin file into the `internals` directory and read the value from there. Not automating isn't terrible either since anyone adding a cfg with higher version will see a nice error about unknown version of Rust and can update it manually.
Because this changes syntax to a more naturally looking version number, as a side effect the `cond_const` macro could be also updated to use the new macro under the hood, providing much nicer experience - it is no longer needed to provide human-readable version of the version string to put in the note about `const`ness requiring a newer version. As such the note is now always there using a single source of truth.
It's also a great moment to introduce this change right now since there's currently no conditional compilation used in `bitcoin` crate making the changes minimal. However it is not yet added to `bitcoin-io` since `bitcoin-io` is not depending on `internals`. It might be a reason to start depending on it but that's for later discussion.
ACKs for top commit:
apoelstra:
ACK ad34a98c61 successfully ran local tests
tcharding:
ACK ad34a98c61
Tree-SHA512: 7a82017fc51ba1b473ca638c7bdbf5c893da0a78c70ea8f1a0241049e7874592fad328abd60b3e09a00439b771e7cfc5ebad5e874314d15a48271ec6cb2d7bb7
In preparation for moving the `ScritpBuf` type to `primitives` add a
public and private extension trait for the functions we want to leave
here in `bitcoin`.
Note, includes a change to the `difine_extension_trait` metavariable
used on `$gent` from `ident` to `path` to support the generic
`AsRef<PushBytes>`.
The impl wasn't previously available because we thought we can do a
blanket impl for `std` traits. We've removed the `std` blanket impl when
we realized it's broken but forgot to add the `&mut T` impls. This adds
them.
Conditional compilation depending on Rust version using `cfg` had the
disadvantage that we had to write the same code multiple times, compile
it multiple times, execute it multiple times, update it multiple
times... Apart from obvious maintenance issues the build script wasn't
generating the list of allowed `cfg`s so those had to be maintained
manually in `Cargo.toml`. This was fixable by printing an appropriate
line but it's best to do it together with the other changes.
Because we cannot export `cfg` flags from a crate to different crates we
take a completely different approach: we define a macro called
`rust_version` that takes a very naturally looking condition such as
`if >= 1.70 {}`. This macro is auto-generated so that it produces
different results based on the compiler version - it either expands to
first block or the second block (after `else`).
This way, the other crates can simply call the macro when needed.
Unfortunately some minimal maintenance is still needed: to update the
max version number when a newer version is used. (Note that code will
still work with higher versions, it only limits which conditions can be
used in downstream code.) This can be automated with the pin update
script or we could just put the pin file into the `internals` directory
and read the value from there. Not automating isn't terrible either
since anyone adding a cfg with higher version will see a nice error
about unknown version of Rust and can update it manually.
Because this changes syntax to a more naturally looking version number,
as a side effect the `cond_const` macro could be also updated to use the
new macro under the hood, providing much nicer experience - it is no
longer needed to provide human-readable version of the version string to
put in the note about `const`ness requiring a newer version. As such the
note is now always there using a single source of truth.
It's also a great moment to introduce this change right now since
there's currently no conditional compilation used in `bitcoin` crate
making the changes minimal.
20ecf5e5d8 Automated update to Github CI to rustc nightly-2024-08-18 (Update Nightly Rustc Bot)
Pull request description:
Automated update to Github CI workflow `rust.yml` by [create-pull-request](https://github.com/peter-evans/create-pull-request) GitHub action
ACKs for top commit:
tcharding:
ACK 20ecf5e5d8
Tree-SHA512: 47b323f67cfb65009ca52d6c0a311ad35da06f9a97d3f64fffd2726324c95e03ca777e4a6bc9ea14dbe098a0cd0a67636c8443b39be3f29d6d5f0191d0cf518d
Add a private type that allows us to mutate the inner vector of a
`ScriptBuf` only using public functions and never touching the inner
field.
Done in preparation for moving the `ScriptBuf` to `primitives`.
Mad hackery by Kix!
In preparation for adding script buf extension make the
`push_slice_no_opt` have the same scope as the other private functions,
this will be the scope of the private extension trait.
In preparation for adding a private extension trait change the scope to
`pub(crate)` because the more specific `pub(in ...)` is not currently
supported by our `define_extension_trait` macro.
In preparation for moving the `ScriptBuf` as a plain old datatype to
`primitives`; separate the POD methods into their own impl block.
Refactor only, no logic changes.
yancy
merge-script
Martin Habovstiak
Rob N
Tobin C. Harding
Update Nightly Rustc Bot
leichak