426f585a47 api: Run just check-api (Tobin C. Harding)
6cf90132bc io: Add traits (Tobin C. Harding)
Pull request description:
So that our `io` crate is not surprising it seems we should generally, unless there is a good reason not to, follow `std::io`.
Copy the derived trait implementations from `std::io` for `Cursor`, and `Sink`. `Take` is correct already, just `Debug`.
Done while investigating C-COMMON-TRAITS
ACKs for top commit:
apoelstra:
ACK 426f585a479ca20b7c3390c589b836f8726b9b03; successfully ran local tests
Tree-SHA512: 0fdacfa0295c36e9ee2bdffd5e649b923f48eeff7baa3afc99fda0836ae30b794610a176c0e76341a268c712baab51139355976c23913a0744692cd1e38a4d11
e315aaf1c6 io: Allow setting position bigger than inner buffer (Tobin C. Harding)
Pull request description:
Currently if one calls `set_position` on a cursor setting the position greater than the total length of the inner buffer future calls to `Read` will panic. Bad `Cursor` - no biscuit.
Mirror the stdlib `Cursor` and allow setting the position to any value. Make reads past the end of the buffer return `Ok(0)`.
ACKs for top commit:
apoelstra:
ACK e315aaf1c65225b89435d0a60907a3233caa5db1; successfully ran local tests
Tree-SHA512: 71b151218e96343a86c8b2c21c6e8212e2d1c2aea6517b4662da3ad01b3b598cec497028b863a8e36a6b1fae1d4f7f975c8610c3b6f39f1c256adc5751d06ea0
96f427a8b8 units: Refactor Send/Sync api test (Tobin C. Harding)
Pull request description:
The `api` test for types implementing `Send` and `Sync` is part of both C-SEND-SYNC and also C-GOOD-ERR (for error types).
Refactor the two tests into a single one and document appropriately. This is mirrors how we do it in `io/tests/api.rs`.
ACKs for top commit:
apoelstra:
ACK 96f427a8b84129179e86f3914be8e4712a89f660; successfully ran local tests; lgtm
Tree-SHA512: 7b24780ac2b4f73d0cad952555f005553d9b8c248da6f92c28e7e9510b58eba6c165720ded9bd2f2db19f9a19d72fe7dd333e68312f1291a47e044a94902be0b
So that our `io` crate is not surprising it seems we should generally,
unless there is a good reason not to, follow `std::io`.
Copy the derived trait implementations from `std::io` for `Cursor`,
`Sink`, and `Take`.
Done while investigating C-COMMON-TRAITS
Currently if one calls `set_position` on a cursor setting the position
greater than the total length of the inner buffer future calls to `Read`
will panic. Bad `Cursor` - no biscuit.
Mirror the stdlib `Cursor` and allow setting the position to any value.
Make reads past the end of the buffer return `Ok(0)`.
The `api` test for types implementing `Send` and `Sync` is part of both
C-SEND-SYNC and also C-GOOD-ERR (for error types).
Refactor the Send/Sync tests in both `units` and `io` and improve
comments.
Currently in the `bridge` module to get a reference and mutable
reference we provide the `as_inner` and `inner_mut` functions. These
names are not in line with the `std::io` module which favours `get_ref`
and `get_mut`.
Add inherent functions `get_ref` and `get_mut` for accessing the wrapped
value in `bridge::ToStd` and deprecate `as_inner` and `inner_mut`.
To convince yourself this API is correct grep the stdlib `io` module for
`fn get_ref` as opposed to `fn as_ref`.
Close: #3832
Our `Cursor` is a replacement of sorts for the `std::io::Cursor`. Users
of the `Cursor` are likely to expect the same API so to get a reference
and/or mutable reference users will likely reach for `get_ref` and
`get_mut`. We should provide these.
Deprecate `inner` since it is the same as `get_ref` but less
idiomatically named (should have been `as_inner`).
Add `get_ref` and `get_mut` methods to the `Cursor` type.
There is a range of different wordings used in the docs of constructor
type functions.
Change all to start with `Constructs a new` or `Constructs an empty`.
In functions that act like constructors there is a mixture of the usage
of `creates` and `constructs`.
Replace all occurrences of `creates` with `constructs` in the first line
of docs of constructor like functions.
This has been fixed and we use nightly to lint so we have access to the
merged fix.
Removing the attribute uncovers a bunch of real lint warnings, fix
them while we are at it.
In preparation for releasing `io v0.2.0` bump the version number,
add a changelog entry, update the lock files, and depend on the new
version in all crates that depend on `io`.
18110a51f2 Bump version of internals to 0.4.0 (Tobin C. Harding)
Pull request description:
In preparation for releasing `internals v0.4.0` bump the version number, add a changelog entry, update the lock files, and depend on the new version in all crates that depend on `internals`.
ACKs for top commit:
apoelstra:
ACK 18110a51f2 successfully ran local tests; lots of nice stuff here
Tree-SHA512: a4d3d5279b7d7fa993cbc3b7b34fc6dc4024dd54c0bfa1ecd0f0d5f09b984871f156c3695092a1f6c44b7571f8b2051699040f5f77636d44d4cae6c972ab597f
Examples in documentation are not linted in the same way as other code,
but should still contain correctly written code.
Throughout all of the crates except internals (another commit) unused
variables have been prefixed with `_`, unused imports have been removed,
and a warn attribute added to all of the `lib.rs` files.
In preparation for releasing `internals v0.4.0` bump the version number,
add a changelog entry, update the lock files, and depend on the new
version in all crates that depend on `internals`.
A new nightly version (`nightly-2024-08-28`) introduces a few warnings
because of our rustdocs. These are valid warnings and should be fixed,
thanks `clippy` team.
(The `bip152` change is a bit sloppy, open to suggestions.)
4ead0adcb5 Add blanket impl of io traits for `&mut T` (Martin Habovstiak)
Pull request description:
The impl wasn't previously available because we thought we can do a blanket impl for `std` traits. We've removed the `std` blanket impl when we realized it's broken but forgot to add the `&mut T` impls. This adds them.
Note: this is in part an experiment to see if this is API breaking. I suspect it might be.
ACKs for top commit:
apoelstra:
ACK 4ead0adcb5 successfully ran local tests
tcharding:
ACK 4ead0adcb5
Tree-SHA512: 1e4411fdf019f3793e6366eda7e8b46246e3263bd7e41802877c20cc5e8ea451a79d89b4c59204ea9c1eb590054975de52a791a9d17a1d180a5cfac316efa959
ad34a98c61 Refactor Rust version checking (Martin Habovstiak)
7d5ce89dad Fix type ambiguity in IO tests (Martin Habovstiak)
Pull request description:
Conditional compilation depending on Rust version using `cfg` had the disadvantage that we had to write the same code multiple times, compile it multiple times, execute it multiple times, update it multiple times... Apart from obvious maintenance issues the build script wasn't generating the list of allowed `cfg`s so those had to be maintained manually in `Cargo.toml`. This was fixable by printing an appropriate line but it's best to do it together with the other changes.
Because we cannot export `cfg` flags from a crate to different crates we take a completely different approach: we define a macro called `rust_version` that takes a very naturally looking condition such as `if >= 1.70 {}`. This macro is auto-generated so that it produces different results based on the compiler version - it either expands to first block or the second block (after `else`).
This way, the other crates can simply call the macro when needed.
Unfortunately some minimal maintenance is still needed: to update the max version number when a newer version is used. (Note that code will still work with higher versions, it only limits which conditions can be used in downstream code.) This can be automated with the pin update script or we could just put the pin file into the `internals` directory and read the value from there. Not automating isn't terrible either since anyone adding a cfg with higher version will see a nice error about unknown version of Rust and can update it manually.
Because this changes syntax to a more naturally looking version number, as a side effect the `cond_const` macro could be also updated to use the new macro under the hood, providing much nicer experience - it is no longer needed to provide human-readable version of the version string to put in the note about `const`ness requiring a newer version. As such the note is now always there using a single source of truth.
It's also a great moment to introduce this change right now since there's currently no conditional compilation used in `bitcoin` crate making the changes minimal. However it is not yet added to `bitcoin-io` since `bitcoin-io` is not depending on `internals`. It might be a reason to start depending on it but that's for later discussion.
ACKs for top commit:
apoelstra:
ACK ad34a98c61 successfully ran local tests
tcharding:
ACK ad34a98c61
Tree-SHA512: 7a82017fc51ba1b473ca638c7bdbf5c893da0a78c70ea8f1a0241049e7874592fad328abd60b3e09a00439b771e7cfc5ebad5e874314d15a48271ec6cb2d7bb7
The impl wasn't previously available because we thought we can do a
blanket impl for `std` traits. We've removed the `std` blanket impl when
we realized it's broken but forgot to add the `&mut T` impls. This adds
them.
Conditional compilation depending on Rust version using `cfg` had the
disadvantage that we had to write the same code multiple times, compile
it multiple times, execute it multiple times, update it multiple
times... Apart from obvious maintenance issues the build script wasn't
generating the list of allowed `cfg`s so those had to be maintained
manually in `Cargo.toml`. This was fixable by printing an appropriate
line but it's best to do it together with the other changes.
Because we cannot export `cfg` flags from a crate to different crates we
take a completely different approach: we define a macro called
`rust_version` that takes a very naturally looking condition such as
`if >= 1.70 {}`. This macro is auto-generated so that it produces
different results based on the compiler version - it either expands to
first block or the second block (after `else`).
This way, the other crates can simply call the macro when needed.
Unfortunately some minimal maintenance is still needed: to update the
max version number when a newer version is used. (Note that code will
still work with higher versions, it only limits which conditions can be
used in downstream code.) This can be automated with the pin update
script or we could just put the pin file into the `internals` directory
and read the value from there. Not automating isn't terrible either
since anyone adding a cfg with higher version will see a nice error
about unknown version of Rust and can update it manually.
Because this changes syntax to a more naturally looking version number,
as a side effect the `cond_const` macro could be also updated to use the
new macro under the hood, providing much nicer experience - it is no
longer needed to provide human-readable version of the version string to
put in the note about `const`ness requiring a newer version. As such the
note is now always there using a single source of truth.
It's also a great moment to introduce this change right now since
there's currently no conditional compilation used in `bitcoin` crate
making the changes minimal.
Previously we've only implemented `bitcoin-io` traits for `BufReader`
and `BufWriter` with the reasoning that people should most likely use
those and implementing for other types is too much work. However since
then there were requests to implement them so we do in this commit.
We want to implement the traits for more types which would be tedious
without a macro and it'd risk forgetting to forward some method since
they may be speialized. This also adds previously-forgotten
specializations. It also relaxes the implicit `Sized` bound on
`BufReader` and `BufWriter` in Rust versions above 1.72.
Without the ability to seek, the `Cursor` type is only useful for
wrapping owned buffers. In `std` the equivalent type also allows
seeking. We currently do not have the `Seek` trait so this simply adds a
method to set the position. Further, this adds a method to access inner
type so seeking from the end can be implemented (to compute the position
relative to the end).
Closes#3174
This adds a reverse of the previous commit however it does not add
convenience methods since user should generally prefer the macro
instead. I had a clever idea to add a convenience conversion trait but
that is blocked by Rust stabilizing overlapping *marker* trait impls.
When we've removed the blanket impl we forgot to replace it with a
bridge that would make the usage with `std` types easy. This change
implements the missing bridge forwarding all the methods and traits.
c72069e921 Bump MSRV to 1.63 (Martin Habovstiak)
Pull request description:
The version 1.63 satisfies our requirements for MSRV and provides significant benefits so this commit bumps it. This commit also starts using some advantages of the new MSRV, namely namespaced features, weak dependencies and the ability to use trait bounds in `const` context.
This however does not yet migrade the `rand-std` feature because that requires a release of `secp256k1` with the same kind of change - bumping MSRV to 1.63 and removing `rand-std` in favor of weak dependency. (Accompanying PR to secp256k1: https://github.com/rust-bitcoin/rust-secp256k1/pull/709 )
Suggested plan:
* merge both PRs
* at some point release `hashes` and `secp256k`
* remove `rand-std` from `bitcoin`
* release the rest of the crates
ACKs for top commit:
apoelstra:
ACK c72069e921
tcharding:
ACK c72069e921
Tree-SHA512: 0b301ef8145f01967318d3ed1c738d33e6cf9e44f835f3762122b460a536f926916dbd6ea39d6f80b4f95402cd845e924401e75427dbb0731ca5b12b4fa6915e
The version 1.63 satisfies our requirements for MSRV and provides
significant benefits so this commit bumps it. This commit also starts
using some advantages of the new MSRV, namely namespaced features, weak
dependencies and the ability to use trait bounds in `const` context.
This however does not yet migrade the `rand-std` feature because that
requires a release of `secp256k1` with the same kind of change - bumping
MSRV to 1.63 and removing `rand-std` in favor of weak dependency.
Some of our CI shell scripts are meant only to be sourced and not
run directly however they include an initial shebang line, implying that
they should be run.
Remove the shebang line from `crates.sh` and the various `test_vars.sh`
scripts. Add a `shellcheck` directive to inhibit the no-shebang warning.
Fix: #2764
26b9782d8b CI: Re-write run_task.sh (Tobin C. Harding)
Pull request description:
Recently we re-wrote CI to increase VM level parallelism, in hindsite this has proved to be not that great because:
- It resulted in approx 180 jobs
- We are on free tier so only get 20 jobs (VMs) at a time so its slow to run
- The UI is annoying to dig through the long job list to find failures
Have another go at organising the jobs with the main aim of shortening total run time and making it easier to quickly see fails.
Re-write the `run_task.sh` script, notable moving manifest handling to the workflow. Also don't bother testing with beta toolchain.
### Note on review
The diff is hard to read for `rust.yml`, I tried splitting out a bunch of separate patches but it resulted in the same thing (because there are so many identical lines in the yaml file). I suggest just looking at the yaml file and not the diff.
ACKs for top commit:
apoelstra:
ACK 26b9782d8b
sanket1729:
ACK 26b9782d8b.
Tree-SHA512: 1b0a0bab5cf729c5890f7150953499b42aebd3b1c31a1b0d3dfa5b5e78fda11e17a62a2df6b610ab4a950d5709f3af6fff1ae64d9e67379338903497ab77ae0e
Recently we re-wrote CI to increase VM level parallelism, in hindsite
this has proved to be not that great because:
- It resulted in approx 180 jobs
- We are on free tier so only get 20 jobs (VMs) at a time so its slow to run
- The UI is annoying to dig through the long job list to find failures
Have another go at organising the jobs with the main aim of shortening
total run time and making it easier to quickly see fails.
Re-write the `run_task.sh` script, notable moving manifest handling
to the workflow. Also don't bother testing with beta toolchain.
WASM Note
Removes the `cdylib` and `rlib` from the manifest patching during wasm
build - I do not know the following:
- Why this breaks on this PR but not on other PRs
- Why I can't get wasm test to run locally on master but PRs are passing
- What the `cdylib` and `rlib` were meant to be doing
This is the docs from: https://doc.rust-lang.org/reference/linkage.html
* --crate-type=cdylib, #![crate_type = "cdylib"] - A dynamic system
library will be produced. This is used when compiling a dynamic library
to be loaded from another language. This output type will create *.so
files on Linux, *.dylib files on macOS, and *.dll files on Windows.
* --crate-type=rlib, #![crate_type = "rlib"] - A "Rust library" file
will be produced. This is used as an intermediate artifact and can be
thought of as a "static Rust library". These rlib files, unlike
staticlib files, are interpreted by the compiler in future linkage. This
essentially means that rustc will look for metadata in rlib files like
it looks for metadata in dynamic libraries. This form of output is used
to produce statically linked executables as well as staticlib outputs.
051c358bcb Remove deprecated legacy numeric methods (Divyansh Gupta)
Pull request description:
As `rustc 1.79.0-nightly (9d79cd5f7 2024-04-05)` is released which solves the issue mentioned , but the release has deperacted legacy numeric methods.
Thus replaced `u16::max_value()` etc with `u32::MAX` & `core::u16` to directly `u16`.
fix#2639
ACKs for top commit:
tcharding:
ACK 051c358bcb
apoelstra:
ACK 051c358bcb thanks! I will remove an equivalent commit from my #2669
Tree-SHA512: c08c856f7f3b281417c29283351eac5e0f75cc1c8d23d9aae58d969219a327b2337fe57932053e53773ebb9dbec04254f90149266b6639a66c5c09f2ad1675ef
As `rustc 1.79.0-nightly (9d79cd5f7 2024-04-05)` is released which solves the issue mentioned , but the release has deperacted legacy numeric methods.
Thus replace `u16::max_value()` etc with `u32::MAX` & `core::u16` to directly `u16`.
fix#2639
This lint triggers when parsing a reference to a large struct as a
generic argument, which is wrong.
Allow it crate wide because [subjectively] this lint never warns for
anything useful.
merge-script
Tobin C. Harding
Jamil Lambert, PhD
Fmt Bot
Martin Habovstiak
Jose Storopoli
Shing Him Ng
Divyansh Gupta