Some of our CI shell scripts are meant only to be sourced and not
run directly however they include an initial shebang line, implying that
they should be run.
Remove the shebang line from `crates.sh` and the various `test_vars.sh`
scripts. Add a `shellcheck` directive to inhibit the no-shebang warning.
Fix: #2764
26b9782d8b CI: Re-write run_task.sh (Tobin C. Harding)
Pull request description:
Recently we re-wrote CI to increase VM level parallelism, in hindsite this has proved to be not that great because:
- It resulted in approx 180 jobs
- We are on free tier so only get 20 jobs (VMs) at a time so its slow to run
- The UI is annoying to dig through the long job list to find failures
Have another go at organising the jobs with the main aim of shortening total run time and making it easier to quickly see fails.
Re-write the `run_task.sh` script, notable moving manifest handling to the workflow. Also don't bother testing with beta toolchain.
### Note on review
The diff is hard to read for `rust.yml`, I tried splitting out a bunch of separate patches but it resulted in the same thing (because there are so many identical lines in the yaml file). I suggest just looking at the yaml file and not the diff.
ACKs for top commit:
apoelstra:
ACK 26b9782d8b
sanket1729:
ACK 26b9782d8b.
Tree-SHA512: 1b0a0bab5cf729c5890f7150953499b42aebd3b1c31a1b0d3dfa5b5e78fda11e17a62a2df6b610ab4a950d5709f3af6fff1ae64d9e67379338903497ab77ae0e
Recently we re-wrote CI to increase VM level parallelism, in hindsite
this has proved to be not that great because:
- It resulted in approx 180 jobs
- We are on free tier so only get 20 jobs (VMs) at a time so its slow to run
- The UI is annoying to dig through the long job list to find failures
Have another go at organising the jobs with the main aim of shortening
total run time and making it easier to quickly see fails.
Re-write the `run_task.sh` script, notable moving manifest handling
to the workflow. Also don't bother testing with beta toolchain.
WASM Note
Removes the `cdylib` and `rlib` from the manifest patching during wasm
build - I do not know the following:
- Why this breaks on this PR but not on other PRs
- Why I can't get wasm test to run locally on master but PRs are passing
- What the `cdylib` and `rlib` were meant to be doing
This is the docs from: https://doc.rust-lang.org/reference/linkage.html
* --crate-type=cdylib, #![crate_type = "cdylib"] - A dynamic system
library will be produced. This is used when compiling a dynamic library
to be loaded from another language. This output type will create *.so
files on Linux, *.dylib files on macOS, and *.dll files on Windows.
* --crate-type=rlib, #![crate_type = "rlib"] - A "Rust library" file
will be produced. This is used as an intermediate artifact and can be
thought of as a "static Rust library". These rlib files, unlike
staticlib files, are interpreted by the compiler in future linkage. This
essentially means that rustc will look for metadata in rlib files like
it looks for metadata in dynamic libraries. This form of output is used
to produce statically linked executables as well as staticlib outputs.
051c358bcb Remove deprecated legacy numeric methods (Divyansh Gupta)
Pull request description:
As `rustc 1.79.0-nightly (9d79cd5f7 2024-04-05)` is released which solves the issue mentioned , but the release has deperacted legacy numeric methods.
Thus replaced `u16::max_value()` etc with `u32::MAX` & `core::u16` to directly `u16`.
fix#2639
ACKs for top commit:
tcharding:
ACK 051c358bcb
apoelstra:
ACK 051c358bcb thanks! I will remove an equivalent commit from my #2669
Tree-SHA512: c08c856f7f3b281417c29283351eac5e0f75cc1c8d23d9aae58d969219a327b2337fe57932053e53773ebb9dbec04254f90149266b6639a66c5c09f2ad1675ef
As `rustc 1.79.0-nightly (9d79cd5f7 2024-04-05)` is released which solves the issue mentioned , but the release has deperacted legacy numeric methods.
Thus replace `u16::max_value()` etc with `u32::MAX` & `core::u16` to directly `u16`.
fix#2639
This lint triggers when parsing a reference to a large struct as a
generic argument, which is wrong.
Allow it crate wide because [subjectively] this lint never warns for
anything useful.
There have only been two PRs merged that touched the `io` crate since it
as last released. The changes are additive so we can do a pre-0.1 point
release.
In preparation for release bump the version and add a changelog entry.
We attempted to release with the current 0.1.0 version forgetting that
we had previously released an empty crate with that version to reserve
the name on crates.io.
Bump the version to 0.1.1 and release the actual code.
Remove the blanket impls of `Read`, `BufRead`, and `Write`. Replace them
with a set of sane impls.
Note, we add code to the `impl_write` macro to implement both
`crate::Write` and `std::io::Write` when "std" feature is
enabled.
Fix: #2432
Done in preparation for an initial v0.1.0 release of the new `io` crate.
Add a changelog file with a brief description of whats in the initial release.
Currently `Take::read_to_end` is private forcing users to use our
"custom" `read_to_limit`, for seasoned Rust hackers
`foo.take(16).read_to_end(buf)` make be more unsurprising.
Make `read_to_end` public.
Make the trait level attributes uniform across all released crates in
the repo. Excludes things that are obviously not needed, eg, bench stuff
if there is not bench code.
- Remove `uninhabited_references` - this is allow by default now.
- Remove `unconditional_recursion` and mark the single false positive we
have with an `allow`.
Note, this does not add `missing_docs` to the `io` crate. There is an
open PR at the moment to add that along with the required docs.
The `std::io::Read` trait includes `read_to_end` but that method
provides a denial of service attack vector since an unbounded reader
will exhaust all system memory.
Add a method to our `Read` trait called `read_to_limit` that does the
same as `std::io::Read::read_to_end` but with memory exhaustion
protection.
Add a `read_to_end` method on our `Take` trait and call through to it
from the new method on our `Read` trait called `read_to_limit`.
Add a `BufRead` trait for types that perform buffered reading.
Implement it for:
- `Take`
- `Cursor`
- `std::io::BufRead` readers
- (in no-std builds) for slice of u8s
Move error code to a private `error` submodule and re-export it at the
crate root.
(Puts private `sealed` module at the bottom of the file i.e., this patch
is strictly a code move but we re-order the code while moving it.)
Its not immediately obvious why we nest the whole `io` code in an `io`
submodule within `lib.rs`. As far as I can tell we can inline it and
re-export from `rust-bitcoin` same as we do for our other dependencies.
This change would effect other users of the crate but since the `io`
crate is unreleased this effects no-one except us.
In preparation for inlining the `io` molule, move the public macros to a
private `macros` module.
Includes removal of the public re-export of `std` as `_std` - flaggin
this because I do not understand why it is here in the first place, we
can use `std::io::Write` in code that is feature gated on "std".
In order to move towards our own I/O traits in the `rust-bitcoin`
ecosystem, we have to slowly replace our use of the `std` and
`core2` traits.
This is the final step in removing the explicit `core2` dependency
for I/O in no-std - replacing the `io::Error` type with our own.
Sadly the `std::io::Error` type requires `std::error::Error` as a
bound on the inner error, which is rather difficult to duplicate in
a way that allows for mapping to `std` and back.
To take a more general approach, we use bound on any `Debug`
instead.
In order to move towards our own I/O traits in the `rust-bitcoin`
ecosystem, we have to slowly replace our use of the `std` and
`core2` traits.
Here we take the second big step, replacing
`{std,core2}::io::Read` with our own `bitcoin_io::io::Read`. We
provide a blanket impl for our trait for all `std::io::Read`, if
the `std` feature is enabled, allowing users who use their own
streams or `std` streams to call `rust-bitcoin` methods directly.
With the new `bitcoin_io` library, implementing `io::Write`
manually is somewhat tricky - for `std` users we really want to
provide an `std::io::Write` implementation, however for `no-std`
users we want to implement against our internal trait.
Sadly we cannot provide a blanket implementation of
`std::io::Write` for all types whcih implement our `io::Write`
trait as its an out-of-crate impl.
Instead, we provide a macro which will either implement
`std::io::Write` or our `io::Write` depending on the feature flags
set on `bitcoin_io`.
Shing Him Ng
Tobin C. Harding
Andrew Poelstra
Jose Storopoli
Fmt Bot
Divyansh Gupta
Liam Aharon
Martin Habovstiak
apoelstra
Matt Corallo