948647a9dd update documentation for changing lockfiles; change script to support mac nanaive cp function (jeremiah)
Pull request description:
1. Updates `CONTRIBUTING.md` with latest instructions on using `just` to create new lock files. See discussion [here](https://github.com/rust-bitcoin/rust-bitcoin/pull/4045/files).
2. For the command `cp`, `--force` is not an option on the Mac. Changed to `-f` which as far as I can tell is equivalent.
```
➜ rust-bitcoin git:(jr_update_contrib) ✗ cp --force
cp: illegal option -- -
```
ACKs for top commit:
tcharding:
ACK 948647a9dd
Kixunil:
ACK 948647a9dd
apoelstra:
ACK 948647a9dd95bedbb8c34f0b9548e859bfc723b1; successfully ran local tests
Tree-SHA512: a9486ec323b15a9cfa78cd70332e153881537ec8fd26e6e23f8b61ed2272251271299723e9aa390542044588357ec072eb1559cd5e32cb707757f58361a6fb05
To ensure that all target features are tested we simply enable all of
them. However some of them are problematic. We already have an exception
to not enable `crt-static`. This change also adds `soft-float` to the
list because `miri` was warning about it being UB.
The regex is still wrong, it misses structs with generics.
Currently `contrib/api.sh io types` returns:
ErrorKind
Error
Sink
But with this patch applied it returns:
FromStd<T>
ToStd<T>
ErrorKind
Cursor<T>
Error
Sink
Take<'a, R: bitcoin_io::Read + ?core::marker::Sized>
Add a simple script that allows one to query the current API. Done by
parsing the API text files and grepping for things.
This is useful as a dev tool as we try to stabalize the leaf crates.
Add a script to generate API files using `cargo public-api` for the
crates that we are trying to stabalise (the so called 'leaf crates').
- hashes
- io
- primitives
- units
We already ran the script and committed the files last patch. The fact
that this patch does not include any changes to the `api/` directory and
that CI passes is enough to convince us that last patch was valid.
Add a CI job that runs the script and checks there are no changes to
the committed text files thereby proving no API changes are made in a
PR without explicitly modifying the API text files.
Add documentation to `CONTRIBUTING.md` on what is expected of devs.
We have a bit of `unsafe` code in the crates which should really be
checked with `miri`. Thus this adds a basic CI check that automatically
determines which crates need `miri` checking and checks them. It also
makes sure to enable all target features so that SIMD code can be
checked as well.
Instead of manually setting the crates list it is less error prone to do
so mechanically. This required some changes to the `run_task` script
which have now merged, so we update to use the new commit hash at the
same time.
- Use shell to set the `CRATES` env var used by `run_task.sh`.
- Use latest revision of rust-bitcoin-maintainer-tools
This should have been done when we introduced the `primitives`
crate - epic fail.
Intentionally put it before `fuzz` because of the known bug in the CI
script where nothing after `fuzz` gets tested.
We intend on splitting the address types and logic out into a separate
crate. In preparation for doing so, and so that we can grab the name on
crates.io, add an empty crate `bitcoin-addresses`.
Tie it in to the CI infrastructure.
Some of our CI shell scripts are meant only to be sourced and not
run directly however they include an initial shebang line, implying that
they should be run.
Remove the shebang line from `crates.sh` and the various `test_vars.sh`
scripts. Add a `shellcheck` directive to inhibit the no-shebang warning.
Fix: #2764
We would like to check for API changes during development and in CI so
that such changes can be discussed separately from the code.
Add tooling and documentation for creating API listings for the public
crates within the workspace (i.e., not `internals`).
Add API text files from an initial run of the script.
As we did for the wasm job.
In preparation for using the `run_task` script from maintainer tools we
want to have all the things that are particular to `rust-bitcoin` out of
the current `run_task` script.
The address/memory sanitizer test is specific to `hashes`. Add a script
in `hashes/contrib` and call it from the ASAN job.
No test coverage change.
In preparation for using the `run_task` script from maintainer tools we
want to have all the things that are particular to `rust-bitcoin` out of
the current `run_task` script.
The wasm test is specific to `hashes`. Add a script in `hashes/contrib`
and call it from the wasm job.
No test coverage change.
Recently we re-wrote CI to increase VM level parallelism, in hindsite
this has proved to be not that great because:
- It resulted in approx 180 jobs
- We are on free tier so only get 20 jobs (VMs) at a time so its slow to run
- The UI is annoying to dig through the long job list to find failures
Have another go at organising the jobs with the main aim of shortening
total run time and making it easier to quickly see fails.
Re-write the `run_task.sh` script, notable moving manifest handling
to the workflow. Also don't bother testing with beta toolchain.
WASM Note
Removes the `cdylib` and `rlib` from the manifest patching during wasm
build - I do not know the following:
- Why this breaks on this PR but not on other PRs
- Why I can't get wasm test to run locally on master but PRs are passing
- What the `cdylib` and `rlib` were meant to be doing
This is the docs from: https://doc.rust-lang.org/reference/linkage.html
* --crate-type=cdylib, #![crate_type = "cdylib"] - A dynamic system
library will be produced. This is used when compiling a dynamic library
to be loaded from another language. This output type will create *.so
files on Linux, *.dylib files on macOS, and *.dll files on Windows.
* --crate-type=rlib, #![crate_type = "rlib"] - A "Rust library" file
will be produced. This is used as an intermediate artifact and can be
thought of as a "static Rust library". These rlib files, unlike
staticlib files, are interpreted by the compiler in future linkage. This
essentially means that rustc will look for metadata in rlib files like
it looks for metadata in dynamic libraries. This form of output is used
to produce statically linked executables as well as staticlib outputs.
af6dc1db02 internals: Bump version to 0.3.0 (Tobin C. Harding)
Pull request description:
In preparation for release add a changelog and bump the version number.
Please note, the changelog is pretty terse.
ACKs for top commit:
apoelstra:
ACK af6dc1db02
sanket1729:
ACK af6dc1db02
Tree-SHA512: b70d4b9de7de90aba3cbff90dd7f25c5ac801d020dbdfe3e64af4c079347cba726aa783a94fc777e7bf177db8402b54948c2dfd4a766d90c1a7a7a6bdfd36136
In #2635 we broke the Manage PR CI job but for some reason CI didn't run
on that PR so we merged the breakage.
Fix the script by setting default variables when they are not set. Done
with ChatGPT.
Add `euo pipefail` to all non-trial shell scripts, note if `x` is
already set we maintain it.
Note we have a pipe in `run_task.sh` that relies on grep not finding
anything i.e., failing, so we cannot use pipefail there. Disable it and
re-enable it after the pipe.
0ca5a43ce5 hashes: Bump version to v0.14.0 (Tobin C. Harding)
Pull request description:
In preparation for release add a changlelog entry and bump the version.
Note the hashes 0.13.0 dependency stays in the dependency graph because of secp, we can update secp after releasing `hashes` then update the secp dependency in `rust-bitcoin` thereby removing the `hashes v0.13.0` dependency - phew.
Note we are right to release this immediately, the two open PRs (#2337 and #2541) that touch `hashes` only add a clippy attribute so can safely be ignored.
ACKs for top commit:
apoelstra:
ACK 0ca5a43ce5
sanket1729:
ACK 0ca5a43ce5
Tree-SHA512: d1d26acb8fbf13f785b25add3f1dac05bb392b5bdbad16ead2bc5dd26f3d668824c4b653c373f88c3562a37e775146766680606cedd19db40e0f197b26ca86b8
In preparation for release add a changlelog entry and bump the version.
I'm not 100% sure that this release is API breaking, dependencies
definitely changed. The rest might be only additives but I didn't bother
looking exactly because I think its better to bump the minor version and
err on the side of caution.
Note the hashes 0.13.0 dependency stays in the dependency graph because
of secp, we can update secp after releasing `hashes` then update the
secp dependency in `rust-bitcoin` thereby removing the `hashes v0.13.0`
dependency - phew.
32f9b1a231 ci: gate coverage analysis on whether source code changed (Andrew Poelstra)
2203c02347 ci: gate fuzztesting on whether source code changed (Andrew Poelstra)
09f7fc3cff ci: gate CI workflow on source being changed (Andrew Poelstra)
9aca8a18c7 ci: introduce `classify-pr.sh` script which determines whether a PR should have CI run (Andrew Poelstra)
Pull request description:
Fixes#2523
ACKs for top commit:
tcharding:
ACK 32f9b1a231
Tree-SHA512: 7d53365bdf4e8ae5480221e43b4c515d4b16048ec1a00cb62ab65b2d2b91a37f8945c39b619a82bfe4ca74f3508176d9879481b908bb8d3fe8d214b173d3bf18
I believe there is currently a bug in the MemorySanitizer, when we pass
various types across the FFI boundry MSAN gives a unititialized variable
error:
- `usize` passed as `size_t` (cannot be uninitialized)
- byte slice passed as `const char *`
In order to let other work continue disable the MSAN job.
If we pin the version of nightly to a specific date then `cargo
+nightly` will stop working. And even locally, if you want to use a
specific version of nightly, you can't if the script is overriding your
version with "+nightly". Instead the user should set RUSTUP_TOOLCHAIN.
These are not run in CI since #2353 and are likely to go out of date. If
we want a script that users can run locally then we should create a new
script that wraps our current CI.
Jamil Lambert, PhD
merge-script
jeremiah
Martin Habovstiak
Tobin C. Harding
Shing Him Ng
Jose Storopoli
Jose Storopoli
Ryan Breen