6def5bc974 CI: Use run_task from maintainer tools (Tobin C. Harding)
c5af52847b CI: Add docs and document start (Tobin C. Harding)
62ba10503a CI: Use correct spacing (Tobin C. Harding)
0c0e88165e CI: Add README file (Tobin C. Harding)
44cb2255d3 CI: Add sanitizer script (Tobin C. Harding)
3407257936 CI: Add WASM script (Tobin C. Harding)
cc14edf63f CI: Run the schemars job directly using cargo (Tobin C. Harding)
1fb12e1917 CI: Remove recent from schemars job (Tobin C. Harding)
8d7117bb0e CI: Use original name (Tobin C. Harding)
Pull request description:
First we do some clean up, then we pull the stuff that is specific to this repo out into separate tests, then as the last patch we switch over to use the new script for `rust-bitcoin-maintainer-tools` that was introduced in https://github.com/rust-bitcoin/rust-bitcoin-maintainer-tools/pull/4.
ACKs for top commit:
apoelstra:
ACK 6def5bc974
Tree-SHA512: 70105d455294d49deb43461958435096d074d19ea8d9adc7036e15d74dfdab466f04b1d9e934574077ca0c32a6fe77d0e5aa11068d8c4d51acef634591227d33
Add a document start and comment to help try to stop the readme from
going stale.
This removes a `yamllint` warning.
While we are clearing lint warnings disable the one causing
warning truthy value should be one of [false, true] (truthy)
This is a know issue with GitHub actions yaml because `on` is a yaml
keyword, or something like that.
As we did for the wasm job.
In preparation for using the `run_task` script from maintainer tools we
want to have all the things that are particular to `rust-bitcoin` out of
the current `run_task` script.
The address/memory sanitizer test is specific to `hashes`. Add a script
in `hashes/contrib` and call it from the ASAN job.
No test coverage change.
In preparation for using the `run_task` script from maintainer tools we
want to have all the things that are particular to `rust-bitcoin` out of
the current `run_task` script.
The wasm test is specific to `hashes`. Add a script in `hashes/contrib`
and call it from the wasm job.
No test coverage change.
I've been a bit confused lately about pinning and the locks, at some
stage recently I changed 'Set dependencies' to 'Copy lock file'. Its no
biggy but the original that Kix wrote was correct and descriptive - it
was me who was confused. Revert it back to the original.
26b9782d8b CI: Re-write run_task.sh (Tobin C. Harding)
Pull request description:
Recently we re-wrote CI to increase VM level parallelism, in hindsite this has proved to be not that great because:
- It resulted in approx 180 jobs
- We are on free tier so only get 20 jobs (VMs) at a time so its slow to run
- The UI is annoying to dig through the long job list to find failures
Have another go at organising the jobs with the main aim of shortening total run time and making it easier to quickly see fails.
Re-write the `run_task.sh` script, notable moving manifest handling to the workflow. Also don't bother testing with beta toolchain.
### Note on review
The diff is hard to read for `rust.yml`, I tried splitting out a bunch of separate patches but it resulted in the same thing (because there are so many identical lines in the yaml file). I suggest just looking at the yaml file and not the diff.
ACKs for top commit:
apoelstra:
ACK 26b9782d8b
sanket1729:
ACK 26b9782d8b.
Tree-SHA512: 1b0a0bab5cf729c5890f7150953499b42aebd3b1c31a1b0d3dfa5b5e78fda11e17a62a2df6b610ab4a950d5709f3af6fff1ae64d9e67379338903497ab77ae0e
Recently we re-wrote CI to increase VM level parallelism, in hindsite
this has proved to be not that great because:
- It resulted in approx 180 jobs
- We are on free tier so only get 20 jobs (VMs) at a time so its slow to run
- The UI is annoying to dig through the long job list to find failures
Have another go at organising the jobs with the main aim of shortening
total run time and making it easier to quickly see fails.
Re-write the `run_task.sh` script, notable moving manifest handling
to the workflow. Also don't bother testing with beta toolchain.
WASM Note
Removes the `cdylib` and `rlib` from the manifest patching during wasm
build - I do not know the following:
- Why this breaks on this PR but not on other PRs
- Why I can't get wasm test to run locally on master but PRs are passing
- What the `cdylib` and `rlib` were meant to be doing
This is the docs from: https://doc.rust-lang.org/reference/linkage.html
* --crate-type=cdylib, #![crate_type = "cdylib"] - A dynamic system
library will be produced. This is used when compiling a dynamic library
to be loaded from another language. This output type will create *.so
files on Linux, *.dylib files on macOS, and *.dll files on Windows.
* --crate-type=rlib, #![crate_type = "rlib"] - A "Rust library" file
will be produced. This is used as an intermediate artifact and can be
thought of as a "static Rust library". These rlib files, unlike
staticlib files, are interpreted by the compiler in future linkage. This
essentially means that rustc will look for metadata in rlib files like
it looks for metadata in dynamic libraries. This form of output is used
to produce statically linked executables as well as staticlib outputs.
The `labeler` job has an input `sync-labels` (default `false`) that
configures:
> Whether or not to remove labels when matching files are reverted or no
> longer changed by the PR
ref: https://github.com/actions/labeler
Currently we are using the default which means labels are not synced
when a PR is modified.
Example
A PR initially touches the `hashes` and `bitcoin` crates and the labeler
action adds `C-hashes` and `C-bitcoin`. Later, a force push removes the
changes to `hashes`, the labeler will not remove the `C-hashes` label.
Now the labels are incorrect.
Note on `pull_request_target`:
We use `on: pull_request_target`, this means the action is run in the
target branch not in the PR branch. So the changes in this patch are not
run when this PR is pushed up. Therefore we cannot test this patch but
have to merge it blindly - please review carefully.
Some further security notes that I learned while doing this patch. The
`pull_request_target` has access to GitHub secrets, and in
`manage-pr.yaml` we run `./contrib/gen_label_config.sh`. This is safe
because its the version of this script on `master` that is run not the
version in the PR (so a malicious attacker cannot patch
`gen_label_config.sh` and have it be run). However we need to be aware
that we do not accidentally merge some funky shell and inadvertently
leak secrets.
Version `0.49` of `kani` broke our daily CI job, there is now a new
release out that fixes the issue.
Remove the explicit `kani` version so we pull the latest version.
Kani version `0.49.0` came out 10 days ago, its odd that our CI just
broke today but in an attempt to see if its release related pin to the
version before the latest release.
The nightly pinning is used by a bunch of different tools outside of
github actions, move the config file to the crate root.
Update the path in the justfile.
Done in preparation for fixing the git pre-commit hook.
Waiting for the fuzzer slows down the dev feedback loop because it makes
CI slow. We still want fuzz coverage but not in a way that slows down
devs.
Instead of running the fuzzer on every PR just run it once a nightly.
As we do for other daily jobs, rename the yaml file to make explicit
what it does.
Open questions:
- This should run for 30 minutes but I can't work out why the current
set up only runs for a shorter time than that?
- Is this less fuzzing i.e., is 30 minutes once a day better or worse that 1
minute 30 times?
32f9b1a231 ci: gate coverage analysis on whether source code changed (Andrew Poelstra)
2203c02347 ci: gate fuzztesting on whether source code changed (Andrew Poelstra)
09f7fc3cff ci: gate CI workflow on source being changed (Andrew Poelstra)
9aca8a18c7 ci: introduce `classify-pr.sh` script which determines whether a PR should have CI run (Andrew Poelstra)
Pull request description:
Fixes#2523
ACKs for top commit:
tcharding:
ACK 32f9b1a231
Tree-SHA512: 7d53365bdf4e8ae5480221e43b4c515d4b16048ec1a00cb62ab65b2d2b91a37f8945c39b619a82bfe4ca74f3508176d9879481b908bb8d3fe8d214b173d3bf18
6b3c8b9f6b ci: use personal access token for daily nightly update (Andrew Poelstra)
775d0c74cd ci: change automatic CI bot committer name to something without special characters (Andrew Poelstra)
aa8655552b ci: change branch of nightly bot to not conflict with rustfmt bot (Andrew Poelstra)
Pull request description:
Rename the bot's committer name to not have special characters, which causes some problem in tcharding's locale; also change the branch name to not overlap with the "weekly CI" job, since it's causing the two jobs to overwrite each others' PRs.
Was part of #2531 but pulled out here to be in a branch that isn't getting clobbered every 24 hours :).
ACKs for top commit:
tcharding:
ACK 6b3c8b9f6b
Tree-SHA512: 2e605991132eb36543d8fb2636bc70c7cc834db6078d2cc6242c65c0f8c5fd270fc9881e6a5c28f5d87a03551fa4f1e0ad3e87c86c226db5e5b8a4c21270ef8c
The chatbot says this should cause CI to trigger on the nightly update
PRs. With the default token this does not happen because it has the
potential to cause recursive CI actions. (How this is avoided by using a
PAT, I don't know, but whatever.)
These are not run in CI since #2353 and are likely to go out of date. If
we want a script that users can run locally then we should create a new
script that wraps our current CI.
Currently we do not build the code in the kani tests when PRs are
pushed, instead we run the verifier once a day. We should at least check
the code builds on each PR. One way to do this is to build the proofs
without running them, `kani --only-codegen` does that.
c6c5f07880 Add automated labeler job (Martin Habovstiak)
Pull request description:
Rather than modifying the labels manually, which we often forget, we can label the PRs automatically. This will make it easier to search PRs.
ACKs for top commit:
apoelstra:
ACK c6c5f07880
tcharding:
ACK c6c5f07880
Tree-SHA512: d3f7ca8c69cc8734d914f871675568ca293e28c3dc7351ee54b9237645204b87d1b47db02863b27d930900ae4b41d6f169f130ed9f4c750d4afbb765921cfb66
Tobin C. Harding
Jose Storopoli
Andrew Poelstra
Update Nightly Rustc Bot
dependabot[bot]
Martin Habovstiak
pool2win