Our decoding code reads bytes in very small chunks. Which is not
efficient when dealing with the OS where the cost of a context switch is
significant. People could already buffer the data but it's easy to
forget it by accident.
This change requires the new `io::BufRead` trait instead of `io::Read`
in all bounds.
Code such as `Transaction::consensus_decode(&mut File::open(foo))` will
break after this is applied, uncovering the inefficiency.
This was originally Kix's work, done before we had the `io` crate.
Changes to `bitcoin` were originally his, any new mistakes are my own.
Changes to `io` are mine.
Co-developed-by: Martin Habovstiak <martin.habovstiak@gmail.com>
There is no advantage in having `io::Read` as opposed to `Read` and
importing the trait. It is surprising that we do so.
Remove `io::` path from `io::Read` and `io::Write`. Some docs keep the
path, leave them as is. Add import `use io::{Read, Write}`.
Refactor only, no logic changes.
We would like all the various hash types to be defined where they
rightly live instead of in the `hash_types` module.
Move transaction hash types to the `transaction` module.
We would like all the various hash types to be defined where they
rightly live instead of in the `hash_types` module.
Move the block hash types to the `block` module. While moving, add full
stops to the rustdoc of each hash.
Re-export _all four_ types from lib.rs (previously `WitnessMerkleNode`
was not re-exported).
We have a convention in `rust-bitcoin` to use external crates directly
when importing them not via `crate::foo`.
Update all the import paths for `io` to use this form.
add371d263 Remove `core2` dependency entirely (Matt Corallo)
b7dd16da99 [IO] Use our own io::Error type (Matt Corallo)
c95b59327a Explicitly use `std::io::Error` when implementing `std` traits (Matt Corallo)
9e1cd372cb Use `io::Error::get_ref()` over `std::error::Error::source()` (Matt Corallo)
3caaadf9bb [IO] Replace the `io::Cursor` re-export with our own `Cursor` (Matt Corallo)
141343edb4 [IO] Move to custom `Read` trait mirroring `std::io::Read` (Matt Corallo)
7395093f94 Stop relying on `Take`'s `by_ref` method (Matt Corallo)
2364e1a877 Stop relying on blanket Read impl for all &mut Read (Matt Corallo)
6aa7ccf841 [IO] Replace `std::io::Sink` usage with our own trivial impl (Matt Corallo)
7eb5d65bda [IO] Provide a macro which implements `io::Write` for types (Matt Corallo)
ac678bb435 [IO] Move to custom `Write` trait mirroring `std::io::Write` (Matt Corallo)
5f2395ce56 Add missing `?Sized` bounds to `io::Write` parameters (Matt Corallo)
2348449d2a Stop relying on `std::io::Write`'s `&mut Write` blanket impl (Matt Corallo)
5e0209569c Use `io::sink` rather than our custom `EmptyWrite` utility (Matt Corallo)
a0ade883b6 [IO] Move io module into selected re-exports (Matt Corallo)
27c7c4e26a Add a `bitcoin_io` crate (Matt Corallo)
Pull request description:
In order to support standard (de)serialization of structs, the
`rust-bitcoin` ecosystem uses the standard `std::io::{Read,Write}`
traits. This works great for environments with `std`, however sadly
the `std::io` module has not yet been added to the `core` crate.
Thus, in `no-std`, the `rust-bitcoin` ecosystem has historically
used the `core2` crate to provide copies of the `std::io` module
without any major dependencies. Sadly, its one dependency,
`memchr`, recently broke our MSRV.
Worse, because we didn't want to take on any excess dependencies
for `std` builds, `rust-bitcoin` has had to have
mutually-exclusive `std` and `no-std` builds. This breaks general
assumptions about how features work in Rust, causing substantial
pain for applications far downstream of `rust-bitcoin` crates.
This is mostly done, I'm still finalizing the `io::Error` commit at the end to drop the `core2` required dep in no-std, but its getting there. Would love further feedback on the approach or code-level review on these first handful of commits.
ACKs for top commit:
tcharding:
ACK add371d263
apoelstra:
ACK add371d263
Kixunil:
ACK add371d263
Tree-SHA512: 18698ea8b1b65108ee0f695d5062d2562c8df2f50bf85d93442648da3b35a4184a5d5d2a493aed0adaadc83f663f0cd2ac735c34941cc9a6fa58d826e548e091
7d695f6b41 Improve public re-exports (Tobin C. Harding)
33774122e0 Remove public re-exports from private module (Tobin C. Harding)
Pull request description:
Improve the public exports in two ways:
1. Inline re-exports into the docs of the module that re-exports them.
2. Separate public and private use statements
Recently we discussed a way to separate the public and private import statements to make the code more clear and prevent `rustfmt` joining them all together.
Separate public exports using a code block and `#[rustfmt::skip]`. Has the nice advantage of reducing the number of `#[doc(inline)]` attributes also.
1. Modules first, as they are part of the project's structure.
2. Private imports
3. Public re-exports (using `rustfmt::skip` to prevent merge)
Use the format
```rust
mod xyz;
mod abc;
use ...;
pub use {
...,
};
```
This patch introduces changes to the rendered HTML docs.
ACKs for top commit:
apoelstra:
ACK 7d695f6b41
Tree-SHA512: dc9121c0fe282e3035d862beadb89e2d5a374a7dab6b1c3147a9b5960f8bc2f5af49892f0f713f55c645c46f53464c32daf390c11d85c75553b3ea7e0efc8246
12d615d900 Use network when calculating difficulty (Tobin C. Harding)
62af5b54f3 Improve difficulty rustdocs (Tobin C. Harding)
Pull request description:
The difficulty is a ratio of the max and current targets, since the max is network specific the difficulty calculation is also network specific.
We already have network specific maximum target constants, use them when calculating the difficulty.
Patch 1 is a trival docs improvement to `block::Header::difficulty`.
ACKs for top commit:
Kixunil:
ACK 12d615d900
apoelstra:
ACK 12d615d900
Tree-SHA512: 8b414c975306667309b0918109b3e5e8774496fc4c0f3413709e95ad7499bebf1a017def4c180a2bb5f1750c69bb505d94c738a28525b7ccc8b36e5e42514000
The difficulty is a ratio of the max and current targets, since the
max is network specific the difficulty calculation is also network
specific.
We already have network specific maximum target constants, use them when
calculating the difficulty.
Improve the public exports in two ways:
1. Inline re-exports into the docs of the module that re-exports them.
2. Separate public and private use statements
Recently we discussed a way to separate the public and private import
statements to make the code more clear and prevent `rustfmt` joining
them all together.
Separate public exports using a code block and `#[rustfmt::skip]`. Has
the nice advantage of reducing the number of `#[doc(inline)]` attributes
also.
1. Modules first, as they are part of the project's structure.
2. Private imports
3. Public re-exports (using `rustfmt::skip` to prevent merge)
Use the format
```rust
mod xyz;
mod abc;
use ...;
pub use {
...,
};
```
This patch introduces changes to the rendered HTML docs.
8eff4d0385 Remove private hex test macro (Tobin C. Harding)
Pull request description:
We have this macro in `hex-conservative` now, remove the version here.
This patch does not change the public API and only touches test code.
ACKs for top commit:
apoelstra:
ACK 8eff4d0385
clarkmoody:
ACK 8eff4d0385
Tree-SHA512: 93a08fff778930071cd1a28c19202e4a94ca8881b2e873538de2e942b71c2cd6184ed6364c572538a8a699295a71761c6f836accaf251a15683138b71f148fab
On our way to v1.0.0 we are defining a standard for our error types,
this includes:
- Uses the following derives (unless not possible, usually because of `io::Error`)
`#[derive(Debug, Clone, PartialEq, Eq)]`
- Has `non_exhaustive` unless we really know we can commit to not adding
anything.
Furthermore, we are trying to make the codebase easy to read. Error code
is write-once-read-many (well it should be) so if we make all the error
code super uniform the users can flick to an error and quickly see what
it includes. In an effort to achieve this I have made up a style and
over recent times have change much of the error code to that new style,
this PR audits _all_ error types in the code base and enforces the
style, specifically:
- Is layed out: definition, [impl block], Display impl, error::Error impl, From impls
- `error::Error` impl matches on enum even if it returns `None` for all variants
- Display/Error impls import enum variants locally
- match uses *self and `ref e`
- error::Error variants that return `Some` come first, `None` after
Re: non_exhaustive
To make dev and review easier I have added `non_exhaustive` to _every_
error type. We can then remove it error by error as we see fit. This is
because it takes a bit of thinking to do and review where as this patch
should not take much brain power to review.
Recently we introduced a bug in the weight/size code, while
investigating I found that our `Transaction`/`Block` weight/size APIs
were in a total mess because:
- The docs were stale
- The concept of weight (weight units) and size (bytes) were mixed up
I audited all the API functions, read some bips (141, 144) and re-wrote
the API with the following goals:
- Use terminology from the bips
- Use abstractions that mirror the bips where possible
55e94b5dea Remove test from test names for Weight type (yancy)
142dde64c3 Use Weight type for stripped_size (yancy)
cb76f3ec43 Add scale_by_witness_factor to Weight type (yancy)
38c9e9947e Add witness scale factor to the Weight type (yancy)
77552987ab Add from_wu_usize to Weight type (yancy)
1a88c887f5 Rename strippedsize to stripped_size (yancy)
3369257c75 Fix grammar (yancy)
Pull request description:
Return Weight type for the strippedize function.
ACKs for top commit:
apoelstra:
ACK 55e94b5dea
tcharding:
ACK 55e94b5dea
Tree-SHA512: ad3e4bc29380f22e20a6302c1b24c201c772be759c655c62ba4717840a01fcaa36f0f8442c9a3ba71c6400d6af47a9a815e6d90877b5f14c6883fb950b9669fd
Throughout the codebase we cast values to `u64` when constructing a
`VarInt`. We can make the code marginally cleaner by adding `From<T>`
impls for all unsigned integer types less than or equal to 64 bits.
Also allows us to (possibly unnecessarily) comment the cast in a single
place.
As part of an ongoing effort to make our error types stable and useful
add a stand set of derives to all error types in the library.
`#[derive(Debug, Clone, PartialEq, Eq)]`
Add `Copy` if possible and the error type does not include
`#[non_exhaustive]`.
If an error type includes `io::Error` it only gets `#[derive(Debug)]`.
We have just released the `hex-conservative` crate, we can now use it.
Do the following:
- Depend on `hex-conservative` in `bitcoin` and `hashes`
- Re-export `hex-conservative` as `hex` from both crate roots.
- Remove all the old hex code from `hashes`
- Fix all the import statements (makes up the bulk of the lines changed
in this patch)
Add a `ValidationError` to the `block` module and remove the two
variants out of `crate::Error`.
This error is only used by the `validate_pow` function, a specific error
better serves our purposes.
Currently we have a mishmash of attribution lines accompanying the SPDX
identifier. These lines are basically meaningless because:
- The date is often wrong
- The original author attributed is not the only contributor to a file
- The term "rust bitcoin developers" is basically just noise
Just remove all the attribution lines and be done with it. While we are
at it add an SPDX line to the few files missing it, whether this license
nonsense is even needed is left as an argument for another day.
If we use `#![cfg_attr(docsrs, feature(doc_auto_cfg))]` instead of
`#![cfg_attr(docsrs, feature(doc_cfg))]` we no longer need to manually
mark types with `#[cfg_attr(docsrs, doc(cfg(feature = "std")))]`.
Sweeeeeet.
We just merged a patch to enable formatting in CI but commit: `05fdead2
Feature: Add difficulty_float method for block::Header.` must have
slipped in.
Run the formatter.
913575ac91 hashes: Run the formatter (Tobin C. Harding)
52c4579057 Enable formatting for hashes (Tobin C. Harding)
3f16b6bf9f util: Run the formatter (Tobin C. Harding)
d210d2ac83 Enable formatting for util (Tobin C. Harding)
5973dce9db blockdata: Run the formatter (Tobin C. Harding)
0dcbed3c7b Enable formatting for blockdata (Tobin C. Harding)
a52746d01c psbt: Run the formatter (Tobin C. Harding)
ef306db5e2 Enable formatting for psbt (Tobin C. Harding)
296f2ed82c Make test panic instead of using code comment (Tobin C. Harding)
3ec8a12428 crypto: Run the formatter (Tobin C. Harding)
c8a3c58786 Enable formatting for crypto (Tobin C. Harding)
314e6786b4 crypto: Add rustfmt::skip attributes (Tobin C. Harding)
450a84f6e8 consensus: Run the formatter (Tobin C. Harding)
89143205f9 Enable formatting for consensus (Tobin C. Harding)
ce773af20f tests: Remove useless use of super imports (Tobin C. Harding)
ef01f4d0f6 consensus: Introduce local variables (Tobin C. Harding)
Pull request description:
One final push crew, 16 patches, only a few are big.
All non-trivial formatting is done in separate patches so the changes can be verified mechanically.
With this applied the whole `rust-bitcoin` crate will be formatted.
Big thanks to everyone for putting up with the ongoing formatting PRs, no-one likes doing these but hopefully this an improvement to the project - especially in helping us get more contributors to the project.
ACKs for top commit:
tcharding:
> ACK [913575a](913575ac91). Went through the workflow locally.
sanket1729:
ACK 913575ac91. Went through the workflow locally.
apoelstra:
ACK 913575ac91
Tree-SHA512: b30eaa2893563155de05f8fa97be4a24a7dd8bf43bb426314c5104598477ca2173af279da796da8b18cc53a0ed525908b3d4edd0504836a443465efa0773632d
Currently we have an associated type on hash types `Inner` with
accompanying methods `into_inner`, `from_inner`, `as_inner`. Also, we
provide a way to create new wrapped hash types. The use of 'inner'
becomes ambiguous with the addition of wrapped types because the inner
could be the inner hash type or the `Inner` byte array of the inner
wrapped hash type.
In an effort to make the API more clear and uniform do the following:
- Rename `Inner` -> `Bytes`
- Rename `*_inner` -> `*_byte_array`
- Rename the inner hash to/from methods to `*_raw_hash`
Correct method prefix `into_` -> `to_` because theses methods convert
owned `Copy` types.
Add the trait Bound `Copy` to the `Bytes` type because we rely on this
trait bound for the conversion methods to be correctly named according
to convention.
Because of the dependency hole created by `secp256k1` this patch changes
the secp dependency to a git tag dependency that includes changes to the
hashes calls required so that we can get green lights on CI in this
repo.
Use of general-purpose integers is often error-prone and annoying. We're
working towards improving it by introducing newtypes.
This adds newtypes for weight and fee rate to make fee computation
easier and more readable. Note however that this dosn't change the type
for individual parts of the transaction since computing the total weight
is not as simple as summing them up and we want to avoid such confusion.
Part of #630
This makes the code less noisy and is a preparation for changing it to
`const`-based literal. Because of the preparation, places that used
variables to store the hex string were changed to constants.
There are still some instances of `Vec::from_hex` left - where they
won't be changeable to `const` and where `hex!` is unavailable
(integration tests). These may be dealt with later.
See also #1189
This renames `Script` to `ScriptBuf` and adds unsized `Script` modeled
after `PathBuf`/`Path`. The change cleans up the API a bit, especially
all functions that previously accepted `&Script` now accept truly
borrowed version. Some functions that perviously accepted `&[u8]` can
now accept `&Script` because constructing it is no loger costly.
We now have an `error` module but the `util::Error`, which is a general
error, is not in it.
Make `Error` more ergonomic to use by doing:
- Move the `util::Error` to `crate::error::Error`
- Re-export it from the crate root since it is our most general error
- Re-export and deprecated it from `util`
The `Version` type uses a signed 32 bit integer inner type but we bit
twiddle as if it was a `u32`. We recently made the inner type private to
hide the data type because of this oddness.
Add methods `from_consensus` and `to_consensus` to facilitate any
possible thing users may want to do with a consensus version value.
The Bitcoin block version is a signed integer for historical reasons,
but we bit twiddle it like an unsigned integer and during consensus
encode/decode we cast the signed value to an unsigned value.
In order to hide this confusion, make the inner value private and add a
couple of constants for v1 and v2 block versions.
Currently the types in the block module have longer names than
necessary, "header" and "version" identifiers contain the word "block",
this is unnecessary because we can write `block::Header` instead of
`BlockHeader` when context is required. This allows us to use the naked
type `Header` inside the `block` module with no loss of clarity.
We are stuck with `BlockHash` because the type is defined along with all
the other hash types in `hash_types`, leave it as is for now but
re-export it from the `block` module to assist in putting types that are
used together in scope in the same place, making import statements more
ergonomic.
Recently we renamed the `hash` module to `merkle_root`, this makes the
public functions provided stutter if used with one layer of path as is
Rust convention:
`merkle_root::bitcoin_merkle_root`
We can improve on this by renaming the functions to 'calculate', then we
get
- `merkle_root::calculate()`
- `merkle_root::calculate_inline()`
The `util::hash` module provides two functions for computing a merkle
root from a list/iterator of hashes.
Rename the module to `merkle_root` and move it to the crate root,
deprecate the original functions.
Done as part of flattening the `util` module.
Tobin C. Harding
apoelstra
Andrew Poelstra
Matt Corallo
Steven Roose
yancy
junderw
Martin Habovstiak