The `DecodeError` (badly named) consensus decodes an object from an
iterator that implements `Read`. The `Read` impl never returns a real IO
error, we use the `io::Error` to temporarily wrap the error returned by
the inner iterator and unwrap it in `IterReader::decode`. As such there
is no reason for the `DecodeError` to hold an `encode::Error`, it can
hold an `encode::ParseError`.
The value of this change is easily seen in the removal of calls to
`unreachable`.
The `consensus::encode::Error` contains an IO error but reading from a
buffer only ever errors for EOF. We converted all instances of EOF to
`MissingData` already so now we can split the IO error apart from the
actual encoding errors variants.
The `io::Error` is troublesome because it contains a bunch of stuff that
never happens when reading from a buffer. However the EOF variant can
occur if the buffer is too short. As an initial step towards reducing
usage of the `io::Error` add a `MissingData` variant to the
`encode::Error` and when converting from an IO error map to
`MissingData` if EOF is encountered.
The `encode::Error::ParseFailed` variant holds an inner string, this is
suboptimal.
In an effort to patch the `encode::Error` while mimizing the diffs
required add a helper function that creates the variant. The benefit is
that later patches that effect this variant will only need to update the
constructor function and not every call site.
Internal change only.
The `consensus` module has a bunch of error types, move them all to a
separate module. Add re-exports so the types are still available at the
same place they were. Make the `error` module private and re-export all
errors from the `consensus` module root.
323e706113 Add rustfmt config option style_edition (Tobin C. Harding)
2e4179ed0f Run the formatter (Tobin C. Harding)
2c40b4f4ec Configure formmater to skip read_compact_size (Tobin C. Harding)
Pull request description:
`rustfmt` is emitting:
Warning: the `version` option is deprecated. Use `style_edition` instead.
As suggested add a config option and set it to 2021.
- Patch 1: Manually configure rustfmt to skip some code
- Patch 2: Run the formmater with current configuration
- Patch 3: Add the new config option (remove old one), introduces no new formatting requirements
ACKs for top commit:
apoelstra:
ACK 323e706113 successfully ran local tests
Tree-SHA512: 7f80cc89f86d2d50936e51704344955fa00532424c29c0ee3fae1a6836e24030f909b770d28da13e1c5efde3d49ad7d52c6d909d120fb09c33abf1755f62cd38
If folk really want to index into a hash they can us `as_byte_array`
then index that.
Includes a bump to the version number of `hashes` to `v0.15.0` - this
is because otherwise `secp` won't build since we are breaking an API
that is used in the current release of secp.
Fix: #3115
At some stage we named the compact encoding `VarInt` (which makes sense
because the compact size encoding is a variable length integer encoding).
However it turns out the term "varint" is used in Core for a different
encoding so this may lead to confusion.
While we fix this naming thing observe also that the `VarInt` type is
unnecessarily complicated, all we need to be able to do is encode and
decode integers in compact form as specified by Core. We can do this
simply by extending our `WriteExt` and `ReadExt` traits.
Add `emit_compact_size` and `read_compact_size` to emit and read compact
endcodings respectively.
Includes addition of `internals::compact_size::encoded_size_const`.
Patch originally written by Steven, Tobin cherry-picked and did a bunch
of impovements after the varint vs compact_size thing (#1016).
ref: https://en.bitcoin.it/wiki/Protocol_documentation#Variable_length_integer
Co-developed-by: Tobin C. Harding <me@tobin.cc>
We would like to add a `emit_varint` function, however doing so requires
that we can get access to the length of a slice when we are encoding it
so we can use `emit_slice` to implement `emit_varint`. It would be
easier to do so if `emit_slice` returned the length of the slice.
In preparation for adding `emit_varint` (and removing the `VarInt` type)
return the encoded length of a slice from `WriteExt::emit_slice`.
(Patch originally written by Steven, cherry-pick and patch description
written by Tobin.)
Co-developed-by: Tobin C. Harding <me@tobin.cc>
579b76b7cb Introduce ToU64 conversion trait (Tobin C. Harding)
Pull request description:
The idea for this was pulled out of Steven's work in #2133
We already explicitly do not support 16 bit machines.
Also, because Rust supports `u182`s one cannot infallibly convert from a `usize` to a `u64`. This is unergonomic and results in a ton of casts.
We can instead limit our code to running only on machines where `usize` is less that or equal to 64 bits then the infallible conversion is possible.
Since 128 bit machines are not a thing yet this does not in reality introduce any limitations on the library.
Add a "private" trait to the `internals` crate to do infallible conversion to a `u64` from `usize`.
Implement it for all unsigned integers smaller than `u64` as well so we have the option to use the trait instead of `u32::from(foo)`.
ACKs for top commit:
Kixunil:
ACK 579b76b7cb
apoelstra:
ACK 579b76b7cb successfully ran local tests
Tree-SHA512: 2eaddfff995987a346e052386c6dfef3510e4732e674e3a2cfab60ee391b4cce1bf7ba4fb2dfd4926f8203d7251eea2198ccb61f0b40332e624c88fda4fa7f48
We already explicitly do not support 16 bit machines.
Also, because Rust supports `u182`s one cannot infallibly convert from a
`usize` to a `u64`. This is unergonomic and results in a ton of casts.
We can instead limit our code to running only on machines where `usize`
is less that or equal to 64 bits then the infallible conversion is
possible.
Since 128 bit machines are not a thing yet this does not in reality
introduce any limitations on the library.
Add a "private" trait to the `internals` crate to do infallible
conversion to a `u64` from `usize`.
Implement it for all unsigned integers smaller than `u64` as well so
we have the option to use the trait instead of `u32::from(foo)`.
Wildcards have been replaced with what is actually used.
In a couple of cases an additional use statement was added to the test
module to import `DisplayHex` which is only used in test, but
previously imported with the wildcard at the top.
the `blockdata` directory is code organisation thing, all the
types/modules are re-exported from other places. In preparation for, and
to make easier, the `primitives` crate smashing work - remove all
explicit usage of `blockdata`.
Note that the few instances remain as they seem required e.g.,
`pub(in crate::blockdata::script)`
Refactor only, no logic changes.
Currently we are defining the two merkle tree hash types in the `block`
module, a better home for them is the `merkle_tree` module.
This is an API breaking change because the types were public in the
`block` module, however the change should/could be unnoticeable to users
if they use the crate level re-export - which is maintained.
051c358bcb Remove deprecated legacy numeric methods (Divyansh Gupta)
Pull request description:
As `rustc 1.79.0-nightly (9d79cd5f7 2024-04-05)` is released which solves the issue mentioned , but the release has deperacted legacy numeric methods.
Thus replaced `u16::max_value()` etc with `u32::MAX` & `core::u16` to directly `u16`.
fix#2639
ACKs for top commit:
tcharding:
ACK 051c358bcb
apoelstra:
ACK 051c358bcb thanks! I will remove an equivalent commit from my #2669
Tree-SHA512: c08c856f7f3b281417c29283351eac5e0f75cc1c8d23d9aae58d969219a327b2337fe57932053e53773ebb9dbec04254f90149266b6639a66c5c09f2ad1675ef
As `rustc 1.79.0-nightly (9d79cd5f7 2024-04-05)` is released which solves the issue mentioned , but the release has deperacted legacy numeric methods.
Thus replace `u16::max_value()` etc with `u32::MAX` & `core::u16` to directly `u16`.
fix#2639
We have `serialize_hex` and `deserialize` but no `deserialize_hex`, add it.
Move the `IterReader` out of `consensus::serde` to the `consensus`
module.
Add some additional logic to the `DecodeError`, I'm not sure why this
wasn't there before?
Use the `HexSliceToBytesIter` by way of the `IterReader` to deserialize
an arbitrary hex string. Add unit tests to check that we consume all
bytes when deserializing a fixed size object (a transaction).
Our decoding code reads bytes in very small chunks. Which is not
efficient when dealing with the OS where the cost of a context switch is
significant. People could already buffer the data but it's easy to
forget it by accident.
This change requires the new `io::BufRead` trait instead of `io::Read`
in all bounds.
Code such as `Transaction::consensus_decode(&mut File::open(foo))` will
break after this is applied, uncovering the inefficiency.
This was originally Kix's work, done before we had the `io` crate.
Changes to `bitcoin` were originally his, any new mistakes are my own.
Changes to `io` are mine.
Co-developed-by: Martin Habovstiak <martin.habovstiak@gmail.com>
There is no advantage in having `io::Read` as opposed to `Read` and
importing the trait. It is surprising that we do so.
Remove `io::` path from `io::Read` and `io::Write`. Some docs keep the
path, leave them as is. Add import `use io::{Read, Write}`.
Refactor only, no logic changes.
We would like all the various hash types to be defined where they
rightly live instead of in the `hash_types` module.
Move the BIP-158 filter hash types to the `bip158` module.
We would like all the various hash types to be defined where they
rightly live instead of in the `hash_types` module.
Move the block hash types to the `block` module. While moving, add full
stops to the rustdoc of each hash.
Re-export _all four_ types from lib.rs (previously `WitnessMerkleNode`
was not re-exported).
We have a convention in `rust-bitcoin` to use external crates directly
when importing them not via `crate::foo`.
Update all the import paths for `io` to use this form.
`std::io::Write` is implemented for all `&mut std::io::Write`. This
makes it easy to have APIs that mix and match owned `Write`s with
mutable references to `Write`s.
However, in the next commit we add our own `Write` trait which we
intend to implement for all `std::io::Write`. Sadly, this is
mutually exclusive with a blanket implementation on our own
`&mut Write`, as that would conflict with an `std::io::Write`
blanket impl.
Thus, in order to use the `Write for all &mut Write` blanket impl
in rust-bitcoin, we'd have to bound all `Write`s by
`std::io::Write`, as we're unable to provide a blanket
`Write for &mut Write` impl.
Here we stop relying on that blanket impl in order to introduce the
new trait in the next commit.
On our way to v1.0.0 we are defining a standard for our error types,
this includes:
- Uses the following derives (unless not possible, usually because of `io::Error`)
`#[derive(Debug, Clone, PartialEq, Eq)]`
- Has `non_exhaustive` unless we really know we can commit to not adding
anything.
Furthermore, we are trying to make the codebase easy to read. Error code
is write-once-read-many (well it should be) so if we make all the error
code super uniform the users can flick to an error and quickly see what
it includes. In an effort to achieve this I have made up a style and
over recent times have change much of the error code to that new style,
this PR audits _all_ error types in the code base and enforces the
style, specifically:
- Is layed out: definition, [impl block], Display impl, error::Error impl, From impls
- `error::Error` impl matches on enum even if it returns `None` for all variants
- Display/Error impls import enum variants locally
- match uses *self and `ref e`
- error::Error variants that return `Some` come first, `None` after
Re: non_exhaustive
To make dev and review easier I have added `non_exhaustive` to _every_
error type. We can then remove it error by error as we see fit. This is
because it takes a bit of thinking to do and review where as this patch
should not take much brain power to review.
Recently we introduced a bug in the weight/size code, while
investigating I found that our `Transaction`/`Block` weight/size APIs
were in a total mess because:
- The docs were stale
- The concept of weight (weight units) and size (bytes) were mixed up
I audited all the API functions, read some bips (141, 144) and re-wrote
the API with the following goals:
- Use terminology from the bips
- Use abstractions that mirror the bips where possible
One encodes to a writer and decodes from a reader, most of the time in
the consensus `Encodable`/`Decodable` traits we use generic `R`/`W` and
variable `r`/`w` but there are other places that use other characters.
While touching these lines note also that there are a bunch of unneeded
`mut`s, I'm not sure why since usually between the compiler and the
linter `mut` is handled correctly.
Make implementations of `Encodable` and `Decodable` uniform by:
- Use R/W and r/w for trait and variable name
- Remove unneeded mut
Throughout the codebase we cast values to `u64` when constructing a
`VarInt`. We can make the code marginally cleaner by adding `From<T>`
impls for all unsigned integer types less than or equal to 64 bits.
Also allows us to (possibly unnecessarily) comment the cast in a single
place.
The `network` module deals with data types and logic related to
internetworking bitcoind nodes, this is commonly referred to as the p2p
layer.
Rename the `network` module to `p2p` and fix all the paths.
Tobin C. Harding
merge-script
Steven Roose
leichak
douglaz
Fmt Bot
Jamil Lambert, PhD
Divyansh Gupta
Liam Aharon
josibake
Matt Corallo