082e185711 Add `consensus_decode_from_finite_reader` optimization (Dawid Ciężarkiewicz)
Pull request description:
As things are right now, memory exhaustion protection in `Decodable`
is based on checking input-decoded lengths against arbitrary limits,
and ad-hoc wrapping collection deserialization in `Take`.
The problem with that are two-fold:
* Potential consensus bugs due to incorrect limits.
* Performance degradation when decoding nested structured,
due to recursive `Take<Take<..>>` readers.
This change introduces a systematic approach to the problem.
A concept of a "size-limited-reader" is introduced to rely on
the input data to finish at enforced limit and fail deserialization.
Memory exhaustion protection is now achived by capping allocations
to reasonable values, yet allowing the underlying collections
to grow to accomodate rare yet legitmately oversized data (with tiny
performance cost), and reliance on input data size limit.
A set of simple rules allow avoiding recursive `Take` wrappers.
Fix#997
ACKs for top commit:
apoelstra:
ACK 082e185711
tcharding:
ACK 082e185711
Tree-SHA512: fa04b62a4799c9a11c5f85ec78a18fa9c2cd4819c83a0d6148fbb203c6fa15c2689cb0847e612b35b8c285a756d81690b31a9bede4486b845f0c16b9fcc6d097
As things are right now, memory exhaustion protection in `Decodable`
is based on checking input-decoded lengths against arbitrary limits,
and ad-hoc wrapping collection deserialization in `Take`.
The problem with that are two-fold:
* Potential consensus bugs due to incorrect limits.
* Performance degradation when decoding nested structured,
due to recursive `Take<Take<..>>` readers.
This change introduces a systematic approach to the problem.
A concept of a "size-limited-reader" is introduced to rely on
the input data to finish at enforced limit and fail deserialization.
Memory exhaustion protection is now achived by capping allocations
to reasonable values, yet allowing the underlying collections
to grow to accomodate rare yet legitmately oversized data (with tiny
performance cost), and reliance on input data size limit.
A set of simple rules allow avoiding recursive `Take` wrappers.
Fix#997
88ce8fe923 Match against an optional single trailing colon (Tobin C. Harding)
Pull request description:
Currently we allow multiple trailing colons when matching within the
`check_format_non_negative` macro. We can be more restrictive with no
loss of usability.
Use `$(;)?` instead of `$(;)*` to match against 0 or 1 semi-colons
instead of 0 or more.
Done as part of the [edition 2018 checklist](https://github.com/rust-bitcoin/rust-bitcoin/issues/510).
ACKs for top commit:
Kixunil:
ACK 88ce8fe923
apoelstra:
ACK 88ce8fe923
Tree-SHA512: 4409c094f6a0aa49ddebdad850fd1d5a31a57dae8828f5a1db0ee5a855e1bce9e43aea69fa0b4d132068c3a43f1f62d35409b9ac5b32ed876e4dd586829e8e68
553a6813c5 Do not pin transitive ryu dependency (Tobin C. Harding)
Pull request description:
We do not need to pin the `ryu` transitive dependency now that MSRV is not 1.29.
ACKs for top commit:
apoelstra:
ACK 553a6813c5
Kixunil:
ACK 553a6813c5
Tree-SHA512: 072a2fea39a0405424579e0e34603f27f12a5271a8979d6f9204b3114827b2c1931105df418ccb5071b641a108b7db803eec953ced04a670509d21652c6a6ca4
Currently we allow multiple trailing colons when matching within the
`check_format_non_negative` macro. We can be more restrictive with no
loss of usability.
Use `$(;)?` instead of `$(;)*` to match against 0 or 1 semi-colons
instead of 0 or more.
07c75304d2 Refactor address byte swapping (Tobin C. Harding)
Pull request description:
Refactor address byte swapping
When encoding a `network::Address` two of the fields are encoded
big-endian instead of little-endian as is done by `consensus_encode`. In
order to achieve this we have a helper function `addr_to_be` that swaps
the bytes. This function is miss-named because it is not converting to a
specific endian-ness (which implies different behaviour on machines with
different endian-ness) but is reversing the byte order irrespective of
the underlying architecture.
- Remove function `addr_to_be`
- Inline the endian-ness code when encoding an address
- Remove TODO and use `to_be_bytes` when encoding port
- Add a function for reading big-endian bytes `read_be_address`
- Use `read_be_address` when decoding `Address` and `Addrv2`
Refactor only, no logic changes. Code path is already covered by
unit tests.
ACKs for top commit:
apoelstra:
ACK 07c75304d2
Kixunil:
ACK 07c75304d2
Tree-SHA512: 186bc86512e264a7b306f3bc2e18d1619f3cd84fc54412148cfc2663e8d6e9616ea9e2fe19eafec72d76cc11367a9b39cac2b73210d9e43eb8f453bd253b33de
97a5bb1439 Implement std::error::source codebase wide (Tobin C. Harding)
0a9191b429 Add parenthesis around left hand side of companion (Tobin C. Harding)
7cf8af2f86 Put Error impl block below Display (Tobin C. Harding)
2384712364 Re-order Display match arms (Tobin C. Harding)
Pull request description:
Now that we have MSRV of 1.41.1 we should use `source` instead of `cause`. Audit the whole codebase and implement `source` for _every_ error type we have.
The first three patches are preparatory cleanup, patch 3 is particularly shameful (adds parenthesis to make my editor work).
CC @Kixunil because he is championing the error stuff.
ACKs for top commit:
apoelstra:
ACK 97a5bb1439
Tree-SHA512: 46313a28929445f32e01e30ca3b0246b30bc9d5e43db5754d4b441e9c30d3e427efaf247100eb6b452f98beec5a4fcde1daba7943a772114aa34f78ab52cbc60
9896f27eae psbt: Improve documentation (Tobin C. Harding)
33a50831ce sighash: Improve documentation (Tobin Harding)
Pull request description:
Done while working on sighash and PSBT signing. Just the usual docs fixes. Note, does not do the whole `psbt` module just the file mentioned.
ACKs for top commit:
apoelstra:
ACK 9896f27eae
Tree-SHA512: 5fbfa258cdb216189922a49a42b7ab9fb78faeee72d82f8cb99a1b3d930d170074013e317b0e7af259a404ac4db93841b4d2b525e933c5e145da71e7522800fd
58f94bee9b Remove sha256t_hash_newtype macro (Tobin C. Harding)
Pull request description:
Since commit `commit 275adc6c335a4326699cfbd444949e1725864ea1` on `bitcoin_hashes` we have the identical implementation of the macro `sha256t1_hash_newtype` in this crate and in `bitcoin_hashes`.
Remove the `sha256t_hash_newtype` macro from this crate in favour of the one in `bitcoin_hashes`.
ACKs for top commit:
apoelstra:
ACK 58f94bee9b
sanket1729:
ACK 58f94bee9b
Tree-SHA512: ec08fd25c1cca71a07ea61cb5838ce8962daae7cbb84d8beccc3d0d285439909721edd643292a8f3f6989e1c2c41fda9addfd5cdb063ef53ebc6ef646da79cf3
90b4f1cde8 Clear TapTreeIter clippy warning (Tobin C. Harding)
e6084a1af8 Improve documentation around EcdsaSig (Tobin Harding)
Pull request description:
Do a couple of trivial docs fixes, done during other work.
- Patch 1 improves docs on the `EcdsaSig` struct
- Patch 2 clears a clippy warning during docs build - no sure if the solution is the best available though
ACKs for top commit:
apoelstra:
re-ACK 90b4f1cde8
sanket1729:
ACK 90b4f1cde8
Tree-SHA512: 0647dc2e6550938ccca658a9dddffba7175d5c4eb8cec0e165d3a7fa8f2b1dfb902e795aca77d96a6c31092baf64244fa1d7151a304134d3b1895619a2823338
7ca30b6aa8 Move Address::payload_as_bytes to Payload::as_bytes (Fredrik Meringdal)
525ea00e0f Make Address::get_payload_bytes public (Fredrik Meringdal)
Pull request description:
Hi, thanks for the amazing work on this crate.
I am trying to upgrade from v0.27 to v0.28, but unable to do so because the `Address::get_payload_bytes` was made private. My use-case is that I have a script hash address and an `Address` and need to compare the two, and in order to do so I need access to the payload bytes of `Address`.
I hope you will consider making this function public again 🙏
ACKs for top commit:
apoelstra:
ACK 7ca30b6
tcharding:
ACK 7ca30b6aa8
sanket1729:
ACK 7ca30b6aa8. Sorry for the delay and congratz on your first time contribution
Tree-SHA512: 02af4565853d93506751ed7cb004f52cb5d8c7936067e06b3e237b448ccdf5716470448eeccbe211958e095b66bb37c7027800c0470c6988dc18d8bd5b48f459
Parenthesis are not needed around this expression but my editor is going
mad and cannot format the code without them. Since it does not hurt
readability add parenthesis around the expression.
When encoding a `network::Address` two of the fields are encoded
big-endian instead of little-endian as is done by `consensus_encode`. In
order to achieve this we have a helper function `addr_to_be` that swaps
the bytes. This function is miss-named because it is not converting to a
specific endian-ness (which implies different behaviour on machines with
different endian-ness) but is reversing the byte order irrespective of
the underlying architecture.
- Remove function `addr_to_be`
- Inline the endian-ness code when encoding an address
- Remove TODO and use `to_be_bytes` when encoding port
- Add a function for reading big-endian bytes `read_be_address`
- Use `read_be_address` when decoding `Address` and `Addrv2`
Refactor only, no logic changes. Code path is already covered by
unit tests.
Since commit `commit 275adc6c335a4326699cfbd444949e1725864ea1` on
`bitcoin_hashes` we have the identical implementation of the macro
`sha256t1_hash_newtype` in this crate and in `bitcoin_hashes`.
Remove the `sha256t_hash_newtype` macro from this crate in favour of the
one in `bitcoin_hashes`.
Clippy emits warning:
public documentation for `script_leaves` links to private item `TapTreeIter`
I'm not exactly sure why this is but adding the generic type place
holder clears the warning.
Improve documentation in `psbt/mod.rs` by doing:
- Use full sentences (full stops and capitalisation)
- Use 100 line column width
- Use back ticks and links as appropriate
- Use `Errors` section
- Use third person tense to describe functions
Improve the rustdoc documentation in the `sighash` module by doing:
- Improve grammar
- Use full sentences (full stops and capitalisation)
- Use 100 line column width
- Use back ticks and links as appropriate
- Improve correctness of `SigHashCache::new` function
9f0c687d89 Enable edition 2018 (Tobin C. Harding)
dca0d67771 Fix in preparation for next edition (Tobin C. Harding)
Pull request description:
This PR supersedes #635 at the permission of @Kixunil in the thread of that PR.
Do a minimal set of changes to enable edition 2018. Patch 1 is the biggest change set, it is done with `cargo`, no other manual changes are included in patch 1. It can verified by running `cargo fix --edition` on master and checking the diffs are the same.
Patch 2 enables 2018 and includes all the manual changes required to get the code to build (with _all_ the feature combinations :)
ACKs for top commit:
dunxen:
re-ACK 9f0c687
apoelstra:
re-ACK 9f0c687d89
RCasatta:
ACK 9f0c687d89,
Tree-SHA512: 7c23554adb4c1dd932af1e80f04397bad0418b4fdae2d0fe243c3f19ba1169686a386bffd38a3c02871c7544b5a7bc8af525b50617a2695726408c091700f081
Use cargo to upgrade from edition 2015 to edition 2018.
cargo fix --edition
No manual changes made. The result of the command above is just to fix
all the use statements (add `crate::`) and fix the fully qualified path
formats i.e., `::Foo` -> `crate::Foo`.
0ab5eeac81 Add method to push an ECDSA sig + sighash type byte on a witness (Matt Corallo)
Pull request description:
We do this all over the place in rust-lightning, and its probably
the most common thing to do with a `Witness` so I figured I'd
upstream the util method to do this. It also avoids an allocation
compared to the naive approach of `SerializedSignature.to_vec()`
with two pushes, which is nice.
ACKs for top commit:
apoelstra:
ACK 0ab5eeac81
tcharding:
ACK 0ab5eeac81
dunxen:
ACK 0ab5eea
Tree-SHA512: d4042eb3f2bc7e6beecd2c17eaeef1fdb7d096b8889a3709924734739557031f338525b685d32ac0dc26404afefd4f2d48defd1753ea8935e54e3a82987feb8c
We do this all over the place in rust-lightning, and its probably
the most common thing to do with a `Witness` so I figured I'd
upstream the util method to do this. It also avoids an allocation
compared to the naive approach of `SerializedSignature.to_vec()`
with two pushes, which is nice.
091688c078 Remove irrelevant notes about version pinning (Martin Habovštiak)
Pull request description:
These are no longer relevant for MSRV 1.41.1+
ACKs for top commit:
tcharding:
ACK 091688c078
sanket1729:
ACK 091688c078
Tree-SHA512: d40ea1cd788ada3172dbedcfd2f9f1472222941b6e3753feee8e2f0c929abe3d12ac9a91488d332a180c9685d53b8c1daf96fe309a586d8f93e49066d5c1dfcc
4f1200d629 Added `amount::Display` - configurable formatting (Martin Habovstiak)
Pull request description:
This significatnly refactors the formatting code to make formatting more
configurable. The main addition is the `Display` type which is a
builder that can configure denomination or other things (possibly more
in the future).
Further, this makes all representations of numbers minimal by default,
so should be documented as a possibly-breaking change.
Because of the effort to support all other `fmt::Formatter` options this
required practically complete rewrite of `fmt_satoshi_in`. As a
byproduct I took the opportunity of removing one allocation from there.
Closes#709
ACKs for top commit:
tcharding:
ACK 4f1200d629
dr-orlovsky:
ACK 4f1200d629
sanket1729:
ACK 4f1200d629
Tree-SHA512: 3fafdf63fd720fd4514e026e9d323ac45dfcd3d3a53a4943178b1e84e4cf7603cb6235ecd3989d46c4ae29453c4b0bb2f2a5996fbddf341cd3f68dc286062144
2c28d3b448 Fix handling of empty slice in Instructions (Martin Habovštiak)
e6ff754b73 Fix doc of take_slice_or_kill (Martin Habovštiak)
0ec6d96a7b Cleanup after `Instructions` refactoring (Martin Habovstiak)
bc763259fe Move repeated code to functions in script (Martin Habovstiak)
1f55edf718 Use iterator in `blockdata::script::Instructions` (Martin Habovstiak)
Pull request description:
This refactors `blockdata::script::Instructions` to use
`::core::slice::Iter<'a, u8>` instead of `&'a [u8]` to better express
the intention and to avoid some slicing mistakes. Similarly to a
previous change this uses a macro to deduplicate the common logic and
the new `read_uint_iter` internal function to automatically advance the
iterator.
Addresses:
https://github.com/rust-bitcoin/rust-bitcoin/pull/662#pullrequestreview-768320603
ACKs for top commit:
tcharding:
ACK 2c28d3b448
sanket1729:
ACK 2c28d3b448. I don't want to hold ACKs on minor things as they can be in a fixup later.
Tree-SHA512: 9dc770b9f7958efbd0df2cc2d3546e23deca5df2f94ea2c42b089df628f4b99f08032ca4aa8822caf6643a8892903e1bda41228b78c8519b90bcaa1255d9acc6
76fcf81474 Override default visit_byte_buf on Script (ass3rt)
add100c20d Removed reimplementations of default methods (ass3rt)
7db03f27e4 Disable Serde's default-features (ass3rt)
Pull request description:
With this patch, existing users of the `use-serde` feature will no longer be
compiling with `serde/std` enabled, but this allows dependent projects
to import serde and enable `serde/alloc` as required by some no-std targets.
ACKs for top commit:
Kixunil:
ACK 76fcf81474
tcharding:
ACK 76fcf81474
apoelstra:
ACK 76fcf81474
Tree-SHA512: 5748e64e1f91f19dbfbf32bead6e6d759e448e92ed0dab731b3059f6b37bd811fad6654edc8fbd113e3be17fefaf9fc4912145d6b61484ced0517712361ecfdc
7854bd7918 Fix `no_std` MSRV Fixes#690, #947 (mcroad)
Pull request description:
`rust-bitcoin` does not work with rust 1.29 under a `no_std` environment. This could be considered a bug. However, `no_std` support is a recent addition and this is likely not breaking anyone's builds.
A decision needs to be made, either `no_std` MSRV is the current stable version while keeping the `std` MSRV as 1.29, or it needs to be fixed.
This pr adds `no_std` to the 1.29 test suite.
This came as I try to get rust-bitcoin/rust-miniscript#277 working and got stuck on the issue of testing `no_std` under 1.29.
ACKs for top commit:
Kixunil:
ACK 7854bd7918
tcharding:
ACK 7854bd7918
sanket1729:
ACK 7854bd7918
apoelstra:
ACK 7854bd7918
Tree-SHA512: 1614fb2193f760ed340592bdb94d076066f6f783bc1dc2b145d97f7151a28316e56b1975f1ad948460eb26db04e7e9382e60076686a681e46dcf33521fda5fca
a0a83e18f6 README: Use correct license (Tobin C. Harding)
5c363bc785 README: Update badge to Rust 1.41.1 (Tobin C. Harding)
Pull request description:
We have two small bugs in our README badges
- Patch 1: sets the MSRV badge to the new 1.41.1
- Patch 2: sets the license badge to the correct license, CC0
(To verify that the changes render correctly, click on the 'from' branch name above.)
ACKs for top commit:
sanket1729:
ACK a0a83e18f6
apoelstra:
ACK a0a83e18f6
Tree-SHA512: e216cd36ac3ea477146c1a8ac842718dfe07762dd47378b7862a6ad5f92a08c25ed92133543fdc083d5edd80d08e7da46e1f87067c14d3240b3e411b2a05baf3
831b0267de Use contains() instead of manual range (Tobin C. Harding)
6410095687 Use chunks_exact (Tobin C. Harding)
3a0097ba49 Use trim_start_matches (Tobin C. Harding)
0a19710906 Use vec! macro instead of new followed by push (Tobin C. Harding)
Pull request description:
Now that 0.28 is out we do not need to support Rust 1.29 on `master`.
Remove trivial MSRV `TODO`s from the code. (All these changes only rely on MSRV bumping to 1.31 so are easily within bounds.)
ACKs for top commit:
Kixunil:
ACK 831b0267de
sanket1729:
ACK 831b0267de
Tree-SHA512: f1ea594216ba7dfa24696b964ce296a8aea72dd2e16e11d3a43fe8b90c851abf59b1612b2b1311146e8070112f3834762584e4f0515b8f546f72af169eb4bda9
c5bb689b60 Update MSRV in clippy.toml (Martin Habovštiak)
Pull request description:
We forgot about this during MSRV bump.
ACKs for top commit:
tcharding:
ACK c5bb689b60
DanGould:
ACK c5bb689b60
sanket1729:
ACK c5bb689b60
Tree-SHA512: 3acd2f3b28fdb7d450923dcec39af980cef95d997b54c9e48f09d78d41ba2b301a1a7637d629cadcaee87b3c9cb2e80c08f971bdf7de06d42fe50158278661f4
5afb0eaf40 API to get an iterator for funding utxos in psbt (violet360)
Pull request description:
### Current status
The API returns a vector of UTXOs and has return type `Result<Vec<&TxOut>, Error>`
### Expected
The return statement should be of type `sighash::Prevouts` as pointed in #849
ACKs for top commit:
Kixunil:
ACK 5afb0eaf40
tcharding:
ACK 5afb0eaf40
sanket1729:
ACK 5afb0eaf40. Thanks for being patient with this.
Tree-SHA512: 724fc3dffdbb1331584f89bbe84527e1af0d193a344fe43b36f2f2a628652d259001a3abf6b3909df53524cd3fbdbe3af760b7004d40d3bee1848fbb83efff5b
Andrew Poelstra
Dawid Ciężarkiewicz
Tobin C. Harding
sanket1729
Riccardo Casatta
Matt Corallo
Matt Corallo
Martin Habovštiak
Fredrik Meringdal