milksad

Custom fork of rust-bitcoin with unsafe modifications for higher speed. Unsuitable for production.

Go to file

Dr. Maxim Orlovsky 86055d9df5 Merge rust-bitcoin/rust-bitcoin#672 : New Witness struct to improve ser/de perfomance `106acdc3ac` Add fuzzing for Witness struct (Riccardo Casatta) `2fd0125bfa` Introduce Witness struct mainly to improve ser/de performance while keeping most usability. (Riccardo Casatta) Pull request description: At the moment the Witness struct is `Vec<Vec<u8>>`, the vec inside a vec cause a lot of allocations, specifically: - empty witness -> 1 allocation, while an empty vec doesn't allocate, the outer vec is not empty - witness with n elements -> n+1 allocations The proposed Witness struct contains the serialized format of the witness. This reduces the allocations to: - empty witness -> 0 allocations - witness with n elements -> 1 allocation for most common cases (you don't know how many bytes is long the entire witness beforehand, thus you need to estimate a good value, not too big to avoid wasting space and not too low to avoid vector reallocation, I used 128 since it covers about 80% of cases on mainnet) The inconvenience is having slightly less comfortable access to the witness, but the iterator is efficient (no allocations) and you can always collect the iteration to have a Vec of slices. If you collect the iteration you end up doing allocation anyway, but the rationale is that it is an operation you need to do rarely while ser/de is done much more often. I had to add a bigger block to better see the improvement (ae860247e191e2136d7c87382f78c96e0908d700), these are the results of the benches on my machine: ``` RCasatta/master_with_block test blockdata::block::benches::bench_block_deserialize ... bench: 5,496,821 ns/iter (+/- 298,859) test blockdata::block::benches::bench_block_serialize ... bench: 437,389 ns/iter (+/- 31,576) test blockdata::block::benches::bench_block_serialize_logic ... bench: 108,759 ns/iter (+/- 5,807) test blockdata::transaction::benches::bench_transaction_deserialize ... bench: 670 ns/iter (+/- 49) test blockdata::transaction::benches::bench_transaction_get_size ... bench: 7 ns/iter (+/- 0) test blockdata::transaction::benches::bench_transaction_serialize ... bench: 51 ns/iter (+/- 5) test blockdata::transaction::benches::bench_transaction_serialize_logic ... bench: 13 ns/iter (+/- 0) branch witness_with_block (this one) test blockdata::block::benches::bench_block_deserialize ... bench: 4,302,788 ns/iter (+/- 424,806) test blockdata::block::benches::bench_block_serialize ... bench: 366,493 ns/iter (+/- 42,216) test blockdata::block::benches::bench_block_serialize_logic ... bench: 84,646 ns/iter (+/- 7,366) test blockdata::transaction::benches::bench_transaction_deserialize ... bench: 648 ns/iter (+/- 77) test blockdata::transaction::benches::bench_transaction_get_size ... bench: 7 ns/iter (+/- 0) test blockdata::transaction::benches::bench_transaction_serialize ... bench: 50 ns/iter (+/- 5) test blockdata::transaction::benches::bench_transaction_serialize_logic ... bench: 14 ns/iter (+/- 0) ``` With an increased performance to deserialize a block of about 21% and to serialize a block of about 16% (seems even higher than expected, need to do more tests to confirm, I'll appreciate tests results from reviewers) ACKs for top commit: apoelstra: ACK `106acdc3ac` sanket1729: ACK `106acdc3ac` dr-orlovsky: utACK `106acdc3ac` Tree-SHA512: e4f23bdd55075c7ea788bc55846fd9e30f9cb76d5847cb259bddbf72523857715b0d4dbac505be3dfb9d4b1bcae289384ab39885b4887e188f8f1c06caf4049a		2021-12-30 01:55:44 +02:00
.github/workflows	Add fuzzing for Witness struct	2021-12-28 09:56:41 +01:00
contrib	no_std support	2021-07-15 09:04:49 +02:00
embedded	Clean up embedded test memory configuration	2021-08-02 22:09:14 +02:00
examples	no_std support	2021-07-15 09:04:49 +02:00
fuzz	Add fuzzing for Witness struct	2021-12-28 09:56:41 +01:00
src	Merge rust-bitcoin/rust-bitcoin#672 : New Witness struct to improve ser/de perfomance	2021-12-30 01:55:44 +02:00
test_data	Add tests from BIP341	2021-12-12 21:49:36 +05:30
.gitignore	Removed Intellij idea files from .gitignore	2020-10-11 21:51:03 -05:00
CHANGELOG.md	fix changelog date	2021-08-06 20:30:38 +02:00
Cargo.toml	Fixed docs.rs metadata	2021-12-17 13:49:54 +01:00
LICENSE	Add LICENSE file with CC0 in it	2014-07-18 17:37:13 -07:00
README.md	Document lack of support for 16-bit pointers	2021-09-20 21:31:46 +02:00
clippy.toml	Fixed a bunch of clippy lints, added clippy.toml	2021-12-21 22:50:13 +01:00

README.md

Rust Bitcoin Library

Library with support for de/serialization, parsing and executing on data structures and network messages related to Bitcoin.

Documentation

Supports (or should support)

De/serialization of Bitcoin protocol network messages
De/serialization of blocks and transactions
Script de/serialization
Private keys and address creation, de/serialization and validation (including full BIP32 support)
PSBT creation, manipulation, merging and finalization
Pay-to-contract support as in Appendix A of the Blockstream sidechains whitepaper

For JSONRPC interaction with Bitcoin Core, it is recommended to use rust-bitcoincore-rpc.

Known limitations

Consensus

This library must not be used for consensus code (i.e. fully validating blockchain data). It technically supports doing this, but doing so is very ill-advised because there are many deviations, known and unknown, between this library and the Bitcoin Core reference implementation. In a consensus based cryptocurrency such as Bitcoin it is critical that all parties are using the same rules to validate data, and this library is simply unable to implement the same rules as Core.

Given the complexity of both C++ and Rust, it is unlikely that this will ever be fixed, and there are no plans to do so. Of course, patches to fix specific consensus incompatibilities are welcome.

Support for 16-bit pointer sizes

16-bit pointer sizes are not supported and we can't promise they will be. If you care about them please let us know, so we can know how large the interest is and possibly decide to support them.

Documentation

Currently can be found on docs.rs/bitcoin. Patches to add usage examples and to expand on existing docs would be extremely appreciated.

Contributing

Contributions are generally welcome. If you intend to make larger changes please discuss them in an issue before PRing them to avoid duplicate work and architectural mismatches. If you have any questions or ideas you want to discuss please join us in #bitcoin-rust on libera.chat.

Minimum Supported Rust Version (MSRV)

This library should always compile with any combination of features on Rust 1.29.

Because some dependencies have broken the build in minor/patch releases, to compile with 1.29.0 you will need to run the following version-pinning command:

cargo update -p cc --precise "1.0.41" --verbose

In order to use the use-serde feature or to build the unit tests with 1.29.0, the following version-pinning commands are also needed:

cargo update --package "serde" --precise "1.0.98"
cargo update --package "serde_derive" --precise "1.0.98"

For the feature base64 to work with 1.29.0 we also need to pin byteorder:

cargo update -p byteorder --precise "1.3.4"

Installing Rust

Rust can be installed using your package manager of choice or rustup.rs. The former way is considered more secure since it typically doesn't involve trust in the CA system. But you should be aware that the version of Rust shipped by your distribution might be out of date. Generally this isn't a problem for rust-bitcoin since we support much older versions than the current stable one (see MSRV section).

Building

The library can be built and tested using cargo:

git clone git@github.com:rust-bitcoin/rust-bitcoin.git
cd rust-bitcoin
cargo build

You can run tests with:

cargo test

Please refer to the cargo documentation for more detailed instructions.

Pull Requests

Every PR needs at least two reviews to get merged. During the review phase maintainers and contributors are likely to leave comments and request changes. Please try to address them, otherwise your PR might get closed without merging after a longer time of inactivity. If your PR isn't ready for review yet please mark it by prefixing the title with WIP: .

Policy on Altcoins/Altchains

Patches which add support for non-Bitcoin cryptocurrencies by adding constants to existing enums (e.g. to set the network message magic-byte sequence) are welcome. Anything more involved will be considered on a case-by-case basis, as the altcoin landscape includes projects which frequently appear and disappear, and are poorly designed anyway and keeping the codebase maintainable is a large priority.

In general, things that improve cross-chain compatibility (e.g. support for cross-chain atomic swaps) are more likely to be accepted than things which support only a single blockchain.

Release Notes

See CHANGELOG.md.

Licensing

The code in this project is licensed under the Creative Commons CC0 1.0 Universal license.