Commit Graph

154 Commits

Author SHA1 Message Date
Matt Corallo 84835f244c Support sendheaders network message decode 2019-05-30 11:25:37 -04:00
Matt Corallo 4f96a87475 Drop LoneHeaders and just use BlockHeader
The protocol has a bug where a 0u8 is pushed at the end of each
block header on the wire in headers messages. WHy this bug came
about is unrealted and shouldn't impact API design.
2019-05-17 17:55:02 -04:00
Dr. Maxim Orlovsky 3c21e301aa Better RawNewtorkMessage deserealization from IO stream (#231)
Follow-up to https://github.com/rust-bitcoin/rust-bitcoin/pull/229

While working with remote peers over the network it is required to deserealize RawNetworkMessage from `TCPStream` to read the incoming messages. These messages can be partial – or one TCP packet can contain few of them. To make the library usable for such use cases, I have implemented the required functionality and covered it with unit tests.

Sample usage:
```rust
fn run() -> Result<(), Error> {
    // Opening stream to the remote bitcoind peer
    let mut stream = TcpStream::connect(SocketAddr::from(([37, 187, 0, 47], 8333));
    let start = SystemTime::now();

    // Constructing and sending `version` message to get some messages back from the remote peer
    let since_the_epoch = start.duration_since(UNIX_EPOCH)
        .expect("Time went backwards");
    let version_msg = message::RawNetworkMessage {
        magic: constants::Network::Bitcoin.magic(),
        payload: message::NetworkMessage::Version(message_network::VersionMessage::new(
            0,
            since_the_epoch.as_secs() as i64,
            address::Address::new(receiver, 0),
            address::Address::new(receiver, 0),
            0,
            String::from("macx0r"),
            0
        ))
    };
    stream.write(encode::serialize(&version_msg).as_slice())?;

    // Receiving incoming messages
    let mut buffer = vec![];
    loop {
        let result = StreamReader::new(&mut stream, None).read_messages();
        if let Err(err) = result {
            stream.shutdown(Shutdown::Both)?;
            return Err(Error::DataError(err))
        }
        for msg in result.unwrap() {
            println!("Received message: {:?}", msg.payload);
        }
    }
}
```

Sample output is the following:
```
Received message: Version(VersionMessage { version: 70015, services: 1037, timestamp: 1548637162, receiver: Address {services: 0, address: [0, 0, 0, 0, 0, 65535, 23536, 35968], port: 33716}, sender: Address {services: 1037, address: [0, 0, 0, 0, 0, 0, 0, 0], port: 0}, nonce: 1370726880972892633, user_agent: "/Satoshi:0.17.99/", start_height: 560412, relay: true })
Received message: Verack
Received message: Alert([1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 255, 255, 127, 0, 0, 0, 0, 255, 255, 255, 127, 254, 255, 255, 127, 1, 255, 255, 255, 127, 0, 0, 0, 0, 255, 255, 255, 127, 0, 255, 255, 255, 127, 0, 47, 85, 82, 71, 69, 78, 84, 58, 32, 65, 108, 101, 114, 116, 32, 107, 101, 121, 32, 99, 111, 109, 112, 114, 111, 109, 105, 115, 101, 100, 44, 32, 117, 112, 103, 114, 97, 100, 101, 32, 114, 101, 113, 117, 105, 114, 101, 100, 0])
```

Working sample code can be found here: https://github.com/dr-orlovsky/bitcoinbigdata-netlistener
2019-02-27 16:41:28 -05:00
Tamás Blummer 1cd2782122
add BIP157 (Client Side Block Filtering) Messages (#225)
* add BIP57 (Client Side Block Filtering) Messages

* rabased after https://github.com/rust-bitcoin/rust-bitcoin/pull/215
2019-02-08 13:00:51 +01:00
Carl Dong 99f63a8ca4 Convert codebase from util::hash to bitcoin_hashes
Also replace unsafe transmute with call to read_u64_into
2019-01-24 16:27:52 -05:00
Dimitris Apostolou 132ca5ea95 Fix typos 2019-01-23 14:17:29 -05:00
Carl Dong 0f42ca69b0 Move relevant names into consensus::encode
- Move network::encodable::* to consensus::encode::*
- Rename Consensus{En,De}codable to {En,De}codable (now under
  consensus::encode)
- Move network::serialize::Error to consensus::encode::Error
- Remove Raw{En,De}coder, implement {En,De}coder for T: {Write,Read}
  instead
- Move network::serialize::Simple{En,De}coder to
  consensus::encode::{En,De}coder
- Rename util::Error::Serialize to util::Error::Encode
- Modify comments to refer to new names
- Modify files to refer to new names
- Expose {En,De}cod{able,er}, {de,}serialize, Params
- Do not return Result for serialize{,_hex} as serializing to a Vec
  should never fail
2018-09-25 21:19:35 +08:00
Carl Dong 8e0e4eb55a Move serialize::BitcoinHash to util:#️⃣:BitcoinHash
- Use Sha256dEncoder for calculating merkle root
- Remove BitcoinHash implementation for Vec<u8>
2018-09-25 21:19:10 +08:00
Carl Dong 97937b1b5f Move network::consensus_params to consensus::params 2018-09-25 21:13:34 +08:00
Carl Dong 7e9d393d03 Remove low-level networking support
- Modify VersionMessage constructor to take in parameters directly that
  would have otherwise been extracted from a Socket (now removed)
2018-09-25 21:13:34 +08:00
Andrew Poelstra ef642295c5 encodable: reject non-compact VarInts on Vec and Box<[T]> lengths 2018-08-24 20:31:46 +00:00
Carl Dong bccdd06794 Replace catch-all match arms with specific ones 2018-08-21 01:58:40 -07:00
Carl Dong 0c172941af Replace serialize::Error::Detail with variants
- Add serialize::Error::ParseFailed(&'static str) variant for
  serialization errors without context
- Add appropriate variants to replace network::Error::Detail for
  serialization error with context
- Remove error method from SimpleDecoders
2018-08-21 01:58:40 -07:00
Carl Dong d12a861f85 Remove unnecessary network::Error::Detail variant 2018-08-21 01:58:40 -07:00
Carl Dong 95303a1d28 Use full path in macros to to eliminate uses 2018-08-21 01:58:40 -07:00
Carl Dong e5b5cbfadb Fix Error type for SimpleDecoder and SimpleEncoder
- Separate serialize::Error and network::Error from util::Error
- Remove unneeded propagate_err and consume_err
- Change fuzzing code to ignore Err type
2018-08-21 01:58:40 -07:00
Aleksei 356d13e465 Add some useful standard trait implementations 2018-08-20 12:11:17 +03:00
Roman Zeyde fdbccf055d
Remove variable shadowing in listener.rs 2018-08-15 09:43:35 +03:00
Carl Dong 70203831d6 Make deserialize error if input bytes not consumed
- Adjust tests in encodable.rs to account for this change
2018-08-14 16:50:33 -07:00
Jean Pierre Dudey b2594087db Use the `?` (try) instead of the `try!` macro.
Signed-off-by: Jean Pierre Dudey <jeandudey@hotmail.com>
2018-08-12 12:47:31 -04:00
Andrew Poelstra ebe5133d1a
Merge pull request #110 from D4nte/regtest
Regtest bech32 address support
2018-08-11 17:21:12 +00:00
Andrew Poelstra 259c5902f1
Merge pull request #116 from jeandudey/2018-08-08-module-docs
Fix modules documentation title.
2018-08-11 17:12:50 +00:00
Jean Pierre Dudey 7ecb6b9dea Refactor and add more documentation for the `Network` type.
Signed-off-by: Jean Pierre Dudey <jeandudey@hotmail.com>
2018-08-10 14:56:19 -04:00
Jean Pierre Dudey 77c185d9ec Fix modules documentation title.
Signed-off-by: Jean Pierre Dudey <jeandudey@hotmail.com>
2018-08-08 17:38:50 -04:00
Savil Srivastava 933dcaeb82 [code hygiene] remove deprecated rustc-serialize
Addresses #96.

Turns out it was being used for hex encoding/decoding, so replaced that with the `hex` crate.

i chose to import the `decode` method as:
```
use hex::decode as hex_decode
```

so that it is clear to the reader what is being decoded when it is called. "decode" is such a generic sounding function name that it would get confusing otherwise.
2018-07-26 09:49:15 -07:00
Franck Royer ba2879cfac Add test for regtest network 2018-07-26 10:51:15 +10:00
Igor Aleksanov 2ff5f8e51e Added module with consensus parameters (#93) 2018-06-23 10:23:34 +02:00
Andrew Poelstra dd176b4177
Merge pull request #84 from popzxc/add-regtest
Added regtest network
2018-06-04 18:42:05 +00:00
Igor Aleksanov 97908ea058 Merge branch 'master' into add-regtest 2018-05-29 12:21:56 +03:00
Igor Aleksanov 34e228c699 Added regtest
Completed regtest integration to the code

Added tests for regtest

Get rid of panics
2018-05-29 12:21:41 +03:00
Tamas Blummer 9f2d737045 add witness inv types 2018-05-28 15:24:35 +02:00
Aleksey Sidorov 5771841144 Replace serde error with the io error. 2018-05-18 12:08:11 +03:00
Aleksei Sidorov 3224cf2b18 Implement `FromStr` for Network constant 2018-05-16 12:22:07 +03:00
Matt Corallo f859dc8b26 Expose VarInt's encoded length 2018-04-02 12:23:05 -04:00
Tamas Blummer ae708447a2 create Address message with SocketAddr, get SocketAddr from Address message. 2018-03-23 15:12:06 +01:00
Tamas Blummer 3351f35583 add documentation 2018-02-28 20:27:52 +01:00
Tamas Blummer f1503866d7 RawNetworkMessage::command should be public. no harm and useful for debug messages. 2018-02-28 10:42:23 +01:00
Andrew Poelstra 9562d8afac
Merge pull request #41 from tamasblummer/minimal_alert_message
minimal implementation of alert message
2018-02-18 15:00:55 +00:00
Andrew Poelstra 9f092a6f31 remove all use of mem::uninitialized and mem::copy_nonoverlapping 2018-02-14 16:53:49 +00:00
Tamas Blummer 9fbe941621 minimal implementation of alert message 2018-02-11 14:08:33 +01:00
Andrew Poelstra 2e7be81ea4 add unit test for overflow panic 2017-06-07 16:09:40 +00:00
Daniel Lockyer a74efe6f8c Sanity checks for vector length 2017-06-05 18:06:30 +01:00
Andrew Poelstra cdb452f79f Sanity-check vector length when deserializing 2017-04-17 00:54:38 +00:00
Steve Bradley f7fed8339d Add GetAddr message 2016-10-10 11:14:12 -04:00
Demur Rumed 5dda3e2602 Replace time with std::time 2016-07-17 04:02:57 +00:00
Andrew Poelstra 1e47019221 cargo-clippy cleanups 2016-06-21 14:35:27 +00:00
Andrew Poelstra f18157e774 Minor code changes for new rustc/clippy. No effects. Update minor version number 2016-02-18 19:44:29 +00:00
Nicholas Alan Cardin e72e01dfa3 Fix decoding of Pong message 2015-12-17 23:47:40 -05:00
Julius Naperkowski 8c707c5bef Added support for mempool messages 2015-11-23 00:58:09 +01:00
Wladimir J. van der Laan 404176a5ee Fix data leak in CommandString::consensus_encode
A fixed buffer of 12 bytes was unsafely copied from the bytes of a
string - if the string was shorter than that, memory from outside would
leak into the packet.

Replace the unsafe copy by a safe loop. Also add a panic if
an attempt is made to use a command string longer than 12 bytes.
2015-11-08 01:14:10 +01:00
Andrew Poelstra 5e03adc9aa Changes for cargo-clippy warnings 2015-10-28 11:27:23 -05:00
Andrew Poelstra c67608a435 Remove `Sized` requirement from ConsensusEncodable 2015-09-20 15:34:13 -05:00
Andrew Poelstra 14db7ee902 Fix for rustc RFC 1214; also add missing docs 2015-09-20 15:30:18 -05:00
Andrew Poelstra 99a4845719 Get library building on stable 2015-09-20 12:22:39 -05:00
Andrew Poelstra 4b1b1c4a74 Fix endianness problems with new librustc ipv6 addresses 2015-05-10 13:08:38 -05:00
Andrew Poelstra 17e27ec09f *** IT COMPILES ON RUSTC NIGHTLY ***
Many unit test failures, but this is progress.
2015-04-10 20:55:59 -05:00
Andrew Poelstra 3117f95b62 Checkpoint commit: into warnings! 2015-04-10 18:15:57 -05:00
Andrew Poelstra 1d78dccb9e Checkpoint commit -- we're onto move errors :D 2015-04-10 13:34:31 -05:00
Andrew Poelstra 5350207ff3 Remove hamcrest; add rust-serde (not compiling right now) 2015-04-09 23:23:53 -05:00
Andrew Poelstra e5a3e84c41 Checkpoint commit (nope, not yet to small commit sizes :))
This is mostly fixing compile errors in `cargo test`. We are down
to 3 in `cargo build` and 14 in `cargo test`, at least for this
round.
2015-04-08 17:23:45 -05:00
Andrew Poelstra 08a20f8764 Checkpoint commit
Work is stalled on some other library work (to give better lifetime
requirements on `eventual::Future` and avoid some unsafety), so
committing here.

There are only three errors left in this round :)

Also all the indenting is done, so there should be no more massive
rewrite commits. Depending how invasive the lifetime-error fixes
are, I may even be able to do sanely sized commits from here on.
2015-04-07 17:52:58 -05:00
Andrew Poelstra 200e0fe8e3 Checkpoint commit
27 files changed, 3944 insertions(+), 3812 deletions(-) :} I've
started doing whitespace changes as well, I want everything to
be 4-space tabs from now on.
2015-04-06 20:51:11 -05:00
Andrew Poelstra 811df8a713 Giant collection of fixes ... we are into lifetime errors now :) 2015-04-05 19:10:37 -05:00
Andrew Poelstra 7b89c15ed5 More changes, incl. dropping DumbHasher in favor of SipHasher
only json stuff left in this round of compiler errors :)
2015-04-05 14:43:44 -05:00
Andrew Poelstra 7738722ab5 Checkpoint commit; tons of disorganized changes for rustc
BTW after all this is done I'm gonna indent the entire codebase...
so `git blame` is gonna be totally broken anyway, hence my
capricious cadence of commits.
2015-04-05 12:58:49 -05:00
Andrew Poelstra 160f2f9ea6 Drop ThinVec, many other changes toward updating for librustc 2015-04-04 22:13:19 -05:00
Andrew Poelstra 7a2cfe95e7 Minor fixes to namespaces 2015-04-04 13:56:30 -05:00
Andrew Poelstra 467f76a37d MemReader/MemWriter become Cursor, other rustc changes 2015-04-04 12:56:40 -05:00
Andrew Poelstra f1aed644c6 More misc cleanup for rustc changes 2015-03-26 14:21:48 -05:00
Andrew Poelstra 6db25db975 Bunch of error type changes 2015-03-26 11:52:20 -05:00
Andrew Poelstra 719f616218 Some more renames for librustc changes 2015-03-26 10:44:49 -05:00
Andrew Poelstra 11dbc717c4 Show -> Debug 2015-03-26 10:35:31 -05:00
Andrew Poelstra df065c143b fix attributes for compiler changes 2015-03-26 10:31:19 -05:00
Andrew Poelstra 2101e4a56d Rename bitcoin-secp256k1-rs to secp256k1 2015-03-26 10:08:36 -05:00
Andrew Poelstra c3377032f8 Many syntax changes for rustc, incomplete 2015-01-18 17:39:51 -06:00
Andrew Poelstra ef11e8273b Destroy socket listener on error rather than trying to reconnect; add #derivings
Reconnecting an existing socket simply was not working; the Rust socket
did not expose any methods for reconnection, so I simply tried calling
connect() again. As near as I can tell, this was a no-op --- which makes
sense because both the sending and receiving threads had their own copy
of the Socket, and it's not clear what the synchronization behaviour
should have been.

Instead if the connection fails, we relay this information to the main
thread, wait for an acknowledgement, then simply destroy the listening
thread. The caller can then simply call `start()` again.
2014-09-10 07:15:48 -05:00
Andrew Poelstra bc7077fac4 Encodable/Decodable improvements; add `user_enum` macro for string enums 2014-09-05 09:16:22 -05:00
Andrew Poelstra 8f5c28a533 Fixes for rustc changes 2014-08-30 16:08:38 -07:00
Andrew Poelstra 6bf553c6fe Add BIP32 key support; unify array newtyping; improve base58 trait
Sorry for so many things in one commit ... it was an iterative
process depending as I worked on BIP32 to get the other stuff
working. (And I was too lazy to separate it out after the fact.)

A breaking change by the array newtyping is that Show for Sha256dHash
now outputs the slice Show. You have to use `{:x}` to get the old hex
output.
2014-08-28 09:49:03 -07:00
Andrew Poelstra 3a921e6b9c Add support for `tx` messages 2014-08-23 16:37:01 -07:00
Andrew Poelstra 7eadf72a1a Add serialize_hex method to do string conversion for serializations 2014-08-22 20:53:49 -07:00
Andrew Poelstra 611b1f57c9 Add a bunch of small hashtypes for cheap'n'collidey hashtables 2014-08-22 11:32:42 -07:00
Andrew Poelstra 1522ab841d Generalize consensus encode/decoders for HashMap 2014-08-18 19:49:09 -07:00
Andrew Poelstra d9d7416e32 Fixes for recent stdlib changes 2014-08-14 15:20:39 -07:00
Andrew Poelstra e01e9ad3be Implement CHECKSIG and CHECKSIGVERIFY
Still need the multisig ops, and p2sh
2014-08-10 13:51:05 -07:00
Andrew Poelstra ecdb750148 Implement script except for crypto opcodes, replace zero_hash by Default
Looks like to implement the crypto opcodes I may need to switch from
rust-crypto to rust-openssl.. or implement RIPEMD-160 for rust-crypto.
In either case I will need to generalize the hash.rs stuff to support
other hashes, so I'm committing here as a checkpoint before doing all
that.
2014-08-05 19:08:06 -07:00
Andrew Poelstra 474d04d154 Manually implement Encodable/Decodable/Show for Network
Since TOML will not encode C-like enums as strings, we do it
ourselves. This is also worthwhile so that we can get the
lowercase "bitcoin" and "testnet" as encodings for the actual
enum values, which are more verbose and camel case.
2014-08-03 06:29:19 -07:00
Andrew Poelstra a6eb33efd0 Actually encode network message
A pretty serious oversight :) this was not noticed because I was
simultaneously dealing with a serious tcp connection bug in rustc,
and I had thought bitcoind's angry disconnects were a further
symptom of that.
2014-08-02 18:22:27 -07:00
Andrew Poelstra 275c93ab6b Add docs for SimpleDecoder and SimpleEncoder traits 2014-08-01 11:16:18 -07:00
Andrew Poelstra a2ce000b2b Revamp Serializable interface to be similar to Encoder/Encodable
This is a massive simplification, fixes a couple endianness bugs (though
not all of them I don't think), should give a speedup, gets rid of the
`serialize_iter` crap.
2014-08-01 09:01:39 -07:00
Andrew Poelstra 020295f8c9 Add a bunch of traits to Network so I can use it as a HashMap key 2014-07-29 15:55:25 -07:00
Andrew Poelstra a34f2642f1 Implement ToJson for BlockHeader
I think this is what I want to do for everything json-visible...perhaps
I will not be able to keep the macro for it though, since there are
some clever variations on it (e.g. blocks should have their header's
hash as a field, txes should appear as txids unless vebose output is
requested, etc.)
2014-07-28 20:12:10 -07:00
Andrew Poelstra b5b7a6b080 Another workaround for https://github.com/rust-lang/rust/issues/15763 2014-07-25 16:50:58 -07:00
Andrew Poelstra cc942a47f3 Workaround for rustc hiccup in `for` loop restructuring, remove assert from deserialization code
Thanks to the assert change there is a segfault happening :(
2014-07-25 15:52:48 -07:00
Andrew Poelstra fe9ca39736 Send new handshake on reconnect 2014-07-24 12:52:28 -07:00
Andrew Poelstra 809bad5ff5 Reconnect on network errors; use Mutexes to control Socket access more precisely 2014-07-24 08:27:40 -07:00
Andrew Poelstra b5c25ff768 Distinguish between network/decode errors in network message decode
It is now visible that EOF (i.e. peer hung up) is interpreted
as a message decode error. Probably what we want to do is reset
the connection on any error. TODO
2014-07-23 11:22:35 -07:00
Andrew Poelstra 9cb86dd21b Change InvalidInput to EndOfFile in several cases 2014-07-22 10:59:58 -07:00
Andrew Poelstra 46969b3396 Replace PatriciaTree with HashMap for UTXO set
We get a speed up (~5%) and memory savings (~10%) on initial sync from
using a HashMap, though it's hard to tell precisely how much savings
because it's quite nonlinear.

I haven't tested de/serialization. Some work needs to be done there to
split up the UTXO set since it takes forever to saveload.
2014-07-20 16:52:00 -07:00
Andrew Poelstra 54e4ea4586 Rename Serializable::hash() to Serializable::bitcoin_hash()
We were conflicting with the Rust stdlib trait Hash, which is used
by various datastructures which need a general hash. Also implement
Hash for Sha256dHash so that we can use bitcoin hashes as keys for
such data structures.
2014-07-19 16:11:55 -07:00