Commit graph

2216 commits

Author SHA1 Message Date
MarcoFalke
0afbeb73cc
Merge #19153: test: mempool compatibility test
16d4b3fd6d test: mempool.dat compatibility between versions (Ivan Metlushko)

Pull request description:

  Rationale: Verify mempool.dat compatibility between versions

  The format of mempool.dat has been changed in #18038
  The tests verifies the fix made in #18807 and ensures that the file format is compatible between current version and v0.19.1
  The test verifies both backward and forward compatibility.

  This PR also adds a log when we fail to add a tx loaded from mempool.dat.
  It was useful when debugging this test and could be potentially useful to debug other scenarios as well.

  Closes #19037

ACKs for top commit:
  Sjors:
    tACK 16d4b3fd6d

Tree-SHA512: 00a38bf528c6478cb0da467af216488f83c1e3ca4d9166c109202ea8284023e99d87a3d6e252c4d88d08d9b5ed1a730b3e1970d6e5c0aef526fa7ced40de7490
2020-06-16 07:25:14 -04:00
MarcoFalke
46bdd4b537
Merge #19178: Make mininode_lock non-reentrant
62068381a3 [tests] Make mininode_lock non-reentrant (John Newbery)
c67c1f2c03 [tests] Don't call super twice in P2PTxInvStore.on_inv() (John Newbery)
9d80762fa0 [tests] Don't acquire mininode_lock twice in wait_for_broadcast() (John Newbery)
edae6075aa [tests] Only acquire lock once in p2p_compactblocks.py (John Newbery)

Pull request description:

  There's no need for mininode_lock to be reentrant.
  Use a simpler non-recursive lock.

ACKs for top commit:
  MarcoFalke:
    ACK 62068381a3 😃
  jonatack:
    ACK 62068381a3

Tree-SHA512: dcbc19e6c986970051705789be0ff7bec70c69cf76d5b468c2ba4cb732883ad512b1de5c3206c2eca41fa3f1c4806999df4cabbf67fc3c463bb817458e59a19c
2020-06-16 05:46:17 -04:00
fanquake
28ce05d06f
Merge #19260: p2p: disconnect peers that send filterclear + update existing filter msg disconnect logic
3a10d935ac [p2p/refactor] move disconnect logic and remove misbehaving (gzhao408)
ff8c430c65 [test] test disconnect for filterclear (gzhao408)
1c6b787e03 [netprocessing] disconnect node that sends filterclear (gzhao408)

Pull request description:

  Nodes that don't have bloomfilters turned on (i.e. no `NODE_BLOOM` service) should disconnect peers that send them `filterclear` P2P messages.

  Non-bloomfilter nodes already disconnect peers for [`filteradd` and `filterload`](19e919217e/src/net_processing.cpp (L2218)), but #8709 removed `filterclear` so it could be used to reset tx relay. This isn't needed now because using `feefilter` message is much better for this purpose (See #19204).

  Also refactors existing disconnect logic for `filteradd` and `filterload` into respective message handlers and removes banning for them.

ACKs for top commit:
  jnewbery:
    Code review ACK 3a10d935ac
  naumenkogs:
    utACK 3a10d93
  gillichu:
    tested ACK: quick test_runner on macOS [`3a10d93`](3a10d935ac)
  MarcoFalke:
    re-ACK 3a10d935ac only change is replacing false with true 🚝

Tree-SHA512: 7aad8b3c0b0e776a47ad52544f0c1250feb242320f9a2962542f5905042f77e297a1486f8cdc3bf0fb93cd00c1ab66a67b2ec426eb6da3fe4cda56b5e623620f
2020-06-16 16:09:22 +08:00
gzhao408
ff8c430c65 [test] test disconnect for filterclear 2020-06-14 11:47:49 -07:00
John Newbery
49236be099 [tests] Don't import asyncio to test magic bytes 2020-06-13 10:49:31 -04:00
MarcoFalke
f154071ec8
Merge #19177: test: Fix and clean p2p_invalid_messages functional tests
af2a145e57 Refactor resource exhaustion test (Troy Giorshev)
5c4648d17b Fix "invalid message size" test (Troy Giorshev)
ff1e7b8844 Move size limits to module-global (Troy Giorshev)
57890abf2c Remove two unneeded tests (Troy Giorshev)

Pull request description:

  This PR touches only the p2p_invalid_messages.py functional test module.  There are two main goals accomplished here.  First, it fixes the "invalid message size" test, which previously made a message that was invalid for multiple reasons.  Second, it refactors the file into a single consistent style.  This file appears to have originally had two authors, with different styles and some test duplication.

  It should now be easier and quicker to understand this module, anticipating the upcoming [BIP324](https://github.com/bitcoin/bitcoin/pull/18242) and [AltNet](https://github.com/bitcoin/bitcoin/issues/18989) changes.

  This should probably go in ahead of #19107, but the two are not strictly related.

ACKs for top commit:
  jnewbery:
    ACK af2a145e57
  MarcoFalke:
    re-ACK af2a145e57 🍦

Tree-SHA512: 9b57561e142c5eaefac5665f7355c8651670400b4db1a89525d2dfdd20e872d6873c4f6175c4222b6f5a8e5210cf5d6a52da69b925b673a2e2ac30a15d670d1c
2020-06-12 15:29:53 -04:00
MarcoFalke
5af16a4db7
Merge #16756: test: Connection eviction logic tests
45eff751c6 Add functional test for P2P eviction logic of inbound peers (Martin Zumsande)

Pull request description:

  This adds a functional test for the eviction logic for inbound peers, which is triggered when the number of maximum connections is exceeded.

  The functional test covers eviction protection for peers that  have sent us blocks or txns recently, or that have faster pings. I couldn't find a way to test the logic of `CConnman::AttemptToEvictConnection` that is based on netgroup (see #14210 for related discussion)

  Fixes #16660 (at least partially).

  [Edit: Earlier, this PR also contained a unit test, which was removed after the discussion]

ACKs for top commit:
  jonatack:
    ACK 45eff751c6
  naumenkogs:
    Tested ACK 45eff75
  fjahr:
    re-ACK 45eff751c6
  andrewtoth:
    re-ACK 45eff751c6

Tree-SHA512: 177208ab6f30dc62da1cc5f51e654f7c9770d8c6b42aca6ae7ecb30e29d3096e04d75739578e7d149a0f29dd92652b4a707e93c0f1be8aa7ed315e6ec3ab07a4
2020-06-12 14:48:20 -04:00
MarcoFalke
19e919217e
Merge #19250: wallet: Make RPC help compile-time static
fadf6bd04f refactor: Remove unused request.fHelp (MarcoFalke)
fad889cbf0 wallet: Make RPC help compile-time static (MarcoFalke)

Pull request description:

  Currently calling `help` on a wallet RPC method will either return `help: unknown command: getnewaddress` or the actual help. This runtime dependency of the help is a bug that complicates any tool that relies on documentation. Also, the code that enables the bug is overly complicated and confusing.

  The fix is split into two commits:

  * First, a commit that can be reviewed with the `--color-moved=dimmed-zebra` option and tested with the included test.
  * Second, a commit that removes the complicated and confusing code.

ACKs for top commit:
  achow101:
    re-ACK fadf6bd04f
  promag:
    Tested ACK fadf6bd04f.

Tree-SHA512: 65d4ff400467f57cb8415c30ce30f814dc76c5c157308b7a7409c59ac9db629e65dfba31cd9c389cfe60a008d3d87787ea0a0e0f2671fd65fd190543c915493d
2020-06-12 13:04:37 -04:00
MarcoFalke
b33136b6ba
Merge #19083: test: msg_mempool, fRelay, and other bloomfilter tests
dca73941eb scripted-diff: rename node to peer for mininodes (gzhao408)
0474ea25af [test] fix race conditions and test in p2p_filter (gzhao408)
4ef80f0827 [test] sending invalid msgs to node with bloomfilters=0 causes disconnect (gzhao408)
497a619386 [test] add BIP 37 test for node with fRelay=false (gzhao408)
e8acc60156 [test] add mempool msg test for node with bloomfilter enabled (gzhao408)

Pull request description:

  This PR adds a few tests that are bloomfilter-related, including behavior for when bloomfilters are turned _off_:
  1. Tests p2p message `msg_mempool`: a node that has `peerbloomfilters` enabled should send its mempool (disabled behavior already tested [here](https://github.com/bitcoin/bitcoin/blob/master/test/functional/p2p_mempool.py)).
  2. Tests that bloomfilter peers with [`fRelay=False`](https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki#extensions-to-existing-messages) in the `version` message should not receive any invs until they set the filter. The rest is the same as what’s already tested in `p2p_filter.py`.
  3. Tests that peers get disconnected if they send `filterload` or `filteradd` p2p messages to a node with bloom filters disabled.
  4. Refactor: renames p2p_mempool.py to p2p_nobloomfilter_messages.py.
  5. Fixes race conditions in p2p_filter.py

ACKs for top commit:
  MarcoFalke:
    ACK dca73941eb only changes is restoring accidentally deleted test 🍮
  jonatack:
    ACK dca73941eb modulo a few nits if you retouch, happy to re-ACK if you take any of them but don't feel obliged to.

Tree-SHA512: 442aeab0755cb8b830251ea170d1d5e6da8ac9029b3276d407a20ee3d588cc61b77b8842368de18c244056316b8c63b911776d6e106bc7c023439ab915b27ad3
2020-06-11 14:35:08 -04:00
MarcoFalke
85f7db2284
Merge #19239: tests: move generate_wif_key to wallet_util.py
3a83a01694 [tests] move generate_wif_key to wallet_util.py (John Newbery)
b216b0b71f [tests] sort imports in rpc_createmultisig.py (John Newbery)
e38081846d Revert "[TESTS] Move base58 to own module to break circular dependency" (John Newbery)

Pull request description:

  generate_wif_key is a wallet utility function. Move it from the EC key module to the wallet util module.

  This fixes the circular dependency issue in #17977

ACKs for top commit:
  MarcoFalke:
    ACK 3a83a01694 🍪

Tree-SHA512: 24985dffb75202721ccc0c6c5b52f1fa5d1ce7963bccde24389feb913cab4dad0c265274ca67892c46c8b64e6a065a0f23263a89be4fb9134dfefbdbe5c7238a
2020-06-11 12:49:38 -04:00
MarcoFalke
fad889cbf0
wallet: Make RPC help compile-time static 2020-06-11 12:38:36 -04:00
fanquake
45a6811d36
Merge #19206: test: Remove leftover comment in mining_basic
fa98e10d5e test: Remove leftover comment in mining_basic (MarcoFalke)
faedb50d89 test: pep-8 mining_basic (MarcoFalke)

Pull request description:

  Remove an accidental leftover comment from #19082, which no longer applies and thus might be confusing

ACKs for top commit:
  adamjonas:
    code review ACK fa98e10

Tree-SHA512: c7f7f8f579b3c6e92f45769be0a7af1a421438a3f5524db5278b2269511a9e0e08f44e3836afb26727644035897ee51ff8296d13ce23030549e7403f57b40e40
2020-06-11 16:38:11 +08:00
John Newbery
3a83a01694 [tests] move generate_wif_key to wallet_util.py
generate_wif_key is a wallet utility function. Move
it from the EC key module to the wallet util module.
2020-06-10 12:10:02 -04:00
John Newbery
b216b0b71f [tests] sort imports in rpc_createmultisig.py 2020-06-10 11:56:44 -04:00
John Newbery
e38081846d Revert "[TESTS] Move base58 to own module to break circular dependency"
This reverts commit c75de5da5f.
2020-06-10 11:54:25 -04:00
gzhao408
dca73941eb scripted-diff: rename node to peer for mininodes
-BEGIN VERIFY SCRIPT-
sed -i 's/FilterNode/P2PBloomFilter/g' test/functional/p2p_filter.py;
sed -i 's/filter_node/filter_peer/g' test/functional/p2p_filter.py;
-END VERIFY SCRIPT-
2020-06-10 07:28:45 -07:00
gzhao408
0474ea25af [test] fix race conditions and test in p2p_filter
-grab mininode_lock for every access to mininode attributes,
otherwise there are race conditions
2020-06-10 07:28:04 -07:00
gzhao408
4ef80f0827 [test] sending invalid msgs to node with bloomfilters=0 causes disconnect
-A node with bloomfilters disabled should disconnect peers that send
msg_mempool, msg_filterload, or msg_filteradd.
-Renamed the test because it now has a wider scope and msg_mempool's
actual functionality makes more sense for p2p_filter.py.
2020-06-10 07:28:04 -07:00
gzhao408
497a619386 [test] add BIP 37 test for node with fRelay=false
A node with fRelay=False should not receive any invs until filter is set using
filterload; all other behavior should be identical.
2020-06-10 07:28:04 -07:00
gzhao408
e8acc60156 [test] add mempool msg test for node with bloomfilter enabled
-msg_mempool is currently only tested with bloomfilter disabled
(node is disconnected) in p2p_mempool.py
-msg_mempool should get mempool txns in response when bloomfilter
is enabled
-edit test that doesn't test msg_mempool as intended
2020-06-10 07:27:58 -07:00
Ivan Metlushko
16d4b3fd6d test: mempool.dat compatibility between versions 2020-06-10 11:06:03 +07:00
Pieter Wuille
c75de5da5f [TESTS] Move base58 to own module to break circular dependency
This breaks the script->key->address->script dependency cycle.
2020-06-09 17:50:50 -07:00
TrentZ
6fc641644f
change blacklist to blocklist
Let's use a more appropriate and clear word and discard the usage of the blacklist. Blocklist is clear. Happy for everyone.
2020-06-09 15:08:41 -05:00
MarcoFalke
297466b49f
Merge #18826: Expose txinwitness for coinbase in JSON form from RPC
34645c4dd0 Test txinwitness is accessible on coinbase vin (Rod Vagg)
3e4421070a Expose txinwitness for coinbase in JSON form (Rod Vagg)

Pull request description:

  ## Rationale

  The CLI can provide you with everything about transactions and blocks that you need to reconstruct the block structure and raw block itself **except** for the witness commitment nonce which is stored in the `scriptWitness` of the coinbase and is not printed. You could manually parse the raw `"hex"` fields for transactions if you really wanted to, but this seems to defeat the point of having a JSONification of the raw block/transaction data.

  Without the nonce you can't:

  1. calculate and validate the witness commitment yourself, you can generate the witness tx merkle root but you don't have the nonce to combine it with
  2. reconstruct the raw block form because you don't have `scriptWitness` stack associated with the coinbase (although you know how big it will be and can guess the common case of `[0x000...000]`)

  I'm building some archiving tooling for block data and being able to do a validated two-way conversion is very helpful.

  ## What

  This PR simply makes the `txinwitness` field not dependent on whether we are working with the coinbase or not. So you get it for the coinbase as well as the rest.

  ## Examples

  Common case of a `[0x000...000]` nonce: 00000000000000000000140a7289f3aada855dfd23b0bb13bb5502b0ca60cdd7

  ```json
        "vin": [
          {
            "coinbase": "0368890904c1fe8d5e2f706f6f6c696e2e636f6d2ffabe6d6d5565843a681160cf7b08b1b74ac90a719e6d6ab28c16d336b924f0dc2fcabdc6010000000000000051bf2ad74af345dbe642154b2658931612a70d195e007add0100ffffffff",
            "txinwitness": [
              "0000000000000000000000000000000000000000000000000000000000000000"
            ],
            "sequence": 4294967295
          }
        ],
  ...
  ```

  Novel nonce value: 000000000000000000008c31945b2012258366cc600a3e9a3ee0598e8f797731

  ```json
        "vin": [
          {
            "coinbase": "031862082cfabe6d6d80c099b5e21f4c186d54eb292e17026932e52b1b807fa1380574c5adc1c843450200000000000000",
            "txinwitness": [
              "5b5032506f6f6c5d5b5032506f6f6c5d5b5032506f6f6c5d5b5032506f6f6c5d"
            ],
            "sequence": 4294967295
          }
        ],
  ...
  ```

  ## Alternatives

  This field could be renamed for the coinbase, `"witnessnonce"` perhaps. It could also be omitted when null/zero (`0x000...000`).

  ## Tests

  This didn't break any tests and I couldn't find an obvious way to include a test for this. If this is desired I'd apreicate some pointers.

ACKs for top commit:
  MarcoFalke:
    ACK 34645c4dd0

Tree-SHA512: b192facc1dfd210a5ec3f0d5d1ac6d0cae81eb35be15eaa71f60009a538dd6a79ab396f218434e7e998563f7f0df2c396cc925cb91619f6841c5a67806148c85
2020-06-08 10:18:42 -04:00
MarcoFalke
b3ec1fe811
Merge #18890: test: disconnect_nodes should warn if nodes were already disconnected
34e641a564 test: Remove unnecessary disconnect_nodes call in rpc_psbt.py (Danny Lee)
e6e7abd51a test: remove redundant two-way disconnect_nodes calls (Danny Lee)
a9bd1f9adf test: warn if nodes not connected before disconnect_nodes (Danny Lee)

Pull request description:

  There's no harm in calling `disconnect_nodes` for nodes that weren't connected (in this case it's a no-op).  However, detecting this case and logging a warning can help ensure that tests are behaving as expected.

  In addition, since `disconnect_nodes` works bidirectionally, I removed all instances of this pattern:

  ```
  disconnect_nodes(self.nodes[0], 1)
  disconnect_nodes(self.nodes[1], 0)
  ```

ACKs for top commit:
  MarcoFalke:
    review ACK 34e641a564 👔
  amitiuttarwar:
    ACK 34e641a564. Thanks for this test improvement!

Tree-SHA512: 344855ceb46c012d43c13d7c09f44d32dcb7645706d10ae1e4645d9edca54c6c6c13fee26b79480755cdfcdf39b4b5770b36bb03ce71ba002d5be8a27fe008af
2020-06-08 09:13:22 -04:00
Troy Giorshev
af2a145e57 Refactor resource exhaustion test
This is a simple refactor of the specified test.  It is now brought in
line with the rest of the tests in the module.  This should make things
easier to debug, as all of the tests are now grouped together at the
top.
2020-06-08 08:52:39 -04:00
MarcoFalke
fa98e10d5e
test: Remove leftover comment in mining_basic 2020-06-08 08:10:37 -04:00
MarcoFalke
faedb50d89
test: pep-8 mining_basic
Can be reviewed with the git options
--word-diff-regex=. --ignore-all-space  -U0
2020-06-08 08:10:23 -04:00
Troy Giorshev
5c4648d17b Fix "invalid message size" test
This test originally made a message with an invalid stated length, and
an invalid checksum.  This was because only the header was changed, but
the checksum stayed the same.  This was fine for now because we check
the header first to see if it has a valid stated size, and we disconnect
if it does not, so we never end up checking for the checksum.  If this
behavior was to change, this test would become a problem.  (Indeed I
discovered this when playing around with this behavior).  By instead
creating a message with an oversized payload from the start, we create a
message with an invalid stated length but a valid checksum, as intended.

Additionally, this takes advantage to the newly module-global
VALID_DATA_LIMIT as opposed to the magic 0x02000000.  Yes, 4MB < 32MiB,
but at the moment when receiving a message we check both, so this makes
the test tighter.
2020-06-08 00:55:34 -04:00
Troy Giorshev
ff1e7b8844 Move size limits to module-global
As well, this renames those variables to match PEP8 and this clears up
the comment relating to VALID_DATA_LIMIT.

Admittedly, this commit is mainly to make the following ones cleaner.
2020-06-08 00:54:52 -04:00
Troy Giorshev
57890abf2c Remove two unneeded tests
Test 1 is a duplicate of test_size() later in the file.  Inexplicably,
this test does not work on macOS, whereas test_size() does.

Test 2 is problematic for two reasons.  First, it always fails with an
invalid checksum, which is probably not what was intended.  Second, it's
not defined at this layer what the behavior should be.  Hypothetically,
if this test was fixed so that it gave messages with valid checksums,
then the message would pass successfully thought the network layer and
fail only in the processing layer.  A priori the network layer has no
idea what the size of a message "actually" is.

The "Why does behavior change at 78 bytes" is because of the following:

print(len(node.p2p.build_message(msg))) # 125
=> Payload size = 125 - 24 = 101
If we take 77 bytes, then there are 101 - 77 = 24 left
That's exactly the size of a header
So, bitcoind deserializes the header and rejects it for some other reason
(Almost always an invalid size (too large))
But, if we take 78 bytes, then there are 101 - 78 = 23 left
That's not enough to fill a header, so the socket stays open waiting for
more data.  That's why we sometimes have to push additional data in
order for the peer to disconnect.

Additionally, both of these tests use the "conn" variable.  For fun, go
look at where it's declared.  (Hint: test_large_inv().  Don't we all
love python's idea of scope?)
2020-06-08 00:54:43 -04:00
Martin Zumsande
45eff751c6 Add functional test for P2P eviction logic of inbound peers 2020-06-06 22:30:29 +02:00
John Newbery
62068381a3 [tests] Make mininode_lock non-reentrant
There's no need for mininode_lock to be reentrant.
Use a simpler non-recursive lock.
2020-06-05 11:01:54 -04:00
John Newbery
c67c1f2c03 [tests] Don't call super twice in P2PTxInvStore.on_inv() 2020-06-05 10:59:40 -04:00
John Newbery
9d80762fa0 [tests] Don't acquire mininode_lock twice in wait_for_broadcast() 2020-06-05 10:58:25 -04:00
John Newbery
edae6075aa [tests] Only acquire lock once in p2p_compactblocks.py 2020-06-05 10:57:13 -04:00
MarcoFalke
fa9604c46f
doc: noban precludes maxuploadtarget disconnects 2020-06-04 16:39:23 -04:00
Wladimir J. van der Laan
39afe5b1c6
Merge #19082: test: Moved the CScriptNum asserts into the unit test in script.py
7daffc6a90 [test] CScriptNum Decode Check as Unit Tests (Gillian Chu)

Pull request description:

  The CScriptNum test (#14816) is a roundtrip test of the test framework. Thus, it would be better suited as a unit test. This is now possible with the introduction of the unit test module for the functional tests. See #18576.

  This PR:
  1. Refactors the CScriptNum tests into 2 unit tests, one in script.py and one in blocktools.py.
  2. Extends the script.py CScriptNum test to trial larger numbers.

ACKs for top commit:
  laanwj:
    ACK 7daffc6a90

Tree-SHA512: 17a04a4bfff1b1817bfc167824c679455d9e06e6e0164c00a7e44f8aa5041c5f5080adcc1452fd80ba1a6d8409f976c982bc481d686c434edf97a5893a32a436
2020-06-04 17:28:55 +02:00
Wladimir J. van der Laan
365f1082e1
Merge #19112: rpc: Remove special case for unknown service flags
fa1433ac1b rpc: Remove special case for unknown service flags (MarcoFalke)

Pull request description:

  The special case to return a bit as an integer is clumsy and undocumented. Probably also irrelevant because there shouldn't currently be a non-misbehaving client that connects to Bitcoin Core and advertises an unknown service flag.

  Thus, simply remove the code.

ACKs for top commit:
  laanwj:
    ACK fa1433ac1b

Tree-SHA512: 942de6a577a9ee076ce12c92be121617640d53ee8c3424064c45a30a7ff789555d3722a4203670768faf81da2a40adfed3ec5cdeb5da06f04be81ddb53b9db7e
2020-06-04 17:16:49 +02:00
Wladimir J. van der Laan
b46fb5cb10
Merge #19131: refactor: Fix unreachable code in init arg checks
eea8114657 build: Enable unreachable-code-loop-increment (Jonathan Schoeller)
d15db4b1fc refactor: Fix unreachable code in init arg checks (Jonathan Schoeller)

Pull request description:

  Closes: #19017

  In #19015 it's been suggested that we add some new compiler warnings to our build. Some of these, such as `-Wunreachable-code-loop-increment`, generate warnings. We'll likely want to fix these up if we're going to turn these warnings on.

  ```shell
  init.cpp:969:5: warning: loop will run at most once (loop increment never executed) [-Wunreachable-code-loop-increment]
       for (const auto& arg : gArgs.GetUnsuitableSectionOnlyArgs()) {
       ^~~
   1 warning generated.
   ```
   aa8d76806c/src/init.cpp (L968-L972)

  To fix this, collect all errors, and output them in a single error message after the loop completes. This resolves the unreachable code warning, and avoids popup hell that could result from outputting a seperate message for each error or warning one by one.

ACKs for top commit:
  laanwj:
    Code review ACK eea8114657
  hebasto:
    re-ACK eea8114657, only suggested changes applied since the [previous](https://github.com/bitcoin/bitcoin/pull/19131#pullrequestreview-421772387) review.

Tree-SHA512: 2aa3ceb7fab581b6ba2580900668388d8eba1c3001c8ff9c11c1f4a9a10fbc37f30e590249862676858446e3f4950140a252953ba1643ba3bfd772f8eae20583
2020-06-04 16:27:53 +02:00
fanquake
234fabab90
Merge #18210: test: type hints in Python tests
bd7e530f01 This PR adds initial support for type hints checking in python scripts. (Kiminuo)

Pull request description:

  This PR adds initial support for type hints checking in python scripts.

  Support for type hints was introduced in Python 3.5. Type hints make it easier to read and review code in my opinion. Also an IDE may discover a potential bug sooner. Yet, as PEP 484 says: "It should also be emphasized that Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention."

  [Mypy](https://mypy.readthedocs.io/en/latest/index.html) is used in `lint-python.sh` to do the type checking. The package is standard so there is little chance that it will be abandoned. Mypy checks that type hints in source code are correct when they are not, it fails with an error.

  **Notes:**

  * [--ignore-missing-imports](https://mypy.readthedocs.io/en/latest/command_line.html#cmdoption-mypy-ignore-missing-imports) switch is passed on to `mypy` checker for now. The effect of this is that one does not need `# type: ignore` for `import zmq`. More information about import processing can be found [here](https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports). This can be changed in a follow-up PR, if it is deemed useful.
  * We are stuck with Python 3.5 until 04/2021 (see https://packages.ubuntu.com/xenial/python3). When Python version is bumped to 3.6+, one can change:
      ```python
      _opcode_instances = []  # type: List[CScriptOp]
      ```
      to
      ```python
      _opcode_instances:List[CScriptOp] = []
      ```
      for type hints that are **not** function parameters and function return types.

  **Useful resources:**

  * https://docs.python.org/3.5/library/typing.html
  * https://www.python.org/dev/peps/pep-0484/

ACKs for top commit:
  fanquake:
    ACK bd7e530f01 - the type checking is not the most robust (there are things it fails to detect), but I think this is worth adopting (in a limited capacity while we maintain 3.5 compat).
  MarcoFalke:
    ACK bd7e530f01 fine with me

Tree-SHA512: 21ef213915fb1dec6012f59ef17484e6c9e0abf542a316b63d5f21a7778ad5ebabf8961ef5fc8e5414726c2ee9c6ae07c7353fb4dd337f8fcef5791199c8987a
2020-06-03 22:19:52 +08:00
Gillian Chu
7daffc6a90 [test] CScriptNum Decode Check as Unit Tests
Migrates the CScriptNum decode tests into a unit test, and moved some
changes made in #14816. Made possible by the integration of
test_framework unit testing in #18576. Further extends the original
test with larger ints, similar to the scriptnum_tests.cpp file. Adds
test to blocktools.py testing fn create_coinbase() with CScriptNum
decode.
2020-06-03 07:18:01 -07:00
MarcoFalke
bdedfcf076
Merge #18974: test: Check that invalid witness destinations can not be imported
facede18a4 test: Check that invalid witness destinations can not be imported (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK facede18a4 -- thanks for adding!

Tree-SHA512: 87000606fac2e6f2780ca75cdeeb2dc1f0528d9b8f13e4156e8304ce7a6b1eb014781b6f0c59d11544bf360ba3dc5f99549470b0876132e189b9107f2c6bb38d
2020-06-03 07:31:06 -04:00
MarcoFalke
3657aee2d2
Merge #18982: wallet: Minimal fix to restore conflicted transaction notifications
7eaf86d3bf trivial: Suggested cleanups to surrounding code (Russell Yanofsky)
b604c5c8b5 wallet: Minimal fix to restore conflicted transaction notifications (Russell Yanofsky)

Pull request description:

  This fix is a based on the fix by Antoine Riard (ariard) in https://github.com/bitcoin/bitcoin/pull/18600.

  Unlike that PR, which implements some new behavior, this just restores previous wallet notification and status behavior for transactions removed from the mempool because they conflict with transactions in a block. The behavior was accidentally changed in two `CWallet::BlockConnected` updates: a31be09bfd and 7e89994133 from https://github.com/bitcoin/bitcoin/pull/16624, causing issue https://github.com/bitcoin/bitcoin/issues/18325.

  The change here could be improved and replaced with a more comprehensive cleanup, so it includes a detailed comment explaining future considerations.

  Fixes #18325

  Co-authored-by: Antoine Riard (ariard)

ACKs for top commit:
  jonatack:
    Re-ACK 7eaf86d3bf
  ariard:
    ACK 7eaf86d, reviewed, built and ran tests.
  MarcoFalke:
    ACK 7eaf86d3bf 🍡

Tree-SHA512: 9a1efe975969bb522a9dd73c41064a9348887cb67883cd92c6571fd2df4321b9f4568363891abdaae14a3b9b168ef8142e95c373fc04677e46289b251fb84689
2020-06-02 18:11:52 -04:00
MarcoFalke
45a1489997
Merge #19122: test: Add missing sync_blocks to wallet_hd
fa7d3a8890 test: Add missing sync_blocks to wallet_hd (MarcoFalke)
eeeed51f58 test: pep-8 wallet_hd (MarcoFalke)

Pull request description:

  This fixes the `                                   test_framework.authproxy.JSONRPCException: non-final (-26)` error when node 1 is one block behind of node 0. (height 122 vs 123)

ACKs for top commit:
  promag:
    Code review ACK fa7d3a8890.

Tree-SHA512: b549dce2e08c58b949168ed2013bfa176802c963d0d7e890f643c8792da5dade14d91441dfa74372f1f1d34d696e8900a2b60b4861c0ba2dce99f2a633ab27ff
2020-06-02 07:13:51 -04:00
Kiminuo
bd7e530f01 This PR adds initial support for type hints checking in python scripts.
Support for type hints was introduced in Python 3.5. Type hints make it easier to read and review code in my opinion. Also an IDE may discover a potential bug sooner. Yet, as PEP 484 says: "It should also be emphasized that Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention."

Mypy is used in lint-python.sh to do the type checking. The package is standard so there is little chance that it will be abandoned. Mypy checks that type hints in source code are correct when they are not, it fails with an error.

Useful resources:

* https://docs.python.org/3.5/library/typing.html
* https://www.python.org/dev/peps/pep-0484/
2020-06-02 08:03:02 +02:00
Jonathan Schoeller
d15db4b1fc refactor: Fix unreachable code in init arg checks
Building with -Wunreachable-code-loop-increment causes a warning
due to always returning on the first iteration of the loop that
outputs errors on invalid args.

Collect all errors, and output them in a single error message
after the loop completes, resolving the warning and avoiding
popup hell by outputting a seperate message for each error.
2020-06-02 06:20:04 +10:00
fanquake
a8327fd71f
Merge #19072: doc: Expand section on Getting Started
facef3d413 doc: Explain that anyone can work on good first issues, move text to CONTRIBUTING.md (MarcoFalke)
fae2fb2a19 doc: Expand section on Getting Started (MarcoFalke)
100000d1b2 doc: Add headings to CONTRIBUTING.md (MarcoFalke)
fab893e0ca doc: Fix unrelated typos reported by codespell (MarcoFalke)

Pull request description:

  Some random doc changes:

  * Add sections to docs, so that they can be linked to
  * Explain that anyone (even maintainers) are allowed to work on good first issues
  * Expand section on Getting Started slightly

ACKs for top commit:
  hebasto:
    ACK facef3d413
  fanquake:
    ACK facef3d413

Tree-SHA512: 8998e273a76dbf4ca77e79374c14efe4dfcc5c6df6b7d801e1e1e436711dbe6f76b436f9cbc6cacb45a56827babdd6396f3bd376a9426ee7be3bb9b8a3b8e383
2020-06-01 15:38:57 +08:00
MarcoFalke
07d0e0d59f
Merge #19044: net processing: Add support for getcfilters
9e36067d8c [test] Add test for cfilters. (Jim Posen)
11106a4722 [net processing] Message handling for getcfilters. (Jim Posen)
e535670726 [indexes] Fix default [de]serialization of BlockFilter. (Jim Posen)
bb911ae7f5 [refactor] Pass CNode and CConnman by reference (John Newbery)

Pull request description:

  Support `getcfilters` requests when `-peerblockfilters` is set.

  Does not advertise compact filter support in version messages.

ACKs for top commit:
  Empact:
    re-Code Review ACK 9e36067d8c
  MarcoFalke:
    re-ACK 9e36067d8c , only change is adding commit "[refactor] Pass CNode and CConnman by reference" 🥑
  jkczyz:
    ACK 9e36067d8c
  fjahr:
    Code review ACK 9e36067d8c

Tree-SHA512: b45b42a25905ef0bd9e195029185300c86856c87f78cbe17921f4a25e159ae0f6f003e61714fa43779017eb97cd89d3568419be88e47d19dc8095562939e7887
2020-05-31 18:20:17 -04:00
MarcoFalke
fa7d3a8890
test: Add missing sync_blocks to wallet_hd 2020-05-31 07:51:44 -04:00