Commit graph

25189 commits

Author SHA1 Message Date
Andrew Chow 74507ce71e walletdb: Remove BerkeleyBatch friend class from BerkeleyDatabase 2020-07-22 23:30:19 -04:00
Andrew Chow 00f0041351 No need to check for duplicate fileids in all dbenvs
Since we have .walletlock in each directory, we don't need the duplicate
fileid checks across all dbenvs as it shouldn't be possible anyways.
2020-07-22 23:30:19 -04:00
Andrew Chow d86efab370 walletdb: Move Db->open to BerkeleyDatabase::Open
Instead of opening the Db handle in BerkeleyBatch, make BerkeleyDatabase
do that.
2020-07-22 23:30:19 -04:00
Andrew Chow 4fe4b3bf1b walletdb: track database file use as m_refcount within BerkeleyDatabase
Instead of having BerkeleyEnvironment track the file use count, make
BerkeleyDatabase do it itself.
2020-07-22 23:30:19 -04:00
Andrew Chow 65fb8807ac Combine BerkeleyEnvironment::Verify into BerkeleyDatabase::Verify 2020-07-22 23:30:19 -04:00
Samuel Dobson 9d4b3d86b6
Merge #19334: wallet: Introduce WalletDatabase abstract class
d416ae560e walletdb: Introduce WalletDatabase abstract class (Andrew Chow)
2179dbcbcd walletdb: Add BerkeleyDatabase::Open dummy function (Andrew Chow)
71d28e7cdc walletdb: Introduce AddRef and RemoveRef functions (Andrew Chow)
27b2766384 walletdb: Move BerkeleyDatabase::Flush(true) to Close() (Andrew Chow)

Pull request description:

  A `WalletDatabase` abstract class is created from `BerkeleyDatabase` and is implemented by `BerkeleyDatabase`. First, to get to the point that this is possible, 4 functions need to be added to `BerkeleyDatabase`: `AddRef`, `RemoveRef`, `Open`, and `Close`.

  First the increment and decrement of `mapFileUseCount` is refactored into separate functions `AddRef` and `RemoveRef`.

  `Open` is introduced as a dummy function. This will raise an exception so that it always fails.

  `Close` is refactored from `Flush`. The `shutdown` argument in `Flush` is removed and instead `Flush(true)` is now the `Close` function.

  Split from #18971

  Requires #19325

ACKs for top commit:
  ryanofsky:
    Code review ACK d416ae560e. Only changes since last review were rebasing after base PR #19334 merge, and adding cs_db lock in BerkeleyDatabase destructor, which should avoid races accessing env->m_databases and env->m_fileids
  fjahr:
    Code review ACK d416ae560e
  meshcollider:
    Code review & test run ACK d416ae560e

Tree-SHA512: 98d05ec093d7446c4488e2b0914584222a331e9a2f4d5be6af98e3f6d78fdd8e75526c12f91a8a52d4820c25bce02aa02aabe92d38bee7eb2fce07d0691b7b0d
2020-07-23 15:22:25 +12:00
Hennadii Stepanov 2aac093a3d
test: Add test coverage for -networkactive option 2020-07-22 22:55:48 +03:00
Hennadii Stepanov 3c58129b12
net: Log network activity status change unconditionally 2020-07-22 22:55:47 +03:00
Hennadii Stepanov 62fe6aa87e
net: Add -networkactive option
The `setnetworkactive' RPC command is already present.
This new option allows to start the client with disabled p2p network
activity for testing or reindexing.
2020-07-22 22:55:11 +03:00
Wladimir J. van der Laan ccef10261e
Merge #18044: Use wtxid for transaction relay
0a4f1422cd Further improve comments around recentRejects (Suhas Daftuar)
0e20cfedb7 Disconnect peers sending wtxidrelay message after VERACK (Suhas Daftuar)
cacd85209e test: Use wtxid relay generally in functional tests (Fabian Jahr)
8d8099e97a test: Add tests for wtxid tx relay in segwit test (Fabian Jahr)
9a5392fdf6 test: Update test framework p2p protocol version to 70016 (Fabian Jahr)
dd78d1d641 Rename AddInventoryKnown() to AddKnownTx() (Suhas Daftuar)
4eb515574e Make TX_WITNESS_STRIPPED its own rejection reason (Suhas Daftuar)
97141ca442 Delay getdata requests from peers using txid-based relay (Suhas Daftuar)
46d78d47de Add p2p message "wtxidrelay" (Suhas Daftuar)
2d282e0cba ignore non-wtxidrelay compliant invs (Anthony Towns)
ac88e2eb61 Add support for tx-relay via wtxid (Suhas Daftuar)
8e68fc246d Add wtxids to recentRejects instead of txids (Suhas Daftuar)
144c385820 Add wtxids of confirmed transactions to bloom filter (Suhas Daftuar)
85c78d54af Add wtxid-index to orphan map (Suhas Daftuar)
08b39955ec Add a wtxid-index to mapRelay (Suhas Daftuar)
60f0acda71 Just pass a hash to AddInventoryKnown (Suhas Daftuar)
c7eb6b4f1f Add wtxid to mempool unbroadcast tracking (Amiti Uttarwar)
2b4b90aa8f Add a wtxid-index to the mempool (Suhas Daftuar)

Pull request description:

  Using txids (a transaction's hash, without witness) for transaction relay is problematic, post-segwit -- if a peer gives us a segwit transaction that fails policy checks, it could be because the txid associated with the transaction is definitely unacceptable to our node (regardless of the witness), or it could be that the transaction was malleated and with a different witness, the txid could be accepted to our mempool.

  We have a bloom filter of recently rejected transactions, whose purpose is to help us avoid redownloading and revalidating transactions that fail to be accepted, but because of this potential for witness malleability to interfere with relay of valid transactions, we do not use the filter for segwit transactions.  This issue is discussed at some length in #8279.  The effect of this is that whenever a segwit transaction that fails policy checks is relayed, a node would download that transaction from every peer announcing it, because it has no way presently to cache failure.  Historically this hasn't been a big problem, but if/when policy for accepting segwit transactions were to change (eg taproot, or any other change), we could expect older nodes talking to newer nodes to be wasting bandwidth because of this.

  As discussed in that issue, switching to wtxid-based relay solves this problem -- by using an identifier for a transaction that commits to all the data in our relay protocol, we can be certain if a transaction that a peer is announcing is one that we've already tried to process, or if it's something new.  This PR introduces support for wtxid-based relay with peers that support it (and remains backwards compatible with peers that use txids for relay, of course).

  Apart from code correctness, one issue to be aware of is that by downloading from old and new peers alike, we should expect there to be some bandwidth wasted, because sometimes we might download the same transaction via txid-relay as well as wtxid-relay.  The last commit in this PR implements a heuristic I want to analyze, which is to just delay relay from txid-relay peers by 2 seconds, if we have at least 1 wtxid-based peer.  I've just started running a couple nodes with this heuristic so I can measure how well it works, but I'm open to other ideas for minimizing that issue.  In the long run, I think this will be essentially a non-issue, so I don't think it's too big a concern, we just need to bite the bullet and deal with it during upgrade.

  Finally, this proposal would need a simple BIP describing the changes, which I haven't yet drafted.  However, review and testing of this code in the interim would be welcome.

  To do items:
  - [x] Write BIP explaining the spec here (1 new p2p message for negotiating wtxid-based relay, along with a new INV type)
  - [ ] Measure and evaluate a heuristic for minimizing how often a node downloads the same transaction twice, when connected to old and new nodes.

ACKs for top commit:
  naumenkogs:
    utACK 0a4f1422cd
  laanwj:
    utACK 0a4f1422cd

Tree-SHA512: d8eb8f0688cf0cbe9507bf738e143edab1f595551fdfeddc2b6734686ea26e7f156b6bfde38bad8bbbe8bec1857c7223e1687f8f018de7463dde8ecaa8f450df
2020-07-22 20:58:55 +02:00
Wladimir J. van der Laan 1397afc5ec
Merge #19526: log: Avoid treating remote misbehvior as local system error
fa56eda58e log: Avoid treating remote misbehvior as local system error (MarcoFalke)
fa492895b5 refactor: Switch ValidationState mode to C++11 enum class (MarcoFalke)

Pull request description:

  When logging failures of `CheckBlockHeader` (high-hash), they are always logged as system error. This is problematic for several reasons:

  * Submitting a blockheader that fails `CheckBlockHeader` over RPC will result in a debug log line that starts with `ERROR`. Proper behaviour should be to log not anything and instead only return the failure reason to the RPC user. This pull does not fix this issue entirely, but is a good first step in the right direction.

  * A misbehaving peer that sends us an invalid block header that fails `CheckBlockHeader` will result in a debug log line that starts with `ERROR`. Proper behavior should be to log the remote peer misbehavior if logging for that category was enabled. This pull fixes this issue for `CheckBlockHeader` and other functions can be adjusted as well if needed in follow-ups. This should be a good first step in the right direction.

ACKs for top commit:
  practicalswift:
    re-ACK fa56eda58e

Tree-SHA512: 9793191f5cb57bdff7c93926e94877e8ca2ef89dcebcf9eb155899c733961839ec7c3f9b9f001dc082ada4234fe6e75f6df431301678d6822325840771166d77
2020-07-22 19:48:55 +02:00
Wladimir J. van der Laan 93decbc7a4
Merge #19370: Static asserts for consistency of fee defaults
1554b54d47 Static asserts for consistency of fee defaults. (Daniel Kraft)

Pull request description:

  This adds `static_assert`'s that ensure that the default values given for fee levels in the wallet (minimum fee and incremental feerate increase) are at least as high as the corresponding levels configured in the core node policy.  Since the core policy values are enforced by the network, it makes sense for the wallet to be conservative and above (or at least not below) this.

ACKs for top commit:
  laanwj:
    code review ACK 1554b54d47, these assumptions seem straightforward

Tree-SHA512: 50e5adf082f467062334377f82a3ee75bcfd436afc65bd0eb33c8d0549d6d90fd1f48c31f60cabe523eb59be9efa8ae0879e9e09cd51ca9c1bd466631ce03cf4
2020-07-22 19:25:07 +02:00
Wladimir J. van der Laan c7b4968552
Merge #19553: build: pass -fcommon when building genisoimage
2a701a1c42 build: pass -fcommon when building genisoimage (fanquake)

Pull request description:

  Starting with the 10.1 release, GCC [defaults to -fno-common](https://gcc.gnu.org/gcc-10/porting_to.html). This causes
  linking issues when building genisoimage:
  ```bash
  [ 98%] Building C object genisoimage/CMakeFiles/genisoimage.dir/checksum.o
  [100%] Linking C executable genisoimage
  /usr/bin/ld: CMakeFiles/genisoimage.dir/apple.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
  /usr/bin/ld: CMakeFiles/genisoimage.dir/boot.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
  /usr/bin/ld: CMakeFiles/genisoimage.dir/desktop.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
  /usr/bin/ld: CMakeFiles/genisoimage.dir/dvd_file.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
  ```

  Rather than patching genisoimage further, just pass -fcommon to preserve
  the legacy GCC behaviour.

  Noticed [while testing #19530](https://github.com/bitcoin/bitcoin/pull/19530#issuecomment-659802512).

ACKs for top commit:
  laanwj:
    ACK 2a701a1c42
  hebasto:
    ACK 2a701a1c42, tested on Fedora 32 (x86_64, GCC 10.1.1):

Tree-SHA512: 0e6dd1b4c7c3f5b6997616da27a7ea54f757c81677766ee6813c776356527787a32b959f834e699fee9798553b13bcb142469f442c5e7a04f810f2c06e3a9505
2020-07-22 13:57:22 +02:00
fanquake ef3d4ce4c3
build: call AC_PATH_TOOL for dsymutil in macOS cross-compile
While testing #19530 I noticed that we couldn't call dsymutil after LTO:
```bash
../libtool: line 10643: x86_64-apple-darwin16-dsymutil: command not found
```

This updates configure to call `AC_PATH_TOOL` so that we end up with the
full path to dsymutil, similar to `otool` and `install_name_tool`, ie:
`/bitcoin/depends/x86_64-apple-darwin16/share/../native/bin/x86_64-apple-darwin16-otool`.
2020-07-22 18:22:56 +08:00
fanquake 2031aa92a3
Merge #19562: test: Fix fuzzer compilation on macOS
c8992e8959 test: Fix fuzzer compilation on macOS fixes #19557 (freenancial)

Pull request description:

  fixes #19557

  Before the fix:
  ```
  ➜  bitcoin git:(fix-fuzzer-macos) make
  Making all in src
    CXX      test/fuzz/addition_overflow-addition_overflow.o
  In file included from test/fuzz/addition_overflow.cpp:7:
  ./test/fuzz/util.h:335:13: error: no matching function for call to 'AdditionOverflow'
          if (AdditionOverflow((uint64_t)fuzzed_file->m_offset, random_bytes.size())) {
              ^~~~~~~~~~~~~~~~
  ./test/fuzz/util.h:201:16: note: candidate template ignored: deduced conflicting types for parameter 'T' ('unsigned long long' vs. 'unsigned long')
  NODISCARD bool AdditionOverflow(const T i, const T j) noexcept
                 ^
  ./test/fuzz/util.h:346:13: error: no matching function for call to 'AdditionOverflow'
          if (AdditionOverflow(fuzzed_file->m_offset, n)) {
              ^~~~~~~~~~~~~~~~
  ./test/fuzz/util.h:201:16: note: candidate template ignored: deduced conflicting types for parameter 'T' ('long long' vs. 'long')
  NODISCARD bool AdditionOverflow(const T i, const T j) noexcept
                 ^
  ```

  After the fix:
  ```
  ➜  bitcoin git:(fix-fuzzer-macos) ./configure --enable-fuzz --with-sanitizers=fuzzer,address,undefined CC=/usr/local/opt/llvm/bin/clang CXX=/usr/local/opt/llvm/bin/clang++ --disable-asm && make clean && make -j5
  ...
  ...
    CXXLD    test/fuzz/uint256_deserialize
  Making all in doc/man
  make[1]: Nothing to be done for `all'.
  make[1]: Nothing to be done for `all-am'.
  ```

ACKs for top commit:
  fanquake:
    ACK c8992e8959 - tested that compiling works on macOS.
  MarcoFalke:
    review ACK c8992e8959

Tree-SHA512: 965cdc61b30db0e2209c91b29f0d42de927a9a5b85e1e70f22d1452e0955f876726c7a8c1d1a5f448f12bf24eec3000802071cd4ae28d8605343fd43d174ca84
2020-07-22 18:03:41 +08:00
fanquake 597d2f905e
Merge #19548: fuzz: add missing overrides to signature_checker
c0f09c2c9d fuzz: add missing overrides to signature_checker (Jon Atack)

Pull request description:

  These functions in `fuzz/signature_checker.cpp` override virtual member functions and should be marked `override` instead of `virtual`, which is for introducing a new virtual function. The overridden virtual functions are in `script/interpreter.h:151/156/161`.

  Also, per MarcoFalke suggestion, add missing parentheses in `fuzz/scriptnum_ops.cpp` and remove useless `unsigned int >= 0` conditional in `fuzz/script.cpp`.

  These changes fix 5 compile warnings in gcc 10 and 3 in clang 11/12.

ACKs for top commit:
  vasild:
    ACK c0f09c2
  MarcoFalke:
    review ACK c0f09c2c9d

Tree-SHA512: 76ce73ec577c1f23cf8646c31d44dcd6c6303732c47187d041a8921d0d24a50163989a375352ebc221abf2ac337bc0902149be46b6f9eebc071d2f364c407f71
2020-07-22 17:32:36 +08:00
MarcoFalke c7007babb7
Merge #18907: walletdb: Don't remove database transaction logs and instead error
d0ea9bab28 walletdb: Don't remove database transaction logs and instead error (Andrew Chow)

Pull request description:

  Instead of removing the database transaction logs and retrying the
  wallet loading, just return an error message to the user. Additionally,
  speciically for DB_RUNRECOVERY, notify the user that this could be due
  to different BDB versions.

  Kind of implements the suggestion from https://github.com/bitcoin/bitcoin/pull/18870#discussion_r421647964

ACKs for top commit:
  Sjors:
    re-utACK d0ea9bab28
  ryanofsky:
    Code review ACK d0ea9bab28. Only changes since last review are rebase and expanding error and commit messages.

Tree-SHA512: f6e67dc70f58188742a5c8af7cdc63a2b58779aa0d26ae7f1e75805a239f1a342433860e5a238d6577fae5ab04b9d15e7f11c55b867065dfd13781a6a62e4958
2020-07-22 08:58:55 +02:00
Jon Atack c0f09c2c9d
fuzz: add missing overrides to signature_checker
and also

- add missing parentheses in fuzz/scriptnum_ops.cpp

- remove useless unsigned int conditional in fuzz/script.cpp

These changes fix 5 compile warnings in gcc 10.
2020-07-22 05:27:13 +02:00
freenancial c8992e8959 test: Fix fuzzer compilation on macOS
fixes #19557
2020-07-21 15:23:49 -07:00
MarcoFalke 2c0c3f8e8c
Merge #19217: p2p: disambiguate block-relay-only variable names from blocksonly variables
ec4c6a17e8 scripted-diff: replace MAX_BLOCKS_ONLY_CONNECTIONS with MAX_BLOCK_RELAY_ONLY_CONNECTIONS (glowang)

Pull request description:

  We have two different concepts that have similar names: `-blocksonly` and `block-relay-only`, and the similarity of names could lead to confusion. `-blocksonly` disables all local receiving & relaying of transactions (with a few exceptions), while `block-relay-only`means that bitcoind will make 2 additional outbound connections that are only used for block relay.

  In net.h and init.cpp, `MAX_BLOCKS_ONLY_CONNECTIONS` is used to represent the maximum number of `block-relay-only` outbound peers, which is 2. But this name sounds ambiguous, and I proposed a better name,  `MAX_BLOCK_RELAY_ONLY_CONNECTION`.

ACKs for top commit:
  jnewbery:
    ACK ec4c6a17e8

Tree-SHA512: cfa592a7ff936f14d10cfc1e926a51b82bc0feaf104885a41ca8111b906cb3d1ec5536bab143a3cfca70aa49e9575c6995941eb6d3d7f4018d4535712342f155
2020-07-21 16:04:33 +02:00
MarcoFalke edfeaf6836
Merge #19552: test: fix intermittent failure in p2p_ibd_txrelay
12410b1feb test: fix intermittent p2p_ibd_txrelay race, add test_framework.py#wait_until (Jon Atack)

Pull request description:

  To fix these intermittent failures in Travis CI.
  ```
  162/163 - p2p_ibd_txrelay.py failed, Duration: 2 s

  stdout:
  2020-07-19T05:44:17.213000Z TestFramework (INFO):
      Check that nodes set minfilter to MAX_MONEY while still in IBD
  2020-07-19T05:44:17.216000Z TestFramework (ERROR): Assertion failed
  Traceback (most recent call last):
    File "/Users/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-apple-darwin16/test/functional/test_framework/test_framework.py", line 117, in main
      self.run_test()
    File "/Users/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-apple-darwin16/test/functional/p2p_ibd_txrelay.py", line 30, in run_test
      assert_equal(conn_info['minfeefilter'], MAX_FEE_FILTER)
    File "/Users/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-apple-darwin16/test/functional/test_framework/util.py", line 49, in assert_equal
      raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))

  AssertionError: not(0E-8 == 0.09170997)
  2020-07-19T05:44:17.293000Z TestFramework (INFO): Stopping nodes
  ```

  At Marco's suggestion, cherry-picked part of #19134 to nicely simplify using `wait_until`.

ACKs for top commit:
  vasild:
    ACK 12410b1fe

Tree-SHA512: 615f509883682fd693e578b259cba35a9fa0bc519f1394e88c857e8b0650bfec5397bfa856cfa9e6d5ef81d0ee6ad02e4ad2b0eb0bd530b4c281cbe3e663790b
2020-07-21 16:01:59 +02:00
fanquake 910a8d9c6f
Merge #19560: contrib: Clean up previous_releases.py
facdf530c7 contrib: Clean up previous_releases.py (MarcoFalke)

Pull request description:

ACKs for top commit:
  fjahr:
    tACK facdf530c7
  Sjors:
    tACK facdf53
  hebasto:
    ACK facdf530c7, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: c3543320572267035aa342dd170128bbdeb83ca4e2e36a8e46596dd76c8ff1b26ed6759a8073884228b133c45b06ec48889cd0ec83a13bef276b48073d8248e4
2020-07-21 20:32:20 +08:00
MarcoFalke 65a54d684f
Merge #18984: Remove unnecessary input blockfile SetPos
5fa067a27d Remove unnecessary blockfile SetPos (Tom Harding)

Pull request description:

  Nothing could have changed the position since we retrieved it a few statements earlier. This dates from commit 16d5194165.

ACKs for top commit:
  LarryRuane:
    ACK 5fa067a27d

Tree-SHA512: 459cc7226e186c231ffb67f0613f550e8eb940f1b8933c3bc4a4e8dd519c8d5d45884e8cfd9347039dab90a093644bbbb31be063baed1c6fc7984b6cb4f17c9f
2020-07-21 11:28:35 +02:00
MarcoFalke facdf530c7
contrib: Clean up previous_releases.py
* Replace curl single char options with their verbose counterpart
* Stricter check for tarballHash
2020-07-21 11:03:35 +02:00
MarcoFalke ea595d39f7
Merge #19205: script: previous_release.sh rewritten in python
9c34aff393 Remove previous_release.sh (Brian Liotti)
e1e5960e10 script: Add previous_release.py (Brian Liotti)

Pull request description:

  Closes #18132

  Added functionality:
  1) checks file hash before untarring when using the binary download option

ACKs for top commit:
  fjahr:
    re-ACK 9c34aff393
  Sjors:
    tACK 9c34aff393

Tree-SHA512: 323f11828736a372a47f048592de8b027ddcd75b38f312dfc73f7b495d1e078bfeb384d9cdf434b3e70f2c6c0ce2da2df48e9a6460ac0e1967c6829a411c52d5
2020-07-21 10:11:39 +02:00
MarcoFalke b763ae02a6
Merge #16878: Fix non-deterministic coverage of test DoS_mapOrphans
4455949d6f Make test DoS_mapOrphans deterministic (David Reikher)

Pull request description:

  This pull request proposes a solution to make the test `DoS_mapOrphans` in denialofservice_tests.cpp have deterministic coverage.

  The `RandomOrphan` function in denialofservice_tests.cpp and the implicitly called function `ecdsa_signature_parse_der_lax` in pubkey.cpp were causing the non-deterministic test coverage.

  In the former, if a random orphan was selected the index of which is bigger than the max. orphan index in `mapOrphanTransactions`, the last orphan was returned from `RandomOrphan`. If the random number generated was never large enough, this condition would not be fulfilled and the corresponding branch wouldn't run. The proposed solution is to force one of the 50 dependant orphans to depend on the last orphan in `mapOrphanTransactions` using the newly introduced function `OrphanByIndex` (and passing it a large uint256), forcing this branch to run at least once.

  In the latter, if values for ECDSA `R` or `S` (or both) had no leading zeros, some code would not be executed. The solution was to find a constant signature that would be comprised of `R` and `S` values with leading zeros and calling `CPubKey::Verify` at the end of the test with this signature forcing this code to always run at least once at the end even if it hadn't throughout the test.

  To test that the coverage is (at least highly likely) deterministic, I ran

  `contrib/devtools/test_deterministic_coverage.sh denialofservice_tests/DoS_mapOrphans 1000`

  and the result was deterministic coverage across 1000 runs.

  Also - removed denialofservice_tests test entry from the list of non-deterministic tests in the coverage script.

ACKs for top commit:
  MarcoFalke:
    ACK 4455949d6f

Tree-SHA512: 987eb1f94b80d5bec4d4944e91ef43b9b8603055750362d4b4665b7f011be27045808aa9f4c6ccf8ae009b61405f9a1b8671d65a843c3328e5b8acce1f1c00a6
2020-07-21 09:38:39 +02:00
David Reikher 4455949d6f Make test DoS_mapOrphans deterministic
The RandomOrphan function and the function ecdsa_signature_parse_der_lax
in pubkey.cpp were causing non-deterministic test coverage.

Force seed in the beginning of the test to make it deterministic.
The seed is selected carefully so that all branches of the function
ecdsa_signature_parse_der_lax are executed. Prior to this fix, the test
was exhibiting non-deterministic coverage since none of the ECDSA
signatures that were generated during the test had leading zeroes in
either R, S, or both, resulting in some branches of said function not
being executed. The seed ensures that both conditions are hit.

Removed denialofservice_tests test entry from the list of non-deterministic
tests in the coverage script.
2020-07-21 09:18:57 +03:00
fanquake caf1766daf
Merge #19547: doc: Update macOS cross compilation dependencies for Focal
6be3eebd74 doc: Update macOS cross compilation dependencies for Focal (Hennadii Stepanov)

Pull request description:

  The [`libtinfo5`](https://packages.ubuntu.com/focal/libtinfo5) package is required on Ubuntu Focal for macOS cross compilation.

  Fixes #19546.

ACKs for top commit:
  fanquake:
    ACK 6be3eebd74 I have been [installing the libtinfo5 package](https://github.com/fanquake/core-review/blob/master/docker/debian9.dockerfile#L18) into my cross-compiling images, so mostly forgot about this.

Tree-SHA512: f40ab1efae549276e520f9cc3200438e70297c3081a3f9acc3075ec11d494fd6176ed3f1d4889b39eb2a5706eac835844a75b79335bb7f3c21c985e0779a7562
2020-07-21 14:02:42 +08:00
fanquake f5000ae66c
Merge #19555: rpc: deduplicate WriteHDKeypath() used in decodepsbt
55057ffc51 rpc: deduplicate WriteHDKeypath() used in decodepsbt (Sebastian Falbesoner)

Pull request description:

  The functionality is already provided in the BIP32 utility library `util/bip32.h` with the exact same name and function signature.

ACKs for top commit:
  achow101:
    ACK 55057ffc51
  instagibbs:
    utACK 55057ffc51
  jonatack:
    ACK 55057ffc51

Tree-SHA512: 074c1a71ffb32908926bf07f0c5428a46309f6e0d21e7c20b1008197c820b97776a441736d0b6fd8ab0c0852522a0b5a5ddb26a1e4a1100ca02aabc65a07a018
2020-07-21 09:11:48 +08:00
Vasil Dimov 100c64a95b
net: document enum Network 2020-07-20 16:03:39 +02:00
MarcoFalke fa2ae0ac8d
span: Add Span::empty() and use it in script/descriptor 2020-07-19 20:51:06 +02:00
Sebastian Falbesoner 55057ffc51 rpc: deduplicate WriteHDKeypath() used in decodepsbt
The functionality is already provided in the BIP32 utility library util/bip32.h
with the exact same name and function signature.
2020-07-19 17:25:07 +02:00
Jon Atack 12410b1feb
test: fix intermittent p2p_ibd_txrelay race, add test_framework.py#wait_until 2020-07-19 13:37:54 +02:00
fanquake 476436b2de
Merge #19551: fuzz: fix unused variable compiler warning in addrdb
1cdc2a642b fuzz: fix unused variable addrdb compiler warning (Jon Atack)

Pull request description:

  Fixes #19549

ACKs for top commit:
  hebasto:
    ACK 1cdc2a642b, tested on Linux Mint 20 (x86_64, clang 10.0).
  MarcoFalke:
    review re-ACK 1cdc2a642b
  practicalswift:
    ACK 1cdc2a642b

Tree-SHA512: 5ce5ce06f0a3121911ae2ac4ed3953e5e56e6a7bd50a60ee6df84873eec75d2997488d2435bd55294e890bbf6fc368dd4302cf2a6ab6073ec74c7711eb0340e1
2020-07-19 19:25:44 +08:00
fanquake 2a701a1c42
build: pass -fcommon when building genisoimage
Starting with the 10.1 release, GCC defaults to -fno-common. This causes
linking issues when building genisoimage:
```bash
[ 98%] Building C object genisoimage/CMakeFiles/genisoimage.dir/checksum.o
[100%] Linking C executable genisoimage
/usr/bin/ld: CMakeFiles/genisoimage.dir/apple.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/boot.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/desktop.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/dvd_file.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
```

Rather than patching genisoimage further, pass -fcommon to preserve the
legacy GCC behaviour.
2020-07-19 14:41:57 +08:00
Jon Atack 1cdc2a642b
fuzz: fix unused variable addrdb compiler warning 2020-07-19 08:31:34 +02:00
Suhas Daftuar 0a4f1422cd Further improve comments around recentRejects 2020-07-19 02:10:42 -04:00
Suhas Daftuar 0e20cfedb7 Disconnect peers sending wtxidrelay message after VERACK 2020-07-19 02:10:42 -04:00
Fabian Jahr cacd85209e test: Use wtxid relay generally in functional tests 2020-07-19 02:10:42 -04:00
Fabian Jahr 8d8099e97a test: Add tests for wtxid tx relay in segwit test
Also cleans up some doublicate lines in the rest of the test.

co-authored-by: Anthony Towns <aj@erisian.com.au>
2020-07-19 02:10:42 -04:00
Fabian Jahr 9a5392fdf6 test: Update test framework p2p protocol version to 70016
This new p2p protocol version allows to use WTXIDs for tx relay.
2020-07-19 02:10:42 -04:00
Suhas Daftuar dd78d1d641 Rename AddInventoryKnown() to AddKnownTx() 2020-07-19 02:10:42 -04:00
Suhas Daftuar 4eb515574e Make TX_WITNESS_STRIPPED its own rejection reason
Previously, TX_WITNESS_MUTATED could be returned during transaction validation
for either transactions that had a witness that was non-standard, or for
transactions that had no witness but were invalid due to segwit validation
rules.

However, for txid/wtxid-relay considerations, net_processing distinguishes the
witness stripped case separately, because it affects whether a wtxid should be
able to be added to the reject filter. It is safe to add the wtxid of a
witness-mutated transaction to the filter (as that wtxid shouldn't collide with
the txid, and hence it wouldn't interfere with transaction relay from
txid-relay peers), but it is not safe to add the wtxid (== txid) of a
witness-stripped transaction to the filter, because that would interfere with
relay of another transaction with the same txid (but different wtxid) when
relaying from txid-relay peers.

Also updates the comment explaining this logic, and explaining that we can get
rid of this complexity once there's a sufficient deployment of wtxid-relaying
peers on the network.
2020-07-19 02:10:42 -04:00
Suhas Daftuar 97141ca442 Delay getdata requests from peers using txid-based relay
Using both txid and wtxid-based relay with peers means that we could sometimes
download the same transaction twice, if announced via two different hashes from
different peers.

Use a heuristic of delaying txid-peer-getdata requests by 2 seconds, if we have
at least one wtxid-based peer.
2020-07-19 02:10:42 -04:00
Suhas Daftuar 46d78d47de Add p2p message "wtxidrelay"
When sent to and received from a given peer, enables using wtxid's for
announcing and fetching transactions with that peer.
2020-07-19 02:10:41 -04:00
Anthony Towns 2d282e0cba ignore non-wtxidrelay compliant invs 2020-07-19 02:05:42 -04:00
Suhas Daftuar ac88e2eb61 Add support for tx-relay via wtxid
This adds a field to CNodeState that tracks whether to relay transactions with
that peer via wtxid, instead of txid. As of this commit the field will always
be false, but in a later commit we will add a way to negotiate turning this on
via p2p messages exchanged with the peer.
2020-07-19 02:05:29 -04:00
Suhas Daftuar 8e68fc246d Add wtxids to recentRejects instead of txids
Previously, we only added txids to recentRejects if we were sure that the
transaction couldn't have had the wrong witness (either because the witness was
malleated or stripped).

In preparation for wtxid-based relay, we can observe that txid == wtxid for
transactions that have no witness, and add the wtxid of rejected transactions,
provided the transaction wasn't a witness-stripped one. This means that we now
add more data to the filter (as prior to this commit, any transaction with a
witness that failed to be accepted was being skipped for inclusion in the
filter) but witness malleation should still not interfere with relay of a valid
segwit transaction, because the txid of a segwit transaction would not be added
to the filter after failing validation.

In the future, having wtxids in the recent rejects filter will allow us to
skip downloading the same wtxid multiple times, once our peers use wtxids for
transaction relay.
2020-07-18 19:00:02 -04:00
Suhas Daftuar 144c385820 Add wtxids of confirmed transactions to bloom filter
This is in preparation for wtxid-based invs (we need to be able to tell whether
we AlreadyHave() a transaction based on either txid or wtxid).

This also double the size of the bloom filter, which is overkill, but still
uses a manageable amount of memory.
2020-07-18 19:00:02 -04:00
Suhas Daftuar 85c78d54af Add wtxid-index to orphan map 2020-07-18 19:00:02 -04:00