Commit graph

24239 commits

Author SHA1 Message Date
Carl Dong d256f91cb1
rel-builds: Directly deploy win installer to OUTDIR 2020-05-01 12:30:20 -04:00
MarcoFalke fab7ee3990
test: Fix p2p_leak intermittent issue 2020-05-01 08:45:26 -04:00
MarcoFalke fa8614aea9
test: Fix intermittent p2p_segwit issue 2020-05-01 08:36:35 -04:00
MarcoFalke fa2cce4391
wallet: Remove trailing whitespace from potential translation strings
If the potential translation strings are translated in the future,
trailing whitespace is going to make translation effort harder.
2020-05-01 07:41:32 -04:00
MarcoFalke fa59cc1c97
wallet: Report full error message in wallettool 2020-05-01 07:39:35 -04:00
MarcoFalke fae7776690
wallet: Avoid translating RPC errors when creating txs
Also, mark feebumper bilingual_str as Untranslated

They are technical and have previously not been translated either.
It is questionable whether they can even appear in the GUI.
2020-05-01 07:39:06 -04:00
MarcoFalke fae51a5c6f
wallet: Avoid translating RPC errors when loading wallets
Common errors and warnings should be translated when displayed in the
GUI, but not translated when displayed elsewhere. The wallet method
CreateWalletFromFile does not know its caller, so this commit changes it
to return a bilingual_str to the caller.
2020-05-01 07:39:00 -04:00
Russell Yanofsky d3a56be77a Revert "gui: Avoid Wallet::GetBalance in WalletModel::pollBalanceChanged"
This reverts commit 0933a37078 from
https://github.com/bitcoin/bitcoin/pull/18160 which no longer an optimization
since commit "gui: Avoid wallet tryGetBalances calls before TransactionChanged
or BlockTip notifications".
2020-05-01 06:59:09 -04:00
Russell Yanofsky 28b112e9bd Get rid of BindWallet
CWalletTx initialization has been fixed so it's no longer necessary to change
which wallet a transaction is bound to.
2020-05-01 05:59:09 -05:00
MarcoFalke 608359b071
Merge #16426: Reverse cs_main, cs_wallet lock order and reduce cs_main locking
6a72f26968 [wallet] Remove locked_chain from CWallet, its RPCs and tests (Antoine Riard)
841178820d [wallet] Move methods from Chain::Lock interface to simple Chain (Antoine Riard)
0a76287387 [wallet] Move getBlockHash from Chain::Lock interface to simple Chain (Antoine Riard)
de13363a47 [wallet] Move getBlockHeight from Chain::Lock interface to simple Chain (Antoine Riard)
b855592d83 [wallet] Move getHeight from Chain::Lock interface to simple Chain (Antoine Riard)

Pull request description:

  This change is intended to make the bitcoin node and its rpc, network and gui interfaces more responsive while the wallet is in use. Currently, because the node's `cs_main` mutex is always locked before the wallet's `cs_wallet` mutex (to prevent deadlocks), `cs_main` currently stays locked while the wallet does relatively slow things like creating and listing transactions.

  Switching the lock order so `cs_main` is acquired after `cs_wallet` allows `cs_main` to be only locked intermittently while the wallet is doing slow operations, so the node is not blocked waiting for the wallet.

  To review the present PR, most of getting right the move is ensuring any `LockAssertion` in `Chain::Lock` method is amended as a `LOCK(cs_main)`. And in final commit, check that any wallet code which was previously locking the chain is now calling a  method, enforcing the lock taking job. So far the only exception I found is `handleNotifications`, which should be corrected.

ACKs for top commit:
  MarcoFalke:
    re-ACK 6a72f26968 🔏
  fjahr:
    re-ACK 6a72f26968
  ryanofsky:
    Code review ACK 6a72f26968. Only difference compared to the rebase I posted is reverting unneeded SetLastBlockProcessed change in wallet_disableprivkeys test

Tree-SHA512: 9168b3bf3432d4f8bc4d9fa9246ac057050848e673efc264c8f44345f243ba9697b05c22c809a79d1b51bf0de1c4ed317960e496480f8d71e584468d4dd1b0ad
2020-05-01 06:59:09 -04:00
Russell Yanofsky bf0a510981 gui: Avoid wallet tryGetBalances calls before TransactionChanged or BlockTip notifications
interfaces::Wallet::tryGetBalances was recently updated in
https://github.com/bitcoin/bitcoin/pull/18160 to avoid computing balances
internally, but this not efficient as it could be with #10102 because
tryGetBalances is an interprocess call.

Implementing the TransactionChanged / BlockTip check outside of tryGetBalances
also allows tryGetBalances to be simplified in next commit 'Revert "gui: Avoid
Wallet::GetBalance in WalletModel::pollBalanceChanged"'.
2020-05-01 06:59:09 -04:00
Russell Yanofsky d002f9d15d Disable CWalletTx copy constructor
Disable copying of CWalletTx objects to prevent bugs where instances get copied
in and out of the mapWallet map and fields are updated in the wrong copy.
2020-05-01 05:59:09 -05:00
Russell Yanofsky 2bc9b92ed8 Cancel wallet balance timer when shutdown requested
This doesn't fix any current problem, but it makes balance checking code less
fragile, and prevents use-after free travis error in next commit:
https://travis-ci.org/github/bitcoin/bitcoin/jobs/675367629#L4240
2020-05-01 06:59:09 -04:00
Russell Yanofsky 65b9d8f8dd Avoid copying CWalletTx in LoadToWallet
The change in walletdb.cpp is easier to review ignoring whitespace.

This change is need to get rid of CWalletTx copy constructor.
2020-05-01 05:59:09 -05:00
Russell Yanofsky 83f69fab3a Switch transaction table to use wallet height not node height
Tweak of #17905 to make gui display of transactions and balances more
consistent. This change shouldn't cause visible effects in normal cases, just
make GUI wallet code more internally correct and consistent.
2020-05-01 06:59:09 -04:00
Russell Yanofsky bd2fbc7cdb Get rid of unneeded CWalletTx::Init parameter 2020-05-01 05:59:09 -05:00
Russell Yanofsky 2b9cba2065 Remove CWalletTx merging logic from AddToWallet
Instead of AddToWallet taking a temporary CWalletTx object and then potentially
merging it with a pre-existing CWalletTx, have it take a callback so callers
can update the pre-existing CWalletTx directly.

This makes AddToWallet simpler because now it is only has to be concerned with
saving CWalletTx objects and not merging them.

This makes AddToWallet calls clearer because they can now make direct updates to
CWalletTx entries without having to make temporary objects and then worry about
how they will be merged.

This is a pure refactoring, no behavior is changing.
2020-05-01 05:59:09 -05:00
Antoine Riard 6a72f26968 [wallet] Remove locked_chain from CWallet, its RPCs and tests
This change is intended to make the bitcoin node and its rpc, network
and gui interfaces more responsive while the wallet is in use. Currently
because the node's cs_main mutex is always locked before the wallet's
cs_wallet mutex (to prevent deadlocks), cs_main currently stays locked
while the wallet does relatively slow things like creating and listing
transactions.

This commit only remmove chain lock tacking in wallet code, and invert
lock order from cs_main, cs_wallet to cs_wallet, cs_main.
must happen at once to avoid any deadlock. Previous commit were only
removing Chain::Lock methods to Chain interface and enforcing they
take cs_main.

Remove LockChain method from CWallet and Chain::Lock interface.
2020-04-30 14:41:24 -04:00
Antoine Riard 841178820d [wallet] Move methods from Chain::Lock interface to simple Chain
Remove findPruned and findFork, no more used after 17954.
2020-04-30 14:37:21 -04:00
Antoine Riard 0a76287387 [wallet] Move getBlockHash from Chain::Lock interface to simple Chain 2020-04-30 14:37:21 -04:00
Antoine Riard de13363a47 [wallet] Move getBlockHeight from Chain::Lock interface to simple Chain
Add HaveChain to assert chain access for wallet-tool in LoadToWallet.
2020-04-30 14:37:21 -04:00
Antoine Riard b855592d83 [wallet] Move getHeight from Chain::Lock interface to simple Chain
Instead of calling getHeight, we rely on CWallet::m_last_block
processed_height where it's possible.
2020-04-30 14:31:19 -04:00
MarcoFalke e5b9308920
Merge #18829: ci: Merge C++17 build with one of the existing ones
aaaacff107 ci: Merge C++17 build with one of the existing ones (MarcoFalke)

Pull request description:

  No need to spin up an extra vm for each pull request for a simple sanity check that any of the other already running machines can test.

Top commit has no ACKs.

Tree-SHA512: ce1609aa4cbc057fc9e85e61f300eac2317cc206647a20f6b001180ffed6623e2243a408476546e16baacb91cb2dd9a2e7b6e2402c8fd829a72860b2c3eb7be6
2020-04-30 14:04:24 -04:00
MarcoFalke ef8ef31fd2
Merge #18775: tests: Add fuzzing harnesses for various classes/functions in policy/ (CBlockPolicyEstimator, IsRBFOptIn(…), etc.)
2bcc2bd742 tests: Clarify how we avoid hitting the signed integer overflow in CFeeRate::GetFeePerK() when fuzzing (practicalswift)
13c1f6b24f tests: Add fuzzing harness for IsRBFOptIn(...) (practicalswift)
3439c88a5d tests: Add fuzzing harness for CBlockPolicyEstimator (practicalswift)

Pull request description:

  Add fuzzing harnesses for various classes/functions in `policy/` (`CBlockPolicyEstimator`, `IsRBFOptIn(…)`, etc.).

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

Top commit has no ACKs.

Tree-SHA512: a756687216802a3b260def5706798a1eb673b3447408561728af77a1d61c8bfb3a7d9b874e16bf619565e9d093f9b595e07d6e6fa430ac08dfeed4f45b01fbc3
2020-04-30 12:53:42 -04:00
MarcoFalke aaaacff107
ci: Merge C++17 build with one of the existing ones 2020-04-30 12:50:36 -04:00
MarcoFalke fa359d14c0
test: Strip down previous releases boilerplate 2020-04-30 11:45:16 -04:00
practicalswift 2bcc2bd742 tests: Clarify how we avoid hitting the signed integer overflow in CFeeRate::GetFeePerK() when fuzzing 2020-04-30 14:19:49 +00:00
MarcoFalke a66ba6d029
Merge #18576: test: use unittest for test_framework unit testing
de8905adf2 test: use unittest and test_runner for test framework unit testing (Gloria Zhao)

Pull request description:

  Proposal for unit testing on test_framework functions:
  1. Use the python `unittest` library. Don't use test_framework to test itself.
  2. Put the tests inside the same file as the functions they are testing.
  3. Call the tests from `test_runner.py`. To include more Test Framework tests, add the filename to the list `TEST_FRAMEWORK_MODULES`. Don't add new files or change the list of accepted script prefixes.

  Makes these changes for `bn2vch` (followup to [this comment](https://github.com/bitcoin/bitcoin/pull/18378#pullrequestreview-377271264)).

ACKs for top commit:
  jnewbery:
    Tested ACK de8905adf2. Great stuff gzhao408 . Thanks for this!

Tree-SHA512: 91572d43e203a1864765b93a9472667994115ec38b271f2b2f9fcd0f0112b393fc24ba7d2371d5a34b0a6a4522f6b934fc5164363819aa7ed8d6c6c9a60cc101
2020-04-30 09:30:13 -04:00
practicalswift 13c1f6b24f tests: Add fuzzing harness for IsRBFOptIn(...) 2020-04-30 13:19:24 +00:00
practicalswift 3439c88a5d tests: Add fuzzing harness for CBlockPolicyEstimator 2020-04-30 13:19:24 +00:00
MarcoFalke 0000ea3265
test: Add test for GetRandMillis and GetRandMicros 2020-04-30 09:19:16 -04:00
MarcoFalke fa0e5b89cf
Add templated GetRandomDuration<> 2020-04-30 09:19:14 -04:00
MarcoFalke 00c1a4d9a9
Merge #18809: rpc: Do not advertise dumptxoutset as a way to flush the chainstate
fac0cf6e55 rpc: Do not advertise dumptxoutset as a way to flush the chainstate (MarcoFalke)

Pull request description:

  The help message leaks several implementation details: leveldb and flush.

  Neither of them are relevant to the end user and I don't see why we should make them part of the API contract.

ACKs for top commit:
  laanwj:
    ACK fac0cf6e55

Tree-SHA512: 273fb85dc5be6cdccf17c43f183fa83c57d0a1cbb30555838f32c074218b713a753930009f6c98c85659421f2285f09c0a713b22f7e34d446e56737ac03870f7
2020-04-30 07:20:14 -04:00
MarcoFalke cf5e3be5ea
Merge #18825: test: fix message for ECC_InitSanityCheck test
06e434d7d9 test: fix message for ECC_InitSanityCheck test (fanquake)

Pull request description:

  OpenSSL is long gone.

ACKs for top commit:
  laanwj:
    Good catch. ACK 06e434d7d9

Tree-SHA512: 1a920fd6493e0374ca00633407e0130f987b136bc68d2062402747bda16a1e588a12bd8b0b8cdef828c9911f210386cfbdb25d478cb9b684d52769d197032064
2020-04-30 07:09:05 -04:00
fanquake 64673b1037
Merge #18780: validation: add const for minimum witness commitment size
692f8307fc test: add test for witness commitment index (fanquake)
06442549f8 validation: Add minimum witness commitment size constant (fanquake)

Pull request description:

  16101de5f3: Per [BIP 141](https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#Commitment_structure), the witness commitment structure is at least 38 bytes,
  OP_RETURN (0x6a) + 36 (0x24) + 4 byte header (0xaa21a9ed) + 32 byte
  SHA256 hash. It can be longer, however any additional data has no
  consensus meaning.

  54f8c48d6a: As per BIP 141, if there is more than 1 pubkey that matches the witness
  commitment structure, the one with the highest output index should be
  chosen. This adds a sanity check that we are doing that, which will fail
  if anyone tries to "optimize" GetWitnessCommitmentIndex() by returning
  early.

ACKs for top commit:
  MarcoFalke:
    ACK 692f8307fc 🌵
  jonatack:
    Code review ACK 692f830
  ajtowns:
    ACK 692f8307fc
  jnewbery:
    utACK 692f8307fc
  laanwj:
    ACK 692f8307fc

Tree-SHA512: 7af3fe4b8a52fea2cdd0aec95f7bb935351a77b73d934bc88d6625a3503311b2a062cba5190b2228f97caa76840db3889032d910fc8e318ca8e7810a8afbafa0
2020-04-30 18:50:26 +08:00
fanquake 0ae8f18dfe
build: add -Wgnu to compile flags
When compiling with Clang, this will warn when GNU extensions are
used.

Info: https://clang.llvm.org/docs/DiagnosticsReference.html#wgnu
2020-04-30 18:02:04 +08:00
Ben Woosley 3a0fd7726b
Remove use of non-standard zero variadic macros
These are a gnu extension warned against by: gnu-zero-variadic-macro-arguments
2020-04-30 18:02:04 +08:00
Ben Woosley 49f6178c3e
Drop unused LOG_TIME_MICROS helper 2020-04-30 18:02:04 +08:00
DesWurstes 5d4999951e
prevector: Avoid unnamed struct, which is a GNU extension 2020-04-30 18:02:03 +08:00
Wladimir J. van der Laan afed2e98b0
Merge #18810: doc: update rest info on block size and json
ff6549c3c8 fix: update rest info on block size and json (Chris Abrams)

Pull request description:

  Addressing the ambiguous block size text in rest docs: https://github.com/bitcoin/bitcoin/issues/18703

  Also makes sure to let developers know there is `.json` option for the rest output format.

ACKs for top commit:
  MarcoFalke:
    ACK ff6549c3c8
  promag:
    ACK ff6549c3c8.

Tree-SHA512: 9ef93c1432d650b1f9599778ba092c1ca5b084a537af257078e1c713c76c5d3a4cc4b1ede8a2489964be8ed0303ad8bea58c1cb4759bbb9b24dbdebfec8001d3
2020-04-30 11:45:06 +02:00
Wladimir J. van der Laan 35ef3c15ef
Merge #18591: Add C++17 build to Travis
c31cbe7cfe Add C++17 test to Travis (Pieter Wuille)
7829685e27 Add configure option for c++17 (Pieter Wuille)
0fbde488b2 Support conversion between Spans of compatible types (Pieter Wuille)
7cbfebbf3d Update ax_cxx_compile_stdcxx.m4 (Pieter Wuille)

Pull request description:

  This adds a `--enable-c++17` option to the configure script, fixes the only C++17 incompatibility (with a commit taken from #18468), and adds a Travis test for it.

  This is all off by default, and release builds remain C++11.

  It implements the first step of the plan in https://github.com/bitcoin/bitcoin/issues/16684.

ACKs for top commit:
  elichai:
    tACK c31cbe7cfe
  practicalswift:
    Tested ACK c31cbe7cfe
  hebasto:
    ACK c31cbe7cfe, tested on Linux Mint 19.3 both C++11 and C++17 modes. Compiled and passed tests locally.

Tree-SHA512: a4b00776dbceef9c12abbb404c6bcd48f7916ce24c8c7a14116355f64e817578b7fcddbedd5ce435322319d1e4de43429b68553f4d96d970c308fe3e3e59b9d1
2020-04-30 11:16:56 +02:00
fanquake 06e434d7d9
test: fix message for ECC_InitSanityCheck test
OpenSSL is long gone.
2020-04-30 16:57:46 +08:00
Wladimir J. van der Laan 63d5ed2fc4
Merge #18437: util: Detect posix_fallocate() instead of assuming
182dbdf0f4 util: Detect posix_fallocate() instead of assuming (Vasil Dimov)

Pull request description:

  Don't assume that `posix_fallocate()` is available on Linux and not
  available on other operating systems. At least FreeBSD has it and we
  are not using it.

  Properly check whether `posix_fallocate()` is present and use it if it
  is.

ACKs for top commit:
  laanwj:
    ACK 182dbdf0f4

Tree-SHA512: f9ed4bd661f33ff6b2b1150591e860b3c1f44e12b87c35e870d06a7013c4e841ed2bf17b41ad6b18fe471b0b23a4b5e42cf1400637180888e0bc56c254fe0766
2020-04-30 10:45:17 +02:00
John Newbery 9847e205bf [docs] Improve commenting in ProcessGetData() 2020-04-29 19:34:01 -04:00
Amiti Uttarwar 2f032556e0 [test] test that an invalid GETDATA doesn't prevent processing of future messages
Co-Authored-By: John Newbery <john@johnnewbery.com>
2020-04-29 19:34:01 -04:00
MarcoFalke 36c0abd8f6
Merge #18726: test: check misbehavior more independently in p2p_filter.py
cd543d9193 test: check misbehavior more independently in p2p_filter.py (Danny Lee)

Pull request description:

  This expands on #18672 in two ways:

  - Check positive cases (`filterload` accepted, `filteradd` accepted) in addition to the negative cases added in #18672
  - Address MarcoFalke 's [suggestion](https://github.com/bitcoin/bitcoin/pull/18672#discussion_r412101752) to successfully load a filter before testing `filteradd`

ACKs for top commit:
  theStack:
    re-ACK cd543d9193

Tree-SHA512: f82402f6287ccddf08b38b6432d5e2b2b2ef528802a981d04c24bac459022f732d9090d4849d72d3d1eb2c757161dcb18c4c036b6e11dc80114e9cd49f21c3bd
2020-04-29 18:56:25 -04:00
MarcoFalke 95a9165016
Merge #18736: test: Add fuzzing harnesses for various classes/functions in util/
32b6b386a5 tests: Sort fuzzing harnesses (practicalswift)
e1e181fad1 tests: Add fuzzing coverage for JSONRPCTransactionError(...) and RPCErrorFromTransactionError(...) (practicalswift)
103b6ecce0 tests: Add fuzzing coverage for TransactionErrorString(...) (practicalswift)
dde508b8b0 tests: Add fuzzing coverage for ParseFixedPoint(...) (practicalswift)
1532259fca tests: Add fuzzing coverage for FormatHDKeypath(...) and WriteHDKeypath(...) (practicalswift)
90b635e84e tests: Add fuzzing coverage for CHECK_NONFATAL(...) (practicalswift)
a4e3d13df6 tests: Add fuzzing coverage for StringForFeeReason(...) (practicalswift)
a19598cf98 tests: Add fuzzing harness for functions in system.h (ArgsManager) (practicalswift)

Pull request description:

  Add fuzzing harnesses for various classes/functions in `util/`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

Top commit has no ACKs.

Tree-SHA512: d27947220850c2a202c7740f44140c17545f45522596912452ccab0c2f5379abeb07cc769982c7855cb465059425206371a2b75ee1c285b03984161c9619d0b0
2020-04-29 18:54:34 -04:00
MarcoFalke 0f204dd3f2
Merge #18727: test: Add CreateWalletFromFile test
7918c1b019 test: Add CreateWalletFromFile test (Russell Yanofsky)

Pull request description:

  Add unit test calling CreateWalletFromFile, which isn't currently called from other unit tests, with some basic checks to make sure it rescans and registers for notifications correctly.

  Motivation for this change was to try to write a test that would fail without the early `handleNotifications` call in ef8c6ca60767cac589d98ca57ee33179608ccda8 from https://github.com/bitcoin/bitcoin/pull/16426, but succeed with it:

  ef8c6ca607/src/wallet/wallet.cpp (L3978-L3986)

  However, writing a full test for the race condition that call prevents isn't possible without the locking changes from #16426. So this PR just adds as much test coverage as is possible now.

  This new test is also useful for https://github.com/bitcoin/bitcoin/pull/15719, since it detects the stale notifications.transactionAddedToMempool notifications that PR eliminates.

ACKs for top commit:
  MarcoFalke:
    ACK 7918c1b019
  jonatack:
    ACK 7918c1b019

Tree-SHA512: 44035aee698ecb722c6039d061d8fac2011e9da0b314e4aff19be1d610b53cacff99016b34d6b84669bb3b61041b2318d9d8e3363658f087802ae4aa36ca17b8
2020-04-29 15:23:39 -04:00
MarcoFalke 978c5a2122
Merge #18485: test: Add mempool_updatefromblock.py
8098dea069 test: Add mempool_updatefromblock.py (Hennadii Stepanov)

Pull request description:

  This PR adds a new test for mempool update of transaction descendants/ancestors information (count, size) when transactions have been re-added from a disconnected block to the mempool.

  It could be helpful for working on PRs like #17925, #18191.

ACKs for top commit:
  ariard:
    ACK 8098dea

Tree-SHA512: 7e808fa8df8d7d7a7dbdc3f79361049b49c7bce9b58fd5539b28c9636bedac747695537e500d7ed68dc8bdb80167ad3f1c01086f7551691405d2ba2e38ef1d06
2020-04-29 12:07:13 -04:00
MarcoFalke ab91a0e0fc
Merge #18798: ci: Fix default retry script usage
45615de26c ci: Fix default retry script usage (Hennadii Stepanov)

Pull request description:

  On master (5352d14b37) `CI_RETRY_EXE=${CI_RETRY_EXE:retry}` works as a [Substring Expansion](https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html), and that is wrong.
  If `CI_RETRY_EXE` variable was unset initially, its new value becomes an empty string, but not "retry" as one could expect. Consequently, the `${CI_RETRY_EXE} ...` command does _not_ use `ci/retry/retry` script.

  This PR makes for `CI_RETRY_EXE` variable a usual parameter expansion, i.e., `${parameter:-word}`.

  Reference: https://github.com/bitcoin/bitcoin/pull/18735#issuecomment-620095489

Top commit has no ACKs.

Tree-SHA512: 108173f6b2677979b9ddf2f9b9df4a6c56f5efa81c36543a1816bb3b984e42984bf3c83fe413ea3a5ca1e2317c4efb02fea7180a6b44863af7cfe6202e9cf94d
2020-04-29 11:14:14 -04:00