Commit graph

2455 commits

Author SHA1 Message Date
MarcoFalke
376294cde6
Merge #18866: test: Fix verack race to avoid intermittent test failures
fae153b409 test: Fix verack race to avoid intermittent test failures (MarcoFalke)

Pull request description:

  Fixes #18832

ACKs for top commit:
  laanwj:
    ACK fae153b409

Tree-SHA512: 071de8c8e2b2787c9433c7460e18b9a54beaf471a52ce848c5ac7263fc2a40f5b976d4f558ecc494fd0fa07284b7c98d29267cade58f80ab74fe9a7d18d94298
2020-05-08 19:31:17 -04:00
MarcoFalke
3930014abc
Merge #18864: Add v0.16.3 backwards compatibility test, bump v0.19.0.1 to v0.19.1
d135c29476 [ci] make list of previous releases to download a setting (Sjors Provoost)
9c246b873c [test] backwards compatibility: bump v0.19.0.1 to v0.19.1 (Sjors Provoost)
89a28e02fa [test] add v0.16.3 backwards compatibility test (Sjors Provoost)

Pull request description:

  Thanks to #18774's `adjust_bitcoin_conf_for_pre_17` we can now test backwards compatibility for v0.16.3, both for sync and loading a recent wallet.

  This PR bumps v0.19.0.1 to v0.19.1.

  I also made the version list consistent for the `contrib/devtools/previous_release.sh` instruction, between both tests.

ACKs for top commit:
  MarcoFalke:
    ACK d135c29476

Tree-SHA512: 5ff137a7a934237fa220f1c2807ce9abeeb75929266558bf3e4045bec7dfcd0a8747fa74d700065c568330b18badf58c60c308eb13d1eed444d4bbfe6decc48b
2020-05-08 11:23:40 -04:00
Wladimir J. van der Laan
7bcc42b403
Merge #18873: test: Fix intermittent sync_blocks failures
fa3f9a0566 test: Fix intermittent sync_blocks failures (MarcoFalke)

Pull request description:

  Fixes #18872
  Fixes #18737
  Fixes #18801

  See docstring for motivation and description

ACKs for top commit:
  laanwj:
    Code review ACK fa3f9a0566

Tree-SHA512: acd52d386a6849f7ff1cb1a51a439dc3c76e0e3a4dd8d22030df0ebb09b44497c61c56331ff65724b695d82d86b0ebb24608f9e637008e5dacb7676b0c448889
2020-05-06 15:30:31 +02:00
fanquake
d96fdc2a39
Merge #18741: guix: Make source tarball using git-archive
bfe1ba2f5b rel-builds: Specify core.abbrev for git-rev-parse (Carl Dong)
27e63e01cc build: Accomodate makensis v2.x (Carl Dong)
1f2c39a30e guix: Remove logical cores requirement (Carl Dong)
a4f6ffa71e lint: Also enable source statements for non-gitian (Carl Dong)
d256f91cb1 rel-builds: Directly deploy win installer to OUTDIR (Carl Dong)
fa791da02f nsis: Specify OutFile path only once (Carl Dong)
14701604d0 guix: Expose GIT_COMMON_DIR in container as readonly (Carl Dong)
f5a6ac4f48 guix: Make source tarball using git-archive (Carl Dong)
395c1137f6 gitian: Limit sourced script to just assignments (Carl Dong)

Pull request description:

  Based on: #18556
  Related: https://github.com/bitcoin/bitcoin/pull/17595#discussion_r399728721

ACKs for top commit:
  fanquake:
    ACK bfe1ba2f5b - I agree with Carl, and am going to merge this. I'd like for Linux Guix builds to be working again, and we can rebase #18818.

Tree-SHA512: c87ada7e3de17ca0b692a91029b86573442ded5780fc081c214773f6b374a0cdbeaf6f6898c36669c2e247ee32aa7f82defb1180f8decac52c65f0c140f18674
2020-05-06 13:13:36 +08:00
MarcoFalke
fa3f9a0566
test: Fix intermittent sync_blocks failures 2020-05-05 14:37:46 -04:00
Wladimir J. van der Laan
23c926d859
Merge #18699: wallet: Avoid translating RPC errors
fa2cce4391 wallet: Remove trailing whitespace from potential translation strings (MarcoFalke)
fa59cc1c97 wallet: Report full error message in wallettool (MarcoFalke)
fae7776690 wallet: Avoid translating RPC errors when creating txs (MarcoFalke)
fae51a5c6f wallet: Avoid translating RPC errors when loading wallets (MarcoFalke)

Pull request description:

  Common errors and warnings should be translated when displayed in the
  GUI, but not translated when displayed elsewhere. The wallet method
  `CreateWalletFromFile` does not know its caller, so this commit changes it
  to return a `bilingual_str` to the caller.

  Fixes #17072

ACKs for top commit:
  laanwj:
    ACK fa2cce4391, checked that no new translation messages are added compared to master.
  hebasto:
    ACK fa2cce4391

Tree-SHA512: c6a943ae9c3689ea3c48c20d26de6e4970de0257a1f1eec57a2bded67a4af9dcc5c45b2d64659d6fb4c4bc4d8103e28483ea3d14bb850df8db0ff9e8e5c77ee2
2020-05-04 16:29:22 +02:00
MarcoFalke
fae153b409
test: Fix verack race to avoid intermittent test failures 2020-05-04 08:56:30 -04:00
Sjors Provoost
9c246b873c
[test] backwards compatibility: bump v0.19.0.1 to v0.19.1 2020-05-04 13:36:40 +02:00
Sjors Provoost
89a28e02fa
[test] add v0.16.3 backwards compatibility test 2020-05-04 13:36:40 +02:00
MarcoFalke
2e6a16b968
Merge #18855: tests: feature_backwards_compatibility.py test downgrade after upgrade
489ebfd7a1 tests: feature_backwards_compatibility.py test downgrade after upgrade (Andrew Chow)

Pull request description:

  After upgrading the node, try to go back to the original version to make sure that using a newer node version does not prevent the wallet file from being downgraded again.

ACKs for top commit:
  MarcoFalke:
    ACK 489ebfd7a1

Tree-SHA512: 86231de6514b3657912fd9d6621212166fd2b29b591fc97120092c548babcf1d6f50b5bd103b28cecde395a26809134f01c1a198725596c3626420de3fd1f017
2020-05-03 10:35:21 -04:00
MarcoFalke
cce034b028
Merge #18839: test: Fix intermittent issues
fab7ee3990 test: Fix p2p_leak intermittent issue (MarcoFalke)
fa8614aea9 test: Fix intermittent p2p_segwit issue (MarcoFalke)

Pull request description:

  Fixes #18801
  Fixes #18802

ACKs for top commit:
  practicalswift:
    ACK fab7ee3990 -- diff looks correct

Tree-SHA512: b5d0473ec1133aded127bef4c0c90f6b55906327cb49cbbd0776c4971cd9d5c9375fb70cc3dc8dd1f4bd8b1245c4414e803dc0d29ed4553b00eb131d27c9c128
2020-05-03 10:30:47 -04:00
MarcoFalke
ddc0a600b3
Merge #18617: test: add factor option to adjust test timeouts
2742c34286 test: add factor option to adjust test timeouts (Harris)

Pull request description:

  This PR adds a new option **factor** that can be used to adjust timeouts in various functional tests.
  Several timeouts and functions from `authproxy`, `mininode`, `test_node` and `util` have been adapted to use this option. The factor-option definition is located in `test_framework.py`.

  Fixes https://github.com/bitcoin/bitcoin/issues/18266
  Also Fixes https://github.com/bitcoin/bitcoin/issues/18834

ACKs for top commit:
  MarcoFalke:
    Thanks! ACK 2742c34286

Tree-SHA512: 6d8421933ba2ac1b7db00b70cf2bc242d9842c48121c11aadc30b0985c4a174c86a127d6402d0cd73b993559d60d4f747872d21f9510cf4806e008349780d3ef
2020-05-03 08:58:56 -04:00
Andrew Chow
489ebfd7a1 tests: feature_backwards_compatibility.py test downgrade after upgrade
After upgrading the node, try to go back to the original version to make
sure that using a newver node version does not prevent the wallet file
from being downgraded again.
2020-05-02 23:45:17 -04:00
Harris
2742c34286
test: add factor option to adjust test timeouts
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-05-03 01:42:40 +02:00
MarcoFalke
844d2070a2
Merge #18828: test: Strip down previous releases boilerplate
fa359d14c0 test: Strip down previous releases boilerplate (MarcoFalke)

Pull request description:

  Reduces code bloat and mental load to write compatibility tests

ACKs for top commit:
  Sjors:
    tACK fa359d14c0 on macOS

Tree-SHA512: dc66286b24b2f137e5bca99412850ec7eee8cc61cf9cdc7ab532d529220808189baea8d1b077f8b7f40d3e8881d981e1ffc5a877adb394816f1225b1186253e4
2020-05-01 14:57:59 -04:00
Carl Dong
a4f6ffa71e
lint: Also enable source statements for non-gitian 2020-05-01 12:31:07 -04:00
MarcoFalke
fab7ee3990
test: Fix p2p_leak intermittent issue 2020-05-01 08:45:26 -04:00
MarcoFalke
fa8614aea9
test: Fix intermittent p2p_segwit issue 2020-05-01 08:36:35 -04:00
MarcoFalke
fa2cce4391
wallet: Remove trailing whitespace from potential translation strings
If the potential translation strings are translated in the future,
trailing whitespace is going to make translation effort harder.
2020-05-01 07:41:32 -04:00
MarcoFalke
fa59cc1c97
wallet: Report full error message in wallettool 2020-05-01 07:39:35 -04:00
MarcoFalke
fa359d14c0
test: Strip down previous releases boilerplate 2020-04-30 11:45:16 -04:00
MarcoFalke
a66ba6d029
Merge #18576: test: use unittest for test_framework unit testing
de8905adf2 test: use unittest and test_runner for test framework unit testing (Gloria Zhao)

Pull request description:

  Proposal for unit testing on test_framework functions:
  1. Use the python `unittest` library. Don't use test_framework to test itself.
  2. Put the tests inside the same file as the functions they are testing.
  3. Call the tests from `test_runner.py`. To include more Test Framework tests, add the filename to the list `TEST_FRAMEWORK_MODULES`. Don't add new files or change the list of accepted script prefixes.

  Makes these changes for `bn2vch` (followup to [this comment](https://github.com/bitcoin/bitcoin/pull/18378#pullrequestreview-377271264)).

ACKs for top commit:
  jnewbery:
    Tested ACK de8905adf2. Great stuff gzhao408 . Thanks for this!

Tree-SHA512: 91572d43e203a1864765b93a9472667994115ec38b271f2b2f9fcd0f0112b393fc24ba7d2371d5a34b0a6a4522f6b934fc5164363819aa7ed8d6c6c9a60cc101
2020-04-30 09:30:13 -04:00
MarcoFalke
36c0abd8f6
Merge #18726: test: check misbehavior more independently in p2p_filter.py
cd543d9193 test: check misbehavior more independently in p2p_filter.py (Danny Lee)

Pull request description:

  This expands on #18672 in two ways:

  - Check positive cases (`filterload` accepted, `filteradd` accepted) in addition to the negative cases added in #18672
  - Address MarcoFalke 's [suggestion](https://github.com/bitcoin/bitcoin/pull/18672#discussion_r412101752) to successfully load a filter before testing `filteradd`

ACKs for top commit:
  theStack:
    re-ACK cd543d9193

Tree-SHA512: f82402f6287ccddf08b38b6432d5e2b2b2ef528802a981d04c24bac459022f732d9090d4849d72d3d1eb2c757161dcb18c4c036b6e11dc80114e9cd49f21c3bd
2020-04-29 18:56:25 -04:00
MarcoFalke
978c5a2122
Merge #18485: test: Add mempool_updatefromblock.py
8098dea069 test: Add mempool_updatefromblock.py (Hennadii Stepanov)

Pull request description:

  This PR adds a new test for mempool update of transaction descendants/ancestors information (count, size) when transactions have been re-added from a disconnected block to the mempool.

  It could be helpful for working on PRs like #17925, #18191.

ACKs for top commit:
  ariard:
    ACK 8098dea

Tree-SHA512: 7e808fa8df8d7d7a7dbdc3f79361049b49c7bce9b58fd5539b28c9636bedac747695537e500d7ed68dc8bdb80167ad3f1c01086f7551691405d2ba2e38ef1d06
2020-04-29 12:07:13 -04:00
MarcoFalke
e302830fae
Merge #18774: test: added test for upgradewallet RPC
66fe7b1a98 test: added test for upgradewallet RPC (Harris)

Pull request description:

  This PR adds tests for the newly merged *upgradewallet* RPC.

  Additionally, it expands `test_framework/util.py` by adding the function `adjust_bitcoin_conf_for_pre_17` to support nodes that don't parse configuration sections.

  This test uses two older node versions, v0.15.2 and v0.16.3, to create older wallet versions to be used by `upgradewallet`.

  Fixes https://github.com/bitcoin/bitcoin/issues/18767

Top commit has no ACKs.

Tree-SHA512: bb72ff1e829e2c3954386cc308842820ef0828a4fbb754202b225a8748f92d4dcc5ec77fb146bfd5484a5c2f29ce95adf9f3fb4483437088ff3ea4a8d2c442c1
2020-04-29 11:09:05 -04:00
Harris
66fe7b1a98
test: added test for upgradewallet RPC 2020-04-29 16:11:49 +02:00
fanquake
0ef0d33f75
Merge #18038: P2P: Mempool tracks locally submitted transactions to improve wallet privacy
50fc4df6c4 [mempool] Persist unbroadcast set to mempool.dat (Amiti Uttarwar)
297a178536 [test] Integration tests for unbroadcast functionality (Amiti Uttarwar)
6851502472 [refactor/test] Extract P2PTxInvStore into test framework (Amiti Uttarwar)
dc1da48dc5 [wallet] Update the rebroadcast frequency to be ~1/day. (Amiti Uttarwar)
e25e42f20a [p2p] Reattempt initial send of unbroadcast transactions (Amiti Uttarwar)
7e93eecce3 [util] Add method that returns random time in milliseconds (Amiti Uttarwar)
89eeb4a333 [mempool] Track "unbroadcast" transactions (Amiti Uttarwar)

Pull request description:

  This PR introduces mempool tracking of unbroadcast transactions and periodic reattempts at initial broadcast. This is a part of the rebroadcast project, and a standalone privacy win.

  The current rebroadcast logic is terrible for privacy because 1. only the source wallet rebroadcasts transactions and 2. it does so quite frequently. In the current system, if a user submits a transaction that does not immediately get broadcast to the network (eg. they are offline), this "rebroadcast" behavior is the safety net that can actually serve as the initial broadcast. So, keeping the attempts frequent is important for initial delivery within a reasonable timespan.

  This PR aims to improve # 2 by reducing the wallet rebroadcast frequency to ~1/day from ~1/15 min. It achieves this by separating the notion of initial broadcast from rebroadcasts. With these changes, the mempool tracks locally submitted transactions & periodically reattempts initial broadcast. Transactions submitted via the wallet or RPC are added to an "unbroadcast" set & are removed when a peer sends a `getdata` request, or the transaction is removed from the mempool. Every 10-15 minutes, the node reattempts an initial broadcast. This enables reducing the wallet rebroadcast frequency while ensuring the transactions will be propagated to the network.

  For privacy improvements around # 1, please see #16698.
  Thank you to gmaxwell for the idea of how to break out this subset of functionality (https://github.com/bitcoin/bitcoin/pull/16698#issuecomment-571399346)

ACKs for top commit:
  fjahr:
    Code review ACK 50fc4df6c4
  MarcoFalke:
    ACK 50fc4df6c4, I think this is ready for merge now 👻
  amitiuttarwar:
    The current tip `50fc4df` currently has 6 ACKs on it, so I've opened #18807 to address the last bits.
  jnewbery:
    utACK 50fc4df6c4.
  ariard:
    Code Review ACK 50fc4df (minor points no need to invalid other ACKs)
  robot-visions:
    ACK 50fc4df6c4
  sipa:
    utACK 50fc4df6c4
  naumenkogs:
    utACK 50fc4df

Tree-SHA512: 2dd935d645d5e209f8abf87bfaa3ef0e4492705ce7e89ea64279cb27ffd37f4727fa94ad62d41be331177332f8edbebf3c7f4972f8cda10dd951b80a28ab3c0f
2020-04-29 16:32:37 +08:00
MarcoFalke
ba348dbc51
Merge #18805: tests: Add missing sync_all to wallet_importdescriptors.py
7b2b06dfe3 tests: Add missing sync_all to wallet_importdescriptors.py (Andrew Chow)

Pull request description:

  node1 will sometimes do sendtoaddress before it has received a funding transaction which will cause the test to fail. sync_all to ensure it gets the transaction first.

  Fixes #18800

ACKs for top commit:
  instagibbs:
    ACK 7b2b06d The wallet endpoint right after is indeed node 1.

Tree-SHA512: b610a771d062b5f955cd70b34337577a1ab8dacbf4be20aa74e1e8234495b0be9faff138eb1713f29decb5574446e0583e221bc2c9a6eea13611b422ea3a296a
2020-04-28 13:14:26 -04:00
MarcoFalke
6a60bfc76c
Merge #18765: test: Fix wallet_bumpfee intermittent error
fa301fec96 test: Fix wallet_bumpfee intermittent error (MarcoFalke)

Pull request description:

  Remove incorrect and undocumented `connect_nodes(self.nodes[0], 1)`.

  Issue is that transactions are re-relayed (going full circle) between the two nodes, that have two connections between each other.

  https://travis-ci.org/github/bitcoin/bitcoin/jobs/679201559#L6992

  Also fix some pep8 while touching the file

  This bug has been introduced by accident in c1dde3a949

ACKs for top commit:
  achow101:
    ACK fa301fec96

Tree-SHA512: a6565ca30dbe44b02e3f58f159d2515c2ea4a74030118fafc1a3391ce980a4b6d4505dcf51315fda24843f72550a7dea7407b877b3b796883dd73d3b6f009e6f
2020-04-28 13:11:57 -04:00
Andrew Chow
7b2b06dfe3 tests: Add missing sync_all to wallet_importdescriptors.py
node1 will sometimes do sendtoaddress before it has received a funding
transaction which will cause the test to fail. sync_all to ensure it
gets the transaction first.
2020-04-28 12:08:33 -04:00
MarcoFalke
fa301fec96
test: Fix wallet_bumpfee intermittent error 2020-04-28 11:53:00 -04:00
fanquake
65fb3dfc8d
Merge #18556: build: Drop make dist in gitian builds
2aa48edec0 refactor: Drop unused ${WRAP_DIR}/${HOST} directory (Hennadii Stepanov)
1362be0447 build: Drop make dist in gitian builds (Hennadii Stepanov)

Pull request description:

  After the merge of #18331, the packaged source tarball is created by `git archive`, but the binaries are built from another one which is made by `make dist`.

  With this PR the only source tarball, created by `git archive`, is used both for binaries building and for packaging to users.

  Close #16588.
  Close #18547.

  As a good side-effect, #18349 becomes redundant.

  **Change in behavior**

  The following variables 1b151e3ffc/configure.ac (L2-L6)

  are no longer used for naming of directories and tarballs.

  Instead of them the gitian descriptors use a git tag (if available) or a commit hash.

  ---

  Also a small refactor commit picked from #18404.

ACKs for top commit:
  dongcarl:
    ACK 2aa48edec0
  MarcoFalke:
    ACK 2aa48edec0
  fanquake:
    ACK 2aa48edec0 - I've had a quick look over this, and don't want to block merging if this actually gets as closer to finally having this all sorted out. Obviously we've still got #18741, and after speaking to Carl this morning, there will likely be even more changes after that (not Guix specific).

Tree-SHA512: d3b16f87e48d1790a3264940c28acd5d881bfd10f3ce94fb0c8a6af76d8039289d01e0cd4972adac49ae24362857251f6c1e5e09e3e9fbf636c10708b4015a7c
2020-04-28 16:44:17 +08:00
Samuel Dobson
eef90c14ed
Merge #16528: Native Descriptor Wallets using DescriptorScriptPubKeyMan
223588b1bb Add a --descriptors option to various tests (Andrew Chow)
869f7ab30a tests: Add RPCOverloadWrapper which overloads some disabled RPCs (Andrew Chow)
cf06062859 Correctly check for default wallet (Andrew Chow)
886e0d75f5 Implement CWallet::IsSpentKey for non-LegacySPKMans (Andrew Chow)
3c19fdd2a2 Return error when no ScriptPubKeyMan is available for specified type (Andrew Chow)
388ba94231 Change wallet_encryption.py to use signmessage instead of dumpprivkey (Andrew Chow)
1346e14831 Functional tests for descriptor wallets (Andrew Chow)
f193ea889d add importdescriptors RPC and tests for native descriptor wallets (Hugo Nguyen)
ce24a94494 Add IsLegacy to CWallet so that the GUI knows whether to show watchonly (Andrew Chow)
1cb42b22b1 Generate new descriptors when encrypting (Andrew Chow)
82ae02b165 Be able to create new wallets with DescriptorScriptPubKeyMans as backing (Andrew Chow)
b713baa75a Implement GetMetadata in DescriptorScriptPubKeyMan (Andrew Chow)
8b9603bd0b Change GetMetadata to use unique_ptr<CKeyMetadata> (Andrew Chow)
72a9540df9 Implement FillPSBT in DescriptorScriptPubKeyMan (Andrew Chow)
84b4978c02 Implement SignMessage for descriptor wallets (Andrew Chow)
bde7c9fa38 Implement SignTransaction in DescriptorScriptPubKeyMan (Andrew Chow)
d50c8ddd41 Implement GetSolvingProvider for DescriptorScriptPubKeyMan (Andrew Chow)
f1ca5feb4a Implement GetKeypoolOldestTime and only display it if greater than 0 (Andrew Chow)
586b57a9a6 Implement ReturnDestination in DescriptorScriptPubKeyMan (Andrew Chow)
f866957979 Implement GetReservedDestination in DescriptorScriptPubKeyMan (Andrew Chow)
a775f7c7fd Implement Unlock and Encrypt in DescriptorScriptPubKeyMan (Andrew Chow)
bfdd073486 Implement GetNewDestination for DescriptorScriptPubKeyMan (Andrew Chow)
58c7651821 Implement TopUp in DescriptorScriptPubKeyMan (Andrew Chow)
e014886a34 Implement SetupGeneration for DescriptorScriptPubKeyMan (Andrew Chow)
46dfb99768 Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file (Andrew Chow)
4cb9b69be0 Implement several simple functions in DescriptorScriptPubKeyMan (Andrew Chow)
d1ec3e4f19 Add IsSingleType to Descriptors (Andrew Chow)
953feb3d27 Implement loading of keys for DescriptorScriptPubKeyMan (Andrew Chow)
2363e9fcaa Load the descriptor cache from the wallet file (Andrew Chow)
46c46aebb7 Implement GetID for DescriptorScriptPubKeyMan (Andrew Chow)
ec2f9e1178 Implement IsHDEnabled in DescriptorScriptPubKeyMan (Andrew Chow)
741122d4c1 Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan (Andrew Chow)
2db7ca765c Implement IsMine for DescriptorScriptPubKeyMan (Andrew Chow)
db7177af8c Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet (Andrew Chow)
78f8a92910 Implement SetType in DescriptorScriptPubKeyMan (Andrew Chow)
834de0300c Store WalletDescriptor in DescriptorScriptPubKeyMan (Andrew Chow)
d8132669e1 Add a lock cs_desc_man for DescriptorScriptPubKeyMan (Andrew Chow)
3194a7f88a Introduce WalletDescriptor class (Andrew Chow)
6b13cd3fa8 Create LegacyScriptPubKeyMan when not a descriptor wallet (Andrew Chow)
aeac157c9d Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet (Andrew Chow)
96accc73f0 Add WALLET_FLAG_DESCRIPTORS (Andrew Chow)
6b8119af53 Introduce DescriptorScriptPubKeyMan as a dummy class (Andrew Chow)
06620302c7 Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it (Andrew Chow)

Pull request description:

  Introducing the wallet of the glorious future (again): native descriptor wallets. With native descriptor wallets, addresses are generated from descriptors. Instead of generating keys and deriving addresses from keys, addresses come from the scriptPubKeys produced by a descriptor. Native descriptor wallets will be optional for now and can only be created by using `createwallet`.

  Descriptor wallets will store descriptors, master keys from the descriptor, and descriptor cache entries. Keys are derived from descriptors on the fly. In order to allow choosing different address types, 6 descriptors are needed for normal use. There is a pair of primary and change descriptors for each of the 3 address types. With the default keypool size of 1000, each descriptor has 1000 scriptPubKeys and descriptor cache entries pregenerated. This has a side effect of making wallets large since 6000 pubkeys are written to the wallet by default, instead of the current 2000. scriptPubKeys are kept only in memory and are generated every time a descriptor is loaded. By default, we use the standard BIP 44, 49, 84 derivation paths with an external and internal derivation chain for each.

  Descriptors can also be imported with a new `importdescriptors` RPC.

  Native descriptor wallets use the `ScriptPubKeyMan` interface introduced in #16341 to add a `DescriptorScriptPubKeyMan`. This defines a different IsMine which uses the simpler model of "does this scriptPubKey exist in this wallet". Furthermore, `DescriptorScriptPubKeyMan` does not have watchonly, so with native descriptor wallets, it is not possible to have a wallet with both watchonly and non-watchonly things. Rather a wallet with `disable_private_keys` needs to be used for watchonly things.

  A `--descriptor` option was added to some tests (`wallet_basic.py`, `wallet_encryption.py`, `wallet_keypool.py`, `wallet_keypool_topup.py`, and `wallet_labels.py`) to allow for these tests to use descriptor wallets. Additionally, several RPCs are disabled for descriptor wallets (`importprivkey`, `importpubkey`, `importaddress`, `importmulti`, `addmultisigaddress`, `dumpprivkey`, `dumpwallet`, `importwallet`, and `sethdseed`).

ACKs for top commit:
  Sjors:
    utACK 223588b1bb (rebased, nits addressed)
  jonatack:
    Code review re-ACK 223588b1bb.
  fjahr:
    re-ACK 223588b1bb
  instagibbs:
    light re-ACK 223588b
  meshcollider:
    Code review ACK 223588b1bb

Tree-SHA512: 59bc52aeddbb769ed5f420d5d240d8137847ac821b588eb616b34461253510c1717d6a70bab8765631738747336ae06f45ba39603ccd17f483843e5ed9a90986
2020-04-27 12:23:05 +12:00
Gloria Zhao
de8905adf2 test: use unittest and test_runner for test framework unit testing
Test the test_framework, but don't use test_framework objects or functions to test itself

Use python unittest library and put test_framework's unit tests inside their respective files
Add the filename to TEST_FRAMEWORK_MODULES in test_runner
Aggregate all test_framework tests into one TestSuite to run before the functional tests in test_runner
Delete framework_test_script, move test_bn2vch to script.py and add to TEST_FRAMEWORK_MODULES in test_runner
2020-04-26 13:31:39 -07:00
MarcoFalke
9ddfce6712
Merge #18753: test: Fix intermittent failure in wallet_importmulti
fa8b9b5d1f test: Fix intermittent failure in wallet_importmulti (MarcoFalke)

Pull request description:

  The wallet is async, so after generating a block, we must call `syncwithvalidationinterfacequeue`. Otherwise the timestamp will be of the previous block.

  https://travis-ci.org/github/bitcoin/bitcoin/jobs/677685073#L2648

ACKs for top commit:
  promag:
    ACK fa8b9b5d1f.

Tree-SHA512: c21f9912aabbe22019d4ac9d0da06d6e46ef7f2a84d2781110e04c9836eb0ecf90a22cf2bae7f608be611670d17b20600135d1c5e5404aa1e762839816285fb4
2020-04-26 10:47:37 -04:00
MarcoFalke
fa489011d9
test: Remove raw-tx byte juggling in mempool_reorg 2020-04-25 19:36:02 -04:00
Danny Lee
cd543d9193 test: check misbehavior more independently in p2p_filter.py 2020-04-25 09:28:22 -07:00
MarcoFalke
3e7c118d65
Merge #18688: fuzz: Run in parallel
fa66280396 fuzz: Run in parallel (MarcoFalke)

Pull request description:

  Can be reviewed with `--ignore-all-space`

ACKs for top commit:
  practicalswift:
    ACK fa66280396 -- patch looks correct and the live demo in https://github.com/bitcoin-core/qa-assets/pull/11#issuecomment-615439226 convinced me: this is good!
  fanquake:
    ACK fa66280396 - this seems sane, and clearly provides some speedup. I'd post benchmarks but can't seem to get through a full run of `./test/fuzz/test_runner.py` without hitting at least a few crashes; see #18763, 18762.

Tree-SHA512: d3b545ca90c75bed27f08fe712399d0ed1ac36b13fb289c83e5606eee8dec4c19f5f5cf91758f0a6b1606d8d6b455fbe46df2588faffe7462b185bd34dc2baaf
2020-04-25 09:28:33 -04:00
Sebastian Falbesoner
0956e46bff test: use zero-argument super() shortcut (Python 3.0+)
as defined in PEP 3135:

"The new syntax:

    super()

is equivalent to:

    super(__class__, <firstarg>)

where __class__ is the class that the method was defined in, and <firstarg> is
the first parameter of the method (normally self for instance methods, and cls
for class methods)."
2020-04-25 13:49:08 +02:00
MarcoFalke
5f19155e5b
Merge #18724: test: add coverage for -rpcwallet cli option
2495110012 test: add coverage for -rpcwallet cli option (Jon Atack)

Pull request description:

  The bitcoin-cli `-rpcwallet=` option is an essential RPC/CLI option when more than one wallet is loaded (see `bitcoin-cli -help | grep -A5 rpcwallet` or `src/bitcoin-cli.cpp::L61`) and it currently has no test coverage.

  It is not only used by users, but also by the test framework and ~10 test files via `get_wallet_rpc()`.

  This PR adds coverage, while simultaneously improving the `-getinfo` coverage when multiple wallets are loaded. This is similar to the test coverage that would be added in #18594.

ACKs for top commit:
  robot-visions:
    ACK 2495110012

Tree-SHA512: caaa8b99fb8fa481ab2c6b2a287ed29720bb4553c3f66657462c44fa2990acaaf36cabeaaf81408678e5fdce4e105d729dd94b5ed8588dd1a6f2cb03fc25acf3
2020-04-24 18:41:22 -04:00
MarcoFalke
a215c61333
Merge #18756: refactor: test: use wait_for_getdata() in p2p_compactblocks.py
c4027e7350 refactor: test: use wait_for_getdata() in p2p_compactblocks.py (Sebastian Falbesoner)

Pull request description:

  The method `wait_for_getdata()` was recently changed to be more precise by waiting for a specified list of hashes, instead of only matching _any_ `getdata` message (see Issue #18614 and PR  #18690). This PR replaces the remaining occurences of manual inspection of `last_messages` with this call.

ACKs for top commit:
  robot-visions:
    ACK c4027e7350

Tree-SHA512: e10b346742f235b6ee2ef1f32f7fd74406c1a277389f020fb9913a93e94cc9530e1e9414872b83c9d2ae652ebce2b09b2c8c8372260c1afb4e0e54fbf7a935b0
2020-04-24 16:37:17 -04:00
MarcoFalke
29637a5c56
Merge #18745: test: Check submitblock return values
fa262712ca test: Check submitblock return values (MarcoFalke)

Pull request description:

  Add `assert_equal` in some tests to check the `submitblock` return value

ACKs for top commit:
  robot-visions:
    ACK fa262712ca

Tree-SHA512: 25d9effe82a4f6852184b9ac848f96336cc2cafb0bb07edb2792f00cd363f0759575bc9c164dd62f64425d3754028b4acd0675600c07d51277aa80bf66c6f960
2020-04-24 16:34:43 -04:00
Jon Atack
2495110012
test: add coverage for -rpcwallet cli option
and add -getinfo coverage with multiple wallets loaded.
2020-04-24 21:51:00 +02:00
Hennadii Stepanov
8098dea069
test: Add mempool_updatefromblock.py 2020-04-24 18:18:34 +03:00
MarcoFalke
d1aa0ae1ad
Merge #18712: test: display command line options passed to send_cli() in debug log
8f5dc8800a test: display command line options passed to send_cli() in debug log (Jon Atack)

Pull request description:

  as per https://github.com/bitcoin/bitcoin/pull/18691#discussion_r411382589, and revert two cli calls changed in #18691 from rpc commands back to command line options (these were the only occurrences).

ACKs for top commit:
  MarcoFalke:
    ACK 8f5dc8800a

Tree-SHA512: fcb3eca00aa4099066028c90d5e50a02e074366e09a17f5f5b937d9f7562dd054ff65681aa0ad4c94f6de1e98b1e2b9ac4cd084ddc297010253989a80483b1b9
2020-04-24 08:41:15 -04:00
MarcoFalke
85bae24d06
Merge #18752: test: Fix intermittent error in mempool_reorg
fae98668d1 test: Fix intermittent error in mempool_reorg (MarcoFalke)

Pull request description:

  Example: https://travis-ci.org/github/bitcoin/bitcoin/jobs/677689899#L4717

  Also speed up tx relay and fix two pep8 errors while touching the file anyway.

ACKs for top commit:
  vasild:
    utACK fae9866

Tree-SHA512: 23a7894e71ad0e1a59c74c73643708fca21b505fa4e980038d554294063fd63c396669eefb233ffdffb0083968e51b702c643cb449df8f656dd8345a20f33907
2020-04-24 07:57:48 -04:00
Sebastian Falbesoner
c4027e7350 refactor: test: use wait_for_getdata() in p2p_compactblocks.py 2020-04-24 13:55:20 +02:00
MarcoFalke
fa8b9b5d1f
test: Fix intermittent failure in wallet_importmulti 2020-04-23 19:01:26 -04:00
MarcoFalke
fae98668d1
test: Fix intermittent error in mempool_reorg 2020-04-23 18:43:44 -04:00
Amiti Uttarwar
50fc4df6c4 [mempool] Persist unbroadcast set to mempool.dat
Ensure that the unbroadcast set will still be meaningful if the node is
restarted.
2020-04-23 14:42:25 -07:00