Commit graph

22771 commits

Author SHA1 Message Date
Jonas Schnelli f05c1ac444
Merge #17937: gui: Remove WalletView and BitcoinGUI circular dependency
cb8a86d9f9 gui: Remove WalletView and BitcoinGUI circular dependency (João Barbosa)
ac3d10777d gui: Add transactionClicked and coinsSent signals to WalletView (João Barbosa)

Pull request description:

  Essentially moves the code in `WalletView::setBitcoinGUI` to the only caller. Two new signals are added beforehand in the first commit so that the connections in `WalletFrame` are all from the wallet view.

ACKs for top commit:
  hebasto:
    ACK cb8a86d9f9, tested on Linux Mint 19.3.
  jonasschnelli:
    utACK cb8a86d9f9

Tree-SHA512: 250316cd3689e51c8cded9ccd75963c836dcafa6db25d684f2aa691dea9738895f9140793e0f925784909e39f8257f7e1c7d611e8bd6d6634e1a50333f4ddb1e
2020-02-01 10:11:24 +01:00
Jonas Schnelli cadb9d3342
Merge #18036: gui: Break trivial circular dependencies
3aee10b80b gui: Drop ShutdownWindow dependency to BitcoinGUI (João Barbosa)
61eb058cc1 gui: Drop BanTableModel dependency to ClientModel (João Barbosa)

Pull request description:

  `ShutdownWindow::showShutdownWindow` just needs a widget to center the shutdown window and to borrow its title.

ACKs for top commit:
  hebasto:
    ACK 3aee10b80b, since previous review only suggested change `QWidget` --> `QMainWindow`
  jonasschnelli:
    utACK 3aee10b80b

Tree-SHA512: e15cb6ee274730bd071d3d97b540c5059e5c655248d69a37c3fd00f2aacc6cfcb36b9a65755718027e15482ec8e5e85534c1dc13d0ddb4e0680df03fbf6571f2
2020-02-01 10:07:42 +01:00
Jonas Schnelli d104aa0ace
Merge #17951: Use rolling bloom filter of recent block txs for AlreadyHave() check
a029e18c2b Use rolling bloom filter of recent block tx's for AlreadyHave() check (Suhas Daftuar)

Pull request description:

  In order to determine whether to download or process a relayed transaction, we first try to check whether we already have the transaction -- either in the mempool, in our filter of recently rejected transactions, in our orphan pool, or already confirmed in a block.

  Prior to this commit, the heuristic for checking whether a transaction was confirmed in a block is based on whether there's a coin cache entry corresponding to the 0- or 1-index vout of the tx. While that is a quick check, it is very imprecise (eg if those outputs were already spent in another block, we wouldn't detect that the transaction has already been confirmed) -- we can do better by just keeping a rolling bloom filter of the transactions in recent blocks, which will better capture the case of a transaction which has been confirmed and then fully spent.

  This should reduce the bandwidth that we waste by requesting transactions which will not be accepted to the mempool.

  To avoid relay problems for transactions which have been included in a recent block but then reorged out of the chain, we clear the bloom filter whenever a block is disconnected.

ACKs for top commit:
  MarcoFalke:
    re-ACK a029e18c2b only stylistic and comment fixups 🍴
  sipa:
    utACK a029e18c2b
  jonatack:
    Code review ACK a029e18c2b also built/ran tests and am running bitcoind with mempool debug logging and custom logging. Looked a bit into CRollingBloomFilter and also the mempool median time past checks mentioned above; I don't have a deep understanding of those areas yet but the concept here and changes LGTM. Tests and other optimisations could be added as a follow-up. In favor of seeing this move forward if no major immediate concerns.

Tree-SHA512: 784c9a35bcd3af5db469063ac7d26b4bac430e451e5637a34d8a538c3ffd1433abdd3f06e5584e7a84bfa9e791449e61819397b5a6c7890fa59d78ec3ba507b2
2020-01-31 14:42:59 +01:00
João Barbosa 3aee10b80b gui: Drop ShutdownWindow dependency to BitcoinGUI 2020-01-31 11:49:51 +00:00
João Barbosa 61eb058cc1 gui: Drop BanTableModel dependency to ClientModel 2020-01-31 08:18:27 +00:00
fanquake 3e1bf71064
Merge #18025: doc: Add missing supported rpcs to doc/descriptors.md
c7ec9a1888 Add missing supported rpcs to doc/descriptors.md (Andrew Toth)

Pull request description:

  Improve descriptor docs by adding missing rpcs.

ACKs for top commit:
  fanquake:
    ACK c7ec9a1888 - I think this has been bikeshed enough.
  jonatack:
    ACK c7ec9a1888

Tree-SHA512: 783219928ed7edc904b507bb30e2eefd8ca9f11225e1460fedecd755f9511055adcc52cc49f66ba840e121883e40753061db76a243ee6e0091daf1fc396ae59a
2020-01-31 08:43:58 +08:00
fanquake d176aeafde
Merge #18031: Remove GitHub Actions CI workflow.
085423b978 Remove GitHub Actions CI workflow. (Aaron Clauson)

Pull request description:

  While the GitHub Action CI workflow has permissions to make commits it's not suitable.

  As per #17803.

ACKs for top commit:
  fanquake:
    ACK 085423b978

Tree-SHA512: 62c19dda6164a563dc87e394314afc57ef2494b7e4af148b83220f1b2b0bef9ff309b5cde5d9bed3633429b77041871993ac5c73b0cbe7f720d6a4e60e3e816a
2020-01-31 08:21:26 +08:00
Andrew Toth c7ec9a1888 Add missing supported rpcs to doc/descriptors.md 2020-01-30 18:34:25 -05:00
MarcoFalke 1d1f8bbf57
Merge #16115: On bitcoind startup, write config args to debug.log
b951b0973c on startup, write config options to debug.log (Larry Ruane)

Pull request description:

  When a developer is examining `debug.log` after something goes wrong, it's often useful to know the exact options the failing instance of `bitcoind` was started with. Sometimes the `debug.log` file is all that's available for the analysis. This PR logs the `bitcoin.conf` entries and command-line arguments to `debug.log` on startup.

ACKs for top commit:
  MarcoFalke:
    ACK b951b0973c 🐪
  jonatack:
    ACK b951b0973c reviewed diff, re-code review, built, ran tests, launched bitcoind and reviewed debug log output, verified value of  `str` debug log in the added unit test.

Tree-SHA512: bbca4fb3d49f99261758302bde0b8b67300ccc72e7380b01f1f66a146ae8a008a045df0ca5ca9664caff034d0ee38ea7ef38a50f38374525608c07ba52790358
2020-01-31 11:10:56 +13:00
Aaron Clauson 085423b978
Remove GitHub Actions CI workflow. 2020-01-30 18:45:28 +00:00
Wladimir J. van der Laan 3b69310beb
Merge #17984: test: Add p2p test for forcerelay permission
aaaae4d0eb test: Add p2p test for forcerelay permission (MarcoFalke)
fa6b57bcaa test: Fix whitespace in p2p_permissions.py (MarcoFalke)
faf40810d7 test: Make msg_tx a witness tx (MarcoFalke)

Pull request description:

  The commit `test: Make msg_tx a witness tx` is needed so that the python mininode does not strip the witness from transactions before sending them over p2p. The commit should also be done to keep symmetry with msg_block. See:

  *  tests: Make msg_block a witness block #15982

ACKs for top commit:
  laanwj:
    ACK aaaae4d0eb

Tree-SHA512: b4b546c88f7f0576cb512f0872bc6bef9d4df65783803f226986e56175937f418aa1ed906417ac909f27f1fd521d64629621fda83250fa925c46ef9513db0e4c
2020-01-30 15:41:59 +01:00
MarcoFalke 7fcaa8291c
Merge #18009: tests: Add fuzzing harness for strprintf(…)
cc668d06fb tests: Add fuzzing harness for strprintf(...) (practicalswift)
ccc3c76e2b tests: Add fuzzer strprintf to FUZZERS_MISSING_CORPORA (temporarily) (practicalswift)
6ef04912af tests: Update FuzzedDataProvider.h from upstream (LLVM) (practicalswift)

Pull request description:

  Add fuzzing harness for `strprintf(…)`.

  Update `FuzzedDataProvider.h`.

  Avoid hitting some issues in tinyformat (reported upstreams in https://github.com/c42f/tinyformat/issues/70).

  ---

  Found issues in tinyformat:

  **Issue 1.** The following causes a signed integer overflow followed by an allocation of 9 GB of RAM (or an OOM in memory constrained environments):

  ```
  strprintf("%.777777700000000$", 1.0);
  ```

  **Issue 2.** The following causes a stack overflow:

  ```
  strprintf("%987654321000000:", 1);
  ```

  **Issue 3.** The following causes a stack overflow:

  ```
  strprintf("%1$*1$*", -11111111);
  ```

  **Issue 4.** The following causes a `NULL` pointer dereference:

  ```
  strprintf("%.1s", (char *)nullptr);
  ```

  **Issue 5.** The following causes a float cast overflow:

  ```
  strprintf("%c", -1000.0);
  ```

  **Issue 6.** The following causes a float cast overflow followed by an invalid integer negation:

  ```
  strprintf("%*", std::numeric_limits<double>::lowest());
  ```

Top commit has no ACKs.

Tree-SHA512: 9b765559281470f4983eb5aeca94bab1b15ec9837c0ee01a20f4348e9335e4ee4e4fecbd7a1a5a8ac96aabe0f9eeb597b8fc9a2c8faf1bab386e8225d5cdbc18
2020-01-31 02:56:49 +13:00
MarcoFalke 0130abbdb7
Merge #18018: tests: reset fIsBareMultisigStd after bare-multisig tests
1b96a3cd1e tests: reset fIsBareMultisigStd after bare-multisig tests (fanquake)

Pull request description:

  Fixes: #18015

  The bug this fixes is two-part.

  1. The `fIsBareMultisigStd` global is being reused by other tests,
  such as [script_p2sh_tests(set)](https://github.com/bitcoin/bitcoin/blob/master/src/test/script_p2sh_tests.cpp#L150), after being set to false.

  2. The order our tests run in doesn't always? seem to be random,
  which meant that the `script_p2sh` tests would only fail if they
  were run in an order where the `transaction_tests` ran first,
  mutating the `fIsBareMultisigStd` global.

  This doesn't seem to happen when running make check, but if you
  run `src/test/test_bitcoin and pass --random=99999`, the failure
  in `script_p2sh` will occur (on most, but maybe not all systems):

  ```bash
  src/test/test_bitcoin --random=99999
  Running 389 test cases...
  test/script_p2sh_tests.cpp:200: error: in "script_p2sh_tests/set": txTo[1].IsStandard
  test/script_p2sh_tests.cpp:200: error: in "script_p2sh_tests/set": txTo[2].IsStandard
  test/script_p2sh_tests.cpp:200: error: in "script_p2sh_tests/set": txTo[3].IsStandard

  *** 3 failures are detected in the test module "Bitcoin Core Test Suite"
  ```

  The new test for bare multisig was introduced in #17502.

ACKs for top commit:
  Empact:
    Code Review ACK 1b96a3cd1e
  theStack:
    ACK https://github.com/bitcoin/bitcoin/pull/18018/commits/1b96a3cd1ebe725896f59614903184289fe62c

Tree-SHA512: fd7578f9f3faa44d236cd007fc25e31f061acabdb8458559fde0e67d11ab5cafed15305993270c9943a50326574bc5f5301b09494a5b0d2de69e64978093ed45
2020-01-31 02:50:47 +13:00
João Barbosa cb8a86d9f9 gui: Remove WalletView and BitcoinGUI circular dependency 2020-01-30 11:38:06 +00:00
João Barbosa ac3d10777d gui: Add transactionClicked and coinsSent signals to WalletView 2020-01-30 11:31:07 +00:00
fanquake 44c2400bcc
Merge #18026: psbt_wallet_tests: use unique_ptr for GetSigningProvider
1115ba693b psbt_wallet_tests: use unique_ptr for GetSigningProvider (Anthony Towns)

Pull request description:

  #17261 changed GetSigningProvider to return a unique_ptr, but #17156 made psbt_wallet_tests use it as well, and wasn't correspondingly updated.

ACKs for top commit:
  fanquake:
    ACK 1115ba693b
  meshcollider:
    Thanks! utACK 1115ba693b

Tree-SHA512: f0191c9b00780e6d1445fa4ec531456758b468b5bca8660474d22b1edb5f48a636a940656c9bdbe466b8bffad7af1e57e0756239906e901d60c69c3124d3bff4
2020-01-30 15:23:53 +08:00
Anthony Towns 1115ba693b psbt_wallet_tests: use unique_ptr for GetSigningProvider 2020-01-30 16:09:13 +10:00
Samuel Dobson 2d6e76af24
Merge #17261: Make ScriptPubKeyMan an actual interface and the wallet to have multiple
3f373659d7 Refactor: Replace SigningProvider pointers with unique_ptrs (Andrew Chow)
3afe53c403 Cleanup: Drop unused GUI learnRelatedScripts method (Andrew Chow)
e2f02aa59e Refactor: Copy CWallet signals and print function to LegacyScriptPubKeyMan (Andrew Chow)
c729afd0a3 Box the wallet: Add multiple keyman maps and loops (Andrew Chow)
4977c30d59 refactor: define a UINT256_ONE global constant (Andrew Chow)
415afcccd3 HD Split: Avoid redundant upgrades (Andrew Chow)
01b4511206 Make UpgradeKeyMetadata work only on LegacyScriptPubKeyMan (Andrew Chow)
4a7e43e846 Store p2sh scripts in AddAndGetDestinationForScript (Andrew Chow)
501acb5538 Always try to sign for all pubkeys in multisig (Andrew Chow)
81610eddbc List output types in an array in order to be iterated over (Andrew Chow)
eb81fc3ee5 Refactor: Allow LegacyScriptPubKeyMan to be null (Andrew Chow)
fadc08ad94 Locking: Lock cs_KeyStore instead of cs_wallet in legacy keyman (Andrew Chow)
f5be479694 wallet: Improve CWallet:MarkDestinationsDirty (João Barbosa)

Pull request description:

  Continuation of wallet boxes project.

  Actually makes ScriptPubKeyMan an interface which LegacyScriptPubkeyMan. Moves around functions and things from CWallet into LegacyScriptPubKeyMan so that they are actually separate things without circular dependencies.

  ***

  Introducing the `ScriptPubKeyMan` (short for ScriptPubKeyManager) for managing scriptPubKeys and their associated scripts and keys. This functionality is moved over from `CWallet`. Instead, `CWallet` will have a pointer to a `ScriptPubKeyMan` for every possible address type, internal and external. It will fetch the correct `ScriptPubKeyMan` as necessary. When fetching new addresses, it chooses the `ScriptPubKeyMan` based on address type and whether it is change. For signing, it takes the script and asks each `ScriptPubKeyMan` for whether that `ScriptPubKeyMan` considers that script `IsMine`, whether it has that script, or whether it is able to produce a signature for it. If so, the `ScriptPubKeyMan` will provide a `SigningProvider` to the caller which will use that in order to sign.

  There is currently one `ScriptPubKeyMan` - the `LegacyScriptPubKeyMan`. Each `CWallet` will have only one `LegacyScriptPubKeyMan` with the pointers for all of the address types and change pointing to this `LegacyScriptPubKeyMan`. It is created when the wallet is loaded and all keys and metadata are loaded into it instead of `CWallet`. The `LegacyScriptPubKeyMan` is primarily made up of all of the key and script management that used to be in `CWallet`. For convenience, `CWallet` has a `GetLegacyScriptPubKeyMan` which will return the `LegacyScriptPubKeyMan` or a `nullptr` if it does not have one (not yet implemented, but callers will check for the `nullptr`). For purposes of signing, `LegacyScriptPubKeyMan`'s `GetSigningProvider` will return itself rather than a separate `SigningProvider`. This will be different for future `ScriptPubKeyMan`s.

  The `LegacyScriptPubKeyMan` will also handle the importing and exporting of keys and scripts instead of `CWallet`. As such, a number of RPCs have been limited to work only if a `LegacyScriptPubKeyMan` can be retrieved from the wallet. These RPCs are `sethdseed`, `addmultisigaddress`, `importaddress`, `importprivkey`, `importpubkey`, `importmulti`, `dumpprivkey`, and `dumpwallet`. Other RPCs which relied on the wallet for scripts and keys have been modified in order to take the `SigningProvider` retrieved from the `ScriptPubKeyMan` for a given script.

  Overall, these changes should not effect how everything actually works and the user should experience no difference between having this change and not having it. As such, no functional tests were changed, and the only unit tests changed were those that were directly accessing `CWallet` functions that have been removed.

  This PR is the last step in the [Wallet Structure Changes](https://github.com/bitcoin-core/bitcoin-devwiki/wiki/Wallet-Class-Structure-Changes).

ACKs for top commit:
  instagibbs:
    re-utACK 3f373659d7
  Sjors:
    re-utACK 3f373659d7 (it still compiles on macOS after https://github.com/bitcoin/bitcoin/pull/17261#discussion_r370377070)
  meshcollider:
    Tested re-ACK 3f373659d7

Tree-SHA512: f8e2b8d9efa750b617691e8702d217ec4c33569ec2554a060141d9eb9b9a3a5323e4216938e2485c44625d7a6e0925d40dea1362b3af9857cf08860c2f344716
2020-01-30 17:21:21 +13:00
fanquake 1b96a3cd1e
tests: reset fIsBareMultisigStd after bare-multisig tests
The bug this fixes is two-part.

1.The fIsBareMultisigStd global is being reused by other tests,
i.e script_p2sh_tests(set), after being set to false.

2. The order our tests run in doesn't always? seem to be random,
which meant that the script_p2sh tests would only fail if they
were run in an order where transaction_tests ran first, mutating
the fIsBareMultisigStd global.

This doesn't seem to happen when running make check, but if you
run src/test/test_bitcoin and pass --random=99999, the failure
in script_p2sh:

test/script_p2sh_tests.cpp:200: error: in "script_p2sh_tests/set": txTo[1].IsStandard

will occur (on most systems).

The new test was introduced in 1bb5d517aa.
2020-01-30 08:41:24 +08:00
MarcoFalke 638239de75
Merge #18022: test: Fix appveyor test_bitcoin build of *.raw
fa1a46e7f4 build: Fix appveyor test_bitcoin build of *.raw (MarcoFalke)

Pull request description:

  Fixes #18020

Top commit has no ACKs.

Tree-SHA512: c0b3ca4f95b46543bb3bc6d254300c832a69feca79f5de4e13cafd4c962ae53903069ec7a8c9573761eefa5cec617992b70750b067ee42231dc74170ba6c3b10
2020-01-29 18:12:29 -05:00
Larry Ruane b951b0973c on startup, write config options to debug.log 2020-01-29 15:44:00 -07:00
MarcoFalke fa1a46e7f4
build: Fix appveyor test_bitcoin build of *.raw 2020-01-29 16:57:41 -05:00
Samuel Dobson aabec94541
Merge #17719: Document better -keypool as a look-ahead safety mechanism
f41d589669 Document better -keypool as a look-ahead safety mechanism (Antoine Riard)

Pull request description:

  If after a backup, an address is issued beyond the initial
  keypool range and none of the addresses in this range
  is seen onchain, if a wallet is restored from backup, even in
  case of rescan, funds may be loss due to the look-ahead
  buffer not being incremented and so restored wallet not detecting
  onchain out-of-range address as derived from its seed.

  This scenario is theoretically unavoidable due to the requirement
  of the keypool to have a max size. However, given the default
  keypool size, this is unlikely. Document better keypool size
  implications to avoid user setting a too low value.

  While reviewing #17681, it took me a while to figure out the safety implications of keypool, I find it would be better to document this a bit farther to avoid users shooting themselves in the foot. For further context & discussion, see https://github.com/bitcoin/bitcoin/pull/17681#issuecomment-563613452

ACKs for top commit:
  ryanofsky:
    Code review ACK f41d589669. Just "Warning:" prefix added since the last review
  jonatack:
    ACK f41d589669 code review and build/test. The added `Warning:` since last review is a good addition.

Tree-SHA512: d3d0ee88fcdfc5c8841a2bd4bada0e4eeb412a0dce5054e5fb023643c2fa57206a0f3efb06890c245528dc4431413ed2fd5645b9319d26245d044c490b7f0db0
2020-01-30 09:10:15 +13:00
Suhas Daftuar a029e18c2b Use rolling bloom filter of recent block tx's for AlreadyHave() check
In order to determine whether to download or process a relayed transaction, we
try to determine if we already have the transaction, either in the mempool, in
our recently rejected filter, in our orphan pool, or already confirmed in the
chain itself.

Prior to this commit, the heuristic for checking the chain is based on whether
there's an output corresponding to the 0- or 1-index vout in our coin cache.
While that is a quick check, it is very imprecise (say if those outputs were
already spent in a block) -- we can do better by just keeping a rolling bloom
filter of the transactions in recent blocks, which will capture the case of a
transaction which has been confirmed and then fully spent already.

To avoid relay problems for transactions which have been included in a recent
block but then reorged out of the chain, we clear the bloom filter whenever a
block is disconnected.
2020-01-29 09:37:21 -05:00
Wladimir J. van der Laan c1607b5df4
Merge #17957: Serialization improvements step 3 (compression.h)
4de934b9b5 Convert compression.h to new serialization framework (Pieter Wuille)
ca34c5cba5 Add FORMATTER_METHODS, similar to SERIALIZE_METHODS, but for formatters (Pieter Wuille)

Pull request description:

  This is the next piece of the puzzle from #10785. It includes:
  * The `FORMATTER_METHODS` macro, similar to `SERIALIZE_METHODS`, for defining a formatter with a unified serialization/deserialization implementation.
  * Updating `compression.h` to consist of 3 formatters, rather than old-style wrappers (`ScriptCompression`, `AmountCompression`, `TxOutCompression`).

ACKs for top commit:
  laanwj:
    code review ACK 4de934b9b5
  ryanofsky:
    Code review ACK 4de934b9b5. Only change since last review is removing REF usages

Tree-SHA512: d52ca21eb1ce87d9bc3c90d00c905bd4fada522759aaa144c02a58b4d738d5e8647c0558b8ce393c707f6e3c4d20bf93781a2dcc1e1dcbd276d9b5ffd0e02cd6
2020-01-29 15:10:59 +01:00
MarcoFalke 3b5b276734
Merge #17942: doc: Improve fuzzing docs for macOS users
b6c3e84e87 doc: Improve fuzzing docs for macOS users (Fabian Jahr)

Pull request description:

  Adds several helpful hints for macOS users trying to get fuzzers to run locally using AFL or libFuzzer. These are partly based on this comment https://github.com/bitcoin/bitcoin/issues/17657#issuecomment-562869600 and discussions in the review club for #17860. See: https://bitcoincore.reviews/17860.html

  Based on the doc in the current state I could not compile fuzzers for AFL or libFuzzer. Using these hints, I can
  - compile and run fuzzers with AFL
  - compile but **not** run fuzzers with libFuzzer

  Fuzzers compiled with libFuzzers may be running but don't produce any output. Looking for others to test this to see if it is an issue with my local system. Especially interesting if you have been running libFuzzer fuzzers successfully on macOS before.

  Edit: Closes #17914

ACKs for top commit:
  MarcoFalke:
    ACK b6c3e84e87
  Sjors:
    ACK b6c3e84
  fanquake:
    ACK b6c3e84e87 - I think this has been nitpicked enough, and importantly the commands look better now.

Tree-SHA512: fdbacbcf10e9353a4ac3d22edf88663e33185ad2f244b986ff74c513de05f9fa62c4d8b17985d2f9288834c124b352cf52280627b5ff095735b411b12482e2ec
2020-01-29 08:45:09 -05:00
Wladimir J. van der Laan 01fc5891fb
Merge #16702: p2p: supplying and using asmap to improve IP bucketing in addrman
3c1bc40205 Add extra logging of asmap use and bucketing (Gleb Naumenko)
e4658aa8ea Return mapped AS in RPC call getpeerinfo (Gleb Naumenko)
ec45646de9 Integrate ASN bucketing in Addrman and add tests (Gleb Naumenko)
8feb4e4b66  Add asmap utility which queries a mapping (Gleb Naumenko)

Pull request description:

  This PR attempts to solve the problem explained in #16599.
  A particular attack which encouraged us to work on this issue is explained here  [[Erebus Attack against Bitcoin Peer-to-Peer Network](https://erebus-attack.comp.nus.edu.sg/)] (by @muoitranduc)

  Instead of relying on /16 prefix to diversify the connections every node creates, we would instead rely on the (ip -> ASN) mapping, if this mapping is provided.

  A .map file can be created by every user independently based on a router dump, or provided along with the Bitcoin release. Currently we use the python scripts written by @sipa to create a .map file, which is no larger than 2MB (awesome!).

  Here I suggest adding a field to peers.dat which would represent a hash of asmap file used while serializing addrman (or 0 for /16 prefix legacy approach).
  In this case, every time the file is updated (or grouping method changed), all buckets will be re-computed.
  I believe that alternative selective re-bucketing for only updated ranges would require substantial changes.

  TODO:
  - ~~more unit tests~~
  - ~~find a way to test the code without including >1 MB mapping file in the repo.~~
  - find a way to check that mapping file is not corrupted (checksum?)
  - comments and separate tests for asmap.cpp
  - make python code for .map generation public
  - figure out asmap distribution (?)

  ~Interesting corner case: I’m using std::hash to compute a fingerprint of asmap, and std::hash returns size_t. I guess  if a user updates the OS to 64-bit, then the hash of asap will change? Does it even matter?~

ACKs for top commit:
  laanwj:
    re-ACK 3c1bc40205
  jamesob:
    ACK 3c1bc40205 ([`jamesob/ackr/16702.3.naumenkogs.p2p_supplying_and_using`](https://github.com/jamesob/bitcoin/tree/ackr/16702.3.naumenkogs.p2p_supplying_and_using))
  jonatack:
    ACK 3c1bc40205

Tree-SHA512: e2dc6171188d5cdc2ab2c022fa49ed73a14a0acb8ae4c5ffa970172a0365942a249ad3d57e5fb134bc156a3492662c983f74bd21e78d316629dcadf71576800c
2020-01-29 13:55:43 +01:00
fanquake c434282d2c
Merge #18008: test: only declare a main() when fuzzing with AFL
b35567fe0b test: only declare a main() when fuzzing with AFL (fanquake)

Pull request description:

  This fixes fuzzing using [libFuzzer](https://llvm.org/docs/LibFuzzer.html) on macOS, which caused a few issues during the recent review club. macOS users could only fuzz using afl, or inside a VM.

  It seems that the `__attribute__((weak))` marking is not quite enough to properly mark `main()` as weak on macOS. See Apples docs on [Frameworks and Weak Linking](https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPFrameworks/Concepts/WeakLinking.html#//apple_ref/doc/uid/20002378-107262-CJBJAEID).

  Have tested fuzzing using libFuzzer and AFL with this patch.

ACKs for top commit:
  MarcoFalke:
    ACK b35567fe0b
  fjahr:
    ACK b35567f

Tree-SHA512: b881fdd98c7e1587fcf44debd31f5e7a52df938059ab91c41d0785077b3329b793e051a2bf2eee64488b9f6029d9288c911052ec23ab3ab8c0561a2be1682dae
2020-01-29 20:48:30 +08:00
fanquake e061b8dc8f
Merge #17971: refactor: Remove redundant conditional
e80317be5f refactor: Remove redundant conditional (Bushstar)

Pull request description:

  Conditional check against fMaster is now redundant as it is already checked as true. This originally made sense as the outer conditional was:

  f9cae832e6/src/checkqueue.h (L86)

  Removal of fQuit happened in the commit below.

  30ded3e3d8 (diff-88316c9aa9514c038c9304297e672da5)

ACKs for top commit:
  theStack:
    ACK e80317be5f
  hebasto:
    ACK e80317be5f, I have reviewed the code, and it looks OK, I agree it can be merged.
  promag:
    ACK e80317be5f.
  emilengler:
    re-ACK e80317be5f
  practicalswift:
    ACK e80317be5f
  Empact:
    ACK e80317be5f

Tree-SHA512: 136ea1d02e3d65100a8758730617ccede7864e08e8404e42e65d45d4bf95a3bfea2ab9895c6e8833abd654557d3efbba02b25297a2a5eefc36a11e97bbe9134f
2020-01-29 20:16:00 +08:00
fanquake 1326092e6c
Merge #17156: psbt: check that various indexes and amounts are within bounds
deaa6dd144 psbt: check output index is within bounds before accessing (Andrew Chow)
f1ef7f0aa4 Don't calculate tx fees for PSBTs with invalid money values (Andrew Chow)

Pull request description:

  Fixes #17149

  Two classes of issues were found by the psbt fuzzer: values out of range and causing overflows, and prevout indexes being out of range. This PR fixes both.

  When accessing a specific output using the index given in the tx, check that it is actually a possible output before trying to access the output.

  When summing and checking amounts for `decodepsbt` and `analyzepsbt`, make sure that the values are actually valid money values.. Otherwise, stop summing and don't show the fee. For `analyzepsbt`, return that the next role is the Creator since the Creator needs to remake the transaction to be valid.

ACKs for top commit:
  practicalswift:
    ACK deaa6dd144 -- only change since last ACK was the addition of tests
  gwillen:
    tested ACK deaa6dd, would also like to see this merged!

Tree-SHA512: 06c36720bbb5a7ab1c29f7d15878bf9f0d3e5760c06bff479d412e1bf07bb3e0e9ab6cca820a4bfedaab71bfd7af813807e87cbcdf0af25cc3f66a53a06dbcfd
2020-01-29 19:39:50 +08:00
fanquake b35567fe0b
test: only declare a main() when fuzzing with AFL
libFuzzer will provide a main(). This also fixes a weak linking
issue when fuzzing with libFuzzer on macOS.
2020-01-29 08:18:22 +08:00
Fabian Jahr b6c3e84e87
doc: Improve fuzzing docs for macOS users 2020-01-29 00:46:21 +01:00
MarcoFalke fe48ac8580
Merge #18013: bench: Fix benchmarks filters
0dae5a5c34 Fix benchmarks filters (Elichai Turkel)

Pull request description:

  The bug was introduced in https://github.com/bitcoin/bitcoin/pull/17781
  before this fix `./src/bench/bench_bitcoin -filter=*` will fail with:

  ```
  # Benchmark, evals, iterations, total, min, max, median
  bench_bitcoin: bench/bench.cpp:119: static void benchmark::BenchRunner::RunAll(benchmark::Printer&, uint64_t, double, const string&, bool): Assertion `g_testing_setup == nullptr' failed.
  Aborted (core dumped)
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 0dae5a5c34

Tree-SHA512: 43de4c7f4a5f29593972cf3bc822429466d0609c159c95d37c9e5370be392ace698b218a65542c7d53bfa52db7377ebdab808501ae109c2249f7f956bd318312
2020-01-28 14:02:13 -05:00
Elichai Turkel 0dae5a5c34
Fix benchmarks filters 2020-01-28 17:10:15 +02:00
fanquake 2755b2b109
Merge #18010: test: rename test suite name "tx_validationcache_tests" to match filename
b3c4d9bac6 test: rename test suite name "tx_validationcache_tests" to match filename (Sebastian Falbesoner)

Pull request description:

  Quoting `src/test/README.md`, '`Adding test cases`':

  >     "The file naming convention is `<source_filename>_tests.cpp`
  >      and such files should wrap their tests in a test suite
  >      called `<source_filename>_tests`."

  Currently the unit test source file `txvalidationcache_tests.cpp` contains a unit test suite with the name `tx_validationcache_tests`, which is fixed by this PR. The following shell script shows that this is the only mismatch and for all other unit test source files the test suite names are correct:

  ```
   #!/bin/bash
   shopt -s globstar
   for test_full_filename in **/*_tests.cpp; do
       test_name_file=`basename $test_full_filename .cpp`
       test_name_suite=`sed -n "s/^.*TEST_SUITE(\(.*_tests\).*$/\1/p" $test_full_filename`
       if [ $test_name_file != $test_name_suite ]; then
           echo "TestFilename: $test_name_file != TestSuitname: $test_name_suite"
       fi
   done
  ```

ACKs for top commit:
  practicalswift:
    ACK b3c4d9bac6 -- expected naming is better than unexpected naming :)
  kristapsk:
    ACK b3c4d9bac6

Tree-SHA512: 29d409b1eb22057ee2cc407508e2580d2bc03f412401df11b8ecf77be5ada6bda8f7d2cb5338c5e079490fa12242c1fd6230a09e47252c1b0d9fe535a828ca4c
2020-01-28 18:25:54 +08:00
fanquake 3774281327
Merge #17933: guix: Pin Guix using guix time-machine
88c83636d5 guix: Update documentation for time-machine (Carl Dong)
e6050884fd guix: Pin Guix using `guix time-machine` (Carl Dong)

Pull request description:

  An alternative to #16519, pinning our version of Guix and eliminating a `guix pull` and changing the default Guix profile of builders.

  I think this method might be superior, as it:
  - Eliminates the possibility of future changes to the `guix environment` command line interface breaking our builds
  - Eliminates the need to set up a separate channel repo

  It is a more general pinning solution than #16519.

  -----

  The reason why I didn't originally propose this is because `guix time-machine` is a recent addition to Guix, only available since `f675f8dec73d02e319e607559ed2316c299ae8c7`

ACKs for top commit:
  fanquake:
    ACK 88c83636d5

Tree-SHA512: 85e03b0987ffa86da73e02801e1cd8b7622698d70c4ba4e60561611be1e9717d661c2811a59b3e137b1b8eef2d0ba37c313867d035ebc89c3bd06a23a078064a
2020-01-28 17:14:50 +08:00
Sebastian Falbesoner b3c4d9bac6 test: rename test suite name "tx_validationcache_tests" to match filename
Quoting src/test/README.md, 'Adding test cases':
    "The file naming convention is `<source_filename>_tests.cpp`
     and such files should wrap their tests in a test suite
     called `<source_filename>_tests`."

Currently the unit test source file txvalidationcache_tests.cpp contains a unit
test suite with the name tx_validationcache_tests, which is fixed by this commit.
The following shell script shows that this is the only mismatch and for all other
unit test source files the test suite names are correct:

 #!/bin/bash
 shopt -s globstar
 for test_full_filename in **/*_tests.cpp; do
     test_name_file=`basename $test_full_filename .cpp`
     test_name_suite=`sed -n "s/^.*TEST_SUITE(\(.*_tests\).*$/\1/p" $test_full_filename`
     if [ $test_name_file != $test_name_suite ]; then
         echo "TestFilename: $test_name_file != TestSuitname: $test_name_suite"
     fi
 done
2020-01-27 22:44:02 +01:00
Carl Dong 88c83636d5
guix: Update documentation for time-machine
Wait a minute, doc. Are you telling me you built a time machine... Out
of a functional package manager?
2020-01-27 16:41:20 -05:00
Carl Dong e6050884fd
guix: Pin Guix using guix time-machine 2020-01-27 16:38:07 -05:00
practicalswift cc668d06fb tests: Add fuzzing harness for strprintf(...) 2020-01-27 21:31:42 +00:00
practicalswift ccc3c76e2b tests: Add fuzzer strprintf to FUZZERS_MISSING_CORPORA (temporarily) 2020-01-27 21:31:42 +00:00
practicalswift 6ef04912af tests: Update FuzzedDataProvider.h from upstream (LLVM)
Upstream revision: a44ef027eb/compiler-rt/include/fuzzer/FuzzedDataProvider.h
2020-01-27 21:31:42 +00:00
Jonas Schnelli b89f2d0599
Merge #17453: gui: Fix intro dialog labels when the prune button is toggled
4f7127d1e3 gui: Make Intro consistent with prune checkbox (Hennadii Stepanov)
4824a7d36c gui: Add Intro::UpdateFreeSpaceLabel() (Hennadii Stepanov)
daa3f3fa90 refactor: Add Intro::UpdatePruneLabels() (Hennadii Stepanov)
e4caa82a03 refactor: Replace static variable with data member (Hennadii Stepanov)
2bede28cd9 util: Add PruneGBtoMiB() function (Hennadii Stepanov)
e35e4b2ba0 util: Add PruneMiBtoGB() function (Hennadii Stepanov)

Pull request description:

  On master (a6f6333ba2) and on 0.19.0.1 the intro dialog with prune enabled (checkbox "Discard blocks..." is checked) provides a user with wrong info about the required disk space:

  ![DeepinScreenshot_bitcoin-qt_20191208112228](https://user-images.githubusercontent.com/32963518/70387510-8daab400-19ae-11ea-9338-29add9c31118.png)

  Also the paragraph "If you have chosen to limit..." is missed.

  ---

  With this PR when prune checkbox is toggled, the related text labels and the amount of required space shown are updated (previously they were only updated when the data directory was updated):
  ![Screenshot from 2019-12-08 11-34-53](https://user-images.githubusercontent.com/32963518/70387542-eed28780-19ae-11ea-9565-49d8a64b2f33.png)

  ---

  This PR is an alternative to #17035.

  **ryanofsky**'s [suggestion](https://github.com/bitcoin/bitcoin/pull/17035#discussion_r337594268) also has been implemented.

ACKs for top commit:
  emilengler:
    ACK 4f7127d1e3
  Sjors:
    tACK 4f7127d1e3
  ryanofsky:
    Code review ACK 4f7127d1e3. It seems like there are a few visible changes here:
  jonasschnelli:
    utACK 4f7127d1e3

Tree-SHA512: fa0bbdcfafde97d7906cda066cbd4608b936a71cae1b4cda3ee3aa2eed3a9795f279f14c6b1b4997278e094db891c7d3bb695368ba0882347aa42165a86e5172
2020-01-27 18:15:45 +01:00
Jonas Schnelli ef8e2cee9f
Merge #18007: Bugfix: GUI: Hide the HD/encrypt icons earlier so they get re-shown if another wallet is open
4c524f0aad Bugfix: GUI: Hide the HD/encrypt icons earlier so they get re-shown if another wallet is open (Luke Dashjr)

Pull request description:

  To reproduce bug, open 2 wallets, and close 1. You end up left without the HD/encrypt icons, despite having a wallet open still.

  This works because the icons are re-shown after we remove the current wallet (if there's another wallet still open).

ACKs for top commit:
  promag:
    Tested ACK 4c524f0aad.
  jonasschnelli:
    utACK 4c524f0aad
  hebasto:
    ACK 4c524f0aad, tested on Linux Mint 19.3.

Tree-SHA512: 4ef1bd4a0ae2f20ace9d02bc5d778640c11e46a86f30b762f8502e577f85114f0644d51a70cfbc4c23b51869c3caf20e94548aa64f51fdb85aea5f194a23fca6
2020-01-27 15:21:57 +01:00
Jonas Schnelli 3253b5dcf4
Merge #17096: gui: rename debug window
44f15cfdcf gui: renamed 'debug window' to 'node window' (Zero)

Pull request description:

  **Edit**: I have now limited the change in this PR to only renaming the window title from `Debug Window` to `Node Window`. Check [this comment](https://github.com/bitcoin/bitcoin/pull/17096#issuecomment-542837511) for more details.

  This PR is in response to #17082, which aims to rename the `Debug window` title to a more user friendly term; `Node window`.

  Closes #17082

ACKs for top commit:
  hebasto:
    ACK 44f15cfdcf, tested on Linux Mint 19.3:
  theStack:
    ACK 44f15cfdcf, tested on Linux (Lubuntu 16.04):

Tree-SHA512: 9fc73f2e67badb38525c550ce4c313288858b3fde30ef17fee85230be5bf31cf94408c699265b5e1256dfed60f8d04f48927d9b2831ba9f25498b98e6fa7180f
2020-01-27 11:31:09 +01:00
Jonas Schnelli 36ded99717
Merge #17998: gui: Shortcut to close ModalOverlay
1a638e1105 gui: Shortcut to close ModalOverlay (Emil Engler)

Pull request description:

  This adds the shortcut `Esc` to hide the ModalOverlay.
  The motivation is that it is annoying to always move the cursor to "Hide" when quickly testing something in the GUI with an outdated chain.

ACKs for top commit:
  kristapsk:
    ACK 1a638e1105. Agree with @promag, Esc feels more natural than Enter here.
  jonasschnelli:
    ACK 1a638e1105

Tree-SHA512: ea764349ec145ce9a34cbc66c3ac0eace9233a3fb3e9c22694a77882478afa22d4e686ce2c1d7b3938f6769f96ba995577b0216ba9d98954dcf3e55d2187f2e0
2020-01-27 11:14:13 +01:00
Luke Dashjr 4c524f0aad Bugfix: GUI: Hide the HD/encrypt icons earlier so they get re-shown if another wallet is open 2020-01-26 22:55:10 +00:00
MarcoFalke b4f832e24e
Merge #18001: Updated appveyor job to checkout a specific vcpkg commit ID
2bcc70531a Updated appveyor job to checkout a specific vcpkg commit ID. (Aaron Clauson)

Pull request description:

  This PR sets the vcpkg packages in stone by checking out a specific commit ID whenever they need to be reinstalled. The commit ID was chosen as the most recent commit at the time of this PR.

  As per discussion on #17995 (and prior PR's/issues).

ACKs for top commit:
  fanquake:
    ACK 2bcc70531a - thanks for trying/suggesting all the different approaches, however this looks like the way to go. Should prevent `vcpkg` packages from changing out from under us.

Tree-SHA512: ced9c9c6df2287214a966d10110edda43a06380bae270a0d9ba1fd3dde48c27c109881423978e173b3e02512f6380600096b30510e90d37a6967fd8bf9186cb6
2020-01-25 20:42:30 -05:00
Aaron Clauson 2bcc70531a
Updated appveyor job to checkout a specific vcpkg commit ID. 2020-01-25 19:07:03 +00:00
MarcoFalke c26b05c2b7
Merge #17770: test: bump test timeouts so that functional tests run in valgrind
2d23082cbe bump test timeouts so that functional tests run in valgrind (Micky Yun Chan)

Pull request description:

  ci/tests: Bump timeouts so all functional tests run on travis in valgrind #17763

Top commit has no ACKs.

Tree-SHA512: 5a8c6e2ea02b715facfcb58c761577be15ae58c45a61654beb98c2c2653361196c2eec521bcae4a9a1bab8e409d6807de771ef4c46d3d05996ae47a22d499d54
2020-01-25 11:20:47 -05:00