Commit graph

25068 commits

Author SHA1 Message Date
MarcoFalke
facdf530c7
contrib: Clean up previous_releases.py
* Replace curl single char options with their verbose counterpart
* Stricter check for tarballHash
2020-07-21 11:03:35 +02:00
MarcoFalke
ea595d39f7
Merge #19205: script: previous_release.sh rewritten in python
9c34aff393 Remove previous_release.sh (Brian Liotti)
e1e5960e10 script: Add previous_release.py (Brian Liotti)

Pull request description:

  Closes #18132

  Added functionality:
  1) checks file hash before untarring when using the binary download option

ACKs for top commit:
  fjahr:
    re-ACK 9c34aff393
  Sjors:
    tACK 9c34aff393

Tree-SHA512: 323f11828736a372a47f048592de8b027ddcd75b38f312dfc73f7b495d1e078bfeb384d9cdf434b3e70f2c6c0ce2da2df48e9a6460ac0e1967c6829a411c52d5
2020-07-21 10:11:39 +02:00
MarcoFalke
b763ae02a6
Merge #16878: Fix non-deterministic coverage of test DoS_mapOrphans
4455949d6f Make test DoS_mapOrphans deterministic (David Reikher)

Pull request description:

  This pull request proposes a solution to make the test `DoS_mapOrphans` in denialofservice_tests.cpp have deterministic coverage.

  The `RandomOrphan` function in denialofservice_tests.cpp and the implicitly called function `ecdsa_signature_parse_der_lax` in pubkey.cpp were causing the non-deterministic test coverage.

  In the former, if a random orphan was selected the index of which is bigger than the max. orphan index in `mapOrphanTransactions`, the last orphan was returned from `RandomOrphan`. If the random number generated was never large enough, this condition would not be fulfilled and the corresponding branch wouldn't run. The proposed solution is to force one of the 50 dependant orphans to depend on the last orphan in `mapOrphanTransactions` using the newly introduced function `OrphanByIndex` (and passing it a large uint256), forcing this branch to run at least once.

  In the latter, if values for ECDSA `R` or `S` (or both) had no leading zeros, some code would not be executed. The solution was to find a constant signature that would be comprised of `R` and `S` values with leading zeros and calling `CPubKey::Verify` at the end of the test with this signature forcing this code to always run at least once at the end even if it hadn't throughout the test.

  To test that the coverage is (at least highly likely) deterministic, I ran

  `contrib/devtools/test_deterministic_coverage.sh denialofservice_tests/DoS_mapOrphans 1000`

  and the result was deterministic coverage across 1000 runs.

  Also - removed denialofservice_tests test entry from the list of non-deterministic tests in the coverage script.

ACKs for top commit:
  MarcoFalke:
    ACK 4455949d6f

Tree-SHA512: 987eb1f94b80d5bec4d4944e91ef43b9b8603055750362d4b4665b7f011be27045808aa9f4c6ccf8ae009b61405f9a1b8671d65a843c3328e5b8acce1f1c00a6
2020-07-21 09:38:39 +02:00
David Reikher
4455949d6f Make test DoS_mapOrphans deterministic
The RandomOrphan function and the function ecdsa_signature_parse_der_lax
in pubkey.cpp were causing non-deterministic test coverage.

Force seed in the beginning of the test to make it deterministic.
The seed is selected carefully so that all branches of the function
ecdsa_signature_parse_der_lax are executed. Prior to this fix, the test
was exhibiting non-deterministic coverage since none of the ECDSA
signatures that were generated during the test had leading zeroes in
either R, S, or both, resulting in some branches of said function not
being executed. The seed ensures that both conditions are hit.

Removed denialofservice_tests test entry from the list of non-deterministic
tests in the coverage script.
2020-07-21 09:18:57 +03:00
fanquake
caf1766daf
Merge #19547: doc: Update macOS cross compilation dependencies for Focal
6be3eebd74 doc: Update macOS cross compilation dependencies for Focal (Hennadii Stepanov)

Pull request description:

  The [`libtinfo5`](https://packages.ubuntu.com/focal/libtinfo5) package is required on Ubuntu Focal for macOS cross compilation.

  Fixes #19546.

ACKs for top commit:
  fanquake:
    ACK 6be3eebd74 I have been [installing the libtinfo5 package](https://github.com/fanquake/core-review/blob/master/docker/debian9.dockerfile#L18) into my cross-compiling images, so mostly forgot about this.

Tree-SHA512: f40ab1efae549276e520f9cc3200438e70297c3081a3f9acc3075ec11d494fd6176ed3f1d4889b39eb2a5706eac835844a75b79335bb7f3c21c985e0779a7562
2020-07-21 14:02:42 +08:00
fanquake
f5000ae66c
Merge #19555: rpc: deduplicate WriteHDKeypath() used in decodepsbt
55057ffc51 rpc: deduplicate WriteHDKeypath() used in decodepsbt (Sebastian Falbesoner)

Pull request description:

  The functionality is already provided in the BIP32 utility library `util/bip32.h` with the exact same name and function signature.

ACKs for top commit:
  achow101:
    ACK 55057ffc51
  instagibbs:
    utACK 55057ffc51
  jonatack:
    ACK 55057ffc51

Tree-SHA512: 074c1a71ffb32908926bf07f0c5428a46309f6e0d21e7c20b1008197c820b97776a441736d0b6fd8ab0c0852522a0b5a5ddb26a1e4a1100ca02aabc65a07a018
2020-07-21 09:11:48 +08:00
Sebastian Falbesoner
55057ffc51 rpc: deduplicate WriteHDKeypath() used in decodepsbt
The functionality is already provided in the BIP32 utility library util/bip32.h
with the exact same name and function signature.
2020-07-19 17:25:07 +02:00
fanquake
476436b2de
Merge #19551: fuzz: fix unused variable compiler warning in addrdb
1cdc2a642b fuzz: fix unused variable addrdb compiler warning (Jon Atack)

Pull request description:

  Fixes #19549

ACKs for top commit:
  hebasto:
    ACK 1cdc2a642b, tested on Linux Mint 20 (x86_64, clang 10.0).
  MarcoFalke:
    review re-ACK 1cdc2a642b
  practicalswift:
    ACK 1cdc2a642b

Tree-SHA512: 5ce5ce06f0a3121911ae2ac4ed3953e5e56e6a7bd50a60ee6df84873eec75d2997488d2435bd55294e890bbf6fc368dd4302cf2a6ab6073ec74c7711eb0340e1
2020-07-19 19:25:44 +08:00
Jon Atack
1cdc2a642b
fuzz: fix unused variable addrdb compiler warning 2020-07-19 08:31:34 +02:00
Hennadii Stepanov
6be3eebd74
doc: Update macOS cross compilation dependencies for Focal 2020-07-18 18:05:40 +03:00
MarcoFalke
090d877160
Merge #19143: tests: Add fuzzing harnesses for CAutoFile, CBufferedFile, LoadExternalBlockFile and other FILE* consumers
ad6c34881d tests: Add fuzzing harness for CBlockPolicyEstimator::{Read,Write} (policy/fees.h) (practicalswift)
614e0807a8 tests: Add fuzzing harness for CBufferedFile::{SetPos,GetPos,GetType,GetVersion} (stream.h) (practicalswift)
7bcc71e5f8 tests: Add fuzzing harness for LoadExternalBlockFile(...) (validation.h) (practicalswift)
9823376030 tests: Add fuzzing harness for CBufferedFile (streams.h) (practicalswift)
f3aa659be6 tests: Add fuzzing harness for CAutoFile (streams.h) (practicalswift)
e507c0799d tests: Add serialization/deserialization fuzzing helpers WriteToStream(…)/ReadFromStream(…) (practicalswift)
e48094a506 tests: Add FuzzedAutoFileProvider which provides a CAutoFile interface to FuzzedDataProvider (practicalswift)
9dbcd6854c tests: Add FuzzedFileProvider which provides a FILE* interface to FuzzedDataProvider using fopencookie (practicalswift)

Pull request description:

  Add fuzzing harnesses for `CAutoFile`, `CBufferedFile`, `LoadExternalBlockFile` and other `FILE*` consumers:
  * Add `FuzzedFileProvider` which provides a `FILE*` interface to `FuzzedDataProvider` using `fopencookie`
  * Add `FuzzedAutoFileProvider` which provides a `CAutoFile` interface to `FuzzedDataProvider`
  * Add serialization/deserialization fuzzing helpers `WriteToStream(…)`/`ReadFromStream(…)`
  * Add fuzzing harness for `CAutoFile` (`streams.h`)
  * Add fuzzing harness for `CBufferedFile` (`streams.h`)
  * Add fuzzing harness for `LoadExternalBlockFile(...)` (`validation.h`)
  * Add fuzzing harness for `CBlockPolicyEstimator::Read` and `CBlockPolicyEstimator::Write` (`policy/fees.h`)

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  Crypt-iQ:
    Tested ACK ad6c348

Tree-SHA512: a38e142608218496796a527d7e59b74e30279a2815450408b7c27a76ed600cebc6b88491e831665a0639671e2d212453fcdca558500bbadbeb32b267751f8f72
2020-07-18 09:24:58 +02:00
MarcoFalke
fd59670642
Merge #19538: ci: Add tsan suppression for race in DatabaseBatch
0cdf2a77dd ci: add tsan debug symbols option (Russell Yanofsky)
9a2f12680b ci: Add tsan suppression for race in DatabaseBatch (Hennadii Stepanov)

Pull request description:

  Since #19325 was merged, the corresponding change in TSan suppression file gets required.

  This PR is:
  - an analogous to #19226 and #19450, and
  - a temporary workaround for CI fail like https://cirrus-ci.com/task/5741795508224000?command=ci#L4993

ACKs for top commit:
  MarcoFalke:
    ACK 0cdf2a77dd

Tree-SHA512: 7832f143887c8a0df99dea03e00694621710378fbe923e3592185fcd3658546a590693b513abffc5ab96e9ef76c9c4bff3330eeee69a0c5dbe7574f34c417220
2020-07-17 17:02:41 +02:00
Russell Yanofsky
0cdf2a77dd
ci: add tsan debug symbols option
https://github.com/bitcoin/bitcoin/issues/19049#issuecomment-639255156
2020-07-17 12:14:41 +03:00
fanquake
c04485850e
Merge #19536: qt, build: Fix QFileDialog for static builds
6457361e90 qt: Fix QFileDialog for static builds (Hennadii Stepanov)

Pull request description:

  This change partially reverts 248e22bbc0 (#16386) and makes `QFileDialog`s work again for static builds.

  Fixes https://github.com/bitcoin-core/gui/issues/32.

ACKs for top commit:
  fanquake:
    ACK 6457361e90. Although it would be good to know exactly _why_ this fixes the issue. At this stage I also don't think this should be a blocker for 0.20.1.
  theuni:
    ACK 6457361e90

Tree-SHA512: 8ad27e0bcae6debd02f73b7c374743e37d4edd806922b103a2fe494cf2d9930fe9ef3107b5a6c61f3c466cf7462de2641171880398954e7f2c4f417f5bb820d7
2020-07-17 14:25:07 +08:00
MarcoFalke
19aaf7945e
Merge #19423: test: add functional test for txrelay during and after IBD
cb31ee01b4 [test] feefilter during and after IBD (gzhao408)

Pull request description:

  This is a followup to #19204 which uses `minfeefilter=MAX_MONEY` to effectively shut off txrelay, thereby reducing inv traffic, when nodes are in IBD. It was [missing](https://github.com/bitcoin/bitcoin/pull/19204#issuecomment-644040070) a functional test.

ACKs for top commit:
  jnewbery:
    utACK cb31ee01b4

Tree-SHA512: a9effc8193fa95fb42a2f9c66b258cc7b0941fc04c1ce3a6092f4426c9bfc7e72f702aca559b3e30e90652497f411f22fae3cf5cdb6cfd6ef6d37fed712cda67
2020-07-17 07:51:24 +02:00
Hennadii Stepanov
9a2f12680b
ci: Add tsan suppression for race in DatabaseBatch 2020-07-16 22:41:10 +03:00
Wladimir J. van der Laan
c57dc566b0
Merge #16525: Dump transaction version as an unsigned integer in RPC/TxToUniv
e80259f197 Additionally treat Tx.nVersion as unsigned in joinpsbts (Matt Corallo)
970de70bdd Dump transaction version as an unsigned integer in RPC/TxToUniv (Matt Corallo)

Pull request description:

  Consensus-wise we already treat it as an unsigned integer (the
  only rules around it are in CSV/locktime handling), but changing
  the underlying data type means touching consensus code for a
  simple cleanup change, which isn't really worth it.

  See-also, https://github.com/rust-bitcoin/rust-bitcoin/pull/299

ACKs for top commit:
  sipa:
    ACK e80259f197
  practicalswift:
    ACK e80259f197
  ajtowns:
    ACK e80259f197 code review -- checked all other uses of tx.nVersion treat it as unsigned (except for policy.cpp:IsStandard anyway), so looks good.
  naumenkogs:
    ACK e80259f

Tree-SHA512: 6760a2c77e24e9e1f79a336ca925f9bbca3a827ce02003c71d7f214b82ed3dea13fa7d9f87df9b9445cd58dff8b44a15571d821c876f22f8e5a372a014c9976b
2020-07-16 21:38:09 +02:00
Hennadii Stepanov
6457361e90
qt: Fix QFileDialog for static builds
This change partially reverts 248e22bbc0.
2020-07-16 20:29:49 +03:00
fanquake
1148f3d630
Merge #17919: depends: Allow building with system clang
de4fedb6c3 depends: Add documentation for FORCE_USE_SYSTEM_CLANG make flag (Carl Dong)
fe98999dcf depends: Reformat make options as definition list (Carl Dong)
60c55b1b9b depends: Add justifications for macOS clang flags (Carl Dong)
6b8e497eea depends: specify libc++ header location for darwin (Cory Fields)
156b604203 depends: force a new host id string if FORCE_USE_SYSTEM_CLANG is in use (Cory Fields)
c9c572a367 depends: Allow building with system clang (Carl Dong)
e6e5c8d6ca depends: Decouple toolchain + binutils (Carl Dong)

Pull request description:

  This replaces: #17099

  -----

  This patchset allows us to force depends to use system clang.

  Previously, #17099 removes our dependency on a specific clang we download from llvm.org, but theuni pointed out that since OSX builds are only ever built with a version of clang that is chosen and "blessed" by Apple, it is more likely that the user will encounter problems if they use their system clang. This patchset forces the user to set `FORCE_USE_SYSTEM_CLANG=1` in order to use their system clang (when they know what they're doing)

ACKs for top commit:
  theuni:
    ACK de4fedb6c3.

Tree-SHA512: 8774121e035f90c27030bcce06e1b79f7729b5e17802c718e49652ab06e19780632db974df47423c1d1b04f1ab1b7a763554fb922fec05d1cd6445b26578be1d
2020-07-16 22:41:09 +08:00
MarcoFalke
0f6900e780
Merge #19533: [tests] Remove unnecessary cs_mains in denialofservice_tests
f58c4b538e [tests] Remove unnecessary cs_mains in denialofservice_tests (Matt Corallo)

Pull request description:

  9fdf05d70c resolved some lock
  inversion warnings in denialofservice_tests, but left in a number
  of cs_main locks that are unnecessary (introducing lock inversion
  warnings in future changes).

ACKs for top commit:
  promag:
    ACK f58c4b538e.
  jonatack:
    ACK f58c4b538e verified the test locks correspond to the locks in net/net_processing, and the debug build is clean/unit tests pass.

Tree-SHA512: de2d9b2a8f08081b2ce31e18585e4677b167a11752b797d790c281575d7dfef3587f8be4fc7f8f16771141b6ff0b0145c7488cf30e79256b0043947c67a6182c
2020-07-16 13:03:49 +02:00
MarcoFalke
6a53c3e390
Merge bitcoin-core/gui#14: scripted-diff: rename movie folder to animation
80968cff68 scripted-diff: rename movie folder to animation (Peter Bushnell)

Pull request description:

  Rename the movies directory and RES_MOVIES make variable to animation and RES_ANIMATION respectively. Movies is a bit of an unexpected term to be found.

ACKs for top commit:
  MarcoFalke:
    ACK 80968cff68
  hebasto:
    ACK 80968cff68, tested on Linux Mint 20 (Qt 5.12.8).

Tree-SHA512: 6bd31ce36e821f6a1bef8a7972086a2387d6258c48fc9df12d3ffdae07d0237036afbc2dec673384b78d9567b91d6e12eafa59fa2305aa79153dfd9b7c3a8655
2020-07-16 09:55:19 +02:00
MarcoFalke
9a714c51dc
Merge bitcoin-core/gui#34: Show permissions instead of whitelisted
784ef8be41 gui: Show permissions instead of whitelisted (Wladimir J. van der Laan)

Pull request description:

  Show detailed permissions instead of legacy "whitelisted" flag in the peer list details.
  These are formatted with `&` in between just like services flags. It reuses the "N/A" translation message if there are no special permissions.
  This removes the one-but-last use of `legacyWhitelisted`.

Top commit has no ACKs.

Tree-SHA512: 11982da4b9d408c74bc56bb3c540c0eb22506be6353aa4d4d6c64461d140f0587be194e2daad1612fddaa2618025a856b33928ad89041558f418f721f6abd407
2020-07-16 08:11:36 +02:00
MarcoFalke
affed844ba
Merge #19174: refactor: replace CConnman pointers by references in net_processing.cpp
0c8461a88e refactor: replace CConnman pointers by references in net_processing.cpp (Sebastian Falbesoner)

Pull request description:

  This is a follow-up to the recently merged PR https://github.com/bitcoin/bitcoin/pull/19053, replacing ~~two more types of~~ one more type of pointer (CConnman) by references to increase the code quality -- pointers should either check for `nullptr` or be replaced by references, and the latter strategy seems to be more reasonable.

  Again, to keep the review burden managable, the changes are kept simple,
  * only tackling `CConnman*` ~~and `BanMan*`~~ pointers
  * only within the net_processing module, i.e. no changes that would need adaption in other modules
  * keeping the names of the variables as they are

ACKs for top commit:
  jnewbery:
    utACK 0c8461a88e
  MarcoFalke:
    ACK 0c8461a88e 🕧

Tree-SHA512: 79dc05144bcfb5e0bbc62180285aadcc6199f044fa3016c0f54f7b7f45037415260970037bd63b18fafefb8aef448549dae14b780bafb540fa2373f493a17f71
2020-07-16 08:07:25 +02:00
gzhao408
cb31ee01b4 [test] feefilter during and after IBD
Co-authored-by: Jon Atack <jon@atack.com>
2020-07-15 16:42:06 -07:00
Wladimir J. van der Laan
784ef8be41 gui: Show permissions instead of whitelisted
Show detailed permissions instead of legacy "whitelisted" flag.
These are formatted with `&` in between just like services flags.
It reuses the "N/A" translation message if not.
This removes the one-but-last use of `legacyWhitelisted`.
2020-07-15 22:54:17 +02:00
Wladimir J. van der Laan
3864219d40
Merge #19360: net: improve encapsulation of CNetAddr
bc74a40a56 net: improve encapsulation of CNetAddr (Vasil Dimov)

Pull request description:

  Do not access `CNetAddr::ip` directly from `CService` methods.

  This improvement will help later when we change the type of
  `CNetAddr::ip` (in the BIP155 implementation).

  (chopped off from https://github.com/bitcoin/bitcoin/pull/19031 to ease review)

ACKs for top commit:
  dongcarl:
    ACK bc74a40a56
  naumenkogs:
    ACK bc74a40
  fjahr:
    Code review ACK bc74a40
  laanwj:
    code review ACK bc74a40a56
  jonatack:
    ACK bc74a40a56
  jnewbery:
    ACK bc74a40a5

Tree-SHA512: 29a203905538e8311e3249b78565abe69ce36dc4ec239bec85c726c30e1a7b55b0aaf5c6659b676935008e068cfa53d716f7a598469064108daf130f94329a5d
2020-07-15 21:17:18 +02:00
Wladimir J. van der Laan
31bdd86631
Merge #19353: Fix mistakenly swapped "previous" and "current" lock orders
0ecff9dd34 Improve "detected inconsistent lock order" error message (Hennadii Stepanov)
bbe9cf4fe4 test: Improve "potential deadlock detected" exception message (Hennadii Stepanov)
35599344c8 Fix mistakenly swapped "previous" and "current" lock orders (Hennadii Stepanov)

Pull request description:

  In master (8ef15e8a86) the "previous" and "current" lock orders are mistakenly swapped.

  This PR:
  - fixes printed lock orders
  - improves the `sync_tests` unit test
  - makes the "detected inconsistent lock order" error message pointing to the lock location rather `tfm::format()` location.

  Debugger output example with this PR (with modified code, of course):
  ```
  2020-06-22T15:46:56Z [msghand] POTENTIAL DEADLOCK DETECTED
  2020-06-22T15:46:56Z [msghand] Previous lock order was:
  2020-06-22T15:46:56Z [msghand]  (2) 'cs_main' in net_processing.cpp:2545 (in thread 'msghand')
  2020-06-22T15:46:56Z [msghand]  (1) 'g_cs_orphans' in net_processing.cpp:1400 (in thread 'msghand')
  2020-06-22T15:46:56Z [msghand] Current lock order is:
  2020-06-22T15:46:56Z [msghand]  (1) 'g_cs_orphans' in net_processing.cpp:2816 (in thread 'msghand')
  2020-06-22T15:46:56Z [msghand]  (2) 'cs_main' in net_processing.cpp:2816 (in thread 'msghand')
  Assertion failed: detected inconsistent lock order for 'cs_main' in net_processing.cpp:2816 (in thread 'msghand'), details in debug log.
  Process 131393 stopped
  * thread #15, name = 'b-msghand', stop reason = signal SIGABRT
      frame #0: 0x00007ffff775c18b libc.so.6`__GI_raise(sig=2) at raise.c:51:1
  (lldb) bt
  * thread #15, name = 'b-msghand', stop reason = signal SIGABRT
    * frame #0: 0x00007ffff775c18b libc.so.6`__GI_raise(sig=2) at raise.c:51:1
      frame #1: 0x00007ffff773b859 libc.so.6`__GI_abort at abort.c:79:7
      frame #2: 0x0000555555e5b196 bitcoind`(anonymous namespace)::potential_deadlock_detected(mismatch=0x00007fff99ff6f30, s1=size=2, s2=size=2, lock_location=0x00007fff99ff7010) at sync.cpp:134:9
      frame #3: 0x0000555555e5a1b1 bitcoind`(anonymous namespace)::push_lock(c=0x0000555556379220, locklocation=0x00007fff99ff7010) at sync.cpp:158:13
      frame #4: 0x0000555555e59e8a bitcoind`EnterCritical(pszName="cs_main", pszFile="net_processing.cpp", nLine=2816, cs=0x0000555556379220, fTry=false) at sync.cpp:177:5
      frame #5: 0x00005555555b0500 bitcoind`UniqueLock<AnnotatedMixin<std::recursive_mutex>, std::unique_lock<std::recursive_mutex> >::Enter(this=0x00007fff99ff8c20, pszName="cs_main", pszFile="net_processing.cpp", nLine=2816) at sync.h:134:9
      frame #6: 0x00005555555b017f bitcoind`UniqueLock<AnnotatedMixin<std::recursive_mutex>, std::unique_lock<std::recursive_mutex> >::UniqueLock(this=0x00007fff99ff8c20, mutexIn=0x0000555556379220, pszName="cs_main", pszFile="net_processing.cpp", nLine=2816, fTry=false) at sync.h:160:13
      frame #7: 0x00005555556aa57e bitcoind`ProcessMessage(pfrom=0x00007fff90001180, msg_type=error: summary string parsing error, vRecv=0x00007fff9c005ac0, nTimeReceived=1592840815980751, chainparams=0x00005555564b7110, chainman=0x0000555556380880, mempool=0x0000555556380ae0, connman=0x000055555657aa20, banman=0x00005555565167b0, interruptMsgProc=0x00005555565cae90) at net_processing.cpp:2816:9
  ```

ACKs for top commit:
  laanwj:
    ACK 0ecff9dd34
  vasild:
    ACK 0ecff9dd

Tree-SHA512: ff285de8dd3198b5b33c4bfbdadf9b1448189c96143b9696bc4f41c07e784c00851ec169cf3ed45cc325f3617ba6783620803234f57fcce28bf6bc3d6a7234fb
2020-07-15 20:53:40 +02:00
MarcoFalke
804ca26629
Merge #19386: rpc: Assert that RPCArg names are equal to CRPCCommand ones (server)
fa7592bfa8 rpc: Update server to use new RPCHelpMan (MarcoFalke)
aaaaad5627 rpc: Add option to hide RPCArg (MarcoFalke)
fa9708f94c rpc: Assert that passed arg names are equal to hardcoded ones (MarcoFalke)
faaeb2b0b3 rpc: Add CRPCCommand constructor which takes RPCHelpMan (MarcoFalke)
fa8ec00061 rpc: Check that left section is not multiline (MarcoFalke)

Pull request description:

  This is split out from #18531 to just touch the RPC methods in server. Description from the main pr:

  ### Motivation

  RPCArg names in the rpc help are currently only used for documentation. However, in the future they could be used to teach the server the named arguments. Named arguments are currently registered by the `CRPCCommand`s and duplicate the RPCArg names from the documentation. This redundancy is fragile, and has lead to errors in the past (despite having linters to catch those kind of errors). See section "bugs found" for a list of bugs that have been found as a result of the changes here.

  ### Changes

  The changes here add an assert in the `CRPCCommand` constructor that the RPCArg names are identical to the ones in the `CRPCCommand`.

  ### Future work

  > Here or follow up, makes sense to also assert type of returned UniValue?

  Sure, but let's not get ahead of ourselves. I am going to submit any further works as follow-ups, including:

  * Removing the CRPCCommand arguments, now that they are asserted to be equal and thus redundant
  * Removing all python regex linters on the args, now that RPCMan can be used to generate any output, including the cli.cpp table
  * Auto-formatting and sanity checking the RPCExamples with RPCMan
  * Checking passed-in json in self-check. Removing redundant checks
  * Checking returned json against documentation to avoid regressions or false documentation
  * Compile the RPC documentation at compile-time to ensure it doesn't change at runtime and is completely static

  ### Bugs found

  * The assert identified issue #18607
  * The changes itself fixed bug #19250

ACKs for top commit:
  laanwj:
    ACK fa7592bfa8
  ryanofsky:
    Code review ACK fa7592bfa8. Looks great! Just some hidden arg and Check() and comment cleanups since last review

Tree-SHA512: e64b6a212f4a3aeedeee47557559bde104d5fd40cdc1746b27eb2f3d4c8885d5e6e4dd287595ea11cdbc6a939654fe103cae765fd505875444d851f0abb11308
2020-07-15 19:20:21 +02:00
Matt Corallo
f58c4b538e [tests] Remove unnecessary cs_mains in denialofservice_tests
9fdf05d70c resolved some lock
inversion warnings in denialofservice_tests, but left in a number
of cs_main locks that are unnecessary (introducing lock inversion
warnings in future changes).
2020-07-15 16:34:05 +01:00
Wladimir J. van der Laan
d626a3be31
Merge #19210: qt: Get rid of cursor in out-of-focus labels
bd315eb5e2 qt: Get rid of cursor in out-of-focus labels (Hennadii Stepanov)

Pull request description:

  After clicking on `QLabel` with selectable text the cursor remains forever:

  ![47532924-65e7b200-d8ba-11e8-9254-7bde658961cb](https://user-images.githubusercontent.com/32963518/84038485-ad945200-a9a8-11ea-89e3-c7c17d02a611.png)

  This PR fixes this visual bug.

  Earlier attempts to fix this issue:
  - #14577
  - #14810 (combined with other UX feature)

ACKs for top commit:
  promag:
    Code review ACK bd315eb5e2.
  laanwj:
    Tested ACK bd315eb5e2

Tree-SHA512: 6bf89362412e5ce9a4dec6944b62fe44fc31ca49cda7f6e2eb37e847fac9dccb68bca7ac6877b19e42add2333e40d0b4265757ead105ac0a5d28f8ab43b322c3
2020-07-15 16:48:03 +02:00
Wladimir J. van der Laan
21209c9cce
Merge #19512: p2p: banscore updates to gui, tests, release notes
fa108d6a75 test: update tests for peer discouragement (Jon Atack)
1a9f462caa gui, doc: rm Ban Score in GUI Peers window/release notes updates (Jon Atack)

Pull request description:

  This is the third `-banscore` PR in the mini-series described in #19464. See that PR for the intention and reasoning.

  - no longer display "Ban Score" in the GUI peers window and add a release note, plus release note fixups per https://github.com/bitcoin/bitcoin/pull/19464#pullrequestreview-447452052
  - update tests (`src/test/denialofservice_tests.cpp` and `test/functional/p2p_leak.py`) from banning to discouragement and per https://github.com/bitcoin/bitcoin/pull/19464#issuecomment-658052518

ACKs for top commit:
  jnewbery:
    ACK fa108d6a75
  laanwj:
    ACK fa108d6a75

Tree-SHA512: 58a449b3f47b8cb5490b34e4442ee8675bfad1ce48af4e4fd5c67715b0c1a596fb8e731d42e576b4c3b64627f76e0a68cbb1da9ea9f588a5932fe119baf40d50
2020-07-15 16:32:27 +02:00
Wladimir J. van der Laan
7ebc365047
Merge #19214: Auto-detect SHA256 implementation in benchmarks
addf18da95 Call SHA256AutoDetect in benchmark setup (Pieter Wuille)

Pull request description:

  It seems `SHA256AutoDetect()` was not being called in benchmarks, making the numbers only reflect the naive implementation. Fix this by calling it in bench_bitcoin's setup.

ACKs for top commit:
  fjahr:
    tested ACK addf18da95
  pstratem:
    ACK addf18da95
  laanwj:
    ACK addf18da95

Tree-SHA512: 3ba4b068145942df1429bf5913e3f685511e6ebeae2c1a3f9b8ac0144f6db1c7df456f88f480a2129f3e1602e3bf6a39530bb96e2c74c03ddb19324cec6799c7
2020-07-15 15:35:11 +02:00
Wladimir J. van der Laan
43125596ce
Merge #19296: tests: Add fuzzing harness for AES{CBC,}256{Encrypt,Decrypt}, poly1305_auth, CHKDF_HMAC_SHA256_L32, ChaCha20 and ChaCha20Poly1305AEAD
cca7c577d5 tests: Add fuzzing harness for ChaCha20Poly1305AEAD (practicalswift)
2fc4e5916c tests: Add fuzzing harness for ChaCha20 (practicalswift)
e9e8aac029 tests: Add fuzzing harness for CHKDF_HMAC_SHA256_L32 (practicalswift)
ec86ca1aaa tests: Add fuzzing harness for poly1305_auth(...) (practicalswift)
4cee53bba7 tests: Add fuzzing harness for AES256CBCEncrypt/AES256CBCDecrypt (practicalswift)
9352c32325 tests: Add fuzzing harness for AES256Encrypt/AES256Decrypt (practicalswift)

Pull request description:

  Add fuzzing harness for `AES{CBC,}256{Encrypt,Decrypt}`, `poly1305_auth`, `CHKDF_HMAC_SHA256_L32`, `ChaCha20` and `ChaCha20Poly1305AEAD`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  laanwj:
    ACK cca7c577d5

Tree-SHA512: cff9acefe370c12a3663aa55145371df835479c6ab8f6d81bbf84e0f81a9d6b0d94e45ec545f9dd5e1702744eaa7947a1f4ffed0171f446fc080369161afd740
2020-07-15 14:45:12 +02:00
Wladimir J. van der Laan
2c4093ebc3
Merge #18882: build: fix -Wformat-security check when compiling with GCC
6cef3652d1 build: fix -Wformat-security check when compiling with GCC (fanquake)

Pull request description:

  GCC expects `-Wformat` to be passed with [`-Wformat-security`](https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html), which means
  when we test for it in configure it currently fails:

  ```bash
  checking whether C++ compiler accepts -Wformat-security... no
  ...
  configure:15907: checking whether C++ compiler accepts -Wformat-security
  configure:15926: g++ -std=c++11 -c -g -O2 -Werror -Wformat-security  conftest.cpp >&5
  cc1plus: error: '-Wformat-security' ignored without '-Wformat' [-Werror=format-security]
  cc1plus: all warnings being treated as errors
  ```

  and never gets added to our CXX flags. Note that Clang does not have this requirement and the check is working correctly there.

  The change in this PR is the simple fix, however we might want to consider using something like `-Wformat=2` in future, which in GCC is equivalent to `-Wformat -Wformat-nonliteral -Wformat-security -Wformat-y2k.` and similar [in Clang](https://clang.llvm.org/docs/DiagnosticsReference.html#wformat-2).

ACKs for top commit:
  practicalswift:
    ACK 6cef3652d1
  laanwj:
    ACK 6cef3652d1

Tree-SHA512: f9230d42af39f85ea9d2f55dbbebd2bae4740fe59b0da2e092af3ac9ef7e6799d3a4cf83eb64574c63982e5f6b14e226d44c84fa0335255d65c9947d86a1ea38
2020-07-15 13:57:41 +02:00
practicalswift
ad6c34881d tests: Add fuzzing harness for CBlockPolicyEstimator::{Read,Write} (policy/fees.h) 2020-07-15 11:41:21 +00:00
practicalswift
614e0807a8 tests: Add fuzzing harness for CBufferedFile::{SetPos,GetPos,GetType,GetVersion} (stream.h) 2020-07-15 11:41:21 +00:00
practicalswift
7bcc71e5f8 tests: Add fuzzing harness for LoadExternalBlockFile(...) (validation.h) 2020-07-15 11:41:21 +00:00
practicalswift
9823376030 tests: Add fuzzing harness for CBufferedFile (streams.h) 2020-07-15 11:41:21 +00:00
practicalswift
f3aa659be6 tests: Add fuzzing harness for CAutoFile (streams.h) 2020-07-15 11:41:21 +00:00
practicalswift
e507c0799d tests: Add serialization/deserialization fuzzing helpers WriteToStream(…)/ReadFromStream(…) 2020-07-15 11:41:21 +00:00
practicalswift
e48094a506 tests: Add FuzzedAutoFileProvider which provides a CAutoFile interface to FuzzedDataProvider 2020-07-15 11:41:21 +00:00
practicalswift
9dbcd6854c tests: Add FuzzedFileProvider which provides a FILE* interface to FuzzedDataProvider using fopencookie 2020-07-15 11:41:21 +00:00
fanquake
6cef3652d1
build: fix -Wformat-security check when compiling with GCC
Debian GCC ignores -Wformat-security, without -Wformat, which
means when we test for it, it currently fails:

```bash
checking whether C++ compiler accepts -Wformat-security... no
...
configure:15907: checking whether C++ compiler accepts -Wformat-security
configure:15926: g++ -std=c++11 -c -g -O2 -Werror -Wformat-security  conftest.cpp >&5
cc1plus: error: '-Wformat-security' ignored without '-Wformat' [-Werror=format-security]
cc1plus: all warnings being treated as errors
```

Fix this by just combining the -Wformat and -Wformat-security checks
together.
2020-07-15 17:50:01 +08:00
MarcoFalke
3d57015aa2
Merge #19491: util: Make Assert work with any value
fa53635381 util: Make Assert work with any value (MarcoFalke)

Pull request description:

  Goal is to avoid compile failures

ACKs for top commit:
  jonatack:
    ACK fa53635381
  ryanofsky:
    Code review ACK fa53635381. Looks like if argument is an lvalue this effectively does:

Tree-SHA512: a5cf47a8bb2fa1bd8b8895774f33de50ad803165d6f7b520351be1cfcd5612d5d97c51d118461331d30640186c470879e5ad19e3333e09e72685c5e4e4f23079
2020-07-15 09:03:41 +02:00
fanquake
032cfdc9a2
Merge #19517: psbt: Increment input value sum only once per UTXO in decodepsbt
75122780e2 Increment input value sum only once per UTXO in decodepsbt (Andrew Chow)

Pull request description:

  Refactors the UTXO processing of `decodepsbt` to extract the relevant `CTxOut` and handle the input amounts from that. This avoids double counting the input value.

  Fixes #19516

ACKs for top commit:
  sipa:
    utACK 75122780e2
  ryanofsky:
    Code review ACK 75122780e2

Tree-SHA512: 004ec1597790a88a98098f1a26534d10ab0130a438dec0913522a529a8d7f18ad679948617dbcad6e541fbab5bcb2682aeed386b67746807c03b64d76ce5441d
2020-07-15 12:01:06 +08:00
Andrew Chow
75122780e2 Increment input value sum only once per UTXO in decodepsbt 2020-07-14 12:20:06 -04:00
MarcoFalke
f4de89edfa
Merge #19429: test: Fix intermittent failure in wallet_encryption
fabd33b541 test: Fix intermittent failure in wallet_encryption (MarcoFalke)

Pull request description:

  Iterating all crypted keys might take time.

  E.g.

  ```
   node0 2020-07-01T14:41:19.227367Z [httpworker.0] ThreadRPCServer method=walletpassphrase user=__cookie__
   node0 2020-07-01T14:41:24.377142Z [httpworker.0] queue run of timer lockwallet() in 100000000 seconds (using HTTP)
  ...
   test  2020-07-01T14:41:24.379000Z TestFramework (ERROR): Assertion failed
                                     Traceback (most recent call last):
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 117, in main
                                         self.run_test()
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/wallet_encryption.py", line 88, in run_test
                                         assert_greater_than(expected_time + 5, actual_time) # 5 second buffer
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/util.py", line 54, in assert_greater_than
                                         raise AssertionError("%s <= %s" % (str(thing1), str(thing2)))
                                     AssertionError: 1693614483 <= 1693614484
  ```

  https://cirrus-ci.com/task/5322429885054976?command=ci#L4517

ACKs for top commit:
  achow101:
    ACK fabd33b541

Tree-SHA512: 7a3ccdfc0cdc05fef1f942d3167d100ed63422eb54c05405c884ed91162b7bdb5ce54cb5a981b99a6df2e4af1ea834ccd7d5156531c8c14ea13e735becd6b377
2020-07-14 17:08:19 +02:00
MarcoFalke
834ac4c0f5
Merge #19323: gui: Fix regression in *txoutset* in GUI console
314b49bd50 gui: Fix regression in GUI console (Hennadii Stepanov)

Pull request description:

  The regression was introduced in #19056: if the GUI is running without `-server=1`, the `*txoutset*` call in the console returns "Shutting down".

  Fix #19255.

ACKs for top commit:
  ryanofsky:
    Code review ACK 314b49bd50. Only change since last review is rebase

Tree-SHA512: 8ff85641a5c249858fecb1ab69c7a1b2850af651ff2a94aa41ce352b5b5bc95bc45c41e1767e871b51e647612d09e4d54ede3e20c313488afef5678826c51b62
2020-07-14 16:33:01 +02:00
MarcoFalke
a924f61cae
Merge #19325: wallet: Refactor BerkeleyDatabase to introduce DatabaseBatch abstract class
b82f0ca4d5 walletdb: Add MakeBatch function to BerkeleyDatabase and use it (Andrew Chow)
eac9200814 walletdb: Refactor DatabaseBatch abstract class from BerkeleyBatch (Andrew Chow)

Pull request description:

  In order to support alternative database systems, we need to have a generic `Batch` class. This PR adds a `DatabaseBatch` abstract class which is implemented by `BerkeleyBatch`. `DatabaseBatch` is now the class that is used by `WalletBatch` to interact with the database. To be able to get the correct type of `DatabaseBatch`, `BerkeleyDatabase` now has a `MakeBatch` function which returns a newly constructed `std::unique_ptr<DatabaseBatch>`. For `BerkeleyDatabase`, that will be `std::unique_ptr<BerkeleyBatch>`.

  The `Read`, `Write`, `Erase`, and `Exists` template functions are moved from `BerkeleyBatch`.

  Part of #18971

  Requires #19308 and #19324

ACKs for top commit:
  Sjors:
    re-utACK b82f0ca4d5
  MarcoFalke:
    ACK b82f0ca4d5 🌘
  meshcollider:
    LGTM, utACK b82f0ca4d5

Tree-SHA512: 6d2d41631c0983391dbecd702e881c6775b155c90b275df97f7157e42608ed251744f9d7ce5173d02a6c5cc38d90b611880fac7fa635d3d8c4d590681f56ac6a
2020-07-14 16:21:01 +02:00
MarcoFalke
b26d62c49a
Merge #18990: log: Properly log txs rejected from mempool
fa9f20b647 log: Properly log txs rejected from mempool (MarcoFalke)

Pull request description:

  Currently `CheckTxInputs` rejections from the mempool are the only rejections that log directly and unconditionally to debug.log instead of leaving it to the caller. This has multiple issues:

  * A rejected RPC transaction will log a redundant failure reason to debug log. All other failures are merely reported to the RPC user.
  * A rejected p2p transaction will log the failure twice. Once with the `MEMPOOLREJ` flag, and once unconditionally.
  * A rejected orphan transaction will log no failure.

  Fix all issues by simply returning the state to the caller, like it is done for all other rejections.

  The patch includes whitespace fixups to highlight relevant parts of the codebase and simplify review.

ACKs for top commit:
  naumenkogs:
    utACK fa9f20b647
  rajarshimaitra:
    Concept ACK. Compiled and ran tests. `fa9f20b`
  jnewbery:
    code review ACK fa9f20b647

Tree-SHA512: 86cc17b2a9239c01c4fc3f254ad48ee1d3883266966b9811030176338b9ac3deaea7ea5babfb8bbf739d7440154e30011fede8f9313175f199d4a062af6494f7
2020-07-14 16:15:07 +02:00