9ff41f6419 tests: Add float to FUZZERS_MISSING_CORPORA (temporarily) (practicalswift)
8f6fb0a85a tests: Add serialization/deserialization fuzzing for integral types (practicalswift)
3c82b92d2e tests: Add fuzzing harness for functions taking floating-point types as input (practicalswift)
c2bd588860 Add missing includes (practicalswift)
Pull request description:
Add simple fuzzing harness for functions with floating-point parameters (such as `ser_double_to_uint64(double)`, etc.).
Add serialization/deserialization fuzzing for integral types.
Add missing includes.
To test this PR:
```
$ make distclean
$ ./autogen.sh
$ CC=clang CXX=clang++ ./configure --enable-fuzz \
--with-sanitizers=address,fuzzer,undefined
$ make
$ src/test/fuzz/float
…
```
Top commit has no ACKs.
Tree-SHA512: 9b5a0c4838ad18d715c7398e557d2a6d0fcc03aa842f76d7a8ed716170a28f17f249eaede4256998aa3417afe2935e0ffdfaa883727d71ae2d2d18a41ced24b5
7e9c7113af compressor: Make the domain of CompressAmount(...) explicit (practicalswift)
4a7fd7a712 tests: Add amount compression/decompression fuzzing to existing fuzzing harness: test compression round-trip (practicalswift)
Pull request description:
Small fuzzing improvement:
Add amount compression/decompression fuzzing to existing fuzzing harness: test compression round-trip (`DecompressAmount(CompressAmount(…))`).
Make the domain of `CompressAmount(…)` explicit.
Amount compression primer:
```
Compact serialization for amounts
Special serializer/deserializer for amount values. It is optimized for
values which have few non-zero digits in decimal representation. Most
amounts currently in the txout set take only 1 or 2 bytes to
represent.
```
**How to test this PR**
```
$ make distclean
$ ./autogen.sh
$ CC=clang CXX=clang++ ./configure --enable-fuzz \
--with-sanitizers=address,fuzzer,undefined
$ make
$ src/test/fuzz/integer
…
```
Top commit has no ACKs.
Tree-SHA512: 0f7c05b97012ccd5cd05a96c209e6b4d7d2fa73138bac9615cf531baa3f614f9003e29a198015bcc083af9f5bdc752bb52615b82c5df3c519b1a064bd4fc6664
470e2ac602 tests: Avoid hitting some known minor tinyformat issues when fuzzing strprintf(...) (practicalswift)
Pull request description:
Avoid hitting some known minor tinyformat issues when fuzzing `strprintf(...)`. These can be removed when the issues have been resolved upstreams :)
Note to reviewers: The `%c` and `%*` issues are also present for `%<some junk>c` and `%<some junk>*`. That is why simply matching on `"%c"` or `"%*"` is not enough. Note that the intentionally trivial skipping logic overshoots somewhat (`c[…]%` is filtered in addition to `%[…]c`).
Top commit has no ACKs.
Tree-SHA512: 2b002981e8b3f2ee021c3013f1260654ac7e158699313849c9e9660462bb8cd521544935799bb8daa74925959dc04d63440e647495e0b008cfe1b8a8b2202d40
353f376277 Convert blockencodings.h to new serialization framework (Pieter Wuille)
e574fff53e Add CustomUintFormatter (Pieter Wuille)
10633398f2 Add DifferenceFormatter (Russell Yanofsky)
56dd9f04c7 Make VectorFormatter support stateful formatters (Russell Yanofsky)
3ca574cef0 Convert CCompactSize to proper formatter (Pieter Wuille)
Pull request description:
This is probably the most involved change in the sequence of changes extracted from #10785.
In order to implement the differential encoding of BIP152, this change changes `VectorFormatter` to permit a stateful sub-formatter, which is then used by `DifferenceFormatter`. A `CustomUintFormatter` is added as well to do the 48-bit serialization of short ids.
ACKs for top commit:
laanwj:
ACK 353f376277, nice change
ryanofsky:
Code review ACK 353f376277. Only changes since last review are suggested assert change and MASK->MAX rename
Tree-SHA512: 976618991a8be62ba0738725b7cfa166a56cde998ebf1031ba6f28557032f1577b666ac7ae25cd498c0e1e740108c3c56a342620b724df41d6cc9d8bdafac037
1891245e73 refactor: Cast ping values to double before output (Ben Woosley)
7a810b1d7a refactor: Convert ping wait time from double to int64_t (Ben Woosley)
e6fc63ec7e refactor: Convert min ping time from double to int64_t (Ben Woosley)
b054c46977 refactor: Convert ping time from double to int64_t (Ben Woosley)
Pull request description:
Alternative to #18252, see motivation there.
This changes `CNodeStats` to handle ping timestamps as their original incoming usec `int64_t` values until the time they need to be displayed.
ACKs for top commit:
vasild:
ACK 1891245
practicalswift:
ACK 1891245e73 -- patch looks correct
promag:
ACK 1891245e73, added cast to double and also braces.
Tree-SHA512: 7cfcba941d9751b522b8c512c25da493338b444637bd0bb711b152d7d86b431ca0968956be3c844ee9dbfea25edab44a0de2afa44f2c9c0bf5b8df53eba66272
2455aa5d7f [rpc] changed MineBlocksOnDemand to IsMockableChain (Gloria Zhao)
Pull request description:
Change: Update the if statement in `setmocktime` to use `IsMockableChain` chainparams function (aka `m_is_mockable_chain`) instead of `MineBlocksOnDemand`
Rationale: It's a more appropriate check for whether or not chain is in RegTest, as [discussed](https://github.com/bitcoin/bitcoin/pull/18037#discussion_r376509388) in #18037
ACKs for top commit:
MarcoFalke:
ACK 2455aa5d7f🙇
jonatack:
ACK 2455aa5d7f
Tree-SHA512: 1d8c8b7ff0b3c1bcbf5755194969b6664fe05a35003375ad08d18e34bcefd2df4f64d0e60078a10bbef3c8f469a9b9d07db467089b55c14cf532304bc965bffc
2a95c7c956 ci: Check for submodules (Emil Engler)
Pull request description:
See #18019.
The current solution looks like this (I also tested with multiple submodules):
```
These submodules were found, delete them:
355a5a310019659d9bf6818d2fd66fbb214dfed7 curl (curl-7_68_0-108-g355a5a310)
```
The submodule example command was `git submodule add https://github.com/curl/curl.git curl`
ACKs for top commit:
laanwj:
ACK 2a95c7c956
Tree-SHA512: 64bf388123f0a88d12e3e41ff29bc190339377a0615c35dc3f2700bb7773470a8fa426e0ff57188a60ed88bded39f75082ff0b73118651ff403b163422395005
1ba3e1cc21 init: move asmap code earlier in init process (Jon Atack)
5ba829e12e rpc: fix getpeerinfo RPCResult `mapped_as` type (Jon Atack)
c90b9a2399 net: extract conditional to bool CNetAddr::IsHeNet (Jon Atack)
819fb5549b logging: asmap logging and #include fixups (Jon Atack)
dcaf543ba0 test: add functional test for an empty, unparsable asmap (Jon Atack)
b8d0412b21 config: separate the asmap finding and parsing checks (Jon Atack)
81c38a2497 config: enable passing -asmap an absolute file path (Jon Atack)
fbe9b024f0 config: use default value in -asmap config (Jon Atack)
08b992675c test: add feature_asmap functional tests (Jon Atack)
Pull request description:
This PR builds on PR #16702 to add functional tests / sanity checks and user-facing refinements for passing `-asmap` to configure ASN-based IP bucketing in addrman. As per our review discussion in that PR, the idea here is to handle aspects like functional tests and config arg handling that can help the PR be merged while enabling the author to focus on the bucketing itself.
- [x] add feature functional tests to verify node behaviour and debug log output when launching
- `bitcoind` with no `-asmap` arg
- `bitcoind -asmap=RELATIVE_FILENAME` to the unit test data skeleton asmap
- `bitcoind -asmap` with no filename specified using the default asmap file
- `bitcoind -asmap` with no filename specified and a missing default asmap file
- [x] add the ability to pass absolute path filenames to the `-asmap` config arg in addition to datadir-relative path filenames as per https://github.com/bitcoin/bitcoin/pull/16702#discussion_r361300447, and add test coverage
- [x] separate the asmap file finding and parsing checks, which allows adding tests for the case of a found but unparseable or empty asmap
- [x] add test for an empty asmap
- [x] various asmap fixups
- [x] move the asmap init code earlier in the init process to provide immediate feedback when passing an `-asmap` config arg. This speeds up the `feature_asmap` functional test from 60 to 5 seconds! Credit to Wladimir J. van der Laan for the suggestion.
ACKs for top commit:
practicalswift:
ACK 1ba3e1cc21 -- diff looks correct
fanquake:
ACK 1ba3e1cc21
Tree-SHA512: e9094460a597ac5597449acfe631c87b71d3ede6a12c7ae61b26d1161b3eefed8e7e25c4fb0505864cebd89300b7c4cf9378060aa9155441029315df15fa3283
fa6df0de53 test: Bump timeouts to accomodate really slow disks (MarcoFalke)
Pull request description:
Needed these patches locally for some arm machines with slow storage
ACKs for top commit:
practicalswift:
ACK fa6df0de53
fanquake:
ACK fa6df0de53
Tree-SHA512: 22f2f6f7ed05f26013431126bb179b029dbc931f02d0e58f8970c6d477f43e3106d76c9732942034cb2cfcb827191e338a082f953ccb69531a19ee6dab9a7e1a
fa8b6020ec doc: Merge release notes for 0.20.0 release (MarcoFalke)
Pull request description:
mostly move-only. Can be reviewed with the `--color-moved=dimmed-zebra` option.
ACKs for top commit:
laanwj:
ACK fa8b6020ec
fanquake:
ACK fa8b6020ec - any changes are basically headers and newlines.
Tree-SHA512: 7273c7625d60c3b28bafc4371e17545bd9fcaa672fde8492a0b4ab88081d616dd41c77389d18a9a3b39b595c5409a354bf511745ddcb9834c9a25d91b94edb28
and update feature_asmap.py and test_runner.py
This commit moves the asmap init.cpp code from the end of "Step 12: start node"
to "Step 6: network initialization" to provide feedback on passing an -asmap
config arg much more quickly. This change speeds up the feature_asmap.py
functional test file from 60 to 5 seconds by accelerating the 2 tests that use
`assert_start_raises_init_error`.
Credit to Wladimir J. van der Laan for the suggestion.
- move asmap #includes to sorted positions in addrman and init (move-only)
- remove redundant quotes in asmap InitError, update test
- remove full stops from asmap logging to be consistent with debug logging,
update tests
to verify node behaviour and debug log when launching bitcoind in these cases:
1. `bitcoind` with no -asmap arg, using /16 prefix for IP bucketing
2. `bitcoind -asmap=<relative path>`, using the unit test skeleton asmap
3. `bitcoind -asmap/-asmap=` with no file specified, using the default asmap
4. `bitcoind -asmap` with no file specified, and a missing default asmap file
The tests are order-independent. The slowest test (missing default asmap file)
is placed last.
fa6b061fc1 rpc: Auto-format RPCResult (MarcoFalke)
fa7d0503d3 rpc: Move OuterType enum to header (MarcoFalke)
Pull request description:
This enforces most syntax rules of the RPCResult at compile time (or some at run time during unit and functional tests)
Apart from normalizing the syntax, by separating stylistic formatting from the structure, we could in theory directly generate the html for e.g. https://bitcoincore.org/en/doc/0.19.0/rpc/wallet/importmulti/
ACKs for top commit:
Sjors:
Indeed, re-ACK fa6b061fc1
ajtowns:
ACK fa6b061fc1 -- skimmed code changes and differences to rpc help output
Tree-SHA512: 5b510b3aa0b7c7b9189a48c77593159409069f939145b9a00c5478e894cf65f994d44d633eb7bb7dbea40ee820645a2930976c24772379d96929002b120efa28
9b0e16226e doc: Correct spelling errors in comments (Ben Woosley)
Pull request description:
And ci script output.
Identified via test/lint/lint-spelling
Before:
```
$ test/lint/lint-spelling.sh
ci/test/05_before_script.sh:29: explicitely ==> explicitly
src/compressor.h:43: Ser ==> Set
src/compressor.h:78: Ser ==> Set
src/logging/timer.h:88: outputing ==> outputting
src/node/psbt.cpp:87: minumum ==> minimum
src/qt/coincontroldialog.cpp:372: UnSelect ==> deselect
src/qt/coincontroldialog.cpp:443: unselect ==> deselect
src/qt/coincontroldialog.cpp:448: UnSelect ==> deselect
src/qt/coincontroldialog.cpp:699: UnSelect ==> deselect
src/serialize.h:211: Ser ==> Set
src/serialize.h:213: Ser ==> Set
src/serialize.h:228: Ser ==> Set
src/serialize.h:246: Ser ==> Set
src/serialize.h:484: Ser ==> Set
src/serialize.h:490: Ser ==> Set
src/serialize.h:510: Ser ==> Set
src/serialize.h:622: Ser ==> Set
src/serialize.h:740: Ser ==> Set
src/test/base32_tests.cpp:14: fo ==> of, for
src/test/base64_tests.cpp:14: fo ==> of, for
src/txmempool.h:756: incomaptible ==> incompatible
src/undo.h:26: Ser ==> Set
src/wallet/coincontrol.h:74: UnSelect ==> deselect
test/functional/feature_backwards_compatibility.py:116: Abondon ==> Abandon
test/functional/rpc_getaddressinfo_label_deprecation.py:7: superceded ==> superseded
test/lint/lint-shell.sh:44: desriptor ==> descriptor
^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt
```
After:
```
$ test/lint/lint-spelling.sh
src/test/base32_tests.cpp:14: fo ==> of, for
src/test/base64_tests.cpp:14: fo ==> of, for
test/functional/rpc_getaddressinfo_label_deprecation.py:7: superceded ==> superseded
^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt
```
ACKs for top commit:
practicalswift:
ACK 9b0e16226e
MarcoFalke:
ACK 9b0e16226e
Tree-SHA512: 9ce203700b11596e4b920b3c5b04f59bc7784fe5b495868d43423608180a9a553ec7efcc5ad70384f3ce462b036c2a682260efebce493c5e6a3d48716b268179
aff2748f8a httpserver: use own HTTP status codes (Filip Gospodinov)
Pull request description:
Before, macros defined in `<event2/http.h>` have been used for some HTTP status codes. `<event2/http.h>` is included implicitly and the usage of its status code macros is inconsistent with the majority HTTP response implementations in this file.
Now, the `HTTPStatusCode` enum from `<rpc/protocol.h>` is consistently used for all HTTP response implementations.
ACKs for top commit:
practicalswift:
ACK aff2748f8a -- patch looks correct
laanwj:
ACK aff2748f8a
Tree-SHA512: 6a7043488b88dcd584215d16b5f16f7bd668fe5553d31298d1beef134d2b0648aef81533014e34d1cd600baa36ee4e853f195352f4a00c866bd5ab1ff688bd50
1ef28b4f7c Make AnalyzePSBT next role calculation simple, correct (Gregory Sanders)
Pull request description:
Sniped test and alternative to https://github.com/bitcoin/bitcoin/pull/18220
Sjors documenting the issue:
```
A PSBT signed by ColdCard was analyzed as follows (see #17509 (comment))
{
"inputs": [
{
"has_utxo": true,
"is_final": false,
"next": "finalizer"
}
],
"estimated_vsize": 141,
"estimated_feerate": 1e-05,
"fee": 1.41e-06,
"next": "signer"
}
I changed AnalyzePSBT so that it returns "next": "finalizer" instead.
```
It makes it much clearer that the role has been decided before hitting the `calc_fee` block, and groups all state-deciding in one spot instead of 2.
Note that this assumes that PSBT roles are a complete ordering, which for now and in the future seems to be a correct assumption.
ACKs for top commit:
Sjors:
ACK 1ef28b4f7c, much nicer. Don't forget to document the bug fix.
achow101:
ACK 1ef28b4f7c
Empact:
ACK 1ef28b4f7c
Tree-SHA512: 22ba4234985c6f9c1445b14565c71268cfaa121c4ef000ee3d5117212b09442dee8d46d9701bceddaf355263fe25dfe40def2ef614d4f2fe66c9ce876cb49934
* GetAvoidReuseFlag: simply gets the flag, without modifying the wallet
* ListReceived: helper function to produce lists
* ListTransactions: produces a list of transactions, without modifications; two cases of map [] -> .at() for verified-existing keys
* DescribeWalletAddress: generates a description of a given wallet address without changing the wallet
* The following functions produce a list without making any modifications to the wallet:
* listaddressgroupings
* listreceivedbyaddress
* listreceivedbylabel
* listtransactions
* listsinceblock
* listlockunspent
* listunspent
* listlabels
* getreceivedbyaddress
* getreceivedbylabel
* getaddressesbylabel
* signmessage: uses the wallet to procure a private key for signing, but does no modifications
* getbalance, getunconfirmedbalance: calculates the wallet balance, without any modifications
* gettransaction: procures transaction without any modifications
* backupwallet: makes a backup of the wallet to disk, without changing said wallet
* getwalletinfo: produces info about wallet without any modifications
* signrawtransactionwithwallet: modifies incoming transaction on the fly by signing with private key procured from within wallet; no modifications to wallet
* getaddressinfo: gets information about the given address, with no modifications done to the wallet; one case of [] -> .at() and one ::iterator -> ::const_iterator
* walletprocesspsbt: processes the given PSBT on the fly, without modifying the wallet
* GetWalletAddressesForKey is, as the name implies, immutable; the one change besides the parameter constness is a [] -> .at() change, to a verified-existing key.
* dumpprivkey and dumpwallet are both similarly immutable, for obvious reasons.
CWallet::CanGetAddresses() is used to check whether the wallet has available or is able to produce keys for addresses. It uses the ScriptPubKeyMan::CanGetAddresses(), which in turn uses the const KeypoolCountExternalKeys() method, all which do counting and no modifications.
