f2b5b0a3b4 build: add linker optimization flags to guix (fanquake)
b8b050a8d6 build: add linker optimization flags to gitian descriptors (fanquake)
Pull request description:
This PR adds `-Wl,O2` to our gitian and guix LDFLAGS. This makes the linker perform certain optimisations (and is different from LTO).
Any -O argument will enable optimizations in GNU ld. We can use -O2 here, as this matches our compile flags. Note that this would also enable additional optimizations if using the lld or gold linkers, when compared to -O0.
A nice writeup + diagrams of some of these optimizations is available here: http://lwn.net/Articles/192624/.
#### master
```bash
# bitcoind
Histogram for `.gnu.hash' bucket list length (total of 3 buckets)
Length Number % of total Coverage
0 1 ( 33.3%) 0.0%
1 0 ( 0.0%) 0.0%
2 1 ( 33.3%) 40.0%
3 1 ( 33.3%) 100.0%
```
```bash
# bitcoin-qt
Histogram for `.gnu.hash' bucket list length (total of 3 buckets)
Length Number % of total Coverage
0 0 ( 0.0%) 0.0%
1 1 ( 33.3%) 10.0%
2 0 ( 0.0%) 10.0%
3 0 ( 0.0%) 10.0%
4 1 ( 33.3%) 50.0%
5 1 ( 33.3%) 100.0%
```
#### this PR:
```bash
# bitcoind
Histogram for `.gnu.hash' bucket list length (total of 8 buckets)
Length Number % of total Coverage
0 3 ( 37.5%) 0.0%
1 5 ( 62.5%) 100.0%
```
```bash
# bitcoin-qt
Histogram for `.gnu.hash' bucket list length (total of 19 buckets)
Length Number % of total Coverage
0 9 ( 47.4%) 0.0%
1 10 ( 52.6%) 100.0%
```
#### GNU ld -O
> If level is a numeric values greater than zero ld optimizes the output. This might take significantly longer and therefore probably should only be enabled for the final binary. At the moment this option only affects ELF shared library generation. Future releases of the linker may make more use of this option. Also currently there is no difference in the linker’s behaviour for different non-zero values of this option. Again this may change with future releases.
#### lld -O
> Optimize output file size
ACKs for top commit:
dongcarl:
ACK f2b5b0a3b4
laanwj:
ACK f2b5b0a3b4
Tree-SHA512: e53f3a4338317dbec65d3a93b57b5a6204aabdf9ac82d99447847a3c8627facc53c58c2cf947376f13edd979fc8129a80f18d9ebeccd191a576c83f1dad5c513
fa6cb00ce5 doc: Fix macos comments in release-notes (MarcoFalke)
Pull request description:
ACKs for top commit:
hebasto:
ACK fa6cb00ce5.
fanquake:
ACK fa6cb00ce5 - this has also been done in the release notes in the wiki.
Tree-SHA512: ce325a9a32e28923ad34bdc7c1c69e9d14636a18d05604158e99f80683686bd2ba5550e84f85479336658b84c40dbe2940840e47d69c10ae3d41180dad49c8f3
fa50d11dd1 appveyor: Disable functional tests for now (MarcoFalke)
Pull request description:
ACKs for top commit:
jonasschnelli:
ACK fa50d11dd1
Tree-SHA512: 86ea9bc0ff47dc309120748627cd83b19aa61fb1db3cca5e04a9e0d56f34c248cd2bec20b037c663ab0f21f7ee6cd0ff8266a36fe79c773de95e5032eb48b118
check the following expected behaviors if no filter is set:
-> filtered block requests are ignored by the node
-> sending a 'filteradd' message is treated as misbehavior
(i.e. the peer's banscore increases by 100)
also fixes a bug in the on_inv() callback method, which
directly modified the type from BLOCK to FILTERED_BLOCK
in the received 'inv' message rather than just for the reply
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
When a wallet uses avoid_reuse and has a large number of outputs in
a single destination, it groups these outputs in OutputGroups that
are no larger than OUTPUT_GROUP_MAX_ENTRIES. The goal is to spend
as many outputs as possible from the destination while not breaking
consensus due to a huge number of inputs and also not surprise the
use with high fees. If there are n outputs in a destination and
n > OUTPUT_GROUP_MAX_ENTRIES then this results in one or many groups
of size OUTPUT_GROUP_MAX_ENTRIES and possibly one group of size
< OUTPUT_GROUP_MAX_ENTRIES.
Prior to this commit the coin selection in the case where
n > OUTPUT_GROUP_MAX_ENTRIES was skewed towards the one group of
size < OUTPUT_GROUP_MAX_ENTRIES if it exists and the amount to be
spent by the transaction is smaller than the aggregate of those
of the group size < OUTPUT_GROUP_MAX_ENTRIES. The reason is that
the coin selection decides between the different groups based on
fees and mostly the smaller group will cause smaller fees.
The behavior that users of the avoid_reuse flag seek is that the
full groups of size OUTPUT_GROUP_MAX_ENTRIES get used first. This
commit implements this by pretending that the small group has
a large number of ancestors (one smallet than the maximum allowed
for this wallet). This dumps the small group to the bottom of the
list of priorities in the coin selection algorithm.
48973402d8 wallet: Avoid use of Chain::Lock in CWallet::GetKeyBirthTimes (Russell Yanofsky)
e958ff9ab5 wallet: Avoid use of Chain::Lock in CWallet::CreateTransaction (Russell Yanofsky)
c0d07dc4cb wallet: Avoid use of Chain::Lock in CWallet::ScanForWalletTransactions (Russell Yanofsky)
1be8ff280c wallet: Avoid use of Chain::Lock in rescanblockchain (Russell Yanofsky)
3cb85ac594 wallet refactor: Avoid use of Chain::Lock in CWallet::RescanFromTime (Russell Yanofsky)
f7ba881bc6 wallet: Avoid use of Chain::Lock in listsinceblock (Russell Yanofsky)
bc96a9bfc6 wallet: Avoid use of Chain::Lock in importmulti (Russell Yanofsky)
25a9fcf9e5 wallet: Avoid use of Chain::Lock in importwallet and dumpwallet (Russell Yanofsky)
c1694ce6bb wallet: Avoid use of Chain::Lock in importprunedfunds (Russell Yanofsky)
ade5f87971 wallet refactor: Avoid use of Chain::Lock in qt wallettests (Russell Yanofsky)
f6da44ccce wallet: Avoid use of Chain::Lock in tryGetTxStatus and tryGetBalances (Russell Yanofsky)
bf30cd4922 refactor: Add interfaces::FoundBlock class to selectively return block data (Russell Yanofsky)
Pull request description:
This is a set of changes updating wallet code to make fewer calls to `Chain::Lock` methods, so the `Chain::Lock` class will be easier to remove in #16426 with fewer code changes and small changes to behavior.
ACKs for top commit:
MarcoFalke:
re-ACK 48973402d8, only change is fixing bug 📀
fjahr:
re-ACK 48973402d8, reviewed rebase and changes since last review, built and ran tests locally
ariard:
Coce Review ACK 4897340, only changes are one suggested by last review on more accurate variable naming, human-readable output, args comments in `findCommonAncestor`
Tree-SHA512: cfd2f559f976b6faaa032794c40c9659191d5597b013abcb6c7968d36b2abb2b14d4e596f8ed8b9a077e96522365261299a241a939b3111eaf729ba0c3ef519b
0306d78cb4 Use getbalances in wallet_address_types tests (Jon Atack)
7eacdc5167 Shift coverage from getunconfirmedbalance to getbalances in wallet_abandonconflict tests (Jon Atack)
3e6f7377f6 Improve getbalances coverage in wallet_balance tests (Jon Atack)
Pull request description:
<strike>This PR updates several tests and then removes the `getunconfirmedbalance` RPC which was deprecated in facfb4111d a year ago.
Next steps: remove the deprecated `getwalletinfo` fields and the `getbalance` RPC in follow-ups, if there seems to be consensus on those removals.</strike>
Update:
`getunconfirmedbalance` RPC was deprecated in facfb4111d a year ago, but following the review comments below, this PR now only updates the test coverage to use `getbalances` while still leaving basic coverage for `getunconfirmedbalance` in wallet_balance.py.
That said, I've seen 3 regular contributors confused in the past 10 days by "DEPRECATED" warnings in the code that are not following the deprecation policy in [JSON-RPC-interface.md#versioning](https://github.com/bitcoin/bitcoin/blob/master/doc/JSON-RPC-interface.md#versioning).
ISTM these warnings should either be removed, or the calls deprecated (`-deprecatedrpc`), or the policy updated to describe these warnings as a pre-deprecation practice.
ACKs for top commit:
jnewbery:
utACK 0306d78cb
Tree-SHA512: 692e43e9bed5afa97d905740666e365f0b64e559e1c75a6a398236d9e943894e3477947fc11324f420a6feaffa0c0c1532aa983c50090ca39d06551399e6ddd1
0753efd9dc rpc: Remove deprecated "size" from mempool txs (Vasil Dimov)
Pull request description:
Remove the "size" property of a mempool transaction from RPC replies.
Deprecated in e16b6a718 in 0.19, about 1 year ago.
ACKs for top commit:
kristapsk:
ACK 0753efd9dc
Tree-SHA512: 392ced6764dd6a1d47c6d1dc9de78990cf3384910d801253f8f620bd1751b2676a6b52bee8a30835d28e845d84bfb37431c1d2370f48c9d4dc8e6a48a5ae8b9e
9cdddae3b4 test: add rpc_signrawtransaction logging (Jon Atack)
4d6cde38ce test: refactor rpc_signrawtransaction witness script tests (Jon Atack)
Pull request description:
As a follow-up to #18484, the new tests are good but bury the one non-duplicate line in each test that sets the witness script, and there is no logging in the testfile. This PR makes it easy to see what is unique to each of the new tests and adds logging.
ACKs for top commit:
theStack:
ACK 9cdddae3b4🥚🐰
Tree-SHA512: 7b1ca303326658afb90b7635abc9fe8bb65f0be004124d4dcf38702bb6f38bc06ce33c0642be4ad5d511453d003cdefeea691e66e3b963a4feb66f6237a3c241
fab9899204 test: Try once more when RPC connection fails on Windows (MarcoFalke)
faa655731e test: Document why connection is re-constructed on windows (MarcoFalke)
fa9f4f663c test: Remove python 3.4 workaround (MarcoFalke)
fae760f2b2 cirrus: Bump freebsd to 12.1 (MarcoFalke)
Pull request description:
Fixes: #18548
ACKs for top commit:
hebasto:
ACK fab9899204, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: c4e9ed8d995b63a820ca66984f152ac216c83ba1f318b61b15c6d375c0e936c08f6bc3d38c255dddf3ee8952f848c7ababf684854e07a7c1b1d8504e6b7208ba
c0af173da2 doc: default minconf for getbalance should be 0 (U-Zyn Chua)
Pull request description:
- Default `minconf` for `getbalance` is `0` but example in doc was showing as `1`.
- `at least 6 blocks confirmed` now updated to be `at least 6 confirmations` to be more consistent with the terminology used elsewhere in the codebase and documentations.
ACKs for top commit:
theStack:
re-ACK c0af173da2
Tree-SHA512: 8f67af78a222a4bd2957658b37fae2224783274f355af84f39a5ce0da90b21f03dc798a6408d44a724c353ff5ed7dfec943fb28726ec423028b64fc579f937ad
555567ace9 test: Extend wallet_dump test to cover comments (MarcoFalke)
Pull request description:
ACKs for top commit:
ryanofsky:
Code review ACK 555567ace9. Nice new checks in this test. I confirmed this catches the missing FormatISO8601DateTime call you discovered in https://github.com/bitcoin/bitcoin/pull/17954#discussion_r406891999
Tree-SHA512: 71aa23dd039f3bcdee642b01151edd1a0d44f48cedd070f5858148c8cb8abd6f5edfd212daeba38e35c843da5ea6c799e5a952105fdecedac355a5a843c05a84
fa404f1e47 test: Check that the version message does not leak the local address of the node (MarcoFalke)
Pull request description:
Add test for #8740
ACKs for top commit:
theStack:
ACK fa404f1e47
Tree-SHA512: 4d1c10d1c02fba4b51bd8b9eb3a0d9a682b6aac8c3f6924e295fdca3faefa5ecc3eaa87d347cfec5d2b2bc49963c10fe0a37c463f36088ed0304a2e3716b963b
854382885f refactor: test: improve wait_for{header,merkleblock} interface (Sebastian Falbesoner)
1356a45ef0 test: complete impl. of msg_merkleblock and wait_for_merkleblock (Sebastian Falbesoner)
Pull request description:
Implements the missing initialization/serialization methods for `msg_merkleblock`, based on the already present class `CMerkleBlock`. Also changes the method `wait_for_merkleblock()` to be more precise by waiting for a merkleblock with a specified blockhash instead of an arbitrary one.
In the BIP37 test `p2p_filter.py`, this new method is used to make the test of receiving merkleblock and tx if a filter is set to be more precise, by checking if they also arrive in the right order.
In the course of this PR, also the interface for the methods `wait_for_merkleblock()` and `wait_for_header()` are improved to take a hex string instead of an integer, which is more typesafe and less of a burden to the caller.
ACKs for top commit:
MarcoFalke:
ACK 854382885f
Tree-SHA512: adaf0ac728ef0b9929cb417a7a7b4c1346c400b2d365bf6914515c67b6cfe8f4a7ecc62fb514afdce9792f0bed833416f6bca6b9620f3d5dcdc66e4d5b0b7ea3
2599d13c94 rpc: Remove deprecated migration code (Vasil Dimov)
Pull request description:
Don't accept a second argument to `sendrawtransaction` and
`testmempoolaccept` of type `bool`. Actually even the code before this
change would not accept `bool`, but it would print a long explanatory
message when rejecting it: "Second argument must be numeric (maxfeerate)
and no longer supports a boolean. To allow a transaction with high fees,
set maxfeerate to 0."
This was scheduled for removal in 6c0a6f73e.
ACKs for top commit:
MarcoFalke:
ACK 2599d13c94📅
Tree-SHA512: e2c74c0bde88e20149d0deab0845851bb3979143530a6bae4f46769d61b607ad2e2347f8969093c2461a80c47661732dc0b3def140f8ce84081719adda3b3811
Any -O argument will enable optimizations in GNU ld. We can use -O2
here, as this matches our compile flags. Note that this would also
enable additional optimizations if using the lld or gold linkers,
when compared to -O0.
Any -O argument will enable optimizations in GNU ld. We can use -O2
here, as this matches our compile flags. Note that this would also
enable additional optimizations if using the lld or gold linkers,
when compared to -O0.
a35e323589 guix: Appease travis. (Carl Dong)
0b66d22da5 guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong)
ba0b99bdd6 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong)
93439a71ed guix: Bump to upstream commit with mingw-w64 changes (Carl Dong)
35a96792dd guix: Check mingw symbols, improve SSP fix docs (Carl Dong)
449d8fe25b guix: Expand on INT trap message (Carl Dong)
3f1f03c67a guix: Spelling fixes (Carl Dong)
ff821dd2a1 guix: Reinstate make-ssp-fixed-gcc (Carl Dong)
360a9e0ad5 guix: Bump time-machine for mingw-w64 patches (Carl Dong)
93e41b7e3b guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong)
ef4f7e4c45 guix: Set the well-known timezone env var (Carl Dong)
acf4b3b3b5 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong)
c4cce00eac guix: Remove dead links from README. (Carl Dong)
df953a4c9a guix: Appease shellcheck. (Carl Dong)
91897c95e1 guix: Improve guix-build.sh documentation (Carl Dong)
570d769c6c guix: Build support for Windows (Carl Dong)
Pull request description:
~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~
Based on: #17933 (Time Machines are... shall we say... superior 😁)
This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon)
ACKs for top commit:
fanquake:
ACK a35e323589 2/3
Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
Add a default constructor to `PrecomputedTransactionData`, which doesn't
initialize the struct's members. Instead they're initialized inside the
`CheckInputScripts()` function. This allows a later commit to add the
spent UTXOs to that structure.
The interfaces for the methods wait_for_header() and wait_for_merkleblock() are
changed to take a hex string instead of an integer, improving type safety and
removing the burden from the caller to always do the transformation via
`int(...)`. As suggested by MarcoFalke in
https://github.com/bitcoin/bitcoin/pull/18593#discussion_r407062253
