Commit graph

1284 commits

Author SHA1 Message Date
Hennadii Stepanov fac86ac7b3
scripted-diff: Add missed copyright headers
-BEGIN VERIFY SCRIPT-
s() { contrib/devtools/copyright_header.py insert "$1"; }

s build_msvc/bitcoin_config.h
s build_msvc/msvc-autogen.py
s build_msvc/testconsensus/testconsensus.cpp
s contrib/devtools/circular-dependencies.py
s contrib/devtools/gen-manpages.sh
s contrib/filter-lcov.py
s contrib/gitian-build.py
s contrib/install_db4.sh
s src/crypto/sha256_avx2.cpp
s src/crypto/sha256_sse41.cpp
s src/fs.cpp
s src/qt/test/addressbooktests.cpp
s src/qt/test/addressbooktests.h
s src/qt/test/util.cpp
s src/qt/test/util.h
s src/qt/test/wallettests.cpp
s src/qt/test/wallettests.h
s src/test/blockchain_tests.cpp
s test/functional/combine_logs.py
s test/lint/lint-locale-dependence.sh
sed -i '1G' test/lint/lint-shebang.sh
s test/lint/lint-shebang.sh
-END VERIFY SCRIPT-
2020-01-04 20:18:28 +02:00
fanquake c491368d8c
scripts: add MACHO dylib checking to symbol-check.py 2020-01-04 11:24:42 +08:00
Hennadii Stepanov 6fde9d5e47
script: Update EXLUDE list in copyright_header.py 2020-01-03 17:48:37 +02:00
Hennadii Stepanov 1998152f15
script: Add empty line after C++ copyright 2020-01-03 17:48:37 +02:00
Hennadii Stepanov 071f2fc204
script: Add ability to insert copyright to *.sh 2020-01-03 17:47:54 +02:00
fanquake 7c9e821c4e
scripts: add MACHO NOUNDEFS check to security-check.py 2020-01-02 14:42:23 +08:00
fanquake 4ca92dc6d3
scripts: add MACHO PIE check to security-check.py 2020-01-02 14:42:21 +08:00
MarcoFalke aaaaad6ac9
scripted-diff: Bump copyright of files changed in 2019
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
2019-12-30 10:42:20 +13:00
Emil Engler 8dc9aa90c3
doc: Update license year range to 2020 2019-12-26 23:11:21 +01:00
MarcoFalke facb416ad5
ci: Add valgrind run 2019-12-10 19:37:37 -05:00
Wladimir J. van der Laan ac09180128
Merge #17680: gitian: fixed SC2001 regex
c966ff14c7 gitian: fixed SC2001 regex (willyk)

Pull request description:

  Currently the gitian-win-signer.yml produces OUTFILE names without `-unsigned` stripped out
  This is due to regex having an`%` in front of it
  ```
  $ INFILE="bitcoin-0.19.0-win64-setup-unsigned.exe"
  $ echo "${INFILE/%-unsigned}"
  bitcoin-0.19.0-win64-setup-unsigned.exe
  $ echo "${INFILE/-unsigned}"
  bitcoin-0.19.0-win64-setup.exe
  ```

  Fixes #17361

ACKs for top commit:
  laanwj:
    ACK c966ff14c7
  hebasto:
    ACK c966ff14c7

Tree-SHA512: 954547f9dfa4cab4def5f284d4837c21f0e6fed7454a04e83e6b1b7d3fd3f9661ea657047f0d8162f6591909d32ef2f72e801b2f3a44cbb1131ac344cb913a69
2019-12-09 16:16:51 +01:00
willyk c966ff14c7
gitian: fixed SC2001 regex
the `-` is not a special symbol and should not have `%` in front of it.
2019-12-05 13:00:34 -08:00
willyk b11d35b5e2 Fixed wget call in gitian-build.py 2019-12-05 02:21:09 -08:00
Emil Engler bd6a243075
script: Add Keyserver to verify-commits README 2019-11-30 04:41:06 +01:00
Wladimir J. van der Laan 1f59885d27
Merge #17361: script: Lint Gitian descriptors with ShellCheck
17f81e9648 script: Enable SC2001 rule for Gitian scripts (Hennadii Stepanov)
61bb21b418 script: Enable SC2155 rule for Gitian scripts (Hennadii Stepanov)
577682d9e8 script: Enable SC2006 rule for Gitian scripts (Hennadii Stepanov)
14aded46df script: Lint Gitian descriptors with ShellCheck (Hennadii Stepanov)

Pull request description:

  This PR extracts shell scripts from Gitian descriptors (`contrib/gitian-descriptors/`) and checks for ShellCheck warnings as any other one.

  Some non-controversial warnings are fixed.

ACKs for top commit:
  practicalswift:
    ACK 17f81e9648 -- diff looks correct

Tree-SHA512: bdfa3d35bbb65ff634c90835d75c3df63e958b558599771d21366724f5cf64da83a68957d926e926a99c3704b9529e96a17697dc8d9ff3adf7154d9cb1999a8d
2019-11-28 10:18:22 +01:00
Hennadii Stepanov 17f81e9648
script: Enable SC2001 rule for Gitian scripts 2019-11-27 19:27:56 +02:00
Hennadii Stepanov 61bb21b418
script: Enable SC2155 rule for Gitian scripts
Also pwd command is replaced with $PWD variable everywhere for
consistency.
2019-11-27 19:25:29 +02:00
Wladimir J. van der Laan eafd259367 build: Add NX workaround for RV64
Work around https://bugs.launchpad.net/ubuntu/+source/gcc-8-cross-ports/+bug/1853740.
2019-11-24 13:40:11 +01:00
Wladimir J. van der Laan f6e42256fe build: Allow export of environ symbols
This export was introduced in #17270 which added
```
//! Necessary on some platforms
extern char** environ;
```
2019-11-23 10:30:41 +01:00
Wladimir J. van der Laan 16a7be1663
build: Bump minimum versions in symbol checker
Debian 8 (Jessie) has:
- g++ version 4.9.2
- libc version 2.19

Ubuntu 16.04.4 (Xenial) has:
- g++ version 5.3.1
- libc version 2.23.0

CentOS 7 has:
- g++ version 4.8.5
- libc version 2.17

Taking the minimum of these as our target.
According to the GNU ABI document this corresponds to:

- GCC 4.8.5: GCC_4.8.0
- (glibc)    GLIBC_2_17

Co-Authored-By: fanquake <fanquake@gmail.com>
2019-11-22 15:40:05 -05:00
Wladimir J. van der Laan b77d5ad59f
build: Disallow dynamic linking against c++ library
Ever since statically linking Qt, we've been linking the C++ library
statically too (-static-libstdc++). Take this into
account in the symbol checker.
2019-11-22 15:38:52 -05:00
fanquake a6f5b6f47d
Merge #17550: build: set minimum supported macOS to 10.12
7d7bf2ff4a build: set minimum supported macOS to 10.12 (fanquake)

Pull request description:

  Extracted from #16392 as this doesn't need to wait for the other build changes.

  Reasoning:
  * `10.10` has been unsupported since July 2017 (~3 years at `v0.20.0` release)
  * `10.11` has been unsupported since July 2018  (~2 years at `v0.20.0` release)
  * macOS users are consistent at upgrading to new releases.
  * Qt 5.12 LTS only supports [macOS > 10.12](https://doc-snapshots.qt.io/qt5-5.12/supported-platforms.html). As long as we're supporting macOS < 10.12 we would not be able to bump Qt in depends to 5.12 for the `v0.20.0` release.

  Once we drop support for 10.12 and start using the 10.15 SDK there are some other follow ups:
  * Enabling support for [`thread_local`](https://github.com/bitcoin/bitcoin/blob/master/configure.ac#L901).
  * Removing some of the macOS notification code
  * Removing macOS [startup item code](https://github.com/bitcoin/bitcoin/blob/master/src/qt/guiutil.cpp#L695).

  There was also some related discussion in the `#bitcoin-builds` channel yesterday arvo.

ACKs for top commit:
  laanwj:
    ACK 7d7bf2ff4a
  dongcarl:
    post-IRC-grilling-ACK 7d7bf2ff4a

Tree-SHA512: 30b0ed75e3b4df2d3f94db2091b7982c1ea7be8fb3a2732d8efd1a53ef1e492d7265a47e90c38e92f66e638d10e9400a6ecb56e5093688bb26d1621645b453e8
2019-11-22 15:35:58 -05:00
fanquake 7d7bf2ff4a
build: set minimum supported macOS to 10.12 2019-11-21 11:08:47 -05:00
fanquake ec89d2882a
build: remove libanl.so.1 from ALLOWED_LIBRARIES 2019-11-20 15:05:35 -05:00
Wladimir J. van der Laan 6fde676f64
Merge #16669: build: use new fork of osslsigncode for windows gitian signing
feb5075777 build: use osslsigncode 2.0 in gitian (fanquake)

Pull request description:

  The original osslsigncode project, https://sourceforge.net/projects/osslsigncode, has been marked as abandonware:
  > This is now - and has been for a long while - abandonware. Feel free to create your own forks etc.".

  However, a fork has emerged, https://github.com/mtrojnar/osslsigncode, that has incorporated
  theuni's patches ([add the -pem option in extract-signature mode ](36715c1183) & [add the attach-signature command](3be7eb1676)) as well as updated the tool to work with OpenSSL 1.1 and other improvements.

  This commit switches the windows signer descriptor to use this new version of `osslsigncode`.

  I've tested using this new version of `osslsigncode` while doing a 0.18.1 gitian build, and it "seems" to work. However this needs a look over from Cory, to check if the tool is still compatible with his usage in the [`detached-sig-create.sh`](https://github.com/bitcoin/bitcoin/blob/master/contrib/windeploy/detached-sig-create.sh) script, as well as some review of the changes to `osslsigncode` itself. Hence WIP and chasing Concept ACKs / NACKs.

ACKs for top commit:
  MarcoFalke:
    Concept ACK feb5075777 given that this upstream is now used in Ubuntu and Debian
  laanwj:
    ACK feb5075777

Tree-SHA512: c48de6dc32751d96dd04b920bfacca40af47a2883330ba0700371d56c580a7e45cedd8d8a913709d56be036762b63cb1825a98cff7aa77b6d7804fab11220850
2019-11-18 14:39:19 +01:00
Wladimir J. van der Laan 94f0749f91
Merge #17436: Add TheCharlatan's pgp key
0ec967164f Add TheCharlatan's pgp key (TheCharlatan)

Pull request description:

  Since I have submitted my pair of signed gitian assertions, I am now adding my key's fingerprint to the gitian keys list.

ACKs for top commit:
  jonasschnelli:
    ACK 0ec967164f - confirm key A8FC55F3B04BA3146F3492E79303B33A305224CB

Tree-SHA512: bbd5e637186ed1659432e4fcc96bdc57fdbdb608325805701e06a51689726e722d7abeb11b5c9de723d051976d9d7ac23602316403fa74029dceb3cf1e837aea
2019-11-18 10:54:49 +01:00
MarcoFalke 1ed3e071df
Merge #17455: tests: Update valgrind suppressions
d604b4cc8c tests: Update valgrind suppressions (practicalswift)

Pull request description:

  Update `valgrind` suppressions.

  To test this PR:

  ```
  $ valgrind --suppressions=contrib/valgrind.supp src/test/test_bitcoin
  $ valgrind --suppressions=contrib/valgrind.supp src/bench/bench_bitcoin -evals=1 \
        -scaling=0.0
  ```

Top commit has no ACKs.

Tree-SHA512: 79cb318b5b9171e74d0bd0b89cc688ad4531b134182b06c2942c46058c19b45723c391b781e8ccd157a14fbf6a14588764c7728c5506c73ae237dde9f44db2f6
2019-11-15 14:43:50 -05:00
MarcoFalke fae75306ba
scripted-diff: Set gitian arch back to amd64
-BEGIN VERIFY SCRIPT-
sed -i -e 's|"linux64"|"amd64"|g' $(git grep -l '"linux64"')
-END VERIFY SCRIPT-
2019-11-14 17:10:18 -05:00
practicalswift d604b4cc8c tests: Update valgrind suppressions 2019-11-13 14:46:09 +00:00
MarcoFalke 333362991c
doc: Explain $LIB in LD_PRELOAD in gitian descriptors 2019-11-12 12:06:34 -05:00
TheCharlatan 0ec967164f
Add TheCharlatan's pgp key 2019-11-10 21:43:50 +01:00
MarcoFalke fab9850ef4
scripted-diff: Avoid hardcoded libfaketime dir in gitian
-BEGIN VERIFY SCRIPT-
sed -i -e "s|'export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/faketime/libfaketime.so.1'|\"export LD_PRELOAD='/usr/\\\\\$LIB/faketime/libfaketime.so.1'\"|g" $(git grep -l 'x86_64-linux-gnu/faketime')
sed -i -e 's|"amd64"|"linux64"|g' $(git grep -l '"amd64"')
-END VERIFY SCRIPT-
2019-11-07 22:35:51 -05:00
Gr0kchain 582e66b6e7 doc: Added regtest config for linearize script
Updated the example-linearize.cfg file to include support for the regtest chain network config which is used by the ./linearize-data.py

Problem:

Without the regtest magic, genesis hash and path config, the `linearize-data.py` script cannot generate a bootstrap.dat file.

Example of error:

./linearize-data.py ./linearize.cfg
Read 102 hashes
Genesis block not found in hashlist

Solution:

Added netmagic, genesis and input example parameters to file.

Resolution

1. Starting bitcoind in regtest mode
2. bitcoin-cli generatetoaddress 101 $(bitcoin-cli getnewaddress)
3. ./linearize-hashes.py ./linearize.cfg > ./hashlist.txt
4. ./linearize-data.py ./linearize.cfg

Example after fix:
$ ./linearize-data.py ./linearize.cfg
Read 102 hashes
Input file /Users/gr0kchain/.bitcoin/regtest/blocks/blk00000.dat
Output file /Users/gr0kchain/Downloads/bootstrap.dat
Done (102 blocks written)
2019-11-07 08:07:10 +02:00
Hennadii Stepanov 577682d9e8
script: Enable SC2006 rule for Gitian scripts 2019-11-06 15:11:54 +02:00
Wladimir J. van der Laan b05b28183c
Merge #16899: UTXO snapshot creation (dumptxoutset)
92b2f5306b test: add dumptxoutset RPC test (James O'Beirne)
c1ccbc3dde devtools: add utxo_snapshot.sh (James O'Beirne)
57cf74c991 rpc: add dumptxoutset (James O'Beirne)
92fafb3a7d coinstats: add coins_count (James O'Beirne)
707fde7b9b add unused SnapshotMetadata class (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal

  ---

  This changeset defines the serialization format for UTXO snapshots and adds an RPC command for creating them, `dumptxoutset`. It also adds a convenience script for generating and verifying snapshots at a certain height, since that requires doing a hacky rewind of the chain via `invalidateblock`.

  All of this is unused at the moment.

ACKs for top commit:
  laanwj:
    ACK 92b2f5306b

Tree-SHA512: 200dff87767f157d627e99506ec543465d9329860a6cd49363081619c437163a640a46d008faa92b1f44fd403bfc7a7c9e851c658b5a4849efa9a34ca976bf31
2019-11-05 19:40:18 +01:00
James O'Beirne c1ccbc3dde devtools: add utxo_snapshot.sh
to allow easy (if not time-consuming) generation and verification of
snapshots.
2019-11-05 13:36:04 -05:00
randymcmillan ac831339cb
doc: Fix some misspellings 2019-11-04 04:22:53 -05:00
Wladimir J. van der Laan feb1a8c03a
Merge #17308: nsis: Write to correct filename in first place
3b3b93174a nsis: Write to correct filename in first place (Carl Dong)

Pull request description:

  Per MarcoFalke's suggestion here https://github.com/bitcoin/bitcoin/pull/17029#discussion_r333216722

ACKs for top commit:
  MarcoFalke:
    unsigned ACK 3b3b93174a, makes sense to name it that way because it will raise the "unsinged" error in Windows

Tree-SHA512: da72aae438505e162d0b3cd27d873b7ad8176178bb459a738e61b6e2ad0fa739d905b3109fab641bb1a3950fe59ad526c5568d12cf48a305166cdb7db6686543
2019-10-31 13:23:47 +01:00
Carl Dong 3b3b93174a
nsis: Write to correct filename in first place 2019-10-29 15:12:52 -04:00
fanquake dd94cc46e4
contrib: remove accounts from bash completion
Also removes setgenerate
2019-10-28 11:53:40 -04:00
Rjected 317fb96de9
Add search for first blk file with pruned node 2019-10-27 23:08:10 -04:00
fanquake 3548e4aac7
Remove BIP70 Support 2019-10-24 16:01:43 -04:00
fanquake feb5075777
build: use osslsigncode 2.0 in gitian
The original osslsigncode project (https://sourceforge.net/projects/osslsigncode/) has been marked as abandonware,
"This is now - and has been for a long while - abandonware. Feel free to create your own forks etc.".

However, a fork at https://github.com/mtrojnar/osslsigncode has emerged that has incorporated
theuni's patches, updated the tool to work with OpenSSL 1.1 and made other improvements.

This commit switches the windows signer descriptor to use this new version of osslsigncode.
2019-10-21 18:32:53 -04:00
Zakk 4441e58497 Update macdeploy README to include correctly named .dmg file produced from make deploy 2019-10-15 11:30:13 +01:00
MarcoFalke 4765b91f50
Merge #16667: build: remove mingw linker workaround from win gitian descriptor
bd3f5a90ec build: remove mingw linker workaround from win gitian descriptor (fanquake)

Pull request description:

  This workaround was added as part of the switch to gitian building using Ubuntu 14.04 (#6900).
  However, it should no longer be required, as we have switched to Bionic (#13171) and that
  has a far newer version of binutils.

  Original discussion: https://github.com/bitcoin/bitcoin/pull/6900
  binutils patch: https://sourceware.org/bugzilla/show_bug.cgi?id=16192

ACKs for top commit:
  MarcoFalke:
    ACK bd3f5a90ec
  theuni:
    ACK bd3f5a90ec
  laanwj:
    ACK bd3f5a90ec

Tree-SHA512: 01a5789994decf8cdedf7aaa0a449d2100a77e2e6b422d6b9dd5a4ac3e2e0b538c3d43aae4a1c3713614782f3c6b09d8d8bb21c20e86ce3c1734183dedd02d0c
2019-10-14 08:38:41 -04:00
Wladimir J. van der Laan 9b4dfec831
Merge #17029: gitian: Various improvements for Windows descriptor
9d1f971c75 gitian: Put things in the right place to begin with (Carl Dong)
71949a97a7 gitian: Eliminate rename dependency (Carl Dong)
999a9a5f5b gitian: Smaller diff with gitian-linux.yml (Carl Dong)
c4a3c25ba1 gitian: Fix README inclusion in archives (Carl Dong)
93cb974980 gitian: Use split-debug.sh for Win builds (Carl Dong)

Pull request description:

  It would seem that our `gitian-win.yml` has not been keeping up with `gitian-linux.yml`, this PR:

  1. Minimizes the diff size between `gitian-{win,linux}.yml`
  2. Eliminates the `rename` dependency

ACKs for top commit:
  laanwj:
    ACK 9d1f971c75

Tree-SHA512: 84ed47c685e12d0064c02811907ae3d0fd3c47db8773d497dcc38f0defbfb3040fd82899fb026cf355f229b906d05a1c8038a95642bb90d044afbc2e0b239af2
2019-10-09 09:38:00 +02:00
Carl Dong 9d1f971c75
gitian: Put things in the right place to begin with 2019-10-08 13:39:48 -04:00
Carl Dong 71949a97a7
gitian: Eliminate rename dependency 2019-10-08 13:39:10 -04:00
Carl Dong 999a9a5f5b
gitian: Smaller diff with gitian-linux.yml 2019-10-08 13:39:09 -04:00
Carl Dong c4a3c25ba1
gitian: Fix README inclusion in archives
Linux:
The README was originally added in 8550f1fb2, but included the README
under the docs directory, which has a bunch of internal links that won't
make sense in a release tarball. In this patch, we include the root
level README instead, which makes more sense.

Windows:
.md files are inconvenient to open on windows and the line endings
differ, so we use README_windows.txt instead.
2019-10-08 13:36:50 -04:00
Wladimir J. van der Laan 99cebc922c
Merge #16569: Increase init file stop timeout
7fb7acfc20 Set init stop timeout to 10 min (setpill)

Pull request description:

  `bitcoind` can take a long time to flush its db cache to disk upon
  shutdown. Systemd sends a `SIGKILL` after a timeout, causing unclean
  shutdowns and triggering a long "Rolling forward" at the next startup.
  Disabling the timeout should prevent this from happening, and does not
  break systemd's `restart` logic.

  Addresses #13736.

ACKs for top commit:
  instagibbs:
    utACK 7fb7acfc20

Tree-SHA512: 16e0ce5a9ecf0628f8d93d68db3f5a78ab36021d9bede05a90c84f144db2e87e17707a6eb910cb7c018c265ce2c81d43de2988bd79e4a2d8554515db8fb5aa36
2019-10-08 11:38:44 +02:00
Wladimir J. van der Laan df50fd194f
Merge #16802: scripts: In linearize, search for next position of magic bytes rather than fail
3284e6c09a scripts: search for next position of magic bytes rather than fail (Tim Akinbo)

Pull request description:

  When using the `linearize-data.py` contrib script to export block data, there are edge cases where the script fails with an `Invalid magic: 00000000` error. This error occurs due to the presence of padding bytes that occasionally appears between consecutive blocks in the block data file.

  There's an ongoing conversation about this in #14986. sipa also admitted that it is a bug in #5028. Fortunately, this is not an issue in bitcoin core as it handles this type of situation gracefully and so no fix in bitcoin core is required.

  This PR is an improvement on how the script handles these "invalid magic bytes". Rather than failing, this patch allows the script to search for the next occurrence of the magic bytes and then starts reading the block from there.

ACKs for top commit:
  laanwj:
    ACK 3284e6c09a

Tree-SHA512: 18067ae0b4b62e822dfc558a86439ad6acaf939b98479e38e8e4248536574643b26eb48e96ec7139375c88b42cbe7705a64deb13a3c239e16025a6aad3d69bfa
2019-10-08 10:42:04 +02:00
fanquake 7b701fef58
Merge #17049: contrib: Bump gitian descriptors for 0.20
fa1ad8f06e build: Bump gitian descriptor versions (MarcoFalke)

Pull request description:

  Bump the gitian descriptor versions as a follow-up to #17007.

  Also fixes #17027 with a cherry-pick, and bump the manpages.

ACKs for top commit:
  fanquake:
    ACK fa1ad8f06e

Tree-SHA512: c3b669c3797e5febb51a8dd01e2621a7544a291e080d73c47a2a12ea9da84ff904533e68792e2e869ebbdc2226b2fee7517214549e6cc7e988f175098f7c412c
2019-10-04 14:39:59 -04:00
gchuf dffae5a5ad scripts: update copyright_header script to include additional files
Includes .sh and .bash-completion files in the script as well
2019-10-04 17:47:55 +02:00
MarcoFalke fa1ad8f06e
build: Bump gitian descriptor versions 2019-10-04 09:17:57 -04:00
Brian Solon 0b1dcd32bf
contrib: fix minor typos in makeseeds.py
See #17020
2019-10-03 14:50:08 -04:00
Carl Dong 93cb974980
gitian: Use split-debug.sh for Win builds
Debug splitting was first introduced in 7e7eb2724, then gitian-linux.yml
changed to using split-debug.sh in 9d2536208. Here we change
gitian-win.yml to use split-debug.sh as well.
2019-10-02 16:21:22 -04:00
Wladimir J. van der Laan 0218171a24 contrib: Remove invalid nodes from seeds list 2019-10-02 08:51:12 +02:00
Wladimir J. van der Laan 3b09f2b9d9 net: 0.19 hardcoded seeds update 2019-10-01 11:38:48 +02:00
Wladimir J. van der Laan 801d341f3a contrib: makeseeds: More fancy output 2019-10-01 11:39:41 +02:00
Wladimir J. van der Laan ed76299bea contrib: makeseeds: Limit per network, instead of total 2019-10-01 11:39:35 +02:00
Wladimir J. van der Laan c254a9ef69 contrib: makeseeds: dedup by ip,port
Handle the multiple ports per IP case (as that's a criterion later).
2019-10-01 11:39:29 +02:00
Wladimir J. van der Laan 3314d87966 contrib: makeseeds: Factor out ASN lookup 2019-10-01 11:39:14 +02:00
Wladimir J. van der Laan 301c2b1ab5 contrib: makeseeds: Improve logging and filtering
- Change regular expression to cover recent versions, as well as
  subversions with custom uacomment, and improve readability.
- Vary uptime requirements per network (onions are allowed to have less
  uptime, to make sure we get enough of them)
- Add deduplication step (to allow simple concatentation of multiple seeds files).
- Log of number of nodes (per network) after every step.
2019-10-01 11:38:48 +02:00
Tim Akinbo 3284e6c09a scripts: search for next position of magic bytes rather than fail
document seek method for next position of magic bytes
2019-09-30 18:10:47 +00:00
fanquake bd3f5a90ec
build: remove mingw linker workaround from win gitian descriptor
This workaround was added as part of the switch to gitian building using Ubuntu 14.04 (#6900).
However, it should no longer be required, as we have switched to Bionic (#13171), and that
has a far newer version of binutils.

binutils patch: https://sourceware.org/bugzilla/show_bug.cgi?id=16192
2019-09-23 08:38:42 +08:00
Jon Layton 6aab7649d3 doc: Fix whitespace errs in .md files, bitcoin.conf, Info.plist.in, and find_bdb48.m4 2019-09-17 03:21:22 -04:00
fanquake 59681beb89
Merge #16477: build: skip deploying plugins we dont use in macdeployqtplus
1ac7b7f66b scripts: filter more qt plugins we don't use in macdeployqtplus (fanquake)
57cdd0697d scripts: misc cleanups in macdeployqtplus (fanquake)
51729a4dfa scripts: use format() in macdeployqtplus (fanquake)
1c37e81694 scripts: add type annotations to macdeployqtplus (fanquake)

Pull request description:

  I frequently run `make deploy` while testing on macOS to get a properly light themed .app. With a brew installed Qt, this currently results in a pretty bloated executable:

  | branch | .app size | .dmg size | `make deploy` time |
  | ------- | --------- | --------- | --------------------- |
  | master (febf3a856b) | 235mb | 86mb | 38s |
  | This PR (da98f6d470d236c027b7eb8b5f5552fdca04e803) | 51mb | 21mb | 22s |

  Similar change to dd367ff8c9.

  ```diff
                           'QtGui.framework'],
    'pluginPath': '/usr/local/opt/qt/plugins',
    'qtPath': '/usr/local/opt/qt'}
  -[('platforminputcontexts', 'libqtvirtualkeyboardplugin.dylib'),
  - ('geoservices', 'libqtgeoservices_esri.dylib'),
  - ('geoservices', 'libqtgeoservices_mapboxgl.dylib'),
  - ('geoservices', 'libqtgeoservices_nokia.dylib'),
  - ('geoservices', 'libqtgeoservices_itemsoverlay.dylib'),
  - ('geoservices', 'libqtgeoservices_osm.dylib'),
  - ('geoservices', 'libqtgeoservices_mapbox.dylib'),
  - ('sceneparsers', 'libgltfsceneexport.dylib'),
  - ('sceneparsers', 'libgltfsceneimport.dylib'),
  - ('platforms', 'libqwebgl.dylib'),
  +[('platforms', 'libqwebgl.dylib'),
    ('platforms', 'libqoffscreen.dylib'),
    ('platforms', 'libqminimal.dylib'),
    ('platforms', 'libqcocoa.dylib'),
    ('platformthemes', 'libqxdgdesktopportal.dylib'),
  - ('printsupport', 'libcocoaprintersupport.dylib'),
  - ('webview', 'libqtwebview_webengine.dylib'),
  - ('webview', 'libqtwebview_darwin.dylib'),
  - ('geometryloaders', 'libdefaultgeometryloader.dylib'),
  - ('geometryloaders', 'libgltfgeometryloader.dylib'),
    ('styles', 'libqmacstyle.dylib'),
  - ('canbus', 'libqttinycanbus.dylib'),
  - ('canbus', 'libqtpassthrucanbus.dylib'),
  - ('canbus', 'libqtvirtualcanbus.dylib'),
  - ('canbus', 'libqtpeakcanbus.dylib'),
    ('bearer', 'libqgenericbearer.dylib'),
  - ('imageformats', 'libqgif.dylib'),
  - ('imageformats', 'libqwbmp.dylib'),
  - ('imageformats', 'libqwebp.dylib'),
  - ('imageformats', 'libqico.dylib'),
  - ('imageformats', 'libqmacheif.dylib'),
  - ('imageformats', 'libqjpeg.dylib'),
  - ('imageformats', 'libqtiff.dylib'),
  - ('imageformats', 'libqicns.dylib'),
  - ('imageformats', 'libqtga.dylib'),
  - ('imageformats', 'libqmacjp2.dylib'),
  - ('texttospeech', 'libqtexttospeech_speechosx.dylib'),
  - ('generic', 'libqtuiotouchplugin.dylib'),
  - ('renderplugins', 'libscene2d.dylib'),
  - ('gamepads', 'libdarwingamepad.dylib'),
  - ('virtualkeyboard', 'libqtvirtualkeyboard_thai.dylib'),
  - ('virtualkeyboard', 'libqtvirtualkeyboard_openwnn.dylib'),
  - ('virtualkeyboard', 'libqtvirtualkeyboard_hangul.dylib'),
  - ('virtualkeyboard', 'libqtvirtualkeyboard_pinyin.dylib'),
  - ('virtualkeyboard', 'libqtvirtualkeyboard_tcime.dylib')]
  + ('generic', 'libqtuiotouchplugin.dylib')]
  ```

ACKs for top commit:
  laanwj:
    ACK 1ac7b7f66b (purely Python code review and the fact that this passes travis, cannot run this on a mac)
  dongcarl:
    tested ACK 1ac7b7f66b

Tree-SHA512: 5974eeaf7229bb5bde2b283c1331ec57ee87f624db146401f6b77dee4ee5502e0bd669958a46205f10398a371f8e6c91ddacb9f0e1943f9f7d042fb6de7957a8
2019-09-10 10:53:59 +10:00
fanquake b5a8d0cff1
Merge #15450: gui: Create wallet menu option
613de61a04 Add Create Wallet menu action (Andrew Chow)
9b41cbb28f Expose wallet creation to the GUI via WalletController (Andrew Chow)
78863e2900 Add CreateWalletDialog to create wallets from the GUI (Andrew Chow)
60adb21c7a Optionally allow AskPassphraseDialog to output the passphrase (Andrew Chow)
bc6d8a3662 gui: Refactor OpenWalletActivity (João Barbosa)

Pull request description:

  This PR adds a menu option to create a new wallet. When clicked, a `CreateWalletDialog` will be created and prompt the user to name the wallet and choose whether to disable private keys, make a blank wallet, and encrypt the wallet. If the wallet is encrypted, the wallet will be born encrypted with the wallet first created blank, then encrypted, and then a new HD seed generated and set.

  To allow the newly created wallets to be encrypted, some changes to how encrypting a wallet works. Instead of encrypting and locking the wallet, the wallet will be encrypted and then unlocked. This is also an extra belt-and-suspenders check to make sure that encryption worked.

ACKs for top commit:
  fanquake:
    ACK 613de61a04 - re-reviewed on macOS. I'm going to merge this now. It's had a stack of review, and as mentioned multiple times above, lets get this into `master` so it can get more testing pre `v0.19.0`.

Tree-SHA512: 3f22cc20b13703ffc90d366ae9133114832fea77f4f319da7fd85eb454f2f0bd5d7e1e6e20284dea2f370d8574f83b45669dcbbe506b994410d32e8e7a6fa877
2019-09-07 14:45:58 +08:00
setpill 7fb7acfc20 Set init stop timeout to 10 min
`bitcoind` can take a long time to flush its db cache to disk upon
shutdown. Most init files send a `SIGKILL` after a timeout of 1 minute,
causing unclean shutdowns and triggering a long "Rolling forward" at the
next startup. Increasing this timeout to 10 minutes should reduce how
often this occurs, especially during IBD.

fixup! Set ProtectHome in systemd service file
2019-09-06 17:03:04 +02:00
Andrew Chow 78863e2900 Add CreateWalletDialog to create wallets from the GUI
Co-authored-by: João Barbosa <joao.paulo.barbosa@gmail.com>
2019-09-05 20:36:57 -04:00
Carl Dong 0065ead5eb
contrib: guix: Remove ssp spec file hack
This hack is no longer needed after fixing our cross-compilation search
paths.
2019-09-05 14:50:56 -04:00
Carl Dong 0093a5869a
contrib: guix: More robust search paths, add checks
- store_path() previously only worked for cross compilation packages, we
  remove this assumption here
- Add CROSS_GCC_LIB variable which points to where gcc libs/headers are
  located
- Add gcc libs/headers to our CROSS_*_PATH environment variables
- Check that all directories in CROSS_*_PATH are sane
2019-09-05 14:50:44 -04:00
Ben Woosley 3d0a82cff8
devtools: Accomodate block-style copyright blocks
Without this, `copyright_header.py report . verbose` reports:
-------------------------------------------------------------------------------
1 with unexpected copyright holder names
	./build_msvc/libsecp256k1_config.h
-------------------------------------------------------------------------------
2019-09-03 14:41:43 -04:00
Ben Woosley b21680baf5
test/contrib: Fix invalid escapes in regex strings
Flagged by flake8 v3.6.0, as W605, plus a few others identified
incidentally, e.g. 59ffecf66cf4d08c4b431e457b083878d66a3fd6.

Note that r"\n" matches to "\n" under re.match/search.
2019-09-03 14:38:38 -04:00
fanquake fc5b756bae
Merge #16556: Fix systemd service file configuration directory setup
f3b57f4a1c Unrecommend making config file owned by bitcoin (setpill)
870d4152df Set ProtectHome in systemd service file (setpill)
639a416e37 Chgrp config dir to bitcoin in systemd service (setpill)
aded0528f0 Improve clarity of systemd service file comments (setpill)

Pull request description:

  Rationale: ran into a bug with the systemd service file, fixed it locally and figured I might as well contribute my fix.

  Also fixed some unrelated confusing phrasing in the comments of the same file, after discussion in IRC.

ACKs for top commit:
  sipsorcery:
    tACK f3b57f4a1c (nothing changed since previous tACK).
  ryanofsky:
    utACK f3b57f4a1c. Only change since last review is removing ConfigurationDirectoryMode churn in early commits

Tree-SHA512: 2188345878925b9e8a5c2c3df8dfba443720e2252a164db54a8e1d8007846721497b2d98c56f1d9b60a9a9ed4fdb1156c7b02c699616b220a9b614671617d32a
2019-08-29 11:55:03 +08:00
setpill 870d4152df Set ProtectHome in systemd service file
Further hardening; the service should be run with as many restrictions
as possible without breaking it.
2019-08-20 10:54:14 +02:00
setpill 639a416e37 Chgrp config dir to bitcoin in systemd service
Rather than making the config dir world-readable, which potentially
leaks RPC credentials, the group of the directory is changed to the one
the service is executed as.
2019-08-20 10:54:14 +02:00
setpill aded0528f0 Improve clarity of systemd service file comments
The phrasing seemed to indicate that the options specified in
ExecStart= could not be specified in the config file, necessitating
their inclusion in the service file. However, the options in the
config file simply get overridden by any specified in ExecStart=.
2019-08-20 10:54:14 +02:00
fanquake 3d50fe2c1f
scripts: move update-translations.py to maintainer-tools repo 2019-08-20 11:14:00 +08:00
fanquake 6c27fa0f09
scripts: remove github-merge.py
This script has been moved to the bitcoin-core/bitcoin-maintainer-tools repository,
after discussion in a core dev meeting. The rationale being that it is also useful to
other projects, and thus should be moved to a more general repository.

Meeting log: http://www.erisian.com.au/bitcoin-core-dev/log-2019-08-15.html
See also: https://github.com/bitcoin-core/bitcoin-maintainer-tools/pull/28
2019-08-19 19:16:57 +08:00
Wladimir J. van der Laan 390874c722 qt: Remove menu icons 2019-08-15 13:05:10 +02:00
Chuf 36f7697cf6
doc: Fix typos in COPYRIGHT
Fixed a couple of typos
2019-08-14 17:28:26 +00:00
Aaron Clauson 9063911017
sipsorcery gitian key. 2019-08-10 05:39:47 +02:00
fanquake 1ac7b7f66b
scripts: filter more qt plugins we don't use in macdeployqtplus
phonon is no longer a part of Qt as of version 5
2019-07-29 09:35:51 +08:00
fanquake 57cdd0697d
scripts: misc cleanups in macdeployqtplus 2019-07-29 08:28:41 +08:00
fanquake 51729a4dfa
scripts: use format() in macdeployqtplus 2019-07-29 08:28:41 +08:00
fanquake 1c37e81694
scripts: add type annotations to macdeployqtplus 2019-07-29 08:28:41 +08:00
Carl Dong 65f8da08df
symbol-check: Disallow libX11-*.so.* shared libraries
They should no longer be needed as we build QT without libX11/XLib
libraries now.
2019-07-17 17:09:48 -04:00
Carl Dong 751549b52a
contrib: guix: Additional clarifications re: substitutes 2019-07-12 12:31:55 -04:00
Carl Dong cd3e947f50
contrib: guix: Various improvements.
- Clearer and more accurate prose
- Pin `guix pull' to commit rather than branch
- Just use `use-module' instead of `define-module'
- Use `bash-minimal' instead of `bash'
- Remove unneeded `tcsh' from manifest
- Explicitly use `python-3.7'
- Add comments about how {native,cross}-toolchains are produced and
  why
2019-07-12 11:42:36 -04:00
Carl Dong 8dff3e48a9
contrib: guix: Clarify SOURCE_DATE_EPOCH. 2019-07-12 11:42:09 -04:00
Carl Dong 3e80ec3ea9
contrib: Add deterministic Guix builds. 2019-07-12 00:48:39 -04:00
fanquake 1088b90cba
Merge #16327: scripts and tools: Update ShellCheck linter
1ac454a384 Enable ShellCheck rules (Hennadii Stepanov)

Pull request description:

  Enable some simple ShellCheck rules.

  Note for reviewers: `bash` and `shellcheck` on macOS are different from ones on Ubuntu.
  For local tests the latest `shellcheck` version 0.6.0 should be used (see #15166).

ACKs for top commit:
  practicalswift:
    utACK 1ac454a384
  dongcarl:
    utACK 1ac454a
  fanquake:
    ACK 1ac454a384

Tree-SHA512: 8d0a3a5c09fe1a0c22120178f5e6b80f81f746f8c3356b7701ff301c117acb2edea8fe08f08fb54ed73f94b1617515fb239fa28e7ab4121f74872e6494b6f20e
2019-07-05 09:19:23 +08:00
Hennadii Stepanov 1ac454a384
Enable ShellCheck rules
Enabled ShellCheck rules:
  SC1087
  SC2001
  SC2004
  SC2005
  SC2006
  SC2016
  SC2028
  SC2048
  SC2066 (note that IFS already contains only a line feed)
  SC2116
  SC2166
  SC2181
  SC2206
  SC2207
  SC2230
  SC2236
2019-07-04 19:35:25 +03:00
fanquake dfdcb3dfe5
Merge #16330: docs: Use placeholder instead of key expiration date
88fd556a96 Use placeholder instead of key expiration date (Hennadii Stepanov)

Pull request description:

  Use a placeholder instead of the actual expiration date, so that the documentation doesn't require updating every time an expiry date changes.

ACKs for top commit:
  fanquake:
    ACK 88fd556a96

Tree-SHA512: 391707833cc0e701cf560ec82fd91368468c90a95f85e4ce2a211b20d12463c85775142f28a3536b57c5f6950b9e6e0785632f6f071fa2180bc8aab53008603b
2019-07-04 11:03:52 +08:00
Hennadii Stepanov 88fd556a96
Use placeholder instead of key expiration date 2019-07-03 17:55:51 +03:00
Hennadii Stepanov ca11606858
Fix: "Bitcoin Core" -> "The Bitcoin Core" 2019-06-29 18:58:52 +03:00
Hennadii Stepanov 621463d7df
Drop no-longer-relevant copyright holder name 2019-06-29 18:50:39 +03:00
Hennadii Stepanov 01fafe55a1
Include Objective-C source files
The copyright_header.py script will process Objective-C source files 
(*.mm) as other ones.
2019-06-29 18:35:45 +03:00
MarcoFalke e115a21f79
Merge #16223: devtools: Fetch and display ACKs at sign-off time in github-merge
0e01e4522e devtools: Fetch and display ACKs at sign-off time in github-merge (Wladimir J. van der Laan)

Pull request description:

  - Fetch the ACKs only at sign-off time. This makes sure that any last-minute ACKs are included (fixes #16200)
  - Show a list of ACKs that will be included and their author before signing off, and warn if there are none

  ![1](https://user-images.githubusercontent.com/126646/59605250-ad070980-910e-11e9-9f9a-d789c7f06ebb.png)
  ![2](https://user-images.githubusercontent.com/126646/59605255-b1332700-910e-11e9-80a5-d1e244f48264.png)

  There's a slight change to the merge commit format—before it was
  ```
      ACKs for commit 88884c:
  (list of ACKs, could be empty)
  ```
  now it is
  ```
  ACKs for top commit:
        jnewbery:
          ACK 5ebc6b0eb
      ... (list of ACKs cannot be empty)
  ```
  or
  ```
  Top commit has no ACKs.
  ```
  I don't think there's a reason to have the abbreviated commit ID there, after all the full commit id is already in the beginning of the merge commit message, and at least the abbreviated one is in every single ACK message.

ACKs for commit 0e01e4:
  fanquake:
    ACK 0e01e4522e

Tree-SHA512: 8576de016137d71cfc101747e9bb6779c13e0953cf2babee7afc9972bf2bd46f6912be4982b54fa5abf4d91e98e8fdae6b4ca3eef7d6892b7a5f04a7017b6882
2019-06-24 15:09:24 -04:00
Wladimir J. van der Laan 0e01e4522e devtools: Fetch and display ACKs at sign-off time in github-merge
- Fetch the ACKs only at sign-off time. This makes sure that any
  last-minute ACKs are included (fixes #16200)
- Show a list of ACKs and their author before signing off, and warn if
  there are none
2019-06-17 14:54:35 +02:00
fanquake 47d981e827
Merge #16186: doc/lint: Fix spelling errors identified by codespell 1.15.0
b748bf6f50 Fix spelling errors identified by codespell 1.15.0 (Ben Woosley)

Pull request description:

  Note all changes are to comments / documentation.

  After this commit, the only remaining output is:

  ```
    $ test/lint/lint-spelling.sh
    src/test/base32_tests.cpp:14: fo  ==> of, for
    src/test/base64_tests.cpp:14: fo  ==> of, for
    ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt
  ```

  Note:
  * I ignore several valid alternative spellings ~, but changed homogenous
  to homogeneous as the latter is a more specific term according to the
  Google dictionary definitions I found~
  * homogenous is present in tinyformat, hence should be addressed upstream
  * process' is correct only if there are plural processes

ACKs for commit b748bf:
  practicalswift:
    ACK b748bf6f50
  fanquake:
    ACK b748bf6f50

Tree-SHA512: 9add7044643ce015e0a44d8b27a3f300d72c485ffff550fb6491a17f14528085289ec5caddfe02f291ea9b2cded38a0dd3079652a054e2d7fe2ff4f7b53db5d7
2019-06-16 09:57:09 +08:00
fanquake 7524376a81
Merge #16201: devtools: Always use unabbreviated commit IDs in github-merge.py
067fba563 devtools: Always use unabbreviated commit IDs in github-merge.py (Wladimir J. van der Laan)

Pull request description:

  Always put the unabbreviated commit IDs in the generated commit messages and other places. This prevents the developer's `core.abbrev` git setting from leaking through and is better against ambiguity too.

ACKs for commit 067fba:
  MarcoFalke:
    ACK 067fba5631 (replaces `h` with `H`, didn't test)
  promag:
    ACK 067fba5631, from the documentation https://git-scm.com/docs/pretty-formats:
  fanquake:
    ACK 067fba5631. Tested by merging this PR into master, then merging a second PR ontop and checking that full commit hashes were being used. Also checked documentation linked to above. Did not check that this works when a different `core.abbrev` is set locally.

Tree-SHA512: a851d10490cd8bcd8bca29094b08a6b9f883cfe1b0767ccda7ca789e4c8eff6260a4d82c33cb3d9bab01dd30ac8c9100cb7adbcb1911bb399d9385c1e1f15ecd
2019-06-14 13:38:52 +08:00
Wladimir J. van der Laan 067fba5631 devtools: Always use unabbreviated commit IDs in github-merge.py
Always put the unabbreviated commit IDs in the generated commit messages
and other places. This prevents the developer's `core.abbrev` git setting
from leaking through and is better against ambiguity too.
2019-06-13 10:21:36 +02:00
MarcoFalke 473c6b7c5a
Merge #16184: scripted-diff: gitian: Use REFERENCE_DATETIME directly.
993aa414d3 scripted-diff: gitian: Use REFERENCE_DATETIME directly. (Carl Dong)

Pull request description:

  Fixes regression introduced by #16141.

  ```
  -BEGIN VERIFY SCRIPT-
  sed -i 's#\$REFERENCE_DATE\\\\\\ \$REFERENCE_TIME#\$REFERENCE_DATETIME#g' contrib/gitian-descriptors/*
  -END VERIFY SCRIPT-
  ```

  -----

  Note that this could have been fixed by escaping properly, but using `REFERENCE_DATETIME` directly is simpler.

  Future note: `REFERENCE_{DATE{,DATETIME},TIME}` is a bit ridiculous. At the very _least_ gitian should use epoch, as it is the most parse-able, and preferably set SOURCE_DATE_EPOCH.

ACKs for commit 993aa4:

Tree-SHA512: 8457e5fffde66e1d2b846547b6807416b884c171f63569f76dfefd498d2a58ad6f9eb93931eb6cfc7ff38c6b460b0c488ca87d1a68bc630c48f365a74b6ee163
2019-06-11 12:10:00 -04:00
Ben Woosley b748bf6f50
Fix spelling errors identified by codespell 1.15.0
After this commit, the only remaining output is:

  $ test/lint/lint-spelling.sh
  src/test/base32_tests.cpp:14: fo  ==> of, for
  src/test/base64_tests.cpp:14: fo  ==> of, for
  ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt

Note:
* I ignore several valid alternative spellings
* homogenous is present in tinyformat, hence should be addressed upstream
* process' is correct only if there are plural processes
2019-06-11 17:18:16 +02:00
MarcoFalke 6899ef3f0e
Merge #16143: tests: Mark unit test blockfilter_index_initial_sync as non-deterministic
c061be1e2f tests: Mark unit test blockfilter_index_initial_sync as non-deterministic (practicalswift)

Pull request description:

  Mark unit test `blockfilter_index_tests/blockfilter_index_initial_sync` as non-deterministic.

  Before this PR:

  ```
  $ contrib/devtools/test_deterministic_coverage.sh 500
  [2019-06-04 09:58:57] Measuring coverage, run #1 of 500
  [2019-06-04 10:00:33] Measuring coverage, run #2 of 500
  [2019-06-04 10:02:19] Measuring coverage, run #3 of 500

  The line coverage is non-deterministic between runs. Exiting.

  The test suite must be deterministic in the sense that the set of lines executed at least
  once must be identical between runs. This is a necessary condition for meaningful
  coverage measuring.

  --- gcovr.run-1.txt     2019-06-04 10:00:33.389059973 +0000
  +++ gcovr.run-3.txt     2019-06-04 10:03:45.619491207 +0000
  @@ -72,7 +72,7 @@
   hash.h                                        54      33    61%   71,74-77,82,85-89,111,113,128,147-148,175,178-181
   httprpc.cpp                                  120       3     2%   31,34-35,38-40,46,49,52,54,56,58,70,73-74,76,78-79,81,83-84,89,91,94-95,97,99-101,103,106-107,111-112,117-119,121-122,125,128,130,132,134-136,138-139,142,145,148,151-153,156-160,163-166,171,173-175,180-182,185,187,189-190,192,195,198-199,201,203-204,212,215,217,219-222,224,227-228,230,232,237,239-240,243-245,247-251,254,256,259,261-264,266-267 [* 205-206,208-209]
   httpserver.cpp                               312       6     1%   46,49-50,53,55,80-81,90,92-93,96-98,101,104,106-109,111-112,114,118,120-122,126,128-129,153,155,157-158,164,166-178,180,182,184-188,192,194-196,198-199,201-202,204-205,207-208,213,216-221,225,228-232,236-239,243-244,247-254,256-258,264-267,270-271,274,279,281-282,286,288-290,292-293,297,299-300,303-307,309-310,312-317,322-328,330,332,335,339,341-342,346,352-353,355,358,360,364,368-369,375,378,381-384,388-391,393-394,398-400,402,404-406,409,411-412,414,416,426,428-431,433-434,438,440-441,443,445-446,449,451-455,457-459,463-464,466-469,471-473,475-477,479,482,484,487,490-493,496-497,499-500,502,504,506,508-509,511,513-514,517,519,521-522,527,529-533,535,538,540-543,550-555,558,560-562,570,572-574,577-582,585-590,594-597,600,602-604,606-609,611,614,616,619,621,625-626,628-629,631-632,634-635,640,642-643,646,648-651,653,655-656
  -index/base.cpp                               149      94    63%   20,22-25,28,66,98,102-103,117-118,140-141,145-146,155,163,175,177-178,181-182,184-185,200-201,203,212,214-215,219-221,228-229,234,236,240,243-244,247-249,258-260,262,270,292-294,308-309 [* 263]
  +index/base.cpp                               149      97    65%   20,22-25,28,66,98,102-103,117-118,140-141,145-146,155,163,175,177-178,181-182,184-185,200-201,203,212,214-215,219-221,228-229,234,236,240,243-244,247-249,258-260,262,270,308-309 [* 263]
   index/base.h                                   3       2    66%   77
   index/blockfilterindex.cpp                   199     134    67%   70,79,81,84-88,91,122,139,142,179-181,184-185,188-189,193-194,201-202,207,233,258,262-263,265-266,268,271-272,274,277,279,284,286,288-289,294,301-302,304,322,329,332-333,350,371,373,438,440-441,444,446,449,455-456,459,461,464,466 [* 162-163]
   index/blockfilterindex.h                       4       4   100%
  @@ -358,7 +358,7 @@
   util/validation.cpp                            5       1    20%   12,15-17
   validation.cpp                              2167     808    37%   291,293,297-300,302,330,332,340,348,355-357,359,362,364-365,368,371,380,382-383,385-386,388-389,396,398-402,406-413,415,417,419,422-425,439-440,442-443,446,449,455-458,461-464,467,469-470,472,474,476,492,494-495,502-503,505-507,511-513,515,517,523,526,528,533,535,540,542-544,550,552-556,558-560,564,574,578-583,586,590-591,594-596,601-602,607-608,611-612,616-617,619-621,635-636,638,640,647-648,651,657-658,660-662,665-667,673,675,677-678,682-683,690,693,700-701,703-705,709-710,713-714,716,719-720,724-727,733-735,737-739,741-743,747-748,751-752,754,757-764,771,773-774,776-779,785-788,793-794,796-800,815-816,818-822,825,827,830,835,838-839,841-843,846-848,850,853,859,864-867,875,877-879,884-885,887-891,895,899-900,904-906,908-909,911,930-931,933,936,942,944-950,952,959,962,965-968,972,978,982-984,990-991,994-996,999,1003-1004,1011,1013,1015-1019,1022-1023,1026-1032,1056,1065,1079,1091,1108,1112,1114-1118,1125,1127-1130,1133-1135,1138-1139,1147,1149,1151-1152,1155,1197,1199-1201,1206-1209,1211-1212,1226,1230,1232-1234,1236,1238-1241,1245-1246,1256,1258,1260-1262,1264-1266,1268,1278-1280,1282-1283,1286,1289,1291-1292,1294-1302,1305-1311,1319-1323,1330,1332-1333,1336-1339,1379,1383-1384,1395,1401,1405-1407,1411-1414,1423-1428,1438-1440,1451,1455,1458,1471,1480,1497,1503,1519,1525,1527-1530,1532-1533,1536,1538-1539,1549,1551,1553,1555,1559-1562,1571,1573,1578,1580,1582-1584,1588-1589,1594-1597,1601-1606,1613-1616,1619-1623,1630,1632,1635,1637,1639-1640,1642-1646,1658,1660,1675,1688,1711,1713-1715,1742,1755,1760,1765,1769,1811,1815,1817,1841-1845,1855,1942,1946-1947,1956,1984-1986,1991-1992,1994,1996-1999,2005-2007,2010-2012,2022-2023,2028-2031,2038-2039,2042,2044,2049,2058-2061,2064,2114-2115,2117-2118,2120-2124,2152-2153,2156,2159-2163,2165-2169,2171-2172,2176-2178,2187-2188,2191-2194,2199,2207-2211,2215-2220,2224,2227-2230,2235,2237-2238,2261-2263,2265,2274,2278,2286,2301,2303-2304,2306-2309,2311,2313-2318,2320,2322,2325,2327-2328,2330,2332-2334,2338,2340,2343-2344,2407-2410,2430,2445-2447,2507-2509,2511-2514,2518,2520-2521,2523-2524,2561,2564,2590,2592-2593,2595-2598,2603,2620,2626,2658,2719,2724,2773,2776-2777,2779,2781,2783,2785-2788,2791,2793-2795,2799,2801-2802,2805,2807-2809,2813,2816,2818-2821,2825-2826,2832-2834,2841-2845,2848,2854,2858-2859,2861,2865-2868,2872-2875,2880,2884-2885,2890-2891,2894-2895,2897,2900-2906,2908,2910,2912,2918-2922,2924,2928-2929,2940,3002-3005,3009-3010,3026-3028,3036-3037,3039-3040,3045,3053,3056,3077,3080,3090,3112,3118,3129,3133,3135-3136,3141-3142,3150,3190-3193,3259,3268,3273,3277,3282-3285,3303,3314,3321-3324,3338-3341,3345-3346,3348-3350,3360,3372,3392,3397,3403,3406,3408,3435-3441,3443,3468-3469,3485,3487-3488,3492-3493,3534-3536,3542,3547-3549,3552,3565-3566,3601-3602,3610,3628,3630,3632,3645,3647,3649-3651,3653,3657,3659,3661-3669,3675-3680,3686-3687,3691,3693-3697,3702,3704,3706-3708,3711-3718,3720,3724,3726-3729,3748,3750-3752,3754,3758-3759,3763,3765,3767,3772,3774,3777-3778,3780-3781,3783,3787-3788,3790,3792-3794,3798-3800,3823,3825,3828,3830,3832,3836-3838,3841-3843,3845,3848,3850,3854-3856,3858-3859,3861-3862,3864-3867,3870-3873,3875-3876,3879,3882-3883,3886-3893,3899,3901,3905-3909,3911-3915,3922-3924,3926-3928,3931,3933-3934,3940-3942,3945-3947,3952,3954-3955,3957,3960-3961,3964,3966,3968-3972,3975,3977,3980,3982,3985,3987-3988,3992-3996,3998-4006,4008-4009,4011-4012,4014,4016,4019,4021-4022,4024-4026,4028-4032,4037-4041,4043-4045,4047,4050,4053-4054,4057,4060-4064,4066-4067,4069-4075,4079-4080,4086,4089-4091,4094-4097,4101,4106,4108,4110,4112-4114,4116-4117,4119,4121,4123-4124,4126,4128-4130,4132-4134,4138-4142,4144-4147,4154,4158-4163,4166-4169,4172-4173,4177,4179-4180,4183,4185,4187-4189,4191-4193,4195,4197-4201,4207-4208,4212,4220-4223,4230,4232-4233,4237,4240,4243,4247,4249,4251,4253-4255,4265-4266,4277,4279,4282,4285-4287,4292-4293,4296,4298,4302,4305-4306,4310-4311,4315-4318,4360,4363-4367,4370,4377,4397,4412,4415-4416,4418,4421-4422,4424,4426-4429,4433-4437,4439-4441,4448-4452,4454-4456,4458,4460,4462-4467,4471-4475,4477,4480-4481,4486-4488,4493,4496-4503,4505,4507-4511,4513-4514,4517-4519,4529-4531,4546,4600,4638-4639,4647,4653,4662-4664,4696,4703-4704,4718,4720,4723,4725,4727,4730,4732-4733,4736,4738-4739,4742,4744-4745,4750,4752-4757,4761-4765,4769-4770,4774-4776,4779-4781,4783-4785,4787-4790,4793-4794,4800-4801,4803,4807,4809-4810,4812-4813,4815-4816,4823,4827,4829,4831-4832,4834-4835,4838-4840,4842,4845,4848-4849,4853,4855-4856,4858-4863,4866-4872,4877,4891,4907 [* 1085-1086,1140-1141,1513-1514,2201-2202,2428,3569-3570,4400-4401,4442,4453,4504,4522-4523,4526-4527,4818-4819,4873-4874]
   validation.h                                  19       5    26%   338,350-352,356-363,366,484
  -validationinterface.cpp                       81      50    61%   78-82,85-86,112-113,116,119-120,123-124,126-128,130,133-136,151-153,163-165,169-171
  +validationinterface.cpp                       83      60    72%   78-82,85-86,112-113,116,133-136,151-153,163-165,169-171
   validationinterface.h                          9       4    44%   94,105,112,118,135
   versionbits.cpp                               92      27    29%   33,35-36,38-39,48-50,52-54,56-57,61-62,67-71,73,75-76,80,82-83,91,98,100,102-103,105,109-110,113-118,121-122,124,127,129-130,134,137,141,149,151,153-155,159,177,179,184,194,196,199,201,204,206 [* 26]
   versionbits.h                                  1       1   100%
  @@ -400,5 +400,5 @@
   zmq/zmqpublishnotifier.h                       5       0     0%   12,31,37,43,49
   zmq/zmqrpc.cpp                                23       3    13%   16,18,20,23,33-35,37,40-47,51,62,64-65
   ------------------------------------------------------------------------------
  -TOTAL                                      52472    7784    14%
  +TOTAL                                      52474    7797    14%
   ------------------------------------------------------------------------------
  $
  ```

  After this PR:

  ```
  $ contrib/devtools/test_deterministic_coverage.sh 500
  [2019-06-03 14:45:25] Measuring coverage, run #1 of 500
  [2019-06-03 14:48:15] Measuring coverage, run #2 of 500
  [2019-06-03 14:50:49] Measuring coverage, run #3 of 500
  [2019-06-03 14:52:20] Measuring coverage, run #4 of 500
  [2019-06-03 14:53:49] Measuring coverage, run #5 of 500
  …
  [2019-06-04 09:04:58] Measuring coverage, run #496 of 500
  [2019-06-04 09:07:42] Measuring coverage, run #497 of 500
  [2019-06-04 09:10:32] Measuring coverage, run #498 of 500
  [2019-06-04 09:13:26] Measuring coverage, run #499 of 500
  [2019-06-04 09:16:32] Measuring coverage, run #500 of 500

  Coverage test passed: Deterministic coverage across 500 runs.
  $
  ```

ACKs for commit c061be:

Tree-SHA512: 00cd55b4371290d8587ab667c64249bc31d26cc9dc3dd519677eb91ddb9dbc5333dfbdef5e90c7a0d74eecd24757113e7ec3eda836859ddc033b1de715df81b6
2019-06-11 07:37:03 -04:00
Carl Dong 993aa414d3
scripted-diff: gitian: Use REFERENCE_DATETIME directly.
Fixes regression introduced by #16141

-BEGIN VERIFY SCRIPT-
sed -i 's#\$REFERENCE_DATE\\\\\\ \$REFERENCE_TIME#\$REFERENCE_DATETIME#g' contrib/gitian-descriptors/*
-END VERIFY SCRIPT-
2019-06-10 15:14:05 -04:00
Wladimir J. van der Laan 70d77fff3a
Merge #16162: scripts: add key for Michael Ford (fanquake) to trusted keys list
8081927c33 scripts: add key for fanquake to trusted keys list (fanquake)

Pull request description:

  Adding my key to the [trusted keys list](https://github.com/bitcoin/bitcoin/blob/master/contrib/verify-commits/trusted-keys) to join the maintainer group. I'll gain merge access and will continue with all triage/repo management work. I'll be focusing primarily on build system development with some guidance from theuni.

  Some maintainer related discussion from the Core Dev meetup in Amsterdam is available [here](http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2019-06-06-maintainers/).

ACKs for commit 808192:
  MarcoFalke:
    ACK 8081927c33
  laanwj:
    ACK, this matches the key I have 8081927c33:
  meshcollider:
    ACK, this matches the key I have from the CoreDev New York 2018 keysigning party 8081927c33
  IlyasRidhuan:
    ACK. 8081927c33 matches the key I have.

Tree-SHA512: 63c390f5ede316263def2f9a897a1046d4ae58e4ea758f379164c6c5afce4928e2c9463fbcac004742838fd4ac1e48679e7a2a0f9095fd134a900f9064584056
2019-06-08 16:45:06 +02:00
Wladimir J. van der Laan af05f36b60
Merge #16141: build: remove GZIP export from gitian descriptors
bc8863b819 depends: remove usage of TAR_OPTIONS (fanquake)
3ff1f2a319 build: remove export GZIP from gitian descriptors (fanquake)

Pull request description:

  The `GZIP` environment variable is [deprecated](https://www.gnu.org/software/gzip/manual/gzip.html#Environment), and everywhere that we invoke `gzip` we are already passing `-9n` directly, i.e:
  ```base
    find bitcoin-* | sort | tar --no-recursion --mode='u+rw,go+r-w,a+X' --owner=0 --group=0 -c -T - | gzip -9n > ../$SOURCEDIST
  ```

  ```bash
  GZIP="-9n" gzip -h
  gzip: warning: GZIP environment variable is deprecated; use an alias or script
  Usage: gzip [OPTION]... [FILE]...
  ```

ACKs for commit bc8863:

Tree-SHA512: 2d5277f7bf096fd5bd0dda47dfaf2dc7a31cc5d91eb8cb42db9cbe060d07dff66bf8e1122a89a3a1b597a3b39dbf8d9a8da4f02e642f58e451ce9fb24cc59769
2019-06-07 14:45:34 +02:00
fanquake 8081927c33
scripts: add key for fanquake to trusted keys list 2019-06-07 09:48:12 +02:00
fanquake bc8863b819
depends: remove usage of TAR_OPTIONS 2019-06-05 15:39:17 +02:00
practicalswift c061be1e2f tests: Mark unit test blockfilter_index_initial_sync as non-deterministic 2019-06-04 12:28:04 +02:00
fanquake 3ff1f2a319
build: remove export GZIP from gitian descriptors 2019-06-03 15:51:59 -04:00
Steven Roose a352d2ae5f
Don't GPG sign intermediate commits with github-merge tool 2019-05-31 19:22:02 +01:00
Wladimir J. van der Laan c7cfd20a77
Merge #16086: contrib: use newer config.guess & config.sub in install_db4.sh
00fade007c contrib: use newer config.guess & config.sub in install_db4.sh (fanquake)

Pull request description:

  The `config.guess` and `config.sub` packaged with `db-4.8.30.NC.tar.gz` are fairly old (2009) and fail to identify some system types i.e `ppc64le`. Replace them with more modern versions before configuring `db4`.

  Fixes #16064.

ACKs for commit 00fade:
  jamesob:
    tACK 00fade007c
  dongcarl:
    utACK 00fade007c
  laanwj:
    Tested ACK 00fade007c

Tree-SHA512: 54f9f2b0fcf802e6a284214035fcb3833577aa5881beb293921b6036b71d0bcd0872f06d14ddc76a4f8c0eee7ba71461dcc99dc907e7ddb5ae403fc02b245538
2019-05-29 14:21:21 +02:00
fanquake 97dce72261
contrib: add curl as a required program in gitian-build.py 2019-05-28 12:51:53 -04:00
fanquake 00fade007c
contrib: use newer config.guess & config.sub in install_db4.sh 2019-05-24 15:48:34 -04:00
Hennadii Stepanov e0eae1b4a4
Make --setup command independent
A user can run 'gitian-build.py --setup' ignoring signer
and version options.
Get rid of warning about macOS build during setup for free.
2019-05-20 21:01:30 +03:00
MarcoFalke f49b8d4783
Merge #13998: Scripts and tools: gitian-build.py improvements and corrections
0f22a0cf2f Fix gitian-build.py --verify option (Hennadii Stepanov)
4c56a798c0 Set/unset USE_LXC, USE_VBOX, USE_DOCKER explicitly (Hennadii Stepanov)
cbbd98863b Fix Docker related issues for gitian-build.py (Hennadii Stepanov)

Pull request description:

  1. The Docker does not depend on `apt-cacher-ng` package. Ref: #14002.

  2. Do not try to install the Docker if `docker.service` is detected on the system (e.g., the Docker was installed manually). Fix https://github.com/bitcoin/bitcoin/pull/13623#issuecomment-405684241 by **Sjors**.

  3. Prevent the setting of more than one environment variable for the `gitian-builder` (an alternative to #13999). E.g., USE_LXC being set shadows USE_DOCKER; for details see [`gitian-builder/libexec/make-clean-vm`](93a62c7d7d/libexec/make-clean-vm (L7)):
  ```sh
  VMSW=KVM
  if [ -n "$USE_LXC" ]; then
      VMSW=LXC
  elif [ -n "$USE_VBOX" ]; then
      VMSW=VBOX
  elif [ -n "$USE_DOCKER" ]; then
      VMSW=DOCKER
  fi
  ```
  4. The [`gitian-builder/bin/gverify`](https://github.com/devrandom/gitian-builder/blob/master/bin/gverify) script returns the exit code 1 if a signature verification ends with 'BAD SIGNATURE' or 'MISMATCH' by design. This PR allows to see the verification results for all signatures without a premature fail of the `gitian-build.py` script. Ref: #14014.

ACKs for commit 0f22a0:

Tree-SHA512: 55f8a5cffa20d0c745f51a687f3199cea015fa616e56a0aee4c25b5ca0985036c61e8cf1922515338d8c6a85f873674ebe7a9a56a5069d65a187e383150f1a83
2019-05-20 13:01:37 -04:00
Wladimir J. van der Laan 149b3477ec
Merge #15840: Contrib scripts: Filter IPv6 by ASN
316b8b2339 Filter IPv6 by ASN (Emil)

Pull request description:

  Improves IP diversity for hardcoded seednodes.

ACKs for commit 316b8b:

Tree-SHA512: ae90427efa317d59125457bf8bfd077fd115c0921e1cc13cebd855206498546a026ccc18f039d1963d64d9be9497c41f4a21214fb565d5d11a9635ad12836421
2019-05-20 17:54:05 +02:00
Hennadii Stepanov 0f22a0cf2f
Fix gitian-build.py --verify option
The gitian-builder/bin/gverify script returns the exit code 1 if a
signature verification ends with 'BAD SIGNATURE' or 'MISMATCH' by
design. This commit allows to see the verification results for all
signatures without a premature fail of the gitian-build.py script.
2019-05-18 19:35:02 +03:00
Hennadii Stepanov 4c56a798c0
Set/unset USE_LXC, USE_VBOX, USE_DOCKER explicitly
This prevents the setting of more than one environment variable for the
gitian-builder (e.g., USE_LXC being set shadows USE_DOCKER; for details
see gitian-builder/libexec/make-clean-vm).
2019-05-18 19:34:31 +03:00
Hennadii Stepanov cbbd98863b
Fix Docker related issues for gitian-build.py
The Docker does not depend on apt-cacher-ng package.
Do not try to install the Docker if docker.service is detected on the
system (e.g., the Docker was installed manually).
Also small style corrections were applied.
2019-05-18 19:34:04 +03:00
MarcoFalke c89a63412e
Merge #15863: scripts and tools: Ensure repos are up-to-date in gitian-build.py
feed98e189 Ensure repos are up-to-date (Hennadii Stepanov)

Pull request description:

  These steps are provided by the [release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#setup-and-perform-gitian-builds).

ACKs for commit feed98:

Tree-SHA512: ad6876d211e524cf6b8dbe4f0f026b77792c8ae3b728e1419f17d5679766603d21c057a7866c183794c814b914a9e4584e16fc501bec77af7e3472a34bd4d913
2019-05-17 16:47:52 -04:00
MarcoFalke 87dbf89271
Merge #15239: scripts and tools: Move non-linux build source tarballs to "bitcoin-binaries/version" directory
5c04814b2d Move non-linux source tarball to bitcoin-binaries (Hennadii Stepanov)

Pull request description:

  Currently, if a user makes a non-linux (`--os=w`, `--os=m` or `--os=wm`) gitian building with the `gitian-build.py` script, source tarballs are not moved to the `bitcoin-binaries/${VERSION}` directory.

  This PR fixes this bug.

  ~~In addition, the `src` subdirectory in the `gitian-builder/build/out` directory is no longer used as unnecessary.~~

ACKs for commit 5c0481:
  fanquake:
    utACK 5c04814
  ken2812221:
    utACK 5c04814b2d

Tree-SHA512: 8648b6cbf502c012b12642783870e37aea385bd5f4cba5cb577fee924c09685e9a117676be502e4d4783c7a8ab31a2bd495970eec42a42d78e86ac5d39323091
2019-05-10 07:59:02 -04:00
Wladimir J. van der Laan 79046d5749
Merge #15939: gitian: Remove Windows 32 bit build
fa193dc8e6 doc: Remove win32 from the release process (MarcoFalke)
faf666f814 Remove Windows 32 bit build (MarcoFalke)

Pull request description:

  The Windows 32 bit build has been removed from https://bitcoincore.org/en/download/, so unless there are complaints, we don't need to build it even

ACKs for commit fa193d:
  fanquake:
    utACK fa193dc8e6

Tree-SHA512: d6f2976a2e0c407698f720b00ac23ec4056626de4eff8621f4c5581120af0460afd1bdef72329cc0e7d92afca48d94ae5fce6777cb36bfabb60b8034ff08fd88
2019-05-09 21:22:24 +02:00
Wladimir J. van der Laan a65fd92f7b
Merge #15766: scripts and tools: Upgrade gitian image before signing
beda0dae95 Upgrade gitian image before signing (Hennadii Stepanov)

Pull request description:

  The package upgrade in the Ubuntu repositories in the period between the building and the signing causes (particularly, using LXC) an error:
  ```
  Creating package manifest
  Could not download some packages, please run gbuild --upgrade
  ```

  For example, the [`busybox-initramfs`](https://packages.ubuntu.com/bionic/busybox-initramfs) package was [upgraded](http://changelogs.ubuntu.com/changelogs/pool/main/b/busybox/busybox_1.27.2-2ubuntu3.2/changelog) from `1:1.27.2-2ubuntu3.1` to `1:1.27.2-2ubuntu3.2` on 2019-03-06.

  This PR forces gitian image upgrade for the `--sign` command.

  Ref:
  [devrandom/gitian-builder/target-bin/grab-packages.sh](04ab7c1218/target-bin/grab-packages.sh)
  ```
  #!/bin/sh

  # Get an installed package manifest

  set -e

  cd /var/cache/apt/archives

  # make sure all packages with installed versions are downloaded
  # (except for held packages, which may not be available for download)
  dpkg-query -W -f '${Status}\t${Package}=${Version}\n' | grep -v ^hold | cut -f2- | xargs -n 50 apt-get install -q --reinstall -y -d > /tmp/download.log
  grep "cannot be downloaded" /tmp/download.log && { echo Could not download some packages, please run gbuild --upgrade 1>&2 ; exit 1 ; }
  sha256sum *.deb | sort --key 2
  ```

ACKs for commit beda0d:
  laanwj:
    utACK beda0dae95
  fanquake:
    utACK beda0da

Tree-SHA512: e2e3b3e3719e098d266ceec39bd69b950344a4eb2f43ae6ad3e696add70f743b363cc83676e339f7caa207d6478029869a8af01fe1f6d5690d2857003f7d8ce8
2019-05-09 17:29:44 +02:00
MarcoFalke fa193dc8e6
doc: Remove win32 from the release process 2019-05-08 11:48:18 -04:00
MarcoFalke faf666f814
Remove Windows 32 bit build 2019-05-03 13:41:27 -04:00
Hennadii Stepanov feed98e189
Ensure repos are up-to-date
These steps are provided by the release process.
2019-04-21 09:52:48 +03:00
nkostoulas 942ff2054b
contrib: gh-merge: Use pagination to fetch all review comments 2019-04-17 20:21:45 +01:00
Emil 316b8b2339 Filter IPv6 by ASN 2019-04-17 17:32:05 +00:00
Hennadii Stepanov beda0dae95
Upgrade gitian image before signing 2019-04-07 16:16:05 +03:00
MarcoFalke 904129b35d
Merge #15255: [tests] Remove travis_wait from lint script
8b8d8eeae9 Remove travis_wait from lint script (Graham Krizek)

Pull request description:

  Using the `travis_wait` command in conjunction with `set -o errexit` causes problems. The `travis_wait` command will correctly log the command's output if successful, but if the command fails the process exits before the `travis_wait` command can dump the logs. This will hide important debugging information like error messages and stack traces. We ran into this in #15196 and it was very hard to debug because output was being suppressed.

  `travis_wait` was being used because the `contrib/verify-commits/verify-commits.py` script can sometimes run for a long time without producing any output. If a script runs for 10 minutes without logging anything, the CI run times out. The `travis_wait` command will extend this timeout by logging a message for you, while sending stderr and stdout to a file.

  This PR removes the `travis_wait` command from our CI system and adds additional logging to the `verify-commits.py` script so it doesn't make Travis timeout.

ACKs for commit 8b8d8e:
  MarcoFalke:
    utACK 8b8d8eeae9

Tree-SHA512: 175a8dd3f4d4e03ab272ddba94fa8bb06875c9027c3f3f81577feda4bc8918b5f0e003a19027f04f8cf2d0b56c68633716a6ab23f95b910121a8d1132428767d
2019-03-29 11:23:08 -04:00
Cory Fields 43ae1e96ed release: Update the Windows Codesigning certificate 2019-03-28 00:07:38 -04:00
MarcoFalke fa1c073154
contrib: gh-merge: Include review comments in merge commit 2019-03-22 12:03:04 -04:00
Douglas Roark 204c729457
Add Gitian key for droark
I've signed 0.18.0rc1 and 0.18.0rc2, and I intend to sign every release moving forward.
2019-03-16 16:33:39 -07:00
Wladimir J. van der Laan cede01b416
Merge #15609: scripts and tools: Set 'distro' explicitly
b8705a0915 Set 'distro' explicitly (Hennadii Stepanov)

Pull request description:

  The [gitian-builder](https://github.com/devrandom/gitian-builder) implicitly uses `ubuntu` as a default distro.

  [bin/gbuild#L237](81edd2fc8e/bin/gbuild (L237)):
  ```ruby
  distro = build_desc["distro"] || "ubuntu"
  ```

  This PR sets a gitian building distro explicitly in description files.

Tree-SHA512: d2a692047f3466a5c637433610854d1100fe68a78fb03b4a81c70911fa14b0228d9cf25bcf115930aab9cc8c5063dacaf79bcd151f1c3f6fee6419389aefcb8b
2019-03-16 13:26:48 +01:00
Hennadii Stepanov b8705a0915
Set 'distro' explicitly 2019-03-16 07:40:50 +02:00
MarcoFalke b83c6f7940
Merge #15444: [docs] Additional productivity tips
ff7f31e07d [doc] productivity: more advanced git range-diff (Sjors Provoost)
3a21905a4e [doc] devtools: mention clang-format dependency (Sjors Provoost)
bf12093191 [doc] productivity: fix broken link (Sjors Provoost)

Pull request description:

  Fixes a broken link to `devtools/README.md`, points out the `clang-format` dependency and adds a `git range-diff` incantation that works even with rebases and squashes.

Tree-SHA512: 36e46282f1e28d1bf3f48ada995fbac548f61b7747091eb032b60919cf76c7bdad0fa8aecb0c47adbdaa9ef986d3ec7752b0bb94c63191401856e2ddeec48f3e
2019-03-14 17:02:28 -04:00
Wladimir J. van der Laan 12408d33c6
Merge #15549: gitian: Improve error handling
32da92bdf6 gitian: Improve error handling (Wladimir J. van der Laan)

Pull request description:

  Improve error handling in gitian builds:

  - Set fail-on-error and pipefail flag, this causes a command to fail when either of the pipe stages fails, not only when the last of the stages fails, so this improves error detection.
  - Also use `xargs` instead of `find -exec`, because `find` will not propagate errors in the executed command, but `xargs` will.

  This will avoid some issues like #15541 where non-determinism is silently introduced due to errors caused by environment conditions (such as lack of disk space in that case).

Tree-SHA512: d5d3f22ce2d04a75e5c25e935744327c3adc704c2d303133f2918113573a564dff3d3243d5569a2b93ee7eb0e97f8e1b1ba81767e966af9015ea711a14091035
2019-03-09 06:45:19 +01:00
MarcoFalke 923d87497c
Merge #15528: contrib: Bump gitian descriptors for 0.19
fa58a2e335 contrib: Bump gitian descriptors for 0.19 (MarcoFalke)

Pull request description:

  Bump the cache directory (as in 263b3777e7)

Tree-SHA512: 2f9a4f8c14c3c6cef30bcdfed638486c6b957068b1e2380cbde1719c9bf8b53bcaff35833272a879af5b031e3ea0a32cc16f78cc7e4d7399037de89132311c5b
2019-03-08 09:12:03 -05:00
Wladimir J. van der Laan 32da92bdf6 gitian: Improve error handling 2019-03-07 17:21:46 +01:00
MarcoFalke a74d588f21
Merge #14954: build: Require python 3.5
fa2797808e test: Remove python3.4 workaround in feature_dbcrash (MarcoFalke)
dddd1d05d3 .python-version: Specify full version 3.5.6 (MarcoFalke)
faa7cdf764 scripted-diff: Update copyright in ./test (MarcoFalke)
fa0e65b772 scripted-diff: test: Remove brackets after assert (MarcoFalke)
fab5a1e0f4 build: Require python 3.5 (MarcoFalke)
fa6bf21f5e scripted-diff: test: Use py3.5 bytes::hex() method (MarcoFalke)

Pull request description:

  Python 3.4 is EOL after March 2019, so switch to 3.5. See https://devguide.python.org/#status-of-python-branches

  This pull does the following in a bunch of commits:
  * scripted diff to use the `bytes::hex()` method in place of previous wrappers (`b2x`, `bytes_to_hex_str`, `hexlify`, ...)
  * Update the build system (gitian and travis) to remove python2.7 and replace it with python3.5
  * Another scripted-diff to remove brackets after `assert`. This is unrelated to the python3.5 switch, but a stylistic commit, so probably not worth to split up. The motivation behind it is to avoid asserting on data structures (such as tuples of length one), which never fails:
  ```py
  >>> assert(False,)   # with brackets
  >>> assert False,    # without brackets
  SyntaxError: invalid syntax
  >>> assert False     # proper assertion
  AssertionError
  ```
  * And then a final scripted diff to update the copyright headers in the `test` subfolder, since I touched most of the files anyway and it wouldn't make sense to split this commit out into a separate pull.

  For reference (contributed by luke-jr):

  Ubuntu LTS (bionic): 3.6.5
  Debian stable (stretch): 3.5.3
  RHEL 8 (expected before v0.19): 3.6.x
  Gentoo stable: 3.6.5
  Arch: 3.7.1

Tree-SHA512: 643c28cd2d5b9543ce4bf8ad2a8b282bc79b37dc5b25c9c8358e6ce201e2a67a546463e5f3430b16652eb2489d7c3ed4b0772cd2e2bf790fe68a5e3cc8a25029
2019-03-05 09:13:13 -05:00
MarcoFalke fa58a2e335
contrib: Bump gitian descriptors for 0.19 2019-03-04 14:44:23 -05:00
MarcoFalke 45f434f44d
Merge #15288: Remove wallet -> node global function calls
f7efd87c8f Change brace formatting (Russell Yanofsky)
a1df1b48a8 Remove use of IsInitialBlockDownload in wallet code (Russell Yanofsky)
1106a6fde4 Remove use of uiInterface.LoadWallet in wallet code (Russell Yanofsky)
318f41fb2c circular-dependencies: Avoid treating some .h/.cpp files as a unit (Russell Yanofsky)
d02b34c8a8 Remove use of AcceptToMemoryPool in wallet code (Russell Yanofsky)
e2c8ba9f6e Remove uses of InitMessage/Warning/Error in wallet code (Russell Yanofsky)
c5e59a96a8 Remove uses of GetAdjustedTime in wallet code (Russell Yanofsky)
6d6bcc77c0 Remove use of g_connman / PushInventory in wallet code (Russell Yanofsky)
00dfb2a440 Remove uses of g_connman in wallet code (Russell Yanofsky)
cc3836e8f9 Remove uses of fPruneMode in wallet code (Russell Yanofsky)
cc02c796d3 Remove uses of fee globals in wallet code (Russell Yanofsky)
1fb0a4a04e Remove use of CalculateMemPoolAncestors in wallet code (Russell Yanofsky)
cd32160af0 Remove use of GetTransactionAncestry in wallet code (Russell Yanofsky)
291276f7f4 Remove use of GetCountWithDescendants in wallet code (Russell Yanofsky)
bdc6628683 Remove use of IsRBFOptIn in wallet code (Russell Yanofsky)
80f52a2267 Remove uses of CheckFinalTx in wallet code (Russell Yanofsky)

Pull request description:

  This change removes wallet calls to node functions that access global chain and mempool state.

  This is the next step in the larger #10973 refactoring change, which removes all other accesses to node global variables from wallet code. Doing this is useful to provide a better defined interface between the wallet and node, and necessary to allow wallet and node code to run in separate processes in #10102.

Tree-SHA512: 40dbaf1f59fb22b32e70b054b30ba5638d638aa3240fa30e0f721d53c721cd6138a7ab4d423a24d7d2fda0b956e68d44c733abc2c9259c3d6c9fd6d4be89aa23
2019-03-04 13:13:36 -05:00
MarcoFalke fab5a1e0f4
build: Require python 3.5 2019-03-02 10:40:23 -05:00
MarcoFalke fa6bf21f5e
scripted-diff: test: Use py3.5 bytes::hex() method
-BEGIN VERIFY SCRIPT-
sed -i -e "s/def bytes_to_hex_str/def b_2_x/g" $(git grep -l bytes_to_hex_str)

export RE_B_0="[^()]*"                          # match no bracket
export RE_B_1="${RE_B_0}\(${RE_B_0}\)${RE_B_0}" # match exactly one ()
export RE_B_2="${RE_B_0}\(${RE_B_1}\)${RE_B_0}" # match wrapped (())

export RE_M="(b2x|bytes_to_hex_str)\(((${RE_B_0}|${RE_B_1}|${RE_B_2})*)\)"

sed -i --regexp-extended -e "s/${RE_M}/\2.hex()/g"      $(git grep -l -E '(b2x|bytes_to_hex_str)')

sed -i --regexp-extended -e "/  +bytes_to_hex_str( as b2x)?,/d"    $(git grep -l bytes_to_hex_str)
sed -i --regexp-extended -e "s/ +bytes_to_hex_str( as b2x)?,//g"   $(git grep -l bytes_to_hex_str)
sed -i --regexp-extended -e "s/, bytes_to_hex_str( as b2x)?//g"    $(git grep -l bytes_to_hex_str)

export RE_M="(binascii\.)?hexlify\(((${RE_B_0}|${RE_B_1}|${RE_B_2})*)\).decode\(${RE_B_0}\)"

sed -i --regexp-extended -e "s/${RE_M}/\2.hex()/g" $(git grep -l hexlify -- ':(exclude)share')

sed -i --regexp-extended -e  "/from binascii import hexlify$/d" $(git grep -l hexlify -- ':(exclude)share')
sed -i --regexp-extended -e "s/(from binascii import) .*hexlify/\1 unhexlify/g" $(git grep -l hexlify -- ':(exclude)share')

sed -i -e 's/ignore-names "/ignore-names "b_2_x,/g' ./test/lint/lint-python-dead-code.sh
-END VERIFY SCRIPT-
2019-03-02 10:40:12 -05:00