Commit graph

60 commits

Author SHA1 Message Date
fanquake
6c27fa0f09
scripts: remove github-merge.py
This script has been moved to the bitcoin-core/bitcoin-maintainer-tools repository,
after discussion in a core dev meeting. The rationale being that it is also useful to
other projects, and thus should be moved to a more general repository.

Meeting log: http://www.erisian.com.au/bitcoin-core-dev/log-2019-08-15.html
See also: https://github.com/bitcoin-core/bitcoin-maintainer-tools/pull/28
2019-08-19 19:16:57 +08:00
Hennadii Stepanov
1ac454a384
Enable ShellCheck rules
Enabled ShellCheck rules:
  SC1087
  SC2001
  SC2004
  SC2005
  SC2006
  SC2016
  SC2028
  SC2048
  SC2066 (note that IFS already contains only a line feed)
  SC2116
  SC2166
  SC2181
  SC2206
  SC2207
  SC2230
  SC2236
2019-07-04 19:35:25 +03:00
fanquake
8081927c33
scripts: add key for fanquake to trusted keys list 2019-06-07 09:48:12 +02:00
MarcoFalke
904129b35d
Merge #15255: [tests] Remove travis_wait from lint script
8b8d8eeae9 Remove travis_wait from lint script (Graham Krizek)

Pull request description:

  Using the `travis_wait` command in conjunction with `set -o errexit` causes problems. The `travis_wait` command will correctly log the command's output if successful, but if the command fails the process exits before the `travis_wait` command can dump the logs. This will hide important debugging information like error messages and stack traces. We ran into this in #15196 and it was very hard to debug because output was being suppressed.

  `travis_wait` was being used because the `contrib/verify-commits/verify-commits.py` script can sometimes run for a long time without producing any output. If a script runs for 10 minutes without logging anything, the CI run times out. The `travis_wait` command will extend this timeout by logging a message for you, while sending stderr and stdout to a file.

  This PR removes the `travis_wait` command from our CI system and adds additional logging to the `verify-commits.py` script so it doesn't make Travis timeout.

ACKs for commit 8b8d8e:
  MarcoFalke:
    utACK 8b8d8eeae9

Tree-SHA512: 175a8dd3f4d4e03ab272ddba94fa8bb06875c9027c3f3f81577feda4bc8918b5f0e003a19027f04f8cf2d0b56c68633716a6ab23f95b910121a8d1132428767d
2019-03-29 11:23:08 -04:00
Graham Krizek
8b8d8eeae9 Remove travis_wait from lint script
Also adding progress logging to verify-commits.py script to prevent Travis from timing out
2019-02-21 11:11:02 -06:00
Douglas Roark
a786c3b306
Slight tweak to the verify-commits script directions
Clarify that GnuPG may be used on both Linux and macOS to obtain the keys required to verify the commits.
2019-01-31 10:15:52 -08:00
Graham Krizek
fdf82ba181
Update all subprocess.check_output functions in CI scripts to be Python 3.4 compatible
Removing the 'universal_newlines' and 'encoding' args from the subprocess.check_outputs fuction. 'universal_newlines' is supported in 3.4, but 'encoding' is not. Without specifying 'encoding' it will make a guess at encoding, which can break things on BSD systems. We must handle encoding/decoding ourselves until we can use Python 3.6
2019-01-18 09:36:39 -06:00
MeshCollider
e946fc7eb1
Merge #14809: Tools: improve verify-commits.py script
45842c3d2 Improve documentation for running verify-commits.py script (Jameson Lopp)

Pull request description:

  I ran into 3 different issues while trying to run the verify-commits script for the first time and I think documenting them would help save time for future developers.

  1. I was trying to just run it with "python" and didn't realize I had multiple python versions installed and this script is only syntactically valid for python 3.x.
  2. I needed to import the trusted keys
  3. The script was hanging because it was triggering my yubikey for signature verification

Tree-SHA512: dfc7a62972ca3de528fae3c9d420c7d2d6658767a555ebbf5f4a27c04748c35ccf8bf63bfc9f264358346de0db49bfbaf2d1540793a609d81c2d9b622ee8182c
2018-12-10 17:18:38 +13:00
Jameson Lopp
45842c3d26
Improve documentation for running verify-commits.py script 2018-12-06 10:41:08 -05:00
Wladimir J. van der Laan
1ca0502541
scripts: Add trusted key for Samuel Dobson
Add trusted GPG key so that Samuel Dobson (MeshCollider), who is now
wallet maintainer, can merge PRs.
2018-12-04 13:51:53 +01:00
Chun Kuan Lee
5d62dcf9cf lint: Make sure we read the command line inputs using utf-8 decoding in python 2018-09-02 21:40:51 +08:00
DesWurstes
000000035b Obsolete #!/bin/bash shebang 2018-06-20 11:12:41 +03:00
Wladimir J. van der Laan
45c00f8416
Merge #13454: Make sure LC_ALL=C is set in all shell scripts
47776a958b Add linter: Make sure all shell scripts opt out of locale dependence using "export LC_ALL=C" (practicalswift)
3352da8da1 Add "export LC_ALL=C" to all shell scripts (practicalswift)

Pull request description:

  ~~Make sure `LC_ALL=C` is set when using `grep` range expressions.~~

  Make sure `LC_ALL=C` is set in all shell scripts.

  From the `grep(1)` documentation:

  > Within a bracket expression, a range expression consists of two characters separated by a hyphen. It matches any single character that sorts between the two characters, inclusive, using the locale's collating sequence and character set. For example, in the default C locale, `[a-d]` is equivalent to `[abcd]`. Many  locales sort characters in dictionary order, and in these locales `[a-d]` is typically not equivalent to `[abcd]`; it might be equivalent to `[aBbCcDd]`, for example. To obtain the traditional interpretation of bracket expressions, you can use the C locale by setting the `LC_ALL` environment variable to the value C.

  Context: [Locale issue found when reviewing #13450](https://github.com/bitcoin/bitcoin/pull/13450/files#r194877736)

Tree-SHA512: fd74d2612998f9b49ef9be24410e505d8c842716f84d085157fc7f9799d40e8a7b4969de783afcf99b7fae4f91bbb4559651f7dd6578a6a081a50bdea29f0909
2018-06-18 13:18:12 +02:00
Wladimir J. van der Laan
a90ca4087a
Merge #13448: Add linter: Make sure we explicitly open all text files using UTF-8 encoding in Python
c8176b3cc7 Add linter: Make sure we explicitly open all text files using UTF-8 or ASCII encoding in Python (practicalswift)
634bd97001 Explicitly specify encoding when opening text files in Python code (practicalswift)

Pull request description:

  Add linter: Make sure we explicitly open all text files using UTF-8 encoding in Python.

  As requested by @laanwj in #13440.

Tree-SHA512: 1651c00fe220ceb273324abd6703aee504029b96c7ef0e3029145901762c733c9b9d24927da281394fd4681a5bff774336c04eed01fafea997bb32192c334c06
2018-06-16 15:23:14 +02:00
Loganaden Velvindron
9e2e5626da Fix CVE-2018-12356 by hardening the regex. 2018-06-15 21:45:32 +04:00
practicalswift
3352da8da1 Add "export LC_ALL=C" to all shell scripts 2018-06-14 15:27:52 +02:00
practicalswift
634bd97001 Explicitly specify encoding when opening text files in Python code 2018-06-12 21:34:52 +02:00
Chun Kuan Lee
e5b2cd8e75 Use python instead of slow shell script on verify-commits 2018-06-12 14:48:02 +00:00
Wladimir J. van der Laan
b2e5fe8b55
Merge #12708: Make verify-commits.sh test that merges are clean
577f111 Make verify-commits.sh test that merges are clean (Pieter Wuille)

Pull request description:

  Unsure if we want this.

  This modifies verify-commits.sh to redo all merges along the leftmost commit branch (which includes all PR merges), and verify whether they match the merge commit's trees.

  The benefit is that it will detect a case where one of the maintainers merges a PR, but makes an unrelated change inside the merge commit. This on itself is not very strong, as unrelated changes can also be included in the merged branch itself - but perhaps the merge commit is not something that people are otherwise likely to look at.

  Fixes #8089

Tree-SHA512: 2c020f5ac3f771ac775aa726832916bb8e03a311b2745d7a9825047239bd0660d838f086f3456f2bb05cea14c1529f74436b8cdd74cc94b70e40b4617309f62c
2018-04-07 18:48:27 +02:00
Matt Corallo
94715767a3 [verify-commits] Add some additional useful documentation. 2018-03-29 10:31:56 -04:00
Matt Corallo
de7e93138a Add Marco-expired-key-signed-commits to allow-revsig-commits 2018-03-28 14:55:50 -04:00
Matt Corallo
99f6d48e23 Revert "test: Update trust git root".
This reverts commit 7deba93bdc.

This is neither a "test" change, nor should the trusted-git-root
have been updated - there is a process for expired PGP keys.
2018-03-28 14:55:50 -04:00
Pieter Wuille
577f11141c Make verify-commits.sh test that merges are clean 2018-03-16 10:24:59 -07:00
Wladimir J. van der Laan
7deba93bdc
test: Update trust git root
Marco Falke's old key expired, causing a travis error while verifying
commits 36afd4db44 and before:

    gpg: Good signature from "Marco Falke <marco.falke@tum.de>" [unknown]
    gpg:                 aka "Marco Falke <falke.marco@gmail.com>" [unknown]
    gpg: Note: This key has expired!
    Primary key fingerprint: B8B3 F1C0 E58C 15DB 6A81  D30C 3648 A882 F431 6B9B
      Subkey fingerprint: FE09 B823 E6D8 3A3B C798  3EAA 2D7F 2372 E50F E137

Update the trusted root commit to the commit after that, to fix
this issue.

Tree-SHA512: 41e5913728099b131f73f8b4621cf6474d8914b2ffd524be8bac356426820f58016cc427fb32d043367688c8dbb60c26a7e34756589b61d0ba4ca3f8529a300f
2018-03-06 15:03:00 +01:00
Matt Corallo
a38686cea0 [verify-commits] Fix gpg.sh's echoing for commits with '\n' 2017-12-05 15:21:23 -05:00
Pieter Wuille
c17f11f7b4
Merge #10773: Shell script cleanups
13a81b19d Add quotes to variable assignment (as requested by @TheBlueMatt) (practicalswift)
683b9d280 Fix valid path output (practicalswift)
193c2fb4c Use bash instead of POSIX sh. POSIX sh does not support arrays. (practicalswift)
80f5f28d3 Fix incorrect quoting of quotes (the previous quotes had no effect beyond unquoting) (practicalswift)
564a172df Add required space to [[ -n "$1" ]] (previously [[ -n"$1" ]]) (practicalswift)
1e44ae0e1 Add error handling: exit if cd fails (practicalswift)
b9e79ab41 Remove "\n" from echo argument. echo does not support escape sequences. (practicalswift)
f6b3382fa Remove unused variables (practicalswift)

Pull request description:

  Shell script cleanups:
  * Add required space to `[ -n ]`.
  * Avoid quote within quote.
  * Exit if `cd` fails.
  * Remove `\n` which is not handled by `echo`.
  * ~~Remove redundant `$` in arithmetic variable expression.~~
  * ~~Use `$(command)` instead of legacy form `` `command` ``.~~
  * Arrays are not supported in POSIX `sh`. Use `bash` when arrays are used.
  * ~~`[ foo -a bar ]` is not well defined, use `[ foo ] && [ bar ]` instead.~~
  * ~~`[ foo -o bar ]` is not well defined, use `[ foo ] || [ bar ]` instead.~~

Tree-SHA512: 80f6ded58bce625b15b4da30d69d2714c633e184e62b21ed67d2c58e2ebaa08b4147593324012694d02bf4f1f252844cdff2fd1cf5e817ddb07e2777db7a6390
2017-12-04 15:52:11 -08:00
Wladimir J. van der Laan
23e9074e0a
Merge #10771: Remove unused variables in shell scripts
ab8e8b9 Remove unused variables in shell scripts. (practicalswift)

Pull request description:

  Remove unused variables in shell scripts. Use `_` where we don't care about the result.

Tree-SHA512: 35049e79ee432c805f061456c32902a92811b5214d50ce6770b22d1442cc5999ed53cfe05bb2347f6995ca33c707a0f3fe92d5829c0385c4a3e254953924cbc4
2017-11-09 21:20:20 +01:00
Matt Corallo
d23be309c2 [verify-commits] Allow revoked keys to expire 2017-10-20 16:35:16 -04:00
practicalswift
13a81b19df Add quotes to variable assignment (as requested by @TheBlueMatt) 2017-10-18 17:10:23 +02:00
practicalswift
683b9d280b Fix valid path output 2017-10-18 17:10:23 +02:00
practicalswift
80f5f28d38 Fix incorrect quoting of quotes (the previous quotes had no effect beyond unquoting) 2017-10-18 17:10:23 +02:00
practicalswift
f6b3382fa3 Remove unused variables 2017-10-18 17:09:36 +02:00
practicalswift
0aacfa43c1 Remove accidental stray semicolon 2017-10-09 16:23:08 +02:00
practicalswift
ab8e8b97a3 Remove unused variables in shell scripts. 2017-08-09 10:45:23 +02:00
Matt Corallo
df5bae2e7b Update trusted-sha512-root-commit for new bad tree hash 2017-03-09 09:53:19 -05:00
Matt Corallo
efc06c2c46 If GNU sha512sum is missing, try perl shasum in verify-commits 2017-03-09 09:53:19 -05:00
Matt Corallo
fd5e905e21 Make verify-commits.sh non-recursive 2017-03-09 09:53:19 -05:00
Matt Corallo
b3ec305f8e Fix bashisms in verify-commits and always check top commit's tree 2017-03-06 16:59:08 -05:00
Matt Corallo
f20e664f40 Check gpg version before setting --weak-digest 2017-03-06 16:59:07 -05:00
Matt Corallo
bbd757940b Fix regsig checking for subkey sigs in verify-commits 2017-03-05 11:21:52 -05:00
Matt Corallo
d025bc7964 Allow any subkey in verify-commits 2017-03-04 09:41:16 -05:00
Peter Todd
eddc77a1b1 Add comment re: why SHA1 is disabled 2017-03-04 09:41:16 -05:00
Matt Corallo
d9c450ffb2 Verify Tree-SHA512s in merge commits, enforce sigs are not SHA1 2017-03-04 09:41:15 -05:00
Matt Corallo
a4b02f4275 Add Pieter's old signed commits to revsig-commits 2017-02-27 20:24:20 -05:00
Matt Corallo
3e900acafa Require merge commits merge branches on top of other merge commits
Specifically, require that the left branch (first restult of git
show -s --format=format:%P) is a signed merge commit, instead of
allowing either. This is fine for now, but might need to be relaxed
in the future.

Also fixes an out-of-file-descriptors issue by holding too many
open FDs writing to /dev/null
2017-02-01 18:22:27 -05:00
isle2983
0766d1cac3 [copyright] add MIT license headers to .sh scripts where missing
Years are set according to 'git log' history
2016-09-11 13:36:22 -06:00
Peter Todd
1e9aab0dbf
Remove sipa's old revoked key from verify-commits
Now that the trusted root is past all commits signed by that key we don't need
it in the trusted-keys list, nor do we need to whitelist those commits in
allow-revsig-commits
2016-06-18 20:53:17 -04:00
Peter Todd
966151e71d
Add README for verify-commits 2016-06-09 13:58:29 -04:00
Peter Todd
11164ec0b4
Remove keys that are no longer used for merging
Also updated trusted git root to be right after gmaxwell's last merge.
2016-05-21 11:29:01 +02:00
Peter Todd
22421faa19 Remove pointless warning
Any attacker who managed to make an evil commit that changed something in the
contrib/verify-commits/ directory could just as easily remove the warning
and/or modify it to not display the evil commits; telling the user to check
those commits specifically misleads them into checking just those commits
rather than the script itself.
2016-05-21 11:26:21 +02:00