Commit graph

118 commits

Author SHA1 Message Date
MarcoFalke fa40e48c50
ci: Remove unparseable lines from supp file for old xenial clang tsan 2019-12-04 12:16:52 -05:00
MarcoFalke fa1bfc476c
ci: ubsan report_error_type=1 and add suppressions 2019-12-04 12:16:38 -05:00
MarcoFalke faa8023ce9
ci: Bump to clang-8 for asan build to avoid segfaults on ppc64le 2019-12-04 09:32:15 -05:00
MarcoFalke 1705f1949a
Merge #17549: ci: misc cleanups
fad88e6f86 ci: Remove use of cd (MarcoFalke)
fa2941bbf4 ci: Remove unmaintained extended_lint (MarcoFalke)
fa04187528 scripted-diff: Use ci DEPENDS_DIR, remove BASE_BUILD_DIR (MarcoFalke)
fa0656d1eb ci: Add DEPENDS_DIR variable, Add documentation for folders (MarcoFalke)
faeeca87b6 scripted-diff: Move various folders to ci scratch dir (MarcoFalke)

Pull request description:

  Some minor cleanups:

  * Remove unused and unmaintained extended_lint to avoid ci bottlenecks and waste of CPU
  * Move all folders that hold temporary ci files to the ci scratch dir (except for the build dirs)
  * Add some documentation to folders and remove the `BASE_BUILD_DIR` alias for the root directory
  * Fixes #17178 by removing `cd`

Top commit has no ACKs.

Tree-SHA512: f6eb9d47ab98c08af59a79c2a4bf62fc331f0f6f3174909a28f2c2f48b0234423da5aee876225d2e188619b71b008d882a20d29a7aca68248039ea5080be5af5
2019-12-03 11:02:06 -05:00
MarcoFalke 69ca6cdd66
Merge #17599: ci: Run functional tests on s390x
fabd71076c ci: Print free disk space (MarcoFalke)
fad9fdbea5 test: Properly deserialize integers in little-endian (MarcoFalke)
fa94fc10c8 ci: Run functional tests on s390x (MarcoFalke)

Pull request description:

Top commit has no ACKs.

Tree-SHA512: 98ba77eb56f283131fdaeb393fda86cc308f1bf9781e1e0e5736b8d616528dc8ff2e494d55ba107c138083025c66a59e382fcfa9962d4349a5fd6cbbc52484c3
2019-12-03 10:34:11 -05:00
Wladimir J. van der Laan 1f59885d27
Merge #17361: script: Lint Gitian descriptors with ShellCheck
17f81e9648 script: Enable SC2001 rule for Gitian scripts (Hennadii Stepanov)
61bb21b418 script: Enable SC2155 rule for Gitian scripts (Hennadii Stepanov)
577682d9e8 script: Enable SC2006 rule for Gitian scripts (Hennadii Stepanov)
14aded46df script: Lint Gitian descriptors with ShellCheck (Hennadii Stepanov)

Pull request description:

  This PR extracts shell scripts from Gitian descriptors (`contrib/gitian-descriptors/`) and checks for ShellCheck warnings as any other one.

  Some non-controversial warnings are fixed.

ACKs for top commit:
  practicalswift:
    ACK 17f81e9648 -- diff looks correct

Tree-SHA512: bdfa3d35bbb65ff634c90835d75c3df63e958b558599771d21366724f5cf64da83a68957d926e926a99c3704b9529e96a17697dc8d9ff3adf7154d9cb1999a8d
2019-11-28 10:18:22 +01:00
MarcoFalke fabd71076c
ci: Print free disk space 2019-11-25 16:23:36 -05:00
MarcoFalke fa94fc10c8
ci: Run functional tests on s390x 2019-11-25 15:07:28 -05:00
Elichai Turkel 2fa65e0de9
Add ci script to install on s390x 2019-11-25 14:34:47 +02:00
MarcoFalke fad88e6f86
ci: Remove use of cd 2019-11-22 13:57:55 -05:00
fanquake 7d7bf2ff4a
build: set minimum supported macOS to 10.12 2019-11-21 11:08:47 -05:00
MarcoFalke fa2941bbf4
ci: Remove unmaintained extended_lint 2019-11-21 11:03:05 -05:00
MarcoFalke fa04187528
scripted-diff: Use ci DEPENDS_DIR, remove BASE_BUILD_DIR
-BEGIN VERIFY SCRIPT-
 # Use ci DEPENDS_DIR
 sed -i -e 's|BASE_BUILD_DIR/depends|DEPENDS_DIR|g' $(git grep -l depends ci)
 sed -i -e 's| depends/| ${DEPENDS_DIR}/|g'         $(git grep -l depends ci)

 # Remove redundant alias
 sed -i -e 's|BASE_BUILD_DIR|BASE_ROOT_DIR|g'       $(git grep -l BASE_BUILD_DIR ci)
-END VERIFY SCRIPT-
2019-11-21 11:01:34 -05:00
MarcoFalke fa0656d1eb
ci: Add DEPENDS_DIR variable, Add documentation for folders 2019-11-21 11:01:17 -05:00
MarcoFalke faeeca87b6
scripted-diff: Move various folders to ci scratch dir
-BEGIN VERIFY SCRIPT-
 # move ci sanitizer-output
 sed -i -e 's|BASE_BUILD_DIR}/sanitizer-output|BASE_SCRATCH_DIR}/sanitizer-output|g' $(git grep -l BASE_BUILD_DIR ci)
 # move qa-assets
 sed -i -e 's|BASE_BUILD_DIR}/qa-assets|BASE_SCRATCH_DIR}/qa-assets|g'               $(git grep -l BASE_BUILD_DIR ci)
 # move out dir
 sed -i -e 's|BASE_BUILD_DIR/out|BASE_SCRATCH_DIR/out|g'                             $(git grep -l BASE_BUILD_DIR ci)
-END VERIFY SCRIPT-
2019-11-20 19:56:26 -05:00
MarcoFalke 41162c0ba0
Merge #17423: ci: Make ci system read-only on the git work tree
fa7523d3aa ci: Extend docs (MarcoFalke)
fa493ef088 ci: Make ci system read-only on the git work tree (MarcoFalke)
fab1333292 ci: Remove git from required packages on host (MarcoFalke)
fa00393bce ci: Make all filesystem operations inside docker (MarcoFalke)

Pull request description:

  Running the ci completely in a docker, without leaving any traces on the host system is not possible right now because the ccache and depends dir needs to be propagated back and picked up by the host for caching.

  Fixes #17372

ACKs for top commit:
  JeremyRubin:
    tested ACK fa7523d3aa

Tree-SHA512: 4bce1a0f883bcbdb34abf409bdbc80d420c5da2045d2f9c5536ac433f9e5b490f23df084546c8c049f688b487572bbfc4f9c4029e9e672f4d9279739d066ed2e
2019-11-20 16:41:55 -05:00
fanquake 76e777df83
Merge #16161: util: Fix compilation errors in support/lockedpool.cpp
30fb598737 Fix segfault in allocator_tests/arena_tests (Jeffrey Czyz)
15c84f53f4 Define ARENA_DEBUG in Travis test runs (Jeffrey Czyz)
ad71548822 Fix compilation errors in support/lockedpool.cpp (Jeffrey Czyz)

Pull request description:

  Changes in #12048 cause a compilation error in Arena::walk() when
  ARENA_DEBUG is defined. Specifically, Arena's chunks_free map was
  changed to have a different value type.

  Additionally, missing includes cause other compilation errors when
  ARENA_DEBUG is defined.

  Reproduced with:

  make CPPFLAGS=-DARENA_DEBUG

ACKs for top commit:
  laanwj:
    ACK 30fb598737
  fanquake:
    ACK 30fb598737 - thanks for following up jkczyz.

Tree-SHA512: 4eec368a4e9c67e4e2a27bc05608a807c2892d50c60d06ed21490cd274c0369f9671bc05b3006acc2a193316caf4896454c9c299603bfed29bd488f1987ec446
2019-11-20 09:54:31 -05:00
Wladimir J. van der Laan ea3c7e585c test: Remove libssl-dev packages from CI scripts 2019-11-19 15:04:43 +01:00
fanquake 397dbae070
ci: remove OpenSSL installation 2019-11-18 08:56:48 -05:00
Jeffrey Czyz 15c84f53f4 Define ARENA_DEBUG in Travis test runs
The definition and uses of Arena::walk() are compiled only if
ARENA_DEBUG is defined. Configure Travis to define ARENA_DEBUG so
compilation errors do not go unnoticed.
2019-11-16 08:45:21 -08:00
MarcoFalke fa6e01b2f3
ci: Use clang-8 for fuzzing to run on aarch64 ci systems 2019-11-13 17:53:04 -05:00
MarcoFalke fa7523d3aa
ci: Extend docs 2019-11-10 22:30:13 -05:00
MarcoFalke fa493ef088
ci: Make ci system read-only on the git work tree 2019-11-09 09:13:41 -05:00
MarcoFalke fab1333292
ci: Remove git from required packages on host 2019-11-09 09:12:24 -05:00
MarcoFalke fa00393bce
ci: Make all filesystem operations inside docker 2019-11-09 09:11:32 -05:00
MarcoFalke faf757a125
ci: Guess the native host when not cross compiling 2019-11-07 13:42:57 -05:00
Hennadii Stepanov 14aded46df
script: Lint Gitian descriptors with ShellCheck 2019-11-06 15:10:11 +02:00
MarcoFalke fa8a60bce9
ci: Run non-cross-compile builds natively 2019-11-04 14:10:24 -05:00
MarcoFalke fa56bcbb01
ci: Run CI_WAIT only on travis 2019-11-04 14:10:07 -05:00
MarcoFalke def69e5a86
Merge #17233: travis: Run unit and functional tests on native arm
facc0da63a travis: Run unit and functional tests on native arm (MarcoFalke)
fafa064d2a ci: Remove ccache requirement on the host (MarcoFalke)

Pull request description:

  This keeps the cross-compilation to make it easy to run the ci on non-arm hardware. To run this locally in qemu-user as it used to be, just `export QEMU_USER_CMD="qemu-arm -L /usr/arm-linux-gnueabihf/"`.

ACKs for top commit:
  laanwj:
    LGTM ACK facc0da63a
  practicalswift:
    ACK facc0da63a -- diff looks correct and Travis seems happy

Tree-SHA512: 0dc1bc82eb93e2bd8b159e044f20fe3055f8cdfd73aaa238bd2e178397582144dfc0c6a87bd8270115dafea1a623e642bde5d5f30254f94140f1a2cdb12fc2da
2019-11-04 08:23:21 -05:00
randymcmillann 58d0393bec
build: update retry to current version
from: 7d2caa94a7
2019-10-30 18:49:57 -04:00
MarcoFalke facc0da63a
travis: Run unit and functional tests on native arm 2019-10-30 14:45:32 -04:00
MarcoFalke fafa064d2a
ci: Remove ccache requirement on the host
ccache is only needed to create the cache dir on the host, if it didn't
already exist. The same can be achieved with mkdir, so just use that
instead.
2019-10-24 18:37:38 -04:00
fanquake 3548e4aac7
Remove BIP70 Support 2019-10-24 16:01:43 -04:00
fanquake 67328bb7ca
build: remove protobuf from depends 2019-10-24 16:01:42 -04:00
MarcoFalke fa710066b9
ci: Disable functional tests on mac host 2019-10-24 12:43:55 -04:00
MarcoFalke 4aaeb04497
Merge #17205: ci: Enable address sanitizer (ASan) stack-use-after-return checking
8d22ab0e50 ci: Enable address sanitizer (ASan) stack-use-after-return checking (practicalswift)

Pull request description:

  Enable address sanitizer (ASan) stack-use-after-return checking (`detect_stack_use_after_return=1`).

  Example:

  ```
  #include <iostream>
  #include <string>

  const std::string& get_string(int i) {
      return std::to_string(i);
  }

  int main() {
      std::cout << get_string(41) << "\n";
  }
  ```

  Without address sanitizer (ASan) stack-use-after-return checking:

  ```
  $ ./stack-use-after-return

  $
  ```

  With address sanitizer (ASan) stack-use-after-return checking:

  ```
  $ ASAN_OPTIONS="detect_stack_use_after_return=1" ./stack-use-after-return
  =================================================================
  ==10400==ERROR: AddressSanitizer: stack-use-after-return on address 0x7f7fa0400030 at pc 0x00000049d2cc bp 0x7ffcbd617070 sp 0x7ffcbd616820
  READ of size 2 at 0x7f7abbecd030 thread T0
      #0 0x439781 in fwrite
      #1 0x7f7ac0504cb3 in std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, long) (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0x113cb3)
      #2 0x4f9b5f in main stack-use-after-return.cpp:9:15
      #3 0x7f7abf440b96 in __libc_start_main
      #4 0x41bbc9 in _start
  …
  $
  ```

Top commit has no ACKs.

Tree-SHA512: 6557a9ff184023380fd9aa433cdf413e01a928ea99dbc59ec138e5d69cb9e13592e8bb5951612f231ff17a37a895bec5c0940c8db5f328a5c840a5771bdeeba5
2019-10-22 13:14:16 -04:00
practicalswift 8d22ab0e50 ci: Enable address sanitizer (ASan) stack-use-after-return checking 2019-10-21 10:04:28 +00:00
MarcoFalke fa677d1801
ci: Remove redundant check for TRAVIS_OS_NAME
Can be reviewed with
git diff --ignore-all-space --function-context
2019-10-17 11:30:36 -04:00
MarcoFalke 4444704ca9
ci: Cleanup macOS runs 2019-10-17 10:47:02 -04:00
Sjors Provoost 1f6c650c99
travis: run tests on macOS native
Review hint:
git show -w

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
Co-authored-by: keneanung <keneanung@googlemail.com>
Co-authored-by: Vadim Peretokin <vperetokin@gmail.com>
2019-10-14 23:00:49 +02:00
MarcoFalke 42d0eca725
Merge #13728: lint: Run the CI lint stage on mac
cd82f75a43 lint: Install grep and git via brew on mac for --perl-regexp (Ben Woosley)
eafa747ca5 lint: Fix shellcheck SC2155 (Ben Woosley)
615ff4e7db lint: Run the linters against Mac OS on Travis (Ben Woosley)

Pull request description:

  This helps ensure ongoing compatibility with macOS-distributed version of GNU bash.

ACKs for top commit:
  MarcoFalke:
    ACK cd82f75a43

Tree-SHA512: 8d56d2303bbebedba8ea2291f4ab35b7fdf3245b7a4c3f04557eee4f19d83573798ad32facc92bfa060aaeb294e6d2c95e6d1c3b795fd7951dcf3aa1cccec107
2019-10-09 14:37:53 -04:00
Ben Woosley cd82f75a43
lint: Install grep and git via brew on mac for --perl-regexp
Particularly `--with-pcre2` is needed to run `git grep --perl-regexp` in
`test/link/check-doc.py`
2019-10-09 15:50:42 +01:00
Ben Woosley eafa747ca5
lint: Fix shellcheck SC2155
Declare and assign separately to avoid masking return values.

https://github.com/koalaman/shellcheck/wiki/SC2155
2019-10-09 15:50:42 +01:00
Ben Woosley 615ff4e7db
lint: Run the linters against Mac OS on Travis
This helps ensure ongoing compatibility with macOS-distributed version
of GNU bash.
2019-10-09 15:49:51 +01:00
MarcoFalke fa79dff624
ci: Run tests on arm 2019-10-09 09:33:07 -04:00
Wladimir J. van der Laan 34919e0033
Merge #17011: ci: Use busybox utils for one build
ddddd8961b ci: Use busybox utils for one build (MarcoFalke)

Pull request description:

  To make sure Bitcoin Core can be built with BusyBox, see https://github.com/bitcoin/bitcoin/pull/16927#issuecomment-536483706

ACKs for top commit:
  laanwj:
    ACK ddddd8961b

Tree-SHA512: da3a4654ee7975206d04643675d309b4973a510ca344acaec97fb1ed19c43cf13489bdf236c92c4a90499ec5b3c18c3338fff096110b26abee5ffe955089f267
2019-10-08 11:31:48 +02:00
MarcoFalke fa95503d64
ci: Make apt-get more verbose, to debug travis timeouts 2019-10-03 10:50:33 -04:00
MarcoFalke ddddd8961b
ci: Use busybox utils for one build 2019-10-02 10:01:04 -04:00
practicalswift aa81e2cc0e Enable UBSan for Travis fuzzer job 2019-10-01 08:23:34 +00:00
MarcoFalke 4c4ff4911a
Merge #16961: test: Remove python dead code linter
f4beb4996d test: Remove python dead code linter (Wladimir J. van der Laan)

Pull request description:

  Primarily I'd like to remove this because it is very imprecise, due to Python's dynamic nature, giving it a large list of false positives that need to be listed as exceptions. See for example #16906.

  It's also a frequent source of complaints. I'm doubtful of the usefulness of checking for dead code in a linter in the first place.
  Having some dead code in the test framework for a while is not a
  disaster.

ACKs for top commit:
  sdaftuar:
    utACK f4beb4996d
  practicalswift:
    ACK f4beb4996d -- diff looks correct
  jamesob:
    ACK f4beb4996d

Tree-SHA512: 329b1555210311d5d15799fd2cb794b3208b0ac4d8a2ffaf4dece1bcc3e0e8b1fe952d5e7a394f94a98919cab579fb579eae7db2a796cc9a1a42ef495dd17507
2019-09-25 13:14:22 -04:00
Wladimir J. van der Laan f4beb4996d test: Remove python dead code linter
Primarily I'd like to remove this because it is very imprecise, due to
Python's dynamic nature, giving it a large list of false positives that
need to be listed as exceptions. See for example #16906.

It's also a frequent source of complaints. I'm doubtful of the
usefulness of checking for dead code in a linter in the first place.
Having some dead code in the test framework for a while is not a
disaster.
2019-09-25 11:16:09 +02:00
MarcoFalke fadd76acc2
ci: Remove TRAVIS env vars 2019-09-24 16:43:44 -04:00
MarcoFalke fa449b89b5
ci: Set $HOST before setting fallback values 2019-09-24 16:21:49 -04:00
MarcoFalke fac35b21e2
test: lint: Add DisabledOpcodeTemplates to whitelist
Also, bump vulture version to include the whitelist for threading module
2019-09-18 11:38:37 -04:00
fanquake 107e030723
build: make protobuf optional in depends
Those that want to build it can now pass PROTOBUF=1.
2019-09-14 12:58:59 +08:00
fanquake 376f4929f8
build: disable BIP70 support by default 2019-09-12 20:28:17 +10:00
Ben Woosley 0ef0e51fe4
lint: Bump flake8 to 3.7.8 2019-09-03 14:41:43 -04:00
MarcoFalke fa27372e65
ci: Move CCACHE_DIR and test_runner tmp dir into ./ci/scratch/ 2019-08-27 15:58:50 -04:00
MarcoFalke fa60583d23
ci: Pass down $MAKEJOBS to test_runner.py 2019-08-27 15:58:49 -04:00
MarcoFalke 8b42db130f
Merge #16623: ci: Add environment files for all settings
fa21737ba7 ci: Add environment files for all settings (MarcoFalke)

Pull request description:

  This moves all environment settings from travis to files in the ci folder. Now, it is possible to easily run each travis configuration with a single command.

Top commit has no ACKs.

Tree-SHA512: 989c6b62eb3839eb1fa5461e986496e9660167e2438a789c7588a6fee4f9c37b332782c010fe5c7de8f606bcf98dffb2481d2777cbce88f87cc9f0c42fb2d7fc
2019-08-19 09:39:53 -04:00
MarcoFalke faba46da07
ci: Set --ansi in test_runner 2019-08-16 08:50:01 -04:00
MarcoFalke fa21737ba7
ci: Add environment files for all settings 2019-08-16 08:46:37 -04:00
MarcoFalke faeacf3269
ci: Add note that this assumes a fresh git clone 2019-08-15 11:13:10 -04:00
MarcoFalke fa6cbdc3c9
ci: Use ./ci/ on non-travis host 2019-08-15 11:13:09 -04:00
MarcoFalke fa31bc35eb
ci: Remove dependence on travis, use it as fallback env 2019-08-15 11:12:34 -04:00
MarcoFalke fa0aac0f43
ci: Add retry
From: a1b1826bdb
2019-08-15 11:12:22 -04:00
MarcoFalke fafe78f6ae
ci: Rename .travis/ to ./ci/ 2019-08-15 11:11:45 -04:00