No description
Find a file
MarcoFalke 80112b17e7
Merge #15296: tests: Add script checking for deterministic line coverage in unit tests
43206239a8 tests: Add script checking for deterministic line coverage (practicalswift)

Pull request description:

  Add script checking for deterministic line coverage in unit tests.

  Context: #14343 ("coverage reports non-deterministic")

  When the coverage is deterministic this script can be invoked from Travis to guard against regressions, but left inactive for now.

  Output in case of determinism:

  ```
  $ contrib/test_deterministic_coverage.sh 2
  [2019-01-30 20:08:46] Measuring coverage, run #1 of 2
  [2019-01-30 20:10:45] Measuring coverage, run #2 of 2

  Coverage test passed: Deterministic coverage across 2 runs.
  ```

  Output in case of non-determinism:

  ```
  $ contrib/test_deterministic_coverage.sh 2
  [2019-01-30 20:08:46] Measuring coverage, run #1 of 2
  [2019-01-30 20:10:45] Measuring coverage, run #2 of 2

  The line coverage is non-deterministic between runs.

  The test suite must be deterministic in the sense that the set of lines executed at least
  once must be identical between runs. This is a neccessary condition for meaningful coverage
  measuring.

  --- gcovr.run-1.txt   2019-01-30 23:14:07.419418694 +0100
  +++ gcovr.run-2.txt   2019-01-30 23:15:57.998811282 +0100
  @@ -471,7 +471,7 @@
   test/crypto_tests.cpp                        270     270   100%
   test/cuckoocache_tests.cpp                   142     142   100%
   test/dbwrapper_tests.cpp                     148     148   100%
  -test/denialofservice_tests.cpp               225     225   100%
  +test/denialofservice_tests.cpp               225     224    99%   363
   test/descriptor_tests.cpp                    116     116   100%
   test/fs_tests.cpp                             24       3    12%   14,16-17,19-20,23,25-26,29,31-32,35-36,39,41-42,45-46,49,51-52
   test/getarg_tests.cpp                        111     111   100%
  @@ -585,5 +585,5 @@
   zmq/zmqpublishnotifier.h                       5       0     0%   12,31,37,43,49
   zmq/zmqrpc.cpp                                21       0     0%   16,18,20,22,33-35,38-45,49,52,56,60,62-63
   ------------------------------------------------------------------------------
  -TOTAL                                      61561   27606    44%
  +TOTAL                                      61561   27605    44%
   ------------------------------------------------------------------------------

  ```

  In this case line 363 of `test/denialofservice_tests.cpp` was executed only in the second run. Non-determinism detected!

Tree-SHA512: 03f45590e70a87146f89aa7838beeff0925d7fd303697ff03e0e69f8a5861694be5f0dd10cb0020e3e3d40c9cf662f71dfcd838f6affb31bd5212314e0a4e3a9
2019-03-02 09:46:25 -05:00
.github Get more info about GUI-related issue on Linux 2018-12-27 06:53:07 +02:00
.travis qa: Add test/fuzz/test_runner.py 2019-02-13 17:12:28 -05:00
.tx qt: Pre-0.18 split-off translations update 2019-02-04 15:24:37 +01:00
build-aux/m4 Bump minimum Qt version to 5.5.1 2019-02-14 11:12:30 +01:00
build_msvc msvc: Use a single file to specify the include path 2019-03-01 02:30:07 +08:00
contrib Merge #15296: tests: Add script checking for deterministic line coverage in unit tests 2019-03-02 09:46:25 -05:00
depends Update zmq to 4.3.1 2019-01-18 10:25:14 +02:00
doc Merge #15492: [rpc] remove deprecated generate method 2019-03-02 09:43:39 -05:00
share [build] Makefile.am: add rule for src/bitcoin-wallet 2019-02-14 16:17:55 +01:00
src Merge #15492: [rpc] remove deprecated generate method 2019-03-02 09:43:39 -05:00
test Merge #15492: [rpc] remove deprecated generate method 2019-03-02 09:43:39 -05:00
.appveyor.yml appveyor: Don't build debug libraries instead of "build and delete" 2019-03-01 07:24:19 +08:00
.gitattributes Separate protocol versioning from clientversion 2014-10-29 00:24:40 -04:00
.gitignore [tools] Add wallet inspection and modification tool 2019-01-30 16:26:52 -05:00
.python-version [test] Travis: enforce Python 3.4 support in functional tests 2018-12-12 10:39:32 +01:00
.travis.yml Merge #15295: fuzz: Add test/fuzz/test_runner.py and run it in travis 2019-02-14 16:32:26 -05:00
autogen.sh Add "export LC_ALL=C" to all shell scripts 2018-06-14 15:27:52 +02:00
configure.ac build: Bump version to 0.18.99 2019-03-02 14:28:48 +01:00
CONTRIBUTING.md Botbot.me (IRC logs) not available anymore 2019-01-01 16:04:38 +02:00
COPYING [Trivial] Update license year range to 2019 2018-12-31 04:27:59 +01:00
INSTALL.md Update INSTALL landing redirection notice for build instructions. 2016-10-06 12:27:23 +13:00
libbitcoinconsensus.pc.in Unify package name to as few places as possible without major changes 2015-12-14 02:11:10 +00:00
Makefile.am Merge #15295: fuzz: Add test/fuzz/test_runner.py and run it in travis 2019-02-14 16:32:26 -05:00
README.md docs: Update Transifex links 2019-03-02 17:42:33 +08:00

Bitcoin Core integration/staging tree

Build Status

https://bitcoincore.org

What is Bitcoin?

Bitcoin is an experimental digital currency that enables instant payments to anyone, anywhere in the world. Bitcoin uses peer-to-peer technology to operate with no central authority: managing transactions and issuing money are carried out collectively by the network. Bitcoin Core is the name of open source software which enables the use of this currency.

For more information, as well as an immediately useable, binary version of the Bitcoin Core software, see https://bitcoincore.org/en/download/, or read the original whitepaper.

License

Bitcoin Core is released under the terms of the MIT license. See COPYING for more information or see https://opensource.org/licenses/MIT.

Development Process

The master branch is regularly built and tested, but is not guaranteed to be completely stable. Tags are created regularly to indicate new official, stable release versions of Bitcoin Core.

The contribution workflow is described in CONTRIBUTING.md and useful hints for developers can be found in doc/developer-notes.md.

Testing

Testing and code review is the bottleneck for development; we get more pull requests than we can review and test on short notice. Please be patient and help out by testing other people's pull requests, and remember this is a security-critical project where any mistake might cost people lots of money.

Automated Testing

Developers are strongly encouraged to write unit tests for new code, and to submit new unit tests for old code. Unit tests can be compiled and run (assuming they weren't disabled in configure) with: make check. Further details on running and extending unit tests can be found in /src/test/README.md.

There are also regression and integration tests, written in Python, that are run automatically on the build server. These tests can be run (if the test dependencies are installed) with: test/functional/test_runner.py

The Travis CI system makes sure that every pull request is built for Windows, Linux, and macOS, and that unit/sanity tests are run automatically.

Manual Quality Assurance (QA) Testing

Changes should be tested by somebody other than the developer who wrote the code. This is especially important for large or high-risk changes. It is useful to add a test plan to the pull request description if testing the changes is not straightforward.

Translations

Changes to translations as well as new translations can be submitted to Bitcoin Core's Transifex page.

Translations are periodically pulled from Transifex and merged into the git repository. See the translation process for details on how this works.

Important: We do not accept translation changes as GitHub pull requests because the next pull from Transifex would automatically overwrite them again.

Translators should also subscribe to the mailing list.