Go to file
MarcoFalke 5029e94f85
Merge #14519: tests: add utility to easily profile node performance with perf
13782b8ba8 docs: add perf section to developer docs (James O'Beirne)
58180b5fd4 tests: add utility to easily profile node performance with perf (James O'Beirne)

Pull request description:

  Adds a context manager to easily (and selectively) profile node performance during functional test execution using `perf`.

  While writing some tests, I encountered some odd bitcoind slowness. I wrote up a utility (`TestNode.profile_with_perf`) that generates performance diagnostics for a node by running `perf` during the execution of a particular region of test code.

  `perf` usage is detailed in the excellent (and sadly unmerged) https://github.com/bitcoin/bitcoin/pull/12649; all due props to @eklitzke.

  ### Example

  ```python
  with node.profile_with_perf("large-msgs"):
      for i in range(200):
          node.p2p.send_message(some_large_msg)
      node.p2p.sync_with_ping()
  ```

  This generates a perf data file in the test node's datadir (`/tmp/testtxmpod0y/node0/node-0-TestName-large-msgs.perf.data`).

  Running `perf report` generates nice output about where the node spent most of its time while running that part of the test:

  ```bash
  $ perf report -i /tmp/testtxmpod0y/node0/node-0-TestName-large-msgs.perf.data --stdio \
    | c++filt \
    | less

  # To display the perf.data header info, please use --header/--header-only options.
  #
  #
  # Total Lost Samples: 0
  #
  # Samples: 135  of event 'cycles:pp'
  # Event count (approx.): 1458205679493582
  #
  # Children      Self  Command          Shared Object        Symbol
  # ........  ........  ...............  ...................  ........................................................................................................................................................................................................................................................................
  #
      70.14%     0.00%  bitcoin-net      bitcoind             [.] CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)
                  |
                  ---CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

      70.14%     0.00%  bitcoin-net      bitcoind             [.] CNetMessage::readData(char const*, unsigned int)
                  |
                  ---CNetMessage::readData(char const*, unsigned int)
                     CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

      35.52%     0.00%  bitcoin-net      bitcoind             [.] std::vector<char, zero_after_free_allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, zero_after_free_allocator<char> > >, unsigned long, char const&)
                  |
                  ---std::vector<char, zero_after_free_allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, zero_after_free_allocator<char> > >, unsigned long, char const&)
                     CNetMessage::readData(char const*, unsigned int)
                     CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

  ...
  ```

Tree-SHA512: 9ac4ceaa88818d5eca00994e8e3c8ad42ae019550d6583972a0a4f7b0c4f61032e3d0c476b4ae58756bc5eb8f8015a19a7fc26c095bd588f31d49a37ed0c6b3e
2019-02-05 17:40:16 -05:00
.github Get more info about GUI-related issue on Linux 2018-12-27 06:53:07 +02:00
.travis travis: Revert "Run extended tests once daily" 2019-02-02 14:38:37 -05:00
.tx qt: Pre-0.18 split-off translations update 2019-02-04 15:24:37 +01:00
build-aux/m4 Bump the minimum Qt version to 5.2 2018-11-14 01:32:51 +02:00
build_msvc Merge #14922: windows: Set _WIN32_WINNT to 0x0601 (Windows 7) 2019-02-05 18:15:01 +01:00
contrib Merge #15245: remove deprecated mentions of signrawtransaction from fundraw help 2019-02-05 10:48:36 -05:00
depends Update zmq to 4.3.1 2019-01-18 10:25:14 +02:00
doc Merge #14519: tests: add utility to easily profile node performance with perf 2019-02-05 17:40:16 -05:00
share Merge #14701: build: Add CLIENT_VERSION_BUILD to CFBundleGetInfoString 2018-12-12 16:24:52 +01:00
src Merge #15332: [Docs] Small updates to getrawtransaction description 2019-02-05 17:18:40 -05:00
test Merge #14519: tests: add utility to easily profile node performance with perf 2019-02-05 17:40:16 -05:00
.appveyor.yml msvc: build leveldb locally 2019-02-01 00:28:50 +08:00
.gitattributes Separate protocol versioning from clientversion 2014-10-29 00:24:40 -04:00
.gitignore [tools] Add wallet inspection and modification tool 2019-01-30 16:26:52 -05:00
.python-version [test] Travis: enforce Python 3.4 support in functional tests 2018-12-12 10:39:32 +01:00
.travis.yml Merge #15303: travis: Remove unused FUNCTIONAL_TESTS_CONFIG 2019-02-01 17:18:03 -05:00
autogen.sh Add "export LC_ALL=C" to all shell scripts 2018-06-14 15:27:52 +02:00
configure.ac Merge #14922: windows: Set _WIN32_WINNT to 0x0601 (Windows 7) 2019-02-05 18:15:01 +01:00
CONTRIBUTING.md Botbot.me (IRC logs) not available anymore 2019-01-01 16:04:38 +02:00
COPYING [Trivial] Update license year range to 2019 2018-12-31 04:27:59 +01:00
INSTALL.md Update INSTALL landing redirection notice for build instructions. 2016-10-06 12:27:23 +13:00
libbitcoinconsensus.pc.in Unify package name to as few places as possible without major changes 2015-12-14 02:11:10 +00:00
Makefile.am build: Add bitcoin-tx.exe into Windows installer 2018-11-09 21:57:13 +08:00
README.md [doc] conf: Remove deprecated options from docs, Other cleanup 2018-11-07 13:30:03 -05:00

Bitcoin Core integration/staging tree

Build Status

https://bitcoincore.org

What is Bitcoin?

Bitcoin is an experimental digital currency that enables instant payments to anyone, anywhere in the world. Bitcoin uses peer-to-peer technology to operate with no central authority: managing transactions and issuing money are carried out collectively by the network. Bitcoin Core is the name of open source software which enables the use of this currency.

For more information, as well as an immediately useable, binary version of the Bitcoin Core software, see https://bitcoincore.org/en/download/, or read the original whitepaper.

License

Bitcoin Core is released under the terms of the MIT license. See COPYING for more information or see https://opensource.org/licenses/MIT.

Development Process

The master branch is regularly built and tested, but is not guaranteed to be completely stable. Tags are created regularly to indicate new official, stable release versions of Bitcoin Core.

The contribution workflow is described in CONTRIBUTING.md and useful hints for developers can be found in doc/developer-notes.md.

Testing

Testing and code review is the bottleneck for development; we get more pull requests than we can review and test on short notice. Please be patient and help out by testing other people's pull requests, and remember this is a security-critical project where any mistake might cost people lots of money.

Automated Testing

Developers are strongly encouraged to write unit tests for new code, and to submit new unit tests for old code. Unit tests can be compiled and run (assuming they weren't disabled in configure) with: make check. Further details on running and extending unit tests can be found in /src/test/README.md.

There are also regression and integration tests, written in Python, that are run automatically on the build server. These tests can be run (if the test dependencies are installed) with: test/functional/test_runner.py

The Travis CI system makes sure that every pull request is built for Windows, Linux, and macOS, and that unit/sanity tests are run automatically.

Manual Quality Assurance (QA) Testing

Changes should be tested by somebody other than the developer who wrote the code. This is especially important for large or high-risk changes. It is useful to add a test plan to the pull request description if testing the changes is not straightforward.

Translations

Changes to translations as well as new translations can be submitted to Bitcoin Core's Transifex page.

Translations are periodically pulled from Transifex and merged into the git repository. See the translation process for details on how this works.

Important: We do not accept translation changes as GitHub pull requests because the next pull from Transifex would automatically overwrite them again.

Translators should also subscribe to the mailing list.