Commit graph

603 commits

Author SHA1 Message Date
Pieter Wuille 8b66bf74e2
Merge #9441: Net: Massive speedup. Net locks overhaul
e60360e net: remove cs_vRecvMsg (Cory Fields)
991955e net: add a flag to indicate when a node's send buffer is full (Cory Fields)
c6e8a9b net: add a flag to indicate when a node's process queue is full (Cory Fields)
4d712e3 net: add a new message queue for the message processor (Cory Fields)
c5a8b1b net: rework the way that the messagehandler sleeps (Cory Fields)
c72cc88 net: remove useless comments (Cory Fields)
ef7b5ec net: Add a simple function for waking the message handler (Cory Fields)
f5c36d1 net: record bytes written before notifying the message processor (Cory Fields)
60befa3 net: handle message accounting in ReceiveMsgBytes (Cory Fields)
56212e2 net: set message deserialization version when it's actually time to deserialize (Cory Fields)
0e973d9 net: remove redundant max sendbuffer size check (Cory Fields)
6042587 net: wait until the node is destroyed to delete its recv buffer (Cory Fields)
f6315e0 net: only disconnect if fDisconnect has been set (Cory Fields)
5b4a8ac net: make GetReceiveFloodSize public (Cory Fields)
e5bcd9c net: make vRecvMsg a list so that we can use splice() (Cory Fields)
53ad9a1 net: fix typo causing the wrong receive buffer size (Cory Fields)
2017-01-13 10:02:18 -08:00
Cory Fields e60360e139 net: remove cs_vRecvMsg
vRecvMsg is now only touched by the socket handler thread.

The accounting vars (nRecvBytes/nLastRecv/mapRecvBytesPerMsgCmd) are also
only used by the socket handler thread, with the exception of queries from
rpc/gui. These accesses are not threadsafe, but they never were. This needs to
be addressed separately.

Also, update comment describing data flow
2017-01-12 23:08:08 -05:00
Cory Fields 991955ee81 net: add a flag to indicate when a node's send buffer is full
Similar to the recv flag, but this one indicates whether or not the net's send
buffer is full.

The socket handler checks the send queue when a new message is added and pauses
if necessary, and possibly unpauses after each message is drained from its buffer.
2017-01-12 23:05:59 -05:00
Cory Fields c6e8a9bcff net: add a flag to indicate when a node's process queue is full
Messages are dumped very quickly from the socket handler to the processor, so
it's the depth of the processing queue that's interesting.

The socket handler checks the process queue's size during the brief message
hand-off and pauses if necessary, and the processor possibly unpauses each time
a message is popped off of its queue.
2017-01-12 23:05:47 -05:00
Cory Fields 4d712e366c net: add a new message queue for the message processor
This separates the storage of messages from the net and queued messages for
processing, allowing the locks to be split.
2017-01-12 23:05:25 -05:00
Cory Fields c5a8b1b946 net: rework the way that the messagehandler sleeps
In order to sleep accurately, the message handler needs to know if _any_ node
has more processing that it should do before the entire thread sleeps.

Rather than returning a value that represents whether ProcessMessages
encountered a message that should trigger a disconnnect, interpret the return
value as whether or not that node has more work to do.

Also, use a global fProcessWake value that can be set by other threads,
which takes precedence (for one cycle) over the messagehandler's decision.

Note that the previous behavior was to only process one message per loop
(except in the case of a bad checksum or invalid header). That was changed in
PR #3180.

The only change here in that regard is that the current node now falls to the
back of the processing queue for the bad checksum/invalid header cases.
2017-01-12 23:05:24 -05:00
Cory Fields ef7b5ecbb7 net: Add a simple function for waking the message handler
This may be used publicly in the future
2017-01-12 23:05:09 -05:00
Cory Fields f5c36d19b6 net: record bytes written before notifying the message processor 2017-01-12 23:05:09 -05:00
Cory Fields 60befa3997 net: handle message accounting in ReceiveMsgBytes
This allows locking to be pushed down to only where it's needed

Also reuse the current time rather than checking multiple times.
2017-01-12 23:05:09 -05:00
Douglas Roark cc0589639c
Remove stray semicolon (Fix empty body warning)
Empty body introduced by commit #9319 should not be empty.
2017-01-07 09:49:14 -08:00
Gregory Maxwell 032ba3f066 RPC help documentation for addnode peerinfo.
Also adds a comment about the netgroup exclusion behavior.
2017-01-05 19:02:15 +00:00
Gregory Maxwell 50bd12ce0c Break addnode out from the outbound connection limits.
Previously addnodes were in competition with outbound connections
 for access to the eight outbound slots.

One result of this is that frequently a node with several addnode
 configured peers would end up connected to none of them, because
 while the addnode loop was in its two minute sleep the automatic
 connection logic would fill any free slots with random peers.
 This is particularly unwelcome to users trying to maintain links
 to specific nodes for fast block relay or purposes.

Another result is that a group of nine or more nodes which are
 have addnode configured towards each other can become partitioned
 from the public network.

This commit introduces a new limit of eight connections just for
 addnode peers which is not subject to any of the other connection
 limitations (including maxconnections).

The choice of eight is sufficient so that under no condition would
 a user find themselves connected to fewer addnoded peers than
 previously.  It is also low enough that users who are confused
 about the significance of more connections and have gotten too
 copy-and-paste happy will not consume more than twice the slot
 usage of a typical user.

Any additional load on the network resulting from this will likely
 be offset by a reduction in users applying even more wasteful
 workaround for the prior behavior.

The retry delays are reduced to avoid nodes sitting around without
 their added peers up, but are still sufficient to prevent overly
 aggressive repeated connections.  The reduced delays also make
 the system much more responsive to the addnode RPC.

Ban-disconnects are also exempted for peers added via addnode since
 the outbound addnode logic ignores bans.  Previously it would ban
 an addnode then immediately reconnect to it.

A minor change was also made to CSemaphoreGrant so that it is
 possible to re-acquire via an object whos grant was moved.
2017-01-05 19:02:09 +00:00
Cory Fields 56212e20ac net: set message deserialization version when it's actually time to deserialize
We'll soon no longer have access to vRecvMsg, and this is more intuitive anyway.
2017-01-04 10:10:35 -05:00
Cory Fields 60425870d7 net: wait until the node is destroyed to delete its recv buffer
when vRecvMsg becomes a private buffer, it won't make sense to allow other
threads to mess with it anymore.
2017-01-04 09:29:14 -05:00
Cory Fields f6315e07f9 net: only disconnect if fDisconnect has been set
These conditions are problematic to check without locking, and we shouldn't be
relying on the refcount to disconnect.
2017-01-04 09:29:14 -05:00
Cory Fields e5bcd9c84f net: make vRecvMsg a list so that we can use splice() 2017-01-04 09:29:14 -05:00
Cory Fields 53ad9a133a net: fix typo causing the wrong receive buffer size
Surprisingly this hasn't been causing me any issues while testing, probably
because it requires lots of large blocks to be flying around.

Send/Recv corks need tests!
2017-01-04 09:29:14 -05:00
Wladimir J. van der Laan d9ae1cefa0
Merge #9289: net: drop boost::thread_group
67ee4ec net: misc header cleanups (Cory Fields)
8b3159e net: make proxy receives interruptible (Cory Fields)
5cb0fce net: remove thread_interrupted catch (Cory Fields)
d3d7056 net: make net processing interruptible (Cory Fields)
0985052 net: make net interruptible (Cory Fields)
799df91 net: add CThreadInterrupt and InterruptibleSleep (Cory Fields)
7325b15 net: a few small cleanups before replacing boost threads (Cory Fields)
2017-01-04 12:21:53 +01:00
Cory Fields 67ee4ec901 net: misc header cleanups 2017-01-03 17:56:21 -05:00
Cory Fields 8b3159ef0a net: make proxy receives interruptible 2017-01-03 17:56:21 -05:00
Cory Fields d3d7056d2a net: make net processing interruptible 2017-01-03 17:56:20 -05:00
Cory Fields 0985052319 net: make net interruptible
Also now that net threads are interruptible, switch them to use std
threads/binds/mutexes/condvars.
2017-01-03 17:53:09 -05:00
Cory Fields 7325b15566 net: a few small cleanups before replacing boost threads
- Drop the interruption point directly after the pnode allocation. This would
    be leaky if hit.
- Rearrange thread creation so that the socket handler comes first
2017-01-03 17:53:09 -05:00
isle2983 27765b6403 Increment MIT Licence copyright header year on files modified in 2016
Edited via:

$ contrib/devtools/copyright_header.py update .
2016-12-31 11:01:21 -07:00
Matt Corallo 2b5f085ad1 Fix non-const mapMultiArgs[] access after init.
Swap mapMultiArgs for a const-reference to a _mapMultiArgs which is
only accessed in util.cpp
2016-12-23 21:30:15 -05:00
Pieter Wuille 3bf06e9bac
Merge #9226: Remove fNetworkNode and pnodeLocalHost.
bdb922b Remove pnodeLocalHost. (Gregory Maxwell)
083f203 Remove fNetworkNode. (Gregory Maxwell)
2016-11-30 17:15:58 -08:00
Wladimir J. van der Laan 5488514b90
Merge #9225: Fix some benign races
dfed983 Fix unlocked access to vNodes.size() (Matt Corallo)
3033522 Remove double brackets in addrman (Matt Corallo)
dbfaade Fix AddrMan locking (Matt Corallo)
047ea10 Make fImporting an std::atomic (Matt Corallo)
42071ca Make fDisconnect an std::atomic (Matt Corallo)
2016-11-29 12:39:33 +01:00
Gregory Maxwell bdb922b34c Remove pnodeLocalHost.
Mostly a legacy of the long removed pub/sub system.
2016-11-27 04:19:37 +00:00
Gregory Maxwell 083f203698 Remove fNetworkNode.
Matt pointed out to me that this appeared to be doing nothing (except involving itself in data races).
2016-11-27 04:13:54 +00:00
Matt Corallo dfed983f19 Fix unlocked access to vNodes.size() 2016-11-26 15:42:55 -08:00
Cory Fields c7be56dcef net: push only raw data into CConnman
This fixes one of the last major layer violations in the networking stack.

The network side is no longer in charge of message serialization, so it is now
decoupled from Bitcoin structures. Only the header is serialized and attached
to the payload.
2016-11-25 12:37:33 -05:00
Jonas Schnelli ab914a6530
Merge #8996: Network activity toggle
19f46f1 Qt: New network_disabled icon (Luke Dashjr)
54cf997 RPC/Net: Use boolean consistently for networkactive, and remove from getinfo (Luke Dashjr)
b2b33d9 Overhaul network activity toggle (Jonas Schnelli)
32efa79 Qt: Add GUI feedback and control of network activity state. (Jon Lund Steffensen)
e38993b RPC: Add "togglenetwork" method to toggle network activity temporarily (Jon Lund Steffensen)
7c9a98a Allow network activity to be temporarily suspended. (Jon Lund Steffensen)
2016-11-11 11:16:40 +01:00
Pieter Wuille 9f554e03eb
Merge #9045: Hash P2P messages as they are received instead of at process-time
fe1dc62 Hash P2P messages as they are received instead of at process-time (Matt Corallo)
2016-11-07 14:19:44 -08:00
Wladimir J. van der Laan c113a651f1
Merge #9052: Use RelevantServices instead of node_network in AttemptToEvict.
d32036a Use RelevantServices instead of node_network in AttemptToEvict. (Gregory Maxwell)
2016-11-07 13:14:04 +01:00
Cory Fields 902768099c net: handle version push in InitializeNode 2016-11-03 13:35:54 -07:00
Cory Fields 440f1d3e4c net: remove now-unused ssSend and Fuzz 2016-11-03 13:35:54 -07:00
Cory Fields 5c2169cc3f drop the optimistic write counter hack
This is now handled properly in realtime.
2016-11-03 13:35:54 -07:00
Cory Fields ea3326891d net: switch all callers to connman for pushing messages
Drop all of the old stuff.
2016-11-03 13:35:53 -07:00
Cory Fields 3e32cd09f6 connman is in charge of pushing messages
The changes here are dense and subtle, but hopefully all is more explicit
than before.

- CConnman is now in charge of sending data rather than the nodes themselves.
  This is necessary because many decisions need to be made with all nodes in
  mind, and a model that requires the nodes calling up to their manager quickly
  turns to spaghetti.

- The per-node-serializer (ssSend) has been replaced with a (quasi-)const
  send-version. Since the send version for serialization can only change once
  per connection, we now explicitly tag messages with INIT_PROTO_VERSION if
  they are sent before the handshake. With this done, there's no need to lock
  for access to nSendVersion.

  Also, a new stream is used for each message, so there's no need to lock
  during the serialization process.

- This takes care of accounting for optimistic sends, so the
  nOptimisticBytesWritten hack can be removed.

- -dropmessagestest and -fuzzmessagestest have not been preserved, as I suspect
  they haven't been used in years.
2016-11-03 13:32:09 -07:00
Gregory Maxwell d32036a47d Use RelevantServices instead of node_network in AttemptToEvict.
Use of node_network here is really meant to be a proxy of "likely to
 send us blocks in the future".  RelevantServices is the right criteria
 now.
2016-11-01 00:11:52 +00:00
Cory Fields 59ac5c5b72 net: Use deterministic randomness for CNode's nonce, and make it const 2016-10-31 17:17:18 -04:00
Cory Fields aff6584e09 net: constify a few CNode vars to indicate that they're threadsafe 2016-10-31 17:17:18 -04:00
Matt Corallo fe1dc62cef Hash P2P messages as they are received instead of at process-time 2016-10-30 18:04:57 -04:00
Wladimir J. van der Laan f14f07cb94
Merge #9008: [net] Remove assert(nMaxInbound > 0)
fa1c3c2 [net] Remove assert(nMaxInbound > 0) (MarcoFalke)
2016-10-25 12:37:05 +02:00
Gregory Maxwell 515e2642eb Make connect=0 disable automatic outbound connections.
Otherwise it just responds to this obvious bit of configuration by
 trying to connect to "0" in a loop.
2016-10-25 06:17:43 +00:00
MarcoFalke fa1c3c2eb0 [net] Remove assert(nMaxInbound > 0)
nMaxInbound might very well be 0 or -1, if the user prefers to keep
a small number of maxconnections.

Note: nMaxInbound of -1 means that the user set maxconnections
to 8 or less, but we still want to keep an additional slot for
the feeler connection.
2016-10-24 22:12:48 +02:00
Jon Lund Steffensen 32efa79e0e Qt: Add GUI feedback and control of network activity state.
Add getNetworkActive()/setNetworkActive() method to client model.
Send network active status through NotifyNetworkActiveChanged.
Indicate in tool tip of gui status bar network indicator whether network activity is disabled.
Indicate in debug window whether network activity is disabled and add button to allow user to toggle network activity state.
2016-10-24 10:23:58 +00:00
Jon Lund Steffensen 7c9a98aac8 Allow network activity to be temporarily suspended.
Added the function SetNetworkActive() which when called with argument set to false disconnects all nodes and sets the flag fNetworkActive to false. As long as this flag is false no new connections are attempted and no incoming connections are accepted. Network activity is reenabled by calling the function with argument true.
2016-10-24 10:23:58 +00:00
Wladimir J. van der Laan e44753c067
Merge #8949: Be more agressive in getting connections to peers with relevant services.
4630479 Make dnsseed's definition of acute need include relevant services. (Gregory Maxwell)
9583477 Be more aggressive in connecting to peers with relevant services. (Gregory Maxwell)
2016-10-19 10:43:54 +02:00
Gregory Maxwell 4630479135 Make dnsseed's definition of acute need include relevant services.
We normally prefer to connect to peers offering the relevant services.

If we're not connected to enough peers with relevant services, we
 probably don't know about them and could use dnsseed's help.
2016-10-19 08:07:02 +00:00