Split off CBlockTreeDB and CCoinsViewDB into txdb-*.{cpp,h} files,
implemented by either LevelDB or BDB.
Based on code from earlier commits by Mike Hearn in his leveldb
branch.
Given that the block tree database (chain.dat) and the active chain
database (coins.dat) are entirely separate now, it becomes legal to
swap one with another instance without affecting the other.
This commit introduces a check in the startup code that detects the
presence of a better chain in chain.dat that has not been activated
yet, and does so efficiently (in batch, while reusing the blk???.dat
files).
During the initial block download (or -loadblock), delay connection
of new blocks a bit, and perform them in a single action. This reduces
the load on the database engine, as subsequent blocks often update an
earlier block's transaction already.
This switches bitcoin's transaction/block verification logic to use a
"coin database", which contains all unredeemed transaction output scripts,
amounts and heights.
The name ultraprune comes from the fact that instead of a full transaction
index, we only (need to) keep an index with unspent outputs. For now, the
blocks themselves are kept as usual, although they are only necessary for
serving, rescanning and reorganizing.
The basic datastructures are CCoins (representing the coins of a single
transaction), and CCoinsView (representing a state of the coins database).
There are several implementations for CCoinsView. A dummy, one backed by
the coins database (coins.dat), one backed by the memory pool, and one
that adds a cache on top of it. FetchInputs, ConnectInputs, ConnectBlock,
DisconnectBlock, ... now operate on a generic CCoinsView.
The block switching logic now builds a single cached CCoinsView with
changes to be committed to the database before any changes are made.
This means no uncommitted changes are ever read from the database, and
should ease the transition to another database layer which does not
support transactions (but does support atomic writes), like LevelDB.
For the getrawtransaction() RPC call, access to a txid-to-disk index
would be preferable. As this index is not necessary or even useful
for any other part of the implementation, it is not provided. Instead,
getrawtransaction() uses the coin database to find the block height,
and then scans that block to find the requested transaction. This is
slow, but should suffice for debug purposes.
Corrupt wallets used to cause a DB_RUNRECOVERY uncaught exception and a
crash. This commit does three things:
1) Runs a BDB verify early in the startup process, and if there is a
low-level problem with the database:
+ Moves the bad wallet.dat to wallet.timestamp.bak
+ Runs a 'salvage' operation to get key/value pairs, and
writes them to a new wallet.dat
+ Continues with startup.
2) Much more tolerant of serialization errors. All errors in deserialization
are reported by tolerated EXCEPT for errors related to reading keypairs
or master key records-- those are reported and then shut down, so the user
can get help (or recover from a backup).
3) Adds a new -salvagewallet option, which:
+ Moves the wallet.dat to wallet.timestamp.bak
+ extracts ONLY keypairs and master keys into a new wallet.dat
+ soft-sets -rescan, to recreate transaction history
This was tested by randomly corrupting testnet wallets using a little
python script I wrote (https://gist.github.com/3812689)
Before, opening a -datadir that was created with a new
version of Berkeley DB would result in an un-caught DB_RUNRECOVERY
exception.
After these changes, the error is caught and the user is told
that there is a problem and is told how to try to recover from
it.
Before, opening a -datadir that was created with a new
version of Berkeley DB would result in an un-caught DB_RUNRECOVERY
exception.
After these changes, the error is caught and the user is told
that there is a problem and is told how to try to recover from
it.
This allows fun stuff such as `bitcoin --help | less`, and more
easy piping to files.
Looking at other tools such as bash, gcc, they all send their help
text to stdout.
- this enables DEP on all Windows version which support the
SetProcessDEPPolicy() call in Kernel32.dll
- use a dynamic approach via GetProcAddress() to not rely on headers or
compiler libs
- this is the same way the Tor-project does it
- ensure warnings always start with "Warning:" and that the first
character after ":" is written uppercase
- ensure the first sentence in warnings ends with an "!"
- remove unneeded spaces from Warning-strings
- add missing Warning-string translation
- remove a "\n" and replace with untranslatable "<br><br>"
- place "-?" option at first
- re-work description and "\n" usage for Gavins new block creation options
to better match current description syntax
- ensure no "\n" is in translated strings, which is better for Transifex
Modify CreateNewBlock so that instead of processing all transactions
in priority order, process the first 27K of transactions in
priority order and then process the rest in fee-per-kilobyte
order.
This is the first, minimal step towards better a better fee-handling
system for both miners and end-users; this patch should be easy
to backport to the old versions of Bitcoin, and accomplishes the
most important goal-- allow users to "buy their way in" to blocks
using transaction fees.
NOTE: These thread names are visible in gdb when using 'info threads'.
Additionally both 'top' and 'ps' show these names *unless* told to
display the command-line instead of task name.
Signed-off-by: Giel van Schijndel <me@mortis.eu>
Useful for developers who need to refer to futher back in debug.log history, but who don't want to
enable the -debug option and all the verbosity that comes with that.
Prior to this change, each TX typically generated 3+ debug messages,
askfor tx 8644cc97480ba1537214 0
sending getdata: tx 8644cc97480ba1537214
askfor tx 8644cc97480ba1537214 1339640761000000
askfor tx 8644cc97480ba1537214 1339640881000000
CTxMemPool::accept() : accepted 8644cc9748 (poolsz 6857)
After this change, there is only one message for each valid TX received
CTxMemPool::accept() : accepted 22a73c5d8c (poolsz 42)
and two messages for each orphan tx received
ERROR: FetchInputs() : 673dc195aa mempool Tx prev not found 1e439346fc
stored orphan tx 673dc195aa (mapsz 19)
The -debugnet option, or its superset -debug, will restore the full debug
output.
This introduces internal types:
* CKeyID: reference (hash160) of a key
* CScriptID: reference (hash160) of a script
* CTxDestination: a boost::variant of the former two
CBitcoinAddress is retrofitted to be a Base58 encoding of a
CTxDestination. This allows all internal code to only use the
internal types, and only have RPC and GUI depend on the base58 code.
Furthermore, the header dependencies are a lot saner now. base58.h is
at the top (right below rpc and gui) instead of at the bottom. For the
rest: wallet -> script -> keystore -> key. Only keystore still requires
a forward declaration of CScript. Solving that would require splitting
script into two layers.
- Signals now go directly from the core to WalletModel/ClientModel.
- WalletModel subscribes to signals on CWallet: Prepares for multi-wallet support, by no longer assuming an implicit global wallet.
- Gets rid of noui.cpp, the few lines that were left are merged into init.cpp
- Rename wxXXX message flags to MF_XXX, to make them UI indifferent.
- ThreadSafeMessageBox no longer returns the value `4` which was never used, converted to void.
Cleans up and organizes several scattered functions and variables related to
the BDB env. Class CDBInit() existed to provide a
guaranteed-via-C++-destructor cleanup of the db environment.
A formal CDBEnv class provides all of this inside a single wrapper.
The best log rotation method formerly available was to configure
logrotate with the copytruncate option. As described in the logrotate
documentation, "there is a very small time slice between copying the
file and truncating it, so some logging data might be lost".
By sending SIGHUP to the server process, one can now reopen the debug
log file without losing any data.
- Solves #1278, attempts to address #1049
- Removes \t's from help message that are removed afterwards anyway
- Moves UI-specific command-line options help to UI code
- Moves "-detachdb" out of #ifdef USE_UPNP
-externalip=<ip> can be used to explicitly set the public IP address
of your node. -discover=0 can be used to disable the automatic public
IP discovery system.
Add an option -detachdb (and entry in OptionDialog), without which no
lsn_reset is called on addr.dat and blkindex.dat. That means these
files cannot be moved to a new environment, but shutdown can be
significantly faster. The wallet file is always lsn_reset'ed.
-detachdb corresponds to the old behaviour, though it is off by
default now to speed up shutdowns.
This commit removes the dependency of serialize.h on PROTOCOL_VERSION,
and makes this parameter required instead of implicit. This is much saner,
as it makes the places where changing a version number can have an
influence obvious.
Where possible, use boost::filesystem::path instead of std::string or
char* for filenames. This avoids a lot of manual string tinkering, in
favor of path::operator/.
GetDataDir is also reworked significantly, it now only keeps two cached
directory names (the network-specific data dir, and the root data dir),
which are decided through a parameter instead of pre-initialized global
variables.
Finally, remove the "upgrade from 0.1.5" case where a debug.log in the
current directory has to be removed.
All client version information is moved to version.cpp, which optionally
(-DHAVE_BUILD_INFO) includes build.h. build.h is automatically generated
on supporting platforms via contrib/genbuild.sh, using git describe.
The git export-subst attribute is used to put the commit id statically
in version.cpp inside generated archives, and this value is used if no
build.h is present.
The gitian descriptors are modified to use git archive instead of a
copy, to create the src/ directory in the output. This way,
src/src/version.cpp will contain the static commit id. To prevent
gitian builds from getting the "-dirty" marker in their git-describe
generated identifiers, no touching of files or running sed on the
makefile is performed anymore. This does not seem to influence
determinism.
- rename wxMessageBox, remove redundant arguments to noui/qtui calls
- also, add flag to force blocking, modal dialog box for disk space warning etc
- clarify function naming
- no more special MessageBox needed from AppInit2, as window object is created before calling AppInit2
Do not automatically change the wallet format unless the user takes an
explicit action that implies an upgrade (encrypting, for now), or uses
-walletupgrade.
-walletupgrade optionally takes an integer argument: the client version
up to which upgrading is allowed. Without an argument, it is upgraded
to latest supported version. If an argument to -walletupgrade is
provided at the time the wallet is created, the new wallet will initially
not use features beyond that version.
Third, the current wallet version number is reported in getinfo.
-checkblocks now takes a numeric argument: the number of blocks that must
be verified at the end of the chain. Default is 2500, and 0 means all
blocks.
-checklevel specifies how thorough the verification must be:
0: only check whether the block exists on disk
1: verify block validity (default)
2: verify transaction index validity
3: check transaction hashes
4: check whether spent txouts were spent within the main chain
5: check whether all prevouts are marked spent
6: check whether spent txouts were spent by a valid transaction that consumes them
Design goals:
* Only keep a limited number of addresses around, so that addr.dat does not grow without bound.
* Keep the address tables in-memory, and occasionally write the table to addr.dat.
* Make sure no (localized) attacker can fill the entire table with his nodes/addresses.
See comments in addrman.h for more detailed information.
