Commit graph

3534 commits

Author SHA1 Message Date
Pieter Wuille
d979e6e36a Use singleton block tree database instance 2012-10-20 23:08:57 +02:00
Pieter Wuille
857c61df0b Prepare database format for multi-stage block processing
This commit adds a status field and a transaction counter to the block
indexes.
2012-10-20 23:08:57 +02:00
Pieter Wuille
4fea06db25 Automatically reorganize at startup to best known block
Given that the block tree database (chain.dat) and the active chain
database (coins.dat) are entirely separate now, it becomes legal to
swap one with another instance without affecting the other.

This commit introduces a check in the startup code that detects the
presence of a better chain in chain.dat that has not been activated
yet, and does so efficiently (in batch, while reusing the blk???.dat
files).
2012-10-20 23:08:57 +02:00
Pieter Wuille
13c51f20f6 Direct CCoins references
To prevent excessive copying of CCoins in and out of the CCoinsView
implementations, introduce a GetCoins() function in CCoinsViewCache
with returns a direct reference. The block validation and connection
logic is updated to require caching CCoinsViews, and exploits the
GetCoins() function heavily.
2012-10-20 23:08:57 +02:00
Pieter Wuille
64dd46fd05 Transaction hash caching
Use CBlock's vMerkleTree to cache transaction hashes, and pass them
along as argument in more function calls. During initial block download,
this results in every transaction's hash to be only computed once.
2012-10-20 23:08:57 +02:00
Pieter Wuille
ae8bfd12da Batch block connection during IBD
During the initial block download (or -loadblock), delay connection
of new blocks a bit, and perform them in a single action. This reduces
the load on the database engine, as subsequent blocks often update an
earlier block's transaction already.
2012-10-20 23:08:57 +02:00
Pieter Wuille
450cbb0944 Ultraprune
This switches bitcoin's transaction/block verification logic to use a
"coin database", which contains all unredeemed transaction output scripts,
amounts and heights.

The name ultraprune comes from the fact that instead of a full transaction
index, we only (need to) keep an index with unspent outputs. For now, the
blocks themselves are kept as usual, although they are only necessary for
serving, rescanning and reorganizing.

The basic datastructures are CCoins (representing the coins of a single
transaction), and CCoinsView (representing a state of the coins database).
There are several implementations for CCoinsView. A dummy, one backed by
the coins database (coins.dat), one backed by the memory pool, and one
that adds a cache on top of it. FetchInputs, ConnectInputs, ConnectBlock,
DisconnectBlock, ... now operate on a generic CCoinsView.

The block switching logic now builds a single cached CCoinsView with
changes to be committed to the database before any changes are made.
This means no uncommitted changes are ever read from the database, and
should ease the transition to another database layer which does not
support transactions (but does support atomic writes), like LevelDB.

For the getrawtransaction() RPC call, access to a txid-to-disk index
would be preferable. As this index is not necessary or even useful
for any other part of the implementation, it is not provided. Instead,
getrawtransaction() uses the coin database to find the block height,
and then scans that block to find the requested transaction. This is
slow, but should suffice for debug purposes.
2012-10-20 23:08:57 +02:00
Pieter Wuille
bba89aa82a Pre-allocate block and undo files in chunks
Introduce a AllocateFileRange() function in util, which wipes or
at least allocates a given range of a file. It can be overriden
by more efficient OS-dependent versions if necessary.

Block and undo files are now allocated in chunks of 16 and 1 MiB,
respectively.
2012-10-20 23:08:57 +02:00
Pieter Wuille
5382bcf8cd Multiple blocks per file
Change the block storage layer again, this time with multiple files
per block, but tracked by txindex.dat database entries. The file
format is exactly the same as the earlier blk00001.dat, but with
smaller files (128 MiB for now).

The database entries track how many bytes each block file already
uses, how many blocks are in it, which range of heights is present
and which range of dates.
2012-10-20 23:08:57 +02:00
Pieter Wuille
8adf48dc9b Preliminary undo file creation
Create files (one per block) with undo information for the transactions
in it.
2012-10-20 23:08:57 +02:00
Pieter Wuille
630fd8dcb6 One file per block
Refactor of the block storage code, which now stores one file per block.
This will allow easier pruning, as blocks can be removed individually.
2012-10-20 23:08:57 +02:00
Pieter Wuille
44ac1c0fe3 Add CTxUndo: transaction undo information
The CTxUndo class encapsulates data necessary to undo the effects of
a transaction on the txout set, namely the previous outputs consumed
by it (script + amount), and potentially transaction meta-data when
it is spent entirely.
2012-10-20 23:08:56 +02:00
Pieter Wuille
10fd8604d7 Add CCoins: pruned list of transaction outputs
The CCoins class represents a pruned set of transaction outputs from
a given transaction. It only retains information about its height in
the block chain, whether it was a coinbase transaction, and its
unspent outputs (script + amount).

It has a custom serializer that has very low redundancy.
2012-10-20 23:08:56 +02:00
Pieter Wuille
0fa593d0fb Compact serialization for amounts
Special serializer/deserializer for amount values. It is optimized for
values which have few non-zero digits in decimal representation. Most
amounts currently in the txout set take only 1 or 2 bytes to
represent.
2012-10-20 23:08:56 +02:00
Pieter Wuille
69fc8047a9 Compact serialization for scripts
Special serializers for script which detect common cases and encode
them much more efficiently. 3 special cases are defined:
* Pay to pubkey hash (encoded as 21 bytes)
* Pay to script hash (encoded as 21 bytes)
* Pay to pubkey starting with 0x02, 0x03 or 0x04 (encoded as 33 bytes)

Other scripts up to 121 bytes require 1 byte + script length. Above
that, scripts up to 16505 bytes require 2 bytes + script length.
2012-10-20 23:08:56 +02:00
Pieter Wuille
4d6144f97f Compact serialization for variable-length integers
Variable-length integers: bytes are a MSB base-128 encoding of the number.
The high bit in each byte signifies whether another digit follows. To make
the encoding is one-to-one, one is subtracted from all but the last digit.
Thus, the byte sequence a[] with length len, where all but the last byte
has bit 128 set, encodes the number:

  (a[len-1] & 0x7F) + sum(i=1..len-1, 128^i*((a[len-i-1] & 0x7F)+1))

Properties:
* Very small (0-127: 1 byte, 128-16511: 2 bytes, 16512-2113663: 3 bytes)
* Every integer has exactly one encoding
* Encoding does not depend on size of original integer type
2012-10-20 23:08:56 +02:00
Pieter Wuille
43b7905e98 LevelDB glue
Database-independent glue for supporting LevelDB databases.

Based on code from earlier commits by Mike Hearn in his leveldb
branch.
2012-10-20 23:08:56 +02:00
Pieter Wuille
3ff3a2bd60 Makefile integration of LevelDB 2012-10-20 23:08:56 +02:00
Pieter Wuille
9f56678fce Backport Win32 LevelDB env from C++0x to C++
Since the gitian mingw compiler doesn't support C++0x yet.
2012-10-20 23:08:56 +02:00
Pieter Wuille
9d503a7285 Disable libsnappy detection in LevelDB 2012-10-20 23:08:56 +02:00
justmoon
94a50fb339 Leveldb Windows port by Edouard Alligand, adapted for MingW by me. 2012-10-20 23:08:56 +02:00
Mike Hearn
5e650d6d2d Import LevelDB 1.5, it will be used for the transaction database. 2012-10-20 23:08:56 +02:00
Jeff Garzik
38ac953b9d Merge pull request #1880 from sipa/threadimport
Move external block import to separate thread
2012-10-20 13:36:36 -07:00
Jeff Garzik
dee0ee2ac9 Merge pull request #1742 from sipa/canonical
Check for canonical public keys and signatures
2012-10-20 10:56:04 -07:00
Pieter Wuille
66b02c93e6 Move external block import to separate thread 2012-10-20 01:54:10 +02:00
Pieter Wuille
8980a50924 Use CHashWriter also in SignatureHash(), and for message signing 2012-10-19 23:18:07 +02:00
Rune K. Svendsen
de91ea0c0c Added script that verifies authenticity of binaries on SourceForge 2012-10-15 22:44:39 +02:00
Wladimir J. van der Laan
485d667748 Revert "Merge pull request #1931 from laanwj/2012_10_newicons"
This reverts commit 199d88cf90, reversing
changes made to 65bc1573e7.

License is worse instead of better. Will only accept public domain and
MIT-licensed icons from now on.
2012-10-14 20:49:06 +02:00
Wladimir J. van der Laan
199d88cf90 Merge pull request #1931 from laanwj/2012_10_newicons
Non-GPL Icons, v2
2012-10-14 07:52:21 -07:00
xanatos
cddda5d9f0 Changed connect?_16.png to non-GPL one and changed the assets attribution. 2012-10-14 13:01:38 +02:00
xanatos
b96238a9c9 Changed the spinner to a non-GPL one, added instructions on how to regenerate it, changed the assets attribution, removed old spinner + old spinner's sources. 2012-10-14 12:34:07 +02:00
Philip Kaufmann
493940038f Bitcoin-Qt: remove unneeded "--param ssp-buffer-size=1" flag
- that flag is not needed when using "-fstack-protector-all", so remove it
  (see:
  http://stackoverflow.com/questions/1629685/when-and-how-to-use-gccs-stack-protection-feature)
2012-10-13 10:25:17 +02:00
Wladimir J. van der Laan
65bc1573e7 Merge pull request #1922 from Diapolo/trans_en_master
Translation update for Bitcoin-Qt 0.7.1
2012-10-12 15:09:47 -07:00
Philip Kaufmann
10d3603ffa Bitcoin-Qt: add new GCC compiler hardening options
- this patch enables several new GCC compiler hardening options that
  allows us to increase the security of our binaries (see:
  https://wiki.debian.org/Hardening)

-D_FORTIFY_SOURCE=2:
Enables compile-time protection against static sized buffer overflows.

-Wl,-z,relro -Wl,-z,now:
Enables full RELRO (RELocation Read-Only), which is a generic mitigation
technique to harden the data sections of an ELF binary/process. See:
http://isisblogs.poly.edu/2011/06/01/relro-relocation-read-only/ for
further details.
2012-10-12 23:16:27 +02:00
Philip Kaufmann
34710818a7 Bitcoin-Qt: intregrate current translations from Transifex 2012-10-12 15:51:49 +02:00
Philip Kaufmann
0e005c3ddc Bitcoin-Qt: update english translation master file 2012-10-12 15:45:27 +02:00
Wladimir J. van der Laan
4bb25e48d7 Merge pull request #1921 from laanwj/2012_10_init_useafterfree
Fix a use-after-free problem in initialization (#1920)
2012-10-11 18:28:47 -07:00
Wladimir J. van der Laan
22bb049011 Fix a use-after-free problem in initialization (#1920)
Don't store the result of c_str().

Luckily, this only affects logging, though it could crash or leak
sensitive data to the log in rare cases.
2012-10-12 03:09:05 +02:00
Wladimir J. van der Laan
16f3618c13 Merge pull request #1879 from sipa/fdatasync
Use fdatasync instead of fsync on supported platforms
2012-10-11 00:42:47 -07:00
Wladimir J. van der Laan
24694e24ed Merge pull request #1913 from sipa/noi2p
Remove I2P support from netbase
2012-10-11 00:41:33 -07:00
Wladimir J. van der Laan
fae3989ffc Merge pull request #1900 from Diapolo/optionsmodel_getters
move most explicit getters in optionsmodel to header
2012-10-11 00:40:20 -07:00
Wladimir J. van der Laan
089b10a28a Merge pull request #1911 from Diapolo/fix_signed_unsigned
fix wrong (signed/unsigned) printf format specifier in bitcoinrpc.cpp
2012-10-11 00:40:08 -07:00
Wladimir J. van der Laan
ac0ad5dc63 Merge pull request #1901 from laanwj/2012_10_remove_strlcpy
get rid of strlcpy.h
2012-10-11 00:39:51 -07:00
Gavin Andresen
eb49457ff2 0.7.1 release notes 2012-10-09 13:22:09 -04:00
Gavin Andresen
1d1f32d16d Bump versions for 0.7.1 release 2012-10-09 12:37:35 -04:00
Gavin Andresen
de038acd96 Fix bad merge, pszDataDir duplication 2012-10-09 12:28:00 -04:00
Gavin Andresen
673021410f Merge branch 'wallet_exceptions' of github.com:gavinandresen/bitcoin-git 2012-10-09 12:14:26 -04:00
Gavin Andresen
cf64347d70 Merge branch 'crash_at_exit' of github.com:gavinandresen/bitcoin-git 2012-10-09 12:10:18 -04:00
Wladimir J. van der Laan
0a5ce30bc1 Merge pull request #1915 from Diapolo/Qt5_compat_leftover
change Q_WS_MAC -> Q_OS_MAC (Qt5 compatibility)
2012-10-08 22:28:24 -07:00
Jeff Garzik
de2b9459bd Merge pull request #1834 from jgarzik/kickblocks
P2P: Do not request blocks from peers with fewer blocks than us
2012-10-08 15:15:58 -07:00