9e49db2 Make --enable-debug to pick better options (Evan Klitzke)
Pull request description:
Cherry-picked (and rebased) 94189645e67f364c4445d62e2b00c282d885cbbf from the "up for grabs" PR: "[build] Make --enable-debug pick better options" (#12695).
See previous review in #12695.
Tree-SHA512: a93cdadcf13e2ef8519acb1ce4f41ce95057a388347bb0a86a5c164dc7d0b0d14d4bb2a466082d5a100b8d50de65c605c40abaed555e8ea77c99e28800a34439
66b0b1b2a6 Add compile time checking for all cs_wallet runtime locking assertions (practicalswift)
Pull request description:
Add compile time checking for `cs_wallet` runtime locking assertions.
This PR is a subset of #12665. The PR was broken up to make reviewing easier.
The intention is that literally all `EXCLUSIVE_LOCKS_REQUIRED`/`LOCKS_EXCLUDED`:s added in this PR should follow either directly or indirectly from `AssertLockHeld(…)`/`AssertLockNotHeld(…)`:s already existing in the repo.
Consider the case where function `A(…)` contains `AssertLockHeld(cs_foo)` (without
first locking `cs_foo` in `A`), and that `B(…)` calls `A(…)` (without first locking `cs_main`):
* It _directly_ follows that: `A(…)` should have an `EXCLUSIVE_LOCKS_REQUIRED(cs_foo)` annotation.
* It _indirectly_ follows that: `B(…)` should have an `EXCLUSIVE_LOCKS_REQUIRED(cs_foo)` annotation.
Tree-SHA512: d561d89e98a823922107e56dbd493f0f82e22edac91e51e6422f17daf2b446a70c143b7b157ca618fadd33d0ec63eb7a57dde5a83bfdf1fc19d71459b43e21fd
b6f0b4d wallet: Improve logging when BerkeleyDB environment fails to close (Tim Ruffing)
264c643 wallet: Reset BerkeleyDB handle after connection fails (Tim Ruffing)
Pull request description:
According to the BerkeleyDB docs, the DbEnv handle may not be accessed
after close() has been called. This change ensures that we create a new
handle after close() is called. This avoids a segfault when the first
connection attempt fails and then a second connection attempt tries to
call open() on the already closed DbEnv handle.
Without the patch, bitcoindd reliably crashes in the second call to `set_lg_dir()` after `close()` if
there is an issue with the database:
```
2018-05-03T13:27:21Z Bitcoin Core version v0.16.99.0-a024a1841-dirty (debug build)
[...]
2018-05-03T13:27:21Z Using wallet directory /home/tim/.bitcoin
2018-05-03T13:27:21Z init message: Verifying wallet(s)...
2018-05-03T13:27:21Z Using BerkeleyDB version Berkeley DB 4.8.30: (April 9, 2010)
2018-05-03T13:27:21Z Using wallet wallet.dat
2018-05-03T13:27:21Z BerkeleyEnvironment::Open: LogDir=/home/tim/.bitcoin/database
2018-05-03T13:27:21Z BerkeleyEnvironment::Open: Error -30974 opening database environment: DB_RUNRECOVERY: Fatal error, run database recovery
2018-05-03T13:27:21Z Moved old /home/tim/.bitcoin/database to /home/tim/.bitcoin/database.1525354041.bak. Retrying.
2018-05-03T13:27:21Z BerkeleyEnvironment::Open: LogDir=/home/tim/.bitcoin/database ErrorFile=/home/tim/.bitcoin/db.log
[1] 14533 segmentation fault (core dumped) ./src/bitcoind
```
After the fix:
```
2018-05-03T17:19:32Z Bitcoin Core version v0.16.99.0-cc09e3bd0-dirty (release build)
[...]
2018-05-03T17:19:32Z Using wallet directory /home/tim/.bitcoin
2018-05-03T17:19:32Z init message: Verifying wallet(s)...
2018-05-03T17:19:32Z Using BerkeleyDB version Berkeley DB 4.8.30: (April 9, 2010)
2018-05-03T17:19:32Z Using wallet wallet.dat
2018-05-03T17:19:32Z BerkeleyEnvironment::Open: LogDir=/home/tim/.bitcoin/database ErrorFile=/home/tim/.bitcoin/db.log
2018-05-03T17:19:32Z scheduler thread start
2018-05-03T17:19:32Z BerkeleyEnvironment::Open: Error -30974 opening database environment: DB_RUNRECOVERY: Fatal error, run database recovery
2018-05-03T17:19:32Z Moved old /home/tim/.bitcoin/database to /home/tim/.bitcoin/database.1525367972.bak. Retrying.
2018-05-03T17:19:32Z BerkeleyEnvironment::Open: LogDir=/home/tim/.bitcoin/database ErrorFile=/home/tim/.bitcoin/db.log
2018-05-03T17:19:32Z Cache configuration:
2018-05-03T17:19:32Z * Using 2.0MiB for block index database
2018-05-03T17:19:32Z * Using 8.0MiB for chain state database
2018-05-03T17:19:32Z * Using 440.0MiB for in-memory UTXO set (plus up to 286.1MiB of unused mempool space)
2018-05-03T17:19:32Z init message: Loading block index..
[...]
```
Tree-SHA512: b809b318e5014ec47d023dc3dc40826b9706bfb211fa08bc2d29f36971b96caa10ad48d9a3f96c03933be46fa4ff7e00e952ac77bfffb6563767fb08aa4f23d6
a8da482 Bump wallet version for pre split keypool (Andrew Chow)
dfcd9f3 Use a keypool of presplit keys after upgrading to hd chain split (Andrew Chow)
5c50e93 Allow -upgradewallet to upgradewallets to HD (Andrew Chow)
2bcf2b5 Test sethdseed (Andrew Chow)
b5ba01a Add 'sethdseed' RPC to initialize or replace HD seed (Chris Moore)
dd3c07a Separate HaveKey function that checks whether a key is in a keystore (Andrew Chow)
Pull request description:
Revival/rebase of #11085
Adds a new command `sethdseed` which allows you to either set or generate a new HD seed to be used. A new keypool can be generated or the original one kept and new keys added to the keypool will come from the new HD seed.
Wallets that are not HD will be upgraded to be version FEATURE_HD_SPLIT when the `sethdseed` RPC command is used.
I have also add some tests for this.
Additionally `-upgradewallet` can now be used to upgrade a wallet from non-HD to HD. When it is used for such an upgrade, the keypool will be regenerated.
Tree-SHA512: e56c792e150590429ac4a1061e8d6f7b20cca06366e184eb9bbade4cd6ae82699a28fe84f87031eadba97ad2c1606517a105f00fb7b45779c979243020071adb
11fa6bb66e Bugfix: ensure consistency of m_failed_blocks after reconsiderblock (Suhas Daftuar)
Pull request description:
This was introduced in 015a5258ad and could cause a node to crash (due to assertion failure) when using the `reconsiderblock` rpc.
Tree-SHA512: 820dcd761bf983e36f5d0f16777ed75c833daaf62a6b3a4dbd17f6caaf9287223e3a202d06540ac62f8ba72926b73b0873bb76c6273ddcb19d9408f4c1cd325e
Bump the wallet version to indicate support for the pre split keypool.
Also prevents any wallets from upgrading to versions between HD_SPLIT
and PRE_SPLIT_KEYPOOL.
After upgrading to HD chain split, we want to continue to use keys
from the old keypool. To do this, before we generate any new keys after
upgrading, we mark all of the keypool entries as being pre-chain
split and move them to a separate pre chain split keypool. Keys are
fetched from that keypool until it is emptied. Only then are the new
internal and external keypools used.
Changes the maximum upgradewallet version to the latest wallet version
number, 159900. Non-HD wallets will be upgraded to use HD derivation.
Non HD chain split wallets will be upgraded to HD chain split.
If a non-HD wallet is upgraded to HD, the keypool will be entirely
regenerated.
Since upgradewallet is effectively run during a first run, all of the
first run initial setup stuff is combined with the upgrade to HD
364bae5 qa: Pad scriptPubKeys to get minimum sized txs (MarcoFalke)
7485488 Policy to reject extremely small transactions (Johnson Lau)
0f8719b Add transaction tests for constant scriptCode (Johnson Lau)
9dabfe4 Add constant scriptCode policy in non-segwit scripts (Johnson Lau)
Pull request description:
This disables `OP_CODESEPARATOR` in non-segwit scripts (even in an unexecuted branch), and makes a positive `FindAndDelete` result invalid. This ensures that the `scriptCode` serialized in `SignatureHash` is always the same as the script passing to the `EvalScript`.
Tree-SHA512: a0552cb920294d130251c48053fa2ff1fbdd26332e62b52147d918837852750f0ce35ce2cd1cbdb86588943312f8154ccb4925e850dbb7c2254bc353070cd5f8
a2f678d Bugfix: the end of a reorged chain is invalid when connect fails (Pieter Wuille)
Pull request description:
Introduced in 4e0eed88ac
When an invalid block is found during a reorg, we know the last of the blocks in the was-to-be-connected chain is invalid, but not necessarily the first. As `vpIndexToConnect` is ordered in decreasing height, the end of the reorg is the front of the vector, and not the back.
This only affected the warning system.
Tree-SHA512: ddf749f8a78083811a5a17152723f545c1463768d09dc9832ec3682e803a3c106fb768de9fa91c03aa95e644d4e41361a7e4ee791940fd7d51cdefea90de31fc
25b7ab9 doc: Add release notes for -includeconf (Karl-Johan Alm)
0f0badd test: Test includeconf parameter. (Karl-Johan Alm)
629ff8c -includeconf=<path> support in config handler, for including external configuration files (Karl-Johan Alm)
Pull request description:
Fixes: #10071.
Done:
- adds `-includeconf=<path>`, where `<path>` is relative to `datadir` or to the path of the file being read, if in a file
- protects against circular includes
- updates help docs
~~~Thoughts:~~~
- ~~~I am not sure how to test this in a neat manner. Feedback on this would be nice. Will dig/think though.~~~
Tree-SHA512: cb31f1b2f69fbc0890d264948eb2e501ac05cf12f5e06a5942f9c1539eb15ea8dc3cae817f4073aecb2fcc21d0386747f14f89d990772003a76e2a6d25642553
Introduced in 4e0eed88ac
When an invalid block is found during a reorg, we know the last of the blocks in
the was-to-be-connected chain is invalid, but not necessarily the first. As
vpIndexToConnect is ordered in decreasing height, the end of the reorg is the
front of the vector, and not the back.
This only affected the warning system.
16be13345 Fix rescanblockchain rpc to property report progress (Ben Woosley)
Pull request description:
Previously it assumed tip in all cases. This also extracts a RescanVerificationProgress helper object whose role is to manage reporting, in order to simplify ScanForWalletTransactions - more lines in total, but much simpler to follow the core logic.
Tree-SHA512: 5ebed0c56fae4ccfe613ff1d7082cb6da5a86635a8993ed3af70b500a4ea43074121aea9219b2f0321fbfeb7efcb964bdc2199297a64ca0fa85d9d07aa637d40
ddebde7 Add Windows shutdown handler (Chun Kuan Lee)
Pull request description:
Exit properly when clicked the red X of Windows Console
Tree-SHA512: f030edd08868390662b42abfa1dc6bd702166c6c19f5b1f8e7482e202451e79fb6f37ea672c26c2eb0d32c367bfca86160fbee624696c53828f280b7070be6a0
20ce5af Print a log message if we fail to shrink the debug log file (practicalswift)
29c9bdc Handle unsuccessful fseek(...):s (practicalswift)
Pull request description:
Handle unsuccessful `fseek(...)`:s.
**Note to reviewers:** What is the most appropriate course of actions for each of these unsuccessful `fseek(...)`:s?
Tree-SHA512: 5b3d82dbdd15d434d3f08dcb4df62888da4df8541d2586f56a4e529083005f6782c39e10645acd1ec403da83061bbfd8dbf2dddc66e09268d410ad0918c61876
43f3dec00 Remove enum specifier (to avoid re-declare scoped enum as unscoped) (donaloconnor)
Pull request description:
MSVC fails to compile with the changes made in #10742
The problem is enum types were changed to scoped (`enum class`) but in some places `enum` as an unscoped is used.
This is a very simple fix and I've tested it.
Edit: Had to remove enum altogether - `enum class` doesn't compile on clang.
Tree-SHA512: 13e21666243585a133c74c81249a1fa4098d6b7aa3cda06be871fa017c0ad9bb7b0725f801160b9d31678448d668718197941fd84702ebdef15128c27d92cd70
fad63eb [logging] Don't incorrectly log that REJECT messages are unknown. (John Newbery)
Pull request description:
Reject messages are logged to debug.log if NET debug logging is enabled.
Because of the way the `ProcessMessages()` function is structured,
processing for REJECT messages will also drop through to the default
branch and incorrectly log `Unknown command "reject" from peer-?`. Fix
that by exiting from `ProcessMessages()` early.
without this PR:
```
2018-05-03T17:37:00.930600Z received: reject (21 bytes) peer=0
2018-05-03T17:37:00.930620Z Reject message code 16: spammy spam
2018-05-03T17:37:00.930656Z Unknown command "reject" from peer=0
```
with this PR:
```
2018-05-03T17:35:04.751246Z received: reject (21 bytes) peer=0
2018-05-03T17:35:04.751274Z Reject message code 16: spammy spam
```
Tree-SHA512: 5c84c98433ab99e0db2dd481f9c2db6f87ff0d39022ff317a791737e918714bbcb4a23e81118212ed8e594ebcf098ab7f52f7fd5e21ebc3f07b1efb279b9b30b
A transaction with 1 segwit input and 1 P2WPHK output has non-witness size of 82 bytes. Anything smaller than this have unnecessary malloc overhead and are not relayed/mined.
This disables OP_CODESEPARATOR in non-segwit scripts (even in an unexecuted branch), and makes a positive FindAndDelete result invalid. This ensures that the scriptCode serialized in SignatureHash() is always the same as the script passing to the EvalScript.
Reject messages are logged to debug.log if NET debug logging is enabled.
Because of the way the `ProcessMessages()` function is structured,
processing for REJECT messages will also drop through to the default
branch and incorrectly log `Unknown command "reject" from peer-?`. Fix
that by exiting from `ProcessMessages()` early.
without this PR:
```
2018-05-03T17:37:00.930600Z received: reject (21 bytes) peer=0
2018-05-03T17:37:00.930620Z Reject message code 16: spammy spam
2018-05-03T17:37:00.930656Z Unknown command "reject" from peer=0
```
with this PR:
```
2018-05-03T17:35:04.751246Z received: reject (21 bytes) peer=0
2018-05-03T17:35:04.751274Z Reject message code 16: spammy spam
```
CWallet::ScanForWalletTransactions did not previously take into account
pindexStop when calculating progress.
Renamed progress vars to progress_*.
rescanblockchain is the only rpc that uses this parameter.
According to the BerkeleyDB docs, the DbEnv handle may not be accessed
after close() has been called. This change ensures that we create a new
handle after close() is called. This avoids a segfault when the first
connection attempt fails and then a second connection attempt tries to
call open() on the already closed DbEnv handle.
a59dac3 refactor: Avoid extra lookups of mapAddressBook in listunspent RPC (João Barbosa)
d76962e rpc: Reduce cs_main lock in listunspent (João Barbosa)
Pull request description:
On my system, where the wallet has 10000 unspents, the `cs_main` lock duration changed from 191ms to 36ms. The loop that generates the response takes around 155ms. So, the lock duration is reduced to around 20%.
Tree-SHA512: ddaae591f39da59a9d1a8e9ffe773d857687789476f566ca273d310ad531da6dacff80cac69f3334c601c251ac7c5ed4136656c725aa3d611c6bbf734111946e
1e46d8a Get rid of ambiguous OutputType::NONE value (Russell Yanofsky)
Pull request description:
Based on suggestion by @sipa https://github.com/bitcoin/bitcoin/pull/12119#issuecomment-357982763
After #12119, the NONE output type was overloaded to refer to either an output type that couldn't be parsed, or to an automatic change output mode. This change drops the NONE enum and uses a simple bool to indicate parse failure, and a new CHANGE_AUTO enum to refer the change output type.
This change is almost a pure refactoring except it makes RPCs reject empty string ("") address types instead of treating them like they were unset. This simplifies the parsing code a little bit and could prevent RPC usage mistakes. It's noted in the release notes.
Follows up #12408 by @MarcoFalke
Followups for future PRs:
- [ ] Add explicit support for specifying "auto" in `ParseOutputType` as suggested by promag and sipa: https://github.com/bitcoin/bitcoin/pull/12729#issuecomment-374799567 and https://github.com/bitcoin/bitcoin/pull/12729#discussion_r175969481
- [ ] Add wallet `AddressChangeType` method to complement `TransactionChangeType`: https://github.com/bitcoin/bitcoin/pull/12729#discussion_r175969618.
Tree-SHA512: 8b08b272bcb177a0a9e556dcd965840a7fe601ef83ca97938b879c9b1a33b5b3f96939e1bceef11ba7c644ac21bfd6c1dbc6ca715cd1da4ace50475240e4ee48
3fdc5fe Make sure initialization occurs in the constructor (practicalswift)
1e7813e Remove redundant initializations from the constructor (practicalswift)
f131872 Initialize non-static class members where they are defined (practicalswift)
73bc1b7 Initialize editStatus and autoCompleter. Previously not initialized where defined or in constructor. (practicalswift)
Pull request description:
Initialize variables previously neither defined where defined nor in constructor:
* `editStatus`
* `autoCompleter`
Also; initialize non-static class members where they are defined in accordance with developer notes.
Tree-SHA512: 84f0cb87ec8394ed7641bfa0731be2ec72e6a920e00ae206ff89e2e7c960358f603c52878311b24601a33aa7cba6ea4f9a78a8ade88112dea0f41efb08e84e25
9cb6cdc Simplify semantics of ChainStateFlushed callback (Matt Corallo)
50b6533 scripted-diff: Rename SetBestChain callback ChainStateFlushed (Matt Corallo)
Pull request description:
Previously, ChainStateFlushed would fire either if a full flush
completed (which can happen due to memory limits, forced flush, or
on its own DATABASE_WRITE_INTERVAL timer) *or* on a
ChainStateFlushed-specific DATABASE_WRITE_INTERVAL timer. This is
both less clear for clients (as there are no guarantees about a
flush having actually happened prior to the call), and reults in
extra flushes not clearly intended by the code. We drop the second
case, providing a strong guarantee without removing the periodit
timer-based flushing.
This is a follow-up to discussion in #11857.
Tree-SHA512: 22ba3a0954d265d28413dbf87040790ca5b439820ee7bbadab14028295ec190de82ce5cd664426c82e58b706dc84278868026fa8d066702eb6e6962c9ace1f8e
`LogPrintStr` returns the number of characters printed. This number is
doubled if both logging to console and logging to file is enabled. As
the return value is never used, I've opted to remove it instead of try
to fix it.
Credit: @laanwj
Wladimir J. van der Laan