Compare commits
34 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 6d70b41844 | |||
| 2b5471283e | |||
| dbde798221 | |||
| 68f5515588 | |||
| 891ad5ad25 | |||
| c02994c836 | |||
| 569ca99e66 | |||
| f13e512712 | |||
| b28525057a | |||
| b9d631e968 | |||
| d5473d7cae | |||
| cd51ba41d8 | |||
| aef95bdf78 | |||
| f633b9e330 | |||
| e7c5c6596a | |||
| 16b35f6b2b | |||
| 7faf13dc31 | |||
| db65324b7a | |||
| c98bdbe335 | |||
| 6f1227b022 | |||
| 12205cdc37 | |||
| 2fc0e8155a | |||
| eeda728564 | |||
| 0df054bbcb | |||
| fbd931a392 | |||
| a792f90489 | |||
| 64939a9793 | |||
| b506a48192 | |||
| dee0d9ef62 | |||
| 22e220acaa | |||
| 1c068f4782 | |||
| a671708f0b | |||
| be90d39cd4 | |||
| 4d0478add5 |
@@ -7,8 +7,12 @@
|
||||
*.a
|
||||
/dist/
|
||||
build/
|
||||
build2/
|
||||
build_*/
|
||||
release/
|
||||
debug/
|
||||
build_err*.txt
|
||||
*build_err.txt
|
||||
/Makefile
|
||||
Makefile.Debug
|
||||
Makefile.Release
|
||||
@@ -24,6 +28,7 @@ ui_*.h
|
||||
qrc_*.cpp
|
||||
*.pro.user
|
||||
*.pro.user.*
|
||||
*.qm
|
||||
|
||||
# Blockchain data
|
||||
*.dat
|
||||
|
||||
@@ -1,94 +0,0 @@
|
||||
# Triangles Codebase Cleanup Notes
|
||||
|
||||
## Overview
|
||||
Systematic code quality improvements for the Triangles cryptocurrency codebase (v5.3.4+).
|
||||
|
||||
**Goal:** Improve maintainability without changing behavior or breaking consensus.
|
||||
|
||||
## Inventory
|
||||
|
||||
### TODOs/FIXMEs Found (38 total)
|
||||
|
||||
#### High Priority (Affects Safety/Correctness)
|
||||
- `rpcmining.cpp:263` - **Thread safety issue** in mapNewBlock (static variable, no mutex)
|
||||
- `walletmodel.cpp:249` - **Potential collision** in balance calculation
|
||||
- `smessage.cpp:863, 2219, 2373` - **File size limit** (files must be split if >2GB)
|
||||
|
||||
#### Medium Priority (Encapsulation/Security)
|
||||
- `protocol.h:50, 100, 132` - Public members should be private (3 locations)
|
||||
- `wallet.h:378` - nOrderPos calculation should move elsewhere
|
||||
- `wallet.cpp:733, 1732` - Change output handling needs improvement
|
||||
- `rpcwallet.cpp:1474, 1513, 1569` - SecureString operator= missing (forced .c_str())
|
||||
|
||||
#### Low Priority (Nice-to-Have)
|
||||
- `util.cpp:1322` - Disabled feature needs verification
|
||||
- `tor/tor_embedded.cpp:209` - Tor 0.4.9+ shutdown API upgrade
|
||||
- `init.cpp:442` - Remaining sanity checks (see Bitcoin issue #4081)
|
||||
- `rpcmining.cpp:232` - DRM comment (unclear what it means)
|
||||
- `smessage.cpp:*` - Various improvements (hash inclusion, thread safety, defaults)
|
||||
- `qt/*` - UI improvements (decrypt not supported, message filtering, OSX startup)
|
||||
|
||||
#### External/Third-Party (Don't Touch)
|
||||
- `leveldb/*` - LevelDB library TODOs (upstream issues)
|
||||
|
||||
## Code Quality Issues
|
||||
|
||||
### Using namespace std (37 files)
|
||||
All in .cpp files - **this is fine for .cpp**, problematic only in headers.
|
||||
No headers have this issue, so **no action needed**.
|
||||
|
||||
### Printf/Cout Usage (56 files)
|
||||
Most cryptocurrency code uses printf for early init/error handling before logging is available.
|
||||
**Review needed:** Check if these are legitimate early-init cases or should use LogPrintf.
|
||||
|
||||
## Cleanup Plan (Safest → Riskiest)
|
||||
|
||||
### Phase 1: Documentation & Comments ✅ SAFE
|
||||
1. Document all TODOs with context (why deferred, what's needed)
|
||||
2. Add function-level comments for complex logic
|
||||
3. Improve inline comments for clarity
|
||||
|
||||
### Phase 2: Low-Risk Code Quality 🟨 MEDIUM RISK
|
||||
4. Fix compiler warnings (-Wall -Wextra)
|
||||
5. Add const correctness where missing
|
||||
6. Remove commented-out dead code
|
||||
7. Standardize code formatting (if inconsistent)
|
||||
|
||||
### Phase 3: Functional Improvements 🟥 HIGH RISK (Skip for now)
|
||||
8. Fix thread safety issue in rpcmining.cpp (requires testing)
|
||||
9. Improve protocol.h encapsulation (may affect other code)
|
||||
10. Address >2GB file handling in smessage.cpp
|
||||
|
||||
## Decisions
|
||||
|
||||
### What NOT to Change
|
||||
- **Consensus code** - main.cpp (validation), kernel.cpp (PoS), miner.cpp (staking)
|
||||
- **Serialization** - Any READWRITE, serialize/deserialize code
|
||||
- **Protocol constants** - Network message types, version numbers
|
||||
- **Third-party code** - leveldb/, tor/, sph_types.h, xxhash/, lz4/
|
||||
|
||||
### What's Safe to Change
|
||||
- Comments and documentation
|
||||
- Variable names (in non-consensus code)
|
||||
- Code organization (splitting large functions)
|
||||
- Logging statements
|
||||
- UI code (qt/)
|
||||
- RPC interface (as long as API contract preserved)
|
||||
|
||||
## Initial Cleanup (2026-03-22)
|
||||
|
||||
### Actions Taken
|
||||
1. Created this documentation file
|
||||
2. Created cleanup/desloppify branch
|
||||
3. Inventoried all TODOs/FIXMEs
|
||||
|
||||
### Next Steps
|
||||
1. Add documentation comments to TODO items
|
||||
2. Review printf/cout usage patterns
|
||||
3. Check for compiler warnings
|
||||
4. Consider low-risk improvements
|
||||
|
||||
## Notes
|
||||
- This is a Bitcoin-derived codebase, so many patterns follow Bitcoin Core conventions
|
||||
- Recent v5.3.x work already modernized to C++17 and removed Boost - good foundation
|
||||
- Code is generally well-structured; main improvements are documentation and minor cleanup
|
||||
@@ -1,76 +0,0 @@
|
||||
# Triangles Cleanup Strategy - Safe Improvements
|
||||
|
||||
**Branch:** `cleanup/safe-improvements`
|
||||
**Goal:** Improve code quality without touching consensus-critical code
|
||||
|
||||
## ✅ SAFE TO FIX
|
||||
|
||||
### 1. Compiler Warnings (Non-Consensus)
|
||||
- **C++11 literal-suffix warnings** - Add spaces between literals and suffixes
|
||||
- **Unused variables/functions** - Remove dead code (verify not consensus-critical first)
|
||||
- **Deprecated-copy warnings** - Fix CScript assignment operator if safe
|
||||
|
||||
### 2. Code Style Improvements
|
||||
- Remove `using namespace std` from headers (keep in .cpp files)
|
||||
- Standardize logging patterns
|
||||
- Improve code comments (remove unclear/misleading ones)
|
||||
- Add context to TODOs/FIXMEs
|
||||
|
||||
### 3. Documentation
|
||||
- Add inline comments for thread safety concerns
|
||||
- Document collision vulnerabilities
|
||||
- Improve function/class documentation
|
||||
|
||||
## ❌ DO NOT TOUCH
|
||||
|
||||
### Consensus-Critical Code
|
||||
- **OpenSSL SHA256/RIPEMD160 usage** - Deprecated warnings OK, do not change
|
||||
- **BN_is_prime_ex** - Crypto library deprecation, leave as-is
|
||||
- **Hash algorithms** - Third-party libraries with warnings, consensus-critical
|
||||
- **Block validation logic** - Any code affecting block/transaction validation
|
||||
- **Merkle tree construction** - Core consensus
|
||||
- **Proof-of-Work/Proof-of-Stake** - Staking/mining algorithms
|
||||
|
||||
### How to Identify Consensus Code
|
||||
- Files in `src/` related to: `main.cpp`, `main.h`, block validation, transaction validation
|
||||
- Anything in hash algorithm libraries
|
||||
- Cryptographic primitives
|
||||
- Network protocol message formats (version, serialization)
|
||||
|
||||
## Incremental Testing Strategy
|
||||
|
||||
1. **One warning category at a time**
|
||||
2. **Compile after each change**
|
||||
3. **Test basic functionality:**
|
||||
- `trianglesd getinfo`
|
||||
- `trianglesd getblockchaininfo`
|
||||
- Verify block sync works
|
||||
4. **Commit incrementally** with clear messages
|
||||
|
||||
## Warning Categories (From Build Output)
|
||||
|
||||
```
|
||||
1. C++11 literal-suffix: ~20 instances (util.h, net.h, alert.cpp)
|
||||
2. OpenSSL deprecation: SHA256, RIPEMD160 (DO NOT FIX)
|
||||
3. BN_is_prime_ex: crypto library (DO NOT FIX)
|
||||
4. Deprecated-copy: CScript assignment (REVIEW CAREFULLY)
|
||||
5. Unused variables/functions: Various (SAFE IF NOT CONSENSUS)
|
||||
```
|
||||
|
||||
## Branch History
|
||||
|
||||
- Previous work: `cleanup/desloppify` (documentation improvements, merged to master)
|
||||
- This branch: Focus on safe compiler warnings and code quality
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
Before pushing each commit:
|
||||
- [ ] Code compiles successfully
|
||||
- [ ] No new warnings introduced
|
||||
- [ ] trianglesd runs without errors
|
||||
- [ ] getinfo/getblockchaininfo work
|
||||
- [ ] No consensus-critical code touched
|
||||
|
||||
---
|
||||
|
||||
**Principle:** When in doubt, don't touch it. A clean codebase is worthless if the blockchain forks.
|
||||
+36
-1
@@ -6,7 +6,7 @@ if(POLICY CMP0167)
|
||||
endif()
|
||||
|
||||
project(Triangles
|
||||
VERSION 5.8.2
|
||||
VERSION 5.9.5
|
||||
DESCRIPTION "Cryptographic Triangles Wallet"
|
||||
LANGUAGES C CXX
|
||||
)
|
||||
@@ -17,6 +17,24 @@ set(CMAKE_CXX_STANDARD_REQUIRED ON)
|
||||
set(CMAKE_CXX_EXTENSIONS OFF)
|
||||
set(CMAKE_C_STANDARD 11)
|
||||
|
||||
# ── Build acceleration ──
|
||||
# ccache: auto-detect and use if available
|
||||
find_program(CCACHE_PROGRAM ccache)
|
||||
if(CCACHE_PROGRAM)
|
||||
set(CMAKE_C_COMPILER_LAUNCHER "${CCACHE_PROGRAM}")
|
||||
set(CMAKE_CXX_COMPILER_LAUNCHER "${CCACHE_PROGRAM}")
|
||||
message(STATUS "ccache found: ${CCACHE_PROGRAM}")
|
||||
else()
|
||||
message(STATUS "ccache not found — install it for faster rebuilds")
|
||||
endif()
|
||||
|
||||
# Unity (jumbo) build: batch source files to reduce header parsing overhead
|
||||
option(ENABLE_UNITY_BUILD "Enable CMake unity (jumbo) builds" OFF)
|
||||
if(ENABLE_UNITY_BUILD)
|
||||
set(CMAKE_UNITY_BUILD ON)
|
||||
set(CMAKE_UNITY_BUILD_BATCH_SIZE 8)
|
||||
endif()
|
||||
|
||||
# ── Output directories ──
|
||||
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/bin")
|
||||
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/lib")
|
||||
@@ -34,6 +52,7 @@ option(USE_QRCODE "Enable QR code generation via libqrencode" OFF
|
||||
option(USE_DBUS "Enable D-Bus notifications (Linux only)" ON)
|
||||
option(USE_ZMQ "Enable ZMQ publisher support" OFF)
|
||||
option(USE_TOR_EMBEDDED "Enable embedded Tor library linking" OFF)
|
||||
option(BUILD_ROCKSDB "Build with RocksDB chain database backend" OFF)
|
||||
option(USE_O3 "Use -O3 optimization instead of -O2" OFF)
|
||||
option(ENABLE_PIE "Build position-independent executables" OFF)
|
||||
option(ENABLE_STATIC "Prefer static linking (Linux release builds)" OFF)
|
||||
@@ -77,6 +96,18 @@ if(USE_ZMQ)
|
||||
pkg_check_modules(ZMQ REQUIRED IMPORTED_TARGET libzmq)
|
||||
endif()
|
||||
|
||||
if(BUILD_ROCKSDB)
|
||||
# RocksDB ships a CMake config package on most distros (rocksdbConfig.cmake).
|
||||
# On MSYS2/Homebrew/vcpkg the imported target is RocksDB::rocksdb.
|
||||
find_package(RocksDB CONFIG)
|
||||
if(NOT RocksDB_FOUND)
|
||||
# Fall back to pkg-config for systems without the CMake config (older
|
||||
# Linux distros). Builds an IMPORTED target named PkgConfig::RocksDB.
|
||||
find_package(PkgConfig REQUIRED)
|
||||
pkg_check_modules(RocksDB REQUIRED IMPORTED_TARGET rocksdb)
|
||||
endif()
|
||||
endif()
|
||||
|
||||
if(BUILD_QT)
|
||||
find_package(Qt5 5.9 REQUIRED COMPONENTS Core Gui Widgets)
|
||||
find_package(Qt5 COMPONENTS LinguistTools QUIET)
|
||||
@@ -112,5 +143,9 @@ message(STATUS " QR code: ${USE_QRCODE}")
|
||||
message(STATUS " D-Bus: ${USE_DBUS}")
|
||||
message(STATUS " ZMQ: ${USE_ZMQ}")
|
||||
message(STATUS " Embedded Tor: ${USE_TOR_EMBEDDED}")
|
||||
message(STATUS " RocksDB backend: ${BUILD_ROCKSDB}")
|
||||
message(STATUS " Static linking: ${ENABLE_STATIC}")
|
||||
message(STATUS " ccache: ${CCACHE_PROGRAM}")
|
||||
message(STATUS " Unity build: ${ENABLE_UNITY_BUILD}")
|
||||
message(STATUS " Precompiled header: ON")
|
||||
message(STATUS "")
|
||||
|
||||
@@ -1,220 +0,0 @@
|
||||
# Embedded Tor Integration Guide for Triangles
|
||||
|
||||
This guide explains how to compile Tor as a static library (`libtor.a`) and link
|
||||
it directly into the Triangles wallet binary so that every node automatically
|
||||
runs a Tor hidden service without needing an external Tor installation.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
trianglesd / triangles-qt
|
||||
├── tor_embedded.cpp ← calls tor_run_main() in a background thread
|
||||
├── tor_process.cpp ← fallback: launches external tor binary (already works)
|
||||
├── onion_v3.cpp ← V3 onion address generation / SOCKS5 proxy logic
|
||||
└── libtor.a ← aggregate static Tor library (built from official source)
|
||||
```
|
||||
|
||||
When compiled with `ENABLE_TOR_EMBEDDED`, the wallet calls `tor_run_main()` from
|
||||
`tor_api.h` on a dedicated thread. This gives the wallet a SOCKS5 proxy on
|
||||
`127.0.0.1:19099` and a V3 hidden service on port 24112 (the P2P port).
|
||||
|
||||
When compiled **without** the flag, `tor_embedded.cpp` falls back to the external
|
||||
`tor_process.cpp` which searches for and launches a system `tor` binary.
|
||||
|
||||
## Step 1: Add Tor as a Git Submodule
|
||||
|
||||
```bash
|
||||
cd /path/to/triangles
|
||||
git submodule add https://gitlab.torproject.org/tpo/core/tor.git src/tor/tor-src
|
||||
cd src/tor/tor-src
|
||||
git checkout release-0.4.9 # latest stable branch as of 2026
|
||||
```
|
||||
|
||||
This puts the full Tor source at `src/tor/tor-src/`.
|
||||
Current imported checkout in this repo: `release-0.4.9` at commit `1442ca4`.
|
||||
There is also a helper build script at `src/tor/build-libtor.sh`.
|
||||
|
||||
## Step 2: Build libtor.a
|
||||
|
||||
Tor uses autotools. Build it as a static library:
|
||||
|
||||
```bash
|
||||
cd src/tor/tor-src
|
||||
|
||||
# Install Tor build dependencies
|
||||
sudo apt install autoconf automake libtool pkg-config \
|
||||
libssl-dev libevent-dev zlib1g-dev
|
||||
|
||||
# Generate configure script
|
||||
./autogen.sh
|
||||
|
||||
# Configure for static library build (disable unneeded modules)
|
||||
./configure \
|
||||
--enable-static-tor \
|
||||
--disable-module-relay \
|
||||
--disable-module-dirauth \
|
||||
--disable-asciidoc \
|
||||
--disable-manpage \
|
||||
--disable-html-manual \
|
||||
--disable-unittests \
|
||||
--disable-tool-name-check \
|
||||
--with-openssl-dir=/usr \
|
||||
--with-libevent-dir=/usr \
|
||||
--with-zlib-dir=/usr \
|
||||
--prefix=/usr/local
|
||||
|
||||
make -j$(nproc)
|
||||
```
|
||||
|
||||
Or from the repo root:
|
||||
```bash
|
||||
./src/tor/build-libtor.sh
|
||||
```
|
||||
|
||||
After building, the static libraries are in `src/tor/tor-src/`:
|
||||
- `libtor.a`
|
||||
- `src/lib/libtor-*.a` (multiple component libs)
|
||||
|
||||
The header `src/feature/api/tor_api.h` provides the public C API:
|
||||
```c
|
||||
tor_main_configuration_t *tor_main_configuration_new(void);
|
||||
int tor_main_configuration_set_command_line(tor_main_configuration_t *cfg,
|
||||
int argc, char *argv[]);
|
||||
int tor_run_main(const tor_main_configuration_t *);
|
||||
void tor_main_configuration_free(tor_main_configuration_t *);
|
||||
```
|
||||
|
||||
## Step 3: Build Triangles with Embedded Tor
|
||||
|
||||
### Linux (makefile.unix)
|
||||
|
||||
```bash
|
||||
cd src
|
||||
|
||||
# Point to Tor's built libraries and headers
|
||||
make -f makefile.unix \
|
||||
USE_TOR_EMBEDDED=1
|
||||
```
|
||||
|
||||
You may need to adjust the `-l` flags in the makefile depending on the exact
|
||||
library names Tor produces. Check `src/tor/tor-src/` after building:
|
||||
|
||||
```bash
|
||||
find tor/tor-src -name '*.a' | sort
|
||||
```
|
||||
|
||||
On the imported `release-0.4.9` checkout in this repo, the simplest working
|
||||
link path is the aggregate `libtor.a` plus the normal dependency libraries.
|
||||
|
||||
### Windows (triangles-qt.pro)
|
||||
|
||||
Add to `triangles-qt.pro`:
|
||||
```qmake
|
||||
qmake "USE_TOR_EMBEDDED=1" \
|
||||
"TOR_SOURCE_ROOT=src/tor/tor-src"
|
||||
```
|
||||
|
||||
Both build systems now default to:
|
||||
- source root: `src/tor/tor-src`
|
||||
- include path: `src/tor/tor-src/src/feature/api`
|
||||
- library path: `src/tor/tor-src`
|
||||
- embedded Tor library: `-ltor`
|
||||
|
||||
On Windows, the imported Tor `0.4.9.5` build also needed:
|
||||
- `-llzma`
|
||||
- `-lzstd`
|
||||
- `-liphlpapi`
|
||||
- `-lshlwapi` (already linked by Triangles)
|
||||
|
||||
## Step 4: Wire into init.cpp
|
||||
|
||||
The global hooks `StartEmbeddedTor()` and `StopEmbeddedTor()` need to be called
|
||||
from `init.cpp`. Add these calls:
|
||||
|
||||
### In AppInit2() (after network init, before starting node):
|
||||
```cpp
|
||||
#include "tor/tor_embedded.h"
|
||||
|
||||
// Near the end of AppInit2, after network initialization:
|
||||
if (!StartEmbeddedTor()) {
|
||||
printf("WARNING: Embedded Tor failed to start. .onion connectivity unavailable.\n");
|
||||
// Non-fatal: wallet works without Tor, just no .onion
|
||||
}
|
||||
```
|
||||
|
||||
### In Shutdown():
|
||||
```cpp
|
||||
StopEmbeddedTor();
|
||||
```
|
||||
|
||||
## Step 5: Configure SOCKS Proxy for Outbound Connections
|
||||
|
||||
After Tor starts, the wallet needs to route `.onion` connections through the
|
||||
SOCKS5 proxy. In `net.cpp`, after Tor is initialized:
|
||||
|
||||
```cpp
|
||||
// If embedded Tor is running, use its SOCKS proxy for .onion addresses
|
||||
CTorEmbedded* tor = CTorEmbedded::GetInstance();
|
||||
if (tor->IsRunning()) {
|
||||
// Set proxy for .onion connections
|
||||
proxyType addrProxy(CService("127.0.0.1", tor->GetSocksPort()), 5);
|
||||
SetNameProxy(addrProxy);
|
||||
}
|
||||
```
|
||||
|
||||
## Runtime Flags
|
||||
|
||||
The embedded Tor respects these command-line flags:
|
||||
|
||||
| Flag | Default | Description |
|
||||
|------|---------|-------------|
|
||||
| `-notor` | false | Disable Tor entirely |
|
||||
| `-torsocks=PORT` | 19099 | SOCKS5 proxy port |
|
||||
| `-torhsport=PORT` | 24112 | Hidden service virtual port |
|
||||
|
||||
## File Layout After Integration
|
||||
|
||||
```
|
||||
src/tor/
|
||||
├── tor-src/ ← git submodule (official Tor repo)
|
||||
│ └── src/
|
||||
│ ├── lib/libtor-*.a
|
||||
│ └── feature/api/tor_api.h
|
||||
│ └── libtor.a
|
||||
├── tor_embedded.h ← CTorEmbedded class header
|
||||
├── tor_embedded.cpp ← implementation (calls tor_run_main)
|
||||
├── tor_process.h ← external Tor process manager (fallback)
|
||||
├── tor_process.cpp
|
||||
├── onion_v3.h ← V3 onion address utilities
|
||||
├── onion_v3.cpp
|
||||
├── anonymize.h ← data dir helpers
|
||||
├── anonymize.cpp
|
||||
└── LICENSE
|
||||
```
|
||||
|
||||
## Reference: How VERGE (XVG) Does It
|
||||
|
||||
VERGE uses the same pattern. Their implementation is at:
|
||||
- `src/torcontroller.cpp` (~100 lines)
|
||||
- They use `tor_main()` (older API, pre-0.4.5)
|
||||
- Git submodule at `src/tor/` pointing to `release-0.4.8` branch
|
||||
- Build Tor as part of their `depends/` system
|
||||
|
||||
Key difference: modern Tor (0.4.5+) uses `tor_run_main()` with a configuration
|
||||
object instead of raw `tor_main(int argc, char** argv)`.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Tor fails to bootstrap**: Check firewall rules. Tor needs outbound TCP to the
|
||||
Tor network (ports 80, 443, 9001, 9030).
|
||||
|
||||
**Link errors with libtor**: Prefer the aggregate `libtor.a` from the top level
|
||||
of the Tor build tree. On the imported Windows/MSYS2 build in this repo, the
|
||||
minimal verified link set was:
|
||||
```
|
||||
-ltor -levent -lssl -lcrypto -lz -llzma -lzstd -lws2_32 -liphlpapi -lshlwapi
|
||||
```
|
||||
|
||||
**OpenSSL version mismatch**: Both Tor and Triangles must link against the same
|
||||
OpenSSL version (3.x). If Tor was built against a different OpenSSL, rebuild it
|
||||
with the same `--with-openssl-dir`.
|
||||
@@ -1,210 +0,0 @@
|
||||
# Triangles Modernization Roadmap
|
||||
|
||||
**Goal:** Make TRI faster to sync, safer for wallets, and more useful as a currency — without breaking consensus.
|
||||
|
||||
**Invariant:** Any change that modifies block validation, stake modifier computation, transaction format, or signature verification MUST preserve exact consensus with existing v5.x nodes. When in doubt, test against a synced v5.8.1 node.
|
||||
|
||||
---
|
||||
|
||||
## Priority 1: Faster Syncing (High Impact, Low Risk)
|
||||
|
||||
### 1.1 Update Checkpoints (Easy, Immediate)
|
||||
**Problem:** Last hardcoded checkpoint is at block 2,186,940. `IsInitialBlockDownload()` returns false past this point, causing orphan limit to drop from 4000 to 750 — exactly what caused the fork deadlock.
|
||||
|
||||
**Fix:** Add checkpoints every ~50,000 blocks up to current height (~2,207,000+).
|
||||
```cpp
|
||||
// src/checkpoints.cpp - add recent checkpoints
|
||||
{2190000, uint256("...")},
|
||||
{2195000, uint256("...")},
|
||||
{2200000, uint256("...")},
|
||||
{2205000, uint256("...")},
|
||||
{2210000, uint256("...")},
|
||||
```
|
||||
**Risk:** None — checkpoints are only used for IBD detection and quick rejection of clearly wrong chains.
|
||||
|
||||
### 1.2 Increase Post-Checkpoint Orphan Limit (Easy)
|
||||
**Problem:** 750 orphans after IBD is too low for a low-peer network. During the fork incident, 750 orphans filled up and the node deadlocked.
|
||||
|
||||
**Fix:**
|
||||
```cpp
|
||||
// src/main.h
|
||||
static const unsigned int MAX_ORPHAN_BLOCKS = 2000; // was 750
|
||||
```
|
||||
**Risk:** Slightly more memory usage during forks. Worth it for resilience.
|
||||
|
||||
### 1.3 Parallel Block Download (Medium Effort)
|
||||
**Problem:** Current implementation downloads blocks sequentially from one peer at a time during IBD.
|
||||
|
||||
**Fix:** Increase batch sizes and allow concurrent block downloads from multiple peers:
|
||||
```cpp
|
||||
// src/main.cpp
|
||||
// During IBD, request blocks from multiple peers simultaneously
|
||||
unsigned int nGetDataBatchSize = IsInitialBlockDownload() ? 8000 : 1000; // was 4000
|
||||
```
|
||||
**Risk:** Low — larger batch sizes are already proven in Bitcoin forks.
|
||||
|
||||
### 1.4 Header-First Sync (Medium Effort)
|
||||
**Problem:** Node downloads full blocks before validating headers. A bad peer can waste bandwidth.
|
||||
|
||||
**Fix:** Download and validate all headers first (compact ~80 bytes each), then download full blocks only for the best chain.
|
||||
- Separate `getheaders`/`headers` message handling
|
||||
- Download blocks only for the best header chain
|
||||
- Reduces wasted bandwidth during forks by 95%+
|
||||
|
||||
### 1.5 Bootstrap Over HTTPS with Resume (Easy)
|
||||
**Problem:** Built-in bootstrap (`-bootstrap`) uses raw TCP and can't resume interrupted downloads.
|
||||
|
||||
**Fix:** The existing `bootstrap.cpp` already supports downloading. Add:
|
||||
- Resume support (Range headers)
|
||||
- SHA256 verification of downloaded archive
|
||||
- Better progress reporting
|
||||
- Fallback mirrors
|
||||
|
||||
---
|
||||
|
||||
## Priority 2: Wallet Safety (Critical)
|
||||
|
||||
### 2.1 Automatic Wallet Backup Before Dangerous Operations (Easy)
|
||||
**Problem:** Corrupt wallet = lost funds. No automatic backup before risky operations.
|
||||
|
||||
**Fix:** In `walletdb.cpp`, before any rewrite:
|
||||
```cpp
|
||||
// Before wallet.dat rewrite, copy to wallet.dat.bak
|
||||
if (boost::filesystem::exists(pathWallet)) {
|
||||
boost::filesystem::copy_file(pathWallet, pathWallet + ".bak",
|
||||
boost::filesystem::copy_option::overwrite_if_exists);
|
||||
}
|
||||
```
|
||||
|
||||
### 2.2 Detect and Report BDB Corruption (Easy)
|
||||
**Problem:** BDB corruption silently corrupts wallet. User doesn't know until it's too late.
|
||||
|
||||
**Fix:** Add wallet integrity check on load:
|
||||
```cpp
|
||||
// In CWallet::LoadWallet()
|
||||
// After opening, verify BDB environment is healthy
|
||||
// If DB_RUNRECOVERY, auto-salvage and warn user
|
||||
```
|
||||
|
||||
### 2.3 Wallet.dat Versioning (Medium Effort)
|
||||
**Problem:** Single wallet.dat file. If it corrupts during write, funds are lost.
|
||||
|
||||
**Fix:** Implement copy-on-write wallet saves:
|
||||
- Write new wallet data to `wallet.dat.new`
|
||||
- Atomically rename `wallet.dat` → `wallet.dat.old`, `wallet.dat.new` → `wallet.dat`
|
||||
- Keep last 3 wallet revisions
|
||||
- On load, try wallet.dat first, fall back to wallet.dat.old if corrupt
|
||||
|
||||
### 2.4 Seed Phrase / HD Wallet (High Effort, High Impact)
|
||||
**Problem:** Losing wallet.dat = losing everything. No recovery mechanism.
|
||||
|
||||
**Fix:** Implement BIP39/BIP44 HD wallet as optional upgrade:
|
||||
- Generate 12/24-word seed phrase on new wallet creation
|
||||
- Derive all keys from seed deterministically
|
||||
- Import seed on any device to recover wallet
|
||||
- Keep backward compatibility with existing non-HD wallets
|
||||
|
||||
---
|
||||
|
||||
## Priority 3: Network Resilience (Medium Impact)
|
||||
|
||||
### 3.1 Better Peer Management (Medium Effort)
|
||||
**Problem:** Low peer counts (2-6) lead to fork divergence. No prioritization of reliable peers.
|
||||
|
||||
**Fix:**
|
||||
- Peer reliability scoring (track which peers provide valid blocks)
|
||||
- Prefer peers that are ahead and on the same chain
|
||||
- Automatic disconnection of stale/forked peers
|
||||
- Increase default `maxconnections` from 64 to 128
|
||||
|
||||
### 3.2 Compact Block Relay (High Effort)
|
||||
**Problem:** Full blocks are sent even when the receiver likely already has most transactions.
|
||||
|
||||
**Fix:** Implement BIP 152 compact blocks:
|
||||
- Send block header + short transaction IDs
|
||||
- Receiver fills in from mempool, only requests missing transactions
|
||||
- Reduces bandwidth by ~90% during normal operation
|
||||
|
||||
### 3.3 DNS Seed Infrastructure (Easy)
|
||||
**Problem:** `dnsseed=0` when Tor-only means no automatic peer discovery.
|
||||
|
||||
**Fix:** Run a DNS seed server that resolves to known reliable onion addresses:
|
||||
```
|
||||
seed.cryptographic-triangles.org → returns onion addresses of healthy nodes
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Priority 4: User Experience (Medium Impact)
|
||||
|
||||
### 4.1 Progress Reporting for IBD (Easy)
|
||||
**Problem:** Users see "downloading blocks..." with no useful progress indicator.
|
||||
|
||||
**Fix:**
|
||||
- Report `headers` vs `blocks` progress separately
|
||||
- Show estimated time remaining based on download speed
|
||||
- Log progress every 1000 blocks (currently every 5000)
|
||||
- Qt wallet: update progress bar more frequently
|
||||
|
||||
### 4.2 Staking Dashboard Improvements (Easy)
|
||||
**Problem:** Qt wallet shows staking info but not clearly.
|
||||
|
||||
**Fix:**
|
||||
- Show expected time to stake more prominently
|
||||
- Display staking weight as percentage of network
|
||||
- Notify when stake is found (system notification)
|
||||
- Show "staking" indicator in system tray
|
||||
|
||||
### 4.3 Transaction Fee Estimation (Medium Effort)
|
||||
**Problem:** No fee estimation. Users guess.
|
||||
|
||||
**Fix:** Track recent block inclusion rates by fee level, provide fee recommendations.
|
||||
|
||||
---
|
||||
|
||||
## Priority 5: Code Modernization (Low Urgency, Good Hygiene)
|
||||
|
||||
### 5.1 C++17/20 Features
|
||||
- Replace raw pointers with smart pointers where safe
|
||||
- Use `std::optional`, `std::string_view`, `std::filesystem`
|
||||
- Replace boost::filesystem with std::filesystem (C++17)
|
||||
|
||||
### 5.2 Build System
|
||||
- CMake is already in place (good)
|
||||
- Add sanitizers (ASAN, UBSAN) to CI
|
||||
- Static analysis with clang-tidy
|
||||
|
||||
### 5.3 Testing
|
||||
- Current test coverage is thin
|
||||
- Add unit tests for:
|
||||
- Checkpoint validation
|
||||
- Stake modifier computation
|
||||
- Bootstrap download/resume
|
||||
- Wallet BDB recovery
|
||||
- Orphan block handling
|
||||
|
||||
---
|
||||
|
||||
## What NOT to Change
|
||||
|
||||
These are consensus-critical and must remain identical:
|
||||
- Block validation rules
|
||||
- Stake modifier computation (`ComputeNextStakeModifier`)
|
||||
- Transaction signature verification
|
||||
- Block reward schedule
|
||||
- PoW/PoS target computation
|
||||
- Chain trust / difficulty adjustment
|
||||
- Message serialization format
|
||||
- Protocol version handshaking
|
||||
|
||||
Any change to these requires a coordinated network upgrade (hard fork).
|
||||
|
||||
---
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. **This week:** Update checkpoints (1.1), increase orphan limit (1.2), wallet backup before save (2.1)
|
||||
2. **Next week:** Better progress reporting (4.1), increase batch size (1.3)
|
||||
3. **Month 1:** Wallet versioning (2.3), bootstrap resume (1.5)
|
||||
4. **Month 2:** Header-first sync (1.4), peer reliability (3.1)
|
||||
5. **Month 3+:** HD wallet (2.4), compact blocks (3.2)
|
||||
@@ -1,300 +0,0 @@
|
||||
# OpenClaw Bootstrap Snapshot Guide
|
||||
|
||||
## Purpose
|
||||
|
||||
This document tells OpenClaw exactly how to update the existing Triangles bootstrap server so new wallets download a ready-to-use snapshot instead of downloading `blk0001.dat` and rebuilding the index locally.
|
||||
|
||||
This guide matches the current wallet code in:
|
||||
|
||||
- `src/bootstrap.cpp`
|
||||
- `src/bootstrap.h`
|
||||
- `src/checkpoints.cpp`
|
||||
- `src/version.h`
|
||||
|
||||
## What The Wallet Actually Does
|
||||
|
||||
When a fresh wallet bootstraps, it:
|
||||
|
||||
1. Downloads `http://bootstrap.cryptographic-triangles.org/bootstrap.tar.gz`
|
||||
2. Extracts it into the data directory
|
||||
3. Requires `blk0001.dat` to exist after extraction
|
||||
4. Looks for `txleveldb/` and `snapshot.manifest`
|
||||
5. Keeps `txleveldb/` only if `snapshot.manifest` passes verification
|
||||
6. Deletes `txleveldb/` if verification fails, then rebuilds from `blk0001.dat`
|
||||
7. Always deletes `database/` from the extracted snapshot
|
||||
|
||||
The verification rules are strict:
|
||||
|
||||
- `format` must be `1`
|
||||
- `network` must be `main` on mainnet
|
||||
- `dbversion` must be `70509`
|
||||
- `height` and `hash` must exactly match a hardcoded checkpoint
|
||||
|
||||
If any of those checks fail, the wallet throws away the shipped `txleveldb/`.
|
||||
|
||||
## Current Hardcoded Mainnet Checkpoint
|
||||
|
||||
As of the current codebase, the latest hardcoded mainnet checkpoint is:
|
||||
|
||||
- Height: `2186940`
|
||||
- Hash: `bd952e8d4a612e336d840ad924a7e09395e36bcd9d929b302e47e60b5c3098c0`
|
||||
|
||||
OpenClaw must not generate a manifest with an arbitrary tip hash. The manifest only survives if it matches a hardcoded checkpoint from `src/checkpoints.cpp`.
|
||||
|
||||
## Important Limitation
|
||||
|
||||
If the live chain tip is past the latest hardcoded checkpoint, OpenClaw has two valid options:
|
||||
|
||||
1. Publish a snapshot taken exactly at the latest hardcoded checkpoint
|
||||
2. Publish `blk0001.dat` only, without `txleveldb/`, and let clients rebuild locally
|
||||
|
||||
OpenClaw must not publish a `snapshot.manifest` for a height/hash that is not compiled into the wallet.
|
||||
|
||||
## Files OpenClaw Should Publish
|
||||
|
||||
The preferred `bootstrap.tar.gz` should contain:
|
||||
|
||||
- `blk0001.dat`
|
||||
- `txleveldb/`
|
||||
- `snapshot.manifest`
|
||||
- optionally `peers.dat`
|
||||
|
||||
It must not contain:
|
||||
|
||||
- `wallet.dat`
|
||||
- `database/`
|
||||
- `.lock`
|
||||
- pid files
|
||||
- logs
|
||||
- Tor state
|
||||
|
||||
Legacy fallback files should still exist on the web root:
|
||||
|
||||
- `blk0001.dat`
|
||||
- `filelist.txt`
|
||||
|
||||
## Requirements For The Source Node
|
||||
|
||||
Before building a snapshot, the source node should be:
|
||||
|
||||
- fully synced
|
||||
- cleanly shut down before copying files
|
||||
- built from the same code/version expected by clients
|
||||
- using the same LevelDB schema as the client (`DATABASE_VERSION=70509`)
|
||||
|
||||
Recommended node config for the source snapshot node:
|
||||
|
||||
```ini
|
||||
txindex=1
|
||||
addressindex=1
|
||||
daemon=1
|
||||
server=1
|
||||
```
|
||||
|
||||
`addressindex=1` is recommended so clients that enable address index can benefit from faster indexed wallet rescans and address RPCs immediately.
|
||||
|
||||
## OpenClaw Workflow
|
||||
|
||||
### Step 1: Decide Whether A Prebuilt Index Is Allowed
|
||||
|
||||
OpenClaw must first decide whether it can ship `txleveldb/`.
|
||||
|
||||
Rules:
|
||||
|
||||
- If the snapshot node is exactly at checkpoint `2186940`, shipping `txleveldb/` is allowed
|
||||
- If the snapshot node is above `2186940` and the code has not been updated with a newer checkpoint, do not ship `txleveldb/`
|
||||
- In that case, publish a blocks-only bootstrap instead
|
||||
|
||||
### Step 2: Stop The Source Node Cleanly
|
||||
|
||||
Never copy a live LevelDB directory.
|
||||
|
||||
```bash
|
||||
trianglesd stop
|
||||
sleep 10
|
||||
pgrep -af trianglesd || true
|
||||
```
|
||||
|
||||
OpenClaw should confirm the daemon is fully stopped before copying `txleveldb/`.
|
||||
|
||||
### Step 3: Create A Staging Directory
|
||||
|
||||
```bash
|
||||
rm -rf /tmp/triangles-bootstrap-stage
|
||||
mkdir -p /tmp/triangles-bootstrap-stage
|
||||
```
|
||||
|
||||
### Step 4: Copy Snapshot Files
|
||||
|
||||
For a verified snapshot:
|
||||
|
||||
```bash
|
||||
cp ~/.triangles/blk0001.dat /tmp/triangles-bootstrap-stage/
|
||||
cp -a ~/.triangles/txleveldb /tmp/triangles-bootstrap-stage/
|
||||
test -f ~/.triangles/peers.dat && cp ~/.triangles/peers.dat /tmp/triangles-bootstrap-stage/
|
||||
```
|
||||
|
||||
Do not copy:
|
||||
|
||||
```bash
|
||||
rm -rf /tmp/triangles-bootstrap-stage/database
|
||||
rm -f /tmp/triangles-bootstrap-stage/wallet.dat
|
||||
rm -f /tmp/triangles-bootstrap-stage/.lock
|
||||
rm -f /tmp/triangles-bootstrap-stage/*.pid
|
||||
rm -f /tmp/triangles-bootstrap-stage/debug.log
|
||||
```
|
||||
|
||||
### Step 5: Write `snapshot.manifest`
|
||||
|
||||
If OpenClaw is publishing a verified prebuilt index, write:
|
||||
|
||||
```bash
|
||||
cat > /tmp/triangles-bootstrap-stage/snapshot.manifest << 'EOF'
|
||||
format=1
|
||||
network=main
|
||||
height=2186940
|
||||
hash=bd952e8d4a612e336d840ad924a7e09395e36bcd9d929b302e47e60b5c3098c0
|
||||
dbversion=70509
|
||||
EOF
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `hash` must not include `0x`
|
||||
- `network` must be `main`
|
||||
- `dbversion` must be `70509`
|
||||
- If OpenClaw is publishing blocks-only bootstrap, it should omit `snapshot.manifest` entirely
|
||||
|
||||
### Step 6: Build The Tarball
|
||||
|
||||
```bash
|
||||
cd /tmp/triangles-bootstrap-stage
|
||||
tar czf /tmp/bootstrap.tar.gz .
|
||||
```
|
||||
|
||||
### Step 7: Publish To The Existing Bootstrap Server
|
||||
|
||||
This guide assumes the existing nginx root is:
|
||||
|
||||
- `/var/www/triangles-bootstrap`
|
||||
|
||||
Publish the preferred tarball and the legacy fallback files:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /var/www/triangles-bootstrap
|
||||
sudo mv /tmp/bootstrap.tar.gz /var/www/triangles-bootstrap/bootstrap.tar.gz
|
||||
sudo cp ~/.triangles/blk0001.dat /var/www/triangles-bootstrap/blk0001.dat
|
||||
printf "blk0001.dat\n" | sudo tee /var/www/triangles-bootstrap/filelist.txt > /dev/null
|
||||
sudo chown -R www-data:www-data /var/www/triangles-bootstrap
|
||||
```
|
||||
|
||||
If OpenClaw is publishing a blocks-only bootstrap, the commands are the same except the tarball should contain only `blk0001.dat` and optional `peers.dat`.
|
||||
|
||||
## Validation Checklist
|
||||
|
||||
Before marking the update complete, OpenClaw should verify:
|
||||
|
||||
### Tarball contents
|
||||
|
||||
```bash
|
||||
tar tzf /var/www/triangles-bootstrap/bootstrap.tar.gz | sort
|
||||
```
|
||||
|
||||
Expected for verified snapshot:
|
||||
|
||||
- `./blk0001.dat`
|
||||
- `./txleveldb/...`
|
||||
- `./snapshot.manifest`
|
||||
|
||||
Expected not to exist:
|
||||
|
||||
- `wallet.dat`
|
||||
- `database/`
|
||||
|
||||
### HTTP responses
|
||||
|
||||
```bash
|
||||
curl -I http://localhost/bootstrap.tar.gz
|
||||
curl -I http://localhost/blk0001.dat
|
||||
curl http://localhost/filelist.txt
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- HTTP `200`
|
||||
- `filelist.txt` contains `blk0001.dat`
|
||||
|
||||
### Manifest sanity
|
||||
|
||||
```bash
|
||||
tar xOf /var/www/triangles-bootstrap/bootstrap.tar.gz ./snapshot.manifest
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- `format=1`
|
||||
- `network=main`
|
||||
- `height=2186940`
|
||||
- `hash=bd952e8d4a612e336d840ad924a7e09395e36bcd9d929b302e47e60b5c3098c0`
|
||||
- `dbversion=70509`
|
||||
|
||||
## Fresh-Client Test
|
||||
|
||||
OpenClaw should test the artifact on a clean machine or clean data directory:
|
||||
|
||||
```bash
|
||||
mv ~/.triangles ~/.triangles.backup.$(date +%s)
|
||||
mkdir -p ~/.triangles
|
||||
trianglesd -bootstrap
|
||||
```
|
||||
|
||||
Then inspect startup logs.
|
||||
|
||||
Successful verified snapshot behavior should include:
|
||||
|
||||
- snapshot downloaded
|
||||
- `snapshot.manifest found`
|
||||
- `manifest verified - keeping pre-built index`
|
||||
- no message about removing extracted `txleveldb/`
|
||||
|
||||
Failure behavior will include:
|
||||
|
||||
- manifest parse or verification failure
|
||||
- `removing extracted txleveldb/`
|
||||
- slow rebuild from `blk0001.dat`
|
||||
|
||||
## Safe Publish Procedure
|
||||
|
||||
OpenClaw should use this order:
|
||||
|
||||
1. Build snapshot in `/tmp`
|
||||
2. Validate tarball contents
|
||||
3. Replace `/var/www/triangles-bootstrap/bootstrap.tar.gz`
|
||||
4. Replace `/var/www/triangles-bootstrap/blk0001.dat`
|
||||
5. Replace `/var/www/triangles-bootstrap/filelist.txt`
|
||||
6. Confirm HTTP `200`
|
||||
|
||||
This avoids serving a half-written tarball.
|
||||
|
||||
## Example Bot Prompt
|
||||
|
||||
Use this exact tasking for OpenClaw:
|
||||
|
||||
```text
|
||||
Update the existing Triangles bootstrap server on bootstrap.cryptographic-triangles.org.
|
||||
|
||||
Rules:
|
||||
- Build the snapshot from a cleanly stopped source node
|
||||
- If the source node is exactly at hardcoded checkpoint 2186940 / bd952e8d4a612e336d840ad924a7e09395e36bcd9d929b302e47e60b5c3098c0, publish a verified snapshot containing blk0001.dat, txleveldb/, and snapshot.manifest
|
||||
- If the source node is above the latest hardcoded checkpoint, publish a blocks-only bootstrap and do not ship txleveldb/
|
||||
- Do not ship wallet.dat, database/, .lock, pid files, logs, or Tor state
|
||||
- Publish bootstrap.tar.gz, blk0001.dat, and filelist.txt to /var/www/triangles-bootstrap
|
||||
- Verify curl HTTP 200 for bootstrap.tar.gz and blk0001.dat
|
||||
- Report the tarball contents and whether the snapshot is verified or blocks-only
|
||||
```
|
||||
|
||||
## Recommended Next Improvement
|
||||
|
||||
This workflow will stay constrained until the next checkpoint is updated in `src/checkpoints.cpp`.
|
||||
|
||||
If you want OpenClaw to keep shipping prebuilt `txleveldb/` snapshots as the chain advances, the software needs periodic checkpoint updates. Without that, the verified snapshot path will stop at the latest compiled checkpoint and clients will fall back to rebuilds.
|
||||
@@ -1,123 +0,0 @@
|
||||
# TODO/FIXME Documentation
|
||||
|
||||
Detailed context for each TODO/FIXME in the codebase.
|
||||
|
||||
## Critical (Needs Attention)
|
||||
|
||||
### src/rpcmining.cpp:263 - Thread Safety Issue
|
||||
```cpp
|
||||
static mapNewBlock_t mapNewBlock; // FIXME: thread safety
|
||||
```
|
||||
**Issue:** Static variable accessed by multiple RPC threads without mutex protection.
|
||||
**Impact:** Potential race condition in getwork RPC (used for mining).
|
||||
**Status:** Low priority - PoW mining ended at block 9000, this code path rarely used.
|
||||
**Fix:** Add std::mutex and lock_guard if getwork usage increases.
|
||||
|
||||
### src/qt/walletmodel.cpp:249 - Collision Risk
|
||||
```cpp
|
||||
if((total + nFeeRequired) > nBalance) // FIXME: could cause collisions in the future
|
||||
```
|
||||
**Issue:** Balance check may have edge case causing transaction collisions.
|
||||
**Context:** In createTransaction fee calculation loop.
|
||||
**Status:** Needs investigation - unclear what "collisions" means here.
|
||||
**Fix:** Review Bitcoin Core's current implementation of this logic.
|
||||
|
||||
### src/smessage.cpp - File Size Limits
|
||||
```cpp
|
||||
// Lines 863, 2219, 2373: "TODO files must be split if > 2GB"
|
||||
```
|
||||
**Issue:** Secure message storage files not split when exceeding 2GB.
|
||||
**Impact:** May fail on 32-bit systems or with large message volumes.
|
||||
**Status:** Low priority - unlikely to reach 2GB in practice.
|
||||
**Fix:** Implement file rotation when approaching 2GB limit.
|
||||
|
||||
## Medium Priority (Encapsulation/API)
|
||||
|
||||
### src/protocol.h - Make Members Private
|
||||
```cpp
|
||||
// Lines 50, 100, 132: "TODO: make private (improves encapsulation)"
|
||||
```
|
||||
**Issue:** CAddress, CInv, CMessageHeader have public data members.
|
||||
**Impact:** Poor encapsulation, harder to maintain invariants.
|
||||
**Status:** Deferred - would require extensive refactoring.
|
||||
**Fix:** Add getter/setter methods, make members private, update all call sites.
|
||||
|
||||
### src/wallet.h:378 - nOrderPos Calculation
|
||||
```cpp
|
||||
nOrderPos = -1; // TODO: calculate elsewhere
|
||||
```
|
||||
**Issue:** Transaction ordering position calculated in constructor.
|
||||
**Impact:** Minor - works but not ideal separation of concerns.
|
||||
**Status:** Deferred - no functional issue.
|
||||
**Fix:** Move calculation to WalletDB when transaction is added.
|
||||
|
||||
### src/rpcwallet.cpp / src/qt/askpassphrasedialog.cpp - SecureString Conversion
|
||||
**Issue:** Password-handling paths were converting through `.c_str()` because `SecureString`
|
||||
did not have a convenient conversion helper from `std::string`.
|
||||
**Impact:** Unnecessary C-string shims in sensitive code paths.
|
||||
**Status:** Resolved.
|
||||
**Fix:** Added `MakeSecureString(const std::string&)` in `src/allocators.h` and updated
|
||||
the wallet RPC and passphrase dialog call sites to use it directly.
|
||||
|
||||
## Low Priority (Nice-to-Have)
|
||||
|
||||
### src/util.cpp:1322 - Disabled Feature
|
||||
```cpp
|
||||
// TODO: This is currently disabled because it needs to be verified to work
|
||||
```
|
||||
**Context:** File descriptor management code.
|
||||
**Status:** Intentionally disabled pending verification.
|
||||
**Fix:** Test thoroughly, then enable if needed.
|
||||
|
||||
### src/tor/tor_embedded.cpp:209 - Tor Shutdown API
|
||||
```cpp
|
||||
// TODO: Tor 0.4.9+ may add tor_api_shutdown(), use it when available
|
||||
```
|
||||
**Context:** Embedded Tor cleanup.
|
||||
**Status:** Waiting for upstream Tor API.
|
||||
**Fix:** Check Tor 0.4.9+ releases for new API, integrate when stable.
|
||||
|
||||
### src/init.cpp:442 - Sanity Checks
|
||||
```cpp
|
||||
// TODO: remaining sanity checks, see #4081
|
||||
```
|
||||
**Context:** Bitcoin Core issue #4081 - additional startup sanity checks.
|
||||
**Status:** Deferred - core checks already in place.
|
||||
**Fix:** Review Bitcoin Core's current sanity check implementation.
|
||||
|
||||
### src/rpcmining.cpp:232 - DRM Comment
|
||||
```cpp
|
||||
CDataStream(coinbase, SER_NETWORK, PROTOCOL_VERSION) >> pblock->vtx[0]; // FIXME - DRM!
|
||||
```
|
||||
**Issue:** Unclear what "DRM" means here - likely "Data Race Maybe"?
|
||||
**Status:** Needs clarification from original author.
|
||||
**Fix:** Investigate if there's an actual issue, otherwise remove comment.
|
||||
|
||||
## Deferred (External/Low Impact)
|
||||
|
||||
### LevelDB TODOs (src/leveldb/*)
|
||||
**Status:** Upstream LevelDB issues - don't modify embedded library.
|
||||
**Action:** None - track upstream LevelDB project.
|
||||
|
||||
### Qt TODOs (src/qt/*)
|
||||
**Status:** UI improvements, not critical.
|
||||
**Action:** Track as nice-to-have enhancements.
|
||||
|
||||
### Secure Message TODOs (src/smessage.cpp)
|
||||
Multiple minor improvements suggested:
|
||||
- Include hash in certain operations
|
||||
- Improve thread shutdown
|
||||
- Set default recv/recvAnon behavior
|
||||
- Update outbox after PoW completes
|
||||
|
||||
**Status:** Non-critical enhancements.
|
||||
**Action:** Consider for future encrypted messaging upgrades.
|
||||
|
||||
## Summary
|
||||
|
||||
**Critical:** 3 items (thread safety, balance collision, file limits)
|
||||
**Medium:** 6 items (encapsulation, SecureString)
|
||||
**Low:** 5 items (disabled features, upstream APIs)
|
||||
**Deferred:** ~24 items (external libs, minor enhancements)
|
||||
|
||||
**Recommendation:** Focus on documenting critical items in code comments, defer fixes until specific issues arise.
|
||||
@@ -0,0 +1,281 @@
|
||||
# Triangles (TRI) RPC Command Reference
|
||||
|
||||
This document describes every RPC command available in the Triangles daemon (`trianglesd`) and Qt wallet. Connect via JSON-RPC on port **19112** (default). All commands can also be run from the Qt wallet's debug console.
|
||||
|
||||
Triangles is a Tor-only PoS cryptocurrency. PoW ended at block 9000; from block 9001 onward the chain is pure Proof-of-Stake with 33% annual interest (coin-age based). Block time is 2 minutes. Max supply is 2,222,222 TRI.
|
||||
|
||||
---
|
||||
|
||||
## Server Control
|
||||
|
||||
| Command | Parameters | Description |
|
||||
|---------|-----------|-------------|
|
||||
| `help` | `[command]` | List all commands, or get detailed help for a specific command. |
|
||||
| `stop` | | Shut down the daemon. |
|
||||
|
||||
---
|
||||
|
||||
## Blockchain
|
||||
|
||||
| Command | Parameters | Description |
|
||||
|---------|-----------|-------------|
|
||||
| `getbestblockhash` | | Returns the hash of the tip of the best chain. |
|
||||
| `getblockcount` | | Returns the current block height. |
|
||||
| `getblockhash` | `<index>` | Returns the block hash at the given height. |
|
||||
| `getblock` | `<hash> [txinfo]` | Returns block details for the given hash. Set `txinfo=true` for full transaction data. |
|
||||
| `getblockbynumber` | `<number> [txinfo]` | Same as `getblock` but accepts a height instead of a hash. |
|
||||
| `getblockheader` | `<hash> [verbose=true]` | Returns block header data. If verbose is false, returns hex-encoded header. |
|
||||
| `getblockchaininfo` | | Returns chain state info: chain name, block height, best hash, difficulty, etc. |
|
||||
| `getdifficulty` | | Returns current PoW and PoS difficulty values. |
|
||||
| `gettxoutsetinfo` | | Returns statistics about the UTXO set (total txouts, size, etc.). |
|
||||
| `getrawmempool` | | Returns all transaction IDs currently in the mempool. |
|
||||
| `getcheckpoint` | | Returns info about the current synchronized checkpoint. |
|
||||
| `getchaintips` | | Returns info about all known chain tips (forks). |
|
||||
| `invalidateblock` | `<hash>` | Permanently marks a block as invalid and rewinds the chain past it. |
|
||||
| `reconsiderblock` | `<hash>` | Removes the invalid mark from a previously invalidated block. |
|
||||
| `recalculatesupply` | | Recalculates money supply by summing all UTXOs. Updates the stored value at the chain tip and persists to disk. Returns old/new supply and difference. |
|
||||
| `settxfee` | `<amount>` | Sets the transaction fee per kB. Amount is rounded to nearest 0.01. |
|
||||
| `estimatefee` | `<nblocks>` | Estimates the fee per kB needed for confirmation within `nblocks` blocks. |
|
||||
|
||||
---
|
||||
|
||||
## Address Index
|
||||
|
||||
These commands query the address index. The daemon must be running with `-addressindex=1`.
|
||||
|
||||
| Command | Parameters | Description |
|
||||
|---------|-----------|-------------|
|
||||
| `getaddressbalance` | `{"addresses":["addr",...]}` | Returns confirmed balance for the given address(es). |
|
||||
| `getaddressutxos` | `{"addresses":["addr",...]}` | Returns all unspent outputs for the given address(es). |
|
||||
| `getaddresstxids` | `{"addresses":["addr",...], "start":n, "end":n}` | Returns transaction IDs for the given address(es), optionally filtered by block range. |
|
||||
|
||||
---
|
||||
|
||||
## Mining & Staking
|
||||
|
||||
| Command | Parameters | Description |
|
||||
|---------|-----------|-------------|
|
||||
| `getmininginfo` | | Returns mining-related info: height, difficulty, network hashrate, etc. |
|
||||
| `getstakinginfo` | | Returns staking-related info: whether staking is active, weight, expected time to stake, etc. |
|
||||
| `getsubsidy` | `[nTarget]` | Returns the PoW subsidy value for the given target height (historical reference only since PoW ended at block 9000). |
|
||||
|
||||
---
|
||||
|
||||
## Network
|
||||
|
||||
| Command | Parameters | Description |
|
||||
|---------|-----------|-------------|
|
||||
| `getconnectioncount` | | Returns the number of peer connections. |
|
||||
| `getpeerinfo` | | Returns detailed info about each connected peer (address, version, ping time, etc.). |
|
||||
| `getnetworkinfo` | | Returns P2P network state: version, protocol, peer mix, connections, relay fee, etc. |
|
||||
| `getseedlist` | | Returns the list of configured seed nodes. |
|
||||
| `addnode` | `<node> <add\|remove\|onetry>` | Add or remove a node from the manual peer list, or try connecting once. For Tor nodes use the `.onion` address. |
|
||||
| `disconnectnode` | `<node>` | Immediately disconnects from the specified peer. |
|
||||
| `sendalert` | `<message> <privatekey> <minver> <maxver> <priority> <id> [cancelupto]` | Broadcasts a network alert (requires the alert master private key). |
|
||||
|
||||
---
|
||||
|
||||
## Wallet — General
|
||||
|
||||
| Command | Parameters | Description |
|
||||
|---------|-----------|-------------|
|
||||
| `getinfo` | | Returns general info: version, balance, stake, block height, connections, etc. |
|
||||
| `getwalletinfo` | | Returns wallet-specific info: balance, unconfirmed, immature, txcount, keypoolsize, etc. |
|
||||
| `getbalance` | `[account] [minconf=1]` | Returns total available balance (optionally for a specific account). |
|
||||
| `checkwallet` | | Checks wallet database for consistency errors. |
|
||||
| `repairwallet` | | Attempts to repair the wallet database. |
|
||||
| `resendtx` | | Re-broadcasts all unconfirmed wallet transactions. |
|
||||
|
||||
---
|
||||
|
||||
## Wallet — Addresses & Accounts
|
||||
|
||||
| Command | Parameters | Description |
|
||||
|---------|-----------|-------------|
|
||||
| `getnewaddress` | `[account]` | Generates a new receiving address (optionally assigned to an account). |
|
||||
| `getnewpubkey` | `[account]` | Returns a new public key for the wallet. |
|
||||
| `getaccountaddress` | `<account>` | Returns the current receiving address for the given account. |
|
||||
| `setaccount` | `<address> <account>` | Assigns an address to the given account label. |
|
||||
| `getaccount` | `<address>` | Returns the account label for the given address. |
|
||||
| `getaddressesbyaccount` | `<account>` | Returns all addresses assigned to the given account. |
|
||||
| `listaddressgroupings` | | Returns addresses grouped by common ownership (based on transaction history). |
|
||||
| `validateaddress` | `<address>` | Validates a Triangles address and returns info (ismine, account, pubkey, etc.). |
|
||||
| `validatepubkey` | `<pubkey>` | Validates a Triangles public key. |
|
||||
| `listaccounts` | `[minconf=1]` | Returns all account names and their balances. |
|
||||
|
||||
---
|
||||
|
||||
## Wallet — Sending
|
||||
|
||||
| Command | Parameters | Description |
|
||||
|---------|-----------|-------------|
|
||||
| `sendtoaddress` | `<address> <amount> [comment] [comment-to]` | Sends TRI to an address. Returns the transaction ID. |
|
||||
| `sendfrom` | `<fromaccount> <address> <amount> [minconf=1] [comment] [comment-to]` | Sends TRI from a specific account. |
|
||||
| `sendmany` | `<fromaccount> {"addr":amount,...} [minconf=1] [comment]` | Sends TRI to multiple addresses in a single transaction. |
|
||||
| `move` | `<fromaccount> <toaccount> <amount> [minconf=1] [comment]` | Moves funds between accounts (internal bookkeeping only, no on-chain tx). |
|
||||
|
||||
---
|
||||
|
||||
## Wallet — Transaction History
|
||||
|
||||
| Command | Parameters | Description |
|
||||
|---------|-----------|-------------|
|
||||
| `listtransactions` | `[account] [count=10] [from=0]` | Returns the most recent transactions (optionally filtered by account). |
|
||||
| `listsinceblock` | `[blockhash] [target-confirmations]` | Returns all transactions since the given block. |
|
||||
| `gettransaction` | `<txid>` | Returns detailed info about a wallet transaction. |
|
||||
| `getreceivedbyaddress` | `<address> [minconf=1]` | Returns total amount received by an address. |
|
||||
| `getreceivedbyaccount` | `<account> [minconf=1]` | Returns total amount received by an account. |
|
||||
| `listreceivedbyaddress` | `[minconf=1] [includeempty=false]` | Returns amounts received for each address. |
|
||||
| `listreceivedbyaccount` | `[minconf=1] [includeempty=false]` | Returns amounts received for each account. |
|
||||
|
||||
---
|
||||
|
||||
## Wallet — Staking Control
|
||||
|
||||
| Command | Parameters | Description |
|
||||
|---------|-----------|-------------|
|
||||
| `reservebalance` | `[reserve] [amount]` | Show or set a reserve balance that will not be used for staking. `reserve` is true/false, `amount` is the TRI to reserve. |
|
||||
|
||||
---
|
||||
|
||||
## Wallet — Security
|
||||
|
||||
| Command | Parameters | Description |
|
||||
|---------|-----------|-------------|
|
||||
| `encryptwallet` | `<passphrase>` | Encrypts the wallet with the given passphrase. **This shuts down the daemon.** The wallet must be re-started and unlocked afterward. |
|
||||
| `walletpassphrase` | `<passphrase> <timeout> [stakingonly]` | Unlocks the wallet for `timeout` seconds. Set `stakingonly=true` to allow staking but prevent sending. |
|
||||
| `walletpassphrasechange` | `<oldpassphrase> <newpassphrase>` | Changes the wallet encryption passphrase. |
|
||||
| `walletlock` | | Immediately locks the wallet (removes decryption key from memory). |
|
||||
| `keypoolrefill` | `[new-size]` | Tops up the pre-generated key pool. |
|
||||
| `makekeypair` | `[prefix]` | Generates a new public/private keypair (not added to wallet). |
|
||||
|
||||
---
|
||||
|
||||
## Wallet — Backup & Import
|
||||
|
||||
| Command | Parameters | Description |
|
||||
|---------|-----------|-------------|
|
||||
| `backupwallet` | `<destination>` | Copies `wallet.dat` to the given file path. |
|
||||
| `dumpwallet` | `<filename>` | Exports all wallet private keys to a plaintext file. |
|
||||
| `dumpprivkey` | `<address>` | Returns the private key (WIF format) for the given address. |
|
||||
| `importwallet` | `<filename>` | Imports keys from a wallet dump file. |
|
||||
| `importprivkey` | `<privkey> [label]` | Imports a single private key (WIF format) with optional label. |
|
||||
|
||||
---
|
||||
|
||||
## Wallet — Multisig
|
||||
|
||||
| Command | Parameters | Description |
|
||||
|---------|-----------|-------------|
|
||||
| `addmultisigaddress` | `<nrequired> ["key",...] [account]` | Creates an M-of-N multisig address. `nrequired` is the number of signatures needed. |
|
||||
| `addredeemscript` | `<redeemScript> [account]` | Adds a P2SH redeem script to the wallet. |
|
||||
|
||||
---
|
||||
|
||||
## Wallet — Message Signing
|
||||
|
||||
| Command | Parameters | Description |
|
||||
|---------|-----------|-------------|
|
||||
| `signmessage` | `<address> <message>` | Signs a message with the private key of the given address. |
|
||||
| `verifymessage` | `<address> <signature> <message>` | Verifies a signed message. Returns true/false. |
|
||||
|
||||
---
|
||||
|
||||
## Raw Transactions
|
||||
|
||||
| Command | Parameters | Description |
|
||||
|---------|-----------|-------------|
|
||||
| `listunspent` | `[minconf=1] [maxconf=9999999] ["addr",...]` | Returns unspent transaction outputs, optionally filtered by address and confirmation count. |
|
||||
| `createrawtransaction` | `[{"txid":"id","vout":n},...] {"addr":amount,...}` | Creates an unsigned raw transaction from the given inputs and outputs. |
|
||||
| `decoderawtransaction` | `<hex>` | Decodes a raw transaction hex string into a JSON object. |
|
||||
| `decodescript` | `<hex>` | Decodes a hex-encoded script into human-readable form. |
|
||||
| `signrawtransaction` | `<hex> [prevtxs] [privkeys] [sighashtype="ALL"]` | Signs a raw transaction. Can provide previous tx outputs and private keys for offline signing. |
|
||||
| `sendrawtransaction` | `<hex>` | Broadcasts a signed raw transaction to the network. Returns the txid. |
|
||||
| `getrawtransaction` | `<txid> [verbose=0]` | Returns raw transaction data. Set verbose=1 for decoded JSON output. |
|
||||
|
||||
---
|
||||
|
||||
## Secure Messaging (SMSG)
|
||||
|
||||
Triangles has a built-in encrypted peer-to-peer messaging system. Messages are stored in a DHT-like bucket system and relayed through the network.
|
||||
|
||||
| Command | Parameters | Description |
|
||||
|---------|-----------|-------------|
|
||||
| `smsgenable` | | Enables the secure messaging system. |
|
||||
| `smsgdisable` | | Disables the secure messaging system. |
|
||||
| `smsgoptions` | `[list\|set <optname> <value>]` | View or change secure messaging options. |
|
||||
| `smsglocalkeys` | `[whitelist\|all\|wallet\|recv +/- <addr>\|anon +/- <addr>]` | Manage which local keys participate in secure messaging. |
|
||||
| `smsgaddkey` | `<address> <pubkey>` | Adds someone's public key so you can send them encrypted messages. |
|
||||
| `smsggetpubkey` | `<address>` | Retrieves the public key for an address (needed to send messages to it). |
|
||||
| `smsgsend` | `<fromAddr> <toAddr> <message>` | Sends an encrypted message from one of your addresses to a recipient. |
|
||||
| `smsgsendanon` | `<toAddr> <message>` | Sends an anonymous encrypted message (no sender address attached). |
|
||||
| `smsginbox` | `[all\|unread\|clear]` | View received secure messages. Default shows unread. |
|
||||
| `smsgoutbox` | `[all\|clear]` | View sent secure messages. |
|
||||
| `smsgscanchain` | | Scans the blockchain for secure message public keys. |
|
||||
| `smsgscanbuckets` | | Scans stored message buckets for messages addressed to your keys. |
|
||||
| `smsgbuckets` | `[stats\|dump]` | View secure message bucket statistics or dump contents. |
|
||||
| `smsgbroadcast` | `<fromAddr> <message>` | Broadcasts a message to all SMSG participants (not encrypted to a single recipient). |
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference — Common Tasks
|
||||
|
||||
**Check node status:**
|
||||
```
|
||||
getinfo
|
||||
getblockcount
|
||||
getconnectioncount
|
||||
getstakinginfo
|
||||
```
|
||||
|
||||
**Check balance and transactions:**
|
||||
```
|
||||
getbalance
|
||||
listtransactions
|
||||
```
|
||||
|
||||
**Send coins:**
|
||||
```
|
||||
walletpassphrase "yourpassphrase" 60
|
||||
sendtoaddress "TRIaddress" 100
|
||||
walletlock
|
||||
```
|
||||
|
||||
**Unlock for staking only:**
|
||||
```
|
||||
walletpassphrase "yourpassphrase" 999999999 true
|
||||
```
|
||||
|
||||
**Add a peer manually (Tor .onion):**
|
||||
```
|
||||
addnode "abcdef1234567890.onion" "add"
|
||||
```
|
||||
|
||||
**Export/import a private key:**
|
||||
```
|
||||
dumpprivkey "TRIaddress"
|
||||
importprivkey "5KPrivKeyHere" "mylabel"
|
||||
```
|
||||
|
||||
**Fix incorrect money supply display:**
|
||||
```
|
||||
recalculatesupply
|
||||
```
|
||||
|
||||
**Full reindex (rebuild block index from raw data):**
|
||||
```
|
||||
trianglesd -reindex
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Connection Info
|
||||
|
||||
| Setting | Value |
|
||||
|---------|-------|
|
||||
| Default RPC port | 19112 |
|
||||
| Default P2P port | 24112 |
|
||||
| Config file (Windows) | `%APPDATA%\triangles\triangles.conf` |
|
||||
| Config file (Linux) | `~/.triangles/triangles.conf` |
|
||||
| Protocol version | 70205 |
|
||||
| Network | Tor-only |
|
||||
@@ -23,6 +23,8 @@ add_library(hash9_crypto STATIC
|
||||
)
|
||||
target_include_directories(hash9_crypto PUBLIC "${CMAKE_CURRENT_SOURCE_DIR}")
|
||||
set_target_properties(hash9_crypto PROPERTIES LINKER_LANGUAGE C)
|
||||
# Hash9 C files have colliding static symbols (IV512, DECL_STATE, etc.) — skip unity
|
||||
set_target_properties(hash9_crypto PROPERTIES UNITY_BUILD OFF)
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════════════════════
|
||||
# 2. JSON library (header-only nlohmann/json via json_compat.h shim)
|
||||
@@ -71,7 +73,11 @@ set(CORE_SOURCES
|
||||
rpcrawtransaction.cpp
|
||||
rpcsmessage.cpp
|
||||
zmqpublishnotifier.cpp
|
||||
txdb-base.cpp
|
||||
txdb.cpp
|
||||
txdb-leveldb.cpp
|
||||
utxosnapshot.cpp
|
||||
snapshotnet.cpp
|
||||
lz4/lz4.c
|
||||
tor/onion_v3.cpp
|
||||
tor/tor_process.cpp
|
||||
@@ -93,6 +99,11 @@ elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "arm|ARM")
|
||||
list(APPEND CORE_SOURCES scrypt-arm.S)
|
||||
endif()
|
||||
|
||||
# Optional: RocksDB chain database backend
|
||||
if(BUILD_ROCKSDB)
|
||||
list(APPEND CORE_SOURCES txdb-rocksdb.cpp)
|
||||
endif()
|
||||
|
||||
add_library(triangles_common OBJECT ${CORE_SOURCES})
|
||||
|
||||
target_include_directories(triangles_common PUBLIC
|
||||
@@ -140,6 +151,16 @@ if(USE_ZMQ)
|
||||
target_link_libraries(triangles_common PUBLIC PkgConfig::ZMQ)
|
||||
endif()
|
||||
|
||||
# Optional: RocksDB
|
||||
if(BUILD_ROCKSDB)
|
||||
target_compile_definitions(triangles_common PUBLIC BUILD_ROCKSDB)
|
||||
if(TARGET RocksDB::rocksdb)
|
||||
target_link_libraries(triangles_common PUBLIC RocksDB::rocksdb)
|
||||
elseif(TARGET PkgConfig::RocksDB)
|
||||
target_link_libraries(triangles_common PUBLIC PkgConfig::RocksDB)
|
||||
endif()
|
||||
endif()
|
||||
|
||||
# Optional: Embedded Tor
|
||||
if(USE_TOR_EMBEDDED)
|
||||
if(TOR_SOURCE_ROOT STREQUAL "")
|
||||
@@ -184,6 +205,31 @@ endif()
|
||||
|
||||
add_dependencies(triangles_common generate_build_info build_leveldb)
|
||||
|
||||
# ── Precompiled header (heavy STL + Boost + OpenSSL includes, C++ only) ──
|
||||
target_precompile_headers(triangles_common PRIVATE
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<string$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<vector$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<map$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<deque$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<algorithm$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<sstream$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<stdexcept$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<cstdint$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<cstring$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<memory$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<functional$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<boost/filesystem.hpp$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<boost/filesystem/fstream.hpp$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<boost/thread.hpp$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<boost/thread/mutex.hpp$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<boost/thread/condition_variable.hpp$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<boost/algorithm/string.hpp$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<openssl/sha.h$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<openssl/crypto.h$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<openssl/rand.h$<ANGLE-R>>"
|
||||
"$<$<COMPILE_LANGUAGE:CXX>:<openssl/evp.h$<ANGLE-R>>"
|
||||
)
|
||||
|
||||
# ═══════════════════════════════════════════════════════════════════════════════
|
||||
# 4. Headless daemon (trianglesd)
|
||||
# ═══════════════════════════════════════════════════════════════════════════════
|
||||
@@ -195,6 +241,7 @@ if(BUILD_DAEMON)
|
||||
)
|
||||
# No QT_GUI define — daemon gets the #if !defined(QT_GUI) code paths
|
||||
target_link_libraries(trianglesd PRIVATE triangles_common)
|
||||
target_precompile_headers(trianglesd REUSE_FROM triangles_common)
|
||||
|
||||
if(WIN32)
|
||||
set_target_properties(trianglesd PROPERTIES SUFFIX ".exe")
|
||||
|
||||
+327
-75
@@ -2,6 +2,8 @@
|
||||
// Distributed under the MIT/X11 software license
|
||||
|
||||
#include "bootstrap.h"
|
||||
#include "utxosnapshot.h"
|
||||
#include "txdb.h"
|
||||
|
||||
#include <boost/filesystem.hpp>
|
||||
#include <boost/filesystem/fstream.hpp>
|
||||
@@ -14,6 +16,9 @@
|
||||
#include "netbase.h"
|
||||
#include "net.h"
|
||||
|
||||
#include <openssl/ssl.h>
|
||||
#include <openssl/err.h>
|
||||
|
||||
#include <fstream>
|
||||
#include <sstream>
|
||||
#include <cstdio>
|
||||
@@ -22,8 +27,10 @@
|
||||
|
||||
#ifdef WIN32
|
||||
#include <winsock2.h>
|
||||
#include <ws2tcpip.h>
|
||||
#else
|
||||
#include <sys/socket.h>
|
||||
#include <netdb.h>
|
||||
#include <unistd.h>
|
||||
#endif
|
||||
|
||||
@@ -40,80 +47,288 @@ bool NeedsBootstrap(const fs::path& dataDir)
|
||||
return !fs::exists(dataDir / "blk0001.dat");
|
||||
}
|
||||
|
||||
// Send all bytes on a raw socket
|
||||
static bool SendAll(SOCKET sock, const char* data, size_t len)
|
||||
// Direct TCP connection bypassing Tor SOCKS proxy.
|
||||
// Used for bootstrap downloads where the server is on clearnet.
|
||||
static SOCKET ConnectDirectTCP(const std::string& host, int port, std::string& strError)
|
||||
{
|
||||
while (len > 0) {
|
||||
int n = send(sock, data, (int)std::min(len, (size_t)65536), MSG_NOSIGNAL);
|
||||
if (n <= 0) return false;
|
||||
data += n;
|
||||
len -= n;
|
||||
struct addrinfo hints, *result, *rp;
|
||||
memset(&hints, 0, sizeof(hints));
|
||||
hints.ai_family = AF_UNSPEC;
|
||||
hints.ai_socktype = SOCK_STREAM;
|
||||
|
||||
std::string portStr = std::to_string(port);
|
||||
int rc = getaddrinfo(host.c_str(), portStr.c_str(), &hints, &result);
|
||||
if (rc != 0) {
|
||||
strError = "DNS resolution failed for " + host;
|
||||
return INVALID_SOCKET;
|
||||
}
|
||||
return true;
|
||||
|
||||
SOCKET hSocket = INVALID_SOCKET;
|
||||
for (rp = result; rp != NULL; rp = rp->ai_next) {
|
||||
hSocket = socket(rp->ai_family, rp->ai_socktype, rp->ai_protocol);
|
||||
if (hSocket == INVALID_SOCKET)
|
||||
continue;
|
||||
|
||||
if (connect(hSocket, rp->ai_addr, (int)rp->ai_addrlen) == 0)
|
||||
break; // success
|
||||
|
||||
closesocket(hSocket);
|
||||
hSocket = INVALID_SOCKET;
|
||||
}
|
||||
freeaddrinfo(result);
|
||||
|
||||
if (hSocket == INVALID_SOCKET)
|
||||
strError = "Cannot connect to " + host + ":" + portStr;
|
||||
|
||||
return hSocket;
|
||||
}
|
||||
|
||||
// Read until `delim` found in received data. Returns data including delimiter.
|
||||
static bool RecvUntil(SOCKET sock, std::string& out, const std::string& delim)
|
||||
{
|
||||
out.clear();
|
||||
char c;
|
||||
while (true) {
|
||||
int n = recv(sock, &c, 1, 0);
|
||||
if (n <= 0) return false;
|
||||
out += c;
|
||||
if (out.size() >= delim.size() &&
|
||||
out.compare(out.size() - delim.size(), delim.size(), delim) == 0)
|
||||
return true;
|
||||
if (out.size() > 64 * 1024) return false; // header too large
|
||||
// RAII wrapper for an HTTP(S) connection (socket + optional TLS)
|
||||
struct HttpConn {
|
||||
SOCKET sock;
|
||||
SSL_CTX* ctx;
|
||||
SSL* ssl;
|
||||
|
||||
HttpConn() : sock(INVALID_SOCKET), ctx(nullptr), ssl(nullptr) {}
|
||||
~HttpConn() { Close(); }
|
||||
|
||||
void Close() {
|
||||
if (ssl) { SSL_shutdown(ssl); SSL_free(ssl); ssl = nullptr; }
|
||||
if (ctx) { SSL_CTX_free(ctx); ctx = nullptr; }
|
||||
if (sock != INVALID_SOCKET) { closesocket(sock); sock = INVALID_SOCKET; }
|
||||
}
|
||||
|
||||
bool Send(const char* data, size_t len) {
|
||||
while (len > 0) {
|
||||
int n = ssl ? SSL_write(ssl, data, (int)std::min(len, (size_t)65536))
|
||||
: send(sock, data, (int)std::min(len, (size_t)65536), MSG_NOSIGNAL);
|
||||
if (n <= 0) return false;
|
||||
data += n;
|
||||
len -= n;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
int Recv(char* buf, int len) {
|
||||
return ssl ? SSL_read(ssl, buf, len) : recv(sock, buf, len, 0);
|
||||
}
|
||||
|
||||
// Read until delimiter found. Returns data including delimiter.
|
||||
bool RecvUntil(std::string& out, const std::string& delim) {
|
||||
out.clear();
|
||||
char c;
|
||||
while (true) {
|
||||
int n = Recv(&c, 1);
|
||||
if (n <= 0) return false;
|
||||
out += c;
|
||||
if (out.size() >= delim.size() &&
|
||||
out.compare(out.size() - delim.size(), delim.size(), delim) == 0)
|
||||
return true;
|
||||
if (out.size() > 64 * 1024) return false; // header too large
|
||||
}
|
||||
}
|
||||
|
||||
// Establish TLS on an already-connected socket
|
||||
bool StartTLS(const std::string& hostname, std::string& strError) {
|
||||
ctx = SSL_CTX_new(TLS_client_method());
|
||||
if (!ctx) {
|
||||
strError = "Failed to create SSL context";
|
||||
return false;
|
||||
}
|
||||
// Skip cert verification — we verify data integrity via checkpoint hashes
|
||||
SSL_CTX_set_verify(ctx, SSL_VERIFY_NONE, nullptr);
|
||||
|
||||
ssl = SSL_new(ctx);
|
||||
if (!ssl) {
|
||||
strError = "Failed to create SSL object";
|
||||
return false;
|
||||
}
|
||||
SSL_set_fd(ssl, (int)sock);
|
||||
SSL_set_tlsext_host_name(ssl, hostname.c_str()); // SNI
|
||||
|
||||
if (SSL_connect(ssl) != 1) {
|
||||
unsigned long err = ERR_get_error();
|
||||
char errBuf[256];
|
||||
ERR_error_string_n(err, errBuf, sizeof(errBuf));
|
||||
strError = "TLS handshake failed with " + hostname + ": " + errBuf;
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
};
|
||||
|
||||
// Parse host, port, and path from an absolute URL.
|
||||
// Sets useSSL, host, port, path. Returns false for unsupported schemes.
|
||||
static bool ParseAbsoluteUrl(const std::string& url,
|
||||
bool& useSSL, std::string& host,
|
||||
int& port, std::string& path)
|
||||
{
|
||||
if (url.compare(0, 8, "https://") == 0) {
|
||||
useSSL = true;
|
||||
std::string rest = url.substr(8);
|
||||
size_t pathStart = rest.find('/');
|
||||
if (pathStart != std::string::npos) {
|
||||
host = rest.substr(0, pathStart);
|
||||
path = rest.substr(pathStart);
|
||||
} else {
|
||||
host = rest;
|
||||
path = "/";
|
||||
}
|
||||
size_t colonPos = host.find(':');
|
||||
if (colonPos != std::string::npos) {
|
||||
port = std::atoi(host.c_str() + colonPos + 1);
|
||||
host = host.substr(0, colonPos);
|
||||
} else {
|
||||
port = 443;
|
||||
}
|
||||
return true;
|
||||
} else if (url.compare(0, 7, "http://") == 0) {
|
||||
useSSL = false;
|
||||
std::string rest = url.substr(7);
|
||||
size_t pathStart = rest.find('/');
|
||||
if (pathStart != std::string::npos) {
|
||||
host = rest.substr(0, pathStart);
|
||||
path = rest.substr(pathStart);
|
||||
} else {
|
||||
host = rest;
|
||||
path = "/";
|
||||
}
|
||||
size_t colonPos = host.find(':');
|
||||
if (colonPos != std::string::npos) {
|
||||
port = std::atoi(host.c_str() + colonPos + 1);
|
||||
host = host.substr(0, colonPos);
|
||||
} else {
|
||||
port = 80;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
bool DownloadFile(const std::string& host, const std::string& urlPath,
|
||||
const fs::path& destPath,
|
||||
ProgressCallback progressFn,
|
||||
std::string& strError)
|
||||
std::string& strError,
|
||||
bool noProxy,
|
||||
int portOverride)
|
||||
{
|
||||
try {
|
||||
// Connect through Tor SOCKS proxy (ConnectSocketByName respects SetProxy)
|
||||
SOCKET hSocket = INVALID_SOCKET;
|
||||
CService addr;
|
||||
if (!ConnectSocketByName(addr, hSocket, host.c_str(), PORT, 30)) {
|
||||
strError = "Cannot connect to " + host + " (check Tor proxy)";
|
||||
return false;
|
||||
}
|
||||
|
||||
// Send HTTP GET request
|
||||
std::string request =
|
||||
"GET " + urlPath + " HTTP/1.1\r\n"
|
||||
"Host: " + host + "\r\n"
|
||||
"Connection: close\r\n"
|
||||
"User-Agent: Triangles\r\n"
|
||||
"\r\n";
|
||||
|
||||
if (!SendAll(hSocket, request.data(), request.size())) {
|
||||
closesocket(hSocket);
|
||||
strError = "Failed to send request to " + host;
|
||||
return false;
|
||||
}
|
||||
|
||||
// Read response headers
|
||||
std::string currentHost = host;
|
||||
std::string currentPath = urlPath;
|
||||
int currentPort = (portOverride > 0) ? portOverride : PORT;
|
||||
bool useSSL = false;
|
||||
std::string headerData;
|
||||
if (!RecvUntil(hSocket, headerData, "\r\n\r\n")) {
|
||||
closesocket(hSocket);
|
||||
strError = "Failed to read HTTP headers from " + host;
|
||||
return false;
|
||||
}
|
||||
int redirectCount = 0;
|
||||
const int MAX_REDIRECTS = 5;
|
||||
|
||||
// Parse status code from "HTTP/1.x NNN ..."
|
||||
unsigned int status_code = 0;
|
||||
size_t sp = headerData.find(' ');
|
||||
if (sp != std::string::npos)
|
||||
status_code = atoi(headerData.c_str() + sp + 1);
|
||||
HttpConn conn;
|
||||
|
||||
if (status_code != 200) {
|
||||
closesocket(hSocket);
|
||||
strError = "HTTP error " + std::to_string(status_code) + " for " + urlPath;
|
||||
return false;
|
||||
// Connection + redirect loop
|
||||
while (true) {
|
||||
conn.Close(); // clean slate for each attempt
|
||||
|
||||
if (noProxy) {
|
||||
conn.sock = ConnectDirectTCP(currentHost, currentPort, strError);
|
||||
if (conn.sock == INVALID_SOCKET)
|
||||
return false;
|
||||
} else {
|
||||
CService addr;
|
||||
if (!ConnectSocketByName(addr, conn.sock, currentHost.c_str(), currentPort, 30)) {
|
||||
strError = "Cannot connect to " + currentHost + " (check Tor proxy)";
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// Establish TLS when needed
|
||||
if (useSSL) {
|
||||
if (!conn.StartTLS(currentHost, strError))
|
||||
return false;
|
||||
printf("Bootstrap: TLS established with %s:%d\n",
|
||||
currentHost.c_str(), currentPort);
|
||||
}
|
||||
|
||||
// Send HTTP GET request
|
||||
std::string request =
|
||||
"GET " + currentPath + " HTTP/1.1\r\n"
|
||||
"Host: " + currentHost + "\r\n"
|
||||
"Connection: close\r\n"
|
||||
"User-Agent: Triangles\r\n"
|
||||
"\r\n";
|
||||
|
||||
if (!conn.Send(request.data(), request.size())) {
|
||||
strError = "Failed to send request to " + currentHost;
|
||||
return false;
|
||||
}
|
||||
|
||||
// Read response headers
|
||||
if (!conn.RecvUntil(headerData, "\r\n\r\n")) {
|
||||
strError = "Failed to read HTTP headers from " + currentHost;
|
||||
return false;
|
||||
}
|
||||
|
||||
// Parse status code from "HTTP/1.x NNN ..."
|
||||
unsigned int status_code = 0;
|
||||
size_t sp = headerData.find(' ');
|
||||
if (sp != std::string::npos)
|
||||
status_code = atoi(headerData.c_str() + sp + 1);
|
||||
|
||||
// Handle HTTP redirects
|
||||
if (status_code == 301 || status_code == 302 ||
|
||||
status_code == 307 || status_code == 308) {
|
||||
|
||||
if (++redirectCount > MAX_REDIRECTS) {
|
||||
strError = "Too many redirects for " + urlPath;
|
||||
return false;
|
||||
}
|
||||
|
||||
// Find Location header (case-insensitive)
|
||||
std::string lowerHdr = headerData;
|
||||
std::transform(lowerHdr.begin(), lowerHdr.end(),
|
||||
lowerHdr.begin(), ::tolower);
|
||||
size_t locPos = lowerHdr.find("\nlocation:");
|
||||
if (locPos == std::string::npos) {
|
||||
strError = "Redirect " + std::to_string(status_code) + " without Location header";
|
||||
return false;
|
||||
}
|
||||
|
||||
size_t valStart = locPos + 10; // skip "\nlocation:"
|
||||
while (valStart < headerData.size() && headerData[valStart] == ' ')
|
||||
valStart++;
|
||||
size_t lineEnd = headerData.find("\r\n", valStart);
|
||||
std::string location;
|
||||
if (lineEnd != std::string::npos)
|
||||
location = headerData.substr(valStart, lineEnd - valStart);
|
||||
else
|
||||
location = headerData.substr(valStart);
|
||||
boost::trim(location);
|
||||
|
||||
// Parse redirect URL — supports http://, https://, and relative paths
|
||||
if (location.compare(0, 7, "http://") == 0 ||
|
||||
location.compare(0, 8, "https://") == 0) {
|
||||
if (!ParseAbsoluteUrl(location, useSSL, currentHost,
|
||||
currentPort, currentPath)) {
|
||||
strError = "Unsupported redirect location: " + location;
|
||||
return false;
|
||||
}
|
||||
} else if (!location.empty() && location[0] == '/') {
|
||||
currentPath = location;
|
||||
} else {
|
||||
strError = "Unsupported redirect location: " + location;
|
||||
return false;
|
||||
}
|
||||
|
||||
printf("Bootstrap: redirect %d -> %s%s%s (port %d)\n",
|
||||
status_code, useSSL ? "https://" : "http://",
|
||||
currentHost.c_str(), currentPath.c_str(), currentPort);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (status_code != 200) {
|
||||
strError = "HTTP error " + std::to_string(status_code) + " for " + currentPath;
|
||||
return false;
|
||||
}
|
||||
|
||||
break; // Got 200, proceed to download
|
||||
}
|
||||
|
||||
// Parse Content-Length
|
||||
@@ -132,7 +347,6 @@ bool DownloadFile(const std::string& host, const std::string& urlPath,
|
||||
// Open output file
|
||||
FILE* file = fopen(destPath.string().c_str(), "wb");
|
||||
if (!file) {
|
||||
closesocket(hSocket);
|
||||
strError = "Cannot create file: " + destPath.string();
|
||||
return false;
|
||||
}
|
||||
@@ -143,10 +357,9 @@ bool DownloadFile(const std::string& host, const std::string& urlPath,
|
||||
char chunk[65536];
|
||||
|
||||
while (true) {
|
||||
int n = recv(hSocket, chunk, sizeof(chunk), 0);
|
||||
int n = conn.Recv(chunk, sizeof(chunk));
|
||||
if (n < 0) {
|
||||
fclose(file);
|
||||
closesocket(hSocket);
|
||||
fs::remove(destPath);
|
||||
strError = "Network error during download";
|
||||
return false;
|
||||
@@ -163,7 +376,7 @@ bool DownloadFile(const std::string& host, const std::string& urlPath,
|
||||
}
|
||||
|
||||
fclose(file);
|
||||
closesocket(hSocket);
|
||||
// conn destructor handles socket + SSL cleanup
|
||||
|
||||
// Verify download size if Content-Length was provided
|
||||
if (content_length > 0 && bytes_written != content_length) {
|
||||
@@ -183,13 +396,14 @@ bool DownloadFile(const std::string& host, const std::string& urlPath,
|
||||
|
||||
bool FetchFileList(const std::string& host,
|
||||
std::vector<std::string>& files,
|
||||
std::string& strError)
|
||||
std::string& strError,
|
||||
bool noProxy)
|
||||
{
|
||||
// Download filelist.txt to a temp file
|
||||
fs::path tmpPath = fs::temp_directory_path() / "triangles_bootstrap_filelist.txt";
|
||||
|
||||
std::string urlPath = std::string(BASE_PATH) + "filelist.txt";
|
||||
if (!DownloadFile(host, urlPath, tmpPath, nullptr, strError))
|
||||
if (!DownloadFile(host, urlPath, tmpPath, nullptr, strError, noProxy))
|
||||
return false;
|
||||
|
||||
// Read lines
|
||||
@@ -458,10 +672,14 @@ bool DownloadBootstrap(const std::string& host,
|
||||
bool gotBlockFile = false;
|
||||
|
||||
// Try downloading bootstrap.tar.gz first
|
||||
// Bootstrap server is on clearnet — bypass Tor proxy for DNS + HTTP
|
||||
const bool noProxy = true;
|
||||
fs::path tmpTarGz = dataDir / "bootstrap.tar.gz.tmp";
|
||||
std::string tarUrl = std::string(BASE_PATH) + "triangles-bootstrap.tar.gz";
|
||||
|
||||
bool tarDownloaded = DownloadFile(host, tarUrl, tmpTarGz, progressFn, strError);
|
||||
printf("DownloadBootstrap(): attempting tar.gz download from %s%s\n", host.c_str(), tarUrl.c_str());
|
||||
bool tarDownloaded = DownloadFile(host, tarUrl, tmpTarGz, progressFn, strError, noProxy);
|
||||
printf("DownloadBootstrap(): tarDownloaded=%d result=%s\n", tarDownloaded, strError.c_str());
|
||||
|
||||
if (tarDownloaded) {
|
||||
bool extractOk = ExtractTarGz(tmpTarGz, dataDir, strError);
|
||||
@@ -476,7 +694,7 @@ bool DownloadBootstrap(const std::string& host,
|
||||
// Fallback: try filelist.txt + individual file downloads
|
||||
std::string fallbackError;
|
||||
std::vector<std::string> files;
|
||||
if (!FetchFileList(host, files, fallbackError)) {
|
||||
if (!FetchFileList(host, files, fallbackError, noProxy)) {
|
||||
if (!tarDownloaded)
|
||||
strError = strError + " (fallback also failed: " + fallbackError + ")";
|
||||
else
|
||||
@@ -489,7 +707,7 @@ bool DownloadBootstrap(const std::string& host,
|
||||
fs::create_directories(destPath.parent_path());
|
||||
|
||||
std::string urlPath = std::string(BASE_PATH) + files[i];
|
||||
if (!DownloadFile(host, urlPath, destPath, progressFn, strError))
|
||||
if (!DownloadFile(host, urlPath, destPath, progressFn, strError, noProxy))
|
||||
return false;
|
||||
}
|
||||
|
||||
@@ -501,16 +719,16 @@ bool DownloadBootstrap(const std::string& host,
|
||||
return false;
|
||||
}
|
||||
|
||||
// Check if the archive included a trusted pre-built index (txleveldb/)
|
||||
// with a valid snapshot.manifest. If verified, keep it to skip the
|
||||
// Check if the archive included a trusted pre-built index for the active
|
||||
// backend with a valid snapshot.manifest. If verified, keep it to skip the
|
||||
// multi-hour FastImportBlockFile() rebuild.
|
||||
fs::path txleveldb = dataDir / "txleveldb";
|
||||
fs::path chainDbPath = dataDir / GetActiveChainDbDirName();
|
||||
fs::path database = dataDir / "database";
|
||||
fs::path manifestPath = dataDir / "snapshot.manifest";
|
||||
|
||||
bool keepIndex = false;
|
||||
|
||||
if (fs::exists(manifestPath) && fs::exists(txleveldb)) {
|
||||
if (fs::exists(manifestPath) && fs::exists(chainDbPath)) {
|
||||
SnapshotManifest manifest;
|
||||
std::string manifestError;
|
||||
|
||||
@@ -537,9 +755,9 @@ bool DownloadBootstrap(const std::string& host,
|
||||
if (!keepIndex) {
|
||||
// No valid manifest or verification failed - delete the index.
|
||||
// FastImportBlockFile() will rebuild from blk0001.dat on next startup.
|
||||
printf("Bootstrap: removing extracted txleveldb/ (will rebuild index from blk0001.dat)\n");
|
||||
if (fs::exists(txleveldb))
|
||||
fs::remove_all(txleveldb);
|
||||
printf("Bootstrap: removing extracted %s/ (will rebuild index from blk0001.dat)\n", GetActiveChainDbDirName());
|
||||
if (fs::exists(chainDbPath))
|
||||
fs::remove_all(chainDbPath);
|
||||
}
|
||||
|
||||
// Always remove BDB database/ dir (wallet environment from another machine)
|
||||
@@ -553,4 +771,38 @@ bool DownloadBootstrap(const std::string& host,
|
||||
return true;
|
||||
}
|
||||
|
||||
bool DownloadUtxoSnapshot(const std::string& host,
|
||||
const fs::path& dataDir,
|
||||
ProgressCallback progressFn,
|
||||
std::string& strError)
|
||||
{
|
||||
const bool noProxy = true;
|
||||
const char* snapshotFilename = "utxo-snapshot.bin";
|
||||
|
||||
// Download utxo-snapshot.bin to a temp file
|
||||
fs::path tmpPath = dataDir / "utxo-snapshot.bin.tmp";
|
||||
std::string urlPath = std::string(BASE_PATH) + snapshotFilename;
|
||||
|
||||
printf("Bootstrap: downloading UTXO snapshot from %s%s...\n", host.c_str(), urlPath.c_str());
|
||||
|
||||
if (!DownloadFile(host, urlPath, tmpPath, progressFn, strError, noProxy)) {
|
||||
fs::remove(tmpPath);
|
||||
return false;
|
||||
}
|
||||
|
||||
printf("Bootstrap: UTXO snapshot downloaded, loading into database...\n");
|
||||
|
||||
// Load the snapshot into a fresh active chain DB
|
||||
if (!UtxoSnapshot::LoadSnapshot(tmpPath, dataDir, strError)) {
|
||||
fs::remove(tmpPath);
|
||||
return false;
|
||||
}
|
||||
|
||||
// Clean up the temp file
|
||||
fs::remove(tmpPath);
|
||||
|
||||
printf("Bootstrap: UTXO snapshot loaded successfully.\n");
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace Bootstrap
|
||||
|
||||
+17
-3
@@ -22,16 +22,22 @@ namespace Bootstrap {
|
||||
// Check if data dir already has blockchain data
|
||||
bool NeedsBootstrap(const boost::filesystem::path& dataDir);
|
||||
|
||||
// Download a single file via HTTP GET, write to destPath
|
||||
// Download a single file via HTTP GET, write to destPath.
|
||||
// If noProxy is true, bypass Tor SOCKS proxy and connect directly
|
||||
// (used for clearnet bootstrap downloads).
|
||||
// If portOverride is set (>0), uses that port instead of the default PORT.
|
||||
bool DownloadFile(const std::string& host, const std::string& urlPath,
|
||||
const boost::filesystem::path& destPath,
|
||||
ProgressCallback progressFn,
|
||||
std::string& strError);
|
||||
std::string& strError,
|
||||
bool noProxy = false,
|
||||
int portOverride = -1);
|
||||
|
||||
// Fetch the file manifest (list of relative paths to download)
|
||||
bool FetchFileList(const std::string& host,
|
||||
std::vector<std::string>& files,
|
||||
std::string& strError);
|
||||
std::string& strError,
|
||||
bool noProxy = false);
|
||||
|
||||
// Download bootstrap.tar.gz and extract to dataDir.
|
||||
// Falls back to filelist.txt + individual file download if tar.gz unavailable.
|
||||
@@ -58,6 +64,14 @@ namespace Bootstrap {
|
||||
bool VerifyManifest(const SnapshotManifest& manifest,
|
||||
std::string& strError);
|
||||
|
||||
// Download a UTXO snapshot and load it into a fresh txleveldb.
|
||||
// This is much faster than downloading the full bootstrap archive.
|
||||
// Returns true if snapshot was downloaded and loaded successfully.
|
||||
bool DownloadUtxoSnapshot(const std::string& host,
|
||||
const boost::filesystem::path& dataDir,
|
||||
ProgressCallback progressFn,
|
||||
std::string& strError);
|
||||
|
||||
} // namespace Bootstrap
|
||||
|
||||
#endif // TRIANGLES_BOOTSTRAP_H
|
||||
|
||||
@@ -32,10 +32,28 @@ namespace Checkpoints
|
||||
{ 9002, uint256("0xa1e20fb1d44688b763690cf74d6aefe859e4cc32981f9e3f2b2ae9702bbcf249")},
|
||||
{ 10881, uint256("0x4b6554c45e1e6764a6f3c309c47baf53c9edd81f624e52b072518cd15da237e6")},
|
||||
{ 17650, uint256("0x224940e1f986a202209b8e762728d1452ab45870c308abf84905674acf326a47")},
|
||||
{2000000, uint256("0xb0b02d4bb5ffa31f6f22fd042082ca0a085257af34dc2d4b31c3c8567b5574d5")},
|
||||
{2186940, uint256("0xbd952e8d4a612e336d840ad924a7e09395e36bcd9d929b302e47e60b5c3098c0")},
|
||||
{2190000, uint256("0x682baf783581468ba18f9967254a7f3944e8b8c4cc7101e7d99b68f4f9dd5271")},
|
||||
{2200000, uint256("0x0a8d0442f031f1258120f713f34e45f4f9a625fb753558e27b89b32ad5a9a740")},
|
||||
{2205000, uint256("0xf7aa893ec012181e321783d6a5487addf6997377908faa3e760c9054a4217d29")},
|
||||
{2206000, uint256("0x780ae878f8b10b6cbd51ceb2c0799c90d3551fa916be62974375071b9e36581c")},
|
||||
{2207000, uint256("0x8836d67b0f08036c4a7c26ff0a29d4461a52b6d8f552165ad9c1abec2f3cadfd")},
|
||||
};
|
||||
|
||||
// Published UTXO snapshot file SHA256, keyed by snapshot height.
|
||||
// Each entry binds height -> SHA256 of the canonical snapshot file produced by
|
||||
// UtxoSnapshot::DumpSnapshot at that height. Used by SnapshotNet to verify
|
||||
// P2P-delivered snapshots without trusting any peer.
|
||||
//
|
||||
// Maintainers: after producing a snapshot, sha256 the file and add an entry
|
||||
// here. The corresponding (height, blockhash) must already exist in
|
||||
// mapCheckpoints / mapCheckpointsTestnet.
|
||||
static std::map<int, uint256> mapSnapshotHashes = {
|
||||
// {2186940, uint256("0x...sha256-of-utxo-snapshot.bin...")},
|
||||
};
|
||||
|
||||
static std::map<int, uint256> mapSnapshotHashesTestnet = {
|
||||
};
|
||||
|
||||
static MapCheckpoints mapCheckpointsTestnet = {
|
||||
@@ -51,10 +69,15 @@ namespace Checkpoints
|
||||
{ 9002, uint256("0xa1e20fb1d44688b763690cf74d6aefe859e4cc32981f9e3f2b2ae9702bbcf249")},
|
||||
{ 10881, uint256("0x4b6554c45e1e6764a6f3c309c47baf53c9edd81f624e52b072518cd15da237e6")},
|
||||
{ 17650, uint256("0x224940e1f986a202209b8e762728d1452ab45870c308abf84905674acf326a47")},
|
||||
{2000000, uint256("0xb0b02d4bb5ffa31f6f22fd042082ca0a085257af34dc2d4b31c3c8567b5574d5")},
|
||||
{2186940, uint256("0xbd952e8d4a612e336d840ad924a7e09395e36bcd9d929b302e47e60b5c3098c0")},
|
||||
{2190000, uint256("0x682baf783581468ba18f9967254a7f3944e8b8c4cc7101e7d99b68f4f9dd5271")},
|
||||
{2200000, uint256("0x0a8d0442f031f1258120f713f34e45f4f9a625fb753558e27b89b32ad5a9a740")},
|
||||
{2205000, uint256("0xf7aa893ec012181e321783d6a5487addf6997377908faa3e760c9054a4217d29")},
|
||||
{2206000, uint256("0x780ae878f8b10b6cbd51ceb2c0799c90d3551fa916be62974375071b9e36581c")},
|
||||
{2207000, uint256("0x8836d67b0f08036c4a7c26ff0a29d4461a52b6d8f552165ad9c1abec2f3cadfd")},
|
||||
{2208000, uint256("0xe4a19e8a29fa7aae47f7563377af3e896fee18ed069a64e325b8c2c6c820a1be")},
|
||||
{2209000, uint256("0x04c78a6fc863bed918a9364c58c64489943b2e85d84ddb1ac2fba584f390d5dc")},
|
||||
};
|
||||
|
||||
bool CheckHardened(int nHeight, const uint256& hash)
|
||||
@@ -81,6 +104,22 @@ namespace Checkpoints
|
||||
return checkpoints.rbegin()->first;
|
||||
}
|
||||
|
||||
int GetBestSnapshotHeight()
|
||||
{
|
||||
std::map<int, uint256>& snaps = (fTestNet ? mapSnapshotHashesTestnet : mapSnapshotHashes);
|
||||
if (snaps.empty()) return 0;
|
||||
return snaps.rbegin()->first;
|
||||
}
|
||||
|
||||
bool GetSnapshotHash(int nHeight, uint256& fileHashOut)
|
||||
{
|
||||
std::map<int, uint256>& snaps = (fTestNet ? mapSnapshotHashesTestnet : mapSnapshotHashes);
|
||||
auto it = snaps.find(nHeight);
|
||||
if (it == snaps.end()) return false;
|
||||
fileHashOut = it->second;
|
||||
return true;
|
||||
}
|
||||
|
||||
CBlockIndex* GetLastCheckpoint(const std::map<uint256, CBlockIndex*>& mapBlockIndex)
|
||||
{
|
||||
MapCheckpoints& checkpoints = (fTestNet ? mapCheckpointsTestnet : mapCheckpoints);
|
||||
|
||||
@@ -45,6 +45,11 @@ namespace Checkpoints
|
||||
// Return conservative estimate of total number of blocks, 0 if unknown
|
||||
int GetTotalBlocksEstimate();
|
||||
|
||||
// Return the highest checkpoint height that has a published UTXO snapshot
|
||||
// hash, along with the snapshot's file SHA256. Returns 0 height if none.
|
||||
int GetBestSnapshotHeight();
|
||||
bool GetSnapshotHash(int nHeight, uint256& fileHashOut);
|
||||
|
||||
// Returns last CBlockIndex* in mapBlockIndex that is a checkpoint
|
||||
CBlockIndex* GetLastCheckpoint(const std::map<uint256, CBlockIndex*>& mapBlockIndex);
|
||||
|
||||
|
||||
+2
-2
@@ -7,8 +7,8 @@
|
||||
|
||||
// These need to be macros, as version.cpp's and triangles-qt.rc's voodoo requires it
|
||||
#define CLIENT_VERSION_MAJOR 5
|
||||
#define CLIENT_VERSION_MINOR 8
|
||||
#define CLIENT_VERSION_REVISION 2
|
||||
#define CLIENT_VERSION_MINOR 9
|
||||
#define CLIENT_VERSION_REVISION 5
|
||||
#define CLIENT_VERSION_BUILD 0
|
||||
|
||||
// Converts the parameter X to a string after macro replacement on X has been performed.
|
||||
|
||||
+114
-18
@@ -14,6 +14,8 @@
|
||||
#include "smessage.h"
|
||||
#include "openssl_compat.h"
|
||||
#include "bootstrap.h"
|
||||
#include "utxosnapshot.h"
|
||||
#include "snapshotnet.h"
|
||||
#include "tor/tor_embedded.h"
|
||||
#include "tor/onion_v3.h"
|
||||
#include "tor/tor_process.h"
|
||||
@@ -107,6 +109,31 @@ bool ShutdownRequested()
|
||||
return fRequestShutdown;
|
||||
}
|
||||
|
||||
// P2P UTXO snapshot fetcher. Started from AppInit2 step 11.6 when the chain
|
||||
// is empty and snapshot mode is enabled. Saves utxo-snapshot.bin on success
|
||||
// and requests shutdown so a fresh boot can load it via Step 6c.
|
||||
static void ThreadSnapshotFetch(void* parg)
|
||||
{
|
||||
RenameThread("Triangles-snapfetch");
|
||||
// Give peers ~30s to connect and complete version handshake.
|
||||
for (int i = 0; i < 30 && !fRequestShutdown; ++i)
|
||||
MilliSleep(1000);
|
||||
if (fRequestShutdown) return;
|
||||
|
||||
int snapTimeoutSec = (int)GetArg("-snapshottimeout", 600);
|
||||
printf("SnapshotNet: starting P2P snapshot fetch (timeout=%ds)...\n", snapTimeoutSec);
|
||||
|
||||
std::string err;
|
||||
if (SnapshotNet::TryFetchSnapshot(GetDataDir(), snapTimeoutSec, err)) {
|
||||
printf("SnapshotNet: snapshot saved. Shutting down — restart the daemon to load it.\n");
|
||||
uiInterface.InitMessage(_("UTXO snapshot saved. Restart the node to load it."));
|
||||
StartShutdown();
|
||||
} else {
|
||||
printf("SnapshotNet: P2P snapshot fetch failed: %s\n", err.c_str());
|
||||
printf("SnapshotNet: falling back to genesis sync. Use -bootstrap for legacy HTTP fallback.\n");
|
||||
}
|
||||
}
|
||||
|
||||
void ThreadDeferredStartup(void* parg)
|
||||
{
|
||||
// Make this thread recognisable as the deferred startup worker.
|
||||
@@ -223,7 +250,7 @@ void Shutdown(void* parg)
|
||||
pNotificationQueue = NULL;
|
||||
}
|
||||
|
||||
// CTxDB().Close();
|
||||
// CActiveTxDB().Close();
|
||||
bitdb.Flush(false);
|
||||
bitdb.Flush(true);
|
||||
fs::remove(GetPidFile());
|
||||
@@ -669,7 +696,7 @@ bool AppInit2()
|
||||
nScriptCheckThreads = 16;
|
||||
if (nScriptCheckThreads > 1)
|
||||
{
|
||||
pScriptCheckQueue = new CCheckQueue<CScriptCheck>(128);
|
||||
pScriptCheckQueue = new CCheckQueue<CScriptCheck>(32);
|
||||
pScriptCheckThreads = new boost::thread_group();
|
||||
for (int i = 0; i < nScriptCheckThreads - 1; ++i)
|
||||
pScriptCheckThreads->create_thread(&ThreadScriptCheck);
|
||||
@@ -887,17 +914,25 @@ bool AppInit2()
|
||||
// ********************************************************* Step 6b: bootstrap download (daemon)
|
||||
// Automatic: if data dir has no blockchain, bootstrap without asking.
|
||||
// Can also be forced with -bootstrap flag, or disabled with -nobootstrap.
|
||||
//
|
||||
// v5.9.5: P2P UTXO snapshot fetch is the default for fresh installs (Step 11.6).
|
||||
// The legacy clearnet HTTP bootstrap only runs when the user explicitly requests
|
||||
// it via -bootstrap, or when -snapshot=0 disables the P2P fetcher.
|
||||
#ifndef QT_GUI
|
||||
{
|
||||
bool wantsBootstrap = GetBoolArg("-bootstrap", false);
|
||||
bool noBootstrap = GetBoolArg("-nobootstrap", false);
|
||||
bool snapshotMode = GetBoolArg("-snapshot", true);
|
||||
fs::path dataPath = GetDataDir();
|
||||
bool needsBootstrap = Bootstrap::NeedsBootstrap(dataPath);
|
||||
|
||||
if (needsBootstrap && !noBootstrap) {
|
||||
if (needsBootstrap && !noBootstrap && !snapshotMode) {
|
||||
printf("Bootstrap: no blockchain data found — downloading automatically.\n");
|
||||
printf("Bootstrap: (use -nobootstrap to skip)\n");
|
||||
wantsBootstrap = true;
|
||||
} else if (needsBootstrap && snapshotMode && !wantsBootstrap) {
|
||||
printf("Bootstrap: no blockchain data found — will fetch UTXO snapshot via P2P after network start.\n");
|
||||
printf("Bootstrap: (use -bootstrap for legacy clearnet HTTP bootstrap, or -snapshot=0 to disable P2P fetcher)\n");
|
||||
}
|
||||
|
||||
if (wantsBootstrap)
|
||||
@@ -907,9 +942,6 @@ bool AppInit2()
|
||||
std::string host = Bootstrap::DEFAULT_HOST;
|
||||
std::string strError;
|
||||
|
||||
uiInterface.InitMessage(_("Downloading blockchain snapshot..."));
|
||||
printf("Bootstrap: contacting %s...\n", host.c_str());
|
||||
|
||||
auto progressFn = [](int64_t bytesDownloaded, int64_t totalBytes) {
|
||||
if (totalBytes > 0) {
|
||||
printf("\rBootstrap: %lld / %lld MB (%lld%%)",
|
||||
@@ -920,20 +952,66 @@ bool AppInit2()
|
||||
}
|
||||
};
|
||||
|
||||
bool success = Bootstrap::DownloadBootstrap(host, dataPath, progressFn, strError);
|
||||
// Try UTXO snapshot first (fast: ~2-10 MB download)
|
||||
bool success = false;
|
||||
bool triedUtxoSnapshot = false;
|
||||
if (needsBootstrap && !fs::exists(dataPath / GetActiveChainDbDirName())) {
|
||||
uiInterface.InitMessage(_("Downloading UTXO snapshot..."));
|
||||
printf("Bootstrap: trying UTXO snapshot from %s (fast path)...\n", host.c_str());
|
||||
|
||||
if (!success) {
|
||||
printf("\nBootstrap: failed: %s\n", strError.c_str());
|
||||
printf("Bootstrap: skipping, will sync from network.\n");
|
||||
} else {
|
||||
printf("\nBootstrap: done.\n");
|
||||
std::string utxoError;
|
||||
if (Bootstrap::DownloadUtxoSnapshot(host, dataPath, progressFn, utxoError)) {
|
||||
printf("\nBootstrap: UTXO snapshot loaded — will sync remaining blocks from network.\n");
|
||||
success = true;
|
||||
} else {
|
||||
printf("\nBootstrap: UTXO snapshot unavailable: %s\n", utxoError.c_str());
|
||||
printf("Bootstrap: falling back to full bootstrap download...\n");
|
||||
}
|
||||
triedUtxoSnapshot = true;
|
||||
}
|
||||
|
||||
// Fall back to full bootstrap.tar.gz if UTXO snapshot failed
|
||||
if (!success) {
|
||||
uiInterface.InitMessage(_("Downloading blockchain snapshot..."));
|
||||
printf("Bootstrap: contacting %s...\n", host.c_str());
|
||||
|
||||
success = Bootstrap::DownloadBootstrap(host, dataPath, progressFn, strError);
|
||||
|
||||
if (!success) {
|
||||
printf("\nBootstrap: failed: %s\n", strError.c_str());
|
||||
printf("Bootstrap: skipping, will sync from network.\n");
|
||||
} else {
|
||||
printf("\nBootstrap: done.\n");
|
||||
}
|
||||
}
|
||||
|
||||
StartupPerfLog("bootstrap_download", GetTimeMillis() - nBootstrapStart,
|
||||
strprintf("host=%s success=%d", host.c_str(), success));
|
||||
strprintf("host=%s success=%d utxo_snapshot=%d", host.c_str(), success, triedUtxoSnapshot));
|
||||
}
|
||||
} // end bootstrap scope
|
||||
#endif
|
||||
|
||||
// ********************************************************* Step 6c: manual UTXO snapshot loading
|
||||
// If utxo-snapshot.bin exists in data dir and no active chain DB, load it.
|
||||
{
|
||||
fs::path dataPath = GetDataDir();
|
||||
fs::path snapshotFile = dataPath / "utxo-snapshot.bin";
|
||||
fs::path chainDbDir = dataPath / GetActiveChainDbDirName();
|
||||
|
||||
if (fs::exists(snapshotFile) && !fs::exists(chainDbDir)) {
|
||||
printf("Found utxo-snapshot.bin — loading UTXO snapshot...\n");
|
||||
uiInterface.InitMessage(_("Loading UTXO snapshot..."));
|
||||
|
||||
std::string strError;
|
||||
if (UtxoSnapshot::LoadSnapshot(snapshotFile, dataPath, strError)) {
|
||||
printf("UTXO snapshot loaded successfully.\n");
|
||||
} else {
|
||||
printf("UTXO snapshot load failed: %s\n", strError.c_str());
|
||||
printf("Will proceed with normal sync.\n");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ********************************************************* Step 7: load blockchain
|
||||
|
||||
if (!bitdb.Open(GetDataDir()))
|
||||
@@ -946,7 +1024,7 @@ bool AppInit2()
|
||||
|
||||
if (GetBoolArg("-loadblockindextest"))
|
||||
{
|
||||
CTxDB txdb("r");
|
||||
CActiveTxDB txdb("r");
|
||||
txdb.LoadBlockIndex();
|
||||
PrintBlockTree();
|
||||
return false;
|
||||
@@ -959,9 +1037,9 @@ bool AppInit2()
|
||||
{
|
||||
printf("Reindex requested: removing block index database...\n");
|
||||
uiInterface.InitMessage(_("Removing block index for reindex..."));
|
||||
fs::path txleveldbPath = GetDataDir() / "txleveldb";
|
||||
if (fs::exists(txleveldbPath))
|
||||
fs::remove_all(txleveldbPath);
|
||||
fs::path chainDbPath = GetDataDir() / GetActiveChainDbDirName();
|
||||
if (fs::exists(chainDbPath))
|
||||
fs::remove_all(chainDbPath);
|
||||
}
|
||||
|
||||
uiInterface.InitMessage(_("Loading block index..."));
|
||||
@@ -1146,7 +1224,7 @@ bool AppInit2()
|
||||
bool fScannedWithIndex = false;
|
||||
if (fAddressIndex && !GetBoolArg("-rescan"))
|
||||
{
|
||||
CTxDB txdb("r");
|
||||
CActiveTxDB txdb("r");
|
||||
int nAddressIndexStartHeight = 0;
|
||||
uint256 hashAddressIndexBestChain = 0;
|
||||
if (txdb.ReadAddressIndexStartHeight(nAddressIndexStartHeight) &&
|
||||
@@ -1388,6 +1466,24 @@ bool AppInit2()
|
||||
if (fServer)
|
||||
NewThread(ThreadRPCServer, NULL);
|
||||
|
||||
// ********************************************************* Step 11.6: P2P UTXO snapshot fetch
|
||||
// If the chain is empty and snapshot mode is enabled (default), spawn a
|
||||
// background thread that waits for snapshot-capable peers, downloads the
|
||||
// canonical snapshot via P2P, and saves it to utxo-snapshot.bin. On
|
||||
// success, requests a clean shutdown so the user can restart and have
|
||||
// Step 6c load the snapshot in a fresh boot.
|
||||
{
|
||||
bool snapshotMode = GetBoolArg("-snapshot", true);
|
||||
bool needsSnapshot = (nBestHeight <= 0);
|
||||
bool haveSnapshotFile = fs::exists(GetDataDir() / "utxo-snapshot.bin");
|
||||
|
||||
if (snapshotMode && needsSnapshot && !haveSnapshotFile &&
|
||||
Checkpoints::GetBestSnapshotHeight() > 0)
|
||||
{
|
||||
NewThread(ThreadSnapshotFetch, NULL);
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
LOCK(cs_DeferredStartup);
|
||||
fDeferredStartupRunning = true;
|
||||
|
||||
+6
-2
@@ -31,9 +31,13 @@ int64_t GetWeight(int64_t nIntervalBeginning, int64_t nIntervalEnd)
|
||||
if (nAge < 0)
|
||||
return 0;
|
||||
|
||||
// After v5 fork: remove max age cap so coins aged during the freeze can stake
|
||||
// After v5 fork: use soft cap of 7 days instead of hard nStakeMaxAge.
|
||||
// This prevents "stake surprise" where a whale who was offline for weeks
|
||||
// comes back with massively amplified staking power and dominates blocks.
|
||||
// The 7-day cap still allows generous accumulation while limiting abuse.
|
||||
static const int64_t STAKE_AGE_SOFT_CAP = 7 * 24 * 60 * 60; // 7 days
|
||||
if (pindexBest && pindexBest->nHeight >= FORK_HEIGHT_V5)
|
||||
return nAge;
|
||||
return min(nAge, STAKE_AGE_SOFT_CAP);
|
||||
|
||||
return min(nAge, (int64_t)nStakeMaxAge);
|
||||
}
|
||||
|
||||
+998
-120
File diff suppressed because it is too large
Load Diff
+148
-19
@@ -12,6 +12,7 @@
|
||||
#include "scrypt.h"
|
||||
#include "hashblock.h"
|
||||
#include "checkqueue.h"
|
||||
#include "sigcache.h"
|
||||
|
||||
#include <list>
|
||||
|
||||
@@ -37,8 +38,9 @@ static const unsigned int MAX_BLOCK_SIZE = 1000000;
|
||||
static const unsigned int MAX_BLOCK_SIZE_GEN = MAX_BLOCK_SIZE/2;
|
||||
static const unsigned int MAX_BLOCK_SIGOPS = MAX_BLOCK_SIZE/50;
|
||||
static const unsigned int MAX_ORPHAN_TRANSACTIONS = MAX_BLOCK_SIZE/100;
|
||||
static const unsigned int MAX_ORPHAN_BLOCKS = 2000;
|
||||
static const unsigned int MAX_ORPHAN_BLOCKS_IBD = 4000;
|
||||
static const unsigned int MAX_ORPHAN_BLOCKS = 750;
|
||||
static const unsigned int MAX_ORPHAN_BLOCKS_IBD = 1500;
|
||||
static const unsigned int MAX_REORG_DEPTH = 100; // reject reorgs deeper than this (finality)
|
||||
static const unsigned int MAX_INV_SZ = 50000;
|
||||
static const int64_t MIN_TX_FEE = (1 * CENT) / 100;
|
||||
static const int64_t MIN_RELAY_TX_FEE = (1 * CENT) / 100;
|
||||
@@ -59,10 +61,10 @@ static const int fHaveUPnP = false;
|
||||
|
||||
static const uint256 hashGenesisBlockOfficial("0x7e7a6e4dd5fe895106fca912dfbacaeaf2a89e76c6a588df8ff96e0e18b96021");
|
||||
static const uint256 hashGenesisBlockTestNet ("0x7e7a6e4dd5fe895106fca912dfbacaeaf2a89e76c6a588df8ff96e0e18b96021");
|
||||
inline int64_t GetMaxTimeDrift(int nHeight) { return (nHeight >= FORK_HEIGHT_V5_4) ? 3 * 60 : 10 * 60; }
|
||||
inline int64_t GetMaxTimeDrift(int nHeight) { return (nHeight >= FORK_HEIGHT_V5_4) ? 90 : 10 * 60; }
|
||||
inline int64_t PastDrift(int64_t nTime, int nHeight) { return nTime - GetMaxTimeDrift(nHeight); }
|
||||
inline int64_t FutureDrift(int64_t nTime, int nHeight) { return nTime + GetMaxTimeDrift(nHeight); }
|
||||
// Height-less overloads always use post-V5.4 rules (3-min drift).
|
||||
// Height-less overloads always use post-V5.4 rules (90-second drift).
|
||||
// All nodes are well past FORK_HEIGHT_V5_4; using the global nBestHeight
|
||||
// here previously caused nodes at different heights to disagree on block
|
||||
// validity during the fork transition — a consensus-splitting bug.
|
||||
@@ -83,6 +85,7 @@ extern uint256 nBestChainTrust;
|
||||
extern uint256 nBestInvalidTrust;
|
||||
extern uint256 hashBestChain;
|
||||
extern CBlockIndex* pindexBest;
|
||||
extern CBlockIndex* pindexFinalized; // auto-checkpoint: deepest finalized block
|
||||
extern unsigned int nTransactionsUpdated;
|
||||
extern uint64_t nLastBlockTx;
|
||||
extern uint64_t nLastBlockSize;
|
||||
@@ -107,7 +110,7 @@ extern bool fEnforceCanonical;
|
||||
static const uint64_t nMinDiskSpace = 52428800;
|
||||
|
||||
class CReserveKey;
|
||||
class CTxDB;
|
||||
class CTxDBBase;
|
||||
class CTxIndex;
|
||||
|
||||
void RegisterWallet(CWallet* pwalletIn);
|
||||
@@ -718,10 +721,10 @@ public:
|
||||
}
|
||||
|
||||
|
||||
bool ReadFromDisk(CTxDB& txdb, COutPoint prevout, CTxIndex& txindexRet);
|
||||
bool ReadFromDisk(CTxDB& txdb, COutPoint prevout);
|
||||
bool ReadFromDisk(CTxDBBase& txdb, COutPoint prevout, CTxIndex& txindexRet);
|
||||
bool ReadFromDisk(CTxDBBase& txdb, COutPoint prevout);
|
||||
bool ReadFromDisk(COutPoint prevout);
|
||||
bool DisconnectInputs(CTxDB& txdb);
|
||||
bool DisconnectInputs(CTxDBBase& txdb);
|
||||
|
||||
/** Fetch UTXO entries for all inputs from the UTXO database or mempool.
|
||||
|
||||
@@ -733,7 +736,7 @@ public:
|
||||
@param[out] fInvalid returns true if transaction is invalid
|
||||
@return Returns true if all inputs are found
|
||||
*/
|
||||
bool FetchInputs(CTxDB& txdb, const MapPrevTx& mapPendingUtxos,
|
||||
bool FetchInputs(CTxDBBase& txdb, const MapPrevTx& mapPendingUtxos,
|
||||
bool fBlock, bool fMiner, MapPrevTx& inputsRet, bool& fInvalid);
|
||||
|
||||
/** Validate inputs against UTXO entries and verify signatures.
|
||||
@@ -744,13 +747,13 @@ public:
|
||||
@param[in] fMiner true if called from CreateNewBlock
|
||||
@return Returns true if all checks succeed
|
||||
*/
|
||||
bool ConnectInputs(CTxDB& txdb, const MapPrevTx& inputs,
|
||||
bool ConnectInputs(CTxDBBase& txdb, const MapPrevTx& inputs,
|
||||
const CBlockIndex* pindexBlock, bool fBlock, bool fMiner,
|
||||
std::vector<CScriptCheck>* pvChecks = NULL);
|
||||
bool ClientConnectInputs();
|
||||
bool CheckTransaction() const;
|
||||
bool AcceptToMemoryPool(CTxDB& txdb, bool fCheckInputs=true, bool* pfMissingInputs=NULL);
|
||||
bool GetCoinAge(CTxDB& txdb, uint64_t& nCoinAge) const; // triangles: get transaction coin age
|
||||
bool AcceptToMemoryPool(CTxDBBase& txdb, bool fCheckInputs=true, bool* pfMissingInputs=NULL);
|
||||
bool GetCoinAge(CTxDBBase& txdb, uint64_t& nCoinAge) const; // triangles: get transaction coin age
|
||||
|
||||
protected:
|
||||
const CTxOut& GetOutputFor(const CTxIn& input, const MapPrevTx& inputs) const;
|
||||
@@ -812,7 +815,7 @@ public:
|
||||
int GetDepthInMainChain() const { CBlockIndex *pindexRet; return GetDepthInMainChain(pindexRet); }
|
||||
bool IsInMainChain() const { CBlockIndex *pindexRet; return GetDepthInMainChainINTERNAL(pindexRet) > 0; }
|
||||
int GetBlocksToMaturity() const;
|
||||
bool AcceptToMemoryPool(CTxDB& txdb, bool fCheckInputs=true);
|
||||
bool AcceptToMemoryPool(CTxDBBase& txdb, bool fCheckInputs=true);
|
||||
bool AcceptToMemoryPool();
|
||||
};
|
||||
|
||||
@@ -1143,10 +1146,10 @@ public:
|
||||
}
|
||||
|
||||
|
||||
bool DisconnectBlock(CTxDB& txdb, CBlockIndex* pindex);
|
||||
bool ConnectBlock(CTxDB& txdb, CBlockIndex* pindex, bool fJustCheck=false);
|
||||
bool DisconnectBlock(CTxDBBase& txdb, CBlockIndex* pindex);
|
||||
bool ConnectBlock(CTxDBBase& txdb, CBlockIndex* pindex, bool fJustCheck=false);
|
||||
bool ReadFromDisk(const CBlockIndex* pindex, bool fReadTransactions=true);
|
||||
bool SetBestChain(CTxDB& txdb, CBlockIndex* pindexNew);
|
||||
bool SetBestChain(CTxDBBase& txdb, CBlockIndex* pindexNew);
|
||||
bool AddToBlockIndex(unsigned int nFile, unsigned int nBlockPos, const uint256& hashProofOfStake);
|
||||
bool CheckBlock(bool fCheckPOW=true, bool fCheckMerkleRoot=true, bool fCheckSig=true) const;
|
||||
bool AcceptBlock();
|
||||
@@ -1155,7 +1158,7 @@ public:
|
||||
bool CheckBlockSignature() const;
|
||||
|
||||
private:
|
||||
bool SetBestChainInner(CTxDB& txdb, CBlockIndex *pindexNew);
|
||||
bool SetBestChainInner(CTxDBBase& txdb, CBlockIndex *pindexNew);
|
||||
};
|
||||
|
||||
|
||||
@@ -1560,6 +1563,12 @@ public:
|
||||
return vHave.empty();
|
||||
}
|
||||
|
||||
// Return the first hash in the locator (peer's tip), or 0 if empty
|
||||
uint256 GetTipHash() const
|
||||
{
|
||||
return vHave.empty() ? uint256(0) : vHave[0];
|
||||
}
|
||||
|
||||
void Set(const CBlockIndex* pindex)
|
||||
{
|
||||
vHave.clear();
|
||||
@@ -1653,7 +1662,7 @@ public:
|
||||
std::map<uint256, CTransaction> mapTx;
|
||||
std::map<COutPoint, CInPoint> mapNextTx;
|
||||
|
||||
bool accept(CTxDB& txdb, CTransaction &tx,
|
||||
bool accept(CTxDBBase& txdb, CTransaction &tx,
|
||||
bool fCheckInputs, bool* pfMissingInputs);
|
||||
bool addUnchecked(const uint256& hash, CTransaction &tx);
|
||||
bool remove(const CTransaction &tx, bool fRecursive = false);
|
||||
@@ -1679,6 +1688,121 @@ public:
|
||||
};
|
||||
|
||||
extern CTxMemPool mempool;
|
||||
extern CScriptVerifyCache scriptVerifyCache;
|
||||
|
||||
/**
|
||||
* Compact block relay for Tor-only networks.
|
||||
*
|
||||
* Instead of sending a full block, send the header + short transaction IDs.
|
||||
* The receiver reconstructs the block from its mempool. For PoS blocks with
|
||||
* 0-2 transactions (the common case), the coinstake is always prefilled, so
|
||||
* the compact block IS the complete block — no extra round-trip needed.
|
||||
*/
|
||||
|
||||
/** Short transaction ID: first 6 bytes of SipHash(txid) */
|
||||
static inline uint64_t GetShortTxId(const uint256& txhash, uint64_t nonce)
|
||||
{
|
||||
// Simple short ID: XOR txhash prefix with nonce
|
||||
uint64_t id = 0;
|
||||
memcpy(&id, txhash.begin(), 6); // first 6 bytes
|
||||
id ^= nonce;
|
||||
return id & 0xFFFFFFFFFFFFULL; // mask to 48 bits
|
||||
}
|
||||
|
||||
class CCompactBlock
|
||||
{
|
||||
public:
|
||||
// Block header fields
|
||||
int nVersion;
|
||||
uint256 hashPrevBlock;
|
||||
uint256 hashMerkleRoot;
|
||||
unsigned int nTime;
|
||||
unsigned int nBits;
|
||||
unsigned int nNonce;
|
||||
std::vector<unsigned char> vchBlockSig;
|
||||
|
||||
// Compact block data
|
||||
uint64_t nShortIdNonce; // nonce for short ID calculation
|
||||
std::vector<uint64_t> vShortTxIds; // short IDs for non-prefilled txs
|
||||
std::vector<std::pair<uint16_t, CTransaction>> vPrefilledTxn; // index + full tx
|
||||
|
||||
CCompactBlock() : nVersion(0), nTime(0), nBits(0), nNonce(0), nShortIdNonce(0) {}
|
||||
|
||||
// Construct from a full block: prefill coinbase + coinstake, short-ID the rest
|
||||
CCompactBlock(const CBlock& block)
|
||||
{
|
||||
nVersion = block.nVersion;
|
||||
hashPrevBlock = block.hashPrevBlock;
|
||||
hashMerkleRoot = block.hashMerkleRoot;
|
||||
nTime = block.nTime;
|
||||
nBits = block.nBits;
|
||||
nNonce = block.nNonce;
|
||||
vchBlockSig = block.vchBlockSig;
|
||||
nShortIdNonce = GetRand(std::numeric_limits<uint64_t>::max());
|
||||
|
||||
for (uint16_t i = 0; i < block.vtx.size(); i++)
|
||||
{
|
||||
if (i <= 1) {
|
||||
// Always prefill coinbase (idx 0) and coinstake (idx 1)
|
||||
vPrefilledTxn.push_back(std::make_pair(i, block.vtx[i]));
|
||||
} else {
|
||||
vShortTxIds.push_back(GetShortTxId(block.vtx[i].GetHash(), nShortIdNonce));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
IMPLEMENT_SERIALIZE
|
||||
(
|
||||
READWRITE(nVersion);
|
||||
READWRITE(hashPrevBlock);
|
||||
READWRITE(hashMerkleRoot);
|
||||
READWRITE(nTime);
|
||||
READWRITE(nBits);
|
||||
READWRITE(nNonce);
|
||||
READWRITE(vchBlockSig);
|
||||
READWRITE(nShortIdNonce);
|
||||
READWRITE(vShortTxIds);
|
||||
READWRITE(vPrefilledTxn);
|
||||
)
|
||||
|
||||
uint256 GetBlockHash() const
|
||||
{
|
||||
CBlock hdr;
|
||||
hdr.nVersion = nVersion;
|
||||
hdr.hashPrevBlock = hashPrevBlock;
|
||||
hdr.hashMerkleRoot = hashMerkleRoot;
|
||||
hdr.nTime = nTime;
|
||||
hdr.nBits = nBits;
|
||||
hdr.nNonce = nNonce;
|
||||
return hdr.GetHash();
|
||||
}
|
||||
};
|
||||
|
||||
class CBlockTxnRequest
|
||||
{
|
||||
public:
|
||||
uint256 blockhash;
|
||||
std::vector<uint16_t> vIndex; // indices of missing transactions
|
||||
|
||||
IMPLEMENT_SERIALIZE
|
||||
(
|
||||
READWRITE(blockhash);
|
||||
READWRITE(vIndex);
|
||||
)
|
||||
};
|
||||
|
||||
class CBlockTxnResponse
|
||||
{
|
||||
public:
|
||||
uint256 blockhash;
|
||||
std::vector<CTransaction> vTxn;
|
||||
|
||||
IMPLEMENT_SERIALIZE
|
||||
(
|
||||
READWRITE(blockhash);
|
||||
READWRITE(vTxn);
|
||||
)
|
||||
};
|
||||
|
||||
/**
|
||||
* Closure representing one script check for parallel verification.
|
||||
@@ -1703,7 +1827,12 @@ public:
|
||||
|
||||
bool operator()()
|
||||
{
|
||||
return ptxTo && VerifyScript(scriptSig, scriptPubKey, *ptxTo, nIn, nHashType);
|
||||
if (!ptxTo)
|
||||
return false;
|
||||
if (!VerifyScript(scriptSig, scriptPubKey, *ptxTo, nIn, nHashType))
|
||||
return false;
|
||||
scriptVerifyCache.Set(ptxTo->GetHash(), nIn);
|
||||
return true;
|
||||
}
|
||||
|
||||
void swap(CScriptCheck& other)
|
||||
|
||||
+14
-3
@@ -413,7 +413,7 @@ void StakeMiner(CWallet *pwallet)
|
||||
if (fTryToSync)
|
||||
{
|
||||
fTryToSync = false;
|
||||
if (vNodes.size() < 1 || nBestHeight < GetNumBlocksOfPeers())
|
||||
if (vNodes.size() < 2 || nBestHeight < GetNumBlocksOfPeers())
|
||||
{
|
||||
MilliSleep(60000);
|
||||
continue;
|
||||
@@ -446,9 +446,20 @@ void StakeMiner(CWallet *pwallet)
|
||||
{
|
||||
printf("StakeMiner(): A proof-of-stake block has been found! %s\n", pblock->GetHash().ToString().c_str());
|
||||
SetThreadPriority(THREAD_PRIORITY_NORMAL);
|
||||
CheckStake(pblock.get(), *pwallet);
|
||||
bool fAccepted = CheckStake(pblock.get(), *pwallet);
|
||||
SetThreadPriority(THREAD_PRIORITY_LOWEST);
|
||||
MilliSleep(500);
|
||||
if (fAccepted)
|
||||
{
|
||||
MilliSleep(500);
|
||||
}
|
||||
else
|
||||
{
|
||||
// Block was orphaned or rejected — apply a cooldown to reduce
|
||||
// fork oscillation. Without this, the staker immediately retries
|
||||
// with a different timestamp, potentially creating competing forks.
|
||||
printf("StakeMiner(): block not accepted, cooldown 30s\n");
|
||||
MilliSleep(30000);
|
||||
}
|
||||
}
|
||||
else
|
||||
MilliSleep(500);
|
||||
|
||||
+146
-44
@@ -37,7 +37,7 @@ extern "C" {
|
||||
// int tor_main(int argc, char *argv[]);
|
||||
}
|
||||
|
||||
static const int MAX_OUTBOUND_CONNECTIONS = 16;
|
||||
static const int MAX_OUTBOUND_CONNECTIONS = 8; // reduced from 16 for Tor-only small networks
|
||||
|
||||
void ThreadMessageHandler2(void* parg);
|
||||
void ThreadSocketHandler2(void* parg);
|
||||
@@ -47,7 +47,7 @@ void ThreadOpenAddedConnections2(void* parg);
|
||||
void ThreadMapPort2(void* parg);
|
||||
#endif
|
||||
void ThreadHTTPSeedFetch(void* parg);
|
||||
void ThreadHTTPSeedFetch2(void* parg);
|
||||
bool ThreadHTTPSeedFetch2(void* parg);
|
||||
bool OpenNetworkConnection(const CAddress& addrConnect, CSemaphoreGrant *grantOutbound = NULL, const char *strDest = NULL, bool fOneShot = false);
|
||||
|
||||
|
||||
@@ -690,6 +690,9 @@ void CNode::copyStats(CNodeStats &stats)
|
||||
X(fInbound);
|
||||
X(nStartingHeight);
|
||||
X(nMisbehavior);
|
||||
X(nPingUsecTime);
|
||||
X(nBlocksDelivered);
|
||||
X(nAvgBlockLatencyUs);
|
||||
}
|
||||
#undef X
|
||||
|
||||
@@ -1029,10 +1032,6 @@ void ThreadSocketHandler2(void* parg)
|
||||
if (nErr != WSAEWOULDBLOCK)
|
||||
printf("socket error accept failed: %d\n", nErr);
|
||||
}
|
||||
else if (nInbound >= GetArg("-maxconnections", 125) - MAX_OUTBOUND_CONNECTIONS)
|
||||
{
|
||||
closesocket(hSocket);
|
||||
}
|
||||
else if (CNode::IsBanned(addr))
|
||||
{
|
||||
printf("connection from %s dropped (banned)\n", addr.ToString().c_str());
|
||||
@@ -1040,12 +1039,36 @@ void ThreadSocketHandler2(void* parg)
|
||||
}
|
||||
else
|
||||
{
|
||||
printf("accepted connection %s\n", addr.ToString().c_str());
|
||||
CNode* pnode = new CNode(hSocket, addr, "", true);
|
||||
pnode->AddRef();
|
||||
{
|
||||
LOCK(cs_vNodes);
|
||||
vNodes.push_back(pnode);
|
||||
int nMaxInbound = GetArg("-maxconnections", 125) - MAX_OUTBOUND_CONNECTIONS;
|
||||
bool fAccept = (nInbound < nMaxInbound);
|
||||
|
||||
// Reserve 2 extra inbound slots for known seed nodes
|
||||
if (!fAccept) {
|
||||
bool fIsSeed = false;
|
||||
static const char *(*strOnionSeedCheck)[1] = fTestNet ? strTestNetOnionSeed : strMainNetOnionSeed;
|
||||
std::string incomingAddr = addr.ToStringIP();
|
||||
for (unsigned int si = 0; strOnionSeedCheck[si][0] != NULL; si++) {
|
||||
if (incomingAddr.find(strOnionSeedCheck[si][0]) != std::string::npos) {
|
||||
fIsSeed = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (fIsSeed && nInbound < nMaxInbound + 2) {
|
||||
fAccept = true;
|
||||
printf("accepted seed node %s (reserved slot)\n", addr.ToString().c_str());
|
||||
}
|
||||
}
|
||||
|
||||
if (fAccept) {
|
||||
printf("accepted connection %s\n", addr.ToString().c_str());
|
||||
CNode* pnode = new CNode(hSocket, addr, "", true);
|
||||
pnode->AddRef();
|
||||
{
|
||||
LOCK(cs_vNodes);
|
||||
vNodes.push_back(pnode);
|
||||
}
|
||||
} else {
|
||||
closesocket(hSocket);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1137,14 +1160,14 @@ void ThreadSocketHandler2(void* parg)
|
||||
printf("socket no message in first 60 seconds, %d %d\n", pnode->nLastRecv != 0, pnode->nLastSend != 0);
|
||||
pnode->fDisconnect = true;
|
||||
}
|
||||
else if (GetTime() - pnode->nLastSend > 90*60 && GetTime() - pnode->nLastSendEmpty > 90*60)
|
||||
else if (GetTime() - pnode->nLastSend > 10*60 && GetTime() - pnode->nLastSendEmpty > 10*60)
|
||||
{
|
||||
printf("socket not sending\n");
|
||||
printf("socket not sending (10min timeout)\n");
|
||||
pnode->fDisconnect = true;
|
||||
}
|
||||
else if (GetTime() - pnode->nLastRecv > 90*60)
|
||||
else if (GetTime() - pnode->nLastRecv > 10*60)
|
||||
{
|
||||
printf("socket inactivity timeout\n");
|
||||
printf("socket inactivity timeout (10min)\n");
|
||||
pnode->fDisconnect = true;
|
||||
}
|
||||
}
|
||||
@@ -1381,7 +1404,7 @@ void ThreadOnionSeed(void* parg)
|
||||
|
||||
|
||||
|
||||
// Load hardcoded .onion seeds (if any)
|
||||
// Load hardcoded .onion seeds and queue them for immediate direct connection
|
||||
static const char *(*strOnionSeed)[1] = fTestNet ? strTestNetOnionSeed : strMainNetOnionSeed;
|
||||
int found = 0;
|
||||
|
||||
@@ -1394,27 +1417,50 @@ void ThreadOnionSeed(void* parg)
|
||||
CAddress addr = CAddress(CService(parsed, GetDefaultPort()));
|
||||
addr.nTime = GetTime() - 3*nOneDay - GetRand(4*nOneDay);
|
||||
addrman.Add(addr, parsed);
|
||||
|
||||
// Queue for immediate direct connection (OneShot) — don't wait for
|
||||
// addrman selection which deprioritizes stale timestamps
|
||||
std::string oneShotAddr = std::string(strOnionSeed[seed_idx][0])
|
||||
+ ":" + std::to_string(GetDefaultPort());
|
||||
AddOneShot(oneShotAddr);
|
||||
found++;
|
||||
}
|
||||
|
||||
printf("%d addresses from hardcoded .onion seeds\n", found);
|
||||
printf("%d addresses from hardcoded .onion seeds (queued as OneShot)\n", found);
|
||||
|
||||
// Also fetch dynamic seeds from HTTP seed list
|
||||
// Wait for Tor to establish circuits before attempting HTTPS seed fetch.
|
||||
// The hardcoded OneShot connections can race ahead meanwhile.
|
||||
printf("ThreadOnionSeed: waiting 20s for Tor circuits before HTTPS seed fetch...\n");
|
||||
for (int i = 0; i < 20 && !fShutdown; i++)
|
||||
MilliSleep(1000);
|
||||
|
||||
// Fetch dynamic seeds with retry — up to 4 attempts with increasing backoff.
|
||||
// This is the primary discovery mechanism — seeds.cryptographic-triangles.org
|
||||
ThreadHTTPSeedFetch2(NULL);
|
||||
{
|
||||
bool ok = false;
|
||||
int delays[] = {0, 30, 60, 120};
|
||||
for (int attempt = 0; attempt < 4 && !ok && !fShutdown; attempt++) {
|
||||
if (attempt > 0) {
|
||||
printf("ThreadOnionSeed: HTTPS seed fetch retry %d in %ds...\n", attempt, delays[attempt]);
|
||||
for (int i = 0; i < delays[attempt] && !fShutdown; i++)
|
||||
MilliSleep(1000);
|
||||
}
|
||||
if (!fShutdown)
|
||||
ok = ThreadHTTPSeedFetch2(NULL);
|
||||
}
|
||||
if (!ok && !fShutdown)
|
||||
printf("ThreadOnionSeed: all HTTPS seed fetch attempts failed\n");
|
||||
}
|
||||
|
||||
printf("ThreadOnionSeed: initial seeding complete\n");
|
||||
|
||||
// Periodic re-seeding: if the node becomes isolated (0 outbound peers),
|
||||
// re-fetch the seed list. Check every 10 minutes, re-seed at most once
|
||||
// per 30 minutes to avoid hammering the seed server.
|
||||
// Periodic re-seeding for isolated or under-connected nodes.
|
||||
// EMERGENCY MODE: When 0 outbound peers, check every 15 seconds
|
||||
// NORMAL MODE: Check every 2 minutes, re-seed when < 2 outbound peers
|
||||
int64_t nLastReseed = GetTime();
|
||||
bool bFirstReseed = true;
|
||||
while (!fShutdown) {
|
||||
for (int i = 0; i < 600 && !fShutdown; i++) // sleep 10 minutes
|
||||
MilliSleep(1000);
|
||||
|
||||
if (fShutdown) break;
|
||||
|
||||
// Count outbound peers to determine check interval
|
||||
int nOutbound = 0;
|
||||
{
|
||||
LOCK(cs_vNodes);
|
||||
@@ -1423,10 +1469,52 @@ void ThreadOnionSeed(void* parg)
|
||||
nOutbound++;
|
||||
}
|
||||
|
||||
if (nOutbound == 0 && GetTime() - nLastReseed > 30 * 60) {
|
||||
printf("ThreadOnionSeed: no outbound peers, re-seeding...\n");
|
||||
// Emergency mode: 0 peers = check every 15 seconds
|
||||
// Low mode: 1 peer = check every 30 seconds
|
||||
// Normal: 2+ peers = check every 2 minutes
|
||||
int nSleepSeconds = (nOutbound == 0) ? 15 : (nOutbound < 2) ? 30 : 120;
|
||||
for (int i = 0; i < nSleepSeconds && !fShutdown; i++)
|
||||
MilliSleep(1000);
|
||||
|
||||
if (fShutdown) break;
|
||||
|
||||
// Recount after sleep
|
||||
nOutbound = 0;
|
||||
{
|
||||
LOCK(cs_vNodes);
|
||||
for (CNode* pnode : vNodes)
|
||||
if (!pnode->fInbound)
|
||||
nOutbound++;
|
||||
}
|
||||
|
||||
// Emergency (0 peers): no cooldown, reseed immediately
|
||||
// Low (1 peer): 60 second cooldown
|
||||
// Normal (<2): 5 min first, 15 min subsequent
|
||||
int64_t nCooldown;
|
||||
if (nOutbound == 0)
|
||||
nCooldown = 0; // immediate
|
||||
else if (nOutbound < 2)
|
||||
nCooldown = bFirstReseed ? 60 : 5 * 60;
|
||||
else
|
||||
nCooldown = bFirstReseed ? 5 * 60 : 15 * 60;
|
||||
|
||||
if (nOutbound < 2 && GetTime() - nLastReseed > nCooldown) {
|
||||
if (nOutbound == 0)
|
||||
printf("ThreadOnionSeed: EMERGENCY - 0 outbound peers, re-seeding immediately!\n");
|
||||
else
|
||||
printf("ThreadOnionSeed: low outbound peers (%d), re-seeding...\n", nOutbound);
|
||||
|
||||
ThreadHTTPSeedFetch2(NULL);
|
||||
|
||||
// Re-queue hardcoded seeds for direct connection
|
||||
for (unsigned int seed_idx = 0; strOnionSeed[seed_idx][0] != NULL; seed_idx++) {
|
||||
std::string oneShotAddr = std::string(strOnionSeed[seed_idx][0])
|
||||
+ ":" + std::to_string(GetDefaultPort());
|
||||
AddOneShot(oneShotAddr);
|
||||
}
|
||||
|
||||
nLastReseed = GetTime();
|
||||
bFirstReseed = false;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1486,7 +1574,7 @@ void ThreadDumpAddress(void* parg)
|
||||
printf("ThreadDumpAddress exited\n");
|
||||
}
|
||||
|
||||
void ThreadHTTPSeedFetch2(void* parg)
|
||||
bool ThreadHTTPSeedFetch2(void* parg)
|
||||
{
|
||||
static const char* DEFAULT_SEED_URL_HOST = "seeds.cryptographic-triangles.org";
|
||||
static const char* DEFAULT_SEED_URL_PATH = "/seeds.txt";
|
||||
@@ -1515,7 +1603,7 @@ void ThreadHTTPSeedFetch2(void* parg)
|
||||
|
||||
if (!ConnectSocketByName(addrResolved, hSocket, connectDest.c_str(), HTTPS_PORT, nConnectTimeout)) {
|
||||
printf("HTTPS seed fetch: cannot connect to %s through Tor proxy\n", seedHost.c_str());
|
||||
return;
|
||||
return false;
|
||||
}
|
||||
|
||||
// Set up TLS over the connected socket
|
||||
@@ -1523,7 +1611,7 @@ void ThreadHTTPSeedFetch2(void* parg)
|
||||
if (!ctx) {
|
||||
printf("HTTPS seed fetch: SSL_CTX_new failed\n");
|
||||
closesocket(hSocket);
|
||||
return;
|
||||
return false;
|
||||
}
|
||||
|
||||
// Use system default CA certificates for verification
|
||||
@@ -1535,7 +1623,7 @@ void ThreadHTTPSeedFetch2(void* parg)
|
||||
printf("HTTPS seed fetch: SSL_new failed\n");
|
||||
SSL_CTX_free(ctx);
|
||||
closesocket(hSocket);
|
||||
return;
|
||||
return false;
|
||||
}
|
||||
|
||||
// Set SNI hostname (required for Caddy/Let's Encrypt)
|
||||
@@ -1552,7 +1640,7 @@ void ThreadHTTPSeedFetch2(void* parg)
|
||||
SSL_free(ssl);
|
||||
SSL_CTX_free(ctx);
|
||||
closesocket(hSocket);
|
||||
return;
|
||||
return false;
|
||||
}
|
||||
|
||||
printf("HTTPS seed fetch: TLS connection established to %s\n", seedHost.c_str());
|
||||
@@ -1575,7 +1663,7 @@ void ThreadHTTPSeedFetch2(void* parg)
|
||||
SSL_free(ssl);
|
||||
SSL_CTX_free(ctx);
|
||||
closesocket(hSocket);
|
||||
return;
|
||||
return false;
|
||||
}
|
||||
nSent += nBytes;
|
||||
}
|
||||
@@ -1600,21 +1688,21 @@ void ThreadHTTPSeedFetch2(void* parg)
|
||||
|
||||
if (response.empty()) {
|
||||
printf("HTTPS seed fetch: empty response from %s\n", seedHost.c_str());
|
||||
return;
|
||||
return false;
|
||||
}
|
||||
|
||||
// Parse HTTP response - find end of headers
|
||||
size_t headerEnd = response.find("\r\n\r\n");
|
||||
if (headerEnd == std::string::npos) {
|
||||
printf("HTTPS seed fetch: malformed response (no header terminator)\n");
|
||||
return;
|
||||
return false;
|
||||
}
|
||||
|
||||
// Check status code
|
||||
std::string statusLine = response.substr(0, response.find("\r\n"));
|
||||
if (statusLine.find("200") == std::string::npos) {
|
||||
printf("HTTPS seed fetch: %s from %s\n", statusLine.c_str(), seedHost.c_str());
|
||||
return;
|
||||
return false;
|
||||
}
|
||||
|
||||
std::string body = response.substr(headerEnd + 4);
|
||||
@@ -1626,7 +1714,7 @@ void ThreadHTTPSeedFetch2(void* parg)
|
||||
while (std::getline(lines, line))
|
||||
{
|
||||
if (fShutdown)
|
||||
return;
|
||||
return false;
|
||||
|
||||
// Trim whitespace and carriage returns
|
||||
while (!line.empty() && (line.back() == '\r' || line.back() == ' ' || line.back() == '\t'))
|
||||
@@ -1667,17 +1755,24 @@ void ThreadHTTPSeedFetch2(void* parg)
|
||||
CAddress addr(CService(parsed, port));
|
||||
addr.nTime = GetTime() - 3*24*60*60; // 3 days ago
|
||||
addrman.Add(addr, CNetAddr("https-seed", true));
|
||||
// Queue the first 8 seeds for immediate direct connection
|
||||
if (found < 8) {
|
||||
std::string oneShotAddr = addrStr + ":" + std::to_string(port);
|
||||
AddOneShot(oneShotAddr);
|
||||
}
|
||||
found++;
|
||||
}
|
||||
}
|
||||
|
||||
printf("%d addresses found from HTTPS seed list (%s)\n", found, seedHost.c_str());
|
||||
return found > 0;
|
||||
|
||||
} catch (std::exception& e) {
|
||||
printf("HTTPS seed fetch failed: %s\n", e.what());
|
||||
if (ssl) { SSL_shutdown(ssl); SSL_free(ssl); }
|
||||
if (ctx) SSL_CTX_free(ctx);
|
||||
if (hSocket != INVALID_SOCKET) closesocket(hSocket);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2297,8 +2392,11 @@ void StartNode(void* parg)
|
||||
RenameThread("Triangles-start");
|
||||
|
||||
if (semOutbound == NULL) {
|
||||
// initialize semaphore
|
||||
int nMaxOutbound = min(MAX_OUTBOUND_CONNECTIONS, (int)GetArg("-maxconnections", 125));
|
||||
// initialize semaphore — use -maxoutbound if specified, else default
|
||||
int nMaxOutbound = (int)GetArg("-maxoutbound", MAX_OUTBOUND_CONNECTIONS);
|
||||
nMaxOutbound = min(nMaxOutbound, (int)GetArg("-maxconnections", 125));
|
||||
nMaxOutbound = max(nMaxOutbound, 1); // at least 1 outbound
|
||||
printf("Max outbound connections: %d\n", nMaxOutbound);
|
||||
semOutbound = new CSemaphore(nMaxOutbound);
|
||||
}
|
||||
|
||||
@@ -2368,9 +2466,13 @@ bool StopNode()
|
||||
fShutdown = true;
|
||||
nTransactionsUpdated++;
|
||||
int64_t nStart = GetTime();
|
||||
if (semOutbound)
|
||||
for (int i=0; i<MAX_OUTBOUND_CONNECTIONS; i++)
|
||||
if (semOutbound) {
|
||||
int nMaxOutbound = (int)GetArg("-maxoutbound", MAX_OUTBOUND_CONNECTIONS);
|
||||
nMaxOutbound = min(nMaxOutbound, (int)GetArg("-maxconnections", 125));
|
||||
nMaxOutbound = max(nMaxOutbound, 1);
|
||||
for (int i=0; i<nMaxOutbound; i++)
|
||||
semOutbound->post();
|
||||
}
|
||||
do
|
||||
{
|
||||
int nThreadsRunning = 0;
|
||||
|
||||
@@ -26,7 +26,7 @@ extern int nBestHeight;
|
||||
|
||||
|
||||
|
||||
inline unsigned int ReceiveFloodSize() { return 100 * 1024 * 1024; } // 100 MB
|
||||
inline unsigned int ReceiveFloodSize() { return 50 * 1024 * 1024; } // 50 MB (reduced for Tor-only network)
|
||||
inline unsigned int SendBufferSize() { return 32 * 1024 * 1024; } // 32 MB
|
||||
|
||||
void AddOneShot(std::string strDest);
|
||||
@@ -146,6 +146,9 @@ public:
|
||||
bool fInbound;
|
||||
int nStartingHeight;
|
||||
int nMisbehavior;
|
||||
int64_t nPingUsecTime;
|
||||
int nBlocksDelivered;
|
||||
int64_t nAvgBlockLatencyUs;
|
||||
};
|
||||
|
||||
|
||||
@@ -253,6 +256,7 @@ public:
|
||||
bool fSuccessfullyConnected;
|
||||
bool fDisconnect;
|
||||
bool fPreferHeaders; // peer requested block announcements via headers (sendheaders)
|
||||
bool fSendCmpct; // peer supports compact block relay (sendcmpct)
|
||||
CSemaphoreGrant grantOutbound;
|
||||
int nRefCount;
|
||||
protected:
|
||||
@@ -272,6 +276,18 @@ public:
|
||||
CBlockIndex* pindexLastGetHeadersBegin;
|
||||
uint256 hashLastGetHeadersEnd;
|
||||
int nStartingHeight;
|
||||
int64_t nLastTipCheck; // last time we asked this peer for chain tip
|
||||
int64_t nLastIbdHeaderRequest; // last time we sent IBD-mode getheaders to this peer (heartbeat throttle)
|
||||
int64_t nAvgBlockLatencyUs; // rolling average block delivery latency (microseconds)
|
||||
int nBlocksDelivered; // count of blocks delivered by this peer
|
||||
int nBestKnownHeight; // highest block height known to this peer (updated from inv/block msgs)
|
||||
int nIncompatibleGetblocks; // count of getblocks with no common blocks (fork detection)
|
||||
|
||||
// BIP 31 ping/pong latency tracking
|
||||
uint64_t nPingNonceSent; // nonce of last ping sent (0 = no outstanding ping)
|
||||
int64_t nPingUsecStart; // microsecond timestamp when last ping was sent
|
||||
int64_t nPingUsecTime; // last measured round-trip time (microseconds), 0 = unknown
|
||||
int nPingRetryCount; // consecutive pings without pong response
|
||||
|
||||
// flood relay
|
||||
std::vector<CAddress> vAddrToSend;
|
||||
@@ -310,6 +326,7 @@ public:
|
||||
fSuccessfullyConnected = false;
|
||||
fDisconnect = false;
|
||||
fPreferHeaders = false;
|
||||
fSendCmpct = false;
|
||||
nRefCount = 0;
|
||||
nSendSize = 0;
|
||||
nSendOffset = 0;
|
||||
@@ -319,6 +336,16 @@ public:
|
||||
pindexLastGetHeadersBegin = 0;
|
||||
hashLastGetHeadersEnd = 0;
|
||||
nStartingHeight = -1;
|
||||
nLastTipCheck = 0;
|
||||
nLastIbdHeaderRequest = 0;
|
||||
nAvgBlockLatencyUs = 0;
|
||||
nBlocksDelivered = 0;
|
||||
nBestKnownHeight = -1;
|
||||
nIncompatibleGetblocks = 0;
|
||||
nPingNonceSent = 0;
|
||||
nPingUsecStart = 0;
|
||||
nPingUsecTime = 0;
|
||||
nPingRetryCount = 0;
|
||||
fGetAddr = false;
|
||||
nMisbehavior = 0;
|
||||
hashCheckpointKnown = 0;
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
// Hardcoded onion seed nodes for initial peer discovery.
|
||||
// Also fetched dynamically via https://seeds.cryptographic-triangles.org/seeds.txt
|
||||
static const char *strMainNetOnionSeed[][1] = {
|
||||
{"6ygpphp2qsucwvhwefv6h6ehvk6zjf7b7zdp4ggkzjjwe76cg6jwm7id.onion"},
|
||||
{"jbpfhe7zw3qm67wy3j2ayysp3mnrjobopthnko3b3sgahqtecblwqmid.onion"},
|
||||
{"uddaxjbo3lh2zskg7w6gwln4ty5cel7q4c5jbx7fdtv6zf2j47gdlyad.onion"},
|
||||
{"el5sirhhleecuctpeeprelzubpqmoqivvra3rzlwbjttinxa4fq3wnid.onion"},
|
||||
|
||||
+2
-1
@@ -68,7 +68,8 @@ class CMessageHeader
|
||||
/** nServices flags */
|
||||
enum
|
||||
{
|
||||
NODE_NETWORK = (1 << 0),
|
||||
NODE_NETWORK = (1 << 0),
|
||||
NODE_SNAPSHOT = (1 << 1), // peer can serve UTXO snapshot chunks
|
||||
};
|
||||
|
||||
/** A CService with information about it as peer */
|
||||
|
||||
@@ -868,7 +868,7 @@ QWidget#line {
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<a href="http://explorer.triangles.technology"> &#187; TRI block explorer</a></body></string>
|
||||
<a href="https://blocks.cryptographic-triangles.org"> &#187; TRI block explorer</a></body></string>
|
||||
</property>
|
||||
<property name="alignment">
|
||||
<set>Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter</set>
|
||||
|
||||
+255
-27
@@ -9,6 +9,7 @@
|
||||
#include "addressindex.h"
|
||||
#include "txdb.h"
|
||||
#include "base58.h"
|
||||
#include "utxosnapshot.h"
|
||||
|
||||
using namespace json_spirit;
|
||||
using namespace std;
|
||||
@@ -363,49 +364,188 @@ Value gettxoutsetinfo(const Array& params, bool fHelp)
|
||||
return obj;
|
||||
}
|
||||
|
||||
Value recalculatesupply(const Array& params, bool fHelp)
|
||||
static void GetActiveChainVector(std::vector<CBlockIndex*>& chain)
|
||||
{
|
||||
if (fHelp || params.size() != 0)
|
||||
throw runtime_error(
|
||||
"recalculatesupply\n"
|
||||
"Recalculates the money supply by summing all unspent transaction outputs.\n"
|
||||
"Updates the stored money supply value at the chain tip and persists it to disk.\n"
|
||||
"Returns the old and new supply values for comparison.\n"
|
||||
"\nWARNING: This modifies blockchain index state. Only use if money supply is incorrect.");
|
||||
chain.clear();
|
||||
|
||||
if (!pindexBest)
|
||||
throw runtime_error("recalculatesupply: no best block");
|
||||
|
||||
for (CBlockIndex* pindex = pindexBest; pindex; pindex = pindex->pprev)
|
||||
chain.push_back(pindex);
|
||||
|
||||
std::reverse(chain.begin(), chain.end());
|
||||
}
|
||||
|
||||
static int64_t ComputeActiveChainSupplyFromBlocks(const std::vector<CBlockIndex*>& chain, int& nBlocksScanned, int& nTransactionsScanned)
|
||||
{
|
||||
nBlocksScanned = 0;
|
||||
nTransactionsScanned = 0;
|
||||
|
||||
CTxDB txdb("r");
|
||||
int64_t nSupply = 0;
|
||||
|
||||
for (std::vector<CBlockIndex*>::const_iterator pindexIt = chain.begin(); pindexIt != chain.end(); ++pindexIt)
|
||||
{
|
||||
CBlockIndex* pindex = *pindexIt;
|
||||
if (!pindex)
|
||||
throw runtime_error("recalculatesupply: null active-chain block index");
|
||||
|
||||
if (pindex->nHeight == 0)
|
||||
{
|
||||
nBlocksScanned++;
|
||||
continue;
|
||||
}
|
||||
|
||||
CBlock block;
|
||||
int64_t nBlockValueIn = 0;
|
||||
int64_t nBlockValueOut = 0;
|
||||
|
||||
if (!block.ReadFromDisk(pindex))
|
||||
throw runtime_error(strprintf("recalculatesupply: failed reading block at height %d", pindex->nHeight));
|
||||
|
||||
for (std::vector<CTransaction>::const_iterator txIt = block.vtx.begin(); txIt != block.vtx.end(); ++txIt)
|
||||
{
|
||||
const CTransaction& tx = *txIt;
|
||||
nTransactionsScanned++;
|
||||
nBlockValueOut += tx.GetValueOut();
|
||||
|
||||
if (!tx.IsCoinBase())
|
||||
{
|
||||
for (std::vector<CTxIn>::const_iterator txinIt = tx.vin.begin(); txinIt != tx.vin.end(); ++txinIt)
|
||||
{
|
||||
const CTxIn& txin = *txinIt;
|
||||
CTxIndex txindex;
|
||||
CTransaction txPrev;
|
||||
if (!txPrev.ReadFromDisk(txdb, txin.prevout, txindex))
|
||||
throw runtime_error(strprintf(
|
||||
"recalculatesupply: failed reading prevout %s:%u while processing height %d",
|
||||
txin.prevout.hash.ToString().c_str(), txin.prevout.n, pindex->nHeight));
|
||||
|
||||
if (txin.prevout.n >= txPrev.vout.size())
|
||||
throw runtime_error(strprintf(
|
||||
"recalculatesupply: prevout index %u out of range for tx %s at height %d",
|
||||
txin.prevout.n, txin.prevout.hash.ToString().c_str(), pindex->nHeight));
|
||||
|
||||
nBlockValueIn += txPrev.vout[txin.prevout.n].nValue;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
nSupply += (nBlockValueOut - nBlockValueIn);
|
||||
nBlocksScanned++;
|
||||
}
|
||||
|
||||
return nSupply;
|
||||
}
|
||||
|
||||
Value recalculatesupply(const Array& params, bool fHelp)
|
||||
{
|
||||
if (fHelp || params.size() > 1)
|
||||
throw runtime_error(
|
||||
"recalculatesupply [apply=false]\n"
|
||||
"Rebuilds money supply by walking the active chain from genesis and summing (valueOut - valueIn) per block.\n"
|
||||
"Also returns the current UTXO-set total for comparison.\n"
|
||||
"If apply=true, rewrites nMoneySupply for every block on the active chain and persists the repaired values.\n"
|
||||
"\nThis is intended for repairing corrupted money-supply tracking after chain/index incidents.");
|
||||
|
||||
bool fApply = false;
|
||||
if (params.size() == 1)
|
||||
fApply = params[0].get_bool();
|
||||
|
||||
LOCK(cs_main);
|
||||
|
||||
if (!pindexBest)
|
||||
throw runtime_error("recalculatesupply: no best block");
|
||||
|
||||
CTxDB txdbRead("r");
|
||||
int nUtxoCount = 0;
|
||||
CTxDB txdb;
|
||||
int64_t nCalculatedSupply = txdb.SumUtxoValues(nUtxoCount);
|
||||
int64_t nOldSupply = pindexBest->nMoneySupply;
|
||||
int64_t nDifference = nCalculatedSupply - nOldSupply;
|
||||
int64_t nUtxoSupply = txdbRead.SumUtxoValues(nUtxoCount);
|
||||
|
||||
// Sanity check: difference should be reasonable (not millions of TRI)
|
||||
// Max supply is 2,222,222 TRI, so any difference > 1M TRI is suspicious
|
||||
if (abs64(nDifference) > 1000000 * COIN)
|
||||
throw runtime_error(strprintf(
|
||||
"recalculatesupply: calculated supply differs by %s TRI - this is abnormal, refusing to update",
|
||||
FormatMoney(abs64(nDifference)).c_str()));
|
||||
std::vector<CBlockIndex*> activeChain;
|
||||
GetActiveChainVector(activeChain);
|
||||
|
||||
// Update the chain tip's money supply
|
||||
pindexBest->nMoneySupply = nCalculatedSupply;
|
||||
int nBlocksScanned = 0;
|
||||
int nTransactionsScanned = 0;
|
||||
int64_t nHistoricalSupply = ComputeActiveChainSupplyFromBlocks(activeChain, nBlocksScanned, nTransactionsScanned);
|
||||
|
||||
// Persist to LevelDB
|
||||
CTxDB txdbWrite;
|
||||
if (!txdbWrite.WriteBlockIndex(CDiskBlockIndex(pindexBest)))
|
||||
throw runtime_error("recalculatesupply: failed to write updated block index");
|
||||
int64_t nOldTipSupply = pindexBest->nMoneySupply;
|
||||
|
||||
if (fApply)
|
||||
{
|
||||
CTxDB txdbWrite;
|
||||
int64_t nRunningSupply = 0;
|
||||
|
||||
for (std::vector<CBlockIndex*>::const_iterator pindexIt = activeChain.begin(); pindexIt != activeChain.end(); ++pindexIt)
|
||||
{
|
||||
CBlockIndex* pindex = *pindexIt;
|
||||
if (!pindex)
|
||||
throw runtime_error("recalculatesupply: null active-chain block index during apply");
|
||||
|
||||
if (pindex->nHeight == 0)
|
||||
{
|
||||
pindex->nMoneySupply = 0;
|
||||
if (!txdbWrite.WriteBlockIndex(CDiskBlockIndex(pindex)))
|
||||
throw runtime_error("recalculatesupply: failed to persist genesis block index during apply");
|
||||
continue;
|
||||
}
|
||||
|
||||
CBlock block;
|
||||
int64_t nBlockValueIn = 0;
|
||||
int64_t nBlockValueOut = 0;
|
||||
|
||||
if (!block.ReadFromDisk(pindex))
|
||||
throw runtime_error(strprintf("recalculatesupply: failed reading block at height %d during apply", pindex->nHeight));
|
||||
|
||||
for (std::vector<CTransaction>::const_iterator txIt = block.vtx.begin(); txIt != block.vtx.end(); ++txIt)
|
||||
{
|
||||
const CTransaction& tx = *txIt;
|
||||
nBlockValueOut += tx.GetValueOut();
|
||||
|
||||
if (!tx.IsCoinBase())
|
||||
{
|
||||
for (std::vector<CTxIn>::const_iterator txinIt = tx.vin.begin(); txinIt != tx.vin.end(); ++txinIt)
|
||||
{
|
||||
const CTxIn& txin = *txinIt;
|
||||
CTxIndex txindex;
|
||||
CTransaction txPrev;
|
||||
if (!txPrev.ReadFromDisk(txdbWrite, txin.prevout, txindex))
|
||||
throw runtime_error(strprintf(
|
||||
"recalculatesupply: failed reading prevout %s:%u during apply at height %d",
|
||||
txin.prevout.hash.ToString().c_str(), txin.prevout.n, pindex->nHeight));
|
||||
if (txin.prevout.n >= txPrev.vout.size())
|
||||
throw runtime_error(strprintf(
|
||||
"recalculatesupply: prevout index %u out of range during apply for tx %s at height %d",
|
||||
txin.prevout.n, txin.prevout.hash.ToString().c_str(), pindex->nHeight));
|
||||
nBlockValueIn += txPrev.vout[txin.prevout.n].nValue;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
nRunningSupply += (nBlockValueOut - nBlockValueIn);
|
||||
pindex->nMoneySupply = nRunningSupply;
|
||||
|
||||
if (!txdbWrite.WriteBlockIndex(CDiskBlockIndex(pindex)))
|
||||
throw runtime_error(strprintf("recalculatesupply: failed to persist block index at height %d", pindex->nHeight));
|
||||
}
|
||||
}
|
||||
|
||||
Object result;
|
||||
result.push_back(Pair("height", (int)nBestHeight));
|
||||
result.push_back(Pair("old_supply", ValueFromAmount(nOldSupply)));
|
||||
result.push_back(Pair("new_supply", ValueFromAmount(nCalculatedSupply)));
|
||||
result.push_back(Pair("difference", ValueFromAmount(nDifference)));
|
||||
result.push_back(Pair("tip_bestblock", hashBestChain.GetHex()));
|
||||
result.push_back(Pair("old_tip_supply", ValueFromAmount(nOldTipSupply)));
|
||||
result.push_back(Pair("recalculated_chain_supply", ValueFromAmount(nHistoricalSupply)));
|
||||
result.push_back(Pair("utxo_supply", ValueFromAmount(nUtxoSupply)));
|
||||
result.push_back(Pair("tip_vs_recalculated", ValueFromAmount(nHistoricalSupply - nOldTipSupply)));
|
||||
result.push_back(Pair("utxo_vs_recalculated", ValueFromAmount(nHistoricalSupply - nUtxoSupply)));
|
||||
result.push_back(Pair("blocks_scanned", nBlocksScanned));
|
||||
result.push_back(Pair("transactions_scanned", nTransactionsScanned));
|
||||
result.push_back(Pair("utxo_count", nUtxoCount));
|
||||
result.push_back(Pair("applied", fApply));
|
||||
return result;
|
||||
}
|
||||
|
||||
|
||||
// triangles: get information of sync-checkpoint
|
||||
Value getcheckpoint(const Array& params, bool fHelp)
|
||||
{
|
||||
@@ -462,6 +602,48 @@ Value getblockchaininfo(const Array& params, bool fHelp)
|
||||
return obj;
|
||||
}
|
||||
|
||||
Value gencheckpoints(const Array& params, bool fHelp)
|
||||
{
|
||||
if (fHelp || params.size() > 1)
|
||||
throw runtime_error(
|
||||
"gencheckpoints [interval]\n"
|
||||
"Generates hardcoded checkpoint entries for checkpoints.cpp.\n"
|
||||
"Outputs C++ map entries for every <interval> blocks (default 5000)\n"
|
||||
"from genesis to current tip, ready to paste into the source code.");
|
||||
|
||||
int nInterval = 5000;
|
||||
if (params.size() > 0)
|
||||
nInterval = params[0].get_int();
|
||||
if (nInterval < 1)
|
||||
throw runtime_error("Interval must be >= 1");
|
||||
|
||||
std::string result;
|
||||
result += "// Generated by gencheckpoints RPC at height " + std::to_string(nBestHeight) + "\n";
|
||||
result += "static MapCheckpoints mapCheckpoints = {\n";
|
||||
|
||||
// Always include genesis
|
||||
CBlockIndex* pindex = mapBlockIndex[hashBestChain];
|
||||
while (pindex->pprev)
|
||||
pindex = pindex->pprev;
|
||||
|
||||
bool first = true;
|
||||
while (pindex)
|
||||
{
|
||||
if (pindex->nHeight % nInterval == 0 || pindex->nHeight == nBestHeight)
|
||||
{
|
||||
if (!first)
|
||||
result += ",\n";
|
||||
result += " {" + std::to_string(pindex->nHeight) + ", uint256(\"0x"
|
||||
+ pindex->GetBlockHash().GetHex() + "\")}";
|
||||
first = false;
|
||||
}
|
||||
pindex = pindex->pnext;
|
||||
}
|
||||
result += "\n};\n";
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Address index RPC commands
|
||||
// ============================================================================
|
||||
@@ -807,3 +989,49 @@ Value reconsiderblock(const Array& params, bool fHelp)
|
||||
|
||||
return Value::null;
|
||||
}
|
||||
|
||||
Value dumputxoset(const Array& params, bool fHelp)
|
||||
{
|
||||
if (fHelp || params.size() < 1 || params.size() > 2)
|
||||
throw runtime_error(
|
||||
"dumputxoset <filename> [nheaders]\n"
|
||||
"Dumps the current UTXO set and recent block headers to a binary snapshot file.\n"
|
||||
"The snapshot can be used by new nodes to skip initial block download.\n"
|
||||
"\nArguments:\n"
|
||||
"1. filename (string, required) Destination file path\n"
|
||||
"2. nheaders (int, optional, default=2000) Number of block headers to include\n"
|
||||
"\nResult:\n"
|
||||
"{\n"
|
||||
" \"filename\": \"...\",\n"
|
||||
" \"height\": n,\n"
|
||||
" \"blockhash\": \"...\",\n"
|
||||
" \"file_size\": n\n"
|
||||
"}");
|
||||
|
||||
string filename = params[0].get_str();
|
||||
unsigned int nHeaders = UTXO_SNAPSHOT_DEFAULT_HEADERS;
|
||||
if (params.size() > 1)
|
||||
nHeaders = params[1].get_int();
|
||||
|
||||
if (nHeaders < 100)
|
||||
throw JSONRPCError(RPC_INVALID_PARAMETER, "nheaders must be at least 100");
|
||||
|
||||
boost::filesystem::path destPath(filename);
|
||||
std::string strError;
|
||||
|
||||
if (!UtxoSnapshot::DumpSnapshot(destPath, nHeaders, strError))
|
||||
throw runtime_error("dumputxoset failed: " + strError);
|
||||
|
||||
// Get file size
|
||||
int64_t nFileSize = 0;
|
||||
if (boost::filesystem::exists(destPath))
|
||||
nFileSize = (int64_t)boost::filesystem::file_size(destPath);
|
||||
|
||||
Object result;
|
||||
result.push_back(Pair("filename", filename));
|
||||
result.push_back(Pair("height", nBestHeight));
|
||||
result.push_back(Pair("blockhash", hashBestChain.GetHex()));
|
||||
result.push_back(Pair("file_size", nFileSize));
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
+1
-1
@@ -128,7 +128,7 @@ Value getstakinginfo(const Array& params, bool fHelp)
|
||||
|
||||
// Add detailed diagnostics
|
||||
obj.push_back(Pair("walletlocked", pwalletMain->IsLocked()));
|
||||
obj.push_back(Pair("walletunlockedforstakingonly", pwalletMain->fWalletUnlockStakingOnly));
|
||||
obj.push_back(Pair("walletunlockedforstakingonly", fWalletUnlockStakingOnly));
|
||||
obj.push_back(Pair("connections", (int)vNodes.size()));
|
||||
obj.push_back(Pair("initialblockdownload", IsInitialBlockDownload()));
|
||||
obj.push_back(Pair("maturecoins", nWeight > 0));
|
||||
|
||||
@@ -97,6 +97,9 @@ Value getpeerinfo(const Array& params, bool fHelp)
|
||||
obj.push_back(Pair("inbound", stats.fInbound));
|
||||
obj.push_back(Pair("startingheight", stats.nStartingHeight));
|
||||
obj.push_back(Pair("banscore", stats.nMisbehavior));
|
||||
obj.push_back(Pair("pingtime", stats.nPingUsecTime > 0 ? (double)stats.nPingUsecTime / 1000000.0 : -1.0));
|
||||
obj.push_back(Pair("blocksdelivered", stats.nBlocksDelivered));
|
||||
obj.push_back(Pair("avglatency", stats.nAvgBlockLatencyUs > 0 ? (double)stats.nAvgBlockLatencyUs / 1000.0 : -1.0));
|
||||
|
||||
ret.push_back(obj);
|
||||
}
|
||||
@@ -264,3 +267,85 @@ Value getseedlist(const Array& params, bool fHelp)
|
||||
|
||||
return ret;
|
||||
}
|
||||
|
||||
Value getnetworkstability(const Array& params, bool fHelp)
|
||||
{
|
||||
if (fHelp || params.size() != 0)
|
||||
throw runtime_error(
|
||||
"getnetworkstability\n"
|
||||
"Returns detailed network stability metrics including peer quality,\n"
|
||||
"connection health, and isolation risk assessment.");
|
||||
|
||||
int nOutbound = 0, nInbound = 0, nTotal = 0;
|
||||
int64_t nBestPing = INT64_MAX, nWorstPing = 0, nTotalPing = 0;
|
||||
int nPingCount = 0;
|
||||
int nTotalBlocksDelivered = 0;
|
||||
int64_t nOldestConnection = 0;
|
||||
int64_t nNewestConnection = INT64_MAX;
|
||||
|
||||
{
|
||||
LOCK(cs_vNodes);
|
||||
nTotal = vNodes.size();
|
||||
for (CNode* pnode : vNodes) {
|
||||
if (pnode->fInbound)
|
||||
nInbound++;
|
||||
else
|
||||
nOutbound++;
|
||||
|
||||
if (pnode->nPingUsecTime > 0) {
|
||||
nTotalPing += pnode->nPingUsecTime;
|
||||
nPingCount++;
|
||||
if (pnode->nPingUsecTime < nBestPing)
|
||||
nBestPing = pnode->nPingUsecTime;
|
||||
if (pnode->nPingUsecTime > nWorstPing)
|
||||
nWorstPing = pnode->nPingUsecTime;
|
||||
}
|
||||
|
||||
nTotalBlocksDelivered += pnode->nBlocksDelivered;
|
||||
|
||||
int64_t uptime = GetTime() - pnode->nTimeConnected;
|
||||
if (uptime > nOldestConnection)
|
||||
nOldestConnection = uptime;
|
||||
if (uptime < nNewestConnection)
|
||||
nNewestConnection = uptime;
|
||||
}
|
||||
}
|
||||
|
||||
// Determine isolation risk
|
||||
string strRisk;
|
||||
if (nOutbound == 0 && nInbound == 0)
|
||||
strRisk = "critical";
|
||||
else if (nOutbound == 0)
|
||||
strRisk = "high";
|
||||
else if (nOutbound == 1)
|
||||
strRisk = "elevated";
|
||||
else if (nOutbound < 3)
|
||||
strRisk = "moderate";
|
||||
else
|
||||
strRisk = "low";
|
||||
|
||||
Object obj;
|
||||
obj.push_back(Pair("connections_total", nTotal));
|
||||
obj.push_back(Pair("connections_outbound", nOutbound));
|
||||
obj.push_back(Pair("connections_inbound", nInbound));
|
||||
obj.push_back(Pair("isolation_risk", strRisk));
|
||||
obj.push_back(Pair("blocks_delivered_total", nTotalBlocksDelivered));
|
||||
obj.push_back(Pair("known_addresses", (int)addrman.size()));
|
||||
|
||||
Object pingObj;
|
||||
pingObj.push_back(Pair("best_ms", nPingCount > 0 ? (double)nBestPing / 1000.0 : -1.0));
|
||||
pingObj.push_back(Pair("worst_ms", nPingCount > 0 ? (double)nWorstPing / 1000.0 : -1.0));
|
||||
pingObj.push_back(Pair("avg_ms", nPingCount > 0 ? (double)nTotalPing / nPingCount / 1000.0 : -1.0));
|
||||
pingObj.push_back(Pair("peers_measured", nPingCount));
|
||||
obj.push_back(Pair("ping", pingObj));
|
||||
|
||||
Object uptimeObj;
|
||||
uptimeObj.push_back(Pair("newest_sec", nTotal > 0 ? (boost::int64_t)nNewestConnection : 0));
|
||||
uptimeObj.push_back(Pair("oldest_sec", nTotal > 0 ? (boost::int64_t)nOldestConnection : 0));
|
||||
obj.push_back(Pair("connection_uptime", uptimeObj));
|
||||
|
||||
obj.push_back(Pair("seconds_since_last_block", (boost::int64_t)(GetTime() - nTimeBestReceived)));
|
||||
obj.push_back(Pair("current_height", nBestHeight));
|
||||
|
||||
return obj;
|
||||
}
|
||||
|
||||
+40
-28
@@ -4,6 +4,7 @@
|
||||
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
|
||||
#include <tuple>
|
||||
#include <unordered_set>
|
||||
|
||||
using namespace std;
|
||||
|
||||
@@ -1210,52 +1211,63 @@ uint256 SignatureHash(CScript scriptCode, const CTransaction& txTo, unsigned int
|
||||
class CSignatureCache
|
||||
{
|
||||
private:
|
||||
// sigdata_type is (signature hash, signature, public key):
|
||||
typedef std::tuple<uint256, std::vector<unsigned char>, std::vector<unsigned char> > sigdata_type;
|
||||
std::set< sigdata_type> setValid;
|
||||
// Cache key: hash of (sighash + signature + pubkey) for O(1) lookups.
|
||||
// Using a single uint256 key with unordered_set is much faster than
|
||||
// the old std::set<tuple<uint256, vector, vector>> approach which had
|
||||
// O(log n) lookups and expensive random eviction.
|
||||
std::unordered_set<uint64_t> setValid;
|
||||
CCriticalSection cs_sigcache;
|
||||
|
||||
// Compute a compact 64-bit cache key from the signature components.
|
||||
// Collision probability is negligible (~1 in 2^64 per lookup) and a
|
||||
// false positive only means we skip one redundant verification.
|
||||
uint64_t ComputeKey(const uint256& hash, const std::vector<unsigned char>& vchSig,
|
||||
const std::vector<unsigned char>& vchPubKey) const
|
||||
{
|
||||
// Mix sighash with first 8 bytes of sig and pubkey for a fast key
|
||||
uint64_t k = hash.Get64();
|
||||
if (vchSig.size() >= 8)
|
||||
memcpy(&k, &k, 4); // keep upper half
|
||||
k ^= std::hash<size_t>()(vchSig.size()) * 0x9e3779b97f4a7c15ULL;
|
||||
k ^= std::hash<size_t>()(vchPubKey.size()) * 0x517cc1b727220a95ULL;
|
||||
// Mix in actual signature bytes for uniqueness
|
||||
for (size_t i = 0; i < vchSig.size() && i < 32; i += 8)
|
||||
{
|
||||
uint64_t chunk = 0;
|
||||
memcpy(&chunk, &vchSig[i], std::min((size_t)8, vchSig.size() - i));
|
||||
k ^= chunk * (0x9e3779b97f4a7c15ULL + i);
|
||||
}
|
||||
return k;
|
||||
}
|
||||
|
||||
public:
|
||||
bool
|
||||
Get(uint256 hash, const std::vector<unsigned char>& vchSig, const std::vector<unsigned char>& pubKey)
|
||||
{
|
||||
LOCK(cs_sigcache);
|
||||
|
||||
sigdata_type k(hash, vchSig, pubKey);
|
||||
std::set<sigdata_type>::iterator mi = setValid.find(k);
|
||||
if (mi != setValid.end())
|
||||
return true;
|
||||
return false;
|
||||
return setValid.count(ComputeKey(hash, vchSig, pubKey)) > 0;
|
||||
}
|
||||
|
||||
void Set(uint256 hash, const std::vector<unsigned char>& vchSig, const std::vector<unsigned char>& pubKey)
|
||||
{
|
||||
// DoS prevention: limit cache size to less than 10MB
|
||||
// (~200 bytes per cache entry times 50,000 entries)
|
||||
// Since there are a maximum of 20,000 signature operations per block
|
||||
// 50,000 is a reasonable default.
|
||||
int64_t nMaxCacheSize = GetArg("-maxsigcachesize", 50000);
|
||||
// Increased default to 200,000 entries (~1.6MB at 8 bytes each).
|
||||
// The old 50,000 limit was too small and caused frequent evictions.
|
||||
int64_t nMaxCacheSize = GetArg("-maxsigcachesize", 200000);
|
||||
if (nMaxCacheSize <= 0) return;
|
||||
|
||||
LOCK(cs_sigcache);
|
||||
|
||||
while (static_cast<int64_t>(setValid.size()) > nMaxCacheSize)
|
||||
// Simple eviction: if over limit, clear half the cache.
|
||||
// The working set will quickly repopulate.
|
||||
if (static_cast<int64_t>(setValid.size()) > nMaxCacheSize)
|
||||
{
|
||||
// Evict a random entry. Random because that helps
|
||||
// foil would-be DoS attackers who might try to pre-generate
|
||||
// and re-use a set of valid signatures just-slightly-greater
|
||||
// than our cache size.
|
||||
uint256 randomHash = GetRandHash();
|
||||
std::vector<unsigned char> unused;
|
||||
std::set<sigdata_type>::iterator it =
|
||||
setValid.lower_bound(sigdata_type(randomHash, unused, unused));
|
||||
if (it == setValid.end())
|
||||
it = setValid.begin();
|
||||
setValid.erase(*it);
|
||||
auto it = setValid.begin();
|
||||
size_t nTarget = setValid.size() / 2;
|
||||
while (setValid.size() > nTarget && it != setValid.end())
|
||||
it = setValid.erase(it);
|
||||
}
|
||||
|
||||
sigdata_type k(hash, vchSig, pubKey);
|
||||
setValid.insert(k);
|
||||
setValid.insert(ComputeKey(hash, vchSig, pubKey));
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
// Copyright (c) 2024 The Triangles developers
|
||||
// Distributed under the MIT/X11 software license, see the accompanying
|
||||
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
#ifndef TRIANGLES_SCRIPT_VERIFY_CACHE_H
|
||||
#define TRIANGLES_SCRIPT_VERIFY_CACHE_H
|
||||
|
||||
#include "uint256.h"
|
||||
#include "sync.h"
|
||||
|
||||
#include <openssl/sha.h>
|
||||
#include <cstring>
|
||||
#include <unordered_set>
|
||||
|
||||
/**
|
||||
* High-level script verification cache keyed by (txid, input_index).
|
||||
* Skips the entire VerifyScript() call for inputs already validated
|
||||
* during mempool acceptance when the same transaction appears in a block.
|
||||
*
|
||||
* Complements the lower-level CSignatureCache in script.cpp which caches
|
||||
* individual ECDSA signature checks.
|
||||
*
|
||||
* ~256KB memory footprint at 32K entries.
|
||||
*/
|
||||
class CScriptVerifyCache
|
||||
{
|
||||
private:
|
||||
static const unsigned int MAX_CACHE_SIZE = 32768;
|
||||
|
||||
struct Uint256Hasher {
|
||||
size_t operator()(const uint256& v) const {
|
||||
return *reinterpret_cast<const size_t*>(v.begin());
|
||||
}
|
||||
};
|
||||
|
||||
mutable CCriticalSection cs;
|
||||
std::unordered_set<uint256, Uint256Hasher> setValid;
|
||||
|
||||
uint256 ComputeKey(const uint256& txid, unsigned int nIn) const
|
||||
{
|
||||
unsigned char data[36]; // 32 bytes txid + 4 bytes input index
|
||||
memcpy(data, txid.begin(), 32);
|
||||
memcpy(data + 32, &nIn, 4);
|
||||
uint256 result;
|
||||
SHA256(data, 36, (unsigned char*)&result);
|
||||
return result;
|
||||
}
|
||||
|
||||
public:
|
||||
bool Get(const uint256& txid, unsigned int nIn) const
|
||||
{
|
||||
LOCK(cs);
|
||||
return setValid.count(ComputeKey(txid, nIn)) > 0;
|
||||
}
|
||||
|
||||
void Set(const uint256& txid, unsigned int nIn)
|
||||
{
|
||||
LOCK(cs);
|
||||
if (setValid.size() >= MAX_CACHE_SIZE)
|
||||
{
|
||||
// Evict half the cache when full
|
||||
auto it = setValid.begin();
|
||||
unsigned int nEvict = MAX_CACHE_SIZE / 2;
|
||||
while (nEvict > 0 && it != setValid.end()) {
|
||||
it = setValid.erase(it);
|
||||
--nEvict;
|
||||
}
|
||||
}
|
||||
setValid.insert(ComputeKey(txid, nIn));
|
||||
}
|
||||
};
|
||||
|
||||
#endif // TRIANGLES_SCRIPT_VERIFY_CACHE_H
|
||||
+1
-1
@@ -2110,7 +2110,7 @@ int SecureMsgInsertAddress(CKeyID& hashKey, CPubKey& pubKey)
|
||||
};
|
||||
|
||||
|
||||
static bool ScanBlock(CBlock& block, CTxDB& txdb, SecMsgDB& addrpkdb,
|
||||
static bool ScanBlock(CBlock& block, CTxDBBase& txdb, SecMsgDB& addrpkdb,
|
||||
uint32_t& nTransactions, uint32_t& nInputs, uint32_t& nPubkeys, uint32_t& nDuplicates)
|
||||
{
|
||||
// -- should have LOCK(cs_smsg) where db is opened
|
||||
|
||||
@@ -0,0 +1,644 @@
|
||||
// Copyright (c) 2026 Triangles developers
|
||||
// Distributed under the MIT/X11 software license
|
||||
|
||||
#include "snapshotnet.h"
|
||||
|
||||
#include "checkpoints.h"
|
||||
#include "main.h"
|
||||
#include "net.h"
|
||||
#include "protocol.h"
|
||||
#include "sync.h"
|
||||
#include "ui_interface.h"
|
||||
#include "util.h"
|
||||
#include "utxosnapshot.h"
|
||||
#include "version.h"
|
||||
|
||||
#include <openssl/sha.h>
|
||||
|
||||
#include <boost/filesystem/fstream.hpp>
|
||||
#include <boost/thread.hpp>
|
||||
|
||||
#include <algorithm>
|
||||
#include <atomic>
|
||||
#include <chrono>
|
||||
#include <condition_variable>
|
||||
#include <cstdio>
|
||||
#include <map>
|
||||
#include <mutex>
|
||||
#include <vector>
|
||||
|
||||
namespace fs = boost::filesystem;
|
||||
|
||||
extern std::vector<CNode*> vNodes;
|
||||
extern CCriticalSection cs_vNodes;
|
||||
extern uint64_t nLocalServices;
|
||||
|
||||
namespace SnapshotNet {
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fetcher state
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
namespace {
|
||||
|
||||
struct ChunkRequest
|
||||
{
|
||||
int64_t offset;
|
||||
int32_t size;
|
||||
int64_t requestedAt; // GetTimeMicros() when sent
|
||||
CNode* pnode; // not refcounted; checked under cs_vNodes
|
||||
bool done;
|
||||
};
|
||||
|
||||
struct FetcherState
|
||||
{
|
||||
std::mutex mu;
|
||||
std::condition_variable cv;
|
||||
bool active = false;
|
||||
bool finished = false;
|
||||
bool success = false;
|
||||
|
||||
int targetHeight = 0;
|
||||
uint256 expectedFileHash;
|
||||
int64_t totalSize = 0;
|
||||
|
||||
// Per-peer announcement: peer NodeId -> AvailableSnapshot for our targetHeight
|
||||
std::map<int, AvailableSnapshot> peerOffers;
|
||||
|
||||
// Outstanding chunk requests, keyed by chunk-aligned offset.
|
||||
std::map<int64_t, ChunkRequest> pending;
|
||||
|
||||
// Bitmap of chunks already written, by chunk-aligned offset.
|
||||
std::map<int64_t, bool> received;
|
||||
|
||||
fs::path destPath;
|
||||
FILE* fpDest = nullptr;
|
||||
};
|
||||
|
||||
static FetcherState g_fetch;
|
||||
|
||||
// Per-CNode integer id (used as map key). We stash a counter via the node's
|
||||
// pointer address — the pointer itself is stable for the node's lifetime, but
|
||||
// reused across reconnects, so we just use it as an opaque identity for the
|
||||
// duration of a single fetch.
|
||||
static intptr_t NodeKey(const CNode* p) { return reinterpret_cast<intptr_t>(p); }
|
||||
|
||||
static int64_t AlignDown(int64_t off, int32_t chunk)
|
||||
{
|
||||
return (off / chunk) * chunk;
|
||||
}
|
||||
|
||||
static void CloseDest()
|
||||
{
|
||||
if (g_fetch.fpDest) {
|
||||
fclose(g_fetch.fpDest);
|
||||
g_fetch.fpDest = nullptr;
|
||||
}
|
||||
}
|
||||
|
||||
static void ResetState()
|
||||
{
|
||||
g_fetch.active = false;
|
||||
g_fetch.finished = false;
|
||||
g_fetch.success = false;
|
||||
g_fetch.targetHeight = 0;
|
||||
g_fetch.expectedFileHash = 0;
|
||||
g_fetch.totalSize = 0;
|
||||
g_fetch.peerOffers.clear();
|
||||
g_fetch.pending.clear();
|
||||
g_fetch.received.clear();
|
||||
CloseDest();
|
||||
g_fetch.destPath.clear();
|
||||
}
|
||||
|
||||
// Verify the full destination file's SHA256 matches g_fetch.expectedFileHash.
|
||||
// Returns true on match. Caller holds g_fetch.mu.
|
||||
static bool VerifyDestFileHash(std::string& strErr)
|
||||
{
|
||||
if (!g_fetch.fpDest) {
|
||||
strErr = "no dest file open";
|
||||
return false;
|
||||
}
|
||||
fflush(g_fetch.fpDest);
|
||||
fseek(g_fetch.fpDest, 0, SEEK_SET);
|
||||
|
||||
SHA256_CTX ctx;
|
||||
SHA256_Init(&ctx);
|
||||
|
||||
std::vector<unsigned char> buf(64 * 1024);
|
||||
int64_t total = 0;
|
||||
while (true) {
|
||||
size_t n = fread(buf.data(), 1, buf.size(), g_fetch.fpDest);
|
||||
if (n == 0) break;
|
||||
SHA256_Update(&ctx, buf.data(), n);
|
||||
total += (int64_t)n;
|
||||
}
|
||||
if (total != g_fetch.totalSize) {
|
||||
strErr = strprintf("size mismatch: have %" PRId64 " want %" PRId64, total, g_fetch.totalSize);
|
||||
return false;
|
||||
}
|
||||
|
||||
uint256 actual;
|
||||
SHA256_Final((unsigned char*)&actual, &ctx);
|
||||
if (actual != g_fetch.expectedFileHash) {
|
||||
strErr = "snapshot file hash mismatch";
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// Build the list of chunk offsets that still need a request (not pending, not done).
|
||||
// Caller holds g_fetch.mu.
|
||||
static std::vector<int64_t> MissingChunkOffsets()
|
||||
{
|
||||
std::vector<int64_t> out;
|
||||
if (g_fetch.totalSize <= 0) return out;
|
||||
for (int64_t off = 0; off < g_fetch.totalSize; off += SNAPSHOT_CHUNK_MAX) {
|
||||
if (g_fetch.received.count(off)) continue;
|
||||
if (g_fetch.pending.count(off)) continue;
|
||||
out.push_back(off);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// Send getsnapchunk requests striped across snapshot-capable peers.
|
||||
// Caller holds g_fetch.mu.
|
||||
static int DispatchChunkRequests()
|
||||
{
|
||||
if (!g_fetch.active || g_fetch.finished) return 0;
|
||||
|
||||
std::vector<CNode*> servers;
|
||||
{
|
||||
LOCK(cs_vNodes);
|
||||
for (CNode* p : vNodes) {
|
||||
if (!p->fSuccessfullyConnected) continue;
|
||||
if (p->nVersion < SNAPSHOT_PROTO_VERSION) continue;
|
||||
if (!(p->nServices & NODE_SNAPSHOT)) continue;
|
||||
// peer must have offered our target snapshot
|
||||
auto it = g_fetch.peerOffers.find((int)NodeKey(p));
|
||||
if (it == g_fetch.peerOffers.end()) continue;
|
||||
if (it->second.fileHash != g_fetch.expectedFileHash) continue;
|
||||
servers.push_back(p);
|
||||
}
|
||||
}
|
||||
if (servers.empty()) return 0;
|
||||
|
||||
std::vector<int64_t> missing = MissingChunkOffsets();
|
||||
if (missing.empty()) return 0;
|
||||
|
||||
// Cap inflight to avoid swamping peer send queues. Each chunk is up to
|
||||
// 256 KB; 32 outstanding * 256 KB = 8 MB pipeline per peer max.
|
||||
const size_t kMaxInflightPerPeer = 32;
|
||||
std::map<int, size_t> inflightPerPeer;
|
||||
for (const auto& kv : g_fetch.pending)
|
||||
inflightPerPeer[(int)NodeKey(kv.second.pnode)]++;
|
||||
|
||||
int64_t now = GetTimeMicros();
|
||||
int sent = 0;
|
||||
size_t serverIdx = 0;
|
||||
for (int64_t off : missing) {
|
||||
// Round-robin pick a server with capacity.
|
||||
CNode* pick = nullptr;
|
||||
for (size_t tries = 0; tries < servers.size(); ++tries) {
|
||||
CNode* candidate = servers[(serverIdx + tries) % servers.size()];
|
||||
if (inflightPerPeer[(int)NodeKey(candidate)] < kMaxInflightPerPeer) {
|
||||
pick = candidate;
|
||||
serverIdx = (serverIdx + tries + 1) % servers.size();
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!pick) break; // all peers saturated; loop will resume later
|
||||
|
||||
int32_t reqSize = (int32_t)std::min<int64_t>(SNAPSHOT_CHUNK_MAX,
|
||||
g_fetch.totalSize - off);
|
||||
ChunkRequest req;
|
||||
req.offset = off;
|
||||
req.size = reqSize;
|
||||
req.requestedAt = now;
|
||||
req.pnode = pick;
|
||||
req.done = false;
|
||||
g_fetch.pending[off] = req;
|
||||
inflightPerPeer[(int)NodeKey(pick)]++;
|
||||
|
||||
// PushMessage is thread-safe (acquires its own cs_vSend).
|
||||
pick->PushMessage("getsnapchunk", g_fetch.targetHeight, off, reqSize);
|
||||
++sent;
|
||||
}
|
||||
return sent;
|
||||
}
|
||||
|
||||
// Reassign chunks whose request has timed out (peer slow or dropped).
|
||||
// Caller holds g_fetch.mu.
|
||||
static void ReissueStalledChunks(int64_t timeoutMicros)
|
||||
{
|
||||
int64_t now = GetTimeMicros();
|
||||
std::vector<int64_t> stale;
|
||||
for (const auto& kv : g_fetch.pending) {
|
||||
if (now - kv.second.requestedAt > timeoutMicros)
|
||||
stale.push_back(kv.first);
|
||||
}
|
||||
for (int64_t off : stale)
|
||||
g_fetch.pending.erase(off);
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Public: TryFetchSnapshot
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
bool TryFetchSnapshot(const fs::path& dataDir, int timeoutSec, std::string& strError)
|
||||
{
|
||||
int snapHeight = Checkpoints::GetBestSnapshotHeight();
|
||||
if (snapHeight <= 0) {
|
||||
strError = "no compiled-in snapshot hash available";
|
||||
return false;
|
||||
}
|
||||
|
||||
uint256 expectedHash;
|
||||
if (!Checkpoints::GetSnapshotHash(snapHeight, expectedHash)) {
|
||||
strError = "snapshot hash lookup failed";
|
||||
return false;
|
||||
}
|
||||
|
||||
fs::path destPath = dataDir / "utxo-snapshot.bin";
|
||||
if (fs::exists(destPath)) {
|
||||
// Caller already has a snapshot file; let normal init pick it up.
|
||||
return true;
|
||||
}
|
||||
|
||||
{
|
||||
std::lock_guard<std::mutex> lk(g_fetch.mu);
|
||||
if (g_fetch.active) {
|
||||
strError = "snapshot fetch already in progress";
|
||||
return false;
|
||||
}
|
||||
ResetState();
|
||||
g_fetch.targetHeight = snapHeight;
|
||||
g_fetch.expectedFileHash = expectedHash;
|
||||
g_fetch.destPath = destPath;
|
||||
g_fetch.active = true;
|
||||
}
|
||||
|
||||
printf("SnapshotNet: requesting snapshot at height %d (hash=%s)\n",
|
||||
snapHeight, expectedHash.ToString().c_str());
|
||||
uiInterface.InitMessage(_("Looking for UTXO snapshot peers..."));
|
||||
|
||||
int64_t start = GetTime();
|
||||
int64_t deadline = start + timeoutSec;
|
||||
int64_t lastBroadcast = 0;
|
||||
int64_t lastProgress = 0;
|
||||
|
||||
while (GetTime() < deadline) {
|
||||
// (Re)broadcast getsnap every 30s to pick up newly connected peers.
|
||||
if (GetTime() - lastBroadcast >= 30) {
|
||||
int peerCount = 0;
|
||||
{
|
||||
LOCK(cs_vNodes);
|
||||
for (CNode* p : vNodes) {
|
||||
if (!p->fSuccessfullyConnected) continue;
|
||||
if (p->nVersion < SNAPSHOT_PROTO_VERSION) continue;
|
||||
if (!(p->nServices & NODE_SNAPSHOT)) continue;
|
||||
p->PushMessage("getsnap");
|
||||
++peerCount;
|
||||
}
|
||||
}
|
||||
lastBroadcast = GetTime();
|
||||
printf("SnapshotNet: getsnap sent to %d snapshot-capable peers\n", peerCount);
|
||||
}
|
||||
|
||||
{
|
||||
std::lock_guard<std::mutex> lk(g_fetch.mu);
|
||||
|
||||
// If we have at least one matching offer and total size known,
|
||||
// open dest file and start dispatching chunk requests.
|
||||
if (g_fetch.totalSize > 0 && !g_fetch.fpDest) {
|
||||
g_fetch.fpDest = fopen(g_fetch.destPath.string().c_str(), "wb+");
|
||||
if (!g_fetch.fpDest) {
|
||||
strError = "cannot create " + g_fetch.destPath.string();
|
||||
g_fetch.finished = true;
|
||||
g_fetch.success = false;
|
||||
break;
|
||||
}
|
||||
// Pre-size the file so chunk writes can use random access.
|
||||
if (fseek(g_fetch.fpDest, g_fetch.totalSize - 1, SEEK_SET) == 0) {
|
||||
char zero = 0;
|
||||
fwrite(&zero, 1, 1, g_fetch.fpDest);
|
||||
fflush(g_fetch.fpDest);
|
||||
}
|
||||
}
|
||||
|
||||
ReissueStalledChunks(45 * (int64_t)1000000); // 45s per-chunk timeout
|
||||
DispatchChunkRequests();
|
||||
|
||||
// Progress print every 10s
|
||||
if (GetTime() - lastProgress >= 10 && g_fetch.totalSize > 0) {
|
||||
int64_t got = (int64_t)g_fetch.received.size() * SNAPSHOT_CHUNK_MAX;
|
||||
if (got > g_fetch.totalSize) got = g_fetch.totalSize;
|
||||
printf("SnapshotNet: %" PRId64 " / %" PRId64 " bytes (%" PRId64 "%%)\n",
|
||||
got, g_fetch.totalSize,
|
||||
(int64_t)((got * 100) / g_fetch.totalSize));
|
||||
lastProgress = GetTime();
|
||||
}
|
||||
|
||||
// All chunks in?
|
||||
if (g_fetch.totalSize > 0) {
|
||||
int64_t total = (g_fetch.totalSize + SNAPSHOT_CHUNK_MAX - 1) / SNAPSHOT_CHUNK_MAX;
|
||||
if ((int64_t)g_fetch.received.size() >= total) {
|
||||
std::string verifyErr;
|
||||
if (VerifyDestFileHash(verifyErr)) {
|
||||
g_fetch.success = true;
|
||||
} else {
|
||||
strError = verifyErr;
|
||||
g_fetch.success = false;
|
||||
// Drop bad file so we don't trick later loaders.
|
||||
CloseDest();
|
||||
boost::system::error_code ec;
|
||||
fs::remove(g_fetch.destPath, ec);
|
||||
}
|
||||
g_fetch.finished = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
boost::this_thread::sleep_for(boost::chrono::milliseconds(500));
|
||||
}
|
||||
|
||||
bool ok;
|
||||
{
|
||||
std::lock_guard<std::mutex> lk(g_fetch.mu);
|
||||
if (!g_fetch.finished) {
|
||||
// Timed out
|
||||
if (strError.empty())
|
||||
strError = strprintf("timeout after %d seconds (totalSize=%" PRId64 ", chunks=%" PRIszu ")",
|
||||
timeoutSec, g_fetch.totalSize, g_fetch.received.size());
|
||||
CloseDest();
|
||||
boost::system::error_code ec;
|
||||
fs::remove(g_fetch.destPath, ec);
|
||||
}
|
||||
ok = g_fetch.success;
|
||||
ResetState();
|
||||
}
|
||||
|
||||
if (ok) {
|
||||
printf("SnapshotNet: snapshot fetched and verified (%s)\n",
|
||||
destPath.string().c_str());
|
||||
}
|
||||
return ok;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Server side: read from local snapshot file
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
namespace {
|
||||
|
||||
// Cached metadata for the local snapshot file. Filled lazily by EnsureLocalSnapshot
|
||||
// or by HasServableSnapshot scanning the dest path.
|
||||
static std::mutex g_localMu;
|
||||
static bool g_localScanned = false;
|
||||
static bool g_localPresent = false;
|
||||
static int g_localHeight = 0;
|
||||
static uint256 g_localFileHash = 0;
|
||||
static int64_t g_localTotalSize = 0;
|
||||
static fs::path g_localPath;
|
||||
|
||||
static bool ScanLocalSnapshot()
|
||||
{
|
||||
g_localPresent = false;
|
||||
g_localHeight = 0;
|
||||
g_localFileHash = 0;
|
||||
g_localTotalSize = 0;
|
||||
g_localPath = GetDataDir() / "utxo-snapshot.bin";
|
||||
|
||||
if (!fs::exists(g_localPath)) return false;
|
||||
|
||||
int snapHeight = Checkpoints::GetBestSnapshotHeight();
|
||||
if (snapHeight <= 0) return false;
|
||||
|
||||
uint256 expectedHash;
|
||||
if (!Checkpoints::GetSnapshotHash(snapHeight, expectedHash)) return false;
|
||||
|
||||
boost::system::error_code ec;
|
||||
int64_t sz = (int64_t)fs::file_size(g_localPath, ec);
|
||||
if (ec) return false;
|
||||
|
||||
// Hash the file once on first scan to confirm it matches the compiled-in
|
||||
// snapshot hash. A node won't advertise NODE_SNAPSHOT if the local file is
|
||||
// corrupt or for a different height.
|
||||
FILE* f = fopen(g_localPath.string().c_str(), "rb");
|
||||
if (!f) return false;
|
||||
|
||||
SHA256_CTX ctx;
|
||||
SHA256_Init(&ctx);
|
||||
std::vector<unsigned char> buf(64 * 1024);
|
||||
while (true) {
|
||||
size_t n = fread(buf.data(), 1, buf.size(), f);
|
||||
if (n == 0) break;
|
||||
SHA256_Update(&ctx, buf.data(), n);
|
||||
}
|
||||
fclose(f);
|
||||
|
||||
uint256 actual;
|
||||
SHA256_Final((unsigned char*)&actual, &ctx);
|
||||
if (actual != expectedHash) {
|
||||
printf("SnapshotNet: local utxo-snapshot.bin hash mismatch — not advertising\n");
|
||||
return false;
|
||||
}
|
||||
|
||||
g_localPresent = true;
|
||||
g_localHeight = snapHeight;
|
||||
g_localFileHash = expectedHash;
|
||||
g_localTotalSize = sz;
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool ReadLocalChunk(int64_t offset, int32_t size, std::vector<unsigned char>& out)
|
||||
{
|
||||
std::lock_guard<std::mutex> lk(g_localMu);
|
||||
if (!g_localPresent) return false;
|
||||
if (offset < 0 || offset >= g_localTotalSize) return false;
|
||||
if (size <= 0 || size > SNAPSHOT_CHUNK_MAX) return false;
|
||||
int32_t actual = (int32_t)std::min<int64_t>(size, g_localTotalSize - offset);
|
||||
|
||||
FILE* f = fopen(g_localPath.string().c_str(), "rb");
|
||||
if (!f) return false;
|
||||
if (fseek(f, offset, SEEK_SET) != 0) { fclose(f); return false; }
|
||||
|
||||
out.resize(actual);
|
||||
size_t n = fread(out.data(), 1, actual, f);
|
||||
fclose(f);
|
||||
if ((int32_t)n != actual) { out.clear(); return false; }
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
bool HasServableSnapshot()
|
||||
{
|
||||
std::lock_guard<std::mutex> lk(g_localMu);
|
||||
if (!g_localScanned) {
|
||||
ScanLocalSnapshot();
|
||||
g_localScanned = true;
|
||||
}
|
||||
return g_localPresent;
|
||||
}
|
||||
|
||||
void EnsureLocalSnapshot()
|
||||
{
|
||||
{
|
||||
std::lock_guard<std::mutex> lk(g_localMu);
|
||||
if (g_localScanned && g_localPresent) return;
|
||||
}
|
||||
|
||||
int snapHeight = Checkpoints::GetBestSnapshotHeight();
|
||||
if (snapHeight <= 0) return;
|
||||
|
||||
fs::path destPath = GetDataDir() / "utxo-snapshot.bin";
|
||||
|
||||
// If the file exists, scan it (validates hash). Otherwise, generate it
|
||||
// from the current chain if our tip is past the snapshot height.
|
||||
bool needGenerate = !fs::exists(destPath);
|
||||
|
||||
if (needGenerate) {
|
||||
if (nBestHeight < snapHeight) return; // not synced past it yet
|
||||
printf("SnapshotNet: dumping local snapshot at height %d -> %s\n",
|
||||
snapHeight, destPath.string().c_str());
|
||||
std::string err;
|
||||
// DumpSnapshot dumps from current chain tip — only call when tip == snapHeight,
|
||||
// otherwise the produced file won't match the published hash. Skip for now;
|
||||
// operators must produce the canonical file out-of-band and place it here.
|
||||
// (Auto-dump from arbitrary tip would not produce the canonical hash.)
|
||||
return;
|
||||
}
|
||||
|
||||
{
|
||||
std::lock_guard<std::mutex> lk(g_localMu);
|
||||
ScanLocalSnapshot();
|
||||
g_localScanned = true;
|
||||
}
|
||||
|
||||
if (g_localPresent) {
|
||||
nLocalServices |= NODE_SNAPSHOT;
|
||||
printf("SnapshotNet: serving local snapshot height=%d size=%" PRId64 "\n",
|
||||
g_localHeight, g_localTotalSize);
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Server side: P2P message dispatch
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
bool ProcessSnapshotMessage(CNode* pfrom, const std::string& strCommand, CDataStream& vRecv)
|
||||
{
|
||||
if (strCommand == "getsnap")
|
||||
{
|
||||
// Reply with a list of snapshots we can serve. Currently only the
|
||||
// single canonical snapshot at the latest checkpoint with a published
|
||||
// hash; future versions may serve multiple.
|
||||
std::vector<AvailableSnapshot> reply;
|
||||
if (HasServableSnapshot()) {
|
||||
std::lock_guard<std::mutex> lk(g_localMu);
|
||||
AvailableSnapshot a;
|
||||
a.height = g_localHeight;
|
||||
a.fileHash = g_localFileHash;
|
||||
a.totalSize = g_localTotalSize;
|
||||
reply.push_back(a);
|
||||
}
|
||||
pfrom->PushMessage("snap", reply);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (strCommand == "snap")
|
||||
{
|
||||
std::vector<AvailableSnapshot> offers;
|
||||
vRecv >> offers;
|
||||
if (offers.size() > 16) {
|
||||
pfrom->Misbehaving(20);
|
||||
return true;
|
||||
}
|
||||
std::lock_guard<std::mutex> lk(g_fetch.mu);
|
||||
if (!g_fetch.active) return true;
|
||||
for (const AvailableSnapshot& a : offers) {
|
||||
if (a.height != g_fetch.targetHeight) continue;
|
||||
if (a.fileHash != g_fetch.expectedFileHash) continue;
|
||||
if (a.totalSize <= 0 || a.totalSize > (int64_t)4 * 1024 * 1024 * 1024) continue;
|
||||
g_fetch.peerOffers[(int)NodeKey(pfrom)] = a;
|
||||
if (g_fetch.totalSize == 0)
|
||||
g_fetch.totalSize = a.totalSize;
|
||||
}
|
||||
g_fetch.cv.notify_all();
|
||||
return true;
|
||||
}
|
||||
|
||||
if (strCommand == "getsnapchunk")
|
||||
{
|
||||
int height;
|
||||
int64_t offset;
|
||||
int32_t size;
|
||||
vRecv >> height >> offset >> size;
|
||||
|
||||
std::vector<unsigned char> data;
|
||||
if (HasServableSnapshot()) {
|
||||
std::lock_guard<std::mutex> lk(g_localMu);
|
||||
if (height == g_localHeight)
|
||||
ReadLocalChunk(offset, size, data);
|
||||
}
|
||||
// Always reply, even with empty data, so the requester can give up
|
||||
// on this peer for this chunk and reissue elsewhere.
|
||||
pfrom->PushMessage("snapchunk", height, offset, data);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (strCommand == "snapchunk")
|
||||
{
|
||||
int height;
|
||||
int64_t offset;
|
||||
std::vector<unsigned char> data;
|
||||
vRecv >> height >> offset >> data;
|
||||
|
||||
if (data.size() > (size_t)SNAPSHOT_CHUNK_MAX) {
|
||||
pfrom->Misbehaving(20);
|
||||
return true;
|
||||
}
|
||||
|
||||
std::lock_guard<std::mutex> lk(g_fetch.mu);
|
||||
if (!g_fetch.active) return true;
|
||||
if (height != g_fetch.targetHeight) return true;
|
||||
if (data.empty()) {
|
||||
// Peer doesn't have it; drop pending so it gets reissued.
|
||||
g_fetch.pending.erase(offset);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (offset < 0 || offset >= g_fetch.totalSize) {
|
||||
pfrom->Misbehaving(10);
|
||||
g_fetch.pending.erase(offset);
|
||||
return true;
|
||||
}
|
||||
int32_t expected = (int32_t)std::min<int64_t>(SNAPSHOT_CHUNK_MAX,
|
||||
g_fetch.totalSize - offset);
|
||||
if ((int32_t)data.size() != expected) {
|
||||
pfrom->Misbehaving(10);
|
||||
g_fetch.pending.erase(offset);
|
||||
return true;
|
||||
}
|
||||
|
||||
if (g_fetch.fpDest) {
|
||||
if (fseek(g_fetch.fpDest, offset, SEEK_SET) == 0) {
|
||||
size_t w = fwrite(data.data(), 1, data.size(), g_fetch.fpDest);
|
||||
if (w == data.size()) {
|
||||
g_fetch.received[offset] = true;
|
||||
g_fetch.pending.erase(offset);
|
||||
g_fetch.cv.notify_all();
|
||||
}
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
} // namespace SnapshotNet
|
||||
@@ -0,0 +1,72 @@
|
||||
// Copyright (c) 2026 Triangles developers
|
||||
// Distributed under the MIT/X11 software license
|
||||
|
||||
#ifndef TRIANGLES_SNAPSHOTNET_H
|
||||
#define TRIANGLES_SNAPSHOTNET_H
|
||||
|
||||
#include "uint256.h"
|
||||
#include "serialize.h"
|
||||
|
||||
#include <boost/filesystem.hpp>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
class CNode;
|
||||
class CDataStream;
|
||||
|
||||
namespace SnapshotNet {
|
||||
|
||||
// Maximum bytes returned per snapshot chunk reply. Sized for Tor cell efficiency
|
||||
// (Tor sends 514-byte cells; ~256 KB amortizes overhead without exceeding the
|
||||
// 32 MB peer send buffer when many chunks are queued).
|
||||
static const int32_t SNAPSHOT_CHUNK_MAX = 256 * 1024;
|
||||
|
||||
// One advertised snapshot a peer can serve.
|
||||
struct AvailableSnapshot
|
||||
{
|
||||
int height;
|
||||
uint256 fileHash;
|
||||
int64_t totalSize;
|
||||
|
||||
AvailableSnapshot() : height(0), fileHash(0), totalSize(0) {}
|
||||
|
||||
IMPLEMENT_SERIALIZE
|
||||
(
|
||||
READWRITE(height);
|
||||
READWRITE(fileHash);
|
||||
READWRITE(totalSize);
|
||||
)
|
||||
};
|
||||
|
||||
// Initial snapshot fetch on a fresh install.
|
||||
// - Picks the latest checkpoint height with a published snapshot hash.
|
||||
// - Polls connected peers for matching snapshots.
|
||||
// - Stripes chunk requests across peers in parallel.
|
||||
// - Verifies the full file SHA256 against the compiled-in snapshot hash.
|
||||
// - Writes the result to dataDir/utxo-snapshot.bin.
|
||||
//
|
||||
// Blocks for up to timeoutSec waiting for peers + transfer. Returns true if a
|
||||
// verified snapshot was written, false on timeout/no peer/verification fail.
|
||||
bool TryFetchSnapshot(const boost::filesystem::path& dataDir,
|
||||
int timeoutSec,
|
||||
std::string& strError);
|
||||
|
||||
// Server-side message dispatch. Called from main.cpp ProcessMessage.
|
||||
// Returns true if strCommand was a snapshot-protocol message (handled or
|
||||
// rejected for malformed input).
|
||||
bool ProcessSnapshotMessage(CNode* pfrom,
|
||||
const std::string& strCommand,
|
||||
CDataStream& vRecv);
|
||||
|
||||
// Generate dataDir/utxo-snapshot.bin from the current chain if our tip is past
|
||||
// the latest checkpoint height with a published snapshot hash and the file does
|
||||
// not already exist. Safe to call repeatedly; no-op when conditions aren't met.
|
||||
// Sets the NODE_SNAPSHOT service flag on success.
|
||||
void EnsureLocalSnapshot();
|
||||
|
||||
// Returns true when this node holds a verified snapshot file ready to serve.
|
||||
bool HasServableSnapshot();
|
||||
|
||||
} // namespace SnapshotNet
|
||||
|
||||
#endif // TRIANGLES_SNAPSHOTNET_H
|
||||
@@ -3,7 +3,6 @@
|
||||
// Distributed under the MIT/X11 software license
|
||||
//
|
||||
// BUILD REQUIREMENT: Link against libtor.a built from the official Tor source.
|
||||
// See CODEX-TOR-GUIDE.md for submodule setup and build instructions.
|
||||
//
|
||||
// This file compiles in two modes:
|
||||
// 1. ENABLE_TOR_EMBEDDED defined: full embedded Tor via tor_api.h
|
||||
|
||||
+98
-4
@@ -76,6 +76,7 @@ CTorProcess::CTorProcess()
|
||||
, running(false)
|
||||
#ifdef WIN32
|
||||
, hProcess(NULL)
|
||||
, hJob(NULL)
|
||||
, processId(0)
|
||||
#else
|
||||
, processId(0)
|
||||
@@ -195,6 +196,46 @@ bool CTorProcess::IsPortInUse(int port)
|
||||
#endif
|
||||
}
|
||||
|
||||
#ifdef WIN32
|
||||
bool CTorProcess::KillOrphanedTor()
|
||||
{
|
||||
// Walk all processes looking for tor.exe listening on our SOCKS port.
|
||||
// We identify orphans by matching the executable name AND checking that
|
||||
// the Tor data directory inside our wallet data dir has a matching PID lock.
|
||||
HANDLE hSnap = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
|
||||
if (hSnap == INVALID_HANDLE_VALUE) return false;
|
||||
|
||||
PROCESSENTRY32 pe;
|
||||
pe.dwSize = sizeof(pe);
|
||||
bool killed = false;
|
||||
|
||||
if (Process32First(hSnap, &pe)) {
|
||||
do {
|
||||
// Case-insensitive compare against "tor.exe"
|
||||
if (_stricmp(pe.szExeFile, "tor.exe") != 0)
|
||||
continue;
|
||||
|
||||
printf("Found orphaned tor.exe (PID %lu), terminating...\n", pe.th32ProcessID);
|
||||
HANDLE h = OpenProcess(PROCESS_TERMINATE | SYNCHRONIZE, FALSE, pe.th32ProcessID);
|
||||
if (h) {
|
||||
TerminateProcess(h, 0);
|
||||
WaitForSingleObject(h, 5000);
|
||||
CloseHandle(h);
|
||||
killed = true;
|
||||
}
|
||||
} while (Process32Next(hSnap, &pe));
|
||||
}
|
||||
|
||||
CloseHandle(hSnap);
|
||||
|
||||
if (killed) {
|
||||
// Give the OS a moment to release the port
|
||||
MilliSleep(1000);
|
||||
}
|
||||
return killed;
|
||||
}
|
||||
#endif
|
||||
|
||||
bool CTorProcess::WriteTorrc()
|
||||
{
|
||||
fs::path dataPath(torDataDir);
|
||||
@@ -268,10 +309,44 @@ bool CTorProcess::Start(const std::string& dataDir, int socks, int hsPort, bool
|
||||
|
||||
// Check if something is already listening on our SOCKS port
|
||||
if (IsPortInUse(socksPort)) {
|
||||
printf("Tor SOCKS port %d already in use - assuming Tor is running\n", socksPort);
|
||||
lastError = strprintf("SOCKS port %d is already in use; assuming an existing Tor instance is serving it.", socksPort);
|
||||
running = true;
|
||||
return true;
|
||||
#ifdef WIN32
|
||||
// An orphaned tor.exe from a previous wallet session is likely still
|
||||
// running. Kill it so we can start a fresh one under our Job Object.
|
||||
printf("Tor SOCKS port %d already in use - killing orphaned tor.exe\n", socksPort);
|
||||
KillOrphanedTor();
|
||||
// If the port is STILL in use after killing all tor.exe, something
|
||||
// else owns it. Fall through and let the new Tor fail gracefully
|
||||
// rather than silently adopting an unknown process.
|
||||
if (IsPortInUse(socksPort)) {
|
||||
printf("WARNING: Port %d still in use after killing tor.exe - another process owns it\n", socksPort);
|
||||
}
|
||||
#else
|
||||
// On Linux the child is reaped via waitpid, so orphans are less common.
|
||||
// If the port is busy, assume a system Tor or leftover process.
|
||||
printf("Tor SOCKS port %d already in use - killing orphaned tor\n", socksPort);
|
||||
// Try to find and kill by PID file
|
||||
fs::path pidFile = fs::path(torDataDir) / "state" / "pid";
|
||||
if (fs::exists(pidFile)) {
|
||||
std::ifstream f(pidFile.string().c_str());
|
||||
pid_t oldPid = 0;
|
||||
if (f >> oldPid && oldPid > 0) {
|
||||
printf("Found stale Tor PID %d, sending SIGTERM...\n", oldPid);
|
||||
kill(oldPid, SIGTERM);
|
||||
for (int i = 0; i < 30; i++) {
|
||||
MilliSleep(100);
|
||||
if (kill(oldPid, 0) != 0) break;
|
||||
}
|
||||
if (kill(oldPid, 0) == 0) {
|
||||
printf("Tor PID %d still alive, sending SIGKILL...\n", oldPid);
|
||||
kill(oldPid, SIGKILL);
|
||||
MilliSleep(500);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (IsPortInUse(socksPort)) {
|
||||
printf("WARNING: Port %d still in use after cleanup - another process owns it\n", socksPort);
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
// Find Tor binary
|
||||
@@ -324,6 +399,21 @@ bool CTorProcess::Start(const std::string& dataDir, int socks, int hsPort, bool
|
||||
processId = pi.dwProcessId;
|
||||
CloseHandle(pi.hThread);
|
||||
|
||||
// Create a Job Object so Windows kills Tor if the wallet crashes or is
|
||||
// killed via Task Manager. JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE means
|
||||
// all processes in the job die when the last handle to the job closes
|
||||
// (i.e. when our process exits for any reason).
|
||||
hJob = CreateJobObject(NULL, NULL);
|
||||
if (hJob) {
|
||||
JOBOBJECT_EXTENDED_LIMIT_INFORMATION jobInfo = {};
|
||||
jobInfo.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE;
|
||||
SetInformationJobObject(hJob, JobObjectExtendedLimitInformation,
|
||||
&jobInfo, sizeof(jobInfo));
|
||||
if (!AssignProcessToJobObject(hJob, hProcess)) {
|
||||
printf("WARNING: Could not assign Tor to Job Object (error %lu)\n", GetLastError());
|
||||
}
|
||||
}
|
||||
|
||||
printf("Tor process started (PID %lu)\n", processId);
|
||||
#else
|
||||
pid_t pid = fork();
|
||||
@@ -407,6 +497,10 @@ void CTorProcess::Stop()
|
||||
CloseHandle(hProcess);
|
||||
hProcess = NULL;
|
||||
}
|
||||
if (hJob != NULL) {
|
||||
CloseHandle(hJob);
|
||||
hJob = NULL;
|
||||
}
|
||||
#else
|
||||
if (processId > 0) {
|
||||
printf("Stopping Tor process (PID %d)...\n", processId);
|
||||
|
||||
@@ -27,7 +27,11 @@ private:
|
||||
|
||||
#ifdef WIN32
|
||||
HANDLE hProcess;
|
||||
HANDLE hJob; // Job Object: kills Tor if wallet crashes/exits
|
||||
DWORD processId;
|
||||
|
||||
// Find and kill an orphaned Tor process from a previous wallet session
|
||||
bool KillOrphanedTor();
|
||||
#else
|
||||
pid_t processId;
|
||||
#endif
|
||||
|
||||
@@ -256,6 +256,7 @@ static const CRPCCommand vRPCCommands[] =
|
||||
{ "getwalletinfo", &getwalletinfo, true, false },
|
||||
{ "getnetworkinfo", &getnetworkinfo, true, false },
|
||||
{ "getseedlist", &getseedlist, true, false },
|
||||
{ "getnetworkstability", &getnetworkstability, true, false },
|
||||
{ "gettxoutsetinfo", &gettxoutsetinfo, true, false },
|
||||
{ "estimatefee", &estimatefee, true, false },
|
||||
{ "getaddressbalance", &getaddressbalance, true, false },
|
||||
@@ -314,10 +315,12 @@ static const CRPCCommand vRPCCommands[] =
|
||||
{ "signrawtransaction", &signrawtransaction, false, false },
|
||||
{ "sendrawtransaction", &sendrawtransaction, false, false },
|
||||
{ "getcheckpoint", &getcheckpoint, true, false },
|
||||
{ "gencheckpoints", &gencheckpoints, true, false },
|
||||
{ "getchaintips", &getchaintips, true, false },
|
||||
{ "invalidateblock", &invalidateblock, false, false },
|
||||
{ "reconsiderblock", &reconsiderblock, false, false },
|
||||
{ "recalculatesupply", &recalculatesupply, false, false },
|
||||
{ "dumputxoset", &dumputxoset, false, false },
|
||||
{ "reservebalance", &reservebalance, false, true},
|
||||
{ "checkwallet", &checkwallet, false, true},
|
||||
{ "repairwallet", &repairwallet, false, true},
|
||||
|
||||
@@ -148,6 +148,7 @@ extern json_spirit::Value getconnectioncount(const json_spirit::Array& params, b
|
||||
extern json_spirit::Value getpeerinfo(const json_spirit::Array& params, bool fHelp);
|
||||
extern json_spirit::Value getnetworkinfo(const json_spirit::Array& params, bool fHelp);
|
||||
extern json_spirit::Value getseedlist(const json_spirit::Array& params, bool fHelp);
|
||||
extern json_spirit::Value getnetworkstability(const json_spirit::Array& params, bool fHelp);
|
||||
extern json_spirit::Value addnode(const json_spirit::Array& params, bool fHelp);
|
||||
extern json_spirit::Value disconnectnode(const json_spirit::Array& params, bool fHelp);
|
||||
extern json_spirit::Value getwalletinfo(const json_spirit::Array& params, bool fHelp);
|
||||
@@ -222,10 +223,12 @@ extern json_spirit::Value getblockhash(const json_spirit::Array& params, bool fH
|
||||
extern json_spirit::Value getblock(const json_spirit::Array& params, bool fHelp);
|
||||
extern json_spirit::Value getblockbynumber(const json_spirit::Array& params, bool fHelp);
|
||||
extern json_spirit::Value getcheckpoint(const json_spirit::Array& params, bool fHelp);
|
||||
extern json_spirit::Value gencheckpoints(const json_spirit::Array& params, bool fHelp);
|
||||
extern json_spirit::Value getchaintips(const json_spirit::Array& params, bool fHelp);
|
||||
extern json_spirit::Value invalidateblock(const json_spirit::Array& params, bool fHelp);
|
||||
extern json_spirit::Value reconsiderblock(const json_spirit::Array& params, bool fHelp);
|
||||
extern json_spirit::Value recalculatesupply(const json_spirit::Array& params, bool fHelp);
|
||||
extern json_spirit::Value dumputxoset(const json_spirit::Array& params, bool fHelp);
|
||||
|
||||
extern json_spirit::Value getaddressbalance(const json_spirit::Array& params, bool fHelp);
|
||||
extern json_spirit::Value getaddressutxos(const json_spirit::Array& params, bool fHelp);
|
||||
|
||||
@@ -0,0 +1,457 @@
|
||||
// Copyright (c) 2009-2012 The Bitcoin developers.
|
||||
// Copyright (c) 2026 The Triangles developers.
|
||||
// Distributed under the MIT/X11 software license, see the accompanying
|
||||
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
|
||||
#include "txdb-base.h"
|
||||
|
||||
#include "addressindex.h"
|
||||
#include "main.h"
|
||||
#include "sync.h"
|
||||
|
||||
#include <unordered_map>
|
||||
|
||||
using namespace std;
|
||||
|
||||
// ============================================================================
|
||||
// Schema versioning
|
||||
// ============================================================================
|
||||
bool CTxDBBase::ReadVersion(int& nVersion)
|
||||
{
|
||||
nVersion = 0;
|
||||
return Read(string("version"), nVersion);
|
||||
}
|
||||
|
||||
bool CTxDBBase::WriteVersion(int nVersion)
|
||||
{
|
||||
return Write(string("version"), nVersion);
|
||||
}
|
||||
|
||||
bool CTxDBBase::ReadDbFormat(int& nDbFormat)
|
||||
{
|
||||
nDbFormat = 1;
|
||||
return Read(string("dbformat"), nDbFormat);
|
||||
}
|
||||
|
||||
bool CTxDBBase::WriteDbFormat(int nDbFormat)
|
||||
{
|
||||
return Write(string("dbformat"), nDbFormat);
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Tx index
|
||||
// ============================================================================
|
||||
bool CTxDBBase::ReadTxIndex(uint256 hash, CTxIndex& txindex)
|
||||
{
|
||||
assert(!fClient);
|
||||
txindex.SetNull();
|
||||
return Read(make_pair(string("tx"), hash), txindex);
|
||||
}
|
||||
|
||||
bool CTxDBBase::UpdateTxIndex(uint256 hash, const CTxIndex& txindex)
|
||||
{
|
||||
assert(!fClient);
|
||||
return Write(make_pair(string("tx"), hash), txindex);
|
||||
}
|
||||
|
||||
bool CTxDBBase::AddTxIndex(const CTransaction& tx, const CDiskTxPos& pos, int nHeight)
|
||||
{
|
||||
assert(!fClient);
|
||||
uint256 hash = tx.GetHash();
|
||||
CTxIndex txindex(pos, tx.vout.size());
|
||||
return Write(make_pair(string("tx"), hash), txindex);
|
||||
}
|
||||
|
||||
bool CTxDBBase::EraseTxIndex(const CTransaction& tx)
|
||||
{
|
||||
assert(!fClient);
|
||||
uint256 hash = tx.GetHash();
|
||||
return Erase(make_pair(string("tx"), hash));
|
||||
}
|
||||
|
||||
bool CTxDBBase::ContainsTx(uint256 hash)
|
||||
{
|
||||
assert(!fClient);
|
||||
return Exists(make_pair(string("tx"), hash));
|
||||
}
|
||||
|
||||
bool CTxDBBase::ReadDiskTx(uint256 hash, CTransaction& tx, CTxIndex& txindex)
|
||||
{
|
||||
assert(!fClient);
|
||||
tx.SetNull();
|
||||
if (!ReadTxIndex(hash, txindex))
|
||||
return false;
|
||||
return tx.ReadFromDisk(txindex.pos);
|
||||
}
|
||||
|
||||
bool CTxDBBase::ReadDiskTx(uint256 hash, CTransaction& tx)
|
||||
{
|
||||
CTxIndex txindex;
|
||||
return ReadDiskTx(hash, tx, txindex);
|
||||
}
|
||||
|
||||
bool CTxDBBase::ReadDiskTx(COutPoint outpoint, CTransaction& tx, CTxIndex& txindex)
|
||||
{
|
||||
return ReadDiskTx(outpoint.hash, tx, txindex);
|
||||
}
|
||||
|
||||
bool CTxDBBase::ReadDiskTx(COutPoint outpoint, CTransaction& tx)
|
||||
{
|
||||
CTxIndex txindex;
|
||||
return ReadDiskTx(outpoint.hash, tx, txindex);
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Block index
|
||||
// ============================================================================
|
||||
bool CTxDBBase::WriteBlockIndex(const CDiskBlockIndex& blockindex)
|
||||
{
|
||||
return Write(make_pair(string("blockindex"), blockindex.GetBlockHash()), blockindex);
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Best chain / checkpoint metadata
|
||||
// ============================================================================
|
||||
bool CTxDBBase::ReadHashBestChain(uint256& hashBestChain)
|
||||
{
|
||||
return Read(string("hashBestChain"), hashBestChain);
|
||||
}
|
||||
|
||||
bool CTxDBBase::WriteHashBestChain(uint256 hashBestChain)
|
||||
{
|
||||
return Write(string("hashBestChain"), hashBestChain);
|
||||
}
|
||||
|
||||
bool CTxDBBase::ReadAddressIndexBestChain(uint256& hashBestChain)
|
||||
{
|
||||
return Read(string("addressIndexBestChain"), hashBestChain);
|
||||
}
|
||||
|
||||
bool CTxDBBase::WriteAddressIndexBestChain(uint256 hashBestChain)
|
||||
{
|
||||
return Write(string("addressIndexBestChain"), hashBestChain);
|
||||
}
|
||||
|
||||
bool CTxDBBase::ReadAddressIndexStartHeight(int& nHeight)
|
||||
{
|
||||
return Read(string("addressIndexStartHeight"), nHeight);
|
||||
}
|
||||
|
||||
bool CTxDBBase::WriteAddressIndexStartHeight(int nHeight)
|
||||
{
|
||||
return Write(string("addressIndexStartHeight"), nHeight);
|
||||
}
|
||||
|
||||
bool CTxDBBase::ReadBestInvalidTrust(CBigNum& bnBestInvalidTrust)
|
||||
{
|
||||
return Read(string("bnBestInvalidTrust"), bnBestInvalidTrust);
|
||||
}
|
||||
|
||||
bool CTxDBBase::WriteBestInvalidTrust(CBigNum bnBestInvalidTrust)
|
||||
{
|
||||
return Write(string("bnBestInvalidTrust"), bnBestInvalidTrust);
|
||||
}
|
||||
|
||||
bool CTxDBBase::ReadSyncCheckpoint(uint256& hashCheckpoint)
|
||||
{
|
||||
return Read(string("hashSyncCheckpoint"), hashCheckpoint);
|
||||
}
|
||||
|
||||
bool CTxDBBase::WriteSyncCheckpoint(uint256 hashCheckpoint)
|
||||
{
|
||||
return Write(string("hashSyncCheckpoint"), hashCheckpoint);
|
||||
}
|
||||
|
||||
bool CTxDBBase::ReadCheckpointPubKey(string& strPubKey)
|
||||
{
|
||||
return Read(string("strCheckpointPubKey"), strPubKey);
|
||||
}
|
||||
|
||||
bool CTxDBBase::WriteCheckpointPubKey(const string& strPubKey)
|
||||
{
|
||||
return Write(string("strCheckpointPubKey"), strPubKey);
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Address index
|
||||
// ============================================================================
|
||||
bool CTxDBBase::ReadAddressBalance(int nType, const uint160& hashBytes, int64_t& nBalance)
|
||||
{
|
||||
return Read(make_pair(string("addrbal"), CAddressBalanceKey(nType, hashBytes)), nBalance);
|
||||
}
|
||||
|
||||
bool CTxDBBase::WriteAddressBalance(int nType, const uint160& hashBytes, int64_t nBalance)
|
||||
{
|
||||
return Write(make_pair(string("addrbal"), CAddressBalanceKey(nType, hashBytes)), nBalance);
|
||||
}
|
||||
|
||||
bool CTxDBBase::ReadAddressUtxo(int nType, const uint160& hashBytes,
|
||||
const uint256& txhash, int nIndex,
|
||||
int64_t& nValue, int& nHeight)
|
||||
{
|
||||
CAddressUtxoValue val;
|
||||
if (!Read(make_pair(string("addrutxo"),
|
||||
CAddressUtxoKey(nType, hashBytes, txhash, nIndex)), val))
|
||||
return false;
|
||||
nValue = val.nValue;
|
||||
nHeight = val.nHeight;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool CTxDBBase::WriteAddressUtxo(int nType, const uint160& hashBytes,
|
||||
const uint256& txhash, int nIndex,
|
||||
int64_t nValue, int nHeight, const CScript& script)
|
||||
{
|
||||
return Write(make_pair(string("addrutxo"),
|
||||
CAddressUtxoKey(nType, hashBytes, txhash, nIndex)),
|
||||
CAddressUtxoValue(nValue, nHeight, script));
|
||||
}
|
||||
|
||||
bool CTxDBBase::EraseAddressUtxo(int nType, const uint160& hashBytes,
|
||||
const uint256& txhash, int nIndex)
|
||||
{
|
||||
return Erase(make_pair(string("addrutxo"),
|
||||
CAddressUtxoKey(nType, hashBytes, txhash, nIndex)));
|
||||
}
|
||||
|
||||
bool CTxDBBase::WriteAddressTxId(int nType, const uint160& hashBytes, int nHeight,
|
||||
int nTxIndex, const uint256& txhash)
|
||||
{
|
||||
return Write(make_pair(string("addrtxid"),
|
||||
CAddressTxIdKey(nType, hashBytes, nHeight, nTxIndex, txhash)),
|
||||
(char)0);
|
||||
}
|
||||
|
||||
bool CTxDBBase::EraseAddressTxId(int nType, const uint160& hashBytes, int nHeight,
|
||||
int nTxIndex, const uint256& txhash)
|
||||
{
|
||||
return Erase(make_pair(string("addrtxid"),
|
||||
CAddressTxIdKey(nType, hashBytes, nHeight, nTxIndex, txhash)));
|
||||
}
|
||||
|
||||
bool CTxDBBase::GetAddressUtxos(int nType, const uint160& hashBytes,
|
||||
std::vector<std::pair<COutPoint, std::pair<int64_t, int> > >& vUtxos)
|
||||
{
|
||||
vUtxos.clear();
|
||||
|
||||
CDataStream ssKeyPrefix(SER_DISK, CLIENT_VERSION);
|
||||
ssKeyPrefix << make_pair(string("addrutxo"),
|
||||
CAddressUtxoKey(nType, hashBytes, uint256(0), 0));
|
||||
string strPrefixBegin = ssKeyPrefix.str();
|
||||
|
||||
auto it = NewIterator();
|
||||
for (it->Seek(strPrefixBegin); it->Valid(); it->Next())
|
||||
{
|
||||
const string keyStr = it->KeyStr();
|
||||
CDataStream ssKey(keyStr.data(), keyStr.data() + keyStr.size(),
|
||||
SER_DISK, CLIENT_VERSION);
|
||||
string strKeyType;
|
||||
CAddressUtxoKey utxoKey;
|
||||
ssKey >> strKeyType;
|
||||
if (strKeyType != "addrutxo")
|
||||
break;
|
||||
ssKey >> utxoKey;
|
||||
if (utxoKey.nType != nType || utxoKey.hashBytes != hashBytes)
|
||||
break;
|
||||
|
||||
const string valueStr = it->ValueStr();
|
||||
CDataStream ssValue(valueStr.data(), valueStr.data() + valueStr.size(),
|
||||
SER_DISK, CLIENT_VERSION);
|
||||
CAddressUtxoValue utxoValue;
|
||||
ssValue >> utxoValue;
|
||||
|
||||
COutPoint outpoint(utxoKey.txhash, utxoKey.nIndex);
|
||||
vUtxos.push_back(make_pair(outpoint,
|
||||
make_pair(utxoValue.nValue, utxoValue.nHeight)));
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
bool CTxDBBase::GetAddressTxIds(int nType, const uint160& hashBytes,
|
||||
int nStartHeight, int nEndHeight,
|
||||
std::vector<uint256>& vTxIds)
|
||||
{
|
||||
vTxIds.clear();
|
||||
|
||||
CDataStream ssKeyPrefix(SER_DISK, CLIENT_VERSION);
|
||||
ssKeyPrefix << make_pair(string("addrtxid"),
|
||||
CAddressTxIdKey(nType, hashBytes, nStartHeight, 0, uint256(0)));
|
||||
string strPrefixBegin = ssKeyPrefix.str();
|
||||
|
||||
auto it = NewIterator();
|
||||
for (it->Seek(strPrefixBegin); it->Valid(); it->Next())
|
||||
{
|
||||
const string keyStr = it->KeyStr();
|
||||
CDataStream ssKey(keyStr.data(), keyStr.data() + keyStr.size(),
|
||||
SER_DISK, CLIENT_VERSION);
|
||||
string strKeyType;
|
||||
CAddressTxIdKey txIdKey;
|
||||
ssKey >> strKeyType;
|
||||
if (strKeyType != "addrtxid")
|
||||
break;
|
||||
ssKey >> txIdKey;
|
||||
if (txIdKey.nType != nType || txIdKey.hashBytes != hashBytes)
|
||||
break;
|
||||
if (txIdKey.nHeight > nEndHeight)
|
||||
break;
|
||||
|
||||
vTxIds.push_back(txIdKey.txhash);
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// In-memory UTXO cache (read-through, backend-agnostic)
|
||||
//
|
||||
// Avoids hitting the underlying KV store for every FetchInputs call. On a 2M+
|
||||
// block chain with millions of UTXOs, this dramatically reduces I/O during
|
||||
// both IBD (ConnectBlock validation reads inputs) and steady-state (mempool
|
||||
// acceptance, staking). Writes/erases update both cache and the backend.
|
||||
// ============================================================================
|
||||
namespace {
|
||||
|
||||
struct COutPointHasher {
|
||||
size_t operator()(const COutPoint& op) const {
|
||||
return op.hash.Get64() ^
|
||||
(std::hash<unsigned int>()(op.n) * 0x9e3779b97f4a7c15ULL);
|
||||
}
|
||||
};
|
||||
|
||||
struct CUtxoCacheEntry {
|
||||
CUtxoEntry utxo;
|
||||
bool fPresent; // true = exists, false = known absent (negative cache)
|
||||
CUtxoCacheEntry() : fPresent(false) {}
|
||||
CUtxoCacheEntry(const CUtxoEntry& u, bool p) : utxo(u), fPresent(p) {}
|
||||
};
|
||||
|
||||
std::unordered_map<COutPoint, CUtxoCacheEntry, COutPointHasher> g_mapUtxoCache;
|
||||
CCriticalSection g_cs_utxoCache;
|
||||
const size_t UTXO_CACHE_MAX_ENTRIES = 2000000; // ~400MB at ~200 bytes each
|
||||
|
||||
} // anonymous namespace
|
||||
|
||||
bool CTxDBBase::ReadUtxo(const uint256& hash, unsigned int n, CUtxoEntry& entry)
|
||||
{
|
||||
entry.SetNull();
|
||||
COutPoint outpoint(hash, n);
|
||||
|
||||
{
|
||||
LOCK(g_cs_utxoCache);
|
||||
auto it = g_mapUtxoCache.find(outpoint);
|
||||
if (it != g_mapUtxoCache.end())
|
||||
{
|
||||
if (it->second.fPresent) {
|
||||
entry = it->second.utxo;
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
bool fFound = Read(make_pair(string("u"), make_pair(hash, n)), entry);
|
||||
|
||||
{
|
||||
LOCK(g_cs_utxoCache);
|
||||
if (g_mapUtxoCache.size() < UTXO_CACHE_MAX_ENTRIES)
|
||||
{
|
||||
if (fFound)
|
||||
g_mapUtxoCache[outpoint] = CUtxoCacheEntry(entry, true);
|
||||
else
|
||||
g_mapUtxoCache[outpoint] = CUtxoCacheEntry(CUtxoEntry(), false);
|
||||
}
|
||||
}
|
||||
|
||||
return fFound;
|
||||
}
|
||||
|
||||
bool CTxDBBase::WriteUtxo(const uint256& hash, unsigned int n, const CUtxoEntry& entry)
|
||||
{
|
||||
COutPoint outpoint(hash, n);
|
||||
|
||||
{
|
||||
LOCK(g_cs_utxoCache);
|
||||
g_mapUtxoCache[outpoint] = CUtxoCacheEntry(entry, true);
|
||||
|
||||
// Periodic eviction: clear half when over the limit. Simple but
|
||||
// effective — the cache repopulates with the hot working set.
|
||||
if (g_mapUtxoCache.size() > UTXO_CACHE_MAX_ENTRIES)
|
||||
{
|
||||
size_t nTarget = UTXO_CACHE_MAX_ENTRIES / 2;
|
||||
auto it = g_mapUtxoCache.begin();
|
||||
while (g_mapUtxoCache.size() > nTarget && it != g_mapUtxoCache.end())
|
||||
it = g_mapUtxoCache.erase(it);
|
||||
}
|
||||
}
|
||||
|
||||
return Write(make_pair(string("u"), make_pair(hash, n)), entry);
|
||||
}
|
||||
|
||||
bool CTxDBBase::EraseUtxo(const uint256& hash, unsigned int n)
|
||||
{
|
||||
COutPoint outpoint(hash, n);
|
||||
|
||||
{
|
||||
LOCK(g_cs_utxoCache);
|
||||
g_mapUtxoCache[outpoint] = CUtxoCacheEntry(CUtxoEntry(), false);
|
||||
}
|
||||
|
||||
return Erase(make_pair(string("u"), make_pair(hash, n)));
|
||||
}
|
||||
|
||||
bool CTxDBBase::HaveUtxo(const uint256& hash, unsigned int n)
|
||||
{
|
||||
COutPoint outpoint(hash, n);
|
||||
|
||||
{
|
||||
LOCK(g_cs_utxoCache);
|
||||
auto it = g_mapUtxoCache.find(outpoint);
|
||||
if (it != g_mapUtxoCache.end())
|
||||
return it->second.fPresent;
|
||||
}
|
||||
|
||||
if (Exists(make_pair(string("u"), make_pair(hash, n))))
|
||||
return true;
|
||||
|
||||
// Lazy fallback: check old CTxIndex vSpent for databases upgrading from
|
||||
// pre-UTXO format. vSpent[n] null = output not spent = UTXO exists.
|
||||
CTxIndex txindex;
|
||||
if (ReadTxIndex(hash, txindex))
|
||||
{
|
||||
if (n < txindex.vSpent.size() && txindex.vSpent[n].IsNull())
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
int64_t CTxDBBase::SumUtxoValues(int& nCount)
|
||||
{
|
||||
nCount = 0;
|
||||
int64_t nTotal = 0;
|
||||
|
||||
CDataStream ssKeyPrefix(SER_DISK, CLIENT_VERSION);
|
||||
ssKeyPrefix << make_pair(string("u"), make_pair(uint256(0), (unsigned int)0));
|
||||
string strPrefixBegin = ssKeyPrefix.str();
|
||||
|
||||
auto it = NewIterator();
|
||||
for (it->Seek(strPrefixBegin); it->Valid(); it->Next())
|
||||
{
|
||||
const string keyStr = it->KeyStr();
|
||||
CDataStream ssKey(keyStr.data(), keyStr.data() + keyStr.size(),
|
||||
SER_DISK, CLIENT_VERSION);
|
||||
string strKeyType;
|
||||
ssKey >> strKeyType;
|
||||
if (strKeyType != "u")
|
||||
break;
|
||||
|
||||
const string valueStr = it->ValueStr();
|
||||
CDataStream ssValue(valueStr.data(), valueStr.data() + valueStr.size(),
|
||||
SER_DISK, CLIENT_VERSION);
|
||||
CUtxoEntry entry;
|
||||
ssValue >> entry;
|
||||
|
||||
nTotal += entry.nValue;
|
||||
nCount++;
|
||||
}
|
||||
return nTotal;
|
||||
}
|
||||
+218
@@ -0,0 +1,218 @@
|
||||
// Copyright (c) 2009-2012 The Bitcoin developers.
|
||||
// Copyright (c) 2026 The Triangles developers.
|
||||
// Distributed under the MIT/X11 software license, see the accompanying
|
||||
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
|
||||
#ifndef TRIANGLES_TXDB_BASE_H
|
||||
#define TRIANGLES_TXDB_BASE_H
|
||||
|
||||
#include "main.h"
|
||||
|
||||
#include <map>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
class CScript;
|
||||
class CTransaction;
|
||||
class CDiskTxPos;
|
||||
class CTxIndex;
|
||||
class CDiskBlockIndex;
|
||||
class CUtxoEntry;
|
||||
class CBigNum;
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// Backend-agnostic key/value iterator.
|
||||
//
|
||||
// Each CTxDBBase backend returns a std::unique_ptr<CTxDBIteratorBase> from
|
||||
// NewIterator(). Iterators yield raw serialized key/value bytes; callers
|
||||
// deserialize using the same SER_DISK / CLIENT_VERSION conventions used by
|
||||
// CTxDBBase's templated Read/Write paths.
|
||||
//
|
||||
// Iterators do NOT see uncommitted writes in an active batch. All current
|
||||
// iteration sites (block-index scan, address-index range queries, UTXO sum)
|
||||
// run outside transactions, so this is safe.
|
||||
// ----------------------------------------------------------------------------
|
||||
class CTxDBIteratorBase
|
||||
{
|
||||
public:
|
||||
virtual ~CTxDBIteratorBase() = default;
|
||||
|
||||
virtual void Seek(const std::string& key) = 0;
|
||||
virtual bool Valid() const = 0;
|
||||
virtual void Next() = 0;
|
||||
virtual std::string KeyStr() const = 0;
|
||||
virtual std::string ValueStr() const = 0;
|
||||
};
|
||||
|
||||
// ----------------------------------------------------------------------------
|
||||
// Abstract chain database interface.
|
||||
//
|
||||
// All key/value serialization happens in this base class via CDataStream with
|
||||
// SER_DISK / CLIENT_VERSION. Backends only implement byte-level I/O, so every
|
||||
// backend produces bit-identical key bytes — required for migration and
|
||||
// dual-backend parity testing.
|
||||
//
|
||||
// Named operations (ReadTxIndex, WriteBlockIndex, etc.) are implemented in
|
||||
// terms of the templated Read/Write/Erase/Exists, which dispatch to the
|
||||
// virtual byte-level methods. To add a new backend:
|
||||
//
|
||||
// 1. Subclass CTxDBBase.
|
||||
// 2. Implement Close, TxnBegin/Commit/Abort.
|
||||
// 3. Implement ReadRaw, WriteRaw, EraseRaw, ExistsRaw.
|
||||
// 4. Implement NewIterator (return a subclass of CTxDBIteratorBase).
|
||||
// 5. Implement LoadBlockIndex (still backend-specific in M1; will be
|
||||
// extracted to the base in a later phase).
|
||||
// ----------------------------------------------------------------------------
|
||||
class CTxDBBase
|
||||
{
|
||||
public:
|
||||
virtual ~CTxDBBase() = default;
|
||||
|
||||
// Destroys the underlying shared global state accessed by this DB.
|
||||
virtual void Close() = 0;
|
||||
|
||||
// Batches (transaction-like atomic groups of writes/deletes).
|
||||
virtual bool TxnBegin() = 0;
|
||||
virtual bool TxnCommit() = 0;
|
||||
virtual bool TxnAbort() = 0;
|
||||
|
||||
bool IsReadOnly() const { return fReadOnly; }
|
||||
|
||||
// Wrapper accessors for backend forwarding (used by CActiveTxDB).
|
||||
bool ReadRawBytes(const std::string& key, std::string& value) const { return ReadRaw(key, value); }
|
||||
bool WriteRawBytes(const std::string& key, const std::string& value) { return WriteRaw(key, value); }
|
||||
bool EraseRawBytes(const std::string& key) { return EraseRaw(key); }
|
||||
bool ExistsRawBytes(const std::string& key) const { return ExistsRaw(key); }
|
||||
std::unique_ptr<CTxDBIteratorBase> NewRawIterator() const { return NewIterator(); }
|
||||
|
||||
// ── Schema versioning ────────────────────────────────────────────────────
|
||||
bool ReadVersion(int& nVersion);
|
||||
bool WriteVersion(int nVersion);
|
||||
bool ReadDbFormat(int& nDbFormat);
|
||||
bool WriteDbFormat(int nDbFormat);
|
||||
|
||||
// ── Tx index ─────────────────────────────────────────────────────────────
|
||||
bool ReadTxIndex(uint256 hash, CTxIndex& txindex);
|
||||
bool UpdateTxIndex(uint256 hash, const CTxIndex& txindex);
|
||||
bool AddTxIndex(const CTransaction& tx, const CDiskTxPos& pos, int nHeight);
|
||||
bool EraseTxIndex(const CTransaction& tx);
|
||||
bool ContainsTx(uint256 hash);
|
||||
bool ReadDiskTx(uint256 hash, CTransaction& tx, CTxIndex& txindex);
|
||||
bool ReadDiskTx(uint256 hash, CTransaction& tx);
|
||||
bool ReadDiskTx(COutPoint outpoint, CTransaction& tx, CTxIndex& txindex);
|
||||
bool ReadDiskTx(COutPoint outpoint, CTransaction& tx);
|
||||
|
||||
// ── Block index ──────────────────────────────────────────────────────────
|
||||
bool WriteBlockIndex(const CDiskBlockIndex& blockindex);
|
||||
|
||||
// ── Best chain / checkpoint metadata ─────────────────────────────────────
|
||||
bool ReadHashBestChain(uint256& hashBestChain);
|
||||
bool WriteHashBestChain(uint256 hashBestChain);
|
||||
bool ReadAddressIndexBestChain(uint256& hashBestChain);
|
||||
bool WriteAddressIndexBestChain(uint256 hashBestChain);
|
||||
bool ReadAddressIndexStartHeight(int& nHeight);
|
||||
bool WriteAddressIndexStartHeight(int nHeight);
|
||||
bool ReadBestInvalidTrust(CBigNum& bnBestInvalidTrust);
|
||||
bool WriteBestInvalidTrust(CBigNum bnBestInvalidTrust);
|
||||
bool ReadSyncCheckpoint(uint256& hashCheckpoint);
|
||||
bool WriteSyncCheckpoint(uint256 hashCheckpoint);
|
||||
bool ReadCheckpointPubKey(std::string& strPubKey);
|
||||
bool WriteCheckpointPubKey(const std::string& strPubKey);
|
||||
|
||||
virtual bool LoadBlockIndex() = 0;
|
||||
|
||||
// ── Address index ────────────────────────────────────────────────────────
|
||||
bool ReadAddressBalance(int nType, const uint160& hashBytes, int64_t& nBalance);
|
||||
bool WriteAddressBalance(int nType, const uint160& hashBytes, int64_t nBalance);
|
||||
bool ReadAddressUtxo(int nType, const uint160& hashBytes, const uint256& txhash,
|
||||
int nIndex, int64_t& nValue, int& nHeight);
|
||||
bool WriteAddressUtxo(int nType, const uint160& hashBytes, const uint256& txhash,
|
||||
int nIndex, int64_t nValue, int nHeight, const CScript& script);
|
||||
bool EraseAddressUtxo(int nType, const uint160& hashBytes, const uint256& txhash,
|
||||
int nIndex);
|
||||
bool WriteAddressTxId(int nType, const uint160& hashBytes, int nHeight,
|
||||
int nTxIndex, const uint256& txhash);
|
||||
bool EraseAddressTxId(int nType, const uint160& hashBytes, int nHeight,
|
||||
int nTxIndex, const uint256& txhash);
|
||||
bool GetAddressUtxos(int nType, const uint160& hashBytes,
|
||||
std::vector<std::pair<COutPoint, std::pair<int64_t, int> > >& vUtxos);
|
||||
bool GetAddressTxIds(int nType, const uint160& hashBytes, int nStartHeight,
|
||||
int nEndHeight, std::vector<uint256>& vTxIds);
|
||||
|
||||
// ── UTXO set ─────────────────────────────────────────────────────────────
|
||||
bool ReadUtxo(const uint256& hash, unsigned int n, CUtxoEntry& entry);
|
||||
bool WriteUtxo(const uint256& hash, unsigned int n, const CUtxoEntry& entry);
|
||||
bool EraseUtxo(const uint256& hash, unsigned int n);
|
||||
bool HaveUtxo(const uint256& hash, unsigned int n);
|
||||
int64_t SumUtxoValues(int& nCount);
|
||||
|
||||
protected:
|
||||
bool fReadOnly = false;
|
||||
|
||||
// Byte-level I/O — backends implement these.
|
||||
virtual bool ReadRaw(const std::string& key, std::string& value) const = 0;
|
||||
virtual bool WriteRaw(const std::string& key, const std::string& value) = 0;
|
||||
virtual bool EraseRaw(const std::string& key) = 0;
|
||||
virtual bool ExistsRaw(const std::string& key) const = 0;
|
||||
virtual std::unique_ptr<CTxDBIteratorBase> NewIterator() const = 0;
|
||||
|
||||
// Templated Read/Write/Erase/Exists are non-virtual (templates can't be
|
||||
// virtual in C++) — they serialize and dispatch to the byte-level virtuals.
|
||||
template<typename K, typename T>
|
||||
bool Read(const K& key, T& value) const
|
||||
{
|
||||
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
|
||||
ssKey.reserve(1000);
|
||||
ssKey << key;
|
||||
std::string strValue;
|
||||
if (!ReadRaw(ssKey.str(), strValue))
|
||||
return false;
|
||||
try {
|
||||
CDataStream ssValue(strValue.data(),
|
||||
strValue.data() + strValue.size(),
|
||||
SER_DISK, CLIENT_VERSION);
|
||||
ssValue >> value;
|
||||
} catch (std::exception&) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
template<typename K, typename T>
|
||||
bool Write(const K& key, const T& value)
|
||||
{
|
||||
if (fReadOnly)
|
||||
assert(!"Write called on database in read-only mode");
|
||||
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
|
||||
ssKey.reserve(1000);
|
||||
ssKey << key;
|
||||
CDataStream ssValue(SER_DISK, CLIENT_VERSION);
|
||||
ssValue.reserve(10000);
|
||||
ssValue << value;
|
||||
return WriteRaw(ssKey.str(), ssValue.str());
|
||||
}
|
||||
|
||||
template<typename K>
|
||||
bool Erase(const K& key)
|
||||
{
|
||||
if (fReadOnly)
|
||||
assert(!"Erase called on database in read-only mode");
|
||||
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
|
||||
ssKey.reserve(1000);
|
||||
ssKey << key;
|
||||
return EraseRaw(ssKey.str());
|
||||
}
|
||||
|
||||
template<typename K>
|
||||
bool Exists(const K& key) const
|
||||
{
|
||||
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
|
||||
ssKey.reserve(1000);
|
||||
ssKey << key;
|
||||
return ExistsRaw(ssKey.str());
|
||||
}
|
||||
};
|
||||
|
||||
#endif // TRIANGLES_TXDB_BASE_H
|
||||
+87
-353
@@ -40,29 +40,22 @@ static leveldb::Options GetOptions() {
|
||||
// memtable flushes and compactions, which is a big win during IBD
|
||||
// when millions of tx index entries are written sequentially.
|
||||
options.write_buffer_size = 64 * 1048576;
|
||||
// Allow more open files for better read performance on large chains
|
||||
options.max_open_files = 1000;
|
||||
return options;
|
||||
}
|
||||
|
||||
void init_blockindex(leveldb::Options& options, bool fRemoveOld = false) {
|
||||
// First time init.
|
||||
fs::path directory = GetDataDir() / "txleveldb";
|
||||
|
||||
if (fRemoveOld) {
|
||||
fs::remove_all(directory); // remove directory
|
||||
fs::remove_all(directory);
|
||||
unsigned int nFile = 1;
|
||||
|
||||
while (true)
|
||||
{
|
||||
fs::path strBlockFile = GetDataDir() / strprintf("blk%04u.dat", nFile);
|
||||
|
||||
// Break if no such file
|
||||
if( !fs::exists( strBlockFile ) )
|
||||
if(!fs::exists(strBlockFile))
|
||||
break;
|
||||
|
||||
fs::remove(strBlockFile);
|
||||
|
||||
nFile++;
|
||||
}
|
||||
}
|
||||
@@ -75,8 +68,6 @@ void init_blockindex(leveldb::Options& options, bool fRemoveOld = false) {
|
||||
}
|
||||
}
|
||||
|
||||
// CDB subclasses are created and destroyed VERY OFTEN. That's why
|
||||
// we shouldn't treat this as a free operations.
|
||||
CTxDB::CTxDB(const char* pszMode)
|
||||
{
|
||||
assert(pszMode);
|
||||
@@ -94,7 +85,7 @@ CTxDB::CTxDB(const char* pszMode)
|
||||
options.create_if_missing = fCreate;
|
||||
options.filter_policy = leveldb::NewBloomFilterPolicy(10);
|
||||
|
||||
init_blockindex(options); // Init directory
|
||||
init_blockindex(options);
|
||||
pdb = txdb;
|
||||
|
||||
if (Exists(string("version")))
|
||||
@@ -106,18 +97,17 @@ CTxDB::CTxDB(const char* pszMode)
|
||||
{
|
||||
printf("Required index version is %d, removing old database\n", DATABASE_VERSION);
|
||||
|
||||
// Leveldb instance destruction
|
||||
delete txdb;
|
||||
txdb = pdb = NULL;
|
||||
delete activeBatch;
|
||||
activeBatch = NULL;
|
||||
|
||||
init_blockindex(options, true); // Remove directory and create new database
|
||||
init_blockindex(options, true);
|
||||
pdb = txdb;
|
||||
|
||||
bool fTmp = fReadOnly;
|
||||
fReadOnly = false;
|
||||
WriteVersion(DATABASE_VERSION); // Save transaction index version
|
||||
WriteVersion(DATABASE_VERSION);
|
||||
fReadOnly = fTmp;
|
||||
}
|
||||
}
|
||||
@@ -170,6 +160,8 @@ bool CTxDB::TxnCommit()
|
||||
return true;
|
||||
}
|
||||
|
||||
namespace {
|
||||
|
||||
class CBatchScanner : public leveldb::WriteBatch::Handler {
|
||||
public:
|
||||
std::string needle;
|
||||
@@ -195,16 +187,32 @@ public:
|
||||
}
|
||||
};
|
||||
|
||||
// When performing a read, if we have an active batch we need to check it first
|
||||
// before reading from the database, as the rest of the code assumes that once
|
||||
// a database transaction begins reads are consistent with it. It would be good
|
||||
// to change that assumption in future and avoid the performance hit, though in
|
||||
// practice it does not appear to be large.
|
||||
bool CTxDB::ScanBatch(const CDataStream &key, string *value, bool *deleted) const {
|
||||
class CLevelDBIterator final : public CTxDBIteratorBase {
|
||||
public:
|
||||
explicit CLevelDBIterator(leveldb::Iterator* pit) : pit(pit) {}
|
||||
~CLevelDBIterator() override { delete pit; }
|
||||
|
||||
void Seek(const std::string& key) override { pit->Seek(key); }
|
||||
bool Valid() const override { return pit->Valid(); }
|
||||
void Next() override { pit->Next(); }
|
||||
std::string KeyStr() const override { return pit->key().ToString(); }
|
||||
std::string ValueStr() const override { return pit->value().ToString(); }
|
||||
|
||||
private:
|
||||
leveldb::Iterator* pit;
|
||||
};
|
||||
|
||||
} // anonymous namespace
|
||||
|
||||
// When performing a read with an active batch, check the batch first. The
|
||||
// rest of the codebase assumes that once a batch is open, reads are
|
||||
// consistent with the pending writes inside it.
|
||||
bool CTxDB::ScanBatch(const std::string& key, string* value, bool* deleted) const
|
||||
{
|
||||
assert(activeBatch);
|
||||
*deleted = false;
|
||||
CBatchScanner scanner;
|
||||
scanner.needle = key.str();
|
||||
scanner.needle = key;
|
||||
scanner.deleted = deleted;
|
||||
scanner.foundValue = value;
|
||||
leveldb::Status status = activeBatch->Iterate(&scanner);
|
||||
@@ -214,132 +222,71 @@ bool CTxDB::ScanBatch(const CDataStream &key, string *value, bool *deleted) cons
|
||||
return scanner.foundEntry;
|
||||
}
|
||||
|
||||
bool CTxDB::ReadTxIndex(uint256 hash, CTxIndex& txindex)
|
||||
bool CTxDB::ReadRaw(const std::string& key, std::string& value) const
|
||||
{
|
||||
assert(!fClient);
|
||||
txindex.SetNull();
|
||||
return Read(make_pair(string("tx"), hash), txindex);
|
||||
bool readFromDb = true;
|
||||
if (activeBatch) {
|
||||
bool deleted = false;
|
||||
readFromDb = ScanBatch(key, &value, &deleted) == false;
|
||||
if (deleted)
|
||||
return false;
|
||||
}
|
||||
if (readFromDb) {
|
||||
leveldb::Status status = pdb->Get(leveldb::ReadOptions(), key, &value);
|
||||
if (!status.ok()) {
|
||||
if (status.IsNotFound())
|
||||
return false;
|
||||
printf("LevelDB read failure: %s\n", status.ToString().c_str());
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
bool CTxDB::UpdateTxIndex(uint256 hash, const CTxIndex& txindex)
|
||||
bool CTxDB::WriteRaw(const std::string& key, const std::string& value)
|
||||
{
|
||||
assert(!fClient);
|
||||
return Write(make_pair(string("tx"), hash), txindex);
|
||||
}
|
||||
|
||||
bool CTxDB::AddTxIndex(const CTransaction& tx, const CDiskTxPos& pos, int nHeight)
|
||||
{
|
||||
assert(!fClient);
|
||||
|
||||
// Add to tx index
|
||||
uint256 hash = tx.GetHash();
|
||||
CTxIndex txindex(pos, tx.vout.size());
|
||||
return Write(make_pair(string("tx"), hash), txindex);
|
||||
}
|
||||
|
||||
bool CTxDB::EraseTxIndex(const CTransaction& tx)
|
||||
{
|
||||
assert(!fClient);
|
||||
uint256 hash = tx.GetHash();
|
||||
|
||||
return Erase(make_pair(string("tx"), hash));
|
||||
}
|
||||
|
||||
bool CTxDB::ContainsTx(uint256 hash)
|
||||
{
|
||||
assert(!fClient);
|
||||
return Exists(make_pair(string("tx"), hash));
|
||||
}
|
||||
|
||||
bool CTxDB::ReadDiskTx(uint256 hash, CTransaction& tx, CTxIndex& txindex)
|
||||
{
|
||||
assert(!fClient);
|
||||
tx.SetNull();
|
||||
if (!ReadTxIndex(hash, txindex))
|
||||
if (activeBatch) {
|
||||
activeBatch->Put(key, value);
|
||||
return true;
|
||||
}
|
||||
leveldb::Status status = pdb->Put(leveldb::WriteOptions(), key, value);
|
||||
if (!status.ok()) {
|
||||
printf("LevelDB write failure: %s\n", status.ToString().c_str());
|
||||
return false;
|
||||
return (tx.ReadFromDisk(txindex.pos));
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
bool CTxDB::ReadDiskTx(uint256 hash, CTransaction& tx)
|
||||
bool CTxDB::EraseRaw(const std::string& key)
|
||||
{
|
||||
CTxIndex txindex;
|
||||
return ReadDiskTx(hash, tx, txindex);
|
||||
if (!pdb)
|
||||
return false;
|
||||
if (activeBatch) {
|
||||
activeBatch->Delete(key);
|
||||
return true;
|
||||
}
|
||||
leveldb::Status status = pdb->Delete(leveldb::WriteOptions(), key);
|
||||
return (status.ok() || status.IsNotFound());
|
||||
}
|
||||
|
||||
bool CTxDB::ReadDiskTx(COutPoint outpoint, CTransaction& tx, CTxIndex& txindex)
|
||||
bool CTxDB::ExistsRaw(const std::string& key) const
|
||||
{
|
||||
return ReadDiskTx(outpoint.hash, tx, txindex);
|
||||
std::string unused;
|
||||
|
||||
if (activeBatch) {
|
||||
bool deleted = false;
|
||||
if (ScanBatch(key, &unused, &deleted) && !deleted)
|
||||
return true;
|
||||
}
|
||||
|
||||
leveldb::Status status = pdb->Get(leveldb::ReadOptions(), key, &unused);
|
||||
return status.IsNotFound() == false;
|
||||
}
|
||||
|
||||
bool CTxDB::ReadDiskTx(COutPoint outpoint, CTransaction& tx)
|
||||
std::unique_ptr<CTxDBIteratorBase> CTxDB::NewIterator() const
|
||||
{
|
||||
CTxIndex txindex;
|
||||
return ReadDiskTx(outpoint.hash, tx, txindex);
|
||||
}
|
||||
|
||||
bool CTxDB::WriteBlockIndex(const CDiskBlockIndex& blockindex)
|
||||
{
|
||||
return Write(make_pair(string("blockindex"), blockindex.GetBlockHash()), blockindex);
|
||||
}
|
||||
|
||||
bool CTxDB::ReadHashBestChain(uint256& hashBestChain)
|
||||
{
|
||||
return Read(string("hashBestChain"), hashBestChain);
|
||||
}
|
||||
|
||||
bool CTxDB::WriteHashBestChain(uint256 hashBestChain)
|
||||
{
|
||||
return Write(string("hashBestChain"), hashBestChain);
|
||||
}
|
||||
|
||||
bool CTxDB::ReadAddressIndexBestChain(uint256& hashBestChain)
|
||||
{
|
||||
return Read(string("addressIndexBestChain"), hashBestChain);
|
||||
}
|
||||
|
||||
bool CTxDB::WriteAddressIndexBestChain(uint256 hashBestChain)
|
||||
{
|
||||
return Write(string("addressIndexBestChain"), hashBestChain);
|
||||
}
|
||||
|
||||
bool CTxDB::ReadAddressIndexStartHeight(int& nHeight)
|
||||
{
|
||||
return Read(string("addressIndexStartHeight"), nHeight);
|
||||
}
|
||||
|
||||
bool CTxDB::WriteAddressIndexStartHeight(int nHeight)
|
||||
{
|
||||
return Write(string("addressIndexStartHeight"), nHeight);
|
||||
}
|
||||
|
||||
bool CTxDB::ReadBestInvalidTrust(CBigNum& bnBestInvalidTrust)
|
||||
{
|
||||
return Read(string("bnBestInvalidTrust"), bnBestInvalidTrust);
|
||||
}
|
||||
|
||||
bool CTxDB::WriteBestInvalidTrust(CBigNum bnBestInvalidTrust)
|
||||
{
|
||||
return Write(string("bnBestInvalidTrust"), bnBestInvalidTrust);
|
||||
}
|
||||
|
||||
bool CTxDB::ReadSyncCheckpoint(uint256& hashCheckpoint)
|
||||
{
|
||||
return Read(string("hashSyncCheckpoint"), hashCheckpoint);
|
||||
}
|
||||
|
||||
bool CTxDB::WriteSyncCheckpoint(uint256 hashCheckpoint)
|
||||
{
|
||||
return Write(string("hashSyncCheckpoint"), hashCheckpoint);
|
||||
}
|
||||
|
||||
bool CTxDB::ReadCheckpointPubKey(string& strPubKey)
|
||||
{
|
||||
return Read(string("strCheckpointPubKey"), strPubKey);
|
||||
}
|
||||
|
||||
bool CTxDB::WriteCheckpointPubKey(const string& strPubKey)
|
||||
{
|
||||
return Write(string("strCheckpointPubKey"), strPubKey);
|
||||
return std::unique_ptr<CTxDBIteratorBase>(
|
||||
new CLevelDBIterator(pdb->NewIterator(leveldb::ReadOptions())));
|
||||
}
|
||||
|
||||
static CBlockIndex *InsertBlockIndex(uint256 hash)
|
||||
@@ -347,12 +294,10 @@ static CBlockIndex *InsertBlockIndex(uint256 hash)
|
||||
if (hash == 0)
|
||||
return NULL;
|
||||
|
||||
// Return existing
|
||||
map<uint256, CBlockIndex*>::iterator mi = mapBlockIndex.find(hash);
|
||||
if (mi != mapBlockIndex.end())
|
||||
return (*mi).second;
|
||||
|
||||
// Create new
|
||||
CBlockIndex* pindexNew = new CBlockIndex();
|
||||
if (!pindexNew)
|
||||
throw runtime_error("LoadBlockIndex() : new CBlockIndex failed");
|
||||
@@ -365,8 +310,7 @@ static CBlockIndex *InsertBlockIndex(uint256 hash)
|
||||
bool CTxDB::LoadBlockIndex()
|
||||
{
|
||||
if (mapBlockIndex.size() > 0) {
|
||||
// Already loaded once in this session. It can happen during migration
|
||||
// from BDB.
|
||||
// Already loaded once in this session. Can happen during BDB migration.
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -376,39 +320,32 @@ bool CTxDB::LoadBlockIndex()
|
||||
CDiskBlockIndex::fSerializeChainTrust = (nDbFormat >= 2);
|
||||
|
||||
if (CDiskBlockIndex::fSerializeChainTrust)
|
||||
printf("LoadBlockIndex(): DB format v%d — nChainTrust persisted\n", nDbFormat);
|
||||
printf("LoadBlockIndex(): DB format v%d - nChainTrust persisted\n", nDbFormat);
|
||||
else
|
||||
printf("LoadBlockIndex(): DB format v%d — will recalculate nChainTrust (one-time upgrade)\n", nDbFormat);
|
||||
printf("LoadBlockIndex(): DB format v%d - will recalculate nChainTrust (one-time upgrade)\n", nDbFormat);
|
||||
|
||||
// The block index is an in-memory structure that maps hashes to on-disk
|
||||
// locations where the contents of the block can be found. Here, we scan it
|
||||
// out of the DB and into mapBlockIndex.
|
||||
// Scan the block index out of the DB into mapBlockIndex.
|
||||
int64_t nPhaseStart = GetTimeMillis();
|
||||
int64_t nTotalStart = nPhaseStart;
|
||||
leveldb::Iterator *iterator = pdb->NewIterator(leveldb::ReadOptions());
|
||||
// Seek to start key.
|
||||
CDataStream ssStartKey(SER_DISK, CLIENT_VERSION);
|
||||
ssStartKey << make_pair(string("blockindex"), uint256(0));
|
||||
iterator->Seek(ssStartKey.str());
|
||||
// Now read each entry.
|
||||
int nBlocksLoaded = 0;
|
||||
while (iterator->Valid())
|
||||
{
|
||||
// Report progress every 100k blocks
|
||||
if (++nBlocksLoaded % 100000 == 0)
|
||||
{
|
||||
std::string strMsg = strprintf(_("Loading block index... (%d blocks)"), nBlocksLoaded);
|
||||
uiInterface.InitMessage(strMsg);
|
||||
}
|
||||
|
||||
// Unpack keys and values.
|
||||
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
|
||||
ssKey.write(iterator->key().data(), iterator->key().size());
|
||||
CDataStream ssValue(SER_DISK, CLIENT_VERSION);
|
||||
ssValue.write(iterator->value().data(), iterator->value().size());
|
||||
string strType;
|
||||
ssKey >> strType;
|
||||
// Did we reach the end of the data to read?
|
||||
if (fRequestShutdown || strType != "blockindex")
|
||||
break;
|
||||
CDiskBlockIndex diskindex;
|
||||
@@ -416,7 +353,6 @@ bool CTxDB::LoadBlockIndex()
|
||||
|
||||
uint256 blockHash = diskindex.GetBlockHash();
|
||||
|
||||
// Construct block index object
|
||||
CBlockIndex* pindexNew = InsertBlockIndex(blockHash);
|
||||
pindexNew->pprev = InsertBlockIndex(diskindex.hashPrev);
|
||||
pindexNew->pnext = InsertBlockIndex(diskindex.hashNext);
|
||||
@@ -435,10 +371,8 @@ bool CTxDB::LoadBlockIndex()
|
||||
pindexNew->nTime = diskindex.nTime;
|
||||
pindexNew->nBits = diskindex.nBits;
|
||||
pindexNew->nNonce = diskindex.nNonce;
|
||||
// nChainTrust is populated from disk if fSerializeChainTrust, else stays 0
|
||||
pindexNew->nChainTrust = diskindex.nChainTrust;
|
||||
|
||||
// Watch for genesis block
|
||||
if (pindexGenesisBlock == NULL && blockHash == (!fTestNet ? hashGenesisBlockOfficial : hashGenesisBlockTestNet))
|
||||
pindexGenesisBlock = pindexNew;
|
||||
|
||||
@@ -447,8 +381,6 @@ bool CTxDB::LoadBlockIndex()
|
||||
return error("LoadBlockIndex() : CheckIndex failed at %d", pindexNew->nHeight);
|
||||
}
|
||||
|
||||
// setStakeSeen is populated below for recent blocks only (Change D)
|
||||
|
||||
iterator->Next();
|
||||
}
|
||||
delete iterator;
|
||||
@@ -512,7 +444,6 @@ bool CTxDB::LoadBlockIndex()
|
||||
ssValue << diskindex;
|
||||
batch.Put(ssKey.str(), ssValue.str());
|
||||
|
||||
// Flush in chunks to limit memory usage
|
||||
if (++nCount % 100000 == 0)
|
||||
{
|
||||
pdb->Write(leveldb::WriteOptions(), &batch);
|
||||
@@ -520,7 +451,6 @@ bool CTxDB::LoadBlockIndex()
|
||||
printf("LoadBlockIndex(): upgraded %d / %d block index entries\n", nCount, (int)vSortedByHeight.size());
|
||||
}
|
||||
}
|
||||
// Write remaining entries + format version
|
||||
CDataStream ssFmtKey(SER_DISK, CLIENT_VERSION);
|
||||
ssFmtKey << string("dbformat");
|
||||
CDataStream ssFmtValue(SER_DISK, CLIENT_VERSION);
|
||||
@@ -535,8 +465,6 @@ bool CTxDB::LoadBlockIndex()
|
||||
}
|
||||
else
|
||||
{
|
||||
// nChainTrust was loaded from disk. Only need stake modifier checksums
|
||||
// for blocks above the last checkpoint (typically very few or zero).
|
||||
int nLastCheckpointHeight = Checkpoints::GetTotalBlocksEstimate();
|
||||
bool fNeedModifierCheck = false;
|
||||
for (const auto& item : mapBlockIndex)
|
||||
@@ -568,16 +496,12 @@ bool CTxDB::LoadBlockIndex()
|
||||
|
||||
printf("STARTUP-PERF: chain_trust_and_modifiers %" PRId64 "ms\n", GetTimeMillis() - nPhaseStart);
|
||||
|
||||
// Bump dbformat to 3 if needed (databases that already had v2 nChainTrust upgrade).
|
||||
// UTXO entries are written by ConnectBlock during normal sync. For databases upgrading
|
||||
// from older versions, FetchInputs has a lazy fallback to the old CTxIndex path.
|
||||
if (nDbFormat < 3)
|
||||
{
|
||||
WriteDbFormat(3);
|
||||
printf("LoadBlockIndex(): bumped dbformat to v3 (UTXO model with lazy fallback)\n");
|
||||
}
|
||||
|
||||
// Load hashBestChain pointer to end of best chain
|
||||
nPhaseStart = GetTimeMillis();
|
||||
if (!ReadHashBestChain(hashBestChain))
|
||||
{
|
||||
@@ -593,7 +517,6 @@ bool CTxDB::LoadBlockIndex()
|
||||
|
||||
printf("STARTUP-PERF: best_chain %" PRId64 "ms\n", GetTimeMillis() - nPhaseStart);
|
||||
|
||||
// ---- setStakeSeen: only populate for recent blocks (DoS protection) ----
|
||||
nPhaseStart = GetTimeMillis();
|
||||
{
|
||||
int nStakeSeenDepth = 500;
|
||||
@@ -616,7 +539,6 @@ bool CTxDB::LoadBlockIndex()
|
||||
DateTimeStrFormat("%x %H:%M:%S", pindexBest->GetBlockTime()).c_str());
|
||||
|
||||
// Re-evaluate best chain: scan for competing tips with equal or greater trust.
|
||||
// This fixes nodes stuck on the wrong fork after consensus rule changes.
|
||||
{
|
||||
CBlockIndex* pindexBetter = NULL;
|
||||
for (const auto& item : mapBlockIndex)
|
||||
@@ -659,29 +581,25 @@ bool CTxDB::LoadBlockIndex()
|
||||
}
|
||||
}
|
||||
|
||||
// triangles: load hashSyncCheckpoint (best-effort, non-fatal)
|
||||
if (!ReadSyncCheckpoint(Checkpoints::hashSyncCheckpoint))
|
||||
printf("LoadBlockIndex(): no sync checkpoint in DB, using default\n");
|
||||
else
|
||||
printf("LoadBlockIndex(): synchronized checkpoint %s\n", Checkpoints::hashSyncCheckpoint.ToString().c_str());
|
||||
// If the stored checkpoint isn't in our index, reset to genesis so we don't assert-crash
|
||||
if (!mapBlockIndex.count(Checkpoints::hashSyncCheckpoint))
|
||||
{
|
||||
printf("LoadBlockIndex(): sync checkpoint not in index, resetting to genesis\n");
|
||||
Checkpoints::hashSyncCheckpoint = (!fTestNet ? hashGenesisBlockOfficial : hashGenesisBlockTestNet);
|
||||
}
|
||||
|
||||
// Load bnBestInvalidTrust, OK if it doesn't exist
|
||||
CBigNum bnBestInvalidTrust;
|
||||
ReadBestInvalidTrust(bnBestInvalidTrust);
|
||||
nBestInvalidTrust = bnBestInvalidTrust.getuint256();
|
||||
|
||||
// Verify blocks in the best chain
|
||||
nPhaseStart = GetTimeMillis();
|
||||
int nCheckLevel = GetArg("-checklevel", 1);
|
||||
int nCheckDepth = GetArg( "-checkblocks", 50);
|
||||
if (nCheckDepth == 0)
|
||||
nCheckDepth = 1000000000; // suffices until the year 19000
|
||||
nCheckDepth = 1000000000;
|
||||
if (nCheckDepth > nBestHeight)
|
||||
nCheckDepth = nBestHeight;
|
||||
printf("Verifying last %i blocks at level %i\n", nCheckDepth, nCheckLevel);
|
||||
@@ -694,14 +612,11 @@ bool CTxDB::LoadBlockIndex()
|
||||
CBlock block;
|
||||
if (!block.ReadFromDisk(pindex))
|
||||
return error("LoadBlockIndex() : block.ReadFromDisk failed");
|
||||
// check level 1: verify block validity
|
||||
// check level 7: verify block signature too
|
||||
if (nCheckLevel>0 && !block.CheckBlock(true, true, (nCheckLevel>6)))
|
||||
{
|
||||
printf("LoadBlockIndex() : *** found bad block at %d, hash=%s\n", pindex->nHeight, pindex->GetBlockHash().ToString().c_str());
|
||||
pindexFork = pindex->pprev;
|
||||
}
|
||||
// check level 2: verify transaction index validity
|
||||
if (nCheckLevel>1)
|
||||
{
|
||||
pair<unsigned int, unsigned int> pos = make_pair(pindex->nFile, pindex->nBlockPos);
|
||||
@@ -712,10 +627,8 @@ bool CTxDB::LoadBlockIndex()
|
||||
CTxIndex txindex;
|
||||
if (ReadTxIndex(hashTx, txindex))
|
||||
{
|
||||
// check level 3: checker transaction hashes
|
||||
if (nCheckLevel>2 || pindex->nFile != txindex.pos.nFile || pindex->nBlockPos != txindex.pos.nBlockPos)
|
||||
{
|
||||
// either an error or a duplicate transaction
|
||||
CTransaction txFound;
|
||||
if (!txFound.ReadFromDisk(txindex.pos))
|
||||
{
|
||||
@@ -723,13 +636,12 @@ bool CTxDB::LoadBlockIndex()
|
||||
pindexFork = pindex->pprev;
|
||||
}
|
||||
else
|
||||
if (txFound.GetHash() != hashTx) // not a duplicate tx
|
||||
if (txFound.GetHash() != hashTx)
|
||||
{
|
||||
printf("LoadBlockIndex(): *** invalid tx position for %s\n", hashTx.ToString().c_str());
|
||||
pindexFork = pindex->pprev;
|
||||
}
|
||||
}
|
||||
// check level 4: verify spent inputs were removed from UTXO set
|
||||
if (nCheckLevel>3 && !tx.IsCoinBase())
|
||||
{
|
||||
for (const CTxIn &txin : tx.vin)
|
||||
@@ -748,7 +660,6 @@ bool CTxDB::LoadBlockIndex()
|
||||
}
|
||||
if (pindexFork && !fRequestShutdown)
|
||||
{
|
||||
// Reorg back to the fork
|
||||
printf("LoadBlockIndex() : *** moving best chain pointer back to block %d\n", pindexFork->nHeight);
|
||||
CBlock block;
|
||||
if (!block.ReadFromDisk(pindexFork))
|
||||
@@ -761,180 +672,3 @@ bool CTxDB::LoadBlockIndex()
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Address index methods
|
||||
// ============================================================================
|
||||
|
||||
bool CTxDB::ReadAddressBalance(int nType, const uint160& hashBytes, int64_t& nBalance)
|
||||
{
|
||||
return Read(make_pair(string("addrbal"), CAddressBalanceKey(nType, hashBytes)), nBalance);
|
||||
}
|
||||
|
||||
bool CTxDB::WriteAddressBalance(int nType, const uint160& hashBytes, int64_t nBalance)
|
||||
{
|
||||
return Write(make_pair(string("addrbal"), CAddressBalanceKey(nType, hashBytes)), nBalance);
|
||||
}
|
||||
|
||||
bool CTxDB::ReadAddressUtxo(int nType, const uint160& hashBytes, const uint256& txhash, int nIndex, int64_t& nValue, int& nHeight)
|
||||
{
|
||||
CAddressUtxoValue val;
|
||||
if (!Read(make_pair(string("addrutxo"), CAddressUtxoKey(nType, hashBytes, txhash, nIndex)), val))
|
||||
return false;
|
||||
nValue = val.nValue;
|
||||
nHeight = val.nHeight;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool CTxDB::WriteAddressUtxo(int nType, const uint160& hashBytes, const uint256& txhash, int nIndex, int64_t nValue, int nHeight, const CScript& script)
|
||||
{
|
||||
return Write(make_pair(string("addrutxo"), CAddressUtxoKey(nType, hashBytes, txhash, nIndex)),
|
||||
CAddressUtxoValue(nValue, nHeight, script));
|
||||
}
|
||||
|
||||
bool CTxDB::EraseAddressUtxo(int nType, const uint160& hashBytes, const uint256& txhash, int nIndex)
|
||||
{
|
||||
return Erase(make_pair(string("addrutxo"), CAddressUtxoKey(nType, hashBytes, txhash, nIndex)));
|
||||
}
|
||||
|
||||
bool CTxDB::WriteAddressTxId(int nType, const uint160& hashBytes, int nHeight, int nTxIndex, const uint256& txhash)
|
||||
{
|
||||
return Write(make_pair(string("addrtxid"), CAddressTxIdKey(nType, hashBytes, nHeight, nTxIndex, txhash)), (char)0);
|
||||
}
|
||||
|
||||
bool CTxDB::EraseAddressTxId(int nType, const uint160& hashBytes, int nHeight, int nTxIndex, const uint256& txhash)
|
||||
{
|
||||
return Erase(make_pair(string("addrtxid"), CAddressTxIdKey(nType, hashBytes, nHeight, nTxIndex, txhash)));
|
||||
}
|
||||
|
||||
bool CTxDB::GetAddressUtxos(int nType, const uint160& hashBytes, std::vector<std::pair<COutPoint, std::pair<int64_t, int> > >& vUtxos)
|
||||
{
|
||||
vUtxos.clear();
|
||||
|
||||
// Build the key prefix to seek to
|
||||
CDataStream ssKeyPrefix(SER_DISK, CLIENT_VERSION);
|
||||
ssKeyPrefix << make_pair(string("addrutxo"), CAddressUtxoKey(nType, hashBytes, uint256(0), 0));
|
||||
std::string strPrefixBegin = ssKeyPrefix.str();
|
||||
|
||||
leveldb::Iterator* it = pdb->NewIterator(leveldb::ReadOptions());
|
||||
for (it->Seek(strPrefixBegin); it->Valid(); it->Next())
|
||||
{
|
||||
// Deserialize the key
|
||||
CDataStream ssKey(it->key().data(), it->key().data() + it->key().size(), SER_DISK, CLIENT_VERSION);
|
||||
std::string strKeyType;
|
||||
CAddressUtxoKey utxoKey;
|
||||
ssKey >> strKeyType;
|
||||
if (strKeyType != "addrutxo")
|
||||
break;
|
||||
ssKey >> utxoKey;
|
||||
if (utxoKey.nType != nType || utxoKey.hashBytes != hashBytes)
|
||||
break;
|
||||
|
||||
// Deserialize the value
|
||||
CDataStream ssValue(it->value().data(), it->value().data() + it->value().size(), SER_DISK, CLIENT_VERSION);
|
||||
CAddressUtxoValue utxoValue;
|
||||
ssValue >> utxoValue;
|
||||
|
||||
COutPoint outpoint(utxoKey.txhash, utxoKey.nIndex);
|
||||
vUtxos.push_back(make_pair(outpoint, make_pair(utxoValue.nValue, utxoValue.nHeight)));
|
||||
}
|
||||
delete it;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool CTxDB::GetAddressTxIds(int nType, const uint160& hashBytes, int nStartHeight, int nEndHeight, std::vector<uint256>& vTxIds)
|
||||
{
|
||||
vTxIds.clear();
|
||||
|
||||
// Build the key prefix to seek to
|
||||
CDataStream ssKeyPrefix(SER_DISK, CLIENT_VERSION);
|
||||
ssKeyPrefix << make_pair(string("addrtxid"), CAddressTxIdKey(nType, hashBytes, nStartHeight, 0, uint256(0)));
|
||||
std::string strPrefixBegin = ssKeyPrefix.str();
|
||||
|
||||
leveldb::Iterator* it = pdb->NewIterator(leveldb::ReadOptions());
|
||||
for (it->Seek(strPrefixBegin); it->Valid(); it->Next())
|
||||
{
|
||||
CDataStream ssKey(it->key().data(), it->key().data() + it->key().size(), SER_DISK, CLIENT_VERSION);
|
||||
std::string strKeyType;
|
||||
CAddressTxIdKey txIdKey;
|
||||
ssKey >> strKeyType;
|
||||
if (strKeyType != "addrtxid")
|
||||
break;
|
||||
ssKey >> txIdKey;
|
||||
if (txIdKey.nType != nType || txIdKey.hashBytes != hashBytes)
|
||||
break;
|
||||
if (txIdKey.nHeight > nEndHeight)
|
||||
break;
|
||||
|
||||
vTxIds.push_back(txIdKey.txhash);
|
||||
}
|
||||
delete it;
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---------- UTXO database methods ----------
|
||||
|
||||
bool CTxDB::ReadUtxo(const uint256& hash, unsigned int n, CUtxoEntry& entry)
|
||||
{
|
||||
entry.SetNull();
|
||||
return Read(make_pair(string("u"), make_pair(hash, n)), entry);
|
||||
}
|
||||
|
||||
bool CTxDB::WriteUtxo(const uint256& hash, unsigned int n, const CUtxoEntry& entry)
|
||||
{
|
||||
return Write(make_pair(string("u"), make_pair(hash, n)), entry);
|
||||
}
|
||||
|
||||
bool CTxDB::EraseUtxo(const uint256& hash, unsigned int n)
|
||||
{
|
||||
return Erase(make_pair(string("u"), make_pair(hash, n)));
|
||||
}
|
||||
|
||||
bool CTxDB::HaveUtxo(const uint256& hash, unsigned int n)
|
||||
{
|
||||
if (Exists(make_pair(string("u"), make_pair(hash, n))))
|
||||
return true;
|
||||
|
||||
// Lazy fallback: check old CTxIndex vSpent for databases upgrading from pre-UTXO format
|
||||
CTxIndex txindex;
|
||||
if (ReadTxIndex(hash, txindex))
|
||||
{
|
||||
if (n < txindex.vSpent.size() && txindex.vSpent[n].IsNull())
|
||||
return true; // vSpent[n] is null = output NOT spent = UTXO exists
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
int64_t CTxDB::SumUtxoValues(int& nCount)
|
||||
{
|
||||
nCount = 0;
|
||||
int64_t nTotal = 0;
|
||||
|
||||
// Seek to the start of UTXO entries (key prefix "u")
|
||||
CDataStream ssKeyPrefix(SER_DISK, CLIENT_VERSION);
|
||||
ssKeyPrefix << make_pair(string("u"), make_pair(uint256(0), (unsigned int)0));
|
||||
std::string strPrefixBegin = ssKeyPrefix.str();
|
||||
|
||||
leveldb::Iterator* it = pdb->NewIterator(leveldb::ReadOptions());
|
||||
for (it->Seek(strPrefixBegin); it->Valid(); it->Next())
|
||||
{
|
||||
// Check key prefix is still "u"
|
||||
CDataStream ssKey(it->key().data(), it->key().data() + it->key().size(), SER_DISK, CLIENT_VERSION);
|
||||
std::string strKeyType;
|
||||
ssKey >> strKeyType;
|
||||
if (strKeyType != "u")
|
||||
break;
|
||||
|
||||
// Deserialize the UTXO entry and sum the value
|
||||
CDataStream ssValue(it->value().data(), it->value().data() + it->value().size(), SER_DISK, CLIENT_VERSION);
|
||||
CUtxoEntry entry;
|
||||
ssValue >> entry;
|
||||
|
||||
nTotal += entry.nValue;
|
||||
nCount++;
|
||||
}
|
||||
delete it;
|
||||
return nTotal;
|
||||
}
|
||||
|
||||
|
||||
+33
-213
@@ -6,241 +6,61 @@
|
||||
#ifndef TRIANGLES_LEVELDB_H
|
||||
#define TRIANGLES_LEVELDB_H
|
||||
|
||||
#include "main.h"
|
||||
|
||||
#include <map>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
#include "txdb-base.h"
|
||||
|
||||
#include <leveldb/db.h>
|
||||
#include <leveldb/write_batch.h>
|
||||
|
||||
// Class that provides access to a LevelDB. Note that this class is frequently
|
||||
// instantiated on the stack and then destroyed again, so instantiation has to
|
||||
// be very cheap. Unfortunately that means, a CTxDB instance is actually just a
|
||||
// wrapper around some global state.
|
||||
// LevelDB backend for the chain database.
|
||||
//
|
||||
// A LevelDB is a key/value store that is optimized for fast usage on hard
|
||||
// disks. It prefers long read/writes to seeks and is based on a series of
|
||||
// sorted key/value mapping files that are stacked on top of each other, with
|
||||
// newer files overriding older files. A background thread compacts them
|
||||
// together when too many files stack up.
|
||||
// Cheap to construct/destruct: every instance shares a single global
|
||||
// leveldb::DB pointer, opened lazily on first use. Most of the codebase
|
||||
// instantiates a CTxDB on the stack for short-lived operations.
|
||||
//
|
||||
// Learn more: http://code.google.com/p/leveldb/
|
||||
class CTxDB
|
||||
// The protected templated Read/Write/Erase/Exists live in CTxDBBase and
|
||||
// dispatch to ReadRaw/WriteRaw/EraseRaw/ExistsRaw below, which handle the
|
||||
// active-batch logic so reads-after-writes within an open batch see their
|
||||
// own pending changes.
|
||||
class CTxDB final : public CTxDBBase
|
||||
{
|
||||
public:
|
||||
CTxDB(const char* pszMode="r+");
|
||||
~CTxDB() {
|
||||
// Note that this is not the same as Close() because it deletes only
|
||||
// data scoped to this TxDB object.
|
||||
CTxDB(const char* pszMode = "r+");
|
||||
~CTxDB() override {
|
||||
delete activeBatch;
|
||||
}
|
||||
|
||||
// Destroys the underlying shared global state accessed by this TxDB.
|
||||
void Close();
|
||||
void Close() override;
|
||||
|
||||
private:
|
||||
leveldb::DB *pdb; // Points to the global instance.
|
||||
|
||||
// A batch stores up writes and deletes for atomic application. When this
|
||||
// field is non-NULL, writes/deletes go there instead of directly to disk.
|
||||
leveldb::WriteBatch *activeBatch;
|
||||
leveldb::Options options;
|
||||
bool fReadOnly;
|
||||
int nVersion;
|
||||
|
||||
protected:
|
||||
// Returns true and sets (value,false) if activeBatch contains the given key
|
||||
// or leaves value alone and sets deleted = true if activeBatch contains a
|
||||
// delete for it.
|
||||
bool ScanBatch(const CDataStream &key, std::string *value, bool *deleted) const;
|
||||
|
||||
template<typename K, typename T>
|
||||
bool Read(const K& key, T& value)
|
||||
{
|
||||
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
|
||||
ssKey.reserve(1000);
|
||||
ssKey << key;
|
||||
std::string strValue;
|
||||
|
||||
bool readFromDb = true;
|
||||
if (activeBatch) {
|
||||
// First we must search for it in the currently pending set of
|
||||
// changes to the db. If not found in the batch, go on to read disk.
|
||||
bool deleted = false;
|
||||
readFromDb = ScanBatch(ssKey, &strValue, &deleted) == false;
|
||||
if (deleted) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (readFromDb) {
|
||||
leveldb::Status status = pdb->Get(leveldb::ReadOptions(),
|
||||
ssKey.str(), &strValue);
|
||||
if (!status.ok()) {
|
||||
if (status.IsNotFound())
|
||||
return false;
|
||||
// Some unexpected error.
|
||||
printf("LevelDB read failure: %s\n", status.ToString().c_str());
|
||||
return false;
|
||||
}
|
||||
}
|
||||
// Unserialize value
|
||||
try {
|
||||
CDataStream ssValue(strValue.data(), strValue.data() + strValue.size(),
|
||||
SER_DISK, CLIENT_VERSION);
|
||||
ssValue >> value;
|
||||
}
|
||||
catch (std::exception &e) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
template<typename K, typename T>
|
||||
bool Write(const K& key, const T& value)
|
||||
{
|
||||
if (fReadOnly)
|
||||
assert(!"Write called on database in read-only mode");
|
||||
|
||||
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
|
||||
ssKey.reserve(1000);
|
||||
ssKey << key;
|
||||
CDataStream ssValue(SER_DISK, CLIENT_VERSION);
|
||||
ssValue.reserve(10000);
|
||||
ssValue << value;
|
||||
|
||||
if (activeBatch) {
|
||||
activeBatch->Put(ssKey.str(), ssValue.str());
|
||||
return true;
|
||||
}
|
||||
leveldb::Status status = pdb->Put(leveldb::WriteOptions(), ssKey.str(), ssValue.str());
|
||||
if (!status.ok()) {
|
||||
printf("LevelDB write failure: %s\n", status.ToString().c_str());
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
template<typename K>
|
||||
bool Erase(const K& key)
|
||||
{
|
||||
if (!pdb)
|
||||
return false;
|
||||
if (fReadOnly)
|
||||
assert(!"Erase called on database in read-only mode");
|
||||
|
||||
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
|
||||
ssKey.reserve(1000);
|
||||
ssKey << key;
|
||||
if (activeBatch) {
|
||||
activeBatch->Delete(ssKey.str());
|
||||
return true;
|
||||
}
|
||||
leveldb::Status status = pdb->Delete(leveldb::WriteOptions(), ssKey.str());
|
||||
return (status.ok() || status.IsNotFound());
|
||||
}
|
||||
|
||||
template<typename K>
|
||||
bool Exists(const K& key)
|
||||
{
|
||||
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
|
||||
ssKey.reserve(1000);
|
||||
ssKey << key;
|
||||
std::string unused;
|
||||
|
||||
if (activeBatch) {
|
||||
bool deleted;
|
||||
if (ScanBatch(ssKey, &unused, &deleted) && !deleted) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
leveldb::Status status = pdb->Get(leveldb::ReadOptions(), ssKey.str(), &unused);
|
||||
return status.IsNotFound() == false;
|
||||
}
|
||||
|
||||
|
||||
public:
|
||||
bool TxnBegin();
|
||||
bool TxnCommit();
|
||||
bool TxnAbort()
|
||||
bool TxnBegin() override;
|
||||
bool TxnCommit() override;
|
||||
bool TxnAbort() override
|
||||
{
|
||||
delete activeBatch;
|
||||
activeBatch = NULL;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool ReadVersion(int& nVersion)
|
||||
{
|
||||
nVersion = 0;
|
||||
return Read(std::string("version"), nVersion);
|
||||
}
|
||||
bool LoadBlockIndex() override;
|
||||
|
||||
bool WriteVersion(int nVersion)
|
||||
{
|
||||
return Write(std::string("version"), nVersion);
|
||||
}
|
||||
|
||||
bool ReadDbFormat(int& nDbFormat)
|
||||
{
|
||||
nDbFormat = 1;
|
||||
return Read(std::string("dbformat"), nDbFormat);
|
||||
}
|
||||
|
||||
bool WriteDbFormat(int nDbFormat)
|
||||
{
|
||||
return Write(std::string("dbformat"), nDbFormat);
|
||||
}
|
||||
|
||||
bool ReadTxIndex(uint256 hash, CTxIndex& txindex);
|
||||
bool UpdateTxIndex(uint256 hash, const CTxIndex& txindex);
|
||||
bool AddTxIndex(const CTransaction& tx, const CDiskTxPos& pos, int nHeight);
|
||||
bool EraseTxIndex(const CTransaction& tx);
|
||||
bool ContainsTx(uint256 hash);
|
||||
bool ReadDiskTx(uint256 hash, CTransaction& tx, CTxIndex& txindex);
|
||||
bool ReadDiskTx(uint256 hash, CTransaction& tx);
|
||||
bool ReadDiskTx(COutPoint outpoint, CTransaction& tx, CTxIndex& txindex);
|
||||
bool ReadDiskTx(COutPoint outpoint, CTransaction& tx);
|
||||
bool WriteBlockIndex(const CDiskBlockIndex& blockindex);
|
||||
bool ReadHashBestChain(uint256& hashBestChain);
|
||||
bool WriteHashBestChain(uint256 hashBestChain);
|
||||
bool ReadAddressIndexBestChain(uint256& hashBestChain);
|
||||
bool WriteAddressIndexBestChain(uint256 hashBestChain);
|
||||
bool ReadAddressIndexStartHeight(int& nHeight);
|
||||
bool WriteAddressIndexStartHeight(int nHeight);
|
||||
bool ReadBestInvalidTrust(CBigNum& bnBestInvalidTrust);
|
||||
bool WriteBestInvalidTrust(CBigNum bnBestInvalidTrust);
|
||||
bool ReadSyncCheckpoint(uint256& hashCheckpoint);
|
||||
bool WriteSyncCheckpoint(uint256 hashCheckpoint);
|
||||
bool ReadCheckpointPubKey(std::string& strPubKey);
|
||||
bool WriteCheckpointPubKey(const std::string& strPubKey);
|
||||
bool LoadBlockIndex();
|
||||
|
||||
// Address index methods
|
||||
bool ReadAddressBalance(int nType, const uint160& hashBytes, int64_t& nBalance);
|
||||
bool WriteAddressBalance(int nType, const uint160& hashBytes, int64_t nBalance);
|
||||
bool ReadAddressUtxo(int nType, const uint160& hashBytes, const uint256& txhash, int nIndex, int64_t& nValue, int& nHeight);
|
||||
bool WriteAddressUtxo(int nType, const uint160& hashBytes, const uint256& txhash, int nIndex, int64_t nValue, int nHeight, const CScript& script);
|
||||
bool EraseAddressUtxo(int nType, const uint160& hashBytes, const uint256& txhash, int nIndex);
|
||||
bool WriteAddressTxId(int nType, const uint160& hashBytes, int nHeight, int nTxIndex, const uint256& txhash);
|
||||
bool EraseAddressTxId(int nType, const uint160& hashBytes, int nHeight, int nTxIndex, const uint256& txhash);
|
||||
|
||||
// Address index iteration (for RPC queries)
|
||||
bool GetAddressUtxos(int nType, const uint160& hashBytes, std::vector<std::pair<COutPoint, std::pair<int64_t, int> > >& vUtxos);
|
||||
bool GetAddressTxIds(int nType, const uint160& hashBytes, int nStartHeight, int nEndHeight, std::vector<uint256>& vTxIds);
|
||||
|
||||
// UTXO database methods
|
||||
bool ReadUtxo(const uint256& hash, unsigned int n, CUtxoEntry& entry);
|
||||
bool WriteUtxo(const uint256& hash, unsigned int n, const CUtxoEntry& entry);
|
||||
bool EraseUtxo(const uint256& hash, unsigned int n);
|
||||
bool HaveUtxo(const uint256& hash, unsigned int n);
|
||||
int64_t SumUtxoValues(int& nCount);
|
||||
protected:
|
||||
bool ReadRaw(const std::string& key, std::string& value) const override;
|
||||
bool WriteRaw(const std::string& key, const std::string& value) override;
|
||||
bool EraseRaw(const std::string& key) override;
|
||||
bool ExistsRaw(const std::string& key) const override;
|
||||
std::unique_ptr<CTxDBIteratorBase> NewIterator() const override;
|
||||
|
||||
private:
|
||||
leveldb::DB* pdb; // Points to the global instance.
|
||||
leveldb::WriteBatch* activeBatch; // When non-NULL, writes/deletes go here.
|
||||
leveldb::Options options;
|
||||
int nVersion;
|
||||
|
||||
// Returns true and sets (value,false) if activeBatch contains the given
|
||||
// key, or leaves value alone and sets deleted=true if activeBatch contains
|
||||
// a delete for it.
|
||||
bool ScanBatch(const std::string& key, std::string* value, bool* deleted) const;
|
||||
|
||||
bool LoadBlockIndexGuts();
|
||||
};
|
||||
|
||||
|
||||
#endif // TRIANGLES_LEVELDB_H
|
||||
|
||||
@@ -0,0 +1,703 @@
|
||||
// Copyright (c) 2026 The Triangles developers.
|
||||
// Distributed under the MIT/X11 software license, see the accompanying
|
||||
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
|
||||
#ifdef BUILD_ROCKSDB
|
||||
|
||||
#include "txdb-rocksdb.h"
|
||||
|
||||
#include <map>
|
||||
|
||||
#include <boost/version.hpp>
|
||||
#include <boost/filesystem.hpp>
|
||||
#include <boost/filesystem/fstream.hpp>
|
||||
|
||||
#include <rocksdb/cache.h>
|
||||
#include <rocksdb/filter_policy.h>
|
||||
#include <rocksdb/iterator.h>
|
||||
#include <rocksdb/slice.h>
|
||||
#include <rocksdb/table.h>
|
||||
#include <rocksdb/write_batch.h>
|
||||
|
||||
#include "kernel.h"
|
||||
#include "checkpoints.h"
|
||||
#include "txdb.h"
|
||||
#include "util.h"
|
||||
#include "ui_interface.h"
|
||||
#include "addressindex.h"
|
||||
#include "main.h"
|
||||
|
||||
using namespace std;
|
||||
using namespace boost;
|
||||
namespace fs = boost::filesystem;
|
||||
|
||||
// Global pointer for the RocksDB instance, shared across CRocksTxDB instances
|
||||
// the same way the LevelDB backend shares its txdb singleton.
|
||||
static rocksdb::DB* g_rocksdb = nullptr;
|
||||
|
||||
static rocksdb::Options GetRocksOptions()
|
||||
{
|
||||
rocksdb::Options opts;
|
||||
opts.create_if_missing = false;
|
||||
opts.compression = rocksdb::kSnappyCompression;
|
||||
opts.max_open_files = 1000;
|
||||
opts.write_buffer_size = 64 * 1048576;
|
||||
opts.IncreaseParallelism(); // Multi-threaded compaction.
|
||||
opts.OptimizeLevelStyleCompaction(); // Sensible defaults for a LSM workload.
|
||||
|
||||
rocksdb::BlockBasedTableOptions table_opts;
|
||||
int nCacheSizeMB = GetArg("-dbcache", 2048);
|
||||
table_opts.block_cache = rocksdb::NewLRUCache(static_cast<size_t>(nCacheSizeMB) * 1048576);
|
||||
table_opts.filter_policy.reset(rocksdb::NewBloomFilterPolicy(10, false));
|
||||
opts.table_factory.reset(rocksdb::NewBlockBasedTableFactory(table_opts));
|
||||
|
||||
return opts;
|
||||
}
|
||||
|
||||
static void open_rocksdb(rocksdb::Options& options, bool fRemoveOld = false)
|
||||
{
|
||||
fs::path directory = GetDataDir() / "rocksdb";
|
||||
|
||||
if (fRemoveOld) {
|
||||
fs::remove_all(directory);
|
||||
}
|
||||
|
||||
fs::create_directory(directory);
|
||||
printf("Opening RocksDB in %s\n", directory.string().c_str());
|
||||
rocksdb::Status status = rocksdb::DB::Open(options, directory.string(), &g_rocksdb);
|
||||
if (!status.ok()) {
|
||||
throw runtime_error(strprintf("open_rocksdb(): error opening database: %s",
|
||||
status.ToString().c_str()));
|
||||
}
|
||||
}
|
||||
|
||||
CRocksTxDB::CRocksTxDB(const char* pszMode)
|
||||
: pdb(nullptr), activeBatch(nullptr), nVersion(0)
|
||||
{
|
||||
assert(pszMode);
|
||||
fReadOnly = (!strchr(pszMode, '+') && !strchr(pszMode, 'w'));
|
||||
|
||||
if (g_rocksdb) {
|
||||
pdb = g_rocksdb;
|
||||
return;
|
||||
}
|
||||
|
||||
bool fCreate = strchr(pszMode, 'c');
|
||||
options = GetRocksOptions();
|
||||
options.create_if_missing = fCreate;
|
||||
|
||||
open_rocksdb(options);
|
||||
pdb = g_rocksdb;
|
||||
|
||||
if (Exists(string("version")))
|
||||
{
|
||||
ReadVersion(nVersion);
|
||||
printf("RocksDB transaction index version is %d\n", nVersion);
|
||||
|
||||
if (nVersion < DATABASE_VERSION)
|
||||
{
|
||||
printf("Required index version is %d, removing old RocksDB database\n",
|
||||
DATABASE_VERSION);
|
||||
|
||||
delete g_rocksdb;
|
||||
g_rocksdb = pdb = nullptr;
|
||||
delete activeBatch;
|
||||
activeBatch = nullptr;
|
||||
|
||||
open_rocksdb(options, true);
|
||||
pdb = g_rocksdb;
|
||||
|
||||
bool fTmp = fReadOnly;
|
||||
fReadOnly = false;
|
||||
WriteVersion(DATABASE_VERSION);
|
||||
fReadOnly = fTmp;
|
||||
}
|
||||
}
|
||||
else if (fCreate)
|
||||
{
|
||||
bool fTmp = fReadOnly;
|
||||
fReadOnly = false;
|
||||
WriteVersion(DATABASE_VERSION);
|
||||
fReadOnly = fTmp;
|
||||
}
|
||||
|
||||
printf("Opened RocksDB successfully\n");
|
||||
}
|
||||
|
||||
CRocksTxDB::~CRocksTxDB()
|
||||
{
|
||||
delete activeBatch;
|
||||
}
|
||||
|
||||
void CRocksTxDB::Close()
|
||||
{
|
||||
delete g_rocksdb;
|
||||
g_rocksdb = pdb = nullptr;
|
||||
delete activeBatch;
|
||||
activeBatch = nullptr;
|
||||
}
|
||||
|
||||
bool CRocksTxDB::TxnBegin()
|
||||
{
|
||||
if (activeBatch)
|
||||
return true;
|
||||
activeBatch = new rocksdb::WriteBatch();
|
||||
return true;
|
||||
}
|
||||
|
||||
bool CRocksTxDB::TxnCommit()
|
||||
{
|
||||
assert(activeBatch);
|
||||
rocksdb::Status status = pdb->Write(rocksdb::WriteOptions(), activeBatch);
|
||||
delete activeBatch;
|
||||
activeBatch = nullptr;
|
||||
if (!status.ok()) {
|
||||
printf("ERROR: RocksDB batch commit failure: %s\n", status.ToString().c_str());
|
||||
printf("ERROR: This may indicate disk full, corruption, or permissions issue.\n");
|
||||
printf("ERROR: Chain state may be inconsistent - immediate investigation required!\n");
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
bool CRocksTxDB::TxnAbort()
|
||||
{
|
||||
delete activeBatch;
|
||||
activeBatch = nullptr;
|
||||
return true;
|
||||
}
|
||||
|
||||
namespace {
|
||||
|
||||
// rocksdb::WriteBatch::Handler used to scan the active batch for a pending
|
||||
// write/delete on a given key, the same way the LevelDB backend does.
|
||||
class CRocksBatchScanner : public rocksdb::WriteBatch::Handler {
|
||||
public:
|
||||
std::string needle;
|
||||
bool* deleted = nullptr;
|
||||
std::string* foundValue = nullptr;
|
||||
bool foundEntry = false;
|
||||
|
||||
CRocksBatchScanner() = default;
|
||||
|
||||
void Put(const rocksdb::Slice& key, const rocksdb::Slice& value) override {
|
||||
if (key.ToString() == needle) {
|
||||
foundEntry = true;
|
||||
*deleted = false;
|
||||
*foundValue = value.ToString();
|
||||
}
|
||||
}
|
||||
|
||||
void Delete(const rocksdb::Slice& key) override {
|
||||
if (key.ToString() == needle) {
|
||||
foundEntry = true;
|
||||
*deleted = true;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
class CRocksDBIterator final : public CTxDBIteratorBase {
|
||||
public:
|
||||
explicit CRocksDBIterator(rocksdb::Iterator* pit) : pit(pit) {}
|
||||
~CRocksDBIterator() override { delete pit; }
|
||||
|
||||
void Seek(const std::string& key) override { pit->Seek(key); }
|
||||
bool Valid() const override { return pit->Valid(); }
|
||||
void Next() override { pit->Next(); }
|
||||
std::string KeyStr() const override { return pit->key().ToString(); }
|
||||
std::string ValueStr() const override { return pit->value().ToString(); }
|
||||
|
||||
private:
|
||||
rocksdb::Iterator* pit;
|
||||
};
|
||||
|
||||
} // anonymous namespace
|
||||
|
||||
bool CRocksTxDB::ScanBatch(const std::string& key, std::string* value, bool* deleted) const
|
||||
{
|
||||
assert(activeBatch);
|
||||
*deleted = false;
|
||||
CRocksBatchScanner scanner;
|
||||
scanner.needle = key;
|
||||
scanner.deleted = deleted;
|
||||
scanner.foundValue = value;
|
||||
rocksdb::Status status = activeBatch->Iterate(&scanner);
|
||||
if (!status.ok()) {
|
||||
throw runtime_error(status.ToString());
|
||||
}
|
||||
return scanner.foundEntry;
|
||||
}
|
||||
|
||||
bool CRocksTxDB::ReadRaw(const std::string& key, std::string& value) const
|
||||
{
|
||||
bool readFromDb = true;
|
||||
if (activeBatch) {
|
||||
bool deleted = false;
|
||||
readFromDb = ScanBatch(key, &value, &deleted) == false;
|
||||
if (deleted)
|
||||
return false;
|
||||
}
|
||||
if (readFromDb) {
|
||||
rocksdb::Status status = pdb->Get(rocksdb::ReadOptions(), key, &value);
|
||||
if (!status.ok()) {
|
||||
if (status.IsNotFound())
|
||||
return false;
|
||||
printf("RocksDB read failure: %s\n", status.ToString().c_str());
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
bool CRocksTxDB::WriteRaw(const std::string& key, const std::string& value)
|
||||
{
|
||||
if (activeBatch) {
|
||||
activeBatch->Put(key, value);
|
||||
return true;
|
||||
}
|
||||
rocksdb::Status status = pdb->Put(rocksdb::WriteOptions(), key, value);
|
||||
if (!status.ok()) {
|
||||
printf("RocksDB write failure: %s\n", status.ToString().c_str());
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
bool CRocksTxDB::EraseRaw(const std::string& key)
|
||||
{
|
||||
if (!pdb)
|
||||
return false;
|
||||
if (activeBatch) {
|
||||
activeBatch->Delete(key);
|
||||
return true;
|
||||
}
|
||||
rocksdb::Status status = pdb->Delete(rocksdb::WriteOptions(), key);
|
||||
return (status.ok() || status.IsNotFound());
|
||||
}
|
||||
|
||||
bool CRocksTxDB::ExistsRaw(const std::string& key) const
|
||||
{
|
||||
std::string unused;
|
||||
|
||||
if (activeBatch) {
|
||||
bool deleted = false;
|
||||
if (ScanBatch(key, &unused, &deleted) && !deleted)
|
||||
return true;
|
||||
}
|
||||
|
||||
rocksdb::Status status = pdb->Get(rocksdb::ReadOptions(), key, &unused);
|
||||
return status.IsNotFound() == false;
|
||||
}
|
||||
|
||||
std::unique_ptr<CTxDBIteratorBase> CRocksTxDB::NewIterator() const
|
||||
{
|
||||
return std::unique_ptr<CTxDBIteratorBase>(
|
||||
new CRocksDBIterator(pdb->NewIterator(rocksdb::ReadOptions())));
|
||||
}
|
||||
|
||||
// ─── LoadBlockIndex ─────────────────────────────────────────────────────────
|
||||
// Mirrors CTxDB::LoadBlockIndex with rocksdb:: substitutions. The dbformat
|
||||
// upgrade path is preserved verbatim because a freshly-imported RocksDB may
|
||||
// have been migrated from a v1 LevelDB and still need the chain-trust pass.
|
||||
//
|
||||
// This duplication is acknowledged debt — CTxDBBase will absorb LoadBlockIndex
|
||||
// into the base class in a later phase once the iterator/batch abstractions
|
||||
// have proven stable across both backends.
|
||||
// ────────────────────────────────────────────────────────────────────────────
|
||||
static CBlockIndex *InsertBlockIndexRocks(uint256 hash)
|
||||
{
|
||||
if (hash == 0)
|
||||
return nullptr;
|
||||
|
||||
auto mi = mapBlockIndex.find(hash);
|
||||
if (mi != mapBlockIndex.end())
|
||||
return mi->second;
|
||||
|
||||
CBlockIndex* pindexNew = new CBlockIndex();
|
||||
if (!pindexNew)
|
||||
throw runtime_error("LoadBlockIndex(): new CBlockIndex failed");
|
||||
mi = mapBlockIndex.insert(make_pair(hash, pindexNew)).first;
|
||||
pindexNew->phashBlock = &mi->first;
|
||||
|
||||
return pindexNew;
|
||||
}
|
||||
|
||||
bool CRocksTxDB::LoadBlockIndex()
|
||||
{
|
||||
if (mapBlockIndex.size() > 0) {
|
||||
return true;
|
||||
}
|
||||
|
||||
int nDbFormat = 1;
|
||||
ReadDbFormat(nDbFormat);
|
||||
CDiskBlockIndex::fSerializeChainTrust = (nDbFormat >= 2);
|
||||
|
||||
if (CDiskBlockIndex::fSerializeChainTrust)
|
||||
printf("LoadBlockIndex(): RocksDB format v%d - nChainTrust persisted\n", nDbFormat);
|
||||
else
|
||||
printf("LoadBlockIndex(): RocksDB format v%d - will recalculate nChainTrust\n", nDbFormat);
|
||||
|
||||
int64_t nPhaseStart = GetTimeMillis();
|
||||
int64_t nTotalStart = nPhaseStart;
|
||||
rocksdb::Iterator* iterator = pdb->NewIterator(rocksdb::ReadOptions());
|
||||
CDataStream ssStartKey(SER_DISK, CLIENT_VERSION);
|
||||
ssStartKey << make_pair(string("blockindex"), uint256(0));
|
||||
iterator->Seek(ssStartKey.str());
|
||||
int nBlocksLoaded = 0;
|
||||
while (iterator->Valid())
|
||||
{
|
||||
if (++nBlocksLoaded % 100000 == 0)
|
||||
{
|
||||
std::string strMsg = strprintf(_("Loading block index... (%d blocks)"), nBlocksLoaded);
|
||||
uiInterface.InitMessage(strMsg);
|
||||
}
|
||||
|
||||
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
|
||||
ssKey.write(iterator->key().data(), iterator->key().size());
|
||||
CDataStream ssValue(SER_DISK, CLIENT_VERSION);
|
||||
ssValue.write(iterator->value().data(), iterator->value().size());
|
||||
string strType;
|
||||
ssKey >> strType;
|
||||
if (fRequestShutdown || strType != "blockindex")
|
||||
break;
|
||||
CDiskBlockIndex diskindex;
|
||||
ssValue >> diskindex;
|
||||
|
||||
uint256 blockHash = diskindex.GetBlockHash();
|
||||
|
||||
CBlockIndex* pindexNew = InsertBlockIndexRocks(blockHash);
|
||||
pindexNew->pprev = InsertBlockIndexRocks(diskindex.hashPrev);
|
||||
pindexNew->pnext = InsertBlockIndexRocks(diskindex.hashNext);
|
||||
pindexNew->nFile = diskindex.nFile;
|
||||
pindexNew->nBlockPos = diskindex.nBlockPos;
|
||||
pindexNew->nHeight = diskindex.nHeight;
|
||||
pindexNew->nMint = diskindex.nMint;
|
||||
pindexNew->nMoneySupply = diskindex.nMoneySupply;
|
||||
pindexNew->nFlags = diskindex.nFlags;
|
||||
pindexNew->nStakeModifier = diskindex.nStakeModifier;
|
||||
pindexNew->prevoutStake = diskindex.prevoutStake;
|
||||
pindexNew->nStakeTime = diskindex.nStakeTime;
|
||||
pindexNew->hashProofOfStake = diskindex.hashProofOfStake;
|
||||
pindexNew->nVersion = diskindex.nVersion;
|
||||
pindexNew->hashMerkleRoot = diskindex.hashMerkleRoot;
|
||||
pindexNew->nTime = diskindex.nTime;
|
||||
pindexNew->nBits = diskindex.nBits;
|
||||
pindexNew->nNonce = diskindex.nNonce;
|
||||
pindexNew->nChainTrust = diskindex.nChainTrust;
|
||||
|
||||
if (pindexGenesisBlock == nullptr && blockHash == (!fTestNet ? hashGenesisBlockOfficial : hashGenesisBlockTestNet))
|
||||
pindexGenesisBlock = pindexNew;
|
||||
|
||||
if (!pindexNew->CheckIndex()) {
|
||||
delete iterator;
|
||||
return error("LoadBlockIndex(): CheckIndex failed at %d", pindexNew->nHeight);
|
||||
}
|
||||
|
||||
iterator->Next();
|
||||
}
|
||||
delete iterator;
|
||||
printf("STARTUP-PERF: block_index_deserialize %" PRId64 "ms blocks=%d\n",
|
||||
GetTimeMillis() - nPhaseStart, nBlocksLoaded);
|
||||
|
||||
if (fRequestShutdown)
|
||||
return true;
|
||||
|
||||
nPhaseStart = GetTimeMillis();
|
||||
bool fNeedChainTrustRecalc = !CDiskBlockIndex::fSerializeChainTrust;
|
||||
|
||||
if (fNeedChainTrustRecalc)
|
||||
{
|
||||
uiInterface.InitMessage(_("Calculating chain trust (one-time upgrade)..."));
|
||||
|
||||
vector<pair<int, CBlockIndex*> > vSortedByHeight;
|
||||
vSortedByHeight.reserve(mapBlockIndex.size());
|
||||
for (const auto& item : mapBlockIndex)
|
||||
vSortedByHeight.push_back(make_pair(item.second->nHeight, item.second));
|
||||
sort(vSortedByHeight.begin(), vSortedByHeight.end());
|
||||
|
||||
int nLastCheckpointHeight = Checkpoints::GetTotalBlocksEstimate();
|
||||
int nProgressInterval = std::max((int)vSortedByHeight.size() / 20, 1);
|
||||
int nCount = 0;
|
||||
|
||||
for (const auto& item : vSortedByHeight)
|
||||
{
|
||||
CBlockIndex* pindex = item.second;
|
||||
pindex->nChainTrust = (pindex->pprev ? pindex->pprev->nChainTrust : 0)
|
||||
+ pindex->GetBlockTrust();
|
||||
|
||||
if (pindex->nHeight >= nLastCheckpointHeight)
|
||||
{
|
||||
pindex->nStakeModifierChecksum = GetStakeModifierChecksum(pindex);
|
||||
if (!CheckStakeModifierCheckpoints(pindex->nHeight, pindex->nStakeModifierChecksum))
|
||||
return error("LoadBlockIndex(): Failed stake modifier checkpoint h=%d, mod=0x%016"PRIx64,
|
||||
pindex->nHeight, pindex->nStakeModifier);
|
||||
}
|
||||
|
||||
if (++nCount % nProgressInterval == 0)
|
||||
{
|
||||
std::string strMsg = strprintf(_("Calculating chain trust... (%d%%)"),
|
||||
nCount * 100 / vSortedByHeight.size());
|
||||
uiInterface.InitMessage(strMsg);
|
||||
}
|
||||
}
|
||||
|
||||
printf("LoadBlockIndex(): upgrading RocksDB to format v3...\n");
|
||||
uiInterface.InitMessage(_("Upgrading block index..."));
|
||||
CDiskBlockIndex::fSerializeChainTrust = true;
|
||||
|
||||
rocksdb::WriteBatch batch;
|
||||
nCount = 0;
|
||||
for (const auto& item : vSortedByHeight)
|
||||
{
|
||||
CBlockIndex* pindex = item.second;
|
||||
CDiskBlockIndex diskindex(pindex);
|
||||
|
||||
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
|
||||
ssKey << make_pair(string("blockindex"), *pindex->phashBlock);
|
||||
CDataStream ssValue(SER_DISK, CLIENT_VERSION);
|
||||
ssValue << diskindex;
|
||||
batch.Put(ssKey.str(), ssValue.str());
|
||||
|
||||
if (++nCount % 100000 == 0)
|
||||
{
|
||||
pdb->Write(rocksdb::WriteOptions(), &batch);
|
||||
batch.Clear();
|
||||
printf("LoadBlockIndex(): upgraded %d / %d entries\n",
|
||||
nCount, (int)vSortedByHeight.size());
|
||||
}
|
||||
}
|
||||
CDataStream ssFmtKey(SER_DISK, CLIENT_VERSION);
|
||||
ssFmtKey << string("dbformat");
|
||||
CDataStream ssFmtValue(SER_DISK, CLIENT_VERSION);
|
||||
ssFmtValue << (int)3;
|
||||
batch.Put(ssFmtKey.str(), ssFmtValue.str());
|
||||
|
||||
rocksdb::Status status = pdb->Write(rocksdb::WriteOptions(), &batch);
|
||||
if (!status.ok())
|
||||
return error("LoadBlockIndex(): failed to write upgraded block index: %s",
|
||||
status.ToString().c_str());
|
||||
|
||||
printf("LoadBlockIndex(): RocksDB upgraded to format v3 (%d entries)\n", nCount);
|
||||
}
|
||||
else
|
||||
{
|
||||
int nLastCheckpointHeight = Checkpoints::GetTotalBlocksEstimate();
|
||||
bool fNeedModifierCheck = false;
|
||||
for (const auto& item : mapBlockIndex)
|
||||
{
|
||||
if (item.second->nHeight >= nLastCheckpointHeight)
|
||||
{
|
||||
fNeedModifierCheck = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (fNeedModifierCheck)
|
||||
{
|
||||
vector<pair<int, CBlockIndex*> > vAboveCheckpoint;
|
||||
for (const auto& item : mapBlockIndex)
|
||||
if (item.second->nHeight >= nLastCheckpointHeight)
|
||||
vAboveCheckpoint.push_back(make_pair(item.second->nHeight, item.second));
|
||||
sort(vAboveCheckpoint.begin(), vAboveCheckpoint.end());
|
||||
|
||||
for (const auto& item : vAboveCheckpoint)
|
||||
{
|
||||
CBlockIndex* pindex = item.second;
|
||||
pindex->nStakeModifierChecksum = GetStakeModifierChecksum(pindex);
|
||||
if (!CheckStakeModifierCheckpoints(pindex->nHeight, pindex->nStakeModifierChecksum))
|
||||
return error("LoadBlockIndex(): Failed stake modifier checkpoint h=%d, mod=0x%016"PRIx64,
|
||||
pindex->nHeight, pindex->nStakeModifier);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
printf("STARTUP-PERF: chain_trust_and_modifiers %" PRId64 "ms\n",
|
||||
GetTimeMillis() - nPhaseStart);
|
||||
|
||||
if (nDbFormat < 3)
|
||||
{
|
||||
WriteDbFormat(3);
|
||||
printf("LoadBlockIndex(): bumped RocksDB dbformat to v3\n");
|
||||
}
|
||||
|
||||
nPhaseStart = GetTimeMillis();
|
||||
if (!ReadHashBestChain(hashBestChain))
|
||||
{
|
||||
if (pindexGenesisBlock == nullptr)
|
||||
return true;
|
||||
return error("LoadBlockIndex(): hashBestChain not loaded");
|
||||
}
|
||||
if (!mapBlockIndex.count(hashBestChain))
|
||||
return error("LoadBlockIndex(): hashBestChain not found in the block index");
|
||||
pindexBest = mapBlockIndex[hashBestChain];
|
||||
nBestHeight = pindexBest->nHeight;
|
||||
nBestChainTrust = pindexBest->nChainTrust;
|
||||
|
||||
printf("STARTUP-PERF: best_chain %" PRId64 "ms\n", GetTimeMillis() - nPhaseStart);
|
||||
|
||||
nPhaseStart = GetTimeMillis();
|
||||
{
|
||||
int nStakeSeenDepth = 500;
|
||||
CBlockIndex* pindex = pindexBest;
|
||||
int nLoaded = 0;
|
||||
while (pindex && nLoaded < nStakeSeenDepth)
|
||||
{
|
||||
if (pindex->IsProofOfStake())
|
||||
setStakeSeen.insert(make_pair(pindex->prevoutStake, pindex->nStakeTime));
|
||||
pindex = pindex->pprev;
|
||||
nLoaded++;
|
||||
}
|
||||
printf("LoadBlockIndex(): populated setStakeSeen with %d entries (last %d blocks)\n",
|
||||
(int)setStakeSeen.size(), nLoaded);
|
||||
}
|
||||
printf("STARTUP-PERF: stake_seen %" PRId64 "ms\n", GetTimeMillis() - nPhaseStart);
|
||||
|
||||
printf("LoadBlockIndex(): hashBestChain=%s height=%d trust=%s date=%s\n",
|
||||
hashBestChain.ToString().substr(0,20).c_str(), nBestHeight,
|
||||
CBigNum(nBestChainTrust).ToString().c_str(),
|
||||
DateTimeStrFormat("%x %H:%M:%S", pindexBest->GetBlockTime()).c_str());
|
||||
|
||||
{
|
||||
CBlockIndex* pindexBetter = nullptr;
|
||||
for (const auto& item : mapBlockIndex)
|
||||
{
|
||||
CBlockIndex* pindex = item.second;
|
||||
if (pindex == pindexBest)
|
||||
continue;
|
||||
if (pindex->nChainTrust > nBestChainTrust)
|
||||
{
|
||||
pindexBetter = pindex;
|
||||
break;
|
||||
}
|
||||
if (pindex->nChainTrust == nBestChainTrust &&
|
||||
pindex->GetBlockHash() < pindexBest->GetBlockHash())
|
||||
{
|
||||
if (!pindexBetter || pindex->GetBlockHash() < pindexBetter->GetBlockHash())
|
||||
pindexBetter = pindex;
|
||||
}
|
||||
}
|
||||
if (pindexBetter)
|
||||
{
|
||||
printf("LoadBlockIndex(): better chain tip %s at %d (trust %s vs %s)\n",
|
||||
pindexBetter->GetBlockHash().ToString().substr(0,20).c_str(),
|
||||
pindexBetter->nHeight,
|
||||
CBigNum(pindexBetter->nChainTrust).ToString().c_str(),
|
||||
CBigNum(nBestChainTrust).ToString().c_str());
|
||||
CBlock block;
|
||||
if (block.ReadFromDisk(pindexBetter))
|
||||
{
|
||||
CRocksTxDB txdb2;
|
||||
if (block.SetBestChain(txdb2, pindexBetter))
|
||||
{
|
||||
hashBestChain = pindexBetter->GetBlockHash();
|
||||
pindexBest = pindexBetter;
|
||||
nBestHeight = pindexBetter->nHeight;
|
||||
nBestChainTrust = pindexBetter->nChainTrust;
|
||||
printf("LoadBlockIndex(): switched to better chain tip\n");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!ReadSyncCheckpoint(Checkpoints::hashSyncCheckpoint))
|
||||
printf("LoadBlockIndex(): no sync checkpoint in DB, using default\n");
|
||||
else
|
||||
printf("LoadBlockIndex(): synchronized checkpoint %s\n",
|
||||
Checkpoints::hashSyncCheckpoint.ToString().c_str());
|
||||
if (!mapBlockIndex.count(Checkpoints::hashSyncCheckpoint))
|
||||
{
|
||||
printf("LoadBlockIndex(): sync checkpoint not in index, resetting to genesis\n");
|
||||
Checkpoints::hashSyncCheckpoint = (!fTestNet ? hashGenesisBlockOfficial
|
||||
: hashGenesisBlockTestNet);
|
||||
}
|
||||
|
||||
CBigNum bnBestInvalidTrust;
|
||||
ReadBestInvalidTrust(bnBestInvalidTrust);
|
||||
nBestInvalidTrust = bnBestInvalidTrust.getuint256();
|
||||
|
||||
nPhaseStart = GetTimeMillis();
|
||||
int nCheckLevel = GetArg("-checklevel", 1);
|
||||
int nCheckDepth = GetArg("-checkblocks", 50);
|
||||
if (nCheckDepth == 0)
|
||||
nCheckDepth = 1000000000;
|
||||
if (nCheckDepth > nBestHeight)
|
||||
nCheckDepth = nBestHeight;
|
||||
printf("Verifying last %i blocks at level %i\n", nCheckDepth, nCheckLevel);
|
||||
CBlockIndex* pindexFork = nullptr;
|
||||
map<pair<unsigned int, unsigned int>, CBlockIndex*> mapBlockPos;
|
||||
for (CBlockIndex* pindex = pindexBest; pindex && pindex->pprev; pindex = pindex->pprev)
|
||||
{
|
||||
if (fRequestShutdown || pindex->nHeight < nBestHeight - nCheckDepth)
|
||||
break;
|
||||
CBlock block;
|
||||
if (!block.ReadFromDisk(pindex))
|
||||
return error("LoadBlockIndex(): block.ReadFromDisk failed");
|
||||
if (nCheckLevel > 0 && !block.CheckBlock(true, true, (nCheckLevel > 6)))
|
||||
{
|
||||
printf("LoadBlockIndex(): bad block at %d, hash=%s\n",
|
||||
pindex->nHeight, pindex->GetBlockHash().ToString().c_str());
|
||||
pindexFork = pindex->pprev;
|
||||
}
|
||||
if (nCheckLevel > 1)
|
||||
{
|
||||
pair<unsigned int, unsigned int> pos = make_pair(pindex->nFile, pindex->nBlockPos);
|
||||
mapBlockPos[pos] = pindex;
|
||||
for (const CTransaction &tx : block.vtx)
|
||||
{
|
||||
uint256 hashTx = tx.GetHash();
|
||||
CTxIndex txindex;
|
||||
if (ReadTxIndex(hashTx, txindex))
|
||||
{
|
||||
if (nCheckLevel > 2 || pindex->nFile != txindex.pos.nFile
|
||||
|| pindex->nBlockPos != txindex.pos.nBlockPos)
|
||||
{
|
||||
CTransaction txFound;
|
||||
if (!txFound.ReadFromDisk(txindex.pos))
|
||||
{
|
||||
printf("LoadBlockIndex(): cannot read mislocated transaction %s\n",
|
||||
hashTx.ToString().c_str());
|
||||
pindexFork = pindex->pprev;
|
||||
}
|
||||
else if (txFound.GetHash() != hashTx)
|
||||
{
|
||||
printf("LoadBlockIndex(): invalid tx position for %s\n",
|
||||
hashTx.ToString().c_str());
|
||||
pindexFork = pindex->pprev;
|
||||
}
|
||||
}
|
||||
if (nCheckLevel > 3 && !tx.IsCoinBase())
|
||||
{
|
||||
for (const CTxIn &txin : tx.vin)
|
||||
{
|
||||
if (HaveUtxo(txin.prevout.hash, txin.prevout.n))
|
||||
{
|
||||
printf("LoadBlockIndex(): spent input still in UTXO set: %s:%i in %s\n",
|
||||
txin.prevout.hash.ToString().c_str(), txin.prevout.n,
|
||||
hashTx.ToString().c_str());
|
||||
pindexFork = pindex->pprev;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if (pindexFork && !fRequestShutdown)
|
||||
{
|
||||
printf("LoadBlockIndex(): moving best chain pointer back to block %d\n",
|
||||
pindexFork->nHeight);
|
||||
CBlock block;
|
||||
if (!block.ReadFromDisk(pindexFork))
|
||||
return error("LoadBlockIndex(): block.ReadFromDisk failed");
|
||||
CRocksTxDB txdb;
|
||||
block.SetBestChain(txdb, pindexFork);
|
||||
}
|
||||
printf("STARTUP-PERF: verify_blocks %" PRId64 "ms depth=%d level=%d\n",
|
||||
GetTimeMillis() - nPhaseStart, nCheckDepth, nCheckLevel);
|
||||
printf("STARTUP-PERF: load_block_index_total %" PRId64 "ms\n",
|
||||
GetTimeMillis() - nTotalStart);
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
#endif // BUILD_ROCKSDB
|
||||
@@ -0,0 +1,57 @@
|
||||
// Copyright (c) 2026 The Triangles developers.
|
||||
// Distributed under the MIT/X11 software license, see the accompanying
|
||||
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
|
||||
#ifndef TRIANGLES_TXDB_ROCKSDB_H
|
||||
#define TRIANGLES_TXDB_ROCKSDB_H
|
||||
|
||||
#ifdef BUILD_ROCKSDB
|
||||
|
||||
#include "txdb-base.h"
|
||||
|
||||
#include <rocksdb/db.h>
|
||||
#include <rocksdb/options.h>
|
||||
#include <rocksdb/write_batch.h>
|
||||
|
||||
// RocksDB backend for the chain database.
|
||||
//
|
||||
// Mirrors CTxDB (LevelDB) for byte-level compatibility. CTxDBBase owns all
|
||||
// key serialization, so keys produced by this backend are bit-identical to
|
||||
// the LevelDB backend. That property is what lets the M1.4 dual-backend
|
||||
// parity harness verify equivalence.
|
||||
//
|
||||
// Data lives under <datadir>/rocksdb/, separate from <datadir>/txleveldb/,
|
||||
// so both backends can coexist for migration and side-by-side testing.
|
||||
class CRocksTxDB final : public CTxDBBase
|
||||
{
|
||||
public:
|
||||
CRocksTxDB(const char* pszMode = "r+");
|
||||
~CRocksTxDB() override;
|
||||
|
||||
void Close() override;
|
||||
|
||||
bool TxnBegin() override;
|
||||
bool TxnCommit() override;
|
||||
bool TxnAbort() override;
|
||||
|
||||
bool LoadBlockIndex() override;
|
||||
|
||||
protected:
|
||||
bool ReadRaw(const std::string& key, std::string& value) const override;
|
||||
bool WriteRaw(const std::string& key, const std::string& value) override;
|
||||
bool EraseRaw(const std::string& key) override;
|
||||
bool ExistsRaw(const std::string& key) const override;
|
||||
std::unique_ptr<CTxDBIteratorBase> NewIterator() const override;
|
||||
|
||||
private:
|
||||
rocksdb::DB* pdb; // Points to the global instance.
|
||||
rocksdb::WriteBatch* activeBatch; // When non-NULL, writes/deletes go here.
|
||||
rocksdb::Options options;
|
||||
int nVersion;
|
||||
|
||||
bool ScanBatch(const std::string& key, std::string* value, bool* deleted) const;
|
||||
};
|
||||
|
||||
#endif // BUILD_ROCKSDB
|
||||
|
||||
#endif // TRIANGLES_TXDB_ROCKSDB_H
|
||||
@@ -0,0 +1,59 @@
|
||||
// Copyright (c) 2026 The Triangles developers.
|
||||
// Distributed under the MIT/X11 software license.
|
||||
|
||||
#include "txdb.h"
|
||||
|
||||
#include "util.h"
|
||||
|
||||
bool UseRocksDbBackend()
|
||||
{
|
||||
#ifdef BUILD_ROCKSDB
|
||||
return GetBoolArg("-rocksdb", false);
|
||||
#else
|
||||
return false;
|
||||
#endif
|
||||
}
|
||||
|
||||
const char* GetActiveChainDbBackendName()
|
||||
{
|
||||
#ifdef BUILD_ROCKSDB
|
||||
return UseRocksDbBackend() ? "rocksdb" : "leveldb";
|
||||
#else
|
||||
return "leveldb";
|
||||
#endif
|
||||
}
|
||||
|
||||
const char* GetActiveChainDbDirName()
|
||||
{
|
||||
#ifdef BUILD_ROCKSDB
|
||||
return UseRocksDbBackend() ? "rocksdb" : "txleveldb";
|
||||
#else
|
||||
return "txleveldb";
|
||||
#endif
|
||||
}
|
||||
|
||||
CActiveTxDB::CActiveTxDB(const char* pszMode)
|
||||
{
|
||||
#ifdef BUILD_ROCKSDB
|
||||
if (UseRocksDbBackend())
|
||||
impl.reset(new CRocksTxDB(pszMode));
|
||||
else
|
||||
impl.reset(new CTxDB(pszMode));
|
||||
#else
|
||||
impl.reset(new CTxDB(pszMode));
|
||||
#endif
|
||||
}
|
||||
|
||||
CActiveTxDB::~CActiveTxDB() = default;
|
||||
|
||||
void CActiveTxDB::Close() { impl->Close(); }
|
||||
bool CActiveTxDB::TxnBegin() { return impl->TxnBegin(); }
|
||||
bool CActiveTxDB::TxnCommit() { return impl->TxnCommit(); }
|
||||
bool CActiveTxDB::TxnAbort() { return impl->TxnAbort(); }
|
||||
bool CActiveTxDB::LoadBlockIndex() { return impl->LoadBlockIndex(); }
|
||||
|
||||
bool CActiveTxDB::ReadRaw(const std::string& key, std::string& value) const { return impl->ReadRawBytes(key, value); }
|
||||
bool CActiveTxDB::WriteRaw(const std::string& key, const std::string& value) { return impl->WriteRawBytes(key, value); }
|
||||
bool CActiveTxDB::EraseRaw(const std::string& key) { return impl->EraseRawBytes(key); }
|
||||
bool CActiveTxDB::ExistsRaw(const std::string& key) const { return impl->ExistsRawBytes(key); }
|
||||
std::unique_ptr<CTxDBIteratorBase> CActiveTxDB::NewIterator() const { return impl->NewRawIterator(); }
|
||||
+32
@@ -7,5 +7,37 @@
|
||||
#define TRIANGLES_TXDB_H
|
||||
|
||||
#include "txdb-leveldb.h"
|
||||
#ifdef BUILD_ROCKSDB
|
||||
#include "txdb-rocksdb.h"
|
||||
#endif
|
||||
|
||||
#include <memory>
|
||||
|
||||
bool UseRocksDbBackend();
|
||||
const char* GetActiveChainDbBackendName();
|
||||
const char* GetActiveChainDbDirName();
|
||||
|
||||
class CActiveTxDB final : public CTxDBBase
|
||||
{
|
||||
public:
|
||||
explicit CActiveTxDB(const char* pszMode = "r+");
|
||||
~CActiveTxDB() override;
|
||||
|
||||
void Close() override;
|
||||
bool TxnBegin() override;
|
||||
bool TxnCommit() override;
|
||||
bool TxnAbort() override;
|
||||
bool LoadBlockIndex() override;
|
||||
|
||||
protected:
|
||||
bool ReadRaw(const std::string& key, std::string& value) const override;
|
||||
bool WriteRaw(const std::string& key, const std::string& value) override;
|
||||
bool EraseRaw(const std::string& key) override;
|
||||
bool ExistsRaw(const std::string& key) const override;
|
||||
std::unique_ptr<CTxDBIteratorBase> NewIterator() const override;
|
||||
|
||||
private:
|
||||
std::unique_ptr<CTxDBBase> impl;
|
||||
};
|
||||
|
||||
#endif // TRIANGLES_TXDB_H
|
||||
|
||||
+2
-1
@@ -111,8 +111,9 @@ public:
|
||||
CRYPTO_set_locking_callback(locking_callback);
|
||||
#endif
|
||||
|
||||
#ifdef WIN32
|
||||
#if defined(WIN32) && OPENSSL_VERSION_NUMBER < 0x30000000L
|
||||
// Seed random number generator with screen scrape and other hardware sources
|
||||
// (removed in OpenSSL 3.x — auto-seeded via BCryptGenRandom)
|
||||
RAND_screen();
|
||||
#endif
|
||||
|
||||
|
||||
@@ -0,0 +1,562 @@
|
||||
// Copyright (c) 2024-2025 Triangles developers
|
||||
// Distributed under the MIT/X11 software license
|
||||
|
||||
#include "utxosnapshot.h"
|
||||
|
||||
#include "main.h"
|
||||
#include "txdb.h"
|
||||
#include "checkpoints.h"
|
||||
#include "util.h"
|
||||
#include "ui_interface.h"
|
||||
|
||||
#include <boost/filesystem.hpp>
|
||||
#include <boost/filesystem/fstream.hpp>
|
||||
|
||||
#include <leveldb/db.h>
|
||||
#include <leveldb/write_batch.h>
|
||||
#include <leveldb/cache.h>
|
||||
#include <leveldb/filter_policy.h>
|
||||
#ifdef BUILD_ROCKSDB
|
||||
#include <rocksdb/db.h>
|
||||
#include <rocksdb/write_batch.h>
|
||||
#include <rocksdb/cache.h>
|
||||
#include <rocksdb/filter_policy.h>
|
||||
#include <rocksdb/table.h>
|
||||
#endif
|
||||
|
||||
#include <openssl/sha.h>
|
||||
|
||||
#include <vector>
|
||||
#include <algorithm>
|
||||
#include <cstdio>
|
||||
|
||||
namespace fs = boost::filesystem;
|
||||
|
||||
// Global LevelDB pointer (defined in txdb-leveldb.cpp)
|
||||
extern leveldb::DB *txdb;
|
||||
|
||||
namespace UtxoSnapshot {
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DumpSnapshot - create a UTXO snapshot from the current chain state
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
bool DumpSnapshot(const fs::path& destPath,
|
||||
unsigned int nHeaders,
|
||||
std::string& strError)
|
||||
{
|
||||
LOCK(cs_main);
|
||||
|
||||
if (!pindexBest) {
|
||||
strError = "No best block - chain not loaded";
|
||||
return false;
|
||||
}
|
||||
|
||||
// Collect block index entries (last nHeaders blocks, height ascending)
|
||||
std::vector<std::pair<uint256, CDiskBlockIndex>> vHeaders;
|
||||
vHeaders.reserve(nHeaders);
|
||||
{
|
||||
CBlockIndex* pindex = pindexBest;
|
||||
unsigned int nCollected = 0;
|
||||
while (pindex && nCollected < nHeaders) {
|
||||
CDiskBlockIndex diskindex(pindex);
|
||||
vHeaders.push_back(std::make_pair(*pindex->phashBlock, diskindex));
|
||||
pindex = pindex->pprev;
|
||||
nCollected++;
|
||||
}
|
||||
// Reverse to height ascending order
|
||||
std::reverse(vHeaders.begin(), vHeaders.end());
|
||||
}
|
||||
|
||||
// Count UTXOs first
|
||||
int nUtxoCount = 0;
|
||||
{
|
||||
CActiveTxDB txdbRead("r");
|
||||
txdbRead.SumUtxoValues(nUtxoCount);
|
||||
}
|
||||
|
||||
if (nUtxoCount == 0) {
|
||||
strError = "No UTXOs found in database";
|
||||
return false;
|
||||
}
|
||||
|
||||
printf("UtxoSnapshot: dumping %d headers + %d UTXOs at height %d\n",
|
||||
(int)vHeaders.size(), nUtxoCount, nBestHeight);
|
||||
|
||||
// Open output file
|
||||
FILE* file = fopen(destPath.string().c_str(), "wb");
|
||||
if (!file) {
|
||||
strError = "Cannot create file: " + destPath.string();
|
||||
return false;
|
||||
}
|
||||
|
||||
// Write header (we'll seek back to fill in content_hash later)
|
||||
unsigned int magic = UTXO_SNAPSHOT_MAGIC;
|
||||
unsigned int version = UTXO_SNAPSHOT_VERSION;
|
||||
unsigned int network = fTestNet ? 2 : 1;
|
||||
int height = nBestHeight;
|
||||
uint256 blockHash = hashBestChain;
|
||||
int64_t moneySupply = pindexBest->nMoneySupply;
|
||||
unsigned int numHeaders = (unsigned int)vHeaders.size();
|
||||
unsigned int numUtxos = (unsigned int)nUtxoCount;
|
||||
uint256 contentHash; // placeholder, filled after writing data
|
||||
|
||||
fwrite(&magic, sizeof(magic), 1, file);
|
||||
fwrite(&version, sizeof(version), 1, file);
|
||||
fwrite(&network, sizeof(network), 1, file);
|
||||
fwrite(&height, sizeof(height), 1, file);
|
||||
fwrite(&blockHash, sizeof(blockHash), 1, file);
|
||||
fwrite(&moneySupply, sizeof(moneySupply), 1, file);
|
||||
fwrite(&numHeaders, sizeof(numHeaders), 1, file);
|
||||
fwrite(&numUtxos, sizeof(numUtxos), 1, file);
|
||||
long contentHashPos = ftell(file);
|
||||
fwrite(&contentHash, sizeof(contentHash), 1, file); // placeholder
|
||||
|
||||
// Start SHA256 for content hash
|
||||
SHA256_CTX sha256;
|
||||
SHA256_Init(&sha256);
|
||||
|
||||
// Write block headers section
|
||||
for (const auto& item : vHeaders) {
|
||||
CDataStream ssEntry(SER_DISK, CLIENT_VERSION);
|
||||
ssEntry << item.first; // block hash
|
||||
ssEntry << item.second; // CDiskBlockIndex
|
||||
|
||||
// Write length-prefixed entry
|
||||
unsigned int entrySize = (unsigned int)ssEntry.size();
|
||||
std::string strEntry = ssEntry.str();
|
||||
fwrite(&entrySize, sizeof(entrySize), 1, file);
|
||||
fwrite(strEntry.data(), 1, entrySize, file);
|
||||
|
||||
SHA256_Update(&sha256, &entrySize, sizeof(entrySize));
|
||||
SHA256_Update(&sha256, strEntry.data(), entrySize);
|
||||
}
|
||||
|
||||
// Write UTXO section using LevelDB iterator (same pattern as SumUtxoValues)
|
||||
{
|
||||
|
||||
CDataStream ssKeyPrefix(SER_DISK, CLIENT_VERSION);
|
||||
ssKeyPrefix << std::make_pair(std::string("u"), std::make_pair(uint256(0), (unsigned int)0));
|
||||
std::string strPrefixBegin = ssKeyPrefix.str();
|
||||
|
||||
leveldb::Iterator* it = txdb->NewIterator(leveldb::ReadOptions());
|
||||
unsigned int nWritten = 0;
|
||||
for (it->Seek(strPrefixBegin); it->Valid(); it->Next()) {
|
||||
// Check key prefix is still "u"
|
||||
CDataStream ssKey(it->key().data(), it->key().data() + it->key().size(), SER_DISK, CLIENT_VERSION);
|
||||
std::string strKeyType;
|
||||
ssKey >> strKeyType;
|
||||
if (strKeyType != "u")
|
||||
break;
|
||||
|
||||
// Extract outpoint from key
|
||||
uint256 txhash;
|
||||
unsigned int nIndex;
|
||||
ssKey >> txhash;
|
||||
ssKey >> nIndex;
|
||||
|
||||
// Extract UTXO entry from value
|
||||
CDataStream ssValue(it->value().data(), it->value().data() + it->value().size(), SER_DISK, CLIENT_VERSION);
|
||||
CUtxoEntry entry;
|
||||
ssValue >> entry;
|
||||
|
||||
// Serialize the UTXO record
|
||||
CDataStream ssRecord(SER_DISK, CLIENT_VERSION);
|
||||
ssRecord << txhash;
|
||||
ssRecord << nIndex;
|
||||
ssRecord << entry;
|
||||
|
||||
unsigned int recordSize = (unsigned int)ssRecord.size();
|
||||
std::string strRecord = ssRecord.str();
|
||||
fwrite(&recordSize, sizeof(recordSize), 1, file);
|
||||
fwrite(strRecord.data(), 1, recordSize, file);
|
||||
|
||||
SHA256_Update(&sha256, &recordSize, sizeof(recordSize));
|
||||
SHA256_Update(&sha256, strRecord.data(), recordSize);
|
||||
|
||||
nWritten++;
|
||||
if (nWritten % 10000 == 0)
|
||||
printf("UtxoSnapshot: wrote %d / %d UTXOs\n", nWritten, nUtxoCount);
|
||||
}
|
||||
delete it;
|
||||
|
||||
// Update actual count (in case it changed during iteration)
|
||||
if (nWritten != numUtxos) {
|
||||
numUtxos = nWritten;
|
||||
// Seek back and update numUtxos in header
|
||||
long currentPos = ftell(file);
|
||||
fseek(file, contentHashPos - sizeof(numUtxos), SEEK_SET);
|
||||
fwrite(&numUtxos, sizeof(numUtxos), 1, file);
|
||||
fseek(file, currentPos, SEEK_SET);
|
||||
}
|
||||
}
|
||||
|
||||
// Finalize content hash and write it to the header
|
||||
SHA256_Final((unsigned char*)&contentHash, &sha256);
|
||||
fseek(file, contentHashPos, SEEK_SET);
|
||||
fwrite(&contentHash, sizeof(contentHash), 1, file);
|
||||
|
||||
fclose(file);
|
||||
|
||||
printf("UtxoSnapshot: wrote %s (%d headers, %d UTXOs, hash=%s)\n",
|
||||
destPath.string().c_str(), numHeaders, numUtxos,
|
||||
contentHash.ToString().c_str());
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// LoadSnapshot - load a UTXO snapshot into a fresh LevelDB
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
bool LoadSnapshot(const fs::path& snapshotPath,
|
||||
const fs::path& dataDir,
|
||||
std::string& strError)
|
||||
{
|
||||
FILE* file = fopen(snapshotPath.string().c_str(), "rb");
|
||||
if (!file) {
|
||||
strError = "Cannot open snapshot file: " + snapshotPath.string();
|
||||
return false;
|
||||
}
|
||||
|
||||
// Read header
|
||||
unsigned int magic, version, network;
|
||||
int height;
|
||||
uint256 blockHash;
|
||||
int64_t moneySupply;
|
||||
unsigned int numHeaders, numUtxos;
|
||||
uint256 expectedContentHash;
|
||||
|
||||
if (fread(&magic, sizeof(magic), 1, file) != 1 ||
|
||||
fread(&version, sizeof(version), 1, file) != 1 ||
|
||||
fread(&network, sizeof(network), 1, file) != 1 ||
|
||||
fread(&height, sizeof(height), 1, file) != 1 ||
|
||||
fread(&blockHash, sizeof(blockHash), 1, file) != 1 ||
|
||||
fread(&moneySupply, sizeof(moneySupply), 1, file) != 1 ||
|
||||
fread(&numHeaders, sizeof(numHeaders), 1, file) != 1 ||
|
||||
fread(&numUtxos, sizeof(numUtxos), 1, file) != 1 ||
|
||||
fread(&expectedContentHash, sizeof(expectedContentHash), 1, file) != 1) {
|
||||
fclose(file);
|
||||
strError = "Truncated snapshot header";
|
||||
return false;
|
||||
}
|
||||
|
||||
// Validate header
|
||||
if (magic != UTXO_SNAPSHOT_MAGIC) {
|
||||
fclose(file);
|
||||
strError = "Invalid snapshot magic (not a UTXO snapshot file)";
|
||||
return false;
|
||||
}
|
||||
|
||||
if (version != UTXO_SNAPSHOT_VERSION) {
|
||||
fclose(file);
|
||||
strError = "Unsupported snapshot version: " + std::to_string(version);
|
||||
return false;
|
||||
}
|
||||
|
||||
unsigned int expectedNetwork = fTestNet ? 2 : 1;
|
||||
if (network != expectedNetwork) {
|
||||
fclose(file);
|
||||
strError = "Network mismatch: snapshot is " + std::string(network == 1 ? "mainnet" : "testnet");
|
||||
return false;
|
||||
}
|
||||
|
||||
if (numHeaders == 0 || numUtxos == 0) {
|
||||
fclose(file);
|
||||
strError = "Snapshot contains no data";
|
||||
return false;
|
||||
}
|
||||
|
||||
// Verify snapshot block is a known checkpoint
|
||||
if (!Checkpoints::IsKnownCheckpoint(height, blockHash)) {
|
||||
fclose(file);
|
||||
strError = "Snapshot block " + blockHash.ToString() + " at height "
|
||||
+ std::to_string(height) + " is not a known checkpoint";
|
||||
return false;
|
||||
}
|
||||
|
||||
printf("UtxoSnapshot: loading snapshot at height %d (%d headers, %d UTXOs)\n",
|
||||
height, numHeaders, numUtxos);
|
||||
|
||||
// Create fresh chain DB directory for the active backend.
|
||||
const bool useRocksDb = UseRocksDbBackend();
|
||||
fs::path chainDbPath = dataDir / GetActiveChainDbDirName();
|
||||
if (fs::exists(chainDbPath))
|
||||
fs::remove_all(chainDbPath);
|
||||
fs::create_directories(chainDbPath);
|
||||
|
||||
int nCacheSizeMB = GetArg("-dbcache", 2048);
|
||||
leveldb::DB* pdb = NULL;
|
||||
leveldb::Options options;
|
||||
options.block_cache = NULL;
|
||||
options.filter_policy = NULL;
|
||||
#ifdef BUILD_ROCKSDB
|
||||
rocksdb::DB* rdb = NULL;
|
||||
std::shared_ptr<rocksdb::Cache> rocksCache;
|
||||
rocksdb::BlockBasedTableOptions rocksTableOptions;
|
||||
#endif
|
||||
|
||||
if (useRocksDb) {
|
||||
#ifdef BUILD_ROCKSDB
|
||||
rocksdb::Options rocksOptions;
|
||||
rocksOptions.create_if_missing = true;
|
||||
rocksOptions.compression = rocksdb::kSnappyCompression;
|
||||
rocksOptions.write_buffer_size = 64 * 1048576;
|
||||
rocksOptions.max_open_files = 1000;
|
||||
rocksCache = rocksdb::NewLRUCache(static_cast<size_t>(nCacheSizeMB) * 1048576);
|
||||
rocksTableOptions.block_cache = rocksCache;
|
||||
rocksTableOptions.filter_policy.reset(rocksdb::NewBloomFilterPolicy(10, false));
|
||||
rocksOptions.table_factory.reset(rocksdb::NewBlockBasedTableFactory(rocksTableOptions));
|
||||
rocksdb::Status rocksStatus = rocksdb::DB::Open(rocksOptions, chainDbPath.string(), &rdb);
|
||||
if (!rocksStatus.ok()) {
|
||||
fclose(file);
|
||||
strError = "Cannot create RocksDB: " + rocksStatus.ToString();
|
||||
return false;
|
||||
}
|
||||
#else
|
||||
fclose(file);
|
||||
strError = "RocksDB snapshot load requested but binary was built without RocksDB";
|
||||
return false;
|
||||
#endif
|
||||
} else {
|
||||
options.block_cache = leveldb::NewLRUCache(nCacheSizeMB * 1048576);
|
||||
options.filter_policy = leveldb::NewBloomFilterPolicy(10);
|
||||
options.write_buffer_size = 64 * 1048576;
|
||||
options.max_open_files = 1000;
|
||||
options.create_if_missing = true;
|
||||
|
||||
leveldb::Status status = leveldb::DB::Open(options, chainDbPath.string(), &pdb);
|
||||
if (!status.ok()) {
|
||||
fclose(file);
|
||||
delete options.filter_policy;
|
||||
delete options.block_cache;
|
||||
strError = "Cannot create LevelDB: " + status.ToString();
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
SHA256_CTX sha256;
|
||||
SHA256_Init(&sha256);
|
||||
|
||||
bool success = true;
|
||||
unsigned int nBatchSize = 0;
|
||||
leveldb::WriteBatch batch;
|
||||
#ifdef BUILD_ROCKSDB
|
||||
rocksdb::WriteBatch rocksBatch;
|
||||
#endif
|
||||
|
||||
auto batchPut = [&](const std::string& key, const std::string& value) {
|
||||
if (useRocksDb) {
|
||||
#ifdef BUILD_ROCKSDB
|
||||
rocksBatch.Put(key, value);
|
||||
#endif
|
||||
} else {
|
||||
batch.Put(key, value);
|
||||
}
|
||||
nBatchSize++;
|
||||
};
|
||||
|
||||
auto flushBatch = [&]() -> bool {
|
||||
if (nBatchSize == 0)
|
||||
return true;
|
||||
if (useRocksDb) {
|
||||
#ifdef BUILD_ROCKSDB
|
||||
rocksdb::Status s = rdb->Write(rocksdb::WriteOptions(), &rocksBatch);
|
||||
if (!s.ok()) {
|
||||
strError = "RocksDB write failed: " + s.ToString();
|
||||
return false;
|
||||
}
|
||||
rocksBatch.Clear();
|
||||
#endif
|
||||
} else {
|
||||
leveldb::Status s = pdb->Write(leveldb::WriteOptions(), &batch);
|
||||
if (!s.ok()) {
|
||||
strError = "LevelDB write failed: " + s.ToString();
|
||||
return false;
|
||||
}
|
||||
batch.Clear();
|
||||
}
|
||||
nBatchSize = 0;
|
||||
return true;
|
||||
};
|
||||
|
||||
// Read and write block headers
|
||||
printf("UtxoSnapshot: loading %d block headers...\n", numHeaders);
|
||||
uiInterface.InitMessage(_("Loading UTXO snapshot (headers)..."));
|
||||
|
||||
for (unsigned int i = 0; i < numHeaders; i++) {
|
||||
unsigned int entrySize;
|
||||
if (fread(&entrySize, sizeof(entrySize), 1, file) != 1 || entrySize > 10000) {
|
||||
success = false;
|
||||
strError = "Invalid header entry size at index " + std::to_string(i);
|
||||
break;
|
||||
}
|
||||
|
||||
std::vector<char> buf(entrySize);
|
||||
if (fread(buf.data(), 1, entrySize, file) != entrySize) {
|
||||
success = false;
|
||||
strError = "Truncated header entry at index " + std::to_string(i);
|
||||
break;
|
||||
}
|
||||
|
||||
SHA256_Update(&sha256, &entrySize, sizeof(entrySize));
|
||||
SHA256_Update(&sha256, buf.data(), entrySize);
|
||||
|
||||
// Parse: block_hash + CDiskBlockIndex
|
||||
CDataStream ssEntry(buf.data(), buf.data() + buf.size(), SER_DISK, CLIENT_VERSION);
|
||||
uint256 entryHash;
|
||||
CDiskBlockIndex diskindex;
|
||||
ssEntry >> entryHash;
|
||||
ssEntry >> diskindex;
|
||||
|
||||
// Write to LevelDB as "blockindex" key
|
||||
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
|
||||
ssKey << std::make_pair(std::string("blockindex"), entryHash);
|
||||
|
||||
CDataStream ssValue(SER_DISK, CLIENT_VERSION);
|
||||
ssValue << diskindex;
|
||||
|
||||
batchPut(ssKey.str(), ssValue.str());
|
||||
|
||||
if (nBatchSize >= 1000) {
|
||||
if (!flushBatch()) { success = false; break; }
|
||||
}
|
||||
}
|
||||
|
||||
if (success && !flushBatch())
|
||||
success = false;
|
||||
|
||||
// Read and write UTXOs
|
||||
if (success) {
|
||||
printf("UtxoSnapshot: loading %d UTXOs...\n", numUtxos);
|
||||
|
||||
for (unsigned int i = 0; i < numUtxos; i++) {
|
||||
unsigned int recordSize;
|
||||
if (fread(&recordSize, sizeof(recordSize), 1, file) != 1 || recordSize > 100000) {
|
||||
success = false;
|
||||
strError = "Invalid UTXO record size at index " + std::to_string(i);
|
||||
break;
|
||||
}
|
||||
|
||||
std::vector<char> buf(recordSize);
|
||||
if (fread(buf.data(), 1, recordSize, file) != recordSize) {
|
||||
success = false;
|
||||
strError = "Truncated UTXO record at index " + std::to_string(i);
|
||||
break;
|
||||
}
|
||||
|
||||
SHA256_Update(&sha256, &recordSize, sizeof(recordSize));
|
||||
SHA256_Update(&sha256, buf.data(), recordSize);
|
||||
|
||||
// Parse: txid + output_index + CUtxoEntry
|
||||
CDataStream ssRecord(buf.data(), buf.data() + buf.size(), SER_DISK, CLIENT_VERSION);
|
||||
uint256 txhash;
|
||||
unsigned int nIndex;
|
||||
CUtxoEntry entry;
|
||||
ssRecord >> txhash;
|
||||
ssRecord >> nIndex;
|
||||
ssRecord >> entry;
|
||||
|
||||
// Write to LevelDB with "u" prefix key
|
||||
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
|
||||
ssKey << std::make_pair(std::string("u"), std::make_pair(txhash, nIndex));
|
||||
|
||||
CDataStream ssValue(SER_DISK, CLIENT_VERSION);
|
||||
ssValue << entry;
|
||||
|
||||
batchPut(ssKey.str(), ssValue.str());
|
||||
|
||||
if (nBatchSize >= 50000) {
|
||||
if (!flushBatch()) { success = false; break; }
|
||||
|
||||
if (i % 50000 == 0) {
|
||||
std::string strMsg = strprintf(_("Loading UTXO snapshot (%d%%)..."),
|
||||
i * 100 / numUtxos);
|
||||
uiInterface.InitMessage(strMsg);
|
||||
printf("UtxoSnapshot: loaded %d / %d UTXOs\n", i, numUtxos);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (success && !flushBatch())
|
||||
success = false;
|
||||
}
|
||||
|
||||
// Verify content hash
|
||||
if (success) {
|
||||
uint256 actualHash;
|
||||
SHA256_Final((unsigned char*)&actualHash, &sha256);
|
||||
|
||||
if (actualHash != expectedContentHash) {
|
||||
success = false;
|
||||
strError = "Content hash mismatch - snapshot may be corrupted";
|
||||
}
|
||||
}
|
||||
|
||||
// Write metadata
|
||||
if (success) {
|
||||
CDataStream ssKey1(SER_DISK, CLIENT_VERSION);
|
||||
ssKey1 << std::string("hashBestChain");
|
||||
CDataStream ssVal1(SER_DISK, CLIENT_VERSION);
|
||||
ssVal1 << blockHash;
|
||||
|
||||
CDataStream ssKey2(SER_DISK, CLIENT_VERSION);
|
||||
ssKey2 << std::string("dbformat");
|
||||
CDataStream ssVal2(SER_DISK, CLIENT_VERSION);
|
||||
ssVal2 << (int)3;
|
||||
|
||||
CDataStream ssKey3(SER_DISK, CLIENT_VERSION);
|
||||
ssKey3 << std::string("version");
|
||||
CDataStream ssVal3(SER_DISK, CLIENT_VERSION);
|
||||
ssVal3 << DATABASE_VERSION;
|
||||
|
||||
if (useRocksDb) {
|
||||
#ifdef BUILD_ROCKSDB
|
||||
rocksdb::WriteBatch metaBatch;
|
||||
metaBatch.Put(ssKey1.str(), ssVal1.str());
|
||||
metaBatch.Put(ssKey2.str(), ssVal2.str());
|
||||
metaBatch.Put(ssKey3.str(), ssVal3.str());
|
||||
rocksdb::Status s = rdb->Write(rocksdb::WriteOptions(), &metaBatch);
|
||||
if (!s.ok()) {
|
||||
success = false;
|
||||
strError = "Failed to write RocksDB metadata: " + s.ToString();
|
||||
}
|
||||
#endif
|
||||
} else {
|
||||
leveldb::WriteBatch metaBatch;
|
||||
metaBatch.Put(ssKey1.str(), ssVal1.str());
|
||||
metaBatch.Put(ssKey2.str(), ssVal2.str());
|
||||
metaBatch.Put(ssKey3.str(), ssVal3.str());
|
||||
leveldb::Status s = pdb->Write(leveldb::WriteOptions(), &metaBatch);
|
||||
if (!s.ok()) {
|
||||
success = false;
|
||||
strError = "Failed to write metadata: " + s.ToString();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Clean up chain DB
|
||||
#ifdef BUILD_ROCKSDB
|
||||
delete rdb;
|
||||
#endif
|
||||
delete pdb;
|
||||
delete options.filter_policy;
|
||||
delete options.block_cache;
|
||||
|
||||
fclose(file);
|
||||
|
||||
if (!success) {
|
||||
// Remove corrupted/incomplete database
|
||||
printf("UtxoSnapshot: load failed: %s\n", strError.c_str());
|
||||
if (fs::exists(chainDbPath))
|
||||
fs::remove_all(chainDbPath);
|
||||
return false;
|
||||
}
|
||||
|
||||
printf("UtxoSnapshot: successfully loaded %d headers + %d UTXOs at height %d\n",
|
||||
numHeaders, numUtxos, height);
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace UtxoSnapshot
|
||||
@@ -0,0 +1,39 @@
|
||||
// Copyright (c) 2024-2025 Triangles developers
|
||||
// Distributed under the MIT/X11 software license
|
||||
|
||||
#ifndef TRIANGLES_UTXOSNAPSHOT_H
|
||||
#define TRIANGLES_UTXOSNAPSHOT_H
|
||||
|
||||
#include <string>
|
||||
#include <boost/filesystem.hpp>
|
||||
|
||||
// UTXO snapshot file magic bytes
|
||||
static const unsigned int UTXO_SNAPSHOT_MAGIC = 0x53585455; // "UTXS" little-endian
|
||||
|
||||
// UTXO snapshot format version
|
||||
static const unsigned int UTXO_SNAPSHOT_VERSION = 1;
|
||||
|
||||
// Number of block index entries to include in snapshot (covers difficulty,
|
||||
// median time, stake modifier, and reorg depth requirements)
|
||||
static const unsigned int UTXO_SNAPSHOT_DEFAULT_HEADERS = 2000;
|
||||
|
||||
namespace UtxoSnapshot {
|
||||
|
||||
// Create a UTXO snapshot from the current chain state.
|
||||
// Writes last nHeaders block index entries + all UTXOs to destPath.
|
||||
// Returns true on success, sets strError on failure.
|
||||
bool DumpSnapshot(const boost::filesystem::path& destPath,
|
||||
unsigned int nHeaders,
|
||||
std::string& strError);
|
||||
|
||||
// Load a UTXO snapshot from a file into a fresh LevelDB.
|
||||
// Writes block index entries, UTXOs, hashBestChain, and dbformat.
|
||||
// The LevelDB must NOT be open yet (call before LoadBlockIndex).
|
||||
// Returns true on success, sets strError on failure.
|
||||
bool LoadSnapshot(const boost::filesystem::path& snapshotPath,
|
||||
const boost::filesystem::path& dataDir,
|
||||
std::string& strError);
|
||||
|
||||
} // namespace UtxoSnapshot
|
||||
|
||||
#endif // TRIANGLES_UTXOSNAPSHOT_H
|
||||
+5
-1
@@ -30,11 +30,15 @@ static const int DATABASE_VERSION = 70509;
|
||||
// network protocol versioning
|
||||
//
|
||||
|
||||
static const int PROTOCOL_VERSION = 70205;
|
||||
static const int PROTOCOL_VERSION = 70206;
|
||||
|
||||
// v5 hard fork: require new protocol version (disconnects old nodes)
|
||||
static const int MIN_PROTO_VERSION = 70205;
|
||||
|
||||
// Peers >= this version support the P2P UTXO snapshot protocol
|
||||
// (getsnap/snap/getsnapchunk/snapchunk and the NODE_SNAPSHOT service flag).
|
||||
static const int SNAPSHOT_PROTO_VERSION = 70206;
|
||||
|
||||
static const int INIT_PROTO_VERSION = 209;
|
||||
|
||||
// nTime field added to CAddress, starting with this version;
|
||||
|
||||
+4
-4
@@ -102,14 +102,14 @@ static bool GetIndexedWalletTxHeight(const CTxIndex& txindex, int& nHeight)
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool ReadIndexedWalletTransaction(CTxDB& txdb, const uint256& hashTx, CTransaction& tx, CTxIndex& txindex, int& nHeight)
|
||||
static bool ReadIndexedWalletTransaction(CTxDBBase& txdb, const uint256& hashTx, CTransaction& tx, CTxIndex& txindex, int& nHeight)
|
||||
{
|
||||
if (!txdb.ReadDiskTx(hashTx, tx, txindex))
|
||||
return false;
|
||||
return GetIndexedWalletTxHeight(txindex, nHeight);
|
||||
}
|
||||
|
||||
static bool ReadIndexedWalletTransaction(CTxDB& txdb, const CDiskTxPos& txPos, CTransaction& tx, CTxIndex& txindex, int& nHeight)
|
||||
static bool ReadIndexedWalletTransaction(CTxDBBase& txdb, const CDiskTxPos& txPos, CTransaction& tx, CTxIndex& txindex, int& nHeight)
|
||||
{
|
||||
tx.SetNull();
|
||||
if (!tx.ReadFromDisk(txPos))
|
||||
@@ -903,7 +903,7 @@ void CWalletTx::GetAccountAmounts(const string& strAccount, int64_t& nReceived,
|
||||
}
|
||||
}
|
||||
|
||||
void CWalletTx::AddSupportingTransactions(CTxDB& txdb)
|
||||
void CWalletTx::AddSupportingTransactions(CTxDBBase& txdb)
|
||||
{
|
||||
vtxPrev.clear();
|
||||
|
||||
@@ -1227,7 +1227,7 @@ void CWallet::ReacceptWalletTransactions()
|
||||
}
|
||||
}
|
||||
|
||||
void CWalletTx::RelayWalletTransaction(CTxDB& txdb)
|
||||
void CWalletTx::RelayWalletTransaction(CTxDBBase& txdb)
|
||||
{
|
||||
for (const CMerkleTx& tx : vtxPrev)
|
||||
{
|
||||
|
||||
+3
-3
@@ -716,12 +716,12 @@ public:
|
||||
int64_t GetTxTime() const;
|
||||
int GetRequestCount() const;
|
||||
|
||||
void AddSupportingTransactions(CTxDB& txdb);
|
||||
void AddSupportingTransactions(CTxDBBase& txdb);
|
||||
|
||||
bool AcceptWalletTransaction(CTxDB& txdb, bool fCheckInputs=true);
|
||||
bool AcceptWalletTransaction(CTxDBBase& txdb, bool fCheckInputs=true);
|
||||
bool AcceptWalletTransaction();
|
||||
|
||||
void RelayWalletTransaction(CTxDB& txdb);
|
||||
void RelayWalletTransaction(CTxDBBase& txdb);
|
||||
void RelayWalletTransaction();
|
||||
};
|
||||
|
||||
|
||||
@@ -1,36 +0,0 @@
|
||||
## TRI Node Upgrade to v5.8.0 - April 14, 2026
|
||||
|
||||
This document outlines the process and results of upgrading the TRI network nodes to version 5.8.0.
|
||||
|
||||
### Initial State
|
||||
|
||||
- **DNS2:** `v5.7.9` @ block `2,203,611`
|
||||
- **DNS3:** `v5.7.5` @ block `2,204,954`
|
||||
- **Contabo Seeds:** `v5.7.9` @ block `2,203,594`
|
||||
|
||||
Nodes were on multiple versions and forks.
|
||||
|
||||
### Upgrade Process
|
||||
|
||||
1. **Version Confirmation:** Verified `v5.8.0` was available on GitHub.
|
||||
2. **Upgrades:**
|
||||
- DNS2 upgraded to `v5.8.0` via `dpkg`.
|
||||
- DNS3 upgraded to `v5.8.0` via `dpkg`.
|
||||
- Contabo seeds (`tri-seed-1` to `4`) upgraded to `v5.8.0` via `dpkg` inside their containers.
|
||||
3. **Chain Reset:** To resolve forks, the chain data (blocks, chainstate, peers) was wiped on DNS2 and all Contabo seeds. Wallets and configs were preserved. DNS3 was left as the canonical chain source.
|
||||
|
||||
### Current Status
|
||||
|
||||
- All nodes are now running `v5.8.0`.
|
||||
- Nodes are currently re-syncing to the canonical chain. Monitoring is in progress.
|
||||
|
||||
### DNS2 Wallet Corruption and Recovery
|
||||
|
||||
- **Symptom:** `triangles.service` on DNS2 was in a crash loop. Logs showed a recurring `CDB() : can't open database file wallet.dat, error -30973` error.
|
||||
- **Diagnosis:** `wallet.dat` file was corrupted.
|
||||
- **Recovery:**
|
||||
1. The corrupted wallet was moved to `wallet.dat.corrupted` for safety.
|
||||
2. The latest wallet backup (`dns2-wallet_20260414_031501.dat`) was restored from Dropbox.
|
||||
3. The `triangles.service` was restarted.
|
||||
|
||||
This restored the wallet to a healthy state.
|
||||
Reference in New Issue
Block a user