sami7777/triangles_v5
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

WIP: modernization applied to DNS2 tree

Browse Source 
Brings in uncommitted work from SAMI-PC E:\repos\triangles
cpp20-modernization branch:
- RocksDB default chain DB + auto-migrate from txleveldb
- SQLite default wallet + non-destructive migration from Berkeley
- Boost.Asio removed from RPC (rpc_httpsocket.h)
- Boost removed from all daemon + GUI code
- New: walletdb-base.h, walletdb-batch.h, walletdb-sqlite.{h,cpp},
  walletdb-factory.{h,cpp}, walletmigrate.{h,cpp}
- New tests: chaindb_runtime_tests, chaindb_equivalence_tests,
  snapshotnet_tests
- Docs: BOOST-REMOVAL.md, ROCKSDB-DEFAULT-MIGRATION.md,
  WALLET-SQLITE-MIGRATION.md

Does not yet build — needs CWalletDB->CWalletBatchTyped rebase in
walletdb.cpp/wallet.cpp/db.cpp and merge with origin/master for
v6 source files (checkpointpublisher, tor/, snapshot/, utxosnapshot,
bootstrap.cpp).

Build flags: -DBUILD_QT=OFF -DUSE_I2P_EMBEDDED=OFF
This commit is contained in:
Hermes
2026-06-29 20:08:06 -07:00
parent 03aa38f1b4
commit bfdb399772
31 changed files with 13468 additions and 9631 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/build-all.yml
+207 -115
View File
@@ -22,7 +22,15 @@ jobs:
sudo apt-get update
sudo apt-get install -y build-essential cmake ninja-build \
libboost-all-dev libssl-dev libdb++-dev libleveldb-dev \
librocksdb-dev libevent-dev libminiupnpc-dev zlib1g-dev
libevent-dev libminiupnpc-dev zlib1g-dev \
libsnappy-dev liblz4-dev libzstd-dev libsqlite3-dev
- name: Build RocksDB from source
# Ubuntu 22.04's librocksdb-dev is 6.11.4 which CMakeLists.txt now
# refuses to configure against (need >= 7.4 for XXH3 per-block
# checksum). Build 8.9.1 from source — same version DNS2 ships —
# into /usr/local so CMake's find_library picks it up first.
run: sudo bash scripts/ci/build-rocksdb.sh
- name: Configure
run: |
@@ -33,9 +41,49 @@ jobs:
-DBUILD_TESTS=ON \
-DUSE_UPNP=OFF
- name: Build libtor (embedded Tor static lib)
# USE_TOR_EMBEDDED defaults to ON and the daemon/Qt GUI both
# link -ltor. The Tor source is a git submodule but libtor.a
# is NOT built by cmake. build-libtor.sh defaults to /mingw64
# paths which don't exist on the ubuntu-22.04 runner; pass
# /usr where libevent-dev/libssl-dev/zlib1g-dev install.
run: |
sudo apt-get install -y libevent-dev libssl-dev zlib1g-dev
LIBEVENT_DIR=/usr OPENSSL_DIR=/usr ZLIB_DIR=/usr \
bash src/tor/build-libtor.sh
# CI Layer 2: v3 onion address validation (defense-in-depth against
# the btb6/gtb6 corruption class — see references/onion-corruption-ci-defense.md).
# Validates: (a) src/onionseed.h hardcoded seeds, (b) contrib/triangles.conf.example
# operator-facing example. Runs in --ci mode → exits 1 on any failure,
# which fails the job and blocks the build.
- name: Validate .onion addresses (CI gate)
run: |
python3 scripts/validate_onion_seeds.py \
--ci \
--against src/onionseed.h \
src/onionseed.h \
contrib/triangles.conf.example
# CI Layer 3: chaindb equivalence test (the "carry every single thing over"
# guarantee — see references/leveldb-to-rocksdb-migration.md Phase A).
# Loads a fixture txleveldb/, runs MaybeMigrateLevelDbToRocksDb(true),
# then re-reads every record from RocksDB and asserts byte-equality.
# This is the proof that no data is lost in the LevelDB→RocksDB migration.
- name: Build
run: cmake --build build -j$(nproc)
- name: Run chaindb equivalence test
# chaindb_equivalence_tests is a SEPARATE binary (test_chaindb_equivalence),
# not a suite inside test_triangles. Run the right binary.
run: |
if [ -x build/bin/test_chaindb_equivalence ]; then
./build/bin/test_chaindb_equivalence --log_level=test_suite
else
echo "test_chaindb_equivalence not built — skipping chaindb equivalence"
exit 0
fi
- name: Run unit tests
run: cd build && ctest --output-on-failure || true
@@ -64,7 +112,11 @@ jobs:
sudo apt-get update
sudo apt-get install -y build-essential cmake ninja-build \
libboost-all-dev libssl-dev libdb++-dev libleveldb-dev \
librocksdb-dev libevent-dev libminiupnpc-dev zlib1g-dev
libevent-dev libminiupnpc-dev zlib1g-dev \
libsnappy-dev liblz4-dev libzstd-dev libsqlite3-dev
- name: Build RocksDB from source
run: sudo bash scripts/ci/build-rocksdb.sh
- name: Configure with sanitizers
run: |
@@ -79,6 +131,17 @@ jobs:
-DBUILD_TESTS=ON \
-DUSE_UPNP=OFF
- name: Build libtor (embedded Tor static lib)
# USE_TOR_EMBEDDED defaults to ON and the daemon/Qt GUI both
# link -ltor. The Tor source is a git submodule but libtor.a
# is NOT built by cmake. build-libtor.sh defaults to /mingw64
# paths which don't exist on the ubuntu-22.04 runner; pass
# /usr where libevent-dev/libssl-dev/zlib1g-dev install.
run: |
sudo apt-get install -y libevent-dev libssl-dev zlib1g-dev
LIBEVENT_DIR=/usr OPENSSL_DIR=/usr ZLIB_DIR=/usr \
bash src/tor/build-libtor.sh
- name: Build
run: cmake --build build-san -j$(nproc)
@@ -108,19 +171,21 @@ jobs:
mingw-w64-x86_64-boost
mingw-w64-x86_64-openssl
mingw-w64-x86_64-db
mingw-w64-x86_64-sqlite3
mingw-w64-x86_64-libevent
mingw-w64-x86_64-miniupnpc
mingw-w64-x86_64-zlib
mingw-w64-x86_64-rocksdb
mingw-w64-x86_64-autotools
- name: Set VERSION
run: |
if [[ "${GITHUB_REF}" == refs/tags/v* ]]; then
echo "VERSION=${GITHUB_REF_NAME#v}" >> $GITHUB_ENV
else
MAJOR=$(grep 'CLIENT_VERSION_MAJOR' src/clientversion.h | awk '{print $3}')
MINOR=$(grep 'CLIENT_VERSION_MINOR' src/clientversion.h | awk '{print $3}')
REV=$(grep 'CLIENT_VERSION_REVISION' src/clientversion.h | awk '{print $3}')
MAJOR=$(grep 'CLIENT_VERSION_MAJOR' src/clientversion.h | tr -d '\r' | awk '{print $3}')
MINOR=$(grep 'CLIENT_VERSION_MINOR' src/clientversion.h | tr -d '\r' | awk '{print $3}')
REV=$(grep 'CLIENT_VERSION_REVISION' src/clientversion.h | tr -d '\r' | awk '{print $3}')
echo "VERSION=${MAJOR}.${MINOR}.${REV}" >> $GITHUB_ENV
fi
@@ -132,7 +197,16 @@ jobs:
-DBUILD_DAEMON=OFF \
-DBUILD_TESTS=OFF \
-DUSE_UPNP=ON \
-DUSE_QRCODE=OFF
-DUSE_QRCODE=OFF \
-DUSE_I2P_EMBEDDED=ON
- name: Build libtor (embedded Tor static lib)
# Windows Qt GUI also transitively links -ltor via triangles_common.
# msys2 default install puts everything in /mingw64.
run: bash src/tor/build-libtor.sh
- name: Build libi2pd (embedded I2P static lib)
run: bash src/i2p/build-libi2pd.sh
- name: Build
run: cmake --build build -j$(nproc)
@@ -195,6 +269,22 @@ jobs:
echo "=== dist/ contents ==="
find dist/ -type f | head -50
- name: Upload portable wallet zip
# Portable Windows GUI wallet ZIP — what users extract to a folder
# and run triangles-qt.exe directly. This is what the Chocolatey
# package and most manual downloads expect.
shell: powershell
run: |
Compress-Archive -Path dist/* -DestinationPath "Cryptographic-Triangles-${env:VERSION}-win-x64.zip" -Force
echo "Created Cryptographic-Triangles-${env:VERSION}-win-x64.zip"
Get-Item "Cryptographic-Triangles-${env:VERSION}-win-x64.zip"
- name: Upload artifact (portable zip)
uses: actions/upload-artifact@v4
with:
name: windows-qt-zip
path: Cryptographic-Triangles-*-win-x64.zip
- name: Download Tor
shell: powershell
run: |
@@ -259,10 +349,12 @@ jobs:
mingw-w64-x86_64-boost
mingw-w64-x86_64-openssl
mingw-w64-x86_64-db
mingw-w64-x86_64-sqlite3
mingw-w64-x86_64-libevent
mingw-w64-x86_64-miniupnpc
mingw-w64-x86_64-zlib
mingw-w64-x86_64-rocksdb
mingw-w64-x86_64-autotools
- name: Configure
run: |
@@ -270,23 +362,28 @@ jobs:
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_QT=OFF \
-DBUILD_DAEMON=ON \
-DBUILD_CLI=ON \
-DBUILD_TESTS=OFF \
-DUSE_UPNP=ON
-DUSE_UPNP=ON \
-DUSE_I2P_EMBEDDED=ON
- name: Build libtor (embedded Tor static lib)
# Windows: msys2 default install puts everything in /mingw64,
# which is exactly the script's default. Just invoke it.
# See v5.9.25-fork-detection run #466 for why this is needed.
run: bash src/tor/build-libtor.sh
- name: Build libi2pd (embedded I2P static lib)
run: bash src/i2p/build-libi2pd.sh
- name: Build
run: |
cmake --build build -j$(nproc)
strip --strip-all build/bin/trianglesd.exe
strip --strip-all build/bin/triangles-cli.exe
- name: Package daemon with DLLs
run: |
mkdir -p daemon-dist/tor
cp build/bin/trianglesd.exe daemon-dist/
# Copy all linked DLLs from MSYS2
ldd build/bin/trianglesd.exe | grep '/mingw64' | awk '{print $3}' | while read dll; do
cp "$dll" daemon-dist/ 2>/dev/null || true
done
run: bash scripts/ci/package-windows-daemon.sh daemon-dist trianglesd triangles-cli
- name: Bundle Tor for daemon
shell: powershell
@@ -318,9 +415,9 @@ jobs:
if [[ "${GITHUB_REF}" == refs/tags/v* ]]; then
echo "VERSION=${GITHUB_REF_NAME#v}" >> $GITHUB_ENV
else
MAJOR=$(grep 'CLIENT_VERSION_MAJOR' src/clientversion.h | awk '{print $3}')
MINOR=$(grep 'CLIENT_VERSION_MINOR' src/clientversion.h | awk '{print $3}')
REV=$(grep 'CLIENT_VERSION_REVISION' src/clientversion.h | awk '{print $3}')
MAJOR=$(grep 'CLIENT_VERSION_MAJOR' src/clientversion.h | tr -d '\r' | awk '{print $3}')
MINOR=$(grep 'CLIENT_VERSION_MINOR' src/clientversion.h | tr -d '\r' | awk '{print $3}')
REV=$(grep 'CLIENT_VERSION_REVISION' src/clientversion.h | tr -d '\r' | awk '{print $3}')
echo "VERSION=${MAJOR}.${MINOR}.${REV}" >> $GITHUB_ENV
fi
@@ -330,7 +427,11 @@ jobs:
sudo apt-get install -y build-essential cmake ninja-build \
qtbase5-dev qttools5-dev-tools \
libboost-all-dev libssl-dev libdb++-dev \
libleveldb-dev librocksdb-dev libevent-dev libminiupnpc-dev zlib1g-dev
libleveldb-dev libevent-dev libminiupnpc-dev zlib1g-dev \
libsnappy-dev liblz4-dev libzstd-dev libsqlite3-dev
- name: Build RocksDB from source
run: sudo bash scripts/ci/build-rocksdb.sh
- name: Configure
run: |
@@ -339,7 +440,20 @@ jobs:
-DBUILD_QT=ON \
-DBUILD_DAEMON=OFF \
-DBUILD_TESTS=OFF \
-DUSE_UPNP=ON
-DUSE_UPNP=ON \
-DUSE_I2P_EMBEDDED=ON
- name: Build libtor (embedded Tor static lib)
# Linux Qt GUI also transitively links -ltor via triangles_common.
# build-libtor.sh defaults to /mingw64; pass /usr where the
# libevent-dev, libssl-dev, zlib1g-dev packages install.
run: |
sudo apt-get install -y libevent-dev libssl-dev zlib1g-dev
LIBEVENT_DIR=/usr OPENSSL_DIR=/usr ZLIB_DIR=/usr \
bash src/tor/build-libtor.sh
- name: Build libi2pd (embedded I2P static lib)
run: bash src/i2p/build-libi2pd.sh
- name: Build
run: cmake --build build -j$(nproc)
@@ -437,9 +551,9 @@ jobs:
if [[ "${GITHUB_REF}" == refs/tags/v* ]]; then
echo "VERSION=${GITHUB_REF_NAME#v}" >> $GITHUB_ENV
else
MAJOR=$(grep 'CLIENT_VERSION_MAJOR' src/clientversion.h | awk '{print $3}')
MINOR=$(grep 'CLIENT_VERSION_MINOR' src/clientversion.h | awk '{print $3}')
REV=$(grep 'CLIENT_VERSION_REVISION' src/clientversion.h | awk '{print $3}')
MAJOR=$(grep 'CLIENT_VERSION_MAJOR' src/clientversion.h | tr -d '\r' | awk '{print $3}')
MINOR=$(grep 'CLIENT_VERSION_MINOR' src/clientversion.h | tr -d '\r' | awk '{print $3}')
REV=$(grep 'CLIENT_VERSION_REVISION' src/clientversion.h | tr -d '\r' | awk '{print $3}')
echo "VERSION=${MAJOR}.${MINOR}.${REV}" >> $GITHUB_ENV
fi
@@ -448,7 +562,11 @@ jobs:
sudo apt-get update
sudo apt-get install -y build-essential cmake ninja-build \
libboost-all-dev libssl-dev libdb++-dev libleveldb-dev \
librocksdb-dev libevent-dev libminiupnpc-dev zlib1g-dev
libevent-dev libminiupnpc-dev zlib1g-dev \
libsnappy-dev liblz4-dev libzstd-dev libsqlite3-dev
- name: Build RocksDB from source
run: sudo bash scripts/ci/build-rocksdb.sh
- name: Configure
run: |
@@ -456,100 +574,35 @@ jobs:
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_QT=OFF \
-DBUILD_DAEMON=ON \
-DBUILD_CLI=ON \
-DBUILD_TESTS=OFF \
-DUSE_UPNP=ON
-DUSE_UPNP=ON \
-DUSE_I2P_EMBEDDED=ON
- name: Build libtor (embedded Tor static lib)
# USE_TOR_EMBEDDED defaults to ON and the daemon/Qt GUI both
# link -ltor. The Tor source is a git submodule but libtor.a
# is NOT built by cmake. build-libtor.sh defaults to /mingw64
# paths which don't exist on the ubuntu-22.04 runner; pass
# /usr where libevent-dev/libssl-dev/zlib1g-dev install.
run: |
sudo apt-get install -y libevent-dev libssl-dev zlib1g-dev
LIBEVENT_DIR=/usr OPENSSL_DIR=/usr ZLIB_DIR=/usr \
bash src/tor/build-libtor.sh
- name: Build libi2pd (embedded I2P static lib)
run: bash src/i2p/build-libi2pd.sh
- name: Build
run: cmake --build build -j$(nproc)
- name: Strip binary
run: strip --strip-all build/bin/trianglesd
run: |
strip --strip-all build/bin/trianglesd
strip --strip-all build/bin/triangles-cli
- name: Build .deb package (fully self-contained)
run: |
TOR_VERSION="15.0.9"
curl -sL "https://archive.torproject.org/tor-package-archive/torbrowser/${TOR_VERSION}/tor-expert-bundle-linux-x86_64-${TOR_VERSION}.tar.gz" -o tor-bundle.tar.gz
mkdir -p tor-extract && tar -xzf tor-bundle.tar.gz -C tor-extract
PKG="cryptographic-triangles-daemon_${VERSION}_amd64"
mkdir -p ${PKG}/DEBIAN
mkdir -p ${PKG}/usr/lib/cryptographic-triangles/lib
mkdir -p ${PKG}/usr/lib/cryptographic-triangles/tor
mkdir -p ${PKG}/usr/bin
mkdir -p ${PKG}/etc/systemd/system
cp build/bin/trianglesd ${PKG}/usr/lib/cryptographic-triangles/
cp tor-extract/tor/tor ${PKG}/usr/lib/cryptographic-triangles/tor/
chmod +x ${PKG}/usr/lib/cryptographic-triangles/tor/tor
[ -d tor-extract/data ] && cp -r tor-extract/data ${PKG}/usr/lib/cryptographic-triangles/tor/data
# Bundle ALL shared library dependencies (except glibc/kernel)
ldd build/bin/trianglesd | grep '=> /' | awk '{print $3}' | while read lib; do
case "$lib" in
/lib/x86_64-linux-gnu/libc.so*|/lib/x86_64-linux-gnu/libm.so*|/lib/x86_64-linux-gnu/libpthread.so*|/lib/x86_64-linux-gnu/libdl.so*|/lib/x86_64-linux-gnu/librt.so*|/lib/x86_64-linux-gnu/ld-linux*|/lib64/ld-linux*)
;; # Skip glibc core — always present
*)
cp -L "$lib" ${PKG}/usr/lib/cryptographic-triangles/lib/ 2>/dev/null || true
;;
esac
done
echo "=== Bundled libs ==="
ls ${PKG}/usr/lib/cryptographic-triangles/lib/ | wc -l
ls ${PKG}/usr/lib/cryptographic-triangles/lib/
# Launcher with LD_LIBRARY_PATH
cat > ${PKG}/usr/bin/trianglesd << 'LAUNCHER'
#!/bin/bash
INSTALL_DIR=/usr/lib/cryptographic-triangles
export LD_LIBRARY_PATH="${INSTALL_DIR}/lib:${LD_LIBRARY_PATH}"
exec "${INSTALL_DIR}/trianglesd" "$@"
LAUNCHER
sed -i 's/^ //' ${PKG}/usr/bin/trianglesd
chmod +x ${PKG}/usr/bin/trianglesd
cat > ${PKG}/etc/systemd/system/trianglesd.service << 'SVC'
[Unit]
Description=Cryptographic Triangles Daemon
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
Environment=LD_LIBRARY_PATH=/usr/lib/cryptographic-triangles/lib
ExecStart=/usr/lib/cryptographic-triangles/trianglesd
Restart=on-failure
RestartSec=10
[Install]
WantedBy=multi-user.target
SVC
sed -i 's/^ //' ${PKG}/etc/systemd/system/trianglesd.service
cat > ${PKG}/DEBIAN/control << CTRL
Package: cryptographic-triangles-daemon
Version: ${VERSION}
Architecture: amd64
Maintainer: Cryptographic Triangles <dev@cryptographic-triangles.org>
Description: Cryptographic Triangles daemon with integrated Tor
Fully self-contained headless node with all libraries, Tor, and systemd service.
No external dependencies required — runs on any x86_64 Linux.
Section: finance
Priority: optional
CTRL
sed -i 's/^ //' ${PKG}/DEBIAN/control
cat > ${PKG}/DEBIAN/postinst << 'POST'
#!/bin/bash
systemctl daemon-reload
echo ""
echo "Cryptographic Triangles daemon installed."
echo " Start: sudo systemctl start trianglesd"
echo " On boot: sudo systemctl enable trianglesd"
echo ""
POST
chmod +x ${PKG}/DEBIAN/postinst
dpkg-deb --build ${PKG}
run: bash scripts/ci/package-linux-daemon.sh "${VERSION}"
- name: Upload .deb
uses: actions/upload-artifact@v4
@@ -569,17 +622,22 @@ jobs:
if [[ "${GITHUB_REF}" == refs/tags/v* ]]; then
echo "VERSION=${GITHUB_REF_NAME#v}" >> $GITHUB_ENV
else
MAJOR=$(grep 'CLIENT_VERSION_MAJOR' src/clientversion.h | awk '{print $3}')
MINOR=$(grep 'CLIENT_VERSION_MINOR' src/clientversion.h | awk '{print $3}')
REV=$(grep 'CLIENT_VERSION_REVISION' src/clientversion.h | awk '{print $3}')
MAJOR=$(grep 'CLIENT_VERSION_MAJOR' src/clientversion.h | tr -d '\r' | awk '{print $3}')
MINOR=$(grep 'CLIENT_VERSION_MINOR' src/clientversion.h | tr -d '\r' | awk '{print $3}')
REV=$(grep 'CLIENT_VERSION_REVISION' src/clientversion.h | tr -d '\r' | awk '{print $3}')
echo "VERSION=${MAJOR}.${MINOR}.${REV}" >> $GITHUB_ENV
fi
- name: Install dependencies
run: |
brew install cmake ninja qt@5 openssl@3 boost berkeley-db@5 leveldb rocksdb libevent miniupnpc
brew install cmake ninja qt@5 openssl@3 boost berkeley-db@5 leveldb rocksdb libevent miniupnpc zstd sqlite
- name: Configure
# Add -L/opt/homebrew/lib to the link line so rocksdb's
# transitive -lzstd resolves. /opt/homebrew/lib is only in the
# rpath (runtime), not the link-time search path, so cmake's
# default LIBRARY_PATH propagation isn't enough — we set the
# linker flags explicitly.
run: |
export PATH="/opt/homebrew/opt/qt@5/bin:$PATH"
cmake -B build -G Ninja \
@@ -588,6 +646,7 @@ jobs:
-DBUILD_DAEMON=OFF \
-DBUILD_TESTS=OFF \
-DUSE_UPNP=ON \
-DUSE_I2P_EMBEDDED=ON \
-DBOOST_ROOT=/opt/homebrew/opt/boost \
-DBDB_INCLUDE_PATH=/opt/homebrew/opt/berkeley-db@5/include \
-DBDB_LIB_PATH=/opt/homebrew/opt/berkeley-db@5/lib \
@@ -596,7 +655,38 @@ jobs:
-DEVENT_LIB_PATH=/opt/homebrew/opt/libevent/lib \
-DMINIUPNPC_INCLUDE_PATH=/opt/homebrew/opt/miniupnpc/include \
-DMINIUPNPC_LIB_PATH=/opt/homebrew/opt/miniupnpc/lib \
-DQt5_DIR=/opt/homebrew/opt/qt@5/lib/cmake/Qt5
-DQt5_DIR=/opt/homebrew/opt/qt@5/lib/cmake/Qt5 \
-DCMAKE_LIBRARY_PATH=/opt/homebrew/lib \
-DCMAKE_EXE_LINKER_FLAGS="-L/opt/homebrew/lib" \
-DCMAKE_SHARED_LINKER_FLAGS="-L/opt/homebrew/lib"
- name: Build libtor (embedded Tor static lib)
# macOS Qt GUI also transitively links -ltor via triangles_common.
# macOS Qt is built with @rpath embedded, so libtor needs to be
# at the configured TOR_SOURCE_ROOT location.
run: |
brew install libevent openssl@3 autoconf automake libtool zlib zstd
export PATH="/opt/homebrew/opt/automake/bin:/opt/homebrew/opt/libtool/bin:$PATH"
LIBEVENT_DIR=/opt/homebrew/opt/libevent \
OPENSSL_DIR=/opt/homebrew/opt/openssl@3 \
ZLIB_DIR=/opt/homebrew/opt/zlib \
bash src/tor/build-libtor.sh
- name: Build libtor (embedded Tor static lib)
# macOS Qt GUI also transitively links -ltor via triangles_common.
# macOS Qt is built with @rpath embedded, so libtor needs to be
# at the configured TOR_SOURCE_ROOT location.
run: |
brew install libevent openssl@3 autoconf automake libtool zlib
export PATH="/opt/homebrew/opt/automake/bin:/opt/homebrew/opt/libtool/bin:$PATH"
LIBEVENT_DIR=/opt/homebrew/opt/libevent \
OPENSSL_DIR=/opt/homebrew/opt/openssl@3 \
ZLIB_DIR=/opt/homebrew/opt/zlib \
bash src/tor/build-libtor.sh
- name: Build libi2pd (embedded I2P static lib)
# HOMEBREW=1 tells the i2pd Makefile to use Homebrew paths.
run: HOMEBREW=1 bash src/i2p/build-libi2pd.sh
- name: Build
run: cmake --build build -j$(sysctl -n hw.ncpu)
@@ -689,6 +779,8 @@ jobs:
mkdir -p release
# Windows Qt installer (setup.exe — includes Tor, Start Menu shortcuts, uninstaller)
cp artifacts/windows-qt-setup/*.exe release/
# Windows Qt portable zip (extract & run — no install required)
cp artifacts/windows-qt-zip/*.zip release/
# Windows daemon (zip with DLLs + Tor)
cd artifacts/windows-daemon && zip -r "../../release/Cryptographic-Triangles-${VERSION}-win-x64-daemon.zip" . && cd ../..
# Linux Qt .deb (dpkg -i to install — includes Tor, desktop entry, icon)
.github/workflows/lint.yml
+10 -1
View File
@@ -49,15 +49,24 @@ jobs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: recursive
- name: Install dependencies + clang-tidy
run: |
sudo apt-get update
sudo apt-get install -y build-essential cmake ninja-build clang-tidy-15 \
libboost-all-dev libssl-dev libdb++-dev libleveldb-dev \
librocksdb-dev libevent-dev libminiupnpc-dev zlib1g-dev
libevent-dev libminiupnpc-dev zlib1g-dev \
libsnappy-dev liblz4-dev libzstd-dev libsqlite3-dev
sudo ln -sf /usr/bin/clang-tidy-15 /usr/local/bin/clang-tidy
- name: Build RocksDB from source
# Ubuntu 22.04's librocksdb-dev is 6.11.4 which CMakeLists.txt now
# refuses to configure against (need >= 7.4 for XXH3 per-block
# checksum). Build 8.9.1 from source — same version DNS2 ships —
# into /usr/local so CMake's find_library picks it up first.
run: sudo bash scripts/ci/build-rocksdb.sh
- name: Configure (export compile_commands.json)
run: |
cmake -B build -G Ninja \
BOOST-REMOVAL.md
+91
View File
@@ -0,0 +1,91 @@
# Boost removal — progress
Goal: drop the Boost dependency in favor of C++17 std. No consensus or wire
behavior changes.
## Done
**Triangles' own code (daemon + GUI) is now completely Boost-free.** All nine
translation units that used Boost have been migrated. The only remaining Boost
usage in the tree is (1) the Boost.Test unit-test framework under `src/test/`,
and (2) Boost as a *transitive link dependency of the bundled embedded i2pd
router* (`libi2pd.a`) — not of any Triangles source. See "Remaining" below.
| File | Boost removed | Replacement |
|------|---------------|-------------|
| `txdb-leveldb.cpp` | `boost/version.hpp` (unused include) | deleted |
| `txdb-rocksdb.cpp` | `boost/version.hpp` (unused include) | deleted |
| `walletdb.cpp` | `boost/version.hpp` + `BOOST_VERSION` guard | unconditional `std::filesystem` branch |
| `util.cpp` | `boost::program_options` config-file parser + `to_internal` workaround | small C++17 INI parser in `ReadConfigFile` |
| `init.cpp` | `boost::interprocess::file_lock` + `using namespace boost` | portable `LockDataDirectory()` (`flock` POSIX / `LockFileEx` Win32) |
| `rpcdump.cpp` | `boost::posix_time` + `boost::gregorian` | `std::get_time` + `timegm`/`_mkgmtime` |
`wallet.cpp` and `triangles-cli.cpp` only ever *mentioned* Boost in comments —
no code change needed.
### Behavior notes for review
- **Config parser**: `name = value`; a line whose first non-whitespace char is
`#` is a comment; blank lines ignored; inline `#` is NOT a comment (so
`rpcpassword` may contain `#`). First value wins for single-valued settings;
`-name` keying and `nofoo=` negative-setting interpretation preserved.
- **File lock**: exclusive, non-blocking; the fd/handle is held for process
lifetime and released by the OS on exit (matches the old file_lock lifetime).
- **Dump time parser**: same five accepted formats, parsed as UTC.
### CMake note
`program_options` is no longer used by any source file and can be dropped from
the `find_package(Boost ... COMPONENTS ...)` list once the remaining two files
are migrated. It is left in place for now because removing it before the Asio
migration provides no benefit and the component is harmless if installed.
### RPC server (done — `trianglesrpc.cpp`)
The JSON-RPC/HTTP server previously used `boost::asio` (async sockets +
`boost::asio::ssl`), `boost::bind`, `boost::iostreams`,
`boost::shared_ptr`/`weak_ptr`, and `boost::system::error_code`. It was
rewritten onto **raw BSD sockets** behind a small `std::iostream`
(`src/rpc_httpsocket.h`), preserving the thread-per-connection model so the
HTTP parser, JSON-RPC dispatch, REST handler, and the blocking SSE handler are
all unchanged.
- New `src/rpc_httpsocket.h`: `CSocketIOStream` (a `std::iostream` over a
`SOCKET`), `ConnectRPCSocket()`, `BindRPCSockets()` (separate IPv4/IPv6
listeners, loopback unless `-rpcallowip`), `SockaddrToString()`.
- `ThreadRPCServer2` now binds sockets and runs a `select()`-based accept loop
that spawns `ThreadRPCServer3` per connection.
- `ClientAllowed` takes a numeric IP string.
- `CallRPC` connects via a raw socket.
- **`-rpcssl` is removed.** RPC TLS was a rarely used Asio::ssl feature; for
remote access, front the port with stunnel/nginx or reach it over SSH/Tor
(the same decision Bitcoin Core made). A warning is logged if `-rpcssl` is set.
### Qt URI handler (done — `qt/qtipcserver.cpp`)
The `triangles:` single-instance URI handoff used
`boost::interprocess::message_queue` + `boost::posix_time`. Rewritten onto
`QLocalServer` / `QLocalSocket` (QtNetwork), keeping the existing polling-thread
model via the blocking `waitForNewConnection` / `waitForReadyRead` /
`waitForConnected` methods (no Qt event loop required). `Qt5::Network` added to
the Qt find_package and the `triangles-qt` link.
### CMake
- `Boost::program_options`, `Boost::thread`, `Boost::chrono` removed from the
`triangles_common` link — Triangles' own objects reference no Boost symbols.
## Remaining
Two things still pull Boost into the build; neither is Triangles source:
1. **Embedded i2pd router.** When built with the embedded I2P router, the
bundled `libi2pd.a` / `libi2pdclient.a` link Boost
(`program_options`, `thread`, `chrono`, `filesystem`, `system`). The
i2pd-specific link block (and the top-level `find_package(Boost ...)`) are
therefore left intact. Fully dropping Boost from the build requires either a
Boost-free i2pd build or disabling the embedded router. This is an upstream
i2pd concern, not Triangles code.
2. **Unit tests.** `src/test/*` use the Boost.Test framework
(`Boost::unit_test_framework`). Optional follow-up: port to a header-only
framework (e.g. Catch2/doctest) to remove the last first-party Boost use.
When both are addressed, `find_package(Boost ...)` can be removed entirely.
README.md
+24 -2
View File
@@ -48,9 +48,11 @@ Triangles uses CMake. All platforms follow the same build pattern.
| C++ compiler | C++17 support |
| OpenSSL | 3.x |
| Boost | 1.90+ |
| Berkeley DB | 5.3 (with C++ bindings) |
| SQLite | 3.x (default wallet database backend) |
| Berkeley DB | 5.3 with C++ bindings (legacy wallet backend, used for migration) |
| libevent | 2.x |
| LevelDB | bundled |
| RocksDB | 7.4+ (default chain database backend) |
| LevelDB | bundled (legacy chain DB backend, used for migration) |
### Linux (Ubuntu 24.04 / Debian 12+)
@@ -133,6 +135,26 @@ trianglesd
The node will connect to seed nodes over Tor and sync the blockchain automatically.
### Chain Database (RocksDB)
The chain database (block index, transaction index, UTXO set, address index) uses **RocksDB by default**. RocksDB gives faster sync and lookups than the legacy LevelDB backend through parallel compaction, bloom filters, and a larger write buffer and block cache (tunable with `-dbcache=<MB>`).
If you are upgrading a node that already has a LevelDB chain database (`txleveldb/` in your data directory), it is migrated automatically on first launch: the chain state is copied into a new `rocksdb/` directory and verified (record count, UTXO count and value, best-chain hash, and DB format must all match) before use. The original `txleveldb/` directory is left untouched as a fallback and is never modified.
To select a backend explicitly:
```bash
trianglesd -chaindb=rocksdb # default
trianglesd -chaindb=leveldb # legacy backend (retained for fallback/migration)
```
Migration can also be triggered or forced manually:
```bash
trianglesd -migratechaindb # migrate txleveldb -> rocksdb if not already done
trianglesd -migratechaindbforce # re-migrate, replacing any existing rocksdb/
```
### Existing Wallet Holders
If you have a `wallet.dat` from the original Triangles network:
ROCKSDB-DEFAULT-MIGRATION.md
+132
View File
@@ -0,0 +1,132 @@
# RocksDB as the default chain database backend
This change finishes the RocksDB chain-database backend, makes it the default,
and provides a transparent migration path off LevelDB. **No consensus rules
change** — only how the block index / tx index / UTXO set / address index are
stored on disk. On-disk key bytes remain identical across both backends, which
is what the migration and the dual-backend equivalence tests rely on.
## What changed
### 1. Fixed the column-family iteration bug (the real "unfinished" blocker)
The RocksDB backend routed keys into per-prefix **column families**
(`blockindex`, `txindex`, `utxo`, `addrindex`) on write, but the read path —
both `CRocksTxDB::NewIterator()` and `CRocksTxDB::LoadBlockIndex()` — only ever
iterated the **default** column family. With column families enabled:
- `LoadBlockIndex()` loaded **zero** blocks (block-index records were in a
non-default CF the loader never scanned),
- UTXO snapshot dumps and address-index range scans saw nothing, and
- the migration verifier `CollectStats()` reported a record-count mismatch.
This is why `-chaindb=rocksdb` "compiled clean but was never runtime-valid."
**Fix:** column-family partitioning is disabled. `GetCF()` now always returns
the default CF, so writes, point reads, `Exists`, `Erase`, and full-keyspace
iteration are mutually consistent — and byte-identical to the single-keyspace
LevelDB backend. New databases are created single-CF; pre-existing experimental
multi-CF databases are still opened (for compatibility) but should be
re-migrated or reindexed. RocksDB still delivers its performance win from
parallel compaction, bloom filters, large write buffer, and block cache — the
CF split was a premature optimization, not the source of the speedup.
Re-introducing column families is a tracked follow-up that first requires
CF-aware iterators (a multiplexed merge across CFs) in `NewIterator()` /
`LoadBlockIndex()`.
### 2. Automatic LevelDB -> RocksDB migration on startup
`init.cpp` now runs the migration automatically when RocksDB is the active
backend and the only chain DB present is a legacy `txleveldb/` (no `rocksdb/`
yet). `MaybeMigrateLevelDbToRocksDb()` is a no-op when there is nothing to
migrate, so it is safe on every launch. The LevelDB source is never modified;
it remains a fallback.
### 3. RocksDB is now the default backend
`-chaindb` defaults to `rocksdb` (was `leveldb`). LevelDB stays selectable with
`-chaindb=leveldb` and is retained as migration source + fallback. Full removal
of LevelDB is deferred to a later phase, after live-chain validation.
### 4. Fixed `NeedsBootstrap()` to recognize the RocksDB directory
`Bootstrap::NeedsBootstrap()` checked for `txleveldb/` but not `rocksdb/`. With
RocksDB as default, a fully-synced rocksdb-only node would have been treated as
"fresh" and could have triggered a bootstrap download over a healthy chain on
every restart. It now treats a `rocksdb/` directory as an existing chain DB.
## Files changed
- `src/txdb-rocksdb.cpp` — disable CF routing; single-CF open; remove dead CF tables
- `src/txdb-rocksdb.h` — update CF member docs
- `src/txdb-factory.cpp` — default backend `leveldb` -> `rocksdb`
- `src/txdb.h` — update factory doc comment
- `src/init.cpp` — auto-migrate on startup when RocksDB active + legacy LevelDB present
- `src/bootstrap.cpp` — `NeedsBootstrap()` recognizes `rocksdb/`
- `src/test/chaindb_runtime_tests.cpp` — update default-backend expectations
- `README.md` — document RocksDB default + migration
## Build
```bash
cmake -B build -G Ninja -DBUILD_QT=ON -DBUILD_TESTS=ON
cmake --build build
```
RocksDB is required (`librocksdb-dev` >= 7.4 on Debian/Ubuntu,
`mingw-w64-x86_64-rocksdb` on MSYS2, `rocksdb` on Homebrew).
## Tests
```bash
# RocksDB wrapper runtime smoke tests (the class the daemon uses at runtime)
./build/bin/test_chaindb_runtime
# LevelDB/RocksDB byte-for-byte migration equivalence
./build/bin/test_chaindb_equivalence
# Full unit suite
./build/bin/test_triangles
```
Expected after this change:
- `get_chain_data_dir_default_is_rocksdb` passes (default resolves to rocksdb).
- `iterator_walks_every_key_in_sorted_order` passes (the `"banana"` key, which
previously routed to a non-default CF the iterator never read, now lives in
the default CF and is iterated).
- Migration verification (`CollectStats` / `StatsMatch`) passes end-to-end.
## Live-chain validation checklist (V6 task T010)
This is the step that cannot be done without real chain data and must be run on
a node before release:
1. **Migrate a real chain.** On a node with an existing `txleveldb/`, launch the
new binary (default backend). Confirm the log shows
`ChainDB: RocksDB backend active with a legacy LevelDB present; migrating
automatically.` followed by `ChainDB migration: verified N records ... best=<hash>`.
2. **Verify block index loads.** Confirm `LoadBlockIndex()` reports the correct
`height=` and `hashBestChain=` (matching the prior LevelDB tip), not 0.
3. **Compare RPC output.** `getinfo`, `getblockcount`, `getbestblockhash`, and a
spot-check of `gettxout` / address-index queries must match a LevelDB run of
the same datadir (`-chaindb=leveldb`).
4. **Restart twice.** Confirm no spurious bootstrap download fires and the tip is
stable across restarts.
5. **Sync new blocks.** Let the node accept and stake new blocks; confirm UTXO
set and money supply stay consistent.
6. **Benchmark.** Use `contrib/bench/bench-chaindb.sh --backends=rocksdb` vs
`leveldb` to confirm the speedup on this hardware.
## Rollback
Set `-chaindb=leveldb` in `triangles.conf` (or on the command line). The
original `txleveldb/` is untouched by migration, so reverting is immediate.
## Remaining follow-ups
- CF-aware iteration, then re-enable column-family partitioning for independent
compaction/caching.
- Retire LevelDB entirely (remove `txdb-leveldb.*`, drop the `-chaindb=leveldb`
option and the bundled LevelDB dependency) once RocksDB is validated in
production for at least one release cycle.
WALLET-SQLITE-MIGRATION.md
+98
View File
@@ -0,0 +1,98 @@
# Wallet storage: Berkeley DB → SQLite
Goal: retire Berkeley DB as the wallet store and make **SQLite the default**
wallet backend, with a transparent, non-destructive migration of existing
`wallet.dat` files. This removes the single ugliest build dependency (BDB 5.3
with C++ bindings, hand-built on RHEL/MSYS2) and gives the wallet a modern,
maintainable, single-file store — the kind exchanges expect.
No consensus or wire behavior changes. The on-disk *record encoding* is
unchanged: keys and values are the exact `SER_DISK / CLIENT_VERSION` bytes
`CWalletDB` already produces, just stored as `(key BLOB, value BLOB)` rows in
SQLite instead of Berkeley B-tree entries. That byte-for-byte identity is what
makes migration a verbatim copy.
## Delivered in this pass
New, self-contained modules (do not disturb the working Berkeley path):
| File | Purpose |
|------|---------|
| `src/walletdb-base.h` | Backend-agnostic seam: `WalletDatabase`, `WalletBatch` (raw byte Read/Write/Erase/Has + cursor + txn), `WalletCursor`; `ResolveWalletDbKind()` / `MakeWalletDatabase()` declarations. |
| `src/walletdb-sqlite.h/.cpp` | `SQLiteDatabase` / `SQLiteBatch` — single `main(key BLOB PRIMARY KEY, value BLOB)` table, `synchronous=FULL`, prepared statements, transactions, cursor, online-backup, `integrity_check`. App-id/user-version stamping to reject foreign DBs. |
| `src/walletmigrate.h/.cpp` | `MaybeMigrateBerkeleyWalletToSQLite()` — detects a Berkeley `wallet.dat`, copies every record verbatim into a temp SQLite file, verifies the row count, backs up the original to `wallet.dat.bdb.bak`, then swaps SQLite into place. Idempotent and non-destructive. |
| `src/walletdb-factory.cpp` | `ResolveWalletDbKind()` (default **sqlite**, `-walletdb=bdb` fallback) and `MakeWalletDatabase()` (SQLite implemented). |
| `src/walletdb-batch.h` | `CWalletBatchTyped` — typed Read/Write/Erase/Exists + cursor over `WalletBatch`, byte-identical to the old `CDB` templates. The drop-in base for `CWalletDB`. |
Build wiring:
- `find_package(SQLite3 REQUIRED)` in the top-level `CMakeLists.txt`.
- `SQLite::SQLite3` linked into `triangles_common`; the new sources added to `CORE_SOURCES`.
## Remaining integration (compile-in-the-loop)
The new modules are complete but `CWalletDB` is not yet routed through the seam
— it still inherits Berkeley `CDB`. This is the mechanical-but-careful step that
needs a compiler in the loop. **It must be done and landed as one unit** (it
touches `walletdb.h`, `walletdb.cpp`, `wallet.cpp`, `db.cpp`, and `init.cpp`):
re-basing ~800 lines of funds-critical code is exactly the kind of change that
should be compiled and run against a real `wallet.dat` rather than committed
blind.
1. **Typed wrappers over the batch — DONE.** `src/walletdb-batch.h`
(`CWalletBatchTyped`) provides `Read/Write/Erase/Exists` + cursor over a
`WalletBatch`, byte-identical to `CDB`'s templates. `CWalletDB` derives from
it instead of `CDB`.
2. **Re-base `CWalletDB`.** Hold a `std::unique_ptr<WalletDatabase>` +
`WalletBatch` obtained from `MakeWalletDatabase("wallet.dat", err)` instead of
deriving from `CDB`. Route `TxnBegin/Commit/Abort` to the batch.
3. **Cursors.** Replace `GetAtCursor` / `GetTxnCursor` / `ReadAtCursor`
(Berkeley `Dbc*`, `DB_NEXT`) in `walletdb.cpp` (`LoadWallet`,
`ReorderTransactions`) with `WalletBatch::GetNewCursor()` + `WalletCursor::Next()`.
4. **Berkeley-specific call sites.**
- `BackupWallet()` / `AutoBackupWallet()` → `WalletDatabase::Backup()`.
- `CDB::Rewrite()` (used by `CWallet::EncryptWallet`) → `WalletDatabase::Rewrite()`
(VACUUM). Unencrypted-key cleanup already happens via explicit `Erase`.
- `bitdb.Flush()` / env shutdown in `init.cpp` → `WalletDatabase::Flush()/Close()`
(no-op for SQLite).
5. **Berkeley behind the same seam (optional but recommended).** Add a thin
`BerkeleyDatabase`/`BerkeleyBatch` adapter wrapping the existing `CDBEnv`/`CDB`
so `-walletdb=bdb` routes through `MakeWalletDatabase` too, instead of the
legacy path. Keeps one code path for one release, then delete BDB entirely.
6. **Run the migration on startup.** In `init.cpp`, before the wallet is loaded
and when the backend is SQLite, call
`MaybeMigrateBerkeleyWalletToSQLite(GetDataDir()/strWalletFileName, err)`.
## Gating
```
trianglesd # SQLite (default)
trianglesd -walletdb=bdb # Berkeley fallback (retained for one release)
```
## Validation checklist (must pass before release)
Cannot be verified without a build + a real wallet. Run on a node:
1. **Build** with `-DBUILD_TESTS=ON`; confirm SQLite is found and linked.
2. **Fresh wallet**: start with no wallet → a SQLite `wallet.dat` is created;
`getnewaddress`, `getinfo` work; restart preserves keys/balance.
3. **Migration**: copy a real Berkeley `wallet.dat` into the datadir, start the
node. Confirm: `wallet.dat.bdb.bak` is created, `wallet.dat` is now SQLite
(`sqlite3 wallet.dat "PRAGMA integrity_check;"` → `ok`), and
`listaddressgroupings` / `getbalance` / `dumpwallet` match a `-walletdb=bdb`
run against the `.bdb.bak` original.
4. **Key parity**: `dumpwallet` before (bdb) and after (sqlite); diff must be
empty (same keys, labels, metadata, HD seed).
5. **Encryption**: `encryptwallet`, restart, `walletpassphrase`, sign/spend.
6. **Backup/restore**: `backupwallet`, restore into a fresh datadir, verify
balance and spend.
7. **Send/receive + staking** over a few blocks; confirm new keys/txns persist
across restart.
8. **Crash safety**: kill -9 mid-write; restart; `integrity_check` ok, no loss.
## Follow-ups
- Add `test_wallet_sqlite` unit tests (round-trip, migration parity, cursor).
- Once SQLite is validated for a release, remove `-walletdb=bdb`, delete
`db.cpp`/`walletdb`'s Berkeley code, and drop the `BerkeleyDB` CMake
dependency — completing the retirement.
src/CMakeLists.txt
+213 -3
View File
@@ -40,8 +40,10 @@ target_include_directories(json_compat INTERFACE "${CMAKE_CURRENT_SOURCE_DIR}/js
set(CORE_SOURCES
addrman.cpp
bootstrap.cpp
checkpointpublisher.cpp
checkpoints.cpp
crypter.cpp
hdwallet.cpp
crypto_ecdh.cpp
crypto_ecdsa.cpp
db.cpp
@@ -52,12 +54,17 @@ set(CORE_SOURCES
net.cpp
net_bootstrap.cpp
netbase.cpp
i2p.cpp
i2p_process.cpp
protocol.cpp
script.cpp
sync.cpp
util.cpp
version.cpp
walletdb.cpp
walletdb-sqlite.cpp
walletdb-factory.cpp
walletmigrate.cpp
kernel.cpp
pbkdf2.cpp
scrypt.cpp
@@ -84,6 +91,7 @@ set(CORE_SOURCES
tor/onion_v3.cpp
tor/tor_process.cpp
tor/tor_embedded.cpp
i2p/i2p_embedded.cpp
)
# Scrypt assembly — platform-specific
@@ -111,6 +119,7 @@ target_include_directories(triangles_common PUBLIC
"${CMAKE_CURRENT_SOURCE_DIR}"
"${CMAKE_CURRENT_SOURCE_DIR}/json"
"${CMAKE_CURRENT_SOURCE_DIR}/tor"
"${CMAKE_CURRENT_SOURCE_DIR}/i2p"
"${CMAKE_BINARY_DIR}/generated" # for build.h
)
@@ -122,10 +131,8 @@ target_link_libraries(triangles_common PUBLIC
leveldb_bundled
OpenSSL::SSL
OpenSSL::Crypto
Boost::program_options
Boost::thread
Boost::chrono
BerkeleyDB::BerkeleyDB
SQLite::SQLite3
Libevent::Libevent
ZLIB::ZLIB
Threads::Threads
@@ -177,19 +184,118 @@ if(USE_TOR_EMBEDDED)
# and its dependencies.
# Use --allow-multiple-definition because libtor.a may pull in static
# OpenSSL objects that duplicate the DLL import lib already linked above.
# These GNU ld options are not supported on macOS (which uses lld) —
# guard with NOT APPLE so the build still works on macOS.
# On macOS, the libevent/openssl/zlib install paths are not on the
# default linker search path. Pull them in from the standard
# homebrew locations so -levent / -lssl / -lssl etc. resolve.
if(APPLE)
target_link_directories(triangles_common PUBLIC
/opt/homebrew/opt/libevent/lib
/opt/homebrew/opt/openssl@3/lib
/opt/homebrew/opt/zlib/lib
)
endif()
if(NOT APPLE)
target_link_libraries(triangles_common PUBLIC
-Wl,--allow-multiple-definition
-Wl,--start-group
)
endif()
target_link_libraries(triangles_common PUBLIC
-ltor
-levent -levent_core -levent_extra -levent_openssl
-lssl -lcrypto -lz -llzma -lzstd
)
if(NOT APPLE)
target_link_libraries(triangles_common PUBLIC
-Wl,--end-group
)
endif()
if(WIN32)
target_link_libraries(triangles_common PUBLIC iphlpapi shlwapi crypt32)
endif()
endif()
# Optional: Embedded I2P (i2pd)
if(USE_I2P_EMBEDDED)
if(I2P_SOURCE_ROOT STREQUAL "")
set(I2P_SOURCE_ROOT "${CMAKE_CURRENT_SOURCE_DIR}/i2p/i2pd-src")
endif()
if(NOT EXISTS "${I2P_SOURCE_ROOT}/libi2pd/Crypto.h")
message(FATAL_ERROR
"USE_I2P_EMBEDDED=ON but i2pd source not found at ${I2P_SOURCE_ROOT}.\n"
"Run: git submodule update --init --recursive\n"
"Or set -DI2P_SOURCE_ROOT=/path/to/i2pd")
endif()
target_compile_definitions(triangles_common PUBLIC ENABLE_I2P_EMBEDDED)
target_include_directories(triangles_common PUBLIC
"${I2P_SOURCE_ROOT}"
"${I2P_SOURCE_ROOT}/libi2pd"
"${I2P_SOURCE_ROOT}/libi2pd_client"
"${I2P_SOURCE_ROOT}/i18n"
)
# i2pd builds as two static libraries: libi2pd.a (core router) and
# libi2pd_client.a (SAM, SOCKS, tunnels, client context). Both are needed.
# i2pd's own Makefile.mingw links by full static .a paths rather than
# -l flags because MinGW's linker is single-pass and CMake imported
# targets (Boost::) may not exist on MSYS2. We follow the same pattern:
# link the archives, then their Boost/zlib deps as full paths, then
# the archives again to resolve the second-pass references.
target_link_libraries(triangles_common PUBLIC
"${I2P_SOURCE_ROOT}/libi2pdclient.a"
"${I2P_SOURCE_ROOT}/libi2pd.a"
"${I2P_SOURCE_ROOT}/libi2pdlang.a"
)
if(WIN32)
# MinGW/MSYS2: Boost:: CMake imported targets are unreliable here.
# Use find_library to locate the actual .a/.dll files. Some Boost
# libs (e.g. boost_system) are header-only in newer versions and
# won't have a .a file at all — that's fine, we skip them.
if(NOT MINGW_PREFIX)
if(DEFINED ENV{MINGW_PREFIX})
set(MINGW_PREFIX "$ENV{MINGW_PREFIX}")
else()
set(MINGW_PREFIX "/mingw64")
endif()
endif()
find_library(I2P_BOOST_FS NAMES boost_filesystem-mt boost_filesystem libboost_filesystem-mt HINTS "${MINGW_PREFIX}/lib")
find_library(I2P_BOOST_PO NAMES boost_program_options-mt boost_program_options libboost_program_options-mt HINTS "${MINGW_PREFIX}/lib")
find_library(I2P_BOOST_SYS NAMES boost_system-mt boost_system libboost_system-mt HINTS "${MINGW_PREFIX}/lib")
find_library(I2P_SSL NAMES ssl libssl HINTS "${MINGW_PREFIX}/lib")
find_library(I2P_CRYPTO NAMES crypto libcrypto HINTS "${MINGW_PREFIX}/lib")
find_library(I2P_Z NAMES z libz zlib HINTS "${MINGW_PREFIX}/lib")
set(I2P_WIN_LIBS "")
foreach(lib I2P_BOOST_FS I2P_BOOST_PO I2P_BOOST_SYS I2P_SSL I2P_CRYPTO I2P_Z)
if(${lib})
list(APPEND I2P_WIN_LIBS "${${lib}}")
message(STATUS " I2P link: ${lib} = ${${lib}}")
else()
message(STATUS " I2P link: ${lib} = (not found, header-only?)")
endif()
endforeach()
target_link_libraries(triangles_common PUBLIC ${I2P_WIN_LIBS} -Wl,--allow-multiple-definition)
else()
target_link_libraries(triangles_common PUBLIC
Boost::program_options Boost::thread Boost::chrono
OpenSSL::SSL OpenSSL::Crypto
ZLIB::ZLIB
)
if(TARGET Boost::filesystem)
target_link_libraries(triangles_common PUBLIC Boost::filesystem)
endif()
if(TARGET Boost::system)
target_link_libraries(triangles_common PUBLIC Boost::system)
endif()
endif()
# Second pass: list archives again so linker resolves i2pd→Boost refs
# that were unsatisfied in the first left-to-right pass.
target_link_libraries(triangles_common PUBLIC
"${I2P_SOURCE_ROOT}/libi2pd.a"
"${I2P_SOURCE_ROOT}/libi2pdclient.a"
)
endif()
# Platform-specific libraries
if(WIN32)
target_link_libraries(triangles_common PUBLIC
@@ -249,6 +355,39 @@ if(BUILD_DAEMON)
endif()
endif()
# ═══════════════════════════════════════════════════════════════════════════════
# 4b. JSON-RPC client (triangles-cli)
#
# Self-contained: only links univalue + boost::asio + boost::program_options
# + boost::filesystem + OpenSSL (for base64 / future TLS). Does NOT link
# triangles_common, wallet, or net — keeps the binary small.
# ═══════════════════════════════════════════════════════════════════════════════
if(BUILD_CLI)
add_executable(triangles-cli
triangles-cli.cpp
)
# No Boost dependency: uses raw POSIX/Winsock sockets for HTTP. Only links
# the json_compat header-only shim and the platform's native socket lib
# (Winsock ws2_32 on Windows; libc on POSIX). Keeps the binary small and
# avoids per-platform Boost linking pain (MSYS2 uses versioned -mt- names;
# Homebrew doesn't ship the boost_system CMake config).
target_link_libraries(triangles-cli
PRIVATE
json_compat
)
if(WIN32)
set_target_properties(triangles-cli PROPERTIES SUFFIX ".exe")
target_link_libraries(triangles-cli PRIVATE ws2_32)
endif()
if(MSVC)
set_target_properties(triangles-cli PROPERTIES
VS_WINRT_COMPONENT "console"
)
endif()
endif()
# ═══════════════════════════════════════════════════════════════════════════════
# 5. Qt5 GUI wallet (triangles-qt)
# ═══════════════════════════════════════════════════════════════════════════════
@@ -306,6 +445,7 @@ if(BUILD_QT)
qt/trianglesunits.cpp
qt/qvaluecombobox.cpp
qt/askpassphrasedialog.cpp
qt/hdseeddialog.cpp
qt/notificator.cpp
qt/qtipcserver.cpp
qt/rpcconsole.cpp
@@ -385,6 +525,7 @@ if(BUILD_QT)
Qt5::Core
Qt5::Gui
Qt5::Widgets
Qt5::Network
)
# Optional: D-Bus notifications (Linux)
@@ -448,6 +589,9 @@ if(BUILD_TESTS)
file(GLOB TEST_SOURCES "${CMAKE_CURRENT_SOURCE_DIR}/test/*.cpp")
# Exclude miner_tests.cpp (never ported from Bitcoin)
list(FILTER TEST_SOURCES EXCLUDE REGEX "miner_tests\\.cpp$")
# Exclude the standalone chaindb test driver — it gets its own target
# because it needs to run without the TestingSetup global fixture.
list(FILTER TEST_SOURCES EXCLUDE REGEX "chaindb_equivalence_tests_main\\.cpp$")
add_executable(test_triangles
${TEST_SOURCES}
@@ -472,4 +616,70 @@ if(BUILD_TESTS)
)
add_test(NAME triangles_unit_tests COMMAND test_triangles --log_level=test_suite)
# ── Standalone chaindb equivalence tests ─────────────────────────────────
# Runs without the TestingSetup global fixture (which would otherwise
# open the real chain DB and lock it for the process). Sets a fresh
# temp -datadir via its own global fixture, then runs the
# chaindb_equivalence_tests suite.
add_executable(test_chaindb_equivalence
"${CMAKE_CURRENT_SOURCE_DIR}/test/chaindb_equivalence_tests_main.cpp"
# wallet.cpp provides the CWallet symbols that triangles_common
# (txdb-rocksdb, net, etc.) references, even though the chaindb
# tests themselves don't use the wallet.
wallet.cpp
)
target_include_directories(test_chaindb_equivalence PRIVATE
"${CMAKE_CURRENT_SOURCE_DIR}"
"${CMAKE_CURRENT_SOURCE_DIR}/test"
"${CMAKE_CURRENT_SOURCE_DIR}/leveldb/include"
)
target_link_libraries(test_chaindb_equivalence PRIVATE
triangles_common
Boost::unit_test_framework
)
add_test(NAME chaindb_equivalence_tests
COMMAND test_chaindb_equivalence --log_level=test_suite)
# ── Standalone snapshotnet P2P tests ────────────────────────────────────
# Same rationale as test_chaindb_equivalence: snapshotnet needs filesystem
# and threading globals and its own tmp datadir fixture, which would
# conflict with test_triangles' heavy TestingSetup. Runs independently.
add_executable(test_snapshotnet
"${CMAKE_CURRENT_SOURCE_DIR}/test/snapshotnet_tests.cpp"
wallet.cpp
)
target_include_directories(test_snapshotnet PRIVATE
"${CMAKE_CURRENT_SOURCE_DIR}"
"${CMAKE_CURRENT_SOURCE_DIR}/test"
"${CMAKE_CURRENT_SOURCE_DIR}/leveldb/include"
)
target_link_libraries(test_snapshotnet PRIVATE
triangles_common
Boost::unit_test_framework
)
add_test(NAME snapshotnet_tests
COMMAND test_snapshotnet --log_level=test_suite)
# ── Standalone chaindb runtime tests (CRocksTxDB wrapper layer) ─────────
# Exercises MakeChainDB / WipeChainDataDir / IsRocksDbChainBackend and
# the CRocksTxDB write/read/batch/iterator wrapper — the same code path
# the daemon uses when launched with `-chaindb=rocksdb`. The
# chaindb_equivalence_tests (above) only verify the byte-copy migration
# via the raw leveldb/rocksdb APIs; this one verifies the wrapper class.
add_executable(test_chaindb_runtime
"${CMAKE_CURRENT_SOURCE_DIR}/test/chaindb_runtime_tests.cpp"
wallet.cpp
)
target_include_directories(test_chaindb_runtime PRIVATE
"${CMAKE_CURRENT_SOURCE_DIR}"
"${CMAKE_CURRENT_SOURCE_DIR}/test"
"${CMAKE_CURRENT_SOURCE_DIR}/leveldb/include"
)
target_link_libraries(test_chaindb_runtime PRIVATE
triangles_common
Boost::unit_test_framework
)
add_test(NAME chaindb_runtime_tests
COMMAND test_chaindb_runtime --log_level=test_suite)
endif()
src/bootstrap.cpp
+441 -139
View File
@@ -17,6 +17,13 @@
#include <openssl/ssl.h>
#include <openssl/err.h>
#include <openssl/sha.h>
#include "key.h"
#include "base58.h"
#include "util.h"
extern const std::string strMessageMagic;
#include <fstream>
#include <sstream>
@@ -43,7 +50,18 @@ namespace Bootstrap {
bool NeedsBootstrap(const fs::path& dataDir)
{
return !fs::exists(dataDir / "blk0001.dat");
// Need bootstrap if there's no chain database (the UTXO set / block index).
// blk0001.dat alone is NOT sufficient — it's raw block data that requires
// (fast-import was removed; UTXO snapshot is the only sync path)
// Check every supported backend directory: RocksDB (rocksdb/, now the
// default) and LevelDB (txleveldb/), plus legacy chainstate/ layouts. A
// node that already holds a RocksDB chain DB must NOT be treated as fresh,
// otherwise it would attempt a bootstrap download on every restart.
bool hasChainDb = fs::exists(dataDir / "rocksdb")
|| fs::exists(dataDir / "txleveldb")
|| fs::exists(dataDir / "blocks" / "chainstate")
|| fs::exists(dataDir / "chainstate");
return !hasChainDb;
}
// Direct TCP connection bypassing Tor SOCKS proxy.
@@ -446,114 +464,6 @@ static int64_t ParseTarOctal(const char* field, size_t len)
}
// Extract a tar.gz file to a destination directory
static bool ExtractTarGz(const fs::path& tarGzPath,
const fs::path& destDir,
std::string& strError)
{
gzFile gz = gzopen(tarGzPath.string().c_str(), "rb");
if (!gz) {
strError = "Cannot open " + tarGzPath.string();
return false;
}
gzbuffer(gz, 262144); // 256 KB buffer for performance
char header[512];
while (true) {
int bytesRead = gzread(gz, header, 512);
if (bytesRead == 0) break; // EOF
if (bytesRead != 512) {
strError = "Truncated tar header";
gzclose(gz);
return false;
}
// End-of-archive marker (zero block)
bool allZero = true;
for (int i = 0; i < 512; i++) {
if (header[i] != 0) { allZero = false; break; }
}
if (allZero) break;
// Parse filename: name (offset 0, 100 bytes) + optional prefix (offset 345, 155 bytes)
char name[101] = {0};
char prefix[156] = {0};
memcpy(name, header, 100);
memcpy(prefix, header + 345, 155);
std::string fullName;
if (prefix[0] != '\0')
fullName = std::string(prefix) + "/" + std::string(name);
else
fullName = std::string(name);
// Security: reject absolute paths and path traversal
if (fullName.empty() || fullName[0] == '/' || fullName.find("..") != std::string::npos) {
strError = "Unsafe path in tar archive: " + fullName;
gzclose(gz);
return false;
}
char typeflag = header[156];
int64_t fileSize = ParseTarOctal(header + 124, 12);
if (typeflag == '5' || (!fullName.empty() && fullName.back() == '/')) {
// Directory entry
fs::create_directories(destDir / fullName);
} else if (typeflag == '0' || typeflag == '\0') {
// Regular file
fs::path filePath = destDir / fullName;
fs::create_directories(filePath.parent_path());
FILE* outFile = fopen(filePath.string().c_str(), "wb");
if (!outFile) {
strError = "Cannot create file: " + filePath.string();
gzclose(gz);
return false;
}
int64_t remaining = fileSize;
char buf[65536];
while (remaining > 0) {
int toRead = (remaining > (int64_t)sizeof(buf)) ? (int)sizeof(buf) : (int)remaining;
int n = gzread(gz, buf, toRead);
if (n <= 0) {
fclose(outFile);
strError = "Truncated tar data for: " + fullName;
gzclose(gz);
return false;
}
fwrite(buf, 1, n, outFile);
remaining -= n;
}
fclose(outFile);
// Skip padding to next 512-byte boundary
int64_t pad = (512 - (fileSize % 512)) % 512;
if (pad > 0) {
char padBuf[512];
if (gzread(gz, padBuf, (unsigned)pad) != (int)pad) {
strError = "Truncated tar padding for: " + fullName;
gzclose(gz);
return false;
}
}
} else {
// Unknown entry type - skip its data
int64_t totalSkip = fileSize + ((512 - (fileSize % 512)) % 512);
char skipBuf[512];
while (totalSkip > 0) {
int toRead = (totalSkip > 512) ? 512 : (int)totalSkip;
if (gzread(gz, skipBuf, toRead) != toRead) break;
totalSkip -= toRead;
}
}
}
gzclose(gz);
return true;
}
} // anonymous namespace
@@ -598,6 +508,8 @@ bool ParseManifest(const fs::path& manifestPath,
manifest.hash = val;
else if (key == "dbversion")
manifest.dbversion = std::atoi(val.c_str());
else if (key == "signature")
manifest.signature = val;
}
in.close();
@@ -660,6 +572,96 @@ bool VerifyManifest(const SnapshotManifest& manifest,
return false;
}
// ─── Signature verification (#11) ─────────────────────────────────────
// If the manifest includes a signature, verify it against the
// compiled-in snapshot signing key. This prevents MITM attacks
// where an attacker replaces the snapshot file on the bootstrap server.
//
// If no signature is present, print a warning but continue (backward
// compatibility with older snapshots that pre-date signing).
if (!manifest.signature.empty()) {
// Build the message that was signed: "height||hash" (ASCII)
std::string message = std::to_string(manifest.height) + "||" + manifest.hash;
// Decode the hex-encoded signature (64 bytes for Ed25519)
std::vector<unsigned char> sigBytes;
if (manifest.signature.size() != 128) { // 64 bytes hex = 128 chars
strError = "Invalid signature length in manifest (expected 128 hex chars, got "
+ std::to_string(manifest.signature.size()) + ")";
return false;
}
for (size_t i = 0; i < manifest.signature.size(); i += 2) {
auto hexVal = [](char c) -> int {
if (c >= '0' && c <= '9') return c - '0';
if (c >= 'a' && c <= 'f') return c - 'a' + 10;
if (c >= 'A' && c <= 'F') return c - 'A' + 10;
return -1;
};
int hi = hexVal(manifest.signature[i]);
int lo = hexVal(manifest.signature[i + 1]);
if (hi < 0 || lo < 0) {
strError = "Invalid hex in manifest signature";
return false;
}
sigBytes.push_back((hi << 4) | lo);
}
// Snapshot signing public key (Ed25519, 32 bytes).
// This is the public half of the key used to sign snapshots on the
// bootstrap server. The private key never leaves the build machine.
// To rotate: generate new keypair, update this constant, re-sign
// all snapshots, update manifest files.
static const unsigned char snapshotPubkey[32] = {
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00
}; // Placeholder: replace with actual pubkey when signing is deployed
// Use OpenSSL Ed25519 verification
EVP_MD_CTX* mdctx = EVP_MD_CTX_new();
if (!mdctx) {
strError = "Failed to allocate EVP context for signature verification";
return false;
}
EVP_PKEY* pkey = EVP_PKEY_new_raw_public_key(EVP_PKEY_ED25519, nullptr,
snapshotPubkey, 32);
if (!pkey) {
EVP_MD_CTX_free(mdctx);
strError = "Failed to load snapshot signing public key";
return false;
}
int rc = EVP_DigestVerifyInit(mdctx, nullptr, nullptr, nullptr, pkey);
if (rc != 1) {
EVP_PKEY_free(pkey);
EVP_MD_CTX_free(mdctx);
strError = "Failed to init signature verification";
return false;
}
rc = EVP_DigestVerify(mdctx,
sigBytes.data(), sigBytes.size(),
(const unsigned char*)message.data(), message.size());
EVP_PKEY_free(pkey);
EVP_MD_CTX_free(mdctx);
if (rc == 1) {
printf("Snapshot manifest signature VERIFIED\n");
} else if (rc == 0) {
strError = "Snapshot manifest signature INVALID — possible tampering detected";
return false;
} else {
// rc < 0 means error (e.g., placeholder zero pubkey not yet deployed)
printf("WARNING: Snapshot manifest signature verification error (rc=%d). "
"Signing key may not be deployed yet. Proceeding without verification.\n", rc);
}
} else {
printf("WARNING: Snapshot manifest has no signature — loading WITHOUT signature verification\n");
}
return true;
}
@@ -670,34 +672,19 @@ bool DownloadBootstrap(const std::string& host,
{
bool gotBlockFile = false;
// Try downloading bootstrap.tar.gz first
// Bootstrap server is on clearnet — bypass Tor proxy for DNS + HTTP
// FastImport removed (commit bdb7253). v2 UTXO snapshot is the ONLY
// supported sync path. Skip the legacy tarball fallback entirely so we
// never hit /triangles-bootstrap.tar.gz (404 since 2026-06-19 cleanup)
// or /tri-bootstrap.tar.gz (also gone; was the URL in the old filelist.txt).
// The remaining path below reads filelist.txt → downloads utxo-snapshot.bin.
const bool noProxy = true;
fs::path tmpTarGz = dataDir / "bootstrap.tar.gz.tmp";
std::string tarUrl = std::string(BASE_PATH) + "triangles-bootstrap.tar.gz";
printf("DownloadBootstrap(): attempting tar.gz download from %s%s\n", host.c_str(), tarUrl.c_str());
bool tarDownloaded = DownloadFile(host, tarUrl, tmpTarGz, progressFn, strError, noProxy);
printf("DownloadBootstrap(): tarDownloaded=%d result=%s\n", tarDownloaded, strError.c_str());
if (tarDownloaded) {
bool extractOk = ExtractTarGz(tmpTarGz, dataDir, strError);
fs::remove(tmpTarGz);
if (extractOk && fs::exists(dataDir / "blk0001.dat"))
gotBlockFile = true;
// If extraction failed, fall through to legacy path
}
if (!gotBlockFile) {
// Fallback: try filelist.txt + individual file downloads
// Try filelist.txt — should contain only utxo-snapshot.bin (v2).
std::string fallbackError;
std::vector<std::string> files;
if (!FetchFileList(host, files, fallbackError, noProxy)) {
if (!tarDownloaded)
strError = strError + " (fallback also failed: " + fallbackError + ")";
else
strError = "Extraction failed: " + strError + " (fallback also failed: " + fallbackError + ")";
strError = "filelist.txt unavailable: " + fallbackError;
return false;
}
@@ -720,7 +707,7 @@ bool DownloadBootstrap(const std::string& host,
// Check if the archive included a trusted pre-built index for the active
// backend with a valid snapshot.manifest. If verified, keep it to skip the
// multi-hour FastImportBlockFile() rebuild.
// multi-hour rebuild (fast-import removed; UTXO snapshot is the only sync path).
fs::path chainDbPath = GetChainDataDir();
fs::path database = dataDir / "database";
fs::path manifestPath = dataDir / "snapshot.manifest";
@@ -753,7 +740,7 @@ bool DownloadBootstrap(const std::string& host,
if (!keepIndex) {
// No valid manifest or verification failed - delete the index.
// FastImportBlockFile() will rebuild from blk0001.dat on next startup.
// The block index will be rebuilt from the UTXO snapshot on next startup.
printf("Bootstrap: removing extracted %s/ (will rebuild index from blk0001.dat)\n",
GetChainDataDir().filename().string().c_str());
if (fs::exists(chainDbPath))
@@ -771,15 +758,312 @@ bool DownloadBootstrap(const std::string& host,
return true;
}
namespace {
// Try to find the canonical UTXO snapshot entry in the bootstrap server's
// manifest.json. Looks for an entry of type "utxo_snapshot" and extracts
// its filename + expected SHA256. Returns true on success.
//
// We deliberately do a simple substring scan rather than full JSON parsing:
// the manifest is operator-controlled, the format is stable, and adding a
// JSON dependency for ~50 lines of code isn't worth it.
//
// On failure, the caller falls back to the legacy "utxo-snapshot.bin" URL,
// which the bootstrap server symlinks to the canonical file.
// Trusted signer addresses for snapshot manifests. A snapshot is accepted
// iff its manifest's signing_address matches one of these AND its signature
// verifies under Triangles' compact-message protocol.
static const char* TRUSTED_SNAPSHOT_SIGNERS[] = {
"TG8f76yktTxDrT7JJymY3wVAusXiD3fVvX", // Sami's snapshot publisher key
};
static const size_t NUM_TRUSTED_SNAPSHOT_SIGNERS =
sizeof(TRUSTED_SNAPSHOT_SIGNERS) / sizeof(TRUSTED_SNAPSHOT_SIGNERS[0]);
bool IsTrustedSnapshotSigner(const std::string& addr)
{
for (size_t i = 0; i < NUM_TRUSTED_SNAPSHOT_SIGNERS; ++i)
if (addr == TRUSTED_SNAPSHOT_SIGNERS[i])
return true;
return false;
}
// Verify a Triangles signed-message compact signature. Returns true iff:
// - The address is valid
// - The signature is valid base64
// - The compact signature recovers to a public key whose hash160 matches
// the address's keyID
// - The hash being verified is Hash(strMessageMagic || message)
//
// Mirrors verifymessage RPC. Caller separately checks trust.
bool VerifySignedMessage(const std::string& strAddress,
const std::string& strSignatureB64,
const std::string& strMessage,
std::string& strError)
{
CTrianglesAddress addr(strAddress);
if (!addr.IsValid()) {
strError = "Invalid signer address: " + strAddress;
return false;
}
CKeyID keyID;
if (!addr.GetKeyID(keyID)) {
strError = "Address does not refer to a key: " + strAddress;
return false;
}
bool fInvalid = false;
std::vector<unsigned char> vchSig = DecodeBase64(strSignatureB64.c_str(), &fInvalid);
if (fInvalid) {
strError = "Malformed base64 in signature";
return false;
}
CDataStream ss(SER_GETHASH, 0);
ss << strMessageMagic;
ss << strMessage;
CKey key;
if (!key.SetCompactSignature(Hash(ss.begin(), ss.end()), vchSig)) {
strError = "Signature does not verify (recovered key mismatch or malformed sig)";
return false;
}
if (key.GetPubKey().GetID() != keyID) {
strError = "Signature recovered to a different key than the claimed signer";
return false;
}
return true;
}
// Extract a string field value from a small JSON object (subset).
std::string ExtractJsonString(const std::string& json, const std::string& field)
{
std::string key = "\"" + field + "\"";
size_t pos = json.find(key);
if (pos == std::string::npos) return "";
pos += key.size();
while (pos < json.size() && (json[pos] == ' ' || json[pos] == ':' || json[pos] == '\t'))
pos++;
if (pos >= json.size() || json[pos] != '\"') return "";
pos++;
size_t end = json.find('\"', pos);
if (end == std::string::npos) return "";
return json.substr(pos, end - pos);
}
bool FindCanonicalSnapshotInManifest(const std::string& manifestText,
std::string& outFilename,
std::string& outSha256,
std::string& outManifestFilename,
std::string& strError)
{
// Look for the "utxo_snapshot" file entry, e.g.:
// "utxo-snapshot-2207680.utx": {
// ...
// "type": "utxo_snapshot",
// "sha256": "eeefe107...",
// ...
// }
size_t typePos = manifestText.find("\"utxo_snapshot\"");
if (typePos == std::string::npos) {
strError = "manifest.json has no utxo_snapshot entry";
return false;
}
// Walk backwards from the typePos to find the start of this file's block.
// Format: "filename": { ... "type": "utxo_snapshot" ...
// We scan for the nearest preceding '"' followed by ':' that introduces a
// top-level file entry. Simple heuristic: find the line containing the
// type marker, then search backwards for the file key.
size_t entryStart = manifestText.rfind('"', typePos);
if (entryStart == std::string::npos || entryStart == 0) {
strError = "malformed manifest.json (no filename before utxo_snapshot entry)";
return false;
}
// Skip the opening quote
size_t filenameStart = entryStart + 1;
size_t filenameEnd = manifestText.find('"', filenameStart);
if (filenameEnd == std::string::npos) {
strError = "malformed manifest.json (unterminated filename)";
return false;
}
outFilename = manifestText.substr(filenameStart, filenameEnd - filenameStart);
// Within this block, extract the sha256.
// Walk forward from the typePos to find the matching closing brace of the
// entry. (Manifest is shallow, so a naive brace-count is fine.)
size_t braceStart = manifestText.find('{', filenameEnd);
if (braceStart == std::string::npos) {
strError = "malformed manifest.json (no body after filename)";
return false;
}
int depth = 0;
size_t bodyEnd = braceStart;
for (size_t i = braceStart; i < manifestText.size(); ++i) {
if (manifestText[i] == '{') depth++;
else if (manifestText[i] == '}') {
depth--;
if (depth == 0) { bodyEnd = i; break; }
}
}
if (depth != 0) {
strError = "malformed manifest.json (unbalanced braces in entry)";
return false;
}
std::string entry = manifestText.substr(braceStart, bodyEnd - braceStart);
size_t shaPos = entry.find("\"sha256\"");
if (shaPos == std::string::npos) {
strError = "manifest entry has no sha256 field";
return false;
}
size_t valStart = entry.find('"', shaPos + 8);
if (valStart == std::string::npos) {
strError = "malformed manifest.json (no sha256 value)";
return false;
}
valStart++;
size_t valEnd = entry.find('"', valStart);
if (valEnd == std::string::npos) {
strError = "malformed manifest.json (unterminated sha256 value)";
return false;
}
outSha256 = entry.substr(valStart, valEnd - valStart);
// Extract manifest filename (optional).
outManifestFilename.clear();
size_t manPos = entry.find("\"manifest\"");
if (manPos != std::string::npos) {
size_t mvStart = entry.find('\"', manPos + 10);
if (mvStart != std::string::npos) {
mvStart++;
size_t mvEnd = entry.find('\"', mvStart);
if (mvEnd != std::string::npos)
outManifestFilename = entry.substr(mvStart, mvEnd - mvStart);
}
}
return true;
}
// Read an entire file into a string. Empty string on error.
std::string ReadFileToString(const fs::path& path)
{
FILE* f = fopen(path.string().c_str(), "rb");
if (!f) return "";
fseek(f, 0, SEEK_END);
long sz = ftell(f);
if (sz < 0) { fclose(f); return ""; }
fseek(f, 0, SEEK_SET);
std::string s(sz, '\0');
size_t nread = fread(&s[0], 1, sz, f);
s.resize(nread);
fclose(f);
return s;
}
// Compute the SHA256 of a file, return as lowercase hex string.
std::string Sha256OfFile(const fs::path& path)
{
FILE* f = fopen(path.string().c_str(), "rb");
if (!f) return "";
SHA256_CTX ctx;
SHA256_Init(&ctx);
unsigned char buf[64 * 1024];
size_t n;
while ((n = fread(buf, 1, sizeof(buf), f)) > 0)
SHA256_Update(&ctx, buf, n);
fclose(f);
unsigned char out[SHA256_DIGEST_LENGTH];
SHA256_Final(out, &ctx);
static const char hex[] = "0123456789abcdef";
std::string s(SHA256_DIGEST_LENGTH * 2, '0');
for (int i = 0; i < SHA256_DIGEST_LENGTH; ++i) {
s[2*i] = hex[(out[i] >> 4) & 0xF];
s[2*i + 1] = hex[out[i] & 0xF];
}
return s;
}
} // anonymous namespace
bool DownloadUtxoSnapshot(const std::string& host,
const fs::path& dataDir,
ProgressCallback progressFn,
std::string& strError)
{
const bool noProxy = true;
const char* snapshotFilename = "utxo-snapshot.bin";
// Download utxo-snapshot.bin to a temp file
// Step 1: discover the canonical snapshot filename + expected SHA256 +
// per-snapshot manifest filename from the big manifest.json. Falls back
// to legacy URL if manifest unavailable.
std::string snapshotFilename = "utxo-snapshot.bin";
std::string expectedSha256;
std::string snapshotManifestFilename;
bool haveManifest = false;
fs::path tmpManifest = dataDir / "manifest.json.tmp";
if (DownloadFile(host, "manifest.json", tmpManifest, nullptr, strError, noProxy)) {
std::string text = ReadFileToString(tmpManifest);
fs::remove(tmpManifest);
std::string mFile, mSha, mManifest;
std::string mErr;
if (FindCanonicalSnapshotInManifest(text, mFile, mSha, mManifest, mErr)) {
snapshotFilename = mFile;
expectedSha256 = mSha;
snapshotManifestFilename = mManifest;
haveManifest = true;
printf("Bootstrap: manifest declares canonical snapshot %s (sha256=%s)\n",
snapshotFilename.c_str(), expectedSha256.substr(0, 16).c_str());
} else {
printf("Bootstrap: manifest parse failed (%s) — falling back to legacy URL\n",
mErr.c_str());
}
} else {
printf("Bootstrap: no manifest.json available — falling back to legacy URL\n");
strError.clear();
}
// Step 2: verify the per-snapshot manifest's signature. This is the
// AUTHENTICATION gate — the signature attests that the listed snapshot
// file came from a trusted operator. No checkpoint required; signature
// alone proves authenticity.
if (!snapshotManifestFilename.empty()) {
fs::path tmpSnapManifest = dataDir / "snapshot-manifest.tmp";
if (!DownloadFile(host, snapshotManifestFilename, tmpSnapManifest, nullptr, strError, noProxy)) {
fs::remove(tmpSnapManifest);
return false;
}
std::string snapManifestText = ReadFileToString(tmpSnapManifest);
fs::remove(tmpSnapManifest);
std::string signerAddr = ExtractJsonString(snapManifestText, "signing_address");
std::string message = ExtractJsonString(snapManifestText, "message");
std::string signature = ExtractJsonString(snapManifestText, "signature");
std::string declaredSha = ExtractJsonString(snapManifestText, "snapshot_sha256");
if (signerAddr.empty() || message.empty() || signature.empty()) {
strError = "per-snapshot manifest missing required fields (signing_address/message/signature)";
return false;
}
if (!IsTrustedSnapshotSigner(signerAddr)) {
strError = "snapshot manifest signer " + signerAddr + " is not in trusted signers list";
return false;
}
std::string vErr;
if (!VerifySignedMessage(signerAddr, signature, message, vErr)) {
strError = "snapshot signature verification failed: " + vErr;
return false;
}
if (!declaredSha.empty())
expectedSha256 = declaredSha;
printf("Bootstrap: snapshot signature verified (signer=%s)\n", signerAddr.c_str());
} else {
printf("Bootstrap: WARNING — no per-snapshot manifest available; "
"loading snapshot WITHOUT signature verification\n");
}
// Step 3: download the canonical snapshot file.
fs::path tmpPath = dataDir / "utxo-snapshot.bin.tmp";
std::string urlPath = std::string(BASE_PATH) + snapshotFilename;
@@ -790,17 +1074,35 @@ bool DownloadUtxoSnapshot(const std::string& host,
return false;
}
// Step 4: verify the downloaded file's SHA256 against the manifest.
if (!expectedSha256.empty()) {
std::string actualSha = Sha256OfFile(tmpPath);
if (actualSha.empty()) {
strError = "Cannot read downloaded snapshot for SHA256 verification";
fs::remove(tmpPath);
return false;
}
if (actualSha != expectedSha256) {
strError = "Snapshot SHA256 mismatch: expected " + expectedSha256
+ ", got " + actualSha
+ " (manifest/snapshot tampering or server misconfiguration)";
fs::remove(tmpPath);
return false;
}
printf("Bootstrap: snapshot SHA256 verified (%s)\n", actualSha.substr(0, 16).c_str());
}
printf("Bootstrap: UTXO snapshot downloaded, loading into database...\n");
// Load the snapshot into a fresh active chain DB
if (!UtxoSnapshot::LoadSnapshot(tmpPath, dataDir, strError)) {
// Step 5: load the snapshot. requireCheckpoint is FALSE — signature is
// the authentication gate; checkpoints would force snapshots only at
// specific heights. Signature alone is sufficient.
if (!UtxoSnapshot::LoadSnapshot(tmpPath, dataDir, strError, /*requireCheckpoint=*/false)) {
fs::remove(tmpPath);
return false;
}
// Clean up the temp file
fs::remove(tmpPath);
printf("Bootstrap: UTXO snapshot loaded successfully.\n");
return true;
}
src/init.cpp
+398 -28
View File
@@ -19,6 +19,8 @@
#include "tor/tor_embedded.h"
#include "tor/onion_v3.h"
#include "tor/tor_process.h"
#include "i2p/i2p_embedded.h"
#include "i2p/i2pseed.h"
#ifdef ENABLE_ZMQ
#include "zmqpublishnotifier.h"
#endif
@@ -28,13 +30,21 @@
#include <memory>
#include <thread>
#include <vector>
// Forward declaration: InitError / InitWarning are defined further down
// in this file but referenced by AppInit (line ~423) before the definition.
static bool InitError(const std::string& str);
static bool InitWarning(const std::string& str);
#include <filesystem>
#include <fstream>
#include <boost/interprocess/sync/file_lock.hpp>
#include <algorithm>
#include <openssl/crypto.h>
#ifndef WIN32
#include <signal.h>
#include <sys/file.h>
#include <fcntl.h>
#include <unistd.h>
#endif
// Windows.h (transitively included) defines these as macros, clobbering Checkpoints:: enum values.
@@ -49,9 +59,41 @@
#endif
using namespace std;
using namespace boost;
namespace fs = std::filesystem;
namespace {
// Acquire an exclusive, non-blocking advisory lock on the datadir .lock file
// and hold it for the lifetime of the process. Replaces
// boost::interprocess::file_lock. The descriptor/handle is intentionally never
// released — the OS drops the lock automatically when the process exits.
bool LockDataDirectory(const std::filesystem::path& pathLockFile)
{
#ifdef WIN32
HANDLE hFile = CreateFileA(pathLockFile.string().c_str(),
GENERIC_READ | GENERIC_WRITE, FILE_SHARE_READ,
nullptr, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, nullptr);
if (hFile == INVALID_HANDLE_VALUE)
return false;
OVERLAPPED ov = {};
if (!LockFileEx(hFile, LOCKFILE_EXCLUSIVE_LOCK | LOCKFILE_FAIL_IMMEDIATELY,
0, MAXDWORD, MAXDWORD, &ov)) {
CloseHandle(hFile);
return false;
}
return true; // handle held until process exit
#else
int fd = open(pathLockFile.string().c_str(), O_RDWR | O_CREAT, 0644);
if (fd < 0)
return false;
if (flock(fd, LOCK_EX | LOCK_NB) != 0) {
close(fd);
return false;
}
return true; // fd held until process exit
#endif
}
} // namespace
std::unique_ptr<CWallet> pwalletMain;
CClientUIInterface uiInterface;
std::string strWalletFileName;
@@ -102,6 +144,97 @@ void ExitTimeout(void* parg)
#endif
}
// Wait up to maxWaitSec for at least minPeers peers to have reported their
// chain height via the version handshake. Returns the median peer height, or
// -1 if we couldn't get enough peers (timeout, no peers, all nStartingHeight=-1).
int WaitForPeerHeights(int minPeers, int maxWaitSec)
{
const int pollIntervalMs = 500;
const int64_t deadline = GetTimeMillis() + (int64_t)maxWaitSec * 1000;
while (GetTimeMillis() < deadline && !fRequestShutdown) {
std::vector<int> heights;
{
LOCK(cs_vNodes);
for (CNode* pnode : vNodes) {
if (pnode && pnode->nStartingHeight > 0)
heights.push_back(pnode->nStartingHeight);
}
}
if ((int)heights.size() >= minPeers) {
std::sort(heights.begin(), heights.end());
int median = heights[heights.size() / 2];
printf("AutoRebuild: got %zu peer heights; median=%d\n", heights.size(), median);
return median;
}
MilliSleep(pollIntervalMs);
}
std::vector<int> heights;
{
LOCK(cs_vNodes);
for (CNode* pnode : vNodes) {
if (pnode && pnode->nStartingHeight > 0)
heights.push_back(pnode->nStartingHeight);
}
}
if (heights.empty()) {
printf("AutoRebuild: no peers reported heights after %ds\n", maxWaitSec);
return -1;
}
std::sort(heights.begin(), heights.end());
int median = heights[heights.size() / 2];
printf("AutoRebuild: timed out with %zu peers; median=%d\n", heights.size(), median);
return median;
}
// If -autorerebuild is set and our local chain is more than that many blocks
// behind the median peer height, wipe the chain DB (preserving wallet.dat +
// onion + smsg state) and request shutdown. On restart, the daemon sees no
// chain DB and the snapshot path takes over.
void MaybeAutoRebuild(int thresholdBlocks)
{
if (thresholdBlocks <= 0)
return;
if (nBestHeight < 0) {
printf("AutoRebuild: local nBestHeight unset — skipping\n");
return;
}
printf("AutoRebuild: enabled (threshold=%d blocks). Local chain tip: %d\n",
thresholdBlocks, nBestHeight);
int medianPeer = WaitForPeerHeights(/*minPeers=*/3, /*maxWaitSec=*/60);
if (medianPeer <= 0) {
printf("AutoRebuild: could not get peer heights — skipping rebuild\n");
return;
}
int lag = medianPeer - nBestHeight;
printf("AutoRebuild: peer median=%d, local=%d, lag=%d\n",
medianPeer, nBestHeight, lag);
if (lag < thresholdBlocks) {
printf("AutoRebuild: lag %d < threshold %d — no rebuild needed\n",
lag, thresholdBlocks);
return;
}
printf("\n*** AutoRebuild: chain is %d blocks behind — wiping chain DB ***\n", lag);
printf("*** Preserving wallet.dat, smsgDB, onion state. ***\n");
printf("*** Daemon will shutdown; restart to load signed UTXO snapshot. ***\n\n");
WipeChainDataDir();
fs::path blkPath = GetDataDir() / "blk0001.dat";
if (fs::exists(blkPath)) {
fs::remove(blkPath);
printf("AutoRebuild: removed stale %s\n", blkPath.string().c_str());
}
StartShutdown();
}
void StartShutdown()
{
fRequestShutdown = true;
@@ -241,9 +374,14 @@ void Shutdown(void* parg)
pScriptCheckQueue.reset();
}
// Stop the I2P SAM session and its accept loop, then the i2pd router.
StopI2P();
StopEmbeddedI2P();
// NOW safe to destroy Tor state - all threads have stopped
ShutdownTorV3();
StopEmbeddedTor();
StopEmbeddedI2P();
#ifdef ENABLE_ZMQ
if (pzmqNotifier)
@@ -321,6 +459,21 @@ bool AppInit(int argc, char* argv[])
}
ReadConfigFile(mapArgs, mapMultiArgs);
// AUDIT: If notorious=1 or -notor was set in triangles.conf, scream
// loudly. This is the silent path that put DNS2 on a 5+ day clearnet
// fork in 2026-06-23 — operator flipped it for troubleshooting, never
// reverted it, and the daemon happily started in clearnet-only mode.
// We refuse to proceed unless -recovery-mode=1 is ALSO set, even if
// the flag was set in the config file rather than on the command line.
if (mapArgs.count("-notor") && !GetBoolArg("-recovery-mode", false)) {
return InitError(_(
"-notor=1 found in triangles.conf or command line. Triangles is "
"Tor-native; running without Tor is unsafe and produces silent "
"clearnet forks (see 2026-06-23 DNS2 incident). If this is an "
"explicit recovery operation, pass -recovery-mode=1 on the command "
"line (in addition to the config file setting) to acknowledge."));
}
if (mapArgs.count("-?") || mapArgs.count("--help"))
{
// First part of help message is specific to trianglesd / RPC client
@@ -414,16 +567,22 @@ std::string HelpMessage()
" -dbcache=<n> " + _("Set database cache size in megabytes (default: 25)") + "\n" +
" -dblogsize=<n> " + _("Set database disk log size in megabytes (default: 100)") + "\n" +
" -timeout=<n> " + _("Specify connection timeout in milliseconds (default: 5000)") + "\n" +
" -torconnecttimeout=<n> " + _("Max time (ms) for the SOCKS5 handshake with the Tor proxy (send+recv of SOCKS5 init/auth/connect). Bounds how long a dead/slow .onion can stall the connector thread (default: 60000, range 5000-180000)") + "\n" +
//" -proxy=<ip:port> " + _("Connect through socks proxy") + "\n" +
//" -socks=<n> " + _("Select the version of socks proxy to use (4-5, default: 5)") + "\n" +
" -tor=<ip:port> " + _("Use proxy to reach tor hidden services (default: same as -proxy)") + "\n"
" -notor " + _("Disable Tor (WARNING: wallet will not start - Tor is required)") + "\n" +
" -notor " + _("Disable Tor - run in clearnet-only mode (no .onion connectivity)") + "\n" +
" -torsocks=<port> " + _("Set embedded or managed Tor SOCKS proxy port (default: 19099)") + "\n" +
" -torhiddenservice " + _("Enable the managed Tor hidden service (default: 1)") + "\n" +
" -torhsport=<port> " + _("Set embedded or managed Tor hidden service port (default: wallet listen port)") + "\n" +
" -torhsport=<port> " + _("Set embedded or managed Tor hidden service port (default: wallet listen port)") + "\n"
" -i2p " + _("Enable embedded I2P router for .b32.i2p connectivity (default: 1)") + "\n"
" -i2psocks=<port> " + _("Set embedded I2P SOCKS proxy port (default: 19100)") + "\n"
" -i2psam=<port> " + _("Set embedded I2P SAM bridge port (default: 7656)") + "\n"
" -i2phsport=<port> " + _("Set I2P server tunnel forward port (default: wallet listen port)") + "\n" +
//" -dns " + _("Allow DNS lookups for -addnode, -seednode and -connect") + "\n" +
" -port=<port> " + _("Listen for connections on <port> (default: 24112 or testnet: 24111)") + "\n" +
" -maxconnections=<n> " + _("Maintain at most <n> connections to peers (default: 125)") + "\n" +
" -maxoutboundconnections=<n> " + _("Maximum outbound connections (default: 8, range 4-32)") + "\n" +
" -addnode=<ip> " + _("Add a node to connect to and attempt to keep the connection open") + "\n" +
" -connect=<ip> " + _("Connect only to the specified node(s)") + "\n" +
" -seednode=<ip> " + _("Connect to a node to retrieve peer addresses, and disconnect") + "\n" +
@@ -440,6 +599,7 @@ std::string HelpMessage()
" -onionseed " + _("Find peers using .onion seeds (default: 1 unless -connect)") + "\n" +
" -seedurl=<host> " + _("HTTP seed list host (default: seeds.cryptographic-triangles.org)") + "\n" +
" -noseedurl " + _("Disable HTTP seed list fetch on startup") + "\n" +
" -autorerebuild=<n> " + _("If our chain is more than <n> blocks behind peers, wipe chain DB and shutdown for clean restart (default: 0=disabled)") + "\n" +
" -banscore=<n> " + _("Threshold for disconnecting misbehaving peers (default: 100)") + "\n" +
" -bantime=<n> " + _("Number of seconds to keep misbehaving peers from reconnecting (default: 86400)") + "\n" +
" -par=<n> " + _("Set the number of script verification threads (default: auto, 0 = auto, 1 = single-threaded)") + "\n" +
@@ -687,6 +847,21 @@ bool AppInit2()
nConnectTimeout = nNewTimeout;
}
// SOCKS5/Tor negotiation timeout. Separate from -timeout (which only covers
// the instant local connect to the Tor SOCKS proxy); this bounds the
// SOCKS5 handshake (send+recv of init/auth/connect). On a dead/slow .onion
// the recv() in Socks5() would otherwise block until Tor's own ~120s
// SocksTimeout fires, holding an outbound connection slot.
if (mapArgs.count("-torconnecttimeout"))
{
int nTorTimeout = GetArg("-torconnecttimeout", 60000);
if (IsValidSocksNegotiationTimeout(nTorTimeout))
nSocksNegotiationTimeout = nTorTimeout;
else
InitWarning("Ignoring -torconnecttimeout=" + mapArgs["-torconnecttimeout"] +
": out of range (5000..180000 ms), using default 60000");
}
if (mapArgs.count("-paytxfee"))
{
if (!ParseMoney(mapArgs["-paytxfee"], nTransactionFee))
@@ -698,6 +873,15 @@ bool AppInit2()
fConfChange = GetBoolArg("-confchange", false);
fEnforceCanonical = GetBoolArg("-enforcecanonical", true);
// Validate -maxoutboundconnections (range 4-32, default 8)
if (mapArgs.count("-maxoutboundconnections"))
{
int nMaxOutboundConn = GetArg("-maxoutboundconnections", 8);
if (nMaxOutboundConn < 4 || nMaxOutboundConn > 32)
InitWarning("Ignoring -maxoutboundconnections=" + mapArgs["-maxoutboundconnections"] +
": out of range (4..32), using default 8");
}
int nScriptCheckThreads = GetArg("-par", 0);
if (nScriptCheckThreads <= 0)
nScriptCheckThreads = std::thread::hardware_concurrency();
@@ -738,8 +922,7 @@ bool AppInit2()
fs::path pathLockFile = GetDataDir() / ".lock";
FILE* file = fopen(pathLockFile.string().c_str(), "a"); // empty lock file; created if it doesn't exist.
if (file) fclose(file);
static boost::interprocess::file_lock lock(pathLockFile.string().c_str());
if (!lock.try_lock())
if (!LockDataDirectory(pathLockFile))
return InitError(strprintf(_("Cannot obtain a lock on data directory %s. Triangles is probably already running."), strDataDir.c_str()));
#if !defined(WIN32) && !defined(QT_GUI)
@@ -927,7 +1110,8 @@ bool AppInit2()
// v5.9.5: P2P UTXO snapshot fetch is the default for fresh installs (Step 11.6).
// The legacy clearnet HTTP bootstrap only runs when the user explicitly requests
// it via -bootstrap, or when -snapshot=0 disables the P2P fetcher.
#ifndef QT_GUI
// Bootstrap auto-download works for both GUI and daemon.
// GUI users get the same automatic bootstrap on fresh installs.
{
bool wantsBootstrap = GetBoolArg("-bootstrap", false);
bool noBootstrap = GetBoolArg("-nobootstrap", false);
@@ -935,13 +1119,11 @@ bool AppInit2()
fs::path dataPath = GetDataDir();
bool needsBootstrap = Bootstrap::NeedsBootstrap(dataPath);
if (needsBootstrap && !noBootstrap && !snapshotMode) {
printf("Bootstrap: no blockchain data found — downloading automatically.\n");
if (needsBootstrap && !noBootstrap) {
printf("Bootstrap: no blockchain data found — downloading UTXO snapshot automatically.\n");
printf("Bootstrap: (use -nobootstrap to skip)\n");
uiInterface.InitMessage(_("Downloading UTXO snapshot..."));
wantsBootstrap = true;
} else if (needsBootstrap && snapshotMode && !wantsBootstrap) {
printf("Bootstrap: no blockchain data found — will fetch UTXO snapshot via P2P after network start.\n");
printf("Bootstrap: (use -bootstrap for legacy clearnet HTTP bootstrap, or -snapshot=0 to disable P2P fetcher)\n");
}
if (wantsBootstrap)
@@ -951,13 +1133,24 @@ bool AppInit2()
std::string host = Bootstrap::DEFAULT_HOST;
std::string strError;
auto progressFn = [](int64_t bytesDownloaded, int64_t totalBytes) {
int64_t lastGuiUpdate = 0;
auto progressFn = [&lastGuiUpdate](int64_t bytesDownloaded, int64_t totalBytes) {
if (totalBytes > 0) {
printf("\rBootstrap: %lld / %lld MB (%lld%%)",
(long long)(bytesDownloaded / (1024*1024)),
(long long)(totalBytes / (1024*1024)),
(long long)((bytesDownloaded * 100) / totalBytes));
fflush(stdout);
// Update GUI status bar every ~1 MB
int64_t now = GetTimeMillis();
if (now - lastGuiUpdate > 1000) {
lastGuiUpdate = now;
std::string msg = strprintf("Downloading blockchain: %lld / %lld MB (%lld%%)",
(long long)(bytesDownloaded / (1024*1024)),
(long long)(totalBytes / (1024*1024)),
(long long)((bytesDownloaded * 100) / totalBytes));
uiInterface.InitMessage(msg);
}
}
};
@@ -999,7 +1192,6 @@ bool AppInit2()
strprintf("host=%s success=%d utxo_snapshot=%d", host.c_str(), success, triedUtxoSnapshot));
}
} // end bootstrap scope
#endif
// ********************************************************* Step 6c: manual UTXO snapshot loading
// If utxo-snapshot.bin exists in data dir and the chain DB hasn't been
@@ -1013,8 +1205,13 @@ bool AppInit2()
printf("Found utxo-snapshot.bin — loading UTXO snapshot...\n");
uiInterface.InitMessage(_("Loading UTXO snapshot..."));
// Local file load: skip the checkpoint gate. The operator has
// filesystem access, so the trust model is already equivalent
// to direct chain state modification — a malicious local file
// is no worse than a malicious chain DB. P2P-delivered
// snapshots (SnapshotNet) keep the checkpoint gate on.
std::string strError;
if (UtxoSnapshot::LoadSnapshot(snapshotFile, dataPath, strError)) {
if (UtxoSnapshot::LoadSnapshot(snapshotFile, dataPath, strError, /*requireCheckpoint=*/false)) {
printf("UTXO snapshot loaded successfully.\n");
} else {
printf("UTXO snapshot load failed: %s\n", strError.c_str());
@@ -1023,15 +1220,33 @@ bool AppInit2()
}
}
// ********************************************************* Step 6d: optional LevelDB -> RocksDB chain DB migration
if (GetBoolArg("-migratechaindb", false) || GetBoolArg("-migratechaindbforce", false))
// ********************************************************* Step 6d: LevelDB -> RocksDB chain DB migration
// Runs when explicitly requested (-migratechaindb[force]) OR automatically
// when RocksDB is the active backend and the only chain DB present is a
// legacy LevelDB (txleveldb). This makes the RocksDB default transparent
// for existing nodes: their chain state is copied (and verified) into a new
// rocksdb/ directory on first launch, leaving the LevelDB source untouched
// as a fallback. MaybeMigrateLevelDbToRocksDb() is a no-op when there is no
// LevelDB source or a RocksDB directory already exists, so it is safe to
// call on every startup.
{
bool fExplicit = GetBoolArg("-migratechaindb", false) ||
GetBoolArg("-migratechaindbforce", false);
bool fAuto = IsRocksDbChainBackend() &&
fs::exists(GetDataDir() / "txleveldb") &&
!fs::exists(GetDataDir() / "rocksdb");
if (fExplicit || fAuto)
{
uiInterface.InitMessage(_("Migrating chain database to RocksDB..."));
if (fAuto && !fExplicit)
printf("ChainDB: RocksDB backend active with a legacy LevelDB present; "
"migrating automatically.\n");
std::string strMigrateError;
bool fForce = GetBoolArg("-migratechaindbforce", false);
if (!MaybeMigrateLevelDbToRocksDb(fForce, strMigrateError))
return InitError(strprintf(_("Chain DB migration failed: %s"), strMigrateError.c_str()));
}
}
// ********************************************************* Step 7: load blockchain
@@ -1052,7 +1267,7 @@ bool AppInit2()
}
// Handle -reindex: delete the chain DB so it gets rebuilt from the raw
// blk*.dat files via FastImportBlockFile(). This recalculates money
// blk*.dat files. This recalculates money
// supply, tx index, and UTXO set from scratch. Backend-agnostic via
// WipeChainDataDir(), which resolves the directory per the configured
// -chaindb backend.
@@ -1069,18 +1284,47 @@ bool AppInit2()
if (!LoadBlockIndex())
return InitError(_("Error loading blkindex.dat"));
// If the block index is empty but blk0001.dat exists (bootstrap download),
// fast-import: build the index directly from the block file without re-writing
// data. Batches LevelDB commits every 200K blocks for speed.
if (nBestHeight == 0 && std::filesystem::exists(GetDataDir() / "blk0001.dat")
&& mapBlockIndex.size() <= 1)
// triangles fix (pitfall #61): initialize pindexFinalized from the
// hardcoded checkpoint on startup, BEFORE the daemon opens any peer
// connections or processes any block messages.
//
// Without this, pindexFinalized stays NULL on a fresh restart even when
// we have 2.2M blocks on disk, because the auto-checkpoint code in
// ActivateBestChain() at main.cpp:2459 only sets it when
// !IsInitialBlockDownload(). If the chain tip is more than 24h stale
// (which happens on every restart with a synced chain), IsInitialBlockDownload()
// returns true and pindexFinalized never gets set.
//
// The downstream reorg guard at main.cpp:2198 short-circuits when
// pindexFinalized is NULL, which allowed a 3,755-block minority fork
// to overwrite a healthy 2,206,004-block chain on 2026-06-16. Loading
// the hardcoded checkpoint from checkpoints.cpp (block 2,205,000) on
// startup means the reorg guard is always active whenever the
// checkpointed block is in our local mapBlockIndex.
{
uiInterface.InitMessage(_("Importing bootstrap blocks..."));
printf("Block index empty but blk0001.dat exists - running fast import...\n");
int64_t nFastImportStart = GetTimeMillis();
FastImportBlockFile();
StartupPerfLog("bootstrap_fast_import", GetTimeMillis() - nFastImportStart, strprintf("bestheight=%d", nBestHeight));
CBlockIndex* pCheckpoint = Checkpoints::GetLastCheckpoint(mapBlockIndex);
if (pCheckpoint && pCheckpoint != pindexFinalized)
{
pindexFinalized = pCheckpoint;
printf("STARTUP-CHECKPOINT: pindexFinalized set to block %d (%s) from hardcoded checkpoint\n",
pindexFinalized->nHeight, pindexFinalized->GetBlockHash().ToString().substr(0,20).c_str());
}
else if (!pCheckpoint)
{
printf("STARTUP-CHECKPOINT: WARNING — hardcoded checkpoint not in local block index, pindexFinalized remains NULL\n");
}
}
// AutoRebuild: if -autorerebuild is set and we are behind peers, wipe chain DB
// and shutdown for clean restart.
MaybeAutoRebuild(GetArg("-autorerebuild", 0));
if (fRequestShutdown) {
printf("AutoRebuild: shutdown requested before chain load complete\n");
return false;
}
// Block index loaded. With fast-import removed, the only supported sync path
// is the UTXO snapshot (auto-downloaded from bootstrap or placed manually in datadir).
// as LoadBlockIndex can take several minutes, it's possible the user
// requested to kill triangles-qt during the last operation. If so, exit.
@@ -1343,6 +1587,30 @@ bool AppInit2()
#ifdef USE_UPNP
fUseUPnP = false;
#endif
} else if (GetBoolArg("-notor", false)) {
// -notor: explicit clearnet mode. Triangles is Tor-native and
// running without Tor is unsafe for normal operation — it can
// produce silent clearnet forks (see 2026-06-23 DNS2 incident,
// 5+ days on a parallel chain because -notor=1 was left on after
// troubleshooting). The flag is preserved for explicit recovery
// workflows (e.g. dumputxoset-from-clearnet when bootstrapping
// a new node) but requires an additional -recovery-mode=1
// confirmation flag so it cannot be flipped by accident.
if (!GetBoolArg("-recovery-mode", false)) {
return InitError(_(
"-notor requires -recovery-mode=1 confirmation. Triangles is Tor-native; "
"running without Tor is unsafe and produces silent clearnet forks. "
"If you need clearnet mode for bootstrap recovery or diagnostics, "
"pass BOTH -notor=1 -recovery-mode=1 on the command line."));
}
printf("WARNING: Tor disabled via -notor AND -recovery-mode=1 set. "
"Running in clearnet-only mode.\n");
printf(" .onion connections will NOT be available.\n");
printf(" This mode is for RECOVERY ONLY — exit and restart without these\n"
" flags as soon as the recovery operation completes.\n");
SetReachable(NET_IPV4, true);
SetReachable(NET_IPV6, true);
SetReachable(NET_TOR, false);
} else {
std::string torError = CTorEmbedded::GetInstance()->GetStartupError();
if (torError.empty())
@@ -1350,6 +1618,47 @@ bool AppInit2()
return InitError(strprintf(_("Tor failed to start. Triangles requires Tor to operate.\n\nDetails: %s"), torError.c_str()));
}
// ════════════════════════════════════════════════════════════════
// Embedded I2P (i2pd) startup
//
// I2P runs as a co-equal anonymity network alongside Tor. When Tor
// starts successfully (tor-native mode), I2P provides an alternative
// anonymous transport via .b32.i2p destinations. When Tor is disabled
// (-notor recovery mode), I2P is still started to maintain anonymity.
//
// I2P's SOCKS proxy (default 19100) handles outbound .i2p connections.
// A server tunnel forwards incoming I2P connections to the P2P port.
// ════════════════════════════════════════════════════════════════
if (torStarted || GetBoolArg("-notor", false)) {
uiInterface.InitMessage(_("Starting embedded I2P router..."));
int64_t nI2PStart = GetTimeMillis();
bool i2pStarted = StartEmbeddedI2P();
StartupPerfLog("i2p_start", GetTimeMillis() - nI2PStart,
strprintf("started=%d", i2pStarted));
if (i2pStarted) {
int i2pSocksPort = CI2PEmbedded::GetInstance()->GetSocksPort();
CService i2pProxyAddr("127.0.0.1", i2pSocksPort);
// Route I2P traffic through i2pd's SOCKS proxy
SetProxy(NET_I2P, i2pProxyAddr, 5);
SetReachable(NET_I2P, true);
printf("I2P-NATIVE MODE: I2P router running\n");
printf(" SOCKS proxy at 127.0.0.1:%d for .b32.i2p connections\n",
i2pSocksPort);
printf(" Dual-network anonymity: Tor (.onion) + I2P (.b32.i2p)\n");
} else {
// I2P failure is non-fatal — Tor-only operation continues.
// The daemon still works with .onion peers.
std::string i2pError = CI2PEmbedded::GetInstance()->GetStartupError();
printf("WARNING: Embedded I2P did not start. Running Tor-only.\n");
if (!i2pError.empty())
printf(" I2P error: %s\n", i2pError.c_str());
SetReachable(NET_I2P, false);
}
}
// Initialize Tor V3 identity (Ed25519 keys, onion address)
uiInterface.InitMessage(_("Initializing Tor V3 identity..."));
printf("Initializing Tor V3 onion identity...\n");
@@ -1416,6 +1725,45 @@ bool AppInit2()
if (!NewThread(ThreadTorMaintenance, nullptr))
printf("Warning: ThreadTorMaintenance could not be started\n");
}
// Bring up I2P (SAM) transport alongside Tor so the wallet has both a
// .onion and a .b32.i2p address. On by default; disable with -i2p=0.
// A bundled i2pd router is launched automatically (mirroring embedded
// Tor); if -i2psam points at a non-loopback bridge, or a router is
// already running, we use that instead.
if (GetBoolArg("-i2p", true)) {
int64_t nI2PStart = GetTimeMillis();
// Resolve the SAM endpoint (default 127.0.0.1:7656).
std::string sam = GetArg("-i2psam", "127.0.0.1:7656");
int samPort = I2P_DEFAULT_SAM_PORT;
std::string samHost = "127.0.0.1";
SplitHostPort(sam, samPort, samHost);
if (samPort <= 0) samPort = I2P_DEFAULT_SAM_PORT;
bool loopback = samHost.empty() || samHost == "127.0.0.1" || samHost == "localhost";
// Auto-launch our own i2pd only when the bridge is local.
if (loopback) {
uiInterface.InitMessage(_("Starting the I2P router..."));
if (!StartEmbeddedI2P((GetDataDir() / "i2pd").string(), samPort)) {
printf("NOTICE: bundled I2P router unavailable (%s).\n",
CI2PProcess::GetInstance()->GetLastError().c_str());
printf(" I2P will use an external router if one is running on %s.\n", sam.c_str());
}
}
uiInterface.InitMessage(_("Connecting to the I2P network..."));
bool i2pStarted = StartI2P();
StartupPerfLog("i2p_start", GetTimeMillis() - nI2PStart, strprintf("started=%d", i2pStarted));
if (i2pStarted) {
SetReachable(NET_I2P, true);
std::string i2pAddr = CI2PSession::GetInstance()->GetB32Address();
printf("I2P network enabled. Our address: %s\n", i2pAddr.c_str());
} else {
printf("NOTICE: I2P not available this session; continuing with Tor only\n");
StopEmbeddedI2P();
}
}
}
// ********************************************************* Step 9: import blocks
@@ -1465,6 +1813,28 @@ bool AppInit2()
addrman.size(), GetTimeMillis() - nStart);
StartupPerfLog("peers_load", GetTimeMillis() - nStart, strprintf("count=%d", addrman.size()));
// Add hardcoded I2P (.b32.i2p) seed addresses to the address manager.
// This enables cross-network peer discovery: Tor-connected nodes can learn
// about I2P peers and vice versa. Onion seeds are loaded separately in
// ThreadOnionSeed (net.cpp), but we add I2P seeds here during init so they
// are available immediately for the outbound connector.
{
static const char *(*strI2PSeed)[1] = fTestNet ? strTestNetI2PSeed : strMainNetI2PSeed;
int nI2PSeeds = 0;
for (unsigned int si = 0; strI2PSeed[si][0] != nullptr; si++) {
CNetAddr parsed;
if (parsed.SetSpecial(strI2PSeed[si][0])) {
int nOneDay = 24 * 3600;
CAddress addr = CAddress(CService(parsed, GetDefaultPort()));
addr.nTime = GetTime() - 3 * nOneDay - GetRand(4 * nOneDay);
addrman.Add(addr, parsed);
nI2PSeeds++;
}
}
if (nI2PSeeds > 0)
printf("Added %d hardcoded I2P (.b32.i2p) seed addresses to addrman\n", nI2PSeeds);
}
// ********************************************************* Step 11: start node
nStart = GetTimeMillis();
src/qt/qtipcserver.cpp
+81 -74
View File
@@ -1,32 +1,27 @@
// Copyright (c) 2009-2012 The Bitcoin developers
// Copyright (c) 2026 The Triangles developers
// Distributed under the MIT/X11 software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include <boost/version.hpp>
#if defined(WIN32) && BOOST_VERSION == 104900
#define BOOST_INTERPROCESS_HAS_WINDOWS_KERNEL_BOOTTIME
#define BOOST_INTERPROCESS_HAS_KERNEL_BOOTTIME
#endif
//
// Single-instance "triangles:" URI handoff. When the wallet is launched with a
// URI argument and an instance is already running, the URI is relayed to the
// running instance over a local socket; otherwise this instance becomes the
// listener. Reworked from Boost.Interprocess message queues onto Qt's
// QLocalServer/QLocalSocket (QtNetwork) — no Boost dependency.
#include "qtipcserver.h"
#include "guiconstants.h"
#include "ui_interface.h"
#include "util.h"
#include <boost/date_time/posix_time/posix_time.hpp>
#include <boost/interprocess/ipc/message_queue.hpp>
#include <boost/version.hpp>
#if defined(WIN32) && (!defined(BOOST_INTERPROCESS_HAS_WINDOWS_KERNEL_BOOTTIME) || !defined(BOOST_INTERPROCESS_HAS_KERNEL_BOOTTIME) || BOOST_VERSION < 104900)
#warning Compiling without BOOST_INTERPROCESS_HAS_WINDOWS_KERNEL_BOOTTIME and BOOST_INTERPROCESS_HAS_KERNEL_BOOTTIME uncommented in boost/interprocess/detail/tmp_dir_helpers.hpp or using a boost version before 1.49 may have unintended results see svn.boost.org/trac/boost/ticket/5392
#endif
using namespace boost;
using namespace boost::interprocess;
using namespace boost::posix_time;
#include <algorithm>
#include <cctype>
#include <string>
#include <QByteArray>
#include <QLocalServer>
#include <QLocalSocket>
#include <QString>
#if defined MAC_OSX || defined __FreeBSD__
// URI handling not implemented on OSX yet
@@ -36,35 +31,49 @@ void ipcInit(int argc, char *argv[]) { }
#else
// Local-socket server name. QLocalServer maps this to a named pipe on Windows
// and a filesystem socket on Unix.
static const QString IPC_SERVER_NAME = QStringLiteral(TRIANGLESURI_QUEUE_NAME);
static void ipcThread2(void* pArg);
static bool IsTrianglesURI(const char* arg)
{
// Case-insensitive match of the "Triangles:" scheme prefix.
return std::equal(std::begin("Triangles:"), std::end("Triangles:") - 1, arg,
[](char a, char b) {
return std::tolower(static_cast<unsigned char>(a)) ==
std::tolower(static_cast<unsigned char>(b));
});
}
static bool ipcScanCmd(int argc, char *argv[], bool fRelay)
{
// Check for URI in argv
// Check for URI in argv and relay it to a running instance, if any.
bool fSent = false;
for (int i = 1; i < argc; i++)
{
if (std::equal(std::begin("Triangles:"), std::end("Triangles:") - 1, argv[i], [](char a, char b) { return std::tolower(static_cast<unsigned char>(a)) == std::tolower(static_cast<unsigned char>(b)); }))
{
if (!IsTrianglesURI(argv[i]))
continue;
const char *strURI = argv[i];
try {
boost::interprocess::message_queue mq(boost::interprocess::open_only, TRIANGLESURI_QUEUE_NAME);
if (mq.try_send(strURI, strlen(strURI), 0))
fSent = true;
else if (fRelay)
break;
}
catch (boost::interprocess::interprocess_exception &ex) {
// don't log the "file not found" exception, because that's normal for
// the first start of the first instance
if (ex.get_error_code() != boost::interprocess::not_found_error || !fRelay)
QLocalSocket socket;
socket.connectToServer(IPC_SERVER_NAME);
if (socket.waitForConnected(1000))
{
printf("main() - boost interprocess exception #%d: %s\n", ex.get_error_code(), ex.what());
socket.write(strURI, static_cast<qint64>(strlen(strURI)));
socket.flush();
socket.waitForBytesWritten(1000);
socket.disconnectFromServer();
fSent = true;
}
else if (fRelay)
{
// No running instance accepted the URI; this process should become
// the listener instead of relaying.
break;
}
}
}
}
return fSent;
}
@@ -95,69 +104,67 @@ static void ipcThread2(void* pArg)
{
printf("ipcThread started\n");
message_queue* mq = (message_queue*)pArg;
char buffer[MAX_URI_LENGTH + 1] = "";
size_t nSize = 0;
unsigned int nPriority = 0;
QLocalServer* server = static_cast<QLocalServer*>(pArg);
// Poll for inbound connections without requiring a Qt event loop:
// waitForNewConnection(timeout) pumps the socket internally.
while (true)
{
ptime d = boost::posix_time::microsec_clock::universal_time() + millisec(100);
if (mq->timed_receive(&buffer, sizeof(buffer), nSize, nPriority, d))
if (server->waitForNewConnection(100))
{
uiInterface.ThreadSafeHandleURI(std::string(buffer, nSize));
QLocalSocket* client = server->nextPendingConnection();
if (client)
{
if (client->waitForReadyRead(1000))
{
QByteArray data = client->readAll();
if (data.size() > MAX_URI_LENGTH)
data.truncate(MAX_URI_LENGTH);
uiInterface.ThreadSafeHandleURI(std::string(data.constData(), data.size()));
MilliSleep(1000);
}
client->disconnectFromServer();
delete client;
}
}
if (fShutdown)
break;
}
// Remove message queue
message_queue::remove(TRIANGLESURI_QUEUE_NAME);
// Cleanup allocated memory
delete mq;
server->close();
delete server;
}
void ipcInit(int argc, char *argv[])
{
message_queue* mq = NULL;
char buffer[MAX_URI_LENGTH + 1] = "";
size_t nSize = 0;
unsigned int nPriority = 0;
// Clear any stale socket/pipe left by a previous crashed instance, then
// listen. If listen() fails, another instance already owns the name — in
// that case relay our own URI args (below) and don't start a server.
QLocalServer::removeServer(IPC_SERVER_NAME);
try {
mq = new message_queue(open_or_create, TRIANGLESURI_QUEUE_NAME, 2, MAX_URI_LENGTH);
// Make sure we don't lose any Triangles: URIs
for (int i = 0; i < 2; i++)
QLocalServer* server = new QLocalServer();
server->setSocketOptions(QLocalServer::UserAccessOption); // owner-only access
if (!server->listen(IPC_SERVER_NAME))
{
ptime d = boost::posix_time::microsec_clock::universal_time() + millisec(1);
if (mq->timed_receive(&buffer, sizeof(buffer), nSize, nPriority, d))
{
uiInterface.ThreadSafeHandleURI(std::string(buffer, nSize));
}
else
break;
}
// Make sure only one Triangles instance is listening
message_queue::remove(TRIANGLESURI_QUEUE_NAME);
delete mq;
mq = new message_queue(open_or_create, TRIANGLESURI_QUEUE_NAME, 2, MAX_URI_LENGTH);
}
catch (interprocess_exception &ex) {
printf("ipcInit() - boost interprocess exception #%d: %s\n", ex.get_error_code(), ex.what());
printf("ipcInit() - QLocalServer listen failed: %s\n",
server->errorString().toUtf8().constData());
delete server;
// Still try to relay any URI passed on our command line to whoever is
// listening.
ipcScanCmd(argc, argv, false);
return;
}
if (!NewThread(ipcThread, mq))
if (!NewThread(ipcThread, server))
{
delete mq;
server->close();
delete server;
return;
}
// Handle a URI passed on our own command line (relayed to the server we
// just started).
ipcScanCmd(argc, argv, false);
}
src/rpc_httpsocket.h
+200
View File
@@ -0,0 +1,200 @@
// Copyright (c) 2026 The Triangles developers.
// Distributed under the MIT/X11 software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
//
// Raw-socket transport for the JSON-RPC / REST HTTP server, replacing the
// previous Boost.Asio implementation. Provides:
//
// - CSocketIOStream : a std::iostream backed by a connected SOCKET, so the
// existing HTTP/JSON/SSE/REST code (which reads and writes std::iostream)
// is unchanged.
// - ConnectRPCSocket() : client-side connect (used by CallRPC).
// - BindRPCSockets() : create listening sockets for the RPC server.
// - SockaddrToString() : numeric host string for a peer address.
//
// TLS for the RPC port is intentionally not supported here (it was a rarely
// used Boost.Asio::ssl feature). For remote access, front the RPC port with a
// TLS terminator (stunnel / nginx) or reach it over SSH / Tor — the same
// guidance Bitcoin Core adopted when it moved its RPC server off Boost.Asio.
#ifndef TRIANGLES_RPC_HTTPSOCKET_H
#define TRIANGLES_RPC_HTTPSOCKET_H
#include "compat.h" // SOCKET, closesocket, INVALID_SOCKET, MSG_NOSIGNAL
#include <cstring>
#include <iostream>
#include <streambuf>
#include <string>
#include <vector>
#ifndef WIN32
#include <sys/socket.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
#include <arpa/inet.h>
#include <netdb.h>
#endif
// ── std::streambuf over a connected socket ──────────────────────────────────
class CSocketStreamBuf : public std::streambuf
{
public:
explicit CSocketStreamBuf(SOCKET s) : m_socket(s)
{
setg(m_in, m_in, m_in); // empty get area to start
}
protected:
// Refill the get area with one recv().
int_type underflow() override
{
if (gptr() < egptr())
return traits_type::to_int_type(*gptr());
int n = ::recv(m_socket, m_in, static_cast<int>(sizeof(m_in)), 0);
if (n <= 0)
return traits_type::eof(); // peer closed or error
setg(m_in, m_in, m_in + n);
return traits_type::to_int_type(*gptr());
}
// Bulk write (operator<< on strings lands here).
std::streamsize xsputn(const char* s, std::streamsize n) override
{
return SendAll(s, n) ? n : 0;
}
int_type overflow(int_type ch) override
{
if (traits_type::eq_int_type(ch, traits_type::eof()))
return traits_type::not_eof(ch);
char c = static_cast<char>(ch);
return SendAll(&c, 1) ? ch : traits_type::eof();
}
int sync() override { return 0; } // sends are immediate; nothing buffered
private:
bool SendAll(const char* s, std::streamsize n)
{
std::streamsize sent = 0;
while (sent < n) {
int r = ::send(m_socket, s + sent, static_cast<int>(n - sent), MSG_NOSIGNAL);
if (r <= 0)
return false;
sent += r;
}
return true;
}
SOCKET m_socket;
char m_in[8192];
};
// std::iostream that owns a CSocketStreamBuf bound to a socket. The socket
// itself is owned by the caller (AcceptedConnection / CallRPC), not closed here.
class CSocketIOStream : public std::iostream
{
public:
explicit CSocketIOStream(SOCKET s) : std::iostream(nullptr), m_buf(s)
{
rdbuf(&m_buf);
}
private:
CSocketStreamBuf m_buf;
};
// Numeric (no DNS) host string for a peer sockaddr, e.g. "127.0.0.1" or "::1".
inline std::string SockaddrToString(const struct sockaddr* sa, socklen_t salen)
{
char host[NI_MAXHOST] = {0};
if (::getnameinfo(sa, salen, host, sizeof(host), nullptr, 0, NI_NUMERICHOST) != 0)
return "unknown";
return std::string(host);
}
// Client connect to host:port. Returns INVALID_SOCKET on failure.
inline SOCKET ConnectRPCSocket(const std::string& host, int port)
{
struct addrinfo hints;
std::memset(&hints, 0, sizeof(hints));
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
struct addrinfo* res = nullptr;
const std::string portStr = std::to_string(port);
if (::getaddrinfo(host.c_str(), portStr.c_str(), &hints, &res) != 0)
return INVALID_SOCKET;
SOCKET hSocket = INVALID_SOCKET;
for (struct addrinfo* rp = res; rp != nullptr; rp = rp->ai_next) {
hSocket = ::socket(rp->ai_family, rp->ai_socktype, rp->ai_protocol);
if (hSocket == INVALID_SOCKET)
continue;
if (::connect(hSocket, rp->ai_addr, static_cast<int>(rp->ai_addrlen)) == 0)
break;
closesocket(hSocket);
hSocket = INVALID_SOCKET;
}
::freeaddrinfo(res);
return hSocket;
}
// Create listening sockets for the RPC server. When loopbackOnly is true the
// server binds the loopback interface(s) only; otherwise it binds the wildcard
// address(es). IPv4 and IPv6 are bound on separate sockets (IPV6_V6ONLY) so the
// two never conflict. Returns the bound, listening sockets; empty + strError on
// total failure (partial success — e.g. only IPv4 — is returned as success).
inline std::vector<SOCKET> BindRPCSockets(int port, bool loopbackOnly, std::string& strError)
{
std::vector<SOCKET> vListen;
struct addrinfo hints;
std::memset(&hints, 0, sizeof(hints));
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
hints.ai_flags = AI_PASSIVE; // wildcard when node == nullptr
struct addrinfo* res = nullptr;
const std::string portStr = std::to_string(port);
// "localhost" resolves to the loopback addresses (127.0.0.1 and ::1);
// nullptr + AI_PASSIVE yields the wildcard addresses.
const char* node = loopbackOnly ? "localhost" : nullptr;
int gai = ::getaddrinfo(node, portStr.c_str(), &hints, &res);
if (gai != 0) {
strError = std::string("RPC bind: getaddrinfo failed: ") + gai_strerror(gai);
return vListen;
}
for (struct addrinfo* rp = res; rp != nullptr; rp = rp->ai_next) {
if (rp->ai_family != AF_INET && rp->ai_family != AF_INET6)
continue;
SOCKET s = ::socket(rp->ai_family, rp->ai_socktype, rp->ai_protocol);
if (s == INVALID_SOCKET)
continue;
int one = 1;
::setsockopt(s, SOL_SOCKET, SO_REUSEADDR,
reinterpret_cast<const char*>(&one), sizeof(one));
if (rp->ai_family == AF_INET6) {
// Keep IPv6 sockets v6-only so a separate IPv4 socket can also bind.
::setsockopt(s, IPPROTO_IPV6, IPV6_V6ONLY,
reinterpret_cast<const char*>(&one), sizeof(one));
}
if (::bind(s, rp->ai_addr, static_cast<int>(rp->ai_addrlen)) != 0 ||
::listen(s, SOMAXCONN) != 0) {
closesocket(s);
continue;
}
vListen.push_back(s);
}
::freeaddrinfo(res);
if (vListen.empty())
strError = "RPC bind: could not bind any address (port in use?)";
return vListen;
}
#endif // TRIANGLES_RPC_HTTPSOCKET_H
src/rpcdump.cpp
+25 -29
View File
@@ -4,14 +4,15 @@
#include <iostream>
#include <fstream>
#include <iomanip>
#include <sstream>
#include <ctime>
#include "init.h" // for pwalletMain
#include "trianglesrpc.h"
#include "ui_interface.h"
#include "base58.h"
#include <boost/date_time/posix_time/posix_time.hpp>
#define printf OutputDebugStringF
using namespace json_spirit;
@@ -19,40 +20,35 @@ using namespace std;
void EnsureWalletIsUnlocked();
namespace bt = boost::posix_time;
// Extended DecodeDumpTime implementation, see this page for details:
// http://stackoverflow.com/questions/3786201/parsing-of-date-time-from-string-boost
const std::locale formats[] = {
std::locale(std::locale::classic(),new bt::time_input_facet("%Y-%m-%dT%H:%M:%SZ")),
std::locale(std::locale::classic(),new bt::time_input_facet("%Y-%m-%d %H:%M:%S")),
std::locale(std::locale::classic(),new bt::time_input_facet("%Y/%m/%d %H:%M:%S")),
std::locale(std::locale::classic(),new bt::time_input_facet("%d.%m.%Y %H:%M:%S")),
std::locale(std::locale::classic(),new bt::time_input_facet("%Y-%m-%d"))
// Accepted timestamp formats, tried in order. Replaces the boost::posix_time
// parser; std::get_time is portable (C++11) and parses against each format.
static const char* const dumptime_formats[] = {
"%Y-%m-%dT%H:%M:%SZ",
"%Y-%m-%d %H:%M:%S",
"%Y/%m/%d %H:%M:%S",
"%d.%m.%Y %H:%M:%S",
"%Y-%m-%d",
};
const size_t formats_n = sizeof(formats)/sizeof(formats[0]);
std::time_t pt_to_time_t(const bt::ptime& pt)
{
bt::ptime timet_start(boost::gregorian::date(1970,1,1));
bt::time_duration diff = pt - timet_start;
return diff.ticks()/bt::time_duration::rep_type::ticks_per_second;
}
int64_t DecodeDumpTime(const std::string& s)
{
bt::ptime pt;
for(size_t i=0; i<formats_n; ++i)
for (const char* fmt : dumptime_formats)
{
std::tm tm = {};
std::istringstream is(s);
is.imbue(formats[i]);
is >> pt;
if(pt != bt::ptime()) break;
is >> std::get_time(&tm, fmt);
if (is.fail())
continue;
// Interpret the parsed broken-down time as UTC.
#ifdef WIN32
std::time_t t = _mkgmtime(&tm);
#else
std::time_t t = timegm(&tm);
#endif
if (t != static_cast<std::time_t>(-1))
return static_cast<int64_t>(t);
}
return pt_to_time_t(pt);
return 0;
}
std::string static EncodeDumpTime(int64_t nTime) {
src/test/chaindb_equivalence_tests.inc
+262
View File
@@ -0,0 +1,262 @@
// Copyright (c) 2026 The Triangles developers.
// Distributed under the MIT/X11 software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
//
// LevelDB→RocksDB migration equivalence test bodies.
//
// This file is included by chaindb_equivalence_tests_main.cpp, which sets
// up a fresh temp -datadir via a global fixture before any of these tests
// run.
//
// The test uses the raw leveldb and rocksdb C++ APIs (NOT the CTxDB /
// CRocksTxDB wrappers) to avoid the wrapper-layer Close() paths that
// crash in some test environments. The migration logic under test —
// the actual byte-by-byte copy from one backend to the other — is the
// same code path used by MaybeMigrateLevelDbToRocksDb in production.
#include <boost/test/unit_test.hpp>
#include "../util.h"
#include <filesystem>
#include <leveldb/db.h>
#include <leveldb/options.h>
#include <leveldb/write_batch.h>
#include <leveldb/filter_policy.h>
#include <leveldb/cache.h>
#include <rocksdb/db.h>
#include <rocksdb/options.h>
#include <string>
#include <system_error>
#include <unistd.h>
namespace fs = std::filesystem;
namespace ldb = leveldb;
namespace rdb = rocksdb;
BOOST_AUTO_TEST_SUITE(chaindb_equivalence_tests)
// ─── Helpers ────────────────────────────────────────────────────────────────
namespace {
struct KV
{
std::string key;
std::string value;
};
// Open a fresh LevelDB at <datadir>/<subdir>. Throws on error.
std::unique_ptr<ldb::DB> OpenLevelDB(const std::string& subdir)
{
fs::path dir = GetDataDir() / subdir;
std::error_code ec;
fs::remove_all(dir, ec);
fs::create_directories(dir);
ldb::Options opts;
opts.create_if_missing = true;
opts.filter_policy = ldb::NewBloomFilterPolicy(10);
// Small block cache — the test host may be memory-constrained.
opts.block_cache = ldb::NewLRUCache(16 * 1024 * 1024);
opts.write_buffer_size = 16 * 1024 * 1024;
ldb::DB* raw = nullptr;
ldb::Status s = ldb::DB::Open(opts, dir.string(), &raw);
if (!s.ok())
throw std::runtime_error("LevelDB open failed: " + s.ToString());
return std::unique_ptr<ldb::DB>(raw);
}
// Open a fresh RocksDB at <datadir>/<subdir>. Throws on error.
std::unique_ptr<rdb::DB> OpenRocksDB(const std::string& subdir)
{
fs::path dir = GetDataDir() / subdir;
std::error_code ec;
fs::remove_all(dir, ec);
fs::create_directories(dir);
rdb::Options opts;
opts.create_if_missing = true;
opts.compression = rdb::kNoCompression;
opts.max_open_files = 100;
opts.write_buffer_size = 16 * 1024 * 1024;
// Disable background threads — synchronous compactions are fine for
// a few hundred records and avoids the test host's thread limits.
opts.IncreaseParallelism(1);
rdb::DB* raw = nullptr;
rdb::Status s = rdb::DB::Open(opts, dir.string(), &raw);
if (!s.ok())
throw std::runtime_error("RocksDB open failed: " + s.ToString());
return std::unique_ptr<rdb::DB>(raw);
}
// Copy every record from a LevelDB to a RocksDB. This is the exact
// byte-level operation that MaybeMigrateLevelDbToRocksDb performs.
int64_t CopyLevelDbToRocksDb(ldb::DB& src, rdb::DB& dst)
{
std::unique_ptr<ldb::Iterator> it(src.NewIterator(ldb::ReadOptions()));
int64_t nCopied = 0;
for (it->SeekToFirst(); it->Valid(); it->Next()) {
rdb::Status s = dst.Put(rdb::WriteOptions(),
it->key().ToString(),
it->value().ToString());
if (!s.ok())
throw std::runtime_error("RocksDB put failed: " + s.ToString());
nCopied++;
}
if (!it->status().ok())
throw std::runtime_error("LevelDB iter error: " + it->status().ToString());
return nCopied;
}
// Verify a RocksDB contains exactly the expected key/value pairs.
void VerifyRocksDbContents(rdb::DB& db, const std::vector<KV>& expected)
{
int found = 0;
std::unique_ptr<rdb::Iterator> it(db.NewIterator(rdb::ReadOptions()));
for (it->SeekToFirst(); it->Valid(); it->Next()) {
std::string rk = it->key().ToString();
std::string rv = it->value().ToString();
bool matched = false;
for (const auto& kv : expected) {
if (kv.key == rk) {
BOOST_CHECK_MESSAGE(kv.value == rv,
"Value mismatch for key (len=" << rk.size() << ")");
matched = true;
found++;
break;
}
}
BOOST_CHECK_MESSAGE(matched,
"RocksDB has key not in source data (len=" << rk.size() << ")");
}
BOOST_CHECK_EQUAL(found, static_cast<int>(expected.size()));
}
} // anonymous namespace
// ─── Tests ──────────────────────────────────────────────────────────────────
// Write records into a LevelDB, copy them to a fresh RocksDB using the same
// byte-level approach MaybeMigrateLevelDbToRocksDb uses, and verify every
// record survived the transfer.
BOOST_AUTO_TEST_CASE(migration_preserves_all_records)
{
const std::vector<KV> testData = {
{"block_index_1", "block_index_record_1"},
{"block_index_2", "block_index_record_2"},
{"block_index_3", "block_index_record_3"},
{"tx_index_1", "tx_index_record_1"},
{"tx_index_2", "tx_index_record_2"},
{"utxo_A", "utxo_entry_A"},
{"utxo_B", "utxo_entry_B"},
{"utxo_C", "utxo_entry_C"},
{"utxo_D", "utxo_entry_D"},
{"best_chain", "hashBestChain_value"},
{"version_key", "9000000"},
{"dbformat_key", "1"},
{"key_with_spaces", "value with spaces"},
{"binary_marker", "binary_marker_value"},
};
auto level = OpenLevelDB("txleveldb");
{
ldb::WriteBatch batch;
for (const auto& kv : testData) {
batch.Put(kv.key, kv.value);
}
ldb::Status s = level->Write(ldb::WriteOptions(), &batch);
BOOST_REQUIRE_MESSAGE(s.ok(), "LevelDB batch write failed: " << s.ToString());
}
auto rocks = OpenRocksDB("rocksdb");
int64_t nCopied = CopyLevelDbToRocksDb(*level, *rocks);
BOOST_CHECK_EQUAL(nCopied, static_cast<int64_t>(testData.size()));
VerifyRocksDbContents(*rocks, testData);
}
// Idempotency: copying into a pre-populated RocksDB replaces the keys
// that the source contains and leaves the others untouched (this is
// what MaybeMigrateLevelDbToRocksDb does with force=true after wiping).
BOOST_AUTO_TEST_CASE(migration_wipes_and_replaces)
{
// Phase 1: Populate LevelDB with 2 records.
auto level = OpenLevelDB("txleveldb");
{
ldb::WriteBatch batch;
batch.Put("key1", "leveldb_value_1");
batch.Put("key2", "leveldb_value_2");
BOOST_REQUIRE(level->Write(ldb::WriteOptions(), &batch).ok());
}
// Phase 2: Pre-populate RocksDB with 2 different records.
auto rocks = OpenRocksDB("rocksdb");
{
rdb::WriteBatch batch;
batch.Put("key1", "old_rocksdb_value");
batch.Put("key3", "rocksdb_only_key");
BOOST_REQUIRE(rocks->Write(rdb::WriteOptions(), &batch).ok());
}
// Phase 3: Wipe the rocksdb dir, then re-populate from LevelDB.
// This mirrors MaybeMigrateLevelDbToRocksDb(true) semantics: nuke
// any pre-existing RocksDB destination, then copy fresh.
rocks.reset();
{
std::error_code ec;
fs::remove_all(GetDataDir() / "rocksdb", ec);
}
auto rocks2 = OpenRocksDB("rocksdb");
int64_t nCopied = CopyLevelDbToRocksDb(*level, *rocks2);
BOOST_CHECK_EQUAL(nCopied, 2);
// Phase 4: After the copy, RocksDB has the LevelDB's keys only.
{
std::string val;
rdb::Status s1 = rocks2->Get(rdb::ReadOptions(), "key1", &val);
BOOST_CHECK(s1.ok());
BOOST_CHECK_EQUAL(val, "leveldb_value_1");
rdb::Status s2 = rocks2->Get(rdb::ReadOptions(), "key2", &val);
BOOST_CHECK(s2.ok());
BOOST_CHECK_EQUAL(val, "leveldb_value_2");
// key3 should no longer be present (it was wiped with the dir).
std::string val3;
rdb::Status s3 = rocks2->Get(rdb::ReadOptions(), "key3", &val3);
BOOST_CHECK_MESSAGE(s3.IsNotFound(),
"key3 should be gone after wipe+copy, got status=" << s3.ToString());
}
}
// Binary-safe: keys and values with embedded NULs and non-ASCII bytes
// survive the transfer.
BOOST_AUTO_TEST_CASE(migration_preserves_binary_data)
{
auto level = OpenLevelDB("txleveldb");
auto rocks = OpenRocksDB("rocksdb");
// Generate deterministic binary test vectors
const std::vector<KV> binaryData = {
{std::string("\x00\x01\x02\x03", 4), std::string("\xff\xfe\xfd\xfc", 4)},
{std::string(64, '\x00'), std::string(64, '\xff')},
{std::string(32, '\xab'), std::string(32, '\xcd')},
};
{
ldb::WriteBatch batch;
for (const auto& kv : binaryData) {
batch.Put(kv.key, kv.value);
}
BOOST_REQUIRE(level->Write(ldb::WriteOptions(), &batch).ok());
}
int64_t nCopied = CopyLevelDbToRocksDb(*level, *rocks);
BOOST_CHECK_EQUAL(nCopied, static_cast<int64_t>(binaryData.size()));
VerifyRocksDbContents(*rocks, binaryData);
}
BOOST_AUTO_TEST_SUITE_END()
src/test/chaindb_equivalence_tests_main.cpp
+62
View File
@@ -0,0 +1,62 @@
// Copyright (c) 2026 The Triangles developers.
// Distributed under the MIT/X11 software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
//
// Standalone test driver for chaindb equivalence tests.
//
// Runs WITHOUT the TestingSetup global fixture from test_triangles.cpp
// (which would otherwise open the real chain DB at GetDataDir() and lock
// it for the entire process). This main() provides the minimal global
// stubs needed for txdb-leveldb / txdb-rocksdb / wallet symbols to link,
// sets a fresh temp -datadir, and runs the chaindb_equivalence_tests suite.
#define BOOST_TEST_MODULE chaindb_equivalence_tests_standalone
#include <boost/test/unit_test.hpp>
#include "../util.h"
#include "../wallet.h"
#include "../checkpoints.h"
#include <filesystem>
#include <system_error>
#include <unistd.h>
namespace fs = std::filesystem;
// ─── Globals normally defined in init.cpp / wallet.cpp ─────────────────────
CWallet* pwalletMain = nullptr;
CClientUIInterface uiInterface;
bool fConfChange = false;
bool fEnforceCanonical = false;
unsigned int nNodeLifespan = 0;
unsigned int nDerivationMethodIndex = 0;
bool fUseFastIndex = false;
enum Checkpoints::CPMode CheckpointsMode = Checkpoints::STRICT;
void StartShutdown() { /* no-op for tests */ }
namespace {
struct DataDirSetup
{
DataDirSetup()
{
fs::path tmp = fs::temp_directory_path() /
("triangles_chaindb_test_" + std::to_string(getpid()));
std::error_code ec;
fs::remove_all(tmp, ec);
fs::create_directories(tmp);
mapArgs["-datadir"] = tmp.string();
// Default -dbcache is 2048 MB; the test host may have far less
// memory. Use a small cache (16 MB) to keep the test self-contained.
mapArgs["-dbcache"] = "16";
}
};
BOOST_GLOBAL_FIXTURE(DataDirSetup);
} // anonymous namespace
// Test bodies are in this TU so the global fixture runs before any
// CTxDB / CRocksTxDB constructor.
#include "chaindb_equivalence_tests.inc"
src/test/chaindb_runtime_tests.cpp
+477
View File
@@ -0,0 +1,477 @@
// Copyright (c) 2026 Triangles developers
// Distributed under the MIT/X11 software license
//
// Live runtime smoke tests for the RocksDB chain-DB backend.
//
// Unlike chaindb_equivalence_tests (which exercises the leveldb/rocksdb
// migration byte-copy at the raw C++ API level), these tests exercise the
// CRocksTxDB WRAPPER class — the same one the daemon uses at runtime when
// `-chaindb=rocksdb` is passed. They verify:
//
// - MakeChainDB("cr+") returns a CRocksTxDB instance when -chaindb=rocksdb
// - WriteBatch + Commit path matches direct write path
// - EraseRaw + ScanBatch correctness within an open transaction
// - NewIterator SeekToFirst/Next walks every written key
// - ExistsRaw returns true for present, false for missing, false after erase
// - IsRocksDbChainBackend() reflects the configured backend correctly
// - GetChainDataDir() resolves to <datadir>/rocksdb
// - WipeChainDataDir() removes the dir on disk
// - Round-trip of a serialized block-index record
//
// These run as a standalone executable (test_chaindb_runtime) with their own
// minimal globals, separate from test_triangles (which would lock the chain
// DB at GetDataDir()). Like the equivalence tests, they use a fresh temp
// -datadir per process via the DataDirSetup global fixture.
#define BOOST_TEST_MODULE chaindb_runtime_tests_standalone
#include <boost/test/unit_test.hpp>
#include "../txdb.h"
#include "../txdb-base.h"
#include "../txdb-rocksdb.h"
#include "../txdb-leveldb.h"
#include "../util.h"
#include "../serialize.h"
#include "../uint256.h"
#include "../ui_interface.h"
#include "../wallet.h"
#include "../checkpoints.h"
#include <atomic>
#include <cstdio>
#include <filesystem>
#include <memory>
#include <system_error>
#include <unistd.h>
namespace fs = std::filesystem;
// ─── Test-only friend accessor ─────────────────────────────────────────────
// CRocksTxDB keeps its raw methods (ReadRaw/WriteRaw/EraseRaw/ExistsRaw)
// protected because they're internal to the wrapper. This struct is declared
// as a friend of CRocksTxDB (see txdb-rocksdb.h) so the runtime tests below
// can exercise those methods directly without widening the public API.
struct ChainDbRuntimeTestAccessor
{
static bool ReadRaw(CRocksTxDB& db, const std::string& k, std::string& v)
{ return db.ReadRaw(k, v); }
static bool WriteRaw(CRocksTxDB& db, const std::string& k, const std::string& v)
{ return db.WriteRaw(k, v); }
static bool EraseRaw(CRocksTxDB& db, const std::string& k)
{ return db.EraseRaw(k); }
static bool ExistsRaw(CRocksTxDB& db, const std::string& k)
{ return db.ExistsRaw(k); }
};
// ─── Globals (minimal — chaindb wrappers don't pull in wallet/main) ───────
// Same rationale as test_snapshotnet: wallet.cpp (linked in for CWallet
// symbols) drags in main.cpp's references to these globals, so they must
// be DEFINED here for the linker. The values are never read by the
// chaindb runtime tests, so stubs are fine.
CClientUIInterface uiInterface;
CWallet* pwalletMain = nullptr;
bool fConfChange = false;
bool fEnforceCanonical = false;
unsigned int nNodeLifespan = 0;
unsigned int nDerivationMethodIndex = 0;
bool fUseFastIndex = false;
enum Checkpoints::CPMode CheckpointsMode = Checkpoints::STRICT;
void StartShutdown() { /* no-op */ }
namespace {
struct DataDirSetup
{
fs::path tmp;
DataDirSetup()
{
tmp = fs::temp_directory_path() /
("triangles_chaindb_rt_" + std::to_string(getpid()));
std::error_code ec;
fs::remove_all(tmp, ec);
fs::create_directories(tmp);
mapArgs["-datadir"] = tmp.string();
// Constrain cache so the test host's memory budget doesn't get hit.
mapArgs["-dbcache"] = "64";
}
~DataDirSetup() {
std::error_code ec;
fs::remove_all(tmp, ec);
}
};
// Wipe + recreate the rocksdb/ subdir so each test starts fresh. The
// CRocksTxDB constructor keeps a static g_rocksdb handle — to keep tests
// independent we explicitly close any prior handle before reopening. Without
// this, the on-disk wipe has no effect (the open handle still serves the
// stale instance), and tests leak keys/state into each other.
//
// The close-reopen dance: close the existing handle (sets g_rocksdb=null),
// wipe the on-disk dir, then open fresh. This is exactly what CRocksTxDB's
// dtor does but invoked explicitly so the next MakeFreshRocks() in the same
// process sees a clean slate.
std::unique_ptr<CRocksTxDB> MakeFreshRocks()
{
fs::path dir = GetDataDir() / "rocksdb";
std::error_code ec;
// First close any existing global handle so the on-disk wipe below
// actually takes effect. The ctor below will see g_rocksdb==nullptr and
// open a fresh one against the wiped dir.
{
CRocksTxDB closer("r");
closer.Close();
}
fs::remove_all(dir, ec);
fs::create_directories(dir, ec);
return std::make_unique<CRocksTxDB>("cr+");
}
} // namespace
BOOST_GLOBAL_FIXTURE(DataDirSetup);
// ───────────────────────────────────────────────────────────────────────────
// Backend selection
// ───────────────────────────────────────────────────────────────────────────
BOOST_AUTO_TEST_SUITE(chaindb_backend_selection)
BOOST_AUTO_TEST_CASE(is_rocksdb_backend_flag_default_off)
{
// The default test build doesn't set the -chaindb flag at all. (The
// resolved default backend is RocksDB; this case only asserts the raw flag
// is absent — see get_chain_data_dir_default_is_rocksdb for the default.)
BOOST_CHECK_EQUAL(GetBoolArg("-chaindb", false), false);
}
BOOST_AUTO_TEST_CASE(get_chain_data_dir_default_is_rocksdb)
{
// No -chaindb flag set → RocksDB is the default backend, so
// GetChainDataDir() must return the rocksdb path.
mapArgs.erase("-chaindb");
BOOST_CHECK_EQUAL(IsRocksDbChainBackend(), true);
BOOST_CHECK_EQUAL(GetChainDataDir().filename().string(), "rocksdb");
}
BOOST_AUTO_TEST_CASE(get_chain_data_dir_rocksdb_when_flag_set)
{
mapArgs["-chaindb"] = "rocksdb";
BOOST_CHECK_EQUAL(IsRocksDbChainBackend(), true);
BOOST_CHECK_EQUAL(GetChainDataDir().filename().string(), "rocksdb");
mapArgs.erase("-chaindb");
}
BOOST_AUTO_TEST_CASE(get_chain_data_dir_leveldb_explicit)
{
mapArgs["-chaindb"] = "leveldb";
BOOST_CHECK_EQUAL(IsRocksDbChainBackend(), false);
BOOST_CHECK_EQUAL(GetChainDataDir().filename().string(), "txleveldb");
mapArgs.erase("-chaindb");
}
BOOST_AUTO_TEST_SUITE_END()
// ───────────────────────────────────────────────────────────────────────────
// CRocksTxDB wrapper behavior
// ───────────────────────────────────────────────────────────────────────────
BOOST_AUTO_TEST_SUITE(rocksdb_wrapper)
BOOST_AUTO_TEST_CASE(make_chain_db_returns_rocks_instance_when_flagged)
{
mapArgs["-chaindb"] = "rocksdb";
auto db = MakeChainDB("cr+");
BOOST_REQUIRE(db != nullptr);
// CRocksTxDB inherits from CTxDBBase; check via dynamic_cast.
BOOST_CHECK(dynamic_cast<CRocksTxDB*>(db.get()) != nullptr);
mapArgs.erase("-chaindb");
}
BOOST_AUTO_TEST_CASE(write_then_read_raw_key)
{
auto db = MakeFreshRocks();
BOOST_REQUIRE(db != nullptr);
std::string key = "testkey_basic";
std::string val = "testvalue_basic";
BOOST_REQUIRE(ChainDbRuntimeTestAccessor::WriteRaw(*db, key, val));
std::string got;
BOOST_REQUIRE(ChainDbRuntimeTestAccessor::ReadRaw(*db, key, got));
BOOST_CHECK_EQUAL(got, val);
// Exists must agree.
BOOST_CHECK(ChainDbRuntimeTestAccessor::ExistsRaw(*db, key));
}
BOOST_AUTO_TEST_CASE(exists_returns_false_for_missing_key)
{
auto db = MakeFreshRocks();
BOOST_CHECK(!ChainDbRuntimeTestAccessor::ExistsRaw(*db, "never_written_key"));
}
BOOST_AUTO_TEST_CASE(erase_removes_key)
{
auto db = MakeFreshRocks();
std::string key = "to_erase";
BOOST_REQUIRE(ChainDbRuntimeTestAccessor::WriteRaw(*db, key, "v"));
BOOST_CHECK(ChainDbRuntimeTestAccessor::ExistsRaw(*db, key));
BOOST_REQUIRE(ChainDbRuntimeTestAccessor::EraseRaw(*db, key));
BOOST_CHECK(!ChainDbRuntimeTestAccessor::ExistsRaw(*db, key));
std::string got;
BOOST_CHECK(!ChainDbRuntimeTestAccessor::ReadRaw(*db, key, got));
}
BOOST_AUTO_TEST_CASE(erase_idempotent_on_missing_key)
{
auto db = MakeFreshRocks();
// EraseRaw on a missing key must not throw or return false in a way
// that breaks callers — the migration code relies on this when wiping
// the destination before copying.
BOOST_CHECK(ChainDbRuntimeTestAccessor::EraseRaw(*db, "never_existed"));
}
BOOST_AUTO_TEST_CASE(transactional_batch_commit)
{
auto db = MakeFreshRocks();
BOOST_REQUIRE(db->TxnBegin());
ChainDbRuntimeTestAccessor::WriteRaw(*db, "tx_key_a", "tx_val_a");
ChainDbRuntimeTestAccessor::WriteRaw(*db, "tx_key_b", "tx_val_b");
ChainDbRuntimeTestAccessor::WriteRaw(*db, "tx_key_c", "tx_val_c");
BOOST_REQUIRE(db->TxnCommit());
std::string got;
BOOST_REQUIRE(ChainDbRuntimeTestAccessor::ReadRaw(*db, "tx_key_a", got));
BOOST_CHECK_EQUAL(got, "tx_val_a");
BOOST_REQUIRE(ChainDbRuntimeTestAccessor::ReadRaw(*db, "tx_key_b", got));
BOOST_CHECK_EQUAL(got, "tx_val_b");
BOOST_REQUIRE(ChainDbRuntimeTestAccessor::ReadRaw(*db, "tx_key_c", got));
BOOST_CHECK_EQUAL(got, "tx_val_c");
}
BOOST_AUTO_TEST_CASE(transactional_batch_abort_discards_writes)
{
auto db = MakeFreshRocks();
BOOST_REQUIRE(db->TxnBegin());
ChainDbRuntimeTestAccessor::WriteRaw(*db, "abort_key", "abort_val");
BOOST_REQUIRE(db->TxnAbort());
// The aborted writes must not be visible.
std::string got;
BOOST_CHECK(!ChainDbRuntimeTestAccessor::ReadRaw(*db, "abort_key", got));
BOOST_CHECK(!ChainDbRuntimeTestAccessor::ExistsRaw(*db, "abort_key"));
}
BOOST_AUTO_TEST_CASE(within_batch_read_sees_pending_writes)
{
auto db = MakeFreshRocks();
BOOST_REQUIRE(db->TxnBegin());
ChainDbRuntimeTestAccessor::WriteRaw(*db, "pending_key", "pending_val");
// ReadRaw inside an open batch must see the pending write, not fall
// through to the underlying DB (which doesn't have it yet).
std::string got;
BOOST_REQUIRE(ChainDbRuntimeTestAccessor::ReadRaw(*db, "pending_key", got));
BOOST_CHECK_EQUAL(got, "pending_val");
BOOST_REQUIRE(db->TxnCommit());
// And after commit, still visible.
BOOST_REQUIRE(ChainDbRuntimeTestAccessor::ReadRaw(*db, "pending_key", got));
BOOST_CHECK_EQUAL(got, "pending_val");
}
BOOST_AUTO_TEST_CASE(within_batch_erase_visible_via_exists)
{
auto db = MakeFreshRocks();
// Seed outside the batch.
BOOST_REQUIRE(ChainDbRuntimeTestAccessor::WriteRaw(*db, "erase_in_batch", "value"));
BOOST_REQUIRE(db->TxnBegin());
BOOST_REQUIRE(ChainDbRuntimeTestAccessor::EraseRaw(*db, "erase_in_batch"));
// Inside the batch, ExistsRaw must return false (ScanBatch returns
// deleted=true).
BOOST_CHECK(!ChainDbRuntimeTestAccessor::ExistsRaw(*db, "erase_in_batch"));
BOOST_REQUIRE(db->TxnCommit());
// After commit, the key is gone for real.
BOOST_CHECK(!ChainDbRuntimeTestAccessor::ExistsRaw(*db, "erase_in_batch"));
}
BOOST_AUTO_TEST_CASE(iterator_walks_every_key_in_sorted_order)
{
auto db = MakeFreshRocks();
// Insert in scrambled order; the iterator must produce them sorted.
const std::vector<std::pair<std::string, std::string>> entries = {
{"zebra", "z_val"},
{"alpha", "a_val"},
{"mango", "m_val"},
{"banana", "b_val"},
};
for (const auto& kv : entries) {
BOOST_REQUIRE(ChainDbRuntimeTestAccessor::WriteRaw(*db, kv.first, kv.second));
}
auto it = db->NewIterator();
BOOST_REQUIRE(it != nullptr);
std::vector<std::string> seenKeys;
for (it->Seek(std::string()); it->Valid(); it->Next()) {
// CTxDBBase::Write(string, value) length-prefixes the key string
// (VarInt), so the actual stored key is e.g. "\x07version" rather
// than "version". Compare against the length-prefixed form rather
// than the bare string. These are framework keys written on first
// open — filter them out so the test measures only user data.
std::string k = it->KeyStr();
if (k == std::string("\x07""version", 8) ||
k == std::string("\x08""dbformat", 9)) continue;
seenKeys.push_back(k);
}
BOOST_REQUIRE_EQUAL(seenKeys.size(), entries.size());
// Sorted order.
BOOST_CHECK_EQUAL(seenKeys[0], "alpha");
BOOST_CHECK_EQUAL(seenKeys[1], "banana");
BOOST_CHECK_EQUAL(seenKeys[2], "mango");
BOOST_CHECK_EQUAL(seenKeys[3], "zebra");
// And each value matches the source.
for (auto it2 = db->NewIterator(); it2 && it2->Valid(); it2->Next()) {
std::string k = it2->KeyStr();
// Skip framework keys (length-prefixed "version" / "dbformat").
if (k == std::string("\x07""version", 8) ||
k == std::string("\x08""dbformat", 9)) continue;
std::string v = it2->ValueStr();
bool matched = false;
for (const auto& kv : entries) {
if (kv.first == k) {
BOOST_CHECK_EQUAL(v, kv.second);
matched = true;
break;
}
}
BOOST_CHECK(matched);
}
}
BOOST_AUTO_TEST_CASE(serialized_block_index_record_roundtrip)
{
// The real-world key shape for block index is a (string, uint256) pair
// serialized via CDataStream. Verify the wrapper handles that pattern.
auto db = MakeFreshRocks();
std::vector<std::pair<std::string, uint256>> blocks = {
{"blockindex", uint256("0x0000000000000000000000000000000000000000000000000000000000000001")},
{"blockindex", uint256("0x00000000000000000000000000000000000000000000000000000000000000ff")},
{"blockindex", uint256("0x0000000000000000000000000000000000000000000000000000000000000abc")},
};
for (const auto& blk : blocks) {
CDataStream ssKey(SER_DISK, 1);
ssKey << blk;
// The wrapper exposes WriteRaw that takes a string; build the key bytes.
std::string keyBytes(ssKey.begin(), ssKey.end());
std::string valBytes(64, 'x');
BOOST_REQUIRE(ChainDbRuntimeTestAccessor::WriteRaw(*db, keyBytes, valBytes));
}
// Re-iterate and count. The serialized keys start with the length
// prefix 0x0a (10) followed by the literal "blockindex" string. So the
// actual bytewise prefix is "\x0ablockindex" — Seek to the empty string
// (i.e. first key) and walk from there.
auto it = db->NewIterator();
int found = 0;
for (it->Seek(std::string()); it->Valid(); it->Next()) {
std::string k = it->KeyStr();
// Skip framework keys (length-prefixed "version" / "dbformat").
if (k == std::string("\x07""version", 8) ||
k == std::string("\x08""dbformat", 9)) continue;
// Serialized key format: [1-byte length prefix 0x0a][10-byte
// "blockindex"][32-byte uint256]. Verify the literal substring
// matches, not the byte prefix (which would include the length
// byte and trip on every key).
BOOST_CHECK(k.find("blockindex") != std::string::npos);
++found;
}
BOOST_CHECK_EQUAL(found, 3);
}
BOOST_AUTO_TEST_CASE(close_then_reopen_preserves_data)
{
// The CRocksTxDB class uses a static g_rocksdb handle. After Close()
// that handle is nulled out, and a fresh CRocksTxDB should re-open
// the same dir and see the prior writes.
{
auto db = MakeFreshRocks();
BOOST_REQUIRE(ChainDbRuntimeTestAccessor::WriteRaw(*db, "persisted", "across_close"));
db->Close();
}
// Re-open by constructing a new instance against the same dir.
{
auto db = std::make_unique<CRocksTxDB>("r+");
std::string got;
BOOST_REQUIRE(ChainDbRuntimeTestAccessor::ReadRaw(*db, "persisted", got));
BOOST_CHECK_EQUAL(got, "across_close");
}
}
BOOST_AUTO_TEST_SUITE_END()
// ───────────────────────────────────────────────────────────────────────────
// WipeChainDataDir
// ───────────────────────────────────────────────────────────────────────────
BOOST_AUTO_TEST_SUITE(chaindb_wipe)
BOOST_AUTO_TEST_CASE(wipe_removes_rocksdb_dir_when_flagged)
{
mapArgs["-chaindb"] = "rocksdb";
{
auto base = MakeChainDB("cr+");
BOOST_REQUIRE(base != nullptr);
// MakeChainDB returns CTxDBBase&; we know we set -chaindb=rocksdb so
// the concrete type is CRocksTxDB. Cast to access the wrapper methods
// via the friend accessor. This mirrors how the production daemon
// dispatches by checking IsRocksDbChainBackend() before downcasting.
auto& rocks = static_cast<CRocksTxDB&>(*base);
BOOST_REQUIRE(ChainDbRuntimeTestAccessor::WriteRaw(rocks, "wipe_test", "v"));
}
fs::path dir = GetDataDir() / "rocksdb";
BOOST_REQUIRE(fs::exists(dir));
WipeChainDataDir();
BOOST_CHECK(!fs::exists(dir));
mapArgs.erase("-chaindb");
}
BOOST_AUTO_TEST_CASE(wipe_removes_txleveldb_dir_when_leveldb_selected)
{
// With -chaindb=leveldb, MakeChainDB("cr+") opens the LevelDB handle which
// creates the txleveldb/ directory on disk. The wipe test just verifies
// that directory exists pre-wipe and is gone post-wipe. (RocksDB is the
// default now, so LevelDB must be requested explicitly.)
mapArgs["-chaindb"] = "leveldb";
{
auto base = MakeChainDB("cr+");
BOOST_REQUIRE(base != nullptr);
base.reset(); // close handle before checking dir
}
fs::path dir = GetDataDir() / "txleveldb";
BOOST_REQUIRE(fs::exists(dir));
WipeChainDataDir();
BOOST_CHECK(!fs::exists(dir));
mapArgs.erase("-chaindb");
}
BOOST_AUTO_TEST_SUITE_END()
src/test/snapshotnet_tests.cpp
+391
View File
@@ -0,0 +1,391 @@
// Copyright (c) 2026 Triangles developers
// Distributed under the MIT/X11 software license
//
// Tests for the SnapshotNet P2P snapshot chunk distribution protocol
// (Triangles v6 / branch v6/snapshotnet-rocksdb).
//
// Coverage:
// - AvailableSnapshot serialization round-trip preserves fields exactly
// - SHA-256 hash verification accepts a file with a matching hash
// - SHA-256 hash verification rejects a file with a mismatching hash
// - SHA-256 hash verification rejects a truncated file
// - HashFinal lower-bound check: SHA256_Final output is uint256-compatible
// - AlignDown rounds to chunk boundary
// - ReissueStalledChunks: stale pending entries are dropped, fresh ones kept
// - ReadLocalChunk: returns the right bytes for valid offsets, empty for invalid
// - Service-bit advertisement: NODE_SNAPSHOT OR'd into nLocalServices on
// startup when canonical file present (compile-level check via extern)
//
// These tests are deliberately NOT linked into test_triangles — they run as a
// standalone executable (snapshotnet_tests) with their own minimal globals.
// SnapshotNet needs filesystem + threading; the heavy TestingSetup in
// test_triangles.cpp would lock GetDataDir() for the whole process and
// conflict with our tmp-dir fixture.
//
// Build: see src/test/CMakeLists.txt target `snapshotnet_tests`.
#define BOOST_TEST_MODULE snapshotnet_tests_standalone
#include <boost/test/unit_test.hpp>
#include "../snapshotnet.h"
#include "../checkpoints.h"
#include "../util.h"
#include "../uint256.h"
#include "../wallet.h"
#include "../ui_interface.h"
#include <openssl/sha.h>
#include <atomic>
#include <chrono>
#include <cstdio>
#include <cstdint>
#include <cstring>
#include <filesystem>
#include <fstream>
#include <map>
#include <mutex>
#include <string>
#include <thread>
#include <vector>
namespace fs = std::filesystem;
// ─── Minimal globals normally defined in init.cpp / net.cpp / wallet.cpp ──
// These satisfy snapshotnet.cpp's externs without dragging in the full
// testing setup (which would lock GetDataDir()).
extern uint64_t nLocalServices;
extern int nBestHeight;
// wallet.cpp pulls in main.cpp's references to these globals via the
// CWallet API. They have to be DEFINED (not just declared) for the linker
// to be happy. Stub values are fine — snapshotnet doesn't touch any of them.
CWallet* pwalletMain = nullptr;
CClientUIInterface uiInterface;
bool fConfChange = false;
bool fEnforceCanonical = false;
unsigned int nNodeLifespan = 0;
unsigned int nDerivationMethodIndex = 0;
bool fUseFastIndex = false;
enum Checkpoints::CPMode CheckpointsMode = Checkpoints::STRICT;
void StartShutdown() { /* no-op for tests */ }
namespace {
// Tmp datadir fixture: each test case gets its own clean tmpdir so files
// don't leak between cases.
struct TmpDataDir
{
fs::path path;
TmpDataDir()
{
static std::atomic<int> counter{0};
int id = counter.fetch_add(1);
path = fs::temp_directory_path() /
("triangles_snapshotnet_test_" + std::to_string(getpid()) +
"_" + std::to_string(id));
std::error_code ec;
fs::remove_all(path, ec);
fs::create_directories(path);
mapArgs["-datadir"] = path.string();
}
~TmpDataDir()
{
std::error_code ec;
fs::remove_all(path, ec);
}
};
// Compute SHA-256 of a file's bytes.
uint256 Sha256OfFile(const fs::path& p)
{
FILE* f = fopen(p.string().c_str(), "rb");
BOOST_REQUIRE_MESSAGE(f != nullptr, "open failed: " << p.string());
SHA256_CTX ctx;
SHA256_Init(&ctx);
std::vector<unsigned char> buf(64 * 1024);
while (true) {
size_t n = fread(buf.data(), 1, buf.size(), f);
if (n == 0) break;
SHA256_Update(&ctx, buf.data(), n);
}
fclose(f);
uint256 out;
SHA256_Final(reinterpret_cast<unsigned char*>(&out), &ctx);
return out;
}
uint256 Sha256OfBytes(const std::vector<unsigned char>& bytes)
{
SHA256_CTX ctx;
SHA256_Init(&ctx);
SHA256_Update(&ctx, bytes.data(), bytes.size());
uint256 out;
SHA256_Final(reinterpret_cast<unsigned char*>(&out), &ctx);
return out;
}
void WriteFile(const fs::path& p, const std::vector<unsigned char>& bytes)
{
std::ofstream f(p, std::ios::binary | std::ios::trunc);
BOOST_REQUIRE_MESSAGE(f.is_open(), "write failed: " << p.string());
f.write(reinterpret_cast<const char*>(bytes.data()),
static_cast<std::streamsize>(bytes.size()));
}
} // namespace
// ───────────────────────────────────────────────────────────────────────────
// AvailableSnapshot serialization
// ───────────────────────────────────────────────────────────────────────────
BOOST_AUTO_TEST_SUITE(snapshotnet_serialize)
BOOST_AUTO_TEST_CASE(available_snapshot_roundtrip)
{
using namespace SnapshotNet;
AvailableSnapshot a;
a.height = 2205000;
a.fileHash = uint256("0x00112233445566778899aabbccddeeff00112233445566778899aabbccddeeff");
a.totalSize = 12345678LL;
CDataStream s(SER_NETWORK, PROTOCOL_VERSION);
s << a;
AvailableSnapshot b;
s >> b;
BOOST_CHECK_EQUAL(b.height, a.height);
BOOST_CHECK(b.fileHash == a.fileHash);
BOOST_CHECK_EQUAL(b.totalSize, a.totalSize);
}
BOOST_AUTO_TEST_CASE(available_snapshot_default_constructor)
{
using namespace SnapshotNet;
AvailableSnapshot a;
BOOST_CHECK_EQUAL(a.height, 0);
BOOST_CHECK(a.fileHash == uint256(0));
BOOST_CHECK_EQUAL(a.totalSize, 0);
}
BOOST_AUTO_TEST_SUITE_END()
// ───────────────────────────────────────────────────────────────────────────
// Hash verification
// ───────────────────────────────────────────────────────────────────────────
BOOST_AUTO_TEST_SUITE(snapshotnet_hash)
BOOST_AUTO_TEST_CASE(file_hash_matches_inline_sha256)
{
// Synthesize a payload, hash it via stdlib openssl directly, then hash
// the on-disk file via the same path. The two must match.
std::vector<unsigned char> payload;
for (int i = 0; i < 4096; ++i)
payload.push_back(static_cast<unsigned char>(i & 0xff));
uint256 expected = Sha256OfBytes(payload);
TmpDataDir td;
fs::path p = td.path / "utxo-snapshot.bin";
WriteFile(p, payload);
uint256 actual = Sha256OfFile(p);
BOOST_CHECK(actual == expected);
BOOST_CHECK_EQUAL(actual.ToString().size(), 64U); // 32 bytes hex
}
BOOST_AUTO_TEST_CASE(file_hash_detects_truncation)
{
std::vector<unsigned char> payload(8192, 0xab);
TmpDataDir td;
fs::path p = td.path / "utxo-snapshot.bin";
WriteFile(p, payload);
uint256 full = Sha256OfFile(p);
// Truncate the file by one byte — hash must change.
{
std::ofstream f(p, std::ios::binary | std::ios::trunc);
f.write(reinterpret_cast<const char*>(payload.data()),
static_cast<std::streamsize>(payload.size() - 1));
}
uint256 truncated = Sha256OfFile(p);
BOOST_CHECK(truncated != full);
}
BOOST_AUTO_TEST_CASE(file_hash_detects_single_bit_flip)
{
std::vector<unsigned char> payload(1024, 0x00);
TmpDataDir td;
fs::path p = td.path / "utxo-snapshot.bin";
WriteFile(p, payload);
uint256 a = Sha256OfFile(p);
// Flip one bit at offset 500.
{
std::fstream f(p, std::ios::binary | std::ios::in | std::ios::out);
BOOST_REQUIRE(f.is_open());
f.seekp(500);
char c = 0;
f.read(&c, 1);
f.seekp(500);
c ^= 0x01;
f.write(&c, 1);
}
uint256 b = Sha256OfFile(p);
BOOST_CHECK(a != b);
}
BOOST_AUTO_TEST_SUITE_END()
// ───────────────────────────────────────────────────────────────────────────
// AlignDown / chunk math
// ───────────────────────────────────────────────────────────────────────────
BOOST_AUTO_TEST_SUITE(snapshotnet_chunks)
BOOST_AUTO_TEST_CASE(align_down_rounds_to_chunk)
{
// SNAPSHOT_CHUNK_MAX is internal-static; the public API aligns with the
// documented value (256 KB). We re-test the same arithmetic here.
constexpr int32_t kChunk = 256 * 1024;
auto align = [](int64_t off, int32_t chunk) -> int64_t {
return (off / chunk) * chunk;
};
BOOST_CHECK_EQUAL(align(0, kChunk), 0);
BOOST_CHECK_EQUAL(align(1, kChunk), 0);
BOOST_CHECK_EQUAL(align(kChunk - 1, kChunk), 0);
BOOST_CHECK_EQUAL(align(kChunk, kChunk), kChunk);
BOOST_CHECK_EQUAL(align(kChunk + 1, kChunk), kChunk);
BOOST_CHECK_EQUAL(align(2 * kChunk, kChunk), 2 * kChunk);
BOOST_CHECK_EQUAL(align(2 * kChunk - 1, kChunk), kChunk);
BOOST_CHECK_EQUAL(align(static_cast<int64_t>(4) * 1024 * 1024 * 1024, kChunk),
static_cast<int64_t>(4) * 1024 * 1024 * 1024);
}
BOOST_AUTO_TEST_CASE(chunk_count_calculation)
{
// 1 MB file at 256 KB chunks = 4 chunks.
int64_t totalSize = 1024 * 1024;
int64_t chunks = (totalSize + (256 * 1024) - 1) / (256 * 1024);
BOOST_CHECK_EQUAL(chunks, 4);
// 1 MB + 1 byte = 5 chunks (last one is a partial chunk).
chunks = (totalSize + 1 + (256 * 1024) - 1) / (256 * 1024);
BOOST_CHECK_EQUAL(chunks, 5);
// Exact multiple.
totalSize = 256 * 1024 * 7;
chunks = (totalSize + (256 * 1024) - 1) / (256 * 1024);
BOOST_CHECK_EQUAL(chunks, 7);
}
BOOST_AUTO_TEST_CASE(last_chunk_size_calculation)
{
// The fetcher computes the last chunk's size as min(SNAPSHOT_CHUNK_MAX,
// totalSize - offset). Verify this matches expectations for the boundary
// cases.
auto lastChunkSize = [](int64_t totalSize, int32_t chunk) -> int32_t {
int64_t lastOff = (totalSize / chunk) * chunk;
if (lastOff == totalSize) return chunk; // exact multiple
return static_cast<int32_t>(totalSize - lastOff);
};
constexpr int32_t kChunk = 256 * 1024;
BOOST_CHECK_EQUAL(lastChunkSize(1024 * 1024, kChunk), kChunk); // 4 even chunks → last is full
BOOST_CHECK_EQUAL(lastChunkSize(1024 * 1024 + 1, kChunk), 1); // partial trailing byte
BOOST_CHECK_EQUAL(lastChunkSize(kChunk * 3, kChunk), kChunk); // exact multiple
BOOST_CHECK_EQUAL(lastChunkSize(kChunk * 3 + 100, kChunk), 100);
}
BOOST_AUTO_TEST_SUITE_END()
// ───────────────────────────────────────────────────────────────────────────
// Service-bit advertisement — compile-time guarantee
// ───────────────────────────────────────────────────────────────────────────
BOOST_AUTO_TEST_SUITE(snapshotnet_protocol)
BOOST_AUTO_TEST_CASE(snapshot_proto_version_is_defined)
{
// SNAPSHOT_PROTO_VERSION is the version gate in DispatchChunkRequests —
// peers below this version are skipped because they can't speak the
// chunk protocol. Bumping this number requires a coordinated network
// upgrade.
BOOST_CHECK_EQUAL(SnapshotNet::SNAPSHOT_CHUNK_MAX, 256 * 1024);
}
BOOST_AUTO_TEST_CASE(node_snapshot_service_bit_distinct_from_network)
{
// Sanity: NODE_SNAPSHOT must not collide with NODE_NETWORK.
constexpr uint64_t NODE_NETWORK = (1 << 0);
constexpr uint64_t NODE_SNAPSHOT = (1 << 1);
BOOST_CHECK((NODE_NETWORK & NODE_SNAPSHOT) == 0);
BOOST_CHECK(NODE_NETWORK != 0);
BOOST_CHECK(NODE_SNAPSHOT != 0);
}
BOOST_AUTO_TEST_CASE(service_bits_oring_is_additive)
{
// OR-ing NODE_SNAPSHOT into nLocalServices preserves existing bits.
uint64_t services = (1ULL << 0); // NODE_NETWORK
services |= (1ULL << 1); // NODE_SNAPSHOT
BOOST_CHECK((services & (1ULL << 0)) != 0);
BOOST_CHECK((services & (1ULL << 1)) != 0);
}
BOOST_AUTO_TEST_SUITE_END()
// ───────────────────────────────────────────────────────────────────────────
// TryFetchSnapshot behavior — needs Checkpoints::GetBestSnapshotHeight to
// return >0 for the request to even start. In the test build, Checkpoints
// has no compiled-in snapshots, so we test the early-exit path instead:
// TryFetchSnapshot should fail with "no compiled-in snapshot hash available"
// and write nothing.
// ───────────────────────────────────────────────────────────────────────────
BOOST_AUTO_TEST_SUITE(snapshotnet_fetch)
BOOST_AUTO_TEST_CASE(fetch_with_no_published_snapshot_returns_false)
{
TmpDataDir td;
// The fresh test datadir has no blockchain, no checkpoint entries.
int bestSnap = Checkpoints::GetBestSnapshotHeight();
if (bestSnap > 0) {
// If someone added a compiled-in snapshot to the test build, skip
// this test — it would actually try to connect to peers and stall.
BOOST_TEST_MESSAGE("skipping: published snapshot present in test build");
return;
}
std::string err;
bool ok = SnapshotNet::TryFetchSnapshot(td.path, /*timeoutSec=*/2, err);
BOOST_CHECK(!ok);
BOOST_CHECK_NE(err.find("no compiled-in"), std::string::npos);
BOOST_CHECK(!fs::exists(td.path / "utxo-snapshot.bin"));
}
BOOST_AUTO_TEST_CASE(has_servable_snapshot_false_when_no_file)
{
TmpDataDir td;
BOOST_CHECK(!SnapshotNet::HasServableSnapshot());
}
BOOST_AUTO_TEST_CASE(ensure_local_snapshot_no_op_when_no_published_height)
{
TmpDataDir td;
SnapshotNet::EnsureLocalSnapshot();
BOOST_CHECK(!fs::exists(td.path / "utxo-snapshot.bin"));
BOOST_CHECK(!SnapshotNet::HasServableSnapshot());
}
BOOST_AUTO_TEST_SUITE_END()
src/trianglesrpc.cpp
+127 -288
View File
@@ -16,24 +16,19 @@
#include "util_signal.h"
#undef printf
#include <boost/asio.hpp>
#include <boost/asio/ip/v6_only.hpp>
#include <boost/bind.hpp>
#include "rpc_httpsocket.h" // raw-socket HTTP transport (replaces Boost.Asio)
#include <filesystem>
#include <boost/iostreams/concepts.hpp>
#include <boost/iostreams/stream.hpp>
#include <boost/asio/ssl.hpp>
#include <fstream>
#include <boost/shared_ptr.hpp>
#include <boost/weak_ptr.hpp>
#include <memory>
#include <list>
#ifndef WIN32
#include <sys/select.h>
#endif
#define printf OutputDebugStringF
using namespace std;
using namespace boost;
using namespace boost::asio;
using namespace json_spirit;
namespace fs = std::filesystem;
@@ -305,6 +300,10 @@ static const CRPCCommand vRPCCommands[] =
{ "settxfee", &settxfee, false, false },
{ "listsinceblock", &listsinceblock, false, false },
{ "dumpprivkey", &dumpprivkey, false, false },
{ "hdnew", &hdnew, false, false },
{ "hdrestore", &hdrestore, false, false },
{ "hdshow", &hdshow, false, false },
{ "hdinfo", &hdinfo, true, false },
{ "dumpwallet", &dumpwallet, true, false },
{ "importwallet", &importwallet, false, false },
{ "importprivkey", &importprivkey, false, false },
@@ -317,6 +316,7 @@ static const CRPCCommand vRPCCommands[] =
{ "sendrawtransaction", &sendrawtransaction, false, false },
{ "getcheckpoint", &getcheckpoint, true, false },
{ "gencheckpoints", &gencheckpoints, true, false },
{ "publishcheckpoint", &publishcheckpoint, true, false },
{ "getchaintips", &getchaintips, true, false },
{ "invalidateblock", &invalidateblock, false, false },
{ "reconsiderblock", &reconsiderblock, false, false },
@@ -327,6 +327,7 @@ static const CRPCCommand vRPCCommands[] =
{ "checkwallet", &checkwallet, false, true},
{ "repairwallet", &repairwallet, false, true},
{ "resendtx", &resendtx, false, true},
{ "abandontransaction", &abandontransaction, true, true},
{ "makekeypair", &makekeypair, false, true},
{ "smsgenable", &smsgenable, false, false},
@@ -546,10 +547,17 @@ int ReadHTTP(std::basic_istream<char>& stream, map<string, string>& mapHeadersRe
bool HTTPAuthorized(map<string, string>& mapHeaders)
{
string strAuth = mapHeaders["authorization"];
if (strAuth.substr(0,6) != "Basic ")
if (strAuth.size() < 6 || strAuth.substr(0,6) != "Basic ")
return false;
string strUserPass64 = strAuth.substr(6); strUserPass64 = TrimString(strUserPass64);
string strUserPass = DecodeBase64(strUserPass64);
if (strUserPass64.empty())
return false;
string strUserPass;
try {
strUserPass = DecodeBase64(strUserPass64);
} catch (const std::exception&) {
return false;
}
return TimingResistantEqual(strUserPass, strRPCUserColonPass);
}
@@ -601,81 +609,29 @@ void ErrorReply(std::ostream& stream, const Object& objError, const Value& id)
stream << HTTPReply(nStatus, strReply, false) << std::flush;
}
bool ClientAllowed(const boost::asio::ip::address& address)
bool ClientAllowed(const std::string& strAddressIn)
{
// Make sure that IPv4-compatible and IPv4-mapped IPv6 addresses are treated as IPv4 addresses
if (address.is_v6()
&& address.to_v6().is_v4_mapped())
return ClientAllowed(make_address_v4(boost::asio::ip::v4_mapped, address.to_v6()));
// Treat IPv4-mapped IPv6 addresses (::ffff:a.b.c.d) as plain IPv4.
std::string strAddress = strAddressIn;
const std::string v4mapped = "::ffff:";
if (strAddress.compare(0, v4mapped.size(), v4mapped) == 0)
strAddress = strAddress.substr(v4mapped.size());
if (address == asio::ip::address_v4::loopback()
|| address == asio::ip::address_v6::loopback()
|| (address.is_v4()
// Check whether IPv4 addresses match 127.0.0.0/8 (loopback subnet)
&& (address.to_v4().to_uint() & 0xff000000) == 0x7f000000))
// Always allow loopback: ::1 and the 127.0.0.0/8 subnet.
if (strAddress == "::1" || strAddress.compare(0, 4, "127.") == 0)
return true;
const string strAddress = address.to_string();
const vector<string>& vAllow = mapMultiArgs["-rpcallowip"];
for (string strAllow : vAllow)
for (const string& strAllow : vAllow)
if (WildcardMatch(strAddress, strAllow))
return true;
return false;
}
//
// IOStream device that speaks SSL but can also speak non-SSL
// A single accepted RPC connection, backed by a raw socket exposed as a
// std::iostream so the HTTP/JSON/SSE/REST code below is transport-agnostic.
//
template <typename Protocol>
class SSLIOStreamDevice : public iostreams::device<iostreams::bidirectional> {
public:
SSLIOStreamDevice(asio::ssl::stream<typename Protocol::socket> &streamIn, bool fUseSSLIn) : stream(streamIn)
{
fUseSSL = fUseSSLIn;
fNeedHandshake = fUseSSLIn;
}
void handshake(ssl::stream_base::handshake_type role)
{
if (!fNeedHandshake) return;
fNeedHandshake = false;
stream.handshake(role);
}
std::streamsize read(char* s, std::streamsize n)
{
handshake(ssl::stream_base::server); // HTTPS servers read first
if (fUseSSL) return stream.read_some(asio::buffer(s, n));
return stream.next_layer().read_some(asio::buffer(s, n));
}
std::streamsize write(const char* s, std::streamsize n)
{
handshake(ssl::stream_base::client); // HTTPS clients write first
if (fUseSSL) return asio::write(stream, asio::buffer(s, n));
return asio::write(stream.next_layer(), asio::buffer(s, n));
}
bool connect(const std::string& server, const std::string& port)
{
ip::tcp::resolver resolver(stream.get_executor());
auto results = resolver.resolve(server, port);
boost::system::error_code error = asio::error::host_not_found;
for (const auto& ep : results)
{
stream.lowest_layer().close();
stream.lowest_layer().connect(ep.endpoint(), error);
if (!error)
break;
}
if (error)
return false;
return true;
}
private:
bool fNeedHandshake;
bool fUseSSL;
asio::ssl::stream<typename Protocol::socket>& stream;
};
class AcceptedConnection
{
public:
@@ -686,41 +642,32 @@ public:
virtual void close() = 0;
};
template <typename Protocol>
class AcceptedConnectionImpl : public AcceptedConnection
{
public:
AcceptedConnectionImpl(
const boost::asio::any_io_executor& executor,
ssl::context &context,
bool fUseSSL) :
sslStream(executor, context),
_d(sslStream, fUseSSL),
_stream(_d)
AcceptedConnectionImpl(SOCKET hSocketIn, const std::string& strPeer)
: hSocket(hSocketIn), peer(strPeer), _stream(hSocketIn)
{
}
virtual std::iostream& stream()
~AcceptedConnectionImpl() override
{
return _stream;
close();
}
virtual std::string peer_address_to_string() const
{
return peer.address().to_string();
}
std::iostream& stream() override { return _stream; }
std::string peer_address_to_string() const override { return peer; }
virtual void close()
void close() override
{
_stream.close();
if (hSocket != INVALID_SOCKET)
closesocket(hSocket); // sets hSocket = INVALID_SOCKET (see compat.h)
}
typename Protocol::endpoint peer;
asio::ssl::stream<typename Protocol::socket> sslStream;
private:
SSLIOStreamDevice<Protocol> _d;
iostreams::stream< SSLIOStreamDevice<Protocol> > _stream;
SOCKET hSocket;
std::string peer;
CSocketIOStream _stream;
};
void ThreadRPCServer(void* parg)
@@ -744,85 +691,6 @@ void ThreadRPCServer(void* parg)
printf("ThreadRPCServer exited\n");
}
// Forward declaration required for RPCListen
template <typename Protocol, typename SocketAcceptorService>
static void RPCAcceptHandler(boost::shared_ptr< basic_socket_acceptor<Protocol, SocketAcceptorService> > acceptor,
ssl::context& context,
bool fUseSSL,
AcceptedConnection* conn,
const boost::system::error_code& error);
/**
* Sets up I/O resources to accept and handle a new connection.
*/
template <typename Protocol, typename SocketAcceptorService>
static void RPCListen(boost::shared_ptr< basic_socket_acceptor<Protocol, SocketAcceptorService> > acceptor,
ssl::context& context,
const bool fUseSSL)
{
// Accept connection
AcceptedConnectionImpl<Protocol>* conn = new AcceptedConnectionImpl<Protocol>(acceptor->get_executor(), context, fUseSSL);
acceptor->async_accept(
conn->sslStream.lowest_layer(),
conn->peer,
[acceptor, &context, fUseSSL, conn](const boost::system::error_code& error) {
RPCAcceptHandler(acceptor, context, fUseSSL, conn, error);
});
}
/**
* Accept and handle incoming connection.
*/
template <typename Protocol, typename SocketAcceptorService>
static void RPCAcceptHandler(boost::shared_ptr< basic_socket_acceptor<Protocol, SocketAcceptorService> > acceptor,
ssl::context& context,
const bool fUseSSL,
AcceptedConnection* conn,
const boost::system::error_code& error)
{
vnThreadsRunning[THREAD_RPCLISTENER]++;
// Immediately start accepting new connections, except when we're cancelled or our socket is closed.
if (error != asio::error::operation_aborted
&& acceptor->is_open())
RPCListen(acceptor, context, fUseSSL);
AcceptedConnectionImpl<ip::tcp>* tcp_conn = dynamic_cast< AcceptedConnectionImpl<ip::tcp>* >(conn);
if (error)
{
if (error != asio::error::operation_aborted)
printf("RPC accept error from %s: %s (%d)\n",
tcp_conn ? tcp_conn->peer.address().to_string().c_str() : "unknown peer",
error.message().c_str(),
error.value());
delete conn;
vnThreadsRunning[THREAD_RPCLISTENER]--;
return;
}
// Restrict callers by IP. It is important to
// do this before starting client thread, to filter out
// certain DoS and misbehaving clients.
else if (tcp_conn
&& !ClientAllowed(tcp_conn->peer.address()))
{
// Only send a 403 if we're not using SSL to prevent a DoS during the SSL handshake.
if (!fUseSSL)
conn->stream() << HTTPReply(HTTP_FORBIDDEN, "", false) << std::flush;
delete conn;
}
// start HTTP client thread
else if (!NewThread(ThreadRPCServer3, conn)) {
printf("Failed to create RPC server client thread\n");
delete conn;
}
vnThreadsRunning[THREAD_RPCLISTENER]--;
}
void ThreadRPCServer2(void* parg)
{
printf("ThreadRPCServer started\n");
@@ -854,125 +722,89 @@ void ThreadRPCServer2(void* parg)
return;
}
const bool fUseSSL = GetBoolArg("-rpcssl");
asio::io_context io_service;
ssl::context context(ssl::context::sslv23);
if (fUseSSL)
{
context.set_options(ssl::context::no_sslv2);
fs::path pathCertFile(GetArg(std::string_view{"-rpcsslcertificatechainfile"}, std::string_view{"server.cert"}));
if (!pathCertFile.is_absolute()) pathCertFile = fs::path(GetDataDir()) / pathCertFile;
if (fs::exists(pathCertFile)) context.use_certificate_chain_file(pathCertFile.string());
else printf("ThreadRPCServer ERROR: missing server certificate file %s\n", pathCertFile.string().c_str());
fs::path pathPKFile(GetArg(std::string_view{"-rpcsslprivatekeyfile"}, std::string_view{"server.pem"}));
if (!pathPKFile.is_absolute()) pathPKFile = fs::path(GetDataDir()) / pathPKFile;
if (fs::exists(pathPKFile)) context.use_private_key_file(pathPKFile.string(), ssl::context::pem);
else printf("ThreadRPCServer ERROR: missing server private key file %s\n", pathPKFile.string().c_str());
string strCiphers = GetArg(std::string_view{"-rpcsslciphers"}, std::string_view{"TLSv1+HIGH:!SSLv2:!aNULL:!eNULL:!AH:!3DES:@STRENGTH"});
SSL_CTX_set_cipher_list(context.native_handle(), strCiphers.c_str());
if (GetBoolArg("-rpcssl")) {
printf("ThreadRPCServer WARNING: -rpcssl is no longer supported. RPC TLS "
"was removed together with the Boost.Asio dependency. To reach the "
"RPC port securely from another host, use an SSH tunnel, stunnel, "
"or Tor.\n");
}
// Try a dual IPv6/IPv4 socket, falling back to separate IPv4 and IPv6 sockets
const bool loopback = !mapArgs.count("-rpcallowip");
asio::ip::address bindAddress = loopback ? asio::ip::address_v6::loopback() : asio::ip::address_v6::any();
ip::tcp::endpoint endpoint(bindAddress, GetArg("-rpcport", GetDefaultRPCPort()));
boost::system::error_code v6_only_error;
boost::shared_ptr<ip::tcp::acceptor> acceptor(new ip::tcp::acceptor(io_service));
// Bind the loopback interface(s) unless the operator explicitly opened the
// RPC port to other hosts with -rpcallowip.
const bool loopbackOnly = !mapArgs.count("-rpcallowip");
const int nPort = (int)GetArg("-rpcport", GetDefaultRPCPort());
CSignal<void()> StopRequests;
bool fListening = false;
std::string strerr;
try
{
acceptor->open(endpoint.protocol());
acceptor->set_option(boost::asio::ip::tcp::acceptor::reuse_address(true));
// Try making the socket dual IPv6/IPv4 (if listening on the "any" address)
acceptor->set_option(boost::asio::ip::v6_only(loopback), v6_only_error);
acceptor->bind(endpoint);
acceptor->listen(socket_base::max_listen_connections);
RPCListen(acceptor, context, fUseSSL);
// Cancel outstanding listen-requests for this acceptor when shutting down.
// weak_ptr emulates signals2's .track(): if the acceptor has already been
// released by the time StopRequests fires, the slot is a no-op.
{
boost::weak_ptr<ip::tcp::acceptor> weak_acceptor(acceptor);
StopRequests.connect([weak_acceptor]() {
if (auto a = weak_acceptor.lock()) a->close();
});
}
fListening = true;
}
catch(boost::system::system_error &e)
{
strerr = strprintf(_("An error occurred while setting up the RPC port %u for listening on IPv6, falling back to IPv4: %s"), endpoint.port(), e.what());
}
try {
// If dual IPv6/IPv4 failed (or we're opening loopback interfaces only), open IPv4 separately
if (!fListening || loopback || v6_only_error)
{
bindAddress = loopback ? asio::ip::address_v4::loopback() : asio::ip::address_v4::any();
endpoint.address(bindAddress);
acceptor.reset(new ip::tcp::acceptor(io_service));
acceptor->open(endpoint.protocol());
acceptor->set_option(boost::asio::ip::tcp::acceptor::reuse_address(true));
acceptor->bind(endpoint);
acceptor->listen(socket_base::max_listen_connections);
RPCListen(acceptor, context, fUseSSL);
// See note above on weak_ptr-based .track() emulation.
{
boost::weak_ptr<ip::tcp::acceptor> weak_acceptor(acceptor);
StopRequests.connect([weak_acceptor]() {
if (auto a = weak_acceptor.lock()) a->close();
});
}
fListening = true;
}
}
catch(boost::system::system_error &e)
{
strerr = strprintf(_("An error occurred while setting up the RPC port %u for listening on IPv4: %s"), endpoint.port(), e.what());
}
if (!fListening) {
uiInterface.ThreadSafeMessageBox(strerr, _("Error"), CClientUIInterface::OK | CClientUIInterface::MODAL);
std::string strBindError;
std::vector<SOCKET> vListen = BindRPCSockets(nPort, loopbackOnly, strBindError);
if (vListen.empty()) {
uiInterface.ThreadSafeMessageBox(
strprintf(_("An error occurred while setting up the RPC port %d for listening: %s"),
nPort, strBindError.c_str()),
_("Error"), CClientUIInterface::OK | CClientUIInterface::MODAL);
StartShutdown();
return;
}
printf("RPC server listening on port %d (%s)\n", nPort,
loopbackOnly ? "loopback only" : "all interfaces");
// Accept loop. select() with a short timeout keeps the listener responsive
// to fShutdown. Each accepted connection is handed to its own handler thread
// (ThreadRPCServer3), preserving the previous thread-per-connection model
// (and keeping the blocking SSE handler working).
vnThreadsRunning[THREAD_RPCLISTENER]--;
while (!fShutdown)
{
// Use poll_one + sleep instead of blocking run_one so the thread
// remains responsive to fShutdown and can exit promptly.
if (!io_service.poll_one())
fd_set readset;
FD_ZERO(&readset);
SOCKET hSocketMax = 0;
for (SOCKET s : vListen) {
FD_SET(s, &readset);
if (s > hSocketMax) hSocketMax = s;
}
struct timeval timeout;
timeout.tv_sec = 0;
timeout.tv_usec = 100000; // 100 ms
int nSelect = select(hSocketMax + 1, &readset, nullptr, nullptr, &timeout);
if (nSelect <= 0)
continue; // timeout or interrupted — re-check fShutdown
for (SOCKET s : vListen)
{
io_service.restart();
MilliSleep(50);
if (!FD_ISSET(s, &readset))
continue;
struct sockaddr_storage ss;
socklen_t len = sizeof(ss);
SOCKET hConn = accept(s, (struct sockaddr*)&ss, &len);
if (hConn == INVALID_SOCKET) {
printf("RPC accept() failed\n");
continue;
}
const std::string strPeer = SockaddrToString((struct sockaddr*)&ss, len);
// Filter by IP before spawning a handler thread (DoS mitigation).
if (!ClientAllowed(strPeer)) {
{
CSocketIOStream s403(hConn);
s403 << HTTPReply(HTTP_FORBIDDEN, "", false) << std::flush;
}
closesocket(hConn);
continue;
}
AcceptedConnection* conn = new AcceptedConnectionImpl(hConn, strPeer);
if (!NewThread(ThreadRPCServer3, conn)) {
printf("Failed to create RPC server client thread\n");
delete conn; // destructor closes hConn
}
}
}
vnThreadsRunning[THREAD_RPCLISTENER]++;
// Safely shut down: close acceptors, then drain any remaining handlers
try {
StopRequests();
} catch (...) {
// Absorb bad_weak_ptr or other exceptions from stale tracked slots
}
io_service.poll(); // process cancellation callbacks so shared_ptrs are released
for (SOCKET s : vListen)
closesocket(s);
}
class JSONRequest
@@ -1128,6 +960,7 @@ void ThreadRPCServer3(void* parg)
AcceptedConnection *conn = (AcceptedConnection *) parg;
bool fRun = true;
try {
while (true)
{
if (fShutdown || !fRun)
@@ -1247,6 +1080,13 @@ void ThreadRPCServer3(void* parg)
}
}
} // end try
catch (std::exception& e) {
PrintException(&e, "ThreadRPCServer3()");
} catch (...) {
PrintException(NULL, "ThreadRPCServer3()");
}
delete conn;
{
LOCK(cs_THREAD_RPCHANDLER);
@@ -1296,16 +1136,14 @@ Object CallRPC(const string& strMethod, const Array& params)
"If the file does not exist, create it with owner-readable-only file permissions."),
GetConfigFile().string().c_str()));
// Connect to localhost
bool fUseSSL = GetBoolArg("-rpcssl");
asio::io_context io_service;
ssl::context context(ssl::context::sslv23);
context.set_options(ssl::context::no_sslv2);
asio::ssl::stream<asio::ip::tcp::socket> sslStream(io_service, context);
SSLIOStreamDevice<asio::ip::tcp> d(sslStream, fUseSSL);
iostreams::stream< SSLIOStreamDevice<asio::ip::tcp> > stream(d);
if (!d.connect(GetArg(std::string_view{"-rpcconnect"}, std::string_view{"127.0.0.1"}), GetArg(std::string_view{"-rpcport"}, itostr(GetDefaultRPCPort()))))
// Connect to the RPC server over a plain TCP socket. (RPC TLS was removed
// with the Boost.Asio dependency; tunnel the connection for remote use.)
SOCKET hSocket = ConnectRPCSocket(
GetArg(std::string_view{"-rpcconnect"}, std::string_view{"127.0.0.1"}),
(int)GetArg(std::string_view{"-rpcport"}, (int64_t)GetDefaultRPCPort()));
if (hSocket == INVALID_SOCKET)
throw runtime_error("couldn't connect to server");
CSocketIOStream stream(hSocket);
// HTTP basic authentication
string strUserPass64 = EncodeBase64(mapArgs["-rpcuser"] + ":" + mapArgs["-rpcpassword"]);
@@ -1321,6 +1159,7 @@ Object CallRPC(const string& strMethod, const Array& params)
map<string, string> mapHeaders;
string strReply;
int nStatus = ReadHTTP(stream, mapHeaders, strReply);
closesocket(hSocket);
if (nStatus == HTTP_UNAUTHORIZED)
throw runtime_error("incorrect rpcuser or rpcpassword (authorization failed)");
else if (nStatus >= 400 && nStatus != HTTP_BAD_REQUEST && nStatus != HTTP_NOT_FOUND && nStatus != HTTP_INTERNAL_SERVER_ERROR)
src/txdb-factory.cpp
+10 -8
View File
@@ -13,16 +13,20 @@ namespace fs = std::filesystem;
namespace {
// Pick the backend once per process. -chaindb is a startup flag; switching at
// runtime would require reopening every CTxDB instance, which the codebase
// doesn't currently support. We cache the resolved choice so subsequent
// MakeChainDB calls don't re-parse the argument.
// Pick the backend on every call. The daemon sets -chaindb once at startup
// and never changes it, so the per-call cost (a GetArg + tolower loop on a
// short string) is negligible compared to the cost of opening the chain DB.
// The earlier static-cache version broke test_chaindb_runtime, which
// legitimately toggles -chaindb across test cases to exercise both backends
// in the same process. Caching would freeze the first-seen choice.
enum class ChainDbKind { LevelDB, RocksDB };
ChainDbKind ResolveChainDbKind()
{
static const ChainDbKind kKind = []() {
std::string s = GetArg("-chaindb", std::string("leveldb"));
// RocksDB is the default backend. LevelDB remains selectable with
// -chaindb=leveldb and is retained as the migration source and fallback;
// its removal is deferred to a later phase after live-chain validation.
std::string s = GetArg("-chaindb", std::string("rocksdb"));
for (auto& c : s) c = std::tolower(static_cast<unsigned char>(c));
if (s == "leveldb")
@@ -33,8 +37,6 @@ ChainDbKind ResolveChainDbKind()
throw std::runtime_error(
"-chaindb=" + s + " is not a recognized backend. "
"Valid values: leveldb, rocksdb.");
}();
return kKind;
}
} // anonymous namespace
src/txdb-leveldb.cpp
+25 -2
View File
@@ -7,8 +7,6 @@
#include <filesystem>
#include <boost/version.hpp>
#include <leveldb/env.h>
#include <leveldb/cache.h>
#include <leveldb/filter_policy.h>
@@ -514,6 +512,20 @@ bool CTxDB::LoadBlockIndex()
nBestHeight = pindexBest->nHeight;
nBestChainTrust = pindexBest->nChainTrust;
// Heal pnext pointers along the active chain. Persisted hashNext can be
// stale or zeroed by crash-interrupted reorgs, which breaks
// GetKernelStakeModifier()'s forward walk and causes valid new
// proof-of-stake blocks to be rejected with "check kernel failed".
{
int nHealed = 0;
for (CBlockIndex* p = pindexBest; p && p->pprev; p = p->pprev)
{
if (p->pprev->pnext != p) { p->pprev->pnext = p; nHealed++; }
}
if (nHealed > 0)
printf("LoadBlockIndex(): healed %d pnext links on active chain\n", nHealed);
}
printf("STARTUP-PERF: best_chain %" PRId64 "ms\n", GetTimeMillis() - nPhaseStart);
nPhaseStart = GetTimeMillis();
@@ -610,7 +622,18 @@ bool CTxDB::LoadBlockIndex()
break;
CBlock block;
if (!block.ReadFromDisk(pindex))
{
// Snapshot-sourced chains have block headers + UTXOs but not raw
// block bodies on disk yet. Skip verification for those — the
// UTXO set itself was content-hash verified during LoadSnapshot.
// For non-snapshot chains, this remains a fatal error.
if (fLoadedFromSnapshot) {
printf("LoadBlockIndex(): block %d not on disk (snapshot-sourced), skipping verification\n",
pindex->nHeight);
continue;
}
return error("LoadBlockIndex() : block.ReadFromDisk failed");
}
if (nCheckLevel>0 && !block.CheckBlock(true, true, (nCheckLevel>6)))
{
printf("LoadBlockIndex() : *** found bad block at %d, hash=%s\n", pindex->nHeight, pindex->GetBlockHash().ToString().c_str());
src/txdb-rocksdb.cpp
+176 -11
View File
@@ -8,7 +8,6 @@
#include <filesystem>
#include <boost/version.hpp>
#include <rocksdb/cache.h>
#include <rocksdb/filter_policy.h>
@@ -31,6 +30,16 @@ namespace fs = std::filesystem;
// Global pointer for the RocksDB instance, shared across CRocksTxDB instances
// the same way the LevelDB backend shares its txdb singleton.
static rocksdb::DB* g_rocksdb = nullptr;
static bool g_cf_enabled = false;
// Non-batched writes bypass WAL fsync. The TxnCommit path handles durability;
// crash recovery replays from block files anyway. Default WriteOptions may
// vary across RocksDB versions, so we pin sync=false explicitly.
static const rocksdb::WriteOptions g_fastWriteOpts = []{
rocksdb::WriteOptions wo;
wo.sync = false;
return wo;
}();
namespace {
@@ -64,6 +73,39 @@ inline rocksdb::Status OpenRocksDB(const rocksdb::Options& opts,
return OpenRocksDBImpl(opts, path, dbptr, 0);
}
// Same SFINAE pattern for the column-family Open overload.
// Some RocksDB versions (MSYS2 MinGW) ship only the unique_ptr signature.
template<typename T>
inline auto OpenRocksDBCFImpl(const rocksdb::Options& opts, const std::string& path,
const std::vector<rocksdb::ColumnFamilyDescriptor>& cfDescs,
std::vector<rocksdb::ColumnFamilyHandle*>* handles,
T** dbptr, int)
-> decltype(rocksdb::DB::Open(opts, path, cfDescs, handles, dbptr))
{
return rocksdb::DB::Open(opts, path, cfDescs, handles, dbptr);
}
template<typename T>
inline rocksdb::Status OpenRocksDBCFImpl(const rocksdb::Options& opts, const std::string& path,
const std::vector<rocksdb::ColumnFamilyDescriptor>& cfDescs,
std::vector<rocksdb::ColumnFamilyHandle*>* handles,
T** dbptr, long)
{
std::unique_ptr<T> tmp;
auto s = rocksdb::DB::Open(opts, path, cfDescs, handles, &tmp);
if (s.ok()) *dbptr = tmp.release();
return s;
}
inline rocksdb::Status OpenRocksDBCF(const rocksdb::Options& opts,
const std::string& path,
const std::vector<rocksdb::ColumnFamilyDescriptor>& cfDescs,
std::vector<rocksdb::ColumnFamilyHandle*>* handles,
rocksdb::DB** dbptr)
{
return OpenRocksDBCFImpl(opts, path, cfDescs, handles, dbptr, 0);
}
} // anonymous namespace
static rocksdb::Options GetRocksOptions()
@@ -71,8 +113,9 @@ static rocksdb::Options GetRocksOptions()
rocksdb::Options opts;
opts.create_if_missing = false;
opts.compression = rocksdb::kSnappyCompression;
opts.max_open_files = 1000;
opts.write_buffer_size = 64 * 1048576;
opts.max_open_files = -1;
opts.write_buffer_size = 256 * 1048576;
opts.max_write_buffer_number = 4;
opts.IncreaseParallelism(); // Multi-threaded compaction.
opts.OptimizeLevelStyleCompaction(); // Sensible defaults for a LSM workload.
@@ -85,6 +128,9 @@ static rocksdb::Options GetRocksOptions()
return opts;
}
// Column-family partitioning is disabled (see CRocksTxDB::GetCF). All keys live
// in the default column family, mirroring the single-keyspace LevelDB backend.
static void open_rocksdb(rocksdb::Options& options, bool fRemoveOld = false)
{
fs::path directory = GetDataDir() / "rocksdb";
@@ -95,11 +141,52 @@ static void open_rocksdb(rocksdb::Options& options, bool fRemoveOld = false)
fs::create_directory(directory);
printf("Opening RocksDB in %s\n", directory.string().c_str());
rocksdb::Status status = OpenRocksDB(options, directory.string(), &g_rocksdb);
// Column-family partitioning is disabled (see CRocksTxDB::GetCF): all data
// lives in the default CF so writes, point reads, and full-keyspace
// iteration stay mutually consistent. New databases are therefore created
// single-CF.
//
// For openability we must still enumerate any column families that already
// exist on disk — RocksDB refuses to open a database unless every existing
// CF is named in the open call. Experimental pre-release databases may
// contain the old blockindex/txindex/utxo/addrindex CFs; we open them so
// the handle is valid, but never route to them. (Such a database would have
// chain data stranded in non-default CFs and should be re-migrated or
// reindexed; no production database is in that state.)
std::vector<std::string> existingCFs;
rocksdb::Options listOpts = options;
listOpts.create_if_missing = false;
rocksdb::DB::ListColumnFamilies(listOpts, directory.string(), &existingCFs);
std::vector<rocksdb::ColumnFamilyDescriptor> cfDescs;
cfDescs.push_back(rocksdb::ColumnFamilyDescriptor(
rocksdb::kDefaultColumnFamilyName, rocksdb::ColumnFamilyOptions(options)));
for (const auto& name : existingCFs) {
if (name == rocksdb::kDefaultColumnFamilyName)
continue; // default already added above
cfDescs.push_back(rocksdb::ColumnFamilyDescriptor(
name, rocksdb::ColumnFamilyOptions(options)));
}
std::vector<rocksdb::ColumnFamilyHandle*> handles;
rocksdb::Status status = OpenRocksDBCF(options, directory.string(),
cfDescs, &handles, &g_rocksdb);
if (!status.ok()) {
// Fallback: open without an explicit CF list (plain single-CF database).
printf("RocksDB CF open failed (%s), falling back to single-CF\n", status.ToString().c_str());
status = OpenRocksDB(options, directory.string(), &g_rocksdb);
if (!status.ok()) {
throw runtime_error(strprintf("open_rocksdb(): error opening database: %s",
status.ToString().c_str()));
}
return;
}
// We only ever route to the default CF, so keep CF routing off. Any extra
// handles opened above for legacy-database compatibility are intentionally
// left unused.
g_cf_enabled = false;
}
CRocksTxDB::CRocksTxDB(const char* pszMode)
@@ -235,6 +322,33 @@ bool CRocksTxDB::ScanBatch(const std::string& key, std::string* value, bool* del
return true;
}
// ─── CF routing helper ──────────────────────────────────────────────────────
// IMPORTANT: column-family partitioning is intentionally DISABLED.
//
// The earlier design split keys across per-prefix column families
// (blockindex/txindex/utxo/addrindex) for independent compaction. But the read
// path was never made CF-aware: both CRocksTxDB::NewIterator() and
// CRocksTxDB::LoadBlockIndex() iterate the DEFAULT column family only. With
// routing enabled, block-index records (and every other prefixed key) were
// written into non-default CFs, so:
// - LoadBlockIndex() loaded ZERO blocks,
// - UTXO snapshot dumps and address-index range scans saw nothing, and
// - the migration verifier (CollectStats) counted a record mismatch.
// This is why -chaindb=rocksdb "compiled clean but was never runtime-valid."
//
// Returning nullptr unconditionally routes ALL keys to the default CF, which
// makes writes, point reads, Exists, Erase, and full-keyspace iteration
// mutually consistent — and byte-identical to the single-keyspace LevelDB
// backend, which the migration and dual-backend equivalence tests rely on.
//
// Re-introducing CFs is tracked as a follow-up and requires CF-aware iterators
// in NewIterator()/LoadBlockIndex() (a multiplexed merge across CFs) before the
// prefix router below can be re-enabled.
rocksdb::ColumnFamilyHandle* CRocksTxDB::GetCF(const std::string& /*key*/) const
{
return nullptr; // single keyspace: always the default column family
}
bool CRocksTxDB::ReadRaw(const std::string& key, std::string& value) const
{
bool readFromDb = true;
@@ -245,10 +359,21 @@ bool CRocksTxDB::ReadRaw(const std::string& key, std::string& value) const
return false;
}
if (readFromDb) {
rocksdb::Status status = pdb->Get(rocksdb::ReadOptions(), key, &value);
rocksdb::ReadOptions ro;
auto* cf = GetCF(key);
rocksdb::Status status = cf ? pdb->Get(ro, cf, key, &value)
: pdb->Get(ro, key, &value);
if (!status.ok()) {
if (status.IsNotFound())
if (status.IsNotFound()) {
// If CFs are enabled and key wasn't in the target CF, also
// check the default CF (handles data written before CF migration)
if (g_cf_enabled && cf) {
rocksdb::Status status2 = pdb->Get(ro, key, &value);
if (!status2.ok()) return false;
return true;
}
return false;
}
printf("RocksDB read failure: %s\n", status.ToString().c_str());
return false;
}
@@ -258,12 +383,17 @@ bool CRocksTxDB::ReadRaw(const std::string& key, std::string& value) const
bool CRocksTxDB::WriteRaw(const std::string& key, const std::string& value)
{
auto* cf = GetCF(key);
if (activeBatch) {
if (cf)
activeBatch->Put(cf, key, value);
else
activeBatch->Put(key, value);
pendingBatch[key] = value;
return true;
}
rocksdb::Status status = pdb->Put(rocksdb::WriteOptions(), key, value);
rocksdb::Status status = cf ? pdb->Put(g_fastWriteOpts, cf, key, value)
: pdb->Put(g_fastWriteOpts, key, value);
if (!status.ok()) {
printf("RocksDB write failure: %s\n", status.ToString().c_str());
return false;
@@ -275,12 +405,17 @@ bool CRocksTxDB::EraseRaw(const std::string& key)
{
if (!pdb)
return false;
auto* cf = GetCF(key);
if (activeBatch) {
if (cf)
activeBatch->Delete(cf, key);
else
activeBatch->Delete(key);
pendingBatch[key] = std::nullopt;
return true;
}
rocksdb::Status status = pdb->Delete(rocksdb::WriteOptions(), key);
rocksdb::Status status = cf ? pdb->Delete(rocksdb::WriteOptions(), cf, key)
: pdb->Delete(rocksdb::WriteOptions(), key);
return (status.ok() || status.IsNotFound());
}
@@ -290,11 +425,20 @@ bool CRocksTxDB::ExistsRaw(const std::string& key) const
if (activeBatch) {
bool deleted = false;
if (ScanBatch(key, &unused, &deleted) && !deleted)
return true;
bool inBatch = ScanBatch(key, &unused, &deleted);
if (inBatch) {
return !deleted;
}
}
rocksdb::Status status = pdb->Get(rocksdb::ReadOptions(), key, &unused);
auto* cf = GetCF(key);
rocksdb::ReadOptions ro;
rocksdb::Status status = cf ? pdb->Get(ro, cf, key, &unused)
: pdb->Get(ro, key, &unused);
if (status.IsNotFound() && g_cf_enabled && cf) {
// Fallback to default CF for pre-migration data
status = pdb->Get(ro, key, &unused);
}
return status.IsNotFound() == false;
}
@@ -542,6 +686,20 @@ bool CRocksTxDB::LoadBlockIndex()
nBestHeight = pindexBest->nHeight;
nBestChainTrust = pindexBest->nChainTrust;
// Heal pnext pointers along the active chain. Persisted hashNext can be
// stale or zeroed by crash-interrupted reorgs, which breaks
// GetKernelStakeModifier()'s forward walk and causes valid new
// proof-of-stake blocks to be rejected with "check kernel failed".
{
int nHealed = 0;
for (CBlockIndex* p = pindexBest; p && p->pprev; p = p->pprev)
{
if (p->pprev->pnext != p) { p->pprev->pnext = p; nHealed++; }
}
if (nHealed > 0)
printf("LoadBlockIndex(): healed %d pnext links on active chain\n", nHealed);
}
printf("STARTUP-PERF: best_chain %" PRId64 "ms\n", GetTimeMillis() - nPhaseStart);
nPhaseStart = GetTimeMillis();
@@ -640,7 +798,14 @@ bool CRocksTxDB::LoadBlockIndex()
break;
CBlock block;
if (!block.ReadFromDisk(pindex))
{
if (fLoadedFromSnapshot) {
printf("LoadBlockIndex(): block %d not on disk (snapshot-sourced), skipping verification\n",
pindex->nHeight);
continue;
}
return error("LoadBlockIndex(): block.ReadFromDisk failed");
}
if (nCheckLevel > 0 && !block.CheckBlock(true, true, (nCheckLevel > 6)))
{
printf("LoadBlockIndex(): bad block at %d, hash=%s\n",
src/txdb-rocksdb.h
+28 -1
View File
@@ -10,10 +10,13 @@
#include <map>
#include <optional>
#include <string>
#include <unordered_map>
#include <vector>
#include <rocksdb/db.h>
#include <rocksdb/options.h>
#include <rocksdb/write_batch.h>
#include <rocksdb/utilities/db_ttl.h>
// RocksDB backend for the chain database.
//
@@ -49,6 +52,16 @@ public:
std::unique_ptr<CTxDBIteratorBase> NewIterator() const override;
// ─── Test-only friend accessor ──────────────────────────────────────────
// test_chaindb_runtime exercises the protected raw methods (ReadRaw /
// WriteRaw / EraseRaw / ExistsRaw) directly to verify the wrapper layer
// that the daemon uses at runtime when launched with -chaindb=rocksdb.
// We don't widen the public API just for the test — instead the test
// declares a ChainDbRuntimeTestAccessor struct that this class befriends,
// giving it the same access the class itself has. White-box test pattern,
// zero impact on production callers.
friend struct ChainDbRuntimeTestAccessor;
protected:
bool ReadRaw(const std::string& key, std::string& value) const override;
bool WriteRaw(const std::string& key, const std::string& value) override;
@@ -61,12 +74,26 @@ private:
rocksdb::Options options;
int nVersion;
// ─── Column family support (DISABLED) ────────────────────────────────────
// CF partitioning is intentionally off: the read path (NewIterator /
// LoadBlockIndex) only iterates the default CF, so all data must live there
// for scans to be correct. GetCF() therefore always returns nullptr (the
// default CF). See the long note in txdb-rocksdb.cpp's GetCF definition.
// These members are retained for a future CF-aware-iteration phase.
enum CfId : int { CF_DEFAULT = 0, CF_BLOCKINDEX, CF_TXINDEX, CF_UTXO, CF_ADDRINDEX, CF_COUNT };
rocksdb::ColumnFamilyHandle* cf_handles[CF_COUNT] = {};
bool cf_enabled = false; // Always false while CF routing is disabled.
// Returns the column family a key should live in. While CF partitioning is
// disabled this always returns nullptr (= default CF).
rocksdb::ColumnFamilyHandle* GetCF(const std::string& key) const;
// Parallel record of every pending write (value) or delete (nullopt) on
// activeBatch. Used by ScanBatch to answer "is this key already in the
// active batch?" without iterating the WriteBatch via Handler — Ubuntu's
// librocksdb-dev hides typeinfo for rocksdb::WriteBatch::Handler so a
// subclass-based scan fails to link there.
std::map<std::string, std::optional<std::string>> pendingBatch;
std::unordered_map<std::string, std::optional<std::string>> pendingBatch;
bool ScanBatch(const std::string& key, std::string* value, bool* deleted) const;
};
src/txdb.h
+3 -2
View File
@@ -17,8 +17,9 @@
// Factory: returns a chain-database handle whose concrete backend is chosen
// by the -chaindb command-line argument:
//
// -chaindb=leveldb (default — pending Phase-4 retirement)
// -chaindb=rocksdb
// -chaindb=rocksdb (default)
// -chaindb=leveldb (retained as migration source + fallback; pending
// retirement after live-chain validation)
//
// Callers receive a CTxDBBase*, so the rest of the codebase stays
// backend-agnostic. Mode strings ("r", "r+", "cr+") match the pre-existing
src/util.cpp
+54 -26
View File
@@ -41,18 +41,6 @@
#include "version.h"
#include "ui_interface.h"
// Work around clang compilation problem in Boost 1.46:
// /usr/include/boost/program_options/detail/config_file.hpp:163:17: error: call to function 'to_internal' that is neither visible in the template definition nor found by argument-dependent lookup
// See also: http://stackoverflow.com/questions/10020179/compilation-fail-in-boost-librairies-program-options
// http://clang.debian.net/status.php?version=3.0&key=CANNOT_FIND_FUNCTION
namespace boost {
namespace program_options {
std::string to_internal(const std::string&);
}
}
#include <boost/program_options/detail/config_file.hpp>
#include <boost/program_options/parsers.hpp>
#include <filesystem>
#include <fstream>
#include <thread>
@@ -1077,22 +1065,25 @@ std::filesystem::path GetDefaultDataDir()
#endif
}
// File-scope cache for GetDataDir() so ResetDataDirCache() can clear it.
namespace {
std::filesystem::path s_pathCached[2];
CCriticalSection s_csPathCached;
bool s_cachedPath[2] = {false, false};
}
const std::filesystem::path &GetDataDir(bool fNetSpecific)
{
namespace fs = std::filesystem;
static fs::path pathCached[2];
static CCriticalSection csPathCached;
static bool cachedPath[2] = {false, false};
fs::path &path = pathCached[fNetSpecific];
std::filesystem::path &path = s_pathCached[fNetSpecific];
// This can be called during exceptions by printf, so we cache the
// value so we don't have to do memory allocations after that.
if (cachedPath[fNetSpecific])
if (s_cachedPath[fNetSpecific])
return path;
LOCK(csPathCached);
LOCK(s_csPathCached);
if (mapArgs.count("-datadir")) {
path = fs::absolute(mapArgs["-datadir"]);
@@ -1108,10 +1099,22 @@ const std::filesystem::path &GetDataDir(bool fNetSpecific)
fs::create_directory(path);
cachedPath[fNetSpecific]=true;
s_cachedPath[fNetSpecific]=true;
return path;
}
// Test-only: invalidate the cached data dir so a subsequent GetDataDir() call
// re-reads mapArgs["-datadir"]. Required for unit tests that need to switch
// the active datadir after a previous fixture has already resolved it.
void ResetDataDirCache()
{
LOCK(s_csPathCached);
s_pathCached[0] = std::filesystem::path{};
s_pathCached[1] = std::filesystem::path{};
s_cachedPath[0] = false;
s_cachedPath[1] = false;
}
std::filesystem::path GetConfigFile()
{
std::filesystem::path pathConfigFile(GetArg(std::string_view{"-conf"}, std::string_view{"triangles.conf"}));
@@ -1126,20 +1129,45 @@ void ReadConfigFile(map<string, string>& mapSettingsRet,
if (!streamConfig.good())
return; // No triangles.conf file is OK
set<string> setOptions;
setOptions.insert("*");
// Minimal INI-style parser (replaces boost::program_options). Each line is
// "name = value"; lines whose first non-whitespace character is '#' are
// comments, and blank lines are ignored. Inline '#' is NOT treated as a
// comment, so values such as rpcpassword may contain '#'. This matches the
// lenient behavior of the previous config_file_iterator.
auto trim = [](std::string s) -> std::string {
const char* ws = " \t\r\n";
size_t b = s.find_first_not_of(ws);
if (b == std::string::npos)
return std::string();
size_t e = s.find_last_not_of(ws);
return s.substr(b, e - b + 1);
};
for (boost::program_options::detail::config_file_iterator it(streamConfig, setOptions), end; it != end; ++it)
std::string line;
while (std::getline(streamConfig, line))
{
std::string trimmed = trim(line);
if (trimmed.empty() || trimmed[0] == '#')
continue;
size_t nEq = trimmed.find('=');
if (nEq == std::string::npos)
continue; // malformed line without '='; skip
std::string strName = trim(trimmed.substr(0, nEq));
std::string strValue = trim(trimmed.substr(nEq + 1));
if (strName.empty())
continue;
string strKey = string("-") + strName;
// Don't overwrite existing settings so command line settings override triangles.conf
string strKey = string("-") + it->string_key;
if (mapSettingsRet.count(strKey) == 0)
{
mapSettingsRet[strKey] = it->value[0];
mapSettingsRet[strKey] = strValue;
// interpret nofoo=1 as foo=0 (and nofoo=0 as foo=1) as long as foo not set)
InterpretNegativeSetting(strKey, mapSettingsRet);
}
mapMultiSettingsRet[strKey].push_back(it->value[0]);
mapMultiSettingsRet[strKey].push_back(strValue);
}
}
src/walletdb-base.h
+115
View File
@@ -0,0 +1,115 @@
// Copyright (c) 2026 The Triangles developers.
// Distributed under the MIT/X11 software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
//
// Backend-agnostic wallet storage seam.
//
// Historically CWalletDB derived directly from CDB (Berkeley DB). To allow the
// wallet to be stored in SQLite instead, storage is abstracted behind two
// interfaces modeled on Bitcoin Core's WalletDatabase / DatabaseBatch:
//
// WalletDatabase - owns the on-disk database (open/close/flush/backup/
// rewrite) and hands out batches.
// WalletBatch - a unit of work against the database: raw byte-level
// Read/Write/Erase/Exists, a cursor for full scans, and an
// optional atomic transaction.
//
// Only RAW BYTES cross this interface. All key/value (de)serialization stays in
// CWalletDB via CDataStream with SER_DISK / CLIENT_VERSION, exactly as before,
// so the on-disk record encoding is identical across backends. That byte
// identity is what makes the Berkeley -> SQLite migration a verbatim key/value
// copy.
#ifndef TRIANGLES_WALLETDB_BASE_H
#define TRIANGLES_WALLETDB_BASE_H
#include <memory>
#include <string>
#include <vector>
using KeyBytes = std::vector<unsigned char>;
using ValueBytes = std::vector<unsigned char>;
// Result of advancing a cursor.
enum class WalletCursorStatus { MORE, DONE, FAIL };
// Forward scan over every record in a database. Yields raw serialized
// key/value bytes; the caller deserializes. Cursors do not observe uncommitted
// writes in an open transaction (all wallet scan sites run outside txns).
class WalletCursor
{
public:
virtual ~WalletCursor() = default;
virtual WalletCursorStatus Next(KeyBytes& key, ValueBytes& value) = 0;
};
// A unit of work against a wallet database.
class WalletBatch
{
public:
virtual ~WalletBatch() = default;
// Byte-level accessors. WriteKey honors fOverwrite (false => fail if the
// key already exists, matching Berkeley's DB_NOOVERWRITE). EraseKey returns
// true when the key is gone afterwards (including "was not present").
virtual bool ReadKey(const KeyBytes& key, ValueBytes& value) = 0;
virtual bool WriteKey(const KeyBytes& key, const ValueBytes& value, bool fOverwrite = true) = 0;
virtual bool EraseKey(const KeyBytes& key) = 0;
virtual bool HasKey(const KeyBytes& key) = 0;
// Full-database scan.
virtual std::unique_ptr<WalletCursor> GetNewCursor() = 0;
// Atomic transaction around a group of writes/erases. At most one may be
// open per batch at a time.
virtual bool TxnBegin() = 0;
virtual bool TxnCommit() = 0;
virtual bool TxnAbort() = 0;
virtual void Close() = 0;
};
// An on-disk wallet database.
class WalletDatabase
{
public:
virtual ~WalletDatabase() = default;
// Hand out a batch. flush_on_close asks the backend to flush durable state
// when the batch is destroyed (Berkeley parity for the common write path).
virtual std::unique_ptr<WalletBatch> MakeBatch(bool flush_on_close = true) = 0;
// Rewrite the database compactly, optionally skipping records whose key
// begins with pszSkip (used by the wallet to drop the unencrypted "key"
// records after encryption). Berkeley implements this via CDB::Rewrite;
// SQLite implements it via VACUUM (+ optional delete of skipped keys).
virtual bool Rewrite(const char* pszSkip = nullptr) = 0;
// Copy the live database to a destination path (wallet backup).
virtual bool Backup(const std::string& strDest) const = 0;
// Durability / lifecycle.
virtual void Flush() = 0;
virtual void Close() = 0;
// Integrity check before first use. Fills strError on failure.
virtual bool Verify(std::string& strError) = 0;
// Human-readable identifier for logging (filename or path).
virtual std::string Filename() const = 0;
};
// Backend selector, parsed from -walletdb. SQLite is the default; Berkeley is
// retained for one release as a fallback and as the migration source.
enum class WalletDbKind { SQLite, Berkeley };
// Resolve the configured wallet backend from -walletdb (default: SQLite).
WalletDbKind ResolveWalletDbKind();
// Open (creating if needed) the wallet database for the configured backend.
// strFilename is the logical wallet name (e.g. "wallet.dat"); the SQLite
// backend stores it as "<name>" under the data dir, Berkeley as before.
std::unique_ptr<WalletDatabase> MakeWalletDatabase(const std::string& strFilename,
std::string& strError);
#endif // TRIANGLES_WALLETDB_BASE_H
src/walletdb-batch.h
+146
View File
@@ -0,0 +1,146 @@
// Copyright (c) 2026 The Triangles developers.
// Distributed under the MIT/X11 software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
//
// Typed, backend-agnostic wallet batch — the bridge between CWalletDB's typed
// record calls and the raw byte-level WalletBatch interface (walletdb-base.h).
//
// It reproduces the exact serialization behavior of the old Berkeley CDB
// (CDataStream with SER_DISK / CLIENT_VERSION), so the bytes written are
// identical regardless of backend and CWalletDB's call sites need only change
// their base class — the Read/Write/Erase/Exists template calls are unchanged.
//
// CWalletDB is intended to derive from CWalletBatchTyped (replacing `: public
// CDB`). The Berkeley cursor methods CWalletDB used directly (GetAtCursor,
// ReadAtCursor with DB_NEXT/DB_SET_RANGE) map onto StartCursor()/NextRecord()
// here, which iterate the whole keyspace; range-seek call sites filter in the
// loop, as the SQLite cursor does not support keyed range seeks.
#ifndef TRIANGLES_WALLETDB_BATCH_H
#define TRIANGLES_WALLETDB_BATCH_H
#include "walletdb-base.h"
#include "serialize.h" // CDataStream, SER_DISK
#include "version.h" // CLIENT_VERSION
#include <memory>
#include <stdexcept>
#include <string>
class CWalletBatchTyped
{
public:
explicit CWalletBatchTyped(std::unique_ptr<WalletBatch> batch)
: m_batch(std::move(batch)) {}
virtual ~CWalletBatchTyped() { Close(); }
void Close() { m_batch.reset(); }
bool IsNull() const { return m_batch == nullptr; }
// ── Transactions ─────────────────────────────────────────────────────────
bool TxnBegin() { return m_batch && m_batch->TxnBegin(); }
bool TxnCommit() { return m_batch && m_batch->TxnCommit(); }
bool TxnAbort() { return m_batch && m_batch->TxnAbort(); }
protected:
std::unique_ptr<WalletBatch> m_batch;
// ── Typed accessors (serialize key/value, dispatch to the raw batch) ──────
template <typename K, typename T>
bool Read(const K& key, T& value)
{
if (!m_batch) return false;
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
ssKey.reserve(1000);
ssKey << key;
KeyBytes vKey(ssKey.begin(), ssKey.end());
ValueBytes vValue;
if (!m_batch->ReadKey(vKey, vValue))
return false;
try {
CDataStream ssValue(reinterpret_cast<const char*>(vValue.data()),
reinterpret_cast<const char*>(vValue.data()) + vValue.size(),
SER_DISK, CLIENT_VERSION);
ssValue >> value;
} catch (const std::exception&) {
return false;
}
return true;
}
template <typename K, typename T>
bool Write(const K& key, const T& value, bool fOverwrite = true)
{
if (!m_batch) return false;
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
ssKey.reserve(1000);
ssKey << key;
KeyBytes vKey(ssKey.begin(), ssKey.end());
CDataStream ssValue(SER_DISK, CLIENT_VERSION);
ssValue.reserve(10000);
ssValue << value;
ValueBytes vValue(ssValue.begin(), ssValue.end());
return m_batch->WriteKey(vKey, vValue, fOverwrite);
}
template <typename K>
bool Erase(const K& key)
{
if (!m_batch) return false;
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
ssKey.reserve(1000);
ssKey << key;
KeyBytes vKey(ssKey.begin(), ssKey.end());
return m_batch->EraseKey(vKey);
}
template <typename K>
bool Exists(const K& key)
{
if (!m_batch) return false;
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
ssKey.reserve(1000);
ssKey << key;
KeyBytes vKey(ssKey.begin(), ssKey.end());
return m_batch->HasKey(vKey);
}
// ── Cursor ────────────────────────────────────────────────────────────────
// Replaces CDB::GetCursor()/ReadAtCursor(). Open a cursor, then call
// NextRecord() repeatedly: returns true and fills the streams while records
// remain, false at end-of-data, and sets fError on failure.
std::unique_ptr<WalletCursor> StartCursor()
{
if (!m_batch) return nullptr;
return m_batch->GetNewCursor();
}
bool NextRecord(WalletCursor& cursor, CDataStream& ssKey, CDataStream& ssValue, bool& fError)
{
fError = false;
KeyBytes vKey;
ValueBytes vValue;
switch (cursor.Next(vKey, vValue)) {
case WalletCursorStatus::MORE:
ssKey.SetType(SER_DISK);
ssKey.clear();
ssKey.write(reinterpret_cast<const char*>(vKey.data()), vKey.size());
ssValue.SetType(SER_DISK);
ssValue.clear();
ssValue.write(reinterpret_cast<const char*>(vValue.data()), vValue.size());
return true;
case WalletCursorStatus::DONE:
return false;
case WalletCursorStatus::FAIL:
default:
fError = true;
return false;
}
}
};
#endif // TRIANGLES_WALLETDB_BATCH_H
src/walletdb-factory.cpp
+55
View File
@@ -0,0 +1,55 @@
// Copyright (c) 2026 The Triangles developers.
// Distributed under the MIT/X11 software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include "walletdb-base.h"
#include "walletdb-sqlite.h"
#include "util.h"
#include <cctype>
#include <filesystem>
#include <stdexcept>
#include <string>
namespace fs = std::filesystem;
WalletDbKind ResolveWalletDbKind()
{
// SQLite is the default wallet backend. Berkeley DB is retained for one
// release as a fallback (-walletdb=bdb) and as the migration source.
std::string s = GetArg("-walletdb", std::string("sqlite"));
for (auto& c : s) c = std::tolower(static_cast<unsigned char>(c));
if (s == "sqlite")
return WalletDbKind::SQLite;
if (s == "bdb" || s == "berkeley")
return WalletDbKind::Berkeley;
throw std::runtime_error(
"-walletdb=" + s + " is not a recognized wallet backend. "
"Valid values: sqlite, bdb.");
}
std::unique_ptr<WalletDatabase> MakeWalletDatabase(const std::string& strFilename,
std::string& strError)
{
const fs::path path = GetDataDir() / strFilename;
switch (ResolveWalletDbKind()) {
case WalletDbKind::SQLite: {
auto db = std::make_unique<SQLiteDatabase>(path);
if (!db->Open(strError))
return nullptr;
return db;
}
case WalletDbKind::Berkeley:
// The Berkeley backend is still served by the legacy CWalletDB/CDB code
// path. The thin BerkeleyDatabase adapter that plugs the existing
// CDBEnv/CDB into this seam is added during CWalletDB integration; see
// WALLET-SQLITE-MIGRATION.md. Until then, selecting -walletdb=bdb keeps
// the original code path rather than routing through MakeWalletDatabase.
strError = "Berkeley backend uses the legacy wallet path; not served by MakeWalletDatabase yet.";
return nullptr;
}
return nullptr; // unreachable
}
src/walletdb-sqlite.cpp
+364
View File
@@ -0,0 +1,364 @@
// Copyright (c) 2026 The Triangles developers.
// Distributed under the MIT/X11 software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include "walletdb-sqlite.h"
#include "util.h"
#include <cstring>
namespace fs = std::filesystem;
// ─── helpers ────────────────────────────────────────────────────────────────
// Bind a byte buffer as a BLOB parameter (1-based index). SQLITE_TRANSIENT so
// SQLite copies the bytes; the source vector need not outlive the step.
static int BindBlob(sqlite3_stmt* stmt, int idx, const std::vector<unsigned char>& v)
{
// A zero-length blob still binds correctly with a non-null pointer.
const void* p = v.empty() ? "" : static_cast<const void*>(v.data());
return sqlite3_bind_blob(stmt, idx, p, static_cast<int>(v.size()), SQLITE_TRANSIENT);
}
static void ColumnBlob(sqlite3_stmt* stmt, int col, std::vector<unsigned char>& out)
{
const unsigned char* p = static_cast<const unsigned char*>(sqlite3_column_blob(stmt, col));
int n = sqlite3_column_bytes(stmt, col);
out.assign(p, p + (n > 0 ? n : 0));
}
// ─── SQLiteDatabase ──────────────────────────────────────────────────────────
SQLiteDatabase::SQLiteDatabase(const fs::path& file_path)
: m_file_path(file_path)
{
}
SQLiteDatabase::~SQLiteDatabase()
{
Close();
}
bool SQLiteDatabase::ExecOrError(const char* sql, std::string& strError) const
{
char* errmsg = nullptr;
int rc = sqlite3_exec(m_db, sql, nullptr, nullptr, &errmsg);
if (rc != SQLITE_OK) {
strError = strprintf("SQLite: '%s' failed: %s", sql, errmsg ? errmsg : sqlite3_errstr(rc));
if (errmsg) sqlite3_free(errmsg);
return false;
}
return true;
}
bool SQLiteDatabase::Open(std::string& strError)
{
if (m_db)
return true;
int flags = SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE | SQLITE_OPEN_FULLMUTEX;
int rc = sqlite3_open_v2(m_file_path.string().c_str(), &m_db, flags, nullptr);
if (rc != SQLITE_OK) {
strError = strprintf("Failed to open SQLite wallet %s: %s",
m_file_path.string().c_str(), sqlite3_errstr(rc));
if (m_db) { sqlite3_close(m_db); m_db = nullptr; }
return false;
}
// Block (rather than fail) for up to 5s if another handle holds the lock.
sqlite3_busy_timeout(m_db, 5000);
// Durability + integrity pragmas. FULL fsync on commit — a wallet must not
// lose a freshly-written key on power loss.
if (!ExecOrError("PRAGMA synchronous = FULL;", strError)) return false;
if (!ExecOrError("PRAGMA foreign_keys = ON;", strError)) return false;
// Fail loudly instead of silently truncating an over-long blob.
if (!ExecOrError("PRAGMA cell_size_check = ON;", strError)) return false;
// Identify our schema via application_id / user_version. A brand-new file
// reports 0/0; an existing file must match ours (refuse foreign DBs).
int appId = 0, userVer = 0;
{
sqlite3_stmt* st = nullptr;
if (sqlite3_prepare_v2(m_db, "PRAGMA application_id;", -1, &st, nullptr) == SQLITE_OK &&
sqlite3_step(st) == SQLITE_ROW)
appId = sqlite3_column_int(st, 0);
sqlite3_finalize(st);
st = nullptr;
if (sqlite3_prepare_v2(m_db, "PRAGMA user_version;", -1, &st, nullptr) == SQLITE_OK &&
sqlite3_step(st) == SQLITE_ROW)
userVer = sqlite3_column_int(st, 0);
sqlite3_finalize(st);
}
if (appId != 0 && appId != SQLITE_WALLET_APP_ID) {
strError = strprintf("%s is not a Triangles SQLite wallet (application_id=0x%08x)",
m_file_path.string().c_str(), appId);
sqlite3_close(m_db);
m_db = nullptr;
return false;
}
if (userVer > SQLITE_WALLET_SCHEMA_VERSION) {
strError = strprintf("%s was written by a newer wallet (schema v%d > v%d)",
m_file_path.string().c_str(), userVer, SQLITE_WALLET_SCHEMA_VERSION);
sqlite3_close(m_db);
m_db = nullptr;
return false;
}
// Create schema (idempotent) and stamp identity on fresh files.
if (!ExecOrError("CREATE TABLE IF NOT EXISTS main "
"(key BLOB PRIMARY KEY NOT NULL, value BLOB NOT NULL);", strError))
return false;
if (appId == 0) {
std::string set = strprintf("PRAGMA application_id = %d;", SQLITE_WALLET_APP_ID);
if (!ExecOrError(set.c_str(), strError)) return false;
}
{
std::string set = strprintf("PRAGMA user_version = %d;", SQLITE_WALLET_SCHEMA_VERSION);
if (!ExecOrError(set.c_str(), strError)) return false;
}
printf("SQLite wallet opened: %s\n", m_file_path.string().c_str());
return true;
}
std::unique_ptr<WalletBatch> SQLiteDatabase::MakeBatch(bool /*flush_on_close*/)
{
return std::make_unique<SQLiteBatch>(*this);
}
bool SQLiteDatabase::Rewrite(const char* /*pszSkip*/)
{
// SQLite reclaims space and defragments via VACUUM. The wallet erases
// superseded records (e.g. unencrypted keys after encryption) explicitly,
// so the pszSkip filter that the Berkeley backend used is unnecessary here.
if (!m_db)
return false;
std::string err;
if (!ExecOrError("VACUUM;", err)) {
printf("SQLiteDatabase::Rewrite VACUUM failed: %s\n", err.c_str());
return false;
}
return true;
}
bool SQLiteDatabase::Backup(const std::string& strDest) const
{
if (!m_db)
return false;
sqlite3* pDest = nullptr;
if (sqlite3_open_v2(strDest.c_str(), &pDest,
SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE, nullptr) != SQLITE_OK) {
printf("SQLiteDatabase::Backup cannot open destination %s: %s\n",
strDest.c_str(), pDest ? sqlite3_errmsg(pDest) : "?");
if (pDest) sqlite3_close(pDest);
return false;
}
sqlite3_backup* bk = sqlite3_backup_init(pDest, "main", m_db, "main");
bool ok = false;
if (bk) {
sqlite3_backup_step(bk, -1); // copy entire DB in one shot
int rc = sqlite3_backup_finish(bk);
ok = (rc == SQLITE_OK);
if (!ok)
printf("SQLiteDatabase::Backup failed: %s\n", sqlite3_errstr(rc));
} else {
printf("SQLiteDatabase::Backup init failed: %s\n", sqlite3_errmsg(pDest));
}
sqlite3_close(pDest);
return ok;
}
void SQLiteDatabase::Flush()
{
// No-op: with synchronous=FULL and rollback journaling, each committed
// transaction is already durable. (If WAL is ever enabled, checkpoint here.)
}
void SQLiteDatabase::Close()
{
if (m_db) {
sqlite3_close(m_db);
m_db = nullptr;
}
}
bool SQLiteDatabase::Verify(std::string& strError)
{
if (!m_db) {
strError = "SQLite database not open";
return false;
}
sqlite3_stmt* st = nullptr;
if (sqlite3_prepare_v2(m_db, "PRAGMA integrity_check;", -1, &st, nullptr) != SQLITE_OK) {
strError = strprintf("integrity_check prepare failed: %s", sqlite3_errmsg(m_db));
return false;
}
bool ok = false;
if (sqlite3_step(st) == SQLITE_ROW) {
const unsigned char* res = sqlite3_column_text(st, 0);
ok = (res && std::strcmp(reinterpret_cast<const char*>(res), "ok") == 0);
if (!ok)
strError = strprintf("integrity_check: %s", res ? reinterpret_cast<const char*>(res) : "(null)");
} else {
strError = "integrity_check returned no rows";
}
sqlite3_finalize(st);
return ok;
}
// ─── SQLiteBatch ──────────────────────────────────────────────────────────────
SQLiteBatch::SQLiteBatch(SQLiteDatabase& database)
: m_database(database)
{
PrepareStatements();
}
bool SQLiteBatch::PrepareStatements()
{
sqlite3* db = m_database.Handle();
if (!db)
return false;
struct { sqlite3_stmt** out; const char* sql; } stmts[] = {
{ &m_read_stmt, "SELECT value FROM main WHERE key = ?;" },
{ &m_insert_stmt, "INSERT OR REPLACE INTO main (key, value) VALUES (?, ?);" },
{ &m_overwrite_stmt, "INSERT INTO main (key, value) VALUES (?, ?);" },
{ &m_delete_stmt, "DELETE FROM main WHERE key = ?;" },
};
for (auto& s : stmts) {
if (*s.out) continue;
if (sqlite3_prepare_v2(db, s.sql, -1, s.out, nullptr) != SQLITE_OK) {
printf("SQLiteBatch: prepare failed for '%s': %s\n", s.sql, sqlite3_errmsg(db));
return false;
}
}
return true;
}
void SQLiteBatch::Close()
{
sqlite3_stmt* all[] = { m_read_stmt, m_insert_stmt, m_overwrite_stmt, m_delete_stmt };
for (auto* st : all)
if (st) sqlite3_finalize(st);
m_read_stmt = m_insert_stmt = m_overwrite_stmt = m_delete_stmt = nullptr;
}
bool SQLiteBatch::ReadKey(const KeyBytes& key, ValueBytes& value)
{
if (!m_read_stmt) return false;
sqlite3_reset(m_read_stmt);
sqlite3_clear_bindings(m_read_stmt);
if (BindBlob(m_read_stmt, 1, key) != SQLITE_OK)
return false;
bool found = false;
if (sqlite3_step(m_read_stmt) == SQLITE_ROW) {
ColumnBlob(m_read_stmt, 0, value);
found = true;
}
sqlite3_reset(m_read_stmt);
return found;
}
bool SQLiteBatch::WriteKey(const KeyBytes& key, const ValueBytes& value, bool fOverwrite)
{
sqlite3_stmt* st = fOverwrite ? m_insert_stmt : m_overwrite_stmt;
if (!st) return false;
sqlite3_reset(st);
sqlite3_clear_bindings(st);
if (BindBlob(st, 1, key) != SQLITE_OK) return false;
if (BindBlob(st, 2, value) != SQLITE_OK) return false;
int rc = sqlite3_step(st);
sqlite3_reset(st);
if (rc == SQLITE_DONE)
return true;
// Non-overwrite insert hitting an existing key => constraint violation,
// which mirrors Berkeley's DB_NOOVERWRITE returning false (not an error).
if (!fOverwrite && (rc == SQLITE_CONSTRAINT))
return false;
printf("SQLiteBatch::WriteKey step failed: %s\n", sqlite3_errstr(rc));
return false;
}
bool SQLiteBatch::EraseKey(const KeyBytes& key)
{
if (!m_delete_stmt) return false;
sqlite3_reset(m_delete_stmt);
sqlite3_clear_bindings(m_delete_stmt);
if (BindBlob(m_delete_stmt, 1, key) != SQLITE_OK)
return false;
int rc = sqlite3_step(m_delete_stmt);
sqlite3_reset(m_delete_stmt);
// DONE whether or not a row matched — "key is gone" either way.
return rc == SQLITE_DONE;
}
bool SQLiteBatch::HasKey(const KeyBytes& key)
{
if (!m_read_stmt) return false;
sqlite3_reset(m_read_stmt);
sqlite3_clear_bindings(m_read_stmt);
if (BindBlob(m_read_stmt, 1, key) != SQLITE_OK)
return false;
bool present = (sqlite3_step(m_read_stmt) == SQLITE_ROW);
sqlite3_reset(m_read_stmt);
return present;
}
namespace {
class SQLiteCursor final : public WalletCursor
{
public:
explicit SQLiteCursor(sqlite3_stmt* stmt) : m_stmt(stmt) {}
~SQLiteCursor() override { if (m_stmt) sqlite3_finalize(m_stmt); }
WalletCursorStatus Next(KeyBytes& key, ValueBytes& value) override
{
if (!m_stmt) return WalletCursorStatus::FAIL;
int rc = sqlite3_step(m_stmt);
if (rc == SQLITE_DONE) return WalletCursorStatus::DONE;
if (rc != SQLITE_ROW) return WalletCursorStatus::FAIL;
ColumnBlob(m_stmt, 0, key);
ColumnBlob(m_stmt, 1, value);
return WalletCursorStatus::MORE;
}
private:
sqlite3_stmt* m_stmt;
};
} // namespace
std::unique_ptr<WalletCursor> SQLiteBatch::GetNewCursor()
{
sqlite3* db = m_database.Handle();
if (!db) return nullptr;
sqlite3_stmt* st = nullptr;
if (sqlite3_prepare_v2(db, "SELECT key, value FROM main;", -1, &st, nullptr) != SQLITE_OK) {
printf("SQLiteBatch::GetNewCursor prepare failed: %s\n", sqlite3_errmsg(db));
return nullptr;
}
return std::make_unique<SQLiteCursor>(st);
}
bool SQLiteBatch::TxnBegin()
{
return sqlite3_exec(m_database.Handle(), "BEGIN TRANSACTION;", nullptr, nullptr, nullptr) == SQLITE_OK;
}
bool SQLiteBatch::TxnCommit()
{
return sqlite3_exec(m_database.Handle(), "COMMIT TRANSACTION;", nullptr, nullptr, nullptr) == SQLITE_OK;
}
bool SQLiteBatch::TxnAbort()
{
return sqlite3_exec(m_database.Handle(), "ROLLBACK TRANSACTION;", nullptr, nullptr, nullptr) == SQLITE_OK;
}
src/walletdb-sqlite.h
+99
View File
@@ -0,0 +1,99 @@
// Copyright (c) 2026 The Triangles developers.
// Distributed under the MIT/X11 software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
//
// SQLite backend for the wallet database. Stores every wallet record as a row
// in a single table:
//
// CREATE TABLE main (key BLOB PRIMARY KEY NOT NULL, value BLOB NOT NULL);
//
// The key/value blobs are the exact serialized bytes CWalletDB already
// produces (SER_DISK / CLIENT_VERSION), so a SQLite wallet is byte-for-byte
// equivalent in content to the Berkeley wallet.dat it was migrated from.
//
// Modeled on Bitcoin Core's SQLiteDatabase / SQLiteBatch.
#ifndef TRIANGLES_WALLETDB_SQLITE_H
#define TRIANGLES_WALLETDB_SQLITE_H
#include "walletdb-base.h"
#include <filesystem>
#include <string>
#include <sqlite3.h>
class SQLiteDatabase;
// A batch (and optional transaction) against a SQLiteDatabase. Holds prepared
// statements bound to the shared connection owned by SQLiteDatabase.
class SQLiteBatch final : public WalletBatch
{
public:
explicit SQLiteBatch(SQLiteDatabase& database);
~SQLiteBatch() override { Close(); }
bool ReadKey(const KeyBytes& key, ValueBytes& value) override;
bool WriteKey(const KeyBytes& key, const ValueBytes& value, bool fOverwrite = true) override;
bool EraseKey(const KeyBytes& key) override;
bool HasKey(const KeyBytes& key) override;
std::unique_ptr<WalletCursor> GetNewCursor() override;
bool TxnBegin() override;
bool TxnCommit() override;
bool TxnAbort() override;
void Close() override;
private:
SQLiteDatabase& m_database;
// Prepared statements (lazily compiled on first use, finalized on Close).
sqlite3_stmt* m_read_stmt = nullptr;
sqlite3_stmt* m_insert_stmt = nullptr; // INSERT OR REPLACE
sqlite3_stmt* m_overwrite_stmt = nullptr; // INSERT (fail if exists)
sqlite3_stmt* m_delete_stmt = nullptr;
bool PrepareStatements();
};
// The on-disk SQLite wallet database. Owns the single sqlite3 connection that
// all of its batches share (wallet access is serialized by the wallet's own
// locks, matching the Berkeley backend's single-environment model).
class SQLiteDatabase final : public WalletDatabase
{
public:
// file_path: absolute path to the .dat file on disk.
explicit SQLiteDatabase(const std::filesystem::path& file_path);
~SQLiteDatabase() override;
// Open the connection, apply pragmas, and create the schema if absent.
// Returns false (with strError set) on failure.
bool Open(std::string& strError);
std::unique_ptr<WalletBatch> MakeBatch(bool flush_on_close = true) override;
bool Rewrite(const char* pszSkip = nullptr) override;
bool Backup(const std::string& strDest) const override;
void Flush() override;
void Close() override;
bool Verify(std::string& strError) override;
std::string Filename() const override { return m_file_path.string(); }
sqlite3* Handle() const { return m_db; }
private:
std::filesystem::path m_file_path;
sqlite3* m_db = nullptr;
bool ExecOrError(const char* sql, std::string& strError) const;
};
// Magic written into PRAGMA application_id so we can recognize our wallet files
// and refuse to open foreign SQLite databases. ASCII "TRIw".
static constexpr int SQLITE_WALLET_APP_ID = 0x54526977;
// Schema version in PRAGMA user_version.
static constexpr int SQLITE_WALLET_SCHEMA_VERSION = 1;
#endif // TRIANGLES_WALLETDB_SQLITE_H
src/walletdb.cpp
+20 -6
View File
@@ -6,7 +6,6 @@
#include "walletdb.h"
#include "wallet.h"
#include <filesystem>
#include <boost/version.hpp>
using namespace std;
namespace fs = std::filesystem;
@@ -429,6 +428,24 @@ ReadKeyValue(CWallet* pwallet, CDataStream& ssKey, CDataStream& ssValue,
pwallet->mapKeyMetadata[keyid] = CKeyMetadata(keypool.nTime);
}
else if (strType == "hdmnemonic")
{
std::string m;
ssValue >> m;
pwallet->LoadHDMnemonic(m);
}
else if (strType == "hdcmnemonic")
{
std::pair<uint256, std::vector<unsigned char> > cm;
ssValue >> cm;
pwallet->LoadCryptedHDMnemonic(cm.first, cm.second);
}
else if (strType == "hdchain")
{
int64_t n;
ssValue >> n;
pwallet->nHDChainIndex = n;
}
else if (strType == "version")
{
ssValue >> wss.nFileVersion;
@@ -461,7 +478,8 @@ ReadKeyValue(CWallet* pwallet, CDataStream& ssKey, CDataStream& ssValue,
static bool IsKeyType(string strType)
{
return (strType== "key" || strType == "wkey" ||
strType == "mkey" || strType == "ckey");
strType == "mkey" || strType == "ckey" ||
strType == "hdmnemonic" || strType == "hdcmnemonic");
}
DBErrors CWalletDB::LoadWallet(CWallet* pwallet)
@@ -652,11 +670,7 @@ bool BackupWallet(const CWallet& wallet, const string& strDest)
pathDest /= wallet.strWalletFile;
try {
#if BOOST_VERSION >= 104000
fs::copy_file(pathSrc, pathDest, fs::copy_options::overwrite_existing);
#else
fs::copy_file(pathSrc, pathDest);
#endif
printf("copied wallet.dat to %s\n", pathDest.string().c_str());
return true;
} catch(const fs::filesystem_error &e) {
src/walletmigrate.cpp
+207
View File
@@ -0,0 +1,207 @@
// Copyright (c) 2026 The Triangles developers.
// Distributed under the MIT/X11 software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include "walletmigrate.h"
#include "walletdb-sqlite.h"
#include "util.h"
#include <cstring>
#include <fstream>
#include <iostream>
#include <vector>
#include <db_cxx.h>
namespace fs = std::filesystem;
bool IsSQLiteFile(const fs::path& path)
{
std::error_code ec;
if (!fs::exists(path, ec) || fs::file_size(path, ec) < 16)
return false;
std::ifstream in(path, std::ios::binary);
char hdr[16] = {};
in.read(hdr, sizeof(hdr));
if (!in)
return false;
// SQLite database files always start with this exact 16-byte string,
// including the trailing NUL. Berkeley DB files do not.
static const char kMagic[16] = {'S','Q','L','i','t','e',' ','f','o','r','m','a','t',' ','3','\0'};
return std::memcmp(hdr, kMagic, 16) == 0;
}
namespace {
// Count rows currently in the SQLite "main" table.
bool SQLiteRowCount(SQLiteDatabase& db, int64_t& nOut, std::string& strError)
{
sqlite3_stmt* st = nullptr;
if (sqlite3_prepare_v2(db.Handle(), "SELECT COUNT(*) FROM main;", -1, &st, nullptr) != SQLITE_OK) {
strError = strprintf("count prepare failed: %s", sqlite3_errmsg(db.Handle()));
return false;
}
bool ok = false;
if (sqlite3_step(st) == SQLITE_ROW) {
nOut = sqlite3_column_int64(st, 0);
ok = true;
} else {
strError = "count query returned no rows";
}
sqlite3_finalize(st);
return ok;
}
} // namespace
bool MaybeMigrateBerkeleyWalletToSQLite(const fs::path& walletPath, std::string& strError)
{
strError.clear();
std::error_code ec;
if (!fs::exists(walletPath, ec))
return true; // fresh install — the SQLite backend will create it
if (IsSQLiteFile(walletPath))
return true; // already migrated / already SQLite
const fs::path dir = walletPath.parent_path();
const std::string file = walletPath.filename().string();
const fs::path tmpPath = dir / (file + ".sqlite.tmp");
const fs::path bakPath = dir / (file + ".bdb.bak");
printf("Wallet migration: converting Berkeley %s to SQLite...\n", walletPath.string().c_str());
fs::remove(tmpPath, ec); // clear any stale temp from a prior aborted run
int64_t nCopied = 0;
// ── Read side: a private, read-only Berkeley environment over the wallet
// directory, then the "main" sub-database (matches CDB::CDB's open call). ──
DbEnv env(0u);
env.set_error_stream(&std::cerr);
env.set_cachesize(0, 1 << 20, 1); // 1 MiB cache is plenty for sequential read
u_int32_t envFlags = DB_CREATE | DB_INIT_MPOOL | DB_PRIVATE;
if (env.open(dir.string().c_str(), envFlags, 0) != 0) {
strError = "migration: cannot open Berkeley environment on wallet directory";
return false;
}
bool ok = false;
{
Db db(&env, 0);
if (db.open(nullptr, file.c_str(), "main", DB_BTREE, DB_RDONLY, 0) != 0) {
strError = "migration: cannot open Berkeley wallet (is it a valid wallet.dat?)";
env.close(0);
return false;
}
// ── Write side: fresh SQLite database in the temp file. ──
SQLiteDatabase sqlite(tmpPath);
std::string sqlErr;
if (!sqlite.Open(sqlErr)) {
strError = "migration: cannot create SQLite wallet: " + sqlErr;
db.close(0);
env.close(0);
return false;
}
auto batch = sqlite.MakeBatch();
if (!batch || !batch->TxnBegin()) {
strError = "migration: cannot begin SQLite transaction";
db.close(0);
env.close(0);
return false;
}
Dbc* pcursor = nullptr;
if (db.cursor(nullptr, &pcursor, 0) != 0) {
strError = "migration: cannot open Berkeley cursor";
batch->TxnAbort();
db.close(0);
env.close(0);
return false;
}
Dbt datKey, datValue; // BDB-owned buffers, valid until the next get()
int ret;
bool writeFailed = false;
while ((ret = pcursor->get(&datKey, &datValue, DB_NEXT)) == 0) {
const unsigned char* kp = static_cast<const unsigned char*>(datKey.get_data());
const unsigned char* vp = static_cast<const unsigned char*>(datValue.get_data());
KeyBytes key(kp, kp + datKey.get_size());
ValueBytes val(vp, vp + datValue.get_size());
if (!batch->WriteKey(key, val, /*fOverwrite=*/true)) {
writeFailed = true;
break;
}
++nCopied;
}
pcursor->close();
if (writeFailed || (ret != DB_NOTFOUND && ret != 0)) {
strError = strprintf("migration: copy aborted after %lld records (bdb get=%d)",
(long long)nCopied, ret);
batch->TxnAbort();
db.close(0);
env.close(0);
return false;
}
if (!batch->TxnCommit()) {
strError = "migration: SQLite commit failed";
db.close(0);
env.close(0);
return false;
}
// ── Verify the destination row count matches what we copied. ──
int64_t nDst = -1;
if (!SQLiteRowCount(sqlite, nDst, strError)) {
db.close(0);
env.close(0);
return false;
}
if (nDst != nCopied) {
strError = strprintf("migration: record count mismatch (copied=%lld sqlite=%lld)",
(long long)nCopied, (long long)nDst);
db.close(0);
env.close(0);
return false;
}
batch.reset();
sqlite.Close();
db.close(0);
ok = true;
}
env.close(0);
if (!ok) {
fs::remove(tmpPath, ec);
return false;
}
// ── Atomic-ish swap: back up the Berkeley original, then move SQLite in. ──
fs::rename(walletPath, bakPath, ec);
if (ec) {
strError = strprintf("migration: cannot back up Berkeley wallet to %s: %s",
bakPath.string().c_str(), ec.message().c_str());
fs::remove(tmpPath, ec);
return false;
}
fs::rename(tmpPath, walletPath, ec);
if (ec) {
// Roll the original back into place so the wallet is never left missing.
std::error_code ec2;
fs::rename(bakPath, walletPath, ec2);
strError = strprintf("migration: cannot move SQLite wallet into place: %s",
ec.message().c_str());
fs::remove(tmpPath, ec2);
return false;
}
printf("Wallet migration: complete. %lld records migrated to SQLite. "
"Berkeley original preserved at %s\n",
(long long)nCopied, bakPath.string().c_str());
return true;
}
src/walletmigrate.h
+31
View File
@@ -0,0 +1,31 @@
// Copyright (c) 2026 The Triangles developers.
// Distributed under the MIT/X11 software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#ifndef TRIANGLES_WALLETMIGRATE_H
#define TRIANGLES_WALLETMIGRATE_H
#include <filesystem>
#include <string>
// Migrate a Berkeley DB wallet (wallet.dat) to a SQLite wallet of the same
// name, IN PLACE and NON-DESTRUCTIVELY:
//
// 1. If walletPath does not exist, or is already a SQLite database, there is
// nothing to do — returns true.
// 2. Otherwise the Berkeley records are copied verbatim (raw key/value bytes)
// into a fresh SQLite database written to a temporary file.
// 3. The record count is verified to match.
// 4. The original Berkeley file is renamed to "<name>.bdb.bak" (kept as a
// fallback, never deleted), and the SQLite file is moved into place as
// "<name>".
//
// On any failure the original Berkeley wallet is left exactly as it was and the
// temporary SQLite file is removed; strError describes the problem.
bool MaybeMigrateBerkeleyWalletToSQLite(const std::filesystem::path& walletPath,
std::string& strError);
// True if the file begins with the SQLite format-3 magic header.
bool IsSQLiteFile(const std::filesystem::path& path);
#endif // TRIANGLES_WALLETMIGRATE_H