Add blockchain bootstrap for fast Pi sync

New features: - Enhanced bootstrap.sh with blockchain download option - create-blockchain-snapshot.sh for creating compressed snapshots - BOOTSTRAP.md documentation Benefits for Raspberry Pi users: - Download pre-synced blockchain (~500MB compressed) - Skip days/weeks of initial sync - Start mining immediately from latest block - Interactive installer asks: bootstrap or sync from scratch Snapshot creation: - Safely stops daemon - Excludes wallet.dat (security) - Compresses to ~70% original size - Includes checksums and metadata TODO: Create first snapshot from DNS2 and host on GitHub releases
2026-04-01 12:04:08 -07:00
parent 247db74393
commit bb6e53f3f6
3 changed files with 304 additions and 27 deletions
@@ -0,0 +1,121 @@
+## TRI-PI Blockchain Bootstrap
+
+Fast-sync your Raspberry Pi node with pre-synced blockchain data.
+
+### Why Use Bootstrap?
+
+**Without bootstrap:**
+- Sync from genesis block (block 0)
+- Takes days/weeks on Raspberry Pi
+- Heavy CPU/disk usage during sync
+- Network bandwidth intensive
+
+**With bootstrap:**
+- Start from latest block height
+- Ready to mine in minutes
+- Minimal resource usage
+- Only ~500MB download
+
+### Creating a Blockchain Snapshot
+
+Run on a fully-synced node (DNS2, DNS3, etc.):
+
+```bash
+# Download snapshot creator
+curl -L -o create-snapshot.sh \
+  https://raw.githubusercontent.com/SamiAhmed7777/tri-pi/main/scripts/create-blockchain-snapshot.sh
+
+chmod +x create-snapshot.sh
+
+# Create snapshot (stops daemon temporarily)
+./create-snapshot.sh
+```
+
+**Output:**
+- `tri-blockchain-v5.4.4-YYYY-MM-DD.tar.gz` - Compressed blockchain
+- `snapshot-info.txt` - Metadata (block height, checksums)
+
+### Hosting the Snapshot
+
+**Option 1: GitHub Release** (recommended for public distribution)
+```bash
+gh release upload v5.4.4 tri-blockchain-*.tar.gz
+```
+
+**Option 2: Dropbox**
+```bash
+dbxcli put tri-blockchain-*.tar.gz /Krystie/TRI/
+dbxcli share /Krystie/TRI/tri-blockchain-*.tar.gz
+```
+
+**Option 3: Custom Web Server**
+```bash
+scp tri-blockchain-*.tar.gz user@server:/var/www/html/tri/
+# Access at: https://example.com/tri/tri-blockchain-v5.4.4-2026-04-01.tar.gz
+```
+
+### Updating Bootstrap Script
+
+After uploading, update `bootstrap.sh`:
+
+```bash
+# Edit line 9
+BLOCKCHAIN_URL="https://your-url-here/tri-blockchain-latest.tar.gz"
+```
+
+Commit and push to make it available to all Pi users!
+
+### Snapshot Contents
+
+**Included:**
+- `blk*.dat` - Blockchain data files
+- `database/` - Block index
+- `txleveldb/` - Transaction index
+- `smsgDB/` - Secure messaging database
+- `peers.dat` - Known peer cache
+- `smsg.ini` - Messaging config
+
+**Excluded (for security/privacy):**
+- `wallet.dat` - NEVER include! Contains private keys
+- `debug.log` - Large, regenerated on run
+- `onion/` - Regenerated automatically
+- `tor_data/` - Regenerated automatically
+
+### Maintenance Schedule
+
+**Recommended:** Create new snapshots weekly/monthly as blockchain grows.
+
+Automation example (cron):
+```bash
+# Every Sunday at 2 AM
+0 2 * * 0 /usr/local/bin/create-snapshot.sh && \
+          gh release upload v5.4.4 /tmp/tmp.*/tri-blockchain-*.tar.gz --clobber
+```
+
+### Compression Stats
+
+Typical compression with tar.gz:
+- Original blockchain: ~2-5GB (depends on block height)
+- Compressed: ~500MB-1.5GB
+- Compression ratio: ~70-75%
+
+### Security Notes
+
+1. **Never include wallet.dat** - Contains private keys!
+2. **Verify checksums** - Always provide SHA256 hashes
+3. **Trust** - Only use snapshots from trusted sources
+4. **Update regularly** - Stale snapshots still require catching up
+
+### Testing
+
+Before releasing publicly:
+
+```bash
+# On a fresh Pi
+rm -rf ~/.triangles
+curl -sSL https://raw.githubusercontent.com/SamiAhmed7777/tri-pi/main/bootstrap.sh | bash
+# Choose option 1 (bootstrap download)
+
+# Verify it starts from correct block
+trianglesd getinfo | grep blocks
+```
@@ -1,11 +1,15 @@
 #!/bin/bash
 # TRI-PI Bootstrap Installer
-# Ultra-lightweight initial download, then fetches optimized binary
+# Ultra-lightweight initial download with optional blockchain bootstrap

 set -e

+VERSION="v5.4.4"
+BINARY_URL="https://github.com/SamiAhmed7777/tri-pi/raw/main/releases/trianglesd-upx"
+BLOCKCHAIN_URL="https://example.com/tri-blockchain-latest.tar.gz"  # TODO: Update with actual URL
+
 echo "╔═══════════════════════════════════════╗"
-echo "║   TRI-PI v5.4.4 Bootstrap Installer  ║"
+echo "║   TRI-PI $VERSION Bootstrap Installer  ║"
 echo "╚═══════════════════════════════════════╝"
 echo ""

@@ -23,45 +27,79 @@ echo "✓ ARM64 architecture detected"
 echo ""
 echo "📦 Installing dependencies..."
 sudo apt-get update -qq
-sudo apt-get install -y tor curl upx-ucl > /dev/null 2>&1
+sudo apt-get install -y tor curl bc > /dev/null 2>&1

-echo "✓ Dependencies installed (tor, curl, upx-ucl)"
+echo "✓ Dependencies installed"

-# Download UPX-compressed binary (1.6MB instead of 5.1MB!)
+# Download binary
 echo ""
-echo "⬇️  Downloading optimized binary (1.6MB, UPX-compressed)..."
+echo "⬇️  Downloading trianglesd binary (1.6MB, UPX-compressed)..."

 TMP_DIR=$(mktemp -d)
 cd "$TMP_DIR"

-# GitHub release URL (we'll create this as a separate release asset)
-BINARY_URL="https://github.com/SamiAhmed7777/tri-pi/releases/download/v5.4.4/trianglesd-upx-arm64"
-
 if ! curl -L -o trianglesd "$BINARY_URL" 2>/dev/null; then
    echo "❌ Download failed. Falling back to full package..."
-    curl -L -o tri-pi.tar.gz "https://github.com/SamiAhmed7777/tri-pi/releases/download/v5.4.4/tri-pi-v5.4.4-arm64.tar.gz"
+    PACKAGE_URL="https://github.com/SamiAhmed7777/tri-pi/releases/download/$VERSION/tri-pi-$VERSION-arm64.tar.gz"
+    curl -L -o tri-pi.tar.gz "$PACKAGE_URL"
    tar xzf tri-pi.tar.gz
-    cd tri-pi-v5.4.4-arm64
+    cd tri-pi-$VERSION-arm64
    sudo cp bin/trianglesd /usr/local/bin/
 else
-    echo "✓ Binary downloaded (UPX-compressed)"
+    echo "✓ Binary downloaded"
    chmod +x trianglesd
    sudo cp trianglesd /usr/local/bin/
 fi

-# Verify
-echo ""
-echo "✅ Installation complete!"
+# Verify installation
 echo ""
+echo "✅ Binary installed!"
 trianglesd --version

 # Create config directory
 mkdir -p ~/.triangles

+# Blockchain bootstrap option
+echo ""
+echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
+echo "  Blockchain Sync Options"
+echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
+echo ""
+echo "Choose sync method:"
+echo "  [1] Download bootstrap blockchain (FAST - recommended for Pi)"
+echo "      ~500MB download, start at current block height"
+echo ""
+echo "  [2] Sync from scratch (SLOW - may take days on Pi)"
+echo "      Start from genesis block"
+echo ""
+read -p "Your choice [1/2]: " SYNC_CHOICE
+
+if [[ "$SYNC_CHOICE" == "1" ]]; then
+    echo ""
+    echo "⬇️  Downloading blockchain bootstrap..."
+    echo "   This will save hours/days of initial sync!"
+    echo ""
+    
+    if curl -L -o blockchain.tar.gz "$BLOCKCHAIN_URL" 2>/dev/null; then
+        echo "✓ Blockchain downloaded"
+        echo "📦 Extracting to ~/.triangles/ ..."
+        
+        tar xzf blockchain.tar.gz -C ~/
+        
+        BLOCK_HEIGHT=$(grep -oP 'Block Height: \K[0-9]+' ~/snapshot-info.txt 2>/dev/null || echo "latest")
+        echo "✓ Blockchain extracted (starting from block $BLOCK_HEIGHT)"
+        rm -f blockchain.tar.gz ~/snapshot-info.txt
+    else
+        echo "⚠️  Bootstrap download failed, will sync from scratch"
+    fi
+else
+    echo "✓ Will sync from genesis block"
+fi
+
 # Generate random RPC password
 RPC_PASS=$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 32)

-# Check if config exists
+# Create config if needed
 if [[ ! -f ~/.triangles/triangles.conf ]]; then
    echo ""
    echo "📝 Creating configuration..."
@@ -81,10 +119,11 @@ rpcport=8332
 port=8333
 maxconnections=50

-# Tor (managed automatically by trianglesd)
-# No manual Tor configuration needed!
+# Performance tuning for Raspberry Pi
+dbcache=100
+maxmempool=50
 CONFIG
-    echo "✓ Configuration created at ~/.triangles/triangles.conf"
+    echo "✓ Configuration created"
 fi

 # Enable Tor service
@@ -98,16 +137,19 @@ rm -rf "$TMP_DIR"

 echo ""
 echo "╔═══════════════════════════════════════╗"
-echo "║          Installation Ready!          ║"
+echo "║          Installation Complete!       ║"
 echo "╚═══════════════════════════════════════╝"
 echo ""
-echo "Start your node:"
-echo "  trianglesd"
+echo "🚀 Start your node:"
+echo "   trianglesd"
 echo ""
-echo "Check status:"
-echo "  trianglesd getinfo"
+echo "📊 Check status:"
+echo "   trianglesd getinfo"
 echo ""
-echo "View your onion address (after first run):"
-echo "  cat ~/.triangles/onion_private_key"
+echo "🔍 View your onion address (after first run):"
+echo "   cat ~/.triangles/onion/private_key"
 echo ""
-echo "Happy mining! 🚀"
+echo "💡 Tip: First sync may take a while. Check progress with:"
+echo "   watch -n5 'trianglesd getinfo | grep blocks'"
+echo ""
+echo "Happy mining! 🎯"
@@ -0,0 +1,114 @@
+#!/bin/bash
+# TRI-PI Blockchain Bootstrap Snapshot Creator
+# Creates compressed blockchain archive for fast Pi sync
+
+set -e
+
+VERSION="v5.4.4"
+SNAPSHOT_DATE=$(date +%Y-%m-%d)
+SNAPSHOT_NAME="tri-blockchain-${VERSION}-${SNAPSHOT_DATE}.tar.gz"
+TRI_DATA_DIR="$HOME/.triangles"
+
+echo "╔═══════════════════════════════════════╗"
+echo "║  TRI-PI Blockchain Snapshot Creator  ║"
+echo "╚═══════════════════════════════════════╝"
+echo ""
+
+# Check if daemon is running
+if pgrep -x trianglesd > /dev/null; then
+    echo "⚠️  trianglesd is running. Stopping for clean snapshot..."
+    trianglesd stop
+    sleep 5
+fi
+
+# Check blockchain exists
+if [[ ! -d "$TRI_DATA_DIR" ]]; then
+    echo "❌ Error: Triangles data directory not found at $TRI_DATA_DIR"
+    exit 1
+fi
+
+# Get block count from last run
+BLOCK_COUNT=$(grep -oP 'height=\K[0-9]+' "$TRI_DATA_DIR/debug.log" 2>/dev/null | tail -1 || echo "unknown")
+
+echo "📊 Blockchain Stats:"
+echo "   Location: $TRI_DATA_DIR"
+echo "   Block height: $BLOCK_COUNT"
+echo "   Size: $(du -sh $TRI_DATA_DIR | cut -f1)"
+echo ""
+
+# Create temp directory
+TEMP_DIR=$(mktemp -d)
+SNAPSHOT_DIR="$TEMP_DIR/triangles-data"
+mkdir -p "$SNAPSHOT_DIR"
+
+echo "📦 Creating snapshot..."
+
+# Copy essential blockchain files (exclude wallet and logs)
+echo "   • Copying blockchain data..."
+cp -r "$TRI_DATA_DIR/blk"* "$SNAPSHOT_DIR/" 2>/dev/null || true
+cp -r "$TRI_DATA_DIR/database" "$SNAPSHOT_DIR/" 2>/dev/null || true
+cp -r "$TRI_DATA_DIR/txleveldb" "$SNAPSHOT_DIR/" 2>/dev/null || true
+cp -r "$TRI_DATA_DIR/smsgDB" "$SNAPSHOT_DIR/" 2>/dev/null || true
+cp "$TRI_DATA_DIR/peers.dat" "$SNAPSHOT_DIR/" 2>/dev/null || true
+cp "$TRI_DATA_DIR/smsg.ini" "$SNAPSHOT_DIR/" 2>/dev/null || true
+
+# DO NOT include:
+# - wallet.dat (user-specific, contains private keys!)
+# - debug.log (grows large, not needed)
+# - db.log (not needed)
+# - onion/ (regenerated on first run)
+# - tor_data/ (regenerated)
+
+echo "   • Compressing (this may take a few minutes)..."
+cd "$TEMP_DIR"
+tar czf "$SNAPSHOT_NAME" triangles-data/
+
+# Calculate size reduction
+ORIGINAL_SIZE=$(du -sb "$TRI_DATA_DIR" | cut -f1)
+COMPRESSED_SIZE=$(stat -c%s "$SNAPSHOT_NAME")
+REDUCTION=$(echo "scale=1; (1 - $COMPRESSED_SIZE / $ORIGINAL_SIZE) * 100" | bc)
+
+echo ""
+echo "✅ Snapshot created successfully!"
+echo ""
+echo "📊 Compression Stats:"
+echo "   Original size: $(numfmt --to=iec-i --suffix=B $ORIGINAL_SIZE)"
+echo "   Compressed size: $(numfmt --to=iec-i --suffix=B $COMPRESSED_SIZE)"
+echo "   Reduction: ${REDUCTION}%"
+echo ""
+echo "📁 Snapshot file:"
+echo "   $TEMP_DIR/$SNAPSHOT_NAME"
+echo ""
+echo "📝 Metadata:"
+cat > "$TEMP_DIR/snapshot-info.txt" << INFO
+TRI-PI Blockchain Bootstrap Snapshot
+=====================================
+Version: $VERSION
+Date: $SNAPSHOT_DATE
+Block Height: $BLOCK_COUNT
+Original Size: $(numfmt --to=iec-i --suffix=B $ORIGINAL_SIZE)
+Compressed Size: $(numfmt --to=iec-i --suffix=B $COMPRESSED_SIZE)
+Compression: ${REDUCTION}%
+
+SHA256: $(sha256sum "$SNAPSHOT_NAME" | cut -d' ' -f1)
+
+Installation:
+1. Extract to ~/.triangles/
+2. Start trianglesd
+3. Blockchain resumes from block $BLOCK_COUNT
+
+Contents:
+- Blockchain files (blk*.dat)
+- Database
+- Transaction index
+- Peer cache
+INFO
+
+cat "$TEMP_DIR/snapshot-info.txt"
+echo ""
+echo "Next steps:"
+echo "1. Upload to hosting: scp $TEMP_DIR/$SNAPSHOT_NAME user@server:/path/"
+echo "2. Update bootstrap.sh with download URL"
+echo "3. Test on fresh Pi"
+echo ""
+echo "Clean up when done: rm -rf $TEMP_DIR"