Permanent fix for the recurring 'Krystie's UMP writes silently drop'
problem. Kills the last manual-process row from the drift stack by
moving krystie's hermes-cli gateway under systemd.
Two new assets in deploy/:
- krystie-hermes-gateway.service
- README.md sections describing install + the manual-to-systemd
migration steps
Run from a non-gateway shell to swap:
sudo cp deploy/krystie-hermes-gateway.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now krystie-hermes-gateway.service
LONG-STANDING PROBLEM (now permanently fixed):
The ump-recall-mcp.js shim spawned a 'npx ump memory' child process
with UMP_DIR inherited from the parent gateway's env. Profile
configs (config.yaml) and parent processes had stale UMP_DIR values
from before the ump store migration, so the child wrote to an empty
directory. Result: krystie's ump.remember calls silently dropped.
NEW PATTERN:
1. ump-memory.service publishes its UMP_DIR/HTTP/STORE via
/run/ump-memory/ump-{dir,port,store}.txt on every (re)start.
This file is the canonical source of truth.
2. On every ump child spawn, the shim:
a. Reads /run/ump-memory/ump-{dir,port,store}.txt (THE source)
b. Strips UMP_* / SIDECAR_URL from parent's env (so stale values
from profile configs cannot leak through)
c. Spawns with the canonical values
d. Audits the spawned child's env via /proc/<pid>/environ and
logs FATAL if anything other than the canonical UMP_DIR was
inherited
This breaks the recurring silent-drop pattern regardless of:
- which profile is launching the gateway (krystie, default, future)
- whether env vars are stale or fresh
- whether the parent process is managed by systemd or launched by hand
- whether config.yaml is pinned to a stale path
The only way to get the wrong UMP_DIR after this patch is if
/run/ump-memory/ump-dir.txt itself is wrong — and that file is
regenerated by ump-memory.service on every (re)start, so the only way
to make it wrong is to corrupt it intentionally.
Companion: krystie-hermes-gateway.service deploy asset (next commit)
makes krystie's gateway systemd-managed so its shim lifetime matches
ump-memory.service's lifetime.
Until now the ump-recall sidecar was started ad-hoc as a background process
and died with the session. This commit ships the systemd unit file that
makes it persist across reboots and Hermes gateway restarts.
The unit:
- Listens on 127.0.0.1:4380 (the sidecar port the MCP shim uses)
- Depends on ump-memory.service (canonical ump store at :4317)
- Reads env from /etc (UMP_URL, QDRANT_URL, OLLAMA_URL, GRAPH_FILE)
- Restarts on crash (RestartSec=3)
- Logs to journal under SyslogIdentifier=ump-recall-sidecar
Verifies cleanly via: curl http://127.0.0.1:4380/health
Returns upstreams reachable + graph loaded.
Install (idempotent):
sudo cp deploy/ump-recall-sidecar.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now ump-recall-sidecar.service
After moving the canonical ump store from /root/.openclaw/.../state/ump-local
to /root/.hermes/state/ump-local, two related fixes:
1. src/ump-recall-mcp.js (the MCP shim):
- UMP_DIR fallback updated: /root/.openclaw/... -> /root/.hermes/state/ump-local
(parent env always provides UMP_DIR explicitly, but the fallback
was a footgun if anyone unset it.)
- tools/call ump.get now routes to the sidecar's GET /get/{urn}
endpoint via HTTP, with fallback to the stdio child for resilience.
This replaces the silent-drop pattern where the npx child held
a stale UMP_DIR and returned "not_found: no record" for everything.
2. scripts/ump_verify.py (new):
File-layer verifier for ump-write verification. Replaces the
manual SOP with an executable that has deterministic exit codes:
0 verified, 2 silent drop, 3 malformed, 4 no store, 5 corrupted
Reads memory.ump.json directly (independent of any HTTP route),
supports --exists, --wait N, --list-last N.
3. .gitignore: ignore state/ (runtime cache: access_log.json, graph.json)
and __pycache__ (eval python tools).
Hermes Agent
sami7777