skill-evolution/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

What this is

A host-agnostic agent skill (skill-evolution) that gives the host agent a self-improvement feedback loop: it reads the agent's past session transcripts through the active HostAdapter (default Hermes → ~/.hermes/state.db; Claude Code → ~/.claude/projects/*/*.jsonl), compares them against the agent's installed skills, generates structured markdown proposals (improve_existing, create_new, merge_skills, deprecate_skill) for a human to review, and — if opted in — auto-applies high-confidence proposals through the host's mutation channel (Hermes: skill_manage instructions; Claude Code: direct file writes), gated on an evaluation framework that scores proposal content before anything mutates.

There is no daemon and no server component. The "pipeline" is a set of standalone Python stdlib scripts that a scheduled job invokes and pipes together; the actual analysis step is an LLM call made by the host agent itself (not by any script here), following an analyzer prompt that the operator maintains outside this repo (see SKILL.md's "Scheduled Runs" section for what that prompt needs to do). The evaluation gate (scripts/evaluate.py), by contrast, is implemented in this repo — it's not delegated to the cron agent.

Git tracking

.gitignore used to use unanchored patterns that matched several directory names anywhere in the tree, silently excluding real pipeline code from version control while git status read clean. This was fixed by anchoring the patterns.

proposals/, reports/, and eval_history.jsonl are gitignored and not shipped by this repo — proposals/ in particular holds real analysis output derived from actual session transcripts, and free-text personal narrative a model can write into a proposal body isn't reliably caught by shape-based redaction, so these stay local-only rather than committed. This has no effect on runtime behavior: the pipeline still reads/writes these paths exactly as described below; only what git tracks is affected. scripts/*.py and tests/*.py remain fully tracked.

proposal.py's get_proposals_dir() defaults to ./proposals/ at the repo root (overridable via SKILL_EVOLUTION_PROPOSALS_DIR), and evaluate.py's history file defaults to ./eval_history.jsonl at the repo root (overridable via SKILL_EVOLUTION_HISTORY_PATH). The skill-quality cron wrapper writes timestamped markdown reports to ./reports/ (overridable via SKILL_EVOLUTION_QUALITY_REPORT_DIR).

Commands

# Run the pipeline manually, end to end (mirrors what the cron job does)
python3 scripts/fetch_sessions.py --dry-run              # preview sessions without marking them processed
python3 scripts/fetch_sessions.py | python3 scripts/analyze.py   # NDJSON sessions -> LLM-ready prompt text
python3 scripts/skill_index.py                            # scan ~/.hermes/skills/ -> JSON index

# Proposal inspection/scaffolding
python3 scripts/proposal.py --example     # print an example proposal
python3 scripts/proposal.py --list        # list current proposals
python3 scripts/proposal.py --show <id>   # show one proposal by id

# Evaluation gate
python3 scripts/evaluate.py --list-evaluators             # show the resolved evaluator set
python3 scripts/evaluate.py --retroactive --dry-run       # preview batch re-evaluation of saved proposals
python3 scripts/evaluate.py --retroactive                 # re-run evaluation against saved proposals, append history
python3 scripts/evaluate.py --prune                       # prune eval history per SKILL_EVOLUTION_HISTORY_RETENTION

# One-off evaluation of any of the four targets (advisory by default — only `skill` and
# `proposal` gate auto-apply; see SKILL_EVOLUTION_GATE_TARGETS below).
# `deterministic` is meaningless for the non-skill targets, so opt out of it.
SKILL_EVOLUTION_EVALUATORS=llm_judge,regression python3 scripts/evaluate.py --eval-target proposal        --proposal-id <id>
SKILL_EVOLUTION_EVALUATORS=llm_judge,regression python3 scripts/evaluate.py --eval-target tool_calls      --session-id <id>
SKILL_EVOLUTION_EVALUATORS=llm_judge,regression python3 scripts/evaluate.py --eval-target analyzer_prompt --session-id <id> --proposal-id <id>

# Inspect processed-session state (read-only; state.py is a library, not a CLI)
python3 -c "import sys; sys.path.insert(0,'scripts'); import state; print(len(state.load_processed()), 'sessions already processed')"
python3 scripts/fetch_sessions.py --prune-state           # prune state per SKILL_EVOLUTION_STATE_RETENTION (reports on stderr)

# Optional GEPA optimizer (needs `pip install -e ".[optimizer]"`)
SKILL_EVOLUTION_OPTIMIZER_ENABLED=true python3 scripts/optimize_skill.py --list-candidates                # read-only report of low-scoring targets (skill: targets only)
- **`scripts/skill_quality.py`** — periodic skill quality tracking (P2-3, shipped 2026-08-01). Evaluates all installed skills against the same rubric as the evaluation gate (correctness, procedure-following, conciseness), records each evaluation in `eval_history.jsonl` under `skill:<name>`, and generates a markdown or JSON report showing trends over time. Reuses `evaluate_skill_text()` with a synthetic proposal (field=body, same old/new value so size guards pass), `installed_skill_body()` for skill content, and the existing provider infrastructure. CLI: `--skill <name>` (evaluate one), `--output <file>` (write report), `--since Nd` (filter by recency), `--below 0.7` (filter by threshold), `--format json|markdown`. Cost: one LLM judge call per skill (~$0.01-0.05), so a full tree of 143 skills costs ~$1.50-7.00 per run — recommend weekly/monthly, not daily. Cron wrapper: `scripts/skill-quality-report.sh`.
SKILL_EVOLUTION_OPTIMIZER_ENABLED=true python3 scripts/optimize_skill.py --list-candidates --target all   # include proposal:/tool_calls:/analyzer_prompt: namespaces
SKILL_EVOLUTION_OPTIMIZER_ENABLED=true python3 scripts/optimize_skill.py --skill <name> [--iterations N]  # run GEPA for one skill

# Tests
pip install -e ".[dev]"
pytest

Interpreter note. The core pipeline is stdlib-only, so any python3 runs it. The optimizer is not: gepa must be importable, so optimize_skill.py --skill needs the interpreter you installed the [optimizer] extra into. A project-local .venv on pyenv 3.12.2 is the setup these commands were verified against — use .venv/bin/python scripts/optimize_skill.py ... for optimizer commands, or the bare python3 shown above for everything else. Running the optimizer under an interpreter without gepa fails fast with _require_gepa()'s actionable RuntimeError rather than misbehaving. pytest passes under either interpreter — the gepa-contract module skips when the extra is absent; see "Tests exist now" below for the currently-verified pass/skip counts rather than a number duplicated here (a stale duplicate is exactly how this line and that one drifted out of sync with each other before).

No build step and no non-stdlib runtime dependencies for the core pipeline. gepa (the standalone PyPI package, not dspy) is an optional extra (pip install -e ".[optimizer]") required only by scripts/optimize_skill.py; nothing else imports it.

Architecture

fetch_sessions.py ──NDJSON──► analyze.py ──formatted prompt──┐
skill_index.py ──JSON index──────────────────────────────────┼──► [host agent + analyzer prompt]
                                                               │
                                                               ▼
                                                proposal.py (write proposal .md files, status: proposed)
                                                               │
                                                     human review (status: approved)
                                                               │
                                                  apply_proposal() ──► evaluate.evaluate_and_record()
                                                               │            (deterministic + llm_judge + regression,
                                                               │             appends to eval_history.jsonl)
                                                               │
                                                        gate passed? ──No──► can_apply: False, proposal stays proposed
                                                               │
                                                              Yes
                                                               │
                                                               ▼
                                                     skill_manage (Hermes tool)

optimize_skill.py (optional, SKILL_EVOLUTION_OPTIMIZER_ENABLED=true)
  --list-candidates: reads eval_history.jsonl, prints low-scoring targets (read-only, no proposal written)
  --skill <name>: fetch_sessions.sessions_for_skill() ──► gepa.optimize_anything() ──► drafts improve_existing
                  proposal from the winning candidate via proposal.save_proposal()
  (never calls apply_proposal() itself — optimizer-originated proposals go through the same gate as any other)

• scripts/fetch_sessions.py — reads ~/.hermes/state.db (SQLite; sessions/messages tables), filters out already-processed sessions (via state.py's state file) and anything older than --lookback-hours, redacts messages via contains_secret() — which layers the fixed SECRET_PATTERNS substrings (known vendor prefixes, specific env-var names) with shape-based SECRET_REGEXES. The regex layer exists because the substring list provably leaked: scanning the real 26,794 message bodies found 27 password= assignments, 23 generic *_TOKEN=/*_SECRET= assignments, 6 generic sk- keys, 5 Authorization: Bearer headers and a credentialed DB URI that it missed, all of which would have been forwarded to a provider verbatim. Length floors (20+ chars for opaque keys, 8+ for assigned values) keep MAX_TOKENS=1024 and DEBUG=true off the list; measured over-redaction is 0.229% of lines. This predicate is the single chokepoint for both redaction points (provider prompts and on-disk proposals), so err toward over-detection — applies redact_pii() (see below), truncates long messages, and emits one JSON object per session to stdout. Marks sessions processed as a side effect unless --dry-run. Its queries select sessions.started_at/messages.timestamp (matching the live schema); the per-session output dict's timestamp key is started_at (there is no created_at key, and total_tokens is not emitted — analyze.py defaults it). sessions_for_skill() (used by the optimizer) is a separate query on the same DB that filters sessions to those touching a given skill, capped at max_sessions most-recent-first. The two functions handle an unreadable DB differently, on purpose: sessions_for_skill() checks existence and the sessions table via _session_db_is_readable() before connecting and returns [] with a stderr warning, because the optimizer is interactive and "no history for this skill" is a normal outcome there; fetch_sessions() still raises, because it is the unattended cron path where an unreadable DB is a failure the operator needs surfaced rather than a run quietly reporting zero sessions every night. Checking before sqlite3.connect() also matters because connect() creates a file for a missing path — a typo'd --db-path previously crashed with a confusing "no such table: sessions" and left a stray 0-byte DB behind.
• Secrets and PII are handled differently, on purpose. A detected secret drops the whole message (_summarize_messages skips it) — there is no version of a message worth sending once it carries a credential. Detected PII is masked in place by redact_pii(), so an email inside a paragraph of useful debugging evidence costs that email, not the paragraph, and the mask names the kind ([PII:EMAIL]) so a reviewer knows what was removed. It is wired at both redaction points: fetch_sessions._summarize_messages() (what reaches the analyzer) and evaluate.redact_secrets() (provider prompts and on-disk proposals). Monetary amounts are deliberately NOT masked. The scope is identifiers, not amounts: an email, phone, Luhn-valid card, IBAN or RUC/DNI identifies a person, while a bare amount identifies nobody and is exactly the evidence the analyzer needs to reason about finance/budgeting-style skills. Measured over a real message corpus, on what actually crosses the boundary: identifier patterns (email/phone/card/RUC) were fully masked while money amounts passed through untouched, confirming the redaction targets identifiers specifically rather than over-redacting numeric content generally. Card detection requires a leading digit of 3–6 and a Luhn check, which is what keeps hash-like 16-digit runs (commit 1234567890123456) out.
• scripts/skill-evolution-fetch.sh — the bash wrapper a cron job invokes, so the cron entry stays one stable command. It resolves fetch_sessions.py relative to its own location, following symlinks, and passes arguments through. Two earlier strategies were broken and are worth not reinventing: probing a hardcoded skills-home path with a fallback that doesn't exist in this repo's layout (always failed), and if git rev-parse --show-toplevel; then to find the project root (that command's stdout isn't captured, so the repo path was printed to stdout and prepended a stray non-JSON line to the NDJSON stream analyze.py consumes). Nothing but fetch_sessions.py may write to stdout. Symlink-following matters whenever the deployed entry point is itself a symlink into this repo, which is a common way to install it.
• scripts/state.py — small JSON-file-backed tracker (~/.hermes/skill_evolution_state.json) for which session IDs have already been analyzed. fetch_sessions.py duplicates this logic inline rather than importing it — if you change one, check the other (tests/test_state_schema_compat.py parametrizes over both modules to catch drift). Two on-disk shapes exist and both must be read. This repo and the deployed SKILL.md document {"processed_sessions": [...], "last_analyzed_at": ..., "version": 1}, but the file actually deployed is a flat {session_id: iso_timestamp} map (101 entries). Reading only the documented shape returned [] every run, which silently disabled dedup entirely — --dry-run --lookback-hours 168 reported 20 unprocessed sessions that had all in fact been processed. mark_processed() now preserves whichever layout the file already has (writing the documented shape only for a fresh file) because rewriting the flat map would destroy timestamps another producer maintains. Don't "normalize" that file without confirming who else writes it. Redirecting state: SKILL_EVOLUTION_STATE_FILE overrides the path, resolved at call time by get_state_file() — present in both modules, since they duplicate deliberately. Its fallback is the module global rather than the literal path, so monkeypatching fetch_sessions.STATE_FILE (what tests/test_state_schema_compat.py does) still works; a blank value is ignored rather than redirecting to "". Note that once dedup works, fetch_sessions.py --dry-run legitimately returns 0 sessions because every real session is already marked processed, which looks like a failure but is correct. Host-prefixed since the host-agnostic read side (U2): load_processed()/mark_processed()/prune_processed() all take a host argument (defaulting to host.resolve_host(), duplicated locally in both modules rather than imported — host.py imports fetch_sessions.py, so importing back would cycle). New entries on the flat-map shape are written "<host>:<session_id>"; a legacy un-prefixed key is read as belonging to "hermes" only, which is what lets the 101 real pre-host entries keep working with zero migration. prune_processed() only ever touches the resolved host's own entries — a different host's rows, prefixed or legacy, pass through untouched, and keep_ids is matched against each entry's bare id rather than its on-disk key for the same reason. This is not yet expressed as a HostAdapter method (scripts/host.py doesn't have a fourth "processed state" method) — see that file's section above for the three methods it does have. Pruning: prune_processed(retention=None, keep_ids=None), also duplicated in both modules (tests/test_state_pruning.py parametrizes over both, same drift-protection convention), gated on SKILL_EVOLUTION_STATE_RETENTION (same bare-int/"Nd"/"Nmo" syntax as evaluate.py's retention parsing, duplicated locally rather than imported — state.py/fetch_sessions.py sit below evaluate.py in this repo's import direction and must not invert it). Real pruning applies only to the flat {id: timestamp} shape — the documented shape carries no per-session timestamp, and its list order isn't chronological either (fetch_sessions() builds it from a Python set union, whose iteration order is hash-based, not insertion-based), so there is no temporal signal to prune by there without a schema change, which this deliberately does not attempt. Configuring retention against a documented-shape file is a no-op with a stderr warning on every call, not a silent gap. keep_ids is structurally required, not a defensive nicety. mark_processed() computes now() once per call and stamps every session in that batch with the identical value, so "keep the N most recent by timestamp" has no defined tiebreak among everything written in the same run — without a floor, a small retention count could drop most of the very batch a run just wrote, making those sessions look unprocessed again on the very next run. fetch_sessions() (the only real orchestrator of prune_processed() for the Hermes path) computes just_processed = {s["session_id"] for s in results} (distinct from new_processed, the full accumulated union) and calls prune_processed(keep_ids=just_processed) immediately after mark_processed(new_processed), structurally after the --dry-run early return so a dry run never prunes either. keep_ids closes the same-run hole, not the cross-run one. If SKILL_EVOLUTION_STATE_RETENTION is configured tighter than the effective --lookback-hours, a session's state entry can be pruned while it's still inside a later run's query window — it no longer looks processed, gets re-fetched, re-marked, and re-analyzed, indefinitely, every time this recurs. There's no clean structural fix without coupling retention to lookback, so the mitigation is a warning, not a fix: _warn_if_state_retention_below_lookback() fires on stderr every run where age-based retention converts to fewer hours than --lookback-hours — count-based retention has no clean unit match to compare, and has its own related limitation (keep_ids overriding the limit means a busy day's batch can floor the file above the configured count, so age-based retention is the better fit for this file specifically). Manual escape hatch: fetch_sessions.py --prune-state, structurally like evaluate.py --prune (before/after count, early return) but reporting to stderr, not stdout — unlike evaluate.py, this script's stdout is the NDJSON channel other tooling parses.
• scripts/skill_index.py — scans ~/.hermes/skills/<category>/<skill>/SKILL.md, hand-parses the YAML frontmatter (no PyYAML dependency — stdlib only) via parse_name_description_frontmatter() for name/description, and emits a JSON index the analyzer prompt uses as "what skills currently exist." evaluate.py's DeterministicEvaluator reuses this same parser to validate a proposal's frontmatter. scan_skills() skips dot-prefixed category directories: a live tree can hold .archive/ (retired skills), .curator_backups/ (timestamped snapshots) and .hub/ (lockfiles, audit log, quarantine). Without that filter the index reported retired skills as currently existing, and an archived twin of a live skill (the same skill name present in both .archive/ and a live category) made optimize_skill.py --skill reject the live skill as an ambiguous duplicate.
• scripts/analyze.py — pure formatter: NDJSON sessions on stdin → markdown-ish text on stdout for LLM consumption. No analysis logic lives here; the actual judgment is made by whatever LLM/agent reads this text plus the analyzer prompt. Its only reusable function is format_session(session) -> str (one session at a time) — there is no batch formatter, so an in-process caller joins the per-session strings itself; main() is the stdin/stdout wrapper. Formatted session text plus the analyzer instructions plus the skill index can add up to a sizeable prompt, so provider context limits are a real design constraint on how many sessions one analysis pass can carry — --lookback-hours/max_sessions exist to bound that.
• scripts/proposal.py — the schema and I/O layer:
  • SkillEvolutionProposal dataclass + ProposalType/ProposalStatus enums define the proposal shape.
  • render()/load_proposal() do round-trip markdown+YAML-frontmatter serialization using a hand-rolled frontmatter parser (_parse_frontmatter, no PyYAML) — it's whitespace-sensitive (2-space list items, 4-space nested change blocks). Be careful preserving exact indentation if you edit proposal rendering.
  • get_proposals_dir() defaults to ./proposals/ at the repo root (overridable via SKILL_EVOLUTION_PROPOSALS_DIR) — see "Git tracking" above.
  • list_proposals() skips files it cannot parse but names each one and its reason on stderr. It used to swallow ValueError/IOError entirely, and that fired for real: a model wrote status: already_covered (not in ProposalStatus), load_proposal() raised, and --list/--retroactive reported 7 of 8 files with no hint the eighth existed. Any model-authored field can do this, since the analyzer writes these files. The parse stays strict — the fix was making the loss audible, not accepting unknown values — because a proposal is a unit of work awaiting a human decision, and a file that errors gets looked at while a file that vanishes does not.
  • save_proposal() pipes the rendered markdown through evaluate.redact_secrets() before writing to disk, so a proposal can never persist a secret that leaked into session evidence or a proposed change.
  • apply_proposal() validates status == proposed and confidence >= min_confidence, resolves the active host adapter, and refuses (before the evaluation gate runs at all) if that host can't write skills or if the proposal can't be expressed as a mutation — a create_new with a missing/placeholder body is a drafting failure, not a valid mutation (a create_new proposal has, in practice, shipped a literal placeholder string instead of real content, which Hermes's skill_manage rejects for create). Then it runs the evaluation gate (evaluate.evaluate_and_record()) before mutating anything. If evaluation raises or the gate fails, it returns can_apply: False (with evaluation_results and, on a raised exception, evaluation_error) and leaves the proposal's status untouched. Only once the gate passes does it build a normalized mutation plan and hand it to adapter.apply_skill_write(plan): the Hermes adapter returns skill_manage instruction dicts (the create instruction carries name + body, fixing the placeholder-body bug above) that the agent runs later, while the Claude Code adapter writes skill files directly and immediately. apply_proposal() flips status to applied only after the adapter reports can_apply: True, and the create_new history migration runs only then too — a refused write must never leave an orphaned migration. It does not call skill_manage itself — this script has no Hermes runtime dependency.
  • proposal.py imports evaluate at module scope; evaluate.py avoids importing proposal at module scope (it imports it locally inside _select_retroactive_proposals) specifically to break that cycle — keep that direction if you touch either file's imports.
• scripts/evaluate.py — the evaluation/gating framework (see "Evaluation framework" below).
• scripts/optimize_skill.py — optional GEPA optimizer, gated behind SKILL_EVOLUTION_OPTIMIZER_ENABLED=true and the optional standalone gepa PyPI package (not dspy). --list-candidates prints find_low_scoring_targets()'s entries (history below DEFAULT_LOW_SCORE_THRESHOLD 0.6) read-only, with no optimization run. --skill <name> [--iterations N] runs run_gepa_optimization(): resolves the skill's installed SKILL.md via skill_index.scan_skills() as the seed candidate, fetches that skill's session history via fetch_sessions.sessions_for_skill() (capped at DEFAULT_MAX_SESSIONS_FOR_SKILL, most recent first; exits cleanly below MIN_SESSIONS without importing gepa at all), then calls gepa.optimize_anything() in Single-Task Search mode with an objective built by _build_objective(seed_body, skill_name) — which states the seed body's byte count and the allowed range, plus an instruction to preserve existing section headings. The range is the intersection of all four limits the gate applies, computed and intersected before any branching: the per-pass percentages (SKILL_EVOLUTION_MAX_GROWTH_PCT/MAX_SHRINK_PCT) against the body being replaced, the absolute per-pass deletion floor (MAX_SHRINK_BYTES), the absolute cap applied as the same ratchet the gate applies (MAX_SKILL_SIZE_KB — an oversized seed gets upper = base, i.e. it may not grow at all), and the cumulative percentages against original_size_for_target(). Each omission was the same bug in a new place: stating only the per-pass window over-advertised headroom for a skill partway toward its cumulative ceiling, and ignoring the cap advertised a ~124KB upper bound for the 103,656B skill that the gate rejects outright. An earlier version returned early from inside the cumulative block, so a window emptied by a different constraint went undetected — hence one intersection, then one branch. Notes explaining why the window is narrow are emitted in precedence order (ratchet → cumulative → byte floor); the byte-floor note is not cosmetic, since without it an oversized skill is handed a 2KB-wide window on a 100KB body with no stated reason. If the intersection is empty the objective says so, and which advice it gives depends on which side is violated: over the growth ceiling → reword within the current length (base is still admissible on the floor side); below the shrink floor → no revision of any length passes, so it says the target needs human attention rather than advising a same-length rewrite that would also be rejected. That second branch fixes advice that was silently wrong before. Without that budget the objective named only the task, so the reflection LM optimized purely for score and only met the size limits after the run: one real run spent its entire metric-call budget converging on a +121.8% candidate that was inadmissible from the first byte. It is a soft defence (a prompt the model may ignore) that does not replace the deterministic check — its value is not wasting budget on candidates born dead, scored by score_candidate() (built on evaluate.py's provider/redaction layer, reusing LLMJudgeEvaluator.parse_and_score()) with gepa's own reflective-mutation step also routed through that same provider layer via _reflection_lm_adapter() rather than gepa's litellm/OpenAI default. draft_proposal_from_gepa_result() builds an improve_existing proposal from the winning candidate (rejecting structurally broken candidates before drafting), with a rationale reporting the full explored-candidate frontier. It never calls apply_proposal() — optimizer-drafted proposals re-enter the same human-review/auto-apply gate as any analyst-drafted one. The score-provider's judge feedback (which gepa embeds verbatim into its own internal reflection prompt) is produced under an explicit instruction not to quote session content verbatim and is wrapped in the same untrusted-content framing used elsewhere — narrowing, but not eliminating, a paraphrased-injection residual risk (gepa's own internal prompt construction is out of reach). The three tuning constants MIN_SESSIONS (3), DEFAULT_MAX_METRIC_CALLS (8), and DEFAULT_MAX_SESSIONS_FOR_SKILL (20) are overridable via SKILL_EVOLUTION_OPTIMIZER_MIN_SESSIONS, SKILL_EVOLUTION_OPTIMIZER_MAX_METRIC_CALLS, and SKILL_EVOLUTION_OPTIMIZER_MAX_SESSIONS_FOR_SKILL; an explicit --iterations always overrides DEFAULT_MAX_METRIC_CALLS regardless. optimize_skill.py has been run against the real gepa==0.1.4 package and real session history, which is what these defaults are tuned from: a real optimizer run against a skill with an explicit --iterations 4 surfaced both a real score gain and a pathological +121.8%-growth failure inside those 4 calls, and took over two minutes wall-clock against a hosted provider (30s+/metric-call) — at the old default of 20 that's a 10+ minute unattended foreground call for what is, by design, a manual/occasional invocation (never cron-triggered). 8 keeps 2x headroom over what was empirically sufficient while bounding the worst case instead of doubling it on no further evidence. MIN_SESSIONS and DEFAULT_MAX_SESSIONS_FOR_SKILL were left unchanged: real runs never tested MIN_SESSIONS's boundary, and both 8- and 20-session trainsets ran fine, so neither constant has evidence pointing anywhere in particular.
• The analyzer prompt (maintained by the operator outside this repo, not shipped here — see SKILL.md) is the "business logic" of the analysis half of the system: it defines the analysis criteria (skill coverage, cross-session patterns, specificity, confidence scoring 0.5–1.0), the output format matching proposal.py's schema, and rules like skipping already-processed sessions and never calling the host's skill-mutation tool itself. The Python scripts in this repo are plumbing around that prompt, not a replacement for it. A structural gotcha worth knowing if you're writing your own: create_new proposals have no target_skill, so target_key_for_proposal() falls back to proposal:<uuid>, giving each its own single-entry lineage unless the target key is derived from proposed_changes instead (which is what this repo's RegressionEvaluator does — see below) — otherwise regression tracking is effectively improve_existing-only. The evaluation half (evaluate.py) is separate: it scores whatever content a proposal already contains, regardless of which agent or prompt produced it.

Evaluation framework (scripts/evaluate.py)

Introduced to gate auto-apply on measured quality rather than just the proposal's self-reported confidence.

• EvalResult (score, feedback, passed, evaluator_name) is the shape every evaluator returns. Evaluator is the ABC; concrete evaluators register themselves into the module-level REGISTRY dict via register_evaluator() at import time — there's no dynamic discovery/plugin scanning.
• Which evaluators run is controlled by SKILL_EVOLUTION_EVALUATORS (comma-separated; default "deterministic,llm_judge,regression", resolved by get_enabled_evaluators()). An unknown name raises ValueError rather than silently skipping.
• Five evaluators, all registered at import:
  • DeterministicEvaluator — binary size (SKILL_EVOLUTION_MAX_SKILL_SIZE_KB, default 15KB), growth-vs-baseline (SKILL_EVOLUTION_MAX_GROWTH_PCT, default 20%), shrink-vs-baseline (SKILL_EVOLUTION_MAX_SHRINK_PCT, default 15% — deliberately tighter than the growth cap: an over-long skill is bounded by the absolute 15KB check and merely costs context, whereas deletion silently removes guidance, and 15% keeps one pass's bite to ~1.4KB of a median 9.7KB skill instead of ~1.9KB), and YAML-frontmatter-structure checks. The shrink floor is not symmetry for its own sake: the LLM judge's conciseness criterion rewards deletion, and a real GEPA run cut a skill body 10258B → 3041B (−70.4%), lost 17 of 29 headings including its Red Flags/Common Rationalizations/Anti-Patterns sections, and scored higher for it (0.85 → 0.90). A cap without a floor guards against bloat but not against content destruction. Both per-pass deltas are gated on context["baseline_size"] — see evaluate_skill_text() below. Plus two cumulative limits measured against context["original_size"] — where the target started, not the body it immediately replaces: SKILL_EVOLUTION_MAX_CUMULATIVE_GROWTH_PCT (default 50%) and SKILL_EVOLUTION_MAX_CUMULATIVE_SHRINK_PCT (default 30%). The per-pass limits reset their reference every pass, so they compound: at 20% per pass, four accepted passes halve a skill while each one looks compliant, and RegressionEvaluator can't catch it because every deletion raises the judge's conciseness score, so each pass legitimately outscores the last. The erosion is self-reinforcing. Verified: simulating repeated exact-floor cuts, the gate halts at pass 3 with 62% of the original retained instead of running to zero. Note the cumulative limit governs the final retained fraction, not the per-pass one — retuning the per-pass floor changes how many passes it takes to get there and how big each bite is, not where erosion stops. Cumulative allowances are deliberately wider than per-pass ones — one pass may move 20%, but total drift stays bounded. The absolute cap is a ratchet, not a flat ceiling. It fails only when the candidate is over the cap and larger than the baseline it replaces. A real installed-skill census found roughly 15% of skills already exceeding the 15KB cap (median size well under it, but with real outliers over 100KB) — a spot-check claiming far fewer was wrong; a full census is what should be trusted. Under a flat ceiling every body proposal for an over-cap skill failed on size regardless of direction, so a proposal shrinking one toward compliance was rejected with the identical message as one growing it: the gate could not tell improvement from worsening. create_new carries no baseline, so it keeps the strict cap and a skill is never born oversized — that exemption is a consequence of the rule ("a change that replaces existing text may not worsen; a change that creates text faces the hard cap"), not a proposal-type check to add later. Note this makes oversized skills improvable, not downsizable: at 2048B/pass, walking a 100KB+ skill to compliance would take dozens of approved passes, so a very large skill still needs a human to split it. Plus an absolute per-pass deletion floor, SKILL_EVOLUTION_MAX_SHRINK_BYTES (default 2048, 0 disables), applied as a second sequential check after the percentage floor — the stricter of the two binds. A percentage scales with the skill, so it is weakest exactly where a deletion does most damage: 15% of the median skill is ~1.5KB, but 15% of the 103,656B one is 15,548B in a single pass. The default comes from the crossover, not from taste: 2048/0.15 = 13,653B, just under the 15,360B cap, so the byte floor is inert for essentially every within-cap skill by construction and operative precisely on the oversized tail the ratchet unblocks (it binds 35 of 143). The two checks are sequential rather than one min() so the feedback names whichever limit actually bound. These two changes are causally coupled and must not be separated: the 15KB single-pass deletion was previously masked (the shrunk candidate was still over the cap and failed there first), so ratcheting the cap without the byte floor is a net regression in deletion safety. Growth needs no byte counterpart: per-pass absolute growth is bounded at cap × g/(1+g) = 2,560B for a within-cap skill, and at 0 for an oversized one under the ratchet. That invariant scales with the cap — raising SKILL_EVOLUTION_MAX_SKILL_SIZE_KB rescales it, so revisit if the cap moves. The frontmatter check only runs when context["content_kind"] == "body" — a bare description change or a merge/deprecate summary+rationale fallback is plain prose and is exempt.
  • LLMJudgeEvaluator — rubric-based (correctness, procedure_following, conciseness, each 0–1, threshold SKILL_EVOLUTION_LLM_JUDGE_THRESHOLD default 0.7). Frames evaluated content as untrusted data between a per-call random hex boundary (not a static tag) specifically so injected content can't forge the closing marker and escape the block; fails closed (self._fail(...)) on any provider error or malformed/out-of-range JSON response rather than guessing.
  • _post_json() sets a descriptive User-Agent (USER_AGENT = "skill-evolution/0.1"), overridable per call via the headers argument. This is load-bearing, not cosmetic: urllib's default Python-urllib/3.12 is blanket-blocked by OpenCode Zen's gateway, which answers 403 for it and 200 for the byte-identical request under any descriptive UA. If a provider starts returning 403 while curl succeeds, check the UA before suspecting the key. Verified harmless against a real Ollama server (ollama==0.30.10): its endpoint doesn't gate on User-Agent at all, so the fix changes nothing there — confirmed via a full round trip (LLMJudgeEvaluator.evaluate() → _call_ollama() → _post_json()) scoring a real installed skill. That pass also surfaced a real, previously unknown friction point unrelated to the UA itself: a reasoning-capable local model (gemma4) used 54–56s of the 60s default call_provider timeout on one judge call, and a heavier one (qwen3.5) timed out outright — local models burn a large share of their output on chain-of-thought before the JSON, in a way a hosted API typically doesn't. That friction is now tunable via SKILL_EVOLUTION_PROVIDER_TIMEOUT (see the provider adapter layer below). Still unverified against Claude (no ANTHROPIC_API_KEY in this environment).
  • OpenCode Zen gotcha: two catalogues exist and they are not interchangeable. Zen (https://opencode.ai/zen/v1, the _call_opencode default) carries big-pickle and the *-free variants (deepseek-v4-flash-free, mimo-v2.5-free, …); the Go subscription tier (https://opencode.ai/zen/go/v1) is a different, smaller list without big-pickle and whose DeepSeek id has no -free suffix. Point SKILL_EVOLUTION_OPENCODE_BASE_URL at the tier matching the model id you set. Both are OpenAI-compatible (/chat/completions, bearer auth), so the caller mirrors the Ollama one; unlike Ollama it reaches a third party, so a missing OPENCODE_API_KEY fails closed with an actionable message instead of sending an unauthenticated request. Model lists are public — curl -s https://opencode.ai/zen/v1/models needs no key.
  • RegressionEvaluator — compares a target's new aggregate score against the last entry that actually passed in that target's history (not just the last entry) so a failed attempt can never lower the bar a later one is compared against. No prior passing entry → auto-passes ("no baseline yet"). Works for both improve_existing and create_new proposals: target_key_for_proposal() extracts the skill name from proposed_changes for create_new, and migrate_proposal_history() rewrites proposal:<id> entries to skill:<name> after creation, connecting the skill's full evaluation lineage.
  • HumanReviewEvaluator — opt-in interactive gate (P2-4). Not in DEFAULT_EVALUATORS; enable via SKILL_EVOLUTION_EVALUATORS=...,human_review. Requires a TTY (stdin/stdout isatty()); fails closed without one. Binary prompt: y/yes → approve, n/no → reject, empty/junk re-prompts up to 3 times then fail-closed; EOF/KeyboardInterrupt fail-closed. Runs last in run_evaluators() after automatic + regression, receives prior_results context. Approve/reject return new_score (the automatic aggregate), mirroring RegressionEvaluator — a hardcoded 1.0 would inflate eval_history.jsonl gate entries that RegressionEvaluator baselines against. Fail-closed returns 0.0. Optional rejection reason captured ("no reason given" if blank). Feedback: "human approved: no note" / "human rejected: <reason>". scripts/skill_quality.py strips human_review from the env var at startup (warns on stderr) to prevent per-skill interactive prompts in the cron wrapper.
  • EmbeddingSimilarityEvaluator — opt-in semantic similarity checks (P2-5). Not in DEFAULT_EVALUATORS; enable via SKILL_EVOLUTION_EVALUATORS=...,embedding_similarity. Requires the embeddings extra (pip install -e ".[embeddings]"), which installs fastembed (~50MB, ONNX-based). Three modes selected automatically by context: duplicate_detection (flags content too similar to existing skills, similarity > threshold), drift_detection (flags content that drifted too far from baseline, similarity < threshold), grounding_check (flags content not grounded in source sessions, average similarity < threshold). Thresholds configurable via SKILL_EVOLUTION_EMBEDDING_DUPLICATE_THRESHOLD (0.85), SKILL_EVOLUTION_EMBEDDING_DRIFT_THRESHOLD (0.70), SKILL_EVOLUTION_EMBEDDING_GROUNDING_THRESHOLD (0.60). Backend architecture is extensible via SKILL_EVOLUTION_EMBEDDING_BACKEND (default fastembed); other backends (ollama, openai, llama_cpp) are stubbed with implementation examples in their docstrings. Registration gotcha: unlike the other four evaluators (registered at evaluate.py module-import time), embedding_similarity's registration only happens inside evaluate.py's if __name__ == "__main__": block (scripts/evaluate.py:2116-2123) — nothing else in the pipeline imports it. Running python3 scripts/evaluate.py ... picks it up automatically; importing evaluate as a library (which is what proposal.py's apply_proposal() and skill_quality.py do) does not, so setting SKILL_EVOLUTION_EVALUATORS=...,embedding_similarity in that path raises "Unknown evaluator" unless something upstream already did import embedding_similarity first.
• Gate combination: run_evaluators() runs in three phases — (1) automatic evaluators (everything except regression and human_review), (2) regression fed the mean of automatic scores as new_score (human score never enters the regression baseline), (3) human_review last with prior_results context (automatic + regression verdicts). combine_gate(results, strictness) then combines per SKILL_EVOLUTION_GATE_STRICTNESS (or a per-proposal-type override, SKILL_EVOLUTION_GATE_STRICTNESS_<TYPE>, or a per-target override SKILL_EVOLUTION_GATE_STRICTNESS_<TARGET>, resolved by resolve_gate_strictness() — target beats type beats global): "strict" (default) requires every evaluator to pass; "majority" requires more than half. Zero configured evaluators never blocks.
• Multi-target gate (P2-1): evaluate_and_record() runs every target in resolve_gate_targets() (SKILL_EVOLUTION_GATE_TARGETS, default skill,proposal; unknown names raise ValueError; empty falls back to default) through the same registry and appends one combined gate entry per gating target — skill:<name>, proposal:<uuid>, and per-session tool_calls:<sid>/analyzer_prompt:<sid> — each combined under its own strictness, then ANDs the per-target decisions. Targets with no data don't block: a proposal without session_ids skips the session-based targets. A provider fault is flagged per-target on the entry it affected. The kind history field tags each entry's lineage (skill_text/proposal/tool_calls/analyzer_prompt); migrate_proposal_history() uses it to leave proposal-document entries under proposal:<id> instead of folding them into the skill lineage.
• Provider adapter layer (call_provider()) is stdlib-only urllib HTTP to Claude (ANTHROPIC_API_KEY + SKILL_EVOLUTION_CLAUDE_MODEL), Ollama/llama.cpp (SKILL_EVOLUTION_OLLAMA_BASE_URL/SKILL_EVOLUTION_OLLAMA_MODEL), OpenCode Zen (OPENCODE_API_KEY + SKILL_EVOLUTION_OPENCODE_BASE_URL/SKILL_EVOLUTION_OPENCODE_MODEL), OpenAI (OPENAI_API_KEY + SKILL_EVOLUTION_OPENAI_BASE_URL/SKILL_EVOLUTION_OPENAI_MODEL, default https://api.openai.com/v1 / gpt-4o), or Gemini (GEMINI_API_KEY + SKILL_EVOLUTION_GEMINI_BASE_URL/SKILL_EVOLUTION_GEMINI_MODEL, default https://generativelanguage.googleapis.com / gemini-2.0-flash), selected by SKILL_EVOLUTION_PROVIDER (default claude) with an optional per-evaluator override SKILL_EVOLUTION_<EVALUATOR_NAME>_PROVIDER. OpenAI and Gemini follow the same _post_json() pattern as the other three branches; Gemini is the one shape outlier — it hits v1beta/models/{model}:generateContent with a contents[].parts[].text request body and candidates[].content.parts[].text response, and authenticates via the x-goog-api-key header rather than a bearer token, keeping the key out of URLs. Both fail closed on a missing key or a malformed response shape, matching OpenCode's posture. GEMINI_API_KEY is also in fetch_sessions.SECRET_PATTERNS, so it's caught by the redaction layer below like OPENAI_API_KEY/ANTHROPIC_API_KEY already were. The HTTP timeout is one generic knob for every provider — SKILL_EVOLUTION_PROVIDER_TIMEOUT (seconds, default 60), resolved by resolve_provider_timeout() inside call_provider(); an explicit timeout= argument always wins. It is deliberately not per-provider (no SKILL_EVOLUTION_OLLAMA_TIMEOUT): the latency it exists for is a property of the model, not the adapter — local reasoning models spend most of their output on chain-of-thought before the JSON (a real gemma4 judge call took 54–56s of the 60s default, and qwen3.5 timed out outright). Every prompt is run through redact_secrets() (reusing fetch_sessions.py's contains_secret()/SECRET_PATTERNS, whole-line replacement) before it ever leaves the machine, independent of the redaction save_proposal() already does before writing to disk — R18/R19 in the plan doc treat "sent to a provider" and "written to disk" as two separate redaction points, not one.
• History: entries now also carry content_size and baseline_size (utf-8 bytes), which is what makes cumulative drift detectable — original_size_for_target() reads the earliest baseline_size for a target (falling back to the earliest content_size) so a candidate can be compared against where the skill began. Both fields are optional, so pre-existing history written before size recording simply yields None and leaves the cumulative check inert. evaluate_and_record() and evaluate_skill_text() both derive the evaluated text via the shared _extract_evaluated_content() helper, so the sizes recorded are measured over exactly the text that was scored. append_history()/read_history() read/write one shared append-only JSONL file (get_history_path(), default ./eval_history.jsonl at the repo root, overridable via SKILL_EVOLUTION_HISTORY_PATH — see "Git tracking" above), keyed per-target (target_key_for_proposal() → "skill:<name>" or "proposal:<id>" fallback; for create_new proposals it extracts the skill name from proposed_changes before falling back, and migrate_proposal_history() rewrites history from proposal:<id> to skill:<name> after the skill is created). prune_history() enforces SKILL_EVOLUTION_HISTORY_RETENTION (a bare int = max versions, "90d"/"6mo" = age-based), and now archives rather than deletes: dropped entries are appended (not overwritten) to a sibling file before the primary is rewritten — get_history_archive_path(), default derived from the primary path's basename (eval_history.jsonl → eval_history.archive.jsonl), overridable independently via SKILL_EVOLUTION_HISTORY_ARCHIVE_PATH. Writing the archive before rewriting the primary matters: a crash mid-prune then loses nothing, where the reverse order could lose data permanently. R22 in the eval plan permits either "archived or compacted" — archiving was chosen because it already satisfies R22's actual goal (a bounded primary file) without a lossy "compacted summary" record needing to independently satisfy three different readers. The survivor rule per target is up to three always-kept anchors, not one — the most recent entry (needed by optimize_skill.find_low_scoring_targets()), the most recent passing entry, and the earliest entry, deduplicated when they coincide. The retained count per target can therefore exceed the configured limit by up to two entries; that's intentional. Two real correctness holes motivated the extra two anchors: RegressionEvaluator baselines against the last entry with passed == True, not just the last entry — the old "keep only the most recent" rule could prune a passing baseline while newer failing retries survived, silently falling back to "no baseline yet" for exactly the targets that most need regression coverage; and original_size_for_target() reads the earliest entry as its cumulative-drift baseline — without protecting it, trimming a target's oldest entries would silently shift that baseline forward, forgiving prior drift. Both are closed by always retaining the anchor entry itself, rather than giving those two functions a fallback read into the archive file — same safety guarantee, no second data-access path to build and maintain. Pruning now runs automatically, not just via --prune. evaluate_and_record() takes auto_prune: bool = True and calls prune_history() right after its append_history() call — a no-op by default, since prune_history() itself early-returns unless SKILL_EVOLUTION_HISTORY_RETENTION is configured. The same env var now controls both whether pruning happens and that it happens on every real evaluation from then on; apply_proposal() calls evaluate_and_record(proposal) with no extra kwargs, so it gets this for free. retroactive_reevaluate()'s loop passes auto_prune=False and calls prune_history() once after the whole batch instead — prune_history() rewrites the shared history file (every target, not just the one being re-evaluated), so pruning after every append in an N-proposal batch would mean N full-file passes for a result identical to pruning once. --prune remains available for one-off backfill of pre-existing bloat, and its report now includes the archived count.
• evaluate_skill_text() extracts the body/description change and builds the evaluator context. It passes content_kind and baseline_size, measured the way DeterministicEvaluator measures the candidate. That wiring is load-bearing: the growth-vs-baseline guard is if baseline_size:-gated, so omitting the key silently disables it — a proposal could balloon a skill +343% and still report "all deterministic checks passed." create_new has no baseline, so the key stays absent and the guard stays inert rather than dividing by a zero baseline. baseline_size comes from the installed SKILL.md, not from the change's old_value (_resolve_baseline() → installed_skill_body(), body changes only). old_value is written by the analyzer LLM — the analyzer prompt asks it to emit old_value: <current value> and proposal.py persists it verbatim — so it is proposal-supplied input, not an observation. That became load-bearing when the cap turned into a ratchet: the cap now asks "is this larger than what it replaces?", so an inflated old_value would raise the very ceiling it is checked against, and a proposal claiming a 951KB baseline could ship a 950KB body past a 15KB cap while every percentage check reported a rounding-error delta. This is not mainly an adversarial concern — a duplicated or truncated transcription is an ordinary LLM failure, and --retroactive re-scores proposals whose old_value may no longer describe anything on disk. optimize_skill._resolve_baseline_size_bytes() already applied this distrust at draft time; this is the same discipline at the gate. Body changes only (for a description change the installed file is not what it replaces — substituting it would read as a ~−98% shrink), best-effort (unresolvable or ambiguous skill → fall back to old_value rather than fail a gate decision on a lookup miss), and shared with evaluate_and_record() so the size recorded in history is the size that was actually judged.
• Three fast-follow evaluation targets (R5) exist as sibling functions to evaluate_skill_text(), reusing the same registry and gate without modifying it or apply_proposal()'s call site:
  • evaluate_proposal(proposal) — scores the proposal as a document (summary, rationale, proposed_changes). Target key: proposal:<uuid> (always, regardless of whether the proposal has a target_skill). No baseline — proposals are evaluated on their own merits.
  • evaluate_tool_calls(session_id_or_messages) — scores tool-call quality from session messages. Input can be a session_id (queries state.db via _fetch_session_messages()) or a list of message dicts (for testing). Target key: tool_calls:<session_id>. Content is JSON-serialized tool_calls with result snippets capped at 200 chars each, total cap 5KB.
  • evaluate_analyzer_prompt(session_id, proposal_id) — scores whether the analyzer's generation step produced a grounded proposal. Reads session messages (truncated to 300 chars each, total cap 10KB) and the saved proposal markdown. Target key: analyzer_prompt:<session_id>. Wiring posture (P2-1): proposal gates auto-apply alongside skill by default; tool_calls/analyzer_prompt gate only if added to SKILL_EVOLUTION_GATE_TARGETS. The gate decision is the AND of every gating target, each combined under its own strictness. deterministic is opt-in only** for these targets (via SKILL_EVOLUTION_EVALUATORS) — its size/growth checks are meaningless for non-skill-text content. The default set (llm_judge,regression) is the right one for proposal/tool-call/analyzer-prompt evaluation.
• evaluate_and_record() is the single call site both apply_proposal() and the retroactive path share — it runs each gating target (default skill,proposal), combines each under its own strictness, ANDs the per-target decisions, and appends one combined EvalResult (evaluator_name="gate") per gating target to history, so both paths produce identical history semantics.
• Retroactive/batch mode (--retroactive, retroactive_reevaluate()) only re-evaluates already-saved proposals with status == proposed by default (_select_retroactive_proposals) — rejected/applied proposals are excluded because they're not live decisions and re-scoring them would inject entries into the same history stream the live gate regresses against; pass include_all_statuses=True (Python-only, no CLI flag yet) to opt in. --target/--since filter further.
• Module-identity gotcha: running python3 scripts/evaluate.py directly loads it as __main__; since proposal.py does import evaluate at module scope, the file registers itself under sys.modules["evaluate"] at the top (if __name__ == "__main__": sys.modules.setdefault(...)) so both names resolve to the same module instance (same REGISTRY, same classes) instead of proposal.py re-executing and re-registering everything as a second, distinct module.

Proposal types and the write side (apply_proposal() → adapter.apply_skill_write(plan))

type	What it means	Hermes emits (applied_by: agent)	Claude Code does (applied_by: direct)
improve_existing	Patch a skill's description/body	one patch instruction per proposed_changes entry	rewrites the installed SKILL.md in place (exact old_value match, else refuse)
create_new	New skill for a recurring uncovered pattern	one create instruction carrying name + full body (fixed to require real content, not a placeholder) + description/category	writes a real skills/<name>/SKILL.md (refuses existing names)
merge_skills	Combine overlapping skills into target_skill	one delete instruction per source_*-prefixed change, each tagged absorbed_into	archives each source into skills/.archive/<name>/ after validating the umbrella exists
deprecate_skill	Remove a stale/unused skill	one delete instruction	archives the skill into skills/.archive/<name>/

All four types pass through the same evaluation gate (above) before the adapter is asked to mutate anything — the gate runs once per apply_proposal() call regardless of type, and apply_proposal() only flips status to applied after the adapter reports can_apply: True.

Conventions specific to this codebase

• Prefer host-agnostic design. Multi-agent support (Codex, Claude Code, Claude, future providers) is an explicit project goal, and the owner's stated principle is that the more agnostic the design, the better — so treat host-neutrality as the default for new work instead of a later retrofit. The Hermes coupling used to be four hardcoded points scattered one-per-module; the read side of that is now a real abstraction, not just an isolation convention. scripts/host.py defines HostAdapter (an ABC with three read methods — iter_sessions(since=None), iter_skills(), read_skill_body(skill_name) — plus the write side: supports_write: bool = False and a fail-closed concrete apply_skill_write(plan) that returns {"can_apply": False, "reason": ...} unless an adapter overrides it), a plain-dict HOST_ADAPTERS registry populated by register_adapter() at import time (instances, not classes — a host adapter carries no per-call state, so nothing is gained by re-instantiating per lookup, unlike evaluate.REGISTRY), and resolve_host()/get_adapter() reading SKILL_EVOLUTION_HOST (default hermes) in the same explicit-arg-beats-env-var-beats-default order evaluate.resolve_provider() already uses — an unknown host name raises rather than silently reading the wrong tree. HermesAdapter reproduces today's behavior byte-for-byte by delegating to fetch_sessions.fetch_sessions()/skill_index.scan_skills(). ClaudeCodeAdapter (root: SKILL_EVOLUTION_CLAUDE_CODE_HOME, default ~/.claude) is the second adapter, reading skills from the flat skills/*/SKILL.md tree (no category level — category is reported as the constant "user") and sessions from projects/*/*.jsonl, one JSONL-per-session with an undocumented, versioned record format handled defensively (allowlist role mapping, best-effort timestamp/field extraction, never raising on a malformed line or record) — it routes through the same fetch_sessions.contains_secret()/redact_pii()/_summarize_messages() redaction chokepoint rather than reimplementing it. evaluate.py (installed_skill_body()) and optimize_skill.py now read skills through host.get_adapter() instead of calling skill_index.scan_skills() directly, so both are host-neutral as of this abstraction landing. The two host-neutral contracts a new adapter must produce are the session dict shape (session_id, started_at, title, model, source, message_count, user_messages, assistant_messages, messages — the last already redacted/truncated) and the skill dict shape (name, category, description, path, size); read_skill_body()'s contract is narrower still — best-effort text or None on no match, an ambiguous match, or a read failure, never raise, since callers like the evaluation gate's baseline-size lookup must degrade to whatever baseline they already had rather than fail a gate decision on an unrelated lookup. The write side shipped with the same abstraction (2026-07-31). proposal.py's apply_proposal() remains the sole mutation path, but it now builds a host-neutral mutation plan and hands it to adapter.apply_skill_write(plan), and it refuses (before the gate runs) on any host that can't write or any proposal that can't be expressed as a mutation — never spending a provider call on a proposal that can never apply. HermesAdapter.supports_write = True and its apply_skill_write() re-emits the legacy skill_manage instruction dicts byte-for-byte (returned applied_by: agent for the cron agent to execute later). ClaudeCodeAdapter.supports_write = True and its apply_skill_write() performs the writes directly and immediately (returned applied_by: direct): real skills/<name>/SKILL.md files via atomic temp-file + os.replace, deprecate/merge archive sources into skills/.archive/<name>/, and symlinked skill directories are refuse-to-write (38/41 installed Claude Code skills are symlinks into ~/.agents/skills/ — the adapter writes around them instead of following or replacing them). The Hermes adapter is instruction-emitting and the Claude Code adapter is file-writing, so they cannot share a code path; what they share is the gate, the plan shape, and the fail-closed posture. The processed-session state seam shipped (2026-08-01). HostAdapter now has three concrete state methods — iter_processed(), mark_processed(session_ids), prune_processed(retention=None, keep_ids=None) — plus a _state_file() hook for path resolution. Each adapter owns its own state file: HermesAdapter inherits the default (~/.hermes/skill_evolution_state.json), while ClaudeCodeAdapter overrides _state_file() to return <CLAUDE_CODE_HOME>/skill_evolution_state.json. SKILL_EVOLUTION_STATE_FILE remains the universal override — when set, it redirects whichever host is active. fetch_sessions_for_host() and main() --prune-state now route through the adapter's state methods instead of calling module-level functions with host=. state.py is now imported by host.py at module scope (no cycle — state.py imports nothing local), making it a real library rather than a test-only module. The host-prefix scheme (<host>:<id> keys) stays for legacy-entry handling and shared-override safety. fetch_sessions() (the Hermes cron path) stays byte-for-byte — it's the critical deployed path and already calls the state functions with host="hermes" and the default path. That includes prose in prompts: the GEPA objective deliberately says "this agent skill", not "this Hermes Agent skill", both to survive generalization and to avoid biasing the reflection LM toward one host's conventions. The remaining write-side follow-up is the project-level .claude/skills/ tree, which is deliberately out of scope — the read side walks one flat root (SKILL_EVOLUTION_CLAUDE_CODE_HOME) and "which project" is undefined at the adapter level.
• Every script is a standalone stdlib-only CLI with its own argparse in main(), importable for its core function (fetch_sessions(), scan_skills(), apply_proposal(), run_evaluators(), etc.) — keep new scripts consistent with this pattern rather than introducing a shared framework/package layout.
• No PyYAML anywhere by design (SKILL.md promises "no pip packages required") — frontmatter is parsed and rendered by hand in both skill_index.py and proposal.py. If a change needs richer YAML (nested structures beyond flat keys + one-level lists), it likely needs a matching parser update in _parse_frontmatter, not just a render() change.
• Auto-apply is opt-in via env vars (SKILL_EVOLUTION_AUTO_APPLY, SKILL_EVOLUTION_MIN_CONFIDENCE), read by the cron agent/prompt, not by the Python scripts themselves. The evaluation gate inside apply_proposal(), by contrast, always runs (it's not behind an opt-in flag) — SKILL_EVOLUTION_EVALUATORS/SKILL_EVOLUTION_GATE_STRICTNESS only change which evaluators run and how strictly, not whether the gate runs at all.
• gepa is imported lazily (_require_gepa() inside optimize_skill.py, called from run_gepa_optimization(), not at module import time) so the rest of the pipeline works with the dependency absent — don't move that import to module scope. _require_gepa() returns the gepa.optimize_anything submodule, not the top-level gepa package — that submodule is the namespace holding the optimize_anything function plus GEPAConfig/EngineConfig/ReflectionConfig. The package deliberately binds the name optimize_anything to the submodule (see gepa/__init__.py: "expose submodule; use from gepa.optimize_anything import optimize_anything for the function"), so reading those names off the package yields a non-callable module and three AttributeErrors. Verified against gepa==0.1.4; the parameter names the call site passes (seed_candidate/evaluator/objective/config, max_metric_calls, reflection_lm) are all correct for that version.
• Stubbed-dependency tests can't catch API drift — every test in tests/test_optimize_skill.py monkeypatches _require_gepa to return a hand-built fake, so it asserts the code against its own assumptions. Pair that with a contract test against the real package (tests/test_optimize_skill_gepa_contract.py, guarded by pytest.importorskip("gepa") so it skips without the optional extra). The same rule applies to seams between modules: the isolated DeterministicEvaluator growth tests pass baseline_size in by hand, which is why nothing caught evaluate_skill_text() never supplying it — assert wiring end-to-end, not just each side of the seam.
• Tests exist now (tests/test_*.py, one file per evaluator/feature area, run via pytest; last verified 2026-08-04: 780 passed + 1 skipped under plain python3 (no gepa, no [embeddings] extra) — the skip is the gepa-contract module, guarded by pytest.importorskip("gepa"). Under a .venv with gepa but not fastembed it's 777 passed + 7 skipped, the 6 extra skips being tests/test_embedding_backends.py cases guarded by pytest.importorskip("fastembed") — so the exact pass/skip split depends on which optional extras are installed, not just gepa alone. Treat any specific number here as a point-in-time sanity check, not a contract — the suite changes; re-run pytest -q -rs for the current count rather than trusting this line) — this replaces an earlier state where pyproject.toml configured pytest but no tests/ directory existed. New evaluators or gate behavior should get a matching tests/test_evaluate_*.py.