skill-evolution/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## What this is

A host-agnostic agent skill (`skill-evolution`) that gives the host agent a self-improvement feedback loop: it reads the agent's past session transcripts through the active `HostAdapter` (default Hermes → `~/.hermes/state.db`; Claude Code → `~/.claude/projects/*/*.jsonl`), compares them against the agent's installed skills, generates structured markdown proposals (`improve_existing`, `create_new`, `merge_skills`, `deprecate_skill`) for a human to review, and — if opted in — auto-applies high-confidence proposals through the host's mutation channel (Hermes: `skill_manage` instructions; Claude Code: direct file writes), gated on an evaluation framework that scores proposal content before anything mutates.

There is no daemon and no server component. The "pipeline" is a set of standalone Python stdlib scripts that a scheduled job invokes and pipes together; the actual analysis step is an LLM call made by the host agent itself (not by any script here), following an analyzer prompt that the operator maintains outside this repo (see SKILL.md's "Scheduled Runs" section for what that prompt needs to do). The evaluation gate (`scripts/evaluate.py`), by contrast, *is* implemented in this repo — it's not delegated to the cron agent.

## Git tracking

`.gitignore` used to use unanchored patterns that matched several directory names anywhere in the tree, silently excluding real pipeline code from version control while `git status` read clean. This was fixed by anchoring the patterns.

`proposals/`, `reports/`, and `eval_history.jsonl` are gitignored and not shipped by this repo — `proposals/` in particular holds real analysis output derived from actual session transcripts, and free-text personal narrative a model can write into a proposal body isn't reliably caught by shape-based redaction, so these stay local-only rather than committed. This has no effect on runtime behavior: the pipeline still reads/writes these paths exactly as described below; only what `git` tracks is affected. `scripts/*.py` and `tests/*.py` remain fully tracked.

`proposal.py`'s `get_proposals_dir()` defaults to `./proposals/` at the repo root
(overridable via `SKILL_EVOLUTION_PROPOSALS_DIR`), and `evaluate.py`'s history file
defaults to `./eval_history.jsonl` at the repo root (overridable via
`SKILL_EVOLUTION_HISTORY_PATH`). The skill-quality cron wrapper writes timestamped markdown
reports to `./reports/` (overridable via `SKILL_EVOLUTION_QUALITY_REPORT_DIR`).

## Commands

```bash
# Run the pipeline manually, end to end (mirrors what the cron job does)
python3 scripts/fetch_sessions.py --dry-run              # preview sessions without marking them processed
python3 scripts/fetch_sessions.py | python3 scripts/analyze.py   # NDJSON sessions -> LLM-ready prompt text
python3 scripts/skill_index.py                            # scan ~/.hermes/skills/ -> JSON index

# Proposal inspection/scaffolding
python3 scripts/proposal.py --example     # print an example proposal
python3 scripts/proposal.py --list        # list current proposals
python3 scripts/proposal.py --show <id>   # show one proposal by id

# Evaluation gate
python3 scripts/evaluate.py --list-evaluators             # show the resolved evaluator set
python3 scripts/evaluate.py --retroactive --dry-run       # preview batch re-evaluation of saved proposals
python3 scripts/evaluate.py --retroactive                 # re-run evaluation against saved proposals, append history
python3 scripts/evaluate.py --prune                       # prune eval history per SKILL_EVOLUTION_HISTORY_RETENTION

# One-off evaluation of any of the four targets (advisory by default — only `skill` and
# `proposal` gate auto-apply; see SKILL_EVOLUTION_GATE_TARGETS below).
# `deterministic` is meaningless for the non-skill targets, so opt out of it.
SKILL_EVOLUTION_EVALUATORS=llm_judge,regression python3 scripts/evaluate.py --eval-target proposal        --proposal-id <id>
SKILL_EVOLUTION_EVALUATORS=llm_judge,regression python3 scripts/evaluate.py --eval-target tool_calls      --session-id <id>
SKILL_EVOLUTION_EVALUATORS=llm_judge,regression python3 scripts/evaluate.py --eval-target analyzer_prompt --session-id <id> --proposal-id <id>

# Inspect processed-session state (read-only; state.py is a library, not a CLI)
python3 -c "import sys; sys.path.insert(0,'scripts'); import state; print(len(state.load_processed()), 'sessions already processed')"
python3 scripts/fetch_sessions.py --prune-state           # prune state per SKILL_EVOLUTION_STATE_RETENTION (reports on stderr)

# Optional GEPA optimizer (needs `pip install -e ".[optimizer]"`)
SKILL_EVOLUTION_OPTIMIZER_ENABLED=true python3 scripts/optimize_skill.py --list-candidates                # read-only report of low-scoring targets (skill: targets only)
- **`scripts/skill_quality.py`** — periodic skill quality tracking (P2-3, shipped 2026-08-01). Evaluates all installed skills against the same rubric as the evaluation gate (correctness, procedure-following, conciseness), records each evaluation in `eval_history.jsonl` under `skill:<name>`, and generates a markdown or JSON report showing trends over time. Reuses `evaluate_skill_text()` with a synthetic proposal (field=body, same old/new value so size guards pass), `installed_skill_body()` for skill content, and the existing provider infrastructure. CLI: `--skill <name>` (evaluate one), `--output <file>` (write report), `--since Nd` (filter by recency), `--below 0.7` (filter by threshold), `--format json|markdown`. Cost: one LLM judge call per skill (~$0.01-0.05), so a full tree of 143 skills costs ~$1.50-7.00 per run — recommend weekly/monthly, not daily. Cron wrapper: `scripts/skill-quality-report.sh`.
SKILL_EVOLUTION_OPTIMIZER_ENABLED=true python3 scripts/optimize_skill.py --list-candidates --target all   # include proposal:/tool_calls:/analyzer_prompt: namespaces
SKILL_EVOLUTION_OPTIMIZER_ENABLED=true python3 scripts/optimize_skill.py --skill <name> [--iterations N]  # run GEPA for one skill

# Tests
pip install -e ".[dev]"
pytest
```

**Interpreter note.** The core pipeline is stdlib-only, so any `python3` runs it. The optimizer is not: `gepa` must be importable, so `optimize_skill.py --skill` needs the interpreter you installed the `[optimizer]` extra into. A project-local `.venv` on pyenv 3.12.2 is the setup these commands were verified against — use `.venv/bin/python scripts/optimize_skill.py ...` for optimizer commands, or the bare `python3` shown above for everything else. Running the optimizer under an interpreter without `gepa` fails fast with `_require_gepa()`'s actionable `RuntimeError` rather than misbehaving. `pytest` passes under either interpreter — the gepa-contract module skips when the extra is absent; see "Tests exist now" below for the currently-verified pass/skip counts rather than a number duplicated here (a stale duplicate is exactly how this line and that one drifted out of sync with each other before).

No build step and no non-stdlib runtime dependencies for the core pipeline. `gepa` (the standalone PyPI package, not `dspy`) is an optional extra (`pip install -e ".[optimizer]"`) required only by `scripts/optimize_skill.py`; nothing else imports it.

## Architecture

```
fetch_sessions.py ──NDJSON──► analyze.py ──formatted prompt──┐
skill_index.py ──JSON index──────────────────────────────────┼──► [host agent + analyzer prompt]
                                                               │
                                                               ▼
                                                proposal.py (write proposal .md files, status: proposed)
                                                               │
                                                     human review (status: approved)
                                                               │
                                                  apply_proposal() ──► evaluate.evaluate_and_record()
                                                               │            (deterministic + llm_judge + regression,
                                                               │             appends to eval_history.jsonl)
                                                               │
                                                        gate passed? ──No──► can_apply: False, proposal stays proposed
                                                               │
                                                              Yes
                                                               │
                                                               ▼
                                                     skill_manage (Hermes tool)

optimize_skill.py (optional, SKILL_EVOLUTION_OPTIMIZER_ENABLED=true)
  --list-candidates: reads eval_history.jsonl, prints low-scoring targets (read-only, no proposal written)
  --skill <name>: fetch_sessions.sessions_for_skill() ──► gepa.optimize_anything() ──► drafts improve_existing
                  proposal from the winning candidate via proposal.save_proposal()
  (never calls apply_proposal() itself — optimizer-originated proposals go through the same gate as any other)
```

- **`scripts/fetch_sessions.py`** — reads `~/.hermes/state.db` (SQLite; `sessions`/`messages` tables), filters out already-processed sessions (via `state.py`'s state file) and anything older than `--lookback-hours`, redacts messages via `contains_secret()` — which layers the fixed `SECRET_PATTERNS` substrings (known vendor prefixes, specific env-var names) with shape-based `SECRET_REGEXES`. The regex layer exists because the substring list provably leaked: scanning the real 26,794 message bodies found 27 `password=` assignments, 23 generic `*_TOKEN=`/`*_SECRET=` assignments, 6 generic `sk-` keys, 5 `Authorization: Bearer` headers and a credentialed DB URI that it missed, all of which would have been forwarded to a provider verbatim. Length floors (20+ chars for opaque keys, 8+ for assigned values) keep `MAX_TOKENS=1024` and `DEBUG=true` off the list; measured over-redaction is 0.229% of lines. This predicate is the single chokepoint for **both** redaction points (provider prompts and on-disk proposals), so err toward over-detection — applies `redact_pii()` (see below), truncates long messages, and emits one JSON object per session to stdout. Marks sessions processed as a side effect unless `--dry-run`. Its queries select `sessions.started_at`/`messages.timestamp` (matching the live schema); the per-session output dict's timestamp key is `started_at` (there is no `created_at` key, and `total_tokens` is not emitted — `analyze.py` defaults it). `sessions_for_skill()` (used by the optimizer) is a separate query on the same DB that filters sessions to those touching a given skill, capped at `max_sessions` most-recent-first. **The two functions handle an unreadable DB differently, on purpose:** `sessions_for_skill()` checks existence and the `sessions` table via `_session_db_is_readable()` *before* connecting and returns `[]` with a stderr warning, because the optimizer is interactive and "no history for this skill" is a normal outcome there; `fetch_sessions()` still raises, because it is the unattended cron path where an unreadable DB is a failure the operator needs surfaced rather than a run quietly reporting zero sessions every night. Checking before `sqlite3.connect()` also matters because `connect()` **creates** a file for a missing path — a typo'd `--db-path` previously crashed with a confusing "no such table: sessions" *and* left a stray 0-byte DB behind.
- **Secrets and PII are handled differently, on purpose.** A detected **secret** drops the whole message (`_summarize_messages` skips it) — there is no version of a message worth sending once it carries a credential. Detected **PII** is *masked in place* by `redact_pii()`, so an email inside a paragraph of useful debugging evidence costs that email, not the paragraph, and the mask names the kind (`[PII:EMAIL]`) so a reviewer knows what was removed. It is wired at both redaction points: `fetch_sessions._summarize_messages()` (what reaches the analyzer) and `evaluate.redact_secrets()` (provider prompts and on-disk proposals).
  **Monetary amounts are deliberately NOT masked.** The scope is *identifiers*, not *amounts*: an email, phone, Luhn-valid card, IBAN or RUC/DNI identifies a person, while a bare amount identifies nobody and is exactly the evidence the analyzer needs to reason about finance/budgeting-style skills. Measured over a real message corpus, on what actually crosses the boundary: identifier patterns (email/phone/card/RUC) were fully masked while money amounts passed through untouched, confirming the redaction targets identifiers specifically rather than over-redacting numeric content generally. Card detection requires a leading digit of 3–6 *and* a Luhn check, which is what keeps hash-like 16-digit runs (`commit 1234567890123456`) out.
- **`scripts/skill-evolution-fetch.sh`** — the bash wrapper a cron job invokes, so the cron entry stays one stable command. It resolves `fetch_sessions.py` **relative to its own location, following symlinks**, and passes arguments through. Two earlier strategies were broken and are worth not reinventing: probing a hardcoded skills-home path with a fallback that doesn't exist in this repo's layout (always failed), and `if git rev-parse --show-toplevel; then` to find the project root (that command's stdout isn't captured, so the repo path was printed to stdout and prepended a stray non-JSON line to the NDJSON stream `analyze.py` consumes). **Nothing but `fetch_sessions.py` may write to stdout.** Symlink-following matters whenever the deployed entry point is itself a symlink into this repo, which is a common way to install it.
- **`scripts/state.py`** — small JSON-file-backed tracker (`~/.hermes/skill_evolution_state.json`) for which session IDs have already been analyzed. `fetch_sessions.py` duplicates this logic inline rather than importing it — if you change one, check the other (`tests/test_state_schema_compat.py` parametrizes over both modules to catch drift). **Two on-disk shapes exist and both must be read.** This repo and the deployed `SKILL.md` document `{"processed_sessions": [...], "last_analyzed_at": ..., "version": 1}`, but the file actually deployed is a flat `{session_id: iso_timestamp}` map (101 entries). Reading only the documented shape returned `[]` every run, which silently disabled dedup entirely — `--dry-run --lookback-hours 168` reported 20 unprocessed sessions that had all in fact been processed. `mark_processed()` now **preserves whichever layout the file already has** (writing the documented shape only for a fresh file) because rewriting the flat map would destroy timestamps another producer maintains. Don't "normalize" that file without confirming who else writes it. **Redirecting state:** `SKILL_EVOLUTION_STATE_FILE` overrides the path, resolved at call time by `get_state_file()` — present in *both* modules, since they duplicate deliberately. Its fallback is the module global rather than the literal path, so monkeypatching `fetch_sessions.STATE_FILE` (what `tests/test_state_schema_compat.py` does) still works; a blank value is ignored rather than redirecting to `""`. Note that once dedup works, `fetch_sessions.py --dry-run` legitimately returns **0 sessions** because every real session is already marked processed, which looks like a failure but is correct. **Host-prefixed since the host-agnostic read side (U2):** `load_processed()`/`mark_processed()`/`prune_processed()` all take a `host` argument (defaulting to `host.resolve_host()`, duplicated locally in both modules rather than imported — `host.py` imports `fetch_sessions.py`, so importing back would cycle). New entries on the flat-map shape are written `"<host>:<session_id>"`; a legacy un-prefixed key is read as belonging to `"hermes"` only, which is what lets the 101 real pre-host entries keep working with zero migration. `prune_processed()` only ever touches the resolved host's own entries — a different host's rows, prefixed or legacy, pass through untouched, and `keep_ids` is matched against each entry's *bare* id rather than its on-disk key for the same reason. This is not yet expressed as a `HostAdapter` method (`scripts/host.py` doesn't have a fourth "processed state" method) — see that file's section above for the three methods it does have.
  **Pruning: `prune_processed(retention=None, keep_ids=None)`, also duplicated in both modules** (`tests/test_state_pruning.py` parametrizes over both, same drift-protection convention), gated on `SKILL_EVOLUTION_STATE_RETENTION` (same bare-int/`"Nd"`/`"Nmo"` syntax as `evaluate.py`'s retention parsing, duplicated locally rather than imported — `state.py`/`fetch_sessions.py` sit below `evaluate.py` in this repo's import direction and must not invert it). **Real pruning applies only to the flat `{id: timestamp}` shape** — the documented shape carries no per-session timestamp, and its list order isn't chronological either (`fetch_sessions()` builds it from a Python `set` union, whose iteration order is hash-based, not insertion-based), so there is no temporal signal to prune by there without a schema change, which this deliberately does not attempt. Configuring retention against a documented-shape file is a no-op with a stderr warning on every call, not a silent gap.
  **`keep_ids` is structurally required, not a defensive nicety.** `mark_processed()` computes `now()` once per call and stamps every session in that batch with the identical value, so "keep the N most recent by timestamp" has no defined tiebreak among everything written in the same run — without a floor, a small retention count could drop most of the very batch a run just wrote, making those sessions look unprocessed again on the very next run. `fetch_sessions()` (the only real orchestrator of `prune_processed()` for the Hermes path) computes `just_processed = {s["session_id"] for s in results}` (distinct from `new_processed`, the full accumulated union) and calls `prune_processed(keep_ids=just_processed)` immediately after `mark_processed(new_processed)`, structurally after the `--dry-run` early return so a dry run never prunes either.
  **`keep_ids` closes the same-run hole, not the cross-run one.** If `SKILL_EVOLUTION_STATE_RETENTION` is configured tighter than the effective `--lookback-hours`, a session's state entry can be pruned while it's still inside a later run's query window — it no longer looks processed, gets re-fetched, re-marked, and re-analyzed, indefinitely, every time this recurs. There's no clean structural fix without coupling retention to lookback, so the mitigation is a warning, not a fix: `_warn_if_state_retention_below_lookback()` fires on stderr every run where age-based retention converts to fewer hours than `--lookback-hours` — count-based retention has no clean unit match to compare, and has its own related limitation (`keep_ids` overriding the limit means a busy day's batch can floor the file above the configured count, so age-based retention is the better fit for this file specifically).
  Manual escape hatch: `fetch_sessions.py --prune-state`, structurally like `evaluate.py --prune` (before/after count, early return) but reporting to **stderr, not stdout** — unlike `evaluate.py`, this script's stdout is the NDJSON channel other tooling parses.
- **`scripts/skill_index.py`** — scans `~/.hermes/skills/<category>/<skill>/SKILL.md`, hand-parses the YAML frontmatter (no PyYAML dependency — stdlib only) via `parse_name_description_frontmatter()` for `name`/`description`, and emits a JSON index the analyzer prompt uses as "what skills currently exist." `evaluate.py`'s `DeterministicEvaluator` reuses this same parser to validate a proposal's frontmatter. `scan_skills()` **skips dot-prefixed category directories**: a live tree can hold `.archive/` (retired skills), `.curator_backups/` (timestamped snapshots) and `.hub/` (lockfiles, audit log, quarantine). Without that filter the index reported retired skills as currently existing, and an archived twin of a live skill (the same skill name present in both `.archive/` and a live category) made `optimize_skill.py --skill` reject the live skill as an ambiguous duplicate.
- **`scripts/analyze.py`** — pure formatter: NDJSON sessions on stdin → markdown-ish text on stdout for LLM consumption. No analysis logic lives here; the actual judgment is made by whatever LLM/agent reads this text plus the analyzer prompt. Its only reusable function is `format_session(session) -> str` (one session at a time) — there is no batch formatter, so an in-process caller joins the per-session strings itself; `main()` is the stdin/stdout wrapper. Formatted session text plus the analyzer instructions plus the skill index can add up to a sizeable prompt, so provider context limits are a real design constraint on how many sessions one analysis pass can carry — `--lookback-hours`/`max_sessions` exist to bound that.
- **`scripts/proposal.py`** — the schema and I/O layer:
  - `SkillEvolutionProposal` dataclass + `ProposalType`/`ProposalStatus` enums define the proposal shape.
  - `render()`/`load_proposal()` do round-trip markdown+YAML-frontmatter serialization using a **hand-rolled frontmatter parser** (`_parse_frontmatter`, no PyYAML) — it's whitespace-sensitive (2-space list items, 4-space nested change blocks). Be careful preserving exact indentation if you edit proposal rendering.
  - `get_proposals_dir()` defaults to `./proposals/` at the repo root (overridable via `SKILL_EVOLUTION_PROPOSALS_DIR`) — see "Git tracking" above.
  - `list_proposals()` skips files it cannot parse but **names each one and its reason on stderr**. It used to swallow `ValueError`/`IOError` entirely, and that fired for real: a model wrote `status: already_covered` (not in `ProposalStatus`), `load_proposal()` raised, and `--list`/`--retroactive` reported 7 of 8 files with no hint the eighth existed. Any model-authored field can do this, since the analyzer writes these files. The parse stays strict — the fix was making the loss audible, not accepting unknown values — because a proposal is a unit of work awaiting a *human decision*, and a file that errors gets looked at while a file that vanishes does not.
  - `save_proposal()` pipes the rendered markdown through `evaluate.redact_secrets()` before writing to disk, so a proposal can never persist a secret that leaked into session evidence or a proposed change.
  - `apply_proposal()` validates `status == proposed` and `confidence >= min_confidence`, resolves the active host adapter, and refuses (before the evaluation gate runs at all) if that host can't write skills or if the proposal can't be expressed as a mutation — a `create_new` with a missing/placeholder `body` is a drafting failure, not a valid mutation (a `create_new` proposal has, in practice, shipped a literal placeholder string instead of real content, which Hermes's `skill_manage` rejects for `create`). Then it runs the evaluation gate (`evaluate.evaluate_and_record()`) *before* mutating anything. If evaluation raises or the gate fails, it returns `can_apply: False` (with `evaluation_results` and, on a raised exception, `evaluation_error`) and leaves the proposal's status untouched. Only once the gate passes does it build a normalized mutation plan and hand it to `adapter.apply_skill_write(plan)`: the Hermes adapter returns `skill_manage` instruction dicts (the `create` instruction carries `name` + `body`, fixing the placeholder-body bug above) that the agent runs later, while the Claude Code adapter writes skill files directly and immediately. `apply_proposal()` flips `status` to `applied` only after the adapter reports `can_apply: True`, and the create_new history migration runs only then too — a refused write must never leave an orphaned migration. It does not call `skill_manage` itself — this script has no Hermes runtime dependency.
  - `proposal.py` imports `evaluate` at module scope; `evaluate.py` avoids importing `proposal` at module scope (it imports it locally inside `_select_retroactive_proposals`) specifically to break that cycle — keep that direction if you touch either file's imports.
- **`scripts/evaluate.py`** — the evaluation/gating framework (see "Evaluation framework" below).
- **`scripts/optimize_skill.py`** — optional GEPA optimizer, gated behind `SKILL_EVOLUTION_OPTIMIZER_ENABLED=true` and the optional standalone `gepa` PyPI package (not `dspy`). `--list-candidates` prints `find_low_scoring_targets()`'s entries (history below `DEFAULT_LOW_SCORE_THRESHOLD` 0.6) read-only, with no optimization run. `--skill <name> [--iterations N]` runs `run_gepa_optimization()`: resolves the skill's installed `SKILL.md` via `skill_index.scan_skills()` as the seed candidate, fetches that skill's session history via `fetch_sessions.sessions_for_skill()` (capped at `DEFAULT_MAX_SESSIONS_FOR_SKILL`, most recent first; exits cleanly below `MIN_SESSIONS` without importing `gepa` at all), then calls `gepa.optimize_anything()` in Single-Task Search mode with an objective built by `_build_objective(seed_body, skill_name)` — which states the seed body's byte count and the allowed range, plus an instruction to preserve existing section headings. The range is the **intersection of all four limits the gate applies**, computed and intersected *before* any branching: the per-pass percentages (`SKILL_EVOLUTION_MAX_GROWTH_PCT`/`MAX_SHRINK_PCT`) against the body being replaced, the absolute per-pass deletion floor (`MAX_SHRINK_BYTES`), the absolute cap applied as the same ratchet the gate applies (`MAX_SKILL_SIZE_KB` — an oversized seed gets `upper = base`, i.e. it may not grow at all), and the cumulative percentages against `original_size_for_target()`. Each omission was the same bug in a new place: stating only the per-pass window over-advertised headroom for a skill partway toward its cumulative ceiling, and ignoring the cap advertised a ~124KB upper bound for the 103,656B skill that the gate rejects outright. An earlier version returned early from inside the cumulative block, so a window emptied by a *different* constraint went undetected — hence one intersection, then one branch. Notes explaining *why* the window is narrow are emitted in precedence order (ratchet → cumulative → byte floor); the byte-floor note is not cosmetic, since without it an oversized skill is handed a 2KB-wide window on a 100KB body with no stated reason. If the intersection is empty the objective says so, and **which advice it gives depends on which side is violated**: over the growth ceiling → reword within the current length (`base` is still admissible on the floor side); below the shrink floor → no revision of any length passes, so it says the target needs human attention rather than advising a same-length rewrite that would also be rejected. That second branch fixes advice that was silently wrong before. Without that budget the objective named only the task, so the reflection LM optimized purely for score and only met the size limits after the run: one real run spent its entire metric-call budget converging on a +121.8% candidate that was inadmissible from the first byte. It is a **soft** defence (a prompt the model may ignore) that does not replace the deterministic check — its value is not wasting budget on candidates born dead, scored by `score_candidate()` (built on `evaluate.py`'s provider/redaction layer, reusing `LLMJudgeEvaluator.parse_and_score()`) with `gepa`'s own reflective-mutation step also routed through that same provider layer via `_reflection_lm_adapter()` rather than `gepa`'s litellm/OpenAI default. `draft_proposal_from_gepa_result()` builds an `improve_existing` proposal from the winning candidate (rejecting structurally broken candidates before drafting), with a rationale reporting the full explored-candidate frontier. It never calls `apply_proposal()` — optimizer-drafted proposals re-enter the same human-review/auto-apply gate as any analyst-drafted one. The score-provider's judge `feedback` (which `gepa` embeds verbatim into its own internal reflection prompt) is produced under an explicit instruction not to quote session content verbatim and is wrapped in the same untrusted-content framing used elsewhere — narrowing, but not eliminating, a paraphrased-injection residual risk (`gepa`'s own internal prompt construction is out of reach). The three tuning constants `MIN_SESSIONS` (3), `DEFAULT_MAX_METRIC_CALLS` (**8**), and `DEFAULT_MAX_SESSIONS_FOR_SKILL` (20) are overridable via `SKILL_EVOLUTION_OPTIMIZER_MIN_SESSIONS`, `SKILL_EVOLUTION_OPTIMIZER_MAX_METRIC_CALLS`, and `SKILL_EVOLUTION_OPTIMIZER_MAX_SESSIONS_FOR_SKILL`; an explicit `--iterations` always overrides `DEFAULT_MAX_METRIC_CALLS` regardless. **`optimize_skill.py` has been run against the real `gepa==0.1.4` package and real session history**, which is what these defaults are tuned from: a real optimizer run against a skill with an explicit `--iterations 4` surfaced both a real score gain and a pathological +121.8%-growth failure inside those 4 calls, and took over two minutes wall-clock against a hosted provider (30s+/metric-call) — at the old default of 20 that's a 10+ minute unattended foreground call for what is, by design, a manual/occasional invocation (never cron-triggered). 8 keeps 2x headroom over what was empirically sufficient while bounding the worst case instead of doubling it on no further evidence. `MIN_SESSIONS` and `DEFAULT_MAX_SESSIONS_FOR_SKILL` were left unchanged: real runs never tested `MIN_SESSIONS`'s boundary, and both 8- and 20-session trainsets ran fine, so neither constant has evidence pointing anywhere in particular.
- **The analyzer prompt** (maintained by the operator outside this repo, not shipped here — see SKILL.md) is the "business logic" of the analysis half of the system: it defines the analysis criteria (skill coverage, cross-session patterns, specificity, confidence scoring 0.5–1.0), the output format matching `proposal.py`'s schema, and rules like skipping already-processed sessions and never calling the host's skill-mutation tool itself. The Python scripts in this repo are plumbing around that prompt, not a replacement for it. A structural gotcha worth knowing if you're writing your own: `create_new` proposals have no `target_skill`, so `target_key_for_proposal()` falls back to `proposal:<uuid>`, giving each its own single-entry lineage unless the target key is derived from `proposed_changes` instead (which is what this repo's `RegressionEvaluator` does — see below) — otherwise regression tracking is effectively `improve_existing`-only. The evaluation half (`evaluate.py`) is separate: it scores whatever content a proposal already contains, regardless of which agent or prompt produced it.

### Evaluation framework (`scripts/evaluate.py`)

Introduced to gate auto-apply on measured quality rather than just the proposal's self-reported `confidence`.

- **`EvalResult`** (`score`, `feedback`, `passed`, `evaluator_name`) is the shape every evaluator returns. **`Evaluator`** is the ABC; concrete evaluators register themselves into the module-level `REGISTRY` dict via `register_evaluator()` at import time — there's no dynamic discovery/plugin scanning.
- **Which evaluators run** is controlled by `SKILL_EVOLUTION_EVALUATORS` (comma-separated; default `"deterministic,llm_judge,regression"`, resolved by `get_enabled_evaluators()`). An unknown name raises `ValueError` rather than silently skipping.
- **Five evaluators**, all registered at import:
  - `DeterministicEvaluator` — binary size (`SKILL_EVOLUTION_MAX_SKILL_SIZE_KB`, default 15KB), growth-vs-baseline (`SKILL_EVOLUTION_MAX_GROWTH_PCT`, default 20%), **shrink-vs-baseline** (`SKILL_EVOLUTION_MAX_SHRINK_PCT`, default **15%** — deliberately tighter than the growth cap: an over-long skill is bounded by the absolute 15KB check and merely costs context, whereas deletion silently removes guidance, and 15% keeps one pass's bite to ~1.4KB of a median 9.7KB skill instead of ~1.9KB), and YAML-frontmatter-structure checks. The shrink floor is not symmetry for its own sake: the LLM judge's `conciseness` criterion rewards deletion, and a real GEPA run cut a skill body 10258B → 3041B (−70.4%), lost 17 of 29 headings including its `Red Flags`/`Common Rationalizations`/`Anti-Patterns` sections, and scored **higher** for it (0.85 → 0.90). A cap without a floor guards against bloat but not against content destruction. Both per-pass deltas are gated on `context["baseline_size"]` — see `evaluate_skill_text()` below.
    **Plus two cumulative limits** measured against `context["original_size"]` — where the target *started*, not the body it immediately replaces: `SKILL_EVOLUTION_MAX_CUMULATIVE_GROWTH_PCT` (default 50%) and `SKILL_EVOLUTION_MAX_CUMULATIVE_SHRINK_PCT` (default 30%). The per-pass limits reset their reference every pass, so they compound: at 20% per pass, four accepted passes halve a skill while each one looks compliant, and `RegressionEvaluator` can't catch it because every deletion *raises* the judge's `conciseness` score, so each pass legitimately outscores the last. The erosion is self-reinforcing. Verified: simulating repeated exact-floor cuts, the gate halts at pass 3 with 62% of the original retained instead of running to zero. Note the **cumulative** limit governs the final retained fraction, not the per-pass one — retuning the per-pass floor changes how many passes it takes to get there and how big each bite is, not where erosion stops. Cumulative allowances are deliberately wider than per-pass ones — one pass may move 20%, but total drift stays bounded.
    **The absolute cap is a ratchet, not a flat ceiling.** It fails only when the candidate is over the cap **and** larger than the baseline it replaces. A real installed-skill census found roughly 15% of skills already exceeding the 15KB cap (median size well under it, but with real outliers over 100KB) — a spot-check claiming far fewer was wrong; a full census is what should be trusted. Under a flat ceiling every body proposal for an over-cap skill failed on size *regardless of direction*, so a proposal shrinking one toward compliance was rejected with the identical message as one growing it: the gate could not tell improvement from worsening. `create_new` carries no baseline, so it keeps the strict cap and a skill is never born oversized — that exemption is a consequence of the rule ("a change that replaces existing text may not worsen; a change that creates text faces the hard cap"), not a proposal-type check to add later. Note this makes oversized skills *improvable*, not *downsizable*: at 2048B/pass, walking a 100KB+ skill to compliance would take dozens of approved passes, so a very large skill still needs a human to split it.
    **Plus an absolute per-pass deletion floor**, `SKILL_EVOLUTION_MAX_SHRINK_BYTES` (default **2048**, `0` disables), applied as a second sequential check after the percentage floor — the *stricter* of the two binds. A percentage scales with the skill, so it is weakest exactly where a deletion does most damage: 15% of the median skill is ~1.5KB, but 15% of the 103,656B one is 15,548B in a single pass. The default comes from the crossover, not from taste: `2048/0.15 = 13,653B`, just under the 15,360B cap, so the byte floor is inert for essentially every within-cap skill *by construction* and operative precisely on the oversized tail the ratchet unblocks (it binds 35 of 143). The two checks are sequential rather than one `min()` so the feedback names whichever limit actually bound. **These two changes are causally coupled and must not be separated:** the 15KB single-pass deletion was previously *masked* (the shrunk candidate was still over the cap and failed there first), so ratcheting the cap without the byte floor is a net regression in deletion safety.
    Growth needs no byte counterpart: per-pass absolute growth is bounded at `cap × g/(1+g)` = 2,560B for a within-cap skill, and at 0 for an oversized one under the ratchet. **That invariant scales with the cap** — raising `SKILL_EVOLUTION_MAX_SKILL_SIZE_KB` rescales it, so revisit if the cap moves.
    The frontmatter check only runs when `context["content_kind"] == "body"` — a bare description change or a merge/deprecate summary+rationale fallback is plain prose and is exempt.
  - `LLMJudgeEvaluator` — rubric-based (`correctness`, `procedure_following`, `conciseness`, each 0–1, threshold `SKILL_EVOLUTION_LLM_JUDGE_THRESHOLD` default 0.7). Frames evaluated content as untrusted data between a **per-call random hex boundary** (not a static tag) specifically so injected content can't forge the closing marker and escape the block; fails closed (`self._fail(...)`) on any provider error or malformed/out-of-range JSON response rather than guessing.
  - **`_post_json()` sets a descriptive `User-Agent`** (`USER_AGENT = "skill-evolution/0.1"`), overridable per call via the `headers` argument. This is load-bearing, not cosmetic: urllib's default `Python-urllib/3.12` is blanket-blocked by OpenCode Zen's gateway, which answers **403 for it and 200 for the byte-identical request** under any descriptive UA. If a provider starts returning 403 while `curl` succeeds, check the UA before suspecting the key. **Verified harmless against a real Ollama server** (`ollama==0.30.10`): its endpoint doesn't gate on User-Agent at all, so the fix changes nothing there — confirmed via a full round trip (`LLMJudgeEvaluator.evaluate()` → `_call_ollama()` → `_post_json()`) scoring a real installed skill. That pass also surfaced a real, previously unknown friction point unrelated to the UA itself: a reasoning-capable local model (`gemma4`) used 54–56s of the 60s default `call_provider` timeout on one judge call, and a heavier one (`qwen3.5`) timed out outright — local models burn a large share of their output on chain-of-thought before the JSON, in a way a hosted API typically doesn't. That friction is now tunable via `SKILL_EVOLUTION_PROVIDER_TIMEOUT` (see the provider adapter layer below). Still unverified against Claude (no `ANTHROPIC_API_KEY` in this environment).
  - **OpenCode Zen gotcha**: two catalogues exist and they are *not* interchangeable. Zen (`https://opencode.ai/zen/v1`, the `_call_opencode` default) carries `big-pickle` and the `*-free` variants (`deepseek-v4-flash-free`, `mimo-v2.5-free`, …); the Go subscription tier (`https://opencode.ai/zen/go/v1`) is a different, smaller list **without** `big-pickle` and whose DeepSeek id has no `-free` suffix. Point `SKILL_EVOLUTION_OPENCODE_BASE_URL` at the tier matching the model id you set. Both are OpenAI-compatible (`/chat/completions`, bearer auth), so the caller mirrors the Ollama one; unlike Ollama it reaches a third party, so a missing `OPENCODE_API_KEY` fails closed with an actionable message instead of sending an unauthenticated request. Model lists are public — `curl -s https://opencode.ai/zen/v1/models` needs no key.
  - `RegressionEvaluator` — compares a target's new aggregate score against the *last entry that actually passed* in that target's history (not just the last entry) so a failed attempt can never lower the bar a later one is compared against. No prior passing entry → auto-passes ("no baseline yet"). Works for both `improve_existing` and `create_new` proposals: `target_key_for_proposal()` extracts the skill name from `proposed_changes` for `create_new`, and `migrate_proposal_history()` rewrites `proposal:<id>` entries to `skill:<name>` after creation, connecting the skill's full evaluation lineage.
  - `HumanReviewEvaluator` — opt-in interactive gate (P2-4). Not in `DEFAULT_EVALUATORS`; enable via `SKILL_EVOLUTION_EVALUATORS=...,human_review`. Requires a TTY (stdin/stdout `isatty()`); fails closed without one. Binary prompt: y/yes → approve, n/no → reject, empty/junk re-prompts up to 3 times then fail-closed; EOF/KeyboardInterrupt fail-closed. Runs **last** in `run_evaluators()` after automatic + regression, receives `prior_results` context. Approve/reject return `new_score` (the automatic aggregate), mirroring `RegressionEvaluator` — a hardcoded 1.0 would inflate `eval_history.jsonl` gate entries that `RegressionEvaluator` baselines against. Fail-closed returns 0.0. Optional rejection reason captured (`"no reason given"` if blank). Feedback: `"human approved: no note"` / `"human rejected: <reason>"`. `scripts/skill_quality.py` strips `human_review` from the env var at startup (warns on stderr) to prevent per-skill interactive prompts in the cron wrapper.
  - `EmbeddingSimilarityEvaluator` — opt-in semantic similarity checks (P2-5). Not in `DEFAULT_EVALUATORS`; enable via `SKILL_EVOLUTION_EVALUATORS=...,embedding_similarity`. Requires the `embeddings` extra (`pip install -e ".[embeddings]"`), which installs `fastembed` (~50MB, ONNX-based). Three modes selected automatically by context: **duplicate_detection** (flags content too similar to existing skills, similarity > threshold), **drift_detection** (flags content that drifted too far from baseline, similarity < threshold), **grounding_check** (flags content not grounded in source sessions, average similarity < threshold). Thresholds configurable via `SKILL_EVOLUTION_EMBEDDING_DUPLICATE_THRESHOLD` (0.85), `SKILL_EVOLUTION_EMBEDDING_DRIFT_THRESHOLD` (0.70), `SKILL_EVOLUTION_EMBEDDING_GROUNDING_THRESHOLD` (0.60). Backend architecture is extensible via `SKILL_EVOLUTION_EMBEDDING_BACKEND` (default `fastembed`); other backends (`ollama`, `openai`, `llama_cpp`) are stubbed with implementation examples in their docstrings.
    **Registration gotcha**: unlike the other four evaluators (registered at `evaluate.py` module-import time), `embedding_similarity`'s registration only happens inside `evaluate.py`'s `if __name__ == "__main__":` block (`scripts/evaluate.py:2116-2123`) — nothing else in the pipeline imports it. Running `python3 scripts/evaluate.py ...` picks it up automatically; importing `evaluate` as a library (which is what `proposal.py`'s `apply_proposal()` and `skill_quality.py` do) does **not**, so setting `SKILL_EVOLUTION_EVALUATORS=...,embedding_similarity` in that path raises "Unknown evaluator" unless something upstream already did `import embedding_similarity` first.
- **Gate combination**: `run_evaluators()` runs in three phases — (1) automatic evaluators (everything except `regression` and `human_review`), (2) `regression` fed the mean of automatic scores as `new_score` (human score never enters the regression baseline), (3) `human_review` last with `prior_results` context (automatic + regression verdicts). `combine_gate(results, strictness)` then combines per `SKILL_EVOLUTION_GATE_STRICTNESS` (or a per-proposal-type override, `SKILL_EVOLUTION_GATE_STRICTNESS_<TYPE>`, or a per-target override `SKILL_EVOLUTION_GATE_STRICTNESS_<TARGET>`, resolved by `resolve_gate_strictness()` — target beats type beats global): `"strict"` (default) requires every evaluator to pass; `"majority"` requires more than half. Zero configured evaluators never blocks.
- **Multi-target gate (P2-1)**: `evaluate_and_record()` runs every target in `resolve_gate_targets()` (`SKILL_EVOLUTION_GATE_TARGETS`, default `skill,proposal`; unknown names raise `ValueError`; empty falls back to default) through the same registry and appends one combined `gate` entry per gating target — `skill:<name>`, `proposal:<uuid>`, and per-session `tool_calls:<sid>`/`analyzer_prompt:<sid>` — each combined under its own strictness, then ANDs the per-target decisions. Targets with no data don't block: a proposal without `session_ids` skips the session-based targets. A provider fault is flagged per-target on the entry it affected. The `kind` history field tags each entry's lineage (`skill_text`/`proposal`/`tool_calls`/`analyzer_prompt`); `migrate_proposal_history()` uses it to leave proposal-document entries under `proposal:<id>` instead of folding them into the skill lineage.
- **Provider adapter layer** (`call_provider()`) is stdlib-only `urllib` HTTP to Claude (`ANTHROPIC_API_KEY` + `SKILL_EVOLUTION_CLAUDE_MODEL`), Ollama/llama.cpp (`SKILL_EVOLUTION_OLLAMA_BASE_URL`/`SKILL_EVOLUTION_OLLAMA_MODEL`), OpenCode Zen (`OPENCODE_API_KEY` + `SKILL_EVOLUTION_OPENCODE_BASE_URL`/`SKILL_EVOLUTION_OPENCODE_MODEL`), OpenAI (`OPENAI_API_KEY` + `SKILL_EVOLUTION_OPENAI_BASE_URL`/`SKILL_EVOLUTION_OPENAI_MODEL`, default `https://api.openai.com/v1` / `gpt-4o`), or Gemini (`GEMINI_API_KEY` + `SKILL_EVOLUTION_GEMINI_BASE_URL`/`SKILL_EVOLUTION_GEMINI_MODEL`, default `https://generativelanguage.googleapis.com` / `gemini-2.0-flash`), selected by `SKILL_EVOLUTION_PROVIDER` (default `claude`) with an optional per-evaluator override `SKILL_EVOLUTION_<EVALUATOR_NAME>_PROVIDER`. OpenAI and Gemini follow the same `_post_json()` pattern as the other three branches; Gemini is the one shape outlier — it hits `v1beta/models/{model}:generateContent` with a `contents[].parts[].text` request body and `candidates[].content.parts[].text` response, and authenticates via the `x-goog-api-key` header rather than a bearer token, keeping the key out of URLs. Both fail closed on a missing key or a malformed response shape, matching OpenCode's posture. `GEMINI_API_KEY` is also in `fetch_sessions.SECRET_PATTERNS`, so it's caught by the redaction layer below like `OPENAI_API_KEY`/`ANTHROPIC_API_KEY` already were. The HTTP timeout is **one generic knob for every provider** — `SKILL_EVOLUTION_PROVIDER_TIMEOUT` (seconds, default 60), resolved by `resolve_provider_timeout()` inside `call_provider()`; an explicit `timeout=` argument always wins. It is deliberately not per-provider (no `SKILL_EVOLUTION_OLLAMA_TIMEOUT`): the latency it exists for is a property of the model, not the adapter — local reasoning models spend most of their output on chain-of-thought before the JSON (a real `gemma4` judge call took 54–56s of the 60s default, and `qwen3.5` timed out outright). Every prompt is run through `redact_secrets()` (reusing `fetch_sessions.py`'s `contains_secret()`/`SECRET_PATTERNS`, whole-line replacement) before it ever leaves the machine, independent of the redaction `save_proposal()` already does before writing to disk — R18/R19 in the plan doc treat "sent to a provider" and "written to disk" as two separate redaction points, not one.
- **History**: entries now also carry `content_size` and `baseline_size` (utf-8 bytes), which is what makes cumulative drift detectable — `original_size_for_target()` reads the *earliest* `baseline_size` for a target (falling back to the earliest `content_size`) so a candidate can be compared against where the skill began. Both fields are optional, so pre-existing history written before size recording simply yields `None` and leaves the cumulative check inert. `evaluate_and_record()` and `evaluate_skill_text()` both derive the evaluated text via the shared `_extract_evaluated_content()` helper, so the sizes recorded are measured over exactly the text that was scored.
  `append_history()`/`read_history()` read/write one shared append-only JSONL file (`get_history_path()`, default `./eval_history.jsonl` at the repo root, overridable via `SKILL_EVOLUTION_HISTORY_PATH` — see "Git tracking" above), keyed per-target (`target_key_for_proposal()` → `"skill:<name>"` or `"proposal:<id>"` fallback; for `create_new` proposals it extracts the skill name from `proposed_changes` before falling back, and `migrate_proposal_history()` rewrites history from `proposal:<id>` to `skill:<name>` after the skill is created). `prune_history()` enforces `SKILL_EVOLUTION_HISTORY_RETENTION` (a bare int = max versions, `"90d"`/`"6mo"` = age-based), and now **archives rather than deletes**: dropped entries are appended (not overwritten) to a sibling file before the primary is rewritten — `get_history_archive_path()`, default derived from the primary path's basename (`eval_history.jsonl` → `eval_history.archive.jsonl`), overridable independently via `SKILL_EVOLUTION_HISTORY_ARCHIVE_PATH`. Writing the archive *before* rewriting the primary matters: a crash mid-prune then loses nothing, where the reverse order could lose data permanently. R22 in the eval plan permits either "archived or compacted" — archiving was chosen because it already satisfies R22's actual goal (a bounded primary file) without a lossy "compacted summary" record needing to independently satisfy three different readers.
  **The survivor rule per target is up to three always-kept anchors, not one** — the most recent entry (needed by `optimize_skill.find_low_scoring_targets()`), the most recent **passing** entry, and the **earliest** entry, deduplicated when they coincide. The retained count per target can therefore exceed the configured limit by up to two entries; that's intentional. Two real correctness holes motivated the extra two anchors: `RegressionEvaluator` baselines against the last entry with `passed == True`, not just the last entry — the old "keep only the most recent" rule could prune a passing baseline while newer failing retries survived, silently falling back to "no baseline yet" for exactly the targets that most need regression coverage; and `original_size_for_target()` reads the *earliest* entry as its cumulative-drift baseline — without protecting it, trimming a target's oldest entries would silently shift that baseline forward, forgiving prior drift. Both are closed by always retaining the anchor entry itself, rather than giving those two functions a fallback read into the archive file — same safety guarantee, no second data-access path to build and maintain.
  **Pruning now runs automatically, not just via `--prune`.** `evaluate_and_record()` takes `auto_prune: bool = True` and calls `prune_history()` right after its `append_history()` call — a no-op by default, since `prune_history()` itself early-returns unless `SKILL_EVOLUTION_HISTORY_RETENTION` is configured. The same env var now controls both *whether* pruning happens and that it happens on every real evaluation from then on; `apply_proposal()` calls `evaluate_and_record(proposal)` with no extra kwargs, so it gets this for free. `retroactive_reevaluate()`'s loop passes `auto_prune=False` and calls `prune_history()` once after the whole batch instead — `prune_history()` rewrites the *shared* history file (every target, not just the one being re-evaluated), so pruning after every append in an N-proposal batch would mean N full-file passes for a result identical to pruning once. `--prune` remains available for one-off backfill of pre-existing bloat, and its report now includes the archived count.
- **`evaluate_skill_text()`** extracts the body/description change and builds the evaluator context. It passes `content_kind` **and `baseline_size`**, measured the way `DeterministicEvaluator` measures the candidate. That wiring is load-bearing: the growth-vs-baseline guard is `if baseline_size:`-gated, so omitting the key silently disables it — a proposal could balloon a skill +343% and still report "all deterministic checks passed." `create_new` has no baseline, so the key stays absent and the guard stays inert rather than dividing by a zero baseline.
  **`baseline_size` comes from the installed `SKILL.md`, not from the change's `old_value`** (`_resolve_baseline()` → `installed_skill_body()`, body changes only). `old_value` is written by the analyzer LLM — the analyzer prompt asks it to emit `old_value: <current value>` and `proposal.py` persists it verbatim — so it is proposal-supplied input, not an observation. That became load-bearing when the cap turned into a ratchet: the cap now asks "is this larger than what it replaces?", so an inflated `old_value` would raise the very ceiling it is checked against, and a proposal claiming a 951KB baseline could ship a 950KB body past a 15KB cap while every percentage check reported a rounding-error delta. This is not mainly an adversarial concern — a duplicated or truncated transcription is an ordinary LLM failure, and `--retroactive` re-scores proposals whose `old_value` may no longer describe anything on disk. `optimize_skill._resolve_baseline_size_bytes()` already applied this distrust at draft time; this is the same discipline at the gate. Body changes only (for a description change the installed *file* is not what it replaces — substituting it would read as a ~−98% shrink), best-effort (unresolvable or ambiguous skill → fall back to `old_value` rather than fail a gate decision on a lookup miss), and **shared with `evaluate_and_record()`** so the size recorded in history is the size that was actually judged.
- **Three fast-follow evaluation targets** (R5) exist as sibling functions to `evaluate_skill_text()`, reusing the same registry and gate without modifying it or `apply_proposal()`'s call site:
  - `evaluate_proposal(proposal)` — scores the proposal as a document (summary, rationale, proposed_changes). Target key: `proposal:<uuid>` (always, regardless of whether the proposal has a `target_skill`). No baseline — proposals are evaluated on their own merits.
  - `evaluate_tool_calls(session_id_or_messages)` — scores tool-call quality from session messages. Input can be a session_id (queries `state.db` via `_fetch_session_messages()`) or a list of message dicts (for testing). Target key: `tool_calls:<session_id>`. Content is JSON-serialized tool_calls with result snippets capped at 200 chars each, total cap 5KB.
  - `evaluate_analyzer_prompt(session_id, proposal_id)` — scores whether the analyzer's generation step produced a grounded proposal. Reads session messages (truncated to 300 chars each, total cap 10KB) and the saved proposal markdown. Target key: `analyzer_prompt:<session_id>`.
  **Wiring posture (P2-1)**: `proposal` gates auto-apply alongside `skill` by default; `tool_calls`/`analyzer_prompt` gate only if added to `SKILL_EVOLUTION_GATE_TARGETS`. The gate decision is the AND of every gating target, each combined under its own strictness. `deterministic` is opt-in only** for these targets (via `SKILL_EVOLUTION_EVALUATORS`) — its size/growth checks are meaningless for non-skill-text content. The default set (`llm_judge,regression`) is the right one for proposal/tool-call/analyzer-prompt evaluation.
- **`evaluate_and_record()`** is the single call site both `apply_proposal()` and the retroactive path share — it runs each gating target (default `skill,proposal`), combines each under its own strictness, ANDs the per-target decisions, and appends one combined `EvalResult` (`evaluator_name="gate"`) **per gating target** to history, so both paths produce identical history semantics.
- **Retroactive/batch mode** (`--retroactive`, `retroactive_reevaluate()`) only re-evaluates already-saved proposals with `status == proposed` by default (`_select_retroactive_proposals`) — rejected/applied proposals are excluded because they're not live decisions and re-scoring them would inject entries into the same history stream the live gate regresses against; pass `include_all_statuses=True` (Python-only, no CLI flag yet) to opt in. `--target`/`--since` filter further.
- **Module-identity gotcha**: running `python3 scripts/evaluate.py` directly loads it as `__main__`; since `proposal.py` does `import evaluate` at module scope, the file registers itself under `sys.modules["evaluate"]` at the top (`if __name__ == "__main__": sys.modules.setdefault(...)`) so both names resolve to the *same* module instance (same `REGISTRY`, same classes) instead of `proposal.py` re-executing and re-registering everything as a second, distinct module.

### Proposal types and the write side (`apply_proposal()` → `adapter.apply_skill_write(plan)`)

| `type` | What it means | Hermes emits (`applied_by: agent`) | Claude Code does (`applied_by: direct`) |
|---|---|---|---|
| `improve_existing` | Patch a skill's description/body | one `patch` instruction per `proposed_changes` entry | rewrites the installed `SKILL.md` in place (exact `old_value` match, else refuse) |
| `create_new` | New skill for a recurring uncovered pattern | one `create` instruction carrying `name` + full `body` (fixed to require real content, not a placeholder) + description/category | writes a real `skills/<name>/SKILL.md` (refuses existing names) |
| `merge_skills` | Combine overlapping skills into `target_skill` | one `delete` instruction per `source_*`-prefixed change, each tagged `absorbed_into` | archives each source into `skills/.archive/<name>/` after validating the umbrella exists |
| `deprecate_skill` | Remove a stale/unused skill | one `delete` instruction | archives the skill into `skills/.archive/<name>/` |

All four types pass through the same evaluation gate (above) before the adapter is asked
to mutate anything — the gate runs once per `apply_proposal()` call regardless of `type`,
and `apply_proposal()` only flips `status` to `applied` after the adapter reports
`can_apply: True`.

## Conventions specific to this codebase

- **Prefer host-agnostic design.** Multi-agent support (Codex, Claude Code, Claude, future providers) is an explicit project goal, and the owner's stated principle is that the more agnostic the design, the better — so treat host-neutrality as the default for new work instead of a later retrofit. The Hermes coupling used to be four hardcoded points scattered one-per-module; the **read side of that is now a real abstraction, not just an isolation convention.** `scripts/host.py` defines `HostAdapter` (an ABC with three read methods — `iter_sessions(since=None)`, `iter_skills()`, `read_skill_body(skill_name)` — plus the write side: `supports_write: bool = False` and a fail-closed concrete `apply_skill_write(plan)` that returns `{"can_apply": False, "reason": ...}` unless an adapter overrides it), a plain-dict `HOST_ADAPTERS` registry populated by `register_adapter()` at import time (instances, not classes — a host adapter carries no per-call state, so nothing is gained by re-instantiating per lookup, unlike `evaluate.REGISTRY`), and `resolve_host()`/`get_adapter()` reading `SKILL_EVOLUTION_HOST` (default `hermes`) in the same explicit-arg-beats-env-var-beats-default order `evaluate.resolve_provider()` already uses — an unknown host name raises rather than silently reading the wrong tree. `HermesAdapter` reproduces today's behavior byte-for-byte by delegating to `fetch_sessions.fetch_sessions()`/`skill_index.scan_skills()`. `ClaudeCodeAdapter` (root: `SKILL_EVOLUTION_CLAUDE_CODE_HOME`, default `~/.claude`) is the second adapter, reading skills from the flat `skills/*/SKILL.md` tree (no category level — `category` is reported as the constant `"user"`) and sessions from `projects/*/*.jsonl`, one JSONL-per-session with an undocumented, versioned record format handled defensively (allowlist role mapping, best-effort timestamp/field extraction, never raising on a malformed line or record) — it routes through the same `fetch_sessions.contains_secret()`/`redact_pii()`/`_summarize_messages()` redaction chokepoint rather than reimplementing it. `evaluate.py` (`installed_skill_body()`) and `optimize_skill.py` now read skills through `host.get_adapter()` instead of calling `skill_index.scan_skills()` directly, so both are host-neutral as of this abstraction landing. **The two host-neutral contracts a new adapter must produce** are the session dict shape (`session_id`, `started_at`, `title`, `model`, `source`, `message_count`, `user_messages`, `assistant_messages`, `messages` — the last already redacted/truncated) and the skill dict shape (`name`, `category`, `description`, `path`, `size`); `read_skill_body()`'s contract is narrower still — best-effort text or `None` on no match, an ambiguous match, or a read failure, *never* raise, since callers like the evaluation gate's baseline-size lookup must degrade to whatever baseline they already had rather than fail a gate decision on an unrelated lookup. **The write side shipped with the same abstraction (2026-07-31).** `proposal.py`'s `apply_proposal()` remains the sole mutation path, but it now builds a host-neutral mutation plan and hands it to `adapter.apply_skill_write(plan)`, and it refuses (before the gate runs) on any host that can't write or any proposal that can't be expressed as a mutation — never spending a provider call on a proposal that can never apply. `HermesAdapter.supports_write = True` and its `apply_skill_write()` re-emits the legacy `skill_manage` instruction dicts byte-for-byte (returned `applied_by: agent` for the cron agent to execute later). `ClaudeCodeAdapter.supports_write = True` and its `apply_skill_write()` performs the writes directly and immediately (returned `applied_by: direct`): real `skills/<name>/SKILL.md` files via atomic temp-file + `os.replace`, deprecate/merge archive sources into `skills/.archive/<name>/`, and **symlinked skill directories are refuse-to-write** (38/41 installed Claude Code skills are symlinks into `~/.agents/skills/` — the adapter writes around them instead of following or replacing them). The Hermes adapter is instruction-emitting and the Claude Code adapter is file-writing, so they cannot share a code path; what they share is the gate, the plan shape, and the fail-closed posture. **The processed-session state seam shipped (2026-08-01).** `HostAdapter` now has three concrete state methods — `iter_processed()`, `mark_processed(session_ids)`, `prune_processed(retention=None, keep_ids=None)` — plus a `_state_file()` hook for path resolution. Each adapter owns its own state file: `HermesAdapter` inherits the default (`~/.hermes/skill_evolution_state.json`), while `ClaudeCodeAdapter` overrides `_state_file()` to return `<CLAUDE_CODE_HOME>/skill_evolution_state.json`. `SKILL_EVOLUTION_STATE_FILE` remains the universal override — when set, it redirects whichever host is active. `fetch_sessions_for_host()` and `main() --prune-state` now route through the adapter's state methods instead of calling module-level functions with `host=`. `state.py` is now imported by `host.py` at module scope (no cycle — `state.py` imports nothing local), making it a real library rather than a test-only module. The host-prefix scheme (`<host>:<id>` keys) stays for legacy-entry handling and shared-override safety. `fetch_sessions()` (the Hermes cron path) stays byte-for-byte — it's the critical deployed path and already calls the state functions with `host="hermes"` and the default path. That includes prose in prompts: the GEPA `objective` deliberately says "this agent skill", not "this Hermes Agent skill", both to survive generalization and to avoid biasing the reflection LM toward one host's conventions. The remaining write-side follow-up is the project-level `.claude/skills/` tree, which is deliberately out of scope — the read side walks one flat root (`SKILL_EVOLUTION_CLAUDE_CODE_HOME`) and "which project" is undefined at the adapter level.
- Every script is a standalone stdlib-only CLI with its own `argparse` in `main()`, importable for its core function (`fetch_sessions()`, `scan_skills()`, `apply_proposal()`, `run_evaluators()`, etc.) — keep new scripts consistent with this pattern rather than introducing a shared framework/package layout.
- No PyYAML anywhere by design (`SKILL.md` promises "no pip packages required") — frontmatter is parsed and rendered by hand in both `skill_index.py` and `proposal.py`. If a change needs richer YAML (nested structures beyond flat keys + one-level lists), it likely needs a matching parser update in `_parse_frontmatter`, not just a `render()` change.
- Auto-apply is opt-in via env vars (`SKILL_EVOLUTION_AUTO_APPLY`, `SKILL_EVOLUTION_MIN_CONFIDENCE`), read by the cron agent/prompt, not by the Python scripts themselves. The evaluation gate inside `apply_proposal()`, by contrast, always runs (it's not behind an opt-in flag) — `SKILL_EVOLUTION_EVALUATORS`/`SKILL_EVOLUTION_GATE_STRICTNESS` only change *which* evaluators run and how strictly, not whether the gate runs at all.
- `gepa` is imported lazily (`_require_gepa()` inside `optimize_skill.py`, called from `run_gepa_optimization()`, not at module import time) so the rest of the pipeline works with the dependency absent — don't move that import to module scope. **`_require_gepa()` returns the `gepa.optimize_anything` *submodule*, not the top-level `gepa` package** — that submodule is the namespace holding the `optimize_anything` function plus `GEPAConfig`/`EngineConfig`/`ReflectionConfig`. The package deliberately binds the name `optimize_anything` to the submodule (see `gepa/__init__.py`: "expose submodule; use `from gepa.optimize_anything import optimize_anything` for the function"), so reading those names off the package yields a non-callable module and three `AttributeError`s. Verified against `gepa==0.1.4`; the parameter names the call site passes (`seed_candidate`/`evaluator`/`objective`/`config`, `max_metric_calls`, `reflection_lm`) are all correct for that version.
- **Stubbed-dependency tests can't catch API drift** — every test in `tests/test_optimize_skill.py` monkeypatches `_require_gepa` to return a hand-built fake, so it asserts the code against its own assumptions. Pair that with a contract test against the real package (`tests/test_optimize_skill_gepa_contract.py`, guarded by `pytest.importorskip("gepa")` so it skips without the optional extra). The same rule applies to seams between modules: the isolated `DeterministicEvaluator` growth tests pass `baseline_size` in by hand, which is why nothing caught `evaluate_skill_text()` never supplying it — assert wiring end-to-end, not just each side of the seam.
- Tests exist now (`tests/test_*.py`, one file per evaluator/feature area, run via `pytest`; last verified 2026-08-04: **780 passed + 1 skipped** under plain `python3` (no `gepa`, no `[embeddings]` extra) — the skip is the gepa-contract module, guarded by `pytest.importorskip("gepa")`. Under a `.venv` with `gepa` but not `fastembed` it's **777 passed + 7 skipped**, the 6 extra skips being `tests/test_embedding_backends.py` cases guarded by `pytest.importorskip("fastembed")` — so the exact pass/skip split depends on which optional extras are installed, not just `gepa` alone. Treat any specific number here as a point-in-time sanity check, not a contract — the suite changes; re-run `pytest -q -rs` for the current count rather than trusting this line) — this replaces an earlier state where `pyproject.toml` configured `pytest` but no `tests/` directory existed. New evaluators or gate behavior should get a matching `tests/test_evaluate_*.py`.