Building an AI Agent Self-Evolution Pipeline: From 8 Papers to Production

Six months. 196 sessions. 29k messages. 134 MB of state.db. Buried inside: countless hard-earned lessons, the optimal solutions I landed on, repeated error patterns — but no mechanism to mine them out. This post documents how I went from 8 concurrent papers to a 6-stage pipeline that lets an Agent extract reusable skills from its own chat history.

It all started with a very practical problem.

I use Hermes Agent for everyday development. After six months, state.db had bloated to 134 MB — 196 sessions, over 29,000 messages. Buried in there were countless "how did I step into this trap again" moments: the same web_search permission issues, the same pip dependency conflicts, the same cron config syntax errors. I'd written plenty of skills manually, but it was entirely reliant on memory and inspiration.

The skills you've written are finite. The traps you've stepped in are infinite.

The question was: could the Agent discover patterns from its own chat history and distill them into skills automatically? If yes, self-evolution goes from a marketing slogan to a real engineering feedback loop.

In H1 2026, eight highly relevant papers appeared on arXiv almost simultaneously. This wasn't a coincidence — the entire field was converging on the same question: how do AI Agents learn from experience? I spent two months researching, experimenting, and shipping, and ended up with a 6-stage pipeline. This post is the complete architecture retrospective.

---

A Flood of Papers: Two Competing Technical Routes

Let's quickly run through all eight papers. They appeared nearly simultaneously, but they followed two distinctly different technical paths.

The six symbolic-route papers each had a different focus:

Trace2Skill (Alibaba, 2603.25158) was the most direct inspiration — analyze execution traces in parallel, then hierarchically merge them into unified skills. Skills learned on a 35B model transferred to a 122B model with a 57-point improvement. Cross-model generalization is the symbolic route's killer advantage.

CoEvoSkills (UIC/Columbia/UBC, 2604.01687) introduced an information-isolated verification approach: a Generator produces candidate skills, a Surrogate Verifier validates them — the verifier never sees the ground truth answer, only scores via tests. This solves a deep trust problem in the Agent domain.

SkillOS (Google Cloud + UIUC + MIT, 2605.06614) might be the one I value most. It clearly articulates the Executor-Curator separation architecture principle: Frozen Executor + RL-trained Curator. An 8B Curator outperformed Gemini-2.5-Pro zero-shot on skill management tasks — insert, update, delete. An 8B model, focused entirely on skill management quality, beats a general-purpose large model.

The conclusion is clear: managing skill quality matters more than generating skills.

The other three — SkillX (plug-and-play knowledge base), SkillClaw (cross-user trajectory aggregation), SkillOpt (bringing deep learning optimization paradigms to text space) — each had their strengths. But for my solo-dev scenario, SkillOS's architectural separation, Trace2Skill's parallel analysis, and CoEvoSkills' adversarial verification formed the strongest combination.

The two RL-route papers (SKILL0 and Skill1) were impressive — SKILL0 uses dynamic curriculum learning, progressively withdrawing skill context to achieve <0.5K tokens/step zero-shot inference; Skill1 hit 97.5% SOTA on ALFWorld. I ultimately didn't go with them.

Why I passed on the RL route?

Honestly, the RL route's results looked better than the symbolic route's on paper. Three reasons drove my choice in the opposite direction:

1. Cross-model portability: I've swapped backbone models 3 times (GPT → Claude → DeepSeek). RL-trained skills are baked into model weights — swap the model and you start over. Symbolic-route skills are Markdown files; a model swap is just a re-read-and-understand problem.

2. Auditability: I need to know "which session did this skill come from, and what pattern was it based on?" RL-trained weights are a black box.

3. Infrastructure cost: RL training needs stable, large-scale compute. The symbolic route only needs LLM calls plus file operations.

That was my first critical decision: go symbolic, skip RL.

---

Six-Stage Pipeline: From Trajectory to Skill

With the direction set, it was time to build. The pipeline went through three iterations before settling into its final 6-stage form.

Step 1: Harvest — Extract structured session trajectories from state.db
  → Sanitize, classify signals (real_error / harmless_warning / permission_prompt)
  → Output: trajectory JSON
Step 2: Mine — Parallel trajectory analysis, cluster patterns
  → parallel-miner.py: 9 parallel groups
  → Analyze: tool errors, repeated patterns, success patterns
  → Output: clustered pattern candidates
Step 3: Distill — Hierarchical merging
  → hierarchical-distiller.py
  → Guardrail filtering + prevalence scoring + merge similar patterns
  → Output: high-quality skill drafts
Step 4: Curate — Validate and refine
  → surrogate-verifier.py (CoEvoSkills-style 6-dimension verification)
  → review-gate.py safety gating
  → Output: ready-to-use skills
Step 5: Feedback — Closed-loop validation
  → validation-curator.py
  → Analyze real_error ratio / outcome distribution / validation pass rate
  → Auto-tune RECURRENCE_THRESHOLD
Step 6: Dual-Form — Dual-form patterns
  → skill pattern → SKILL.md
  → subagent pattern → Pi agent config
  → Output: 18 proposals (15 skill + 3 subagent)

Let me break this down by causality, not just feature-list it.

Step 1: Harvest — Signal Classification Was the First Trap

Extracting trajectories from state.db isn't hard — read the message table from SQLite, group by session. The hard part is signal classification.

Is a message a real error or a harmless warning? A web_search returning permission_denied is a genuine error. But the Agent fixing its own typo is just normal dev-process noise. I started with a dead-simple rule — flag anything containing 'error' as red — and hit a 55% false positive rate.

I had to build three-tier signal classification: real_error (things that genuinely block a task), harmless_warning (non-blocking noise), permission_prompt (tool calls intercepted by the user). The accuracy of this classification system directly determines the quality of every downstream stage.

Classify wrong, and everything downstream is garbage in, garbage out.

Step 2: Mine — Why Batch Parallel Instead of Online Incremental?

196 sessions. If analyzed one at a time, roughly 30-40 minutes. But split them into 9 parallel groups, and it finishes in <5 minutes.

I had a choice: build an online incremental analyzer — analyze each session as it finishes, update the pattern library in real time. Sounds elegant. I passed.

Why?

Incremental mode has a brutal cold-start problem. The first 10 sessions don't have enough data for meaningful clustering — the "patterns" you see are probably random noise. And incremental state management gets gnarly fast: a pattern appears 3 times today — is that a signal? One more time tomorrow — does it decay? You end up writing a state machine that never ends.

Batch parallel is much simpler: fixed 9 parallel sub-agents, each reads 20-25 sessions, each produces a pattern list. Then merge and deduplicate. 9-way parallel finishes in <5 minutes; including downstream LLM API consumption, 14 minutes total. Simple, predictable, easy to debug.

Step 3: Distill — Frequency Is Not Value

This is where I stepped in the deepest trap.

The Mine phase produced a lot of pattern candidates — some patterns appeared across 20 sessions. Intuition says: high frequency = important.

Completely wrong.

Many high-frequency patterns were cron retry noise — an API fails due to rate limiting, the Agent retries 3 times, each recorded as an independent event. Looks like "error pattern appears 3 times" — it's actually the same event counted 3 times.

Another class: web_search returning empty results. Happens in almost every session. Is that a valuable pattern? Most of the time it's just an imprecise search query, not a systemic issue.

Pattern frequency ≠ pattern value.

The Hierarchical Distiller solves this with prevalence scoring plus guardrail filtering. Prevalence scoring considers "how many different contexts does this pattern appear in" rather than "how many times total does it show up." Guardrails directly filter known noise categories — cron retries, empty search results, permission prompts.

These two filtering layers cut candidate patterns from 130+ down to 18, while dramatically improving quality.

Step 4: Curate — LLMs Let Themselves Off the Hook

The Surrogate Verifier is CoEvoSkills' smartest contribution. It scores skill drafts across 6 dimensions (clarity, correctness, completeness, relevance, actionability, safety). The key: the verifier never sees the generator's prompt. Information isolation prevents collusion.

In implementation I hit a real problem: when the same LLM instance acts as both verifier and reviewer, it systematically rates its own output higher — LLMs are bad at criticizing their own work. Switching to a Separated Verifier (different model instance, different prompt template) solved it.

This lesson generalizes: any "self-checking" step must account for the Agent's tendency to let itself off the hook.

The Review Gate is another safety net. It checks token efficiency, section coverage, and potential security issues. A scope filter ensures it only reviews skills under its own jurisdiction, avoiding false positives on third-party installed skills.

Step 5: Feedback — Closing the Loop Lets Thresholds Self-Tune

This step was added later — I realized the first three stages were all open-loop.

RECURRENCE_THRESHOLD (how many times a pattern must appear to be a signal) started at 3. After two weeks, I found that many valuable patterns appeared only twice — but were genuinely rare, critical issues (like compatibility problems with a specific API). Meanwhile, some patterns appearing 5+ times were cron noise.

The Feedback stage adds a simple closed loop: each auto-generated skill is tagged as applied/ignored, then we observe applied skills in production — how many times were they invoked? Did they reduce the corresponding real_error rate? Based on this data, RECURRENCE_THRESHOLD adjusts automatically.

The stable threshold currently sits between 2 and 3. It's not fixed — it's dynamically tuned based on real-world results.

Step 6: Dual-Form — One Pattern, Two Shapes

This last step is the icing on the cake.

The same pattern can have two representations: as a SKILL.md for Hermes Agent to read in context, or as a Pi agent configuration (YAML-format tool definitions and instructions). The former suits complex tasks where the Agent needs to "understand"; the latter suits deterministic workflows where the Agent needs to "execute strictly."

Of the 18 proposals, 15 took the skill path, 3 took the subagent path. The dual-form approach doesn't bias the outcome — it lets the pattern "choose" the most suitable form.

---

Key Decisions: Every Tradeoff in One Table

The most valuable part of this entire pipeline might not be the code — it's these decisions:

I covered the symbolic route decision earlier. Executor-Curator separation was the most far-sighted call — splitting "how to do" and "what to learn" into two systems so their optimization targets never interfere.

The Surrogate Verifier choice over real execution is one where peers might disagree. But for me it was straightforward: real execution requires building isolated environments, mocking external dependencies, and handling edge cases. For a solo dev, that investment's ROI is far below using an LLM for automated scoring. The latter has bias, but the bias is known and manageable — we observed a verifier score standard deviation of 12-15 points, with a consistent bias pattern.

---

Real-World Results: By the Numbers

After 8 weeks of pipeline operation:

Eight auto-generated skills doesn't sound like a lot. But each one has actually been used — they're not filler "generated for the sake of generating." The 3 CRITICAL security findings were genuinely valuable discoveries, including one where an API key was written in plaintext into a trajectory JSON.

But what struck me most was the journey from 55% false positive rate to 0%. At the start, virtually every auto-detected pattern was noise. It took two full iterations to add enough filtering layers.

False positive management takes longer than true positive mining — and is more worth the investment. Because users' tolerance for "the machine gave a bad suggestion" is far lower than "the machine didn't suggest anything."

---

Traps I Stepped In

A few lessons worth calling out separately.

Trap 1: LLMs as QA Agents Let Themselves Off the Hook

Mentioned this earlier. The single most important principle for any verification system: the verifier and the generator must not be the same LLM instance. Separating them isn't double cost — it's the minimum bar for verification to mean anything.

Trap 2: Pattern Frequency ≠ Pattern Value

Worth repeating. High-frequency patterns are often cron retries, tool-failure retries, empty-search retries — all systemic noise. Valuable patterns tend to appear only 2-3 times, but each time in a different context. Diversity matters more than frequency.

Trap 3: EVOLUTION_LOG Noise Pollution

I had an EVOLUTION_LOG to record evolution events. Then the SkillClaw bridge sync cron agent (runs every hour) started auto-writing "Hourly bridge sync: skills + sessions" to the log. After 8 weeks, the log had ballooned to 202 lines — 160 of which were this kind of noise.

The fix was simple: explicitly forbid writing to EVOLUTION_LOG in the cron prompt. But until that fix, the noise had drowned out genuinely valuable entries.

The lesson: the more successful your cron automation, the worse your log pollution. Every automated side effect must be explicitly controlled.

Trap 4: Cron Model Drift

If you create a cron job without pinning the model, switching backbone models later causes the cron runner to flag model mismatch and alert. One tiny config oversight, and you get paged at 3 AM. Fix: always specify both --model and --model-provider when creating cron jobs.

---

Leveling Up: Self-Evolution Isn't Just a Technical Problem

I've covered the engineering. Let me zoom out.

This 6-stage pipeline is essentially making an implicit knowledge management process explicit. Without this system, experience lives in a human's head — writing skills, taking notes, remembering the gotchas for next time. With it, experience lives in the Agent's skill library — automatically extracted, validated, and updated.

But I'm increasingly convinced that Agent self-evolution isn't purely a technical problem.

It's first an organizational knowledge management problem.

Think about knowledge management in traditional software engineering — wikis, docs, onboarding guides. What goes wrong? Nobody writes docs, nobody reads what's written, nobody remembers what they read. The self-evolution pipeline does essentially the same thing: turning tacit experience into explicit assets. The difference is the executor is an Agent, not a person.

This pipeline won't turn a mediocre engineer into a great one. But it can capture a great engineer's experience so the team doesn't have to reinvent the wheel every time they hit the same problem.

Think about it: if every Agent session ended not just as a chat log, but as a validated, reusable pattern — then six months later, a year later, the system wouldn't have accumulated 134 MB of data. It would have a growing experience asset library.

There's a line in the SkillOS paper I've kept with me:

Curator is 10x more important than Executor.

Let me rephrase it:

Teaching an Agent how to learn is 10x more important than teaching it how to execute.

The pipeline will keep evolving. Next steps might be Pi agent subagent mode integration, cross-machine SkillClaw aggregation, tighter verification loops. But the direction is clear: make every interaction produce reusable value, instead of silently rotting in 134 MB of chat history.

Not every problem needs AI to solve it. But a problem that AI has already solved should never need to be solved again by the next developer.

---

The full code and configurations for this pipeline live in the Hermes Agent self-evolution system directory. Paper references: Trace2Skill (arXiv 2603.25158), CoEvoSkills (arXiv 2604.01687), SkillOS (arXiv 2605.06614), Skill0 (arXiv 2604.02268), Skill1 (arXiv 2605.06130).