Show IH: I was my AI coding agent's memory — so I automated myself out of that job
For months I had an unpaid part-time job: being my coding agent's memory.
Long sessions (Codex, Claude Code, Cursor — the pattern shows up across harnesses) eventually "compact": the context window fills, old state gets summarized, and the agent keeps going — minus the why. Decisions turn into vague one-liners, dead ends get retried, and I re-type the same constraint for the third time that week.
So I built agent-crystallize and open-sourced it: a local-first CLI that saves durable Markdown artifacts into your repo before the boundary eats the context — lightweight checkpoints while you work, a fuller "crystal" (decisions + evidence + open loops + resume prompt) before compaction or session end. The next session resumes from the artifact instead of from my memory of what I told it last time.
Favorite moment so far: the session where I published it hit compaction mid-launch and recovered from its own checkpoint. The product demoed itself.
- Repo: github.com/stewie-sh/agent-crystallize (Apache-2.0, no account, no server; install is one
npm install -gaway — command in the README) - Why compaction loss compounds — the full write-up
Curious about your coping ritual: do you babysit the context meter and save by hand, spawn a fresh session per task, or just accept the amnesia?
This is exactly the kind of problem that quietly compounds once agents start touching real product work.
That's the part that surprised me too — it's invisible on toy tasks and only compounds once the agent is touching real product work over weeks. Thanks for reading.
The detail that lands for me is the 'why behind every decision quietly vaporizes' line. Code survives compaction but intent doesn't, so the next session starts rebuilding assumptions from scratch.
That's the asymmetry the whole thing hinges on — code survives compaction because Git holds it, intent doesn't because nothing does. Glad that line landed.
My coping ritual landed somewhere next to this, and your framing named the half I was missing.
What I do: for anything I run more than once, the context lives in a version-controlled runbook in the repo, and the agent reads it at the START of every run. So a task on run #50 boots from the same written context as run #1, instead of me reconstructing it from memory each session. The rule that keeps it honest is dumb but works: any time I had to re-explain something by hand mid-run, I write it back into the runbook before I close the task. That's the "unpaid memory job" slowly disappearing.
Where your crystal idea clicks for me is the other direction. The runbook is the proactive, authored-once context going in; a crystal is the durable capture of what a live session figured out, coming back out before compaction eats it. Feeding a session's crystal back into the standing runbook would close the loop, the reactive capture becomes the next run's proactive context.
Nice work open-sourcing it, the self-recovery-mid-launch moment is a great demo.
The inbound/outbound framing is sharp — your runbook is the durable instructions in, and this is the durable work-state out, and they genuinely are different lifecycles. "Write it back before I close the task" is the exact discipline; the only thing the tooling adds is making that reflex cheap enough to happen at every boundary instead of only when you remember to.
The "lose the constraint before the agent even sees it" framing is the part I hadn't named, and it's real. Half my context loss happens in that gap between having the thought and getting it into the session cleanly, not during compaction at all.
You're right that they're two halves of one loop. Voice-capture on the way in keeps the constraint from evaporating at the speed-of-thought bottleneck, and writing it back before I close the task keeps it from evaporating at the compaction bottleneck. It's the same failure showing up at two different ends of the pipe.
The thing I'm still working out is where the handoff sits: does the spoken constraint from DictaFlow become a durable artifact on its own, or does it still need a second pass to survive past the current task? That seam is where I keep losing things.
The compaction-eats-the-why problem is real, and I think it shows up one layer earlier too. Sometimes I lose the constraint before the agent even sees it, because typing the whole thing is slower than the thought. That's part of why I built DictaFlow. I hold a key, say the constraint out loud, and drop it straight into Claude, Codex, or Cursor while the context is still fresh. Your crystal idea feels like the other half of the loop, fast capture going in, durable reasoning coming back out.
Glad it resonated — the capture-in and resume-out ends really are two different problems. Thanks for reading!
this is way too relatable
being the “memory layer” for these agents is honestly one of the most annoying hidden costs of using them seriously. the compaction issue doesn’t feel like a small inconvenience, it literally breaks continuity and decision quality over time
i like the idea of turning sessions into actual artifacts instead of hoping the model remembers. especially the “decisions + evidence + open loops” part, that’s exactly what gets lost and causes repeated mistakes
also that moment where it recovered mid-launch from its own checkpoint is kind of the perfect proof this is a real problem
feels like this direction makes a lot of sense, less chat history, more structured state
Agreed — chat history is a recording, an artifact is a state you can reload. The difference shows up most in decision quality over long projects, exactly as you said.
the failure mode i'd design hardest against: the crystal written at the compaction boundary is authored by the agent at its most degraded — window full, attention smeared, quality already dropping. that's the worst possible moment to ask it to distill the why.
the capture you can actually trust is the one taken at decision-time, while the reasoning is still fresh in the window — an append the moment the decision lands, not a reconstruction at the end. it's basically event-sourcing: record the event when it happens, don't rebuild state from a degraded snapshot later. the boundary "crystal" then becomes a rollup of those appends, not the primary source.
bonus: decision-time capture gives invalidation (the append-only-eventually-lies point above) a natural home — when a new decision supersedes an old one, you mark it right there as it happens, instead of hoping the end-of-session pass notices the contradiction.
Fully agree, and that's actually the split: checkpoints are the decision-time, event-sourced layer — written while quality is still high, when the decision happens. The crystal at the boundary doesn't originate context under a smeared attention budget; it consolidates checkpoints that were already captured at their good moment. So the boundary is a roll-up, not a first capture.
my ritual is fresh session per task plus a hand-kept decisions.md, and honestly your tool is the thing i kept meaning to build. the rejected-alternatives point someone made above is the big one, so i'll add two others.
capture at decision-time, not at the compaction boundary. by the time the meter is near full the why is already getting summarized away, so a crystal written at the boundary is reconstructing from a degraded copy. the high-value moment is when the agent picks Z over X, snapshot the reasoning right there, event-triggered, not on a timer.
and the failure mode nobody warns you about: crystals go stale. a decision you saved three sessions ago that a later change quietly invalidated, then replayed confidently in the next resume prompt, is worse than amnesia. the agent re-anchors on something that's no longer true and defends it. so the artifact needs invalidation/expiry, not just accumulation. an append-only memory eventually lies. (this is basically continuous ADRs btw, and ADR templates already solved the "what to record" question: context / decision / considered alternatives / consequences.)
The ADR comparison is the right frame — it's closer to a rolling ADR log for work-state than to a spec. On staleness: the artifacts are inspectable Markdown, and each capture lands as a new timestamped file rather than silently overwriting the last one. They also point at commits and test runs instead of duplicating them, so when a decision is superseded you can see it go stale against the code instead of trusting a silent summary. Explicit-and-visibly-wrong beats implicit-and-lost.
the compaction-eats-the-why problem is so real, i re-type the same constraint three times a week too. curious how you decide what's worth crystallizing vs noise, is it manual or does it flag decision points automatically?
Both. You can run it manually at any point, and there are hook examples for a couple of harnesses (Codex, Claude Code) that fire a checkpoint automatically before compaction. The automatic part is best-effort by design — it never blocks your session — so an explicit run is always the reliable path. And "explicit" doesn't mean you type it yourself — you can just ask your agent to run the command. :)
This resonates. The thing that quietly kills long agent sessions is not losing the state, it is losing the why. State can be re-derived, but a decision without its rejected alternatives gets re-litigated every few turns, and the agent happily retries a dead end you already ruled out.
Two things I would be curious about in your crystal format. Do you capture the alternatives you rejected and the reason, not just the decision, since that is what actually stops the retries. And how do you decide what not to write down, because the failure mode on the other side is a bloated artifact where the resume prompt drowns in noise and the next session anchors on the wrong thing.
Also worth saying, this generalizes past coding. Any agent you put on a real ongoing job needs a durable decisions and why layer, not just chat history. That layer is usually the difference between an agent that holds a role and one that resets to zero every morning.
Honest answer: there's no enforced "rejected alternatives" schema. The format is editable Markdown with Decisions, Findings, and Reality Checks sections — that's where you record the path you rejected and why it lost. So it's a convention the format invites, not a field it validates. But your instinct is right: a decision without its rejected alternatives is how a team re-litigates a settled call next week, so I write the "why not X" straight into the decision bullet. On the bloat worry — the Resume Prompt is its own section, separate from the reasoning archive, so the sections can grow while the thing the next session reads first stays short.
The part that resonates is preserving negative results, not just decisions.
When an agent forgets a failed path, the next session can waste time with total confidence. A durable artifact is useful because it turns "we tried this and it failed" into repo state instead of human memory.
Do you imagine people using this mostly at compaction boundaries, or as a lightweight checkpoint after any meaningful dead end?
After any meaningful dead end, not just at boundaries. A checkpoint is cheap enough — thirty seconds — that a failed path is worth writing the moment it fails, as repo state, so the agent doesn't confidently retry it a week later. The boundary crystal just rolls up whatever checkpoints already exist.
This is exactly the kind of problem that quietly compounds once agents start touching real product work.
The detail I like in this approach is that it saves the negative results too. In my experience the expensive failures are not only "the agent forgot what worked"; it is "the agent forgot what was already ruled out" and confidently retries the same dead path.
I have been using a cruder version of this: a project-local journal with decisions, rejected options, open loops, and resume notes. The missing piece is making it automatic enough that the agent treats it as part of the workflow rather than a separate docs chore. Your crystallize/checkpoint framing feels like the right direction.
That last mile — making it feel like part of the workflow rather than a separate docs chore — is exactly the gap. A manual journal works right up until the session where you're heads-down and skip it, which is usually the session you most needed it. That's the whole reason for the hook path: let the boundary trigger the write instead of your willpower.
The compaction-loss problem is real and it compounds in a specific way I didn't expect until it bit us: it's not just the reasoning that gets lost, it's the negative results. We built a small digital-products storefront and every dead end (a marketplace signup that hit a bot wall, a subreddit that bans self-promo) needs to survive the next session's compaction or the agent just retries the same dead thing a week later, confidently. Our fix was cruder than yours: a plain append-only journal file the agent is told to read before acting, git-style, instead of trusting its own context. Your crystallize idea of writing the artifact before compaction eats it, not after, is the better version of that. Following the repo.
Yes — negative results are the loss category nobody plans for. Reasoning at least feels worth saving, so people try; a dead end feels like nothing happened, so it leaves no trace, and then the agent re-runs it with full confidence a week later. Your "read the journal before acting instead of trusting its own context" is the right discipline — the artifact has to be the source of truth, not the window. Writing it before the boundary eats it rather than after is really the only tweak; the rest you're already doing. Thanks for following the repo 🙏
What stood out to me is that you're treating context as an asset instead of a temporary resource.
The problem isn't just that AI forgets. It's that the reasoning behind important decisions gradually disappears, forcing humans to recreate it. Preserving that reasoning may end up being more valuable than preserving the conversation itself.
"Context as an asset instead of a temporary resource" is the right reframe — and you're right that the reasoning outlives the conversation in value. The transcript is the raw material; the decision is the thing worth keeping.
Glad it resonated.
Your reply made me think there's one strategic decision sitting underneath that idea of preserving decisions instead of conversations which becomes much more significant as the product grows, but I don't think I can explain the reasoning properly in a thread without oversimplifying it.
If you're interested, what's the best email to reach you on?