How We Decide What's "Relevant" When Retrieving Memory
The hard part of building persistent AI memory isn't storing data, it's retrieving the right slice of it without dumping everything back into the prompt (which just recreates the token-bloat problem you were trying to avoid).
Our current approach: semantic search over stored memories, scoped by vault, ranked by relevance to the current query rather than recency. Store wide, retrieve narrow.
The tricky edge case we're still tuning: a memory that's contextually relevant but was saved months ago sometimes loses to a shallower, more recent one. Recency bias creeps in even when you don't want it to.
Curious how others handling long-term memory/RAG-style retrieval are weighting relevance vs. recency, still feels more art than science right now.
One approach that has worked well for us is to keep relevance and freshness separate instead of blending them too early. First apply hard scope filters (product, workflow, or decision type), then rank semantically, and use recency mostly as a tie-breaker when scores are close.
The tricky part is durable “negative” memory: failed experiments should stay retrievable for a long time, but only become prominent when the current query is about a similar decision or repeated mistake. A small regression set of high-cost-to-miss queries has been more useful than optimizing similarity scores in isolation.
The harder problem seems to be defining what “relevant” means when the memory that matters most may have little semantic overlap with the current query. That feels like a deeper retrieval problem than simply tuning the relevance-versus-recency weighting.