design: add long term memory design#844
Conversation
opieter-aws
had a problem deploying
to
manual-approval
GitHub Actions
Error
opieter-aws
requested a deployment
to
manual-approval
GitHub Actions
Waiting
opieter-aws
force-pushed
the
opieter-aws/push-local-commits
branch
from
f280dba to
c0bfdb5
Compare
opieter-aws
had a problem deploying
to
manual-approval
GitHub Actions
Error
opieter-aws
requested a deployment
to
manual-approval
GitHub Actions
Waiting
|
|
||
| #### Context Injection | ||
|
|
||
| Context injection guarantees that relevant knowledge is always present, at the cost of paying for retrieval every turn. This is useful when baseline context is more important than token efficiency, or when the model can't reliably judge when to search. |
There was a problem hiding this comment.
Why did we decide on a system prompt vs a distinct message?
There was a problem hiding this comment.
The agent takes system prompt injection with higher priority than a message. We can run additional benchmarking here, but that's what research shows
There was a problem hiding this comment.
lol I have the same mental model of keeping system prompt for instructions only, but the industry tends to dump the data to system prompt. That keeps conversational flow normal (sequential), and it makes system prompt into this dynamic component.
There was a problem hiding this comment.
qq, given that we keep appending xml to system prompt (skills, memory, etc), should we make it more configurable? should we improve our system prompt handling? 🤔
|
|
||
| ### Knowledge Ingestion | ||
|
|
||
| Knowledge can enter the system in two ways: the agent explicitly writes it (via a `store_memory` tool registered by MemoryManager), or MemoryManager automatically extracts it from conversation messages using an extractor (a component that distills messages into discrete facts via a model call). |
There was a problem hiding this comment.
should store/search memory be separate tools? can't we do dynamic tooling instead? just have a memory tool with search and store as actions? 🤔
There was a problem hiding this comment.
as we split the tools more, it'll cause pain/annoyance (on toolspec and token side + tool shortlisting side [lke manage tools])
that said if it performs better on evals, sure
There was a problem hiding this comment.
If they can be registered separately, we can consider that. We shouldn't couple them in case of a readOnly store.
| model, | ||
| memoryManager: new MemoryManager({ | ||
| stores: [{ store, namespace: 'user-123' }], | ||
| tools: true, // Agent gets search_memory tool. |
There was a problem hiding this comment.
nit: would workshop the config UX more. tools: true is surprising to me
There was a problem hiding this comment.
Maybe like retrieveTools and writeTools could be more granular/clear
There was a problem hiding this comment.
+, do we want to allow users to provide their tools?
There was a problem hiding this comment.
See https://github.com/strands-agents/docs/pull/844/changes#diff-bbb536e8ceda86b8e7bbc8a665ee8f4eca2ac0d5ead2259f6c043ded15e54d1fR305 . AFAIK we're using the same setup for context offloading. It's a situation of boolean | defaults
| }) | ||
| ``` | ||
|
|
||
| ## Alternatives Considered |
There was a problem hiding this comment.
Did we consider thinking about memories as files instead of in-memory representations?
| ### What Becomes Harder or Requires Attention | ||
|
|
||
| - **Eventual consistency**: writes are async; a fact may not be searchable in the next turn. | ||
| - **Extraction cost**: `perTurn` triggers a model call every turn. We need sensible defaults and good documentation for users to navigate this. |
There was a problem hiding this comment.
How does this compare generally to what other frameworks use? And/or how do most services handle this? Are they mostly doing this server side anyways?
|
|
||
| When no extractor is configured, messages are serialized as plain text and passed directly to the store's `add()` method. This is the correct setup for managed backends that handle extraction internally. | ||
|
|
||
| ## Developer Experience |
There was a problem hiding this comment.
what are the defaults? you enabled both direct ingestion and tools with a param, so we do we need customers to enable one or the other?
|
|
||
| ### Context injection lifecycle | ||
|
|
||
| When injection is enabled, MemoryManager hooks into `BeforeInvocationEvent` (once per turn). The lifecycle: |
There was a problem hiding this comment.
Side-topic, but we should push for a better way to do this IMHO; the whole "modify the system-prompt on every term" seems like a non-ideal hack at this point
|
|
||
| ### Migration | ||
|
|
||
| No breaking changes. `memoryManager` is a new optional parameter on `AgentConfig`. |
There was a problem hiding this comment.
how does this relate to conversation manager, session manager, snapshots?
opieter-aws
requested a deployment
to
manual-approval
GitHub Actions
Waiting
opieter-aws
requested a deployment
to
manual-approval
GitHub Actions
Waiting
8 participants
Description
Related Issues
Type of Change
Checklist
npm run devBy submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.