How do memory stores (vector DBs, SQL stores) enhance agent performance?

What you need to know

The rule of thumb

• If the question is "what is the value now?", use SQL.
• If the question is "what have we said or seen that is like this?", use vector search.
• If the question is about relationships ("which services depend on this one?"), a graph store or plain relational joins may fit.

Many teams keep both in one database, such as Postgres with a vector extension, so filters and similarity search run in one query.

Filter inside the search

Apply user_id, tenant_id and permission filters as part of the vector query. If you fetch the top 10 across all tenants and filter afterwards, you may end with zero results, or worse, a bug that forgets the filter leaks another customer's data.

How this improves the agent

• Fewer hallucinated facts, because exact facts come from rows, not from fuzzy text.
• Better personalisation, because relevant past episodes are found even when worded differently.
• Smaller prompts, because the agent retrieves 5 relevant memories instead of the whole history.

A real-life example

A travel-booking agent stored everything about travellers as text memories in a vector store. A traveller changed her home city from Pune to Bengaluru. The next week the agent booked a Pune departure, because the old memory "Home airport: PNQ" was closer in meaning to the query than the new one.

The fix split the data:

• SQL traveller_profile: home airport, seat preference, loyalty numbers, policy grade. One row per person, updated in place.
• Vector store: free-text notes from past trips, such as "hates red-eye flights after a bad Delhi trip in March", with timestamps and a user filter.

The agent now reads the profile row through get_profile() every time, and searches notes only for soft preferences. Wrong-airport bookings stopped completely.

Follow-up questions to expect

• "Why not store everything in the vector store with timestamps?" — You can sort by time, but every query then needs extra logic to pick the latest version, and deletion is still hard. Changing facts belong in rows.
• "When is a graph database worth adding?" — When the questions are about relationships and paths, like dependencies between services. Start with relational joins; add a graph only if they get painful.
• "How do you evaluate memory retrieval?" — Build a small labelled set of queries with the memories that should come back, and measure recall at k.