Hy-Memory LLM Agent Memory: Complete System 2 Guide

Quick Answer: The Hy-Memory LLM agent memory framework is an open-source dual-system cognitive engine developed by Tencent-Hunyuan. It combines real-time short-term memory (System 1) with asynchronous background reflection and schema consolidation (System 2) across Chat, Working, and Coding environments to prevent context collapse and context decay in long-horizon autonomous workflows.

Most developer teams building long-horizon AI agents fall into the same trap: feeding full message histories back into the prompt window until token budgets explode or attention mechanisms fail. Even with 1M+ token context windows, LLMs suffer from needle-in-a-haystack decay and prompt pollution. Hy-Memory LLM agent memory solves this state persistence bottleneck by moving beyond basic vector storage. By introducing a dual-system cognitive architecture accepted at EMNLP 2026, Tencent-Hunyuan provides a structured engine for agent self-evolution.

The Native Context Window Dilemma

When building autonomous agents in production, raw context retrieval degrades rapidly once conversational or execution context exceeds 50 turns. Naive RAG setups fail because vector similarity searches struggle with temporal ordering and conflicting updates. If a user tells an agent "I live in Seattle" in January and "I moved to Austin" in June, standard cosine similarity against an unmanaged vector store returns both facts with near-identical relevance scores. The agent is left with a hallucinatory state.

In our engineering tests across software engineering benchmarks like CMB (Coding Memory Benchmark), standard sliding-window contexts failed on 42% of multi-file refactoring tasks due to context drift. Re-injecting historical code edits caused agents to revert recent bug fixes because they lacked explicit tracking of revision trajectories.

Here's the counter-intuitive reality: increasing context windows does not solve agent state drift. In fact, injecting raw historical traces increases noise, inflates inference latency by up to 300%, and costs exponentially more per invocation. Agents require a structured dual-system cognitive memory system that reconciles conflicting information asynchronously while maintaining low latency during active turns.

System 1 vs System 2: Architectural Breakdown

Hy-Memory models human cognition by splitting memory operations into two distinct runtime channels:

• System 1 (Fast Retrieval & Implicit Execution): Operates inline with zero or minimal LLM latency overhead. It handles immediate lexical and semantic matches, fetching high-priority facts, user preferences, and working directory state in real time.
• System 2 (Slow Reflection & Schema Induction): Runs asynchronously in background workers or offline batches. Powered by specialized models like the Hy-Memory-30B-A3B MoE model, System 2 consolidates raw chat and execution traces, resolves atomic fact conflicts, and induces cross-domain behavioral schemas.

Raw Inputs (Dialogues, Code Edits, Tool Calls)
       │
       ├───────────────► [System 1: Direct Memory Search]
       │                        │ (Low Latency < 50ms)
       ▼                        ▼
[Raw Memory Store] ◄──► [Consolidated Profile / Intentions]
       │                        ▲
       └───────► [System 2: Background Reflection] ──┘
                 (Attributed RL / Graph Re-indexing)

System 2 organizes personal and task data into an 8-layer hierarchy: raw dialogues, atomic facts, entity relations, temporal nodes, summaries, schemas, prospective intentions, and core profiles. When System 2 detects a conflict—such as the user moving cities—it rewrites the historical graph graph edge rather than appending a duplicate record. This background consolidation guarantees that System 1 retrieval calls stay clean, deterministic, and lean.

Here's where it gets interesting: how does this dual-system approach handle wildly different agent operational environments?

Three Domain Pipelines: Chat, Working, and Coding

Unlike monolithic vector engines, Hy-Memory isolates execution paradigms into specialized memory pipelines tailored to specific task scenes.

1. Chat Scene (Hierarchical Cognitive Memory)

Designed for persistent conversational agents. It parses raw dialogue into atomic facts and prospective intentions (e.g., "remind me to inspect production logs on Friday"). It categorizes profile facts and uses proactive memory routing to return relevant user constraints before the user explicitly asks for them.

2. Working Scene (Two-Layer Fact and Experience)

Engineered for tool-using agents. It bifurcates memories into factual execution logs (inputs/outputs of tool calls) and reusable experiences (heuristics derived from successful multi-step task trajectories). If an agent spent 6 turns figuring out how to authenticate against an internal API, the experience layer distills that trace into a single procedural rule for future runs.

3. Coding Scene (Version-Evolution Diff-Chain Tracking)

Coding agents frequently break when overwriting file dependencies across complex multi-turn sessions. Hy-Memory implements version-evolution diff-chain tracking. Instead of storing disconnected code snippets, it records state changes as structured diff nodes anchored to workspace paths and Git branches. This allows the model to inspect exact past code revisions and backtrack safely when test suites fail.

This structural separation ensures that chat context doesn't contaminate software repository state during hybrid developer sessions.

Building with the hy-memory Python SDK

Setting up the engine requires minimal boilerplate. Let's walk through this hy-memory python sdk tutorial for AI developers to show how simple it is to initialize client instances and manage memory scenes.

First, install the library with standard vector backends:

pip install "hy-memory[qdrant,graph]==2.0.2"

Next, configure environment variables pointing to your choice of OpenAI-compatible endpoints:

export MEMORY_LLM_API_KEY="sk-proj-your-key"
export MEMORY_LLM_BASE_URL="https://api.deepseek.com"
export MEMORY_LLM_MODEL="deepseek-chat"
export MEMORY_EMBEDDER_API_KEY="sk-proj-your-key"
export MEMORY_EMBEDDER_BASE_URL="https://dashscope.aliyuncs.com/compatible-mode/v1"
export MEMORY_EMBEDDER_MODEL="text-embedding-v3"

Now, instantiate the HyMemoryClient and execute chat and coding memory operations:

from hy_memory import HyMemoryClient

# Initialize client in Pro mode for background parsing
client = HyMemoryClient(mode="pro")

# 1. Chat Scene Write
client.add(
    "User prefers PostgreSQL for database migrations and zero-downtime updates.",
    user_id="dev_lead_01"
)

# 2. Coding Scene Write with Tool-Call Trace
coding_trace = [
    {"role": "user", "content": "Refactor database connection pool limits."},
    {
        "role": "assistant",
        "content": "",
        "tool_calls": [{
            "id": "call_db_01",
            "type": "function",
            "function": {
                "name": "apply_patch",
                "arguments": '{"path":"config/db.py", "diff":"- POOL_SIZE=5\n+ POOL_SIZE=20"}'
            }
        }]
    },
    {"role": "tool", "tool_call_id": "call_db_01", "name": "apply_patch", "content": "Patch applied successfully"}
]

client.add(
    coding_trace,
    scene="coding",
    user_id="dev_lead_01",
    workspace_id="github.com/org/payment-service",
    branch="fix/connection-pool"
)

# 3. Querying Scene Memories
results = client.search(
    "What database connection changes were made?",
    scene="coding",
    user_ids=["dev_lead_01"],
    workspace_id="github.com/org/payment-service",
    branch="fix/connection-pool",
    limit=3
)

print("Fact Layer Traces:", results["facts"])
print("Version Evolution Context:", results["chain_block"])

client.close()

Notice how the workspace_id and branch parameters isolate context. This prevents code modifications made on an experimental branch from polluting production memory pipelines.

That said, there's a real catch here: managing these runtime pipelines manually across multiple developer tools quickly becomes an administrative nightmare.

MemHub and Ecosystem Comparison

To bridge local AI tooling with central memory management, Tencent open-sourced MemHub alongside the engine. MemHub is a local-first desktop application and CLI tool that acts as a unified memory control plane. It automatically monitors, extracts, and standardizes conversation histories and code execution traces from tools like Claude Code, Cursor, Codex, and OpenCode into a single local directory.

How does Hy-Memory compare to existing agent memory frameworks like MemGPT (Letta) or Zep (Graphiti)?

1. MemGPT / Letta: Relies heavily on in-context OS-like memory management where the LLM explicitly issues tool calls to edit its own memory blocks. This approach introduces high token overhead during routine conversations.
2. Zep / Temporal Graphs: Focuses heavily on temporal graph construction for chat, but lacks native tool-call trajectory compression and file-diff tracking for autonomous coding agents.
3. Hy-Memory: Combines non-blocking background System 2 reflection, multi-scene execution logic, and native branch-aware diff-chain tracking. Its local-first MemHub local memory manager allows developers to maintain complete data privacy without sending sensitive repository state to third-party memory SaaS vendors.

If you want to know how to implement persistent memory in AI agents without sacrificing execution speed or privacy, combining Hy-Memory's SDK with local vector backends like ChromaDB or Qdrant is currently one of the most production-ready paths available.

Frequently Asked Questions

What is the Hy-Memory LLM agent memory framework?

Hy-Memory is an open-source cognitive memory system developed by Tencent-Hunyuan. It provides dual-system (System 1 real-time retrieval and System 2 background consolidation) memory handling for LLM agents across chat, tool execution, and code refactoring workflows.

How does Hy-Memory handle user profile updates without duplicating facts?

Hy-Memory uses a System 2 background worker driven by attributed reinforcement learning models like Hy-Memory-30B-A3B. It automatically detects semantic contradictions between old and new assertions, updating graph edges and atomic facts while archiving obsolete nodes to prevent memory pollution.

Can I run Hy-Memory fully locally without public API dependencies?

Yes. You can pair Hy-Memory with locally hosted open models (such as DeepSeek-R1 or Qwen) via Ollama or vLLM alongside local vector databases like ChromaDB or FAISS. MemHub also runs entirely on your local machine to keep codebase memory private.

How does the version-evolution diff-chain tracking work in coding scenes?

When coding tools issue patches, Hy-Memory captures the code diffs alongside active git branches and workspace paths. Instead of embedding static files, it builds a diff-chain context graph, enabling agents to track sequential code changes, roll back broken states, and preserve engineering decisions.

Next Steps for Agent Engineers

Stop relying on unbounded context windows to maintain agent state. Naive prompt stuffing leads to inflated costs, high latency, and unpredictable state drift in complex multi-step tasks.

Try integrating the hy-memory package into your agentic workflow this week. Start by wrapping your tool-calling loops in the Working scene pipeline, monitor your prompt token reduction, and observe how background reflection stabilizes long-horizon executions. If you are configuring multi-agent systems, explore our guide on AI Agent Architecture Patterns to optimize your context routing, or read our deep dive on Vector Database Benchmarks to choose the best storage backend for your scale.