Checkpointers
In shortCheckpointer implementations for state persistence. BaseCheckpointer, InMemoryCheckpointer, PgCheckpointer, SqliteCheckpointer.
- 8 min read
- 8 sections
- Updated
- v0.10.0
- Markdown
A checkpointer persists graph state, messages and thread metadata between requests, so conversations survive restarts and multi-turn runs stay coherent. 10xGraph ships four: the BaseCheckpointer abstract class, plus InMemoryCheckpointer, SqliteCheckpointer and PgCheckpointer. Pass one to graph.compile(checkpointer=...).
For choosing between them, see Checkpointing and threads and Set up checkpointing.
Import paths
All four classes are exported from one package. PgCheckpointer and SqliteCheckpointer raise ImportError on construction if their optional dependencies are missing.
from tenxgraph.storage.checkpointer import BaseCheckpointer, InMemoryCheckpointer
# Optional: requires the pg_checkpoint extra (asyncpg and redis)
from tenxgraph.storage.checkpointer import PgCheckpointer
# Optional: requires the sqlite_checkpoint extra (aiosqlite)
from tenxgraph.storage.checkpointer import SqliteCheckpointerBaseCheckpointer[StateT]
Abstract base class for all checkpointer implementations. It declares async methods and a sync wrapper for each one, so a custom backend only implements the async side.
Abstract async methods
The methods below are abstract and must be implemented by a subclass:
| Method | Signature | Description |
|---|---|---|
asetup |
async () -> Any |
Initialise the storage backend (create tables, connect pools). |
aput_state |
async (config, state) -> StateT |
Persist a state snapshot for the thread in config["thread_id"]. |
aget_state |
async (config) -> StateT | None |
Load the latest state snapshot for the thread. |
aclear_state |
async (config) -> Any |
Delete all state for the thread. |
aput_state_cache |
async (config, state) -> Any |
Write to the fast cache (Redis or in-memory). |
aget_state_cache |
async (config) -> StateT | None |
Read from the fast cache. |
aput_messages |
async (config, messages, metadata=None) -> Any |
Append messages for the thread. |
aget_message |
async (config, message_id) -> Message |
Fetch a single message by ID. |
alist_messages |
async (config, search=None, offset=None, limit=None) -> list[Message] |
List messages. |
adelete_message |
async (config, message_id) -> Any | None |
Delete a single message. |
aput_thread |
async (config, thread_info) -> Any | None |
Store thread metadata. |
aget_thread |
async (config) -> ThreadInfo | None |
Get thread metadata. |
alist_threads |
async (config, search=None, offset=None, limit=None) -> list[ThreadInfo] |
List threads. |
aclean_thread |
async (config) -> Any | None |
Delete a thread and its state and messages. |
arelease |
async () -> Any | None |
Release resources (close pools and connections). |
Sync wrappers
For every async axxx() method there is a sync xxx() wrapper that runs the async method to completion. Use these only from non-async contexts such as a management script:
checkpointer.put_state(config, state)
checkpointer.get_state(config)Optional, non-abstract async methods also exist on the base class, for example aput_checkpoint, aput_cache_value, aget_cache_value, arequest_stop and aget_thread_owner. Backends override them as needed.
Wiring into a graph
Pass the checkpointer to compile. The same call works for every backend.
from tenxgraph.storage.checkpointer import InMemoryCheckpointer
app = graph.compile(checkpointer=InMemoryCheckpointer())InMemoryCheckpointer
In-process dictionary-based storage. Zero dependencies.
from tenxgraph.storage.checkpointer import InMemoryCheckpointer
checkpointer = InMemoryCheckpointer()
app = graph.compile(checkpointer=checkpointer)When to use:
- Unit tests and CI.
- Local development when you don’t need state to survive a restart.
- Ephemeral single-process jobs.
When NOT to use:
- Any multi-process or multi-worker deployment.
- Production applications where conversation history must survive crashes.
Storage behaviour
| Storage | Key | Data |
|---|---|---|
_states |
thread_id |
Latest serialised state snapshot. |
_state_cache |
thread_id |
Hot cache for the running execution. |
_messages |
thread_id |
Ordered message list. |
_threads |
thread_id |
Thread metadata (name, created_at, etc.). |
All access is guarded by per-bucket asyncio.Lock instances for safe concurrent use within a single process.
PgCheckpointer
PostgreSQL-backed checkpointer with a Redis cache for the hot state. Use it for production and multi-worker deployments. Both Postgres and Redis connection details are required.
from tenxgraph.storage.checkpointer import PgCheckpointer
checkpointer = PgCheckpointer(
postgres_dsn="postgresql://user:pass@localhost:5432/mydb",
redis_url="redis://localhost:6379/0", # required unless redis or redis_pool is given
)
await checkpointer.asetup() # creates tables if they don't exist
app = graph.compile(checkpointer=checkpointer)Constructor parameters
The constructor takes the connection arguments below as named parameters; the remaining options are read from keyword arguments. It raises ValueError if neither postgres_dsn nor pg_pool is given, or if none of redis_url, redis_pool or redis is given.
| Parameter | Type | Default | Description |
|---|---|---|---|
postgres_dsn |
str | None |
None |
PostgreSQL DSN. Required unless pg_pool is provided. |
pg_pool |
asyncpg.Pool | None |
None |
Pre-created asyncpg connection pool. |
pool_config |
dict | None |
None |
Config passed to asyncpg.create_pool() (min_size, max_size, etc.). |
redis_url |
str | None |
None |
Redis URL for caching. One of redis_url, redis or redis_pool is required. |
redis |
Redis | None |
None |
Pre-created Redis instance. |
redis_pool |
ConnectionPool | None |
None |
Pre-created Redis connection pool. |
redis_pool_config |
dict | None |
None |
Config passed to redis.asyncio.ConnectionPool() if creating a new pool. |
schema |
str |
"public" |
PostgreSQL schema name where checkpointer tables live. Must match ^[a-zA-Z_][a-zA-Z0-9_]*$, otherwise ValueError. |
cache_ttl (kwarg) |
int |
86400 |
Redis cache TTL in seconds (24 hours). |
state_history_limit (kwarg) |
int |
20 |
Number of historical state snapshots to retain per thread. |
release_resources (kwarg) |
bool |
False |
On arelease(), close the Postgres and Redis resources even if you passed them in. By default only resources the checkpointer created itself are closed. |
enforce_user_isolation (kwarg) |
bool |
True |
Treat user_id as an ownership boundary. When enabled, threads and state are scoped to the requesting user. Set to False for single-tenant apps or when there is no real user identity. |
ID types
PgCheckpointer adapts its schema to the thread and message ID type:
id_type |
SQL column type |
|---|---|
string |
VARCHAR(255) |
int |
SERIAL |
bigint |
BIGSERIAL |
The value comes from the generated_id_type registered with InjectQ by the compiled graph, defaulting to string. You can override it with the id_type keyword argument. The user_id column type follows the user_id_type keyword argument (default string).
Schema migration
asetup() creates the required tables if they do not exist and applies pending schema migrations. It is idempotent and safe to call on every startup. A failed migration raises SchemaVersionError, a subclass of StorageError.
SqliteCheckpointer
Single-file SQLite checkpointer. Stores everything in one local .db file: durable state, the hot “realtime” state cache, generic TTL cache, messages, and threads. No Postgres, no Redis. Fully async via aiosqlite.
from tenxgraph.storage.checkpointer import SqliteCheckpointer
# Defaults to ~/.10xgraph/checkpointer.db; pass a path to override.
checkpointer = SqliteCheckpointer("agent_state.db")
await checkpointer.asetup() # creates tables (also runs lazily on first use)
app = graph.compile(checkpointer=checkpointer)Constructor parameters
| Parameter | Type | Description |
|---|---|---|
db_path |
str | Path | None |
Path to the SQLite database file. Defaults to ~/.10xgraph/checkpointer.db. Parent directories are created on setup. Pass ":memory:" for an ephemeral in-process database (useful for tests). |
When to use:
- Client-side / embedded agents. A desktop app that ships a Python sidecar (Tauri, Electron, PyInstaller) or a local CLI agent. The checkpointer runs entirely on the user’s machine, right next to the app.
- Dedicated-room deployments. Each user (or tenant, or session) has their own process and their own
.dbfile, so there is exactly one writer per database.
When NOT to use:
- Multi-user servers where many users share one backend. SQLite serializes writers and does not scale horizontally. Reach for
PgCheckpointer(Postgres + optional Redis) instead.
Design notes
- One file, no external services. The realtime state cache lives in a separate SQLite table (
af_state_cache) rather than Redis; durable state lives inaf_states. Reads check the cache table first, then fall back to durable state. This is the same two-layer read path asPgCheckpointer, collapsed into one file. - Custom state is preserved. State is serialized with an embedded
__class_path__and reconstructed into its exactAgentStatesubclass on read, matchingPgCheckpointer. - Concurrency. A single persistent connection is opened lazily with
PRAGMA journal_mode=WAL; writes are serialized behind anasyncio.Lock. Callawait checkpointer.arelease()(orcheckpointer.release()) at shutdown to close it. - Isolation. Data is keyed purely by
thread_id;user_idis not required.alist_threadsreturns every thread in the file, which matches the one-database-per-user model.
Writing a custom checkpointer
from typing import Any
from tenxgraph.storage.checkpointer import BaseCheckpointer
from tenxgraph.core.state import AgentState, Message
from tenxgraph.utils.thread_info import ThreadInfo
class DynamoDBCheckpointer(BaseCheckpointer):
async def asetup(self) -> Any:
# Create DynamoDB tables
...
async def aput_state(self, config: dict, state: AgentState) -> AgentState:
thread_id = config["thread_id"]
# Serialise and PUT to DynamoDB
...
return state
async def aget_state(self, config: dict) -> AgentState | None:
thread_id = config["thread_id"]
# GET from DynamoDB and deserialise
...
async def aclear_state(self, config: dict) -> Any:
...
async def aput_state_cache(self, config: dict, state: AgentState) -> Any:
... # in-memory dict, no external call
async def aget_state_cache(self, config: dict) -> AgentState | None:
...
async def aput_messages(
self, config: dict, messages: list[Message], metadata: dict | None = None
) -> Any:
...
async def aget_message(self, config: dict, message_id: str | int) -> Message:
...
async def alist_messages(
self,
config: dict,
search: str | None = None,
offset: int | None = None,
limit: int | None = None,
) -> list[Message]:
...
async def adelete_message(self, config: dict, message_id: str | int) -> Any | None:
...
async def aput_thread(self, config: dict, thread_info: ThreadInfo) -> Any | None:
...
async def aget_thread(self, config: dict) -> ThreadInfo | None:
...
async def alist_threads(
self,
config: dict,
search: str | None = None,
offset: int | None = None,
limit: int | None = None,
) -> list[ThreadInfo]:
...
async def aclean_thread(self, config: dict) -> Any | None:
...
async def arelease(self) -> Any | None:
... # close connectionsConfig dictionary
All checkpointer methods accept a config dict. The required key is:
config = {"thread_id": "session-abc123"}Optional keys:
user_id: scopes message searches by user.run_id: tracks a specific invocation.
Common errors
| Error | Cause | Fix |
|---|---|---|
StorageError |
Unrecoverable PostgreSQL error. | Check Postgres logs and DSN config. |
TransientStorageError |
Temporary Postgres failure. | Retry the call. |
ImportError: PgCheckpointer requires 'asyncpg' or 'redis' |
PgCheckpointer used without both packages installed. |
Run pip install "10xgraph[pg_checkpoint]". |
ValueError: Either redis_url, redis_pool or redis instance must be provided. |
PgCheckpointer created without Redis details. |
Pass redis_url, redis or redis_pool. |
SchemaVersionError |
A schema migration failed during asetup(). |
Check database permissions and the schema name. |
ImportError: SqliteCheckpointer requires 'aiosqlite' |
SqliteCheckpointer used without aiosqlite installed. |
Run pip install "10xgraph[sqlite_checkpoint]". |
| State lost between requests | Using InMemoryCheckpointer with multiple workers. |
Switch to PgCheckpointer or ensure a single-process deployment. |