Here is the practical decision framework.

The crisp definitions​

Workflow: the LLM does not decide control flow. You wrote the steps; the model fills in language-shaped tasks at specific points.

Agent: the LLM decides control flow. You give it tools and a goal; it picks the next action at each step.

A "graph with conditional edges" is a workflow if the conditions are deterministic Python; it is an agent if the conditions involve the model picking the next node.

This is a continuum, not a binary. Most systems are 80% workflow and 20% agent.

Workflow code​

def summarize_call(transcript: str) -> dict:
    # Step 1: extract entities (LLM)
    entities = llm.extract(transcript, schema=EntitySchema)

    # Step 2: classify intent (LLM)
    intent = llm.classify(transcript, labels=INTENTS)

    # Step 3: write summary (LLM)
    summary = llm.summarize(transcript, max_words=120)

    # Step 4: store (deterministic)
    db.write(call_id, {"entities": entities, "intent": intent, "summary": summary})

    return {"entities": entities, "intent": intent, "summary": summary}

Three LLM calls, in a fixed order. The model never picks the next step.

Agent code​

agent = Agent(
    model="google/gemini-2.5-flash",
    system_prompt=[{"role": "system", "content": (
        "Help the user with their support request. "
        "Use lookup_order to find their order, refund_order to issue refunds, "
        "or escalate_to_human if you cannot resolve it."
    )}],
    tool_node="TOOL",
)

# The model decides which tools to call, in what order, and when to stop
app.invoke(
    {"messages": [Message.text_message("My order #123 hasn't arrived.")]},
    config={"thread_id": "support-1"},
)

The model picks: look up the order, check status, decide whether to refund or escalate, etc. That decision is what makes this an agent.

When workflows win​

Workflows are the right answer when:

• The steps are predictable. Most data pipelines, ETL with LLM enrichment, batch processing.
• You need determinism. Compliance review, regulated outputs, anything where the path matters as much as the result.
• Cost matters. Each LLM call is paid; a fixed pipeline pays only for what you need.
• Latency matters. No "the model is thinking" loops. Steps run sequentially with predictable timings.
• You need to test it. Workflows are easy to unit-test; agent loops are not.

A workflow with three LLM calls is a far simpler operational target than an agent that might make three calls, or seven, or none.

When agents win​

Agents are the right answer when:

• The path depends on the input. Customer support: did the user ask for a refund or a status check?
• You can't enumerate the steps in advance. Research tasks, debugging tasks, anything with branching.
• The model needs to decide when to stop. Open-ended exploration with a clear goal.
• Tool selection matters. When the right tool depends on what the user said, the model is the natural router.

The killer feature of agents is deciding when to stop. Workflows always run to the end; agents can finish early.

Hybrid: the most common production shape​

Most production systems are hybrids:

[ Deterministic workflow ]
    Step 1: Extract entities
    Step 2: Classify intent
    Step 3: ──── Branch on intent ────
                    │
                    ├─→ [ Simple workflow: format response ]
                    │
                    └─→ [ Agent: handle complex cases ]
                            └─ tool calls, loops, etc.
    Step 4: Log result

The deterministic shell handles the boring parts (extraction, logging, routing). The agent handles the parts where the model needs to decide.

Code shape:

def handle_request(req):
    intent = llm.classify(req.text, labels=INTENTS)

    if intent == "simple_lookup":
        return run_lookup_workflow(req)

    if intent == "complex_support":
        return support_agent.invoke(
            {"messages": [Message.text_message(req.text)]},
            config={"thread_id": req.user_id},
        )

This is the right shape for a typical SaaS support system. 80% workflow (cheap, deterministic, fast), 20% agent (handles the long tail).

A decision tree​

For any subsystem in your architecture, ask:

1. Can I enumerate the steps in advance?

   • Yes → workflow
   • No → agent

2. Does the path depend on the input?

   • Yes → agent (or branching workflow)
   • No → workflow

3. Does the model need to decide when to stop?

   • Yes → agent
   • No → workflow

4. Does this need to be deterministic for compliance / testing?

   • Yes → workflow (even if the obvious shape is an agent)
   • No → either is fine

When in doubt, start with a workflow. It is easier to upgrade a workflow to an agent than to downgrade.

Operational cost differences​

Agents pay an "intelligence tax". The freedom to decide costs you in token spend, latency variance, and operational complexity. Charge for that intelligence only when you need it.

Same primitive, different uses​

A graph runtime like AgentFlow or LangGraph supports both. The runtime is the same; the wiring is different:

• Workflow: edges are deterministic, conditional functions are pure Python
• Agent: at least one node is an Agent whose tool choice drives an edge

You can mix them in one graph. A workflow that calls into a small agent for one branch, then resumes deterministic processing.

graph = StateGraph(AgentState)
graph.add_node("EXTRACT", extract_node)             # workflow
graph.add_node("CLASSIFY", classify_node)           # workflow
graph.add_node("AGENT", complex_support_agent)      # agent
graph.add_node("LOG", log_node)                     # workflow

graph.add_edge("EXTRACT", "CLASSIFY")
graph.add_conditional_edges(
    "CLASSIFY",
    lambda s: "AGENT" if s.intent == "complex" else "LOG",
    {"AGENT": "AGENT", "LOG": "LOG"},
)
graph.add_edge("AGENT", "LOG")
graph.add_edge("LOG", END)

This is the production shape for most non-trivial AI systems.

Common mistakes​

1. Agent-everything. Wrapping a 3-step pipeline in an agent because "agents are cool." You pay 3× the tokens for the same outcome with worse predictability.
2. Workflow-everything. A long if-else chain trying to anticipate every case the model could handle in two tool calls. Use the agent.
3. No determinism boundary. When the model gets to call any tool any time, your audit log is unreadable. Define agent boundaries explicitly.
4. Skipping the workflow shell. A pure agent in production with no deterministic guardrails is a footgun. Wrap it.
5. Not setting recursion_limit. Agents loop. Always cap them.

Further reading​

• State graph concept. Runtime shared by both shapes
• Multi-agent orchestration patterns
• ReAct agent with real APIs. Pure agent example
• Production AI agents (observability + retries)

If your system is mostly a workflow with one agent loop, Get started. The same runtime handles both, with the same primitives.

We will walk a real, two-agent example end-to-end: imports, state, nodes, edges, checkpointing, API serving, and the TypeScript client.

What changes, what stays​

The shape is identical. The names and types are different.

The example: ReAct agent + tool​

Our starting point. A LangGraph ReAct agent that calls a get_weather tool.

Before (LangGraph)​

from langchain_core.messages import HumanMessage
from langchain_core.tools import tool
from langgraph.graph import StateGraph, END
from langgraph.prebuilt import create_react_agent
from langgraph.checkpoint.memory import MemorySaver

@tool
def get_weather(location: str) -> str:
    """Get current weather for a city."""
    return f"It is sunny and 22°C in {location}."

memory = MemorySaver()
app = create_react_agent(
    "openai:gpt-4o-mini",
    tools=[get_weather],
    checkpointer=memory,
)

result = app.invoke(
    {"messages": [HumanMessage("Weather in Tokyo?")]},
    config={"configurable": {"thread_id": "demo-1"}},
)
print(result["messages"][-1].content)

After (AgentFlow)​

from agentflow.core.graph import Agent, StateGraph, ToolNode
from agentflow.core.state import AgentState, Message
from agentflow.storage.checkpointer import InMemoryCheckpointer
from agentflow.utils import END

def get_weather(location: str) -> str:
    """Get current weather for a city."""
    return f"It is sunny and 22°C in {location}."

tool_node = ToolNode([get_weather])
agent = Agent(
    model="google/gemini-2.5-flash",  # or "openai/gpt-4o-mini"
    system_prompt=[{"role": "system", "content": "Helpful assistant."}],
    tool_node="TOOL",
)

graph = StateGraph(AgentState)
graph.add_node("MAIN", agent)
graph.add_node("TOOL", tool_node)

def route(state):
    last = state.context[-1] if state.context else None
    if last and getattr(last, "tools_calls", None) and last.role == "assistant":
        return "TOOL"
    if last and last.role == "tool":
        return "MAIN"
    return END

graph.add_conditional_edges("MAIN", route, {"TOOL": "TOOL", END: END})
graph.add_edge("TOOL", "MAIN")
graph.set_entry_point("MAIN")

memory = InMemoryCheckpointer()
app = graph.compile(checkpointer=memory)

result = app.invoke(
    {"messages": [Message.text_message("Weather in Tokyo?")]},
    config={"thread_id": "demo-1"},
)
print(result["messages"][-1].text())

The notable differences:

• No @tool decorator. AgentFlow reads the type hints + docstring directly.
• Explicit graph construction. LangGraph's create_react_agent hides the graph; AgentFlow shows it. Once you've built one, the boilerplate becomes a snippet.
• Message.text_message(...) instead of HumanMessage(...).
• thread_id at top level of config, not inside configurable.
• message.text() instead of message.content.

Step-by-step migration checklist​

1. Replace imports​

# - from langchain_core.messages import HumanMessage, AIMessage
# - from langgraph.graph import StateGraph, END
# - from langgraph.prebuilt import create_react_agent
# - from langgraph.checkpoint.memory import MemorySaver
# - from langgraph.checkpoint.postgres import PostgresSaver

# + from agentflow.core.graph import Agent, StateGraph, ToolNode
# + from agentflow.core.state import AgentState, Message
# + from agentflow.storage.checkpointer import InMemoryCheckpointer, PgCheckpointer
# + from agentflow.utils import END

2. Convert tools​

LangGraph @tool → plain function. The @tool decorator does mostly type extraction, which AgentFlow does automatically.

# - @tool
# - def get_weather(location: str) -> str:
# -     """Get weather."""
# -     return ...

# + def get_weather(location: str) -> str:
# +     """Get weather."""
# +     return ...

Wrap them in ToolNode([fn1, fn2, ...]).

3. Replace create_react_agent with explicit graph​

This is the biggest mechanical change. LangGraph's prebuilt hides the graph; AgentFlow asks you to define it. Use the snippet above as the template. It is the same for any ReAct agent.

If you have many ReAct agents, factor the graph construction into a helper:

def build_react_graph(agent: Agent, tool_node: ToolNode):
    g = StateGraph(AgentState)
    g.add_node("MAIN", agent)
    g.add_node("TOOL", tool_node)
    g.add_conditional_edges("MAIN", _route, {"TOOL": "TOOL", END: END})
    g.add_edge("TOOL", "MAIN")
    g.set_entry_point("MAIN")
    return g.compile()

4. Convert state schema​

If you used a custom TypedDict:

# LangGraph
class MyState(TypedDict):
    messages: Annotated[list[AnyMessage], add_messages]
    user_id: str
    cart: list[dict]

AgentFlow's AgentState already has messages and context. Add custom fields by subclassing:

from agentflow.core.state import AgentState

class MyState(AgentState):
    user_id: str = ""
    cart: list[dict] = []

Pass MyState to StateGraph(MyState). The runtime treats your fields as state slots.

5. Convert checkpointer​

# - memory = MemorySaver()
# + memory = InMemoryCheckpointer()

# - with PostgresSaver.from_conn_string("postgresql://...") as cp:
# -     app = workflow.compile(checkpointer=cp)
# + cp = PgCheckpointer(
# +     db_url="postgresql+asyncpg://user:password@localhost/agentflow",
# +     redis_url="redis://localhost:6379/0",
# + )
# + app = graph.compile(checkpointer=cp)

PgCheckpointer requires both Postgres and Redis. Redis is for hot-path access; if you do not have one yet, run redis:7-alpine in Docker. See production checkpointing.

6. Update invoke calls​

# - app.invoke(
# -     {"messages": [HumanMessage("hi")]},
# -     config={"configurable": {"thread_id": "u1"}},
# - )

# + app.invoke(
# +     {"messages": [Message.text_message("hi")]},
# +     config={"thread_id": "u1", "recursion_limit": 25},
# + )

recursion_limit lives on the invoke config in AgentFlow. Set it; LangGraph's default is also bounded but AgentFlow asks you to be explicit.

7. Update reading state​

# - response = result["messages"][-1].content
# + response = result["messages"][-1].text()

Message.text() returns plain text. For multimodal messages, use .parts.

8. Drop your custom FastAPI server​

If you were running a hand-rolled FastAPI service to expose LangGraph over HTTP:

agentflow init
agentflow api --host 0.0.0.0 --port 8000

agentflow.json:

{"agent": "graph.react:app"}

You now have POST /v1/graph/invoke, POST /v1/graph/stream, and GET /v1/graph/threads/{id}. The hand-rolled FastAPI usually deletes 100–300 lines.

9. Replace your TypeScript fetcher​

If you wrote a custom fetch + SSE parser to call LangGraph from a frontend:

// - const response = await fetch("/api/agent", { method: "POST", body: JSON.stringify(...) });
// - // ... custom SSE parsing ...

// + import {AgentFlowClient, Message} from "@10xscale/agentflow-client";
// + const client = new AgentFlowClient({baseUrl: "/api"});
// + const result = await client.invoke([Message.text_message(text)], {config: {thread_id}});

The typed client handles SSE, reconnection, and types.

Multi-agent migrations​

For LangGraph multi-agent workflows (router → specialists, supervisors, handoffs), the migration follows the same pattern:

• LangGraph Command for handoffs → AgentFlow create_handoff_tool
• LangGraph supervisor pattern → AgentFlow router node + handoff tools

See the handoff tutorial for the full pattern. The multi-agent orchestration post covers when each shape is the right call.

What you keep​

You do not have to rewrite:

• Tool implementations. Plain Python functions move over as-is.
• External integrations. Vector stores, databases, custom retrievers. All unchanged.
• System prompts. Copy them across.
• Evals and tests. Most of them only care about the input/output of the agent, which stays the same.

What changes is the orchestration layer in the middle.

After the migration​

Once the port is done, you usually want to:

1. Add streaming. Use app.stream() / app.astream() and the SSE endpoint
2. Move to PgCheckpointer. Production durability
3. Add the TypeScript client. Kill the hand-rolled fetcher
4. Set recursion_limit. Explicitly cap loops
5. Add OpenTelemetry traces. See production observability

These are all optional but each saves real time downstream.

Common migration gotchas​

1. Forgetting recursion_limit. LangGraph's default is 25; AgentFlow asks you to set it. If your agent loops, this is the first place to check.
2. message.content → message.text(). Easy to miss in tests.
3. configurable config key. AgentFlow uses top-level config, not nested configurable. Update everywhere.
4. Custom state schema. If you used Annotated reducers (e.g., add_messages), AgentFlow handles the messages reducer for you. Custom reducers go on subclassed state fields.
5. Tool docstrings. AgentFlow reads them as the tool description for the model. Make sure they are accurate.

Further reading​

• AgentFlow vs LangGraph. Head-to-head
• State graph concept
• Add memory. Checkpointer setup
• Handoff between agents
• Multi-agent orchestration patterns

If you want to compare the runtime first, Get started and reproduce your LangGraph agent in 30 minutes.

Here is the path from compiled graph to production on AWS. Docker image, ECS Fargate, RDS, secrets, scaling.

The shape of a production deployment​

For a Python agent service on AWS, the minimum viable architecture is:

ALB → ECS Fargate (your agent) → RDS Postgres (checkpointer)
                              → ElastiCache Redis (hot path)
                              → Secrets Manager (API keys)
                              → CloudWatch (logs)

Each box is one or two days of work. The agent itself is the small part.

Step 1: Dockerfile​

A small, reproducible image:

FROM python:3.12-slim

# Reduce image size, avoid pyc files baked in by package install
ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    PIP_NO_CACHE_DIR=1 \
    PIP_DISABLE_PIP_VERSION_CHECK=1

WORKDIR /app

# Install system deps if you need them (e.g. for asyncpg)
RUN apt-get update && apt-get install -y --no-install-recommends \
    libpq-dev gcc \
    && rm -rf /var/lib/apt/lists/*

# Install Python deps first for layer caching
COPY pyproject.toml uv.lock ./
RUN pip install uv && uv pip install --system --no-cache-dir -e .

# Copy app code last
COPY . .

# AgentFlow CLI
RUN pip install --no-cache-dir 10xscale-agentflow-cli

EXPOSE 8000

# Use exec form so signals reach the process
CMD ["agentflow", "api", "--host", "0.0.0.0", "--port", "8000"]

If you prefer to skip the Dockerfile, AgentFlow's CLI can scaffold one:

agentflow init
agentflow generate-docker

See Generate Docker files for the generated content.

Step 2: Local docker-compose for development​

version: "3.9"

services:
  agent:
    build: .
    ports: ["8000:8000"]
    environment:
      DATABASE_URL: postgresql+asyncpg://postgres:postgres@db:5432/agentflow
      REDIS_URL: redis://redis:6379/0
      OPENAI_API_KEY: ${OPENAI_API_KEY}
      GOOGLE_API_KEY: ${GOOGLE_API_KEY}
    depends_on: [db, redis]

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: agentflow
    ports: ["5432:5432"]
    volumes:
      - pgdata:/var/lib/postgresql/data

  redis:
    image: redis:7-alpine
    ports: ["6379:6379"]

volumes:
  pgdata:

Run docker compose up, send a request to localhost:8000. If this works, ECS will work. Only the configuration changes.

Step 3: Build and push to ECR​

aws ecr create-repository --repository-name agentflow-app
aws ecr get-login-password --region us-east-1 \
  | docker login --username AWS --password-stdin <account>.dkr.ecr.us-east-1.amazonaws.com

docker build -t agentflow-app:latest .
docker tag agentflow-app:latest <account>.dkr.ecr.us-east-1.amazonaws.com/agentflow-app:latest
docker push <account>.dkr.ecr.us-east-1.amazonaws.com/agentflow-app:latest

For CI, use the official aws-actions/amazon-ecr-login GitHub Action.

Step 4: RDS Postgres for the checkpointer​

Create the smallest instance that fits your throughput. For a starting point: db.t4g.medium, single-AZ, 50 GB storage.

aws rds create-db-instance \
  --db-instance-identifier agentflow-prod \
  --db-instance-class db.t4g.medium \
  --engine postgres \
  --master-username agentflow \
  --master-user-password 'use-secrets-manager-instead' \
  --allocated-storage 50 \
  --vpc-security-group-ids sg-xxx \
  --db-subnet-group-name your-private-subnet-group

Connection string format: postgresql+asyncpg://agentflow:<password>@<host>:5432/agentflow. Pass via Secrets Manager (next step), not env vars in the task definition.

Step 5: ElastiCache Redis for hot-path state​

cache.t4g.small works for most production loads. Single-shard, no clustering needed initially.

Connection string: redis://<host>:6379/0.

Step 6: Secrets Manager​

Never put OPENAI_API_KEY in plaintext in your task definition. Create a secret per credential:

aws secretsmanager create-secret \
  --name agentflow/prod/openai \
  --secret-string '{"OPENAI_API_KEY":"sk-..."}'

Reference it in the ECS task definition's secrets block, not environment.

Step 7: ECS Fargate task definition​

{
  "family": "agentflow-app",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "1024",
  "memory": "2048",
  "containerDefinitions": [
    {
      "name": "agent",
      "image": "<account>.dkr.ecr.us-east-1.amazonaws.com/agentflow-app:latest",
      "portMappings": [{"containerPort": 8000, "protocol": "tcp"}],
      "environment": [
        {"name": "DATABASE_URL", "value": "postgresql+asyncpg://agentflow:_/agentflow.cluster-xxx.us-east-1.rds.amazonaws.com:5432/agentflow"},
        {"name": "REDIS_URL", "value": "redis://agentflow.xxx.cache.amazonaws.com:6379/0"}
      ],
      "secrets": [
        {"name": "OPENAI_API_KEY", "valueFrom": "arn:aws:secretsmanager:...:secret:agentflow/prod/openai:OPENAI_API_KEY::"}
      ],
      "healthCheck": {
        "command": ["CMD-SHELL", "curl -f http://localhost:8000/health || exit 1"],
        "interval": 30,
        "timeout": 5,
        "retries": 3,
        "startPeriod": 30
      },
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/agentflow-app",
          "awslogs-region": "us-east-1",
          "awslogs-stream-prefix": "agent"
        }
      }
    }
  ]
}

Notes:

• 1 vCPU + 2 GB is plenty for an I/O-bound agent service. Add CPU only if you have CPU-bound tools.
• Health checks hit /health (AgentFlow exposes this by default).
• Logs flow to CloudWatch; route from there to your log platform.

Step 8: ALB and SSL​

Put the service behind an Application Load Balancer with HTTPS:

• Target group: HTTP 8000, health check on /health
• Listener: HTTPS 443 with an ACM certificate
• Idle timeout: 300 seconds. Important for SSE streams

If your idle timeout is too low (default is 60s), long agent streams get cut. Tune it.

Step 9: Autoscaling​

For an agent service, scale on CPU (less likely the bottleneck) or active connections per task (more likely). Define a target tracking policy:

• Target: 30 active connections per task
• Min: 2 tasks (HA across AZs)
• Max: 20 tasks

LLM-bound services are usually I/O-bound. CPU autoscaling underestimates load. Connection-based scaling is closer to truth.

Step 10: Observability​

CloudWatch is the default; most teams ship logs onward to Datadog, Honeycomb, or self-hosted Loki. Either way:

• Subscribe the log group to your platform's ingestion
• Add the OTEL collector as a sidecar if you want traces
• Define alarms on:
  • Service UnhealthyHostCount > 0 for 5 min
  • p95 latency > 5 s for 5 min
  • Stream-error rate > 5% for 5 min

The production observability post covers what to log and trace inside the agent.

CI/CD with GitHub Actions​

A reasonable pipeline:

name: deploy
on:
  push:
    branches: [main]

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4
      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::<account>:role/github-actions-deploy
          aws-region: us-east-1
      - uses: aws-actions/amazon-ecr-login@v2
      - run: |
          docker build -t agentflow-app:${{ github.sha }} .
          docker tag agentflow-app:${{ github.sha }} ${{ env.ECR_REGISTRY }}/agentflow-app:${{ github.sha }}
          docker push ${{ env.ECR_REGISTRY }}/agentflow-app:${{ github.sha }}
      - uses: aws-actions/amazon-ecs-deploy-task-definition@v2
        with:
          service: agentflow-app
          cluster: agentflow-prod
          task-definition: ./.aws/task-def.json

Use OIDC for credentials (the id-token: write block). Never long-lived AWS keys in GitHub.

What about Cloud Run / Kubernetes?​

The same architecture works on:

• Google Cloud Run. Replace ECS with Cloud Run, RDS with Cloud SQL, ElastiCache with Memorystore. Slightly simpler ops; same code.
• GKE / EKS. Replace ECS task definition with a Deployment + Service. More flexible, more ops.
• Fly.io / Render. Containers are containers. Mostly the same.

Pick whichever your team already operates. The Python agent does not care.

Cost expectations (rough)​

A small production agent service on AWS, moderate traffic:

Most teams' bills are 10× the infra in token costs. Plan accordingly.

Common gotchas​

1. ALB idle timeout < SSE stream length. Streams get cut mid-response. Bump to 300s.
2. Database connection pool too small. A PgCheckpointer writes per node; under load, sized it for n_tasks × n_concurrent_streams × 2.
3. Secrets in plaintext env vars. Use Secrets Manager and secrets blocks.
4. No health check. ECS happily routes traffic to dead pods. /health is one line of code.
5. Single-AZ RDS. Cheap until your AZ has an outage. For real production, multi-AZ.

Further reading​

• Production runtime concept
• Deployment guide. Full deploy walkthrough
• Generate Docker files
• Production observability + retries
• Auth and authorization

For a working stack to start from, Get started. The same agentflow api server you run locally is what runs in your ECS task.

Here is the end-to-end pattern, with the failure modes baked in.

What ReAct actually is​

ReAct is a loop:

1. Reason. The model thinks about what to do
2. Act. The model picks a tool and calls it with arguments
3. Observe. The tool returns a result; the model reads it
4. Loop. Back to Reason, until the model produces a final answer

In a graph runtime, this is two nodes (MAIN and TOOL) with a conditional edge between them.

The shape of a real tool​

Let's build a tool that hits a real API. The Open-Meteo weather service (no auth required for testing).

import httpx
from dataclasses import dataclass

@dataclass
class WeatherResult:
    location: str
    temperature_c: float
    description: str

def get_weather(location: str) -> str:
    """Get current weather for a city. Returns a one-line summary."""
    try:
        # Step 1 — geocode
        geo = httpx.get(
            "https://geocoding-api.open-meteo.com/v1/search",
            params={"name": location, "count": 1},
            timeout=5.0,
        ).json()
        if not geo.get("results"):
            return f"Could not find a location named '{location}'. Ask the user to clarify."

        latitude = geo["results"][0]["latitude"]
        longitude = geo["results"][0]["longitude"]

        # Step 2 — current weather
        weather = httpx.get(
            "https://api.open-meteo.com/v1/forecast",
            params={"latitude": latitude, "longitude": longitude, "current_weather": True},
            timeout=5.0,
        ).json()
        cw = weather["current_weather"]
        return f"{location}: {cw['temperature']}°C, wind {cw['windspeed']} km/h."

    except httpx.TimeoutException:
        return f"Weather API timed out for {location}. Tell the user to try again later."
    except httpx.HTTPError as e:
        return f"Weather API error for {location}: {e}. Suggest a fallback or move on."

Key choices:

• Plain Python function with type hints and a docstring. AgentFlow exposes this to the model automatically.
• Returns a string, not a dict. The model reads strings best; structure goes in the wording, not the type.
• Errors become results, not exceptions. The agent reads the error message and decides how to recover.
• Timeouts are explicit. 5.0 seconds is a reasonable budget per tool call.

The agent​

from agentflow.core.graph import Agent, StateGraph, ToolNode
from agentflow.core.state import AgentState, Message
from agentflow.utils import END

tool_node = ToolNode([get_weather])

agent = Agent(
    model="google/gemini-2.5-flash",
    system_prompt=[{
        "role": "system",
        "content": (
            "You are a weather assistant. Use get_weather to answer questions. "
            "If the tool returns an error, acknowledge the error and suggest the user try again or pick a different city. "
            "Do not invent weather data."
        ),
    }],
    tool_node="TOOL",
)

graph = StateGraph(AgentState)
graph.add_node("MAIN", agent)
graph.add_node("TOOL", tool_node)

def route(state):
    last = state.context[-1] if state.context else None
    if last and getattr(last, "tools_calls", None) and last.role == "assistant":
        return "TOOL"
    if last and last.role == "tool":
        return "MAIN"
    return END

graph.add_conditional_edges("MAIN", route, {"TOOL": "TOOL", END: END})
graph.add_edge("TOOL", "MAIN")
graph.set_entry_point("MAIN")
app = graph.compile()

The system prompt is doing important work. It tells the model how to handle tool errors. Without that instruction, models often hallucinate weather data when the tool fails.

Running it​

result = app.invoke(
    {"messages": [Message.text_message("What's the weather in Tokyo and Bengaluru?")]},
    config={"thread_id": "weather-1", "recursion_limit": 10},
)
print(result["messages"][-1].text())

The agent will call get_weather twice (once per city), then synthesize the answer. With recursion_limit=10 we cap the loop in case the model decides to retry forever.

Tool design tips​

Keep the surface small​

A tool with 8 optional parameters confuses the model. Two common patterns:

• One tool, one job. get_weather(location), not query_weather(location, date, unit, fields).
• Multiple specific tools. get_current_weather, get_forecast, get_historical_weather. Each does one thing.

Use string-typed parameters when you can​

Models pass strings reliably. They sometimes mangle complex types. If your underlying API takes a date, accept a date string in the tool:

def get_forecast(location: str, date: str) -> str:
    """Get weather forecast for a city on a specific date.

    Args:
        location: City name.
        date: ISO date like 2026-04-15.
    """
    try:
        d = datetime.date.fromisoformat(date)
    except ValueError:
        return f"Invalid date '{date}'. Use ISO format like 2026-04-15."
    ...

Validate inside the tool. Don't trust the model to obey type hints.

Return useful, scannable results​

Models read tool results like a human reads a search result. A useful result has:

• A clear answer line at the top
• Source / context so the model can cite it
• An error explanation when something goes wrong, with what to do next

Bad: {"data": {"temp": 22, "wind": 3.4}}. The model has to interpret it.

Good: Tokyo: 22°C, wind 3.4 km/h. Source: open-meteo.com. The model just reads it.

Bound the result size​

A tool that dumps 50 KB of API response into the context will blow your token budget on the next turn. Trim before returning:

def search_internal_docs(query: str) -> str:
    """Search internal documentation."""
    hits = vector_client.search(query, top_k=3)
    summary = "\n\n".join(
        f"[{h.id}] {h.title}\n{h.text[:300]}..." for h in hits
    )
    return summary or "No matching docs found."

Three results × 300 chars = 1 KB. Plenty for the model, manageable cost.

Multi-tool, structured-output ReAct​

A common pattern: ReAct agent that needs to return a structured result, not free-form text. Use a final "submit answer" tool:

from typing import TypedDict

class WeatherSummary(TypedDict):
    location: str
    temperature_c: float
    advice: str

def submit_summary(summary: WeatherSummary) -> str:
    """Call this when you have a final answer. Stores the structured summary."""
    # Persist or return; AgentFlow surfaces this in state for the caller
    return f"Stored summary for {summary['location']}."

tool_node = ToolNode([get_weather, submit_summary])

The agent loops with get_weather, then calls submit_summary once it has all the data. You read the structured result from state after the run finishes.

For a runnable structured-output example, see react-agent-validation.

Persistence​

The agent above is single-shot. For a real assistant, add a checkpointer:

from agentflow.storage.checkpointer import PgCheckpointer

app = graph.compile(checkpointer=PgCheckpointer(
    db_url="postgresql+asyncpg://user:password@localhost/agentflow",
    redis_url="redis://localhost:6379/0",
))

# Same thread_id in subsequent calls reuses prior history
app.invoke(..., config={"thread_id": "user-42"})

Now turn 2 of the conversation knows what was said in turn 1. See add memory for the full pattern.

Common ReAct pitfalls​

1. Forgetting recursion_limit. Default it to 10–25; the model will sometimes loop on errors.
2. Letting tools throw. Catch and return as a string. The agent should read the error.
3. Tools that block too long. Set HTTP timeouts (3–10 s). A 30-second tool kills the user experience.
4. Tools with hidden side effects. A tool that writes to a database needs an idempotency key. See the production post.
5. Ambiguous tool names. lookup vs get_weather. Be specific. Models choose tools by name and description.

Further reading​

• Add a tool. Beginner walkthrough
• Tool decorator tutorial. Advanced patterns
• ReAct agent example. Runnable code
• Streaming agent responses
• Multi-agent orchestration patterns

To get a working ReAct agent in five minutes, Get started. The quickstart walks the same pattern, end to end.

These are the patterns we see in production Python agent codebases, and the failure modes they prevent.

Observability: what to log​

The minimum you should record per agent run:

Emit one log line per node boundary, plus one summary line per request. Anything more granular goes to traces, not logs.

Correlation​

Use thread_id as your trace correlation ID. Every downstream call (tool HTTP request, vector query, database write) should carry it. When a user reports "the agent went weird at 3pm," you grep thread_id and reconstruct the entire run.

Structured logging​

import structlog

logger = structlog.get_logger()

def log_node(state, node):
    logger.info("agent.node",
        thread_id=state.config.get("thread_id"),
        node=node,
        message_count=len(state.context),
        last_role=state.context[-1].role if state.context else None,
    )

JSON logs go to your platform of choice (Loki, Datadog, OpenSearch). Avoid pretty-printed logs in production. They cost CPU and lose searchability.

Tracing with OpenTelemetry​

For deeper insight (per-tool latency, model timing, span trees):

from opentelemetry import trace

tracer = trace.get_tracer("agentflow.app")

def get_weather(location: str) -> str:
    with tracer.start_as_current_span("tool.get_weather") as span:
        span.set_attribute("location", location)
        result = weather_client.get(location)
        span.set_attribute("weather.summary", result.summary)
        return result.text()

Pipe to Tempo, Jaeger, Honeycomb, or whatever you have. The agent runtime is one boundary; the tools are the other. Trace both.

Cost tracking​

Tokens are money. Add a per-run total:

async for chunk in app.astream(...):
    if chunk.event == StreamEvent.MESSAGE and chunk.message.usage:
        total_prompt_tokens += chunk.message.usage.prompt_tokens
        total_completion_tokens += chunk.message.usage.completion_tokens

logger.info("agent.cost",
    thread_id=tid,
    prompt_tokens=total_prompt_tokens,
    completion_tokens=total_completion_tokens,
    model=model,
)

A daily SQL query against this gives you cost per user, per feature, per model. Invaluable when finance asks why the bill jumped.

Retries: where to put them​

LLM calls and tool calls fail. There are three failure layers, each with different retry rules:

Layer 1: provider transient errors​

Rate limits (429), gateway timeouts (502/504), connection resets. Always retry with exponential backoff:

from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception_type
from openai import RateLimitError, APIConnectionError

@retry(
    stop=stop_after_attempt(5),
    wait=wait_exponential(multiplier=1, min=1, max=30),
    retry=retry_if_exception_type((RateLimitError, APIConnectionError)),
)
def call_llm(...):
    ...

Most provider SDKs (OpenAI, Anthropic, Google) have built-in retry; configure it once.

Layer 2: tool failures​

Your tool calls a real API. The API returns 500. What do you do?

The wrong answer: retry inside the tool silently and let the agent think the call succeeded.

The right answer: surface the error to the agent as a tool result, and let the agent decide:

def get_weather(location: str) -> str:
    """Get current weather for a location."""
    try:
        return weather_client.get(location).text()
    except Exception as e:
        return f"Error fetching weather: {e}. Try a different city or fall back to a general estimate."

The agent, seeing the error, can retry the tool, ask the user, or move on. This is far more robust than swallowing errors.

Layer 3: graph-level retries​

For a critical action (a payment, an email send), wrap the node in an idempotency-aware retry:

def charge_node(state):
    idempotency_key = f"{state.config['thread_id']}-{state.config.get('attempt', 0)}"
    return payment_client.charge(amount=state.amount, key=idempotency_key)

Every external write needs an idempotency key. Without it, retries become double-billing.

Idempotency​

Two rules:

1. Reads are free to retry. Search, look-up, fetch. Retry as much as you want.
2. Writes need an idempotency key. Charges, emails, ticket creation, file uploads.

The key is usually thread_id + node_name + attempt_count. Provide it to the upstream API; if the API does not support keys, use a small dedup table:

CREATE TABLE agent_idempotency (
  key VARCHAR PRIMARY KEY,
  result JSONB,
  created_at TIMESTAMPTZ DEFAULT NOW()
);

INSERT ... ON CONFLICT DO NOTHING RETURNING result. If the key existed, return the previous result without re-doing the action.

Graceful shutdown​

When your container gets a SIGTERM (deploy, autoscale-down, spot reclaim), you have ~30 seconds to clean up. For an agent server, that means:

1. Stop accepting new requests. Most ASGI servers do this automatically.
2. Let in-flight requests finish. Or at least checkpoint their state.
3. Close the database pool.
4. Flush logs and traces.

AgentFlow's CLI handles 1, 3, and 4. For 2. Letting in-flight runs finish. The checkpointer is your friend: even if the request is killed, the graph state is durable, and a new replica can resume the thread.

The pattern:

# In your FastAPI / agent entrypoint
import signal

shutdown = asyncio.Event()

def _on_term(*args):
    shutdown.set()

signal.signal(signal.SIGTERM, _on_term)
signal.signal(signal.SIGINT, _on_term)

# In long-running loops, periodically check shutdown.is_set()

For a runnable example, see the graceful shutdown tutorial.

SLOs that actually mean something​

For an agent service, the useful SLOs are:

• TTFB (time-to-first-byte) p95 < 1.5 s. First token latency the user feels
• Stream-success rate ≥ 99.5%. Streams that complete without error
• Tool-error rate < 2%. Tool failures bubbling up to user
• Cost per request p95 < $X. Pick a number for your model mix

Latency to completion is less useful for streaming agents. Users care about first token, not total time.

Failure modes you will hit​

1. Recursion runaway. Always set recursion_limit (10–25 for most workflows).
2. Token blowout. A long thread + verbose tool output can blow past the model's context. Trim aggressively.
3. Provider outage. Have a fallback model; provider abstractions make this a config change.
4. Vector store unavailable. If long-term memory is down, degrade gracefully. Note in the logs and continue without it.
5. Database lock contention. PgCheckpointer writes per node; under load, you need a connection pool sized appropriately.

A quick checklist before you go live​

• Structured logs with thread_id correlation
• OpenTelemetry traces for tool calls and LLM calls
• Cost (token) tracking per run
• Provider-level retries configured
• Tool errors surfaced as results, not exceptions
• Idempotency keys on every external write
• recursion_limit set on every invoke
• Fallback model for provider outages
• Graceful shutdown with checkpointer-backed resume
• SLOs defined and dashboards in place

This is the boring stuff that keeps the pager quiet.

Further reading​

• Production runtime concept
• Auth and authorization
• Production checkpointing
• Deployment guide
• Graceful shutdown example

If you are starting from a prototype and shipping to prod, Get started. The runtime gives you most of this checklist for free.

Here is the production-shaped pattern.

Why SSE, not WebSockets​

For agent responses, SSE wins on the things that matter:

• HTTP/2 friendly. Works through every proxy and CDN
• Automatic reconnect. Browser does it for you
• One-way. Exactly the data shape you need
• Auth is just a header. No special handshake

WebSockets win for full-duplex, low-latency interaction (collaborative editing, multiplayer). For chat / agent streaming, SSE is simpler and more reliable.

What an event stream actually looks like​

Each event is a line beginning with data: followed by a JSON payload. AgentFlow's stream emits events with a discriminator field:

event: message_chunk
data: {"role": "assistant", "content": "Looking up "}

event: message_chunk
data: {"role": "assistant", "content": "the weather"}

event: tool_start
data: {"name": "get_weather", "args": {"location": "Tokyo"}}

event: tool_end
data: {"name": "get_weather", "output": "Sunny, 22°C"}

event: message_chunk
data: {"role": "assistant", "content": "Tokyo is sunny..."}

event: done
data: {"thread_id": "user-1", "tokens": 124}

Your frontend parses each event and updates the UI accordingly: append text on message_chunk, show a "calling tool..." indicator on tool_start, hide it on tool_end.

Server side: AgentFlow's built-in API​

The fastest path: use agentflow api, which already serves SSE-correctly.

agentflow init
agentflow api --host 0.0.0.0 --port 8000

POST /v1/graph/stream is the streaming endpoint. No FastAPI to wire yourself.

If you need custom routing, auth, or pre-processing, mount AgentFlow inside your existing FastAPI app instead.

Server side: custom FastAPI integration​

Embedding AgentFlow's stream in a FastAPI route:

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
import json

from agentflow.core.state import Message
from agentflow.utils import ResponseGranularity
from agentflow.core.state.stream_chunks import StreamEvent

from my_app.graph import app as agent_app  # your compiled graph

api = FastAPI()

class Body(BaseModel):
    thread_id: str
    text: str

@api.post("/agent/stream")
async def stream(body: Body):
    async def gen():
        try:
            async for chunk in agent_app.astream(
                {"messages": [Message.text_message(body.text)]},
                config={"thread_id": body.thread_id, "recursion_limit": 25},
                response_granularity=ResponseGranularity.LOW,
            ):
                if chunk.event == StreamEvent.MESSAGE and chunk.message:
                    payload = {"role": chunk.message.role, "content": chunk.message.text()}
                    yield f"event: message_chunk\ndata: {json.dumps(payload)}\n\n"
                elif chunk.event == StreamEvent.TOOL_START:
                    yield f"event: tool_start\ndata: {json.dumps(chunk.payload)}\n\n"
                elif chunk.event == StreamEvent.TOOL_END:
                    yield f"event: tool_end\ndata: {json.dumps(chunk.payload)}\n\n"
            yield "event: done\ndata: {}\n\n"
        except Exception as e:  # noqa: BLE001
            yield f"event: error\ndata: {json.dumps({'message': str(e)})}\n\n"

    return StreamingResponse(gen(), media_type="text/event-stream", headers={
        "Cache-Control": "no-cache",
        "X-Accel-Buffering": "no",  # disable nginx buffering
    })

Notes:

• X-Accel-Buffering: no prevents nginx from holding events in a buffer
• Trailing \n\n ends an SSE event. Required by the spec
• recursion_limit caps runaway loops at the API boundary
• Catch and emit error events rather than crashing mid-stream

Frontend: vanilla browser​

Modern browsers ship EventSource, but it does not support custom headers (auth) for non-GET requests. For POST + auth, use fetch with a manual SSE parser or @microsoft/fetch-event-source.

If you do not need auth, a vanilla EventSource is fine:

const es = new EventSource("/agent/stream?thread_id=u1&text=hello");
es.addEventListener("message_chunk", (e) => {
  const {content} = JSON.parse(e.data);
  appendToUI(content);
});
es.addEventListener("done", () => es.close());

Frontend: TypeScript with auth​

Use AgentFlow's typed client. It handles SSE parsing, reconnection, and Auth headers:

import {AgentFlowClient, Message} from "@10xscale/agentflow-client";

const client = new AgentFlowClient({
  baseUrl: "/api",  // or full URL
  headers: {Authorization: `Bearer ${token}`},
});

for await (const chunk of client.stream(
  [Message.text_message("Plan my trip to Tokyo.")],
  {config: {thread_id: "user-1"}},
)) {
  if (chunk.type === "message_chunk") {
    appendToUI(chunk.content ?? "");
  } else if (chunk.type === "tool_start") {
    showToolIndicator(chunk.name);
  } else if (chunk.type === "tool_end") {
    hideToolIndicator();
  }
}

The type definitions match the server's event shape. No untyped JSON in your UI.

Reconnection and resume​

EventSource reconnects automatically with the last-event-id header. For agent streams, this is harder: you do not want to replay the whole conversation, just the events the client missed.

The robust pattern:

1. Client requests ?thread_id=X
2. Server starts streaming from the current graph state (the checkpointer holds it)
3. On disconnect, the server keeps running (do not bind the agent to the SSE response)
4. Client reconnects with the same thread_id and reads only new events

Implementing step 3 cleanly requires a publisher / pub-sub between the agent and the SSE handler. For the "good enough" version: persist progress to the checkpointer and have the client poll-then-reconnect.

For the production version, see Streaming concept and stream-sync tutorial.

Backpressure​

If the client is slow (mobile, bad network), and the agent is fast, the buffer fills. Two choices:

• Drop intermediate token chunks when the client is behind (acceptable for chat)
• Batch tokens into 50–100ms windows (smooths jitter, costs ~1 frame of latency)

AgentFlow's ResponseGranularity setting controls this.LOW for individual tokens, higher for batched chunks.

Auth and rate limits​

A few rules of thumb:

1. Authenticate the request, not the SSE connection. Pass a JWT in the body of the initial POST; the SSE response is implicitly authenticated.
2. Rate limit by user, not by IP. Agent users sit behind shared NATs.
3. Cap recursion_limit server-side. Do not trust the client to send a sane value.
4. Add a per-stream timeout (60–180 s). Long stalls usually mean the model is stuck; better to fail than hang.

See Auth and authorization for AgentFlow's auth options.

Common gotchas​

• CORS preflight blocks SSE if the browser sends Access-Control-Request-Headers with custom headers. Use client.stream or fetch-event-source, not raw EventSource, when auth is involved.
• Server-side proxy buffering. Nginx, Cloudflare, and AWS ALBs all need explicit "no buffering" hints.
• Long-running connections + autoscaling. SSE connections are sticky; account for that in your scaling policy (use connection draining, not hard kills).
• Mobile networks drop frequently. Reconnection logic is not optional.

Further reading​

• Streaming concept
• Stream-sync tutorial
• Production deployment
• Stop a stream. Handling client cancellation

When you are ready, Get started. The default agentflow api command serves correctly-formatted SSE out of the box.

Here is the practical taxonomy and how to wire each one in Python.

Three kinds of memory, three tools​

Most teams need all three. Most "memory" libraries solve one and leave you to assemble the rest.

Short-term memory: threads and checkpointers​

A thread is a single conversation. AgentFlow snapshots the entire graph state after every node, keyed by thread_id:

from agentflow.core.graph import Agent, StateGraph
from agentflow.core.state import AgentState, Message
from agentflow.storage.checkpointer import InMemoryCheckpointer
from agentflow.utils import END

agent = Agent(model="google/gemini-2.5-flash",
    system_prompt=[{"role": "system", "content": "Helpful assistant."}])

graph = StateGraph(AgentState)
graph.add_node("assistant", agent)
graph.set_entry_point("assistant")
graph.add_edge("assistant", END)

app = graph.compile(checkpointer=InMemoryCheckpointer())

# Turn 1
app.invoke(
    {"messages": [Message.text_message("My name is Alex.")]},
    config={"thread_id": "user-42"},
)
# Turn 2 — same thread_id pulls the prior history
app.invoke(
    {"messages": [Message.text_message("What's my name?")]},
    config={"thread_id": "user-42"},
)

Key properties:

• No code changes when you swap the storage backend
• Same thread_id = same conversation context
• Automatic. You do not assemble the prompt yourself

Production storage​

InMemoryCheckpointer is for development. Switch to Postgres + Redis with one import:

from agentflow.storage.checkpointer import PgCheckpointer

checkpointer = PgCheckpointer(
    db_url="postgresql+asyncpg://user:password@localhost/agentflow",
    redis_url="redis://localhost:6379/0",
)
app = graph.compile(checkpointer=checkpointer)

Postgres holds the durable history; Redis is used for fast hot-path access. See checkpointing concept and the production checkpointing guide for tuning.

Trimming long threads​

Threads grow. At ~100 turns, you start paying for tokens you do not need. Two common patterns:

1. Sliding window. Keep last N turns, drop the rest
2. Summary head. Every 20 turns, replace the oldest 15 with a summary message

agent = Agent(
    model="google/gemini-2.5-flash",
    system_prompt=[{"role": "system", "content": "Helpful assistant."}],
    trim_context=True,  # AgentFlow's built-in trimming
)

For custom strategies, see State and messages.

Long-term memory: semantic recall​

Threads cover "what did this user say earlier?" They do not cover "what did any user say about pricing in the last 90 days?" or "what did this user prefer 6 months ago?"

For that, expose a vector store as a tool:

from agentflow.core.graph import Agent, StateGraph, ToolNode

def remember(query: str) -> str:
    """Search long-term memory for relevant past interactions."""
    # qdrant / pgvector / pinecone / your choice
    hits = vector_client.search(query, top_k=3)
    return "\n".join(f"[{h.id}] {h.text}" for h in hits)

def write_to_memory(text: str, tags: str) -> str:
    """Store a notable fact for future recall."""
    vector_client.upsert(text=text, metadata={"tags": tags})
    return "Saved."

tool_node = ToolNode([remember, write_to_memory])
agent = Agent(
    model="google/gemini-2.5-flash",
    system_prompt=[{"role": "system", "content": (
        "Use `remember` to recall past info and `write_to_memory` "
        "to save important new facts. Cite memory IDs."
    )}],
    tool_node="TOOL",
)

The agent decides when to recall and when to write. This is materially different from naively dumping the entire vector index into context. It scales and stays auditable.

For a runnable example with Qdrant, see the qdrant-memory tutorial.

When to write vs when to query​

A useful default:

• Always query at the start of a conversation if the user is identified
• Write when the agent encounters a stable fact (preference, identity, decision). Not raw chat
• Tag aggressively so retrieval can filter by user, tenant, or topic

This keeps the long-term store as a curated knowledge base, not a chat dump.

Resumable runs: checkpointing for interrupts and replays​

The same checkpointer that powers thread history also powers two other features:

Interrupt and resume​

Pause the graph mid-flow, get human approval, then resume:

result = app.invoke(
    {"messages": [Message.text_message("Refund order #123 for $400.")]},
    config={"thread_id": "approval-1"},
)
# Graph paused at APPROVE node; surface state to a human

# Later — same thread_id resumes from the checkpoint
app.invoke(
    {"approval": True},
    config={"thread_id": "approval-1"},
)

Time travel / replays​

Because every node-boundary state is snapshotted, you can replay a thread from any prior state. Useful for debugging "why did the agent choose tool X here?" without rerunning the whole conversation.

See the interrupts-and-resume tutorial for the details.

Putting it together: a memory architecture​

A production agent typically wires:

┌──────────────────────────────────────────┐
│  Agent (LLM)                              │
│  ├── short-term: PgCheckpointer (thread)  │
│  ├── long-term: vector tool (Qdrant)      │
│  └── resumable: same PgCheckpointer       │
└──────────────────────────────────────────┘

Each layer has a different SLA:

These are budgets, not guarantees. The checkpointer is on the hot path; the vector store is opt-in per query. Treat them differently when you scale.

Common mistakes​

1. Storing everything in the vector store. It becomes noise. Curate.
2. Skipping thread_id in dev. Then surprised it doesn't remember in prod. Always pass thread_id from day one.
3. One vector index per user. Hard to maintain. Use one index with user_id metadata and filter at query time.
4. No expiry policy. Memory is a feature; stale memory is a bug. Add TTL on long-term entries.
5. Treating chat history as long-term memory. They are not the same. The full chat goes in the checkpointer; facts extracted from chat go in the vector store.

Further reading​

• Checkpointing and threads. Concept guide
• Memory and store. Full memory architecture
• Add memory tutorial. Beginner walkthrough
• Qdrant memory example. Long-term recall
• Production checkpointing. Postgres + Redis tuning

When you are ready, Get started and you will have working short-term memory in under five minutes.

This is a survey of the five strongest LangGraph alternatives in 2026, what each one is good at, and how to decide.

Why people look for alternatives​

The most common reasons we hear:

1. LangChain dependency tax. LangGraph builds on LangChain core. If you do not need LangChain, you carry the dependencies anyway.
2. No built-in production server. LangGraph is the runtime. Serving an agent over REST/SSE is BYO FastAPI or paid LangGraph Platform.
3. No first-party TypeScript client. Most products have a JS frontend; you wrap fetch and parse SSE by hand.
4. Vendor gravity. LangSmith and LangGraph Platform are good products and pull you toward a paid stack.

If those tradeoffs work for you, stay with LangGraph. It is mature and well-supported. Otherwise, here are the five strongest alternatives.

1. AgentFlow: the closest drop-in alternative​

AgentFlow is an MIT-licensed graph-based runtime with the same mental model as LangGraph (typed StateGraph, conditional edges, checkpointers) plus a built-in REST + SSE server (agentflow api) and a typed TypeScript client (@10xscale/agentflow-client).

from agentflow.core.graph import Agent, StateGraph, ToolNode
from agentflow.core.state import AgentState, Message

agent = Agent(
    model="google/gemini-2.5-flash",
    system_prompt=[{"role": "system", "content": "Helpful assistant."}],
    tool_node="TOOL",
)

graph = StateGraph(AgentState)
graph.add_node("MAIN", agent)
graph.add_node("TOOL", ToolNode([get_weather]))
# ... edges ...
app = graph.compile()

Strengths

• Graph mental model identical to LangGraph (one of the most mechanical migrations in the field)
• Production server + TypeScript client included
• No required SaaS account, MIT license
• Multi-provider out of the box (OpenAI, Anthropic, Google, Vertex AI)

Weaknesses

• Smaller community than LangGraph today
• No equivalent of LangSmith for tracing yet (use OpenTelemetry + your existing observability)

Pick it when: you want a graph runtime plus the production stack in one project. → AgentFlow vs LangGraph

2. CrewAI: fastest path to a role-based crew​

CrewAI optimizes for declarative role + task DSL. Five lines for "researcher → writer → editor."

from crewai import Agent, Task, Crew, Process

researcher = Agent(role="Researcher", goal="...", backstory="...")
writer = Agent(role="Writer", goal="...", backstory="...")
crew = Crew(agents=[researcher, writer], tasks=[...], process=Process.sequential)
crew.kickoff()

Strengths

• Shortest distance from idea to working crew
• Hierarchical processes for runtime-dispatched delegation
• Clear conceptual model for non-engineers

Weaknesses

• Less explicit control flow than a graph
• Production patterns (persistence, API serving) require more glue than LangGraph or AgentFlow
• CrewAI Enterprise is the recommended production hosting

Pick it when: the workflow really is "roles + tasks" and you want fast prototyping. → AgentFlow vs CrewAI

3. Microsoft AutoGen: research-grade multi-agent conversations​

AutoGen 0.4 split into core + agentchat + extensions, with strong primitives for multi-agent chat, group chats, and selectors.

Strengths

• Excellent for emergent multi-agent dynamics
• Actor-style architecture in autogen-core
• AutoGen Studio is a great UI for designing flows visually

Weaknesses

• LLM-driven selectors make routing harder to debug under load
• Production server is BYO
• API surface still evolving across 0.4.x

Pick it when: you are exploring multi-agent dynamics, working in Microsoft / Azure, or want the visual designer. → AgentFlow vs AutoGen

4. LlamaIndex Agents: RAG-first agent layer​

LlamaIndex's FunctionAgent, ReActAgent, and Workflow build on top of best-in-class retrieval and indexing primitives. If your product is "chat with my documents," it is the natural starting point.

Strengths

• Best retrieval + parsing + indexing stack in Python
• Tight integration between agents and query engines
• LlamaCloud for managed parsing/indexing

Weaknesses

• Agent layer is thinner than the retrieval layer
• Less explicit multi-agent orchestration
• No bundled production server

Pick it when: retrieval is the core feature, not just one tool. (Common pattern: LlamaIndex for retrieval, AgentFlow for the runtime.) → AgentFlow vs LlamaIndex Agents

5. Google ADK: Vertex AI native​

Google's Agent Development Kit (ADK) is the official Google framework with first-party Gemini and Vertex AI support, plus Vertex AI Agent Engine for hosted execution.

Strengths

• Best-in-class Gemini integration
• Vertex AI Agent Engine for managed deployment
• Apache-2.0 license, no required commercial tier

Weaknesses

• Provider-neutral by intent, but truly best on Vertex AI
• No first-party TypeScript client
• Smaller community outside the Google ecosystem

Pick it when: you are committed to Vertex AI across data, models, and ops. → AgentFlow vs Google ADK

A decision tree​

When LangGraph is still the right pick​

We will not pretend LangGraph is wrong for everyone. It is the right answer when:

• Your codebase already depends on LangChain runnables and retrievers
• You use LangSmith for tracing and debugging
• LangGraph Platform's hosted infrastructure works for your team
• You want the largest community and the most third-party content

The "alternatives" framing is about fit, not quality. All six frameworks here ship production agents in 2026.

How to evaluate without committing​

Pick a real, small use case (2 agents, 1 tool, persistent threads) and implement it in your top two candidates. Measure:

• Cold-start time
• p95 latency under your real load
• Code size of the agent + the deployment surface
• How long it takes a new engineer to read the flow

The differences become obvious in a week. That signal is worth more than any roundup post, including this one.

Further reading​

• Get started with AgentFlow
• Best Python agent frameworks in 2026. Full roundup
• AgentFlow vs LangGraph. Head-to-head
• Multi-agent orchestration patterns

If you are curious about migration mechanics, the LangGraph → AgentFlow walkthrough covers a full port in one sitting.

Here are seven orchestration patterns we see ship in production Python codebases, and when each one is the right tool.

A quick mental model​

A graph-based runtime models multi-agent flows as nodes (agents or tools) connected by edges (control flow). State is shared across the graph, and each pattern below is just a different way of wiring the edges.

We use AgentFlow's StateGraph for the examples; the patterns translate to LangGraph, AutoGen, or CrewAI with minor syntax changes.

Pattern 1: Sequential pipeline​

The simplest case: agent A → agent B → agent C, in order.

from agentflow.core.graph import Agent, StateGraph
from agentflow.core.state import AgentState
from agentflow.utils import END

researcher = Agent(model="google/gemini-2.5-flash",
    system_prompt=[{"role": "system", "content": "Find three sources for the topic."}])
writer = Agent(model="google/gemini-2.5-flash",
    system_prompt=[{"role": "system", "content": "Use the sources to write a 200-word brief."}])
editor = Agent(model="google/gemini-2.5-flash",
    system_prompt=[{"role": "system", "content": "Tighten and copy-edit the brief."}])

graph = StateGraph(AgentState)
graph.add_node("RESEARCH", researcher)
graph.add_node("WRITE", writer)
graph.add_node("EDIT", editor)

graph.set_entry_point("RESEARCH")
graph.add_edge("RESEARCH", "WRITE")
graph.add_edge("WRITE", "EDIT")
graph.add_edge("EDIT", END)

Use when: the order is fixed and each step depends on the previous output. Content pipelines, ETL with LLM enrichment, document synthesis.

Pattern 2: Parallel fan-out + fan-in​

Run multiple specialists in parallel, then combine.

graph = StateGraph(AgentState)
graph.add_node("FETCH", source_agent)
graph.add_node("SUMMARIZE_A", summarizer_a)  # different angle
graph.add_node("SUMMARIZE_B", summarizer_b)
graph.add_node("MERGE", merger_agent)

graph.set_entry_point("FETCH")
graph.add_edge("FETCH", "SUMMARIZE_A")
graph.add_edge("FETCH", "SUMMARIZE_B")
graph.add_edge("SUMMARIZE_A", "MERGE")
graph.add_edge("SUMMARIZE_B", "MERGE")
graph.add_edge("MERGE", END)