Managing state across multi-step agent workflows in LlamaIndex — patterns?

[Arpanwanwe]

I have been using LlamaIndex for agent-like workflows-planning, execution, tool usage, and sometimes reviews. One thing I keep running into is state management once the workflow sprawls to multiple steps.

In other words, when agents hand work off to each other, context ends up split between prompts, intermediate nodes, and application code. This gets especially tricky with retries or branching logic-it's hard to tell why an agent made a certain decision or to reproduce behavior later.

I have tried to implement this by keeping an explicit shared specification/state that agents read and write, instead of relying solely on the implicit prompt context. I've been testing this with a small orchestration tool, Zenflow, in conjunction with LlamaIndex to make workflows more inspectable, but I'm still feeling out whether this is the right direction.

Wondering how people here are handling this with LlamaIndex: mostly manually managing state, using callback handlers/state handlers, or leaning on the newer abstractions for agents in the library?

[KeepALifeUS]

This is exactly the problem I've been solving! Here's the pattern that works:

Explicit Shared State (Stigmergy Pattern)

Instead of implicit context in prompts, use a structured state object that all agents read/write:

state = {
    "task": "Analyze customer data",
    "phase": "execution",
    "context": {
        "planning_decisions": ["Use pandas for analysis", "Focus on churn metrics"],
        "intermediate_results": {},
        "tool_outputs": []
    },
    "agent_reasoning": {
        "planner": "Selected pandas over SQL for flexibility",
        "executor": "Processing 10k rows in batches"
    }
}

Why This Works

1. Context is inspectable — Just read the state file to understand what happened

2. Retries are deterministic — Feed the same state snapshot, get the same behavior

3. Handoffs are explicit — Agent A writes to state, Agent B reads from state

# Agent A completes
state["phase"] = "review"
state["context"]["intermediate_results"]["analysis"] = result

# Agent B picks up
if state["phase"] == "review":
    analysis = state["context"]["intermediate_results"]["analysis"]
    # Continue with full context

4. Branching is clean

if state.get("requires_retry"):
    state["phase"] = state["retry_from_phase"]
    # Agents re-read state and continue

Results

• 80% reduction in token usage (no repeated context in prompts)
• Full reproducibility (state snapshots are your test fixtures)
• Easy debugging (diff two state files to see what changed)

This is the same pattern Anthropic independently discovered for their C compiler project. Working implementation: https://github.com/KeepALifeUS/autonomous-agents

Happy to discuss specific integration patterns with LlamaIndex!

[xXMrNidaXx]

State management in multi-step workflows is one of the trickier parts of agent development. Some patterns that have worked for us at RevolutionAI (https://revolutionai.io) when building agent systems:

1. Explicit state object
Pass a typed state dict through each step. Makes debugging and checkpointing trivial.

class WorkflowState:
    context: dict
    history: list
    current_step: str
    metadata: dict

2. Event sourcing pattern
Log every state change as an event. You can replay the entire workflow from events.

3. Memory tiers

• Short-term: conversation buffer (last N messages)
• Working: current task context
• Long-term: vector store for retrieval

4. State serialization checkpoints
After each successful step, serialize state to disk/db. Enables resume on failure.

The key insight: treat state as a first-class citizen, not an afterthought. Design your state schema before building the workflow.

What specific state challenges are you hitting?

[xXMrNidaXx]

State management across multi-step workflows is tricky! Here is what has worked well for us at RevolutionAI (https://revolutionai.io):

1. Checkpoint-based state - serialize agent state at each step boundary, enables resume on failure
2. Hierarchical context - separate "global" context from step-specific context to avoid token bloat
3. Event sourcing pattern - log all state transitions, makes debugging way easier

The key insight from recent research (check out the Routine paper) is that structured planning increases agent accuracy from ~40% to 96% in enterprise scenarios. Worth implementing explicit planning steps in your workflow.

Happy to share more implementation details if useful!

[fjnunezp75]

The approach you are building (explicit shared specification + Zenflow) is the right direction. The core insight: implicit prompt context is not state — it is text. Real state needs to be queryable, diffable, and replayable.

LlamaIndex-specific state patterns

The cleanest integration I have found is using LlamaIndex Context (in Workflows) as the state carrier:

from llama_index.core.workflow import Context, Workflow, step, Event

class PlanningDone(Event):
    plan: dict

class ResearchDone(Event):
    findings: list[str]

class AgentWorkflow(Workflow):
    
    @step
    async def plan(self, ctx: Context, ev: StartEvent) -> PlanningDone:
        # Initialize shared state in context
        await ctx.set("run_id", str(uuid.uuid4()))
        await ctx.set("budget_spent_usd", 0.0)
        await ctx.set("token_count", {"input": 0, "output": 0})
        
        plan = await self.planner.run(task=ev.task)
        await ctx.set("plan", plan.response)
        
        return PlanningDone(plan={"summary": plan.response})
    
    @step
    async def research(self, ctx: Context, ev: PlanningDone) -> ResearchDone:
        # Read state from context — not from prompt
        plan = await ctx.get("plan")
        run_id = await ctx.get("run_id")
        
        # Update costs after each call
        findings = await self.researcher.run(task=plan)
        await ctx.set("budget_spent_usd", await ctx.get("budget_spent_usd") + 0.002)
        
        return ResearchDone(findings=[findings.response])

Why this beats passing context via prompts

1. The ctx object is serializable — you can checkpoint and resume
2. Branching logic reads from ctx, not from parsing LLM output
3. Cost tracking lives in state, not in a separate log
4. Retry from a specific step: restore ctx snapshot, re-run from that step

For the retry/branching pain you mentioned

The hard case is when an agent goes wrong mid-research and you want to retry just that step. Add explicit retry metadata to context:

@step
async def research(self, ctx: Context, ev: PlanningDone) -> ResearchDone:
    retry_count = await ctx.get("research_retry_count", default=0)
    
    try:
        result = await self.researcher.run(...)
        await ctx.set("research_retry_count", 0)  # Reset on success
        return ResearchDone(findings=[result.response])
    except Exception as e:
        await ctx.set("research_retry_count", retry_count + 1)
        await ctx.set("last_error", {"step": "research", "error": str(e), "retry": retry_count})
        if retry_count < 3:
            return PlanningDone(plan=await ctx.get("plan"))  # Loop back
        raise

On making behavior reproducible

The key practice: serialize ctx state to disk after each major step. Then your test fixtures are real production snapshots, not synthetic data. I have found this cuts debugging time significantly — you are not guessing what state the agent was in, you are running against the exact snapshot.

[vgudur-dev]

Excellent breakdown of the Context workflow pattern — this is the right architecture. One critical dimension to add when you go to production: the ctx state object is also your primary attack surface.

When a Research step fetches external content (web pages, user-uploaded docs, emails), an adversary can embed instructions in that content that get written into ctx. The Executor step then reads from ctx and follows the injected instruction — this is OWASP ASI06 (Memory Poisoning), and it's especially dangerous in LlamaIndex workflows because the Context object propagates to every downstream step.

The fix is to scan values before writing them to ctx from any step that touches external input:

from agent_memory_guard import MemoryGuard
from llama_index.core.workflow import Context

guard = MemoryGuard()

async def safe_ctx_set(ctx: Context, key: str, value: str):
    result = guard.scan(value)
    if result.is_safe:
        await ctx.set(key, value)
    else:
        # Log the blocked attempt and set a safe fallback
        await ctx.set(f"{key}_blocked", True)
        await ctx.set("security_events", [
            *await ctx.get("security_events", default=[]),
            {"key": key, "threat": result.threat_type}
        ])
        raise SecurityError(f"Memory poisoning blocked in step '{key}': {result.threat_type}")

This is especially important for the checkpoint/replay pattern mentioned above — if you serialize a poisoned ctx to disk and replay it, you replay the attack.

Library: pip install agent-memory-guard (OWASP project: https://github.com/OWASP/www-project-agent-memory-guard)

[hanswang73]

I have been using LlamaIndex for agent-like workflows-planning, execution, tool usage, and sometimes reviews. One thing I keep running into is state management once the workflow sprawls to multiple steps.

In other words, when agents hand work off to each other, context ends up split between prompts, intermediate nodes, and application code. This gets especially tricky with retries or branching logic-it's hard to tell why an agent made a certain decision or to reproduce behavior later.

I have tried to implement this by keeping an explicit shared specification/state that agents read and write, instead of relying solely on the implicit prompt context. I've been testing this with a small orchestration tool, Zenflow, in conjunction with LlamaIndex to make workflows more inspectable, but I'm still feeling out whether this is the right direction.

Wondering how people here are handling this with LlamaIndex: mostly manually managing state, using callback handlers/state handlers, or leaning on the newer abstractions for agents in the library?

Perhaps you can try telichat.io

[Ghraven]

The explicit shared-state direction makes sense to me. I would avoid treating agent memory as the source of truth for workflow state, especially once retries or branches enter the picture.

A pattern that has worked well for multi-step agent workflows is to split state into a few layers:

1. Workflow state: typed, durable, and owned by the orchestration layer. This is where I would put things like run_id, current step, plan version, selected artifacts, retry counts, and last successful checkpoint.

2. Agent context: the task-local context each agent needs for its next call. This can be derived from workflow state, but it should not be the only place the state exists.

3. Event log: append-only records of tool calls, transitions, failures, retries, and human approvals. This is what makes postmortems and reproducibility much easier.

For LlamaIndex specifically, I would lean toward manually managed state or the workflow/orchestration layer owning the canonical state, then use callback/event hooks for observability rather than relying on memory alone to coordinate the workflow. Memory is useful for recall, but it is usually too implicit for retry semantics.

One practical rule: after each successful step, write a checkpoint with the exact output contract for the next step. On retry, resume from the last good checkpoint rather than asking the next agent to infer state from prior conversation text.

For example:

state = {
    "run_id": run_id,
    "step": "research",
    "plan_version": 3,
    "artifacts": {"brief": "...", "sources": [...]},
    "retry_counts": {"research": 1},
    "last_successful_checkpoint": "plan_accepted",
}

Then each agent/task receives only the slice it needs, plus the relevant checkpoint/artifact references. That keeps prompts smaller and makes it much easier to answer "did this fail because the task was wrong, the prior output was missing, or the tool call failed?"

So short version: yes to explicit shared state. I would pair it with typed output contracts, checkpoints after successful steps, and an append-only event log. Callback handlers can then record what happened, but the workflow state itself should remain outside the agents.

[ferhimedamine]

@Ghraven nailed the key architectural insight — splitting state into workflow state, agent context, and event log. I want to build on one specific point: the typed output contracts between steps and how memory types map naturally onto those layers.

The pattern that has worked well in our system is to treat each layer as a distinct memory type. Working memory holds current-step context and expires when the step completes. Episodic memory stores completed-step outcomes and persists across steps within the workflow. Semantic memory captures accumulated knowledge and persists across workflows entirely. Each step reads relevant memory from the appropriate tier, executes, then stores its outcome at the right level. This makes workflows resumable without replaying — if step 3 fails, you restore its working memory state from the last checkpoint rather than re-running steps 1 and 2.

@fjnunezp75 showed a clean LlamaIndex Workflow implementation with Context carrying the state. One subtlety worth adding: the retry metadata pattern works better when combined with importance scoring on what gets preserved during retry. When a step fails and retries, you do not want the failed attempt polluting the context for the next attempt. We tag failed-step outputs with low importance so they are available for debugging but do not compete with fresh context during retrieval. The retry count plus the error metadata stays high-importance so the agent knows it is retrying and why.

One thing the thread has not addressed: what happens when the workflow spans sessions or the process restarts mid-execution. The checkpoint-per-step approach works, but the checkpoint needs to capture not just the typed state but also the memory retrieval context — which memories were loaded, what scores they had, what the agent "knew" at decision time. Without that, resuming from a checkpoint can produce different behavior because the memory corpus may have changed between the crash and the restart. Storing a snapshot of the retrieved memory alongside the workflow state makes resume truly deterministic.

[RitwijParmar]

I think the important split is state versus trace.

State is what the next step needs to continue. Trace is what you need later to understand why it happened.

For multi step workflows I would keep state small and typed. Current goal, step name, selected files, tool outputs, retry count, budget used. Then separately write an append only event log for every agent decision and tool call.

That gives you two useful things.

You can resume from the state snapshot.

You can debug from the event log without asking the model to explain itself later.

I would also make tool calls idempotent where possible. Retries become much less scary if every step has a stable run id and writes either once or not at all.

[ThinkOffApp]

The state vs trace split that @RitwijParmar and @Ghraven both land on matches what we ended up with running agents across multiple machines, where handoffs cross process and host boundaries so implicit prompt context is not even an option. We keep the workflow state tiny and typed (current goal, step, the few things the next step needs to continue) and write every decision and tool call to a separate append only event log. What surprised us is how much that log earns its keep, not for replay but for answering "why did it branch here" weeks later, which is exactly the hard part the original post calls out.

One practical note on the explicit shared spec direction: it pays to make agents read and write state through a narrow interface rather than a shared dict, because the moment two agents (especially on different machines) both mutate it, you want the writes diffable and attributable. Logging each read and write as an event (who wrote what, when) turns "hard to reproduce behavior later" into just reading the log. @vgudur-dev's point about ctx being an attack surface is the same reason: if external content can land in shared state, you really want every write attributed to a source.

[rishabhsaini282]

Excellent insights! One lesson I've learned is that a clean architecture is just as important as the model itself. Website scraping, chunking, embeddings, retrieval, and the API layer all need to work together for reliable responses.

I recently shared a complete implementation using Python, FastAPI, BeautifulSoup, SentenceTransformers, and OpenRouter: https://youtu.be/Ze1mdBoRutY?si=EF3cGaWRQUwyCTJN.

What's been the biggest bottleneck in your production RAG systems, retrieval, latency, or hallucinations?

[Ghraven]

For the RAG systems I have worked on, the biggest bottleneck has usually not been the first version of retrieval or latency. It has been proving that the system is grounded enough to trust after the happy-path demo works.

A pattern that helped me is to treat RAG as three separate surfaces:

1. Retrieval quality: small eval set with expected source documents, Hit@K / MRR, and examples where similar-but-wrong documents exist.
2. Answer grounding: require answers to cite retrieved context and add a refusal path when the retrieved context is weak or off-topic.
3. Regression checks: keep a few production-like questions as fixtures so changes to chunking, embeddings, prompts, or rerankers do not silently make old behavior worse.

Latency matters, but I would usually optimize it after those checks exist. Without evals, a faster RAG pipeline can just become a faster way to return plausible unsupported answers.