OpenClaw Multi-Step Workflow Orchestration: Building Reliable Agent Pipelines

Single-agent tasks are table stakes. The transformative capability—what separates toy projects from production systems—is orchestrating multi-step workflows where agents pass context, handle failures gracefully, and maintain state across execution boundaries.

In this guide, I'll walk through the patterns I use daily to build reliable agent pipelines in OpenClaw. We'll cover everything from simple sequential flows to complex conditional routing with error recovery.

Why Workflow Orchestration Matters

Most AI agent tutorials stop at "here's how to call an API." Real-world problems require coordination:

• Content generation pipelines: Research → Outline → Write → Edit → Publish
• Data processing workflows: Extract → Transform → Analyze → Visualize → Report
• Customer support automation: Classify → Route → Respond → Escalate → Follow-up
• Development workflows: Plan → Code → Test → Review → Deploy

Each step has different failure modes, latency profiles, and resource requirements. Orchestration is what makes these pipelines reliable.

The Three Core Orchestration Patterns

1. Sequential Execution (The Simple Chain)

The most basic pattern: Agent A completes, passes output to Agent B, which passes to Agent C. This works for linear processes where each step depends on the previous.

#!/bin/bash
# Example: Content generation pipeline
# Step 1: Research agent
RESEARCH_OUTPUT=$(sessions_spawn \
  runtime="subagent" \
  agentId="mira-sage" \
  task="Research latest trends in AI agent orchestration for a blog post" \
  mode="run")

# Step 2: Outline agent (uses research output)
OUTLINE=$(sessions_spawn \
  runtime="subagent" \
  agentId="mira-wren" \
  task="Create a detailed outline for a blog post about AI agent orchestration. Research context: $RESEARCH_OUTPUT" \
  mode="run")

# Step 3: Writing agent (uses outline)
ARTICLE=$(sessions_spawn \
  runtime="subagent" \
  agentId="mira-wren" \
  task="Write a 2000-word blog post using this outline: $OUTLINE" \
  mode="run")

# Step 4: Editing agent
FINAL_ARTICLE=$(sessions_spawn \
  runtime="subagent" \
  agentId="mira-forge" \
  task="Edit and polish this article for technical accuracy and readability: $ARTICLE" \
  mode="run")

When to use: Simple linear processes with clear dependencies. Limitation: No error recovery—if step 2 fails, the whole pipeline stops.

2. Parallel Execution with Aggregation

When you have independent tasks that can run simultaneously, spawn them in parallel and aggregate results.

// TypeScript example for parallel execution
import { exec } from 'child_process';
import { promisify } from 'util';

const execAsync = promisify(exec);

async function parallelResearch(topics: string[]) {
  const promises = topics.map(topic => 
    execAsync(`sessions_spawn runtime="subagent" agentId="mira-sage" task="Research ${topic}" mode="run"`)
  );
  
  const results = await Promise.allSettled(promises);
  
  // Aggregate successful results
  const aggregated = results
    .filter((r): r is PromiseFulfilledResult<{ stdout: string }> => r.status === 'fulfilled')
    .map(r => r.value.stdout)
    .join('\n\n---\n\n');
    
  return aggregated;
}

// Usage
const topics = [
  "AI agent orchestration patterns",
  "OpenClaw workflow examples", 
  "Multi-agent system best practices"
];

const researchReport = await parallelResearch(topics);

When to use: Independent research tasks, data collection from multiple sources, A/B testing variations. Key benefit: Dramatically reduces total execution time.

3. Conditional Routing (Decision-Based Workflows)

The most powerful pattern: Use a router agent to analyze input and decide which path to take.

#!/bin/bash
# Example: Customer support ticket routing
TICKET="Customer says their OpenClaw cron jobs aren't running consistently."

# Router agent analyzes and decides
ROUTING_DECISION=$(sessions_spawn \
  runtime="subagent" \
  agentId="mira-forge" \
  task="Analyze this support ticket and decide which specialist agent should handle it. 
  Ticket: $TICKET
  Options: 
  1. 'cron-issue' - For cron/scheduling problems
  2. 'config-issue' - For configuration problems  
  3. 'bug-report' - For potential software bugs
  4. 'documentation' - For how-to questions
  Return ONLY the option label, nothing else." \
  mode="run")

# Trim whitespace and route
DECISION=$(echo "$ROUTING_DECISION" | tr -d '[:space:]')

case $DECISION in
  "cron-issue")
    sessions_spawn runtime="subagent" agentId="mira-forge" task="Handle cron issue: $TICKET" mode="run"
    ;;
  "config-issue")
    sessions_spawn runtime="subagent" agentId="mira-forge" task="Debug configuration: $TICKET" mode="run"
    ;;
  "bug-report")
    sessions_spawn runtime="subagent" agentId="mira-forge" task="Investigate potential bug: $TICKET" mode="run"
    ;;
  "documentation")
    sessions_spawn runtime="subagent" agentId="mira-wren" task="Create documentation response: $TICKET" mode="run"
    ;;
  *)
    # Default fallback
    sessions_spawn runtime="subagent" agentId="mira-forge" task="Handle general support: $TICKET" mode="run"
    ;;
esac

When to use: Customer support, content categorization, priority-based processing, A/B testing with different treatment paths.

State Management Across Workflow Steps

The hardest part of workflow orchestration isn't chaining agents—it's preserving context across steps. Here are the patterns I use:

Pattern 1: Workspace Files (Simple & Reliable)

#!/bin/bash
# Create a shared workspace directory
WORKSPACE="/tmp/workflow_$(date +%s)"
mkdir -p "$WORKSPACE"

# Step 1: Write output to workspace
sessions_spawn \
  runtime="subagent" \
  agentId="mira-sage" \
  task="Research AI agent trends" \
  mode="run" > "$WORKSPACE/research.txt"

# Step 2: Read from workspace, write new output
RESEARCH=$(cat "$WORKSPACE/research.txt")
sessions_spawn \
  runtime="subagent" \
  agentId="mira-wren" \
  task="Outline based on: $RESEARCH" \
  mode="run" > "$WORKSPACE/outline.txt"

# Step 3: Continue chain
OUTLINE=$(cat "$WORKSPACE/outline.txt")
sessions_spawn \
  runtime="subagent" \
  agentId="mira-wren" \
  task="Write article from outline: $OUTLINE" \
  mode="run" > "$WORKSPACE/article.md"

Pros: Simple, debuggable, survives agent crashes. Cons: File I/O overhead, cleanup needed.

Pattern 2: Environment Variables (Lightweight Context)

#!/bin/bash
# Export context as environment variables
export RESEARCH_CONTEXT="$(sessions_spawn runtime='subagent' agentId='mira-sage' task='Brief research' mode='run')"

# Subsequent agents inherit the environment
sessions_spawn \
  runtime="subagent" \
  agentId="mira-wren" \
  task="Outline using research: $RESEARCH_CONTEXT" \
  mode="run"

# For complex state, use JSON in env var
export WORKFLOW_STATE='{"step": 2, "data": {"research": "completed", "outline": "in_progress"}}'

Pros: Fast, no disk I/O. Cons: Size limits, doesn't survive shell termination.

Pattern 3: Database-Backed State (Production Ready)

// Using Supabase for workflow state
import { createClient } from '@supabase/supabase-js';

const supabase = createClient(
  process.env.SUPABASE_URL!,
  process.env.SUPABASE_SERVICE_ROLE_KEY!
);

interface WorkflowState {
  id: string;
  workflow_id: string;
  step: number;
  status: 'pending' | 'running' | 'completed' | 'failed';
  data: Record<string, any>;
  created_at: string;
  updated_at: string;
}

async function updateWorkflowState(
  workflowId: string, 
  step: number, 
  status: WorkflowState['status'],
  data: Record<string, any>
) {
  const { error } = await supabase
    .from('workflow_states')
    .upsert({
      workflow_id: workflowId,
      step,
      status,
      data,
      updated_at: new Date().toISOString()
    }, {
      onConflict: 'workflow_id,step'
    });
    
  if (error) throw error;
}

// Usage in workflow
await updateWorkflowState('content-gen-123', 1, 'completed', {
  research: researchOutput,
  keywords: ['orchestration', 'workflow', 'agents']
});

Pros: Durable, queryable, supports resumable workflows. Cons: Infrastructure dependency, more complex.

Error Handling and Recovery Patterns

Workflows fail. The difference between amateur and professional orchestration is how you handle failures.

Retry with Exponential Backoff

#!/bin/bash
retry_with_backoff() {
  local cmd="$1"
  local max_retries=${2:-3}
  local retry_count=0
  
  while [ $retry_count -lt $max_retries ]; do
    if eval "$cmd"; then
      return 0
    fi
    
    retry_count=$((retry_count + 1))
    if [ $retry_count -eq $max_retries ]; then
      echo "Failed after $max_retries attempts"
      return 1
    fi
    
    # Exponential backoff: 1, 2, 4, 8 seconds
    sleep $((2 ** (retry_count - 1)))
    echo "Retry $retry_count/$max_retries..."
  done
}

# Usage
retry_with_backoff \
  'sessions_spawn runtime="subagent" agentId="mira-forge" task="Process data batch" mode="run"' \
  3

Circuit Breaker Pattern

class CircuitBreaker {
  private failures = 0;
  private lastFailureTime = 0;
  private readonly threshold: number;
  private readonly resetTimeout: number;
  private state: 'CLOSED' | 'OPEN' | 'HALF_OPEN' = 'CLOSED';

  constructor(threshold = 5, resetTimeout = 60000) {
    this.threshold = threshold;
    this.resetTimeout = resetTimeout;
  }

  async execute(fn: () => Promise<any>): Promise<any> {
    if (this.state === 'OPEN') {
      const now = Date.now();
      if (now - this.lastFailureTime > this.resetTimeout) {
        this.state = 'HALF_OPEN';
      } else {
        throw new Error('Circuit breaker OPEN');
      }
    }

    try {
      const result = await fn();
      if (this.state === 'HALF_OPEN') {
        this.state = 'CLOSED';
        this.failures = 0;
      }
      return result;
    } catch (error) {
      this.failures++;
      this.lastFailureTime = Date.now();
      
      if (this.failures >= this.threshold) {
        this.state = 'OPEN';
      }
      
      throw error;
    }
  }
}

// Usage with agent calls
const breaker = new CircuitBreaker(3, 30000);

try {
  const result = await breaker.execute(() =>
    sessions_spawn({
      runtime: "subagent",
      agentId: "mira-forge",
      task: "High-risk data processing",
      mode: "run"
    })
  );
} catch (error) {
  console.error('Circuit breaker prevented cascade failure:', error);
}

Fallback Strategies

#!/bin/bash
# Try primary, fall back to secondary
execute_with_fallback() {
  local primary_cmd="$1"
  local fallback_cmd="$2"
  
  if eval "$primary_cmd"; then
    echo "Primary succeeded"
    return 0
  else
    echo "Primary failed, trying fallback..."
    if eval "$fallback_cmd"; then
      echo "Fallback succeeded"
      return 0
    else
      echo "Both primary and fallback failed"
      return 1
    fi
  fi
}

# Usage
execute_with_fallback \
  'sessions_spawn runtime="subagent" agentId="claude-opus" task="Complex analysis" mode="run"' \
  'sessions_spawn runtime="subagent" agentId="claude-sonnet" task="Simpler version of analysis" mode="run"'

Monitoring and Observability

You can't improve what you can't measure. Here's how I instrument workflows:

#!/bin/bash
# Simple workflow telemetry
log_workflow_event() {
  local event="$1"
  local data="$2"
  local timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
  
  echo "{\"timestamp\": \"$timestamp\", \"event\": \"$event\", \"data\": $data}" \
    >> "/tmp/workflow_telemetry_$(date +%Y%m%d).jsonl"
}

# Instrument each step
log_workflow_event "workflow_started" '{"workflow_id": "content-gen-123", "steps": 4}'

START_TIME=$(date +%s)
RESEARCH_OUTPUT=$(sessions_spawn runtime="subagent" agentId="mira-sage" task="Research" mode="run")
RESEARCH_TIME=$(( $(date +%s) - $START_TIME ))
log_workflow_event "step_completed" '{"step": 1, "agent": "mira-sage", "duration_seconds": '$RESEARCH_TIME', "output_length": '${#RESEARCH_OUTPUT}'}'

# Continue for each step...
log_workflow_event "workflow_completed" '{"workflow_id": "content-gen-123", "total_duration_seconds": '$(( $(date +%s) - $START_TIME ))', "success": true}'

Real-World Example: Content Generation Pipeline

Let's walk through a complete, production-ready content generation workflow:

#!/bin/bash
# content_pipeline.sh
set -euo pipefail

WORKFLOW_ID="content-$(date +%Y%m%d-%H%M%S)"
WORKSPACE="/tmp/${WORKFLOW_ID}"
mkdir -p "$WORKSPACE"

log_event() {
  echo "[$(date -u +"%Y-%m-%dT%H:%M:%SZ")] $1" | tee -a "$WORKSPACE/log.txt"
}

# 1. Topic Research (parallel)
log_event "Starting parallel research"
TOPICS=("AI agent orchestration" "Workflow patterns" "Error handling in agents")
for topic in "${TOPICS[@]}"; do
  (sessions_spawn runtime="subagent" agentId="mira-sage" \
    task="Research: $topic. Focus on practical implementation details." \
    mode="run" > "$WORKSPACE/research_${topic// /_}.txt" 2>&1 &
  ) 
done
wait
log_event "Parallel research completed"

# 2. Aggregate research
RESEARCH_AGGREGATE=""
for topic in "${TOPICS[@]}"; do
  RESEARCH_AGGREGATE+="$(cat "$WORKSPACE/research_${topic// /_}.txt")\n\n---\n\n"
done
echo "$RESEARCH_AGGREGATE" > "$WORKSPACE/research_aggregate.txt"

# 3. Outline generation
log_event "Generating outline"
OUTLINE=$(sessions_spawn runtime="subagent" agentId="mira-wren" \
  task="Create detailed outline for article about AI workflow orchestration. Research: $RESEARCH_AGGREGATE" \
  mode="run")
echo "$OUTLINE" > "$WORKSPACE/outline.txt"

# 4. Article writing (with retry)
log_event "Writing article"
retry_with_backoff() {
  local cmd="$1"
  for i in {1..3}; do
    if eval "$cmd"; then
      return 0
    fi
    sleep $((i * 2))
  done
  return 1
}

ARTICLE=$(retry_with_backoff \
  'sessions_spawn runtime="subagent" agentId="mira-wren" \
    task="Write 2000-word article using outline: $OUTLINE" \
    mode="run"')
echo "$ARTICLE" > "$WORKSPACE/article.md"

# 5. Technical review
log_event "Technical review"
TECH_REVIEW=$(sessions_spawn runtime="subagent" agentId="mira-forge" \
  task="Review article for technical accuracy: $ARTICLE" \
  mode="run")
echo "$TECH_REVIEW" > "$WORKSPACE/tech_review.txt"

# 6. Final polish
log_event "Final polish"
FINAL_ARTICLE=$(sessions_spawn runtime="subagent" agentId="mira-wren" \
  task="Polish article incorporating technical feedback: $ARTICLE\n\nFeedback: $TECH_REVIEW" \
  mode="run")
echo "$FINAL_ARTICLE" > "$WORKSPACE/final_article.md"

log_event "Pipeline completed successfully"
echo "Workflow $WORKFLOW_ID completed. Output: $WORKSPACE/final_article.md"

Common Pitfalls and How to Avoid Them

1. Context Loss Between Steps

Problem: Agents forget what happened in previous steps. Solution: Always pass explicit context, never assume agents remember. Use the state management patterns above.

2. Cascading Failures

Problem: One failed step kills the entire workflow. Solution: Implement circuit breakers, retries with backoff, and fallback strategies.

3. Unbounded Execution Time

Problem: Workflows run forever if an agent gets stuck. Solution: Set timeouts at every level:

# Agent-level timeout
sessions_spawn runtime="subagent" agentId="mira-forge" \
  task="Process data" \
  mode="run" \
  timeoutSeconds=300  # 5 minute timeout

# Workflow-level timeout (using timeout command)
timeout 1800 ./content_pipeline.sh  # 30 minute overall timeout

4. No Visibility Into Progress

Problem: You can't tell what's happening in a running workflow. Solution: Implement structured logging and telemetry as shown above.

Next Steps: From Scripts to Systems

Once you've mastered these patterns, consider evolving your workflow orchestration:

• Workflow as Code: Define workflows in TypeScript/JSON for version control and reuse
• Visual Orchestrator: Build a UI to monitor and manage running workflows
• Dynamic Routing: Implement ML-based routing decisions based on historical performance
• Cost Optimization: Route tasks to different models based on complexity and cost targets

The patterns in this article are battle-tested from running OpenClaw in production. They scale from simple 2-step scripts to complex pipelines with dozens of agents and conditional branches.

FAQ

This article represents months of production experience orchestrating OpenClaw workflows. The patterns here handle everything from daily content generation to complex customer support automation.