Conversational Claude coordinator for dynamic agent orchestration

[james-in-a-box]

Summary

Replace the rigid SDLC CLI pipeline with a conversational Claude session that dynamically manages the orchestrator and agent teams. Instead of fixed phase sequences driven by egg-orch commands, a top-level Claude session acts as the coordinator — understanding the task, deciding which agents to spawn, delegating work, and adapting the workflow in real-time.

Motivation

The current SDLC pipeline (egg-orch pipeline create → fixed phases → fixed agent roles per phase) is effective but rigid:

• Fixed phase ordering — analysis → plan → implement → test → document, always in that order, even when the task doesn't need all phases
• Manual CLI orchestration — advancing phases, spawning agents, and handling decisions all require explicit CLI commands or pre-configured automation
• No adaptive reasoning — the pipeline can't decide "this is a simple bug fix, skip the analysis phase" or "the tester found a problem, loop back to the coder"
• Steep operational overhead — managing pipelines requires knowledge of egg-orch, egg-contract, phase mechanics, and HITL protocols

A Claude session as the coordinator brings judgment, flexibility, and natural language interaction to the process.

Proposed Design

Conversational Coordinator

A top-level Claude session receives the task (from a user, issue, or Slack message) and acts as the project manager:

• Understands the task — reads the issue, gathers context, determines scope and complexity
• Plans the approach — decides which agents are needed, whether phases should be sequential or concurrent, and what the success criteria are
• Delegates to agents — spawns agent teams via the orchestrator API, provides them with context and instructions
• Monitors progress — receives status updates from agents (via the cross-agent communication system), intervenes when needed
• Adapts dynamically — if a tester finds a bug, the coordinator can loop the coder back in without restarting the entire pipeline
• Escalates to humans — uses judgment to decide when human input is needed, rather than relying on pre-configured HITL checkpoints

Interaction with Orchestrator

The coordinator doesn't replace the orchestrator — it drives it:

• Uses orchestrator APIs to spawn containers, manage agent lifecycle, route messages
• The orchestrator remains the execution layer (container management, gateway enforcement, checkpoint storage)
• The coordinator is the decision layer (what to do, when, with whom)

Example Workflow

User: "Fix the auth bug described in issue #432"

Coordinator (Claude session):
  1. Reads issue #432, understands the bug
  2. Checks codebase — determines it's a focused fix in gateway/auth.py
  3. Spawns a coder agent with targeted context
  4. Coder pushes a fix, signals progress
  5. Coordinator spawns a tester agent, points it at the fix
  6. Tester finds an edge case — coordinator relays back to coder
  7. Coder addresses it, tester confirms
  8. Coordinator decides docs aren't needed for this change
  9. Creates PR, summarizes work to user

Compare this to the current flow: create pipeline → wait for analysis agent → review analysis → advance phase → wait for plan agent → review plan → advance → spawn coder → etc.

Relationship to SDLC Pipeline

The existing SDLC pipeline becomes one possible workflow template that the coordinator can choose to follow, rather than the only option. For complex features, the coordinator might follow a full SDLC-like flow. For a typo fix, it skips straight to a coder.

Key Components to Build

1. Coordinator session framework — Claude session with tools for orchestrator interaction (spawn agents, send messages, monitor status)
2. Orchestrator API extensions — endpoints the coordinator needs beyond current CLI (e.g., flexible agent spawning without rigid phase constraints)
3. Agent instruction protocol — how the coordinator communicates task context and expectations to spawned agents
4. Progress monitoring — how the coordinator tracks agent work and decides when to intervene
5. Workflow templates — optional pre-defined patterns (full SDLC, quick fix, docs-only) the coordinator can choose from or adapt
6. Human escalation interface — how the coordinator surfaces decisions to users (Slack, GitHub comments, etc.)

Dependencies

• Enable cross-agent communication and concurrent phase execution #1027 — Cross-agent communication and concurrent phase execution. The coordinator needs the message routing and concurrent agent infrastructure to function.

Open Questions

• Coordinator persistence: Does the coordinator session stay alive for the entire task lifecycle, or can it checkpoint and resume?
• Authority boundaries: What can the coordinator do autonomously vs. what requires human approval? (e.g., spawning agents = auto, merging PR = human)
• Cost management: A long-running coordinator session plus multiple agent sessions could be expensive — how to bound this?
• Failure recovery: If the coordinator session crashes, how do running agents behave? Do they continue autonomously or pause?
• Multi-task coordination: Can one coordinator manage multiple related issues simultaneously?

Success Criteria

• A user can describe a task in natural language and have a Claude coordinator session manage the full lifecycle
• The coordinator dynamically decides which agents to spawn and how to sequence work
• Simple tasks complete with fewer steps and agents than the current rigid pipeline
• Complex tasks still get thorough multi-agent treatment, but with adaptive feedback loops
• The existing SDLC CLI continues to work as an alternative for users who prefer explicit control