Reference Implementation PYTHON Project

This section walks through a (non-existing) reference implementation based on the AI Blueprint – a multi-agent orchestration framework built on LangChain and LangGraph. The goal is not to prescribe a single architecture but to illustrate why each layer exists, what problem it solves, and how the pieces compose. You are free to adopt as much or as little as fits your context.

From Minimal to Production-Ready

The shortest possible agent is a single LLM call:

from langchain_openai import ChatOpenAI   
from langchain_core.messages import HumanMessage, SystemMessage   
   
llm = ChatOpenAI(model="gpt-4o", api_key="sk-...")   
   
def run_agent(goal: str) -> str:   
 response = llm.invoke([   
 SystemMessage(content="You are a helpful assistant."),   
 HumanMessage(content=goal),   
 ])   
 return str(response.content)

The above code snippet works, it answers a single question in a single turn. However, as soon as you move beyond a demo, there are missing capabilities that are needed to make it production ready:

The reference implementation addresses all nine capabilities, and the remainder of this section explains each layer in turn.

LLM Router – Resilience, Fallback, and Capability Routing

All LLM calls in the reference implementation flow through a single LLMRouter. No agent calls a provider SDK directly. This indirection enables three things: automatic retry with exponential backoff, transparent provider fallback, and SHA-256-keyed response caching.

LLMRouter   
 ├── invoke(messages, tools?, model\_selection?) → AIMessage   
 ├── invoke\_with\_history(system, user\_prompt, session\_state) → AIMessage   
 ├── Retry: exponential backoff, configurable attempts and bounds   
 ├── Fallback: primary → fallback₁ → fallback₂ → … → AllLLMProvidersFailed   
 └── Cache: LLMCacheBase (file JSON or Redis), TTL-based, thread-safe

Capability-based model selection

Each provider configuration declares its capabilities – context window size, input/output modalities, and reasoning level. Agents that need a large context or image understanding set a model_selection criterion on their AgentConfig. The router filters the provider list at call time and raises NoModelsMatchCriteria if nothing qualifies.

config.yaml

 models:   
 primary:   
 name: "openai: gpt-4o"   
 provider: openai   
 model: gpt-4o   
 api_key: "$ENV:API_KEY"   
 capabilities:   
 max_context_size: 128000   
 input_modalities: ["text", "image"]   
 reasoning_capability: "high"   
 fallbacks:   
 - name: "anthropic: claude-3-5-sonnet"   
 provider: anthropic   
 model: claude-3-5-sonnet-20241022   
 capabilities:   
 max_context_size: 200000   
 reasoning_capability: "high"   
   
 retry:   
 max_retries: 3   
 backoff_min_seconds: 1   
 backoff_max_seconds: 8   
   
 cache:   
 type: file # or: redis   
 enabled: true   
 ttl_seconds: 3600

The LLMCacheBase abstraction means switching from file-based caching to Redis requires only a one-line configuration change.

Design principle: Route through one object. Every cross-cutting concern – retry, fallback, caching, capability filtering – lives in the router, not scattered across agents.

Multi-Agent Architecture – Specialized Roles

The reference implementation defines four specialized (role) agents, each extending a common BaseAgent abstract class. All are injected with the same LLMRouter, ContextMemory, and ToolRegistry instances; they differ only in their system prompt and, optionally, their model selection criteria.

BaseAgent (abstract)   
 ├── PlannerAgent – decomposes the goal into an ordered task list   
 ├── ExecutorAgent – executes the plan, calling tools as needed   
 ├── CriticAgent – reviews the output; signals APPROVED or REVISE   
 └── EditorAgent – polishes the final user-facing response

Agents are stored in an AgentRegistry. At runtime the Coordinator looks up agents by name. This means swapping an agent – replacing the default ExecutorAgent with a domain-specific one – requires a single AgentRegistry.register() call with no graph rewiring.

Setting retain_message_history=True enables per-agent conversation memory. The LLM router’s invoke_with_history() wraps the call in LangChain’s RunnableWithMessageHistory, keying history by session ID. This is how a multi-turn REST or A2A conversation maintains coherent context inside each agent without any agent knowing about HTTP or gRPC.

Context Memory – Shared State Across Agents

All role agents in a single orchestration pass share a ContextMemory instance. It stores, per session: agent outputs, conversation turns (user and assistant messages), tool results, and an arbitrary key/value scratchpad.

The internal structure is ContextState → {session_id: SessionContextState}. File persistence uses atomic writes to avoid corruption on concurrent access. In a production ready, multi-process deployment (e.g., combined REST + A2A servers), the implementation must be extended to use a (fast) database to store context successfully.

Why a shared memory? Each agent receives the full accumulated context – the plan text, previous execution output, prior revision notes – as part of its prompt. Without a shared store, each agent would need to pass state explicitly through the graph, coupling the agents to one another’s output shape.

Orchestration Graph – Structured Multi-Step Workflow

The workflow is a compiled LangGraph StateGraph. Nodes call role agents; edges encode conditional routing. The default graph implements a Plan → Execute → Review loop with a “simple query” shortcut:

The evaluate_query_complexity setting determines whether the simple-query shortcut is active. When enabled, the Coordinator classifies the incoming goal before entering the graph; a direct factual question goes straight to Execute → Edit, bypassing the full review cycle.

Cooperative cancellation is woven into every node: a threading.Event is checked at each boundary. Callers that set the cancellation event receive an OrchestrationCancelled exception rather than waiting for the current node to complete.

The graph itself is swappable. The DynamicGraphBuilder API lets callers supply an entirely custom StateGraph while reusing the same Coordinator, LLMRouter, ContextMemory, and ToolRegistry. Examples in the repository demonstrate this pattern for domain-specific workflows that need different node topologies.

Tool Registry and MCP Integration

Agents gain access to external capabilities through a ToolRegistry. The executor’s tool-call loop is native LangChain: tools are bound to the LLM, and the model issues tool_calls in its response; the executor invokes them and feeds results back, up to a configurable depth (MAX_TOOL_CALL_DEPTH=10).

Built-in tools (only for demonstration):

Beyond built-ins, any Model Context Protocol (MCP) server is automatically discovered at startup – stdio and HTTP transports are both supported. If an MCP server is unavailable, the orchestrator logs a warning and continues; the agent simply lacks those tools.

Guardrails – Input and Output Validation

The Coordinator validates every request before it enters the graph and every response before it leaves. Validation is a two-stage pipeline: fast regex pattern matching first (currently matches English words), then an optional LLM classification pass for ambiguous cases.

Input categories checked: harmful actions, hate speech and discrimination, system manipulation, prompt injection.

Output categories checked: dangerous content, internal system exposure, tone analysis.

Each category is independently toggled in configuration. The optional LLM classifier accepts a confidence threshold; cases where the regex stage is inconclusive are escalated to the classifier rather than defaulting to block or allow.

 guardrails:   
 enabled: true   
 input_validation:   
 harmful_actions: true   
 prompt_injection: true   
 hate_speech: false   
 output_validation:   
 dangerous_content: true   
 internal_exposure: true   
 llm_classification:   
 enabled: true   
 confidence_threshold: 0.7

Configuration Management

All runtime behaviour is controlled through a single config.yaml. The Settings class (Pydantic BaseSettings) merges file configuration, environment variables, and .env – in that order of precedence. All values are validated at startup; a misconfigured system fails fast rather than at invocation time.

Secret values use $ENV:VAR_NAME substitution in the YAML file. The literal secret never appears in source control.

The configuration tree covers every subsystem:

 models: # providers, capabilities, retry, cache   
 agents: # per-agent system prompts, history, model selection   
 tools: # built-in tool toggles and parameters   
 orchestration: # mode (direct / bus), complexity evaluation   
 a2a: # server identity, skills, task store, runtime   
 guardrails: # categories, LLM classification   
 memory: # persistence path   
 telemetry: # OpenTelemetry / Langfuse endpoints

Because configuration is a Pydantic model, every setting has a documented type, a default, and a validation rule. Operators can override any setting via environment variable, which makes the system straightforward to deploy in containers.

Agent-to-Agent Protocol – Durable Server Mode

When the orchestrator is deployed as a service, it implements the Agent-to-Agent (A2A) protocol – a JSON-RPC and gRPC interface for structured agent communication. The combined server exposes the REST chat-completions API and the A2A endpoints simultaneously on separate ports.

Durable task persistence is the key operational feature. The DurableRequestHandler persists every in-progress task to a configurable backend and replays it from the last user message after a process restart. No in-flight work is lost when a deployment rolls or a process crashes.

Three storage backends are available:

The PlanExecuteReviewAgentExecutor bridges the A2A task lifecycle to the Coordinator. From the A2A caller’s perspective, it issues a task and polls for completion; internally the full Plan → Execute → Review → Edit workflow runs transparently.

Observability – Structured Logs and Traces

Every significant event – LLM cache hit or miss, provider failure, graph node entry, revision loop iteration – is recorded as a structured log entry via structlog. Log output is machine-parseable JSON in production and human-readable key=value in development.

For distributed tracing and LLM cost analytics, the implementation integrates OpenTelemetry and optionally Langfuse. When telemetry is configured, every orchestration produces a trace spanning goal → plan → execute → review → edit, with LLM token counts and latencies attached to each span.

System Architecture

The system architecture of the reference project is depicted in the following diagram: