AWS Agent Squad Cheat Sheet

• An open-source framework for orchestrating and routing user queries across multiple specialized AI agents.
• It uses LLM-based intent classification to dynamically assign tasks to the best-suited agent, such as Amazon Bedrock models, Lex bots, or Lambda functions, while maintaining unified conversation context for seamless interactions.

Key Features

• Intelligent Intent Classification:

  • Dynamically routes queries to the most suitable agent by analyzing context and content.

• Flexible Agent Responses:

  • Natively supports both streaming (token-by-token) and non-streaming (complete block) responses.

• Context Management:

  • Maintains and leverages conversation context across multiple agents to support coherent, multi-turn interactions.

• Extensible Architecture:

  • Modular design allows for easy integration of new agents or customization of existing ones.


• Universal Deployment:

  • Capable of running anywhere, from AWS Lambda to local environments or any cloud platform.

• Scalable Design:

  • Built to handle multiple concurrent conversations, scaling from simple chatbots to complex AI systems.

• Agent Overlap Analysis:

  • Built-in tooling to analyze and optimize agent configurations by detecting functional overlaps.

• Pre-configured Agents:

  • Ready-to-use implementations powered by Amazon Bedrock models.

Use Cases

• Chainlit Chat App:
  • Full-stack chat with real-time streaming and multi-agent routing (e.g., Tech, Travel, Health agents).
• Chat Demo:
  • Lightweight web demo for testing routing logic and conversation history.
• E-commerce Simulator:
  • Simulates customer support with agents for orders, product info, and human handoff.
• FastAPI Streaming:
  • REST API example using Server-Sent Events (SSE) for token-by-token streaming.
• TypeScript Local Demo:
  • Node.js/TypeScript setup for local agent routing tests.
• Python Local Demo:
  • Python setup for local prototyping of agents and classification logic.
• API Agent:
  • Pattern for agents that fetch real-time data from REST or GraphQL APIs.
• Ollama Agent:
  • Integrates local LLMs via Ollama for private or offline use.
• Ollama Classifier:
  • Uses a local Ollama model for intent classification to reduce cloud costs.

How AWS Squad Agents Work

1. Input Processing:
   • The system receives the user query and session metadata.
2. Context Retrieval:
   • The Orchestrator fetches conversation history from the configured storage.
3. Intent Classification:
   • The Classifier evaluates input, context, and agent descriptions to determine intent.
4. Agent Selection:
   • The system identifies the optimal agent(s) to handle the request.
5. Request Execution:
   • The query is routed to the selected agent for processing.
6. Response Generation:
   • The agent executes logic and generates a response (stream or text).
7. Context Update:
   • The system captures the interaction and updates the conversation history.
8. Output Delivery:
   • The final response is delivered to the user.

Agents Supported (Built-in Agents in AWS Agent Squad)

• Bedrock LLM Agent:

  • A direct interface for Amazon Bedrock Foundation Models.

• Amazon Bedrock Agent:

  • A wrapper for managed Amazon Bedrock Agents (supports Action Groups/Knowledge Bases).

• Amazon Lex Bot:

  • An interface for invoking Amazon Lex chatbots for structured intent fulfillment.

• Lambda Agent:

  • An invoker for AWS Lambda functions to execute deterministic business logic.

• OpenAI Agent:

  • An integration for OpenAI models (e.g., GPT-4).

• Custom Agents:

  • Extensible classes for creating bespoke agents with custom logic.

Core Concepts

• Orchestrator

  • The Orchestrator is the central component responsible for managing the lifecycle of user interactions. It coordinates the flow of data between the Classifier, Agents, and Storage components, handling error resolution, fallback logic, and context propagation.

• Classifiers

  • The Classifier is the component responsible for analyzing user input and conversation history to determine the appropriate agent for a given request.
    • Bedrock Classifier: Uses Amazon Bedrock models for intent classification.
    • Anthropic Classifier: Uses Anthropic models for intent classification.
    • OpenAI Classifier: Uses OpenAI models for intent classification.

• Conversation Storage

  • Conversation Storage is the persistence layer that stores and retrieves conversation history to maintain context across sessions.
    • In-Memory: Stores history in RAM (non-persistent).
    • DynamoDB: Persists history in Amazon DynamoDB (scalable, serverless).
    • Redis: Persists history in a Redis cluster (low-latency).

• Agents

  • An Agent is an encapsulated unit of logic or AI capability that handles specific tasks.
  • Key Properties:
    • name: (String) A unique display name for the agent.
    • id: (String) A unique identifier for the agent (auto-generated from name if omitted).
    • description: (String) A detailed description of the agent’s capabilities, used by the classifier for routing.
    • save_chat: (Boolean) Determines if the chat history for this agent should be persisted.
    • callbacks: (AgentCallbacks) An optional object for handling lifecycle events (e.g., on_llm_new_token).

• Retrievers

  • Retrievers are components that fetch relevant external data to augment the orchestration process. They can provide additional context to the Classifier for better routing or supply data to an Agent before execution (RAG).

Pricing

• AWS Agent Squad is free, open-source software. Operational costs are based on AWS service usage:
  • Inference: Pay-per-token for Classifiers and Agents (Amazon Bedrock, etc.).
  • Compute: Pay-for-duration for hosting (AWS Lambda, Amazon EC2/ECS).
  • Storage: Pay-for-capacity/storage for history (Amazon DynamoDB).

AWS Squad Agents Cheat Sheet References:

https://awslabs.github.io/agent-squad/

https://github.com/awslabs/agent-squad

https://aws.amazon.com/solutions/guidance/multi-agent-orchestration-on-aws/