Role-Based Agent Teams, Roles, Tasks, Processes
Four primitives: Agent, Task, Crew, Process. Two top-level shapes: Crews (autonomous, role-based collaboration) and Flows (event-driven, deterministic). CrewAI is the 2026 reference implementation, and its docs are blunt: "for any production-ready application, start with a Flow."
Learning Objectives
- Name CrewAI's four primitives (Agent, Task, Crew, Process) and what each owns.
- Distinguish Sequential, Hierarchical, and the planned Consensus process; pick one per workload.
- Distinguish Crews (autonomous role-based) from Flows (event-driven deterministic), and explain the docs' production recommendation.
- Wire tools with the
@tooldecorator andBaseToolsubclass; reason about structured outputs vs free text. - Name the four CrewAI memory types and when each pays off.
- Implement a stdlib three-agent crew (researcher, writer, editor) that produces a brief.
- Spot the three CrewAI failure modes: prompt-bloat, manager-LLM tax, brittle handoffs.
The Problem
Teams adopting multi-agent frameworks hit the same wall. "Autonomous collaboration" sounds great in a demo. Then a customer files a bug and you need deterministic replay. Or finance asks how much an LLM-routed crew costs per run. Or on-call needs to know which agent stalled at 3 AM.
Free-form LLM-routed crews answer none of those cleanly. Pure DAGs answer them all but lose the exploratory shape a brainstorming agent needs.
CrewAI's split is honest about the trade. Crews for collaborative, role-based, exploratory work. Flows for event-driven, code-owned, auditable production. Same framework, two shapes, pick per surface.
The Concept
Four primitives
CrewAI's surface is small. Memorize this and the rest is config.
- Agent.
role + goal + backstory + tools + (optional) llm. The backstory is load-bearing. It shapes tone, judgment, when the agent stops. Tools are functions the agent can call (more below). - Task.
description + expected_output + agent + (optional) context + (optional) output_pydantic. A reusable unit of work.expected_outputis the contract.contextlists upstream tasks whose outputs are passed in.output_pydanticforces a structured shape. - Crew. Container. Owns the list of
agentsthe list oftaskstheprocessand optionalmemory+verbose+manager_llmsettings. - Process. Execution strategy. Sequential, Hierarchical, Consensus (planned). Picks the shape of the run.
Agents do not see each other directly. Tasks reference agents. The Crew sequences tasks. The Process decides who picks the next task. That is the whole mental model.
Validated against CrewAI 0.86 (2026-05). Newer versions may rename or merge process types; check the CrewAI Processes docs before relying on a specific shape.
Sequential vs Hierarchical vs Consensus
- Sequential. Tasks run in declaration order. Output of task N is available as
contextto task N+1. Lowest cost. Most predictable. Use when the order is fixed. - Hierarchical. A manager Agent (separate LLM call) routes between specialists. CrewAI spawns the manager either from your
manager_llmconfig or a default. The manager picks the next task each round and can refuse or re-route. Use when you have four or more specialists and order genuinely depends on prior output. - Consensus. Planned, not currently implemented in the public API. The docs reserve the name for a future voting-based process. Do not rely on it today.
Hierarchical adds a per-round LLM call (the manager) on top of every specialist call. Token cost can triple on a five-step run. Pay for it only when you need the routing.
Crews vs Flows
This is the framing the docs lead with in 2026.
- Crew. LLM-driven autonomy. The framework picks the shape at runtime. Good for: research, brainstorming, first drafts, anywhere the path is part of the answer. Hard to replay. Hard to test. Cheap to prototype.
- Flow. Event-driven graph you own.
@startmarks the entry.@listen(topic)marks a step that fires when another step emits that topic. Each step is plain Python (can call a Crew internally). Good for: production. Observable. Testable. Deterministic.
The docs' 2026 production recommendation: start with a Flow. Fold Crews in as Crew.kickoff() calls from inside Flow steps when autonomy earns its cost. The Flow gives you the audit trail, the Crew gives you the exploration. Compose, do not pick.
Tool integration
Three ways to give an Agent a tool. Pick the simplest one that fits.
@tooldecorator. Pure functions become tools. Signature is the schema; docstring is the description the LLM sees. Best for one-off helpers.Pythonfrom crewai.tools import tool @tool("Search the web") def search(query: str) -> str: """Return top results for the query.""" return run_search(query)BaseToolsubclass. Class-based tool with explicit args schema, async support, retries. Use when the tool has state (a client, a cache) or needs structured args.Pythonfrom crewai.tools import BaseTool from pydantic import BaseModel class SearchArgs(BaseModel): query: str limit: int = 10 class SearchTool(BaseTool): name = "web_search" description = "Search the web and return top results." args_schema = SearchArgs def _run(self, query: str, limit: int = 10) -> str: return self.client.search(query, limit=limit)- Built-in toolkits. CrewAI ships first-party adapters:
SerperDevToolFileReadToolDirectoryReadToolCodeInterpreterToolRagToolWebsiteSearchTool. Wired with one import.
Structured outputs use Pydantic. Pass output_pydantic=MyModel on the Task. CrewAI validates the LLM response against the model and either coerces or retries. Pair this with a tight expected_output string. Free-text outputs are fine for drafts; structured outputs are what downstream Flows can consume.
Memory hooks
CrewAI ships four memory types out of the box. They compose: a Crew can enable all four at once.
Validated against CrewAI 0.86 (2026-05). Recent releases route everything through a unified
Memorysystem that wraps these four stores. The conceptual model below still holds, but the public class surface may collapse to a singleMemoryentry-point in newer versions; check CrewAI memory docs for the current API.
- Short-term. Conversation buffer within a single run. Wiped at the end.
- Long-term. Persisted across runs. Stored in a vector DB (Chroma by default, swappable). Retrieved by similarity to the current task.
- Entity. Per-entity facts. "Customer X is on the enterprise plan." Keyed by entity, not by similarity. Survives across runs.
- Contextual. Assembly-time retrieval. Pulls relevant memory at the moment the Agent needs it, not preloaded.
Enable on the Crew with memory=True or per-type config. Backed by an embeddings provider you configure (defaults to OpenAI, swappable to local). Memory is one of the places CrewAI earns its keep against thinner frameworks; pure LangGraph requires you to wire each of these yourself.
When role-based teams fit
- Three to six agents with named roles and a collaborative workflow. Drafting, reviewing, planning, brainstorming.
- Routing where the LLM's judgment about the next step is part of the value (Hierarchical).
- Anywhere the team is happier reading
role + goal + backstorythan reading a graph definition.
When they do not
- Deterministic DAGs with strict ordering. Use LangGraph (Lesson 13). The graph shape is the right abstraction; CrewAI's role framing is friction.
- Sub-second latency budgets. Hierarchical adds round trips. Even Sequential serializes prompts that include backstories and prior outputs.
- Single-agent loops. Skip the framework; an agent loop (Lesson 1) plus a tool registry is shorter.
Lesson 17 (Agent Framework Tradeoffs) lays this out in a matrix. The short version: CrewAI sits in the "collaborative role-based" corner.
Dependency shape
Independent of LangChain. Python 3.10 to 3.13. Uses uv. Star count: see crewAIInc/crewAI (snapshot as of 2026-05). AWS Bedrock integration is documented; vendor benchmarks report a substantial speedup vs LangGraph on QA workloads, but the methodology (dataset, hardware, evaluation metric) is not published, so treat framework-vendor numbers as directional only.
Where this pattern goes wrong
- Prompt-bloat from backstories. A 2000-word backstory per agent and a five-agent crew burns the context budget before the first tool call. Keep backstories under 200 words. Reuse phrases across agents; do not repeat house style five times.
- Manager-LLM token tax. Hierarchical process adds a manager LLM call before every specialist call. On a five-task crew that is six LLM calls instead of five, and the manager call carries the full task list plus prior outputs. Switch to Sequential unless routing depends on output.
- Brittle handoffs. Task N's
expected_outputis "an outline". Task N+1 reads it ascontextand tries to parse three sections. The LLM produced four. The downstream Agent ad-libs. Fix withoutput_pydanticon Task N so Task N+1 reads a typed object, not free text. - Crew-as-prod. Free-form Crew shipped to production without a Flow wrapper. Output variability is high; replay is impossible; on-call cannot diff a bad run against a good one. Wrap with a Flow.
Build It
code/main.py implements stdlib versions of both shapes plus a three-agent crew.
Shape:
AgentTaskdataclasses matching CrewAI's surface.SequentialCrew.kickoff(inputs)runs tasks in declaration order, threading outputs ascontext.HierarchicalCrew.kickoff(topic)adds a manager Agent picking the next specialist each round, stops at "done".Flowwith@startand@listen(topic)decorators, a tiny event loop, and a trace.tool(name)decorator mirroring CrewAI's@toolshape.Memorywithshort_termlong_termentitystores; mocked similarity uses numpy.- Mock LLM responses are hardcoded strings keyed off role plus input prefix. No network. Deterministic.
Concrete demo: researcher, writer, editor crew producing a brief on "agent engineering 2026". Researcher pulls (mocked) sources. Writer drafts. Editor tightens. Same crew runs through a Flow to show the deterministic shape.
Run it:
python3 code/main.pyTrace covers: sequential crew threading outputs through contexthierarchical crew with manager picks (researcher, writer, editor, then "done"), flow running the same three steps with explicit topics (researcheddraftededited), tool calls routed through @tooland long-term memory surviving across two kickoffs.
The Crew trace is fluid; the manager could in principle re-order. The Flow trace is fixed. That choice is the lesson.
Use It
- CrewAI Flow for production. Even when the Flow is one step that calls
Crew.kickoff(). The Flow gives the audit boundary. - CrewAI Crew (Sequential) for clear-ordering collaborative work, especially first drafts and review loops.
- CrewAI Crew (Hierarchical) when routing depends on output and you have four or more specialists.
- LangGraph (Lesson 13) for explicit state machines, durable resume, strict ordering.
- AutoGen v0.4 (Lesson 14) for actor-model concurrency and fault isolation.
- OpenAI Agents SDK (Lesson 16) for OpenAI-first products with handoffs and guardrails.
- Claude Agent SDK (Lesson 17) for Claude-first products with subagents and session store.
Ship It
outputs/skill-crew-or-flow.md picks Crew vs Flow for a task and scaffolds the minimal implementation. Hard rejects on Crew-without-backstory, Flow-without-explicit-topics, Hierarchical with under three specialists.
Pitfalls
- Backstory as flavor. It shapes outputs. Test three variants per agent; variance is real. Pick one, freeze it.
- Skipping
expected_output. Without a contract per task, downstream tasks pick up whatever the LLM produced. Crew runs; audit fails. - Memory always-on. Long-term writes every run. Vector DB grows. Retrieval gets noisy. Scope writes to tasks where the fact is persistent.
- Manager prompt drift. Hierarchical's manager prompt is implicit. If routing gets weird, dump it in verbose mode and read.
- Tool side effects in Crews. A Crew can call a tool more times than expected. POST, DELETE, payment belong in a Flow step, never a Crew tool.
Exercises
- Convert the Sequential crew to a Flow. Count the touchpoints where variability drops. Note where readability dropped.
- Add entity memory to the crew: facts about a customer persist across kickoffs. Verify retrieval pulls the right entity.
- Implement a Hierarchical process where the manager refuses to route to the editor until the writer's output has at least three paragraphs. Trace the retry.
- Wire a
BaseToolsubclass for a (mocked) web search. Compare the trace shape vs the@tooldecorator version. - Add
output_pydantic=Briefto the editor task, whereBriefhastitlesummarysections. Make the writer task output malformed JSON once; verify CrewAI's retry behavior in the trace. - Read CrewAI's docs intro. Port the toy to the real
crewaiAPI. Which guarantees did the stdlib version skip? - Wire AgentOps or Langfuse (Lesson 24) to a real run. Which traces did you miss in the stdlib version?
Key Terms
| Term | What people say | What it actually means |
|---|---|---|
| Agent | "Persona" | Role + goal + backstory + tools |
| Task | "Unit of work" | Description + expected output + assignee + optional structured output |
| Crew | "Agent team" | Container for Agents + Tasks + Process |
| Process | "Execution strategy" | Sequential / Hierarchical / Consensus (planned) |
| Flow | "Deterministic workflow" | Event-driven, code-owned, testable |
| Backstory | "Persona prompt" | Tone and judgment shaper for the Agent |
@tool | "Function tool" | Decorator that turns a function into a tool the Agent can call |
BaseTool | "Class tool" | Class-based tool with args schema, retries, async support |
| Entity memory | "Per-entity facts" | Memory scoped to a customer / account / issue |
| Long-term memory | "Cross-run memory" | Vector-backed memory that survives between kickoffs |
| Contextual memory | "Just-in-time retrieval" | Memory pulled at the moment the Agent needs it |
| Manager LLM | "Router agent" | Extra LLM in Hierarchical process that picks the next task |
expected_output | "Task contract" | String that tells the Agent (and audit) what shape to return |
Further Reading
- CrewAI docs introduction: concepts and the recommended production path
- CrewAI Flows guide: event-driven shape,
@start@listen - CrewAI tools reference:
@toolBaseToolbuilt-in toolkits - CrewAI memory: short-term, long-term, entity, contextual
- Anthropic, Building Effective Agents: when multi-agent helps and when it does not
- LangGraph overview: the state-machine alternative
Check what you learned
7 questions. Answer each one to see why.
What are CrewAI's four primitives?
The full code
Every file for this lesson. Copy it, run it, then change it.
"""CrewAI-shaped Crew and Flow primitives in stdlib.
Three-agent crew (researcher, writer, editor) producing a brief on
"agent engineering 2026". Same crew is run Sequential, Hierarchical, and
through a Flow to show all three execution shapes.
Stdlib + numpy. Mock LLM responses are deterministic hardcoded strings
keyed off agent role and input prefix.
"""
from __future__ import annotations
import hashlib
from dataclasses import dataclass, field
from typing import Any, Callable
import numpy as np
def tool(name: str) -> Callable[[Callable[..., str]], Callable[..., str]]:
"""Mirror of CrewAI's @tool decorator. Marks a function as a tool the
Agent can call. Docstring is the description; signature is the schema."""
def decorator(fn: Callable[..., str]) -> Callable[..., str]:
fn.tool_name = name # type: ignore[attr-defined]
fn.is_tool = True # type: ignore[attr-defined]
return fn
return decorator
@tool("Search the web")
def search(query: str) -> str:
"""Return top results for the query."""
fixtures = {
"agent engineering": "src1: agent loop, src2: tool use, src3: memory",
"crewai": "src1: docs intro, src2: flows guide, src3: tools ref",
}
for key, value in fixtures.items():
if key in query.lower():
return value
return "src1: generic, src2: generic, src3: generic"
@dataclass
class Agent:
role: str
goal: str
backstory: str
fn: Callable[..., str]
tools: list[Callable[..., str]] = field(default_factory=list)
@dataclass
class Task:
description: str
expected_output: str
agent: Agent
context: list["Task"] = field(default_factory=list)
@dataclass
class SequentialCrew:
agents: list[Agent]
tasks: list[Task]
memory: "Memory | None" = None
def kickoff(self, inputs: dict[str, Any]) -> list[str]:
outputs: list[str] = []
prior = inputs.get("topic", "")
by_task: dict[int, str] = {}
for task in self.tasks:
if task.context:
# CrewAI behavior: feed outputs of every declared upstream task
# into the current one. Falls back to prior when none declared.
joined = "\n\n".join(
by_task[id(t)] for t in task.context if id(t) in by_task
)
agent_input = joined or prior
else:
agent_input = prior
out = task.agent.fn(agent_input, task.agent.tools, self.memory)
outputs.append(f"[{task.agent.role}] {out}")
by_task[id(task)] = out
prior = out
if self.memory is not None:
self.memory.write_short_term(task.agent.role, out)
self.memory.write_long_term(task.agent.role, out)
return outputs
@dataclass
class HierarchicalCrew:
manager: Agent
specialists: dict[str, Agent]
max_steps: int = 5
memory: "Memory | None" = None
def kickoff(self, topic: str) -> list[str]:
outputs: list[str] = []
current = topic
done: set[str] = set()
for _ in range(self.max_steps):
pick = self.manager.fn(done, [], None)
if pick == "done":
outputs.append("[manager] done")
break
specialist = self.specialists.get(pick)
if specialist is None:
outputs.append(f"[manager] unknown pick {pick!r}")
break
out = specialist.fn(current, specialist.tools, self.memory)
outputs.append(f"[manager -> {specialist.role}] {out}")
current = out
done.add(pick)
if self.memory is not None:
self.memory.write_short_term(specialist.role, out)
return outputs
class Flow:
"""Deterministic event-driven workflow. @start fires on kickoff;
@listen(topic) fires when another step emits that topic.
"""
def __init__(self) -> None:
self.start_step: Callable[[Any], tuple[str, Any]] | None = None
self.listeners: dict[str, Callable[[Any], tuple[str, Any] | None]] = {}
self.trace: list[tuple[str, str, Any]] = []
def start(self, fn: Callable[[Any], tuple[str, Any]]) -> Callable[..., Any]:
self.start_step = fn
return fn
def listen(self, topic: str) -> Callable[[Callable[..., Any]], Callable[..., Any]]:
def decorator(fn: Callable[[Any], tuple[str, Any] | None]) -> Callable[..., Any]:
self.listeners[topic] = fn
return fn
return decorator
def kickoff(self, payload: Any) -> list[tuple[str, str, Any]]:
if self.start_step is None:
return []
self.trace = []
topic, out = self.start_step(payload)
self.trace.append(("start", topic, out))
while topic in self.listeners:
step = self.listeners[topic]
result = step(out)
if result is None:
break
topic, out = result
self.trace.append((step.__name__, topic, out))
return self.trace
class Memory:
"""Four-store memory matching CrewAI's short, long, entity, contextual.
Long-term retrieval uses numpy cosine similarity on hashed token vectors.
"""
def __init__(self, dim: int = 16) -> None:
self.dim = dim
self.short_term: list[tuple[str, str]] = []
self.long_term: list[tuple[str, str, np.ndarray]] = []
self.entity: dict[str, dict[str, str]] = {}
def _embed(self, text: str) -> np.ndarray:
seed = int.from_bytes(
hashlib.sha256(text.encode("utf-8")).digest()[:8],
"little",
)
rng = np.random.default_rng(seed)
v = rng.standard_normal(self.dim)
n = np.linalg.norm(v)
return v / n if n > 0 else v
def write_short_term(self, role: str, value: str) -> None:
self.short_term.append((role, value))
def write_long_term(self, role: str, value: str) -> None:
self.long_term.append((role, value, self._embed(value)))
def write_entity(self, entity_id: str, key: str, value: str) -> None:
self.entity.setdefault(entity_id, {})[key] = value
def recall_long_term(self, query: str, k: int = 2) -> list[tuple[str, str, float]]:
if not self.long_term:
return []
q = self._embed(query)
scored = [(r, v, float(np.dot(q, e))) for r, v, e in self.long_term]
scored.sort(key=lambda row: row[2], reverse=True)
return scored[:k]
def reset_short_term(self) -> None:
self.short_term = []
def _researcher(prior: Any, tools: list[Callable[..., str]], memory: Memory | None) -> str:
topic = prior if isinstance(prior, str) else ""
# Run whichever search-ish tool the agent was wired with, in order.
search_fn = next(
(t for t in tools if getattr(t, "is_tool", False) and "search" in getattr(t, "tool_name", "").lower()),
None,
)
sources = search_fn(topic) if search_fn else "src1, src2, src3"
return f"3 sources on {topic}: {sources}"
def _writer(prior: Any, tools: list[Callable[..., str]], memory: Memory | None) -> str:
text = prior if isinstance(prior, str) else ""
return f"draft (3 paragraphs) from sources: {text[:60]}"
def _editor(prior: Any, tools: list[Callable[..., str]], memory: Memory | None) -> str:
text = prior if isinstance(prior, str) else ""
return f"final brief (tightened, 800 words): {text[:60]}"
def _manager(prior: Any, tools: list[Callable[..., str]], memory: Memory | None) -> str:
done = prior if isinstance(prior, set) else set()
if "researcher" not in done:
return "researcher"
if "writer" not in done:
return "writer"
if "editor" not in done:
return "editor"
return "done"
def build_agents() -> tuple[Agent, Agent, Agent]:
researcher = Agent(
role="researcher",
goal="find 3 credible sources",
backstory="former librarian. terse. cites primaries.",
fn=_researcher,
tools=[search],
)
writer = Agent(
role="writer",
goal="turn sources into a draft",
backstory="editorial voice. paragraphs of three.",
fn=_writer,
)
editor = Agent(
role="editor",
goal="tighten draft to final brief",
backstory="cuts adjectives. enforces house style.",
fn=_editor,
)
return researcher, writer, editor
def main() -> None:
print("=" * 70)
print("CREWAI CREW AND FLOW - Phase 14, Lesson 15")
print("=" * 70)
researcher, writer, editor = build_agents()
memory = Memory()
print("\n1. SequentialCrew (researcher -> writer -> editor)")
seq = SequentialCrew(
agents=[researcher, writer, editor],
tasks=[
Task("research the topic", "3 sources", researcher),
Task("write a draft", "3 paragraphs", writer),
Task("edit to final brief", "800 words", editor),
],
memory=memory,
)
for line in seq.kickoff({"topic": "agent engineering 2026"}):
print(f" {line}")
print("\n2. HierarchicalCrew (manager routes)")
manager = Agent(
role="manager",
goal="pick next specialist",
backstory="PM background. routes by missing role.",
fn=_manager,
)
hcrew = HierarchicalCrew(
manager=manager,
specialists={"researcher": researcher, "writer": writer, "editor": editor},
memory=memory,
)
for line in hcrew.kickoff("agent engineering 2026"):
print(f" {line}")
print("\n3. Flow (deterministic, event-driven)")
flow = Flow()
@flow.start
def kickoff(topic: str) -> tuple[str, str]:
out = _researcher(topic, [search], memory)
memory.write_short_term("researcher", out)
memory.write_long_term("researcher", out)
return "researched", out
@flow.listen("researched")
def on_researched(prior: str) -> tuple[str, str]:
out = _writer(prior, [], memory)
memory.write_short_term("writer", out)
memory.write_long_term("writer", out)
return "drafted", out
@flow.listen("drafted")
def on_drafted(prior: str) -> tuple[str, str]:
out = _editor(prior, [], memory)
memory.write_short_term("editor", out)
memory.write_long_term("editor", out)
return "edited", out
@flow.listen("edited")
def on_edited(prior: str) -> None:
return None
for step_name, topic, output in flow.kickoff("agent engineering 2026"):
print(f" [{step_name}] topic={topic!r} out={output[:60]}")
print("\n4. Memory: recall_long_term('brief')")
for role, value, score in memory.recall_long_term("brief"):
print(f" [{role}] score={score:+.3f} value={value[:50]}")
print("\n5. Second kickoff (long-term memory survives)")
memory.reset_short_term()
seq2 = SequentialCrew(
agents=[researcher, writer, editor],
tasks=[
Task("research", "3 sources", researcher),
Task("draft", "3 paragraphs", writer),
Task("edit", "800 words", editor),
],
memory=memory,
)
seq2.kickoff({"topic": "agent engineering 2026"})
print(f" long_term entries: {len(memory.long_term)}")
print(f" short_term entries (this run): {len(memory.short_term)}")
print()
print("Crew: LLM picks the shape. Flow: code owns the shape.")
print("Docs (2026): start production with a Flow; fold Crews in as sub-steps.")
if __name__ == "__main__":
main()