How do you coordinate parallel task execution in a crew?

What you need to know

Level 1: async tasks inside a crew

1pricing = Task(description="Collect prices for {brand} rivals.",2               expected_output="A price table", agent=pricing_analyst,3               async_execution=True)4reviews = Task(description="Summarise app-store reviews for {brand} rivals.",5               expected_output="Top 5 complaints per rival", agent=review_analyst,6               async_execution=True)7report = Task(description="Combine into a competitor report.",8              expected_output="A 1-page report", agent=writer,9              context=[pricing, reviews])

pricing and reviews run at the same time; report waits for both. Rules to remember: a crew may end with at most one async task, and an async task cannot use another async task from the same run of async tasks as context.

Level 2: parallel branches in a Flow

1from crewai.flow.flow import Flow, start, listen, and_23class CompetitorFlow(Flow[CompetitorState]):4    @start()5    async def prices(self):6        out = await PricingCrew().crew().akickoff(inputs={"brand": self.state.brand})7        self.state.prices = out.raw89    @start()10    async def reviews(self):11        out = await ReviewCrew().crew().akickoff(inputs={"brand": self.state.brand})12        self.state.reviews = out.raw1314    @listen(and_(prices, reviews))15    def write_report(self):16        ...

Both @start() methods run concurrently. and_ waits for both; or_ would fire on the first.

Level 3: many inputs

await crew.akickoff_for_each(inputs=[...]) runs one copy of the crew per input, concurrently. The plain kickoff_for_each runs them one after another.

Things that go wrong

• Rate limits. Five parallel agents can hit the provider's limit at once. Set max_rpm on the crew or agents.
• No shared scratchpad. Branches do not see each other's progress; share results only at the join.
• Spiky cost. The same tokens, but all at once — watch per-minute quotas.
• Errors at the join. A failure in an async task shows up when results are collected; log per task with task_callback.

A real-life example

A market-research crew for a two-wheeler EV brand collected pricing, reviews and dealer counts for 6 rivals, sequentially. One run took about 7 minutes.

The team made the three collection tasks async and joined them in a report task. Time fell to about 3 minutes. The first week, about 1 run in 5 failed with rate-limit errors, because three agents plus their tool calls fired at once. Adding max_rpm=40 on the crew and a retry with backoff inside the search tool fixed it, at a cost of about 20 extra seconds per run. Token cost did not change — parallelism saves time, not money.

Follow-up questions to expect

• "Does parallelism reduce cost?" — No. The same work is done; only wall-clock time falls. It can raise cost if it leads to retries from rate limits.
• "Can parallel tasks share memory?" — They can read the shared memory store, but they should not depend on each other's in-progress results; join through context.
• "Threads or asyncio?" — Async crew tasks run on threads; Flows and akickoff support native async def methods. Either way the waiting is on network calls, so both help.