Python asyncio Recipes/Bound a worker pool with Semaphore

Throttle async work to a safe level.

Section: Bounded Concurrency

Bound a worker pool with Semaphore

python
python
import asyncio

sem = asyncio.Semaphore(5)

async def work(item):
    async with sem:
        await asyncio.sleep(0.2)
        return item * 10

async def main():
    results = await asyncio.gather(*(work(i) for i in range(20)))
    print(results[:5])

asyncio.run(main())
Explanation

A standard pattern for APIs, crawlers, and batch processors.

Learn the surrounding workflow

Compare similar commands or jump into common fixes when this command is part of a bigger troubleshooting path.

Related commands

Same sheet · prioritizing Bounded Concurrency
Process input in batches
Break large workloads into fixed-size async chunks.
OpenIn sheetpythonsame section
Retry an async operation
Retry with simple exponential backoff.
Inspect running tasks
List currently active tasks in the running loop.
Cancel cleanly on shutdown
Maintain and cancel outstanding tasks at shutdown time.
Enable asyncio debug mode
Turn on extra diagnostics while developing.