“Make it async” is common advice when a Python service slows down. It is also incomplete. If the service spends most of its time waiting on network or disk I/O, concurrency can reduce elapsed time. If the bottleneck is CPU or a saturated dependency, an async rewrite changes the scheduling while the underlying constraint remains.

I use sleep-based examples below because they isolate one behavior: overlapping waits. They prove very little about a real service. That limitation is the point of the walkthrough, which moves from scheduling to the controls I would add before putting concurrent work behind an API.

Understand cooperative scheduling

An event loop works like a dispatcher. It advances work that is ready and parks work that is waiting. When that work can continue, the loop returns to it. A coroutine is a function declared with async def. Calling it creates a coroutine object; you still need to await it or schedule it as a task.

Tasks cooperate by yielding control while waiting. An await expression can suspend the current coroutine. An operation that completes immediately may keep control in the same task. CPU-heavy work and blocking calls inside a coroutine still occupy the event-loop thread. See Python’s coroutines and tasks documentation for the scheduling model.

Concurrency lets operations make progress over overlapping periods. Parallelism involves simultaneous execution. The example below uses concurrency on one event-loop thread.

Compare the same workload

Use the same delay list for both runs so scheduling is the only variable. time.sleep blocks the current thread; asyncio.sleep lets other scheduled tasks run while the delay elapses.

Save this as compare_waits.py and run it with Python:

import asyncio
import time


async def wait_one(delay):
    await asyncio.sleep(delay)


async def run_concurrently(delays):
    await asyncio.gather(*(wait_one(delay) for delay in delays))


def main():
    delays = [0.1, 0.2, 0.3, 0.4]
    start = time.perf_counter()
    for delay in delays:
        time.sleep(delay)
    print(f"Sequential: {time.perf_counter() - start:.3f} seconds")

    start = time.perf_counter()
    asyncio.run(run_concurrently(delays))
    print(f"Concurrent: {time.perf_counter() - start:.3f} seconds")


if __name__ == "__main__":
    main()

Expect the sequential run to take roughly the sum of the delays, 1 second, and the concurrent run roughly the longest delay, 0.4 seconds. Scheduling and system load add overhead, so exact timings vary.

For n equal, independent waits of duration d, the idealized sequential time is n × d; overlapping all waits approaches d plus overhead. The potential speedup grows linearly with the number of equal waits in this idealized model. Real services impose limits through connection pools, rate limits, server capacity, and memory.

The result establishes one fact which is that the event loop can overlap these waits. Throughput for an HTTP client, database, or production service requires measurement with representative inputs and concurrency limits.

Limit work in flight

A semaphore limits how many coroutines can enter a section at once. I reach for one when the workload is finite and already in memory, or when several call paths need to share a fixed amount of downstream capacity. This runnable example allows two simulated requests to wait concurrently:

import asyncio


async def request(item, slots):
    async with slots:
        await asyncio.sleep(0.1)
        return item * 2


async def main():
    slots = asyncio.Semaphore(2)
    results = await asyncio.gather(
        *(request(item, slots) for item in range(6))
    )
    print(results)


if __name__ == "__main__":
    asyncio.run(main())

The result is [0, 2, 4, 6, 8, 10]. gather returns results in input order, even if tasks finish in a different order.

This semaphore bounds active work in its protected section. All six tasks are still scheduled. That distinction is easy to miss.

For a continuous or potentially large input stream, I prefer a bounded queue and a fixed worker pool. It limits active work and pending work, which makes overload visible at one boundary. Decide whether a full queue should block producers, reject work, or drop items. A semaphore alone is overused for this case because it can leave an unbounded number of tasks waiting for a slot.

Keep blocking work off the event loop

A synchronous HTTP request made directly inside async def blocks the event-loop thread until it returns. I would use an asynchronous client when the surrounding stack is already async. For an isolated blocking library, asyncio.to_thread is often the smaller and clearer change. Python’s asyncio development guide covers blocking code and cross-thread interactions.

CPU-heavy Python work needs a separate execution strategy, such as a process pool, when it would otherwise monopolize the event loop. A compute-intensive function keeps the same execution needs after adding async. Small sequential transformations can remain ordinary functions.

When replacing simulated waits with real I/O, add timeouts and define cancellation and failure behavior. Try a slow dependency and a failed request, and inspect whether pending tasks finish or are cancelled as intended. The log-pipeline walkthrough applies these ideas to a reader thread, queue, and asynchronous workers.