Abstract flowing gradient in deep indigo and blue tones, smooth and luminous, evoking a modern digital learning atmosphere

Computer Science and programming articles. We do not sell courses.

Running concurrent I/O in Python with the asyncio library

Most server-side Python programs spend more time waiting than they do computing. A scraper fetching five hundred pages, a chat backend holding open a thousand WebSockets, or a data pipeline pinging dozens of SaaS endpoints will sit almost entirely idle while the operating system moves bytes. asyncio reclaims that idle time without the weight of OS threads. It is a library, a syntax, and a runtime convention that lets one thread juggle thousands of in-flight operations by yielding voluntarily whenever it would block.

The promise is concrete: while one coroutine waits for bytes from a socket, another can parse the previous response, write to a database, or compute the next hash. In Australia, where the National Broadband Network has trimmed average latencies to a few tens of milliseconds, asyncio matters because it lets you stack hundreds of those waits on top of each other. The library cannot make a single disk read faster, but it lets a program do dozens of things in the time one would normally wait for the slowest of them.

This article walks through the moving parts of asyncio in a pragmatic order. We begin with the conceptual model so the later syntax makes sense, work through a fan-out URL fetch, then cover the patterns and pitfalls that show up in real codebases.

How asyncio compares to threads and processes

Threads hand scheduling to the kernel, which is preemptive: the OS can stop a thread at any instruction and switch to another. asyncio inverts that. Your code decides when to pause via an await, and the runtime then chooses what runs next. The trade-off is that you never have to reason about locks for shared state, since only one coroutine runs at a time on a given thread, but you must never call a blocking function from inside a coroutine, or the loop stalls behind it.

For CPU-bound work — anything where the GIL dominates, like training a model or compressing a file — neither threads nor asyncio help within one process; reach for multiprocessing, NumPy, or a C extension instead. For I/O, asyncio is usually the lighter option. A thread carries roughly a megabyte of stack and a kernel handle; a coroutine carries a few hundred bytes. That ratio is what lets a single process in a Brisbane datacentre keep tens of thousands of slow clients happy on commodity hardware.

A useful test before reaching for asyncio: if the work is mostly waiting on sockets, files, subprocesses, timers, or queues, asyncio will probably pay off. If the work is mostly crunching numbers, it will not. Mixing the two is fine, but run CPU work in a ProcessPoolExecutor so it never touches the loop.

Coroutines, tasks, and the event loop in plain terms

A coroutine is a function declared with async def. Calling it does not execute it; it returns a coroutine object the loop must drive. The most common way is to await it from another coroutine, or to wrap it with asyncio.create_task(), which schedules it to run soon on the loop. Tasks are the unit of concurrency in asyncio: you cancel them, collect them with gather, and time them out with wait_for.

The event loop is a queue of callbacks plus a registry of file descriptors and timers. When you await something, the runtime asks the loop whether the awaited thing is ready. If yes, the coroutine resumes; if no, it parks and the loop picks up another task. Once the kernel signals that a socket has bytes, the loop schedules the waiting task back, and the coroutine resumes with its locals intact.

The single line that almost every script ends up containing is asyncio.run(main()). It creates a loop, runs the supplied coroutine to completion, then tears the loop down cleanly. Behind that call sit create_task, gather, wait_for, sleep, and the various transport classes, but most real code lives in the top half of that surface area. For a comparable look at how a small set of primitives composes into something useful, the separate chaining walkthrough on this site shows a similar design pattern.

A worked example: fetching many pages concurrently

The canonical asyncio demo is fetching a list of URLs. The synchronous requests library forces you to sit through each round trip in series; httpx.AsyncClient lets you launch them all and await them in parallel:

import asyncio
import httpx

async def fetch(client, url):
    r = await client.get(url, timeout=10.0)
    return url, r.status_code, len(r.content)

async def main(urls):
    async with httpx.AsyncClient() as client:
        tasks = [fetch(client, u) for u in urls]
        return await asyncio.gather(*tasks)

if __name__ == "__main__":
    urls = ["https://example.com"] * 20
    for result in asyncio.run(main(urls)):
        print(result)

The interesting lines are gather and the list comprehension. Each call returns a task immediately because create_task is invoked under the hood when gather is given coroutines, and the coroutines then interleave on the single thread. Twenty 200 ms requests finish in roughly 200 ms total instead of four seconds, with no extra processes and no GIL contention.

Reuse a single AsyncClient; opening one per request defeats most of the speedup and can upset the ACSC's connection-hygiene expectations. Wrap each fetch with a timeout so a stalled server cannot pin a worker. Pass return_exceptions=True to gather when each URL should be independent — otherwise the first failure cancels its siblings. When responses come back, you often want to look them up against a local index, and the B-tree database indexing article on this site walks through the kind of storage layer that sits well behind an async workload.

Patterns that earn their keep in real codebases

A handful of recipes covers most production needs. The same constructs come up repeatedly in code reviews, and a working knowledge of them sees most jobs through:

Two patterns deserve attention. The first is graceful shutdown in long-running services. When SIGTERM arrives, stop accepting new work, cancel in-flight requests with a deadline, and flush logs. A common idiom is a main task that owns the server, plus an asyncio.Event the signal handler sets. Cancelling the main task cancels its children, and asyncio.shield can protect pieces you do not want interrupted, such as a half-written audit row that the Privacy Act 1988 expects intact.

The second is backpressure on producers. If a slow consumer cannot keep up, an unbounded queue grows without limit and your memory bill climbs. Use queue with a maxsize, or have the producer await queue.put() conditionally so it slows when the buffer is full. This is what separates a script that runs in a notebook from a service that sits in production for months.

Pitfalls that bite beginners and experienced engineers alike

The most common asyncio bug is still the blocking call. time.sleep becomes await asyncio.sleep; requests.get becomes an httpx or aiohttp call; a synchronous file read in newer free-threaded builds stays synchronous unless it is wrapped in a library that knows about the loop. If you genuinely must run blocking code, hand it to loop.run_in_executor and let a thread pool handle it while the loop stays responsive.

The second pitfall is losing track of tasks. Calling create_task and never awaiting leaves it in limbo until the loop shuts down, swallowing exceptions silently. Either keep the task handle and await it later, or use TaskGroup, which raises all exceptions at the exit of the with block. A three-line wrapper that logs unhandled exceptions on dangling tasks catches more production incidents than people expect.

The third is mixing sync and async without a boundary. A Django ORM call inside an async view is fine if you wrap it in sync_to_async; a database driver that talks to psycopg directly without an async wrapper will block the loop. Most Australian teams adopting asyncio audit their dependency list, checking each entry for an async client, before shipping a hybrid release.

Debugging, testing, and measuring the payoff

The first sign of an asyncio bug is usually a hang. The loop is stuck waiting for something that will never finish because a task was never awaited, or because a blocking call is wedging the run queue. Printing asyncio.all_tasks() reveals forgotten tasks; PYTHONASYNCIODEBUG=1 or loop.set_debug(True) warns about unawaited coroutines and slow callbacks. The third-party aiomonitor attaches a web shell so you can poke a live process from a browser, invaluable during an outage in a Sydney region.

A few habits make async debugging far less painful:

Testing async code is straightforward once you stop calling coroutines directly. The pytest-asyncio plugin turns async test functions into proper tests, and pytest-anyio provides the same for anyio-based libraries. Keep the tests small, isolate the loop, and prefer async fake HTTP servers such as aiohttp's test_utils over mocks that reimplement too much transport logic. Coverage tools generally work with async code; tracing does too, although expect a few extra frames compared to synchronous runs.

Finally, measure. A single time.perf_counter pair around the main call gives an honest answer: run the synchronous baseline, run the gather-based version, and report both numbers in the pull request. The AEST time of day does not change the result, but it is worth running the test twice — morning when the Adelaide-to-Sydney path is uncongested, evening when it is not — because real concurrency wins compound across noisy networks. If the speedup is in the right ballpark, ship it; if not, the bottleneck is elsewhere, and asyncio was never the lever you needed.