Python asyncio Streams and Subprocesses

Network streams, subprocesses, and non-blocking integration patterns.

View
StandardDetailedCompact
Export
Copy the compact sheet, download it, or print it.
Download
`D` dense toggle · `C` copy all

Streams

TCP clients and stream I/O.

Open a TCP connection

Use stream reader and writer objects.

pythonANYstreamstcpclient
python
import asyncio

async def main():
    reader, writer = await asyncio.open_connection("example.com", 80)
    writer.write(b"GET / HTTP/1.1
Host: example.com
Connection: close

")
    await writer.drain()
    data = await reader.read(200)
    print(data.decode(errors="replace"))
    writer.close()
    await writer.wait_closed()

asyncio.run(main())

High-level streams are convenient for simple protocol clients.

Start a TCP server

Serve client connections with async handlers.

pythonANYserverstreamstcp
python
import asyncio

async def handle(reader, writer):
    data = await reader.readline()
    writer.write(b"echo: " + data)
    await writer.drain()
    writer.close()
    await writer.wait_closed()

async def main():
    server = await asyncio.start_server(handle, "127.0.0.1", 9000)
    async with server:
        await server.serve_forever()

asyncio.run(main())

Use `start_server()` for lightweight TCP services.

Subprocesses

Run and communicate with child processes asynchronously.

Run a subprocess with exec

Capture stdout and stderr asynchronously.

pythonANYsubprocessexecstdout
python
import asyncio

async def main():
    proc = await asyncio.create_subprocess_exec(
        "python",
        "-c",
        "print('hello from child')",
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
    )
    stdout, stderr = await proc.communicate()
    print(stdout.decode().strip(), proc.returncode)

asyncio.run(main())

Use `create_subprocess_exec()` when you want argument-safe subprocess calls.

Run a shell command asynchronously

Spawn a shell pipeline with `create_subprocess_shell()`.

pythonANYsubprocessshellpipeline
python
import asyncio

async def main():
    proc = await asyncio.create_subprocess_shell(
        "echo one && echo two",
        stdout=asyncio.subprocess.PIPE,
    )
    stdout, _ = await proc.communicate()
    print(stdout.decode())

asyncio.run(main())

Convenient for shell pipelines, but avoid untrusted input.

Threads and Blocking Code

Offload blocking work without freezing the event loop.

Run blocking I/O in a thread

Use `asyncio.to_thread()` for simple thread offloading.

pythonANYto-threadblockingthreads
python
import asyncio
import time

def blocking_read():
    time.sleep(1)
    return "finished"

async def main():
    result = await asyncio.to_thread(blocking_read)
    print(result)

asyncio.run(main())

`asyncio.to_thread()` is a high-level helper for running I/O-bound blocking functions in a separate thread.

Use `run_in_executor()`

Offload work with explicit loop executor integration.

pythonANYexecutorrun-in-executorblocking
python
import asyncio
import time

def blocking():
    time.sleep(1)
    return 42

async def main():
    loop = asyncio.get_running_loop()
    result = await loop.run_in_executor(None, blocking)
    print(result)

asyncio.run(main())

A lower-level alternative to `to_thread()` when you need more control.

Recommended next

No recommendations yet.