The Complete Production SOP: How to Deploy and Run HTTPX Modern Asynchronous & Synchronous HTTP Client in Autonomous AI Agents
Production SOP to install and run the HTTPX Async HTTP Client skill in Codex, Claude Code, and Cursor. Master connection pools, timeouts, and streaming.
Technical Overview & Architectural Context
When autonomous coding agents generate HTTP client logic in Python without strict procedural boundaries, they frequently fall back to blocking `requests` calls inside asynchronous loops, instantiate unmanaged `httpx.AsyncClient` instances on every invocation leading to socket exhaustion, and omit connection pooling configurations. Furthermore, agents regularly read unbounded streaming responses into memory all at once with `response.read()` rather than processing chunks via `aiter_bytes()`, precipitating Out-Of-Memory (OOM) failures under heavy load. The `httpx-async-http-client` skill enforces deterministic best practices across autonomous runtime environments like Codex, Claude Code, Cursor, and Windsurf. It establishes rigorous guardrails around connection pooling using `httpx.Limits`, configures explicit granular timeouts across connect, read, and write phases via `httpx.Timeout`, and dictates resilient resource cleanup through context managers (`async with`). It also mandates comprehensive unit testing boundaries through declarative `respx` mocking, preventing external network calls during automated CI/CD and self-verification runs.
Prerequisites & Environment Requirements
Verified Search Intents & Long-Tail Queries Solved
This operational tutorial is engineered to provide deterministic solutions for these developer search queries:
Step-by-Step Execution Playbook
Local Environment Setup & SKILL.md Mounting
Create the canonical skill manifest directory in the project repository and inject the verified procedural rules.
Autonomous agent runtimes inspect the `.agents/skills` or `.claude/skills` directories to index deterministic operational runbooks. By placing the `httpx-async-http-client` manifest inside the workspace, the agent reads frontmatter triggers such as 'async http request httpx' and 'configure httpx connection pool' during plan formulation, ensuring it selects modern async patterns over obsolete synchronous patterns.
mkdir -p .agents/skills/httpx-async-http-client && curl -fsSL https://askills.dev/api/skills/httpx-async-http-client/raw -o .agents/skills/httpx-async-http-client/SKILL.md---
name: "httpx-async-http-client"
description: "Make asynchronous HTTP requests with HTTP/2 support, connection pooling, and connection timeouts."
metadata:
author: "Encode"
version: "1.0.0"
runtime_compatibility:
- "codex"
- "claude-code"
- "cursor"
- "windsurf"
triggers:
- "async http request httpx"
- "configure httpx connection pool"
- "stream large file httpx"
permissions:
fs_read: true
fs_write: "workspace"
shell_exec: false
net_access: false
---Keep the SKILL.md rules concise and trigger-driven. Modern agents only ingest the YAML header during discovery and pull full procedural rules on trigger match, saving up to 80% context tokens.
Ensure directory permissions match current agent runner user UID; read-only mounts in Docker-based runners will cause silent registration failures.
Environment Secrets & Dependency Validation
Install httpx with HTTP/2 support and respx for testing while configuring non-leaking local environment variables.
HTTPX requires the `h2` dependency to support HTTP/2 handshakes. When implementing connection pooling and authentication headers, autonomous agents must pull credentials from OS environment variables rather than hardcoding tokens into the request headers or commit logs.
pip install "httpx[http2]" respx pytest pytest-asyncio && echo 'HTTPX_POOL_MAX_CONNECTIONS=100' >> .env.localimport os
import httpx
limits = httpx.Limits(
max_keepalive_connections=int(os.getenv("HTTPX_POOL_KEEPALIVE", "20")),
max_connections=int(os.getenv("HTTPX_POOL_MAX_CONNECTIONS", "100")),
keepalive_expiry=30.0,
)
timeout = httpx.Timeout(connect=5.0, read=15.0, write=5.0, pool=10.0)
async def get_client() -> httpx.AsyncClient:
return httpx.AsyncClient(limits=limits, timeout=timeout, http2=True)Never pass API tokens or target endpoints directly via agent prompts; reference `.env.local` variables and instruct the agent to use `os.environ.get()`.
Verify `.env.local` is listed inside `.gitignore` before allowing the agent to run `git add` or `git status`.
Intent Matching & Trigger Invocation
Trigger deterministic HTTP client generation using canonical natural language requests.
Provide prompts that explicitly match the YAML frontmatter triggers. When prompting Codex, Claude Code, or Cursor, use declarative phrasing like 'async http request httpx' or 'stream large file httpx'. This immediately activates the loaded client lifecycle and streaming rules.
import httpx
from pathlib import Path
async def stream_large_file(url: str, destination: Path) -> None:
timeout = httpx.Timeout(connect=10.0, read=60.0, write=10.0, pool=5.0)
async with httpx.AsyncClient(timeout=timeout, http2=True) as client:
async with client.stream("GET", url) as response:
response.raise_for_status()
with open(destination, "wb") as buffer:
async for chunk in response.aiter_bytes(chunk_size=8192):
buffer.write(chunk)Instruct the agent to instantiate `httpx.AsyncClient` once as a lifecycle dependency (e.g., FastAPI lifespan or shared singleton) rather than inside single loop iterations.
Avoid unhandled client exceptions by ensuring the agent attaches `response.raise_for_status()` inside the `client.stream` context block.
Verification, Linting & Output Quality Gate
Run pytest with respx mocks and syntax linters to guarantee zero broken network calls.
The skill mandates complete verification using mock adapters. The agent must generate resilient unit tests using `respx` to test timeouts, status codes, and HTTP/2 fallbacks locally without making external network calls that could fail in restricted sandboxes.
pytest tests/ --asyncio-mode=auto -v && ruff check .import pytest
import httpx
import respx
@pytest.mark.asyncio
@respx.mock
async def test_async_fetch():
mock_route = respx.get("https://api.example.internal/items").respond(
status_code=200,
json={"status": "ok", "items": [1, 2, 3]}
)
async with httpx.AsyncClient(http2=True) as client:
response = await client.get("https://api.example.internal/items")
assert response.status_code == 200
assert response.json()["status"] == "ok"
assert mock_route.calledExamine the git diff via `git diff --stat` before confirming the agent pull request to confirm no extraneous debug headers or credentials were added.
If running tests in a sandbox with `net_access: false`, ensure no tests attempt real DNS resolution or live host connections.
Real-World Scenario: High-Throughput Asynchronous Asset Downloader with Chunked Disk Streaming
“Build a resilient download pipeline in Python using httpx to download multiple large dataset archives concurrently. Use async http request httpx triggers, stream large file httpx with 16KB chunking, configure httpx connection pool limits to 50 max connections, and write unit tests using respx.”
Runtime Troubleshooting & Failure Modes
Root Cause: The agent spawned multiple concurrent coroutines without releasing connection context or configured `max_connections` lower than asyncio task batch size, with a short `pool` timeout in `httpx.Timeout`.
limits = httpx.Limits(max_connections=100, max_keepalive_connections=20)
timeout = httpx.Timeout(connect=5.0, read=30.0, write=5.0, pool=30.0)
# Ensure AsyncClient is shared via singleton or context manager rather than recreated per request.Root Cause: The agent yielded or returned a streaming response object outside the scope of the `async with httpx.AsyncClient(...)` context manager block, causing the client session to close before read completion.
# Keep the async client active during consumer iteration:
async with httpx.AsyncClient() as client:
async with client.stream('GET', url) as response:
async for chunk in response.aiter_bytes():
await handle_chunk(chunk)Frequently Asked Questions (HTTPX Modern Asynchronous & Synchronous HTTP Client SOP)
Q:Can I use HTTPX Modern Asynchronous & Synchronous HTTP Client across Codex, Claude Code, and Cursor simultaneously?
Yes. The skill conforms to the universal SKILL.md specification. Placing the file in `.agents/skills/httpx-async-http-client/` allows Codex, Claude Code, Cursor, and Windsurf to detect the triggers and apply the exact same async patterns and limits across any IDE.
Q:How does HTTPX Modern Asynchronous & Synchronous HTTP Client prevent context window overflow during long agent sessions?
The skill provides concise, declarative rules and modular boilerplate for limits, timeouts, and streaming. Instead of generating hundreds of lines of exploratory networking logic and hallucinating options, the agent immediately writes deterministic code using the specified SOP.
Q:Does this skill require escalated root or container permissions?
No. The skill is audited with a security score of 98/100. It requires only standard workspace file read/write permissions (`fs_write: workspace`) to update project files and run unit test commands in user space without requiring root privileges.
Ready to equip your autonomous agent with HTTPX Modern Asynchronous & Synchronous HTTP Client?
Copy the complete audited SKILL.md specification or download the 1-click zip bundle directly from our verified registry.