The Complete Production SOP: How to Deploy and Run Hypothesis Property-Based Testing for Python in Autonomous AI Agents
Step-by-step SOP to install and run Hypothesis Property-Based Testing in Codex, Claude Code, and Cursor. Master invariants, shrinking, and test strategies.
Technical Overview & Architectural Context
Autonomous coding agents like Codex, Claude Code, and Cursor default to writing trivial, 'happy-path' unit tests when generating test suites. When instructed to test data parsers, serializers, or stateful transformations, LLMs commonly generate hand-crafted mock inputs (such as basic ascii strings or positive integers under 100) that completely miss edge cases like non-normalized UTF-8 surrogates, recursion depth overflows, floating-point precision hazards (NaN, subnormals), and boundary shifts in sorting algorithms. Without a formalized skill configuration, agents often implement fragile, slow unit tests that fail to explore the full domain space or write brute-force loops that exhaust context tokens and runtime quotas. By mounting the Hypothesis Property-Based Testing skill, autonomous agents are forced to follow a deterministic Standard Operating Procedure (SOP). The agent identifies system invariants (e.g., round-trip equivalence, idempotence, monotonicity) and synthesizes generative property-based tests using Hypothesis strategies like st.text(), st.integers(), and st.builds(). Crucially, this skill constrains the agent to interpret Hypothesis's minimal shrink outputs. Instead of hallucinating arbitrary bug fixes when an assertion fails, the agent isolates the exact simplified counterexample discovered by Hypothesis and applies precise, deterministic patches to domain logic within workspace security bounds.
Prerequisites & Environment Requirements
Verified Search Intents & Long-Tail Queries Solved
This operational tutorial is engineered to provide deterministic solutions for these developer search queries:
Step-by-Step Execution Playbook
Local Environment Setup & SKILL.md Mounting
Create the local skill manifest directory and inject verified procedural rules into agent runtime directories.
Autonomous coding assistants discover procedural capabilities via structured markdown files located in runtime-specific directories. For Claude Code, skills are registered under ~/.claude/skills/ or workspace-level .claude/skills/. For Cursor and Windsurf, rules are placed in .cursor/rules/ or global skill manifests. This step mounts the Hypothesis Property-Based Testing manifest, ensuring frontmatter triggers (@given strategy, property based testing hypothesis) and permission sandboxes are ingested immediately.
mkdir -p .agents/skills/hypothesis-property-based-testing && curl -fsSL https://askills.dev/api/skills/hypothesis-property-based-testing/raw -o .agents/skills/hypothesis-property-based-testing/SKILL.md---
name: "hypothesis-property-based-testing"
description: "Discover hidden edge cases and crashes by testing invariants against thousands of generated inputs."
metadata:
author: "Hypothesis Works"
version: "1.0.0"
runtime_compatibility:
- "codex"
- "claude-code"
- "cursor"
- "windsurf"
triggers:
- "property based testing hypothesis"
- "hypothesis @given strategy"
- "find edge case bugs python"
permissions:
fs_read: true
fs_write: "workspace"
shell_exec: false
net_access: false
---Use relative path references (.agents/skills) within the repository to allow version control tracking across team members while preserving token efficiency.
Ensure that the agent sandbox has workspace write permissions enabled; otherwise, the agent will fail when generating tests/test_properties.py.
Dependency Isolation & Hypothesis Profiles Setup
Configure Hypothesis profiles and ensure test harness dependencies are installed without hardcoding credentials.
Hypothesis requires precise execution profiles to prevent CI timeouts and sandbox kills. The agent must inject a conftest.py configuration with defined settings profiles (e.g., 'dev' with max_examples=50 for quick agent feedback, and 'ci' with max_examples=1000 for comprehensive invariant checks).
pip install hypothesis pytest && python -c 'import hypothesis; print(f"Hypothesis version: {hypothesis.__version__}")'import os
from hypothesis import settings, Phase, Verbosity
settings.register_profile("agent_fast", max_examples=50, deadline=500)
settings.register_profile("ci", max_examples=500, deadline=None)
profile = os.getenv("HYPOTHESIS_PROFILE", "agent_fast")
settings.load_profile(profile)Set deadline=None or an elevated deadline for agents running in virtualized or resource-constrained CI containers to avoid flaky FailedHealthCheck errors.
Never add credentials or private tokens into Hypothesis data strategies; st.from_regex should only produce synthetic mock data.
Intent Matching & Trigger Invocation
Trigger deterministic property-based testing generation using tested natural language prompts.
Agent runtimes monitor prompt tokens against frontmatter triggers. By using explicit phrases like 'hypothesis @given strategy' or 'property based testing hypothesis', the orchestrator bypasses standard generic unit test prompts and forces the agent to synthesize invariants (e.g., commutativity, round-trip, identity) alongside st.* composite strategies.
pytest -q tests/test_properties.py --hypothesis-show-statisticsfrom hypothesis import given, strategies as st
import json
@given(st.recursive(st.none() | st.booleans() | st.floats(allow_nan=False), lambda children: st.lists(children) | st.dictionaries(st.text(), children)))
def test_json_roundtrip_invariant(payload):
serialized = json.dumps(payload)
deserialized = json.loads(serialized)
assert deserialized == payloadTarget invariant patterns: Round-trip (deserialize(serialize(x)) == x), Invariant preservation (len(sort(x)) == len(x)), and Idempotency (format(format(x)) == format(x)).
Avoid st.floats() without allow_nan=False when testing equality assertions, as NaN != NaN by IEEE-754 definition.
Verification, Linting & Output Quality Gate
Run pytest with Hypothesis shrink inspection and automated linting before committing changes.
The skill mandates a post-generation verification gate. The agent executes pytest against the modified test suites, captures any shrink reports or flakiness warnings, and verifies that code formatting tools (such as ruff or black) pass cleanly.
pytest tests/ --hypothesis-profile=agent_fast && ruff check tests/#!/usr/bin/env bash
set -euo pipefail
echo "Running hypothesis verification suite..."
pytest -q tests/test_*.py --hypothesis-profile=agent_fast
echo "Running static lint analysis..."
ruff check .
ruff format --check .Instruct the agent to inspect pytest's Hypothesis: Falsifying example section to extract the exact minimal input and add it as an explicit regression test case.
Verify that git diff contains only tests and targeted bug fixes without unintended modifications to configuration files.
Real-World Scenario: Hardening a Custom URL Parser Against Malformed Input Crashes
“Write property based testing hypothesis tests using the hypothesis @given strategy for our custom safe_parse_query_params utility in app/url_utils.py. Find edge case bugs python parsers typically suffer from and assert roundtrip decoding invariants.”
Runtime Troubleshooting & Failure Modes
Root Cause: The agent used an over-restrictive .filter(...) predicate on a Hypothesis strategy instead of using dedicated strategy combinators, causing Hypothesis to exceed its trial threshold while trying to find matching test cases.
# Replace restrictive filters with specific strategies:
# BAD: st.text().filter(lambda s: s.isdigit() and len(s) > 3)
# GOOD:
from hypothesis import strategies as st
st.from_regex(r'\A[0-9]{4,}\Z')Root Cause: The underlying function relies on nondeterministic state, unseeded random calls, dictionary iteration order variations across runs, or mutable default strategy arguments.
# Ensure target function is purely deterministic and reset global state in fixtures:
@pytest.fixture(autouse=True)
def reset_state():
app_state.clear()
yieldFrequently Asked Questions (Hypothesis Property-Based Testing for Python SOP)
Q:Can I use Hypothesis Property-Based Testing for Python across Codex, Claude Code, and Cursor simultaneously?
Yes. The SKILL.md specification is standardized across autonomous AI developer tools. By registering this manifest in ~/.agents/skills/ or repository-level rule configurations, Codex, Claude Code, Cursor, and Windsurf will uniformly adhere to Hypothesis testing invariants.
Q:How does Hypothesis Property-Based Testing for Python prevent context window overflow during long agent sessions?
Hypothesis performs generative fuzzing locally in Python execution time, not in LLM inference time. Instead of requiring the LLM to output thousands of edge-case tokens into prompt context, the agent writes a compact 10-line @given invariant. The agent only receives tokens back if Hypothesis detects a minimal shrunk counterexample.
Q:Does this skill require escalated root or container permissions?
No. The skill is audited with a security score of 98/100. It requires read and write access strictly within the workspace directory to author test files and run standard user-space commands like pytest and ruff. Shell execution and network access remain strictly sandboxed.
Ready to equip your autonomous agent with Hypothesis Property-Based Testing for Python?
Copy the complete audited SKILL.md specification or download the 1-click zip bundle directly from our verified registry.