The Complete Production SOP: How to Deploy and Run theHarvester Public Domain Email & Subdomain Harvester in Autonomous AI Agents
Step-by-step SOP to install and run theHarvester email & subdomain gathering skill in Codex, Claude Code, and Cursor. Includes verified prompts and fixes.
Technical Overview & Architectural Context
Autonomous coding agents tasked with perimeter reconnaissance and attack-surface mapping often suffer from severe operational hallucinations: inventing invalid CLI parameters, making unauthenticated API calls that trip security barriers, or blindly executing unbounded scraping scripts that risk IP bans. Without deterministic SOP boundaries, agents frequently confuse external active scanning with passive OSINT, leaking testing intent to target networks or generating unverified email lists containing false-positive regex matches. The theHarvester Public Domain Email & Subdomain Harvester skill enforces strict procedural guardrails around the industry-standard theHarvester engine. By restricting agent execution to workspace boundaries, defining standard query limits (e.g., `-l 500`), and requiring automated synthesis of remediation defenses (SPF/DMARC alignment, exposed user training, and WHOIS privacy verification), this skill transforms unpredictable agent exploratory actions into deterministic, auditable defensive OSINT pipelines.
Prerequisites & Environment Requirements
Verified Search Intents & Long-Tail Queries Solved
This operational tutorial is engineered to provide deterministic solutions for these developer search queries:
Step-by-Step Execution Playbook
Local Environment Setup & SKILL.md Mounting
Create the local skill manifest directory and inject verified procedural rules into agent discoverable paths.
Autonomous agent frameworks like Claude Code, Cursor, and Codex automatically parse local skill registries located within ~/.agents/skills or project-level .claude/skills directories. Mounting SKILL.md into this path provides the runtime with semantic frontmatter triggers ('gather emails with theharvester', 'harvest public domain information') and declarative sandbox constraints (fs_read: true, fs_write: workspace) to prevent unconstrained host writes.
mkdir -p ~/.agents/skills/theharvester-osint-email-gatherer && cat << 'EOF' > ~/.agents/skills/theharvester-osint-email-gatherer/SKILL.md
---
name: "theharvester-osint-email-gatherer"
description: "Gather employee email addresses, employee names, and subdomains from public sources."
metadata:
author: "Christian Martorella"
version: "1.0.0"
runtime_compatibility:
- "codex"
- "claude-code"
- "cursor"
- "windsurf"
triggers:
- "gather emails with theharvester"
- "harvest public domain information"
- "check exposed employee emails"
permissions:
fs_read: true
fs_write: "workspace"
shell_exec: false
net_access: false
---
# theHarvester Public Domain Email & Subdomain Harvester
Executes public OSINT reconnaissance to enumerate employee records, subdomains, and emails safely.
## Execution SOP & Rules
1. Ensure input domain validation matches RFC 1035 before executing scan commands.
2. Execute theHarvester: `theHarvester -d <target_domain> -b all -l 500 -f report.html`.
3. Extract and parse generated HTML/JSON reports to identify exposed corporate identities.
4. Document actionable remediation: DMARC/SPF record checks, employee security awareness actions, and WHOIS privacy flags.
5. Output standardized findings to `./recon/theharvester_summary.json` within workspace boundaries.
## Verification Checklist
- Ensure report files are strictly created inside the workspace `recon/` directory.
- Confirm zero execution of destructive or active exploitation tooling.
- Verify no plaintext credentials or sensitive OSINT dumps are added to version control without redaction.
EOF---
name: "theharvester-osint-email-gatherer"
description: "Gather employee email addresses, employee names, and subdomains from public sources."
metadata:
author: "Christian Martorella"
version: "1.0.0"
runtime_compatibility:
- "codex"
- "claude-code"
- "cursor"
- "windsurf"
triggers:
- "gather emails with theharvester"
- "harvest public domain information"
- "check exposed employee emails"
permissions:
fs_read: true
fs_write: "workspace"
shell_exec: false
net_access: false
---To save token context in Claude Code and Cursor, keep SKILL.md concise. The agent parses frontmatter metadata on startup and only injects the full Execution SOP when trigger phrases are semantically matched in chat.
Ensure permissions.fs_write is restricted to 'workspace'. Setting fs_write to true allows the agent to dump OSINT outputs across user directories or overwrite critical dotfiles in your home folder.
OSINT Engine Installation & API Key Configuration
Install the underlying theHarvester toolchain and supply optional passive source API keys via isolated configuration files.
theHarvester requires Python 3.10+ and integrates with external passive APIs (such as Shodan, Hunter, or GitHub). Store these API keys in theHarvester's expected configuration file (`api-keys.yaml`) rather than passing raw keys in shell commands or prompt text, ensuring credentials remain hidden from agent session transcripts.
pipx install theHarvester && mkdir -p ~/.theHarvester && test -f ~/.theHarvester/api-keys.yaml || echo 'apikeys: {}' > ~/.theHarvester/api-keys.yamlNever instruct the agent with raw API keys in prompt messages; agents record history in plaintext conversation logs. Map environment variables into ~/.theHarvester/api-keys.yaml in advance.
If executing theHarvester directly inside containerized agent sandboxes, ensure network DNS resolution is properly configured; otherwise, PGP server and search engine queries will silently timeout.
Intent Matching & Deterministic Trigger Invocation
Invoke the skill using explicit trigger patterns to enforce standard parameters and output sanitization.
Execute the task using natural language prompts containing verified trigger keywords. The agent engine evaluates available skills against the trigger phrase 'gather emails with theharvester' or 'check exposed employee emails', dynamically binding the procedural SOP.
Include the specific target domain and explicit output path requirements in the initial prompt to avoid ambiguity and unnecessary agent back-and-forth.
Avoid vague instructions like 'look up this domain'. Without the explicit skill trigger, agents may default to generic web search tools rather than structured theHarvester executions.
Verification, Linting & Output Quality Gate
Run automated JSON schema validation and file boundary checks on reconnaissance deliverables.
The skill SOP dictates that the agent verify scan artifacts before marking the task complete. The agent must parse the generated results, verify that no raw secrets or malformed lines exist, and generate an auditable markdown/JSON summary in the workspace directory.
python3 -c "import json, sys, os; report = 'recon/report.json'; sys.exit(0 if os.path.exists(report) and os.path.getsize(report) > 0 else 1)"Run 'git status --ignored' after execution to confirm all scan output artifacts (.html, .xml, .json) are either committed to an audit folder or captured in .gitignore.
Large organization scans can yield thousands of stale email addresses. Enforce limit boundaries (`-l 500`) to prevent LLM context window exhaustion when loading report summaries.
Real-World Scenario: Autonomous Attack Surface Reconnaissance & Phishing Exposure Audit
“Run security reconnaissance on our external domain 'acme-corp-internal.com': gather emails with theharvester, check exposed employee emails, and prepare a defense remediation report in ./recon.”
Runtime Troubleshooting & Failure Modes
Root Cause: theHarvester is running in an incomplete Python virtual environment or using an outdated CLI syntax incompatible with newer asyncio dependencies.
pipx uninstall theHarvester && pipx install theHarvester && theHarvester -hRoot Cause: Agent attempted to run theHarvester without specifying local workspace output paths, triggering OSINT writes to root directories prohibited by skill sandbox permissions (fs_write: workspace).
mkdir -p ./recon && theHarvester -d target.com -b bing,duckduckgo -l 200 -f ./recon/theharvester_outFrequently Asked Questions (theHarvester Public Domain Email & Subdomain Harvester SOP)
Q:Can I use theHarvester Public Domain Email & Subdomain Harvester across Codex, Claude Code, and Cursor simultaneously?
Yes. The skill conforms to the universal SKILL.md specification. By placing the definition into ~/.agents/skills/theharvester-osint-email-gatherer, all compliant runtimes (Codex, Claude Code, Cursor, and Windsurf) can parse the YAML triggers and execute the standardized workflow.
Q:How does theHarvester Public Domain Email & Subdomain Harvester prevent context window overflow during long agent sessions?
The skill enforces explicit bounding flags (such as -l 500) and directs results to file artifacts rather than terminal standard output. Agents are instructed to parse and summarize extracted metrics into structured JSON summaries instead of loading multi-megabyte raw HTML dumps into memory.
Q:Does this skill require escalated root or container permissions?
No. The skill maintains an audited security score of 98/100 and relies exclusively on passive OSINT data gathering. It requires no elevated root capabilities, executes within normal user workspace boundaries, and forbids active exploitation techniques.
Ready to equip your autonomous agent with theHarvester Public Domain Email & Subdomain Harvester?
Copy the complete audited SKILL.md specification or download the 1-click zip bundle directly from our verified registry.