---
name: "colly-golang-web-scraper"
description: "Build blazing-fast, concurrent web scrapers in Go with Colly collector framework."
metadata:
  author: "gocolly"
  version: "1.0.0"
  runtime_compatibility:
    - "codex"
    - "claude-code"
    - "cursor"
    - "windsurf"
triggers:
  - "scrape website with colly go"
  - "write colly onhtml callback"
  - "rate limit colly collector"
permissions:
  fs_read: true
  fs_write: "workspace"
  shell_exec: false
  net_access: false
---

# Colly Golang Fast Distributed Web Scraper

Develops high-throughput scrapers in Go using Colly: Collector initialization, HTML callback hooks (OnHTML, OnResponse), asynchronous worker threads, Redis storage backend, and domain rate limiting.

## When to Invoke
Invoke this skill whenever you need to:
- scrape website with colly go
- write colly onhtml callback
- rate limit colly collector

## Execution SOP & Rules
1. Initialize collectors with c := colly.NewCollector(colly.AllowedDomains("example.com")).
2. Attach parsing callbacks using c.OnHTML("a[href]", func(e *colly.HTMLElement) { ... }).
3. Configure rate limits using c.Limit(&colly.LimitRule{ DomainGlob: "*", Delay: 1 * time.Second }).
4. Always call c.Wait() after asynchronous visits when running with c.Async = true.

## Verification Checklist
- Run local linting, syntax validation, and type-checks before reporting completion.
- Ensure all created or modified files adhere to workspace boundaries and avoid uncommitted secrets.
- Verify tests or build commands exit cleanly with return code 0.