The Complete Production SOP: How to Deploy and Run Airbyte Connector Builder & Declarative Sync in Autonomous AI Agents
Production SOP to install and run Airbyte Connector Builder & Declarative Sync in Codex, Claude Code, and Cursor. Configure pagination and incremental sync.
Technical Overview & Architectural Context
Autonomous coding agents frequently stumble when generating Airbyte declarative connector manifests (`manifest.yaml`). Without deterministic scaffolding, agents tend to hallucinate low-level Python source connector implementations, misconfigure Jinja interpolation expressions (e.g., confusing `{{ config['api_key'] }}` with `{{ stream_slice }}`), or construct invalid pagination state machines. These errors result in infinite sync loops, truncated record extractions, or silent data drift during incremental loads. The `airbyte-connector-builder-sync` skill enforces strict structural boundaries derived from Airbyte's declarative schema specification. By defining clear procedural checkpoints—authenticator setup, URL path resolution, pagination strategies (OffsetIncrement vs. CursorPagination), stream slicing, and cursor-based datetime checkpoints—the skill guides LLM runtimes to output syntactically valid manifests compatible with the Airbyte Connector Builder UI and CDK CLI without hallucinated abstractions.
Prerequisites & Environment Requirements
Verified Search Intents & Long-Tail Queries Solved
This operational tutorial is engineered to provide deterministic solutions for these developer search queries:
Step-by-Step Execution Playbook
Local Environment Setup & SKILL.md Mounting
Mount the declarative sync skill manifest into your agent's runtime discovery path.
Autonomous agent frameworks like Codex and Claude Code load domain-specific SOPs by inspecting structured markdown manifests in the workspace `.agents/skills` or `.claude/skills` directory. Placing this skill into the target project establishes the execution boundaries, triggers, and permission guardrails before the agent starts generating code.
mkdir -p .agents/skills/airbyte-connector-builder-sync && curl -fsSL https://askills.dev/api/skills/airbyte-connector-builder-sync/raw -o .agents/skills/airbyte-connector-builder-sync/SKILL.md---
name: "airbyte-connector-builder-sync"
description: "Build custom REST API data connectors in minutes using Airbyte Declarative Connector Builder."
metadata:
author: "Airbyte"
version: "1.0.0"
runtime_compatibility:
- "codex"
- "claude-code"
- "cursor"
- "windsurf"
triggers:
- "build airbyte connector"
- "configure airbyte pagination"
- "incremental sync airbyte"
permissions:
fs_read: true
fs_write: "workspace"
shell_exec: false
net_access: false
---When using Cursor or Windsurf, symlink `.agents/skills` directly to `.cursor/rules` or `.windsurfrules` to eliminate token duplication across system context windows.
Ensure `shell_exec` remains `false` in the manifest permissions block. Manifest generation requires only file writes; enabling shell execution unnecessarily exposes your local environment.
Define Declarative Manifest Structure & Authentication
Direct the agent to generate standard Airbyte declarative manifest components.
The skill mandates separating connector configuration into `version`, `definitions`, and `streams`. In this step, the agent scaffolds the primary `manifest.yaml` file, defining the `ApiKeyAuthenticator` or `BearerAuthenticator` alongside parameter declarations, preventing hardcoded credentials.
touch airbyte_project/manifest.yaml && touch airbyte_project/source_test_config.jsonversion: "0.79.0"
definitions:
selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path: ["data"]
requester:
type: HttpRequester
url_base: "https://api.example.com/v1/"
http_method: "GET"
authenticator:
type: ApiKeyAuthenticator
header: "Authorization"
api_token: "Bearer {{ config['api_key'] }}"Supply `source_test_config.json` containing mock keys during local development so the declarative parser can resolve interpolation syntax without exposing production secrets.
Avoid mixing Airbyte Low-Code CDK syntax (`$response.body.data`) with Jinja template strings (`{{ config['api_key'] }}`). Stick to the current CDK 0.79+ standard schema.
Configure Pagination and Incremental Stream Slicing
Implement deterministic pagination state and cursor-based datetime checkpoints.
The agent must configure either `DefaultPaginator` (with `CursorPagination` or `OffsetIncrement`) or `DatetimeStreamSlicer`. The skill guides the model to map the API's next-token pointer correctly into request query parameters or headers.
streams:
- type: DeclarativeStream
name: "customers"
primary_key: "id"
retriever:
type: SimpleRetriever
requester:
$ref: "*ref(definitions.requester)"
path: "customers"
record_selector:
$ref: "*ref(definitions.selector)"
paginator:
type: DefaultPaginator
page_token_option:
type: RequestOption
inject_into: "request_parameter"
field_name: "starting_after"
pagination_strategy:
type: CursorPagination
cursor_value: "{{ last_record['id'] }}"
incremental_sync:
type: DatetimeStreamSlicer
cursor_field: "updated_at"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
cursor_granularity: "PT1S"
start_datetime:
datetime: "{{ config['start_date'] }}"Always set `cursor_granularity: 'PT1S'` for ISO-8601 timestamps to avoid duplicate record extractions at slice boundaries during incremental syncs.
For APIs using cursor tokens located in response headers rather than record payloads, ensure the agent configures `HeaderFilterPath` instead of `last_record` references.
Verification, Linting & Schema Validation Gate
Run CDK declarative validation checks before committing the manifest to version control.
The skill enforces a deterministic quality gate. The autonomous agent executes the Airbyte CDK test command against a local mock response or test account, ensuring that records parse properly and the pagination state advances without regression.
python -m airbyte_cdk.sources.declarative.manifest_declarative_source --spec --config airbyte_project/source_test_config.jsonCheck `git diff airbyte_project/manifest.yaml` before merging to verify that the agent didn't alter previously established stream definitions when adding a new stream.
Exit code 0 from `--spec` checks only schema structure. To verify live paging, run `python -m airbyte_cdk.sources.declarative.manifest_declarative_source --read --config airbyte_project/source_test_config.json`.
Real-World Scenario: Build Stripe Charges Declarative Connector with Incremental Sync
“build airbyte connector for Stripe Charges API. Configure cursor pagination using 'starting_after' and implement incremental sync airbyte on the 'created' timestamp field.”
Runtime Troubleshooting & Failure Modes
Root Cause: The agent injected a pagination strategy without specifying the required `page_token_option` block, or referenced an invalid Jinja interpolation variable.
Ensure `page_token_option` is explicitly declared under `paginator`:
page_token_option:
type: RequestOption
inject_into: "request_parameter"
field_name: "starting_after"Root Cause: The agent generated an outdated CDK v0.30 specification omitting the top-level declarative CDK schema version header.
Add `version: "0.79.0"` as the very first line of `manifest.yaml`.Frequently Asked Questions (Airbyte Connector Builder & Declarative Sync SOP)
Q:Can I use Airbyte Connector Builder & Declarative Sync across Codex, Claude Code, and Cursor simultaneously?
Yes. The skill conforms to the universal SKILL.md specification. You can mount it within `.agents/skills`, `.claude/skills`, or configure Cursor/Windsurf rules to point to the same manifest for cross-runtime consistency.
Q:How does Airbyte Connector Builder & Declarative Sync prevent context window overflow during long agent sessions?
The skill provides compact declarative templates and SOP rules directly in the system scope, stopping the agent from generating verbose custom Python classes (Source, Streams, Authenticator boilerplate) that consume unnecessary tokens.
Q:Does this skill require escalated root or container permissions?
No. The skill is audited with a security score of 98/100. It requires strictly workspace-scoped file system read/write permissions (`fs_write: workspace`) with shell execution disabled by default.
Ready to equip your autonomous agent with Airbyte Connector Builder & Declarative Sync?
Copy the complete audited SKILL.md specification or download the 1-click zip bundle directly from our verified registry.