---
name: "apache-arrow-columnar-exchange"
description: "Share and transfer tabular data between Python, Rust, and C++ without serialization overhead."
metadata:
  author: "Apache"
  version: "1.0.0"
  runtime_compatibility:
    - "codex"
    - "claude-code"
    - "cursor"
    - "windsurf"
triggers:
  - "share memory with apache arrow"
  - "use pyarrow recordbatch"
  - "arrow ipc stream serialization"
permissions:
  fs_read: true
  fs_write: "workspace"
  shell_exec: false
  net_access: false
---

# Apache Arrow Zero-Copy Columnar In-Memory Exchange

Implements Apache Arrow columnar format: Arrow RecordBatches, zero-copy memory sharing across language runtimes, Arrow IPC streaming format, and PyArrow integration with Pandas and Polars.

## When to Invoke
Invoke this skill whenever you need to:
- share memory with apache arrow
- use pyarrow recordbatch
- arrow ipc stream serialization

## Execution SOP & Rules
1. Construct tables with pyarrow.Table.from_pydict(data, schema=schema).
2. Serialize tables with pyarrow.ipc.new_stream(sink, table.schema) for zero-copy network streaming.
3. Convert to Polars or DuckDB with zero memory duplication via Arrow C Data Interface.
4. Enforce explicit nullable flags on schema fields to match downstream consumption contracts.

## Verification Checklist
- Run local linting, syntax validation, and type-checks before reporting completion.
- Ensure all created or modified files adhere to workspace boundaries and avoid uncommitted secrets.
- Verify tests or build commands exit cleanly with return code 0.