---
name: "vaex-outofcore-dataframe"
description: "Visualize and compute statistics on 1 billion rows without loading into RAM using Vaex."
metadata:
  author: "Vaex"
  version: "1.0.0"
  runtime_compatibility:
    - "codex"
    - "claude-code"
    - "cursor"
    - "windsurf"
triggers:
  - "explore billion row dataset"
  - "use vaex out of core"
  - "calculate histogram vaex"
permissions:
  fs_read: true
  fs_write: "workspace"
  shell_exec: false
  net_access: false
---

# Vaex Out-of-Core Billion Row Data Exploration

Analyzes massive tabular datasets using Vaex: memory-mapped HDF5/Arrow/Parquet files, lazy evaluations, zero-copy virtual columns, and sub-second statistical histograms on billion-row tables.

## When to Invoke
Invoke this skill whenever you need to:
- explore billion row dataset
- use vaex out of core
- calculate histogram vaex

## Execution SOP & Rules
1. Open memory-mapped files using df = vaex.open('massive_data.hdf5') or vaex.open('*.parquet').
2. Create virtual columns cleanly with df['speed'] = df['distance'] / df['time'].
3. Compute aggregations lazily: df.mean(df.amount, selection='state == "CA"').
4. Export filtered subsets directly to disk with df.export('clean.parquet').

## Verification Checklist
- Run local linting, syntax validation, and type-checks before reporting completion.
- Ensure all created or modified files adhere to workspace boundaries and avoid uncommitted secrets.
- Verify tests or build commands exit cleanly with return code 0.