The Complete Production SOP: How to Deploy and Run Polars LazyFrame Query Optimization in Autonomous AI Agents

---
name: "polars-lazy-query-plan-optimizer"
description: "Optimize and execute analytical query graphs with Polars LazyFrame and scan_* APIs."
metadata:
  author: "Polars"
  version: "1.0.0"
  runtime_compatibility:
    - "codex"
    - "claude-code"
    - "cursor"
    - "windsurf"
triggers:
  - "use polars lazyframe"
  - "inspect polars query plan explain"
  - "scan parquet with polars"
permissions:
  fs_read: true
  fs_write: "workspace"
  shell_exec: false
  net_access: false
---

# Polars LazyFrame Query Optimization

Leverages Polars Lazy API (pl.scan_parquet, pl.scan_csv): predicate pushdown, projection pushdown, common subplan elimination, query graph inspection with explain(), and lazy execution.

## Execution SOP & Rules
1. Start data ingestion with pl.scan_parquet() or pl.scan_csv() rather than eager read_* functions.
2. Apply filters as early as possible in the query chain to maximize predicate pushdown efficiency.
3. Call print(lf.explain()) to audit physical and logical query optimization trees before execution.
4. Execute final query execution with lf.collect(streaming=True) when processing out-of-core datasets.

import polars as pl

assert int(pl.__version__.split('.')[0]) >= 1, f"Expected Polars >= 1.0.0, found {pl.__version__}"
print(f"Polars {pl.__version__} initialized with thread pool size: {pl.thread_pool_size()}")

import ast
import pathlib
import pytest

def test_no_eager_read_calls():
    src = pathlib.Path('pipeline.py').read_text()
    tree = ast.parse(src)
    for node in ast.walk(tree):
        if isinstance(node, ast.Attribute) and node.attr in {'read_csv', 'read_parquet'}:
            pytest.fail(f"Forbidden eager call detected: pl.{node.attr}. Use pl.scan_* instead.")