Skip to content
Back to the work
Public repo

cacheopt

Cost-based query routing across a three-tier cache.

SQLRedisDuckDBQuery rewriting
Open the live app ↗Read the source ↗

Problem

A three-tier cache only helps if queries reach the right tier. Sending everything to the fastest store wastes memory, sending everything to the slowest one wastes time, and most systems choose with a static rule that ignores what a query actually costs.

Approach

cacheopt estimates the cost of each query the way a database planner would, rewrites it when a cheaper shape exists, then routes it to whichever of the in-memory, Redis or DuckDB tiers should serve it.

Outcome

Throughput rose 2.74 times at 10M rows, and p95 latency fell from 126ms to 31ms. p95 rather than mean, because the tail is the number that wakes somebody up.