Cloud cost
Bytes scanned & partition pruning
On-demand BigQuery bills by how much data a query reads, not how long it runs. Partition pruning and column selection are the two levers that shrink that number.
Last updated
Definition
A query's cost is set by the bytes it scans, full stop — a query that finishes in one second but reads an entire table costs more than one that takes a minute but reads a single day's partition. Two habits inflate that number for no benefit: leaving out a filter on the partitioning column (forcing a scan of every partition instead of one), and selecting every column with SELECT * when only a few are actually used, since BigQuery's columnar storage means unreferenced columns would otherwise cost nothing to skip.
require_partition_filter turns a missing partition filter into a query error instead of a silent full scan, which is the more durable fix of the two.
Where it shows up
Bytes scanned appears in the BigQuery console's query validator before a query even runs, and again in the job details once it completes. The require_partition_filter table setting is visible in the table's own configuration, and turning it on converts a missing partition filter into a query error instead of a silent scan.
What makes it expensive
SELECT * against a partitioned table, with no filter on the partitioning column, is the expensive default. It reads every partition and every column for a query that only ever needed one day and three fields. The console shows bytes scanned before the query runs. Almost nobody looks, and the habit spreads across a team quickly.
Related
More in Cloud cost
- Committed use discount (CUD)
- Reserved instance (RI)
- Savings plan (SP)
- On-demand vs. spot instances
- Data egress
- Amortized, blended, and unblended cost
- Billed vs. effective vs. list vs. contracted cost
- Rightsizing
- Idle and orphaned resources
- Storage class
- Lifecycle policy
- Incremental snapshot
- Allocatable capacity
- Requests vs. limits
- Cluster idle
- Workload idle
- Bin packing
- Cost allocation tags
- Tag coverage
- Unallocated spend
- Showback vs. chargeback
- Unit economics
- RI/SP coverage and utilization
- Observability tax
- Metric cardinality
- High-water-mark billing
- Log ingest vs. index
- Snowflake credit
- Auto-suspend / resume minimum
- BigQuery slot
- Time Travel & Fail-safe
Connect a provider. See one number.
Connect your first provider and CostMon normalizes it alongside everything else you run. Your team gets one number everyone can check.