Databricks Cost Optimization on AWS: 8 Ways to Cut Costs

Your Databricks bill went up last month, and now someone wants to know why. Was it more data? A cluster somebody forgot to turn off? That new pipeline? And once you know, how do you bring the number back down without breaking the jobs that feed tomorrow morning’s dashboards?

Databricks on AWS makes this extra fun, because you’re really paying two bills. Databricks charges for DBUs (Databricks Units), and on classic compute AWS separately charges for the EC2 instances, EBS volumes and network traffic underneath. On serverless compute, the servers are already included in Databricks’ price.

If you want a refresher on how that pricing works, our Databricks pricing guide covers the billing model. This article is for AWS teams that already run Databricks: the data platform engineers who own the jobs, and the FinOps or finance people who get asked about the bill.

In this article, we’ll start with eight ways to reduce Databricks costs on AWS. Then we’ll walk one nightly job through them: measure it, pick the right tip, change one thing and prove it worked.

One piece of advice if you’re tempted to skip straight to the tips: measure before you cut. Step 1 shows you how.

8 Databricks Cost Optimization Tips for AWS

These are the eight cost optimization levers we’d check first. Each tip below starts with a signal: something you can spot in your billing data, cluster settings or metrics that points to that tip. Use this table to find the one that matches what you’re seeing.

Tip	The signal that points to it	What it changes	Main risk
1. Tag workloads	Spend you can’t assign to a team	Nothing on its own; it makes every other change measurable	New labels don’t cover past usage
2. Move scheduled work off all-purpose compute	Scheduled jobs on an all-purpose cluster	Price per DBU, plus idle time	A library or setting the old cluster had is missing
3. Set auto-termination	Interactive clusters running with no commands	Databricks usage and AWS server hours	People wait longer for a restart
4. Right-size from bottlenecks	Low CPU or memory use, or heavy spill to disk	Databricks usage and AWS server hours	Longer runtimes or out-of-memory failures
5. Use Spot instances	Batch jobs with room in their deadline	AWS server price	Interruptions make runs longer
6. Test Photon	Long SQL or DataFrame jobs where nobody has measured Photon	Usage per hour (up) and runtime (maybe down)	A higher bill if the job doesn’t speed up enough
7. Try serverless	Short, bursty or frequent workloads, or long startup waits	Moves the server cost into the Databricks price	Unsupported tasks, and the cost leaves your AWS bill
8. Use Committed Use Contracts	Steady usage you expect to keep	Price per DBU	Paying for a commitment you don’t use

Whichever tip you pick, change one thing at a time so you know which change did what.

1. Tag Workloads So Every Cost Has an Owner

If you have spend you can’t assign to a team, start here, because every other tip needs an owner who can approve a change. Databricks already adds default tags to classic clusters (Vendor, ClusterId, ClusterName and Creator), and job clusters also get JobId and RunName. Add your own custom tags for team, project and environment to every cluster, job cluster, SQL warehouse and pool. On classic compute, Databricks passes these labels to both the custom_tags column of its billing table, system.billing.usage, and the AWS resources it launches, such as EC2 instances and EBS volumes. To see them in the CUR or Cost Explorer, activate them as cost allocation tags in AWS Billing. AWS says new keys can take up to 24 hours to appear, then up to 24 hours to activate.

There are two gotchas. Clusters created from a pool pass only the workspace’s and the pool’s labels to EC2, not their own cluster labels, so put the team label on the pool too. And serverless compute runs in a Databricks-managed account, so there are no EC2 instances in your account to carry them. For serverless, use serverless usage policies (in Public Preview), which add your labels to serverless usage in the billing table.

Labels only cover usage from the moment you add them. Databricks’ cost best practices are blunt about it: “missing tags can’t be added to past events.” AWS’s cost allocation tag backfill doesn’t rescue you either, because it only works for tags that were already on the resource back then. Test it with the loop in Step 4.

2. Move Scheduled Work Off All-Purpose Compute

If scheduled jobs run on an all-purpose cluster, this tip is for you. All-purpose compute is built for interactive work in notebooks, and it carries a higher list price than jobs compute for the same usage. Databricks’ own cost best practices say non-interactive workloads “will cost significantly less” on job clusters, and the jobs docs don’t recommend all-purpose compute for production jobs. You’ll find the rate difference in our rates by compute type section.

The change itself lives in the job’s Compute section: point the task at a new job cluster instead of the existing all-purpose one (the Swap button changes it for every task that uses that compute). A new job cluster starts when the run begins and terminates when the run ends, so any idle time after the run disappears too. Serverless compute for jobs is the other option, and it’s tip 7.

Check two things before you switch. Libraries someone installed on the shared cluster don’t come along, so add them to the task’s Dependent libraries setting (or to the job’s environment spec, if the compute uses environments). And if several tasks share one job cluster, it stays on (and idle) while tasks that don’t use it are running. Test it with the loop in Step 4.

3. Set Auto-Termination on Interactive Clusters

If interactive clusters are running with no commands, they’re still billing you. Databricks says idle compute “continue to accumulate DBU and cloud instance charges” until it terminates, so you pay both Databricks and AWS for the wait.

The setting is Terminate after … minutes of inactivity, under the compute form’s performance settings. It accepts 10 to 10,000 minutes, and a cluster counts as inactive once every command, Spark job, JDBC call and web-terminal activity has finished. Watch out for clusters created through the Clusters API: if autotermination_minutes isn’t set, the cluster is never terminated automatically. Databricks’ best practices use one hour as an example of “a reasonable value” in a compute policy, but the right number depends on how long your users will put up with a restart.

Pools are the sneaky exception. Databricks doesn’t bill for idle instances sitting in a pool, but AWS still does, so check the pool’s own idle-instance setting as well. Test it with the loop in Step 4.

4. Right-Size From Bottlenecks, Not DBU Totals

If a cluster shows low CPU or memory use, or heavy spill to disk, its size is worth a look. The catch: the billing table tells you how much a cluster used, not how busy it was. For that, Databricks has another system table, system.compute.node_timeline, with minute-by-minute records of each node’s cpu_user_percent, mem_used_percent and network traffic. It covers classic all-purpose and job clusters, not serverless compute or SQL warehouses, and nodes that ran for under 10 minutes might not show up.

What you’re looking for decides the change. Worker nodes that sit at low CPU and memory for most of a run suggest fewer or smaller nodes. Significant spill to disk or out-of-memory errors point the other way, and Databricks’ sizing guidance says to add memory per instance. For complex jobs with lots of joins, it recommends fewer workers on larger instances, to cut down on shuffling. The settings you’ll change are the Worker type and the number of workers (or the autoscaling range). Autoscaling is worth a look on its own, too: Databricks says it “can reduce overall costs compared to a statically sized compute instance.” For instance families and Graviton, see the AWS section of our pricing guide.

Smaller isn’t automatically cheaper. A cluster with half the nodes that runs twice as long costs about the same, and one that starts failing costs more. Test it with the loop in Step 4.

5. Use Spot Instances Where Interruptions Are Acceptable

If a batch job has room in its deadline, its workers are candidates for EC2 Spot instances: spare AWS capacity at a lower price that AWS can take back. Databricks recommends Spot for workloads “that have lax latency requirements”, and says the first instance, the Spark driver, should always be on-demand. Our post on when to use Spot instances covers the AWS side in more depth.

In the compute form, it’s the Use spot instance checkbox under Advanced performance. With it checked, the driver stays on-demand and the other nodes use Spot. If you create clusters through the API, look at aws_attributes: availability takes SPOT, ON_DEMAND or SPOT_WITH_FALLBACK, and first_on_demand sets how many nodes stay on-demand. The API reference lists its default as 0, and any value above 0 puts the driver on an on-demand instance.

Keep two limits in mind. Spot only changes the EC2 part of the bill; the Databricks charges are the same. And it’s a classic-compute setting, so there’s no Spot option for serverless. Test it with the loop in Step 4.

6. Test Photon on Supported Work

If you have long SQL or DataFrame jobs and nobody has measured Photon on them, this one’s interesting. It’s Databricks’ vectorized query engine. On classic compute, it doesn’t change the price per DBU, but Photon compute uses more DBUs per hour: the Databricks pricing pages list a “2.9X” DBU emission rate for jobs compute and “2X” for all-purpose compute (October 2026). So the engine only saves money if the job finishes fast enough to make up for that.

Now for the more interesting bit. Databricks turns Photon on by default for classic all-purpose and job clusters created in the UI, through the Use Photon Acceleration checkbox. (Through the Clusters or Jobs API, you have to set runtime_engine to PHOTON yourself.) That means you may already be paying Photon’s higher DBU rate on jobs that don’t benefit from it. Photon doesn’t support UDFs, RDD APIs or Dataset APIs, it doesn’t improve queries that normally run in under two seconds, and it needs a supported instance type. So the test can go either way: turn it on for a job that’s been running without it, or off for one that never sped up. Our pricing guide’s cost factors section explains how the runtime engine affects the bill. Test it with the loop in Step 4.

7. Try Serverless Where It Fits

If a workload is short, bursty or frequent, or people keep waiting on cluster startup, serverless is worth a test. With serverless, the servers run in Databricks’ account and their cost is included in the Databricks price, so there’s no EC2 line to add. Serverless compute is also the default for job tasks that support it, and serverless jobs get autoscaling automatically. For serverless jobs, the setting to know is Performance optimized: with it turned off, the job uses standard mode, which consumes fewer DBUs and accepts a startup latency of 4 to 6 minutes.

Serverless SQL warehouses aren’t free when idle, though. They keep charging until Auto Stop kicks in (10 minutes by default). Not every task runs on serverless either: Spark Submit tasks need classic compute, and continuous scheduling on serverless only works with bounded streaming triggers such as Trigger.AvailableNow. Check the serverless limitations first.

One side effect catches teams out. Serverless spend no longer shows up as EC2 in your AWS bill, so a report built on the CUR’s EC2 lines loses it. You’ll find it in system.billing.usage instead, up to 24 hours after the work ran. Our serverless pricing section explains what the serverless price includes. Test it with the loop in Step 4.

8. Use Committed Use Contracts Once Usage Is Steady

If your Databricks usage is steady and you expect to keep it, committing to a level of usage can lower your Databricks rate. Databricks calls these commitments Committed Use Contracts (you’ll also see them called pre-purchased DBUs), and its pricing page says “the larger your usage commitments, the greater your benefits.” There are no published discount bands, so this is a conversation with your Databricks account team.

Don’t expect AWS to help with the Databricks side. Savings Plans apply to Amazon EC2, AWS Fargate, AWS Lambda and Amazon SageMaker AI usage, so they can lower the EC2 part of classic compute, but Databricks’ own charges aren’t on that list. Our pricing guide’s billing model section covers pay-as-you-go versus committed use.

Do this one last. If you commit before working through tips 2 to 7, you lock in a discount on usage you were about to remove. Test it with the loop in Step 4.

Putting the Tips to Work: Cutting the Cost of One Nightly Job

With the tips under your belt, let’s see them in action on one job. Meet Team Pelican. Last month, Pelican added a nightly ETL job and pointed it at the shared all-purpose cluster their analysts use during the day, because it already had the right libraries installed. Then the FinOps lead, who reviews the bill each month, saw the workspace’s all-purpose spend jump and asked Pelican what happened.

Team Pelican is made up, and so are its usage figures. The prices are real, dated list prices, but this is an illustration, not a customer result. Here are our assumptions:

• Plan and region: Databricks Premium on AWS, US East (N. Virginia), pay-as-you-go list prices in USD, checked October 21, 2026.
• Job: one nightly notebook task that loads Delta Lake tables in S3, 30 nights a month, due by 06:00.
• Cluster: the shared all-purpose cluster, 3 × m5.xlarge (1 driver and 2 workers), all on-demand, fixed size (no autoscaling). Photon is off (Databricks’ pricing calculator doesn’t list m5.xlarge as a supported instance for it).
• Runtime: 2 hours of billed cluster time per run, startup included, from 01:00 UTC.
• Idle tail: the cluster’s auto-termination is 45 minutes, and nothing else runs on it at night, so it sits idle for 45 minutes after each run.
• DBU rate: 0.690 DBUs per hour per m5.xlarge node, the same for all-purpose and jobs compute (Databricks pricing calculator).
• Databricks price: $0.55 per DBU for all-purpose compute and $0.15 per DBU for jobs compute (Databricks product pricing pages).
• AWS price: $0.192 per hour for an on-demand Linux m5.xlarge in us-east-1 (AWS Price List).
• Left out: EBS volumes, data transfer, S3 and other AWS charges. In a real check, they go in the AWS column too.

That makes one node-hour cost 0.690 × $0.55 + $0.192 = $0.5715 on all-purpose compute, and 0.690 × $0.15 + $0.192 = $0.2955 on jobs compute.

Step 1: Measure the Full Cost of the Job

First things first: what does the job cost today, including the AWS half? The comparison only works if both sides use the same rules, so write them down:

• Comparable cost = Databricks cost on your chosen price basis + AWS cost that isn’t already included in it.
• Same window on both sides, in UTC, with matching nights or weekdays.
• Same price basis: list price on both sides, or contract price on both sides, never one of each.

On the Databricks side, usage lives in the system tables: system.billing.usage has the DBUs, and system.billing.list_prices holds the list price history you can join to them. On the AWS side, use the CUR (hourly granularity helps here) or Cost Explorer, filtered by the cluster’s ClusterId tag. Then decide what goes inside the boundary:

Cost	Classic compute	Serverless compute	Where you see it
DBUs	Yes	Yes, and the price includes the servers	system.billing.usage, the Databricks invoice
EC2 instances	Yes, billed by AWS	Included in the Databricks price, so don’t add them	CUR or Cost Explorer, by ClusterId or team label
EBS volumes	Yes, billed by AWS	None in your AWS account	CUR
Networking	Billed by AWS	Some data transfer is billed by Databricks; in-region S3 access isn’t	CUR, or system.billing.usage for serverless
S3 storage	Billed by AWS for your own buckets	Same; Databricks default storage is billed by Databricks	CUR, or system.billing.usage
Databricks paid through AWS Marketplace	The Databricks charges also show up in the CUR, with a billing entity of “AWS Marketplace”	Same	CUR bill/BillingEntity; count these once, not twice

That last row is easy to miss. If you pay Databricks through AWS Marketplace, adding “Databricks billing table” to “everything in the CUR” counts that usage twice. Shared costs, like a NAT gateway or an S3 bucket every team uses, go in an unallocated bucket rather than onto one job.

Here’s Pelican’s job last month, at list prices:

• Node-hours per night: 3 × 2.75 = 8.25
• Node-hours for the month: 8.25 × 30 = 247.5
• DBUs: 247.5 × 0.690 = 170.775
• Databricks (all-purpose): 170.775 × $0.55 = $93.93
• AWS EC2: 247.5 × $0.192 = $47.52
• Total for the month: $93.93 + $47.52 = $141.45, or about $4.71 per run

Hold onto this $141.45 number.

Step 2: Find What Changed and Who Owns It

Now for the detective work. Narrow the increase down one level at a time: SKU (sku_name or billing_origin_product), then workspace (workspace_id), then cluster (usage_metadata.cluster_id), then owner and labels (custom_tags), then job.

Billing data has blind spots, and it helps to know them before you start:

• No job ID on all-purpose compute. usage_metadata.job_id is filled in for runs on job clusters and serverless, but not for jobs run on all-purpose compute.
• The identity isn’t always the owner. For all-purpose compute, identity_metadata.run_as is whoever created the cluster. The docs point you to owned_by in system.compute.clusters for the owner.
• Corrections. When Databricks corrects a record, it adds a RETRACTION row that cancels the original and, usually, a RESTATEMENT row with the right numbers. Sum usage_quantity across all three record_type values instead of keeping only ORIGINAL.
• Delays. Original records typically arrive within 12 hours, corrections don’t follow that timing, and serverless usage can take up to 24 hours. Don’t compare a day that isn’t finished yet.
• No utilization data. Billing records say how much you used, not how busy the nodes were. That’s system.compute.node_timeline (tip 4).

If there’s no owner label, find the owner through the cluster’s owned_by or the job’s owner, then apply tip 1 so you don’t have to do this again.

Pelican’s drill-down went like this. The increase was all ALL_PURPOSE usage, and almost all of it came from one cluster_id: the shared analytics cluster. That cluster had new usage records between 01:00 and 04:00 UTC every night, which weren’t there the month before. The records carry no job_id, because it’s an all-purpose cluster, but the job’s run history shows Pelican’s job running from 01:00 to 03:00, and the cluster’s event log shows it terminating at 03:45.

Since two groups share the cluster, we need a rule for splitting its cost, and it should be written down. Ours is a time-window split: from the moment the nightly job starts the cluster until the cluster terminates afterwards, the cost belongs to Pelican, because nothing else runs on it at night (Pelican checked with the analysts). The analysts’ daytime hours stay with the analysts. If a notebook had been running during the night window, those hours would go into a shared bucket instead of being guessed at.

Step 3: Pick the Tip the Evidence Points To

Back to the quick picker. Pelican’s signal is “scheduled jobs on an all-purpose cluster”, which is tip 2. Moving the job to its own job cluster also gets rid of the 45-minute idle tail, because a job cluster ends with the run. So Pelican doesn’t need tip 3. (The shared cluster’s daytime idle time belongs to the analysts, and so does that decision.)

Why not try a few tips at once and save a month? Because if the cost drops and a run fails, you won’t know which change did which. Change one thing at a time, and start with the one that has an owner and a way back. Pelican owns the job, and the way back is simple: swap the task back to the shared cluster.

Step 4: Change One Thing and Test It

Before touching anything, Pelican wrote a short test plan:

• What changes: the task’s compute, from the shared all-purpose cluster to a new job cluster with the same instance type, node count, Databricks Runtime version and settings. The job cluster gets Pelican’s team label (tip 1), so its cost is easy to find. A label doesn’t change cost, so it doesn’t count as a second change.
• What stays the same: the code, the input data, the schedule and the hardware.
• Checks first: notebook tasks are supported on job clusters; libraries installed on the shared cluster move to the task’s Dependent libraries; the job cluster gets the same access to data (for example, the same instance profile or Unity Catalog permissions).
• Success: the output matches (row counts and checksums on the target Delta Lake tables), every run finishes before 06:00, and the cost per successful run goes down.
• Stop: two failed runs in a row, or one missed deadline, means switching back.
• Rollback: a saved copy of the job’s current configuration, plus the Swap button.
• Who approves: Pelican’s lead, with a heads-up to the analysts.
• Window: a full month, so it lines up with last month’s 30 nights.

Step 5: Verify the Saving

Once the month is over, wait for billing to settle (at least a day for original records), then pull the numbers again a few days later in case corrections arrive. Then compare with two formulas:

• Comparable cost = Databricks cost on the declared basis + AWS cost not already included
• Cost per successful run = (all in-scope cost, including failed attempts) ÷ successful runs

Failed attempts count in the cost but not in the runs, and that matters here. On night 2, the job failed after 30 minutes because of a library someone had installed by hand on the shared cluster months ago, which nobody had listed. Pelican added it to the task and the rerun worked, so the month ended with 30 successful runs and one failed attempt.

Here are Pelican’s numbers for the new month:

• Node-hours for successful runs: 3 × 2 × 30 = 180
• Node-hours for the failed attempt: 3 × 0.5 = 1.5
• Node-hours in total: 180 + 1.5 = 181.5
• DBUs: 181.5 × 0.690 = 125.235
• Databricks (jobs compute): 125.235 × $0.15 = $18.79
• AWS EC2: 181.5 × $0.192 = $34.85
• Total for the month: $53.63, or about $1.79 per successful run

(We round each line to the cent. Totals use the unrounded figures, so line items can be a cent off. The saving below is worked from the totals as shown.)

	Before: shared all-purpose cluster	After: job cluster
Window	Last month, 30 nights	This month, 30 nights
Successful runs	30	30
Failed attempts	0	1 (30 minutes)
Node-hours	247.5	181.5
DBUs	170.775	125.235
Price basis	List, $0.55 per DBU	List, $0.15 per DBU
Databricks cost	$93.93	$18.79
AWS EC2 cost	$47.52	$34.85
Combined cost	$141.45	$53.63
Cost per successful run	$4.71	$1.79
Billed time per run	2 h plus a 45-minute idle tail	2 h
Deadline met	30 of 30	30 of 30
Output matches	Baseline	Yes
Decision		Keep

At list prices, that’s $141.45 − $53.63 = $87.82 less for the month. Where did it come from? The idle tail was 67.5 node-hours, worth 67.5 × $0.5715 = $38.58. The cheaper Databricks rate on the 180 working node-hours saved 180 × $0.5715 − 180 × $0.2955 = $49.68. The failed attempt added 1.5 × $0.2955 = $0.44 back. And $38.58 + $49.68 − $0.44 = $87.82. Whew!

Three caveats before Pelican celebrates. First, these are list prices. If you have a contract, the invoice your FinOps or finance team sees reflects contract prices instead. With a committed-use discount, the Databricks dollars shrink, but the DBU counts and node-hours don’t, so compare those first. Second, check that the shared cluster’s total all-purpose cost really dropped by the night window. If it didn’t, something else is using that cluster. Third, if output quality or the deadline had slipped, the right call would be to revert, even with a lower bill.

Step 6: Repeat With the Next Tip

One win down! Pelican’s next candidates, each as its own test:

• Sizing (tip 4). Check system.compute.node_timeline for the new job cluster. If both worker nodes sit at low CPU and memory for most of the run, test one worker (or an autoscaling range of one to two) and watch the runtime.
• Photon (tip 6). m5.xlarge isn’t on the calculator’s Photon list, so this test also means a different instance, such as m5d.xlarge (same 4 vCPUs and 16 GB). At list prices, a jobs node-hour on m5d.xlarge with the engine on costs 2.001 × $0.15 + $0.226 = $0.52615, against $0.2955 today. Against today’s m5.xlarge job cluster, the job has to finish in $0.2955 ÷ $0.52615 of its current time to break even, or about 56% (roughly 67 minutes instead of 120), and that figure bundles two changes: the instance and the engine. For a clean read, test m5d.xlarge on the standard engine first. At 0.690 DBUs an hour, its node-hour costs 0.690 × $0.15 + $0.226 = $0.3295, so the engine on its own then has to finish in about 63% of the time (roughly 75 minutes).
• Spot (tip 5). Workers on Spot, driver on-demand. A 06:00 deadline after a 03:00 finish leaves three hours of slack for interruptions.
• Serverless (tip 7). Notebook tasks are supported, so the job can be compared on full cost, with nothing added for EC2.

The shared cluster’s daytime idle time (tip 3) goes to the analysts’ lead, with Pelican’s numbers attached. Same loop every time: one change, a test plan, a month of data, a decision.

Copy This Worksheet

Here’s the whole loop as a worksheet you can paste into a doc or ticket. Use one sheet per change.

DATABRICKS COST INVESTIGATION WORKSHEET (one change per sheet)

1. Question and owner
   Workload (workspace / cluster / warehouse / job):
   Business purpose and deadline:
   Owner:                         Approver:

2. Window (UTC, start inclusive, end exclusive)
   Before: ________ to ________
   After:  ________ to ________   Matching nights or weekdays? Y / N
   Data pulled on:

3. Price basis
   Databricks: list price / contract price    Currency:
   AWS: CUR cost column used:                 Savings Plans or RIs applied? Y / N
   Paying Databricks through AWS Marketplace? Y / N  (if Y, count those CUR lines once)

4. Cost boundary
   Compute: classic / serverless
   Classic: DBUs + EC2 + EBS + network   Tag used to match AWS lines:
   Serverless: Databricks charges only (DBUs plus any serverless networking SKUs); no EC2 line
   Shared costs and the split rule:

5. What changed
   SKU > workspace > cluster > owner/label > job:
   Gaps (no job_id, no owner label, corrections, delays):

6. Useful work (before)
   Successful runs:        Failed attempts:      Input size:
   Runtime per run:        Deadline met:         Output checks:

7. The one change
   Tip:                    Setting, before > after:
   What stays the same:
   Success criteria:       Stop criteria:
   Rollback config saved at:                     Approved by:

8. Before / after            Before       After
   DBUs (by SKU)
   Databricks cost
   AWS cost
   Combined cost
   Successful runs
   Failed attempts
   Cost per successful run
   Runtime / deadline met
   Output matches?

9. Decision
   Keep / retest / revert:     Date:     Billing settled? Y / N
   Evidence:

10. Next
   Next tip to test:          Review date:          Alerts go to:

Filling in the AWS half of this sheet every month is the tedious part, since it means joining the billing table to the CUR by hand. If your team would rather not maintain that join, CloudForecast’s Databricks add-on, in early access for Databricks on AWS, is a read-only connection that shows DBUs and the AWS servers underneath in one dashboard, by workspace, cluster, owner and tag. Learn about Databricks early access.

Keep Costs Down After the First Win

Keeping a saving is mostly unglamorous meat and potatoes. You need three habits.

• Owners. Keep every cluster, job and warehouse labeled with its team (tip 1), and make the label part of your new-cluster checklist or compute policy.
• A monthly review. Re-run the worksheet’s steps 1 and 5 for your biggest jobs and clusters, with your FinOps or platform team in the room. Databricks recommends regular cost audits and monthly cost reports for the same reason.
• Spike alerts with a name on them. An alert nobody owns is just an email. Route each alert to the person who can change the workload.

Know what Databricks’ native tools do and don’t do. Budgets track spend in USD at list price and email you when a monthly threshold is crossed. Notifications can take up to 24 hours, and budgets ignore credits and negotiated discounts. They don’t stop your clusters, jobs or warehouses. Blocking exists only for specific AI products such as Unity Gateway endpoints, and even that is enforced approximately. Serverless usage policies are a separate feature that labels serverless usage for attribution. They don’t limit spend either.

If you’d rather build the monitoring yourself, the native route is solid: the system tables system.billing.usage and system.billing.list_prices for the Databricks side (account admins can import a pre-built cost dashboard), joined with the CUR for the AWS side by ClusterId or your own labels.

Start With One Job This Week

So, why did the bill go up, and how do you bring it down? Databricks cost optimization comes down to one loop: measure the full cost of a job, Databricks plus AWS, find what changed and who owns it, then change one thing and check the cost per successful run. For Pelican, one setting moved an illustrative $141.45 a month to $53.63 at list prices. Your numbers will differ, but the loop works the same way, so pick one job this week and fill in the worksheet.

If you’d rather not maintain the combined Databricks and AWS report yourself, CloudForecast’s Databricks add-on is in early access for Databricks on AWS.

It’s a read-only connection that shows DBUs and the AWS servers underneath in one dashboard, by workspace, cluster, owner and tag, plus a daily report and spike alerts in your inbox, Slack or Teams. It shows you where the money goes; your team still decides what to change. See Databricks early access.