How to Detect AWS Cost Anomalies Before They Spiral Out of Control

Cost awareness is becoming an increasingly important part of engineering culture. With tighter corporate budgets, especially following the COVID-19 outbreak, many organizations are putting AWS resources under a microscope. The good news is that there are several ways that you can automatically identify cost anomalies and address them before they become a bigger problem.

Let’s take a look at how to identify these anomalies with AWS Budget, AWS Cost Explorer, or third-party solutions like CloudForecast that can save you time and money.

AWS Budgets Monitoring

AWS Budgets lets you set custom budgets that alert you when your costs or usage exceed (or are forecasted to exceed) the budgeted amount. You can also use the tool to set reservation utilization or coverage targets and receive alerts when utilization drops below your thresholds. Alerts can be sent via email at a daily, weekly or monthly cadence.

It’s easy to get started with a new budget:

1. Sign into your AWS Management Console and navigate to the Billing and Cost Management console.
2. Choose Budgets > Create Budget > Cost Budget > Set up your budget.
3. Choose a name for your budget and select the period, or how often you’d want the budget to reset the actual and forecasted spend.
4. Set a fixed Budgeted Amount for the period or enter the amounts for Monthly and Quarterly Planning budgets.
5. Optionally, set the Budget effective dates or other parameters.
6. Choose Configure Alerts.
7. Set an Actual or Forecast alert and enter the threshold that you’d like to use as a percentage or absolute value.
8. Setup the Email contacts and optionally set the SNS topic ARN to use the Amazon Simple Notification Service.
9. Choose Confirm Budget.
10. Choose Create after reviewing the budget settings.

The problem with AWS Budgets is that you’re limited to static thresholds, which aren’t very useful across many accounts or in dynamic environments. While it’s helpful to know how much you’ve spent, the real goal of cost awareness is to shift from what are the costs? to why are the costs trending in this way? in order to find opportunities for savings.

The only way to get more granular insight is by using AWS Cost Explorer. While this tool provides all of the data you need, it takes significant resources to aggregate the data in a way that derives real and actionable insights. These are engineering hours that may be better spent focusing on the product rather than generating cost reports for stakeholders.

Using CloudForecast

CloudForecast provides everything you need to monitor and eliminate wasted cost in AWS without significant engineering time and resources. Rather than spending engineering hours hacking AWS Cost Explorer, you can use our out-of-the-box reports that contain all of the actionable metrics engineering teams require and different metrics for stakeholders.

Download our Checklist of Ways to Lower Your AWS Bill to learn how you can slim down your team’s expenses:

Email cost reports can help engineering teams become more cost-aware in less than 30 seconds, while our monthly financial report keeps Finance and C-Level teams up-to-date in a familiar Excel spreadsheet with breakdowns and trends that they can understand. There’s no need to spend hours in meetings generating reports and explaining numbers.

In addition to high-level data, our AWS Cost Charts let you dive into any specific product, region, tag or sub-account graphs to take a deeper dive into AWS costs. You don’t have to spend hours digging into reports to find what’s really contributing to the costs — you can quickly drill down into the relevant details in minutes.

Sign up for a free trial to see it in action!

How to Spot Anomalies

There are many different ways to define a cost anomaly. For example, you may look for an absolute change in dollars or percentage, or for dynamic usage, look at standard deviations from an average (e.g. the z-score). You can adjust these levels and timeframes to create alerts that look for both sudden spikes in usage or slow increases over time.

If you’re using the AWS Cost Explorer API, you must programmatically query cost and usage data to compute these values and send alerts for any anomalies. Using the API, you can access up to 12 months of monthly level granularity and up to three months of daily level granularity, and the billing information is updated three times daily.

These techniques are great for engineers looking for cost trends and anomalies, but stakeholders require an entirely different kind of report. CFOs and other C-Level executives require higher level reports that show longer-term trends, preferably in an Excel spreadsheet that they can incorporate into their other business spreadsheets.

CloudForecast eliminates the need to spend valuable engineering hours on cost reports by automatically building reports and alerting you of any cost anomalies in a way that’s extremely easy to understand with visualizations. In addition, you can receive reports via email, Slack, PagerDuty, and Opsgenie for faster follow-up from engineers.

The Bottom Line

Cost awareness is a necessity for engineering teams, but it can be challenging to truly understand those costs. While AWS Budgets provides basic alerts, they are insufficient for organizations with complex resource usage. AWS Cost Explorer provides everything you need to build your own reports, but it takes significant time and effort to do so.

When looking for cost anomalies, it’s important to look for both sudden spikes in usage and slower and more insidious trends higher. You can find these problems by looking at everything from absolute dollar values to percentages to standard deviations (z-scores). Most of this data can be found in AWS Cost Explorer’s API.

CloudForecast simplifies the process by automatically aggregating data from your AWS instances and creating actionable reports for both engineers and stakeholders. Even better, these reports can be accessed via email, Slack or countless other integrations.

Sign up for a free trial to see how it works!