How Dynata Uses CloudForecast to Socialize AWS Costs to Their Team

Dynata is the world’s largest first-party data and insights platform. Nearly 6,000 customers globally use its market research solutions to understand the attitudes and buying habits of ~62 million consumers and business decision-makers worldwide. Its robust, trustworthy data, collected directly from individuals, can target profiles based on thousands of attributes.

Dynata’s customers access this data on-demand to create targeted questionnaires and surveys for their research, marketing, and CRM needs. They can use Dynata’s platform to analyze, visualize, and share critical insights quickly and at scale.

Recently, we interviewed Michael Schueler, Dynata’s Lead Engineer for DevOps and Site Reliability Engineering. He leads the engineering team that manages Dynata’s IT infrastructure, which is, by now, predominantly on the cloud. In this role, he reports to the VP of Engineering, and indirectly to the VP of IT Operations and the SVP Software Engineering.

His duties include managing cloud governance, access control, provisioning accounts, and monitoring AWS costs.

Challenges

Cloud Governance and Monitoring Costs Across Teams

While such an environment offers flexibility, it can lead to increased operational costs and inefficiencies as different departments can deploy applications and programs with little or no reuse. 

Dynata wanted a solution that could help provide the tools for better cloud governance – primarily monitoring costs across their different teams, and secondly allowing their security team a way to identify untagged resources.

Michael’s team inherited a third-party solution acquired previously to monitor costs, inventory their assets and improve security. While this solution offered some of the required features, it was not particularly good at addressing a key use case – providing more fine-grained insights into costs without the time investment required digging into complex dashboards. It was better at inventorying their assets, though. On the whole, Michael says that the solution’s subscription model, based on a percentage of Dynata’s AWS spend, did not offer them sufficient value.

Michael notes how well the CloudForecast product performed during a trial. He also observed how the CloudForecast team “went head over heels to try to get our business.” “Trent [the VP of IT operations] saw that too”, he adds.

Solutions & Outcomes

The Benefits Of Using CloudForecast

Michael’s responsibility includes identifying and implementing the most efficient use of cloud resources. To that end, his team’s initial challenge was to figure out which department to charge for the use of resources associated with each AWS account. 

Michael’s team tag all their resources and associate these with individuals, teams, and services, and track costs related to each of these entities. The only immediately available tools for this were from Amazon, which, while improving over time, still do not offer much more than a console or raw CSV output for reporting. They could have spent valuable engineering resources to automate the reporting, but realized quickly that it was more efficient to defer to a ready-made solution already targeting these precise requirements. CloudForecast’s solution had the ability to “slice and dice” (as he put it) the tags to create many different ways of reporting and visualizing the cost breakdown – per resources, per accounts and many other slices.

CloudForecast’s reports also highlight compliance to tagging policies, which are particularly useful to Dynata’s security team for ensuring that all resources have owners and are deployed in a compliant manner. A particular favorite of the security team is the report with the top 500 out-of-compliance tags along with their spend. The team tackles about 50 of these a week to bring them into compliance, assigning a person or team to each that is responsible for ensuring proper resource usage.

Another valuable feature is CloudForecast’s ZeroWaste feature, which helps Michael’s team discover idle or underutilized resources within EC2 instances and EBS Volumes that can unwittingly cost them money. ZeroWaste also gives them a list of all their old EBS snapshots, allowing them to decide if they want to eliminate some of these for additional cost savings.

While Amazon’s Trusted Advisor has some of these features in ZeroWaste, it still requires creating reports by hand, which the CloudForecast’s solution creates automatically and proactively delivers a report to his inbox. In fact, Michael spoke enthusiastically of these customized reports, noting particularly the engaging UX allowing the ability to see very quickly (via the sunny and cloudy icons, among other features) a sudden uptick in cost and alerting the responsible individual or teams of these.

Michael estimates a 15x ROI using CloudForecast this past year, from eliminating the costs for unnecessary cloud usage.