How We Saved Over $1,000 By Building CloudForecast With Serverless And AWS Lambda

Francois Lagier co-founder and chief technology officer

Last updated 30 May, 2024

4 mins read

How We Saved Over $1,000 By Building CloudForecast With Serverless And AWS Lambda150

In a previous blog post, we described our journey to launch CloudForecast and some of the questions every startup face. In this post, we will answer the question “What’s the right way to build this product with our resources?”.

The “$100 Startup” concept

I’ve been obsessed with a book called “$100 Startup” by Chris Guillebeau for over a year now. In his book, Chris advises his readers to get their product out fast while being cost-effective to limit the damage if your idea fails (definitely possible, unfortunately!). We try to apply this advice on every side of our business including our technical decisions.

CloudForecast’s Goals

With the $100 startup concept in mind, we started by listing our requirements that were important to us while building CloudForecast:

  • Ship it out fast: We were excited to build CloudForecast to help companies save money on AWS. We wanted to put it in front of our clients ASAP to keep our excitement going. Spending time debugging config, deploy process, etc. will not make the end product better. We were looking for an out of the box solution that we can grow with and allowed us to focus on our MVP the right away.
  • Cost effective: As an early startup, we wanted to avoid fixed costs by building a system that will scale seamlessly based on our client base.

AWS Lambda + Serverless = Easy + Focus + Cost Effective

With our next steps, we brainstormed on how can we achieve our goals and requirements. We discussed managing our own instances, using containers and half a dozen other ideas but we wanted something simple so we decided to use a Serverless solution (or FaaS). While there are few downsides (e.g cold starts, …), we believe going Serverless better suited our use case (zero administration, pay-per-execution with no idle cost and auto-scaling).

For the dev setup and deploy process, we decided to use Serverless
Framework
with AWS Lambda for the following reasons:

  • We can focus on writing our product and let Serverless manage the rest
    (permission, event management)
  • Easy to configure and easy to deploy
  • Serverless supports multiple platforms (GCP, AWS, … ) which could prevent
    headaches in the future. We picked AWS Lambdas to start with since our clients
    data will be stored in S3 bucket. It was a logical decision in order to reduce the
    reduce the network cost
  • Great documentation and community (Github, Gitter, Slack)

Our original requirements were fairly simple: We needed 4 AWS lambda functions
each on their own cron job, and each lambda would need to talk to various AWS
products (RDS, DynamoDb, SQS et al.). All that with an easy way to manage
multiple environments (dev vs prod) easily with an easy/effective way to manage
resource permissions.

Here is how we do it:

serverless 1
serverless 2

With these two code snippets, we were able to set up most of our architecture. 4 functions that will interact with each other through SNS. Two functions will run check on a cron schedule (via the ‘schedule.rate’ params) to check a file need to be reprocessed and trigger another function via an SNS. This configuration will be able to scale effortlessly while keeping our cost under control. We are able to fully silo-ed our environments using iamRoleStatements to configure the permissions. We used an external variable to manage environment variables from a simple YAML file.

We considered running a couple of small instances to do a job which would cost us at least $1k for the year. Meanwhile, the AWS Lambda cost is meaningless since we are only running tens of thousands of functions a day.

What’s Next?

Using Serverless helped us get CloudForecast off the ground quickly. We saved our first clients thousands of dollars by giving them visibility into their AWS spend.

We are in the process of integrating AWS Athena into CloudForecast for our next project. Integrating with AWS Athena will enable new features that will allow end users to dive deeper into their AWS spend. Every company has unique spending patterns, and this will allow them to fully understand where their money is going. Check out our new blog post on how to save money on AWS by using Parquet on AWS Athena.

If you have any questions related to this post or what we do at CloudForecast, feel free to reach out to me at [email protected]. We would love to hear from you!

-Francois & the CloudForecast Team

Haven’t tried our features? Sign up today and get started with a risk-free 30 day free trial. No credit card required.

Special Thanks to Kyle Galbraith for reviewing this draft. Check out his weekly Newsletter, Learn By Doing. Learn by Doing is a free weekly curated cloud, blockchain, and coding newsletter delivered to your inbox every week.

Manage, track, and report your AWS spending in seconds — not hours

CloudForecast’s focused daily AWS cost monitoring reports to help busy engineering teams understand their AWS costs, rapidly respond to any overspends, and promote opportunities to save costs.

Monitor & Manage AWS Cost in Seconds — Not Hours

CloudForecast makes the tedious work of AWS cost monitoring less tedious.

AWS cost management is easy with CloudForecast

We would love to learn more about the problems you are facing around AWS cost. Connect with us directly and we’ll schedule a time to chat!

AWS daily cost reports