How Smaily scales infrastructure to send 100M+ emails per month
Smaily uses SecretHub to manage a large server infrastructure with just a small DevOps team.
- Scaling out server infrastructure to serve more and more customers put a strain on the small DevOps team and how they managed the ever growing infrastructure.
- Updating root secrets was so time-consuming and complex, the team started to avoid the procedure.
- Now, everything is automated so important procedures are never skipped and the team can focus on delivering product instead of time-consuming operations.
Sending over 100 million emails per month, Smaily is responsible for the management and distribution of email marketing campaigns for some of the biggest brands of the Baltics and Scandinavia: the world’s largest security company G4S, Finnish telecommunications leader Elisa, grocery retailer Maxima Group and hundreds of others.
Helping enterprises deliver their deals, newsletters and other messages to millions of customers around the world comes with many technical challenges, chief among them being scale.
“Business is growing and we need to serve more and more customers, so we’re always looking for ways to scale. Scaling vertically is not an option for us. We need to add more and more servers as we grow to distribute the load horizontally,” says Kait Todesk, CTO of Smaily.
But scaling out is not the kind of problem you just throw more CPUs and RAM at. With bigger and bigger infrastructure, comes the need for more scalable processes to manage and provision that ever growing infrastructure. One of those fundamental processes is secret management.
“The main reason I went looking for a solution was that we didn’t have a good solution for root infrastructure secrets like DNS, PKI infrastructure, etc. These secrets needed to be distributed securely to wherever they were needed in our infrastructure and everyone on the team needed to be kept up to date with the latest version of the secrets. Updating the secrets and re-distributing them across the infrastructure was always a very involved operation, which caused us to start avoiding the procedure.” says Janar Todesk, Principal Engineer at Smaily.
“In the very beginning we used git-crypt to PGP-encrypt the secrets and store them alongside our infrastructure configuration in version control. Next we tried LUKS-encrypted thumb drives, but neither of these approaches scaled. And since we have a small infrastructure team, we don’t have the resources for overwhelmingly complex procedures.”
Our goal was to automate everything in a way that was simple, scaled well and integrated with our configuration management, so we would no longer avoid updating those root secrets.
“Our main use case for SecretHub is a safe place to store the root secrets, and distribute these securely to wherever these might be needed. We use Ansible to provision our infrastructure and integrating it with SecretHub was really simple. We just call the SecretHub CLI from the playbooks and that provides everything we need. It’s the only integration we need right now and we don’t need any more complexity in that part of the pipeline.”
“Now we can update secrets whenever we feel the need, we don’t have to worry about backups, and everybody’s always up to date with the latest state.”
“When making the decision to buy SecretHub, the main thing I researched was their security model and how they handle everything. The main selling point for us was that all the data we push to SecretHub is always encrypted and they don’t even know what we send them. Only we have the keys to decrypt the data stored in SecretHub.”
“Also, their code quality was actually quite impressive, because I checked out their open source projects and they’re really nicely written and structured. That was a big bonus for me, because that shows me they know what they’re doing.”
“In short, we’ve tried pretty much every solution out there for secret management and SecretHub is The Thing filling the void in our pipeline. The experience was super nice, we had zero friction integrating SecretHub with our infrastructure. I liked the ease of use of SecretHub. It’s really simple to use and the simplicity of updating the secrets, keeping everybody on the team up to date, and integrating all that with configuration management is something others must know about.”
“I would definitely recommend SecretHub to anyone who wants to automate anything in their infrastructure.”