Challenge
With AWS adoption increasing throughout the organization, the scale at which cost management initiatives need to function is now global. How can cost management be applied in a manner that ensures compliance requirements for data retention and management are unaffected?
Many of the modern tools available for discovering, analyzing and actioning managed data simply don’t function well yet at Petabytes scale. How can the native features in AWS technologies be combined with modern tools to make this possible?
Solution
Hestio combined several of the native data management features available in AWS with it’s own tools to deliver an approach that provides repeatable and automatable cost management. This involved;
- Building inventory to understand what data the customer already had
- Mapping each of the data-sets to it’s required data management policy
- Codifying the data management policies
- Leveraging automation to have data management policies apply on both existing and new data
Figure: Matching data-sets to policies
To handle the immense scale of data involved, we took advantage power of both AWS’s and Hestio’s rich suite of tools to codify and automate taking actions in line with each of the data management policies.
Business benefits for the customer
The customer regained control over the costs of their data lake by identifying and automating the removal more than 1 PB of unnecessary data, at a saving of over $250,000 USD per year in AWS costs.
Along the way, we unveiled invaluable insights that provided a foundation for ongoing cost savings and operational improvements.
If you’re interested in more of the details about how we delivered this success, check out our deep-dive article here
