Most of the time, when people talk about working with DevOps philosophies, they have in mind the creation of self-sufficient teams (“you-build-it-you-run-it“) applied to the development of a specific product.
DevOps allows in these scenarios, through automation and changes in the way processes and people interact, to break down the typical silos between development and operations and deliver value with greater speed and quality.
Based on these principles, other terms have appeared to explain the change of philosophy in other scenarios: SecDevOps for the application of these principles to security, MLOps for Machine Learning processes or FinOps, focused on cost processes.
When organizations grow, Cloud strategy may impose the need to govern several public cloud providers, hundreds of accounts and dozens of DevOps teams that deliver value based on certain standards defined by teams such as Cloud Centers of Excellence. In this scenario, it is essential to scale the DevOps model to the teams that govern the cloud, applying its principles to tasks that allow standardizing and governing the public cloud based on frameworks such as the Well Architected Framework.
An example of this type of transformation applied to automation can be the following architecture for creating governance dashboards in an automated way for hundreds of accounts in AWS.
In this case we start from a data catalog in Athena that aggregates -like AWS Config- all the resources of the different accounts. In this solution we have a non-relational database (AWS DynamoDB) that stores Athena queries that reflect our governance cases.
For example, thanks to this solution, queries can be created that reflect the number of AWS workspaces that have not been deployed in 90 days, the instances with a security group open to unauthorized IPs, or the correlation between the number of requests from a user and the times at which they connect to the AWS console. Any request to Athena (whether to CloudTrail, inventory data or the account cost table) can help us in creating governance dashboards.
A Step Function then automatically orchestrates each query, storing the results in an S3 bucket and launching an AWS Glue crawler. This is a database that can also be queried from Athena, containing the views of the governance cases to be painted from Quicksight.
Finally, the Step Function retrieves table names and creates shared datasets with governance teams to be able to make analysis and graphs with data.
With solutions of this type, the task of control, automation, governance and establishment of best practices within the organization is simplified.
The creation of multidisciplinary Cloud governance teams, with skills to develop their own governance products following DevOps principles, can greatly facilitate the work of large organizations in their public cloud governance strategy.
Leave A Comment