Case Study | Acciona: CECOC Platform: Unifying Data and Optimizing Processes
Acciona is a Spanish company with a global presence in 65 countries across five continents. They are a group that specializes in developing and managing infrastructure such as water, concessions, construction, and services, with a focus on renewable energies. Acciona invests in sustainable projects and is committed to leading the transition to a low-carbon economy by prioritizing technical excellence and innovation.
Their activities cover the entire value chain, including design, construction, operation and maintenance, providing sustainable solutions to the major challenges of our era, including the energy transition and environmental conservation.
Leading the Energy transition through digitization.
The energy transition has become a necessity, especially in recent years, due to the implementation of mandatory decarbonization and zero emissions policies. This highlights the importance of focusing on renewable energies and digitalization.
The goal is to lead the transition to a low-carbon economy, so it is necessary to invest in projects of technical excellence and technological innovation to design a better planet and contribute to economic and social development.
The challenge Acciona Construction had to face was to build, develop and deploy a data platform in Google’s public cloud for the ingestion, consolidation and productivity of data generated from multiple sources of on-site production.
In order to meet the challenge, Acciona Construction had to address several requirements, which included:
The construction of big data architectures in the public cloud.
The ingestion of data from the identified sources.
The processing and consolidation of data through the development of ETLs.
The accompaniment of the Acciona Construcción IT team.
Solution on Google Cloud Platform
Keepler teamed with Acciona IT to design CECOC data platform, a data lake based on three layers that are ingesting a series of sources following a mainly batch model, specifically databases and some APIs. Although, other types of ingest based on streaming were also started to be considered for other cases, such as heavy machinery data. In 2022 was made a proof of concept for tunnel boring machines, which will be taken up again during 2023.
The first layer is a Raw layer based on Cloud Storage.
The intermediate layer, which we call Common, is based on BigQuery.
The final layer is a consumption layer in which the technology is proposed specifically for each case. Mainly in BigQuery, although other storage systems such as relational databases or noSQL databases, such as DataStore, are being considered.
From this consumption layer there are a number of applications, especially reports in PowerBI and analytics applications that consume this data. Recently, CloudComposer, a managed AirFlow environment, is being introduced as an orchestrator of the platform’s ingest and transformation processes.
Google Cloud Platform services used were as follows
Cloud Storage: Raw Layer / Common Layer
Cloud IoT Core: Connection
API Gateaway: Connection / Serving
Cloud Pub/Sub: Ingestion
Cloud Functions: Ingestion / Transform and Load / Serving
Cloud Dataflow: Ingestion / Transform and Load
SFTP Server Marketplace: Ingestion / Serving
Cloud Monitoring and Cloud Logging: Monitoring
Cloud IAM: Security / Serving
Secret Manager: Security
Key Management Service: Security
Cloud Data Catalog: Validation and Metadata / Functional Validations
Cloud Dataproc: Validation and Metadata
Cloud Composer: Common Layer
Cloud Bigtable: Consumption Layer
Cloud Datastore: Consumption Layer
Cloud SQL: Consumption Layer
BigQuery: Common Layer / Consumption Layer
Benefits for the client
Unified Data Platform
Acciona Construction now has a platform in which to unify all data generated from multiple sources with the creation of the CECOC platform they achieved a centralized repository designed to store, process and protect large amounts of data.
Enhanced Data Management
Acciona Construction has adopted data governance, data quality and process monitoring.
Data Process Optimization
Establishment of best practices and optimization of processes regarding data.
Keepler is a full-stack analytics services company specialized in the design, construction, deployment and operation of advanced public cloud analytics custom-made solutions. We bring to the market the Data Product concept, which is a fully automated, public cloud services-based, tailored software that adds advanced analytics, data engineering, massive data processing, and monitoring features. In addition, we help our customers transition to using public cloud services securely and improve data governance to make the organization more data-centric.
Would you like to talk about your business?
We can help you leverage the power of data to enhance your operations.