The Cloud Data & Analytics trends highlighted below are already helping enterprises that we work with to modernise, automate, and predict customer or competitive behaviours.
Pioneer organisations have gone in early on some of these trends to gain competitive advantage. Instead of viewing these trends from a long-term lens, the pandemic has accelerated the need for some, if not all of them to be looked at today itself!
Cloud data technologies are making it easier for organisations to not only unlock value from their own internal data but also from data that is sourced externally from a data marketplace or from data received from partners and suppliers.
Data sharing has become easier to setup for the data source provider and straightforward for the data consumer to consume the data. Organisations are finding that they can derive immense business value if they share and receive data from their partners and suppliers. This can lead to mutually beneficial revenue or operational efficiency opportunities.
For example: An organisation’s supply chain has benefited tremendously from data sharing in both directions with a supplier due to better visibility of orders and invoices.
Data Marketplaces have been setup by multiple cloud technology vendors, and it has becomes easier for an organisation to monetise their data or for an organisation to find a dataset that can add value in conjunction with the data they already have.
Data Sharing is now possible across regions and across cloud vendors too.
A note of caution here would be to understand if you need to preserve the privacy of your data in anyway before sharing the data. There are various privacy preserving techniques that can be implemented before you share the data across the organisation or outside.
All these use cases can be executed knowing that your data is secure and protected. This has opened up a host of data driven opportunities.
Public cloud providers like Microsoft Azure, AWS and Google Cloud have been using Infrastructure as Code (IaC) for some time now. IaC is the managing and provisioning of infrastructure like servers, containers, connections, endpoints, permissions etc. through code instead of manual processes.
With IaC, configuration files are created that contain infrastructure specifications, which makes it easier to edit, distribute, automate, and repeat configurations.
Organisations are now learning that they can take a leaf out of these cloud vendor playbooks and move away from “click-ops” - i.e. move away from manual creation and administration of cloud based resources and into leveraging Infrastructure as Code tools that allow you to build, change, and version infrastructure safely and efficiently.
One technique that organisations have leveraged successfully is to conduct an initial pilot to identify the various types of cloud resources and configurations required for a successful cloud data lake or data warehouse implementation and use that knowledge gained to automate the production environments and if required the test and development ones too.
More and more System Integrators (SI’s) and cloud data vendors are building industry specific cloud focused data and analytics solutions. These solutions are quick to put together and integrate with an organisations data. The solutions are typically use case focused or sometimes focused on particular hard to get to source systems.
A large number of organisations have already or are looking to migrate data from on-premise to the cloud. They are in need of solutions that quickly and easily move data to the cloud, irrespective of the legacy technology they may reside in, the number of tables they may have or the volume of data they may house.
From there it's a matter of cleansing the data (if needed), integrating multiple business data sets together and implementing organisation specific business rules. Cloud based technology solutions have made this easier than it was in the past, in the on-premise world where one came across a complex web of extract-transform-load (ETL) code.
Data is then modelled using one of the standard data modelling techniques to make it suitable for reporting and predictive analytics. Some organisations have tried skipping this step in an attempt to quicken the cycle of getting data to the business but have since found that it doesn't stand the test of time.
Finally, industry specific predictive or data analytics solutions have been designed to extract value from the data like “Member Lifetime Value” or “Member Churn”.
Many Cloud Data tools released new features in 2021 that made it easier to perform tasks like:
Features like these have and will make it much easier for Business stakeholders, Data Analytics Leads, Data Analysts, Data Engineers, and Data Scientists to collaborate and work towards a common goal.
We have seen in 2021, that more and more organisations that have gone down the cloud data journey and those that are planning to do so are thinking about Data Governance from the outset. This is mainly because they have felt the pain of not having an explicit Data Governance program and a Data Governance roadmap in the on-premise world.
In the on-premise world, organisations have been living through the challenges like:
In the Cloud Data world, we have seen that organisations want to make a change by firstly conducting a review of their current state and then setting up a data governance roadmap. One of the important directions that have come from such exercises has been the use of explicit Data Governance tools to ensure that:
In conclusion, in 2022 organisations will need ask themselves these questions so that they are prepared for tomorrow:
If you need help with your cloud data conversation please do reach out to us here.
Copyright © Tridant Pty Ltd.
