Where big data is headed this year

The global big data market is forecast to top $70 billion by the end of this year and more than $103 billion by 2027. Making sense of this data will be a key differentiator for any organisation in this ultra-connected world. Andreas Bartsch, head of innovation and services at PBT Group, and Chad Gouws, data analyst at PBT Group, discuss several of the key data and analytics trends to be aware of in 2022.

“For South Africa, the most relevant trends will centre on the familiar themes we have seen over the past 12-months. It stands to reason that local companies have vastly different data and analytic maturity levels to those in the North American and European markets,” says Bartsch.

To this end, the four key topics will focus on the continuing move to the cloud, the practical use of artificial intelligence (AI), the focus on DataOps, and the importance of governance in a data-driven environment.

Cloud is a constant

“Those organisations still undecided about the benefits of the cloud have seen how effective it can be to manage a hybrid workforce. But more than a singular approach, it is the hybrid and multi-cloud that will gain increasing traction in South Africa,” Bartsch says.

Business and technology leaders cannot blindly charge into this as there will be significant cost implications to consider. This will see data platform readiness become even more important.

“Things like data architecture, data modelling, data engineering, and data governance remain key ingredients to a successful data and analytics strategy, even more so in preparing for the rocky path to the cloud,” says Bartsch.

Realising artificial intelligence

“AI and machine learning (ML) will be vital for organisational success this year. I expect these technologies to become indispensable as more businesses become data-driven and garner the insights necessary to differentiate themselves on the global stage,” says Gouws.

This is where a well-designed data architecture will be essential to analytics and AI projects. Having this in place will make the integration process with AI and ML not only faster to do, but also more robust when navigating the complexities of legacy systems.

“AI will become smarter at an accelerated pace. Companies will more easily migrate from pilot projects into more practical ones as they start operationalising the AI models they have running. It is now about creating an internal product from AI that focuses on automation and managing the repetitiveness of tasks,” says Bartsch.

DataOps focus

Bartsch believes that the push to reduce time to market on data products driven by the increasing automation resulting from AI, will see DataOps mature inside the organisation.

“The merging of data pipeline process and IT operations will help to improve the velocity, quality, and predictability of these data and analytics environments. DataOps will strengthen collaboration, orchestration, monitoring, and ease of use, along with much-improved maintainability when it comes to data and analytics,” says Bartsch.

Foundation on governance

From a governance perspective, machine-driven models to automatically monitor for breaches and identify potential vulnerabilities in data systems will be a significant area for growth.

“Cybersecurity providers will leverage this to monitor customer networks in real-time and proactively identify weak points to mitigate against the risk of them being exploited. At a fundamental level, data will remain a key ingredient to any digital transformation strategy,” says Gouws.

Part of this is the ongoing regulatory and compliance pressure on organisations. This could very well assist in enforcing the focus on all aspects of data governance (people, processes, and technology).

Bartsch says: “It provides an opportunity for the business to use data governance to necessitate ownership and replace certain structures and put in place the means of establishing a data-driven culture. Now, more than ever, companies will realise that when it comes to data and analytics they need to walk before they run. 2022 is going to be a massive year for Big Data strategies. But taking small steps and getting the basics right will be critical to success.”