The business world is undergoing a total transformation as new technologies enable the rapid processing and analysis of huge volumes of data in real-time, equipping business leaders with unprecedented insight into granular detail of every aspect of their business. A new wave of applications leveraging real-time data to provide accurate insights into challenges and opportunities is heralding the entry of the Intelligent Enterprise. But such changes do not come without significant challenges.
The escalating volume of data and the organisational complexity required to collect and store the data and then analyse it for insights that can deliver business value is forcing IT leaders to fundamentally rethink their data storage and database management strategies. As technologies such as IoT generate exponentially more data, IT leaders will have to transform their approach not only to how data is stored and managed right now but build strategies that can equip the enterprise with the agility to manage tomorrow’s challenges too.
The escalating challenge of data storage
Not too long ago, data scientists maintained that the volume of data would double every two years to reach 40 zettabytes – or 40 trillion gigabytes – by 2020. As the Internet of Things becomes increasingly pervasive, though, that figure has been revised to 44ZB, incorporating increasing volumes of non-text data such as video, graphics and other visual media.
A recent SAP study of more than 500 IT decision-makers around the world found that 74% of business leaders felt their data landscape is so complex that it limits their agility, with 86% saying they were not getting the most out of their data. When the billions of sensors connected to the Internet of Everything comes online (according to Gartner anything between 12 and 30 billion sensors will be active by 2020) the volume of data will increase exponentially, making it even more important to store data in the most effective and efficient manner possible.
Compounding the problem is the typical enterprise’s reliance on a mix of older and next-generation data storage tools, ranging from high-end in-memory computing platforms to tapes and external hard drives. Without integrating these data sets into a cohesive core, the data residing in such environments holds little to no value to the business. And considering the sheer volume of data a modern enterprise generates, data storage strategies need to evolve quickly to enable IT leaders to deliver real-time value to the business using accurate insights gained from verified data.
Plotting big data on a temperature heat map
One measure enterprises may take in storing and managing data more effectively is to classify data according to its ‘temperature’, from hot to cold. Hot data is the most accessed and includes the likes of the latest sales data. According to studies, this type of data accounts for less than 20% of a typical enterprise’s stored data. Hot data needs to be accessed frequently and as such performs best when stored in-memory where it can be retrieved quickly.
Cold data is the remaining 80% of an enterprise’s data which is accessed less than 10% of the time. It’s not typically cost-effective to store this type of data in-memory, despite the falling cost of memory.
Enterprises wishing to allocate a temperature to different types of data to more effectively store and retrieve such data need to look at a form of dynamic storage hierarchy. While it’s important to have experienced and skilled database administrators driving the initial process of developing the data storage strategy, it is impossible for humans to determine the heat of data manually.
Here, a database management platform that can support different data management strategies can deliver the best value. SAP HANA enables advanced analytics running alongside high-speed transactions in real-time for hot data while also integrating with cold data solutions such as Hadoop as well as other external data storage solutions. In one IDC white paper, businesses using SAP HANA realised a five-year ROI of 575% while enabling 40% higher productivity and 29% more efficient database management.
