Few people can dispute that the amount of data that is generated and disseminated on any given day is increasing daily, with no slowing down in sight. According to business intelligence company, Bernard Marr & Co, every two days the population creates as much data as was recorded from the beginning of time until the year 2000.
With this continuing increase in data creation, the question remains – where does this data come from? Could this be what is referred to as Big Data? Where does all this data end up going and what purpose does it serve?
“Big Data” has become somewhat of a catch phrase in the past few years. Many consumers are still confused by what it means and how it is shaping the world we live in.
In a nutshell, Big Data refers to any category of information, from databases to photo and video libraries, collections of sound recordings, written text and sensor data that is generated and shared on devices and machines.
Due to the widespread adoption of smart devices and data services, almost every action we take leaves a digital footprint.
Big Data is the driving force behind many ongoing waves of digital transformation including artificial intelligence, data science, and the Internet of Things (IoT).
The amount of data we are creating does not show any sign of slowing down. Statistics Portal makes an interesting projection of data traffic patterns, forecasting global data traffic to grow from 7 exabyte per month to a staggering 49 exabytes per month, accounting for about 20% of all electricity usage by 2025.
We generate data whenever we go online, when we carry our GPS-equipped smartphones, when we communicate with our friends through social media or chat applications, and when we shop.
“As we continue to generate data, the amount of machine-generated data is rapidly growing as well. Additional volumes of data are generated and shared when our ‘smart’ home devices communicate with each other or with their home servers. As industries become digitised and adopt smart technologies, we expect the volumes of data that is churned out by sensors that gather and transmit information to increase exponentially,” says Phinda Ncala, Executive: Information Technology, IT Management for MTN SA.
So, what happens to this large volume of data that is generated each day? Through data analytics, this data is mined to gain meaningful insights that convert knowledge into action.
Big Data works on the principle that the more you know about something or a situation, the more reliably you can gain new insights and make predictions about what will happen in the future. By comparing more data points, relationships begin to emerge that were previously hidden, and these relationships enable us to learn and make smarter decisions.
“Most commonly, this is done through a process that involves building models, based on the data we can collect, and then running simulations, tweaking the value of data points each time and monitoring how it impacts our results,” Ncala explains.
This process is automated through advanced analytics instruments that are capable of running millions of simulations, tweaking all the possible variables until it finds a pattern – or an insight – that helps solve the problem it is working on.
Many industries utilise Big Data and data analytics to make informed decisions that are geared at predicting future outcomes and enhancing customer experience. Needless to say, each industry is different, and will therefore customise data analytics to achieve industry-speficic outcomes.
For example, utilities may use Big Data and data analytics to analyse consumption which then helps them to improve customer feedback, manage their field workforce more efficiently, and identify and correct system failures proactively.
Data analytics can be put to good use to impove healthcare by analysing vast numbers of medical records and images for patterns that can help spot disease early and develop new medical treatment plans.
In distaster-prone areas, data analytics has proven very helpful as sensor data can be analysed to predict where natural disasters such as earthquakes are likely to strike next, and patterns of human behaviour give clues that help organisations provide relief to survivors.
On the entertainment front, Spotify, an on-demand music service, uses big data analytics to collect data from its millions of users worldwide and then uses the analysed data to offer informed music recommendations to individual users.
Despite the multiple benefits that Big Data and data analytics presents for many industries and the public sector, there remains a minefield of challenges that have to be navigated. The absence of a cohesive ecosystem that is fully integrated renders the outcome of the feedback susceptible to errors. This is understandable as data analytics is not an exact science and is still in its early days.
The recent bout of data breaches experienced in the past few months have put into sharp focus the issue of privacy and the role that organisations need to play to safeguard their customer’s confidential information.
“The industry is going through a teething process, and we are learning important lessons along the way. We believe that Big Data and data analytics has immense potential of transforming customer experience across the broad range of applications. The onus is on the industry and other role players to ensure that this valuable knowledge is harnessed for the betterment of humankind and to help us address the socio-economic challenges that we are facing,” Ncala concludes.