By Kana Sabaratnam, Former General Manager, Lava Labs
You are probably wondering, how Facebook knows who are your friends in a photo that you just uploaded? Or how Google knows the events in your life and the videos start listing automatically in your YouTube account.
Well, the use of Big Data and Web Intelligence has been around for awhile but enterprise and corporate entities are still early in taking advantage of this new technology for their own data analytics.
It is imperative that C-level executives know the power and ability of Big Data to move their organisation to newer heights. Big Data is a competitive advantage and it will bring customers closer than ever before. Big Data is a game changer that can improve customer service and how an organisation is perceived.
Big Data and Web Intelligence Analytics is an automated process of pattern recognition from large amounts of data, which results in a predictive decision.
Big Data introduces new data storage technologies that does not rely on the traditional database relationships and indexing but it focuses on primarily improving the speed of how data is stored and retrieved. Further, the information is stored very differently from the traditional row and column architectures.
Web Intelligence is an artificial intelligent predictive decision making process that is based on various pattern mapping and data reduction methods. Together, Big Data and Web Intelligence have created associated new technologies such as NoSQL, Hadoop and MapReduce that is used in today’s predictive analytics. These are the core technologies in Big Data and Web Intelligence Analytics.
Since the birth of the Social Networking sites, these sites have been generating unprecedented amounts of data. Therefore, it is only natural that the Social Networking giants have been very instrumental in creating these core technologies. Google for example, has single-handedly developed Google File Distributed Systems, to store, process and retrieve data.
This new technology is now being introduced to corporates and enterprise as Big Data and Web Intelligence tools and frameworks. This new technology is called Hadoop and it is available as an open source project from the Apache group. It allows for distributed and parallel computing using the Hadoop framework. Many computers can read and write during the Big Data and Web Intelligence analysis process. This distributed file system brings the power of parallel computing, where very large amounts of data can be processed in seconds.
Now we shall look at how the data is stored in the file system. When the traditional relational databases were not able to handle the size of current data sizes, it was back to the drawing boards of how a data storage should behave. It was very clear that the two limiting factors are: 1.) relationships and 2.) indexes had to be removed to drastically to increase the speed of data storage systems.
Traditional database use what is called Structure Query Language (SQL) to manipulate data. Big Data employes NoSQL technology where the use of complex SQL statements are not permitted.
This does not mean that data is not structured in a NoSQL database. However, the data is retrieved in a persistent manner that proves to be much better than traditional databases.
Persistent data is when data is stored with respect to data and timestamps. All modifications to the data are registered with the modified date and time stamps. The previous version is always available in persistent data storage. The retrieval of persistent data delivers all the information at once, this allows for analytics based on the history of the data modifications.
The integral piece in Big Data and Web Intelligence Analytics which forms the core technology that makes this whole experience applicable is MapReduce.
MapReduce is a process that maps the data into reduced structures that are specific to facilitate a required structure for analysis. This analysis may be rolled-up or aggregated to produce any number of mapped reductions.
Finally the data is synthesized by additional pattern recognition and artificial intelligence to deliver decision making actions. The decision making process is carried out by distributed intelligent systems that extract information from the reduced data sets. The entire process is automated and the decision will be taking place in real time.
The power of Big Data has already taken root in many government policy decision making processes so why not in every organization’s operational decision making? The tools and frameworks are already available and the proof is evident. The most important takeaway from this initiative is that C-Level executives have to start using Big Data and Web Intelligence to empower the decision making process in your own organization.
Lava is an authorised Salesforce Partner in Malaysia and has more than a decade of experience in cloud solutions which includes marketing automation, CRM implementation, change management, and consultation. We pride ourselves in not just being a CRM partner but in also understanding the needs of our customers and taking their business to the next level.