Big Data is changing the world. The ability to store, manage and extract actionable insights from vast and deep lakes of disparate data pouring in from multiple sources has transformed businesses, industries, and our lives.
And for this we can thank the cloud.
Big Data in the beginning
Before cloud computing came along, Big Data was confined to Apache Hadoop platforms running on large numbers of commodity servers in a physical environment. And while Hadoop was very effective at managing and storing massive volumes of raw unstructured data, analyzing that data was a slow, costly and cumbersome process that required teams of data analysts and often took weeks. Faced with rapidly growing data volumes, organizations and IT departments became frustrated and overwhelmed as they lacked the technology required to fully leverage Big Data for the valuable insights locked within.
Here comes the cloud
The world is being flooded with information at an alarming rate. Companies today process 1,000 times more data than they did only a decade ago, and 80 percent of that data is unstructured. Those data demands call for a new Big Data solution---a solution that democratizes Big Data.
That’s where the cloud comes in.
First came the public clouds, search engines and social networks. Their arrival provided the innovative spark needed to heat up interest in Big Data and Hadoop. Soon Google and other public cloud providers such as Amazon Web Services and Microsoft began offering reasonably inexpensive plug-in platforms for Big Data systems.
Today’s cloud-based Hadoop platforms provide virtually unlimited resources to both large and small organizations, on demand. With the ability to scale up or down quickly, the cloud gives organizations an easy and affordable way to acquire the level of processing power and storage they need to gain rapid Big Data insights---not in weeks but in a matter of moments.
Thanks to the cloud
Big Data is a game-changer for businesses, industries, and each of our lives. For example:
The Healthcare Industry is using Big Data to improve the quality of patient care by reducing the time it takes to accurately diagnose a patient’s condition and start providing appropriate treatment. Big Data analytics also helps to reduce hospital admissions by providing physicians with richer and more detailed information about their patients with which to make decisions that will ensure the best outcomes. And sensor devices used by patients both at home and on the go deliver streams of constant data, monitored and analyzed in real-time, to help patients self-manage their conditions and enjoy a better quality of life.
Marketers are using Big Data to get to know consumers like never before. Using analytics to drill down through segments, demographics and sample sizes to get a granular view of the individual customer or “market of one”, marketers gain insights they can use to create targeted and personalized marketing to better meet customer needs and enhance the customer experience. Add geo-location data to the mix and marketers have a way to interact with customers on their mobile devices to offer them relevant promotions in context and in real-time.
The Design Industry is using Big Data to spur creativity and innovation. Using computer modeling and simulation software, designers can now render, test and refine products in a virtual world to arrive at the best design possible before going into production. But that’s just the beginning. Big Data now makes “experience design” possible, which is the seamless fusion of form, function, and interfaces to create a product that provides an individualized and irresistible experience for the end user.
People everywhere are benefitting from Big Data in the cloud. Today smartphones, tablets, wearables and apps are connecting us and the objects that surround us---our cars, our home appliances, our watches, even our toothbrushes---to the Web like never before. This new emerging world, known as the Internet of Things, will create unprecedented data demands and challenges as data streaming in from objects is added to the already astronomical volumes of data generated by texts, transactions, emails, social media, sensors and more.
Fortunately, sophisticated analytics platforms such as cloud-based Hadoop stand ready to meet the formidable challenges posed by Big Data and the Internet of Things.