Smart Data vs. Data Lake

The world is creating a massive amount of data every day and organizations need a technology, information and data science strategy to collect, store, analyze and distribute data from multiple sources.

The "Data Lake" strategy is to collect and store all of this data.

The "Smart Data" strategy is to ONLY collect and store high value data relevant to achieve specific goals.

The "Data Lake" strategy rests on the concept of "Big Data" - defined as data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time. The premise is technology to collect, store and crunch huge amounts of data will provide value and advantage. While large data sets may provide great value in specific situations, it is very difficult to get any value out of large data sets without clear goals, employing sophisticated data science techniques and the right technologies.

The "Smart Data" strategy is to select high value data for specific goals versus collecting and storing huge volumes of data. Smart data avoids the curse of big data: more correlations and patterns between data yet more noise than signal. In other words, more correlations without causation leading to an illusion of reality resulting in bad decisions.

Smart organizations are planning to use the power of 5G, the Internet of Things and data science to better measure performance, discover patterns, prevent problems, and improve operations and processes. Sensors and smart devices are just starting to be embedded in many physical and biological things to collect data allowing organizations to better measure and improve performance. With smart devices and sensors everywhere - combined with data science techniques and processes organizations can make better decisions on the operational level, innovate new products and services on the tactical level, and make game changing shifts on the strategic level.

It is advantageous to take a process or activity that was previously invisible and turn it into Smart Data that can be measured, tracked and monitored to optimize processes, make better decisions and innovate.

Smart Data versus the Data Lake - choose wisely!