Big data needs to become smart data

There was hardly any presentation or technical article over the past several years that was released without the buzzword "big data". They unanimously mentioned the major potentials and challenges as well. However, there are fewer articles about the topic. It is obvious that a great disillusionment is prevalent.

Use of data silos

The (still) growing quantities of data are still the least of the problems in light of the challenges that arise when handling big data. The consistent increase in computing performance is able to keep up with it. But before any value can even be obtained from the data, it must first be merged. Insurance companies have administrated their data in silos far too often and for far too long. And let's not even begin to talk about the quality of the data.

Using interfaces to implement a seamless flow of all the available data—possibly even in real-time—using one's own resources has proven to be too complex. After all, daily operations still need to completed, which remain challenging enough anyway thanks to changes in customer behavior and regulations.

IT architecture concepts that enable general networking and thus an analysis of data quantities often fail particularly it comes to which data repositories should even be used first. Professionals can assist insurance companies with dissolving data silos and merging different sources. However, there is one very essential task that they cannot assume for them: defining visions and goals. This is where disillusionment mostly sets in for insurance companies, because many big data projects fail precisely because of no visions and goals, because merging data in and of itself does not yet constitute any value.

You shouldn't get ahead of yourself

Many companies (and insurance companies are no exception in this case) are like those home builders who have fulfilled their dream of owning their own home primarily due to the expected appreciation in value. And, ultimately, they have their dream house. Perhaps it will also increase in value, but the owner does not receive any of it because the owner does not sell at all.

In the same vein, knowledge and insights continue to lie dormant in the data warehouse, which could help to improve understanding of consumers or form the basis for new pricing models or service offerings.

Integrating data and dissolving data silos is a complex task. However, it is actually just the second step. In order that big data can serve a purpose, you should first ask and answer the question of what will even be done with the data. And people need ideas.

AI can convert "big data" into "smart data".

Instead of "big data", smart data from which new insights can be obtained is needed. However, ideas need to be sought first. During this phase, you must ask questions and hypotheses which are confirmed or rejected using data analyses. Artificial intelligence and machine learning assist with data analyses and help to increase data quality. But it does so only if useful hypothesis are established.

Simply taking a major data repository and purchasing software that uses AI is not enough. Ideas should first be generated across departments: whether it is about identifying new customer segments, processes for detecting fraud, or ideas for new products. Additional discussions in a company must then lead to a portfolio of concepts that are verified with the assistance of data.

"Big data" can become "smart data", but only if people and AI work hand-in-hand. And you also need ideas to do so.