What and Why Data Intelligence Management?
When we think about enterprise-level management, a lot of tools and systems are available for different key positions in organizations:
- ITSM for CIO, CTO, Head of Engineering;
- CRM for Head of Sales, Head of Marketing;
- ERP for COO, CFO, Management.
Albeit CDO, CDMO, Head of Data, Head of Analytics will leverage Data Management, the critical aspects of the data lifecycle won’t be intrinsically covered:
Managing the value of data throughout its various usages.
This is what I’ll cover here as the first entry of this series of blogs about Data Intelligence Management.
Here are the key takeaways in this article:
1. The only way the value of data can grow is with smart data applications.
2. Manage your data application to overcome the limitations of Data Management.
3. ML&AI (Machine Learning & Artificial Intelligence) projects won’t be sustainable without a tailored intelligence management system.
Data is an asset
Like any asset, data takes value along a processing line. Consider oil, for example, its value is only increased with refinements.
And like any other assets, especially nowadays, data can be bought, or resold (e.g. this article lists ~120 data brokers).
Furthermore, as any value generation process works, the better it is, the more value you generate…
But what means “better”? Well, simply, smarter! For which, a definition is presented below:
This makes clear that in our super fast and highly competitive era, you have to differentiate your company on the market by the level of intelligence you put on the data.
Data Intelligence generates the value
After all… data is nothing but the result of applications — I mean data is never naturally:
- Created.
- Modified.
- Deleted.
Hence, you are responsible to increase the value of your data by creating applications. Obviously, those applications will only increase the value of the data by adding a smart twist. The kinds of smart twists have evolved over the last few decades, from ad-hoc business rules to automated AI.
And that is Data Intelligence which aims to increase the value of data with smart applications that can be, potentially, automated, …
Examples of such applications are:
▹ Analytical tools (Excel, …);
▹ Artificial Intelligence models (Tensorflow, …);
▹ BI reports (Tableau, …);
▹ Data Services (API, …).
The smarter, the more automated, and the more trusted your applications are, the better will be your position compared to your competition.
That said, let’s consider their trustworthiness in the coming section.
The responsibility of data applications
To create continuous value with data, all applications must be executed autonomously (say, “in production”).
Therefore automation is key to success.
One of the biggest challenges that come with that is the trustworthiness of the automated results. You might be tempted to call this “data quality”, however, this is reductive, because you would discard the biggest information needed to assess it: its context.
Let’s take an example if you are considering a CSV file generated automatically that will create marketing campaigns from your CRM using an extractor tool that has been parameterized to extract several segments automatically.
You will trust that file as long as you trust the parameters for the extractor are accurate, and that the data in the CRM is satisfying its constraints.
In other terms, the perceived quality of the results depends on the quality of the extraction and the data. This is somewhat disturbing because the data in CRM is outside your perimeter of influence, you have no control, you are not responsible for it. Note that, this is true whether or not silos exist, it is not because you can access some data, or you know it exists, that you are responsible for it. Nevertheless, how you will be using the data is undoubtedly your responsibility?
So you have two choices:
- Accept that if you get garbage, you create garbage (Garbage in, Garbage out).
- Take some of the responsibilities to cover your case, and anticipate issues (Garbage in… no out … or less out).
If you are worried about the quality of your results, and you want to be proactive, you have again two options:
- Scan periodically your results to assess the quality;
- Report metrics from your extractor.
Whilst the first option is part of Data Management (e.g. data monitoring, or data observability), the second option is part of Data Intelligence Management.
In that matter, if you are willing to do “Results Management”, you must first consider Data Intelligence Management.
Data Intelligence Management a must-have for AI/ML
Taking a data management approach to managing your results, or the value generated in your data, has several advantages that won’t be discussed here. Nevertheless, there are crucial limitations in this approach — it doesn’t work for AI/ML projects.
If you want to be smarter than your competition, you are probably already investing in AI/ML. These are the most promising Data Intelligence applications of today, yet you must find a way to manage their results.
The results of an AI/ML are called a model, such model is actually itself an artifact used to generate other results (prediction of churn, the recommendation of an article, …).
Such that the generation of those results is, by design, done without the intervention of a human (somewhat automatically) as shown below:
To trust a model, therefore, you must trust its robustness which is highly dependent on what I call Datastrophes (more on this topic later in the series). This is why, in AI/ML, it is important to know when the model must be recreated.
Consequently, to do AI/ML in an efficient, sustainable, and scalable way Data Intelligence Management is a prerequisite.
What’s next?
Throughout this article, you have learned what data intelligence is and why it is a key element of your data management strategy.
In the coming weeks, I’ll cover how this approach will help your team in saving time and energy considering Datastrophes.
Interested in enhancing the Intelligence of your data? Don’t hesitate to reach me out to or on our website Kensu.io.
