Authors: Dr. Lourival Carmo Monaco Neto, Postdoctoral Research Associate
Dr. Allan W. Gray, Executive Director and Professor
In this multi-part blog series about data-driven decision making in times of crisis, we have argued that intelligence, not data itself, is the critical component in decision making. We have also pointed out the importance of understanding the critical data needed for a specific decision and the processes involved in collecting, organizing and storing this data. In this article, we focus on what may be considered the most important step in the intelligence cycle: the processing and analysis of data and information.
The analysis step transforms raw data and information into actionable intelligence. The key to this step is taking our collection of facts, figures, statistics and other quantitative data points and organizing and interpreting the data to reveal patterns, tendencies and relationships. As we’ve discussed, the transformation of data and information into actionable intelligence is the heart and soul of good decision making.
Several concepts are fundamental in the process of transforming data into intelligence. The following concepts permeate the various tools and techniques used to achieve this transformation:
- Inductive reasoning: the ability to combine separate pieces of information or specific answers to problems to form general rules or conclusions. This involves the ability to think of possible reasons why things go together, as well as discovering logical explanation for series of events that seem unrelated.
- Deductive reasoning: the ability to apply general rules to specific problems in order to discover a logical resolution. This involves an analysis to determine if the resolution makes sense.
- Pattern recognition: the ability to identify or detect a known pattern (a figure, word or object) that is hidden in other material.
A major point to be observed here is that some of the tools used to collect, organize and store data and information may do a great deal of the heavy lifting when the time comes for analysis. Not only will these tools make this process faster, but they may also jumpstart the transformation. For example, a text mining tool may help you collect textual data about a topic in a document or on a website. This same tool may also be used to analyze pre-collected content, providing, for instance, a count for how many times a word was used and its various contexts.
Often, particularly in research-focused parts of organizations, there is inertia or pressure to use the latest and most complex method for transforming data into intelligence. However, it is critical to maintain a focus on using the tool that is best suited for the job at hand. You wouldn’t use a sledgehammer to tighten a screw, and you probably wouldn’t use tweezers to bring down a wall.
The number of analysis tools and frameworks is practically limitless. We do not intend to present all possible analysis tools, but some examples include:
- Classical analysis strategies and tools:
- BCG Matrix
- GE Business Matrix
- Industry Analysis (Porter’s Five Forces Model)
- Strategic Group Analysis
- SWOT Analysis
- Financial Ratios (ROA, ROE, ROI, etc.)
- Value Chain Analysis
- Scenario Analysis
- Data mining and statistical analysis tools:
- Class/Concept Description: Characterization and Discrimination
- Data Mining for Frequent Patterns
- Outlier Analysis
- Correlation Analysis
- Discriminate Analysis
- Factor Analysis
- Cluster Analysis
- Forecasting/Time Series Analysis
- Classification and Regression for Predictive Analysis
- Visual analysis strategies and tools:
- Tableau
- Qlikview
- FusionCharts
- Plotly
- Sisense
- Power Bi
- Decision modeling tools:
- Business Experiments
- Sentiment Analysis
- Monte Carlo Simulation
- Linear Programming
- Cohort Analysis
We would be remiss to ignore the modern-day role of Big Data in the data to intelligence transformation process. To start, it is important to understand the definition of Big Data:
High-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making.
Additionally (and extremely important to remember), Big Data is not a thing — it is a capability. Therefore, while the dimensions of Big Data — volume, variety and velocity — are important descriptors of what makes data “big”, the more critical dimension of Big Data is analysis, or the capability of transforming all of this data into insights. In fact, one of the most sought-after degrees in higher education today is data analytics, as this set of technical skills is what allows a company to truly leverage its big data to create differentiation in today’s marketplace.
Big Data Dimensions
Source: SONKA and CHENG, 2015.Big Data in agriculture is helping us to better understand the factors that drive what has happened or is currently happening on our farms and in our food and agribusiness firms. This is a powerful tool to provide insights into our decision making and help us identify areas we should research further to learn how we can better predict what may happen in the future. Combining Big Data analysis with what is known as “small data” (conducting scientific research in a systematic way) is the true essence of converting data into intelligence. For example, we have seen real-time Big Data analysis of weather patterns, soil conditions and farm management practices combined with small data associated with disciplined agronomic research improve in-season farm management practices that enhance production, improve plant health, preserve soil nutrients and maintain cleaner water systems.
Regardless of the tool or strategy we decide on to transform our critical data sets, it is imperative that it achieves a simple and important goal: to answer the critical intelligence needs or questions required by the decision maker.
Once the right tool or strategy is employed and data and information are transformed, the core of the intelligence cycle is done, yet far from over. There is no value in analytical efforts if they don’t reach the decision maker in a timely and appropriate manner. Stay tuned for our next blog covering this issue: communicating intelligence.
Transformation of Data into Intelligence
Source: Adapted from STEFANIKOVAA and MASAROVAA, 2013..
