Predictive Analytics; the Future of Business Intelligence
Featured Author - Mukhles Zaman - November 8, 2005

Introduction

The market is witnessing an unprecedented shift in business intelligence (BI), largely because of technological innovation and increasing business needs. The latest shift in the BI market is the move from traditional analytics to predictive analytics. Although predictive analytics belongs to the BI family, it is emerging as a distinct new software sector.

Analytical tools enable greater transparency, and can find and analyze past and present trends, as well as the hidden nature of data. However, past and present insight and trend information are not enough to be competitive in business. Business organizations need to know more about the future, and in particular, about future trends, patterns, and customer behavior in order to understand the market better. To meet this demand, many BI vendors developed predictive analytics to forecast future trends in customer behavior, buying patterns, and who is coming into and leaving the market and why.

Traditional analytical tools claim to have a real 360° view of the enterprise or business, but they analyze only historical data—data about what has already happened. Traditional analytics help gain insight for what was right and what went wrong in decision-making. Today’s tools merely provide rear view analysis. However, one cannot change the past, but one can prepare better for the future and decision makers want to see the predictable future, control it, and take actions today to attain tomorrow’s goals.

What is Predictive Analytics?

Predictive analytics are used to determine the probable future outcome of an event or the likelihood of a situation occurring. It is the branch of data mining concerned with the prediction of future probabilities and trends. Predictive analytics is used to automatically analyze large amounts of data with different variables; it includes clustering, decision trees, market basket analysis, regression modeling, neural nets, genetic algorithms, text mining, hypothesis testing, decision analytics, and more.

The core element of predictive analytics is the predictor, a variable that can be measured for an individual or entity to predict future behavior. For example, a credit card company could consider age, income, credit history, other demographics as predictors when issuing a credit card to determine an applicant’s risk factor.

Multiple predictors are combined into a predictive model, which, when subjected to analysis, can be used to forecast future probabilities with an acceptable level of reliability. In predictive modeling, data is collected, a statistical model is formulated, predictions are made, and the model is validated (or revised) as additional data become available.

Predictive analytics combine business knowledge and statistical analytical techniques to apply with business data to achieve insights. These insights help organizations understand how people behave as customers, buyers, sellers, distributors, etc.

Multiple related predictive models can produce good insights to make strategic company decisions, like where to explore new markets, acquisitions, and retentions; find up-selling and cross-selling opportunities; and discovering areas that can improve security and fraud detection. Predictive analytics indicates not only what to do, but also how and when to do it, and to explain what-if scenarios.

A Microscopic and Telescopic View of Your Data

Predictive analytics employs both a microscopic and telescopic view of data allowing organizations to see and analyze the minute details of a business, and to peer into the future. Traditional BI tools cannot accomplish this functionality. Traditional BI tools work with the assumptions one creates, and then will find if the statistical patterns match those assumptions. Predictive analytics go beyond those assumptions to discover previously unknown data; it then looks for patterns and associations anywhere and everywhere between seemingly disparate information.

Let’s use the example of a credit card company operating a customer loyalty program to describe the application of predictive analytics. Credit card companies try to retain their existing customers through loyalty programs. The challenge is predicting the loss of customer. In an ideal world, a company can look into the future and take appropriate action before customers switch to competitor companies. In this case, one can build a predictive model employing three predictors: frequency of use, personal financial situations, and lower annual percentage rate (APR) offered by competitors. The combination of these predictors creates a predictive model, which works to find patterns and associations.

This predictive model can be applied to customers who are start using their cards less frequently. Predictive analytics would classify these less frequent users differently than the regular users. It would then find the pattern of card usage for this group and predict a probable outcome. The predictive model could identify patterns between card usage; changes in one’s personal financial situation; and the lower APR offered by competitors. In this situation, the predictive analytics model can help the company to identify who are those unsatisfied customers. As a result, company’s can respond in a timely manner to keep those clients loyal by offering them attractive promotional services to sway them away from switching to a competitor. Predictive analytics could also help organizations, such as government agencies, banks, immigration departments, video clubs etc., achieve their business aims by using internal and external data.

On-line books and music stores also take advantage of predictive analytics. Many sites provide additional consumer information based on the type of book one purchased. These additional details are generated by predictive analytics to potentially up-sell customers to other related products and services.

Predictive Analytics and Data Mining

The future of data mining lies in predictive analytics. However, the terms data mining and data extraction are often confused with each other in the market. Data mining is more than data extraction It is the extraction of hidden predictive information from large databases or data warehouses. Data mining, also known as knowledge-discovery in databases, is the practice of automatically searching large stores of data for patterns. To do this, data mining uses computational techniques from statistics and pattern recognition. On the other hand, data extraction is the process of pulling data from one data source and loading them into a targeted database; for example, it pulls data from source or legacy system and loading data into standard database or data warehouse. Thus the critical difference between the two is data mining looks for patterns in data.

A predictive analytical model is built by data mining tools and techniques. Data mining tools extract data by accessing massive databases and then they process the data with advance algorithms to find hidden patterns and predictive information. Though there is an obvious connection between statistics and data mining, because methodologies used in data mining have originated in fields other than statistics.

Data mining sits at the common borders of several domains, including data base management, artificial intelligence, machine learning, pattern recognition, and data visualization. Common data mining techniques include artificial neural networks, decision trees, genetic algorithms, nearest neighbor method, and rule induction.

Major Predictive Analytics Vendors

Some vendors have been in the predictive analytical tools sector for decades; others have recently emerged. This section will briefly discuss the capabilities of key vendors in predictive analytics.

SAS

SAS is one of the leaders in predictive analytics. Though it is a latecomer to BI, SAS started making tools for statistical analysis at least thirty years prior, which has helped it to move into data mining and create predictive analytic tools. Its application, SAS Enterprise Miner, streamlines the entire data mining process from data access to model deployment by supporting all necessary tasks within a single, integrated solution. Delivered as a distributed client-server system, it is well suited for data mining in large organizations. SAS provides financial, forecasting, and statistical analysis tools critical for problem-solving and competitive agility.

SAS is geared towards power users, and is difficult to learn. Additionally, in terms of real-time analytics, building dashboards and scorecards, SAS is a laggard compared to competitors like Cognos, Business Objects, and Hyperion; however, its niche product in data mining and predictive analytics has made it stand out of the crowd.

SPSS

SPSS Inc. is another leader in providing predictive analytics software and solutions. Founded in 1968, SPSS has a long history of creating programs for statistical analysis in social sciences. SPSS today is known more as a predictive analytics software developer than statistical analysis software.

SPSS has played a thought-leadership role in the emergence of predictive analytics, showcasing predictive analytics as an important, distinct segment within the broader business intelligence software sector. SPSS performs almost all general statistical analyses (regression, logistic regression, survival analysis, analysis of variance, factor analysis, and multivariate analysis) and now has a full set of data mining and predictive analytical tools.

Though the program comes in modules, it is necessary to have the SPSS Base System in order to fully benefit from the product. SPSS focuses on ease; thus beginners enjoy it, while power users may quickly outgrow it. SPSS is strong in the area of graphics, and weak in more cutting edge statistical procedures and lacks robust methods and survey methods. The latest SPSS 14.0 release has improved links to third-party data sources and programming languages.

Insightful

Along similar lines is Insightful Corporation, a supplier of software and services for statistical data analysis, data mining of numeric, and text data. It delivers software and solutions for predictive analytics and provides enterprises with scalable data analysis solutions that drive better decisions by revealing patterns, trends, and relationships. Insightful’s S-PLUS 7, is a standard software platform for statistical data analysis and predictive analytics. Designed with an open architecture and flexible interfaces, S-PLUS 7 is an ideal platform for integrating advanced statistical techniques into existing business processes.

Another tool offered by Insightful is Insightful Miner, a data mining tool. Its ability to scale to large data sets in an accessible manner in one of its strengths. Insightful Miner is also a good tool for data import/export, data exploration, and data cleansing tasks, and its reduces dimensionality prior to modeling. While it has powerful reporting and modeling capabilities, it has relatively low levels of automation.

StatSoft Inc.

StatSoft, Inc. is a global provider of analytic software. Its flagship product is Statistica, a suite of analytics software products. Statistica provides comprehensive array of data analysis, data management, data visualization and data mining procedures. Its features include the wide selection of predictive modeling, clustering, classification and exploratory techniques made available in one software platform. Because of its open architecture, it is highly customizable and can be tailored to meet very specific and demanding analysis requirements. Statistica has a relatively easy to use graphical programming user interface, and provides tools for all common data mining tasks; however, its charts are not easily available for the evaluation of neural net models. Statistica Data Miner another solution that offers a collection comprehensive data mining solutions. It is one of two suites that provides a support vector machine (SVM), which provides the framework for modeling learning algorithms.

Knowledge Extractions Engines (KXEN)

Knowledge Extraction Engines (KXEN) is the other vendor that provides a suite that includes SVM. KXEN is a global provider of business analytics software. Its self-named tool, KXEN provides (SVM) and merges the fields of machine learning and statistics.

KXEN Analytic Framework is a suite of predictive and descriptive modeling engines that create analytic models. It places the latest data mining technology within reach of business decision makers and data mining professionals. The key components of KXEN are robust regression, smart segmenter, time series, association rules, support vector machine, consistent coder, sequence coder, model export, and event log.

One can embed the KXEN data mining tool into existing enterprise applications and business processes. No advanced technical knowledge is required to create and deploy models and KXEN is highly accurate data mining tool and it is almost fully automatic. However, one record must be submitted for every entity that must be modeled, and this record must contain a clean data set.

Unica

Affinium Model is Unica’s data mining tool. It is used for response modeling to understand and anticipate customer behavior. Unica is enterprise marketing management (EMM) software vendor and Affinium Model is a core component of the market-leading Affinium EMM software suite.

The software empowers marketing professionals to recognize and predict customer behaviors and preferences—and use that information to develop relevant, profitable, and customer-focused marketing strategies and interactions. The automatic operation of the modeling engine shields the user from many data mining operations that must be manually performed by users of other packages, including a choice of algorithms.

Affinium is an easy to use response modeling product on the market and is suitable for the non-data miner or statistician, who lacks statistical and graphical knowledge. New variables can be derived in the spreadsheet with a rich set of macro functions; however, the solution lacks data exploration tools and data preparation functions.

Angoss Software Corporation

Another leading provider of data mining and predictive analytics tools is Angoss Software Corporation.

Its products provide information on customer behavior and marketing initiatives to help in the development of business strategies. Main products include KnowledgeSTUDIO and KnowledgeSEEKER, which are data mining and predictive analytics tools. The company also offers customized training to its clients, who are primarily in the financial services industry.

Angoss developed industry specific predictive analytics software like Angoss Expands FundGuard, Angoss Telecom Marketing Analytics, and Angoss Claims & Payments Analytics. Apart from financial industry Angoss software is used by telecom, life sciences, and retail organizations.

Fair Isaac Corporation

Along similar lines, Fair Isaac Corporation is the leading provider of credit scoring systems. The firm offers statistics-based predictive tools for the consumer credit industry. Model Builder 2.1 addresses predictive analytics, and is an advanced modeling platform specifically designed to jump-start the predictive modeling process, enabling rapid development, and deployment of predictive models into enterprise-class decision applications. Fair Isaac's analytic and decision-management products and services are used around the world, and include applicant scoring for insurers, and financial risk and database management products for financial concerns.

IBM

Not to be left out, the world’s largest information and technology company, IBM also offers predictive analytics tools. DB2 Intelligent Miner for Data is a predictive analytical tool and can be used to gain new business insights and to harvest valuable business intelligence from enterprise data. Intelligent Miner for Data mines high-volume transaction data generated by point-of-sale, automatic transfer machine (ATM), credit card, call center, or e-commerce activities. It better equips an organization to make insightful decisions, whether the problem is how to develop more precisely targeted marketing campaigns, reduce customer attrition, or increase revenue generated by Internet shopping.

The Intelligent Miner Scoring is built as an extension to the DB2 tool and works directly from the relational database. It accelerates the data mining process, resulting in the ability to make quicker decisions from a host of culled data. Additionally, because D2B Intelligent Miner Scoring is compatible with Oracle databases, companies no longer have to wait for Oracle to incorporate business intelligence capabilities into their database product.

User Recommendations

Depending on an organization’s needs, some predictive analytics tools will be more relevant than others. Each has its strengths and weakness and can be highly industry-and model-specific—the algorithms and models built for one industry are not applicable to other industries. Financial industries, for example, have different models than what are used in manufacturing and research industries.

Selecting the appropriate predictive analytics tools is not a simple task. The following capabilities must be taken into consideration: algorithm richness, degree of automation, scalability, model portability, web enablement, ease of use, and the capability to access large data sets. The more diversified the business, the more functions and unique models are required. Model portability is important even within different business units in the same company. The scalability of the solution and its ability to handle expanded functionality should also be verified and based on a business’ growth.

The tools also have to be tested by the right experts. To understand and interpret predictive analytics results, one has to be knowledgeable about statistical modeling. One should look for the main functions and features of the tool and try to match them with their main requirements, as well as measure the trade off between functionality and cost. For example, some functionalities might be more important for some companies and less important for others.

Buyers should also beware. Although marketing campaigns for predictive analytics solutions claim ”ease of use”, these tools are not for beginners. Users require extensive training and expertise to use the core functionalities of the predictive analytics solutions, such as identifying data, building the predictive model with right predictors, data mining knowledge to align with business strategy etc. Furthermore, predictive analytics automates model building, but does not automate the integration of business processes and knowledge. Thus expertise and training are required to evaluate the best software relevant to an organization’s unique business model.

Nonetheless, if a company has or is willing to attain the expertise required to use predictive analytics it can definitely benefit from the tool. Although most large enterprises use some sort of traditional BI tool or platform, their tools do not provide predictive analytics functionality. Incorporating predictive analytics into an existing BI infrastructure can provide organizations’ a competitive advantage in their industry. Consequently, the integration of BI tools is a key consideration when selecting a predictive analytical tool, as is its integration with key applications such as enterprise resource planning, (ERP), customer resource management (CRM), and supply chain management (SCM) etc. Ultimately, since predictive analytics is currently the only way to analyze and monitor the business trends of the past, present, and future, selecting the right tool can be a key success factor in your BI strategy.

About the author

Mukhles Zaman has more than twenty five years experience in the IT industry specializing in business intelligence (BI), customer relationship management (CRM), project management, database design, and reporting software. He is a leading BI expert and has worked as a senior project manager on IT projects for Fortune 1000 companies in India, the Middle East, US, and Canada. He has also developed call center systems, software architecture, and portfolio management systems. He holds an MA in Economics, and a BA in Economics and Statistics from the University of Dhaka and is an Oracle Certified Professional.

He can be reached at mukhleszaman@yahoo.com.