WO2017019078A1

WO2017019078A1 - Providing a probability for a customer interaction

Info

Publication number: WO2017019078A1
Application number: PCT/US2015/042829
Authority: WO
Inventors: Xin Zhang; Julie Ward Drew; Shailendra K. Jain; Debora BIELECKI; Cesar Alberto GALVIS; Keh-Chang HUANG; Rosemeire Aparecida Fasanelli GIORDANO
Original assignee: Hewlett Packard Enterprise Development LP
Current assignee: Hewlett Packard Enterprise Development LP
Priority date: 2015-07-30
Filing date: 2015-07-30
Publication date: 2017-02-02
Anticipated expiration: 2018-01-30

Abstract

An example system provides a probability for a customer interaction. The system includes a network interface to access a plurality of data sources. A modeler predicts a probability of a customer interaction from data collected from the plurality of data sources. A prediction displayer provides the probability predicted for the customer interaction.

Description

PROVIDING A PROBABILITY FOR A CUSTOMER INTERACTION

BACKGROUND

[0001] A contract may be an agreement to provide a service, such as reactive repair service, preventive maintenance, or other services. Improving contract renewal rates may be important to a business.

DESCRIPTION OF THE DRAWINGS

[0002] Certain exemplary embodiments are described in the following detailed description and in reference to the drawings, in which:

[0003] Fig. 1 A is an example of a system for providing a probability for a customer interaction;

[0004] Fig. 1 B is an example system for providing a probability for a customer interaction;

[0005] Fig. 2 is a screenshot of an example of a customer control panel that allows the display of customer information.

[0006] Fig. 3 is a screenshot of an example of a probability of a customer interaction being displayed on the customer control panel;

[0007] Fig. 4 is an example of a modelling process to identify probabilities;

[0008] Fig. 5 is a bar chart of an example of classifying probabilities into different prediction groups based on the value of the probability;

[0009] Fig. 6 is an example of a hierarchy of customer interactions that can be predicted;

[0010] Fig. 7A is an example of a method that can be used to predict a probability of a customer interaction;

[0011] Fig. 7B is a simplified example of the method that can be used to predict a probability of a customer interaction;

[0012] Fig. 8 is a screenshot of an example of a probability of a group of customer interactions being displayed on the customer control panel; [0013] Fig. 9 is a screenshot of an example of a probability of a group of customer interactions being displayed on the customer control panel; and

[0014] Fig. 10 is an example of a non-transitory, computer readable medium that includes code to direct a processor to determine a probability for a customer interaction.

DETAILED DESCRIPTION

[0015] Methods and systems are provided for providing a prediction of the probability that a customer will renew a contract or convert an extended warranty to a contract. For example, a customer may have purchased a service contract to maintain previously purchased equipment and the contract may be renewed each year. Further, the customer may choose to convert an extended warranty, for example, that is expiring, to a service contract. After conversion, the service contract may come up for renewal annually.

[0016] The prediction is based on historical data and factors that contribute to, or inhibit, this probability may be identified. The prediction is presented in the context of a customer control panel that allows the predictions for multiple contracts and customers to be located and displayed.

[0017] The prediction techniques include using and applying statistical models in logistic regression and machine learning techniques, for example, in a decision tree and random forest. Further, the system includes connections to extract the relevant data from various data systems, such as enterprise databases, customer databases, and the like, and to pre-process the raw data into attributes or predictors, which will then feed into the prediction models. The prediction models are run on a computer and then the predictions may be aggregated into account-level views. The prediction accuracy is validated on a quarterly basis by checking the prior predictions against actual business outcomes.

[0018] The prediction of which contracts are less likely to be renewed allows efforts and resources to be focused to increase the likelihood of renewal of the contracts which are at-risk. Currently, for many large companies, there are hundreds of thousands of contracts, expiring every year from a diverse customer base in terms of size, geography, industry segments, and relationship history. This makes identifying individual "good" and "at-risk" contracts a difficult task from a methodology and systems perspective.

[0019] Accordingly, the techniques described here aid the process making the companies personnel more efficient. For example, predicting the probability of renewal of a contract enables sales teams to ensure a high level of renewal confidence and business growth, by providing an advance indictor of renewal, such as High, Medium High, Medium Low, and Low, for each sales document. The information gives the sales person valuable intelligence about the customer account and enhanced visibility of further opportunities, which allows different level of sales efforts for each sales document and customer interaction, helping to optimize the use of sales resources.

[0020] Furthermore, by identifying customer interactions that are at-risk for nonrenewal, sales teams may avoid potential loss of business due to competitors' actions and other market. This may be termed "hot-spot" and "cold-spot"

identification, and may allow sales organizations to identify both business growth opportunities and revenue erosion threats, to formulate strategies, and to set goals and prepare financial projections.

[0021] Examples described herein provide a system-wide visibility of all contracts, utilizing the full range of data sources, thus giving higher levels of prediction accuracy. The prediction output may be presented as part of a customer control panel, or central display showing all of the information for a customer or group of customers, which also enhances the visibility and consistency in communication and reporting across geographies and sales organizations. To improve prediction accuracy, different models may be used for different document types, such as contracts or conversions of extended warranties, and for different dependent variable (y) outcomes, such as binary outcomes, renewal/non-renewal, or multi-class outcomes. Further, each model may be run on different data subsets corresponding to different regions and sub-regions.

[0022] Fig. 1 A is an example of a system 102 for providing a probability for a customer interaction. As used herein, a customer interaction may be a contract renewal, a conversion of an extended warranty to a contract, or any number of other interactions that have historical data. The system 102 may be a server, a desktop, a cloud computing system, a virtual machine, or any number of other computing devices.

[0023] The system 102 may include a processor 104 that is configured to execute stored instructions, as well as a memory device 106 that stores instructions that are executable by the processor 104. The processor 104 can be a single core processor, a dual-core processor, a multi-core processor, a computing cluster, a virtual processor, or the like. The processor 104 may be coupled to the memory device 106 by a bus 108 where the bus 108 may be a communication system that transfers data between various components of the system 102. In embodiments, the bus 108 may be a PCI, ISA, PCI-Express, HyperT ran sport®, NuBus, or the like.

[0024] The memory device 106 can include random access memory (RAM), e.g., SRAM, DRAM, zero capacitor RAM, eDRAM, EDO RAM, DDR RAM, RRAM, PRAM, read only memory (ROM), e.g., Mask ROM, PROM, EPROM, EEPROM, flash memory, or any other suitable memory systems. The memory device 106 may be shared among a group of processors 104, or may be specifically allocated to a single processor 104.

[0025] The system 102 may also include a storage device 1 10. The storage device 1 10 may include any number of volatile or non-volatile storage devices, such as any of the RAMs above, in addition to a static RAM, a non-volatile RAM

(NVRAM), a solid-state drive, a hard drive, a flash drive, an array of drives, or any combinations thereof.

[0026] The processor 104 may be connected through the bus 108 to a human machine interface (HMI) 1 12 configured to couple the system 102 to one or more I/O devices. The I/O devices may include an input device 1 14, such as a keyboard, a mouse, or a pointing device, wherein the pointing device may include a touchpad or a touchscreen, among others. The HMI 1 12 may include a display driver to couple the system 102 to a display device 1 16. The display device 1 16 may include a display screen, a computer monitor, a television, or a projector, among others. In some examples, the HMI 1 12, display device 1 16, and input device 1 14, may be omitted, for example, if the system is part of a server. [0027] A network interface controller (NIC) 1 18 may also be linked to the processor 104. The NIC 133 may link the system 102 to a computing cloud 120, such as data sources 122 connected over a local area network (LAN), a wide area network (WAN), or the Internet. The computing cloud 120 may link one or more remote user devices 124 to the system 102, allowing users to query the system 102 for probabilities of customer interactions.

[0028] The storage device 1 10 may include a number of modules configured to provide the system 102 with the customer control panel and predictive functionality. For example, a data retriever 126 may use the NIC 1 18 to access the data sources 122 in the cloud 120. The data retriever 126 may also extract and preprocess the data, for example, pulling from a number of different enterprise databases.

[0029] A user interface 128 may provide a customer control panel display to a user device 124 or on the display 1 16. As described herein, the customer control panel may allow a user, such as a sales representative to access a display of relevant customer interactions for a region, time period, or type of customer interaction, among others. The user interface 128 may also allow queries from other devices. For example, a user may send a text query in through an SMS service 130, such as a mobile phone provider. The user interface 128 may format a text reply and return it to the user via the SMS service 130.

[0030] A modeler 132 may use various modeling techniques, as described herein, to model the probability of a customer interaction occurring. The modeling may be based on learning techniques, such as neural networks or genetic algorithms, among others, that use training sets to teach the modeler 132. In some examples, the modeling may be based on iterative error minimization techniques, such as least squares regression, among others. Any combinations of these techniques or other modeling techniques may be used.

[0031] Once the modeler 132 has generated probabilities for customer interactions, for example, at the level of the individual customer interaction, an aggregator 134 may aggregate the probabilities and amounts into higher level ensembles. For example, as a customer may have multiple customer interactions, the aggregator 134 may pool the probabilities for all of these into a single value that may indicate a relationship with the customer. Further, the aggregator 134 may indicate the value of customer interactions as a function of the probabilities. The modeler 132 may include validation functions that compare outcomes over a period to the predicted outcomes, allowing the models to be improved or new factors to be identified.

[0032] A prediction displayer 136 may display the values for the probabilities on a region of the customer control panel. The values may be displayed as aggregated predictions or the probabilities for individual customer interactions may be displayed.

[0033] The block diagram of Fig. 1 A is not intended to indicate that the system 102 is to include all of the components shown in Fig. 1 A. For example, the HMI 1 12, display 1 16, and input device 1 14 may not be used in some implementations, as described in the example in Fig. 1 B. Further, any number of additional components may be included within the system 102, depending on the details of the specific implementation. For example, the system 102 may include an updater to update the model at each run.

[0034] Fig. 1 B is an example system for providing a probability for a customer interaction. Like numbers are as described with respect to Fig. 1 A. In the example shown in Fig. 1 B, the system 102 includes a few core elements. These are a network interface 1 18 to access a plurality of data sources, a modeler 132 to predict a probability of a customer interaction from data collected from the plurality of data sources, and a prediction displayer to provide the probability predicted for the customer interaction.

[0035] Fig. 2 is a screenshot 200 of an example of a customer control panel 200 that allows the display of customer information. The customer control panel 200 may have any number of selection regions to assist in locating and accessing information about a particular customer, group of customers, geographic region, or value of contracts, among many others. For example, a group of links 202 may allow a user to quickly access regions of the site. Further regions of the site may allow a user to access customer data by global accounts 204, local accounts 206, regions 208, or countries 210, among others.

[0036] Fig. 3 is a screenshot 300 of an example of a probability of a customer interaction being displayed on the customer control panel. In this screenshot 300, a specific customer 302 has been selected for display. Information about the business relationship and customer interactions, such as total sales 304 and existing contracts 306 may be displayed, among others. A region 308 of the control panel may display probabilities for particular customer interactions. In this case, the region 308 is indicating contracts and other customer interactions having a "lower" repurchase indication and, thus, needing attention from a sales representative.

[0037] Fig. 4 is an example of a modelling process 400 to identify probabilities. The process 400 begins at block 402 with the acquisition of historical data 404 and business inputs 406. The historical data 404 includes items such as prior sales, prior contract renewals, age of contracts, and the like. The business inputs 406 may include length of the relationships, subjective factors from sales personnel, competitive forces and the like. As discussed herein, the input may be preprocessed for use.

[0038] Once the input has been obtained, modelling may be performed at block 408. As described herein, any number of statistical algorithms 410 may be used, including least squares error minimization, neural networks, or genetic algorithms, among many others.

[0039] As part of the interactive process of the modelling, at block 412, the working principle may be identified or adjusted. This may include, for example, studying 414 the historical data in light of the predictions. This may lead to an identification 416 of the attributes that influence the customer interaction. Further, the attributes maybe analyzed 418 to analyze the impact of each on the probability of the customer interaction. The cumulative impact of the attributes on each customer interaction may be determined 420, and the probability score for each customer interaction may be calculated 422 and classified, for example, into ranges. The process then returns to 402 to iteratively strengthen the relationships.

[0040] Fig. 5 is a bar chart 500 of an example of classifying probabilities into different prediction groups based on the value of the probability. The prediction group is a label that summarizes the probability: high (H) 502, low (L) 504, and intermediate probabilities, medium high (MH) 506 and medium low (ML) 508. More or less granular classification may be used.

[0041] For renewal of contracts, the renewal prediction group may be determined by the absolute probability value, although this classification could be used for any type of customer interaction. For example, high 502 may be chosen when renewal probability is greater than 0.83. Medium high 506 may be chosen when probability is between 0.63 and 0.83. Medium low 508 may be chosen when probability is between 0.43 and 0.63. Low 504 may be chosen when probability is less than or equal to 0.43. Different probability thresholds may be used for classification into renewal prediction groups.

[0042] For conversion of extended warranties to contracts, the conversion prediction group is determined by the percentile of the predicted conversion probability value among all the extended warranties in a country, which is a relative measure. The most likely 30% of the extended warranties to convert to a service contract may be labeled high 502, the next 20% may be labeled medium high 506, the following 20% may be labeled medium low 508, and the bottom 30%, e.g., least likely to convert, may be labeled low 504. This is illustrated in the bar chart in Fig. 5.

[0043] For extended warranties, since the H/MH/ML/L labels are determined by comparing the conversion probabilities within a country, it is possible that an extended warranty that is labeled High 502 in one country might have a lower probability than an extended warranty that is labeled Medium High 506 in another country. However, the use of the H/MH/ML/L labels gives country sales teams a clear sense of the conversion outlook, which may guides the allocation of sales resources.

[0044] The verbal classifications account for the nature of the probability scores, for example, by allowing a sales representative to focus attention on the customer interactions that need the most attention rather than spending time interpreting the relative value of the numbers. Further, the bar chart 500 shows that the customer interactions can be aggregated and compared over time, such as the four quarters in the bar chart 500, to identify or learn how the customer interactions are changing. In combinations with value indications, the sales representatives may focus their efforts on the customer interactions that are going to provide the highest value. These customer interactions may be individual contracts or groups of contracts.

[0045] Fig. 6 is an example of a hierarchy 600 of customer interactions that can be predicted. The customer interactions may be examined at hierarchical levels, for example, starting with a worldwide (WW) summary 602. Below that, particular sales regions, such as the continental regions 604 shown in Fig. 6 may be broken out. Sales in each of the countries 606 in the continental regions 604 can also be examined. In each of the countries 606, particular customer accounts 608 can be analyzed for particular customer interactions.

[0046] Each customer interaction may be uniquely identified by a "Sales

Document Number" or "sales doc" 610 for short. The customer interactions may be divided out by type 612, such as contracts or extended warranties, among others. A sales doc 610 can be further divided into multiple items 614. These items 614 may include any number of different customer interactions, such as service packages specifying the service level agreements, e.g., availability and response time terms, or goods packages describing the hardware under coverage.

[0047] Multiple sales docs 610 can belong to the same customer accounts 608. A large customer account 608 may have thousands of sales docs 610, while a small customer account 608 may have only one sales doc 610. Multiple customer accounts 608 may then be aggregated into a sales territory, such as East, West, South, Central, or a sales organization, such as State and Local Government, Federal Government, Higher Education, etc. Further, the results may be aggregated and reported, for example, for a continental region 604.

[0048] The basic unit for calculating a prediction is a sales doc 610. Thus, a prediction is computed for the probability of renewal or conversion of a sales doc 610. The predictions may be aggregated to the account level, for example, through a dollar-weighted averaging method described herein. Account level predictions can be further rolled up into country, region and world-wide levels through the same dollar-weighted averaging method.

[0049] Fig. 7A is an example of a method 700 that can be used to predict a probability of a customer interaction. The method 700 begins at block 702 with the retrieval of data from various data systems and databases. The data may include information such as orders, contracts, shipments, entitlements, support incidents, and the like, pulled from sources such as enterprise databases, customer databases, and the like.

[0050] The sales docs 610, as referred to in Fig. 6, may include information such as sales document type, contract expiration date, contract duration, multi-year status; annualized dollar value of contract; and other information including special pricing, sales organization and the like. Further data for the customer may include product lines owned by the customer, products owned by the customer, the age of products owned by the customer, and the like. Many other items can be used as input for the prediction, such as geography, the customer's competition, and the like. It can be understood that these items are merely examples, and many other items can be used for the prediction.

[0051] The items may be predictive in any number of ways, both known before the modeling and identified by the modeling. For example, contract duration is indicative of the renewal or conversion likelihood, as very short and very long contracts have lower renewal rates than contracts with more intermediate terms, such as 1 -3 years of duration. The contract dollar value, which reflects the value of the hardware under coverage and the service level, may also be indicative of the importance and criticality of the system to the customer, and therefore indicative of the renewal probability. Other elements, such as the age of the items purchase, the prior resolution of issues, and the effort placed into accounts by sales

representatives may all have effects on the probability.

[0052] The data elements may form the predictors, or factors, which feed the prediction models. Although the specific form of the prediction model, such as a logistic model, a regression model, a decision tree, or a random forest model may be selected, the set of predictors may remain the same. The subsequent prediction modeling process will perform a variable selection step to remove data elements which do not provide significant predictive value or are redundant, e.g., highly correlated with other data elements.

[0053] At block 702 the data is preprocessed, for example, by joining on sales document number, and by creating derived predictors, such as the number of contracts or enhanced warranties belonging to the same customer, and the like. The raw data coming from the databases, such as an SAP system or an Enterprise Data Warehouse (EDW) system go through a processing step before being fed to the prediction algorithm. First, data from various databases are linked (joined) through the Sales Document (Sales Doc) Number, to provide a unique identifier for customer interactions, such as contracts and extended warranties. Customers may be identified and linked through other identifications, such as contract numbers and customer IDs.

[0054] The data preparation may also include the computation of the time-interval attributes (variables or predictors), or time dynamic attributes, such as the number of extended warranties converted to service contracts in the past 365 days of a customer (such as the Sold-To Party or Ship-To Party). For each time dynamic attribute, two timing elements may be defined, the reference time point, such as the expiration date of an extended warranty, and the time window length, such as 365 days, 120 days, or 60 days. Hence, a time dynamic attribute is also a "moving time window" attribute. A prediction model may then be trained using both time-invariant attributes, such as the customer's geography and industry segment, which typically does not change over time, and time dynamic attributes described above.

[0055] The modelling may include, for example, a decision tree technique or a random forest technique. A decision tree uses a tree and leaf graph of decisions and possible consequences, including possible event outcomes, resource costs, and utility. The random forest technique creates an ensemble of decision trees, and may provide classification, mean probability, or both.

[0056] Both of these techniques produce metrics that indicate the importance of metrics used. However, these metrics may be at the overall model aggregate level, for example, for customer interactions in the model's training data set. This may not indicate the variable importance for a specific customer interaction. For example, for one customer interaction, geography may be the most important factor contributing to the probability. However, for another customer interaction, an annualized dollar value may be the most important factor.

[0057] The ability to identify a list of important factors contributing to the customer interaction may explain why the specific customer interaction has a predicted probability value. This may allow a determination to be made as to what factors have made a customer interaction's probability high or low.

[0058] For example, for a customer interaction model, a logistic regression involving j = 1 , 2, n terms, may be used, which may be written as the following equation: ^l0g(^ ~) = β + βΐΧΐ + p2*2 + ^■■■ + βη*τ

[0059] In the equation above, Xj is a column vector containing each customer interactions values for variable j. After the model is fitted and coefficients

determined, an individual customer interaction i, and compute the values using the following equation:

wi,j = ^XU ^{~ x}j)'j = ¹' ²' - > ⁿ

[0060] In this equation, x_j represents the average value of the j-th column in the dataset, i.e., the average over the elements of Xj. The term x_i:j represents the i-th element of the vector Xj. The order W_i:j may then be ranked for j = 1 , 2, n. The Xj that gives the highest value W _j is the factor that contributes most to the probability. Similarly, the Xj which gives the lowest value W _j is the factor that contributes least to the probability. Typically, the lowest W _j is negative, indicating that the probability would have been higher if this term had not existed. Thus, this may be termed an inhibiting factor. The factors may be ordered by their influence on the probability to assist personnel is obtaining a beneficial outcome. For example, the top three contributing factors and top three inhibiting factors for each customer interaction, i, may be determined to allow actions to be taken to improve the probabilities.

[0061] It may be noted that the right-hand-side terms j = 1 , 2, n of the equation are additive. Thus, it will work for both numeric values, such as dollar amounts, and indicator variables, e.g., that take 0 or 1 values. The coefficients themselves, βι , β2, .. . , β_η may not indicate the importance of a term, since the scaling or the "unit of measurement" for the corresponding Xj value also influences the coefficient ft. For example, when Xj is revenue measured in millions of dollars, the coefficient ft will also re-scale by a million.

[0062] At block 706 separate prediction models may be run for each of the types of customer interactions. For example, several prediction models may be created by distinguishing contracts from extended warranties, although the contract renewal model and the extended warranty conversion model share many of the predictors. The use of different models may be reflective of the difference in factors that may influence these customer interactions, such as age of the equipment, previous relationship, and the like. Each model can use predictors describing the other type of customer interaction. For example, a contract renewal model may use a predictor describing the number of extended warranties at the customer account, and the extended warranty conversion model may use a predictor describing the number of contracts at the customer account.

[0063] The outcome is generally not a binary classification, e.g., renewal or nonrenewal, or conversion or non-conversion. For example, a multi-class outcome may be used in which the non-renewal outcome is further divided into sub-categories that indicate the reason for the loss. The use of a multi class renewal, e.g., six class including five for poor outcomes in the customer interaction and one class for a good outcome, allows further prediction and identification of the type of non-renewal risk.

[0064] Many more models may be used than simply the type of contract renewal, including for example, sub-models may be based on a sub-region that the customer interaction is located in, such as Canada, the United States, Latin America, Eastern Europe, Western Europe, and the like. The sub-models allow the generation of more accurate models by taking advantage of data that is available only within some regions and sub-regions.

[0065] The models may be rerun at a regular interval, such as quarterly, using the latest data. The history length for obtaining predictors may be selected to give adequate accuracy without substantial overhead, for example, covering the past 12 quarters (3 years), past 16 quarters (4 years), past 20 quarters (5 years), and the like. Similarly, the prediction window may be chosen to be useful while remaining within error bounds, for example, a future 5 or 6 quarters. Predicting the probabilities for a sales doc expiring in the next 5 quarters, provides a full fiscal year's visibility into the future to facilitate fiscal year financial planning. The prediction models are run in the middle of the first month of a quarter, thus a delay of a fraction of a quarter may be removed by including at least five quarters. [0066] At block 708, the predicted probabilities may be aggregated into account level assessments to identify cold spots, e.g., points where extra attention from a sales representative may be helpful. For each sales doc that has an expiration date in the next 5 quarters and a renewal status that indicates neither won nor lost, the prediction output includes the predicted probability of a customer interaction, such as a renewal of a contract or a conversion of an extended warranty. The predicted probability is a number between zero and 1 such as 0.84 or 0.08. The prediction also includes the prediction group, as discussed with respect to Fig. 5.

[0067] At block 708, the predictions for the individual sales docs may be aggregated to the customer account level. A dollar-value weighted averaging method may be used to aggregate multiple probabilities to provide an account level probability. For example, if an account has k sales docs, such as contracts, each sales doc having a probability, pj, and an annualized dollar value d then the weighted average probability for the account, p_c, may be calculated as:

Pc = (di* i + d₂*p₂ + ... + d_k*p_k)/(di + d₂ + ... + d_k)

Thus, sales docs that have a higher dollar value will have a greater influence on the account level probability. A prediction group label (High, Medium High, Medium Low, and Low) may be applied at the account level as well. The same dollar weighted averaging method can be repeatedly applied to even higher levels of aggregation, such as country (or sales organization), sub-region, region, and world-wide.

[0068] Using the account level probability and an account level annualized dollar value, "account value", a grid of sales docs by renewal prediction group and by account dollar value, may be displayed on the customer control panel. Those accounts which have a high dollar value but a low renewal probability are worthy of greater attention, hence we call them "cold-spot" accounts, as they represent a significant risk of loss in potential revenue.

[0069] A number of additional account level metrics that indicate risk may be calculated for display on the customer control panel. For example, the number of sales docs that have a renewal probability that is below a pre-specified threshold value. Further, a value at risk may be calculated. Other values that may be calculated include a sum of dollar values of the sales docs that have a renewal probability below a threshold. The value potential of accounts at risk may be calculated as the sum of (dollar values ^* max(0,threshold - renewal probability)), which represents the potential gain in dollar value of at-risk contracts, e.g., a renewal probability below the threshold, if the renewal probability of these may be raised to the threshold.

[0070] At block 710, the prediction may be provided to the user. This may be done through the customer control panel as described with respect to Figs. 2, 3, 8 and 9.

[0071] At block 712, the accuracy of the previous predictions may be evaluated by comparing with the recent quarter's actual renewal/loss outcomes. Various metrics may be used to determine the accuracy the prediction accuracy against the actual renewals and conversions, which may occur about a quarter after the models have been run. The focus may be on business evaluation of prediction accuracy, rather than on the "internal" goodness-of-fit of the models.

[0072] Examples of three metrics that may be used for the evaluation are relative odds by probability group (H/MH/ML/L), e.g., the number of sales docs in the high category that have renewed or the number of sales docs in the low category that have renewed. If the model is accurately predicting the customer interactions, most of the sales docs that were labeled high will have been renewed, while very few of the sales docs that were labeled low will have been renewed.

[0073] Further, the difference in probability between renewed and lost contracts may be determined. For the sales docs that have renewed, the average probability that was predicted is calculated. For the sales docs that were lost, the average probability that was predicted is also calculated. If the model is accurately predicting the customer interactions, the sales docs that have renewed will have an average probability that is close to 1 . The sales docs that have been lost may have an average probability an average probability that is close to 0, e.g., less than about 0.3. Further, a capture curve may be generated and objective measured may be applied. For example, if the predicted probabilities are ranked from the highest to the lowest, one measure of the model accuracy could be the number of the top 100 sales docs, based on our predicted probabilities, that have renewed. Other measures may be the number of the top 200 sales docs that have renewed or the number of the top 300 sales docs that have renewed.

[0074] At block 714, the model may be updated by incorporating new predictors formed from new data elements, and removing outdated data elements. This may be used to improve the prediction accuracy by incorporating new data sources, new data elements, and by adopting new modeling methods and algorithms.

[0075] A particular challenge in maintaining a prediction model is to deal with the "new" definitions of data elements, such as new product lines and codes, new organizations, new sales areas, new customer industry segments, even new countries and regions. Updating the model at each cycle recognizes that these new or "unseen" elements, from the historical data perspective, may be commonplace occurrences that continually arise in every modeling run.

[0076] New "levels", such as a new sales organization or region, may be included in the prediction model, as leaving out a new level causes the prediction for any sales doc involving the new level to not be produced. Further, if a new level appears in both training data set and prediction data set, and with a sufficiently high frequency count (sample size), then the new level may be added as a new variable to the model. However, if the new level appears only in the prediction data set, such as a new product line, but not in the training data set, or if the new level has a low frequency count, the new level may be merged into an existing level which is closest in characteristics to the new level. Over time, the new level will become more frequent in the data sets, and will make its way into the model as an independent variable.

[0077] When a new data column, representing another factor, becomes available, the new column may be included as a predictor. The variable selection process may then be used to see if the new predictor from the new column is selected into the final model.

[0078] Similar approaches apply to "old" levels and "old" data columns. As an old product line is gradually phased out, its appearance frequency in the dataset will decrease. The variable selection step may then eventually drop out the old level, or the old data column. [0079] Fig. 7B is a simplified example of the method 700 that can be used to predict a probability of a customer interaction. Like numbered items are as described with respect to Fig. 7A. This simplified version of the method may be used in some examples as the core actions for the prediction.

[0080] Fig. 8 is a screenshot of an example of a probability of an aggregated group of customer interactions being displayed on the customer control panel. The company 802 is selected using the customer control panel. A first region 804 may show renewal opportunities for an ensemble of the customer interactions, for example, in dollar values. A second region 806 may provide the renewal prediction groups for an ensemble of the customer interactions. The probability may also be displayed for individual customer interactions as discussed with respect to Fig. 9

[0081] Fig. 9 is a screenshot 900 of an example of a probability of a group of customer interactions being displayed on the customer control panel. In this screen shot, a region 902 shows a list of individual customer interactions for a single customer. The prediction group 904 for each of the customer interactions. In this example, all of the customer interactions are in the medium high chance of renewal. The customer control panel may also show the annualized value of each of the customer interactions.

[0082] Fig. 10 is an example of a non-transitory, computer readable medium 1000 that includes code to direct a processor to determine a probability for a customer interaction. The non-transitory, computer readable medium 1000 is coupled to a bus 1004 and to a processor 1006 via the bus 1004. The non-transitory, computer readable medium 1000 may include a number of code modules. For example, a first code module 1008 may direct the processor 1006 to retrieve data from various sources, as described herein. A second code module 1010 may direct the processor 1006 to preprocess the data to form predictors. A third code module 1012 may direct the processor 1006 to determine the probability of a customer interaction. A fourth code module 1014 may display the probability, for example, on a customer control panel, as described herein.

[0083] While the present techniques may be susceptible to various modifications and alternative forms, the exemplary examples discussed above have been shown only by way of example. It is to be understood that the technique is not intended to be limited to the particular examples disclosed herein. Indeed, the present techniques include all alternatives, modifications, and equivalents falling within the scope of the present techniques.

Claims

CLAIMS What is claimed is:

1. A system for providing a probability for a customer interaction, comprising:

a network interface to access a plurality of data sources;

a modeler to predict the probability of the customer interaction from data collected from the plurality of data sources; and

a prediction displayer to provide the probability predicted for the customer interaction.

2. The system of claim 1 , comprising an application on a portable device to display the probability predicted for the customer interaction.

3. The system of claim 1 , comprising a display to show the probability predicted for the customer interaction in a region on the display.

4. The system of claim 3, comprising a customer control panel on the display.

5. The system of claim 4, comprising a customer selection region on the customer control panel, wherein the customer selection region allows a selection of customers.

6. The system of claim 1 , wherein the customer interaction comprises a contract renewal.

7. The system of claim 1 , wherein the customer interaction comprises a conversion of an extended warranty into a service contract.

8. A method for providing a probability for a customer interaction, comprising: retrieving data from a plurality of data sources;

preprocessing the data to create factors for a prediction model;

determining a probability of a customer interaction from a prediction model; and

providing the probability of the customer interaction to a user.

9. The method of claim 8, comprising:

determining separate probabilities for the customer interaction from individual prediction models for a plurality of data points; and

aggregating the separate probabilities from the individual prediction models to determine the probability of the customer interaction.

10. The method of claim 8, comprising ordering the factors by their effects on the probability of the customer interaction.

1 1 . The method of claim 8, comprising:

evaluating an accuracy of the probability of the customer interaction; and updating the models.

12. The method of claim 8, wherein providing the probability of the customer interaction comprises displaying a value of the probability in a region of a display, wherein the display comprises a customer control panel.

13. The method of claim 8, wherein preprocessing the data comprises: joining data sets;

cleaning up the data; and

creating time-interval predictors.

14. A non-transitory, machine readable medium comprising code configured to direct a processor to:

retrieve data from a plurality of data sources;

preprocess the data; determine a probability of a customer interaction from a prediction model; and display the probability of the customer interaction on a customer control panel.

15. The non-transitory, computer readable medium of claim 14, comprising code configured to:

determine separate probabilities for the customer interaction from individual prediction models for a plurality of data points; and

aggregate the separate probabilities from the individual prediction models to determine the probability of the customer interaction.