CA3081276A1 - Systems and methods for generating and adjusting recommendations provided on a user interface - Google Patents

Abstract

A recommendation system includes at least one processor and a memory coupled to the at least one processor. The memory stores a profile database, a historical recommendation database, a calculator database, and instructions that, upon execution, cause the at least one processor to, in response to receiving a recommendation request for a requesting user identifier via the web portal, obtain a first profile of the requesting user identifier from the profile database.
The instructions include, from a set of groups, identifying a first group of the set of groups corresponding to the requesting user identifier based on the first profile and determining a bounded range based on (i) the first group and (ii) the first profile. The instructions include selecting a calculator version, based on the bounded range, estimating the initial recommendation using the selected calculator version, and transmitting the initial recommendation to the web portal for display.

Description

SYSTEMS AND METHODS FOR GENERATING AND ADJUSTING
RECOMMENDATIONS PROVIDED ON A USER INTERFACE
FIELD
[0001] The present disclosure relates to user interfaces and more particularly to systems and methods for generating and adjusting recommendations.
BACKGROUND

[0002] Significant amounts of information can be considered to provide a user with a more desirable margin rate or interest rate when engaged in a financial relationship with an entity. Many user relationships can be improved by contacting users with improved rates for the user's account, whether the account supplies the entity with a certain margin rate, commission rate, loan rate, etc.
Further, considering all information related to the user's account is important to the entity to avoid offering or accepting unreasonable rates. However, considering all the necessary information is time consuming when engaged in a conversation with a user as well as when attempting to improve a user relationship by periodically reviewing user accounts.

[0003] The background description provided here is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in this background section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present disclosure.
SUMMARY

[0004] A recommendation system includes at least one processor and a memory coupled to the at least one processor. The memory stores a profile database including a plurality of profiles corresponding to a plurality of user identifiers.
Each profile of the plurality of profiles includes a set of parameters for a Date Recue/Date Received 2020-05-11 corresponding user identifier. The memory stores a historical recommendation database including previously generated recommendations transmitted to a web portal. The memory stores a calculator database including a plurality of calculator versions. The historical recommendation database includes, for each user identifier, a corresponding calculator version identifier used to generate an initial recommendation.

[0005] The memory stores instructions that, upon execution, cause the at least one processor to, in response to receiving a recommendation request for a requesting user identifier via the web portal, obtain a first profile of the requesting user identifier from the profile database. The instructions further cause the at least one processor to, from a set of groups, identify a first group of the set of groups corresponding to the requesting user identifier based on the first profile.
Each profile of the plurality of profiles is associated with a corresponding group of the set of groups based on a similarity.

[0006] The instructions further cause the at least one processor to determine a bounded range based on (i) the first group and (ii) the first profile and select a calculator version from the plurality of calculator versions. The instructions further cause the at least one processor to, based on the bounded range, estimate the initial recommendation using the selected calculator version and transmit the initial recommendation to the web portal for display.

[0007] In other features, the instructions, upon execution, cause the at least one processor to, in response to receiving a response for a responding user identifier via the web portal, update the first profile with a most recent recommendation in response to the response indicating an acceptance. The instructions further cause the at least one processor to generate a subsequent recommendation in response to the response indicating a denial. In other features, the instructions, upon execution, cause the at least one processor to, in response to the response indicating the denial, obtain a first user entry from the historical recommendation database. The instructions further cause the at least one processor to generate the Date Recue/Date Received 2020-05-11 subsequent recommendation based on parameters of the most recent recommendation stored in the first user entry.

[0008] In other features, the instructions, upon execution, cause the at least one processor to, prior to transmitting the initial recommendation, adjust the initial recommendation in response to the recommendation request including a desired recommendation. In other features, the instructions, upon execution, cause the at least one processor to, prior to transmitting the initial recommendation, for each user of the first group, obtain a corresponding entry from the historical recommendation database. The instructions further cause the at least one processor to determine a likelihood of acceptance of the initial recommendation based on the obtained entries and adjust the initial recommendation according to the likelihood of acceptance.

[0009] In other features, the set of parameters for each user profile includes (i) a present margin rate, (ii) a margin, (iii) a profitability score, (iv) a risk score, (v) an amount of revenue, (vi) a margin percentage of revenue, and (vii) a risk relative to revenue. In other features, the similarity is determined using a clustering algorithm to identity and group similar profiles. In other features, each group of the set of groups is associated with a respective calculator version. In other features, each calculator version of the plurality of calculator versions is marked as testing or validated.

[0010] In other features, the instructions, upon execution, cause the at least one processor to, after a predetermined testing period has elapsed, select each calculator version of the plurality of calculator versions marked as testing.
The instructions further cause the at least one processor to calculate, for each selected calculator version, a performance metric over the predetermined testing period and identify a highest performance metric of the calculated performance metrics.
The instructions further cause the at least one processor to mark a corresponding calculator version that corresponds to the highest performance metric as validated.

Date Recue/Date Received 2020-05-11

[0011] A recommendation method includes, in response to receiving a recommendation request for a requesting user identifier via a web portal, obtaining a first profile of the requesting user identifier from a profile database.
The profile database stores a plurality of profiles corresponding to a plurality of user identifiers. Each profile of the plurality of profiles includes a set of parameters for a corresponding user identifier. The method further includes, from a set of groups, identifying a first group of the set of groups corresponding to the requesting user identifier based on the first profile. Each profile of the plurality of profiles is associated with a corresponding group of the set of groups based on a similarity.

[0012] The method further includes determining a bounded range based on (i) the first group and (ii) the first profile. The method further includes selecting a calculator version from a plurality of calculator versions. A calculator database stores the plurality of calculator versions. A historical recommendation database stores, for each user identifier, a corresponding calculator version identifier used to generate an initial recommendation. The method further includes, based on the bounded range, estimating the initial recommendation using the selected calculator version and transmitting the initial recommendation to the web portal for display.

[0013] In other features, the method includes, in response to receiving a response for a responding user identifier via the web portal, updating the first profile with a most recent recommendation in response to the response indicating an acceptance. The method further includes generating a subsequent recommendation in response to the response indicating a denial. In other features, the method includes, in response to the response indicating the denial, obtaining a first user entry from the historical recommendation database. The method further includes generating the subsequent recommendation based on parameters of the most recent recommendation stored in the first user entry.

[0014] In other features, the method includes, prior to transmitting the initial recommendation, adjusting the initial recommendation in response to the Date Recue/Date Received 2020-05-11 recommendation request including a desired recommendation. In other features, the method includes, prior to transmitting the initial recommendation, for each user of the first group, obtaining a corresponding entry from the historical recommendation database. The method further includes determining a likelihood of acceptance of the initial recommendation based on the obtained entries. The method further includes adjusting the initial recommendation according to the likelihood of acceptance.

[0015] In other features, the set of parameters for each user profile includes (i) a present margin rate, (ii) a margin, (iii) a profitability score, (iv) a risk score, (v) an amount of revenue, (vi) a margin percentage of revenue, and (vii) a risk relative to revenue. In other features, the similarity is determined using a clustering algorithm to identity and group similar profiles. In other features, each group of the set of groups is associated with a respective calculator version. In other features, each calculator version of the plurality of calculator versions is marked as testing or validated.

[0016] In other features, the method includes, after a predetermined testing period has elapsed, selecting each calculator version of the plurality of calculator versions marked as testing. The method further includes calculating, for each selected calculator version, a performance metric over the predetermined testing period. The method further includes identifying a highest performance metric of the calculated performance metrics. The method further includes marking a corresponding calculator version that corresponds to the highest performance metric as validated.

[0017] Further areas of applicability of the present disclosure will become apparent from the detailed description, the claims, and the drawings. The detailed description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the disclosure.
Date Recue/Date Received 2020-05-11 BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The present disclosure will become more fully understood from the detailed description and the accompanying drawings.

[0019] FIG. 1 is a high-level block diagram of an example network communication system including a recommendation system according to the principles of the present disclosure.

[0020] FIG. 2 is an example user interface displaying input to be received for generating and adjusting recommendations.

[0021] FIG. 3 is an example user interface displaying generated recommendations.

[0022] FIG. 4 is a functional block diagram of an example recommendation device.

[0023] FIG. 5 is a flowchart depicting example operation of generating and adjusting recommendations based on user input.

[0024] FIG. 6 is a flowchart depicting example operation of comparison testing of calculator versions.

[0025] In the drawings, reference numbers may be reused to identify similar and/or identical elements.
DETAILED DESCRIPTION

[0026] A recommendation system generates recommendation metrics for users based on account information and user input to provide a more desirable rate to the user and improve a relationship with the user. The recommendations system is an automated system that generates the recommendation metrics for the user in response to the user requesting an updated recommendation metric. In various implementations, the recommendation system may intermittently generate recommendation metrics for users to determine if improved recommendation metrics can be offered to the user, allowing the user to be approached by an entity with a recommendation metric as opposed to the user requesting the Date Recue/Date Received 2020-05-11 recommendation metric. The more desirable rate may pertain to a rate associated with an account the user holds with the entity. For example, the more desirable rate may be a margin rate, commission rate, loan rate, etc. In various implementations, the recommendation metrics provided more desirable metrics pertaining to a relationship of the user and entity other than the rate.

[0027] As described in further detail below, recommendation metrics may include an initial recommendation and subsequent recommendations that are both included within a bounded range. The bounded range may be determined based on a set of parameters that correspond to the first user. The recommendation system accesses a plurality of databases that maintain the set of parameters to calculate the recommendations. These calculations may be conducted in real-time to assist an entity in negotiating with an individual during a conversation.
In this context, real-time means that the calculations are conducted quickly enough to be used in a single interaction (such as a phone call) between the entity and the individual. In other words, real-time in this context means that calculations are conducted in less than 5 minutes, in less than 1 minute, or in less than 10 seconds.

[0028] One database maintains account information including a present margin rate, a margin, a profitability score, a risk score, an amount of revenue, a margin percentage of revenue, a risk relative to revenue, etc. The recommendation system also considers user input when the user is requesting updated recommendation metrics. For example, the recommendation system may consider information provided by the user that indicate a retention likelihood based on a mention of a competitor. The recommendation system generates the recommendations based on the above listed factors.

[0029] Referring now to FIG. 1, a high-level block diagram of an example network communication system including a recommendation system 100 according to the principles of the present disclosure is shown. An analyst device 104 communicates via the Internet 108 with a recommendation device 112.

In various implementations, the analyst device 104 may interact directly with the recommendation device 112 using an internal network connection. In response to Date Recue/Date Received 2020-05-11 the analyst device 104 receiving a user request via a web portal from the analyst, the recommendation device 112 accesses multiple databases, internal or via the Internet 108, to generate an initial or subsequent recommendation for a requesting user submitting the user request.

[0030] The recommendation device 112 can access a profile database 116 via the Internet 108 or via the internal network connection. The profile database stores a profile for users maintaining an account with an entity associated with the profile database 116. In various implementations, the entity may operate a financial trading platform, a loan providing platform, or another financial platform. A profile stored in the profile database 116 includes demographic information of the corresponding user, such as age, account duration, location, etc. The profile also stores account information including the present margin rate, the margin, the profitability score, the risk score, the amount of revenue, the margin percentage of revenue, the risk relative to revenue, etc.

[0031] The recommendation device 112 can also access a historical recommendation database 120. Similar to the profile database 116, the historical recommendation database 120 may also be accessed via the internal network connection to generate recommendations on a user interface of the analyst device 104. In various implementations, the user may be communicating with an analyst via phone or in-person. For the purpose of the present disclosure, it is assumed that the user is communicating with the analyst using a chat system via the web portal. In various implementations, the chat system is operated by a bot (which may be implemented using artificial intelligence) instead of by an analyst.
The historical recommendation database 120 is indexed by user and stores information generated from each interaction between the user and the recommendation device 112.

[0032] For example, the historical recommendation database 120 will create an entry in response to the user submitting user input. The user input may include a request for a recommendation, a response to a previously provided recommendation, an indication of a switch to a competitor, etc. Additionally, as Date Recue/Date Received 2020-05-11 will be discussed in more detail below, when the recommendation device 112 generates the initial recommendation for the user, the historical recommendation database 120 will store the initial recommendation as well as the user's response to the initial recommendation, which may include acceptance or denial.
Further, any subsequent recommendations that the recommendation device 112 generates will also be stored in the historical recommendation database 120 with the parameters of the recommendation as well as the parameters of the user's response.

[0033] The recommendation device 112 can also access a grouping database 124 that stores a group to which each user belongs. For example, a group may be based on user parameters, such as demographic information, account information, etc. The recommendation system 100 has recognized a set of groups of generally similar users based on the user parameters described above. In one example, the users may be grouped according to their risk score, each group corresponding to a different risk score range.

[0034] In various implementations, a similarity amongst the users may be determined. To identify similar users, a grouping system categorizes each user into a group based on features listed above. In various implementations, the recommendation system 100 may use unsupervised learning, such as K-means clustering, to cluster users into the set of groups. In various implementations, lower or upper limits on the number of clusters may be set. These limits may be adaptive based on, for example, the total number of users. Additionally or alternatively, the number of clusters may be determined based on upper and/or lower limits placed on how many users can be in each cluster.

[0035] The recommendation device 112 determines the grouping of the user to influence or adjust the initial and subsequent recommendations. For example, the recommendation device 112 can obtain a likelihood that a member of a particular group would accept a particular recommendation. Similarly, based on the grouping of the user, the recommendation device 112 can assess if a different user of the same group that has obtained a competitor's recommendation would accept Date Recue/Date Received 2020-05-11 a particular recommendation that the recommendation device 112 generates, for example, based on a distance metric between a requested rate included in the user request and a historical distance metric between a similar user's request and accepted rate. The grouping database 124 may also be accessed via the Internet 108 or internal connection.

[0036] The recommendation device 112 can also access a calculator database 128 via the Internet 108 or internal connection. The calculator database 128 stores a variety of versions of calculators. Each calculator version may have a different set of operating constraints. The operating constraints may include a preferred revenue to rate ratio, a preferred margin percentage of revenue, etc. These operating constraints affect the bounded range for the particular user as well as the initial and subsequent recommendations described above. The operating constraints of each calculator version vary so that the calculator versions may be compared to one another using automated AB testing.

The historical recommendation database 120 stores which calculator version from the calculator database 128 is used in each recommendation generation of each entry for comparison and continuity of user recommendations.

[0037] In various implementations, when generating an initial recommendation for a user, the recommendation device 112 may randomly select a calculator version from the calculator database 128. In another implementation, the calculator version stored in the calculator database 128 may be determined based on the requesting user's profile selected from the profile database 116 and/or the group of the requesting user. When the recommendation device 112 is generating subsequent recommendations, the recommendation device 112 determines which calculator version was used to calculate the initial recommendation from the historical recommendation database 120 and continues to use the same calculator version to further improve the automated AB testing calculator comparison and continuity.

[0038] Referring now to FIG. 2, an example user interface 200 displaying input to be received for generating and adjusting recommendations is shown. As shown Date Recue/Date Received 2020-05-11 in the example user interface 200, a calculator version 7.6 of the recommendation device may generate recommendations using an account number 204 of a user, the trailing 12 month revenue 208 of the user's account, the margin percentage of revenue 212 of the user's account, and a current margin rate 216 of the user's account. In various implementations, when a user requests a recommendation, the user only needs to provide their account number and the recommendation device automatically populates the remaining information. The recommendation device may automatically generate a recommendation once the input information is received by the recommendation generator shown in the example user interface 200. In various limitations, the analyst may be accessing the recommendation device using a web portal on the analyst device and may select a generate button 220 to generate the recommendation.

[0039] FIG. 3 is an example user interface 300 displaying generated recommendations. Once the generate button is selected, the recommendation device will generate at least one recommendation. In various implementations, the recommendation device may generate an initial recommendation 304, providing the initial recommendation to suggest to the user based on the information previously provided. The recommendation device may simultaneously generate a subsequent recommendation 308 that may be subsequently provided to the user in the event that the initial recommendation is not accepted. The analyst may also be presented with a floor value 312, which is an absolute minimum of the bounded range that the recommendation device calculates to determine the range of recommendations that may be provided to the user. When an analyst is using the recommendation device, a return button 316 may be present to return to the user interface shown in FIG. 2. In an instance when the recommendation device is automatic, the recommendation device may select a set of users, calculate recommendations, and store the recommendations in a database for an analyst to access in the future. In various implementations, the automatic recommendation device may generate alerts when an initial recommendation is beyond a Date Recue/Date Received 2020-05-11 predetermined threshold difference from a present rate, indicating to an analyst to approach the user to improve the relationship.

[0040] In FIG. 4, a functional block diagram of the recommendation device 112 is shown. The recommendation device 112 receives user input. The user input may be received directly from a user, for example, when the recommendation device 112 is operable by users. In various implementations, the user input is submitted by an analyst using an analyst device communicating with a user requesting a recommendation. In an automated version, the recommendation device 112 may select profiles of users from the profile database 116 to compare present user rates to generated recommendations.

[0041] As described in more detail below, each recommendation interaction is stored and learned from by selecting particular calculator versions to generate recommendations. In the automated version, the recommendation device 112 selects a set of users that are classified into groups based on stored profile data, such as account information, demographic information, portfolio composition (for example, stocks, mutual funds, stock sector, etc.), trading activity, risk score, profitability, attrition probability, a similarity metric, a desired retention value, etc. Then, in a scenario where a member of one of the groups has a successful negotiation, the recommendation device 112 may cycle through the remaining users in the group to identify whether another user can be recommended a reduced rate. Approaching the user with a lower rate without the user having to prompt any recommendation interaction improves the user relationship and may increase profitability.

[0042] The user input includes identifying information of the user, for example, an account number as well as a recommendation request. The user input may further include additional information that may be used to adjust a generated recommendation. The additional information may include a desired recommendation, an indication of switching to a competitor, a name of the competitor, a retention likelihood, an indication of intent (i.e., an indication in the recommendation request that suggests the user desires a recommendation that Date Recue/Date Received 2020-05-11 significantly changes their present circumstances, such as salient terms used to indicate aggression), etc.

[0043] The recommendation device 112 includes an input type determination module 404. The input type determination module 404 receives the user input, as described above, from the analyst, from the corresponding user, or from the profile database, and determines whether the user input is a request for a recommendation or a response to a transmitted recommendation. If the user request includes an initial request for initial recommendation, the recommendation device 112 obtains user parameters to determine the initial recommendation. The user input of the requesting user is forwarded to a recommendation request module 408 and a grouping module 412. The recommendation request module 408 obtains a profile of the requesting user from the profile database 116. The grouping module 412 receives the user input including the identification information identifying the requesting user and similarly obtains the profile of the requesting user from the profile database 116.
The grouping module 412 also determines a group of the requesting user as indexed in the grouping database 124. In various implementations, the grouping module 412 may determine a group of the requesting user based on the grouping of users stored in the grouping database 124. Alternatively, the grouping module 412 may obtain which group the requesting user belongs to from the grouping database 124.

[0044] In various implementations, the grouping module 412 may group the user based on the user input indicating that the user is looking to switch to a competitor. Optionally, the profile of the user may include website interactions of the user with their present account. The website interactions may indicate whether the user has recently downloaded a statement, viewed performance, rates, etc., and determine whether the website interactions indicate the user may be switching to another entity or competitor. Based on the website interactions or user input indicating a switch, the grouping module 412 may group the user based on a retention likelihood.

Date Recue/Date Received 2020-05-11

[0045] A range determination module 416 receives the recommendation request including parameters of the user input from the recommendation request module 408 as well as the group of the requesting user from the grouping module 412. The range determination module 416 also obtains the profile of the requesting user from the profile database 116 and determines a bounded range for the requesting user based on the obtained profile and the group. For example, the bounded range includes a floor value as well as a ceiling value. The floor value is a lowest value that the recommendation device 112 can generate without losing margin. The ceiling value is a highest reasonable value that the recommendation device 112 can request.

[0046] The range determination module 416 determines the bounded range based on parameters obtained or included in the profile of the requesting user including the present margin rate, the margin, the profitability score, the risk score, the amount of revenue, the margin percentage of revenue, the risk relative to revenue, etc. In various implementations, a desired margin rate included in the user input that is received from the requesting user may also influence the bounded range determined by the range determination module 416. In various implementations, the range determination module 416 may communicate (communication not shown) with the calculator database 128 to determine the bounded range based on operating parameters of a particular calculator version.

[0047] A recommendation generation module 420 receives the bounded range from the range determination module 416. In various implementations, the recommendation generation module 420 selects a calculator version stored in the calculator database 128. As mentioned previously, the recommendation generation module 420 may randomly select the calculator version or may select the calculator version based on the parameters of the requesting user's profile, the user input, etc. The recommendation generation module 420 also stores parameters of the requesting user's request in the historical recommendation database 120 and, as previously mentioned, the historical recommendation database 120 may be indexed according to user. For each entry for each user Date Recue/Date Received 2020-05-11 stored in the historical recommendation database 120, the user input for an initial recommendation request and subsequent recommendation request may be stored along with the calculator version used by the recommendation generation module 420, the bounded range determined by the range determination module 416, the group of the user, and other information used to generate the initial or subsequent recommendation.

[0048] Based on the parameters of the requesting user's profile, the bounded range, and the group of the requesting user, the recommendation generation module 420 generates an initial recommendation. The recommendation generation module 420 accounts for the present margin rate, the margin, the profitability score, the risk score, the amount of revenue, the margin percentage of revenue, the risk relative to revenue, etc., to generate the initial recommendation.

[0049] When generating the initial recommendation, the recommendation generation module 420 may also generate, using the calculator version, a subsequent recommendation that may be transmitted to the analyst communicating with the requesting user in the event that the requesting user does not accept the initial recommendation. The recommendation generation module 420 then transmits the initial recommendation to a web portal to be displayed to the analyst using the analyst device, which can then be communicated to the requesting user. In various implementations, the initial and subsequent recommendations may be a set percentage of the bounded range. For example, the initial recommendation may be set at the 70% point of the bounded range. In other words, if the bounded range is defined by values named Lower and Upper, the 70% point can be calculated as Lower + 0.7 * (Upper ¨ Lower). As a further example, the subsequent recommendation may be set at the 50% point of the bounded range.

[0050] If the user input is a response to an initial or subsequent recommendation, the recommendation device 112 determines whether the response is an acceptance or denial of the initial or subsequent recommendation and stores the acceptance or generates a subsequent recommendation in response Date Recue/Date Received 2020-05-11 to the denial. When the input type determination module 404 receives the response, the input type determination module 404 identifies the response as a response and transmits the user input to a recommendation response module 424.

[0051] The recommendation response module 424 determines whether the user input is an acceptance of a most recent recommendation or a denial of the most recent recommendation. In response to receiving an acceptance, the recommendation response module 424 updates the responding user profile with the most recent recommendation that the responding user has accepted. The recommendation response module 424 also stores in the historical recommendation database 120 that the responding user has accepted the most recent recommendation.

[0052] In response to receiving a denial, the recommendation response module 424 retrieves an entry corresponding to the responding user from the historical recommendation database 120 and transmits the responding user's response as well as the history of previous recommendations provided to the responding user to the recommendation generation module 420. The recommendation generation module 420 then generates a subsequent recommendation based on the stored bounded range associated with the responding user, the most recent recommendation provided to the responding user that the responding user denied, and the original factors considered, such as the profitability score. The recommendation generation module 420 may also determine the subsequent recommendation based on user input from the responding user. For example, if the responding user indicated that a competitor offered a more preferable recommendation, the recommendation generation module 420 may determine that, based on the competitor or based on the more preferable recommendation offered by the competitor, a more aggressive subsequent recommendation should be provided to the responding user due to the user's account information, such as the present margin rate, the margin, the profitability score, the risk score, the amount of revenue, the margin percentage of revenue, the risk relative to revenue, etc. Once generated, the subsequent Date Recue/Date Received 2020-05-11 recommendation is transmitted to the web portal to be displayed on the analyst device for communication to the user.

[0053] Referring now to FIG. 5, a flowchart depicting example operation of generating and adjusting recommendations based on user input is shown. Control begins upon receiving user input. At 504, control determines whether a recommendation request or a response to a recommendation has been received. If a recommendation request was received, control continues to 508. Otherwise, if a response to a recommendation is received, control proceeds to 512.

[0054] At 508, control obtains request parameters for a requesting user. The request parameters include what the requesting user is requesting, such as a reduced margin rate as well as any peripheral information including the desired rate, a competitor's rate, a retention likelihood, etc. The received request also includes identification information of the requesting user, such as account information, user name, etc. Control continues to 516 to set a response_number equal to zero for the requesting user. Control continues to 520 to obtain a first user profile of the requesting user. As described above, the user profile for each user that maintains an account (for example, for each user who has a financial account or a loan account) includes information such as the present margin rate, the margin, the profitability score, the risk score, the amount of revenue, the margin percentage of revenue, the risk relative to revenue, etc., regarding the requesting user's account.

[0055] At 524, control identifies a first group of the first profile from among a set of groups. Each user profile is categorized into a group based on account information, demographic information, a similarity metric, a distance metric between a desired rate and the present rate, and/or the desired retention value. For example, users may be grouped based on similarity among the users. Control may then analyze previous recommendations made to users in the same group as the requesting user to determine a likelihood of acceptance or to adjust a recommendation provided to the requesting user to increase the likelihood of acceptance.

Date Recue/Date Received 2020-05-11

[0056] Control continues to 528 to determine a bounded range based on the first group and the first profile. The bounded range is a floor value to a ceiling value.
Each value is determined based on the requesting user's account information and information of users within the first group that was used to categorize the users.
Control continues to 532 to select a calculator version from a set of calculators based on the requesting user. The selected calculator version may be random or may be based on account information of the requesting user. In various implementations, each group may have a corresponding calculator version. For example, each calculator version may have at least one parameter that varies with respect to other versions, allowing for varying recommendations as well as varying bounded ranges. In various implementations, each calculator is compared to other calculator versions based on one or more resulting parameters, such as profitability, risk score, etc.

[0057] At 536, control generates, using the selected calculator version, a first recommendation. The first recommendation is within the bounded range and based on the parameters of the first profile, which may include the profitability score and risk score, as described above. Control continues to 540 adjust the first recommendation based on user input of the requesting user. That is, if the user input indicates a desired rate or an indication that the requesting user is likely to leave, the first recommendation may be adjusted to be more similar to the desired rate or may be adjusted to be more aggressive or preferable to the user if it appears that the user is likely to close their account.

[0058] Control continues to 544 to transmit the first recommendation to the analyst device for display to the analyst and communication to the user. At 548, control stores the request, the first recommendation, and calculator version in association with the requested user in a database for future analysis and reference.
Storing recommendations and user responses allows for the generation of recommendations that a similar user is more likely to accept, reducing the number of recommendations generated and the amount of back and forth between the user and the analyst or the recommendation device. Then, control ends.

Date Recue/Date Received 2020-05-11

[0059] Returning to 512, control obtains response parameters after determining that the user input is a response to a recommendation. The response parameters may include an acceptance or a denial as well as additional information pertaining to the most recent recommendation. For example, the response to the most recent recommendation may include, along with the denial, a name of a competitor and an offered rate from the competitor or an indication of frustration of the user.
These response parameters may be used to generate a subsequent recommendation that is more likely to be accepted by the user. At 552, control obtains the user profile of the responding user. At 556, control obtains the most recent recommendation parameters, including calculator version, account information used to calculate the most recent recommendation, the bounded range, etc.

[0060] Control continues to 560 to determine whether the response parameters indicate acceptance of the most recent recommendation. If yes, control proceeds to 564. Otherwise, control continues to 568. If the most recent recommendation is accepted, at 564, control marks the most recent recommendation of the responding user as accepted. Then, control proceeds to 572 to update the responding user's profile with the accepted recommendation, including the new margin rate or other parameter of the user profile that is being recommended.
Then, control ends.

[0061] Returning to 568, if the most recent recommendation is not accepted, control determines if the response_number is greater than a predetermined threshold. The predetermined threshold is a number of times that a recommendation may be transmitted to the responding user. Therefore, if the response_number exceeds the predetermined threshold, control has generated a maximum number of recommendations for the responding user and will no longer generate recommendations for the responding user. Instead, control will proceed to 576 to generate and transmit a response_number exceeded alert. After receiving the alert, the analyst can evaluate how to proceed with the responding user instead of continuing to generate recommendations that the user is not accepting.

Date Recue/Date Received 2020-05-11

[0062] Returning to 568, if the response_number is not greater than the predetermined threshold, control proceeds to 580 to increment the response_number. Then, control proceeds to 584 to generate a subsequent recommendation based on the most recent recommendation and the response received in the user input. For example, the subsequent recommendation may be set to equal 60% of the most recent recommendation. In various implementations, if the response includes information indicating that the responding user retention is low (for example, based on use of argumentative words) and the profitability score of the user account is high, control may set the subsequent recommendation equal to 50% of the most recent recommendation to increase the likelihood of user acceptance.

[0063] Further, at 588, control adjusts the subsequent recommendation based on the original request stored in association with the responding user.
Therefore, control may adjust subsequent recommendation further according to a desired rate that was supplied in the original request for a recommendation from the responding user. Similar to the previous percentage example given above, control may further reduce the percentage of the most recent recommendation provided to the user. At 592, control transmits the subsequent recommendation to the analyst device via the web portal. At 596, control stores the subsequent recommendation in association with the responding user and then control ends.

[0064] Referring now to FIG. 6, a flowchart depicts example comparison testing of calculator versions. In various implementations, a calculator comparison module (not shown) may be included in the example recommendation device of FIG. 4, accessing the historical recommendation database and profile database to compare the results of each calculator version. For example, the resulting profitability from each accepted recommendation may be compared across calculator versions. A first calculator version may have an adjusted bounded range that prevents generated recommendations from being 80% below a user requested rate while a second calculator version and a third calculator version Date Recue/Date Received 2020-05-11 may prevent generated recommendations from being 60% and 40% below the user requested rate, respectively.

[0065] During a comparison of calculator versions, a highest performing calculator version may be selected as validated. For example, each new calculator version that is developed or has a variable (such as the bounded range) that is altered may be added to the calculator database and marked as a beta version.
The calculator database may also store calculator versions that are marked as validated. When a recommendation interaction occurs, a particular calculator version is randomly selected from a set of validated calculator versions.

[0066] To test beta calculator versions prior to validation, a predetermined number of recommendation interactions may be conducted using beta calculator versions. During testing, a beta calculator version is randomly selected from the set of beta calculator versions for each of the predetermined number of recommendation interactions. After a sufficient amount of testing is conducted using the beta calculator versions, the predetermined number of recommendation interactions may be compared to validate one or more of the beta calculator versions.

[0067] As shown in FIG. 6, control begins upon expiration of a predetermined testing period. That is, beta calculator versions are compared after enough time has passed for a sufficient number of recommendation interactions to have occurred. Control continues to 604 to obtain a list of test calculator versions. At 608, control selects a comparison metric, such as profitability, risk score, etc. At 612, control selects a first calculator version from the list of test calculator versions. Control proceeds to 616 to obtain data related to the selected comparison metric for the selected calculator version. Data related to each comparison metric is stored after a recommendation interaction is complete, also storing the corresponding calculator version. For example, if the comparison metric is profitability, control obtains profitability values for each recommendation interaction that used the selected calculator version. Control continues to 620 to calculate performance of the selected calculator version based Date Recue/Date Received 2020-05-11 on the comparison metric. In various implementations, the calculated performance is an average of the comparison metric across each recommendation interaction that used the selected calculator version. Alternatively, depending on the selected comparison metric, the calculated performance may be a weighted average of the comparison metric, and the weight may be based on a size of the account.

[0068] At 624, control adds the calculated performance of the selected calculator version to a comparison list. Control continues to 628 to determine if another calculator version is in the list of test calculator versions. If yes, control proceeds to 632 to select the next test calculator version and return to 616.
Otherwise, control continues to 636 to select a best performance metric from the comparison list. For example, the calculator version that results in the highest profitability is selected while the calculator version that results in the lowest risk score is selected. Control then continues to 640 to mark the best performing calculator version as validated. Then, control ends.
CONCLUSION

[0069] The foregoing description is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses. The broad teachings of the disclosure can be implemented in a variety of forms. Therefore, while this disclosure includes particular examples, the true scope of the disclosure should not be so limited since other modifications will become apparent upon a study of the drawings, the specification, and the following claims. It should be understood that one or more steps within a method may be executed in different order (or concurrently) without altering the principles of the present disclosure.
Further, although each of the embodiments is described above as having certain features, any one or more of those features described with respect to any embodiment of the disclosure can be implemented in and/or combined with features of any of the other embodiments, even if that combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and permutations Date Recue/Date Received 2020-05-11 of one or more embodiments with one another remain within the scope of this disclosure.

[0070] Spatial and functional relationships between elements (for example, between modules) are described using various terms, including "connected,"
"engaged," "interfaced," and "coupled." Unless explicitly described as being "direct," when a relationship between first and second elements is described in the above disclosure, that relationship encompasses a direct relationship where no other intervening elements are present between the first and second elements, and also an indirect relationship where one or more intervening elements are present (either spatially or functionally) between the first and second elements. As used herein, the phrase at least one of A, B, and C should be construed to mean a logical (A OR B OR C), using a non-exclusive logical OR, and should not be construed to mean "at least one of A, at least one of B, and at least one of C."

[0071] In the figures, the direction of an arrow, as indicated by the arrowhead, generally demonstrates the flow of information (such as data or instructions) that is of interest to the illustration. For example, when element A and element B
exchange a variety of information but information transmitted from element A
to element B is relevant to the illustration, the arrow may point from element A
to element B. This unidirectional arrow does not imply that no other information is transmitted from element B to element A. Further, for information sent from element A to element B, element B may send requests for, or receipt acknowledgements of, the information to element A. The term subset does not necessarily require a proper subset. In other words, a first subset of a first set may be coextensive with (equal to) the first set.

[0072] In this application, including the definitions below, the term "module"
or the term "controller" may be replaced with the term "circuit." The term "module"
may refer to, be part of, or include processor hardware (shared, dedicated, or group) that executes code and memory hardware (shared, dedicated, or group) that stores code executed by the processor hardware.

Date Recue/Date Received 2020-05-11

[0073] The module may include one or more interface circuits. In some examples, the interface circuit(s) may implement wired or wireless interfaces that connect to a local area network (LAN) or a wireless personal area network (WPAN). Examples of a LAN are Institute of Electrical and Electronics Engineers (IEEE) Standard 802.11-2016 (also known as the WWI wireless networking standard) and IEEE Standard 802.3-2015 (also known as the ETHERNET wired networking standard). Examples of a WPAN are the BLUETOOTH wireless networking standard from the Bluetooth Special Interest Group and IEEE
Standard 802.15.4.

[0074] The module may communicate with other modules using the interface circuit(s). Although the module may be depicted in the present disclosure as logically communicating directly with other modules, in various implementations the module may actually communicate via a communications system. The communications system includes physical and/or virtual networking equipment such as hubs, switches, routers, and gateways. In some implementations, the communications system connects to or traverses a wide area network (WAN) such as the Internet. For example, the communications system may include multiple LANs connected to each other over the Internet or point-to-point leased lines using technologies including Multiprotocol Label Switching (MPLS) and virtual private networks (VPNs).

[0075] In various implementations, the functionality of the module may be distributed among multiple modules that are connected via the communications system. For example, multiple modules may implement the same functionality distributed by a load balancing system. In a further example, the functionality of the module may be split between a server (also known as remote, or cloud) module and a client (or, user) module.

[0076] The term code, as used above, may include software, firmware, and/or microcode, and may refer to programs, routines, functions, classes, data structures, and/or objects. Shared processor hardware encompasses a single microprocessor that executes some or all code from multiple modules. Group Date Recue/Date Received 2020-05-11 processor hardware encompasses a microprocessor that, in combination with additional microprocessors, executes some or all code from one or more modules.
References to multiple microprocessors encompass multiple microprocessors on discrete dies, multiple microprocessors on a single die, multiple cores of a single microprocessor, multiple threads of a single microprocessor, or a combination of the above.

[0077] Shared memory hardware encompasses a single memory device that stores some or all code from multiple modules. Group memory hardware encompasses a memory device that, in combination with other memory devices, stores some or all code from one or more modules.

[0078] The term memory hardware is a subset of the term computer-readable medium. The term computer-readable medium, as used herein, does not encompass transitory electrical or electromagnetic signals propagating through a medium (such as on a carrier wave); the term computer-readable medium is therefore considered tangible and non-transitory. Non-limiting examples of a non-transitory computer-readable medium are nonvolatile memory devices (such as a flash memory device, an erasable programmable read-only memory device, or a mask read-only memory device), volatile memory devices (such as a static random access memory device or a dynamic random access memory device), magnetic storage media (such as an analog or digital magnetic tape or a hard disk drive), and optical storage media (such as a CD, a DVD, or a Blu-ray Disc).

[0079] The apparatuses and methods described in this application may be partially or fully implemented by a special purpose computer created by configuring a general purpose computer to execute one or more particular functions embodied in computer programs. The functional blocks and flowchart elements described above serve as software specifications, which can be translated into the computer programs by the routine work of a skilled technician or programmer.
Date Recue/Date Received 2020-05-11

[0080] The computer programs include processor-executable instructions that are stored on at least one non-transitory computer-readable medium. The computer programs may also include or rely on stored data. The computer programs may encompass a basic input/output system (BIOS) that interacts with hardware of the special purpose computer, device drivers that interact with particular devices of the special purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0081] The computer programs may include: (i) descriptive text to be parsed, such as HTML (hypertext markup language), XML (extensible markup language), or JSON (JavaScript Object Notation), (ii) assembly code, (iii) object code generated from source code by a compiler, (iv) source code for execution by an interpreter, (v) source code for compilation and execution by a just-in-time compiler, etc. As examples only, source code may be written using syntax from languages including C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, Java , Fortran, Perl, Pascal, Curl, OCaml, JavaScript , HTML5 (Hypertext Markup Language 5th revision), Ada, ASP (Active Server Pages), PHP (PHP:
Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Flash , Visual Basic , Lua, MATLAB, SIMULINK, and Python .

Date Recue/Date Received 2020-05-11

Claims (20)

What is claimed is:

1. A recommendation system comprising:
at least one processor; and a memory coupled to the at least one processor, wherein the memory stores:
a profile database including a plurality of profiles corresponding to a plurality of user identifiers, wherein each profile of the plurality of profiles includes a set of parameters for a corresponding user identifier;
a historical recommendation database including previously generated recommendations transmitted to a web portal;
a calculator database including a plurality of calculator versions, wherein the historical recommendation database includes, for each user identifier, a corresponding calculator version identifier used to generate an initial recommendation; and instructions that, upon execution, cause the at least one processor to, in response to receiving a recommendation request for a requesting user identifier via the web portal:
obtain a first profile of the requesting user identifier from the profile database;
from a set of groups, identify a first group of the set of groups corresponding to the requesting user identifier based on the first profile, wherein each profile of the plurality of profiles is associated with a corresponding group of the set of groups based on a similarity;
determine a bounded range based on (i) the first group and (ii) the first profile;
select a calculator version from the plurality of calculator versions;

based on the bounded range, estimate the initial recommendation using the selected calculator version; and transmit the initial recommendation to the web portal for display.

2. The recommendation system of claim I wherein the instructions, upon execution, cause the at least one processor to:
in response to receiving a response for a responding user identifier via the web portal:
update the first profile with a most recent recommendation in response to the response indicating an acceptance; and generate a subsequent recommendation in response to the response indicating a denial.

3. The recommendation system of claim 2 wherein the instructions, upon execution, cause the at least one processor to:
in response to the response indicating the denial:
obtain a first user entry from the historical recommendation database; and generate the subsequent recommendation based on parameters of the most recent recommendation stored in the first user entry.

4. The recommendation system of claim I wherein the instructions, upon execution, cause the at least one processor to, prior to transmitting the initial recommendation, adjust the initial recommendation in response to the recommendation request including a desired recommendation.

5. The recommendation system of claim I wherein the instructions, upon execution, cause the at least one processor to, prior to transmitting the initial recommendation:

for each user of the first group, obtain a corresponding entry from the historical recommendation database;
determine a likelihood of acceptance of the initial recommendation based on the obtained entries; and adjust the initial recommendation according to the likelihood of acceptance.

6. The recommendation system of claim 1 wherein the set of parameters for each user profile includes (i) a present margin rate, (ii) a margin, (iii) a profitability score, (iv) a risk score, (v) an amount of revenue, (vi) a margin percentage of revenue, and (vii) a risk relative to revenue.

7. The recommendation system of claim 1 wherein the similarity is determined using a clustering algorithm to identity and group similar profiles.

8. The recommendation system of claim 1 wherein each group of the set of groups is associated with a respective calculator version.

9. The recommendation system of claim 1 wherein each calculator version of the plurality of calculator versions is marked as testing or validated.

10. The recommendation system of claim 9 wherein the instructions, upon execution, cause the at least one processor to, after a predetermined testing period has elapsed:
select each calculator version of the plurality of calculator versions marked as testing;
calculate, for each selected calculator version, a performance metric over the predetermined testing period;
identify a highest performance metric of the calculated performance metrics; and mark a corresponding calculator version that corresponds to the highest performance metric as validated.

11. A recommendation method comprising, in response to receiving a recommendation request for a requesting user identifier via a web portal:
obtaining a first profile of the requesting user identifier from a profile database, wherein the profile database stores a plurality of profiles corresponding to a plurality of user identifiers, and wherein each profile of the plurality of profiles includes a set of parameters for a corresponding user identifier;
from a set of groups, identifying a first group of the set of groups corresponding to the requesting user identifier based on the first profile, wherein each profile of the plurality of profiles is associated with a corresponding group of the set of groups based on a similarity;
determining a bounded range based on (i) the first group and (ii) the first profile;
selecting a calculator version from a plurality of calculator versions, wherein a calculator database stores the plurality of calculator versions, and wherein a historical recommendation database stores, for each user identifier, a corresponding calculator version identifier used to generate an initial recommendation;
based on the bounded range, estimating the initial recommendation using the selected calculator version; and transmitting the initial recommendation to the web portal for display.

12. The recommendation method of claim 11 further comprising:
in response to receiving a response for a responding user identifier via the web portal:
updating the first profile with a most recent recommendation in response to the response indicating an acceptance; and generating a subsequent recommendation in response to the response indicating a denial.

13. The recommendation method of claim 12 further comprising:
in response to the response indicating the denial:
obtaining a first user entry from the historical recommendation database; and generating the subsequent recommendation based on parameters of the most recent recommendation stored in the first user entry.

14. The recommendation method of claim 11 further comprising, prior to transmitting the initial recommendation, adjusting the initial recommendation in response to the recommendation request including a desired recommendation.

15. The recommendation method of claim 11 further comprising, prior to transmitting the initial recommendation:
for each user of the first group, obtaining a corresponding entry from the historical recommendation database;
determining a likelihood of acceptance of the initial recommendation based on the obtained entries; and adjusting the initial recommendation according to the likelihood of acceptance.

16. The recommendation method of claim 11 wherein the set of parameters for each user profile includes (i) a present margin rate, (ii) a margin, (iii) a profitability score, (iv) a risk score, (v) an amount of revenue, (vi) a margin percentage of revenue, and (vii) a risk relative to revenue.

17. The recommendation method of claim 11 wherein the similarity is determined using a clustering algorithm to identity and group similar profiles.

18. The recommendation method of claim 11 wherein each group of the set of groups is associated with a respective calculator version.

19. The recommendation method of claim 11 wherein each calculator version of the plurality of calculator versions is marked as testing or validated.

20. The recommendation method of claim 19 further comprising, after a predetermined testing period has elapsed:
selecting each calculator version of the plurality of calculator versions marked as testing;
calculating, for each selected calculator version, a performance metric over the predetermined testing period;
identifying a highest performance metric of the calculated performance metrics; and marking a corresponding calculator version that corresponds to the highest performance metric as validated.