RU2680760C1 - Scoring models development and control computerized method - Google Patents

Abstract

FIELD: computer equipment.

SUBSTANCE: invention relates to the field of computer equipment. Disclosed is the scoring models development and control computerized method, including the following steps: obtaining data for the specified period of time, containing the scoring model affecting factors; performing the obtained data partitioning into samples for the scoring model development, validation and testing; performing the factors transformation by establishing relationships between groups of the transformed factor values and the defaults levels; excluding from the samples at least one transformed factor, correlating with at least one other factor; forming the credit scoring model through the binary multiple logistic regression training; automatically selecting the cut-off zones for at least one scoring model for its installation into the credit procedure.

EFFECT: technical result is increase in the developed credit scoring models quality.

10 cl, 2 dwg

Description

FIELD OF TECHNOLOGY

[1] This technical solution, in General, relates to the field of computer technology, and in particular to methods for the automatic development of credit scoring models and their automatic implementation in the loan process.

BACKGROUND

[2] Currently, financial institutions are using standard statistical approaches to the analysis of historical data to describe potential customers in terms of risk. This allows you to classify borrowers as “good” and “bad” and thus make the final decision on lending. Most credit institutions have created units that develop credit scoring models based on their own statistics, taking into account the specifics of the client profile. However, these credit institutions often turn to the credit bureaus, which is why the process of assessing the creditworthiness of the borrower is dragged out and becomes inaccurate, as it depends on the algorithms used by the credit bureaus.

SUMMARY OF THE INVENTION

[3] This technical solution is aimed at eliminating the disadvantages inherent in existing solutions known from the prior art.

[4] The technical problem (or technical problem) in this technical solution is the automatic development of credit scoring models with their subsequent implementation in the decision-making system and monitoring.

[5] The technical result manifested in solving the above problem is to improve the quality of the created credit scoring models.

[6] An additional technical result that manifests itself in solving the technical problem is to increase the speed of developing credit scoring models.

[7] The need for the amount of resources necessary for the development and support of models, the increase in the speed and ease of implementation of models in the industrial circuit, as well as the monitoring of the operation of models and the operational response to changes are also decreasing.

[8] The specified technical result is achieved through the implementation of a method for the development and management of scoring models, in which data is obtained for a given period of time, the content of factors affecting the scoring model; after that, the obtained data is divided into samples for the development, validation and testing of the scoring model; then the transformation of factors is carried out by establishing relationships between groups of values of the converted factor and default levels; at least one transformed factor correlating with at least one other factor is further excluded from the samples; form a credit scoring model through training of binary multiple logistic regression; automatically select clipping zones for at least one scoring model for its installation in the credit procedure.

[9] In some embodiments, data is obtained for a predetermined period of time from a user's mobile communication device.

[10] In some embodiments, when dividing the data into samples, non-intersecting parts of the original population or random subsamples are obtained.

[11] In some embodiments, factors affecting the scoring model are annual income and / or amount of outstanding debt, and / or ownership of real estate, and / or possession of a car, and / or work experience in last place, and / or age.

[12] In some embodiments, the factors affecting the scoring model are discrete or continuous.

[13] In some embodiments, the implementation of the transformation of factors determines the degree of deviation of the level of defaults in the data group from the average level of defaults throughout the sample.

[14] In some embodiments, when transforming factors into factors that are on the excluded list, the algorithm for splitting factor values with a new set of settings is launched.

[15] In some embodiments, when excluding transformed factors from the samples, a table is created with the values of the pair correlation coefficients of the transformed factors.

[16] In some embodiments, when excluding transformed factors from the samples, the factor that has the highest number of factors correlated with it is selected in the cycle.

[17] In some embodiments, when creating a credit scoring model, a logistic model is constructed using stepwise regression to select the final set of factors.

BRIEF DESCRIPTION OF THE DRAWINGS

[18] The features and advantages of the present invention will become apparent from the following detailed description of the invention and the accompanying drawings, in which:

[19] In FIG. 1 shows an example implementation of a method for developing and managing scoring models in the form of a flowchart.

[20] In FIG. 2 shows a top-level example implementation diagram of a method for developing and managing scoring models. The main core is made up of two blocks - this is retraining and selection / adjustment of cut-off zones, and without adaptation of cut-off zones it is impossible to organize automatic implementation of the model in the decision-making system. The results of these two blocks are integrated into the industrial environment (in this embodiment, in SAS RTDM). In addition, each of these two blocks is subject to routine checks in the form of daily monitoring of the target indicator, depending on the cut-off zones (level of approval) and monthly model validation.

DETAILED DESCRIPTION OF THE INVENTION

[21] This technical solution can be implemented on a computer, in the form of an automated system (AS) or a machine-readable medium containing instructions for performing the above method.

[22] The technical solution can be implemented as a distributed computer system.

[23] In this solution, a system means a computer system, a computer (electronic computer), CNC (numerical control), PLC (programmable logic controller), computerized control systems, and any other devices that can perform a given, well-defined sequence of computing operations (actions, instructions).

[24] A command processing device is understood to mean an electronic unit or an integrated circuit (microprocessor) that executes machine instructions (programs).

[25] An instruction processing device reads and executes machine instructions (programs) from one or more data storage devices. Data storage devices may include, but are not limited to, hard disks (HDDs), flash memory, ROM (read only memory), solid state drives (SSDs), and optical drives.

[26] A program is a sequence of instructions intended for execution by a computer control device or an instruction processing device.

[27] Terms and concepts necessary for the implementation of this technical solution will be described below.

[28] Credit scoring is a method of modeling a borrower's credit risk based on numerical statistical methods. The purpose of credit scoring is to make decisions on granting loans to individuals or legal entities.

[29] P-value is a value used in testing statistical hypotheses. In fact, this is the probability of error in rejecting the null hypothesis (errors of the first kind).

[30] Representativeness is the correspondence of the characteristics of the sample to the characteristics of the population or the population as a whole. Representativeness determines how much it is possible to generalize the results of the study with the involvement of a specific sample for the entire population.

[31] DR - default rate. It is calculated as the number of default cases in the group divided by the number of all cases in the group.

[32] Bootstrap is a practical computer method for studying the distribution of statistics of probability distributions, based on the multiple generation of samples based on the available sample.

[33] Default probability - the probability of a transaction occurring within one year from the date of rating assignment / adjustment.

[34] A sample is a set of transactions and their parameters that meet specified characteristics and are part of the analyzed population.

[35] A sample for training is a set of transactions and their parameters used to evaluate the model.

[36] A sample for assessing stability is a set of transactions and their parameters that are used to assess the stability of the ranking ability of factors and their partitions.

[37] Sampling for testing - data on all available contracts for all available reporting dates. It is defined in relation to the segment on which the model is developed.

[38] General population - a set of transaction-date pairs related to a selected segment.

[39] Discrete factors - factors with a limited number of value options.

[40] Continuous factors - factors with an unlimited number of possible values.

[41] A training sample is a set of transactions and their parameters used to develop a model.

[42] Conversion of factors - replacement of the values of factors by the calculated values (scores, WOE) associated with the assessment of the probability of default related to the value of the factor.

[43] Scoring score - the value of the indicator of the quality of transactions in terms of the probability of their default.

[44] Test sample - a sample used to verify the effectiveness of the resulting model (not involved in development)

[45] The transformation of factors is the same as the transformation of factors.

[46] PD is the probability of default.

[47] WOE (English weight of evidence) - a value that characterizes the degree of deviation of the level of defaults in the group from the average level of defaults throughout the sample.

[48] A computerized method for developing and managing scoring models, schematically shown in FIG. 1 includes the following steps.

[49] Step 101: Obtain data for a given period of time containing factors that influence the scoring model.

[50] User data may include the current state of accounts (including closed ones) - opening dates, current balances, term, currency, type and name of product, number of extensions, current status, and so on, without limitation.

[51] The data obtained may also include monthly balances (at the end of each month) for each account for the last period of time (for example, for the last six months), all transactions for the same period with the amount, type and subtype, with the sign “debit / credit ".

[52] The above data, which is a sample, can be obtained from a mobile communication device of a user, for example, such as a tablet, mobile phone, smartphone, or from an automated system of a financial and credit organization in which data is stored.

[53] Based on the obtained user data, credit scoring is automatically determined, ie predict the non-repayment of the loan by the user. To do this, use a training sample: a set of objects (users), each of which is characterized by a set of characteristics (such as age, salary, type of loan, state of accounts, monthly balances, defaults, etc.), as well as the target attribute. A target may be, for example, a loan overdue. If this target attribute is simply a fact of non-repayment of the loan (takes the value 1 or 0, i.e. the financial and credit institution knows about its customers who repaid the loan and who did not), then this is a (binary) classification task. If you know how much time the client delayed in repaying the loan and you want to predict the same thing for new customers, then this will be a regression task.

[54] For each group of accounts (deposits and other accounts), the following data or factors may be considered:

[55] Number of accounts;

[56] Number of accounts with the status of "Active";

[57] Number of accounts with the status "Closed";

[58] Number of accounts with the status "Account Arrested";

[59] Worst status across all customer accounts;

[60] The number of accounts in foreign currency;

[61] The number of accounts in precious metals;

[62] Minimum term on accounts;

[63] The average term on the accounts;

[64] The maximum term for the accounts;

[65] Minimum term on current accounts;

[66] Average term on current accounts;

[67] The maximum period for current accounts;

[68] Weighted average term for the current balance in rubles;

[69] The total amount of current balances;

[70] Maximum balance of all accounts;

[71] The weighted average share of foreign currency accounts at the current balance;

[72] Weighted average share of accounts in precious metals for the current balance;

[73] Time in days elapsed since the opening date of the earliest account.

[74] It will be apparent to those skilled in the art that the above data set is exemplary and, in some embodiments, may differ from the above.

[75] Next, at least one sample is generated to develop a scoring model. For this, the most relevant loans issued for one calendar year are used, which have been in the portfolio for at least 12 months. Since scoring models are developed to predict the behavior of all borrowers, its development solely on issued applications may lead to inaccurate results. In this case, the model will be trained on a biased sample, therefore, they analyze the applications for which failures of the previous scoring model were received. In order to account for these failures, a certain percentage of the worst applications for which the failure of the previous model was received is added to the sample for developing the scoring model. All such applications are considered defaulted by default.

[76] Step 102: carry out the partitioning of the data into samples for training, validation and testing of the scoring model.

[77] At this stage, the initial data set is divided into a training, validation and test sample in a given ratio. In the future, the training sample is used at all stages of the process, the validation sample is used to select the most stable factors and the final quality control of the scoring model, and the test sample is used for complex independent testing. Samples in some embodiments may be formed as consecutive, non-intersecting parts of the original population, or as random subsamples.

[78] Step 103: carry out the transformation of factors by establishing relationships between groups of values of the transformed factor and default levels.

[79] The factors used as input parameters for scoring models and potentially related to the user's creditworthiness may include, but are not limited to, annual income, outstanding debt, ownership of real estate or a car, work experience in last place, age, etc. P.

[80] Among the factors describing the data of the loan application, the majority are usually discrete (education, gender, marital status, purpose of the loan, type of ownership of housing, occupation, etc.). Moreover, if some factors lend themselves to some ordering (for example, education - we can assume that the higher the level, the greater the value of the variable), then for others there is no meaningful linear order (for example, marital status or purpose of the loan). Therefore, such variables cannot even be considered approximately continuous, since their values are the numbers of answers to the corresponding questions, which can be arranged in an arbitrary order. If the scoring model used requires the use of continuous variables, then the discreteness of the variables can be circumvented by replacing them with a larger number of variables taking values from 0 to 1.

[81] The transformation of each factor under consideration consists in replacing its values with a calculated value - WOE.

[82] WOE - weight of evidence, characterizes the degree of deviation of the level of defaults for a group of data from the average level of defaults for the entire sample. Thus, each factor is replaced by its corresponding WOE factor as follows:

[83] where f is the factor under consideration, i is the number of the group of values of the factor f, WOE _i (f) is the value of WOE corresponding to the group of values of i.

[84] In some embodiments, the WOE metric can be any value. Positive values of WOE indicate that the segment in question has a lower default level than the sample as a whole (the higher the WOE, the lower the level of defaults). A value of WOE less than zero indicates that the segment in question has a higher default level than the sample as a whole. The WOE values for group i can be determined as follows:

[85] where N _G (i) and N _G are the number of non-default cases in group i and for the entire sample, respectively, N _B (i) and N _B are the number of default cases in group i and for the entire sample, respectively.

[86] If N _G (i) = 0 or N _B (i) = 0, then the WOE value for the group is determined by the formula:

[87] For continuous factors, the grouping is carried out in such a way that observations with a comparable level of defaults (DR) fall into each range. As a result of the grouping process, the continuous factor is divided into several groups, for each of which it is possible to assess the level of defaults based on observations that fall into this range.

[88] The grouping of variables with a discrete set of values is carried out similarly to the grouping of continuous factors - based on a comparable level of defaults (DR). One or several factor values may fall into each group. The level of defaults is calculated for all the observations included in the group.

[89] The use of WOE factors has the following advantages:

[90] Linearization of factors in accordance with the prerequisites of logistic regression.

[91] Automatic processing of missing values: they are either combined with the group most similar in defaults or act as a separate group. In the case when the missing value is not interpreted or is absent in the sample, then it belongs to the group with the worst risk level.

[92] Automatic processing of anomalous values, since they are not able to adversely affect the model and their actual value is not used in the model. They will enter the model as an element of one of the extreme groups, characterized by their WOE value, based only on the ratio of default and non-default observations in the group.

[93] The ability to evaluate and control the logical direction of the relationship between the factor values and the level of defaults (business logic), which ensures that the final scoring points will make sense (for example, older people usually score more points than young ones). Logical connections confirm business experience, therefore, they allow to obtain a more stable model.

[94] Reduces the risk of retraining. Each random data change is not included in the model, which would be the case for ungrouped attributes. Such a model has greater flexibility and is able to withstand some changes in the population, which ensures stability over a longer period of time.

[95] The initial grouping of factor values can occur using one-factor decision trees. This makes it possible to increase the discriminatory ability of the obtained factors in comparison with manual groupings, since the obtained groups will be as homogeneous as possible inside and different from each other based on the statistical criterion used.

[96] The discriminatory power of a factor is understood to mean its ability to differentiate between default and non-default observations. To assess the discriminatory ability of a variable, a Gini index can be used.

[97] Based on the practice of the prior art, on the interpretability of the factors used in scoring, it is necessary to pay attention not only to the ranking ability of WoE-transformed factors, but also to their business logic. For this reason, at this stage, not only the automatic partition of the values of the factors and calculation of WoE for them occurs, but also the verification of the resulting partitions into business logic. If the received partition does not pass this test, then the algorithm tries to obtain a new partition using alternative settings. A method for obtaining the resulting WoE factors includes the steps below.

[98] First, the splitting of factor values with the specified set of settings is started.

[99] Then, the obtained groups are merged by the proximity of the WoE values if the WoE distance between the groups does not exceed a predetermined threshold. For interval factors, the order of the groups ordered by the factor values is also taken into account. Factors that have only one group left after associations are transferred to the list of excluded ones.

[100] In the next step, small groups are merged according to a predetermined threshold value with the group closest to the WoE. For interval factors, the order of the groups ordered by the factor values is also taken into account. Factors that have only one group left after associations are transferred to the list of excluded ones. After each merger, you need to return to the second paragraph.

[101] It is also important to merge the obtained groups according to the proximity of the WoE values if more groups are formed than the initially specified maximum number for a given predictor. For interval factors, the order of the groups ordered by the factor values is also taken into account. Factors that have only one group left after associations are transferred to the list of excluded ones. After each merger, you need to return to the second paragraph.

[102] In some embodiments, monotonicity, nonmonotonicity conditions, and risk directions for interval variables are checked in accordance with a manual. Factors that do not meet the conditions from the directory go into the list of excluded.

[103] In some embodiments, the minimum number of groups is checked. If the variable has fewer groups than the initially specified minimum number, then it goes into the excluded list.

[104] In some embodiments, the implementation of the check conditions of the ratio of risk in different groups for categorical and binary variables in accordance with the directory (verification of business logic). Conditions are set using a special language that allows you to describe patterns of risk ratios in groups of any complexity. Factors that do not meet the conditions from the directory go into the list of excluded.

[105] In some embodiments, a drop in the Gini coefficient is checked. If the given coefficient according to the predictor in the validation sample is less than the initially set threshold value or falls compared to the Gini coefficient in the training sample by more than a specified number of percent, then this factor goes into the excluded list.

[106] In some embodiments, the stability of the order of groups ordered by WoE is checked. A comparison of the training sample and 20 samples randomly selected from the combination of the training and validation is performed. Factors for which instability in the order of the groups ordered by WoE is detected is transferred to the list of excluded ones.

[107] For factors that are on the list of excluded, you must run the algorithm for splitting the values of factors with a new set of settings. If there are no available settings or all of them have already been checked, then the formation of partitions is considered complete. The number of settings is determined by the capabilities of the statistical package used, for example, based on SAS Enterprise Miner. Thus, according to the results of the application of the algorithm, WoE factors are formed. Initial factors that fail the test with any set of split settings are excluded from the process.

[108] Step 104: exclude from the samples at least one transformed factor that correlates with at least one other factor.

[109] Pair correlation analysis is used to identify collinear relationships between variables. The presence of correlations between factors increases the standard deviations of the regression coefficients, which reduces their stability and reliability in multivariate analysis. For correlation analysis, the correlation matrix is calculated — a table with the values of the pair correlation coefficients of the transformed WOE factors. The analysis of this table allows us to determine variables that have high linear relationships with other factors. The value starting from which the correlation coefficients are considered high is set in the directory. The recommended value, starting from which the correlation coefficients are considered high, is in the range from 0.5 to 1 modulo. Of each pair of correlating factors, only one should be left on the basis of either a higher individual predictive ability or a greater importance of the factor from the point of view of business logic. The following algorithm is used in the system: in the cycle, the factor that has the largest number of factors correlated with it is selected (the correlation value is above the selected threshold). If there are several, then the factor with the lowest Gini index is selected from them. This factor is excluded from consideration. After that, the next factor is selected with the largest number of remaining factors correlated with it and the lowest value of the Gini index. Thus, factors without correlations above the selected threshold remain at the exit from the cycle. This approach provides the largest number of uncorrelated factors in the final list of factors for modeling.

[110] Step 105: form a credit scoring model by training binary multiple logistic regression, having the following form:

where Y is the dependent variable (sign of default), Y = 1 is the event of default, X ₁ , X ₂ , ..., X _n is the set of independent explanatory WOE factors, β ₀ , β ₁ , β ₂ , ..., β _n - logistic regression coefficients, PD - probability of default.

[111] The values of the probability of default (PD - Probability of Default) are in the interval [0, 1]. It shows the probability of default for each calculated rating.

[112] In some embodiments, the default probability values may range from 0 to 100 in percentage or numerical terms.

[113] Despite the absence of correlating pairs excluded in the previous step, multicollinearity can occur between the factors of the scoring model, therefore, at the stage of constructing the scoring model, it is necessary to check its absence. In addition, since the scoring model is developed based on WoE factors, and the more WoE, the less risk, it is necessary to check the correctness of the coefficient sign in the scoring model (all regression coefficients must be negative). In addition, it is necessary to ensure high stability of the model, so the significance of each of the factors included in it is checked using the statistical bootstrap procedure: each of the factors should be significant based on Wald statistics in at least 85% of cases. The method of forming the final scoring model is as follows.

[114] Based on all the factors that have reached this stage, a logistic model is constructed using stepwise regression (stepwise) to select the final set of factors.

[115] For such factors, the Variance Inflation Factor (VIF) is calculated. To determine VIF, it is necessary to evaluate a linear regression model, where the factor under consideration will be the factor under consideration, and the remaining factors included in the model will act as independent variables . The final VIF value for the factor can be found by the formula:

where R ² is the coefficient of determination of the model described above. A variable whose VIF value is greater than the specified value and the value of the Gini coefficient is minimal is excluded. The first and second steps are repeated until all factors included in the model have a VIF value lower than the specified value.

[116] Then a check is made for the presence of factors with a positive sign of the regression coefficient. If they are detected, the factor is excluded with the minimum value of the Gini coefficient, after which it is necessary to return to the first step. If there are no such factors, then go to the next paragraph.

[117] Next, the combination of training and validation samples is carried out. Several dozen samples of the same size as the training one are randomly selected from their association. On each of the obtained samples, a scoring model is trained with the current set of factors. If there are factors that are significant, according to Wald's statistics, in less than 85% of cases, then the one with the least Gini coefficient is excluded. After the exception, you must return to the first step. If there are no such factors, then the scoring model is considered to be successfully built.

[118] Thus, the algorithm allows the automatic development of scoring models that meet all reasonable quality requirements. In addition, he guarantees that each factor will correspond to the business logic described in a special guide.

[119] In some embodiments, the model is automatically validated in accordance with any statistical model validation technique known in the art. At this stage, quantitative tests are calculated to assess the quality of the model. The validation process uses the test sample generated in step 102 and a population of data. In the case of validation, proceed to the next step, otherwise a notification will be sent to the system user that the validation has not been completed, as well as a detailed report on the identified deficiencies. By varying the algorithm settings, the user can adjust the modeling approaches and ensure the success of the next validation.

[120] The choice of the optimal cut-off threshold value depends on the price of making errors of the first and second kind in the classification. The model should more accurately classify “bad” borrowers, as in credit scoring, the price of a pen-type error is higher. With a decrease in the cutoff threshold, the sensitivity will increase in the model, i.e. the ability of the model to correctly identify those borrowers who will have late payment. For the optimal cutoff threshold, you can take the balance point between sensitivity and specificity.

[121] Step 106: automatically select clipping zones for at least one scoring model for its installation in the credit procedure.

[122] Next, automatic selection of cut-off zones for scoring models by scoring points is carried out for their installation in the credit procedure. The algorithm for selecting cut-off zones consists of two parts: external and internal. The external part is responsible for iterative selection of cutoff levels, the internal part is for calculating the expected level of approval of a loan application corresponding to the current set of cutoffs. It is worth noting that the criterion for the internal part of the algorithm can be not only the level of approval, but any indicator of interest that depends on cut-off levels, for example, the risk level or portfolio NPV. The algorithm works on a historical sample of data on loan applications. Due to the fact that the approval level is characterized by seasonality within the week, this technical solution deals with the target approval level only within seven days, as otherwise, you have to define it separately for each day of the week. Based on this, the number of days for which the history of applications is considered should be a multiple of seven. Suppose that in the process of making a credit decision a combination of three models is used:

1. The quality of credit history, or scoring credit bureaus (BKI scoring);

2. Personal data (application scoring);

3. The tendency to fraud, or FDC-scoring (Fraud Detection Card Scoring).

[123] Suppose we have a combination of cut-off points for bid, FDC, and BKI scoring models. Let (t1, t2, t3) be the value of adjustments for cutoffs for the corresponding models, and (n1, n2, n3) be the number of consecutive repetitions of the adjustment for each of the corresponding models. Then the external algorithm for selecting cutoff points will be as follows. Consistently for each of the scoring models, it is necessary to carry out the following actions:

1. add the appropriate t adjustment from (t1, t2, t3) to the cutoff level for this model;

2. run the internal part of the algorithm described below to calculate the expected level of approval;

3. if the deviation of the expected level of approval has changed direction, then choose a combination of cut-off levels according to the application, FDC and BKI scoring models, in which the deviation of the expected level of approval is the smallest (in fact, the choice is made from the last two tested combinations);

4. if the deviation of the expected level of approval from the target did not change direction and the first paragraph was repeated less than n from (n1, n2, n3) times, then go to the first paragraph, i.e. to adjust the next scoring model.

[124] In some embodiments, the above procedure is repeated until a target level of approval is obtained or an upper / lower score is reached for each of the models.

[125] The internal part of the algorithm evaluates the change in the level of approval when changing the cutoff points for working scoring models. As noted earlier, the effect of changing cut-off zones can be evaluated on various indicators, whether it is risk or profitability, but in any case, it is necessary to assess who will be approved under the new cut-off zones and who is rejected (or the probability of one of these events). In this regard, an algorithm for assessing changes in the level of approval will be considered.

[126] As a rule, a decision-making system (DSS) in a financial-credit organization is a sequence of checks and application of the rules and may include the following stages of the application:

1. failure according to the minimum requirements, based on data from the Hunter system, stop list, etc .;

2. use of application, BKI and FDC scoring;

3. application of reliability models;

4. underwriting;

5. failures in subsequent stages.

[127] For this reason, in order to assess the level of approval in case of redefinition of actual scoring refusals on applications, it is necessary to know the decision on them at each of the stages following the second stage (using scoring). For any application approved for all working scoring models (application, BKI, FDC scoring, etc.), the necessary information is available on the process of its passage through the subsequent stages of the DSS. For applications for which a rejection of at least one of the models is received, uncertainty arises with respect to the subsequent stages, since such applications do not reach these stages. In order to eliminate this uncertainty, within the framework of the algorithm, failure modeling is performed after the scoring stage for applications for which a failure has previously been received. The algorithm can be represented as a sequence of the following actions.

1. For applications that have reached the scoring stage, failures are simulated for three types of models with new cutoff points. All applications for which an actual rejection was received at the scoring stage and approval for all models during the simulation are marked (they need a separate simulation of the probability of failure at the subsequent stages of the DSS).

2. The simulation of the probability of failure at the stage of application of the reliability model is carried out. To build the model, applications are used that successfully passed the scoring procedure before changing the cutoff points.

3. The simulation of the probability of failure at the stage of underwriting. To do this, in addition to the previous sample, applications are excluded for which a refusal was received at the stage of application of reliability models.

4. The simulation of the probability of failure at subsequent stages is carried out. Additionally excluded are applications for which a rejection has been received at the underwriting stage.

5. The probability of failure after passing the scoring procedure is calculated for marked applications requiring separate modeling (paragraph 1).

[128] In order to determine the probability of failure for applications for which a failure was received at the scoring stage, the following formula is used:

where P _blag is the probability of failure for the application according to the reliability model; P _underr - probability of failure for the application at the underwriting stage; P _next - the probability of failure for the application in subsequent stages.

[129] To determine the likelihood of approval on an application, the probability of rejection is deducted from the unit. After that, the approval level is calculated as the ratio of the number of approved applications (the sum of the probabilities of approval) to the number of all applications. By averaging this value over the portfolio under consideration, a level of approval is obtained for the selected cut-off zones.

[130] If you select a risk level for the target indicator, then the obtained value must be multiplied by the risk level obtained from the PD model. When averaging this product, the level of risk in the portfolio issued for the selected cut-off zones is obtained.

[131] After the automatic selection of cut-off zones, these zones are optimized for different segments of the portfolio. The principle of operation of the optimization algorithm is based on an iterative optimization calculation of optimal decision thresholds for individual customer segments from the point of view of the ratio “Approval Level - Risk Level”. Below are the main premises that are critical for the obtained results of the algorithm:

1. The risk level is estimated as the average level of probability of delay in each segment.

2. A forecast of the probability of delay is made on the latest available data, taking into account segmentation.

[132] The main idea of the calculation algorithm is an iterative shift of the cutoff threshold for an individual client segment, which ultimately leads to an increase in the overall level of approval while maintaining the current level of risk.

[133] At each iteration of the algorithm, the client segment that is optimal from the point of possible improvement of the AR / DR ratio is considered, within which there are “spin” - “spin” operations in this sequence with a predetermined step of 15 points (this step can be increased in accordance with the rules formation of the cycle, but not more than 60 points). Thus, an optimal neighborhood of the base cutoff threshold is sought, leading to an improvement in the overall AR / DR ratio.

[134] Next, the scoring model (or models) and cut-off zones are introduced into the industrial environment.

[135] As a result of automatic selection of cut-off levels, the target level of approval may not be achieved with the required accuracy. As a result, it is necessary to adaptively adjust clipping according to scores to maximize approximation to the target AR. To do this, 7 full days after the last change in the target approval level or immediately after adjusting the cut-off points without changing the target approval level, adaptive adjustment of the obtained cut-off points begins. It continues until the actual level of approval falls within acceptable limits at least once. Correction is performed as follows. Add to all the clipping zone the following value:

where Δ = AR-high _AR , if the last exit of AR beyond the established boundaries occurred upwards;

where Δ = low _AR -AR, if less. The size of the adjustment is set expertly in the directory empirically.

[136] Adjustments are triggered daily until the approval level returns to the acceptable interval between high _AR and low _AR .

[137] In some embodiments, a monthly automatic validation of the scoring model is carried out in accordance with the methodology adopted by the financial institution. If the model does not pass validation, it is sent for retraining.

[138] In some embodiments, the level of approval is monitored daily. This methodology is suitable for monitoring not only the level of approval, but also other indicators, such as risk. In the framework of monitoring, the average moving approval level with a window of 7 days is considered as a time series, the elements of which are modeled using independent normally distributed random variables. In order to maintain the level of approval at any target level, first of all, a criterion is needed, with which you can understand that the change really happened, since this indicator has natural fluctuations. Based on this, a CUSUM test can be used to detect deviations in the target approval level. For this, it is determined that there is a moment of change in the level of approval (disorder) - this is the moment when the law of distribution in the flow of incoming data about the level of approval changes. This technical solution considers the change in the average value. Let X _n , n≥1 be a sequence of observations that are modeled using independent normally distributed random variables, θ∈ [1, n] be an unknown point in time at which the distribution of observations changes from ƒ ₀ ~ N (μ ₀ , σ ² ) on ƒ ₁ ~ N (μ ₁ , σ ² ), n is the current time. Since the exact time of the breakdown is not known, the hypothesis of H ₀ - breakdown on the interval [1, n] is not, and H ₁ - breakdown occurred on the segment [1, n]. To distinguish between these two hypotheses, it is necessary to determine a generalized likelihood ratio criterion:

[139] where with the help of C ₀ the number of false positives is controlled. This expression is known as the CUSUM test. The resulting test record will be computationally inefficient, but in the case of independent random variables, statistics can be represented by a recurrence relation:

[140] Since we assume that ƒ ₀ and ƒ _{1 are} distributed normally:

[141] Let μ ₁ = μ ₀ ± δ, where δ is the permissible error, which is selected depending on which deviation we consider acceptable. Then the expression for calculating CUSUM can be rewritten as:

- для отклонений в сторону снижения и

- for deviations downward and

- для отклонений в сторону увеличения.

- for deviations upward.

[142] The final decision is found from the condition

[143] The described approach allows detecting deviations of the approval level from the target level with a minimum delay and a small number of false positives.

[144] If the change in the target approval level occurred less than 7 days ago, we cannot conduct the CUSUM test, as there are no observations of the moving average approval level for 7 days, not including the days before the adjustment. In addition, you need to insure yourself against the incorrect operation of the CUSUM test. For this, an alternative, simpler test is used, based on setting the boundaries of the acceptable range for the observed indicator.

[145] Aspects of the present invention can also be implemented using a data processing device that is a computer or system (or tools such as a central / graphics processor or microprocessor) that reads and executes a program recorded on a storage device to perform the functions of the above Embodiment (s) and the method shown in FIG. 1, the steps of which are performed by a computer or device by, for example, reading and executing a program recorded on a memory device to perform the functions of the above-described embodiment (s) of implementation. To this end, the program is recorded on a computer, for example, through a network or from a recording medium of various types, serving as a storage device (for example, a computer-readable medium).

[146] The data processing device may have additional features or functionality. For example, the data processing device may also include additional data storage devices (removable and non-removable), such as, for example, magnetic disks, optical disks or tape. Storage devices may include volatile and non-volatile, removable and non-removable media implemented in any way or using any technology for storing information, such as machine-readable instructions, data structures, program modules or other data. A storage device, removable storage, and non-removable storage are examples of computer storage media. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact ROM a disc (CD-ROM), universal digital disks (DVDs) or other optical storage devices, magnetic tapes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that may be used to store the desired information and to which the data processing device can access. The data processing device may also include an input device (s), such as a keyboard, mouse, pen, voice input device, touch input device, and so on. An output device (a), such as a display, speakers, printer, and the like, may also be included in the system.

[147] The data processing device comprises communication connections that allow the device to communicate with other computing devices, for example over a network. Networks include local area networks and wide area networks along with other large, scalable networks, including, but not limited to, corporate networks and extranets. Communication connection is an example of a communication environment. Typically, a communication medium can be implemented using computer-readable instructions, data structures, program modules or other data in a modulated information signal, such as a carrier wave, or in another transport mechanism, and includes any information delivery medium. The term "modulated information signal" means a signal, one or more of its characteristics are changed or set in such a way as to encode information in this signal. By way of example, but without limitation, communication media include wired media such as a wired network or a direct wired connection, and wireless media such as acoustic, radio frequency, infrared, and other wireless media. The term “machine-readable medium”, as used herein, includes both storage media and communication media. The process sequences described in this document may be performed using hardware, software, or a combination thereof. When processes are performed using software, the program in which the sequence of processes is recorded can be installed and can be executed in the memory of a computer built into specialized hardware, or the program can be installed and can be run on a general-purpose computer that can perform various processes .

[148] For example, a program may be pre-recorded on a recording medium such as a hard disk or ROM (read only memory). Alternatively, the program can be temporarily or permanently saved (recorded) on a removable recording medium such as a floppy disk, CD-ROM (compact disc, designed for playback only), MO (magneto-optical) disc, DVD (digital universal disc) , magnetic disk or semiconductor memory. Removable recording media may be distributed as so-called software sold through a retail network.

[149] The program may be installed from a removable recording medium described above to a computer, or may be transmitted via cable from a download site to a computer, or may be transmitted to a computer via network data channels such as a LAN (local area network) or The Internet. The computer can receive the program transmitted in this way and can install it on a recording medium such as an internal hard drive.

[150] The processes described in this document can be performed sequentially in time, in accordance with the description, or can be performed in parallel or separately, depending on the processing characteristics of the device that performs the processes, or in accordance with the need. The system described in this document is a logical set of multiple devices and is not limited to the structure in which these devices are installed in one enclosure.

Claims (16)

1. A computerized way to develop and manage scoring models, which includes the following steps: - receive data for a given period of time containing factors affecting the scoring model; - carry out the division of the data into samples for the development, validation and testing of the scoring model; - carry out the transformation of factors by establishing relationships between groups of values of the converted factor and default levels; - exclude from the samples at least one transformed factor that correlates with at least one other factor; - form a credit scoring model through the training of binary multiple logistic regression; - automatically select clipping zones for at least one scoring model for its installation in the credit procedure. 2. The method according to p. 1, characterized in that they receive data for a given period of time from the user's mobile communication device. 3. The method according to claim 1, characterized in that when dividing the obtained data into samples, parts of the original population that do not intersect in time or random subsamples are obtained. 4. The method according to claim 1, characterized in that the factors affecting the scoring model are annual income and / or the amount of outstanding debt, and / or ownership of real estate, and / or possession of a car, and / or work experience in last place, and / or age. 5. The method according to p. 1, characterized in that the factors affecting the scoring model are discrete or continuous. 6. The method according to p. 1, characterized in that during the transformation of factors determine the degree of deviation of the level of defaults in the data group from the average level of defaults throughout the sample. 7. The method according to claim 1, characterized in that when transforming the factors according to the factors included in the excluded list, the algorithm for splitting the values of the factors with a new set of settings is launched. 8. The method according to p. 1, characterized in that, when excluding transformed factors from the samples, a table is formed with the values of the pair correlation coefficients of the transformed factors. 9. The method according to p. 1, characterized in that when excluding transformed factors from the samples in the cycle, the factor that has the largest number of factors correlated with it is selected. 10. The method according to p. 1, characterized in that when forming the credit scoring model, a logistic model is built using step-by-step regression to select the final set of factors.

Images