TWM622331U - System and device for risk prediction therefor - Google Patents

Abstract

A system and a device for risk prediction are provided. The system and the device include: generating a plurality of models by using a plurality of data sets of a training group and a plurality of algorithms after training, wherein each data set includes a plurality of training data and each model is trained and generated by using one of the data sets and one of the algorithms; inputting each validation data of a validation group into each model in order to calculate a score of each data set based on an output value of each model for each validation data; selecting the data set with the highest score as the optimal set; and predicting a risk of a real-time data by using the models of the optimal set.

Description

Risk prediction system and its equipment

This creation relates to risk prediction technology based on artificial intelligence, and in particular, relates to a system and equipment for risk prediction.

For risk judgment, for example, in some product inspection fields, the original design is based on historical records, and logical conditions are set to judge the risk level, and then determine the probability of random inspection. For example, under normal circumstances, the probability of random inspection is low, such as below 20%; if there is a failure in the historical record, the probability of random inspection will be increased to between 20% and 50%; Batch sampling, the sampling probability is 100%.

Therefore, there is a need for automated risk prediction technology to automatically predict risks based on real-time data based on various data about risks to improve efficiency and reduce costs.

These techniques can be used in most areas of risk forecasting. At present, there are automatic risk predictions using artificial intelligence models. However, the existing model predictions are mostly the application of a single algorithm, and the possible impact of the data combination used for training the model itself is ignored. Therefore, this technology is still inefficient and accurate. low accuracy and high cost.

In order to solve the above problems, this creation provides a risk prediction system, including a training unit, a score unit, a selection unit and a prediction unit. The training unit is used to generate a complex number model after training using the complex number data combination in the training set and the complex number algorithm, wherein each data combination includes complex number training data, and each the model uses one of the data combination and the complex number model. One of the other algorithms is generated by training; the fractional unit is coupled to the training unit, and is used to input each verification data in the verification group into each of the models, and calculate the output value of each of the verification data according to each of the models. The score of each data combination is obtained; the selection unit is coupled to the score unit, and is used to select the data combination with the highest score among the data combinations as the best combination; the prediction unit is coupled to the selection unit for using the The best combination of these models to predict the risk of real-time data.

The present invention also provides a risk prediction device, which has a processor and stores instructions to execute the content of the above unit.

This creation uses a variety of different algorithms to ensemble models, and through a variety of different data combinations, to find the best data combination for risk prediction, so as to avoid the prediction bias of only using a single algorithm in the past, and Solve the problem of ignoring the impact of the data portfolio itself, so as to reduce the cost of risk response or processing, while taking into account the hit rate of risk prediction, and improve the overall efficiency.

300: Risk Prediction System

310: Data Unit

320: Training Unit

330: Fractional Unit

340:Select unit

350: Prediction Unit

S101~S112, S201~S205: method steps

FIG. 1 is a flow chart of a model building procedure of a risk prediction method.

FIG. 2 is a flow chart of a prediction procedure of a risk prediction method.

Figure 3 is a block diagram of a risk prediction system.

The following describes the implementation of the present invention by means of specific embodiments, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification.

FIG. 1 is a flow chart of a model building procedure of a risk prediction method.

In step S101, data collection and consolidation of the field targeted by the risk prediction method is performed. These areas may be products, goods, services or other areas that involve risk. For example, if the field is electronic products, the risk may be whether the electronic product complies with safety regulations or various other regulations; if the field is loan services provided by financial institutions, the risk may be whether the loan applicant can normally repayment risk.

In addition, the above-mentioned data may include various data related to risks in this field, such as relevant data in the internal information systems of various organizations, customer data, the above-mentioned products, commodities, services or other fields of information, as well as external openings such as weather data The above-mentioned organizations can be government agencies, foundations, foundations and other non-governmental organizations involved in this field, or organizations such as company bank numbers; the above-mentioned customers can be customers of the above-mentioned organizations or customers who use the risk prediction method.

For example, in the field of whether the financial industry provides personal credit, the above-mentioned information may include the individual's gender, age, education level, monthly income, annual income, residential area, as well as the educational level of the residential area, the number of public facilities and so on.

Furthermore, in step S101, the correspondence of multiple data sources is performed, that is, the data related to the same object from each data source are merged into the same data, and then all the data are assembled into a unified format. For example, in the field of financial credit, each object can be a loan application; in the field of electronic products, each object can be the same batch of imported or the same batch of electronic products to be inspected, and so on.

In step S102, pre-processing of the above-mentioned data is performed, for example, the simplified version of the same character is corrected to the traditional version, or the residential areas of 368 townships and urban areas in Taiwan are simplified into high, medium and low risk areas, etc. deal with. In this step, various relevant information required for training the artificial intelligence model in the above data are also defined as features, that is, variables that may have an impact on the risk prediction results.

In step S103, the data processed in steps S101 and S102 are divided into training group, verification group and test group, wherein the training group is used to train the model, the verification group is used to find the best data combination, and the test group is used to Benefit evaluation of this risk prediction method. In one embodiment, the training group, the verification group, and the test group can be used according to the usage time of each piece of data (that is, the time when each piece of data in the historical record is used for risk-related inspection or evaluation, such as a certain year, a certain month, and a certain day). Differentiate, for example, the training group is the earliest, the validation group is in the middle, and the test group is the latest. Usually the training group contains the most data, for example, the number of data in the above three groups can be 8:1:1. In addition, in order to emphasize or distinguish the data in each group, the data in the training group, the validation group, and the test group may be referred to as training data, validation data, and test data, respectively.

The training data in the training group can be divided into multiple data sets according to their usage time and oversampling methods. For example, the four data sets in Table 1 below correspond to the two data sets starting from 2015 and starting from 2018, respectively. Different data usage times and four combinations of two different resampling methods, namely proportional enlargement and synthetic minority oversampling technique (SMOTE).

Depending on the source of the obtained data, the integrity and reliability of the data may lead to differences in the results of different data combinations in different algorithms. Therefore, the data combination can be combined according to the time of use of the data. For example, the data owned by the organizations using the above risk forecasting method were collected in 2015, but most of the external data obtained were relatively complete after 2018. Therefore, the data usage time can be divided into data from 2015 and data from 2015. Data from 2018 (data from 2015 includes data from 2018). At the same time, among the target variables, there are usually a small number of high-risk data. For example, the ratio of low-risk and high-risk data may be 10:1 or 8:1. Therefore, it is necessary to increase the ratio of high-risk data through unbalanced data processing, for example, to 5 : 5 or 7: 3, so that the algorithm has better learning effect. Therefore, different resampling methods for dealing with unbalanced data, such as proportional upscaling and SMOTE, are another aspect of the combination, so there are a total of four training group data combinations shown in Table 1.

The present creation is not limited to having only four training set data combinations. For example, in another embodiment, the training set may include T*O data combinations, which correspond to T different data usage times and O different resampling methods respectively. There are T*O different combinations of , and each data combination includes a plurality of training data, wherein T and O are both integers greater than one. In other words, each data combination is composed of training data starting from one of T data usage times, and is generated according to one of O resampling methods.

In step S104, feature screening is performed on the training group to determine features included in model training. The feature screening refers to removing features that are useless for model training. For example, if the risk of a product has nothing to do with the capital of the manufacturing company, the capital of the manufacturing company is removed and not used for subsequent model training.

In step S105, a complex number model is generated after training using the complex number data combination in the training set and the complex number algorithm. In practice, because of the random component of resampling, for each data combination and each An algorithm needs to use the data combination and the algorithm to train to generate N models, so as to have a relatively stable prediction result.

In addition, for the above N models, different training data are randomly generated for each model. Therefore, if there are S types of data combinations (S=T*O) and A types of algorithms, a total of S*A*N different models are generated by training in this step, where A and N are both integers greater than one. For example, in one embodiment, S is equal to 4 (that is, the four data combinations shown in Table 1), A is equal to 5, and the five algorithms are logistic regression and decision tree respectively. C5.0, classification and regression tree (CART), naive Bayes classifier, and random forest, and N is equal to 10, so in step S105 a total of 200 different Model.

In step S106, the next verification data in the verification group is read, and if this step is performed for the first time, the first verification data in the verification group is read.

In step S107, the read verification data is input into each model generated by the training in step S105, so as to generate the output values of the verification data for each model of each data combination. For example, if S equals 4, A equals 5, and N equals 10, then each of the 4 data sets has 50 models for 5 algorithms that are fed into the validation data and produce corresponding outputs value.

In step S108, according to the output value of the verification data of each model of each data combination, a prediction result of each algorithm of each data combination for the verification data is generated, wherein, for each data combination and each algorithm, the data combination The prediction result of the algorithm for the verification data is the average of the output values of the data combination and the models corresponding to the algorithm for the verification data. For example, if S equals 4, A equals 5, and N equals 10, then there are 50 models for each of the 5 algorithms for each of the 4 data sets, and the prediction for each algorithm for each data set The result is the average of the 10 outputs of the 10 models of the algorithm for the validation data.

In step S109, the prediction results of the verification data are voted on according to the algorithms of each data combination to generate a voting result of the verification data for each data combination.

In detail, for each data combination, if more than half of the prediction results for the verification data by the algorithms of the data combination are greater than the corresponding threshold The voting result is high risk, otherwise it is low risk. For example, if S equals 4 and A equals 5, then for each of the 4 data combinations, if at least 3 of the 5 prediction results of the 5 algorithms are greater than the corresponding threshold, then the The voting result of the data combination for the verification data is high risk, otherwise it is low risk. The thresholds corresponding to each algorithm can be the same or different. In another embodiment, there may be at least two risks of voting results depending on the detection results.

In step S110, it is checked whether the verification materials in the verification group have been processed. If there is still unprocessed verification data in the verification group, the flow returns to step S106, otherwise the flow goes to S111.

In step S111, for each data combination, the score of the data combination is calculated according to the voting result of the data combination for each piece of verification data.

In detail, a confusion matrix is used to evaluate each data combination based on the historical records and voting results of each verification data. For example, the historical record of the subject of each piece of verification data being inspected or evaluated may be qualified or unqualified, and the qualified and unqualified historical records may correspond to the low risk and high risk of the voting result, respectively. In addition, using confusion matrix to analyze the relationship between historical records and voting results, the accuracy, recall, specificity, positive predictive value, The negative predictive value (negative predictive value) and the final F1 score (F1 score/measure) are shown in Table 2 below, where the F1 score is the above score of each data combination.

In step S112, the data combination with the highest score among the data combinations is selected as the best combination. For example, in the two fields shown in Table 2, the data combination corresponding to 2015 and proportionally enlarged in the X field has the highest score of 0.49, so this data combination is selected as the best combination in the X field. On the other hand, the data combination corresponding to 2018 and proportionally enlarged in the Y field has the highest score of 0.45, so this data combination is selected as the best combination in the Y field. The optimal combination includes A*N different models in total. From the evaluation results shown in Table 2, it can be seen that through the same various algorithms, but different combinations of training data, the effects in different fields are also different.

FIG. 2 is a flow chart of a forecasting procedure of a risk forecasting method, the forecasting procedure continuing the model building procedure of FIG. 1 .

In step S201, real-time data is obtained, wherein the real-time data is a piece of risk-related data of an object to be predicted in the application field of the risk prediction method, and the format and content of the real-time data correspond to the above-mentioned training data and inspection data. For example, in the field of financial credit, the object may be a loan application.

In step S202, the preprocessing of the real-time data is performed, and the preprocessing of this step is the same as that of step S102.

In step S203, the risk of the real-time data is predicted using the best combination of the models selected in step S112.

Specifically, the real-time data is input into each model of the optimal combination to generate an output value of the real-time data for each model of the optimal combination. Then, for each algorithm, the average value of the output values of each model of the algorithm in the optimal combination is calculated as the prediction result of the algorithm for the real-time data, and then the real-time data is calculated according to each algorithm. The prediction results of the data are voted to predict the risk of the real-time data, wherein if more than half of the prediction results of the real-time data by the algorithms are greater than a threshold, the real-time data is high risk , otherwise it is low risk.

In step S204, the voting result (high risk or low risk) and related records of the real-time data are written into the database.

In step S205, it is checked whether the voting result is high risk, and if it is high risk, an early warning is issued. For example, a short message or an email can be sent as a warning to notify the management personnel, or an early warning can be issued by means of sound, voice and/or light, etc. ; otherwise, a general notice or no processing will be issued.

Regarding the voting results of real-time data, various responses can be made according to the characteristics of the field. For example, in the field of financial credit, if the voting result is high risk, the loan application corresponding to the real-time data can be strictly reviewed, and vice versa.

In addition, the model of the optimal combination can also be used to perform risk prediction on the test data in the test group to evaluate the effectiveness of the risk prediction method. For example, for product sampling inspection, it is possible to evaluate how much the risk prediction method can reduce the sampling inspection cost, and whether it can improve the hit rate of inspection problems in the sampled products.

In another embodiment, the present invention further provides a risk prediction device, which is applied or set in a computing device or a computer. For example, the risk prediction device may have a memory, a floppy disk, a hard disk or an optical disk. In addition, the risk prediction device stores instructions to execute the risk prediction method or to execute the content of each unit shown in FIG. 3 .

In another embodiment, the present creation further provides a risk prediction system, such as the risk prediction system 300 shown in FIG. 3 . The risk prediction system 300 includes a data unit 310 , a training unit 320 , a score unit 330 , a selection unit 340 and a prediction unit 350 . These units are coupled in series in the above order, and each unit can be implemented by hardware, firmware or software.

The risk prediction system 300 executes the above-mentioned risk prediction method, wherein the data unit 310 is used to execute steps S101 to S104 in FIG. 1 , including data collection and aggregation, data preprocessing, dividing data into training groups, and verification groups With the test group and feature screening, the training unit 320 is used to perform step S105, including training and generating models, and the scoring unit 330 is used to perform steps S106 to S111, including reading the next verification data, and generating each model pair for each data combination. To verify the output value of the data, to generate the prediction result of each data combination on the verification data, to generate the voting result of each data combination to the verification data, to verify whether the data has been processed, and to calculate the score of each data combination, select unit 340 is used to execute step S112, including selecting the best combination, and the prediction unit 350 is used to execute steps S201 to S205 in FIG. 2, including obtaining real-time data, pre-processing the data, and predicting the risk of real-time data using the model of the best combination , write to the database, and issue an alert.

To sum up, this creation finds the best data combination for risk prediction through the robust integration architecture of various different algorithm models and comprehensive cross-evaluation of various data combinations, thereby improving the performance of model prediction and avoiding the traditional Use the prediction bias of a single algorithm and solve the problem of ignoring the impact of the data portfolio itself to reduce risk response or processing costs, while providing accurate risk predictions and improving overall efficiency.

The above-mentioned embodiments are only used to illustrate the principles and effects of the present invention, but are not intended to limit the present invention. Anyone with ordinary knowledge in the technical field can modify and change the above-mentioned embodiments without departing from the spirit and scope of the present creation. Therefore, the protection scope of the rights of this creation should be listed in the patent application scope described later.

300: Risk Prediction System

310: Data Unit

320: Training Unit

330: Fractional Unit

340:Select unit

350: Prediction Unit

Claims (9)

A risk prediction system comprising: a training unit for training complex data sets and complex algorithms in the training set to generate complex models, wherein each data set includes complex training data, and each of the models uses one of the data sets and one of those algorithms is the result of training; a score unit, coupled to the training unit, for inputting each verification data in the verification group into each of the models, so as to calculate the scores of each of the data combinations according to the output values of each of the models for each of the verification data; a selection unit, coupled to the score unit, for selecting the data combination with the highest score among the data combinations as the best combination; and The prediction unit, coupled to the selection unit, is used for predicting the risk of real-time data using the models corresponding to the optimal combination. The risk prediction system of claim 1, wherein each data combination is composed of the training data starting from one of plural data usage times, and each of the data sets is generated according to one of plural resampling methods Data combination. The risk prediction system of claim 1, wherein, for each of the data sets and each of the algorithms, there are plural sub-models in the models corresponding to the data sets and the algorithms. The risk prediction system of claim 1, wherein the fractional unit is multiplexed to: According to the output value of each of the models for each of the verification data, the prediction results of each of the algorithms for each of the data combinations for each of the verification data are generated; for each of the verification data, voting on the prediction results of the verification data according to the algorithms for each of the data combinations to generate a voting result for the verification data for each of the data combinations; and For each data combination, the score for the data combination is calculated based on the voting results of the data combination on the verification data. The risk prediction system of claim 4, wherein, for each of the data combinations, each of the algorithms, and each of the verification data, the prediction result of the verification data by the algorithm of the data combination is the data combination and the verification data. The average value of the output values of the verification data of the sub-models corresponding to the algorithm. The risk prediction system of claim 4, wherein, for each of the data combinations and each of the verification data, if the prediction results of the verification data by the algorithms of the data combination have more than half of the prediction results If it is greater than the threshold, the voting result of the data combination for the verification data is high risk, otherwise, it is low risk. The risk prediction system of claim 1, wherein the prediction unit is multiplexed to: inputting the real-time data into each of the models of the optimal combination to generate prediction results for the real-time data of the algorithms based on the output values of the real-time data of each of the models of the optimal combination; and The prediction results of the real-time data are voted on according to the algorithms to predict the risk of the real-time data. The risk prediction system as described in claim 7, wherein if more than half of the prediction results of the real-time data by the algorithms are greater than a threshold, the real-time data is high risk, otherwise, the real-time data is considered as high risk. low risk. A risk prediction apparatus having a processor storing instructions to execute the content of the unit of the risk prediction system as claimed in any one of claims 1 to 8.

Images