KR102499182B1 - Loan regular auditing system using artificia intellicence - Google Patents

Abstract

In the household loan fraud/insolvency regular audit support system using artificial intelligence according to an embodiment of the present invention, when a user submits a predetermined document to apply for household loan fraud/insolvency, the credit rating and the possible amount of household loan fraud/insolvency In addition to the household loan fraud/insolvency system that conducts fraudulent/insolvent transactions in household loans by investigating fraudulent/insolvent transactions, etc. An audit information system that conducts regular audits in consideration of the risk rate of insolvent transactions and its basis items, and supports efficient business audits for high-risk groups, and the first database of the above household loan fraud / insolvency system through wired and wireless Internet Collect data, perform preprocessing for learning, analyze, learn, and model data to create and store an abnormal transaction prediction model with the help of an audit terminal, and use the abnormal transaction prediction model to perform the household loan fraud/insolvency system Household loan fraud/non-performing transaction data is predicted to be an abnormal transaction and provided to the audit information system, so that the audit terminal detects household loan fraud/insolvency using the fraud/insolvent household loan fraud/insolvency detection monitoring of the audit information system. It is characterized by including an artificial intelligence audit analysis system that enables efficient regular auditing for insolvency.

Description

Household loan fraud/insolvency regular audit support system using artificial intelligence {LOAN REGULAR AUDITING SYSTEM USING ARTIFICIA INTELLICENCE}

The present invention relates to a household loan fraud/insolvency regular audit support system using artificial intelligence. It is possible to develop a machine learning prediction model and an automatic business rule learning system, to research and develop new types of abnormal transactions that are not found in existing rules, and to develop a connection between an analysis server equipped with machine learning and an audit system. It is about a household loan fraud/insolvency regular audit support system using artificial intelligence.

In general, audits were carried out on a one-to-one basis for each audited organization by detecting and preventing various accidents such as embezzlement, misappropriation, or negligence of audited organizations such as branch offices.

However, as the number of organizations to be audited increased due to the large size of the organization, the need for more systematic and efficient improvement of the one-on-one audit method was raised. As a result, various audit systems were introduced for each organization.

However, the conventional audit system is limited to simply providing the information to the auditor after collecting and collecting information on the audited organization. In addition, each auditor performed regular audits at intervals of 2 to 3 years for audited operations that had not been imprisoned for a long time.

However, in the conventional auditing method, since the auditor has to directly review and audit the subject for each case, it is reaching its limits both temporally and geographically.

Moreover, as the organization's business structure is reorganized due to organizational restructuring or increased asset size, a new type of accident occurs and the amount of the accident is increasing, raising the need for a new audit system.

The background art of the present invention is disclosed in Korean Patent Publication No. 10-2006-0086619 (2006.08.01), 'ERP system-based audit information system and audit information operation method using the same, and a recording medium on which the program is recorded' there is.

For corporations, companies, institutions, organizations, etc. (hereinafter collectively referred to as corporations), it is a major task to protect the assets of the organization and to increase work efficiency, to control business risks, and to understand the current status of these business losses. An example is the audit system as a management technique for management and management.

However, since the conventional audit system is mainly performed based on past audit experiences or audit results, problems such as waste of audit resources or unreasonable audits have been present. As a result, there is a disadvantage in that the quality of audit activities deteriorates and repetitive work losses occur, resulting in a vicious cycle in which unnecessary audit resources are wasted again.

Furthermore, in the management of audit activities, audit reports, audit results, and corrective/improvement plans are all managed only as paper documents with a high risk of loss or extinction, making it difficult to obtain accumulated data on previous audit data. there is.

In addition, audit data in paper form is difficult to analyze, search, and process, so it is not properly utilized in audit activities, and work productivity such as delays in work processing and decision-making as the flow of information provision is greatly affected by the speed of circulation of documents It acts as a factor that slows down the improvement.

Therefore, it has been required to develop a new concept audit management system audit management method that can improve work productivity and speed in audit activities and implement functions that can systematically store, search, and statistically process audit-related data.

On the other hand, in the case of household loan fraud for a predetermined target, the fraudulent loan ratio for credit and deposit cases is 0.018%, and data collection and analysis are limited to develop a model that predicts normal and fraudulent loans by analyzing new loan applications there is

In addition, there is a problem in that the discriminative power of bad prediction is low in the case of bad household loans for a predetermined target, and there are many cases in which bad predictions are made when they are not actually bad, or normal predictions are made when bad household loans are bad.

The present invention has been made to solve these problems, and an object of the present invention is to timely detect abnormal symptoms related to fraudulent household loan fraud / insolvency as the scale of loss due to fraudulent household loan fraud / insolvency increases, and focuses on post-detection. It is possible to prevent fraud and fraudulent household loan fraud / insolvency by moving away from human resources, and it is possible to increase audit efficiency and build an integrated inspection management system through the risk assessment and automatic relearning function of loans subject to inspection, and to manage risks through advanced monitoring at all times. It is to provide a regular audit support system for household loan fraud/insolvency using artificial intelligence that can build a system.

In the household loan fraud/insolvency regular audit support system using artificial intelligence according to an embodiment of the present invention, when a user submits a predetermined document to apply for household loan fraud/insolvency, the credit rating and the possible amount of household loan fraud/insolvency In addition to the household loan fraud/insolvency system that conducts fraudulent/insolvent transactions in household loans by investigating fraudulent/insolvent transactions, etc. An audit information system that conducts regular audits in consideration of the risk rate of insolvent transactions and its basis items, and supports efficient business audits for high-risk groups, and the first database of the above household loan fraud / insolvency system through wired and wireless Internet Collect data, perform preprocessing for learning, analyze, learn, and model data to create and store an abnormal transaction prediction model with the help of an audit terminal, and use the abnormal transaction prediction model to perform the household loan fraud/insolvency system Household loan fraud/non-performing transaction data is predicted to be an abnormal transaction and provided to the audit information system, so that the audit terminal detects household loan fraud/insolvency using the fraud/insolvent household loan fraud/insolvency detection monitoring of the audit information system. It includes an artificial intelligence audit analysis system that enables efficient regular auditing of insolvency,

The data includes demographic information, employment and income information, bank transaction information, and fraudulent loan-related attribute information in consideration of the possibility of fraudulent loans, focusing on items indicating signs of fraudulent loans.

The above data includes personal loan application, personal credit information summary details, personal credit review approval performance details, and derived variables for insolvency prediction model analysis.
The audit information system conducts regular audits when the result of the fraud/insolvency prediction model of the artificial intelligence audit analysis system is high for fraudulent/insolvent transactions of household loans through the fraud/insolvency system for household loans, and detects fraud and fraudulent households. It includes a regular audit module to prevent loan fraud/insolvency and a regular sales audit module for organizations to identify and manage business losses due to fraud/insolvency in fraudulent household loans.
The regular audit module includes a fraud probability display unit for displaying a fraud probability for household loan fraud/insolvency transactions through the household loan fraud/insolvency system, and a fraud probability for household loan fraud/insolvency transactions through the household loan fraud/insolvency system. Further comprising: an insolvency probability display unit displaying and a basis presentation unit for presenting a basis for the probability of the household loan fraud/insolvency transaction with respect to the fraud probability or insolvency probability displayed through the fraud probability display unit or the insolvency probability display unit. to be characterized

According to the household loan fraud / insolvency regular audit support system using artificial intelligence according to an embodiment of the present invention, it is possible to prevent losses of financial institutions due to fraud / insolvency handling of false / fraudulent household loans and to prepare for large-scale accidents, internally and externally A new type of fraud (planned household loan fraud/insolvency) in response to environmental changes enables loss prevention and reputation risk management of financial institutions by responding to household loan fraud/insolvency. The scope of inspection can be expanded by increasing the efficiency of

In particular, according to the household loan fraud/insolvency regular audit support system using artificial intelligence according to an embodiment of the present invention, not only when fraud/insolvency data is insufficient or non-uniform, but also in cases where the discriminative power is low, it is classified as a high-risk fraud loan account. It is possible to expand the scope of inspection by increasing the efficiency of inspection work by the auditor through selective and intensive inspection.

1 shows the overall configuration of a regular audit support system using artificial intelligence according to one system of the present invention.
Figure 2 is a block diagram explaining the internal configuration of the audit information system of Figure 1.
Figure 3 is an exemplary screen provided by the audit information system of Figure 1 to the audit terminal.
Figure 4 is a block diagram explaining the internal configuration of the artificial intelligence audit analysis system according to an embodiment of the present invention of Figure 1.
Figure 5 shows the configuration of the interface between the artificial intelligence audit analysis system of Figure 4 and the audit information system of Figure 2.
6 is a flowchart of a regular audit support method for fraudulent/insolvent household loans using artificial intelligence according to an embodiment of the present invention.
7 is a conceptual diagram explaining the interlocking relationship between supervised learning and unsupervised learning for fraud/insolvent household loan fraud/insolvency used in the household loan fraud/insolvency constant audit support method using artificial intelligence according to an embodiment of the present invention. am.
8A and 8B are flowcharts illustrating a method of deriving candidate variables to be used for modeling among the analysis data of steps S30 and S35 of FIG. 6 and a method of processing learning data.
9 is a conceptual diagram illustrating the concept of developing a household loan fraud/insolvency regular audit model and model evaluation.
10A is a diagram showing the performance of a second model and a third model with different ratios of learning data and test data for household fraud prediction performance, and FIG. 10B is a modified example of a screen example provided to an audit terminal in FIG. 3 FIG. 10C is a diagram showing performance indicators in the case of using a DNN model.
11 is a graph showing household insolvency prediction results when using the household loan fraud/insolvency constant audit support method using artificial intelligence according to an embodiment of the present invention.
12a and 12b show the distribution of personal credit ratings by predicted insolvency probability (low insolvency/medium insolvency/high insolvency) as a result of analysis by the DNN model, respectively. This is a table showing the results of performing post-hoc verification on predicted cases.

Hereinafter, a continuous auditing device according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In this process, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or practice of a user or operator. Therefore, definitions of these terms will have to be made based on the content throughout this specification.

Figure 1 shows the overall configuration of a regular audit support system using artificial intelligence according to one system of the present invention, Figure 2 is a block diagram explaining the internal configuration of the audit information system of Figure 1, Figure 3 is a It is an example of a screen provided by an audit information system to an audit terminal, and FIG. 4 is a block diagram illustrating the internal configuration of an artificial intelligence audit analysis system according to an embodiment of the present invention in FIG.

As shown in FIGS. 1 to 4 , the household loan fraud/insolvency regular audit support system 1 using artificial intelligence according to an embodiment of the present invention allows a user to submit predetermined documents to detect fraud/insolvency in household loans. In case of application, the household loan fraud/insolvency system 100 conducts household loan fraud/insolvency transactions by examining the credit rating and possible amount of household loan fraud/insolvency, and the household loan fraud/insolvency system 100 performs In addition to periodic audits for household loan fraud/insolvent transactions, regular audits are conducted in consideration of the risk rate of fraud or insolvency transactions and the basis items through the wired/wireless Internet (200), so that efficient business audits can be performed for high-risk groups. Collects data from the supporting audit information system 300 and the first database 101 of the household loan fraud/insolvency system 100 through the wired and wireless Internet 200, performs pre-processing for learning, and analyzes the data and learning and modeling to generate and store an abnormal transaction prediction model with the help of the audit terminal 10, and use the abnormal transaction prediction model to obtain the household loan fraud/insolvency transaction data of the household loan fraud/insolvency system 100 ( 101), predicts whether or not it is an abnormal transaction and provides it to the audit information system 300 so that the audit terminal 10 uses the fraud/insolvency household loan fraud/insolvency detection monitoring of the audit information system 300 to It includes an artificial intelligence audit analysis system 500 that enables efficient regular auditing of loan fraud/insolvency.

The second database 501 of the artificial intelligence audit analysis system 500 includes original data as a Hadoop file system and data staging from the first database 101 for AI audit analysis work. A learning data storage unit 501a for storing , a predictive model storage unit 501b for machine learning learning using the learning data, and prediction results such as fraud/insolvency of fraudulent household loans using the prediction model are stored A prediction result storage unit 501c may be further included.

The audit information system 300, when the result value of the fraud/insolvency prediction model of the artificial intelligence audit analysis system 500 is high for fraudulent/insolvent transactions in household loans through the fraudulent/insolvent household loan system 100 A regular audit module (310) to perform regular audits to prevent fraud and fraudulent household loan fraud/insolvency, and a regular sales audit module for organizations to identify and manage business losses due to fraudulent household loan fraud/insolvency ( 330).

The regular audit module 310 includes a fraud probability display unit 311 displaying a fraud probability for household loan fraud/insolvency transactions through the household loan fraud/insolvency system 100, and the household loan fraud/insolvency system 100 The insolvency probability display unit 313 for displaying the insolvency probability for fraudulent/insolvent transactions of household loans through the fraud probability display unit 311 or the insolvency probability display unit 313, and the fraud probability or insolvency probability displayed through the above A basis presentation unit 315 for presenting a basis for the probability of household loan fraud/non-performing transaction may be further included.

In general, the evidence presentation unit 315 provides a sophisticated risk level with high precision and continuous risk estimation in the case of a data-based machine learning model, but in order to solve the problem that it is difficult to clearly explain the results, the audit terminal 10 Alternatively, with the support of the artificial intelligence audit analysis system 500, by presenting business rules or models and grounds to be reviewed first, the efficiency of the inspection task of the audit terminal 10 is selectively and intensively inspected for fraudulent high-risk classified loan accounts. can be increased, and the inspection range can be expanded by presenting a new business rule or model other than the conventional business rule or model that the monitoring terminal 10 can consider.

The regular audit module 310 and the sales audit module 330 interlock with each other to conduct statistical analysis and machine learning not only for high-risk groups but also for fraudulent/non-performing transactions in household loans, so as to achieve a risk management system by advancing regular monitoring. it is possible to build

The artificial intelligence audit analysis system 500 includes an abnormal transaction detection server 510, a model/business rule development server 530, an automatic learning server 550, an AI prediction server 560, and an interworking server 570. ).

The abnormal transaction detection server 510 includes an abnormal transaction management unit 511 that defines abnormal transactions for household loan fraud/insolvent transactions and establishes a management system such as tagging, and abnormal transactions defined by the abnormal transaction management unit 511. Since the abnormal transaction data collection unit 513 classifies and collects data, and the abnormal transaction data is very small, less than 1% of the total household loan fraud / non-performing transaction data, there is a problem of low accuracy when used as a machine learning model. Data pre-processing unit 515 that removes noise from abnormal transaction data and processes it before processing it into learning data, and learning that can be used in the model/business rule development server 530 or automatic learning server 550, which will be described in detail later. It includes a learning data processing unit 517 for processing into data.

The learning data processing unit 517 identifies abnormal transaction data and defines it as target data for a machine learning solution.

The model/business rule development server 530 develops a machine learning prediction model for elaborating abnormal transaction predictions, optimizes business rules that can prevent fraudulent household loan fraud/insolvency transactions or fraudulent transaction household loan fraud/insolvency, or It is characterized by new development.

The model/business rule development server 530 analyzes the business requirements provided by the abnormal transaction detection model development unit 531 that develops a machine learning-based abnormal transaction detection model, and detects fraudulent household loan fraud/insolvent transactions or fraudulent transaction households. A business rule development unit 533 that optimizes or newly develops business rules that can prevent loan fraud/insolvency and develops regular audit check items to be checked by the regular audit module 310 of the audit information system 300 The audit check item development unit 535 detects a new type of fraud pattern not found in the existing rules, continuously self-develops a new type of abnormal transaction, and generates a predictive model through the automatic learning server 550, fraud or insolvency A new fraud pattern detection unit 537 is further included to provide prediction information on household loan fraud/insolvency.

The regular audit check item development unit 535 applies rules and machine learning learning (supervised learning) to detect applications with a high probability of false / fraud in case of fraud / insolvency of household loans, and sends the result value to the audit terminal 10 It can be provided for regular auditing.

In the case of corporate loans, the regular audit check items development unit 535 applies rules through machine learning learning (unsupervised learning) to financial/non-financial information of companies subject to new credit evaluation, and predicts fraud/high-risk group information through the audit It can be provided to the terminal 10 so that it can be audited at all times.

In this way, the work load of the audit terminal 10 can be reduced by differentiating machine learning learning for household loan fraud/insolvency and corporate loan.

The automatic learning server 550 performs supervised learning using the abnormal transaction model developed in the abnormal transaction detection model development unit 531 using the learning data processed in the learning data processing unit 517 using a machine learning algorithm. and the model/business rule development server ( An unsupervised learning unit 553 that automatically performs unsupervised learning by applying the abnormal transaction detection model or business rule developed in step 530, or a regular check item or a new fraud pattern, and the fraud/insolvent household loan fraud/insolvency model A model comparison and analysis unit 555 that compares and analyzes and applies a model with high accuracy, and a prediction model to be applied to the household loan fraud / insolvent transaction is finalized using the analysis result of the model comparison and analysis unit 555 It may include a predictive model generator 557 to select.

The AI prediction server 560 uses the supervised learning unit 551 and the unsupervised learning unit 553 using the prediction model generated by the prediction model generation unit 557 to perform fraud or fraud by machine learning learning. An AI prediction unit 561 for predicting the probability of fraudulent/non-performing transactions in bad household loans, and a prediction information storage unit 563 for storing information predicted by the AI prediction unit 561 in a prediction information DB 567; An interface unit 565 for connecting the AI prediction server 560 and the audit information system 300 may be included.

The interlocking server 570 is a transmission/reception interface 571 for connecting to the household loan fraud/insolvency system 100, audit terminal 10, and audit information system 300 through the Internet network 200 so as to transmit and receive ), and information to be provided according to the needs of the audit terminal 10 in the regular audit module 310 and the sales audit module 330 of the audit information system 300, for example, the percentage or bar of fraud or insolvency probability A customizing unit 573 capable of customizing audit information, such as displaying a graph, providing evidence items in rank, or highlighting, may be included.

Now, referring to FIG. 5, the interface between the artificial intelligence audit analysis system of FIG. 4 and the audit information system of FIG. 2 will be described in detail.

Figure 5 shows the configuration of the interface between the artificial intelligence audit analysis system of Figure 4 and the audit information system of Figure 2.

As shown in FIG. 5, the automatic learning server 550 extracts, converts, and loads past household loan fraud/insolvency transaction data from the first database 101 of the household loan fraud/insolvency system 100 at regular intervals. It can be stored in the second database 501, which is the Hadoop system, and the learning data processed by the learning data processing unit 517 is learned in the supervised learning unit 551 or the unsupervised learning unit 553, The models calculated in the model comparison and analysis unit 555 are compared and analyzed, stored in the prediction model storage area 501b, and continuously re-learned in the supervised learning unit 551 or the unsupervised learning unit 553, and the model If the comparison and analysis unit 555 does not satisfy the distribution criteria such as accuracy, data or parameters are changed and re-learned, and in the predictive model generator 557, a model that satisfies the model performance criteria such as accuracy is finally converted into a predictive model. If selected, the AI prediction server 560 extracts, converts, and loads the household loan fraud/insolvency transaction data loaded in a single batch from the first database 101 of the household loan fraud/insolvency system 100 and the AI prediction unit 561 predicts a fraud probability or a non-performing probability value for the household loan fraud/non-performing transaction data loaded in a batch to be predicted using the prediction model, and the prediction result is stored in the prediction result storage area 501c ), and provided through the regular audit module 310 of the audit information system 300 through the interface unit 565, and constantly audited and monitored to prevent bank losses and accidents due to fraudulent / insolvent handling of false / fraudulent household loans You can prepare for an increase in the amount.

Therefore, it is possible to prevent losses and manage reputational risks by responding to new types of fraud (planned household loan fraud/insolvency) and household loan fraud/insolvency according to changes in the internal and external environment. The inspection range can be expanded by increasing the efficiency of the inspection task of the terminal 10 .

Now, with reference to FIGS. 6 and 7 , a method for supporting fraud/insolvency auditing of household loans using artificial intelligence according to an embodiment of the present invention will be described in detail.

6 is a flowchart of a household loan fraud/insolvency constant audit support method using artificial intelligence according to an embodiment of the present invention, and FIG. 7 is a flowchart of a household loan fraud/insolvency constant using artificial intelligence according to an embodiment of the present invention. It is a conceptual diagram explaining the interlocking relationship between supervised learning and unsupervised learning for fraud/insolvent household loan fraud/insolvency used in the audit support method.

First, as shown in FIGS. 6 and 7, in the household loan fraud/insolvency regular audit support method using artificial intelligence according to an embodiment of the present invention, the artificial intelligence audit analysis system 500 is an abnormal transaction detection server In the abnormal transaction management unit (511) of (510), the business requirements for fraudulent/non-performing transactions in household loans are organized, and the abnormal transactions to be audited and analyzed by artificial intelligence are selected and the requirements are defined (S10).

The abnormal transaction management unit 511 may define an abnormal transaction prediction target, select an analysis target, an observation period, model segment definition, and an analysis method.

When the abnormal transaction data defined by the abnormal transaction management unit 511 is classified and collected by the abnormal transaction data collection unit 513, the data pre-processing unit 515 removes noise from the abnormal transaction data and increases speed or accuracy. The data pre-processing is performed, and the learning data processing unit 517 processes and prepares learning data that can be used in the model/business rule development server 530 or the automatic learning server 550 (S20).

The learning data processing unit 517 explores and analyzes data (S30) using short-term bankruptcy data, past fraud (accident) related credit data, false household loan fraud / insolvency suspected registered borrower data, etc. as the range of analysis data. can be processed with

In particular, in relation to household loan fraud/insolvency, the fraud prediction model analysis data defined candidate variables under four categories by considering the possibility of fraudulent loans, focusing on items that indicate signs of fraudulent loans.

Category Meaning main candidate variable demographic information Basic demographic information of customers such as age and gender Number of ages, gender, personal credit rating Employment and Income Information Information about your company and income Current workplace classification, current workplace service month, annual income amount, Nice estimated income amount banking information Relational information that can determine the degree of customer use of the bank
No complaints within 6 months of eating
Average wage transfer amount for the latest 3 months Number of months of the current transaction period Security Fraudulent Loan Property Information Attribute information commonly found in fraudulent loans (information identified through past credit audit experiences of the Sales Audit Department) Employment Classification_High-Risk Group
Suspicious job title similarity

Also, for the insolvency prediction model analysis data, the analysis data can be configured by considering personal loan application, personal credit information summary details, personal credit review approval record, and derived variables.

The learning data processing unit 517 is a machine learning prediction model developed or optimized by the model/business rule development server 530, which can prevent fraudulent household loan fraud/non-performing transactions or fraudulent transaction household loan fraud/insolvency. It is possible to search and analyze learning data using business rules, detect new types of fraud patterns not found in existing rules, and predict new types of abnormal transactions through the automatic learning server 550, and new fraud patterns. A new type of fraud pattern provided by the detection unit 537 may be used, and the learning data may be processed by applying a highly accurate model after comparative analysis by the model comparison and analysis unit 555 (S35).

The automatic learning server 550 uses the machine learning algorithm through the supervised learning unit 551, and the learning data processed by the learning data processing unit 517 and the abnormal transaction detection model developed by the development unit 531. It is possible to probabilistically determine the possibility of fraud by performing supervised learning using the abnormal transaction model, and the abnormal transaction detection model developed in the model/business rule development server 530 using the unsupervised learning unit 553. Alternatively, unsupervised learning may be performed automatically by applying business rules, regular check items, or new fraud patterns (S50).

In the data learning/modeling step (S40), a prediction model to be applied to the household loan fraud/insolvent transaction using the predictive model generator 557 is finalized using the analysis result of the model comparison and analysis unit 555. The selection model/business rule development/comparative analysis step (S35) may be repeated.

The AI prediction server 560 uses the supervised learning unit 551 and the unsupervised learning unit 553 using the prediction model generated by the prediction model generation unit 557 to perform fraud or fraud by machine learning learning. Probability of insolvent household loan fraud/insolvent transaction can be predicted (S50)

The AI prediction server 560 and the audit information system 300 are connected by the interface unit 565 and the regular audit module 310 of the audit information system 300 displays the probability of fraud or insolvency in % or a bar graph In addition, it is possible to prevent bank losses due to fraudulent/insolvent handling of false/fraudulent household loans and to prepare for an increase in accident amount by providing evidence items in order and auditing and monitoring by the auditing terminal 10 interlocking with them.

Now, with reference to FIGS. 8A and 9 , a method of deriving candidate variables to be used for modeling from analysis data and a method of processing learning data will be described.

8A and 8B are flowcharts illustrating a method of deriving candidate variables to be used for modeling among the analysis data of steps S30 and S35 of FIG. 6 and a method of processing learning data, and FIG. 9 is a regular audit of household loan fraud/insolvency It is a conceptual diagram showing the concept of model development and model evaluation.

As shown in FIG. 8A, in the optimal model search and analysis step (S30), basic analysis is performed to derive candidate variables to be used for modeling among the analyzed data, and through this, errors or outliers are found and data processing is performed to improve model performance. Do it.

To this end, in the optimal model search and analysis step (S30), percentage analysis is performed to perform a distribution analysis step (S301) in which the candidate variable to be used for modeling among the analyzed data is excluded when the ratio of the same value is 90% or more, and the candidate variable is excluded. Continuous variable significance test, which performs statistical tests to see if the average difference between a variable and target is significant, and univariate significance test, which performs categorical variable significance test, which tests the statistical difference between observed values and expected values by category Analysis step (S302) is performed, and check item variables to be checked for fraud / insolvency of household loans, for example, whether it is a mandatory check subject, number of past mortgage loans, number of small credit loans to other banks in the last 3 months, past loans to other banks The step of verifying the model (S303) is performed while adding household check item variables such as the number of account holdings, the number of past loan account holdings, the number of non-requested non-reporting records within 2 years, whether or not the retirement allowance is held, and the remaining variables after significance verification , but if necessary, change the variable type to generate significant variables to derive candidate variables, and if there are many candidate variables, select multivariate variables as features before modeling to derive final candidate variables to determine the final candidate variables (candidate variables). Derivation, feature selection) step (S304) is performed, and a modeling variable setting step (S305) of generating modeling applied variables by selecting a modeling variable set as a business model and a rule may be included.

As shown in [Table 2], the final 61 variables can be set through data exploration and analysis of demographic information, employment and income information, bank transaction information, and attribute information related to fraudulent loans.

[Table 2]

As shown in FIGS. 8B and 9, in the regular audit model development/comparison analysis step (S35) for household loan fraud/insolvency, a model is developed using learning data consisting of candidate variables and targets, and loans are obtained through this model. In order to select the optimal model by evaluating the performance of the model after calculating the fraud probability for each application, developing a selection model (S351) and using the selection model to calculate the fraud probability for each loan application The step of deriving the probability of fraudulent loans (S352), which calculates the probability of fraud for each case of fraudulent/insolvent application for household loans, using learning data such as candidate variables and targets such as bank transactions; It may include determining whether the application is normal or fraudulent, evaluating the model (S353), and calculating and comparing performance indicators for the model (S354).

In the performance index calculation and comparison step (S354) for the model, a curve may be generated by dividing upper and lower cases into normal and fraudulent cases for each application case and obtaining precision and recall rate for each case.

Precision refers to the ratio of the actual number of fraud cases to the number of model fraud predictions, and recall refers to the ratio of the number of model fraud predictions to the actual number of fraud cases. After analyzing the area, the closer to 1 is, the better the model is.

On the other hand, in order to develop and evaluate a fraud prediction model, the second and third verifications can be performed by changing the data such as the date and time of consultation and the date and time of sending loan documents for the number of fraud cases compared to the total number of data. It can be performed while varying the ratio of training data and test data.

10A is an example screen showing the performance of a second model and a third model in which the ratio of learning data and test data is different for household fraud prediction performance.

As shown in FIG. 10A, the performance of the 2nd model and the 3rd model with different ratios of training data and test data are evaluated differently, and the performance of the 3rd model shows a PRAUC value close to 1 compared to the 2nd model. It has a high F1-Score, so it can be seen that it is an excellent model.

FIG. 10B is a diagram showing a modified example of the exemplary screen provided to the auditing terminal of FIG. 3 .

As shown in FIG. 10B, in order to increase the explanatory power of the fraud prediction result in the audit terminal 10, the machine learning model interpretation library SHAP (SHapely additive exPlanation) is used to calculate and display the basis items to direct the influence of each prediction case variable. It can be displayed in size and size, and the rank, content and value of important fraudulent items can be displayed as evidence items, and the rank, content and value of normal important items can be displayed.

To this end, it is possible to derive important variables fitted with the model determined to be the optimal model.

On the other hand, it is possible to develop and evaluate not only the household fraudulent loan model but also the insolvency prediction model. In the case of the insolvency model, the insolvency rate is 0.73%, which accounts for a large portion compared to the fraud rate. In addition to algorithms, deep neural network models can be used.

Deep Neural Network (DNN) is a learning method that mimics the human nervous system and can be analyzed regardless of continuous or categorical variables. Its predictive power is significantly superior to other machine learning techniques, and feature extraction is also It is performed automatically and can have the feature that performance increases as the data size increases.

Therefore, it is used to improve accuracy and reduce overfitting, and FIG. 10c is a diagram comparing performance indicators in the case of using the learning data of FIG. 10a and in the case of using the DNN model.

As shown in FIG. 10c, when the user developed a poor prediction model using a DNN model, compared to when the user processed and used the training data, the case of mispredicting the actual normal as poor was significantly reduced, and the PRAUC value in the DNN model It was closer to 1 and the F1-Score was larger.

11 is a graph showing household insolvency prediction results when using the household loan fraud/insolvency constant audit support method using artificial intelligence according to an embodiment of the present invention.

As shown in FIG. 11, when the household loan fraud/insolvency constant audit support method using artificial intelligence according to an embodiment of the present invention is used, for example, based on the prediction probability of 0.4, for the audit terminal for household insolvency prediction When displaying the result of classification as normal/poor, prediction probabilities of 0.2 or less can be classified as low, 0.2 to less than 0.8 as medium, and 0.8 or more as high.

12a and 12b show the distribution of individual credit ratings by predicted insolvency probability (low/medium/high insolvency) as a result of analysis by the DNN model, respectively. This is a table showing the results of performing post-hoc verification on predicted cases.

As shown in FIG. 12a, as a result of analysis with the DNN model, discrimination power could be provided through the distribution of individual credit ratings for each predicted insolvency probability (low insolvency/medium insolvency/high insolvency).

In addition, as shown in FIG. 12B, the validity of the predicted value of the model was confirmed as a result of conducting a post-hoc verification on the case where the developed model predicted the normal/fraud classification in the opposite way to the actual one.

Claims (8)

In addition to regular audits of the loan system and household loan fraud/insolvent transactions performed by the household loan fraud/insolvency system, regular audits are conducted in consideration of the risk rate of fraud or insolvency transactions through the wired/wireless Internet and the basis items. Collects data from the audit information system and the first database of the household loan fraud/insolvency system through the wired and wireless Internet, performs pre-processing for learning, analyzes and learns data, Modeling, creating and storing an abnormal transaction prediction model with the help of the audit terminal, and predicting whether or not it is an abnormal transaction for the household loan fraud / insolvency transaction data of the household loan fraud / insolvency system using the abnormal transaction prediction model Artificial intelligence audit analysis that is provided to the audit information system so that the audit terminal can efficiently perform regular audits on household loan fraud / insolvency by using the fraud / insolvency household loan fraud / insolvency detection monitoring of the audit information system including the system,
The data includes demographic information, employment and income information, bank transaction information, and fraudulent loan-related attribute information, taking into account the possibility of fraudulent loans, focusing on items that indicate signs of fraudulent loans.
The above data includes personal loan application, personal credit information summary details, personal credit review approval record details, and derived variables for insolvency prediction model analysis.
The audit information system performs a regular audit when the fraud/insolvency prediction model result value of the artificial intelligence audit analysis system is high for fraudulent/insolvent transactions of household loans through the fraud/insolvency system for household loans, and detects fraud and fraudulent households. It includes a regular audit module to prevent loan fraud/insolvency and a regular sales audit module for organizations to identify and manage business losses due to fraud/insolvency in fraudulent household loans.
The regular audit module includes a fraud probability indicator displaying a fraud probability for household loan fraud/insolvency transactions through the household loan fraud/insolvency system, and an insolvency probability for household loan fraud/insolvency transactions through the household loan fraud/insolvency system. Artificial intelligence further comprising a non-performing probability display unit for displaying and a grounds presentation unit for presenting a basis for the probability of the household loan fraud/non-performing transaction with respect to the fraud probability or non-performing probability displayed through the fraud probability display unit or the insolvency probability display unit. Household loan fraud/insolvency regular audit support system using intelligence. According to claim 1,
The artificial intelligence audit analysis system is a Hadoop file system, and a learning data storage unit for storing changed data for artificial intelligence audit analysis work from the first database, and machine learning learning using the learning data Household loan fraud/insolvency regular audit support using artificial intelligence, including a second database further including a prediction model storage unit for storing household loans and a prediction result storage unit for storing prediction results such as fraudulent household loan fraud/insolvency using the prediction model system. delete delete According to claim 1
The artificial intelligence audit analysis system defines abnormal transaction data, collects and preprocesses abnormal transaction data, identifies accident case data and defines target data for machine learning solutions, and detects and analyzes abnormal transaction data. An abnormal transaction detection server and machine learning A model/business rule development server that develops business rules by developing an abnormal transaction detection model based on household loan fraud/non-performing transactions and analyzing requirements for fraudulent/non-performing transactions in household loans, and learning to define, detect, analyze, and develop new rule models through machine learning learning An automatic learning server that automates, an AI prediction server that predicts abnormal transactions with artificial intelligence using the machine learning model developed in the automatic learning server and learning data, and an AI prediction server that links the audit information system. Including an interlocking server,
The model/business rule development server performs the steps of verifying the model while adding checklist variables to be checked for fraud/insolvency of household loans, generating meaningful variables by changing variable types, deriving candidate variables, and deriving candidate variables. In many cases, a step of determining a final candidate variable by selecting a multivariate variable as a feature before modeling, and a modeling variable setting step of selecting a modeling variable set as a business model and a rule to create a modeling application variable,
After developing a model using the learning data and calculating the probability of fraud for each loan application through the developed model, evaluating the performance of the model and selecting a model; The step of deriving the probability of fraud by calculating the probability of fraud by household loan fraud / insolvent application by using learning data such as employment / income, bank transaction candidate variables and targets, and sorting in descending order by model prediction score to match the predicted probability and actual Household loan fraud/insolvency regular audit support system using artificial intelligence that performs the model evaluation step by determining whether the application is normal or fraudulent, and the performance index calculation and comparison step for the model. According to claim 5
The model/business rule development server prevents fraudulent household loan fraud/insolvent transactions or fraudulent transaction household loan fraud/insolvency by analyzing the business requirements provided by the abnormal transaction detection model development department that develops a machine learning-based abnormal transaction detection model. A business rule development department that newly develops possible business rules, a regular audit check item development unit that develops regular audit check items to be inspected in the regular audit module of the audit information system, and a new type of fraud pattern not found in the existing rules. New fraud pattern detection unit for detecting and continuously self-developing new abnormal transaction types to generate a predictive model through the automatic learning server and to provide prediction information on fraud or insolvent household loan fraud / insolvency Further comprising Household loan fraud/insolvency regular audit support system using artificial intelligence. According to claim 6
The automatic learning server proceeds with supervised learning using the learning data processed in the learning data processing unit using the machine learning algorithm and the abnormal transaction model developed in the abnormal transaction detection model development unit to probabilistically determine the possibility of fraud an unsupervised learning unit that automatically performs unsupervised learning by applying an abnormal transaction detection model or business rule developed in the model/business rule development server, a regular inspection item, or a new fraud pattern; / A model comparison analysis unit that compares and analyzes fraudulent household loan fraud / insolvency models, and a prediction model generation unit that finally selects a prediction model to be applied to the household loan fraud / insolvency transaction using the analysis results of the model comparison analysis unit Household loan fraud/insolvency regular audit support system using artificial intelligence including According to claim 7
The AI prediction server predicts the probability of fraudulent or insolvent household loan fraud/insolvent transaction by machine learning learning using the supervised learning unit or the unsupervised learning unit using the prediction model generated by the prediction model generator AI. Including a prediction unit,
The AI prediction server uses a deep neural network model, and in order to increase the explanatory power of fraud prediction results, household loan fraud / insolvency using artificial intelligence that displays the degree of influence of each prediction case in direction and size in order to calculate and display evidence items audit support system.

Images