WO2023163405A1

WO2023163405A1 - Method and apparatus for updating or replacing credit evaluation model

Info

Publication number: WO2023163405A1
Application number: PCT/KR2023/001448
Authority: WO
Inventors: 강정석
Original assignee: 주식회사 에이젠글로벌
Priority date: 2022-02-25
Filing date: 2023-02-01
Publication date: 2023-08-31
Also published as: KR20230127861A; KR102480518B1

Abstract

The present invention relates to a method and apparatus for updating or replacing a credit evaluation model. The method for updating or replacing a credit evaluation comprises the steps of: generating, by an artificial intelligence engine generation unit, a plurality of artificial intelligence engines for credit evaluation; generating, by a credit evaluation model generation unit, a credit evaluation model on the basis of the plurality of artificial intelligence engines; and generating, by a credit evaluation unit, credit evaluation data for a seller on the basis of the credit evaluation model, wherein the credit evaluation model is generated on the basis of scaling and/or layering of the plurality of artificial intelligence engines, and evaluation of the credit evaluation model may be performed on the basis of a core module, a related module, and a non-core module constituting the credit evaluation model.

Description

Method and apparatus for updating or replacing credit evaluation model

The present invention relates to a method and apparatus for updating or replacing a credit rating model. More specifically, it relates to a method and apparatus for updating or replacing a credit evaluation model used for credit evaluation of a seller to provide financial services to the seller.

As we enter the intelligent information society triggered by the 4th industrial revolution, the possibility of unlimited utilization of data is bringing about changes in the data industry. With the arrival of the data era, the level of the future data industry will determine the difference in competitiveness between countries.

In particular, the establishment of big data infrastructure in the financial market is not only very urgent, but it has soon become an important asset enough to determine the direction of the country's data industry. The financial big data infrastructure consists of big data open systems, data exchanges, and data specialized institutions.

Research on new financial products based on these big data-based user financial data is needed. It is possible to analyze various risks through artificial intelligence-based learning of user financial products, and based on risk analysis, new financial services that have not been available before can be provided to users.

Therefore, it is necessary to study specific methods for utilizing user's financial data and providing various financial services based on the user's financial data.

The object of the present invention is to solve all of the above problems.

In addition, an object of the present invention is to perform replacement or shifting of an artificial intelligence engine by searching for judgment errors for each module constituting a credit evaluation model in consideration of a feedback result for the credit evaluation model.

Another object of the present invention is to enable correction of a credit evaluation model through replacement of an artificial intelligence engine starting with an error in judgment and shifting to an artificial intelligence engine that affects an artificial intelligence engine with an error in judgment.

Representative configurations of the present invention for achieving the above object are as follows.

According to an embodiment of the present invention, a method for updating or replacing a credit evaluation model includes generating a plurality of artificial intelligence engines for credit evaluation by an artificial intelligence engine generator, and generating a credit evaluation model based on the plurality of artificial intelligence engines. Generating a credit rating model and generating credit rating data for a seller based on the credit rating unit by a credit rating unit, wherein the credit rating model is based on extension and/or layering of a plurality of artificial intelligence engines. , and evaluation of the credit evaluation model may be performed based on core modules, related modules, and non-core modules constituting the credit evaluation model.

On the other hand, the core module is a reference artificial intelligence engine, the related module is an artificial intelligence engine directly connected to the reference artificial intelligence engine, and the non-core module is the credit evaluation model in addition to the core module and the related module. It may be the rest of the artificial intelligence engine that constitutes it.

In addition, the evaluation of the credit rating model is performed through a search for a module that generated a judgment error when the judgment result of the financial service based on the credit rating data is incorrect, and the search generates the judgment error. Dividing a plurality of lower-level credit evaluation data constituting the credit evaluation data, determining target lower-level credit evaluation data that caused a judgment error among the divided plurality of lower-level credit evaluation data, and determining the target lower-level credit evaluation data Thereafter, the core module, the relevant module or the non-core module related to the target sub-credit evaluation data is determined, and a judgment error starting artificial intelligence engine from which the propagation of the judgment error starts and a judgment error effect affected by the judgment error You can decide the artificial intelligence engine.

According to another embodiment of the present invention, a credit evaluation apparatus for generating an artificial intelligence-based credit evaluation model includes an artificial intelligence engine generating unit implemented to generate a plurality of artificial intelligence engines for credit evaluation, based on the plurality of artificial intelligence engines A credit evaluation model generating unit implemented to generate a credit evaluation model and a credit evaluation unit implemented to generate credit evaluation data for a seller based on the credit evaluation model, wherein the credit evaluation model includes a plurality of artificial intelligence engines is generated based on the extension and/or stratification of, and evaluation of the credit evaluation model may be performed based on core modules, related modules, and non-core modules constituting the credit evaluation model.

According to the present invention, replacement or shifting of the artificial intelligence engine may be performed by searching for judgment errors for each module constituting the credit evaluation model in consideration of the feedback result for the credit evaluation model.

In addition, according to the present invention, it is possible to correct the credit evaluation model through replacement of the judgment error start artificial intelligence engine and shifting of the judgment error impact artificial intelligence engine.

1 is a conceptual diagram illustrating a credit evaluation apparatus of a credit bureau (CB) company according to an embodiment of the present invention.

2 is a conceptual diagram illustrating the operation of an artificial intelligence engine generation unit according to an embodiment of the present invention.

3 is a conceptual diagram illustrating the operation of a candidate credit evaluation model generation unit according to an embodiment of the present invention.

4 is a conceptual diagram illustrating the operation of a credit evaluation model determination unit according to an embodiment of the present invention.

5 is a conceptual diagram illustrating the operation of a credit evaluation unit according to an embodiment of the present invention.

6 is a conceptual diagram illustrating the operation of a credit evaluation model management unit according to an embodiment of the present invention.

7 is a conceptual diagram illustrating the operation of a credit evaluation model management unit according to an embodiment of the present invention.

8 is a conceptual diagram illustrating a credit evaluation model monitoring method according to an embodiment of the present invention.

9 is a conceptual diagram illustrating a credit evaluation apparatus according to an embodiment of the present invention.

10 is a conceptual diagram illustrating a method for generating an artificial intelligence-based credit evaluation model according to an embodiment of the present invention.

11 is a conceptual diagram illustrating extension of an artificial intelligence engine for implementing a credit evaluation model according to an embodiment of the present invention.

12 is a conceptual diagram illustrating layering of an artificial intelligence engine for implementing a credit evaluation model according to an embodiment of the present invention.

13 is a conceptual diagram illustrating a credit evaluation model evaluation method according to an embodiment of the present invention.

14 is a conceptual diagram illustrating a credit evaluation model replacement method according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The detailed description of the present invention which follows refers to the accompanying drawings which illustrate, by way of illustration, specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable any person skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different from each other but are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented from one embodiment to another without departing from the spirit and scope of the present invention. It should also be understood that the location or arrangement of individual components within each embodiment may be changed without departing from the spirit and scope of the present invention. Therefore, the detailed description to be described later is not performed in a limiting sense, and the scope of the present invention should be taken as encompassing the scope claimed by the claims and all scopes equivalent thereto. Like reference numbers in the drawings indicate the same or similar elements throughout the various aspects.

Hereinafter, various preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to enable those skilled in the art to easily practice the present invention.

In FIG. 1 , a credit evaluation device for generating a credit evaluation model for performing credit evaluation of a seller by CB, a credit evaluation agency, and automating management of the credit evaluation model is disclosed.

Referring to FIG. 1, the CB company credit evaluation device includes an artificial intelligence engine generation unit 100, a credit evaluation model generation unit 110, a credit evaluation unit 170, a credit evaluation model management unit 180, and a processor 190. can include

The artificial intelligence engine generation unit 100 may be implemented to generate various artificial intelligence engines for credit evaluation. In order to determine the credit evaluation data, individual artificial intelligence engines generating various sub-credit evaluation data may be utilized. The artificial intelligence engine generation unit 100 may generate an artificial intelligence engine capable of generating various sub-level credit evaluation data as a basis for determining credit evaluation data.

The artificial intelligence engine generated by the artificial intelligence engine generator 100 may be transferred to the credit evaluation model generator 110 and used to generate a credit evaluation model.

The credit evaluation model generation unit 110 may be implemented to generate a credit evaluation model. The credit evaluation model may be a model for generating seller's credit evaluation data.

The credit evaluation model may be implemented based on a plurality of artificial intelligence engines and a plurality of algorithms for credit evaluation. The credit evaluation model may generate credit evaluation data for the seller by receiving preprocessed credit evaluation basic data for credit evaluation.

The credit evaluation model generator 110 may include a candidate credit evaluation model generator 120 and a credit evaluation model determiner 160 .

The candidate credit evaluation model generating unit 120 may include an artificial intelligence engine determining unit 130 , an artificial intelligence engine deploying unit 140 , and a credit evaluation data generating unit 150 .

The artificial intelligence engine determination unit 130 may be implemented to determine at least one artificial intelligence engine to be included in the candidate credit evaluation model. The candidate credit evaluation model may generate credit evaluation data based on a plurality of artificial intelligence engines, and the artificial intelligence engine determination unit 130 may determine at least one artificial intelligence engine for generating the candidate credit evaluation model.

The artificial intelligence engine deployer 140 may be implemented to generate a candidate credit evaluation model by arranging the artificial intelligence engine determined by the artificial intelligence model determiner 130 . Each of the plurality of artificial intelligence engines may be arranged sequentially or in parallel to generate a plurality of lower level credit evaluation data, and credit evaluation data may be determined by synthesizing the plurality of lower level credit evaluation data.

For example, each of the plurality of artificial intelligence engines may be arranged in parallel to generate sub-credit evaluation data, and credit evaluation data of the seller may be determined based on the sub-credit evaluation data. Alternatively, a plurality of artificial intelligence engines may be arranged hierarchically (or sequentially) to generate lower credit evaluation data, and credit evaluation data of the seller may be determined based on the hierarchically generated n-th lower credit evaluation data. Alternatively, a candidate credit evaluation model for generating credit evaluation data having a structure in which a result generated by a specific artificial intelligence engine is input to another artificial intelligence engine may be determined. Accordingly, the artificial intelligence engine deployment unit 140 may determine a candidate credit evaluation model based on the deployment of an appropriate artificial intelligence engine to generate credit evaluation data.

The credit evaluation data generation unit 150 may be implemented to generate credit evaluation data based on the candidate credit evaluation model determined by the artificial intelligence engine deployment unit 140 .

The credit evaluation model determination unit 160 may be implemented to determine, as a credit evaluation model, a model having a critical level of reliability in credit evaluation among a plurality of candidate credit evaluation models generated by the candidate credit evaluation model generation unit 120 . Among the candidate credit evaluation models generated by the credit evaluation model generation unit 110 , a credit evaluation model to be used for credit evaluation may be transmitted to the credit evaluation unit 170 . Reliability may be determined by comparing a credit evaluation result of a candidate credit evaluation model based on existing credit evaluation basic data and an actual financial service result. As the credit evaluation result of the candidate credit evaluation model and the actual financial service result are relatively similar, the candidate credit evaluation model may be determined to have relatively high reliability.

The credit evaluation unit 170 may generate credit evaluation data for the seller based on the credit evaluation model generated by the credit evaluation model generating unit 180 .

The credit evaluation unit 170 may generate credit evaluation data through credit evaluation of the seller using at least one credit evaluation model. At least one credit evaluation model located in the credit evaluation unit 170 may be used to determine seller's credit evaluation data by being replaced or combined by evaluation.

In addition, the credit evaluation unit 170 may include various credit evaluation models that are adaptively applicable according to credit evaluation subjects, and credit evaluation data considering seller characteristics may be generated based on these various credit evaluation models.

The credit evaluation model management unit 180 may be implemented for management of credit evaluation models. The credit evaluation model of the credit evaluation unit 170 may be replaced by periodic/non-periodic evaluation. The credit evaluation model management unit 180 may determine a credit evaluation model currently used by the credit evaluation unit 170 and determine whether to replace the credit evaluation model.

Alternatively, the credit evaluation model management unit 180 determines whether to perform partial correction (or shifting) of the resulting value rather than replacing the credit evaluation model of the credit evaluation unit 170 and converts the correction value (or shifting value) to the credit evaluation. It can also be sent to unit 170. For example, when the error between the prediction result of the credit evaluation model of the credit evaluation unit 170 and the actual financial service result can be narrowed based on the correction value, the credit evaluation model management unit 180 calculates the prediction result of the credit evaluation model. A correction value to be corrected may be determined and transmitted to the credit evaluation unit 170 .

The processor 190 may be implemented to control operations of the artificial intelligence engine generator 100 , the credit evaluation model generator 110 , the credit evaluation unit 170 , and the credit evaluation model manager 180 .

2 discloses a method of generating an artificial intelligence engine for credit evaluation in an artificial intelligence engine generation unit.

Referring to FIG. 2 , an artificial intelligence engine generation unit may be implemented to generate an artificial intelligence engine included in a credit evaluation model.

The artificial intelligence engine may be implemented to receive the pre-processed credit evaluation base data as an input value and generate lower-level credit evaluation data for determining the seller's credit evaluation data.

The artificial intelligence engine generation unit may include a candidate artificial intelligence engine generation unit 210 , a reliability determination unit 260 , and an artificial intelligence engine determination unit 270 .

The candidate artificial intelligence engine generation unit 210 may be implemented to generate candidate artificial intelligence engines to be included in the candidate credit evaluation model. The candidate artificial intelligence engine generating unit 210 may include an updated artificial intelligence engine generating unit 220 and a new artificial intelligence engine generating unit 230 .

The updated artificial intelligence engine generation unit 220 may be implemented to generate an updated artificial intelligence engine as a candidate artificial intelligence engine by re-performing artificial intelligence learning based on new training data or new feedback data on an existing artificial intelligence engine. can

The update artificial intelligence engine generating unit 220 may include a new data database 222 , a feedback data database 224 , and an update cycle determining unit 226 .

The new data database 222 may be a database that stores newly input credit evaluation basic data.

The feedback data database 224 may be a database that stores feedback data related to the previously created artificial intelligence engine.

The update cycle determining unit 226 may be implemented to determine an update cycle for the artificial intelligence engine. The update cycle determining unit 226 may determine an update through additional learning data based on a preset cycle. Also, the update cycle determining unit 226 may determine an update cycle in consideration of characteristics of newly input credit evaluation basic data and feedback data. For example, when the credit evaluation basic data and feedback data stored in the database have a relatively high similarity to data used for existing learning, the update cycle may be set to be relatively long. Conversely, when the credit evaluation basic data and feedback data stored in the database have a relatively low similarity to the data used in the existing learning, the update cycle may be set to be relatively short.

The new artificial intelligence engine generation unit 230 may be implemented to generate a candidate artificial intelligence engine through artificial intelligence learning as a new artificial intelligence engine. The new artificial intelligence engine can be a learning model that performs different learning layers, artificial intelligence learning algorithms, learning preprocessing, etc. than existing artificial intelligence engines.

The new artificial intelligence engine generation unit 230 may include a target data determination unit 233 and a learning data determination unit 236 .

The target data determination unit 233 may be implemented to determine lower-level credit evaluation data output by a newly generated artificial intelligence engine.

The learning data determination unit 236 may be implemented to determine learning data for learning of the artificial intelligence engine according to lower credit evaluation data.

In the present invention, the candidate artificial intelligence engine generated by the update artificial intelligence engine generation unit 220 is the candidate artificial intelligence engine (update) 240, and the artificial intelligence engine generated by the new artificial intelligence engine generation unit 230 is a candidate It can be expressed by dividing it into terms of artificial intelligence engine (new) 250 .

The reliability determination unit 260 may be implemented to determine the reliability of the candidate artificial intelligence engine (updated) 240 and the candidate artificial intelligence engine (new) 250 . The prediction value of the candidate artificial intelligence engine (update) 240 and the candidate artificial intelligence engine (new) 250 is compared with the existing financial service result, and the candidate artificial intelligence engine (update) 240 and the candidate artificial intelligence engine (new) ) 250 reliability of each may be determined.

The artificial intelligence engine determination unit 270 may be implemented to determine an artificial intelligence engine to be used in the credit evaluation model among candidate artificial intelligence engines.

If the reliability of the candidate artificial intelligence engine is higher than that of the reference artificial intelligence engine, the candidate artificial intelligence engine may be determined as an artificial intelligence engine and used in the credit evaluation model.

The reference artificial intelligence engine of the candidate artificial intelligence engine (update) 240 and the reference artificial intelligence engine of the candidate artificial intelligence engine (new) 250 may be different from each other. The reference artificial intelligence engine of the candidate artificial intelligence engine (update) 240 may be a pre-update artificial intelligence engine corresponding to the candidate artificial intelligence engine (update) 240 . The reference artificial intelligence engine of the candidate artificial intelligence engine (new) 250 may be an artificial intelligence engine having the highest reliability among artificial intelligence engines generating the same lower credit evaluation data as the candidate artificial intelligence engine (new) 250. . Alternatively, the reference artificial intelligence engine of the candidate artificial intelligence engine (new) 250 is an artificial intelligence engine corresponding to the median confidence level of the artificial intelligence engine generating the same sub-credit evaluation data as the candidate artificial intelligence engine (new) 250. can be

3 discloses a method in which the candidate credit evaluation model generation unit generates the candidate credit evaluation model by combining an artificial intelligence engine.

Referring to FIG. 3 , the candidate credit evaluation model generator 300 may be included in the credit evaluation model generator and implemented to generate a candidate credit evaluation model.

The candidate credit evaluation model generating unit 300 may include an artificial intelligence engine determining unit 310 , an artificial intelligence engine deploying unit 320 and a credit evaluation data generating unit 330 .

The artificial intelligence engine determination unit 310 may determine an artificial intelligence engine to be used when generating a candidate credit evaluation model. The candidate credit evaluation model may be a model specialized for sellers in a specific sector or a model generally applicable to all sellers. Accordingly, the artificial intelligence engine determination unit 310 may generate an artificial intelligence engine in consideration of the characteristics of the credit evaluation model.

The artificial intelligence engine determination unit 310 may determine lower-level credit evaluation data to be used for generating credit evaluation data in order to determine an artificial intelligence engine. After determining the low-level credit evaluation data, an artificial intelligence engine corresponding to the determined low-level credit evaluation data may be determined.

Artificial intelligence engines can be defined in two types. The artificial intelligence engine (first type) 350 may determine lower-level credit evaluation data based on the preprocessed credit evaluation basic data. The artificial intelligence engine (second type) 360 may determine other low-level credit evaluation data based on the input low-level credit evaluation data.

The artificial intelligence engine (first type) 350 receives credit evaluation basic data (eg, credit evaluation factor a (sales), credit evaluation factor b (net profit), credit evaluation factor c (return rate)), and Credit evaluation data can be determined.

The artificial intelligence engine (second type) 360 may receive lower level credit evaluation data generated by at least one artificial intelligence engine (first type) 350 and generate another lower level credit evaluation data. For example, the artificial intelligence engine (second type) 360 receives sub-credit evaluation data 1 (possibility of delinquency), sub-credit evaluation data 2 (possibility of default), and sub-credit evaluation data 3 (possibility of market change) as input, Another sub-credit evaluation data, credit rating, can be determined.

The artificial intelligence model determination unit determines the reliability of the artificial intelligence engine, the connection relationship between the artificial intelligence engine (first type) 350 and the artificial intelligence engine (second type) 360, the artificial intelligence engine layout structure, and the characteristics of the artificial intelligence engine. Considering this, an artificial intelligence engine can be determined.

The connection relationship between the artificial intelligence engine (first type) 350 and the artificial intelligence engine (second type) 360 is an artificial intelligence engine (first type) considering the input value of the artificial intelligence engine (second type) 360. ) 350 may include information about the connection possibility.

The artificial intelligence engine deployment structure is a hierarchical deployment between the artificial intelligence engine (first type) 350 and the artificial intelligence engine (second type) 360, the deployment of artificial intelligence engines in the same layer, and the same/similar sub-level credit evaluation data. It may be determined by considering whether or not the artificial intelligence engine generating the redundant deployment, the frequency of generation of credit evaluation basic data input to the artificial intelligence engine, and the like.

The characteristics of the artificial intelligence engine take into account characteristics of output data for the same/similar input data, and it may be determined that the characteristics of the artificial intelligence engine are relatively similar as values of the output data are similar.

The artificial intelligence engine deployment unit 320 may be implemented to generate a candidate credit evaluation model by deploying an artificial intelligence engine. An artificial intelligence engine may be deployed to generate highly reliable credit evaluation data. The artificial intelligence engine deployment unit 320 includes the reliability of the artificial intelligence engine, the connection relationship between the artificial intelligence engine (first type) 350 and the artificial intelligence engine (second type) 360, the artificial intelligence engine deployment structure, and artificial intelligence Various artificial intelligence engines can be deployed in consideration of engine characteristics and generation frequency of credit rating base data.

The credit evaluation data generation unit 330 may be implemented to generate credit evaluation data in a candidate credit evaluation model. The credit evaluation data generating unit 330 may input credit evaluation basic data to the candidate credit evaluation model generated by the artificial intelligence engine arranging unit 320 and generate credit evaluation data as an output value.

4 discloses a method of determining a model to be transmitted as a credit evaluation model from among candidate credit evaluation models in a credit evaluation model determination unit to the credit evaluation unit.

Referring to FIG. 4 , the credit evaluation model determiner determines the reliability of the candidate credit evaluation model as a first evaluation factor, and compares the candidate credit evaluation model with a reference credit evaluation model as a second evaluation factor to determine the candidate credit evaluation model. It can be determined by the credit rating model and transmitted to the credit rating department.

(1) Reliability determination (S400)

The credit evaluation model determination unit may determine reliability of the candidate credit evaluation model based on a result value output from the candidate credit evaluation model.

As described above, the credit evaluation model determiner may be implemented to determine, as a credit evaluation model, a model having a critical level of reliability in credit evaluation among a plurality of candidate credit evaluation models generated by the candidate credit evaluation model generation unit. Among the candidate credit evaluation models generated by the credit evaluation model generation unit, a credit evaluation model to be used for credit evaluation may be transmitted to the credit evaluation unit. Reliability may be determined by comparing a credit evaluation result of a candidate credit evaluation model based on existing credit evaluation basic data and an actual financial service result. As the credit evaluation result of the candidate credit evaluation model and the actual financial service result are relatively similar, it may be determined to have relatively high reliability.

(2) Comparison with the standard credit evaluation model (S410)

In the credit rating unit, various credit rating models may exist so that different credit rating models can be used adaptively according to circumstances. The credit evaluation model determination unit may determine a credit evaluation model of the credit evaluation unit having the most similar characteristics to the candidate credit evaluation model as the reference credit evaluation model.

The similarity of characteristics between credit rating models may be determined based on the similarity of input credit rating basic data, the similarity of output credit rating data, and the similarity of artificial intelligence models.

The credit rating model determining unit compares the reliability between the standard credit rating model and the candidate credit rating model, and evaluates the candidate's credit through difference between the output values of the standard credit rating model and the candidate credit rating model and error detection when inputting the same input values. You can decide whether or not to decide the model as a credit rating model.

5 discloses a method of generating credit evaluation data based on a credit evaluation model in a credit evaluation unit.

Referring to FIG. 5 , the credit evaluation unit may generate credit evaluation data 540 through a seller's credit evaluation based on at least one credit evaluation model.

The credit rating unit may generate the seller's credit rating data based on one credit rating model, but the credit rating unit determines a target credit rating model 520 adaptively applicable to the seller based on the seller characteristic information 500. And, credit evaluation data 540 may be generated based on the determined target credit evaluation model 520 .

For example, a target credit evaluation model 520 for credit evaluation that is most suitable for seller characteristic information 500 based on seller information such as seller's sales product, seller's product sales platform, seller's sales, seller's net profit, etc. can be determined

The credit evaluation unit may determine reliability of the credit evaluation model according to seller characteristic information based on feedback information of each of the plurality of credit evaluation models. Also, the credit evaluation unit may determine a reliability level for each seller characteristic information for each of the plurality of credit evaluation models. Specifically, the seller characteristic information 500 may be vectorized and expressed on a space based on each sub-seller characteristic information, and a seller group may be formed based on the seller characteristic information through distance information between spaces, and each seller group Reliability of each seller group of the credit evaluation model may be determined by comparing the credit evaluation data with the financial service result data. The reliability level may be determined in consideration of statistical characteristics of reliability for a seller group for each credit rating model.

The credit evaluation unit may generate credit evaluation data for the seller by determining a credit evaluation model having a relatively high reliability level as a target credit evaluation model based on the seller characteristic information.

6 discloses a method for determining shifting (or correction) of a credit evaluation model or replacement of a credit evaluation model in a credit evaluation model management unit.

Referring to FIG. 6 , when the credit evaluation data generated by the credit evaluation model differs from the financial service result data, the credit evaluation model management unit may shift the credit evaluation model or replace the credit evaluation model.

The credit rating model management unit may determine prediction accuracy of the credit rating model through comparison between credit rating data of the credit rating model and financial service result data. When the prediction accuracy of the credit rating model is less than or equal to the critical accuracy, the credit rating model management unit may determine the credit rating model as a credit rating model to be managed.

The credit evaluation model manager may determine shifting or replacement of a credit evaluation model to be managed.

Credit rating model shifting can be performed when there is a difference between the credit rating data of the credit rating model to be managed and the financial service result data, but the difference has a negative or positive tendency and exists within a critical range. there is.

Credit evaluation model replacement is performed when there is a difference between the credit evaluation data of the managed credit evaluation model and the financial service result data, and the difference does not have a negative or positive trend and exceeds a critical range. can

(1) Credit rating model shifting (600)

1) When the financial service result data is more positive in terms of financial services than the credit evaluation data of the managed credit evaluation model

When the credit evaluation model shifting 600 is not performed, a conservative financial service (low loan amount, high interest rate) may be provided based on credit evaluation data of a credit evaluation model to be managed. Accordingly, the credit evaluation data may be set to be close to actual financial service data by shifting the credit evaluation data through the credit evaluation model shifting 600 .

2) When financial service outcome data is more negative in terms of financial services than credit evaluation data

When the credit evaluation model shifting 600 is not performed, a high-risk financial service (high loan amount, low interest rate) may be provided based on credit evaluation data of a credit evaluation model to be managed. Accordingly, the credit evaluation data may be set to be close to actual financial service data by shifting the credit evaluation data through the credit evaluation model shifting 600 .

(2) Replace credit rating model (620)

1) Creating a new credit evaluation model

To replace the credit evaluation model to be managed, the credit evaluation data and the financial service result data may be transmitted as feedback data to the credit evaluation model generator, and the replacement may be performed with the credit evaluation model newly created by the credit evaluation model generator.

2) Using existing credit rating models

A replaceable credit evaluation model among existing credit evaluation models may be searched for and replacement with the corresponding credit evaluation model may be performed.

Credit that can predict actual financial service results with high accuracy when compared to actual financial service results when existing input values of credit rating models to be managed are inserted, or when credit rating model shifting is performed If an evaluation model exists, a corresponding credit evaluation model may be replaced by shifting the model.

7 discloses a method for determining shifting (or correction) of a credit evaluation model or replacement of a credit evaluation model in a credit evaluation model management unit.

Referring to FIG. 7 , replacement of a credit evaluation model may be determined differently according to the classification of credit evaluation data.

The credit evaluation data may be classified into a high-risk class 710, a medium-risk class 720, and a low-risk class 730. The higher the risk class 710, the higher the probability of default when providing financial services, and the lower the probability of default when providing financial services, the lower the risk class 730 may be.

When erroneous judgments of the credit evaluation data of the credit evaluation model accumulate, replacement and updating of the credit evaluation model may be determined. When an error occurs in the credit evaluation data toward the high-risk class 710, the medium-risk class 720, and the low-risk class 730, it may be determined that the credit evaluation model needs to be replaced or updated. For example, when the credit rating model evaluates the low-risk class 730, but a problem actually occurs in the financial service, the credit rating model evaluates the high-risk class 710 as the case where a problem actually occurs in the financial service. Could be a problem.

The credit evaluation model evaluation unit may track and accumulate financial service problems occurring in each of the low-risk class 730 , the medium-risk class 720 , and the high-risk class 710 . Accumulate the financial service problems that occurred in the low-risk class (730), medium-risk class (720), and high-risk class (710), but gradually increase in the high-risk class (710), medium-risk class (720), and low-risk class (730). By assigning a higher weight, the evaluation value 750 of the credit evaluation model may be negatively evaluated by minus the reference value.

When the evaluation value 750 of the credit evaluation model falls below a threshold value, replacement or update (or shifting) may be determined.

As described above, update is a case in which credit evaluation data can be corrected with some correction values, and replacement is a case in which credit evaluation data can be corrected with some correction values.

According to an embodiment of the present invention, a separate replacement or update for each class is also possible. For example, if a financial service problem occurs only in a specific class among the low-risk class 730, the medium-risk class 720, and the high-risk class 710, the credit evaluation model corresponding to the class is replaced or updated, and the remaining In the case of classes, the credit rating model can be kept as is.

8 discloses a method for monitoring a credit rating model in real time.

Referring to FIG. 8 , after credit evaluation data 800 for a specific seller is determined and a financial service according to the credit evaluation data 800 is provided, tracking of the seller may be performed. Tracking of the seller may be tracking whether the seller's credit evaluation basic data 810 is changed or tracking the financial service result 820 .

In performing tracking, the time point of generating the credit evaluation basic data 810 and the time point of evaluating the financial service result 820 840 may be set in plurality.

In addition, by setting a plurality of evaluation time points 840 , evaluation errors for the credit evaluation basic data 810 and the financial service result 820 generated according to the evaluation time point 840 may be reduced.

The timing of evaluating the change in the credit evaluation basic data 810 and the financial service result 820 is set to a plurality of times periodically/non-periodically, and the financial service problem ( 850) may be determined.

The timing of evaluating the change in the basic credit evaluation data 810 and the financial service result 820 may be determined in consideration of the seller characteristic information 830 . A sharp decrease in sales of sellers selling products not related to seasonality is a factor that can affect financial services, but a sharp decrease in sales of sellers selling products related to seasonality can affect financial services when seasonal factors are considered. It may be an element that does not exist.

Therefore, the timing of evaluating the change in the credit evaluation basic data 810 and the financial service result 820 is determined in consideration of the seller characteristic information 830, and it is determined whether the result changing according to the evaluation timing is a financial service problem. The criteria may also be set differently.

In FIG. 9 , a credit evaluation device for monitoring basic credit evaluation data and financial service results of a seller and updating a credit evaluation model is disclosed.

Referring to FIG. 9 , the credit evaluation device includes a credit evaluation unit 910, a monitoring information collection unit 920, a class classification unit 930, a credit evaluation model evaluation unit 940, and a credit evaluation model management unit 950. can do.

The credit evaluation unit 910 may be implemented to determine seller's credit evaluation data based on the determined credit evaluation model in consideration of seller characteristics.

The monitoring information collection unit 920 may be implemented to monitor the seller's basic credit evaluation data and financial service results, and to collect the credit evaluation basic data and financial service results. The credit rating base data and financial service results collected by the monitoring information can be used to update or replace the credit rating model.

The class classification unit 930 may be implemented to classify the seller's class into a low-risk class, a medium-risk class, and a high-risk class based on the seller's credit data.

The credit evaluation model evaluation unit 940 may be implemented for evaluation of the credit evaluation model. The credit evaluation model evaluation unit 940 is implemented to evaluate the credit evaluation model applied to the seller based on monitoring information (basic credit evaluation data and financial service results) of the seller corresponding to the low-risk class, medium-risk class, and high-risk class. It can be.

The credit evaluation model management unit 950 may be implemented to replace or update (or shift) the credit evaluation model based on the evaluation result of the credit evaluation model.

10 discloses a method of generating a credit evaluation model based on an artificial intelligence engine.

Referring to FIG. 10 , a credit evaluation model may be generated based on extension and/or stratification of a plurality of artificial intelligence engines.

A reference artificial intelligence engine 1000 that is preferentially a reference for extension of an artificial intelligence engine for implementing a credit evaluation model may be determined. The reference artificial intelligence engine 1000 is an engine having relatively high reliability, and related artificial intelligence engines 1010 having relevance based on the reference artificial intelligence engine 1000 may be expanded and set. A plurality of reference artificial intelligence engines 1000 may be set.

In order to determine the reference artificial intelligence engine 1000, seller characteristics according to the purpose of the credit evaluation model may be considered, or scalability of the reference artificial intelligence engine 1000 may be considered. When the reference artificial intelligence engine 1000 is an artificial intelligence engine (second type), the scalability of the reference artificial intelligence engine 1000 is the connectability of the artificial intelligence engine (first type) considering the input value and the reference artificial intelligence engine ( 1000) may be determined in consideration of the generation frequency of input data. When the reference artificial intelligence engine 1000 is an artificial intelligence engine (first type), the generation frequency of input data input to the reference artificial intelligence engine 1000 may be considered.

After the reference artificial intelligence engine 1000 is determined, extension to connect other artificial intelligence engines based on the reference artificial intelligence engine 1000 may be performed. Through an extension process, other related artificial intelligence engines for generating credit evaluation data based on the reference artificial intelligence engine 1000 may be connected.

In the present invention, when generating a credit evaluation model, each artificial intelligence engine is modularized (or made into parts) to be designed, and the artificial intelligence engine to improve the modularized (or made into parts) credit evaluation model may be replaced in module units.

The plurality of artificial intelligence engines constituting the credit evaluation model may be divided into a core module 1020, a related module 1030, and a non-core module 1040. The core module 1020 may be a reference artificial intelligence engine, and the related module 1030 may be an artificial intelligence engine directly connected to the reference artificial intelligence engine. In addition to the core module 1020 and related modules 1030, other artificial intelligence engines may be classified as non-core modules 1040.

Also, according to an embodiment of the present invention, one credit evaluation model may be created, and specialized artificial intelligence engines applicable to a specific seller group may be layered and deployed. That is, a plurality of artificial intelligence engines may be layered in a vertical direction, and credit evaluation data for the seller may be generated.

The generalized artificial intelligence engine 1075 is an artificial intelligence engine generated to generate universal credit evaluation data without considering seller characteristics, and the credit evaluation model formed based on the generalized artificial intelligence engine 1075 is a generalized credit evaluation model. (1070). An artificial intelligence engine generated to generate general-purpose credit evaluation data for a seller in consideration of seller characteristics is a specialized artificial intelligence engine 1085, and a credit evaluation model formed based on the specialized artificial intelligence engine 1085 is a specialized credit evaluation model. (1080).

The first layer 1050 may be implemented based on a generalized artificial intelligence engine 1075 as a generalized credit evaluation model 1070 that generates seller's credit evaluation data based on input values regardless of seller characteristics. The second layer 1060 to the n-th layer may include a specialized artificial intelligence engine 1085 as a specialized credit evaluation model 1080 generated according to seller characteristics in consideration of seller characteristics.

Hereinafter, a method for extending and layering an artificial intelligence engine for implementing a credit evaluation model is disclosed in the present invention.

11 discloses a reference artificial intelligence engine and an extension of an artificial intelligence engine based on the reference artificial intelligence engine.

Referring to FIG. 11 , after the reference artificial intelligence engine 1100 is determined to expand the artificial intelligence engine, other related artificial intelligence engines 1120 corresponding to input values and output values of the reference artificial intelligence engine 1100 may be determined. there is.

When the sub-credit evaluation data used as the input value of the reference artificial intelligence engine 1100 is generated based on another artificial intelligence engine, selection of a related artificial intelligence engine 1120 to be connected with the reference artificial intelligence engine 1100 is performed. It can be.

For example, the lower credit evaluation data A is input to the reference artificial intelligence engine 1100, and the lower credit evaluation data A may be data generated based on a specific artificial intelligence engine. In this case, a related artificial intelligence engine 1120 to be connected to the reference artificial intelligence engine 1100 may be selected from among a plurality of artificial intelligence engines capable of generating the lower credit evaluation data A. The reliability of the artificial intelligence engine may be considered for selection of the relevant artificial intelligence engine 1120 . In addition, the more similar the generation cycles between the input data are, the more accurate the credit evaluation result considering the change in the seller's credit over time can be generated. Therefore, the data generation cycle of the input data input to the reference artificial intelligence engine 1100 can be considered. there is.

As the reliability of the artificial intelligence engine is relatively high, the probability of being determined as a related artificial intelligence engine increases relatively, and as the data generation cycles of the input data of the artificial intelligence engine are relatively similar to each other, the probability of being determined as a related artificial intelligence engine 1120 increases. can be increased relatively.

In the present invention, in order to increase the accuracy of credit evaluation data, extension considering the data generation cycle may be performed. If the data generation cycle does not match, the accuracy of the generated credit evaluation data may be lowered. Thus, the artificial intelligence engine can be extended to fit data generation cycles.

In Figure 12, a layering of artificial intelligence engines for implementing the credit rating model is revealed.

Referring to FIG. 12 , the first layer of the credit evaluation model may be a generalized credit evaluation model 1250 applicable to all sellers.

The second to nth layers of the credit evaluation model may be a specialized credit evaluation model 1260 applied to a specific seller. Layers of the credit evaluation model may be connected to each other in the input and output of the artificial intelligence engine constituting the credit evaluation model, and based on the input and output connected to each other, learning by sharing feedback results is performed, and between the output credit evaluation data A comparison can be made.

For example, the second layer may include a characterization artificial intelligence engine and/or a generalization artificial intelligence engine as a credit rating model for a seller of clothes. The specialized artificial intelligence engine and the generalized artificial intelligence engine included in the second layer may be connected and shared with other layers, for example, the first layer or the third through nth layers. That is, an artificial intelligence engine included in one layer may be vertically connected to another layer and used to generate credit evaluation data.

Also, parallel sharing of artificial intelligence engines between layers is possible in addition to sharing of artificial intelligence engines between layers. For example, an artificial intelligence engine of another layer may be used in parallel with a second layer of artificial intelligence engine to generate different credit evaluation data.

The input and output results of the artificial intelligence engine between these layers can generate a plurality of credit evaluation data, and the plurality of credit evaluation data can be utilized to provide financial services to sellers.

As shown in the example of FIG. 12 , the second layer may share an artificial intelligence engine with the first layer and the n-th layer or may generate a credit evaluation model by configuring them in parallel. A shared artificial intelligence engine may be expressed as a shared artificial intelligence engine 1200, and an artificial intelligence engine deployed in parallel may be expressed as a parallel artificial intelligence engine 1220.

Credit evaluation data generated based on sharing by the sharing artificial intelligence engine 1200 may be reference credit evaluation data. The credit evaluation data generated based on the parallel arrangement of the parallel artificial intelligence engine 1220 may be additional credit evaluation data. Depending on the reliability of the additional credit evaluation data, ON/OFF of the parallel artificial intelligence engine in parallel arrangement is determined, and additional credit evaluation data can be selectively generated and used to provide financial services to the seller.

For example, the first artificial intelligence engine 1270 is shared and used as a shared artificial intelligence engine 1200, and each of the second artificial intelligence engine 1280 and the third artificial intelligence engine 1290 is a parallel artificial intelligence engine 1220. ), it can be assumed that they are arranged in parallel on the second layer and the third layer. Each of the second artificial intelligence engine 1280 and the third artificial intelligence engine 1290, which are parallel artificial intelligence engines, may be controlled ON/OFF to generate various credit evaluation data. That is, according to the feedback result, artificial intelligence ON/OFF arranged in parallel may be determined, and additional credit evaluation data may be generated.

In addition to this, the shared artificial intelligence engine 1200 and/or the parallel artificial intelligence engine 1220 may be additionally configured based on the feedback result. In addition, in the present invention, parallel artificial intelligence engines 1220 are defined as pairs based on relevance for each artificial intelligence engine, and additional credit evaluation is performed through various combinations based on ON/OFF of the artificial intelligence engines defined as pairs. data can be created. ON/OFF of the parallel artificial intelligence engine 1220 takes into account the reliability of the parallel artificial intelligence engine 1220, and as a result of the feedback, the parallel artificial intelligence engine 1220 having a reliability level higher than or equal to the critical level reliability parallel artificial intelligence engine 1220 lower than the critical level of reliability Credit evaluation data may be generated by combining them so as to have a higher probability of operating in the ON state.

13 discloses a method of evaluating a credit evaluation model in consideration of a feedback result for the credit evaluation model.

Referring to FIG. 13 , evaluation of the credit evaluation model may be performed for each module.

As described above, the plurality of artificial intelligence engines constituting the credit evaluation model may be divided into a core module 1310, a related module 1320, and a non-core module 1330. The core module 1310 may be a reference artificial intelligence engine, and the related module 1320 may be an artificial intelligence engine directly connected to the reference artificial intelligence engine. Artificial intelligence engines other than the core module 1310 and related modules 1320 may be classified as non-core modules 1330.

Evaluation of the credit evaluation model of the present invention may be performed for core module 1310, related module 1320 and non-core module 1330. When the judgment result of the financial service based on the credit evaluation data is incorrect for evaluation of the credit evaluation model, a search for a module that caused a judgment error may be performed.

A plurality of lower level credit evaluation data constituting the credit evaluation data causing the judgment error may be divided, and target lower level credit evaluation data 1300 causing the judgment error may be determined from among the divided plurality of lower level credit evaluation data. The target lower credit evaluation data 1300 may be determined by considering an error between actual lower credit evaluation data based on actual financial service results and prediction lower credit evaluation data predicted by an artificial intelligence engine.

After determining the target lower credit evaluation data 1300 that caused the judgment error, a core module 1310, a related module 1320, or a non-core module 1330 related to the target lower credit evaluation data may be determined. The judgment error starting artificial intelligence engine 1350 and the judgment error effecting artificial intelligence engine 1360 affected by the judgment error may be determined.

If the error in the credit evaluation data that determines the financial service result exceeds a set threshold, replacement or shifting is performed for the judgment error starting artificial intelligence engine 1350 and the judgment error effecting artificial intelligence engine 1360 affected by the judgment error. It can be. The replacement and shifting may be performed in the same manner as shifting (or correcting) the credit evaluation model or replacement of the credit evaluation model in the credit evaluation model management unit disclosed in FIG. 6 .

14 shows a method for determining a core module, a related module, or a non-core module related to the target sub-credit evaluation after determining the target sub-credit evaluation data that caused the judgment error, and a judgment error starting artificial intelligence engine from which propagation of the judgment error started; and A method for determining an error-influenced artificial intelligence engine affected by an error-of-decision is disclosed.

Referring to FIG. 14 , in order to search for an artificial intelligence engine that lowers the reliability of credit evaluation data, an artificial intelligence engine that generates lower credit evaluation data that determines credit evaluation data may be searched in reverse.

After the financial service is provided to the seller, the feedback result of the financial service may be delivered as feedback on lower credit evaluation data that generated the seller's credit evaluation data. An artificial intelligence engine that generates misjudged low-level credit evaluation data among the low-level credit evaluation data may be a target artificial intelligence engine.

There may be a plurality of target artificial intelligence engines 1400, and the plurality of target artificial intelligence engines 1400 may be judgment error starting artificial intelligence engines or error effect artificial intelligence engines.

After the target artificial intelligence engine 1400 determines, the cause of the target artificial intelligence engine 1400's erroneous determination may be an input data error 1440 and/or an artificial intelligence engine error 1460.

When the parallel artificial intelligence engine 1420 of the target artificial intelligence engine 1400 makes the same error determination, it is determined as an input data error 1440, and the parallel artificial intelligence engine 1420 of the target artificial intelligence engine 1400 has another error. When making a decision, it may be determined as an artificial intelligence engine error 1460.

If it is determined to be an artificial intelligence engine error 1460, the target artificial intelligence engine 1400 is set to the artificial intelligence engine 1480 starting with a judgment error, and if it is determined to be an input data error, the target artificial intelligence engine affects the artificial intelligence engine with a judgment error. It can be set to intelligence engine 1470. The credit rating model may be corrected through replacement of the judgment error start artificial intelligence engine 1480 and shifting of the judgment error impact artificial intelligence engine 1470 .

Embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. medium), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes generated by a compiler. A hardware device may be modified with one or more software modules to perform processing according to the present invention and vice versa.

Although the present invention has been described above with specific details such as specific components and limited embodiments and drawings, these are only provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments, and the present invention Those with ordinary knowledge in the technical field to which the invention belongs may seek various modifications and changes from these descriptions.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments and should not be determined, and all scopes equivalent to or equivalently changed from the claims as well as the claims described below are within the scope of the spirit of the present invention. will be said to belong to

Claims

How to update or replace your credit scoring model:

Generating a plurality of artificial intelligence engines for credit evaluation by the artificial intelligence engine generating unit;

generating a credit evaluation model based on the plurality of artificial intelligence engines; and

A credit rating unit generating credit rating data for the seller based on the credit rating model,

The credit evaluation model is generated based on the expansion and/or layering of a plurality of artificial intelligence engines;

Wherein the evaluation of the credit evaluation model is performed based on core modules, related modules, and non-core modules constituting the credit evaluation model.
According to claim 1,

The core module is a reference artificial intelligence engine,

the related module is an artificial intelligence engine directly connected to the reference artificial intelligence engine;

Wherein the non-core module is the remaining artificial intelligence engine constituting the credit rating model besides the core module and the related module.
According to claim 2,

The evaluation of the credit evaluation model is performed through a search for a module that caused a judgment error when the determination result of the financial service based on the credit evaluation data is incorrect,

The search divides a plurality of lower-level credit evaluation data constituting the credit evaluation data that caused the judgment error, and determines target lower-level credit evaluation data that caused the judgment error among the divided plurality of lower-level credit evaluation data; After determining the target sub-credit rating data, determining the core module, the related module or the non-core module related to the target sub-credit rating data, and a judgment error starting artificial intelligence engine from which the propagation of the judgment error begins, and the judgment A method comprising determining a judgment error impact artificial intelligence engine affected by the error.
A credit evaluation device that generates an artificial intelligence-based credit evaluation model,

an artificial intelligence engine generation unit configured to generate a plurality of artificial intelligence engines for credit evaluation;

a credit evaluation model generator configured to generate a credit evaluation model based on the plurality of artificial intelligence engines; and

Including a credit rating unit configured to generate credit rating data for a seller based on the credit rating model;

The credit evaluation model is generated based on the expansion and/or layering of a plurality of artificial intelligence engines;

The credit evaluation apparatus, characterized in that the evaluation of the credit evaluation model is performed based on core modules, related modules and non-core modules constituting the credit evaluation model.
According to claim 4,

The core module is a reference artificial intelligence engine,

the related module is an artificial intelligence engine directly connected to the reference artificial intelligence engine;

The non-core module is the remaining artificial intelligence engine constituting the credit evaluation model in addition to the core module and the related module.
According to claim 5,

The evaluation of the credit evaluation model is performed through a search for a module that caused a judgment error when the determination result of the financial service based on the credit evaluation data is incorrect,

The search divides a plurality of lower-level credit evaluation data constituting the credit evaluation data that caused the judgment error, and determines target lower-level credit evaluation data that caused the judgment error among the divided plurality of lower-level credit evaluation data; After determining the target sub-credit rating data, determining the core module, the related module or the non-core module related to the target sub-credit rating data, and a judgment error starting artificial intelligence engine from which the propagation of the judgment error begins, and the judgment A credit rating apparatus characterized by determining an error-affected judgment error-affected artificial intelligence engine.