KR102534396B1 - Method of operating artificial intelligence algorithms, apparatus for operating artificial intelligence algorithms and storage medium for storing a software operating artificial intelligence algorithms - Google Patents

Abstract

The disclosed embodiments relate to an artificial intelligence (AI) algorithm execution method, an artificial intelligence algorithm execution device, and a storage medium storing software for performing an artificial intelligence algorithm.
According to an embodiment, a class distribution detector detecting a binary label distribution of binary label data; a label distribution processing unit that classifies the data according to the detected binary label distribution and processes the classified data according to the binary label distribution; and a learning processing unit learning the data processed by the label distribution processing unit using artificial intelligence algorithms selected according to the label distribution of the data processed by the label distribution processing unit.

Description

Artificial intelligence (AI) algorithm execution method, artificial intelligence algorithm execution device, and storage medium for storing artificial intelligence algorithm execution software OPERATING ARTIFICIAL INTELLIGENCE ALGORITHMS}

The following disclosure relates to an artificial intelligence (AI) algorithm execution method, an artificial intelligence algorithm execution device, and a storage medium for storing software for performing an artificial intelligence algorithm.

In the process of creating an artificial intelligence model, it is very important to secure a high-quality data set for learning. However, in most cases, the distribution and characteristics of actual data are biased. Under this environment, artificial intelligence models require a lot of time and material costs because the process of data modification, selection, and processing must be pre-determined.

For example, even if the same data and learning algorithm are used from the perspective of using an artificial intelligence model, the application method can vary greatly depending on the user's purpose.

Data processing, including data imbalance processing, and when selecting a learning algorithm suitable for the purpose of use, a person intervened to process such data, which greatly limited the use and selection of artificial intelligence models.

In addition, although attempts are being made to utilize artificial intelligence technology in various fields, there is also a problem in that the usability of artificial intelligence models is extremely limited as artificial intelligence models are generally developed using data specialized for each field.

For example, in the field of network security control, such as an intrusion detection system in the field of computer security, the class distribution of input data is very flexible, and there is also a problem that AI model results are easily biased depending on the nature of the data.

Security data collected in the real environment, such as true positives or false positives, which are detection results of security events, includes imbalanced data of binary classes (classification of elements of a set into two groups according to classification rules, binary class).

In the case of security events, since there is a large difference between the true positive rate and the false positive rate, it is necessary to apply the optimal algorithm when performing an artificial intelligence algorithm on such imbalanced data.

However, the user's arbitrary data pre-processing may affect the model performance because there is a clear difference from the data collected in the real environment.

If a user who does not know the characteristics of data applies an artificial intelligence algorithm, or if a user who lacks understanding of an artificial intelligence algorithm makes a wrong choice of algorithm or data processing, bad results can be obtained.

The following disclosure is to solve this problem, and to store an artificial intelligence algorithm performing method, an artificial intelligence algorithm performing device, and an artificial intelligence algorithm performing software that can optimally apply an artificial intelligence algorithm according to an imbalance of input data. to provide a storage medium.

The purpose of the disclosure below is to store an artificial intelligence algorithm performing method, an artificial intelligence algorithm performing device, and an artificial intelligence algorithm performing software that can automatically apply an artificial intelligence algorithm according to the imbalance of input data without user's arbitrary data preprocessing. to provide a storage medium.

Another object of the disclosure below is to provide a method for performing an artificial intelligence algorithm, a device for performing an artificial intelligence algorithm, and an artificial intelligence algorithm capable of performing an optimal artificial intelligence algorithm even for a user who does not know the characteristics of data or a user who lacks understanding of an artificial intelligence algorithm. It is to provide a storage medium for storing software for performing an algorithm.

One disclosed embodiment includes a class distribution detection unit for detecting a binary label distribution of binary label data; a label distribution processing unit that classifies the data according to the detected binary label distribution and processes the classified data according to the binary label distribution; and a learning processing unit learning the data processed by the label distribution processing unit using artificial intelligence algorithms selected according to the label distribution of the data processed by the label distribution processing unit.

The label distribution processor may process the classified data according to whether a label distribution of the classified data is one of a single label distribution, an imbalanced label distribution, and a balanced label distribution.

The label distribution processing unit converts the label distribution of the classified data into a single label distribution by over-fitting or under-fitting the classified data when the label distribution of the classified data is an imbalanced label distribution. Alternatively, it can be changed to a data set with a balanced label distribution.

The learning processing unit may apply an unsupervised learning artificial intelligence algorithm to the classified data when the label distribution of the classified data is a single label distribution.

The learning processing unit, when the label distribution of the classified data is a balanced label distribution, selects any one of a Deep Neural Network (DNN), a Convolution Neural Network (CNN), a Recurrent Neural Network (RNN), and a Binarized Support Vector Machine (BSVM) One artificial intelligence algorithm can be applied.

Another disclosed embodiment includes detecting a binary label distribution of input data from the input data; classifying the data according to the detected binary label distribution and processing the classified data according to the binary label distribution; and learning each of the processed data by applying AI algorithms selected according to the label distribution.

Another disclosed embodiment detects a binary label distribution of input data from the data; classifying the data according to the detected binary label distribution and processing the classified data according to the binary label distribution; and a storage medium storing software for performing an artificial intelligence algorithm for learning each of the processed data by applying artificial intelligence algorithms selected according to the label distribution of the classified data.

According to the example disclosed below, even if the input data has an imbalanced distribution, the artificial intelligence algorithm can be optimally performed in consideration of this.

According to the example disclosed below, an artificial intelligence algorithm may be automatically applied according to an imbalance of input data without arbitrary data preprocessing by a user.

According to the example disclosed below, even a user who does not know the characteristics of data or a user who lacks understanding of an artificial intelligence algorithm can perform an optimal artificial intelligence algorithm.

1 is a conceptual diagram conceptually showing an embodiment of an artificial intelligence model platform proposed in the present invention.
2 is a block diagram of an artificial intelligence model platform according to an embodiment of the present invention
3 is a block diagram of a device for recommending feature information according to an embodiment of the present invention.
4 is a block diagram of a normalization method recommendation apparatus according to an embodiment of the present invention.
5 is a diagram illustrating an embodiment of performing an artificial intelligence algorithm according to a label distribution of imbalanced data;
6 is a diagram illustrating an example of a label (class) distribution processing unit according to an embodiment.
7 is a diagram illustrating an example of selectively applying an artificial intelligence algorithm using data processed according to the distribution of input data;
8 is a diagram illustrating an example of a method of performing an artificial intelligence algorithm that selectively applies an artificial intelligence algorithm using data processed according to the distribution of data.

In order to easily describe the following embodiments, embodiments will be described with reference to the accompanying drawings.

Currently, the real-time security control service provided by the Science and Technology Cyber Safety Center is based on security events detected and collected by the threat management system (TMS), and rule-based analysis and response support by security control personnel. has a service structure.

However, the number of security events detected by TMS is increasing exponentially, and it is reaching a realistic limit for security control personnel to analyze such large-capacity entire security events.

In addition, since existing security control services depend on the expertise and experience of security control personnel, analysis leveling is not realized, such as work concentration in which analysis of specific security events is concentrated or deviations in analysis results occur. Situation is also happening.

After all, in the current situation where the number of security events detected by TMS is explosively increasing, it is necessary to innovate the service structure of the existing security control service that depends on the analysis of security control personnel.

Accordingly, it is possible to consider a security control service structure that utilizes an artificial intelligence model that can replace the security control agent's analysis.

In the present invention, it is intended to provide an artificial intelligence model platform capable of generating an artificial intelligence model for security control.

In particular, in the present invention, it is intended to provide an artificial intelligence model platform that allows even general users who are not familiar with security control technology to create an optimal artificial intelligence model for security control.

1 conceptually shows an embodiment of the artificial intelligence model platform proposed in the present invention.

As shown in FIG. 1, the artificial intelligence model platform of the present invention collects and processes various data necessary for generating an artificial intelligence model for security control and artificial intelligence based on various data collected and processed in the collection function. Management responsible for various settings and user management related to collection/AI functions based on AI functions that create intelligence models and manage performance and history related to them, and UI (User Interface) provided to system administrators and general users function can be distinguished.

In addition, the artificial intelligence model platform of the present invention includes a search engine that periodically collects source security data newly generated from big data integrated storage, and loads various data from the collection function into the search engine to turn the search engine into data. Can be used as storage.

In this case, various modules (eg, collection/feature extraction/normalization/output) belonging to the collection function may operate based on the search engine (data storage).

Hereinafter, the configuration of the artificial intelligence model platform and the role of each configuration will be described in detail in an embodiment of the present invention with reference to FIG. 2 .

The artificial intelligence model platform 100 of the present invention includes a data collection module 110, a feature extraction module 120, a normalization module 130, a data output module 140, and a model generation module 150.

Furthermore, the artificial intelligence model platform 100 of the present invention may further include a performance management module 160 and a UI module 170.

All or at least part of the configuration of the artificial intelligence model platform 100 may be implemented in the form of hardware modules or software modules, or may be implemented in the form of a combination of hardware modules and software modules.

Here, the software module may be understood as, for example, a command executed by a processor that controls operation within the artificial intelligence model platform 100, and such a command may have a form loaded in a memory within the artificial intelligence model platform 100. will be able to have

As a result, the artificial intelligence model platform 100 according to an embodiment of the present invention realizes the technology proposed in the present invention, that is, a technology that can generate an optimal artificial intelligence model for security control, through the above-described configuration, , Hereinafter, each component in the artificial intelligence model platform 100 for realizing this will be described in more detail.

First, the UI module 170 performs at least one of a specific search condition of the data collection module 110, feature information of the feature extraction module 120, a normalization method of the normalization module 130, and a condition of the data output module 140. Provides a UI (User Interface) for setting.

For example, the UI module 170, according to the operation of a system manager or general user (hereinafter collectively referred to as a user) who wants to create an artificial intelligence model for security control in the artificial intelligence model platform 100 of the present invention, data, A UI for setting at least one of a specific search condition of the collection module 110, feature information of the feature extraction module 120, a normalization method of the normalization module 130, and a condition of the data output module 140 is provided.

Accordingly, the UI module 170, based on the provided UI, various settings related to the collection/artificial intelligence function, specific search conditions of the data collection module 110 for the artificial intelligence model to be created later, and feature extraction module. Characteristic information of 120, normalization method of normalization module 130, condition of data output module 140, etc. are stored/managed in user information/setting information storage.

The data collection module 110 collects security events to be used as learning/test data from source security data according to a specific search condition previously set by a user.

For example, as specific search conditions for the data collection module 110, a date (or period), number of cases, and IP to be used as learning/test data may be set.

Accordingly, when a specific search condition is a date (or period), the data collection module 110 may collect security events belonging to a set date (or period) from source security data.

Alternatively, when a specific search condition is the number of cases, the data collection module 110 may collect a set number of security events (eg, 500,000 cases) from source security data at a designated time point.

Alternatively, when the specific search condition is IP, the data collection module 110 may collect security events whose IP set from source security data matches the source IP or destination IP.

Of course, as a specific search condition, a combination of date (or period), number of cases, IP, etc. may be set.

In this case as well, the data collection module 110 may collect security events according to a combination of a set date (or period), case number, IP, etc. from source security data.

More specifically, in collecting security events from source security data as described above, the data collection module 110 may limit the maximum number of concurrently performed collections in order to reduce system load.

For example, when collecting security events belonging to a set date (or period) from source security data, the total number of security event collections belonging to the set date (or period) is 1000,000, and the maximum number of concurrent collections is We can assume 500,000 cases.

In this case, the data collection module 110 determines that the total number of collections this time exceeds the maximum number of simultaneous collections, and stores the collections exceeding the maximum number among the total number of collections this time in a queue. After that, you can proceed sequentially.

That is, the data collection module 110 collects/processes 500,000 of the total number of collections in chronological order among the total number of collections of 1000,000 this time, but queues 500,000 collections exceeding the maximum of 500,000. After saving in the queue, it can be collected/processed sequentially.

In this case, in the case of 500,000 collected cases that are stored in the queue and then proceeded, the data collection module 110 collects security events only for data prior to the point of occurrence of the collected cases in the source security data.

In other words, in the case of 500,000 collection cases that are stored in the queue and proceeding after being stored in the queue, out of the total number of 1000,000 cases collected this time, there is a difference between the time when the collection event occurred and the time when the actual collection/procedure occurred, preventing errors in security event collection caused by this. In order to do this, security events are collected only from the original security data prior to the point of occurrence of the collection case.

On the other hand, it has been mentioned that the artificial intelligence model platform 100 of the present invention includes a search engine that periodically collects newly generated original security data from big data integrated storage.

In this case, the data collection module 110 may collect security data from source security data in a search engine (data store).

Since the big data integrated storage storage is a storage used not only by the artificial intelligence model platform 100 of the present invention but also by other systems, when a large amount of data (security event) is collected from the big data integrated storage storage, the big data integrated storage storage It can create a load and affect other systems.

However, in the present invention (data collection module 110), the data collection module 110 does not collect security events directly from the big data integrated storage, but only periodically generates original security data from the big data integrated storage. Since security events are collected based on the collected search engine, the aforementioned big data integrated storage storage load problem can be avoided.

The feature extraction module 120 extracts feature information previously set for the security event collected by the data collection module 110, that is, feature information previously set by the user.

When creating an artificial intelligence model, in order to classify data (security events) with an artificial intelligence algorithm, it is necessary to find out what characteristics the data (security events) have and make them into vectors. This process is called feature information extraction process.

The feature extraction module 120 is responsible for performing a feature information extraction process with respect to security events collected by the data collection module 110 .

In addition, the feature information of each security event extracted by the feature extraction module 120 will be used for machine learning (eg, deep learning) when generating an artificial intelligence model described later.

In particular, in the present invention, the user can set a single feature as feature information and set a composite feature.

Here, a single feature means features that can be extracted from one security event.

For example, detection time, source IP, source port, destination IP, destination port, protocol, security event name, security event type, number of attacks, attack direction, packet size, automatic analysis result, dynamic analysis result, agency number, jumbo Whether or not the payload is present, the payload, and the payload to which the word2vec conversion method is applied may belong to a single feature.

For reference, the payload conversion method through Word2Vec is a method of converting a word into a vector, and is a method of determining a vector of a corresponding word through a relationship between neighboring words. General sentences can distinguish words based on spacing, but payload is very difficult to classify into semantic units and contains a large amount of special characters, so pre-processing is required to apply word2vec.

In the present invention, as a pre-processing for applying word2vec, the following 4 steps can be performed.

1) Converts hexadecimal-encoded strings to ASCII strings (except for ASCII code values (32 to 127), converts to blanks)

2) Processing the url-encoded part (%25 -> ‘%’, %26 -> ‘&’, %2A -> ‘*’ ...)

3) Replace special symbols except '@', '\', '-', ':', '%', '_', '.', '!', '/', '`' with spaces Replace all uppercase letters with lowercase letters

4) Excluding words composed of one letter, word2vec algorithm is applied

On the other hand, a complex feature means a single feature that can be extracted by using aggregation between several security events and statistical techniques.

For example, security event groups are formed based on period or number of cases, and one feature (eg, result value of operation) that can be extracted through calculation within the group (eg, aggregation, statistical technique, etc.) is a composite feature. can belong to

For example, it is assumed that a security event group as shown in Table 1 is formed based on the period (8.22 to 9.3).

One feature (e.g. 4) that can be extracted through calculation within the security event group (e.g. source IP, destination IP, number of security events with security event names 100.100.100.100/111.111.111.11/AAA) may belong to the feature.

Accordingly, the feature extraction module 120 may extract preset feature information (single feature and/or composite feature) from the security event collected by the data collection module 110 .

The normalization module 130 performs preset normalization on extracted feature information of security events.

Normalization refers to a process of adjusting the range of values of extracted features to be constant. If field A has a range of 50 to 100 and field B has a range of 0 to 100, the meaning is different because the same 50 is a value measured by different scales. Therefore, a process of adjusting the values of different fields to a common scale to have a certain meaning is required, and this is called normalization.

The normalization module 130 performs normalization of the extracted feature information of the security event to have a certain meaning by adjusting values of different fields to a common scale according to a preset normalization method.

In this case, the preset normalization method means a normalization method previously set by the user.

In the artificial intelligence model platform 100 of the present invention, the following three normalization methods are provided so that the user can set them in advance.

Equation 1 means the feature scaling [a,b] normalization method, Equation 2 means the Mean normalization [-1,1] normalization method, and Equation 3 means the standard score normalization method.

The normalization module 130 normalizes the extracted feature information of the security event according to a normalization method preset by a user among the three normalization methods described above.

The data output module 140 extracts learning data or test data from the security event for which specific information normalization has been completed under a given condition, that is, a condition previously set (given) by a user.

Specifically, the data output module 140 outputs the security event for which specific information has been normalized to a screen or a file according to a user's desired value, sequence, format, training/test data ratio, file division method, and the like.

The output learning data or test data is managed through a database or file storage by date and user so that it can be used immediately when creating an artificial intelligence model.

The model generation module 150 applies an artificial intelligence algorithm to the learning data managed in the output/file storage in the data output module 140 to generate an artificial intelligence model for security control.

That is, the model generation module 150 may generate an artificial intelligence model for security control, for example, an artificial intelligence model of a function required by a user, by applying an artificial intelligence algorithm to learning data.

For example, the model generation module 150 may generate an artificial intelligence detection model for detecting whether a security event is malicious according to a user request, and an artificial intelligence classification model for classifying true positives/false positives of a security event. can also create

Specifically, the model generation module 150, based on the learning data managed in the output / file storage in the data output module 140, an artificial intelligence algorithm, for example, a machine learning (eg, Deep Learning) algorithm previously selected by the user. Accordingly, an artificial intelligence model for security control can be created.

For example, the model generation module 150 uses a learning loss function (Loss function) representing the deviation between the result value predicted through the model and the actual result value in the backward propagation (error backpropagation) calculation-based machine learning technology. Thus, it is possible to generate an artificial intelligence model in which the deviation of the learning loss function becomes 0 based on the learning data.

As described above, according to the artificial intelligence model platform 100 of the present invention, by providing a platform environment that enables the creation of an artificial intelligence model for security control based on UI without separate programming, security control technology Even ordinary users who are not familiar with it can create artificial intelligence models suitable for their own purposes and requirements for security control.

Furthermore, in the artificial intelligence model platform 100 of the present invention, the performance management module 160 utilizes the test data managed in the output/file storage in the data output module 140 to generate the artificial intelligence model generated above. Test accuracy.

The performance management module 160 is for managing the artificial intelligence model generated by the model generation module 150, and includes 'who', 'when', 'what data', 'what field', 'what sampling method', 'what normalization method' Record and manage performance information such as 'what model' was used to create the AI model and how much performance (correct answer rate) the generated AI model has in the system (file storage).

Also, the performance management module 160 compares conditions for model generation and performance at a glance based on the performance information management, so that a correlation between the conditions and performance can be easily grasped.

In the present invention, in that a platform environment is provided so that even general users who are not familiar with security control technology can create an artificial intelligence model, the accuracy (performance) test of the artificial intelligence model created in the platform environment of the present invention may be essential.

Specifically, the performance management module 160 utilizes the test data managed in the output/file storage in the data output module 140 (security events that know the actual result values of true positive/false positive classification and malicious detection), Test the accuracy of the artificial intelligence model created above.

For example, the performance management module 160 tests the artificial intelligence model generated above using the test data, and calculates the accuracy of the model (performance ), that is, it can be output as a test result.

To create an artificial intelligence model, which features are used and which regularization method is applied have a great influence on model performance (accuracy).

However, it will be difficult for general users, especially those who are not familiar with security control technology, to combine/set feature information that can produce optimal performance in creating the artificial intelligence model they want.

Therefore, in the present invention, the feature extraction module 120, based on the accuracy test result of the performance management module 160, recommends changes to feature information to increase the accuracy of the artificial intelligence model generated above. can

It will also be difficult for ordinary users, especially those who are not familiar with security control technology, to know and set a normalization method that can produce optimal performance in creating the artificial intelligence model they want.

In addition, in the present invention, the normalization module 130 may recommend a normalization method change for normalization to increase the accuracy of the artificial intelligence model.

Hereinafter, with reference to FIG. 3, a technique for recommending a feature change to increase the accuracy of an artificial intelligence model, and a feature information recommending device that realizes the technique, will be described in detail.

3 illustrates the configuration of an apparatus for recommending feature information according to an embodiment of the present invention.

As shown in FIG. 3 , the apparatus 200 for recommending feature information according to the present invention includes a model performance checking unit 210 , a combination performance checking unit 220 , and a recommendation unit 230 .

All or at least part of the configuration of the feature information recommendation device 200 may be implemented in the form of a hardware module or a software module, or may be implemented in a combination of a hardware module and a software module.

Here, a software module may be understood as, for example, a command executed by a processor that controls an operation within the feature information recommendation device 200, and such a command may have a form loaded in a memory within the feature information recommendation device 200. will be able to have

As a result, the feature information recommendation device 200 according to an embodiment of the present invention, through the above configuration, the technology proposed in the present invention, that is, the technology of recommending a change in feature information to increase the accuracy of the artificial intelligence model. Hereinafter, each component in the feature information recommendation device 200 for realizing this will be described in more detail.

The model performance checking unit 210 checks the model performance of the artificial intelligence model generated based on learning of preset feature information among all feature information that can be set when the artificial intelligence model is created.

That is, the model performance checking unit 210 checks the performance (accuracy) of the artificial intelligence model generated based on learning feature information set by the user.

For detailed description, hereinafter, it will be described assuming an artificial intelligence model that has learned/generated feature information (hereinafter, user-set feature information) set by a user in the artificial intelligence model platform 100 of the present invention.

As described above, the model performance check unit 210 checks the model performance of the artificial intelligence model generated by learning the user-set feature information in the artificial intelligence model platform 100.

For example, the model performance confirmation unit 210, for the artificial intelligence model, the test data output from the artificial intelligence model platform 100 (particularly, the data output module 140) of the present invention (true positive/false positive classification and malicious status) Using the security event for which the actual result of the detection is known), the model performance (accuracy) can be tested/confirmed.

Accordingly, the model performance confirmation unit 210 tests the artificial intelligence model using the test data for the artificial intelligence model generated by the artificial intelligence model platform 100 (particularly, the data output module 140) of the present invention, The matching ratio between the result value predicted by the model and the actual result value known can be output as the accuracy (performance) of the model, that is, the test result.

The combination performance checking unit 220 sets a plurality of feature information combinations from all feature information and checks the performance of the artificial intelligence model generated based on learning for each feature information combination.

Specifically, the combination performance confirmation unit 220 sets various feature information combinations in addition to the user-set feature information learned when the artificial intelligence model is created from the entire feature information that can be set when the artificial intelligence model is created, and sets each of a plurality of feature information combinations. You can check the performance of the artificial intelligence model created based on learning.

The recommendation unit 230, among the performance of a plurality of feature information combinations confirmed by the combination performance confirmation unit 220, the model performance confirmed by the model performance confirmation unit 210, that is, the performance of the artificial intelligence model generated based on the current user setting A specific feature information combination with higher performance may be recommended.

Hereinafter, specific embodiments for recommending a specific feature information combination will be described.

According to an embodiment, the combination of a plurality of feature information set by the combination performance check unit 220 includes the user-set feature information learned at the time of generating the artificial intelligence model this time, and the remaining specific feature information other than the user-set feature information from all feature information. It may be a combination of sequentially adding at least one of the pieces of information.

In the following, among all feature information (eg a,b,c...,z (n=26)), the learned user-set feature information (eg a,b,c,d,e , f (k = 6)) will be explained. And in this case, it is assumed that the artificial intelligence model performance (mk) checked by the model performance confirmation unit 210 is 85%.

Accordingly, the combination performance check unit 220 assigns the user-set feature information (a, b, c, d, e, f) to the user-set feature information (a, b, c, d, e) among the entire feature information (n). , It is possible to set a plurality of characteristic information combinations by sequentially adding at least one of the remaining specific information except for f).

For example, the combination performance check unit 220, in the user-set feature information (a, b, c, d, e, f) set by the user, among the entire feature information (n), the user-set feature information (a, b, c, d, e, f), 1 to (n-k) feature information among the remaining specific information can be sequentially added to set a plurality of feature information combinations as follows.

a,b,c,d,e,f,g -> m(k+1)1 -> 82%

a,b,c,d,e,f,h -> m(k+1)2 -> 80%

...

a,b,c,d,e,f,g,h,i -> m(k+3)1 -> 88%

...

a,b,c,d,e,f,...,z -> m(n) -> 85%

And, the combination performance check unit 220, as described above, performs 82%, 80%, ... 88%, ... 85% of the performance of the artificial intelligence model generated based on learning for each combination of feature information. You can check.

In this case, the recommendation unit 230 selects the top N (e.g., 4 ) can be selected/recommended as a specific feature information combination.

Of course, the top N is a number that can be designated/changed by the system administrator or user.

For another example, the combination performance check unit 220 may include the user-set feature information (a, b, c, d, e, f) set by the user among the entire feature information (n), the user-set feature information (a). , b, c, d, e, f), the remaining specific information can be sequentially added one by one to set a plurality of characteristic information combinations as follows.

a,b,c,d,e,f,g -> m(k+1)1 -> 82%

a,b,c,d,e,f,h -> m(k+1)2 -> 80%

...

a,b,c,d,e,f,z -> m(k+1)ζ+1 -> 90%

In addition, the combination performance check unit 220 can check the performance of the artificial intelligence model generated based on learning for each feature information combination, 82%, 80%, ... 90%, as described above.

In this case, the recommendation unit 230 selects the top N (e.g., 3 ) can be selected/recommended as a specific feature information combination.

Of course, the top N is a number that can be designated/changed by the system administrator or user.

On the other hand, according to another embodiment, the combination performance check unit 220 includes the user-set feature from the entire feature information (n) to preset feature information, that is, user-set feature information (a, b, c, d, e, f). A combination setting process of setting a plurality of characteristic information combinations may be performed by sequentially adding one of the remaining specific information excluding information (a, b, c, d, e, and f).

In this case, the combination performance checking unit 220 can check the following performance for each feature information combination, 82%, 80%, ... 90%, as in the above.

a,b,c,d,e,f,g -> m(k+1)1 -> 82%

a,b,c,d,e,f,h -> m(k+1)2 -> 80%

...

a,b,c,d,e,f,z -> m(k+1)ζ+1 -> 90%

The combination performance confirmation unit 220 resets each of the feature information combinations having a higher performance than the performance (mk = 85%) of the artificial intelligence model generated based on the current user setting among a plurality of feature information combinations as feature information In this case, it is possible to perform a reset process in which the combination setting process is repeatedly performed for each reset feature information.

That is, the combination performance check unit 220 deletes a feature information combination having a performance lower than or equal to the performance of the artificial intelligence model (mk = 85%) among a plurality of feature information combinations, and the performance of the artificial intelligence model (mk = 85%) %) A reset process in which only feature information combinations with higher performance are left as follows, and each of them is reset as feature information so that the combination setting process is repeatedly performed for each reset feature information as shown in Table 2 below. can be done

a,b,c,d,e,f,l -> m(k+1)1 -> 87%

a,b,c,d,e,f,m -> m(k+1)2 -> 86%

a,b,c,d,e,f,n -> m(k+1)3 -> 86%

The combination performance confirmation unit 220 repeats the above-described combination setting process and resetting process, and performs higher than the performance (mk = 85%) of the artificial intelligence model generated based on the user setting this time among a plurality of feature information combinations. If the feature information combination does not exist, a process of selecting the previous feature information as a specific feature information combination and transmitting it to the recommendation unit 230 is performed.

In this case, the recommendation unit 230 selects the feature information transmitted from the combination performance checker 220 among the performance of a plurality of feature information combinations, and the performance of the artificial intelligence model generated based on the current user setting (mk = 85 %) can be recommended as a specific feature information combination with higher performance.

As described above, according to the present invention, an optimal feature having optimal performance (accuracy) is recommended to a user who creates an artificial intelligence model for security control based on a UI in an environment provided by the artificial intelligence model platform 100. / By enabling application, even general users who are not familiar with security control technology can create an optimal artificial intelligence model for security control.

Hereinafter, with reference to FIG. 4, a technique for recommending a normalization method change to increase the accuracy of an artificial intelligence model, and a normalization method recommendation device for realizing the technology will be described in detail.

4 illustrates the configuration of a normalization method recommendation apparatus according to an embodiment of the present invention.

As shown in FIG. 4 , the normalization method recommendation apparatus 300 of the present invention includes an attribute confirmation unit 310 , a determination unit 320 , and a recommendation unit 330 .

All or at least part of the configuration of the normalization method recommendation device 300 may be implemented in the form of a hardware module, a software module, or a combination of a hardware module and a software module.

Here, the software module may be understood as, for example, a command executed by a processor that controls an operation within the normalization method recommendation device 300, and such a command may have a form loaded in a memory within the normalization method recommendation device 300. will be able to have

As a result, the normalization method recommendation apparatus 300 according to an embodiment of the present invention realizes the technique proposed in the present invention, that is, a technique of recommending a change in normalization method to increase the accuracy of an artificial intelligence model, through the above configuration, Hereinafter, each component in the normalization method recommendation apparatus 300 for realizing this will be described in more detail.

The property confirmation unit 310 checks the properties of the feature information used for learning when the artificial intelligence model is created.

Here, the feature information used for learning when the artificial intelligence model is created may be feature information directly set by the user based on the UI among all feature information that can be set when the artificial intelligence model is created, or recommended specific information among all feature information. It may be feature information to which a feature information combination is applied/set.

In addition, properties of feature information can be largely divided into numerical properties and category properties.

That is, the attribute check unit 310 can check whether the attribute of the feature information (directly set or applied as a recommendation) used for learning when creating the artificial intelligence model is a number attribute, a category attribute, or a number and category combination attribute. .

The determination unit 320 determines a normalization method according to the property of the characteristic information confirmed by the property checking unit 310 among all normalization methods that can be set.

Specifically, before determining the normalization method according to the attribute of the feature information, the determination unit 320 determines whether the same normalization method is applied to all fields of the current feature information or whether the normalization method for each field is applied in all fields of the current feature information. You can first distinguish whether it applies.

The determination unit 320 may determine that the same normalization method is applied to all fields of the current feature information when only numeric and/or category data exists in all fields of the current feature information (including a single feature case).

In this case, the determining unit 320 determines the first normalization method according to the entire number pattern of the feature information when the attribute of the feature information is a numeric attribute, and if the attribute of the feature information is a category attribute, the entirety of the feature information A second normalization method is determined in which a non-zero characteristic value is expressed only in a position designated for each category of feature information in a vector defined by the number of categories, and when the attribute of feature information is a number and category combination attribute, the second normalization method is determined. A normalization method and a first normalization method may be determined.

Specifically, the first normalization method includes a standard score normalization method, a mean normalization normalization method, and a feature scaling normalization method according to predefined priorities (see Equations 1, 2, and 3).

The determination unit 320 determines that the attribute of the feature information is a numeric attribute when only numeric data exists in the entire field of feature information, and in this case, determines the first normalization method according to the entire number pattern of feature information.

At this time, the determination unit 320 determines the standard score normalization method, the mean normalization normalization method, and the feature scaling normalization method in order according to the priority among the first normalization methods, and standard deviation and normalization for all numerical patterns of feature information. Based on whether upper/lower limits of the scaling range exist, a normalization method having the highest applicable priority among the first normalization methods may be determined.

In addition, the determination unit 320 classifies the property of the characteristic information as a category property when only category data exists in all fields of the characteristic information, and in this case, the characteristic information in a vector defined by the total number of categories of the characteristic information. It is possible to determine a second normalization scheme that expresses a non-zero characteristic value only at positions designated for each category of .

In order to generate an artificial intelligence model by applying an artificial intelligence algorithm (e.g., machine learning) to learning data, it is necessary to convert data into numerical data that machines can understand. In the present invention, this conversion method (second normalization) method) can adopt One Hot Encoding.

Accordingly, when the attribute of the feature information is a category attribute, the determiner 320 sets a non-zero feature value (e.g., 1 ), it is possible to determine the second normalization method_One Hot Encoding.

To briefly explain the second normalization method_One Hot Encoding, it is assumed that feature information has a category attribute of fruit, and apples, pears, and persimmons (expressed as a 3D vector since there are three types of fruit) are the total number of categories.

At this time, each characteristic information having apples, pears, and persimmons as data can be expressed as follows according to the second normalization method_One Hot Encoding.

apple = {1, 0, 0}

times = {0, 1, 0}

Persimmon = {0, 0, 1}

In addition, the determination unit 320, when number and category data exist in all fields of the feature information, classifies the attribute of the feature information as a number and category combination attribute, and in this case, the second normalization method and the first normalization method described above. can determine

That is, when the property of the feature information is a number and category combination attribute, the determination unit 320 first applies the second normalization method_One Hot Encoding to the data of the category attribute in the feature information, and then applies the entirety of the feature information. A second normalization method and a first normalization method may be determined in order to determine a normalization method having the highest priority among the first normalization methods based on the standard deviation of the number pattern and whether there is an upper/lower limit of the normalization scaling range. .

On the other hand, the determination unit 320 determines that, when the feature information is a composite feature (a feature that can be extracted using an aggregation of several security events and statistical techniques), the normalization method is applied for each field in the entire feature information field this time. can be distinguished.

In this case, the determination unit 320 may determine a normalization method having the highest priority among the mean normalization normalization method and the feature scaling normalization method for a field of attribute type in feature information.

In addition, the determiner 320 may determine a normalization method having the highest priority among mean normalization normalization methods and feature scaling normalization methods for a field of the number of attributes in feature information.

In addition, the determination unit 320 may decide to exclude a normalization method for a field of a ratio attribute in feature information without determining a normalization method, or may determine a standard score normalization method.

In addition, the determination unit 320 may determine whether a normalization method is not determined for a field of whether an attribute exists (eg, presence/absence of an operation result value) in feature information and excludes it from the normalization target.

The recommendation unit 330 recommends the normalization method determined by the determination unit 320 .

As described above, according to the present invention, an optimal normalization method having optimal performance (accuracy) is recommended/applied to a user who creates an artificial intelligence model for security control based on a UI in an environment provided by the artificial intelligence model platform 100. By enabling this, even general users who are not familiar with security control technology can create an optimal artificial intelligence model for security control.

As described above, according to the present invention, an artificial intelligence model platform capable of generating an artificial intelligence model for security control is implemented, but in particular, feature information directly related to artificial intelligence model performance and normalization method are optimally recommended/recommended. By making it applicable, it is possible to implement an artificial intelligence model platform that allows even general users who are not familiar with security control technology to create an optimal artificial intelligence model for security control.

For this reason, according to the present invention, since the optimal artificial intelligence model suitable for the purpose and requirements for security control can be created and applied flexibly and variously, it is possible to maximize the quality improvement of the security control service, and at the same time, large-scale It can even be expected to support the establishment of an artificial intelligence-based breach response system to efficiently analyze signs of cyberattacks and abnormal behavior.

Hereinafter, embodiments that can further enhance the performance of an artificial intelligence algorithm by considering the characteristics of data will be disclosed.

Disclosed is an embodiment for utilizing imbalanced data collected in real environments, such as security data, in machine learning and artificial intelligence techniques without separate processing and classification procedures.

According to the embodiment, an artificial intelligence algorithm capable of learning in an optimal way may be automatically selected according to the label distribution of retained data, or an optimal algorithm may be applied in consideration of a user's purpose.

5 discloses an embodiment of performing an artificial intelligence algorithm according to a label distribution of imbalanced data.

This embodiment includes a database 510, a label (class) distribution processor 520, and a learning processor 530.

The database 510 stores user data, and may store security-related data such as security events or security log data.

The label (class) distribution processing unit 520 classifies data stored in the database 510 according to the label distribution or detects the classified data, and processes the data according to the classification or detected label distribution.

The label (class) distribution processing unit 520 may include a plurality of label processing distribution processing units according to the label distribution.

In an embodiment, the label (class) distribution processing unit 520 includes a label (class) distribution detection unit 521, a first label distribution processing unit 523, a second label distribution processing unit 525, and a third label distribution processing unit 527. Including examples are disclosed.

The label (class) distribution detection unit 521 may classify data stored in the database 510 according to the label distribution of the data or detect the classified data.

For convenience of description, the following example illustrates a single label distribution, an unbalanced label distribution, or a balanced label distribution according to the label (class) distribution ratio.

In this case, the first label distribution processing unit 523 processes single label distribution data detected by the label (class) distribution detection unit 521 . An example of processing single label distribution data is detailed below.

The second label distribution processing unit 525 processes the imbalanced label distribution data detected by the label (class) distribution detection unit 521 . An example of processing unbalanced label distribution data is detailed below.

The third label distribution processing unit 527 processes balanced label distribution data detected by the label (class) distribution detection unit 521 . An example of processing balanced label distribution data is detailed below.

Here, an example of dividing the label distribution into three cases (single, unbalanced, and balanced) has been disclosed, but the label (class) distribution can be classified in more detail according to the user's purpose. In that case, the label (class) distribution processing unit 520 may include a plurality of label processing distribution processing units other than those illustrated.

For example, if the user classifies the label distribution ratio in more detail, the ratio of data labels included in single label, unbalanced label, and balanced label may be different. In that case, for example, an imbalanced label may also include a plurality of imbalanced labels according to a ratio of data labels, such as a first imbalanced label, a second imbalanced label, a third imbalanced label, and the like.

Here, the single label means a case in which it can be determined that the input data set consists of almost a single class because the distribution ratio of the first class and the second class, which are binary labels of data included in the input data set, is one-sidedly large. The distribution ratio between the first class and the second class means a case where the distribution ratio is maximum:minimum or conversely minimum:maximum. The ratio of the two classes can be defined by the user.

Also, here, the balanced label refers to a case where the distribution ratios of the first class and the second class, which are binary labels of data included in the input data set, are similar, and the ratio of the two classes may be defined by a user.

In addition, an imbalanced label means a case that is neither a single label ratio nor a balanced label ratio, and similarly, the ratio of two classes of data included in the input data set may follow the user's definition.

Classification and classification ratios of a plurality of label distributions will be described in detail in the following embodiments.

The learning processing unit 530 selects each artificial intelligence learning algorithm according to the data classifications processed by the label (class) distribution processing unit 520 according to the label distribution, and learns data of the corresponding classification with the selected algorithm.

Among the embodiments of this drawing, the database 510 may correspond to the data collection module 110 of FIG. 2 . Also, the label distribution processing unit 520 may correspond to the feature extraction module 120 of FIG. 2 , and the learning processing unit 530 may correspond to the model generation module 150 of FIG. 2 .

6 discloses an example of a label (class) distribution processing unit according to an embodiment.

In this example, it is assumed that the label or class distribution of data is a binary class.

And in this example, if the binary label distribution of the data is greater than or equal to 9:1, it is a single label distribution, if the binary label distribution is greater than or equal to 6:4 and less than or equal to 9:1, it is an unbalanced label distribution, and if the binary label distribution is greater than or equal to 5:5 and 6:4 or less is assumed to be a balanced label distribution.

In this example, according to the distribution of each label, the first label distribution processing unit 523 is referred to as a single label distribution unit processing single label distribution data, and the second label distribution processing unit 525 is called an unbalanced label distribution processing unit processing unbalanced label distribution data. It is called the label distribution unit. Also, according to the label distribution, the third label distribution processing unit 527 is called a balanced label distribution unit that processes balanced label distribution data.

The label (class) distribution detector 521 may detect a label distribution of stored or input data according to a label distribution criterion. Each detected data is output to the first label distribution processing unit 523, the second label distribution processing unit 525, and the third label distribution processing unit 527 according to the label distribution. '

If the label (class) distribution detection unit 521 has one label distribution in the input data set (the case where the ratio of the binary class data distribution is 9:1 or more is exemplified as a single label), this label distribution is detected and detected. The input data is output to the first label distribution processing unit 523 according to the label distribution.

For example, in the binary class, the amount of data of class 1 is extremely large and the amount of data of class 2 is extremely small, or conversely, the amount of data of class 1 is extremely small and the amount of data of class 2 is extremely large. .

If the label (class) distribution detection unit 521 has an unbalanced label distribution (binary class data distribution ratio of 6:4 or more and 9:1 or less), the label (class) distribution detection unit 521 detects this label distribution, Input data is output to the second label distribution processing unit 525 according to the detected label distribution.

For example, this case may correspond to a binary class in which the amount of data of class 1 is large, the amount of data of class 2 is small, or, conversely, the amount of data of class 1 is small and the amount of data of class 2 is large.

If the label (class) distribution detection unit 521 has a balanced label distribution for the input data set (the case where the ratio of the binary class data distribution is 6:4 or less and 5:5 or more is exemplified as a balanced label), this label The distribution is detected and the input data is output to the third label distribution processor 527 according to the detected label distribution.

For example, when detecting security data, a binary class distribution of true positives and false positives can be obtained. In the case of such security data detection, since the ratio of true positives and false positives has a very large difference, it can be determined that the data class has a single label distribution.

Accordingly, in this case, the label (class) distribution detection unit 521 may output the input data set to the first label distribution processing unit 523 according to the corresponding data distribution. In addition, an artificial intelligence algorithm may be selected and applied to the data processed by the first label distribution processor 523 in consideration of data distribution.

For example, the first label distribution processor 523 may review and process the label distribution of the received data set. For example, the first label distribution processing unit 523 may perform detection using data similarity or anomaly detection between data for a single label distribution data set.

7 discloses an example of selectively applying an artificial intelligence algorithm using data processed according to the distribution of input data.

In this example as well, according to the distribution of each label, the first label distribution processing unit 523 is referred to as a single label distribution unit processing single label distribution data, and the second label distribution processing unit 525 processes unbalanced label distribution data. The third label distribution processing unit 527 is called a balanced label distribution unit that processes balanced label distribution data.

Depending on the embodiment, an artificial intelligence algorithm for each label distribution group may be selected differently, and data belonging to the distribution group may be learned using the artificial intelligence algorithm selected for each label distribution.

In this example, the first label distribution processing unit 523 verifies and processes the data distribution for data of a single label distribution, and the learning processing unit 530 applies an artificial intelligence algorithm suitable for the processed data.

In this case, the artificial intelligence algorithm applied by the learning processing unit 530 applies an unsupervised learning artificial intelligence algorithm rather than supervised learning. For example, when the label distribution is a single label distribution, the learning processing unit 530 classifies data into groups based on similarity, such as a clustering algorithm, and performs data mining in consideration of characteristics of the classified clusters. carry out The learning processing unit 530 performs machine learning in consideration of the characteristics of the entire data using the representative value of the cluster according to the cluster of the classified data.

The learning processing unit 530 may apply an algorithm of the same method as an unsupervised learning generative adversarial network (GAN). In this case, the learning processing unit 530 makes the artificial neural network create the distribution by estimating the probability distribution of the data in the manner exemplified in the normalization module of FIG. 2 or FIG. 4 above.

In this case, when new data is introduced, the label (class) distribution detection unit 520 compares the similarity between the new data and the previously processed data according to the generated probability distribution and substitutes the new data into the artificial intelligence model created by learning. determine the label of

The third label distribution processing unit 527 verifies and processes the data distribution for the balanced label distribution data, and the learning processing unit 530 applies an artificial intelligence algorithm suitable for the balanced label distribution to the processed data. In this case, the learning processing unit 530 may apply supervised learning-based artificial intelligence algorithms (Deep Neural Network (DNN), Convolution Neural Network (CNN), Recurrent Neural Network (RNN), Binarized Support Vector Machine (BSVM), etc.) .

The second label distribution processor 525 processes data of an imbalanced label distribution. The learning processing unit 530 may perform an artificial intelligence algorithm on the imbalanced label distribution data processed by the second label distribution processing unit 525 .

In this case, as another example, the second label distribution processor 525 performs over- or under-sampling of the data set having an imbalanced label distribution. As a result, the over- or under-sampled data set through the second label distribution processor 526 may become a single label distribution or a balanced label distribution.

The second label distribution processor 525 processes the unbalanced label distribution data set and changes it to a single label distribution and transmits the changed data to the first label distribution processor 523, or changes it to a balanced label distribution and converts the changed data It can be transmitted to the third label distribution processor 527.

The learning processing unit 530 may apply suitable artificial intelligence algorithms to the data processed by the first, second, and third label distribution processing units 523, 525, and 527, respectively.

8 discloses an example of a method of performing an artificial intelligence algorithm that selectively applies an artificial intelligence algorithm using data processed according to data distribution.

The distribution of binary class data is detected from the input data (S610). An example of detecting the binary class of data is described above in FIGS. 5 and 6 . For example, the data distribution of the binary class can be divided according to single label distribution, unbalanced label distribution, and balanced label distribution. Since an example of detecting a binary class may vary according to a distribution ratio of data labels, there may be other label distributions other than the exemplified label distribution when the distribution ratio is changed. In addition, after dividing the data distribution into single label distribution, unbalanced label distribution, and balanced label distribution, each of the single label distribution, unbalanced label distribution, and balanced label distribution can be further subdivided according to the ratio of the label distributions to be divided.

Data included in the detected label or class is processed according to the distribution of the detected label or class (S620). A method of processing data included in a class has been described in detail with reference to FIGS. 6 and 7 .

For example, if the ratio of the data label distribution corresponds to an unbalanced label distribution, the data can be changed to a single label or a balanced label through overfitting or underfitting in consideration of the user's purpose. .

An artificial intelligence algorithm is selected for each processed data according to the label distribution, and the processed data is learned using the selected artificial intelligence algorithm (S630).

Examples of selecting an artificial intelligence algorithm for data processed according to label distribution are described in FIGS. 6 and 7 .

It can also be subdivided by users according to label distribution. The embodiment discloses an example in which each label is classified into a single label, an imbalanced label, and a balanced label. If the label distribution is different, the AI algorithm selected may also be selected differently according to the label distribution.

Even if the class distribution of data is a single label, unsupervised learning artificial intelligence algorithms can be applied differently by internally subdividing the label distribution ratios of a data set classified as a single label.

In addition, even if the class distribution of the data is a balanced label, the label distribution ratios of the data set classified as balanced labels can be subdivided and artificial intelligence algorithms such as DNN, CNN, RNN, and BSVM can be selected and applied according to the subdivided label distribution ratio. .

According to this embodiment, when applying artificial intelligence and machine learning methods to data, even if the data distribution of the class greatly affects the results or causes the learning or performance degradation of the artificial intelligence model, by applying different artificial intelligence algorithms according to the data distribution It enables the creation of high-performance artificial intelligence models.

In addition, it is possible to lower entry barriers to the use of artificial intelligence and maximize productivity through automatic algorithm recommendation according to the purpose of using artificial intelligence algorithms or data distribution of classes.

100: AI model platform
110: data collection module
120: feature extraction module
130: normalization module
140: data output module
150: model generation module
160: performance management module
170: UI module
210: model performance confirmation unit
220: combination performance confirmation unit
230: recommendation unit
310: property confirmation unit
320: decision unit
330: recommendation unit
510: database
520: label distribution processing unit
521: label distribution detection unit
523, 525, 527: first, second, and third label distribution processors
530: learning processing unit

Claims (13)

a class distribution detector detecting a binary label distribution of binary label data;
a label distribution processing unit that classifies the data according to the detected binary label distribution and processes the classified data according to the binary label distribution; and
A learning processing unit learning each of the data processed by the label distribution processing unit using artificial intelligence algorithms selected according to the label distribution of the data processed by the label distribution processing unit;
The label distribution processing unit,
Check the performance of artificial intelligence models generated for each combination of feature information for the binary label data;
Recommending any one of the feature information combinations based on the performance of the artificial intelligence models,
An artificial intelligence algorithm execution device. According to claim 1,
The label distribution processing unit,
An artificial intelligence algorithm execution device for processing the classified data according to whether the label distribution of the classified data is any one of a single label distribution, an unbalanced label distribution, and a balanced label distribution. According to claim 1,
The label distribution processing unit,
If the label distribution of the classified data is an unbalanced label distribution, the label distribution of the classified data is obtained by over-fitting or under-fitting the classified data to data of a single label distribution or balanced label distribution. A device that performs an artificial intelligence algorithm that changes to a set. According to claim 1,
The learning processing unit,
An artificial intelligence algorithm performing device for applying an unsupervised learning artificial intelligence algorithm to the classified data when the label distribution of the classified data is a single label distribution. According to claim 1,
The learning processing unit,
If the label distribution of the classified data is a balanced label distribution, any artificial intelligence algorithm of Deep Neural Network (DNN), Convolution Neural Network (CNN), Recurrent Neural Network (RNN), or Binarized Support Vector Machine (BSVM) An artificial intelligence algorithm implementation device that applies. In the method of performing an artificial intelligence algorithm in which each step is performed by a computing device,
detecting a binary label distribution of the data from the input data;
classifying the data according to the detected binary label distribution and processing the classified data according to the binary label distribution; and
Learning each of the processed data by applying artificial intelligence algorithms selected according to the label distribution;
The processing step is
Check the performance of artificial intelligence models generated for each combination of feature information for binary label data,
Recommending any one of the feature information combinations based on the performance of the artificial intelligence models,
How to perform artificial intelligence algorithms. According to claim 6,
The step of processing according to the binary label distribution,
The method of performing the artificial intelligence algorithm for processing the classified data according to whether the label distribution of the classified data is any one of a single label distribution, an imbalanced label distribution, and a balanced label distribution. According to claim 6,
The step of processing according to the binary label distribution,
When the label distribution of the classified data is an unbalanced label distribution, the label distribution of the classified data is obtained by overfitting or underfitting the classified data to a single label distribution or a balanced label distribution. How to perform an artificial intelligence algorithm that transforms into a data set. According to claim 6,
The learning step is
An artificial intelligence algorithm performing method of applying an unsupervised learning artificial intelligence algorithm to the classified data when the label distribution of the classified data is a single label distribution. According to claim 6,
The learning step is
If the label distribution of the classified data is a balanced label distribution, any artificial intelligence algorithm of Deep Neural Network (DNN), Convolution Neural Network (CNN), Recurrent Neural Network (RNN), or Binarized Support Vector Machine (BSVM) How to perform an artificial intelligence algorithm that applies . detecting a binary label distribution of the data from the input data; classifying the data according to the detected binary label distribution and processing the classified data according to the binary label distribution; and applying artificial intelligence algorithms selected according to the label distribution of the classified data to learn each of the processed data;
The processing process is
Check the performance of artificial intelligence models generated for each combination of feature information for binary label data,
Recommending any one of the feature information combinations based on the performance of the artificial intelligence models,
A computer-readable storage medium that stores software that performs artificial intelligence algorithms. According to claim 11,
The process of learning each of the processed data,
A computer-readable storage medium storing software for performing an artificial intelligence algorithm for applying an unsupervised learning artificial intelligence algorithm to the classified data when the label distribution of the classified data is a single label distribution. According to claim 11,
The process of learning each of the processed data,
If the label distribution of the classified data is a balanced label distribution, any artificial intelligence algorithm of Deep Neural Network (DNN), Convolution Neural Network (CNN), Recurrent Neural Network (RNN), or Binarized Support Vector Machine (BSVM) A computer-readable storage medium that stores software that performs artificial intelligence algorithms that apply

Images