KR20220132804A - Apparatus and method of recommending sampling method and classification algorithm by using metadata set - Google Patents

Abstract

The present invention relates to technology which generates a metadata set using dataset features of an open dataset, and when a dataset to be used by a user is input, recommends an optimal algorithm type, a parameter value, and a data sampling method for the dataset to be used by the user by using a metadata set. An embodiment of the present invention provides a sampling method and classification algorithm recommendation device using a metadata set. The device comprises: a dataset collection unit which collects an open dataset from an open database; a feature extraction unit which extracts a plurality of dataset features of the collected open dataset and pre-processes the extracted plurality of dataset features; a mapping processing unit for mapping a sampling method and classification algorithm according to the features of the plurality of preprocessed datasets; a metadata set generation unit which generates a selection rule base for selecting a recommended sampling method and a recommended classification algorithm based on the mapped sampling method and the mapped classification algorithm, and generates a metadata set including the generated selection rule base and the preprocessed plurality of dataset features; and a recommendation unit for recommending at least one of a customized sampling method and a customized classification algorithm using the generated metadata set for a user dataset input from a user. The present invention can reduce a time and costs.

Description

Apparatus and method for recommending sampling method and classification algorithm using metadata set

The present invention relates to a technical idea for recommending a sampling method and a classification algorithm in relation to an artificial intelligence algorithm optimized using a metadata set. Specifically, a metadata set is generated using the dataset characteristics of an open dataset, and , relates to a technique for recommending an optimal algorithm type, parameter value, and data sampling method for a dataset to be used by the user using a metadata set when a data set to be used by the user is input.

Efforts are being made to improve customer service and the accuracy of decision-making within the company through machine learning algorithms in order to preoccupy the competitive advantage of companies in all industries.

Existing AI developers are recklessly consuming time and computer hardware resources by repeating experiments to develop high-performance algorithms.

In other words, it consumes a lot of iterative experiments and time and resources to select an optimized AI algorithm.

Also, although many AI algorithm researchers have published algorithms with excellent performance, performance differences may exist depending on the characteristics of the dataset.

Recently, efforts are being made to build a smart factory equipped with artificial intelligence in the manufacturing and production field, and it can be used for artificial intelligence consulting companies and smart factory construction that lack prior knowledge in the early stage of AI construction, so sampling related to artificial intelligence algorithms Selection of methods and classification algorithms is inevitably an important issue.

In addition, as a core component of a platform for developing solutions, systems, and services based on machine learning, automatic selection and recommendation of artificial intelligence algorithms is required, and the machine learning-based development market continues to grow significantly.

The reason why the performance of the algorithm differs depending on the dataset is that the developed algorithm has been developed so that it can achieve excellent performance on the dataset generated in a special situation.

The accuracy of the classification algorithm can be determined by the characteristics of the data set as well as the characteristics and hyperparameters of the algorithm.

Since classification algorithms used in artificial intelligence have different classification strategies, efficient algorithms exist according to the characteristics of a specific dataset, so it may be important to understand the characteristics of a dataset.

Research on the correlation between the characteristics of a dataset called meta-features and the performance of classification algorithms is still insufficient, and studies on meta-features that reflect the disproportionate characteristics of multi-classes have not been conducted. have.

Korean Patent Registration No. 10-2103902, "Component-based Machine Learning Automation Prediction Apparatus and Method" Korea Patent Registration No. 10-2098897, "Self-learning system based on machine learning knowledge and automated machine learning procedure" Korean Patent Registration No. 10-1864286, "Method and Apparatus Using Machine Learning Algorithm" US Patent Publication No. 2020/0210775, "DATA STITCHING AND HARMONIZATION FOR MACHINE LEARNING"

The present invention generates a metadata set using the dataset characteristics of an open dataset, and when a dataset to be used by a user is input, an optimal algorithm type and An object of the present invention is to provide an apparatus for recommending a sampling method and a classification algorithm using a metadata set that recommends a parameter value and a data sampling method.

The present invention aims to recommend an optimal algorithm type, parameter value, and data sampling method that shows the best performance based on a metadata set created by learning and metadataizing the dataset characteristics of an open dataset in advance. do.

An object of the present invention is to reduce repetitive experiments in data science and artificial intelligence development companies to reduce time and cost, and to support eco-friendly artificial intelligence development and data analysis by reducing the use of hardware.

In the present invention, when a user enters a field to which data science is not applied, the user may experience difficulties due to lack of prior knowledge. It aims to automatically find and recommend

The present invention provides a sampling method and classification related to artificial intelligence algorithms by referring to the method of automatically finding and recommending similar machine learning algorithms through pre-learned datasets when novice data scientists and AI developers lack data know-how. The purpose is to support the acquisition of know-how on the algorithm selection method.

The apparatus for recommending a sampling method and classification algorithm using a metadata set according to an embodiment of the present invention includes a dataset collection unit that collects an open dataset from an open database, and extracts characteristics of a plurality of datasets of the collected open dataset. and a feature extracting unit for pre-processing the plurality of extracted data set characteristics, a mapping processing unit for mapping a sampling method and a classification algorithm according to the pre-processed plurality of data set characteristics, and the mapped sampling method and the mapped classification algorithm. a metadata set generator for generating a selection rule base for selecting a recommended sampling method and a recommended classification algorithm based on and a recommender for recommending at least one of a custom sampling method and a custom classification algorithm by using the generated metadata set with respect to the user data set input from the user.

The feature extraction unit includes the number of variables, the number of instances, the number of classes, the degree of class bias, the entropy of the class, the degree of overlap of the variables in the collected open dataset, the silhouette score, the hub score, the entropy of the variable, and the The plurality of dataset features including linearity and neighborhood may be extracted, and the extracted plurality of dataset features may be pre-processed.

The feature extractor classifies the collected open dataset into a plurality of folds, and uses the dataset included in the remaining folds except for one of the classified plurality of folds into a plurality of training datasets. to be determined, and the plurality of dataset characteristics may be extracted from the determined plurality of training datasets.

When a missing value exists in the dataset from which the plurality of dataset characteristics are extracted, and the variable in which the missing value exists is a numeric type, the characteristic extractor processes the missing value by using the average value of the corresponding class, thereby processing the extracted plurality of data It is possible to preprocess three properties.

When a missing value exists in the dataset from which the plurality of dataset characteristics are extracted, and the variable in which the missing value exists is a nominal type, the feature extractor processes the missing value using the mode value of the corresponding class to process the plurality of extracted data It is possible to preprocess three properties.

When class imbalance exists in the dataset from which the plurality of dataset characteristics are extracted, the characteristic extraction unit removes a majority class according to the existing class imbalance. An under-sampling method and a minority class ( Minority class) according to the majority class (majority class) by using any one of the class imbalance resolution method of the over sampling method to resolve the existing class imbalance, the extracted plurality of dataset characteristics can be pre-processed.

The mapping processing unit applies the characteristics of the plurality of preprocessed datasets to a plurality of sampling methods, calculates sampling method accuracy in each of the plurality of applied sampling methods, and according to the calculated sampling method accuracy, the plurality of preprocessed data sets Data set characteristics and sampling methods can be mapped.

The mapping processing unit applies the characteristics of the plurality of preprocessed datasets to a plurality of classification algorithms, calculates classification algorithm accuracy in each of the plurality of applied classification algorithms, and calculates the accuracy of the plurality of preprocessed classification algorithms according to the calculated classification algorithm accuracy. You can map data set characteristics and classification algorithms.

The mapping processing unit may calculate the classification algorithm accuracy based on the characteristics and hyperparameters of the classification algorithm in each of the plurality of classification algorithms applied to the preprocessed characteristics of the plurality of datasets.

The metadata set generating unit filters the plurality of preprocessed dataset characteristics applied to the mapped sampling method and the mapped classification algorithm, and selects a plurality of datasets related to the filtered plurality of dataset characteristics as the recommended sampling method and machine learning by inputting the recommendation classification algorithm, and generating a selection rule base for selecting the recommendation sampling method and the recommendation classification algorithm based on the machine learning.

The metadata set generator may generate a metadata set including a plurality of datasets related to characteristics of the plurality of filtered datasets and the generated selection rule base.

According to an embodiment of the present invention, the apparatus for recommending a sampling method and classification algorithm using a metadata set may further include a metadata set storage unit for storing the generated metadata set.

The feature extractor includes the number of variables, the number of instances, the number of classes, the degree of bias in the class, the entropy of the class, the degree of overlap of the variables in the input user dataset, the silhouette score, the hub score, the entropy of the variable, and the It is possible to extract a plurality of data set features including linearity and neighborliness.

The feature extractor classifies the input user dataset into a plurality of folds, and uses the dataset included in the remaining folds except for one of the classified plurality of folds into a plurality of training datasets. to be determined, and the plurality of dataset characteristics may be extracted from the determined plurality of training datasets.

The feature extractor uses the average value of the class when a missing value exists in the dataset from which the plurality of dataset features are extracted, and the variable in which the missing value exists is of a numeric type, and when the variable with the missing value is of a nominal type The plurality of extracted dataset characteristics are preprocessed as the missing value is processed using the class mode, and when class imbalance exists in the dataset from which the plurality of dataset characteristics are extracted, according to the existing class imbalance One of the methods of resolving class imbalance is the undersampling method that removes the majority class and the oversampling method that duplicates the minority class to fit the majority class. As a result of resolving the existing class imbalance by using the method, it is possible to pre-process the extracted plurality of data set characteristics.

The recommendation unit recognizes a plurality of preprocessed dataset characteristics of the user dataset, identifies a plurality of dataset characteristics related to the recognized plurality of dataset characteristics in the generated metadata set, and identifies the plurality of identified dataset characteristics. At least one of the customized sampling method and the customized classification algorithm may be recommended based on a dataset characteristic and the generated selection rule base.

According to an embodiment of the present invention, a sampling method and a classification algorithm recommendation method using a metadata set includes the steps of: in a dataset collecting unit, collecting an open data set from an open database; in a feature extracting unit, the collected open data set extracting a plurality of dataset characteristics of , and pre-processing the extracted plurality of dataset characteristics, mapping a sampling method and a classification algorithm according to the plurality of pre-processed dataset characteristics in a mapping processing unit, a metadata set In the generator, a selection rule base for selecting a recommended sampling method and a recommended classification algorithm is generated based on the mapped sampling method and the mapped classification algorithm, and the generated selection rule base and the plurality of preprocessed datasets are generated. generating a metadata set including characteristics, and recommending, in the recommendation unit, at least one of a customized sampling method and a customized classification algorithm by using the generated metadata set with respect to a user dataset input from a user. can do.

The step of extracting a plurality of dataset characteristics of the collected open dataset and pre-processing the extracted plurality of dataset characteristics includes classifying the collected open dataset into a plurality of folds, A dataset included in the remaining folds except for one of a plurality of folds is determined as a plurality of training datasets, and the number of variables in the collected open dataset from the determined plurality of training datasets, instances The number of, the number of classes, the degree of bias of the class, the entropy of the class, the degree of overlap of the variable, the silhouette score, the hub score, the entropy of the variable, the linearity of the data set, Step, when a missing value exists in the dataset from which the plurality of dataset characteristics are extracted, and the variable with the missing value is of a numeric type, the missing value is processed using the average value of the class, and the variable with the missing value is nominal preprocessing the plurality of extracted dataset characteristics as the missing value is processed using the mode of the corresponding class in the case of type, and if there is a class imbalance in the dataset from which the plurality of dataset characteristics are extracted, the existing One of the classes of undersampling, which removes a majority class according to class imbalance, and oversampling, which replicates a minority class to fit a majority class The method may include pre-processing the extracted characteristics of the plurality of datasets as the existing class imbalance is resolved using an imbalance resolution method.

The mapping of the sampling method and the classification algorithm according to the plurality of preprocessed dataset characteristics may include applying the plurality of preprocessed dataset characteristics to a plurality of sampling methods, and sampling method accuracy in each of the plurality of applied sampling methods. , mapping the plurality of preprocessed dataset characteristics to the sampling method according to the calculated sampling method accuracy, and applying the plurality of preprocessed dataset characteristics to a plurality of classification algorithms, and the applied plurality of classifications Calculating the classification algorithm accuracy based on the characteristics and hyperparameters of the classification algorithm in each algorithm, and mapping the preprocessed plurality of data set characteristics and the classification algorithm according to the calculated classification algorithm accuracy.

generating a selection rule base for selecting a recommended sampling method and a recommended classification algorithm based on the mapped sampling method and the mapped classification algorithm, and including the generated selection rule base and the preprocessed plurality of data set characteristics The generating of the metadata set may include filtering the plurality of preprocessed dataset characteristics applied to the mapped sampling method and the mapped classification algorithm, and generating a plurality of datasets related to the filtered plurality of dataset characteristics. machine learning by inputting the recommended sampling method and the recommended classification algorithm, generating a selection rule base for selecting the recommended sampling method and the recommended classification algorithm based on the machine learning, and characteristics of the filtered plurality of datasets and generating a metadata set including a plurality of datasets related to and the generated selection rule base.

The present invention generates a metadata set using the dataset characteristics of an open dataset, and when a dataset to be used by a user is input, an optimal algorithm type and A sampling method and classification algorithm recommendation apparatus using a metadata set that recommends a parameter value and a data sampling method may be provided.

The present invention can recommend an optimal algorithm type, parameter value, and data sampling method showing the best performance based on the metadata set generated by learning the dataset characteristics of the open dataset in advance and making it metadata.

The present invention can reduce time and cost by reducing repetitive experiments in data science and artificial intelligence development companies, and can support eco-friendly artificial intelligence development and data analysis by reducing the use of hardware.

In the present invention, when a user enters a field to which data science is not applied, the user may experience difficulties due to lack of prior knowledge. can be automatically found and recommended.

The present invention provides a sampling method and classification related to artificial intelligence algorithms by referring to the method of automatically finding and recommending similar machine learning algorithms through pre-learned datasets when novice data scientists and AI developers lack data know-how. Acquisition of know-how on how to select an algorithm can be supported.

1 is a diagram for explaining a sampling method and a classification algorithm recommendation apparatus using a metadata set according to an embodiment of the present invention.
2 is a diagram for explaining a procedure for generating a metadata set by a sampling method and classification algorithm recommendation apparatus using a metadata set according to an embodiment of the present invention.
3 to 5 are diagrams for explaining a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention.
6 is a diagram for explaining the structure of a data set characteristic collected by a sampling method and classification algorithm recommendation apparatus using a metadata set according to an embodiment of the present invention.
7 is a diagram illustrating a metadata set generated by a sampling method and classification algorithm recommendation apparatus using a metadata set according to an embodiment of the present invention.
8 is a diagram illustrating a result of performance evaluation in relation to a classification algorithm by the apparatus for recommending a sampling method and a classification algorithm using a metadata set according to an embodiment of the present invention.
9 is a diagram illustrating a result of performance evaluation in relation to a sampling method by the apparatus for recommending a sampling method and a classification algorithm using a metadata set according to an embodiment of the present invention.

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings.

Examples and terms used therein are not intended to limit the technology described in this document to specific embodiments, and should be understood to include various modifications, equivalents, and/or substitutions of the embodiments.

In the following, when it is determined that a detailed description of a known function or configuration related to various embodiments may unnecessarily obscure the gist of the present invention, a detailed description thereof will be omitted.

In addition, terms to be described later are terms defined in consideration of functions in various embodiments, which may vary according to intentions or customs of users and operators. Therefore, the definition should be made based on the content throughout this specification.

In connection with the description of the drawings, like reference numerals may be used for like components.

The singular expression may include the plural expression unless the context clearly dictates otherwise.

In this document, expressions such as “A or B” or “at least one of A and/or B” may include all possible combinations of items listed together.

Expressions such as "first," "second," "first," or "second," can modify the corresponding elements, regardless of order or importance, and to distinguish one element from another element. It is used only and does not limit the corresponding components.

When an (eg, first) component is referred to as being “connected (functionally or communicatively)” or “connected” to another (eg, second) component, that component is It may be directly connected to the component or may be connected through another component (eg, a third component).

As used herein, "configured to (or configured to)" according to the context, for example, hardware or software "suitable for," "having the ability to," "modified to ," "made to," "capable of," or "designed to," may be used interchangeably.

In some contexts, the expression “a device configured to” may mean that the device is “capable of” with other devices or components.

For example, the phrase “a processor configured (or configured to perform) A, B, and C” refers to a dedicated processor (eg, an embedded processor) for performing the operations, or by executing one or more software programs stored in a memory device. , may refer to a general-purpose processor (eg, a CPU or an application processor) capable of performing corresponding operations.

Also, the term 'or' means 'inclusive or' rather than 'exclusive or'.

That is, unless stated otherwise or clear from context, the expression 'x employs a or b' means any one of natural inclusive permutations.

Terms such as '.. unit' and '.. group' used below mean a unit for processing at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

1 is a diagram for explaining a sampling method and a classification algorithm recommendation apparatus using a metadata set according to an embodiment of the present invention.

1 illustrates components of a sampling method and classification algorithm recommendation apparatus using a metadata set according to an embodiment of the present invention.

Referring to FIG. 1 , an apparatus 100 for recommending a sampling method and classification algorithm using a metadata set according to an embodiment of the present invention includes a dataset collecting unit 110 , a feature extracting unit 120 , and a mapping processing unit 130 . , a metadata set generating unit 140 and a recommendation unit 150 .

According to an embodiment of the present invention, the data set collecting unit 110 collects an open data set from an open database.

Here, the open database may refer to a database storing open data.

That is, the dataset collection unit 110 collects an open dataset from an open database for prior data learning. It may be data input based on an open database, or it may be data collected by the apparatus 100 for recommending a sampling method and classification algorithm using a metadata set by accessing an open database.

Meanwhile, the data set collection unit 110 may collect a user data set even when a user data set for testing is input by a user.

According to an embodiment of the present invention, the feature extraction unit 120 may extract a plurality of data set characteristics of an open data set or a user data set, and preprocess the plurality of extracted data set characteristics.

As an example, the feature extraction unit 120 may include the number of variables, the number of instances, the number of classes, the degree of bias of classes, entropy of classes, degree of overlap of variables, silhouette score, hub score, and number of variables in the collected open dataset. A plurality of dataset characteristics including entropy, linearity and neighborliness of the dataset may be extracted, and the extracted plurality of dataset characteristics may be pre-processed.

In addition, the feature extraction unit 120 is the number of variables in the input user dataset, the number of instances, the number of classes, the degree of class bias, the entropy of the class, the degree of overlap of the variables, the silhouette score, the hub score, the entropy of the variable , it is possible to extract a plurality of dataset characteristics including linearity and neighborliness of the dataset, and preprocess the extracted plurality of dataset characteristics.

That is, the feature extraction unit 120 determines the number of variables, the number of instances, the number of classes, the degree of class bias, the entropy of the class, the degree of overlap of the variables, the silhouette score, the hub score, the entropy of the variable, the linearity of the dataset and It is possible to extract a plurality of data set features including neighborliness.

Specifically, the feature extractor 120 classifies a user dataset or an open dataset into a plurality of folds, and a dataset included in the remaining folds except for one of the classified plurality of folds. may be determined as a plurality of training datasets, and a plurality of dataset characteristics may be extracted from the determined plurality of training datasets.

Here, a dataset corresponding to one of a plurality of classified folds may be used as a test dataset of a selected classification algorithm after a training process using the plurality of training datasets is completed.

According to an embodiment of the present invention, the feature extractor 120 may determine whether missing values and class imbalance exist in a plurality of data set characteristics, and perform a preprocessing process for resolving the missing values and class imbalance.

As an example, the feature extraction unit 120 includes missing values in a dataset from which a plurality of data set characteristics are extracted, and when a variable having missing values is a numeric type, the plurality of extracted values are processed using the average value of the corresponding class. You can preprocess the data set characteristics of .

According to an embodiment of the present invention, when a missing value exists in a dataset from which a plurality of dataset characteristics are extracted, and the variable in which the missing value exists is a nominal type, the characteristic extraction unit 120 processes the missing value using the mode value of the corresponding class. Accordingly, it is possible to pre-process the extracted plurality of data set characteristics.

For example, when class imbalance exists in a dataset from which a plurality of dataset characteristics are extracted, the feature extractor 120 removes a majority class according to the existing class imbalance. Under sampling method And a plurality of data set characteristics can be preprocessed by resolving the class imbalance using any one of the over sampling methods of replicating the minority class to fit the majority class. can

According to an embodiment of the present invention, the mapping processing unit 130 may map a sampling method and a classification algorithm according to the plurality of data set characteristics preprocessed by the characteristic extraction unit 120 .

For example, the mapping processing unit 130 applies a plurality of preprocessed data set characteristics to a plurality of sampling methods, calculates a sampling method accuracy in each of the plurality of applied sampling methods, and calculates a plurality of preprocessed data sets according to the calculated sampling method accuracy. of data set characteristics and sampling methods can be mapped.

According to an embodiment of the present invention, the mapping processing unit 130 applies a plurality of preprocessed dataset characteristics to a plurality of classification algorithms, calculates the classification algorithm accuracy in each of the plurality of classification algorithms applied, and calculates the classification algorithm accuracy. It is possible to map the characteristics of a plurality of preprocessed datasets and classification algorithms.

Here, for the sampling method accuracy and classification algorithm accuracy, F1-Score and G-mean are used, and F1-score is a value determined by considering a True Positive Rate and a Positive Predictive Vale, G-mean can be viewed as an arithmetic mean considering true positive and true negative values.

For example, the F1-score is characterized by using a harmonic average to calculate precision and recall.

F1-socre may be calculated using Equation 1 below, and G-mean may be calculated using Equation 2 below.

[Equation 1]

Equation 1 may be calculated in consideration of precision and recall.

[Equation 2]

Equation 2 may be calculated using a true positive (TP) ratio and a true negative (TN) ratio.

According to an embodiment of the present invention, the mapping processing unit 130 may calculate the classification algorithm accuracy based on the characteristics and hyperparameters of the classification algorithm in each of the plurality of classification algorithms applied to the plurality of preprocessed data set characteristics.

That is, the mapping processing unit 130 classifies in consideration of precision and recall or True Positive (TP) ratio and True Negative (TN) ratio with respect to the characteristics and hyperparameters of the classification algorithm. Algorithm accuracy can be calculated.

According to an embodiment of the present invention, the metadata set generating unit 140 selects the recommended sampling method and the recommended classification algorithm based on the mapping method mapped by the mapping processing unit 130 and the mapped classification algorithm. can create

Also, the metadata set generating unit 140 may generate a metadata set including the generated selection rule base and a plurality of preprocessed data set characteristics.

Here, the selection rule base may be a criterion for selecting a recommended sampling method and a recommended classification algorithm when a test dataset is additionally input.

According to an embodiment of the present invention, the metadata set generating unit 140 filters the plurality of preprocessed dataset characteristics applied to the mapped sampling method and the mapped classification algorithm, A data set may be input to a recommended sampling method and a recommended classification algorithm to perform machine learning, and a selection rule base for selecting a recommended sampling method and a recommended classification algorithm may be generated based on the machine learning. Here, based on machine learning may refer to using a machine learning result.

Also, the metadata set generating unit 140 may generate a plurality of datasets related to the filtered plurality of dataset characteristics and a metadata set including the generated selection rule base.

According to an embodiment of the present invention, the recommendation unit 150 may recommend at least one of a custom sampling method and a custom classification algorithm by using a metadata set generated with respect to a user data set input from a user.

That is, when a user inputs a user dataset to be used, the recommendation unit 150 scans the input dataset, automatically recognizes the characteristics of the user dataset through the characteristic extraction unit 120 , and a metadata set generated by pre-learning. Based on this, it is possible to automatically recommend or select the optimal algorithm type, parameter value, and sampling method for sampling data suitable for the characteristics of the user dataset.

As an example, the recommendation unit 150 recognizes a plurality of preprocessed dataset characteristics of the user dataset, identifies a plurality of dataset characteristics related to the plurality of dataset characteristics recognized in the generated metadata set, and checks the At least one of a custom sampling method and a custom classification algorithm may be recommended based on the plurality of data set characteristics and the generated selection rule base.

According to an embodiment of the present invention, the apparatus 100 for recommending a sampling method and classification algorithm using a metadata set includes a metadata set storage unit (not shown) that stores the metadata set generated by the metadata set generator 140 . ) may be further included.

According to an embodiment of the present invention, the apparatus 100 for recommending a sampling method and classification algorithm using a metadata set directly calls a metadata set stored in a metadata set storage unit (not shown) when a user dataset is input. It can be used to recommend sampling methods and classification algorithms suitable for user datasets.

Therefore, the present invention generates a metadata set using the dataset characteristics of an open dataset, and when a dataset to be used by the user is input, an optimal algorithm for the dataset the user wants to use using the metadata set It is possible to provide a sampling method and a classification algorithm recommendation apparatus using a metadata set that recommends a type and parameter value and a data sampling method.

In addition, the present invention can recommend the optimal algorithm type, parameter value, and data sampling method that shows the best performance based on the metadata set generated by learning the dataset characteristics of the open dataset in advance and making it metadata. have.

2 is a diagram for explaining a procedure for generating a metadata set by a sampling method and classification algorithm recommendation apparatus using a metadata set according to an embodiment of the present invention.

2 illustrates a process in which the apparatus for recommending a sampling method and classification algorithm using a metadata set according to an embodiment of the present invention generates a metadata set using an open dataset.

Referring to FIG. 2 , in step S201 , the apparatus for recommending a sampling method and classification algorithm using a metadata set according to an embodiment of the present invention classifies an open dataset collected from an open database into a plurality of folds.

For example, the apparatus for recommending a sampling method and classification algorithm using a metadata set according to an embodiment of the present invention classifies one open dataset into 10 folds, and uses a dataset of 9 folds as a training dataset. and use it as a test data set in one fold.

In step S202, the apparatus for recommending a sampling method and classification algorithm using a metadata set according to an embodiment of the present invention extracts a plurality of dataset characteristics from nine training datasets and one test dataset.

Here, the extracted plurality of dataset characteristics are the number of variables, the number of instances, the number of classes, the degree of class bias, the entropy of the class, the degree of overlap of the variables, the silhouette score, the hub score, the entropy of the variable, and the linearity of the dataset. and neighborhood.

That is, the apparatus for recommending a sampling method and classification algorithm using a metadata set according to an embodiment of the present invention may extract a plurality of data set characteristics from one data set. In addition, the plurality of extracted data set characteristics may be included in the metadata set.

In step S203, the apparatus for recommending a sampling method and classification algorithm using a metadata set according to an embodiment of the present invention may interpolate missing values of the extracted data set characteristics.

That is, the apparatus for recommending a sampling method and classification algorithm using a metadata set according to an embodiment of the present invention interpolates the missing value using the average value of the class when the variable having the missing value is a numeric type, and the variable with the missing value is In the case of a nominal type, missing values can be interpolated using the mode of the corresponding class.

In step S204, the apparatus for recommending a sampling method and a classification algorithm using a metadata set according to an embodiment of the present invention may resolve class imbalance by applying the class imbalance resolution method.

Class imbalance indicates that the property of the target variable to be classified is imbalanced as it has a great influence on classification performance. In addition, cost-sensitive learning may be a method of reducing classification errors by using a cost matrix for misclassified data differently from the data extraction method according to the distribution of classes.

In addition, there are an oversampling method and an undersampling method. The undersampling method balances the class distribution by removing a majority class. The problem with the undersampling method is the loss of information. On the other hand, the over-sampling method may be a method of balancing a minority class by duplicating it to fit a majority class.

In the apparatus for recommending a sampling method and classification algorithm using a metadata set according to an embodiment of the present invention, when class imbalance exists in a dataset from which a plurality of dataset characteristics are extracted, a majority class (majority class) according to the existing class imbalance ) and the oversampling method of duplicating a minority class to fit the majority class. It is possible to apply the method of resolving the class imbalance to resolve.

In step S205, the apparatus for recommending a sampling method and classification algorithm using a metadata set according to an embodiment of the present invention extracts the preprocessed data set characteristics.

In addition, in step S206 , the apparatus for recommending a sampling method and classification algorithm using a metadata set according to an embodiment of the present invention applies the preprocessed dataset characteristic corresponding to the training dataset to the classification algorithm to perform a test.

In step S207, the apparatus for recommending a sampling method and classification algorithm using a metadata set according to an embodiment of the present invention performs performance evaluation based on step S206.

That is, the sampling method and classification algorithm recommendation device using the metadata set performs sampling for each fold using K-fold cross validation, and data characteristics and performance for each fold A performance evaluation may be performed by performing a test and deriving a performance test result.

In step S208, the apparatus for recommending a sampling method and classification algorithm using a metadata set according to an embodiment of the present invention generates a metadata set.

That is, the apparatus for recommending a sampling method and classification algorithm using a metadata set includes a plurality of dataset characteristics extracted in step S202, a preprocessed dataset characteristic extracted in step S205, a classification algorithm applied in step S206 and A metadata set is generated based on the result of the performance evaluation performed in step S207.

In other words, the apparatus for recommending a sampling method and a classification algorithm using a metadata set filters the plurality of preprocessed dataset characteristics applied to the mapped sampling method and the mapped classification algorithm, and a plurality of data related to the filtered plurality of dataset characteristics The set is put into a recommended sampling method and a recommended classification algorithm for machine learning, and a selection rule base for selecting a recommended sampling method and a recommended classification algorithm based on machine learning is generated and metadata including both the results and characteristics of a plurality of datasets. create three

3 is a view for explaining a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention.

3 illustrates an embodiment in which a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention identify characteristics of a dataset extracted from an open dataset and recommend a customized algorithm.

Referring to FIG. 3 , in step 301 , the sampling method and the classification algorithm recommendation method using a metadata set according to an embodiment of the present invention collects an open data set from an open database.

That is, the sampling method and classification algorithm recommendation method using the metadata set can collect the open data set from the open database.

In step 302, a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention extracts a plurality of data set characteristics of an open data set.

That is, the sampling method and classification algorithm recommendation method using the metadata set classifies the collected open data set into a plurality of folds, and the collected open dataset is divided into a plurality of folds except for one of the classified folds. The included dataset is determined as a plurality of training datasets, and the number of variables, the number of instances, the number of classes, the degree of class bias, the entropy of the classes, and the overlap of variables in the open dataset collected from the plurality of training datasets. Multiple dataset characteristics including degree, silhouette score, hub score, entropy of variables, linearity and neighborhood of the dataset are extracted.

In step 303, a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention pre-process a plurality of data set characteristics.

That is, a sampling method and a classification algorithm recommendation method using a metadata set performs a preprocessing process for interpolating missing values of a plurality of dataset characteristics and resolving class imbalance.

In step 304, a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention map a sampling method according to characteristics of a plurality of datasets.

That is, the sampling method and classification algorithm recommendation method using the metadata set applies the characteristics of the plurality of preprocessed datasets to the plurality of sampling methods, calculates the sampling method accuracy in each of the plurality of sampling methods applied, and calculates the sampling method According to the accuracy, it is possible to map the characteristics of a plurality of preprocessed datasets and the sampling method.

In step 305, a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention map a classification algorithm according to characteristics of a plurality of datasets.

That is, the sampling method and the classification algorithm recommendation method using the metadata set apply the characteristics of a plurality of preprocessed datasets to the plurality of classification algorithms, and classify them based on the characteristics and hyperparameters of the classification algorithm in each of the plurality of classification algorithms applied. Algorithm accuracy may be calculated, and a plurality of preprocessed dataset characteristics may be mapped to a classification algorithm according to the calculated classification algorithm accuracy.

In step 306, a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention generate a selection rule base based on the mapped sampling method and classification algorithm.

That is, the sampling method and classification algorithm recommendation method using the metadata set filters the plurality of preprocessed dataset characteristics applied to the mapped sampling method and the mapped classification algorithm, and a plurality of datasets related to the filtered plurality of dataset characteristics. can be input to a recommended sampling method and a recommended classification algorithm to perform machine learning, and a selection rule base for selecting a recommended sampling method and a recommended classification algorithm can be generated based on machine learning.

In step 307, the sampling method and classification algorithm recommendation method using a metadata set according to an embodiment of the present invention stores the selection rule base as a metadata set.

That is, the sampling method and the classification algorithm recommendation method using the metadata set may store a plurality of data set characteristics as a metadata set together with the selection rule base generated in step 306 .

In step 308, the sampling method and classification algorithm recommendation method using a metadata set according to an embodiment of the present invention uses at least one of a customized sampling method and a customized classification algorithm using a metadata set for a user dataset input from a user. I can recommend one.

That is, the sampling method and classification algorithm recommendation method using the metadata set compares the plurality of dataset characteristics according to the user dataset input by the user with the plurality of dataset characteristics included in the metadata set to provide a customized sampling method and customized method. At least one of the classification algorithms may be recommended.

4 is a diagram for explaining a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention.

4 illustrates an embodiment of recommending at least one of a sampling method and a classification algorithm recommendation method using a metadata set according to a user dataset inputted according to an embodiment of the present invention.

Referring to FIG. 4 , in step 401 , a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention receive a user data set.

That is, a sampling method and a classification algorithm recommendation method using a metadata set receives a user data set from a user, and the user data set can be viewed as a data set for the user to use in an artificial intelligence algorithm.

In step 402, a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention extracts a plurality of data set characteristics of a user data set.

That is, the sampling method and classification algorithm recommendation method using the metadata set include the number of variables, the number of instances, the number of classes, the degree of class bias, the entropy of classes, the degree of overlap of variables, the silhouette score, and the number of variables in the input user dataset. A plurality of data set characteristics including hub scores, entropy of variables, and linearity and neighborliness of the data set can be extracted.

In step 403, a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention pre-process a plurality of data set characteristics.

That is, the sampling method and the classification algorithm recommendation method using the metadata set perform a preprocessing process for interpolating missing values and resolving class imbalance for a plurality of data set characteristics extracted from a user data set.

In step 404, a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention determine a recommended sampling method according to characteristics of a plurality of datasets based on a metadata set.

That is, the sampling method and classification algorithm recommendation method using the metadata set applies a plurality of dataset characteristics of the user dataset to the pre-learned recommended sampling methods included in the metadata set, and the best performance is obtained according to the application result. The sampling method is determined as the recommended sampling method.

In step 405, the sampling method and the classification algorithm recommendation method using a metadata set according to an embodiment of the present invention determine a recommended classification algorithm according to characteristics of a plurality of datasets based on the metadata set.

That is, the sampling method and classification algorithm recommendation method using the metadata set applies a plurality of dataset characteristics of the user dataset to the pre-learned recommendation classification algorithms included in the metadata set, and the best performance is obtained according to the application result. A classification algorithm is determined as a recommended classification algorithm. Here, according to the determination of the recommended classification algorithm, the type and parameter value of the classification algorithm may also be determined.

In step 406, the sampling method and classification algorithm recommendation method using the metadata set according to an embodiment of the present invention learns the recommended sampling method and the recommended classification algorithm, and recommends at least one of the customized sampling method and the customized classification algorithm. can

That is, the sampling method and the classification algorithm recommendation method using the metadata set learn the recommended sampling method and the recommended classification algorithm determined by simultaneously considering the plurality of dataset characteristics of the user dataset and the plurality of dataset characteristics included in the metadata set. Accordingly, it is possible to automatically determine and recommend a classification algorithm and a data sampling method customized to a user dataset.

5 is a diagram for explaining a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention.

5 illustrates an embodiment in which a metadata set is generated by a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention by collecting a plurality of dataset characteristics.

Referring to FIG. 5 , in step 501 , a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention apply a sampling method to a training dataset to generate a plurality of training datasets.

That is, a sampling method and a classification algorithm recommendation method using a metadata set generate a plurality of training datasets using a sampling method, which is a method of sampling training data to be suitable for a classification algorithm, which is one of the data sampling methods.

In step 502, a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention extract characteristics of a dataset.

In other words, the sampling method and classification algorithm recommendation method using the metadata set are the number of variables in the dataset, the number of instances, the number of classes, the degree of class bias, the entropy of classes, the degree of overlap of variables, the silhouette score, the hub score, A plurality of dataset characteristics including entropy of variables, linearity and neighborliness of the dataset can be extracted from each training dataset.

In step 503, the sampling method and the classification algorithm recommendation method using the metadata set according to an embodiment of the present invention determine whether missing values exist.

The sampling method and classification algorithm recommendation method using a metadata set according to an embodiment of the present invention proceeds to step 504 if there is a missing value, and proceeds to step 507 if there is no missing value.

In step 504, the sampling method and classification algorithm recommendation method using a metadata set according to an embodiment of the present invention determines whether a variable having a missing value is a numeric type.

As an example, the sampling method and classification algorithm recommendation method using the metadata set proceeds to step 505 when the variable with the missing value is of a numeric type, and goes to step 506 when the variable with the missing value is not a numeric type. proceed with

In step 505, the sampling method and the classification algorithm recommendation method using the metadata set according to an embodiment of the present invention process the missing value by inputting the average of the corresponding class.

That is, the sampling method and the classification algorithm recommendation method using the metadata set can interpolate the missing value by inputting the class average into the missing value in order to interpolate the numerical missing value.

In step 506, the sampling method and classification algorithm recommendation method using the metadata set according to an embodiment of the present invention input the mode value of the corresponding class to process the missing value.

That is, the sampling method and classification algorithm recommendation method using the metadata set can interpolate the missing value by inputting the class mode to the missing value in order to interpolate the nominally missing value.

In step 507, the sampling method and classification algorithm recommendation method using a metadata set according to an embodiment of the present invention performs sampling for each fold using a K-fold cross-validation method, and stores the test result.

That is, the sampling method and classification algorithm recommendation method using the metadata set samples the training dataset characteristics for each fold classified by K, and derives and stores test results according to the sampled training dataset characteristics. For example, K may be 10.

In step 508, a sampling method and a classification algorithm recommendation method using a metadata set according to an embodiment of the present invention apply a class imbalance method and a classification algorithm.

That is, a sampling method and a classification algorithm recommendation method using a metadata set may apply a method for resolving class imbalance, and a test dataset characteristic related to the sampled training dataset characteristic may be applied to the classification algorithm.

In step 509, the sampling method and the classification algorithm recommendation method using the metadata set according to an embodiment of the present invention measure classification performance.

That is, the sampling method and the classification algorithm recommendation method using the metadata set measure the classification performance for the application result of the classification algorithm, and F1-Score and G-mean can be used to measure the classification performance.

6 is a diagram for explaining the structure of a data set characteristic collected by a sampling method and classification algorithm recommendation apparatus using a metadata set according to an embodiment of the present invention.

Referring to FIG. 6 , a structure of a dataset characteristic 600 may represent a structure in which a dataset complexity 610 and a basic dataset characteristic 620 can be distinguished.

For example, the dataset complexity 610 includes class bias, class entropy, variable overlap, silhouette score, hub score, variable entropy, data set linearity, neighborhood, and Herfindahl-Hirschman Index (HHI). and Silhouette.

Meanwhile, the basic dataset characteristic 620 may include the number of instances, the number of missing values, the number of variables, and the number of classes.

The Herfindahl-Hirschman Index (HHI) may represent a market situation, which is a competitive situation in the industry, on a scale of 0 to 1, and the Silhouette may be an evaluation index used for cluster evaluation in cluster analysis.

The higher the Silhouette, the better the clusters are grouped, the more homogeneous within the cluster, and the better the heterogeneity between clusters.

The number of instances indicates the size of the data set, which means that more information means more information.

However, the larger the data, the more noisy, the more time it takes to train in the field of machine learning, the more noise it can generate, and if it is too small, it can be difficult to classify correctly due to lack of information.

If too few variables are considered for the number of variables, the complexity of the predictive model will decrease, but the accuracy may be lower than when considering a large number of variables.

As the number of variables decreases, the G-mean increases.

The number of classes is a dimension of a class. If it is a binary type class, the expected probability of each class is 0.5, and if the number of classes is 4, it will be 0.25.

In addition, since the expected probability is different in the case of probabilistic binary form and the case of 4 classes, and the expected probability decreases as the number of classes is increased, it may be closely related to the accuracy of the class.

Missing values occur in various fields. Missing values can have adverse effects in data mining, machine learning, and other information systems.

Missing values can usually be caused by sensor defects, lack of response in scientific experiments, measurement deficiencies, data transmission problems in digital systems, or the reluctance of respondents to respond to surveys.

If there are many missing values, there is not enough information to classify the class, which may cause misclassification.

HHI is one of the indices indicating the concentration of a company's market in an industry. The advantage of HHI is that it can be intuitively known at once by representing the market conditions of the entire industry as 0 to 1.

HHI can be applied to the equilibrium state of a class because, when the market share is viewed as the ratio of each class, the equilibrium state of the class and the competitive state of the market have a similar shape.

Entropy indicates the amount and purity of information and is a method that can quantitatively measure the uncertainty about the amount of information generated by given data. will increase the amount of

In other words, it can be said that the rarer the information, the greater the amount of information, and the more universal the data, the more consistent the data.

Therefore, entropy can be an indicator of whether there are not many specific data and consistently high purity data.

Class entropy is an expression of class imbalance using entropy, and as the class imbalance is severe, it can be expressed as a higher number.

The Silhouette coefficient is an indicator used to verify that the cluster is correctly formed after clustering analysis.

It can be determined how densely the values are clustered in the calculated cluster and whether the distance between the clusters is sufficient. The closer the value of Silhouette is to 1, the more appropriate the number of clusters can be.

As for data nonlinearity, the more complex the data type, the more difficult it may be in classification problems, and some classification algorithms use linear models to solve classification problems.

For example, the present invention may use a nonlinearity of linear classifier measurement method, which is a method for simultaneously considering linearity of data and outliers.

A method of classifying through a non-linear classifier and comparing the classified results through a linear kernel function of a support vector machine (SVM) to compare errors may be used. .

The hub score is an index for measuring the cohesion of data using network connectivity.

The hub score can be measured through a number connected to a node by constructing a network using the given data.

The degree of overlap of variables (feature overlap) suggests the degree of overlap between variables as a ratio. The more non-overlapping the data, the clearer the decision boundaries are. Therefore, the higher the overlap of variables, the more advantageous the undersampling methods such as Tomek link, ENN, and CNN.

Neighborhood can be related to the cause of poor classification performance in linear separation problems because the data is placed in the wrong place when it is incorrectly labeled.

For each instance, the distance to the nearest neighbor within a class and the distance to the nearest neighbor of another class are calculated, and the ratio of the sum of distances within a class to the sum of distances between classes is used as the number of datasets.

Dimensionality may represent the difference between the number of variables reduced through Principal Component Analysis (PCA) and the number of original data variables as a ratio.

7 is a diagram illustrating a metadata set generated by a sampling method and classification algorithm recommendation apparatus using a metadata set according to an embodiment of the present invention.

Referring to FIG. 7 , an example 700 of a metadata set may include various information such as identification number, accuracy, G-mean score, F-score score, algorithm data, method, ratio, and characteristics of a plurality of datasets. .

8 is a diagram illustrating a result of performance evaluation in relation to a classification algorithm by a sampling method and a classification algorithm recommendation apparatus using a metadata set according to an embodiment of the present invention.

Referring to FIG. 8 , the graph 800 indicates the type of classification algorithm on the horizontal axis and frequency on the vertical axis.

ve Bayes), RF(random forest) 및 SVM(support vector machine)을 포함한다.The classification algorithms are knn (Nearest Neighbor), LR (Logistic Regression), NB (na

ve Bayes), random forest (RF) and support vector machine (SVM).

The graph 800 shows that a random forest (RF), which is an algorithm for classifying classes, exhibits superior performance compared to other algorithms.

ve Bayes) 분류기나, SVM, 의사결정 나무 모형과 다르게 학습데이터를 이용하지 않다가 실증데이터가 주어져야 움직이는 lazy학습법이다.The k-NN method is naive Bayes (na

ve Bayes) Unlike classifiers, SVMs, and decision tree models, it is a lazy learning method that does not use learning data and moves only when empirical data is given.

The logistic regression model has a functional form of a logistic cumulative distribution function (CDF).

The naive Bayes classifier is a probabilistic approach during supervised learning and uses Bayes theorem. The naive Bayes model is relatively simple and the calculation process is not complicated, but it is known to show excellent performance.

Random forest (RF) is a classification algorithm based on a decision tree model, and it is an ensemble technique of a majority vote method by generating several decision tree models.

SVM (Support Vector Machine) is a method used to classify data using an n-1 dimensional hyperplane in n dimensional data.

9 is a diagram illustrating a result of performance evaluation in relation to a sampling method by the apparatus for recommending a sampling method and a classification algorithm using a metadata set according to an embodiment of the present invention.

Referring to FIG. 9 , a graph 900 indicates the type of sampling method on the horizontal axis and frequency on the vertical axis.

The sampling methods are Adasyn(Adaptive Synthetic Sampling Approach for Imbalanced Learning), ENN(Edited Nearest Neighbors), NCL(Neighborhood Cleaning Rule), ROS(Random Over Sampling), RUS(Random Under Sampling), SMOTE(Synthetic Minority Oversampling TEchnique) ) and Tomek.

Graph 900 confirms that Adasyn and SMOTE exhibit excellent performance.

Oversampling methods include ROS, SMOTE, and ADASYN, and undersampling methods include RUS, ENN, Tomek link method, CNN and NCL.

ROS is a method of repetitively restoring and extracting data by randomly selecting a small number of classes until the data size of the majority class is the same as the random over-extraction method.

SMOTE is a method of selecting random data of a prime class and generating new artificial data among k-Nearest Neighbors (NN).

ADASYN (Adaptive Synthetic Sampling) is a method of generating data based on SMOTE considering the density distribution of a prime class.

RUS is a sampling method that arbitrarily deletes the majority class and matches the ratio of the minority class. RUS has the advantage of being convenient to use like ROS. However, there is a high possibility of losing important information because the data is reduced arbitrarily.

The CNN method is a method proposed by Hart (1968), which selects random data from training data and stores it in set X, and selects another piece of data and includes it in set X.

The data of set X is classified using the NN (Nearest Neighbors) rule, and when it is misclassified, it is stored in the set X. Then, random data is selected again and included in the set X to use the NN rule.

The CNN method is a method that iterates until all the training data are classified well except for the misclassified X set, and leaves data with clear boundaries of different classes.

ENN is a variant of CNN. Unlike CNN, if a value included in set X is misclassified, it can be excluded from X.

Tomek link is a method of removing internal data near decision boundaries based on CNN sampling method.

NCL is a mixture of Condensed Nearest Neighbors (CNN) and Edited Nearest Neighbors (ENN).

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

Although the embodiments have been described with reference to the limited drawings as described above, various modifications and variations are possible by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

100: Sampling method and classification algorithm recommendation device using metadata set
110: data set collection unit 120: feature extraction unit
130: mapping processing unit 140: metadata set generation unit
150: recommendation

Claims (20)

a data set collecting unit that collects open data sets from open databases;
a feature extracting unit for extracting a plurality of data set characteristics of the collected open data set and pre-processing the plurality of extracted data set characteristics;
a mapping processing unit for mapping a sampling method and a classification algorithm according to the characteristics of the plurality of preprocessed datasets;
generating a selection rule base for selecting a recommended sampling method and a recommended classification algorithm based on the mapped sampling method and the mapped classification algorithm, and including the generated selection rule base and the preprocessed plurality of data set characteristics a metadata set generator for generating a metadata set; and
and a recommender for recommending at least one of a custom sampling method and a custom classification algorithm using the generated metadata set with respect to a user data set input from a user.
Sampling method and classification algorithm recommendation device using metadata set. According to claim 1,
The feature extraction unit includes the number of variables, the number of instances, the number of classes, the degree of class bias, the entropy of the class, the degree of overlap of the variables in the collected open dataset, the silhouette score, the hub score, the entropy of the variable, and the Extracting the plurality of dataset features including linearity and neighbor, and pre-processing the extracted plurality of dataset features
Sampling method and classification algorithm recommendation device using metadata set. 3. The method of claim 2,
The feature extractor classifies the collected open dataset into a plurality of folds, and uses the dataset included in the remaining folds except for one of the classified plurality of folds into a plurality of training datasets. and extracting the plurality of dataset characteristics from the determined plurality of training datasets.
Sampling method and classification algorithm recommendation device using metadata set. 4. The method of claim 3,
When a missing value exists in the dataset from which the plurality of dataset characteristics are extracted, and the variable in which the missing value exists is a numeric type, the characteristic extractor processes the missing value by using the average value of the corresponding class, thereby processing the extracted plurality of data Characterized in pre-processing the set characteristics
Sampling method and classification algorithm recommendation device using metadata set. 4. The method of claim 3,
When a missing value exists in the dataset from which the plurality of dataset characteristics are extracted, and the variable in which the missing value exists is a nominal type, the feature extractor processes the missing value using the mode value of the corresponding class to process the plurality of extracted data Characterized in pre-processing the set characteristics
Sampling method and classification algorithm recommendation device using metadata set. 4. The method of claim 3,
When class imbalance exists in the dataset from which the plurality of dataset characteristics are extracted, the characteristic extraction unit removes a majority class according to the existing class imbalance. An under-sampling method and a minority class ( Minority class) according to the majority class (majority class) by using any one of the class imbalance resolution method of the over sampling method to resolve the existing class imbalance, the extracted plurality of dataset characteristics characterized by pre-treatment
Sampling method and classification algorithm recommendation device using metadata set. 3. The method of claim 2,
The mapping processing unit applies the characteristics of the plurality of preprocessed datasets to a plurality of sampling methods, calculates sampling method accuracy in each of the plurality of applied sampling methods, and according to the calculated sampling method accuracy, the plurality of preprocessed data sets Characterized in mapping data set characteristics and sampling methods
Sampling method and classification algorithm recommendation device using metadata set. 3. The method of claim 2,
The mapping processing unit applies the characteristics of the plurality of preprocessed datasets to a plurality of classification algorithms, calculates classification algorithm accuracy in each of the plurality of applied classification algorithms, and calculates the accuracy of the plurality of preprocessed classification algorithms according to the calculated classification algorithm accuracy. Characterized in mapping dataset characteristics and classification algorithms
Sampling method and classification algorithm recommendation device using metadata set. 9. The method of claim 8,
The mapping processing unit calculates the accuracy of the classification algorithm based on the characteristics and hyperparameters of the classification algorithm in each of the plurality of classification algorithms applied to the characteristics of the plurality of preprocessed datasets
Sampling method and classification algorithm recommendation device using metadata set. According to claim 1,
The metadata set generating unit filters the plurality of preprocessed dataset characteristics applied to the mapped sampling method and the mapped classification algorithm, and selects a plurality of datasets related to the filtered plurality of dataset characteristics as the recommended sampling method and machine learning by inputting the recommendation classification algorithm, and generating a selection rule base for selecting the recommendation sampling method and the recommendation classification algorithm based on the machine learning.
Sampling method and classification algorithm recommendation device using metadata set. 11. The method of claim 10,
The metadata set generating unit generates a metadata set including a plurality of datasets related to the filtered plurality of dataset characteristics and the generated selection rule base.
Sampling method and classification algorithm recommendation device using metadata set. 12. The method of claim 11,
and a metadata set storage unit configured to store the generated metadata set.
Sampling method and classification algorithm recommendation device using metadata set. According to claim 1,
The feature extractor includes the number of variables, the number of instances, the number of classes, the degree of bias in the class, the entropy of the class, the degree of overlap of the variables in the input user dataset, the silhouette score, the hub score, the entropy of the variable, and the Characterized in extracting a plurality of data set characteristics including linearity and neighbor
Sampling method and classification algorithm recommendation device using metadata set. 14. The method of claim 13,
The feature extractor classifies the input user dataset into a plurality of folds, and uses the dataset included in the remaining folds except for one of the classified plurality of folds into a plurality of training datasets. and extracting the plurality of dataset characteristics from the determined plurality of training datasets.
Sampling method and classification algorithm recommendation device using metadata set. 15. The method of claim 14,
The feature extractor uses the average value of the class when a missing value exists in the dataset from which the plurality of dataset features are extracted, and the variable in which the missing value exists is of a numeric type, and when the variable with the missing value is of a nominal type The plurality of extracted dataset characteristics are preprocessed as the missing value is processed using the class mode, and when class imbalance exists in the dataset from which the plurality of dataset characteristics are extracted, according to the existing class imbalance One of the methods of resolving class imbalance is the undersampling method that removes the majority class and the oversampling method that duplicates the minority class to fit the majority class. Pre-processing the extracted plurality of data set characteristics as the existing class imbalance is resolved using
Sampling method and classification algorithm recommendation device using metadata set. 16. The method of claim 15,
The recommendation unit recognizes a plurality of preprocessed dataset characteristics of the user dataset, identifies a plurality of dataset characteristics related to the recognized plurality of dataset characteristics in the generated metadata set, and identifies the plurality of identified dataset characteristics. Recommending at least one of the customized sampling method and the customized classification algorithm based on a dataset characteristic and the generated selection rule base
Sampling method and classification algorithm recommendation device using metadata set. In the data set collecting unit, collecting an open data set from an open database;
extracting, in a feature extraction unit, a plurality of data set features of the collected open data set, and pre-processing the extracted plurality of data set features;
mapping, in a mapping processing unit, a sampling method and a classification algorithm according to the characteristics of the plurality of preprocessed datasets;
In the metadata set generator, a selection rule base for selecting a recommended sampling method and a recommended classification algorithm is generated based on the mapped sampling method and the mapped classification algorithm, and the generated selection rule base and the preprocessed plurality are generated. generating a metadata set including data set characteristics of ; and
and recommending, by the recommendation unit, at least one of a custom sampling method and a custom classification algorithm using the generated metadata set with respect to the user data set input by the user.
Sampling method and classification algorithm recommendation method using metadata set. 18. The method of claim 17,
Extracting a plurality of dataset characteristics of the collected open dataset, and pre-processing the extracted plurality of dataset characteristics,
Classifying the collected open dataset into a plurality of folds, and determining a dataset included in the remaining folds except for one of the classified plurality of folds as a plurality of training datasets, The number of variables, the number of instances, the number of classes, the degree of class bias, the entropy of the classes, the degree of overlap of the variables, the silhouette score, the hub score, the entropy of the variables in the collected open dataset from the determined plurality of training datasets , extracting the plurality of dataset characteristics including linearity and neighborliness of the dataset;
When a missing value exists in the dataset from which the plurality of dataset characteristics are extracted and the variable with the missing value is of a numeric type, the missing value is processed using the average value of the corresponding class, and the variable with the missing value is of a nominal type pre-processing the plurality of data set characteristics extracted by processing the missing value using the mode value of the corresponding class; and
When class imbalance exists in the dataset from which the plurality of dataset characteristics are extracted, an undersampling method that removes a majority class according to the existing class imbalance and a minority class Pre-processing the characteristics of the plurality of extracted datasets as the existing class imbalance is resolved using any one of the class imbalance resolution methods of oversampling that replicates for a majority class characterized by including
Sampling method and classification algorithm recommendation method using metadata set. 18. The method of claim 17,
The mapping of the sampling method and the classification algorithm according to the characteristics of the plurality of preprocessed datasets comprises:
applying the plurality of preprocessed dataset characteristics to a plurality of sampling methods, calculating sampling method accuracy in each of the applied plurality of sampling methods, and calculating the plurality of preprocessed dataset characteristics according to the calculated sampling method accuracy; mapping the sampling method; and
Applying the preprocessed plurality of dataset characteristics to a plurality of classification algorithms, calculating the classification algorithm accuracy based on the characteristics and hyperparameters of the classification algorithm in each of the applied plurality of classification algorithms, and adding to the calculated classification algorithm accuracy and mapping the preprocessed plurality of data set characteristics and a classification algorithm according to
Sampling method and classification algorithm recommendation method using metadata set. 18. The method of claim 17,
generating a selection rule base for selecting a recommended sampling method and a recommended classification algorithm based on the mapped sampling method and the mapped classification algorithm, and including the generated selection rule base and the preprocessed plurality of data set characteristics The steps to create a metadata set are:
Filtering the plurality of preprocessed dataset characteristics applied to the mapped sampling method and the mapped classification algorithm, and adding a plurality of datasets related to the filtered plurality of dataset characteristics to the recommended sampling method and the recommended classification algorithm generating a selection rule base for performing machine learning by input and selecting the recommended sampling method and the recommended classification algorithm based on the machine learning; and
and generating a metadata set including a plurality of datasets related to the filtered plurality of dataset characteristics and the generated selection rule base.
Sampling method and classification algorithm recommendation method using metadata set.

Images