KR20080095708A - A method for modelling a huge amount of data - Google Patents

Abstract

A modeling method of high capacity data is provided to select representative data aggregate from high capacity data by using a genetic algorithm and to obtain an optimal hyperplane by applying a KRR(Kernel Ridge Regression) method to the selected representative data aggregate, consequently data modeling of high capacity data can be accomplished efficiently by using the small amount of calculation. A method for modeling high capacity data comprises the steps of: selecting representative data aggregate(415) by applying a genetic algorithm to high capacity data(410); extracting a feature vector(425) from the selected data aggregate(420); obtaining an optimal hyperplane(435), that is a data model, by applying KRR to the extracted feature vectors(430). In particular, the number of data which belong to the representative data aggregate can be set up optionally in the process to which the genetic algorithm is applied, and the trade-off between the accuracy of the optimal hyperplane and the calculation amount which is required from the step 430 can be adjusted by the optional setting. In addition, the feature vectors extracted from the step 420 which is the feature vector extraction step can be expanded to nonlinear space by using a Kernel function prior to the step 430 to which the KRR is applied.

Description

A METHOD FOR MODELLING A HUGE AMOUNT OF DATA

1 is a diagram showing a data set used in a comparative experiment, FIG. 1A is a diagram showing a data set forming one large circular cluster, and FIG. 1B is another data set forming four small circular clusters. drawing.

FIG. 2 is a diagram illustrating a result of modeling a data set of FIG. 1 using a support vector machine (SVM) method, and FIGS. 2A and 2B are results of modeling the data set of FIG. 1A by different parameters of kernel functions. 2C and 2D are diagrams illustrating the results of modeling the data set of FIG. 1B with different parameters of a kernel function.

3 is a diagram illustrating a result of modeling the data set of FIG. 1 using a kernel ridge regression (KRR) method, and FIGS. 3A and 3B illustrate the results of modeling the data set of FIG. 1A by different parameters of kernel functions. 3C and 3D are diagrams illustrating the results of modeling the data set of FIG. 1B with different parameters of a kernel function.

4 is a block diagram illustrating respective steps constituting a method for modeling a large amount of data according to an embodiment of the present invention.

5 shows an example of a basic genetic algorithm.

6 shows data used in an experiment.

Fig. 7 is a diagram showing experimental results of KRR in the case where 50 pieces of data are randomly selected from 2000 pieces of data.

FIG. 8 is a view showing experimental results of KRR in the case where 50 data are selected from 2000 total data using a genetic algorithm according to an embodiment of the present invention. FIG.

FIG. 9 is a view showing experimental results of KRR in the case where 20 data are selected from 2000 total data using a genetic algorithm according to an embodiment of the present invention. FIG.

<Explanation of symbols for main parts of the drawings>

400 : modeling method of large data (according to an embodiment of the present invention)

405: large data

410: Applying the genetic algorithm

415: representative data set

420: Extracting feature vectors

425: Feature Vector

430: Steps to Apply Ridge Regression Method

435: Optimal Transcendental Surface (Data Model)

The present invention relates to a method for modeling a large amount of data, and more specifically, to select a representative data set from the large data using a genetic algorithm (Genetic Algorithm), and then the selected representative data set Kernel Ridge Regression ; KRR) method to obtain an optimal transcendental plane, and to efficiently model a large amount of data with a smaller amount of computation.

Modeling large amounts of data can be used in many applications. In particular, subject modeling that models documents with the same subject can be applied in specialized search fields such as subject-based search and personalized search. The data modeling method includes a support vector machine (SVM) method using a support vector, a ridge regression (RR) method, and the like. The support vector machine (SVM) method is a method of extracting a region in which data exists and expressing the region as a support vector. Data outside the expressed region is regarded as outliers. The SVM method can naturally express complex regions (including nonlinear regions) in which data exists by mapping data into kernel space, which is a high dimensional feature space, using a kernel function. In contrast, the ridge regression (RR) method is a form of a regularized least-square error method that finds an optimal hyperplane with a minimum distance from all given data. The RR method is suitable for modeling the condensation of data. Like the SVM method, by using a kernel function, a nonlinear transcendental can be obtained by mapping a given data into kernel space. The RR method using the kernel function is called the Kernel Ridge Regression (KRR) method.

Comparing the support vector machine (SVM) method with the ridge regression (RR) method, there is a problem that the SVM method merely represents the boundary of a given data, so that the representation of the internal distribution of the data is insufficient. That is, when the SVM method is used, it is possible to express the distribution form of given data, but it is not suitable to indicate the nature of the data distribution.

1 to 3 are diagrams comparing data modeling results using the SVM method and the KRR method. 1 is a diagram showing a data set used in a comparative experiment, FIG. 1A is a diagram showing a data set forming one large circular cluster, and FIG. 1B is another data set forming four small circular clusters. Drawing. FIG. 2 is a diagram illustrating a result of modeling a data set of FIG. 1 using a support vector machine (SVM) method, and FIGS. 2A and 2B are results of modeling the data set of FIG. 1A by different parameters of kernel functions. 2C and 2D are diagrams illustrating the results of modeling the data set of FIG. 1B by using different kernel function parameters. 3 is a diagram illustrating a result of modeling the data set of FIG. 1 using a kernel ridge regression (KRR) method, and FIGS. 3A and 3B illustrate the results of modeling the data set of FIG. 1A by different parameters of kernel functions. 3C and 3D are diagrams illustrating the results of modeling the data set of FIG. 1B by using different kernel function parameters. Through comparison between FIG. 2 and FIG. 3, it can be clearly seen that the data model generated by the SVM method can only represent the boundaries of the data, whereas the data model generated by the KRR method can express the internal distribution of the data. have.

As can be clearly seen through Figures 1 to 3, the KRR method has the advantage that it can represent the internal distribution of the data as compared to the SVM method. However, the KRR method has the disadvantage of including an inverse matrix operation that requires a large amount of computation in the process of finding an optimal transcendental plane. In particular, a large amount of data requires a lot of time and memory space for the inverse matrix operation.

The present invention has been proposed to solve the above problems, after selecting a representative data set from a large amount of data using a genetic algorithm (Genetic Algorithm), and then the selected representative data set Kernel Ridge Regression (KRR) The purpose of the present invention is to provide a method for efficiently modeling large data with a smaller amount of computation by obtaining an optimal transcendental plane by applying the method.

According to a feature of the present invention for achieving the above object, a method of modeling a large amount of data,

(1) a first step of selecting a representative data set that optimally approximates a large amount of total data using a genetic algorithm;

(2) a second step of extracting a feature vector from the representative data set selected in the first step; And

And (3) a third step of applying the feature vector extracted in the second step to a Ridge Regression method to obtain an optimal hyperplane representing a large amount of total data. Features of the jacket.

Preferably, in the third step, the feature vector may be applied to a kernel function to extend into a nonlinear space and then applied to the ridge regression method.

More preferably, in the first step, the number of data belonging to the representative data set may be arbitrarily set.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

4 is a block diagram illustrating respective steps constituting a method for modeling a large amount of data according to an embodiment of the present invention. As shown in FIG. 4, in the large data modeling method 400 according to the present invention, selecting a representative data set 415 by applying a genetic algorithm to the large data 405, selecting the representative data set 415, and selecting the representative data. Extracting the feature vector 425 from the data set 415 and applying the ridge regression method to the extracted feature vector 425 to obtain an optimal transcendental surface 435, i.e., a data model (430). ). In particular, in the step 410 of applying the genetic algorithm, the number of data belonging to the representative data set 415 may be arbitrarily set, thereby applying the accuracy and ridge regression method of the optimal transcendental plane 435 as the final result. In operation 430, a trade-off between the amounts of computation required may be adjusted. In addition, prior to the step 430 of applying the ridge regression method, the feature vector 425 extracted in the feature vector extraction step 420 may be extended to a nonlinear space using a kernel function.

Next, the KRR method to which the genetic algorithm used for the large-capacity data modeling in the present invention will be described in more detail, including the mathematical basis thereof.

One. Ridge  return( Ridge Regression ) Way

(1) Linear Ridge Regression (LRR)

, 레이블과 오차는 각각

과

인 데이터에서, LRR은 다음 수학식 1과 같은 선형 함수 f(x)로 찾을 수 있다.Training data is

, Label and error are respectively

and

In the data, LRR can be found by the linear function f (x) as shown in Equation 1 below.

[Equation 1]

A parameter ( w , b) that minimizes the sum of squared differences of data should be determined, which may be defined as an objective function as shown in Equation 2 below.

[Equation 2]

Equation 2 is expressed by the Lagrangian coefficient method as shown in Equation 3 below.

[Equation 3]

에 대하여 미분하여 정리하면 다음 수학식 4와 같다.Equation 3 is represented by w , b,

Differentiated and summarized as in Equation 4 below.

[Equation 4]

Solving Equation 4 by the dual objective function is as follows.

[Equation 5]

By differentiating Equation 5 with respect to a , a can be determined as shown in Equation 6 below.

[Equation 6]

When the equation 6 is substituted into the decision function f (x) of Equation 1, Equation 7 is obtained.

[Equation 7]

(2) Kernel Ridge Regression (KRR)

을 이용하여 커널 공간으로 사상하는 과정이 추가된다. 상기 수학식 1의 결정함수 f(x)는 커널함수를 이용하여 다음 수학식 8과 같이 다시 적을 수 있다.KRR Kernel Function for Data x in LRR

Mapping into kernel space is added using. The determination function f (x) of Equation 1 may be rewritten as in Equation 8 using the kernel function.

[Equation 8]

는 커널함수

에 의해

로 변환되며, KRR의 목적함수는 다음 수학식 9와 같이 정의될 수 있다.Where training data

Is a kernel function

By

The KRR objective function may be defined as in Equation 9 below.

[Equation 9]

The equation 9 is solved using the Lagrangian coefficient method.

[Equation 10]

에 대하여 미분하여 정리하면 다음 수학식 11과 같다.Equation 10 is represented by w , b,

Differentiated and summarized as in Equation 11 below.

[Equation 11]

If Equation 11 is solved by the dual objective function, Equation 12 is obtained.

[Equation 12]

By differentiating Equation 12 with respect to a , a can be determined as shown in Equation 13.

[Equation 13]

When the equation 13 is substituted for the decision function f (x) of Equation 8, the determination function of KRR is expressed by Equation 14 below.

[Equation 14]

2. Feature Vector

(1) Feature Extraction

A typical web document consists of a set of terms, which can be used as feature vectors. Web documents are largely divided into HTML tags and plain text. After parsing the HTML to remove the tag information, the words included in the plain text are selected as feature vectors. In general, the attribute value of the feature vector may use TF (Term Frequency) and TF-IDF. TF document d _i In is defined as the frequency of the word _{w j, (Inverse Document Frequency)} IDF may be determined by the number n _j of the document d _i including the word w _j. This expression is expressed by the following equation (15).

[Equation 15]

)로 결정된다. 웹 문서의 크기는 서로 다르므로 한 문서 내의 특징 벡터를 단위 길이(unit length)로 정규화(normalization)하여 사용한다.IDF has a higher weight as the number of documents containing a word is smaller. The weight of the word w _j contained in document d _i is TF-IDF (

Is determined. Since web documents have different sizes, feature vectors within a document can be normalized to unit length.

(2) Feature Reduction

A typical web document contains a very large number of words, which can be mapped into higher dimensions. In this case, misspelled words or words with high frequency (pronouns, surveys, etc.) may reduce the search performance. That is, stopwords are removed by removing words that are inappropriate for classification. In addition, the number of feature vectors is reduced by expressing words having the same meaning as the same feature vectors through stemming.

(3) Feature Selection

통계치(Chi-Squared statistics) 등이 있다.Applying all the words extracted from the web document as a feature vector introduces a problem of computation execution time and memory. This problem can be solved by excluding words that do not affect performance and selecting important words. The method for selecting a feature may include document frequency (DF), thresholding, information gain (IG), mutual information (MI),

Chi-Squared statistics.

3. Genetic Algorithm

The optimal hyperplane of KRR can be obtained by calculating a of the Lagrangian coefficient method of Equation (14). If the number of training data is m, in order to obtain a using Equation 14, the inverse of the square matrix of size m must be calculated. There is a problem that the operation execution time and memory usage are very large. As a method to compensate for this shortcoming of the KRR, the present invention uses a genetic algorithm, which is one of the evolutionary algorithms.

5 is a diagram illustrating an example of a basic genetic algorithm. In FIG. 5, population P (t) is composed of n individuals, and each individual is composed of k learning data, that is, a gene. The initial P (t) is initialized by randomly selecting from all the training data. After evaluating P (t) using a fitness function, a new P (t) is selected by performing a process of selection, crossover, and mutation. When selecting a new p (t), the last individual is the highest in the assessment to date. Finally selected P (t) Among the subjects with the best evaluation values, the training data approximates the entire training data.

The evaluation function may be defined as in Equation 16 below.

[Equation 16]

Here, N is a total number of learning data, P _j is a result of evaluating the entire data at the best beyond surface obtained by performing the KRR for each object, it learned that P _j greater approximate the entire training data optimally Means data.

The selection process is probabilistically selected according to the evaluation value among each individual, and crossover and mutation are probabilistically determined according to a predetermined value. If the optimal evaluation value does not change, the genetic algorithm is terminated and the entity with the largest evaluation value is selected as the learning data that best represents the given total learning data.

4. Experimental Results

The experiment was conducted by comparing the results of applying the KRR after selecting the data using the genetic algorithm according to the present invention and comparing the results of applying the KRR by randomly selecting the data. 6 is a diagram illustrating data used in an experiment. As shown in FIG. 6, the data used in the experiment is in the form of chess and is arbitrary data according to a normal distribution. The total number of data is 2000, and the number of noise is 400.

First, the size of a population was set to 10, and the size of an individual constituting each population was set to 50 data (2.5% of the total data). FIG. 7 is a diagram showing an experimental result of KRR in the case where 50 pieces of data are randomly selected from 2000 pieces of data. The white part in the figure means the transcendental surface that best represents the given data, and the darker the darker, the more irrelevant data to the learning data. Looking at Figure 7, it can be seen that the transcendental surface does not appear clearly. FIG. 8 is a diagram showing an experimental result of KRR in a case where 50 data are selected from 2000 total data using a genetic algorithm according to an embodiment of the present invention. In applying the genetic algorithm, the algorithm was performed until the final result converged, and the probability value for the variation was set to 0.3. Comparing the experimental results of FIG. 8 with the experimental results of FIG. 7, it can be seen that the optimal transcendental surface is much more clearly seen. In other words, it can be clearly seen from the experimental results that the method according to the present invention can efficiently model a large amount of data even with a smaller amount of computation.

Next, an experiment was conducted in which the size of the individuals constituting each population was set to 20 data (1% of the total data). FIG. 9 is a diagram showing an experimental result of KRR in the case where 20 data are selected from 2000 total data using a genetic algorithm according to an embodiment of the present invention. The set value for the genetic algorithm was set the same as in the previous experiment. Comparing the experimental result of FIG. 9 with the experimental result of FIG. 8, it can be seen that the white part is slightly blurred. This means that the optimal transcendental surface is a little less pronounced, ie, the data modeling is a little less pronounced, which is a natural consequence of the reduced amount of data used in KRR, and in return, the amount of computation for inverse computation is much reduced. do. 8 and 9, it can be seen that the user can arbitrarily adjust the trade-off between the accuracy of data modeling and the required amount of calculation (memory size) by arbitrarily setting the size of the objects constituting the population. .

The present invention described above may be variously modified or applied by those skilled in the art, and the scope of the technical idea according to the present invention should be defined by the following claims.

According to the method for modeling a large amount of data according to the present invention, a representative data set is selected from the large amount of data using a genetic algorithm, and the selected representative data set is applied to a kernel ridge regression (KRR) method. By finding the optimal transcendental plane, we can effectively model large amounts of data with smaller amounts of computation.

Claims (3)

Selecting a representative data set for optimally approximating a large amount of total data using a genetic algorithm; A second step of extracting a feature vector from the representative data set selected in the first step; And A third step of obtaining an optimal hyperplane representative of a large amount of data by applying the feature vector extracted in the second step to a Ridge Regression method; Way. The method of claim 1, In the third step, the feature vector is applied to a kernel function (kernel function) to expand to a nonlinear space, and then to the ridge regression method for modeling a large amount of data characterized in that. The method according to claim 1 or 2, And in the first step, the number of data belonging to the representative data set can be arbitrarily set.

Images