KR101247307B1 - Data processing method for classifying data, media for writing the method, and data processing apparatus for applying the method - Google Patents

Abstract

The present invention is to provide a data processing method for selecting a feature suitable for classification in the feature extraction process by removing the non-boundary data that degrades the classification performance as a data preprocessing process when data classification using a linear subspace technique It is.

Description

Data processing method for classifying data, a recording medium recording the method, and a data processing apparatus for executing the method, and a data processing apparatus for applying the method.

The present invention provides a data processing method for performing a data preprocessing step of obtaining a transformation matrix for reducing the dimension of data in order to improve the performance of linear feature extraction methods in the field of pattern recognition, a recording medium recording the same and A data processing apparatus for executing the method.

In a typical computer-based classification system, statistical characteristics of a given data are analyzed to determine the type of new data judged to have the same statistical characteristics. In this case, in order to improve the classification performance, it is necessary to process the given data to select the data having high reliability. This process is called preprocessing. This process is a preliminary operation for separating unnecessary information by selecting the partial data to be currently noticed or shaping the data so that the essential information can be extracted from the given input data. This preprocessing includes processing such as data normalization, data selection, and noise data removal. After the data preprocessing process, it is necessary to convert the original data into new data in a lower dimension in order to lower the calculation amount to an appropriate level and improve the quality of the data. The new data obtained for this dimension reduction is called the feature of the original data, and the process of extracting such feature is called feature extraction. Once features are extracted from the data, the type information of the given data is finally determined using a classifier.

What should be considered in the feature extraction for data classification is that information suitable for classifying the data according to the type information must be sufficiently included in the extracted feature. Generally, feature extraction is divided into global method and local method. The method in which all the dimensions included in the original data are considered in the feature extraction is called a global method. The way in which some dimensions of the original data and the geometric relationships between them are taken into account is called a local method.

One widely used global method is the subspace method. The subspace method includes a preprocessing step of processing raw data so as to extract essential information for classification, a step of finding a transformation matrix which is a set of axes for projecting data using the processed data, Projecting and extracting features, and classifying the data using the projected features instead of the raw data.

A representative example of the subspace technique is principal component analysis (PCA), which is a multivariate data analysis technique in which variables are correlated to reduce dimensions. In other words, it is a statistical technique that reduces the loss of information contained in the multidimensional data to be interpreted into low-dimensional data while reducing the loss of information contained therein. .

Principal component analysis finds a set of axes maximizing the variance included in each dimension of the data, projects the data onto the axes included therein, and uses the projected values as features instead of the original image data.

However, features extracted through principal component analysis are often not suitable for use in classification systems. The characteristics obtained through the principal component analysis do not consider the classification performance of the original data, and thus there is a problem that the classification performance decreases due to the reduction of the dimension. In particular, the boundary data at the decision boundary is often mixed, and when the linear subspace technique is used, the transformation matrix unsuitable for classification may be obtained due to the influence of the boundary data.

To solve this drawback, linear discriminant analysis has been proposed. The linear discriminant analysis comprises the steps of: finding an axis maximizing defined scatter to enhance classification performance; Projecting data onto the found axes; And using the projected value as a feature instead of the original image.

This linear discriminant analysis is a technique based on principal component analysis. Instead of maximizing variance, it finds the axis that maximizes the defined scatter and improves the classification performance.

The scatter matrix, which represents scattering, consists of the ratio of between-class variance between the data that should be classified into the same class and between-class variance of the mean of those data. Use this to find the axis that maximizes the variance between the data that should be classified differently.

The features obtained through linear discriminant analysis show higher classification performance with less or equal dimensions than those obtained through principal component analysis. However, even in the linear discriminant analysis, since the data at the decision boundary is mixed with data having different kinds of information, the transformation matrix unsuitable for classification may be obtained due to the influence of the boundary data.

The present invention has been proposed to solve the above-described conventional problems and other various problems. In the case of using the linear subspace technique by removing some data that degrade classification performance by using type information in the data preprocessing process, It is an object of the present invention to provide a data processing method for selecting a feature suitable for classification in a feature extraction process.

The present invention provides a data processing method for data classification, comprising: deriving non-boundary data from boundary data from boundary data, and deriving a transformation matrix using the non-boundary data.

 In the data processing method, distinguishing the non-boundary data from a plurality of data is to derive a probability according to the type of at least one data included in one region of the plurality of data, and to use the probability The non-boundary data may be derived by dividing the plurality of data into boundary data and non-boundary data using the disorder.

In the data processing method, the probability may be a probability of how much data includes the same kind of data as the i th data among k data included in the i th data and one region around the i th data.

In the data processing method, the disorder indicates how much different kinds of data are included in the one region.

In the data processing method, dividing the plurality of data into boundary data and non-boundary data may determine whether the disorder of the i-th data is greater than a reference value. If the first data is boundary data and the disorder is not greater than the reference value, the i-th data may include dividing the data into non-boundary data.

The data processing method may further include extracting a feature of data by projecting the plurality of data onto the transformation matrix, and classifying the plurality of data by using the extracted feature of the data.

The present invention also provides a recording medium on which the data processing methods are recorded.

The present invention also provides a data processing apparatus including a data derivation unit for deriving non-boundary data from boundary data from boundary data and a transformation matrix derivation unit for deriving a transformation matrix using the non-boundary data. .

In the data processing apparatus, the data derivation unit may include a probability derivation unit deriving a probability according to a type of at least one data included in one region of the plurality of data, and a disorder using the probability to derive a disorder figure. A derivation unit and a non-boundary data derivation unit for deriving distinction of non-boundary data from boundary data using the disordered diagram may be provided.

In the data processing apparatus, the non-boundary data deriving unit includes a determining unit that determines whether the disorder of any one of the plurality of data is greater than a reference value, and if the disorder is greater than a reference value, the data. Is boundary data and if the disorder is not greater than a reference value, the data may include a control unit that divides the data into non-boundary data.

The data processing apparatus may further include a feature extractor configured to extract a feature by projecting the plurality of data onto the transformation matrix, and a classifier that classifies the data using the feature extracted by the feature extractor.

As described above, the present invention can improve the classification performance by extracting features more bonded to the classification using non-boundary data except boundary data as a preprocessing method for data classification.

Hereinafter, a data processing method according to the present invention, a recording medium recording the same, and a data processing apparatus performed by the above method will be described with reference to the accompanying drawings.

1 is a flowchart illustrating a data processing method according to the present invention.

First, at least two pieces of a plurality of data are collected. Define an area around the data before or after data collection and sorting. The one area is a local area around the data and may be determined manually or automatically by a user or the like. The one region may be defined as in Equation 1 below.

Where Ν (x _i, k) is a set of k pieces of data on the x _i data with a predetermined distance, and determines the work area including the x _i data. The size of the one region may be determined by k value.

When the one region is determined, a probability according to the type of data included in the one region is derived (S11). For example, the probability indicates how much of the k-th data included in the i-th data and one region around the i-th data includes the same kind of data as the i-th data including the i-th data. The probability may be derived in the same manner as in Equation 2 below.

Where x _i is the value data I j (n) n-th-up data having the same type of information and data in the data x _i is 1, or if the value is determined to zero. j means kind information. That is, P _j (x _i ) means a probability value that x _i data belongs to the j th type.

For if example, with reference to FIG. 2 x _i data ○ has the type information, the x _i when to calculate the probability of having the same ○ type information and the x _i data in an area containing the data P _j (x _i ) Is 3/5. Referring to FIG. 3, the probability P _j (x _{i + 1} ) having the same type information as the x _{i + 1} data which is the i + 1 th data is 1.

The probability is used to derive an entropy in the one region (S12). The disorder is an indication of how different kinds of data are included in the one area. It can be derived according to the following equation (3).

Here, Nc means the number of data types.

If the disorder chart is calculated using Equation 3 with respect to the x _i data shown in FIG. 2, it has a value greater than zero. However, the disorder of the x _{i +1} data shown in FIG. 3 has zero. The larger the degree of chaos, the more likely that the data will be located in the boundary area with the set of data having different kinds of information.

Therefore, it is determined whether the derived disorder is greater than a reference value (S13).

The reference value is a value determined based on empirical rules, and may be appropriately determined in consideration of a variable such as noise data included in the one region.

The determination may be represented by Equation 4 below.

Where Xb is a set of boundary data, and θ ( k , Nc ) is a reference threshold predetermined by the user, and is determined as an appropriate value according to the number and size of the proximal region according to the determination of k value as described above. Can be.

It is determined that x _i if not larger than the disorder is also a reference value of a data x _i data is a non-boundary data, and (S14), x disorders of _i data is greater than the reference value x _i data can be classified into boundary data ( S15).

In operation S16, it may be determined whether the x _i data is the last data. If it is not the last data, the next data is selected (S17), and the probabilities for the next data, the derivation of the disorder, and the subsequent steps are performed.

If it is the last data, a transformation matrix to be used for feature extraction is derived using non-boundary data separated by the determination S13 (S18).

The transformation matrix derivation may be performed in various ways, and for example, the transformation matrix may be generated by Equation 5 or 6 below.

S ^(nb) is a covariance matrix of non-boundary data, and W _PCA ^(nb) is a transformation matrix representing a set of axes to project data.

Here, S _b ^{( nb )} means the scattering matrix between types of non-boundary data, and S _w ^{( nb )} means the scattering matrix within the type of non-boundary data. . In addition, W _LDA ^(nb) is a transformation matrix representing a set of axes on which data is projected.

The final feature is extracted by projecting the collected data on the derived transformation matrix (S19). Equation 7 may be used for feature extraction.

F is a transformation matrix representing the set of axes on which to project data.

In such a feature set, data may be classified using a classifier. As described above, the transform matrix may be generated using only non-boundary data except boundary data before feature extraction, and the data may be effectively classified by extracting the feature of the data from the transform matrix.

The present invention includes a recording medium recording the above method, for example, a computer readable recording medium such as an optical disc, a magnetic disc, a hard disc.

The present invention also includes a data processing apparatus for distinguishing non-boundary data from collected data, generating a transformation matrix using the non-boundary data, and then extracting a feature and classifying the data using the transformation matrix. . Specifically, this will be described with reference to FIG. 2.

According to FIG. 2, the data processing apparatus according to the present invention includes a data preprocessor 100. The data preprocessor 100 includes a data derivation unit 110 for deriving non-boundary data and a transformation matrix derivation unit 120 for generating a transformation matrix using the non-boundary data.

In addition, the data processing apparatus may include a feature extractor 200 for extracting features of data using the transformation matrix and a classifier 300 for classifying data from the features.

The data derivation unit 110 will be described in more detail with reference to FIG. 3.

As shown in FIG. 3, the data derivation unit 110 may include a probability derivation unit 111, a chaotic derivation unit 112, and a non-boundary data derivation unit 113.

The probability deriving unit 111 derives a probability according to the type of the data in one region around the selected data. Specifically, the probability here relates to how much data of the same kind as the i-th data is included among the i-th data and k data included in one region around the i-th data. For example, the method may be performed in the same manner as in Equation 2 above. Here, the one area may be preset by a user or the like.

The disorder degree deriving unit 112 derives the disorder degree which is a degree of how different kinds of data are included in one region by using the probability. The disorder can be calculated by, for example, Equation 3 described above.

The non-boundary data deriving unit 113 may derive the non-boundary data from the boundary data by using the disorder. In detail, the non-boundary data deriving unit 113 determines whether the disorder of the data is greater than the reference value, and if the disorder of the data is greater than the reference value, the data is boundary data and the disorder is Is not greater than the reference value, the data may include a controller 113b that divides the data into non-boundary data.

The transformation matrix is derived by using the non-boundary data extracted by the data deriving unit 110 as described above except for boundary data. This is performed by the transformation matrix derivation unit 120. Therefore, in the present invention, the data preprocessing unit 100 derives a transformation matrix using non-boundary data.

The feature is extracted by projecting the data collected on the transform matrix derived from the feature extractor 200, and the data is classified by the classifier 300 using the feature.

Hereinafter, an embodiment in which the data preprocessing effect is evaluated in the data processing method according to the present invention will be described.

First, FIGS. 6 and 7 are graphs showing results of extracting and classifying features by principal component analysis on data collected without data pretreatment as in the present invention as a control.

8 and 9 are graphs illustrating a result of classification after generating a transformation matrix by using non-boundary data, extracting a feature using the transformation matrix, according to the data processing method of the present invention.

The control group collected data of a and b and extracted the collected data using conventional principal component analysis. According to FIG. 6, a vertical straight axis was extracted as an axis for feature extraction. Accordingly, as a result of projecting all data on the axis, referring to FIG. 7, it is difficult to distinguish between a data and b data.

 On the other hand, after collecting different types of c data and d data, and selectively extracting non-boundary data from the collected data to derive a transformation matrix, the transformation matrix was derived. According to FIG. 8, a horizontal linear axis was extracted as an axis for feature extraction. When all the data are projected on the axis, as shown in FIG. 9, it can be seen that c and d data exist in a state where they are easier to classify.

In addition, description will be given of evaluating classification performance with respect to the data processing method of the present invention.

Classification experiments were performed using various classification databases published in the UCI Machine Learning Depository shown in FIG. 10.

Two types of cross validation were performed for performance comparison. First, we extracted the features of the data set by excluding the data from one set of data, learned the classifier, and recorded whether the data that were excluded were correctly classified. This process was repeated for each of the data, and the cross validation of the whole was performed and the classification performance was confirmed.

Next, we divided the dataset into 10 subsets, extracted features using the data from the 9 subsets, trained the classifier, and recorded whether they correctly classify the data in the other subset. As a classifier, a neighborhood neighborhood classifier was used.

The classification performance obtained from the public classification data was compared with the method of classifying the data according to the principal component analysis and the linear discrimination analysis without applying the data preprocessing method provided by the present invention. Here, data preprocessing is to extract non-boundary data from the collected data and derive a transformation matrix using the non-boundary data.

11 and 12 are results obtained by comparing the performance of the data preprocessing to the principal component analysis (RPS + PCA: e) and the performance of the conventional principal component analysis (PCA: f) according to the data processing method of the present invention. . 11 shows the result of using the first cross-valuation method, and FIG. 12 shows the result of using the second cross-valuation method.

FIG. 13 and FIG. 14 show the results obtained by applying data preprocessing to the linear discrimination analysis (RPS + LDA: g) according to the data processing method of the present invention and comparing the performance with the conventional linear discrimination analysis (LDA: h). . FIG. 13 shows the result of using the first cross-evaluation method, and FIG. 14 shows the result of using the second cross-evaluation method.

11 to 14, it can be seen that the classification performance is better when applying the data processing method proposed in the present invention.

As such, the present invention is to improve the classification performance of subspace techniques such as principal component analysis and linear discriminant analysis by performing data preprocessing using the type information for classification.

While the present invention has been described in connection with certain exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the invention. Therefore, the present invention can be applied to modify these embodiments as appropriate, it will be natural that such applications also fall within the scope of the present invention based on the technical idea described in the claims.

1 is a flowchart illustrating a data processing method according to the present invention.

2 and 3 are diagrams for explaining a step of deriving probability and disorder for the i th data and the i + 1 th data according to the data processing method of the present invention.

4 and 5 are block diagrams for explaining a data processing apparatus according to the present invention.

6 is a graph showing an axis on which data is projected using principal component analysis.

FIG. 7 is a graph illustrating distribution of data having different type information when the data is projected on the axis illustrated in FIG. 6.

8 is a graph showing an axis to project data using principal component analysis after data preprocessing according to the present invention.

FIG. 9 is a graph illustrating distribution of data having different type information when data is projected on the axis illustrated in FIG. 8.

10 is a diagram showing a UCI Machine Learning database used for performance evaluation of the present invention.

11 to 14 are graphs comparing the classification performance according to the data processing method and the conventional data classification method for data classification according to the present invention using the database shown in FIG.

Claims (11)

A probability according to a type is derived for at least one data included in one region among a plurality of data, a disorder chart is derived using the probability, and the disorder chart is used to compare the plurality of data with boundary data. Deriving the non-boundary data by dividing by boundary data; And Deriving a transformation matrix representing a set of axes to project the data using the covariance matrix of the non-boundary data. delete Claim 3 has been abandoned due to the setting registration fee. The method of claim 1, wherein the probability is a probability of how much data includes the same kind of data as the i-th data among the k-th data included in the i-th data and one region around the i-th data. Data processing method for data classification. Claim 4 has been abandoned due to the setting registration fee. The method of claim 1, wherein the plurality of data are divided into boundary data and non-boundary data. Determining whether the disorder of the i-th data of the plurality of data is greater than a reference value; And if the disorder is greater than the reference value, the i-th data is boundary data, and if the disorder is not greater than the reference value, the i-th data is divided into non-boundary data. How data is processed. Claim 6 has been abandoned due to the setting registration fee. The method of claim 1, wherein the deriving of the transformation matrix comprises: And a transformation matrix representing a set of axes on which the data is to be projected, by using the dispersion matrix between the types of the non-boundary data and the distribution matrix within the type of the non-boundary data. A recording medium on which the data processing method of any one of claims 1 to 3 is recorded. A probability derivation unit for deriving probabilities according to types of at least one data included in one region of the plurality of data; A chaotic derivation unit for deriving a chaotic diagram using the probability; A non-boundary data derivation unit for deriving distinction of non-boundary data from boundary data using the disorder diagram; And And a transformation matrix derivation unit for deriving a transformation matrix representing a set of axes to project the data using the covariance matrix of the non-boundary data. delete The method of claim 8, wherein the non-boundary data derivation unit, Determination unit for determining whether the disorder of any one of the plurality of data is greater than the reference value; And The data is boundary data if the disorder is greater than the reference value, and the controller divides the data into non-boundary data if the disorder is not greater than the reference value. Processing unit. Claim 11 was abandoned when the registration fee was paid. The method of claim 8, wherein the conversion matrix derivation unit, And a transformation matrix representing a set of axes on which the data is to be projected by using a dispersion matrix between the types of non-boundary data and a dispersion matrix within the type of the non-boundary data.

Images