KR101580202B1 - Svm-bta updating apparatus and method for large scale dataset - Google Patents

Abstract

A plurality of SVMs (Support Vector Machines) configured with a binary tree structure are provided to receive input data sets including at least one input data when updating a binary tree structure of a support vector machine for a large- ) Node, and updates the binary tree structure by generating a new SVM node to which the input data branched from any of the detected leaf nodes is to be allocated, The weighting factors are calculated based on the depth and the class size in the binary tree structure and the leaf nodes to be branched out are selected based on the weighting coefficients.

Description

Technical Field [0001] The present invention relates to a support vector machine binary tree structure update apparatus and method for large-

The present invention relates to an apparatus and method for updating a support vector machine-binary tree architecture (SVM-BTA) of a binary tree structure useful for large-scale data classification such as large-scale face recognition.

SVM (Support Vector Machine) is a classification technique that performs very well because it guarantees a global optimum solution of a given problem, and is applied to pattern classification and function approximation.

Basically, since SVM is a binary classifier, a separate construction method and a learning method therefor are required in order to apply to a multi-class classification problem such as a face recognition system. Examples of such SVM schemes include SVMs for multi-class classification problems such as one-to-many, one-to-one voting, and one-on-one tournament methods. Conventional SVM schemes provide high recognition rates in facial recognition, It is not suitable for the real-time recognition field in which real-time recognition work must be performed in a large-capacity database.

Korean Patent Laid-Open Publication No. 2013-0067758 (entitled " Person Detection Apparatus and Method Using SVM Learning ") relates to a technique for detecting a person from an input image acquired through a camera, A histogram of the learning binary feature patterns extracted from the learning image is generated through the analysis of the change, and a learning feature vector is generated based on the histogram, and the generated learning feature vector is SVM learned to generate a classifier, A histogram is generated for input binary feature patterns extracted from the input image, an input feature vector is generated based on the histogram, and an input feature vector is generated for the entire input The search area for the image is set, and the set search area and the database There is a configuration that detects a person is disclosed by using a classifier stored.

SVM with binary tree structure (BTA) has been proposed to solve the structural limitations of SVM. Due to the nature of the binary tree structure, SVM-BTA is suitable for real-time recognition system field because it can guarantee faster and higher recognition rate such as existing one-to-one voting method and one-on-one tournament method.

However, due to the structural nature of SVM-BTA, which constructs a hierarchical tree with clustering algorithms such as the K-Means algorithm and performs pattern recognition by learning SVMs present in each node, considerable time is required to reconstruct the BTA when adding new data And it takes a lot of time to re-learn SVMs existing in each node, which makes it difficult to maintain the system.

Therefore, a faster and more accurate SVM-BTA update method is required to operate SVM-BTA-based classification processing in systems where databases are dynamically changing.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: FIG.

Also, an embodiment of the present invention is to provide an apparatus and method for updating a binary tree structure of a support vector machine for parallel learning processing of a plurality of SVMs.

It should be understood, however, that the technical scope of the present invention is not limited to the above-described technical problems, and other technical problems may exist.

According to an aspect of the present invention, there is provided an apparatus for updating a binary tree structure of a support vector machine, comprising: a data input unit receiving an input data set including at least one input data; And a plurality of SVM (Support Vector Machine) nodes constituted by a binary tree structure, and allocating the input data branched from any one of the detected leaf nodes to a leaf node And a binary tree structure update unit for updating the binary tree structure by generating a new SVM node to be updated based on the depth and class size of the binary tree structure for each of the plurality of SVM nodes, Calculates a weighting coefficient, and selects the leaf node to be branched out of the detected leaf nodes based on the weighting coefficient.

According to another aspect of the present invention, a method for updating a binary tree structure of a support vector machine via a binary tree structure update apparatus of a support vector machine includes the steps of: receiving an input data set including at least one input data; Detecting at least one leaf node among a plurality of SVM (Support Vector Machine) nodes constituted by a binary tree structure; And generating a new SVM node to which the input data branched from one of the detected leaf nodes is to be allocated to update the binary tree structure, wherein the updating of the binary tree structure comprises: Calculating a weighting coefficient based on the depth and the class size in the binary tree structure for each node, and selecting the branching leaf node among the detected leaf nodes based on the weighting coefficient.

According to the above-mentioned object of the present invention, the SVM-BTA can be quickly and accurately updated in pattern recognition of large-scale data such as real-time large-scale face recognition.

In addition, according to the task of the present invention, it is possible to process a fast BTA update time using the SVM and the weighting coefficient that have already been learned by using the structural characteristics of the SVM-BTA. Also, by inducing a full binary tree with fast BTA update, a high recognition rate can be maintained.

Further, according to the task solution of the present invention, the SVM learning time can be effectively reduced by parallel learning processing of a plurality of SVMs. In addition, it is possible to re-interpret the parallel learning problem of SVM-BTA as an advanced backpack problem, and to efficiently schedule the workload to be re-learned based on the learning execution time, thereby providing a time and spatially optimized parallel SVM learning result .

1 is a block diagram showing a configuration of an apparatus for updating a binary tree structure of a support vector machine according to an embodiment of the present invention.
FIG. 2 is a diagram for explaining a binary tree structure (BTA) applied to an embodiment of the present invention.
3 is a flowchart illustrating a method of updating a binary tree structure according to an embodiment of the present invention.
4 is a diagram for explaining an initial classification process of an input data set according to an embodiment of the present invention.
5 and 6 are diagrams for explaining a leaf node selecting method according to a weighting coefficient according to an embodiment of the present invention.
7 is a flowchart illustrating an SVM parallel processing method according to an embodiment of the present invention.
8 is a diagram for explaining a workload allocation method for an SVM processor according to an embodiment of the present invention.
9 is a view for explaining a sorting method of a workload order for an SVM learning process according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as "comprising ", it means that it can include other elements as well, without departing from the other elements unless specifically stated otherwise.

Hereinafter, a network to which the present invention is applied will be described as a data center network. However, the network to which the present invention is applied is not limited to the data sensor network, but may be applied to various networks in which data traffic is generated by connecting at least one of wired and wireless links between a plurality of node devices.

1 is a block diagram showing a configuration of an apparatus for updating a binary tree structure of a support vector machine according to an embodiment of the present invention. 2 is a diagram for explaining a binary tree structure (BTA) applied to an embodiment of the present invention.

1, an apparatus 100 for updating a binary tree structure of a support vector machine according to an embodiment of the present invention includes a data input unit 110, a binary tree structure update unit 120, a parallel learning control unit 130, an S-VAM learning processing unit 140, and a database 150.

The data input unit 110 receives an input data set including at least one new input data.

At this time, in an embodiment of the present invention, the binary tree structure update apparatus 10 of the support vector machine processes real-time face recognition, and the input data set is face image data for face recognition.

2, the apparatus for updating a binary tree structure 100 of a support vector machine according to an embodiment of the present invention classifies large-scale data using a plurality of SVM nodes configured in a binary tree structure.

2, two nodes (SVM 2 and SVM 3) are configured as a lower layer of the SVM 1 node, which is the root node and the uppermost node, in the embodiment of the present invention. SVM 3 has a plurality of child nodes as parent nodes. SVM 8 is configured as the lowest node. In FIG. 2, SVM 5, SVM 6, and SVM 10 are included as leaf nodes that are not the lowest node but are not the parent node (i.e., there are no child nodes).

Referring back to FIG. 1, the binary tree structure update unit 120 detects a leaf node that is not the lowest node among a plurality of SVM (Support Vector Machine) nodes constituted by a Binary Tree Architecture (BTA) And branching at a leaf node of the detected leaf nodes generates a new SVM node to which new input data is to be allocated. At this time, the binary tree structure update unit 120 repeats the above procedures until all the input data included in the input data set is allocated, and when all the input data is allocated to the SVM, And updates the binary tree structure.

In addition, the binary tree structure update unit 120 calculates a predetermined weight factor for each of the learned SVM nodes, and determines a leaf node to which new input data is to be allocated based on the calculated weighting factor . For reference, SVM-BTA has few child nodes, and the number of SVM classifiers that need to be updated is smaller, and it has little effect on degradation of recognition rate. Accordingly, the binary tree structure update unit 120 directs the binary tree structure update to the full binary tree so that there is no leaning in updating the SVM-BTA, thereby preventing the degradation of the recognition rate.

Specifically, the method for updating the binary tree structure update unit 120 will be described in detail with reference to FIG.

3 is a flowchart illustrating a method of updating a binary tree structure according to an embodiment of the present invention.

First, when a new input data set is generated (S310), new input data is classified through the SVM of the highest node and input data is allocated to an appropriate class (S320).

For example, FIG. 4 illustrates an initial classification process of an input data set according to an embodiment of the present invention. That is, as shown in FIG. 4, it is determined whether to branch to the left cluster G1 or the right cluster G2 through the SVM in the root node, and branch the branch. For reference, the new input data and the centroid and normalized Euclidean distances of the two largest clusters of the top node are measured, and a closer candidate is adopted as the candidate group based on the measured distance, Can be assigned to the cluster.

Next, predetermined weighting coefficients are calculated for each SVM node in the binary tree structure (S330).

Specifically, the weighting coefficient is calculated by the following equation (1).

&Quot; (1) "

According to Equation (1), the weighting factor has a value ranging from 0 to 1, and when it is 0, it means a parent node, and when it is 1, it means a lowest node.

Then, based on the calculated weighting factors for each SVM node, the leaf node to which the input data is to be finally branched is selected (S340), and an SVM node to which the input data is finally allocated is generated by branching at the selected SVM leaf node ).

For example, FIGS. 5 and 6 are diagrams for explaining a leaf node selecting method according to a weighting coefficient according to an embodiment of the present invention.

In this case, if the input data is classified into the left cluster in FIG. 4, the weighting factors of the respective SVM nodes may be as shown in FIG. If the value of the weighting coefficient is not 0 or 1, the node having the largest value branches.

5 (a), when the leaf node having the same weighting coefficient value as the leaf node is present, as shown in FIG. 5 (b), input is performed through the parent node of the corresponding node It is possible to classify the belonging class of data and determine the leaf node to be finally branched. 5 (a), when the leaf nodes SVM 5 and SVM 6 having the same weighting factor of 0.8 exist, the leaf nodes SVM 5 and SVM 6 branch to the left from the highest node SVM 1 and finally branches to the SVM 5 leaf node, 13, and then assigns the input data.

In FIG. 5, it is explained that the class of the input data belongs to the left cluster at the highest node, but it is also possible to determine the belonging class of the input data as the right cluster at the highest node as shown in FIG. In FIG. 6, it is shown that the SVM10 leaf node having the largest weighting factor 0.75 is finally branched.

5 and 6, the input data D1 is allocated to the newly created SVM node (denoted by 'SVM 13' in FIG. 5 and FIG. 6) by finally branching at the leaf node selected by the weighting coefficient.

Thus, the steps from S320 to S350 are repeated until all the input data of the new input data set is allocated. Then, when the SVM generation for allocating all the input data is completed, the binary tree structure of the corresponding state is determined and the binary tree structure is updated (S360).

For example, a series of processes described above may be performed by the binary tree structure update unit 120, as shown in Table 1 below, when the input data set is generated, a pseudo code code.

<Table 1>

Referring back to FIG. 1, the parallel learning control unit 130 controls the SVM learning processing unit 140 to control the SVM nodes in the updated state of the binary tree structure to be learned in parallel.

The SVM learning processing unit 140 includes a plurality of learning processors (not shown) each processing learning of a plurality of SVM nodes. For example, the SVM learning processor 140 according to an exemplary embodiment of the present invention may be a multicore processor, and each learning processor may be a core. The SVM learning processing unit 140 processes the workload (that is, the data size of the SVM node to be processed) for each learning processor (not shown) under the control of the parallel learning control unit 130. [

The database 150 stores previously learned input data sets, and the newly input data sets are also stored in the database 150 after the learning processing through the AV learning processing unit 140. At this time, the database 150 provides input data of SVM nodes (i.e., SVM nodes requiring new learning and re-learning) required by the S / V learning processing unit 140.

Hereinafter, a method for controlling the parallel learning of the SVM nodes by the parallel learning control unit 130 will be described in detail with reference to FIG.

7 is a flowchart illustrating an SVM parallel processing method according to an embodiment of the present invention.

According to the structural characteristics of SVM-BTA, SVMs existing in each node can be learned independently without dependency with other SVMs. Accordingly, the parallel learning control unit 130 allocates workloads of SVMs to be learned in a plurality of learning processors (not shown) of the S / V learning processing unit 130, and controls the parallel processing.

That is, the learning execution time of each SVM node to which the updated binary tree structure is applied is estimated according to the generation of the new input data set through the binary tree structure update unit 120 (S710).

In this case, since the sizes of the data sets received by the SVM nodes are different from each other, the parallel learning control unit 130 calculates the amount of resources (Memory Resource) required for each learning processor and the training time ) As much as possible.

That is, the workloads are equally allocated to the plurality of learning processors based on the data size and the learning execution time for each SVM node (S720).

As a reference, it is a variation of the traditional Knapsack Problem which contains the items to be most profitable in a single backpack. It is a variant of the Knapsack Problem, in which the Data Scale and the Training Time are different, , It is reinterpreted as a problem of distributing the given backpack (the learning processor, Core) so as to have similar volume and weight as possible. That is, in one embodiment of the present invention, the learning problem of the SVM-BTA is regarded as a modified backpack problem, and the same level of workload is assigned to each learning processor.

For example, FIG. 8 illustrates a workload allocation method for an SVM processor according to an embodiment of the present invention.

As shown in FIG. 8, the amount of resources (Memory Resource) required for each learning processor and the training execution time (Training Time) are equally distributed as much as possible. FIG. 8A shows a state in which the workload is not evenly distributed, and FIG. 8B shows a state in which the workload is evenly distributed according to an embodiment of the present invention.

(T₁ = 순차 처리 시간, T_n= n개의 학습 처리기를 통한 병렬 처리 수행 시간)만큼 학습 처리 시간을 단축할 수 있다. 또한, 데이터의 규모와 학습 수행 시간은 거의 선형적인(linear) 특성을 갖는다. 따라서, SVM-BTA 병렬 처리는 각 학습 처리기에서 수행될 SVM 학습 시간이

(이하, ‘최적 학습 수행 시간’이라고 지칭함)이 되도록 작업 부하를 분배하되, SVM의 학습이 진행되는 각 순간마다 전체 필요한 메모리의 양이 최소가 되도록 작업 부하의 우선 순위를 스케줄링하여 최적의 병렬처리 결과를 얻을 수 있다.On the other hand, parallel processing of SVM-BTA is faster than sequential processing

(T ₁ = sequential processing time, T _n = parallel processing execution time through n learning processors). In addition, the size of the data and the execution time are almost linear. Therefore, the SVM-BTA parallel processing is performed by the SVM learning time

(Hereinafter referred to as &quot; optimal learning execution time &quot;), scheduling the priority of the workload so that the total amount of required memory is minimized at each instant of learning of the SVM, Results can be obtained.

That is, the SVM processing sequence for each learning processor is sorted based on the predetermined optimal learning execution time and the learning execution time for each SVM node (S730).

Then, a plurality of SVMs are parallel-learned according to the schedule according to the alignment (S740).

)을 계산한다. 그리고 데이터 셋의 규모를 고려하여, 작업 부하 별로 예상되는 학습 시간을 추정하고, 작업 부하 할당 전략을 통하여 첫 번째 학습 처리기부터 순차적으로 n번 째(단, n는 자연수) 학습 처리기까지 작업 부하 할당을 수행한다.As described above, when the SVM and the data set to be newly learned are inputted, the parallel learning control unit 130 calculates the optimum value to be allocated to each learning processor in consideration of the number of learning processors of the S / V learning processing unit 140 Learning time (

). In addition, considering the size of the dataset, the estimated learning time for each workload is estimated, and the workload allocation from the first learning processor to the n-th learning processor (n is a natural number) .

For reference, the workload allocation strategy performs the following procedure according to the state of each learning processor.

First, if the sum of the estimated time of the workload allocated to the ith learning processor and the estimated time of the newly inputted workload is smaller than the optimal learning execution time, the input workload is allocated to the ith core.

On the other hand, if the sum of the estimated time of the workload allocated to the ith learning processor and the estimated time of the newly inputted workload exceeds the optimal learning execution time, the workload is temporarily allocated to the (i + 1) The total sum is again obtained, and it is repeatedly checked whether it meets the condition of the optimum learning execution time, and assigned to the learning processor which is less than the optimal learning execution time.

In addition, even after confirming that the sum of all the workloads for the available learning processors is the optimum learning performance time, the unallocated workloads are allocated to the learning processor having the smallest sum of the predicted learning times according to the workloads.

Meanwhile, the parallel learning control unit 120 repeats all the processes according to the workload allocation strategy until the workloads are all allocated through the advanced quick sort algorithm. Advanced Quick Sort is designed so that SVM node-specific learning can be terminated at the same time when learning load accumulates in each learning processor.

For example, FIG. 9 is a diagram for explaining an arrangement method of a workload order for an SVM learning process according to an embodiment of the present invention.

As shown in FIG. 9, the parallel learning control unit 120 searches workload combinations having the same size to sort workload orders. If you can not find a workload combination of the same size, use the Quick Sort algorithm to sort the workload.

In this way, the parallel learning control unit 120 predicts the learning execution time from the different data set sizes of the SVMs existing in each node in the binary tree structure, compares the predicted learning execution time with the optimal learning execution time, To the workload. And sort the work order so that the allocated workload ends at the same time and occupies a minimum of resources. Thus, an optimal parallel processing result can be obtained.

That is, the parallel learning control unit 130 predicts the resource size and the predicted learning time required for each SVM node, and equally allocates the data for each SVM node to be processed to each of the plurality of learning processors based on the resource size and the learning predicted time . The parallel learning controller 130 detects a combination of SVM nodes having the same data size among the SVM nodes to which the data is allocated for each of the learning processors, and instructs the SVM node to process the combination of the detected SVM nodes at the same time Sort data by node.

For example, a series of processes described above can be expressed by pseudo code as shown in Table 2 below.

<Table 2>

Embodiments of the present invention may also be embodied in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

100: binary tree structure update apparatus of support vector machine
110: Data input unit
120: Binary tree structure update unit
130: parallel learning control unit
140: S-VEM processing processor
150: Database

Claims (14)

1. A binary tree architecture update apparatus of a support vector machine,
A data input for receiving an input data set including at least one input data; And
A method for detecting at least one leaf node among a plurality of SVM (Support Vector Machine) nodes constituted by a binary tree structure and allocating the input data branched from any one of the detected leaf nodes And a binary tree structure update unit for creating a new SVM node to update the binary tree structure,
Wherein the binary tree structure update unit comprises:
Determining a branching direction of new input data through a learned highest SVM node, calculating a weighting coefficient based on a depth and a class size in the binary tree structure for each of the plurality of SVM nodes, Selecting leaf nodes to branch from among the detected leaf nodes,
Wherein the multiplication value of the class size and the total depth is denominator for each of the SVM nodes and the multiplication value of the number of leaf nodes from the highest SVM node to the corresponding SVM node and the depth of the corresponding SVM node is denominator, Calculates a weighting coefficient,
The leaf node having the highest value of the calculated weighting coefficient among the detected leaf nodes is selected as the leaf node to be branched A binary tree structure update apparatus of a support vector machine. delete The method according to claim 1,
Wherein the binary tree structure update unit comprises:
Judging a belonging class of the input data through a plurality of parent nodes for each leaf node having a highest weighting coefficient value when a plurality of leaf nodes having the highest weighting coefficient value are detected, And selects the leaf node to be branched according to the branching direction determined for the class. The method according to claim 1,
A parallel learning control unit for controlling the SVM nodes of the updated binary tree structure to learn in parallel; And
And a plurality of learning processors each processing learning of the plurality of SVM nodes, wherein the SVM learning processing unit further includes an S / V learning processing unit for performing parallel processing of SVM nodes of the updated binary tree structure under the control of the parallel learning control unit A binary tree structure update device of the support vector machine. 5. The method of claim 4,
Wherein the parallel learning control unit comprises:
A binary tree structure of a support vector machine for predicting a required resource size and a predicted learning time for each SVM node and allocating data for each SVM node to be processed to each of the plurality of learning processors on the basis of the resource size and the predicted learning time Update device. 6. The method of claim 5,
Wherein the parallel learning control unit comprises:
A support vector for sorting SVM node-specific data such that a combination of SVM nodes having the same data size among the SVM nodes to which data is allocated for each learning processor is detected and the learning processor simultaneously processes the combination of the detected SVM nodes at the same time A binary tree structure update mechanism for a machine. The method according to claim 1,
Wherein the input data set is training face image data for face recognition. A method for updating a binary tree structure of a support vector machine via a binary tree structure update apparatus of a support vector machine,
The method comprising: receiving an input data set including at least one input data;
Detecting at least one leaf node among a plurality of SVM (Support Vector Machine) nodes constituted by a binary tree structure; And
Generating a new SVM node to which the input data branched from one of the detected leaf nodes is to be allocated and updating the binary tree structure,
Wherein updating the binary tree structure comprises:
Determining a branching direction of new input data through a learned highest SVM node, calculating a weighting coefficient based on a depth and a class size in the binary tree structure for each of the plurality of SVM nodes, Selecting leaf nodes to branch from among the detected leaf nodes,
Wherein the multiplication value of the class size and the total depth is denominator for each of the SVM nodes and the multiplication value of the number of leaf nodes from the highest SVM node to the corresponding SVM node and the depth of the corresponding SVM node is denominator, Calculates a weighting coefficient,
The leaf node having the highest value of the calculated weighting coefficient among the detected leaf nodes is selected as the leaf node to be branched A method for updating a binary tree structure of a support vector machine. delete 9. The method of claim 8,
Wherein updating the binary tree structure comprises:
Judging a belonging class of the input data through a plurality of parent nodes for each leaf node having a highest weighting coefficient value when a plurality of leaf nodes having the highest weighting coefficient value are detected, And selecting the leaf node to be branched according to the branching direction determined for the class. 9. The method of claim 8,
After updating the binary tree structure,
Further comprising parallel processing of learning of SVM nodes of the updated binary tree structure by a plurality of learning processors each processing learning of a plurality of SVM nodes. 12. The method of claim 11,
Wherein the parallel processing of learning of the SVM nodes comprises:
Estimating a resource size and a predicted learning time required for each SVM node; And
And equally allocating data for each SVM node to be processed to the plurality of learning processors based on the resource size and the predicted learning time. 13. The method of claim 12,
Wherein the parallel processing of learning of the SVM nodes comprises:
After uniformly allocating the data for each SVM node,
Detecting combinations of SVM nodes having the same data size among SVM nodes to which data is allocated for each learning processor; And
Further comprising sorting SVM node-specific data to process the combination of detected SVM nodes together at the same time in the learning processor. 9. The method of claim 8,
Wherein the input data set is training face image data for face recognition.

Images