KR20150025508A - Multi-view object detection method using shared local features - Google Patents

Abstract

The present invention relates to a multi-view object detection method using shared local features and, more specifically, to a multi-view object detection method using shared local features including the steps of: generating randomly N regions which have a random location and a random size in a training image with a multi-view including an object; extracting orientation center symmetric local binary patterns (OCS-LBP); learning feature vectors extracted by each N region through random forest based on a region; and selecting the region of local features.

Description

[0001] MULTI-VIEW OBJECT DETECTION METHOD USING SHARED LOCAL FEATURES [

The present invention relates to a method of detecting a multi-view object using a shared area feature, and more particularly, to a method of detecting a multi-view object using a local OCS-LBP (Orientation Center Symmetric Local Binary Patterns) The present invention relates to a method of detecting a multi-view object using a shared area feature that enables fast and accurate detection of an object having various viewpoints in an image using a forest classifier.

In general, object detection system is one of the important research fields in computer vision such as surveillance system, behavior recognition, and augmented reality. Object detection in a general image has a problem that it is difficult to detect an object by various sizes of objects, complex backgrounds, and the like. In order to solve this problem, active research is being conducted related to object detection.

Prior art 1 (Korean Patent Laid-Open Publication No. 10-2011-0131727, titled "Object Recognition Method in Image Processing System") and prior art 2 (CH Chang, CC Wang, and Reference is made to JJ Lien, " Multi-view vehicle detection using gentle boost with sharing HOG feature, " in Proceedings of 22nd IPPR Conference on Computer Vision, Graphics and Image Processing (IPPR, 2009), pp.1688-1694) There is a bar. Prior Art 1 discloses a method of recognizing an object using a global feature-based method and a region feature-based method through feature extraction using a Sobel filter and a GIST algorithm in a still image. Prior Art 2 discloses an algorithm for generating a patch for an image including an object, extracting a histogram of oriented gradient (HOG) characteristic for the patch, and learning and object detection using a Gentle Boost after feature extraction . The algorithm used in Prior Art 2 uses HOG as a feature extraction method, and HOG has a problem that it has a very slow operation speed because it has many feature dimension numbers.

The present invention has been proposed in order to solve the above-mentioned problems of the previously proposed methods. The present invention proposes a method of extracting local OCS-LBP features for objects having various viewpoints in a still image and learning and learning using a random- The object of the present invention is to provide a method of detecting a multi-view object using a shared area feature, which enables quick and accurate detection of multi-view objects having various views in an image.

In addition, the present invention uses OCS-LBP feature extraction so that it can exhibit very fast performance in spite of a small number of characteristic dimensions including direction information, and it is possible to classify all regions generated using a random forest classifier into object classification A multi-view object using a shared area feature, which enables fast learning speed and detection speed for objects having various viewpoints even if less training data is used, by learning and classifying by using optimal shared area features without using It is still another object to provide a detection method.

In addition, the present invention is capable of detecting multi-point objects even in complex backgrounds (backgrounds) with partial occlusion using shared area features, and in particular sharing shared area features rather than being independent for each class Another object of the present invention is to provide a method of detecting a multi-view object using shared area features, which can minimize the number of classifiers and reduce the complexity of calculation.

According to an aspect of the present invention, there is provided a method of detecting a multi-view object using a shared area feature,

(1) generating candidate regions (Randomly N Regions) having arbitrary positions and sizes in a training image having multiple views including an object;

(2) extracting an OCS-LBP (Orientation Center Symmetric Local Binary Patterns) characteristic for each candidate region;

(3) learning region-based feature vectors extracted for each candidate region through a random forest;

(4) applying a test image to the learned random forest to estimate a probability value, and then selecting an area of Local Features having higher values;

(5) finding and selecting a shared region for a region of the selected region feature for each viewpoint, and learning the regions of the shared region feature in a random forest;

(6) generating a probability value histogram by estimating a probability value for each view when a test image is input to detect an object after performing a training process;

(7) Finding the maximum value among the probability values estimated in the generated probability value histogram, estimating the final probability value by applying a weight to the five surrounding probability values having the maximum value, and linearly combining the weight values; And

(8) detecting as an object when the estimated final probability value is larger than a predetermined threshold value.

Preferably, the candidate regions in step (1)

And N pieces of training samples collected in the window region adjusted by adjusting the size of 80 × 40 including the object in the original image of the training image.

More preferably, the candidate regions include,

It is possible to randomly generate a local feature having a size of at least 40 × 20 (half of the window area) from a size of at least 10 × 10 in the window area having the size of 80 × 40.

Preferably, the candidate regions in step (1)

It is possible to specify 12 training sets in the range of 0 ° to 360 ° at intervals of 30 ° for the local features of the rotating object necessary for multi-point object detection.

More preferably, the training set comprises:

A learning group that selects a specific location and size of a local feature, and a test group.

Preferably,

The negative set can be extracted from the background (background) area.

Preferably, in the step (2)

For each of the candidate regions, it is possible to divide into 4 (2 x 2) sub-regions and to extract 8-dimensional OCS-LBP features in each sub-region.

More preferably, the candidate regions include,

By combining the extracted OCS-LBP features in each sub-region, it is possible to extract 32 (8 × 4) dimensional feature vectors for each candidate region.

Preferably, the random forest in step (3)

And an ensemble classifier based on a decision tree.

More preferably,

Decision trees can be used to handle high training speeds and large amounts of data.

More preferably,

The structure of each tree can be implemented in a binary top-down fashion.

Preferably, the random forest in step (5)

You can train a shared classifier using training data collected from each view class that shares the same local characteristics to create a multi-class classifier for shared area features.

More preferably, in the step (6)

A probability histogram can be generated from the histogram probability distribution of a multi-class classifier shared by each shared classifier.

Even more preferably, the maximum value in the step (7)

The maximum function (Max Operation) for finding the most basic point in the generated probability value histogram can be used.

According to the multi-viewpoint object detection method using the shared area feature proposed in the present invention, the local OCS-LBP feature extraction and the shared area based random forest classifier are used for learning and classifying the objects having various viewpoints in the still image , It is possible to perform fast and accurate detection on multi-view objects having various viewpoints in the image.

In addition, according to the present invention, by using the OCS-LBP feature extraction, it is possible to exhibit very fast performance despite the small number of characteristic dimensions including direction information, and all regions generated using the random forest classifier can be classified into object classification Learning and classification using optimal shared area features instead of using training data can enable fast learning speed and detection speed for objects having various viewpoints even when using less training data.

In addition, according to the present invention, it is possible to detect a multi-point object even in a complicated background (background) with partial occlusion using a shared area feature, and more particularly, So that the number of classifiers can be reduced to a minimum and the complexity of the calculation can be reduced.

1 is a flowchart illustrating a method of detecting a multi-view object using a shared area feature according to an exemplary embodiment of the present invention.
2 is a block diagram illustrating a multi-view object detection method using shared area features according to an exemplary embodiment of the present invention.
FIG. 3 is a block diagram illustrating a method of detecting a multi-view object using shared area features according to an exemplary embodiment of the present invention, in which OCS-LBP features are extracted by dividing each candidate region into four sub-regions.
4 is a block diagram of a process used to select a shared area feature from 12 view classes in a still image in a multi-view object detection method using a shared area feature according to an exemplary embodiment of the present invention.
5 is a view illustrating an example of a shared area feature and its location in four viewpoints according to a method of detecting a multi-view object using a shared area feature according to an exemplary embodiment of the present invention.
6 is a diagram illustrating an example of an algorithm applied to a multi-viewpoint detection method using shared area features according to an embodiment of the present invention.
FIG. 7 to FIG. 13 illustrate experimental results of a multi-point detection method using shared area features according to an embodiment of the present invention. FIG.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, in order that those skilled in the art can easily carry out the present invention. In the following detailed description of the preferred embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. In the drawings, like reference numerals are used throughout the drawings.

In addition, in the entire specification, when a part is referred to as being 'connected' to another part, it may be referred to as 'indirectly connected' not only with 'directly connected' . Also, to "include" an element means that it may include other elements, rather than excluding other elements, unless specifically stated otherwise.

FIG. 1 is a flowchart illustrating a method of detecting a multi-view object using a shared area feature according to an exemplary embodiment of the present invention. FIG. 2 is a flowchart illustrating a method of detecting multi-view objects using a shared area feature according to an exemplary embodiment of the present invention As a block diagram. 1, a method of detecting a multi-view object using a shared area feature according to an exemplary embodiment of the present invention includes generating candidate regions having an arbitrary position and size in a training image (S110) Extracting the OCS-LBP feature for each candidate region (S120), learning the feature vectors extracted for each candidate region region-based through a random forest (S130), applying the test image to the learned random forest, (S140), searching for a region to be shared for each viewpoint, and learning the region of the shared region feature in a random forest (S150) A probability value is estimated for each viewpoint, a probability value histogram is generated (S160), a class having a maximum value in the generated probability value histogram (S170) of estimating a final probability value by applying a weight to five probability values around the object and linearly combining the estimated probability values, and detecting the object as an object when the estimated final probability value is greater than a preset threshold value S180).

In step S110, candidate regions (randomly N regions) having arbitrary positions and sizes are generated in the training image having the multi-view including the object. Here, the candidate regions in step S110 are composed of N training samples collected in the window region adjusted by adjusting the size of 80 × 40 including the object in the original image of the training image. In addition, the candidate regions use randomly generated local features having a size of at least 40 × 20 (half of the window region) from a size of at least 10 × 10 in a window region having a size of 80 × 40. Further, candidate regions in step S110 can be designated as 12 training sets in the range of 0 DEG to 360 DEG at intervals of 30 DEG for the local feature of the rotating object necessary for multi-point object detection, as shown in FIG. 4 . This is because the local feature of one view can detect rotation objects from -15 ° to + 15 °. A training set can be divided into a learning group that selects a specific location and size of a local feature, and a test group. On the other hand, the negative set can be extracted and used in the background (background) area.

In step S120, the OCS-LBP (Orientation Center Symmetric Local Binary Patterns) feature is extracted for each candidate region generated in step S110. As shown in FIG. 3, in step S120, four (2 × 2) sub-regions are divided for each candidate region, and 8-dimensional OCS-LBP features are extracted in each sub-region. Here, the candidate regions can extract 32 (8 × 4) dimensional feature vectors for each candidate region by connecting the extracted OCS-LBP features in each sub-region. In the present invention, OCS-LBP features are individually calculated for each region, and an integrated histogram can be used to maintain local changes to reduce computation time.

FIG. 3 illustrates a block diagram for extracting OCS-LBP features by dividing each candidate region into four sub-regions in a multi-point object detection method using a shared area feature according to an exemplary embodiment of the present invention. 3 (a) shows a process of dividing the candidate region into four sub-regions, and FIG. 3 (b) shows a process of extracting the 8-dimensional OCS-LBP in the divided sub- c) shows the OCS-LBP feature with the tilting direction and magnitude.

The OCS-LBP generation is applied to Equation (1) below.

Where n _i and n _{i + (N / 2)} correspond to intensity values of a pixel-centered symmetric pair for pixels of constant spacing N in a circle with a radius R, and K is a gradient- ) Number.

In step S130, the feature vectors extracted for each candidate region are learned on an area basis through a random forest. As shown in FIG. 2 and FIG. 4, the random forest in step S130 can be configured as an ensemble classifier based on a decision tree. The structure of each tree applied to the random forest is top- . Random forests also use decision trees to process fast data and large amounts of data. In particular, the random forest in the present invention is used to select shared area features based on the importance of the area. That is, the present invention uses a random forest as a shared classifier to generate a random set of rectangular local features in a normalized object window to prevent experiential selection of regions, and to determine the location of meaningful size and local features do.

In step S140, a test image is applied to the learned random forest to estimate a probability value, and then an area of Local Features having higher values is selected.

In step S150, a shared area is selected and selected for each region of the selected region, and the area is shared in a random forest. The random forest in step S150 generates a multi-class classifier for the shared area feature by training with the shared classifier using the training data collected in each view class sharing the same local feature. Further, some features are selected several times from different viewpoints. Figure 4 shows a block diagram of a process used to select a shared area feature from 12 view classes in a still image.

In step S160, after a training process is performed, when a test image is input to detect an object, a probability value is estimated for each viewpoint, and a probability value histogram is generated. Here, in step S160, a probability value histogram is generated from the histogram probability distribution of the multi-class classifiers shared by the respective shared classifiers.

In step S170, the maximum value of the probability values estimated in the generated probability value histogram is searched for, and a final probability value is estimated by applying a weight to five probability values around the class having the maximum value and linearly combining them. That is, for each of the first and second maximum values, weights are applied to the five probability values around each other, and the result is linearly combined to use the case where the value is larger as the final probability value. The maximum value in step S170 uses a maximum function (Max Operation) for finding the most basic point in the generated probability value histogram.

In step S180, when the estimated final probability value is larger than a predetermined threshold value, it is detected as an object. FIG. 5 illustrates an example of a shared area feature and its location in four viewpoints according to a method of detecting a multi-view object using a shared area feature according to an exemplary embodiment of the present invention. Fig. 5 (a) shows an airplane, Fig. 5 (b) shows a motor vehicle, and Fig. 5 (c) shows a motorcycle. That is, in order to create a multi-class classifier for shared area features, the present invention uses a random forest to train shared classifiers using training data collected from each view class that shares the same local characteristics.

FIG. 6 illustrates an example of an algorithm applied to the multi-viewpoint detection method using the shared area feature according to an exemplary embodiment of the present invention. FIGS. 7 to 13 illustrate an example of an algorithm applied to the multi- And the experimental result according to the point detection method is shown. In the present invention, the algorithm of FIG. 6 is used for the multi-point detection method using the shared area feature, and the experimental results are verified through FIGS. 7 to 13. First, in order to evaluate the detection performance of multi-point object detection, we used 1200 object samples with multiple views for training and randomly selected 700 negative samples in the background. The PASCAL VOC 2012 data set comprises 6475 images containing 9415 objects from different scales and from different perspectives. The data of the test set includes an object of a different view from the table of FIG. 8 and the other scale shown in FIG.

FIG. 7 shows experimental results using six values for determining an appropriate number of shared area features in a multi-point detection method using shared area features according to an exemplary embodiment of the present invention. In other words, we show the results of experiments on the accuracy and recall of processing time by using 6 values for the threshold value. Here, four samples (airplane, bicycle, car, motorcycle) objects were selected in the test category to determine the appropriate number of shared area features with relatively little distortion and certain pattern pattern objects.

The following equations (2) and (3) express the precision and the recall formula.

FIG. 8 shows a comparison of the average accuracy results according to the multi-viewpoint detection method using the shared area feature according to the embodiment of the present invention and the average accuracy results of the conventional four methods without considering the classification of the viewpoints. Performance was evaluated according to the average density of each class for all classes using the same PASCAL VOC 2012 challenge data set. The present invention shows better object detection performance than the conventional four algorithms. The accuracy of algorithm 1 was 10.3%, that of algorithm 2 was 8.7%, that of algorithm 3 was 4.8%, and that of algorithm 4 was 12.6%.

FIG. 9 shows an accurate evaluation according to classification of viewpoints according to a PASCAL VOC 2012 data set using a multi-point detection method using shared area features according to an embodiment of the present invention, and FIG. The results show that the recall is classified according to the PASCAL VOC 2012 dataset. FIG. 11 illustrates a comparison of recall results using 10 objects without considering classification of viewpoints according to a PASCAL VOC 2012 data set using a multi-point detection method using shared area features according to an embodiment of the present invention.

12 is a diagram illustrating a result of PASCAL VOC 2012 image sample object detection using a multi-point detection method using a shared area feature according to an exemplary embodiment of the present invention. FIG. 12A shows an airplane, FIG. 12B shows a bicycle, (D) is a bus, (e) is a car, (f) is a dog, (g) is a horse, (h) is a motorcycle, (i) is a sofa, and (j) is a train. Figure 13 shows sample detection results in various object samples, showing sample detection results in various object samples where there is a serious duplication of objects due to the presence of similar structures in the foreground or background view, or in the background, It displays the detection of the error, and the red rectangle shows the exact detection.

The invention of the multi-point object detection method using the shared area feature according to an embodiment of the present invention can be implemented as a computer-readable code on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and a carrier wave (transmission via the Internet) . In addition, the computer-readable recording medium may be distributed over a network-connected computer system so that computer-readable code can be stored and executed in a distributed manner.

The present invention may be embodied in many other specific forms without departing from the spirit or essential characteristics of the invention.

S110: generating candidate regions having an arbitrary position and size in the training image
S120: Extracting the OCS-LBP feature for each candidate region
S130: learning feature vectors extracted for each candidate region on a region basis through a random forest
S140: applying a test image to the learned random forest to estimate a probability value, and then selecting a region of a local feature having higher values
S150: Finding a shared area for each viewpoint and learning in a random forest for an area of a shared area feature
S160: When a test image is input, a probability value is estimated for each viewpoint and a probability value histogram is generated
S170: Step of estimating a final probability value by imposing a weight on the five probability values having the largest value in the generated probability value histogram and linearly combining the weight values
S180: detecting as an object when the estimated final probability value is larger than a predetermined threshold value

Claims (14)

(1) generating candidate regions (Randomly N Regions) having arbitrary positions and sizes in a training image having multiple views including an object;
(2) extracting an OCS-LBP (Orientation Center Symmetric Local Binary Patterns) characteristic for each candidate region;
(3) learning region-based feature vectors extracted for each candidate region through a random forest;
(4) applying a test image to the learned random forest to estimate a probability value, and then selecting an area of Local Features having higher values;
(5) finding and selecting a shared region for a region of the selected region feature for each viewpoint, and learning the regions of the shared region feature in a random forest;
(6) generating a probability value histogram by estimating a probability value for each view when a test image is input to detect an object after performing a training process;
(7) Finding the maximum value among the probability values estimated in the generated probability value histogram, estimating the final probability value by applying a weight to the five surrounding probability values having the maximum value, and linearly combining the weight values; And
(8) detecting an object as an object when the estimated final probability value is greater than a predetermined threshold value.
2. The method of claim 1, wherein the candidate regions in step (1)
Wherein the training object is N training samples collected in a window region adjusted to a size of 80x40 including an object in the original image of the training image.
3. The apparatus of claim 2,
And a local feature having a size of at least 40 × 20 (half of the window area) is randomly generated from a size of at least 10 × 10 in the window area having the size of 80 × 40. Point object detection method.
2. The method of claim 1, wherein the candidate regions in step (1)
And a 12 training set in a range of 0 ° to 360 ° at intervals of 30 ° for a local feature of a rotating object necessary for detecting a multi-view object.
5. The method of claim 4,
A learning group for selecting a specific location and size of a region feature, and a dividing region for dividing the same into a test group.
The method according to claim 1,
And wherein the negative set is extracted from the background region.
2. The method according to claim 1, wherein in the step (2)
(2 × 2) sub-regions for each of the candidate regions, and extracts 8-dimensional OCS-LBP features in each sub-region. .
8. The method of claim 7,
And extracting 32 (8x4) -dimensional feature vectors for each candidate region by connecting the extracted OCS-LBP features in each sub-region.
2. The method of claim 1, wherein the random forest in step (3)
And an ensemble classifier based on a decision tree.
10. The method of claim 9,
Wherein the decision tree is used to process a high training rate and a large amount of data.
10. The method of claim 9,
Wherein the structure of each tree is implemented in a binary top-down manner.
2. The method of claim 1, wherein the random forest in step (5)
Wherein training is performed with a shared classifier using training data collected from each view class sharing the same local feature to generate a multi-class classifier for the shared area feature.
13. The method of claim 12, wherein in step (6)
And generating a probability value histogram from the histogram probability distribution of the multi-class classifiers shared by the respective shared classifiers.
14. The method of claim 13, wherein the maximum value in step (7)
And a maximum function (Max Operation) for finding a most basic point in the generated probability value histogram is used.

Images