KR101899993B1 - Object recognition method based on restricted region on image using disparity map - Google Patents

Abstract

A method for recognizing an object based on an area defined by a parallax map includes the steps of an image processing apparatus generating a disparity map for a source image, the image processing apparatus including a V-parallax map and a U- Wherein the image processing apparatus includes a step of determining a candidate region by using the information indicated in the V-parallax map and the U-parallax map, and a step of performing a deep run of the candidate region of the source image .

Description

[0001] OBJECT RECOGNITION METHOD BASED ON RESTRICTED REGION ON IMAGE USING DISPARITY MAP [0002]

The technique described below relates to a technique of recognizing an object of an image using deep learning.

Deep Learning (Deep Learning) is a method in which the machine learns the information provided by using the machine learning function and automatically recognizes the information of the learned objects from the input image information.

Korean Patent Publication No. 10-2008-0069601

The technique described below is to provide a technique of determining a candidate region to be deep-runned in an input image using a parallax map, and performing a deep run on a candidate region.

A method for recognizing an object based on an area defined by a parallax map includes the steps of an image processing apparatus generating a disparity map for a source image, the image processing apparatus including a V-parallax map and a U- Wherein the image processing apparatus includes a step of determining a candidate region by using the information indicated in the V-parallax map and the U-parallax map, and a step of performing a deep run of the candidate region of the source image .

The technique described below improves the speed of recognizing an object in an image by performing a deep run on a limited area without performing a deep run on the entire input image.

1 is an example of an image processing system that performs deep learning.
2 is an example of a process of performing deep learning.
3 is an example of a process of generating the V-disparity map and the U-disparity map.
4 is an example of determining a candidate region for object detection in a source image using a depth map.
5 is an example of a result of performing a deep run on a candidate region in a source image.

The following description is intended to illustrate and describe specific embodiments in the drawings, since various changes may be made and the embodiments may have various embodiments. However, it should be understood that the following description does not limit the specific embodiments, but includes all changes, equivalents, and alternatives falling within the spirit and scope of the following description.

The terms first, second, A, B, etc., may be used to describe various components, but the components are not limited by the terms, but may be used to distinguish one component from another . For example, without departing from the scope of the following description, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

As used herein, the singular " include "should be understood to include a plurality of representations unless the context clearly dictates otherwise, and the terms" comprises & , Parts or combinations thereof, and does not preclude the presence or addition of one or more other features, integers, steps, components, components, or combinations thereof.

Before describing the drawings in detail, it is to be clarified that the division of constituent parts in this specification is merely a division by main functions of each constituent part. That is, two or more constituent parts to be described below may be combined into one constituent part, or one constituent part may be divided into two or more functions according to functions that are more subdivided. In addition, each of the constituent units described below may additionally perform some or all of the functions of other constituent units in addition to the main functions of the constituent units themselves, and that some of the main functions, And may be carried out in a dedicated manner.

Also, in performing a method or an operation method, each of the processes constituting the method may take place differently from the stated order unless clearly specified in the context. That is, each process may occur in the same order as described, may be performed substantially concurrently, or may be performed in the opposite order.

The following description relates to a deep learning technique for recognizing or detecting an object included in an image. 1 is an example of an image processing system 100 that performs deep learning. The image processing system 100 includes client devices 110a and 110b, an image database 120, and an image processing device 130. [ 1 shows an example of a system connected through a network.

The client devices 110a and 110b are devices for transmitting images. The client devices 110a and 110b transmit images photographed by the camera or images stored in the storage medium. The image database 120 stores images transmitted by the client devices 110a and 110b. The image processing apparatus 130 is an apparatus that learns and analyzes an image stored in the image database 120. The image processing apparatus 130 may be a server or a computer apparatus on which the image processing program operates. The image processing apparatus 130 accesses the image stored in the image database 120 and learns the image. The image processing apparatus 130 can classify the learned image according to a certain criterion. For example, the image processing apparatus 130 can classify an image based on various conditions such as an image including a specific object, an image including a specific background, an image including a specific color, and an image size. To this end, the image processing apparatus 130 must recognize or detect an object included in the image. The image processing apparatus 130 can recognize or detect an object included in the image using a deep learning technique. Specific techniques related to deep running can use various algorithms well known in the art.

Deep running may be performed in a closed system, such as an automobile, unlike the system 100 shown in FIG. For example, an automobile system can acquire an image using a camera and perform deep running to determine whether there is a collision object such as a person or an automobile.

2 is an example of a process of performing deep learning. 2 is an example of a process of performing deep learning on a conventional image. In the conventional deep-drawing technique, a window 5 of a predetermined size is applied to the input image 10 as shown on the left side of FIG. 2 to scan the whole image 10. As a result, an object to be recognized or detected is searched as shown in the result image shown in the right side of FIG. However, the conventional deep learning method performs repeated calculations to the area where no object exists in the image 10. Therefore, unnecessary operations cause the performance of the system to deteriorate.

The deep-learning technique described below does not scan the whole image but scans only a region where the object is estimated to exist in the image. That is, the following description first sets an area to be scanned by applying a window in an image. A region to be scanned by the image processing apparatus 130 among the entire images is referred to as a candidate region. The technique described below sets a candidate region using a disparity map. The image processing apparatus scans an image based on the candidate region and performs deep running.

The image processing apparatus generates a parallax map, and generates a V-parallax map and a U-parallax map from the parallax map. The image processing apparatus determines a candidate region based on the V-parallax map and the U-parallax map.

First, the process of generating the V-disparity map and the U-disparity map will be briefly described. 3 is an example of a process of generating the V-disparity map and the U-disparity map. The image processing apparatus generates a parallax map based on the binocular image acquired by the stereo camera. 3 (a) is a stereo image obtained by a stereo camera or a plurality of cameras. There are various techniques for generating a parallax map from a stereo image. The image processing apparatus can generate a parallax map using one of various techniques. For example, the image processing apparatus can classify pixels in a stereo image, which is a source image, and perform stereo matching based on the classified pixels. FIG. 3 (b) is a result of classifying pixels for a stereo image using a pixel classification technique such as LBP (Local Binary Pattern). The image processing apparatus performs stereo matching based on the image as shown in FIG. 3 (b) to generate a parallax map. 3 (c) is an example of the generated parallax map.

The image processing apparatus generates the V-parallax map and the U-parallax map on the basis of the parallax map. Fig. 3 (d) is an example of the V-parallax map generated based on Fig. 3 (c). Fig. 3 (e) is an example of a U-parallax map generated based on Fig. 3 (c).

The V-parallax map is obtained by accumulating the pixels having the same parallax value on the same vertical axis while being located on the same vertical axis in the depth map. In the V-parallax map, the horizontal axis represents the parallax value, and the vertical axis is the vertical axis of the parallax map. The algorithm for generating the V-parallax map is as follows.

[V-Parallax Map Generation Algorithm]

for each i ^th column on disp do

for each j ^th line on disp do

if disp (i, j)> 0 then

vdisp (disp (i, j), j) ++

end

end

end

In the above code, disp is a time difference map. disp (x, y) represents the parallax value of the position (x, y) in the depth map. vdisp is the V-disparity map. vdisp (x, y) represents the value of position (x, y) in the V-parallax map.

The U-parallax map is the same as the process of generating the V-parallax map except that the direction of accumulation is the horizontal axis. The U-parallax map accumulates pixels having the same parallax value on the same horizontal axis while having the same parallax value in the depth map. In the U-parallax map, the vertical axis represents the parallax value, and the horizontal axis is the horizontal axis of the parallax map.

The image processing apparatus determines a region showing a linear change in the V-parallax map and the U-parallax map as a candidate region. The V-disparity map and the U-disparity map are expressed by a constant value. FIG. 3 is a representation of a V-parallax map and a U-parallax map having a constant value in a gray image like the parallax map. The image processing apparatus may determine the candidate region based on any one of the V-parallax map and the U-parallax map. For example, the image processing apparatus can determine, as a candidate region, an area in which any one of the V-parallax map and the U-parallax map shows a large change with reference to the horizontal axis or the vertical axis. Furthermore, the image processing apparatus may determine a region showing a large change in both the V-parallax map and the U-parallax map as a candidate region.

The linear change means that the values constituting the V-parallax map or the U-parallax map change abruptly with respect to the horizontal direction or the vertical direction. For example, in the data such as '3, 5, 4, 6, 15, 16, 20, 12, 7, 5 and 5', "15, 16, 20" The image processing apparatus can preset the range of the change for determining the candidate region. For example, reference values such as 5, 10, and the like can be used.

4 is an example of determining a candidate region for object detection in a source image using a depth map. The image processing apparatus generates a V-parallax map or a U-parallax map for the input image (10). The image processing apparatus determines a candidate region showing a linear change based on the V-parallax map or the U-parallax map. In FIG. 4, the image 20 is an example of determining a candidate region. The image processing apparatus can remove the remaining region excluding the candidate region from the input image 10. In FIG. 4, the image 150 is an example in which pixel information is left only in the candidate region in the input image 10. Of course, the image processing apparatus may process the window only in the candidate area without changing the pixel information as shown in FIG.

5 is an example of a result of performing a deep run on a candidate region in a source image. FIG. 5 is a result obtained by recognizing or detecting an object by performing a deep run of the image processing apparatus for the candidate region determined in FIG.

It should be noted that the present embodiment and the drawings attached hereto are only a part of the technical idea included in the above-described technology, and those skilled in the art will readily understand the technical ideas included in the above- It is to be understood that both variations and specific embodiments which can be deduced are included in the scope of the above-mentioned technical scope.

100: Image processing system
110a, 110b: Client device
120: image database
130: Image processing device

Claims (6)

Generating a disparity map for the source image by the image processing apparatus;
Generating a V-parallax map and a U-parallax map using the parallax map;
The V-parallax map and the U-parallax map are set as a reference area on the basis of the horizontal axis or the vertical axis, respectively, as an area in which the image processing apparatus displays a linear change using the information represented by the V-parallax map and the U- Determining a region indicating a change in disparity over a range as a candidate region;
Removing an area other than the candidate area from the source image; And
And performing the Deep Learning on the candidate region of the source image by the image processing apparatus,
Wherein the candidate region includes a region having a variation of the parallax value equal to or larger than a first reference value in the V-parallax map and a region having a parallax value corresponding to a region having a change in parallax value greater than or equal to a second reference value in the U- An object recognition method based on an area defined by a map. The method according to claim 1,
Wherein the parallax map is generated by performing stereo matching based on an image obtained by classifying pixels of the source image using a pixel classification technique. The method according to claim 1,
Wherein the V-parallax map is formed by accumulating pixels having the same parallax value included in the parallax map in the vertical direction on the map in which the horizontal axis is the parallax value and the vertical axis is the vertical axis in the parallax map,
Wherein the U-parallax map is formed by accumulating pixels having the same parallax value included in the parallax map in the horizontal direction on the map in which the vertical axis is the parallax value and the horizontal axis is the same as the horizontal axis in the parallax map, A method for recognizing an object based on a region defined by a region. delete delete delete

Images