JP2012185684A - Object detection device and object detection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enhance detection accuracy in the case that a plurality of detection objects exist at great and short distances without increasing a processing load.SOLUTION: An object detection device includes: a distance information acquiring part for acquiring distance information to an imaging object about an input image; a distance section setting part for setting a distance section to the input image on the basis of the distance information; a window size setting part for setting a size of a retrieval window for each distance section; a scanning part for using a retrieval window of the set size to scan the input image for each distance section; and a detection processing part for performing detection processing of a detection object in the retrieval window.

Description

  The present invention relates to an object detection technique, and relates to a technique for improving detection accuracy when a plurality of detection objects exist at a distance.

  2. Description of the Related Art Conventionally, an object detection technique for detecting an object such as a person or a vehicle using features such as an outline on a captured image generated by an imaging device such as a video camera has been proposed. In such an object detection technique, a method of detecting a target object by scanning a search window on a captured image using a template image or a discriminator created by machine learning is the mainstream.

  When the detection method using features such as contours is applied to surveillance camera devices that detect and track people, unlike intrusion methods that use color and movement, an intruder can be detected without being swayed by small animals or trees. It is possible to detect and deal with tracking.

  In this case, the size of the detection target in the image varies depending on the position of the detection target and the state of the monitoring camera device such as the zoom / shooting direction. In the surveillance camera device, multiple search windows of different sizes are set so that the target object can be detected reliably regardless of the size of the image, and the entire captured image is displayed using each search window. Scanning is done. For example, in the examples shown in FIGS. 6A to 6C, a person who appears in different sizes is detected by scanning the entire search window W1, search window W2, and search window W3 of different sizes. ing.

  In terms of mounting, since a search window of a larger size is used and scanning is performed while overlapping by a predetermined amount, the detection process takes time, and a large memory capacity is required, resulting in a heavy processing load.

  To deal with this problem, Patent Document 1 describes that the size of a subject is estimated based on distance information obtained from the focus information of the subject, and an object is detected using a search window corresponding to the size. Has been.

JP 2004-334836 A

  However, the monitoring camera device cannot obtain distance information based on the focus state of the subject because the subject is not fixed unlike the shooting state of a general camera. For this reason, even if the technique described in Patent Document 1 is applied as it is, a sufficient effect cannot be obtained. Further, in the monitoring camera device, it is necessary to increase the detection accuracy for all the objects even when there are a plurality of objects to be detected at a distance, and the sizes of the objects in the captured image are different.

  Therefore, an object of the present invention is to increase the detection accuracy when a plurality of detection objects exist at a distance from each other without increasing the processing load.

In order to solve the above-described problem, an object detection apparatus according to a first aspect of the present invention includes a distance information acquisition unit that acquires distance information about an input image to an imaging target, and the input image based on the distance information. A distance category setting unit that sets a distance category for each, a window size setting unit that sets a size of a search window for each distance category, and the search window having a set size for each distance category A scanning unit that scans the input image and a detection processing unit that performs detection processing of a detection target from the search window are provided.
Here, the distance information acquisition unit acquires distance information for each pixel of the input image, and the distance segment setting unit divides the input image into a plurality of blocks, and based on the distance information of the pixels in the block. The distance classification can be set in block units.
The window size setting unit sets a size ratio of a search window to the input image, and the scan unit performs the scan using an input image and a search window having a size according to the size ratio. You may do it.
In order to solve the above-described problem, an object detection method according to a second aspect of the present invention includes a distance information acquisition step for acquiring distance information to an imaging target related to an input image, and the input image based on the distance information. A distance division setting step for setting a distance division for the window, a window size setting step for setting a search window size for each of the distance divisions,
A detection step of scanning the input image using the search window of a set size for each distance section and detecting a detection target object from within the search window.

  According to the present invention, it is possible to improve the detection accuracy when a plurality of detection objects exist at a distance from each other without increasing the processing load.

It is a block diagram which shows the structure of the target object detection apparatus which concerns on this embodiment. It is a figure which shows the example of the distance division set. It is a figure explaining the size setting of a search window. It is the figure which showed typically the state which is scanning in the search window of the size set for every distance division. It is a flowchart explaining operation | movement of the target object detection apparatus of this embodiment. It is a figure explaining the scanning operation | movement of the conventional search window.

  Embodiments of the present invention will be described in detail with reference to the drawings. This embodiment is an example when the object detection apparatus of the present invention is applied to an object detection system. FIG. 1 is a block diagram illustrating a configuration of an object detection system according to the present embodiment.

  As shown in the figure, the object detection system 10 includes an object detection device 100, a camera 200, and a display device 300. The object detection system 10 can function as, for example, a monitoring camera system. In this case, the camera 200 functions as a monitoring camera device, and the display device 300 functions as a monitoring monitor device. The object detection device 100 can be configured using, for example, an information processing device including a CPU, a memory, an interface, and the like.

  The camera 200 includes an imaging unit 210, a camera driving unit 220, and a distance measuring unit 230. The imaging unit 210 includes an optical system such as a lens, a light receiving unit such as a light receiving element, an image processing unit such as an arithmetic device, and the like, captures a moving image and outputs image data to the object detection device 100. It is assumed that the optical system includes a zoom function.

  The camera driving unit 220 includes a driving mechanism such as a motor, and based on a control signal from the object detection device 100, a function for changing the zoom function of the imaging unit 210 and a pan / tilt function for changing the imaging direction. It has. Here, zoom and pan / tilt are collectively referred to as PTZ.

  The distance measuring unit 230 measures the distance to the imaging target in real time on the entire surface in the image captured by the imaging unit 210. For the distance measurement, various methods such as a TOF (Time of Flight) method, a stereo camera method, and the like can be employed. Here, the TOF method, for example, flashes the infrared LED for a moment during imaging or before and after imaging, and measures the time until the infrared ray bounces back on the imaging target for each light receiving element of the imaging unit 210, to the imaging target. The distance is calculated. The distance information measured by the distance measurement unit 230 is recorded in the alpha channel for each pixel of the image data output from the imaging unit 210 and is output to the object detection device 100, for example.

  The object detection apparatus 100 includes an image input unit 110, a distance information acquisition unit 120, a distance category setting unit 130, a window size setting unit 140, a scanning unit 150, a detection processing unit 160, and a camera control unit 170.

  The image input unit 110 continuously inputs image data from the camera 200. The distance information acquisition unit 120 acquires distance information from the alpha channel for each pixel of the image data and outputs the distance information to the distance category setting unit 130.

  The distance category setting unit 130 sets a distance category for the image based on the distance information for each pixel. The distance classification is, for example, by dividing the image into blocks of a predetermined size, and assigning each block to a distance classification such as less than 10 m, less than 10 m, less than 20 m, and more than 20 m and less than 30 m based on the average distance of the pixels in the block. Can be set.

  FIG. 2B shows an example of distance divisions set for the image shown in FIG. In this example, the distance division is divided into five stages in advance, the image is divided into blocks having a size as shown in the region B of FIG. 2B, and the average value of the distance information of the pixels in each block is used. Thus, each block is assigned to a distance category. For example, if the average value of the distance information of the pixels included in a certain block is 12 m, the block is a distance category of 10 m or more and less than 20 m.

  The window size setting unit 140 sets a search window size for each distance category. That is, the size on the image changes according to the distance from the camera 200 even if the objects have the same size. Specifically, the farther away the object, the smaller the image appears. Accordingly, the window size setting unit 140 sets the search window size to be relatively smaller as the distance is further away.

  The size on the image also changes depending on the zoom state of the imaging unit 210. Specifically, even if the distance between the object and the camera 200 is the same, the larger the zoom focal length, the larger the image. For this reason, the window size setting unit 140 may acquire zoom information of the camera 200 from the camera control unit 170 described later, and may further set the search window size according to the focal length.

  In the present embodiment, for example, as shown in FIG. 3, the search window is set according to the distance category and the focal length. That is, the search window is set to be smaller as the distance section is farther, and the larger search window is set as the focal length is longer if the distance section is the same. The size of each search window is acquired experimentally and theoretically based on the size of the detection target itself such as a person / vehicle and the lens characteristics of the imaging unit 210, and the inside or outside of the window size setting unit 140 Make a note of it.

  The scan unit 150 scans the image input by the image input unit 110 using a search window having a size set for each distance section. The search window is scanned over the entire distance section with a predetermined number of overlaps.

  FIG. 4 is a diagram schematically showing a state in which scanning is performed in a search window having a size set for each distance section. The set distance division corresponds to FIG. In the example of this figure, scanning is performed using the search window Wa having the smallest size in the distance section A having the longest distance, and using the search window Wb having the next smallest size in the distance section B having the next longest distance. Scanning. Then, scanning is performed using the search window We having the largest size in the distance section E having the shortest distance. The scan direction, procedure, and the like can be determined in advance as a scan sequence.

  During scanning, the search window is moved while being overlapped, so that the search window may protrude from the corresponding distance section. If the same distance segment is not continuous but discrete, scanning is performed using a search window of the same size for each region. In this embodiment, the search window size is changed. However, the search window size may be fixed, and the image size may be relatively changed for scanning.

  As described above, in this embodiment, since a scan is performed for each distance section using a search window having a size corresponding to the distance section, detection accuracy can be improved even when a plurality of detection objects exist at a distance. Can do. In addition, since it is not necessary to scan the entire screen using search windows of various sizes, it is possible to reduce the time required for detection processing and reduce the necessary memory capacity.

  The detection processing unit 160 determines whether or not a detection target exists in the search window, and outputs an image including the detection result to the display device 300. Whether or not a detection target exists in the search window can be determined by extracting features such as contours from the search window and using a template image or a classifier created by machine learning. As features such as contours, for example, HOG (Histograms of Oriented Gradients) feature values can be used. As machine learning, for example, the AdaBoost algorithm can be used.

  The camera control unit 170 controls the operation of the camera driving unit 220 according to the detection result, and causes the camera 200 to perform the PTZ operation. The PTZ operation of the camera 200 to be performed according to the detection result can be performed with various logics. For example, a zoom operation is performed to capture a large image of the detection object, a plurality of detection objects are all within the angle of view, a PT operation is performed according to the position of the detection object, and a detection is performed. The PT operation can be performed so as to track the object. When the zoom operation is performed, the focal length information is notified to the window size setting unit 140.

  Next, the operation of the object detection apparatus 100 in the present embodiment will be described with reference to the flowchart of FIG. First, the image input unit 110 inputs image data from the camera 200 (S101). Then, the distance information acquisition unit 120 determines whether the PTZ has changed with the previous image under the control of the camera control unit 170 (S102). This is because if there is no change in PTZ (S102: No), there is no need to change the distance division in the image.

  If there is a change in the PTZ (S102: Yes), and if the distance category is not set, the distance information acquisition unit 120 acquires distance information for each pixel from the alpha channel of the image data (S103). Then, the distance segment setting unit 130 divides the region in the image into blocks, and sets the distance segment in units of blocks as shown in FIG. 2B (S104).

  When the distance segment is set, the window size setting unit 140 sets the size of the search window for each distance segment (S105). At this time, the search window size may be set in consideration of the zoom state of the camera 200, lens characteristics, and the like.

  If there is a change in the PTZ, the scanning unit 150 displays the newly set distance section as a target, and if there is no change in the PTZ, the scanning unit 150 displays the set distance section as a target in FIG. As shown, scanning is performed using a search window having a size set for each distance section, and the detection processing unit 160 performs object detection processing from within the search window (S106).

  The camera control unit 170 controls the camera 200 with PTZ based on the detection result (S107). The PTZ control of the camera 200 based on the detection result can be performed as in the conventional case.

  The target object detection apparatus 100 repeats the above process until the detection process ends (S108: Yes). With the above procedure, the object detection apparatus 100 can detect the detection object with high accuracy without increasing the processing load even when there are a plurality of detection objects at a distance.

DESCRIPTION OF SYMBOLS 10 ... Object detection system 100 ... Object detection apparatus 110 ... Image input part 120 ... Distance information acquisition part 130 ... Distance classification setting part 140 ... Window size setting part 150 ... Scan part 160 ... Detection processing part 170 ... Camera control part 200 ... Camera 210 ... Imaging unit 220 ... Camera drive unit 230 ... Distance measurement unit 300 ... Display device

Claims (4)

A distance information acquisition unit that acquires distance information to the imaging target related to the input image;
A distance segment setting unit that sets a distance segment for the input image based on the distance information;
A window size setting unit for setting the size of the search window for each distance section;
A scan unit that scans the input image using the search window of a set size for each distance section;
An object detection apparatus comprising: a detection processing unit configured to detect a detection object from within the search window. The distance information acquisition unit acquires distance information for each pixel of the input image,
The object detection according to claim 1, wherein the distance segment setting unit divides the input image into a plurality of blocks, and sets a distance segment in units of blocks based on distance information of pixels in the block. apparatus. The window size setting unit sets a size ratio of a search window with respect to the input image;
The object detection device according to claim 1, wherein the scan unit performs the scan using an input image having a size according to the size ratio and a search window. A distance information acquisition step of acquiring distance information to the imaging target regarding the input image;
A distance segment setting step for setting a distance segment for the input image based on the distance information;
A window size setting step for setting the size of the search window for each distance section;
A detection step in which the input image is scanned using the search window of a set size for each distance section, and a detection target object is detected from within the search window. Object detection method.