JP2010108262A - Object detection apparatus and control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect an object at high speed by narrowing a possible position and size of a partial image for object detection by use of a distance image. <P>SOLUTION: In a scannable range as the partial image, points in which a distance gap is present within the distance image is extracted so that the center of the partial image cannot move in the vicinity of the points, the partial image is scanned by moving it while changing the position and size within an input image corresponding to the distance image, and it is determined whether the scanned partial image is an object. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an object detection apparatus and method for detecting an object such as a face.

  In the face detection method described in Patent Document 1, a distance image is created, a skin color region image and an edge image are extracted, and a contour model corresponding to depth information of the distance image is created. Then, the face area is extracted based on the correlation between the contour model, the skin color image, and the edge image.

  Further, in the object detection method described in Patent Document 2, the partial image moves while changing the position and size in the input image. Then, it is determined whether or not the partial image is a face by comparison with a rectangular feature (weak hypothesis) (weak identification). Whether the weak hypothesis is correct is calculated by the integral image method. The weighting coefficient of each weak hypothesis is acquired in advance by learning. Whether a partial image is a face is determined by weighted majority of weak hypotheses (strong identification).

In the image processing method described in Patent Document 3, the distance from the vehicle to the traffic signal is calculated by referring to the position of the traffic signal from the position of the vehicle and the map data. And a traffic light is detected with the template which adjusted the magnitude | size based on the position and distance of the traffic signal in front of a vehicle.
JP 2002-216129 A US7094991 JP 2007-004256 A

  In Patent Document 1, the face area is extracted based on the distance image and the skin color. However, since the actual skin color is ambiguous, it is difficult to accurately extract the face area. This method is not suitable for detecting an object having a feature whose position and size are determined to some extent, such as the eyes and mouth.

  In Patent Document 2, since the partial image is moved while changing the position and size in the entire area of the input image, the amount of calculation increases.

  Further, the method according to Patent Document 3 cannot detect objects that can exist in various positions such as a human face and have different patterns.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to enable high-speed object detection by narrowing a range that can be taken by a partial image for object detection using a distance image. To do.

  An object detection apparatus according to the present invention includes an imaging unit that inputs an input image, a distance measuring unit that measures a distance to each object shown in the input image, and a distance image that creates a distance image corresponding to the input image. A creation unit, an object detection frame setting unit that sets an object detection frame according to a distance to an object using the distance image, and a region that can be scanned as a partial image among the set object detection frames is set. A scan area setting unit; a dictionary that stores object characteristics; and an object detection unit that determines whether the partial image scanned using the set area is an object using the dictionary. The scan area setting means includes a range that can be scanned as the partial image, in which there is a disparity in distance in the distance image. Extraction is made so that the center of the partial image cannot move in the vicinity of the point, and the object detection means scans by moving the partial image while changing the position and size in the input image corresponding to the distance image. It is characterized by that.

  In the present invention, the position and size that can be taken by the partial image for object detection are narrowed down using the distance image. As a result, it is possible to quickly detect an object having features whose positions and sizes are determined such as eyes and mouth.

<First Embodiment>
Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a configuration diagram of an object detection apparatus according to the present invention. In this embodiment, a camera (or a video camera) having a face detection function will be described. In this embodiment, face detection will be described, but detection of an object other than a face can be performed in the same manner. As shown in FIG. 1, the object detection apparatus 10 includes an imaging unit 11, a distance measurement unit 12, a distance image creation unit 13, an object detection frame setting unit 14, a scan area setting unit 15, an object detection unit 16, a face dictionary 17, The CPU 18, the ROM 19, and the RAM 20 are included.

  FIG. 2 is a flowchart of face detection processing in the object detection apparatus 10.

  First, in step S101, an image is input from the imaging unit 11. An input image is shown in FIG.

  In step S102, the distance to each object reflected in the input image is measured using the distance measuring unit 12.

  In step S103, the distance image measurement unit 13 creates a distance image (FIG. 3B) corresponding to the input image (FIG. 3A) based on the distance to each object. Here, the distance image may be created by using any method such as a lens focus method, a stereo method, or an optical radar method. In the distance image in FIG. 3B, the closer the object is, the lighter the color is displayed. As shown in FIG. 3B, an area that is regarded as having substantially the same distance from the imaging unit is divided into A1, A2, A3,.

  In step S104, the object detection frame setting unit 14 sets a detection frame for detecting a face. Here, first, as shown in FIG. 3C, a detection frame Frame1 including the closest region A1 is set.

  In step S <b> 105, the scan area setting unit 15 extracts points where there is a distance difference in the distance image. Within the detection frame Frame1 set as shown in FIG. 4A, a distance image is scanned at a constant interval, and there is a distance difference with respect to the region A1 that is a certain value or more (for example, a depth of 20 cm or more). A point is extracted (FIG. 4B). The distance difference point GP for the region A1 is as shown in FIG. Further, the non-scan area NG is as shown in FIG.

  Next, in step S106, the scan area setting unit 15 sets the non-scan area NG of the partial image SW (FIG. 5D) for the vicinity of the distance difference point GP as shown in FIG. . The size of the non-scanning area at this time is the size of the partial image SW (FIG. 5D) for detecting the face while changing the size in the target area AX (X = 1, 2,...). Set to the minimum size. When the non-scan area NG is set for all the distance disparity points GP extracted for the area A1, the result is as shown in FIG. As a result, the center of the partial image SW for face detection moves within the detection frame Frame1 set for the area A1 (FIG. 5D). The scan area OK (scannable area) is shown in FIG. )become that way.

In step S107, the object detection unit 16 uses the scan area (FIG. 6A) obtained from the distance image in the input image corresponding to the distance image (FIG. 6B) to display the partial image SW ( (Rectangle) is scanned to detect the face. The face detection will be described with reference to FIGS. As shown in FIG. 6B, the partial image SW moves while changing its position and size in the detection frame Frame1 set in the input image on the condition that the center thereof is in the scan region OK. Whether or not the partial image SW (FIG. 6B) is a face is determined by the same method as in US Pat. No. 7,099,510 as described below. First, a face dictionary storing rectangular features Feature (i) (examples shown in FIGS. 8A to 8D) having different positions and sizes and corresponding weighting coefficients C (i) (FIG. 9). A DIC (FIG. 9) is prepared. In the face dictionary DIC of FIG. 9, the rectangular features Feature (i) are arranged in an important order by learning using the training image set. First, it is determined using the integral image method whether the partial image SW (FIG. 6B) satisfies the condition for weak face identification. In the integral image method, for each point (x, y) of the input image, an integral image value,
II (x, y) = ∫ [0, x] ∫ [0, y] I (x, y) dxdy
Is assigned. Here, I (x, y) is a luminance value on the image plane. The integral image value II (x, y) can be calculated by a single reference of the luminance value I (x, y) of each pixel by the following recurrence formula.
S (x, y) = S (x, y-1) + I (x, y)
II (x, y) = II (x-1, y) + S (x, y)
S (x, -1) = 0
II (-1, y) = 0

Once II (x, y) in the entire image is calculated, the average luminance value of any rectangular area in the image can be calculated only from the integrated image value at each vertex. For example, the average luminance value M (ACDB) of the rectangular area ACDB in FIG. 7 is as follows.
M (ACDB) = (II (D) -II (B) -II (C) + II (A)) / | ACDB |

Next, in the partial image SW of the input image of FIG. 10, a value f (i) obtained by subtracting the average luminance value of the black small rectangular area from the average luminance value of the white small rectangular area corresponding to the rectangular feature Feature (i) is integrated. Obtained by image method. If f (i) satisfies a certain threshold condition, it is determined that the partial image SW satisfies the condition for weak facial recognition, and the weak identification condition h (i) = 1, otherwise h (i) = 0. Next, it is determined whether or not the partial image SW satisfies the condition for strong face identification. The face dictionary of FIG. 9 stores rectangular features constituting the face and the corresponding coefficient C (t) (t = 1 to T | T is a value determined during learning). If the next strong discrimination condition is established for each weak discrimination condition h (t) (t = 1 to T), the partial image SW in FIG. 10 is finally determined to be a face.
Σ [t = 1 to T] C (t) × h (t) ≧ (1/2) × θ × Σ [t = 1 to T] C (t) (θ is a threshold value)

  By the way, although Frame1 of FIG.3 (c) was set with respect to area | region A1 using the distance image, the size of the face reflected in area | region A1 from the approximate distance to area | region A1 and the size of a real face. The range of is narrowed. A standard face size that can exist in the area A1 is defined as FSIZE (A1). Then, the partial image SW in Frame 1 in FIG. 6 (b) is moved by changing the size at a magnification of 1.2 in the range of 1 / √2 times to √2 times of FSIZE (A 1) to detect the face. To do. Assume that the input image is 240 × 360 pixels horizontally and the minimum size of the partial image is 20 × 20 pixels. In the conventional method of US70959910, the range of the size of the partial image is approximately log (240/20) / log (1.2). In this embodiment, the size of the face is estimated based on the distance from the imaging unit, and the size of the partial image is changed in a range of 1 / √2 times to √2 times FSIZE (A1). Then, the size range of the partial image is approximately log (2) / log (1.2). That is, the size range of the partial image is approximately log (2) / log (240/20) to 0.29 times that of the conventional example. If the ratio of the scan area OK in Frame 1 in FIG. 5C is P1 (<1), the amount of calculation for detecting a face in Frame 1 is 0.29 × P1 times (<0.29). become.

  In step S108, it is determined whether or not there is a next detection frame. If there is a next detection frame, the process of steps S104 to S107 is performed on the next detection frame to detect a face. In this embodiment, as shown in FIG. 11A, the detection frame Frame2 is set for the region A2, and the process is performed. Similarly, as shown in FIGS. 11B, 11C, and 11D, detection frames are set for the areas A3, A4, and A5 to detect a face. The results of face detection in all the regions A1 to A10 are as shown in FIG.

<Modification 1>
In step S105 in FIG. 2 of the first embodiment, the interval between the scanning lines (FIG. 4A) when extracting the points where the distance difference exists may be narrower, and scanning is performed with a minimum interval of 1 pixel. Then, a point where a disparity in distance exists may be extracted. On the contrary, the interval between the scanning lines (FIG. 4A) when extracting the points where the distance difference exists may be wider.

<Modification 2>
The distance measurement in step S102 of FIG. 2 and the distance image generation in step S103 are not performed at any time, but may be performed at regular time intervals (for example, intervals of 10 seconds, 30 seconds, 60 seconds, etc.).

<Other embodiments>
The above-described first embodiment is merely a specific example for carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

  The present invention can take the form of, for example, a system, apparatus, method, program, or storage medium (recording medium). Specifically, the present invention may be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, a photographing device, a web application, etc.), or may be applied to a device composed of one device. good.

  The present invention also provides a software program that implements the functions of the above-described embodiments directly or remotely to a system or apparatus, and the system or apparatus computer reads out and executes the supplied program code. Achieved. The program in this case is a computer-readable program corresponding to the flowchart shown in the drawing in the embodiment.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  Recording media for supplying the program include the following media. For example, floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD- R).

  As a program supply method, the following method is also possible. That is, the browser of the client computer is connected to a homepage on the Internet, and the computer program itself (or a compressed file including an automatic installation function) of the present invention is downloaded to a recording medium such as a hard disk. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. It is also possible to make it. That is, the user can execute the encrypted program by using the key information and install it on the computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. Furthermore, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can be realized by the processing.

  Further, the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, and then executed, so that the program of the above-described embodiment can be obtained. Function is realized. That is, based on the instructions of the program, the CPU provided in the function expansion board or function expansion unit can perform part or all of the actual processing.

It is a block diagram of the object detection apparatus in this invention. It is a flowchart of a face detection process. (A) is a figure of an input image. (B) is a figure of the distance image corresponding to an input image. (C) is the figure which set the detection frame with respect to area | region A1. (A) is a figure which scans in order to extract the point in which the disparity of distance exists within a detection frame. (B) is a figure which scans a distance image with a fixed space | interval, and extracts the point where the disparity of distance exists in a detection frame. (C) is the figure which extracted the point where the disparity of distance exists within a detection frame. (D) is a figure of a non-scan area | region. (A) is a figure of the non-scan area | region of the vicinity of the point where the disparity of distance exists. (B) is the figure which set the non-scan area | region with respect to the point where the difference of all the distances exists. (C) is a flowchart of the scan region finally obtained. (D) is a figure in which the center of the partial image for detecting the face moves. (A) is a figure of the partial image which detects the face obtained from the distance image. (B) is a figure of the partial image which detects a face in an input image. It is a figure which shows the integral image method. It is a figure of a rectangle feature. It is a figure of a face dictionary. It is a figure of weak identification. (A)-(d) is the figure which set the detection frame with respect to area | region A2-A5, respectively. It is a figure showing a face detection result.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Image pick-up part 12 Distance measurement part 13 Distance image creation part 14 Object detection frame setting part 15 Scan area setting part 16 Object detection part 17 Face dictionary 18 CPU
19 ROM
20 RAM

Claims (6)

Imaging means for inputting an input image;
Ranging means for measuring the distance to each object shown in the input image,
Distance image creating means for creating a distance image corresponding to the input image;
An object detection frame setting means for setting an object detection frame according to the distance to the object using the distance image;
Of the set object detection frame, scan area setting means for setting an area that can be scanned as a partial image;
A dictionary that stores the features of the object,
Object detection means for determining whether the partial image scanned using the set region is an object, using the dictionary;
The scan area setting means extracts a point where a disparity in distance exists in the distance image, and a range that can be scanned as the partial image, so that the center of the partial image cannot move in the vicinity of the point,
The object detection device is characterized in that scanning is performed by moving a partial image while changing its position and size in an input image corresponding to the distance image.   The object detection apparatus according to claim 1, wherein the object is a face.   The object detection apparatus according to claim 1, wherein the distance image is created by any one of a lens focus method, a stereo method, and an optical radar method.   The object detection apparatus according to claim 3, wherein the creation of the distance image is performed at regular time intervals. An imaging step of inputting an input image by an imaging means;
A distance measuring step of measuring a distance to each object reflected in the input image by a distance measuring means;
A distance image creating step of creating a distance image corresponding to the input image by a distance image creating means;
An object detection frame setting step of setting an object detection frame according to the distance to the object using the distance image by the object detection frame setting means;
A scan area setting step for setting an area that can be scanned as a partial image in the set object detection frame by a scan area setting means;
An object detection step of determining whether or not the partial image scanned using the set area is an object by using an object detection means, using a dictionary in which object characteristics are stored;
In the scan region setting step, a range that can be scanned as the partial image is extracted from the distance image where there is a difference in distance so that the center of the partial image cannot move in the vicinity of the point,
The object detection step is a method for controlling an object detection apparatus, wherein scanning is performed by moving a partial image while changing a position and a size in an input image corresponding to the distance image.   The program for functioning a computer as a means which comprises the object detection apparatus of any one of Claims 1-4.

Images