JP2007272420A - Device, method and program for generating data for object detection, and device, method and program for object detection - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate data for object detection to accurately detect an object. <P>SOLUTION: A characteristic quantity calculation device 22 calculates a characteristic quantity h<SB>k</SB>by applying a characteristic quantity calculation function f to a sample image I<SB>k</SB>stored in an image database 21. A comparative determination device 24 calculates a distance d(h<SB>k</SB>, g<SB>p</SB>) between the sample image I<SB>k</SB>and a non-target image J<SB>p</SB>. The comparative determination device 24 determines whether distances (h<SB>k</SB>, g<SB>p</SB>) to all p exceed a threshold θp or not, and stores each sample image I<SB>k</SB>while attaching a label of "incorrect" when it exceeds and a label "correct" when it does not exceeds in a learning database 25. A learning device 26 performs learning by use of the sample images I<SB>k</SB>with the label of "correct" to generate a detection dictionary. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an object detection data generation apparatus, method and program, and an object detection apparatus, method and program, for example, an object detection data generation apparatus suitable for use in detecting an object at night, The present invention relates to a method, a program, an object detection device, a method, and a program.

  In order to improve safety during night driving, there is a need for an object detection device that performs image processing of images of the front of the vehicle imaged by an in-vehicle camera, detects objects such as pedestrians, and presents detection results to the driver Has been. In general, the object detection device prepares a detection dictionary in which the pattern of the object is learned in advance, and determines whether the input image is close to data registered in the detection dictionary. To detect. Therefore, pedestrians of various postures are prepared and learned in the detection dictionary. In this case, the image quality of the night image to be prepared becomes a problem.

  In particular, when the learning data has a low contrast, problems such as 1) the detection performance is degraded, 2) learning is not converged, and 3) the dictionary size is increased. Causes of low contrast include halation caused by pedestrian flashlights and bicycle headlights, and when the entire image is low contrast.

  Specifically, an imaging apparatus that presents an image to a driver by controlling imaging while suppressing halation is disclosed (see Patent Documents 1 and 2). The on-vehicle imaging device of Patent Document 1 fixes the shutter speed so as to obtain an exposure amount that does not cause halation when imaging infrared light reflected from a subject. The imaging apparatus of Patent Document 2 adjusts the peak value of the video signal when a predetermined test pattern is imaged under the reference condition to be 50 to 70 IRE.

  In addition, a photographing system that displays an appropriate video corresponding to the driving environment is disclosed (see Patent Document 3). The imaging system of Patent Literature 3 captures the outside of the vehicle, acquires current traveling environment information of the vehicle from the car navigation system, and controls the output of the captured video based on the traveling environment information.

On the other hand, an in-vehicle camera exposure control apparatus that controls exposure so as to obtain an image suitable for image processing is disclosed (see Patent Document 4). The on-vehicle camera exposure control device of Patent Document 4 calculates the density of the position corresponding to the lane mark in the road image captured by the camera, and controls the exposure of the camera from the calculated density value.
JP 2004-153425 A JP 2004-96345 A JP 2004-123061 A JP 2004-343303 A

  The techniques described in Patent Documents 1 to 3 improve the visibility of a captured image, but the captured image may not have an image quality suitable for image processing. For example, when the image output level decreases due to halation control, the contrast of the pedestrian in the image may decrease, or the pedestrian's gradation may be crushed by a bicycle with a strong headlight. is there. Therefore, there is a problem in using these images for learning.

  Moreover, although the technique described in Patent Document 4 can predict where the lane mark is located relative to the vehicle, it cannot predict the position of a pedestrian that is one of the detection targets. . Therefore, since the technique described in Patent Document 4 cannot detect the pedestrian density value and control the exposure, the pedestrian cannot obtain a photographed image having an image quality suitable for image processing. is there.

  Therefore, even with the techniques described in Patent Documents 1 to 4, there is a problem that an image suitable for image processing for recognizing an object cannot be acquired, and the object cannot be reliably recognized.

  The present invention has been proposed in order to solve the above-described problem, and an object detection data generation device, method, and program for generating object detection data capable of detecting an object with high accuracy, and An object of the present invention is to provide an object detection apparatus, method, and program for recognizing an object using the object detection data.

  The object detection data generation apparatus of the present invention includes a storage unit that stores a non-target feature amount that is a feature amount of a non-target image, and a feature amount calculation that calculates a sample feature amount that is a feature amount of an input sample image. Determining whether to use the sample image as the object detection image based on the means, the sample feature quantity calculated by the feature quantity calculation means, and the non-target feature quantity stored in the storage means And learning means for generating object detection data by learning using the sample image determined to be used as the object detection image by the determination means.

  The storage unit stores a non-target feature amount that is a feature amount of the non-target image. Here, the non-target image refers to an image that should not be a learning target. The feature amount calculation means calculates a sample feature amount that is a feature amount of the input sample image. The determination unit determines whether to use the sample image as the object detection image based on the calculated sample feature amount and non-target feature amount. Here, the sample images are randomly collected images, and include non-target images that are not used as target object detection images. Therefore, the determination unit removes the non-target image from the sample image and selects the target object detection image. The learning means learns using the sample image determined to be used as the object detection image.

  Therefore, the object detection data generation apparatus of the present invention calculates a sample feature amount that is a feature amount of the input sample image, and targets the sample image based on the calculated sample feature amount and non-target feature amount. An object that can be used as an object detection image and learns using a sample image that is determined to be used as an object detection image, so that learning can be easily converged and the object can be easily detected. Detection data can be generated. Note that the present invention can also be applied to an object detection data generation method and program.

  An object detection apparatus according to the present invention is an object detection data storage for storing object detection data generated by the object detection data generation apparatus according to any one of claims 1 to 4. Means, an image input means for inputting an image, an image feature quantity calculation means for calculating an image feature quantity that is a feature quantity of the input image, an image feature quantity calculated by the image feature quantity calculation means, Identification means for identifying the object based on the object detection data stored in the object detection data storage means.

  Therefore, the object detection apparatus of the present invention detects an object using the object detection data generated by the object detection data generation apparatus according to any one of claims 1 to 4. Therefore, the object can be detected with high accuracy. The present invention can also be applied to an object detection method and program.

  The present invention can generate object detection data that can detect an object with high accuracy.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an object detection system according to an embodiment of the present invention. The object detection system according to the present embodiment includes an object detection device 10 that detects an object and a learning database generation device 20 that generates a detection dictionary used in the object detection device 10. .

  For example, as shown in FIG. 2A, the object detection device 10 is used as a near-infrared camera mounted on a vehicle, and detects driving of a night pedestrian or a bicycle occupant and presents it to a driver. I do. As shown in FIG. 2 (B), the near-infrared camera projects near-infrared light onto an object such as a pedestrian, thereby setting a range wider than the low beam area of the vehicle as an object range. Can be detected.

  The object detection device 10 includes an image input device 11 that inputs image data, a feature amount calculation device 12 that calculates a feature amount based on the image data, and a feature for detection that stores in advance the feature amount of the target object to be detected. A dictionary storage device 13, an identification device 14 for identifying an object using the calculated feature value and the detection dictionary storage device 13, and an output device 15 for outputting the identification result of the object are provided.

  The image input device 11 inputs image data captured by an imaging device (not shown), for example. Note that the imaging device captures an image according to reflected light from a near-infrared light target. The image input device 11 may use image data read from a recording medium or image data transmitted via a network (not shown) as long as the image data can be input to the feature amount calculation device 12. Good.

  The feature amount calculation device 12 calculates a feature amount based on the image data input from the image input device 11. The feature amount is not particularly limited, but for example, there are the following.

  FIG. 3 is a diagram illustrating an example of a feature amount calculated by the feature amount calculation device 12. The feature amount calculation device 12 generates a luminance histogram by counting the frequency of luminance values using luminance data constituting the image data input from the image input device 11. Further, the feature quantity calculation device 12 may calculate the frequency component of the luminance data by performing Fourier transform on the luminance data constituting the image data.

  The detection dictionary storage device 13 stores the feature amount of the object as a detection dictionary. In the present embodiment, the object is, for example, a pedestrian or a bicycle occupant, but is not limited thereto. Note that the detection dictionary storage device 13 does not include a feature amount calculated from image data in which halation has occurred or low-contrast image data as a feature amount of the object. As will be described in detail later, a detection dictionary related to the feature amount of the object is generated by the learning database generation device 20.

  The identification device 14 refers to the detection dictionary storage device 13 and identifies whether there is an object in the image data input to the image input device 11 based on the feature amount calculated by the feature amount calculation device 12. Process. The output device 15 gives the identification result of the identification device 14. The output device 15 may be either a display device or an audio output device.

  Such an object detection device 10 has different object detection performance depending on the detection dictionary stored in the detection dictionary storage device 13. Hereinafter, generation of a detection dictionary that improves the detection performance of an object will be described.

  As illustrated in FIG. 1, the learning database generation device 20 includes an image database 21 that stores image data, a feature amount calculation device 22 that calculates feature amounts, and a non-target portion that stores feature amounts of non-target image data. Image database 23, comparison determination device 24 that compares the calculated feature amount with the feature amount of the non-target image data, learning database 25 that stores the learning result of comparison determination device 24 together with the image data, and object detection And a learning device 26 that learns using the business image.

In the image database 21, for example, various sample images (image data) relating to objects such as night pedestrians and bicycle occupants are stored. These sample images include images suitable for object detection (for example, high-contrast images) and images not suitable for object detection (for example, low-contrast images, images with halation). Here, it is assumed that N sample images are stored in the image database 21, and in the following, the k (k = 1, 2,..., N) -th sample image is I _k .

  The non-target image database 23 stores in advance the feature amounts of images (non-target images) to be excluded from detection targets. The feature amount of the non-target image is calculated in advance by the feature amount calculation device 22. Here, the non-target image is, for example, as follows.

  FIG. 4 is a diagram illustrating an example of a non-target image. FIGS. 4A to 4C are images in which halation has occurred, and FIGS. 4D to 4F are low-contrast images. An image in which halation has occurred is blurred by, for example, a bicycle light. In addition, the low-contrast image lacks information for identifying the object. Therefore, such a non-target image is excluded from the sample image.

Here, when there are M non-target images, the p-th non-target image is J _p (p = 1, 2,..., M). The feature amount calculation device 22 calculates the feature amount g _p in advance using the feature amount calculation function f for the p-th non-target image J _p . That is, the feature quantity calculation device 22 calculates the following equation (1).

FIG. 5A shows a luminance histogram of a non-target image in which halation has occurred, and FIG. 5B shows a luminance histogram of a low-contrast non-target image. The feature quantity g _p of these non-target images calculated by the feature quantity calculation device 22 is stored in the non-target image database 23.

  FIG. 6 is a flowchart showing a learning database generation routine. That is, the learning database generation device 20 executes the processing after the next step S1.

In step S _< b> 1, the feature quantity calculation device 22 calculates a feature quantity h _k using the feature quantity calculation function f for the sample image I _k stored in the image database 21. That is, the feature quantity calculation device 22 calculates the following equation (2) and proceeds to step S2.

In step S2, the comparison / determination device 24 evaluates the distance between the sample image I _k and the non-target image J _p . Specifically, the comparison determination device 24 includes the feature amount h _k of the sample image I _k calculated by the feature amount calculation device 22 and the feature amount g _{p of the} non-target image J _p stored in the non-target image database 23. Are applied to the distance calculation function d to calculate the distance d (h _k , g _p ). Then, the comparison / determination device 24 calculates the distance d (h _k , g _p ) for all p, and proceeds to step S3.

In step S3, the comparison / determination device 24 determines whether or not all the distances (h _k , g _p ) exceed the threshold θ _p for all _p , that is, whether or not the following equation (3) is satisfied. If the determination is affirmative, the process proceeds to step S4. If the determination is negative, the process proceeds to step S5.

  Examples of the distance calculation function d include Euclidean distance, Manhattan distance, and angle reference. That is, the comparison / determination device 24 may calculate, for example, Expression (4).

  Equation (4) represents the Euclidean distance when t = 2, and the Manhattan distance when t = 1. Further, the comparison determination device 24 may calculate, for example, the following equation (5).

Expression (5) is obtained from an angle θ _ij formed by two vectors when the sample image I _k and the non-target image J _p are represented by vectors.

In step S _< b> 4, the comparison determination device 24 labels the sample image I _k as “incorrect” and stores the sample image I _k in the learning database 25. In step S _< b> 5, the comparison determination device 24 labels the sample image I _k as “correct”, and stores the sample image I _k in the learning database 25. When the processing from step S1 to step S5 described above is performed on the N sample images I _k (k = 1 to N), the process proceeds to step S6.

  In step S6, the learning device 26 learns by using, for example, a well-known Adaboost technique, using the learning database 25. The Adaboost method uses a plurality of element classifiers, and determines a coupling coefficient of each element classifier by learning. For example, each training pattern receives a weight such that the probability of that pattern being selected is determined for a training set for an individual element classifier. Therefore, if a certain training pattern is correctly identified, the chance of being used again in the next element classifier is reduced. Conversely, if the pattern is not correctly identified, the chance of being used again increases. In this way, the Adaboost method focuses on patterns with useful information and difficult patterns. The discriminant function finally constructed as a result of learning is expressed by the following equation (6).

Here, h _k (x) is an element discriminator, and a _k is a coupling weight (coupling coefficient). k is the number of element identifiers. The learning device 26 adjusts the learning accuracy by learning by increasing / decreasing this k. The discriminant function of equation (6) outputs a positive value when the input data is determined to be correct, and outputs a negative value when it is determined to be incorrect. The learning device 26 stores the values of k, h _k (x), and a _{k in} the detection dictionary storage device 13.

  As a learning method of the learning device 26, a learning method using a neural network can also be used. Even in that case, of course, the detecting dictionary storage device 13 for storing the learning result is configured.

  In addition, the conventional learning database generation device prepares a learning database that collects thousands to tens of thousands of data for learning, and it is very complicated to determine whether or not all data should be learned. Yes, not easy.

  In contrast, the learning database generation apparatus 20 according to the present embodiment prepares data that is not to be learned as a non-target image in the non-target image database 23, and samples a flag indicating whether or not the image is similar to the non-target image. Give to the image. Then, the learning database generation device 20 learns using only the sample images that are not similar to the non-target images, so that the learning results are easily converged, and the detection dictionary stored in the detection dictionary storage device 13 is stored. It can be made compact.

  As a result, the object detection device 10 can detect the object with high accuracy from an image with high contrast by detecting the object using the detection dictionary created by the learning database generation device 20. .

  Note that the present invention is not limited to the above-described embodiment, and it is needless to say that the present invention can also be applied to a design modified within the scope of the claims. For example, examples of non-target images include low-contrast images and images with halation, but are not limited thereto.

It is a block diagram which shows the structure of the target object detection system which concerns on embodiment of this invention. (A) is a figure which shows the near-infrared camera mounted in a vehicle, (B) is a figure for demonstrating detecting a target object for the range still wider than the low beam area | region of a vehicle. It is a figure which shows an example of the feature-value calculated by the feature-value calculation apparatus. It is a figure which shows an example of a non-target image. (A) is a luminance histogram of a non-target image in which halation has occurred, and (B) is a diagram showing a luminance histogram of a low-contrast non-target image. It is a flowchart which shows the database production | generation routine for learning.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Object detection apparatus 11 Image input device 12 Feature quantity calculation device 13 Detection dictionary 14 Identification device 15 Output device 20 Learning database generation device 21 Image database 22 Feature quantity calculation device 23 Non-target image database 24 Comparison judgment device 25 For learning Database 26 Learning device

Claims (9)

Storage means for storing a non-target feature quantity that is a feature quantity of the non-target image;
Feature quantity calculation means for calculating a sample feature quantity that is a feature quantity of the input sample image;
A determination unit that determines whether to use the sample image as an object detection image based on the sample feature amount calculated by the feature amount calculation unit and the non-target feature amount stored in the storage unit;
Learning means for generating object detection data by learning using a sample image determined to be used as the object detection image by the determination means;
An object detection data generation apparatus comprising: The determination unit calculates each distance between the sample feature amount and all the non-target feature amounts, and uses the sample image as an object detection image when at least one distance does not exceed a threshold value. The object detection data generation device according to claim 1. The object detection data generation device according to claim 2, wherein the distance is one of a Euclidean distance, a Manhattan distance, and an angle reference. The object generation data generation device according to any one of claims 1 to 3, wherein the feature amount is a luminance histogram or a luminance frequency component. Calculate the sample feature value that is the feature value of the input sample image,
Based on the calculated sample feature amount and the non-target feature amount that is the feature amount of the non-target image, it is determined whether to use the sample image as an object detection image;
An object detection data generation method for generating object detection data by learning using a sample image determined to be used as the object detection image. On the computer,
Calculate the sample feature value that is the feature value of the input sample image,
Based on the calculated sample feature amount and a non-target feature amount that is a feature amount of a non-target image excluded from the target image, it is determined whether to use the sample image as a target detection image,
An object detection data generation program for generating object detection data by learning using a sample image determined to be used as the object detection image. Object detection data storage means for storing object detection data generated by the object detection data generation device according to any one of claims 1 to 4,
An image input means for inputting an image;
An image feature amount calculating means for calculating an image feature amount which is a feature amount of the input image;
Identification means for identifying an object based on the image feature quantity calculated by the image feature quantity calculation means and the object detection data stored in the object detection data storage means;
An object detection apparatus comprising: Enter an image,
Calculate the image feature amount that is the feature amount of the input image,
An object detection method for identifying an object based on the calculated image feature amount and the object detection data generated by the object detection data generation method according to claim 5. On the computer,
Calculate the image feature amount that is the feature amount of the input image,
An object detection program for identifying an object based on the calculated image feature amount and the object detection data generated by the object detection data generation method according to claim 5.