JP2010262576A - Subject detecting apparatus and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve the accuracy of detecting a subject even if the way in which the subject is seen varies as the size and the position of the subject in an input image vary. <P>SOLUTION: A window image extraction unit 20 extracts a window image from an input image. A reliability obtaining unit 22 obtains from a reliability storage unit 58 a reliability corresponding to the probability of existence of the subject of detection with the size and the position of the window image which are calculated on the basis of the size and the position of a subject image extracted in advance from a learning image. A feature quantity calculation unit 24 calculates the feature quantity of the window image. An evaluation value calculation unit 26 compares the feature quantity of the window image with the feature quantity of an identification model stored in an identification model storage unit 74 and calculates an evaluation value indicating the likelihood that the subject in the window image is the subject of detection. A determination unit 28 determines whether or not the window image is the subject of detection based on the reliability and the evaluation value. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an object detection apparatus and program, and more particularly, to an object detection apparatus and program for detecting an object from a captured image.

  2. Description of the Related Art In recent years, an increasing number of vehicles are equipped with an object detection device that performs image processing on an image around a vehicle imaged by an in-vehicle camera, detects an object such as a surrounding vehicle, and presents a detection result to a driver.

  For example, an area where an object may exist in an image captured by a camera is specified as a processing target area, and the edge direction and edge strength distribution of each edge component in the processing target area are obtained, and according to the type of the object There has been proposed an object type determination apparatus that determines the type of an object from the obtained distribution based on the features that appear in the distribution (see Patent Document 1).

  Whether the partial area is a partial area of a person by dividing the input identification target image into a plurality of partial areas and using a threshold value for distinguishing between the target image and the non-target image calculated in advance. Determining whether or not the area type of the environment where the input image was captured from the environment where the input image was captured, changing the weight of the partial area of the target object based on the area type, and the identification result for each partial area Has been proposed (see Patent Document 2), which finally identifies whether or not the input image is a person.

JP 2007-156626 A JP 2007-310805 A

  However, in the object type determination device disclosed in Patent Document 1, there is an object in the case where the appearance of the object, such as the shape and orientation, changes due to different positions where the object exists on the input image. Since the edge information also changes depending on the position, there is a problem that it is difficult to improve the identification accuracy.

  In addition, the object recognition device of Patent Document 2 can cope with partial hiding and partial shape change to some extent, but it can cope with the case where the appearance of the entire object changes in shape and orientation. There is a problem that is difficult.

  The present invention has been made to solve the above-described problems, and improves the accuracy of object detection even when the object looks different due to the difference in size and position of the object in the input image. It is an object of the present invention to provide an object detection device and a program that can be executed.

  In order to achieve the above object, the object detection apparatus according to the first aspect of the present invention is configured to move each of a plurality of window frames having different sizes with respect to the input image by a predetermined amount from the input image. By comparing the window image with an extraction means for extracting the image of the image as a window image, and comparing each of the window images with an identification model generated in advance for detecting the detection object, the detection object likelihood of each of the window images is determined. An evaluation value calculating means for calculating an evaluation value to be expressed, and the size of the window image in the input image obtained based on the size and position of each of the detection objects in a plurality of learning images including the detection object prepared in advance, and A reliability indicating that the window image corresponding to the presence probability of the detection object at the position is the detection object, and the evaluation value Based on the window image is configured to include a judging means for judging whether the detected object.

  The object detection program according to the second aspect of the present invention is a program for detecting an image in each window frame from the input image while moving each of a plurality of window frames having different sizes with respect to the input image by a predetermined amount. An extraction value that is extracted as a window image, and an evaluation value that represents the likelihood of the detection object for each of the window images by comparing each of the window images with an identification model generated in advance to detect the detection object. And an evaluation value calculation means for calculating the size of the window image in the input image obtained based on the size and position of each of the detection objects in a plurality of learning images including the detection object prepared in advance. Reliability indicating that the window image corresponding to the existence probability of the detection target is a detection target, and the evaluation Based on the bets, the window image is a program for functioning as a determination means for determining whether or not the detection object.

  According to the object detection device of the first invention and the object detection program of the second invention, the extraction means moves the input image while moving each of a plurality of window frames having different sizes with respect to the input image by a predetermined amount. To extract an image in each window frame as a window image. Then, the evaluation value calculation means calculates an evaluation value representing the likelihood of the detection object for each of the window images by comparing each of the window images with an identification model generated in advance for detecting the detection object. . Since the identification model used here is generated so as to correspond to window images of all sizes and positions, it is difficult to cope with the difference in the appearance of the detection target due to the difference in the size and position of the window image, For window images of a specific size and position, the identification performance may be low.

  Thus, the presence of the detection object at the size and position of the window image in the input image obtained based on the size and position of each of the detection objects in the plurality of learning images including the detection object prepared in advance. It is determined whether or not the window image is the detection target based on the reliability indicating that the window image corresponding to the probability is the detection target and the evaluation value.

  As described above, the reliability representing that the window image is a detection target according to the existence probability of the detection target at the size and position of the window image in the input image together with the evaluation value calculated by comparison with the identification model. By using them together in the determination, it is possible to reflect the difference in the appearance of the detection object due to the difference in the size and position of the window image that is difficult to be compared with the identification model. Accuracy can be improved.

  In the object detection device of the first invention and the object detection program of the second invention, the reliability is extracted from a plurality of learning images including the detection object according to the size of the detection object. Based on the size of each of the plurality of object images and the position of the object image in the learning image, the position of each of the plurality of object images having substantially the same size as the size of the window image and the window image It can be determined that the smaller the average value of the distance to the position, the larger the value. Thereby, the difference in the appearance of the detection object due to the difference in the size and position of the window image can be reflected in the determination by simple calculation.

  The object detection device of the third invention extracts an image in each window frame from the input image as a window image while moving each of a plurality of window frames having different sizes with respect to the input image by a predetermined amount. An extraction unit; a feature amount calculation unit that calculates a feature amount of the window image; and a window in the input image according to a size of the detection target from a plurality of learning images including a detection target prepared in advance. A trust representing that the window image corresponding to the distance between the feature quantity of the plurality of object images extracted at the size and position corresponding to the size and position of the image and the feature quantity of the window image is the detection object. A degree-of-reliability calculation means for calculating the degree of comparison, and a feature amount of the identification model generated in advance for detecting the detection object and a feature amount of the window image And an evaluation value calculating means for calculating an evaluation value representing the likelihood of the detection target of the window image, and a determination for determining whether the window image is a detection target based on the evaluation value and the reliability. Means.

  The object detection program according to a fourth aspect of the present invention is a program for detecting an image in each window frame from the input image while moving a plurality of window frames having different sizes with respect to the input image by a predetermined amount. Extraction means for extracting as a window image, feature amount calculation means for calculating the feature amount of the window image, a plurality of learning images including detection objects prepared in advance according to the size of the detection object, and The window image corresponding to the distance between the feature amount of the plurality of object images extracted at the size and position corresponding to the size and position of the window image in the input image and the feature amount of the window image is the detection object. A reliability calculation means for calculating a reliability indicating that there is a feature, a feature amount of an identification model generated in advance to detect a detection object, and the The window image is detected based on the evaluation value calculating means for calculating the evaluation value representing the likelihood of the detection target of the window image by comparing the feature value of the window image, and the evaluation value and the reliability. It is a program for functioning as a determination means for determining whether or not an object.

  According to the object detection device of the third invention and the object detection program of the fourth invention, the extraction means moves the input image while moving each of a plurality of window frames having different sizes with respect to the input image by a predetermined amount. Then, the image in each window frame is extracted as a window image, and the feature amount calculating means calculates the feature amount of the window image. Then, the reliability calculation means extracts from a plurality of learning images including the detection target prepared in advance according to the size of the detection target and at a size and position corresponding to the size and position of the window image in the input image. A reliability indicating that the window image corresponding to the distance between the feature amount of the plurality of target object images and the feature amount of the window image is a detection target is calculated.

  Further, the evaluation value calculation means compares the feature quantity of the identification model generated in advance to detect the detection target object with the feature quantity of the window image, thereby obtaining an evaluation value representing the likelihood of the detection target object of the window image. calculate. Since the identification model used here is generated so as to correspond to window images of all sizes and positions, it is difficult to cope with the difference in the appearance of the detection target due to the difference in the size and position of the window image, For window images of a specific size and position, the identification performance may be low. Therefore, the determination unit determines whether the window image is a detection target based on the evaluation value and the reliability.

  As described above, according to the evaluation value calculated by comparison with the identification model, the distance between the feature amount of the object image at the size and position corresponding to the size and position of the window image in the input image and the feature amount of the window image. In addition, by using the reliability indicating that the window image is the detection object together for the determination, it is difficult to compare with the identification model, and the appearance of the detection object due to the difference in the size and position of the window image is difficult. Differences can also be reflected in the determination, and the accuracy of object detection can be improved.

  Further, in the object detection device of the third invention and the object detection program of the fourth invention, the reliability calculation means includes the size of each of the plurality of object images, the object image in the learning image. And the position of the object image in the learning image and the position of the window image in the input image based on the position of the object image and the feature amount of the object image. The reliability can be calculated such that the smaller the average value of the distance between the feature amount of each of the plurality of object images within the predetermined range and the feature amount of the window image is, the larger the value is. . Thereby, the difference in the appearance of the detection object due to the difference in the size and position of the window image can be reflected in the determination by simple calculation.

  In the target object detection apparatus and program, an image obtained by photographing the input image and the learning image with an in-vehicle camera that photographs a predetermined area in front of or behind the vehicle can be used.

  As described above, according to the present invention, the size and position of the target object in the input image are different, so that even if the appearance of the target object is different, the size and position of the window image in the input image are different. By using the reliability indicating that the window image is the detection target according to the existence probability of the detection target for the determination, it is possible to improve the accuracy of the target detection.

It is a block diagram which shows schematic structure of the target object detection apparatus which concerns on 1st Embodiment. It is a figure which shows an example of the search window in this Embodiment. It is a figure for demonstrating calculation of the feature-value of a window image. It is a figure for demonstrating the reliability in 1st Embodiment. It is a flowchart which shows the content of the reliability information generation process routine in 1st Embodiment. It is a figure which shows an example of the target object information in 1st Embodiment. It is a flowchart which shows the content of the target object detection processing routine in the target object detection apparatus of 1st Embodiment. It is a block diagram which shows schematic structure of the target object detection apparatus which concerns on 2nd Embodiment. It is a figure for demonstrating the reliability in 2nd Embodiment. It is a flowchart which shows the content of the target object information generation process routine in 2nd Embodiment. It is a flowchart which shows the content of the target object detection processing routine in the target object detection apparatus of 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, a case will be described in which the present invention is applied to an object detection device that detects a surrounding vehicle as a detection object.

  As illustrated in FIG. 1, the object detection device 10 according to the first embodiment includes an imaging device 12 that captures a range including an identification target region, and an object based on a captured image output from the imaging device 12. The computer 16 that executes the object detection processing routine for detecting the object and the display device 18 for displaying the processing result in the computer 16 are provided.

  The imaging device 12 captures an area including an identification target region and generates an image signal (not shown), and A / D conversion that converts an analog image signal generated by the imaging unit into a digital signal And an image memory (not shown) for temporarily storing the A / D converted image signal.

  The computer 16 includes a CPU that controls the entire object detection device 10, a ROM as a storage medium that stores various programs such as an object detection program described later, a RAM that temporarily stores data as a work area, and a bus that connects these It is comprised including. In the case of such a configuration, a program for realizing the function of each component is stored in a storage medium such as a ROM or HDD, and each function is realized by executing the program by the CPU. To.

  When the computer 16 is described with functional blocks divided for each function realizing means determined based on hardware and software, a predetermined image is input from an input image captured by the imaging device 12 and input to the computer 16 as shown in FIG. A window image extraction unit 20 that extracts an image of a region, that is, a window image in a region surrounded by a window, and a reliability indicating that the window image extracted by the window image extraction unit 20 is a detection target. The reliability acquisition unit 22 acquired from the storage unit 58, the feature amount calculation unit 24 that calculates the feature amount of the window image extracted by the window image extraction unit 20, and the feature amount calculated by the feature amount calculation unit 24 The feature quantity of the identification model stored in the model storage unit 74 is compared, and the detection pair of the object in the window image is detected. Based on the evaluation value calculation unit 26 that calculates an evaluation value that represents physicality, the reliability acquired by the reliability acquisition unit 22, and the evaluation value calculated by the evaluation value calculation unit 26, the object in the window image is displayed. A determination unit 28 that determines whether or not the object is a detection target, and a display control unit 30 that performs control so that the determination result by the determination unit 28 is superimposed on the captured image captured by the imaging device 12 and displayed on the display device 18. It can be expressed by a configuration including

  The window image extraction unit 20 moves a window of a predetermined size (referred to as a search window) from the input image by one predetermined amount of movement (referred to as a search step) per step, and moves the image at each step. cut out. Here, the cut image is referred to as a window image. The size of the window image (that is, the size of the search window) is set so that at least one of a plurality of sizes and shapes is different in order to detect surrounding vehicles appearing in various sizes on the input image. The image extraction unit 20 extracts window images using search windows of all sizes set. Here, for example, as shown in FIG. 2, search windows having a size of (A) 16 pixels × 16 pixels (B) 24 pixels × 16 pixels and (C) 64 pixels × 64 pixels are prepared. (A) and (C) are windows having the same shape and different sizes, and (B) is a window having different shapes and sizes from (A) and (C).

  Further, the window image extraction unit 20 normalizes the extracted window image into an image having a preset number of pixels.

  The reliability acquisition unit 22 obtains the reliability corresponding to the size of the extracted window image and the position in the input image from the reliability storage unit 58 in which the reliability generated by the reliability information generation device 50 described later is stored. get.

  The feature amount calculation unit 24 calculates the feature amount of the window image. Here, a histogram in the edge direction is used as the feature amount. As shown in FIG. 3A, an image extracted by the window image extraction unit 20 and normalized to a predetermined size is divided into small regions. Then, according to the following (1) and (2), the size and direction of the edge are calculated for each pixel.

  Where m (u, v) is the edge size at image coordinates (u, v), θ (u, v) is the edge direction at image coordinates (u, v), and X (u, v) is This is the pixel value of the image coordinates (u, v) of the normalized window image X. The image coordinates are represented by the upper left pixel of the normalized window image X as image coordinates (1, 1), the horizontal direction being u, and the vertical direction v.

  Using the edge size m (u, v) and the direction θ (u, v) calculated for each pixel in this way, as shown in FIG. Generate. In the voting of the histogram, the angle from 0 to 180 degrees is divided into predetermined classes (for example, less than 20 degrees, 20 degrees or more and less than 40 degrees, 40 degrees or more and less than 60 degrees, etc., one class every 20 degrees). And voting after multiplying the corresponding class in the edge direction θ (u, v) by the size (u, v) of the edge. A vector created by concatenating histograms in the edge direction generated for each small area is used as a feature amount of the window image.

  The evaluation value calculation unit 26 refers to the identification model stored in the identification model storage unit 74 and calculates an evaluation value representing the object likeness of the window image. The identification model is previously learned by the identification model generation unit 72 of the identification model generation device 70 using a learning image including a detection target (positive data) and a learning image including a non-detection target (negative data). Thus, a boundary surface between the object and the non-object in the feature space is generated and stored in the identification model storage unit 74. As a learning method, SVM (Support Vector Machine) can be used. The evaluation value is determined based on the distance between the feature amount of the window image calculated by the feature amount calculation unit 24 and the boundary surface, which is an identification model, from the boundary surface to the object (positive) side in the feature space. The evaluation value increases as the distance increases, and the evaluation value decreases as the distance from the boundary surface to the non-object (negative) side increases.

  The determination unit 28 uses the reliability p acquired by the reliability acquisition unit 22 and the evaluation value E calculated by the evaluation value calculation unit 26, and the window image is a detection target according to the following equation (3). It is determined whether or not.

       E + α × p> Th (3)

  However, α is a weighting factor of reliability and is determined in advance depending on how much reliability is reflected in the determination. Further, Th is a determination threshold value, and when the expression (3) is established, it is determined that the window image is a detection target.

  The reliability information generating apparatus 50 can be configured by a computer configured to include a CPU, ROM, RAM, built-in HDD, and the like. In the case of such a configuration, a program for realizing the function of each component is stored in a storage medium such as a ROM or HDD, and each function is realized by executing the program by the CPU. To. In addition, the reliability information generation device 50 may be configured by a microcomputer independent of the object detection device 10 or may be configured on the same computer. Further, the above-described identification model generation device 70 can also have the same configuration as the reliability information generation device 50.

  If the reliability information generating device 50 is described by function blocks divided for each function realizing means determined based on hardware and software, as shown in FIG. 1, it is detected from an input learning image including a detection object. An object image extracting unit 52 that extracts an image (object image) of an object region having a size corresponding to the size of the object, the size of the object image extracted by the object image extracting unit 52, and a learning image And a reliability calculation for calculating the reliability of each window image size and position based on the object information generated by the object information generation unit 54. And a reliability storage unit 58 in which the reliability calculated by the reliability calculation unit 56 is stored.

  The object image extraction unit 52 extracts an object image having a size corresponding to the size of the detection object from the object area including the detection object of the learning image including the vehicle that is the detection object. As the learning image including the detection target, the same learning image as the learning image as the positive data used when the identification model generation device 70 described above generates the identification model can be used.

  The object information generating unit 54 generates, as object information, the center position representing the size of the object image and the position of the object image in the learning image for each object image extracted by the object image extracting unit 52. And stored in a predetermined storage area.

  The reliability calculation unit 56 assumes image coordinates having the same number of vertical and horizontal pixels as the input image (hereinafter referred to as input image coordinates) in order to calculate the center position of the window image. The center position of the search window when the search window used in the detection process is set is calculated as the center position of the window image extracted from the input image corresponding to the position where the search window is set. Then, referring to the object information stored in the predetermined area, the window image is based on the object information indicating the center position that is substantially the same size as the size of the search window and is close to the center position of the window image. The reliability corresponding to the average value of the distance between the center position of the object and the center position indicated by the object information is calculated. Note that substantially the same size means that the ratio of the width and height of the size of the window image extracted in the search window of the selected size and the size indicated by the object information is within a predetermined range, and the width of the window image And the difference between the height ratio and the size width and height ratio indicated by the object information is within a predetermined range.

  For example, as shown in FIG. 4A, it is assumed that a window image a and a window image b are extracted using a search window of 24 pixels × 16 pixels. Further, among the object information stored in the predetermined storage area, an example in which the center position indicated by the object information whose size is approximately the same size as 24 pixels × 16 pixels is plotted on the input image coordinates (circle mark). This is shown in FIG. When the center positions of the window image a and the window image b are plotted (x marks) on the input image coordinates, the distance between the center position of the window image a and the center position of the object information is large. As for the center position, the distance from the center position of the object information is small. This indicates that the window image b is extracted from a position where the object existence probability is higher, so the reliability value is increased as the distance from the center position of the object information is smaller. Thereby, even if the evaluation value calculated by comparison with the identification model is the same value for the window image a and the window image b, by determining the reliability together, The existence probability due to the position can be reflected, and the detection accuracy is improved.

  The reliability storage unit 58 is a built-in or external storage unit, such as a hard disk drive (HDD) or a CD-ROM, and is configured of a medium that can store reliability information. The reliability information stored in the reliability storage unit 58 is referred to as the reliability of the window image in the object detection process.

  Here, the reliability information generation processing routine will be described with reference to FIG.

  In step 100, one learning image is acquired from a predetermined number (for example, 1000) of learning images prepared in advance and including vehicles that are detection objects of various sizes included in various positions.

  Next, in step 102, an object image having a size corresponding to the size of the detection object is extracted from the object region including the detection object of the learning image. Next, in step 104, the size of the extracted object image and the center position representing the position of the object image in the learning image are temporarily stored as object information in a predetermined storage area. Next, in step 106, it is determined whether or not the processing in step 102 and step 104 has been completed for all prepared learning images. If completed, the process proceeds to step 108. If not completed, the process returns to step 100 to acquire the next learning image and repeat the process. When the processing is completed for all the learning images, as shown in FIG. 6, the object information for the number of learning images prepared in a predetermined storage area is stored.

  In step 108, one size (for example, 16 pixels × 16 pixels) is selected from the search window used when the window image is extracted from the input image in the object detection process, and the assumed upper left corner of the input image coordinates is selected. Set the selected search window. Next, in step 110, the center position of the search window is calculated as the center position of the window image extracted from the input image corresponding to the position where the search window is set.

  Next, in step 112, with reference to the target object information (FIG. 6) stored in the predetermined area, target object information having approximately the same size as the size of the search window selected in step 108 is extracted and extracted. From the target object information, K target object information indicating a center position that is close to the center position calculated in step 110 is selected.

  Next, in step 114, an average value of the distance between the center position indicated by each of the selected K pieces of object information and the center position calculated in step 110 is calculated, and the average distance of the calculated center position is calculated. Convert value to confidence. The conversion from the average value of the center position distance to the reliability is performed by multiplying the average value of the center position distance by a coefficient that increases the reliability as the average value of the center position distance is smaller. Conversion can be performed based on a predetermined conversion table.

  Next, in step 116, the reliability converted in step 114 is set to the size of the search window selected in step 108 and the center position of the search window set to the input image coordinates, that is, the input image in the search window. Is generated as reliability information corresponding to the size and center position of the window image extracted from the image and stored in the reliability storage unit 58.

  Next, in step 118, it is determined whether or not the processing for calculating the reliability is completed for all positions of the input image coordinates using the search window selected in step 108. If not completed yet, the process proceeds to step 120 where the search window is set to a position moved by one step in the assumed input image coordinates, and the process returns to step 110 to repeat the process. On the other hand, when the process of calculating the reliability is completed for all the positions of the input image coordinates, the process proceeds to step 122 to determine whether the process is completed for all prepared search windows. If there is an unprocessed size search window, the process returns to step 108 and the process is repeated for the next size search window. When the processing is finished for all the search windows, this routine is finished.

  Thereby, the reliability is calculated in advance for all the window images extracted in the object detection process.

  Next, an object detection processing routine in the object detection device 10 according to the first embodiment will be described with reference to FIG. This routine is performed when the CPU executes the object detection program stored in the ROM when imaging of the target area is started by the imaging device 12.

  In step 130, an image picked up by the image pickup device 12 is acquired. Next, in step 132, a search window of, for example, 16 pixels × 16 pixels is set on the input image, for example, a predetermined area (for example, the upper left corner area) of the input image. ), A window image of 16 × 16 pixels is extracted from the input image using the set search window, and is normalized to a predetermined size.

  Next, in step 134, the reliability p corresponding to the size and the center position of the window image is acquired with reference to the reliability information stored in the reliability storage unit 58.

  Next, in step 136, the normalized window image is divided into small regions, the size and direction of the edge are calculated for each pixel in accordance with the above (1) and (2), and the edge direction of each small region is calculated. A histogram is generated, and a vector created by concatenating histograms in the edge direction generated for each small region is calculated as a feature amount of the window image.

  Next, in step 138, the window image calculated in step 136 is referred to with reference to the identification model generated as a boundary surface between the object and the non-object in the feature space and stored in the identification model storage unit 74. An evaluation value E corresponding to the distance between the feature quantity and the boundary surface that is the identification model is calculated.

  Next, in step 140, whether or not the window image is a detection object according to the above equation (3) using the reliability p acquired in step 134 and the evaluation value E calculated in step 138. Determine whether. If the expression (3) is satisfied, it is determined that the window image is a detection target, and the process proceeds to step 142 where information such as the position and size of the search window is stored in the RAM as a list, and the next step Proceed to 144. On the other hand, when the expression (3) is not satisfied, it is determined that the window image is not the detection target, and step 142 is skipped and the process proceeds to step 144.

  In step 144, it is determined whether or not the search has been completed by scanning the search window for the entire input image. If not completed, the process proceeds to step 146, the position of the search window is moved by a predetermined search step, the process returns to step 132, and the processes from step 132 to step 144 are repeated. When the search of the entire input image in the search window of the current size is completed, the process proceeds to step 148.

  In step 148, it is determined whether or not the search in the search windows of all sizes is completed. If not completed, the process proceeds to step 150, the size of the search window is changed to another size (for example, 24 pixels × 16 pixels), the process returns to step 132, and the processes of steps 132 to 148 are repeated. When the search is completed in the search windows of all sizes, the process proceeds to step 152.

  In step 152, based on the information stored in the list, the display device 18 is controlled so that the detected object is displayed surrounded by a window with respect to the input image.

  As described above, according to the object detection device of the first embodiment, not only the evaluation value calculated by comparing the feature amount of the window image extracted from the input image with the identification model generated in advance, Since the reliability corresponding to the existence probability of the detection target object at the size and position of the window image calculated based on the size and position of the target object image extracted in advance from the learning image is used for the determination, the identification model Therefore, it is possible to improve the detection accuracy in response to the difference in the appearance of the detection target due to the difference in the size and position of the detection target in the input image that is difficult to cope with.

  Moreover, since the calculation of the reliability can be easily calculated as the existence probability of the detection target object in the size and position of the window image, an identification model is generated for each size and position of the detection target object in the input image. Compared to the case, the calculation time can be reduced.

  In the first embodiment, the case where the reliability information is generated in advance by the reliability information generation device has been described. However, the object information generated by the object information generation unit is stored and the target information is stored. In the object detection apparatus, the object information corresponding to the size and position of the window image may be acquired to calculate the reliability.

  Next, a second embodiment will be described. The object detection device according to the second embodiment is different from the object detection device according to the first embodiment in that the feature amount of the window image is also used for calculating the reliability. In addition, about the structure and process same as 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  As illustrated in FIG. 8, the object detection device 210 according to the second embodiment includes an imaging device 12, a computer 216, and a display device 18.

  The function block obtained by dividing the computer 216 for each function realizing means determined based on hardware and software will be described. As shown in FIG. 8, a window image extraction unit 20, a feature amount calculation unit 24, a window image extraction, and the like. A reliability calculation unit 32 that calculates a reliability indicating that the window image extracted by the unit 20 is a detection target, an evaluation value calculation unit 26, and the reliability and evaluation values calculated by the reliability calculation unit 32 Based on the evaluation value calculated by the calculation unit 26, it can be expressed by a configuration including a determination unit 28 that determines whether or not the window image is a detection target and a display control unit 30.

  The reliability calculation unit 32 refers to the object information stored in the object information storage unit 62 of the object information generation device 250 described later, and is substantially the same as the size of the window image extracted by the window image extraction unit 20. Object information indicating the center position that is the size and is close to the center position of the window image is acquired. Then, the average of the distance between the feature amount of the window image calculated by the feature amount calculation unit 24 and the feature amount of the object image indicated by each of the acquired object information is calculated, and the average of the calculated distance is determined. Calculate reliability.

  For example, as shown in FIG. 9A, it is assumed that a window image a is extracted using a search window of 24 pixels × 16 pixels. Also, the object information close to the size and center position of the window image a is acquired, and the learning image including the object image corresponding to the acquired object information is as shown in FIG. To do. In this case, since both the window image a and the object image are images of the oncoming vehicle in the right region of the input image, the appearance on the image is similar and the distance between the feature amounts is also short. On the other hand, when the window image b as shown in FIG. 5C is extracted and the object information corresponding to the learning image including the object image as shown in FIG. The distance between the feature amount and the feature amount of the object image is large. This indicates that the feature amount indicated by the window image differs depending on the size and position of the window image. Therefore, the reliability value increases as the distance between the feature amount of the window image and the feature amount of the object image decreases. To do. The identification model is generated using object images and non-object images extracted at all sizes and positions. Therefore, for example, when the feature amounts of the window image a and the window image b are approximated as shown by the histograms in FIGS. It will be calculated. However, by determining the reliability as well, the difference in appearance depending on the position of the detection target can be reflected, and the detection accuracy is improved.

  The object information generation device 250 can be configured by a computer including a CPU, a ROM, a RAM, a built-in HDD, and the like. In the case of such a configuration, a program for realizing the function of each component is stored in a storage medium such as a ROM or HDD, and each function is realized by executing the program by the CPU. To. In addition, the object information generation device 250 may be configured by a microcomputer independent of the object detection device 210, or may be configured on the same computer.

  When the target object information generating apparatus 250 is described by function blocks divided for each function realizing means determined based on hardware and software, as shown in FIG. 8, the target object image extracting unit 52 and the feature amount calculating unit 60 are provided. An object information generation unit 254 that generates, as object information, the size of the object image extracted by the object image extraction unit 52, the position in the learning image, and the feature amount calculated by the feature amount calculation unit 60; The object information storage unit 62 that stores the object information generated by the object information generation unit 254 can be used.

  The feature amount calculation unit 60 calculates the feature amount of the object image extracted from the learning image by the same method as the feature amount calculation unit 24 in the object detection device 210.

  Here, the object information generation processing routine will be described with reference to FIG.

  In step 100, one learning image is acquired from a predetermined number (for example, 1000) of learning images including the detection target. Next, in step 102, an object image is extracted from the learning image. Next, in step 200, a histogram in the edge direction is calculated as the feature amount of the object image extracted in step 102.

  Next, in step 202, the size of the extracted object image, the center position representing the position of the object image in the learning image, and the feature amount calculated in step 200 are generated as object information, and the object The information is stored in the information storage unit 62.

  Next, in step 106, it is determined whether or not the processing of step 102, step 200, and step 202 has been completed for all prepared learning images. If not completed, the process returns to step 100, the next learning image is acquired, and the process is repeated. When the processing is completed for all the learning images, this routine is terminated.

  Next, an object detection processing routine in the second embodiment will be described with reference to FIG. This routine is performed when the CPU executes the object detection program stored in the ROM when imaging of the target area is started by the imaging device 12.

  In step 130, an input image is acquired, and in step 132, a window image is extracted from the input image and normalized to a predetermined size. Next, in step 136, the feature amount of the window image extracted in step 132 is calculated.

  Next, in step 220, an object indicating substantially the same size as the size of the window image extracted in step 132 with reference to the object information stored in the object information storage unit 62 of the object information generating device 250. Object information is extracted, and K pieces of object information indicating a center position that is close to the center position of the window image are acquired from the extracted object information.

  Next, in step 222, an average value of the distance between the feature amount indicated by each of the acquired K pieces of object information and the feature amount of the window image is calculated, and the calculated average value of the distance of the feature amount is used as the reliability. Convert to The distance between the feature amounts represents a difference between the feature amounts. For example, the Euclidean distance between the feature vectors can be used. In addition, the conversion from the average value of the feature amount to the reliability is performed by multiplying the average value of the distance of the feature amount by a coefficient such that the reliability increases as the average value of the feature amount decreases. The conversion can be performed based on a predetermined conversion table.

  Next, in step 138, an evaluation value E corresponding to the distance between the feature amount of the window image calculated in step 136 and the boundary surface as the identification model is calculated. Next, in step 140, in step 222 described above. Based on the calculated reliability p and the evaluation value E calculated in step 138, it is determined whether or not the window image is a detection object. Thereafter, similarly to the first embodiment, steps 142 to 152 are executed.

  As described above, according to the target object detection device of the second embodiment, not only the evaluation value calculated by comparing the feature amount of the window image extracted from the input image with the identification model generated in advance. The size of the window image calculated in correspondence with the distance between the feature amount of the target image and the feature amount of the window image, which is close to the size and position of the window image, among the target images extracted from the learning image in advance. Since the reliability corresponding to the difference in the appearance of the detection object due to the difference in position is used for the determination, the detection object due to the difference in the size and position of the detection object in the input image that is difficult to cope with the identification model. Corresponding to the difference in appearance, detection accuracy can be improved.

  In addition, since the reliability can be easily calculated based on the feature amount of the window image and the feature amount of the corresponding object image, the reliability is identified for each size and position of the detection object in the input image. Compared to the case of generating a model, the calculation time can be reduced.

  In the first and second embodiments, the case where the identification model is generated using the SVM method has been described. For example, another identification model such as a cascade type classifier using a boosting algorithm is generated. You may do it.

  In the first and second embodiments, the case where the edge direction histogram is used as the feature amount of the window image and the object image has been described. However, for example, other feature amounts such as a luminance histogram may be used. Good.

  Further, both the reliability in the first embodiment and the reliability in the second embodiment may be used for the determination. For example, the reliability based on the size and position of the window image in the first embodiment is defined as p1, and the reliability p2 based on the size, position, and feature amount of the window image in the second embodiment is used as a weighting factor for reliability. When the following expression expressed using α and β and the determination threshold Th is satisfied, it is possible to determine that the window image is a detection target.

     E + α × p1 + β × p2> Th

  In addition, even when there is a difference in mounting position or mounting angle between the imaging device that captures the input image and the imaging device that captures the learning image, this difference is corrected by image processing or the like, The present invention can be applied by taking the correspondence between the size of the window image and the position in the input image and the size of the object image and the position in the learning image.

DESCRIPTION OF SYMBOLS 10,210 Object detection apparatus 12 Imaging device 16,216 Computer 20 Window image extraction part 22 Reliability acquisition part 24 Feature-value calculation part (window image)
26 evaluation value calculation unit 28 determination unit 32 reliability calculation unit 50 reliability information generation device 52 object image extraction unit 54 object information generation unit 56 reliability calculation unit 58 reliability storage unit 60 feature amount calculation unit (object image )
62 Object information storage unit 70 Identification model generation device 250 Object information generation device

Claims (8)

Extracting means for extracting an image in each window frame from the input image as a window image while moving each of a plurality of window frames having different sizes with respect to the input image;
An evaluation value calculating means for calculating an evaluation value representing the likelihood of a detection object for each of the window images by comparing each of the window images with an identification model generated in advance to detect a detection object;
According to the existence probability of the detection object at the size and position of the window image in the input image obtained based on the size and position of each of the detection objects in a plurality of learning images including the detection object prepared in advance Determination means for determining whether or not the window image is a detection object based on the reliability indicating that the window image is a detection object and the evaluation value;
An object detection apparatus including:   The reliability is determined based on the size of each of the plurality of object images extracted according to the size of the detection object from the plurality of learning images including the detection object, and the position of the object image in the learning image. 2. The method according to claim 1, wherein the smaller the average value of the distance between the position of each of the plurality of object images having substantially the same size as the size of the window image and the position of the window image, the larger the value. Object detection device. Extracting means for extracting an image in each window frame from the input image as a window image while moving each of a plurality of window frames having different sizes with respect to the input image;
Feature amount calculating means for calculating the feature amount of the window image;
A plurality of objects extracted from a plurality of learning images including a detection object prepared in advance according to the size of the detection object and at a size and position corresponding to the size and position of the window image in the input image Reliability calculation means for calculating a reliability indicating that the window image is a detection target according to the distance between the feature quantity of the image and the feature quantity of the window image;
Evaluation value calculating means for calculating an evaluation value representing the likelihood of the detection target of the window image by comparing the feature amount of the identification model generated in advance to detect the detection target object with the feature amount of the window image. When,
Determining means for determining whether the window image is a detection object based on the evaluation value and the reliability;
An object detection apparatus including:   The reliability calculation means is substantially the same as the size of the window image based on the size of each of the plurality of object images, the position of the object image in the learning image, and the feature amount of the object image. A feature amount of each of the plurality of object images having a size and a distance between the position of the object image in the learning image and the position of the window image in the input image, and the feature of the window image The object detection apparatus according to claim 3, wherein the reliability is calculated so that the value increases as the average value of the distance to the quantity decreases.   The target object detection apparatus of any one of Claims 1-4 which image | photographed the said input image and the said image for learning with the vehicle-mounted camera which image | photographs the predetermined area | region of the front or back of a vehicle. Computer
Extracting means for extracting an image in each window frame from the input image as a window image while moving each of a plurality of window frames having different sizes with respect to the input image;
An evaluation value calculating means for calculating an evaluation value representing the likelihood of a detection object for each of the window images by comparing each of the window images with an identification model generated in advance to detect a detection object;
According to the existence probability of the detection object at the size and position of the window image in the input image obtained based on the size and position of each of the detection objects in a plurality of learning images including the detection object prepared in advance Determination means for determining whether or not the window image is a detection object based on the reliability indicating that the window image is a detection object and the evaluation value;
The object detection program to make it function. Computer
Extracting means for extracting an image in each window frame from the input image as a window image while moving each of a plurality of window frames having different sizes with respect to the input image;
Feature amount calculating means for calculating the feature amount of the window image;
A plurality of objects extracted from a plurality of learning images including a detection object prepared in advance according to the size of the detection object and at a size and position corresponding to the size and position of the window image in the input image Reliability calculation means for calculating a reliability indicating that the window image is a detection target according to the distance between the feature quantity of the image and the feature quantity of the window image;
Evaluation value calculating means for calculating an evaluation value representing the likelihood of the detection target of the window image by comparing the feature amount of the identification model generated in advance to detect the detection target object with the feature amount of the window image. When,
Determining means for determining whether the window image is a detection object based on the evaluation value and the reliability;
The object detection program to make it function.   An object detection program for causing a computer to function as each means constituting the object detection device according to any one of claims 1 to 5.

Images