JP6423594B2 - Object detection device - Google Patents

Description

  The present invention relates to an object detection device, and more particularly to an object detection device that detects a detection target from a frame image.

  There is an object detection device that determines whether or not a detection target exists in an image taken by a camera. For example, the object detection device is mounted on a vehicle together with a camera. The object detection device can notify the driver of the presence of a pedestrian by determining whether or not a person is present in an image captured by the camera. By using the object detection device, the driver can easily grasp the situation outside the vehicle.

  The object detection apparatus uses pattern matching to determine whether or not a detection target exists in an image. Examples of pattern matching algorithms include neural networks and support vector machines. The object detection device sets an area (window area) for detecting an object for the input image. The object detection device determines whether or not the detection target is included in the window region using a program in which the above algorithm is implemented.

  Patent Documents 1 and 2 disclose object detection apparatuses using pattern matching.

  The device disclosed in Patent Literature 1 detects a pedestrian from an image captured by an in-vehicle camera. Specifically, this apparatus detects a candidate object that may be a pedestrian from an image using a neural network, and compares the candidate object with a pedestrian's head, limbs, etc. It is determined whether it is a person.

  The apparatus disclosed in Patent Document 2 sets a plurality of detection windows in which detection target areas partially overlap with respect to an image captured by a camera. This apparatus performs pattern matching for each detection window using a reference pattern of a recognition target (such as a pedestrian). This apparatus integrates the results of each pattern matching for an area where pattern matching has been executed a plurality of times. Based on the integration result, the position of the pedestrian is specified.

JP 2008-21034 A JP 2009-70344 A

  In order to detect a detection target from an image, an object detection device using pattern matching learns a detection target pattern in advance using an image (sample image data) including the detection target. For example, when the detection target is a pedestrian, the sample image data is generated by cutting out a region including the pedestrian from a learning image obtained by photographing the pedestrian. The learning image is taken under a predetermined exposure condition.

  The object detection device detects a pedestrian from a captured image input from the camera after learning a pedestrian pattern. However, since the exposure condition when the captured image is generated does not match the exposure condition when the learning image is generated, the detection accuracy of the pedestrian may be lowered.

  For example, let us consider a case where a learning image is generated by shooting in a positive location and the object detection device detects a pedestrian from a shot image shot in the shade. In this case, since the exposure condition of the captured image is different from the exposure condition of the learning image, the object detection device cannot detect the pedestrian even though the captured image captured in the shade includes the pedestrian. There is a fear. Therefore, in order to improve the accuracy of detecting the detection target from the photographed image, it is desirable to use the photographed image photographed under various exposure conditions.

  In order to improve the detection accuracy, a method in which the camera switches a plurality of exposure conditions and generates a plurality of captured images photographed under each exposure condition is conceivable. However, with this method, it takes time to shoot and switch between exposure conditions.

  Also, a method of controlling the exposure of the camera so that the object detection device meets the exposure condition of the learning image can be considered. In this method, the object detection device analyzes a captured image input from the camera and feeds back the exposure condition to the camera. However, this method cannot be used when the camera does not accept external exposure condition control.

  In view of the above problems, an object of the present invention is to provide an object detection apparatus that can improve the detection accuracy of a detection target without changing the exposure conditions of the camera.

  In order to solve the above-mentioned problem, the invention according to claim 1 is an object detection device for detecting a detection target from a frame image, wherein a window region is set in the frame image and the image of the window region is included. A window region setting unit that acquires a one-window image, and if the pixel value of a pixel in the first window image is smaller than a correction reference value, a second window image is generated by acquiring a value smaller than the pixel value A pixel value correcting unit, a first identification value indicating the degree to which the detection target is present in the first window image, and a second identification value indicating the degree to which the detection target is present in the second window image Based on feature data indicating features of the detection target, and based on the first identification value and the second identification value, the detection target exists in the window region of the frame image. A determination unit that determines whether or not to

  Invention of Claim 2 is the object detection apparatus of Claim 1, Comprising: The said pixel value correction | amendment part WHEREIN: When the pixel value of the pixel in the said 1st window image is smaller than a zero reference value, the said pixel If the pixel value of the pixel is greater than or equal to the zero reference value and smaller than the correction reference value, the pixel value of the pixel is set to a value greater than zero. The zero reference value is smaller than the correction reference value.

  Invention of Claim 3 is the object detection apparatus of Claim 1 or 2, Comprising: The said determination part makes the said 1st identification value whether the said detection target exists in the said 1st window image. And determining whether or not the detection target exists in the second window image based on the second identification value, and the detection target is at least one of the first window image and the second window image. It is determined that the detection target exists in the window area of the frame image.

  Invention of Claim 4 is the object detection apparatus of Claim 1 or 2, Comprising: The said determination part, The integration part which calculates the integrated value of a said 1st identification value and a said 2nd identification value, And a final determination unit that determines whether or not the detection target exists in a window region in the frame image based on an integrated value.

  A fifth aspect of the present invention is the object detection device according to the fourth aspect, wherein the integration unit calculates a multiplication value obtained by multiplying the first identification value by a first weighting coefficient, and a second weighting coefficient. The integrated value is obtained by integrating the multiplied value obtained by multiplying the two identification values.

  A sixth aspect of the present invention is the object detection device according to any one of the first to fifth aspects, wherein the pixel value correction unit has a pixel value of a pixel in the first window image different from the correction reference value. When it is smaller than a reference value, a changed pixel value smaller than the pixel value is acquired to generate a third window image, and the identification value calculation unit includes the detection target in the third window image. A third identification value indicating a degree is calculated based on the feature data, and the determination unit determines whether the detection target exists in the window region of the frame image based on the first to third identification values. Determine whether or not.

  A seventh aspect of the present invention is the object detection device according to any one of the first to sixth aspects, wherein the frame image is generated by performing gray scale conversion on a captured image captured by a camera. .

  The invention according to an eighth aspect is the object detection device according to any one of the first to sixth aspects, wherein the frame image is generated by performing edge enhancement on a captured image captured by a camera.

  The invention according to claim 9 is an object detection device for detecting a detection target from a frame image, wherein a window region setting unit that sets a window region in the frame image, and a pixel of a pixel in the image having the window region When the value is smaller than the first correction reference value, a first window image is generated by acquiring a value smaller than the pixel value, and the pixel value of the pixel in the image having the window region is the first correction reference value. A pixel value correction unit that generates a second window image by acquiring a value smaller than the pixel value when the second correction reference value is different from the value, and the detection target is present in the first window image The identification value calculation that calculates the first identification value indicating the degree of the detection and the second identification value indicating the degree that the detection target exists in the second window image based on the feature data indicating the feature of the detection target Part, the first identification value and the Based on the second identification value, and a determination unit configured to determine whether or not present in the window area of the detection target is the frame image.

  In a tenth aspect of the present invention, in a computer mounted on an object detection apparatus that detects a detection target from a frame image, a window area is set in the frame image, and a first window image having an image of the window area is provided. Obtaining a second window image by obtaining a value smaller than the pixel value when the pixel value of the pixel in the first window image is smaller than the correction reference value; and the detection target Is a feature that indicates the characteristics of the detection object, and a first identification value that indicates the degree to which the detection object exists in the first window image and a second identification value that indicates the degree to which the detection object exists in the second window image. A step of calculating based on data, and a step of determining whether or not the detection target exists in a window region of the frame image based on the first identification value and the second identification value. Object detection process A gram.

  According to an eleventh aspect of the present invention, a step of setting a window area in the frame image in a computer mounted on an object detection apparatus that detects a detection target from the frame image, and a pixel of the pixel in the image having the window area When the value is smaller than the first correction reference value, a first window image is generated by acquiring a value smaller than the pixel value, and the pixel value of the pixel in the image having the window region is the first correction reference value. If the second correction reference value is different from the value, a step of generating a second window image by obtaining a value smaller than the pixel value, and a degree to which the detection target exists in the first window image Calculating a first identification value indicating a second identification value indicating a degree to which the detection target exists in the second window image based on feature data indicating a characteristic of the detection target; identification On the basis of the second identification value, which is the detection target object detection program for executing, and determining whether or not there in the window region of the frame image.

  The object detection apparatus according to the present invention generates a first window image having a window region in a frame image. When the pixel value of the pixel of the first window image is smaller than the correction reference value, the object detection device acquires the second window image by acquiring a value smaller than the pixel value. The first window image and the second window image correspond to images cut out from two captured images having different exposure conditions. The object detection device determines whether or not the detection target exists in the window region based on the first and second identification values calculated from the first window image and the second window image. Therefore, the detection accuracy of the detection target can be improved without changing the exposure condition of the camera.

It is a functional block diagram which shows the structure of the object detection apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the object detection apparatus shown in FIG. It is a figure which shows an example of the picked-up image input into the object detection apparatus shown in FIG. It is a figure which shows an example of the window image produced | generated by the object detection apparatus shown in FIG. It is a figure explaining the production | generation of the correction | amendment window image 24 produced | generated by the pixel value correction | amendment part shown in FIG. It is a figure explaining the production | generation of the correction | amendment window image 25 produced | generated by the pixel value correction | amendment part shown in FIG. It is a figure which shows an example of the sample image data used for the production | generation of the characteristic data shown in FIG. It is a flowchart of the determination process shown in FIG. It is a figure which shows an example of the coefficient table shown in FIG. It is a figure which shows an example of the identification value and multiplication value corresponding to the window image shown in FIG. It is a figure which shows an example of the detection result of the pedestrian in the object detection apparatus which concerns on the 2nd Embodiment of this invention. It is a flowchart of the determination process in the object detection apparatus which concerns on the 2nd Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
{1. overall structure}
FIG. 1 is a functional block diagram of an object detection apparatus 1 according to the first embodiment of the present invention. The object detection apparatus 1 detects a detection target from a color captured image captured by the camera 100. The object detection device 1 is mounted on a vehicle such as an automobile together with the camera 100. In the present embodiment, the detection target is a pedestrian.

  The camera 100 is installed on the dashboard of the automobile, and generates image data 20 by photographing the scenery in front. The image data 20 is a moving image composed of a plurality of frames, and is input to the object detection apparatus 1 in units of frames. The object detection device 1 determines whether or not a pedestrian exists in one frame (hereinafter referred to as “captured image 21”).

  As shown in FIG. 1, the object detection apparatus 1 includes an edge image generation unit 11, a window area setting unit 12, a pixel value correction unit 13, an identification value calculation unit 14, a determination unit 15, and a storage unit 16. Is provided.

  The edge image generation unit 11 converts the captured image 21 into a grayscale image, and generates an edge image 22 from the grayscale image. The edge image generation unit 11 generates an edge image 22 by performing processing for enhancing horizontal and vertical edges on the grace scale image. The edge image 22 is one frame image.

  The window area setting unit 12 sets a window area for the edge image 22 and generates a normal window image 23 by cutting out the set window area from the edge image 22. The window area is a unit area for detecting a pedestrian from the captured image 21. One normal window image 23 is generated for one window region. The generated normal window image 23 is supplied to the pixel value correction unit 13 and the identification value calculation unit 14.

  The pixel value correcting unit 13 generates corrected window images 24 and 25 by correcting the pixel value of each pixel in the normal window image 23. When the pixel value of the pixel included in the normal window image 23 is smaller than the correction reference value 35a, the pixel value correction unit 13 generates the correction window image 24 by replacing the pixel value with a value smaller than the original value. . When the pixel value of the pixel of the normal window image 23 is smaller than the correction reference value 35b, the pixel value correction unit 13 generates the correction window image 25 by replacing the pixel value with a value smaller than the original value.

  The correction reference values 35a and 35b are different from each other, and are recorded in the reference data 32 as pixel value correction references. The zero reference values 36a and 36b recorded in the reference data 32 will be described later.

  Hereinafter, when the normal window image 23 and the corrected window images 24 and 25 are collectively referred to, they are described as “window images 23 to 25”.

  The identification value calculation unit 14 uses the feature data 31 stored in the storage unit 16 to calculate an identification value 43 indicating the degree of presence of a pedestrian in the normal window image 23. The identification value 43 is a real number between 0 and 1. The closer the identification value 43 is to 1, the higher the possibility that a pedestrian exists in the normal window image 23. Similar to the calculation of the identification value 43, the identification value calculation unit 14 uses the feature data 31 to calculate identification values 44 and 45 indicating the degree to which pedestrians exist in the correction window images 24 and 25.

  The determination unit 15 calculates the integrated value by weighting and adding the identification values 43 to 45, and determines whether or not a pedestrian exists in the window region of the captured image 21 based on the calculated integrated value. The determination result is output from the determination unit 15 as result data 25.

  The storage unit 16 is, for example, a hard disk device or a flash memory. The storage unit 16 stores feature data 31, reference data 32, and a coefficient table 33. Details of each data stored in the storage unit 16 will be described later.

{2. Operation of Object Detection Device 1}
FIG. 2 is a flowchart showing the operation of the object detection apparatus 1. Each time the captured image 21 (frame) is input from the camera 100, the object detection device 1 executes the process shown in FIG. 2 and detects a pedestrian from the input captured image 21.

  FIG. 3 is a diagram illustrating an example of the captured image 21 input to the object detection device 1. As shown in FIG. 3, the captured image 21 has a coordinate system in which the upper left vertex is the origin, the horizontal direction is the x axis, and the vertical direction is the y axis. The window areas 51 to 54 arranged on the captured image 21 will be described later. Hereinafter, the process shown in FIG. 2 will be described in detail by taking as an example the case where the captured image 21 shown in FIG.

{2.1. Edge image generation}
The edge image generation unit 11 generates an edge image 22 from the color photographed image 21 (step S1). Specifically, the edge image generation unit 11 converts the captured image 21 into a grayscale image, and executes edge enhancement processing on the grayscale image. The gray scale image and the edge image 22 have the same coordinate system as the captured image 21. When the pixel value of each pixel of the grayscale image is f (x, y), the edge image generation unit 11 generates the edge image 22 by executing the processes of the following formulas (1) to (3). .

  The above equation (1) represents Sobel filter processing for emphasizing vertical edges. The above equation (2) represents Sobel filter processing for enhancing the edge in the horizontal direction. Expression (3) indicates that the pixel value of each pixel of the edge image 22 is calculated by adding the calculation result of Expression (1) and the calculation result of Expression (2).

{2.2. Setting of window area (step S2)}
The window area setting unit 12 receives the edge image 22 generated by the edge image generation unit 11. The window area setting unit 12 sets one window area for detecting a pedestrian in the input edge image 22 (step S2). In order to show the positional relationship between the detection target pedestrian and the set window area, the position of the window area set by the window area setting unit 12 is shown on the captured image 21 in FIG.

  As shown in FIG. 2, a plurality of window regions are set in the captured image 21 by repeating steps S <b> 2 to S <b> 7. The window area setting unit 12 sets the window area so that the captured image 21 is scanned from the upper left to the lower right. When a new window area is set, it is desirable that the window area setting unit 12 sets a new window area so as to overlap a part of the already set window area.

  Hereinafter, unless otherwise specified, the operation of the object detection apparatus 1 will be described using the case where the window area setting unit 12 sets the window area 52 as an example.

  FIG. 4 is a diagram showing a normal window image 23 and correction window images 24 and 25 corresponding to each window region. The window area setting unit 12 generates the normal window image 23 by cutting out the window area 52 from the edge image 22 (step S3). As shown in FIG. 4, one normal window image 23 is generated from one window region 52. The generated normal window image 23 is supplied to the pixel value correction unit 13 and the identification value calculation unit 14.

{2.3. Generation of Correction Window Image (Step S4)}
Reference is again made to FIG. The pixel value correcting unit 13 generates corrected window images 24 and 25 by correcting the pixel value of each pixel of the normal window image 23 input from the window area setting unit 12. A coring process is used for correcting the pixel value of each pixel of the normal window image 23. The parameters used in the coring process are different from each other in generating the correction window images 24 and 25.

  The generation of the correction window image 24 will be described with reference to FIG. FIG. 5 is a diagram for explaining a coring process when the correction window image 24 is generated. As shown in FIG. 5, in the grayscale image, the maximum pixel value is 255 and the minimum value is 0.

  In the coring process, two parameters are used. One parameter is a parameter for determining whether to maintain the pixel value of the pixel as it is or to change the pixel value of the pixel to a value smaller than the current pixel value (correction reference value 35a). The other parameter is a parameter for determining whether or not the corrected pixel value is set to zero when the pixel value is corrected (zero reference value 36a). Since the correction reference value 35b and the zero reference value 36b are used when the correction window image 25 is generated, they will be described later.

  In FIG. 5, a one-dot chain line indicates a relationship between the pixel value before correction and the pixel value after correction when the correction reference value 35a and the zero reference value 36a are set to zero. In this case, since the pixel value after correction is equal to the pixel value before correction, the alternate long and short dash line is a straight line having an inclination of 1.

  In FIG. 5, a solid line 56 indicates the relationship between the pixel value before correction and the pixel value after correction when the correction window image 24 is generated. The pixel values before correction corresponding to the inflection points 56A and 56B of the solid line 56 correspond to the correction reference value 35a and the zero reference value 36a. In the generation of the correction window image 24, the correction reference value 35a is set to 100, and the zero reference value 36a is set to 50.

  The pixel value correction unit 13 corrects the pixel value of each pixel in the normal window image 23 using the correspondence relationship represented by the solid line 56.

  Specifically, when the pixel value of the target pixel in the normal window image 23 is the correction reference value 35a or more, that is, when the pixel value of the target pixel is 100 or more, the pixel value of the target pixel is not changed. . When the pixel value of the target pixel is smaller than the zero reference value 36a, that is, when the pixel value of the target pixel is smaller than 50, the pixel value of the target pixel is corrected to zero.

  When the pixel value of the target pixel is smaller than the correction reference value 35a (100) and equal to or greater than the zero reference value 36a (50), that is, when the pixel value of the target pixel is smaller than 100 and equal to or greater than 50 The pixel value of the pixel is corrected based on the correspondence represented by the solid line 56 from the inflection points 56A to 56B. When the pixel value before correction is P, the slope of the solid line 56 in the range of 50 ≦ P <100 is larger than 1. Therefore, the pixel value of the target pixel is corrected to a value larger than zero and smaller than the current pixel value. In the range of 50 ≦ P <100, the solid line 56 may not be a straight line as long as the pixel value before correction and the pixel value after correction correspond one-to-one.

  Next, generation of the correction window image 25 will be described with reference to FIG. FIG. 6 is a diagram for explaining coring processing when the correction window image 25 is generated.

  In FIG. 6, the solid line 57 indicates the relationship between the pixel value before correction and the pixel value after correction when the correction window image 25 is generated. The pixel values before correction corresponding to the inflection points 57A and 57B of the solid line 57 correspond to the correction reference value 35b and the zero reference value 36b. In the generation of the correction window image 25, the correction reference value 35b is set to 150, and the zero reference value 36b is set to 100.

  The pixel value correction unit 13 corrects the pixel value of each pixel in the normal window image 23 using the correspondence relationship represented by the solid line 57.

  Specifically, when the pixel value of the target pixel in the normal window image 23 is the correction reference value 35b or more, that is, when the pixel value of the target pixel is 150 or more, the pixel value of the target pixel is not changed. When the pixel value of the target pixel is smaller than the zero reference value 36b, when the pixel value of the target pixel is 100 or less, the pixel value of the target pixel is corrected to zero.

  When the pixel value of the target pixel is smaller than the correction reference value 35b and equal to or greater than the zero reference value 36b, that is, when the pixel value of the target pixel is smaller than 150 and equal to or greater than 100, the pixel value of the target pixel is Correction is performed based on the correspondence relationship indicated by the solid line 57 from the inflection points 57A to 57B. When the pixel value before correction is P, the slope of the solid line 57 in the range of 100 ≦ P <150 is larger than 1. Therefore, the pixel value of the target pixel is corrected to a value larger than zero and smaller than the current pixel value.

  Window images 23 to 25 corresponding to the window region 52 correspond to window regions cut out from the three captured images 21 captured at the same timing under different exposure conditions. This will be described in detail below.

  The pixel value correction unit 13 maintains the pixel value P when the pixel value P of the normal window image 23 is equal to or greater than the correction reference value 35a, and decreases the pixel value P when the pixel value P is smaller than the correction reference value 35a. By doing so, the correction window image 24 is generated. As a result of this processing, the difference between the minimum pixel value (0) and the maximum pixel value (255) is the same in the normal window image 23 and the correction window image 24, but the distribution of intermediate pixel values in the correction window image 24 is It changes from the distribution of intermediate pixel values in the normal window image 23. The intermediate pixel value is a pixel value other than the minimum pixel value and the maximum pixel value. As shown in FIG. 5, in the normal window image 23, the distribution of intermediate pixel values ranges from 1 to 254. On the other hand, in the corrected window image 24, the distribution of intermediate pixel values is in a range from pixel values (51) to 254 that are one greater than the zero reference value 36a. Since the distribution range of the intermediate pixel values in the correction window image 24 is narrower than the distribution range of the intermediate pixel values in the normal window image 23, the brightness difference of the correction window image 24 is steeper than the brightness difference of the normal window image 23. Become.

  The pixel value correction unit 13 maintains the pixel value P when the pixel value P of the normal window image 23 is equal to or greater than the correction reference value 35b, and decreases the pixel value P when the pixel value P is smaller than the correction reference value 35b. By doing so, the correction window image 24 is generated. Therefore, the brightness difference of the correction window image 25 is steeper than the brightness difference of the normal window image 23 as in the correction window image 24. Further, since the correction reference value 35b and the zero reference value 36b are larger than the correction reference value 35a and the zero reference value 36a, the contrast difference of the correction window image 25 is steeper than the contrast difference of the correction window image 24.

  As described above, the brightness differences of the window images 23 to 25 are different from each other. In general, the contrast of the captured image 21 generated by the camera 100 changes according to the exposure condition at the time of shooting. Accordingly, the window images 23 to 25 correspond to window regions cut out from each of the three captured images 21 captured at the same timing under different exposure conditions.

  In addition, since the pixel values of the pixels of the normal window image 23 are partially maintained in the correction window images 24 and 25, the characteristics of the object included in the normal window image 23 are also maintained in the correction window images 24 and 25. Is done. In other words, the correction window images 24 and 25 having a difference in brightness and darkness from the normal window image 23 can be generated without impairing the characteristics of the object included in the normal window image 23 by the processing of the three functional units described above.

{2.4. Calculation of identification value (step S5)
The identification value calculation unit 14 calculates the identification values 43 to 45 corresponding to the window images 23 to 25 corresponding to the window region 52 using the feature data 31 stored in the storage unit 16 (step S5). The identification values 43 to 45 indicate the degree to which a pedestrian exists in each of the window images 23 to 25, and are numerical values of 0 or more and 1 or less. The identification values 43 to 45 are closer to 1 as the pedestrian is more likely to be present in the window image.

  The identification value calculation unit 14 uses a neural network as an algorithm for calculating the identification values 43 to 45. A neural network is one type of pattern matching. Therefore, the identification value calculation unit 14 learns the characteristics of the pedestrian and generates the feature data 31 before starting the process of detecting the pedestrian from the captured image 21 generated by the camera 100. Hereinafter, generation of the feature data 31 will be described.

  FIG. 7 is a diagram illustrating an example of the sample image data 55 used by the identification value calculation unit 14 for pedestrian learning. In the three sample image data 55 shown in FIG. 7, the size in the vertical direction and the size in the horizontal direction are the same as the sizes of the window areas 51 to 54. The sample image data 55 includes pedestrians, but the pedestrian edge strength in each sample image data 55 is not normalized. “Edge strength is normalized” means that the pedestrian's edge strength is adjusted to be within a preset range in each of the sample image data 55.

  That is, in the present embodiment, the sample image data 55 is generated from images taken under various exposure conditions. The identification value calculation unit 14 learns pedestrian patterns included in images taken under various exposure conditions by using the non-normalized sample image data 55.

{2.5. Determination process (step S6)}
The determination unit 15 inputs identification values 43 to 45 corresponding to the three window images 23 to 25 from the identification value calculation unit 14. The determination unit 15 determines whether or not there is a pedestrian in the window area 52 based on the identification values 43 to 45 (step S6). Details of step S6 will be described later.

  Next, the window area setting unit 12 confirms whether or not the setting of the window area for the captured image 21 is completed (step S7). When the setting of the window area is not completed (No in step S7), the window area setting unit 12 returns to step S2 in order to set a new window area. On the other hand, when the setting of the window area is completed (Yes in step S7), the object detection device 1 ends the process shown in FIG. When a new frame (captured image 21) is input, the object detection device 1 executes the process illustrated in FIG. 2 again.

  Details of the determination process (step S6) will be described below. FIG. 8 is a flowchart of the determination process (step S6).

  FIG. 9 is a diagram illustrating an example of the coefficient table 33 stored in the storage unit 16. As shown in FIG. 9, weighting coefficients 34 a to 34 c corresponding to the window images 23 to 25 are set in the coefficient table 33. The determination unit 15 acquires the weighting coefficients 34a to 34c from the storage unit 16 (step S611).

  The weighting coefficient 34a set for the window image (normal window image 23) whose pixel value is not corrected is 1.8. The weighting coefficient 34b set for the correction window image 24 is 1.6, and the weighting coefficient 34c set for the correction window image 25 is 1.4. In the example shown in FIG. 9, the weighting coefficient 34a is the maximum, but each weighting coefficient may be changed as appropriate.

  The determination unit 15 obtains multiplication values corresponding to the respective window images 23 to 25 by multiplying the identification values 43 to 45 by the weighting coefficients 34a to 34c, respectively (step S612). FIG. 10 is a table showing calculation results of identification values and multiplication values of the window images 23 to 25. In FIG. 10, the calculation results in each of the window regions 52 to 54 are shown.

  In FIG. 10, the numerical values (without parentheses) of the window images 23 to 25 indicate multiplication values. Numerical values in parentheses of the window images 23 to 25 indicate identification values of the window images. The determination unit 15 accumulates the multiplication values of the three window images 23 to 25 corresponding to the window region 52 (step S613). The integrated value shown in FIG. 10 is the sum of the multiplied values of the window images.

  The determination unit 15 compares the integrated value corresponding to the window region 52 with a preset threshold value in order to determine the presence or absence of a pedestrian in the window region 52 (step S614). It is assumed that the threshold set in the determination unit 15 is 2.5. In this case, since the integrated value (3.82) of the window area 52 is larger than the threshold value (Yes in step S614), the determination unit 15 determines that a detection target (pedestrian) exists in the window area 52 ( Step S615).

  Similarly, the determination unit 15 performs a determination process (step S6) using the identification values 43 to 45 of the window images 23 to 25 corresponding to the window region 53. In this case, the integrated value corresponding to the window region 53 is 3.14. Since the integrated value corresponding to the window area 54 is larger than the threshold value (Yes in step S614), the determination unit 15 determines that there is a pedestrian in the window area 53 (step S615).

  The determination unit 15 executes determination processing (step S6) using the identification values 43 to 45 of the window images 23 to 25 corresponding to the window region 54. In this case, the integrated value corresponding to the window area 52 is 1.23. Since the integrated value corresponding to the window area 54 is equal to or less than the threshold value (Yes in step S614), the determination unit 15 determines that there is no pedestrian in the window area 54 (step S616).

  As shown in FIG. 4, the pedestrian is present in the window areas 52 and 53 and is not present in the window area 54. As a result of the determination process (step S6), it can be seen that the presence or absence of a pedestrian in the window regions 52 to 54 is accurately determined.

  Thus, the object detection apparatus 1 determines the presence or absence of a pedestrian in the window region by weighting and integrating the identification values 43 to 45 of the three window images and comparing the integrated value with the threshold value. . Thereby, the object detection apparatus 1 can determine with high precision whether a pedestrian exists in a detection window area | region. The reason for this will be described with reference to FIGS.

  As described above, since the plurality of sample image data 55 is not normalized, the edge strength of the normal window image 23 is likely to be substantially equal to the edge strength of any one of the sample image data 55. Become.

  When the edge strength of the normal window image 23 is substantially equal to the edge strength of any one of the sample image data 55, the identification value calculation unit 14 is likely to have a person as the identification value 43 of the normal window image 23. Is calculated (for example, 0.5 or more). Similarly, the identification value calculation unit 14 calculates a value of 0.5 or more for the identification values 44 and 45 of the correction window images 24 and 25. As a result, the integrated value corresponding to the window region 52 can greatly exceed the threshold value (2.5).

  On the other hand, there are no pedestrians and trees in the window area 54. In the identification value calculation unit 14, the identification values 43 to 45 corresponding to the window area 54 are values (for example, 0.5 or less) that have a low possibility that a person exists. In this case, the integrated value corresponding to the window region 54 is significantly below the threshold value.

  In this way, by integrating the identification values of the normal window image 23 and the corrected window images 24 and 25 by weighting, the integrated value of the window area including the pedestrian and the integrated value of the window area not including the pedestrian are calculated. The difference can be highlighted. Therefore, a pedestrian can be detected with high accuracy.

  Note that any one of the window images 23 to 25 corresponding to the window region 54 may have a value larger than 0.5, unlike the value shown in FIG. For example, the tree pattern of the normal window image 23 corresponding to the window region 54 is similar to the pattern of a pedestrian included in a certain sample image data 55, and the edge strength of the normal window image 23 corresponding to the window region 54 is A case where the edge strengths of the sample image data 55 are approximately equal may occur accidentally. In this case, the identification value 43 of the window area 54 is a value close to 1. However, like the identification value 43, the identification values 44 and 45 are unlikely to be 0.5 or more. Therefore, since it is suppressed that the integrated value of the window area | region 54 exceeds, it becomes possible to prevent a misdetection.

  Further, even when any one of the identification values 43 to 45 corresponding to the window region 52 is 0.5 or less, the possibility that the other two identification values are 0.5 or less is low. In such a case, since the integrated value of the window area 52 is suppressed from falling below the threshold value, the object detection device 1 does not move even though the pedestrian is actually present in the window area. It is possible to prevent erroneous determination that no pedestrian exists.

  As described above, the object detection apparatus 1 according to the present embodiment generates the correction window images 24 and 25 using the two types of coring processes for the normal window image 23, and each of the window images 23 to 25. The identification values 43 to 45 are calculated from the above. As a result, the object detection apparatus 1 performs the same processing as when a pedestrian is detected using a plurality of captured images 21 that have different exposure conditions and are captured at the same timing. 21 can be used. Therefore, it is possible to improve the detection accuracy of pedestrians without controlling the exposure conditions of the camera 100.

[Second Embodiment]
Hereinafter, a second embodiment of the present invention will be described. The second embodiment will be described with a focus on differences from the first embodiment.

  In the second embodiment, the edge strength is normalized in the sample image data learned by the identification value calculation unit 14. Moreover, the determination part 15 determines whether a pedestrian exists for every three window images in determination processing (step S6). The determination unit 15 determines that there is a pedestrian in the window area when there is even one window image in which it is determined that there is a pedestrian.

  Hereinafter, the present embodiment will be described in detail by taking as an example the case of determining the presence or absence of a pedestrian in the window region 52.

  As described above, the identification value calculation unit 14 generates the feature data 31 by learning in advance the sample image data 55 with normalized edge strength. The normalized sample image data 55 is adjusted so that the edge intensity of each image data is in a certain range. The identification value calculation unit 14 calculates identification values 43 to 45 corresponding to the three window images 23 to 25 using the generated feature data 31.

  FIG. 11 is a diagram illustrating identification values 43 to 45 and determination results of the window areas 52 to 54 in the present embodiment. The identification value of each window image shown in FIG. 11 is the same as the value shown in FIG.

  The object detection device 1 executes the processes of steps S1 to S5 shown in FIG. 2 to calculate the identification values 43 to 45 of the window images 23 to 25 in the window area 52. And the determination part 15 performs the process shown in FIG. 12 as a determination process (step S6).

  FIG. 12 is a flowchart showing the contents of the determination process in the second embodiment. Hereinafter, the case where the presence or absence of the pedestrian in the window area 52 is determined will be described as an example. The determination unit 15 specifies a window image to be determined from the window images 23 to 25 (step S621), and determines whether a pedestrian exists in the specified window image (step S622). The determination unit 15 is preset with a threshold value for determining whether or not there is a pedestrian for each window image. The determination unit 15 compares the specified window image identification value with a threshold value to determine whether or not there is a pedestrian in each window image.

  Here, consider a case where the threshold value is set to 0.8 and the normal window image 23 is designated as a determination target. Since the identification value (0.92) of the normal window image 23 exceeds the threshold value (Yes in Step S622), the determination unit 15 determines that a pedestrian exists in the normal window image 23 (Step S623).

  Of the normal window image 23 and the corrected window images 24 and 25 in the window region 52, the identification value of the window image having an edge strength substantially equal to the edge strength of the learned sample image data 55 is considered to have a value close to 1. It is done. Therefore, when there is even one window image for which an identification value exceeding the threshold is calculated, the determination unit 15 determines that a pedestrian exists in the window region 52.

  Note that the window area 54 does not include a pedestrian. Therefore, even if the edge strength of any one of the three window images corresponding to the window region 54 matches the edge strength of the sample image data, there is little possibility that the value will exceed the threshold value. As described above, when learning the normalized sample image data 55, the presence / absence of a pedestrian in each window region can be calculated with high accuracy without using weighting calculation as in the above embodiment. .

  The determination unit 15 checks whether or not all the window images 23 to 25 corresponding to the window region 52 have been designated (step S625). If not all window images have been designated (No in step S625), the determination unit 15 returns to the process of step S621. Thereby, the process of step S622 is performed with respect to all the window images. As shown in FIG. 11, the identification values of the correction window images 24 and 25 are 0.73 and 0.54, which are not more than the threshold value (No in step S622). For this reason, the determination unit 15 determines that the pedestrian does not exist in any of the correction window images 24 and 25 (step S624).

  When all the window images have been designated (Yes in step S625), the determination unit 15 determines whether or not there is a pedestrian in the detection window region 41 based on the determination results of the window images 23 to 25. . Specifically, the determination unit 15 determines that a pedestrian exists in the window region 52 when there is one or more window images determined to have a pedestrian (Yes in step S626) (step S627). If there is no window image at which it is determined that there is a pedestrian (No in step S626), the determination unit 15 determines that there is no pedestrian in the window area 52 (step S628).

  Note that the determination unit 15 may determine whether or not there is a pedestrian in the window area using another determination criterion in step S626. For example, when the number of window images determined to have a pedestrian is more than a majority, the determination unit 15 may determine that there is a pedestrian in the window region 52. Alternatively, when there is at least one window image that is determined to have no pedestrian, the determination unit 15 may determine that no pedestrian exists in the window region 52. Further, the determination unit 15 may set a threshold value according to the correction reference value and the zero reference value in the process of step S622. Further, the determination unit 15 may multiply the window images 23 to 25 in the window region 52 by weighting coefficients 34a to 34c, respectively, in step S622. In this case, since the multiplication value may be 1 or more, a threshold value corresponding to the multiplication value is set.

  As described above, in the second embodiment, the determination unit 15 performs the process of determining whether or not there is a pedestrian for each window image, and based on the determination result of each window image, the window region It is finally determined whether or not there is a pedestrian. The object detection apparatus 1 according to the second embodiment can determine whether or not there is a pedestrian in the detection window area without multiplying the identification value of each window image by a weighting coefficient.

  In the above embodiment, the pixel value correction unit 13 may not use the correction reference value. That is, when the pixel value of the pixel in the normal window image is smaller than a predetermined reference value, the pixel value correction unit 13 may generate a correction window image by acquiring a value smaller than the pixel value. In this case, it is possible to generate a correction window image having a sharper contrast than the correction window image generated when the correction reference value is used without changing a pixel value larger than the zero reference value. Can do.

  In the first and second embodiments, the example has been described in which the object detection device 1 executes processing for detecting a pedestrian from the captured image 21 captured by a camera (not shown) in real time. However, the object detection apparatus 1 may perform the above-described object detection process on an image stored in a storage device such as a hard disk device.

  Moreover, although the example which produces | generates the correction window images 24 and 25 from the normal window image 23 was demonstrated in the said embodiment, it is not restricted to this. The number of correction window images may be one, or may be three or more. When three or more correction window images are generated, the coring process for generating each correction window image is different from each other.

  Moreover, although the pixel value correction | amendment part 13 demonstrated the example which performs a coring process with respect to the normal window image 23 in the said embodiment, it is not restricted to this. The coring process may be executed on the edge image 22 (frame image). In this case, a corrected window image may be generated by cutting out the window region from the edge image that has been subjected to the coring process.

  Moreover, although the said embodiment demonstrated the example which calculates the identification values 43-45 from the window images 23-25, it is not restricted to this. The object detection device 1 does not have to use the normal window image 23 for detecting a pedestrian. In this case, the object detection apparatus 1 executes the determination process (step S6) using the identification values 44 and 45 of the correction window images 24 and 25.

  In the object detection apparatus 1 described in the above embodiment, each functional unit may be individually made into one chip by a semiconductor device such as an LSI, or may be made into one chip so as to include a part or the whole. . The method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  In addition, a part or all of the processing of each functional block of the above embodiment may be realized by a program. A part or all of the processing of each functional block in the above embodiment is performed by a central processing unit (CPU) in the computer. In addition, a program for performing each processing is stored in a storage device such as a hard disk or a ROM, and is read out and executed in the ROM or the RAM.

  Each processing of the above embodiment may be realized by hardware, or may be realized by software (including a case where the processing is realized together with an OS (Operating System), middleware, or a predetermined library). Further, it may be realized by mixed processing of software and hardware. Needless to say, when the object detection apparatus 1 according to the above embodiment is realized by hardware, it is necessary to adjust the timing for performing each process.

  A computer program that causes a computer to execute the above-described method and a computer-readable recording medium that records the program are included in the scope of the present invention. Here, examples of the computer-readable recording medium include a flexible disk, a hard disk, a CD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, a BD (Blu-ray Disc), and a semiconductor memory. .

  The computer program is not limited to the one recorded on the recording medium, and may be transmitted via a telecommunication line, a wireless or wired communication line, a network represented by the Internet, or the like.

DESCRIPTION OF SYMBOLS 1 Object detection apparatus 11 Edge image generation part 12 Window area | region setting part 13 Pixel value correction | amendment part 14 Discrimination value calculation part 15 Determination part 16 Memory | storage part 31 Feature data 35a, 35b Correction reference value 36a, 36b Zero reference value 100 Camera

Claims (9)

An object detection device that detects a detection target from a frame image generated by performing grayscale conversion and edge enhancement on a captured image captured by a camera ,
A window area setting unit that sets a window area in the frame image and obtains a first window image having an image of the window area;
A pixel value correction unit that generates a second window image by obtaining a value smaller than the pixel value when the pixel value of the pixel in the first window image is smaller than the correction reference value;
A first identification value indicating the degree to which the detection object exists in the first window image and a second identification value indicating the degree to which the detection object exists in the second window image are characterized by the detection object. An identification value calculation unit for calculating based on feature data indicating
A determination unit that determines whether or not the detection target exists in a window region of the frame image based on the first identification value and the second identification value;
An object detection apparatus comprising: The object detection device according to claim 1,
When the pixel value of the pixel in the first window image is smaller than a zero reference value, the pixel value correction unit acquires a pixel value in which the pixel value of the pixel is set to zero, and is equal to or greater than the zero reference value. When the pixel value of the pixel is set to a value greater than zero when the pixel value is smaller than the correction reference value,
The zero reference value is an object detection device that is smaller than the correction reference value. The object detection device according to claim 1 or 2,
The determination unit determines whether the detection target is present in the first window image based on the first identification value, and determines whether the detection target is present in the second window image. 2 based on the identification value, and when it is determined that the detection target exists in at least one of the first window image and the second window image, it is determined that the detection target exists in the window region of the frame image Object detection device. The object detection device according to claim 1 or 2,
The determination unit
An integration unit for calculating an integrated value of the first identification value and the second identification value;
Based on the integrated value, a final determination unit that determines whether or not the detection target exists in a window region in the frame image;
An object detection apparatus including: The object detection device according to claim 4,
The integration unit obtains the integrated value by integrating a multiplication value obtained by multiplying the first identification value by a first weighting coefficient and a multiplication value obtained by multiplying the second identification value by a second weighting coefficient. Detection device. The object detection device according to any one of claims 1 to 5,
When the pixel value of the pixel in the first window image is smaller than a reference value different from the correction reference value, the pixel value correction unit acquires a changed pixel value smaller than the pixel value, thereby obtaining a third window image. Produces
The identification value calculation unit calculates a third identification value indicating the degree to which the detection target exists in the third window image based on the feature data,
The said determination part is an object detection apparatus which determines whether the said detection target exists in the said window area | region of the said frame image based on the said 1st-3rd identification value. An object detection device that detects a detection target from a frame image generated by performing grayscale conversion and edge enhancement on a captured image captured by a camera,
  A window area setting unit for setting a window area in the frame image;
  When a pixel value of a pixel in the image having the window region is smaller than a first correction reference value, a first window image is generated by acquiring a value smaller than the pixel value, and the image having the window region A pixel value correction unit that generates a second window image by obtaining a value smaller than the pixel value when the pixel value of the pixel is smaller than a second correction reference value different from the first correction reference value;
  A first identification value indicating the degree to which the detection object exists in the first window image and a second identification value indicating the degree to which the detection object exists in the second window image are characterized by the detection object. An identification value calculation unit for calculating based on feature data indicating
  A determination unit that determines whether or not the detection target exists in a window region of the frame image based on the first identification value and the second identification value;
An object detection apparatus comprising: In a computer mounted on an object detection device that detects a detection target from a frame image generated by performing grayscale conversion and edge enhancement on a captured image captured by a camera,
  Setting a window area in the frame image and obtaining a first window image having an image of the window area;
  Generating a second window image by obtaining a value smaller than the pixel value when the pixel value of the pixel in the first window image is smaller than the correction reference value;
  A first identification value indicating the degree to which the detection object exists in the first window image and a second identification value indicating the degree to which the detection object exists in the second window image are characterized by the detection object. Calculating based on feature data indicating
  Determining whether or not the detection target exists in a window region of the frame image based on the first identification value and the second identification value;
An object detection program that executes In a computer mounted on an object detection device that detects a detection target from a frame image generated by performing grayscale conversion and edge enhancement on a captured image captured by a camera,
  Setting a window region in the frame image;
  When a pixel value of a pixel in the image having the window region is smaller than a first correction reference value, a first window image is generated by acquiring a value smaller than the pixel value, and the image having the window region Generating a second window image by obtaining a value smaller than the pixel value when the pixel value of the pixel is smaller than a second correction reference value different from the first correction reference value;
  A first identification value indicating the degree to which the detection object exists in the first window image and a second identification value indicating the degree to which the detection object exists in the second window image are characterized by the detection object. Calculating based on feature data indicating
  Determining whether or not the detection target exists in a window region of the frame image based on the first identification value and the second identification value;
An object detection program that executes