JP3948450B2 - Object detection apparatus and object detection method - Google Patents

Description

  The present invention relates to an object detection device and an object detection method mounted on a vehicle.

Conventionally, an object detection device mounted on a vehicle is known (for example, Patent Document 1 ).

In the technique described in Patent Document 1 , a front image of a vehicle is captured to obtain a thermal image (an image having thermal data ahead of the vehicle), and a detection target candidate region having human thermal data is extracted from the thermal image. Then, the aspect ratio of the detection target object candidate area is calculated, and when the calculated aspect ratio matches the aspect ratio of the person, a person is detected from the detection target object candidate area.
JP 2001-28050 A

However, this technique performs the detection based only on the aspect ratio of the detection target object candidate region, and thus has the following problems.

  That is, in the technique described in Patent Document 1, an object other than a person and having an aspect ratio comparable to that of a person, for example, a part of a pillar or a wall (particularly, a part having a different material from other parts) is erroneously detected as a person. There was a possibility.

  The present invention has been made to solve such a conventional problem, and a main object thereof is to provide an object detection device and an object detection method capable of detecting a person more reliably. It is.

In order to achieve the above object, the invention described in the claims of the present application is an object detection device mounted on a vehicle, wherein an image acquisition means for observing the periphery of the vehicle and acquiring an image having thermal data around the vehicle is provided. Detecting from the detection object candidate area based on at least one of the area extraction means for extracting the detection object candidate area having human thermal data from the image and the position and size of the detection object candidate area A position calculating means for calculating a position of the detected object, a surrounding of the vehicle is scanned with a predetermined detection wave, a measurement point area reflecting the detection wave is detected, and the detected measurement point area detection wave output means for detecting the position, of the measurement point region, position of extracting the measuring point area to be in the same position as the position of the detected object to be detected from the detection target candidate region as the same candidate region Means out, the width of the detection object detected from the detection object candidate region, consistent with the width of the same candidate region feature quantity of the detection object candidate region coincides with the feature value of the person, and the position calculated Detecting means for detecting a person from the detection object candidate area or the same candidate area when at least one of the movement of the position calculated by the means or the movement of the position of the same candidate area matches the walking speed of the person ; The feature amount is at least one of the thermal data and the aspect ratio .

In the invention described in the claims of the present application, the following effects can be mainly obtained. That is, a detection object that is different from a person and that moves in the same manner as a person is not erroneously detected as a person. In addition, a detection object having the same size as a person and moving differently from the person is not erroneously detected as a person. Therefore, it is possible to reliably detect a person.

(First embodiment)
A first embodiment of the present invention will be described below with reference to the drawings. First, based on FIGS. 1-14, the main function of the component of the object detection apparatus 1 which concerns on one embodiment of this invention and each component is demonstrated.

  FIG. 1 is a block diagram illustrating the configuration of the object detection apparatus 1, FIG. 2 is a side view illustrating the installation positions of the camera 10 and the detection wave output unit 13, and FIG. 3 illustrates the camera 10 and the detection wave output unit 13. It is a top view which shows an installation position. 4-5 is explanatory drawing which shows an example of the thermal image A acquired with the camera 10. As shown in FIG. In the following description, as shown in FIGS. 2 to 3, the point at the tip of the detection wave output unit 13 is the origin, and the direction parallel to the central axis of the detection wave output unit 13 and the optical axis of the camera 10 is used. A coordinate system in which a direction perpendicular to the z axis and the z axis and parallel to the road surface is the x axis, and a direction perpendicular to the z axis and the road surface is the y axis is a reference coordinate system. Here, the optical axis of the camera 10 is an axis that passes through the center of the camera lens of the camera 10 and is perpendicular to the camera lens. The central axis of the detection wave output unit 13 is an axis that passes through the center of the tip surface of the detection wave output unit 13 and is perpendicular to the tip surface.

  Further, as shown in FIG. 4, a coordinate system in which the upper left end of the thermal image A is the origin, the direction from the upper end to the lower end of the thermal image A is the y axis, and the direction from the left end to the right end of the thermal image A is the x axis. Is the thermal image coordinate system.

  As shown in FIG. 1, the object detection apparatus 1 includes a camera (image acquisition unit) 10, an image memory 11, a detection object temperature zone region extraction unit (region extraction unit) (hereinafter referred to as “region extraction unit”). 12, a detection wave output unit (detection wave output unit) 13, a radar measurement result storage memory (hereinafter referred to as “storage memory”) 14, and a detection object distance measurement unit (position calculation unit, (Position extraction means) (hereinafter referred to as “distance measurement section”) 15, identical object determination section 16, object detection section (detection means) 17, and detection result output section 18.

  The camera 10 is installed in the front part of the vehicle P so that the optical axis is parallel to the central axis of the detection wave output unit 13 and the road surface, as shown in FIGS. Further, the front of the vehicle P is observed, and a thermal image A having thermal data ahead of the vehicle P is acquired in the form of thermal image data. An example of the thermal image A is shown in FIG. FIG. 4 shows a thermal image A acquired by the camera 10 when there are two persons and a wall in front of the vehicle P. The thermal image A includes person images a0 to a1 and wall images a3 to a4.

  The thermal image data is composed of a plurality of pixel data, and each pixel data includes coordinate data and thermal data. Then, the camera 10 outputs the thermal image data to the image memory 11. The image memory 11 stores thermal image data given from the camera 10.

  The region extraction unit 12 acquires thermal image data from the image memory 11. And based on the said thermal image data, the detection target object candidate area | region which has a person's thermal data is extracted from the thermal image A. FIG. Here, the thermal data value of the person is about 36 (degrees), but the longer the distance from the vehicle P to the person, the smaller the thermal data value that the person image on the thermal image A has. Therefore, the region extraction unit 12 extracts pixels having thermal data values of around 36 (degrees) from the thermal image A, and puts adjacent pixels together into one region, so that a detection target candidate region is obtained. To extract.

  Then, the region extraction unit 12 calculates the size (specifically, the width and height) of the detection target object candidate region and the position on the thermal image A based on the pixel data of the detection target candidate region. To do. Then, detection target candidate area data relating to the calculation result and pixel data included in the detection target candidate area is generated and output to the distance measuring unit 15.

  For example, when the thermal image A is the one shown in FIG. 4, the region extraction unit 12 extracts the detection target candidate region A0 including the person image a0 and the person image a1 from the thermal image A as shown in FIG. The detection object candidate area A1 to be included is extracted. In addition, since the thermal data which wall image a3-a4 has differs from the thermal data which person image a0-a1 has, wall image a3-a4 is not contained in detection target object candidate area | region A0-A1. Then, the region extraction unit 12 calculates the position (xc0, yc0), the width wc0, and the height hc0 of the detection target candidate region A0, and similarly, the position (xc1, yc1) of the detection target candidate region A1. The width wc1 and the height hc1 are calculated. Then, detection target candidate area data relating to the calculation result and pixel data included in the detection target candidate areas A0 to A1 is generated and output to the distance measuring unit 15. In addition, depending on the environment around the vehicle P, an object other than a person (for example, a vehicle) may have a heat similar to that of a person. In this case, the region extraction unit 12 may include an object other than a person in the thermal image A. There is a possibility that the drawn area is extracted as a detection object candidate area. Therefore, the region extraction unit 12 calculates the aspect ratio of the detection target candidate region so that only the detection target candidate region in which the person is drawn can be output, and the calculated aspect ratio is the aspect ratio of the person. Only the detection object candidate region that matches the ratio may be output to the distance measurement unit 15.

  The detection wave output unit 13 shown in FIG. 1 is, for example, a laser radar or a millimeter wave radar, and the vehicle has an optical axis parallel to the road surface and the optical axis of the camera 10 as shown in FIGS. Installed in front of P. Then, scanning is performed with a predetermined detection wave (for example, a laser or a millimeter wave), and a measurement point region that reflects the detection wave is detected. Then, the size (specifically, the width of the measurement point area, etc.) and the position (specifically, the z direction distance and direction from the vehicle P to the measurement point area) of the detected measurement point area are detected. To do. Further, the reflection intensity of the detection wave in the measurement point region is detected. Then, measurement point area data relating to the detected size, position, and reflection intensity is generated and output to the storage memory 14.

  FIG. 6 shows an example of the measurement point region acquired by the detection wave output unit 13 when the detection wave output unit 13 is a laser radar. FIG. 7 shows a thermal image A acquired by the camera 10 in the case shown in FIG. In FIG. 6, in addition to the persons Q0 to Q1 corresponding to FIG.

  As shown in FIG. 6, since the persons Q0 to Q1 and the vehicle P0 all reflect the laser, the detection wave output unit 13 scans the front of the vehicle P with the laser, and the persons Q0 to Q1 and the vehicle P0. To measure measurement point regions B0 to B2. Then, the size, position, and reflection intensity of the detected measurement point regions B0 to B2 are detected. For example, the z-direction distance of the measurement point regions B0 and B2 is z0, and the z-direction distance of the measurement point region B1 is z1. Then, measurement point area data relating to the detected size, position, and reflection intensity is generated and output to the storage memory 14. Here, the reflection intensity of the laser depends on the type of the detection object that reflects the laser. For example, the reflection intensity on the vehicle is higher than the reflection intensity on the person. This is because the reflector reflects the radar on the vehicle and the person's clothes on the person reflect the radar, and the reflector usually reflects the radar more efficiently than the person's clothes. Further, since the radar attenuates in the air, the reflection intensity of the measurement point area detected by the detection wave output unit 13 becomes smaller as the distance in the z direction from the vehicle P to the measurement point area becomes longer. As described above, the reflection intensity of the measurement point region B2 is larger than the reflection intensity of the measurement point region B1. Further, the reflection intensity of the measurement point region B1 is larger than the reflection intensity of the measurement point region B0.

  On the other hand, as shown in FIG. 7, the thermal image A includes a vehicle image a2 in addition to the person images a0 to a1 and the wall images a3 to a4. However, since the thermal data included in the vehicle image a2 and the wall images a3 to a4 are different from the thermal data included in the person images a0 to a1, the detection object candidate areas A0 to A1 include the vehicle image a2 and the wall images a3 to a4. Absent.

  FIG. 8 shows an example of a measurement point region acquired by the detection wave output unit 13 when the detection wave output unit 13 is a millimeter wave radar. Moreover, the thermal image A which the camera 10 acquires in the case shown in FIG. 8 is shown in FIG. The situation in front of the vehicle P shown in FIG. 8 is the same as the situation shown in FIG.

  As shown in FIG. 8, since the person Q0 to Q1, the vehicle P0, and the walls R0 to R1 (corresponding to the wall images a3 to a4 shown in FIG. 4) all reflect millimeter waves, the detection wave output unit 13 The front of the vehicle P is scanned with millimeter waves, and the measurement point regions B0 to B4 are detected from the persons Q0 to Q1, the vehicle P0, and the walls R0 to R1. Then, the size, position, and reflection intensity of the detected measurement point regions B0 to B4 are detected. For example, the z-direction distance of the measurement point regions B0 and B2 is z0, and the z-direction distance of the measurement point region B1 is z1. Then, measurement point area data relating to the detected size, position, and reflection intensity is generated and output to the storage memory 14. Here, although the wall only reflects the millimeter wave slightly, the millimeter wave radar can detect the measurement point region even from a detection object that slightly reflects the millimeter wave. Therefore, the detection wave output unit 13 can detect the measurement point regions B3 to B4 from the walls R0 to R1. Further, the reflection intensity does not depend on the type of the detected object but depends on the size of the measurement point area.

  On the other hand, as shown in FIG. 9, the thermal image A includes a vehicle image a2 in addition to person images a0 to a1 and wall images a3 to a4. However, since the thermal data included in the vehicle image a2 and the wall images a3 to a4 are different from the thermal data included in the person images a0 to a1, the detection object candidate areas A0 to A1 include the vehicle image a2 and the wall images a3 to a4. Absent.

  The storage memory 14 shown in FIG. 1 stores the measurement point area data given from the detection wave output unit 13.

  The distance measurement unit 15 acquires measurement point region data from the storage memory 14, and based on the measurement point region data and the detection target candidate region data given from the region extraction unit 12, out of the measurement point regions, A measurement point area corresponding to the detection object candidate area is extracted as the same candidate area.

  Specifically, first, based on the detection object candidate area data, the position of the detected object detected from the detection object candidate area (position in the reference coordinate system) is calculated. Here, a specific calculation method will be described with reference to FIGS. Here, FIG. 10 is a plan view showing the positional relationship between the person Q2 and the camera lens 100 and the imaging surface 101, and FIG. 12 is a side view showing the positional relationship between the person Q2 and the camera lens 100 and the imaging surface 101. It is. FIG. 14 is a side view showing the positional relationship between the person Q2, the camera 10, and the imaging surface 101, and FIGS. 11, 13, and 15 are explanatory diagrams showing an example of the thermal image A. FIG. Note that the camera lens 100 is included in the camera 10, and the imaging surface 101 is a surface on which the thermal image A is captured.

  As shown in FIG. 10, when the focal length is f (constant) and the x-coordinate of the person image in which the person Q2 is drawn in the thermal image A is xcd0, the orientation θ of the person Q2 with respect to the camera lens 100 is expressed by the following equation: It is represented by (1).

tan θ = xcd0 / f (1)
Therefore, when the camera 10 acquires the thermal image A shown in FIG. 11 (the same thermal image A as in FIG. 4) and the region extraction unit 12 extracts the detection target candidate region A0, the distance measurement unit 15 By substituting the x-coordinate xc0 of the position of the detection object candidate area A0 into xcd0 in Expression (1), the azimuth θ0 of the detected object (in this case, the person Q0) detected from the detection object candidate area A0 is calculated. To do. The z-direction distance zd1 from the person Q2 to the camera lens 100 is as follows. The height of the person Q2 is H, the height of the person image in which the person Q2 is drawn in the thermal image A is hc, and the focal length as shown in FIG. Is represented by the following formula (2).

zd1 = f * H / hc (2)
Therefore, when the camera 10 acquires the thermal image A shown in FIG. 13 (the same thermal image A as in FIG. 4) and the region extraction unit 12 extracts the detection target candidate regions A0 to A1, the distance measurement unit 15 Substitutes the heights hc0 to hc1 of the detection object candidate areas A0 to A1 into hc in the equation (2), respectively, and the normal height H1 of the person into H in the equation (2). Thus, the z-direction distances z0 ′ to z1 ′ of the detected object (in this case, a person) detected from the detection object candidate areas A0 to A1 are calculated.

  Note that the method of calculating the z-direction distance in this way is applied when there is almost no variation in the height of the person existing in front of the vehicle P (for example, when most of the persons existing around the vehicle P are adults). Is done. If the height varies, even if the normal height H1 is substituted for H in Equation (2), the detected object detected from the detection target candidate region does not necessarily have the normal height H1. This is because the accurate z-direction distance of the detected object may not be calculated.

  Therefore, when the height of the person varies, the distance measurement unit 15 calculates the z-direction distance by the following method. That is, as shown in FIG. 14, when the person Q2 is present in front of the vehicle P, the height from the road surface to the camera 10 is L (constant), the focal length is f, and the person Q2 is depicted in the thermal image A. When the y coordinate of the lower end of the image is ycd0 ′, the z-direction distance zd1 from the vehicle P to the person Q2 is expressed by the following equation (3).

zd1 = f * L / ycd0 ′ (3)
Therefore, when the camera 10 acquires the thermal image A shown in FIG. 14 (the same thermal image A as in FIG. 4) and the region extraction unit 12 extracts the detection target object regions A0 to A1, the distance measurement unit 15 Calculates the y-coordinates yc0 ′ to yc1 ′ of the lower end portions of the detection object candidate areas A0 to A1 by the following equations (4) to (5).

yc0 ′ = yc0 + hc0 (4)
yc1 ′ = yc1 + hc1 (5)
Then, by substituting the calculated yc0 ′ to yc1 ′ into ycd0 ′ in Expression (3), respectively, the z direction of the detected object (in this case, a person) detected from the detection target candidate areas A0 to A1 The distances z0 ′ to z1 ′ are calculated. Here, since the calculated azimuth and z-direction distance are based on the camera lens 100, the distance measuring unit 15 uses the calculated azimuth and z-direction distance as a reference, as shown in FIG. Convert to azimuth and z-direction distance with reference to coordinate system. Thereby, the position of the detected object detected from the detection target candidate region is calculated. Then, position data regarding the calculated position is acquired.

  Then, the distance measuring unit 15 adds the calculated position data to the detection target candidate area data. Further, based on the measurement point area data acquired from the storage memory 14, a measurement point area corresponding to the detection target object candidate area, that is, a measurement point area having position data matching the position data of the detection target object candidate area is obtained. Extract as the same candidate area. Then, the measurement point region data relating to the extracted measurement point region is output to the same object determination unit 16 as the same candidate region data. Further, the detection target candidate area data is output to the same object determination unit 16.

  The same object determination unit 16 determines again that the detection object candidate region and the same candidate region correspond to each other. Specifically, the movement of the position and the same candidate area calculated by the distance measuring unit 15 based on the plurality of detection object candidate area data and the same candidate area data given from the distance measuring unit 15 as time passes. The movement of the position of is calculated. Then, it is determined whether or not these motions match each other. If they match, it is determined that the detection target candidate region and the same candidate region correspond to each other. Then, it is determined that the movement of the position calculated by the distance measuring unit 15 and the movement of the position of the same candidate area coincide with the movement of the detected object detected from the detection target object candidate area. This determination can be made for the following reason. That is, the detection wave output unit 13 can accurately detect the position of the measurement point area, that is, the same candidate area, and therefore the movement of the position of the same candidate area is the movement of the detected object detected from the same candidate area. Match. And when a detection target object candidate area | region respond | corresponds to the same candidate area | region, the detection object detected from a detection target object candidate area | region corresponds with the detection object detected from the same candidate area | region. Therefore, when the detection target object candidate areas correspond to the same candidate area, the movement of the position calculated by the distance measuring unit 15 and the movement of the position of the same candidate area are detected object movements detected from the detection target object candidate area. Can be determined to match.

  Then, it is determined whether the movement of the position calculated by the distance measuring unit 15, that is, the movement of the detected object is included in the predetermined range. As a result, when it is included in the predetermined range, motion data relating to the motion of the detected object, detection target candidate region data, and measurement point region data are output to the target detection unit 17.

  The determination can also be performed by the following method. That is, the same object determination unit 16 calculates the width wd of the detected object detected from the detection target candidate area based on the detection target candidate area data. Here, the width wd of the detected object is calculated by the following equation (6), where wc is the width of the detection target object candidate region, f is the focal length, and zd2 is the distance from the vehicle P to the detected object. .

wd = zd2 * wc / f (6)
Then, it is determined whether or not the calculated width wd of the detected object matches the width of the same candidate region. If they match, it is determined that the detection target candidate region and the same candidate region correspond to each other.

  The object detection unit 17 detects the feature amount of the detection target candidate region and the same candidate region based on the detection target candidate region data and the same candidate region data given from the same object determination unit 16 over time. The average value is calculated for the feature amount and the motion of the detected object. Here, examples of the feature amount include thermal data of the detection target object candidate area, an aspect ratio of the detection target object candidate area, and a position of the same candidate area. The reason for calculating the average value is to suppress variations in the calculation results. Then, the average value is set as the feature amount of the detection target candidate region, the feature amount of the same candidate region, and the motion of the detected object.

  Then, a person is detected based on the calculated feature amount and motion. Specifically, for example, it is determined whether or not the thermal data of the detection target object candidate region matches the thermal data of the person. As a result, when they match, the calculated movement is compared with the walking speed of the person. As a result, if they match, a person is detected from the detection object candidate area or the same candidate area.

  Moreover, the target object detection part 17 can also detect a person with the following method. That is, the object detection unit 17 compares the average aspect ratio of the detection object candidate area with the aspect ratio of the person. As a result, when they match, the calculated movement is compared with the walking speed of the person. As a result, if they match, a person is detected from the detection object candidate area or the same candidate area.

  Then, when a person is detected, the object detection unit 17 calculates the position of the person based on the detection object candidate area data or the same candidate area data, and the person position relating to the calculated person position Data is generated and output to the detection result output unit 18.

  The detection result output unit 18 determines the reliability of the person position data given from the object detection unit 17. Specifically, when the person position data having the same content is given only once, it is determined that the reliability of the person position data is low. This is because the person position data may be noise. Conversely, when a plurality of pieces of person position data having the same contents are given, it is determined that the person position data is highly reliable, and the person position data is converted into an apparatus that requires the person position data (for example, To car navigation).

  Next, a processing procedure by the object detection apparatus 1 will be described with reference to a flowchart shown in FIG.

  In step S <b> 1 shown in FIG. 16, the camera 10 observes the front of the vehicle P, acquires a thermal image A having thermal data ahead of the vehicle P in the form of thermal image data, and stores the thermal image data in the image memory 11. Output to. The image memory 11 stores thermal image data given from the camera 10.

  Next, in step S2, the region extraction unit 12 acquires thermal image data from the image memory 11, and extracts a detection target candidate region having human thermal data from the thermal image A based on the thermal image data. To do.

  Next, the region extraction unit 12 calculates the size (specifically, the width and height) of the detection target candidate region and the position on the thermal image A based on the pixel data included in the detection target candidate region. To do. Then, detection target candidate area data relating to the calculation result and pixel data included in the detection target candidate area is generated and output to the distance measuring unit 15.

  The object detection device 1 performs the processes in steps S1 to S2 while performing the processes in steps S3 to S4. That is, in step S3 to step S4, the detection wave output unit 13 scans the front of the vehicle P with a predetermined detection wave, and extracts a measurement point region that reflects the detection wave.

  Next, the size, position, and reflection intensity of the detected measurement point area are detected, and measurement point area data relating to the detected size, position, and reflection intensity is generated. Next, the generated measurement point area data is output to the storage memory 14. The storage memory 14 stores the measurement point area data.

  Next, in step S <b> 5, the distance measurement unit 15 acquires measurement point region data from the storage memory 14, and based on the measurement point region data and the detection target candidate region data given from the region extraction unit 12. Through the above-described processing, the measurement point area corresponding to the detection target candidate area is extracted as the same candidate area from the measurement point area.

  Next, the measurement point region data extracted as the same candidate region is output to the same object determination unit 16 as the same candidate region data. Further, the detection target candidate area data is output to the same object determination unit 16.

  Next, in step S6, the same object determination unit 16 determines again whether or not the detection target object candidate region and the same candidate region correspond to each other by the above-described processing. Next, the detection object candidate area data and the same candidate area data determined to correspond to each other are output to the object detection unit 17.

  Next, in step S <b> 7, the target detection unit 17 detects a person by the above-described processing based on the detection target candidate region data and the same candidate region data given from the same object determination unit 16.

  Next, when a person is detected, the position of the person is calculated based on the detection object candidate area data or the same candidate area data, and the person position data relating to the calculated person position is generated and the detection result Output to the output unit 18.

  Next, in step S8, the detection result output unit 18 determines the reliability of the person position data given from the object detection unit 17 by the above-described processing. As a result, the person position data determined to have high reliability is output to an apparatus that requires the person position data.

  As described above, in the first embodiment, the object detection device 1 moves the position calculated by the distance measurement unit 15, that is, the detected object calculated based on the feature amount of the detection target candidate region. Are matched with the movements of the same candidate area and these movements are within a predetermined range, and then a person is detected based on the feature amount of the detection target candidate area and the movement.

  Therefore, a detection object that is different from a person and that moves in the same manner as the person is not erroneously detected as a person. In addition, a detection object having the same size as a person and moving differently from the person is not erroneously detected as a person. Therefore, it is possible to reliably detect a person.

  In addition, when the motion of the detected object calculated based on the feature amount of the detection target candidate region matches the motion of the same candidate region, the motion matches the motion of the detected object detected from the detection target candidate region. . Therefore, the object detection apparatus 1 can detect a person based on the actual movement of the detection object, and thus can accurately detect the person.

(Second Embodiment)
Hereinafter, a second embodiment of the present invention will be described with reference to the drawings. First, based on FIG. 1, FIG. 6, FIG. 8, FIG. 17, and FIG. 18, components of the object detection device 2 according to an embodiment of the present invention and main functions of each component will be described.

  FIG. 17 is a side view showing the installation positions of the camera 10, the laser radar 230, and the millimeter wave radar 231, and FIG. 18 is a plan view showing the installation positions of the camera 10, the laser radar 230, and the millimeter wave radar 231. . 17 to 18, the point at the tip of the laser radar 230 is the origin, the z-axis is the direction parallel to the central axis of the laser radar 230, and x is the direction perpendicular to the z-axis and parallel to the road surface. A coordinate system having the y-axis in the direction perpendicular to the axis, the z-axis, and the road surface is the reference coordinate system. The center axis of the laser radar 230 is an axis that passes through the center of the tip surface of the laser radar 230 and is perpendicular to the tip surface.

  As shown in FIG. 1, the object detection device 2 is obtained by replacing the detection wave output unit 13 and the distance measurement unit 15 of the object detection device 1 with a detection wave output unit 23 and a distance measurement unit 25.

  The detection wave output unit 23 shown in FIG. 1 includes a laser radar 230 and a millimeter wave radar 231 as shown in FIGS. Here, the laser radar 230 is installed at the front of the vehicle P so that its optical axis is parallel to the road surface and the optical axis of the camera 10. Similarly, the millimeter wave radar 231 is installed at the front portion of the vehicle P so that its optical axis is parallel to the road surface and the optical axis of the camera 10. Further, the laser radar 230 and the millimeter wave radar 231 are installed at different positions in the vertical direction so that these optical axes are all on the xy plane.

  The detection wave output unit 23 scans the front of the vehicle P with a laser and a millimeter wave, and detects a measurement point region that reflects the laser or the millimeter wave.

  Then, the size (specifically, the width of the measurement point area, etc.) and the position (specifically, the z direction distance and direction from the vehicle P to the measurement point area) of the detected measurement point area are detected. To do. Further, the reflection intensity of the detection wave in the measurement point region is detected. Then, measurement point area data relating to the detected size, position, and reflection intensity is generated and output to the storage memory 14.

  The distance measurement unit 25 acquires measurement point region data from the storage memory 14, and based on the measurement point region data and the detection target candidate region data given from the region extraction unit 12, among the measurement point regions, A measurement point area that is suitable for calculating the position of the detection object candidate area and that corresponds to the detection object candidate area is extracted as the same candidate area.

  Specifically, first, similarly to the distance measurement unit 15, the z-direction distance and direction, that is, the position of the detected object detected from the detection target candidate area are calculated based on the detection target candidate area data. Then, position data regarding the calculated position is added to the detection target candidate area data.

  Further, the calculated z-direction distance is compared with a predetermined reference distance. As a result, when the calculated z-direction distance is longer than the reference distance, it is determined that the measurement point region detected by the millimeter wave radar 231 is suitable for calculating the position of the detection target candidate region. Then, from the measurement point area detected by the millimeter wave radar 231, a measurement point area corresponding to the detection target object candidate area, that is, a measurement point area having position data matching the position data of the detection target object candidate area is obtained. Extract as the same candidate area.

  On the other hand, when the z-direction distance is equal to or smaller than the reference distance, or when the z-direction distance is larger than the reference distance but the detection target candidate areas are dense, the measurement detected by the laser radar 230 is performed. It is determined that the point area data is suitable for calculating the position of the detection target object candidate area. Then, the measurement point area corresponding to the detection object candidate area is extracted as the same candidate area from the measurement point area detected by the laser radar 230.

  Then, the measurement point region data relating to the extracted measurement point region is output to the same object determination unit 16 as the same candidate region data. Further, the detection target candidate area data is output to the same object determination unit 16.

  As described above, the reason why the type of the measurement point area to be extracted is changed according to the size of the distance in the z direction or the like is as follows. That is, the laser is attenuated by the influence of air, while the millimeter wave is not affected. Therefore, the detectable range of the laser radar 230 is narrower than the detectable range of the millimeter wave radar 231. Further, when the measurement point area is detected by the laser radar 230, when the distance in the z direction of the measurement point area is increased, the reflection intensity of the measurement point area is reduced, and thus the reliability of the measurement point area is lowered.

  On the other hand, when the measurement point area is detected by the millimeter wave radar 231, the reflection intensity becomes a certain level or more regardless of the distance in the z direction of the measurement point area. For this reason, even when the distance in the z direction from the vehicle P to the detection object ahead is long and the laser radar 230 cannot detect the measurement point area from the detection object or the reliability of the measurement point area is low, A highly reliable measurement point region can be detected. Therefore, when the z-direction distance is long, the reliability of the measurement point region detected by the millimeter wave is larger than the reliability of the measurement point region detected by the laser.

  On the other hand, the millimeter wave radar 231 detects the measurement point region from a detection object that slightly reflects the millimeter wave, such as a wall. Further, the reflection intensity in the measurement point region does not depend on the type of the detection object that reflects the millimeter wave. Therefore, for example, when a person is approaching a wall, it is difficult to accurately detect the person from the measurement point area. On the other hand, since the laser radar 230 does not detect the measurement point area data from the detection object that slightly reflects the laser, for example, even when the person is approaching the wall, the measurement point area is determined only from the person. Can be detected. Therefore, when the detection object candidate areas are dense, the reliability of the measurement point area detected by the laser is larger than the reliability of the measurement point area detected by the millimeter wave.

  Therefore, the distance measurement unit 25, as described above, sets the measurement points to be extracted according to the calculated z-direction distance and the like so that the more reliable measurement point area data is the same candidate area data. It was decided to change the area data.

  Since the procedure of the process by the object detection device 2 is the same as the procedure of the process by the object detection device 1, the description thereof is omitted.

  As described above, in the second embodiment, the distance measuring unit 25 corresponds to the detection target object candidate area in the measurement point area and is suitable for calculating the position of the detection target object candidate area (in other words, The measurement point area (which has higher reliability) is extracted as the same candidate area. Accordingly, the extracted motion of the same candidate region is closer to the actual motion of the detected object than in the first embodiment, and therefore the motion of the same candidate region is the same as the motion of the position calculated by the distance measuring unit 25. When they match, the movement of the calculated position becomes closer to the actual movement of the detected object. And since the object detection apparatus 2 detects a person based on the said calculated motion, it can detect a person more correctly.

(Third embodiment)
Hereinafter, a third embodiment of the present invention will be described with reference to the drawings. First, based on FIG. 1, FIG. 19, and FIG. 20, components of the object detection device 3 according to an embodiment of the present invention and main functions of each component will be described.

  FIG. 1 is a block diagram showing the configuration of the object detection device 3, and FIGS. 19 to 20 are graphs showing the relationship between the z-direction distance and various parameters.

  As shown in FIG. 1, the object detection device 3 is obtained by replacing the object detection unit 17 of the object detection device 1 with an object detection unit (detection unit, storage unit) 37. In the third embodiment, the detection wave output unit 13 is a laser radar.

  The object detection unit 37 includes first person correspondence data indicating the relationship between the size of the detection object area in which a person is detected in the detection object candidate areas and the distance in the z direction from the person to the vehicle P. Remember. Moreover, the 2nd person corresponding | compatible data which show the relationship between the thermal data value which the said detection object area | region has and the z direction distance from the said person to the vehicle P are memorize | stored. Further, in the same candidate area, the third is a relationship between the reflection intensity data value (laser reflection intensity data value) of the detection target area where the person is detected and the z-direction distance from the person to the vehicle P. The person correspondence data is stored.

  FIG. 19A shows a graph 360 showing the contents of the first person correspondence data, FIG. 19B shows a graph 362 showing the contents of the second person correspondence data, and FIG. 3 shows a graph 364 showing the contents of the three person correspondence data.

  As shown in the graph 360, as the z-direction distance increases, the detection target area decreases. Also, as shown in graph 362, the thermal data value decreases as the z-direction distance increases. This is because the heat released from the object is attenuated in the air. Further, as shown in the graph 364, the reflection intensity data value decreases as the z-direction distance increases. This is because the laser attenuates in the air.

  Note that other person correspondence data may be stored in the object detection unit 37. Other person-corresponding data includes, for example, data indicating the relationship between the width of the detection target region and the distance in the z direction from the person to the vehicle P, or the reflection intensity data value (millimeter wave of the detection target region). And the like indicating the relationship between the reflection intensity data value) and the distance in the z direction from the person to the vehicle P. When the object detection unit 37 stores a graph relating to millimeter wave radar, the detection wave output unit 13 needs to be a millimeter wave radar.

  Moreover, the target object detection part 37 performs the following processes. That is, similar to the object detection unit 17, based on the detection object candidate area data and the same candidate area data given from the same object determination unit 16 over time, the feature amount of the detection object candidate area In addition, an average value is calculated for the feature amount of the same candidate region and the motion of the detected object detected from the detection target candidate region. Here, the feature amount includes the position of the detection object detected from the detection object candidate area (such as the distance in the z direction from the vehicle P to the detection object), the size of the detection object candidate area, and the detection object candidate area. The thermal data included in the data and the reflection intensity included in the same candidate area data are included. Then, the calculated average value is used as the feature amount of the detection target object candidate region, the feature amount of the same candidate region, and the motion of the detected object.

  Then, a person is detected based on the feature amount of the detection target object candidate region, the feature amount of the same candidate region, the movement of the detected object detected from the detection target candidate region, and the person correspondence data described above.

  Specifically, it is determined whether or not the coordinate point having the ordinate as the size of the detection object candidate area and the abscissa as the z-direction distance of the detected object detected from the detection object candidate area is on the graph 360. . For example, when the z-direction distance is included in the range from the distance z0 to the distance z1, it is determined whether or not the coordinate point is on the graph 361 corresponding to the range from the distance z0 to the distance z1 in the graph 360.

  As a result, when the coordinate point is on the graph 360, it is determined whether or not the coordinate point having the thermal data value of the detection target candidate region as the ordinate and the abscissa as the z-direction distance of the detection object is on the graph 362. . For example, when the z-direction distance is included in the range from the distance z0 to the distance z1, whether or not the coordinate point is on the graph 363 corresponding to the range from the distance z0 to the distance z1 in the graph 362 is determined.

  As a result, when the coordinate point is on the graph 362, it is determined whether or not a coordinate point having the reflection intensity data value of the same candidate region as the ordinate and the abscissa as the z-direction distance of the detected object is on the graph 364. For example, when the z-direction distance is included in the range from the distance z0 to the distance z1, it is determined whether or not the graph 365 corresponds to the graph 365 corresponding to the range from the distance z0 to the distance z1. As a result, when the coordinate point is on the graph 364, the movement calculated by the above-described process is compared with the walking speed of the person. As a result, if they match, a person is detected from the detection object candidate area or the same candidate area.

  Then, when a person is detected, the object detection unit 37 calculates the position of the person based on the detection object candidate area data or the same candidate area data, and the person position related to the calculated person position Data is generated and output to the detection result output unit 18.

  The processing procedure by the object detection device 3 is the same as the procedure by the object detection device 1, and thus the description thereof is omitted.

  As described above, in the third embodiment, the object detection device 3 detects the parameter value (thermal data value or size) indicating the feature amount of the detection target candidate region from the ordinate and the detection target candidate region. It is determined whether or not the coordinate points having the abscissa as the z-direction distance of the detected object are on the graphs 360 and 362. When the coordinate points are on the graphs 360 and 362, coordinate points having a parameter value (reflection intensity data value) indicating the feature amount of the same candidate area as the ordinate and a z-direction distance of the same candidate area as the abscissa are obtained. Whether to ride on the graph 364 is determined. As a result, when the coordinate point is on the graph 364, the person is detected based on the motion detected from the detection target candidate region.

  Here, the graphs 360, 362, and 364 show the relationship between the detection target region in which a person is detected and the z-direction distance. Therefore, the object detection apparatus 3 performs such a determination to ensure that the detection target object area and the same candidate area do not have the feature amount of the detection target area, that is, the person is not detected. It is possible to exclude a person from the remaining detection object candidate area and the same candidate area. Therefore, it is possible to reliably detect a person.

  Since the z-direction distance is immediately calculated from the size of the detection target area (see Expression (2)), the z-direction distance in the person correspondence data may be used as the detection target area size.

  In addition, the relationship between the size of the detection target area, the thermal data of the detection target area, and the reflection intensity data value of the detection target area and the distance in the z direction is determined by the person's clothes, age, height, It varies somewhat depending on the environment (weather). For this reason, even if the above-described coordinate points do not completely occupy the graphs 360, 362, and 364, there is a possibility that a person is detected from the detection object candidate region corresponding to the coordinate points.

  Therefore, the object detection unit 37 may be configured as follows. That is, as shown in FIG. 20, the object detection unit 37 stores peripheral area correspondence data indicating the peripheral areas 360a to 365a of the graphs 360, 362, and 364 in addition to the above-described person correspondence data. And the target object detection part 37 performs the following processes. That is, it is determined whether or not a coordinate point having the size of the detection target object region as the ordinate and the abscissa as the z-direction distance of the detected object detected from the detection target candidate region is on the peripheral region 360a. For example, when the z-direction distance is included in the range from the distance z0 to the distance z1, it is determined whether or not the coordinate point is on the peripheral area 361a corresponding to the range from the distance z0 to the distance z1 in the peripheral area 360a.

  As a result, when the coordinate point is on the peripheral region 360a, whether the coordinate point having the thermal data value of the detection target candidate region data as the ordinate and the abscissa as the z-direction distance of the detection object is on the peripheral region 362a. Determine. For example, when the z-direction distance is included in the range from the distance z0 to the distance z1, it is determined whether or not the coordinate point is on the peripheral area 363a corresponding to the range from the distance z0 to the distance z1 in the peripheral area 362a.

  As a result, when the coordinate point rides on the peripheral region 362a, whether or not the coordinate point having the reflection intensity data value of the same candidate region data as the ordinate and the abscissa as the z-direction distance of the detected object is on the peripheral region 364a. judge. For example, when the z-direction distance is included in the range from the distance z0 to the distance z1, it is determined whether the peripheral area 365a corresponding to the range from the distance z0 to the distance z1 is included in the peripheral area 364a. As a result, when the coordinate point is on the peripheral region 364a, the movement calculated by the above-described process is compared with the walking speed of the person. As a result, if they match, a person is detected from the detection object candidate area or the same candidate area.

  Thereby, although the coordinate points do not completely ride on the graphs 360, 362, and 364, it is possible to prevent the detection target candidate area from which the person is detected from being excluded from the person detection process.

  In addition, the distance measurement unit 15 may perform the same process as the process performed by the object detection unit 37. In this case, the distance measuring unit 15 generates the same candidate area data as the detection target candidate area data by the same process as described above. Thereafter, by performing the same process as the process performed by the target object detection unit 37, only the same candidate area data as the detection target object candidate area data that can detect a person is the same object determination unit. 16 is output. As a result, data to be processed by the same object determination unit 16, the object detection unit 37, and the detection result output unit 18 is reduced, so that the processing by the object detection device 3 is speeded up.

(Fourth embodiment)
Hereinafter, a third embodiment of the present invention will be described with reference to the drawings. First, based on FIG. 1 and FIG. 21, the components of the object detection device 4 according to an embodiment of the present invention and the main functions of each component will be described.

  FIG. 1 is a block diagram showing the configuration of the object detection device 4, and FIG. 21 is a graph showing the relationship between the z-direction distance and various parameters.

  As shown in FIG. 1, the object detection device 4 is obtained by replacing the object detection unit 17 of the object detection device 1 with an object detection unit (detection unit, storage unit) 47. In the fourth embodiment, the detection wave output unit 13 is a laser radar.

  Similar to the object detection unit 37, the object detection unit 47 stores second to third person correspondence data.

  Here, FIGS. 21A to 21B show graphs 460 indicating the contents of the second person correspondence data, and FIGS. 21C to 21D show graphs showing the contents of the third person correspondence data. 463 is shown.

  Note that other person correspondence data may be stored in the object detection unit 47. As other person-corresponding data, for example, in addition to the first person-corresponding data described above, the relationship between the width of the detection target area where a person is detected and the distance in the z direction from the person to the vehicle P is shown. In addition, there is a relationship between the reflection intensity data value (millimeter wave reflection intensity data value) of the detection target area and the z-direction distance from the person to the vehicle P. When the object detection unit 47 stores a graph related to millimeter wave radar, the detection wave output unit 13 needs to be a millimeter wave radar.

  The object detection unit 47 performs the following processing. That is, similar to the object detection unit 17, the average value is calculated for the feature amount of the detection target candidate region, the feature amount of the same candidate region, and the motion of the detected object detected from the detection target candidate region. Here, the feature amount includes the position of the detection object detected from the detection object candidate area (such as the distance in the z direction from the vehicle P to the detection object), the size of the detection object candidate area, and the detection object candidate area. And the reflection intensity of the same candidate region are included. Then, the calculated average value is used as the feature amount of the detection target object candidate region, the feature amount of the same candidate region, and the motion of the detected object.

  Then, the processing is performed a plurality of times, the first object correspondence data indicating the movement of the thermal data value associated with the movement of the detected object in the z direction distance, and the reflection intensity data value associated with the movement of the detected object in the z direction distance. Second object correspondence data indicating movement is generated.

  Then, a person is detected based on the generated first to second object correspondence data and second to third person correspondence data.

  Specifically, the object detection unit 47 determines whether or not the relationship indicated by the first object correspondence data matches the relationship indicated by the second person correspondence data. FIG. 21A shows an example in the case of coincidence, and FIG. 21B shows an example in the case of no coincidence. A graph 460 shows the content of the second person correspondence data, and graphs 461 to 462 show the content of the first object correspondence data.

  As a result, if they match, it is determined whether or not the relationship indicated by the second object correspondence data matches the relationship indicated by the third person correspondence data. FIG. 21C shows an example in the case of coincidence, and FIG. 21D shows an example in the case of no coincidence. The graph 463 shows the content of the third person correspondence data, and the graphs 464 to 465 show the content of the second object correspondence data.

  As a result, if they match, the movement calculated by the above-described process is compared with the walking speed of the person. As a result, if they match, a person is detected from the detection object candidate area or the same candidate area.

  Then, when a person is detected, the object detection unit 47 calculates the position of the person based on the detection object candidate area data or the same candidate area data, and the person position related to the calculated person position Data is generated and output to the detection result output unit 18.

  The processing procedure by the object detection device 4 is the same as the procedure by the object detection device 1, and thus the description thereof is omitted.

  As described above, in the fourth embodiment, the object detection device 4 detects the movement of the thermal data value accompanying the movement of the detected object in the z-direction distance (that is, the relationship indicated by the first object correspondence data) and the detection. Whether the movement of the reflection intensity data value accompanying the movement of the distance in the z direction of the object (that is, the relationship indicated by the second object correspondence data) matches the relationship indicated by the second to third person correspondence data described above. Determine if. If they match, a person is detected from the detection target object candidate area or the same candidate area based on the feature quantity of the motion of the detected object. Therefore, it can be determined more reliably than the third embodiment whether or not the detection target object region and the same candidate region have the same feature amount as the detection target region. In other words, the detection target candidate area and the same candidate area in which no person is detected can be more reliably excluded than in the third embodiment, and then a person can be detected from the remaining detection target candidate areas and the same candidate area. it can.

  In addition, since the object correspondence data is generated by calculating a plurality of feature amounts and the like of the detection target candidate region, the object detection device 4 is indicated by the object correspondence data even if the feature amount or the like includes noise. It is possible to accurately determine whether the relationship matches the relationship indicated by the person correspondence data.

(Fifth embodiment)
Hereinafter, a fifth embodiment of the present invention will be described with reference to the drawings. First, based on FIG. 1 and FIG. 22, components of the object detection device 5 according to one embodiment of the present invention and main functions of each component will be described.

  FIG. 1 is a block diagram showing the configuration of the object detection device 5, and FIG. 22 is a graph showing the relationship between the z-direction distance and various parameters.

  As shown in FIG. 1, the object detection device 5 replaces the detection wave output unit 13 and the object detection unit 17 of the object detection device 1 with a detection wave output unit 23 and an object detection unit (detection unit, storage unit) 57. It is a thing.

  In addition to the first to third person correspondence data described above, the object detection unit 57 includes a reflection intensity data value (millimeter wave reflection intensity data value) included in a detection object area in which a person is detected, and the person. 4th person corresponding | compatible data which show the relationship with the z direction distance to the vehicle P are memorize | stored.

  FIG. 22A shows a graph 560 showing the contents of the fourth person correspondence data, and FIG. 22B shows a graph 560 showing the contents of the third person correspondence data. The graph 560 indicates that the detectable distance of the millimeter wave radar is from the distance zMRd to the distance zMRu, and the detectable distance of the laser radar is from the distance zLRd to the distance zLRu. As described in the second embodiment, the distance zMRu is longer than the distance zLRu.

  In addition to the first to fourth person correspondence data, the object detection unit 57 also stores peripheral region correspondence data indicating the above-described peripheral regions (for example, the peripheral regions 560a to 561a shown in FIG. 22).

  It should be noted that other person correspondence data may be stored in the object detection unit 57. Other person-corresponding data includes, for example, a relationship between the width of a detection target area where a person is detected and the z-direction distance from the person to the vehicle P.

  The reliability of the first to fourth person correspondence data and the peripheral region correspondence data varies depending on the position of the detected object detected from the detection target candidate region.

  For example, when the z-direction distance of the detection object is included in the detectable distance of the laser radar, the reliability of the third person correspondence data is greater than the reliability of the fourth person correspondence data. As described in the second embodiment, the reflection intensity data value of the laser differs depending on the type of the object. In this case, the reflection intensity data value indicated by the third person correspondence data is the reflection intensity data of the person. This is because the values match the values but differ from the reflection intensity data values of other objects. On the other hand, when the z-direction distance of the detected object is not included in the detectable distance of the laser radar, the reliability of the third person correspondence data is smaller than the reliability of the fourth person correspondence data. This is because the third person correspondence data does not have a corresponding reflection intensity data value.

  Further, when the detected object is present at the end of the field angle of the millimeter wave radar or the laser radar, the reliability of the third to fourth person correspondence data is the reliability of the first to second person correspondence data. Smaller than. In this case, since the reflection intensity data value of the same candidate area is not accurately detected, even if the determination process described later is performed using the third to fourth person correspondence data, accurate determination cannot be performed. It is.

  Moreover, the target object detection part 57 performs the following processes. That is, similar to the object detection unit 17, based on the detection object candidate area data and the same candidate area data given from the same object determination unit 16 over time, the feature amount of the detection object candidate area In addition, an average value is calculated for the feature amount of the same candidate region and the motion of the detected object detected from the detection target candidate region. Here, the feature amount includes the position of the detected object (such as the distance in the z direction from the vehicle P to the detected object), the size of the detection target object candidate area, the thermal data of the detection target object candidate area data, and the same candidate The reflection intensity of the area data is included. Then, the calculated average value is used as the feature amount of the detection target object candidate region, the feature amount of the same candidate region, and the motion of the detected object.

  Then, correspondence data (corresponding data with high reliability) corresponding to the calculated position of the detected object is acquired, and the correspondence data, the feature amount of the detection target candidate region, the feature amount of the same candidate region, and the detection Based on the movement of the object, a determination process similar to that of the object detection unit 37 is performed to detect a person.

  As described above, in the fifth embodiment, the object detection device 5 acquires the correspondence data corresponding to the position of the detected object, and performs the same determination process as that of the object detection unit 37 based on the correspondence data. Therefore, it can be determined more reliably than the third embodiment whether or not the detection target object region and the same candidate region have the same feature amount as the detection target region. In other words, the detection target candidate area and the same candidate area in which no person is detected can be more reliably excluded than in the third embodiment, and then a person can be detected from the remaining detection target candidate areas and the same candidate area. it can.

  Needless to say, the techniques described in the first to fifth embodiments can be combined with each other.

  In the first to fifth embodiments, a person is detected as a detection target. However, other detection objects such as road structures may be detected.

It is a block diagram which shows the structure of the object detection apparatus which concerns on one embodiment of this invention. It is the side view which showed the installation position of a camera and a detection wave output part. It is the top view which showed the installation position of a camera and a detection wave output part. It is explanatory drawing which shows an example of the thermal image acquired with a camera. It is explanatory drawing which shows an example of the thermal image acquired with a camera. It is a top view which shows an example of the measurement point area | region detected by a laser radar. It is explanatory drawing which shows an example of the thermal image acquired with a camera. It is a top view which shows an example of the measurement point area | region detected by a millimeter wave radar. It is explanatory drawing which shows an example of the thermal image acquired with a camera. It is a top view which shows the positional relationship of a camera lens and a person. It is explanatory drawing which shows an example of the thermal image acquired with a camera. It is a side view which shows the positional relationship of a camera lens and a person. It is explanatory drawing which shows an example of the thermal image acquired with a camera. It is a side view which shows the positional relationship of a camera and a person. It is explanatory drawing which shows an example of the thermal image acquired with a camera. It is a flowchart which shows the procedure of the process by an object detection apparatus. It is the side view which showed the installation position of a camera and a detection wave output part. It is the top view which showed the installation position of a camera and a detection wave output part. (A) It is a graph which shows the relationship between the magnitude | size of a detection target object area | region, and z direction distance. (B) It is a graph which shows the relationship between a thermal data value and z direction distance. (C) It is a graph which shows the relationship between a reflection intensity data value and z direction distance. (A) It is a graph which shows the relationship between the magnitude | size of a detection target object area | region, and z direction distance. (B) It is a graph which shows the relationship between a thermal data value and z direction distance. (C) It is a graph which shows the relationship between a reflection intensity data value and z direction distance. (A) It is a graph which shows the relationship between a thermal data value and z direction distance. (B) It is a graph which shows the relationship between a thermal data value and z direction distance. (C) It is a graph which shows the relationship between a reflection intensity data value and z direction distance. (D) It is a graph which shows the relationship between a reflection intensity data value and z direction distance. (A) It is a graph which shows the relationship between the reflection intensity data value by a millimeter wave, and z direction distance. (B) It is a graph which shows the relationship between the reflection intensity data value by a laser, and z direction distance.

Explanation of symbols

1-5 ... Object detection device 10 ... Camera (image acquisition means)
DESCRIPTION OF SYMBOLS 11 ... Image memory 12 ... Area extraction part (area extraction means)
13, 23 ... Detection wave output unit (detection wave output means)
14 ... Memory for storage 15, 25 ... Distance measuring unit (position calculating means, position extracting means)
16: Same object determination unit 17, 37, 47 ... Object detection unit (detection unit, storage unit)
18 ... Detection result output section

Claims (6)

In an object detection device mounted on a vehicle,
Image acquisition means for observing the vehicle periphery and acquiring an image having thermal data around the vehicle;
A region extraction means for extracting a detection object candidate region having thermal data of a person from the image;
Position calculating means for calculating the position of the detected object detected from the detection target candidate region based on at least one of the position and size of the detection target candidate region;
A detection wave output means for scanning the periphery of the vehicle with a predetermined detection wave, detecting a measurement point region that reflects the detection wave, and detecting a position of the detected measurement point region;
Position extraction means for extracting, as the same candidate area, a measurement point area existing at the same position as the position of the detected object detected from the detection target object area, among the measurement point areas;
The width of the detected object detected from the detection target object candidate region matches the width of the same candidate region, the feature amount of the detection target candidate region matches the feature amount of a person, and the position calculation When at least one of the position movement calculated by the means or the position movement of the same candidate area matches the walking speed of the person, a person is detected from the detection target candidate area or the same candidate area. Detection means;
Of the first correspondence data indicating the relationship between the position of the person and the size of the detection target area in which the person is detected among the detection target candidate areas, the position of the person, and the detection target candidate area The second correspondence data indicating the relationship with the thermal data value of the detection target area where the person is detected, the position of the person, and the reflection intensity of the detection target area where the person is detected among the same candidate areas Storage means for storing third corresponding data indicating a relationship with the data value,
The feature amount is at least one of thermal data and an aspect ratio,
When the width of the detected object detected from the detection target candidate area matches the width of the same candidate area, the detection means determines the size of the detection target candidate area and the detection target candidate area. A determination is made as to whether or not the relationship between the position of the detected object detected from the data matches the first corresponding data, and if it matches, the thermal data value of the detection object candidate region and the detection A determination is made as to whether or not the relationship between the position of the detected object detected from the object candidate area matches the second corresponding data, and if it matches, the reflection intensity data value of the same candidate area Then, a determination is made as to whether or not the relationship with the position of the detected object detected from the detection object candidate region matches the third corresponding data, and if it matches, it is calculated by the position calculation means. Movement of the position Or of the movement of the position of the same candidate region, it is determined whether at least one of which coincides with the walking speed of a person, if they match, detects the person from the detection target object candidate region or the same candidate region An object detection apparatus characterized by that. In an object detection device mounted on a vehicle,
Image acquisition means for observing the vehicle periphery and acquiring an image having thermal data around the vehicle;
A region extraction means for extracting a detection object candidate region having thermal data of a person from the image;
Position calculating means for calculating the position of the detected object detected from the detection target candidate region based on at least one of the position and size of the detection target candidate region;
A detection wave output means for scanning the periphery of the vehicle with a predetermined detection wave, detecting a measurement point region that reflects the detection wave, and detecting a position of the detected measurement point region;
Position extraction means for extracting, as the same candidate area, a measurement point area existing at the same position as the position of the detected object detected from the detection target object area, among the measurement point areas;
The width of the detected object detected from the detection target object candidate region matches the width of the same candidate region, the feature amount of the detection target candidate region matches the feature amount of a person, and the position calculation When at least one of the position movement calculated by the means or the position movement of the same candidate area matches the walking speed of the person, a person is detected from the detection target candidate area or the same candidate area. Detection means;
Of the first correspondence data indicating the relationship between the position of the person and the size of the detection target area in which the person is detected among the detection target candidate areas, the position of the person, and the detection target candidate area The second correspondence data indicating the relationship with the thermal data value of the detection target area where the person is detected, the position of the person, and the reflection intensity of the detection target area where the person is detected among the same candidate areas Storage means for storing third corresponding data indicating a relationship with the data value,
The feature amount is at least one of thermal data and an aspect ratio,
The detection wave output means scans the periphery of the vehicle with a plurality of detection waves having different detectable ranges, and detects the measurement point region and the position of the measurement point region for each detection wave,
The position extracting means extracts a measurement point area having the highest reliability, which is present at the same position as the position of the detected object detected from the detection target candidate area, as the same candidate area. And
When the width of the detected object detected from the detection target candidate area matches the width of the same candidate area, the detection means determines the size of the detection target candidate area and the detection target candidate area. A determination is made as to whether or not the relationship between the position of the detected object detected from the data matches the first corresponding data, and if it matches, the thermal data value of the detection object candidate region and the detection A determination is made as to whether or not the relationship between the position of the detected object detected from the object candidate area matches the second corresponding data, and if it matches, the reflection intensity data value of the same candidate area Then, a determination is made as to whether or not the relationship with the position of the detected object detected from the detection object candidate region matches the third corresponding data, and if it matches, it is calculated by the position calculation means. Movement of the position Or of the movement of the position of the same candidate region, it is determined whether at least one of which coincides with the walking speed of a person, if they match, detects the person from the detection target object candidate region or the same candidate region An object detection apparatus characterized by that. In an object detection device mounted on a vehicle,
  Image acquisition means for observing the vehicle periphery and acquiring an image having thermal data around the vehicle;
  A region extracting means for extracting a detection target candidate region having thermal data of a person from the image;
  Position calculating means for calculating the position of the detected object detected from the detection target candidate region based on at least one of the position and size of the detection target candidate region;
  A detection wave output means for scanning the periphery of the vehicle with a predetermined detection wave, detecting a measurement point region that reflects the detection wave, and detecting a position of the detected measurement point region;
  Position extraction means for extracting, as the same candidate area, a measurement point area existing at the same position as the position of the detected object detected from the detection target object area, among the measurement point areas;
  The width of the detected object detected from the detection target object candidate region matches the width of the same candidate region, the feature amount of the detection target candidate region matches the feature amount of a person, and the position calculation When at least one of the position movement calculated by the means or the position movement of the same candidate area matches the walking speed of the person, a person is detected from the detection target candidate area or the same candidate area. Detection means;
  The second corresponding data indicating the relationship between the position of the person and the thermal data value of the detection target area where the person is detected among the detection target candidate areas, the position of the person, and the same candidate area Storage means for storing a plurality of third correspondence data indicating the relationship with the reflection intensity data value of the detection target region where the person is detected, according to the position of the person,
  The feature amount is at least one of thermal data and an aspect ratio,
  When the width of the detected object detected from the detection target object region matches the width of the same candidate region, the detection means moves the position of the detected object detected from the detection target candidate region It is determined whether or not the movement of the thermal data value of the detection object candidate area accompanying the movement matches the movement of the second corresponding data accompanying the movement of the position of the person. Whether the movement of the reflection intensity data value of the same candidate area accompanying the movement of the position of the detected object detected from the target object area matches the movement of the third corresponding data accompanying the movement of the position of the person A match determination is made, and if they match, it is determined whether at least one of the position movement calculated by the position calculation means or the position movement of the same candidate region matches the walking speed of the person, If matches are object detecting device and detecting a person from the detection target object candidate region or the same candidate region. In an object detection device mounted on a vehicle,
  Image acquisition means for observing the vehicle periphery and acquiring an image having thermal data around the vehicle;
  A region extracting means for extracting a detection target candidate region having thermal data of a person from the image;
  Position calculating means for calculating the position of the detected object detected from the detection target candidate region based on at least one of the position and size of the detection target candidate region;
  A detection wave output means for scanning the periphery of the vehicle with a predetermined detection wave, detecting a measurement point region that reflects the detection wave, and detecting a position of the detected measurement point region;
  Position extraction means for extracting, as the same candidate area, a measurement point area existing at the same position as the position of the detected object detected from the detection target object area, among the measurement point areas;
  The width of the detected object detected from the detection target object candidate region matches the width of the same candidate region, the feature amount of the detection target candidate region matches the feature amount of a person, and the position calculation When at least one of the position movement calculated by the means or the position movement of the same candidate area matches the walking speed of the person, a person is detected from the detection target candidate area or the same candidate area. Detection means;
  The second corresponding data indicating the relationship between the position of the person and the thermal data value of the detection target area where the person is detected among the detection target candidate areas, the position of the person, and the same candidate area Storage means for storing a plurality of third correspondence data indicating the relationship with the reflection intensity data value of the detection target region where the person is detected, according to the position of the person,
  The feature amount is at least one of thermal data and an aspect ratio,
The detection wave output means scans the periphery of the vehicle with a plurality of detection waves having different detectable ranges, and detects the measurement point region and the position of the measurement point region for each detection wave,
  The position extracting means extracts a measurement point area having the highest reliability, which is present at the same position as the position of the detected object detected from the detection target candidate area, as the same candidate area. And
  When the width of the detected object detected from the detection target object region matches the width of the same candidate region, the detection means moves the position of the detected object detected from the detection target candidate region It is determined whether or not the movement of the thermal data value of the detection object candidate area accompanying the movement matches the movement of the second corresponding data accompanying the movement of the position of the person. Whether the movement of the reflection intensity data value of the same candidate area accompanying the movement of the position of the detected object detected from the target object area matches the movement of the third corresponding data accompanying the movement of the position of the person A match determination is made, and if they match, it is determined whether at least one of the position movement calculated by the position calculation means or the position movement of the same candidate region matches the walking speed of the person, If matches are object detecting device and detecting a person from the detection target object candidate region or the same candidate region. In the object detection device according to any one of claims 1 to 4 ,
The reliability of the first correspondence data, the second correspondence data, and the third correspondence data is different from each other according to the position of the person,
The object detection apparatus characterized in that the detection means performs the matching determination based on correspondence data corresponding to a position of the detected object. A first step of observing the vehicle periphery and obtaining an image having thermal data of the vehicle periphery;
A second step of extracting a detection object candidate region having thermal data of a person from the image;
A third step of calculating the position of the detected object detected from the detection target candidate region based on at least one of the position and size of the detection target candidate region;
A fourth step of scanning the periphery of the vehicle with a predetermined detection wave, detecting a measurement point region reflecting the detection wave, and detecting a position of the detected measurement point region;
A fifth step of extracting, as the same candidate area, a measurement point area existing at the same position as the position of the detected object detected from the detection target object area, among the measurement point areas;
The width of the detection object detected from the detection target object candidate region matches the width of the same candidate region, the feature amount of the detection target candidate region matches the feature amount of a person, and the third If at least one of the position movement calculated by the step or the position movement of the same candidate area matches the walking speed of the person, the person is detected from the detection target candidate area or the same candidate area. And a sixth step of
The feature amount is at least one of thermal data and an aspect ratio ,
In the sixth step, when the width of the detected object detected from the detection object candidate area matches the width of the same candidate area, the size of the detection object candidate area and the detection object The first correspondence data indicating the relationship between the position of the detected object detected from the candidate area and the position of the person and the size of the detection target area where the person is detected among the detection target candidate areas If they match, the relationship between the thermal data value of the detection object candidate area and the position of the detected object detected from the detection object candidate area is In the case of matching, it is determined whether or not the second matching data matches the position and the thermal data value of the detection target area where the person is detected among the detection target candidate areas. Is the reflection strength of the same candidate region The relationship between the data value and the position of the detected object detected from the detection object candidate area is the position of the person, and the reflection intensity data value of the detection object area where the person is detected in the same candidate area. Whether or not the third corresponding data indicating the relationship is matched, and if there is a match, out of the movement of the position calculated by the third step or the movement of the position of the same candidate region, An object detection method comprising determining whether or not at least one of them matches the walking speed of a person and, if they match, detecting a person from the detection object candidate area or the same candidate area .