JP2007334511A - Object detection device, vehicle, object detection method and program for object detection - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object detection device for precisely identifying objects such as pedestrians extracted in different time from an image around the vehicle acquired by an image pickup means loaded on the vehicle by reducing the influence of a residual image to be generated on an image in turning or pitching of a vehicle, and to provide a vehicle, an object detection method and a program for object detection. <P>SOLUTION: When the value of a yaw rate is a predetermined value or more (or pitching is detected) at a time Ta, an identifying means 16 decides whether or not a first object is identical to a second object from the featured values of a section near an edge opposite to the turning direction (or pitching direction) of the vehicle between the both right and left side sections of the image section of the first object in an image acquired at the time Ta and the featured values of the section near the edge opposite to the turning direction (or pitching direction) of the vehicle 10 between the both right and left side sections of the image section of the second object in an image acquired at a time Tb. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an apparatus and a method for detecting an object such as a pedestrian existing in the vicinity of a vehicle from an image acquired via an imaging unit such as a camera mounted on the vehicle. Furthermore, the present invention relates to a program for causing a computer to execute processing of the apparatus.

  In recent years, an imaging means such as a CCD camera is mounted on a vehicle, the surroundings are imaged, a pedestrian or other object existing in the vicinity of the vehicle is detected from the captured image, and walking with high possibility of coming into contact with the vehicle A technique for determining an object such as a driver and presenting information to a driver is known (see, for example, Patent Document 1).

  In the vehicle periphery monitoring device of Patent Document 1, binarization processing, labeling processing, or the like is performed on one image (reference image) of images obtained from two infrared cameras, and an object around the vehicle is displayed. Extract. Then, the object extracted on the reference image is tracked between times, and the object corresponding to the object extracted on the reference image is searched on the other image, and images obtained from two infrared cameras. Based on the above deviation (parallax) of the object, the relative position of the object present around the vehicle with respect to the vehicle is detected as position data. Thereby, time-series data of the position of the object is acquired, and a movement vector of the object with respect to the vehicle is calculated based on the time-series data. Then, an object that is highly likely to come into contact with the vehicle is determined based on the position data of the object and the movement vector.

At this time, in the vehicle periphery monitoring device of Patent Document 1, the process of tracking the object between times is a process of recognizing the identity of the objects extracted at different times. When the object T _k is extracted from the reference image at _k and the object T _{k + 1} is extracted from the reference image at the time k + 1 of the next calculation processing cycle, the object T _{k + 1} and the object T _k are extracted. identity with _k (whether or not the object T _{k + 1} is the same object as the object T _k ) is determined. This identity determination is performed based on the overall characteristics of the object such as the position of the center of gravity, area, and aspect ratio of the object. If it is determined that the object T _k and the object T _{k + 1} are the same object, the label of the object T _{k + 1} is changed to the same label as the label of the object T _k. The As a result, the object T _k and the object T _{k + 1} are recognized as the same object and are tracked between times.

  On the other hand, when the vehicle is turning or pitching, the temporal change in the relative position of the object with respect to the vehicle is larger than the imaging performance of the imaging means, so the image around the road obtained from the imaging means mounted on the vehicle Thus, an afterimage of an object such as a pedestrian may occur. For example, when the vehicle is turning in the right direction, the object is shifted to the left side on the image, so an afterimage is generated on the right side. When the vehicle is turning in the left direction, the object is on the image. Since it shifts to the right side, an afterimage occurs on the left side. Further, for example, when the vehicle is pitched upward, the object is shifted downward on the image, so an afterimage is generated on the upper side. When the vehicle is pitched downward, the image is Since the object is shifted upward, an afterimage is generated on the lower side. When an object is extracted from an image in which an afterimage is generated in this way, characteristics such as the position of the center of gravity, area, and aspect ratio of the object change due to the influence of the afterimage. For this reason, even in the case of the same object, features such as the position of the center of gravity, area, and aspect ratio of the object on the image are different from the characteristics that the object showed on the image in which no afterimage has occurred. It will be.

However, since the influence of such an afterimage is not taken into consideration in the vehicle periphery monitoring device of Patent Document 1, it is the same when determining whether or not the object is the same when turning or pitching the vehicle. If it is not the target object, there is a possibility of being erroneously determined. For this reason, the technique which can recognize appropriately the identity of the target object extracted at mutually different time at the time of turning of a vehicle or pitching was desired.
JP 2001-6096 A

  The present invention has been made in view of such a background, and reduces the influence of an afterimage generated on an image during turning or pitching of the vehicle, from an image around the vehicle obtained by an imaging means mounted on the vehicle. The object detection apparatus, the vehicle, the object detection method and the object detection apparatus capable of accurately recognizing the identity of objects such as pedestrians extracted at different times are executed by a computer. An object is to provide a program for detecting an object.

  In order to achieve this object, the object detection apparatus according to the first aspect of the present invention is an object for sequentially extracting objects existing in the vicinity of a vehicle from an image acquired via an imaging means mounted on the vehicle. The object extraction means and the first object extracted by the object extraction means from the image acquired at an arbitrary time Ta are extracted from the image acquired at a time Tb before the time Ta. And an identity determination means for determining whether or not the second object extracted by the means is the same. The object detection apparatus further comprises a yaw rate detection means for sequentially detecting the yaw rate of the vehicle, and the same The sex determination means has at least the left and right side portions of the image portion of the first object in the image acquired at the time Ta when the magnitude of the yaw rate at the time Ta is a predetermined threshold value or more. Of the feature amount of the portion near the edge opposite to the turning direction of the vehicle and the left and right side portions of the image portion of the second object in the image acquired at the time Tb. It is characterized in that it is determined whether or not the first object is the same as the second object based on the feature amount of the portion near the edge opposite to the turning direction (first invention). ).

  In the object detection device according to the first aspect of the present invention, the identity determination means is configured such that the first object extracted by the object extraction means from an image acquired at an arbitrary time Ta is earlier than the time Ta. It is determined whether or not it is the same as the second object extracted by the object extraction means from the image acquired at time Tb. At this time, an afterimage of the object may occur on the acquired image when the vehicle turns. For example, when the vehicle is turning rightward, an afterimage is generated on the right side of the object on the image, and when the vehicle is turning leftward, an afterimage is displayed on the left side of the object on the image. Arise. Then, when the object is extracted from the image in which the afterimage is generated in this way, the image portion of the afterimage is included and extracted in the image portion of the object. For this reason, the overall feature amount such as the position of the center of gravity, area, and aspect ratio of the image portion of the extracted object changes due to the influence of the afterimage.

  Therefore, the identity determination means determines that the yaw rate at the time Ta is greater than or equal to a predetermined threshold (when the vehicle turns and the yaw rate is large enough to cause an afterimage on the image). , The feature amount of the portion near the edge opposite to the turning direction of the vehicle in the left and right side portions of the image portion of the first object in the image acquired at time Ta, and acquired at time Tb. The first object is determined based on the feature amount of the portion near the edge opposite to the turning direction of the vehicle in the left and right side portions of the image portion of the second object in the captured image. It is determined whether or not it is the same as the second object.

  That is, since the portion near the edge opposite to the turning direction of the vehicle is an image portion on the side where no afterimage is generated, the feature amount of this portion is one in which the influence of the afterimage is reduced. Therefore, the identity determination means determines whether the first object is the same as the second object based on the feature amount of this portion, so that when the vehicle turns, Thus, it is possible to reduce the influence of the afterimage that occurs in the vehicle and accurately recognize the identity of the objects extracted from the images around the vehicle at different times.

  The identity determination means determines that at the time Ta when the magnitude of the yaw rate at the time Ta is less than a predetermined threshold (when the magnitude of the yaw rate is sufficiently small so that no afterimage is generated on the image). The overall feature amount of the image portion of the first object in the acquired image (the feature amount of the entire image portion including both the left and right side portions of the object) and the image acquired at time Tb What is necessary is just to determine whether this 1st target object is the same as this 2nd target object based on the whole feature-value of the image part of a said 2nd target object.

  In the object detection device of the first invention, the object extracted by the object extraction unit from an image acquired when the magnitude of the yaw rate detected by the yaw rate detection unit is less than the predetermined threshold value. Object image storage means for sequentially storing the image portion as an object image, wherein the identity determination means is configured such that the time Ta and the time Tb are at least the magnitude of the yaw rate less than the predetermined value. The image of the first object in the image acquired at the time Ta immediately after the change and when the yaw rate is a time within a predetermined period in which the magnitude of the yaw rate is maintained above the predetermined value. The time immediately before the feature amount of the portion near the edge opposite to the turning direction of the vehicle and the magnitude of the yaw rate change beyond the predetermined threshold value in the left and right side portions of the portion The first feature amount of the left and right side portions of the object image stored by the object image storage means from the image acquired in c is the feature amount of the portion near the edge opposite to the turning direction of the vehicle. The degree of similarity, and the feature amount of the edge portion on the opposite side to the turning direction of the vehicle, of the left and right side portions of the image portion of the second object in the image acquired at the time Tb, Based on the second degree of similarity between the left and right side parts of the object image at the time Tc and the feature amount of the part near the edge on the opposite side to the turning direction of the vehicle, the first object It is preferable to determine whether or not an object is the same as the second object (second invention).

  According to this, the object image storage means is sufficiently large when the magnitude of the yaw rate detected by the yaw rate detection means is less than the predetermined threshold (the magnitude of the yaw rate does not cause an afterimage on the image). The image portion of the target object extracted by the target object extraction means from the acquired image is sequentially stored as the target object image. This object image is an image portion of the object in a state in which no afterimage is generated. Since the object image does not include noise due to the afterimage, the feature amount of the object appears remarkably.

  On the other hand, the time Ta and the time Tb are at least immediately after the magnitude of the yaw rate changes from less than the predetermined value to the predetermined value or more, and a predetermined period in which the magnitude of the yaw rate is maintained above the predetermined value. Is an afterimage both on the image acquired at time Ta and on the image acquired at time Tb, and the first image extracted from these images. The image portions of the first and second objects include afterimage noise. The predetermined period is a period until the same object does not exist on the image.

  Here, both of the first object in the image acquired at the time Ta and the second object in the image acquired at the time Tb have a magnitude of the yaw rate equal to the predetermined threshold value. The object corresponding to the object image stored by the object image storage means from the image acquired at time Tc immediately before the change (extracted by the object extraction means from the image acquired at time Tc). If it is found that it is the same as the third object), it is found that the first object is the same as the second object. Note that the object image at the time Tc immediately before the magnitude of the yaw rate changes to the predetermined threshold value or more is an object image stored before the value of the yaw rate changes to the predetermined threshold value or more. It is the object image made.

  Therefore, the identity determination means includes a portion near the edge opposite to the turning direction of the vehicle, out of the left and right side portions of the image portion of the first object in the image acquired at the time Ta. The first similarity degree between the feature amount and the feature amount of the portion near the edge opposite to the turning direction of the vehicle in the left and right side portions of the object image at the time Tc, and the time Tb Among the left and right side portions of the image portion of the second object in the acquired image, the feature amount of the edge side portion opposite to the turning direction of the vehicle, and the object image at the time Tc The first object is the second object based on the second degree of similarity between the left and right side portions and the feature amount of the portion near the edge on the opposite side to the turning direction of the vehicle. It is determined whether or not the same.

  That is, of the left and right side portions of the image portions of the first and second objects, the portion near the edge opposite to the turning direction of the vehicle is an image portion on the side where no afterimage is generated. The feature amounts of these portions are those in which the influence of the afterimage is reduced. Further, since the feature values of these portions are compared with the feature values of the object image in which no afterimage is generated, the influence of the afterimage is eliminated, and the first and second similarities include the first and second similarities. And whether the second object is the same as the third object is accurately indicated. Therefore, by determining whether or not the first object is the same as the second object based on the first and second similarity degrees, the identity determination means At the time of turning, it is possible to reduce the influence of an afterimage generated on the image, and to accurately recognize the identity of the objects extracted from the images around the vehicle at different times.

  Next, the object detection device according to the second aspect of the present invention includes an object extraction unit that sequentially extracts an object existing around the vehicle from an image acquired via an imaging unit mounted on the vehicle, The first object extracted by the object extracting unit from the image acquired at an arbitrary time Ta is extracted by the object extracting unit from the image acquired at the time Tb before the time Ta. An object detection device comprising identity determination means for determining whether or not the second object is the same, further comprising pitching detection means for sequentially detecting the pitching of the vehicle, wherein the identity determination means includes: At least when the pitching is detected by the pitching detection means at the time Ta, the vehicle in the upper and lower side portions of the image portion of the first object in the image acquired at the time Ta. Of the feature amount of the portion near the edge opposite to the pitching direction and the upper and lower side portions of the image portion of the second object in the image acquired at the time Tb, the side opposite to the pitching direction of the vehicle It is characterized in that it is determined whether or not the first object is the same as the second object based on the feature amount of the portion closer to the edge (third invention).

  In the object detection device according to the third aspect of the present invention, as in the first aspect, the identity determination means is the first object extracted by the object extraction means from an image acquired at an arbitrary time Ta. Is the same as the second object extracted by the object extracting means from the image acquired at time Tb before time Ta. At this time, when the vehicle is pitched, an afterimage of the object may occur on the acquired image. For example, when the vehicle is pitching upward, an afterimage is generated above the object on the image, and when the vehicle is pitching downward, the afterimage is displayed below the object on the image. Occurs. Then, when the object is extracted from the image in which the afterimage is generated in this way, the image portion of the afterimage is included and extracted in the image portion of the object. For this reason, the overall feature amount such as the position of the center of gravity, area, and aspect ratio of the image portion of the extracted object changes due to the influence of the afterimage.

  Therefore, the identity determination means is acquired at time Ta when the pitching detection means detects the pitching of the vehicle at time Ta (when the vehicle is pitching enough to cause an afterimage on the image). Of the upper and lower side parts of the image part of the first object in the image, the feature amount of the part near the edge opposite to the pitching direction of the vehicle, and the second in the image acquired at time Tb The first object is the same as the second object on the basis of the feature amount of the portion near the edge opposite to the pitching direction of the vehicle in the upper and lower side portions of the image portion of the object. It is determined whether or not. The pitching detection means has a state quantity detection value (for example, a translational velocity in the vertical direction of the vehicle, an angular velocity around the pitching axis of the vehicle, etc.) that is changed by the pitching of the vehicle being a predetermined threshold value or more. In addition, it is determined that pitching has been detected, and it is determined that no pitching has been detected when the magnitude of the detected value is less than the predetermined threshold.

  That is, since the portion near the edge opposite to the pitching direction of the vehicle is an image portion on the side where no afterimage is generated, the feature amount of this portion has a reduced influence of the afterimage. Therefore, the identity determination means determines whether the first object is the same as the second object based on the feature amount of this portion, so that when the vehicle is pitched, Thus, it is possible to reduce the influence of the afterimage that occurs in the vehicle and accurately recognize the identity of the objects extracted from the images around the vehicle at different times.

  The identity determination means is acquired at time Ta when pitching is not detected by the pitching detection means at time Ta (when the vehicle is not pitched enough to cause an afterimage on the image). The overall feature amount of the image portion of the first object in the image (the feature amount of the entire image portion including both the left and right side portions of the object) and the second feature amount in the image acquired at time Tb Whether or not the first object is the same as the second object may be determined based on the overall feature amount of the image portion of the object.

Further, in the object detection device of the third invention, an image portion of the object extracted by the object extraction means from an image acquired when no pitching is detected by the pitching detection means is obtained as an object image. As an object image storage means for sequentially storing as,
The identity determination means is a state immediately after the time Ta and the time Tb are changed from a state in which no pitching is detected by the pitching detection means to a state in which pitching is detected, and in which the pitching is detected. Of the left and right side portions of the image portion of the first object in the image acquired at the time Ta, on the opposite side to the pitching direction of the vehicle. Of the left and right side portions of the object image stored by the object image storage means from the feature amount of the portion near the edge and the image acquired at the time Tc immediately before the pitching is detected, The first similarity degree with the feature quantity of the portion near the edge opposite to the pitching direction of the vehicle, and the image of the second object in the image acquired at the time Tb Of the left and right side portions of the minute, and the feature amount of the edge side portion opposite to the pitching direction of the vehicle, and the pitching direction of the vehicle of the left and right side portions of the object image at the time Tc It is preferable to determine whether or not the first object is the same as the second object based on the second degree of similarity with the feature amount of the portion near the opposite edge ( Fourth invention).

  According to this, the object image storage means is configured to detect the object from an image acquired when pitching is not detected by the pitching detection means (when the vehicle is not pitched to the extent that an afterimage is generated on the image). The image portion of the object extracted by the object extracting means is sequentially stored as the object image. This object image is an image portion of the object in a state in which no afterimage is generated. Since the object image does not include noise due to the afterimage, the feature amount of the object appears remarkably.

  On the other hand, the time Ta and the time Tb are at least immediately after changing from a state where no pitching is detected by the pitching detection means to a state where pitching is detected, and a state where the state where the pitching is detected is maintained. When the time is within the period, an afterimage is generated both on the image acquired at the time Ta and on the image acquired at the time Tb, and extracted from these images. The image portions of the first and second objects include afterimage noise. The predetermined period is a period until the same object does not exist on the image.

  Here, both the first object in the image acquired at the time Ta and the second object in the image acquired at the time Tb are in a state in which the pitching is detected. The object corresponding to the object image stored by the object image storage means from the image acquired at the previous time Tc (the third object extracted by the object extraction means from the image acquired at the time Tc) If it is found that the first object is the same as the second object, the first object is the same as the second object. Note that the object image at the time Tc immediately before the pitching is detected is the object image stored before the pitching is detected, and is stored most recently. It is an object image.

  Therefore, the identity determination means includes a portion of the left and right side portions of the image portion of the first object in the image acquired at the time Ta that is closer to the edge opposite to the pitching direction of the vehicle. The degree of similarity between the feature amount and the feature amount of the left and right side portions of the object image at the time Tc and the portion near the edge opposite to the pitching direction of the vehicle is acquired at the time Tb. Among the left and right side portions of the image portion of the second object in the captured image, the feature amount of the edge portion on the opposite side to the pitching direction of the vehicle, and both the left and right sides of the object image at the time Tc Whether or not the first object is the same as the second object based on the degree of similarity between the feature amount of the portion near the edge opposite to the pitching direction of the vehicle. Determine whether.

  That is, of the left and right side portions of the image portions of the first and second objects, the portion near the edge opposite to the vehicle pitching direction is an image portion on the side where no afterimage is generated. The feature amounts of these portions are those in which the influence of the afterimage is reduced. Further, since the feature values of these portions are compared with the feature values of the object image in which no afterimage is generated, the influence of the afterimage is eliminated, and the first and second similarities include the first and second similarities. And whether the second object is the same as the third object is accurately indicated. Therefore, by determining whether or not the first object is the same as the second object based on the first and second similarity degrees, the identity determination means At the time of pitching, it is possible to reduce the influence of an afterimage generated on the image, and to accurately recognize the identity of objects extracted from images around the vehicle at different times.

  Furthermore, in the object detection device according to any one of the first to fourth inventions, the object detection apparatus includes a distance detection unit that detects a distance of the object extracted by the object extraction unit with respect to the vehicle, and the identity determination unit includes: The distance detected by the distance detector of the first object in the image acquired at time Ta and the object corresponding to the object image at the time Tc detected by the distance detector. Means for correcting the size of the object image based on the distance, the first similarity degree is calculated using the corrected object image, and acquired at the time Tb Based on the distance detected by the distance detection means of the second object in the image and the distance detected by the distance detection means of the object corresponding to the object image at the time Tc. Comprising means for correcting the size of the object image, by using the corrected target image, we are preferable to calculate the second similarity degree (the fifth invention).

  That is, when the distance of the object to the vehicle changes, the size of the image portion of the object changes in accordance with the change of the distance on the image acquired via the imaging means mounted on the vehicle. Therefore, the identity determination unit includes a distance detected by the distance detection unit of the first object in the image acquired at the time Ta, and an object corresponding to the object image at the time Tc. The size of the object image is corrected based on the distance detected by the distance detecting means. Also, the distance detected by the distance detection unit of the second object in the image acquired at the time Tb and the distance detection unit of the object corresponding to the object image at the time Tc. Based on the detected distance, the size of the object image is corrected. That is, the size of the object image is corrected so as to reduce the influence of a change in the distance of the object to the vehicle.

  Then, the identity determination means calculates the first and second similar degrees using the corrected object images. Therefore, the calculated first and second similarity degrees are the feature amounts of the image portions of the first and second objects and the object image regardless of the change in the distance of the object to the vehicle. The degree of similarity with the object corresponding to is more appropriately indicated. Therefore, the identity determination means determines whether the first object is the same as the second object based on the calculated first and second similar degrees. It is possible to reduce the influence of afterimages generated on the image at the time of turning or pitching of the vehicle, and accurately recognize the identity of objects extracted at different times from images around the vehicle.

  Next, the vehicle of the present invention is characterized in that any one of the object detection devices of the first to fifth inventions is mounted (sixth invention). According to the vehicle of the sixth aspect of the invention, a vehicle having the same effects as the object detection device of the present invention can be realized.

  Next, the object detection method according to the first aspect of the present invention includes an object extraction step of sequentially extracting objects existing around the vehicle from an image acquired via an imaging unit mounted on the vehicle; The first object extracted by the object extraction step from the image acquired at an arbitrary time Ta is extracted by the object extraction step from the image acquired at time Tb prior to the time Ta. An object detection method comprising: an identity determination step for determining whether or not the second object is the same; and a yaw rate detection step for sequentially detecting the yaw rate of the vehicle, wherein the identity determination step includes: When at least the magnitude of the yaw rate at the time Ta is equal to or greater than a predetermined threshold, both left and right sides of the image portion of the first object in the image acquired at the time Ta Of the minute, the feature amount of the portion near the edge opposite to the turning direction of the vehicle, and the left and right side portions of the image portion of the second object in the image acquired at the time Tb, It is characterized in that it is determined whether or not the first object is the same as the second object based on the feature amount of the portion near the edge on the opposite side to the turning direction of the vehicle ( (Seventh invention).

  According to the object detection method of the seventh aspect of the invention, as described in relation to the first aspect of the invention, when the magnitude of the yaw rate at the time Ta is equal to or greater than the predetermined threshold (the vehicle turns and the magnitude of the yaw rate is an image). Of the left and right sides of the image portion of the object extracted at time Ta), the portion near the edge opposite to the turning direction of the vehicle is an afterimage. Therefore, the feature amount of this part is one in which the influence of the afterimage is eliminated. Then, when the magnitude of the yaw rate at the time Ta is equal to or greater than a predetermined threshold, the first object is detected at the time Tb based on the feature amount of the portion in which the influence of the afterimage is reduced in the identity determination step. It is determined whether or not it is the same as the second object in the acquired image. Therefore, according to the present invention, it is possible to reduce the influence of an afterimage generated on an image when the vehicle turns, and to accurately recognize the identity of objects extracted from images around the vehicle at different times. .

  Next, the object detection method according to the second aspect of the present invention includes an object extraction step of sequentially extracting objects existing around the vehicle from an image acquired through an imaging unit mounted on the vehicle; The first object extracted by the object extraction step from the image acquired at an arbitrary time Ta is extracted by the object extraction step from the image acquired at time Tb prior to the time Ta. An object detection method comprising: an identity determination step for determining whether or not the second object is the same; and a pitching detection step for sequentially detecting pitching of the vehicle, wherein the identity determination step includes: , At least when the pitching is detected in the pitching detection step at the time Ta, the upper portion of the image portion of the first object in the image acquired at the time Ta Of the two side portions of the vehicle, the feature amount of the portion closer to the edge opposite to the pitching direction of the vehicle, and the upper and lower side portions of the image portion of the second object in the image acquired at the time Tb And determining whether or not the first object is the same as the second object based on the feature amount of the portion near the edge opposite to the pitching direction of the vehicle. (8th invention).

  According to the object detection method of the eighth invention, as described with respect to the third invention, when pitching is detected in the pitching detection step at time Ta (the vehicle is pitched to the extent that an afterimage is generated on the image). In the image acquired at time Ta, an afterimage is generated in the upper and lower side portions of the image portion of the first object near the edge opposite to the vehicle pitching direction. Since this is an image portion on the side that is not, the feature amount of this portion is one in which the influence of the afterimage is eliminated. In the pitching detection step, when the detected value of the state quantity that changes due to the pitching of the vehicle (for example, the vertical translational velocity of the vehicle, the angular velocity around the pitching axis of the vehicle, etc.) is greater than or equal to a predetermined threshold value. In addition, it is determined that pitching has been detected, and it is determined that no pitching has been detected when the magnitude of the detected value is less than the predetermined threshold.

  Then, when pitching is detected at time Ta, in the identity determination step, the first object in the image acquired at time Tb is based on the feature amount of the portion where the influence of the afterimage is reduced. It is determined whether or not it is the same as the second object. Therefore, according to the present invention, it is possible to reduce the influence of an afterimage generated on an image when the vehicle is pitched, and accurately recognize the identity of objects extracted at different times from images around the vehicle. .

  Next, the object detection program according to the first aspect of the present invention includes an object extraction process for sequentially extracting objects existing around the vehicle from an image acquired via an imaging unit mounted on the vehicle. The first object extracted from the image acquired at an arbitrary time Ta by the object extraction process is extracted from the image acquired at the time Tb before the time Ta by the object extraction process. An object detection program for causing a computer to execute an identity determination process for determining whether or not the second object is the same, and a yaw rate acquisition process for sequentially acquiring a detected value of the yaw rate of the vehicle A function to be executed by a computer, and as the identity determination process, at least when the yaw rate at the time Ta is greater than or equal to a predetermined threshold, an image acquired at the time Ta Of the left and right side portions of the image portion of the first object, the feature amount of the portion near the edge opposite to the turning direction of the vehicle, and the second target in the image acquired at time Tb The first object is the same as the second object based on the feature amount of the left and right side parts of the image portion of the object and the portion near the edge opposite to the turning direction of the vehicle. A program having a function of causing a computer to execute a process for determining whether or not there is (ninth invention).

  According to the object detection program of the ninth aspect of the invention, it is possible to cause a computer to execute a process that can achieve the effects described with respect to the first aspect of the invention.

  Next, the object detection program according to the second aspect of the present invention includes an object extraction process for sequentially extracting objects existing around the vehicle from an image acquired through an imaging unit mounted on the vehicle. The first object extracted by the object extraction process from the image acquired at an arbitrary time Ta is extracted by the object extraction means from the image acquired at the time Tb before the time Ta. An object determination program for causing a computer to execute an identity determination process for determining whether or not the second object is the same as the second object, and sequentially detecting detection values of state quantities that change due to pitching of the vehicle. A function for causing a computer to execute a pitching acquisition process to be acquired, and as the identity determination process, at least a pitch based on a detection value acquired by the pitching acquisition process at the time Ta The feature of the portion near the edge opposite to the pitching direction of the vehicle in the upper and lower side portions of the image portion of the first object in the image acquired at the time Ta when the motion is detected Based on the amount and the feature amount of the portion near the edge opposite to the pitching direction of the vehicle in the upper and lower side portions of the image portion of the second object in the image acquired at the time Tb A program having a function of causing a computer to execute a process of determining whether or not the first object is the same as the second object (a tenth invention).

  According to the object detection program of the tenth aspect of the present invention, it is possible to cause a computer to execute processing that can achieve the effects described with respect to the third aspect of the present invention.

  An embodiment of the present invention will be described with reference to the accompanying drawings. First, a first embodiment of the present invention will be described. FIG. 1 is a functional block diagram of the object detection device according to the present embodiment, FIG. 2 is an explanatory view of an attachment mode of the object detection device shown in FIG. 1 to the vehicle, and FIG. It is a flowchart which shows the target object detection / alerting | wakening start work in the target object detection apparatus. FIG. 4 is a view showing an example of a processed image in the object detection / attention activation work of FIG. In addition, this embodiment is corresponded to the target object detection apparatus of the 1st aspect of this invention.

  With reference to FIGS. 1 and 2, the object detection device of the present embodiment includes an image processing unit 1 that is an electronic unit including a CPU (central processing unit). Two infrared cameras 2R and 2L are connected to the image processing unit 1, and the yaw rate sensor 3 for sequentially detecting the yaw rate of the host vehicle 10 as a sensor for detecting the traveling state of the host vehicle 10; A vehicle speed sensor 4 for sequentially detecting a traveling speed (vehicle speed) and a brake sensor 5 for sequentially detecting a brake operation of the host vehicle 10 are connected. The acceleration sensor 17 is not provided in the present embodiment, but is provided in a second embodiment to be described later, and thus description thereof is omitted here.

  Further, the image processing unit 1 includes a speaker 6 mounted on the host vehicle 10 for outputting auditory alert information by voice and the like, images captured by the infrared cameras 2R and 2L, and visual attention. A display device 7 for displaying the arousal information is connected. The display device 7 includes, for example, a HUD (head-up display) 7 a that displays information such as an image on the front window of the host vehicle 10. The HUD 7 a is provided so that the screen is displayed at a position that does not obstruct the driver's forward view of the front window of the host vehicle 10.

  The infrared cameras 2R and 2L are cameras that can detect far-infrared rays, and have a characteristic that the output signal level increases (the luminance increases) as the temperature of the object increases. The infrared cameras 2R and 2L correspond to the imaging means of the present invention.

  As shown in FIG. 2, the infrared cameras 2 </ b> R and 2 </ b> L are attached to the front portion of the host vehicle 10 at a predetermined interval in order to capture the front of the host vehicle 10. The infrared cameras 2R and 2L are fixed to the front portion of the host vehicle 10 so that their optical axes are parallel to each other and the heights of the respective optical axes from the road surface are equal.

  Although not shown in detail, the image processing unit 1 is configured by an electronic circuit including an A / D conversion circuit, a CPU, a RAM, a ROM, an image memory, and the like, and includes infrared cameras 2R and 2L, a yaw rate sensor 3, and a vehicle speed sensor 4. The output (analog signal) of the brake sensor 5 is converted into a digital signal via the A / D conversion circuit and input. Then, the image processing unit 1 performs processing for detecting an object such as a pedestrian based on the input data and whether or not a predetermined requirement regarding the detected object is satisfied for each predetermined arithmetic processing cycle. When the requirement is satisfied, a process of calling attention (calling the driver's attention to the object) through the speaker 6 or the display device 7 is executed. These processes are realized by causing the image processing unit 1 to execute a program mounted in advance in the ROM of the image processing unit 1, and the program includes the object detection program of the present invention.

  More specifically, the image processing unit 1 includes, as functions realized by the above-described program, the object extraction unit 11 that extracts an object existing around the host vehicle 10 from the acquired image, and the extracted object. Pedestrian determination means 12 for determining whether or not the object is a pedestrian from the image portion, and a pedestrian image is sequentially stored when the magnitude of the yaw rate detected by the yaw rate sensor 3 is less than a predetermined threshold. Pedestrian image storage means 13 and distance detection means 14 for detecting the distance of the object to the host vehicle 10 are provided. In addition, the image processing unit 1 has, as its function, the magnitude of the yaw rate detected by the identity determination means 16 for determining the identity of the object extracted every calculation processing cycle and the yaw rate sensor 3 is less than a predetermined threshold. The object image storage means 15 for sequentially storing the object images.

  The object extraction means 11 extracts an object existing around the host vehicle 10 from images acquired via the infrared cameras 2R and 2L. Specifically, the object extracting means 11 is a predetermined reference image among images acquired via the infrared cameras 2R and 2L (in this embodiment, it is assumed that the image is acquired via the infrared camera 2R). A binarization process or the like is performed on the image, and a labeling process or the like is further performed to extract an image portion of the object.

  The distance detection means 14 searches for an object corresponding to the object extracted by the object extraction means 11 on the reference image on the image acquired via the infrared camera 2L, and from the two infrared cameras 2R and 2L. Based on the deviation (parallax) of the object on the obtained image, the distance of the object to the host vehicle 10 is detected. In addition, as a specific method for detecting the distance of the object based on the image, for example, a method as described in Patent Document 1 described above can be used.

  The pedestrian image storage means 13 is an object that has been determined to be a pedestrian by the pedestrian determination means 12 from an image acquired when the magnitude of the yaw rate detected by the yaw rate sensor 3 is less than a predetermined threshold. Image portions are sequentially stored as pedestrian images.

  The pedestrian determination unit 12 determines whether or not the target object is a pedestrian from the image portion of the target object extracted by the target object extraction unit 11. Specifically, when the magnitude of the yaw rate is greater than or equal to a predetermined threshold, the pedestrian determination unit 12 determines that the host vehicle is out of the left and right side portions of the image portion of the object extracted from the image acquired at this time. Among the left and right side portions of the pedestrian image stored immediately before the feature amount of the portion near the edge opposite to the turning direction of 10 and the magnitude of the yaw rate change to a predetermined threshold value or more, It is determined whether or not the object is a pedestrian based on the degree of similarity between the turning direction and the feature amount of the portion near the edge on the opposite side. Note that the pedestrian determination unit 12 corrects the size of the pedestrian image based on the detection result of the distance detection unit 14, and calculates the degree of similarity using the corrected pedestrian image.

  In addition, when the magnitude of the yaw rate is less than the predetermined threshold, the pedestrian determination unit 12 determines that the target object is based on the overall feature amount of the image portion of the target object extracted from the image acquired at this time. It is determined whether or not the person is a pedestrian. Note that the overall feature amount of the image portion of the object used for the determination of the pedestrian is, for example, the shape of the object, luminance dispersion, and the like.

  The object image storage means 15 obtains an image portion of the object extracted by the object extraction means 11 from an image acquired when the magnitude of the yaw rate detected by the yaw rate sensor 3 is less than a predetermined threshold. Store sequentially as images.

  The identity determination unit 16 acquires the first object extracted by the object extraction unit 11 from the image acquired at an arbitrary time Ta at the time Tb of the arithmetic processing cycle before the time Ta. It is determined whether or not the second object extracted from the image by the object extracting means 11 is the same. Specifically, when the magnitude of the yaw rate at the time Ta is greater than or equal to a predetermined threshold, the identity determination unit 16 determines the left and right side portions of the image portion of the first object in the image acquired at the time Ta. Of the vehicle, the feature amount of the portion near the edge opposite to the turning direction of the host vehicle 10 and the left and right side portions of the image portion of the second object in the image acquired at time Tb. Whether the first object is the same as the second object or not is determined based on the turning direction and the feature value of the portion near the edge on the opposite side.

  In addition, the identity determination unit 16 determines, when the magnitude of the yaw rate at the time Ta is less than a predetermined threshold, the overall feature amount of the image portion of the first object in the image acquired at the time Ta, and the time Whether or not the first object is the same as the second object is determined based on the overall feature amount of the image portion of the second object in the image acquired at Tb. Note that the overall feature amount of the image portion of the target used for identity determination is, for example, the position of the center of gravity of the target, the area, the aspect ratio of the circumscribed rectangle of the target, and the like.

  Next, the overall operation (object detection / alarm operation) of the object detection device of the present embodiment will be described with reference to the flowchart shown in FIG. Referring to FIG. 3, image processing unit 1 repeats the processing of STEP 001 to STEP 020 for each predetermined calculation processing cycle to execute object detection / attention activation work. First, the image processing unit 1 acquires an infrared image that is an output signal of the infrared cameras 2R and 2L (STEP001), performs A / D conversion (STEP002), and stores a grayscale image in an image memory (STEP003). The right image is obtained by the infrared camera 2R, and the left image is obtained by the infrared camera 2L. Since the right image and the left image are displayed with the positions in the horizontal direction (x direction) shifted on the image of the same object, the distance to the object can be calculated based on this shift (parallax). it can.

  Next, the image processing unit 1 binarizes the image signal with respect to the reference image in the grayscale image (STEP 004). That is, a process is performed in which a region where the luminance value of the image signal of the reference image is brighter than the threshold value Ith is set to “1” (white) and a dark region is set to “0” (black). The threshold value Ith is a value that is experimentally determined in advance.

  FIG. 4A illustrates an image obtained by binarizing an image obtained by the infrared camera 2R at a time (discrete system time) k in a certain arithmetic processing cycle. In FIG. 4A, the hatched area is black, and the area surrounded by the thick solid line is white. A region surrounded by a thick solid line is a region of an object that has a high luminance level (at a high temperature) and is displayed as white on the screen in an image obtained from the infrared camera 2R. The example shown in FIG. 4A is an example where a pedestrian is present in front of the host vehicle 10 and the host vehicle 10 is traveling straight ahead.

  Next, the image processing unit 1 creates run-length data from an area that has become “white” in the binarization process (hereinafter referred to as a binarized area) (STEP005). The generated run-length data represents a binarized area as a set of lines that are one-dimensional connected pixels in the horizontal direction of the image, and represents each line constituting the binarized area from the coordinates of the start point and the start point. This is indicated by the length to the end point (number of pixels).

  Next, the image processing unit 1 extracts the target object by labeling the target object (STEP 006) based on the generated run length data (STEP 007). That is, among the lines represented by the run-length data, a line having a portion overlapping in the vertical direction (y direction) of the image is regarded as one object, and a label (identifier) is attached, thereby connecting connected regions in the image. Is extracted as an object.

By the above-described processing of STEP 005 to 007, the binarized area shown in FIG. 4A is extracted as the target (binarized target) T _k as illustrated in FIG. 4B. At this time, for example, the object of the label T _k is indicated by n pieces of run-length data L1 to Ln. The extracted objects (binarized objects) include, for example, other vehicles, artificial structures such as electric poles and vending machines, in addition to pedestrians around the road. In the example of FIG. 4B, the target T _k is a binarized target corresponding to a pedestrian existing in front of the host vehicle 10.

Next, the image processing unit 1 calculates the area S, the gravity center position Gc, the height Hb, the width Wb, the gravity center position Gb, and the aspect ratio ASP of the circumscribed rectangle of the object (STEP008). Specifically, in the example of FIG. 4 (b), the area S _k of the object T _k are each run-length data Li (i = 1, ..., n) the length of the line indicated by the subject Calculation is performed by integrating the n run-length data of the object T _k . The center-of-gravity position Gc _k of the object T _k is the length of the line indicated by each run-length data Li and the coordinates (x [i], y [i]) of the midpoint of each line of the run-length data Li. the multiplied respectively, those obtained by integrating the further this object T1 n pieces of run-length data, is calculated by dividing the area S _k. Further, the aspect ratio ASP _k of the circumscribed quadrangle of the object T _k is the height of the rectangle circumscribing the object T _k (vertical length) Hb _k and width (lateral length) Wb _k and the ratio Hb _k / Wb _k is calculated. Note that the image portion R _k of the object T _k is the entire area of the rectangle circumscribing the object T _k.

  Next, the image processing unit 1 reads the yaw rate YR detected by the yaw rate sensor 3 and the vehicle speed VCAR detected by the vehicle speed sensor 4 (STEP 009). In STEP 009, the turning angle θr of the host vehicle 10 is also calculated by integrating the yaw rate YR over time.

Next, the image processing unit 1 performs tracking of the object during the time, that is, a process of recognizing the same object every calculation processing period of the image processing unit 1 (STEP010). In the same object recognition process, when the object T _k is extracted by the process of STEP 007 at time k, and the object T _{k + 1} is extracted by the process of STEP 007 at time k + 1 of the next calculation processing cycle, This is a process of determining the identity between the object T _{k + 1} and the object T _k (whether the object T _{k + 1} is the same object as the object T _k ). If it is determined that the object T _k and the object T _{k + 1} are the same object, the label of the object T _{k + 1} is changed to the same label as the label of the object T _k. The As a result, the object T _k and the object T _{k + 1} are recognized as the same object, and are tracked between times. This same object recognition process is performed on the reference image. The details of the same object recognition process will be described later.

  On the other hand, the image processing unit 1 executes the processes of STEP 011 to 014 in parallel with the processes of STEP 009 and 010. The processing of STEPs 011 to 014 is processing for calculating a distance z between the object and the host vehicle 10 (a distance in the front-rear direction of the host vehicle 10).

  First, the image processing unit 1 selects one of the objects to be tracked by the binarized image of the reference image, and selects the search image R1 (the entire circumscribed square area surrounding the selected object) from the reference image. The search image is extracted (STEP 011).

  Next, the image processing unit 1 starts from the reference image (the right image obtained from the infrared cameras 2R and 2L and the left image that is not the standard image, in this example, the left image obtained from the infrared camera 2L). A search area for searching for an image corresponding to the search image R1 (hereinafter referred to as “corresponding image”) is set, and a correlation operation is executed to extract the corresponding image (STEP012). Specifically, the search area R2 is set in the reference image according to each vertex coordinate of the search image R1, and the coordinates (x0, y0) are set as the base point (upper left vertex of the area) in the search area R2. A local region R3 having the same shape as the search image R1 is set. Then, by changing the coordinates (x0, y0) of the base point and moving the local region R3 within the search region R2, the absolute difference sum of luminance values (SAD) indicating the degree of correlation between the local region R3 and the search image R1 , Sum of Absolute Difference) C (x0, y0) is calculated by the following equation (1).

  Here, the absolute difference sum C (x0, y0) is a local area based on the luminance value IR of the pixel at the coordinates (m, n) in the search image R1 and the coordinates (x0, y0) in the search area R2. The absolute value of the difference from the luminance value IL of the pixel at the coordinates (x0 + m, y0 + n) in R3 is taken, and all pixels (m = 0,...,. M−1, n = 0,..., N−1) is obtained as a total value. Note that the smaller the value of the absolute difference sum C (x0, y0), the higher the degree of correlation between the search image R1 and the local region R3. Thus, the coordinates (x0, y0) of the base point at which the absolute difference sum C (x0, y0) is minimum are obtained, and the local region R3 at this position is extracted as the corresponding image R4. This correlation calculation is performed using a grayscale image instead of a binarized image. Through the processing in STEPs 011 to 012, the search image R1 in the standard image and the corresponding image R4 in the reference image are extracted.

  Next, the image processing unit 1 calculates the parallax Δd (number of pixels) based on the centroid position of the search image R1 and the centroid position of the corresponding image R4 (STEP013). Then, the image processing unit 1 calculates the distance z between the host vehicle 10 and the object by the following equation (2) using the calculated parallax Δd (STEP014).

z = B × F / (Δd × p) (2)
Note that B is the base line length (interval of the optical axis) of the infrared cameras 2R and 2L, F is the focal length F of the infrared cameras 2R and 2L, and p is the pixel interval.

  After the processing of STEP010 and STEP014, the image processing unit 1 next converts the coordinates (x, y) and the distance z in the image into real space coordinates, and the position of each object in the real space. The real space position (relative position with respect to the host vehicle 10) is calculated (STEP015). Here, the real space position is, as shown in FIG. 2, a real space coordinate system (XYZ coordinate system) set with the midpoint of the attachment position of the infrared cameras 2R and 2L (the position fixed to the host vehicle 10) as the origin. ) At position (X, Y, Z). The X direction and Y direction of the real space coordinate system are the left-right direction (vehicle width direction) and the up-down direction of the host vehicle 10, respectively. These X direction and Y direction are the x direction (horizontal direction) of the right image and the left image. Direction) and y direction (longitudinal direction). The Z direction in the real space coordinate system is the front-rear direction of the host vehicle 10. The real space position (X, Y, Z) is calculated by the following equations (3), (4), and (5).

X = x × z × p / f (3)
Y = y × z × p / f (4)
Z = z (5)
Next, the image processing unit 1 uses the turning angle θr calculated in STEP 009 to compensate for the influence of the change in the turning angle of the host vehicle 10 and increase the accuracy of the real space position of the object. The real space position of is corrected. (STEP016). In the turning angle correction, when the host vehicle 10 turns, for example, to the left by the turning angle θr during the period from time k to time k + 1, the range of the image is in the x direction on the images obtained by the infrared cameras 2R and 2L. This is a process for correcting this. In the following description, “the real space position of the object” means the real space position of the object subjected to the turning angle correction.

  Next, the image processing unit 1 obtains a movement vector of the object relative to the host vehicle 10 (STEP017). Specifically, a straight line that approximates time-series data in a predetermined period dT (period from the current time to a predetermined time) of the real space position of the same object is obtained, and the straight line at the time before the predetermined time is obtained. The object moves from the position of the target object (coordinate PvdT = (XvdT, YvdT, ZvdT)) to the position of the target object on the straight line (coordinate Pv0 = (Xv0, Yv0, Zv0)) at the current time. Ask as a vector. Note that the technique described in Patent Document 1 is used for specific calculation processing of the approximate straight line.

  Next, the image processing unit 1 determines the possibility that the detected object and the host vehicle 10 are in contact with each other, and whether the target object is an avoidance target (target to avoid contact with the host vehicle 10). An avoidance determination process is performed to determine whether or not (STEP 018). Details of the avoidance determination process will be described later. If it is determined in STEP 018 that the detected object is not an avoidance target (the determination result in STEP 018 is NO), the process returns to STEP 001 and the above-described processing is repeated. If it is determined in STEP018 that the detected object is an avoidance target (YES in STEP018), the process proceeds to STEP019.

  In STEP019, the image processing unit 1 performs a warning output determination process for determining whether or not the driver of the vehicle 10 should call attention to the object. In this alerting output determination process, it is confirmed from the output BR of the brake sensor 5 that the driver has operated the brake of the host vehicle 10, and the deceleration acceleration of the host vehicle 10 (acceleration in the vehicle speed decreasing direction is determined). When (positive) is larger than a predetermined threshold value (> 0), it is determined not to call attention. Further, when the driver does not perform a braking operation, or even when the braking operation is performed, if the deceleration acceleration of the host vehicle 10 is equal to or less than a predetermined threshold value, it is determined that attention should be given. The

  When the image processing unit 1 determines that alerting should be performed (the determination result of STEP019 is YES), attention is given to the driver of the host vehicle 10 by the speaker 6 and the display device 7. Arousing process is executed (STEP 020), the process returns to STEP 001, and the above process is repeated. In this alarm output process, for example, a reference image is displayed on the display device 7 and an image of an object to be avoided in the reference image is displayed in an emphasized manner. Furthermore, a voice guidance is provided from the speaker 6 to the driver that there is an object to be avoided. Note that the driver may be alerted only by either the speaker 6 or the display device 7.

  If it is determined in STEP019 that alerting is not performed (when it is determined that alerting is not performed for all objects), the determination result in STEP019 is NO. In this case, the process directly returns to STEP001, The above process is repeated.

  The above is the object detection / attention activation work in the image processing unit 1 of the object detection apparatus of the present embodiment. As a result, an object such as a pedestrian in front of the host vehicle 10 is detected from the infrared image around the host vehicle 10 and a signal indicating the traveling state of the host vehicle 10, and the driver about the target object to be avoided. Attention is given to.

  Next, the same object recognition processing in STEP010 of the flowchart shown in FIG. 3 will be described in detail with reference to the flowchart shown in FIG. In the following description, at time k, when a pedestrian is present in front of the host vehicle 10 and the host vehicle 10 is traveling straight, the host vehicle 10 continues to travel straight at time k + 1. 2, the case where the host vehicle 10 is turning rightward will be described as an example. The processed image at time k is as illustrated in FIG.

FIG. 4C illustrates an image obtained by binarizing an image obtained by the infrared camera 2R at the above-described STEP 004 at time k + 1. In FIG. 4C, the hatched area is black, and the area surrounded by the thick solid line is white. The binarized area shown in FIG. 4C is extracted as an object (binarized object) T _{k + 1} as illustrated in FIG. 4D by the processing in STEP 005 to 007 described above. . At this time, the image portion R _{k + 1} of the object T _{k + 1} has a width Wb _{k + 1} and a height Hb _{k + 1} .

FIG. 4E illustrates an image obtained by binarizing the image obtained by the infrared camera 2R at the above STEP 004 at the time k + 2. In FIG. 4E, the hatched area is black, and the area surrounded by the thick solid line is white. At this time, in the example of FIG. 4E, since the host vehicle 10 is turning rightward, an afterimage is generated on the right side of the image portion of the pedestrian. For this reason, the binarized area shown in FIG. 4 (e) includes an area F2 corresponding to an afterimage (area with stippling) as well as an area F1 corresponding to a pedestrian. Then, the binarized area shown in FIG. 4E is extracted as an object (binarized object) T _{k + 2} as illustrated in FIG. 4F by the above-described processing of STEP 005 to 007. Is done. At this time, the image portion R _{k + 2} of the object T _{k + 2} has a width Wb _{k + 2} and a height Hb _{k + 2} .

  Referring to FIG. 5, in the same object recognition process, first, image processing unit 1 determines whether or not the yaw rate YR read in STEP 009 of FIG. 3 is equal to or greater than a predetermined threshold YRth (STEP 101).

  When the determination result in STEP 101 is NO (| YR | <YRth), the magnitude of the yaw rate of the host vehicle 10 is sufficiently small so that no afterimage is generated on the image, and the following processing in STEPs 102 to 104 is performed. The process proceeds to STEP110 or STEP111. The example of the time k + 1 shown in FIG. 4C described above corresponds to this case, and will be described below using this example.

First, proceeding to STEP 102, the image processing unit 1 stores the extracted image portion R _{k + 1} of the target object T _{k + 1 in} the image memory as the target object image. The object image is updated and stored sequentially (every calculation processing cycle).

Then, calculated in STEP103, the image processing unit 1, the object T _{k + 1,} which is extracted at time k + 1, the features for the determination of the identity between the object T _k extracted at time k To do. In the present embodiment, as the feature quantity for identity determination, the center-of-gravity position Gc _{k + 1} , area S _{k + 1} , and aspect ratio ASP _{k of} the object T _{k + 1} calculated in STEP 008 at time k + 1. ₊₁ and the center-of-gravity position Gc _k , area S _k , and aspect ratio ASP _{k of} the object T _k calculated at STEP 008 at time k are used. Then, the image processing unit 1 calculates respective differences ΔGc (= Gc _{k + 1} −Gc _k ), ΔS (= S _{k + 1} −S _k ), ΔASP (= ASP _{k + 1} −ASP _k ).

Next, in STEP 104, the image processing unit 1 determines whether or not ΔGc, ΔS, and ΔASP are within predetermined ranges. The predetermined range is a range that is predetermined as a range of values that can be taken when the object T _k and the object T _{k + 1} are the same for each of ΔGc, ΔS, and ΔASP.

If the determination result in STEP 104 is YES (all ΔGc, ΔS, and ΔASP are within a predetermined range), the process proceeds to STEP 110, where it is determined that the object T _k and the object T _{k + 1} are the same. The label of the object T _{k + 1} is changed to the same label as the label of the object T _k , and the same object recognition process is terminated. If the determination result in STEP 104 is NO (at least one of ΔGc, ΔS, and ΔASP is not within the predetermined range), the process proceeds to STEP 111, where it is determined that the object T _k and the object T _{k + 1} are not the same. Then, the same object recognition process is completed.

On the other hand, when the determination result in STEP 101 is YES (| YR | ≧ YRth), the vehicle 10 turns and the yaw rate is large enough to cause an afterimage on the image. The process proceeds to STEP110 or STEP111. The example of the time k + 2 shown in FIG. 4E described above corresponds to this case, and will be described below using this example. The image portion R _{k + 2} of the target T _{k + 2} extracted at time k + 2 is as shown in FIG.

First, proceeding to STEP 105, the image processing unit 1 determines that the object image R _{k + 1} (the object image immediately before the magnitude of the yaw rate YR is equal to or larger than the predetermined threshold YRth) stored in the image memory at STEP k + 1 at time k + 1. ). Next, in STEP 106, the image processing unit 1 corrects the size (width Wb _{k + 1} and height Hb _{k + 1} ) of the read object image R _{k + 1} . Specifically, by using the distance Zv0 _{k + 1} with respect to the vehicle 10 of the object T _{k + 1} at time k + 1, and a distance Zv0 _{k + 2} with respect to the vehicle 10 of the object T _{k + 2} at time k + 2, Width Wb _{k + 1} ′ of corrected object image R _{k + 1} ′ = Wb _{k + 1} × Zv0 _{k + 2} / Zv0 _{k + 1} , height Hb _{k + 1} ′ = Hb _{k + 1} × Zv0 _{k + 2} / Zv0 _{k + 1} is corrected.

Note that the value calculated in STEP 014 at time k + 1 is used as the distance Zv0 _{k + 1} of the object T _{k + 1} at time k + 1 to the host vehicle 10. The distance Zv0 _{k + 2} with respect to the vehicle 10 of the object T _{k + 2} at time k + 2, for example, assuming that the object T _{k + 2} is identical to the object T _{k + 1,} time k + 1 is calculated by estimating the movement trajectory based on the position and speed of the object T _{k + 1 in 1} , the yaw rate YR, and the host vehicle speed VCAR. The corrected object image R _{k + 1} ′ is as shown in FIG.

Next, in STEP 107, the image processing unit 1 reverses the turning direction of the host vehicle 10 to the corrected object image R _{k + 1} ′ and the image portion R _{k + 2} of the object T _{k + 2.} The partial areas C _{k + 1} and C _{k + 2} that are closer to the edge on the side are set as shown in FIGS. 6A and 6B (the partial areas C _{k + 1} and C _{k + 2} As illustrated, this is a region including an edge closer to the opposite side of the turning direction of the host vehicle 10). The partial area C _{k + 2} has the same shape as the partial area C _{k + 1} . Thereby, the partial areas C _{k + 1} and C _{k + 2} are set in the image portion on the side where no afterimage is generated.

Next, in STEP 108, the image processing unit 1 calculates an absolute difference sum of luminance values, which is a value indicating the degree of correlation between the partial area C _{k + 1} and the partial area C _{k + 2} . In the present embodiment, as a feature amount for determination identity, and using the luminance distribution of the partial region _{C k + 1, C k +} 2 , the partial region _{C k + 1, C k +} 2 of the luminance distribution The absolute difference sum is used as the degree of similarity.

Next, in STEP 109, the image processing unit 1 determines the degree of similarity of the luminance distributions of the partial areas C _{k + 1} and C _{k + 2} based on the calculated absolute difference sum. Specifically, when the calculated absolute difference sum is equal to or less than a predetermined threshold (the degree of correlation is high), it is determined that the degree of similarity is high.

If the determination result in STEP 109 is YES (the degree of similarity is high), the process proceeds to STEP 110, where it is determined that the object T _{k + 1} and the object T _{k + 2} are the same, and the object T _{k +} The label of ₂ is changed to the same label as the label of the object T _{k + 1} , and the same object recognition process is terminated. In the example illustrated in FIG. 6, it is determined that the degree of similarity is high, and it is determined that the object T _{k + 1} and the object T _{k + 2} are the same. If the determination result in STEP 109 is NO (the degree of similarity is low), the process proceeds to STEP 111, where it is determined that the object T _{k + 1} and the object T _{k + 2} are not the same, and the same object recognition process is performed. Is terminated. Thereby, the influence of the afterimage which arises on the acquired image at the time of the turn of the own vehicle 10 can be reduced, and the identity of the target object extracted for every arithmetic processing cycle can be determined with high accuracy.

Further, at time k + 3, the object T _{k + 3} is extracted on the image (the same object as the object T _{k + 2} ), and the host vehicle 10 turns right like the time k + 2. If it is determined (| YR | ≧ YRth), the processing of STEPs 105 to 109 is performed in the same manner as at time k + 2. In this case, times k + 1, k + 2, and k + 3 correspond to times Tc, Tb, and Ta of the second invention, respectively.

At this time, in STEP 106, the image processing unit 1 determines the distance Zv0 _{k + 1} of the object T _{k + 1} at time k + 1 to the own vehicle 10 and the distance of the object T _{k + 3} at time _{k + 3 from} the own vehicle 10. The size of the object image R _{k + 1} is corrected using Zv0 _{k + 3} . Next, in STEP 107, the image processing unit 1 reverses the turning direction of the host vehicle 10 to the corrected object image R _{k + 1} ″ and the image portion R _{k + 3} of the object T _{k + 3} , respectively. The partial areas C _{k + 1} and C _{k + 3} which are parts near the edge on the side are set, and then in STEP 108, the absolute difference sum of the luminance values of the partial areas C _{k + 1} and C _{k + 3} Is calculated, and in STEP 109, the degree of similarity of the luminance distribution between the partial region C _{k + 1} and the partial region C _{k + 3} is determined.

Then, the determination result in STEP 109 is YES, and in STEP 110, it is determined that the object T _{k + 1} and the object T _{k + 3} are the same, and the label of the object T _{k + 3} is the object T _{k +.} It is changed to the same label as ₁ label. At this time, in STEP 109, it is determined that the degree of similarity of the luminance distributions of the partial areas C _{k + 1} and C _{k + 2} is high in STEP 109, and in STEP 110, the object T _{k + 1} and the object are detected. It is determined that the object T _{k + 2} is the same, and the label of the object T _{k + 2} is changed to the same label as the label of the object T _{k + 1} . Therefore, the object T _{k + 3} of the label becomes to be changed to the same label as the label of the object T _{k + 2,} the time k + 3 of the object T _{k + 3} the time k + 2 of the object T _k Recognized to be identical to ₊₂ .

  The above is the details of the same object recognition process in the present embodiment.

  In the present embodiment, the image processing unit 1 includes the object image storage unit 13 as its function, and the image processing unit 1 uses the object image storage unit 13 immediately before the magnitude of the yaw rate changes to a predetermined threshold value or more. The identity of the object is determined based on the degree of similarity between the stored object image and the image portion of the object. However, as another embodiment, the image processing unit 1 may Instead of the image, for example, the degree of similarity may be calculated by using the image portion of the object extracted in the immediately preceding calculation processing cycle to determine the identity of the object.

At this time, for example, in the case of the above-mentioned time k + 3, the image processing unit 1 is the self-of-the-art image portion of the objects T _{k + 2} and T _{k + 3 at} the times k + 2 and k + 3. The degree of similarity is calculated by directly comparing the feature amounts of the edge portions C _{k + 2} and C _{k + 3} on the opposite side to the turning direction of the vehicle 10, and the object T _{k + 2} and the object T _{k + 3} are calculated. Determine the identity of. In this case, the times k + 2 and k + 3 correspond to the times Tb and Ta of the first invention, respectively. Even in this case, the feature amount of the portion near the edge opposite to the turning direction of the host vehicle 10 is the one in which the influence of the afterimage is reduced. It is possible to reduce the influence of the generated afterimage and accurately determine the identity of the object extracted at every calculation processing cycle.

  Next, the avoidance determination processing in STEP018 of the flowchart shown in FIG. 3 will be described in detail with reference to the flowchart shown in FIG. FIG. 7 is a flowchart showing the avoidance determination process of the present embodiment. The avoidance determination process includes a first contact determination process, a second contact determination process, an approach contact determination process, a pedestrian determination process, and an artificial structure determination process, which are described below. This is a process of determining the possibility and the type of the object and determining whether or not the object is an avoidance target.

  Referring to FIG. 7, first, the image processing unit 1 performs a first contact determination process as one of the processes for determining the degree of possibility that the object contacts the own vehicle 10 (STEP 201). The first contact determination process is a process for determining whether or not contact between the object and the host vehicle 10 can be avoided with sufficient margin by steering or braking operation of the host vehicle 10. Specifically, in the first contact determination process, out of the area AR0 in front of the host vehicle 10 (area within the viewing angle of the infrared cameras 2R and 2L) AR0 captured by the infrared cameras 2R and 2L from the host vehicle 10. It is determined whether or not the current real space position of the object exists in an area AR1 (hereinafter referred to as a first contact determination area) in which the distance in the Z direction (the distance in the front-rear direction of the host vehicle 10) is a predetermined value or less. Is done.

  In this case, the predetermined value regarding the distance from the host vehicle 10 is set for each object. Specifically, a relative speed Vs (= (Zv0−ZvdT) / dT) in the Z direction between the object and the host vehicle 10 is obtained, and a predetermined time T1 (for example, about 2 to 5 seconds) is obtained as the relative speed Vs. The multiplied value Vs × T1 is set as the predetermined value that defines the boundary in the Z direction of the first contact determination area AR1. When the relative speed Vs is a relative speed away from the host vehicle 10, it is determined that the object does not exist in the first contact determination area AR1.

  Here, with reference to FIG. 8, FIG. 8 is a view of the road on which the host vehicle 10 travels as viewed from above, and the area section ahead of the host vehicle 10 is shown. As shown in FIG. 8, when the area AR0 is an outer triangular area indicated by a thick solid line, the first contact determination area AR1 is an area closer to the host vehicle 10 than Z1 (= Vs × T1) in the area AR0. Become. The first collision determination area AR1 is an area having a predetermined height H (for example, a height about twice the vehicle height of the host vehicle 10) in the vertical direction. Therefore, when the coordinate value (distance) Zv0 in the current Z direction of the object is Vs × T1 or less and the current coordinate value (height) Yv0 in the Y direction of the object is H or less, the object Is present in the first contact determination area AR1.

  If the determination result in STEP 201 is NO (the object does not exist in the first contact determination area AR1), contact between the object and the vehicle 10 can be avoided with a margin by steering or braking the vehicle 10. Is the situation. In this case, proceeding to STEP 207, the image processing unit 1 determines that the object is not an avoidance target, and ends the avoidance determination process.

  When the determination result in STEP 201 is YES (the object is present in the first contact determination area AR1), the process proceeds to STEP 202, and the image processing unit 1 may cause the object to contact the host vehicle 10. As one of the processes for determining the degree, the second contact determination process is performed. The second contact determination process is a process for determining whether or not there is a high possibility of contact between the object and the vehicle 10 when the real space position of the object is maintained at the current position. Specifically, in the second contact determination process, as shown in FIG. 8, the object is a width obtained by adding a margin β to both sides of the vehicle width α of the host vehicle 10 in the first contact determination area AR1. It is determined whether or not it exists in an area AR2 (hereinafter referred to as a second contact determination area) having (α + 2β). The second contact determination area AR2 also has a predetermined height H.

  When the determination result in STEP 202 is YES (the object is present in the second contact determination area AR2), the object is determined to be the host vehicle 10 when it is assumed that the object remains at the current real space position. There is a high possibility of contact. In this case, proceeding to STEP 203, the image processing unit 1 performs a pedestrian determination process for determining whether or not the object is a pedestrian. The details of the pedestrian determination process will be described later.

  If the determination result in STEP 203 is YES (the object may be a pedestrian), the process proceeds to STEP 204, and in order to further increase the reliability of the determination of the possibility that the object is a pedestrian, 1 performs an artificial structure determination process for determining whether or not the object is an artificial structure. Artificial structure determination processing is performed when an image of a target object is detected as a feature that is not a pedestrian, for example, matches the shape of a pre-registered artificial structure. This is a process of determining an object and excluding it from the avoidance target.

  When the determination result in STEP 204 is NO (the object is not an artificial structure), the process proceeds to STEP 206, where the image processing unit 1 determines that the object is an avoidance target and ends the avoidance determination process. Therefore, if the object is present in the second contact determination area within the first contact determination area, the object is highly likely to be a pedestrian, and is determined not to be an artificial structure, It is determined that the object is an avoidance target.

  If the determination result in STEP 203 is NO (the object is not a pedestrian) or if the determination result in STEP 204 is YES (the object is an artificial structure), the process proceeds to STEP 207. The image processing unit 1 determines that the object is not an avoidance target, and ends the avoidance determination process.

  On the other hand, if the determination result in STEP 202 is NO (the object does not exist in the second contact determination area AR2), the process proceeds to STEP 205, and the image processing unit 1 may cause the object to contact the host vehicle 10. As one of the processes for determining the degree, an approach contact determination process is performed. The approach contact determination process is a process for determining whether or not there is a high possibility that the target object enters the second contact determination area AR2 and comes into contact with the host vehicle 10. In the approach contact determination process, as shown in FIG. 8, a region in which the absolute value of the X coordinate is larger than the second contact determination region AR2 in the first contact determination region AR1 (outside in the lateral direction of the second contact determination region AR2). Whether an object in AR3, AR4 (hereinafter referred to as an entry determination area) enters the second contact determination area AR2 and contacts the host vehicle 10 is determined based on the movement vector of the object. The entry determination areas AR3 and AR4 also have a predetermined height H.

  Specifically, the X coordinate (position in the vehicle width direction) of the intersection of the XY plane (the plane perpendicular to the front-rear direction of the host vehicle 10) of the host vehicle 10 and a straight line including the movement vector of the target object is When the vehicle 10 is within a predetermined range slightly wider than the vehicle width α of the host vehicle 10 (when the target object is relatively toward the host vehicle 10), there is a possibility of entering and contacting the second contact determination area AR2. Is determined to be high.

  When the determination result in STEP 205 is YES, there is a high possibility that the target object will come into contact with the host vehicle 10 in the future. Therefore, in this case, the process proceeds to STEP 206, in which the image processing unit 1 determines that the object is an avoidance target, and ends the avoidance determination process. If the determination result in STEP 205 is NO, there is a low possibility that the target object will come into contact with the host vehicle 10, so that the process proceeds to STEP 207 and the image processing unit 1 determines that the target object is not an avoidance target and avoids it. The determination process ends.

  The above is the detail of the avoidance determination process in this embodiment.

  Next, the pedestrian determination processing in STEP 203 of the flowchart shown in FIG. 7 will be described in detail with reference to the flowchart shown in FIG. In the following description, at time k, when a pedestrian is present in front of the host vehicle 10 and the host vehicle 10 is traveling straight, the host vehicle 10 is traveling straight at time k + 1, and time k + 2, the case where the host vehicle 10 is turning rightward will be described as an example. The processed images at times k, k + 1, and k + 2 are as illustrated in FIGS. 4A, 4C, and 4E.

  Referring to FIG. 9, first, image processing unit 1 determines whether or not the yaw rate YR detected in STEP 009 of FIG. 3 is equal to or greater than a predetermined threshold YRth (STEP 301).

  When the determination result in STEP 301 is NO (| YR | <YRth), the magnitude of the yaw rate of the host vehicle 10 is sufficiently small to prevent an afterimage from appearing on the image, and the following processing in STEPs 302 to 304 is performed. The process proceeds to STEP 310 or STEP 311. The example of the time k + 1 shown in FIG. 4C described above corresponds to this case, and will be described below using this example.

First, proceeding to STEP 302, the image processing unit 1 calculates a feature amount used to determine whether or not the extracted target object T _{k + 1} is a pedestrian. Specifically, first, the image processing unit 1 determines the area S _{k + 1} , the gravity center position Gc _{k + 1} , and the target object T _{k + of} the target object T _{k + 1} (binarized target object) calculated in STEP008. aspect ratio ASP _{k + 1} of the circumscribed rectangle _1, the height Hb _{k + 1,} the width Wb _{k + 1,} the center of gravity position Gb _{k + 1,} the area _{(Hb k + 1 × Wb k} + 1), calculated in STEP014 Using the distance Zv0 _{k + 1} to the own vehicle 10, the shape feature amount of the binarized object in the real space is calculated as a feature amount for pedestrian determination. The shape feature amount of the binarized object in the real space includes, for example, the width ΔWb of the binarized object in the real space, the height ΔHb of the binarized object in the real space, and the binarized object in the real space. The upper end position Yt of the object, the ratio of the area of the binarized object and the area of the circumscribed rectangle Rate = S _{k + 1} / (Hb _{k + 1} × Wb _{k + 1} ), the center of gravity position of the binarized object and the circumscribing The distance Dis in the real space from the center of gravity of the square, the aspect ratio ASP _{k + 1 of the} circumscribed square, and the like.

Further, the image processing unit 1 binarizes the image portion of the object T _{k + 1} on the image R _{k + 1} based, as shown in FIG. 10, the object in the gray scale image T _{k + One} image portion AREA0 _{k + 1} (height Hg _{k + 1} , width Wg _{k + 1} ) is obtained. In FIG. 10, the region of the target T _{k + 1} on the binarized image is indicated by a region surrounded by a thick solid line. As a specific method for calculating the image portion of the object on the gray scale image, for example, the method described in S43 of FIG. 8 of Japanese Patent Application Laid-Open No. 2003-284057 by the present applicant is used. it can. Then, the image processing unit 1 sets the mask areas AREA1, AREA2, and AREA3 in the image portion AREA0 _{k + 1} as shown in FIG. , AREA3 luminance variance Var_A3 is calculated as a feature amount for pedestrian determination. AREA1 is an area corresponding to the head of the object, AREA2 is an area corresponding to the arm and torso of the object, and AREA3 corresponds to the whole body from the head of the object to the lower body. It is an area. Further, the image processing unit 1 performs a correlation operation between the AREA 1 and a template image indicating a pre-registered pedestrian head pattern, and the degree of similarity between the AREA 1 and the head pattern is determined as a feature for pedestrian determination. Calculated as a quantity.

  Next, in STEP 303, the image processing unit 1 determines whether or not each of the above-described feature amounts is within a predetermined range. The predetermined range is a range that is predetermined as a range of values that can be taken when the object is the upper body or the whole body of a pedestrian.

If the determination result in STEP303 is YES (all feature quantity is within a predetermined range), the process proceeds to STEP 304, the image processing unit 1 includes an image portion in the gray scale of the object T _{k + 1} AREA0 _{k + 1} Is stored in the image memory as a pedestrian image. Note that the pedestrian image is updated and stored sequentially (every calculation processing cycle). And it progresses to STEP310, it determines with a target object being a pedestrian, and a pedestrian determination process is complete | finished. If the determination result in STEP 303 is NO (at least one of the feature values is not within the predetermined range), the process proceeds to STEP 311, where it is determined that the object is not a pedestrian, and the pedestrian determination process is terminated. In addition, about the method of determining this pedestrian, it describes in detail in the above-mentioned patent document 1. FIG.

On the other hand, when the determination result in STEP 301 is YES (| YR | ≧ YRth), the vehicle 10 turns and the yaw rate is large enough to cause an afterimage on the image. Processing of 309 is performed, and the process proceeds to STEP 310 or STEP 311. The example of the time k + 2 shown in FIG. 4E described above corresponds to this case, and will be described below using this example. An image portion AREA0 _{k + 2} (height Hg _{k + 2} , width Wg _{k + 2} ) on the gray scale of the object T _{k + 2} extracted at time k + 2 is shown in FIG. It becomes like this. In FIG. 11B, the region of the target T _{k + 1} on the binarized image is indicated by a region surrounded by a thick solid line.

First, proceeding to STEP 305, the image processing unit 1 determines that the pedestrian image AREA0 _{k + 1} stored in the image memory at STEP 304 at time _{k + 1} (the pedestrian image immediately before the yaw rate YR is greater than or equal to the predetermined threshold YRth). ). Next, in STEP 306, the image processing unit 1 corrects the size of the read pedestrian image AREA0 _{k + 1} . Specifically, the distance Zv0 _{k + 1 of} the object T _{k + 1} calculated in STEP014 at time k + 1 to the host vehicle 10 and the host vehicle of the object T _{k + 2} calculated in STEP014 at time k + 2. 10 by using the distance Zv0 _{k + 2} for the 'width Wg _{k + 1} of _{the' = Wg k + 1 × Zv0} k + 2 / Zv0 k + 1, the height Hg _k corrected pedestrian image AREA0 _{k + 1} of _{+1 '= Hg k + 1 ×} Zv0 k + 2 / Zv0 k + 1 and so as to be corrected. The corrected pedestrian image AREA0 _{k + 1} ′ is as shown in FIG. In FIG. 11A, the target object T _{k + 2} region on the binarized image is indicated by a region surrounded by a thick solid line.

Next, in STEP 307, the image processing unit 1 determines that the corrected pedestrian image AREA0 _{k + 1} ′ and the image portion AREA0 _{k + 2} on the grayscale image of the object T _{k + 2} The partial regions D _{k + 1} and D _{k + 2} of the edge portion closer to the opposite side in the turning direction of 10 are set as shown in FIGS. In the present embodiment, the partial areas D _{k + 1} and D _{k + 2} are set so that the head and shoulders are included when the object is a pedestrian. The partial region D _{k + 2} has the same shape as the partial region D _{k + 1} . Thereby, the partial regions D _{k + 1} and D _{k + 2} are set in the image portion on the side where no afterimage is generated.

Next, in STEP 308, the image processing unit 1 calculates an absolute difference sum of luminance values, which is a value indicating the degree of correlation between the partial area D _{k + 1} and the partial area D _{k + 2} . In the present embodiment, as the feature quantity for pedestrian determination, and using the luminance distribution of the partial regions D _{k + 1,} D k _{+ 2,} subregion D _{k + 1,} D k _{+ 2} of the luminance distribution The absolute difference sum is used as the degree of similarity.

Next, in STEP 309, the image processing unit 1 determines the degree of similarity of the luminance distributions of the partial areas D _{k + 1} and D _{k + 2} based on the calculated absolute difference sum. Specifically, when the calculated absolute difference sum is equal to or less than a predetermined threshold (the degree of correlation is high), it is determined that the degree of similarity is high. If the determination result in STEP 309 is YES (the degree of similarity is high), the process proceeds to STEP 310, where it is determined that the object T _{k + 2} is a pedestrian (possibly a pedestrian), and the pedestrian determination process. Is terminated. In the example illustrated in FIG. 11, it is determined that the degree of similarity is high, and it is determined that the object T _{k + 2} is a pedestrian. If the determination result in STEP 309 is NO (the degree of similarity is low), the process proceeds to STEP 311, where it is determined that the object T _{k + 2} is not a pedestrian (no pedestrian possibility), and the pedestrian determination process is performed. Is terminated. Thereby, the influence of the afterimage generated on the acquired image can be reduced when the host vehicle 10 is turning, and it can be accurately determined whether or not the extracted object T _{k + 2} is a pedestrian. .

  Through the above processing, according to the present embodiment, the influence of the afterimage generated on the image is reduced when the host vehicle 10 is turning, and the image is obtained around the host vehicle 10 from the images obtained by the infrared cameras 2R and 2L. It is possible to accurately detect a pedestrian who performs. Thereby, information can be appropriately presented to the driver of the host vehicle 10.

  In the present embodiment, as described above, the image processing unit 1 includes, as its function, the object extraction means 11, the pedestrian determination means 12, the pedestrian image storage means 13, and the distance detection means 14. Contains. More specifically, STEPs 004 to 007 in FIG. 3 correspond to the object extraction unit 11, STEPs 011 to 014 in FIG. 3 correspond to the distance detection unit 14, STEP 203 in FIG. 7, STEPs 301 to 303 in FIG. 9, STEP 305. ˜311 corresponds to the pedestrian determination unit 12, and STEP 304 in FIG. 9 corresponds to the pedestrian image storage unit 13.

  In this embodiment, as described above, the image processing unit 1 includes the identity determination unit 16 and the object image storage unit 15 as its functions. More specifically, STEP 010 in FIG. 3 and STEPs 101 and 103 to 111 in FIG. 5 correspond to the identity determination unit 16, and STEP 102 in FIG. 5 corresponds to the object image storage unit 15.

  Next, a second embodiment of the present invention will be described. In the present embodiment, in the first embodiment, the acceleration sensor 17 that sequentially detects the vertical acceleration of the host vehicle 10 is connected to the image processing unit 1 to calculate the vertical speed of the host vehicle 10. The operations of the object image storage means 15 and the identity determination means 16 that are functions of the image processing unit 1 are different from those of the first embodiment. In the following description, the same components as those of the first embodiment are denoted by the same reference numerals as those of the first embodiment, and the description thereof is omitted. In addition, this embodiment is corresponded to the target object detection apparatus of the 2nd aspect of this invention.

  In the object detection device of the present embodiment, the object image storage means 15 that is a function of the image processing unit 1 is obtained by integrating the vertical acceleration of the own vehicle 10 detected by the acceleration sensor 17. The image portion of the object extracted by the object extraction means 11 from the image acquired when the magnitude of the vertical speed of 10 is less than a predetermined threshold (when pitching of the host vehicle 10 is not detected) , And sequentially stored as an object image.

  The identity determination means 16 is the first target in the image acquired at the time Ta when the magnitude of the vertical speed at the time Ta is equal to or greater than a predetermined threshold (when pitching of the host vehicle 10 is detected). Of the left and right side parts of the image part of the object, the feature amount of the part near the edge opposite to the pitching direction of the host vehicle 10 and the left and right of the image part of the second object in the image acquired at time Tb Whether the first object is the same as the second object or not is determined based on the feature amount of the part near the edge on the opposite side to the pitching direction of the host vehicle 10. . In addition, the identity determination unit 16 determines that the first speed in the image acquired at the time Ta when the magnitude of the vertical speed at the time Ta is less than a predetermined threshold (when pitching of the host vehicle 10 is not detected). Based on the overall feature amount of the image portion of the first object and the overall feature amount of the image portion of the second object in the image acquired at time Tb, the first object is the second feature amount. It is determined whether or not the object is the same. The configuration other than that described above is the same as that of the first embodiment.

  Next, the overall operation (object detection / alarm operation) of the object detection device of the present embodiment will be described. Note that the object detection / alarm operation of the object detection device in the present embodiment is the same as that in the first embodiment and the process (STEP 009 in FIG. 3) for reading the detection value related to the traveling state of the host vehicle 10. Only the object recognition process (STEP010 in FIG. 3) is different.

  In STEP 009 of the present embodiment, the image processing unit 1 moves the vertical direction detected by the acceleration sensor 17 together with the yaw rate YR detected by the yaw rate sensor 3 and the vehicle speed VCAR (horizontal speed) detected by the vehicle speed sensor 4. The acceleration GY is read (STEP 009). In STEP 009, the turning angle θr of the host vehicle 10 is also calculated by integrating the yaw rate YR over time. Further, the vertical speed VY of the host vehicle 10 is also calculated by integrating the acceleration GY over time.

  Next, in STEP 010, the image processing unit 1 performs tracking of the object for the time, that is, processing for recognizing the same object for each arithmetic processing cycle of the image processing unit 1. Here, the same object recognition process of this embodiment is demonstrated with reference to the flowchart shown in FIG. In this embodiment, STEP 010 in FIG. 3 and STEPs 401 and 403 to 411 in FIG. 12 correspond to the identity determination unit 16, and STEP 402 in FIG. 12 corresponds to the object image storage unit 15.

  In the following description, at time k, when a pedestrian is present in front of the host vehicle 10 and the host vehicle 10 is traveling straight, the host vehicle 10 continues to travel straight at time k + 1. 2, the case where the host vehicle 10 is pitching downward will be described as an example. The processed images at times k and k + 1 are as illustrated in FIGS. 4A and 4C.

FIG. 13A illustrates an image obtained by binarizing an image obtained by the infrared camera 2R at the above-described STEP 004 at time k + 2. In FIG. 13A, the hatched area is black, and the area surrounded by the thick solid line is white. At this time, in the example of FIG. 13A, since the host vehicle 10 is pitched downward, an afterimage is generated below the image portion of the pedestrian. For this reason, the binarized area shown in FIG. 13A includes an area F4 corresponding to an afterimage as well as an area F3 corresponding to a pedestrian. Then, the binarized area shown in FIG. 13A is extracted as an object (binarized object) T _{k + 2} as illustrated in FIG. 13B by the above-described processing of STEP 005 to 007. Is done. At this time, the image portion R _{k + 2} of the object T _{k + 2} has a width Wb _{k + 2} and a height Hb _{k + 2} .

  Referring to FIG. 12, in the same object recognition process, first, the image processing unit 1 determines whether or not the vertical speed VY of the host vehicle 10 calculated in STEP 009 of FIG. 3 is greater than or equal to a predetermined threshold value VYth. Judgment is made (STEP 401).

  When the determination result in STEP 401 is NO (| VY | <VYth), the magnitude of the vertical speed of the host vehicle 10 is sufficiently small so that no afterimage is generated on the image, and the following STEPs 402 to 404 are performed. The process proceeds to STEP410 or STEP411. The example of the time k + 1 shown in FIG. 4C corresponds to this case. In addition, since the process of STEP402-404,410,411 is the same as the process of STEP102-104,110,111 of 1st Embodiment, respectively, description is abbreviate | omitted.

On the other hand, when the determination result in STEP 401 is YES (| VY | ≧ VYth), the vehicle 10 is pitched and the vertical speed is large enough to cause an afterimage on the image. Processing in STEPs 405 to 409 is performed, and the process proceeds to STEP 410 or STEP 411. The example of the time k + 2 shown in FIG. 13A described above corresponds to this case, and will be described below using this example. The image portion R _{k + 2} of the target T _{k + 2} extracted at time k + 2 is as shown in FIG.

First, proceeding to STEP 405, the image processing unit 1 determines that the object image R _{k + 1} stored in the image memory at STEP _{k + 1} (the object image immediately before the vertical speed VY is equal to or greater than the predetermined threshold value VRth). ). Next, in STEP 406, the image processing unit 1 corrects the size (width Wb _{k + 1} and height Hb _{k + 1} ) of the read object image R _{k + 1} . Specifically, as in STEP 105 of the first embodiment, the distance Zv0 _{k + 1} of the object T _{k + 1} at time k + 1 to the own vehicle 10 and the own vehicle of the object T _{k + 2} at time k + 2. 10 by using the distance Zv0 _{k + 2} with respect to, 'the width Wb k _{+ 1} of _{the' = Wb k + 1 × Zv0} k + 2 / Zv0 k + 1, the height Hb _k corrected object image R _{k + 1 +1 '= Hb k + 1 ×} Zv0 k + 2 / Zv0 k + 1 and so as to be corrected. The corrected object image R _{k + 1} ′ is as shown in FIG.

Next, in STEP 407, the image processing unit 1 converts the corrected object image R _{k + 1} ′ and the image portion R _{k + 2} of the object T _{k + 2} in the reverse direction of the pitching direction of the host vehicle 10, respectively. The partial regions C _{k + 1} and C _{k + 2} of the side edge portions are set as shown in FIGS. 13 (a) and 13 (b). The partial area C _{k + 2} has the same shape as the partial area C _{k + 1} . Thereby, the partial areas C _{k + 1} and C _{k + 2} are set in the image portion on the side where no afterimage is generated.

Next, in STEP 408, the image processing unit 1 calculates the absolute difference sum of luminance values, which is a value indicating the degree of correlation between the partial region C _{k + 1} and the partial region C _{k + 2} . In the present embodiment, as a feature amount for determination identity, and using the luminance distribution of the partial region _{C k + 1, C k +} 2 , the partial region _{C k + 1, C k +} 2 of the luminance distribution The absolute difference sum is used as the degree of similarity.

Next, in STEP 409, the image processing unit 1 determines the degree of similarity of the luminance distributions of the partial areas C _{k + 1} and C _{k + 2} based on the calculated absolute difference sum. Specifically, when the calculated absolute difference sum is equal to or less than a predetermined threshold (the degree of correlation is high), it is determined that the degree of similarity is high.

If the determination result in STEP 409 is YES (the degree of similarity is high), the process proceeds to STEP 410, where it is determined that the object T _{k + 1} and the object T _{k + 2} are the same, and the object T _{k +} The label of ₂ is changed to the same label as the label of the object T _{k + 1} , and the same object recognition process is terminated. In the example illustrated in FIG. 13, it is determined that the degree of similarity is high, and it is determined that the object T _{k + 1} and the object T _{k + 2} are the same. When the determination result in STEP 409 is NO (the degree of similarity is low), the process proceeds to STEP 411, where it is determined that the object T _{k + 1} and the object T _{k + 2} are not the same, and the same object recognition process is performed. Is terminated. Thereby, at the time of pitching of the own vehicle 10, the influence of the afterimage generated on the acquired image can be reduced, and the identity of the object extracted for each arithmetic processing cycle can be accurately recognized.

Further, at time k + 3, the object T _{k + 3} (the same object as the object T _{k + 2} ) is extracted on the image, and the host vehicle 10 pitches downward similarly to the time k + 2. If it is (VY ≧ VYth), the processing of STEPs 405 to 409 is performed in the same manner as at time k + 2. In this case, times k + 1, k + 2, and k + 3 correspond to times Tc, Tb, and Ta of the fourth invention, respectively.

At this time, in STEP 406, the image processing unit 1 determines the distance Zv0 _{k + 1} of the object T _{k + 1} at time k + 1 to the own vehicle 10 and the distance of the object T _{k + 3} at time _{k + 3 from} the own vehicle 10. The size of the object image R _{k + 1} is corrected using Zv0 _{k + 3} . Next, in STEP 407, the image processing unit 1 makes the corrected object image R _{k + 1} ″ and the image portion R _{k + 3} of the object T _{k + 3} opposite to the pitching direction of the host vehicle 10, respectively. The partial areas C _{k + 1} and C _{k + 3} that are close to the edge on the side are set.Next, in STEP 408, the absolute difference sum of the luminance values of the partial area C _{k + 1} and the partial area C _{k + 3} is set. Is calculated, and in STEP 409, the degree of similarity of the luminance distribution between the partial region C _{k + 1} and the partial region C _{k + 3} is determined.

Then, the determination result in STEP 409 is YES, and in STEP 410, it is determined that the object T _{k + 1} and the object T _{k + 3} are the same, and the label of the object T _{k + 3} is the object T _{k +.} It is changed to the same label as ₁ label. At this time, it is determined in STEP 409 that the degree of similarity of the luminance distributions of the partial regions C _{k + 1} and C _{k + 2} is high in the calculation processing cycle at time k + 2, and in STEP 410, the target T _{k + 1} and target It is determined that the object T _{k + 2} is the same, and the label of the object T _{k + 2} is changed to the same label as the label of the object T _{k + 1} . Therefore, the object T _{k + 3} of the label becomes to be changed to the same label as the label of the object T _{k + 2,} the time k + 3 of the object T _{k + 3} the time k + 2 of the object T _k Recognized to be identical to ₊₂ .

  The above is the details of the same object recognition process in the present embodiment.

  In the pedestrian determination process according to the present embodiment, the influence of an afterimage in the vertical direction generated on the image due to pitching of the host vehicle 10 on a feature amount (for example, a luminance distribution of the head) that is a pedestrian is Assuming that there are few, like STEP302-303 of 1st Embodiment, what is necessary is just to determine whether this target object is a pedestrian based on the whole feature-value of the image part of a target object.

  In the present embodiment, the image processing unit 1 includes the object image storage unit 13 as its function, and the image processing unit 1 is stored by the object image storage unit 13 immediately before the pitching is detected. The identity of the object is determined based on the degree of similarity between the object image and the image portion of the object. However, as another embodiment, the image processing unit 1 may replace the object image. In addition, for example, the degree of similarity may be calculated using the image portion of the object extracted in the immediately preceding calculation processing cycle to determine the identity of the object.

At this time, for example, in the case of the above-mentioned time k + 3, the image processing unit 1 is the self-of-the-art image portion of the objects T _{k + 2} and T _{k + 3 at} the times k + 2 and k + 3. The feature amounts of the portions C _{k + 2} and C _{k + 3} on the opposite side to the pitching direction of the vehicle 10 are directly compared to calculate the degree of similarity, and the object T _{k + 2} and the object T _{k + 3} are Determine the identity of. In this case, the times k + 2 and k + 3 correspond to the times Tb and Ta of the third invention, respectively. Even in this case, since the feature amount of the portion near the edge opposite to the pitching direction of the host vehicle 10 is reduced in the influence of the afterimage, the feature amount is displayed on the acquired image when the host vehicle 10 is pitched. It is possible to reduce the influence of the generated afterimage and accurately determine the identity of the object extracted at every calculation processing cycle.

  In the present embodiment, the image processing unit 1 uses the vertical speed (translation speed) VY as the detection value of the state quantity that changes due to pitching, and the speed VY is greater than or equal to a predetermined threshold value VYth. It is determined that pitching has been detected, and it is determined that no pitching has been detected when the velocity VY is less than the predetermined threshold value VYth. On the other hand, as another embodiment, the image processing unit 1 uses the angular velocity PR around the pitching axis of the host vehicle 10 instead of the vertical velocity VY, according to the magnitude of the angular velocity PR around the pitching axis. Then, the pitching of the host vehicle 10 may be detected and the process of determining the identity of the target object may be executed.

  At this time, as the pitching detection means, vertical and longitudinal acceleration sensors attached to the front part of the host vehicle 10 (in the vicinity of the infrared cameras 2R and 2L) are used, and the detected vertical and longitudinal accelerations are sequentially detected. The angular velocity PR around the pitching axis may be calculated by time integration. Alternatively, a rate sensor that sequentially detects the angular velocity PR around the pitching axis of the host vehicle 10 may be used as the pitching detection means. Alternatively, the pitching angle (angle around the pitching axis) of the host vehicle 10 may be sequentially detected, and the amount of change in the pitching angle per unit time (angular velocity PR around the pitching axis) may be calculated.

In the first and second embodiments, the image processing unit 1 corrects the size of the target image R _{k + 1} based on the distances Zv0 _{k + 2} and Zv0 _{k + 1} , and corrects the corrected target image. Although the degree of similarity is calculated using R _{k + 1} ′ (STEP 106 to 108, STEP 406 to 408), the image processing unit 1 determines the time k + 1 and k + 2 when the calculation processing cycle is sufficiently short. Thus, assuming that there is little change in the size of the object on the image, the degree of similarity may be calculated using the object image R _{k + 1} as it is without correction.

  In the first and second embodiments, when the image processing unit 1 calculates the feature quantity for identity determination in the same object recognition process, the image portion of the object on the binarized image is calculated. Although used, you may use the image part of the target object on a gray scale image like the pedestrian determination process of 1st Embodiment.

In the first and second embodiments, the image processing unit 1 calculates the brightness of the partial areas C _{k + 1} and C _{k + 2} when calculating the feature quantity for identity determination in the same object recognition process. Although the distribution is used (STEP 108, STEP 408), as another embodiment, for example, when calculating the feature value for identity determination, the contour of the object in the partial regions C _{k + 1} and C _{k + 2} is calculated. A line shape may be used. Further, as a feature amount for determination identity, subregion _{C k + 1, C k +} 2 of the luminance distribution, the partial area C _{k + 1,} a plurality of features of shape of the contour of the object in C k _{+ 2} You may use combining quantity.

  In the first and second embodiments, the image processing unit 1 alerts the driver of the host vehicle 10 about the object determined to be the avoidance object in STEP 018. As an embodiment, for example, when the vehicle 10 can operate any one of the vehicle steering device, the brake device, and the accelerator device by an actuator (and consequently, the traveling behavior of the vehicle 10 can be operated), it is avoided in STEP018. The steering device, the brake device, and the accelerator device of the host vehicle 10 may be controlled so as to avoid contact with an object that is determined to be a target, or to facilitate the avoidance. For example, the accelerator device is controlled so that the required pedaling force of the accelerator pedal by the driver is greater than when there is no target object to be avoided (normal case), thereby making acceleration difficult. Alternatively, the required rotational force of the steering handle toward the steering direction of the steering device required to avoid contact between the avoidance target and the vehicle 10 is made lower than the required rotational force of the steering handle toward the opposite side, The steering handle can be easily operated in the steering direction. Or the increase speed of the braking force of the vehicle 10 according to the depression amount of the brake pedal of a brake device is made higher than usual. By doing in this way, the driving | operation of the vehicle 10 for avoiding a contact with an avoidance object becomes easy. Further, the control of the steering device as described above and the alerting by the display device 7 or the speaker 6 may be performed in parallel.

  In the first and second embodiments, the distance detection unit 14 calculates the distance of the object to the host vehicle 10 based on the images acquired via the infrared cameras 2R and 2L. As a form, the distance detection unit 14 may calculate the distance of the object with respect to the host vehicle 10 based on the position information of the object detected by another sensor such as a millimeter wave radar.

  In the first and second embodiments, an infrared camera is used as the imaging means. However, for example, a CCD camera that can detect only normal visible light may be used. However, by using an infrared camera, it is possible to simplify the extraction process of pedestrians, running vehicles, and the like, and it can be realized even when the calculation capability of the calculation device is relatively low.

The functional block diagram of the target object detection apparatus by 1st Embodiment of this invention. Explanatory drawing of the attachment aspect to the vehicle of the target object detection apparatus shown in FIG. The flowchart which shows the target object detection and attention starting operation | work in the image processing unit of the target object detection apparatus of FIG. FIG. 4 is a view showing an example of a processed image in the object detection / attention activation work of FIG. 3. The flowchart of the same target object recognition process in the target object detection / alerting | wakening start work of FIG. Explanatory drawing regarding the same target object recognition process of FIG. The flowchart of the avoidance determination process in the target object detection / alerting | wakening start work of FIG. Explanatory drawing which shows the area | region division ahead of the vehicle in the avoidance determination process of FIG. The flowchart of the pedestrian determination process in the avoidance determination process of FIG. Explanatory drawing regarding the pedestrian determination process of FIG. Explanatory drawing regarding the pedestrian determination process of FIG. The flowchart of the same target object recognition process in the target object detection apparatus of 1st Embodiment of this invention. FIG. 13 is an exemplary view of a processed image in the object detection / attention activation work of FIG. 12. Explanatory drawing regarding the same target object recognition process of FIG.

Explanation of symbols

  2R, 2L ... Infrared camera (imaging means), 3 ... Yaw rate sensor (yaw rate detection means), 10 ... Vehicle, 11 ... Object extraction means, 14 ... Distance detection means, 15 ... Object image storage means, 16 ... Identity Determination means, 17... Acceleration sensor (pitching detection means).

Claims (10)

Object extraction means for sequentially extracting objects existing around the vehicle from an image acquired via an imaging means mounted on the vehicle, and object extraction means from an image acquired at an arbitrary time Ta The same determination as to whether or not the extracted first object is the same as the second object extracted by the object extraction means from the image acquired at time Tb prior to the time Ta In the object detection device provided with sex determination means,
Yaw rate detection means for sequentially detecting the yaw rate of the vehicle,
The identity determination unit is configured to detect the left and right side portions of the image portion of the first object in the image acquired at the time Ta when at least the magnitude of the yaw rate at the time Ta is a predetermined threshold value or more. Of the characteristic amount of the portion near the edge opposite to the turning direction of the vehicle and the left and right side portions of the image portion of the second object in the image acquired at the time Tb, the vehicle The object detection is characterized in that it is determined whether or not the first object is the same as the second object based on the feature amount of the portion near the edge on the opposite side to the turning direction of apparatus. A target for sequentially storing, as a target image, an image portion of a target extracted by the target extraction unit from an image acquired when the magnitude of the yaw rate detected by the yaw rate detection unit is less than the predetermined threshold. Comprising object image storage means,
The identity determination unit is configured so that the time Ta and the time Tb are at least immediately after the magnitude of the yaw rate changes from less than the predetermined value to the predetermined value or more, and the magnitude of the yaw rate exceeds the predetermined value. The edge on the opposite side to the turning direction of the vehicle in the left and right side portions of the image portion of the first object in the image acquired at the time Ta when the time is within a predetermined period to be maintained The left and right side portions of the object image stored by the object image storage means from the image acquired at the time Tc immediately before the feature amount of the near portion and the magnitude of the yaw rate change to the predetermined threshold value or more. The degree of similarity between the feature amount of the portion near the edge opposite to the turning direction of the vehicle, and the left and right side portions of the image portion of the second object in the image acquired at the time Tb My car Of the portion closer to the edge opposite to the turning direction of the vehicle and the feature amount of the portion closer to the edge opposite to the turning direction of the vehicle of the left and right side portions of the object image at the time Tc. 2. The object detection device according to claim 1, wherein it is determined whether or not the first object is the same as the second object based on a second similarity degree. Object extraction means for sequentially extracting objects existing around the vehicle from an image acquired via an imaging means mounted on the vehicle, and object extraction means from an image acquired at an arbitrary time Ta The same determination as to whether or not the extracted first object is the same as the second object extracted by the object extraction means from the image acquired at time Tb prior to the time Ta In the object detection device provided with sex determination means,
Pitching detection means for sequentially detecting the pitching of the vehicle,
The identity determination means includes at least upper and lower side portions of the image portion of the first object in the image acquired at the time Ta when pitching is detected by the pitching detection means at least at the time Ta. Of the characteristic amount of the portion near the edge opposite to the pitching direction of the vehicle and both the upper and lower side portions of the image portion of the second object in the image acquired at the time Tb. An object detection device that determines whether or not the first object is the same as the second object based on a feature amount of a portion near the edge opposite to the pitching direction. . An object image storage means for sequentially storing, as an object image, an image portion of the object extracted by the object extraction means from an image acquired when no pitching is detected by the pitching detection means;
The identity determination means is a state immediately after the time Ta and the time Tb are changed from a state in which no pitching is detected by the pitching detection means to a state in which pitching is detected, and in which the pitching is detected. Of the left and right side portions of the image portion of the first object in the image acquired at the time Ta, on the opposite side to the pitching direction of the vehicle. Of the left and right side portions of the object image stored by the object image storage means from the feature amount of the portion near the edge and the image acquired at the time Tc immediately before the pitching is detected, The first similarity degree with the feature quantity of the portion near the edge opposite to the pitching direction of the vehicle, and the image of the second object in the image acquired at the time Tb Of the left and right side portions of the minute, and the feature amount of the edge side portion opposite to the pitching direction of the vehicle, and the pitching direction of the vehicle of the left and right side portions of the object image at the time Tc Determining whether or not the first object is the same as the second object based on the second degree of similarity with the feature amount of the portion near the opposite edge. The object detection device according to claim 3. A distance detecting means for detecting a distance of the object extracted by the object extracting means with respect to the vehicle;
The identity determination means includes a distance detected by the distance detection means of the first object in the image acquired at the time Ta, and an object corresponding to the object image at the time Tc. Based on the distance detected by the distance detection means, a means for correcting the size of the object image is calculated, and the first similarity degree is calculated using the corrected object image. The distance detected by the distance detecting means of the second object in the image acquired at the time Tb and the object corresponding to the object image at the time Tc are detected by the distance detecting means. And a means for correcting the size of the object image based on the calculated distance, and calculating the second similarity degree using the corrected object image. 2 or claim 4 The system for detecting the object.   A vehicle on which the object detection device according to any one of claims 1 to 5 is mounted. By an object extraction step for sequentially extracting objects existing around the vehicle from an image acquired via an imaging means mounted on the vehicle, and an object extraction step from an image acquired at an arbitrary time Ta The same determination as to whether or not the extracted first object is the same as the second object extracted by the object extraction step from the image acquired at time Tb before the time Ta An object detection method comprising a sex determination step,
A yaw rate detection step for sequentially detecting the yaw rate of the vehicle;
In the identity determination step, at least when the magnitude of the yaw rate at the time Ta is equal to or greater than a predetermined threshold, the left and right side portions of the image portion of the first object in the image acquired at the time Ta Of the characteristic amount of the portion near the edge opposite to the turning direction of the vehicle and the left and right side portions of the image portion of the second object in the image acquired at the time Tb, the vehicle The object detection is characterized in that it is determined whether or not the first object is the same as the second object based on the feature amount of the portion near the edge on the opposite side to the turning direction of Method. By an object extraction step for sequentially extracting objects existing around the vehicle from an image acquired via an imaging means mounted on the vehicle, and an object extraction step from an image acquired at an arbitrary time Ta The same determination as to whether or not the extracted first object is the same as the second object extracted by the object extraction step from the image acquired at time Tb before the time Ta An object detection method comprising a sex determination step,
A pitching detection step for sequentially detecting the pitching of the vehicle;
In the identity determination step, at least when the pitching is detected in the pitching detection step at the time Ta, out of both upper and lower side portions of the image portion of the first object in the image acquired at the time Ta Of the characteristic amount of the portion near the edge opposite to the pitching direction of the vehicle and both the upper and lower side portions of the image portion of the second object in the image acquired at the time Tb. An object detection method for determining whether or not the first object is the same as the second object based on a feature amount of a portion near the edge opposite to the pitching direction. . By an object extraction process for sequentially extracting objects existing around the vehicle from an image acquired via an imaging means mounted on the vehicle, and an object extraction process from an image acquired at an arbitrary time Ta The same determination as to whether or not the extracted first object is the same as the second object extracted by the object extraction process from the image acquired at time Tb prior to the time Ta An object detection program for causing a computer to execute sex determination processing,
A function of causing a computer to execute a yaw rate acquisition process for sequentially acquiring a detected value of the yaw rate of the vehicle,
As the identity determination processing, when at least the magnitude of the yaw rate at the time Ta is equal to or greater than a predetermined threshold, the left and right side portions of the image portion of the first object in the image acquired at the time Ta Of the characteristic amount of the portion near the edge opposite to the turning direction of the vehicle and the left and right side portions of the image portion of the second object in the image acquired at the time Tb, the vehicle A function of causing a computer to execute a process of determining whether or not the first object is the same as the second object based on a feature amount of a portion near the edge on the opposite side to the turning direction of A program for detecting an object, characterized by comprising a program. By an object extraction process for sequentially extracting objects existing around the vehicle from an image acquired via an imaging means mounted on the vehicle, and an object extraction process from an image acquired at an arbitrary time Ta The same determination as to whether or not the extracted first object is the same as the second object extracted by the object extraction means from the image acquired at time Tb prior to the time Ta An object detection program for causing a computer to execute sex determination processing,
A function of causing a computer to execute a pitching acquisition process for sequentially acquiring detection values of state quantities that change due to the pitching of the vehicle,
As the identity determination processing, at least when the pitching is detected by the detection value acquired by the pitching acquisition processing at the time Ta, the image portion of the first object in the image acquired at the time Ta Of the upper and lower side parts, the feature amount of the part near the edge opposite to the pitching direction of the vehicle, and the upper and lower side parts of the image part of the second object in the image acquired at the time Tb A process for determining whether or not the first object is the same as the second object based on the feature amount of the portion near the edge opposite to the pitching direction of the vehicle. A program for detecting an object, which is a program having a function to be executed.

Images