JP4377439B1 - Vehicle periphery monitoring device - Google Patents

Abstract

To provide a vehicle periphery monitoring device capable of reporting at a suitable timing or frequency that the possibility of contact between a vehicle and an object is high in view of the type of the object.
According to the vehicle periphery monitoring device of the present invention, when it is determined that the object is a human and the actual space position of the object is included in the first contact determination area A1, It is reported that the possibility of contact between the vehicle 10 and the object is high. On the other hand, when it is determined that the object is a quadruped and the real space position of the object is included in the second contact determination area A2, the notification is made. Here, the second contact determination area A2 has an overlapping area with the first contact determination area A1 and an area at least partially protruding from the first contact determination area A1.
[Selection] Figure 4

Description

  The present invention relates to a vehicle periphery monitoring device that monitors the periphery of a vehicle using a captured image obtained by an imaging device mounted on the vehicle.

Based on the time-series position data of objects such as animals existing around the vehicle, a movement vector in the real space of the object is calculated, and based on this movement vector, the possibility of contact between the vehicle and the object is calculated. There has been proposed a vehicle periphery monitoring device that determines the height and determines that the possibility of contact is high, and notifies the driver to that effect (see Patent Document 1).
JP 2001-006096 A

  However, when the target object is a quadruped animal such as a deer, it is generally difficult to predict its behavior, so if the possibility of contact is determined based on the movement vector, Report timing may be delayed. On the other hand, in spite of an object whose behavior can be predicted relatively easily like a human being, the notification frequency to the driver may become unnecessarily high simply by advancing the notification timing.

  Accordingly, the present invention provides a vehicle periphery monitoring device capable of reporting at a suitable timing or frequency that the possibility of contact between a vehicle and an object is high in view of the type of the object. With the goal.

A vehicle peripheral device according to a first aspect of the present invention is a vehicle periphery monitoring device that monitors the periphery of a vehicle using a captured image obtained by an imaging device mounted on the vehicle, and extracts an object from the captured image. Means, position measuring means for measuring the real space position of the object extracted by the object extracting means, and the object extracted by the object extracting means corresponds to either a human or a quadruped animal An object determining means for determining whether or not the object is a human by the object determining means for determining whether the vehicle and the human being as the object are likely to contact each other While the first contact determination area is set, when the target determination unit determines that the target is a quadruped, the possibility of contact between the vehicle and the quadruped as the target is high or low The A region for the contact determination area setting a second contact determination area having at least a portion of which protruded region from the overlap region and the first contact determination area of the first contact determination area When the real space position of the person as the object measured by the position measuring unit is included in the first contact determination region set by the setting unit and the contact determination region setting unit , or the contact determination When the real space position of the quadruped as the object is included in the real space position measured by the position measurement means in the second contact determination area set by the area setting means, It is provided with the object notification means which reports presence of a target object.

  According to the vehicle periphery monitoring device of the first invention, when it is determined that the object is a human and the real space position of the object is included in the first contact determination area, the vehicle and the object It is reported that there is a high possibility of contact with. On the other hand, when it is determined that the object is a quadruped and the real space position of the object is included in the second contact determination area, the notification is made. Here, the second contact determination area has an overlapping area with the first contact determination area and an area at least partially protruding from the first contact determination area. For this reason, when the target object is a quadruped animal, even if the real space position of the target object is not included in the first contact determination area, the second contact determination area extends beyond the first contact determination area. Since the notification is made if it is included in the area, the notification timing can be advanced compared to the case where the object is a human being. On the other hand, when the target object is a human, the notification is unconditionally made even if the real space position of the target object is included in the second contact determination area that is outside the first contact determination area. Therefore, it is possible to prevent the frequency of notifying the driver that there is a high possibility that the vehicle and the person are in contact with each other. Therefore, it is possible to report that there is a high possibility that the vehicle and the object are in contact with each other at an appropriate timing or frequency in view of the type of the object.

The vehicle periphery monitoring device according to a second aspect of the present invention is the vehicle periphery monitoring device according to the first aspect of the invention, wherein the object measured at different time points by the position measuring means calculates a movement vector of the object from a plurality of real space positions. Movement vector calculation means, and when the object determination means determines that the object is a human, the contact determination area setting means has a third contact determination area outside the first contact determination area. The object notifying means includes the real space position of the person as the object measured by the position measuring means in the third contact determination area set by the contact determination area setting means. and, if the motion vector of the human being as the object calculated by the motion vector calculation means is toward the first contact determination area, the relative the driver Wherein the notifying the existence of a human as elephants thereof.

  According to the vehicle periphery monitoring device of the second invention, it is determined that the object is a human, the real space position of the object is included in the third contact determination area, and the movement vector of the object is the first When the vehicle is heading toward the contact determination area 1, it is reported that the possibility of contact between the vehicle and the object is high. For this reason, since the human real space position is included in the third contact determination area and the human movement vector is not directed to the first contact determination area, the notification is not issued. It is prevented that the frequency of notifying the driver that there is a high possibility of contact with the vehicle is unnecessarily high. On the other hand, if it is determined that the object is a quadruped and the real space position of the object is included in the third contact determination area as a part of the second contact determination area, the movement vector thereof Since the notification is made regardless of the direction of the notification, the notification timing can be advanced compared to the case where the object is a human being. Therefore, it is possible to report that there is a high possibility that the vehicle and the object are in contact with each other at an appropriate timing or frequency in view of the type of the object.

  A vehicle periphery monitoring device according to a third aspect of the invention is the vehicle periphery monitoring device according to the first or second aspect of the invention, wherein the contact determination area setting means extends parallel to the traveling direction of the vehicle in front of the vehicle. An area having a width obtained by adding a predetermined margin on both the left and right sides is set as the first contact determination area.

  According to the vehicle periphery monitoring device of the third aspect of the invention, when it is determined that the object is a human, the first contact determination region extends in parallel to the traveling direction of the vehicle in front of the vehicle and It is set to an area having a width with a predetermined margin on both sides. The area is an area where the vehicle and the object are highly likely to come into contact, and the area outside the area is an area where the possibility that the vehicle and the object are in contact is not necessarily high. When the real space position of the object is included in the first contact determination area, the driver is notified at a suitable frequency that the vehicle and the object are likely to contact each other. Moreover, since the notification is not made outside the first contact determination region even if the real space position of the object is included in the region, the frequency of reporting to the driver is unnecessarily high. Is prevented. On the other hand, since the second contact determination region is included outside the first contact determination region, when the target is a quadruped, the notification timing is advanced compared to the case where the target is a human. be able to. Therefore, it is possible to report that there is a high possibility that the vehicle and the object are in contact with each other at an appropriate timing or frequency in view of the type of the object.

  The vehicle periphery monitoring device according to a fourth aspect of the present invention is the vehicle periphery monitoring device according to any one of the first to third aspects, wherein the contact determination area setting means determines the imaging area captured by the imaging apparatus as the second contact determination. It is set to an area.

  According to the vehicle periphery monitoring device of the fourth aspect of the invention, when it is determined that the object is a quadruped, the second contact determination area is imaged by the imaging device that is the maximum range that the vehicle can recognize. In view of being set in the imaging region, when the real space position of the object is included in the region, the notification timing that the possibility of contact between the vehicle and the object is high can be advanced. On the other hand, when it is determined that the object is a human, the first contact determination area is set to an area limited to the imaging area, and the real space position of the object protrudes from the area. In view of the fact that the notification is not made unconditionally even if it is included in the vehicle, the frequency of reporting to the driver is prevented from becoming unnecessarily high. Therefore, it is possible to report that there is a high possibility that the vehicle and the object are in contact with each other at an appropriate timing or frequency in view of the type of the object.

  An embodiment of the present invention will be described below with reference to FIGS. First, the structure of the vehicle periphery monitoring apparatus of this embodiment is demonstrated. With reference to FIGS. 1 and 2, the vehicle periphery monitoring device of the present embodiment includes an image processing unit 1. The image processing unit 1 is connected with two infrared cameras 2R and 2L as imaging devices for capturing an image in front of the host vehicle 10, and as a sensor for detecting the traveling state of the host vehicle 10, the image processing unit 1 A yaw rate sensor 3 that detects the yaw rate, a vehicle speed sensor 4 that detects the traveling speed (vehicle speed) of the host vehicle 10, and a brake sensor 5 that detects the presence or absence of a brake operation of the host vehicle 10 are connected. Further, the image processing unit 1 includes a speaker 6 for outputting auditory notification information such as voice, and a display device for displaying captured images and visual notification information captured by the infrared cameras 2R and 2L. 7 is connected. The infrared cameras 2R and 2L correspond to the imaging device in the present invention.

  Although not shown in detail, the image processing unit 1 includes electronic circuits including an A / D conversion circuit, a microcomputer (having a CPU, a RAM, a ROM, and the like), an image memory, and the like. The infrared cameras 2R, 2L, Analog signals output from the yaw rate sensor 3, the vehicle speed sensor 4, and the brake sensor 5 are converted into digital data by the A / D conversion circuit and input to the microcomputer. The microcomputer detects an object such as a person (pedestrian or person riding a bicycle) based on the input data. When the detected object satisfies a predetermined notification requirement, the microcomputer 6 And the process which issues a report to a driver by the display device 7 is executed. The image processing unit 1 has functions as an object extraction means, an object determination means, and a contact area setting means in the present invention.

  As shown in FIG. 2, the infrared cameras 2 </ b> R and 2 </ b> L are attached to the front portion (the front grill portion in the figure) of the host vehicle 10 in order to image the front of the host vehicle 10. In this case, the infrared cameras 2R and 2L are respectively disposed at a position on the right side and a position on the left side of the center of the host vehicle 10 in the vehicle width direction. These positions are symmetrical with respect to the center of the vehicle 10 in the vehicle width direction. The infrared cameras 2R and 2L are fixed so that their optical axes extend in parallel in the front-rear direction of the vehicle 10, and the heights of the respective optical axes from the road surface are equal to each other. The infrared cameras 2R and 2L have sensitivity in the far-infrared region, and have a characteristic that the higher the temperature of an object to be imaged, the higher the level of the output video signal (the higher the luminance of the video signal). is doing. As shown in FIG. 2, the real space coordinate system is defined with the center of the front end of the vehicle 10 as the origin O, the right direction of the vehicle 10 as + X direction, the vertically downward direction as + Y direction, and the front of the vehicle 10 as + Z direction. ing.

  In the present embodiment, the display device 7 includes a head-up display 7a (hereinafter referred to as HUD 7a) that displays image information on the front window of the host vehicle 10. As the display device 7, instead of the HUD 7a or together with the HUD 7a, a display provided integrally with a meter for displaying a traveling state such as the vehicle speed of the host vehicle 10 or a display provided in an in-vehicle navigation device is provided. It may be used.

  Next, basic functions of the vehicle periphery monitoring apparatus having the above-described configuration will be described with reference to the flowchart of FIG. The basic processing content of the flowchart of FIG. 3 is the same as the processing content described in FIG. 3 of Japanese Patent Laid-Open No. 2001-6096 and FIG.

  Specifically, first, the image processing unit 1 receives an infrared image from output signals of the infrared cameras 2R and 2L (FIG. 3 / STEP 11). Next, the image processing unit 1 A / D converts the output signals of the infrared cameras 2R and 2L (FIG. 3 / STEP 12). Then, the image processing unit 1 acquires a gray scale image from the A / D converted infrared image (FIG. 3 / STEP 13). Thereafter, the image processing unit 1 binarizes the reference image (right image) (FIG. 3 / STEP 14). Next, the image processing unit 1 performs the processing of STEPs 15 to 17 on the binarized image, and extracts an object (more precisely, an image region corresponding to the object) from the binarized image. Specifically, the pixel group constituting the high luminance area of the binarized image is converted into run length data (FIG. 3 / STEP 15). Next, a label (identifier) is attached to each line group that overlaps in the vertical direction of the reference image (FIG. 3 / STEP 16). Then, each line group is extracted as an object (FIG. 3 / STEP 17). Thereafter, the image processing unit 1 calculates the position of the center of gravity of each object extracted as described above (position on the reference image), the area, and the aspect ratio of the circumscribed rectangle (FIG. 3 / STEP 18). Next, the image processing unit 1 tracks the object extracted in STEP 18 during time, that is, recognizes the same object for each calculation processing period of the image processing unit 1 (FIG. 3 / STEP 19). Then, the image processing unit 1 reads the outputs (vehicle speed detection value and yaw rate detection value) of the vehicle speed sensor 4 and the yaw rate sensor 5 (FIG. 3 / STEP 20).

  On the other hand, the image processing unit 1 executes the processing of STEP 31 in parallel with the processing of STEP 19 and STEP 20. First, in the reference image, an area corresponding to each object (for example, a circumscribed quadrangular area of the object) is extracted as a search image (FIG. 3 / STEP 31). Next, a search area for searching for an image (corresponding image) corresponding to the search image from the left image is set, and a correlation operation is executed to extract the corresponding image (FIG. 3 / STEP 32). And the distance from the own vehicle 10 of the target object (distance in the front-back direction of the own vehicle 10) is calculated (FIG. 3 / STEP33).

  Next, the image processing unit 1 calculates a real space position that is a position (relative position with respect to the host vehicle 10) of each object in the real space (FIG. 3 / STEP 21). Then, the image processing unit 1 corrects the position X in the X direction of the real space position (X, Y, Z) of the object according to the time-series data of the turning angle obtained in STEP 20 (FIG. 3 / (Step 22). Thereafter, the image processing unit 1 estimates a relative movement vector of the object with respect to the host vehicle 10 (FIG. 3 / STEP 23). Next, when the relative movement vector is obtained in STEP 23, the possibility of contact with the detected object is determined, and a notification determination process for issuing a notification when the possibility is high is executed (FIG. 3 / STEP 24). The above is the overall operation of the periphery monitoring device of this embodiment. In addition, the structure which performs the process of STEP11-18 by the image processing unit 1 is equivalent to the target object extraction means of this invention.

  Next, notification processing, which is a main function of the vehicle periphery monitoring device of the present invention, will be described with reference to the flowchart of FIG.

  First, an object determination process for determining whether the object extracted by the object extraction means is a human or a quadruped animal is executed (FIG. 4 / STEP 100).

  Specifically, the type of the object is determined from the characteristics such as the shape and size of the image area of the object on the gray scale image and the luminance dispersion. For example, the head (specifically corresponding to the first high-intensity image area) and the shoulder, torso, arm or leg (specifically, below the first high-intensity image area) In addition, an object having a second high-intensity image region having the same arrangement relationship as a standard arrangement relationship such as a shoulder portion with respect to the head with respect to the first high-intensity image region is a human being. It is determined. Further, the body (specifically, the first high-luminance image region) and the head or leg (specifically, located on the side or the lower side of the first high-luminance image region), and the first It is determined that the object having the second high-intensity image region smaller than the high-intensity image region is a quadruped animal such as a deer, a sheep, a dog, or a horse. Further, the type of the object may be determined by pattern matching between the outer shape of the target object and the outer shape of the target object stored in advance in the memory.

  When it is determined that the object is a human (FIG. 4 / STEP 100... A), the first contact determination area A1 and the third contact determination area A3 are set (FIG. 4 / STEP 102). Specifically, as shown in FIG. 5 (a), the vehicle of the host vehicle 10 extends parallel to the Z direction inside the triangular area A0 that can be monitored by the infrared cameras 2R and 2L, and extends in the X direction. A region having a width obtained by adding a first margin β (for example, 50 to 100 cm) to both the left and right sides of the width α is set as the first contact determination region A1 (see the hatched portion). The depth Z1 in the Z direction as viewed from the vehicle 1 in the first contact determination area A1 is determined to be a distance obtained by adding the margin time T to the relative speed Vs between the host vehicle 10 and the object. Further, as shown in FIG. 5A, in the triangular area A0, the areas adjacent to the left and right sides of the first contact determination area are the third contact determination areas A3L. Set as A3R.

  Further, it is determined whether or not the real space position of the object is included in the first contact determination area A1 (FIG. 4 / STEP 104). Note that the real space position of the object is measured in STEP 21 of FIG. The configuration in which the processing of STEP 21 is performed by the image processing unit 1 corresponds to the position measuring unit of the present invention. When it is determined that the real space position of the object exists in the first contact determination area A1 (FIG. 4 / STEP 104... YES), a notification output determination process is executed (FIG. 4 / STEP 106). On the other hand, when it is determined that the object does not exist in the first contact determination area A1 (FIG. 4 / STEP 104... NO), the object exists in the third contact determination areas A3R and A3L, and in STEP23. It is determined whether or not the obtained movement vector of the object is toward the first contact determination area A1 (FIG. 4 / STEP 105).

  If the determination result is affirmative (FIG. 4 / STEP 105... YES), it is determined from the output of the brake sensor 5 whether the driver of the host vehicle 10 is performing an appropriate brake operation (FIG. 4 / STEP 106). ). When it is determined that the driver of the vehicle 10 does not perform an appropriate brake operation (FIG. 4 / STEP 106... NO), the driver is informed that the possibility of contact between the vehicle and the object is high. (FIG. 4 / STEP 108). Specifically, the situation is reported by outputting sound through the speaker 6. The situation is also reported by highlighting the object on the HUD 7a. Note that the determination process in STEP 106 may be omitted.

  On the other hand, when it is determined that the object is a quadruped animal (FIG. 4 / STEP 100... B), the second contact determination area A2 is set (FIG. 4 / STEP 202). Specifically, as shown in FIG. 5B, inside the triangular area A0 that can be monitored by the infrared cameras 2R and 2L, it extends parallel to the Z direction, and the vehicle 10 is in the X direction. A region having a width obtained by adding a second margin γ larger than the first margin β on both the left and right sides of the vehicle width α is set as the second contact determination region A2 (see the hatched portion). The depth Z1 in the Z direction as viewed from the vehicle 1 in the second contact determination area A2 is obtained by adding the margin time T to the relative speed Vs between the host vehicle 10 and the object, as in the first contact determination area A1. It is determined at a given distance. The second contact determination region A2 includes a region that overlaps the entire first contact determination region A1, and a deviation (γ−β) between the second margin γ and the first margin β on both the left and right sides of the first contact determination region A1. Only).

  Next, it is determined whether or not the real space position of the object is included in the second contact determination area A2 (FIG. 4 / STEP 204). Note that the real space position of the object is measured in STEP 21 of FIG. When it is determined that the real space position of the object exists in the second contact determination area A2 (FIG. 4 / STEP 204... YES), the driver of the host vehicle 10 performs an appropriate brake operation from the output of the brake sensor 5. It is determined whether or not it exists (FIG. 4 / STEP 206). When it is determined that the driver of the host vehicle 10 is not performing an appropriate brake operation (FIG. 4 / STEP 206... NO), the driver is notified that the possibility of contact between the vehicle 10 and the object is high. (FIG. 4 / STEP 208).

  Note that the configuration in which the processing of STEP 100 is performed by the image processing unit 1 is the object determination unit of the present invention, the configuration of executing the processing of STEP 102 and STEP 202 is the contact determination region setting unit of the present invention, and STEP 108 and STEP 208 The configuration for executing the processing corresponds to the object notification means of the present invention.

  According to the vehicle periphery monitoring device that exhibits the function, when the object is determined to be a human and the real space position of the object is included in the first contact determination area A1, the host vehicle 10 It is reported that there is a high possibility of contact with the object (see FIG. 4 / S100... A, S102 to 108, FIG. 5A). On the other hand, when it is determined that the object is a quadruped and the real space position of the object is included in the second contact determination area A2, the notification is made (FIG. 4 / S100... B). , S202 to 208, see FIG. 5B). Here, the second contact determination region A2 has an overlapping region with the first contact determination region A1 and a region at least partially protruding from the first contact determination region A1 (FIG. 5 ( a) (b)). For this reason, when the target object is a quadruped animal, even if the real space position of the target object is not included in the first contact determination area A1, the first contact determination area in the second contact determination area A2 Since the notification is made if it is included in the area that protrudes from A1, the notification timing can be advanced compared to the case where the object is a human. On the other hand, when the object is a human being, the notification is unconditionally made even if the real space position of the object is included in the area of the second contact determination area A2 that extends beyond the first contact determination area A1. Therefore, it is possible to prevent the frequency of notifying the driver that there is a high possibility that the host vehicle 10 and the person are in contact with each other. Therefore, it is possible to report at a suitable timing or frequency in view of the type of the object that there is a high possibility that the host vehicle 10 and the object are in contact with each other.

  In the embodiment, the second contact determination area A2 shown in FIG. 5B is set so as to overlap the entire first contact determination area A1 shown in FIG. 5A. However, as shown in FIG. 6A, the second contact determination area A2 (shaded portion) may be set to overlap a part of the first contact determination area A1. Moreover, in the said embodiment, in 2nd contact determination area | region A2, although the overlap area | region with 1st contact determination area | region A1 and the protrusion area | region from 1st contact determination area | region A1 were adjacent, FIG. As shown in b), the overlapping area and the protruding area may be separated from each other. Further, the protruding area from the first contact determination area A1 in the second contact determination area A2 may be provided only on either the left or right side of the overlapping area with the first contact determination area. Alternatively, the shape or arrangement may be asymmetrical. In the embodiment, the second contact determination area A2 is set to an area corresponding to a range obtained by adding the second margin γ (γ> β) to both sides of the vehicle width α of the host vehicle 10. A region obtained by combining the contact determination region A1 and the third contact determination regions A3R and A3L may be used as the second contact determination region A2.

  Further, the second contact determination area AR4 may be an imaging area that is imaged by the infrared cameras 2R and 2L. Moreover, in this embodiment, although it is comprised so that a predetermined | prescribed report may be performed based on the process result of the image processing unit 1, you may comprise so that vehicle behavior may be controlled based on this process result. . Furthermore, although the two infrared cameras 2R and 2L are provided in the embodiment, one infrared camera may be mounted on the host vehicle 10. In this case, the distance to the object is detected by a radar or the like.

The figure which shows the whole structure of one Embodiment of the periphery monitoring apparatus of the vehicle of this invention. The perspective view of the vehicle provided with the periphery monitoring apparatus of FIG. The flowchart which shows the process of the image processing unit with which the periphery monitoring apparatus of FIG. 1 was equipped. The flowchart which shows the report determination processing in this embodiment The figure which shows the contact determination area imaged with the imaging device in this embodiment. The figure which shows the modification of the contact determination area imaged with the imaging device in this embodiment

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image processing unit (object extraction means, object determination means, contact area setting means), 2R, 2L ... Infrared camera (imaging device), 6 ... Speaker (object notification means), 7a ... HUD (object notification) means)

Claims (4)

A vehicle periphery monitoring device that monitors the periphery of a vehicle using a captured image obtained by an imaging device mounted on the vehicle,
Object extraction means for extracting an object from the captured image;
Position measuring means for measuring the real space position of the object extracted by the object extracting means;
An object determination means for determining whether the object extracted by the object extraction means corresponds to a human or a quadruped animal;
When the object determining means determines that the object is a human, a first contact determination area for determining the level of possibility of contact between the vehicle and a human being as the object is set. , if the object is determined to be a quadruped animal by the object determination unit, an area for determining the level of possibility of contact between tetrapods as the object and the vehicle, Contact determination area setting means for setting a second contact determination area having an overlapping area with the first contact determination area and an area at least partially protruding from the first contact determination area;
When the first contact determination area set by the contact determination area setting means includes a human real space position as the object measured by the position measurement means, or by the contact determination area setting means When the real space position of the quadruped animal as the object is included in the real space position measured by the position measuring means in the set second contact determination area, the presence of the object to the driver A vehicle surroundings monitoring device comprising object reporting means for reporting The vehicle periphery monitoring device according to claim 1,
The object measured at different time points by the position measuring means comprises a movement vector calculating means for calculating a movement vector of the object from a plurality of real space positions,
The contact determination area setting means sets a third contact determination area outside the first contact determination area when the object determination means determines that the object is a human,
The object notifying means includes a human real space position as the object measured by the position measuring means in the third contact determination area set by the contact determination area setting means, and When the movement vector of the person as the object calculated by the movement vector calculation means is toward the first contact determination area, the presence of the person as the object is notified to the driver. A vehicle periphery monitoring device. In the vehicle periphery monitoring device according to claim 1 or 2,
The contact determination area setting means includes, in front of the vehicle, an area extending in parallel to the traveling direction of the vehicle and having a width with a predetermined margin on both the left and right sides of the vehicle. A vehicle periphery monitoring device characterized by being set as follows. In the vehicle periphery monitoring device according to any one of claims 1 to 3,
The vehicle periphery monitoring device, wherein the contact determination region setting means sets an imaging region imaged by the imaging device as the second contact determination region.

Images