JP2007293487A - Periphery monitoring method for vehicle, vehicle, periphery monitoring method for vehicle and periphery monitoring program for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide the periphery monitoring device of a vehicle for establishing a status in which the contact of an object such as a pedestrian to a vehicle is avoidable as quickly as possible under such circumstances that the object intends to cross in front of the vehicle. <P>SOLUTION: The periphery monitoring method of a vehicle comprises: a speed component judging means (STEP 16) for successively judging whether a relative speed to a vehicle 10 of an object detected in front of a vehicle 10 has the speed components of a vehicle width direction, that is, lateral speed components whose scale is a prescribed positive lower limit value or less; and avoidance object judging means (STEP 16 to 20) for selecting a judgement algorithm (STEP 19 or 20) for judging whether the object is an avoidance object whose contact to the vehicle 10 should be avoided according to at least the judgement result of the speed component judging means, and for executing the selected judgment algorithm to judge whether the object is the avoidance object (STEP 19 or 20). <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an apparatus and a method for monitoring an object existing around a vehicle using an imaging device or the like. Furthermore, the present invention relates to a vehicle on which the device is mounted, and a program for causing a computer to execute processing of the device.

  As this type of device, for example, the one shown in Patent Document 1 has been proposed by the present applicant.

In this technique, the front of a vehicle is imaged by two infrared cameras, and an object such as a pedestrian or a vehicle is detected from the captured image. Then, based on the parallax of the object in the captured images of the two infrared cameras, the relative position (real space position) of the object with respect to the vehicle is sequentially detected, and a certain time interval (detection of the relative position) of the relative position. A movement vector representing a relative movement direction of the object with respect to the vehicle is obtained based on a time series in a time interval longer than the period. Then, after the calculation of the movement vector, a determination process including determination regarding the movement direction of the object represented by the movement vector, the object is to be an alarm object (avoids contact with the vehicle) that should be an alarm object for the driver. It is determined whether it is a target to be). Furthermore, when it is determined that the object is an alarm target, it is determined whether or not an actual alarm is to be issued based on the state of the brake operation of the vehicle. The vehicle driver is alerted by a speaker or an image. Accordingly, the driver of the vehicle is alerted to the object to be alarmed, and the driver can execute a measure for avoiding contact between the object and the vehicle.
JP 2001-6096 A

  In the technique found in Patent Document 1, the movement vector is calculated based on the time series of the relative position of the object, so that the movement vector represents the relative movement direction of the object with respect to the vehicle. Is highly reliable.

  On the other hand, when an object such as a pedestrian or an animal is about to cross the front of the vehicle, the presence of the object is quickly recognized by the driver of the vehicle to avoid contact between the object and the vehicle. It is desirable to establish a possible state. In particular, when a pedestrian is trying to cross the vehicle so as to jump out in front of the vehicle, it is desirable to establish as soon as possible a state in which contact between the object and the vehicle can be avoided.

  However, in the technique found in Patent Document 1, it takes time to calculate the movement vector. And in the thing of the said patent document 1, in the process which determines whether the said target object is a warning target, in order to perform the determination regarding the moving direction of the target object which this movement vector represents, until the determination result is obtained Takes time. For this reason, in particular, when a pedestrian suddenly jumps out, there is a demand for a technique that can more quickly determine whether an object is a target that should avoid contact with a vehicle. It was.

  The present invention has been made in view of such a background, and in a situation where an object such as a pedestrian is about to cross the front of the vehicle, a state in which contact between the object and the vehicle can be avoided as quickly as possible. It is an object of the present invention to provide a vehicle periphery monitoring device, a periphery monitoring method, and a vehicle that can be established. Furthermore, the present invention relates to a vehicle periphery monitoring program that causes a computer to execute the processing of the periphery monitoring device.

  In order to achieve the above object, the vehicle periphery monitoring device of the present invention has at least object detection means for detecting an object existing on the front side of the vehicle, and the detected object makes contact with the vehicle. In a vehicle periphery monitoring device including an avoidance target determination unit that determines whether or not the target is an avoidance target to be avoided, a relative speed of the target detected by the target detection unit with respect to the vehicle is determined by the vehicle. A speed component determination unit that sequentially determines whether or not the vehicle has a speed component in the vehicle width direction, the magnitude of which is equal to or greater than a predetermined positive lower limit value, and the avoidance target determination unit includes: A means capable of selectively executing a plurality of types of determination algorithms for determining whether or not the object is an avoidance target, and at least the plurality of types of determinations according to a determination result of the speed component determination unit; Select one determination algorithm of the algorithm, the object is characterized in that it is determined whether or not the avoidance target by executing the decision algorithm (first invention).

  According to the present invention (first invention), when the speed component determination means determines that the relative speed of the object to the vehicle has the lateral speed component, the object tries to cross the front of the vehicle. It is highly possible that On the other hand, when it is determined that the relative speed does not have a lateral speed component, the object is stopped or moving in the same direction as the vehicle. It is unlikely that you are trying to cross the front of the vehicle. Therefore, the determination result of the speed component determination unit represents the degree of possibility of crossing the object detected by the object detection unit in front of the vehicle. For this reason, while the determination process of the speed component determination unit is sequentially performed, the target is about to cross the front of the vehicle by selecting the determination algorithm executed by the avoidance target determination unit according to at least the determination result. The determination process of the avoidance target determination unit can be executed by a determination algorithm suitable for each situation and each other. For example, in the former situation (a situation where the determination result of the speed component determination unit is positive), the determination result is determined in a shorter time than in the latter situation (a situation where the determination result of the speed component determination unit is negative). One possible determination algorithm may be selected from the plurality of determination algorithms and executed. Conversely, in the latter situation, for example, one judgment algorithm that can determine a judgment result with higher reliability than the former situation may be selected from the plurality of judgment algorithms and executed.

  Therefore, according to the present invention, in a situation where an object such as a pedestrian is about to cross the front of the vehicle, it is possible to determine whether or not the object is an avoidance target as quickly as possible. Become. Eventually, it is possible to appropriately execute measures such as alerting (arousing the driver's attention to the avoidance target) according to the determination result. Further, in situations other than the situation, it is possible to increase the reliability of the determination as to whether or not the object is an avoidance target.

  In the present invention, it is preferable that the lateral velocity component is a velocity component whose magnitude is not more than a predetermined upper limit value that is larger than the lower limit value (second invention).

  According to this, in addition to the magnitude of the lateral speed component being equal to or greater than the lower limit value, the determination result of the speed component determination means is affirmative when the magnitude is equal to or less than the upper limit value (lateral speed To be determined to have an ingredient). Here, prompting the driver's attention quickly is particularly necessary when a specific type of object such as a person or an animal is about to cross the front of the vehicle. Also, objects that need to be alerted, such as people and animals, generally have a higher speed when trying to cross a road than artificial moving objects such as other vehicles. The upper limit is relatively low. Therefore, by making the size of the lateral velocity component equal to or smaller than the upper limit value as a necessary requirement for the determination result of the velocity component determination means to be affirmative, the determination result can be affirmative. There is an increased possibility that the types of objects are limited to specific types such as people and animals. As a result, it is possible to quickly determine whether or not the specific type of object is an avoidance target.

  The upper limit value is set to a value near the maximum value of the speed magnitude that can be generally taken by a specific type of object that can be determined as an avoidance target in the determination process of the avoidance target determination means, for example. Just keep it.

  Further, in the first invention or the second invention, the apparatus further comprises relative position detecting means for sequentially detecting a relative position of the object detected by the object detecting means with respect to the vehicle, and the direction of the lateral velocity component is determined by the direction. It may be a velocity component having a direction that satisfies a predetermined requirement determined according to the relative position detected by the relative position detecting means (third invention).

  That is, even if the magnitude of the lateral speed component of the object is equal to or greater than the lower limit value, the object may be an avoidance target depending on the combination of the direction of the lateral speed component and the relative position of the object with respect to the vehicle. There may be a low need for a quick determination of whether or not there is. For example, an object existing outside (on the right or left side) of the area corresponding to the vehicle width in front of the vehicle has a direction in which the lateral velocity component approaches the vehicle (within the vehicle width in front of the vehicle If the direction of the lateral velocity component is away from the vehicle, the determination of the avoidance target is quick. The need for is low. Therefore, the speed component determination unit can perform the determination process of the avoidance target quickly by setting the horizontal direction speed component to a speed component whose direction satisfies a predetermined requirement determined according to the relative position of the object. It is possible to carry out by adapting to the degree of necessity of determination.

  As the predetermined requirement, for example, the relative position of the object exists in a region outside a region corresponding to the vehicle width in front of the vehicle (region having a width equal to or slightly wider than the vehicle width). When the direction of the lateral speed component is the direction approaching the vehicle, and when the relative position is in a region corresponding to the vehicle width in front of the vehicle, the direction of the lateral speed component is either left or right There are requirements such as that.

  In the first invention, the relative position detecting means for sequentially detecting the relative position of the object detected by the object detecting means with respect to the vehicle, and the relative position exists in a predetermined area in front of the vehicle. Relative position determination means for determining whether the plurality of determination algorithms are based on a set of a determination result of the speed component determination means and a determination result of the relative position determination means. One of the determination algorithms may be selected and executed (fourth invention).

  That is, when the relative position of the object is sufficiently far away from the vehicle in the front-rear direction or the vehicle width direction, even if the determination result of the speed component determination means is positive, the object In general, the necessity for quickly determining the avoidance target is low. Therefore, by selecting and executing one determination algorithm of the plurality of determination algorithms according to the combination of the determination result of the speed component determination unit and the determination result of the relative position determination unit, the avoidance target determination unit This determination process can be performed more appropriately.

  Supplementally, the predetermined area according to the fourth aspect of the present invention is an area in which the object can be detected by the object detecting means, and the distance in the front-rear direction of the vehicle is a predetermined value or less (hereinafter referred to as the A area). Or a region between a pair of boundary lines set so as to extend in the front-rear direction on both sides of the vehicle among the detectable regions (hereinafter also referred to as a B-th region). Alternatively, a composite region of the A region and the B region may be used. In this case, if the determination result of the speed component determination unit is affirmative and the determination result of the relative position determination unit is affirmative, the determination result of the speed component determination unit is negative or the relative position When the determination result of the determination unit is negative, it is preferable to select and execute one determination algorithm that can determine the determination result in a shorter time from the plurality of determination algorithms.

  The fourth invention may be combined with the second invention or the third invention.

  In the first aspect of the invention, the relative position detecting means for sequentially detecting the relative position of the object to the vehicle, and the relative position detecting means at a time interval longer than the execution cycle of the determination process of the speed component determining means. A moving direction feature amount calculating means for calculating a moving direction feature amount representing a moving direction of the target object with respect to the vehicle based on a detected time series of the relative position of the target object; and the determination result of the speed component determining means is The determination algorithm selected by the avoidance target determination unit in the case of negative is configured by an algorithm including at least a determination process related to the moving direction feature amount, and the determination result of the speed component determination unit is positive when the determination result is positive The determination algorithm selected by the avoidance target determination means is an algorithm that does not include at least a determination process related to the moving direction feature amount. It is preferably composed of a beam (fifth aspect).

  According to this, when the determination result of the speed component determination unit is negative, the determination algorithm executed by the avoidance target determination unit includes a determination process related to the moving direction feature amount, and thus a highly reliable determination The result can be determined. However, since the calculation of the moving direction feature amount is performed at a time interval longer than the execution cycle of the determination process of the speed component determination unit, it takes time to determine the determination result by the avoidance target determination unit. On the other hand, when the determination result of the speed component determination unit is affirmative, the determination algorithm executed by the avoidance target determination unit does not include the determination process related to the moving direction feature amount. The result can be determined in a short time. As a result, it is possible to quickly perform a procedure such as alerting an object having a lateral velocity component.

  Similarly, in the fourth aspect of the present invention, based on the time series of the relative position of the object detected by the relative position detection means in a time interval longer than the execution cycle of the determination process of the speed component determination means, A moving direction feature amount calculating unit that calculates a moving direction feature amount representing a moving direction with respect to the vehicle, and the avoidance target when the determination result of the speed component determination unit or the determination result of the relative position determination unit is negative The determination algorithm selected by the determination unit is configured by an algorithm including at least a determination process related to the moving direction feature quantity, and the determination result of the speed component determination unit and the determination result of the relative position determination unit are affirmative. The determination algorithm selected by the avoidance target determination means is an algorithm that does not include at least determination processing related to the moving direction feature amount. It is preferably made of the rhythm (sixth invention).

  According to this, when the determination result of the speed component determination unit or the determination result of the relative position determination unit is negative, the determination algorithm executed by the avoidance target determination unit is a determination process related to the moving direction feature amount. Therefore, it is possible to determine a highly reliable determination result. On the other hand, when the determination result of the speed component determination unit and the determination result of the relative position determination unit are affirmative, the determination algorithm executed by the avoidance target determination unit does not include the determination process regarding the moving direction feature amount. The determination result by the avoidance target determination means can be determined in a short time. As a result, it is possible to quickly perform a procedure such as alerting an object having a lateral velocity component.

  Supplementally, in the fifth or sixth invention, the determination process related to the moving direction feature value is a direction in which the moving direction of the object represented by the moving direction feature value is directed toward the front of the vehicle. And a determination process such as whether or not. The fifth invention may be combined with the first to third inventions.

  Each of the determination algorithms executed by the avoidance target determination unit preferably includes a determination process for the type of the object. Further, each determination algorithm may include a determination process related to the relative position of the object or the lateral velocity component.

  Further, the vehicle of the present invention is equipped with the above-described periphery monitoring device of the present invention (seventh invention). According to this vehicle, a vehicle having the same effects as the periphery monitoring device of the present invention can be realized.

  Further, the vehicle periphery monitoring method of the present invention detects at least an object existing on the front side of the vehicle, and determines whether or not the detected object is an avoidance object that should avoid contact with the vehicle. In the vehicle periphery monitoring method, a lateral direction in which a relative speed of the detected object to the vehicle is a speed component in a vehicle width direction of the vehicle, the magnitude of which is equal to or greater than a predetermined positive lower limit value. A speed component determination step for sequentially determining whether or not it has a speed component, and a determination algorithm for determining whether or not the object is an avoidance target, from among a plurality of types of determination algorithms prepared in advance The selection is made at least according to the determination result of the speed component determination step, and the determination is made as to whether or not the object is an avoidance object by executing the selected determination algorithm. Characterized by comprising a elephant determination step (Eighth invention).

  According to the eighth aspect of the invention, the determination result of the speed component determination step represents the degree of possibility that the detected object crosses in front of the vehicle, as in the vehicle periphery monitoring device of the present invention. Therefore, while performing the determination process of the speed component determination step sequentially, the target tries to cross the front of the vehicle by selecting the determination algorithm to be executed in the avoidance target determination step according to at least the determination result. It is possible to execute the determination process of the avoidance target determination step with a determination algorithm suitable for each of the existing situation and the other situation.

  Therefore, according to the present invention, in a situation where an object such as a pedestrian is about to cross the front of the vehicle, it is possible to determine whether or not the object is an avoidance target as quickly as possible. Become. As a result, it is possible to execute appropriate measures such as alerting according to the determination result. Further, in situations other than the situation, it is possible to increase the reliability of the determination as to whether or not the object is an avoidance target.

  The vehicle periphery monitoring program according to the present invention is a process for determining whether or not an object detected to be present at least on the front side of the vehicle is an avoidance object that should avoid contact with the vehicle. A vehicle periphery monitoring program that causes a computer to execute the following: a relative speed of the detected object with respect to the vehicle is a speed component in a vehicle width direction of the vehicle, the magnitude of which is a predetermined positive value A speed component determination process for sequentially determining whether or not a lateral speed component is equal to or greater than a lower limit value, and a determination algorithm for determining whether or not the object is an avoidance target are determined in advance. It is selected from a plurality of determination algorithms according to at least the determination result of the speed component determination process, and whether or not the object is the avoidance target by the selected determination algorithm And determining avoidance judging process, characterized in that it comprises a function to be executed by the computer (ninth invention).

  According to the program of the ninth aspect of the invention, it is possible to cause a computer to execute a process that can achieve the effects described with respect to the first aspect of the invention.

  A first embodiment of the present invention will be described below with reference to FIGS.

  First, with reference to FIG. 1 and FIG. 2, the system configuration | structure of the vehicle periphery monitoring apparatus of this embodiment is demonstrated. FIG. 1 is a block diagram showing the overall configuration of the periphery monitoring device, and FIG. 2 is a perspective view showing the appearance of a vehicle equipped with the periphery monitoring device. In FIG. 2, illustration of some components of the periphery monitoring device is omitted.

  With reference to FIGS. 1 and 2, the periphery monitoring device of the present embodiment includes an image processing unit 1. The image processing unit 1 is connected to two infrared cameras 2R and 2L as imaging devices for capturing an image in front of the vehicle 1 and also uses the yaw rate of the vehicle 10 as a sensor for detecting the traveling state of the vehicle 10. A yaw rate sensor 3 for detecting, a vehicle speed sensor 4 for detecting a traveling speed (vehicle speed) of the vehicle 10, and a brake sensor 5 for detecting a brake operation (specifically, whether or not a brake pedal is operated) of the vehicle 10 are connected. Has been. Further, the image processing unit 1 displays a speaker 6 for outputting auditory alert information such as voice, and images captured by the infrared cameras 2R and 2L and visual alert information. A display device 7 is connected.

  Although not shown in detail, the image processing unit 1 is composed of an electronic circuit including an A / D conversion circuit, a microcomputer (CPU, RAM, ROM), an image memory, and the like. The infrared cameras 2R and 2L, the yaw rate sensor 3. Outputs (analog signals) of the vehicle speed sensor 4 and the brake sensor 5 are digitized and inputted via an A / D conversion circuit. Then, the image processing unit 1 performs processing for detecting an object such as a person (pedestrian) based on the input data, and whether the detected object is an avoidance target that should avoid contact with the vehicle. The microcomputer performs a process for determining whether or not, a process for alerting the driver to an object determined to be an avoidance target, and the like. These processes are realized by the microcomputer executing a program pre-installed in the ROM of the microcomputer, and the program includes the vehicle periphery monitoring program of the present invention.

  Note that the image processing unit 1 includes an object detection unit, a speed component determination unit, an avoidance target determination unit, and a relative position detection unit according to the present invention as functions realized by the program.

  As shown in FIG. 2, the infrared cameras 2 </ b> R and 2 </ b> L are attached to the front portion of the vehicle 10 (the front grill portion in the figure) in order to image the front of the vehicle 10. In this case, the infrared cameras 2R and 2L are respectively arranged at a position on the right side and a position on the left side of the center of the vehicle 10 in the vehicle width direction. These positions are symmetrical with respect to the center of the host vehicle 10 in the vehicle width direction. The infrared cameras 2R and 2L are arranged in front of the vehicle 10 such that their optical axes extend in parallel in the front-rear direction of the vehicle 10 and the heights of the respective optical axes from the road surface are equal to each other. It is fixed to the part. Each infrared camera 2R, 2L is an imaging device having sensitivity in the far-infrared region, and the higher the temperature of the object imaged by the infrared camera 2R, 2L, the higher the level of the output signal of the object image (the object's image). (The brightness of the image is increased).

  In the present embodiment, the display device 7 includes a head-up display 7a (hereinafter referred to as HUD 7a) that displays information such as an image on the front window of the vehicle 10, for example. The display device 7 may be a display provided integrally with a meter that displays a traveling state such as the vehicle speed of the host vehicle 10 in place of the HUD 7a or together with the HUD 7a, or a display provided in an in-vehicle navigation device. May be included.

  Next, the overall operation of the periphery monitoring apparatus of this embodiment will be described with reference to the flowcharts of FIGS. Note that many of the processes in the flowchart of FIG. 3 are the same as the processes described in Patent Document 1, for example, and thus detailed description of the same processes is omitted in this specification. . Supplementally, the process of the flowchart of FIG. 3 is a process realized by a program executed by the microcomputer of the image processing unit 1.

  First, the image processing unit 1 acquires infrared images that are output signals of the infrared cameras 2R and 2L (STEP 1), performs A / D conversion (STEP 2), and stores each image in an image memory (STEP 3). ). As a result, images captured by the infrared cameras 2R and 2L are taken into the image processing unit 1. Hereinafter, an image obtained from the infrared camera 2R is referred to as a right image, and an image obtained from the infrared camera 2L is referred to as a left image. These right image and left image are both grayscale images.

  Next, the image processing unit 1 uses one of the right image and the left image as a reference image, and binarizes this reference image (STEP 4). The reference image is a right image in the present embodiment. In this binarization process, the luminance value of each pixel of the reference image is compared with a predetermined luminance threshold value, and an area (relatively bright area) having a luminance value higher than the predetermined luminance threshold value in the reference image. This is a process of setting “1” (white) and setting an area having a luminance value lower than the luminance threshold (a relatively dark area) to “0” (black). Hereinafter, an image (monochrome image) obtained by the binarization process is referred to as a binarized image. In the binarized image, an area “1” is referred to as a high luminance area. The binarized image is stored in the image memory separately from the grayscale image (right image and left image).

  Supplementally, the processing of STEP1 to STEP4 is the same as the processing of S11 to S14 of FIG.

  Next, the image processing unit 1 performs the processing of STEPs 5 to 7 on the binarized image, and extracts an object (more precisely, an image portion corresponding to the object) from the binarized image. That is, the pixel group constituting the high luminance area of the binarized image is classified into lines having a width of one pixel in the vertical direction (y direction) of the reference image and extending in the horizontal direction (x direction). Each line is converted into run-length data composed of coordinates and length (number of pixels) of the position (two-dimensional position on the reference image) (STEP 5). Of the lines represented by the run length data, labels (identifiers) are attached to the respective line groups that overlap in the vertical direction of the reference image (STEP 6), and each of the line groups is extracted as an object. (STEP 7).

  The objects extracted by the processing in STEPs 5 to 7 generally include not only people (pedestrians) but also artificial structures such as other vehicles. In addition, a plurality of local parts of the same object may be extracted as the target object.

  Next, the image processing unit 1 obtains the position of the center of gravity of each object extracted as described above (position on the reference image), the area, and the aspect ratio of the circumscribed rectangle (STEP 8). The position of the center of gravity of each object is obtained by multiplying the coordinates of the position of each line (the center position of each line) of the run length data included in the object by the length of the line. It is obtained by adding all the lines of included run length data and dividing the addition result by the area of the object. Further, instead of the center of gravity of each object, the position of the center (center) of the circumscribed rectangle of the object may be obtained.

  Next, the image processing unit 1 traces the object extracted in STEP 7 for the time, that is, recognizes the same object for each calculation processing period of the image processing unit 1 (STEP 9). In this processing, when the object A is extracted by the processing of STEP 7 at a time (discrete system time) k in a certain arithmetic processing cycle, and the object B is extracted by the processing of STEP 7 at the time k + 1 of the next arithmetic processing cycle. The identity of these objects A and B is determined. This identity determination may be performed based on, for example, the shape and size of the objects A and B on the binarized image, the correlation of the luminance distribution on the reference image (grayscale image), and the like. . When it is determined that the objects A and B are the same, the label of the object B extracted at time k + 1 (the label attached in STEP 6) is changed to the same label as the label of the object A. The

  Note that the processing in STEPs 5 to 9 described above is the same as the processing in S15 to S19 in FIG. Moreover, the object detection means in this invention is comprised by the process of STEP1-9.

  Next, the image processing unit 1 reads the outputs (vehicle speed detection value and yaw rate detection value) of the vehicle speed sensor 4 and the yaw rate sensor 5 (STEP 10). In STEP 10, the turning angle (azimuth angle) of the host vehicle 10 is also calculated by integrating the read detection value of the yaw rate.

  On the other hand, the image processing unit 1 executes the processes of STEPs 11 to 13 in parallel with the processes of STEPs 9 and 10. The processing of STEP 11 to 13 is processing for obtaining the distance of each object extracted in STEP 7 from the host vehicle 10, and is the same as the processing of S31 to S33 in FIG. The process is schematically described. First, an area corresponding to each object (for example, a circumscribed square area of the object) in the right image (reference image) is extracted as a search image R1 (STEP 11).

  Next, in the left image, a search area R2 that is an area for searching for the same object as the object included in the search image R1 of the right image is set, and the correlation with the search image R1 is set in the search area R2. The region having the highest probability is extracted as a corresponding image R3 that is an image corresponding to the search image R1 (an image equivalent to the search image R1) (STEP 12). In this case, an area having a luminance distribution that most closely matches the luminance distribution of the search image R1 of the right image is extracted as the corresponding image R3 from the search area R2 of the left image. Note that the processing in STEP 12 is performed using a grayscale image, not a binarized image.

  Next, the number of pixels of the difference between the lateral position (x-direction position) of the center of gravity of the search image R1 in the right image and the lateral position (x-direction position) of the center of gravity of the corresponding image R3 in the left image is defined as parallax Δd. The distance z of the object from the host vehicle 10 (the distance in the front-rear direction of the host vehicle 10) is calculated using the parallax Δd (STEP 13). The distance z is calculated by the following equation (1).

z = (f × D) / (Δd × p) (1)

Note that f is the focal length of the infrared cameras 2R and 2L, D is the base length (interval of the optical axis) of the infrared cameras 2R and 2L, and p is the pixel pitch (length of one pixel).

  The above is the outline of the processing of STEPs 11 to 13. In addition, the process of STEP11-13 is performed with respect to each target object extracted by said STEP7.

  After the processing of STEP 10 and STEP 13, the image processing unit 1 next calculates a real space position that is a position (relative position with respect to the host vehicle 10) of each object in the real space (STEP 14). Here, as shown in FIG. 2, the real space position is a position (X, Y, Z) in the real space coordinate system (XYZ coordinate system) set with the midpoint of the attachment position of the infrared cameras 2R, 2L as the origin. ). The X direction and the Y direction of the real space coordinate system are the vehicle width direction and the vertical direction of the host vehicle 10, respectively. These X direction and Y direction are the x direction (lateral direction) and y of the right image and the left image, respectively. It is the same direction as the direction (vertical direction). The Z direction in the real space coordinate system is the front-rear direction of the host vehicle 10. Then, the real space position (X, Y, Z) of the object is calculated by the following equations (2), (3), (4).

X = x × z × p / f (2)
Y = y × z × p / f (3)
Z = z (4)

Note that x and y are the x-coordinate and y-coordinate of the object on the reference image.

  Next, the image processing unit 1 compensates for the influence of the change in the turning angle of the host vehicle 10 and increases the accuracy of the real space position of the target object, so that the real space position (X, Y, Z) of the target object is increased. The position X in the X direction is corrected according to the time-series data of the turning angle obtained in STEP 10 from the value obtained by the above equation (2) (STEP 15). Thereby, the real space position of the object is finally obtained. In the following description, “the real space position of the object” means the real space position of the object subjected to this correction. Note that the real space position of the object is sequentially calculated at a predetermined calculation processing cycle.

  Note that the processing of STEPs 14 and 15 is the same as the processing of S21 and S22 of FIG. In the following description, the X-direction component (X-direction position), Y-direction component (Y-direction position), and Z-direction component (Z-direction position) of the actual space position (in the current calculation processing cycle) of the object are described. Represented by X0, Y0 and Z0, respectively.

  Supplementally, the processing of STEPs 14 and 15 constitutes the relative position detecting means in the present invention.

  Next, the image processing unit 1 determines for each object (each object with the same label) whether or not the object has a lateral velocity component (STEP 16). Supplementally, the processing after STEP 16 (including the processing of STEP 16) is different from the processing of FIG.

  In the present embodiment, the determination process in STEP 16 is performed as follows.

  That is, first, the X direction component (X direction position) in the previous value of the real space position of the object (the real space position obtained in the previous calculation processing cycle) and the current value of the real space position of the object (current time) A difference ΔX with respect to the X direction component X 0 at the (current space position obtained in the current processing cycle) is obtained. Then, a value Vx obtained by dividing ΔX by the time of the arithmetic processing cycle is obtained as a speed component in the vehicle width direction of the current relative speed of the object (relative speed with respect to the vehicle 10). Further, when the magnitude (absolute value) of the velocity component Vx is not less than a predetermined lower limit value VxL (> 0) and not more than a predetermined upper limit value VxH (> lower limit value) (VxL ≦ | Vx | ≦ VxH). In the case where the object has a lateral velocity component with respect to the vehicle 10. If VxL> | Vx | or VxH <| Vx |, it is determined that the object does not have a lateral velocity component. Note that an object extracted for the first time in the current calculation processing cycle (an object not extracted in the previous calculation processing cycle) is determined not to have a lateral velocity component. Further, in this embodiment, Vx is obtained from two values, the previous value and the current value of the real space position of the object. However, for example, the latest value including the current value such as the last time value, the previous value, and the current value. Vx may be obtained from three or more real space positions. In that case, Vx may be obtained by a sequential moving average process.

  Here, in the present embodiment, the lower limit value VxL and the upper limit value VxH are set so that the magnitude of the speed that can generally be taken when a person crosses the road falls between the lower limit value VxL and the upper limit value VxH. It is set in advance. Note that an object having a speed component in the vehicle width direction larger than the upper limit value VxH is highly likely to be a moving object other than a pedestrian, such as another vehicle. Such an object is determined in STEP 16 as a lateral speed. It is determined that there is no component. Therefore, the object determined to have the lateral velocity component in STEP 16 is an object that is highly likely to be a pedestrian (person). In addition, since the lower limit value VxL is set to a value slightly larger than 0, the lower limit value VxL is moving in the same direction or the opposite direction to the vehicle 10 due to a detection error of the real space position of the object. It is avoided that a pedestrian or a stationary object (for example, a utility pole or a vending machine) is determined to have a lateral velocity component.

  Supplementally, the lower limit value VxL and the upper limit value VxH may be set in consideration of not only the movement speed of a person but also the movement speed of an animal other than a person. The determination process of STEP 16 constitutes the speed component determination means and the speed component determination step in the present invention, and is a process executed by the microcomputer of the image processing unit 1 by the speed component determination program.

  When the determination result in STEP 16 is YES (when there is an object determined to have a lateral speed component), the image processing unit 1 determines each object determined to have the lateral speed component. A first avoidance target determination process (first avoidance target determination algorithm) described later is executed (STEP 19). When the determination result in STEP 16 is NO (when there is an object determined not to have a lateral velocity component), the image processing unit 1 performs time series of the real space position for the object. It is determined whether or not the number of data has reached a predetermined number N (N is a number larger than the number of samples when the lateral speed component is obtained in STEP 16, for example, 10) (STEP 17). When the time series data of the real space position of the object is less than the predetermined number N, the processing from STEP 1 is repeated. Thus, a time interval N-1 times the calculation processing cycle for calculating the real space position (which is longer than the calculation processing cycle of the real space position) is set as a monitoring period (observation period) of the real space position of the object. Time series data of the real space position of the object in this monitoring period is obtained. In the process of STEP 17, instead of determining the number of time-series data, the elapsed time from the time when each object is first extracted has reached a predetermined time (time corresponding to the predetermined number N). It may be determined whether or not.

  Supplementally, in each calculation processing cycle, when an object for which the determination result in STEP 16 is YES and an object for which NO is mixed, the execution of the first avoidance object determination process for the object for which YES is executed. Priority is given, and after the execution, processing from STEP 17 is performed.

  Further, when the number of the time-series data of the real space position for the object for which the determination result in STEP 16 is NO (the object determined to have no lateral velocity component) reaches a predetermined number N, The image processing unit 1 obtains a movement vector as a movement direction feature amount representing a relative movement direction of the object with respect to the vehicle 10 (STEP 18). Specifically, a straight line that approximates N time-series data of the real space position for the object is obtained by an approximation method such as the least square method, and (N−1) at the time of the calculation processing cycle before the cycle. A vector from the position (point) of the object on the straight line to the position (point) of the object on the straight line at the current time (the time of the current computation processing cycle) is obtained as a movement vector of the object. This movement vector is proportional to the relative velocity vector of the object with respect to the vehicle 10. Instead of the movement vector, for example, the azimuth angle of the straight line with respect to the front-rear direction or the vehicle width direction of the vehicle 10 may be calculated as the moving direction feature amount of the object.

  Supplementally, the processing in STEP 18 constitutes the moving direction feature amount calculating means in the present invention.

  Then, after calculating the movement vector as described above, the image processing unit 1 executes a second avoidance target determination process (second avoidance target determination algorithm) to be described later for the target object for which the movement vector has been calculated (STEP 20). ). It should be noted that in each arithmetic processing cycle, the object of which the determination result of STEP 16 is YES and the object of NO are mixed, and the time series data of the real space position for the object of which the determination result is NO When the number of the N reaches N, priority is given to the execution of the first avoidance target determination process for an object for which the determination result of STEP 16 is YES (an object determined to have a lateral speed component), After the execution, a second avoidance target determination process is executed for an object for which the determination result of STEP 16 is NO (an object determined not to have a lateral speed component).

  The first avoidance target determination process and the second avoidance target determination process are processes that determine whether or not the target object is an avoidance target that should avoid contact with the vehicle 10, as will be described in detail later. Specifically, whether or not the target object is an avoidance target is determined by a determination process regarding the degree of possibility that the target object contacts the vehicle 10 and the type of the target object. Note that these avoidance target determination processes are realized by the microcomputer executing a subroutine program installed in the ROM of the microcomputer.

  If the determination result in STEP 19 is YES, or if the determination result in STEP 20 is YES, that is, if it is determined in STEP 19 or 20 that the object is an avoidance target, the process proceeds to STEP 21. The image processing unit 1 executes a warning output determination process for determining whether or not the driver of the vehicle 10 should be alerted to the object.

  In this alerting output determination process, it is confirmed from the output of the brake sensor 5 that the driver has operated the brake of the vehicle 10 and the vehicle 10 is decelerating acceleration (acceleration in the vehicle speed decreasing direction is positive). ) Is greater than a predetermined threshold (> 0), it is determined that no alerting is to be performed. In addition, when the driver does not perform a braking operation or when the deceleration acceleration of the vehicle 10 is equal to or less than a predetermined threshold even if the braking operation is performed, it is determined that attention should be given. .

  When the image processing unit 1 determines that attention should be given (when the determination result of STEP 21 is YES), the image processing unit 1 issues a warning by the speaker 6 and the display device 7 to the driver of the vehicle 10. Is executed (STEP 21), and the processing from STEP 1 is repeated. In this alerting process, for example, the reference image is displayed on the display device 7 and the image of the target object (the target object determined as the avoidance target in STEP 19 or 20) in the reference image is highlighted. . Furthermore, the driver is instructed by voice from the speaker 6 that such an object to be avoided exists. As a result, the driver's attention is drawn to the object, and the driver can prepare a system for avoiding contact between the object and the vehicle 10. Note that the driver may be alerted only by either the speaker 6 or the display device 7. Further, when the vehicle 10 is a vehicle that can perform automatic steering, the steering control of the vehicle 10 is executed so as to avoid contact with an object that is determined as an avoidance target in STEP 19 or 20. Also good.

  Further, when it is determined in STEP 21 that alerting is not performed (when it is determined that alerting is not performed for all objects to be avoided), the determination result in STEP 21 is NO, and in this case, STEP 1 is maintained as it is. Processing from is resumed.

  Supplementally, the processing of STEP16 to STEP20 constitutes the avoidance target determination means and the avoidance target determination step in the present invention. In this case, in this embodiment, two algorithms of STEP 19 and 20 are provided as determination algorithms for determining whether or not the target object is an avoidance target.

  The above is the overall operation of the periphery monitoring device of this embodiment.

  Next, the determination processing of STEPs 19 and 20 which will be described later will be described in detail with reference to FIGS. 5 is a flowchart showing the first avoidance target determination process in STEP 19, FIG. 6 is a flowchart showing the second avoidance target determination process in STEP 20, and FIG. 7 is a first area AR1, second area AR2, third area AR3, etc., which will be described later. FIG.

  First, the determination process in STEP 19 will be described with reference to FIGS. The first avoidance target determination process of STEP 19 is a determination process (determination algorithm) that is executed for each of the objects determined to have a lateral velocity component in STEP 16, and is executed as shown in the flowchart of FIG. .

  First, a first object position determination process is performed as one determination process regarding the real space position of the object (relative position with respect to the vehicle 10) (STEP 31). The first object position determination process is a process for determining whether or not contact between the vehicle 10 and the object can be avoided with sufficient margin by steering or braking operation of the vehicle 10. Specifically, in the first object position determination process, an area in front of the vehicle 10 imaged by the infrared cameras 2R and 2L (an area within the viewing angle of the infrared cameras 2R and 2L; hereinafter referred to as an imaging area). Among them, the current real space position (real space position) of the object in the area AR1 (hereinafter referred to as the first area AR1) in which the distance in the Z direction from the vehicle 10 (the distance in the front-rear direction of the vehicle 10) is a predetermined value or less. It is determined whether or not there is a current value).

  In this case, the predetermined value related to the distance from the vehicle 10 is set for each object. Specifically, the difference between the Z direction component (Z direction position) in the previous value of the real space position of the target object and the Z direction component (Z direction position) in the current value of the real space position of the target object is calculated. Is obtained by dividing the relative speed Vz1 of the object in the longitudinal direction (Z direction) of the vehicle 10 by multiplying the relative speed Vz1 by a predetermined constant T (time dimension constant). Vz1 · T is set as the predetermined value that defines the boundary in the Z direction of the first area AR1. Thus, the predetermined value is set according to the relative speed Vz1 of the object in the front-rear direction (Z direction) of the vehicle 10. The predetermined value may be a fixed value, or may be set according to the relative speed Vz1 and the traveling speed (ground speed) of the vehicle 10.

  Here, as shown in FIG. 7, when the imaging region is a region between the straight lines L1 and L2 in the drawing in a plan view (the straight lines L1 and L2 are the left and right boundary lines of the viewing angles of the infrared cameras 2R and 2L). The first area AR1 is an area surrounded by the triangle abc in FIG. 7 in plan view. In addition, the first area AR1 is an area having a predetermined height (for example, about twice the height of the vehicle 10) in the vertical direction. Accordingly, when the Z direction position Z0 of the current real space position of the object is Vz1 · T or less and the Y direction position Y0 is a position of a predetermined height or less, the object is in the first area AR1. It is determined that it exists. When the relative speed Vz1 of the object in the front-rear direction of the vehicle 10 is a relative speed away from the vehicle 10, it is determined that the object does not exist in the first area AR1.

  In STEP 31, when it is determined that the object does not exist in the first area AR1 (when the determination result in STEP 31 is NO), the vehicle 10 is brought into contact with the object by steering or braking operation. It is a situation that can be avoided with a margin. In this case, the image processing unit 1 determines in STEP 36 that the target object is not an avoidance target, and ends the first avoidance target determination process for the target object.

  On the other hand, when it is determined in STEP 31 that the object is present in the first area AR1 (when the determination result in STEP 31 is YES), depending on the real space position of the object or the movement form of the object, There is a possibility that the object and the vehicle 10 will come into contact in the future. Therefore, in this case, the image processing unit 1 further executes a second object position determination process as a determination process related to the real space position of the object (STEP 32). This second object position determination process is a process for determining whether or not the possibility of contact between the vehicle 10 and the object is high when the real space position of the object is maintained at the current position. is there. Specifically, in the second object position determination process, the object extends in the front-rear direction on both sides of the vehicle 10 as shown in FIG. 7 (extending in parallel with the vehicle width center line L0 of the vehicle 10). It is determined whether or not it exists in an area AR2 (hereinafter referred to as a second area AR2) between a pair of boundary lines L3 and L4 set as follows.

  In this case, the left and right boundary lines L3 and L4 of the second area AR2 have the same distance W / 2 left and right from the vehicle width center line L0 of the vehicle 10, as shown in FIG. It is set to the position to have. The interval W between the boundary lines L3 and L4 is set to be slightly wider than the vehicle width α of the vehicle 10. Whether or not the object exists in the second area AR2 depends on whether the value of the X direction component X0 of the current real space position of the object is the X direction position of the boundary line L3 and the X direction position of the boundary line L4. It is determined by whether the value is between.

  In STEP 32, when it is determined that the real space position of the object exists in the second area AR2 (when the determination result in STEP 32 is YES), the object remains at the current real space position. There is a high possibility that the object comes into contact with the vehicle 10. In this case, the image processing unit 1 next performs a process of determining whether the object is a pedestrian (person) (more accurately, whether or not there is a high possibility of being a person) (STEP 33). A determination as to whether or not the object is a pedestrian may be made using a known method (for example, a method described in Japanese Patent Application Laid-Open No. 2003-284057), and the shape of the object on the grayscale image This can be done based on features such as size and luminance distribution.

  In this pedestrian determination process of STEP 33, if it is determined that the object is likely to be a pedestrian, the process proceeds to STEP 34 in order to increase the reliability of the determination that the object is a pedestrian. A determination process (vehicle determination process) is performed as to whether or not the object is another vehicle. Whether or not the object is another vehicle may be determined by using a known method (for example, a method described in Japanese Patent Application Laid-Open No. 2003-230134), and an image of the object (grayscale image or binary). For example, the presence or absence of a window shield part of another vehicle. Or you may make it determine whether a target object is another vehicle by the method of pattern matching.

  Supplementally, since the first avoidance target determination process is a process executed on the target determined to have the lateral velocity component in STEP 16, stationary objects such as utility poles and vending machines are excluded. . On the other hand, another vehicle that is about to turn relatively slowly in front of the vehicle 10 or another vehicle that is changing course may be a target of the first avoidance target determination process. For this reason, in this embodiment, the vehicle determination process of STEP34 is performed.

  When it is determined in STEP 34 that the object is not another vehicle (when the determination result in STEP 34 is NO), the image processing unit 1 determines in STEP 35 that the object is an avoidance target. The first avoidance target determination process is terminated. Therefore, when it is determined that the object exists in the second area AR2 in the first area AR1, the object is likely to be a pedestrian, and is not another vehicle, the object is It is determined to be an avoidance target.

  If it is determined in STEP 33 that the object is not a pedestrian, or if it is determined in STEP 34 that the object is an artificial structure, the process proceeds to STEP 36 and it is determined that the object is not an avoidance target. Then, the first avoidance target determination process ends.

  On the other hand, when it is determined in STEP 32 that the object does not exist in the second area AR2 (when the determination result in STEP 32 is NO), the image processing unit 1 next moves the real space position of the object. And a first approach determination process regarding the direction of the lateral velocity component Vx of the object (STEP 37). The first entry determination process determines whether or not the object is likely to enter the second area AR2 (the object is likely to cross the road in front of the vehicle 10). It is processing to do. Specifically, in the first entry determination process, the left and right sides of the second area AR2 are set in the pair of left and right third areas AR3 and AR3 (see FIG. 7) set to extend in the front-rear direction. And the direction (sign) of the lateral velocity component Vx of the object is determined whether or not the requirement that the direction is toward the second area AR2 is satisfied.

  Here, referring to FIG. 7, the left third area AR3 includes a left boundary line L3 of the second area AR2, and a boundary line L5 extending parallel to the boundary line L3 outside the second area AR2. The third region on the right side is a region between the right boundary line L4 of the second region and the boundary line L6 extending in parallel with the outer boundary line L4 of the second region. It is. In other words, these third areas AR3 and AR3 are areas extending in the front-rear direction adjacent to the second area AR2 outside the second area AR2 in the imaging area. The widths of the left and right third regions AR3 and AR3 are the same. The width of the third region AR3 may be a fixed value, but is changed according to the magnitude (absolute value) of the lateral velocity component Vx of the object (the larger the magnitude of the lateral velocity component, the third The width of the area AR3 may be increased). Further, the width of the third region AR3 may be increased as it approaches the vehicle 10 in the Z direction.

  Supplementally, whether or not the object exists in the third area AR3 depends on whether the current X direction position X0 of the object is between the X direction position of the boundary line L3 and the X direction position of the boundary line L5, or the boundary It is determined by whether or not the value is between the X direction position of the line L4 and the X direction position of the boundary line L6. Also, the direction of the lateral velocity component Vx of the object is determined by the sign of Vx. In this case, when the object is present on the left side of the second area AR2, when the direction of the lateral velocity component Vx is rightward, the direction is the direction approaching the second area AR2, and the object is When present on the right side of the third area AR2, when the direction of the lateral velocity component Vx is leftward, the direction is the direction approaching the second area AR2.

  In STEP 32, the object is present in any one of the third areas AR3 and AR3, and the direction (sign) of the lateral velocity component Vx of the object is a direction toward the second area AR2. Is satisfied (when the determination result in STEP 32 is YES), the object is about to cross the front of the vehicle 10, and there is a high possibility that the object will enter the second area AR2 in the near future. Therefore, in this case, the image processing unit 1 determines in STEP 35 that the target object is an avoidance target, and ends the first avoidance target determination process.

  In STEP 32, the real space position of the object is present in the third areas AR3 and AR3, and the direction (sign) of the lateral velocity component Vx of the object is the direction toward the second area AR2. If the requirement is not satisfied (when the determination result of STEP 32 is NO), that is, the object is an area outside the third area AR3 (the area on the left side of the boundary line L5 or the In the area on the right side of the boundary line L6) or when the direction of the lateral velocity component Vx of the object is away from the second area AR2, the image processing unit 1 determines in STEP 36 that It is determined that the object is not an avoidance target, and the first avoidance target determination process is terminated.

  The above is the details of the first avoidance target determination process. In addition, you may make it set the area | region except a 2nd area | region among imaging areas as 3rd area | region AR3.

  Next, the second avoidance target determination process in STEP 20 will be described. This second avoidance target determination process is executed as shown in the flowchart of FIG.

  In the second avoidance target determination process, first, a third target position determination process related to the real space position (relative position with respect to the vehicle 10) of the target whose movement vector is obtained in STEP 18 is executed (STEP 41). In the present embodiment, in the third object position determination process, in the same manner as the first object position determination process, the distance in the Z direction from the vehicle 10 (the distance in the front-rear direction of the vehicle 10) in the imaging region. However, it is determined whether or not an object exists in the first area AR1 that is equal to or less than a predetermined value. However, in this third object position determination process, the predetermined value that defines the boundary in the Z direction of the first area AR1 is the relative speed (Z direction) of the object in the monitoring period for obtaining the movement vector in STEP18. Of the velocity component) is set to a value Vz2 · T (see FIG. 7) obtained by multiplying the average value Vz2 by the constant T. In this case, the average value Vz2 of the relative speed of the object in the Z direction is calculated by dividing the Z direction component of the movement vector calculated in STEP 18 by the time of the monitoring period. The vertical height of the first area AR1 is set to be the same as that in the case of the first object position determination process.

  In STEP 41, when the object does not exist in the first area AR1 (when the determination result in STEP 41 is NO), the contact between the object and the vehicle 10 can be avoided with sufficient margin by steering or braking operation of the vehicle 10. For this reason, in this case, the image processing unit 1 determines in STEP 46 that the object is not an avoidance target, and ends the second avoidance target determination process.

  In STEP 41, when the object is present in the first area AR1 (when the determination result in STEP 41 is YES), the image processing unit 1 next selects the fourth object position relating to the real space position of the object. A determination process is executed (STEP 42). The fourth object position determination process is the same as the second object position determination process, and it is determined whether or not the object exists in the second area AR2 (see FIG. 7).

  In STEP 41, when the object is present in the second area AR2, the same pedestrian determination process (determination process as to whether or not the object is likely to be a pedestrian) is performed in STEP 43. If the determination result in STEP 44 is YES, a determination process (artificial structure determination process) for determining whether or not the target object is an artificial structure such as a utility pole or another vehicle is further performed in STEP 44. . Whether or not the object is an artificial structure may be determined by using a known method (for example, a method described in JP2003-16429A or JP2003-230134A). The image (grayscale image or binarized image) can be performed based on the presence or absence of a straight line portion or a right angle portion, the presence or absence of a window shield portion of another vehicle, and the like. Alternatively, it may be determined whether or not the object is an artificial structure by a pattern matching method.

  Supplementally, since the second avoidance target determination process is a process executed on the target object determined not to have the lateral velocity component in STEP 16, the utility pole or automatic that is not the target of the first avoidance target determination process A stationary object such as a vending machine, or another vehicle moving in the same direction as or opposite to the vehicle 10 may be a target of the second avoidance target determination process. For this reason, in this embodiment, the artificial structure determination process of STEP44 is performed.

  At this time, if the determination result in STEP 44 is NO, it is determined in STEP 45 that the object is an avoidance target, and the second avoidance target determination process ends. If the determination result in STEP 43 is NO, or if the determination result in STEP 44 is YES, it is determined in STEP 46 that the target is not an avoidance target, and the second avoidance target determination process ends. .

  On the other hand, when it is determined in STEP42 that the object does not exist in the second area AR2 (when the determination result in STEP42 is NO), that is, the object is located outside the second area AR2 in the imaging area. If it exists in the area, the image processing unit 1 next executes a second approach determination process regarding the moving direction of the object (STEP 47). The second entry determination process is a process of determining whether or not the object enters the second area AR2 and has a high possibility of contact with the vehicle 10. Specifically, assuming that the movement vector of the object is maintained as it is, the position in the X direction of the intersection point between the straight line including this movement vector and the XY plane of the real space coordinate system at the front end position of the vehicle 10 is obtained. It is done. Then, it is a requirement that the X-direction position thus obtained exists in a predetermined range (a range slightly wider than the vehicle width of the host vehicle 10) centered on the position in the X direction of the vehicle width center line L0 of the host vehicle 10 (hereinafter referred to as a requirement). It is determined whether or not this approach contact requirement is satisfied.

  In STEP 47, when the object satisfies the entry contact requirement (when the determination result in STEP 47 is YES), there is a high possibility that the object will contact the vehicle 10 in the future. Therefore, in this case, the image processing unit 1 determines in STEP 45 that the target object is an avoidance target, and ends the second avoidance target determination process.

  Further, in STEP 47, when the object does not satisfy the entry contact requirement (when the determination result of STEP is NO), the object is unlikely to contact the vehicle 10, so the image processing unit 1 In STEP 46, it is determined that the object is not an avoidance target, and the second avoidance target determination process is terminated.

  The above is the details of the second avoidance target determination process.

  In the first embodiment described above, when it is determined in STEP 16 that the object has a lateral velocity component, the first avoidance target determination is performed by the image processing unit 1 (specifically, the microcomputer of the image processing unit 1). When the process is executed and it is determined that there is no lateral velocity component in the monitoring period, the second avoidance target determination process is executed by the image processing unit 1. In this case, since the first avoidance target determination process does not include a determination process related to the movement vector determined based on the time-series data of the object in the monitoring period, the pedestrian is crossing the road ahead of the vehicle 10. When or when trying to cross (or when the possibility is high), the pedestrian is quickly determined to be an avoidance target, and the driver of the vehicle 10 regarding the pedestrian is alerted. can do. Further, since the second avoidance target determination process includes the second entry determination process of STEP 47 that is a determination process related to the movement vector, it is not executed unless the movement vector is calculated. However, since the possibility that the target object contacts the vehicle 10 can be determined with high reliability by the second approach determination processing, appropriate alerting can be performed to the driver of the vehicle 10.

  Next, a second embodiment of the present invention will be described with reference to FIG. This embodiment is different from the first embodiment only in the determination process of STEP 16 (determination process for determining whether or not it has a lateral velocity component), and FIG. 8 shows the determination process.

  As will be described below, in the present embodiment, in the determination process of STEP 16, as described in the first embodiment, first, the speed component Vx of the object in the vehicle width direction of the vehicle 10 is calculated (STEP 51). It is determined whether or not the magnitude of the velocity component Vx is not less than a predetermined lower limit value VxL and not more than an upper limit value VxH (STEP 52). At this time, if the determination result in STEP 52 is NO (when VxL> | Vx | or VxH <| Vx |), it is determined in STEP 56 that the object does not have a lateral velocity component.

  On the other hand, when the determination result in STEP 52 is YES (when VxL ≦ | Vx | ≦ VxH), the object is determined based on the comparison between the current real space position of the object and the second area AR2. It is determined whether or not it exists in the second area AR2 (STEP 53). At this time, if the object is present in the second area AR2 and the determination result in STEP 53 is YES, the object is in the horizontal direction in STEP 55 regardless of the direction (sign) of the velocity component Vx of the object. It is determined that it has a velocity component.

  On the other hand, if the determination result in STEP 53 is NO, that is, if the object is present outside the second area AR2 (the left area or the right area of the second area AR2) within the imaging area, Whether the direction of the velocity component Vx of the object is a direction approaching the second area AR2 is determined based on the sign of Vx (STEP 54). In this case, when the object is present in the left area of the second area AR2, the direction approaching the second area AR2 is rightward, and the object is present in the right area of the second area AR2. The direction approaching the second area AR2 is leftward.

  When the determination result in STEP 54 is YES (when the direction of Vx is the direction approaching the second area AR2), it is determined in STEP 55 that the object has a lateral velocity component. When the determination result in STEP 54 is NO (when the direction of Vx is a direction away from the second area AR2), it is determined in STEP 56 that the object does not have a lateral velocity component.

  As described above, in the present embodiment, in addition to the requirement of VxL ≦ | Vx | ≦ VxH, the direction of Vx satisfies the requirement determined according to the real space position of the object (relative position with respect to the vehicle 10). The object is determined to have a lateral velocity component. In this case, the requirement regarding the direction of Vx is a requirement that Vx is in the left or right direction when the object is present in the second area AR2, and when the object is not present in the second area AR2. , Vx is in a direction approaching the second area AR2.

  By determining whether or not the vehicle has the lateral speed component in this way, the object determined to have the lateral speed component can cross the front of the vehicle 10 as compared with the case of the first embodiment. It becomes a high-quality object.

  Supplementally, in the present embodiment, when the third area is set as an area outside the second area AR2, the determination result of STEP 37 in the first avoidance target determination process is always YES. Therefore, in this case, when the determination result of STEP 32 in FIG. 5 is NO, the determination process (first entry determination process) of STEP 37 is omitted, and it is determined that the object is an avoidance target. May be.

  Next, a third embodiment of the present invention will be described with reference to FIG. In the present embodiment, the processing from STEP 16 to STEP 17 in FIG. 4 in the first embodiment is changed, and FIG. 9 is a flowchart partially showing the processing after the change. The rest is the same as in the first embodiment.

  Referring to FIG. 9, in the present embodiment, for an object determined to have a lateral velocity component in STEP 16, a determination process related to the real space position of the object is further executed in STEP 60. Specifically, in STEP 60, it is determined whether or not the current Z-direction position Z0 of the object, that is, the distance of the object from the vehicle 10 in the front-rear direction of the vehicle 10 is equal to or less than a predetermined value. In this case, the predetermined value is set according to the current relative speed Vz1 of the object in the Z direction, and is, for example, the same value as a predetermined value (= Vz1 · T) that defines the boundary in the Z direction of the first area AR1. Set to However, the predetermined value in the determination process of STEP 60 and the predetermined value (= Vz1 · T) defining the boundary in the Z direction of the first area AR1 do not have to be the same value. For example, the former value is changed to the latter value. Different values may be set, or the former value may be set to a fixed value.

  If the determination result in STEP 60 is YES (when Z0 ≦ predetermined value), the process proceeds to STEP 19 in FIG. 4 for the object, and the first avoidance object determination process is executed. If the determination result in STEP 60 is NO, the processing from STEP 17 is executed, and when the number of time-series data at the real space position finally reaches N, STEP 20 in FIG. Then, the second avoidance target determination process is executed.

  As described above, in the present embodiment, the first avoidance target determination process and the first avoidance process are performed according to the set of the determination result of the presence or absence of the lateral velocity component in STEP 16 and the determination result regarding the real space position of the object in STEP 60. 2 The avoidance target determination process is selectively executed.

  In this case, in the present embodiment, the first avoidance target determination process is executed not only when the target object has a lateral velocity component but also when the relative position of the target object in the Z direction is relatively close to the vehicle 10. The Rukoto. In other words, the first avoidance target determination process, which is a simple avoidance target determination process that does not use a movement vector, is executed only in a situation in which it is more quickly determined whether or not an object should be an avoidance target. In this situation, the second avoidance target determination process is executed to increase the reliability of the determination as to whether or not the target object should be the avoidance target, so that alerting can be performed more appropriately.

  Supplementally, in the present embodiment, the determination process in STEP 60 may be the same as the determination process in STEP 31 (first object position determination process) in the first avoidance target determination process. In that case, the determination result of STEP 31 in the first avoidance target determination process is always YES, and therefore the determination process of STEP 31 may be omitted.

  In this embodiment, only the position of the object in the Z direction is determined in STEP 60. However, in addition to or instead of this, the position of the object in the X direction may be determined. Good. For example, when the X direction position of the object is present in the left area of the left boundary line L5 of the third area AR3 or the right area of the right boundary line L6, the first avoidance object determination process in STEP 19 is executed. Instead, the processing from STEP 17 may be executed.

  In the first to third embodiments, the object is detected based on the captured images of the infrared cameras 2R and 2L. However, the object is detected based on the captured image of the visible light camera. Alternatively, the object may be detected by a radar or the like. Moreover, you may make it detect the relative position and relative velocity with respect to the vehicle 10 of a target object using a radar.

The figure which shows the whole structure of the apparatus of 1st Embodiment of this invention. The perspective view of the vehicle provided with the periphery monitoring apparatus of FIG. 3 is a flowchart showing processing of an image processing unit provided in the periphery monitoring device of FIG. 1. 3 is a flowchart showing processing of an image processing unit provided in the periphery monitoring device of FIG. 1. The flowchart which shows the process of STEP19 of FIG. The flowchart which shows the process of STEP20 of FIG. The figure for demonstrating the area | region used by the process of STEP19, 20 of FIG. The flowchart which shows the determination process of STEP16 in 2nd Embodiment of this invention. 10 is a flowchart showing a part of processing of an image processing unit in the third embodiment of the present invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image processing unit (Object detection means, speed component determination means, avoidance target determination means, relative position detection means, movement direction feature quantity calculation means), STEP 1 to 9 ... Object detection means, STEP 14, 15 ... Relative position detection Means, STEP 16 ... speed component determination means, STEP 16 to STEP 20 ... avoidance target determination means, STEP 18 ... movement direction feature quantity calculation means.

Claims (9)

Object detection means for detecting an object present at least on the front side of the vehicle, and avoidance object determination means for determining whether or not the detected object is an avoidance object to avoid contact with the vehicle; In a vehicle periphery monitoring device equipped with
The relative speed of the object detected by the object detection means is a speed component in the vehicle width direction of the vehicle, and a lateral speed component whose magnitude is equal to or greater than a predetermined positive lower limit value. Speed component determination means for sequentially determining whether or not it has,
The avoidance target determination means is a means capable of selectively executing a plurality of types of determination algorithms for determining whether or not the object is an avoidance target, and at least the determination result of the speed component determination means Accordingly, a vehicle is characterized in that one of the plurality of types of determination algorithms is selected and the determination algorithm is executed to determine whether or not the object is an avoidance target. Perimeter monitoring device.   2. The vehicle periphery monitoring device according to claim 1, wherein the lateral speed component is a speed component whose magnitude is not more than a predetermined upper limit value that is larger than the lower limit value.   Relative position detection means for successively detecting the relative position of the object detected by the object detection means with respect to the vehicle is provided, and the direction of the lateral velocity component is the relative position detected by the relative position detection means. The vehicle periphery monitoring device according to claim 1, wherein the vehicle periphery monitoring device is a speed component in a direction that satisfies a predetermined requirement determined in accordance with the requirements.   Relative position detecting means for sequentially detecting the relative position of the object detected by the object detecting means with respect to the vehicle, and a relative position for determining whether or not the relative position exists in a predetermined area in front of the vehicle A determination unit, wherein the avoidance target determination unit selects one determination algorithm from among the plurality of determination algorithms according to a set of a determination result of the speed component determination unit and a determination result of the relative position determination unit The vehicle periphery monitoring device according to claim 1, wherein the vehicle periphery monitoring device is executed. Relative position detecting means for sequentially detecting a relative position of the object to the vehicle;
A moving direction representing a moving direction of the object relative to the vehicle based on a time series of the relative position of the object detected by the relative position detecting unit in a time interval longer than the execution period of the determination process of the speed component determining unit. A moving direction feature amount calculating means for calculating the feature amount;
The determination algorithm selected by the avoidance target determination unit when the determination result of the speed component determination unit is negative is configured by an algorithm including at least a determination process related to the moving direction feature amount, and the determination of the speed component determination unit 2. The vehicle according to claim 1, wherein when the result is affirmative, the determination algorithm selected by the avoidance target determination unit is configured by an algorithm that does not include at least a determination process related to the moving direction feature quantity. Perimeter monitoring device. A moving direction representing a moving direction of the object relative to the vehicle based on a time series of the relative position of the object detected by the relative position detecting unit in a time interval longer than the execution period of the determination process of the speed component determining unit. A moving direction feature amount calculating means for calculating the feature amount;
The determination algorithm selected by the avoidance target determination unit when the determination result of the speed component determination unit or the determination result of the relative position determination unit is negative is constituted by an algorithm including at least a determination process related to the moving direction feature amount The determination algorithm selected by the avoidance target determination unit when the determination result of the speed component determination unit and the determination result of the relative position determination unit are affirmative does not include at least a determination process related to the moving direction feature amount The vehicle periphery monitoring device according to claim 4, wherein the vehicle periphery monitoring device is configured by an algorithm.   A vehicle comprising the vehicle periphery monitoring device according to any one of claims 1 to 6. In the vehicle periphery monitoring method for detecting an object present at least on the front side of the vehicle and determining whether the detected object is an avoidance object to avoid contact with the vehicle,
Whether the relative speed of the detected object with respect to the vehicle is a speed component in the vehicle width direction of the vehicle and has a lateral speed component whose magnitude is equal to or greater than a predetermined positive lower limit value. A speed component determining step for sequentially determining;
A determination algorithm for determining whether or not the object is an avoidance target is selected from a plurality of types of determination algorithms prepared in advance according to the determination result of at least the speed component determination step, A vehicle periphery monitoring method comprising: an avoidance target determination step for determining whether or not the target object is an avoidance target by executing a selected determination algorithm. A vehicle periphery monitoring program that causes a computer to execute a process of determining whether or not an object detected at least in front of the vehicle is an avoidance target that should avoid contact with the vehicle. And
Whether the relative speed of the detected object with respect to the vehicle is a speed component in the vehicle width direction of the vehicle and has a lateral speed component whose magnitude is equal to or greater than a predetermined positive lower limit value. Speed component determination processing for sequential determination;
A determination algorithm for determining whether or not the object is an avoidance target is selected from a plurality of predetermined determination algorithms according to at least the determination result of the speed component determination process, A vehicle periphery monitoring program comprising a function of causing the computer to execute an avoidance target determination process for determining whether or not the object is the avoidance target by the selected determination algorithm.

Images