JP2007241898A - Stopping vehicle classifying and detecting device and vehicle peripheral monitoring device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To distinguish/detect with high reliability whether or not another vehicle stopping ahead of an own vehicle is a temporary stopping one and to alert a driver of the own vehicle with a proper alarm. <P>SOLUTION: This device detects the other stopping vehicle from an image ahead of the own vehicle 10 picked up by image pickup devices 2R, 2L and detects the temperature state of the other vehicle based on the brightness state of the image of the other vehicle. The device determines whether or not the other vehicle is a temporary stopping one based on the temperature state. According to whether or not it is a temporary stopping one, the device variably sets up at least one of a detection region of an object and a requirement for alarm regarding the object. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a device for separately detecting whether another vehicle stopped in front of the host vehicle is a temporarily stopped vehicle or a long-term stopped vehicle, and surrounding monitoring of the vehicle using the device Relates to the device.

When an imaging device mounted on a vehicle captures an image of the front of the host vehicle and detects an object such as a pedestrian or another vehicle from the captured image, or when the detected object is a pedestrian, or is detected When an object that meets the required alarm generation requirements is detected, such as when there is a risk that the target object will collide with the host vehicle, the driver of the host vehicle is alerted and the driver's attention is given. Devices that are designed to evoke are conventionally known (see, for example, Patent Document 1 and Patent Document 2).
JP 11-215487 A JP 2003-230134 A

  By the way, when other vehicles existing in front of the host vehicle (including not only the front front of the host vehicle but also diagonally forward) are stopped (not moving), what is the stop status of the other vehicle? In many cases, it is desirable to change the environmental range that the driver of the host vehicle should be aware of in order to avoid a collision. For example, when the other vehicle that is stopped in front of the host vehicle (especially diagonally forward) is a vehicle that is temporarily stopped, the driver of the host vehicle has not been recognized yet, or little attention is paid. There are many situations where the driver of the other vehicle is trying to cross a person who is not facing or getting on the other vehicle, or a passenger of the other vehicle is getting off the vehicle. In addition, even when the other vehicle is in a stop state immediately after parking, there are many situations where a passenger of the other vehicle is about to get off. In such a situation, a situation in which a person who crossed the road, a person who got off from another vehicle, etc. unexpectedly jumped out in front of the host vehicle stopped (parked) constantly. It is more likely to occur than if it is.

  Therefore, in the situation as described above, there is a wider range of people than when the other vehicle is stationary in front of the own vehicle or when there is no other vehicle stopped in front of the own vehicle. It is desirable to alert the driver to

  On the other hand, as a method for detecting another vehicle in front of the host vehicle from a captured image of the imaging device, various methods such as the method described in Patent Document 2 have been proposed. Whether or not the other vehicle is a stopped vehicle is recognized by, for example, detecting the relative speed of the other vehicle with respect to the own vehicle and comparing the relative speed with the detected speed of the own vehicle. Is possible.

  However, conventionally, whether the other vehicle stopped in front of the host vehicle is temporarily stopped, just after parking, or in a state where it is parked regularly. In fact, an appropriate method for distinguishing and detecting the state of the stop state of the other vehicle has not been proposed yet. Therefore, a method for distinguishing and detecting what type of stop state the other vehicle is in is desired.

  In this case, for example, when an imaging device in the visible light range is mounted on the own vehicle, the other vehicle temporarily stops depending on whether or not the brake light of the other vehicle stopped in front of the own vehicle is lit. It is conceivable to detect whether or not there is. However, in this case, when the other vehicle is present in the oncoming lane, it cannot be detected whether the other vehicle is temporarily stopped, and further, for parking such as a side brake. It is also impossible to detect other vehicles that are temporarily stopped by operating the brake device, or other vehicles that have just been parked.

  The present invention has been made in view of such a background, and can detect with high reliability whether or not another vehicle stopped in front of the host vehicle is a temporarily stopped vehicle. It aims at providing a stop vehicle classification detection device. It is another object of the present invention to provide a vehicle periphery monitoring device capable of providing an appropriate alarm to the driver of the host vehicle using the stopped vehicle classification detection device.

  In the present invention, the “temporarily stopped vehicle” is a vehicle that is stopped while the prime mover (propulsion source) of the vehicle is operated, or is stopped immediately after the operation of the prime mover of the vehicle is stopped. Means a vehicle.

  In order to achieve the above object, the stop vehicle classification detection device of the present invention detects whether or not another vehicle stopped in front of the host vehicle is a temporary stop vehicle. An imaging device mounted on the front of the host vehicle and having sensitivity in the far-infrared region, and stopped vehicle detection for detecting other vehicles that are stopped from an image in front of the host vehicle captured by the imaging device And a stopped vehicle image that is a partial image corresponding to the stopped other vehicle among the images captured by the imaging device when the stopped vehicle detecting means detects the stopped other vehicle. And a temperature state detecting means for detecting a temperature state of the other vehicle being stopped, and whether the other vehicle being stopped is a temporarily stopped vehicle based on the temperature state detected by the temperature state detecting means. Judgment means for distinguishing whether or not Characterized in that it obtain.

  According to the present invention, since the imaging device is an imaging device having sensitivity in the far-infrared region, the stop corresponding to the other vehicle being stopped is displayed in the image ahead of the host vehicle imaged by the imaging device. When a vehicle image is included, the stopped vehicle image is an image according to the temperature state of the other vehicle. Specifically, the luminance of each part of the stopped vehicle image corresponds to the temperature of the part of the other vehicle corresponding to that part. Therefore, the temperature state of the other vehicle can be detected based on the stopped vehicle image.

  Here, the temperature state (temperature distribution or the like) of the other vehicle that is stopped is significantly different depending on whether or not the other vehicle is a temporarily stopped vehicle. That is, when the other vehicle being stopped is not a temporarily stopped vehicle, generally, the temperature of each part of the other vehicle is not so high and the variation in the temperature is small. On the other hand, when the other vehicle is a temporarily stopped vehicle, the temperature state of the other vehicle is a temperature state having a local high temperature portion. For example, in each part of the other vehicle, the temperature of the exhaust pipe, the tire, or the prime mover mounting part is higher than the temperature of the other part. In addition, the temperature of each part of the other vehicle also varies. Therefore, by detecting the temperature state of the other vehicle based on the stopped vehicle image, it is possible to distinguish and determine whether or not the other vehicle is a temporarily stopped vehicle.

  Therefore, according to the stop vehicle classification detection device of the present invention, it is possible to distinguish and detect with high reliability whether another vehicle stopped in front of the host vehicle is a temporarily stopped vehicle.

  In addition, what is necessary is just to perform the detection of the other vehicle by the said stop vehicle detection means by a well-known method. At this time, whether or not the other vehicle is stopped is determined by, for example, a known distance measuring method using a radar or a known distance measuring method using parallax by stereo image processing. The relative position with respect to the vehicle may be detected in time series, and a determination may be made based on a comparison between the relative speed obtained from the time series of the relative position and the detected speed of the host vehicle.

  In the stopped vehicle classification detection device of the present invention, it is preferable that the temperature state detected by the temperature state detection means includes at least a temperature state of a predetermined part of the other vehicle that is stopped. In this case, the predetermined portion is preferably at least one of an exhaust pipe, a tire, and a prime mover mounting portion of the other vehicle, for example.

  That is, when the other vehicle being stopped is a temporarily stopped vehicle, the temperature state of the other vehicle is a temperature state having a local high temperature portion as described above. By detecting the temperature state of the predetermined part as the predetermined part, it is possible to easily determine whether the other vehicle is a temporarily stopped vehicle based on the predetermined temperature state it can. In particular, when the other vehicle is a temporarily stopped vehicle, the temperature of the exhaust pipe, tire, and prime mover mounting portion of the other vehicle becomes significantly high. By detecting the temperature state of the predetermined part as the predetermined part, it can be appropriately determined with high reliability whether or not the other vehicle is a temporarily stopped vehicle.

  Further, in the stopped vehicle classification detection device according to the present invention, the brightness of each part of the stopped vehicle image depends on the temperature of the part of the other vehicle corresponding to the part. The brightness state of the vehicle image may be detected as indicating the temperature state of the stopped vehicle. Thereby, appropriate detection of the temperature state of another vehicle can be performed easily.

  Next, in order to achieve the above object, the vehicle periphery monitoring apparatus of the present invention has a first aspect and a second aspect, and the first aspect is at least a predetermined detection in front of the host vehicle. An object detection means for detecting an object existing in the area and an alarm to the driver of the host vehicle are issued as a necessary requirement that the object detected by the object detection means satisfies at least a predetermined alarm generation requirement. In a vehicle periphery monitoring device comprising an alarm generation means, the detection region is variably set according to the determination result of the stop vehicle classification detection device of the present invention and the determination means of the stop vehicle classification detection device. And a detection area setting means.

  According to the first aspect, the detection area (spatial area) is variably set according to the determination result of the determination means of the stopped vehicle classification detection device. The detection area where the object is to be detected can be different depending on whether the vehicle is a temporarily stopped vehicle or not. Therefore, the object can be detected in a detection region suitable for each stop situation depending on whether or not the other vehicle being stopped is a temporarily stopped vehicle, and thus suitable for each stop situation. Warning can be given to the driver of the vehicle.

  Therefore, according to the first aspect of the vehicle periphery monitoring device of the present invention, it is possible to effectively use the stopped vehicle classification detection device of the present invention to give an appropriate warning to the driver of the host vehicle.

  In this first aspect, it is needless to say that the detection of the object by the object detection means may be performed based on the image captured by the imaging device of the stopped vehicle classification detection device. The object may be detected using an imaging device different from the imaging device of the stopped vehicle classification detection device or using another detection means such as a radar.

  In the first aspect, more specifically, the detection area setting means determines that the detected other vehicle being stopped is a temporarily stopped vehicle by the determination means of the stopped vehicle classification detection device. When the other vehicle that is stopped is not detected, or the detected other vehicle that is stopped is determined not to be a temporarily stopped vehicle, the detection region is at least in the width direction of the host vehicle. It is preferable to set so as to enlarge.

  According to this, when another vehicle that is stopped in front of the host vehicle is detected and it is determined that the other vehicle is a temporarily stopped vehicle, that is, the possibility of a person jumping out increases. In a situation where the surrounding area to be noted by the driver of the vehicle is wide, the stopped other vehicle is not detected, or the detected other vehicle that is stopped is determined not to be a temporarily stopped vehicle. Since the detection area is expanded at least in the width direction of the host vehicle, a spatial range in which an object that satisfies the alarm generation requirement can be detected increases at least in the vehicle width direction of the host vehicle. For this reason, it is possible to detect an object that satisfies the alarm generation requirement (particularly, an object for which the driver's attention has not yet been directed) at an early stage, and to issue an alarm to the driver. In addition, when the other vehicle that is stopped is not detected or the detected other vehicle that is stopped is not a temporarily stopped vehicle, the detected region is the temporary other vehicle that is detected. Since it is narrower in the vehicle width direction of the host vehicle than in the case of a stopped vehicle, it is possible to prevent the frequency of alarms from being increased more than necessary.

  In the first aspect of the vehicle periphery monitoring device of the present invention, the object detection means is based on an image in front of the own vehicle amount captured by the imaging device of the stopped vehicle classification detection device. The detection area setting means is a means for detecting an object, and the detection area setting means determines an image area in which the object is to be detected, among the images captured by the imaging apparatus, a determination result of the determination means of the stop vehicle classification detection apparatus The detection area can be variably set by variably setting according to the above. In this case, the alarm object detection means and the stopped vehicle classification detection device can share the imaging device.

  According to a second aspect of the vehicle periphery monitoring apparatus of the present invention, in order to achieve the above object, at least an object detection means for detecting an object existing in front of the host vehicle, and the object detection means. In the vehicle periphery monitoring device, comprising the alarm generation means for issuing an alarm to the driver of the host vehicle, on the assumption that the object detected by the vehicle satisfies at least a predetermined alarm generation requirement, the stop of the present invention described above A vehicle classification detection device and an alarm generation requirement setting unit that variably sets the alarm generation requirement according to a determination result of the determination unit of the stopped vehicle classification detection device.

  According to this second aspect, since the alarm generation requirement is variably set according to the determination result of the determination means of the stopped vehicle classification detection device, the other vehicle that is stopped ahead of the host vehicle is temporarily A warning can be given to the driver of the host vehicle with different alarm generation requirements depending on whether the vehicle is a stopped vehicle or not. For this reason, depending on whether or not the other vehicle being stopped is a temporarily stopped vehicle, it is possible to appropriately issue a warning to the driver of the host vehicle according to the alarm generation requirement suitable for each stop situation.

  Therefore, according to the second aspect of the vehicle periphery monitoring device of the present invention, it is possible to effectively use the stopped vehicle classification detection device of the present invention to give an appropriate warning to the driver of the host vehicle.

  In the second aspect, as in the case of the first aspect, the detection of the object by the object detection means is performed based on the image captured by the imaging device of the stopped vehicle classification detection device. Of course, the object may be detected using an imaging device different from the imaging device of the stopped vehicle classification detection device or using another detection means such as a radar. It may be.

  In the second aspect, when the alarm generation requirement includes a requirement regarding a positional relationship of the alarm object detected by the object detection means with respect to a predetermined alarm area in front of the host vehicle, the alarm is generated. The requirement setting means is preferably means for variably setting the warning generation requirement by variably setting the warning area according to the determination result of the determination means of the stopped vehicle classification detection device.

  According to this, the warning area is set to different areas depending on whether the other vehicle in front of the host vehicle is temporarily stopped or not, and the change in the warning area causes the Even if the object is the same, the positional relationship of the object with respect to the alarm region can be changed. Thereby, the variable setting of the alarm generation requirement can be easily performed.

  Note that the alarm generation requirement in this case includes conditions such as the presence of an object in the alarm area and the prediction that the object will enter the alarm area within a predetermined time.

  In this case, more specifically, when the alarm generation requirement setting unit determines that the other vehicle being stopped detected by the determination unit of the stopped vehicle classification detection device is a temporarily stopped vehicle, The warning area is expanded at least in the width direction of the own vehicle as compared with the case where it is determined that the stopped other vehicle is not detected or the detected stopped other vehicle is not a temporarily stopped vehicle. It is preferable to set so.

  According to this, when another vehicle that is stopped in front of the host vehicle is detected and it is determined that the other vehicle is a temporarily stopped vehicle, that is, the possibility of a person jumping out increases. In a situation where the surrounding area to be noted by the driver of the vehicle is wide, the stopped other vehicle is not detected, or the detected other vehicle that is stopped is determined not to be a temporarily stopped vehicle. Since the warning area is expanded at least in the width direction of the host vehicle, the chance that the detected object comes to satisfy the warning generation requirement can be increased. As a result, the chance of warning for the driver of the host vehicle can be increased, and the driver's attention can be effectively evoked.

  In the first and second aspects of the vehicle periphery monitoring device of the present invention, it is preferable that the object includes at least a person.

  However, the object may include animals other than humans, other vehicles, etc., and a plurality of types of objects may be detected. When other vehicles are included in the object, for example, a warning to the driver may be issued based on the alarm generation requirement that the temporarily stopped vehicle is detected by the stopped vehicle classification detection device.

  In the first and second aspects of the vehicle periphery monitoring device of the present invention, the alarm generation requirement is not limited to the requirement related to the positional relationship of the alarm object with respect to the predetermined alarm area in front of the host vehicle. For example, it may be a requirement regarding a spatial or temporal relationship between a plurality of types of objects (for example, people and other vehicles).

  Moreover, you may use together the 1st aspect and 2nd aspect of the periphery monitoring apparatus of the vehicle of this invention. That is, both the detection area and the alarm generation requirement may be variably set according to the determination result of the determination means of the stopped vehicle classification detection device.

  An embodiment of the present invention will be described below with reference to FIGS.

  First, with reference to FIG. 1 and FIG. 2, the system configuration | structure of the vehicle periphery monitoring apparatus of this embodiment is demonstrated. FIG. 1 is a block diagram showing the overall configuration of the periphery monitoring device, and FIG. 2 is a perspective view showing the appearance of a vehicle (own vehicle) equipped with the periphery monitoring device. In FIG. 2, illustration of some components of the periphery monitoring device is omitted.

  With reference to FIGS. 1 and 2, the periphery monitoring device of the present embodiment includes an image processing unit 1. The image processing unit 1 is connected with two infrared cameras 2R and 2L as imaging devices for capturing an image in front of the host vehicle 1, and the host vehicle 10 is used as a sensor for detecting the traveling state of the host vehicle 10. A yaw rate sensor 3 for detecting the yaw rate of the vehicle, a vehicle speed sensor 4 for detecting the traveling speed (vehicle speed) of the host vehicle 10, and a brake for detecting a brake operation of the host vehicle 10 (specifically, whether or not the brake pedal is operated). The sensor 5 is connected. Further, the image processing unit 1 includes a speaker 6 for outputting auditory alarm information such as sound, and a display device for displaying images taken by the infrared cameras 2R and 2L and visual alarm information. 7 is connected.

  Although not shown in detail, the image processing unit 1 is composed of an electronic circuit including an A / D conversion circuit, a microcomputer (CPU, RAM, ROM), an image memory, and the like. The infrared cameras 2R and 2L, the yaw rate sensor 3. Outputs (analog signals) of the vehicle speed sensor 4 and the brake sensor 5 are digitized and inputted via an A / D conversion circuit. Then, the image processing unit 1 determines whether a process for detecting an object such as a person (pedestrian) or a predetermined alarm generation requirement regarding the detected object is satisfied based on the input data. When the alarm generation requirement is satisfied, a process for issuing an alarm to the driver via the speaker 6 and the display device 7 is executed by the microcomputer.

  Therefore, the image processing unit 1 has functions as the object detection means and alarm generation means in the present invention. Further, the image processing unit 1 includes a function as a stopped vehicle classification detection device of the present invention (more specifically, functions of a stopped vehicle detection means, a temperature state detection means, and a determination means).

  As shown in FIG. 2, the infrared cameras 2 </ b> R and 2 </ b> L are attached to the front portion (front grill portion in the figure) of the host vehicle 10 in order to image the front of the host vehicle 10. In this case, the infrared cameras 2R and 2L are respectively disposed at a position on the right side and a position on the left side of the center of the host vehicle 10 in the vehicle width direction. These positions are symmetrical with respect to the center of the host vehicle 10 in the vehicle width direction. The infrared cameras 2R and 2L have their optical axes extending in the front-rear direction of the host vehicle 10 in parallel with each other, and the heights of the respective optical axes from the road surface are equal to each other. It is fixed to the front of the. Each infrared camera 2R, 2L is an imaging device having sensitivity in the far-infrared region, and the higher the temperature of the object imaged by the infrared camera 2R, 2L, the higher the level of the output signal of the object image (the object's image). (The brightness of the image is increased).

  In the present embodiment, the display device 7 includes a head-up display 7a (hereinafter referred to as a HUD 7a) that displays information such as an image on the front window of the host vehicle 10, for example. The display device 7 may be a display provided integrally with a meter that displays a traveling state such as the vehicle speed of the host vehicle 10 in place of the HUD 7a or together with the HUD 7a, or a display provided in an in-vehicle navigation device. May be included.

  Next, the overall operation of the periphery monitoring apparatus of this embodiment will be described with reference to the flowcharts of FIGS. Note that many of the processes in the flowcharts of FIGS. 3 and 4 are the same as the processes described in FIG. 3 of Patent Document 2, for example. Detailed description is omitted.

  First, the image processing unit 1 acquires infrared images that are output signals of the infrared cameras 2R and 2L (STEP 1), performs A / D conversion (STEP 2), and stores each image in an image memory (STEP 3). ). As a result, images captured by the infrared cameras 2R and 2L are taken into the image processing unit 1. Hereinafter, an image obtained from the infrared camera 2R is referred to as a right image, and an image obtained from the infrared camera 2L is referred to as a left image. These right image and left image are both grayscale images. Note that the processing of STEPs 1 to 3 is the same as the processing of S1 to S3 in FIG.

  Next, the image processing unit 1 uses one of the right image and the left image as a reference image, and binarizes this reference image (STEP 4). The reference image is a right image in the present embodiment. In this binarization process, the luminance value of each pixel of the reference image is compared with a predetermined luminance threshold value, and an area (relatively bright area) having a luminance value higher than the predetermined luminance threshold value in the reference image. This is a process of setting “1” (white) and setting an area having a luminance value lower than the luminance threshold (a relatively dark area) to “0” (black). Hereinafter, an image (monochrome image) obtained by the binarization process is referred to as a binarized image. In the binarized image, an area “1” is referred to as a high luminance area. The binarized image is stored in the image memory separately from the grayscale image (right image and left image).

  Supplementally, in the present embodiment, the binarization process is performed in an object extraction image area set in STEPs 18 and 19 described later in the reference image. In the present embodiment, the object extraction image area is set to either the entire area of the reference image or an area smaller than the entire area.

  Next, the image processing unit 1 performs the processing of STEPs 5 to 7 on the binarized image, and extracts an object (more precisely, an image portion corresponding to the object) from the binarized image. That is, the pixel group constituting the high luminance area of the binarized image is classified into lines having a width of one pixel in the vertical direction (y direction) of the reference image and extending in the horizontal direction (x direction). Each line is converted into run-length data composed of coordinates and length (number of pixels) of the position (two-dimensional position on the reference image) (STEP 5). Of the lines represented by the run length data, labels (identifiers) are attached to the respective line groups that overlap in the vertical direction of the reference image (STEP 6), and each of the line groups is extracted as an object. (STEP 7).

  The objects extracted by the processing in STEPs 5 to 7 generally include not only people (pedestrians) but also artificial structures such as other vehicles. In addition, a plurality of local parts of the same object may be extracted as the target object.

  Next, the image processing unit 1 obtains the position of the center of gravity of each object extracted as described above (position on the reference image), the area, and the aspect ratio of the circumscribed rectangle (STEP 8). The position of the center of gravity of each object is obtained by multiplying the coordinates of the position of each line (the center position of each line) of the run length data included in the object by the length of the line. It is obtained by adding all the lines of included run length data and dividing the addition result by the area of the object. Further, instead of the center of gravity of each object, the position of the center (center) of the circumscribed rectangle of the object may be obtained.

  Next, the image processing unit 1 traces the object extracted in STEP 7 for the time, that is, recognizes the same object for each calculation processing period of the image processing unit 1 (STEP 9). In this processing, when the object A is extracted by the processing of STEP 7 at a time (discrete system time) k in a certain arithmetic processing cycle, and the object B is extracted by the processing of STEP 7 at the time k + 1 of the next arithmetic processing cycle. The identity of these objects A and B is determined. This identity determination may be performed based on, for example, the shape and size of the objects A and B on the binarized image, the correlation of the luminance distribution on the reference image (grayscale image), and the like. . When it is determined that the objects A and B are the same, the label of the object B extracted at time k + 1 (the label attached in STEP 6) is changed to the same label as the label of the object A. The

  Note that the processing in STEPs 5 to 9 described above is the same as the processing in S5 to S9 in FIG. Moreover, the object detection means in this invention is comprised by the process of STEP1-9.

  Next, the image processing unit 1 reads the outputs (vehicle speed detection value and yaw rate detection value) of the vehicle speed sensor 4 and the yaw rate sensor 5 (STEP 10). In STEP 10, the turning angle (azimuth angle) of the host vehicle 10 is also calculated by integrating the read detection value of the yaw rate.

  On the other hand, the image processing unit 1 executes the processes of STEPs 11 to 13 in parallel with the processes of STEPs 9 and 10. The processing of STEP 11 to 13 is processing for obtaining the distance of each object extracted in STEP 7 from the host vehicle 10, and is the same as the processing of S11 to S13 of FIG. The process is schematically described. First, an area corresponding to each object (for example, a circumscribed square area of the object) in the right image (reference image) is extracted as a search image R1 (STEP 11).

  Next, in the left image, a search area R2 that is an area for searching for the same object as the object included in the search image R1 of the right image is set, and the correlation with the search image R1 is set in the search area R2. The region having the highest probability is extracted as a corresponding image R3 that is an image corresponding to the search image R1 (an image equivalent to the search image R1) (STEP 12). In this case, an area having a luminance distribution that most closely matches the luminance distribution of the search image R1 of the right image is extracted as the corresponding image R3 from the search area R2 of the left image. Note that the processing in STEP 12 is performed using a grayscale image, not a binarized image.

  Next, the number of pixels of the difference between the lateral position (x-direction position) of the center of gravity of the search image R1 in the right image and the lateral position (x-direction position) of the center of gravity of the corresponding image R3 in the left image is defined as parallax Δd. The distance z of the object from the host vehicle 10 (the distance in the front-rear direction of the host vehicle 10) is calculated using the parallax Δd (STEP 13). The distance z is calculated by the following equation (1).

z = (f × D) / (Δd × p) (1)

Note that f is the focal length of the infrared cameras 2R and 2L, D is the base length (interval of the optical axis) of the infrared cameras 2R and 2L, and p is the pixel pitch (length of one pixel).

  The above is the outline of the processing of STEPs 11 to 13. In addition, the process of STEP11-13 is performed with respect to each target object extracted by said STEP7.

  After the processing of STEP 10 and STEP 13, the image processing unit 1 next calculates a real space position that is a position (relative position with respect to the host vehicle 10) of each object in the real space (STEP 14). Here, as shown in FIG. 2, the real space position is a position (X, Y, Z) in the real space coordinate system (XYZ coordinate system) set with the midpoint of the attachment position of the infrared cameras 2R, 2L as the origin. ). The X direction and the Y direction of the real space coordinate system are the vehicle width direction and the vertical direction of the host vehicle 10, respectively. These X direction and Y direction are the x direction (lateral direction) and y of the right image and the left image, respectively. It is the same direction as the direction (vertical direction). The Z direction in the real space coordinate system is the front-rear direction of the host vehicle 10. Then, the real space position (X, Y, Z) of the object is calculated by the following equations (2), (3), (4).

X = x × z × p / f (2)
Y = y × z × p / f (3)
Z = z (4)

Note that x and y are the x-coordinate and y-coordinate of the object on the reference image.

  Next, the image processing unit 1 compensates for the influence of the change in the turning angle of the host vehicle 10 and increases the accuracy of the real space position of the target object, so that the real space position (X, Y, Z) of the target object is increased. The position X in the X direction is corrected according to the time-series data of the turning angle obtained in STEP 10 from the value obtained by the above equation (2) (STEP 15). Thereby, the real space position of the object is finally obtained. In the following description, “the real space position of the object” means the real space position of the object subjected to this correction.

  Note that the processing in STEPs 14 and 15 is the same as the processing in S14 and S15 in FIG.

  Next, the image processing unit 1 obtains a movement vector of the object relative to the host vehicle 10 (STEP 16). Specifically, a straight line that approximates time-series data in a predetermined period (a period from the current time to a predetermined time) of the real space position of the same object is obtained, and the straight line at the time before the predetermined time is obtained. A vector from the position (point) of the object toward the position (point) of the object on the straight line at the current time is obtained as a movement vector of the object. This movement vector is proportional to the relative velocity vector of the object with respect to the host vehicle 10. Note that the processing of STEP 16 is the same as the processing of S16 of FIG.

  Next, the image processing unit 1 temporarily moves in front of the host vehicle 10 (more specifically, a spatial region in front of the host vehicle 10 corresponding to the object extraction image region in which the target is extracted in STEP 7). It is determined whether or not there is a stationary vehicle (STEP 17). Then, in accordance with the determination result of STEP 17, a width of an alarm region described later and an object extraction image region to be binarized in the subsequent processing of STEP 4 are set (STEPs 18 and 19). Further, the image processing unit 1 executes an alarm determination process for determining whether or not the extracted object satisfies a predetermined alarm generation requirement (STEP 20). Details of the processing of STEPs 17 to 20 will be described later.

  Then, the image processing unit 1 determines in the alarm determination process of STEP 20 that none of the objects satisfies the alarm generation requirement (there is no object corresponding to the alarm generation requirement) (the determination result of STEP 20 is NO) In such a case, the processing from STEP1 is resumed. If it is determined in STEP 20 that any of the objects satisfies the alarm generation requirement (when the determination result in STEP 20 is NO), the process proceeds to STEP 21 and an actual alarm regarding the object that satisfies the alarm generation requirement. A warning output determination process for determining whether or not should be performed is executed (STEP 21). In this alarm output determination process, it is confirmed from the output of the brake sensor 5 that the driver has operated the brake of the host vehicle 10, and the deceleration acceleration of the host vehicle 10 (the acceleration in the vehicle speed decreasing direction is corrected). ) Is larger than a predetermined threshold (> 0), it is determined that no alarm is given. In addition, when the driver does not perform a brake operation or when the deceleration acceleration of the host vehicle 10 is equal to or less than a predetermined threshold even if the brake operation is performed, it is determined that an alarm should be issued. .

  When the image processing unit 1 determines that an alarm should be issued (when the determination result in STEP 21 is YES), the image processing unit 1 issues an alarm by the speaker 6 and the display device 7 to the driver of the host vehicle 10. The alarm generation process to be issued is executed (STEP 22), and the process from STEP 1 is resumed. In this alarm generation process, for example, the reference image is displayed on the display device 7, and an image of an object that satisfies the alarm generation requirement in the reference image is highlighted. Furthermore, a voice guidance is provided from the speaker 6 to the driver that such an object exists. This alerts the driver to the object. Note that the warning for the driver may be performed by only one of the speaker 6 and the display device 7.

  Further, when it is determined in STEP 21 that no warning is given (when it is determined that no warning is given for all objects), the determination result in STEP 21 is NO, and in this case, the processing from STEP 1 is resumed as it is. Is done.

  The above is the overall operation of the periphery monitoring device of this embodiment.

  Next, the processing of STEP17 to STEP20 which will be described later will be described in detail below. In the following description, a specific description will be given by taking the situation shown in FIG. 5 as an example as appropriate. In this example, another vehicle 22 that is stopped on the traveling lane 21 of the host vehicle 10 exists diagonally left front of the host vehicle 10 and is traveling in a direction approaching the host vehicle 10 on the opposite lane 23. There is another vehicle 24. Furthermore, people (pedestrians) P1 and P2 represented by circles with diagonal lines exist diagonally left front of the host vehicle 10. Further, an area in front of the host vehicle 10 (hereinafter referred to as an imaging monitoring area) imaged by the infrared cameras 2R and 2L is generally an area within the triangle abc in FIG. The other vehicles 22 and 24 and the persons P1 and P2 exist inside.

  Supplementally, the imaging monitoring area abc is more specifically from the own vehicle 10 in the spatial area in front of the own vehicle 10 (the area within the viewing angle of the infrared cameras 2R and 2L) imaged by the infrared cameras 2R and 2L. Is a region whose distance (distance in the Z direction of the real space coordinate system) is a predetermined value or less. The angle formed by the line segments ab and ac in the imaging monitoring area abc corresponds to the viewing angle of the infrared cameras 2R and 2L. In the present embodiment, an object that can be extracted as an object is an object that exists in the imaging monitoring area abc.

  In the situation shown in FIG. 5, for example, the right image (reference image) of the right image and the left image is schematically illustrated. As shown in FIG. 6, as shown in FIG. The image includes P2. In addition, the image shown in FIG. 6 represents each other vehicle 22, 24, person P1, P2, etc. as an image with which a clear outline appears for convenience of explanation. However, since the image is a gray scale image, the contours of the other vehicles 22 and 24 and the people P1 and P2 are not always clear. For example, only a local part such as another vehicle or a person may be a clear image.

  Based on the above description, first, the processing of STEP 17 (processing for determining whether or not a temporarily stopped vehicle exists) out of the processing of STEPs 17 to 20 will be described. FIG. 8 is a flowchart showing this process. The process of STEP 17 is a process that constitutes a stopped vehicle detection unit, a temperature state detection unit, and a determination unit of the stopped vehicle classification detection device of the present invention.

  With reference to FIG. 8, in the process of STEP 17, it is first determined whether or not there is another vehicle among the extracted objects (whether or not there is an object corresponding to the other vehicle) ( (STEP 101). In this determination process, a process of detecting another vehicle (or its local part) is executed from the extracted objects, and if no other vehicle is detected by the detection process, it is determined that no other vehicle exists. The

  In this case, the process for detecting the other vehicle may use a known method. In the present embodiment, for example, the technique described in paragraphs 0038 to 0053 of Patent Document 1 is used. The method is not described in detail in this specification, but the outline is as follows.

  That is, a set of objects having a symmetrical positional relationship (hereinafter referred to as a symmetrical object pair) is searched for among the objects extracted from the binarized image in STEP8. The symmetry object pair has a distance (Z-direction position in the real space coordinate system) from the host vehicle 10 and a height (Y-direction position in the real space coordinate system) that are substantially the same as each other. Is a set of objects such that the interval (space in the X direction in the real space coordinate system) is within a predetermined range corresponding to the vehicle width of a general other vehicle, This corresponds to a pair of left and right taillights.

  In this case, in the situation shown in FIG. 5, it is assumed that the other vehicle 22 diagonally left front of the host vehicle 10 is a vehicle that is temporarily stopped, for example, while driving a motor (usually an engine). In this case, for this other vehicle 22, for example, objects OBJ1 to OBJ5 shown in FIG. 7 are extracted in the binarized image. OBJ1 and 2 correspond to the tail lamps of the other vehicle 22, OBJ3 corresponds to the vicinity of the exhaust pipe of the other vehicle 22, and OBJ4 and 5 correspond to the left and right rear tires of the other vehicle 22.

  At this time, among these objects OBJ1 to OBJ5, OBJ1 and OBJ1 and OBJ1 and OBJ2 are extracted as symmetrical object pairs corresponding to the pair of left and right taillights of the other vehicle 22.

  When a symmetrical object pair is extracted, a predetermined search area MASK1 (see FIG. 5) above the symmetrical object pair (a part corresponding to a window shield of another vehicle) in the right image (reference image). 7), and calculates the distance (distance from the vehicle 10 in the real space coordinate system) and the average luminance value of the search area MASK1, and based on these calculated values, the object of symmetry It is determined whether or not a window shield of another vehicle exists above the object pair. Specifically, when the distance of the portion of the search area MASK1 is substantially the same as the distance from the host vehicle 10 of the symmetrical object pair and the average luminance value is smaller than a predetermined value, the search area on the reference image It is determined that the window shield of the other vehicle exists at the location of MASK1. When the presence of the window shield is confirmed above the symmetrical object pair in this way, it is determined that another vehicle exists at the location of the symmetrical object pair, thereby detecting the presence of the other vehicle. The

  If a symmetrical object pair is not extracted, a search area MASK2 (not shown) is set immediately above each object. Based on the distance from the host vehicle 10 corresponding to the search area MASK2 and the average luminance values of the left and right portions of the search area MASK2, the presence or absence of the window shield is determined, and the window shield When the presence is confirmed, it is determined that there is another vehicle including the object. In this case, simultaneously with the confirmation of the presence of the window shield, it is also determined whether the window shield exists in the left-side portion or the right-side portion of the search area MASK2. If the presence of the window shield is not confirmed, it is determined that no other vehicle exists among the objects extracted in STEP 7.

  The above is the outline of the processing of STEP101. By the processing of STEP 101, for example, in the situation shown in FIG. 5, the presence of the other vehicles 22, 24 is detected. The detection of other vehicles is not limited to the above method, and other known methods (for example, a pattern matching method using a template) may be used. In addition, other vehicles may be detected using a visible light camera, a radar, or the like.

  When no other vehicle is detected in STEP 101 (when the determination result in STEP 101 is NO), the image processing unit 1 determines that there is no temporarily stopped vehicle (STEP 105), and ends the processing in STEP 17. To do. When at least one other vehicle is detected (when the determination result in STEP 101 is YES), it is next determined whether or not the detected other vehicle is stopped (STEP 102).

  This determination is performed for all detected other vehicles as follows. That is, the relative position (actual space position of the object) of at least one object (for example, OBJ1 or OBJ2 in FIG. 7) included in each other vehicle with respect to the own vehicle 10 or the distance in the front-rear direction from the own vehicle 10 Based on the time-series data of (the position in the Z direction of the object in the real space coordinate system), the relative speed of the other vehicle with respect to the host vehicle 10 is calculated. When the relative speed is a speed toward the host vehicle 10 and the magnitude (absolute value) is substantially the same as the detected value of the vehicle speed of the host vehicle 10 obtained by the vehicle speed sensor 4 ( When the absolute value of the difference from the detected value is equal to or less than a predetermined value near 0), it is determined that the other vehicle is stopped. In other cases, it is determined that the other vehicle has not stopped. Note that the relative speed of the other vehicle may be measured using a radar or the like.

  The processing of STEP 101 and 102 corresponds to the stopped vehicle detection means of the stopped vehicle classification detection device of the present invention, and other vehicles that are stopped are detected by the processing of STEP 101 and 102.

  When the image processing unit 1 determines in STEP 102 that the other vehicle has not stopped (all other vehicles detected in STEP 101 have not stopped) (when the determination result in STEP 102 is NO), Proceeding to STEP 105, it is determined that there is no temporarily stopped vehicle, and the processing of STEP 17 is terminated. Further, when it is determined that at least one of the detected other vehicles is stopped (when the determination result in STEP 102 is NO), the other vehicle is temporarily stopped. It is determined whether or not (STEP 103).

  This process is performed as follows. That is, referring to FIG. 7, the image processing unit 1 relates to each other vehicle determined to be stopped, and an image of a predetermined part of the other vehicle is displayed on the reference image (grayscale image). A plurality of (four in this embodiment) areas AREA1 to AREA4 that are estimated to be included are set as luminance determination areas. Here, the luminance determination areas AREA 1 and 2 are set so that the image of the exhaust pipe portion of the other vehicle and the image of the prime mover mounting portion of the other vehicle are included therein, and the luminance determination areas AREA 3 and 4 The left and right tires are set to be included. Note that these brightness determination areas AREA1 to AREA4 are, for example, the positions of the symmetrical object pairs OBJ1 and OBJ1 and OBJ2 extracted in STEP 101 on the reference image, and the symmetrical object pairs OBJ1 and OBJ1 from the host vehicle 10. Based on the distance and height, the position of the exhaust pipe portion of the other vehicle may be roughly estimated and set near the estimated position. In addition, the sizes of the luminance determination areas AREA1 to AREA4 are set according to the distance so as to decrease as the distance from the host vehicle 10 to the other vehicle increases.

  Then, the image processing unit 1 calculates the luminance average value in each of the luminance determination areas AREA1 to AREA4 on the reference image (grayscale image). Here, since the reference image is a far-infrared image, the average luminance value in each luminance determination area AREA1 to AREA4 corresponds to the temperature (average temperature) of the object existing in each luminance determination area AREA1 to AREA4. The higher the temperature, the higher the brightness average value. Also, when the other vehicle is a temporarily stopped vehicle, the heat associated with the operation of the motor (usually the engine) of the other vehicle, or the other vehicle's Due to frictional heat between the tire and the road surface during traveling before the stop, the temperatures of the exhaust pipe portion, the prime mover mounting portion, and the tire portion of the other vehicle are higher than the temperatures of other portions of the other vehicle. Therefore, when these high-temperature portions are present in the luminance determination areas AREA1 to AREA4 on the reference image, the average luminance values of the luminance determination areas AREA1 to AREA4 are relatively high.

  Therefore, in the present embodiment, the average luminance value of each of the luminance determination areas AREA1 to AREA4 is obtained as indicating the temperature state of the corresponding part of the other vehicle. Then, each of the obtained luminance average values is compared with a predetermined luminance threshold value, and if any luminance average value in the luminance determination areas AREA1 to AREA4 is higher than the luminance threshold value, the other vehicle is temporarily stopped. Judged to be a vehicle. Further, in any of the luminance determination areas AREA1 to AREA4, when the average luminance value is equal to or lower than the luminance threshold, it is determined that the other vehicle is not a temporarily stopped vehicle.

  Specifically, in the example of FIG. 5, when the other vehicle 22 diagonally left front of the host vehicle 10 is a temporarily stopped vehicle, the areas of the brightness determination areas AREA 1 to AREA 4, AREA 1, 3, Any one of the luminance average values of 4 exceeds the luminance threshold value. For this reason, it is determined that the other vehicle 22 is a temporarily stopped vehicle. In the example of FIG. 5, the other vehicle 22 is a rear-view vehicle, and thus the luminance threshold value of the luminance determination area AREA2 corresponding to the prime mover mounting unit is generally lower than the luminance threshold value.

  On the other hand, if the other vehicle 22 is not a temporarily stopped vehicle, that is, if it is a vehicle that is regularly parked for a sufficiently long time with the operation of the prime mover stopped, Since the overall temperature is relatively low, the average luminance value is lower than the luminance threshold value in any of the luminance determination areas AREA1 to AREA4. For this reason, it is determined that the other vehicle 22 is not a temporarily stopped vehicle.

  Supplementally, in the situation of FIG. 5, if the other vehicle 24 on the opposite oblique line 23 side is a temporarily stopped vehicle, the luminance in the luminance determination areas AREA1 to AREA4 regarding the other vehicle 24 will be described. The average luminance value of any one of the determination areas AREA2 to 4 becomes higher than the luminance threshold value, whereby it is possible to determine that the other vehicle 24 is a temporarily stopped vehicle.

  In STEP 103, the process for obtaining the average brightness value of the brightness determination areas AREA1 to AREA4 as described above corresponds to the temperature state detection means of the stopped vehicle classification detection device of the present invention. In this case, the luminance determination areas AREA1 to AREA4 correspond to the stopped vehicle image. Then, in STEP 103, the process for determining whether or not the other stopped vehicle is a temporarily stopped vehicle based on the average luminance values of the luminance determination areas AREA1 to AREA4 is the stopped vehicle classification detection device of the present invention. It corresponds to the determination means.

  In this embodiment, the luminance average value of the luminance determination areas AREA1 to AREA4 is used to determine whether or not the other stopped vehicle is a temporarily stopped vehicle. Instead, for example, the maximum luminance value in the luminance determination areas AREA1 to AREA4 may be used. Further, a part of the luminance determination areas AREA1 to AREA4, for example, only AREA1 and 2, or only AREA3 and 4 may be used. Further, the luminance threshold may be set for each of the luminance determination areas AREA1 to AREA4, and may be changed according to the temperature or the like.

  The image processing unit 1 determines in STEP 103 that the other vehicle stopped as described above (at least one of the other vehicles determined to be stopped in STEP 103) is a temporarily stopped vehicle. In the case (when the determination result in STEP 103 is YES), the process proceeds to STEP 104, where it is determined that there is a temporarily stopped vehicle, and the processing in STEP 17 is terminated. If it is determined that all other vehicles determined to be stopped in STEP 103 are not temporarily stopped vehicles (when the determination result in STEP 103 is NO), the process proceeds to STEP 105 to temporarily stop. It is determined that there is no vehicle, and the processing of STEP 17 ends.

  The above is the details of the process of determining whether or not there is a temporarily stopped vehicle in STEP 17.

  Next, details of the processing of STEPs 18 and 19 will be described. As described above, the image processing unit 1 executes the process of setting the width of the alarm area and the object extraction image area in STEPs 18 and 19 according to the determination result in STEP17.

  Here, the warning area and the object extraction image area will be described. First, since the object extraction image area is an area on the reference image to be binarized in the subsequent processing of STEP 4 as described above, the extraction of the object in STEP 7 is performed using this object extraction image area. Is done within. In other words, the object extraction image area is an area in the reference image for searching for an object that can be an alarm target. In this embodiment, the object extraction image area to be set includes a rectangular area S1 (hereinafter referred to as a standard image area S1) indicated by a broken line frame in the reference image (right image) shown in FIG. There are two types: an entire area S2 of the image (hereinafter, an enlarged image area S2). In the present embodiment, the standard image region S1 is a region whose center point is the same as the center point of the enlarged image region S2, and whose vertical and horizontal widths are smaller than the enlarged image region S2 by a predetermined ratio. The standard image area S1 may be set to an area in which only the width in the horizontal direction is smaller than that of the enlarged image area S2.

  Supplementally, when the object extraction image area is changed, a spatial area (hereinafter referred to as an object detection space area) for detecting an object that can be an alarm target in the imaging monitoring area in front of the host vehicle 10 is changed. It will be. Specifically, referring to FIG. 6, when the enlarged image area S2 is set as the object extraction image area, the corresponding object detection space area is shown in FIG. 5 in plan view. It matches the region abc. When the standard image area S1 is set as the object extraction image area, the corresponding object detection space area is reduced to a triangular area ade in the imaging monitoring area abc shown in FIG. 5 in plan view. The Therefore, the object detection space area is also changed by changing the object extraction image area.

  The warning area is a spatial area in front of the host vehicle 10 related to one of the warning generation requirements. This warning area is an area where it is determined that there is a high possibility that the object and the host vehicle 10 will collide when an object exists in the area. The alarm area set in the present embodiment is a standard alarm area afghhi that is an area of the pentagon afghhi in FIG. 5 and an enlarged alarm area that is an area of the pentagon ajkmn in the same figure that includes the standard alarm area afghhi in plan view. There are two types, ajkmn. Here, in the object detection space area abc or ade, the distance in the front-rear direction (Z direction in the real space coordinate system) from the own vehicle 10 is the relative speed Vs of the object with respect to the own vehicle 10 (the own vehicle 10 An area (triangle aop or aqr area in FIG. 5) that is equal to or smaller than a predetermined value Vs · T (T: a positive constant in the dimension of time) according to the front-rear direction relative speed) is referred to as a collision determination area. In other words, the collision determination areas aop and aqr are similar in shape to the object detection space areas abc and ade, respectively, in the plan view, and the length in the front-rear direction of the host vehicle 10 for each object. This is an area set to a value Vs · T proportional to the relative speed Vs of the object with respect to the host vehicle 10. In FIG. 5, the collision determination areas aop and aqr are areas corresponding to the person P1 or P2, for example.

  In the present embodiment, one alarm generation requirement is that the object exists in the collision determination area aop or aqr. The standard alarm area afghi and the enlarged alarm area ajkmn are areas set in the collision determination areas aqr and aop, respectively. Hereinafter, the collision determination area aqr is referred to as a standard collision determination area aqr, and the collision determination area aop is referred to as an enlarged collision determination area.

  More specifically, the standard warning area afghhi is parallel to the boundary lines aq, ar, qr of the standard collision determination area aqr and the vehicle width center line L of the host vehicle 10 and to the left and right of the vehicle width center line L. This is an area surrounded by the vehicle width center line L and two straight lines (straight lines each including line segments fg and hi) having a predetermined interval Wa / 2. In this case, the width Wa of the standard alarm area afghhi is set to be slightly larger than the vehicle width of the host vehicle. Wa is smaller than the length of the bottom side (line segment qr) of the standard collision determination area aqr.

  Further, the enlarged warning area ajkmn is parallel to the boundary lines ao, ap, op of the enlarged collision determination area aop and the vehicle width center line L of the host vehicle 10, and the vehicle width on the left and right of the vehicle width center line L. This is an area surrounded by a center line L and two straight lines (straight lines each including line segments jk and mn) having a predetermined interval Wb / 2. In this case, the width Wb of the enlarged alarm area ajkmn is set to be larger than the width Wa of the standard alarm area afghhi. Note that Wb is smaller than the length of the bottom side (line segment op) of the enlarged collision determination area aop.

  STEP 18 is a process when it is determined in STEP 17 that there is a temporarily stopped vehicle, and STEP 19 is a process when it is determined in STEP 17 that there is no temporarily stopped vehicle.

  In STEP 18, the image processing unit 1 sets the width Wb of the enlarged warning area ajkmn as the width of the warning area, and sets the enlarged image area S2 as the target extraction image area (target extraction space). The area abc is set as the area). In STEP 19, the width Wa of the standard alarm area afghhi is set as the width of the alarm area, and the standard image area S1 is set as the object extraction image area (the area ade is set as the object extraction space area). To do). Thereby, the width | variety of a warning area | region and the target object extraction space area | region will be changed variably by whether the temporary stop vehicle exists ahead of the own vehicle 10. FIG. In this case, both the warning area and the object extraction space area are expanded in the vehicle width direction of the host vehicle 10 when there is no temporarily stopped vehicle. .

  Supplementally, the processing of STEPs 18 and 19 corresponds to detection area setting means and alarm generation requirement setting means of the periphery monitoring device of the present invention.

  Next, the alarm determination process in STEP 20 will be described. FIG. 9 is a flowchart showing this alarm determination processing.

  In the present embodiment, for each extracted object, it is determined whether or not the object satisfies the alarm generation requirement as described below. First, in STEP 120, it is determined whether or not the object exists in the collision determination area. Specifically, the relative speed Vs of the object with respect to the host vehicle 10 is calculated based on the time series data of the real space position (position in the Z direction) of the object. Then, depending on whether or not the Z position (the distance from the vehicle 10 in the Z direction) of the current real space position of the object is equal to or less than the value obtained by multiplying the relative speed Vs by a constant T, Is present in the collision determination area determined as described above. If the relative speed Vs of the object is a speed away from the host vehicle 10, it is determined that the object does not exist in the collision determination area. Further, even when the height position of the target object (Y direction position in the real space coordinate system) exceeds a predetermined height (a height higher than the vehicle height of the host vehicle 10), the target object is in the collision determination area. Is determined not to exist.

  Supplementally, the collision determination area in this case is the standard collision determination area aqr when the current object extraction image area is set to the standard image area S1, and the current object extraction image area is When it is set in the enlarged image area S2, it is the enlarged collision determination area aop. Therefore, in the situation of FIG. 5, when the object extraction image area is set to the standard image area S1, it is determined that the person P1 out of the persons P1 and P2 exists in the collision determination area, but the person P2 Is determined not to exist in the collision determination area. On the other hand, when the object extraction image area is set to the enlarged image area S2, it is determined that both the persons P1 and P2 exist in the collision determination area. In the description here, it is assumed that the relative speeds Vs regarding the persons P1 and P2 are substantially the same.

  When the object does not exist in the collision determination area (when the determination result of STEP 120 is NO), the process proceeds to STEP 126, where it is determined that the object does not satisfy the alarm generation requirement, and the alarm determination process of STEP 20 Ends.

  If the object exists in the collision determination area in STEP 120 (when the determination result in STEP 120 is YES), it is next determined whether or not the object exists in the alarm area (STEP 121). . Specifically, the X position in the real space coordinate system of the object (the position in the vehicle width direction of the host vehicle 10) is the left and right of the warning area determined by the width Wa or Wb set in STEP 18 or 19. Whether or not the object is present in the alarm area is determined based on whether or not the object exists in the range between the boundary lines.

  In this case, for example, in the situation shown in FIG. 5, when it is determined in STEP 17 that the other vehicle 22 is a temporarily stopped vehicle, the warning area is the enlarged warning area ajkmn. Are a straight line including the line segment jk and a straight line including the line segment mn. Further, when it is determined in STEP 17 that the other vehicle 22 is not a temporarily stopped vehicle, the warning area is the standard warning area afghhi, so the left and right boundary lines of this warning area include the line segment fg. A straight line including a straight line and a line segment hi. Therefore, when the other vehicle 22 is a temporarily stopped vehicle, the object extraction image area (the image area that is binarized in STEP 4 in the current calculation processing cycle of the image processing unit 1) is an enlarged image area. If it is set to S2, it is determined that both of the people P1 and P2 are present in the alarm area. On the other hand, when the other vehicle 22 is not a temporarily stopped vehicle and the object extraction image area is set to the standard image area S2 in the processing of STEP4 in the current calculation processing cycle, the person P1, It is determined that both of P2 do not exist in the alarm area.

  If it is determined in STEP 121 that the object is present in the alarm area (when the determination result in STEP 121 is YES), the process proceeds to STEP 122, and the image processing unit 1 determines whether or not the object is a person. (More precisely, whether or not there is a high possibility of being a person). Whether or not the object is a person may be determined by using a known method, and the characteristics of the object are determined from characteristics such as the shape and size of the object on the gray scale image and the luminance distribution. do it.

  If it is determined in step 122 that the object is likely to be a person, the process proceeds to step 123, where it is determined whether the object is an artificial structure. In this case, the object determined to be another vehicle in the processing of STEP 17 is determined to be an artificial structure. In addition to other vehicles, an object such as a utility pole may be determined as an artificial structure.

  If it is determined in STEP 122 that the object is a person (it is determined that there is a high possibility of being a person) and it is determined in STEP 123 that the object is not an artificial structure, the object is It is determined that the alarm generation requirement is satisfied (STEP 124). If it is determined in STEP 122 that the object is not a person, or if it is determined in STEP 123 that the object is an artificial structure, the process proceeds to STEP 126 and the object does not satisfy the alarm generation requirement. Judge.

  On the other hand, if it is determined in STEP 121 that the object does not exist in the warning area, the image processing unit 1 performs an approach collision determination (STEP 125). In this approach collision determination, the possibility of a collision with the host vehicle 10 is determined based on the movement vector of the object obtained in STEP16. Specifically, assuming that the movement vector of the object is maintained as it is, the X direction position of the intersection of the straight line including this movement vector and the XY plane of the real space coordinate system at the front end position of the host vehicle 10 is Desired. And it is a requirement (hereinafter referred to as the following) that the obtained X direction position exists in a predetermined range (a range slightly wider than the vehicle width of the own vehicle 10) centered on the X direction position of the vehicle width center line of the own vehicle 10. As the entry collision requirement), it is determined whether or not the object is highly likely to collide with the host vehicle 10. That is, when the entry collision requirement is satisfied, it is determined that there is a high possibility that the target object enters the warning area and collides with the own vehicle 10, and otherwise, there is a possibility that the object collides with the own vehicle 10. It is judged that there is no or low.

  Then, if it is determined in STEP 125 that the object is highly likely to collide with the host vehicle 10 (when the determination result in STEP 125 is YES), the process proceeds to STEP 124, where the object is detected. It is judged that the alarm generation requirement is satisfied. Further, when it is determined in STEP 125 that the collision of the object is not likely to collide with the host vehicle 10 (when the determination result in STEP 125 is NO), the process proceeds to STEP 126 and the object is detected. Is determined not to meet the alarm requirements.

The above is the details of the warning determination process of STEP20. In this alarm judgment process,
(1) The object exists in the alarm area, and the type of the object is a person.

(2) The object is present in an area other than the warning area within the collision determination area, and satisfies the approach collision requirement.
That is, it is determined whether or not this alarm generation requirement is satisfied.

  In this case, in the situation of FIG. 4, when the other vehicle 22 diagonally to the left of the host vehicle 10 is not a temporarily stopped vehicle, the people P1 and P2 do not exist in the warning area (standard warning area afghhi). Unless the persons P1 and P2 satisfy the entry collision requirement, it is determined that the persons P1 and P2 do not satisfy the alarm generation requirement. On the other hand, when the other vehicle 22 is a temporarily stopped vehicle, the person P1, from the calculation processing cycle next to the calculation processing cycle first determined that the vehicle 22 is a temporarily stopped vehicle. Since P2 exists in the warning area (enlarged warning area ajkmn), it is determined that those persons P1 and P2 satisfy the warning generation requirement.

  As a result, when it is determined in the warning output determination process in STEP 21 that a warning should be given, a warning regarding the persons P1 and P2 is issued to the driver.

  Supplementally, the processing of STEP20 to STEP22 corresponds to an alarm generation means in the periphery monitoring device of the present invention.

  As described above, according to the periphery monitoring device of the present embodiment, when a temporarily stopped vehicle is detected, the object extraction image area (and thus the object detection space area) and the alarm area are temporarily displayed. The vehicle 10 is enlarged in the vehicle width direction of the host vehicle 10 as compared with a case where a stationary vehicle is not detected. For this reason, an object such as a person existing in the vicinity of another vehicle, which has not been targeted for warning until now, is targeted for warning. For this reason, the driver | operator of the own vehicle 10 can be alerted to presence of a person etc. over a wider range.

  For example, in the situation of FIG. 5, when the other vehicle 22 is a temporarily stopped vehicle, the person P1 crosses the front of the other vehicle 22 or the people P1 and P2 enter the other vehicle 22. The case where it jumps out to the traveling lane 21 side of the own vehicle 10 etc. is assumed. In addition, it may be assumed that the passenger on the passenger seat gets off the other vehicle 22 and jumps out to the driver's seat because the passenger of the other vehicle 22 changes driving. In such a case, the driver can be alerted to the people P1, P2 and the like that have not been noted by the driver of the host vehicle 10 until now.

  In addition, when there is no temporarily stopped vehicle, the object extraction image area (and thus the object detection space area) and the warning area are compared to the case where the own vehicle 10 is compared to the case where the temporarily stopped vehicle exists. Since the vehicle is reduced in the vehicle width direction, it is possible to prevent a situation in which an alarm is generated more frequently than necessary.

  In the embodiment described above, both the object extraction image area and the width of the alarm area are variably set depending on whether or not a temporarily stopped vehicle exists. Only one of them may be set variably.

  Moreover, in the said embodiment, when the temporarily stopped vehicle is detected, you may make it alarm the driver | operator of the own vehicle 10 about the presence of the stopped vehicle. If it does in this way, a driver's attention to the situation where the crew member of the temporary stop vehicle gets off suddenly can be called. Alternatively, for example, when a temporarily stopped vehicle is detected, it is assumed that the presence of a person in the vicinity for a predetermined period of time is an alarm generation requirement, and an alarm regarding the stopped vehicle and the person is issued to the driver. Good. In this way, in particular, the driver's attention can be alerted to a situation where a person gets on a temporarily stopped vehicle. In this way, the alarm generation requirement when a temporarily stopped vehicle is detected may be set in consideration of the positional relationship and temporal relationship between the stopped vehicle and a person.

  In the above embodiment, when the object extraction image area is changed in STEPs 18 and 19, the object extraction image area after the change is used in the next arithmetic processing cycle of the image processing unit 1. When the object extraction image region is changed in STEPs 18 and 19, the processing from STEP 4 may be performed again within the same calculation processing cycle. In that case, in particular, when the object extraction image area is changed from the standard image area S1 to the enlarged image area S2, the already extracted object is held and the standard image area S1 in the enlarged image area S2 is retained. The object may be newly extracted from only the other area.

  In the embodiment, the enlarged image area S2 of the object extraction image area is enlarged to the left and right sides with respect to the standard image area S1. Alternatively, the temporary stop vehicle may be enlarged more than the other side. Similarly, the enlarged warning area ajkmn is enlarged only on the side where the temporarily stopped vehicle is present on the left and right sides of the standard warning area afjgh, or the side where the temporarily stopped vehicle is present is enlarged more than the other side. You may make it do.

  In the above embodiment, the two infrared cameras 2R and 2L are provided. However, when the distance to the object is detected by a radar or the like, the single infrared camera 2R or 2L is connected to the host vehicle 10. You may make it mount in.

  Furthermore, the detection of the object may be performed using not only the infrared cameras 2R and 2L but also a visible light camera or a radar.

The figure which shows the whole structure of one Embodiment of the periphery monitoring apparatus (a stop vehicle classification | category detection apparatus) of the vehicle of this invention. The perspective view of the vehicle provided with the periphery monitoring apparatus of FIG. 2 is a flowchart showing processing of an image processing unit provided in the periphery monitoring device of FIG. 1. 2 is a flowchart showing processing of an image processing unit provided in the periphery monitoring device of FIG. 1. The figure for demonstrating the process of an image processing unit. The figure which shows the captured image in embodiment exemplarily. The figure which shows typically the example of an image of the other vehicle in embodiment. The flowchart which shows the process of STEP17 of the flowchart of FIG. The flowchart which shows the process of STEP20 of the flowchart of FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image processing unit (stop vehicle detection means, temperature state detection means, determination means, object detection means, alarm generation means, detection area setting means, alarm generation requirement setting means), 2R, 2L ... infrared camera (imaging device) , STEP 1 to 9 ... Object detection means, STEP 101 and 102 ... Stopped vehicle detection means, STEP 103 ... Temperature state detection means and determination means, STEP 18 and 19 ... Detection area setting means and alarm generation requirement setting means, STEP 20 to 22 ... Alarm means .

Claims (10)

It is a stop vehicle classification detection device that distinguishes and detects whether or not another vehicle stopped in front of the host vehicle is a temporary stop vehicle,
An imaging device mounted on the front of the host vehicle and having sensitivity in the far-infrared region;
A stopped vehicle detection means for detecting other vehicles that are stopped from an image ahead of the host vehicle imaged by the imaging device;
When another vehicle that is stopped is detected by the stopped vehicle detection means, based on a stopped vehicle image that is a partial image corresponding to the other vehicle that is stopped, of the images captured by the imaging device, Temperature state detection means for detecting the temperature state of the other vehicle being stopped;
Stop vehicle classification, comprising: a determination unit that distinguishes and determines whether or not the other vehicle being stopped is a temporarily stopped vehicle based on the temperature state detected by the temperature state detection unit Detection device.   The stopped vehicle classification detection device according to claim 1, wherein the temperature state detected by the temperature state detection means includes at least a temperature state of a predetermined part of the other vehicle that is stopped.   The stopped vehicle classification detection device according to claim 2, wherein the predetermined part is at least one of an exhaust pipe, a tire, and a prime mover mounting portion of the other vehicle.   The said temperature state detection means detects the brightness | luminance state of the said stop vehicle image as what shows the temperature state of the said stop vehicle, The stop vehicle classification of any one of Claims 1-3 characterized by the above-mentioned. Detection device. Object detection means for detecting an object present in at least a predetermined detection area in front of the host vehicle, and the object detected by the object detection means satisfy at least a predetermined alarm generation requirement, In the vehicle periphery monitoring device comprising an alarm generating means for issuing an alarm to the driver of the own vehicle,
The stop vehicle classification detection device according to any one of claims 1 to 4, and a detection region setting unit that variably sets the detection region according to a determination result of the determination unit of the stop vehicle classification detection device. A vehicle periphery monitoring device comprising:   The detection area setting means determines whether or not the other vehicle being stopped is detected when it is determined by the determination means of the stopped vehicle classification detection device that the detected other vehicle being stopped is a temporarily stopped vehicle. Alternatively, the detection area is set to expand at least in the width direction of the host vehicle, compared to when it is determined that the detected other stopped vehicle is not a temporarily stopped vehicle. Item 6. A vehicle periphery monitoring device according to Item 5.   The object detection means is means for detecting the object based on an image ahead of the own vehicle amount imaged by the imaging device of the stopped vehicle classification detection device, and the detection area setting means is Of the images captured by the imaging device, the detection region can be variably set by variably setting an image region in which the object is to be detected according to a determination result of the determination unit of the stopped vehicle classification detection device. The vehicle periphery monitoring device according to claim 5 or 6, wherein An object detection means for detecting an object present at least in front of the own vehicle, and a driver of the own vehicle as a necessary requirement that the object detected by the object detection means satisfy at least a predetermined alarm generation requirement In the vehicle periphery monitoring device comprising an alarm generating means for issuing an alarm to
An alarm generation requirement setting that variably sets the alarm generation requirement according to a determination result of the stop vehicle classification detection device according to any one of claims 1 to 4 and the determination unit of the stop vehicle classification detection device. And a vehicle periphery monitoring device. The alarm generation requirement includes a requirement regarding a positional relationship of a warning object detected by the object detection unit with respect to a predetermined warning area in front of the host vehicle,
The alarm generation requirement setting means is means for variably setting the alarm generation requirement by variably setting the alarm area according to the determination result of the determination means of the stopped vehicle classification detection device. The vehicle periphery monitoring device according to claim 8.   The warning generation requirement setting means does not detect the stopped other vehicle when the determining means of the stopped vehicle classification detection device determines that the detected other stopped vehicle is a temporarily stopped vehicle. Alternatively, the warning area is set to be expanded at least in the width direction of the host vehicle as compared with a case where it is determined that the detected other stopped vehicle is not a temporarily stopped vehicle. The vehicle periphery monitoring apparatus according to claim 9.

Images