JP2007323578A - Vehicle periphery monitoring device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle periphery monitoring device reducing the unnecessary attraction of attention and the bothersome attraction of attention for performing the highly reliable attraction of attention to a driver of a vehicle. <P>SOLUTION: This vehicle periphery monitoring device which detects an object existing around the vehicle from images acquired by imaging means 2R, 2L, and 2' mounted to the vehicle 10 is provided with a road area recognition means recognizing a road area in the image (STEP 100), an target extraction means extracting a target from the image (STEP 200), attention attracting means 6 and 7 attracting attention from the driver, and an attention attracting control means outputting first attention from the attention attracting means when the target is extracted in the road area by the target extraction means and outputting second attention different from the first attention from the attention attracting means when the target is extracted outside the road area (STEP 500). <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a vehicle periphery monitoring device that detects an object that affects driving of a vehicle, and more particularly to a vehicle periphery monitoring device that alerts a driver of a vehicle when the object is detected.

  Conventionally, as this kind of vehicle periphery monitoring device, an object such as a pedestrian who may be in contact with a vehicle is extracted from an image around the vehicle captured by an infrared camera, and the information is obtained from the driver of the vehicle. There has been proposed a display processing device that is visually provided (see, for example, Patent Document 1). This device binarizes an infrared image to search for a high brightness area, and the area is determined based on head determination conditions such as the center of gravity position, area ratio, and actual area in the infrared image of the area. It is determined whether or not. And when it determines with it being the area | region of a pedestrian's head, the area | region which includes a pedestrian's body is displayed separately from another area | region. Thereby, pedestrian information is displayed to the driver of the vehicle, and assists the driver's vision.

  Further, in such a vehicle periphery monitoring device, there is provided a vehicle periphery device that visually displays a pedestrian who has a high possibility of contact with the vehicle in order to suppress unnecessary alerting to the driver of the vehicle. It has been proposed (see, for example, Patent Document 2). This device calculates the distance between an object around the vehicle and the vehicle from images obtained by two infrared cameras, and further calculates a movement vector of the object from position data of the object obtained in time series. Yes. Then, an object that is highly likely to come into contact with the vehicle is extracted and displayed from the relationship between the traveling direction of the vehicle and the movement vector of the object. That is, as shown in FIG. 11, an approach determination area AR1 for performing contact determination in front of the vehicle and an intrusion determination area AR2 and AR3 are set, and the object and the vehicle can be contacted from the movement vector of the object in each area. Processing to determine the sex.

However, the inventors of the present application have found that when the possibility of contact is determined by this processing, unnecessary alerting or troublesome alerting may be performed.
Japanese Patent Laid-Open No. 11-328364 JP 2001-6096 A

  The present invention has been made in view of the above background, and provides a vehicle periphery monitoring device that suppresses unnecessary alerts and annoying alerts and provides highly reliable alerts to vehicle drivers. The issue is to provide.

  A vehicle periphery monitoring device according to the present invention for solving the above-described problem is a vehicle periphery monitoring device that monitors an object existing in the periphery of a vehicle from an image obtained by an imaging unit mounted on the vehicle. By the roadway area identification means for identifying the roadway area, the object extraction means for extracting the object from the image, the alerting means for alerting the driver of the vehicle, and the object extraction means, When an object is extracted within the road area identified by the road area identification means, a first alert is output by the alert means, and when the object is extracted outside the road area Is provided with a warning control means for outputting a second warning different from the first warning by the warning means.

  According to such a vehicle periphery monitoring device of the present invention, the vehicle road area identifying means for identifying the road area is provided, and the object extracted from the image is inside or outside the road area identified by the road area identifying means. Depending on whether or not it has been extracted, different warnings of the first warning and the second warning are given to the driver of the vehicle. For this reason, the driver of the vehicle has a possibility that the object exists in the road area and is likely to come into contact with the own vehicle, and that the object exists outside the road area and may come into contact with the own vehicle. The low case can be distinguished and recognized by different alerts. As a result, even when the possibility of contact with the host vehicle is low, the inconvenience of alerting caused by the same alert as when the possibility of contact is high is suppressed, and the driver of the vehicle is thus reliable. High alerts can be made.

  Further, in the vehicle periphery monitoring device of the present invention, the imaging means images the front of the vehicle, and the alert control means is in real space between the vehicle and an object extracted in the road area of the image. Calculating an object distance, which is a distance along the direction of the roadway area ahead, and performing the first alerting by the alerting means when the object distance is equal to or less than a predetermined determination distance; Features.

  According to such a vehicle periphery monitoring device, the object distance calculated by the alerting control unit is a distance between the vehicle and the object in the traveling direction of the vehicle. Since the first alerting is performed only when the object distance is equal to or less than the predetermined distance, the alert is limited to the object located in the traveling direction of the host vehicle, so that unnecessary alerting is performed. Or annoying annoying alerts can be suppressed.

  Furthermore, the vehicle periphery monitoring device of the present invention is a movement vector calculation unit that calculates a movement vector of the same object from a change in position of the same object extracted from a plurality of images acquired at different timings by the imaging unit. And the alerting control means is estimated that the object extracted from the road area by the object extraction means reaches the outside of the road area within a predetermined time from the movement vector of the object. The content of the first alert is changed depending on when it is estimated that it will not reach.

  According to such a vehicle periphery monitoring device, the alerting control means, when the object is extracted in the road area, the object is reached outside the road area within a predetermined time by the movement vector of the object. The content of the first alert is changed according to the estimation result of whether or not. As a result, a similar alert is issued when there is a low possibility that the object will reach the outside of the road area and come into contact with the own vehicle, and when there is a high possibility that the object will not come into the road area and come into contact with the own vehicle. It is possible to suppress the troublesomeness of alerting due to being performed and perform highly reliable alerting to the driver of the vehicle.

  Further, the vehicle periphery monitoring device of the present invention is a movement vector calculation unit that calculates a movement vector of the target object from a change in position of the same target object extracted from a plurality of images acquired at different timings by the imaging unit. The alerting control means is estimated that the object extracted from the outside of the road area by the object extracting means reaches the road area within a predetermined time from the movement vector of the object And the content of said 2nd alerting is changed when it is estimated that it does not reach | attain.

  According to such a vehicle periphery monitoring device, when the target is extracted outside the road area, the alert control means allows the target to reach the road area within a predetermined time based on the movement vector of the target. The content of the second alert is changed according to the estimation result of whether or not. As a result, it is possible to reduce the annoyance of alerting due to the same alerting when the object is likely to enter the roadway area and come into contact with the host vehicle and when the possibility is low. Thus, it is possible to alert the driver of the vehicle with high reliability.

  Furthermore, the vehicle periphery monitoring device according to the present invention is a vehicle periphery monitoring device that monitors an object present in the periphery of a vehicle from an image obtained by an imaging unit mounted on the vehicle, and identifies a roadway region in the image. The roadway area identifying means, the object extracting means for extracting an object from the image, the alerting means for alerting the driver of the vehicle, and the object extracting means by the roadway area identifying means. When an object is extracted within the identified road area, a predetermined alert is output by the alert means, and when an object is extracted outside the road area, an alert is output It is provided with the alerting control means which does not do.

  According to such a vehicle periphery monitoring device of the present invention, a roadway area that alerts only when an object is extracted in a roadway area that is highly likely to come into contact with the host vehicle and that is less likely to come into contact with the host vehicle. If an object is extracted outside, no alert is given. Therefore, it is possible to avoid making the driver of the vehicle feel the troublesomeness of alerting by alerting even when the possibility of contact with the host vehicle is low.

  Further, in the vehicle periphery monitoring device of the present invention, the imaging means images the front of the vehicle, and the alert control means is in real space between the vehicle and an object extracted in the road area of the image. An object distance, which is a distance along the direction of the front road area, is calculated, and when the object distance is equal to or less than a predetermined determination distance, alerting by the alerting means is performed.

  According to such a vehicle periphery monitoring device, the object distance calculated by the alerting control unit is a distance between the vehicle and the object in the traveling direction of the vehicle. Then, by performing the alert only when the object distance is equal to or less than the predetermined distance, the alert is limited to the object located in the traveling direction of the own vehicle, so unnecessary alerts and troublesome attentions are made. Arousing can be suppressed.

  Furthermore, the vehicle periphery monitoring device of the present invention includes, as the imaging unit, a visible image imaging unit that acquires a visible image, and an infrared image imaging unit that acquires an infrared image. When the lane mark of the roadway can be detected from the visible image acquired by the visible image pickup means, the roadway region is identified by the lane mark, and when the lane mark cannot be detected, the infrared image pickup An edge extraction process for extracting an edge part of an infrared image acquired by the means is performed, and a roadway area is identified by the edge part extracted by the edge extraction process.

  According to such a vehicle periphery monitoring device of the present invention, the imaging means includes the visible image imaging means and the infrared image imaging means, and the road area identification means identifies the road area from the visible image obtained by the visible image imaging means. When a high lane mark can be detected, the road area is identified using the lane mark as a first determination factor. When the lane mark cannot be detected, the edge portion of the infrared image obtained by the infrared image capturing means is extracted, and the road area is identified using the edge portion as the second determination element. In this way, by giving priority to the lane mark with high identification accuracy of the road area, the road area can be identified with high accuracy, and when the lane mark cannot be detected, the road area is determined by the edge portion of the infrared image. By identifying, the road area can be identified under a wider range of road conditions. Here, the visible image means an image that can be detected by a lane mark and is captured by a CCD camera, a near infrared camera, or the like.

  Further, in the vehicle periphery monitoring device of the present invention, the attention control unit detects a lower end portion of the object extracted from the image, and the lower end portion is within the road area extracted by the road area extraction unit. It is determined that the object is extracted in the road area when the vehicle is located in the roadway.

  According to the vehicle periphery monitoring device of the present invention, the lower end of the object extracted from the image is detected, and the object is detected depending on whether the lower end is located in the road area or outside the road area. It is determined whether or not it has been extracted from inside the road area or outside the road area. Here, the lower end portion of the target object can be detected as, for example, a pixel portion positioned at the lower end on the image in the region where the same distance portion of the target object is extracted. Then, by determining the position of the object with reference to the lower end of the object, whether or not the object is extracted in the vehicle area even when the object is located near the boundary of the roadway area. Whether or not can be determined easily and reliably.

  Furthermore, the vehicle periphery monitoring device of the present invention includes a position calculation unit that calculates a position in real space corresponding to an image portion included in the image, and the alerting control unit is an object extracted from the image. It is determined that the object has been extracted in the road area when the position in the real space is within the real space area of the road area extracted by the road area extraction means. .

  According to the vehicle periphery monitoring device of the present invention, the position calculating means calculates the position of the object extracted from the image in the real space and the area range of the roadway area in the real space, and It is determined whether or not the object is extracted from the road area or outside the road area depending on whether or not the position of the object is located within the area range or outside the area range. Thereby, for example, even when the shape of the object extracted from the image is a part lacking a part of the actual shape in the real space, the object is extracted in the vehicle area. It can be accurately determined whether or not.

  A vehicle according to the present invention for solving the above-described problems is characterized in that the vehicle periphery monitoring device is mounted.

  According to the vehicle of the present invention, the driver of the vehicle has a case where the object is present in the road area and is likely to come into contact with the own vehicle, and the object exists outside the road area. The case where the possibility of contact with the vehicle is low can be distinguished and recognized by a predetermined alert (or alerting only when the possibility of contact with the host vehicle is high). Thereby, it is possible to suppress unnecessary alerting and troublesome alerting and to perform highly reliable alerting to the driver of the vehicle.

  A vehicle periphery monitoring program according to the present invention for solving the above-described problems includes a means for accessing data of an image obtained by an imaging means mounted on a vehicle, and an alert for alerting a driver of the vehicle. A vehicle-side computer having a means for controlling the operation of the means for realizing a function of monitoring an object existing in the vicinity of the vehicle, the computer using the computer to identify a road area in the image Means for extracting an object from the image, and when the object is extracted in the road area identified by the road area identification means by the object extraction means, When a warning is output by the warning means and the object is extracted outside the roadway area, a second warning different from the first warning is issued. Wherein the function as reminder control means for outputting the electromotive unit.

  The vehicle periphery monitoring program according to the present invention is executed on the computer, the road area in the image is identified by the road area identification means, the object is extracted from the image by the object extraction means, and the attention When the object is extracted within the road area by the alert control means, a first alert is output by the attention means, and when the object is extracted outside the road area, When the second alerting that is different from the first alerting is output by the alerting means, the driver of the vehicle is likely to be in contact with the subject vehicle in the road area. And the case where the object exists outside the road area and the possibility of contact with the host vehicle is low can be distinguished and recognized by different alerts. As a result, even when the possibility of contact with the host vehicle is low, the inconvenience of alerting caused by the same alert as when the possibility of contact is high is suppressed, and the driver of the vehicle is thus reliable. High alerts can be made.

  The vehicle periphery monitoring method of the present invention for solving the above-described problems includes a means for accessing data of an image obtained by an imaging means mounted on a vehicle, and a warning means for calling attention to a driver of the vehicle. A vehicle periphery monitoring method for monitoring an object existing around the vehicle by an in-vehicle computer having a means for controlling the operation of the vehicle, wherein the computer identifies a roadway region in the image, and An object extraction step in which the computer extracts an object from the image; and the computer extracts an object in the road area identified by the road area identification means by the object extraction means. , When a first alert is output by the alert means and the object is extracted outside the road area, the first alert The different second alert and having a reminder control step of outputting by the attention drawing means.

  According to the vehicle periphery monitoring method of the present invention, the computer identifies a road area in the image at a road area identification step, extracts an object from the image at an object extraction step, and includes a warning control step. When the object is extracted within the road area, a first alert is output by the alert means, and when the object is extracted outside the road area, the first alert By outputting different second alerts by the alerting means, the driver of the vehicle has a high possibility that the object is in the roadway area and is in contact with the host vehicle, and the object is A case in which the vehicle is outside the road area and is unlikely to come into contact with the host vehicle can be distinguished and recognized by different alerts. As a result, even when the possibility of contact with the host vehicle is low, the inconvenience of alerting caused by the same alert as when the possibility of contact is high is suppressed, and the driver of the vehicle is thus reliable. High alerts can be made.

  An embodiment of the present invention will be described below with reference to FIGS.

  First, with reference to FIG. 1 and FIG. 2, the system configuration | structure of the vehicle periphery monitoring apparatus of this embodiment is demonstrated. FIG. 1 is a block diagram showing the overall configuration of the periphery monitoring device, and FIG. 2 is a perspective view showing the appearance of a vehicle (own vehicle) equipped with the periphery monitoring device. In FIG. 2, illustration of some components of the periphery monitoring device is omitted.

  With reference to FIGS. 1 and 2, the periphery monitoring device of the present embodiment includes an image processing unit 1. The image processing unit 1 is connected with two infrared cameras 2R, 2L and one visible camera 2 'as imaging means for capturing an image in front of the host vehicle 10, and the traveling state of the host vehicle 10 is also monitored. As sensors to detect, a yaw rate sensor 3 that detects the yaw rate of the host vehicle 10, a vehicle speed sensor 4 that detects the traveling speed (vehicle speed) of the host vehicle 10, and a brake sensor 5 that detects whether or not the host vehicle 10 is braked. Is connected. Further, the image processing unit 1 has a speaker 6 for outputting auditory alert information such as sound, and a display for displaying images taken by the infrared cameras 2R and 2L and visual alert information. Device 7 is connected.

  The infrared cameras 2R and 2L correspond to the infrared image capturing unit in the present invention, and the visible camera 2 'corresponds to the visible image capturing unit in the present invention. Further, the infrared cameras 2R and 2L and the visible camera 2 ′ correspond to the imaging means of the present invention. The speaker 6 and the display device 7 correspond to the alerting means of the present invention.

  Although not shown in detail, the image processing unit 1 includes electronic circuits including an A / D conversion circuit, a microcomputer (having a CPU, a RAM, a ROM, and the like), an image memory, and the like. The infrared cameras 2R, 2L, Analog signals output from the visible camera 2 ′, the yaw rate sensor 3, the vehicle speed sensor 4, and the brake sensor 5 are converted into digital data by the A / D conversion circuit and input to the microcomputer. The microcomputer detects an object such as a person (pedestrian or person riding a bicycle) based on the input data, and the speaker detects when the detected object satisfies a predetermined alerting requirement. 6 or a display device 7 is used to execute a process for alerting the driver.

  The image processing unit 1 has functions as a roadway area identification unit, an object extraction unit, and an attention control unit in the present invention. Then, by causing the microcomputer to execute the vehicle periphery monitoring program of the present invention, the computer functions as a roadway area identifying unit, an object extracting unit, and an alerting control unit of the present invention. Furthermore, by causing the microcomputer to function as a roadway area identifying means, an object extracting means, and an alerting control means, the roadway area identifying step, the object extracting step, and the alerting control step in the vehicle periphery monitoring method of the present invention are performed. Executed.

  As shown in FIG. 2, the infrared cameras 2 </ b> R and 2 </ b> L are attached to the front portion (the front grill portion in the figure) of the host vehicle 10 in order to image the front of the host vehicle 10. In this case, the infrared cameras 2R and 2L are respectively disposed at a position on the right side and a position on the left side of the center of the host vehicle 10 in the vehicle width direction. These positions are symmetrical with respect to the center of the host vehicle 10 in the vehicle width direction. The infrared cameras 2R and 2L are fixed so that their optical axes extend in parallel in the front-rear direction of the host vehicle 10 and the heights of the optical axes from the road surface are equal to each other. . The infrared cameras 2R and 2L have sensitivity in the far-infrared region, and the higher the temperature of the object to be imaged, the higher the level of the output video signal (the luminance of the video signal increases). have.

  Further, the visible camera 2 ′ is attached in front of a room mirror (not shown) of the host vehicle 10 with the imaging direction facing the front of the host vehicle 10. The visible camera 2 ′ is configured by a CCD camera or the like that can capture an image in the visible light region, and the higher the illuminance of the object to be imaged, the higher the level of the output video signal (the luminance of the image of the video signal is higher). (Becomes higher).

  Further, in the present embodiment, the display device 7 includes a head-up display 7a (hereinafter referred to as HUD 7a) that displays image information on the front window of the host vehicle 10. In addition, as a display device 7, instead of the HUD 7a, or together with the HUD 7a, a display provided integrally with a meter for displaying a traveling state such as the vehicle speed of the host vehicle 10 or a display provided in an in-vehicle navigation device is provided. It may be used.

  Next, the overall operation of the periphery monitoring device of this embodiment will be described with reference to the flowchart of FIG.

  With reference to FIG. 3, the image processing unit 1 of the present embodiment first performs “roadway area identification processing” in which a roadway area is identified and set from a visible image or an infrared image in STEP 100, and the object is detected from the infrared image in STEP200. Perform “object extraction processing” to extract objects.

  In STEP 300, it is determined whether or not an object has been extracted in the “object extraction process”. If extracted, the process proceeds to STEP 400, and whether or not a road area has been set in the “road area identification process”. Determine whether. On the other hand, when no object is extracted in STEP 300, the process returns to STEP 100 and the processes after the “roadway area identification process” are executed again.

  In STEP 400, when the road area is set, the process proceeds to STEP 500, and when the object is extracted within the alert area in the road area, the level of attention is higher than when the object is extracted outside the road area. The “attention calling process (I)” for calling out is executed.

  On the other hand, when the road area is not set in STEP 400, the process branches to STEP 600 and alerts when an object is detected in the front area of the host vehicle 10 without distinguishing between the inside and the outside of the road area. Execute “Attention Calling Process (II)”.

  The above is the overall operation of the periphery monitoring device of the present embodiment.

  Next, specific contents of the “roadway area identification process” will be described with reference to the flowchart shown in FIG. 4.

  First, the image processing unit 1 acquires a visible image by the visible camera 2 ′ (STEP 101). Specifically, the acquired visible image is a grayscale image obtained by A / D converting the output signal of the visible camera 2 ', and is stored in the image memory.

  Subsequently, lane mark detection is performed to detect a travel line (lane mark) such as a white line on the visible image (STEP 102). Specifically, an edge point in the visible image is extracted by a differential filter or the like, and a straight line component corresponding to a traveling division line such as a white line is detected as a lane mark from the edge point.

  For example, in STEP 101, when the visible image shown in FIG. 5A is acquired, the visible image is subjected to a differentiation process to generate an edge image, and the edge image is subjected to a well-known Hough transformation. A straight line component corresponding to the travel line is extracted. When a straight line component corresponding to a travel line such as a white line is extracted, the lane mark is detected as indicated by a circle in FIG. 5B by detecting the straight line component as a lane mark. can do.

  Next, the image processing unit 1 determines whether or not a lane mark is detected in STEP 102 (STEP 103).

  When a lane mark is detected (when the determination result in STEP 103 is YES), the process proceeds to STEP 104, and a road area is set based on the position of the lane mark detected in STEP 102. Then, the process proceeds to STEP 105, and the “roadway area identification process” is terminated.

  On the other hand, if it is determined in STEP 103 that no lane mark is detected (when the determination result in STEP 103 is NO), the process branches to STEP 106, and the image processing unit 1 receives an infrared image by the infrared cameras 2R and 2L. get. The infrared image is a grayscale image obtained by A / D converting the infrared image that is the output signal of each of the infrared cameras 2R and 2L, and is stored in the image memory. Hereinafter, an image obtained from the infrared camera 2R is referred to as a right image, an image obtained from the infrared camera 2L is referred to as a left image, and in particular, the right image is referred to as a reference image. In addition, the reference image is a right image in the present embodiment, but may be a left image.

  Subsequently, the image processing unit 1 detects a road area from the infrared image (STEP 107). Specifically, an edge point in the reference image is extracted by a differential filter or the like, and a road area is detected in a front area of the host vehicle 10 from the edge point.

  For example, when the infrared image shown in FIG. 6A is acquired as the reference image in STEP 106, as in the lane mark detection in STEP 102 described above, a differential process is performed on the reference image to generate an edge image. A straight line component corresponding to the end of the road region is extracted from the edge image by a well-known Hough transform. Then, when a straight line component corresponding to the end of the road area is extracted, the road area is detected as indicated by a circle in FIG. 6B by detecting the straight line component as the end of the road area. Can be detected.

  Next, the image processing unit 1 determines whether or not a road area is detected in STEP 107 (STEP 108).

  When the road area is detected (when the determination result in STEP 108 is YES), the end position of the road area detected in STEP 107 is set as the road area (STEP 109), and the “road area identification process” is performed. finish.

  On the other hand, when the road area is not detected (when the determination result in STEP 108 is NO), the process branches to STEP 105, and the “road area identification process” is terminated without setting the road area.

  In addition, the structure which performs the process of STEP100 by the image processing unit 1 corresponds to the roadway area | region identification means of this invention. Further, the processing of STEP 100 corresponds to a roadway area identification step in the vehicle periphery monitoring method of the present invention.

  Next, the specific contents of the “object extraction process” (STEP 300 in FIG. 3) will be described with reference to the flowchart shown in FIG.

  First, the image processing unit 1 acquires an infrared image by the infrared cameras 2R and 2L (STEP 201).

  Next, the image processing unit 1 binarizes the reference image (right image) (STEP 202). In this binarization process, the luminance value of each pixel of the reference image is compared with a predetermined luminance threshold value, and an area (relatively bright area) having a luminance value higher than the predetermined luminance threshold value in the reference image. This is a process of setting “1” (white) and setting an area having a luminance value lower than the luminance threshold (a relatively dark area) to “0” (black). Hereinafter, an image (monochrome image) obtained by the binarization process is referred to as a binarized image. In the binarized image, an area “1” is referred to as a high luminance area. The binarized image is stored in the image memory separately from the grayscale image (right image and left image).

  Next, the image processing unit 1 performs the processing of STEPs 203 to 205 on the binarized image, and extracts an object (more precisely, an image portion corresponding to the object) from the binarized image. That is, the pixel group constituting the high luminance area of the binarized image is classified into lines having a width of one pixel in the vertical direction (y direction) of the reference image and extending in the horizontal direction (x direction). Each line is converted into run-length data composed of coordinates and length (number of pixels) at the position (two-dimensional position on the reference image) (STEP 203). Of the lines represented by the run length data, labels (identifiers) are attached to each of the line groups that overlap in the vertical direction of the reference image (STEP 204), and each of the line groups is extracted as an object. (STEP 205).

  Note that the objects extracted by the processing of STEPs 203 to 205 include not only people (pedestrians) but also artificial structures such as other vehicles. In addition, a plurality of local parts of the same object may be extracted as the target object.

  Next, the image processing unit 1 obtains the position of the center of gravity of each object extracted as described above (position on the reference image), the area, and the aspect ratio of the circumscribed rectangle (STEP 206). The area is calculated by accumulating the length of the run length data for the same object, and the coordinates of the center of gravity are the x coordinate of the line that bisects the area in the x direction and the y of the line that bisects the y direction. Calculated as coordinates, the aspect ratio is calculated as the aspect ratio of the circumscribed rectangle of the line converted to run length data. Note that the position of the center of gravity G may be substituted by the position of the center of gravity of the circumscribed rectangle.

  Next, the image processing unit 1 tracks the target object extracted in STEP 205, that is, recognizes the same target object every calculation processing period of the image processing unit 1 (STEP 207). In this process, when the object A is extracted by the processing of STEP 205 at a time (discrete system time) k in a certain arithmetic processing cycle, and the object B is extracted by the processing of STEP 205 at time k + 1 of the next arithmetic processing cycle. The identity of these objects A and B is determined. This identity determination may be performed based on, for example, the shape and size of the objects A and B on the binarized image, the correlation of the luminance distribution on the reference image (grayscale image), and the like. . When it is determined that the objects A and B are the same, the label of the object B extracted at time k + 1 (the label attached in STEP 204) is changed to the same label as the label of the object A. .

  Next, the image processing unit 1 reads the outputs (the detected vehicle speed value and the detected yaw rate value) of the vehicle speed sensor 4 and the yaw rate sensor 5 (STEP 208). In STEP 208, the turning angle (azimuth angle) of the host vehicle 10 is also calculated by integrating the read detection value of the yaw rate.

  On the other hand, the image processing unit 1 executes the processing of STEP 210 in parallel with the processing of STEPs 207 and 208. The processing in STEPs 210 to 212 is processing for obtaining the distance from the subject vehicle 10 of each object extracted in STEP 205. The process is schematically described. First, an area corresponding to each object (for example, a circumscribed square area of the object) is extracted as a search image R1 from the reference image (STEP 210).

  Next, in the left image, a search area R2 that is an area for searching for the same object as the object included in the search image R1 of the right image is set as the left image, and the search image R1 is set in the search area R2. Is extracted as a corresponding image R3 that is an image corresponding to the search image R1 (an image equivalent to the search image R1) (STEP 211). In this case, an area having a luminance distribution that most closely matches the luminance distribution of the search image R1 of the right image is extracted as the corresponding image R3 from the search area R2 of the left image. Note that the processing of STEP 211 is performed using a grayscale image, not a binarized image.

  Next, the number of pixels of the difference between the lateral position (x-direction position) of the center of gravity of the search image R1 in the right image and the lateral position (x-direction position) of the center of gravity of the corresponding image R3 in the left image is defined as parallax Δd. The distance z of the object from the host vehicle 10 (the distance in the front-rear direction of the host vehicle 10) is calculated using the parallax Δd (STEP 212). The distance z is calculated by the following equation (1).

z = (f × D) / (Δd × p) (1)

Note that f is the focal length of the infrared cameras 2R and 2L, D is the base length (interval of the optical axis) of the infrared cameras 2R and 2L, and p is the pixel pitch (length of one pixel).

  The above is the outline of the processing of STEPs 210 to 212. Note that the processing in STEPs 210 to 212 is executed for each object extracted in STEP 205.

  After the processing of STEP 208 and STEP 212 is completed, the image processing unit 1 next calculates a real space position that is a position (relative position with respect to the host vehicle 10) of each object in the real space (STEP 213). Here, as shown in FIG. 2, the real space position is a position (X, Y, Z) in the real space coordinate system (XYZ coordinate system) set with the midpoint of the attachment position of the infrared cameras 2R, 2L as the origin. ). The X direction and the Y direction of the real space coordinate system are the vehicle width direction and the vertical direction of the host vehicle 10, respectively. These X direction and Y direction are the x direction (lateral direction) and y of the right image and the left image, respectively. It is the same direction as the direction (vertical direction). The Z direction in the real space coordinate system is the front-rear direction of the host vehicle 10. Then, the real space position (X, Y, Z) of the object is calculated by the following equations (2), (3), (4).

X = x × z × p / f (2)
Y = y × z × p / f (3)
Z = z (4)

Note that x and y are the x-coordinate and y-coordinate of the object on the reference image.

  Next, the image processing unit 1 compensates for the influence of the change in the turning angle of the host vehicle 10 and increases the accuracy of the real space position of the target object, so that the real space position (X, Y, Z) of the target object is increased. The position X in the X direction is corrected according to the time-series data of the turning angle obtained in STEP 208 from the value obtained by the above formula (2) (STEP 214). Thereby, the real space position of the object is finally obtained. In the following description, “the real space position of the object” means the real space position of the object subjected to this correction.

  Next, the image processing unit 1 obtains a movement vector of the object relative to the host vehicle 10 (STEP 215). Specifically, a straight line that approximates time-series data in a predetermined period (a period from the current time to a predetermined time) of the real space position of the same object is obtained, and the straight line at the time before the predetermined time is obtained. A vector from the position (point) of the object toward the position (point) of the object on the straight line at the current time is obtained as a movement vector of the object. This movement vector is proportional to the relative velocity vector of the object with respect to the host vehicle 10.

Further, the image processing unit 1 executes the processes of STEPs 216 to 217 following the processes of STEPs 213 to 215. In the process of STEP 216, when each target object extracted in STEP 205 is an artificial structure, an artificial structure exclusion process for excluding the target object from the alerting target is performed (STEP 216). The artificial structure exclusion process is a process in which, when a feature that is impossible for a pedestrian is detected in a target object image, the target object is determined as an artificial structure and is excluded from an alert target. For example, when a feature that is impossible for a pedestrian is detected as in the conditions shown in the following (a) to (d), the target object is determined as an artificial structure and excluded from the target of alerting.
(A) When the image of the object includes a portion indicating a straight edge.
(B) The corner of the image of the object is a right angle.
(C) A case where a plurality of objects having the same shape are included in the image of the object.
(D) When the image of the object matches the shape of the artificial structure registered in advance.

  Next, in STEP 216, after performing the process of excluding the artificial structure from the object, in order to further increase the reliability of the determination, the pedestrian recognition process for certifying whether the object is a pedestrian or not. (STEP 217). In the pedestrian recognition process, the object is alerted unless the object is recognized as a pedestrian due to the shape and size of the object image, the luminance dispersion, etc. on the reference image. It is processing to be excluded from the target.

  The above is the details of the “object extraction process”. In addition, the structure which performs the process of STEP200 by the image processing unit 1 is equivalent to the target object extraction means of this invention. Further, the processing of STEP 200 corresponds to an object extraction step in the vehicle periphery monitoring method of the present invention.

  Further, the configuration for executing the processing of STEP 213 by the image processing unit 1 corresponds to the position calculating means of the present invention, and the configuration of executing the processing of STEP 215 corresponds to the movement vector calculating means of the present invention.

  Next, the specific content of the “attention calling process (I)” (STEP 500 in FIG. 3) will be described according to the flowchart shown in FIG.

  First, the image processing unit 1 determines whether or not the object has been extracted from the alerting range in the road area set by the “road area identification process” (STEP 501).

  Specifically, for example, when the image shown in FIG. 9 is acquired as the reference image, the part of the object Q where a certain luminance value continues or the same distance part is set as the range of the same object. The lower end q of Q is determined. Then, it is determined whether or not the object Q is extracted from the road area ARE based on whether or not the lower end q is located in the road area ARE. For example, in the case shown in FIG. 9, since the lower end q of the object Q is located in the road area ARE, it is determined that the object Q is extracted from the road area ARE.

  Further, as shown in FIG. 10A, the direction from the own vehicle 10 to the positions er1, er2,... At the predetermined intervals at the right end of the road area ARE and the distances dr1, dr2,. Further, the direction and distances dl1, dl2,... From the host vehicle 10 to the positions el1, el2,... At predetermined intervals at the left end of the road area ARE are calculated. The image processing unit 1 recognizes the shape of the road area ARE and the distance along the direction of the front road area ARE based on the direction and distance to the right end and the direction and distance to the left end. Then, as shown in FIG. 10 (b), a range (a hatched portion in the figure) in which the distance along the direction of the forward road area ARE is a predetermined distance L (a distance determined by the own vehicle speed or a relative speed with the object). ) As the alerting range, it is determined whether or not the object Q exists in the alerting range.

  For example, the object Q shown in FIG. 9 is determined to be the object extracted in the road area ARE from the position of the lower end q of the object Q on the image as described above. ) Since the object distance dq between the host vehicle 10 and the object Q along the direction of the forward road area ARE is equal to or less than the predetermined distance L, the object Q is the alert in the road area. It is determined that the object is extracted from the range.

  In the present embodiment, the image processing unit 1 detects the lower end portion of the object on the reference image, and whether the lower end is located in the road area or not, the object is detected from the road area. Although it is determined whether or not it is an extracted one, the real space position (X, Z) of the object is on the real space corresponding to the roadway region ((2) and (4) above) ( It may be determined whether or not the object is extracted from the roadway area depending on whether it is located within the area range (on the XZ coordinate axis).

  Next, when it is determined in STEP 501 that the object has been extracted from the alerting range in the road area (when the determination result in STEP 501 is YES), the object exists in the road area and the host vehicle It is estimated that the possibility of contact with 10 is high. For this reason, a warning of level S having the highest warning level indicating the degree of danger is performed (STEP 502, corresponding to the first warning in the present invention). More specifically, the speaker 6 emits an audible alert and a predetermined alert sound indicating that an object that may be a pedestrian exists in the road area, and the reference image of the image display device 7. An image of an object that may be a pedestrian is highlighted in the interior. This alerts the driver to an object that may come into contact with the host vehicle 10.

  On the other hand, when it is determined in STEP 501 that the object is not extracted from the alerting range in the road area (when the determination result in STEP 501 is NO), the process branches to STEP 503, and the above-described FIG. It is determined whether or not the movement vector of the object calculated in STEP 215 of the “object extraction process” has a size that reaches the alerting range during a predetermined time (STEP 503). Here, the predetermined time is the time required for the host vehicle 10 to pass the object distance dq at the current vehicle speed.

  For example, in the case of the object Q1 having the movement vector u1 shown in FIG. 10B, when the movement vector u1 has a size that reaches the warning range during the predetermined time, the object Q1 It is determined that the object has a movement vector that reaches the arousal region. On the other hand, in the case of the object Q2 having the movement vector u2, since the movement vector u2 does not have a size that reaches the attention calling range during the predetermined time, the object Q2 has a movement vector that reaches the attention calling range. It is determined that the object is not included.

  Next, when it is determined in STEP 503 that the object is an object having a movement vector that reaches the alert range within a predetermined time (when the determination result in STEP 503 is YES), the object is in the alert range. It is estimated that there is a possibility of entering the vehicle and coming into contact with the host vehicle 10. Therefore, a warning of level A having the second highest warning level is given (STEP 504, corresponding to the second alerting in the present invention). Specifically, a warning to the effect that a target object that may be a pedestrian enters the roadway area is issued via the speaker 6, and the pedestrian can be included in the reference image of the image display device 7. The image of the target object is highlighted. This alerts the driver to an object that may come into contact with the host vehicle 10.

  On the other hand, when it is determined in STEP 503 that the object is not an object having a movement vector that reaches the alerting range (when the determination result in STEP 503 is NO), the contact between the object and the host vehicle 10 is determined. The possibility is estimated to be low. Therefore, in this case, attention is given to level B, which is a low alert level indicating the degree of danger (STEP 505, corresponding to the second alert in the present invention). Specifically, the driver's situation around the vehicle is either given a voice alert that there is an object with low possibility of contact around the vehicle, or highlighting the object image. Call attention to.

  In the present embodiment, the image processing unit 1 alerts the level S with the highest warning level indicating the degree of danger when it is determined in STEP 501 that the object has been extracted from the alerting range in the road area. (STEP502), the movement vector of the object extracted from the alerting range is calculated, and depending on whether or not the movement vector has a magnitude that reaches the outside of the alerting range during a predetermined time Level S alerts may be changed. Specifically, when an object extracted from the alerting range has a movement vector that reaches outside the alerting range within a predetermined time, the alerting level is lowered and alerting corresponding to level A is performed. It may be.

  The above is the details of the “attention calling process (I)”. In addition, the structure which performs the process of STEP500 by the image processing unit 1 corresponds to the alerting control means of the present invention. Further, the processing in STEP 500 corresponds to the alerting control step in the vehicle periphery monitoring method of the present invention.

  Next, an outline of the attention calling process (II) (STEP 600) will be described. Note that the alerting process (II) (STEP 600) is the same as the process described in FIG. 4 (STEPs 41 to 45) of Patent Document 2, and thus detailed description thereof will be omitted.

  First, the image processing unit 1 determines that there is a possibility of contact when the distance between the host vehicle 10 and the object is equal to or less than the value obtained by multiplying the relative speed Vs and the margin time T.

  Next, when there is a possibility that the host vehicle 10 and the target object contact within the margin time T, in order to further increase the reliability of the determination, as shown in FIG. It is determined whether or not the vehicle 10 is in the approach determination area AR1 where the possibility of contact with the vehicle 10 is extremely high.

  Further, when the object does not exist in the approach determination area AR1, the image processing unit 1 may enter the approach determination area AR1 from the entry determination areas AR2 and AR3 and come into contact with the host vehicle 10. Determine whether or not.

  Next, when the object is present in the approach determination area AR1, or when it is determined that the object may enter the approach determination area AR1 from the entry determination areas AR2 and AR3 and come into contact with the host vehicle 10 In this case, a warning output determination process for determining whether or not actual warning should be performed is executed.

  If the image processing unit 1 determines that alerting should be performed, in this case, the object is estimated to be in front of the host vehicle 10 and possibly in contact with the host vehicle 10. The For this reason, a warning is issued to the effect that there is an object that can be contacted in front of the vehicle via the speaker 6, and the image of the object is highlighted in the reference image of the image display device 7. To do. This alerts the driver to an object that may come into contact with the host vehicle 10.

As described above, according to the vehicle periphery monitoring device of the present embodiment, the driver of the host vehicle 10 determines that there is a possibility of a pedestrian in STEPs 216 to 217 in the “target extraction process” of STEP 200. Against the target object
(1) When the object exists in the road area and is likely to come into contact with the host vehicle 10,
(2) When the object exists outside the road area but may enter the road area and come into contact with the host vehicle 10, and
(3) Although the possibility of contact is low, it is possible to recognize when there is an object with the possibility of a pedestrian around the host vehicle 10 by different alerting. Therefore, unnecessary alerting and troublesome alerting can be suppressed, and highly reliable alerting can be performed to the driver of the vehicle.

  Further, even when the roadway area is not detected, and there is a high possibility that an object is present in front of the host vehicle 10 and is in contact with the host vehicle 10, the same as the above (1) to (3) By performing alerting or alerting differently, the driver of the vehicle can be made aware of the situation around the vehicle.

  In the present embodiment, when the object is detected outside the road area (when the determination result in STEP 503 is NO), the alert level B or alert level C is alerted ( (STEP 504 or 505), when the object is detected outside the road area, the warning output may not be performed.

  Moreover, in this embodiment, although it is comprised so that predetermined alerting may be performed based on the process result of the image processing unit 1, a vehicle behavior may be controlled based on this process result.

  Furthermore, in the above-described embodiment, the two infrared cameras 2R and 2L are provided. However, when the distance to the object is detected by a radar or the like, one infrared camera 2R or 2L is attached to the host vehicle 10. You may make it mount in.

The figure which shows the whole structure of one Embodiment of the periphery monitoring apparatus of the vehicle of this invention. The perspective view of the vehicle provided with the periphery monitoring apparatus of FIG. 3 is a flowchart showing processing of an image processing unit provided in the periphery monitoring device of FIG. 1. 3 is a flowchart showing processing of an image processing unit provided in the periphery monitoring device of FIG. 1. The figure which shows the captured image in this embodiment exemplarily. The figure which shows the captured image in this embodiment exemplarily. 3 is a flowchart showing processing of an image processing unit provided in the periphery monitoring device of FIG. 1. 3 is a flowchart showing processing of an image processing unit provided in the periphery monitoring device of FIG. 1. The figure which shows the captured image in this embodiment exemplarily. The figure for demonstrating the process of an image processing unit. The figure for demonstrating the periphery monitoring apparatus of the conventional vehicle.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image processing unit (Roadway area identification means, object extraction means, alerting control means), 2R, 2L ... Infrared camera (infrared image imaging means), 2 '... Visible camera (visible image imaging means), 6 ... Speaker (attention calling means), 7 ... Display device (attention calling means), STEP 100 ... Processing by roadway area identification means, STEP 200 ... Processing by object extraction means, STEP 213 ... Processing by position calculation means, STEP 215 ... By movement vector calculation means Processing, STEP500: Processing by the alerting control means.

Claims (12)

A vehicle periphery monitoring device for monitoring an object existing around a vehicle from an image obtained by an imaging means mounted on the vehicle,
Road area identification means for identifying the road area in the image;
Object extraction means for extracting an object from the image;
Alerting means for alerting the driver of the vehicle;
When the object is extracted in the road area identified by the road area identification means by the object extraction means, a first alert is output by the alert means, and the object is output from the road A vehicle periphery monitoring device, comprising: an alert control means for outputting a second alert different from the first alert by the alert means when extracted outside the area. The imaging means images the front of the vehicle,
The alerting control means calculates an object distance, which is a distance along the direction of the forward road area in the real space between the vehicle and the object extracted in the road area of the image, and the object distance 2. The vehicle periphery monitoring device according to claim 1, wherein the first alerting by the alerting means is performed when is equal to or less than a predetermined determination distance. Movement vector calculation means for calculating a movement vector of the object from a change in position of the same object extracted from a plurality of images acquired at different timings by the imaging means;
The alerting control unit is configured to reach when the object extracted from the road area by the object extraction unit is estimated to reach the outside of the road area within a predetermined time from the movement vector of the object. The vehicle periphery monitoring device according to claim 1 or 2, wherein the content of the first alert is changed depending on when it is estimated not to be performed. Movement vector calculation means for calculating a movement vector of the object from a change in position of the same object extracted from a plurality of images acquired at different timings by the imaging means;
The alerting control unit is configured to reach when the object extracted from outside the road area by the object extraction unit is estimated to reach the road area within a predetermined time from the movement vector of the object. The vehicle periphery monitoring device according to claim 1 or 2, wherein the content of the second alerting is changed depending on when it is estimated that it will not be performed. A vehicle periphery monitoring device for monitoring an object existing around a vehicle from an image obtained by an imaging means mounted on the vehicle,
Road area identification means for identifying the road area in the image;
Object extraction means for extracting an object from the image;
Alerting means for alerting the driver of the vehicle;
When the object is extracted in the road area identified by the road area identification means by the object extraction means, a predetermined alert is output by the attention calling means, and the object is output from the road area. A vehicle periphery monitoring device, comprising: a reminder control unit that does not output a reminder when extracted outside. The imaging means images the front of the vehicle,
The alerting control means calculates an object distance, which is a distance along the direction of the forward road area in the real space between the vehicle and the object extracted in the road area of the image, and the object distance 6. The vehicle periphery monitoring device according to claim 5, wherein alerting is performed by alerting means when is equal to or less than a preset determination distance. As the imaging means, a visible image imaging means for acquiring a visible image, and an infrared image imaging means for acquiring an infrared image,
The roadway area identifying unit identifies the roadway area by the lane mark when the lane mark of the roadway can be detected from the visible image acquired by the visible image capturing unit, and the lane mark cannot be detected. The method includes performing edge extraction processing for extracting an edge portion of an infrared image acquired by the infrared image capturing unit, and identifying a roadway region by the edge portion extracted by the edge extraction processing. The vehicle periphery monitoring device according to any one of claims 1 to 6.   The alerting control means detects a lower end portion of the object extracted from the image, and when the lower end portion is located in the road area extracted by the road area extraction means, the object is The vehicle periphery monitoring device according to any one of claims 1 to 7, wherein it is determined that the vehicle has been extracted in a roadway region. A position calculating means for calculating a position in real space corresponding to an image portion included in the image;
When the position of the object extracted from the image in the real space is within the area of the roadway area extracted by the roadway area extraction means, the alerting control means The vehicle periphery monitoring device according to any one of claims 1 to 7, wherein it is determined that the vehicle has been extracted within the roadway region.   A vehicle comprising the vehicle periphery monitoring device according to any one of claims 1 to 9. An on-vehicle computer having means for accessing image data obtained by imaging means mounted on a vehicle and means for controlling the operation of alert means for alerting the driver of the vehicle A program for realizing a function of monitoring an object existing in the computer,
Road area identification means for identifying the road area in the image;
Object extraction means for extracting an object from the image;
When the object is extracted in the road area identified by the road area identification means by the object extraction means, a first alert is output by the alert means, and the object is output from the road A vehicle periphery monitoring program that causes a second alert that is different from the first alert to be output by the alert means when it is extracted outside the area, as an alert control means. An in-vehicle computer having means for accessing data of an image obtained by an image pickup means mounted on a vehicle and means for controlling the operation of the alerting means for alerting the driver of the vehicle. A vehicle periphery monitoring method for monitoring an object existing in a vehicle,
A road area identifying step for identifying a road area in the image by the computer;
An object extraction step in which the computer extracts an object from the image;
When the object is extracted in the road area identified by the road area identification means by the object extraction means, the computer outputs a first warning by the attention calling means, A vehicle surrounding monitoring, comprising: an alerting control step for outputting, by the alerting means, a second alert that is different from the first alert when an object is extracted outside the road area. Method.