JP5292047B2 - Vehicle periphery monitoring device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To change a processing range in which objects are recognized according to a reaction time. <P>SOLUTION: In a picked up image, the device recognizes objects on the periphery of a vehicle by regarding an image area corresponding to a predetermined range on the periphery of the vehicle as a processing object, and gives an alarm to a driver of the vehicle about the objects recognized. A reaction time of the driver for a predetermined operation with respect to the alarm is detected. When the reaction time is detected to be a predetermined value or more, the predetermined range is enlarged and the recognition of the objects is carried out by making an image area corresponding to the enlarged predetermined range a processing object. The enlargement is carried out in a horizontal and/or distance direction. In the case of long reaction time, therefore, the recognition of the objects by the driver and driving operation through the recognition are achieved earlier. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to an apparatus for monitoring the periphery of a vehicle, which recognizes an object existing around the vehicle and issues a warning to a driver.

  2. Description of the Related Art Conventionally, a technique has been proposed in which the periphery of a vehicle is imaged by a camera, an object existing around the vehicle is recognized, and an alarm is issued to the driver. In response to this alarm, the driver can perform a driving operation that avoids the object, such as a brake operation or a steering operation.

The following Patent Document 1 discloses a device that changes the timing for issuing a collision warning according to the timing of a brake operation by a driver. Patent Document 2 below discloses a technique for learning the driver's braking operation and changing the alarm timing.
JP 59-105587 A Japanese Patent Laid-Open No. 2006-190090

  There are individual differences in the response of the driver's driving operation to the alarm. The longer the reaction time, the more delayed the recognition of the presence of the object and the operation of avoiding the object corresponding thereto, and the time margin for executing the avoidance operation may be reduced.

  Recently, an imaging camera such as an infrared camera is mounted on a vehicle, a predetermined object is recognized by performing image processing on an image captured by the imaging camera, and an alarm is given to the driver according to the recognition of the object. The technology of emitting is proposed. In order to increase the accuracy of object recognition, the contents of image processing have become increasingly complex in recent years. As the range around the vehicle to be processed is wider, the computational load of image processing increases, and the possibility of an error in the object recognition processing due to the presence of noise or the like increases. Therefore, in order to further improve the recognition accuracy, it is desirable to limit the range around the vehicle to be processed.

  However, there is a possibility that the object far from the vehicle is not recognized as the range to be processed is limited. If such a restriction is applied uniformly, if the driver's response time to the alarm is long, the recognition of the object by the driver and the driving operation may be delayed even if the recognition accuracy of the object is high. There is.

  Therefore, according to the present invention, by adjusting the size of the range to be processed according to the reaction time, even when the reaction time is long, the recognition of the object and the early avoidance operation for the object can be achieved. The purpose is to.

  According to one aspect of the present invention, in an image captured by an imaging unit mounted on a vehicle, an object around the vehicle is recognized using an image region corresponding to a predetermined range around the vehicle as a processing target, A vehicle periphery monitoring device that issues a warning to a vehicle driver about the recognized object, a reaction time detection unit that detects a reaction time of a predetermined operation of the driver with respect to the issue of the warning, and If it is detected that the reaction time is equal to or greater than a predetermined value, the predetermined range is expanded so that the recognition of the object is performed on an image area corresponding to the expanded predetermined range as a processing target. Range changing means.

  According to this invention, when the reaction time is equal to or greater than the predetermined value, the predetermined range in which the object is recognized is expanded. Therefore, even when the reaction time is long, the recognition of the object and the avoidance operation for the object can be accelerated.

  Other features and advantages of the present invention will be apparent from the detailed description that follows.

  Next, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a vehicle periphery monitoring device according to an embodiment of the present invention. The apparatus is mounted on a vehicle and includes two infrared cameras 1R and 1L capable of detecting far-infrared rays, a yaw rate sensor 5 that detects the yaw rate of the vehicle, and a vehicle speed sensor 6 that detects a travel speed (vehicle speed) VCAR of the vehicle. A brake sensor 7 for detecting the operation amount of the brake, an image processing unit 2 for detecting an object in front of the vehicle based on image data obtained by the cameras 1R and 1L, and a sound based on the detection result The head-up display (hereinafter referred to as the speaker 3) that displays an image obtained by the speaker 3 and the camera 1R or 1L and that causes the driver to recognize an object determined to have a high possibility of a collision. , Called HUD).

  As shown in FIG. 2, the cameras 1 </ b> R and 1 </ b> L are disposed at the front part of the vehicle 10 at positions symmetrical with respect to the central axis passing through the center of the vehicle width. The two cameras 1R and 1L are fixed to the vehicle so that their optical axes are parallel to each other and their height from the road surface is equal. The infrared cameras 1R and 1L have a characteristic that the level of the output signal becomes higher (that is, the luminance in the captured image becomes higher) as the temperature of the object is higher than the background temperature.

  The image processing unit 2 includes an A / D conversion circuit that converts an input analog signal into a digital signal, an image memory that stores a digitized image signal, a central processing unit (CPU) that performs various arithmetic processing, and a data RAM (Random Access Memory) used to store data, ROM (Read Only Memory) that stores programs executed by the CPU and data used (including tables and maps), driving signals for the speaker 3, display signals for the HUD 4, etc. Is provided. The output signals of the cameras 1R and 1L and the output signals of the sensors 5 to 7 are converted into digital signals and input to the CPU. As shown in FIG. 2, the HUD 4 is provided so that a screen 4 a is displayed at a front position of the driver on the front window of the vehicle 10. Thus, the driver can visually recognize the screen displayed on the HUD 4.

  Further, the apparatus includes a reaction time detection unit 8 and a processing range change unit 9, and after a warning is issued for an object recognized by the image processing unit 2, a predetermined operation of the driver is started. The reaction time until the vehicle is detected, and a predetermined range around the vehicle to be processed for object recognition is changed according to the reaction time. The reaction time detection unit 8 and the processing range changing unit 9 can be realized in a predetermined electronic control device that is a computer including a central processing unit (CPU) and a memory. It may be realized in the same control device as the image processing unit 2. Details of these processes will be described later.

  FIG. 3 is a flowchart showing a process executed by the image processing unit 2. The process is performed at predetermined time intervals.

  In steps S11 to S13, the output signals of the cameras 1R and 1L (that is, captured image data) are received as input, A / D converted, and stored in the image memory. The stored image data is a grayscale image, and has a higher luminance value (a luminance value close to white) as the temperature of the object is higher. The horizontal position on the image of the same object is shifted between the image captured by the camera 1R, that is, the right image, and the image captured by the camera 1L, that is, the left image. The distance can be calculated.

  In step S14, the right image is used as a reference image (alternatively, the left image may be used as a reference image), and the image signal is binarized. Specifically, a process is performed in which a region brighter than the luminance threshold ITH determined in advance by simulation or the like is set to “1” (white) and a dark region is set to “0” (black). Here, the threshold value ITH is set to a value that distinguishes an object having a temperature higher than a predetermined value, such as a human being and an animal, from the background. Thus, the high temperature object is extracted as a white region.

  In step S15, the binarized image data is converted into run-length data. For the area that has become white by binarization, for each pixel line, the start point of the line (the pixel at the left end of the line) and the length from the start point to the end point (the pixel at the right end of the line) Represents run-length data. For example, if a certain line starts from coordinates (x3, y5) and consists of three pixels, it is represented by run-length data (x3, y5, 3).

  In steps S16 and S17, the object is labeled and a process for extracting the object is performed. That is, of the lines converted into run length data, a line having a portion overlapping in the y direction is regarded as one object and given a predetermined label. In this way, the object to which the label is attached is extracted.

  In step S18, the center G of the extracted object, the area S, and the aspect ratio ASPECT of the circumscribed rectangle circumscribed by the extracted object, that is, the circumscribed rectangle, are calculated. The area S is calculated by integrating the lengths of run length data for the same object. The coordinates of the center of gravity G are calculated as the x coordinate of a line that bisects the area S in the x direction and the y coordinate of a line that bisects the area S in the y direction. The aspect ratio ASPECT is calculated as a ratio Dy / Dx between the length Dy in the y direction and the length Dx in the x direction of the circumscribed square. Note that the position of the center of gravity G may be substituted by the position of the center of gravity of the circumscribed rectangle.

  In step S19, tracking of the target object for the time (tracking), that is, recognition of the same target object is performed every predetermined sampling period. The sampling period may be the same as the period in which the process of FIG. 3 is performed. Specifically, when the time obtained by discretizing the time t as an analog quantity with the sampling period is k and the object A is extracted at the time k, the object A and the time at the next sampling period are considered. The identity with the object B extracted at (k + 1) is determined. The identity determination can be performed according to a predetermined condition. For example, 1) the difference between the X and Y coordinates of the position of the center of gravity G on the images of the objects A and B is smaller than a predetermined allowable value, and 2) the area on the image of the object B on the image of the object A If the ratio of the aspect ratio of the circumscribed rectangle of the object B to the aspect ratio of the circumscribed rectangle of the object A is smaller than the predetermined allowable value, the objects A and B Can be determined to be the same.

  Thus, in each sampling period, with respect to the extracted object, the position of the object (in this embodiment, the position coordinate of the center of gravity G) is stored in the memory as time-series data together with the assigned label.

  Note that the processes in steps S14 to S19 described above are executed for a binarized reference image (in this example, the right image).

  In step S20, the vehicle speed VCAR detected by the vehicle speed sensor 6 and the yaw rate YR detected by the yaw rate sensor 5 are read, and the turning angle θr (described later) of the vehicle 10 is calculated by integrating the yaw rate YR over time.

  On the other hand, in steps S31 to S33, a process of calculating a distance z from the vehicle 10 to the object is performed in parallel with the processes of steps S19 and S20. Since this calculation requires a longer time than steps S19 and S20, it may be executed at a cycle longer than that of steps S19 and S20 (for example, a cycle about three times the execution cycle of steps S11 to S20).

In step S31, one of the objects to be tracked by the binarized image of the reference image (in this example, the right image) is selected, and this is surrounded by a search image R1 (here, a circumscribed rectangle). Let the image area be a search image). In step S32, an image of the same object as the search image R1 (hereinafter referred to as a corresponding image) is searched for in the left image. Specifically, it can be performed by executing a correlation calculation between the search image R1 and the left image. The correlation calculation is performed according to the following formula (1). This correlation calculation is performed using a grayscale image, not a binary image.

  Here, the search image R1 has M × N number of pixels, and IR (m, n) is a luminance value at the position of the coordinates (m, n) in the search image R1, and IL (a + m−M , B + n−N) is a luminance value at the position of the coordinates (m, n) in the local area having the same shape as the search image R1 with the predetermined coordinates (a, b) in the left image as a base point. The position of the corresponding image is specified by determining the position where the luminance difference sum C (a, b) is minimized by changing the coordinates (a, b) of the base point.

  Alternatively, a region to be searched for in the left image may be set in advance, and correlation calculation may be performed between the search image R1 and the region.

In step S33, the distance dR (number of pixels) between the centroid position of the search image R1 and the image center line (the line that bisects the captured image in the x direction) LCTR of the captured image, the centroid position of the corresponding image, and the image center line LCTR Distance dL (number of pixels) is obtained and applied to Equation (2) to calculate the distance z to the object of the vehicle 10.

  Here, B is the base line length, that is, the distance in the x direction (horizontal direction) between the center position of the image sensor of the camera 1R and the center position of the image sensor of the camera 1L (that is, the distance between the optical axes of both cameras). F denotes the focal length of the lenses 12R and 12L provided in the cameras 1R and 1L, and p denotes the pixel interval in the image sensors 11R and 11L. Δd (= dR + dL) indicates the magnitude of parallax.

In step S21, the coordinates (x, y) in the image of the position of the object (as described above, the position of the center of gravity G in this embodiment) and the distance z calculated by Expression (2) are expressed in Expression (3). Apply and convert to real space coordinates (X, Y, Z). Here, the real space coordinates (X, Y, Z) are, as shown in FIG. As shown in the figure, it is expressed in a coordinate system in which the x-axis is defined in the vehicle width direction of the vehicle 10, the y-axis is defined in the vehicle height direction, and the z-axis is defined in the traveling direction of the vehicle 10. As shown in FIG. 4B, the coordinates on the image are represented by a coordinate system in which the center of the image is the origin, the horizontal direction is the x axis, and the vertical direction is the y axis.

  Here, (xc, yc) is the coordinate (x, y) on the right image based on the relative position relationship between the mounting position of the camera 1R and the origin O of the real space coordinate system and the image. Are converted into coordinates in a virtual image that coincides with the center of the image. F is a ratio between the focal length F and the pixel interval p.

  In step S22, the turning angle correction for correcting the positional deviation on the image due to the turning of the vehicle 10 is performed. During the period from time k to (k + 1), when the vehicle 10 turns, for example, by a turning angle θr in the left direction, the image obtained by the camera is shifted in the x direction (positive direction) by Δx. Therefore, this is corrected.

Specifically, the real space coordinates (X, Y, Z) are applied to Equation (4) to calculate the corrected coordinates (Xr, Yr, Zr). The calculated real space position data (Xr, Yr, Zr) is stored in the memory in time series in association with each object. In the following description, the corrected coordinates are indicated as (X, Y, Z).

In step S23, from the N real space position data (for example, about N = 10) after turning angle correction obtained within the period of ΔT for the same object, that is, from the time-series data, the vehicle 10 of the object. An approximate straight line LMV corresponding to the relative movement vector with respect to is obtained. Specifically, when the direction vector L = (lx, ly, lz) (| L | = 1) indicating the direction of the approximate straight line LMV, the straight line represented by the equation (5) is obtained.

Here, u is a parameter that takes an arbitrary value. Xav, Yav, and Zav are the average value of the X coordinate, the average value of the Y coordinate, and the average value of the Z coordinate of the real space position data string, respectively. By deleting the parameter u, the equation (5) is expressed as the following equation (5a).
(X-Xav) / lx = (Y-Yav) / ly = (Z-Zav) / lz
(5a)

  FIG. 5 is a diagram for explaining the approximate straight line LMV. P (0), P (1), P (2),..., P (N-2), P (N-1) indicate time-series data after the turning angle correction, and the approximate straight line LMV is It is obtained as a straight line that passes through the average position coordinates Pav (= (Xav, Yav, Zav)) of the time series data and has the minimum mean value of the square of the distance from each data point. Here, the numerical value in () attached to P indicating the coordinates of each data point indicates that the data is past data as the value increases. For example, P (0) indicates the latest position coordinates, P (1) indicates the position coordinates one sample period before, and P (2) indicates the position coordinates two sample periods before. The same applies to X (j), Y (j), Z (j) and the like in the following description. A more detailed method for calculating the approximate straight line LMV is described in Japanese Patent Laid-Open No. 2001-6096.

Next, the latest position coordinate P (0) = (X (0), Y (0), Z (0)) and the position coordinate P (N−) before (N−1) (that is, before time ΔT). 1) = (X (N−1), Y (N−1), Z (N−1)) is corrected to a position on the approximate straight line LMV. Specifically, the corrected position coordinates Pv (0) = (Xv (0) are obtained by using the expression (6) obtained by applying the Z coordinates Z (0) and Z (n−1) to the expression (5a). ), Yv (0), Zv (0)) and Pv (N-1) = (Xv (N-1), Yv (N-1), Zv (N-1)).

  A vector from the position coordinates Pv (N−1) calculated by Expression (6) toward Pv (0) is calculated as a relative movement vector.

  In this way, by calculating the approximate straight line LMV that approximates the relative movement trajectory of the object with respect to the vehicle 10 from a plurality (N) of data within the monitoring period ΔT, and determining the relative movement vector, the influence of the position detection error is obtained. And the possibility of collision with the object can be predicted more accurately.

  Here, referring to FIG. 6, an imaging range AR0, which is an area that can be imaged by the cameras 1R and 1L, is shown. The processing in steps S11 to S23 in FIG. 3 is performed on the captured image corresponding to the imaging range AR0.

  The area AR1 corresponds to an area corresponding to a range obtained by adding a margin β (for example, about 50 to 100 cm) on both sides of the vehicle width α of the vehicle 10, in other words, the central axis of the vehicle 10 in the vehicle width direction. It is an area having a width of (α / 2 + β) on both sides, and is an approach determination area with a high possibility of collision if the object continues to exist as it is. Areas AR2 and AR3 are areas where the absolute value of the X coordinate is larger than that of the approach determination area (outside in the lateral direction of the approach determination area), and an object in this area may enter the approach determination area. It is a judgment area. These areas AR1 to AR3 have a predetermined height H in the Y direction and a predetermined distance Z1 in the Z direction.

In step S24, objects existing in the areas AR1 to AR3 are extracted. In other words, the relative velocity Vs in the Z direction is calculated by the equation (7), and the object that satisfies the equations (8) and (9) is extracted.
Vs = (Zv (N−1) −Zv (0)) / ΔT (7)
Zv (0) / Vs ≦ T (8)
| Yv (0) | ≦ H (9)

  Here, Zv (0) is attached to indicate that the latest distance detection value (v is data after correction by the approximate straight line LMV, but the Z coordinate is the same value as before correction. Zv (N−1) is a distance detection value before time ΔT. T is an allowance time, and is intended to determine the possibility of a collision by a time T before the predicted collision time, and is set to about 2 to 5 seconds, for example. Vs × T corresponds to the predetermined distance Z1 between the areas AR1 to AR3. H defines a range in the Y direction, that is, the height direction, and is set to about twice the vehicle height of the host vehicle 10, for example. This indicates the predetermined height H of the areas AR1 to AR3. Thus, the objects in the areas AR1 to AR3 that are limited to the predetermined height H in the vertical direction and limited to the predetermined distance Z1 in the distance direction are determined to have a possibility of collision and extracted.

  Next, an approach determination process is performed to determine whether or not the object thus extracted is within the approach determination area AR1. Specifically, it is determined whether or not the X coordinate Xv (0) of the position Pv (0) of the object exists in the area AR1. If this determination is Yes, it is determined that the possibility of collision is high, and the process immediately proceeds to the object determination process in step S25. If this determination is No, it indicates that the object exists in the area AR2 or AR3, and an intrusion determination process is performed.

In the intrusion determination process, the latest x-coordinate xc (0) on the image of the object (c is a coordinate that has been corrected to make the center position of the image coincide with the real space origin O as described above. And the difference between the x coordinate xc (N−1) before the time ΔT satisfies whether or not the expression (12) is satisfied. If satisfied, it is determined that there is a high possibility of a collision with the vehicle 10 due to the object entering the approach determination area AR1 by movement, and the process proceeds to the object determination process in step S25. If Expression (10) is not satisfied, it is determined that there is no object that may possibly collide this time in the areas AR1 to AR3, and the process of FIG.

Here, the basis of the above formula will be briefly described. Referring to FIG. 7, a straight line passing through the latest position coordinates of the object 20 and the position coordinates before time ΔT, that is, an approximate straight line LMV, and an XY plane (a plane including the X axis and the Y axis, that is, the tip of the vehicle 10). When the X coordinate of the intersection point with the line (X axis) corresponding to the section and the plane perpendicular to the traveling direction of the vehicle 10 is XCL, the condition for the occurrence of the collision considering the vehicle width α is expressed by Equation (11). Given.
-Α / 2 ≦ XCL ≦ α / 2 (11)

On the other hand, a straight line obtained by projecting the approximate straight line LMV onto the XZ plane is given by Expression (12).

Substituting Z = 0 and X = XCL into this equation yields XCL as shown in equation (13).

Further, since there is a relationship as shown in Equation (3) between the real space coordinate X and the coordinate xc on the image, the following equation is established.
Xv (0) = xc (0) × Zv (0) / f (14)
Xv (N-1) = xc (N-1) * Zv (N-1) / f (15)

When these are applied to Expression (13), the intersection X coordinate XCL is given by Expression (16). Substituting this into equation (11) and rearranging gives equation (10) above.

  Returning to step S25 in FIG. 3, is the object determination process performed on the object that is determined to have a high possibility of collision in the approach determination process and the intrusion determination process, and is the object to be a target for alerting? Judge whether. For example, pedestrians and animals can be targeted for attention. The process of determining a pedestrian and an animal can be performed by any appropriate technique (for example, described in JP 2006-185434 A and JP 2007-310705 A).

  If it is determined that the object is a target for alerting, an alarm determination process is performed in step S26 to determine whether or not an alarm is actually output to the driver. If the determination result is affirmative, an alarm is output.

  In this embodiment, in the alarm determination process, it is determined whether an alarm is actually output according to the operation of the brake. Specifically, it is determined from the output of the brake sensor 7 whether or not the driver of the vehicle 10 is performing a brake operation. If the brake operation is not performed, an alarm is output. When the brake operation is performed, the acceleration Gs generated by the braking operation is calculated (the deceleration direction is positive). When the acceleration Gs is equal to or less than a predetermined threshold GTH (Gs ≦ GTH), an alarm is output. When Gx> GTH, it is determined that the collision is avoided by the brake operation, and the process is terminated. As a result, when an appropriate brake operation is performed, an alarm is not issued, so that the driver is not bothered excessively. However, alternatively, an alarm may be issued without determining brake operation.

Here, the threshold value GTH can be determined as shown in Expression (17). This is a value corresponding to a condition in which the vehicle 10 stops at a travel distance equal to or less than the distance Zv (0) when the brake acceleration Gs is maintained as it is.

  In the alarm output, an audio alarm is issued through the speaker 3 and, as shown in FIG. 8, an image obtained by the camera 1R, for example, is displayed on the screen 4a by the HUD 4, and the recognized object is displayed. The sound alarm may be a simple buzzer sound or a message sound to call attention. Further, the display form of the object may be arbitrary, and for example, it can be highlighted by surrounding it with a colored frame. An alarm may be issued by one of sound and image display. In this way, the driver can more reliably recognize an object with a high possibility of collision.

  The above is the overall overview of the object recognition process. Next, the processing range changing process according to the reaction time by the reaction time detecting unit 8 and the processing range changing unit 9 referred to with reference to FIG. 1 will be described.

  The reaction time detection unit 8 measures the time from when the alarm is issued until a predetermined operation by the driver is started, that is, the reaction time using, for example, a timer. The predetermined operation is detected by the reaction time detector 8, and in this embodiment, includes an operation of braking the vehicle and an operation on the steering wheel (handle) of the vehicle. The operation of braking the vehicle is an operation of decelerating the vehicle. For example, an operation in which the driver depresses the brake pedal, an operation in which the driver releases the depression of the accelerator pedal, and an operation in which a so-called engine brake is activated. Operations (e.g., changing the shift position, etc.) can be included. The braking operation can be detected based on, for example, whether the vehicle speed has been reduced from the detection value of the vehicle speed sensor 6. Alternatively or additionally, whether or not the brake pedal has been depressed can be detected by the brake sensor 7, and whether or not the depression of the accelerator pedal has been released is determined by an accelerator pedal opening sensor (not shown). Can be detected. The operation on the steering wheel of the vehicle is an operation of turning the vehicle, and can be detected by, for example, a sensor that detects the steering angle of the steering wheel.

  The processing range changing unit 9 changes a predetermined range (hereinafter simply referred to as “processing range”) around the vehicle, which is a processing target for recognizing an object, according to the detected reaction time. Specifically, if the detected reaction time is smaller than a predetermined threshold, the processing range is set as a reference range so that an object is recognized from the reference range, and if the detected reaction time is equal to or greater than the threshold, For example, the processing range is expanded from the reference range, and the object is recognized from the expanded range.

Here, the processing range includes the following two cases A and B.
A: Range to be processed in steps S11 to S23 in FIG.
B: Range to be processed in steps S24 and S25 in FIG.

  As described with reference to FIG. 6, in steps S21 to S23, processing is performed on the captured image corresponding to the imaging range AR0. AR0 is determined by the viewing angles (horizontal (X) and vertical (Y) directions) of cameras 1R and 1L, and is not limited in the distance (Z) direction. In addition, since the camera 1R and the camera 1L are separately arranged, the imaging range of both is actually different, but AR0 in the figure indicates a common part of the imaging range of both cameras, and both cameras Are assumed to have the same viewing angle (this also applies to the following description). On the other hand, in step S24, the range for recognizing the object is limited by setting the areas AR1 to AR3. In the regions AR1 to AR3, the viewing angle in the horizontal direction is the same as that of AR0. However, as described above, in this case, the vertical direction is limited by the predetermined height H, and the distance direction is the predetermined distance Z1 (= Vs × T). In step S25, an object in the limited range is determined.

  In the following description, when the processing ranges of cases A and B are distinguished and described, the former is referred to as “image processing range” and the latter is referred to as “recognition processing range”. In any case, the calculation load can be reduced as the processing range is reduced.

The following sub-cases can be considered for changing the processing range for case A.
A-1: The imaging range AR0 is set as a processing range, and the size of the imaging range AR0 itself is changed.
A-2: A predetermined image area is set as a processing range in the captured image of the imaging range AR0, and the size of the image area is changed.

  Hereinafter, a method for changing the processing range will be described for each case with reference to FIGS.

  FIG. 9 shows Case A-1. In this case, the imaging cameras 1R and 1L are configured with cameras that can change at least the viewing angle in the horizontal direction. (A) shows the image processing range PR0 set as the reference range, which coincides with the range AR0 captured by the cameras 1R and 1L at a predetermined viewing angle in the horizontal and vertical directions. There is no restriction in the distance direction. In the figure, the viewing angle θd1 in the horizontal direction is shown. In the image processing range PR0, as described with reference to FIG. 6, the approach determination area AR1 having the width W1 (= α / 2 + β) and the intrusion determination areas AR2 and AR3 are set. It is limited by Z1 in the direction and H in the vertical direction.

  (B) shows the imaging range AR0 enlarged by expanding the horizontal viewing angle of the cameras 1R and 1L to θd2 (> θd1), that is, the image processing range PR0 set to the enlarged range. The viewing angle in the vertical direction may or may not be widened depending on the camera specifications. The image processing range PR0 is not limited in the distance direction. With this expansion, the intrusion determination areas AR2 and AR3 are also expanded in the horizontal direction. The approach determination area AR1 is not enlarged. Restrictions on the distance direction and the vertical direction of the areas AR1 to AR3 are maintained at Z1 and H, respectively.

  The processing range changing unit 9 maintains the image processing range PR0 within the reference range (a) when the reaction time is smaller than the predetermined threshold. The image processing unit 2 executes the processes of steps S11 to S23 in FIG. 3 for the image processing range PR0 set as the reference range. On the other hand, if it is detected that the reaction time is equal to or greater than the predetermined threshold value, the processing range changing unit 9 expands the horizontal viewing angle of the cameras 1R and 1L to θd2 (> θd1), so that (b) As shown in FIG. 4, the imaging range AR0 is enlarged, and thereby the image processing range PR0 is changed to the enlarged range. As a result, an image corresponding to the image processing range expanded in the horizontal direction is acquired. The image processing unit 2 performs the processes of steps S11 to S23 in FIG. 3 on the captured image corresponding to the enlarged image processing range. In both cases (a) and (b), in step S24, the objects existing in the areas AR1 to AR3 restricted in the distance direction and the vertical direction by the expressions (8) and (9) are recognized.

  Since the processing range PR0 of (b) is expanded in the horizontal direction as compared with the processing range PR0 of (a), it is possible to recognize an object that is located farther from the vehicle 10 in the horizontal direction. . Therefore, even when the reaction time is long, the driver can recognize an object at a farther place at an early stage and can start a predetermined driving operation at an early stage.

  FIG. 10 shows Case A-2. In this case, since the horizontal and vertical viewing angles of the cameras 1R and 1L are constant, the imaging range AR0 is not changed. However, by changing the size of the image area to be processed in the captured image, The processing range PR0 is changed.

  That is, (a) shows the image processing range PR0 set as the reference range, and the horizontal viewing angle θd3 is smaller than the predetermined viewing angle θd4 in the horizontal direction of the cameras 1R and 1L. Therefore, the image processing range PR0 is smaller in the horizontal direction than the imaging range AR0. The vertical direction may be small or not small. There is no restriction in the distance direction. In the image processing range PR0, as described above, the approach determination area AR1 and the intrusion determination areas AR2 and AR3 are set, and these are limited by Z1 in the distance direction and by H in the vertical direction.

  (B) shows the image processing range PR0 set as the enlargement range, which is enlarged to coincide with the imaging range AR0. The vertical direction may be enlarged or may not be enlarged. The image processing range PR0 is not limited in the distance direction. With this expansion, the intrusion determination areas AR2 and AR3 are also expanded in the horizontal direction. The approach determination area AR1 is not enlarged. Restrictions on the distance direction and the vertical direction of the areas AR1 to AR3 are maintained at Z1 and H, respectively.

  When the reaction time is smaller than the predetermined threshold, the processing range changing unit 9 maintains the image processing range PR0 in the reference range of (a), and the image processing unit 2 sets the image processing range PR0 set to the reference range. On the other hand, the process of step S11-S23 of FIG. 3 is performed. Since the image processing range PR0 at this time is smaller than the imaging range AR0, it corresponds to a partial image area of the captured image. For example, only a part of the image area of the captured image acquired in step S11 is processed by the A / D conversion in step S12, and the subsequent image processing is performed only for the part of the image area.

  On the other hand, if it is detected that the reaction time is equal to or greater than the predetermined threshold, the processing range changing unit 9 changes the image processing range PR0 to the enlarged range as shown in (b). As a result, the entire captured image corresponding to the image processing range PR0 expanded in the horizontal direction is the processing target of steps S11 to S23 of FIG. In both cases (a) and (b), in step S24, objects existing in the areas AR1 to AR3 restricted in the distance direction and the vertical direction by the expressions (8) and (9) are recognized.

  Since the image processing range PR0 of (b) is expanded in the horizontal direction as compared with the image processing range PR0 of (a), it is possible to recognize an object that is located farther from the vehicle 10 in the horizontal direction. it can. Therefore, even when the reaction time is long, the driver can recognize an object at a farther place at an early stage and can start a predetermined driving operation at an early stage.

  In the above example, the image processing range PR0 is enlarged so as to coincide with the imaging range AR0. However, if it falls within the imaging range AR0, it may be enlarged so as not to coincide.

  FIG. 11 is another example of case A-2. In FIG. 10, the image processing range PR0 is changed in the horizontal direction, but in FIG. 11, the image processing range PR0 is changed in the distance direction.

  That is, (a) shows the image processing range PR0 set as the reference range, which is a predetermined viewing angle in the horizontal and vertical directions of the cameras 1R and 1L (in the figure, the viewing angle θd5 in the horizontal direction is It is an area limited by a predetermined value Zth1 in the distance direction of the range AR0 to be imaged. In the image processing range PR0, as described with reference to FIG. 6, the approach determination area AR1 having the width of W1 (= α / 2 + β) and the intrusion determination areas AR2 and AR3 are set. Is limited by Z1 and by H in the vertical direction. Here, Z1 is set to be equal to or less than Zth1.

  (B) shows the image processing range PR0 set as the enlargement range, which is enlarged to a predetermined value Zth2 (> Zth1) in the distance direction. It is not enlarged in the horizontal and vertical directions. As this enlargement is performed, the areas AR1 to AR3 are also enlarged to the predetermined value Z2 in the distance direction. Z2 is set to be equal to or less than Zth2. The restriction in the vertical direction of the areas AR1 to AR3 remains the predetermined value H.

  The processing range changing unit 9 maintains the image processing range PR0 within the reference range (a) when the reaction time is smaller than the predetermined threshold. If the distance value z calculated in step S33 is larger than the predetermined value Zth1, the image processing unit 2 excludes the pixel region having the distance value z from the subsequent processing. In this way, the reference range in which the distance value z is equal to or smaller than the predetermined value Zth1 is set as the processing target in steps S21 to S23. The restriction in the distance direction and the vertical direction in step S24 is realized by predetermined values Z1 (= Vs × T) and H, respectively, as shown in the above-described equations (8) and (9).

  On the other hand, if it is detected that the reaction time is equal to or greater than the predetermined threshold, the processing range changing unit 9 changes the image processing range PR0 to the enlarged range as shown in (b). As a result, if the distance value z calculated in step S33 is larger than the predetermined value Zth2, the image processing unit 2 excludes the pixel region having the distance value z from the subsequent processing. Thus, an enlarged range in which the distance value z is equal to or smaller than the predetermined value Zth2 is set as a processing target in steps S21 to S23. In step S24, the object existing in the areas AR1 to AR3 limited by the predetermined value Z2 in the distance direction and limited by H in the vertical direction is recognized. Here, the predetermined value Z2 can be determined, for example, by multiplying the predetermined time T2 (> T) by the relative speed Vs. Alternatively, Z1 in (a) and Z2 in (b) may be the same, in which case Z2 is determined by (T × Vs).

  Since the image processing range PR0 of (b) is expanded in the distance direction as compared with the image processing range PR0 of (a), an object existing farther from the vehicle 10 is also recognized in the distance direction. Therefore, even when the reaction time is long, the driver can recognize an object at a farther place at an early stage and can start a predetermined driving operation at an early stage.

  FIG. 12 shows an enlargement of case B in the horizontal direction. In case B, as described above, the image processing range PR0 that is the processing target of steps S11 to S23 in FIG. 3 matches the imaging range AR0, but the target is finally recognized in step S24 of FIG. The recognition processing range PR1 (= AR1 + AR2 + AR3), which is a region, is changed according to the reaction time.

  That is, in both (a) and (b), the image processing range PR0 is a predetermined viewing angle in the horizontal and vertical directions of the cameras 1R and 1L (the viewing angle θd6 in the horizontal direction is shown in the figure). ) Matches the range AR0 imaged. There is no restriction in the distance direction. In the image processing range PR0, as described with reference to FIG. 6, the approach determination area AR1 having the width W1 (= α / 2 + β) and the intrusion determination areas AR2 and AR3 are set. It is limited by Z1 in the direction and H in the vertical direction. The sum of AR1 to AR3 is the recognition processing range PR1.

  The recognition processing range PR1 in (a) is set as a reference range, and is smaller than the image processing range PR0 (= imaging range AR0) as represented by the horizontal viewing angle θd7 (<θd6). (B) shows the recognition processing range PR1 expanded in the horizontal direction so as to coincide with the image processing range PR0. The vertical and distance directions are not enlarged.

  When the reaction time is smaller than the predetermined threshold, the processing range changing unit 9 maintains the recognition processing range PR0 in the reference range of (a), and the image processing unit 2 recognizes the recognition processing range PR1 set in the reference range. On the other hand, the process of step S24 in FIG. 3 is executed. Since the recognition processing range PR1 at this time is smaller than the imaging range AR0, it corresponds to a part of the image area of the captured image. For example, in step S24, in addition to the conditions of the expressions (8) and (9) described above, the condition that the latest position Pv (0) of the object exists within the recognition processing range PR1 as shown in (a). When these conditions are satisfied, the approach determination process and the intrusion determination process described above can be performed.

  On the other hand, if it is detected that the reaction time is equal to or greater than the predetermined threshold, the processing range changing unit 9 changes the recognition processing range PR1 to an enlarged range as shown in (b). As a result, the process of step S24 in FIG. 3 is performed on the recognition processing range PR1 expanded in the horizontal direction.

  Since the recognition processing range PR1 in (b) is expanded in the horizontal direction as compared with the recognition processing range PR1 in (a), an object existing farther from the vehicle 10 is also recognized in the horizontal direction. Therefore, even when the reaction time is long, the driver can recognize an object at a farther place at an early stage and can start a predetermined driving operation at an early stage.

  In the above example, the recognition processing range PR1 is enlarged so as to match the image processing range PR0 (= imaging range AR0), but if it falls within the image processing range PR0, it is enlarged so as not to match. Also good.

  FIG. 13 shows an enlargement of case B in the distance direction. In both (a) and (b), the image processing range PR0 is a predetermined viewing angle in the horizontal and vertical directions of the cameras 1R and 1L (the viewing angle θd8 in the horizontal direction is shown in the figure). The range AR0 to be imaged matches. There is no restriction in the distance direction. As described with reference to FIG. 6, the approach determination area AR1 having the width of W1 (= α / 2 + β) and the intrusion determination areas AR2 and AR3 are set in the image processing range PR0. The direction is limited by H. The sum of AR1 to AR3 is the recognition processing range PR1.

  (A) shows the recognition processing range PR1 set as the reference range, which is limited by a predetermined value Z1 in the distance direction. (B) shows the recognition processing range PR1 limited by the predetermined value Z2 (> Z1) in the distance direction, which is enlarged in the distance direction compared to (a). It is not enlarged in the horizontal and vertical directions.

  When the reaction time is smaller than the predetermined threshold, the processing range changing unit 9 maintains the recognition processing range PR0 in the reference range of (a), and the image processing unit 2 recognizes the recognition processing range PR1 set in the reference range. On the other hand, the process of step S24 in FIG. 3 is executed. As described above, the predetermined value Z1 is Vs × T, and the limitation in the distance direction by the predetermined value Z1 is realized by the equation (8) in step S24.

  On the other hand, if it is detected that the reaction time is equal to or greater than the predetermined threshold, the processing range changing unit 9 changes the recognition processing range PR1 to an enlarged range as shown in (b). Specifically, Z2 is set to (relative speed Vs × predetermined time T2), where T2 is set such that T2> T. By changing the equation (8) in step S24 to Zv (0) / Vs ≦ T2, the limitation in the distance direction by the predetermined value Z2 can be realized.

  Since the recognition processing range PR1 of (b) is expanded in the distance direction as compared with the recognition processing range PR1 of (a), an object existing farther from the vehicle 10 is also recognized in the distance direction. Therefore, even when the reaction time is long, the driver can recognize an object at a farther place at an early stage and can start a predetermined driving operation at an early stage.

  In this embodiment, the processing range changing unit 9 has been described with reference to FIGS. 11 and 13 when the reaction time of the operation of braking the vehicle by the driver with respect to the issuance of the alarm is not less than a predetermined threshold. In this way, the processing range (either the image processing range PR0 and the recognition processing range PR1 or both may be expanded) in the distance direction. The operation of braking the vehicle is an operation of limiting the vehicle to travel in the distance direction. A long response time of the braking operation indicates that there is a possibility that a delay may occur in limiting the vehicle travel. Therefore, by expanding the processing range in the distance direction, the driver can recognize an object farther in the distance direction at an early stage. As a result, the driver can start the braking operation earlier.

  On the other hand, the processing range changing unit 9 refers to FIGS. 9, 10, and 12 when the response time of the operation on the steering wheel of the vehicle by the driver with respect to the issue of the alarm is equal to or greater than a predetermined threshold value. As described above, the processing range (either the image processing range PR0 and the recognition processing range PR1 or both may be expanded) in the horizontal direction. The operation on the steering wheel is an operation for turning the vehicle. A long reaction time with respect to the operation of the steering hole indicates that there is a risk of delay in turning the vehicle. Therefore, by enlarging in the horizontal direction, the driver can recognize an object farther in the horizontal direction at an early stage. As a result, the driver can start the turning operation earlier.

  Depending on the issuance of the alarm, both braking operation and steering wheel operation are detected, and if the reaction time is more than the threshold value for both, the processing range is expanded in the horizontal direction and in the distance direction. Also good.

  As an alternative, if the reaction time is smaller than the threshold, the width of the approach determination area AR1 is maintained at W1, and the width of the approach determination area AR1 is increased in response to the detection that the reaction time is equal to or greater than the threshold. It may be. Referring to FIG. 14, a recognition processing range PR1 having an approach determination area AR1 in which the width W1 (which is α / 2 + β as described above) is expanded to become the width W2 is shown. By such an enlargement, the capture range of the object existing in the approach determination area AR1 is expanded, so that the driver has a higher possibility of a collision earlier than the approach determination area AR1 having a narrower width W1. Recognize the existence of things. Along with the expansion of the width of the approach determination area AR1, as described with reference to FIGS. 9 to 13, the horizontal direction and / or the distance direction may be expanded.

  In the above embodiment, the size of the processing range is switched between the reference range (FIGS. 9 to 13A) and the enlarged range (FIGS. 9 to 13B) according to the reaction time. Alternatively, the treatment range may be gradually expanded as the reaction time increases. Each time the reaction time increases by a predetermined time, the horizontal direction and / or the distance direction of the processing range may be expanded by a predetermined size.

  Further, in this embodiment, the alarm output form is changed according to the detected reaction time. Specifically, if the reaction time detected by the reaction time detection unit 8 is smaller than the predetermined threshold, the image processing unit 2 issues an alarm output through the speaker 3 at a reference volume in step S26. If the reaction time is equal to or greater than the threshold value, the volume of the alarm to be output is set larger than the reference volume. As described above, the warning sound may be a simple buzzer sound or a voice message that calls attention. The volume may be gradually increased as the reaction time becomes longer.

  If the reaction time detected by the reaction time detection unit 8 is equal to or more than the above-described predetermined threshold instead of or in addition to the alarm sound, a display indicating the object on the screen 4a of the HUD 4 is displayed. Highlight. An example of the highlight display is shown in FIG. (A) is a reference display when the reaction time is smaller than a predetermined threshold, and is displayed with some emphasis by surrounding the recognized object with a frame 201. (B) is a display when the reaction time is equal to or greater than a threshold, and the frame 203 indicating the object is thicker than (a), and the display is more emphasized than (a). It has become. It can be emphasized in any form. For example, the color of the frame indicating the object is changed between (a) and (b), or some symbol or the like is displayed around the frame indicating the object. May be. As the reaction time becomes longer, the highlighted form may be changed. For example, as the reaction time becomes longer, the thickness of the frame may be gradually increased or the color of the frame may be gradually made more conspicuous to attract more attention.

  In still another embodiment, the alarm output timing may be changed according to the detected reaction time. An example of such a change technique is shown in FIG. (A) shows the alarm output timing when the reaction time is smaller than the predetermined threshold, that is, the reference alarm output timing, and (b) shows the alarm output timing when the reaction time is determined to be equal to or greater than the threshold. An object to be alerted is recognized at time t1 (step S25 in FIG. 3), and an alarm output is issued at time t2. As described above, the alarm output may be a sound or an emphasis display in an image.

  If the reaction time detection unit 8 detects that the reaction time is smaller than the predetermined threshold, the image processing unit 2 determines the time from the object recognition in step S25 to the actual output of the alarm in step S26. , Set to a predetermined period T1. On the other hand, if it is detected that the reaction time is equal to or greater than the predetermined threshold, the image processing unit 2 accordingly determines the time from the object recognition in step S25 to the actual output of the alarm in step S26. The predetermined period T1 is shortened to T2 (> T1). By doing so, an alarm is output early when the reaction is slow. Thereby, the driver can start the driving operation according to the warning at an early stage.

  In the example of the figure, the alarm output timing is set in two stages, but more stages may be provided so that the alarm output timing is gradually advanced as the reaction time becomes longer.

  FIG. 17 is a flow of a process range and alarm output mode change process executed by the reaction time detection unit 8, the processing range change unit 9, and the image processing unit 2. As shown in step S41, this process is started in response to the alarm being output in step S26 of FIG. Time is measured by a timer according to the alarm output.

  In step S42, a predetermined driving operation is detected. In this embodiment, as described above, the operation of braking the vehicle and the operation of the steering wheel are detected.

  In step S43, the time from the alarm output to the detection of the driving operation in step S42, that is, the reaction time, is obtained by the time counting.

  In step S44, the reaction time is compared with a predetermined threshold value. If the reaction time is smaller than the threshold value, the processing range (one of the image processing range PR0 and the recognition processing range PR1 described above or both may be used) When a reference range as shown in (a) of FIG. 9 to FIG. 13 is selected and the reaction time is equal to or greater than the threshold, an expanded range as shown in (b) of FIG. select. As described above, the type of the enlarged range is preferably selected according to the detected driving operation. For example, if a braking operation is detected, a processing range expanded in the distance direction as shown in FIGS. 11 and 13B is selected, and if a steering wheel operation is detected, The processing range expanded in the horizontal direction as shown in FIGS. 9, 10, and 12B is selected. If both the braking operation and the steering wheel operation are detected, the processing range expanded in both the horizontal direction and the distance direction may be selected.

  Further, in step S44, if the reaction time is smaller than the threshold, a reference volume is selected as the volume of the alarm sound, and if the reaction time is equal to or greater than the threshold, the volume of the alarm sound is larger than the reference volume. Select the volume. Further, if the reaction time is smaller than the threshold, a reference display form (for example, (a) in FIG. 15) or a display form without emphasis is selected as an alarm display, and if the reaction time is equal to or greater than the threshold, As a warning display, a display form (for example, (b) in FIG. 15) with a higher emphasis than the standard display form is selected.

  In step S45, it is determined whether or not the selected processing range is the same as the processing range that is currently used. If it is the same, exit the process. If they are different, the currently used processing range is changed to the selected processing range. The process of FIG. 3 executed thereafter is executed based on the changed processing range. For example, if the selected processing range is an enlarged range, and the currently used processing range is also an enlarged range, the size of the processing range is not changed. On the other hand, for example, if the selected processing range is an enlarged range and the currently used processing range is a reference range, the processing range is changed to the enlarged range, and the object is recognized from the enlarged range. So that

  In step S45, if the volume of the selected alarm sound and the form of the alarm display are the same as the volume of the alarm sound and the form of the alarm display currently used, the volume of the alarm sound and the form of the alarm display If it is different, the alarm sound volume and the alarm display form that are already used are changed to the selected alarm sound volume and the alarm display form. In this way, in step S26 of FIG. 3 performed after that, alarm output is performed in the form of the changed alarm sound volume and alarm display.

  In this flow, the processing range, the alarm sound, and the alarm display form are included as targets to be changed according to the reaction time. However, of course, only the processing range may be changed, or in addition to the change of the processing range. Only one of the alarm sound and the alarm display may be changed. A change in the size of the approach determination area and a change in the alarm output timing as described with reference to FIGS. 14 and 16 may be added to these changes.

  Thus, according to the present invention, when the reaction time is short, by limiting the processing range, it is possible to reduce the amount of calculation related to image processing and to improve the recognition accuracy of the object. . On the other hand, when the reaction time is long, priority is given to prompting the driver to recognize the object earlier than the amount of calculation and the recognition accuracy, so that the driver can drive for avoiding the object earlier. Allow the operation to begin. In this way, it is possible to prompt the driver to perform a driving operation so as to more reliably avoid a collision with the object.

  In addition, this invention is not restricted to said embodiment, Various deformation | transformation forms are possible. For example, in the above-described embodiment, an infrared camera is used as the imaging unit. However, for example, a television camera that can detect only normal visible light may be used (for example, JP-A-2-26490). However, by using an infrared camera, the object extraction process can be further simplified, and the calculation load can be reduced.

The block diagram which shows the structure of the periphery monitoring apparatus according to one Example of this invention. The figure for demonstrating the attachment position of the camera according to one Example of this invention. 3 is a flowchart illustrating a process in an image processing unit according to an embodiment of the present invention. The figure which shows the real space coordinate system and image coordinate system according to one Example of this invention. The figure for demonstrating the calculation method of a relative movement vector according to one Example of this invention. The figure which shows the area division ahead of the vehicle according to one Example of this invention. The figure for demonstrating the intrusion determination process according to one Example of this invention. The figure for demonstrating an example of the alarm output according to one Example of this invention. The figure for demonstrating an example which changes the process range ahead of a vehicle according to one Example of this invention. The figure for demonstrating the other example which changes the process range ahead of the vehicle according to one Example of this invention. The figure for demonstrating the other example which changes the process range ahead of the vehicle according to one Example of this invention. The figure for demonstrating the other example which changes the process range ahead of the vehicle according to one Example of this invention. The figure for demonstrating the other example which changes the process range ahead of the vehicle according to one Example of this invention. The figure for demonstrating the other example which changes the process range ahead of the vehicle according to one Example of this invention. The figure for demonstrating an example which changes the output form of the alarm according to one Example of this invention. The figure for demonstrating an example which changes the output timing of an alarm according to one Example of this invention. The flowchart of the process which changes the processing range and the output form of an alarm according to reaction time according to one Example of this invention.

Explanation of symbols

1R, 1L infrared camera (imaging means)
2 Image processing unit 3 Speaker 4 Head-up display 5 Yaw rate sensor 6 Vehicle speed sensor 7 Brake sensor 8 Reaction time detection unit 9 Processing range change unit

Claims (2)

A vehicle peripheral device that recognizes an object for a predetermined range around the vehicle based on an image captured by an imaging unit mounted on the vehicle, and issues an alarm to the driver of the vehicle with respect to the recognized object Because
A reaction time detecting means for detecting a response time of the driver's steering operation and an operation of braking the vehicle with respect to the issuing of the alarm;
If it is detected that the reaction time is equal to or greater than a predetermined value, the predetermined range is expanded, and the image capturing range for recognizing the object or a range change for expanding the image processing target range in the image Means,
Bei to give a,
If the range changing means detects that the response time of the steering operation is equal to or greater than a first predetermined value, the size of the predetermined range in the distance direction along the traveling direction of the vehicle is increased. If it is detected that the response time of the braking operation is equal to or greater than a second predetermined value, the size of the predetermined range in the vehicle width direction is increased. Expand the predetermined range,
Vehicle periphery monitoring device. Issuing the warning includes alerting the driver by sound or image display;
Further, if said the steering operation or reaction times of the braking is thereby operated by the response time detecting unit is each of the first or second predetermined value or more is detected, increasing the the sound or the image display With means to emphasize
The vehicle periphery monitoring apparatus according to claim 1 .