JP4176558B2 - Vehicle periphery display device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a vehicle periphery display device that displays a warning object that affects the traveling of the host vehicle.
[0002]
[Prior art]
A conventional vehicle periphery display device extracts an object such as a pedestrian that affects the traveling of the host vehicle from an image of the periphery of the host vehicle captured by an imaging means such as an infrared camera, and uses the information of the host vehicle. Provide to the driver. In this device, a portion having a high temperature in an image around the host vehicle captured by a pair of left and right stereo cameras is used as a target, and the distance to the target is calculated by obtaining the parallax of the target. An object that affects the traveling of the host vehicle is detected from the moving direction of the object and the position of the object. In addition, when an object that affects the traveling of the host vehicle is detected, the display of the image around the host vehicle captured by the imaging unit and the detected object are displayed on the display arranged in front of the driver's seat of the host vehicle. Is displayed simultaneously, and information on the detected object is provided to the driver of the host vehicle (see, for example, Patent Document 1).
[0003]
[Patent Document 1]
Japanese Patent Laid-Open No. 2001-6096
[Problems to be solved by the invention]
By the way, in the prior art, an object that affects the traveling of the host vehicle is detected and displayed on a display arranged in front of the driver's seat of the host vehicle, so that the driver of the host vehicle can run the host vehicle. Although it is possible to quickly recognize an object that affects the vehicle, the display around the host vehicle captured by the imaging unit and the highlighted image of the detected object are displayed on the display arranged in front of the driver's seat of the host vehicle. Is performed only when an object that affects the traveling of the host vehicle is detected, the display of the image around the host vehicle and the highlighted image of the detected object is performed on the display. If not, there is a problem that it cannot be determined whether or not the apparatus is operating normally.
[0005]
That is, for example, when the temperature around the vehicle (outside temperature) is high, or when there are many objects that emit infrared rays in the target image, this is obtained from the characteristics of the imaging means using infrared rays. Since there is no change in the brightness distribution of the infrared image and the brightness distribution is acquired uniformly, it is difficult to detect objects such as pedestrians that are infrared emitting objects, and the surrounding image of the vehicle and the detected objects are difficult to detect. In some cases, the number of times the highlighted video is displayed is reduced. In addition, even when a signal indicating the running state of the vehicle, such as the traveling speed and rotation angle information input to the device, is interrupted due to a failure of the sensor or the like, the display of the image around the own vehicle and the emphasized image of the detected object is displayed. Not executed. Therefore, if the driver of the own vehicle does not display an image around the own vehicle and an emphasized image of the detected object on the display, the driver affects the driving of the own vehicle around the own vehicle. It was not possible to determine whether there was an object or the device was not operating properly.
[0006]
The present invention has been made in view of the above problems, and an object of the present invention is to provide a vehicle periphery display device that can accurately notify the driver of the host vehicle of the operation status of the device.
[0007]
[Means for Solving the Problems]
In order to solve the above-described problem, a vehicle periphery display device according to the invention of claim 1 is an infrared camera (for example, the infrared camera 2R of the embodiment) capable of photographing the periphery of the host vehicle (for example, the host vehicle 20 of the embodiment). 2L), warning object determination means (for example, the image processing unit 11 of the embodiment) for determining a warning object that affects the traveling of the host vehicle from the image captured by the infrared camera, and the warning object When the warning object is detected by the object determination means, the image taken by the infrared camera and the warning are displayed on the display device (for example, the display device 8 in the embodiment) arranged in front of the driver's seat of the host vehicle. In a vehicle peripheral display device including warning display means (for example, the display control unit 12 of the embodiment) that simultaneously displays an emphasized image showing an object, the warning object determination means performs the If you can not perform the determination of WARNING object, the warning display means, and displaying constantly the image of the infrared camera to the display device.
[0008]
In the vehicle periphery display device having the above-described configuration, even when the determination by the warning object determination unit cannot be performed, the warning display unit always displays the image of the infrared camera on the display device, thereby allowing the driver of the host vehicle to At least the situation around the host vehicle can be confirmed.
[0009]
The vehicle periphery display device according to a second aspect of the present invention is the vehicle periphery display device according to the first aspect, wherein the warning display means displays the operation state of the warning object determination means together with the display of the image of the infrared camera. The display is performed on the display device.
[0010]
In the vehicle periphery display device having the above-described configuration, the warning display means displays the image of the infrared camera and the display indicating the operation state of the warning object determination means together on the display device. It is possible to reliably notify and grasp the operating state of the warning object determination means by an easy-to-understand method.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of a vehicle periphery display device according to an embodiment of the present invention. In FIG. 1, reference numeral 1 denotes an in-vehicle image processing unit including a CPU (Central Processing Unit) that determines a warning object that affects the traveling of the host vehicle from an image captured by an infrared camera. Two infrared cameras 2R, 2L that can detect infrared rays, a yaw rate sensor 3 that detects the yaw rate of the vehicle, a vehicle speed sensor 4 that detects the traveling speed (vehicle speed) of the vehicle, and a brake operation A brake sensor 5 and a temperature sensor 6 for measuring the temperature around the host vehicle (outside air temperature) are connected. Thereby, the image processing unit 1 detects a pedestrian or an animal in front of the vehicle from the infrared image around the vehicle and a signal indicating the running state or the surrounding environment of the vehicle, and determines that the possibility of a collision is high. Sometimes alerts.
[0012]
In addition, the image processing unit 1 is connected with a speaker 7 for issuing a warning by voice, and when it is determined that there is a high risk of collision between the detected object and the host vehicle, the infrared camera 2R, A display device 8 is connected to display an image captured by 2L together with a highlighted object image (emphasized image) and to allow the vehicle driver to recognize an object with a high risk of collision.
[0013]
On the other hand, the image processing unit 1 will be described in more detail. The image processing unit 1 includes an A / D conversion circuit that converts an input analog video signal into a digital signal, an image memory that stores a digitized image signal, and various arithmetic processes. CPU (Central Processing Unit) to be performed, RAM (Random Access Memory) used for storing data being calculated by the CPU, ROM (Read Only Memory) storing programs, tables, maps, etc. executed by the CPU, speaker 7 And an image processing unit 11 having an output circuit for outputting the driving signal, and a display control unit for outputting a control signal related to display of the video signal and alarm still image data (icon) output from the image processing unit 1 to the display device 8. 12 and an alarm stationary output that the display control unit 12 outputs to the display device 8 according to the content of the alarm or the state of the image processing unit 1 And a warning for the still image data storage unit 13 for holding data.
[0014]
In the image processing unit 11, the output signals of the infrared cameras 2R and 2L, the yaw rate sensor 3, the vehicle speed sensor 4, the brake sensor 5, and the temperature sensor 6 are converted into digital signals and input to the internal CPU. It is configured.
The display control unit 12 also includes a display position adjustment switch and a luminance adjustment volume (both shown) provided on the console of the host vehicle for manually adjusting the position and luminance of the video display area in the display device 8. Connected).
[0015]
In addition, the display device 8 specifically includes, for example, a HUD (Head Up Display) 8a that displays information at a position that does not obstruct the driver's front view of the front window shown in FIG. A meter-integrated display 8b integrated with the meter represented by (2), NAVID display 8c installed on the console of the host vehicle, and the like.
[0016]
As shown in FIG. 3, the infrared cameras 2R and 2L are disposed at the front portion of the host vehicle 20 at positions that are substantially symmetrical with respect to the center of the host vehicle 20 in the vehicle width direction. The optical axes of 2R and 2L are parallel to each other and are fixed so that the heights from both road surfaces are equal. The infrared cameras 2R and 2L have a characteristic that the output signal level increases (the luminance increases) as the temperature of the object increases.
The HUD 8a is provided so that the display screen is displayed at a position that does not obstruct the driver's front view of the front window of the host vehicle 20.
[0017]
Next, the operation of the present embodiment will be described with reference to the drawings.
4 and 5 are flowcharts showing an object detection / alarm operation by the image processing unit 1 of the vehicle periphery display device of the present embodiment.
First, the image processing unit 1 acquires infrared video that is an output signal of the infrared cameras 2R and 2L (step S1), performs A / D conversion (step S2), and stores a grayscale image in an image memory (step S1). S3). Here, the right image is obtained by the infrared camera 2R, and the left image is obtained by the infrared camera 2L. In addition, since the horizontal position of the same object on the display screen is shifted in the right image and the left image, the distance to the object can be calculated from this shift (parallax).
If a grayscale image is obtained in step S3, the right image obtained by the infrared camera 2R is used as a reference image, and the density value of the image signal is statistically processed to calculate the contrast of the reference image ( Step S4).
[0018]
Then, it is determined whether the calculated contrast of the reference image is lower than a predetermined value (step S5).
If the calculated contrast of the reference image is greater than or equal to a predetermined value in step S5 (NO in step S5), the reference image is binarized, that is, an area brighter than the luminance threshold ITH is “1” (white). And processing for setting the dark area to “0” (black) (step S6).
[0019]
If binarized image data is acquired from the infrared image, a process of converting the binarized image data into run-length data is performed (step S7). The line represented by the run-length data is an area that is whitened by binarization at the pixel level, and has a width of one pixel in the y direction, and each has a width in the x direction. It has the length of the pixels constituting the run-length data.
[0020]
Next, the target object is labeled from the image data converted into run-length data (step S8), thereby performing a process of extracting the target object (step S9). That is, a binarized object is extracted by regarding a line having a portion overlapping in the y direction as one object among the lines converted to run length data. When the extraction of the binarized object is completed, the center G of the extracted object, the area S, and the aspect ratio ASPECT of the circumscribed rectangle are calculated (step S10).
[0021]
Here, the area S is (x [i], y [i], run [i], A) (i = 0, 1, 2,... N−1) ), The length of the run length data (run [i] −1) is calculated for the same object (N pieces of run length data). Also, the coordinates (xc, yc) of the center of gravity G of the object A are the length (run [i] -1) of each run length data and the coordinates x [i] or y [i] of each run length data. Are multiplied by the area S and calculated by multiplying them by the area S. Further, the aspect ratio ASPECT is calculated as a ratio Dy / Dx between the length Dy in the vertical direction and the length Dx in the horizontal direction of the circumscribed rectangle of the object.
Since the run-length data is indicated by the number of pixels (number of coordinates) (= run [i]), the actual length needs to be “−1” (subtract 1) (= run [i]. ] -1). Further, the position of the center of gravity G may be substituted by the position of the center of gravity of the circumscribed rectangle.
[0022]
Once the center of gravity, area, and circumscribing aspect ratio of the object can be calculated, next, the object is tracked between times, that is, the same object is recognized for each sampling period (step S11). For tracking between times, the time obtained by discretizing the time t as an analog quantity with the sampling period is set as k. For example, when the objects A and B are extracted at the time k, the objects C and D extracted at the time (k + 1) and the target The identity determination with the objects A and B is performed. When it is determined that the objects A and B and the objects C and D are the same, the objects C and D are changed to labels of the objects A and B, respectively, so that tracking is performed for a time.
Further, the position coordinates (center of gravity) of each object recognized in this way are stored in the memory as time-series position data, and are used for later calculation processing.
[0023]
Note that the processes in steps S4 to S11 described above are executed for a binarized reference image (a right image in the present embodiment).
Next, the vehicle speed VCAR detected by the vehicle speed sensor 4, the yaw rate YR detected by the yaw rate sensor 3, and the temperature (outside temperature) around the host vehicle 20 detected by the temperature sensor 6 are acquired (step S12). .
[0024]
Then, by determining whether or not the acquired temperature around the host vehicle 20 (outside temperature) is within a predetermined temperature range, or whether the values of vehicle information such as the vehicle speed VCAR and the yaw rate YR are within a predetermined range. Then, it is determined whether or not each data is normal (step S13).
If the acquired ambient temperature (outside air temperature) of the host vehicle 20 is within a predetermined temperature range and vehicle information values such as the vehicle speed VCAR and the yaw rate YR are within the predetermined range in step S13, each data Is determined to be normal (YES in step S13), and the image processing unit 1 calculates the turning angle θr of the host vehicle 20 by integrating the yaw rate YR acquired in step S12 with time. Using the turning angle θr, the turning angle correction for correcting the positional deviation on the image due to the turning of the host vehicle 20 is performed (step S14).
[0025]
Here, in the turning angle correction, when the host vehicle 20 turns, for example, to the left by the turning angle θr during the period from time k to (k + 1), the range of the image is Δx in the x direction on the image obtained by the camera. This is a process for correcting this.
In the following description, the coordinates after the turning angle correction are displayed as (X, Y, Z).
[0026]
On the other hand, in parallel with the processing in step S11 and step S12, in step S15 to step S17, processing for calculating the distance z between the object and the host vehicle 20 is performed. Since this calculation requires a longer time than steps S11 and S12, the calculation is executed in a cycle longer than that in steps S11 and S12 (for example, a cycle about three times the execution cycle of steps S1 to S12).
First, by selecting one of the objects tracked by the binarized image of the reference image (right image), the search image R1 (here, the entire region surrounded by the circumscribed rectangle is searched for from the right image). Are extracted (step S15).
[0027]
Next, a search area for searching for an image corresponding to the search image R1 (hereinafter referred to as “corresponding image”) from the left image is set, and a correlation operation is executed to extract the corresponding image (step S16). Specifically, a search area R2 is set in the left image according to each vertex coordinate of the search image R1, and a luminance difference total value C (a indicating the level of correlation with the search image R1 within the search area R2 , B), and a region where the total value C (a, b) is minimum is extracted as a corresponding image. This correlation calculation is performed using a grayscale image instead of a binarized image.
When there is past position data for the same object, an area R2a narrower than the search area R2 is set as a search area based on the position data.
[0028]
The search image R1 is extracted from the reference image (right image) and the corresponding image R4 corresponding to the target object is extracted from the left image by the processing in step S16. Next, the search image R1 corresponds to the center of gravity position of the search image R1. The position of the center of gravity of the image R4 and the amount of parallax Δd (number of pixels) are obtained, and the distance z between the host vehicle 20 and the object is calculated therefrom (step S17).
[0029]
When the turning angle correction in step S14 is completed, the coordinates (x, y) in the image and the distance z to the object calculated in step S17 are converted into real space coordinates (X, Y, Z) (steps). S18).
Here, the real space coordinates (X, Y, Z) are, as shown in FIG. 3, with the origin O as the midpoint position (position fixed to the host vehicle 20) of the attachment position of the infrared cameras 2R, 2L. The coordinates in the image are determined as shown in the figure, with the center of the image as the origin and the horizontal direction as x and the vertical direction as y.
[0030]
Once the real space coordinates are obtained, next, N pieces (for example, about N = 10) of real space position data after turning angle correction, that is, time-series data obtained within the monitoring period of ΔT for the same object. From this, an approximate straight line LMV corresponding to the relative movement vector between the object and the host vehicle 20 is obtained.
Next, the latest position coordinates P (0) = (X (0), Y (0), Z (0)) and (N-1) position coordinates P (N−1) before the sample (before time ΔT). = (X (N-1), Y (N-1), Z (N-1)) is corrected to a position on the approximate straight line LMV, and the corrected position coordinates Pv (0) = (Xv (0), Yv (0), Zv (0)) and Pv (N-1) = (Xv (N-1), Yv (N-1), Zv (N-1)) are obtained.
[0031]
Thereby, a relative movement vector is obtained as a vector from the position coordinates Pv (N−1) to Pv (0) (step S19).
In this way, by calculating an approximate straight line that approximates the relative movement locus of the object with respect to the host vehicle 20 from a plurality (N) of data within the monitoring period ΔT, the influence of the position detection error is reduced. Thus, the possibility of collision with the object can be predicted more accurately.
[0032]
If the relative movement vector is obtained in step S19, an alarm determination process for determining the possibility of collision with the detected object is performed (step S20). In the alarm determination process, when the object approaches the host vehicle 20 at the relative speed Vs, it is assumed that both move while maintaining the relative speed Vs. The collision determination process for determining whether or not the target object is in a range in which a margin β (for example, about 50 to 100 cm) is added to both sides of the vehicle width α of the host vehicle 20 and continues to exist if it continues to exist. A determination process for determining whether or not the vehicle 20 is in the approach determination area where the possibility of a collision with the vehicle 20 is extremely high. Further, the target object enters the above-described approach determination area and collides with the host vehicle 20. An approach collision determination process for determining whether or not there is a possibility is performed, and the possibility of a collision between the detected object and the host vehicle 20 is determined.
[0033]
If it is determined in step S20 that there is no possibility of collision between the host vehicle 20 and the detected object (NO in step S20), the process returns to step S1 and the above-described processing is repeated.
If it is determined in step S20 that there is a possibility of a collision between the host vehicle 20 and the detected object (YES in step S20), the process proceeds to an alarm output determination process in step S21.
[0034]
In step S21, it is determined whether or not the driver of the host vehicle 20 is performing a brake operation from the output BR of the brake sensor 5, thereby determining whether or not to perform alarm output determination processing, that is, whether or not alarm output is performed ( Step S21).
If the driver of the host vehicle 20 is performing a brake operation, the acceleration Gs generated (the deceleration direction is positive) is calculated. If the acceleration Gs is greater than a predetermined threshold GTH, the brake is It is determined that the collision is avoided by the operation, the alarm output determination process is terminated (NO in step S21), the process returns to step S1, and the above process is repeated. As a result, when an appropriate brake operation is performed, an alarm is not issued, and the driver is not bothered excessively.
[0035]
Further, when the acceleration Gs is equal to or less than the predetermined threshold GTH, or when the driver of the host vehicle 20 is not performing a brake operation (YES in Step S21), the process immediately proceeds to Step S22 and may contact the object. Is high, an audio warning is issued through the speaker 7 (step S22).
The predetermined threshold GTH is a value corresponding to a condition in which the host vehicle 20 stops at a travel distance equal to or less than the distance between the target object and the host vehicle 20 when the acceleration Gs during the brake operation is maintained as it is.
[0036]
In addition, when an alarm by voice is issued through the speaker 7 in step S22, the image processing unit 1 is obtained by the display control unit 12, for example, by the infrared camera 2R as shown in FIG. 6A. The video signal 30 and the emphasized video signal 31 of the object for providing the information of the approaching object to the driver of the host vehicle 20 are displayed on the display device 8, and part of the video display area, Alarm still image data (icon) 32 indicating “operating pedestrian detection function” selected from alarm still image data (icon) held in alarm still image data storage unit 13 is displayed (step S23). ).
[0037]
On the other hand, when the calculated contrast of the reference image is smaller than the predetermined value in step S5 (YES in step S5), the image processing unit 1 stops the object detection process, and the display is performed as shown in FIG. The control unit 12 displays the video signal 30 obtained by, for example, the infrared camera 2R on the display device 8 and, at the same time, displays the alarm still image data (stored in the alarm still image data storage unit 13 in a part of the video display area. Alarm still image data (icon) 33 indicating “inactive pedestrian detection function” selected from the icons is displayed (step S24).
[0038]
Similarly, in step S13, when the acquired ambient temperature (outside air temperature) of the host vehicle 20 is outside the predetermined temperature range, or the values of the vehicle information such as the vehicle speed VCAR and the yaw rate YR are outside the predetermined range. In both cases, it is determined that one of the data is not normal (NO in step S13), the image processing unit 1 stops the object detection process, and as shown in FIG. 12, for example, the video signal 30 obtained by the infrared camera 2 </ b> R is displayed on the display device 8, and alarm still image data (icon) held in the alarm still image data storage unit 13 in a part of the video display area. The alarm still image data (icon) 33 indicating that “the pedestrian detection function is not operating” selected from the above is displayed (step S24).
[0039]
In the object detection / alarm operation by the image processing unit 1 described above, the processing from step S1 to step S22 corresponds to the processing executed by the image processing unit 11, and the processing of step S23 and step S24 is performed by the display control unit 12. This corresponds to the process to be executed.
[0040]
As described above, the vehicle periphery display device according to the present embodiment travels the host vehicle 20 from the infrared cameras 2R and 2L that can shoot the surroundings of the host vehicle 20 and the images captured by the infrared cameras 2R and 2L. When the warning object is detected by the image processing unit 11 that determines the warning object that affects the vehicle, and the image processing unit 11, the infrared camera 2R, The image processing unit 1 includes a display control unit 12 that simultaneously displays an image captured by 2L and an emphasized image indicating a warning object. For example, when the temperature around the vehicle 20 (outside temperature) is high or When the image processing unit 11 cannot determine the warning object, such as when signals from various sensors indicating the traveling state of the vehicle 20 are interrupted, the display control unit 12 performs the infrared cameras 2R, 2L. The video is always displayed on the display device 8 and is displayed by alarm still image data (icon) indicating the operating state of the image processing unit 1, so that the driver of the own vehicle can at least know the situation around the own vehicle. At the same time, it is possible to reliably notify and grasp that the determination of the warning object by the image processing unit 11 has not been executed.
[0041]
In addition, when only the alarm still image data (icon) is displayed by displaying the alarm still image data (icon) indicating the operating state of the image processing unit 1 together with the image display of the infrared cameras 2R and 2L. Compared to the above, it is possible to more reliably notify and grasp the operating state of the image processing unit 1 to the driver of the host vehicle.
[0042]
Accordingly, in any case, by providing the driver of the host vehicle with the situation around the host vehicle in some form and notifying the operating state of the image processing unit 1 in an easy-to-understand manner, the driver can appropriately This makes it possible to execute an appropriate judgment and response.
[0043]
【The invention's effect】
As described above, according to the vehicle periphery display device of the present invention, even when the determination by the warning object determination unit cannot be performed, the warning display unit always displays the image of the infrared camera on the display device, thereby The driver can check at least the situation around the host vehicle. Further, the warning display means displays the image of the infrared camera and the display indicating the operation state of the warning object determination means together on the display device, so that the driver of the own vehicle can operate the warning object determination means. The state can be surely notified and grasped by an easy-to-understand method.
[0044]
Therefore, it is possible to cause the driver of the own vehicle to execute appropriate determination and response corresponding to the situation.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a vehicle periphery display device according to an embodiment of the present invention.
FIG. 2 is a diagram showing a specific example of a display device provided in a vehicle.
FIG. 3 is a diagram showing attachment positions of an infrared camera, a sensor, a display, and the like provided in the vehicle.
FIG. 4 is a flowchart showing an object detection / alarm operation of the vehicle periphery display device according to the embodiment;
FIG. 5 is a flowchart showing an object detection / alarm operation of the vehicle periphery display device according to the embodiment;
FIG. 6 is a diagram showing a display example of an image displayed on the display device of the vehicle periphery display device of the embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Image processing unit 2R, 2L Infrared camera 3 Yaw rate sensor 4 Vehicle speed sensor 5 Brake sensor 6 Temperature sensor 7 Speaker 8 Display apparatus 11 Image processing part (warning object determination means)
12 Display control unit (Warning display means)
13 Alarm Still Image Data Storage Unit 20 Own Vehicle

Claims (2)

An infrared camera capable of photographing the surroundings of the host vehicle, a warning target determining unit that determines a warning target that affects the traveling of the host vehicle from an image captured by the infrared camera, and the warning target determining unit Warning display means for simultaneously displaying an image captured by the infrared camera and an emphasized image showing the warning object on a display device disposed in front of the driver's seat of the host vehicle when the warning object is detected; In the vehicle periphery display device comprising:
The vehicle periphery display device, wherein the warning display means always displays an image of the infrared camera on the display device when the warning object determination by the warning object determination means cannot be executed. 2. The vehicle periphery display device according to claim 1, wherein the warning display unit displays on the display device an operation state of the warning object determination unit together with display of an image of the infrared camera. .

Images