JP5023872B2 - Image display control device and image display control system - Google Patents

Description

  The present invention relates to an image display control device and an image display control system that are mounted on, for example, an automobile and can display surrounding images such as the front of the host vehicle on a monitor display, a windshield display, or the like.

  Conventionally, in order to reduce the burden on the driver when driving the vehicle, an image is taken with the electronic camera mounted on the vehicle for the purpose of detecting the surrounding situation quickly and accurately without overlooking the surroundings. Indirect visual recognition devices for vehicles that display the above on a monitor display or a windshield display in front of the driver's seat have been proposed.

  In these technologies, objects that require safety confirmation when driving, for example, objects requiring safety confirmation such as pedestrians and other vehicles, are highlighted on the display to alert the driver. ing.

As a highlighting method, for example, a method of highlighting a predetermined area in front of the vehicle (see Patent Document 1) or a safety confirmation target by suppressing the background of the safety confirmation target so as not to stand out. A method of indirectly highlighting an object has been proposed (see Patent Documents 2 and 3).
JP 2003-274393 A JP-A-9-48282 Japanese Patent Laid-Open No. 11-328364

However, for example, when directly emphasizing the image of the safety confirmation target object, there is a problem that the appearance of the safety confirmation target object greatly changes and becomes difficult to recognize.
In addition, as in the technique of Patent Document 1, when all of the traveling direction of the vehicle is clarified, attention is also drawn to objects other than the safety confirmation target that should be taken care of. It was.

  Furthermore, in Patent Documents 2 and 3, since all but the safety confirmation target object are suppressed, it is difficult for the driver to grasp the positional relationship between the host vehicle and the safety confirmation target object, and the position of the safety confirmation target object is recognized. There was a problem that it was difficult.

  The present invention has been made in view of such a problem, and is capable of easily recognizing a safety confirmation target object and easily grasping a positional relationship between the host vehicle and the safety confirmation target object and an image. An object is to provide a display control system.

(1) According to the first aspect of the present invention, based on a signal obtained from an imaging device that captures the surroundings of the host vehicle, an object image indicating a safety confirmation target existing around the host vehicle is displayed as an image display device. In the image display control device to be displayed, the predicted travel region calculation means for calculating the predicted travel region of the host vehicle, the position of the predicted travel region and the safety confirmation target object are related to each other, While displaying the image and the object image on the image display device, suppress the display of the region other than the predicted travel region than the display of the predicted travel region, and display other than the region of the target image, In addition to suppressing the display in the area of the object image, and further setting a priority order for the safety-required confirmation target object according to the degree of risk, and according to the priority order, Surrounding display And changing the degree of control.

In the present invention, since the positions of the predicted travel area and the safety confirmation target object are related and displayed on the image display device, the future positional relationship between the host vehicle and the safety confirmation target object can be easily grasped. In addition, since the display of images other than the predicted travel area is suppressed at this time, the predicted travel area becomes clear, and as a result, the positional relationship between the host vehicle and the object requiring safety confirmation can be understood very easily. There is a remarkable effect of being able to.
Further, in the present invention, the display other than the area of the object image is suppressed more than the display in the area of the object image.
Thereby, the recognition property of a target object can be improved. For example, when the image of the region other than the predicted travel region and the object image overlap, the display of the predicted travel region is suppressed from the display of the target image. Further, when the image of the predicted travel area and the object image overlap, the display of the predicted travel area is suppressed from the display of the target image.
Furthermore, in the present invention, priorities are set according to the degree of risk for the safety confirmation target object, and the degree of suppression of display around the object image is changed according to the priority order.
As a result, the display according to the degree of danger becomes possible, so that the driver can easily grasp the degree of the degree of danger and the safety is further improved.

Here, “suppressing image display” is processing for reducing image recognition, such as blurring, as opposed to enhancement processing for enhancing image recognition.
The safety target object is an obstacle that may collide with the host vehicle in the future, such as a pedestrian or a vehicle.

  -Although the said driving | running | working prediction area | region can consider the whole road (including an oncoming lane) where the own vehicle drive | works, it is good also as only the track of the same direction except an oncoming lane. In addition, when there are a plurality of lanes in the same direction (lanes such as lanes and overtaking lanes), it is preferable to include all the lanes, but only a single lane on which the vehicle travels may be included.

(2) In the invention of claim 2, the object image is an image of the object requiring safety confirmation itself or an image within a frame including a periphery of the image of the object necessary for safety confirmation. .
The present invention exemplifies an object image indicating a safety confirmation target object. In addition, it becomes easier to recognize the presence of the safety confirmation target object by surrounding the image of the safety confirmation target object itself with a frame.

( 3 ) In the invention of claim 3 , a band-shaped portion is set around the object image, and the degree of suppression of the display is changed in the band-shaped portion.

The present invention exemplifies a region where the degree of suppression of display is changed according to priority.
( 4 ) The invention of claim 4 is characterized in that the higher the priority is, the greater the degree of suppression of the display.

Accordingly, the driver, since what priority of a main safety check object (hence risk) is high seen, it is further improved safety.

( 5 ) In the invention of claim 5, the suppression process for suppressing the display of the image includes a process of painting with a predetermined color, a process of painting with a predetermined color translucently, a process of blurring with a predetermined blurring degree, and a process of reducing contrast And at least one of a process for reducing luminance and a process for reducing the saturation of a color image.

The present invention illustrates the suppression process.
(6) The invention of claim 6 is based on a signal obtained from an image pickup device that captures the surroundings of the host vehicle, and displays an object image indicating a safety confirmation target that exists around the host vehicle. In the image display control device to be displayed, the predicted travel region calculation means for calculating the predicted travel region of the host vehicle, the position of the predicted travel region and the safety confirmation target object are related to each other, An image and the object image are displayed on the image display device, a suppression process for suppressing display of an area other than the predicted travel area from the display of the predicted travel area is performed, and a predetermined process is performed as the suppression process. Processing that fills with color, processing that is translucent with a predetermined color, processing that blurs with a predetermined blurring degree, processing that lowers contrast, processing that lowers brightness, and processing that lowers the saturation of a color image Characterized by using at least one.
In the present invention, since the positions of the predicted travel area and the safety confirmation target object are related and displayed on the image display device, the future positional relationship between the host vehicle and the safety confirmation target object can be easily grasped. In addition, since the display of images other than the predicted travel area is suppressed at this time, the predicted travel area becomes clear, and as a result, the positional relationship between the host vehicle and the object requiring safety confirmation can be understood very easily. There is a remarkable effect of being able to.
In the present invention, as the suppression process, a process of painting with a predetermined color, a process of painting with a predetermined color and semi-transparent, a process of blurring with a predetermined blurring degree, a process of reducing contrast, a process of reducing luminance, and saturation of a color image Among the processes for lowering, at least one kind is used.
Thereby, the recognition degree of the image of an important obstacle and the own lane travel area can be raised by the driving | operation of a driver.
(7) The invention according to claim 7 is characterized in that the object image is an image of the object requiring safety confirmation itself or an image within a frame including a periphery of the image of the object requiring safety confirmation itself. .
The present invention exemplifies an object image indicating a safety confirmation target object. In addition, it becomes easier to recognize the presence of the safety confirmation target object by surrounding the image of the safety confirmation target object itself with a frame.
(8) The invention of claim 8 is characterized in that the display other than the area of the object image is suppressed more than the display in the area of the object image.
Thereby, the recognition property of a target object can be improved. For example, when the image of the region other than the predicted travel region and the object image overlap, the display of the predicted travel region is suppressed from the display of the target image. Further, when the image of the predicted travel area and the object image overlap, the display of the predicted travel area is suppressed from the display of the target image.
(9) The invention of claim 9 sets priority for the safety-required confirmation target object according to the degree of risk, and suppresses display around the object image according to the priority order. It is characterized by changing the degree.
As a result, the display according to the degree of danger becomes possible, so that the driver can easily grasp the degree of the degree of danger and the safety is further improved.
(10) The invention of claim 10 is characterized in that a band-shaped portion is set around the object image, and the degree of suppression of the display is changed in the band-shaped portion.
The present invention exemplifies a region where the degree of suppression of display is changed according to priority.
(11) The invention of claim 11 is characterized in that the higher the priority is, the greater the degree of suppression of the display.
As a result, the driver knows which safety-required confirmation target object has a higher priority (thus, a risk level), and thus the safety is further improved.
(12) the invention (image display control system) according to claim 12, the image display control device according to any one of the preceding claims 1 to 11, and characterized in that it comprises at least the image pickup device and an image display device To do.

  The present invention shows a system configuration using an image display control device. In addition to the imaging device, signals from radars and various sensors can be used.

Embodiments of the present invention will be described below with reference to the drawings.
[First Embodiment]
a) First, the configuration of the image display control system of the present embodiment will be described.

  As shown in FIG. 1, the image display control system 1 according to the present embodiment detects a target object (safety confirmation target object: obstacle) existing around the vehicle and displays the obstacle on the display 3. It is an indirect vehicular visual recognition system that performs image processing that can be easily recognized, and includes an image display control device 5 that performs processing such as image suppression display as a main part thereof.

  The image display control device 5 includes an electronic control device having a known microcomputer 7 and memory 9 as main parts. The image display control device 5 includes an electronic camera 11 for photographing the front of the vehicle and the like, a vehicle Are connected to a radar 13 for detecting an obstacle in front of the vehicle, a vehicle speed sensor 15 for detecting a vehicle speed, a steering angle sensor 17 for detecting a steering angle, a navigation device 19, and a display 3 for displaying an image.

  The electronic camera 11 can be a visible light camera or an infrared camera, and the radar 13 can be a millimeter wave radar (for vehicles) or a laser radar (for pedestrians). ) Can be used. Furthermore, a sensor that detects the pitch angle of the vehicle may be used as the sensor.

  As will be described in detail later, the image display control device 5 recognizes the driving environment and the driving state based on information obtained from the electronic camera 11, the radar 13, the vehicle speed sensor 15, the steering angle sensor 17, the navigation device 19, and the like. Recognition, obstacles and predicted travel areas, and the surrounding images, and based on the processing results, the display 3 displays the images of the obstacles and predicted travel areas and blurs the surroundings. Display suppressed by processing.

b) Next, processing contents performed in the image display control device 5 will be described.
As shown in the flowchart of FIG. 2, in the image display control device 5, first, in step (S) 100, image data in front of the vehicle photographed by the electronic camera 11 is captured.

In the subsequent step 110, data on the distance to the obstacle detected by the radar 13 (ranging data) is captured.
In the following step 120, the vehicle information of the own vehicle obtained from the vehicle speed sensor 15 and the steering angle sensor 17 is captured. At this time, information such as the lane in which the host vehicle is traveling can be taken from the navigation device 19.

  In the following step 130, the own lane travel area is calculated. The own lane travel region is a predicted travel region (trajectory) where the host vehicle is likely to travel from now on, and shows the entire road including the oncoming lane.

  As a method of calculating the own lane travel area, for example, when there is a white line on the road, the left and right white lines are detected from the camera image and vehicle information (steering angle, etc.), and the inside of the left and right white lines is set as the own lane travel area. The method of extracting is mentioned. For example, the technique described in Japanese Patent Application Laid-Open No. 11-85999 can be adopted as the white line detection method.

  When there is no white line, a method of obtaining a road plane area from the camera image by clustering processing and extracting it as the own lane running area can be adopted. As a road plane detection method, for example, the technique described in JP-A-2005-21788 can be adopted.

In the subsequent step 140, obstacles such as pedestrians and vehicles that are safety-required confirmation objects are detected. As the obstacle detection method, for example, the following methods (1) to (4) can be employed.
(1) Method for detecting a solid object (obstacle) The parallax is calculated from two electronic cameras 11 arranged in parallel, and the part where the parallax is generated is clustered to obtain a three-dimensional object region (that is, an obstacle). Extract. As a method for detecting a three-dimensional object from a stereo image, for example, a technique described in JP 2000-217883 A can be employed.

(2) Pedestrian detection method A high brightness area is extracted from an infrared camera image, and a pedestrian is detected by an identification method focusing on the shape and movement of the pedestrian. In addition, the method as described in Unexamined-Japanese-Patent No. 11-328364 etc. can be employ | adopted for the method of detecting a pedestrian from an infrared image.

(3) Pedestrian detection method A three-dimensional object region including a pedestrian is extracted by laser radar, and a pedestrian is detected in the three-dimensional object region of the camera image by an identification method that focuses on the shape and movement of the pedestrian. . In addition, the method as described in Unexamined-Japanese-Patent No. 2003-302470 etc. can be employ | adopted for the method of detecting a pedestrian from a laser and an image.

(4) Vehicle detection method A three-dimensional object region including a vehicle is extracted by a millimeter wave radar, and the vehicle is detected in the three-dimensional object region of the camera image by an identification method focusing on the shape and movement of the vehicle. As a method for detecting a vehicle from a millimeter wave and an image, a technique described in Japanese Patent Application Laid-Open No. 2005-141517 can be employed.

  In the subsequent step 150, a display image to be displayed on the display 3 is calculated. That is, a display image is set on the display 3 so that the positional relationship between the obstacle and the own lane travel area can be understood, and which part of the display is suppressed to what extent.

  Specifically, as shown in FIG. 3, a rectangular frame is set so as to surround the pedestrian and vehicle detected in step 150, and the inside of the frame is photographed by the electronic camera 11. The selected image is displayed as it is.

  In the own lane travel area calculated in step 130, the image captured by the electronic camera 11 is displayed as it is. In addition, you may perform the process which suppressed the display rather than the image (object image) in a frame with respect to the own lane travel area. For example, a semi-transparent process may be performed. Thereby, the recognition degree of the image in a frame improves.

  Furthermore, image suppression processing is performed for areas other than the own lane travel area (out-travel area). In other words, image processing such as blurring the entire out-of-travel area is performed in order to reduce the recognition level (attention level) more than the image in the frame and the own lane travel area. It should be noted that in the region where the frame and the non-running region overlap, it is set to display the image in the frame.

Therefore, here, the order in which the images are conspicuous (that is, the order in which the degree of suppression is low) is the image in the frame (including the image of the obstacle itself) and the image in the own lane travel area> the image in the non-travel area. If the inside of the own lane driving area is made translucent, the image in the frame> the image in the own lane driving area> the image of the outside driving area is obtained. In the following step 160, the figure calculated in the step 150 is obtained. 3 is displayed on the display 3, and the process is temporarily terminated.

  c) Thus, in this embodiment, the image of the obstacle and the image of the own lane travel area are displayed so as to overlap with each other in accordance with the position where the obstacle is present. It is possible to easily grasp the current and future positional relationship with In addition, since the area outside the own lane travel area is subjected to image suppression processing such as blurring, it is possible to increase the recognition degree of important obstacles and the image of the own lane travel area by driving the driver.

  In addition, when suppressing the image by making the own lane travel area, for example, translucent, there is an advantage that the recognition degree of the obstacle with high risk can be increased and the shape of the road can be easily grasped. is there.

[Second Embodiment]
Next, the second embodiment will be described, but the description of the same contents as the first embodiment will be omitted.

In the present embodiment, display is performed according to the danger level of the obstacle.
Below, the processing content performed by the image display control apparatus 5 of this embodiment is demonstrated.
As shown in the flowchart of FIG. 4, in the image display control device 5, first, in step 200, image capturing data of an image captured by the electronic camera 11 is captured.

In the subsequent step 210, distance measurement data with the obstacle detected by the radar 13 is captured.
In the following step 220, the vehicle information of the own vehicle obtained from the vehicle speed sensor 15 and the steering angle sensor 17 is fetched.

In subsequent step 230, the own lane travel area is calculated in the same manner as in step 130 of the first embodiment.
In subsequent step 240, obstacles such as pedestrians and vehicles are detected in the same manner as in step 140 of the first embodiment.

In subsequent step 250, the degree of danger of the obstacle is calculated.
For example, the following methods (1) to (3) can be adopted as a method of calculating the degree of danger of the obstacle.

  (1) For example, based on information from the radar 13, the relative distance between each obstacle and the host vehicle is calculated, and the risk of the obstacle is set higher as the distance from the obstacle is shorter. In this case, it is desirable to have the degree of danger with respect to the distance as a map.

  (2) Calculate the relative distance and relative speed between each obstacle and the vehicle, and the shorter the distance from the obstacle, the more the obstacle is in the direction of the relative speed with the obstacle and the vehicle. The higher the relative speed of, the higher the degree of danger. In this case, it is desirable to have a risk with respect to distance and relative speed as a map.

  (3) The relative distance and relative speed between each obstacle and the host vehicle are calculated, and the shorter the distance from the obstacle, the more the direction of the relative speed with the obstacle is facing the host vehicle and the own lane. The higher the relative speed of, the higher the degree of danger. Further, the degree of danger is changed depending on the attribute of the obstacle. For example, the degree of danger is set such that “three-dimensional object (stationary object) <vehicle <pedestrian”. Also in this case, it is desirable to have a degree of risk for the distance and relative speed as a map.

  Thus, for example, as shown in FIG. 5, the risk level of the pedestrian A who is on the left lane of the own lane travel area and approaches the center of the lane is set to the maximum risk level 4, Next, the risk level of pedestrian B who is on the left lane and moves away from the lane is set to the next risk level 3, and then the risk level of pedestrian C who moves away from the right opposite lane is Then, the next risk level 2 is set, and then the risk level of the other vehicle D approaching from the opposite lane on the right side is set to the next risk level 1. As is well known, the movement direction of a pedestrian or vehicle can be obtained from, for example, the feature points extracted from those images and the movement directions of the feature points.

In the subsequent step 260, a display image to be displayed on the display 3 is calculated.
Basically, as shown in FIG. 5, a rectangular belt-like frame (band-like portion) is set so as to surround the pedestrian or vehicle detected in step 240 (the lower side is a belt-like shape). However, the image captured by the electronic camera 11 is displayed as it is without image processing inside the belt-like frame.

In addition, the own lane travel area may display the captured image as it is, as in the first embodiment, or may suppress the display by making the own lane travel area translucent, for example.
Further, for areas other than the own lane travel area (out-travel area), image suppression processing such as blurring the entire image is performed as in the first embodiment.

  In the present embodiment, in particular, the display of the band-shaped frame set around the obstacle increases the degree of blur as the degree of danger increases. Further, here, the color of the belt-like frame is also changed according to the degree of danger. For example, the lightness is changed from the one with the highest degree of danger to red> orange> yellow> blue, or the lightness difference with the background is increased.

That is, the display of the belt-like frame is set so that the obstacle having a higher degree of danger attracts the driver's attention.
It should be noted that the degree of suppression of the image between the area other than the own lane driving area and the belt-like frame (the degree of blurring is not limited as long as the area other than the own lane driving area and the belt-like frame can be distinguished. Increase the degree of blurring in areas other than the lane driving area.

In the following step 270, an image as shown in FIG. 5 is displayed on the display 3, and the process is temporarily terminated.
c) As described above, in this embodiment, the display of the band-shaped frame set around the obstacle increases the degree of suppression of the image display as the degree of danger increases. Compared with the method of increasing the degree of emphasis on the vehicle, the driver can easily recognize obstacles with a high degree of danger.

In addition, this invention is not limited to the said embodiment, It is possible to implement in various aspects.
(1) For example, as a highlighting method, there is no particular limitation as long as it draws attention of the driver or the like. For example, an image showing an object may be highlighted by vibrating on the screen. In addition, when there are a plurality of objects, images indicating the objects may be sequentially highlighted in order from the highest risk level.

(2) The present invention can also be applied to a program for executing a display suppression process based on the above-described algorithm and a recording medium storing the program.
Examples of the recording medium include various recording media such as an electronic control device configured as a microcomputer, a microchip, a flexible disk, a hard disk, a DVD, and an optical disk. That is, there is no particular limitation as long as it stores a program that can execute the processing of the image display control device described above.

  The program is not limited to a program stored in a recording medium, but can be applied to a program transmitted / received through a communication line such as the Internet.

It is a block diagram which shows the whole control system in 1st, 2nd embodiment. It is a flowchart which shows the process in the image display control apparatus of 1st Embodiment. It is explanatory drawing which shows the screen displayed on the display of 1st Embodiment. It is a flowchart which shows the process in the image display control apparatus of 2nd Embodiment. It is explanatory drawing which shows the screen displayed on the display of 2nd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image display control system 3 ... Display 5 ... Image display control apparatus 11 ... Electronic camera 13 ... Radar

Claims (12)

In an image display control device for displaying an object image indicating a safety-required confirmation object existing around the host vehicle on an image display device based on a signal obtained from an imaging device that captures the periphery of the host vehicle.
A predicted travel region calculation means for calculating a predicted travel region of the host vehicle;
The position of the predicted travel area and the safety required target object are related to each other, and the image of the predicted travel area and the object image are displayed on the image display device, and the area other than the predicted travel area is displayed. Is suppressed from the display of the predicted travel area ,
In addition, the display other than the area of the object image is suppressed more than the display in the area of the object image, and the priority order is set for the safety confirmation target object according to the risk level. An image display control apparatus that changes a degree of suppression of display around the object image according to the priority order .   The image display control device according to claim 1, wherein the object image is an image of the safety confirmation target object itself or an image within a frame including a periphery of the image of the safety confirmation target object itself. . The Set strip portion around the object image, the image display control apparatus according to claim 1 or 2, characterized in that to change the degree of the display of suppression in the band portion. The priority higher, the image display control device according to any one of claims 1 to 3, characterized in that to increase the degree of the display of the suppression. The suppression process for suppressing the display of the image includes a process of painting with a predetermined color, a process of painting with a predetermined color and semi-transparent, a process of blurring with a predetermined blurring degree, a process of reducing contrast, a process of reducing brightness, and a color image of the process of lowering the degree, the image display control apparatus according to any one of claims 1 to 4, characterized in that at least one. In an image display control device for displaying an object image indicating a safety-required confirmation object existing around the host vehicle on an image display device based on a signal obtained from an imaging device that captures the periphery of the host vehicle.
  A predicted travel region calculation means for calculating a predicted travel region of the host vehicle;
  The position of the predicted travel area and the safety required target object are related to each other, and the image of the predicted travel area and the object image are displayed on the image display device, and the area other than the predicted travel area is displayed. Is suppressed from the display of the predicted travel area,
  In addition, as the suppression process, a process of painting with a predetermined color, a process of painting with a predetermined color and semi-transparent, a process of blurring with a predetermined blurring degree, a process of reducing contrast, a process of reducing brightness, and a process of reducing the saturation of a color image An image display control device using at least one of them. The image display control device according to claim 6, wherein the object image is an image of the safety confirmation target object itself or an image in a frame including a periphery of the image of the safety confirmation target object itself. . The image display control apparatus according to claim 6, wherein display other than the region of the object image is suppressed more than display within the region of the object image. A priority order is set for the object requiring safety confirmation according to the degree of risk, and a degree of suppression of display around the object image is changed according to the priority order. Item 9. The image display control device according to Item 8. The image display control apparatus according to claim 9, wherein a band-shaped portion is set around the object image, and the degree of suppression of the display is changed in the band-shaped portion. The image display control apparatus according to claim 9, wherein the degree of suppression of the display is increased as the priority is higher. An image display control system according to any one of claims 1 to 11, comprising at least an imaging device and an image display device.