JP7521268B2 - Vehicle image processing device - Google Patents

Description

The present invention relates to a video processing device for vehicles.

Patent document 1 describes a vehicle display device that includes a center console with at least one of a display unit and an operation unit in the center of the instrument panel in the vehicle width direction, an imaging means for capturing images of the rear of the vehicle, and an electronic mirror monitor that is provided in the center console or between the center console and the steering wheel in the vehicle width direction of the instrument panel and displays an image of the rear of the vehicle captured by the imaging means. This vehicle display device includes an inferred information acquisition means that acquires inferred information that is inferred when the driver looks at the center console, a determination means that determines whether or not to display a degradation in the visibility of the vehicle rear image based on the inferred information, and a degradation display means that reduces the visibility of the vehicle rear image based on the determination result of the determination means.

JP 2016-172526 A

Unlike optical mirrors such as fender mirrors and door mirrors, the display device of an electronic mirror in an automobile is installed inside the vehicle cabin. The viewing distance when checking both the front of the vehicle and the display device of the electronic mirror inside the vehicle cabin is greater than the viewing distance between the front of the vehicle and the optical mirror outside the vehicle cabin, so the driver of a vehicle equipped with an electronic mirror places a heavy strain on the eyes.

The purpose of this invention is to reduce the strain on the eyes of drivers of vehicles equipped with electronic mirrors.

In order to achieve the above object, the image processing device for a vehicle according to the present invention includes an acquisition unit that acquires an image of the surroundings of the vehicle, and an image processing unit that creates a visual stimulation image having linear parts that flow in a direction opposite to the optic flow of the surroundings image acquired by the acquisition unit, and generates an image for display by superimposing the visual stimulation image on the surroundings image. The visual stimulation image is an image that is reproduced intermittently.

The present invention can reduce the strain on the eyes of drivers of vehicles equipped with electronic mirrors.

FIG. 1 is an explanatory diagram of optic flow. FIG. 1 is an explanatory diagram of an electronic mirror system. FIG. 13 is a flowchart showing a flow of video processing. FIG. 2 is an explanatory diagram showing an example of a display image. FIG. 11 is an explanatory diagram showing another example of a display image. FIG. 11 is an explanatory diagram showing yet another example of a display image.

The present invention will be described below based on the illustrated embodiment. However, the present invention is not limited to the embodiment described below.

In conventional optical mirrors installed outside the vehicle cabin, the distance at which the eyes should focus (visual distance) is equal to the distance to the object reflected by the mirror's reflective surface. Therefore, even if you move your gaze from the road in front of the vehicle to the optical mirror, there is little strain on the eyes in adjusting the focus.

In contrast, with a digital mirror device, the driver needs to focus his or her eyes on a display device (such as a liquid crystal panel) inside the vehicle, which is located at a relatively short distance from the driver. For this reason, when the driver moves his or her gaze from the road in front of the vehicle, which is relatively far away, to a display device at a close distance, the range of adjustment required to adjust the refraction of the crystalline lens and the convergence of the eyes in accordance with the change in visual distance becomes larger, resulting in a greater physiological strain on the driver's eyes.

Figure 1 shows an image IM1 from an electronic side mirror attached to the vehicle C1 while it is traveling forward. This electronic side mirror is used to check the rear of the vehicle and the right side as one faces the front of the vehicle. The image IM1 shows the lane A1 in which the vehicle C1 is traveling and part of the body of the vehicle C1, the lane A2 adjacent to lane A1, another vehicle C2 traveling on lane A2, and the boundary line BL between lanes A1 and A2. For convenience, the vehicle C1 and vehicle C2 are illustrated as rectangular parallelepiped shapes.

In the vision of living things, the movement on the retina caused by the movement of the eye is called optic flow. When the eye moves forward, optic flow is generated that spreads across the entire visual field, centered on the direction of gaze. On the other hand, when the eye moves backward, optic flow is generated that converges from the entire visual field towards the direction of gaze.

The angle formed by the axes of both eyes when looking at an object is called the angle of convergence. The angle of convergence becomes smaller when looking at an object at a distance and becomes larger when looking at an object at a close distance. In addition, eye movement when looking at an object from a distance to a close distance is called convergence movement, and eye movement when looking at an object from a close distance to a far distance is called divergence movement. It is known that convergence movement is induced by optic flow in the divergent direction, and divergence movement is induced by optic flow in the convergent direction.

Since image IM1 captures the rear of vehicle C1, which is traveling forward, the multiple optic flows OF have directions that converge toward vanishing point P1 of image IM1. Such converging optic flows OF induce divergence movement. This divergence movement is thought to hinder the eye's focusing (i.e., convergence movement) from the front of the vehicle to the electronic mirror display device inside the vehicle cabin.

The embodiment described below is intended to reduce the physiological strain on the driver's eyes when using an electronic mirror.

Figure 2 shows an electronic mirror system 1. This electronic mirror system includes imaging units 2a and 2b, an MCU (microcontroller unit) 3, and display units 4a and 4b. The MCU 3 is also called a vehicle image processing device.

Figure 3 is a plan view of the vehicle C1. The roof of the vehicle C1 is not shown. Arrows F and B indicate the front and rear of the vehicle, respectively, and arrows L and R indicate the left and right sides in the vehicle width direction, respectively. The imaging unit 2a is attached to the outside of the left door of the vehicle C1, and captures images of the rear of the vehicle and the left side relative to the front of the vehicle. The imaging unit 2b is attached to the outside of the right door of the vehicle C1, and captures images of the rear of the vehicle and the right side relative to the front of the vehicle. In other words, the optical axes of both imaging units 2a and 2b are directed toward the rear side of the vehicle C1.

Display unit 4a is located inside the vehicle cabin near the passenger seat and displays the image captured by imaging unit 2a. Display unit 4b is located inside the vehicle cabin near the driver's seat where the steering wheel SW is located and displays the image captured by imaging unit 2b.

The MCU 3 has a hardware configuration including a CPU (Central Processing Unit) 31 and a memory 32. The CPU 31 has a functional configuration including an acquisition unit 31a and a video processing unit 31b. The processing of the acquisition unit 31a and the video processing unit 31b will be described with reference to FIG. 4.

First, in step S1, the imaging units 2a and 2b each capture an image of the area to the rear of the vehicle (surrounding image).

In step S2, the acquisition unit 31a acquires the surrounding images captured by the imaging units 2a and 2b from the imaging units 2a and 2b.

Also in step S2, the image processing unit 31b generates a visual stimulation image (visual clue) for each surrounding image acquired by the acquisition unit 31a. This visual stimulation image is an image of a line portion radiating from a certain point (such as a vanishing point) in the surrounding image. The direction in which the line portion of this visual stimulation image flows is the diffusion direction (at least two directions), and is the opposite direction to the optic flow (the flow of the scenery of each image), which has a direction converging toward the vanishing point.

Furthermore, in this step, the image processing unit 31b creates a display image to be displayed on the display unit 4a by superimposing the visual stimulation image calculated for the surrounding image of the imaging unit 2a on the surrounding image of the imaging unit 2a. In addition, the image processing unit 31b creates a display image to be displayed on the display unit 4b by superimposing the visual stimulation image calculated for the surrounding image of the imaging unit 2b on the surrounding image of the imaging unit 2b. Each display image is sent from the image processing unit 31b to the display units 4a and 4b.

In step S3, a mirror image of the display image is created and displayed on the display units 4a and 4b.

Figure 5 shows the display image IM1a displayed on the right display unit 4b. This display image IM1a is an image formed by superimposing a visual stimulation image having a semi-transparent linear portion VS1a on the image IM1 shown in Figure 1. The linear portion VS1a flows so as to diffuse radially from the vanishing point P1 as the diffusion source. In other words, the flow direction (diffusion direction) of the linear portion VS1a is opposite to the flow direction (convergence direction) of the optic flow OF in Figure 1.

Figure 6 shows the display image IM2 displayed on the right display unit 4b. This display image IM2 is an image formed by superimposing a visual stimulation image having a semi-transparent linear portion VS2 on the image IM1 shown in Figure 1. The linear portion VS2 flows so as to diffuse radially from the center P2 of the area in which the vehicle C2 is captured as the diffusion source. In other words, the flow direction (diffusion direction) of the linear portion VS2 is opposite to the flow direction (convergence direction) of the optic flow OF in Figure 1. The image processing unit 31b calculates the movement speed of the rear vehicle C2 by analyzing the surrounding image, and creates a display image so that the diffusion speed of the linear portion VS2 is faster than the movement speed of the vehicle C2 in the image.

As another example, FIG. 7 shows a display image IM3 displayed on the right display unit 4b. In addition to the subject vehicle C1 and vehicle C2, this display image also shows another vehicle C3. In addition, a semi-transparent linear portion VS3 is shown flowing radially from the center of the area in which vehicle C3 is shown. Similarly, the flow direction (diffusion direction) of linear portion VS3 is opposite to the flow direction (convergence direction) of the optic flow OF in FIG. 1. The diffusion source of linear portion VS3 is the center P3 of the area in which vehicle C3 is shown.

The above embodiment utilizes the fact that the focusing function of the human eye is affected by the flow of the scenery being viewed. That is, to avoid the flow of the scenery reflected in the electronic mirror (optic flow in the convergence direction) inducing a divergence movement and impeding the convergence movement when adjusting the focus from the front of the vehicle to the display device, a visual stimulation image having linear parts flowing in the divergence direction is superimposed on the surrounding image. This visual stimulation image assists the convergence movement when adjusting the focus from the front of the vehicle to the display device, reducing the physiological burden on the driver's eyes. Furthermore, it is expected that the time spent gazing at the image on the electronic mirror will be reduced, and the driver's concentration on driving will increase.

It is generally known that the gaze is guided to the center point of the optic flow flowing in the diffusion direction. Therefore, by aligning the center point of the linear portion flowing in the diffusion direction (the diffusion source of the optic flow) with the position of a vehicle appearing in the image, it is expected that the driver's attention will be guided to that vehicle.

The linear portion may be semi-transparent or have other translucent properties so as not to impede the driver's search within the image. The edges (contours) of the linear portion may be blurred. Brightness may be added during the superimposition process.
Also, when the linear portion is a dotted line or the like that moves regularly, the driver's attention tends to be drawn to that line. Therefore, it is preferable to generate the linear portion intermittently and discontinuously (randomly or irregularly).

[Other embodiments]
The diffusion source of the linear part can be aligned with the center of the vehicle shown in the image. The recognition of the vehicle shown in the image can be performed by the image processing unit 31b. This guides the driver's line of sight to the diffusion source of the diffusing linear part, i.e., the center of the vehicle, making it easier for the driver to find the vehicle. When multiple vehicles are shown in the image, the vehicle closest to the vehicle traveling in the lane adjacent to the lane in which the vehicle is traveling can be recognized, and the diffusion source can be placed in the area in which the vehicle is shown. It is also possible to place a diffusion source on each of the multiple vehicles.

The speed at which visual stimuli flow can be changed according to the relative vehicle speed between the vehicle itself and the vehicle behind.
The image processor 31b calculates the relative speed based on the speed of the vehicle itself obtained from the CAN 5 (FIG. 2) of the vehicle itself and the speed of the rear vehicle obtained by analyzing the surrounding image, thereby visually notifying the driver that the speed of the rear vehicle is high.

The brightness of the visual stimulus can also be changed depending on the illuminance of the environment surrounding the vehicle. When the surroundings are dark, such as at night, the brightness must be reduced. It is sufficient to have a brightness that is visible to the scenery and the vehicle. This prevents the visual stimulus from impeding the driver's search. The illuminance of the surrounding environment can be obtained, for example, from a light receiving sensor if the vehicle is equipped with automatic headlights. The illuminance may also be obtained through image analysis by an image processing unit.

The above embodiment can be applied not only to side mirrors but also to rearview mirrors.

Regarding the embodiments described above, the following supplementary notes are disclosed.
[Appendix 1]
An acquisition unit that acquires an image of the surroundings of the host vehicle;
and an image processing unit that creates a visual stimulation image having linear portions that flow in a direction opposite to the optic flow of the surrounding image acquired by the acquisition unit, and generates a display image by superimposing the visual stimulation image on the surrounding image.
[Appendix 2]
The vehicle image processing device according to claim 1, wherein the direction of the optic flow is a convergence direction and the flow direction of the linear portion is a divergence direction.
[Appendix 3]
3. The vehicle image processing device according to claim 1, wherein the linear portion has a blurred outline and is transparent.
[Appendix 4]
4. The vehicle image processing device according to claim 1, wherein the visual stimulation image is an image that is played back intermittently.
[Appendix 5]
5. The vehicle image processing device according to claim 2, wherein a diffusion source of the linear portion is a vanishing point of the surrounding image.
[Appendix 6]
The vehicle image processing device according to any one of appendices 2 to 5, wherein the diffusion source of the linear portion is in an area within the surrounding image in which a vehicle following the host vehicle is captured.
[Appendix 7]
The vehicle image processing device according to claim 6, wherein the following vehicle is the vehicle closest to the host vehicle.
[Appendix 8]
The vehicle image processing device according to claim 6 or 7, wherein the speed at which the linear portion flows is determined according to a relative speed between the host vehicle and the following vehicle.
[Appendix 9]
The luminance of the linear portion is determined according to the brightness of the surroundings of the vehicle.

Although the embodiment of the present invention has been described above, the present invention is not limited to the embodiment described above, and various modifications and changes are possible based on the technical concept of the present invention.

1 Digital mirror system 2a, 2b Imaging unit 31a Acquisition unit 31b Image processing unit 4a, 4b Display unit

IM1-IM3 Images C1-C3 Vehicle OF Optic flow VS2, VS3 Linear part

Claims (8)

An acquisition unit that acquires an image of the surroundings of the host vehicle;
a video processing unit that creates a visual stimulation image having a linear portion that flows in a direction opposite to the optic flow of the surrounding image acquired by the acquisition unit, and generates a display image by superimposing the visual stimulation image on the surrounding image ,
The visual stimulation image is an image that is reproduced intermittently.
Vehicle image processing device. The vehicle image processing device according to claim 1, wherein the direction of the optic flow is a convergence direction, and the direction of the flow of the linear portion is a divergence direction. The vehicle image processing device according to claim 1 or 2, wherein the linear portion has a blurred outline and is transparent. The image processing device for a vehicle according to claim 2 , wherein a diffusion source of the linear portion is a vanishing point of the surrounding image. The vehicle image processing device according to claim 2 , wherein the diffusion source of the linear portion is in an area in the surrounding image in which a following vehicle of the host vehicle is captured. The vehicle image processing device according to claim 5 , wherein the following vehicle is the vehicle closest to the host vehicle. The image processing device for a vehicle according to claim 5 or 6 , wherein a speed at which the linear portion flows is determined in accordance with a relative speed between the host vehicle and the following vehicle. The vehicle image processing device according to claim 1 , wherein a luminance of the linear portion is determined according to a brightness of an area around the vehicle.