JP7028536B2 - Display system - Google Patents

Description

The present invention relates to a display control using a head-up display, and more particularly to a display control for displaying a road surface condition of a road.

A head-up display (hereinafter abbreviated as HUD) has been put into practical use as one means for displaying driving support information for supporting the driver. A typical HUD projects an image from a projection device installed in the car onto the windshield or the screen installed on the windshield, and the driver can see the virtual image as if an object exists in the real space through the windshield. To see. By using the HUD, the driving support information can be displayed in the direction of the driver's line of sight, so that the movement of the driver's line of sight can be reduced as compared with the case of looking at the liquid crystal display in the vehicle.

As a technique related to HUD, for example, there is a display device for a vehicle described in Patent Document 1. In Patent Document 1, a head-up display device is used to easily grasp the road shape in front of the own vehicle, and in particular, the driver is alerted by highlighting the road shape of a sharp curve. It has become a thing.

Japanese Unexamined Patent Publication No. 2006-40223

FIG. 11 is a diagram illustrating a conventional display device. The display display 10 is a display medium provided on the front surface of the driver or on the dashboard, and displays vehicle information such as a speedometer 12 and a tachometer 14, as well as an image 16 captured by an in-vehicle camera provided in the vehicle. be able to. When the vehicle travels in the daytime, a relatively bright image 16 is displayed, but when the vehicle travels at night, the surroundings are dark, so that the image 16 is difficult to see. If the in-vehicle camera is a night-vision camera (night-vision mode), it is possible to supplement the brightness to some extent, but it is still difficult to understand the road surface condition of the driving lane, and the distance from the vehicle or obstacle in front is also large. It's hard to grasp.

The present invention solves the above-mentioned conventional problems, and an object of the present invention is to provide a display system that makes it easy to recognize the road surface condition of the road in front of the vehicle.

The display system according to the present invention includes an image pickup means for imaging a road, a recognition means for recognizing a road surface state related to steps, waviness, unevenness, etc. of the road surface from the image captured by the image pickup means, and a grid corresponding to the road surface state. It has a generation means for generating an image of the shape and a projection means for projecting the grid-like image on the road in front of the own vehicle.

Preferably, the grid-like image includes a plurality of vertical lines and a plurality of horizontal lines, and the vertical line and the horizontal line include a curve corresponding to the road surface condition of the road. Preferably, the generation means generates the grid-like image so that the portion including the curve corresponding to the road surface condition of the road has a different display color from the other portions. Preferably, the grid-like image includes a plurality of grid points, and the plurality of grid points are arranged according to the road surface condition of the road. Preferably, in the grid-like image, portions that may be obstacles such as steps, waviness, and unevenness are represented by grid-like lines, and other portions are represented by grid points. Preferably, the display system further has detection means for detecting the brightness around the vehicle, and the projection means adjusts the brightness of the grid-like image based on the brightness detected by the detection means. .. Preferably, the detection means determines daytime or nighttime based on the brightness of the surroundings of the vehicle, and the projection means increases the brightness of the grid-like image during daytime travel and increases the brightness of the grid-like image during nighttime travel. Decrease the brightness of the image. Preferably, the display system further includes a display means for superimposing and displaying an image captured by the image pickup means and a grid-like image generated by the generation means.

According to the present invention, a grid-like image according to the condition of the road surface is projected on the road in front of the vehicle, so that the driver can directly see the road surface in front of the vehicle, but it is difficult to visually grasp the road surface. You will be able to grasp the situation.

It is a block diagram which shows the structure of the display system which concerns on embodiment of this invention. It is a figure which shows an example of the structure of a HUD. It is a figure which shows the functional structure of a display control program. It is a figure explaining the operation of a display control program concretely. It is a figure which illustrates the transformation example of the image created by a grid image creating part. It is a figure which shows the display example of the image by the projection image output part. It is a flow chart which shows the operation of a display control program. It is a figure which shows the functional structure of the display control program 200A which concerns on a modification. It is a flow diagram which shows the operation of the display control program which concerns on a modification. It is a figure which shows the example which displays the grid grid image on the liquid crystal display. It is a figure which illustrates the conventional display device.

Next, embodiments of the present invention will be described in detail with reference to the drawings. The display system according to the present invention includes an image projection device typified by HUD, which projects or projects an image on the front glass of a vehicle by HUD, superimposes this image on the real space, and projects the projected image (or projection) to the driver. The virtual image of the image) can be visually recognized.

Further, the display system according to the present invention has an image pickup camera or the like that captures the surroundings of the vehicle, and a composite image in which an image captured by the image pickup camera and an image projected on the HUD are superimposed is displayed on a liquid crystal display or the like. It can also be displayed on a display medium.

FIG. 1 is a block diagram showing a configuration of a display system according to an embodiment of the present invention. The display system 100 according to this embodiment includes an image pickup camera 110, an illuminance sensor 120, a HUD 130, a storage unit 140, an I / F unit 150, and a control unit 160.

The image pickup camera 110 includes, for example, an image pickup camera that captures the front of the own vehicle and an image pickup camera that captures the rear of the own vehicle. In this case, the image pickup camera 110 is installed at a position where the front vehicle and the rear vehicle can be imaged. The image captured by the image pickup camera 110 is provided to the control unit 160 as image pickup data. The image pickup camera 110 may be provided with a night-vision mode (a function of an infrared camera or the like) capable of capturing a road surface condition even at night. The illuminance sensor 120 is a sensor that detects the brightness around the vehicle, and is mounted on the exterior or interior of the vehicle.

The HUD 130 projects an image for displaying the HUD on the front window of the own vehicle, and the driver can see a virtual image of the image projected in the line-of-sight direction. FIG. 2 is a diagram showing an example of the configuration of the HUD. The HUD 130 projects an image modulated by a light source 131, an optical system 132 that collects light from the light source 131, an optical modulation means 133 that modulates the light incident from the optical system 132 based on image data, and an optical modulation means 133. Includes an actuator 135 for adjusting the projection optical system 134 and the optical system (for example, optical axis, focal distance, etc.) of the projection optical system 134.

The optical modulation means 133 is composed of, for example, a liquid crystal device or a digital mirror device (DMD) in which a plurality of variable mirrors are two-dimensionally arranged. The optical modulation means 133, for example, modulates the incident light based on the image data generated by the control unit 160 to generate a projected image. Further, when another device such as a navigation device is connected via the I / F unit 150, it is also possible to generate a projected image based on the image data provided by the navigation device or the like. In this embodiment, a grid display image can be displayed according to the road surface condition in front of the own vehicle (see FIGS. 4, 5, and 6).

The projection optical system 134 includes, for example, an optical member such as a lens or a mirror, and the actuator 135 adjusts, for example, the position of the lens of the projection optical system 134, the angle of the mirror, and the like. By varying the focal length of the projection optical system 134, the position of the virtual image seen by the driver can be changed. The display system 100 may be equipped with a plurality of HUD 130s and each display an image for augmented reality.

The storage unit 140 stores information and the like necessary for the display system 100. For example, image data for HUD display can be stored, and other programs for controlling the display system 100 can be stored. The image data includes at least data for generating the grid display image of FIGS. 4 to 6, which will be described later.

The I / F unit 150 enables connection with a navigation device, an in-vehicle bus, and other devices, and the control unit 160 links the operation of the device connected to the I / F unit 150 with the display system 100. For example, when a navigation device is connected, the display system 100 can display an image of intersection guidance, speed information, and the like based on the information provided by the navigation device.

The control unit 160 controls each unit of the display system 100, and in a preferred embodiment, the control unit 160 includes a microcontroller including a ROM, a RAM, and the like, and executes a display control program for controlling the operation of the display system 100. do.

FIG. 3 is a diagram showing a functional configuration of a display control program. The display control program 200 includes a road surface state recognition unit 210, a grid image creation unit 220, and a projection image output unit 230. Further, FIG. 4 is a diagram specifically explaining the operation of the display control program.

The road surface condition recognition unit 210 acquires image pickup data in front of the vehicle from the image pickup camera 110 and recognizes the road surface condition in front of the vehicle. In recognition of the road surface condition, in addition to recognizing the shape of the road on which the own vehicle travels, the shape of steps, swells, irregularities, grooves, etc. existing on the road is recognized. A known image analysis method can be used for recognizing the road surface condition. For example, as shown in Japanese Patent Application Laid-Open No. 2010-287015, three-dimensional image data is acquired by using two cameras, and unevenness is recognized. Can recognize the part. This makes it possible to recognize information such as the depth and shape of unevenness and the slope of the road. Further, the brightness of the image pickup data may be image-analyzed to extract unevenness and grooves in the road surface state, or steps, waviness, unevenness, grooves and the like may be stored in advance in the storage unit 140 to be used as the image pickup data. The road surface condition may be recognized by matching, or the feature points may be extracted from the image pickup data and the road surface condition may be recognized. Further, a device such as a distance measuring sensor or a millimeter wave radar may be provided in the vehicle, and the distance to an obstacle may be recognized by acquiring the detection results thereof.

The grid image creation unit 220 creates image data of an image projected by the HUD 130 based on the road surface condition recognized by the road surface condition recognition unit 210. FIG. 4 is a diagram illustrating an image created by the grid image creating unit. When the road surface condition recognized by the road surface condition recognition unit 210 is flat, that is, when there are no conspicuous uneven portions or grooves, the grid image creation unit 220 has a depth as shown in FIG. 4 (A). Create a flat image D1. The image D1 is an image that appears to be superimposed on the actual road R by the display of the HUD 130. The lengths X and Y between the grids of the image D1 are preferably determined based on the distance in real space. For example, the length X1 immediately before the vehicle and the length X2 at the back are planar (two-dimensional). In the target), X1> X2 due to the perspective effect, but both represent the same length (for example, 50 cm) in the real space. The length Y1 immediately before the vehicle and the length Y2 at the back also have Y1> Y2 in a plane, but both represent the same length (for example, 1 m) in the real space. Further, the grid-shaped image D1 may be an image created as a trapezoidal image that matches the shape of the road, and the spacing between the grid points has no meaning.

Further, when the road shape recognized by the road surface condition recognition unit 210 includes irregularities and grooves, the grid image creation unit 220 according to the present embodiment creates a grid image in consideration of these obstacles. FIG. 4B is a diagram showing an image D2 in consideration of unevenness and grooves. The convex portion recognized by the road surface condition recognition unit 210 includes a portion D21 distorted as if the road R is raised, and the concave portion or the groove includes a portion D22 distorted as if the road R is depressed. The degree of swelling and denting of D21 and D22 are created based on the depth and shape recognized by the road surface condition recognition unit 210. In order to make the distortion of D21 and D22 more clearly visible, the image data may be created so that the color of the distorted portion is different from the color of the other non-distorted grid portion. For example, the portion of D21 or D22 may be red, and the other grids may be a color other than red.

FIG. 5 is a diagram illustrating a modified example of the image created by the grid image creating unit. As shown in FIG. 5A, the grid image creation unit 220 may create image data for displaying a plurality of grid points. Further, as shown in FIG. 5B, image data may be created in which the image D21 of the raised portion is displayed in a grid pattern and the other portions are displayed as grid points. In the case of FIG. 5B, it is also possible to display the portion of D21 in a color different from the grid points.

The projection image output unit 230 provides the image data created by the grid image creation unit 220 to the HUD 130, and projects the image. FIG. 6 is a diagram showing an image display example by the projection image output unit. The HUD 130 mounted in the vehicle can receive the image data provided from the projection image output unit 230 and display the image D2. The projected image output unit 230 transmits a control signal to the actuator 135 to adjust the focal length so that the displayed image based on the provided image data can be seen superimposed on the road R2 in the actual scene.

FIG. 7 is a flow chart showing the operation of the display control program. The display control program 200 is repeatedly executed every fixed time or every fixed distance when the HUD 130 is used. The display control program 200 first recognizes the road surface condition in front of the own vehicle by the road surface condition recognition unit 210 (S100). Next, the grid image creation unit 220 creates image data for displaying a grid-like image based on the road surface condition recognized in S100 (S102). The projected image output unit 230 provides the image data generated in S102 to the HUD 130 and displays a grid image (S104).

Since the display system 100 according to the present embodiment can display an image that matches the uneven shape of the road, it is possible to instantly grasp the shape of steps, waviness, unevenness, grooves, etc. existing on the road. In addition, it becomes possible to grasp the road surface condition that is difficult to visually grasp while directly looking at the road surface in front.

In this embodiment, as shown in FIGS. 4 and 5, an example of displaying a grid image along the road shape is shown, but only the uneven portion is displayed in the grid, that is, only the portions D21 and D22 are displayed. The display form may be such that the grid display is not performed on other road areas.

Next, a modified example of this embodiment will be described. The display system 100 according to the modified example can include the configuration, functions, and the like shown in the above-described embodiment. FIG. 8 is a diagram showing a functional configuration of the display control program 200A according to the modified example. The display control program 200A includes an illuminance information acquisition unit 222 and a day / night determination unit 224 in addition to the road surface condition recognition unit 210, the grid image creation unit 220, and the projection image output unit 230 described above. The illuminance information acquisition unit 222 acquires illuminance information from the illuminance sensor 120. The day / night determination unit 224 determines whether the surroundings of the vehicle are in the daytime brightness state or the nighttime brightness state based on the illuminance information acquired by the illuminance information acquisition unit 222. If the display by HUD130 is too bright during night driving, it may be difficult to see the real world. Therefore, the day / night determination unit 224 determines the brightness, and if it is in the daytime brightness state, the brightness of the grid image is determined. Try to raise it. On the contrary, if the brightness is at night, it is assumed that the vehicle will run at night, so the brightness of the grid image should be increased. The determination by the day / night determination unit 224 can be made, for example, in the daytime when the illuminance above a certain threshold value is acquired, and in the nighttime when the illuminance below the threshold value is acquired. The determination by the day / night determination unit 224 determines the actual day or night, and even when the illuminance is similar to the night state such as a tunnel, it is determined to be night and the brightness of the grid image is determined. Can be changed. The projection image output unit 230 according to the modified example transmits a control signal to the HUD 130 based on the determination result of the day / night determination unit 224, and changes the brightness of the light source 131.

FIG. 9 is a flow chart showing the operation of the display control program according to the modified example. S100 and S102 are the same as the processes described in the flow of FIG. After creating the projected image, the display control program 200A according to the modified example acquires the information provided from the illuminance sensor 120 by the illuminance information acquisition unit 222 (S106). The day / night determination unit 224 determines whether it is nighttime brightness or daytime brightness based on the illuminance information acquired in S106 (S108). When the brightness is nighttime, the projected image output unit 230 projects the image at low brightness (S110), and when the brightness is not nighttime, that is, when the brightness is daytime, the projection image output unit 230 has high brightness. Image projection is performed in (S112).

Since the display system 100 according to the modified example changes the brightness based on the illuminance around the vehicle, the image projection by the HUD 130 becomes difficult to see due to the brightness of the surroundings, or becomes too visible and interferes with the actual scene. , Etc. can be alleviated.

The display system 100 described in this embodiment (including a modification) shows an example of projecting image data by the HUD 130, but the display system 100 is another display mode in which the captured image captured by the image pickup camera 110 is used. A grid-like grid image may be superimposed and displayed on a flat display medium such as a liquid crystal display. FIG. 10 is a diagram showing an example of displaying a grid grid image on a liquid crystal display. When the display 300 is provided in the display system 100 or the navigation device connected to the I / F unit 150, a composite image in which a grid image is superimposed on the image pickup data captured by the image pickup camera 110 is created. , The composite image can be displayed on the display 300. The generation and display control of the composite image is performed by the control unit 160. Such a display display may be performed in place of the display of the HUD 130, or may be performed in combination with the display of the HUD 130.

Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the specific embodiments, and various aspects of the present invention are described within the scope of the claims. It can be transformed and changed.

100: Display system 110: Imaging camera 120: Illuminance sensor 130: HUD
131: Light source 132: Optical system 133: Optical modulation means 134: Projection optical system 135: Actuator 140: Storage unit 150: I / F unit 160: Control unit 200: Display control program 210: Road surface condition recognition unit 220: Lattice image creation Unit 222: Illuminance information acquisition unit 224: Day / night determination unit 230: Projection image output unit 300: Display

Claims (8)

An imaging means that captures the road in front of the vehicle ,
A recognition means for recognizing an uneven portion of a road surface based on three-dimensional image data captured by the image pickup means, and a recognition means.
A generating means for generating a grid-like image that matches the shape of a road , wherein the grid-like image includes a plurality of vertical lines or points and a plurality of horizontal lines or points, and the vertical direction thereof. The line or point and the lateral line or point include a portion distorted so that the road surface rises when a convex portion is recognized, and includes a portion distorted so that the road surface is dented when a concave portion is recognized. , The generation means and
A display system comprising a projection means for projecting the grid-like image onto a front window so that the grid-like image appears to be superimposed on the road in front of the vehicle. The display system according to claim 1, wherein the distorted portion of the grid-like image is generated based on the depth and shape of the recognized uneven portion. The display system according to claim 1 or 2, wherein the generation means generates the grid-like image so that the distorted portion has a display color different from that of other portions. The display system according to any one of claims 1 to 3, wherein the grid spacing of the grid-like image is narrower on the back side than on the front side in order to represent the perspective effect. The display system according to claim 4, wherein the grid spacing of the grid-like images represents equal distances in real space. The display system further includes a detection means for detecting the brightness around the vehicle, and the projection means adjusts the brightness of the grid-like image based on the brightness detected by the detection means. The display system according to any one of 1 to 5. The detection means determines daytime or nighttime based on the brightness around the vehicle, and the projection means increases the brightness of the grid-like image during daytime travel and increases the brightness of the grid-like image during nighttime travel. The display system according to claim 6, which reduces the brightness of the image. The display system according to any one of claims 1 to 7, further comprising a display means for superimposing and displaying an image captured by the image pickup means and a grid-like image generated by the generation means. Display system.

Images