JP6726412B2 - Image display device, moving body, image display method and program - Google Patents

Description

The present invention relates to an image display device, the moving member and at about the image display method and a program.

Is in this type of image display apparatus, a head-up display for displaying vehicle that moves carrying the driver, a ship, an aircraft, an image for providing information to the driver of the moving object, such as industrial robots A device using a (HUD) device or the like is known.

Patent Document 1 discloses a HUD device that projects image light onto a windshield or the like (light-transmitting member) and displays the image on a front landscape visually recognized by a driver of a vehicle (moving body) through the windshield. Has been done. This HUD device superimposes and displays an arrow indicating the direction of travel, a display object indicating speed, attention, warning, and the like as a virtual image on the front landscape. The HUD device includes a position detection unit (viewpoint position detection unit) that images the driver and detects the position of one eye of the driver, and determines the position of each display object in the virtual image according to the detection result. Perform the process of changing. In this process, when the driver moves his/her head to move the viewpoint position (the position of one eye), the amount of movement in the virtual image of each display object is made different, so that the motion parallax causes a difference in the depth direction of each display object. The driver is made to perceive that the display position (subjective depth position) is different.

Passive information that is passively recognized by the driver is generally information that the driver wants to provide at the timing determined by the information providing device, and normally, the timing when the passive information is provided and the content of the passive information. It is information that has a certain relationship with. Therefore, it is important that the driver recognizes the passive information image showing the passive information immediately after displaying the passive information image.

In order to solve the above-described problems, the present invention provides a position detection unit that detects a viewpoint position of a driver who drives a moving body, an active information image that is actively recognized by the driver, and provision of predetermined information. A passive information image that is displayed so that the driver can recognize it when the conditions are satisfied, and a display unit that visually displays it to the driver via a light transmissive member, and the display unit is the position by displaying by changing the display position of the passive information image on the basis of the detection information of the detecting section in a direction orthogonal to the depth direction in a predetermined display area, the passive information displayed on the predetermined display area the perceptual metric of the driver image, wherein the distance in Table Shimesuru than perceptual metric of the driver of the active information image displayed in a separate display area which is located below the said predetermined display region And

According to the present invention, the driver displays information (surrounding object information) about the relative movement of an object (detection target) around the moving body with respect to the moving body, using an image displayed in the display area in front of the moving body. It has an excellent effect that it can be easily recognized.

In an embodiment, it is an explanatory view showing an example of a virtual image displayed on a display area by superimposing it on a scenery in front of the vehicle viewed from the driver through the windshield. It is the schematic diagram which represented typically the structure of the motor vehicle which mounts the HUD apparatus for motor vehicles in embodiment. It is a schematic diagram which represented typically the internal structure of the HUD apparatus for motor vehicles. It is a hardware block diagram of a control system in the HUD device for the vehicle. It is a block diagram showing a schematic structure of a driver information providing system in an embodiment. It is explanatory drawing which shows the hardware constitutions of the object recognition apparatus in the same driver information provision system. It is a hardware block diagram which shows the main hardware of the image control apparatus in the HUD apparatus for vehicles. 6 is an explanatory diagram for describing an image processing method of a virtual image having a sense of depth due to motion parallax in the embodiment. FIG. It is an explanatory view showing an example of a picture of a situation where a course is changed at the latest intersection. (A)-(e) is explanatory drawing which shows the example 1 of an image which the image which shows front vehicle approach information changes according to the inter-vehicle distance with a preceding vehicle. (A) And (b) is explanatory drawing which shows the other image example 2 in which the image which shows front vehicle approach information changes according to the inter-vehicle distance with a preceding vehicle. (A)-(c) is explanatory drawing which shows the example 3 of further another image from which the image which shows front vehicle approach information changes according to the inter-vehicle distance with a preceding vehicle. (A)-(c) is explanatory drawing which shows the other example image 4 of the condition which changes a course in the nearest intersection. It is explanatory drawing which shows the example 5 of an image which displayed the school road warning image in the middle stage display area, when the road where the own vehicle is driving is a school road. It is explanatory drawing which shows the image example 6 which displayed the humanoid warning image in the middle stage display area, when the person trying to cross the pedestrian crossing in the advancing direction of the own vehicle was detected.

An embodiment in which the present invention is applied to a driver information providing system including a vehicle head-up display (HUD) device as an information providing device will be described below.
FIG. 1 is an explanatory diagram showing an example of a virtual image G displayed in a display area 700 overlaid on a landscape in front of the vehicle 301 viewed from the driver 300 through the windshield 302.
FIG. 2 is a schematic diagram schematically showing the configuration of an automobile equipped with the automobile HUD device according to the present embodiment.
FIG. 3 is a schematic diagram schematically showing the internal configuration of the automobile HUD device in the present embodiment.

The automobile HUD device 200 according to the present embodiment is installed, for example, in a dashboard of a vehicle 301 that is a traveling body as a moving body. Projection light L, which is image light emitted from the automobile HUD device 200 in the dashboard, is reflected by the windshield 302 as a light transmitting member, and travels toward the driver 300. Thereby, the driver 300 can visually recognize a HUD display image such as a navigation image described later as a virtual image. A combiner as a light transmitting member may be installed on the inner wall surface of the windshield 302 so that the driver can visually recognize the virtual image by the projection light L reflected by the combiner.

In the present embodiment, the optical system and the like of the HUD device 200 for automobiles are configured such that the distance from the driver 300 to the virtual image G is 5 m or more. In the conventional general HUD device for automobiles, the distance from the driver 300 to the virtual image G is about 2 m. The driver 300 is normally gazing at an infinite point in front of the vehicle or gazing at a preceding vehicle several tens of meters ahead. When the driver 300 focusing on such a distant place wants to visually recognize the virtual image G 2 m ahead, the focal lengths are significantly different, and therefore the crystalline lens of the eyeball needs to be largely moved. Therefore, the focus adjustment time for focusing on the virtual image G becomes long, it takes time to recognize the contents of the virtual image G, and the eyes of the driver 300 are easily fatigued. Further, it is difficult for the driver to notice the contents of the virtual image G, and it is difficult to appropriately provide information to the driver by the virtual image G.

If the distance to the virtual image G is 5 m or more as in the present embodiment, the amount of movement of the crystalline lens of the eyeball is smaller than in the conventional case, the focus adjustment time to the virtual image G is shortened, and the contents of the virtual image G are recognized earlier. Therefore, the fatigue of the eyes of the driver 300 can be reduced. Further, the driver can easily notice the contents of the virtual image G, and the virtual image G facilitates appropriately providing information to the driver.

Further, when the distance to the virtual image G is about 2 m, when the eyeball is focused on the virtual image G having such a short distance, a vergence movement of the eyeball is usually required. The vergence movement is a factor that greatly affects the sense of distance and the sense of depth to the visual target. In the present embodiment, as will be described later, display control is performed so that the perceived distance of the image displayed as the virtual image G is perceived by the motion parallax. In this case, if the eyeball performs a vergence movement to focus on the virtual image G, the effect of perceiving a sense of distance (change in perceptual distance) or a sense of depth (difference in perceptual distance) due to motion parallax is diminished. The information recognition effect of the driver, which will be described later, using the difference or change in the perceived distance of the vehicle will be reduced.

When the distance to the virtual image G is 5 m or more, it is possible to focus on the virtual image G with almost no vergence movement of the eyeball. Therefore, the effect of perceiving a sense of distance (change in perceived distance) or a sense of depth (difference in perceived distance) by using motion parallax is suppressed from fading due to the vergence movement of the eyeball. Therefore, the driver's information perception effect using the sense of distance and the sense of depth of the image can be effectively exhibited.

The HUD device 200 for an automobile includes red, green, and blue laser light sources 201R, 201G, and 201B, collimator lenses 202, 203, and 204 provided for the respective laser light sources, and two dichroic mirrors 205 in a HUD main body 230. , 206, a light quantity adjusting unit 207, an optical scanning device 208 as an optical scanning unit, a free-form curved mirror 209, a microlens array 210 as a light diverging member, and a projection mirror 211 as a light reflecting member. ing. In the light source unit 220 in this embodiment, the laser light sources 201R, 201G, 201B, the collimator lenses 202, 203, 204, and the dichroic mirrors 205, 206 are unitized by an optical housing.

An LD (semiconductor laser element) can be used as the laser light sources 201R, 201G, and 201B. The wavelength of the light beam emitted from the red laser light source 201R is, for example, 640 nm, the wavelength of the light beam emitted from the green laser light source 201G is, for example, 530 nm, and the wavelength of the light beam emitted from the blue laser light source 201B is, for example, 445 nm. ..

The HUD device 200 for automobiles of the present embodiment projects an intermediate image formed on the microlens array 210 onto the windshield 302 of the host vehicle 301, so that an enlarged image of the intermediate image is displayed to the driver 300 as a virtual image G. To make it visible. The laser light of each color emitted from the laser light sources 201R, 201G and 201B is made into substantially parallel light by the collimator lenses 202, 203 and 204, respectively, and is combined by the two dichroic mirrors 205 and 206. The combined laser light is two-dimensionally scanned by the mirror of the optical scanning device 208 after the light amount is adjusted by the light amount adjusting unit 207. The scanning light L′ that has been two-dimensionally scanned by the optical scanning device 208 is reflected by the free-form surface mirror 209 to have its distortion corrected, and then condensed on the microlens array 210 to draw an intermediate image.

In the present embodiment, the microlens array 210 is used as the light diverging member that individually diverges and emits the light flux of each pixel of the intermediate image (one point of the intermediate image), but other light diverging members are used. May be. As a method of forming the intermediate image G', a method using a liquid crystal display (LCD) or a fluorescent display tube (VFD) may be used.

However, in order to display the large virtual image G with high brightness, the laser scanning method as in this embodiment is preferable.
Further, in the method using a liquid crystal display (LCD) or a fluorescent display tube (VFD), a slight amount of light is also applied to the non-image portion in the display area where the virtual image G is displayed, so that it is completely blocked. Is difficult. Therefore, there is a demerit that the visibility of the front landscape of the host vehicle 301 through the non-image portion is poor. On the other hand, according to the laser scanning method as in the present embodiment, with respect to the non-image portion in the display area of the virtual image G, the laser light sources 201R, 201G, and 201B are turned off to irradiate the non-image portion with light. Can be completely blocked. Therefore, it is possible to avoid a situation where the visibility of the front landscape of the host vehicle 301 through the non-image portion is reduced by the light emitted from the automotive HUD device 200, and there is an advantage that the visibility of the front landscape is high.

Further, when the degree of warning is strengthened by gradually increasing the brightness of the warning image for warning the driver, only the brightness of the warning image among the various images displayed in the display area 700 is displayed. It is necessary to control the display by gradually increasing. The laser scanning method is also suitable when the display control is performed so as to partially increase the brightness of a partial image in the display area 700. In a method using a liquid crystal display (LCD), a fluorescent display tube (VFD), or the like, the brightness of images other than the warning image displayed in the display area 700 is increased, and the warning image and the other images are not displayed. This is because it is not possible to widen the difference in brightness between them, and it is not possible to sufficiently obtain the effect of strengthening the degree of warning by increasing the brightness of the warning image stepwise.

The optical scanning device 208 tilts the mirror in the main scanning direction and the sub-scanning direction by a known actuator driving system such as MEMS (Micro Electro Mechanical Systems) and two-dimensionally scans (raster scans) the laser light incident on the mirror. .. The drive control of the mirror is performed in synchronization with the light emission timing of the laser light sources 201R, 201G, and 201B. The optical scanning device 208 is not limited to the configuration of this embodiment, and may be configured by, for example, a mirror system including two mirrors that respectively swing or rotate about two axes orthogonal to each other.

FIG. 4 is a hardware block diagram of a control system in the automobile HUD device 200 of the present embodiment.
The control system of the automobile HUD device 200 mainly includes an FPGA 251, a CPU 252, a ROM 253, a RAM 254, an I/F 255, a bus line 256, an LD driver 257, and a MEMS controller 258. The FPGA 251 controls the operation of the laser light sources 201R, 201G, and 201B of the light source unit 220 by the LD driver 257, and controls the operation of the MEMS 208a of the optical scanning device 208 by the MEMS controller 258. The CPU 252 controls each function of the automotive HUD device 200. The ROM 253 stores various programs such as an image processing program executed by the CPU 252 to control each function of the automotive HUD device 200. The RAM 254 is used as a work area for the CPU 252. The I/F 255 is an interface for communicating with an external controller or the like, and for example, via a CAN (Controller Area Network) of the host vehicle 301, to an object recognition device 100, a vehicle navigation device 400, various sensors 500, and the like described later. Connected.

FIG. 5 is a block diagram showing a schematic configuration of the driver information providing system in the present embodiment.
In the present embodiment, an object recognition device 100, a vehicle navigation device 400, a sensor device 500, etc. are provided as information acquisition means for acquiring the driver provision information provided to the driver by the virtual image G. The vehicle HUD device 200 in the present embodiment mainly includes a HUD main body 230 as an image light projection unit and an image control device 250 as a display control unit. The information acquisition means in the present embodiment is mounted on the host vehicle 301, but the information acquired by the information acquisition means is input via the communication means using the information acquisition means installed outside the host vehicle 301. It may be configured.

FIG. 6 is an explanatory diagram showing the hardware configuration of the object recognition device 100 in this embodiment.
The object recognition device 100 of the present embodiment is present in the imaging area based on the stereo camera unit 110 as an imaging unit for imaging the front area of the host vehicle 301 as the imaging area, and the captured image data captured by the stereo camera unit 110. The information processing unit 120 is an image processing unit that executes image processing for recognizing a predetermined recognition target object. Instead of the stereo camera unit 110, a configuration in which a monocular camera as an image pickup unit and a laser radar (millimeter wave radar) as a distance measuring unit are combined may be adopted.

The stereo camera unit 110 is configured by assembling two camera units in parallel, a first camera unit 110A for the left eye and a second camera unit 110B for the right eye. Each of the camera units 110A and 110B includes a lens 115, an image sensor 116, and a sensor controller 117, respectively. As the image sensor 116, for example, an image sensor including a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) can be used. The sensor controller 117 controls exposure of the image sensor 116, image reading control, communication with an external circuit, transmission control of image data, and the like. The stereo camera unit 110 is installed near the rearview mirror of the windshield 302 of the vehicle 301.

The information processing unit 120 includes a data bus line 121, a serial bus line 122, a CPU (Central Processing Unit) 123, an FPGA (Field-Programmable Gate Array) 124, a ROM (Read Only Memory) 125, a RAM (Random Access Memory) 126, and It has a serial IF (Interface) 127 and a data IF 128.

The stereo camera unit 110 is connected to the information processing unit 120 via a data bus line 121 and a serial bus line 122. The CPU 123 controls execution of each sensor controller 117 of the stereo camera unit 110, operation of the entire information processing unit 120, image processing, and the like. The brightness image data of the captured image captured by the image sensor 116 of each of the camera units 110A and 110B is written to the RAM 126 of the information processing unit 120 via the data bus line 121. Sensor exposure value change control data, image read parameter change control data, and various setting data from the CPU 123 or FPGA 124 are transmitted and received via the serial bus line 122.

The FPGA 124 performs processing requiring real-time processing on the image data stored in the RAM 126, for example, gamma correction, distortion correction (parallelization of left and right images), and parallax calculation by block matching to generate a parallax image, Rewrite to RAM 18. The ROM 125 includes a recognition program for recognizing a three-dimensional object such as a vehicle or a pedestrian, or a predetermined recognition target object such as a lane boundary line such as a white line on the road surface, a curb stone or a median strip existing on the side of the road surface. Is remembered. The recognition program is an example of an image processing program.

The CPU 123 acquires CAN information such as vehicle speed, acceleration, steering angle, yaw rate, etc. from the sensor device 500 via the data IF 128, for example, the CAN of the host vehicle 301. Then, the CPU 123 executes image processing using the luminance image and the parallax image stored in the RAM 126 according to the recognition program stored in the ROM 125, and recognizes a recognition target object such as the preceding vehicle 350 or a lane boundary line. I do.

The recognition result data of the recognition target object is supplied to external devices such as the image control device 250 and the vehicle traveling control unit via the serial IF 127. The vehicle traveling control unit uses the recognition result data of the recognition target object to perform brake control, speed control, steering control, etc. of the host vehicle 301, and, for example, controls the host vehicle 301 so as to maintain a preset inter-vehicle distance. It realizes cruise control that automatically tracks the preceding vehicle and automatic brake control that avoids and reduces collisions with obstacles ahead.

The vehicle navigation device 400 according to the present embodiment can widely use known vehicle navigation devices mounted on automobiles and the like. Information necessary for generating the route navigation image displayed on the virtual image G is output from the vehicle navigation device 400, and this information is input to the image control device 250. For example, as shown in FIG. 1, the number of lanes (driving lanes) of the road on which the vehicle 301 is traveling, the distance to a point where the course should be changed next (turn right, turn left, branch, etc.), and the course change next. An image showing information such as a direction to do is included. By inputting these pieces of information from the vehicle navigation device 400 to the image control device 250, the traveling lane instruction image 711, the route designation image 721, and the remaining distance image are controlled by the automobile HUD device 200 under the control of the image control device 250. Navigation images such as 722 and name images 723 such as intersections are displayed in the upper display area A and the middle display area B of the display area 700.

Further, in the image example shown in FIG. 1, an image showing the unique information of the road (road name, speed limit, etc.) is displayed in the lower display area C of the display area 700. The unique information of this road is also input from the vehicle navigation device 400 to the image control device 250. The image control device 250 displays the road name display image 701, the speed limit display image 702, the overtaking prohibition display image 703, etc. corresponding to the road specific information in the lower display area C of the display area 700 by the automotive HUD device 200. ..

The sensor device 500 according to the present embodiment includes one or more sensors for detecting various information indicating the behavior of the host vehicle 301, the state of the host vehicle 301, the situation around the host vehicle 301, and the like. The sensor device 500 outputs sensing information necessary for generating an image to be displayed as the virtual image G, and this sensing information is input to the image control device 250. For example, in the image example shown in FIG. 1, a vehicle speed display image 704 (a character image “83 km/h” in FIG. 1) indicating the vehicle speed of the host vehicle 301 is displayed in the lower display area C of the display area 700. Therefore, the vehicle speed information included in the CAN information of the host vehicle 301 is input from the sensor device 500 to the image control device 250, and under the control of the image control device 250, the vehicle HUD device 200 displays a character image indicating the vehicle speed. It is displayed in the lower display area C of 700.

The sensor device 500 is, for example, a sensor other than the sensor that detects the vehicle speed of the host vehicle 301, and is, for example, another vehicle existing around the host vehicle 301 (front, side, or rear), a pedestrian, a building (guardrail, telephone pole, or the like). A laser radar device or an imaging device for detecting the distance to the vehicle, a sensor for detecting external environment information (outside temperature, brightness, weather, etc.) of the vehicle, a driving operation of the driver 300 (brake scanning, accelerator opening/closing degree, etc.) ), a sensor for detecting the remaining amount of fuel in the fuel tank of the host vehicle 301, a sensor for detecting the states of various in-vehicle devices such as an engine and a battery, and the like. By detecting such information with the sensor device 500 and sending it to the image control device 250, it is possible to display the information as a virtual image G on the automotive HUD device 200 and provide it to the driver 300.

FIG. 7 is a hardware block diagram showing the main hardware of the image control device 250.
In the image control device 250, a CPU 251, a RAM 252, a ROM 253, an input data IF 254, and an output data IF 255 are connected to each other by a data bus line. Various recognition result data output from the object recognition device 100, sensing information output from the sensor device 500, various information output from the vehicle navigation device 400, and the like are input to the input data IF 254. From the output data IF 255, a control signal or the like for the HUD device 200 for an automobile is output. The CPU 251 executes various computer programs such as an information providing control program stored in the ROM 253 and the like, and causes the image control device 250 to perform various controls and various processes described below.

Next, the virtual image G displayed by the automobile HUD device 200 will be described.
In the vehicle HUD device 200 according to the present embodiment, the driver-provided information provided to the driver by the virtual image G may be any information that is useful for the driver. In this embodiment, the driver-provided information is roughly classified into passive information and active information.

The passive information is information that is passively recognized by the driver at the timing when a predetermined information providing condition is satisfied. Therefore, the information provided to the driver at the setting timing of the automobile HUD device 200 is included in the passive information, and the information having a certain relationship between the timing when the information is provided and the content of the information is passive. Included in the information. Examples of the passive information include information related to safety during driving, route navigation information, and the like. As information related to safety during driving, for example, vehicle-to-vehicle distance information between the own vehicle 301 and the preceding vehicle 350 (vehicle-to-vehicle distance presentation image 712), urgent information related to driving (instruct the driver to perform an emergency operation). Warning information such as emergency operation instruction information or warning information). The route navigation information is information for guiding a preset traveling route to a destination, and is provided to the driver by a known vehicle navigation device. The route navigation information includes travel lane instruction information (travel lane instruction image 711) for instructing a travel lane to be traveled at the nearest intersection, and a course change operation at an intersection or a branch point where the course should be changed from the next straight direction. Examples include route change operation instruction information to be instructed. As the route change operation instruction information, specifically, the route designation information (route designation image 721) for designating the route to be taken at the intersection or the like, and the rest up to the intersection or the like for performing the route change operation. Examples include distance information (remaining distance image 722), name information of the intersection and the like (intersection name image 723), and the like.

The active information is information that is actively recognized by the driver at a timing determined by the driver himself. The active information is sufficient information if it is provided to the driver at the timing desired by the driver.For example, information that has a low or no relationship between the timing at which the information is provided and the content of the information is available. , Included in active information. The active information is information that is acquired by the driver at the timing desired by the driver, and is thus information that is continuously displayed for a certain long period or at all times. For example, specific information of the road on which the host vehicle 301 is traveling, vehicle speed information of the host vehicle 301 (vehicle speed display image 704), current time information, and the like can be given. The specific information of the road includes, for example, road name information (road name display image 701), restriction content information such as speed limit of the road (speed limit display image 702, overtaking prohibition display image 703), and other related roads. The information includes information useful to the driver.

In the present embodiment, the passive information and the active information thus roughly classified are displayed in the corresponding display areas in the display area 700 capable of displaying the virtual image G. Specifically, in the present embodiment, the display area 700 is vertically divided into three display areas, and the passive information image corresponding to the passive information is displayed in the upper display area A and the middle display area B among them. In the lower display area C, an active information image corresponding to active information is displayed. Although a part of the active information image may be displayed in the upper display area A or the middle display area B, in that case, the visibility of the passive information image displayed in the upper display area A or the middle display area B may be improved. The active information image is displayed with priority.

The passive information of the present embodiment includes operation instruction information indicating operation instruction contents to the driver 300, such as route navigation information. When such operation instruction information is provided to the driver, normally, not only the operation instruction image showing the latest operation instruction content but also the operation instruction image showing the subsequent operation instruction content is displayed together. It is preferable in that it can eliminate the anxiety (because it is unclear what operation will be performed thereafter). Also in the example shown in FIG. 1, together with the traveling lane instruction image 711 that indicates the traveling lane to be traveled at the nearest intersection, the route changing operation is instructed at the intersection or branch point where the route should be changed next from the straight traveling direction. The route change operation instruction images 721, 722, 723 are displayed.

In this way, when a plurality of operation instruction images with different operation sequences, that is, the traveling lane instruction image 711 and the course change operation instruction images 721, 722, 723 are displayed in the display area 700 at the same time, each operation instruction image is displayed. It is desirable to avoid as much as possible a situation in which the driver is confused about the operation sequence of the operation instruction content shown in the image.

Therefore, in the present embodiment, image display control is performed so that a plurality of operation instruction images with different operation orders are displayed on the upper side in the display area 700 as the operation instruction image with which the driver operates the later operation instruction images. Is going. That is, the route change operation instruction images 721, 722, 723 whose operation sequence comes later than the traveling lane instruction image 711 are displayed above the traveling lane instruction image 711. As a result, it is possible to avoid a situation in which the driver is confused about the operation sequence regarding the operation instruction content of the traveling lane instruction image 711 and the operation instruction content of the route change operation instruction images 721, 722, 723. This is for the following reason.

The driver 300 who is driving normally has a concept of distance in which an object located in front of the traveling own vehicle 301 approaches the present position over time. On the other hand, the driver 300 also has the idea of time in which future events will approach the current time as time passes. An object existing in front of the traveling own vehicle 301 has a shorter time to reach the position of the own vehicle 301 as the distance to the own vehicle 301 is shorter, and a position of the own vehicle 301 as the distance to the own vehicle 301 is farther. It takes a long time to reach. Therefore, an object far away from the traveling own vehicle 301 in front has a high cognitive affinity with a far future event, and an object located near the front of the traveling own vehicle 301 has a closer future event. Has a high cognitive affinity with.

In consideration of this affinity, an operation instruction image of a distant future (an operation instruction image showing operation instruction contents whose operation sequence is later, so that it overlaps with a position where an object far away from the traveling own vehicle 301 is visually recognized, That is, the route change operation instruction images 721, 722, 723) are displayed, and the operation in the closer future (including the present) is performed so that the object located near the front of the traveling vehicle 301 is overlapped with the visible position. By displaying an instruction image (an operation instruction image indicating the operation instruction content that comes first in the operation order, that is, the traveling lane instruction image 711), the driver can intuitively recognize the operation order of these operation instruction content. You can

Here, as described above, the driver 300 who is driving is usually gazing at the infinity point in front of the own vehicle or gazing at the back surface of the preceding vehicle 350 traveling several tens of meters ahead. The gazing point is approximately in the vertical center of the front view seen from the windshield 302. In the present embodiment, the display area 700 in which the traveling lane instruction image 711 and the route change operation instruction images 721, 722, 723 having different operation sequences are displayed is positioned at the lower part of the front landscape seen from the windshield 302. .. Therefore, the traveling lane instruction image 711 and the route change operation instruction images 721, 722, 723 are displayed below the gazing point of the driver 300 so as to overlap the front scenery. The forward scenery portion below the gazing point of the driver 300 is viewed as an upper portion for an object located farther from the host vehicle 301 and is viewed as a lower portion for an object located closer to the host vehicle 301.

Therefore, in the present embodiment, the route change operation instruction images 721, 722, 723 whose operation sequence comes later than the traveling lane instruction image 711 are displayed above the traveling lane instruction image 711. Specifically, in the upper display area A of the display area 700 that overlaps with the visual recognition position of the object far away from the traveling vehicle 301, the route change operation instruction images 721, 722, 723 whose operation sequence is later displayed are displayed. In the middle display area B of the display area 700 that overlaps with the visual recognition position of an object located near the front of the traveling own vehicle 301, a traveling lane instruction image 711 having an earlier operation sequence is displayed. This allows the driver to intuitively recognize the operation order of these operation instruction contents due to the high degree of recognition affinity described above, and avoids a situation where the driver gets lost in the operation order. be able to.

On the contrary, for example, the display positions of the traveling lane instruction image 711 and the course change operation instruction images 721, 722, 723 may be set upside down in the present embodiment without considering the affinity of the recognition. If the drivers are placed side by side, the idea of distance and the idea of time described above hinder the driver from intuitively recognizing the operation sequence.

Further, by utilizing this recognition affinity, in the present embodiment, a plurality of operation instruction images related to the operation instruction contents having the same operation order are arranged side by side. Specifically, for each of the three route change operation instruction images 721, 722, 723 displayed in the upper display area A, the route change operation at the intersection or branch point where the route should be changed next from the straight direction is performed. Since it indicates the route change operation instruction information to be instructed, it is arranged side by side in the upper display area A. According to the concept of distance and the concept of time described above, the three course changing operation instruction images 721, 722, 723 arranged side by side at the same vertical position are information regarding the operation instruction content at the same time. Easy to recognize intuitively. Therefore, the information of the route change operation instruction images 721, 722, 723 can be appropriately recognized.

Further, in the present embodiment, as the virtual image G displayed in the display area 700, a stereoscopic image represented using stereoscopic vision is used. Specifically, the perspective image represented by the perspective method is used as the inter-vehicle distance presentation image 712 and the traveling lane instruction image 711 displayed in the middle display area B of the display area 700.

In detail, the perspectives drawn by perspective drawing are such that the lengths of the five horizontal lines forming the inter-vehicle distance presentation image 712 become shorter toward the upper side, and the inter-vehicle distance presentation image 712 goes to one vanishing point. It is a legal image. In particular, in the present embodiment, the inter-vehicle distance presentation image 712 is displayed so that its vanishing point is set near the gazing point of the driver, so that the driver 300 who is driving perceives the sense of depth of the inter-vehicle distance presentation image 712. Easy to make. Further, in this embodiment, the perspective image is such that the thickness of the horizontal line becomes thinner toward the upper side and the brightness of the horizontal line becomes lower toward the upper side. This makes it easier for the driver 300 who is driving to perceive the depth of the inter-vehicle distance presentation image 712.

By giving a sense of depth to the virtual image G displayed in the display area 700 by using such a stereoscopic image, the vertical position of the image displayed in the display area 700 and the distance to the object in the front landscape are determined. It becomes easier for the driver to perceive the relevance. As a result, the above-described cognitive affinity is more easily obtained, and the operation instruction content of the driving lane instruction image 711 displayed in the middle display area B of the display area 700 is displayed in the upper display area A of the display area 700. It becomes easier for the driver to intuitively recognize that the contents should be operated before the displayed route change operation instruction images 721, 722, 723.

Similarly to the inter-vehicle distance presentation image 712, the traveling lane instruction image 711 is also drawn by the perspective drawing so that the same point as the vanishing point of the inter-vehicle distance presentation image 712 becomes the vanishing point. However, the driving lane instruction image 711 has an aspect of providing the driver with operation instruction content. Therefore, when giving priority to more clearly providing the driver with the operation instruction content of the traveling lane instruction image 711 rather than giving a more sense of depth, for example, the three lanes constituting the traveling lane instruction image 711 are set. The brightness of the arrow image may be increased toward the tip of the arrow. In the laser scanning method, the brightness adjustment for each image area is easier than in other methods, and therefore the contrast between the low brightness area and the high brightness area can be increased. Therefore, as described above, it is effective when the brightness of the tip of the arrow is increased and information is clearly provided to the driver. In this case, since the brightness of the traveling lane instruction image 711 becomes lower toward the upper side, it is easier to give a feeling of depth by decreasing the brightness toward the upper side. By increasing the brightness of the arrow, the visibility of the direction of the arrow is enhanced, and the operation instruction content of the traveling lane instruction image 711 can be more clearly provided to the driver. Even in this case, since the driving lane instruction image 711 is drawn by the perspective drawing method, the sense of depth is not significantly impaired.

Next, a method of making a sense of distance and a sense of depth by making the distance to the virtual image G perceive by using motion parallax will be described.
In this embodiment, a motion parallax image represented by motion parallax is used as the virtual image G. The motion parallax means a parallax caused by the movement of the driver's 300 eye position (viewpoint position). When the eyes of the driver 300 move, the sense of distance and the sense of depth due to the motion parallax is recognized as a shift in movement that is recognized such that the closer an object in the front scene is, the larger the object moves, and the far the object is, the less the object moves. The person 300 perceives a sense of distance and a sense of depth to each object.

In the present embodiment, as shown in FIG. 2, the driver camera 150 as a viewpoint position detecting means for observing the position of the eyes (viewpoint position) of the driver 300 is provided near the rearview mirror of the windshield 302 of the host vehicle 301. is set up. It is preferable that the driver camera 150 is installed near the midline of the driver 300 sitting in the driver's seat in order to accurately acquire the vertical and horizontal movements of the driver 300. However, it is preferable that the driver 300 is arranged at an upper position so as not to obstruct the view of the driver 300.

The driver camera 150 is a monocular camera that is set so as to capture an image of a range in which the driver 300 sitting in the driver's seat is expected to move his/her head while driving, and each of the stereo camera units 110. Similar to the camera units 110A and 110B, it is composed of a lens, an image sensor, a sensor controller, and the like. A stereo camera may be used as the driver camera 150, and the position of the driver's eyes in the front-rear direction may be grasped.

The luminance image data of the captured image captured by the driver camera 150 is input to the image control device 250. The image control device 250 recognizes the position of the eyes of the driver 300 based on the brightness image data from the driver camera 150 by the CPU 251 executing the information providing control program stored in the ROM 253 or the like. In the present embodiment, the head position of the driver 300 is simply recognized based on the brightness image data from the driver camera 150, and the eye position of the driver 300 is estimated from the recognition result. As a method of recognizing the head position of the driver 300, a general recognition processing method can be widely adopted.

FIG. 8 is an explanatory diagram for describing the image processing method of the virtual image G having a sense of depth due to the motion parallax in the present embodiment.
As shown in FIG. 8, when the head of the driver 300 moves by Dd, the visual recognition position of the object Oa located at a distance La close to the driver 300 moves by Da and moves at a distance Lb far from the driver 300. The visual recognition position of the object Ob to be moved moves by Db smaller than Da, and the visual recognition position of the object Ob located at a distance Lc farther from the driver 300 moves by Dc smaller than Db. Due to the difference in the movement amount Da, Db, Dc of the visual recognition position among these objects Oa, Ob, Oc, the driver 300 is present at the position where the object Oa is separated by the distance La and the object Ob is separated by the distance Lb. It can be perceived that the object Oc exists at a position and the object Oc exists at a position separated by the distance Lc.

The virtual image G in the present embodiment is displayed at a position at a distance of 5 m from the driver 300, and any image portion on the virtual image G is displayed at a position at a distance of 5 m from the driver 300. In the present embodiment, by using the above-described motion parallax, the driver 300 is made to perceive as if the plurality of image portions on the virtual image G are displayed at different distances from each other.

Specifically, the image control device 250 recognizes the head position of the driver 300 at predetermined time intervals based on the brightness image data of the captured image captured by the driver camera 150. Then, the image control device 250 calculates a driver head movement amount Dd in which the head of the driver 300 has moved during the time interval. At this time, the visual recognition position of the virtual image G displayed at a distance of 5 m moves by Da.

In the present embodiment, the image portion displayed in the lower display area C has a fixed display position in the display area 700. Therefore, the visible position of the image portion displayed in the lower display area C moves by the same movement amount Da as the movement amount of the virtual image G. Therefore, the image portion displayed in the lower display area C is perceived by the driver 300 as if it is displayed at the distance La (5 m).

On the other hand, the image control device 250 moves the driver within the display area 700 for the image portion displayed in the middle display area B of the display area 700 of the virtual image G according to the calculated driver head movement amount Dd. Only Da-Db is moved in the direction opposite to the head moving direction. As a result, with respect to the image portion displayed in the middle display area B, the visual recognition position seen from the driver 300 moves by the movement amount Db. As a result, the image portion displayed in the middle display area B is perceived by the driver 300 as if it is displayed at the distance Lb.

Similarly, the image control device 250 drives the display area 700 for the image portion displayed in the upper display area A of the display area 700 of the virtual image G according to the calculated driver head movement amount Dd. The head is moved by Da-Dc in the opposite direction. As a result, for the image portion displayed in the upper display area A, the visual recognition position seen by the driver 300 moves by the movement amount Dc. As a result, the image portion displayed in the upper display area A is perceived by the driver 300 as if it is displayed at the distance Lc.

As described above, the virtual image G is projected while controlling the movement amounts Db and Dc of the visual recognition positions of the image portions displayed in the upper display area A and the middle display area B according to the driver head movement amount Dd. As a result, the driver 300 determines that the image portion of the middle display area B (the driving lane instruction) is located at a position farther than the image portions of the lower display area C (road name display image 701, speed limit display image 702, overtaking prohibition display image 703, etc.). An image 711, an inter-vehicle distance presentation image 712, etc. are displayed, and an image portion of the upper display area A (route change operation instruction images 721, 722, 723, etc.) is displayed at a position farther than the image portion of the middle display area B. Perceive as In this way, the image portion on the virtual image G displayed at the same distance can be perceived by the driver 300 as if they are displayed at different distances, so that the virtual image G can have a sense of depth.

In particular, in the present embodiment, the image portion of the middle display area B is divided into a plurality of parts in the vertical direction, and the movement amount is changed for each division according to the driver's head movement amount Dd. It is perceived that the image portion located at is displayed farther away. As a result, the inter-vehicle distance presentation image 712 and the traveling lane instruction image 711 displayed in the middle display area B are not only represented by the perspective method but also the motion parallax is used.

FIG. 9 is an explanatory diagram illustrating an image example of a situation where the course is changed at the latest intersection.
When the situation is such that the course is changed at the latest intersection, the image example shown in FIG. 1 is switched to the image example shown in FIG. That is, instead of the traveling lane instruction image 711 displayed in the middle display area B, a route designation image 717 similar to the route designation image 721 displayed in the upper display area A in the image example shown in FIG. Display in the display area B. This is because the operation instruction content after the latest operation content has become the latest operation instruction content. Thereby, the driver 300 can recognize that the driver should operate according to the operation instruction content of the route designation image 721 displayed in the middle display area B at the next intersection or branch point.

Further, in the image example shown in FIG. 9, the driver 300 is displayed in the route designation image 717 displayed in the middle display region B at the position of the route designation image 721 displayed in the upper display region A in the image example shown in FIG. The downward mark image 725 for guiding the line of sight of is displayed. This facilitates the driver 300 to recognize the route designation image 717 displayed in the middle display area B.

Next, display control of the inter-vehicle distance presentation image 712 and the preceding vehicle image 724 for notifying the driver of front vehicle approach information indicating that the inter-vehicle distance to the preceding vehicle 350 is narrowed will be described.
In the present embodiment, the inter-vehicle distance from the preceding vehicle 350 is recognized by the distance detection means such as the object recognition device 100 and the sensor device 500. Then, the image control device 250 receives the recognition result data (forward distance information), and based on the recognition result data, the preceding vehicle image 724 displayed in the upper display area B and the inter-vehicle distance displayed in the middle display area B. With respect to the distance presentation image 712, image display control is executed to change the display position, brightness, color, image shape, etc. in the display area 700. In the present embodiment, the change in the inter-vehicle distance presentation image 712 and the preceding vehicle image 724 informs the driver 300 who visually recognizes the inter-vehicle distance presentation image 712 and the preceding vehicle 350 how narrow the inter-vehicle distance is.

[Image example 1]
FIGS. 10A to 10E show image examples (hereinafter, this image example is referred to as “image example 1”) in which the image indicating the front vehicle approach information changes in accordance with the inter-vehicle distance from the preceding vehicle 350. FIG.
In this image example 1, the destination is not set in the vehicle navigation device 400, and the operation instruction image related to the route navigation information is not displayed. However, even when the operation instruction image related to the route navigation information is displayed. , Is the same.

In the present image example 1, when the preceding vehicle 350 is not recognized, the image control device 250 has a shorter length and a lower brightness in the middle display area B as it goes upward, as shown in FIG. An inter-vehicle distance presentation image 712, which is composed of five horizontal lines and has a low luminance as a whole, is displayed. At this time, nothing is displayed in the upper display area A.

Further, when the preceding vehicle 350 is recognized and the inter-vehicle distance to the preceding vehicle 350 is so vacant as to exceed the predetermined safety distance range, the image control device 250 causes the image control device 250 to move to the middle stage as shown in FIG. Display control is performed to increase the brightness of the inter-vehicle distance presentation image 712 displayed in the display area B. Further, in the present image example 1, instead of the horizontal line image at the bottom of the inter-vehicle distance presentation image 712, the thickness of the line is made thicker, and both ends of the vehicle bumper image 712a are extended downward toward the outside. Display control is performed to change the shape. This own vehicle bumper image reminds the driver 300 of the front bumper of the own vehicle 301.

Furthermore, in the present image example 1, as shown in FIG. 10B, the preceding vehicle image 724 with a low brightness simulating the vehicle rear surface is displayed in the upper display area A. The preceding vehicle image 724 is display-controlled so that it is perceived by the driver 300 as being displayed at the same perceived distance (Lc) as the other images displayed in the upper display area A due to the driving parallax. It

When the preceding vehicle 350 is recognized and the inter-vehicle distance from the preceding vehicle 350 is within the predetermined safety distance range, the image control device 250 displays the middle display area B as shown in FIG. 10C. Display control is performed to increase the brightness of the displayed inter-vehicle distance presentation image 712 and the brightness of the preceding vehicle image 724 displayed in the upper display area A as compared with the case of FIG. 10B. Further, in the present image example 1, display control for changing the second horizontal line image from the bottom to the own vehicle bumper image 712a is also executed. As a result, the own vehicle bumper image 712a is closer to the preceding vehicle image 724 than in the case of FIG. 10(b), and the driver's own vehicle 301 is presented to the driver 300 more than the preceding vehicle image of FIG. 10(b). It is possible to recognize that the user has approached 350.

Further, in the present image example 1, the perceived distance Le1 (Lb<Le1<Lc) of the preceding vehicle image 724 shown in FIG. 10C is closer than the other images displayed in the upper display area A due to the driving parallax. The display is controlled so that the driver 300 perceives it as shown in FIG. As a result, the driver perceives that the preceding vehicle image 724 is closer than in the case of FIG. 10B, and thus the driver's vehicle 301 is ahead of the driver 300 as compared with the case of FIG. 10B. It is possible to further strongly recognize that the vehicle 350 is approaching.

Further, when the preceding vehicle 350 is recognized and the inter-vehicle distance to the preceding vehicle 350 is closer than the predetermined safety distance range, but is longer than the predetermined required braking distance, the image control device 250, the image control device 250, FIG. 10C, the brightness of the inter-vehicle distance presentation image 712 displayed in the middle display area B and the brightness of the preceding vehicle image 724 displayed in the upper display area A are shown in FIG. The display control is performed to be higher than that. Further, in the present image example 1, display control for changing the third horizontal line image from the bottom to the own vehicle bumper image 712a is also performed. As a result, the own vehicle bumper image 712a is closer to the preceding vehicle image 724 than in the case of FIG. 10C, and the driver's vehicle 301 is ahead of the case of FIG. 10C with respect to the driver 300. It can be recognized that the vehicle 350 has come closer.

Further, in the present image example 1, the perceived distance Le2 (Lb<Le2<Le1) of the preceding vehicle image 724 shown in FIG. 10D is further closer than the other images displayed in the upper display area A due to the driving parallax. The display is controlled so that the driver 300 perceives it as displayed in (). As a result, the driver perceives that the preceding vehicle image 724 is closer than in the case of FIG. 10C, and therefore the driver's vehicle 301 is ahead of the driver 300 of FIG. 10C. It is possible to further strongly recognize that the vehicle 350 is getting closer.

In addition, when the preceding vehicle 350 is recognized and the inter-vehicle distance from the preceding vehicle 350 is narrowed to within the predetermined braking distance range, the image control device 250 causes the middle display area B to be displayed as shown in FIG. Display control is performed to change the inter-vehicle distance presentation image 712 displayed therein to a brake warning image 714 in which a character image of "brake!" is drawn in a substantially red trapezoidal shape. At this time, by reducing the brightness of the preceding vehicle image 724 displayed in the upper display area A or stopping the image filling to reduce the visibility of the preceding vehicle image 724, the brake warning image 714 becomes more visible. It is preferable to make it conspicuous. The leading vehicle image 724 in the upper display area A may be hidden so that the brake warning image is conspicuous. At this time, if the laser scanning method is used, the light in the non-image portion can be completely blocked, so that the brake warning image can be made more noticeable and the warning effect can be enhanced.

The predetermined safety distance and the predetermined required braking distance described above may be fixed distances determined in advance, or may be distances that vary according to the vehicle speed of the host vehicle 301 or the like.

[Image example 2]
The expression method of notifying the driver 300 that the inter-vehicle distance from the preceding vehicle 350 has become narrow is not limited to the image example 1 described above, and, for example, image examples as shown in FIGS. 11A and 11B. (Hereinafter, this image example will be referred to as “image example 2”).

Similar to the image example 1 shown in FIGS. 10A to 10E, the present image example 2 is also an image example in which the image indicating the front vehicle approach information changes according to the inter-vehicle distance from the preceding vehicle 350. More specifically, the inter-vehicle distance presentation image 715 of the present image example 2 is a trapezoidal image drawn by the perspective drawing so as to face the vanishing point, instead of the five horizontal lines of the inter-vehicle distance presentation image 712 of the above-mentioned image example 1. It is divided into four parts in the vertical direction. Then, in the image example 1 described above, similarly to the case where the host vehicle bumper image 712a moves upward so as to approach the preceding vehicle image 724 as the distance from the preceding vehicle 350 decreases, in the present image example 2, The trapezoidal section 715a having high brightness moves upward so as to approach the preceding vehicle image 724 as the distance becomes smaller. The image in FIG. 11A corresponds to the image in FIG. 10D, the preceding vehicle 350 is recognized, and the inter-vehicle distance to the preceding vehicle 350 is closer than the predetermined safety distance range. Is an example of a case where the distance is longer than a predetermined required braking distance.

Here, in the present image example 2, the destination is set in the vehicle navigation device 400, and the operation instruction image related to the route navigation information is displayed. Therefore, in the situation where the preceding vehicle 350 is not recognized, as in the example shown in FIG. 1, the route designation image 721 is displayed near the center in the left-right direction of the upper display area A (near the vanishing point), and remains on both sides thereof. A distance image 722 and an intersection name image 723 are displayed respectively.

In this situation, when the preceding vehicle 350 is recognized and the preceding vehicle image 724 is displayed in the upper display area A, all the route changing operation instruction images 721, 722, 723 are hidden from the upper display area A. Can also be considered. However, in this case, the operation instruction content (the operation instruction content later in the operation sequence) by the route change operation instruction images 721, 722, and 723 is not provided to the driver 300, and the driver 300 may be anxious.

On the other hand, when moving the course change operation instruction images 721, 722, 723 displayed in the upper display area A to a display area different from the upper display area A and continuing the display, the driver is It may take a long time to search for the route change operation instruction images 721, 722, 723, and the operation instruction contents by the route change operation instruction images 721, 722, 723 may not be provided to the driver 300 accurately.

According to this image example 2, even when the preceding vehicle image 724 is displayed in the upper display area A in which the route change operation instruction images 721, 722, 723 are displayed, the route change operation instruction images 721, 722, 723 are displayed. The display is continued in the upper display area A. Therefore, even when the preceding vehicle image 724 is displayed in the upper display area A, the operation instruction content by the route change operation instruction images 721, 722, 723 can be accurately provided to the driver 300.

When the preceding vehicle image 724 is displayed in the upper display area A where the route change operation instruction images 721, 722, 723 are displayed, all of the route change operation instruction images 721, 722, 723 are displayed in the upper display area A. Need not be displayed continuously, and for example, only the name image 723 such as the intersection may be hidden.

Further, in the present image example 2, when the inter-vehicle distance from the preceding vehicle 350 approaches within a predetermined braking distance range, the image control device 250 causes the upper display area A and the middle display as shown in FIG. 11B. A brake warning image 716 that straddles the display area B is displayed. This brake warning image 716 is a combination of a red image obtained by enlarging the preceding vehicle image 724 displayed in the upper display area A and an image in which all the trapezoidal sections of the inter-vehicle distance presentation image 715 of this image example 2 are displayed in red. It is a thing. Moreover, in this image example 2, in order to make the brake warning image 716 more noticeable, all of the course changing operation instruction images 721, 722, 723 are hidden.

[Image example 3]
As an expression method for notifying the driver 300 that the inter-vehicle distance from the preceding vehicle 350 has become narrower, for example, an image example as shown in FIGS. Example 3”).
Similar to the image example 1 and the image example 2 described above, the present image example 3 is also an image example in which the image indicating the front vehicle approach information changes according to the inter-vehicle distance from the preceding vehicle 350. Specifically, as the image showing the front vehicle approach information in the third image example, a simple figure (circle in the third image example 3) is used instead of the preceding vehicle image 724 simulating the vehicle rear surface, the inter-vehicle distance presentation image 712, and the like. The constructed simple graphic image 713 is used. Then, in the present image example 3, display control is performed such that the perceived distance from the simple graphic image 713 becomes narrower as the distance to the preceding vehicle 350 becomes shorter.

Specifically, as the distance from the preceding vehicle 350 is narrowed, as shown in FIGS. 12A to 12C, the display control is performed such that the simple graphic image 713 is enlarged, or the brightness of the simple graphic image 713 is increased. The display is controlled to increase. Particularly, in the third image example, the display control is performed such that the perceived distance due to the motion parallax of the simple graphic image 713 becomes shorter as the distance to the preceding vehicle 350 becomes shorter.

That is, the virtual image G including the simple graphic image 713 is displayed at a position 5 m away from the driver 300, but in the case of FIG. 12A where the distance from the preceding vehicle 350 is large, the virtual image G is displayed. The display control is performed so that the perceptual distance Lc of the area A is the same as or close to the perceptual distance Lc (for example, 10 m). Specifically, the image control device 250 simply displays the virtual image G in the middle display area B of the display area 700 according to the driver's head movement amount Dd calculated from the captured image data of the driver camera 150. By changing the position of the graphic image 713 in the display area 700, display control is performed so that the perceived distance of the simple graphic image 713 becomes about 10 m described above.

Further, in the case of FIG. 12B in which the distance to the preceding vehicle 350 is closer than in the case of FIG. 12A, the perceptual distance (for example, 7 m) is closer than the perceptual distance in the case of FIG. 12A. Display control is performed. Specifically, the image control device 250 simply displays the virtual image G in the middle display area B of the display area 700 according to the driver's head movement amount Dd calculated from the captured image data of the driver camera 150. By changing the position of the graphic image 713 in the display area 700, display control is performed so that the perceived distance of the simple graphic image 713 becomes about 7 m described above.

Similarly, in the case of FIG. 12C in which the distance from the preceding vehicle 350 is closer than in the case of FIG. 12B, the perceptual distance (for example, 4 m) closer than the perceptual distance in the case of FIG. The display control is performed so that Specifically, the image control device 250 simply displays the virtual image G in the middle display area B of the display area 700 according to the driver's head movement amount Dd calculated from the captured image data of the driver camera 150. By changing the position of the graphic image 713 in the display area 700, display control is performed so that the perceived distance of the simple graphic image 713 becomes about 4 m described above. The perceptual distance (4 m) is shorter than the virtual image G distance (5 m), but this can also be realized by the display control described above.

[Image example 4]
Next, another image example (hereinafter, this image example is referred to as “image example 4”) showing a situation where the course is changed at the nearest intersection will be described.
When it becomes a situation of changing the course at the latest intersection, the above-described embodiment uses the image example shown in FIG. 9 to provide the driver 300 with information about which intersection to change the course to. In this image example 4, the image examples shown in FIGS. 13A to 13C are used.

In the present image example 4, the route designation image 717 is displayed in the display area 700 so as to overlap the actual intersection visually recognized by the driver through the windshield 302. In the image example shown in FIG. 9, the remaining distance information as passive information indicating the remaining distance to the intersection or the like where the route change operation is performed is provided to the driver 300 by the remaining distance image 722 indicating the remaining distance as a numerical image. doing. On the other hand, in the present image example 4, by changing the perceptual distance of the route designation image 717 in accordance with the remaining distance information, the perceived distance of the route designation image 717 indicates the remaining distance to the intersection or the like at which the route change operation is performed. The information is provided to the driver 300.

Specifically, the image control device 250, according to the driver's head movement amount Dd calculated from the captured image data of the driver camera 150, the path displayed in the middle display area B of the display area 700 of the virtual image G. By changing the position of the designated image 717 in the display area 700, display control is performed so that the perceived distance of the simple graphic image 713 matches the distance to the actual intersection. The actual distance to the intersection can be obtained from the vehicle navigation device 400, for example.

It is not always necessary to match the perceived distance of the simple graphic image 713 with the distance to the actual intersection, but it is possible to reduce the perceived distance of the simple graphic image 713 by gradually reducing the distance to the actual intersection. It is sufficient if the driver can recognize it.

[Image example 5]
Further, among the information regarding the prohibition, restriction, and designation of traffic on the road on which the vehicle 301 is traveling, information that is particularly preferable to be provided to the driver 300 is displayed in the middle display area B as a warning image. You may For example, when the road on which the vehicle 301 is traveling is a school road, as in the image example shown in FIG. 14 (hereinafter, this image example is referred to as “image example 5”), the school road indicating that fact is shown. The warning image 718 is displayed in the middle display area B. Specifically, the image control device 250 acquires unique information of the road on which the vehicle 301 is traveling from various information output from the vehicle navigation device 400, and the acquired unique road information is displayed in the middle display area B. If it is the target information to be displayed on the display, the display control is performed to display the warning image corresponding to the road specific information in the middle display area B.

As described above, the middle display area B is an area in which the operation instruction image indicating the operation instruction content to be operated first among the plurality of operation instruction content is displayed. Therefore, by displaying a warning image in the middle display area B as in the present image example 5, the driver prohibits, limits, or specifies traffic on the road on which the host vehicle 301 is currently traveling according to the warning image. You can intuitively recognize that

[Image example 6]
In addition, when a person trying to cross a pedestrian crossing in the traveling direction of the vehicle 301 is detected, the warning image may be displayed in the middle display area B. This is because human beings are one of the most important targets for vehicles and must be dealt with most recently. For example, like the image example shown in FIG. 15 (hereinafter, this image example is referred to as “image example 6”), the humanoid warning image 719 is displayed in the middle display area B. Specifically, the image control device 250 recognizes a person in front of the own vehicle by the object recognition device 100 and the sensor device 500. Then, the image control device 250 receives the recognition result data and performs display control for displaying the humanoid warning image 719 in the middle display area B based on the recognition result data. In this image example 6, the school road warning image 718 in the above-described image example 5 is also displayed. When the information providing device according to the present embodiment is mounted on a ship other than the vehicle, for example, it is possible to display other ships around the own ship instead of humans.

What has been described above is an example, and the following unique effects can be obtained.
(Aspect A)
A driver-provided information image showing driver-provided information provided to the driver 300 of the moving body such as the own vehicle 301 is visually recognized by the driver through the light transmitting member such as the windshield 302. The display device 700 is an information providing device such as a vehicle HUD device 200 provided with an image light projecting unit such as a HUD main body 230 that projects image light onto the light transmitting member so as to be displayed in the display area 700. Distance detection means such as the object recognition device 100 that detects the relative distance of the moving object traveling direction of the detection target such as the preceding vehicle 350 that exists in the surroundings, and the viewpoint position of the driver camera 150 that detects the viewpoint position of the driver. By changing the display position of the object image such as the preceding vehicle image 724 showing the object to be detected according to the detection means and the detection result of the viewpoint position detection means, the driver based on the motion parallax of the object image. Display control means such as an image control device 250 for controlling the image light projection means so that the perceived distance of the object changes according to the moving object traveling direction relative distance of the detection target detected by the distance detection means. It is characterized by
It is useful to provide the driver with information about the relative movement of an object existing around the moving body with respect to the own vehicle (surrounding object information) by the image projected by the image light projection means. For example, when it is difficult for the driver to directly visually recognize an object around the moving body due to poor visibility around the moving body or the like, an object display image corresponding to the object displayed by using the HUD device allows such surroundings to be detected. If the object information can be provided to the driver, it is possible to drive more safely. Not only when it is difficult for the driver to directly see the vehicle, but if it is possible to provide the driver with surrounding object information regarding the movement of an object that is easy to overlook due to lack of attention of the driver, it is possible to drive more safely. is there. Moreover, when the surrounding object information is to be provided to the driver in this way, the driver can perceive that the depth position of the object display image changes according to the relative movement of the object with respect to the moving body. , It is effective in making it easier for the driver to recognize surrounding object information. Then, as a method of making the driver perceive, it is effective to use motion parallax.
According to this aspect, the predetermined display area in front of the moving body traveling direction in accordance with the relative distance (moving body traveling direction relative distance) to the moving body in the moving body traveling direction of the detection target existing around the moving body. It is possible to change the perceived distance of the object image displayed on the screen. In this aspect, since the driver perceives the perceived distance of the object image by the motion parallax, it is easy for the driver to recognize the information (surrounding object information) of the movement of the detected object approaching or moving away from the moving body. ..

(Aspect B)
In the aspect A, the detection target is another moving body such as a preceding vehicle 350 traveling in front of the moving body.
According to this, it becomes easy for the driver 300 to recognize the information that the distance to another preceding moving body is narrowed (front vehicle approach information or the like).

(Aspect C)
A driver-provided information image showing driver-provided information provided to the driver 300 of the moving body such as the own vehicle 301 is visually recognized by the driver through the light transmitting member such as the windshield 302. An information providing device such as an automobile HUD device 200 having an image light projecting means such as a HUD main body 230 for projecting image light onto the light transmitting member so as to be displayed in the display area 700 of FIG. The area is active information such as a road name display image 701, a speed limit display image 702, an overtaking prohibition display image 703, a vehicle speed display image 704, and the like, which show active information such as road-specific information and vehicle speed information that are actively recognized by the driver. An active information image display area such as a lower display area C where images are continuously displayed, a driving lane instruction image 711 indicating passive information such as route navigation information and front vehicle approach information passively recognized by a driver, and a course change operation. Passive information such as the upper display area A and the middle display area B for temporarily displaying the passive information images such as the instruction images 721, 722, 723, the inter-vehicle distance presentation image 712, and the preceding vehicle image 724 at the timing satisfying the predetermined information providing condition. An image display area is provided, which is displayed in the passive information image display area according to the detection result of the viewpoint position detection means such as the driver camera 150 for detecting the viewpoint position of the driver and the viewpoint position detection means. By changing the display position of the passive information image, the driver's perception distance of the active information image displayed in the active information image display area by the driver's perceptual distances Lb and Lc of the passive information image due to the motion parallax. The display control means such as an image control device 250 for controlling the image light projection means is provided so as to be farther than the distance La.
A driver who is driving usually looks at an infinite point in front of the moving object or another moving object several tens of meters ahead through the light transmitting member. For a driver who is driving while focusing on such a distant place, the farther the image is displayed, the smaller the difference in the focal lengths with respect to the gazing point, and thus the image is easier to recognize (easier to notice).
Here, the passive information that is passively recognized by the driver is generally the information that the driver wants to provide at a timing determined by the information providing device, and normally, the passive information and the passive information are provided. It is information that has a certain relationship with the content of information. Therefore, it is important that the driver 300 recognizes the passive information image indicating the passive information immediately after displaying the passive information image.
On the other hand, the active information actively recognized by the driver is generally sufficient information if it is provided to the driver at the timing desired by the driver, and the timing at which the active information is provided and the active information There is little or no relationship with the content. Since the active information needs to be provided at the timing desired by the driver, the active information continues to be displayed for a long period of time or at all times.
In this aspect, the driver perceives that the passive information image is displayed farther than the active information image due to the motion parallax. Therefore, the passive information image is easier to recognize than the active information image for a driver who is driving while focusing on a distant place. Therefore, when the passive information image is displayed, the driver easily notices the passive information image even if the active information image is displayed. Therefore, the driver 300 can easily recognize the passive information image as soon as it is displayed, and can quickly provide the driver with the passive information.
It should be noted that it is difficult for a driver who is driving to focus on a distant place to recognize an active information image that is perceived to be displayed at a closer distance than a passive information image. However, since the active information image is visually recognized by the driver 300 with the intention of acquiring the active information, even if the active information image is harder to recognize than the passive information image, the active information image is provided. Does not matter.
Rather, since the active information image is continuously displayed for a long period of time or at all times as described above, if the active information image is easy to recognize, it may cause the driver to lose consciousness. However, it may bring disadvantages.

(Aspect D)
In the aspect C, the passive information includes a plurality of operation instruction information indicating different operation instruction contents to the driver of the moving body, and the display control unit includes the travel lane instruction image 711 corresponding to each operation instruction information. And a plurality of operation instruction images such as the route change operation instruction images 721, 722, 723 are displayed on the upper side in the passive information image display area as the operation instruction contents that are later in the operation order for the driver to operate. Thus, the image light projection means is controlled.
The driver generally thinks that an object located in front of a moving moving body approaches the current position over time, and that future events will approach the current time over time. It also has the idea of time. Then, for an object existing in front of the moving body, the shorter the distance to the moving body is, the faster it takes to reach the position of the moving body, and the farther the distance to the moving body is to reach the position of the moving body. Time is late. Therefore, an object located far from the moving body has a high cognitive affinity with a far future event, and an object located near the moving body has a cognitive affinity with a near future event. Is high. According to this aspect, in consideration of this affinity, the operation instruction image indicating the operation instruction content that the operation order to be operated by the driver later is displayed on the upper side in the predetermined display area. It is easy to make the driver intuitively recognize the operation order of the plurality of operation instruction contents indicated by the instruction image and avoid the situation where the driver is confused about the operation order.

(Aspect E)
In the aspect D, the image light projecting unit is configured to transmit the light so that the passive information image display area is located below a landscape in front of a moving body traveling direction visually recognized by a driver via the light transmitting member. The image light is projected onto the member.
The driver 300 who visually recognizes the front scenery in the traveling direction of the moving body through the light transmitting member usually operates while gazing at the vicinity of the vertical center of the front scenery as the gazing point. In this aspect, since the predetermined display area 700 in which a plurality of operation instruction images is displayed is located in the lower part of the front landscape, the plurality of operation instruction images are displayed vertically below the gazing point of the driver. Will be done. In the landscape portion where the plurality of operation instruction images displayed at this position overlap, an object at a point farther from the moving body is visually recognized on the upper side in the vertical direction, and an object at a point closer to the moving body is visually recognized on the lower side in the vertical direction. Therefore, the position where the object in the landscape portion is visually recognized is gradually displaced downward as the moving body approaches the moving body.
When a plurality of operation instruction images are overlapped and displayed on such a landscape portion, as in the present aspect, the operation instruction image indicating the operation instruction content to be operated later by the driver is in the predetermined display area. Displaying on the upper side can minimize the driver's hesitation about the operation sequence. This is an operation instruction image showing the operation instruction content in the far future (operation instruction content in which the operation order is later) so as to overlap with the position where an object far from the moving body is visually recognized in consideration of the cognitive affinity described above. Is displayed, and an operation instruction image indicating the operation instruction content in the near future (operation instruction content that comes first in the operation order) so as to overlap the position where an object close to the moving body is visually recognized can be displayed. is there. With such a display, the driver can intuitively recognize the operation sequence of the plurality of operation instruction contents indicated by these operation instruction images. On the contrary, when the predetermined display area 700 is located in the lower part of the front landscape, for example, the display positions of the plurality of operation instruction images are reversed from the above-described positions, or the plurality of operation instruction images are displayed. If the positions are arranged side by side, it becomes difficult for the driver to intuitively recognize the operation sequence because the idea described above interferes.

(Aspect F)
In the aspect D or E, the plurality of operation instruction images are displayed by the driver at different points on the moving route of the moving body (the nearest intersection and an intersection or branch point where the course should be changed next from the straight traveling direction). It is characterized in that it indicates the operation instruction content for instructing the operation content to be operated.
According to this, it is possible to avoid a situation in which the driver is confused about the operation order of a plurality of operation instruction contents having different operation orders according to the route navigation information.

(Aspect G)
In any one of the aspects A to F, the image light projecting means causes the operation instruction image to be displayed as a virtual image G in the predetermined display area by the image light to be projected, and the virtual image from the driver. Is characterized by a distance of 5 m or more.
When the distance to the virtual image G is about 2 m, which is a general distance, in order to focus the eyeball on the virtual image G having such a short distance, a vergence movement of the eyeball is usually required. As described above, the vergence movement is a factor that greatly affects the sense of distance to the visual target and the sense of depth, and when the eye converges to focus on the virtual image G, the sense of distance (perception) due to motion parallax. The effect of perceiving a change in distance) and a sense of depth (difference in perceived distance) is diminished.
According to this aspect, since the distance to the virtual image G is 5 m or more, the virtual image G can be focused with almost no vergence movement of the eyeball. Therefore, the effect of perceiving a sense of distance (change in perceived distance) and a sense of depth (difference in perceived distance) using motion parallax is suppressed from being diluted by the vergence movement of the eyeball.

(Aspect H)
In any one of the aspects A to G, the image light projection unit optically scans the image light emitted from the light irradiation unit such as the light source unit 220 that emits the image light according to the image information of the operation instruction image. It is characterized in that the driver-provided information image is displayed in the predetermined display area by two-dimensionally scanning by an optical scanning means such as the device 208 and projecting it on the light transmitting member.
As described above, it is easier to display a large virtual image G with higher brightness than a method using a liquid crystal display (LCD) or a fluorescent display tube (VFD). According to this aspect, the non-image portion of the virtual image G can be completely eliminated by not irradiating the non-image portion with the image light. Therefore, it is possible to avoid the situation where the visibility of the front scene of the moving body through the non-image portion is deteriorated by the light emitted from the light irradiating means, and the visibility of the front scene is high.

(Aspect I)
The driver-provided information image showing the driver-provided information provided to the driver of the moving body is displayed through a light transmitting member in a predetermined display area in front of the moving body traveling direction visually recognized by the driver. An information providing method for providing the driver with the driver-provided information by projecting image light onto a transparent member, wherein a moving body traveling direction relative distance of a detection target object existing around the moving body is detected. A distance detection step, a viewpoint position detection step of detecting the viewpoint position of the driver, and by changing the display position of the object image showing the detection object according to the detection result in the viewpoint position detection step, And a display step of changing a driver's perceived distance based on the motion parallax of the object image according to the relative distance of the moving object traveling direction of the detected object detected in the distance detecting step.
According to this aspect, since the driver perceives the perceived distance of the object image by the motion parallax, the driver recognizes information about the movement of the detected object approaching or moving away from the moving body (surrounding object information). Easy to make.

(Aspect J)
The driver-provided information image showing the driver-provided information provided to the driver of the moving body is displayed through a light transmitting member in a predetermined display area in front of the moving body traveling direction visually recognized by the driver. Image light projecting means for projecting image light to the transparent member, distance detecting means for detecting the relative distance of the moving object in the moving body traveling direction around the moving body, viewpoint position for detecting the viewpoint position of the driver An information providing control program for causing a computer of an information providing apparatus including a detection unit to function, wherein a display position of an object image showing the detection object is changed according to a detection result of the viewpoint position detection unit. By doing so, the image light projection unit changes the perceived distance of the driver due to the motion parallax of the target image in accordance with the moving object traveling direction relative distance of the detected target detected by the distance detection unit. The computer is made to function as display control means for controlling the.
According to this aspect, since the driver perceives the perceived distance of the object image by the motion parallax, the driver recognizes information about the movement of the detected object approaching or moving away from the moving body (surrounding object information). Easy to make.

(Aspect K)
The driver-provided information image showing the driver-provided information provided to the driver of the moving body is displayed through a light transmitting member in a predetermined display area in front of the moving body traveling direction visually recognized by the driver. An information providing method for providing the driver with the driver-provided information by projecting image light onto a transparent member, wherein the predetermined display area is active indicating active information that is actively recognized by the driver. An active information image display area for continuously displaying an information image and a passive information image display area for temporarily displaying a passive information image indicating passive information passively recognized by the driver at a timing satisfying a predetermined information providing condition. And a viewpoint position detecting step of detecting the viewpoint position of the driver, and a display position of the passive information image displayed in the passive information image display area is changed according to a detection result in the viewpoint position detecting step. Thereby, the image light projection means is configured such that the driver's perceived distance of the passive information image due to the motion parallax is longer than the driver's perceived distance of the active information image displayed in the active information image display area. And a display control step for controlling.
According to this aspect, since the passive information image is easier to recognize than the active information image for the driver who is driving while focusing on a distant place, the driver can be active when the passive information image is displayed. Even if the information image is displayed, it is easy to notice the passive information image. Therefore, the driver 300 can easily recognize the passive information image as soon as it is displayed, and can quickly provide the driver with the passive information.

(Aspect L)
The driver-provided information image showing the driver-provided information provided to the driver of the moving body is displayed through a light transmitting member in a predetermined display area in front of the moving body traveling direction visually recognized by the driver. An information providing control program for operating a computer of an information providing apparatus, comprising: an image light projecting means for projecting image light onto a transmitting member; and a viewpoint position detecting means for detecting a viewpoint position of the driver, The predetermined display area is an active information image display area for continuously displaying an active information image showing active information actively recognized by the driver, and a passive information image showing passive information passively recognized by the driver. Is provided with a passive information image display area for temporarily displaying at a timing satisfying a predetermined information providing condition, and the passive information image displayed in the passive information image display area according to the detection result of the viewpoint position detecting means. By changing the display position, the perceived distance of the driver for the passive information image due to the motion parallax becomes longer than the perceived distance of the driver for the active information image displayed in the active information image display area, The computer is made to function as a display control unit that controls the image light projection unit.
According to this aspect, since the passive information image is easier to recognize than the active information image for the driver who is driving while focusing on a distant place, the driver can be active when the passive information image is displayed. Even if the information image is displayed, it is easy to notice the passive information image. Therefore, the driver 300 can easily recognize the passive information image as soon as it is displayed, and can quickly provide the driver with the passive information.

The above-mentioned program can be distributed or obtained while being recorded in a recording medium such as a CD-ROM. Further, by distributing the above-mentioned program and distributing or receiving a signal transmitted by a predetermined transmission device via a transmission medium such as a public telephone line, a leased line, or another communication network, distribution, It is available. At the time of this distribution, at least a part of the computer program may be transmitted in the transmission medium. That is, it is not necessary that all the data making up the computer program exist on the transmission medium at one time. The signal carrying the above-mentioned program is a computer data signal embodied on a predetermined carrier wave containing the computer program. Further, the transmission method of transmitting the computer program from the predetermined transmission device includes a case of continuously transmitting the data making up the program and a case of transmitting the data intermittently.

100 Object Recognition Device 110 Stereo Camera Unit 120 Information Processing Unit 150 Driver Camera 200 Automotive HUD Devices 201R, 201G, 201B Laser Light Source 207 Light Amount Adjusting Unit 208 Optical Scanning Device 209 Free-form Mirror 210 Microlens Array 211 Projection Mirror 220 Light Source Unit 230 HUD main body 250 Image control device 300 Driver 301 Own vehicle 302 Windshield 350 Preceding vehicle 400 Vehicle navigation device 500 Sensor device 700 Display area 701 Road name display image 702 Speed limit display image 703 Overpass prohibition display image 704 Vehicle speed display image 711 Traveling Lane instruction image 712, 715 Inter-vehicle distance presentation image 713 Simple graphic image 712a Vehicle bumper image 714, 716 Brake warning image 718 School road warning image 719 Humanoid warning image 717, 721 Route designation image 722 Remaining distance image 723 Name image of intersection, etc. 724 Leading vehicle image 725 Downward mark image A Upper display area B Middle display area C Lower display area G Virtual image

Japanese Patent No. 4686586

Claims (10)

A position detection unit that detects a viewpoint position of a driver who drives a moving body,
An active information image that is actively recognized by the driver, and a passive information image that is displayed to be recognized by the driver when predetermined information provision conditions are met are displayed to the driver via a light transmissive member. And a display unit for visually displaying,
In the predetermined display area, the display section changes the display position of the passive information image to a direction orthogonal to the depth direction in the predetermined display area based on the detection information of the position detection section . the perceived distance of the driver of the passive information image to be displayed, farther than perceptual metric of the driver of the active information image displayed in a separate display area which is located below the said predetermined display region an image display device, wherein the table Shimesuru to. The image display device according to 請 Motomeko 1,
An acquisition unit that acquires distance information from an object around the moving body,
An image display device, wherein the passive information image is displayed at a position corresponding to the object. The image display device according to claim 2 ,
The image display device, wherein the display unit displays the passive information image so that the perceived distance of the passive information image becomes short when the relative distance to the target object becomes short. The image display device according to claim 2 or 3,
The image display device, wherein the object is another moving body that travels ahead of the moving body. The image display device according to claim 4 ,
The image display device, wherein the passive information image is an image regarding a distance to the another moving body. The image display device according to any one of claims 1乃Itaru 5,
The image display device, wherein the viewpoint position is a position of the driver's eyes. The image display device according to any one of claims 1乃Itaru 5,
The image display device, wherein the viewpoint position is a head position of the driver. A light transmitting member that allows the driver to visually see the outside of the moving body,
A position detection unit that detects a viewpoint position of a driver who drives the moving body,
The active information image actively recognized by the driver, and the passive information image displayed to be recognized by the driver when a predetermined information provision condition is satisfied are displayed on the driver through the light transmitting member. And a display section for visually displaying the
In the predetermined display area, the display section changes the display position of the passive information image to a direction orthogonal to the depth direction in the predetermined display area based on the detection information of the position detection section . the perceived distance of the driver of the passive information image to be displayed, farther than perceptual metric of the driver of the active information image displayed in a separate display area which is located below the said predetermined display region mobile, wherein the table Shimesuru to. A position detection unit that detects a viewpoint position of a driver who drives a moving body,
An active information image that is actively recognized by the driver, and a passive information image that is displayed to be recognized by the driver when predetermined information provision conditions are met are displayed to the driver via a light transmissive member. An image display method carried out by an image display device comprising a display unit for visually displaying,
Acquiring the driver's viewpoint position information,
By displaying by changing the display position of the passive information image on the basis of the detection information of the position detection unit in a direction orthogonal to the depth direction in a predetermined display area, the displayed in the predetermined display area the perceptual metric of the driver of a passive information images, tables Shimesuru step further than perceptual metric of the driver of the active information image displayed in a separate display area which is located below the said predetermined display region An image display method comprising: A position detection unit that detects a viewpoint position of a driver who drives a moving body,
An active information image that is actively recognized by the driver, and a passive information image that is displayed to be recognized by the driver when predetermined information provision conditions are met are displayed to the driver via a light transmissive member. A program that is executed by a computer of an image display device that includes a display unit that visually displays,
Acquiring the driver's viewpoint position information,
By displaying by changing the display position of the passive information image on the basis of the detection information of the position detection unit in a direction orthogonal to the depth direction in a predetermined display area, the displayed in the predetermined display area the perceptual metric of the driver of a passive information images, tables Shimesuru step further than perceptual metric of the driver of the active information image displayed in a separate display area which is located below the said predetermined display region And a program that causes the computer to execute.