JP7385834B2 - Image display method and image display device - Google Patents

Description

The present invention relates to an image display method and an image display device .

This type of image display device includes a head-up display (HUD) that displays images to provide information to the driver of a moving object such as a vehicle, ship, aircraft, or industrial robot that carries a driver. ) devices are known.

Patent Document 1 discloses a HUD device that projects image light onto a windshield or the like (light-transmitting member) and displays an image superimposed on the scenery ahead that is viewed by the driver of a vehicle (moving object) through the windshield. has been done. This HUD device displays virtual images of arrows indicating the direction of travel, display objects indicating speed, cautions, warnings, etc., superimposed on the scenery in front of the vehicle. This HUD device is equipped with a position detection unit (viewpoint position detection means) that images the driver and detects the position of one eye of the driver, and determines the position of each display object in the virtual image according to the detection result. Perform processing to vary. In this process, when the driver moves his head and the viewpoint position (position of one eye) moves, the amount of movement of each display object in the virtual image is varied, and the depth direction of each display object is changed due to motion parallax. The driver is made to perceive that the display position (subjective depth position) is different.

However, Patent Document 1 does not mention making the driver perceive image parts on a virtual image that are displayed at the same distance as if they were displayed at different distances, thereby giving a sense of depth to the virtual image. .

In order to solve the above-mentioned problems, the present invention provides an image display method, which includes a detection step of detecting the movement of a driver driving a mobile object, and a detection step of detecting the movement of a driver who is driving a mobile object, and detecting the movement of a driver who is away from the driver by a first distance. The driver moves a first image portion that is made to be perceived by the driver and a second image portion that is made to be made to be perceived by the driver as being separated from the driver by a second distance that is greater than the first distance. a display step of moving and displaying at least one of the first image portion and the second image portion according to the movement of the driver detected by the detection step when One includes an image showing a human being, and the displaying step includes moving the second image portion by a greater amount than a first movement amount for moving and displaying the first image portion. The display is characterized in that the second movement amount for displaying is larger.

According to the present invention, an excellent effect can be obtained in that a driver can perceive image parts on a virtual image that are displayed at the same distance as if they were displayed at different distances, thereby giving a sense of depth to the virtual image. It is played.

FIG. 3 is an explanatory diagram illustrating an example of a virtual image displayed in a display area superimposed on the scenery in front of the own vehicle as seen by the driver through the windshield in the embodiment. FIG. 1 is a schematic diagram schematically showing the configuration of an automobile equipped with an automobile HUD device according to an embodiment. FIG. 2 is a schematic diagram schematically showing the internal configuration of the automotive HUD device. FIG. 2 is a hardware block diagram of a control system in the automotive HUD device. FIG. 1 is a block diagram showing a schematic configuration of a driver information providing system in an embodiment. FIG. 2 is an explanatory diagram showing the hardware configuration of an object recognition device in the driver information providing system. FIG. 2 is a hardware block diagram showing main hardware of an image control device in the automotive HUD device. FIG. 3 is an explanatory diagram for explaining an image processing method for a virtual image that gives a sense of depth using motion parallax in an embodiment. It is an explanatory view showing an example of an image of a situation where the driver changes course at the nearest intersection. (a) to (e) are explanatory diagrams showing image example 1 in which an image indicating front vehicle approach information changes depending on the inter-vehicle distance from the preceding vehicle. (a) and (b) are explanatory diagrams showing another image example 2 in which an image indicating front vehicle approach information changes depending on the inter-vehicle distance from the preceding vehicle. (a) to (c) are explanatory diagrams showing still another image example 3 in which an image indicating front vehicle approach information changes depending on the inter-vehicle distance from the preceding vehicle. (a) to (c) are explanatory diagrams showing another image example 4 of a situation where the driver changes course at the nearest intersection. FIG. 12 is an explanatory diagram showing image example 5 in which a school road warning image is displayed in the middle display area when the road on which the host vehicle is traveling is a school road. FIG. 12 is an explanatory diagram showing image example 6 in which a human-shaped warning image is displayed in the middle display area when a person attempting to cross a crosswalk in the traveling direction of the own vehicle is detected.

Hereinafter, an embodiment in which the present invention is applied to a driver information providing system including an automobile head-up display (HUD) device as an information providing device will be described.
FIG. 1 is an explanatory diagram showing an example of a virtual image G displayed in a display area 700 superimposed on the scenery in front of the host vehicle 301 as seen from the driver 300 through the windshield 302.
FIG. 2 is a schematic diagram schematically showing the configuration of an automobile equipped with an automobile HUD device according to the present embodiment.
FIG. 3 is a schematic diagram schematically showing the internal configuration of the automotive HUD device in this embodiment.

The automobile HUD device 200 in this embodiment is installed, for example, in the dashboard of a host vehicle 301 that is a running object as a moving object. Projection light L, which is image light emitted from the automobile HUD device 200 in the dashboard, is reflected by the windshield 302 as a light-transmitting member and is directed toward the driver 300. Thereby, the driver 300 can visually recognize a HUD display image such as a navigation image to be described later as a virtual image. Note that a combiner as a light transmitting member may be installed on the inner wall surface of the windshield 302, and the driver may be able to visually recognize a virtual image by the projection light L reflected by the combiner.

In this embodiment, the optical system and the like of the automobile HUD device 200 are configured so that the distance from the driver 300 to the virtual image G is 5 m or more. In conventional general automotive HUD devices, the distance from the driver 300 to the virtual image G is about 2 meters. The driver 300 usually gazes at an infinite point in front of the vehicle or at a preceding vehicle several tens of meters ahead. When the driver 300 who focuses on such a distant object attempts to visually recognize the virtual image G 2 meters ahead, the lens of the eyeball needs to be moved significantly because the focal lengths are significantly different. Therefore, it takes a long time to adjust the focus until the virtual image G is brought into focus, and it takes time to recognize the contents of the virtual image G, and the driver's 300 eyes easily become fatigued. Furthermore, it is difficult for the driver to notice the contents of the virtual image G, and it is difficult to appropriately provide information to the driver using the virtual image G.

If the distance to the virtual image G is 5 m or more as in this embodiment, the amount of movement of the crystalline lens of the eyeball will be reduced compared to the conventional method, and the time to adjust the focus on the virtual image G will be shortened, allowing for early recognition of the contents of the virtual image G. In addition, the eye fatigue of the driver 300 can be reduced. Furthermore, it becomes easier for the driver to notice the contents of the virtual image G, and it becomes easier to appropriately provide information to the driver using the virtual image G.

Further, when the distance to the virtual image G is about 2 m, convergence movement of the eyeballs is usually required to focus the eyeballs on the virtual image G at such a short distance. Convergence motion is a factor that greatly affects the sense of distance and depth to a visual object. In this embodiment, as will be described later, display control is performed in which the perceived distance of an image displayed as a virtual image G is perceived by motion parallax. In this case, if the eyeballs make a convergence movement to focus on the virtual image G, the effect of motion parallax on perceiving the sense of distance (change in perceived distance) and depth (difference in perceived distance) will be weakened, and the image This reduces the driver's information recognition effect, which will be described later, by utilizing differences and changes in perceived distance.

If the distance to the virtual image G is 5 m or more, it is possible to focus on the virtual image G with almost no convergence movement of the eyeballs. Therefore, the effect of using motion parallax to perceive a sense of distance (change in perceived distance) and a sense of depth (difference in perceived distance) is suppressed from being weakened by the convergence movement of the eyeballs. Therefore, the driver's information perception effect using the sense of distance and depth of the image can be effectively exerted.

The automotive HUD device 200 includes red, green, and blue laser light sources 201R, 201G, and 201B, collimator lenses 202, 203, and 204 provided for each laser light source, and two dichroic mirrors 205 in the HUD main body 230. , 206, a light amount adjusting section 207, a light scanning device 208 as a light scanning means, a free-form mirror 209, a microlens array 210 as a light diverging member, and a projection mirror 211 as a light reflecting member. ing. In the light source unit 220 in this embodiment, laser light sources 201R, 201G, 201B, collimator lenses 202, 203, 204, and dichroic mirrors 205, 206 are unitized by an optical housing.

LDs (semiconductor laser devices) can be used as the laser light sources 201R, 201G, and 201B. The wavelength of the light flux emitted from the red laser light source 201R is, for example, 640 nm, the wavelength of the light flux emitted from the green laser light source 201G is, for example, 530 nm, and the wavelength of the light flux emitted from the blue laser light source 201B is, for example, 445 nm. .

The automotive HUD device 200 of this embodiment projects an intermediate image formed on a microlens array 210 onto a windshield 302 of the host vehicle 301, and displays an enlarged image of the intermediate image to the driver 300 as a virtual image G. Make it visible as. The laser beams of each color emitted from the laser light sources 201R, 201G, and 201B are made into substantially parallel beams by collimator lenses 202, 203, and 204, respectively, and combined by two dichroic mirrors 205 and 206. The light intensity of the combined laser beams is adjusted by a light intensity adjustment unit 207, and then two-dimensionally scanned by a mirror of an optical scanning device 208. The scanning light L' that has been two-dimensionally scanned by the optical scanning device 208 is reflected by the free-form mirror 209 to correct distortion, and then is focused on the microlens array 210 to draw an intermediate image.

Note that in this embodiment, the microlens array 210 is used as a light diverging member that individually diverges and emits the light flux for each pixel of the intermediate image (one point of the intermediate image), but other light diverging members may be used. It's okay. Further, as a method of forming the intermediate image G', a method using a liquid crystal display (LCD) or a fluorescent display tube (VFD) may be used.

However, in order to display a large virtual image G with high brightness, a laser scanning method as in this embodiment is preferable.
In addition, in systems using liquid crystal displays (LCDs), fluorescent display tubes (VFDs), etc., a small amount of light is irradiated to non-image areas within the display area where the virtual image G is displayed, and this cannot be completely blocked. is difficult. Therefore, there is a disadvantage that visibility of the scenery in front of the own vehicle 301 through the non-image portion is poor. On the other hand, according to the laser scanning method as in this embodiment, light is irradiated to the non-image portion within the display area of the virtual image G by turning off the laser light sources 201R, 201G, and 201B. This can be completely blocked. Therefore, it is possible to avoid a situation in which the visibility of the scenery in front of the own vehicle 301 through the non-image portion is reduced by the light emitted from the automobile HUD device 200, and there is an advantage that the visibility of the scenery in front of the vehicle 301 is high.

Furthermore, if the degree of warning is to be strengthened by gradually increasing the brightness of a warning image to warn the driver, only the brightness of the warning image among the various images displayed in the display area 700 may be increased. Display control is required to increase the value in stages. The laser scanning method is also suitable when performing display control such as partially increasing the brightness of some images in the display area 700 in this way. In a method using a liquid crystal display (LCD) or a fluorescent display tube (VFD), the brightness of images other than the warning image displayed within the display area 700 also increases, causing the difference between the warning image and other images. This is because the difference in brightness between the warning images cannot be widened, and the effect of increasing the degree of warning by increasing the brightness of the warning image in stages cannot be sufficiently achieved.

The optical scanning device 208 tilts the mirror in the main scanning direction and the sub-scanning direction using a known actuator drive system such as MEMS (Micro Electro Mechanical Systems), and performs two-dimensional scanning (raster scanning) of the laser light incident on the mirror. . The drive control of the mirror is performed in synchronization with the emission timing of the laser light sources 201R, 201G, and 201B. The optical scanning device 208 is not limited to the configuration of this embodiment, but may be configured with a mirror system including two mirrors that respectively swing or rotate about two mutually orthogonal axes, for example.

FIG. 4 is a hardware block diagram of a control system in the automotive HUD device 200 of this embodiment.
The control system of the automotive HUD device 200 mainly includes an FPGA 251, a CPU 252, a ROM 253, a RAM 254, an I/F 255, a bus line 256, an LD driver 257, and a MEMS controller 258. The FPGA 251 controls the operation of the laser light sources 201R, 201G, and 201B of the light source unit 220 using the LD driver 257, and controls the operation of the MEMS 208a of the optical scanning device 208 using the MEMS controller 258. The CPU 252 controls each function of the automotive HUD device 200. The ROM 253 stores various programs such as an image processing program executed by the CPU 252 to control each function of the automotive HUD device 200. RAM 254 is used as a work area for CPU 252. The I/F 255 is an interface for communicating with an external controller, etc., and for example, communicates with an object recognition device 100, a vehicle navigation device 400, various sensors 500, etc., which will be described later, via a CAN (Controller Area Network) of the host vehicle 301. Connected.

FIG. 5 is a block diagram showing a schematic configuration of the driver information providing system in this embodiment.
In this embodiment, an object recognition device 100, a vehicle navigation device 400, a sensor device 500, and the like are provided as information acquisition means for acquiring driver-provided information provided to the driver using the virtual image G. The automotive HUD device 200 in this embodiment mainly includes a HUD main body 230 as an image light projection means and an image control device 250 as a display control means. Although the information acquisition means in this embodiment is mounted on the own vehicle 301, the information acquired by the information acquisition means is inputted via the communication means using the information acquisition means installed outside the own vehicle 301. It may be a configuration.

FIG. 6 is an explanatory diagram showing the hardware configuration of the object recognition device 100 in this embodiment.
The object recognition device 100 of the present embodiment includes a stereo camera section 110 as an imaging means that images an area in front of the own vehicle 301 as an imaging region, and an object recognition device 100 based on captured image data captured by the stereo camera section 110. The information processing section 120 serves as an image processing means for performing image processing for recognizing a predetermined recognition target object. Note that instead of the stereo camera section 110, a configuration may be adopted in which a monocular camera as an imaging means and a laser radar (millimeter wave radar) as a distance measuring means are combined.

The stereo camera section 110 is constructed by assembling two camera sections in parallel, a first camera section 110A for the left eye and a second camera section 110B for the right eye. Each camera section 110A, 110B includes a lens 115, an image sensor 116, and a sensor controller 117, respectively. As the image sensor 116, an image sensor configured of, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) can be used. The sensor controller 117 performs exposure control of the image sensor 116, image readout control, communication with an external circuit, image data transmission control, and the like. The stereo camera unit 110 is installed near the rearview mirror of the windshield 302 of the own vehicle 301.

The information processing unit 120 includes a data bus line 121, a serial bus line 122, a CPU (Central Processing Unit) 123, an FPGA (Field-Programmable Gate Array) 124, a ROM (Read Only Memory) 125, a RAM (Random Access Memory) 126, It has a serial IF (Interface) 127 and a data IF 128.

The stereo camera section 110 is connected to the information processing section 120 via a data bus line 121 and a serial bus line 122. The CPU 123 executes and controls each sensor controller 117 of the stereo camera section 110, the operation of the entire information processing section 120, image processing, and the like. Luminance image data of captured images captured by the image sensors 116 of each camera unit 110A, 110B is written to the RAM 126 of the information processing unit 120 via the data bus line 121. Sensor exposure value change control data, image readout parameter change control data, various setting data, etc. from the CPU 123 or the FPGA 124 are transmitted and received via the serial bus line 122.

The FPGA 124 generates a parallax image by performing processing that requires real-time performance on the image data stored in the RAM 126, such as gamma correction, distortion correction (parallelization of left and right images), and parallax calculation using block matching. Write again to RAM18. The ROM 125 contains a recognition program for recognizing three-dimensional objects such as vehicles and pedestrians, or predetermined recognition objects such as lane boundaries such as white lines on the road surface, curbs and median strips on the side of the road surface. is memorized. The recognition program is an example of an image processing program.

The CPU 123 acquires CAN information such as vehicle speed, acceleration, steering angle, and yaw rate from the sensor device 500 via the data IF 128, for example, via the CAN of the host vehicle 301. Then, the CPU 123 executes image processing using the luminance image and the parallax image stored in the RAM 126 according to the recognition program stored in the ROM 125, and recognizes objects to be recognized, such as the preceding vehicle 350 and lane boundaries. I do.

The recognition result data of the recognition target object is supplied to external devices such as the image control device 250 and the vehicle travel control unit via the serial IF 127, for example. The vehicle running control unit performs brake control, speed control, steering control, etc. of the host vehicle 301 using the recognition result data of the recognition target, and for example, controls the host vehicle 301 to maintain a preset inter-vehicle distance. This includes cruise control, which automatically follows the vehicle in front, and automatic brake control, which helps avoid and reduce collisions with obstacles ahead.

The vehicle navigation device 400 in this embodiment can widely utilize known vehicle navigation devices installed in automobiles and the like. The vehicle navigation device 400 outputs information necessary to generate a route navigation image to be displayed on the virtual image G, and this information is input to the image control device 250. For example, as shown in FIG. 1, the number of lanes (driving lanes) on the road on which the host vehicle 301 is traveling, the distance to the next point where the course should be changed (right turn, left turn, branch, etc.), and the next course change. Contains images that show information such as directions. By inputting this information from the vehicle navigation device 400 to the image control device 250, the vehicle HUD device 200, under the control of the image control device 250, displays a driving lane instruction image 711, a course designation image 721, and a remaining distance image. Navigation images such as 722 and intersection name image 723 are displayed in the upper display area A and the middle display area B of the display area 700.

Further, in the image example shown in FIG. 1, an image showing road specific information (road name, speed limit, etc.) is displayed in the lower display area C of the display area 700. This road specific information is also input from the vehicle navigation device 400 to the image control device 250. The image control device 250 causes the automobile HUD device 200 to display a road name display image 701, a speed limit display image 702, an overtaking prohibition display image 703, etc. corresponding to the road-specific information in the lower display area C of the display area 700. .

The sensor device 500 in this embodiment includes one or more sensors for detecting various information indicating the behavior of the own vehicle 301, the state of the own vehicle 301, the situation around the own vehicle 301, and the like. The sensor device 500 outputs sensing information necessary to generate an image to be displayed as the virtual image G, and this sensing information is input to the image control device 250. For example, in the image example shown in FIG. 1, a vehicle speed display image 704 (in FIG. 1, a character image "83 km/h") indicating the vehicle speed of the host vehicle 301 is displayed in the lower display area C of the display area 700. Therefore, vehicle speed information included in the CAN information of the own vehicle 301 is input from the sensor device 500 to the image control device 250, and under the control of the image control device 250, the character image indicating the vehicle speed is displayed in the display area by the automobile HUD device 200. 700 is displayed in the lower display area C.

In addition to the sensor that detects the vehicle speed of the host vehicle 301, the sensor device 500 also detects other vehicles, pedestrians, and buildings (guardrails, telephone poles, etc.) around the host vehicle 301 (in front, sides, and rear). A laser radar device or an imaging device that detects the distance from ), a sensor for detecting the remaining amount of fuel in the fuel tank of the host vehicle 301, a sensor for detecting the status of various on-vehicle devices such as an engine and a battery, and the like. By detecting such information with the sensor device 500 and sending it to the image control device 250, the information can be displayed as a virtual image G by the automobile HUD device 200 and provided to the driver 300.

FIG. 7 is a hardware block diagram showing the main hardware of the image control device 250.
In the image control device 250, a CPU 251, a RAM 252, a ROM 253, an input data IF 254, and an output data IF 255 are connected to each other by a data bus line. The input data IF 254 receives various recognition result data output from the object recognition device 100, sensing information output from the sensor device 500, various information output from the vehicle navigation device 400, and the like. The output data IF 255 outputs control signals for the automobile HUD device 200 and the like. The CPU 251 executes various computer programs such as an information provision control program stored in the ROM 253 and the like, and causes the image control device 250 to perform various controls and processes to be described later.

Next, the virtual image G displayed by the automobile HUD device 200 will be explained.
In the automobile HUD device 200 according to the present embodiment, the driver-provided information provided to the driver by the virtual image G may be any information as long as it is useful for the driver. In this embodiment, driver-provided information is roughly divided into passive information and active information.

Passive information is information that is passively recognized by the driver when a predetermined information provision condition is met. Therefore, information provided to the driver at the setting timing of the automotive HUD device 200 is included in passive information, and information that has a certain relationship between the timing at which information is provided and the content of the information is passive information. Included in the information. Examples of passive information include information related to driving safety and route navigation information. Information related to safety during driving includes, for example, information on the distance between the own vehicle 301 and the preceding vehicle 350 (inter-vehicle distance presentation image 712), and information on urgent matters related to driving (instructing the driver to perform emergency operations). Warning information such as emergency operation instruction information, caution information, etc.). Further, the route navigation information is information for guiding a driving route to a preset destination, and is provided to the driver by a known vehicle navigation device. Route navigation information includes driving lane instruction information (driving lane instruction image 711) that instructs the driving lane in which to drive at the nearest intersection, and course change operations at intersections and branch points where the next course should be changed from the straight direction. Examples include route change operation instruction information. Specifically, the route change operation instruction information includes route designation information (route designation image 721) that specifies which route to take at the intersection, etc., and the remaining distance to the intersection, etc. where the route change operation is to be performed. Examples include distance information (remaining distance image 722), name information of the intersection, etc. (intersection name image 723), and the like.

Active information is information that is actively recognized by the driver at a timing determined by the driver. Active information is sufficient information if it is provided to the driver at the timing desired by the driver.For example, information where there is little or no relationship between the timing at which the information is provided and the content of the information is , included in active information. Active information is information that the driver acquires at a timing that the driver desires, and therefore is information that continues to be displayed for a fairly long period of time or all the time. Examples include unique information of the road on which the host vehicle 301 is traveling, vehicle speed information of the host vehicle 301 (vehicle speed display image 704), current time information, and the like. The unique information of a road includes, for example, the road name information (road name display image 701), regulation content information such as the speed limit of the road (speed limit display image 702, overtaking prohibition display image 703), and other information related to the road. The information includes information that is useful to the driver.

In this embodiment, passive information and active information, which are broadly classified in this way, are displayed in corresponding display areas within the display area 700 where the virtual image G can be displayed. Specifically, in this embodiment, the display area 700 is divided into three display areas in the vertical direction, and the upper display area A and the middle display area B display passive information images corresponding to passive information, In the lower display area C, an active information image corresponding to the active information is displayed. Note that a part of the active information image may be displayed in the upper display area A and the middle display area B, but in that case, the visibility of the passive information image displayed in the upper display area A and the middle display area B may be Display the active information image to give priority.

The passive information of this embodiment includes operation instruction information indicating the content of operation instructions to the driver 300, such as route navigation information. When providing such operation instruction information to the driver, it is usually recommended to display not only an operation instruction image showing the most recent operation instruction content, but also an operation instruction image showing the subsequent operation instruction content. This is preferable because it eliminates anxiety (anxiety caused by not knowing what operations will be performed afterwards). In the example shown in FIG. 1 as well, along with the driving lane instruction image 711 that instructs the driving lane in which to travel at the nearest intersection, the direction change operation at the intersection or branch point where the next course should be changed from the straight direction is instructed. Route change operation instruction images 721, 722, and 723 are displayed.

In this way, when displaying a plurality of operation instruction images having different operation orders, that is, the driving lane instruction image 711 and the course change operation instruction images 721, 722, and 723 in the display area 700 at the same time, each operation instruction It is desirable to avoid as much as possible a situation in which the driver becomes confused about the order of operation instructions shown in the image.

Therefore, in the present embodiment, among a plurality of operation instruction images having different operation orders, the image display control is performed so that the operation instruction images that are operated by the driver later in the operation order are displayed higher in the display area 700. Is going. That is, the course change operation instruction images 721, 722, and 723, which are operated later than the driving lane instruction image 711, are displayed above the driving lane instruction image 711. Thereby, it is possible to avoid a situation in which the driver becomes confused about the order of operations regarding the operation instruction contents of the driving lane instruction image 711 and the operation instruction contents of the course change operation instruction images 721, 722, and 723. This is due to the following reason.

The driver 300 who is driving usually has the concept of distance that an object located in front of the traveling own vehicle 301 approaches the current position as time passes. On the other hand, the driver 300 also has the concept of time that future events will approach the current time as time passes. The shorter the distance to the own vehicle 301, the faster the object existing in front of the traveling own vehicle 301 will reach the own vehicle 301, and the farther the distance to the own vehicle 301, the faster the object will reach the own vehicle 301's position. The time it takes to reach is slow. Therefore, an object that is far away in front of the traveling own vehicle 301 has a high recognition affinity with an event in the distant future, and an object located near the front of the traveling own vehicle 301 has a high recognition affinity with an event in the near future. There is a high cognitive affinity between the two.

In consideration of this affinity, an operation instruction image in the distant future (an operation instruction image indicating the content of the operation instruction whose operation order will be later) is created so that it overlaps the position where an object far away in front of the traveling host vehicle 301 is visible. That is, display the course change operation instruction images 721, 722, 723), and perform operations in the nearer future (including the present) so that the object located near the front of the traveling own vehicle 301 overlaps with the visible position. By displaying an instruction image (operation instruction image showing the operation instruction contents whose operation order comes first, that is, driving lane instruction image 711), the driver can intuitively recognize the operation order of these operation instruction contents. I can do it.

Here, as mentioned above, the driver 300 while driving is usually gazing at an infinite point in front of his own vehicle or at the back of the preceding vehicle 350 that is running several tens of meters ahead. The point of view is approximately in the vicinity of the vertical center of the front scenery seen through the windshield 302. In this embodiment, the display area 700 in which the driving lane instruction image 711 and the course change operation instruction images 721, 722, and 723 with different operation orders are displayed is positioned at the bottom of the front scenery visible from the windshield 302. . Therefore, the driving lane instruction image 711 and the course change operation instruction images 721, 722, and 723 are displayed below the gaze point of the driver 300 so as to overlap with the scenery ahead. In the forward scenery portion below the gaze point of the driver 300, objects farther from the own vehicle 301 are viewed higher, and objects closer to the own vehicle 301 are viewed lower.

Therefore, in this embodiment, the course change operation instruction images 721, 722, and 723, which are operated later than the driving lane instruction image 711, are displayed above the driving lane instruction image 711. Specifically, course change operation instruction images 721, 722, and 723 whose operation order is later are displayed in the upper display area A of the display area 700 that overlaps with the visible position of an object that is far forward from the traveling own vehicle 301. , a driving lane instruction image 711 that is operated first is displayed in the middle display region B of the display area 700, which overlaps with the visible position of an object located near the front of the traveling own vehicle 301. This allows the driver to intuitively recognize the order of operation of these operation instructions due to the high recognition affinity mentioned above, and avoids situations where the driver becomes confused about the order of operations. be able to.

Conversely, without considering this cognitive affinity, for example, the display positions of the driving lane instruction image 711 and the course change operation instruction images 721, 722, 723 may be set upside down from the present embodiment. , side-by-side, it becomes difficult for the driver to intuitively recognize the order of operations because the above-mentioned concept of distance and time interferes with the driver.

Further, by utilizing this cognitive affinity, in this embodiment, a plurality of operation instruction images related to operation instruction contents having the same operation order are arranged side by side. Specifically, the three course change operation instruction images 721, 722, and 723 displayed in the upper display area A all indicate the next course change operation at an intersection or branch point where the course should be changed from the straight direction. Since these information indicate the course change operation instruction information, they are arranged side by side in the upper display area A. According to the concept of distance and the concept of time described above, the three course change operation instruction images 721, 722, and 723 arranged side by side at the same vertical position are information regarding the operation instruction contents at the same time. Easy to recognize intuitively. Therefore, the information in the course change operation instruction images 721, 722, and 723 can be appropriately recognized.

Furthermore, in this embodiment, a stereoscopic image expressed using stereoscopic vision is used as the virtual image G displayed in the display area 700. Specifically, as the inter-vehicle distance presentation image 712 and driving lane instruction image 711 displayed in the middle display area B of the display area 700, a perspective image expressed by a perspective method is used.

Specifically, the length of the five horizontal lines constituting the inter-vehicle distance presentation image 712 becomes shorter toward the top, and the perspective drawing method is used to draw the inter-vehicle distance presentation image 712 toward one vanishing point. It is considered a legal image. In particular, in this embodiment, since the inter-vehicle distance presentation image 712 is displayed such that its vanishing point is determined near the driver's gaze point, the driver 300 who is driving has a sense of depth in the inter-vehicle distance presentation image 712. Easy to do. Furthermore, in this embodiment, the perspective image is such that the thickness of the horizontal line becomes thinner as it goes upward, and the brightness of the horizontal line becomes lower as it goes upward. This makes it easier for the driver 300 while driving to perceive the sense of depth of the inter-vehicle distance presentation image 712.

By using such a stereoscopic image to create a sense of depth in the virtual image G displayed in the display area 700, the vertical position of the image displayed in the display area 700 and the distance to the object in the front scenery can be adjusted. It becomes easier for the driver to perceive the relevance. As a result, the above-mentioned recognition affinity is more easily obtained, and the operation instruction content of the driving lane instruction image 711 displayed in the middle display area B of the display area 700 is more likely to be displayed in the upper display area A of the display area 700. It becomes easier for the driver to intuitively recognize that this is the content that should be operated before the displayed course change operation instruction images 721, 722, and 723.

Furthermore, like the inter-vehicle distance presentation image 712, the driving lane instruction image 711 is also drawn using perspective drawing so that the vanishing point is the same as the vanishing point of the inter-vehicle distance presentation image 712. However, the driving lane instruction image 711 has the aspect of providing operation instruction content to the driver. Therefore, if priority is given to clearly providing the operation instruction content of the driving lane instruction image 711 to the driver rather than creating a sense of depth, for example, the three composing the driving lane instruction image 711 The brightness of the arrow image may increase toward the tip of the arrow. Compared to other methods, the laser scanning method makes it easier to adjust the brightness for each image area, so it is possible to increase the contrast between low and high brightness areas. Therefore, as described above, this is effective when increasing the brightness at the tip of the arrow to clearly provide information to the driver. In this case, since the brightness of the driving lane instruction image 711 decreases toward the top, it is easier to create a sense of depth by lowering the brightness toward the top, so the sense of depth cannot be enhanced, but the tip of the arrow By increasing the brightness, the visibility of the direction of the arrow increases, and it becomes possible to more clearly provide the operation instruction content of the driving lane instruction image 711 to the driver. Note that even in this case, since the driving lane instruction image 711 is drawn using perspective drawing, the sense of depth will not be significantly impaired.

Next, a method of creating a sense of distance and depth by making the user perceive the distance to the virtual image G using motion parallax will be described.
In this embodiment, a motion parallax image expressed by motion parallax is used as the virtual image G. Motion parallax refers to parallax caused by movement of the eye position (viewpoint position) of driver 300. The sense of distance and depth due to motion parallax is caused by the difference in movement in which when the position of the driver's 300 eyes moves, the closer objects in the front scenery are perceived as moving more, and the farther objects are visually perceived as moving less. The person 300 perceives the sense of distance and depth to each object.

In this embodiment, as shown in FIG. 2, a driver camera 150 as a viewpoint position detection means for observing the eye position (viewpoint position) of the driver 300 is placed near the rearview mirror of the windshield 302 of the own vehicle 301. is set up. The installation position of the driver camera 150 is preferably close to the midline of the driver 300 sitting in the driver's seat in order to accurately capture the vertical and horizontal movements of the driver 300. However, it is preferable to arrange it, for example, in an upper position so as not to obstruct the driver's 300's view.

The driver camera 150 is a monocular camera that is set to capture an image area in which the driver 300 sitting in the driver's seat is expected to move his or her head while driving. Like the camera sections 110A and 110B, it is composed of a lens, an image sensor, a sensor controller, and the like. As the driver camera 150, a stereo camera may be used to grasp the position of the driver's eyes in the front and rear directions.

The brightness image data of the captured image captured by the driver's camera 150 is input to the image control device 250. The image control device 250 recognizes the position of the eyes of the driver 300 based on the brightness image data from the driver camera 150 when the CPU 251 executes the information provision control program stored in the ROM 253 or the like. In this embodiment, the head position of the driver 300 is simply recognized based on the brightness image data from the driver camera 150, and the positions of the driver's 300 eyes are estimated from the recognition result. As a method for recognizing the head position of the driver 300, a wide variety of general recognition processing methods can be adopted.

FIG. 8 is an explanatory diagram for explaining an image processing method for the virtual image G that produces a sense of depth by using motion parallax in this embodiment.
As shown in FIG. 8, when the head of the driver 300 moves by Dd, the visible position of the object Oa located at a distance La near from the driver 300 moves by Da, and is located at a distance Lb far from the driver 300. The visible position of the object Ob moves by Db, which is less than Da, and the visible position of the object Oc, which is located at a further distance Lc from the driver 300, moves by Dc, which is even less than Db. Due to the difference in the movement amounts Da, Db, and Dc of the visible positions among these objects Oa, Ob, and Oc, the driver 300 can determine that the object Oa is located at a distance La, and the object Ob is located at a distance Lb. It is possible to perceive that the object Oc is present at a distance Lc.

The virtual image G in this embodiment is displayed at a distance of 5 m from the driver 300, and any image portion on the virtual image G is displayed at a distance of 5 m from the driver 300. In this embodiment, by using the above-mentioned motion parallax, the driver 300 is made to perceive a plurality of image parts on the virtual image G as if they were displayed at different distances from each other.

Specifically, the image control device 250 recognizes the head position of the driver 300 based on the brightness image data of the captured image captured by the driver camera 150 at predetermined time intervals. The image control device 250 then calculates the amount of driver head movement Dd that the driver's 300 head has moved during the time interval. At this time, the visible position of the virtual image G displayed at a distance of 5 m will move by Da.

In this embodiment, the display position of the image portion displayed in the lower display area C is fixed within the display area 700. Therefore, the visible position of the image portion displayed in the lower display area C moves by the same amount of movement Da as the amount of movement of the virtual image G. Therefore, the image portion displayed in the lower display area C is perceived by the driver 300 as being displayed at a distance La (5 m).

On the other hand, the image control device 250 allows the driver to move within the display area 700 regarding the image portion displayed in the middle display area B of the display area 700 of the virtual image G, according to the calculated driver head movement amount Dd. The head is moved by Da-Db in the opposite direction to the head movement direction. As a result, the visual position of the image portion displayed in the middle display area B as seen from the driver 300 is moved by the amount of movement Db. As a result, the image portion displayed in the middle display area B is perceived by the driver 300 as being displayed at a distance Lb.

Similarly, the image control device 250 controls the driver to drive within the display area 700 regarding the image portion displayed in the upper display area A of the display area 700 of the virtual image G according to the calculated amount of driver head movement Dd. The person's head is moved by Da-Dc in the opposite direction to the direction of movement. As a result, the visual position of the image portion displayed in the upper display area A as seen from the driver 300 is moved by the amount of movement Dc. As a result, the image portion displayed in the upper display area A is perceived by the driver 300 as being displayed at a distance Lc.

As described above, the virtual image G is projected while controlling the amount of movement Db, Dc of the visual recognition position of the image portion displayed in the upper display area A and the middle display area B according to the amount of movement Dd of the driver's head. Therefore, the driver 300 displays the image portion of the middle display area B (driving lane instructions) at a position farther from the image portion of the lower display area C (road name display image 701, speed limit display image 702, overtaking prohibition display image 703, etc.). Image 711, inter-vehicle distance presentation image 712, etc.) are displayed, and image portions of upper display area A (route change operation instruction images 721, 722, 723, etc.) are displayed in a position further away from the image portion of middle display area B. Perceive it as if it were. In this way, it is possible for the driver 300 to perceive image portions on the virtual image G that are displayed at the same distance as if they were displayed at different distances, thereby giving a sense of depth to the virtual image G.

In particular, in this embodiment, the image portion of the middle display area B is divided into a plurality of sections in the vertical direction, and the amount of movement is made different for each section according to the amount of movement Dd of the driver's head. The part of the image that is located closer to the viewer is perceived to be displayed farther away. As a result, not only perspective expression but also motion parallax is used for the inter-vehicle distance presentation image 712 and driving lane instruction image 711 displayed in the middle display area B, so that further depth can be obtained.

FIG. 9 is an explanatory diagram showing an example image of a situation where the driver changes course at the nearest intersection.
When a situation arises in which the driver changes course at the nearest intersection, the image example shown in FIG. 1 is switched to the image example shown in FIG. 9. That is, instead of the driving lane instruction image 711 that was displayed in the middle display area B, a route designation image 717 similar to the route designation image 721 that was displayed in the upper display area A in the image example shown in FIG. Display in display area B. This is because the content of the operation instruction after the most recent one has become the content of the most recent operation instruction. Thereby, the driver 300 can recognize that the driver 300 should operate according to the operation instruction content of the route designation image 721 displayed in the middle display area B at the next intersection or branch point.

In the image example shown in FIG. 9, the driver 300 is displayed in the route designation image 717 displayed in the middle display area B at the position of the route designation image 721 that was displayed in the upper display area A in the image example shown in FIG. A downward mark image 725 for guiding the line of sight is displayed. This makes it easier for the driver 300 to recognize the route designation image 717 displayed in the middle display area B.

Next, display control of the following distance presentation image 712 and the preceding vehicle image 724 for informing the driver of the preceding vehicle approaching information indicating that the following vehicle distance to the preceding vehicle 350 is narrowing will be explained.
In this embodiment, the distance between the vehicle and the preceding vehicle 350 is recognized by distance detection means such as the object recognition device 100 and the sensor device 500. Then, the image control device 250 receives the recognition result data (front distance information), and based on this recognition result data, the preceding vehicle image 724 displayed in the upper display area B and the inter-vehicle distance displayed in the middle display area B. For the distance presentation image 712, image display control is performed to change the display position, brightness, color, image shape, etc. within the display area 700. In the present embodiment, changes in the inter-vehicle distance presentation image 712 and the preceding vehicle image 724 inform the driver 300 who visually recognizes the changes to how much the inter-vehicle distance to the preceding vehicle 350 has narrowed.

[Image example 1]
FIGS. 10(a) to (e) show image examples (hereinafter, this image example will be referred to as "image example 1") in which an image indicating front vehicle approach information changes depending on the distance between the vehicle and the preceding vehicle 350. It is an explanatory diagram.
In this image example 1, no destination is set in the vehicle navigation device 400 and no operation instruction image related to route navigation information is displayed. However, even if an operation instruction image related to route navigation information is displayed. , is similar.

In this image example 1, when the preceding vehicle 350 is not recognized, the image control device 250 displays the image in the middle display area B such that the length becomes shorter and the brightness decreases toward the top, as shown in FIG. 10(a). An overall low-luminance inter-vehicle distance presentation image 712 consisting of five horizontal lines is displayed. At this time, nothing is displayed in the upper display area A.

In addition, when the preceding vehicle 350 is recognized and the distance between the preceding vehicle 350 and the preceding vehicle 350 is so empty that it exceeds a predetermined safe distance range, the image control device 250 displays the middle stage as shown in FIG. 10(b). Display control is performed to increase the brightness of inter-vehicle distance presentation image 712 displayed in display area B. Furthermore, in this image example 1, instead of the horizontal line image at the bottom of the inter-vehicle distance presentation image 712, the own vehicle bumper image 712a is created with thicker lines and both ends thereof extending downward and outward. Perform display control to change the shape. This own vehicle bumper image reminds the driver 300 of the front bumper of the own vehicle 301.

Furthermore, in this image example 1, as shown in FIG. 10(b), a low-luminance preceding vehicle image 724 simulating the back of the vehicle is displayed in the upper display area A. The display of this preceding vehicle image 724 is controlled by driving parallax so that it is perceived by the driver 300 as being displayed at the same perceived distance (Lc) as other images displayed in the upper display area A. Ru.

Further, when the preceding vehicle 350 is recognized and the distance between the preceding vehicle 350 and the preceding vehicle 350 is within a predetermined safe distance range, the image control device 250 displays the image within the middle display area B as shown in FIG. 10(c). Display control is performed to make the brightness of the displayed inter-vehicle distance presentation image 712 and the brightness of the preceding vehicle image 724 displayed in the upper display area A higher than in the case of FIG. 10(b). Furthermore, in this image example 1, display control is also performed to change the second horizontal line image from the bottom to the own vehicle bumper image 712a. As a result, the own vehicle bumper image 712a becomes closer to the preceding vehicle image 724 than in the case of FIG. It can be made to recognize that it is approaching 350.

Furthermore, in this image example 1, the preceding vehicle image 724 shown in FIG. 10(c) has a perceived distance Le1 (Lb<Le1<Lc) that is shorter than other images displayed in the upper display area A due to driving parallax. The display is controlled so that it is perceived by the driver 300 as being displayed. As a result, the driver perceives that the preceding vehicle image 724 is coming closer than in the case of FIG. The fact that the vehicle 350 is approaching can be more strongly recognized.

Further, when the preceding vehicle 350 is recognized and the distance between the preceding vehicle 350 and the preceding vehicle 350 is closer than the predetermined safe distance range but is further than the predetermined required braking distance, the image control device 250 performs the following operation as shown in FIG. 10(d). As shown in FIG. 10(c), the brightness of the inter-vehicle distance presentation image 712 displayed in the middle display area B and the brightness of the preceding vehicle image 724 displayed in the upper display area A are compared to the image example of FIG. 10(c). Implement display control that enhances the display. Furthermore, in this image example 1, display control is also performed to change the third horizontal line image from the bottom to the own vehicle bumper image 712a. As a result, the own vehicle bumper image 712a becomes closer to the preceding vehicle image 724 than in the case of FIG. It is possible to recognize that the vehicle 350 is getting closer.

Furthermore, in this image example 1, regarding the preceding vehicle image 724 shown in FIG. 10(d), due to driving parallax, the perceived distance Le2 (Lb<Le2<Le1 ) The display is controlled so that it is perceived by the driver 300 as shown in FIG. As a result, the driver perceives that the preceding vehicle image 724 is coming closer than in the case of FIG. The fact that the vehicle 350 is getting closer can be more strongly recognized.

Further, when the preceding vehicle 350 is recognized and the distance between the preceding vehicle 350 and the preceding vehicle 350 is narrowed to within a predetermined required braking distance range, the image control device 250 displays the middle display area B as shown in FIG. 10(e). Display control is performed to change the inter-vehicle distance presentation image 712 displayed within the frame to a brake warning image 714 in which a character image of "Brake!" is drawn in a red substantially trapezoidal shape. At this time, by lowering the brightness of the preceding vehicle image 724 displayed in the upper display area A or stopping filling the image, the visibility of the preceding vehicle image 724 is lowered, thereby making the brake warning image 714 more visible. It is preferable to make it visible. Note that the preceding vehicle image 724 in the upper display area A may be hidden so that the brake warning image stands out. At this time, if the laser scanning method is used, it is possible to completely block the light in the non-image area, so the brake warning image can be made more conspicuous, and the warning effect can be enhanced.

Note that the above-mentioned predetermined safety distance and predetermined required braking distance may be fixed distances determined in advance, or may be distances that vary depending on the vehicle speed of the host vehicle 301 or the like.

[Image example 2]
The expression method for making the driver 300 aware that the distance between the preceding vehicle 350 and the preceding vehicle 350 has become narrow is not limited to the image example 1 described above, but also image examples such as those shown in FIGS. 11(a) and 11(b). (Hereinafter, this image example will be referred to as "image example 2.").

Similar to image example 1 shown in FIGS. 10(a) to 10(e), image example 2 is also an image example in which the image indicating the front vehicle approach information changes depending on the inter-vehicle distance from the preceding vehicle 350. Specifically, the inter-vehicle distance presentation image 715 of this image example 2 uses a trapezoidal image drawn by perspective drawing toward the vanishing point instead of the five horizontal lines of the inter-vehicle distance presentation image 712 of the above-mentioned image example 1. It is divided into four sections in the vertical direction. In the image example 1 described above, as the distance from the preceding vehicle 350 decreases, the host vehicle bumper image 712a moves upward to approach the preceding vehicle image 724. As the distance narrows, the high-intensity trapezoid section 715a moves upward so as to approach the preceding vehicle image 724. Note that the image in FIG. 11(a) corresponds to the image shown in FIG. 10(d), in which the preceding vehicle 350 is recognized and the distance between the preceding vehicle 350 and the preceding vehicle 350 is closer than a predetermined safe distance range. This is an example where the distance is farther than the predetermined required braking distance.

Here, in this image example 2, a destination is set on the vehicle navigation device 400, and an operation instruction image related to route navigation information is displayed. Therefore, in a situation where the preceding vehicle 350 is not recognized, as in the example shown in FIG. A distance image 722 and an intersection name image 723 are displayed.

In this situation, when the preceding vehicle 350 is recognized and the preceding vehicle image 724 is to be displayed in the upper display area A, all of the course change operation instruction images 721, 722, and 723 should be hidden from the upper display area A. can also be considered. However, in this case, the operation instruction contents (operation instruction contents later in the operation order) by the route change operation instruction images 721, 722, and 723 are not provided to the driver 300, which may cause anxiety to the driver 300.

On the other hand, when moving the course change operation instruction images 721, 722, 723 displayed in the upper display area A to a display area different from the upper display area A to continue displaying them, the driver It may take time to search for the route change operation instruction images 721, 722, 723, etc., and there is a possibility that the operation instruction contents by the route change operation instruction images 721, 722, 723 cannot be accurately provided to the driver 300.

According to this image example 2, even when displaying the preceding vehicle image 724 in the upper display area A where the course change operation instruction images 721, 722, 723 are displayed, the course change operation instruction images 721, 722, 723 are displayed. It continues to be displayed in the upper display area A. Therefore, even when displaying the preceding vehicle image 724 in the upper display area A, the operation instruction contents by the course change operation instruction images 721, 722, and 723 can be accurately provided to the driver 300.

Note that when displaying the preceding vehicle image 724 in the upper display area A where the course change operation instruction images 721, 722, and 723 are displayed, all of the course change operation instruction images 721, 722, and 723 are displayed in the upper display area A. It is not necessary to continuously display the image 723; for example, only the intersection name image 723 may be hidden.

In this image example 2, when the inter-vehicle distance with the preceding vehicle 350 approaches within the predetermined braking distance range, the image control device 250 displays the upper display area A and the middle display area as shown in FIG. 11(b). A brake warning image 716 that straddles display area B is displayed. This brake warning image 716 is a combination of a red image obtained by enlarging the preceding vehicle image 724 displayed in the upper display area A and an image in which all trapezoidal sections of the inter-vehicle distance presentation image 715 of image example 2 are displayed in red. It is something that Moreover, in this image example 2, in order to make the brake warning image 716 more noticeable, all of the course change operation instruction images 721, 722, and 723 are hidden.

[Image example 3]
As an expression method to make the driver 300 aware that the distance between the preceding vehicle 350 and the preceding vehicle 350 has become narrow, for example, image examples as shown in FIGS. Example 3).
Similar to image example 1 and image example 2 described above, this image example 3 is also an image example in which an image indicating front vehicle approach information changes depending on the inter-vehicle distance from the preceding vehicle 350. Specifically, the image showing the front vehicle approach information in this image example 3 is a simple figure (a circle in this image example 3) instead of the preceding vehicle image 724 that imitates the rear of the vehicle, the inter-vehicle distance presentation image 712, etc. A simple graphic image 713 composed of the following elements is used. In this image example 3, display control is performed such that as the distance to the preceding vehicle 350 decreases, the perceived distance to the simple graphic image 713 decreases.

Specifically, as the distance to the preceding vehicle 350 narrows, the display is controlled so that the simple graphic image 713 is enlarged, or the brightness of the simple graphic image 713 is decreased, as shown in FIGS. 12(a) to (c). The display can be controlled so that it increases. In particular, in this image example 3, display control is performed so that the perceived distance due to motion parallax of the simple graphic image 713 becomes shorter as the distance to the preceding vehicle 350 becomes smaller.

That is, the virtual image G including the simple graphic image 713 is displayed at a distance of 5 m from the driver 300, but in the case of FIG. Display control is performed so that the perceived distance is the same as or close to the perceived distance Lc of area A (for example, 10 m). Specifically, the image control device 250 controls a simple image displayed in the middle display area B of the display area 700 of the virtual image G according to the driver's head movement amount Dd calculated from the captured image data of the driver camera 150. By changing the position of the graphic image 713 within the display area 700, the display is controlled so that the perceived distance of the simple graphic image 713 is approximately 10 m as described above.

In addition, in the case of FIG. 12(b) in which the distance to the preceding vehicle 350 is closer than that in FIG. 12(a), the perceived distance is shorter (for example, 7 m) than the perceived distance in the case of FIG. Perform display control. Specifically, the image control device 250 controls a simple image displayed in the middle display area B of the display area 700 of the virtual image G according to the driver's head movement amount Dd calculated from the captured image data of the driver camera 150. By changing the position of the graphic image 713 within the display area 700, the display is controlled so that the perceived distance of the simple graphic image 713 is approximately 7 m as described above.

Similarly, in the case of FIG. 12(c) where the distance to the preceding vehicle 350 is even closer than that of FIG. 12(b), the perceived distance is shorter than the perceived distance in FIG. 12(b) (for example, 4 m). Implement display control so that Specifically, the image control device 250 controls a simple image displayed in the middle display area B of the display area 700 of the virtual image G according to the driver's head movement amount Dd calculated from the captured image data of the driver camera 150. By changing the position of the graphic image 713 within the display area 700, the display is controlled so that the perceived distance of the simple graphic image 713 is approximately 4 m as described above. Note that this perceived distance (4 m) is shorter than the distance of the virtual image G (5 m), but this can also be realized by the display control described above.

[Image example 4]
Next, another image example (hereinafter, this image example will be referred to as "image example 4") showing a situation where the driver changes course at the nearest intersection will be described.
When the situation arises to change course at the nearest intersection, in the embodiment described above, the image example shown in FIG. 9 is used to provide the driver 300 with information on which intersection and in which direction to change course. In this image example 4, image examples shown in FIGS. 13(a) to (c) are used.

In this image example 4, a route designation image 717 is displayed in the display area 700 so as to overlap an actual intersection that is visible to the driver through the windshield 302. In the image example shown in FIG. 9, remaining distance information as passive information indicating the remaining distance to the intersection, etc. where the course change operation is performed is provided to the driver 300 as a remaining distance image 722 indicating the remaining distance as a numerical image. are doing. On the other hand, in this image example 4, by changing the perceived distance of the route designation image 717 according to the remaining distance information, the remaining distance to the intersection, etc. at which the course change operation is performed can be calculated based on the perceived distance of the route designation image 717. Information is provided to the driver 300.

Specifically, the image control device 250 adjusts the course displayed in the middle display area B of the display area 700 of the virtual image G according to the amount of driver head movement Dd calculated from the captured image data of the driver camera 150. By changing the position of the designated image 717 within the display area 700, the display is controlled so that the perceived distance of the simple graphic image 713 matches the actual distance to the intersection. The distance to the actual intersection can be obtained from the vehicle navigation device 400, for example.

Note that it is not always necessary to match the perceived distance of the simple figure image 713 with the distance to the actual intersection; by gradually narrowing the perceived distance of the simple figure image 713, it is possible to detect that the distance to the actual intersection is getting closer. It is sufficient if the driver can be made aware of it.

[Image example 5]
Further, among information regarding prohibitions, restrictions, and designations of traffic on the road on which the own vehicle 301 is traveling, information that is particularly preferred to be provided to the driver 300 is displayed as a warning image in the middle display area B. You can also do this. For example, as in the image example shown in FIG. 14 (hereinafter, this image example will be referred to as "image example 5"), when the road on which the host vehicle 301 is traveling is a school road, the school road is shown to that effect. A warning image 718 is displayed in the middle display area B. Specifically, the image control device 250 acquires unique information of the road on which the host vehicle 301 is traveling from various information outputted from the vehicle navigation device 400, and displays the acquired road unique information in the middle display area B. If the road-specific information is the target information to be displayed, display control is performed to display a warning image corresponding to the road-specific information in the middle display area B.

As described above, the middle display area B is an area where an operation instruction image indicating the operation instruction content to be operated first among a plurality of operation instruction contents is displayed. Therefore, by displaying a warning image in the middle display area B as in image example 5, the driver can confirm that the road on which the own vehicle 301 is currently traveling is prohibited, restricted, or designated by the warning image. You can intuitively recognize what is being done.

[Image example 6]
Furthermore, when a person attempting to cross a crosswalk in the direction of travel of the host vehicle 301 is detected, a warning image thereof may be displayed in the middle display area B. This is because humans are one of the objects that vehicles should pay the most attention to, and they are the ones that should be dealt with immediately. For example, a humanoid warning image 719 is displayed in the middle display area B, as in the image example shown in FIG. 15 (hereinafter, this image example will be referred to as "image example 6"). Specifically, the image control device 250 uses the object recognition device 100 and the sensor device 500 to recognize a person in front of the host vehicle. Then, the image control device 250 receives the recognition result data and performs display control to display the humanoid warning image 719 in the middle display area B based on the recognition result data. Note that this image example 6 is an example in which the school route warning image 718 in image example 5 described above is also displayed. Note that when the information providing device of the present embodiment is installed in a vehicle other than a vehicle, for example, a ship, other ships, etc. around the own ship can be displayed instead of humans.

What has been described above is just an example, and each of the following aspects has its own unique effects.
(Aspect A)
A driver-provided information image indicating driver-provided information to be provided to a driver 300 of a mobile object such as the host vehicle 301 is displayed at a predetermined location in front of the moving object in the direction in which the driver views it through a light-transmitting member such as a windshield 302. An information providing device such as an automobile HUD device 200 that is equipped with an image light projection means such as a HUD main body 230 that projects image light onto the light transmitting member so as to display the image light on the display area 700 of the moving object. Distance detection means such as the object recognition device 100 that detects the relative distance in the moving direction of the moving body of objects to be detected such as the preceding vehicle 350 existing in the surroundings, and the viewpoint position of the driver camera 150 that detects the viewpoint position of the driver. By changing the display position of an object image such as the preceding vehicle image 724 showing the detection object according to the detection result of the detection means and the viewpoint position detection means, the driver can detect the object image by motion parallax with respect to the object image. display control means such as an image control device 250 for controlling the image light projection means so that the perceived distance of the detection object changes in accordance with the relative distance in the traveling direction of the moving object detected by the distance detection means. It is characterized by
It is beneficial to provide the driver with information on the relative movement of objects around the moving body with respect to the own vehicle (surrounding object information) using images projected by the image light projection means. For example, if it is difficult for the driver to directly see objects around the moving object due to poor visibility around the moving object, an object display image corresponding to the object displayed using the HUD device may be used to If object information can be provided to the driver, it will be possible to drive more safely. In addition, not only in situations where it is difficult for the driver to see directly, but if the driver can be provided with information on surrounding objects that are often overlooked due to lack of attention, it will be possible to drive more safely. be. Moreover, when attempting to provide surrounding object information to the driver in this way, it is difficult to make the driver perceive that the depth position of the object display image changes according to the relative movement of the object with respect to the moving object. This is effective in making it easier for the driver to recognize surrounding object information. An effective way to make the driver perceive this is to use motion parallax.
According to this aspect, the predetermined display area in front of the moving body is determined according to the relative distance (relative distance in the moving direction of the moving body) of the detection target existing around the moving body to the moving body in the moving direction of the moving body. The perceived distance of the object image displayed can be changed. In this aspect, since the driver is made to perceive the perceived distance of the target object image using motion parallax, it is easy for the driver to recognize the movement information (surrounding object information) in which the detected object approaches or moves away from the moving body. .

(Aspect B)
Aspect A is characterized in that the object to be detected is another moving object such as a preceding vehicle 350 traveling in front of the moving object.
According to this, it becomes easier for the driver 300 to recognize information that the distance to another preceding moving object is narrowing (front vehicle approach information, etc.).

(Aspect C)
A driver-provided information image indicating driver-provided information to be provided to a driver 300 of a mobile object such as the host vehicle 301 is displayed at a predetermined location in front of the moving object in the direction in which the driver views it through a light-transmitting member such as a windshield 302. An information providing device such as an automobile HUD device 200 comprising an image light projection means such as a HUD main body 230 that projects image light onto the light transmitting member so as to be displayed in a display area 700 of the predetermined display area 700. The area includes active information such as a road name display image 701, a speed limit display image 702, an overtaking prohibition display image 703, and a vehicle speed display image 704, which show active information such as road specific information and vehicle speed information that are actively recognized by the driver. An active information image display area such as a lower display area C that continuously displays an image, a driving lane instruction image 711 that shows passive information such as route navigation information and front vehicle approach information that are passively recognized by the driver, and a course change operation. Passive information such as upper display area A and middle display area B that temporarily display passive information images such as instruction images 721, 722, 723, inter-vehicle distance presentation image 712, and preceding vehicle image 724 at a timing when predetermined information provision conditions are met. and a viewpoint position detection means such as a driver camera 150 for detecting the viewpoint position of the driver, and displaying information in the passive information image display area according to the detection result of the viewpoint position detection means. By changing the display position of the passive information image, the driver's perceived distance Lb, Lc for the passive information image due to motion parallax is changed to the driver's perception for the active information image displayed in the active information image display area. It is characterized by comprising display control means such as an image control device 250 that controls the image light projection means so that the image light projection means is farther than the distance La.
A driver who is driving usually gazes at an infinite point in front of the moving body or at another moving body several tens of meters ahead of the moving body through a light-transmitting member. For a driver who is focusing on such a distant object, the further away the image is displayed, the smaller the difference in focal length with respect to the point of gaze, so the image is easier to recognize (easier to notice).
Here, the passive information that is passively recognized by the driver is generally information that is desired to be provided to the driver at a timing determined by the information providing device, and usually the timing at which the passive information is provided and the This is information that has a certain relationship with the content of the information. Therefore, it is important that the passive information image indicating the passive information is recognized by the driver 300 as soon as it is displayed.
On the other hand, active information that is actively recognized by the driver is generally sufficient information if it is provided to the driver at the timing desired by the driver. There is little or no relationship between the content and the content. Since the active information needs to be provided at the timing desired by the driver, it continues to be displayed for a certain amount of time or all the time.
In this aspect, the driver perceives the passive information image to be displayed farther away than the active information image due to motion parallax. Therefore, for a driver who is focusing on a distant object, a passive information image is easier to recognize than an active information image. Therefore, when the passive information image is displayed, the driver is likely to notice the passive information image even if the active information image is displayed. Therefore, the driver 300 can easily recognize the passive information image as soon as it is displayed, and the passive information can be quickly provided to the driver.
Note that for a driver who is focusing on a distant object, it is difficult to recognize an active information image that is perceived to be displayed at a closer distance than a passive information image. However, since the active information image is viewed by the driver 300 with the intention of acquiring the active information, even if the active information image is more difficult to recognize than the passive information image, it is difficult to recognize the active information image when providing the active information. is not a problem.
Rather, as mentioned above, active information images are displayed continuously for a certain amount of time or all the time, so if active information images are easy to recognize, they may distract the driver's attention while driving. , there is a risk that it may actually cause a disadvantage.

(Aspect D)
In aspect C, the passive information includes a plurality of pieces of operation instruction information each indicating different operation instruction contents to the driver of the mobile object, and the display control means displays a driving lane instruction image 711 corresponding to each piece of operation instruction information. and route change operation instruction images 721, 722, 723, etc., the more the operation instruction contents are displayed later in the order of operation for the driver, the higher the position is displayed in the passive information image display area. It is characterized in that the image light projection means is controlled so as to
Drivers usually have the idea of distance that objects located in front of a moving vehicle will approach their current position as time passes, and that future events will approach the current time as time passes. It also has the concept of time. For objects in front of a moving object, the closer the distance to the moving object, the faster it will take to reach the moving object's position, and the farther the distance to the moving object, the faster it will take to reach the moving object's position. time is late. Therefore, objects located far from the moving body have a high cognitive affinity with events in the distant future, and objects located close to the moving body have a high cognitive affinity with events in the near future. is high. According to this aspect, in consideration of this affinity, the operation instruction images that indicate the contents of the operation instructions that are to be operated later in the driver's order are displayed higher in the predetermined display area. To allow a driver to intuitively recognize the operation order of a plurality of operation instruction contents indicated by an instruction image, and to easily avoid a situation where the driver is confused about the operation order.

(Aspect E)
In the aspect D, the image light projection means transmits the light so that the passive information image display area is located at a lower part of the scenery in front of the moving body in the traveling direction that the driver visually recognizes through the light transmitting member. It is characterized by projecting image light onto the member.
The driver 300 who visually recognizes the scenery ahead in the moving direction of the moving object through the light-transmitting member usually drives while gazing near the vertical center of the scenery ahead. In this aspect, since the predetermined display area 700 in which a plurality of operation instruction images are displayed is located at the bottom of the front scenery, the plurality of operation instruction images are displayed vertically below the driver's gaze point. will be done. In the landscape portion where a plurality of operation instruction images displayed at this position overlap, objects farther from the moving body are viewed vertically upward, and objects closer to the moving body are viewed vertically lower. Therefore, the position at which the object in the landscape portion is visually recognized gradually shifts downward as the object approaches the moving object as the moving object advances.
When displaying a plurality of operation instruction images superimposed on such a landscape part, as in this embodiment, the operation instruction images that indicate the content of the operation instructions that are later in the order of operation for the driver are displayed in the predetermined display area. By displaying the information on the upper side, it is possible to minimize the driver's confusion regarding the operation order. In consideration of the above-mentioned cognitive affinity, this is an operation instruction image that shows operation instruction contents in the distant future (operation instruction contents whose operation order will be later) so that it overlaps the position where an object far from the moving object is visible. This is because it is possible to display an operation instruction image indicating the operation instruction content in the near future (operation instruction content whose operation order is first) so as to overlap the position where an object close to the moving object is visible. be. By displaying in this manner, the driver can intuitively recognize the order of operations of the plurality of operation instruction contents indicated by these operation instruction images. Conversely, when the predetermined display area 700 is located at the bottom of the front scenery, for example, the display position of the plurality of operation instruction images may be reversed from the above-mentioned position, or the display of the plurality of operation instruction images may be changed. If the positions are arranged side by side, the driver will be hindered by the above-mentioned concept, and it will be difficult for the driver to intuitively recognize the order of operations.

(Aspect F)
In the aspect D or E, the plurality of operation instruction images are provided by the driver at different points on the moving route of the mobile object (such as the nearest intersection and the intersection or junction where the next course should be changed from the straight direction). It is characterized in that it indicates the content of an operation instruction that instructs the content of the operation to be performed.
According to this, it is possible to avoid a situation in which the driver is confused about the order of operations for a plurality of operation instruction contents having different orders of operations based on the route navigation information.

(Aspect G)
In any one of the aspects A to F, the image light projection means displays the operation instruction image as a virtual image G in the predetermined display area by projecting image light, and the image light projection means displays the operation instruction image as a virtual image G in the predetermined display area, and It is characterized by a distance of 5 m or more.
When the distance to the virtual image G is generally about 2 m, convergence movement of the eyeballs is usually required to focus the eyeballs on the virtual image G at such a short distance. As mentioned above, convergence movement is a factor that greatly affects the sense of distance and depth to the visual object, and if the eyeballs move convergence to focus on the virtual image G, the sense of distance (perception) due to motion parallax will be affected. The effect of perceiving the sense of depth (change in distance) and sense of depth (difference in perceived distance) is weakened.
According to this aspect, since the distance to the virtual image G is 5 m or more, it is possible to focus on the virtual image G with almost no convergence movement of the eyeballs. Therefore, the effect of using motion parallax to perceive a sense of distance (change in perceived distance) and a sense of depth (difference in perceived distance) is suppressed from being weakened by the convergence movement of the eyeballs.

(Aspect H)
In any of the aspects A to G, the image light projection means optically scans image light emitted from a light emitting means such as a light source unit 220 that emit image light according to image information of the operation instruction image. The driver-provided information image is displayed within the predetermined display area by performing two-dimensional scanning using a light scanning means such as the device 208 and projecting the image onto the light-transmitting member.
As described above, it is easier to display a large virtual image G with high brightness than a method using a liquid crystal display (LCD), a fluorescent display tube (VFD), or the like. Further, according to this aspect, by not irradiating the non-image portion of the virtual image G with image light from the light irradiation means, it is possible to completely eliminate light from the non-image portion. Therefore, it is possible to avoid a situation in which the visibility of the scenery in front of the mobile body through the non-image portion is reduced due to the light emitted from the light irradiation means, and the visibility of the scenery in front of the moving object is high.

(Aspect I)
The light transmits the light so that a driver-provided information image indicating the driver-provided information to be provided to the driver of the moving object is displayed in a predetermined display area in front of the moving object in the direction in which the driver views it through the light-transmitting member. An information providing method for providing the driver-provided information to a driver by projecting image light onto a transparent member, the method detecting the relative distance in the moving direction of the moving object of a detection target existing around the moving object. A distance detection step, a viewpoint position detection step of detecting the viewpoint position of the driver, and a display position of an object image indicating the detection target object according to the detection results in the viewpoint position detection step, The present invention is characterized by comprising a display step of changing the perceived distance of the driver due to motion parallax regarding the object image in accordance with the relative distance in the traveling direction of the moving body of the detection object detected in the distance detection step.
According to this aspect, the perceived distance of the target object image is made to be perceived by the driver using motion parallax, so the driver recognizes information about the movement of the detected target object toward or away from the moving body (surrounding object information). Easy to do.

(Aspect J)
The light transmits the light so that a driver-provided information image indicating the driver-provided information to be provided to the driver of the moving object is displayed in a predetermined display area in front of the moving object in the direction in which the driver views it through the light-transmitting member. image light projection means for projecting image light onto a transparent member; distance detection means for detecting a relative distance in the moving direction of the moving object of a detection target existing around the moving object; and a viewpoint position for detecting the viewpoint position of the driver. An information provision control program for operating a computer of an information provision apparatus comprising a detection means, the program changing a display position of an object image indicating the detection object according to a detection result of the viewpoint position detection means. the image light projecting means so that the distance perceived by the driver due to motion parallax regarding the object image changes in accordance with the relative distance in the traveling direction of the moving body of the detection object detected by the distance detection means. The computer is characterized in that it functions as a display control means for controlling.
According to this aspect, the perceived distance of the target object image is made to be perceived by the driver using motion parallax, so the driver recognizes information about the movement of the detected target object toward or away from the moving body (surrounding object information). Easy to do.

(Aspect K)
The light transmits the light so that a driver-provided information image indicating the driver-provided information to be provided to the driver of the moving object is displayed in a predetermined display area in front of the moving object in the direction in which the driver views it through the light-transmitting member. An information providing method for providing the driver-provided information to a driver by projecting image light onto a transparent member, wherein the predetermined display area is an active area showing active information that is actively recognized by the driver. An active information image display area that continuously displays an information image, and a passive information image display area that temporarily displays a passive information image indicating passive information that is passively recognized by the driver at a timing when a predetermined information provision condition is met. a viewpoint position detection step of detecting the viewpoint position of the driver; and changing the display position of the passive information image displayed in the passive information image display area according to the detection result in the viewpoint position detection step. Accordingly, the image light projecting means may be configured such that the perceived distance of the passive information image by the driver due to motion parallax is farther than the perceived distance of the driver regarding the active information image displayed in the active information image display area. and a display control step for controlling.
According to this aspect, the passive information image is easier to recognize than the active information image for a driver who is focusing on a distant object, so when the passive information image is displayed, the driver can Passive information images are easy to notice even when information images are displayed. Therefore, the driver 300 can easily recognize the passive information image as soon as it is displayed, and the passive information can be quickly provided to the driver.

(Aspect L)
The light transmits the light so that a driver-provided information image indicating the driver-provided information to be provided to the driver of the moving object is displayed in a predetermined display area in front of the moving object in the direction in which the driver views it through the light-transmitting member. An information provision control program for operating a computer of an information provision apparatus comprising an image light projection means for projecting image light onto a transparent member, and a viewpoint position detection means for detecting the viewpoint position of the driver, the program comprising: The predetermined display area includes an active information image display area that continuously displays an active information image showing active information that is actively recognized by the driver, and a passive information image display area that shows passive information that is passively recognized by the driver. a passive information image display area that temporarily displays a passive information image display area at a timing when a predetermined information provision condition is met; By changing the display position, the driver's perceived distance for the passive information image due to motion parallax is farther than the driver's perceived distance for the active information image displayed in the active information image display area, The computer is characterized in that the computer functions as display control means for controlling the image light projection means.
According to this aspect, the passive information image is easier to recognize than the active information image for a driver who is focusing on a distant object, so when the passive information image is displayed, the driver can Passive information images are easy to notice even when information images are displayed. Therefore, the driver 300 can easily recognize the passive information image as soon as it is displayed, and the passive information can be quickly provided to the driver.

Note that the above-mentioned program can be distributed or obtained in a state recorded on a recording medium such as a CD-ROM. It is also possible to distribute or receive a signal carrying the above-mentioned program and transmitted by a predetermined transmitting device via a transmission medium such as a public telephone line, leased line, or other communication network. Available. At the time of this distribution, it is sufficient that at least a part of the computer program is transmitted in the transmission medium. That is, all the data making up a computer program does not need to exist on the transmission medium at one time. The above-mentioned signal carrying a program is a computer data signal embodied in a predetermined carrier wave containing a computer program. Furthermore, the transmission method for transmitting a computer program from a predetermined transmitting device includes both continuous transmission of data constituting the program and intermittent transmission.

100 Object recognition device 110 Stereo camera section 120 Information processing section 150 Driver camera 200 Automobile HUD device 201R, 201G, 201B Laser light source 207 Light amount adjustment section 208 Light scanning device 209 Free-form surface mirror 210 Microlens array 211 Projection mirror 220 Light source unit 230 HUD body 250 Image control device 300 Driver 301 Own vehicle 302 Windshield 350 Leading vehicle 400 Vehicle navigation device 500 Sensor device 700 Display area 701 Road name display image 702 Speed limit display image 703 Overtaking prohibited display image 704 Vehicle speed display image 711 Driving Lane instruction images 712, 715 Inter-vehicle distance presentation image 713 Simple graphic image 712a Own vehicle bumper image 714, 716 Brake warning image 718 School route warning image 719 Humanoid warning image 717, 721 Course designation image 722 Remaining distance image 723 Intersection, etc. name image 724 Leading vehicle image 725 Downward mark image A Upper display area B Middle display area C Lower display area G Virtual image

Patent No. 4686586

Claims (4)

a detection step for detecting the movement of a driver driving the mobile object;
a first image portion that causes the driver to perceive that it is a first distance away from the driver; and a first image portion that causes the driver to perceive that it is a second distance that is greater than the first distance from the driver; a display step of moving and displaying a second image portion to be perceived in accordance with the movement of the driver detected by the detection step when the driver moves;
At least one of the first image portion and the second image portion includes an image showing a human being,
In the displaying step, a second movement amount for moving and displaying the second image portion is larger than a first movement amount for moving and displaying the first image portion. An image display method characterized by displaying an image so that it appears as follows . In the image display method according to claim 1 ,
The image display method is characterized in that the detecting step detects the movement of the driver using an imaging device installed in the moving object. In the image display method according to claim 2 ,
The image display method is characterized in that the detecting step estimates the position of the driver's eyes from the position of the driver's head photographed by the imaging device. a detection unit that detects the movement of a driver driving a mobile object;
a first image portion that causes the driver to perceive that it is a first distance away from the driver; and a first image portion that causes the driver to perceive that it is a second distance that is greater than the first distance from the driver; a display section that moves and displays a second image portion to be perceived in accordance with the movement of the driver detected by the detection section when the driver moves;
At least one of the first image portion and the second image portion includes an image showing a human being,
The display unit has a second movement amount for moving and displaying the second image portion that is larger than a first movement amount for moving and displaying the first image portion. An image display device characterized by displaying images as follows.

Images