WO2017061026A1

WO2017061026A1 - Image display device

Info

Publication number: WO2017061026A1
Application number: PCT/JP2015/078728
Authority: WO
Inventors: 裕己永野; 真希花田; 壮太佐藤
Original assignee: 日立マクセル株式会社
Priority date: 2015-10-09
Filing date: 2015-10-09
Publication date: 2017-04-13
Also published as: JPWO2017061026A1; JP6637986B2

Abstract

Disclosed is an image display device, which outputs light for projecting an image, and displays the image by constituting a virtual image. The image display device is provided with a control module that sets, for a plurality of conditions, angles for outputting light, and a view point position detection module for selecting one condition from among the conditions, said image display device displaying the image by outputting the light under the condition thus selected.

Description

Image display device

The present invention relates to a so-called head-up display (hereinafter referred to as HUD: Head Up Display) that is particularly suitable for being mounted on an aircraft or a vehicle and displaying various information.

In recent years, technology for displaying image information in a real space is drawing attention in the field of entertainment and work support systems. As an example of this, there is a display device in which an optically generated image is reflected to the user side by an optical branching element or the like to generate a virtual image, and the virtual image and the real space are overlapped and shown to the user (see, for example, Patent Document 1). .

Such technology has been put into practical use as a device for displaying various types of information to, for example, automobile occupants.

JP 2014-225017 JP 2011-105306 JP

In the technique described in the background art, the appearance of the image information varies depending on the eye position (viewpoint) of the viewer (for example, an occupant in the case of a car), for example, the height. That is, the state of the image formed on the retina of the occupant is determined by the relationship between the position of the virtual image formed by the HUD and the position of the occupant's eyes. Further, when the position of the virtual image formed by the HUD and the position of the occupant's eyes do not satisfy a predetermined relationship, the occupant may not be able to visually recognize the image information. The range in which the position of the occupant's eyes must enter in order to view the image is called an eyebox.

Patent Document 1 has a mechanism for moving the optical system in order to adjust the projection of the image information with respect to the occupant's viewpoint so that the occupant can visually recognize the image information without depending on the eye height. A device has been proposed.

Patent Document 2 discloses a technology that allows a virtual image projected on a windshield to flexibly cope with variations in the occupant's viewpoint and differences in the shape of the windshield.

However, considering the usage environment of a car or the like, for example, if the occupant gets into the car and sits in the driver's seat, the image information cannot be visually recognized, so it cannot be recognized that the car is originally equipped with a HUD. Even if the presence of the HUD can be recognized by some method, it is necessary to make an adjustment for positioning his / her eyes on the eye box before using the HUD. It is also complicated to associate a virtual image with the viewpoint after positioning the eye in the eye box.

An object of the present invention is to provide a system in which a HUD user can easily use a HUD with a simpler operation.

One aspect of the invention for solving the above-described problems is an image display device that emits light for projecting an image, forms a virtual image, and displays the image, and controls to set the angle at which the light is emitted to a plurality of conditions An image display device includes a module and a viewpoint position detection module that selects one of a plurality of conditions, and displays the image by emitting light according to the selected conditions.

If a specific configuration example is shown, the apparatus further includes a video processing module, the video processing module displays different images for each of a plurality of conditions, and the viewpoint position detection module is configured to specify a specific image among different images depending on a user. By specifying one, a condition corresponding to the specified image is selected.

If another specific configuration example is shown, the viewpoint position detection module selects a condition corresponding to the timing by designating a timing set to a specific one of a plurality of conditions by the user. To do.

In another specific configuration example, the camera further includes a camera that acquires an image of the user's eye, and the viewpoint position detection module detects the timing at which the user's pupil contracts from the image of the user's eye, A condition corresponding to the timing is selected.

In another specific configuration example, the camera further includes a camera that acquires an image of the user's eye, and the viewpoint position detection module detects, from the image of the user's eye, the timing at which the user's pupil is enlarged, The condition immediately before the condition corresponding to the timing is selected.

If you show other specific configuration examples,
A distortion correction module is further provided, and image distortion correction settings are changed in accordance with the condition of the angle at which light is emitted.

It is possible to provide a system that allows HUD users to easily use HUD with simpler operations.

The conceptual diagram explaining the Example of this invention Flow chart for explaining the first embodiment of the present invention The system block diagram explaining Example 1 of this invention Block diagram for explaining the first embodiment of the present invention Table for explaining data used in Example 1 of the present invention Block diagram for explaining a second embodiment of the present invention Flow chart for explaining a third embodiment of the present invention Flow chart for explaining a fourth embodiment of the present invention Block diagram for explaining a fourth embodiment of the present invention Conceptual diagram for explaining a fifth embodiment of the present invention.

Embodiments will be described in detail with reference to the drawings. However, the present invention is not construed as being limited to the description of the embodiments below. Those skilled in the art will readily understand that the specific configuration can be changed without departing from the spirit or the spirit of the present invention.

In the structure of the invention described below, the same portions or portions having similar functions are denoted by the same reference numerals in different drawings, and redundant description may be omitted.

In this specification and the like, notations such as “first”, “second”, and “third” are attached to identify the constituent elements, and do not necessarily limit the number or order. In addition, a number for identifying a component is used for each context, and a number used in one context does not necessarily indicate the same configuration in another context. Further, it does not preclude that a component identified by a certain number also functions as a component identified by another number.

The position, size, shape, range, etc. of each component shown in the drawings and the like may not represent the actual position, size, shape, range, etc. in order to facilitate understanding of the invention. For this reason, the present invention is not necessarily limited to the position, size, shape, range, and the like disclosed in the drawings and the like.

An embodiment of the present invention will be described with reference to FIG. FIG. 1 is a schematic diagram for explaining a situation where a driver (or an occupant) 101 riding in a driver's seat of an automobile visually recognizes image information. Reference numeral 102 denotes a windshield of an automobile. Light 103 for projecting image information emitted from a light source (not shown) is irradiated to the windshield 102 with a first mirror (for example, a folding mirror: a folding mirror hereinafter). : Explained using Folding Mirror) 104 and second mirror (for example, concave mirror: Concave mirror, hereinafter explained using Concave mirror) 105 to reflect and irradiate. The HUD includes a folding mirror 104, a concave mirror 105, a light source, a transmissive liquid crystal panel that displays an image to be projected, a lens, and other optical systems. Note that the transmissive liquid crystal panel that displays an image to be projected may be replaced with a reflective liquid crystal panel, a DMD (Digital Micromirror Device) panel, a MEMS (Micro Electro Mechanical Systems), or the like.

The light 103 is reflected by the windshield 102, enters the eye 106 of the occupant 101, and forms an image on the retina so that the occupant 101 can visually recognize the image information. At this time, the occupant 101 is looking at the virtual image 107 on the other side of the windshield 102.

The occupant 101 may have different eye positions (heights) such as Person A, Person B, and Person C in FIG. In this case, depending on the position (height) of the virtual image 107 to be formed, the occupant 101 cannot visually recognize the image information.

The figure which expanded and showed the virtual image 107 at the upper part of FIG. 1 is shown. The size of the virtual image formed by the HUD is the size of a rectangle 108a indicated by a solid line in the figure, and the rectangles 108b and 108c indicated by dotted lines indicate a state in which the HUD has moved the position of the virtual image. That is, the virtual image (image information) that appears at the same time is indicated by any one of the

rectangles

108a, 108b, and 108c. In the example of FIG. 1, the projected images are switched to circled

numbers

1, 2, and 3 as the virtual image moves.

When the virtual images are projected in order like 108a, 108b, and 108c, the occupant A (Person A) can visually recognize the circled number 2 in the image information of FIG. Cannot be seen. The passenger B (PersonPerB) can visually recognize the image information of number 3 in FIG. 1, but cannot visually recognize the image information of number 2 and number 4.

That is, for example, when the position of the virtual image 107 is fixed at 108b, the occupant B (Person B) can see the image information, but the occupant A (Person A) and the occupant C (Person c) cannot see the image information. become.

Here, it is possible to move the position of the virtual image 107, for example, by changing the angle of the concave mirror 105. For example, assuming a mirror surface and a rotation axis in a direction perpendicular to the paper surface of the figure, and rotating the rotation axis, the imaging position of the virtual image 107 can be changed in the vertical direction.

Hereinafter, an embodiment will be described in which any occupant can recognize the image information of the HUD and set the HUD in an appropriate state with an easy operation.

FIG. 2 shows a control flow of the system of the embodiment of the present invention. Here, the description will be made assuming the initial setting of the HUD immediately after the occupant enters the automobile.

First, the system starts the initial setting (S201). The start of the setting may be started when the occupant actively inputs a command, or may be started when the occupant is seated and the seat belt is worn, for example. Since it is desirable to perform the initial setting after the occupant's posture (eye position) is stabilized, the start after the seat belt is worn is desirable. It is also desirable to call attention by voice guidance such as “Start initial setting of HUD from now on” at the start of setting in order to stabilize the eye position.

Next, the concave mirror 105 is moved to the home position (S202). Although the home position is arbitrary, since it is desirable to drive the entire movable range of the concave mirror 105 in subsequent operations, it is desirable that the upper limit or the lower limit of the movable range. However, in FIG. 2, it is assumed that the home position is at an arbitrary position in the middle of the movable range, and such a form is also possible.

Next, the system displays an identification image corresponding to the current mirror angle (S203). The image may be anything such as an icon or ID number. For example, the numbers as described in FIG. 1 may be used. After the display, the occupant (user) is caused to input information specifying the identification image visually recognized by the occupant (user). As an input method, identification information that looks like “No. 2” can be uttered and input to the system by voice recognition. Alternatively, the user's gesture may be recognized by an in-vehicle camera. Or you may make it input from input means like a touch panel. Or you may make it operate so that a user may push a button at the timing when the recognition image was seen. This method requires a complicated operation for the user, but has an advantage that it is not necessary to change the identification image.

In order to allow the user to perform a predetermined operation as described above, it is desirable to give an instruction such as “Please enter the number that appears on the windshield” by voice guidance or the like.

After the angle change, the presence / absence of user operation is detected (S206).

If there has been a user operation during the setting operation so far, it is determined whether or not a mirror angle designation has been received by the user (S207). That is, when the user inputs identification information that looks like “No. 2”, the system specifies that the mirror angle at the time when the “No. 2” identification information was displayed, that is, the projection position of the virtual image is specified. Can be recognized.

If the mirror angle is not specified by the user, the system changes the angle of the concave mirror 105 (S204). The amount of angle change is arbitrary, but if the amount of change is small, a setting suitable for the sitting height and posture of the occupant is possible, but the setting takes time. The opposite is true if the amount of change is increased.

After changing the mirror angle, determine whether the mirror angle has reached the upper limit or lower limit of the movable range (S205), display the image corresponding to the mirror angle until the mirror angle reaches the upper limit or lower limit, and change the mirror angle , And the presence / absence detection of the user operation is continued (S203 to S206).

If the mirror angle is designated in process S207, the system sets the designated mirror angle as a fixed position for the user (S208). Further, if necessary, distortion correction corresponding to the mirror angle is set (S209). The distortion correction will be described later. Thereafter, a normal HUD operation is executed at the mirror angle (S210).

Also, in the determination of whether the angle has reached the upper limit or lower limit of the movable range (S205), if the upper limit or lower limit is reached, the angle change direction of the concave mirror 105 is reversed (S211). Then, the identification image is displayed (S203) over the entire movable range of the mirror.

If the mirror angle designation by the user cannot be recognized even after displaying the identification image over the entire movable range, the change (increase / decrease) of the mirror angle is continued by the determination process of S205. It is also possible to return to S201 and start the operation from the beginning, issue a warning such as “Could not be recognized”, or change the mirror angle to the home position and perform the HUD operation.

FIG. 3 is an overall configuration diagram of the system of this embodiment. This is an image of HUD mounted on a car. The same components as those in FIG. The HUD 301 includes a folding mirror 104 and a concave mirror 105, for example. Although the configuration of the HUD 301 itself may be the same as a known system configuration such as Patent Document 1, in this embodiment, the projection position of the virtual image 107 can be controlled under the control of the information processing apparatus 302.

The information processing apparatus 302 can be configured as a microcomputer including an input device 303, an output device 304, a processing device 305, and a storage device 306. The microcomputer may be dedicated to HUD control, or may be shared with a car stereo or engine control. The input device 303 may include a known voice input device, a keyboard, and a touch panel (not shown). The output device 304 includes an interface for controlling the HUD 301. The control content includes an image signal of an image displayed on the HUD and a control signal for adjusting the angle of the concave mirror 105. The output device 304 may include a voice output interface that gives instructions to the occupant by voice and a display device other than the HUD.

FIG. 4 is a block diagram illustrating in detail the part of the configuration shown in FIG. In the example of FIG. 4, the control function of the HUD is such that the program stored in the storage device 306 of the information processing device 302 is executed by the processing device 305 so that the determined processing cooperates with other hardware. Realized. A program executed by the processing device 305 or the like or means for realizing the function may be referred to as “function”, “means”, “unit”, “module”, or the like.

The program provided in the storage device 306 includes a viewpoint position detection module 401, a voice recognition module 402 that constitutes a part of the viewpoint position detection module 401, a video processing module 403, a distortion correction module 404, and a mirror control module 405. The input device 303 includes a microphone 407. The HUD 301 includes a video display unit 406 and a mirror unit 407.

In the example of FIG. 4, after starting the initial setting, the viewpoint position detection module 401 instructs the video processing module 403 and the mirror control module 405 to drive the concave mirror 105 while the video processing module 403 performs the concave operation. A display (number 1, number 2, number 3, etc.) corresponding to the position of the mirror is output. When the number at the optimal display position for the passenger is input to the microphone 407 by voice, the voice recognition module 402 recognizes the input number. The mirror control unit 405 adjusts the display position to the mirror angle corresponding to the recognized number, and the initial setting is completed.

In this example, the mirror control unit 405 adjusts the angle of the concave mirror 105, but other optical systems such as lenses and prisms may be driven as long as the position of the virtual image can be changed. The video display unit 406 includes a light source, a liquid crystal panel for displaying video, and the like. However, since a known one may be used, details thereof are omitted. The distortion correction module 404 will be described later.

FIG. 5 is a table showing the contents of data stored in the storage device 306. In accordance with the angle 501 of the concave mirror 105, the identification image 502 to be projected in the initial setting process S203 is stored. Also, distortion correction information 503 is stored according to the angle 501 of the concave mirror 105. The distortion correction information will be described later. Further, the user ID 504 is stored according to the angle 501 of the concave mirror 105. In the initial setting of FIG. 2, when the mirror angle is specified by the user (S207), it is convenient to use the same setting for the same user without performing the initial setting again. Therefore, the ID of the user is registered in the process of setting the mirror to the designated angle (S208). For registration, the user may input his / her ID from the input device 303 such as a touch panel. Alternatively, an ID associated with the engine key may be automatically stored. The HP of the user ID means a home position and indicates an angle set for a user who has not registered ID.

As the storage device 306, a magnetic disk device or a non-volatile semiconductor memory can be used.

Example 2 will be described with reference to FIG. The same components as those in FIG. In this example, after starting the initial setting, a specific image is output while driving the mirror 105. When the optimal display position for the user is reached, the hard switch 601 that is a part of the input device 303 is pressed. The switch input can be detected by the switch detection module 602, the mirror control can be stopped at that timing, and the display position can be stopped. The switch 601 may be a push switch or a tact switch, or may be detected by a pedal operation or the like.

In the example of FIG. 5, the video processing module 403 does not need to change the video depending on the mirror angle, but the mirror control is stopped at the timing of the switch 601, so there is a problem that imposes a burden on the user.

FIG. 7 shows the overall processing flow of the third embodiment. In the third embodiment, an example using image recognition will be described. The same components as those in FIG. In this example, the displayed image may not be changed depending on the mirror angle (S710).

Here, instead of detecting a predetermined operation from the user, an image in the vehicle (for example, the driver's seat) is acquired (S701). For this purpose, an in-vehicle camera for taking an image inside the vehicle is installed, for example, at the position of the room mirror.

Detect the pupil of the user (driver) from the acquired image (S702). In order to detect the pupil part, a known face authentication technique may be applied to cut out the eye part from the face image. In order to detect the pupil part, a voice like “Start the initial setting of the HUD. Stop the vehicle in a place where it is not directly exposed to sunlight. Do not move your head when looking at the front.” It is also desirable to provide guidance by

After detecting the pupil, it is determined whether or not the user's pupil has changed (S703). For this purpose, for example, it can be determined by cutting out the user's eye part from the acquired image, detecting the pupil and iris part, and determining the ratio of the diameters of both.

When the light from the HUD is incident on the user's eye, the user's pupil is narrowed by the brightness of the incident light as a physiological reaction. For example, when the pupil diameter is smaller than half of the iris diameter, it can be determined that light from the HUD has entered the user's eye (that is, the image has been visually recognized). Therefore, the mirror angle is adjusted to the angle at which the user's pupil is narrowed down (S704), the distortion correction corresponding to the mirror angle is set (S209), and then the operation shifts to the HUD normal operation (S210).

FIG. 8 is an overall processing flow of the fourth embodiment. In the fourth embodiment, another example using image recognition will be described. The same configurations as those in FIG. 2 and FIG. In this example, the displayed image may not be changed depending on the mirror angle (S710).

In the example of FIG. 8, a change in the pupil of the user is detected (S710), and the timing for opening the pupil is detected (S803). That is, when the mirror angle is changed (S204) and the image is displayed, the user visually recognizes the image at a certain timing. The pupil contracts when visually recognized, but if the mirror angle is further changed, the image cannot be visually recognized (light does not enter the eye), so the pupil opens. This timing is detected in process S803.

When the pupil is opened, external light is detected in order to determine whether or not it is the influence of external light (S804). If the external light is dark (S805), the pupil may have opened due to the influence of the external light, so that the result is ignored as an erroneous determination, and the mirror angle change is continued (S204).

If the external light is not dark (S805), the mirror angle is returned by one step so that the image can be seen (S806). At this angle, the HUD normal operation is performed (S210).

As in the third and fourth embodiments, in the method of detecting the appropriate position of the mirror by the change of the pupil, the user's pupil is most closed because it looks brightest at the proper position of the mirror. The setting of the brightness of the image is arbitrary, but it is preferable that the screen is brightened in order to increase the pupil change. On the other hand, if it is too bright, the user may feel dazzled, so it is desirable to set an upper limit. In order to avoid the influence of outside light and pay attention to safety during the initial setting, voice guidance is given as "Please stop the car in a place where it is not directly exposed to sunlight" and the side brake is pulled. You may comprise so that initial setting may not be performed if it is not in a certain state.

FIG. 9 is a block diagram for executing the processing of FIG. The same components as those in FIG. In the example of FIG. 9, the input device 303 includes a viewpoint measurement camera 901 and an external light detection sensor 902 for acquiring a user image in step S701. An image acquired by the viewpoint measurement camera 901 is input to the pupil detection unit 903 of the viewpoint position detection unit 401. The pupil detection unit 903 performs the processing of S702 and S803. In the video processing module 403, it is not necessary to change the video depending on the mirror angle, but in order to make it easier to detect a pupil change, a process of increasing the brightness of the image at the initial setting than at the normal time may be performed. .

Using the external light measured by the external light detection sensor 902, the external light detection unit 904 performs the processes S804 and S805.

In the fifth embodiment, the distortion correction processing S209 of FIGS. 2, 7, and 8 and the distortion correction modules of FIGS. 4, 6, and 9 will be described. In the first to fourth embodiments, the above distortion correction is not necessarily required, but is an optional position.

FIG. 10 is a conceptual diagram for explaining the purpose of distortion correction, and describes the positional relationship between the user's eye 106 and the virtual image 107 and how the virtual image is seen by the user. As shown in FIG. 10, when the position (viewpoint) of the eye 106 moves up and down with respect to the virtual image 107, the virtual image appearance 1001 is distorted.

Such a phenomenon is caused by changing the shape of the image by changing the optical path from the light source to the eye by changing the mirror angle (S204) in the first to fourth embodiments. In particular, when the HUD forms an image using the reflection of the windshield 102, the windshield of the automobile has a complicated curved surface shape, and the shape varies depending on the vehicle type. For this reason, by changing the mirror angle, the incident position and angle of light on the windshield 102 change in a complicated manner. Therefore, the appearance of the image is different (that is, distorted) due to the change in the mirror angle. Japanese Patent Application Laid-Open No. 2004-228867 discloses a technique for previously distorting an image to be projected so as to cancel such image distortion.

In the example of Example 1 (FIGS. 2 to 5), the position of the light source, virtual image, and eye is uniquely determined by the process of setting the mirror to a specified angle (S208). In this positional relationship, the projected image is distorted in advance so that an image without distortion can be visually recognized. How to distort the original image can be determined experimentally with a real machine. For example, a process of setting the mirror to a specified angle (S208) is performed at a plurality of viewpoints, and a perfect circle image is projected at that position. If the image looks distorted, the original image is converted to look like a perfect circle.

The conversion rule (for example, a function) is stored as distortion correction information 503 for each mirror angle as shown in FIG. Alternatively, the image itself may be stored.

In this embodiment, functions equivalent to those configured by software can be realized by hardware such as FPGA (Field Programmable Gate Array) and ASIC (Application Specific Integrated Circuit). Such an embodiment is also included in the scope of the present invention.

The present invention is not limited to the above-described embodiment, and includes various modifications. For example, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Further, it is possible to add, delete, and replace the configurations of other embodiments with respect to a part of the configurations of the embodiments.

It can be used for HUD, which is used for displaying information such as automobiles.

Driver (or occupant) 101, windshield 102, light 103 for projecting image information, folding mirror 104, concave mirror 105
Eye 106, virtual image 107

Claims

An image display device that emits light for projecting an image and forms a virtual image to display the image,
A control module for setting the angle at which the light is emitted to a plurality of conditions;
A viewpoint position detection module for selecting one of the plurality of conditions;
With
Emitting the light according to selected conditions to display the image;
Image display device.
A video processing module;
The video processing module includes:
Displaying different images for each of the plurality of conditions,
The viewpoint position detection module includes:
By selecting a specific one of the different images by the user, a condition corresponding to the specified image is selected.
2. The image display device according to claim 1.
The viewpoint position detection module includes:
By selecting a timing set to a specific one of the plurality of conditions by the user, a condition corresponding to the timing is selected.
2. The image display device according to claim 1.
A camera for acquiring an image of the user's eye;
The viewpoint position detection module includes:
By detecting the timing at which the user's pupil contracts from the image of the user's eye, a condition corresponding to the timing is selected.
2. The image display device according to claim 1.
A camera for acquiring an image of the user's eye;
The viewpoint position detection module includes:
By detecting the timing at which the user's pupil is enlarged from the image of the user's eye, the condition immediately before the condition corresponding to the timing is selected.
2. The image display device according to claim 1.
It further includes a sensor that detects the state of external light,
The viewpoint position detection module includes:
Disable the selection based on external light conditions;
The image display device according to claim 5.
The control module is
By controlling the angle of the mirror that reflects the light, the angle at which the light is emitted is set to a plurality of conditions.
The image display device according to claim 1.
A distortion correction module;
The distortion correction module includes:
Corresponding to the condition of the angle at which the light is emitted, the distortion correction setting of the image is changed.
The image display device according to claim 7.
The image distortion correction setting is
It is a conversion rule for converting the original image conveyed by the light to be projected.
The image display device according to claim 8.