JP2018041011A - Display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display device that enables a user to arrange information to be added and improve recognition of information.SOLUTION: A head-mounted display device 100 causes a user to visually recognize a display image including predetermined information overlapped on a landscape. A control part 10 virtually sets a plurality of display zones having different positions in the depth direction in front of the user, specifies the display zone gazed by the user on the basis of gazing position information obtained from a gazing position detection part 20, acquires image information indicating information associated with the specified display zone from a cloud server 200 via a communication part 60, drives an imaging position adjustment part 80 so as to match the depth position of the specified display zone with the imaging position of the information associated with at least the specified display zone in the display image, and displays the display image on a display unit 70 according to the acquired image information.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a display device that allows a user to visually recognize a display image including predetermined information superimposed on a landscape.

  2. Description of the Related Art Conventionally, a transmissive head mounted display (HMD) device that allows a user to visually recognize a virtual image over a landscape is known (see, for example, Patent Document 1). The HMD device is a display device that is worn on the head, projects display light from a display device to a user using a projection optical system or a light guide, and the like, and displays a virtual image of a display image indicated by the display light to the user. It is to be visually recognized. The user can visually recognize the virtual image obtained by the HMD device by superimposing a front real scene. Further, a technique called augmented reality (AR) can be applied to the HMD device. That is, by displaying an AR image related to a real landscape as a display image, information can be added to the real landscape.

  Further, as HMD devices, Patent Documents 2 and 3 disclose techniques capable of changing the imaging position of a virtual image visually recognized by a user. The technique disclosed in Patent Document 2 changes the imaging position of a virtual image by mechanically switching the distance between the display panel and the magnifying optical member by a virtual image distance control means. In addition, the technique disclosed in Patent Document 3 changes the imaging position of a virtual image by image processing by moving an emission region of an image formed by display light from a pair of display driving units.

JP 2014-119786 A Japanese Patent Laid-Open No. 10-262165 JP-A-2015-32131

  As described above, a transmissive HMD device can provide information superimposed on a real scene by a virtual image. However, since a virtual image is always visible in the user's field of view, much information is visually recognized. In this case, there is a problem that it is troublesome for the user, and the user cannot organize the information, and the recognizability of each information is lowered.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a display device capable of organizing information added by a user and improving information recognizability.

A display device of the present invention is a display device that allows a user to visually recognize a display image including predetermined information superimposed on a landscape in order to solve the above-described problem,
A gaze position detection unit for detecting a gaze position of the user;
A display that emits display light indicating the display image;
A communication unit capable of communicating with an external server;
A control unit that causes the display to display the display image in accordance with image information indicating the information acquired from the external server via the communication unit;
An imaging position adjustment unit capable of adjusting an imaging position of at least a part of the display image visually recognized by the user by the display light,
The control unit virtually sets a plurality of display zones with different positions in the depth direction in front of the user, and the display zone in which the user is gazing based on gaze position information from the gaze position detection unit Image information indicating information associated with the display zone identified from the external server via the communication unit is acquired, and at least the depth position of the identified display zone and the display image are identified. Driving the imaging position adjustment unit so that the imaging position of the information associated with the display zone matches, and displaying the display image according to the image information acquired on the display,
In the external server, information related to the warning state is stored in association with the display zone located on the most front side,
Furthermore, the control unit determines the occurrence of the warning state, and when it is determined that the warning state has occurred, the warning state is transmitted from the external server via the communication unit regardless of the determined display zone. The image information indicating the information related to the information is acquired, and the display image is displayed according to the image information indicating the information related to the acquired warning state on the display device, and then the gaze position is detected from the gaze position detection unit. Information is obtained to determine whether the information related to the warning state has been visually recognized in the use, and when it is determined that the information regarding the warning state is not visually recognized by the user, The imaging position adjustment unit is driven so that at least the imaging position of the information related to the warning state coincides with the depth position of the display zone located closest to the front side. To,
It is characterized by that.

  According to the present invention, it is possible to organize information added by a user and improve information recognizability.

It is a figure which shows the general view of the head mounted display apparatus which is 1st embodiment of this invention. It is a block diagram which shows the electrical constitution of a head mounted display apparatus same as the above. It is a figure which shows the state which the user with which the head mounted display apparatus same as the above boarded. It is a figure which shows an example of the information linked | related with the display zone and display zone which are virtually set in a head mounted display apparatus same as the above. It is a flowchart figure which shows the display process in a head mounted display apparatus same as the above. It is a figure which shows the example of a display of a head mounted display apparatus same as the above. It is a figure which shows the other example of a display of a head mounted display apparatus same as the above. It is a flowchart figure which shows the warning display process in a head mounted display apparatus same as the above. It is a figure which shows the example of a display by the warning display process of a head mounted display apparatus same as the above.

  Hereinafter, embodiments according to the present invention will be described with reference to the drawings. In addition, this invention is not limited by the following embodiment (The content of drawing is also included). It goes without saying that changes (including deletion of components) can be added to the following embodiments. Moreover, in the following description, in order to make an understanding of this invention easy, description of a known technical matter is abbreviate | omitted suitably.

FIG. 1 is a diagram illustrating an overview of a head mounted display device (hereinafter also referred to as an HMD device) 100 according to the first embodiment of the present invention, and FIG. 2 is a diagram illustrating an electrical configuration of the HMD device 100. .
The HMD device 100 includes a control unit 10, a gaze position detection unit 20, an imaging unit 30, a position information acquisition unit 40, a state information acquisition unit 50, a communication unit 60, a display unit 70, and an imaging position adjustment. Unit 80.
The HMD device 100 is a glasses-type display device worn on the user U's head. The HMD device 100 is connected to the cloud server (external server) 200 by wireless communication, and constitutes a display system including the HMD device 100 and the cloud server 200. The cloud server 200 stores biological information such as personal information and physical information of the user U, map information, and various types of information associated with first, second, and third display zones Z1, Z2, and Z3 described later. The image information shown is accumulated by network communication. The HMD device 100 receives arbitrary image information from the cloud server 200 by display processing to be described later, displays a display image indicating various information, and displays display light indicating the display image via a lens or an optical member (not shown). The light is incident on the eyes of the user U, and the display image (virtual image) is visually recognized by the user U so as to overlap the scenery in front.

  The control unit 10 is composed of, for example, a microcomputer, and a CPU (Central Processing Unit) that actually performs processing (such as control of the entire HMD device 100) executed by the control unit 10 and a RAM (Random Access) that functions as a main memory of the CPU. Memory), ROM (Read Only Memory) for storing various programs for causing the control unit 10 to execute processing described later, and information (signals) input / output to / from the control unit 10 are digitally converted for output to the CPU And various converters that perform analog conversion. In addition to the CPU, the control unit 10 may include various dedicated circuits (for example, an image processing circuit) for executing part of the processing performed by the control unit 10 instead of the CPU.

As shown in FIG. 1, the control unit 10 virtually sets first, second, and third display zones Z1, Z2, and Z3 having different depth directions in front of the user U.
The first display zone Z1 is a display zone located between the second display zone Z2 and the third display zone Z3. The second display zone Z2 is a display zone located closer to the depth side (most depth side) than the first display zone Z1. The third display zone Z3 is a display zone located on the user U side (most front side) with respect to the first display zone Z1.
The depth positions of the display zones Z1, Z2, and Z3 are arbitrarily set. As shown in FIG. 3, the user U sits in the driver's seat of the vehicle C and looks at the front of the vehicle C. For example, it can be set as follows. That is, in the third display zone Z3, the vicinity of the steering wheel C1 located between the user U (driver) and the instrument panel C2 is set as the depth position. In the first display zone Z1, the depth position is near the instrument panel C2 on the depth side of the steering C1. The second display zone Z2 has a depth position in the vicinity of a landscape 15 m away from the user U that can be viewed from the windshield, on the depth side of the instrument panel C2. In addition, the above description demonstrated the example of the depth position of 1st, 2nd, 3rd display zone Z1, Z2, Z3, Comprising: 1st, 2nd, 3rd display zone Z1, Z2 , Z3 is not set only in a portion overlapping with each part of the vehicle, but is set at a point where the face of the user U is facing.

  As described above, the cloud server 200 stores image information indicating information associated with the first, second, and third display zones Z1, Z2, and Z3. FIG. 4 shows an example of information associated with the display zones Z1, Z2, and Z3. In the first display zone Z1, the second display zone Z2, and the third display zone Z3, the information having the highest priority in the order of the third display zone Z3, the first display zone Z1, and the second display zone Z2. Associated. That is, with the first display zone Z1 as a reference, information with lower priority is associated with the second display zone Z2, and information with higher priority is associated with the third display zone Z3. The first display zone Z1 has the second highest priority (priority “medium”), for example, the state of the user U (heart rate, pulse, brain wave, etc.) and the vehicle C (use) Information (first information) related to the user U and the mobile object moving with the user, such as a state (traveling speed, engine speed, remaining fuel, etc.) of the moving object moving with the user U is associated. In the second display zone Z2, information with the lowest priority (priority level “low”) is superimposed with a predetermined positional relationship with information such as surrounding store information, route guidance information, and near miss information. Is associated with AR information (second information). The third display zone Z3 has the highest priority (priority “high”), for example, information related to a warning state with a high degree of urgency such as warning information such as a physical condition of the user U or a failure of the vehicle C (third Information).

  The gaze position detection unit 20 detects the gaze position of the user U by gaze detection or eye axis measurement. In the case of gaze detection, the eyes of the user U are imaged with a CCD camera or the like, and the gaze direction of the user U is detected as a gaze position by pattern matching processing of image processing technology. In the case of the measurement of the axial axis, the user's U's eyeball is irradiated with ultrasonic waves, and the distance to the front lens surface, the rear surface, and the front surface of the retina is calculated from the echo propagation time. The length of the lens including the axial length of the lens and vitreous tissue from the front of the cornea to the front of the retina) and the lens thickness are measured, and the focus position of the user U is detected as the gaze position from the measurement result. The gaze position detection unit 20 transmits gaze position information of the user U to the control unit 10.

  The imaging unit 30 is formed of a CCD camera, for example, and images a landscape in front of the user U. The imaging unit 30 transmits imaging information (landscape image information) obtained by imaging a landscape in front of the user U to the control unit 10.

  The position information acquisition unit 40 includes a GPS antenna, for example, and acquires the current position of the user U. The position information acquisition unit 40 transmits position information indicating the current position of the user U to the control unit 10.

  The state information acquisition unit 50 is connected to various sensors that detect the state of the user U and a moving body (vehicle C) that moves together with the user U, and the user state information indicating the state of the user U and the state of the moving body The moving body state information shown is acquired and transmitted to the control unit 10. In addition, as a state of the user U, a heart rate, a pulse, a brain wave etc. are mentioned, for example. Further, when the moving body is the vehicle C, examples of the state of the moving body include travel speed, engine speed, remaining fuel, travel distance, and the like.

  The communication unit 60 includes a transmission unit and a reception unit that perform wireless communication with the cloud server 200, such as gaze position information from the control unit 10, gaze display zone information, imaging information, position information, and state information described below. Various information is transmitted to the cloud server 200, image information is received from the cloud server 200, and is transmitted to the control unit 10.

  The display device 70 is composed of, for example, a TFT liquid crystal display device or an organic EL display device, displays a predetermined display image, and emits display light representing the display image.

  The imaging position adjustment unit 80 can change the imaging position of at least a part of the display image visually recognized by the user U. Specifically, as an example, in order to change the optical path length of the display light from the display unit 70 to the user U, the display unit 70 can be moved to change the distance between the display unit 70 and a lens (not shown). It consists of a motor that performs. Further, when an optical member is provided between the display 70 and the lens, the position of the optical member may be changed. Further, the image formation position may be adjusted by performing image processing, such as adjusting the position of the image incident on the left and right eyes in order to change the image formation position according to the left and right parallax, and an arbitrary focus. The image may be displayed so as to be blurred so that the image is clearly visible at a distance. In this case, the control unit 10 may also serve as the imaging position adjustment unit 80. Therefore, the adjustment and change of the imaging position includes the case based on the illusion of the user U.

  Next, normal display processing based on the gaze position of the HMD device 100 will be described with reference to FIG.

  First, in step S <b> 101, the control unit 10 acquires gaze position information indicating the gaze position of the user U from the gaze position detection unit 20. Note that the acquisition of information includes the case of calculating based on the received information in addition to the case of receiving.

  Next, in step S102, the control unit 10 gazes the user U from the first, second, and third display zones Z1, Z2, and Z3 based on the gaze position information acquired in step S101. A display zone (hereinafter also referred to as a gaze display zone) is identified. Specifically, the control unit 10 specifies a display zone whose depth position is closest to the gaze position as the gaze display zone. A case where it is determined that there is no gaze display zone based on the gaze position information may be included.

  In step S <b> 103, the control unit 10 acquires imaging information obtained by imaging a landscape in front of the user U from the imaging unit 30.

  In step S <b> 104, the control unit 10 acquires position information indicating the current position of the user U from the position information acquisition unit 40.

  Further, in step S105, the control unit 10 receives the user state information and the mobile body state information (hereinafter collectively referred to as state information) indicating the state of the mobile body moving together with the user U and the user U from the state information acquisition unit 50. Say).

  Next, in step S106, the control unit 10 determines whether or not a new warning state has occurred based on the gaze position information, imaging information, and state information acquired in steps S101 and S103 to S105. Note that the warning state in the present embodiment means a state in which the content indicated by the highly urgent information displayed in the third display zone Z3 has occurred. In addition to the information described above, the control unit 10 may receive information indicating the occurrence of a warning state from the outside and determine whether or not a new warning state has occurred. If it is determined in step S106 that a new warning condition has not occurred (step S106; No), the control unit 10 executes the processing from step S107 onward, and if it is determined that a new warning condition has occurred ( In step S106; Yes), the control unit executes warning display processing in step S111. The warning display process will be described in detail later.

In step S107, the control unit 10 cloudes the gaze display zone information indicating the gaze display zone determined in step S102, the gaze position information, the imaging information, and the state information acquired in steps S101 and S103 to S105 via the communication unit 60. Transmit to server 200.
The cloud server 200 that has received these pieces of information from the HMD device 100 (the communication unit 60 thereof) is associated with the display zone specified from the image information stored in itself based on the received gaze display zone information and various types of information. The image information indicating the acquired information is extracted. Specifically, when the gaze display zone is the first display zone Z1, the cloud server 200 sets the type of information to be extracted as “information about the user U and the moving object” associated with the first display zone Z1. Furthermore, image information to be displayed based on the state information (for example, numbers, characters, designs, etc. representing the value and degree of the state information) is extracted. When the gaze display zone is the second display zone Z2, the cloud server 200 sets the type of information to be extracted as “AR information superimposed on the landscape” associated with the second display zone Z2, and further Image information to be displayed based on the position information, the imaging information, and the position information (for example, store information around the gaze position of the user U, a route guidance mark superimposed on the road, and near-miss information around the current position) is extracted. When the gaze display zone is the third display zone Z3, the cloud server 200 sets the type of information to be extracted as “information related to a warning state with a high degree of urgency” associated with the third display zone Z3. Then, image information to be displayed based on the state information or the like (for example, characters or symbols for notifying the user U of poor physical condition, moving body failure or malfunction) is extracted. The cloud server 200 transmits the extracted image information to the HMD device 100.

  Next, in step S <b> 108, the control unit 10 receives the image information extracted by the cloud server 200 via the communication unit 60. Note that the HMD device 100 previously receives a part of image information indicating information associated with each of the display zones Z1 to Z3 from the cloud server 200, and stores the image information in an internal storage unit (not shown), in step S102. Based on the gaze display zone information indicating the gaze display zone determined in step S1, and the gaze position information, the imaging information, and the state information acquired in steps S101 and S103 to S105, the information is displayed from the storage unit in relation to the gaze display zone. Image information to be read may be read out.

  Next, in step S109, the control unit 10 drives the imaging position adjustment unit 80 so that the depth position of the gaze display zone specified in step S102 matches the imaging position of the display image. If the image formation position can be adjusted for a part of the display image, the image formation position is set only for information associated with the determined gaze display zone (part corresponding to the received image information) in the display image. You may adjust.

  Next, in step S110, the control unit 10 causes the display device 70 to display a display image according to the image information received from the cloud server 200 in step S107 (update the display image).

  By repeatedly executing the above processing, the HMD device 100 can cause the user U to visually recognize information related to the gaze position as a virtual image according to the gaze position of the user U. As a result, the information related to the position where the user U is gazing is automatically switched and displayed, so that the information added by the user U can be easily organized and the information recognizability can be improved. it can.

  FIG. 6 shows a display example of the HMD device 100 when the user U is driving the vehicle C. In addition, in FIG. 6, the virtual image of the display image visually recognized by the user U is shown. FIG. 6A shows an example in which the gaze display zone is the first display zone Z1. In the example shown in FIG. 6A, a display image V <b> 1 indicating the traveling speed (60 Km / h) of the vehicle C and the pulse (80 bpm) of the user U is displayed on the display unit 70 as information regarding the user U and the moving body. The virtual image of the display image V1 is visually recognized by the user U. FIG. 6B shows an example in which the gaze display zone is the second display zone Z2. In the example shown in FIG. 6B, a display image V2 indicating a route guidance mark (arrow mark) and a parking lot mark (icon including “P”) is displayed on the display unit 70 as AR information superimposed on the landscape. The virtual image of the display image V2 is visually recognized by the user U. FIG. 6C shows an example in which the gaze display zone is the third display zone Z3. In the example shown in FIG. 6C, a display showing a text message (letter “fuel reduction. Please refuel”) that informs the decrease of the remaining fuel (warning state) as information on the warning state with a high degree of urgency. The image V3 is displayed on the display device 70, and the virtual image of the display image V3 is visually recognized by the user U. In addition to the above text message, the information related to the warning state with a high degree of urgency includes a guidance content that guides the user U's line of sight in the direction related to the warning state (such as the direction of an obstacle that is not included in the fuel gauge or view). (A mark such as an arrow).

  FIG. 7 shows another display example of the HMD device 100 when the user U is driving the vehicle C. In another display example, based on the gaze position information acquired in step S101, all of the information included in the display image is information in the vicinity of the gaze position P, all of which is near the gaze position P of the user U. The display image is displayed on the display unit 70 so as to be emphasized with respect to the information located outside. In the present embodiment, the control unit 10, from the gaze position information and the position of the information in the display image displayed on the display 70, at least a part of the gaze area E including the gaze position P of the user U. If it is determined that it is located near the gaze position P of the user U, it is located outside the vicinity of the gaze position P of the user U when it is judged that all are located outside the gaze area E. Judge that. Note that the gaze region E is a circular range that extends in the horizontal direction of the vehicle C (a direction perpendicular to the front-rear direction of the user U) with the gaze position P as the center. In addition, in FIG. 7, the virtual image of the display image visually recognized by the user U is shown. FIG. 7 shows an example in which the gaze display zone is the second display zone Z2. In the example illustrated in FIG. 7, a display image V22 including a route guidance mark (arrow mark) and a parking lot mark (icon including “P”) is displayed on the display unit 70 as AR information to be superimposed on the landscape, and the display image V22 is displayed. Is visually recognized by the user U. Furthermore, among the AR information included in the display image V22, part of the route guidance mark is located inside the gaze area E, and all the parking lot marks are located outside the gaze area E. The mark is emphasized more than the parking lot mark. Here, emphasizing specific information over other information means that the specific information is visually recognized by the user U more clearly than the other information. In addition to making the display brightness higher than other information, changing the display color, etc., lowering the display brightness of other information, changing the display color of other information from color to monochrome (black and white), other information There is a method of relatively emphasizing by lowering specific information that emphasizes the visibility of other information, such as blurring display (shifting the focus). In the other display examples, when the gaze display zones are the first and third display zones Z1 and Z3, the information included in the display image is similarly based on the gaze position information acquired in step S101. Among them, at least a part of the information located in the gaze area E including the gaze position P of the user U, that is, the information located in the vicinity of the gaze position P is all the information located outside the gaze area E of the user U. The display image is displayed on the display unit 70 so as to be emphasized, and the virtual image is visually recognized by the user U.

Next, the warning display process in step S111 will be described with reference to FIG.
First, in step S <b> 121, the control unit 10 transmits warning state occurrence information indicating the occurrence of a warning state, gaze position information, imaging information, and state information acquired in steps S <b> 101 and S <b> 103 to S <b> 105 via the communication unit 60 to the cloud server 200. Send to. At this time, gaze display zone information may also be transmitted.
The cloud server 200 that has received the information from the HMD device 100 (the communication unit 60 thereof) associates the image information stored in itself with the third display zone Z3 based on the received warning state occurrence information and various types of information. Image information indicating the obtained information is extracted. Specifically, the cloud server 200 sets the type of information to be extracted as “information related to a warning state with a high degree of urgency” associated with the third display zone Z3, and image information to be displayed based on the state information and the like. (Image information related to a newly generated warning state) is extracted. The cloud server 200 transmits the extracted image information to the HMD device 100. Moreover, when the cloud server 200 receives the gaze display zone information, the cloud server 200 may also extract image information indicating information associated with the gaze display zone from the image information stored in itself.

  Next, in step S <b> 122, the control unit 10 receives the image information extracted by the cloud server 200 via the communication unit 60. At this time, the image information received from the cloud server 200 includes at least image information indicating “information related to a warning state with a high degree of urgency” associated with the third display zone Z3.

  Next, in step S123, the control unit 10 drives the imaging position adjustment unit 80 so that the depth position of the gaze display zone specified in step S102 matches the imaging position of the display image.

  Next, in step S124, the control unit 10 displays, on the display device 70, the “information related to a warning state with a high degree of urgency” associated with at least the third display zone Z3 received from the cloud server 200 in step S122. A display image is displayed according to the information (the display image is updated). Moreover, the control part 10 starts time measurement with the display of a display image.

  Next, in step S <b> 125, the control unit 10 acquires gaze position information indicating the gaze position of the user U from the gaze position detection unit 20. The gaze position information acquired here is gaze position information after the display image is displayed (updated).

  Next, in step S126, based on the gaze position information acquired in step S125, the control unit 10 determines whether or not the user U has visually recognized information related to a highly urgent warning state in the display image. . When it is determined that the user U visually recognizes information related to a highly urgent warning state among the virtual images of the display image (YES in step S126), the control unit 10 ends the warning display process, and the user U When it is determined that the information regarding the warning state with high urgency is not visually recognized among the virtual images of the display image (NO in step S126), the process proceeds to step S127.

  In step S127, the control unit 10 determines whether or not a predetermined time has elapsed from the display of the display image. If it is determined that the predetermined time has elapsed (YES in step S127), the control unit 10 proceeds to step S127. If it is determined that the predetermined time has not elapsed (NO in step S127), step S125 is performed. Returning to step S125, the processes after step S125 are performed again.

  In step S128, the control unit 10 drives the imaging position adjustment unit 80 so that the depth position of the third display zone Z3 matches the imaging position of the display image. Note that when the image formation position can be adjusted for a part of the display image, the image formation position may be adjusted only for information related to a highly urgent warning state in the display image. Further, the adjustment of the image formation position of the display image may be performed so as to gradually reach the third display zone Z3. For example, when adjusting the imaging position of the display image from the second display zone Z2 to the third display zone Z3, the second display zone Z2 → the first display zone Z1 → the third display zone Z3. It may be adjusted in order, and may further include a step of adjusting the imaging position between the display zones Z1 to Z3.

  By executing the above processing, the HMD device 100 causes the user U to visually recognize high priority information as a virtual image regardless of the user U's gaze position when a new warning state occurs. Can do. Thereby, since newly generated high priority information is automatically displayed at the gaze position of the user U, the information added by the user U can be easily organized and the information recognizability can be improved. Can do. Furthermore, when the user U does not visually recognize high priority information even after a predetermined time has elapsed, the image formation position of the display image is adjusted to a position closer to the user U. High information can be emphasized. When the warning state continues, the control unit 10 repeatedly displays the display image indicating information associated with the gaze display zone of the user U by repeatedly executing the processing from step S101. That is, when the gaze position of the user U is not maintained in the third display zone Z3, the image formation position of the display image moves to another display zone, and the display content of the visually recognized display image has an urgency level. The information regarding the high warning state is switched to the information associated with the gaze display zone of the user U.

  FIG. 9 shows a display example by the warning display process of the HMD device 100 when the user U is driving the vehicle C. In addition, in FIG. 8, the virtual image of the display image visually recognized by the user U is shown. In the first example shown in FIG. 9A, information related to a warning state with a high degree of urgency (notification of a decrease in remaining fuel) is issued instead of displaying information according to the gaze position of the user U by warning display processing. The display image V4 including only the text message (letter “fuel reduction. Please refuel”) is displayed on the display 70, and the virtual image of the display image V4 is visually recognized by the user U. In the second example shown in FIG. 9B, information associated with the gaze display zone of the user U (AR information superimposed on the landscape (route guidance mark (arrow mark) and parking mark ( The icon containing “P”)))) shows a warning message with a high degree of urgency (a text message that informs the reduction of the remaining fuel (letter “Fuel reduction. Please refuel”)) and the user U side. The display image V5 in which the information related to the warning state with high urgency is emphasized more than the information associated with the gaze display zone of the user U is displayed on the display unit 70, and the virtual image of the display image V5 is displayed. It is visually recognized by the user U. As a specific method for emphasizing information, the above-described methods such as making the display brightness higher than other information and changing the display color are used. In the third example shown in FIG. 9C, information associated with the gaze display zone of the user U (AR information superimposed on the landscape (route guidance mark (arrow mark) and parking lot mark ( The icon containing “P”)))) shows a warning message with a high degree of urgency (a text message that informs the reduction of the remaining fuel (letter “Fuel reduction. Please refuel”)) and the user U side. The display image V6 is displayed on the display unit 70, and the display image V6 in which the information associated with the gaze display zone of the user U is lower in visibility than the information on the warning state with a high degree of urgency is displayed on the display unit 70. Is visually recognized by the user U. Specific methods for lowering the visibility of information include lowering its display brightness compared to other information, lowering saturation and / or brightness, blurring the display (shifting the focus), and part (eg, There are methods such as hiding a portion overlapping with information related to a high warning state and a surrounding portion, and reducing the size. Furthermore, as a method for reducing the visibility of information as in the display image V7 in the example shown in FIG. 9D, the information associated with the gaze display zone of the user U is overlapped with the information related to the warning state with a high degree of urgency. It may be moved to a display position that does not become necessary.

The HMD device 100 according to the present embodiment is a display device that allows the user U to visually recognize the display images V1 to V7 and V22 including predetermined information superimposed on the scenery, and detects the gaze position of the user U. Unit 20, display 70 that emits display light indicating display images V <b> 1 to V <b> 7, V <b> 22, communication unit 60 that can communicate with cloud server 200, and the information acquired from cloud server 200 via communication unit 60. The control unit 10 that displays the display images V1 to V7 and V22 on the display 70 according to the image information to be displayed, and the imaging of at least a part of the display images V1 to V7 and V22 that are visually recognized by the user U by the display light. An imaging position adjusting unit 80 capable of adjusting the position,
The control unit 10 virtually sets a plurality of display zones Z1, Z2, and Z3 having different positions in the depth direction in front of the user U, and the user U gazes based on the gaze position information from the gaze position detection unit 20. Image information indicating information associated with the specified display zone is acquired from the cloud server 200 via the communication unit 60, and the depth position of the specified display zone and the display image V1 are acquired. The image forming position adjustment unit 80 is driven so that the image forming position of the information associated with at least the specified display zone out of V3 and V22 coincides, and the display 70 responds to the acquired image information. Display images V1 to V3 and V22.

  According to this, since the information related to the position where the user U is gazing is automatically switched and displayed, the information added by the user U can be easily arranged and the information recognizability is improved. be able to.

Further, in the cloud server 200, information related to a warning state is stored in association with the display zone (third display zone Z3) located closest to the front side,
The control unit 10 determines the occurrence of the warning state, and when it is determined that the warning state has occurred, the control unit 10 transmits the warning state from the cloud server 200 via the communication unit 60 regardless of the determined display zone. Image information indicating information is acquired, and display images V4 to V7 are displayed on the display device 70 according to the image information indicating the acquired information related to the warning state. Thereafter, the gaze position is detected from the gaze position detection unit 20. When the information is acquired to determine whether the information related to the warning state is visually recognized by the user U, and when it is determined that the information related to the warning state is not visually recognized by the user U, the display image V4 Image forming position adjustment unit 8 so that the image forming position of at least the information related to the warning state among V7 coincides with the depth position of the display zone located closest to the front side To drive the.
According to this, it is possible to preferentially display information related to a newly generated warning state, and it is possible to provide optimal information according to the situation. Further, it is possible to further emphasize information related to a newly generated warning state as necessary.

Further, in the HMD device 100, when it is determined that the information related to the warning state is not visually recognized by the user, the control unit 10 forms an imaging position of the information related to at least the warning state among the display images. The image forming position adjusting unit is driven so that the depth gradually coincides with the depth position of the display zone located on the most front side.
According to this, the information regarding the newly generated warning state can be emphasized more naturally as necessary.

Further, in the HMD device 100, when the control unit 10 causes the display device 70 to display the display image V5 according to the image information indicating the information related to the warning state, the information associated with the identified display zone. And the information related to the warning state, and the display image V5 is displayed on the display unit 70 so that the information related to the warning state is emphasized more than the information associated with the specified display zone.
According to this, when information related to a newly generated warning state is preferentially displayed, it is easy for the user U to recognize high-priority information while recognizing the presence of information related to the gaze position of the user U. Can grasp.

Further, in the HMD device 100, the control unit 10 determines that the warning state has occurred, and causes the display 70 to display the display images V6 and V7 according to the image information indicating the information related to the warning state. Including the information associated with the identified display zone and the information regarding the warning state, and the information associated with the identified display zone is less visible than the information regarding the warning state In this way, display images V6 and V7 are displayed on the display unit 70.
According to this, when information related to a newly generated warning state is preferentially displayed, it is easy for the user U to recognize high-priority information while recognizing the presence of information related to the gaze position of the user U. Can grasp.

  In the present embodiment, the HMD device 100 has been described as an example. However, the present invention provides, as a display device, a display that emits display light indicating a display image, and at least one eye of a user on the vehicle. The present invention may be applied to a vehicle display device (non-wearable display device) that is provided so as to cover a field of view and includes a transmissive combiner (display member) that causes the display light to enter the user's eyes. . Further, the present invention may be applied to a non-transmissive display device in which a display device is arranged so as to block a user's field of view as a display device. In the non-transmissive display device, a display image in which an additional image indicating predetermined information is superimposed on a landscape image obtained by capturing a landscape in front of the user is displayed on the display, and is displayed to the user via a lens or an optical member. A display image (real image) is visually recognized. That is, the “landscape” in the present invention may be a landscape image in addition to a real landscape, and the display image visually recognized by the user may be a virtual image or a real image.

  The present invention is suitable for a display device that allows a viewer to visually recognize a display image including predetermined information superimposed on a landscape.

100 Head-mounted display device (HMD device)
DESCRIPTION OF SYMBOLS 10 Control part 20 Gaze position detection part 30 Imaging part 40 Position information acquisition part 50 State information acquisition part 60 Communication part 70 Display device 80 Imaging position adjustment part C Vehicle U User V1-V7, V22 Display image

Claims (4)

A display device that allows a user to visually recognize a display image including predetermined information superimposed on a landscape,
A gaze position detection unit for detecting a gaze position of the user;
A display that emits display light indicating the display image;
A communication unit capable of communicating with an external server;
A control unit that causes the display to display the display image in accordance with image information indicating the information acquired from the external server via the communication unit;
An imaging position adjustment unit capable of adjusting an imaging position of at least a part of the display image visually recognized by the user by the display light,
The control unit virtually sets a plurality of display zones with different positions in the depth direction in front of the user, and the display zone in which the user is gazing based on gaze position information from the gaze position detection unit Image information indicating information associated with the display zone identified from the external server via the communication unit is acquired, and at least the depth position of the identified display zone and the display image are identified. Driving the imaging position adjustment unit so that the imaging position of the information associated with the display zone matches, and displaying the display image according to the image information acquired on the display,
In the external server, information related to the warning state is stored in association with the display zone located on the most front side,
Furthermore, the control unit determines the occurrence of the warning state, and when it is determined that the warning state has occurred, the warning state is transmitted from the external server via the communication unit regardless of the determined display zone. The image information indicating the information related to the information is acquired, and the display image is displayed according to the image information indicating the information related to the acquired warning state on the display device, and then the gaze position is detected from the gaze position detection unit. Information is obtained to determine whether the information related to the warning state has been visually recognized in the use, and when it is determined that the information regarding the warning state is not visually recognized by the user, The imaging position adjustment unit is driven so that at least the imaging position of the information related to the warning state coincides with the depth position of the display zone located closest to the front side. To,
A display device characterized by that. When it is determined that the information regarding the warning state is not visually recognized by the user, the control unit positions at least the imaging position of the information regarding the warning state among the display images on the front side. Driving the imaging position adjustment unit to gradually match the depth position of the display zone;
The display device according to claim 1. The control unit determines that the warning state has occurred, and causes the display to display the display image according to the image information indicating the information related to the warning state, and is associated with the specified display zone. The display image is displayed on the display unit so that the information on the warning state is emphasized more than the information associated with the specified display zone. Let
The display device according to claim 1. The control unit determines that the warning state has occurred, and causes the display to display the display image according to the image information indicating the information related to the warning state, and is associated with the specified display zone. The display image on the display so that the information associated with the identified display zone is less visible than the information regarding the warning state. To display,
The display device according to claim 1.

Images