JP2018041362A - Display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an HMD device that enables a user to arrange information to be added and improve recognition of information.SOLUTION: A head-mounted display device 100 causes a user to visually recognize a display image including predetermined information overlapped on a landscape. A control part 10 virtually sets a plurality of display zones having different positions in the depth direction in front of the user, specifies the display zone gazed by the user on the basis of gazing position information obtained from a gazing position detection part 20, acquires image information indicating information associated with the specified display zone from a cloud server 200 via a communication part 70, drives an imaging position adjustment part 90 so as to match the depth position of the specified display zone with the imaging position of the information associated with at least the specified display zone in the display image, and displays the display image on a display unit 80 according to the acquired image information.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a display device that allows a user to visually recognize a display image including predetermined information superimposed on a landscape.

  2. Description of the Related Art Conventionally, a transmissive head mounted display (HMD) device that allows a user to visually recognize a virtual image over a landscape is known (see, for example, Patent Document 1). The HMD device is a display device that is worn on the head, projects display light from a display device to a user using a projection optical system or a light guide, and the like, and displays a virtual image of a display image indicated by the display light to the user. It is to be visually recognized. The user can visually recognize the virtual image obtained by the HMD device by superimposing a front real scene. Further, a technique called augmented reality (AR) can be applied to the HMD device. That is, by displaying an AR image related to a real landscape as a display image, information can be added to the real landscape.

  Further, as HMD devices, Patent Documents 2 and 3 disclose techniques capable of changing the imaging position of a virtual image visually recognized by a user. The technique disclosed in Patent Document 2 changes the imaging position of a virtual image by mechanically switching the distance between the display panel and the magnifying optical member by a virtual image distance control means. In addition, the technique disclosed in Patent Document 3 changes the imaging position of a virtual image by image processing by moving an emission region of an image formed by display light from a pair of display driving units.

JP 2014-119786 A Japanese Patent Laid-Open No. 10-262165 JP-A-2015-32131

  As described above, a transmissive HMD device can provide information superimposed on a real scene by a virtual image. However, since a virtual image is always visible in the user's field of view, much information is visually recognized. In this case, there is a problem that it is troublesome for the user, and the user cannot organize the information, and the recognizability of each information is lowered.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a display device capable of organizing information added by a user and improving information recognizability.

A display device of the present invention is a display device that allows a user to visually recognize a display image including predetermined information superimposed on a landscape in order to solve the above-described problem,
A gaze position detection unit for detecting a gaze position of the user;
A recognition determination unit that determines whether the user has recognized specific information among the information based on the brain activity of the user;
A display that emits display light indicating the display image;
A communication unit capable of communicating with an external server;
A control unit that causes the display to display the display image in accordance with image information indicating the information acquired from the external server via the communication unit;
An imaging position adjustment unit capable of adjusting an imaging position of at least a part of the display image visually recognized by the user by the display light,
The control unit virtually sets a plurality of display zones having different positions in the depth direction in front of the user, and displays the display zone being watched by the user based on gaze position information from the gaze position detection unit. Specifying and acquiring image information indicating information associated with the display zone specified from the external server via the communication unit, and specifying at least the depth position of the specified display zone and the display image Driving the imaging position adjustment unit to match the imaging position of the information associated with the display zone, and causing the display to display the display image according to the acquired image information; When it is determined that the user has recognized the specific information based on the recognition information from the recognition determination unit after the display image is displayed, the specific of the display image Reducing the visibility of the broadcast,
It is characterized by that.

  According to the present invention, it is possible to organize information added by a user and improve information recognizability.

It is a figure which shows the general view of the head mounted display apparatus which is 1st embodiment of this invention. It is a block diagram which shows the electrical constitution of a head mounted display apparatus same as the above. It is a figure which shows the state which the user with which the head mounted display apparatus same as the above boarded. It is a figure which shows an example of the information linked | related with the display zone and display zone which are virtually set in a head mounted display apparatus same as the above. It is a flowchart which shows the display process in a head mounted display apparatus same as the above. It is a figure which shows the example of a display of a head mounted display apparatus same as the above. It is a figure which shows the other example of a display of a head mounted display apparatus same as the above. It is a flowchart which shows the warning display process in a head mounted display apparatus same as the above. It is a figure which shows the example of a display by the warning display process of a head mounted display apparatus same as the above. It is a flowchart which shows the visibility fall process of a head mounted display apparatus same as the above. It is a figure which shows the example of a display by the visibility fall process of a head mounted display apparatus same as the above.

  Hereinafter, embodiments according to the present invention will be described with reference to the drawings. In addition, this invention is not limited by the following embodiment (The content of drawing is also included). It goes without saying that changes (including deletion of components) can be added to the following embodiments. Moreover, in the following description, in order to make an understanding of this invention easy, description of a known technical matter is abbreviate | omitted suitably.

FIG. 1 is a diagram illustrating an overview of a head mounted display device (hereinafter also referred to as an HMD device) 100 according to the first embodiment of the present invention, and FIG. 2 is a diagram illustrating an electrical configuration of the HMD device 100. .
The HMD device 100 includes a control unit 10, a gaze position detection unit 20, an imaging unit 30, a position information acquisition unit 40, a brain measurement unit 50, a state information acquisition unit 60, a communication unit 70, and a display 80. And an imaging position adjusting unit 90.
The HMD device 100 is a glasses-type display device worn on the user U's head. The HMD device 100 is connected to the cloud server (external server) 200 by wireless communication, and constitutes a display system including the HMD device 100 and the cloud server 200. The cloud server 200 stores biological information such as personal information and physical information of the user U, map information, and various types of information associated with first, second, and third display zones Z1, Z2, and Z3 described later. The image information shown is accumulated by network communication. The HMD device 100 receives arbitrary image information from the cloud server 200 by display processing to be described later, displays a display image indicating various information, and displays display light indicating the display image via a lens or an optical member (not shown). The light is incident on the eyes of the user U, and the display image (virtual image) is visually recognized by the user U so as to overlap the scenery in front.

  The control unit 10 is composed of, for example, a microcomputer, and a CPU (Central Processing Unit) that actually performs processing (such as control of the entire HMD device 100) executed by the control unit 10 and a RAM (Random Access) that functions as a main memory of the CPU. Memory), ROM (Read Only Memory) for storing various programs for causing the control unit 10 to execute processing described later, and information (signals) input / output to / from the control unit 10 are digitally converted for output to the CPU And various converters that perform analog conversion. In addition to the CPU, the control unit 10 may include various dedicated circuits (for example, an image processing circuit) for executing part of the processing performed by the control unit 10 instead of the CPU.

As shown in FIG. 1, the control unit 10 virtually sets first, second, and third display zones Z1, Z2, and Z3 having different depth directions in front of the user U.
The first display zone Z1 is a display zone located between the second display zone Z2 and the third display zone Z3. The second display zone Z2 is a display zone located closer to the depth side (most depth side) than the first display zone Z1. The third display zone Z3 is a display zone located on the user U side (most front side) with respect to the first display zone Z1.
The depth positions of the display zones Z1, Z2, and Z3 are arbitrarily set. As shown in FIG. 3, the user U sits in the driver's seat of the vehicle C and looks at the front of the vehicle C. For example, it can be set as follows. That is, in the third display zone Z3, the vicinity of the steering wheel C1 located between the user U (driver) and the instrument panel C2 is set as the depth position. In the first display zone Z1, the depth position is near the instrument panel C2 on the depth side of the steering C1. The second display zone Z2 has a depth position in the vicinity of a landscape 15 m away from the user U that can be viewed from the windshield, on the depth side of the instrument panel C2. In addition, the above description demonstrated the example of the depth position of 1st, 2nd, 3rd display zone Z1, Z2, Z3, Comprising: 1st, 2nd, 3rd display zone Z1, Z2 , Z3 is not set only in a portion overlapping with each part of the vehicle, but is set at a point where the face of the user U is facing.

  As described above, the cloud server 200 stores image information indicating information associated with the first, second, and third display zones Z1, Z2, and Z3. FIG. 4 shows an example of information associated with the display zones Z1, Z2, and Z3. In the first display zone Z1, the second display zone Z2, and the third display zone Z3, the information having the highest priority in the order of the third display zone Z3, the first display zone Z1, and the second display zone Z2. Associated. That is, with the first display zone Z1 as a reference, information with lower priority is associated with the second display zone Z2, and information with higher priority is associated with the third display zone Z3. The first display zone Z1 has the second highest priority (priority “medium”), for example, the state of the user U (heart rate, pulse, brain wave, etc.) and the vehicle C (use) Information (first information) related to the user U and the mobile object moving with the user, such as a state (traveling speed, engine speed, remaining fuel, etc.) of the moving object moving with the user U is associated. In the second display zone Z2, information with the lowest priority (priority level “low”) is superimposed with a predetermined positional relationship with information such as surrounding store information, route guidance information, and near miss information. Is associated with AR information (second information). The third display zone Z3 has the highest priority (priority “high”), for example, information related to a warning state with a high degree of urgency such as warning information such as a physical condition of the user U or a failure of the vehicle C (third Information).

  The gaze position detection unit 20 detects the gaze position of the user U by gaze detection or eye axis measurement. In the case of gaze detection, the eyes of the user U are imaged with a CCD camera or the like, and the gaze direction of the user U is detected as a gaze position by pattern matching processing of image processing technology. In the case of the measurement of the axial axis, the user's U's eyeball is irradiated with ultrasonic waves, and the distance to the front lens surface, the rear surface, and the front surface of the retina is calculated from the echo propagation time. The length of the lens including the axial length of the lens and vitreous tissue from the front of the cornea to the front of the retina) and the lens thickness are measured, and the focus position of the user U is detected as the gaze position from the measurement result. The gaze position detection unit 20 transmits gaze position information of the user U to the control unit 10.

  The imaging unit 30 is formed of a CCD camera, for example, and images a landscape in front of the user U. The imaging unit 30 transmits imaging information (landscape image information) obtained by imaging a landscape in front of the user U to the control unit 10.

  The position information acquisition unit 40 includes a GPS antenna, for example, and acquires the current position of the user U. The position information acquisition unit 40 transmits position information indicating the current position of the user U to the control unit 10.

  The brain measurement unit 50 has a function of measuring the brain activity (consciousness / intention) of the user U. The brain measurement unit 50 is provided integrally with the glasses of the HMD device 100. The brain measurement unit 50 may be provided separately from the glasses of the HMD device 100 and installed on the headrest of the vehicle C, and connected to the main body of the HMD device 100 by wire or wirelessly. More specifically, the brain measurement unit 50 measures at least the brain waves of the user U, determines whether or not the user U has recognized specific information included in the display image mainly from changes in the brain waves, and measures the measured numerical values. The determination result is transmitted to the control unit 10 as brain measurement information and cognitive information. That is, the brain measurement unit 50 functions as a recognition determination unit. In the present embodiment, it is assumed that the specific information is “information related to a warning state with a high degree of urgency”. The determination of the recognition of the user U is performed, for example, by measuring an electroencephalogram in a state where the user U recognizes an image indicating specific information in advance and storing it as reference data. This is done by judging whether or not it matches or approximates. If it does not match or approximate the reference data, it is judged as “not recognized”. Another possible method is to use a brain wave that is generated transiently when a human recognizes a stimulus such as light or sound. Specifically, specific information in the display image is blinked at a predetermined cycle, or a predetermined icon is blinked at a predetermined cycle near the specific information. When the brain waves are measured from the brain waves of the user U with the same period as the specific information or icon, it can be determined that the user U has recognized the specific information. In addition to the brain wave, the brain measuring unit 50 may measure a cerebral blood flow or a brain electromagnetic wave. In addition, the brain measurement unit 50 performs only brain activity measurement, transmits brain measurement information to the control unit 10 or the cloud server 200 via the communication unit 70, and stores specific information on the control unit 10 or the cloud server 200. Cognitive determination may be made. That is, the control unit 10 or the cloud server 200 may function as a recognition determination unit.

  The state information acquisition unit 60 is connected to various sensors that detect the state of the user U and a moving body (vehicle C) that moves with the user U, and indicates the state of the user U (excluding brain activity). The mobile body state information indicating the information and the state of the mobile body is acquired and transmitted to the control unit 10. In addition, as a state of the user U, a heart rate, a pulse, etc. are mentioned, for example. Further, when the moving body is the vehicle C, examples of the state of the moving body include travel speed, engine speed, remaining fuel, travel distance, and the like.

  The communication unit 70 includes a transmission unit and a reception unit that perform wireless communication with the cloud server 200, and includes gaze position information, brain measurement information, gaze display zone information, imaging information, position information, which will be described later, from the control unit 10. Various information such as status information is transmitted to the cloud server 200, image information is received from the cloud server 200, and is transmitted to the control unit 10.

  The display device 80 includes, for example, a TFT liquid crystal display device or an organic EL display device, displays a predetermined display image, and emits display light representing the display image.

  The imaging position adjustment unit 90 can change the imaging position of at least a part of the display image visually recognized by the user U. Specifically, as an example, in order to change the optical path length of the display light from the display device 80 to the user U, the display device 80 can be moved to change the distance between the display device 80 and a lens (not shown). It consists of a motor that performs. Moreover, when an optical member is provided between the display device 80 and the lens, the position of the optical member may be changed. Further, the image formation position may be adjusted by performing image processing, such as adjusting the position of the image incident on the left and right eyes in order to change the image formation position according to the left and right parallax, and an arbitrary focus. The image may be displayed so as to be blurred so that the image is clearly visible at a distance. In this case, the control unit 10 may also serve as the imaging position adjustment unit 90. Therefore, the adjustment and change of the imaging position includes the case based on the illusion of the user U.

  Next, normal display processing based on the gaze position of the HMD device 100 will be described with reference to FIG.

  First, in step S <b> 101, the control unit 10 acquires gaze position information indicating the gaze position of the user U from the gaze position detection unit 20. Note that the acquisition of information includes the case of calculating based on the received information in addition to the case of receiving.

  Next, in step S <b> 102, the control unit 10 acquires brain measurement information from the brain measurement unit 50.

  Next, in step S103, the control unit 10 displays the display in which the user U is gazing intentionally from the first, second, and third display zones Z1, Z2, and Z3 based on the gaze position information. A zone (hereinafter also referred to as a gaze display zone) is specified. Specifically, the control unit 10 identifies a display zone whose depth position is closest to the gaze position based on the gaze position information as the gaze display zone. In addition, it may include a case where it is determined that there is no gaze display zone based on the gaze position information, and when the gaze display zone cannot be identified based on the gaze position information, the last gaze display zone identified this time is displayed. You may specify a zone.

  In step S <b> 104, the control unit 10 acquires imaging information obtained by imaging the landscape in front of the user U from the imaging unit 30.

  In step S <b> 105, the control unit 10 acquires position information indicating the current position of the user U from the position information acquisition unit 40.

  Further, in step S106, the control unit 10 transmits the user state information and the moving body state information (hereinafter collectively referred to as state information) indicating the state of the moving body moving together with the user U and the user U from the state information acquiring unit 60. Say).

  Next, in step S107, the control unit 10 determines whether or not a new warning state has occurred based on the gaze position information, brain measurement information, imaging information, and state information acquired in steps S101, S102, and S104 to S106. Determine. Note that the warning state in the present embodiment means a state in which the content indicated by the highly urgent information displayed in the third display zone Z3 has occurred. In addition to the information described above, the control unit 10 may receive information indicating the occurrence of a warning state from the outside and determine whether or not a new warning state has occurred. If it is determined in step S107 that a new warning state has not occurred (step S107; No), the control unit 10 executes the processes in and after step S108, and if it is determined that a new warning state has occurred (step S107; No). In step S107; Yes), the control unit executes warning display processing in step S113. The warning display process will be described in detail later.

In step S108, the control unit 10 communicates gaze display zone information indicating the gaze display zone determined in step S103, gaze position information, brain measurement information, imaging information, and state information acquired in steps S101, S102, and S104 to S106. To the cloud server 200 via the unit 70.
The cloud server 200 that has received these pieces of information from the HMD device 100 (the communication unit 70 thereof) is associated with the display zone specified from the image information stored therein based on the received gaze display zone information and various information. The image information indicating the acquired information is extracted. Specifically, when the gaze display zone is the first display zone Z1, the cloud server 200 sets the type of information to be extracted as “information about the user U and the moving object” associated with the first display zone Z1. Furthermore, image information to be displayed based on the state information (for example, numbers, characters, designs, etc. representing the value and degree of the state information) is extracted. When the gaze display zone is the second display zone Z2, the cloud server 200 sets the type of information to be extracted as “AR information superimposed on the landscape” associated with the second display zone Z2, and further Image information to be displayed based on the position information, the imaging information, and the position information (for example, store information around the gaze position of the user U, a route guidance mark superimposed on the road, and near-miss information around the current position) is extracted. When the gaze display zone is the third display zone Z3, the cloud server 200 sets the type of information to be extracted as “information related to a warning state with a high degree of urgency” associated with the third display zone Z3. Then, image information to be displayed based on the state information or the like (for example, characters or symbols for notifying the user U of poor physical condition, moving body failure or malfunction) is extracted. The cloud server 200 transmits the extracted image information to the HMD device 100.

  Next, in step S <b> 109, the control unit 10 receives the image information extracted by the cloud server 200 via the communication unit 70. Note that the HMD device 100 previously receives a part of image information indicating information associated with each of the display zones Z1 to Z3 from the cloud server 200, and stores the image information in an internal storage unit (not shown), in step S103. Based on the gaze display zone information indicating the gaze display zone determined in step S1, the gaze position information acquired in steps S101, S101, and S104 to S106, the brain measurement information, the imaging information, and the state information, the gaze display zone is associated with the gaze display zone. Image information to be displayed may be read out of the received information.

  Next, in step S110, the control unit 10 drives the imaging position adjustment unit 90 so that the depth position of the gaze display zone specified in step S103 matches the imaging position of the virtual image of the display image. If the image formation position can be adjusted for a part of the display image, the image formation position is set only for information associated with the determined gaze display zone (part corresponding to the received image information) in the display image. You may adjust.

  Next, in step S111, the control unit 10 causes the display device 80 to display a display image corresponding to the image information received from the cloud server 200 in step S109 (update the display image).

  After step S111 or step S112 is executed, that is, after the display image is displayed, in step S113, the control unit 10 executes a visibility reduction process based on the recognition determination information from the brain measurement unit 50. Details of the visibility reduction process will be described later.

  By repeatedly executing the above processing, the HMD device 100 can cause the user U to visually recognize information related to the gaze position as a virtual image according to the gaze position of the user U. As a result, the information related to the position where the user U is gazing is automatically switched and displayed, so that the information added by the user U can be easily organized and the information recognizability can be improved. it can.

  FIG. 6 shows a display example of the HMD device 100 when the user U is driving the vehicle C. In addition, in FIG. 6, the virtual image of the display image visually recognized by the user U is shown. FIG. 6A shows an example in which the gaze display zone is the first display zone Z1. In the example shown in FIG. 6A, a display image V1 indicating the traveling speed (60 Km / h) of the vehicle C and the pulse (80 bpm) of the user U is displayed on the display device 80 as information about the user U and the moving body. The virtual image of the display image V1 is visually recognized by the user U. FIG. 6B shows an example in which the gaze display zone is the second display zone Z2. In the example shown in FIG. 6B, a display image V2 indicating a route guidance mark (arrow mark) and a parking lot mark (icon including “P”) is displayed on the display device 80 as AR information superimposed on the landscape. The virtual image of the display image V2 is visually recognized by the user U. FIG. 6C shows an example in which the gaze display zone is the third display zone Z3. In the example shown in FIG. 6C, a display showing a text message (letter “fuel reduction. Please refuel”) that informs the decrease of the remaining fuel (warning state) as information on the warning state with a high degree of urgency. The image V3 is displayed on the display device 80, and a virtual image of the display image V3 is visually recognized by the user U.

  FIG. 7 shows another display example of the HMD device 100 when the user U is driving the vehicle C. In another display example, based on the gaze position information acquired in step S101, all of the information included in the display image is information in the vicinity of the gaze position P, all of which is the gaze position P of the user U. A display image is displayed on the display device 80 so as to be emphasized with respect to information located outside the vicinity. In the present embodiment, the control unit 10, from the gaze position information and the position of the information in the display image displayed on the display device 80, at least a part of the inside of the gaze area E including the gaze position P of the user U. If it is determined that it is located near the gaze position P of the user U, it is located outside the vicinity of the gaze position P of the user U when it is judged that all are located outside the gaze area E. Judge that. Note that the gaze region E is a circular range that extends in the horizontal direction of the vehicle C (a direction perpendicular to the front-rear direction of the user U) with the gaze position P as the center. In addition, in FIG. 7, the virtual image of the display image visually recognized by the user U is shown. FIG. 7 shows an example in which the gaze display zone is the second display zone Z2. In the example shown in FIG. 7, a display image V22 including a route guidance mark (arrow mark) and a parking lot mark (icon including “P”) is displayed on the display device 80 as AR information to be superimposed on the landscape, and the display image V22 is displayed. Is visually recognized by the user U. Furthermore, among the AR information included in the display image V22, part of the route guidance mark is located inside the gaze area E, and all the parking lot marks are located outside the gaze area E. The mark is emphasized more than the parking lot mark. Here, emphasizing specific information over other information means that the specific information is visually recognized by the user U more clearly than the other information. In addition to making the display brightness higher than other information, changing the display color, etc., lowering the display brightness of other information, changing the display color of other information from color to monochrome (black and white), other information There is a method of relatively emphasizing by lowering specific information that emphasizes the visibility of other information, such as blurring display (shifting the focus). In the other display examples, when the gaze display zones are the first and third display zones Z1 and Z3, the information included in the display image is similarly based on the gaze position information acquired in step S101. Among them, at least a part of the information located in the gaze area E including the gaze position P of the user U, that is, the information located in the vicinity of the gaze position P is all the information located outside the gaze area E of the user U. The display image is displayed on the display unit 80 so as to be emphasized, and the virtual image is visually recognized by the user U.

Next, the warning display process in step S112 will be described with reference to FIG.
First, in step S121, the control unit 10 transmits warning state occurrence information indicating the occurrence of a warning state, gaze position information, brain measurement information, imaging information, and state information acquired in steps S101, S102, and S104 to S106 to the communication unit 70. To the cloud server 200 via At this time, gaze display zone information may also be transmitted.
The cloud server 200 that has received these pieces of information from the HMD device 100 (the communication unit 70 thereof) associates the image information stored in itself with the third display zone Z3 based on the received warning state occurrence information and various types of information. Image information indicating the obtained information is extracted. Specifically, the cloud server 200 sets the type of information to be extracted as “information related to a warning state with a high degree of urgency” associated with the third display zone Z3, and image information to be displayed based on the state information and the like. (Image information related to a newly generated warning state) is extracted. The cloud server 200 transmits the extracted image information to the HMD device 100. Moreover, when the cloud server 200 receives the gaze display zone information, the cloud server 200 may also extract image information indicating information associated with the gaze display zone from the image information stored in itself.

  Next, in step S <b> 122, the control unit 10 receives the image information extracted by the cloud server 200 via the communication unit 70. At this time, the image information received from the cloud server 200 is image information indicating “information regarding a warning state with a high degree of urgency” associated with at least the third display zone Z3.

  Next, in step S123, the control unit 10 drives the imaging position adjustment unit 90 so that the depth position of the third display zone Z3 matches the imaging position of the display image.

  Next, in step S124, the control unit 10 displays, on the display device 80, image information indicating “information related to a warning state with a high degree of urgency” associated with the third display zone Z3 received from the cloud server 200 in step S107. A display image corresponding to is displayed (the display image is updated).

  By executing the above processing, the HMD device 100 causes the user U to visually recognize high priority information as a virtual image regardless of the user U's gaze position when a new warning state occurs. Can do. As a result, the newly generated high priority information is automatically displayed at the imaging position close to the user U, so that the information added by the user U can be easily arranged and the information recognizability is improved. can do. When the warning state continues, the control unit 10 repeatedly displays the display image indicating information associated with the gaze display zone of the user U by repeatedly executing the processing from step S101. That is, when the gaze position of the user U is not maintained in the third display zone Z3, the image formation position of the display image moves to another display zone, and the display content of the visually recognized display image has an urgency level. The information regarding the high warning state is switched to the information associated with the gaze display zone of the user U.

  FIG. 9 shows a display example by the warning display process of the HMD device 100 when the user U is driving the vehicle C. In addition, in FIG. 8, the virtual image of the display image visually recognized by the user U is shown. In the first example shown in FIG. 9A, information related to a warning state with a high degree of urgency (notification of a decrease in remaining fuel) is issued instead of displaying information according to the gaze position of the user U by warning display processing. A display image V4 including only a text message (letter “Fuel reduction. Please refuel”) is displayed on the display device 80, and a virtual image of the display image V4 is visually recognized by the user U. In the second example shown in FIG. 9B, display of information associated with the gaze display zone of the user U (AR information superimposed on the landscape (route guidance mark (arrow mark) and parking lot) is performed by warning display processing. Mark (icons including “P”))), a warning message with a high degree of urgency (a text message that informs the reduction of the remaining fuel (a letter “fuel reduction. Please refuel”)) is displayed on the user U. The display image V5 is displayed on the display device 80, and the information related to the warning state with high urgency is emphasized more than the information associated with the gaze display zone of the user U. A virtual image is visually recognized by the user U. As a specific method for emphasizing information, the above-described methods such as making the display brightness higher than other information and changing the display color are used. In the third example shown in FIG. 9C, display of information associated with the gaze display zone of the user U (AR information superimposed on the landscape (route guidance mark (arrow mark) and parking lot) is performed by warning display processing. Mark (icons including “P”))), a warning message with a high degree of urgency (a text message that informs the reduction of the remaining fuel (a letter “fuel reduction. Please refuel”)) is displayed on the user U. The display image V6 is displayed on the display device 80 so that the information associated with the gaze display zone of the user U is lower in visibility than the information on the warning state with a high degree of urgency. A virtual image of the image V6 is visually recognized by the user U. Specific methods for lowering the visibility of information include lowering its display brightness compared to other information, lowering saturation and / or brightness, blurring the display (shifting the focus), and part (eg, There are methods such as hiding a portion overlapping with information related to a high warning state and a surrounding portion, and reducing the size. Furthermore, as a method for reducing the visibility of information as in the display image V7 in the example shown in FIG. 9D, the information associated with the gaze display zone of the user U is overlapped with the information related to the warning state with a high degree of urgency. It may be moved to a display position that does not become necessary.

Next, the visibility reduction process in step S113 will be described with reference to FIG.
First, in step S131, the control unit 10 acquires recognition determination information indicating whether or not the user U has recognized specific information (information on a warning state with a high degree of urgency) from the brain measurement unit 50. The recognition determination information acquired here indicates whether or not the user U has recognized after the display image is displayed in step S111 or step S112.

  Next, in step S132, the control unit 10 determines whether or not the user U has recognized the specific information included in the display image based on the recognition determination information acquired in step S131. If it is determined that the information is recognized (YES in step S132), the process proceeds to step S133. If it is determined that the user U does not recognize the specific information (NO in step S132), the process proceeds to step S134.

  In step S133, the control unit 10 performs processing so that the visibility of the specific information in the display image is lower than that at the start of display, and updates the display image. Here, specific methods for reducing the visibility of specific information include non-display, lowering the display brightness, lowering saturation and / or brightness, blurring the display (shifting the focus), and some There are methods such as hiding (for example, a portion overlapping with other information and a peripheral portion) and reducing the size. Furthermore, as a method of reducing the visibility, the specific information may be moved to a display position that does not overlap with other information. When the display image is displayed by the warning display process in step S112, the image formation position of the display image is changed to the gaze display zone as the visibility of the information related to the warning state with high urgency that is the specific information is reduced. The image position adjustment unit 90 may be driven so as to coincide with the depth position of the image and information associated with the gaze display zone may be displayed on the display image. FIG. 11 shows a display example after the visibility lowering process is performed. In addition, in FIG. 11, the virtual image of the display image which the user U can visually recognize is shown. 11A to 11B correspond to the first example by the warning display process shown in FIG. 9A. In the example shown in FIG. 11 (a), the information about the warning state with a high degree of urgency recognized by the user U (text message to notify the decrease in the remaining fuel ("Fuel reduction. .) Is displayed on the display device 80, and the virtual image of the display image V41 is visually recognized by the user U. In the example shown in FIG. 11 (b), the information about the warning state with a high degree of urgency recognized by the user U (the text message ("Fuel reduction. ")" And the display of information associated with the gaze display zone of the user U (AR information (route guidance mark (arrow mark) and parking lot mark ("P") superimposed on the landscape)). The display image V42 including the icon))) is displayed, and the virtual image of the display image V42 is visually recognized by the user U. In this example, the imaging position adjustment unit 90 may be further driven so that the imaging position of the display image matches the depth position of the gaze display zone. Moreover, as a display image V43 in the example shown in FIG. 11C, as a method of reducing the visibility of information, information on a warning state with a high degree of urgency recognized by the user U is used as a gaze display zone of the user U. You may move to the display position which does not overlap with the information linked | related.

  By executing the above processing, information according to the actual condition of the user U can be provided.

The HMD device 100 according to the present embodiment is a display device that allows the user U to view display images V1 to V7, V22, and V41 to V43 including predetermined information superimposed on the landscape, and detects the gaze position of the user U. Gaze position detection unit 20, display device 80 that emits display light indicating display images V 1 to V 7, V 22, V 41 to V 43, a communication unit 70 that can communicate with the cloud server 200, and a cloud via the communication unit 70. The control unit 10 that displays the display images V1 to V7, V22, and V41 to V43 on the display device 80 according to the image information indicating the information acquired from the server 200, and the display image that is visually recognized by the user U by the display light. An imaging position adjustment unit 90 that can adjust at least a part of the imaging positions of V1 to V7, V22, and V41 to V43, and
The control unit 10 virtually sets a plurality of display zones Z1, Z2, and Z3 having different positions in the depth direction in front of the user U, and the user U gazes based on the gaze position information from the gaze position detection unit 20. Image information indicating information associated with the specified display zone is acquired from the cloud server 200 via the communication unit 70, and the depth position of the specified display zone and the display image V1 are acquired. The imaging position adjustment unit 90 is driven so that the imaging position of the information associated with at least the specified display zone of ~ V3 and V22 coincides, and the display unit 80 responds to the acquired image information. Display images V1 to V3 and V22.

  According to this, since the information related to the position where the user U is gazing is automatically switched and displayed, the information added by the user U can be easily arranged and the information recognizability is improved. be able to.

Further, the HMD device 100 includes a brain measurement unit 50 that determines whether the user U has recognized specific information among the information based on the brain activity of the user U,
When it is determined that the user U has recognized the specific information based on the recognition information from the brain measurement unit 50 after the display images V1 to V7 and V22 are displayed, the control unit 10 displays the display images V1 to V7 and V22. Among them, the visibility of the specific information is reduced.
According to this, since the information which the user U recognized and the display priority fell is not visually recognized or becomes difficult to visually recognize, the information according to the actual condition of the user U can be provided more.

In addition, the HMD device 100 is configured with a first display zone Z1 and a second display zone Z2 located on the depth side of the first display zone Z1 as a plurality of display zones,
In the cloud server 200, first information related to the user U and / or a moving body (vehicle C) moving with the user U in association with the first display zone Z1 is stored, and the second display zone Z2 is stored. Second information to be superimposed on the landscape is stored.
Furthermore, in the HMD device 100, as a plurality of display zones, a third display zone Z3 positioned further on the near side than the first display zone Z1 is set,
In the cloud server 200, the third information related to the warning state is stored in association with the third display zone Z3.
The specific information is the third information.
According to this, since information with different priorities is automatically switched and displayed according to the position where the user U is gazing, the information added by the user U can be easily organized and the information can be recognized. Can be improved. Moreover, information with high priority can be appropriately provided according to the actual state of the user U by using specific information as information with high priority.

Further, in the HMD device 100, when the control unit 10 displays the display image V22 on the display unit 80 according to the acquired image information, the control unit 10 includes the information included in the display image V22 based on the gaze position information. The display image is displayed on the display unit 80 so that at least a part of the information located near the user's U gaze position P is emphasized with respect to the information located outside the gaze position P of the user U. V22 is displayed.
According to this, among the information added by the user U, information closer to the gaze position can be recognized instantly, and the visibility of the information can be improved.

Further, in the HMD device 100, the control unit 10 determines the occurrence of the warning state, and when it is determined that the warning state has occurred, the cloud server via the communication unit 70 is determined regardless of the specified display zone. Image information indicating the third information is acquired from 200, and image formation is performed so that at least the image formation position of the third information in the display images V4 to V7 matches the depth position of the third display zone Z3. The position adjustment unit 90 is driven, and the display images V4 to V7 are displayed according to the image information indicating the acquired third information on the display device 80.
According to this, it is possible to preferentially display information related to a newly generated warning state, and it is possible to provide optimal information according to the situation.

Further, in the HMD device 100, the control unit 10 determines that the warning state has occurred, and when the display image V5 is displayed according to the image information indicating the third information on the display device 80, the control unit 10 identifies The display image is displayed on the display unit 80 so as to include the information associated with the display zone and the third information, and the third information is more emphasized than the information associated with the specified display zone. V5 is displayed.
According to this, when information related to a newly generated warning state is preferentially displayed, it is easy for the user U to recognize high-priority information while recognizing the presence of information related to the gaze position of the user U. Can grasp.

Further, in the HMD device 100, the control unit 10 determines that the warning state has occurred, and specifies the display image V6 or V7 on the display device 80 according to the image information indicating the third information. The display includes the information associated with the display zone and the third information, and the information associated with the identified display zone is less visible than the third information. The display image is displayed on a container.
According to this, when information related to a newly generated warning state is preferentially displayed, it is easy for the user U to recognize high-priority information while recognizing the presence of information related to the gaze position of the user U. Can grasp.

In the present embodiment, the HMD device 100 has been described as an example. However, the present invention provides, as a display device, a display that emits display light indicating a display image, and at least one eye of a user on the vehicle. The present invention may be applied to a vehicle display device (non-wearable display device) that is provided so as to cover a field of view and includes a transmissive combiner (display member) that causes the display light to enter the user's eyes. . Further, the present invention may be applied to a non-transmissive display device in which a display device is arranged so as to block a user's field of view as a display device. In the non-transmissive display device, a display image in which an additional image indicating predetermined information is superimposed on a landscape image obtained by capturing a landscape in front of the user is displayed on the display, and is displayed to the user via a lens or an optical member. A display image (real image) is visually recognized. That is, the “landscape” in the present invention may be a landscape image in addition to a real landscape, and the display image visually recognized by the user may be a virtual image or a real image.
Further, the specific information of the present invention is not limited to information related to a warning state with a high degree of urgency, and arbitrary information can be selected. Information set as specific information by setting by the user U may be changeable.

  The present invention is suitable for a display device that allows a viewer to visually recognize a display image including predetermined information superimposed on a landscape.

100 Head-mounted display device (HMD device)
DESCRIPTION OF SYMBOLS 10 Control part 20 Gaze position detection part 30 Imaging part 40 Position information acquisition part 50 Brain measurement part (cognitive determination part)
60 Status information acquisition unit 70 Communication unit 80 Display 90 Imaging position adjustment unit C Vehicle U User V1 to V7, V22, V41 to V43 Display image

Claims (7)

A display device that allows a user to visually recognize a display image including predetermined information superimposed on a landscape,
A gaze position detection unit for detecting a gaze position of the user;
A recognition determination unit that determines whether the user has recognized specific information among the information based on the brain activity of the user;
A display that emits display light indicating the display image;
A communication unit capable of communicating with an external server;
A control unit that causes the display to display the display image in accordance with image information indicating the information acquired from the external server via the communication unit;
An imaging position adjustment unit capable of adjusting an imaging position of at least a part of the display image visually recognized by the user by the display light,
The control unit virtually sets a plurality of display zones having different positions in the depth direction in front of the user, and displays the display zone being watched by the user based on gaze position information from the gaze position detection unit. Specifying and acquiring image information indicating information associated with the display zone specified from the external server via the communication unit, and specifying at least the depth position of the specified display zone and the display image Driving the imaging position adjustment unit to match the imaging position of the information associated with the display zone, and causing the display to display the display image according to the acquired image information; When it is determined that the user has recognized the specific information based on the recognition information from the recognition determination unit after the display image is displayed, the specific of the display image Reducing the visibility of the broadcast,
A display device characterized by that. As the plurality of display zones, a first display zone and a second display zone located on the depth side of the first display zone are set,
In the external server, first information related to the user and / or a moving body that moves together with the user is stored in association with the first display zone, and is associated with the second display zone. Second information to be superimposed on the landscape is stored.
The display device according to claim 1. As the plurality of display zones, a third display zone located further on the front side than the first display zone is set,
In the external server, third information related to the warning state is stored in association with the third display zone,
The specific information is the third information.
The display device according to claim 2. The control unit determines the occurrence of the warning state, and when it is determined that the warning state has occurred, the third information is transmitted from the external server via the communication unit regardless of the specified display zone. And driving the imaging position adjustment unit so that at least the imaging position of the third information in the display image matches the depth position of the third display zone, Displaying the display image according to image information indicating the third information acquired by the display;
The display device according to claim 3. The control unit determines that the warning state has occurred, and displays the display image in accordance with the image information indicating the third information on the display, and is associated with the specified display zone Including the information and the third information, and causing the display to display the display image such that the third information is emphasized more than the information associated with the specified display zone.
The display device according to claim 4. The control unit determines that the warning state has occurred, and displays the display image in accordance with the image information indicating the third information on the display, and is associated with the specified display zone Including the information and the third information, and causing the display to display the display image so that the information is less visible than the third information.
The display device according to claim 4. When the control unit displays the display image on the display according to the acquired image information, at least a part of the information included in the display image based on the gaze position information is used. Display the display image on the display so that all the information located near the gaze position of the person is emphasized with respect to information located outside the gaze position near the user.
The display device according to claim 1.

Images