JP2018055416A - Display device, head-mounted display device, method for controlling display device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable a user to more easily operate a display device having a plurality of functions so as to improve convenience.SOLUTION: A display device comprises: a display part that displays an image; a detection part 170 that is directly or indirectly attached to a user's body; a state determination part 195 that determines a state of the user's body on the basis of a detection result of the detection part 170; a processing execution part 158 that executes functions including display in the display part; and an application execution control part 154 that selects, from the executable functions by the processing execution part 158, a function corresponding to a determination result of the state determination part 195, and causes the processing execution part 158 to execute the selected function.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a display device, a head-mounted display device, a display device control method, and a program.

  Conventionally, a head mounted display (HMD) is known as an example of a display device (see, for example, Patent Document 1). The HMD described in Patent Document 1 is worn in front of the user's eyes and immerses the user in the image of the virtual world. Moreover, since HMD of patent document 1 is provided with a touch panel, the user can operate HMD, paying attention to a display image.

JP-A-2015-118332

The HMD described in Patent Document 1 performs various processes such as content playback, stop, audio output, and output restriction in accordance with user operations.
By the way, in a display device capable of executing a plurality of functions, it is necessary for the user to select a function to be executed and to perform several operations in order to instruct execution. In particular, when the display device has multiple functions, the number of operations is large. In order to perform this kind of operation, the user has to release his hand, and it has been desired to improve convenience.
It is an object of the present invention to improve convenience by allowing a display device having a plurality of functions to be operated more easily.

In order to solve the above problems, the present invention provides a display unit that displays an image, a detection unit that is directly or indirectly attached to the user's body, and the user's body based on the detection result of the detection unit. A determination unit that determines a state of the display unit, an execution unit that executes a function including display on the display unit, and the function corresponding to the determination result of the determination unit is selected from the functions that can be executed by the execution unit And an execution control unit to be executed by the execution unit.
According to the present invention, the display device selects and executes a function according to the state of the user's body. For this reason, an appropriate function is executed without the user performing an operation such as selecting a function. For example, since the user can cause the display device to execute the function without using a hand, the convenience of the display device can be improved.

Further, according to the present invention, the determination unit specifies the state of the user's body from a plurality of preset states based on the detection result of the detection unit, and the execution control unit includes the determination unit. Causes the execution unit to execute the function corresponding to the specified state.
According to the present invention, a function suitable for the state of the user's body can be selected more appropriately and promptly according to the setting.

In the present invention, the execution unit causes the display unit to execute display including at least one of an image and text related to the function by executing the function.
ADVANTAGE OF THE INVENTION According to this invention, the image and text regarding the function selected according to a user's body state can be displayed, and the improvement of convenience can be aimed at.

The present invention further includes a storage unit that stores a program corresponding to a function executed by the execution unit, and the execution control unit is capable of executing the function based on the program stored in the storage unit. Is selected and executed by the execution unit.
ADVANTAGE OF THE INVENTION According to this invention, in the structure which a display apparatus implement | achieves a function by running the program memorize | stored in the memory | storage part, the function suitable for a user's body state can be selected appropriately.

In the present invention, when the execution control unit installs the program in the display device, the execution control unit stores the program to be installed in the storage unit in association with the state of the user's body.
ADVANTAGE OF THE INVENTION According to this invention, the program installed in a display apparatus can be matched with a user's body state. For this reason, the installed program can be appropriately selected according to the state of the user's body.

In addition, the present invention includes a communication unit that communicates with an external device, and the execution control unit selects the function that can be executed by the execution unit based on a program that can be acquired from the external device by the communication unit. Then, the communication unit receives the program corresponding to the selected function, and causes the execution unit to execute the function corresponding to the received program.
ADVANTAGE OF THE INVENTION According to this invention, in the structure which implement | achieves a function with the program acquired from an external apparatus, the function suitable for a user's body state can be selected appropriately.

In the present invention, the execution control unit may determine the number of executions of the function, the evaluation of the function, and the display area in the display unit required when the function is executed, for each of the functions that can be executed by the execution unit. Obtaining priority information indicating the size and at least one of the costs associated with the execution of the function, and the weighting set for each of the conditions, and based on the acquired condition and the priority information, Select the function.
According to the present invention, a function can be selected more appropriately by combining the state of the user's body and the conditions relating to the execution of each function.

The present invention further includes a condition storage unit that stores the condition and the priority information.
According to the present invention, the display device stores the conditions relating to the execution of each function and the information for weighting the conditions, and the function can be quickly selected based on the stored information.

In the present invention, the execution control unit obtains the condition and the priority information from an external device.
According to the present invention, conditions relating to the execution of each function and information for weighting the conditions can be acquired from an external device, and functions can be quickly selected based on the stored information.

Further, according to the present invention, when the execution control unit selects the function, the display unit sets a size of a display area used for displaying the function on the display unit, and the display related to the function is displayed on the set display area. Before the execution unit executes the function, the display related to the function is displayed in the display area smaller than the set size, and input of whether to execute or not is awaited.
According to the present invention, the size of the display area for executing a function is set, and before executing the function, the display size related to the function is reduced, and input of whether to execute or not is waited. For this reason, the display size related to the function can be appropriately set, and when the user inputs whether or not to execute, the user's operability can be improved by reducing the display.

In the present invention, a tag or an identifier is assigned to each of the functions that can be executed by the execution unit, and the execution control unit assigns a plurality of the functions to the tag before causing the execution unit to execute the functions. Alternatively, it is displayed so as to be selectable together with the identifier, and waits for the input relating to the selection and execution of the function.
According to the present invention, a function can be selected according to a user input. Further, based on the tag and the identifier, a function suitable for the state of the user's body can be selected.

Further, according to the present invention, the execution control unit sets a size of a display area used for displaying the function on the display unit based on the state of the user determined by the determination unit, and the set display area To display the function.
According to the present invention, it is possible to appropriately set the display size related to the function in accordance with the state of the user.

Further, according to the present invention, the execution control unit executes a preset interrupt process when an external factor is generated or an abnormal value is detected by the detection unit while the execution unit is executing the function. Let the department execute.
According to the present invention, interrupt processing can be executed when an external factor is generated or an abnormal value is detected during execution of a function.

Further, according to the present invention, the determination unit determines the state of the user every preset time, and the execution control unit, when the state of the user determined by the determination unit has changed, Change the function executed by the execution unit when it is determined whether or not the state of the user determined by the determination unit returns to the previously determined state within a predetermined time. Let
According to the present invention, when the state of the user's body changes, the function to be executed can be changed in response to the change, and the execution of the function can be continued when the need for changing is small. For this reason, inconvenience due to frequent function changes can be avoided, and changes in the state of the user's body can be accommodated.

In order to solve the above problems, the present invention provides a display unit that is mounted on a user's head and displays an image so that an outside scene can be visually recognized, and detection that is mounted directly or indirectly on the user's body. A determination unit that determines a state of the user's body based on a detection result of the detection unit, an execution unit that executes a function including display on the display unit, and the function that can be executed by the execution unit An execution control unit that selects the function corresponding to the determination result of the determination unit and causes the execution unit to execute the function.
According to the present invention, a display device having a display unit mounted on the user's head selects and executes a function in accordance with the state of the user's body, and performs display regarding the function to be executed. For this reason, an appropriate function is executed without the user performing an operation such as selecting a function. For example, since the user can cause the display device to execute the function without using a hand, the convenience of the display device can be improved.

In order to solve the above-mentioned problem, the present invention controls a display device including a display unit that displays an image and a detection unit that is directly or indirectly attached to a user's body, and the detection unit A determination step of determining the state of the user's body based on the detection result of the user, a selection step of selecting the function corresponding to the determination result of the determination step among a plurality of executable functions, and the selection step The execution step of executing the function including the display on the display unit, selected in step.
According to the present invention, the display device selects and executes a function according to the state of the user's body. For this reason, an appropriate function is executed without the user performing an operation such as selecting a function. For example, since the user can cause the display device to execute the function without using a hand, the convenience of the display device can be improved.

In order to solve the above problems, the present invention provides a computer that controls a display device including a display unit that displays an image and a detection unit that is directly or indirectly attached to a user's body. A determination step of determining a state of the user's body based on a detection result of the unit, a selection step of selecting the function corresponding to the determination result of the determination step among a plurality of executable functions, and the selection An execution step of executing the function including the display on the display unit selected in the step.
According to the present invention, the display device selects and executes a function according to the state of the user's body. For this reason, an appropriate function is executed without the user performing an operation such as selecting a function. For example, since the user can cause the display device to execute the function without using a hand, the convenience of the display device can be improved.

The present invention can be implemented in various forms other than the above-described display device, head-mounted display device, and display device control method. For example, a display system including the above display device can be configured.
Further, when the program is realized as a program executed by a computer (or a processor) to execute the control method, a recording medium that records the program, a server device that distributes the program, a transmission medium that transmits the program, and the program Can be realized in the form of a data signal or the like embodied in a carrier wave.

1 is a schematic configuration diagram of a display system 1 according to an embodiment. Explanatory drawing which shows the external appearance structure of HMD. The figure which shows the structure of the optical system of an image display part. Explanatory drawing which shows a response | compatibility with an image display part and an imaging range. The block diagram of each part which comprises HMD. The functional block diagram of the control system of HMD. The schematic diagram which shows the structural example of the application information which HMD memorize | stores. The schematic diagram which shows the structural example of the application execution conditions which HMD memorize | stores. The schematic diagram which shows the structural example of the application additional information which HMD memorize | stores. The flowchart which shows operation | movement of HMD. The flowchart which shows operation | movement of HMD. The flowchart which shows operation | movement of HMD. The figure which shows the example of a display of HMD.

FIG. 1 is a schematic configuration diagram of a display system 1 according to an embodiment to which the present invention is applied.
As shown in FIG. 1, the display system 1 includes a plurality of HMDs 100. The plurality of HMDs 100 have a common configuration. There is no restriction | limiting in the number of HMD100 with which the display system 1 is provided, In this embodiment, the example provided with two HMD100 is shown. In the following description, reference numerals 100A and 100B are described when two HMDs 100 included in the display system 1 are distinguished.

The HMD 100 is a display device that a user wears on the head as shown in FIG. 1, and is also called a head-mounted display device (head mounted display). The specific configuration of the HMD 100 does not limit the present invention. The HMD 100 described in the present embodiment has a configuration in which a user can visually recognize an outside scene at the same time as visually recognizing a virtual image. The HMD 100 is a so-called optical transmission type HMD.
In the following description, for convenience, a virtual image visually recognized by the user wearing the HMD 100 by the function of the HMD 100 is referred to as a “display image”. The HMD 100 defines a process or operation of emitting image light generated based on image data as “display an image”.
Reference sign UA indicates a user (user) who wears the HMD 100A on the head. Reference UB indicates a user who wears the HMD 100B on the head. When the users UA and UB are not distinguished, they are described as the user U.

In the display system 1, the HMDs 100A and 100B can communicate with each other by various wireless communication methods described later.
The HMDs 100A and 100B can be connected to the server 3 via the communication network 4. The communication network 4 is realized by various communication lines or a combination thereof. The communication network 4 may be a wide area communication line network that can connect remote locations, or may be a LAN (Local Area Network) laid in a specific facility or building. The specific configuration of the communication network 4 is not limited. For example, the communication network 4 may include a public line network, a dedicated line, a wireless communication line including a mobile phone line, a backbone communication line of these lines, and the like. The communication network 4 may include network devices such as a server device, a gateway device, and a router device that connect the communication lines. Further, the communication network 4 may include a plurality of communication lines.

The server 3 (external device) is connected to the communication network 4 via a wired or wireless communication line.
The HMD 100A and the HMD 100B are connected to the communication network 4 via a wireless communication line by the communication unit 117 (FIG. 5). The HMDs 100 </ b> A and 100 </ b> B perform data communication with the server 3 using the communication network 4. For example, the HMDs 100 </ b> A and 100 </ b> B can receive an application program and various data related to execution of the application program from the server 3. Various data related to the execution of the application program includes, for example, application information 127 (FIG. 7), application execution conditions 128 (FIG. 8), application additional information 129 (FIG. 9), and the like.
The HMDs 100A and 100B perform wireless communication complying with standards such as Bluetooth (registered trademark) and wireless LAN (including Wi-Fi (registered trademark)), for example.
The server 3 and each of the HMDs 100A and 100B may perform data communication directly without going through the communication network 4.

  FIG. 2 is an explanatory diagram showing an external configuration of the HMD 100. The HMD 100 includes an image display unit 20 (display unit) that causes the user U to visually recognize a virtual image while being mounted on the head of the user U, and a control device 10 that controls the image display unit 20. As shown in FIG. 2, the control device 10 includes a flat box-shaped case 10 </ b> A (which can also be referred to as a housing or a main body), and the case 10 </ b> A includes each unit described below. On the surface of the case 10 </ b> A, various buttons 11, switches, a track pad 14, and the like that receive the operation of the user U are provided. When the user U operates these, the control apparatus 10 functions as a controller of the HMD 100.

  The image display unit 20 is a mounting body that is mounted on the head of the user U, and has a glasses shape in the present embodiment. The image display unit 20 includes a right display unit 22, a left display unit 24, a right light guide plate 26, and a left light guide plate 28 in a main body having a right holding unit 21, a left holding unit 23, and a front frame 27. .

  Each of the right holding unit 21 and the left holding unit 23 extends rearward from both ends of the front frame 27 and holds the image display unit 20 on the head of the user U like a temple of glasses. Here, of the both ends of the front frame 27, the end located on the right side of the user U in the mounted state of the image display unit 20 is defined as the end ER, and the end located on the left side of the user U is defined as the end. Let it be EL. The right holding unit 21 extends from the end ER of the front frame 27 to a position corresponding to the right head of the user U when the image display unit 20 is mounted. The left holding portion 23 is provided to extend from the end portion EL to a position corresponding to the left side of the user U when the image display portion 20 is worn.

  The right light guide plate 26 and the left light guide plate 28 are provided on the front frame 27. The right light guide plate 26 is positioned in front of the right eye of the user U when the image display unit 20 is mounted, and causes the right eye to visually recognize the image. The left light guide plate 28 is positioned in front of the left eye of the user U when the image display unit 20 is mounted, and causes the left eye to visually recognize the image.

  The front frame 27 has a shape in which one end of the right light guide plate 26 and one end of the left light guide plate 28 are connected to each other. This connection position is used when the user U wears the image display unit 20. It corresponds to the eyebrows of the person U. The front frame 27 may be provided with a nose pad portion that comes into contact with the nose of the user U when the image display unit 20 is mounted at a connection position between the right light guide plate 26 and the left light guide plate 28. In this case, the image display unit 20 can be held on the head of the user U by the nose pad, the right holding unit 21 and the left holding unit 23. In addition, a belt (not shown) that contacts the back of the user U when the image display unit 20 is attached may be connected to the right holding unit 21 and the left holding unit 23. In this case, the image display unit 20 is connected by the belt. It can be held on the head of the user U.

  The right display unit 22 is a unit related to image display by the right light guide plate 26 and is provided in the right holding unit 21 and is located in the vicinity of the right side of the user U in the mounted state. The left display unit 24 is a unit related to image display by the left light guide plate 28 and is provided in the left holding unit 23 and is positioned in the vicinity of the left side of the user U in the mounted state. The right display unit 22 and the left display unit 24 are also collectively referred to simply as a “display driving unit”.

The right light guide plate 26 and the left light guide plate 28 of the present embodiment are optical units formed of a light-transmitting resin or the like, and are, for example, prisms, and output image light output from the right display unit 22 and the left display unit 24. To the user U's eyes.
A light control plate (not shown) may be provided on the surfaces of the right light guide plate 26 and the left light guide plate 28. The light control plate is an optical element on a thin plate having different transmittance depending on the wavelength region of light, and functions as a so-called wavelength filter. For example, the light control plate is disposed so as to cover the front side of the front frame 27 that is the side opposite to the eye side of the user U. By appropriately selecting the optical characteristics of the light control plate, the transmittance of light in an arbitrary wavelength region such as visible light, infrared light, and ultraviolet light can be adjusted. The amount of external light incident on the light plate 28 and transmitted through the right light guide plate 26 and the left light guide plate 28 can be adjusted.

  The image display unit 20 guides the image light generated by the right display unit 22 and the left display unit 24 to the right light guide plate 26 and the left light guide plate 28, respectively, and makes the user U visually recognize the virtual image by the image light. Display an image. When external light is incident on the user U's eyes through the right light guide plate 26 and the left light guide plate 28 from the front of the user U, image light and external light constituting a virtual image are displayed on the user U's eyes. Is incident, and the visibility of the virtual image is affected by the intensity of external light. For this reason, for example, by attaching a light control plate to the front frame 27 and appropriately selecting or adjusting the optical characteristics of the light control plate, the visibility of the virtual image can be adjusted. In a typical example, it is possible to use a light control plate having a light transmittance that allows the user U wearing the HMD 100 to visually recognize at least the outside scenery. In addition, when the dimming plate is used, it is possible to protect the right light guide plate 26 and the left light guide plate 28 and to suppress the damage to the right light guide plate 26 and the left light guide plate 28, the adhesion of dirt, and the like. The light control plate may be detachable with respect to the front frame 27 or each of the right light guide plate 26 and the left light guide plate 28, or may be mounted by replacing a plurality of types of light control plates. May be omitted.

  The camera 61 is disposed on the front frame 27 of the image display unit 20. It is desirable that the camera 61 captures an outside scene direction that is visually recognized by the user U with the image display unit 20 attached, and external light that passes through the right light guide plate 26 and the left light guide plate 28 on the front surface of the front frame 27. It is provided in a position where it does not block. In the example of FIG. 2, the camera 61 is disposed on the end ER side of the front frame 27. The camera 61 may be disposed on the end EL side, or may be disposed at a connection portion between the right light guide plate 26 and the left light guide plate 28.

The camera 61 is a digital camera that includes an imaging element such as a CCD or CMOS, an imaging lens, and the like. The camera 61 of the present embodiment is a monocular camera, but may be a stereo camera. The camera 61 images at least a part of the outside scene (real space) in the front direction of the HMD 100, in other words, in the viewing direction of the user U when the HMD 100 is worn. In another expression, it can be said that the camera 61 captures a range or direction that overlaps the field of view of the user U and captures a direction in which the user U is gazing. Although the angle of view of the camera 61 can be set as appropriate, in the present embodiment, as will be described later, it includes the outside world that the user U visually recognizes through the right light guide plate 26 and the left light guide plate 28. More preferably, the imaging range of the camera 61 is set so that the entire field of view of the user U visible through the right light guide plate 26 and the left light guide plate 28 can be imaged.
The camera 61 executes imaging according to the control of the control unit 150 and / or the sensor hub 192 (FIG. 6), and outputs captured image data to the imaging control unit 149. Here, the camera 61 may include a so-called wide-angle lens as an imaging lens, and may be configured to capture a wide angle of view. The wide-angle lens may include a lens called a super-wide-angle lens or a quasi-wide-angle lens, may be a single focus lens or a zoom lens, and the camera 61 includes a lens group including a plurality of lenses. There may be.

FIG. 3 is a main part plan view showing the configuration of the optical system provided in the image display unit 20. FIG. 3 shows the left eye LE and right eye RE of the user U for explanation.
As shown in FIG. 3, the right display unit 22 and the left display unit 24 are configured symmetrically. As a configuration for causing the right eye RE of the user U to visually recognize an image, the right display unit 22 includes an OLED (Organic Light Emitting Diode) unit 221 that emits image light, a lens group that guides the image light L emitted from the OLED unit 221, and the like. And a right optical system 251 provided. The image light L is guided to the right light guide plate 26 by the right optical system 251.

The OLED unit 221 includes an OLED panel 223 and an OLED drive circuit 225 that drives the OLED panel 223. The OLED panel 223 is a self-luminous display configured by arranging light emitting elements that emit light of R (red), G (green), and B (blue) light by organic electroluminescence in a matrix. It is a panel. The OLED panel 223 includes a plurality of pixels, each of which includes one R, G, and B element, and forms an image with pixels arranged in a matrix. The OLED drive circuit 225 selects a light emitting element included in the OLED panel 223 and energizes the light emitting element under the control of the control unit 150 (FIG. 6), and causes the light emitting element of the OLED panel 223 to emit light. The OLED drive circuit 225 is fixed to the back surface of the OLED panel 223, that is, the back surface of the light emitting surface by bonding or the like. The OLED drive circuit 225 may be formed of a semiconductor device that drives the OLED panel 223, for example, and may be mounted on a substrate (not shown) fixed to the back surface of the OLED panel 223. A temperature sensor 217 is mounted on this substrate.
Note that the OLED panel 223 may have a configuration in which light emitting elements that emit white light are arranged in a matrix, and color filters corresponding to R, G, and B colors are stacked. Further, an OLED panel 223 having a WRGB configuration provided with a light emitting element that emits W (white) light in addition to the light emitting elements that respectively emit R, G, and B color light may be used.

  The right optical system 251 includes a collimator lens that converts the image light L emitted from the OLED panel 223 into a parallel light flux. The image light L converted into a parallel light beam by the collimator lens enters the right light guide plate 26. A plurality of reflecting surfaces that reflect the image light L are formed in an optical path that guides light inside the right light guide plate 26. The image light L is guided to the right eye RE side through a plurality of reflections inside the right light guide plate 26. A half mirror 261 (reflection surface) located in front of the right eye RE is formed on the right light guide plate 26. The image light L is reflected by the half mirror 261 and emitted from the right light guide plate 26 toward the right eye RE. The image light L forms an image on the retina of the right eye RE, and causes the user U to visually recognize the image.

  Further, as a configuration for allowing the left eye LE of the user U to visually recognize the image, the left display unit 24 includes a left optical unit including an OLED unit 241 that emits image light, a lens group that guides the image light L emitted from the OLED unit 241, and the like. A system 252; The image light L is guided to the left light guide plate 28 by the left optical system 252.

  The OLED unit 241 includes an OLED panel 243 and an OLED drive circuit 245 that drives the OLED panel 243. The OLED panel 243 is a self-luminous display panel configured similarly to the OLED panel 223. The OLED drive circuit 245 selects a light emitting element included in the OLED panel 243 and energizes the light emitting element under the control of the control unit 150 (FIG. 6), and causes the light emitting element of the OLED panel 243 to emit light. The OLED drive circuit 245 is fixed to the back surface of the OLED panel 243, that is, the back surface of the light emitting surface by bonding or the like. The OLED drive circuit 245 may be configured by a semiconductor device that drives the OLED panel 243, for example, and may be mounted on a substrate (not shown) fixed to the back surface of the OLED panel 243. A temperature sensor 239 is mounted on this substrate.

  The left optical system 252 includes a collimating lens that converts the image light L emitted from the OLED panel 243 into a light beam in a parallel state. The image light L converted into a parallel light beam by the collimator lens enters the left light guide plate 28. The left light guide plate 28 is an optical element formed with a plurality of reflecting surfaces that reflect the image light L, and is, for example, a prism. The image light L is guided to the left eye LE side through a plurality of reflections inside the left light guide plate 28. The left light guide plate 28 is formed with a half mirror 281 (reflection surface) located in front of the left eye LE. The image light L is reflected by the half mirror 281 and emitted from the left light guide plate 28 toward the left eye LE. The image light L forms an image on the retina of the left eye LE, and causes the user U to visually recognize the image.

According to this configuration, the HMD 100 functions as a see-through display device. That is, the image light L reflected by the half mirror 261 and the external light OL transmitted through the right light guide plate 26 are incident on the right eye RE of the user U. Further, the image light L reflected by the half mirror 281 and the external light OL transmitted through the half mirror 281 are incident on the left eye LE. In this way, the HMD 100 superimposes the image light L of the internally processed image and the external light OL on the eyes of the user U, and for the user U, the right light guide plate 26 and the left light guide plate 28 are watermarked. The outside scene can be seen, and the image by the image light L is visually recognized over the outside scene.
The half mirrors 261 and 281 are image extraction units that extract image by reflecting image light output from the right display unit 22 and the left display unit 24, respectively, and can be called display units.

  The left optical system 252 and the left light guide plate 28 are collectively referred to as a “left light guide”, and the right optical system 251 and the right light guide plate 26 are collectively referred to as a “right light guide”. The configurations of the right light guide unit and the left light guide unit are not limited to the above example, and any method can be used as long as a virtual image is formed in front of the user U using image light. For example, a diffraction grating Or a transflective film may be used.

  The camera 61 (FIG. 2) of the present embodiment images the direction in which both eyes of the user U face, that is, the front for the user U. For example, the optical axis of the camera 61 may be a direction including the line-of-sight direction of the right eye RE and the left eye LE (FIG. 3). The outside scene that the user U can visually recognize while wearing the HMD 100 is not always at infinity. For example, when the user U gazes at an object (not shown) located in the field of view with both eyes, the line of sight of the user U is directed toward the object. In this case, the distance from the user U to the target object is often about 30 cm to 10 m, and more often about 1 m to 4 m. Therefore, for the HMD 100, an upper limit and a lower limit of the distance from the user U to the object during normal use may be determined. This standard may be obtained by investigation or experiment, or may be set by the user U. The optical axis and the angle of view of the camera 61 are determined when the distance to the object in normal use corresponds to the set upper limit guide and the lower limit guide. It is preferable that it is set so as to be included.

  In general, the viewing angle of a human is about 200 degrees in the horizontal direction and about 125 degrees in the vertical direction, and of these, the effective field of view with excellent information receiving capability is about 30 degrees in the horizontal direction and about 20 degrees in the vertical direction. Furthermore, the stable gaze field in which the gaze point that the human gazes at appears to be quickly and stably is about 60 to 90 degrees in the horizontal direction and about 45 to 70 degrees in the vertical direction. Therefore, about 30 degrees in the horizontal direction and about 20 degrees in the vertical direction centering on the line of sight of the right eye RE and the left eye LE are effective visual fields of the right eye RE and the left eye LE, respectively. Further, the stable viewing field is about 60 to 90 degrees in the horizontal direction and about 45 to 70 degrees in the vertical direction, and the viewing angle is about 200 degrees in the horizontal direction and about 125 degrees in the vertical direction. Furthermore, an actual field of view that the user U permeates through the image display unit 20 and permeates through the right light guide plate 26 and the left light guide plate 28 can be referred to as a real field of view (FOV). In the configuration of the present embodiment shown in FIGS. 2 and 3, the actual visual field corresponds to an actual visual field that is transmitted through the right light guide plate 26 and the left light guide plate 28 and visually recognized by the user U. The real field of view is narrower than the viewing angle and stable field of view, but wider than the effective field of view.

  The field of view of the camera 61 is preferably capable of capturing a wider range than the field of view of the user U. Specifically, the field of view of the camera 61 is preferably at least wider than the effective field of view of the user U. . Further, it is more preferable that the angle of view of the camera 61 is wider than the actual field of view of the user U. More preferably, the angle of view of the camera 61 is wider than the stable viewing field of the user U, and most preferably, the angle of view of the camera 61 is wider than the viewing angle of both eyes of the user U.

  Returning to FIG. 2, the control device 10 and the image display unit 20 are connected by a connection cable 40. The connection cable 40 is detachably connected to a connector (not shown) provided at the lower part of the case 10 </ b> A, and is connected to various circuits provided in the image display unit 20 from the tip of the left holding unit 23. The connection cable 40 includes a metal cable or optical fiber cable that transmits digital data, and may include a metal cable that transmits an analog signal. A connector 46 is provided in the middle of the connection cable 40. The connector 46 is a jack for connecting a stereo mini-plug, and the connector 46 and the control device 10 are connected by a line for transmitting an analog audio signal, for example. In the configuration example shown in FIG. 2, a headset 30 having a right earphone 32 and a left earphone 34 and a microphone 63 constituting a stereo headphone is connected to a connector 46.

  For example, as shown in FIG. 2, the microphone 63 is arranged so that the sound collection unit of the microphone 63 faces the line of sight of the user U, collects sound, and sends the sound signal to the sound interface 182 (FIG. 5). Output. For example, the microphone 63 may be a monaural microphone or a stereo microphone, may be a directional microphone, or may be an omnidirectional microphone.

The control device 10 includes a button 11, an LED indicator 12, a track pad 14, an up / down key 15, a changeover switch 16, and a power switch 18 as operated parts operated by a user U. These operated parts are arranged on the surface of the case 10A.
The button 11 includes a menu key, a home key, a return key, and the like for operating the operating system 143 (FIG. 6) executed by the control device 10, and in particular, is displaced by a pressing operation of these keys and switches. Including things. The LED indicator 12 lights up or flashes in accordance with the operating state of the HMD 100. The up / down key 15 is used to input an instruction to increase / decrease the volume output from the right earphone 32 and the left earphone 34 and to input an instruction to increase / decrease the display brightness of the image display unit 20. The changeover switch 16 is a switch for changing an input corresponding to the operation of the up / down key 15. The power switch 18 is a switch for switching on / off the power of the HMD 100, and is configured by a slide switch, for example.

The track pad 14 has an operation surface for detecting a contact operation, and outputs an operation signal according to an operation on the operation surface. The detection method on the operation surface is not limited, and an electrostatic method, a pressure detection method, an optical method, or the like can be adopted. Contact (touch operation) to the track pad 14 is detected by the touch sensor 13 (FIG. 5).
Further, as indicated by a broken line in FIG. 2, an LED display unit 17 is installed on the track pad 14. The LED display unit 17 includes a plurality of LEDs, and the operation unit can be visually recognized on the LED display unit 17 by lighting each LED. In the example of FIG. 2, three symbols Δ (triangle), ○ (circle), and □ (square) appear when the LED of the LED display unit 17 is turned on. When the LED display unit 17 is turned off, these symbols are not visible. This configuration can be realized, for example, by configuring the track pad 14 as a colored or colorless and transparent flat plate having translucency, and disposing an LED directly under the flat plate.

  The LED display unit 17 functions as a software button. For example, when the LED display unit 17 is lit, the lighting position (display position) of the symbol functions as a button for giving an instruction corresponding to the symbol. In the example of FIG. 2, the symbol ○ (circle) functions as a home button, and when an operation for touching the position of the symbol ○ (circle) is performed, the control unit 150 performs the home button based on the detection value of the touch sensor 13. Detect operation of. The symbol □ (square) functions as a history button. When an operation to contact the position of the symbol □ (square) is performed, the control unit 150 detects the operation of the history button based on the detection value of the touch sensor 13. The symbol Δ (triangle) functions as a return button. When an operation to contact the position of the symbol Δ (triangle) is performed, the control unit 150 detects the operation of the return button based on the detection value of the touch sensor 13.

  FIG. 4 is a diagram illustrating a configuration of a main part of the image display unit 20, and FIG. In FIG. 4, the connection cable 40 is not shown.

  FIG. 4 is a side of the image display unit 20 that is in contact with the head of the user U, in other words, a side that is visible to the right eye RE and the left eye LE of the user U. In other words, the back sides of the right light guide plate 26 and the left light guide plate 28 are visible. In FIG. 4, the half mirror 261 that irradiates the right eye RE of the user U with the image light and the half mirror 281 that irradiates the left eye LE with the image light appear as a substantially rectangular area. Further, the entire right light guide plate 26 and left light guide plate 28 including the half mirrors 261 and 281 transmit external light as described above. For this reason, the user U sees the outside scene through the entire right light guide plate 26 and the left light guide plate 28, and sees a rectangular display image at the positions of the half mirrors 261 and 281. An area in which an image is formed by the rectangular half mirrors 261 and 281 is referred to as a display area V (FIG. 13) where the image display unit 20 displays an image.

FIG. 5 is a block diagram illustrating a configuration of each unit included in the HMD 100.
The control device 10 includes a main processor 140 that executes a program and controls the HMD 100. A memory 118 and a non-volatile storage unit 121 are connected to the main processor 140. The main processor 140 is connected to the track pad 14 and the operation unit 110 as input devices. In addition, a six-axis sensor 111, a magnetic sensor 113, and a GPS 115 are connected to the main processor 140 as sensors. The main processor 140 is connected to a communication unit 117, an audio codec 180, an external connector 184, an external memory interface 186, a USB connector 188, a sensor hub 192, and an FPGA 194. These function as an interface with the outside.

  The main processor 140 is mounted on the controller board 120 built in the control device 10. In addition to the main processor 140, a memory 118, a nonvolatile storage unit 121, and the like may be mounted on the controller board 120. In the present embodiment, a six-axis sensor 111, a magnetic sensor 113, a GPS 115, a communication unit 117, a memory 118, a nonvolatile storage unit 121, an audio codec 180, and the like are mounted on the controller board 120. Further, the external connector 184, the external memory interface 186, the USB connector 188, the sensor hub 192, the FPGA 194, and the interface 196 may be mounted on the controller board 120.

  The memory 118 constitutes a work area that temporarily stores a program to be executed and data to be processed when the main processor 140 executes the program. The nonvolatile storage unit 121 is configured by a flash memory or an eMMC (embedded Multi Media Card). The nonvolatile storage unit 121 stores a program executed by the main processor 140 and various data processed by the main processor 140 executing the program.

The main processor 140 detects a touch operation on the operation surface of the track pad 14 based on an operation signal input from the track pad 14 and acquires an operation position.
The operation unit 110 includes a button 11, a touch sensor 13, and an LED display unit 17. The touch sensor 13 detects a touch operation on the track pad 14 and specifies an operation position of the detected touch operation. When the operation of the button 11 is performed and when the touch sensor 13 detects the touch operation, an operation signal is output from the operation unit 110 to the main processor 140.
The LED display unit 17 is disposed immediately below the track pad 14 (FIG. 2) and includes an LED (not shown) and a drive circuit that lights the LED. The LED display unit 17 turns on, blinks, and turns off the LED according to the control of the main processor 140.

The 6-axis sensor 111 is a motion sensor (inertial sensor) including a 3-axis acceleration sensor and a 3-axis gyro (angular velocity) sensor. The 6-axis sensor 111 may employ an IMU (Inertial Measurement Unit) in which the above sensors are modularized.
The magnetic sensor 113 is, for example, a triaxial geomagnetic sensor.
A GPS (Global Positioning System) 115 includes a GPS antenna (not shown), receives a radio signal transmitted from a GPS satellite, and detects the coordinates of the current position of the control device 10.
The six-axis sensor 111, the magnetic sensor 113, and the GPS 115 output detection values to the main processor 140 according to a sampling period specified in advance. Alternatively, the six-axis sensor 111, the magnetic sensor 113, and the GPS 115 output detection values to the main processor 140 at a timing designated by the main processor 140 in response to a request from the main processor 140.

  The communication unit 117 performs wireless communication with an external device. In the display system 1, the communication unit 117 performs data communication with the server 3 using the communication network 4 (FIG. 1) or directly. The communication unit 117 includes an antenna, an RF circuit, a baseband circuit, a communication control circuit, and the like, or a device in which these are integrated. The communication unit 117 performs wireless communication complying with standards such as Bluetooth and wireless LAN (including Wi-Fi), for example.

  The audio interface 182 is an interface for inputting and outputting audio signals. The audio interface 182 includes a connector 46 (FIG. 2) provided on the connection cable 40. The audio codec 180 is connected to the audio interface 182 and encodes / decodes an audio signal input / output via the audio interface 182. The audio codec 180 may include an A / D converter that converts analog audio signals to digital audio data, and a D / A converter that performs the reverse conversion. For example, the HMD 100 according to the present embodiment outputs sound by the right earphone 32 and the left earphone 34 and collects sound by the microphone 63. The audio codec 180 converts the digital audio data output from the main processor 140 into an analog audio signal and outputs the analog audio signal via the audio interface 182. The audio codec 180 converts an analog audio signal input to the audio interface 182 into digital audio data and outputs the digital audio data to the main processor 140.

  The external connector 184 is a connector that connects an external device that communicates with the main processor 140. The external connector 184 is an interface for connecting an external device when, for example, an external device is connected to the main processor 140 to debug a program executed by the main processor 140 or collect a log of the operation of the HMD 100. It is.

  The external memory interface 186 is an interface to which a portable memory device can be connected. For example, the external memory interface 186 includes a memory card slot in which a card type recording medium is mounted and data can be read and an interface circuit. In this case, the size, shape and standard of the card type recording medium are not limited and can be changed as appropriate.

  A USB (Universal Serial Bus) connector 188 includes a connector conforming to the USB standard and an interface circuit, and can connect a USB memory device, a smartphone, a computer, and the like. The size and shape of the USB connector 188 and the compatible USB standard version can be appropriately selected and changed.

  Further, the HMD 100 includes a vibrator 19. The vibrator 19 includes a motor (not shown), an eccentric rotor (not shown), and the like, and generates vibrations under the control of the main processor 140. The HMD 100 generates vibration by the vibrator 19 with a predetermined vibration pattern, for example, when an operation on the operation unit 110 is detected, or when the power of the HMD 100 is turned on / off.

  The sensor hub 192 and the FPGA 194 are connected to the image display unit 20 via an interface (I / F) 196. The sensor hub 192 acquires detection values of various sensors included in the image display unit 20 and outputs them to the main processor 140. The FPGA 194 executes processing of data transmitted and received between the main processor 140 and each unit of the image display unit 20 and transmission via the interface 196.

  The right display unit 22 and the left display unit 24 of the image display unit 20 are each connected to the control device 10. As shown in FIG. 2, in the HMD 100, the connection cable 40 is connected to the left holding unit 23, and the wiring connected to the connection cable 40 is laid inside the image display unit 20, and each of the right display unit 22 and the left display unit 24 is Connected to the control device 10.

The right display unit 22 includes a display unit substrate 210. The display unit substrate 210 includes an interface (I / F) 211 connected to the interface 196, a receiving unit (Rx) 213 that receives data input from the control device 10 via the interface 211, and an EEPROM 215 (storage unit) ) Is implemented.
The interface 211 connects the receiving unit 213, the EEPROM 215, the temperature sensor 217, the camera 61, the illuminance sensor 65, and the LED indicator 67 to the control device 10.

  An EEPROM (Electrically Erasable Programmable Read-Only Memory) 215 stores various data so that the main processor 140 can read the data. The EEPROM 215 stores, for example, data relating to the light emission characteristics and display characteristics of the OLED units 221 and 241 provided in the image display unit 20, data relating to the characteristics of sensors provided in the right display unit 22 or the left display unit 24, and the like. Specifically, parameters relating to gamma correction of the OLED units 221, 241 and data for compensating the detection values of the temperature sensors 217, 239 are stored. These data are generated by the factory inspection of the HMD 100 and written in the EEPROM 215. After the shipment, the main processor 140 can perform processing using the data in the EEPROM 215.

The camera 61 executes imaging according to a signal input via the interface 211 and outputs captured image data or a signal indicating the imaging result to the control device 10.
As shown in FIG. 2, the illuminance sensor 65 is provided at the end ER of the front frame 27 and is arranged to receive external light from the front of the user U wearing the image display unit 20. The illuminance sensor 65 outputs a detection value corresponding to the amount of received light (received light intensity).
As shown in FIG. 2, the LED indicator 67 is disposed in the vicinity of the camera 61 at the end ER of the front frame 27. The LED indicator 67 is lit during execution of imaging by the camera 61 to notify that imaging is in progress.

  The temperature sensor 217 detects the temperature and outputs a voltage value or a resistance value corresponding to the detected temperature as a detected value. The temperature sensor 217 is mounted on the back side of the OLED panel 223 (FIG. 4). For example, the temperature sensor 217 may be mounted on the same substrate as the OLED drive circuit 225. With this configuration, the temperature sensor 217 mainly detects the temperature of the OLED panel 223.

  The receiving unit 213 receives data transmitted from the main processor 140 via the interface 211. When receiving image data of an image to be displayed on the OLED unit 221, the receiving unit 213 outputs the received image data to the OLED drive circuit 225 (FIG. 3).

The left display unit 24 includes a display unit substrate 210. On the display unit substrate 210, an interface (I / F) 231 connected to the interface 196 and a receiving unit (Rx) 233 that receives data input from the control device 10 via the interface 231 are mounted. In addition, a 6-axis sensor 235 and a magnetic sensor 237 are mounted on the display unit substrate 210.
The interface 231 connects the receiving unit 233, the six-axis sensor 235, the magnetic sensor 237, and the temperature sensor 239 to the control device 10.

The 6-axis sensor 235 is a motion sensor (inertial sensor) including a 3-axis acceleration sensor and a 3-axis gyro (angular velocity) sensor. The 6-axis sensor 235 may employ an IMU (Inertial Measurement Unit) in which the above sensors are modularized.
The magnetic sensor 237 is, for example, a triaxial geomagnetic sensor.

The temperature sensor 239 detects the temperature and outputs a voltage value or a resistance value corresponding to the detected temperature as a detected value. The temperature sensor 239 is mounted on the back side of the OLED panel 243 (FIG. 4). The temperature sensor 239 may be mounted on the same substrate as the OLED drive circuit 245, for example. With this configuration, the temperature sensor 239 mainly detects the temperature of the OLED panel 243.
Further, the temperature sensor 239 may be incorporated in the OLED panel 243 or the OLED drive circuit 245. The substrate may be a semiconductor substrate. Specifically, when the OLED panel 243 is mounted as an Si-OLED as an integrated circuit on an integrated semiconductor chip together with the OLED drive circuit 245 and the like, the temperature sensor 239 may be mounted on the semiconductor chip.

The camera 61, the illuminance sensor 65, the temperature sensor 217, and the 6-axis sensor 235, the magnetic sensor 237, and the temperature sensor 239 included in the left display unit 24 are connected to the sensor hub 192. The sensor hub 192 sets and initializes the sampling period of each sensor according to the control of the main processor 140. The sensor hub 192 executes energization to each sensor, transmission of control data, acquisition of a detection value, and the like in accordance with the sampling period of each sensor. The sensor hub 192 outputs detection values of the sensors included in the right display unit 22 and the left display unit 24 to the main processor 140 at a preset timing. The sensor hub 192 may have a function of temporarily holding the detection value of each sensor in accordance with the output timing to the main processor 140. In addition, the sensor hub 192 may have a function of converting to data in a unified data format and outputting the data to the main processor 140 in response to a difference in signal format or data format of the output value of each sensor.
The sensor hub 192 starts and stops energization of the LED indicator 67 according to the control of the main processor 140, and lights or blinks the LED indicator 67 in accordance with the timing when the camera 61 starts and ends imaging.

  The control device 10 includes a power supply unit 130 and operates with power supplied from the power supply unit 130. The power supply unit 130 includes a rechargeable battery 132 and a power supply control circuit 134 that detects the remaining capacity of the battery 132 and controls the charging of the battery 132. The power control circuit 134 is connected to the main processor 140 and outputs a detected value of the remaining capacity of the battery 132 or a detected value of the voltage to the main processor 140. Further, the power may be supplied from the control device 10 to the image display unit 20 based on the power supplied from the power supply unit 130. The power supply state from the power supply unit 130 to each unit of the control device 10 and the image display unit 20 may be configured to be controllable by the main processor 140.

  The HMD 100 may include an interface (not shown) for connecting various external devices that are content supply sources. For example, an interface corresponding to a wired connection such as a USB interface, a micro USB interface, or a memory card interface may be used, or a wireless communication interface may be used. The external device in this case is an image supply device that supplies an image to the HMD 100, and a personal computer (PC), a mobile phone terminal, a portable game machine, or the like is used. In this case, the HMD 100 can output images and sounds based on content data input from these external devices.

  FIG. 6 is a functional block diagram of the control system of the HMD 100. The control system of the HMD 100 includes a storage unit 122, a control unit 150, and a sensor hub 192. FIG. 6 illustrates a detection unit 170 that is a control target of the control system.

The storage unit 122 is a logical storage unit configured by the nonvolatile storage unit 121 (FIG. 5), and may include an EEPROM 215.
The control unit 150 and various functional units included in the control unit 150 are formed by cooperation of software and hardware as the main processor 140 executes a program. The control unit 150 and each functional unit constituting the control unit 150 include, for example, a main processor 140, a memory 118, and a nonvolatile storage unit 121.

  The sensor hub 192 includes a processor that executes software or an element that executes a hardware program. In FIG. 6, the function of the sensor hub 192 is shown as a state determination unit 195 (determination unit) and a detection value acquisition unit 197, which are functional blocks.

  Various sensors connected to the sensor hub 192 are shown as a detection unit 170. The detection value acquisition unit 197 of the sensor hub 192 acquires detection values of various sensors that constitute the detection unit 170.

  The detection unit 170 is a sensor provided in the HMD 100 and includes a sensor that is directly or indirectly attached to the body of the user U. In particular, the detection unit 170 includes a sensor that detects the movement of the user U's body. For example, the detection unit 170 includes a 6-axis sensor 235. The 6-axis sensor 235 is a sensor that is attached to the body (head) of the user U with the left holding unit 23 of the image display unit 20 interposed therebetween.

  The detection unit 170 is not limited to a motion sensor, and may include other types of sensors. For example, the detection unit 170 may include a magnetic sensor 237. The detection unit 170 may include the camera 61. In this case, the detection value acquired by the detection value acquisition unit 197 is captured image data of the camera 61 or a signal output from the imaging element of the camera 61. The detection unit 170 may include an illuminance sensor 65. Further, the detection unit 170 may include a temperature sensor 239. The detection unit 170 may include a sensor included in the control device 10, and specifically may include a 6-axis sensor 111, a magnetic sensor 113, a GPS 115, and a touch sensor 13. The detection unit 170 may include an audio codec 180. In this case, the detection value acquired by the detection value acquisition unit 197 is digital audio data generated by the audio codec 180.

The detection value acquisition unit 197 acquires detection values of various sensors constituting the detection unit 170 at a preset sampling cycle.
The state determination unit 195 determines the body state of the user U based on the detection value acquired by the detection value acquisition unit 197. The state determination unit 195 outputs the determination result to the control unit 150. Since the state determination unit 195 repeatedly executes the determination at a preset cycle, the determination result is periodically output from the state determination unit 195 to the control unit 150. The determination cycle of the state determination unit 195 is set according to the cycle in which the detection value acquisition unit 197 acquires the detection value of the detection unit 170 and the like. In the state determination unit 195, the detection value acquisition unit 197 may perform determination using the detection values of all the sensors included in the detection unit 170, or the detection values of some sensors included in the detection unit 170. The determination may be made by using.

  The state of the body of the user U determined by the state determination unit 195 is, for example, the exercise state of the user U, the posture of the user U, and the operation performed by the user U. Specifically, the motion state of the user U is a state in which the user U is stationary, walking, running, swimming, riding on a moving body such as a vehicle, and moving. is there. In addition, the posture of the user U is standing, sitting, crouching, squatting, and the like. Moreover, the state determination part 195 may determine an exercise | movement state and the site | part of the body which is exercising, when a part of the user's U body is exercising. For example, the state determination unit 195 determines that a part of the body is moving, and identifies the moving parts as the head, arms, legs, upper body, lower body, and the like. You may determine right and left about an arm and a leg.

  In addition, the state determination unit 195 may determine that the body state of the user U is dangerous or an emergency state. For example, a state in which the user U has fallen, is in a drowning state, does not move for more than a set time in a specified posture such as prone position, and the like may be determined.

  The state determination unit 195 may determine the state of the user U's body by integrally processing detection values of a plurality of sensors constituting the detection unit 170. For example, by comparing the detection value of the 6-axis sensor 111 with the detection value of the 6-axis sensor 235, the relative position between the region where the control device 10 is mounted (for example, the waist, chest, or upper arm) and the head A change in position or relative movement may be obtained. Moreover, the state determination part 195 may perform a sensor fusion process. The sensor fusion process is a process in which a calculation process using detection values of a plurality of sensors is performed, and a kind of value that cannot be directly detected by one sensor is obtained in a pseudo manner. The state determination unit 195 can obtain a kind of value that cannot be directly detected by one or a small number of sensors of the detection unit 170 by sensor fusion processing. Further, in the sensor fusion process, it can be used also for the purpose of obtaining with higher accuracy a kind of value that can be directly detected by one or a plurality of sensors and output by the sensor as a detected value. In other words, a system value that cannot be directly detected by the sensor can be obtained by calculation processing based on the detection value of the sensor. For example, based on the detected angular velocity value output from the 6-axis sensor 235, sensor fusion processing can be performed to obtain a more accurate detected angular velocity value. The same applies to detection values of acceleration and geomagnetism, GPS position information, detection values of the illuminance sensor 65, and the like.

  In addition, the state determination unit 195 may determine the state of the user U in a multifaceted manner by processing the detection values of a plurality of sensors included in the detection unit 170 in an integrated manner. For example, the state determination unit 195 may comprehensively determine the position information that is the detected value of the acceleration and angular acceleration of the six-axis sensor 235 and the detected value of the GPS 115. In this example, the state determination unit 195 may obtain a determination result indicating that the user U is sleeping at home, resting on a sofa, sleeping on a bullet train, and sleeping on a seat. . In addition, when the state determination unit 195 determines that the user U is sleeping, the state determination unit 195 further determines the frequency of turning over and the like, and determines whether the sleep state of the user U is REM sleep or non-REM sleep. You may judge.

  The HMD 100 may include a distance sensor (not shown), and the distance sensor may be included in the detection unit 170. In this case, the detection value acquired by the detection value acquisition unit 197 is, for example, the distance detected by the distance sensor. The distance sensor can be disposed at the boundary between the right optical image display unit 26 and the left optical image display unit 28, for example. For example, in a state where the user U wears the image display unit 20, the position of the distance sensor is substantially in the middle of both eyes of the user U in the horizontal direction and above the both eyes of the user U in the vertical direction. It can be. As the distance sensor, a sensor that detects a distance to a measurement object located in a preset measurement direction can be used. For example, a distance sensor having a light source such as an LED or a laser diode and a light receiving unit that receives reflected light reflected by a measurement object from light emitted from the light source can be used. In this case, the distance sensor performs a triangular distance measurement process or a distance measurement process based on a time difference according to the control of the detection value acquisition unit 197. In addition, the distance sensor may include a sound source that emits ultrasonic waves and a detection unit that receives ultrasonic waves reflected by the measurement object. In this case, the distance sensor performs a distance measurement process based on the time difference until the reflection of the ultrasonic wave according to the control of the detection value acquisition unit 197. Note that the distance sensor may include a light source, a light receiving unit, or a sound source and a detection unit, and the state determination unit 195 may perform distance measurement processing. The measurement direction of the distance sensor is, for example, the front side direction of the HMD 100 and can be a direction overlapping with the imaging direction of the camera 61. The state determination unit 195 may determine the state of the user U using the detection value of the distance sensor and the detection value of another sensor.

  Further, the HMD 100 includes a myoelectric potential sensor attached to the body of the user U, and detects the movement of the user U's muscle by detecting a change in myoelectric potential according to the control of the detection value acquisition unit 197. It may be. The myoelectric potential sensor may be a sensor that detects the movement of the eye muscle of the user U, for example. In this case, the state determination unit 195 can detect the eye movement of the user U using a myoelectric potential sensor. The myoelectric potential sensor may be a sensor that detects the movement of the muscles of the user U's jaw. In this case, the state determination unit 195 can detect a movement of the user U opening and closing the mouth as a movement of the body using a myoelectric potential sensor. In addition, the state determination unit 195 may determine the state of the user U using the detection value of the myoelectric potential sensor and the detection value of another sensor.

  The HMD 100 may include an inner camera on the user U side of the image display unit 20. In this case, the detection value acquired by the detection value acquisition unit 197 is captured image data of the inner camera or a signal output from the image sensor of the inner camera. For example, a pair of inner cameras are provided at the center positions of the right optical image display unit 26 and the left optical image display unit 28 so as to correspond to the right eye RE and the left eye LE of the user U, respectively. it can. The inner camera can be a pair of cameras that respectively capture the right eye RE and the left eye LE of the user U. Further, for example, one inner camera may be provided at the center position of the image display unit 20. In this case, the configuration may be such that one inner camera has an angle of view capable of capturing the right eye RE and the left eye LE. For example, only one of the right eye RE or the left eye LE may be imaged by the inner camera.

  The inner camera may be configured to perform imaging in accordance with the control of the detection value acquisition unit 197. The state determination unit 195 analyzes captured image data of the inner camera. For example, the state determination unit 195 may detect reflected light and pupil images on the eyeball surfaces of the right eye RE and the left eye LE from the imaged image data of the inner camera, and specify the user's U gaze direction. Further, the state determination unit 195 may obtain a change in the direction of the line of sight of the user U, and may obtain respective eye movements of the right eye RE and the left eye LE. Further, the state determination unit 195 may extract eyelid movements of the right eye RE and left eye LE of the user U from the captured image data of the inner camera to detect eyelid movement, and the state of the eyelid May be detected. Further, the state determination unit 195 may detect the line-of-sight directions of the right eye RE and the left eye LE from the captured image of the inner camera, and obtain the convergence angles of the right eye RE and the left eye LE based on the detected line-of-sight directions. Also good. The state determination unit 195 may determine the state of the user U using the gaze direction of the right eye RE and the left eye LE, the movement of the pupil, the eyelid movement, and the detection values of other sensors.

The control unit 150 executes various processes using data stored in the storage unit 122 and controls the HMD 100.
The storage unit 122 stores a program executed by the control unit 150. That is, the storage unit 122 stores the control program 141 and the application program 125.
The control program 141 is a basic control program for the control unit 150 to control each unit of the HMD 100, and is specifically executed by the main processor 140. The control program 141 constitutes an operating system for realizing various functions of the HMD 100. The main processor 140 reads out and executes the control program 141, whereby the control unit 150 is configured. The control unit 150 includes an operating system 143 of the HMD 100. The operating system 143 includes function modules corresponding to various functions of the control unit 150. The functional modules include an image processing unit 145, a display control unit 147, an imaging control unit 149, an information acquisition unit 152, an application execution control unit 154, a display setting unit 156, and a process execution unit 158.

  The application program 125 is a program executed on the operating system 143. That is, when the main processor 140 reads and executes the application program 125, an API (Application Programming Interface) provided by the operating system 143 is called. The main processor 140 executes various operations or processes using the functions of the operating system 143 by executing the application program 125. Thus, the application program 125 can be referred to as, for example, a program that expands a function that the HMD 100 executes by the operating system 143 or adds a function of the HMD 100. A plurality of functions may be executed by the main processor 140 executing one application program 125. Alternatively, one application program 125 may be a program that realizes one function of the HMD 100.

  The entity data of the application program 125 executed by the main processor 140 is stored in the storage unit 122. Further, the entity data of the application program 125 may be downloaded from the server 3 by the HMD 100 and executed by the main processor 140. In this case, the storage unit 122 may store information for downloading the application program 125 instead of the application program 125. For example, the storage unit 122 may store a server 3 for downloading the application program 125, a URL (Uniform Resource Locator) designating a storage area, a network address, a file path, and the like.

  The HMD 100 can execute the application program 125 installed by the installation process shown in FIG. In the installation process, the application program 125 is stored in the storage unit 122 or information for downloading the application program 125 is stored. Then, the name of the application program 125 installed by the installation process is recorded in the executable list 126 as an executable program.

  A plurality of application programs 125 can be installed in the HMD 100. The storage unit 122 can store a plurality of installed application programs 125.

  The storage unit 122 stores various data processed by the control unit 150. The storage unit 122 stores setting data 123, content data 124, application program 125, executable list data 126, application information 127, application execution conditions 128, and application additional information 129.

  The setting data 123 includes various setting values related to the operation of the HMD 100. When the control unit 150 controls the HMD 100, parameters, determinants, arithmetic expressions, LUTs (Look Up Tables), and the like may be included in the setting data 123.

The content data 124 is content data including images and videos displayed by the image display unit 20 under the control of the control unit 150, and includes image data or video data. Further, the content data 124 may include audio data. The content data 124 may include image data of a plurality of images. In this case, the plurality of images are not limited to images displayed on the image display unit 20 at the same time.
The content data 124 is a bidirectional type in which when the content is displayed on the image display unit 20, the control device 10 receives an operation of the user U and the control unit 150 executes a process according to the received operation. It may be the content. In this case, the content data 124 may include image data of a menu screen that is displayed when an operation is accepted, data that defines processing corresponding to items included in the menu screen, and the like.

  The executable list 126 is a list in which application programs 125 installed in the HMD 100 and executable by the HMD 100 are registered. The executable list 126 may include data indicating whether or not the entity data of the application program 125 is stored in the storage unit 122 in addition to the executable application program 125. Information for executing the application program 125 may be included in the executable list 126. The control unit 150 can know the executable application program 125 by referring to the executable list 126.

The application information 127 stores a tag and an identifier corresponding to the application program 125 registered in the executable list 126.
FIG. 7 is a schematic diagram illustrating a configuration example of the application information 127. In the example of FIG. 7, a three-digit numerical value is used as a code for distinguishing the installed application program 125. The tag is additional information indicating the function or purpose of use of each application program 125. For example, the tag of the application program 125 with the number 001 is set as “pedometer”. Therefore, the application program 125 of number 001 is a program for the HMD 100 to realize the function of a pedometer (registered trademark). In addition, the image 011 is set as an identifier in the application program 125 of number 001. For example, when presenting to the user U that the application program 125 of number 001 can be executed, the HMD 100 displays the image 011 in the display area V. The identifier may be a character (text), a symbol, or an image, and is an image in the example of FIG. The entity of the image data of the image (for example, image 011) as the identifier is stored in the storage unit 122 in association with the application information 127, for example.

  The application execution condition 128 illustrated in FIG. 6 includes a condition for executing each application program 125 installed in the HMD 100. The application execution condition 128 includes information about the application program 125 registered in the executable list 126.

  FIG. 8 is a schematic diagram illustrating a configuration example of the application execution condition 128. In the present embodiment, the HMD 100 selects and executes the installed application program 125 according to the state of the user U's body. The application execution condition 128 can be said to be information that associates the physical state of the user U with the application program 125 to be executed.

  In the example of FIG. 8, the application execution condition 128 sets the determination result output from the state determination unit 195 and the application program 125 to be executed in association with each other. That is, the application execution condition 128 associates the output value of the sensor hub 192 with the number of the application program 125. In the example of FIG. 8, the application execution condition 128 includes characters indicating the state of the user U corresponding to the output value of the sensor hub 192, but it is not necessary to include this character information. When the state determination unit 195 determines that the state of the user U is “walking”, the output value of the sensor hub 192 is “001”. As a program corresponding to this output value, an application program 125 of number 011 is set.

  In the application execution condition 128, a plurality of application programs 125 can be set in association with the output value of one sensor hub 192. Further, one (common) application program 125 may be set in association with a plurality of output values of the sensor hub 192.

  For example, the output value “004” of the sensor hub 192 is an output value indicating that the user U moving in a preset speed range (for example, less than 10 km / h) has fallen. The output value “005” is an output value indicating that the user U moving in a preset speed range (for example, 10 km / h or more) has fallen. The output value “007” is an output value indicating that the user U is in an abnormal state during swimming. Among these, the application program 125 of number 015 is associated with each of the output values “002” and “005”. According to the application information 127 illustrated in FIG. 7, the application program 125 having the number 015 is defined as an application program for reporting a severe abnormality. According to the application execution condition 128 of FIG. 8, when the user U falls within a preset speed range, the application program 125 that makes a rescue request (rescue request, emergency report) or the like to the public institution operates. Is set to In addition, when the state determination unit 195 determines that the user U is in a state that requires notification during swimming, the same application program 125 is activated to make a rescue request to a public institution. .

  The application additional information 129 illustrated in FIG. 6 includes additional information other than the determination result of the state determination unit 195 as a condition for selecting the application program 125 installed in the HMD 100. The application additional information 129 includes information about the application program 125 registered in the executable list 126.

  FIG. 9 is a schematic diagram illustrating a configuration example of the application additional information 129. The application additional information 129 associates information used when the HMD 100 selects the application program 125 with each application program 125. Information included in the application additional information 129 is used together with the output value of the sensor hub 192 in the process of selecting the application program 125 to be executed.

  In the example of FIG. 9, the application additional information 129 includes the number of executions, evaluation, the position and size of the display area, and the cost required for execution in association with each application program 125. The number of executions is a history of executing the application program 125 in the past. The evaluation is a statistic obtained as a result of evaluation of the application program 125 by the user U and / or another user. The information included in the application additional information 129 is generated after the application program 125 is installed in the HMD 100 or when it is installed, and is updated as necessary. For example, the control unit 150 communicates with the server 3 via the communication unit 117, downloads the evaluation data of the application program 125, and updates the application additional information 129 based on the downloaded data. Data may be downloaded every time the application program 125 is executed. The position and size of the display area are the position and size of the area in the display area V where images and text related to the application program 125 are displayed when the application program 125 is executed. The position of the display area can be a relative position with respect to the entire display area V, and may be information for selecting any one of a preset upper part, center, lower part, and the like. Further, the size of the display area may be the number of pixels constituting the display area V.

  Based on the size of the display area included in the application additional information 129, the display setting unit 156 sets the size of the display area for the application program 125. The size and position of the display area set here may be a display area that can be displayed according to the operation of the user U. That is, the size and position of the display area may be determined based on the determination result of the sensor hub 192. For example, the size of the display area when the user U is stationary may be 100% of the display area V. Further, when the user U is walking, the display area may be a display area other than the user U's effective visual field (± 15 degrees from the center). In the state where the user U is running, the display area may be the remaining area in the display area V excluding the stable focus (± 30 degrees from the center).

  The cost is a fee necessary for executing or downloading the application program 125, for example. In the configuration in which the actual program of the application program 125 is downloaded from the server 3 when the application program 125 is executed, there may be a charge associated with the download. The cost included in the application additional information 129 indicates a fee charged when the application program 125 is downloaded.

  The control unit 150 includes an operating system (OS) 143, an image processing unit 145, a display control unit 147, an imaging control unit 149, an information acquisition unit 152, an application execution control unit 154, a display setting unit 156, and a process execution unit 158. Have.

The image processing unit 145 generates a signal to be transmitted to the right display unit 22 and the left display unit 24 based on image data of an image or video displayed by the image display unit 20. The signal generated by the image processing unit 145 may be a vertical synchronization signal, a horizontal synchronization signal, a clock signal, an analog image signal, or the like.
Further, the image processing unit 145 may perform resolution conversion processing for converting the resolution of the image data to a resolution suitable for the right display unit 22 and the left display unit 24 as necessary. The image processing unit 145 also performs image adjustment processing for adjusting the brightness and saturation of image data, 2D image data from 3D image data, 2D / 3D conversion processing for generating 3D image data from 2D image data, and the like. May be executed. When these image processes are executed, the image processing unit 145 generates a signal for displaying an image based on the processed image data, and transmits the signal to the image display unit 20 via the connection cable 40.

  The image processing unit 145 may be configured by hardware (for example, DSP (Digital Signal Processor)) different from the main processor 140 in addition to the configuration realized by the main processor 140 executing a program.

  The display control unit 147 generates a control signal for controlling the right display unit 22 and the left display unit 24, and controls the generation and emission of image light by the right display unit 22 and the left display unit 24 based on the control signal. . Specifically, the display control unit 147 controls the OLED drive circuits 225 and 245 to cause the OLED panels 223 and 243 to display an image. The display control unit 147 controls the timing at which the OLED drive circuits 225 and 245 draw on the OLED panels 223 and 243, controls the luminance of the OLED panels 223 and 243, and the like based on the signal output from the image processing unit 145.

  When the application program 125 is executed under the control of the application execution control unit 154, the display control unit 147 displays characters and images related to the application program 125 according to the position and size set by the display setting unit 156.

  The imaging control unit 149 controls the camera 61 to execute imaging, generates captured image data, and temporarily stores the captured image data in the storage unit 122. When the camera 61 is configured as a camera unit including a circuit that generates captured image data, the imaging control unit 149 acquires captured image data from the camera 61 and temporarily stores the captured image data in the storage unit 122. In the present embodiment, when the camera 61 is included in the detection unit 170, the camera 61 executes imaging under the control of the sensor hub 192, and the sensor hub 192 acquires an output signal or captured image data of the imaging device. The imaging control unit 149 may execute imaging via the sensor hub 192 and acquire captured image data via the sensor hub 192.

  The information acquisition unit 152 acquires data necessary for processing in which the application execution control unit 154 selects the application program 125. The information acquisition unit 152 acquires the output value of the sensor hub 192. The information acquisition unit 152 acquires necessary information from the executable list 126, the application information 127, the application execution condition 128, and the application additional information 129 stored in the storage unit 122.

  The application execution control unit 154 (execution control unit) selects the application program 125 to be executed based on the information acquired by the information acquisition unit 152. Further, the application execution control unit 154 causes the process execution unit 158 (execution unit) to execute the selected application program 125.

  The display setting unit 156 sets the position and size of the display area for displaying the application program 125 executed by the process execution unit 158. The display setting unit 156 sets the position and size of the display area based on the information included in the application additional information 129. Further, as will be described later, when the application execution control unit 154 requests confirmation input by the user U, the display setting unit 156 temporarily sets the display area so as to reduce the display area of the application program 125. Set.

  The process execution unit 158 executes the application program 125 selected by the application execution control unit 154 and instructed to be executed. If the actual program of the application program 125 to be executed is not stored in the storage unit 122, the application execution condition 128 is downloaded from the server 3 by the communication unit 117 and executed.

  FIG. 10 is a flowchart showing the operation of the HMD 100. FIG. 10 particularly shows an installation process for installing the application program 125 in the HMD 100. The installation process of FIG. 10 is an example of storing the actual program of the application program 125 in the storage unit 122.

  The control unit 150 controls the communication unit 117 to connect to the server 3 (step S11). When the application program 125 to be installed is specified by an operation detected by the operation unit 110 and installation is instructed (step S12), the control unit 150 determines whether there is a history of installing the specified application program 125 in the past. Determination is made (step S13). For example, the control unit 150 refers to the executable list 126 stored in the storage unit 122 and performs the determination in step S13. In this case, it is preferable that the executable list 126 includes data indicating a history of past installation of the application program 125 that has been uninstalled from the HMD 100 and becomes unusable.

  If there is no history of installing the application program 125 specified in step S12 in the past (step S13; No), the control unit 150 generates information related to the application program 125 (step S14). In step S14, the control unit 150 generates information related to the application program 125 specified in step S12, and adds the information to the executable list 126, application information 127, application execution condition 128, and application additional information 129.

The control unit 150 executes a process of installing the application program 125, and stores the actual program of the application program 125 in the storage unit 122 by this process (step S15).
The control unit 150 updates the information created in step S14 based on, for example, information input by an operation detected by the operation unit 110 (step S16).

  The user U may determine the association between the application program 125 installed by the HMD 100 and the output value of the state determination unit 195. For example, the user determines whether the user U uses the application program 125 corresponding to the function of presenting speed and position information to the user U during jogging while the user U is running or the user U is walking. There is a need for U to choose. Further, for example, there is a need for the user U to select whether the user U uses the application program 125 corresponding to the function of measuring the heart rate while the user U is traveling or when the user U is at rest. . For such an application program 125, the control unit 150 does not set specific information in the application execution condition 128 in step S14. In step S16, the user U performs an operation of associating the application program 125 with the output value of the sensor hub 192 or the state of the user U, and the control unit 150 executes the application based on the information input by this operation. Update condition 128.

  If the application program 125 to be installed and the state of the user U are clear, the control unit 150 may register information in the application execution condition 128 in step S14.

  If there is a history of installing the application program 125 specified in step S12 in the past (step S13; Yes), the control unit 150 associates information created in the past (step S17). In step S <b> 17, the control unit 150 searches for information corresponding to the application program 125 to be installed among the information included in the application information 127, the application execution condition 128, and the application additional information 129. And the process which matches the application program 125 designated by step S12 and the number of the application program 125 specified by the search is performed. Thereby, the information created in the past can be utilized. Thereafter, the control unit 150 proceeds to step S15.

  11 and 12 are flowcharts showing the operation of the HMD 100. FIG. 11 particularly shows the operation of selecting and executing the application program 125. FIG. 12 shows in detail the operation of step S24 of FIG.

In the HMD 100, for example, after the HMD 100 is activated, each unit is initialized, and the sensor hub 192 starts operating (step S21).
The sensor hub 192 acquires the detection value of the detection unit 170, determines the state of the user U, and outputs the determination result to the control unit 150. The control unit 150 acquires the output value of the sensor hub 192 (step S22).

  The control unit 150 determines whether or not the application program 125 is already being executed (step S23). The HMD 100 can realize so-called multitasking according to the specifications of the operating system 143. In this case, the HMD 100 can execute a plurality of application programs 125 in parallel on the operating system 143. Further, among the application programs 125, there is also an application program 125 that is executed regardless of the state of the user U. In step S23, the control unit 150 determines whether or not the application program 125 selected based on the state of the user U determined by the sensor hub 192 is already being executed (step S23).

  When it is determined that the application program 125 is not being executed (step S23; No), the control unit 150 performs an application program selection process (step S24). In step S24, the application program 125 that matches the determination result of the user U is selected.

  The control unit 150 starts executing the application program 125 selected in step S24 (step S25). The control unit 150 executes the application program 125 and causes the image display unit 20 to display an image and text related to the function of the application program 125. For example, the application program 125 of number 011 illustrated in FIG. 7 corresponds to the pedometer function. When this application program 125 is executed, the control unit 150 causes the image display unit 20 to display an image indicating the count value of the pedometer. Further, for example, when executing the application program 125 for performing cycling recording, the control unit 150 displays a screen on which the GPS coordinates of the current position of the user U, the traveling speed, the traveling direction, contents recorded in the past, and the like are arranged. The image is displayed by the image display unit 20. For example, when executing the translation application program 125, the control unit 150 causes the image display unit 20 to display an image related to the translation function. Specifically, the control unit 150 performs speech recognition on the speech collected by the microphone 63, a language determined from the speech recognized text, a translation processing result obtained by translating the speech recognized text into a set language, and the like. A screen on which the text and image are arranged is displayed by the image display unit 20.

  Thereafter, the control unit 150 determines whether or not to end the operation (step S26). When the execution stop of the application program 125 or the operation stop of the HMD 100 is instructed by an operation detected by the operation unit 110 or the like, the control unit 150 determines that the operation is ended (step S26; Yes), and ends this process. Moreover, when it determines with continuing operation | movement (step S26; No), the control part 150 returns to step S22.

  In addition, when it is determined that the application program 125 selected based on the state of the user U is already being executed (step S26; Yes), the control unit 150 determines whether or not the output value of the sensor hub 192 has changed. (Step S27). In step S27, the control unit 150 determines whether or not the output value of the sensor hub 192 when the application program 125 is selected based on the state of the user U is different from the output value acquired in step S22. In step S27, the control unit 150 determines the state of the user U that is similar to each other even if the output value of the sensor hub 192 when the application program 125 is selected does not completely match the output value acquired in step S22. If so, it is determined that there is no change.

When it determines with there being no change of an output value (step S27; No), the control part 150 returns to step S22.
If it is determined that the output value has changed (step S27; Yes), the control unit 150 starts monitoring the output value of the sensor hub 192 (step S28).
The controller 150 determines whether or not the output value after determining that the output value of the sensor hub 192 has changed in Step S27 has continued for a predetermined time or more (Step S29). In step S29, even if the same output value does not continue, if it is an output value indicating the state of a similar user U, it is determined that it is continued. When the output value after the change continues for the set time or longer (step S29; Yes), the control unit 150 proceeds to an application program selection process (step S24). Thereby, when the state of the user U changes, the application program 125 suitable for the state of the user U after the change can be selected again.

  Further, when the control unit 150 determines that the output value of the sensor hub 192 after determining that it has changed in step S27 has not continued for a predetermined time or longer (step S29; No), the control unit 150 returns to step S22. Thereby, when the state of the user U changes temporarily, since the application program 125 to be executed is not changed, the operation of the HMD 100 can be stabilized.

The application program selection process shown in step S24 will be described in detail.
In step S31 of FIG. 12, the control unit 150 determines whether the list display setting is turned on (step S31). The list display setting is a setting for performing an operation of displaying a list of application programs 125 suitable for the state of the user U in the display area so that the user U can see them. When the list display setting is on, the user U can select the application program 125 by looking at the tag or identifier of the application program 125 displayed as a list in the display area. Then, the application program 125 selected by the user U is executed. The setting state (ON or OFF) of the list display setting is included in the setting data 123 stored in the storage unit 122, for example.

When the list display setting is off (step S31; No), the control unit 150 selects the application program 125 set in the application execution condition 128 according to the output value of the sensor hub 192 acquired in step S22 ( Step S32).
In step S32, the control unit 150 selects any one application program 125. When there are a plurality of application programs 125 set in the application execution condition 128 corresponding to the output value of the sensor hub 192, the control unit 150 uses any one of the applications using the additional information registered in the application additional information 129. The program 125 is selected. A plurality of additional information registered in the application additional information 129 may be weighted in advance. In this case, for example, the control unit 150 quantifies a plurality of additional information registered in the application additional information 129, and calculates a score of each application program 125 based on the quantified additional information and a weighting value. The control unit 150 selects one application program 125 according to the calculated score.

  The control unit 150 causes the image display unit 20 to display an execution availability screen (step S33). The execution availability input screen is a screen including characters and images that request the user U to confirm whether or not the selected application program 125 can be executed. In step S33, the function of the display setting unit 156 reduces the size of the display area for display related to the application program 125, and moves the display position to a position that does not interfere with the field of view.

  After displaying the execution enable / disable input, the control unit 150 starts detecting the executable input (step S34), and waits until no executable input is made (step S34; No). When the executable input is input (step S34; Yes), the control unit 150 proceeds to step S25 in FIG. 11 and executes the application program 125.

  The display of the availability input and the standby for input in steps S33 to S34 may be omitted. Further, whether to execute steps S33 to S34 may be set depending on the type of the selected application program 125. For example, steps S33 to S34 may be omitted when the application program 125 (number 015 in FIG. 7) for requesting rescue in case of an abnormality or emergency is selected. Moreover, you may set to the application execution conditions 128 whether steps S33-S34 are abbreviate | omitted or performed.

  When the list display setting is ON (step S31; Yes), the control unit 150 selects a plurality of application programs 125 set in the application execution condition 128 corresponding to the output value of the sensor hub 192 acquired in step S22. (Step S35). The control unit 150 causes the image display unit 20 to display a list of tags and / or identifiers of the selected application program 125 (step S36). The control unit 150 detects the operation of the user U by the operation unit 110 (step S37). Since the operation detected in step S37 is an operation for designating the application program 125, one operation program 125 is specified by this operation. Thereafter, the control unit 150 proceeds to step S33.

  FIG. 13 is a diagram illustrating a display example of the image display unit 20. FIG. 13 shows an example of a state in which a list of application programs 125 executed in step S36 (FIG. 12) is displayed. In the figure, the symbol VR indicates the field of view of the user U wearing the image display unit 20. Reference numeral V is an area where the image can be visually recognized by the user U by the half mirrors 261 and 281, in other words, a displayable area where the image display unit 20 can display an image.

  In the example of FIG. 13, identifiers 301, 302, and 303 are displayed side by side in the display area V located in the visual field VR. An identifier 301 is an identifier (image) of the application program 125 that provides the function of the heart rate monitor. The identifier 302 is an identifier of the application program 125 that provides a function for displaying the moving speed in a meter. An identifier 303 is an identifier of the application program 125 that provides a function for displaying a bearing. In this manner, by displaying identifiers such as images for selectable application programs 125 and prompting the user U to select, the user U can select the application program 125 intuitively and promptly.

  In a conventional head-mounted display device or a device that realizes a function by executing an application program such as a smartphone, the user U needs to perform an operation in order to select the application program. This operation has many operation procedures in order to select an arbitrary application from among a plurality of applications installed in the apparatus by visual inspection or the like and determine an application program to be executed. When an operation with such a conventional apparatus is performed while the user U is walking or performing other operations, the user U may feel difficulty in operation because there are many operation procedures. In addition, it is difficult for the user U to concentrate on the operation of the apparatus, making it difficult to focus on other actions, and the possibility of a problem in safety cannot be denied. In this regard, the HMD 100 according to the present embodiment requires fewer operations for selecting the application program 125. Further, based on the information registered in the application information 127, the application execution condition 128, and the application additional information 129, the application program 125 can be selected and started without any operation. Thereby, it becomes easy for the user U to execute the application program 125 by the HMD 100 while walking or performing other operations, and the operability of the HMD 100 can be improved.

  The control unit 150 may detect the occurrence of an external factor by the sensor hub 192 while the processing execution unit 158 is executing the application program 125. Further, the control unit 150 may detect the occurrence of abnormality by the sensor hub 192 based on the detection value of the detection unit 170. In this case, the configuration may be such that when the occurrence of an external factor or abnormality is detected, the process execution unit 158 executes a preset interrupt process. In this case, interrupt processing can be executed when an external factor is generated or an abnormal value is detected during execution of the function. The content of the interrupt processing may be set according to the type of the application program 125 being executed or for each individual application program 125. For example, for an application program 125 that uses a large display area, such as an application program 125 that displays a map and performs route guidance (navigation), an interrupt process for reducing the display area and newly displaying an abnormality notification screen is performed. May be executed. Further, interrupt processing may be prohibited during execution of the rescue request application program 125. Further, when the external factor is resolved, the HMD 100 may return to the execution state of the original application program 125.

Further, the selection and selection of the application program 125 may take into account the movement and / or state of the user U of another HMD 100.
That is, while the user UA is using the HMD 100A, the application program 125 may be selected using the determination result determined by the state determination unit 195 by the HMD 100B used by the user UB. The method for the HMD 100A to acquire the determination result of the HMD 100B includes a method in which the communication unit 117 performs communication between the HMDs 100A and 100B, but a captured image of the camera 61 may be used. The HMD 100B blinks the LED indicator 67 in a pattern corresponding to the determination result, and the control unit 150 of the HMD 100A detects blinking of the LED indicator 67 of the HMD 100B from the captured image data of the camera 61, analyzes it, and calculates the determination result May be.

  A specific example is given. The HMDs 100A and 100B may collect the voices of the users UA and UB through the voice interface 182 and detect voices uttered by the users UA and UB from the collected voices. In this case, the HMDs 100A and 100B may specify the languages spoken by the users UA and UB by the state determination unit 195, respectively. The determination result in this case is data indicating that the users UA and UB are speaking and the language. The HMD 100A specifies the language of the user UB from the determination result of the HMD 100B, selects and executes the application program 125 that translates this language into the language of the user UA specified by the HMD 100A. Thereby, it is possible to communicate between different language users UA and users UB using the application program 125 for translation. In this case, the HMD 100B may select and start the translation application program 125 in the same manner as the HMD 100A.

Further, when the HMD 100A has image information and position information of recommended tourist spots as experience information of the user UB, it is possible to realize an aspect in which these pieces of information are shared. In this case, the HMD 100A can acquire image information and position information from the HMD 100B, select and execute the route guidance application program 125, and perform route guidance to the tourist spot for the user UA.
In addition, when the HMD 100A and the HMD 100B execute a battle-type game, it is possible to assume an aspect in which the HMD 100A acquires and shares information such as the level and handy of the user UB that the HMD 100B has. In this case, when the HMD 100B determines that the user UB is playing a game, the HMD 100A acquires the determination result and executes the application program 125 of the same game in the HMD 100A. Then, the HMD 100A sets information such as the level and handy of the user UB in the HMD 100B as the execution condition of the game application program 125. As a result, the users UA and UB can easily find each other an opponent who plays a battle game, can execute a game reflecting the opponent's level and the like, and can enhance the interest.

  Further, as described above, when the determination result of the state determination unit 195 is a result that the user U is sleeping at home, the application program 125 that executes the power saving function may be activated. In this case, it is possible to save power by stopping the operation of the function unit, for example, GPS 115, which is not required to be operated during the sleep of the user U.

  In addition, when the determination result of the state determination unit 195 is a result indicating that the user U is in a public facility, the determination function of the state determination unit 195 based on the captured image data of the camera 61 from the viewpoint of public nature It is also possible to turn on the imaging function by operating the operation unit 110. In this case, the execution state of the functions of the HMD 100 can be controlled from the viewpoint of privacy protection and the like. Further, the execution state of the application program 125 being executed may be controlled from the viewpoint of energy saving in accordance with the change in the state of the user U. For example, when the HMD 100 determines that the user U is at home, the application program 125 may perform a function of stopping execution and / or restricting start of execution of the application program 125 that is not frequently used at home. Good. In addition, when the state of the user U is determined to be a state in which an application program 125 used in an emergency, such as an application program 125 that makes a rescue request, is determined, a specific function may be turned on. Specifically, WiFi communication by the communication unit 117, and the communication function via the 3G, 3.9G, 4G (LTE: Long Term Evolution) network by the cellular communication unit (not shown) are turned on, and the notification is performed. A mode in which the application program 125 that transmits the biometric information of the user U is executed first is also conceivable.

  Further, the control unit 150 may select the application program 125 based on an appropriate display size corresponding to the state of the user U. For example, when the state determination unit 195 determines that the state of the user U wearing the HMD 100 is a stationary state at home or the like, the application execution control unit 154 displays a screen over the entire display area of the image display unit 20. The application program 125 to be selected may be selected. Specifically, the application execution control unit 154 selects an application program 125 that performs a so-called full-screen display that displays a screen including text and images related to the application program 125 over the entire display area. Alternatively, the application program 125 whose display size is set to the maximum is selected. On the other hand, when the state determination unit 195 determines that the state of the user U is an outing place (moving state by train or car), the application execution control unit 154 displays an application program with a narrowed angle of view. 125 is selected. Narrowing the angle of view means, for example, displaying the screen in the display area with a display size that allows the user U to see a little outside scene through the image display unit 20. In addition, when the state determination unit 195 determines that the state of the user U is a walking state or a running state, the application execution control unit 154 displays an image centered so as not to obstruct the user U's field of view. An application program 125 is selected which is set to a peripheral display mode in which no icons are arranged and no central area is displayed. For example, a state in which no image or text is arranged in a predetermined range including the center of the display area of the image display unit 20 and no background color is displayed in the predetermined range can be referred to as center area non-display.

  In particular, a map display application program 125 for displaying a map, or a navigation application program 125 for displaying a current position, a destination, or a movement route of the HMD 100 together with a map are given as examples. When the application execution control unit 154 selects such an application program 125, when the user U is moving, the application program 125 that displays in the above-described manner of not displaying the central region may be selected. In this case, the corresponding application program 125 may be selected from a plurality of map display application programs 125 or a plurality of navigation application programs 125. On the other hand, when the user U stops and faces down, the application program 125 for map display or navigation that performs the above-described full screen display may be selected.

  Further, the brightness of the image displayed on the image display unit 20, that is, the amount of image light incident on the user's U eye from the right display unit 22 and / or the left display unit 24 is combined with the display size or independently. Then, the application execution control unit 154 may be a criterion for selecting the application program 125. The application execution control unit 154 calculates or acquires an appropriate luminance or light amount from the state of the user U determined by the state determination unit 195, and selects an application program 125 corresponding to the luminance or light amount. For example, when the user U is moving, the application program 125 having a small luminance or light quantity is selected so that the outside scene can be easily seen. Furthermore, in addition to the selection of the application program 125, for example, when there is a movement in the state determined by the state determination unit 195, the right display unit 22 and the left display unit 24 are so far that the user U can visually recognize the outside scene. Subtle adjustments such as lowering the luminance (light quantity) of the emitted image light are also possible in the HMD 100 having the see-through image display unit 20.

  Similarly, for the voice, the application execution control unit 154 is walking whether the state of the user U determined by the state determination unit 195 is a stationary state, a place where the user U is away (moving state by train or car), or the like. Adjustments may be made depending on whether the vehicle is in a state or running. In this case, the application execution control unit 154 executes the application program 125 by performing processing such as lowering the volume when the application program 125 is executed or changing from both ears to one ear output. An application program 125 that performs output may be selected.

  Further, the control unit 150 may control the display change when the application program 125 for starting or executing the application program 125 is switched by the function of the display control unit 147. In addition, when performing so-called animation display in which an image is changed during execution of the application program 125, the mode of animation display may be controlled. In this case, the display control unit 147 may change the display switching speed and the animation change speed in accordance with the state of the user U determined by the state determination unit 195, the moving speed of the user U, and the like. As a parameter for determining the rate of change, for example, an inertia force, a time constant, or the like may be used. The inertia force is a parameter for changing the display change speed during the display change, and may be determined in accordance with the moving speed of the user U detected by the control unit 150. The time constant is a parameter for determining the display change speed, and may be determined according to the moving speed of the user U detected by the control unit 150. For example, the state determination unit 195 detects the moving speed of the user U, determines a time constant, and changes the display at a speed corresponding to the time constant. Then, during the display change, the control unit 150 detects the moving speed of the user U and determines the inertial force. The controller 150 changes the display change speed based on the determined inertial force. The time constant and the inertial force may be set in advance before the display change is started. Thereby, the speed at which the screen and the image appear on the image display unit 20 and the speed at which the image is retracted are adjusted, and for example, it is possible to prevent a phenomenon in which the image suddenly appears and blinds. In addition, the control unit 150 immediately erases (stops) the display of the image display unit 20 or the image display when the user U is in a state where danger avoidance is considered necessary based on the determination result of the state determination unit 195. An image or the like may be saved in the periphery of the display area of the unit 20. When the display is changed in this way, the speed can be changed depending on the state of the user U.

  As described above, the HMD 100 configuring the display system 1 according to the embodiment to which the present invention is applied includes the image display unit 20 that displays an image and the detection unit 170 that is directly or indirectly attached to the body of the user U. With. The HMD 100 includes a state determination unit 195 that determines the state of the body of the user U based on the detection result of the detection unit 170, and a process execution unit 158 that executes functions including display on the image display unit 20. The HMD 100 includes an application execution control unit 154 that selects a function corresponding to the determination result of the state determination unit 195 from functions that can be executed by the process execution unit 158 and causes the process execution unit 158 to execute the function. In this configuration, the HMD 100 selects and executes a function in accordance with the state of the user U's body. Therefore, an appropriate function is executed without the user U performing an operation such as selecting a function. For example, since the user U can cause the HMD 100 to execute a function without using a hand, the convenience of the HMD 100 can be improved.

  In addition, the state determination unit 195 identifies the body state of the user U from a plurality of preset states based on the detection result of the detection unit 170, and the application execution control unit 154 includes the state determination unit 195. The process execution unit 158 is caused to execute a function corresponding to the identified state. For this reason, according to the setting, the function suitable for the state of the user U's body can be selected more appropriately and promptly.

  Further, the process execution unit 158 causes the image display unit 20 to execute display including at least one of an image and text related to the application program 125 by executing the application program 125. Thereby, the image and text regarding the application program 125 selected according to the state of the user's U body can be displayed, and the convenience can be improved.

  The HMD 100 also includes a storage unit 122 that stores an application program 125 corresponding to the function executed by the process execution unit 158. The application execution control unit 154 selects a function that can be executed by the process execution unit 158 based on the application program 125 stored in the storage unit 122 and causes the process execution unit 158 to execute the function. Thereby, in the configuration in which the HMD 100 executes the application program 125 stored in the storage unit 122 to realize the function, the function suitable for the state of the user U's body can be appropriately selected.

  Further, when installing the application program 125 in the HMD 100, the application execution control unit 154 stores the installed application program 125 in the storage unit 122 in association with the body state of the user U. Thereby, the application program 125 installed in the HMD 100 can be associated with the body state of the user U. For this reason, the installed application program 125 can be appropriately selected according to the state of the user U's body.

  The HMD 100 also includes a communication unit 117 that communicates with the server 3. The application execution control unit 154 may select a function that can be executed by the process execution unit 158 based on the application program 125 that can be acquired from the server 3 by the communication unit 117. In this case, the application execution control unit 154 downloads (receives) the application program 125 corresponding to the selected function through the communication unit 117. The HMD 100 may be configured to cause the process execution unit 158 to execute a function corresponding to the program received by the communication unit 117. In this case, in a configuration in which the function is realized by a program acquired from the server 3, a function suitable for the state of the user U's body can be appropriately selected.

  Further, the application execution control unit 154 stores various types of information, such as the application additional information 129 illustrated in FIG. 9, for each function that can be executed by the process execution unit 158. For example, these pieces of information include at least any of the following conditions: the number of executions of the function, the evaluation of the function, the size of the display area in the image display unit 20 required when the function is executed, and the cost associated with the execution of the function. The HMD 100 may be configured to acquire priority information indicating the above-described conditions and the weighting set for each of the above-described conditions, and select a function based on the acquired conditions and priority information. Thereby, the function can be selected more appropriately by combining the state of the user U's body and the conditions related to the execution of each function.

  Further, the HMD 100 may be configured to store the application execution condition 128 and the application additional information 129 in the storage unit 122. In this configuration, the HMD 100 stores conditions relating to the execution of each function and information for weighting the conditions, and functions can be selected quickly based on the stored information.

  Further, the application execution control unit 154 may be configured to acquire the application execution condition 128 and the application additional information 129 from the server 3. In this case, the condition regarding the execution of each function and the information for weighting the condition are acquired from the server 3, and the function can be selected promptly based on the stored information.

  Further, when selecting a function, the application execution control unit 154 sets the size of the display area used for displaying the function in the image display unit 20 by the function of the display setting unit 156. The application execution control unit 154 executes display related to the function in the set display area, and executes display related to the function in the display area smaller than the set size before executing the function in the process execution unit 158. It waits for the input of availability. Thus, the size of the display area when executing the function is set, and before executing the function, the display size related to the function is reduced, and input of whether to execute the function is waited. For this reason, the display size related to the function can be set appropriately, and when the user U inputs whether or not to execute, the operability of the user U can be improved by reducing the display.

  Further, as described with reference to the example of the application information 127 in FIG. 7, a tag or an identifier is assigned to each function that can be executed by the process execution unit 158. Prior to causing the process execution unit 158 to execute the function, the application execution control unit 154 displays a plurality of functions together with tags or identifiers, and waits for input regarding whether the function is selected and executed. Thereby, a function can be selected according to the input of the user U. Moreover, the function suitable for the state of the user's U body can be selected based on a tag and an identifier.

  Further, the application execution control unit 154 sets and sets the size of the display area V (display area) used for displaying the function in the image display unit 20 based on the state of the user U determined by the state determination unit 195. The display related to the function is executed in the display area V. Thereby, the size of the display regarding a function can be set appropriately according to the state of the user U.

  Further, the application execution control unit 154 sends a preset interrupt process to the process execution unit 158 when an external factor is generated or an abnormal value is detected by the detection unit 170 while the process execution unit 158 is executing the function. It may be configured to be executed. In this case, interrupt processing can be executed when an external factor is generated or an abnormal value is detected during execution of the function.

  Moreover, the state determination part 195 determines the state of the user U for every preset time. When the state of the user U determined by the state determination unit 195 has changed, the application execution control unit 154 has previously changed within the time (predetermined time) when the state of the user U determined by the state determination unit 195 is set. It is determined whether or not to return to the determined state. The application execution control unit 154 changes the function executed by the process execution unit 158 when it is determined that the state determined by the state determination unit 195 does not return within a predetermined time. Thereby, when the state of the user's U body changes, the function performed according to this change can be changed, and when the necessity to change is small, execution of a function can be continued. For this reason, inconvenience due to frequent function changes can be avoided, and changes in the state of the user U's body can be accommodated.

In addition, this invention is not restricted to the structure of the said embodiment, In the range which does not deviate from the summary, it can be implemented in a various aspect.
For example, in the above embodiment, the configuration in which the control device 10 is connected to the image display unit 20 by wire is illustrated, but the present invention is not limited to this, and the image display unit 20 is wirelessly connected to the control device 10. It may be a configuration. In this case, the wireless communication method may be the method exemplified as the communication method supported by the communication unit 117, or may be another communication method.

  In addition, some functions of the control device 10 may be provided in the image display unit 20, and the control device 10 may be realized by a plurality of devices. For example, instead of the control device 10, a wearable device that can be attached to the body of the user U, clothes, or an accessory worn by the user U may be used. The wearable device in this case may be, for example, a watch-type device, a ring-type device, a laser pointer, a mouse, an air mouse, a game controller, a pen-type device, or the like.

  Furthermore, in the above embodiment, the configuration in which the image display unit 20 and the control device 10 are separated and connected via the connection cable 40 has been described as an example. The present invention is not limited to this, and the control device 10 and the image display unit 20 may be configured integrally and mounted on the head of the user U.

  In the above embodiment, the configuration in which the user U visually recognizes the outside scene through the display unit is not limited to the configuration in which the right light guide plate 26 and the left light guide plate 28 transmit the external light. For example, the present invention can be applied to a display device that displays an image in a state where an outside scene cannot be visually recognized. Specifically, the present invention is applied to a display device that displays a captured image of the camera 61, an image or CG generated based on the captured image, video based on video data stored in advance or video data input from the outside, and the like. Can be applied. This type of display device can include a so-called closed display device in which an outside scene cannot be visually recognized. In addition, as described in the above embodiment, AR display that displays an image superimposed on real space, MR (Mixed Reality) display that combines a captured real space image and a virtual image, or VR (VR that displays a virtual image) It can also be applied to a display device that does not perform processing such as virtual reality display. For example, a display device that displays video data or an analog video signal input from the outside is naturally included as an application target of the present invention.

  Further, for example, instead of the image display unit 20, an image display unit of another method such as an image display unit worn like a hat may be adopted, and an image corresponding to the left eye LE of the user U is used. And a display unit that displays an image corresponding to the right eye RE of the user U. The display device of the present invention may be configured as a head mounted display mounted on a vehicle such as an automobile or an airplane. Further, for example, it may be configured as a head-mounted display built in a body protective device such as a helmet. In this case, the portion for positioning the position of the user U with respect to the body and the portion positioned with respect to the portion can be used as the mounting portion.

Further, as an optical system that guides image light to the eyes of the user U, a configuration in which virtual images are formed by the half mirrors 261 and 281 on a part of the right light guide plate 26 and the left light guide plate 28 is illustrated. The present invention is not limited to this, and an image may be displayed on a display region having an area that occupies the entire or most of the right light guide plate 26 and the left light guide plate 28. In this case, a process of reducing the image may be included in the operation of changing the display position of the image.
Furthermore, the optical element of the present invention is not limited to the right light guide plate 26 and the left light guide plate 28 having the half mirrors 261 and 281, and may be any optical component that allows image light to enter the user's U eye. In this case, a diffraction grating, a prism, or a holographic display unit may be used.

  In addition, at least a part of each functional block shown in the block diagram may be realized by hardware, or may be realized by cooperation of hardware and software, and independent as shown in the figure. The configuration is not limited to the arrangement of the hardware resources. The program executed by the control unit 150 may be stored in the nonvolatile storage unit 121 or another storage device (not shown) in the control device 10. A program stored in an external device may be acquired and executed via the communication unit 117 or the external connector 184. Of the components formed in the control device 10, the operation unit 110 may be formed as a user interface (UI). Further, the configuration formed in the control device 10 may be formed in the image display unit 20 in an overlapping manner. For example, a processor similar to the main processor 140 may be disposed in the image display unit 20, or the main processor 140 and the processor of the image display unit 20 included in the control device 10 perform functions separately divided. Also good.

  DESCRIPTION OF SYMBOLS 1 ... Display system, 3 ... Server (external apparatus), 10 ... Control apparatus, 20 ... Image display part (display part), 22 ... Right display unit, 24 ... Left display unit, 30 ... Headset, 32 ... Right earphone 34 ... Left earphone, 40 ... Connection cable, 61 ... Camera, 63 ... Microphone, 65 ... Illuminance sensor, 100 ... HMD (display device, head-mounted display device), 110 ... Operation unit, 111 ... 6-axis sensor, DESCRIPTION OF SYMBOLS 113 ... Magnetic sensor, 115 ... GPS, 117 ... Communication part, 118 ... Memory, 120 ... Controller board | substrate, 121 ... Nonvolatile memory | storage part, 122 ... Memory | storage part (memory | storage part, condition memory | storage part), 123 ... Setting data, 124 ... Content data 125 ... Application program (program), 126 ... Executable list, 127 ... Application information, 128 ... Application execution condition (Conditions) 129 ... Application additional information (priority information), 130 ... Power supply unit, 132 ... Battery, 134 ... Power supply control circuit, 140 ... Main processor, 141 ... Control program, 143 ... Operating system, 145 ... Image processing unit, 147 ... Display control section, 149 ... Imaging control section, 150 ... Control section, 152 ... Information acquisition section, 154 ... Application execution control section (execution control section), 156 ... Display setting section, 158 ... Process execution section (execution section) , 170 ... detection unit, 180 ... audio codec, 182 ... audio interface, 184 ... external connector, 186 ... external memory interface, 188 ... USB connector, 192 ... sensor hub, 194 ... FPGA, 195 ... status determination unit, 196 ... interface 197 ... Detection value acquisition unit 211 ... Interface 213: Receiver, 215 ... EEPROM, 217 ... Temperature sensor, 221 ... OLED unit, 231 ... Interface, 233 ... Receiver, 235 ... 6-axis sensor, 237 ... Magnetic sensor, 239 ... Temperature sensor, 241 ... OLED unit, U, UA, UB ... user, V ... display area.

Claims (17)

A display for displaying an image;
A detection unit attached directly or indirectly to the user's body;
A determination unit that determines a state of the user's body based on a detection result of the detection unit;
An execution unit for executing functions including display in the display unit;
Among the functions that can be executed by the execution unit, an execution control unit that selects and executes the function corresponding to the determination result of the determination unit;
A display device comprising: The determination unit specifies the state of the user's body from a plurality of preset states based on the detection result of the detection unit,
The display device according to claim 1, wherein the execution control unit causes the execution unit to execute the function corresponding to the state specified by the determination unit.   The display device according to claim 2, wherein the execution unit causes the display unit to execute display including at least one of an image and text related to the function by executing the function. A storage unit for storing a program corresponding to the function executed by the execution unit;
The display device according to claim 1, wherein the execution control unit selects the function executable by the execution unit based on a program stored in the storage unit and causes the execution unit to execute the function. .   The display device according to claim 4, wherein, when the program is installed in the display device, the execution control unit stores the installed program in the storage unit in association with a state of the user's body. It has a communication unit that communicates with external devices,
The execution control unit selects the function executable by the execution unit based on a program that can be acquired from the external device by the communication unit, and receives the program corresponding to the selected function by the communication unit. The display device according to claim 1, wherein the execution unit executes the function corresponding to the received program.   The execution control unit, for each of the functions that can be executed by the execution unit, the number of executions of the function, the evaluation of the function, the size of the display area in the display unit required when the function is executed, and the function Claims that acquire priority information indicating at least one of the conditions associated with the execution of the cost and the weight set for each of the conditions, and select the function based on the acquired condition and the priority information. Item 7. The display device according to any one of Items 1 to 6.   The display device according to claim 7, further comprising a condition storage unit that stores the condition and the priority information.   The display device according to claim 7, wherein the execution control unit acquires the condition and the priority information from an external device. The execution control unit sets the size of a display area used for displaying the function on the display unit when selecting the function, and causes the set display area to display the function.
The display according to the function is displayed in the display area smaller than a set size before the execution unit executes the function, and an input as to whether execution is possible is waited for. The display device described. A tag or identifier is given to each of the functions that can be executed by the execution unit,
The execution control unit displays a plurality of the functions together with the tag or the identifier so that the execution unit can execute the functions, and waits for the input regarding the selection and execution of the functions. Item 11. A display device according to Item 10.   The execution control unit sets a size of a display area used for displaying the function on the display unit based on the state of the user determined by the determination unit, and displays the function according to the set display region. The display device according to claim 1, wherein:   The execution control unit causes the execution unit to execute a preset interrupt process when an external factor is generated or an abnormal value is detected by the detection unit while the execution unit is executing the function. 13. The display device according to any one of 1 to 12. The determination unit determines the state of the user every preset time,
The execution control unit determines whether or not the state of the user determined by the determination unit returns to a previously determined state within a predetermined time when the state of the user determined by the determination unit changes. The display device according to claim 1, wherein the function executed by the execution unit is changed when it is determined that the function is not returned. A display unit that is mounted on the user's head and displays an image so that the outside scene can be visually recognized;
A detection unit mounted directly or indirectly on the user's body;
A determination unit that determines a state of the user's body based on a detection result of the detection unit;
An execution unit for executing functions including display in the display unit;
Among the functions that can be executed by the execution unit, an execution control unit that selects and executes the function corresponding to the determination result of the determination unit;
A head-mounted display device comprising: Controlling a display device comprising a display unit that displays an image and a detection unit that is mounted directly or indirectly on the user's body,
A determination step of determining a state of the user's body based on a detection result of the detection unit;
A selection step of selecting the function corresponding to the determination result of the determination step among a plurality of executable functions;
An execution step of executing the function including the display on the display unit selected in the selection step. A computer that controls a display device that includes a display unit that displays an image and a detection unit that is directly or indirectly attached to the user's body,
A determination step of determining a state of the user's body based on a detection result of the detection unit;
A selection step of selecting the function corresponding to the determination result of the determination step among a plurality of executable functions;
An execution step of executing the function including the display on the display unit selected in the selection step;
A program for running

Images