JP5953714B2 - Device, head-mounted display device, device control method, and head-mounted display device control method - Google Patents

Description

  The present invention relates to a device, a head-mounted display device, a device control method, and a head-mounted display device control method.

  A head-mounted display device (head mounted display) that is a display device to be mounted on the head is known. The head-mounted display device, for example, generates image light representing an image using a liquid crystal display and a light source, and guides the generated image light to a user's eye using a projection optical system or a light guide plate This causes the user to visually recognize the virtual image.

  In a head-mounted display device, it is preferable that the user can visually recognize an input device such as a mouse or a keyboard in addition to visually recognizing the generated virtual image. Therefore, a sensor that can sense the position of the user's hand (for example, an infrared sensor) Etc.), and a technique for changing the position of a virtual image to be visually recognized according to the position of a user's hand is known (see, for example, Patent Document 1).

  Further, in the head-mounted display device, it is required to reduce power consumption. In particular, in a head-mounted display device driven by a battery in order to improve portability, it is more required to reduce power consumption in order to extend the usable time. In this regard, a head-mounted display device that does not require a light-emitting light source in order to reduce power consumption is known (for example, see Patent Document 2).

JP-A-6-314331 JP-A-7-209600 JP 2009-192583 A JP 2006-105889 A JP 2000-353046 A

  However, in the conventional head-mounted display device described above, the user needs to use his hand for each operation, and when the display is not necessary, a button operation is required in advance. There was room for improvement. Such a problem is not limited to the head-mounted display device, but is a problem common to all devices.

  In addition, the conventional head-mounted display device has a problem that a clear image cannot be seen in an environment where there is not enough extraneous light because there is no light emitting light source.

  The present invention has been made to solve the above-described problems, and a first problem is to provide a technique that allows a user to operate the apparatus without using a hand. A second problem is to provide a technique for suppressing power consumption without degrading the quality of a display image of a head-mounted display device.

In order to solve at least a part of the above problems, the present invention can be realized as the following forms or application examples. One embodiment of the present invention is an apparatus; a detection unit that detects an eyelid state including an open / closed state of a user's right eyelid and an open / closed state of a left eyelid at a plurality of predetermined timings; provided with a control unit for instructing said corresponding eyelid state, a; wherein the control unit; first and eyelids state wherein the detection unit is a pre-Symbol eyelid state that put the first timing detected, the second second and eyelids state, the case where only differ, the first eyelid with said second of each pattern a predetermined rule defined from the combination of the eyelid condition is before Symbol eyelid state that put the timing According to the input information and performing a predetermined operation according to the input information; a predetermined motion corresponding to a combination of the eyelid states when the first eyelid state and the second eyelid state are the same The not performed. In such a form, the detection unit detects the eyelid pattern, and the control unit converts the instruction assigned to the pattern into input information according to a predetermined rule, so a plurality of instructions assigned to the pattern Can be executed and convenience can be improved. In addition, the present invention can be realized as the following forms.

[Application Example 1] An apparatus,
A detection unit for detecting the state of the user's eyelid;
A control unit that gives instructions according to the state of the user's eyelid detected by the detection unit.

  In this apparatus, the detection unit detects the state of the user's eyelid, and the control unit gives an instruction according to the state of the eyelid, so that the user can operate without using a hand and improve operability. be able to.

[Application Example 2] A head-mounted display device,
An image display unit that generates and emits image light based on image data, and a light guide unit that guides the image light to the user's eyes, and allows the user to visually recognize a virtual image;
The apparatus according to Application Example 1 connected to the image display unit,
The control unit transmits the image data to the image display unit, controls image display by the image display unit, and has a plurality of different power consumptions according to the state of the user's eyelid detected by the detection unit. A head-mounted display device that switches the operation mode.

  In this head-mounted display device, the detection unit detects the state of the user's eyelid. The control unit switches to a plurality of operation modes with different power consumption according to the detected eyelid state. Therefore, the head-mounted display device can reduce power consumption by operating in an appropriate operation mode according to the state of the user's eyelid.

[Application Example 3] The head-mounted display device according to Application Example 2,
The control unit performs a power saving transition, which is a transition to an operation mode with a lower power consumption, when the detection unit detects that the user's eyelid is kept closed for a predetermined time. apparatus.

  If the user's eyelid is closed for a predetermined time, it is highly likely that the user has not seen the image. In this head-mounted display device, the control unit shifts to power saving, which is a shift to an operation mode with lower power consumption when the detection unit detects that the user's eyelid is kept closed for a predetermined time. Therefore, the power consumption can be reduced as compared with the conventional one while suppressing the deterioration of the convenience for the user.

[Application Example 4] The head-mounted display device according to Application Example 3,
When the detection unit detects that the user's eyelid has opened for a predetermined time after the power saving transition, the control unit performs a return transition that is a transition to an operation mode with higher power consumption. Part-mounted display device.

  If the user's eyelid is open for a predetermined time, the user is likely to be viewing the image. In this head-mounted display device, the control unit shifts to an operation mode with higher power consumption when the detection unit detects that the user's eyelid has opened for a predetermined time after shifting to power saving. Since the return transition is performed, the return transition can be promptly performed.

[Application Example 5] The head-mounted display device according to Application Example 3 or Application Example 4,
The image light generation unit includes: a light source that emits light; and an image generation unit that converts the light emitted from the light source based on image data into image light representing an image,
The head-mounted display device, wherein the control unit reduces light emitted from the light source during the power saving transition.

  In this head-mounted display device, the control unit reduces the light emitted from the light source when shifting to power saving, and thus can reduce power consumption.

[Application Example 6] The head-mounted display device according to Application Example 5,
The light source includes a right eye light source and a left eye light source,
The head-mounted display device that reduces the light emitted from one of the light source for the right eye and the light source for the left eye during the power saving transition.

  In this head-mounted display device, the control unit reduces light emitted from one of the right-eye light source and the left-eye light source at the time of power saving transition, thereby realizing a flexible reduction in power consumption. Can do.

[Application Example 7] The head-mounted display device according to Application Example 3 or Application Example 4,
The image light generation unit includes a light source that emits light, and an image generation unit that converts the light emitted from the light source based on image data into image light representing an image,
The head-mounted display device, wherein the control unit stops the operation of the image generation unit during the power saving transition.

  In this head-mounted display device, the control unit stops the operation of the image generation unit when shifting to power saving, and thus power consumption can be reduced.

[Application Example 8] The head-mounted display device according to Application Example 4,
The image light generation unit includes a light source that emits light, and an image generation unit that converts the light emitted from the light source based on image data into image light representing an image,
The head-mounted display device, wherein the control unit stops the operation of the image generation unit at the time of the power saving transition and restarts the operation of the image generation unit at the return transition.

  In this head-mounted display device, the control unit starts the operation of the image generation unit again at the time of return transition, so that the return shift can be performed quickly.

[Application Example 9] The head-mounted display device according to Application Example 3 or Application Example 4,
The head-mounted display device, wherein the control unit includes a function unit that realizes a plurality of functions, and stops a part of the operation of the function unit that realizes the plurality of functions during the power saving transition.

  In this head-mounted display device, the control unit stops the operation of some of the functional units that realize a plurality of functions of the control unit when shifting to power saving, so that power consumption can be reduced.

[Application Example 10] The head-mounted display device according to Application Example 9,
The control unit further includes an operation unit, and the head-mounted display device restarts the operation of the function unit that has been stopped when the operation unit receives a specific operation after the power saving transition. .

  In this head-mounted display device, the control unit restarts the operation of the function unit that has been stopped when the operation unit receives a specific operation after shifting to power saving. Even if this state is not detected, the return transition can be promptly performed.

[Application Example 11] The head-mounted display device according to Application Example 10,
The control unit stores information of an image displayed by the image display unit at the time of the power saving transition, and resumes display from the image displayed at the time of the power saving transition at the time of the return transition. Wearable display device.

  In this head-mounted display device, the display is restarted from the image that was displayed at the time of the power saving transition at the time of the transition to the return, so that the target image can be efficiently viewed at the time of the transition to the return and the convenience is improved. Can do.

[Application Example 12] The head-mounted display device according to Application Example 3,
When the control unit detects that the user's eyelid keeps closing for a predetermined time after the power saving transition, the control unit performs the power saving transition to an operation mode with lower power consumption. Head-mounted display device.

  In this head-mounted display device, the control unit performs a power saving transition to an operation mode in which the power consumption is gradually reduced according to the state of the user's eyelid. Therefore, the head-mounted display device can perform a return transition promptly while reducing power consumption.

[Application Example 13] The apparatus according to Application Example 1,
The said control part is an apparatus which converts each pattern of the state of the eyelid which the said detection part detected into input information according to a predetermined rule.

  In this device, the detection unit detects the eyelid pattern, and the control unit converts the instruction assigned to the pattern into input information according to a predetermined rule, so that a plurality of instructions assigned to the pattern are executed. It is possible to improve convenience.

Application Example 14 The apparatus according to Application Example 13,
The said control part is an apparatus which performs predetermined | prescribed operation | movement according to the said input information.

  In this apparatus, the control unit performs the operation assigned according to the input information converted according to each pattern, so that a plurality of functions assigned to the pattern can be executed, and convenience can be improved.

[Application Example 15] The apparatus according to Application Example 13 or Application Example 14,
The said detection part is an apparatus which detects the opening / closing state of the eyelid in predetermined timing.

  In this device, the detection unit detects and determines the state of the eyelid in two types of an open state and a closed state at a predetermined timing, so that there is little misdetection of the state of the eyelid and highly accurate operability. Can be realized.

Application Example 16 The apparatus according to Application Example 15,
Each said pattern contains the pattern prescribed | regulated by the combination of the opening / closing state of the eyelid in each of several said timing.

  In this device, the eyelid opening / closing state pattern is a pattern defined by a combination of eyelid opening / closing states at a plurality of predetermined timings, so it is possible to give instructions according to a plurality of patterns and improve convenience. Can do.

[Application Example 17] The apparatus according to Application Example 15 or Application Example 16,
The detection unit detects the state of the right eyelid and the state of the left eyelid,
Each pattern includes a pattern defined by a combination of the state of the right eyelid and the state of the left eyelid.

  In this device, the detection unit detects the open / closed state of the right eyelid and the left eyelid, and the pattern of the open / closed state of the eyelid is a pattern defined by the combination of the open / closed state of the right eyelid and the left eyelid. The operation | movement according to it can be performed and the convenience can be improved.

[Application Example 18] The apparatus according to any one of Application Example 13 to Application Example 17,
The said control part is an apparatus which converts each pattern except the one part pattern of each said pattern into the said input information according to the said predetermined rule, and does not convert the said one part pattern.

  In this device, the control unit converts the instructions assigned to each pattern excluding a part of the eyelid pattern into input information according to a predetermined rule, and the instructions assigned to the part of the pattern Is not converted to input information. Therefore, in this apparatus, when an instruction assigned to each pattern is given, it is possible to avoid giving an instruction not intended by the user.

Application Example 19 The apparatus according to Application Example 13,
The said control part is an apparatus which inputs the input data according to the said input information.

  In this device, the control unit inputs the input data according to the input information obtained by converting the eyelid state detected by the detection unit, so that the user can input the input data without using a hand, and the operability is improved. Can be made.

Application Example 20 The apparatus according to any one of Application Examples 13 to 19, further comprising:
It has an operation unit that accepts user operations,
The said control part is an apparatus which converts the combination of each said pattern and the said operation into the said input information according to the said predetermined rule.

  In this device, the control unit converts the input information according to the combination of the operation received by the operation unit in addition to the eyelid state pattern, so that it is possible to prevent erroneous detection of the eyelid state, and more users It is possible to instruct by the pattern.

[Application Example 21] The apparatus according to any one of Application Example 13 to Application Example 20,
The detection unit detects the number of times the eyelid is opened and closed at a predetermined time,
Each said pattern contains the pattern prescribed | regulated by the frequency | count of opening and closing of the eyelid in the said predetermined time.

  In this device, the detection unit detects the number of times the eyelid is opened and closed for a predetermined time, and the eyelid pattern is defined by the number of times the eyelid is opened and closed, so that the user can give an instruction by a simple eyelid state change. , Can improve convenience.

[Application Example 22] The apparatus according to any one of Application Example 13 to Application Example 21,
The said detection part is an apparatus which interprets that the eyelid is an open state, when the time which the eyelid is closed is shorter than predetermined value among the states of the said eyelid.

  This device interprets the eyelid as being open when the eyelid is closed for less than a predetermined value in the eyelid state, so the eyelid can be detected without detecting eye blinks. An erroneous detection of the state can be avoided.

Application Example 23 The apparatus according to any one of Application Examples 13 to 22, further comprising:
It has a sensing unit that senses the movement of the user's head,
The said control part is an apparatus which converts the combination of each said pattern and the motion of the said head into the said input information according to the said predetermined rule.

  This apparatus further includes a sensing unit that senses the movement of the user's head, and the control unit converts the information into input information according to a combination of each pattern of the eyelid state and the sensed movement of the user's head. Therefore, in this device, the user can operate without using a hand, can prevent erroneous detection of the state of the eyelid, and can instruct more patterns by combining with the movement of the head. .

[Application Example 24] A head-mounted display device,
An image display unit that generates and emits image light based on image data, and a light guide unit that guides the image light to the user's eyes, and allows the user to visually recognize a virtual image;
The apparatus according to any one of Application Examples 13 to 23, which is connected to the image display unit, transmits the image data to the image display unit, and controls image display by the image display unit. Head-mounted display device.

  Since the head-mounted display device includes the control unit and the image display unit, an image display device with high portability can be realized.

  Note that the present invention can be realized in various modes. For example, the device and the device control method, the head-mounted display device and the head-mounted display device control method, the detection instruction device, and the state detection Control apparatus, eyelid detection control apparatus, information processing system, head-mounted display system, computer program for realizing the functions of these methods, apparatus or system, recording medium storing the computer program, including the computer program It can be realized in the form of a data signal embodied in a carrier wave.

It is explanatory drawing which shows the external appearance structure of the head mounted display apparatus 100 in 1st Example of this invention. 3 is an explanatory diagram illustrating a detailed configuration of an image display unit 20. FIG. 3 is an explanatory diagram functionally showing the configuration of the head-mounted display device 100. FIG. It is explanatory drawing which shows an example of the virtual image recognized by the user. It is a flowchart which shows the flow of the control processing of the head mounted display apparatus 100 in 1st Example. It is a flowchart which shows the flow of the control processing of the head mounted display apparatus 100 in 2nd Example. It is explanatory drawing which shows the relationship between the time in 2nd Example, and the open / closed state of an eyelid. It is explanatory drawing which shows an example of the function according to each pattern of the user's eyelid of the head mounted display apparatus 100 in 2nd Example. It is explanatory drawing which shows an example of the function according to each pattern of the user's eyelid of the head mounted display apparatus 100 in a modification. It is explanatory drawing which shows an example of the input data according to the state of the user's eyelid of the head mounted display apparatus 100 in a modification.

Next, embodiments of the present invention will be described in the following order based on examples.
A. First embodiment:
A-1. Device configuration:
A-2. Control processing:
B. Second embodiment:
C. Variations:

A. First embodiment:
A-1. Device configuration:
FIG. 1 is an explanatory diagram showing an external configuration of a head-mounted display device 100 according to the first embodiment of the present invention. The head-mounted display device 100 is a display device mounted on the head, and is also called a head mounted display (HMD). The head-mounted display device 100 according to this embodiment is an optically transmissive head-mounted display device that allows a user to visually recognize a virtual image and at the same time directly view an outside scene.

  The head-mounted display device 100 includes an image display unit 20 that allows a user to visually recognize a virtual image when mounted on the user's head, and a control unit (controller) 10 that controls the image display unit 20. ing.

  The image display unit 20 is a mounting body that is worn on the user's head, and has a shape like glasses in this embodiment. The image display unit 20 includes an ear hook unit 21 that functions as a temple, a right optical panel 26 that is positioned in front of the right and left eyes of the user when the user wears the image display unit 20, and And a left optical panel 28. Further, a right display drive unit 22 is disposed at a connection portion between the right ear hook portion 21 and the right optical panel 26, and a connection portion between the left ear hook portion 21 and the left optical panel 28 is provided. The left display driving unit 24 is disposed. In the following description, the right optical panel 26 and the left optical panel 28 are collectively referred to simply as “optical panels 26, 28”.

  The image display unit 20 also includes a right earphone 32 for the right ear and a left earphone 34 for the left ear. The right earphone 32 and the left earphone 34 are respectively attached to the right and left ears when the user wears the image display unit 20.

  The image display unit 20 further includes a connection unit 40 for connecting the image display unit 20 to the control unit 10. The connection unit 40 includes a main cable 48 connected to the control unit 10, a right cable 42 and a left cable 44 in which the main cable 48 is branched into two, and a connection unit 46 provided at a branch point. . The right cable 42 is connected to the right display drive unit 22, and the left cable 44 is connected to the left display drive unit 24. The image display unit 20 and the control unit 10 transmit various signals via the connection unit 40. Each of the end of the cable 48 opposite to the connecting portion 46 and the control unit 10 is provided with a connector (not shown) that fits with each other. The connector of the cable 48 and the connector of the control unit 10 are provided. The control unit 10 and the image display unit 20 are connected to or disconnected from each other by the fitting / releasing.

  The control unit 10 is a device that controls image display by the image display unit 20. The control unit 10 includes a power switch 18 that switches power ON / OFF, a lighting unit 12 that notifies an operation state (for example, power ON / OFF state) of the head-mounted display device 100 according to a light emission state, and a user. A touch pad 14 that detects a finger operation and outputs a signal corresponding to the finger operation; and a cross key 16 that detects a pressing operation of a key corresponding to the vertical and horizontal directions and outputs a signal corresponding to the operation. doing. As the lighting unit 12, for example, one or a plurality of LED lamps can be used. As the touch pad 14, various touch pads such as an electrostatic type, a pressure detection type, and an optical type can be adopted.

  FIG. 2 is an explanatory diagram showing a detailed configuration of the image display unit 20. The image display unit 20 includes an infrared sensor 50 that detects the state of the user's eyelid. The infrared sensor 50 includes an infrared light emitting unit 51 that emits infrared light toward both eyes of the user and an infrared light receiving unit 56 that receives infrared light reflected by both eyes of the user. The infrared light emitting unit 51 includes a right infrared light emitting unit 52 that emits infrared rays toward the user's right eye and a left infrared light emitting unit 54 that emits infrared rays toward the user's left eye. The right infrared light emitting unit 52 is provided in the right display driving unit 22, and the left infrared light emitting unit 54 is provided in the left display driving unit 24. The infrared light receiver 56 is provided at a boundary portion that is intermediate between the right optical panel 26 and the left optical panel 28.

  The infrared light emitted from the right infrared light emitting unit 52 is reflected by the right light guide plate 261 included in the image display unit 20 described later and enters the right eye 62 of the user. The reflected infrared light from the right eye 62 with respect to the incident light reaches the infrared light receiving unit 56 and is detected by the infrared sensor 50. Since the infrared ray emitted from the left infrared light emitting unit 54 is the same as the infrared ray emitted from the right infrared light emitting unit 52, the description thereof is omitted here. Since the infrared reflectance is different between the state where the user's eyes are open and the state where the eyes are closed, the output value at the infrared light receiving unit 56 is different. Therefore, the infrared sensor 50 can determine a state where the user's eyes are open and a state where the eyes are closed.

  FIG. 3 is an explanatory diagram functionally showing the configuration of the head-mounted display device 100. As illustrated in FIG. 3, the control unit 10 includes a CPU 140, a storage unit 120 configured by a ROM, a RAM, and the like, a power supply 130 that supplies power to each unit of the head-mounted display device 100, and a user operation. Operation unit 110 (in this embodiment, touch pad 14, cross key 16, power switch 18) and various external devices OA (sources of content data such as images (still images, moving images) and audio) ( For example, an interface 180 for connecting a personal computer PC, a mobile phone terminal, or a game terminal) is provided. As the power source 130, for example, a secondary battery can be used, and as the interface 180, for example, a USB interface, a memory card interface, a wireless LAN interface, or the like can be employed.

  Various computer programs are stored in the storage unit 120, and the CPU 140 reads out and executes the computer programs from the storage unit 120, thereby operating the operating system (OS) 150, the image processing unit 160, and the display control unit 190. , Function as a detection control unit 183 and a sound processing unit 170.

  The image processing unit 160 generates a vertical synchronization signal VSync, a horizontal synchronization signal HSync, and image data Data based on the content data input via the interface 180, and these signals are output to the image display unit 20 via the connection unit 40. To supply. Specifically, the image processing unit 160 acquires an image signal (for example, an analog signal composed of 30 frame images per second) included in the content data, and the vertical synchronization signal VSync or the like from the acquired image signal. A synchronizing signal such as the horizontal synchronizing signal HSync is separated, and an analog image signal from which the synchronizing signal is separated is converted into a digital image signal to generate image data Data. Note that the image processing unit 160 may execute resolution conversion processing, color tone correction processing, and the like on the image data as necessary.

  The detection control unit 183 is connected to the infrared sensor 50 and detects the output value of the infrared light received by the infrared light receiving unit 56. The detection control unit 183 controls the infrared light emission of the infrared light emitting unit 51. The detection control unit 183 determines a state where the user's eyes are open and a state where the eyes are closed based on the output value from the infrared light receiving unit 56. Based on the determination, the detection control unit 183 generates a control signal and supplies the control signal to the display control unit 190. The detection control unit 183 and the infrared sensor 50 correspond to the detection unit 185 of the present invention.

  The display control unit 190 receives control signals from the detection unit 185 and the like, generates control signals for controlling the right display drive unit 22 and the left display drive unit 24, and supplies them to the image display unit 20 via the connection unit 40. Thus, the image display state in the image display unit 20 is controlled. Specifically, the display control unit 190 uses the generated control signal to turn on / off the right LCD 241 by the right LCD control unit 211, to turn on / off the right light source 221 by the right light source control unit 201, and to control the left LCD. By individually controlling the driving ON / OFF of the left LCD 242 by the unit 212 and the driving ON / OFF of the left light source 222 by the left light source control unit 202, the image light of each of the right display driving unit 22 and the left display driving unit 24 is controlled. Controls the generation. For example, the display control unit 190 may cause both the right display driving unit 22 and the left display driving unit 24 to generate image light, generate only one image light, or neither may generate image light.

  The audio processing unit 170 acquires an audio signal included in the content data, amplifies the acquired audio signal, and supplies the acquired audio signal to the right earphone 32 and the left earphone 34 of the image display unit 20 via the connection unit 40.

  The right display drive unit 22 of the image display unit 20 includes, for example, a right light source 221 configured by an LED lamp, a right light source control unit 201 that drives the right light source 221 based on a control signal supplied from the control unit 10, and a right A right liquid crystal display (LCD) 241 that converts light emitted from the light source 221 into image light representing an image, and a right based on a vertical synchronization signal VSync, a horizontal synchronization signal HSync, and image data Data supplied from the control unit 10 And a right LCD control unit 211 for driving and controlling the LCD 241. The right LCD 241 is configured using, for example, a transmissive liquid crystal panel. The right LCD 241 and the right LCD control unit 211 constitute a right eye image generation unit 244. The right display drive unit 22 includes a right projection optical system 251 that projects the generated image light. The right projection optical system 251 is configured using, for example, a collimator lens.

  The right optical panel 26 (FIG. 1) of the image display unit 20 has a right light guide plate 261. The right light guide plate 261 receives image light output from the right projection optical system 251 along a predetermined optical path. It is guided to the user's right eye 62 (FIG. 2) while reflecting. The right projection optical system 251 and the right light guide plate 261 correspond to the light guide unit in the present invention.

  Similar to the right display drive unit 22, the left display drive unit 24 includes a left light source 222, a left light source control unit 202, a left LCD 242, a left LCD control unit 212, and a left projection optical system 252. . Since the configuration and function of each element included in the left display driving unit 24 are the same as those included in the right display driving unit 22, the description thereof is omitted here. The left optical panel 28 of the image display unit 20 includes a left light guide plate 262. The left light guide plate 262 reflects the image light output from the left projection optical system 252 along a predetermined optical path. While guiding to the left eye 64 (FIG. 2) of the user. The left LCD 242 and the left LCD control unit 212 constitute a left eye image generation unit 246. The left projection optical system 252 and the left light guide plate 262 correspond to the light guide unit in the present invention. Hereinafter, the right light source control unit 201 and the left light source control unit 202 are collectively referred to as “light source control units 201 and 202”, and the right light source 221 and the left light source 222 are collectively referred to as “light sources 221 and 222”. The eye image generation unit 244 and the left eye image generation unit 246 are collectively referred to simply as “image generation units 244 and 246”. The light sources 221 and 222 and the image generation units 244 and 246 correspond to the image light generation unit in the present invention.

  In this way, the image light guided to both eyes of the user of the head-mounted display device 100 forms an image on the retina, so that the user visually recognizes the virtual image. FIG. 4 is an explanatory diagram illustrating an example of a virtual image recognized by the user. As shown in FIG. 4, the virtual image VI is visually recognized in the visual field VR of the user of the head-mounted display device 100. In addition, the user can see the outside scene SC through the right optical panel 26 and the left optical panel 28 except the portion where the virtual image VI is visually recognized in the visual field VR of the user. In the head-mounted display device 100 according to the present embodiment, the outside scene SC can be seen through the virtual image VI even in the portion where the virtual image VI is visually recognized in the user's visual field VR.

  The right optical panel 26 and the left optical panel 28 are provided on the front side of the right light guide plate 261 and the left light guide plate 262 (on the side opposite to the user's eye side), and can adjust the light transmittance. It may have a light plate. When the light control plate is provided, by adjusting the light transmittance of the light control plate, the amount of external light entering the user's eyes can be adjusted, and the ease of visual recognition of the virtual image can be adjusted.

A-2. Control Processing FIG. 5 is a flowchart showing the flow of control processing of the head-mounted display device 100 in the first embodiment. In a state where the head-mounted display device 100 of this embodiment is used, content data such as a moving image and an image is supplied to the CPU 140 from the external device OA connected via the interface 180. The display control unit 190 generates a control signal based on the supplied content data. The control signal is supplied to the light source controllers 201 and 202 and the image generators 244 and 246. The light source controllers 201 and 202 increase or decrease the luminance of the light sources 221 and 222 based on the supplied control signal. The image generation units 244 and 246 convert light emitted from the light sources 221 and 222 into image light based on the supplied control signal. The left projection optical system 252 and the left light guide plate 262 guide the image light to the user's eyes. The image light is visually recognized by the user as a virtual image, and the sound processing unit 170 included in the CPU 140 reproduces the sound of the content supplied from the external device OA (step 310). In the present embodiment, since the head-mounted display device 100 is a transmissive head-mounted display device, the user can see the outside scene SC through the virtual image VI.

  During the operation of the head-mounted display device 100, the detection unit 185 detects the state of the user's eyelid (the open / closed state of the eyelid) (step S320). When the user is viewing the content supplied from the external device OA, the detection unit 185 may detect that the user's eyelid is kept closed. At this time, the user may have slept. Therefore, in this embodiment, the control unit 10 and the display control unit 190 switch the operation mode according to the open / closed state of the user's eyelid detected by the detection unit 185.

  While the detection unit 185 does not detect that the user's eyelid has been closed for 10 seconds or more (step S330: NO), it is determined that the user is not sleeping, and the head-mounted display device 100 operates normally. Continues (step S331).

  When the head-mounted display device 100 is in a normal operation, the detection unit 185 detects that the user is sleeping when detecting that the user's eyelid has been closed for 10 seconds or more (step S330: YES). judge. At this time, the display control unit 190 reduces the luminance of the light sources 221 and 222 (step S340). By reducing the light emitted from the light sources 221 and 222, the head-mounted display device 100 shifts to an operation mode with lower power consumption. This transition is called power saving transition.

  In the present embodiment, the display control unit 190 controls the right light source control unit 201 and the left light source control unit 202 separately. The right light source control unit 201 controls the right light source 221, and the left light source control unit 202 controls the left light source 222. Therefore, the display control unit 190 can also reduce the luminance of only one of the right light source 221 and the left light source 222. The display control unit 190 can also reduce the luminance separately for the right light source 221 and the left light source 222.

  When the detection unit 185 detects that the user's eyelid has been opened within 30 seconds after the luminance of the light sources 221 and 222 is reduced (step S350: NO), the detection unit 185 determines that the user has awakened from sleep. To do. At this time, the display control unit 190 maximizes the luminance of the light sources 221 and 222 (step S351). By maximizing the luminance of the light sources 221 and 222, the head-mounted display device 100 shifts to an operation mode with higher power consumption. This transition is called a return transition. Thereafter, the display control unit 190 shifts to a normal operation (step S331). Note that the increase in the luminance of the light sources 221 and 222 may not be equal to the decrease in the luminance of the light sources 221 and 222 in the process of step S340. Further, the display control unit 190 may gradually increase the luminance of the light sources 221 and 222 as time elapses.

  When the detection unit 185 continues to detect that the user's eyelid has been closed for 30 seconds or more after the luminance of the light sources 221 and 222 has been reduced (step S350: YES), the user continues to sleep. It is determined that At this time, the display control unit 190 turns off the light sources 221 and 222 (step S360). As a result, the head-mounted display device 100 performs a power saving transition to an operation mode with lower power consumption.

  When the detection unit 185 detects that the user's eyelid has been opened within one minute after the light sources 221 and 222 are turned off (step S370: NO), the detection unit 185 determines that the user has woke up from sleep. At this time, the display control unit 190 causes the image display unit 20 to display a recreation video (for example, a video with low luminance) (step S371). Thereafter, the display control unit 190 maximizes the luminance of the light sources 221 and 222 (step S351), and shifts to normal operation (step S331). Thereby, the head-mounted display device 100 performs a return transition to an operation mode with higher power consumption. In the present embodiment, if the brightness of the light sources 221 and 222 increases in a short time from the extinguished state, the burden on the user's eyes is great, so the display control unit 190 displays the resting image as an image display unit. 20 is displayed. Note that the display control unit 190 may increase the luminance of the light sources 221 and 222 in a short time without displaying the recreation video, or may gradually increase the luminance of the light sources 221 and 222 as time elapses. Good.

  When the light sources 221 and 222 are turned off, the detection unit 185 continuously detects that the user's eyelid has been closed for more than one minute (step S370: YES), the user continues to sleep. It is determined that At this time, the display control unit 190 stops the operations of the image generation units 244 and 246 (step S380). As a result, the head-mounted display device 100 performs a power saving transition to an operation mode with lower power consumption.

  In the present embodiment, the display control unit 190 controls the right eye image generation unit 244 and the left eye image generation unit 246 separately. Therefore, the display control unit 190 can also stop the operation of only one of the right-eye image generation unit 244 and the left-eye image generation unit 246.

  When the detection unit 185 detects that the user's eyelid has been opened within 5 minutes after the operation of the image generation units 244 and 246 is stopped (step S390: NO), the user wakes up from sleep. It is determined that At this time, the display control unit 190 starts the operations of the image generation units 244 and 246 again (step S391). Thereafter, the display control unit 190 displays the recreation video on the image display unit 20 (step S371), maximizes the luminance of the light sources 221 and 222 (step S351), and shifts to normal operation (step S331). Thereby, the head-mounted display device 100 performs a return transition to an operation mode with higher power consumption.

  When the detection unit 185 detects that the user's eyelid continues to be closed for more than 5 minutes after the operation of the image generation units 244 and 246 is stopped (step S390: YES), the user sleeps. It is determined that it continues. At this time, the control unit 10 stops the operations of the image processing unit 160, the display control unit 190, the detection control unit 183, and the sound processing unit 170 (step S400). As a result, the head-mounted display device 100 performs a power saving transition to an operation mode with lower power consumption. An operation mode in which the operations of the image processing unit 160, the display control unit 190, the detection control unit 183, and the sound processing unit 170 are stopped is referred to as a standby mode. In addition, the image processing unit 160, the display control unit 190, the detection control unit 183, and the sound processing unit 170 correspond to each one of the functional units that realize a plurality of functions of the control unit 10 in the present invention.

  As long as the power switch 18 of the operation unit 110 is not pressed in the standby mode, the detection unit 185 determines that the user continues to sleep, and the standby mode continues (step S410: NO).

  When the power switch 18 of the operation unit 110 is pressed in the standby mode (step S410: YES), the control unit 10 determines that the user has awakened from sleep. At this time, the control unit 10 cancels the standby mode, and the operations of the image processing unit 160, the display control unit 190, the detection control unit 183, and the display control unit 190 are started again (step S411). Thereafter, the display control unit 190 displays the recreation video on the image display unit 20 (step S371), maximizes the luminance of the light sources 221 and 222 (step S351), and shifts to normal operation (step S331). Thereby, the head-mounted display device 100 performs a return transition to an operation mode with higher power consumption.

  In the present embodiment, the storage unit 120 stores image information of content data that the display control unit 190 displayed on the image display unit 20 before shifting to the standby mode. Therefore, when the control unit 10 cancels the standby mode, the operations of the image processing unit 160, the display control unit 190, the detection control unit 183, and the display control unit 190 are started again, and the display control unit 190 is stored in the storage unit 120. Based on the stored content data image information, it can be resumed from where it left off.

  As described above, in the head-mounted display device 100 of the present embodiment, the detection unit 185 detects the open / closed state of the user's eyelid. The display control unit 190 controls the operation of the image generation units 244 and 246 by increasing or decreasing the luminance of the light sources 221 and 222 according to the open / closed state of the eyelids. Further, the control unit 10 shifts to the standby mode or cancels the standby mode depending on the open / closed state of the eyelid. When the detection unit 185 detects that the user's eyelid has been closed for a predetermined time, the display control unit 190 determines that the user is sleeping. At this time, the display control unit 190 and the control unit 10 perform power saving transition. In this embodiment, as the power saving transition, the display control unit 190 reduces the luminance of the light sources 221 and 222 or stops the operation of the image generation units 244 and 246. Further, the control unit 10 shifts to the standby mode. On the contrary, in the operation mode in which the power consumption is lower than that in the normal operation, the detection unit 185 determines that the user has awakened from sleep when it does not detect that the user's eyelid has been closed for a predetermined time. At this time, the display control unit 190 and the image display unit 20 perform return transition. In the present embodiment, as the return transition, the display control unit 190 increases the luminance of the light sources 221 and 222 and starts the operations of the image generation units 244 and 246 again. The control unit 10 cancels the standby mode. Therefore, in the head-mounted display device 100 according to the present embodiment, the display control unit 190 and the control unit 10 switch to an appropriate operation mode with different power consumption according to the detected opening / closing state of the user's eyelid. Power consumption can be reduced.

  Further, in the head-mounted display device 100 of the present embodiment, during normal operation, the detection unit 185 detects that the user's eyelid has been closed for 10 seconds or more, and the user is sleeping. Is determined. At this time, the display control unit 190 reduces the luminance of the light sources 221 and 222. Further, when the luminance of the light sources 221 and 222 is reduced, the detection unit 185 determines that the user continues to sleep when the detection unit 185 detects that the user's eyelid continues to be closed for 30 seconds or more. To do. At this time, the display control unit 190 turns off the light sources 221 and 222. Therefore, the head-mounted display device 100 reduces the luminance of the light sources 221 and 222 when the user is asleep, and shifts to an operation mode with lower power consumption. Can do.

  Further, in the head-mounted display device 100 according to the present embodiment, when the light sources 221 and 222 are turned off, the detection unit 185 continuously detects that the user's eyelid has been closed for more than one minute. It is determined that the user continues to sleep. At this time, the display control unit 190 stops the operations of the image generation units 244 and 246. Therefore, the head-mounted display device 100 stops the operation of the image generation units 244 and 246 when the user continues to sleep, and shifts to an operation mode with lower power consumption. can do.

  In the head-mounted display device 100 according to the present embodiment, when the operation of the image generation units 244 and 246 is stopped, the detection unit 185 continues to close the user's eyelid for 5 minutes or more. When it is detected, it is determined that the user continues to sleep. At this time, the control unit 10 shifts to the standby mode. Therefore, in the head-mounted display device 100, when the user continues to sleep, the operations of the image processing unit 160, the display control unit 190, the detection control unit 183, and the sound processing unit 170 are stopped, and more consumption is performed. Since the operation mode shifts to a low power mode, the power consumption can be reduced.

  Further, as described above, the head-mounted display device 100 according to the present embodiment performs the power saving transition step by step according to the time during which the user's eyelid is kept closed. When the elapsed time since the user's eyelid is closed is short, it can be considered that there is a higher possibility that the user will wake up than when the elapsed time is long. When the user wakes up, it is desirable that the time required to return to normal operation is shorter. Accordingly, the head-mounted display device 100 shifts to an operation mode with lower power consumption step by step according to the elapsed time after the user's eyelids are closed. It is possible to return to normal operation.

  In the head-mounted display device 100 according to the present embodiment, when the luminance of the light sources 221 and 222 is reduced, the detection unit 185 does not detect that the user's eyelid has been closed for another 30 seconds or more. It is determined that the user woke up from sleep. At this time, the display control unit 190 maximizes the luminance of the light sources 221 and 222. Further, when the light sources 221 and 222 are turned off, the detection unit 185 determines that the user has awakened from sleep when it does not detect that the user's eyelid has been closed for another minute or more. At this time, the display control unit 190 causes the image display unit 20 to display a rest video. Thereafter, the display control unit 190 maximizes the luminance of the light sources 221 and 222. Therefore, the head-mounted display device 100 can reduce power consumption when the user is asleep, and can quickly return to normal operation when the user wakes up from sleep.

  Further, in the head-mounted display device 100 of the present embodiment, when the operation of the image generation units 244 and 246 is stopped, the detection unit 185 keeps the user's eyelid closed for more than 5 minutes. When it is not detected, it is determined that the user woke up from sleep. At this time, the display control unit 190 starts the operations of the image generation units 244 and 246 again. Thereafter, the display control unit 190 displays the recreation video on the image display unit 20 and maximizes the luminance of the light sources 221 and 222. Therefore, the head-mounted display device 100 can reduce power consumption when the user is asleep, and can quickly return to normal operation when the user wakes up from sleep.

  In the head-mounted display device 100 according to the present embodiment, when the power switch 18 of the operation unit 110 is pressed in the standby mode, the control unit 10 determines that the user woke up from sleep. At this time, the control unit 10 cancels the standby mode, and the operations of the image processing unit 160, the display control unit 190, the detection control unit 183, and the display control unit 190 are started again. Thereafter, the display control unit 190 displays the recreation video on the image display unit 20 and maximizes the luminance of the light sources 221 and 222. Therefore, the head-mounted display device 100 can reduce power consumption when the user is asleep, and can quickly return to normal operation when the user wakes up from sleep.

  In addition, as described above, the head-mounted display device 100 according to the present embodiment performs the return transition step by step according to the time during which the user's eyelid is kept closed. When the elapsed time since the user's eyelid is closed is long, it can be considered that the user is deeply asleep. Therefore, when the detection unit 185 detects that the user's eyelid is opened from the closed state, if the luminance of the light sources 221 and 222 increases in a short time, the burden on the user's eyes is large. . Therefore, the head-mounted display device 100 can reduce the burden on the user's eyes by performing a return transition step by step.

  In the head-mounted display device 100 according to the present embodiment, the image processing unit 160 causes the storage unit 120 to display the content displayed on the image display unit 20 by the display control unit 190 before shifting to the standby mode. The information is memorized. The content information is, for example, a frame number or frame data. Therefore, when the control unit 10 cancels the standby mode, the operations of the image processing unit 160, the display control unit 190, the detection control unit 183, and the display control unit 190 are started again, and the display control unit 190 is stored in the storage unit 120. Based on the stored content information, it can be resumed from where it left off. Therefore, in the head-mounted display device 100, the user can resume the content he / she wants to see efficiently.

  Further, in the head-mounted display device 100 according to the present embodiment, the display control unit 190 controls the right light source control unit 201 and the left light source control unit 202 separately. The right light source control unit 201 controls the right light source 221, and the left light source control unit 202 controls the left light source 222. Therefore, the display control unit 190 can also reduce the luminance of only one of the right light source 221 and the left light source 222. In addition, the display control unit 190 performs separate control on the right eye image generation unit 244 and the left eye image generation unit 246. Therefore, the display control unit 190 can also stop the operation of only one of the right-eye image generation unit 244 and the left-eye image generation unit 246. Therefore, since the head-mounted display device 100 controls the right light source 221 and the left light source 222, and the right eye image generation unit 244 and the left eye image generation unit 246 separately, it is flexible. Reduction of power consumption can be realized.

B. Second embodiment:
FIG. 6 is a flowchart showing a flow of control processing of the head-mounted display device 100 in the second embodiment. The head-mounted display device 100 according to the second embodiment is configured such that a user can operate it with an eyelid without using a hand. The control process is a process in which the head-mounted display device 100 performs control according to an operation by the user's eyelid.

  In the control process of the head-mounted display device 100, the detection unit 185 detects the open / closed state of the right eye 72 (FIG. 2) of the right eye 62 and the left eye 74 of the left eye 64 (step S510). FIG. 7 is an explanatory diagram showing the relationship between time and the open / closed state of the eyelid in the second embodiment. FIG. 7A shows an example of a detection opening / closing signal ES1 indicating the opening / closing state of the right eyelid 72 (or the left eyelid 74) at each time detected by the infrared sensor 50. FIG. 7B shows an example of the interpretation opening / closing signal ES2 indicating the opening / closing state of the right eyelid 72 (or the left eyelid 74) at each time interpreted by the detection control unit 183 that has received the detection result by the infrared sensor 50. Yes. In FIG. 7B, the detection opening / closing signal ES1 is also shown by a one-dot chain line as a reference. In FIG. 7, the state where the eyelid is closed is “0”, and the state where the eyelid is open is “1”. As shown in FIG. 3, the infrared sensor 50 and the detection control unit 183, that is, the detection unit 185 are attached to the image display unit 20, but are a part of the configuration of the control unit 10.

  Usually, the user of the head-mounted display device 100 is in a state where both eyelids are opened. The infrared sensor 50 continuously detects the open / closed state of both eyelids of the user, and outputs a signal indicating the open / closed state of both eyelids (detected open / close signal ES1 shown in FIG. 7A) to the detection control unit 183. Yes. The detection control unit 183 determines the open / closed state of both eyelids based on the detection open / close signal ES1. For example, when detecting the state where only the user's right eyelid 72 or left eyelid 74 is closed, the detection time starts. At time T0, the open / closed state of each eyelid is detected at a first detection time T1 0.5 seconds after the start time T0 and at a second detection time T2 after 1 second. The time from the start time T0 to the first detection time T1 is 0.5 seconds, and the time from the first detection time T1 to the second detection time T2 is 1 second. However, the time is not limited to this value. The user may set it freely.

  FIG. 7 shows an example in which the user of the head-mounted display device 100 of the second embodiment performs an operation of opening the eyelid at the first detection time T1 and closing the eyelid at the second detection time T2. .

  Usually, the user of the head-mounted display device 100 may unconsciously blink even when performing an operation of opening the eyelid. The section of time BT shown in FIG. 7A is a section where the eyelid is closed by blinking. As shown in FIG. 7A, when the first detection time T1 overlaps the section of the blink time BT, the infrared sensor 50 detects a state where the eyelid is closed at the first detection time T1.

  As shown in FIG. 7B, the detection control unit 183 converts the detection opening / closing signal ES1 detected by the infrared sensor 50 into an interpretation opening / closing signal ES2. In order not to determine that the eyelid is in a closed state even if eye blinking is detected, the detection control unit 183 blinks the eye when the eyelid closing time is shorter than a preset threshold value Tth. It is determined that the eyelid is in an open state, not a closed state. In the head-mounted display device 100 according to the second embodiment, the preset threshold value Tth is set to 0.3 seconds in consideration of the blink time BT (0.10 to 0.15 seconds), and the detection control unit 183 has the eyelids. The detection unit 185 detects the open / closed state of the eyelid by interpreting the detection result so as to remove the blink of the eye from the open / closed state of the eyelid. Note that the preset threshold value Tth is not limited to 0.3 seconds.

  After the detection unit 185 detects the open / closed state of the eyelids, the display control unit 190 is assigned to each pattern that combines the open / closed states of the right eyelid 72 and the left eyelid 74 at the first detection time T1 and the second detection time T2. It is determined whether or not there is a function (step S520).

  FIG. 8 is an explanatory diagram illustrating an example of a function corresponding to each eyelid pattern of the user of the head-mounted display device 100 in the second embodiment. Since the detection unit 185 detects the open / closed state of the right eyelid 72 and the left eyelid 74 at the time of the first detection time T1 and the second detection time T2, the patterns of the open / closed state of the eyelids are 16 patterns P01 to P16. Exist.

  As shown in FIG. 8, there are a pattern to which a function is assigned and a pattern to which no function is assigned. For example, since the pattern P16 is always the same as the normal state where the eyelids of both eyes are open, it is preferable not to assign a function to the pattern P16. Further, the eyelid opening / closing state differs between the pattern P03 and the pattern P04 in that the left eyelid 74 changes from the open state to the closed state or remains in the closed state at the second detection time T2. is there. When executing the function assigned to the pattern P03, if the left eyelid 74 is kept open until the second detection time T2, the display control unit 190 converts the control signal to the function assigned to the pattern P04 and displays the image. To the unit 20. For this reason, in the head-mounted display device 100 according to the second embodiment, a function different from the function intended by the user may be executed. Therefore, a function is assigned to one of the patterns P03 and P04. Preferably not. Similarly, it is preferable not to assign a function to the patterns P01, P04, and P13 in which both eyelids do not change from the first detection time T1 to the second detection time T2. Further, there is a pattern in which it is preferable not to assign a function in order to prevent execution of a function that is not intended by the user depending on the interval between the first detection time T1 and the second detection time T2. In addition, the function allocated according to a pattern is not restricted to the function shown in FIG.

  When the function according to the pattern detected by the detection unit 185 is not assigned (step S520: NO), the control process of the head-mounted display device 100 in the second example is finished.

  When the function according to the pattern detected by the detection unit 185 is assigned (step S520: YES), the display control unit 190 executes the function assigned to the pattern (step S530). For example, when the user opens the right eyelid 72 at the first detection time T1 and closes the left eyelid 74 and then closes the right eyelid 72 and opens the left eyelid 74 at the second detection time T2, the display control unit 190 The function assigned to the pattern P10 is converted into a control signal, and the “return” operation of the browser is performed. The control signal in the second embodiment corresponds to the input information of the present invention.

  As shown in FIG. 8, in addition to the pattern P10, functions corresponding to the patterns are also assigned to the pattern P02, the pattern P03, the pattern P05, the pattern P07, the pattern P12, and the pattern P15. The function assigned to the pattern P12 is a function performed on a program such as a browser as in the pattern P10. The other functions of the patterns are the same as the operation modes in the head-mounted display device 100 of the first embodiment. Unlike the functions of the patterns P10 and P12, the contents of the image that the user is viewing. It is executed regardless of (for example, the Internet and moving image content).

  As described above, in the head-mounted display device 100 in the second embodiment, the detection unit 185 detects the state of the user's eyelids, and the display control unit 190 functions assigned to each detected eyelid pattern. Therefore, the user can operate without using a hand, and the convenience can be improved.

  Further, in the head-mounted display device 100 according to the second embodiment, the detection control unit 183 detects the open / closed state of the right eyelid 72 and the left eyelid 74 at the first detection time T1 and the second detection time T2, and detects them. The unit 185 detects the pattern of the open / closed state of the eyelid. The display controller 190 operates by converting the function assigned to the eyelid pattern into a control signal. Therefore, in the head-mounted display device 100 in the second embodiment, the user can operate without using a hand, and has a plurality of functions assigned according to the pattern of the open / closed state of both eyelids. Since it can be executed, the convenience can be further improved. Further, in the head-mounted display device 100 according to the second embodiment, the detection unit 185 detects only two types of the eyelid open state and the closed state, so that there are few erroneous detections and highly accurate operation. Can be realized. Moreover, since the head-mounted display device 100 in the second embodiment includes the control unit 10 and the image display unit 20, it is possible to realize a highly portable image display device.

  In the head-mounted display device 100 according to the second embodiment, the detection control unit 183 determines that the eye blinks when the eyelid closing time is smaller than a preset threshold value Tth, and the eyelid is closed. The detection unit 185 detects the open / closed state of the eyelid, interpreting it as an open state rather than a closed state. Therefore, in the head-mounted display device 100 according to the second embodiment, the detection unit 185 does not detect a state where the eye blink eyelid is closed, so that erroneous detection of the eyelid state can be avoided.

  In the head-mounted display device 100 according to the second embodiment, since the pattern P16 is always the same as the normal state where the eyes of both eyes are open, no function is assigned to the pattern P16. No function is assigned to any of the patterns P04. Further, no function is assigned to the patterns P01, P04, and P13 in which both eyelids do not change from the first detection time T1 to the second detection time T2. Therefore, in the head-mounted display device 100 according to the second embodiment, since functions are not assigned to some of the eyelid patterns, when the functions assigned to each pattern are executed, the user can Unintended function execution can be avoided.

C. Variations:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

C1. Modification 1:
The configuration of the head-mounted display device 100 in the above embodiment is merely an example, and various modifications can be made. In the first embodiment, the detection unit 185 determines that the user is asleep when detecting that the user's eyelid has been closed for a predetermined time. However, the user sometimes has a closed eyelid and is not sleeping, or has a thin eye. Therefore, the display control unit 190 may cause the image display unit 20 to display a notification that the power saving transition is performed immediately before the power saving transition is performed. At the same time, the voice processing unit 170 may send a notification to the effect of shifting to power saving with sound. Note that the notification that the power saving transition is performed may be only the display of the display control unit 190 or only the sound of the audio processing unit 170. When the display control unit 190 detects that the user's eyelid has been opened after the notification that the power saving transition is performed and before the power saving transition is performed, the display control unit 190 Do not migrate. Therefore, the head-mounted display device 100 does not perform the power saving transition only by the state of the user's eyelid detected by the detection unit 185, but also confirms the user's intention and performs the power saving transition. Can do.

C2. Modification 2:
In the first embodiment, the power saving transition and the return transition are stepwise transitions according to the state of the user's eyelid, but may be power saving transitions and return transitions that are not stepwise. For example, the head-mounted display device 100 may suddenly turn off the power switch 18 according to the state of the user's eyelid from the normal operation, or may suddenly shift to the standby mode. The image generation units 244 and 246 may be stopped.

C3. Modification 3:
In the above embodiment, the detection unit 185 detects the state of the user's eyelid using the infrared sensor 50, but may detect it using another detection device. For example, the detection unit 185 may detect the state of the eyelid using a photographing element such as a CCD. Further, the detection unit 185 may detect the state of the eyelid by detecting the movement of the muscle when the eye blinks using a pressure sensor arranged in the ear hook unit 21.

  In the above embodiment, the detection unit 185 realizes the detection of the state of the user's eyelid using the infrared sensor 50, but can be substituted by various known methods. For example, it can be realized by a detection method using a reflective sensor in which an infrared light emitting diode and a photodiode are placed in front of the eye.

C4. Modification 4:
The predetermined time during which the user's eyelid continues to be closed, which is the determination criterion for the power saving transition in the first embodiment, is merely an example, and can be set to an arbitrary value. In determining the return shift, it may be determined that the return shift is performed when the user's eyelid opens for a predetermined time (for example, 3 seconds).

C5. Modification 5:
In the first embodiment, the control unit 10 cancels the standby mode when the power switch 18 of the operation unit 110 is pressed. However, the control unit 10 may cancel the standby mode by other operations. The control unit 10 may cancel the standby mode when a button other than the power switch 18 of the operation unit 110 is pressed. Further, an acceleration sensor may be arranged in the operation unit 110, and the control unit 10 may cancel the standby mode when the operation unit 110 detects a predetermined acceleration.

C6. Modification 6:
In the above embodiment, the control unit 10 stops the operations of the image processing unit 160, the display control unit 190, the detection control unit 183, and the audio processing unit 170 in the standby mode, but all the operations of these functional units are performed. Does not have to be stopped. For example, the control unit 10 can reduce the power consumption by stopping only the operation of the image processing unit 160.

  In the above embodiment, the operating system 150, the image processing unit 160, the display control unit 190, the detection unit 185, and the sound processing unit 170 are realized by the CPU 140 executing software. It may be realized by hardware. For example, when the image processing unit 160 and the display control unit 190 are realized by electronic circuits, the control unit 10 may reduce the power consumption by stopping the operations of the image processing unit 160 and the display control unit 190. it can.

C7. Modification 7:
In 2nd Example, although specific time was illustrated about the timing which the detection part 185 detects a user's eyelid state, the timing to detect is not restricted to this. The length of time is not limited, and the user of the head-mounted display device 100 may be able to set the detection timing. The state of the eyelid detected at the start time T0 is not limited to the state of the above embodiment, and the start time T0 may be determined by the detection unit 185 detecting an eyelid state different from the second embodiment. For example, the start time T0 may be when a state in which both eyelids are closed for a predetermined period is detected, or when a predetermined number of blinks is detected within a predetermined period. The time when the operation unit 110 accepts the operation may be the start time T0, or the start time T0 is when the acceleration sensor or the like provided in the head-mounted display device 100 senses the movement of the user's head. It is good. Further, the state of the eyelid detected by the detection unit 185 is not limited to the open / closed state of the eyelid in the above-described embodiment, and the eyelid state may be detected by the CCD or the like with the opening area of the eyelid. Also, the function assigned according to the state of the eyelid is not limited to that of the above embodiment, and may be a function of stopping or playing music content, for example.

  Also, the pattern of the open / closed state of the right eyelid 72 and the left eyelid 74 is not limited to the pattern of the above embodiment. FIG. 9 is an explanatory diagram illustrating an example of a function corresponding to each eyelid pattern of the user of the head-mounted display device 100 according to the modification. In the pattern shown in FIG. 9, unlike the pattern shown in FIG. 8 in the second embodiment, the display control unit 190 functions according to the pattern of only the open / closed state of the right eyelid 72 and the left eyelid 74 at the first detection time T1. Assigned. In the pattern shown in FIG. 9, the function assigned to the content being viewed by the user differs even in the same pattern. For example, the function assigned to the pattern P21 performs a “return” operation when the user is looking at the browser, and “goes to the next chapter” when the user is playing a video. Perform the action. Therefore, in the head-mounted display device 100 in this modified example, the user can operate without using a hand, and convenience can be improved. Note that since the pattern P24 shown in FIG. 9 is in a normal state with both eyelids open, no function is assigned to the pattern P24.

  FIG. 10 is an explanatory diagram illustrating an example of input data corresponding to the state of the user's eyelid of the head-mounted display device 100 according to the modification. FIG. 10A shows an example of input data corresponding to the eyelid pattern. FIG. 10B shows the relationship between the open / closed state of both eyelids and the pattern. In FIG. 10, the display control unit 190 converts the pattern of the open / closed state of the eyelid into input data. As shown in FIG. 10A, an address to be input when the right eyelid 72 is opened / closed is designated, and input data to be inputted when the left eyelid 74 is opened / closed is designated. For example, in the head-mounted display device 100 according to this modification, the detection unit 185 detects the open / closed state of the right eyelid 72 and the left eyelid 74 alternately four times every 0.5 seconds from the start time T0. To do. As shown in FIG. 10B, each eyelid pattern in the modified example can be expressed in binary and there are 16 patterns. As shown in FIG. 10A, the address specified by the combination of the open / closed states of the right eyelid 72 is “6” in hexadecimal notation, and the input data by the combination of the open / closed states of the left eyelid 74 is “F”. is there. The display control unit 190 inputs the input data corresponding to “F” to the address corresponding to “6”. Therefore, in the head-mounted display device 100 in this modified example, the user can input input data without using a hand, and convenience can be improved. The input data and the designated address correspond to the input information of the present invention. The instruction according to the state of the eyelid of the present invention includes address designation and input data input.

  In the second embodiment, the display control unit 190 executes a function assigned according to the detected eyelid opening / closing state pattern, but the operation unit 110 provided in the control unit 10 in addition to the eyelid pattern. You may convert into a control signal according to the combination with the operation which (the touchpad 14, the cross key 16, the power switch 18, etc.) received. For example, if the detection unit 185 is set to detect the eyelid state only when the cross key 16 of the control unit 10 is pressed, the timing of the start time T0, the first detection time T1, and the second detection time T2 is used. Therefore, it is possible to avoid erroneous detection of the state of the eyelid. In addition, more functions can be assigned by combining eyelid patterns and operations than functions assigned only by eyelid patterns.

  In addition to the eyelid pattern, the display control unit 190 is connected to the control unit 10 according to a combination with the movement of the user's head sensed by an acceleration sensor or the like provided in the head-mounted display device 100 or the like. You may convert into a control signal. In the head-mounted display device 100 according to this modification, the user can operate without using a hand, and the user can freely determine the detection timing, thereby avoiding erroneous detection of the eyelid state. Can be assigned more functions. Further, it may be converted into a control signal in combination with an operation received by the operation unit 110. Note that the control signal of the head-mounted display device 100 in the modification corresponds to the input information of the present invention.

  In the second embodiment, the detection unit 185 detects the open / closed state of each eyelid at the first detection time T1 and the second detection time T2, but the eyelid detection condition is not limited to this. For example, in the head-mounted display device 100 according to the second embodiment, the detection unit 185 detects the number of times the eyelid is opened and closed in the section from the start time T0 to the second detection time T2, and the display control unit 190 sets the number of times. A function assigned to the corresponding pattern may be executed. In the head-mounted display device 100 according to this modified example, the display control unit 190 executes the function assigned to the pattern according to the number of times the eyelid is opened and closed for a predetermined time detected by the detection unit 185. The operation can be performed by changing the state of the eyelid, and convenience can be improved.

  In the second embodiment, the display control unit 190 executes a function assigned according to the pattern of the open / closed state of the right eyelid 72 and the left eyelid 74, but only the right eyelid 72 or the left eyelid. The function may be executed according to only the open / closed state of 74. In the second embodiment, the timing at which the detection unit 185 detects the eyelid state is based on the relationship with time, but this need not be the case. For example, the display control unit 190 may notify the user of the timing at which the detection unit 185 detects the eyelid state by sound. Further, the display control unit 190 displays the image displayed on the image display unit 20, for example, by displaying a count until the detection of the eyelid state on the image display unit 20. You may alert | report and the pattern detected simultaneously may be displayed on the image display part 20. FIG.

C8. Modification 8:
In the above embodiment, the cross key 16 provided in the control unit 10 may be omitted, or another operation interface such as an operation stick may be provided in addition to the cross key 16 and the touch pad 14. Moreover, the control part 10 is a structure which can connect input devices, such as a keyboard and a mouse | mouth, and is good also as what receives an input from a keyboard or a mouse | mouth. In addition, the control unit 10 may be provided with a communication unit such as a wireless LAN.

  Moreover, in the said Example, although the control part 10 and the image display part 20 are assumed to be wired-connected by the connection part 40, the control part 10 and the image display part 20 are wireless LAN, infrared communication, Bluetooth (Bluetooth: Bluetooth: It may be connected via a signal transmission line such as a registered trademark.

  In the above embodiment, the image display unit 20 is an HMD worn like glasses, but the image display unit 20 is a normal flat display device (liquid crystal display device, plasma display device, organic electroluminescence (Electro-Luminescence). : EL) display device, etc.). Also in this case, the connection between the control unit 10 and the image display unit 20 may be a connection via a wired signal transmission path or a connection via a wireless signal transmission path. . In this way, the control unit 10 can be used as a remote controller for a normal flat display device.

  Further, instead of the image display unit 20 worn like glasses, an image display unit having another shape such as an image display unit worn like a hat may be adopted as the image display unit. Further, the earphones 32 and 34 may be omitted.

  Moreover, in the said Example, although LCD and a light source are utilized as a structure which produces | generates image light, it replaces with these and you may employ | adopt other display elements, such as an organic EL display.

  In the above embodiment, the head-mounted display device 100 is a binocular optically transmissive head-mounted display device. However, the present invention can be applied to, for example, a video transmission type, a non-transmission type, or a monocular type. The present invention can be similarly applied to other types of head mounted display devices. Furthermore, the present invention is applicable not only to a head-mounted display device but also to operations of devices such as a personal computer, a projector, and a television.

  In the above-described embodiment, the head-mounted display device 100 includes the control unit 10 and the image display unit 20, but the configuration is not limited thereto. A device other than the head-mounted display device 100 (for example, a portable terminal, a machine operation unit such as a factory, a medical terminal, a measuring instrument such as an oscilloscope, a projection image display device, a monitor of a personal computer, a television, etc.) In addition, the control unit 10 gives an instruction according to the state of the eyelid detected by the detection unit 185, so that the user can operate without using a hand, and convenience can be improved.

  In addition, even devices such as detection instruction devices, state detection control devices, and eyelid detection control devices give instructions according to the state of the user's eyelid detected by the control unit, and the user can operate without using his / her hands. , Can improve convenience. For example, in the remote control of the apparatus, the operation lock can be set to perform on / off control for opening / closing the eyelid. The computer mouse pointer can also be controlled to give the same instruction as double-clicking the mouse for two blinks of both eyes within a predetermined time, or the mouse pointer can be controlled when the right eyelid is closed. It can also be controlled to move to the right at speed.

  In the above embodiment, a part of the configuration realized by hardware may be replaced by software, and conversely, a part of the configuration realized by software may be replaced by hardware. Good. For example, in the above embodiment, the image processing unit 160 and the sound processing unit 170 are realized by the CPU 140 reading and executing a computer program. However, these functional units may be realized by a hardware circuit. Good.

  In addition, when part or all of the functions of the present invention are realized by software, the software (computer program) can be provided in a form stored in a computer-readable recording medium. In the present invention, the “computer-readable recording medium” is not limited to a portable recording medium such as a flexible disk or a CD-ROM, but an internal storage device in a computer such as various RAMs and ROMs, a hard disk, etc. It also includes an external storage device fixed to the computer.

C9. Modification 9:
In the above embodiment, the head-mounted display device 100 may guide the image light representing the same image to the left and right eyes of the user to make the user visually recognize the two-dimensional image, or the left and right eyes of the user. Alternatively, image light representing different images may be guided to allow the user to visually recognize the three-dimensional image.

  In the above embodiment, the content data is supplied from the external device OA. However, the content data is stored in the storage unit 120 of the control unit 10 of the head-mounted display device 100 and stored in the storage unit 120. Image display may be performed based on the content data.

  In addition, among the constituent elements in the above-described embodiment, elements other than the elements described in the independent claims are additional elements, and can be omitted or combined as appropriate.

DESCRIPTION OF SYMBOLS 10 ... Control part 12 ... Lighting part 14 ... Touch pad 16 ... Cross key 18 ... Power switch 20 ... Image display part 21 ... Ear hook part 22 ... Right display drive part 24 ... Left display drive part 26 ... Right optical panel (optical panel) )
28 ... Left optical panel (optical panel)
32 ... Right earphone (earphone)
34 ... Left earphone (earphone)
DESCRIPTION OF SYMBOLS 40 ... Connection part 42 ... Right cable 44 ... Left cable 46 ... Connection part 48 ... This cable 50 ... Infrared sensor 51 ... Infrared light emission part 52 ... Right infrared light emission part 54 ... Left infrared light emission part 56 ... Infrared light reception part 62 ... Right eye 64 ... Left eye 72 ... Right eyelid 74 ... Left eyelid 100 ... Head-mounted display device 110 ... Operation unit 120 ... Storage unit 130 ... Power supply 140 ... CPU
DESCRIPTION OF SYMBOLS 150 ... Operating system 160 ... Image processing part 170 ... Audio | voice processing part 180 ... Interface 183 ... Detection control part 185 ... Detection part 190 ... Display control part 201 ... Right light source control part (light source control part)
202 ... Left light source controller (light source controller)
211 ... Right LCD control unit 212 ... Left LCD control unit 221 ... Right light source (light source)
222 ... Left light source (light source)
241 ... Right LCD
242 ... Left LCD
244 ... Right eye image generation unit (image generation unit)
246 ... Left eye image generation unit (image generation unit)
251 ... Right projection optical system (light guide unit)
252 ... Left projection optical system (light guide)
261 ... Right light guide plate (light guide part)
262 ... Left light guide plate (light guide part)
T0 ... start time T1 ... first detection time T2 ... second detection time ES1 ... detection open / close signal ES2 ... interpretation open / close signal Tth ... threshold

Claims (18)

A device,
A detection unit that detects an eyelid state including an open / closed state of the right eyelid and an open / closed state of the left eyelid at a plurality of predetermined timings;
A control unit that gives instructions according to the eyelid state detected by the detection unit,
The controller is
For the case where the first and eyelids state wherein the detection unit is a pre-Symbol eyelid state that put the first timing detected, the second and the eyelid condition is pre Symbol eyelid state that put the second timing, different Only, each pattern defined by the combination of the first eyelid state and the second eyelid state is converted into input information according to a predetermined rule, and a predetermined operation according to the input information is performed,
An apparatus that does not perform a predetermined operation corresponding to a combination of the eyelid states when the first eyelid state and the second eyelid state are the same. A head-mounted display device,
An image display unit that generates and emits image light based on image data, and a light guide unit that guides the image light to the user's eyes, and allows the user to visually recognize a virtual image;
The apparatus according to claim 1 connected to the image display unit,
The control unit transmits the image data to the image display unit, controls image display by the image display unit, and has a plurality of different power consumptions according to the state of the user's eyelid detected by the detection unit. A head-mounted display device that switches the operation mode. The head-mounted display device according to claim 2 ,
The image light generation unit includes a light source that emits light, and an image generation unit that converts the light emitted from the light source based on image data into image light representing an image,
The head-mounted display device, wherein the control unit reduces the light emitted from the light source during a power saving transition, which is a transition to an operation mode with lower power consumption . The head-mounted display device according to claim 2 ,
The control unit includes a function unit that realizes a plurality of functions, and stops part of the operation of the function unit that realizes the plurality of functions at the time of power saving transition that is a transition to an operation mode with lower power consumption. A head-mounted display device. The head-mounted display device according to claim 4 ,
The control unit further includes an operation unit, and the head-mounted display device restarts the operation of the function unit that has been stopped when the operation unit receives a specific operation after the power saving transition. . The apparatus of claim 1, comprising:
Pattern wherein the control unit includes a first Noma pig state, the second Noma pig state, but if the same is to be the default by the combination with the second eyelid with said first eyelid state A device that does not set input information corresponding to. An apparatus according to claim 1 or claim 6 , wherein
In the combination of a plurality of eyelid states, the control unit has the same first eyelid state, and when only one eyelid in the second eyelid state is different, the plurality of eyelid states A predetermined operation is set only for one of the combinations of the eyelid states among the combinations. An apparatus according to claim 6 or claim 7 , wherein
The control unit sets some predetermined operations among predetermined operations set corresponding to a combination of the plurality of eyelid states as operations of functions common to a plurality of programs executed by the device, and others. An apparatus that sets a part of a predetermined operation as an operation of a function that changes according to a plurality of programs executed by the apparatus. The apparatus according to any one of claims 6 to 8 , further comprising:
It has an audio output unit that outputs audio,
The said control part is an apparatus which alert | reports to a user by the timing of the audio | voice which the said audio | voice output part outputs at least one of the said 1st timing and the said 2nd timing. An apparatus according to any one of claims 6 to 9 , comprising:
The said control part is an apparatus which inputs the input data according to the said input information. The apparatus according to any one of claims 6 to 10 , further comprising:
It has an operation unit that accepts user operations,
The said control part is an apparatus which converts the combination of each said pattern and the said operation into the said input information according to the said predetermined rule. The apparatus of claim 11 , comprising:
The control unit starts the first timing when the operation unit receives a predetermined operation or when the detection unit detects a change in the eyelid state a predetermined number of times within a predetermined period. Set as a device. A device according to any of claims 6 to 12 ,
The said detection part is an apparatus which interprets that the eyelid is an open state, when the time which the eyelid is closed is shorter than predetermined value among the states of the said eyelid. The apparatus according to any one of claims 6 to 13 , further comprising:
It has a sensing unit that senses the movement of the user's head,
The said control part is an apparatus which converts the combination of each said pattern and the motion of the said head into the said input information according to the said predetermined rule. A head-mounted display device,
An image display unit that generates and emits image light based on image data, and a light guide unit that guides the image light to the user's eyes, and allows the user to visually recognize a virtual image;
The apparatus according to any one of claims 6 to 14 , which is connected to the image display unit, transmits the image data to the image display unit, and controls image display by the image display unit. Head-mounted display device. The head-mounted display device according to claim 15 ,
The control unit is a head-mounted display device that notifies a user of at least one of the first timing and the second timing by image display by the image display unit. An apparatus control method comprising:
Detecting the eyelid state including the open / closed state of the right eyelid and the open / closed state of the left eyelid at a plurality of predetermined timings;
A control step of instructing the device according to the detected state of the eyelid,
The control step includes
When the first and eyelid condition is first pre Symbol eyelid state that put the timing of which is detected, the second and the eyelid condition is pre Symbol eyelid state that put the second timing is different, the first Each pattern defined by a combination of one eyelid state and the second eyelid state is converted into input information according to a predetermined rule, and a predetermined operation according to the input information is performed,
A control method including a step of not performing a predetermined operation corresponding to a combination of the eyelid states when the first eyelid state and the second eyelid state are the same. A head having an image light generation unit that generates and emits image light based on image data, and a light guide unit that guides the image light to a user's eyes, and has an image display unit that allows the user to visually recognize a virtual image. A method for controlling a part-mounted display device,
A control step of transmitting the image data to the image display unit and controlling image display by the image display unit;
A detection step of detecting an eyelid state including an open / closed state of the user's right eyelid and an open / closed state of the left eyelid at a plurality of predetermined timings,
The control step includes
According to the state of the eyelid detected in the detection step, a step of switching a plurality of operation modes with different power consumption,
When the first and eyelid condition is first pre Symbol eyelid state that put the timing of which is detected, the second and the eyelid condition is pre Symbol eyelid state that put the second timing is different, the first Each pattern defined by a combination of one eyelid state and the second eyelid state is converted into input information according to a predetermined rule, and a predetermined operation according to the input information is performed,
And a step of not performing a predetermined operation corresponding to a combination of the eyelid states when the first eyelid state and the second eyelid state are the same.

Images