WO2010064361A1

WO2010064361A1 - Head-mounted display

Info

Publication number: WO2010064361A1
Application number: PCT/JP2009/006012
Authority: WO
Inventors: 矢田裕紀
Original assignee: ブラザー工業株式会社
Priority date: 2008-12-02
Filing date: 2009-11-11
Publication date: 2010-06-10
Also published as: JP2010134057A

Abstract

Disclosed is a head-mounted display, which visibly provides a content image, which is depicted with content data, to a user's eyes. The head-mounted display comprises a control means that controls the operation of the head-mounted display, and a detection means, which detects that the state of specified parts of the user's face, which do not move unless the user wills the movement thereof, is a predetermined state. The control means controls for operations that are associated with the predetermined state, per step S210, on condition that the detection means detects that the state of the specified parts of the user's face is the predetermined state, per steps S202, S206, and S208.

Description

Head mounted display

The present disclosure relates to a head-mounted display that presents a content image indicated by content data to a user's eyes so that the user can recognize the content image.

A technology related to a head-mounted display has been proposed in which a content image indicated by content data is presented to a user's eyes so that the user can recognize the content image. For example, in order to improve the operability of the head mounted display, the gaze direction, focal length, pupil state, fundus pattern, eyelid movement of the user (user) wearing the head mounted display is detected by a sensor. A technique has been proposed in which a user's intention or situation is determined from information acquired by a sensor, and based on the determination result, a control is performed on display operation in a display unit that displays an image (content image) ( For example, see Patent Document 1).

JP 2008-65169 A

With regard to the operation of the head-mounted display, the operation that instructs the user's intended operation is not performed via the user interface that the user directly operates through the hand etc., but the state of the pupil of the user's body, the fundus pattern, the eyelid A technique that can be performed based on movement is a useful technique because it can reduce the operation burden on the user. For example, in a production site of a product, a work instruction is presented to a user who wears the head-mounted display, and when the user assembles a product, both hands of the user who is working are It is used for product assembly work and it is difficult to operate the head mounted display.

In such a situation, the above technique can reduce the burden of operation on the head mounted display by the user wearing the head mounted display. However, in this case, it is important that an operation that matches the user's intention is accurately executed in the head mounted display. For example, maintaining a state in which the line-of-sight direction is artificially fixed in a certain direction is a painful operation for humans, and usually the direction of the user's line-of-sight is constantly changing. Furthermore, depending on various stimuli from the outside world, the line of sight may be changed without intention to operate. Therefore, it is difficult to distinguish whether the eye direction is detected by the method of detecting the direction of the line of sight, and erroneous detection occurs. The movement of the eyelids is the same, and the eyelids are usually unintentionally blinked or the eyelids are closed without intention to operate due to fatigue or the like, and erroneous detection is likely to occur. Also, it is difficult to intentionally change the state of the pupil and the fundus pattern.

This disclosure aims to provide a highly reliable head mounted display that is excellent in operability with respect to an operation for instructing the operation of the head mounted display.

The head-mounted display according to the present disclosure has a specific state when there is almost no unintentional movement and the user consciously moves among the parts constituting the face of the user wearing the head-mounted display. Detecting the state of a specific part and controlling the operation associated with the detected state to be executed on the head mounted display on condition that the state of the specific part is a specific state It is what.

According to an aspect of the present disclosure, a head-mounted display that presents a content image indicated by content data to a user's eye so that the user can recognize the content image, the head-mounted display Control means for controlling the operation of the control unit, and detection means for detecting that a state of a specific part of the user's face that does not move unless the user is conscious is a predetermined state, the control means Is characterized in that the operation associated with the predetermined state is controlled on condition that the detection unit detects that the state of a specific part of the user's face is the predetermined state. A head mounted display can be obtained.

According to such a head mounted display, an erroneous operation due to an unconscious operation is unlikely to occur with respect to an operation for instructing an operation of the head mounted display, and a reliable operation is performed according to the user's intention without using a hand. be able to.

According to another aspect of the present disclosure, the detection unit detects that the state of the eyebrow of the user is the predetermined state as a state of a specific part of the user's face, and the control unit A head-mounted display that controls the operation associated with the predetermined state on condition that the detection unit detects that the state of the eyebrows of the user is the predetermined state; Obtainable.

According to still another aspect of the present disclosure, the detection unit detects that the state of the user's cheek is the predetermined state as a state of a specific part of the user's face, and the control unit The head-mounted display controls the operation associated with the predetermined state on condition that the detecting unit detects that the state of the cheek of the user is the predetermined state. Can be obtained.

According to still another aspect of the present disclosure, the detection unit detects that the state of the user's mouth is the predetermined state as a state of a specific portion of the user's face, and the control unit The head-mounted display controls the operation associated with the predetermined state on condition that the detection unit detects that the state of the mouth of the user is the predetermined state. Can be obtained.

In such a head-mounted display, unless the user wearing the head-mounted display is particularly conscious, a large state change is unlikely to occur, and the operation of the head-mounted display can be controlled based on the state of the eyebrows, cheeks or mouth. Therefore, the operability of the head mounted display can be improved while suppressing malfunction.

The “operation of the head mounted display” includes, for example, processing related to selection of content data, playback / stop of content data executed when presented to the user's eyes, presentation of content images to the user's eyes, Alternatively, it includes supplying or shutting off power to the head mounted display and other various operations realized in the head mounted display. Further, the head-mounted display is not necessarily a single device, and may be configured by two devices, for example. For example, the control means may be configured as another device and connected to the head mounted display with a predetermined signal cable.

According to still another aspect of the present disclosure, the detection unit includes an imaging unit that can capture a specific image of a specific part of the user's face, and an analysis unit that analyzes the specific image captured by the imaging unit. In accordance with the analysis result of the analysis means, it is possible to obtain a head-mounted display that detects that the state of a specific part of the user's face is the predetermined state . In such a head mounted display, the state of a specific part of the user's face can be detected appropriately.

According to still another aspect of the present disclosure, the detection unit includes: a light emitting element capable of irradiating a specific part of the user's face; and the light applied to the specific part of the user's face. A light detecting element for detecting reflected light, and when the intensity of the reflected light detected by the light detecting element deviates from a predetermined reference value, the state of a specific part of the user's face is: It is possible to obtain a head-mounted display that detects the predetermined state. In such a head mounted display, the state of a specific part of the user's face can be detected appropriately.

According to still another aspect of the present disclosure, the control unit controls a first operation and a second operation as operations of the head mounted display, and the detection unit is configured to detect a specific part of the user's face. When the state is the first state, the first state is detected as the predetermined state, and when the state of the specific part of the user's face is the second state, the second state is the predetermined state. The control means controls the first operation associated with the first state on the condition that the detection means detects the first state, and the detection means It is possible to obtain a head mounted display that controls the second operation associated with the second state on condition that the state is detected in two states. In such a head mounted display, a plurality of operations can be assigned to the state of a specific part of the user's face, and the plurality of operations can be controlled.

(A) thru | or (c) is a figure which shows the external appearance of a head mounted display main body. It is a figure which shows a head mounted display. It is a figure which shows the functional block of the control box of a head mounted display. (A) thru | or (d) is a figure which shows notionally the registration state of a table. It is a figure which shows the flow of a main process. It is a figure which shows the flow of a state determination process. It is a figure which shows notionally the registration state (registration state regarding the state of an eyebrow) of another table.

Embodiments reflecting the present disclosure will be described in detail below with reference to the drawings. Note that the present disclosure is not limited to the configurations described below, and various configurations can be employed in the same technical idea. For example, in the following description, a head mounted display configured by connecting a head mounted display main body and a control box that provides a content image to the head mounted display main body will be described as an example. It can also be configured as a device. In the following description, the head-mounted display body is simply referred to as a head-mounted display (hereinafter referred to as “HMD”).

(Overview of head mounted display)
As shown in FIG. 1, the HMD 100 includes

temples

104A and 104B,

armatures

106A and 106B, and a front frame 108. Moderns 102A and 102B that hit the user's ear are attached to one end of the

temples

104A and 104B. Trillion

numbers

112A and 112B are provided at the other ends of the

temples

104A and 104B.

Temples

104A and 104B and Yoroi 106A and 106B are connected via these trillion

numbers

112A and 112B. The front frame 108 connects the

end pieces

106A and 106B. A nose pad 110 that contacts the user's nose is attached to the center of the front frame 108. The skeleton of the HMD 100 is formed by the

temples

104A and 104B, the

armatures

106A and 106B, the front frame 108, and the nose pad 110. The

temples

104A and 104B can be folded at the

trillions

112A and 112B formed on the

armatures

106A and 106B. The structure of the skeleton part of HMD100 is the same as that of normal glasses, for example. As shown in FIG. 2, the HMD 100 is supported on the user's face by the modern 102 A and 102 B and the nose pad 110 when the HMD 100 is worn on the user. In FIG. 1B, drawing of the

moderns

102A and 102B and the

temples

104A and 104B is omitted.

The image presentation device 114 is attached to the skeleton part of the HMD 100 via an attachment part 122 provided (provided) near the armor 106A. The image presentation device 114 is disposed at a position that is substantially the same height as the left eye 118 of the user wearing the HMD 100 in a state where the image presentation device 114 is attached to the vicinity of the armor 106A via the attachment portion 122. As shown in FIG. 2, the image presentation device 114 is connected to the control box 200 via a predetermined signal cable 250. Although details will be described later, the control box 200 executes a rendering process on the content data stored in the predetermined area. The control box 200 controls the input / output interface (hereinafter referred to as “I / F”) included in the own device, thereby transmitting the content image signal including the content image obtained by the rendering process (reproduction process) to the signal cable. The image is output to the image presentation device 114 via 250. The image presentation device 114 acquires the content image signal output from the control box 200 via an input / output I / F that is not drawn in FIGS. Then, the content image based on the content image signal is optically emitted toward the half mirror 116.

The content image (light beam) emitted from the image presentation device 114 is reflected by the half mirror 116 and is incident on the user's left eye 118, in other words, presented (projected) so as to be visible. Thereby, the user recognizes the content image. Here, in FIG. 1A, reference numeral 120a indicates a light beam related to the content image emitted from the image presentation device 114, and reference numeral 120b indicates a light beam that reflects the half mirror 116 and enters the left eye 118 of the user. Show. The image presentation device 114 scans the light rays 120a and 120b corresponding to the acquired content image signal in a two-dimensional direction, guides the scanned

light rays

120a and 120b to the user's left eye 118, and displays the content image on the retina. In addition to a retinal scanning type display that forms a liquid crystal display, a liquid crystal display, an organic EL (Organic Electroluminescence) display, and other devices may be used.

The eyebrow sensor 214 is attached to the upper surface of the image presentation device 114, and the cheek sensor 216 is attached to the lower surface. As shown in FIG. 2, a stay 124 (not shown in FIG. 1) with a mouth sensor 218 attached to the tip is attached to the temple 104A.

(Control box configuration)
For example, the control box 200 is attached to a user's waist or the like. As shown in FIG. 3, the control box 200 includes a CPU 202 that controls the apparatus itself, a ROM 204 that stores various programs, a RAM 206 as a work area, a storage unit 208 that stores content data 2082 and a table 2084. , An input / output I / F 210 that transmits / receives various signals to / from the HMD 100, and an operation unit 212 that is operated by the user and receives an instruction from the user. Further, the control box 200 includes an eyebrow sensor 214 attached to the upper surface of the image presentation device 114, a cheek sensor 216 attached to the lower surface, and a mouth sensor 218 attached to the tip of the stay 124 (see FIG. 2). , Is connected.

Here, the storage unit 208 is constituted by, for example, a hard disk. The content data 2082 stored in the storage unit 208 includes, for example, a work instruction describing a method for assembling a predetermined product (hereinafter, the case where the content data 2082 is a work instruction is taken as an example). explain.). The table 2084 associates each state of the eyebrows, cheeks, and mouth detected by the eyebrow sensor 214, cheek sensor 216, and mouth sensor 218 attached to the HMD 100 with each action realized in the HMD 100 and the control box 200. Registered table. The operation unit 212 includes, for example, keys, and receives instructions to start and end (stop) playback of the content data 2082.

The eyebrow sensor 214 detects the state (movement) of the user's left eyebrow, and the cheek sensor 216 detects the state (movement) of the user's cheek. The mouth sensor 218 detects the state (shape) of the user's mouth, more specifically, the user's lips. As the eyebrow sensor 214, the cheek sensor 216, and the mouth sensor 218, for example, an image sensor composed of a CCD (Charge Coupled Devices) or a photoelectric sensor composed of a light emitting element and a light detecting element can be used. Detection signals including states detected by the

sensors

214, 216, and 218 are input to the control box 200 and stored on the RAM 206. The state detected by each

sensor

214, 216, 218 is indicated by an image indicating a state such as eyebrows in the case of an image sensor, and is indicated by the intensity of reflected light in the case of a photoelectric sensor.

The CPU 202 acquires a content image by executing a program for reproducing (rendering) the content data 2082 stored in the ROM 204 on the RAM 206. Then, a program for controlling the input / output I / F 210 stored in the ROM 204 is executed on the RAM 206, and a content image signal including a content image is output from the input / output I / F 210 to the HMD 100. The CPU 202 detects an analysis program (for example, a program for pattern matching) stored in the ROM 204 by the

sensors

214, 216, and 218, and uses the detection signal and the table 2084 stored in the RAM 206. By executing on the RAM 206, the state of the user's eyebrows, cheeks and mouth is analyzed. Further, the CPU 202 executes a program for controlling the HMD 100 stored in the ROM 204 on the RAM 206, thereby performing the operation of the HMD 100 instructed by the operation unit 212 and the operation of the HMD 100 based on the analysis result. Control. Therefore, the CPU 202 uses the various data such as the content data 2082, the table 2084, and the detection signal, and executes various programs stored in the ROM 204 on the RAM 206, whereby various functional units (for example, a control unit and an analysis unit) are used. Composed.

As shown in FIG. 4A, a standard face image, which is a face image of the user in the standard state, is registered in the table 2084. The operation of the HMD 100 is not associated with the standard face image. In other words, the standard face image is associated with continuing the operation in progress on the HMD 100, for example, continuously executing the reproduction operation.

As shown in FIG. 4B, the face image indicating the state of “raising the eyebrows” is associated with the operation “next”. For example, in a state where a predetermined content image is presented to the user's left eye 118, when the user raises his / her eyebrows, the content image next to the content image being presented is presented. More specifically, when a user who has finished the work in the first process raises the eyebrows in the state where the content image including the work instruction in the first process is presented, the work instruction related to the second process is given. A content image as content is presented.

Also, the action “return to the front” is associated with the face image indicating the state of “drawing the eyebrows”. Specifically, in the state where the content image including the work instruction relating to the second step is presented, when the user brings his / her eyebrows, the content image including the work instruction relating to the first step is presented. In the HMD 100 shown in FIGS. 1 and 2, the eyebrow sensor 214 is disposed so as to face the user's left eyebrow. Therefore, whether the eyebrow is raised or not is determined by the state of the left eyebrow. Judgment based on.

As shown in FIG. 4C, the action “decision” is associated with the face image indicating the state of “raised cheek”. Further, the operation “cancel” is associated with the face image indicating the state of “dented cheek”. For example, a content image containing the work instruction of the first step, a content image containing the work instruction of the second step, and a content image containing the work instruction of the third step are sequentially arranged at predetermined intervals. It is displayed while being switched. In this state, if the user raises the cheek while the content image including the work instruction in the second step is presented, the content image including the work instruction in the second step continues. The content image to be presented is determined. After this decision, if the user dents his cheek, this decision is canceled. Note that the cheek sensor 216 is also disposed opposite the user's left cheek, as with the eyebrow sensor 214, so whether or not the cheek is raised is determined based on the state of the left cheek. The

As shown in FIG. 4D, the “brightness adjustment” of the presented content image is associated with the face image indicating the state of the mouth when “e” is uttered. In addition, “brightness up” is associated with an image indicating the state of the mouth when “a” is uttered. Further, “brightness down” is associated with the state of the mouth when “o” is uttered. More specifically, if the user's left eye 118 is presented with a content image that contains the work instruction of the first step, it will become bright when the user speaks “e”. A setting screen for adjustment is presented. If the user speaks “A” with the setting screen presented, the brightness increases by one level. On the other hand, if the user speaks “O”, the brightness decreases by one level. Note that if the user makes “a” or “o” into the shape of a mouth that utters two times in succession, the brightness increases or decreases by two levels.

(Processing executed by the control box)
The main processing shown in FIG. 5 is started when the CPU 202 executes a program stored in the ROM 204 on the RAM 206 on condition that the HMD 100 and the control box 200 are turned on. In the execution of this process, the detection signal detected by the eyebrow sensor 214, cheek sensor 216, and mouth sensor 218 and input to the control box 200, the content data 2082 stored in the storage unit 208, and the table 2084 are displayed at a predetermined timing. Used.

The CPU 202 that has started the process first initializes the components of the control box 200 (S100), initializes the

sensors

214, 216, and 218 (S102), and moves the process to S104. In step S 104, the CPU 202 determines whether the user has input an instruction to start playback of the content data 2082 via the operation unit 212. As a result of the determination, if a playback start instruction is not input (S104: No), the CPU 202 waits until a playback start instruction is input. On the other hand, when an instruction to start reproduction is input (S104: Yes), the CPU 202 starts content image signal output processing (S106). Here, the content image signal output process is a process in which the content data 2082 is read from the storage unit 208 to the RAM 206, the content data 2082 is rendered, and the content image signal including the content image obtained by the rendering is input / output I / F 210. This is a process of controlling and outputting to the HMD 100.

The CPU 202 that has started the content image signal output process in S106 determines whether a face motion trigger has been detected from any one of the

sensors

214, 216, and 218 (S108). S108 will be described in detail. For example, the CPU 202 determines the state of the user's eyebrow (left eyebrow) included in the detection signal input from the eyebrow sensor 214 to the control box 200 and the state of the left eyebrow of the standard face image (see FIG. 4A). , A pattern matching process is executed to determine whether or not matching can be achieved between the two. Further, the CPU 202 determines between the state of the user's cheek (left cheek) included in the detection signal input from the cheek sensor 216 and the state of the left cheek of the standard face image (see FIG. 4A). A pattern matching process is executed, and it is determined whether or not matching can be obtained between the two. Furthermore, the CPU 202 determines between the mouth state (shape) of the user included in the detection signal input from the mouth sensor 218 and the mouth state (shape) of the standard face image (see FIG. 4A). A pattern matching process is executed in step (b), and it is determined whether or not matching can be achieved.

When matching is obtained in S108, in other words, when the state of the user's eyebrows, cheeks, and mouth matches the state of the left eyebrow indicated by the standard image, the CPU 202 has not input a face motion trigger. (S108: No), the process proceeds to S112. On the other hand, if any one of the eyebrows, cheeks, and mouth states of the user does not match the state of the left eyebrow indicated by the standard image, the CPU 202 determines that a face motion trigger has been input (S108: Yes), a state determination process is executed (S110).

The CPU 202 that has started the state determination process shown in FIG. 6 determines whether or not the face motion trigger detected in S108 is based on a change in eyebrow state (S200). If the state of the user's left eyebrow matches the state of the left eyebrow indicated by the standard image in S108, the CPU 202 determines that the change is not based on the change in eyebrow state (S200: No), and performs the process. The process proceeds to S204. On the other hand, if the two do not match, the CPU 202 determines that the face motion trigger is based on a change in the state of the eyebrows (S200: Yes), and moves the process to S202.

In S202, the CPU 202 determines again by the pattern matching process what the state of the user's left eyebrow included in the detection signal input to the control box 200 by the eyebrow sensor 214 is. The determination in S202 will be specifically described. The CPU 202 determines that the eyebrow state of the user included in the detection signal input by the eyebrow sensor 214 is the eyebrow state registered in the table 2084 (see FIG. 4B). Which of the following is true is determined. As a result of the determination, for example, when the state of the user's eyebrows included in the detection signal matches (matches) an image indicating the raised state, the CPU 202 specifies the operation “next”. Similarly, when the state of the eyebrows of the user included in the detection signal matches the image indicating the close state, the CPU 202 specifies the operation “return to the previous”. After executing S202, the CPU 202 shifts the process to S210.

In S204, the CPU 202 determines whether or not the face motion trigger detected in S108 is based on a change in cheek condition (S204). If the state of the user's left cheek matches the state of the left cheek indicated by the standard image in S108, the CPU 202 determines that the change is not based on a change in the state of the cheek (S204: No), and performs processing. The process proceeds to S208. On the other hand, if the two do not match, the CPU 202 determines that the face motion trigger is based on a change in cheek state (S204: Yes), and proceeds to S206.

In S206, the CPU 202 executes pattern matching processing to determine what the state of the user's left cheek included in the detection signal input to the control box 200 by the cheek sensor 216 is. The determination in S206 will be described in detail. The CPU 202 determines that the state of the user's cheek included in the detection signal input by the cheek sensor 216 is the state of the cheek registered in the table 2084 (see FIG. 4C). Which of the following is true is determined. As a result of the determination, for example, when the state of the user's cheek included in the detection signal matches (matches) the image indicating the raised state, the CPU 202 specifies the operation “decision”. Similarly, when the state of the user's cheek included in the detection signal matches the image indicating the depressed state, the CPU 202 specifies the operation “cancel”. After executing S206, the CPU 202 shifts the processing to S210.

In S 208, the CPU 202 determines that the face motion trigger detected in S 108 is based on a change in the mouth state (shape), and the user included in the detection signal input to the control box 200 by the mouth sensor 218. A pattern matching process is performed to determine whether the mouth (lips) state is “e”, “a”, or “o”. Continuing the description of the determination in S208, the CPU 202 determines that the state (shape) of the user's mouth included in the detection signal input by the mouth sensor 218 is the state of the mouth registered in the table 2084 (FIG. 4D). It is determined whether it corresponds to (see). As a result of the determination, for example, when the state of the mouth of the user included in the detection signal matches (matches) the image when “e” is uttered, the CPU 202 specifies the operation “brightness adjustment”. Similarly, when the state of the user's mouth included in the detection signal matches the image when “a” is uttered, the CPU 202 identifies the operation “brightness up” and when “o” is uttered If the image matches the image, the CPU 202 identifies the operation “brightness down”. After executing S206, the CPU 202 shifts the processing to S210.

In S210, the CPU 202 controls so that the operation specified in S202, S206, or S208 is executed in the content image signal output process being executed. For example, in the state where the content image including the work instruction related to the first process is presented, when the operation “proceed to the next” is specified in S202, the CPU 202 includes the content including the work instruction related to the second process. The rendering process is executed so that the image is presented, and the content image signal including the content image including the work instruction related to the second step is output from the input / output I / F 210 to the HMD 100. A content image based on the content image signal including the work instructions regarding the two steps is optically emitted toward the half mirror 116. After executing S210, the CPU 202 ends the state determination process and proceeds to S112 (see FIG. 5).

Returning to FIG. 5, in S112, the CPU 202 determines whether or not the reproduction of the content data 2082 has ended, in other words, whether or not the content data has been processed to the end. If the result of determination is that playback has not ended (S112: No), the process proceeds to S108. On the other hand, when the reproduction is completed (S112: Yes), the CPU 202 executes a content image signal output process termination process (S114), and terminates the main process. Note that the reproduction is also terminated based on a reproduction termination (stop) instruction input by the user via the operation unit 212.

In the above description, the eyebrow sensor 214, cheek sensor 216, and mouth sensor 218 have been described based on image sensors. However, as described above, each of the

sensors

214, 216, and 218 can be a photoelectric sensor. In this case, as shown in FIG. 7, the intensity of the reflected light is registered in the table 2084 in association with each operation of the HMD 100. The CPU 202 determines the state of the user's eyebrows, cheeks, and mouth based on the intensity registered in the table 2084 and the intensity of the reflected light input from the

sensors

214, 216, and 218 to the HMD 100. In FIG. 7, “hatched rectangle” indicates the eyebrows of the user, and “◯” indicates the “light spot” from the light emitting element.

For example, the intensity (reference value) of the reflected light when the user's face is in the standard state is “brow: 80 ± 15 (65 to 95)” (see FIG. 7), “cheek: 120 ± 15 ( 105 to 135) ”and“ mouth: 100 ± 15 (85 to 115) ”are registered in the table 2084. From the eyebrow sensor 214, the cheek sensor 216, and the mouth sensor 218, detection signals including reflected light intensities of “brow: 82”, “cheek: 118”, and “mouth: 100” are respectively sent to the control box 200. If it is input, the CPU 202 determines that a face motion trigger has not been detected in S108 (see FIG. 5) (S108: No).

Further, in the table 2084, for example, “110 ± 14 (96 ≦ intensity ≦ 124)” deviating from the reference value “80 ± 15 (65 to 95)” of the amount of change as the amount of change in the state where the eyebrows are raised. Are registered (see FIG. 7). When the detection signal including that the intensity of the reflected light is “brow: 115” is input from the eyebrow sensor 214, the CPU 202 affirms the determination of S108 (see FIG. 5) and S200 (see FIG. 6) (S108). , S200: Yes), the operation “next” is specified in S202.

Further, as the amount of change in the state of “drawing the eyebrows” in the table 2084, for example, “50 ± 14 (36 ≦ intensity ≦ 64)” deviating from the reference value is registered (see FIG. 7). . When the detection signal including that the intensity of the reflected light is “brow: 45” is input from the eyebrow sensor 214, the CPU 202 affirms the determination of S108 (see FIG. 5) and S200 (see FIG. 6) (S108). , S200: Yes), the operation “return to the previous” is specified in S202.

In the above description, the state of a specific part of the user's face alone included in the detection signal input to the control box 200 from each of the eyebrow sensor 214, the cheek sensor 216, and the mouth sensor 218, and the operation of the HMD 100. The description is based on the associated configuration. In addition to such a configuration, the following configuration may be adopted. For example, the operation of the HMD 100, for example, “fast forward” is set to a combination of a state where the eyebrows are raised and a state where the cheeks are raised, in other words, a state where the eyebrows are raised while raising the eyebrows. It can also be set as the structure matched. In this case, the CPU 202 determines that a face motion trigger has been input based on detection signals input from the eyebrow sensor 214 and cheek sensor 216 (see S108 in FIG. 5: Yes), and executes state determination processing (FIG. 5). (See FIG. 6 for details). In the state determination process, the CPU 202 makes the determination in S200 of FIG. 6 and, after affirming the determination (see S200: Yes in FIG. 6), again performs the same determination as in S204 of FIG. If both the determinations in S200 and S204 are affirmative (see S200 and S204 in FIG. 6: Yes), the CPU 202 determines that the state is “raised the eyebrows and raised the cheek”, and the CPU 202 performs the operation “fast forward”. Identify.

Furthermore, in the above description, for example, the eyebrow sensor 214 is configured to detect the state (movement) of one eyebrow, specifically, the left eyebrow. In addition to such a configuration, the configuration of detecting the state of the right and left eyebrows, a combination of the state of the left and right eyebrows, specifically, the HMD 100 in a state where the left eyebrow is raised and the right eyebrow is lowered. For example, a configuration in which “reduction” is associated can be adopted. In this case, the CPU 202 determines that a face motion trigger has been input based on the detection signals input from the right and left eyebrow sensors 214 (see S108 in FIG. 5: Yes), and executes state determination processing (see FIG. 5). S110 (Refer to FIG. 6 for details). In the state determination process, the CPU 202 performs the determination in S200 of FIG. If the CPU 202 affirms the determination regarding the state of the right and left eyebrows (see S200 in FIG. 6: Yes), the CPU 202 determines that the left eyebrow is raised and the right eyebrow is lowered. Specify “reduction”.

Further, in the above description, among the portions constituting the face of the user wearing the HMD 100, specific portions that are brought into a specific state by the user's conscious movement, particularly eyebrows, cheeks and mouths It was. In addition to this, for example, it may be configured to detect the state of the forehead or the jaw.

Furthermore, as operations realized by the HMD 100 and the control box 200, “next”, “return to previous”, “decision”, “cancel”, “brightness adjustment”, “brightness up”, and “brightness down” However, other operations can be registered in the table 2084. For example, an operation related to “selection / confirmation of content data 2082 to be played back” or an operation related to “start or end (stop) of playback” of content data 2082 in a state where a plurality of content data 2082 is stored in the storage unit 208 (In the above description, an example of inputting via the operation unit 212 will be described. Refer to S104 in FIG. 5 for the start of reproduction) may be registered in the table 2084 in association with the state of the eyebrows, cheeks or mouth. Further, an operation related to “power supply or power supply cutoff” to the HMD 100 or the control box 200 may be registered in association with a predetermined state such as an eyebrow.

(Advantageous effects based on the configuration of the embodiment)
According to the configuration of the HMD 100 and the control box 200 in the above embodiment, the state of the eyebrows, cheeks and mouth does not change unless the user wearing the HMD 100 consciously moves, and the HMD 100 and the control box 200 The movements to be realized are registered in association with each other, the state of each of these parts is sensed, and when the state of the eyebrows, cheeks and mouth is the registered state, the movement associated with the state is executed. The configuration is adopted.

According to this, a hands-free operation can be realized while preventing an operation instruction related to an operation not intended by the user from being input to the control box 200. Specifically, for example, in a state where a user is viewing a content image including a work instruction related to a first process for assembling a predetermined product, the user can use the HMD 100 or his / her hands while working. Without operating the control box 200, it is possible to visually recognize and recognize the work instruction related to the second step, for example, by the act of raising the eyebrows.

Claims

A head-mounted display that presents a content image indicated by content data to a user's eye so that the user can recognize the content image, and allows the user to recognize the content image,
Control means for controlling the operation of the head mounted display;
Detecting means for detecting that a state of a specific part of the user's face that does not move unless the user is conscious, is a predetermined state;
The control means controls the action associated with the predetermined state on condition that the detection means detects that the state of a specific part of the user's face is the predetermined state. Head-mounted display featuring
The detecting means detects that the state of the user's eyebrows is the predetermined state as a state of a specific part of the user's face,
The control means controls the operation associated with the predetermined state on condition that the detection means detects that the state of the eyebrows of the user is the predetermined state. The head mounted display according to claim 1.
The detection means detects that the state of the user's cheek as the state of a specific part of the user's face is the predetermined state,
The control unit controls the operation associated with the predetermined state on condition that the detection unit detects that the state of the cheek of the user is the predetermined state. The head mounted display according to claim 1.
The detecting means detects that the state of the mouth of the user is the predetermined state as a state of a specific part of the user's face,
The control unit controls the operation associated with the predetermined state on condition that the detection unit detects that the state of the mouth of the user is the predetermined state. The head mounted display according to claim 1.
The detection means includes
Imaging means capable of imaging a specific image of a specific part of the user's face;
Analyzing means for analyzing the specific image picked up by the image pickup means,
5. The method according to claim 1, wherein the state of the specific part of the user's face is detected as the predetermined state according to an analysis result of the analysis unit. The described head mounted display.
The detection means includes
A light emitting element capable of irradiating a specific part of the user's face;
A light detecting element for detecting reflected light reflected by the light irradiated on a specific part of the user's face,
When the intensity of the reflected light detected by the light detection element deviates from a predetermined reference value, it is detected that the state of a specific part of the user's face is the predetermined state. The head mounted display of any one of Claim 1 thru | or 4.
The control means controls the first operation and the second operation as the operation of the head mounted display,
When the state of the specific part of the user's face is the first state, the detecting means detects the first state as the predetermined state, and the state of the specific part of the user's face is the first state. In the case of two states, the second state is detected as the predetermined state,
The control unit controls the first operation associated with the first state on the condition that the detection unit detects that the first state is in the first state, and the detection unit is in the second state. The head mounted display according to any one of claims 1 to 4, wherein the second operation associated with the second state is controlled on the condition that the detection is detected.