JP2017146715A - Display device, control method of display device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display device capable of easily changing display in response to an operation and suppressing changes in display which are not intended by a user, a control method of the display device, and a program.SOLUTION: An HMD 100 includes an image display part 20 for visualizing an image for a user, and a control device 10 operable independent from the image display part 20 and having a 6-axis sensor 111 for detecting movement of the control device 10. The HMD 100 executes, as a process on an image displayed by the image display part 20, a first display change in response to the movement in a first movement direction detected by the 6-axis sensor 111 and a second display change in response to the movement in a second movement direction. The HMD 100 starts the first display change when the movement in the first movement direction is detected, and then stops the first display change when the movement in the second direction is detected by the 6-axis sensor 111.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a display device, a display device control method, and a program.

  2. Description of the Related Art Conventionally, an example is known in which a display device is operated by a device that a user (user) holds and uses (see, for example, Patent Document 1). The remote control device described in Patent Document 1 includes a ring-shaped housing or body and is operated by a user by gripping, and the movement of the remote control device is reflected in the display.

Special table 2009-500923

In the configuration described in Patent Document 1, the operation of the remote control device is reflected in the display. The configuration for controlling the display in response to the movement in this way has an advantage of excellent operability because the operation is simple, but the display changes in accordance with the movement regardless of the user's intention. For this reason, there is a possibility that a display change unintended by the user may occur.
The present invention has been made in view of the above circumstances. A display device, a display device control method, and a program capable of easily changing a display in response to an operation and suppressing a display change unintended by a user. The purpose is to provide.

In order to achieve the above object, the present invention is a head-mounted display device having a display unit that allows a user to visually recognize an image, and is an operation device that can be moved independently of the display unit. An operation device having a motion detection unit that detects the movement of the operation device, and a control unit that controls display of the display unit according to the motion detected by the motion detection unit, As processing for the image displayed by the display unit, the first display change corresponding to the movement in the first movement direction detected by the movement detection unit and the movement in the second movement direction detected by the movement detection unit. The corresponding second display change is executed, and when the movement in the first movement direction is detected, the first display change is started, and then the movement in the second movement direction is performed by the movement detection unit. Is detected, Stop showing changes in 1, characterized in that.
According to the present invention, display on the display unit can be easily controlled by an operation of moving the operation device. Further, by combining the movements of the operation device in two directions, the image displayed on the display unit can be moved to a desired position.

In the display device according to the aspect of the invention, the control unit may execute the first display change when the motion detection unit detects a motion in the first motion direction, and the motion detection unit When the movement in the second movement direction is detected, the second display change is executed.
According to the present invention, the image displayed on the display unit can be moved by moving the operation device, and the direction of movement of the image can be indicated by the movement direction of the operation device.

Further, according to the present invention, in the display device, the operation device includes an operation reception unit that receives an operation, and the control unit detects the motion detection unit in a state where the operation reception unit receives an operation. The display of the display unit is changed according to movement.
According to the present invention, it is possible to prevent a display change unintended by the user depending on the operation state of the operation reception unit.

Further, the present invention provides the display device, wherein the control unit detects the movement in the first movement direction by the movement detection unit in a state where the operation reception unit receives an operation. 1 display change is executed.
According to the present invention, it is possible to prevent a display change unintended by the user depending on the operation state of the operation reception unit.

In the display device according to the aspect of the invention, the control unit starts the first display change when the motion detection unit detects a motion in the first motion direction, and then performs the operation. The first display change is stopped when the accepting unit stops accepting the operation.
According to the present invention, it is possible to prevent a display change unintended by the user depending on the operation state of the operation reception unit.

In the display device according to the aspect of the invention, the control unit may detect the movement in the second movement direction when the movement detection unit detects a movement in the second movement direction while the operation reception unit is receiving an operation. The display change of 2 is performed.
According to the present invention, the image displayed on the display unit can be moved by moving the operation device, and the direction of movement of the image can be indicated by the movement direction of the operation device.

Moreover, the present invention is characterized in that, in the display device, the operation receiving unit receives an operation different from the detection of the motion detecting unit.
According to the present invention, it is possible to select a state in which the display is controlled by the movement of the operation device and a state in which the display is not controlled by an operation different from the motion detected by the motion detection unit. Can be more reliably prevented.

In the display device according to the present invention, the operation device is a wearable device attached to the body of the user.
ADVANTAGE OF THE INVENTION According to this invention, the display of a display part can be easily controlled by operation which a user moves a body. In addition, since the display state unintended by the user can be prevented depending on the operation state of the operation reception unit, an operation for controlling the display can be performed more intuitively.

In order to achieve the above object, the present invention provides a display unit that allows a user to visually recognize an image, and an operation device that can be moved independently of the display unit, and detects the movement of the operation device. As a process for an image displayed on the display unit by controlling a display device including an operation device having a motion detection unit, a first display corresponding to the motion in the first motion direction detected by the motion detection unit A second display change corresponding to a change and a movement in the second movement direction detected by the movement detector, and the first display change when the movement in the first movement direction is detected. The first display change is stopped when a motion in the second motion direction is detected by the motion detection unit.
According to the present invention, display on the display unit can be easily controlled by an operation of moving the operation device. Further, by combining the movements of the operation device in two directions, the image displayed on the display unit can be moved to a desired position.

In order to achieve the above object, the present invention provides a display unit that allows a user to visually recognize an image, and an operation device that can be moved independently of the display unit, and detects the movement of the operation device. A program that can be executed by a computer that controls a display device including an operation device having a motion detection unit, and as a process for an image displayed by the display unit, the first motion direction detected by the motion detection unit The first display change corresponding to the movement and the second display change corresponding to the movement in the second movement direction detected by the movement detector are executed, and the movement in the first movement direction is detected. In this case, the first display change is started, and when the movement in the second movement direction is detected by the movement detection unit, the first display change is stopped.
According to the present invention, display on the display unit can be easily controlled by an operation of moving the operation device. Further, by combining the movements of the operation device in two directions, the image displayed on the display unit can be moved to a desired position.

  The present invention can also be realized in various forms other than the above-described display device, display device control method, and program. For example, the present invention can be realized in the form of a recording medium that records the program, a server device that distributes the program, a transmission medium that transmits the program, a data signal that embodies the program in a carrier wave, and the like.

Explanatory drawing which shows the external appearance structure of HMD. The figure which shows the structure of the optical system of an image display part. Explanatory drawing which shows a response | compatibility with an image display part and an imaging range. The block diagram of each part which comprises HMD. The block diagram of a control part and a memory | storage part. Explanatory drawing which shows the mode of lighting of a LED display part. Explanatory drawing which shows the example of operation using a control apparatus. Explanatory drawing which shows the change of the display corresponding to operation of HMD. Explanatory drawing which shows the change of the display corresponding to operation of HMD. Explanatory drawing which shows the change of the display corresponding to operation of HMD. The flowchart which shows operation | movement of HMD. The flowchart which shows operation | movement of HMD. The external view of a clock-type device. External view of a ring-type device.

FIG. 1 is an explanatory diagram showing an external configuration of an HMD (Head Mounted Display) 100 according to an embodiment to which the present invention is applied.
HMD100 is a display apparatus provided with the image display part 20 (display part) which makes a user visually recognize a virtual image in the state with which the user (user) was mounted | worn, and the control apparatus 10 which controls the image display part 20. It is. As shown in FIG. 1, the control device 10 (operation device) includes a flat box-shaped case 10 </ b> A (which can also be referred to as a housing or a main body), and the case 10 </ b> A includes each unit described later. On the surface of the case 10 </ b> A, various buttons 11, switches, track pads 14, and the like that receive user operations are provided. When the user operates these, the control device 10 functions as a controller of the HMD 100.

The image display unit 20 is a wearing body that is worn on the user's head, and has a glasses shape in the present embodiment. The image display unit 20 includes a right display unit 22, a left display unit 24, a right light guide plate 26, and a left light guide plate 28 in a main body having a right holding unit 21, a left holding unit 23, and a front frame 27. .
Each of the right holding unit 21 and the left holding unit 23 extends rearward from both ends of the front frame 27, and holds the image display unit 20 on the user's head like a temple of glasses. Here, of both ends of the front frame 27, an end located on the right side of the user in the mounted state of the image display unit 20 is defined as an end ER, and an end located on the left side of the user is defined as an end EL. To do. The right holding unit 21 extends from the end ER of the front frame 27 to a position corresponding to the right side of the user when the image display unit 20 is mounted. The left holding unit 23 is provided to extend from the end EL to a position corresponding to the left side of the user when the image display unit 20 is mounted.

  The right light guide plate 26 and the left light guide plate 28 are provided on the front frame 27. The right light guide plate 26 is positioned in front of the right eye of the user when the image display unit 20 is mounted, and causes the right eye to visually recognize the image. The left light guide plate 28 is positioned in front of the left eye of the user when the image display unit 20 is mounted, and causes the left eye to visually recognize the image.

  The front frame 27 has a shape in which one end of the right light guide plate 26 and one end of the left light guide plate 28 are connected to each other, and this connection position is in a wearing state in which the user wears the image display unit 20. Corresponding to the eyebrows. The front frame 27 may be provided with a nose pad portion that comes into contact with the user's nose when the image display unit 20 is mounted at a connection position between the right light guide plate 26 and the left light guide plate 28. In this case, the image display unit 20 can be held on the user's head by the nose pad, the right holding unit 21 and the left holding unit 23. Further, a belt (not shown) that contacts the back of the user when the image display unit 20 is mounted may be connected to the right holding unit 21 and the left holding unit 23. In this case, the image display unit 20 is used by the belt. Can be held on the person's head.

  The right display unit 22 is a unit related to display of an image by the right light guide plate 26 and is provided in the right holding unit 21 and is located in the vicinity of the right side of the user in the mounted state. The left display unit 24 is a unit related to display of an image by the left light guide plate 28 and is provided in the left holding unit 23 and is located in the vicinity of the left head of the user in the mounted state. The right display unit 22 and the left display unit 24 are also collectively referred to simply as a “display driving unit”.

The right light guide plate 26 and the left light guide plate 28 of the present embodiment are optical units formed of a light-transmitting resin or the like, and are, for example, prisms, and output image light output from the right display unit 22 and the left display unit 24. Lead to the eyes of the user.
A light control plate (not shown) may be provided on the surfaces of the right light guide plate 26 and the left light guide plate 28. The light control plate is an optical element on a thin plate having different transmittance depending on the wavelength region of light, and functions as a so-called wavelength filter. The light control board is arrange | positioned so that the front side of the front frame 27 which is the opposite side to the user's eyes side may be covered, for example. By appropriately selecting the optical characteristics of the light control plate, the transmittance of light in an arbitrary wavelength region such as visible light, infrared light, and ultraviolet light can be adjusted. The amount of external light incident on the light plate 28 and transmitted through the right light guide plate 26 and the left light guide plate 28 can be adjusted.

  The image display unit 20 guides the image light generated by the right display unit 22 and the left display unit 24 to the right light guide plate 26 and the left light guide plate 28, respectively, and makes the user visually recognize the virtual image by using the image light, thereby Is displayed. When the external light enters the user's eyes through the right light guide plate 26 and the left light guide plate 28 from the front of the user, the image light and the external light constituting the virtual image enter the user's eyes. In other words, the visibility of the virtual image is affected by the intensity of external light. For this reason, for example, by attaching a light control plate to the front frame 27 and appropriately selecting or adjusting the optical characteristics of the light control plate, the visibility of the virtual image can be adjusted. In a typical example, it is possible to use a light control plate having a light transmittance that allows a user wearing the HMD 100 to visually recognize at least the outside scenery. In addition, when the dimming plate is used, it is possible to protect the right light guide plate 26 and the left light guide plate 28 and to suppress the damage to the right light guide plate 26 and the left light guide plate 28, the adhesion of dirt, and the like. The light control plate may be detachable with respect to the front frame 27 or each of the right light guide plate 26 and the left light guide plate 28, or may be mounted by replacing a plurality of types of light control plates. May be omitted.

  The camera 61 is disposed on the front frame 27 of the image display unit 20. The camera 61 desirably captures an outside scene direction viewed by the user wearing the image display unit 20, and external light transmitted through the right light guide plate 26 and the left light guide plate 28 on the front surface of the front frame 27. It is provided at a position that does not block. In the example of FIG. 1, the camera 61 is disposed on the end ER side of the front frame 27. The camera 61 may be disposed on the end EL side, or may be disposed at a connection portion between the right light guide plate 26 and the left light guide plate 28.

The camera 61 is a digital camera that includes an imaging element such as a CCD or CMOS, an imaging lens, and the like. The camera 61 of the present embodiment is a monocular camera, but may be a stereo camera. The camera 61 images at least a part of the outside scene (real space) in the front side direction of the HMD 100, in other words, the user's viewing direction when the HMD 100 is worn. In another expression, the camera 61 captures a range or direction that overlaps the user's field of view, and captures the direction in which the user gazes. Although the angle of view of the camera 61 can be set as appropriate, in the present embodiment, as will be described later, the camera 61 includes the outside world visually recognized through the right light guide plate 26 and the left light guide plate 28. More preferably, the imaging range of the camera 61 is set so that the entire field of view of the user visible through the right light guide plate 26 and the left light guide plate 28 can be imaged.
The camera 61 performs imaging in accordance with the control of the imaging control unit 149 included in the control unit 150 (FIG. 5), and outputs captured image data to the imaging control unit 149.

  The HMD 100 may include a distance sensor (not shown) that detects a distance to a measurement object positioned in a preset measurement direction. The distance sensor can be disposed, for example, at a connection portion between the right light guide plate 26 and the left light guide plate 28 in the front frame 27. In this case, when the image display unit 20 is mounted, the position of the distance sensor is substantially in the middle between the user's eyes in the horizontal direction and above the user's eyes in the vertical direction. The measurement direction of the distance sensor can be, for example, the front side direction of the front frame 27, in other words, the direction overlapping the imaging direction of the camera 61. The distance sensor can be configured to include, for example, a light source such as an LED or a laser diode, and a light receiving unit that receives reflected light that is reflected from the light to be measured. The distance sensor may perform a triangulation process or a distance measurement process based on a time difference in accordance with the control of the control unit 150. The distance sensor may be configured to include a sound source that emits ultrasonic waves and a detection unit that receives ultrasonic waves reflected by the measurement object. In this case, the distance sensor may perform the distance measurement process based on the time difference until the reflection of the ultrasonic wave according to the control of the control unit 150.

FIG. 2 is a principal plan view showing the configuration of the optical system provided in the image display unit 20. FIG. 2 shows the user's left eye LE and right eye RE for explanation.
As shown in FIG. 2, the right display unit 22 and the left display unit 24 are configured symmetrically. As a configuration for allowing the user's right eye RE to visually recognize an image, the right display unit 22 includes an OLED (Organic Light Emitting Diode) unit 221 that emits image light, a lens group that guides image light L emitted from the OLED unit 221, and the like. Right optical system 251. The image light L is guided to the right light guide plate 26 by the right optical system 251.

The OLED unit 221 includes an OLED panel 223 and an OLED drive circuit 225 that drives the OLED panel 223. The OLED panel 223 is a self-luminous display configured by arranging light emitting elements that emit light of R (red), G (green), and B (blue) light by organic electroluminescence in a matrix. It is a panel. The OLED panel 223 includes a plurality of pixels, each of which includes one R, G, and B element, and forms an image with pixels arranged in a matrix. The OLED drive circuit 225 selects a light emitting element included in the OLED panel 223 and energizes the light emitting element under the control of the control unit 150 (FIG. 5), and causes the light emitting element of the OLED panel 223 to emit light. The OLED drive circuit 225 is fixed to the back surface of the OLED panel 223, that is, the back surface of the light emitting surface by bonding or the like. The OLED drive circuit 225 may be formed of a semiconductor device that drives the OLED panel 223, for example, and may be mounted on a substrate (not shown) fixed to the back surface of the OLED panel 223. A temperature sensor 217 is mounted on this substrate.
Note that the OLED panel 223 may have a configuration in which light emitting elements that emit white light are arranged in a matrix, and color filters corresponding to R, G, and B colors are stacked. Further, an OLED panel 223 having a WRGB configuration provided with a light emitting element that emits W (white) light in addition to the light emitting elements that respectively emit R, G, and B color light may be used.

  The right optical system 251 includes a collimator lens that converts the image light L emitted from the OLED panel 223 into a parallel light flux. The image light L converted into a parallel light beam by the collimator lens enters the right light guide plate 26. A plurality of reflecting surfaces that reflect the image light L are formed in an optical path that guides light inside the right light guide plate 26. The image light L is guided to the right eye RE side through a plurality of reflections inside the right light guide plate 26. A half mirror 261 (reflection surface) located in front of the right eye RE is formed on the right light guide plate 26. The image light L is reflected by the half mirror 261 and emitted from the right light guide plate 26 toward the right eye RE. The image light L forms an image on the retina of the right eye RE, and allows the user to visually recognize the image.

  In addition, as a configuration for allowing the user's left eye LE to visually recognize an image, the left display unit 24 includes a left optical system including an OLED unit 241 that emits image light, a lens group that guides the image light L emitted from the OLED unit 241, and the like. 252. The image light L is guided to the left light guide plate 28 by the left optical system 252.

  The OLED unit 241 includes an OLED panel 243 and an OLED drive circuit 245 that drives the OLED panel 243. The OLED panel 243 is a self-luminous display panel configured similarly to the OLED panel 223. The OLED driving circuit 245 causes the light emitting elements of the OLED panel 243 to emit light by selecting a light emitting element included in the OLED panel 243 and energizing the light emitting element according to the control of the control unit 150 (FIG. 5). The OLED drive circuit 245 is fixed to the back surface of the OLED panel 243, that is, the back surface of the light emitting surface by bonding or the like. The OLED drive circuit 245 may be configured by a semiconductor device that drives the OLED panel 243, for example, and may be mounted on a substrate (not shown) fixed to the back surface of the OLED panel 243. A temperature sensor 239 is mounted on this substrate.

  The left optical system 252 includes a collimating lens that converts the image light L emitted from the OLED panel 243 into a light beam in a parallel state. The image light L converted into a parallel light beam by the collimator lens enters the left light guide plate 28. The left light guide plate 28 is an optical element formed with a plurality of reflecting surfaces that reflect the image light L, and is, for example, a prism. The image light L is guided to the left eye LE side through a plurality of reflections inside the left light guide plate 28. The left light guide plate 28 is formed with a half mirror 281 (reflection surface) located in front of the left eye LE. The image light L is reflected by the half mirror 281 and emitted from the left light guide plate 28 toward the left eye LE. The image light L forms an image on the retina of the left eye LE, and allows the user to visually recognize the image.

According to this configuration, the HMD 100 functions as a see-through display device. That is, the image light L reflected by the half mirror 261 and the external light OL transmitted through the right light guide plate 26 are incident on the user's right eye RE. Further, the image light L reflected by the half mirror 281 and the external light OL transmitted through the half mirror 281 are incident on the left eye LE. In this way, the HMD 100 superimposes the image light L of the internally processed image and the external light OL so as to enter the user's eyes, and for the user, the outside light can be seen through the right light guide plate 26 and the left light guide plate 28. The image by the image light L is visually recognized over the outside scene.
The half mirrors 261 and 281 are image extraction units that extract image by reflecting image light output from the right display unit 22 and the left display unit 24, respectively, and can be called display units.

  The left optical system 252 and the left light guide plate 28 are collectively referred to as a “left light guide”, and the right optical system 251 and the right light guide plate 26 are collectively referred to as a “right light guide”. The configuration of the right light guide and the left light guide is not limited to the above example, and any method can be used as long as a virtual image is formed in front of the user's eyes using image light. It may be used, or a semi-transmissive reflective film may be used.

  Returning to FIG. 1, the control device 10 and the image display unit 20 are connected by a connection cable 40. The connection cable 40 is detachably connected to a connector (not shown) provided at the lower part of the case 10 </ b> A, and is connected to various circuits provided in the image display unit 20 from the tip of the left holding unit 23. The connection cable 40 includes a metal cable or optical fiber cable that transmits digital data, and may include a metal cable that transmits an analog signal. A connector 46 is provided in the middle of the connection cable 40. The connector 46 is a jack for connecting a stereo mini-plug, and the connector 46 and the control device 10 are connected by a line for transmitting an analog audio signal, for example. In the configuration example shown in FIG. 1, a headset 30 having a right earphone 32 and a left earphone 34 and a microphone 63 constituting a stereo headphone is connected to a connector 46.

  For example, as shown in FIG. 1, the microphone 63 is arranged so that the sound collection unit of the microphone 63 faces the user's line of sight, collects sound, and outputs a sound signal to the sound interface 182 (FIG. 4). To do. For example, the microphone 63 may be a monaural microphone or a stereo microphone, may be a directional microphone, or may be an omnidirectional microphone.

The control device 10 includes a button 11, an LED indicator 12, a track pad 14, an up / down key 15, a changeover switch 16, and a power switch 18 as operated parts operated by a user. These operated parts are arranged on the surface of the case 10A.
The button 11 includes a menu key, a home key, a return key, and the like for operating the operating system 143 (FIG. 5) executed by the control device 10, and in particular, is displaced by a pressing operation of these keys and switches. Including things. The LED indicator 12 lights up or flashes in accordance with the operating state of the HMD 100. The up / down key 15 is used to input an instruction to increase / decrease the volume output from the right earphone 32 and the left earphone 34 and to input an instruction to increase / decrease the display brightness of the image display unit 20. The changeover switch 16 is a switch for changing an input corresponding to the operation of the up / down key 15. The power switch 18 is a switch for switching on / off the power of the HMD 100, and is configured by a slide switch, for example.

The track pad 14 (operation reception unit, position instruction operation unit) has an operation surface for detecting a contact operation, and outputs an operation signal according to an operation on the operation surface. The detection method on the operation surface is not limited, and an electrostatic method, a pressure detection method, an optical method, or the like can be adopted. Contact (touch operation) to the track pad 14 is detected by a touch sensor 13 (FIG. 4) described later.
Further, as indicated by a broken line in FIG. 1, an LED display unit 17 is installed on the track pad 14. The LED display unit 17 includes a plurality of LEDs, and the operation unit can be visually recognized on the LED display unit 17 by lighting each LED. In the example of FIG. 1, three symbols Δ (triangle), ○ (circle), and □ (square) appear when the LED of the LED display unit 17 is turned on. When the LED display unit 17 is turned off, these symbols are not visible. This configuration can be realized, for example, by configuring the track pad 14 as a colored or colorless and transparent flat plate having translucency, and disposing an LED directly under the flat plate.

  The LED display unit 17 functions as a software button. For example, when the LED display unit 17 is lit, the lighting position (display position) of the symbol functions as a button for giving an instruction corresponding to the symbol. In the example of FIG. 1, the symbol ○ (circle) functions as a home button, and when an operation of touching the position of the symbol ○ (circle) is performed, the control unit 150 described later is based on the detection value of the touch sensor 13. Detects home button operation. The symbol □ (square) functions as a history button. When an operation of touching the position of the symbol □ (rectangle) is performed, the control unit 150 described later detects the operation of the history button based on the detection value of the touch sensor 13. The symbol Δ (triangle) functions as a return button. When an operation of touching the position of the symbol Δ (triangle) is performed, the control unit 150 described later detects the operation of the return button based on the detection value of the touch sensor 13.

  3A and 3B are diagrams illustrating a configuration of a main part of the image display unit 20, where FIG. 3A is a main part perspective view of the image display unit 20 viewed from the user's head side, and FIG. 3B is an angle of view of the camera 61. It is explanatory drawing of. Note that the connection cable 40 is not shown in FIG.

FIG. 3A shows the side of the image display unit 20 in contact with the user's head, in other words, the side visible to the user's right eye RE and left eye LE. In other words, the back sides of the right light guide plate 26 and the left light guide plate 28 are visible.
In FIG. 3A, the half mirror 261 that irradiates the user's right eye RE with the image light and the half mirror 281 that irradiates the left eye LE with the image light appear as substantially rectangular areas. Further, the entire right light guide plate 26 and left light guide plate 28 including the half mirrors 261 and 281 transmit external light as described above. For this reason, the user sees the outside scene through the entire right light guide plate 26 and the left light guide plate 28, and sees a rectangular display image at the positions of the half mirrors 261 and 281.

  The camera 61 is arranged at the right end of the image display unit 20 as described above, and images the direction in which both eyes of the user face, that is, the front for the user. FIG. 3B is a diagram schematically showing the position of the camera 61 in plan view together with the user's right eye RE and left eye LE. An angle of view (imaging range) of the camera 61 is indicated by C. FIG. 3B shows a horizontal angle of view C, but the actual angle of view of the camera 61 extends in the vertical direction as in a general digital camera.

  The optical axis of the camera 61 is a direction including the line-of-sight directions of the right eye RE and the left eye LE. The outside scene that the user can visually recognize in a state where the HMD 100 is worn is not always at infinity. For example, as shown in FIG. 3B, when the user gazes at the object OB with both eyes, the user's line of sight is directed toward the object OB as indicated by reference signs RD and LD in the figure. In this case, the distance from the user to the object OB is often about 30 cm to 10 m, and more often about 1 m to 4 m. Therefore, for the HMD 100, an upper limit and a lower limit guideline for the distance from the user to the object OB during normal use may be set. This standard may be obtained by investigation or experiment, or may be set by the user. The optical axis and the angle of view of the camera 61 are determined when the distance to the object OB during normal use corresponds to the set upper limit guide and when the target OB corresponds to the lower limit guide. It is preferably set so as to be included in the angle of view.

  In general, the viewing angle of a human is about 200 degrees in the horizontal direction and about 125 degrees in the vertical direction, and of these, the effective field of view with excellent information receiving capability is about 30 degrees in the horizontal direction and about 20 degrees in the vertical direction. Furthermore, the stable gaze field in which the gaze point that the human gazes at appears to be quickly and stably is about 60 to 90 degrees in the horizontal direction and about 45 to 70 degrees in the vertical direction. In this case, when the gazing point is the object OB in FIG. 3B, the effective visual field is about 30 degrees in the horizontal direction and about 20 degrees in the vertical direction around the lines of sight RD and LD. Further, the stable viewing field is about 60 to 90 degrees in the horizontal direction and about 45 to 70 degrees in the vertical direction, and the viewing angle is about 200 degrees in the horizontal direction and about 125 degrees in the vertical direction. Furthermore, the actual field of view that the user perceives through the image display unit 20 and through the right light guide plate 26 and the left light guide plate 28 can be referred to as a real field of view (FOV). In the configuration of the present embodiment shown in FIGS. 1 and 2, the actual visual field corresponds to an actual visual field that is transmitted through the right light guide plate 26 and the left light guide plate 28 and visually recognized by the user. The real field of view is narrower than the viewing angle and stable field of view, but wider than the effective field of view.

  The angle of view C of the camera 61 is preferably capable of capturing a wider range than the user's field of view, and specifically, the angle of view C is preferably at least wider than the effective field of view of the user. Further, it is more preferable that the angle of view C is wider than the actual field of view of the user. More preferably, the angle of view C is wider than the stable viewing field of the user, and most preferably, the angle of view C is wider than the viewing angle of both eyes of the user.

  The camera 61 may include a so-called wide-angle lens as an imaging lens so that a wide angle of view can be captured. The wide-angle lens may include a lens called a super-wide-angle lens or a quasi-wide-angle lens, may be a single focus lens or a zoom lens, and the camera 61 includes a lens group including a plurality of lenses. There may be.

FIG. 4 is a block diagram illustrating a configuration of each unit included in the HMD 100.
The control device 10 includes a main processor 140 that executes a program and controls the HMD 100. A memory 118 and a non-volatile storage unit 121 are connected to the main processor 140. The main processor 140 is connected to the track pad 14 and the operation unit 110 as input devices. In addition, a six-axis sensor 111, a magnetic sensor 113, and a GPS 115 are connected to the main processor 140 as sensors. The main processor 140 is connected to a communication unit 117, an audio codec 180, an external connector 184, an external memory interface 186, a USB connector 188, a sensor hub 192, and an FPGA 194. These function as an interface with the outside.

  The main processor 140 is mounted on the controller board 120 built in the control device 10. In addition to the main processor 140, a memory 118, a nonvolatile storage unit 121, and the like may be mounted on the controller board 120. In the present embodiment, a six-axis sensor 111, a magnetic sensor 113, a GPS 115, a communication unit 117, a memory 118, a nonvolatile storage unit 121, an audio codec 180, and the like are mounted on the controller board 120. Further, the external connector 184, the external memory interface 186, the USB connector 188, the sensor hub 192, the FPGA 194, and the interface 196 may be mounted on the controller board 120.

  The memory 118 constitutes a work area that temporarily stores a program to be executed and data to be processed when the main processor 140 executes the program. The nonvolatile storage unit 121 is configured by a flash memory or an eMMC (embedded Multi Media Card). The nonvolatile storage unit 121 stores a program executed by the main processor 140 and various data processed by the main processor 140 executing the program.

The main processor 140 detects a touch operation on the operation surface of the track pad 14 based on an operation signal input from the track pad 14 and acquires an operation position.
The operation unit 110 includes a button 11, a touch sensor 13, and an LED display unit 17. The touch sensor 13 detects a touch operation on the track pad 14 and specifies an operation position of the detected touch operation. When the operation of the button 11 is performed and when the touch sensor 13 detects the touch operation, an operation signal is output from the operation unit 110 to the main processor 140.
The LED display unit 17 is disposed immediately below the track pad 14 (FIG. 1) and includes an LED (not shown) and a drive circuit that lights the LED. The LED display unit 17 turns on, blinks, and turns off the LED according to the control of the main processor 140.

The light emission pattern of the LED of the LED display part 17 is shown in FIG.
FIGS. 6A and 6B are timing charts showing changes in the luminance of the LEDs of the LED display unit 17. In the light emission pattern shown in FIG. 6A, the LED display unit 17 periodically turns on the LED, and the rise and fall of the brightness of the LED are steep. On the other hand, in the lighting pattern of FIG. 6B, the luminance of the LED changes gently. The LED display unit 17 causes the LED to emit light according to the pattern shown in FIG. 6A or 6B under the control of the main processor 140. Moreover, the LED display part 17 may light up LED always. In the pattern shown in FIG. 6A or 6B, the time T1 during which the LED emits light and the time T2 during which the LED is turned off are arbitrary. For example, the times T1 and T2 may be 100 milliseconds or less. In this case, even if the LED is repeatedly turned on and off, the user can visually recognize that the light is continuously turned on. Further, the LED display unit 17 may perform PWM control on the luminance of the LED by controlling the ratio between the times T1 and T2.

Returning to FIG. 4, the 6-axis sensor 111 is a motion sensor (inertial sensor) including a 3-axis acceleration sensor and a 3-axis gyro (angular velocity) sensor. The 6-axis sensor 111 may employ an IMU (Inertial Measurement Unit) in which the above sensors are modularized.
The magnetic sensor 113 is, for example, a triaxial geomagnetic sensor.
A GPS (Global Positioning System) 115 includes a GPS antenna (not shown), receives a radio signal transmitted from a GPS satellite, and detects the coordinates of the current position of the control device 10.
The six-axis sensor 111, the magnetic sensor 113, and the GPS 115 output detection values to the main processor 140 according to a sampling period specified in advance. Alternatively, the six-axis sensor 111, the magnetic sensor 113, and the GPS 115 output detection values to the main processor 140 at a timing designated by the main processor 140 in response to a request from the main processor 140.

The communication unit 117 performs wireless communication with an external device. The communication unit 117 includes an antenna, an RF circuit, a baseband circuit, a communication control circuit, and the like, or a device in which these are integrated. The communication unit 117 performs wireless communication conforming to standards such as Bluetooth (registered trademark) and wireless LAN (including Wi-Fi (registered trademark)).
The audio interface 182 is an interface for inputting and outputting audio signals. In the present embodiment, the audio interface 182 includes a connector 46 (FIG. 1) provided on the connection cable 40. The audio codec 180 is connected to the audio interface 182 and encodes / decodes an audio signal input / output via the audio interface 182. The audio codec 180 may include an A / D converter that converts analog audio signals to digital audio data, and a D / A converter that performs the reverse conversion. For example, the HMD 100 according to the present embodiment outputs sound by the right earphone 32 and the left earphone 34 and collects sound by the microphone 63. The audio codec 180 converts the digital audio data output from the main processor 140 into an analog audio signal and outputs the analog audio signal via the audio interface 182. The audio codec 180 converts an analog audio signal input to the audio interface 182 into digital audio data and outputs the digital audio data to the main processor 140.

The external connector 184 is a connector that connects an external device that communicates with the main processor 140. The external connector 184 is an interface for connecting an external device when, for example, an external device is connected to the main processor 140 to debug a program executed by the main processor 140 or collect a log of the operation of the HMD 100. It is.
The external memory interface 186 is an interface to which a portable memory device can be connected. For example, the external memory interface 186 includes a memory card slot in which a card type recording medium is mounted and data can be read and an interface circuit. In this case, the size, shape and standard of the card type recording medium are not limited and can be changed as appropriate.
A USB (Universal Serial Bus) connector 188 includes a connector conforming to the USB standard and an interface circuit, and can connect a USB memory device, a smartphone, a computer, and the like. The size and shape of the USB connector 188 and the compatible USB standard version can be appropriately selected and changed.

  Further, the HMD 100 includes a vibrator 19. The vibrator 19 includes a motor (not shown), an eccentric rotor (not shown), and the like, and generates vibrations under the control of the main processor 140. The HMD 100 generates vibration by the vibrator 19 with a predetermined vibration pattern, for example, when an operation on the operation unit 110 is detected, or when the power of the HMD 100 is turned on / off.

  The sensor hub 192 and the FPGA 194 are connected to the image display unit 20 via an interface (I / F) 196. The sensor hub 192 acquires detection values of various sensors included in the image display unit 20 and outputs them to the main processor 140. The FPGA 194 executes processing of data transmitted and received between the main processor 140 and each unit of the image display unit 20 and transmission via the interface 196.

  The right display unit 22 and the left display unit 24 of the image display unit 20 are each connected to the control device 10. As shown in FIG. 1, in the HMD 100, a connection cable 40 is connected to the left holding unit 23, wiring connected to the connection cable 40 is laid inside the image display unit 20, and each of the right display unit 22 and the left display unit 24 is Connected to the control device 10.

The right display unit 22 includes a display unit substrate 210. The display unit substrate 210 includes an interface (I / F) 211 connected to the interface 196, a receiving unit (Rx) 213 that receives data input from the control device 10 via the interface 211, and an EEPROM 215 (storage unit) ) Is implemented.
The interface 211 connects the receiving unit 213, the EEPROM 215, the temperature sensor 217, the camera 61, the illuminance sensor 65, and the LED indicator 67 to the control device 10.

  An EEPROM (Electrically Erasable Programmable Read-Only Memory) 215 stores various data so that the main processor 140 can read the data. The EEPROM 215 stores, for example, data relating to the light emission characteristics and display characteristics of the OLED units 221 and 241 provided in the image display unit 20, data relating to the characteristics of sensors provided in the right display unit 22 or the left display unit 24, and the like. Specifically, parameters relating to gamma correction of the OLED units 221, 241 and data for compensating the detection values of the temperature sensors 217, 239 are stored. These data are generated by the factory inspection of the HMD 100 and written in the EEPROM 215. After the shipment, the main processor 140 can perform processing using the data in the EEPROM 215.

The camera 61 executes imaging according to a signal input via the interface 211 and outputs captured image data or a signal indicating the imaging result to the control device 10.
As shown in FIG. 1, the illuminance sensor 65 is provided at the end ER of the front frame 27 and is arranged to receive external light from the front of the user wearing the image display unit 20. The illuminance sensor 65 outputs a detection value corresponding to the amount of received light (received light intensity).
As shown in FIG. 1, the LED indicator 67 is disposed near the camera 61 at the end ER of the front frame 27. The LED indicator 67 is lit during execution of imaging by the camera 61 to notify that imaging is in progress.

  The temperature sensor 217 detects the temperature and outputs a voltage value or a resistance value corresponding to the detected temperature as a detected value. The temperature sensor 217 is mounted on the back side of the OLED panel 223 (FIG. 3). For example, the temperature sensor 217 may be mounted on the same substrate as the OLED drive circuit 225. With this configuration, the temperature sensor 217 mainly detects the temperature of the OLED panel 223.

  The receiving unit 213 receives data transmitted from the main processor 140 via the interface 211. When receiving the image data of the image displayed on the OLED unit 221, the receiving unit 213 outputs the received image data to the OLED drive circuit 225 (FIG. 2).

The left display unit 24 includes a display unit substrate 210. On the display unit substrate 210, an interface (I / F) 231 connected to the interface 196 and a receiving unit (Rx) 233 that receives data input from the control device 10 via the interface 231 are mounted. In addition, a 6-axis sensor 235 and a magnetic sensor 237 are mounted on the display unit substrate 210.
The interface 231 connects the receiving unit 233, the six-axis sensor 235, the magnetic sensor 237, and the temperature sensor 239 to the control device 10.

The 6-axis sensor 235 (display unit motion detection unit) is a motion sensor (inertia sensor) including a 3-axis acceleration sensor and a 3-axis gyro (angular velocity) sensor. The 6-axis sensor 235 may employ an IMU (Inertial Measurement Unit) in which the above sensors are modularized.
The magnetic sensor 237 is, for example, a triaxial geomagnetic sensor.

The temperature sensor 239 detects the temperature and outputs a voltage value or a resistance value corresponding to the detected temperature as a detected value. The temperature sensor 239 is mounted on the back side of the OLED panel 243 (FIG. 3). The temperature sensor 239 may be mounted on the same substrate as the OLED drive circuit 245, for example. With this configuration, the temperature sensor 239 mainly detects the temperature of the OLED panel 243.
Further, the temperature sensor 239 may be incorporated in the OLED panel 243 or the OLED drive circuit 245. The substrate may be a semiconductor substrate. Specifically, when the OLED panel 243 is mounted as an Si-OLED as an integrated circuit on an integrated semiconductor chip together with the OLED drive circuit 245 and the like, the temperature sensor 239 may be mounted on the semiconductor chip.

The camera 61, the illuminance sensor 65, the temperature sensor 217, and the 6-axis sensor 235, the magnetic sensor 237, and the temperature sensor 239 included in the left display unit 24 are connected to the sensor hub 192. The sensor hub 192 sets and initializes the sampling period of each sensor according to the control of the main processor 140. The sensor hub 192 executes energization to each sensor, transmission of control data, acquisition of a detection value, and the like in accordance with the sampling period of each sensor. The sensor hub 192 outputs detection values of the sensors included in the right display unit 22 and the left display unit 24 to the main processor 140 at a preset timing. The sensor hub 192 may have a function of temporarily holding the detection value of each sensor in accordance with the output timing to the main processor 140. In addition, the sensor hub 192 may have a function of converting to data in a unified data format and outputting the data to the main processor 140 in response to a difference in signal format or data format of the output value of each sensor.
The sensor hub 192 starts and stops energization of the LED indicator 67 according to the control of the main processor 140, and lights or blinks the LED indicator 67 in accordance with the timing when the camera 61 starts and ends imaging.

  The control device 10 includes a power supply unit 130 and operates with power supplied from the power supply unit 130. The power supply unit 130 includes a rechargeable battery 132 and a power supply control circuit 134 that detects the remaining capacity of the battery 132 and controls the charging of the battery 132. The power control circuit 134 is connected to the main processor 140 and outputs a detected value of the remaining capacity of the battery 132 or a detected value of the voltage to the main processor 140. Further, the power may be supplied from the control device 10 to the image display unit 20 based on the power supplied from the power supply unit 130. The power supply state from the power supply unit 130 to each unit of the control device 10 and the image display unit 20 may be configured to be controllable by the main processor 140.

  FIG. 5 is a functional block diagram of the storage unit 122 and the control unit 150 that constitute the control system of the control device 10. The storage unit 122 illustrated in FIG. 5 is a logical storage unit configured by the nonvolatile storage unit 121 (FIG. 4), and may include an EEPROM 215. The control unit 150 and various functional units included in the control unit 150 are formed by cooperation of software and hardware by the main processor 140 executing a program. The control unit 150 and each functional unit constituting the control unit 150 include, for example, a main processor 140, a memory 118, and a nonvolatile storage unit 121.

The control unit 150 executes various processes using data stored in the storage unit 122 and controls the HMD 100.
The storage unit 122 stores various data processed by the control unit 150. The storage unit 122 stores setting data 123, content data 124, operation setting data 125, and display control data 126. The setting data 123 includes various setting values related to the operation of the HMD 100. When the control unit 150 controls the HMD 100, parameters, determinants, arithmetic expressions, LUTs (Look Up Tables), and the like may be included in the setting data 123.

The content data 124 is content data including images and videos displayed by the image display unit 20 under the control of the control unit 150, and includes image data or video data. Further, the content data 124 may include audio data. The content data 124 may include image data of a plurality of images. In this case, the plurality of images are not limited to images displayed on the image display unit 20 at the same time.
The content data 124 is an interactive type in which when the content is displayed on the image display unit 20, the control device 10 receives a user operation and the control unit 150 executes a process according to the received operation. It may be content. In this case, the content data 124 may include image data of a menu screen that is displayed when an operation is accepted, data that defines processing corresponding to items included in the menu screen, and the like.

  The operation setting data 125 is data used by the control unit 150 in a process of detecting a motion operation that moves the control device 10. The operation setting data 125 includes data for determining each motion of one or a plurality of motion patterns preset as the motion of the case 10A. Specifically, for each type of motion, a detection value of motion detected by the 6-axis sensor 111 or data of a change in detection value is included. This data may include a plurality of detection values of the six-axis sensor 111, or may be feature amount data obtained by calculation from the detection values. The operation setting data 125 may be data that can determine from the detection value of the six-axis sensor 111 whether or not the case 10A has moved corresponding to the set motion. More preferably, the data can identify a motion corresponding to the movement of the case 10 </ b> A detected by the 6-axis sensor 111 from a plurality of set motions.

  The display control data 126 is data used in processing for changing the display state of an image or the like displayed by the image display unit 20 in accordance with the motion of the case 10A. More specifically, it includes data indicating a mode of changing the display state in association with the set motion of the case 10A. When the control unit 150 detects and identifies the motion of the case 10A based on the operation setting data 125, the control unit 150 changes the display of the image display unit 20 in a manner set in the display control data 126 in association with the identified motion. Let

  The control unit 150 has functions of an operating system (OS) 143, an image processing unit 145, a display control unit 147, an imaging control unit 149, and an operation detection control unit 152. The functions of the operating system 143 are functions of a control program stored in the storage unit 122, and the other units are functions of application programs executed on the operating system 143.

The image processing unit 145 generates a signal to be transmitted to the right display unit 22 and the left display unit 24 based on image data of an image or video displayed by the image display unit 20. The signal generated by the image processing unit 145 may be a vertical synchronization signal, a horizontal synchronization signal, a clock signal, an analog image signal, or the like.
Further, the image processing unit 145 may perform resolution conversion processing for converting the resolution of the image data to a resolution suitable for the right display unit 22 and the left display unit 24 as necessary. The image processing unit 145 also performs image adjustment processing for adjusting the brightness and saturation of image data, 2D image data from 3D image data, 2D / 3D conversion processing for generating 3D image data from 2D image data, and the like. May be executed. When these image processes are executed, the image processing unit 145 generates a signal for displaying an image based on the processed image data, and transmits the signal to the image display unit 20 via the connection cable 40.

  The image processing unit 145 may be configured by hardware (for example, DSP (Digital Signal Processor)) different from the main processor 140 in addition to the configuration realized by the main processor 140 executing a program.

The display control unit 147 generates a control signal for controlling the right display unit 22 and the left display unit 24, and controls the generation and emission of image light by the right display unit 22 and the left display unit 24 based on the control signal. . Specifically, the display control unit 147 controls the OLED drive circuits 225 and 245 to cause the OLED panels 223 and 243 to display an image. The display control unit 147 controls the timing at which the OLED drive circuits 225 and 245 draw on the OLED panels 223 and 243, controls the luminance of the OLED panels 223 and 243, and the like based on the signal output from the image processing unit 145.
Further, the display control unit 147 changes the display mode of an image or the like displayed on the image display unit 20 according to the display control data 126 when the operation detection control unit 152 detects the motion of the control device 10.

  The imaging control unit 149 controls the camera 61 to execute imaging, generates captured image data, and temporarily stores the captured image data in the storage unit 122. When the camera 61 is configured as a camera unit including a circuit that generates captured image data, the imaging control unit 149 acquires captured image data from the camera 61 and temporarily stores the captured image data in the storage unit 122.

  The operation detection control unit 152 detects an operation on the track pad 14 and the operation unit 110, and outputs data corresponding to the operation. For example, when a button or the like of the operation unit 110 is operated, the operation detection control unit 152 generates operation data indicating the operation content and outputs the operation data to the display control unit 147. The display control unit 147 changes the display state according to the operation data input from the operation detection control unit 152.

The operation detection control unit 152 detects the operation of the software button while the LED display unit 17 is lit. The operation detection control unit 152 detects a contact operation with respect to the position of the symbol displayed when the LED display unit 17 is lit while the LED display unit 17 is lit as an operation of a button (software operator) to which a function is assigned in advance. .
For example, in the example of FIG. 1, an operation that touches the position of the symbol ◯ (circle) is detected as an operation of a home button assigned in advance to the position of the symbol ◯ (circle). In this case, the operation detection control unit 152 shifts the display screen of the image display unit 20 and the operation state of the control device 10 to the basic state of the operating system 143.
Further, for example, in the example of FIG. 1, an operation that touches the position of the symbol □ (square) is detected as an operation of a history button assigned in advance to the position of the symbol □ (square). In this case, the operation detection control unit 152 causes the image display unit 20 to display a display screen of the image display unit 20 and a screen indicating a past operation state of the control device 10.

The operation detection control unit 152 identifies the movement of the case 10A based on the detection value of the 6-axis sensor 111, and determines whether or not the movement of the case 10A corresponds to a preset motion. In the present embodiment, the operation detection control unit 152 performs a process of specifying a motion corresponding to the movement of the case 10A. In this process, the operation detection control unit 152 acquires the detection value of the six-axis sensor 111. Further, a plurality of detection values detected by the six-axis sensor 111 may be acquired. For example, when the six-axis sensor 111 performs detection at a predetermined cycle, a plurality of detection values that the six-axis sensor 111 continuously detects. May be acquired continuously. The operation detection control unit 152 may store the detection value of the six-axis sensor 111 in the memory 118 or the nonvolatile storage unit 121 as necessary. For example, a ring buffer may be formed in the memory 118 or the nonvolatile storage unit 121, and the operation detection control unit 152 may constantly acquire the detection value of the six-axis sensor 111 and store it in the ring buffer. In this case, the latest detection values corresponding to the capacity of the ring buffer are stored in the ring buffer. The operation detection control unit 152 can perform the above processing based on the detection value stored in the ring buffer and the operation setting data 125.
The operation detection control unit 152 acquires the coordinates of the operation position in the operation detection area (detection area) of the track pad 14 when an operation on the track pad 14 is detected.

  The HMD 100 may include an interface (not shown) for connecting various external devices that are content supply sources. For example, an interface corresponding to a wired connection such as a USB interface, a micro USB interface, or a memory card interface may be used, or a wireless communication interface may be used. The external device in this case is an image supply device that supplies an image to the HMD 100, and a personal computer (PC), a mobile phone terminal, a portable game machine, or the like is used. In this case, the HMD 100 can output images and sounds based on content data input from these external devices.

  FIG. 7 is an explanatory diagram showing an operation example using the control device 10, and specifically shows an example of a motion for moving the case 10A. 7A and 7B show a state where the user holds the case 10A in his / her hand, and FIG. 7A is a front view of the control device 10 as viewed from the user side. (B) is a side view. In addition, illustration of the connection cable 40 is abbreviate | omitted in FIG. 7 (A), (B).

In the present embodiment, six motions are defined in advance as modes of operation for moving the control device 10.
FIG. 7A shows four types of motions A, B, C, and D. FIG. 7B shows two types of motions E and F.
In the illustrated example, the user holds the case 10A with the right hand RH, but may hold it with the left hand. Further, the user may grasp the case 10A jointly using a jig or the like.

  The motion A is an operation of rotating the control device 10 clockwise (clockwise: CW) as indicated by an arrow A with reference to the left and right sides of the user's body. The 6-axis sensor 111 detects acceleration and angular velocity indicating the clockwise movement of the case 10A. Motion B is an operation of rotating the control device 10 counterclockwise (counterclockwise: CCW) as indicated by an arrow B in the figure. Motion A can be called a right rotation operation, and Motion B can be called a left rotation operation.

  As shown in the figure, the control device 10 is preset in the left-right direction when the user holds it in his / her hand. For example, as shown in FIGS. 7A and 7B, when the user holds the case 10A in his / her hand, the upper end of the case 10A is turned up and the surface on which the track pad 14 is arranged faces the right side of the user. It may be specified in advance as follows. The upper end surface of the case 10A is a surface on which the LED indicator 12 and the power switch 18 are provided. Moreover, the structure which a user can set by the operation of HMD100 which surface of case 10A is made into the right side and the left side when a user holds the control apparatus 10 in hand may be sufficient. In this case, the operation setting data 125 includes data that associates the detection direction of the six-axis sensor 111 with the left and right sides of the user's body.

Motion C is an operation of moving the control device 10 so as to draw an arc on the right side as indicated by an arrow C in the drawing, and Motion D is an operation of moving the control device 10 on the left side as indicated by an arrow D in the drawing. It is an operation to move as if drawing. Motion C can be referred to as a right movement operation, and Motion D can be referred to as a left movement operation.
Note that, as indicated by an arrow C ′ in the figure, an operation for linearly moving the control device 10 to the right may be detected as motion C. Similarly, as indicated by an arrow D ′ in the figure, a configuration in which an operation of linearly moving the control device 10 to the left may be detected as a motion D.

  The motion E is an operation of moving the control device 10 so as to draw an arc upward as indicated by an arrow E in FIG. The motion F is an operation of moving the control device 10 so as to draw an arc downward as indicated by F in FIG. Motion E can be referred to as a swing-up operation, and motion D can be referred to as a swing-down operation. As indicated by an arrow E ′ in the figure, an operation for linearly moving the control device 10 upward may be detected as motion E. Further, as indicated by an arrow F ′ in the figure, an operation for linearly moving the control device 10 downward may be detected as motion E.

  Some of these six types of motion are associated with each other as a pair of motions (motions). Motion A and motion B are set as a pair of motions. Similarly, motion C and motion D, motion E and motion F are set as a pair of motions. Hereinafter, a pair of motions is called reverse motion. For example, motion B corresponds to the reverse motion of motion A, and vice versa. The same applies to the motions C, D, E, and F.

  One of these paired motions corresponds to an operation (movement, motion) in the first movement direction, and the other corresponds to an operation in the second movement direction. The first movement direction and the second movement direction are not limited to the opposite directions. Any direction may be used as long as it can be detected as a movement in a different direction by the six-axis sensor 111 or the magnetic sensor 113 and / or the camera 61 used instead of the six-axis sensor 111. Each of the motions A to F, C ′, D ′, E ′, and F ′ is set as a movement in a specific direction, but a movement close to the specific direction is regarded as a movement in the specific direction. Good. For example, the motion C shown in FIG. 7A is a horizontal movement in the figure, but a movement in an angle range of about 5 degrees with respect to the horizontal may be detected as the motion C. The range regarded as the motion in the same direction can be arbitrarily set, for example, ± 5 degrees or ± 10 degrees. The same applies to the first movement direction and the second movement direction.

  The operation detection control unit 152 detects the movement of the six cases 10A of motions A to F, and the display control unit 147 displays the image display unit 20 so as to correspond to the motion detected by the operation detection control unit 152. Control.

8, 9 and 10 are explanatory diagrams showing changes in display corresponding to the operation of the HMD.
FIG. 8 shows an example of enlarging and reducing the display. 8A shows a display example before enlargement, FIG. 8B shows a display example after enlargement, FIG. 8C shows a display example before reduction, and FIG. 8D shows reduction. The following display example is shown. Reference sign VR in the figure indicates a user's visual field, and reference sign V indicates a display area in which an image can be displayed by the right display unit 22 and the left display unit 24. Reference numerals W1, W2, and W3 denote windows as examples of images displayed in the display area V.

  In the state shown in FIG. 8A, when the display is enlarged as shown by an arrow in the drawing corresponding to the motion of the case 10A, the display control unit 147 determines the center position CE of the enlargement process in the display region V. To do. The center position CE may be designated by an operation on the track pad 14 or may be automatically determined based on the position of a pointer (not shown) when starting the enlargement process. In the example of FIG. 8B, the window W1 of the display area V shown in FIG. 8A is enlarged and displayed with the center position CE as the center. Comparing the display example of FIG. 8A and the display example of FIG. 8B, the enlargement ratio for enlarging the window W1 is determined by an operation on the control device 10, as will be described later with reference to a flowchart. It may be set in advance.

  In the case where the display is reduced as shown by an arrow in the figure from the state shown in FIG. 8C in response to the motion of the case 10A, the display control unit 147 also performs the center of the reduction process in the display region V. A position CE is determined. The center position CE may be designated by an operation on the track pad 14 or may be automatically determined based on the position of a pointer (not shown) when starting the enlargement process. In the example of FIG. 8D, the window W1 of the display area V shown in FIG. 8C is reduced and displayed around the center position CE. Comparing the display example of FIG. 8C and the display example of FIG. 8D, the reduction ratio for reducing the window W1 is determined by an operation on the control device 10, as will be described later with reference to a flowchart. It may be set in advance.

FIG. 9 shows an example of scrolling the display. FIG. 9A shows a display example before scrolling, FIG. 9B shows a display example scrolled upward, and FIG. 9C shows a display example scrolled downward.
From the state shown in FIG. 9A, the display control unit 147 can scroll the display up and down as shown by the arrows in the drawing in accordance with the motion of the case 10A. When scrolling upward, as shown in FIG. 9B, the display position of the window W1 moves upward. When scrolling downward, the display position of the window W1 moves downward as shown in FIG. 9C. As shown in the display examples of FIGS. 9A to 9C, the scroll amount for scrolling the window W1 may be determined by an operation on the control device 10, as will be described later with reference to a flowchart. It may be set in advance.
Although not shown, the display control unit 147 may execute a horizontal scroll process in which an image displayed in the display area V is moved leftward or rightward.

  Either one of the upward scroll in FIG. 9B and the downward scroll in FIG. 9C corresponds to the first display operation direction, and the other corresponds to the second display operation direction. These are merely examples, and the first display operation direction and the second display operation direction may be display movements in different directions. For example, the horizontal display direction (which may be either right or left) and the vertical direction (which may be either up or down) may be used as the first display operation direction and the second display operation direction. The display movement in the oblique direction may be the first display operation direction and / or the second display operation direction. That is, the display movement in two directions that can be identified as different directions for the user can be the first display operation direction and the second display operation direction.

  Further, in FIGS. 9B and 9C, in response to the operation of moving the case 10A upward, scrolling of the display in the upward direction is executed, and in response to the operation of moving the case 10A downward. An example of scrolling the display in the downward direction was shown. These are merely examples, and the movement of the case 10A detected by the 6-axis sensor 111 and the movement direction of the display need not be the same relative to the user. The direction of movement detected by the six-axis sensor 111 and the direction of movement of display need only be associated in advance, and may be in the opposite direction, for example.

FIG. 10 shows an example of switching the display. 10A shows a display example before processing, FIG. 10B shows a display example in which the display is changed to the previous screen, and FIG. 10C is a table in which the display is changed to the next screen. An example is shown.
In the example shown in FIGS. 10A to 10C, the content data 124 displayed by the display control unit 147 includes a plurality of images in which the display order is set, and when the content data 124 is reproduced, the display control unit 147 is displayed. Corresponds to an example of switching and displaying images according to the display order. These images are not limited to images such as a single photo, but may be, for example, a combination of images and text that constitute one screen. Specifically, in a configuration in which the control unit 150 executes an application program on the operating system 143, a web page displayed by a web browser is given. In a configuration in which a presentation program is executed as an application program, a presentation slide sheet having a plurality of slide sheets can be cited as the content data 124 including a plurality of images.

FIG. 10A shows an example in which a slide sheet for presentation is displayed as an example. For example, in FIG. 10A, a window W1 including a slide sheet is displayed. When the display control unit 147 changes the screen to the previous screen in this state, the window W2 including the previous slide sheet is displayed in the display area V. Is displayed. In addition, when the display control unit 147 changes the screen to the next screen in FIG. 10A, a window W3 including the next slide sheet is displayed in the display area V.
That is, in the example of FIGS. 10A to 10C, in an operation of displaying a plurality of images in which the display order is set, a display change for sequentially switching images and a display change for switching images in reverse order are shown.

  In the present embodiment, the display control unit 147 is set to enlarge the display corresponding to the motion A (right rotation), and the display control unit 147 is set to reduce the display corresponding to the motion B (left rotation). Further, the display control unit 147 scrolls the display upward corresponding to the motion E (up), and the display control unit 147 scrolls the display downward corresponding to the motion F (down). In addition, the display control unit 147 changes the display to the next screen corresponding to the motion C (right movement), and the display control unit 147 changes the display to the next screen corresponding to the motion D (left movement).

11 and 12 are flowcharts showing the operation of the HMD 100.
When starting a motion operation for moving the control device 10 (step S11), the control unit 150 determines whether to use a position instruction (position input) by the track pad 14 (step S12). When the position instruction is used, the center position CE is designated by operating the track pad 14 in the process of enlarging the display when the motion A is executed and the process of reducing the display when the motion B is executed. Whether to use the position instruction may be designated by the user through an input operation on the operation unit 110, or may be set in advance.

When the position instruction is used (step S12; Yes), the control unit 150 proceeds to step S31 (FIG. 12). This case will be described later.
When the position instruction is not used (step S12; No), the control unit 150 starts obtaining the detection value of the six-axis sensor 111 (step S13). Thereafter, the control unit 150 acquires the detection value of the six-axis sensor 111 at a preset cycle.

  Subsequently, the control unit 150 determines whether or not there is an operation serving as a trigger for controlling the display (step S14), and when there is no operation (step S14; No), the control unit 150 waits until there is an operation. Examples of the operation that serves as a trigger include operations on various operators such as the button 11 provided on the case 10A. In the present embodiment, a contact operation on the track pad 14 is set as a trigger.

  When the control unit 150 determines that the contact operation with respect to the track pad 14 has been performed (step S14; Yes), the elapsed time from the start of obtaining the detection value at step S13 is set to the set time (set time). It is determined whether or not it has been reached (step S15).

Functions are assigned in advance to operators corresponding to operations that serve as triggers for controlling display. For example, in the present embodiment, the contact operation of the track pad 14 is set as the trigger operation, but the track pad 14 is assigned with the function of the position input operation and the software button corresponding to the display position of the LED display unit 17. Yes. Whether the operation detection control unit 152 detects a trigger operation when the touch operation of the track pad 14 is performed, or detects a position input operation of the track pad 14 or a software button operation at the display position of the LED display unit 17 Can be switched. When the elapsed time from the start of obtaining the detection value of the six-axis sensor 111 to the operation of the track pad 14 reaches the set time (step S15; Yes), the operation detection control unit 152 performs the motion operation. Execute normal operation mode without accepting. The operation detection control unit 152 clears the detection value of the six-axis sensor 111 that has already been acquired and stored (step S16), and executes the function assigned as the normal operation for the trigger operation detected in step S14 (step S16). S17), the process ends.
For example, when the contact of the position of the symbol ◯ (circle) on the track pad 14 is detected in step S14, the operation detection control unit 152 performs control corresponding to the operation of the home button in step S17.

  When the elapsed time has not reached the set time (step S15; No), the operation detection control unit 152 starts the display operation mode (step S18). The display operation mode is an operation mode in which a trigger operation is detected as a motion operation trigger. The operation detection control unit 152 analyzes the movement of the case 10A based on the detection value of the six-axis sensor 111 (step S19). The detection values analyzed in step S19 include the past detection values that have been acquired in step S13 and stored in the memory 118 or the nonvolatile storage unit 121. In particular, the detection values until the trigger operation is detected in step S14. Includes detected values acquired in between.

  For example, when the user moves the control device 10 in his / her hand and touches the track pad 14 while the case 10A is moving, the control unit 150 moves the case 10A from before the contact operation of the track pad 14. Is analyzed. For example, the movement of the case 10A intentionally performed by the user is analyzed even when the timing between the operation of moving the control device 10 and the operation as a trigger is shifted against the intention of the user.

  The operation detection control unit 152 determines whether the motion corresponding to the movement of the case 10A analyzed in step S19 is in a preset motion (for example, motions A to F) (step S20). When there is no corresponding motion (step S20; No), the operation detection control unit 152 analyzes the movement of the case 10A again (step S21), and returns to step S20 to make a determination. In step S21, since the analysis is performed including the detection value of the six-axis sensor 111 acquired between step S19 and step S21, the motion of the new case 10A may be detected.

  When there is a motion corresponding to the movement of the case 10A (step S20; Yes), the operation detection control unit 152 determines whether or not to perform a combination operation (step S22). The combination operation is an operation method for controlling display by combining a plurality of pairs of motions. Whether or not to perform the combination operation may be determined by an operation on the control device 10 or may be set in advance.

  When the combination operation is not performed (step S23; No), the display control unit 147 determines the content of the display change corresponding to the motion determined by the operation detection control unit 152 in step S20 and the display change amount (step). S23). For example, the enlargement / reduction ratio is determined corresponding to the motions A and B, and the scroll direction and the scroll amount are determined corresponding to the motions E and F. The amount of change in display can be set in advance for each type of motion.

  The display control unit 147 changes the display of the image display unit 20 according to the display control content determined in step S23 and the amount of change in display (step S24), and proceeds to step S29 described later.

  When performing a combination operation (step S23; Yes), the display control unit 147 starts a display change corresponding to the motion determined by the operation detection control unit 152 in step S20 (step S25). After starting the display change, the operation detection control unit 152 analyzes the movement of the case 10A based on the detection value of the six-axis sensor 111 (step S26). The operation detection control unit 152 determines whether or not the movement of the case 10A corresponding to the reverse motion of the motion determined in step S20 has been performed (step S27). While the motion of the reverse motion is not performed (step S27; No), the analysis of steps S26 to S27 is performed and the process waits.

When the operation detection control unit 152 determines that the motion of the reverse motion has been performed (step S27; Yes), the display control unit 147 stops the display change (step S28), and proceeds to step S29.
For example, the control unit 150 starts scrolling in the upward direction in response to the motion E in step S25, and then stops the scrolling in the upward direction when it is determined in step S27 that the motion F has been performed. As described above, since the timing for stopping the change in display corresponding to the motion can be specified by the reverse motion, the user can freely adjust the amount of change in the display.

  In step S29, the operation detection control unit 152 determines whether or not the trigger operation has been released (step S29). If the trigger operation has not been released (step S29; No), the process returns to step S20. Further, when the operation of the operator assigned to the trigger operation is released (step S29; Yes), this process is terminated.

  On the other hand, when the position instruction is used (step S12; Yes), the operation detection control unit 152 determines whether or not the position instruction operation by the track pad 14 has been performed (step S31). S31; No), it waits until there is an operation. When the operation detection control unit 152 determines that the position instruction operation by the track pad 14 has been performed (step S31; Yes), the elapsed time waiting for the position instruction operation in step S31 reaches the set time (set time). It is determined whether or not (step S32).

  If the operation detection control unit 152 determines that the time waiting for the position instruction operation has reached the set time (step S32; Yes), the operation detection control unit 152 executes the normal operation mode without accepting the motion operation. The operation detection control unit 152 executes the above-described operations of steps S16 to S17, and ends this process.

  When the elapsed time has not reached the set time (step S32; No), the operation detection control unit 152 starts the display operation mode (step S33). The operation detection control unit 152 analyzes the movement of the case 10A based on the detection value of the 6-axis sensor 111 (step S34). In step S <b> 33, the operation detection control unit 152 may perform analysis including past detection values that have already been acquired and stored in the memory 118 or the nonvolatile storage unit 121.

  The operation detection control unit 152 determines whether the motion corresponding to the movement of the case 10A analyzed in step S34 is in a preset motion (for example, motions A to F) (step S35). When there is no corresponding motion (step S35; No), the operation detection control unit 152 analyzes the motion of the case 10A again (step S36), and returns to step S35 to make a determination.

  When there is a motion corresponding to the movement of the case 10A (step S35; Yes), the operation detection control unit 152 acquires the coordinates of the position indicated by the position instruction operation detected in step S31 (step S37). Here, the operation detection control unit 152 may obtain the display positions of the right display unit 22 and the left display unit 24 corresponding to the acquired designated position.

  The operation detection control unit 152 determines whether to perform a combination operation (step S38). The combination operation is as described above. When the combination operation is not performed (step S38; No), the display control unit 147 displays the content of the display change corresponding to the motion determined by the operation detection control unit 152 in step S36 and the display change amount, as in step S23. Are determined (step S39).

  The display control unit 147 changes the display of the image display unit 20 according to the display change content and the change amount determined in step S39, with the instruction position acquired by the operation detection control unit 152 in step S37 as a reference (step S40). Then, the process proceeds to step S45 described later.

When performing the combination operation (step S38; Yes), the display control unit 147 starts a display change corresponding to the motion determined by the operation detection control unit 152 in step S36 (step S41). In step S41, the indication position acquired by the operation detection control unit 152 in step S37 is used as a reference for display change.
In steps S39 and S41, the display control unit 147 executes display enlargement / reduction with the indicated position as the center position CE, for example, as shown in FIGS.

  After starting the display change, the operation detection control unit 152 analyzes the movement of the case 10A based on the detection value of the six-axis sensor 111 (step S42). The operation detection control unit 152 determines whether or not the movement of the case 10A corresponding to the reverse motion of the motion determined in step S20 has been performed (step S43). While the motion of the reverse motion is not performed (Step S43; No), the analysis of Steps S42 to S43 is performed and the process waits.

When the operation detection control unit 152 determines that the motion of the reverse motion has been performed (step S43; Yes), the display control unit 147 stops the display change (step S44), and proceeds to step S45.
In step S45, the operation detection control unit 152 determines whether or not the trigger operation has been released (step S45). If the trigger operation has not been released (step S45; No), the process returns to step S35. Further, when the operation of the operation element assigned to the trigger operation is released (step S45; Yes), this process ends.

If the trigger operation is canceled before determining that the reverse motion has been performed in steps S26 to S27, the display control unit 147 stops the display change in step S28 as in the case where the reverse motion has been performed. Then, this process may be terminated.
Similarly, in step S42 to S43, when the trigger operation is released before determining that the reverse motion has been performed, the display control unit 147 changes the display in step S44 in the same manner as when the reverse motion has been performed. You may stop and complete | finish this process.
Further, in addition to the reverse motion, a specific motion for stopping the change in time may be set in advance. For example, the movement that moves the case 10A in small increments with a smaller amount of motion than the motions E and F may be set as a stop motion that serves as a trigger to stop the change in time. In this case, in steps S27 and S43, the operation detection control unit 152 may determine that a reverse motion or a stop motion has been performed.

  In step S31 described above, when the operation detection control unit 152 detects the first contact operation with respect to the track pad 14, it may be determined that the position instruction operation has been performed, but the instruction position is determined by the user's instruction. May be. For example, when the contact operation with respect to the track pad 14 is continuously performed, it may be determined that the position instruction operation has been performed at the timing instructed by the user, and the designated position at the timing may be acquired in step S37. . The method of instructing the timing by the user may be, for example, an operation of an operator other than the track pad 14 provided in the control device 10. Further, for example, the timing may be designated by the movement of the image display unit 20. Specifically, the movement of moving the image display unit 20 up and down in small increments may be set as the motion of the display unit that specifies the timing of the position instruction. The movement of the image display unit 20 can be detected and determined by the operation detection control unit 152 based on the detection value of the 6-axis sensor 235. In this case, the user can designate the timing by moving the head while operating the track pad 14. Therefore, a user wearing the image display unit 20 on the head, particularly in front of the eyes, can easily perform an operation of designating an arbitrary position without looking at the control device 10. Also, an image indicating the position indicated by the track pad 14, that is, the contact position, is displayed from the start of the contact operation of the track pad 14 until the timing is designated by the movement of the image display unit 20. 20 may be displayed. In this case, the user can adjust the operation position of the track pad 14 while observing the display on the image display unit 20 to determine an arbitrary designated position by the movement of the head, thereby further improving the operability. be able to.

  As described above, the HMD 100 is a head-mounted display device having the image display unit 20 that allows a user to visually recognize an image. The HMD 100 includes a control device 10 as an operation device that can be moved independently of the image display unit 20. The control device 10 includes a 6-axis sensor 111 that detects the movement of the control device 10. The HMD 100 also includes a control unit 150 that controls the display of the image display unit 20 according to the movement detected by the 6-axis sensor 111. The control unit 150 performs, as processing on the image displayed by the image display unit 20, the first display change corresponding to the movement in the first movement direction detected by the 6-axis sensor 111 and the first detection detected by the 6-axis sensor 111. The second display change corresponding to the movement in the second movement direction is executed. The first display change is started when movement in the first movement direction is detected, and then the first display change is stopped when movement in the second movement direction is detected by the six-axis sensor 111. To do. Thereby, the display of the image display unit 20 can be easily controlled by an operation of moving the control device 10. Further, by combining the movements of the control device 10 in two directions, the image displayed by the image display unit 20 can be moved to a desired position.

  Here, the motion detector that detects the motion in the first motion direction and / or the motion in the second motion direction is not limited to the six-axis sensor 111 but may be the magnetic sensor 113. In addition, it is also possible to detect the movement in the first movement direction and / or the movement in the second movement direction using the captured image of the camera 61. In this case, the camera 61 corresponds to a movement detection unit. . Further, the motion detection unit may include a function of the control unit 150 that detects a motion by analyzing a captured image of the camera 61.

  In addition, the control unit 150 executes the first display change when the movement in the first movement direction is detected by the six-axis sensor 111, and the movement in the second movement direction is detected by the six-axis sensor 111. If so, a second display change is performed. As a result, the image displayed by the image display unit 20 can be moved by moving the control device 10, and the direction of movement of the image can be indicated by the movement direction of the control device 10.

  For example, when the controller 150 detects the motion E as the movement in the first movement direction, the control unit 150 performs the upward scroll as the first display change, and detects the motion F as the movement in the second movement direction. Down scrolling can be executed as the second display change. Further, after the motion E is detected and the upward scrolling is started, the upward scrolling can be stopped when the motion F is detected as the motion in the second motion direction.

  In addition, the control unit 150 changes the display of the image display unit 20 according to the movement detected by the six-axis sensor 111 in a state where the track pad 14 receives an operation. Thereby, the change of the display which a user does not intend by the operation state of the track pad 14 can be prevented.

  In addition, the control unit 150 executes the first display change when the six-axis sensor 111 detects a movement in the first movement direction in a state where the track pad 14 receives an operation. Thereby, the change of the display which a user does not intend by the operation state of the track pad 14 can be prevented.

  In addition, the control unit 150 starts the first display change when the movement in the first movement direction is detected by the six-axis sensor 111, and thereafter, when the track pad 14 no longer accepts the operation, The display change of 1 is stopped. Thereby, the change of the display which a user does not intend by the operation state of the track pad 14 can be prevented.

  In addition, the control unit 150 executes the second display change when the six-axis sensor 111 detects a movement in the second movement direction in a state where the track pad 14 receives an operation. As a result, the image displayed by the image display unit 20 can be moved by moving the control device 10, and the direction of movement of the image can be indicated by the movement direction of the control device 10.

  The operation receiving unit including the track pad 14 receives an operation different from the detection of the 6-axis sensor 111 or the like as the motion detecting unit. For this reason, a state in which display is controlled and a state in which display control is not performed can be selected by an operation different from the movement of the control device 10 detected by the 6-axis sensor or the like. For this reason, the change of the display which a user does not intend can be prevented more reliably.

  Here, as a specific example of the operation different from the detection of the six-axis sensor 111 as the motion detection unit, the operation of the track pad 14 is illustrated in the above embodiment. That is, the configuration in which the operation detection unit corresponds to the track pad 14 is illustrated. The present invention is not limited to this, and the up / down key 15, the changeover switch 16, the power switch 18, and the like may detect an operation different from the detection of the motion detection unit as the operation detection unit. Further, when the 6-axis sensor 111 that is the motion detection unit detects an operation of moving the case 10A, the operation of hitting the case 10A may be the above-mentioned “different operation”. In this case, the control device 10 may include a knock sensor and a pressure sensor, and the detection of these sensors may be an operation different from the detection of the six-axis sensor 111 as the motion detection unit.

  In the said embodiment, although the control apparatus 10 which has case 10A was demonstrated as an example of the operation device of this invention, this invention is not limited to this. For example, it may be a card-type operation device that does not have a housing, and may include a sensor corresponding to the motion detection unit and a functional unit corresponding to the operation reception unit. The present invention can also be applied to an operation device configured integrally with another apparatus. The operation device can also be called, for example, a remote controller, a control device, a control device, a small device, a motion device, or the like.

  In the above embodiment, the configuration in which the operation device is connected to the image display unit 20 by wire is illustrated. However, the present invention is not limited to this, and the operation device and the image display unit 20 are configured to be wirelessly connected. May be. In this case, the wireless communication method may be the method exemplified as the communication method supported by the communication unit 117, or may be another communication method.

  Further, the operation device need not have the function of the control device 10. In addition to the control apparatus 10, an apparatus, device, unit, or instrument used as an operation device may be provided.

  For example, instead of the control device 10, the present invention can also be applied using a wearable device that can be attached to a user's body, clothes, or an accessory worn by the user as an operation device. The wearable device in this case may be, for example, a watch-type device, a ring-type device, a laser pointer, a mouse, an air mouse, a game controller, a pen-type device, or the like.

Here, another example of the operation device will be described.
FIG. 13 is an external view of the watch-type device 3. The watch-type device 3 can be used as another example of the operation device. The watch-type device 3 is used together with the control device 10, for example. In this case, the control device 10 may be configured not to function as an operation device.

  The watch-type device 3 has a band portion 300 having a shape similar to that of a wristwatch band. The band part 300 has a fixing part such as a buckle (not shown), and can be wound and fixed around, for example, a user's forearm. The band portion 300 of the timepiece device 3 is formed with a substantially disk-shaped flat surface portion 300A at a position corresponding to the timepiece dial. The plane portion 300A is provided with a bezel 301, an LCD 303, a button 305, a crown-type operation element 307, and a plurality of buttons 309.

  The bezel 301 is a ring-shaped operation element, and is disposed on the peripheral portion of the flat surface portion 300A. The bezel 301 is provided so as to be rotatable in the circumferential direction with respect to the band unit 300. As will be described later, the timepiece device 3 includes a mechanism for detecting the rotation direction and the rotation amount of the bezel 301. The mechanism that instructs the band unit 300 to rotate the bezel 301 may include a memory that generates a notch sound when the bezel 301 is rotated.

The LCD 303 is a liquid crystal display that displays characters and images.
The button 305 is a push button type switch disposed outside the bezel 301. The button 305 is positioned below the bezel 301 when viewed from the user when the watch-type device 3 is attached. The button 305 is larger than the crown-type operation element 307 and the button 309 and can be operated by groping.

  The crown-type operation element 307 is an operation element having a shape imitating a crown of a wristwatch, and can be rotated as indicated by an arrow in the drawing. The watch-type device 3 includes a mechanism for detecting the rotation direction and the rotation amount of the crown-type operation element 307 when the user rotates the crown-type operation element 307. In addition, the mechanism that instructs the band unit 300 to rotate the crown-type operation element 307 may include a memory that generates a notch sound when the crown unit 307 is rotated.

The button 309 is a push button type switch provided on the outer peripheral portion of the plane portion 300A. The number of buttons 309 is not particularly limited, and in the present embodiment, an example in which four buttons 309 are provided is shown.
Different functions can be assigned to the buttons 309, and the functions assigned to the buttons 309 can be displayed on the LCD 303.

The watch-type device 3 can use the bezel 301, the button 305, the crown-type operation element 307, and the button 309 as an operation receiving unit. These operation accepting units are, for example, a function of instructing execution and / or cancellation of a mute function for temporarily stopping display of the image display unit 20 during normal operation, and types of pointers displayed by the image display unit 20. Assigned to the function that instructs change.
The watch-type device 3 includes a communication unit (not shown) that functions in the same manner as the communication unit 117 and a motion sensor (not shown) that is similar to the 6-axis sensor 111. The watch-type device 3 detects an operation on each unit functioning as an operation receiving unit, and wirelessly transmits the detection result to the control device 10. The watch-type device 3 wirelessly transmits the detection result of the motion sensor to the control device 10.

  In this configuration, the control device 10 can detect an operation for moving the timepiece device 3 to, for example, the motions A to F, C ′, D ′, E ′, and F ′ described above. Further, the control device 10 detects the operations of the bezel 301, the button 305, the crown-type operation element 307, and the button 309 as operations different from the detection of the motion sensor. In this case, the display on the image display unit 20 can be controlled by operating the bezel 301, the button 305, the crown type operator 307, and the button 309 while moving the watch-type device 3. The movement direction of the timepiece device 3 and the display control of the image display unit 20 can be the same as those in the above-described embodiment, for example.

  FIG. 14 is an external view of the ring-type device 4. The ring-type device 4 has the same shape as a ring, and is attached to, for example, a user's finger. On the surface of the ring-type device 4, a track pad 401 capable of detecting a contact operation is installed similarly to the track pad 14. The ring type device 4 detects contact with the track pad 401 by a detection circuit (not shown). In this configuration, the track pad 401 functions as an operation detection unit. The track pad 401 performs, for example, a function for instructing execution and / or cancellation of a mute function for temporarily stopping the display of the image display unit 20 and a change in the type of pointer displayed by the image display unit 20 during normal operation. Assigned to the commanding function.

  The ring device 4 includes a communication unit (not shown) that functions in the same manner as the communication unit 117 and a motion sensor (not shown) that is similar to the 6-axis sensor 111. The ring type device 4 detects an operation on each unit functioning as an operation receiving unit, and wirelessly transmits the detection result to the control device 10. The watch-type device 3 wirelessly transmits the detection result of the motion sensor to the control device 10.

  In this configuration, the control device 10 can detect an operation for moving the ring device 4 to, for example, the motions A to F, C ′, D ′, E ′, and F ′ described above. Further, the control device 10 detects the operation of the track pad 401 as an operation different from the detection of the motion sensor. In this case, the display on the image display unit 20 can be controlled by the operation of moving the ring device 4 while operating the track pad 401. The movement direction of the ring-type device 4 and the display control of the image display unit 20 can be the same as those in the above-described embodiment, for example.

  Further, when a wearable device that is worn on the user's body, such as the watch-type device 3 or the ring-type device 4, is used as the operation device, the display can be easily controlled by an operation of the user moving the body. Further, since an operation different from the detection of the motion sensor is detected and the display is controlled based on the detection value of the motion sensor, a display change unintended by the user can be prevented. Therefore, an intuitive operation can be realized, an erroneous operation can be prevented, and the display can be controlled as intended by the user.

  The direction in which the watch-type device 3 and the ring-type device 4 are moved can be detected and specified based on, for example, the X-axis direction, the Y-axis direction, and the Z-axis direction shown in the drawing. The motion for moving the watch-type device 3 and the ring-type device 4 in a specific direction is the same as the motions A to F, C ′, D ′, E ′, and F ′ described above, This corresponds to the operation in the second movement direction. In addition, the movement in the first movement direction and the second movement direction are set as movements in a specific direction, but movements close to the specific direction may be regarded as movements in the specific direction. . The range regarded as the motion in the same direction can be arbitrarily set, for example, ± 5 degrees or ± 10 degrees. The same applies to the first movement direction and the second movement direction. Further, the reference of the movement direction of the watch-type device 3 and the ring-type device 4 is set in advance in association with the detection direction of the motion sensor incorporated in each of the watch-type device 3 and the ring-type device 4. Just do it. Therefore, any direction can be set as a reference, and is not limited to the X-axis direction, the Y-axis direction, and the Z-axis direction shown in the drawing.

  Further, the watch-type device 3 and the ring-type device 4 are not limited to the configuration used with the control device 10. The HMD 100 only needs to have a configuration that has a function corresponding to the control device 10 and that can control the display of the image display unit 20. Accordingly, if the image display unit 20 is configured to accommodate each unit including the main processor 140, for example, the image display unit 20 and the clock device 3 or the image display unit 20 and the ring device 4 constitute the HMD 100. it can.

In addition, this invention is not restricted to the structure of the said embodiment, In the range which does not deviate from the summary, it can be implemented in a various aspect.
For example, in the above-described embodiment, the configuration in which the user visually recognizes the outside scene through the display unit is not limited to the configuration in which the right light guide plate 26 and the left light guide plate 28 transmit external light. For example, the present invention can be applied to a display device that displays an image in a state where an outside scene cannot be visually recognized. Specifically, the present invention is applied to a display device that displays a captured image of the camera 61, an image or CG generated based on the captured image, video based on video data stored in advance or video data input from the outside, and the like. Can be applied. This type of display device can include a so-called closed display device in which an outside scene cannot be visually recognized. In addition, as described in the above embodiment, AR display that displays an image superimposed on real space, MR (Mixed Reality) display that combines a captured real space image and a virtual image, or VR (VR that displays a virtual image) It can also be applied to a display device that does not perform processing such as virtual reality display. For example, a display device that displays video data or an analog video signal input from the outside is naturally included as an application target of the present invention.

  In the above embodiment, as an example of changing the display mode of the image displayed on the image display unit 20, enlargement, reduction, scrolling, and image switching based on the display order are illustrated, but the present invention is not limited to this. For example, control for rotating an image, control for changing the display brightness of an image, control for enlarging or reducing the display size of some of the images or characters being displayed, and the like may be performed in accordance with the motion. In this case, the rotation direction of the image, the change direction of the display luminance (brightness increase or decrease) of the image, and the enlargement or reduction of the size may be associated with different motions. In addition, the operation of the trigger for starting the operation by the motion is not limited to an operation element such as a button provided in the track pad 14 or the control device 10, and may be an operation that can be detected by a method different from the movement of the case 10A. Specifically, since it is only necessary to detect a detection value different from the detection value of the 6-axis sensor 111, for example, the movement of the image display unit 20 detected by the 6-axis sensor 235 may be used as a trigger operation. Further, for example, a touch sensor (not shown) may be provided in the image display unit 20, and an operation on the touch sensor may be a trigger operation.

  Further, for example, instead of the image display unit 20, another type of image display unit such as an image display unit worn like a hat may be adopted, and an image is displayed corresponding to the left eye of the user. And a display unit that displays an image corresponding to the right eye of the user. The display device of the present invention may be configured as a head mounted display mounted on a vehicle such as an automobile or an airplane. Further, for example, it may be configured as a head-mounted display built in a body protective device such as a helmet. In this case, the part for positioning the position with respect to the user's body and the part positioned with respect to the part can be used as the mounting portion.

Further, in the above embodiment, the image display unit 20 and the control device 10 are separated and connected via the connection cable 40 as an example. However, the control device 10 and the image display unit 20 are integrated. It is also possible to be configured to be mounted on the user's head.
As the control device 10, a notebook computer, a tablet computer, or a desktop computer may be used. The control device 10 may be a game machine, a portable phone, a portable electronic device including a smartphone or a portable media player, other dedicated devices, or the like. Further, the control device 10 may be configured separately from the image display unit 20, and various signals may be transmitted and received between the control device 10 and the image display unit 20 by wireless communication.

  Moreover, in the said embodiment, although the image display part 20 and the control apparatus 10 were isolate | separated and it demonstrated and demonstrated as an example the structure connected via the connection cable 40, the control apparatus 10 and the image display part 20 are wireless. It may be configured to be connected by a communication line.

In addition, as an optical system that guides image light to the user's eyes, a configuration in which virtual images are formed by the half mirrors 261 and 281 on a part of the right light guide plate 26 and the left light guide plate 28 is illustrated. The present invention is not limited to this, and an image may be displayed on a display region having an area that occupies the entire or most of the right light guide plate 26 and the left light guide plate 28. In this case, a process of reducing the image may be included in the operation of changing the display position of the image.
Furthermore, the optical element of the present invention is not limited to the right light guide plate 26 and the left light guide plate 28 having the half mirrors 261 and 281, and may be any optical component that allows image light to enter the user's eyes. A diffraction grating, a prism, or a holographic display unit may be used.

  In addition, at least a part of the functional blocks shown in FIGS. 4 and 5 may be realized by hardware, or may be realized by cooperation of hardware and software. As described above, the present invention is not limited to a configuration in which independent hardware resources are arranged. The program executed by the control unit 150 may be stored in the nonvolatile storage unit 121 or another storage device (not shown) in the control device 10. A program stored in an external device may be acquired and executed via the communication unit 117 or the external connector 184. Moreover, the operation part 110 may be formed as a user interface (UI) among the structures formed in the control apparatus 10. Further, the configuration formed in the control device 10 may be formed in the image display unit 20 in an overlapping manner. For example, a processor similar to the main processor 140 may be disposed in the image display unit 20, or the main processor 140 and the processor of the image display unit 20 included in the control device 10 perform functions separately divided. Also good.

  DESCRIPTION OF SYMBOLS 3 ... Clock type device (operation device), 4 ... Ring type device (operation device), 10 ... Control apparatus (operation device), 14 ... Trackpad (operation reception part), 20 ... Image display part (display part), 22 ... right display unit, 24 ... left display unit, 26 ... right light guide plate, 28 ... left light guide plate, 30 ... headset, 40 ... connection cable, 61 ... camera (motion detector), 63 ... microphone, 65 ... illuminance sensor , 67 ... LED indicator, 100 ... HMD (display device), 110 ... operation unit, 111 ... 6-axis sensor (motion detection unit), 113 ... magnetic sensor (motion detection unit), 115 ... GPS, 117 ... communication unit, 118 ... Memory, 119 ... Vibrator, 120 ... Controller board, 121 ... Nonvolatile memory, 122 ... Memory, 123 ... Setting data, 124 ... Computer Data, 125 ... operation setting data, 126 ... display control data, 130 ... power supply unit, 132 ... battery, 134 ... power supply control circuit, 140 ... main processor, 143 ... operating system, 145 ... image processing unit, 147 ... display control unit 149: Imaging control unit, 150 ... Control unit, 152 ... Operation detection control unit, 180 ... Audio codec, 182 ... Audio interface, 184 ... External connector, 186 ... External memory interface, 188 ... USB connector, 192 ... Sensor hub, 194 ... FPGA, 196 ... interface, 211,231 ... interface, 213,233 ... receiver, 215 ... EEPROM, 217,239 ... temperature sensor, 221,241 ... OLED unit, 235 ... 6-axis sensor, 237 ... magnetic Sensor: 300 ... Band part, 300A ... Flat part, 301 ... Bezel (operation receiving part), 303 ... LCD, 305 ... Button (operation receiving part), 307 ... Crown-type operation element (operation receiving part), 309 ... Button ( Operation accepting unit), 401... Trackpad (operation accepting unit).

Claims (10)

A head-mounted display device having a display unit that allows a user to visually recognize an image,
An operation device that can be moved independently of the display unit, the operation device having a motion detection unit that detects a motion of the operation device;
A control unit that controls display of the display unit in accordance with movement detected by the motion detection unit,
The control unit performs a process on the image displayed by the display unit, and includes a first display change corresponding to the movement in the first movement direction detected by the movement detection unit, and a second detection detected by the movement detection unit. Perform a second display change corresponding to the movement in the movement direction of
When the movement in the first movement direction is detected, the first display change is started. After that, when the movement in the second movement direction is detected by the movement detection unit, the first display direction is changed. Stop the display change,
A display device characterized by that. The control unit performs the first display change when the motion detection unit detects a motion in the first motion direction, and the motion detection unit detects a motion in the second motion direction. Performing the second display change when
The display device according to claim 1. The operation device includes an operation reception unit that receives an operation,
The control unit is configured to change the display of the display unit according to the movement detected by the movement detection unit in a state where the operation reception unit is receiving an operation;
The display device according to claim 1 or 2. The control unit performs the first display change when the movement detecting unit detects a movement in the first movement direction in a state where the operation receiving unit is receiving an operation;
The display device according to claim 3. The control unit starts the first display change when a movement in the first movement direction is detected by the movement detection unit, and then, when the operation reception unit no longer receives an operation, Stopping the first display change;
The display device according to claim 3 or 4, characterized by the above-mentioned. The control unit executes the second display change when the movement detecting unit detects a movement in the second movement direction in a state where the operation receiving unit is receiving an operation;
The display device according to claim 3, wherein: The operation receiving unit receives an operation different from the detection of the motion detection unit;
The display device according to claim 3, wherein: The operation device is a wearable device worn on the user's body;
The display device according to claim 1, wherein: A display device comprising: a display unit that allows a user to visually recognize an image; and an operation device that can be moved independently of the display unit and includes a motion detection unit that detects a motion of the operation device. Control
As processing for the image displayed by the display unit, the first display change corresponding to the movement in the first movement direction detected by the movement detection unit, and the movement in the second movement direction detected by the movement detection unit Perform a second display change corresponding to
When the movement in the first movement direction is detected, the first display change is started. After that, when the movement in the second movement direction is detected by the movement detection unit, the first display direction is changed. Stopping the display change,
A control method of a display device characterized by the above. A display device comprising: a display unit that allows a user to visually recognize an image; and an operation device that can be moved independently of the display unit and includes a motion detection unit that detects a motion of the operation device. A computer executable program that controls
As processing for the image displayed by the display unit, the first display change corresponding to the movement in the first movement direction detected by the movement detection unit, and the movement in the second movement direction detected by the movement detection unit Perform a second display change corresponding to
When the movement in the first movement direction is detected, the first display change is started. After that, when the movement in the second movement direction is detected by the movement detection unit, the first display direction is changed. A program for stopping display changes.

Images