JP6776578B2 - Input device, input method, computer program - Google Patents

Description

The present invention relates to an input device.

Flick input is known as one of the input methods in devices such as smartphones (see, for example, Patent Document 1). Flick input refers to all operations performed by quickly moving or flipping a finger on the touchpad. This flick input is known as a means for quickly realizing character input using, for example, a software keyboard.

Japanese Unexamined Patent Publication No. 2013-33395

In order to perform flick input, it is necessary for the input device to be provided with a sensor device such as a touch pad or a touch panel that can acquire the locus of a finger or the like. On the other hand, there are many input devices that do not have a sensor device that can acquire the trajectory of a finger or the like, such as a remote controller or a controller that has a cross key and some buttons. Conventionally, there has been a problem that flick input cannot be realized with such an input device.

Therefore, an input device capable of performing flick input without using a sensor device capable of acquiring a locus of a finger or the like has been desired.

The present invention has been made to solve at least a part of the above-mentioned problems, and can be realized as the following forms.
According to one embodiment of the present invention, an input device is provided. The input device includes two or more buttons that can individually turn inputs on and off, and a control unit that controls the input device. The control unit has a pattern in which the detection result of each button is predetermined. When is included, the output corresponding to the flick input is executed, when the predetermined pattern is not included, the detection result of each of the buttons is output as it is, and the predetermined pattern is the two or more buttons. of after detecting at least one time on and off of one of the buttons state, and are the detection of the on-the other buttons, the two or more buttons, a decision button, a direction corresponding to each of the vertical and horizontal The predetermined pattern, including the button, is that after detecting the on / off of the decision button, the on of any one of the direction buttons is detected .
According to this form of input device, in two or more buttons that can be individually turned on and off, if the detection result of each button includes a predetermined pattern, an output corresponding to a flick input is produced. Therefore, the user can perform flick input without using a sensor device capable of acquiring a trajectory such as a finger. Further, according to the input device of this form, when the enter button is pressed (or touched) and then any one of the direction buttons is pressed (or touched), in other words, the press / touch is sandwiched. As the finger moves, an output equivalent to a flick input is produced. As a result, the user can perform flick input by a finger movement motion that imitates an actual flick motion.
According to another embodiment of the present invention, an input device is provided. The input device includes two or more buttons that can individually turn inputs on and off, and a control unit that controls the input device. The control unit has a pattern in which the detection result of each button is predetermined. When is included, the output corresponding to the flick input is executed, when the predetermined pattern is not included, the detection result of each of the buttons is output as it is, and the predetermined pattern is the two or more buttons. Of these, the on / off of one button is detected at least once, and then the on / off of the other button is detected. The two or more buttons are the enter button and the direction buttons corresponding to the up / down / left / right directions. The predetermined pattern includes, and, after detecting the on / off of the decision button twice, the on of any one of the direction buttons is detected. According to this form of input device, in two or more buttons that can be individually turned on and off, if the detection result of each button includes a predetermined pattern, an output corresponding to a flick input is produced. Therefore, the user can perform flick input without using a sensor device capable of acquiring a trajectory such as a finger. Further, according to the input device of this form, the output corresponding to the flick input is produced only when the direction button is pressed (or touched) after the enter button is pressed (or touched) twice. Therefore, the user can properly use the normal decision-direction input and the flick input.

(1) According to one embodiment of the present invention, an input device is provided. The input device comprises; two or more buttons that can individually turn inputs on and off; and a control unit that controls the input device; the control unit has a predetermined pattern of detection results of each button. When is included, the output corresponding to the flick input is executed, and when the predetermined pattern is not included, the detection result of each button is output as it is.
According to this form of input device, in two or more buttons that can be individually turned on and off, if the detection result of each button includes a predetermined pattern, an output corresponding to a flick input is produced. Therefore, the user can perform flick input without using a sensor device capable of acquiring a trajectory such as a finger.

(2) In the input device of the above embodiment; the two or more buttons include a decision button and a direction button corresponding to each of the up, down, left, and right; the predetermined pattern; turns the decision button on and off. After the detection, it may be said that the on of any one of the direction buttons is detected.
According to this form of input device, when the enter button is pressed (or touched) and then any one of the direction buttons is pressed (or touched), in other words, the finger holding the press / touch. As the movement of, the output corresponding to the flick input is made. As a result, the user can perform flick input by a finger movement motion that imitates an actual flick motion.

(3) In the input device of the above-described embodiment; the predetermined pattern; after detecting the on / off of the determination button, any one of the direction buttons is turned on within a predetermined first time. May be detected.
According to the input device of this form, the output corresponding to the flick input is produced only when the pressing (or touch) of the enter button and the pressing (or touch) of the direction button are within the first time. .. By appropriately setting this first time, the user can properly use the normal decision-direction input and the flick input.

(4) In the input device of the above-described embodiment; the predetermined pattern; after detecting the on of the decision button, the off of the decision button is detected after a predetermined second time has elapsed, and then the decision button is detected. It may be that the on of any one of the direction buttons is detected.
According to the input device of this form, the output corresponding to the flick input is made only when the period from the start of pressing (or touching) the enter button to the end of pressing (or touching) of the enter button is longer than the second time. To. By appropriately setting this second time, the user can consciously use the normal decision-direction input and the flick input properly by pressing and holding the button.

(5) In the input device of the above form; the two or more buttons include a decision button and a direction button corresponding to each of the up, down, left, and right; the predetermined pattern; turns the decision button on and off. It may be that after the detection is performed twice, the on of any one of the direction buttons is detected.
According to this form of input device, an output corresponding to a flick input is produced only when the direction button is pressed (or touched) after the enter button is pressed (or touched) twice. Therefore, the user can properly use the normal decision-direction input and the flick input.

(6) In the input device of the above embodiment; the two or more buttons may be cross keys in which the direction buttons are arranged in the vertical and horizontal directions with the decision button as the center.
According to this form of input device, the enter button and the direction button can be configured as a cross key that is easy for the user to intuitively grasp and operate.

(7) In the input device of the above form; each of the two or more buttons may be a mechanical contact type button or a capacitance type button.
According to this form of input device, the input device can be configured by using either a mechanical contact type button or a capacitance type button.

(8) In the input device of the above-described embodiment, the control unit further includes an interface for connecting to an image display unit which is attached to the head of the user to make the user visually recognize a virtual image. It includes a function of generating an image light and outputting the generated image light to the image display unit; an operating system capable of executing various applications; the control unit outputs an output corresponding to the flick input. Alternatively, the detection result may be output to at least one of the application and the operating system.
In this form of input device, the input device can be configured as a controller that operates and controls the image display device.

(9) In the input device of the above form; the control unit further includes an operating system capable of executing various applications; the control unit produces an output corresponding to the flick input or the detection result. It may be output to at least one of the application and the operating system.
In this form of input device, the input device can be configured as a computer capable of executing various applications.

(10) In the input device of the above form, further; an interface for connecting to an external device; the control unit sends an output corresponding to the flick input or the detection result to the external device via the interface. You may output it.
In this form of input device, the input device can be configured as a device that inputs to an external device.

The plurality of components of each form of the present invention described above are not all essential, and may be used to solve some or all of the above-mentioned problems, or part or all of the effects described herein. In order to achieve the above, it is possible to change or delete some of the components of the plurality of components, replace them with new components, and partially delete the limited contents. Further, in order to solve a part or all of the above-mentioned problems, or to achieve a part or all of the effects described in the present specification, the technical features included in the above-mentioned embodiment of the present invention. It is also possible to combine some or all with some or all of the technical features contained in the other embodiments of the invention described above to form an independent embodiment of the invention.

The present invention can be realized in various aspects, for example, an input device, a head-mounted display device combining an input device and an image display unit, and an input device and a head-mounted display device. A system that combines input devices and an information processing device, an input method, a computer program that realizes the functions of these devices, systems, and methods, a server device that distributes the computer program, and the computer. It can be realized in the form of a storage medium or the like that stores the program.

It is a figure which shows the appearance structure of the HMD in one Embodiment of this invention. It is a main part plan view which shows the structure of the optical system provided in the image display part. It is a figure which shows the main part structure of the image display part seen from the user. It is a figure for demonstrating the angle of view of a camera. It is a block diagram which shows the structure of HMD functionally. It is a block diagram which shows the structure of the control function part of an HMD functionally. It is a figure explaining the cross key. It is a figure explaining the method of flick input using a cross key. It is a figure which shows the example of the operation pattern which the input / output control part outputs corresponding to a flick input. It is a figure explaining operation of a pattern. It is a figure explaining operation of a pattern. It is a figure explaining operation of a pattern. It is a figure explaining operation of a pattern and operation. It is a figure explaining another structure of a cross key. It is a figure explaining another structure of a cross key. It is a figure explaining another structure of a cross key. It is a main part plan view which shows the structure of the optical system provided in the image display part of the modification.

A. Embodiment:
A-1. HMD configuration:
FIG. 1 is a diagram showing an external configuration of an HMD (Head Mounted Display) 100 according to an embodiment of the present invention. The HMD 100 includes a display unit 20 (display unit) that allows the user to visually recognize a virtual image while being mounted on the user's head, and a control device 10 (control unit) that controls the image display unit 20. It is a device. The control device 10 includes various input devices for acquiring the user's operation, and functions as a controller for the user to operate the HMD 100. Further, the control device 10 of the present embodiment also functions as an "input device" capable of performing flick input without using a sensor device capable of acquiring a locus of a finger or the like.

The image display unit 20 is a wearer that is worn on the user's head, and has a spectacle shape in the present embodiment. The image display unit 20 has a right display unit 21, a left display unit 24, a right light guide plate 26, and a left light guide plate 28 in a main body having a right holding unit 21, a left holding unit 23, and a front frame 27. And.

The right holding portion 21 and the left holding portion 23 extend rearward from both ends of the front frame 27, respectively, and hold the image display portion 20 on the user's head like a temple (vine) of eyeglasses. Here, of both ends of the front frame 27, the end located on the right side of the user when the image display unit 20 is attached is referred to as the end ER, and the end located on the left side of the user is referred to as the end EL. To do. The right holding portion 21 extends from the end ER of the front frame 27 to a position corresponding to the right head of the user in the mounted state of the image display portion 20. The left holding portion 23 is provided extending from the end EL of the front frame 27 to a position corresponding to the left head of the user in the mounted state of the image display portion 20.

The right light guide plate 26 and the left light guide plate 28 are provided on the front frame 27. The right light guide plate 26 is located in front of the user's right eye when the image display unit 20 is attached, and allows the right eye to visually recognize the image. The left light guide plate 28 is located in front of the left eye of the user when the image display unit 20 is attached, and allows the left eye to visually recognize the image.

The front frame 27 has a shape in which one end of the right light guide plate 26 and one end of the left light guide plate 28 are connected to each other. This connection position corresponds to the position between the user's eyebrows when the image display unit 20 is worn. The front frame 27 may be provided with a nose pad that comes into contact with the user's nose when the image display unit 20 is attached at the connection position between the right light guide plate 26 and the left light guide plate 28. In this case, the image display unit 20 can be held on the user's head by the nose pad portion, the right holding portion 21, and the left holding portion 23. Further, a belt in contact with the back of the head of the user may be connected to the right holding portion 21 and the left holding portion 23 while the image display unit 20 is attached. In this case, the image display unit 20 can be firmly held on the user's head by the belt.

The right display unit 22 displays an image by the right light guide plate 26. The right display unit 22 is provided in the right holding unit 21 and is located near the right head of the user in the mounted state of the image display unit 20. The left display unit 24 displays an image by the left light guide plate 28. The left display unit 24 is provided on the left holding unit 23 and is located near the left side head of the user when the image display unit 20 is attached. The right display unit 22 and the left display unit 24 are also collectively referred to as a "display drive unit".

The right light guide plate 26 and the left light guide plate 28 of the present embodiment are optical portions (for example, prisms) formed of a light-transmitting resin or the like, and use the image light output by the right display unit 22 and the left display unit 24. Lead to the eyes of others. A dimming plate may be provided on the surfaces of the right light guide plate 26 and the left light guide plate 28. The dimmer is a thin plate-shaped optical element having different transmittance depending on the wavelength range of light, and functions as a so-called wavelength filter. The dimmer is arranged so as to cover, for example, the surface of the front frame 27 (the surface opposite to the surface facing the user's eyes). By appropriately selecting the optical characteristics of the dimmer, the transmittance of light in any wavelength range such as visible light, infrared light, and ultraviolet light can be adjusted, and the right light guide plate 26 and left guide can be adjusted from the outside. The amount of external light incident on the light plate 28 and transmitted through the right light guide plate 26 and the left light guide plate 28 can be adjusted.

The image display unit 20 guides the image light generated by the right display unit 22 and the left display unit 24 to the right light guide plate 26 and the left light guide plate 28, respectively, and makes the user visually recognize the virtual image by the image light. Also called "display an image"). When external light enters the user's eyes through the right light guide plate 26 and the left light guide plate 28 from the front of the user, the image light constituting the virtual image and the external light are incident on the user's eyes. To do. Therefore, the visibility of the virtual image in the user is affected by the intensity of external light.

Therefore, for example, by mounting a dimmer on the front frame 27 and appropriately selecting or adjusting the optical characteristics of the dimmer, the ease of viewing the virtual image can be adjusted. In a typical example, it is possible to select a dimmer plate having at least a light transmittance that allows a user wearing the HMD 100 to visually recognize the outside scenery. When the dimming plate is used, the effect of protecting the right light guide plate 26 and the left light guide plate 28 and suppressing damage to the right light guide plate 26 and the left light guide plate 28 and adhesion of dirt can be expected. The light control plate may be detachable from the front frame 27 or the right light guide plate 26 and the left light guide plate 28, respectively. Further, a plurality of types of dimmers may be exchanged so that they can be attached and detached, and the dimmers may be omitted.

The camera 61 is arranged in the front frame 27 of the image display unit 20. The camera 61 is provided on the front surface of the front frame 27 at a position that does not block the external light transmitted through the right light guide plate 26 and the left light guide plate 28. In the example of FIG. 1, the camera 61 is arranged on the end ER side of the front frame 27. The camera 61 may be arranged on the end EL side of the front frame 27, or may be arranged at the connecting portion between the right light guide plate 26 and the left light guide plate 28.

The camera 61 is a digital camera including an image pickup element such as a CCD or CMOS, an image pickup lens, or the like. Although the camera 61 of the present embodiment is a monocular camera, a stereo camera may be adopted. The camera 61 captures at least a part of the outside view (real space) in the front side direction of the HMD 100, in other words, in the field of view direction visually recognized by the user when the image display unit 20 is attached. In other words, the camera 61 captures a range or direction that overlaps with the user's field of view, and captures the direction that the user sees. The width of the angle of view of the camera 61 can be set as appropriate. In the present embodiment, the wide angle of view of the camera 61 is set so that the user can image the entire field of view of the user who can see through the right light guide plate 26 and the left light guide plate 28. The camera 61 executes imaging under the control of the control unit 150 (FIG. 5), and outputs the obtained imaging data to the control unit 150.

The HMD 100 may include a distance measuring sensor that detects a distance to a measurement object located in a preset measurement direction. The distance measuring sensor can be arranged, for example, at the connecting portion between the right light guide plate 26 and the left light guide plate 28 of the front frame 27. The measurement direction of the distance measuring sensor can be the front side direction of the MD 100 (the direction overlapping the imaging direction of the camera 61). The distance measuring sensor can be composed of, for example, a light emitting unit such as an LED or a laser diode, and a light receiving unit that receives the reflected light that the light emitted from the light source reflects on the object to be measured. In this case, the distance is obtained by triangular distance measurement processing or distance measurement processing based on the time difference. The distance measuring sensor may be composed of, for example, a transmitting unit that emits ultrasonic waves and a receiving unit that receives ultrasonic waves reflected by the object to be measured. In this case, the distance is obtained by distance measurement processing based on the time difference. Like the camera 61, the distance measuring sensor is controlled by the control unit 150 and outputs the detection result to the control unit 150.

FIG. 2 is a plan view of a main part showing the configuration of the optical system included in the image display unit 20. For convenience of explanation, FIG. 2 illustrates the user's right eye RE and left eye LE. As shown in FIG. 2, the right display unit 22 and the left display unit 24 are symmetrically configured.

The right display unit 22 includes an OLED (Organic Light Emitting Diode) unit 221 and a right optical system 251 so that the right eye RE can visually recognize the virtual image. The OLED unit 221 emits image light. The right optical system 251 includes a lens group and the like, and guides the image light L emitted by the OLED unit 221 to the right light guide plate 26.

The OLED unit 221 has an OLED panel 223 and an OLED drive circuit 225 for driving the OLED panel 223. The OLED panel 223 is a self-luminous display panel composed of light emitting elements that emit light by organic electroluminescence and emit R (red), G (green), and B (blue) colored lights, respectively. In the OLED panel 223, a plurality of pixels having a unit including one R, G, and B element as one pixel are arranged in a matrix.

The OLED drive circuit 225 selects and energizes the light emitting element included in the OLED panel 223 according to the control of the control unit 150 (FIG. 5), and causes the light emitting element to emit light. The OLED drive circuit 225 is fixed to the back surface of the OLED panel 223, that is, the back side of the light emitting surface by bonding or the like. The OLED drive circuit 225 is composed of, for example, a semiconductor device that drives the OLED panel 223, and may be mounted on a substrate fixed to the back surface of the OLED panel 223. A temperature sensor 217, which will be described later, is mounted on this substrate. The OLED panel 223 may adopt a configuration in which light emitting elements that emit white light are arranged in a matrix, and color filters corresponding to each of the R, G, and B colors are arranged in an overlapping manner. Further, an OLED panel 223 having a WRGB configuration including a light emitting element that emits W (white) light in addition to a light emitting element that emits R, G, and B colored light may be adopted.

The right optical system 251 has a collimating lens that converts the image light L emitted from the OLED panel 223 into a light flux in a parallel state. The image light L converted into a light flux in a parallel state by the collimating lens is incident on the right light guide plate 26. A plurality of reflecting surfaces that reflect the image light L are formed in the optical path that guides the light inside the right light guide plate 26. The image light L is guided to the right eye RE side through a plurality of reflections inside the right light guide plate 26. A half mirror 261 (reflection surface) located in front of the right eye RE is formed on the right light guide plate 26. After being reflected by the half mirror 261, the image light L is emitted from the right light guide plate 26 to the right eye RE, and the image light L forms an image on the retina of the right eye RE so that the user can visually recognize the virtual image.

The left display unit 24 includes an OLED unit 241 and a left optical system 252 as a configuration for allowing the left eye LE to visually recognize a virtual image. The OLED unit 241 emits image light. The left optical system 252 includes a lens group and the like, and guides the image light L emitted by the OLED unit 241 to the left light guide plate 28. The OLED unit 241 includes an OLED panel 243 and an OLED drive circuit 245 that drives the OLED panel 243. The details of each part are the same as those of the OLED unit 221 and the OLED panel 223, and the OLED drive circuit 225. The temperature sensor 239 is mounted on the substrate fixed to the back surface of the OLED panel 243. The details of the left optical system 252 are the same as those of the right optical system 251.

According to the configuration described above, the HMD 100 can function as a see-through type display device. That is, the image light L reflected by the half mirror 261 and the external light OL transmitted through the right light guide plate 26 are incident on the user's right eye RE. The image light L reflected by the half mirror 281 and the external light OL transmitted through the left light guide plate 28 are incident on the left eye LE of the user. In this way, the HMD 100 superimposes the image light L of the internally processed image and the external light OL and causes them to enter the user's eye. As a result, the user can see the outside view (real world) through the right light guide plate 26 and the left light guide plate 28, and can visually recognize the virtual image by the image light L so as to overlap the outside view.

The half mirror 261 and the half mirror 281 function as an "image extraction unit" that reflects the image light output by the right display unit 22 and the left display unit 24 to extract an image. Further, the right optical system 251 and the right light guide plate 26 are collectively referred to as a “right light guide unit”, and the left optical system 252 and the left light guide plate 28 are collectively referred to as a “left light guide unit”. The configuration of the right light guide unit and the left light guide unit is not limited to the above-mentioned example, and any method can be used as long as an image light is used to form a virtual image in front of the user's eyes. For example, a diffraction grating may be used for the right light guide portion and the left light guide portion, or a semitransmissive reflective film may be used.

In FIG. 1, the control device 10 and the image display unit 20 are connected by a connection cable 40. The connection cable 40 is detachably connected to a connector provided at the bottom of the control device 10, and is connected to various circuits inside the image display unit 20 from the tip of the left holding unit 23. The connection cable 40 includes a metal cable or an optical fiber cable for transmitting digital data. The connection cable 40 may further include a metal cable for transmitting analog data. A connector 46 is provided in the middle of the connection cable 40.

The connector 46 is a jack for connecting a stereo mini plug, and the connector 46 and the control device 10 are connected by, for example, a line for transmitting an analog audio signal. In the example of the present embodiment shown in FIG. 1, the right earphone 32 and the left earphone 34 constituting the stereo headphones and the headset 30 having the microphone 63 are connected to the connector 46.

As shown in FIG. 1, for example, the microphone 63 is arranged so that the sound collecting portion of the microphone 63 faces the line-of-sight direction of the user. The microphone 63 collects sound and outputs a voice signal to the voice interface 182 (FIG. 5). The microphone 63 may be a monaural microphone or a stereo microphone, and may be a directional microphone or an omnidirectional microphone.

The control device 10 includes a button 11, an LED indicator 12, a cross key 13, a track pad 14, an up / down key 15, a changeover switch 16, and a power switch 18 as input / output interfaces for the user.

The button 11 includes a menu key, a home key, a back key, and the like for operating the OS (operating system) 143 (FIG. 6) executed by the control device 10. The button 11 of the present embodiment is a type of button that is displaced by a pressing operation among these keys and switches. The LED indicator 12 turns on or off according to the operating state of the HMD 100. The cross key 13 detects a touch or pressing operation on the keys corresponding to the up / down / left / right and the center portion, respectively, and outputs a signal according to the detected content. The trackpad 14 has an operation surface for detecting a contact operation, detects a contact operation with the operation surface, and outputs a signal according to the detected content. As the contact operation detection method, various methods such as electrostatic type, pressure detection type, and optical type can be adopted.

The up / down key 15 detects an instruction operation for increasing / decreasing the volume output from the right earphone 32 and the left earphone 34, and an instruction operation for increasing / decreasing the brightness in the image display unit 20. The changeover switch 16 detects a change operation of the referent (volume, brightness) of the up / down keys 15. The power switch 18 detects an on / off (ON / OFF) switching operation of the power supply of the HMD 100. As the power switch 18, for example, a slide switch can be adopted.

FIG. 3 is a diagram showing a configuration of a main part of the image display unit 20 as seen from the user. In FIG. 3, the connection cable 40, the right earphone 32, and the left earphone 34 are not shown. In the state of FIG. 3, the back side of the right light guide plate 26 and the left light guide plate 28 can be visually recognized, and the half mirror 261 for irradiating the right eye RE with image light and the left eye LE for irradiating the image light. The half mirror 281 can be visually recognized as a substantially quadrangular area. The user visually recognizes the outside view through the entire left and right light guide plates 26 and 28 including the half mirrors 261 and 281 and visually recognizes a rectangular display image at the positions of the half mirrors 261 and 281.

FIG. 4 is a diagram for explaining the angle of view of the camera 61. In FIG. 4, the camera 61 and the user's right eye RE and left eye LE are schematically shown in a plan view, and the angle of view (imaging range) of the camera 61 is shown by C. The angle of view C of the camera 61 is widened in the horizontal direction as shown in the drawing, and is also widened in the vertical direction like a general digital camera.

As described above, the camera 61 is arranged at the right end of the image display unit 20 and images the direction of the user's line of sight (that is, in front of the user). Therefore, the optical axis of the camera 61 is a direction including the line-of-sight directions of the right eye RE and the left eye LE. The external view that the user can see while wearing the HMD100 is not always infinity. For example, when the user gazes at the object OB with both eyes, the user's line of sight is directed to the object OB as shown by the symbols RD and LD in the figure. In this case, the distance from the user to the object OB is often about 30 cm to 10 m, and more often 1 m to 4 m. Therefore, for the HMD 100, a guideline for the upper limit and the lower limit of the distance from the user to the object OB during normal use may be set. This guideline may be obtained in advance and preset in the HMD 100, or may be set by the user. The optical axis and angle of view of the camera 61 are set so that the object OB is included in the angle of view when the distance to the object OB during such normal use corresponds to the set upper limit and lower limit. It is preferable to be done.

Generally, the viewing angle of a human is about 200 degrees in the horizontal direction and about 125 degrees in the vertical direction. Among them, the effective visual field having excellent information receiving ability is about 30 degrees in the horizontal direction and about 20 degrees in the vertical direction. The stable gazing field at which a human gaze can be seen quickly and stably is 60 to 90 degrees in the horizontal direction and 45 to 70 degrees in the vertical direction. In this case, when the gazing point is the object OB (FIG. 4), the effective field of view is about 30 degrees in the horizontal direction and about 20 degrees in the vertical direction with the lines of sight RD and LD as the center. The stable gaze is 60 to 90 degrees in the horizontal direction and 45 to 70 degrees in the vertical direction. The actual field of view that the user sees through the image display unit 20 and through the right light guide plate 26 and the left light guide plate 28 is called an actual field of view (FOV). The actual field of view is narrower than the viewing angle and stable gaze, but wider than the effective field of view.

The angle of view C of the camera 61 of the present embodiment is set so that a range wider than the field of view of the user can be imaged. The angle of view C of the camera 61 is preferably set so that at least a range wider than the user's effective field of view can be imaged, and more preferably a range wider than the actual field of view can be imaged. The angle of view C of the camera 61 is more preferably set so as to be able to image a wider range than the user's stable gazing field, and is set to be able to image a wider range than the viewing angle of both eyes of the user. Most preferred. Therefore, the camera 61 may be provided with a so-called wide-angle lens as an image pickup lens, and may be configured to be able to image a wide angle of view. The wide-angle lens may include a lens called an ultra-wide-angle lens or a quasi-wide-angle lens. Further, the camera 61 may include a single focus lens, a zoom lens, or a lens group including a plurality of lenses.

FIG. 5 is a block diagram functionally showing the configuration of the HMD 100. The control device 10 includes a main processor 140 that executes a program to control the HMD 100, a storage unit, an input / output unit, sensors, an interface, and a power supply unit 130. These storage units, input / output units, sensors, interfaces, and power supply units 130 are connected to the main processor 140, respectively. The main processor 140 is mounted on the controller board 120 in which the control device 10 is built.

The storage unit includes a memory 118 and a non-volatile storage unit 121. The memory 118 constitutes a work area for temporarily storing a computer program executed by the main processor 140 and data to be processed. The non-volatile storage unit 121 is composed of a flash memory and an eMMC (embedded Multi Media Card). The non-volatile storage unit 121 stores a computer program executed by the main processor 140 and various data processed by the main processor 140. In this embodiment, these storage units are mounted on the controller board 120.

The input / output unit includes a track pad 14 and an operation unit 110. The operation unit 110 includes the button 11, the LED indicator 12, the cross key 13, the track pad 14, the up / down key 15, the changeover switch 16, and the power switch 18. The main processor 140 controls each of these input / output units and acquires signals output from each input / output unit.

The sensors include a 6-axis sensor 111, a magnetic sensor 113, and a GPS (Global Positioning System) 115. The 6-axis sensor 111 is a motion sensor (inertia sensor) including a 3-axis acceleration sensor and a 3-axis gyro (angular velocity) sensor. The 6-axis sensor 111 may employ an IMU (Inertial Measurement Unit) in which these sensors are modularized. The magnetic sensor 113 is, for example, a three-axis geomagnetic sensor. The GPS 115 includes a GPS antenna (not shown), receives a radio signal transmitted from a GPS satellite, and detects the coordinates of the current position of the control device 10. These sensors (6-axis sensor 111, magnetic sensor 113, GPS 115) output the detected value to the main processor 140 according to a sampling frequency specified in advance. The timing at which each sensor outputs the detected value may be in response to an instruction from the main processor 140.

The interface includes a communication unit 117, a voice codec 180, an external connector 184, an external memory interface 186, a USB (Universal Serial Bus) connector 188, a sensor hub 192, an FPGA 194, and an interface 196. There is. These serve as interfaces to the outside world. The communication unit 117 executes wireless communication between the HMD 100 and the external device. The communication unit 117 is configured to include an antenna (not shown), an RF circuit, a baseband circuit, a communication control circuit, and the like, or is configured as a device in which these are integrated. The communication unit 117 performs wireless communication conforming to standards such as a wireless LAN including Bluetooth (registered trademark) and Wi-Fi (registered trademark).

The voice codec 180 is connected to the voice interface 182 and encodes / decodes a voice signal input / output via the voice interface 182. The voice interface 182 is an interface for inputting / outputting a voice signal. The voice codec 180 may include an A / D converter that converts an analog voice signal into digital voice data, and a D / A converter that performs the reverse conversion. The HMD 100 of the present embodiment outputs sound from the right earphone 32 and the left earphone 34, and collects the sound by the microphone 63. The voice codec 180 converts the digital voice data output by the main processor 140 into an analog voice signal and outputs it via the voice interface 182. Further, the voice codec 180 converts the analog voice signal input to the voice interface 182 into digital voice data and outputs it to the main processor 140.

The external connector 184 is a connector for connecting an external device (for example, a personal computer, a smartphone, a game device, etc.) that communicates with the main processor 140 to the main processor 140. The external device connected to the external connector 184 can be used as a source of contents, debug a computer program executed by the main processor 140, and collect an operation log of the HMD 100. The external connector 184 can adopt various aspects. As the external connector 184, for example, an interface corresponding to a wired connection such as a USB interface, a micro USB interface, and a memory card interface, or an interface corresponding to a wireless connection such as a wireless LAN interface and a Bluetooth interface can be adopted.

The external memory interface 186 is an interface to which a portable memory device can be connected. The external memory interface 186 includes, for example, a memory card slot in which a card-type recording medium is attached to read / write data, and an interface circuit. The size, shape, standard, etc. of the card-type recording medium can be appropriately selected. The USB connector 188 is an interface to which a memory device, a smartphone, a personal computer, or the like conforming to the USB standard can be connected. The USB connector 188 includes, for example, a connector conforming to the USB standard and an interface circuit. The size, shape, USB standard version, etc. of the USB connector 188 can be appropriately selected.

The HMD 100 also includes a vibrator 19. The vibrator 19 includes a motor (not shown), an eccentric rotor, and the like, and emits vibration under the control of the main processor 140. The HMD 100 generates vibration by the vibrator 19 in a predetermined vibration pattern, for example, when an operation on the operation unit 110 is detected or when the power supply of the HMD 100 is turned on / off.

The sensor hub 192 and the FPGA 194 are connected to the image display unit 20 via the interface (I / F) 196. The sensor hub 192 acquires the detection values of various sensors included in the image display unit 20 and outputs them to the main processor 140. The FPGA 194 executes processing of data transmitted and received between the main processor 140 and each unit of the image display unit 20 and transmission via the interface 196. The interface 196 is connected to the right display unit 22 and the left display unit 24 of the image display unit 20, respectively. In the example of the present embodiment, the connection cable 40 is connected to the left holding unit 23, the wiring connected to the connection cable 40 is laid inside the image display unit 20, and the right display unit 22 and the left display unit 24 are respectively. It is connected to the interface 196 of the control device 10.

The power supply unit 130 includes a battery 132 and a power supply control circuit 134. The power supply unit 130 supplies electric power for operating the control device 10. The battery 132 is a rechargeable battery. The power supply control circuit 134 detects the remaining capacity of the battery 132 and controls the charging of the OS 143. The power supply control circuit 134 is connected to the main processor 140, and outputs the detected value of the remaining capacity of the battery 132 and the detected value of the voltage of the battery 132 to the main processor 140. The electric power may be supplied from the control device 10 to the image display unit 20 based on the electric power supplied by the power supply unit 130. The power supply state from the power supply unit 130 to each unit of the control device 10 and the image display unit 20 may be controlled by the main processor 140.

The right display unit 22 includes a display unit board 210, an OLED unit 221, a camera 61, an illuminance sensor 65, an LED indicator 67, and a temperature sensor 217. An interface (I / F) 211 connected to the interface 196, a receiving unit (Rx) 213, and an EEPROM (Electrically Erasable Programmable Read-Only Memory) 215 are mounted on the display unit board 210. The receiving unit 213 receives the data input from the control device 10 via the interface 211. When the receiving unit 213 receives the image data of the image displayed by the OLED unit 221, the receiving unit 213 outputs the received image data to the OLED drive circuit 225 (FIG. 2).

The EEPROM 215 stores various data in a manner readable by the main processor 140. The EEPROM 215 stores, for example, data on the light emission characteristics and display characteristics of the OLED units 221 and 241 of the image display unit 20, data on the sensor characteristics of the right display unit 22 or the left display unit 24, and the like. Specifically, for example, parameters related to gamma correction of the OLED units 221 and 241 and data for compensating for the detected values of the temperature sensors 217 and 239 described later are stored. These data are generated by the factory inspection of the HMD100 and written to the EEPROM 215. After shipment, the main processor 140 reads the data of the EEPROM 215 and uses it for various processes.

The camera 61 executes imaging according to a signal input via the interface 211, and outputs captured image data or a signal representing the imaging result to the control device 10. As shown in FIG. 1, the illuminance sensor 65 is provided at the end ER of the front frame 27 and is arranged so as to receive external light from the front of the user who wears the image display unit 20. The illuminance sensor 65 outputs a detected value corresponding to the amount of received light (light receiving intensity). As shown in FIG. 1, the LED indicator 67 is arranged in the vicinity of the camera 61 at the end ER of the front frame 27. The LED indicator 67 lights up during the image pickup by the camera 61 to notify that the image pickup is in progress.

The temperature sensor 217 detects the temperature and outputs a voltage value or a resistance value corresponding to the detected temperature. The temperature sensor 217 is mounted on the back surface side of the OLED panel 223 (FIG. 3). The temperature sensor 217 may be mounted on the same substrate as the OLED drive circuit 225, for example. With this configuration, the temperature sensor 217 mainly detects the temperature of the OLED panel 223. The temperature sensor 217 may be built in the OLED panel 223 or the OLED drive circuit 225. For example, when the OLED panel 223 is mounted as a Si-OLED together with the OLED drive circuit 225 as an integrated circuit on an integrated semiconductor chip, the temperature sensor 217 may be mounted on the semiconductor chip.

The left display unit 24 includes a display unit board 230, an OLED unit 241 and a temperature sensor 239. An interface (I / F) 231 connected to the interface 196, a receiving unit (Rx) 233, a 6-axis sensor 235, and a magnetic sensor 237 are mounted on the display unit board 230. The receiving unit 233 receives the data input from the control device 10 via the interface 231. When the receiving unit 233 receives the image data of the image displayed by the OLED unit 241, the receiving unit 233 outputs the received image data to the OLED drive circuit 245 (FIG. 2).

The 6-axis sensor 235 is a motion sensor (inertia sensor) including a 3-axis acceleration sensor and a 3-axis gyro (angular velocity) sensor. The 6-axis sensor 235 may employ an IMU in which the above sensor is modularized. The magnetic sensor 237 is, for example, a three-axis geomagnetic sensor. The temperature sensor 239 detects the temperature and outputs a voltage value or a resistance value corresponding to the detected temperature. The temperature sensor 239 is mounted on the back surface side of the OLED panel 243 (FIG. 3). The temperature sensor 239 may be mounted on the same substrate as the OLED drive circuit 245, for example. With this configuration, the temperature sensor 239 mainly detects the temperature of the OLED panel 243. The temperature sensor 239 may be built in the OLED panel 243 or the OLED drive circuit 245. The details are the same as those of the temperature sensor 217.

The camera 61, the illuminance sensor 65, and the temperature sensor 217 of the right display unit 22, and the 6-axis sensor 235, the magnetic sensor 237, and the temperature sensor 239 of the left display unit 24 are connected to the sensor hub 192 of the control device 10. The sensor hub 192 sets and initializes the sampling cycle of each sensor according to the control of the main processor 140. The sensor hub 192 executes energization of each sensor, transmission of control data, acquisition of detected values, and the like in accordance with the sampling cycle of each sensor. The sensor hub 192 outputs the detection value of each sensor included in the right display unit 22 and the left display unit 24 to the main processor 140 at a preset timing. The sensor hub 192 may have a cache function that temporarily holds the detected value of each sensor. The sensor hub 192 may include a signal format or data format conversion function (for example, a conversion function to a unified format) of the detection value of each sensor. The sensor hub 192 turns on or off the LED indicator 67 by starting and stopping the energization of the LED indicator 67 according to the control of the main processor 140.

FIG. 6 is a block diagram functionally showing the configuration of the control function unit of the HMD 100. The control function unit of the HMD 100 includes a storage unit 122 and a control unit 150. The storage unit 122 is a logical storage unit composed of the non-volatile storage unit 121 (FIG. 5). Although FIG. 6 shows an example in which only the storage unit 122 is used, the EEPROM 215 or the memory 118 may be used in combination with the non-volatile storage unit 121. Further, the control unit 150 is configured by the main processor 140 executing a computer program, that is, by the hardware and software collaborating with each other.

The storage unit 122 stores various data to be processed by the control unit 150. Specifically, the storage unit 122 of the present embodiment stores the setting data 123 and the content data 124. The setting data 123 includes various setting values related to the operation of the HMD 100. For example, the setting data 123 includes parameters, a determinant, an arithmetic expression, a LUT (Look Up Table), and the like when the control unit 150 controls the HMD 100.

The content data 124 includes content data (image data, video data, audio data, etc.) including an image and a video displayed by the image display unit 20 under the control of the control unit 150. The content data 124 may include bidirectional content data. The bidirectional content means that the control device 10 acquires an operation of the user, the control unit 150 executes a process according to the acquired operation content, and the content according to the process content is displayed on the image display unit 20. Means the type of content that you want to do. In this case, the content data may include image data of the menu screen for acquiring the operation of the user, data for defining the process corresponding to the item included in the menu screen, and the like.

The control unit 150 serves as the OS 143, the image processing unit 145, the display control unit 147, the image pickup control unit 149, and the input / output control unit 151 by executing various processes using the data stored in the storage unit 122. Perform the function. In this embodiment, each functional unit other than OS143 is configured as an application program executed on OS143.

The image processing unit 145 generates a signal to be transmitted to the right display unit 22 and the left display unit 24 based on the image data of the image / video displayed by the image display unit 20. The signal generated by the image processing unit 145 may be a vertical synchronization signal, a horizontal synchronization signal, a clock signal, an analog image signal, or the like. The image processing unit 145 may be configured by a hardware other than the main processor 140 (for example, a DSP (Digital Signal Processor)) in addition to the configuration realized by the main processor 140 executing a computer program.

The image processing unit 145 may execute resolution conversion processing, image adjustment processing, 2D / 3D conversion processing, and the like, if necessary. The resolution conversion process is a process of converting the resolution of the image data into a resolution suitable for the right display unit 22 and the left display unit 24. The image adjustment process is a process for adjusting the brightness and saturation of image data. The 2D / 3D conversion process is a process of generating two-dimensional image data from three-dimensional image data or generating three-dimensional image data from two-dimensional image data. When these processes are executed, the image processing unit 145 generates a signal for displaying an image based on the processed image data and transmits the signal to the image display unit 20 via the connection cable 40.

The display control unit 147 generates a control signal for controlling the right display unit 22 and the left display unit 24, and controls the generation and emission of image light by each of the right display unit 22 and the left display unit 24 by the control signal. To do. Specifically, the display control unit 147 controls the OLED drive circuits 225 and 245 to display an image on the OLED panels 223 and 243. The display control unit 147 controls the timing at which the OLED drive circuits 225 and 245 draw on the OLED panels 223 and 243, and controls the brightness of the OLED panels 223 and 243, based on the signal output by the image processing unit 145.

The image pickup control unit 149 controls the camera 61 to execute image pickup, generate captured image data, and temporarily store it in the storage unit 122. When the camera 61 is configured as a camera unit including a circuit for generating captured image data, the imaging control unit 149 acquires the captured image data from the camera 61 and temporarily stores the captured image data in the storage unit 122.

The input / output control unit 151 acquires the on / off detection result of each button on the cross key 13. The input / output control unit 151 executes an output corresponding to a flick input when the detection result includes a predetermined pattern (details will be described later), and when the detection result does not include a predetermined pattern, the on / off detection result. Is output as it is. Here, the output destination of the flick input or on / off detection result can be arbitrarily determined, but in the case of this embodiment, it is OS143. The input / output control unit 151 may output the flick input or on / off detection result to another application running on the OS 143 instead of the OS 143 or together with the OS 143. The other application running on the OS 143 is a display control unit 147, an imaging control unit 149, or another application installed on the OS 143.

The input / output control unit 151 may output the flick input or on / off detection result to the image display unit 20 connected via the interface 196, or output to an external device connected via the external connector 184. You may. In this case, the flick input or on / off detection result is used by the control unit mounted on the image display unit 20 or the control unit of the external device.

A-2. Cross key configuration:
FIG. 7 is a diagram illustrating the cross key 13. In the HMD 100 of the present embodiment, flick input can be performed using the cross key 13. The cross key 13 of the present embodiment is composed of one decision button 139 and four direction buttons 135 to 138 corresponding to each of the up, down, left, and right directions. The enter button 139 has a circular shape, and the direction buttons 135 to 138 have a rectangular shape. These buttons are in the form of a so-called cross key in which four direction buttons 135 to 138 are arranged in the vertical and horizontal directions around the decision button 139 on the housing surface of the control device 10. Hereinafter, when the four direction buttons 135 to 138 are collectively described, they are simply referred to as "direction buttons", and when they are individually described, they are referred to as a left button 135, a right button 136, an upper button 137, and a lower button 138. In FIG. 7, the characters for determination, up, down, left, and right are shown for convenience, but the actual cross key 13 may not have these indications and may be replaced with symbols or the like.

The decision button 139 of the present embodiment is a switch provided internally when the user presses the decision button 139, and the switch provided inside is turned on when the user stops pressing the decision button 139. Is a so-called mechanical contact type button that turns off. On the other hand, the four directional buttons 135 to 138 are provided internally when the user touches (touches) the directional button to turn on the switch provided inside, and when the user releases the directional button. It is a so-called capacitive button that switches off. Note that the specific example of the method of the decision button 139 and each direction button is only an example, and various modes can be adopted. For example, all of the decision button 139 and each direction button may be of the mechanical contact type, or all of them may be of the capacitance type.

As described above, according to the input device (control device 10) of the present embodiment, the decision button 139 and the direction buttons 135 to 138 can be configured as a cross key 13 that is easy for the user to intuitively grasp and operate. it can. Further, the input device can be configured by using either a mechanical contact type button or a capacitance type button.

A-3. I / O control (general purpose example):
A general-purpose example of flick input by the input / output control unit 151 will be described.

FIG. 8 is a diagram illustrating a method of flick input using the cross key 13. In FIG. 8, a right flick will be described as an example.
(1) First, the user presses the decision button 139 with the finger of the hand hd, and then stops pressing the decision button 139. In the present embodiment, a series of operations in which the user presses and releases the enter button 139 is also referred to as "operation 1". By operation 1, the input / output control unit 151 acquires the on / off of the decision button 139.
(2) Next, the user moves the hand hd to the position of the right button 136 (Fig. 8: arrow). In the present embodiment, the operation of the user moving his / her hand to the next direction button is also referred to as “operation 2”.
(3) Finally, the user touches the right button 136 with the finger of the hand hd. In the present embodiment, the operation of the user touching the direction button is also referred to as "operation 3". By operation 3, the input / output control unit 151 acquires the on of the right button 136.
In addition, this series of operations 1 to 3 functions as a "predetermined pattern".

The input / output control unit 151 acquires the on / off of the decision button 139 by the operation 1, and the on / off of the right button 136 (direction button) by the operation 3. In this way, when any one of the direction buttons is detected on after detecting the on / off of the enter button 139, the input / output control unit 151 executes an output corresponding to a flick input to the OS 143. Here, as the output corresponding to the flick input, various modes can be adopted as long as it can be interpreted as "a flick input has been made" on the receiving side (OS143 in the case of this embodiment). For example, the input / output control unit 151 can perform either of the following outputs a1 and a2.

(A1) Generate a flick event.
(A2) Generate a series of events (for example, the following) in which a flick input can be detected on the receiving side.
-Set up a virtual touch panel.
-Generate a touched event, for example, getAction (ACTION_DOWN) on a virtual touch panel.
-On a virtual touch panel, generate an event that is moving in a touched state, for example, getAction (ACTION_MOVE).
-On a virtual touch panel, the touch is separated, for example, getAction (ACTION_UP) is generated.
The input / output control unit 151 determines in advance various states (X coordinate, Y coordinate, number of touched fingers, touch pressure, touched time, etc.) related to the touch event, if necessary. You can set the dummy value that is set.

The user may press the button using a tool such as a stylus instead of the finger of the hand hd. Further, in the above example, the determination button 139, which is a mechanical contact type button, is described as "press", and the four directional buttons 135 to 138, which are capacitance type buttons, are described as "touch". However, the operation content (pressing / touching) of the user can be appropriately changed from each other according to the configuration content (mechanical contact type / capacitance type) of each button 135-139. As long as the on / off detection results of the buttons 135 to 139 include a predetermined pattern (operations 1 to 3), the input / output control unit 151 may execute an output corresponding to a flick input. Therefore, for example, even if the on / off detection result of each button 135 to 139 does not completely match the predetermined pattern (operations 1 to 3) as a result of switch chattering or the like, the input / output control unit 151 Can perform output equivalent to flick input.

As described above, according to the input device (control device 10) of the above embodiment, the detection result of each button is predetermined in the two or more buttons (each button 135 to 139) that can individually turn the input on and off. When the patterns (operations 1 to 3) are included, the outputs a1 and a2 corresponding to the flick input are made. Specifically, when the enter button 139 is pressed (or touched) and then any one of the four direction buttons 135 to 138 is pressed (or touched), in other words, the push / touch is sandwiched. As the finger moves, outputs a1 and a2 corresponding to flick input are produced. As a result, the user can perform flick input by using the cross key 13 without using a sensor device capable of acquiring a trajectory of a finger or the like by a finger moving motion imitating an actual flick motion. .. Further, in this example, since the outputs a1 and a2 are output to the OS143 or the like in the form of an event, they can be used for various purposes depending on the application on the receiving side.

A-4. Input / output control (example of character input):
An example of character input using flick input by the input / output control unit 151 will be described. FIG. 9 is a diagram showing an example of an operation pattern in which the input / output control unit 151 outputs an output corresponding to a flick input. Each of the patterns 1 to 4 in FIG. 9 functions as a "predetermined pattern". Each pattern in FIG. 9 can also be used in the above-mentioned input / output control (general-purpose example). When each pattern of FIG. 9 is used in input / output control (general-purpose example), the “screen display” of FIG. 9 is unnecessary.

<Pattern 1>
FIG. 10 is a diagram illustrating operation 1 of pattern 1. FIG. 11 is a diagram illustrating operation 2 of pattern 1. FIG. 12 is a diagram illustrating operation 3 of pattern 1. First, the input / output control unit 151 causes the image display unit 20 to display the software keyboard pd (FIG. 10: left side). On the software keyboard pd, candidates for the Japanese syllabary from "a" to "wa" are arranged in a tile shape, and in the initial state, "a" is surrounded by a thick rectangle. It is in. By touching the direction button of the cross key 13, the user moves the bold rectangle from the candidate of the Japanese syllabary of the software keyboard pd to the desired candidate and puts it in the selected state. In the example of FIG. 10, "na" is selected.

The user presses the enter button 139 with the finger of the hand hd, and then stops pressing the enter button 139 (FIG. 10: operation 1). By operation 1, the input / output control unit 151 acquires the on / off of the decision button 139. At this time, the input / output control unit 151 determines that the "operation 1 condition" of the pattern 1 shown in FIG. 9 is satisfied. Therefore, the input / output control unit 151 emphasizes the selected candidate "na" and the consonant candidate "na" for the selected candidate "na" on the software keyboard pd displayed on the image display unit 20. "Ni-no" is displayed (Fig. 11: left side). In addition, in the drawings after FIG. 11, for convenience of illustration, the display of the Japanese syllabary not selected is omitted.

The user moves the hand hd to the position of the direction button representing the consonant to be selected (Fig. 11: Operation 2). At this time, the input / output control unit 151 continues to display the consonant candidates on the software keyboard pd.

The user touches the direction button (right button 136 in the example of the figure) with the finger of the hand hd (FIG. 12: operation 3). By operation 3, the input / output control unit 151 acquires the on of the right button 136. At this time, the input / output control unit 151 determines that the "operation 3 condition" of the pattern 1 shown in FIG. 9 is satisfied. Therefore, the input / output control unit 151 emphasizes the selected consonant "ne" on the software keyboard pd displayed on the image display unit 20 (FIG. 12: left side). Further, the input / output control unit 151 outputs the selected consonant "ne" to the OS 143 as an output corresponding to the flick input. As a result, for example, "ne" is displayed on the character input pad or the like. As described above, in this example, as the output corresponding to the flick input, the finally selected character is output instead of the above-mentioned outputs a1 and a2.

<Pattern 2>
FIG. 13 is a diagram illustrating operation 2 and operation 3 of the pattern 2. First, the input / output control unit 151 causes the image display unit 20 to display the software keyboard pd. The details are the same as in FIG. By touching the direction button of the cross key 13, the user moves the bold rectangle from the candidate of the Japanese syllabary of the software keyboard pd to the desired candidate and puts it in the selected state.

The user presses the enter button 139 with the finger of the hand hd, and then stops pressing the enter button 139 (FIG. 10: operation 1). By operation 1, the input / output control unit 151 acquires the on / off of the decision button 139. At this time, the input / output control unit 151 determines that the "operation 1 condition" of the pattern 2 shown in FIG. 9 is satisfied. Therefore, the input / output control unit 151 emphasizes the selected candidate "na" on the software keyboard pd displayed on the image display unit 20. In this example, consonant candidates are not displayed.

The user moves the hand hd to the position of the direction button representing the consonant to be selected (FIG. 13: operation 2).

The user touches the direction button (right button 136 in the example of the figure) with the finger of the hand hd (FIG. 13: operation 3). By operation 3, the input / output control unit 151 acquires the on of the right button 136. Next, the input / output control unit 151 determines whether or not the time t1 (sec) from the off acquisition of the decision button 139 of the operation 1 to the on acquisition of the direction button of the operation 3 is within a predetermined time n (sec). judge. The predetermined time n functions as a "first time" and is stored in a predetermined storage unit 122. The predetermined time n can be arbitrarily set, but it is preferably set to a somewhat short time assuming the finger movement time in a general flick operation. When the time t1 is within the predetermined time n, the input / output control unit 151 determines that the “operation 2 condition” of the pattern 2 shown in FIG. 9 is satisfied.

Further, since the input / output control unit 151 has acquired the on of the right button 136 by the operation 3, the input / output control unit 151 determines that the "operation 3 condition" of the pattern 2 shown in FIG. 9 is also satisfied. Therefore, the input / output control unit 151 displays and emphasizes the selected consonant "ne" on the software keyboard pd displayed on the image display unit 20 (FIG. 13: left side). Further, the input / output control unit 151 outputs the selected consonant "ne" to the OS 143 as an output corresponding to the flick input.

According to the input / output control using the pattern 2, the time (t1) between the pressing (or touch) of the enter button 139 and the pressing (or touch) of the direction buttons 135 to 138 is within the first time (n). Only in that case, the output corresponding to the flick input is made. By appropriately setting this first time, the user can properly use the normal decision-direction input using the cross key 13 and the flick input.

<Pattern 3>
Pattern 3 is the same as pattern 1 except that the determination criteria for "operation 1 condition" in the input / output control unit 151 are different. In the pattern 3, the input / output control unit 151 satisfies the "operation 1 condition" when the user repeatedly presses the decision button 139 twice, in other words, when the decision button 139 is turned on and off twice in a row. Judge that it was done.

According to the input / output control using the pattern 3, only when the direction buttons 135 to 138 are pressed (or touched) after the enter button 139 is pressed (or touched) twice, it corresponds to the flick input. Output is made. Therefore, the user can properly use the normal decision-direction input using the cross key 13 and the flick input.

<Pattern 4>
Pattern 4 is the same as pattern 1 except that the determination criteria of "operation 1 condition" in the input / output control unit 151 are different. In the pattern 4, when the user presses and holds the decision button 139, the input / output control unit 151, in other words, the time t2 (sec) from the on acquisition to the off acquisition of the decision button 139 is a predetermined time m (sec). ), It is determined that the "operation 1 condition" is satisfied. The predetermined time m functions as a "second time" and is predetermined and stored in the storage unit 122. The predetermined time m can be arbitrarily set, but it is preferable to set it to a time assuming a general button long press.

According to the input / output control using the pattern 4, when the period (t2) from the start of pressing (or touching) the decision button 139 to the end of pressing (or touching) the decision button 139 is longer than the second time (m). Only, the output corresponding to the flick input is made. By appropriately setting this second time, the user can consciously use the normal decision-direction input using the cross key 13 and the flick input properly when the decision button 139 is pressed and held. Can be done.

As described above, according to the input device (control device 10) of the above embodiment, the detection result of each button is predetermined in the two or more buttons (each button 135 to 139) that can individually turn the input on and off. When the patterns 1 to 4 (operations 1 to 3) are included, an output corresponding to the flick input (output of the character selected by the flick character input) is performed. As a result, the user can perform flick input by using the cross key 13 without using a sensor device capable of acquiring a trajectory of a finger or the like by a finger moving motion imitating an actual flick motion. ..

FIG. 14 is a diagram illustrating another configuration of the cross key 13. The cross key 13a shown in FIG. 14 is the same as the cross key 13 described above, except that the enter button 139 is not provided. When the above-mentioned input control is performed by using such a cross key 13a, another direction button may be used instead of the enter button 139. For example, the user touches and releases the left button 135 and touches the right button 136. As a result, the input / output control unit 151 can acquire the on / off of the left button 135 and the on / off of the right button 136, so that the output corresponding to the right flick input is performed. Similarly, the user touches and releases the upper button 137 and touches the lower button 138. As a result, the input / output control unit 151 outputs the output corresponding to the lower flick input. As described above, even when there is no decision button 139, the input / output control unit 151 can perform the same operation as that of the above embodiment as long as there are two or more direction buttons.

FIG. 15 is a diagram illustrating another configuration of the cross key 13. The cross key 13b shown in FIG. 15 is the same as the cross key 13 described above, except that the surface of each direction button 135 to 139 is covered with a circular cover. Even if the cross key 13b having such a configuration is used, the same effect as that of the above embodiment can be obtained.

FIG. 16 is a diagram illustrating another configuration of the cross key 13. The key 13c shown in FIG. 16 is the cross key 13 described above, except that the decision button 139 is formed in a rectangular shape and the decision button 139 and the direction buttons 135 to 138 are arranged in a row. It is the same. Even if the key 13c having such a configuration is used, the same effect as that of the above embodiment can be obtained.

B. Modification example:
In the above embodiment, a part of the configuration realized by the hardware may be replaced with software, and conversely, a part of the configuration realized by the software may be replaced with the hardware. Good. In addition, the following modifications are possible.

-Modification example 1:
In the above embodiment, the configuration of the HMD has been illustrated. However, the configuration of the HMD can be arbitrarily determined within a range that does not deviate from the gist of the present invention, and for example, components can be added, deleted, converted, and the like.

In the above embodiment, the so-called transmissive HMD100 in which the right light guide plate 26 and the left light guide plate 28 transmit external light has been described. However, the present invention can also be applied to, for example, a so-called non-transparent HMD 100 that displays an image in a state where the outside view cannot be visually recognized. Further, in these HMD 100s, in addition to the AR (Augmented Reality) display that displays an image superimposed on the real space described in the above embodiment, MR (Mixed) that displays the captured real space image and the virtual image in combination is displayed. Reality) display or VR (Virtual Reality) display that displays the virtual space can also be performed. The present invention can also be applied to an apparatus that does not display AR, MR, or VR.

In the above embodiment, the functional units of the control device 10 and the image display unit 20 have been described, but these can be arbitrarily changed. For example, the following aspects may be adopted. A mode in which a storage unit 122 and a control unit 150 are mounted on the control device 10, and only a display function is mounted on the image display unit 20. A mode in which a storage unit 122 and a control unit 150 are mounted on both the control device 10 and the image display unit 20. A mode in which the control device 10 and the image display unit 20 are integrated. In this case, for example, the image display unit 20 includes all the components of the control device 10 and is configured as a glasses-type wearable computer. A mode in which a smartphone or a portable game device is used instead of the control device 10. A mode in which the control device 10 and the image display unit 20 are connected by wireless communication and a connection cable 40 is arranged. In this case, for example, power supply to the control device 10 and the image display unit 20 may be performed wirelessly.

-Modification example 2:
In the above embodiment, the configuration of the control device has been illustrated. However, the configuration of the control device can be arbitrarily determined within a range that does not deviate from the gist of the present invention, and for example, components can be added, deleted, converted, and the like.

In the above embodiment, an example of the input means included in the control device 10 has been described. However, the control device 10 may be configured by omitting some of the illustrated input means, or may include other input means not described above. For example, the control device 10 may include an operation stick, a keyboard, a mouse, and the like. For example, the control device 10 may include an input means for interpreting a command associated with the movement of the user's body or the like. The movement of the user's body can be acquired by, for example, line-of-sight detection for detecting the line of sight, gesture detection for detecting the movement of the hand, a foot switch for detecting the movement of the foot, or the like. The line-of-sight detection can be realized by, for example, a camera that images the inside of the image display unit 20. Gesture detection can be realized, for example, by image analysis of an image taken with time by the camera 61.

In the above embodiment, the control unit 150 operates by the main processor 140 executing a computer program in the storage unit 122. However, the control unit 150 can adopt various configurations. For example, the computer program may replace the storage unit 122, or together with the storage unit 122, with the non-volatile storage unit 121, the EEPROM 215, the memory 118, and other external storage devices (storage devices such as USB memory inserted in various interfaces). It may be stored in an external device such as a server connected via a network). Further, each function of the control unit 150 may be realized by using an ASIC (Application Specific Integrated Circuit) designed to realize the function.

-Modification example 3:
In the above embodiment, the configuration of the image display unit has been illustrated. However, the configuration of the image display unit can be arbitrarily determined within a range that does not deviate from the gist of the present invention, and for example, components can be added, deleted, converted, and the like.

FIG. 17 is a plan view of a main part showing the configuration of the optical system included in the image display unit of the modified example. In the image display unit of the modified example, an OLED unit 221a corresponding to the user's right eye RE and an OLED unit 241a corresponding to the left eye LE are provided. The OLED unit 221a corresponding to the right eye RE includes an OLED panel 223a that develops a white color and an OLED drive circuit 225 that drives the OLED panel 223a to emit light. A modulation element 227 (modulation device) is arranged between the OLED panel 223a and the right optical system 251. The modulation element 227 is composed of, for example, a transmissive liquid crystal panel, and modulates the light emitted by the OLED panel 223a to generate image light L. The image light L transmitted through the modulation element 227 and modulated is guided to the right eye RE by the right light guide plate 26.

The OLED unit 241a corresponding to the left eye LE includes an OLED panel 243a that emits light in white and an OLED drive circuit 245 that drives the OLED panel 243a to emit light. A modulation element 247 (modulation device) is arranged between the OLED panel 243a and the left optical system 252. The modulation element 247 is composed of, for example, a transmissive liquid crystal panel, and modulates the light emitted by the OLED panel 243a to generate image light L. The image light L transmitted through the modulation element 247 and modulated is guided to the left eye LE by the left light guide plate 28. The modulation elements 227 and 247 are connected to a liquid crystal driver circuit (not shown). This liquid crystal driver circuit (modulation device driving unit) is mounted on a substrate arranged in the vicinity of, for example, the modulation elements 227 and 247.

According to the image display unit of the modified example, the right display unit 22 and the left display unit 24 have the OLED panels 223a and 243a as the light source unit and the image light including a plurality of colored lights by modulating the light emitted by the light source unit, respectively. It is configured as a video element including a modulation element 227 and 247 that output the light source. The modulation device that modulates the light emitted by the OLED panels 223a and 243a is not limited to the configuration in which the transmissive liquid crystal panel is adopted. For example, a reflective liquid crystal panel may be used instead of the transmissive liquid crystal panel, a digital micromirror device may be used, or a laser retinal projection type HMD100 may be used.

In the above embodiment, the eyeglass-shaped image display unit 20 has been described, but the aspect of the image display unit 20 can be arbitrarily changed. For example, the image display unit 20 may be worn like a hat, or may be built into a body protective device such as a helmet. Further, the image display unit 20 may be configured as a HUD (Head Up Display) mounted on a vehicle such as an automobile or an airplane, or other means of transportation.

In the above embodiment, as an optical system that guides the image light to the eyes of the user, a configuration in which a virtual image is formed on a part of the right light guide plate 26 and the left light guide plate 28 by the half mirrors 261 and 281 is illustrated. However, this configuration can be changed arbitrarily. For example, a virtual image may be formed in a region that occupies the entire surface (or most) of the right light guide plate 26 and the left light guide plate 28. In this case, the image may be reduced by the operation of changing the display position of the image. Further, the optical element of the present invention is not limited to the right light guide plate 26 and the left light guide plate 28 having the half mirrors 261 and 281, and an optical component (for example, a diffraction grating, a prism, etc.) that allows image light to enter the user's eye. Any aspect can be adopted as long as holography or the like is used.

-Modification example 4:
In the above embodiment, input / output control has been illustrated. However, the processing content in the input / output control can be arbitrarily determined without departing from the gist of the present invention.

In the above embodiment, a form surrounded by a thick rectangular frame is illustrated to emphasize the selected candidate. However, emphasis on selected candidates can be performed in various ways. For example, for the selected candidate (character itself), the display is enlarged, the brightness of the display is changed, the display position is moved, and the display form is changed, as compared with other candidates that are not selected. Etc. can be adopted. When changing the brightness of the display, it may be possible to change whether the selected candidate is brightened or darkened according to the ambient brightness. When moving the display position, the position may be changed so that the selected candidate moves to the center of the cross display and the unselected candidate moves up, down, left, and right of the cross display. When changing the display form, the selected candidate may be displayed in black or white, or the typeface of the selected candidate may be changed. Further, the selected candidate (character itself) may be notified by voice together with the above-mentioned highlighting or in place of the above-mentioned highlighting. By doing so, it is possible to emphasize the selected candidates not only by the user's eyesight but also by hearing.

-Modification example 5:
The present invention is not limited to the above-described embodiments, examples, and modifications, and can be realized with various configurations within a range not deviating from the gist thereof. For example, the technical features in the embodiments, examples, and modifications corresponding to the technical features in each embodiment described in the column of the outline of the invention may be used to solve some or all of the above-mentioned problems. , It is possible to replace or combine as appropriate in order to achieve some or all of the above effects. Further, if the technical feature is not described as essential in the present specification, it can be appropriately deleted.

10 ... Control device, 11 ... Button, 12 ... LED indicator, 13, 13a, 13b ... Cross key, 13c ... Key, 14 ... Track pad, 15 ... Up and down key, 16 ... Changeover switch, 18 ... Power switch, 19 ... Vibrator , 20 ... image display unit, 21 ... right holding unit, 22 ... right display unit, 23 ... left holding unit, 24 ... left display unit, 26 ... right light guide plate, 27 ... front frame, 28 ... left light guide plate, 30 ... headset, 32 ... right earphone, 34 ... left earphone, 40 ... connection cable, 46 ... connector, 61 ... camera, 63 ... microphone, 65 ... illuminance sensor, 67 ... LED indicator, 110 ... operation unit, 111 ... 6 axes Sensor, 113 ... Magnetic sensor, 117 ... Communication unit, 118 ... Memory, 120 ... Controller board, 121 ... Non-volatile storage unit, 122 ... Storage unit, 123 ... Setting data, 124 ... Content data, 130 ... Power supply unit, 132 ... Battery, 134 ... Power control circuit, 135 ... Left button, 136 ... Right button, 137 ... Up button, 138 ... Down button, 139 ... Enter button, 140 ... Main processor, 145 ... Image processing unit, 147 ... Display control unit, 149 ... Imaging control unit, 150 ... Control unit, 151 ... Input / output control unit, 180 ... Voice codec, 182 ... Voice interface, 184 ... External connector, 186 ... External memory interface, 188 ... USB connector, 192 ... Sensor hub, 196 ... Interface, 210 ... Display unit board, 211 ... Interface, 213 ... Receiver, 215 ... EEPROM, 217 ... Temperature sensor, 221 221a ... OLED unit, 223, 223a ... OLED panel, 225 ... OLED drive circuit, 227 ... Modulation Element, 230 ... Display unit board, 231 ... Interface, 233 ... Receiver, 235 ... 6-axis sensor, 237 ... Magnetic sensor, 239 ... Temperature sensor, 241, 241a ... OLED unit, 243, 243a ... OLED panel, 245 ... OLED Drive circuit, 247 ... Modulator, 251 ... Right optical system, 252 ... Left optical system, 261 ... Half mirror, 281 ... Half mirror

Claims (13)

It is an input device
Two or more buttons that can be turned on and off individually,
A control unit that controls the input device and
With
When the detection result of each button includes a predetermined pattern, the control unit executes an output corresponding to a flick input, and when the detection result of each button does not include the predetermined pattern, the detection result of each button is used as it is. Output and
Said predetermined pattern of said two or more buttons, after detecting at least one time on and off of one of the buttons state, and are the detection of the on-the other buttons,
The two or more buttons include a decision button and a direction button corresponding to each of the top, bottom, left, and right.
The predetermined pattern is
An input device that detects the on / off of any one of the direction buttons after detecting the on / off of the decision button . The input device according to claim 1 .
The predetermined pattern is
An input device that detects the on / off of any one of the direction buttons within a predetermined first time after detecting the on / off of the decision button. The input device according to claim 1 .
The predetermined pattern is
After detecting the on of the decision button, the off of the decision button is detected after a lapse of a predetermined second time, and then the on of any one of the direction buttons is detected. Input device. It is an input device
Two or more buttons that can be turned on and off individually,
A control unit that controls the input device and
With
When the detection result of each button includes a predetermined pattern, the control unit executes an output corresponding to a flick input, and when the detection result of each button does not include the predetermined pattern, the detection result of each button is used as it is. Output and
The predetermined pattern is that, of the two or more buttons, the on / off of one button is detected at least once, and then the on / off of the other button is detected.
The two or more buttons include a decision button and a direction button corresponding to each of the top, bottom, left, and right.
The predetermined pattern is
An input device that detects the on / off of the decision button twice and then detects the on / off of any one of the direction buttons. The input apparatus according to any one of claims 1 to 4,
The two or more buttons are input devices in which each of the direction buttons is a cross key arranged in the up, down, left, and right directions about the decision button. The input device according to any one of claims 1 to 5 .
Each of the two or more buttons is an input device, which is a mechanical contact type button or a capacitance type button. The input device according to any one of claims 1 to 6 , and further
It is equipped with an interface for connecting to an image display unit that allows the user to visually recognize a virtual image by being attached to the user's head.
The control unit further
A function of generating image light and outputting the generated image light to the image display unit,
Including an operating system that allows the execution of various applications,
The control unit is an input device that outputs an output corresponding to the flick input or the detection result to at least one of the application and the operating system. The input device according to any one of claims 1 to 6 .
The control unit further includes an operating system that enables execution of various applications.
The control unit is an input device that outputs an output corresponding to the flick input or the detection result to at least one of the application and the operating system. The input device according to any one of claims 1 to 6 , and further
Equipped with an interface for connecting to an external device
The control unit is an input device that outputs an output corresponding to the flick input or the detection result to the external device via the interface. It is an input method using two or more buttons that can turn input on and off individually.
When the detection result of each button includes a predetermined pattern, the computer executes an output corresponding to a flick input, and when the detection result of each button does not include the predetermined pattern, the detection result of each button is output as it is. With a process,
Said predetermined pattern of said two or more buttons, after detecting at least one time on and off of one of the buttons state, and are the detection of the on-the other buttons,
The two or more buttons include a decision button and a direction button corresponding to each of the top, bottom, left, and right.
The predetermined pattern is
An input method for detecting the on / off of any one of the direction buttons after detecting the on / off of the enter button . It ’s a computer program,
When the detection result of each button in two or more buttons whose inputs can be turned on and off individually includes a predetermined pattern, the output corresponding to the flick input is executed, and the predetermined pattern is not included. In the case, it has a function to output the detection result of each button as it is.
Said predetermined pattern of said two or more buttons, after detecting at least one time on and off of one of the buttons state, and are the detection of the on-the other buttons,
The two or more buttons include a decision button and a direction button corresponding to each of the top, bottom, left, and right.
The predetermined pattern is
A computer program that detects the on / off of any one of the direction buttons after detecting the on / off of the enter button . It is an input method using two or more buttons that can turn input on and off individually.
When the detection result of each button includes a predetermined pattern, the computer executes an output corresponding to a flick input, and when the detection result of each button does not include the predetermined pattern, the detection result of each button is output as it is. With a process
The predetermined pattern is that, of the two or more buttons, the on / off of one button is detected at least once, and then the on / off of the other button is detected.
The two or more buttons include a decision button and a direction button corresponding to each of the top, bottom, left, and right.
The predetermined pattern is
An input method in which the on / off of the decision button is detected twice, and then the on / off of any one of the direction buttons is detected. It ’s a computer program,
When the detection result of each button in two or more buttons whose inputs can be turned on and off individually includes a predetermined pattern, the output corresponding to the flick input is executed, and the predetermined pattern is not included. In the case, it has a function to output the detection result of each button as it is.
The predetermined pattern is that, of the two or more buttons, the on / off of one button is detected at least once, and then the on / off of the other button is detected.
The two or more buttons include a decision button and a direction button corresponding to each of the top, bottom, left, and right.
The predetermined pattern is
A computer program that detects the on / off of the decision button twice and then detects the on / off of any one of the direction buttons.

Images