JPWO2017065050A1 - Input device, electronic device, input method of electronic device, and input program thereof - Google Patents

Abstract

Provided are a user-friendly input device, an electronic device, an input method for an electronic device, and an input program thereof that can suppress erroneous input while realizing non-contact operation. When the detection device detects that the indicator has moved in the positive direction of the specific direction within the detection area, the control device determines the input content corresponding to the movement in the positive direction of the specific direction, When the detection device detects that the body has moved in the negative direction opposite to the positive direction of the specific direction within the detection area, the determination of the input content corresponding to the movement in the negative direction of the specific direction has been suspended On the other hand, when the detection device detects that the indicator has moved in the positive direction of the specific direction within the detection area within the predetermined time, the input corresponding to the movement of the reserved direction in the negative direction of the specific direction Determine the content.

Description

  The present invention relates to an input device, an electronic device, an input method of the electronic device, and an input program thereof.

  In recent years, mobile terminals such as smartphones that have been rapidly developed are often used for business and home work assistance. A typical mobile terminal has a touch panel screen that serves both as an image display and a user interface. By touching this screen, the user can make necessary inputs to display a desired image or input information. Can be performed. However, there are cases where you want to operate the mobile device without touching the screen, such as when the user's hand is wet or dirty, but how to perform input at that time is an issue Yes.

  Patent Document 1 discloses a computing device including an infrared LED and an infrared proximity sensor. According to the technique of Patent Document 1, the infrared light emitted from the infrared LED is reflected by the hand approaching the computing device and reflected, and the reflected light can be detected by the infrared proximity sensor to detect the movement of the hand. The user can perform an input operation desired by the user such as swipe without contact by the movement (gesture) of the hand. Therefore, even if the user's hand is dirty, there is no risk of contaminating the mobile computing device.

Special table 2013-534209

  However, in the computing device disclosed in Patent Document 1, when performing gesture detection using an infrared proximity sensor, a user may desire multiple scrolls or page feeds in one direction. If the user reciprocates his / her hand within the detection area, the computing device that has received the signal from the infrared proximity sensor that has detected this will recognize it as reciprocating scrolling or reciprocating page feed. Therefore, in the computing device disclosed in Patent Document 1, when the user wants to scroll or page forward multiple times in one direction according to the user's intention, the user moves the hand in one direction within the detection region of the infrared proximity sensor. After returning the hand to the original position through the outside of the detection area, it is necessary to move again in the same direction within the detection area. However, requiring such an operation is a heavy burden on the user, and it is difficult to unconsciously perform such an operation, which may cause erroneous input.

  The present invention has been made in view of the above circumstances, and is a user-friendly input device, an electronic device, an input method for an electronic device, and an input program thereof that can suppress erroneous input while realizing a non-contact operation. The purpose is to provide.

In order to achieve at least one of the above-described objects, an input device reflecting one aspect of the present invention is:
A detection device having a detection area, detecting a direction in which the indicator moved by the user moves in the detection area, and generating an output corresponding thereto;
A control device for determining input contents according to the output of the detection device, and an input device comprising:
When the detection device detects that the indicator has moved in the positive direction of the specific direction within the detection area, the control device determines an input content corresponding to the movement of the indicator in the positive direction of the specific direction. And
The control device has moved in the negative direction of the specific direction when the detection device detects that the indicator has moved in the negative direction opposite to the positive direction of the specific direction in the detection area. When the detection device detects that the indicator has moved in the positive direction of the specific direction within the detection area within a predetermined time, the input content corresponding to is suspended. The input content corresponding to the movement in the negative direction of the specific direction is determined.

In order to achieve at least one of the above-described objects, an electronic device input method reflecting one aspect of the present invention has a detection area and detects a direction in which an indicator moved by a user moves in the detection area. An input method of an electronic device comprising a detection device that generates an output corresponding to the detection device, and determining input contents according to the output of the detection device,
When the detection device detects that the indicator has moved in the positive direction of the specific direction within the detection area, the input content corresponding to the movement in the positive direction of the specific direction is determined, and the indicator Is detected in the negative direction opposite to the positive direction of the specific direction in the detection area, the input content corresponding to the movement in the negative direction of the specific direction is determined. When the detection device detects that the indicator has moved in the positive direction of the specific direction within the detection area within a predetermined time after being put on hold, the negative direction of the specific direction put on hold The input content corresponding to the movement to is determined.

In order to achieve at least one of the above-described objects, an electronic device input program reflecting one aspect of the present invention has a detection area, and detects a direction in which an indicator moved by the user moves in the detection area. And an electronic device input program that includes a detection device that generates an output corresponding to the detection device, and that determines input contents according to the output of the detection device,
When the detection device detects that the indicator has moved in the positive direction of the specific direction within the detection area, the input content corresponding to the movement in the positive direction of the specific direction is determined, and the indicator Is detected in the negative direction opposite to the positive direction of the specific direction in the detection area, the input content corresponding to the movement in the negative direction of the specific direction is determined. When the detection device detects that the indicator has moved in the positive direction of the specific direction within the detection area within a predetermined time after being put on hold, the negative direction of the specific direction put on hold The input content corresponding to the movement to is determined.

  According to the present invention, it is possible to provide a user-friendly input device, an electronic device, an input method for an electronic device, and an input program thereof that can suppress erroneous input while realizing a non-contact operation.

It is a perspective view of the head mounted display (HMD) concerning this embodiment. It is the figure which looked at HMD from the front. It is the figure which looked at HMD from the upper part. It is a schematic sectional drawing which shows the structure of a display unit. It is an enlarged view of a proximity sensor. It is a block diagram of the main circuit of HMD. It is a front view when a user wears HMD. It is a side view when a user wears HMD. It is a top view when a user wears an HMD. It is a figure which shows the example of the signal waveform which a some light reception area | region outputs. It is a figure which shows the other example of the signal waveform which a some light reception area | region outputs. It is a figure which shows the other example of the signal waveform which a light reception area | region outputs. It is a figure which shows the example of a motion of user's US hand HD. It is a figure which shows the example of a motion of user's US hand HD. It is a flowchart for performing the input method by gesture operation. It is the figure which represented the example of (a) hand movement, (b) proximity sensor output, and (c) screen display control with the time chart. It is a figure which shows an example of the image switched by gesture operation.

  Embodiments according to the present invention will be described below with reference to the drawings. FIG. 1 is a perspective view of an HMD 100 that is an electronic apparatus according to the present embodiment. FIG. 2 is a front view of the HMD 100 according to the present embodiment. FIG. 3 is a view of the HMD 100 according to the present embodiment as viewed from above. Hereinafter, the right side and the left side of the HMD 100 refer to the right side and the left side for the user wearing the HMD 100.

  As shown in FIGS. 1-3, HMD100 of this embodiment has the flame | frame 101 which is a supporting member. A frame 101 that is U-shaped when viewed from above has a front part 101a to which two spectacle lenses 102 are attached, and side parts 101b and 101c extending rearward from both ends of the front part 101a. The two spectacle lenses 102 attached to the frame 101 may or may not have refractive power.

  A cylindrical main body 103 serving as a support member is fixed to the front portion 101 a of the frame 101 on the upper side of the right eyeglass lens 102 (which may be the left side according to the user's dominant eye). The main body 103 is provided with a display unit 104. A display control unit 104DR (see FIG. 6 to be described later) that controls display control of the display unit 104 based on an instruction from a processor 121 described later is disposed in the main body 103. If necessary, a display unit may be arranged in front of both eyes.

  FIG. 4 is a schematic cross-sectional view showing the configuration of the display unit 104. The display unit 104 includes an image forming unit 104A and an image display unit 104B. The image forming unit 104A is incorporated in the main body unit 103, and includes a light source 104a, a unidirectional diffuser 104b, a condenser lens 104c, and a display element 104d. On the other hand, the image display unit 104B, which is a so-called see-through type display member, is disposed on the entire plate so as to extend downward from the main body unit 103 and extend in parallel to one eyeglass lens 102 (see FIG. 1). The eyepiece prism 104f, the deflecting prism 104g, and the hologram optical element 104h.

  The light source 104a has a function of illuminating the display element 104d. For example, 462 ± 12 nm (B light), 525 ± 17 nm (G light), 635 ± 11 nm (R It is composed of RGB-integrated LEDs that emit light in three wavelength bands. Thus, the light source 104a emits light having a predetermined wavelength width, whereby the image light obtained by illuminating the display element 104d can have a predetermined wavelength width, and the hologram optical element 104h transmits the image light. When diffracted, the image can be observed by the user over the entire observation angle of view at the position of the pupil B. Further, the peak wavelength for each color of the light source 104a is set in the vicinity of the peak wavelength of the diffraction efficiency of the hologram optical element 104h, so that the light use efficiency is improved.

  Further, since the light source 104a is composed of LEDs that emit RGB light, the cost of the light source 104a can be reduced, and a color image is displayed on the display element 104d when the display element 104d is illuminated. The color image can be visually recognized by the user. In addition, since each of the RGB LED elements has a narrow emission wavelength width, the use of a plurality of such LED elements enables high color reproducibility and bright image display.

  The display element 104d displays an image by modulating the light emitted from the light source 104a in accordance with image data, and is configured by a transmissive liquid crystal display element having pixels that serve as light transmitting regions in a matrix. Has been. Note that the display element 104d may be of a reflective type.

  The eyepiece prism 104f totally reflects the image light from the display element 104d incident through the base end face PL1 by the opposed parallel inner side face PL2 and outer side face PL3, and passes through the hologram optical element 104h to the user's pupil. On the other hand, external light is transmitted and guided to the user's pupil, and is composed of, for example, an acrylic resin together with the deflecting prism 104g. The eyepiece prism 104f and the deflection prism 104g are joined by an adhesive with the hologram optical element 104h sandwiched between inclined surfaces PL4 and PL5 inclined with respect to the inner surface PL2 and the outer surface PL3.

  The deflecting prism 104g is joined to the eyepiece prism 104f, and is integrated with the eyepiece prism 104f to form a substantially parallel plate. By joining the deflecting prism 104g to the eyepiece prism 104f, it is possible to prevent distortion in the external image observed by the user through the display unit 104.

  That is, for example, when the deflecting prism 104g is not joined to the eyepiece prism 104f, external light is refracted when passing through the inclined surface PL4 of the eyepiece prism 104f, so that an external image observed through the eyepiece prism 104f is distorted. . However, by joining the deflecting prism 104g having the inclined surface PL5 complementary to the eyepiece prism 104f to form an integral substantially parallel plate, external light is transmitted through the inclined surfaces PL4 and PL5 (hologram optical element 104h). The refraction at that time can be canceled by the deflecting prism 104g. As a result, it is possible to prevent distortion in the external image observed through the see-through. In addition, if the spectacle lens 102 (refer FIG. 1) is mounted | worn between the display unit 104 and a user's pupil, it will be possible for the user who normally uses spectacles to observe an image without a problem.

  The hologram optical element 104h diffracts and reflects the image light (light having a wavelength corresponding to the three primary colors) emitted from the display element 104d, guides it to the pupil B, enlarges the image displayed on the display element 104d, and enlarges the user's pupil. It is a volume phase type reflection hologram guided as a virtual image. The hologram optical element 104h has, for example, three wavelength ranges of 465 ± 5 nm (B light), 521 ± 5 nm (G light), and 634 ± 5 nm (R light) with a peak wavelength of diffraction efficiency and a wavelength width of half the diffraction efficiency. The light is diffracted (reflected). Here, the peak wavelength of diffraction efficiency is the wavelength at which the diffraction efficiency reaches a peak, and the wavelength width at half maximum of the diffraction efficiency is the wavelength width at which the diffraction efficiency is at half maximum of the diffraction efficiency peak. is there.

  The reflection-type hologram optical element 104h has high wavelength selectivity, and only diffracts and reflects light having a wavelength in the above-mentioned wavelength range (near the exposure wavelength). The hologram optical element 104h is transmitted, and a high external light transmittance can be realized.

  Next, the operation of the display unit 104 will be described. The light emitted from the light source 104a is diffused by the unidirectional diffusion plate 104b, condensed by the condenser lens 104c, and enters the display element 104d. The light incident on the display element 104d is modulated for each pixel based on the image data input from the display control unit 104DR, and is emitted as image light. Thereby, a color image is displayed on the display element 104d.

  The image light from the display element 104d enters the eyepiece prism 104f from its base end face PL1, is totally reflected a plurality of times by the inner side face PL2 and the outer side face PL3, and enters the hologram optical element 104h. The light incident on the hologram optical element 104h is reflected there, passes through the inner side surface PL2, and reaches the pupil B. At the position of the pupil B, the user can observe an enlarged virtual image of the image displayed on the display element 104d, and can visually recognize it as a screen formed on the image display unit 104B. In this case, it can be considered that the hologram optical element 104h constitutes a screen, or it can be considered that a screen is formed on the inner surface PL2. In the present specification, “screen” may refer to an image to be displayed.

  On the other hand, the eyepiece prism 104f, the deflecting prism 104g, and the hologram optical element 104h transmit almost all the external light, so that the user can observe an external image (real image) through them. Therefore, the virtual image of the image displayed on the display element 104d is observed so as to overlap with a part of the external image. In this manner, the user of the HMD 100 can simultaneously observe the image provided from the display element 104d and the external image via the hologram optical element 104h. Note that when the display unit 104 is in the non-display state, the image display unit 104B is transparent, and only the external image can be observed. In this example, a display unit is configured by combining a light source, a liquid crystal display element, and an optical system. However, instead of a combination of a light source and a liquid crystal display element, a self-luminous display element (for example, an organic EL display) is used. Element) may be used. Further, instead of a combination of a light source, a liquid crystal display element, and an optical system, a transmissive organic EL display panel having transparency in a non-light emitting state may be used. In any case, when the screen is arranged so as to fall within the visual field of the user's eye facing the image display unit 104B, and preferably at least partially overlaps the effective visual field, the user can easily visually recognize the image. Can do.

  Further, in FIGS. 1 and 2, a proximity sensor 105 disposed near the center of the frame 101 and a lens 106 a of a camera 106 disposed near the side are provided in front of the main body 103 so as to face the front. It has been.

  In this specification, a “proximity sensor” which is an example of a detection device refers to a proximity sensor in order to detect that an object, for example, a part of a human body (such as a hand or a finger) is in proximity to the user's eyes. This means that a signal is output by detecting whether or not it exists in a detection region in the proximity range in front of the detection surface. The proximity range may be set as appropriate according to the operator's characteristics and preferences. For example, the proximity range from the detection surface of the proximity sensor may be within a range of 200 mm. If the distance from the proximity sensor is 200mm or less, the user can put the palm and fingers into and out of the user's field of view with the arm bent, so the user can easily operate with gestures using the hands and fingers. In addition, the risk of erroneous detection of a human body or furniture other than the user is reduced. Here, the control circuit determines that the object exists in the proximity range based on a signal output from the proximity sensor when the object enters the detection area in the proximity range in front of the proximity sensor. When an object enters the detection area, a valid signal may be output from the proximity sensor to the control circuit.

  The proximity sensor includes a passive type and an active type. A passive proximity sensor has a detection unit that detects invisible light and electromagnetic waves emitted from an object when the object approaches. As a passive proximity sensor, there are a pyroelectric sensor that detects invisible light such as infrared rays emitted from an approaching human body, and a capacitance sensor that detects a change in electrostatic capacitance between the approaching human body. The active proximity sensor includes an invisible light or sound wave projection unit, and a detection unit that receives the invisible light or sound wave reflected and returned from the object. Active proximity sensors include infrared sensors that project infrared rays and receive infrared rays reflected by objects, laser sensors that project laser beams and receive laser beams reflected by objects, and project ultrasonic waves. Then, there is an ultrasonic sensor that receives ultrasonic waves reflected by an object. Note that a passive proximity sensor does not need to project energy toward an object, and thus has excellent low power consumption. Active proximity sensors can improve detection reliability. For example, even if the user wears gloves that do not transmit detection light emitted from the human body such as infrared light, the movement of the user's hand Can be detected. A plurality of types of proximity sensors may be used in combination.

  Proximity sensors are generally smaller and cheaper than cameras, and consume less power. In addition, it is difficult for the proximity sensor to perform complicated detection such as detection of the shape of the object, but it can determine the approach and separation of the object from the detection area, so that the user can pass the hand or finger or touch the palm. By holding it over, the HMD can be operated, and complicated image processing required for gesture recognition by analysis of the image captured by the camera is also unnecessary.

  FIG. 5 is an enlarged view of the proximity sensor 105 used in the present embodiment as viewed from the front. In this embodiment, an example in which a pyroelectric sensor is used as the proximity sensor 105 will be described. In FIG. 5, the proximity sensor 105 includes a light receiving unit 105 a that receives invisible light such as infrared light emitted from a human body as detection light. The light receiving unit 105a forms light receiving areas RA to RD arranged in 2 rows and 2 columns, and when receiving invisible light, signals corresponding to the light receiving areas RA to RD are individually output from the light receiving areas RA to RD. It has become. The outputs of the light receiving regions RA to RD change in intensity according to the distance from the light receiving unit 105a to the object, and the intensity increases as the distance decreases. If it is a pyroelectric sensor that detects infrared light emitted from the human body, it is difficult to falsely detect objects other than the exposed human body. For example, when working in a confined space, false detection is effectively prevented. There is an advantage that you can.

  1 and 2, a right sub-body portion 107 is attached to the right side portion 101b of the frame 101, and a left sub-body portion 108 is attached to the left side portion 101c of the frame 101. The right sub-main body portion 107 and the left sub-main body portion 108 have an elongated plate shape, and have elongated protrusions 107a and 108a on the inner side, respectively. By engaging the elongated protrusion 107 a with the elongated hole 101 d of the side portion 101 b of the frame 101, the right sub-body portion 107 is attached to the frame 101 in a positioned state, and the elongated protrusion 108 a is attached to the side of the frame 101. By engaging with the long hole 101e of the portion 101c, the left sub-main body portion 108 is attached to the frame 101 in a positioned state.

  In the right sub main body 107, there are a geomagnetic sensor 109 (see FIG. 6 described later) for detecting geomagnetism, and an angular velocity sensor 110B and an acceleration sensor 110A (see FIG. 6 described later) that generate an output corresponding to the posture. The left sub-main body 108 is provided with a speaker 111A, an earphone 111C, and a microphone 111B (see FIG. 6 to be described later). The main main body 103 and the right sub main body 107 are connected so as to be able to transmit signals through a wiring HS, and the main main body 103 and the left sub main body 108 are connected so as to be able to transmit signals through a wiring (not shown). Yes. As schematically illustrated in FIG. 3, the right sub-main body 107 is connected to the control unit CTU via a cord CD extending from the rear end. A 6-axis sensor in which an angular velocity sensor and an acceleration sensor are integrated may be used. Further, the HMD can be operated by sound based on an output signal generated from the microphone 111B according to the input sound. Further, the main main body 103 and the left sub main body 108 may be configured to be wirelessly connected. Although not shown in the figure, a power circuit 113 (see FIG. 6 described later) for converting the voltage applied from the battery control unit 128 of the control unit CTU into an appropriate voltage for each unit is provided in the right sub-main unit 107. Yes.

  FIG. 6 is a block diagram of main circuits of the HMD 100. The control unit CTU receives the radio waves from the processor 121, the operation unit 122, and the GPS satellite as a control device for controlling each functional device by generating a control signal for the display unit 104 and other functional devices. A GPS receiving unit 123, a communication unit 124 for exchanging data with the outside, a ROM 125 for storing programs, a RAM 126 for storing image data, a battery 127, a battery control unit 128, and a battery control unit 128. It has a power supply circuit 130 for converting the applied voltage into an appropriate voltage for each part, and a storage device 129 such as an SSD or a flash memory. The processor 121 can use an application processor used in a smartphone or the like, but the type of the processor 121 is not limited. For example, if an application processor includes hardware necessary for image processing such as GPU or Codec as a standard, it can be said that the processor is suitable for a small HMD.

  Furthermore, when the light receiving unit 105 a detects invisible light as detection light emitted from the human body, the processor 121 receives a signal from the proximity sensor 105. Further, the processor 121 controls image display of the display unit 104 via the display control unit 104DR. The processor 121 and the proximity sensor 105 constitute an input device.

  The processor 121 receives power from the power supply circuit 130, operates according to a program stored in at least one of the ROM 124 and the storage device 129, and receives image data from the camera 106 according to an operation input such as power-on from the operation unit 122. The data can be input and stored in the RAM 126, and can be communicated with the outside via the communication unit 124 as necessary. Furthermore, as will be described later, the processor 121 executes image control according to the output from the proximity sensor 105, so that the user can perform screen control of the HMD 100 by gesture operation using a hand or a finger.

  FIG. 7 is a front view when the user US wears the HMD 100 of the present embodiment. FIG. 8 is a side view when the user US wears the HMD 100, and FIG. 9 is a top view thereof, which is shown together with the user's hand. Here, the gesture operation is an operation in which at least the hand HD or finger of the user US passes through the detection area of the proximity sensor 105 and can be detected by the processor 121 of the HMD 100 via the proximity sensor 105.

  Next, the basic principle of gesture operation detection will be described. If there is nothing in front of the user US when the proximity sensor 105 is operating, the light receiving unit 105a does not receive invisible light as detection light, so the processor 121 determines that no gesture operation is performed. To do. On the other hand, as shown in FIG. 8, when the user US's own hand HD is approached in front of the user US, the light receiving unit 105a detects the invisible light emitted from the hand HD, and the proximity sensor 105 based on the invisible light is detected. In response to the output signal, the processor 121 determines that a gesture operation has been performed. In the following description, it is assumed that the gesture operation is performed by the user's hand HD. However, the gesture operation may be performed by the user using a pointing device made of a material that can emit invisible light. May be performed. An indicator can be moved by the user's intention. An imaging device (camera) can be used as the detection device instead of the proximity sensor. In such a case, the moving indicator is imaged by the imaging device, and the moving direction is determined by the image processing device (part of the control device) based on the image data output from the imaging device.

  As described above, the light receiving unit 105a includes the light receiving regions RA to RD arranged in 2 rows and 2 columns (see FIG. 5). Therefore, when the user US brings the hand HD closer to the front of the HMD 100 from either the left, right, up, or down directions, the output timings of signals detected in the light receiving areas RA to RD differ.

  10 and 11 are examples of signal waveforms of the light receiving regions RA to RD, where the vertical axis represents the signal intensity of the light receiving regions RA to RD and the horizontal axis represents time. For example, when the user US moves the hand HD from the right to the left in front of the HMD 100 with reference to FIGS. 8 and 9, invisible light emitted from the hand HD enters the light receiving unit 105a. In this case, the light receiving areas RA and RC first receive invisible light. Therefore, as shown in FIG. 10, first, the signals in the light receiving areas RA and RC rise, the signals in the light receiving areas RB and RD rise later, and the signals in the light receiving areas RA and RC fall, and then the light receiving areas RB, The RD signal falls. The processor 121 detects the timing of this signal, and determines that the user US has performed the gesture operation by moving the hand HD from the right to the left.

  In the example of FIG. 11, the signals of the light receiving areas RA and RB rise first, the signals of the light receiving areas RC and RD rise after a delay, and the signals of the light receiving areas RA and RB further fall, and then the signals of the light receiving areas RC and RD. Is falling. The processor 121 detects the timing of this signal, and determines that the user US has performed the gesture operation by moving the hand HD from the top to the bottom.

  Based on the output from the proximity sensor 105, the processor 121 can also detect that the hand HD has moved in the front-rear direction. FIG. 12 shows an example of a signal waveform in which the vertical axis represents the average signal intensity of the light receiving regions RA to RD and the horizontal axis represents time. The waveform indicated by the solid line is high in intensity immediately after output and decreases with time. Therefore, when such a waveform is output from the proximity sensor 105, the processor 121 determines that the gesture operation is performed by moving the hand HD away from the vicinity of the face. On the other hand, the waveform indicated by the dotted line is low in intensity immediately after output and increases with time. Therefore, when such a waveform is output from the proximity sensor 105, the processor 121 determines that the gesture operation is performed by moving the hand HD so as to approach the face from a distance.

  According to the present embodiment, the presence and movement of the hand HD can be reliably detected by using the proximity sensor and positioning the detection area within the visual field of the user's eye facing the image display unit. Accordingly, the user US can view the hand operation and the operation in conjunction with each other, and can realize an intuitive operation. Furthermore, since the proximity sensor is small and consumes less power than the camera, the continuous operation time of the HMD 100 can be extended.

  By the way, in general, humans tend to be in a standby posture in a posture where the burden on muscles or the like is reduced (for example, a state where the hand is lowered), and a position where the hand or the like takes a posture other than the standby posture (for example, the hand is raised) When it is moved to (state), the burden on muscles and the like becomes relatively large, and it is often difficult to maintain the posture for a long time. Further, when returning the hand from a posture other than the standby posture to the standby posture, it is generally considered that the natural operation is to pass the shortest path. In view of such human characteristics, there is a risk that hand movements that are not intended by the user will be mistakenly recognized as gesture detection.

  More specifically, in the HMD 100 worn on the head of the user US as in the present embodiment, the proximity sensor 105 is detected when the hand once raised according to the user's intention for the gesture operation is lowered to the standby position. It may pass through the area, and this may be detected as a gesture operation not intended by the user. On the other hand, if the user US keeps his hand raised for gesture operation, or extends his arm or turns it down while avoiding the detection area, the gesture operation unintended by the user is detected. Avoidance is possible. However, since these operations are different from natural operations based on human characteristics, requesting such operations may increase the burden on the user or deteriorate usability. In this embodiment, this problem is dealt with as follows.

  More specifically, gesture detection by the processor 121 will be described. FIGS. 13 and 14 show examples of the movement of the hand HD of the user US, and the SA indicated by the dotted line is the detection area. Hereinafter, the “specific direction” is the vertical direction, the “positive direction” is the direction from the top to the bottom, and the “negative direction” is the direction from the bottom to the top. Here, when the hand HD is moved in the forward direction from the position where the hand HD is lowered, the waveform shown in FIG. On the other hand, when the hand HD is moved in the negative direction from the position where the hand HD is raised, the signals from the light receiving areas RC and RD of the proximity sensor 105 rise up in advance, contrary to the waveform of FIG. The signal from RB will rise.

  First, in FIG. 13A, the user US holds the hand HD over the detection area SA, and from this state, as shown in FIG. 13B, while passing through the detection area SA, downward (in the forward direction). It shall be moved. At this time, the processor 121 determines that the hand HD has moved from top to bottom by the gesture operation based on the output of the proximity sensor 105 (confirms the input content), for example, referring to FIG. Control based on input such as movement from top to bottom.

  On the other hand, as shown in FIG. 14 (a), the hand HD positioned below the detection area SA by the user US moves upward (negative) while passing through the detection area SA as shown in FIG. 14 (b). In the case of movement, the processor 121 determines that the hand HD has moved from the bottom to the top by a gesture operation based on the output of the proximity sensor 105. The processor 121 waits for the next gesture operation for a predetermined time (for example, 2 to 3 seconds: it is desirable that the user can change it). When the user US moves the hand HD downward while passing through the detection area SA within a predetermined time (FIGS. 14 (c) and 14 (d)), the processor 121 is based on the output of the proximity sensor 105. Then, it is determined that the hand HD has moved from top to bottom by the gesture operation, and the gesture operation (input content) in which the hand HD that has been suspended has moved from bottom to top is determined. Accordingly, the processor 121 executes control based on an input such as movement from the bottom to the top of the image shown in FIG.

  That is, even when the user US reciprocates up and down so as to pass the detection area SA within a predetermined time as shown in FIG. 14, the processor 121 performs the gesture operation from the top to the bottom, It is not determined that the gesture operation from the top to the top has been repeated. Thereby, the gesture operation prevents, for example, a problem that images G1 and G2 shown in FIG. 17 to be described later are repeatedly displayed in the center. For example, the user US intends to move a plurality of screens unconsciously. Even when the hand HD is reciprocated, there is an advantage that the possibility of erroneous input is reduced. When instructing multiple screen feeds in the forward direction, of the up and down reciprocating movements of the hand HD, the downward movement from the top to the bottom is allowed to pass through the detection area, and the upward movement from the bottom to the outside is outside the detection area. By passing the, it is possible to input a continuous forward direction instruction instead of a round-trip instruction.

  In particular, when the user US lowers the hand HD from the top to the bottom by a gesture operation, this position becomes a standby posture, so even if the hand HD is held at this position, the user US is less burdensome and natural. I can say that. Therefore, it is desirable that the positive direction is a direction from top to bottom. On the other hand, when the user US raises the hand HD from the bottom to the top, the state in which the hand is raised is not the standby posture, so it is a burden on the user US to maintain this posture until, for example, the detection of the gesture operation times out. growing. Therefore, in the gesture operation, the user US raises the hand HD from the bottom to the top and then lowers the hand HD again from the top to the bottom to confirm the input content. From the above, it is desirable that the negative direction is from bottom to top. However, the positive and negative directions may be reversed depending on the user's preference.

  The same applies to the case where the user US moves the hand HD left and right so as to pass through the detection area SA with the specific direction as the left and right direction. In such a case, since it is natural to use the dominant hand (right-handed person is the right hand) for the gesture operation, it is preferable that the hand used for the gesture operation be on the dominant hand side of the body, so that the standby posture is used. On the assumption that the right-handed user US uses the right hand, when the proximity sensor 105 detects that the hand HD has moved from the left to the right while passing through the detection area SA, the processor 121 moves the hand HD by a gesture operation. You can immediately determine that you have moved from left to right.

  On the other hand, when the proximity sensor 105 detects that the hand HD has moved from the right to the left while passing through the detection area SA, the processor 121 waits for a predetermined time, and then the hand HD further passes through the detection area SA from the left. When the proximity sensor 105 detects that the hand has moved to the right, it can be determined that the hand HD has moved from the right to the left by a gesture operation. Other than that, it is the same as the above-mentioned control. In the above example, the positive direction is the left-to-right direction and the negative direction is the right-to-left direction. However, the positive / negative direction may be reversed depending on the preference of the left-handed person or the user.

  The same can be said when the specific direction is the front-rear direction. As described above, when the hand HD is moved in the previous sign direction within the detection area SA, the proximity sensor 105 outputs the waveform of FIG. At this time, since the direction in which the user US is approaching the hand HD is closer to the standby posture, the positive direction is the direction approaching the user US (dotted line waveform in FIG. 12), and the negative direction is the direction away from the user US (in FIG. 12). (Solid line waveform) is desirable. Furthermore, the specific direction may be an oblique direction. Other than that, it is the same as the above-mentioned control.

  By the way, after the proximity sensor 105 detects that the hand is moved in the negative direction with respect to the specific direction, the processor 121 waits for a predetermined gesture operation for a predetermined time. It is also assumed that no output is performed from the proximity sensor 105 without being performed. In such a case, the processor 121 can execute one of two types of determination control. First, if no gesture operation is performed until a predetermined time has elapsed, the processor 121 determines that the suspended gesture operation in the negative direction is invalidated (in other words, the hand HD moves in the negative direction). The other control is a determination control that immediately confirms the suspended gesture operation in the negative direction. In the latter case, a continuous negative direction instruction can be input simply by performing a gesture operation in the negative direction after a predetermined time. These can be selected according to preference.

  In addition, although it is assumed that gesture operations are continuously performed, for example, when the proximity sensor 105 detects that the hand is moved in a positive direction with respect to a specific direction (for example, the vertical direction), the processor 121 Since the input content by the gesture operation in the positive direction in the specific direction is immediately determined, when the proximity sensor 105 detects another gesture operation (including the gesture operation in the positive direction in the specific direction), the processor 121 performs the gesture operation. This judgment control can be performed immediately. However, when the proximity sensor 105 detects that the hand is moved in a negative direction with respect to a specific direction (for example, the vertical direction), the processor 121 holds the determination for a predetermined time and waits for the next gesture operation. During the predetermined time, when the proximity sensor 105 detects a gesture operation in a direction other than the positive direction in the specific direction (separate direction: including the negative direction in the specific direction), the determination control becomes a problem.

  In such a case, the processor 121 can execute one of two types of determination control. First, when a gesture operation in a separate direction is performed within a predetermined time, the processor 121 determines that the suspended gesture operation in the negative direction is invalidated (in other words, the hand HD moves in the negative direction). The other control is a determination control that immediately confirms the suspended gesture operation in the negative direction. These can be selected according to preference. In any case, the processor 121 cancels the state of waiting for the gesture operation, and continues the determination control based on a new gesture operation in another direction.

  FIG. 15 is a flowchart for executing the input method by the gesture operation according to the above contents, and this input method can be executed by the input program stored in the ROM 204. Here, it is assumed that the processor 121 predetermines a specific direction of each gesture operation and its positive and negative directions. In the following embodiments, the specific direction is the vertical direction, the positive direction is the direction from top to bottom, and the negative direction is the direction from bottom to top. Other than the vertical direction, no specific direction is set. However, the two directions of the up and down direction and the left and right direction may be specified directions, and three or more directions may be specified directions.

  In step S100 of FIG. 15, the user US holds the indicator (here, the hand HD) and moves it within the detection area SA, and in step S101, the proximity sensor 105 reacts and outputs a signal. Next, in step S102, if the processor 121 determines the moving direction of the indicator from the output of the proximity sensor 105 and determines that it is not the specific direction, in step S110, the input content is immediately determined based on the gesture operation. On the other hand, if it is determined that the moving direction of the indicator is the specific direction, in the subsequent step S103, the processor 121 determines that the moving direction of the indicator is the positive direction (up → down) of the specific direction from the output of the proximity sensor 105. If it is recognized that there is, the input content is determined in step S108 as if a positive gesture operation in a specific direction was performed.

  On the other hand, if the processor 121 determines in step S103 that the moving direction of the indicator is a specific direction but not a positive direction (up → down) (that is, a negative direction), a new gesture operation is performed in step S104. The next gesture operation is waited for a predetermined time while confirming whether or not (No in Step S105). That is, at this time, the processor 121 suspends the determination of the input content of the gesture operation in the negative direction. On the other hand, if it is determined that the next gesture operation is not performed even after a predetermined time has elapsed (Yes in step S105), the processor 121 determines in step S107 that the gesture operation in the negative direction of the specific direction that has been suspended is held. It is assumed that the input content corresponding to is determined, or the input content is confirmed, or a negative gesture operation in a specific direction is not performed.

  On the other hand, if it is determined in step S104 that the next gesture operation has been performed within the predetermined time, the processor 121 determines whether or not the gesture operation in the negative direction (down → up) in the specific direction has been performed in step S106. If it is determined that it has been performed, the input content is determined in step S108 as a negative gesture operation in a specific direction that has been suspended.

  On the other hand, if it is determined in step S106 that the gesture operation in the positive direction (up to down) in the specific direction has not been performed, the processor 121 responds to the gesture operation in the negative direction in the specific direction that has been suspended in step S109. It is assumed that the input content is confirmed as having been determined, or that a negative gesture operation in a specific direction has not been performed.

  FIG. 16 is a time chart showing an example of (a) hand movement, (b) proximity sensor output, and (c) screen display control. Here, the specific direction is the vertical direction, the direction from the top to the bottom is the positive direction, and the direction from the bottom to the top is the negative direction. The output of the proximity sensor is assumed to be on within the detection area and off outside the detection area. FIG. 17 is a diagram illustrating an example of an image displayed on the image forming unit 104A.

  In FIG. 16, when the hand moves from the bottom to the top during the time T0 to T1, the processor 121 does not determine the input content, so the home screen (the screen on which the image G1 shown in FIG. ) Will continue to be displayed. However, if the hand moves from top to bottom during time T1 to T2, the processor 121 determines the input content at time T2 when the hand moves out of the detection area, and the first screen from the home screen. The display is changed to (a screen in which the image G2 shown in FIG. 17B is located at the center).

  Further, when the hand moves from the bottom to the top during the time T2 to T3, the processor 121 does not determine the input content, so the home screen (screen in which the image G2 shown in FIG. 17B is located at the center) is displayed. Keep doing. However, if the hand moves from top to bottom at times T3 to T4, the processor 121 determines the input content at time T4 when the hand moves out of the detection area, and the second screen from the home screen (FIG. The display is changed to a screen in which an image G3 shown in FIG.

  Thereafter, if the hand moves from top to bottom during time T4 to T5, the processor 121 determines the input content at time T5 when the hand moves out of the detection area, and the first screen from the home screen. The display is changed to (a screen in which the image G2 shown in FIG. 17B is located at the center).

  As shown in FIG. 17A, if the hand is moved from the left to the right while the image G1 of various adjustments (Settings) is displayed at the center, the OK button B1 is detected by gesture detection by the processor 121. Is input, and various adjustments can be made. Also, as shown in FIG. 17B, when the hand is moved from left to right with the brightness image G2 displayed in the center, the OK button B1 is detected by gesture detection by the processor 121. Input is executed, and brightness adjustment of the image forming unit 104A can be performed. Further, as shown in FIG. 17 (c), if the hand is moved from left to right in the state where the image G3 of the speaker (Sound) is displayed in the center, the OK button B1 is detected by gesture detection by the processor 121. The input is executed, and the volume of the speaker 111A can be adjusted.

  Note that the input direction of the gesture operation changes depending on the direction in which the proximity sensor 105 is facing. For example, if the user US moves his hand in the direction from the chin to the forehead in front of his eyes while looking up, the processor 121 detects this as a negative direction in a specific direction and performs a gesture. Although the operation is put on hold, there is a case where the hold is not necessarily required. Therefore, for example, when the specific direction is the up-down direction, the processor 121 described above only when the acceleration sensor 110A or the like detects that the face of the user US wearing the HMD 100 is almost directly facing the vertical plane. You may make it perform control.

  The present invention is not limited to the embodiments described in the specification, and other embodiments and modifications are included for those skilled in the art from the embodiments and technical ideas described in the present specification. it is obvious. The description and the embodiments are for illustrative purposes only, and the scope of the present invention is indicated by the following claims.

100 HMD
101 Frame 101a Front part 101b Side part 101c Side part 101d Long hole 101e Long hole 102 Eyeglass lens 103 Main body part 104 Display 104A Image forming part 104B Image display part 104DR Display control part 104a Light source 104b Unidirectional diffuser 104c Condensing lens 104d Display element 104f Eyepiece prism 104g Deflection prism 104h Hologram optical element 104i Screen 105 Proximity sensor 105a Light receiving part 106 Camera 106a Lens 107 Right sub-main part 107a Protrusion 108 Left sub-main part 108a Protrusion 109 Acceleration sensor 110 Gyro 111A Speaker 111B Microphone 111C Earphone 113 Power supply circuit 121 Processor 122 Operation unit 123 GPS reception unit 124 Communication unit 125 ROM
126 RAM
127 Battery 128 Battery control unit 129 Storage device 130 Power supply circuit CD Code CTU Control unit HD Hand HS Wiring PL1 Base end surface PL2 Inner surface PL3 Outer surface PL4 Inclined surface PL5 Inclined surface RA-RD Light receiving area SA Detection area US User

Claims (13)

A detection device having a detection area, detecting a direction in which the indicator moved by the user moves in the detection area, and generating an output corresponding thereto;
A control device for determining input contents according to the output of the detection device, and an input device comprising:
When the detection device detects that the indicator has moved in the positive direction of the specific direction within the detection area, the control device determines an input content corresponding to the movement of the indicator in the positive direction of the specific direction. And
The control device has moved in the negative direction of the specific direction when the detection device detects that the indicator has moved in the negative direction opposite to the positive direction of the specific direction in the detection area. When the detection device detects that the indicator has moved in the positive direction of the specific direction within the detection area within a predetermined time, the input content corresponding to is suspended. An input device for determining an input content corresponding to the movement in the negative direction of the specific direction.   The control device holds the indicator when the detection device does not detect that the indicator has moved in the positive direction of the specific direction within the detection area during the predetermined time. The input device according to claim 1, wherein an input content corresponding to a movement in a negative direction of the specific direction is determined.   When the detection device does not detect that the indicator has moved in the positive direction of the specific direction within the detection area during the predetermined time, the control device detects that the indicator is in the specific direction. The input device according to claim 1, wherein the output of the detection device according to the movement in the negative direction is not used for determining the input content.   The control device is put on hold when the detection device detects that the indicator has moved in a separate direction other than the positive direction in the specific direction within the detection region during the predetermined time. The input device according to claim 1, wherein the input content corresponding to the movement of the indicator in the negative direction of the specific direction is determined.   When the detection device detects that the indicator has moved in a separate direction other than the positive direction in the specific direction within the detection region during the predetermined time, the indicator is The input device according to claim 1, wherein the output of the detection device according to the movement in the negative direction of the specific direction is not used for determining the input content.   The specific direction is a vertical direction in the detection region, the positive direction is a direction from top to bottom in the detection region, and the negative direction is a direction from bottom to top in the detection region. Item 6. The input device according to any one of Items 1 to 5.   The input device according to claim 1, wherein the specific direction is a left-right direction in the detection region.   The input device according to claim 1, wherein the detection device is a proximity sensor that detects an object in a proximity range and generates an output.   An electronic device comprising the input device according to claim 1 and attached to an operator's body.   The electronic device includes a display member having a screen capable of displaying an image and a support member attached to the head of the operator, and the support member positions the display member in front of the operator's eyes. The electronic apparatus according to claim 9, wherein the input member is supported so that a detection area of the input device is in front of the operator's eyes.   An electronic apparatus comprising the input device according to claim 1 and an image display unit, and operating a display screen of the image display unit by an output from the input device. An electronic device having a detection area, including a detection device that detects a direction in which the indicator moved by the user moves in the detection region and generates an output corresponding thereto, and determines input contents according to the output of the detection device A device input method,
When the detection device detects that the indicator has moved in the positive direction of the specific direction within the detection area, the input content corresponding to the movement in the positive direction of the specific direction is determined, and the indicator Is detected in the negative direction opposite to the positive direction of the specific direction in the detection area, the input content corresponding to the movement in the negative direction of the specific direction is determined. When the detection device detects that the indicator has moved in the positive direction of the specific direction within the detection area within a predetermined time after being put on hold, the negative direction of the specific direction put on hold The input method of the electronic device which determines the input content corresponding to having moved to. An electronic device having a detection area, including a detection device that detects a direction in which the indicator moved by the user moves in the detection region and generates an output corresponding thereto, and determines input contents according to the output of the detection device A device input program,
When the detection device detects that the indicator has moved in the positive direction of the specific direction within the detection area, the input content corresponding to the movement in the positive direction of the specific direction is determined, and the indicator Is detected in the negative direction opposite to the positive direction of the specific direction in the detection area, the input content corresponding to the movement in the negative direction of the specific direction is determined. When the detection device detects that the indicator has moved in the positive direction of the specific direction within the detection area within a predetermined time after being put on hold, the negative direction of the specific direction put on hold An input program for an electronic device that determines the input content corresponding to the movement to.

Images