WO2016190057A1

WO2016190057A1 - Wearable electronic device and gesture detection method for wearable electronic device

Info

Publication number: WO2016190057A1
Application number: PCT/JP2016/063617
Authority: WO
Inventors: 卓天野; 軌行石井
Original assignee: コニカミノルタ株式会社
Priority date: 2015-05-22
Filing date: 2016-05-06
Publication date: 2016-12-01

Abstract

In this wearable electronic device and gesture detection method therefor, first movement of a mounted member mounted on a first region of a living body is measured, second movement of a second region different from the first region on the living body or a pointing body provided at the second region is measured, and on the basis of the results of the first and second measurements, gestures based on the second movement of the second region or the pointing body are recognized.

Description

Wearable electronic device and gesture detection method for wearable electronic device

The present invention relates to a wearable electronic device that can be worn, and particularly relates to a wearable electronic device that can recognize a predetermined gesture. The present invention relates to a gesture detection method for wearable electronic devices.

2. Description of the Related Art In recent years, mobile terminal devices such as smart phones (multifunctional mobile phones) that have been rapidly developed are generally equipped with a touch panel, and a user can display a desired image or input information, for example. In addition, the necessary input operation is performed using the touch panel. However, there are cases where it is desired to perform the input operation without touching the touch panel, such as when the user's hand is wet or dirty. For this reason, a device capable of performing the input operation by one gesture of gesture transmission means or a gesture that is a hand gesture has been researched and developed. For example, there is a technique disclosed in Patent Document 1.

The mobile computing device disclosed in this document is configured to acquire data related to a three-dimensional user movement comprising an infrared (IR) light emitting diode (LED) and an IR proximity sensor. A sensor system, communicatively coupled to the sensor system, and data clarity related to the three-dimensional user movement acquired by the sensor system and correct input gesture identification of the three-dimensional user movement. Configured to identify a property of the device that is indicative of a probability and regulate power consumption of at least one of the IR LED or the IR proximity sensor of the sensor system based on the device property A sensor control device module.

By the way, the sensor system disclosed in Patent Document 1 is based on the assumption that the IR proximity sensor is stationary, and the IR proximity sensor is used to perform a three-dimensional user movement with respect to the IR proximity sensor, that is, a gesture. Relevant data is acquired. For this reason, if the IR proximity sensor moves, it is difficult to detect a gesture by the IR proximity sensor.

Special table 2013-534209

The present invention has been made in view of the above-described circumstances, and an object thereof is to provide a wearable electronic device capable of detecting a gesture even when a wearing part moves, and a gesture detection method for the wearable electronic device. It is to be.

In the wearable electronic device and the gesture detection method thereof according to the present invention, the first movement of the mounting member to be mounted on the first part in the living body is measured and mounted on the mounting member, which is different from the first part in the living body. The second movement of the indicator provided in the second part or the second part is measured, and based on the first and second measurement results, the second movement of the second part or the indicator is determined. Gesture is recognized. Therefore, the wearable electronic device and the gesture detection method for the wearable electronic device according to the present invention can detect a gesture even if the wearing site moves.

The above and other objects, features and advantages of the present invention will become apparent from the following detailed description and the accompanying drawings.

It is a perspective view which shows the structural structure of the wearable electronic device in embodiment. It is a front view which shows the structural structure of the said wearable electronic device. It is a top view which shows the structural structure of the said wearable electronic device. It is a schematic sectional drawing which shows the structure of the display unit in the said wearable electronic device. It is a block diagram which shows the electrical structure of the said wearable electronic device. It is a figure which shows the structure of the proximity sensor as an example of the 2nd motion measurement part in the said wearable electronic device. It is a front view at the time of mounting | wearing with the said wearable electronic device. It is the side view at the time of mounting | wearing with the said wearable electronic device, and a partial top view. It is a figure which shows an example of the image which a user visually recognizes through a see-through type image display part. It is a figure which shows an example of the output of the proximity sensor as an example of the 2nd motion measurement part in the said wearable electronic device. It is a flowchart which shows the gesture detection process (main routine) in the said wearable electronic device. It is a flowchart which shows the gesture determination process (subroutine) in the said gesture detection process of the said wearable electronic device.

Hereinafter, an embodiment according to the present invention will be described with reference to the drawings. In addition, the structure which attached | subjected the same code | symbol in each figure shows that it is the same structure, The description is abbreviate | omitted suitably. In this specification, when referring generically, it shows with the reference symbol which abbreviate | omitted the suffix, and when referring to an individual structure, it shows with the reference symbol which attached the suffix.

The wearable electronic device in the present embodiment is an electronic device that can be worn and can detect a predetermined gesture, and includes a mounting member for mounting on a predetermined first part of a living body, and a first of the mounting member A first movement measuring unit for measuring movement; and a second indicator mounted on the mounting member and used for performing a gesture provided in the second part or the second part different from the first part in the living body. A second motion measuring unit for measuring a motion; and a gesture processing unit for recognizing a gesture based on the second motion based on the first and second measurement results measured by the first and second motion measuring units. Such a wearable electronic device may be an electronic device for any purpose, but here, as an example, a member (head) for mounting the mounting member on the head with the first part as the head More specifically, the case where it is a so-called head mounted display (HMD) will be described below.

FIG. 1 is a perspective view showing a structural configuration of a wearable electronic device according to an embodiment. FIG. 2 is a front view illustrating a structural configuration of the wearable electronic device according to the embodiment. FIG. 3 is a top view illustrating a structural configuration of the wearable electronic device according to the embodiment. FIG. 4 is a schematic cross-sectional view illustrating a configuration of a display unit in the wearable electronic device of the embodiment. FIG. 5 is a block diagram illustrating an electrical configuration of the wearable electronic device according to the embodiment. FIG. 6 is a diagram illustrating a configuration of a proximity sensor as an example of a second motion measurement unit in the wearable electronic device of the embodiment. Hereinafter, the right side and the left side of the HMD 100 refer to the right side and the left side for the user wearing the HMD 100. As shown in FIGS. 1 and 2, the direction from the right side to the left side is the XYZ orthogonal. A coordinate system is set, and the X axis, Y axis, Z axis, X direction, Y direction, and Z direction are used as appropriate.

First, the structural configuration of the HMD 100 will be described. As shown in FIGS. 1 to 3, the HMD 100 according to the present embodiment includes a frame 101 that is an example of a head mounting member to be mounted on the head. A frame 101 that is substantially U-shaped when viewed from above includes a front part 101a to which two spectacle lenses 102 are attached, and

side parts

101b and 101c that extend rearward (Y direction) from both ends of the front part 101a. . The two spectacle lenses 102 attached to the frame 101 may or may not have refractive power (optical power, reciprocal of focal length).

The cylindrical main body 103 is fixed to the front part 101 a of the frame 101 at the upper part of the right eyeglass lens 102 (which may be the left side according to the user's dominant eye etc.). The main body 103 is provided with a display unit 104. In the main body 103, a display control unit 104DR (see FIG. 5 described later) that controls display of the display unit 104 based on an instruction from the control processing unit 121 described later is disposed. Note that a display unit may be disposed in front of both eyes as necessary.

In FIG. 4, the display unit 104 includes an image forming unit 104A and an image display unit 104B. The image forming unit 104A is incorporated in the main body unit 103, and includes a light source 104a, a one-way diffusing plate 104b, a condenser lens 104c, and a display element 104d. On the other hand, the image display unit 104B, which is a so-called see-through type display member, is disposed on the entire plate so as to extend downward from the main body unit 103 and extend in parallel to one eyeglass lens 102 (see FIG. 1). The eyepiece prism 104f, the deflecting prism 104g, and the hologram optical element 104h.

The light source 104a has a function of illuminating the display element 104d. For example, the peak wavelength of light intensity and the half width of the light intensity are 462 ± 12 nm (blue light (B light)), 525 ± 17 nm (green light (G light )), 635 ± 11 nm (red light (R light)), and is composed of RGB integrated light emitting diodes (LEDs) that emit light in three wavelength bands.

The display element 104d displays an image by modulating the light emitted from the light source 104a in accordance with image data, and is configured by a transmissive liquid crystal display element having pixels that serve as light transmitting regions in a matrix. Is done. Note that the display element 104d may be of a reflective type.

The eyepiece prism 104f totally reflects the image light from the display element 104d incident through the base end face PL1 by the opposed parallel inner side face PL2 and outer side face PL3, and passes through the hologram optical element 104h to the user's pupil. On the other hand, it transmits external light and guides it to the user's pupil, and is formed of, for example, an acrylic resin together with the deflecting prism 104g. The eyepiece prism 104f and the deflection prism 104g are joined by an adhesive with the hologram optical element 104h sandwiched between inclined surfaces PL4 and PL5 inclined with respect to the inner surface PL2 and the outer surface PL3.

The deflection prism 104g is joined to the eyepiece prism 104f, and becomes a substantially parallel flat plate integrated with the eyepiece prism 104f. In addition, if the spectacle lens 102 (refer FIG. 1) is mounted | worn between the display unit 104 and a user's pupil, even the user who uses normal spectacles can observe an image.

The hologram optical element 104h diffracts and reflects the image light (light having a wavelength corresponding to the three primary colors) emitted from the display element 104d, guides it to the pupil B, enlarges the image displayed on the display element 104d, and enlarges the user's pupil. It is a volume phase type reflection hologram guided as a virtual image. The hologram optical element 104h has, for example, three wavelength ranges of 465 ± 5 nm (B light), 521 ± 5 nm (G light), and 634 ± 5 nm (R light) with a peak wavelength of diffraction efficiency and a wavelength width of half the diffraction efficiency. The light is diffracted (reflected). Here, the peak wavelength of diffraction efficiency is the wavelength at which the diffraction efficiency reaches a peak, and the wavelength width at half maximum of the diffraction efficiency is the wavelength width at which the diffraction efficiency is at half maximum of the diffraction efficiency peak. is there.

In the display unit 104 having such a configuration, light emitted from the light source 104a is diffused by the unidirectional diffusion plate 104b, condensed by the condenser lens 104c, and incident on the display element 104d. The light incident on the display element 104d is modulated for each pixel based on the image data input from the display control unit 104DR, and is emitted as image light. Thereby, a color image is displayed on the display element 104d. The image light from the display element 104d enters the eyepiece prism 104f from its base end face PL1, is totally reflected a plurality of times by the inner side face PL2 and the outer side face PL3, and enters the hologram optical element 104h. The light incident on the hologram optical element 104h is reflected there, passes through the inner side surface PL2, and reaches the pupil B. At the position of the pupil B, the user can observe an enlarged virtual image of the image displayed on the display element 104d, and can visually recognize it as a screen formed on the image display unit 104B.

On the other hand, since the eyepiece prism 104f, the deflecting prism 104g, and the hologram optical element 104h transmit almost all of the external light, the user can observe the external image (real image) through these. Therefore, the virtual image of the image displayed on the display element 104d is observed so as to overlap with a part of the external image. In this way, the user of the HMD 100 can simultaneously observe the image provided from the display element 104d and the external image via the hologram optical element 104h. In addition, when the display unit 104 is in a non-display state, the image display unit 104B is transparent and can observe only an external image. In this embodiment, the display unit is configured by combining a light source, a liquid crystal display element, and an optical system. However, instead of the combination of the light source and the liquid crystal display element, a self-luminous display element (for example, an organic EL display) is used. Element) may be used. Further, instead of a combination of a light source, a liquid crystal display element, and an optical system, a transmissive organic EL display panel having transparency in a non-light emitting state may be used.

Returning to FIGS. 1 to 3, on the front surface of the main body 103, a second motion measuring unit (for example, a proximity sensor) 105 disposed near the center of the frame 101 and a camera 106 disposed near the side. The lens 106a and the illuminance sensor 107 disposed between the second motion measuring unit 105 and the lens 106a are provided so as to face forward. Accordingly, in the example shown in FIGS. 1 to 3, the measurement direction of the second motion measurement unit 105, the optical axis of the camera 106, and the measurement direction of the illuminance sensor 107 are the same. Here, the measurement direction of the second motion measuring unit 105, the optical axis of the camera 106, and the measurement direction of the illuminance sensor 107 are the same as the central axis of the detection range of the proximity sensor 105A, the optical axis of the camera 106, and the illuminance sensor 107. A position where not only when the central axes of the measurement range are parallel, but also when the three axes are slightly crossed, an output with a tendency similar to that when the three axes are parallel is obtained from the three parties. The case where these are arranged in a relationship is also included.

The right sub-body portion 108-R is attached to the right side portion 101b of the frame 101, and the left sub-body portion 108-L is attached to the left side portion 101c of the frame 101. The right sub-main body portion 108-R and the left sub-main body portion 108-L have an elongated plate shape, and have elongated projections 108a-R and 108a-L, respectively, inside. By engaging the elongated protrusion 108a-R with the elongated hole 101d of the side portion 101b of the frame 101, the right sub-main body portion 108-R is attached to the frame 101 in a positioned state, and the elongated protrusion 108a-L is By engaging with the long hole 101e of the side portion 101c of the frame 101, the left sub-main body portion 108-L is attached to the frame 101 in a positioned state.

A geomagnetic sensor 109 (see FIG. 5) and a first motion measuring unit (for example, a gyroscope and an acceleration sensor) 110 (see FIG. 5) are arranged in the right sub-main body portion 108-R. A speaker (or earphone) 111A and a microphone 111B (see FIG. 5) are arranged in the left sub-main body portion 108-L. The main main body 103 and the right sub main body 108-R are connected so as to be able to transmit signals through a wiring HS, and the main main body 103 and the left sub main body 108-L can transmit signals through a wiring (not shown). It is connected to the. As schematically shown in FIG. 3, the right sub-main body 108-R is connected to the control unit CTU via a cord CD extending from the rear end thereof. Note that the HMD 100 may be configured to be operated by voice based on an output signal generated from the microphone 111B according to the input voice. Further, the main main body 103 and the left sub main body 108 may be configured to be wirelessly connected.

Next, the electrical configuration of the HMD 100 will be described. 5, the HMD 100 includes a control unit CTU, a camera 106, a geomagnetic sensor 109, a first motion measuring unit 110, a second motion measuring unit 105, a microphone 111B, an illuminance sensor 107, and an image forming unit 104A. And a display control unit 104DR and a speaker 111A. The control unit CTU includes a control processing unit 121, an operation unit 122, a GPS reception unit 123, a communication unit 124, a storage unit 125, a battery 126, and a power supply circuit 127.

The camera 106 is an apparatus that is connected to the control processing unit 121 and generates an image of a subject under the control of the control processing unit 121. The camera 106 is, for example, an image forming optical system that forms an optical image of a subject on a predetermined image forming surface, and a light receiving surface that matches the image forming surface. An image sensor that converts the image sensor into a digital signal processor (DSP) that performs known image processing on the output of the image sensor to generate an image (image data). The imaging optical system includes one or more lenses, and includes the lens 106a as one of them. The camera 106 outputs the generated image data to the control processing unit 121.

The geomagnetic sensor 109 is a circuit that is connected to the control processing unit 121 and measures the front (−Y direction) orientation in the HMD 100 by measuring the earth magnetism. The geomagnetic sensor 109 outputs the measured orientation to the control processing unit 121.

The first motion measuring unit 110 is connected to the control processing unit 121 and is a device for measuring the motion of the frame 101 which is an example of the head-mounted member. The first motion measuring unit 110 is mounted on the frame 101. The first motion measurement unit 110 outputs the measurement result to the control processing unit 121. More specifically, the first motion measurement unit 110 is, for example, a gyroscope and an acceleration sensor 110a, and is disposed in the right sub-main body unit 108-R as described above. The gyroscope and acceleration sensor 110a is configured to determine the angular velocity of the roll around the X axis, the angular velocity of the pitch around the Y axis, the angular velocity of the yaw around the Z axis, the acceleration in the X direction, the acceleration in the Y direction, and the Z according to the posture of the frame 101. It is a circuit that measures each acceleration in the direction. The gyro and acceleration sensor 110 a outputs the measured angular velocities and accelerations to the control processing unit 121. Note that the gyro and acceleration sensor 110a may be a six-axis sensor in which these are integrated.

The second movement measuring unit 105 is connected to the control processing unit 121, and is a second indicator for performing a second movement of a second part different from the first part or a gesture provided in the second part. It is a device for measuring movement. In this embodiment, since the first part is the head of a living body, the second part in the living body is a part different from the head, for example, a hand or a finger. The indicator provided in the second part of the living body is, for example, a rod-shaped member (for example, a pen or a pointing rod) gripped by a user's hand, or a rod-shaped member mounted on a user's finger or arm with a mounting member. These members. In the present embodiment, the second motion measuring unit 105 further detects the presence / absence of the second part. The second motion measuring unit 105 is mounted on the frame 101 which is an example of the head mounting member. The second motion measuring unit 105 outputs the measurement result to the control processing unit 121. More specifically, the second motion measuring unit 105 is, for example, a proximity sensor 105a and is mounted on the main body 103 fixed to the front part 101a of the frame 101 as described above.

In the present specification, the “proximity sensor” refers to a proximity range in front of the detection surface of the proximity sensor in order to detect that an object, for example, a part of a human body (such as a hand or a finger) is close to the user's eyes. The signal is output by detecting whether or not it exists within the detection area. The proximity range may be set as appropriate according to the operator's characteristics and preferences. For example, the proximity range from the detection surface of the proximity sensor may be within a range of 200 mm. If the distance from the proximity sensor is within 200 mm, the user can easily put the palm and fingers into and out of the user's field of view with the arm bent, so that the user can easily operate with gestures using the hands and fingers. In addition, the possibility of erroneous detection of a human body or furniture other than the user is reduced.

There are two types of proximity sensors: passive type and active type. A passive proximity sensor has a detection unit that detects invisible light and electromagnetic waves emitted from an object when the object approaches. As a passive proximity sensor, there are a pyroelectric sensor that detects invisible light such as infrared rays emitted from an approaching human body, an electrostatic capacitance sensor that detects a change in electrostatic capacitance between the approaching human body, and the like. The active proximity sensor includes an invisible light and sound wave projection unit, and a detection unit that receives the invisible light and sound wave reflected and returned from the object. Active proximity sensors include infrared sensors that project infrared rays and receive infrared rays reflected by objects, laser sensors that project laser beams and receive laser beams reflected by objects, and project ultrasonic waves. Then, there is an ultrasonic sensor that receives ultrasonic waves reflected by an object. Note that a passive proximity sensor does not need to project energy toward an object, and thus has excellent low power consumption. An active proximity sensor is easy to improve the certainty of detection. For example, even when a user wears a glove that does not transmit detection light emitted from a human body such as infrared light, Can detect movement. A plurality of types of proximity sensors may be combined.

In this embodiment, a pyroelectric sensor including a plurality of pyroelectric elements arranged in a two-dimensional matrix is used as the proximity sensor 105a. In FIG. 6, the proximity sensor 105a includes four pyroelectric elements RA, RB, RC, and RD arranged in 2 rows and 2 columns, and detects invisible light such as infrared light emitted from the human body. Light is received as light, and a corresponding signal is output from each pyroelectric element RA to RD. The outputs of the pyroelectric elements RA to RD vary in intensity according to the distance from the light receiving surface of the proximity sensor 105a to the object, and the intensity increases as the distance decreases. The proximity sensor 105a outputs each output of the pyroelectric elements RA to RD to the control processing unit 121.

The microphone 111B is a circuit that is connected to the control processing unit 121 and converts acoustic vibration of sound into an electric signal. The microphone 111 </ b> B outputs the converted electric signal representing the external sound to the control processing unit 121.

The illuminance sensor 107 is a circuit that is connected to the control processing unit 121 and measures illuminance. The illuminance sensor 107 outputs the measured illuminance to the control processing unit 121. The illuminance sensor 107 is, for example, a photodiode that outputs a current having a magnitude corresponding to the light intensity of incident light by photoelectric conversion, and an IV conversion circuit that converts the current value of the photodiode into a voltage value. The peripheral circuit is provided.

The display control unit 104DR is a circuit that is connected to the control processing unit 121 and causes the image forming unit 104A to form an image by controlling the image forming unit 104A according to the control of the control processing unit 121. The image forming unit 104A is as described above.

The speaker 111 </ b> A is a circuit that is connected to the control processing unit 121 and generates and outputs a sound corresponding to an electric signal representing a sound according to the control of the control processing unit 121.

The operation unit 122 is connected to the control processing unit 121 and is a device that inputs a predetermined instruction, such as power on / off, to the HMD 100, for example, one or a plurality of switches assigned a predetermined function Etc.

The GPS receiving unit 123 is connected to the control processing unit 121, and is a device that measures the position of the HDM 100 by a satellite positioning system for measuring the current position on the earth according to the control of the control processing unit 121. The result (latitude X, longitude Y, altitude Z) is output to the control processing unit 121. The GPS receiving unit 123 may be a GPS having a correction function for correcting an error such as DGSP (Differential GSP).

The communication unit 124 is a circuit that is connected to the control processing unit 121 and inputs / outputs data to / from an external device according to the control of the control processing unit 121. For example, an RS232C interface circuit that is a serial communication method, Bluetooth An interface circuit using the (registered trademark) standard, an interface circuit performing infrared communication such as an IrDA (Infrared Data Association) standard, and an interface circuit using the USB (Universal Serial Bus) standard.

The communication unit 124 is a communication card or the like that communicates by wire or wireless, and may communicate with an external device such as a server device via a communication network such as an Ethernet environment (Ethernet is a registered trademark). ). Such a communication unit 124 generates a communication signal containing data to be transferred input from the control processing unit 121 according to a communication protocol used in the communication network, and generates the generated communication signal via the communication network. To the external device. The communication unit 124 receives a communication signal from an external device via the communication network, extracts data from the received communication signal, and converts the extracted data into data in a format that can be processed by the control processing unit 121. Output to the control processing unit 121. For example, the communication unit 124 transmits and receives communication signals according to the Wi-Fi (Wireless Fidelity) standard, which is one of the wireless LAN standards, and Wi-Fi Module (communication card) compatible with IEEE802.11b / g / n. It is configured with.

The battery 126 is a battery that accumulates electric power and supplies the electric power. The battery 126 may be a primary battery or a secondary battery. The power supply circuit 127 is a circuit that supplies power supplied from the battery 126 to each part of the HMD 100 that requires power at a voltage corresponding to each part.

The storage unit 125 is a circuit that is connected to the control processing unit 121 and stores various predetermined programs and various predetermined data under the control of the control processing unit 121. Examples of the various predetermined programs include a control program that controls each part of the HMD 100 according to the function of each part, and first and second measurement results measured by the first and second motion measuring units 110 and 105. And a control processing program such as a gesture processing program for recognizing a gesture by the second movement of the second part based on the above. The various predetermined data includes data necessary for controlling the HMD 100. The storage unit 125 includes, for example, a ROM (Read Only Memory) that is a nonvolatile storage element, an EEPROM (Electrically Erasable Programmable Read Only Memory) that is a rewritable nonvolatile storage element, and the like. The storage unit 125 includes a RAM (Random Access Memory) that serves as a working memory of the control processing unit 121 that stores data generated during the execution of the predetermined program.

The control processing unit 121 controls each part of the HMD 100 according to the function of each part, and based on the first and second measurement results measured by the first and second motion measuring units 110 and 105, A gesture based on the second movement is recognized, and processing corresponding to the recognized gesture is executed. The control processing unit 121 includes, for example, a CPU (Central Processing Unit) and its peripheral circuits. In the control processing unit 121, a control processing program is executed, so that a control unit 1211, a gesture processing unit 1212, and a detection result utilization unit 1213 are functionally configured. Part or all of the control unit 1211, the gesture processing unit 1212, and the detection result utilization unit 1213 may be configured by hardware.

The control unit 1211 controls each unit of the HMD 100 according to the function of each unit.

The gesture processing unit 1212 recognizes a gesture caused by the second movement of the second part based on the first and second measurement results measured by the first and second movement measuring units 110 and 105. More specifically, in the present embodiment, the gesture processing unit 1212 includes the first measurement result of the gyroscope and the acceleration sensor 110a and a plurality of pyroelectric elements in the proximity sensor 105a. Based on each output of RD, a predetermined gesture set in advance by the second movement of the hand or fingers is determined. The gesture processing unit 1212 notifies the detection result utilization unit 1213 of the determination result (detection result). Preferably, the gesture processing unit 1212 determines the first motion measurement unit 110 (gyroscope and acceleration sensor 110a in this embodiment) from the second measurement result measured by the second motion measurement unit 105 (proximity sensor 105a in this embodiment). The first movement component due to the first movement is removed based on the first measurement result measured in step, and the second portion of the second part is removed based on the second measurement result from which the first movement component is removed. Recognize movement gestures.

The gesture processing unit 1212 starts its operation when the second motion measuring unit 105 detects the presence of the second part. More specifically, in the present embodiment, the gesture processing unit 1212 starts its operation by switching to the active state when the second motion measuring unit 105 detects the presence of the second part from the sleep state. The gesture processing unit 1212 may start its operation by turning on the power supply when the second motion measurement unit 105 detects the presence of the second part from the state where the power supply is turned off.

In addition, in this embodiment, the gesture processing unit 1212 is mounted not on the frame 101 but on the control unit CTU that is separate from the frame 101.

The detection result utilization unit 1213 executes predetermined processing based on the determination result of the gesture processing unit 1212. For example, when the determination result of the gesture processing unit 1212 is a so-called “flick”, the detection result using unit 1213 performs the second operation from the first image formed in the image forming unit 104A by the display control of the display control unit 104DR. Change the display to turn the page. Further, for example, when the determination result of the gesture processing unit 1212 is a so-called “slide”, the detection result using unit 1213 moves the image formed in the image forming unit 104A by the display control of the display control unit 104DR. Change the display to.

First, the basic operation of detecting a gesture in the HMD 100 will be described. FIG. 7 is a front view when the wearable electronic device of the embodiment is mounted. FIG. 8 is a side view and a partial top view when the wearable electronic device of the embodiment is mounted. FIG. 8 also shows the hand HD of the user US. 8A is a side view, and FIG. 8B is a partial top view. FIG. 9 is a diagram illustrating an example of an image visually recognized by the user through the see-through image display unit. FIG. 10 is a diagram illustrating an example of the output of the proximity sensor in the wearable electronic device of the embodiment. 10A shows the output of the pyroelectric element RA, FIG. 10B shows the output of the pyroelectric element RB, FIG. 10C shows the output of the pyroelectric element RC, and FIG. 10D shows the output of the pyroelectric element RD. Show. The horizontal axis of each figure in FIG. 10 is time, and the vertical axis thereof is intensity (output level). Here, the gesture operation is an operation in which at least the hand HD or the finger of the user US enters or leaves the detection area of the proximity sensor 105a, and the gesture processing unit 1212 of the control processing unit 121 of the HMD 100 via the proximity sensor 105a. Can be detected.

As shown in FIG. 9, the screen 104i of the image display unit 104B is arranged so as to overlap the effective visual field EV of the user's eye facing the image display unit 104B (here, positioned in the effective visual field EV). The The detection area SA of the proximity sensor 105a is in the visual field of the user's eye facing the image display unit 104B. Preferably, the detection area SA is located within the stable field of view of the user's eye or the field inside thereof (within about 90 ° horizontal and within about 70 ° vertical), and more preferably located inside the stable field of view. The proximity sensor 105a may be installed with its arrangement and orientation adjusted so as to overlap with the effective visual field EV or the inner visual field (horizontal within about 30 °, vertical within about 20 °).

FIG. 9 shows an example in which the detection area SA overlaps the screen 104i. In this way, by setting the detection area SA of the proximity sensor 105a within the visual field of the eye of the user US while the user US is wearing the frame 101 that is the head mounting member on the head, the screen While observing the hand HD through 104i, the approach and retraction of the hand to the detection area SA of the proximity sensor 105a can be reliably visually recognized without moving the eye. In particular, by setting the detection area SA of the proximity sensor 105a within the stable visual field or the inside visual field, the gesture operation can be performed reliably while recognizing the detection area SA even when the user observes the screen. . Further, by making the detection area SA of the proximity sensor 105a within the effective visual field EV or the visual field inside the effective visual field EV, the gesture operation can be performed more reliably. If the detection area SA overlaps the screen 104i, the gesture operation can be performed more reliably. When the proximity sensor 105a includes a plurality of pyroelectric elements RA to RD as in the present embodiment, the entire light receiving area of the plurality of pyroelectric elements RA to RD is regarded as one light receiving unit, and The maximum detection range shall be regarded as the detection area. As shown in FIG. 9, when the detection area SA of the proximity sensor 105a is set to overlap the screen 104i, an image showing the detection area SA is displayed on the screen 104i (for example, the range of the area SA is displayed by a solid line). ), The user can surely recognize the detection area SA, so that the operation by the gesture can be more reliably performed.

Next, the basic principle of gesture detection will be described. When there is nothing in front of the user US when the proximity sensor 105a is operating, the proximity sensor 105a does not receive invisible light as detection light, so that the control processing unit 121 is performing a gesture. The gesture processing unit 1212 is set in the sleep state. On the other hand, as shown in FIG. 8, when the user US's own hand HD is approached in front of the user US, the proximity sensor 105a detects invisible light radiated from the hand HD, and the proximity sensor 105a based on the invisible light is detected. In response to the output signal, the control processing unit 121 determines that a gesture has been performed, and activates the gesture processing unit 1212. In the following description, it is assumed that a gesture is performed with the hand HD of the user US. However, the gesture may be performed by the user US using an indicator made of a material capable of emitting invisible light. You may go.

As described above, the proximity sensor 105a has four pyroelectric elements RA to RD arranged in two rows and two columns (see FIG. 6). Therefore, when the user US brings the hand HD close to the front of the HMD 100 from either the left, right, up, or down directions, the output timings of signals detected by the pyroelectric elements RA to RD are different.

For example, referring to FIG. 8 and FIG. 9, in the case of a gesture in which the user US moves the hand HD from the right to the left in front of the HMD 100, the invisible light emitted from the hand HD is applied to the proximity sensor 105a. Incident. In this case, the pyroelectric elements RA and RC first receive invisible light. Therefore, as shown in FIG. 10, first, the signals of the pyroelectric elements RA and RC rise, and the signals of the pyroelectric elements RB and RD rise after a delay. Thereafter, the signals of the pyroelectric elements RA and RC fall, and the signals of the pyroelectric elements RB and RD fall after a delay (not shown). The gesture processing unit 1212 detects the timing of this signal, and the gesture processing unit 1212 determines that the user US has made a gesture by moving the hand HD from right to left.

Similarly, the signals of the pyroelectric elements RB and RD rise and are delayed, the signals of the pyroelectric elements RA and RC rise, and then the signals of the pyroelectric elements RB and RD fall and are delayed with the pyroelectric elements RA, RC. When the signal falls, the gesture processing unit 1212 can determine that the user US has made a gesture by moving the hand HD from left to right.

Similarly, the signals of the pyroelectric elements RA and RB rise and are delayed, and the signals of the pyroelectric elements RC and RD rise, and then the signals of the pyroelectric elements RA and RB fall and are delayed. When the RD signal falls, the gesture processing unit 1212 can determine that the user US has made a gesture by moving the hand HD from above to below.

Similarly, the signals of the pyroelectric elements RC and RD rise and are delayed, the signals of the pyroelectric elements RA and RB rise, and then the signals of the pyroelectric elements RC and RD fall, and the pyroelectric elements RA, RD are delayed. When the RB signal falls, the gesture processing unit 1212 can determine that the user US has made a gesture by moving the hand HD from the bottom to the top.

Similarly, the signal of the pyroelectric element RA rises, the signal of the pyroelectric elements RB and RC rises with a delay, the pyroelectric element RD rises with a delay, and then the signal of the pyroelectric element RA falls and is delayed When the pyroelectric elements RB and RC fall and the pyroelectric element RD falls after a further delay, the gesture processing unit 1212 performs the gesture in which the user US moves the hand HD from the upper right to the lower left. Can be judged.

Similarly, the signal of the pyroelectric element RB rises, the signal of the pyroelectric elements RA and RD rises with a delay, the pyroelectric element RC rises with a delay, and then the signal of the pyroelectric element RB falls and delays When the pyroelectric elements RA and RD fall and the pyroelectric element RC falls later, the gesture processing unit 1212 performs a gesture in which the user US moves the hand HD from the upper left to the lower right. Can be determined.

Similarly, the signal of the pyroelectric element RD rises, the signal of the pyroelectric elements RB and RC rises with a delay, the pyroelectric element RA rises with a delay, and then the signal of the pyroelectric element RD falls and delays When the pyroelectric elements RB and RC fall and the pyroelectric element RA falls after a further delay, the gesture processing unit 1212 has performed the gesture in which the user US moves the hand HD from the lower left to the upper right. Can be judged.

Similarly, the signal of the pyroelectric element RC rises, the signals of the pyroelectric elements RA and RD rise after a delay, the pyroelectric element RB rises after a further delay, and then the signal of the pyroelectric element RC falls and delays When the pyroelectric elements RA and RD fall and the pyroelectric element RB falls after a further delay, the gesture processing unit 1212 performs a gesture in which the user US moves the hand HD from the lower right to the upper left. Can be determined.

Next, gesture detection in the present embodiment based on the basic principle described above will be described. FIG. 11 is a flowchart illustrating gesture detection processing (main routine) in the wearable electronic device according to the embodiment. FIG. 12 is a flowchart illustrating a gesture determination process (subroutine) in the gesture detection process of the wearable electronic device according to the embodiment.

In such an HMD 100, when the proximity sensor 105a, which is an example of the second motion measuring unit 105, is stationary, the gesture processing unit 1212 uses the hand or finger, which is an example of the second part, according to the basic principle described above. The gesture by can be detected. However, since the proximity sensor 105a is disposed on the main body 103 of the frame 101, which is an example of a head mounting part, when the head, which is an example of the first part, is moved, only the basic principle described above is used. It is difficult for the gesture processing unit 1212 to detect the gesture correctly only based on the processing based on it. For example, even when the hand or finger is stationary, if the user swings the gesture, the gesture processing unit 1212 may detect a gesture with the hand or finger and make a misjudgment. .

Therefore, in this embodiment, the HMD 100 operates as follows. In FIG. 11, the control unit 1211 samples the output of the proximity sensor 105a (each output of each current collecting element RA to RD) at a predetermined sampling interval (for example, 10 ms, 20 ms, or 30 ms). By determining whether or not the output exceeds a predetermined threshold (first threshold) th1 set in advance, the presence or absence of the second part, in this example, the hand HD is determined (S11). The first threshold th1 is a value set to avoid erroneous determination due to noise. The control unit 1211 also samples the outputs of the gyroscope and the acceleration sensor 110a at the timing of sampling the output of the proximity sensor 105a. Similarly, the control unit 1211 also samples the outputs of the gyroscope and the acceleration sensor 110a at the timing of sampling the output of the proximity sensor 105a.

As a result of the determination, when the output of the proximity sensor 105a does not exceed the first threshold th1 (No), the control unit 1211 returns the process to the process S11 and executes the process S11 at the next sampling timing. . On the other hand, as a result of the determination, if the output of the proximity sensor 105a exceeds the first threshold th1 (Yes), that is, the output of any of the current collecting elements RA to RD in the proximity sensor 105a is When the first threshold value th1 is exceeded (Yes), the control unit 1211 activates the gesture processing unit 1212 when the gesture processing unit 1212 is in the sleep state, and operates the gesture determination processing. The gesture is provisionally determined (S12).

This gesture determination process will be described with reference to FIG. In FIG. 12, the gesture processing unit 1212 first determines the rising timing and falling timing of each output of the plurality of pyroelectric elements RA to RD in the proximity sensor 105a (S21).

Next, the gesture processing unit 1212 determines whether or not a gesture can be determined (S22). As described above, the gesture is determined based on each rising timing and each falling timing of each signal output from each of the pyroelectric elements RA to RD in the proximity sensor 105a. In executing the process, the gesture cannot be determined. Therefore, whether or not the gesture processing unit 1212 can tentatively determine a gesture by combining the current processing result and a plurality of past processing results such as the previous processing result and the previous processing result in the current processing. Judging.

As a result of this determination, if the gesture cannot be determined, the gesture processing unit 1212 acquires the output of the proximity sensor 105a (each output of each current collector RA to RD) at the next sampling timing (S23). Is returned to step S21. On the other hand, as a result of the determination, if the gesture can be determined, the gesture processing unit 1212 ends the determination process of the gesture shown in FIG. 12 as the temporarily determined gesture, and performs the process S13 of FIG. Execute.

Returning to FIG. 11, in process S13, the gesture processing unit 1212 determines whether the output of the gyroscope and the acceleration sensor 110a exceeds a predetermined threshold value (second threshold value) th2 or not. One part, in this example, the presence or absence of head movement is determined. The second threshold th2 is a value set to avoid erroneous determination due to noise.

If the result of this determination is that the output of the gyroscope and the acceleration sensor 110a does not exceed the second threshold th2 (No), the gesture using the first measurement result as performed in the processing S14 described later is not performed. The processing unit 1212 finally determines (mainly determines) a gesture with the tentatively determined gesture (S15), and then executes processing S16. On the other hand, as a result of the determination, when the output of the gyro and acceleration sensor 110a exceeds the second threshold th2 (Yes), that is, out of the acceleration in the X direction and the acceleration in the Z direction of the gyro and acceleration sensor 110a. When the output of any of the above exceeds the second threshold th2 (Yes), the gesture processing unit 1212 corrects the tentatively determined gesture and finally determines (mainly determines) the gesture (S14). Then, the process S16 is executed.

In this process S14, more specifically, first, the gesture processing unit 1212 obtains the temporarily determined speed vector (speed in the X direction, speed in the Z direction) of the gesture. The output of the pyroelectric elements RA to RD is sampled at a predetermined sampling interval as described above, and the size (horizontal length (length in the X direction), vertical length (length in the Z direction) of each pyroelectric element RA to RD. , Diagonal length) can be measured in advance. Therefore, the sizes of the pyroelectric elements RA to RD are measured in advance and stored in the storage unit 125, and the sizes of the pyroelectric elements RA to RD are changed from the rising timing to the falling timing according to the tentatively determined gesture. By dividing by the time, the velocity vector is obtained.

More specifically, when the tentatively determined gesture is a gesture of moving the hand HD from right to left, the gesture processing unit 1212 determines whether the pyroelectric element RA or the pyroelectric element is based on the sampling interval. The time from the rise timing of the signal in RC to the fall timing is obtained, and the horizontal length of the pyroelectric element RA or the pyroelectric element RC is divided by the obtained time, whereby the speed in the X direction of the tentatively determined gesture is obtained. Ask. In the gesture of moving the hand HD from right to left, the speed in the Z direction is zero. Therefore, the velocity vector in the gesture of the tentatively determined gesture moving from right to left of the hand HD is obtained as (X-direction velocity, 0). The time from the rising timing to the falling timing may be obtained by multiplying the number of samplings from the rising timing to the falling timing by a sampling interval, but a timer corresponding to each pyroelectric element RA to RD. May be functionally provided in the control processing unit 121, restarting the timer at the rise timing, and measuring the time until the fall timing with the timer.

In this case, the gesture processing unit 1212 obtains the first time from the rising timing of the signal in the pyroelectric element RA or the pyroelectric element RC to the falling timing based on the sampling interval, and the obtained first time. By dividing the first horizontal length of the pyroelectric element RA or pyroelectric element RC in one hour, a first speed (first speed in the X direction) is obtained, and similarly, based on the sampling interval, the pyroelectric element RB Alternatively, the second time from the rising timing of the signal in the pyroelectric element RD to the falling timing is obtained, and the second horizontal length in the pyroelectric element RB or pyroelectric element RD is divided by the obtained second time, 2 speeds (second speed in the X direction) are obtained, an average speed of the first and second speeds is obtained, and the obtained average speed is calculated as the temporary speed. May be used as the speed of the X direction of the boss was gesture. Alternatively, the gesture processing unit 1212 may obtain and average each corresponding speed for each pyroelectric element RA to RD, and this average speed may be used as the speed in the X direction of the tentatively determined gesture. The same applies to the following cases.

The velocity vector in the case where the tentatively determined gesture is a gesture of moving the hand HD from left to right is the same as that in the case where the tentatively determined gesture is the gesture of moving the hand HD from right to left. (−X direction velocity, 0).

When the tentatively determined gesture is a gesture of moving from the top to the bottom of the hand HD, the gesture processing unit 1212 determines the signal rise timing in the pyroelectric element RA or the pyroelectric element RB based on the sampling interval. The time from the first to the fall timing is obtained, and the vertical length of the pyroelectric element RA or pyroelectric element RB is divided by the obtained time to obtain the speed in the −Z direction of the tentatively determined gesture. In the gesture of moving from the top to the bottom of the hand HD, the velocity in the X direction is zero. Therefore, the velocity vector in the gesture of the tentatively determined gesture moving from the top to the bottom of the hand HD is obtained as (speed in the 0, −Z direction).

The velocity vector in the case where the tentatively determined gesture is a movement gesture from the bottom to the top of the hand HD is the same as that in the case where the tentatively determined gesture is a movement gesture from the top to the bottom of the hand HD described above. (0, velocity in the Z direction).

When the tentatively determined gesture is a gesture for moving the hand HD from the upper right to the lower left, the gesture processing unit 1212 determines the rise timing of the signal in the pyroelectric element RA or the pyroelectric element RD based on the sampling interval. Is calculated from the upper right to the lower left of the tentatively determined gesture by dividing the diagonal length of the pyroelectric element RA or the pyroelectric element RD by the obtained time. Ask. By dividing the obtained velocity in the direction from the upper right to the lower left into the respective components of the velocity in the X direction and the velocity in the Z direction based on the aspect ratio of the pyroelectric element RA or the aspect ratio of the pyroelectric element RD. The velocity vector in the gesture of the tentatively determined gesture moving from the upper right to the lower left of the hand HD is obtained as (velocity in the X direction, velocity in the Z direction).

A velocity vector in the case where the tentatively determined gesture is a movement gesture from the upper left to the lower right of the hand HD, a velocity vector in a case where the tentatively determined gesture is a movement gesture from the lower left to the upper right of the hand HD, and Each of the velocity vectors in the case where the tentatively determined gesture is a gesture of moving the hand HD from the lower right to the upper left is the above-described velocity vector which is the gesture of moving the hand HD from the upper right to the lower left. It is required in the same way as the case.

Next, the gesture processing unit 1212 obtains a velocity vector in the first movement of the head, which is an example of the first part, from the acceleration in the X direction and the acceleration in the Z direction of the gyroscope and the acceleration sensor 110a. Here, a plurality of outputs are acquired from the gyro and the acceleration sensor 110a until the gesture is provisionally determined. For this reason, for example, the average value of each of the plurality of outputs is used as the acceleration in the X direction and the acceleration in the Z direction of the gyroscope and the acceleration sensor 110a in the above-described processing. Further, for example, an output acquired at the center timing among the plurality of outputs arranged in time series is used as the acceleration in the X direction and the acceleration in the Z direction of the gyroscope and the acceleration sensor 110a in the above-described processing.

Next, the gesture processing unit 1212 subtracts the velocity vector in the first movement of the head from the velocity vector in the tentatively determined gesture, and the velocity in the second movement of the hand HD that is an example of the second part. Find a vector. Then, the gesture processing unit 1212 finally determines (mainly determines) a gesture with a gesture corresponding to the obtained velocity vector. Thereby, the temporarily determined gesture is corrected, and the gesture is finally determined (mainly determined).

For example, when the head HD is moved from the right to the left while the head is moved from the top to the bottom, the process HD is repeated from the lower right to the upper left of the hand HD by the process S12 (repetitive processes of the processes S21 to S23). Is temporarily determined as a movement gesture. Then, in step S13 and step S14, a velocity vector in the case of the tentatively determined movement of the hand HD from the lower right to the upper left is obtained, and the head obtained from the gyro and the acceleration sensor 110a is obtained from this velocity vector. The velocity vector of the part HD is subtracted to obtain the velocity vector of the hand HD. In the velocity vector of the hand HD, the velocity component due to the movement from the top to the bottom of the head is removed, and the velocity in the Z direction becomes substantially zero. Therefore, the tentatively determined gesture of moving the hand HD from the lower right to the upper left is corrected to the gesture of moving the hand HD from the right to the left and finally determined.

In step S16, the gesture processing unit 1212 notifies the finally determined gesture to the detection result utilization unit 1213.

Next, the control unit 1211 samples the output of the proximity sensor 105a (each output of each of the current collecting elements RA to RD) at the next sampling timing, and the output of the proximity sensor 105a is the first output as in the process S11. By determining whether or not the threshold value th1 is exceeded, the presence or absence of the second part, in this example, the hand HD is determined (S17).

As a result of this determination, when the output of the proximity sensor 105a does not exceed the first threshold th1 (No), the control unit 1211 sets the gesture processing unit 1212 to the sleep state and returns the processing to the processing S11. Process S11 is executed at the timing of sampling. On the other hand, as a result of the determination, if the output of the proximity sensor 105a exceeds the first threshold th1 (Yes), that is, the output of any of the current collecting elements RA to RD in the proximity sensor 105a is When the first threshold value th1 is exceeded (Yes), the control unit 1211 returns the process to the process S12 and executes the next gesture determination process.

On the other hand, upon receiving a notification from the gesture processing unit 1212 in the process S16, the detection result utilization unit 1213 executes a predetermined process based on the determination result of the gesture processing unit 1212. For example, when the determination result of the gesture processing unit 1212 is a so-called “flick”, the detection result using unit 1213 performs the second operation from the first image formed in the image forming unit 104A by the display control of the display control unit 104DR. Change the display to turn the page.

As described above, the HMD 100 as an example of the wearable electronic device in this embodiment and the gesture detection method implemented in the HMD 100 are measured by the second motion measurement unit 110 (the gyro and the acceleration sensor 110a in this embodiment). In consideration of not only the two measurement results but also the first measurement result measured by the first movement measurement unit (proximity sensor 105a in this embodiment), the gesture by the second movement of the second part (hand HD in this embodiment) is performed. Since it recognizes, even if a mounting | wearing site | part (head part in this embodiment) moves, a gesture can be detected.

In the HMD 100 as an example of the wearable electronic device in this embodiment and the gesture detection method implemented therein, the gesture processing unit 1212 does not operate until the second motion measurement unit 105 detects the presence of the second part (gesture processing). Since the unit 1212 is stopped), power can be saved.

In the HMD 100 as an example of the wearable electronic device in this embodiment and the gesture detection method implemented therein, the gesture processing unit 1212 is mounted on a control unit CTU that is separate from the frame 101 that is an example of a head-mounted member. Therefore, by mounting the minimum necessary configuration on the head mounting member, the head mounting member can be reduced in size and weight, and the mounting feeling such as discomfort and annoyance due to the mounting of the head mounting member can be reduced. it can.

In the above-described embodiment, the HMD 100 is based on the first measurement result obtained by measuring the predetermined motion in the living body by the first motion measuring unit 110 functionally in the control processing unit 121 as indicated by a broken line in FIG. The gesture processing unit 1212 further includes a motion detection processing unit 1214 that detects the predetermined motion based on the second measurement result measured by the second motion measurement unit 105 when the motion detection processing unit 1214 detects the predetermined motion. Then, the gesture by the second movement of the second part may be recognized. According to this, when the living body performs a predetermined operation (for example, walking) excluding the movement (for example, swinging) of the first part, the gesture can be recognized without considering the predetermined movement. More specifically, a measurement result measured by the first motion measurement unit 110 (in this embodiment, the gyroscope and the acceleration sensor 110a) when the living body performs the predetermined operation, for example, walking, is obtained in advance. The obtained measurement result is stored in advance in the storage unit 125 as a signal pattern corresponding to the predetermined operation. When determining the gesture described above, the motion detection processing unit 1214 correlates the first measurement result measured by the first motion measurement unit 110 with the signal pattern stored in the storage unit 125 within a predetermined range. It is determined whether or not the living body is performing the predetermined operation. When the motion detection processing unit 1214 determines that the living body is performing the predetermined motion, the gesture processing unit 1212 performs measurement by the second motion measurement unit 105 (proximity sensor 105a in the present embodiment). Based on the measurement result, the gesture by the second movement of the second part (in the present embodiment, the hand HD) is recognized. Note that the motion detection processing unit 1214 may be configured by hardware.

In the above-described embodiment, the gesture processing unit 1212 receives the first motion measurement unit 110 (the gyro and acceleration sensor 110a in the present embodiment) when the output exceeds a predetermined threshold (third threshold) th3. The output of the two-motion measuring unit 105 (proximity sensor 105a in this embodiment) may be canceled and the gesture temporary determination process may be performed again.

In the above-described embodiment, the first motion measurement unit 110 is the gyro and the acceleration sensor 110a, but is not limited thereto. The first motion measuring unit 110 includes an acceleration sensor that measures acceleration, an angular velocity sensor that measures angular velocity, a speed sensor that measures velocity, a vibration sensor that measures vibration, an inclinometer that measures inclination, and a geomagnetic sensor that measures orientation. One or more of them may be provided.

This specification discloses various modes of technology as described above, and the main technologies are summarized below.

A wearable electronic device according to one aspect is mounted on a mounting member for mounting on a first part in a living body, a first movement measuring unit for measuring a first movement of the mounting member, and the mounting member, A second movement measuring unit for measuring a second movement of an indicator for performing a gesture provided in the second part or the second part different from the first part in the living body; and the first and first A gesture processing unit for recognizing a gesture based on the second movement based on the first and second measurement results measured by the two-motion measurement unit. Preferably, in the above-described wearable electronic device, the first motion measurement unit is one of an acceleration sensor that measures acceleration, an angular velocity sensor that measures angular velocity, a speed sensor that measures velocity, and a vibration sensor that measures vibration. It is configured with a plurality. Preferably, in the above-described wearable electronic devices, the second motion measuring unit includes a passive sensor including a plurality of pyroelectric elements arranged in a two-dimensional matrix. Preferably, in the above-described wearable electronic devices, the second motion measuring unit includes an active sensor including an infrared light source that emits infrared light and a plurality of pyroelectric elements arranged in a two-dimensional matrix. Configured. Preferably, in these above-described wearable electronic devices, the gesture processing unit is configured to determine the first measurement result based on the first measurement result measured by the first motion measurement unit from the second measurement result measured by the second motion measurement unit. A first movement component due to one movement is removed, and gesture recognition based on the second movement of the second part is recognized based on the second measurement result from which the first movement component has been removed. Preferably, in the wearable electronic device described above, the gesture processing unit temporarily determines a gesture by the indicator provided in the second part of the living body or the second part based on the second measurement result, Based on the tentatively determined gesture and the first measurement result, the gesture by the second movement of the indicator provided in the second part or the second part of the living body is recognized. Preferably, in the wearable electronic device described above, the gesture processing unit modifies the gesture provisionally determined based on the second measurement result based on the first measurement result, and performs the gesture based on the second movement. recognize. Preferably, in the above-described wearable electronic device, the gesture processing unit obtains the velocity vector of the first part and the velocity vector of the second part, and the velocity vector of the first part from the velocity vector of the second part. By subtracting, the gesture by the second movement is recognized.

Such a wearable electronic device is based on the second movement of the second part in consideration of not only the second measurement result measured by the second movement measurement unit but also the first measurement result measured by the first movement measurement unit. Since the gesture is recognized, the gesture can be detected even if the wearing part moves.

In another aspect, the above-described wearable electronic device further includes an operation detection processing unit that detects a predetermined operation in the living body based on a first measurement result measured by the first motion measurement unit, and the gesture When the motion detection processing unit detects the predetermined motion, the processing unit is provided in the second part or the second part based on the second measurement result measured by the second motion measurement unit. And recognizing a gesture caused by the second movement of the indicator.

Such a wearable electronic device can recognize a gesture without considering the predetermined movement when the living body performs a predetermined movement (for example, walking) other than the movement of the first part (for example, swinging). .

In another aspect, in these above-described wearable electronic devices, the gesture processing unit cancels the tentatively determined gesture when the first measurement result exceeds a predetermined threshold.

In another aspect, in the above-described wearable electronic device, the second motion measuring unit further detects the presence or absence of the second part, and the gesture processing unit is configured such that the second motion measuring unit is the second part. The operation is started when the presence of the is detected. Preferably, in the above-described wearable electronic device, the gesture processing unit performs the operation by turning on the power supply when the second motion measurement unit detects the presence of the second part from a state in which the power supply is turned off. Start. Preferably, in the above-described wearable electronic device, the gesture processing unit starts its operation by changing to an active state when the second motion measuring unit detects the presence of the second part from a sleep state.

Such a wearable electronic device can save power because the gesture processing unit does not operate until the second motion measurement unit detects the presence of the second part (because the gesture processing unit is paused).

In another aspect, in these above-described wearable electronic devices, the gesture processing unit is mounted on a member separate from the mounting member.

Such a wearable electronic device can reduce the size of the mounting member by mounting the minimum necessary configuration on the mounting member, and can reduce a feeling of discomfort or annoyance due to mounting of the mounting member.

In another aspect, in these above-described wearable electronic devices, the mounting member is a member for mounting on the head with the first part as the head.

According to this, a wearable electronic device to be worn on the head can be provided, and even if such a wearable electronic device has a first movement such as a swing, for example, a gesture of a second movement by, for example, a hand or fingers. Can be detected.

A gesture detection method for a wearable electronic device according to another aspect is a gesture detection method for a wearable electronic device that is attached to a first part of a living body by a mounting member, and the first movement of the mounting member is measured. The first measurement result is acquired, and the second measurement result is measured by measuring the second movement of the indicator for performing the gesture provided in the second part or the second part different from the first part in the living body. Acquiring and recognizing a gesture by the second movement based on the first and second measurement results. Preferably, in the gesture detection method for the wearable electronic device described above, a gesture based on the second part of the living body or an indicator provided in the second part is provisionally determined based on the second measurement result, and the provisional determination is performed. Based on the gesture and the first measurement result, the gesture by the second movement of the indicator provided in the second part or the second part of the living body is recognized. Preferably, in the gesture detection method for the wearable electronic device described above, the gesture temporarily determined based on the second measurement result is modified based on the first measurement result, and the gesture caused by the second movement is recognized. . Preferably, in the above-described gesture detection methods for wearable electronic devices, the second measurement result is further detected when the predetermined action in the living body is detected based on the first measurement result and the predetermined action is detected. Based on the above, the gesture by the second movement is recognized.

Such a wearable electronic device gesture detection method takes into account not only the second measurement result measured in the second motion measurement step but also the first measurement result measured in the first motion measurement step, and the second portion of the second part. Since the gesture by two movements is recognized, even if the wearing part moves, the gesture can be detected.

This application is based on Japanese Patent Application No. 2015-104830 filed on May 22, 2015, the contents of which are included in the present application.

In order to express the present invention, the present invention has been properly and fully described through the embodiments with reference to the drawings. However, those skilled in the art can easily change and / or improve the above-described embodiments. It should be recognized that this is possible. Therefore, unless the modifications or improvements implemented by those skilled in the art are at a level that departs from the scope of the claims recited in the claims, the modifications or improvements are not covered by the claims. To be construed as inclusive.

ADVANTAGE OF THE INVENTION According to this invention, the gesture detection method of a wearable electronic device and wearable electronic device can be provided.

Claims

A mounting member for mounting on the first part of the living body;
A first movement measuring unit for measuring a first movement of the mounting member;
A second motion measuring unit mounted on the mounting member and for measuring a second motion of the indicator for performing a gesture provided in the second part or the second part different from the first part in the living body When,
A gesture processing unit for recognizing a gesture by the second movement based on the first and second measurement results measured by the first and second movement measurement units,
Wearable electronics.
The gesture processing unit tentatively determines a gesture by the indicator provided in the second part or the second part of the living body based on the second measurement result, the tentatively determined gesture, and the first measurement Recognizing a gesture by the second movement of the indicator provided in the second part of the living body or the second part based on the result,
The wearable electronic device according to claim 1.
The gesture processing unit modifies the gesture temporarily determined based on the second measurement result based on the first measurement result, and recognizes the gesture due to the second movement.
The wearable electronic device according to claim 2.
The gesture processing unit obtains the velocity vector of the first part and the velocity vector of the second part, and subtracts the velocity vector of the first part from the velocity vector of the second part, thereby Recognize gestures by movement,
The wearable electronic device according to claim 3.
An operation detection processing unit configured to detect a predetermined operation in the living body based on a first measurement result measured by the first motion measurement unit;
When the motion detection processing unit detects the predetermined motion, the gesture processing unit applies the second part or the second part based on the second measurement result measured by the second motion measurement unit. Recognizing a gesture caused by the second movement of the indicator provided;
The wearable electronic device according to any one of claims 1 to 4.
The gesture processing unit cancels the tentatively determined gesture when the first measurement result exceeds a predetermined threshold;
The wearable electronic device according to claim 3.
The second motion measurement unit further detects the presence or absence of the second part,
The gesture processing unit starts an operation when the second motion measuring unit detects the presence of the second part,
The wearable electronic device according to claim 1 or 2.
The gesture processing unit is mounted on a member separate from the mounting member.
The wearable electronic device according to any one of claims 1 to 7.
The mounting member is a member for mounting to the head with the first part as a head.
9. The wearable electronic device according to any one of claims 1 to 8.
A gesture detection method for a wearable electronic device mounted on a first part of a living body by a mounting member,
Measuring a first movement of the mounting member to obtain a first measurement result;
Measuring a second movement of the indicator for performing a gesture provided in the second part or the second part different from the first part in the living body to obtain a second measurement result;
Recognizing a gesture by the second movement based on the first and second measurement results;
A gesture detection method for wearable electronic devices.
Based on the second measurement result, a gesture based on the second part of the living body or an indicator provided in the second part is provisionally determined, and based on the provisionally determined gesture and the first measurement result, the first part Recognize gestures by the movement of 2.
The gesture detection method of the wearable electronic device according to claim 10.
Correcting the gesture based on the first measurement result to the gesture temporarily determined based on the second measurement result, and recognizing the gesture caused by the second movement;
The gesture detection method of the wearable electronic device according to claim 11.
Furthermore, a predetermined operation in the living body is detected based on the first measurement result,
If the predetermined movement is detected, the gesture based on the second movement is recognized based on the second measurement result.
The gesture detection method of the wearable electronic device according to claim 10 or 11.