JP2015002939A - Biopotential input interface system, biopotential input sensor device, biopotential input method, and program thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a myoelectric potential input interface device for operating equipment by using gesture.SOLUTION: A biopotential input interface system includes: a biopotential measurement part which measures a potential difference that occurs linked with movement of a user; an intentional action detection part for detecting control intention of a user from a time change of the potential difference that is measured; an action determination part which, when intentional movement is detected by the intentional movement detection part, determines action of a user according to a reference stored in a determination reference accumulation part; a determination result transmission part which transmits the determined action of a user to the outside for controlling equipment that is a control object; a determination result receiving part which receives action result of the user; a screen transition control part which calculates a transition state of a screen displayed to a user based on the determination result received by the determination result receiving part; and a display control part which, after the screen transition control part switches determination reference accumulated in the determination reference accumulation part, controls display content based on an output of the screen transition control part.

Description

  The present invention relates to a biopotential input interface system, a biopotential input sensor device, a biopotential input method, and a program for operating a device using a biopotential (for example, myoelectric potential).

  Patent Documents 1 and 2 disclose an interface using an electromyogram.

U.S. Pat. No. 8,170,656 U.S. Pat. No. 8,447,704

  In Patent Documents 1 and 2, there is no detailed disclosure of controlling an operation device using an operation for controlling a user's device and other operations.

  The biopotential input interface system according to an aspect of the present disclosure refers to a bioelectric potential measurement unit that measures a user's biopotential using a plurality of measurement electrodes, and a detection standard that includes a first threshold voltage. An operation detection unit that detects an operation corresponding to the first operation of the user from the bioelectric potential of the user, and when the operation detection unit detects an operation corresponding to the first operation, The operation determination unit that determines the user action corresponding to the measured bioelectric potential of the user, and the operation reference that associates the operation of the control target device and the user action are referred to And a device control unit that determines an operation corresponding to the determined user action and transmits the determined operation to the control target device main body.

  The biopotential input interface according to the present disclosure controls the operation device using an operation for controlling the user's device and other operations.

The figure which shows the implementation | achievement form of a biopotential input interface system. The figure which shows the external appearance of a biopotential input interface system. The figure which shows some hollow bodies 110. FIG. The figure which shows the sensor part. The figure which shows the individual difference of a forearm maximum circumference. The figure which shows the sensor part. FIG. 3 is an enlarged view of a part of the sensor unit 100. The figure which looked at the appearance of an example of sensor part 100 from the side. The figure which shows an example of a structure of the bioelectrical potential input interface system. The figure which shows the flowchart of a process of the bioelectrical potential input interface system. The figure which shows the example of the measured myoelectric potential. The figure which shows the reference | standard currently accumulate | stored in the determination reference | standard storage part 5. FIG. The figure which shows the example of the operation reference | standard which the apparatus control part 8 hold | maintains previously. The figure which shows the information which matched the user's bioelectric potential and operation of the control object apparatus 102. FIG. 1 is a diagram illustrating a configuration of a biopotential input interface system 1001. FIG. The figure which shows the hardware constitutions of the sensor part 100. FIG. The figure which shows the structure of the biopotential input interface system 1002. FIG. The figure which shows the example of the display of the display part 12, and a criterion. The figure which shows the example of a display of the display part 12. FIG. The figure which shows the example of a display of the display part 12. FIG. The figure which shows the flowchart of the transition of the display screen of the display part. The figure which shows the change of the moving speed of the display screen according to a user's operation | movement. The figure which shows the flowchart of the preparation method of DB of gesture. The figure which shows the example of the myoelectric potential which the sensor part 100 measured. The figure which shows the flowchart of the process which recognizes a user's operation | movement from an electric potential.

  Hereinafter, an embodiment of a biopotential input interface system will be described with reference to the accompanying drawings.

(Embodiment 1)
FIG. 1 shows an example of an implementation form of the biopotential input interface system 1000. The biopotential input interface system 1000 includes a sensor unit 100, a recognition unit 101, and a control target device 102. Although FIG. 1 shows a user 1 who uses the biopotential input interface system 1000, the user 1 is not in the configuration of the biopotential input interface system 1000.

  The sensor unit 100 is attached to the user 1. FIG. 2A shows an example in which the sensor unit 100 is mounted on the arm of the user 1. FIG. 2B shows an example of the shape of the sensor unit 100. Details of the sensor unit 100 will be described later.

  The user 1 makes a gesture corresponding to the operation of the control target device 102 with the sensor unit 100 mounted. The sensor unit 100 measures the bioelectric potential when the user 1 is gesturing. The bioelectric potential means a potential generated in response to a user gesture. An example of the bioelectric potential of the user 1 is myoelectric potential.

  The recognizing unit 101 recognizes a gesture of the user 1 corresponding to the bioelectric potential measured by the sensor unit 100 with reference to a reference held in advance. The recognition unit 101 transmits an operation instruction corresponding to the recognized gesture to the control target device 102. An example of the control target device 102 is a display device including a TV, a device for recording and reproducing video information including a BD recorder, and an electronic device including a lighting device.

  Hereinafter, an example of the sensor unit 100 will be described in detail.

<Sensor part 100>
The sensor unit 100 includes a plurality of hollow bodies 110, a measurement hollow body 111, and a storage hollow body 112. The hollow body means a configuration in which at least a part of a side surface is surrounded and a space is provided inside.

  A plurality of hollow bodies 110, measurement hollow bodies 111, and storage hollow bodies 112 are coupled to each other to form an annular shape. The internal space of the plurality of hollow bodies 110, the internal space of the measurement hollow body 111, and the internal space of the storage hollow body 112 are connected.

  Each hollow body 110 has a measurement electrode 113. The measurement hollow body 111 has the bioelectric potential measurement unit 2 inside. The bioelectric potential measurement unit 2 is electrically connected to each measurement electrode 113 by metal wiring.

  Moreover, the measurement hollow body 111 may have a ground electrode and a reference electrode. The bioelectric potential measurement unit 2 is electrically connected to the ground electrode and the reference electrode.

  The bioelectric potential measurement unit 2 measures the bioelectric potential of the user 1 using the measurement electrode 113.

  FIG. 3 shows hollow bodies 110a, 110b, and 110c included in the plurality of hollow bodies 110.

  Each hollow body 110 has a side surface 114, a first opening 115, and a second opening 116. Inside the hollow body 110, an internal space that is connected from the first opening 115 to the second opening 116 is formed. An example of the side surface 114 of the hollow body 110 is a surface surrounding the entire side surface of the hollow body 110. The same applies to the measurement hollow body 111 and the storage hollow body 112.

  Of the side surface 114 of the hollow body 110, the measurement electrode 113 is disposed on the inner surface of the annular shape formed by the plurality of hollow bodies 110. As shown in FIG. 2A, when the user 1 wears the sensor unit 100, the measurement electrode 113 comes into contact with the user 1.

  Of the plurality of hollow bodies 110 shown in FIG. 3, the first hollow body 110a and the second hollow body 110b will be described as an example. The second opening 116a of the first hollow body 110a and the first opening 115b of the second hollow body 110b are positioned so as to face each other. Similarly, the plurality of hollow bodies 110, the measurement hollow bodies 111, and the storage hollow bodies 112 are connected in order, so that the sensor unit 100 forms an annular shape. Thereby, the internal space of the plurality of hollow bodies 110, the internal space of the measurement hollow body 111, and the internal space of the storage hollow body 112 are connected.

  FIG. 4A shows the shape of the sensor unit 100 as in FIG. 4 (b) and 4 (c) show examples of development of the sensor in which the housing hollow body 112 is removed from the sensor unit 100. FIG. A and B in FIGS. 4B and 4C correspond to A and B in FIG.

  FIG. 4B shows a case where the user 1 with a thin forearm wears a sensor on the forearm, and FIG. 4C shows a case where the user 1 with a thick forearm wears a sensor on the forearm. In FIG. 4B, the hollow bodies 110 are in contact with each other at the openings. In FIG.4 (c), since the length of the inner surface of a sensor respond | corresponds to the thickness of a forearm, between the hollow body 110a and the hollow body 110b has spread. Metal wiring for electrically connecting the biopotential measurement unit 2 and each measurement electrode 113 shown in FIG. 4B, and the biopotential measurement unit 2 and each measurement electrode 113 shown in FIG. The length is different from the metal wiring to be connected.

  In order to cope with this difference, it is difficult for the user 1 to expand and contract the metal wiring that electrically connects the bioelectric potential measurement unit 2 and each measurement electrode 113 according to the thickness of the part on which the sensor is mounted. .

  FIG. 5 shows an example of the maximum circumference (length) of the forearm, which is an example of a part where the user 1 wears the sensor unit 100 (AIST human body size database 1991-92). Data of 90% or more of the maximum circumference (length) of the forearm shown in the numerical value (mm) of the percentile in FIG. 5 is included in the range of 210 mm to 280 mm. That is, it is desirable that the ring-shaped length (circumference length) of the sensor unit 100 can be adjusted within a range of at least 70 mm.

  Further, in order to measure the bioelectric potential of the user 1, it is desirable to increase the contact strength (adhesion degree) between the user 1 and the measurement electrode 113 and reduce the contact impedance. Therefore, when the user 1 is wearing the sensor unit 100, it is desirable that the part of the user 1 wearing the sensor unit 100 has the same size as the ring of the sensor unit 100 having a ring shape. Thereby, the contact of the user 1 and the measurement electrode 113 can be improved.

  Further, for example, a case where the plurality of hollow bodies 110, the measurement hollow body 111, and the storage hollow body 112 of the sensor unit 100 are configured so as not to be disconnected can be considered. For example, the plurality of hollow bodies 110, the measurement hollow bodies 111, and the storage are connected to the internal spaces of the plurality of hollow bodies 110, the internal spaces of the measurement hollow bodies 111, and the internal spaces of the storage hollow bodies 112. The hollow body 112 may be connected. The connecting line is made of a stretchable member. An example of the connecting line is an elastic rubber including natural rubber or synthetic rubber.

  At this time, when the user 1 wears the sensor unit 100 on the forearm, the user 1 wears the sensor unit 100 on the forearm by passing the back from the hand. Therefore, it is desirable that the metal wiring of the sensor unit 100 has a size corresponding to not only the thickness of the part where the user 1 wears the sensor part 100 but also the size of the part of the user 1 that passes when the user 1 wears.

  The sensor unit 100 according to the present disclosure includes the housing hollow body 112 to constitute a metal wiring corresponding to stretchability. FIG. 6A shows the shape of the sensor unit 100 according to the present disclosure. 6 (b) and 6 (c) show examples of development of the sensor unit 100 having the housing hollow body 112. FIG. A and B in FIGS. 6B and 6C correspond to A and B in FIG.

  6A and 6B, the sensor unit 100 includes a housing hollow body 112 between a plurality of hollow bodies 110. 6A and 6B, the measurement hollow body 111, a plurality of hollow bodies 110 (also referred to as a first hollow body group 1101), a storage hollow body 112, and a plurality of hollow bodies 110 (first 2 in the order of the hollow body group 1102). The storage hollow body 112 does not have a measurement electrode, and stores metal wiring that connects the biopotential measurement unit 2 and the measurement electrodes of the hollow bodies included in the second hollow body group 1102. The housing hollow body 112 has a protrusion. The metal wiring rotates from the bioelectric potential measurement unit 2 around the protrusion of the housing hollow body 112 and is connected to the measurement electrode. The protrusion of the storage hollow body 112 may be disposed so as to contact the first side surface in the internal space of the storage hollow body 112 and the second side surface opposite to the first side surface. Alternatively, the protrusion only needs to have a predetermined height or more from the first side surface in the internal space of the housing hollow body 112. For example, an example of the height of the protrusion is a height at which all the metal wirings connected from the bioelectric potential measurement unit 2 to the measurement electrode 114 of the second second hollow body group 1102 are caught by the protrusion.

  The metal wiring when the user 1 with the small forearm thickness shown in FIG. 6B wears the sensor unit 100 has the small forearm thickness shown in FIG. 6B shown in FIG. 6C. The length located inside the housing hollow body 112 is larger than the metal wiring when the user 1 is wearing the sensor unit 100. That is, by changing the length of the metal wiring located in the housing hollow body 112, the diameter of the ring-shaped ring of the sensor unit 100 can be changed, which corresponds to the thickness of the wearing part of the user 1 and the like. Can do. Specifically, as shown in FIGS. 6B and 6C, the length of the metal wiring arranged inside the housing hollow body 112 allows the first hollow even if the metal wiring itself does not expand and contract. The length between the body group 1101 and the second hollow body group 1102 and the housing hollow body 112 can be adjusted.

  7A and 7B are enlarged views of a part of the sensor unit 100 as viewed from the inside of the ring shape formed by the sensor unit 100. FIG. 7A and 7B omit a part of the plurality of hollow bodies 110 and show only the hollow bodies 110a and 110b. 7A and 7B, a part of the metal wiring included in the sensor unit 100 is omitted, and the bioelectric potential measurement unit 2 included in the measurement hollow body 111 and the measurement electrode 114b positioned in the hollow body 110b. Only the metal wiring for electrically connecting the two is shown.

  The measurement hollow body 111, the hollow body 110a, the storage hollow body 112, and the hollow body 110b are connected by the connection lines 117a and 117b.

  The metal wiring shown in FIGS. 7A and 7B rotates around the protrusion 1121 of the housing hollow body 112 to connect the biopotential measurement unit 2 and the measurement electrode 114b.

  The distance between the hollow body 110a and the storage hollow body 112 shown in FIG. 7B is larger than the distance between the hollow body 110a and the storage hollow body 112 shown in FIG. This difference in distance corresponds to the ring-shaped diameter formed by the sensor unit 100.

  In FIG. 7 (a), the metal wiring wraps around the projection of the housing hollow body 112 so that the length of the metal wiring located inside the housing hollow body 112 increases, and the sensor unit 100 is formed. The ring-shaped diameter can be reduced. On the other hand, in FIG. 7B, since the metal wiring circulates so as to be in contact with the periphery of the protrusion of the housing hollow body 112, the length of the metal wiring located inside the housing hollow body 112 becomes small, and the sensor The diameter of the ring shape formed by the portion 100 can be increased.

  As described above, the ring-shaped diameter of the sensor unit 100 can be changed without expanding or contracting the metal wiring by being arranged by rotating around the protrusion 1121 of the housing hollow body 112.

  FIG. 8A shows a view of an external appearance of an example of the sensor unit 100 as viewed from the side. FIG. 8B shows a view of an external appearance of an example of the sensor unit 100 as viewed obliquely.

  When the sensor unit 100 shown in FIG. 8A is attached to the arm of the user 1 and the myoelectric potential is measured, the palm direction of the user 1 (hereinafter also referred to as the lower surface direction of the arm) is the measurement hollow body 111. The direction of the back of the hand of the user 1 (hereinafter also referred to as the upper surface direction of the arm) is in contact with the housing hollow body 112.

  The arm muscles (mainly extensors) of the user are located on the upper surface of the arm and the side surface of the arm. In order to acquire the myoelectric potential of the user, it is desirable that the measurement electrode 114 of the sensor unit 100 be disposed so as to contact the upper surface of the arm and the side surface of the arm. In particular, the measurement electrode 114 is preferably positioned so as to be in contact with a portion where the movement of the user's muscle is large.

  Since the lower surface of the arm has fewer muscles than other parts, the measurement hollow body 111 is arranged so that the measurement hollow body 111 without the measurement electrode 114 is in contact with the lower surface of the arm. When the measurement hollow body 111 has a reference electrode or a ground electrode, the lower surface of the arm is relatively flat, which is desirable because the reference electrode or the ground electrode easily contacts the user 1. At this time, the length of the measurement hollow body 111 in the circumferential direction of the ring formed by the sensor unit 100 is preferably larger than the length of the hollow body 110. This is because the reference electrode or the ground electrode of the measurement hollow body 111 is easily in contact with the user 1 and it is easy to set a standard for measuring the myoelectric potential.

  In addition, bones are located on the upper surface of the user's arm, and it may be difficult to bring the measurement electrode 114 and the upper surface of the arm into close contact with each other. In particular, bones overhang at positions close to the elbows, and it may be difficult to bring the electrodes and arms into contact. Therefore, the housing hollow body 112 that does not have the measurement electrode 114 is disposed so as to contact the upper surface of the user's arm.

  The storage hollow body 112 and the measurement hollow body 111 are disposed at positions facing each other with the hollow body 110 interposed therebetween. It is also conceivable that the position of the flat portion of the lower surface of the arm and the position of the portion of the upper surface of the arm where the bone is located do not completely face each other. As shown in FIG. 8A, the housing hollow body 112 and the measurement hollow body 111 are not symmetrical positions with respect to the center of the ring shape of the sensor unit 100, but are symmetrical according to the shape of the user's arm. You may arrange | position in the asymmetrical position shifted | deviated the predetermined distance or less from the position. The housing hollow body 112 and the measurement hollow body 111 are also included in the case where they are arranged at an asymmetrical position shifted by a predetermined distance or less from a symmetric position according to the shape of the user's arm.

<Biopotential input interface system 1000>
FIG. 9 shows an example of the configuration of the biopotential input interface system 1000. A biopotential input interface system 1000 illustrated in FIG. 9 includes a sensor unit 100 and a control target device 102. The sensor unit 100 includes a bioelectric potential measurement unit 2, an operation detection unit 3, an operation determination unit 4, a determination reference accumulation unit 5, and a determination result transmission unit 6. The control target device 102 includes a determination result receiving unit 7, a device control unit 8, and a control target device main body 9. The sensor unit 100 is connected to the control target device 102 by wire or wirelessly, and transmits and receives information. Note that only the sensor unit 100 is also referred to as a biopotential input sensor device.

<Sensor part 100>
The bioelectric potential measurement unit 2 measures the bioelectric potential of the user 1 using a plurality of electrically connected measurement electrodes. An example of the bioelectric potential of the user 1 is myoelectric potential.

  The motion detection unit 3 refers to detection criteria held in advance, and detects a plurality of biopotentials corresponding to the intentional motion (gesture) of the user 1 from the plurality of biopotentials measured by the biopotential measurement unit 2. . For example, the motion detection unit 3 holds a voltage having a predetermined threshold as a detection reference, and detects a potential equal to or higher than the predetermined threshold from the bioelectric potential measured by the bioelectric potential measurement unit 2. The potential equal to or higher than the predetermined threshold value may be an absolute value, or may be the magnitude of the potential change from the potential at the previous time.

  Alternatively, a myoelectric potential corresponding to a user's predetermined action is held in advance as a detection reference. The myoelectric potential corresponding to a predetermined action is the magnitude of the potential of each channel or a change in potential. When the measured bioelectric potential and the myoelectric potential held in advance are equal to or greater than a predetermined threshold value, the measured bioelectric potential is detected as an intentional action. The motion detection unit 3 holds in advance a predetermined threshold value for determining the degree of coincidence. The myoelectric potential corresponding to a predetermined operation is information on the magnitude of the potential of each measurement electrode. The magnitude of the potential of each measurement electrode may be an absolute value or a ratio of the magnitude of the potential.

  The operation determination unit 4 determines a gesture corresponding to the bioelectric potential detected by the operation detection unit 3 with reference to the determination criterion held by the determination criterion accumulation unit 5.

  The determination reference accumulation unit 5 holds a determination reference that associates a potential with an action (gesture). An example of the reference is information in which the magnitude of the potential measured using each measurement electrode is associated with an action (gesture). The magnitude of the potential of each measurement electrode may be an absolute value or a ratio of the magnitude of the potential.

  The determination result transmission unit 6 transmits the gesture determined by the operation determination unit 4 to the control target device 102.

  Note that the operation determination unit 4 may acquire a determination reference from the determination reference storage unit 5 existing outside the sensor unit 100 through a wired or wireless connection, and may determine an operation based on the acquired determination reference. That is, the sensor unit 100 may not include the determination reference accumulation unit 5. Similarly, the operation detection unit 3 may acquire a detection reference from an external storage unit and detect it based on the acquired detection reference.

  Hereinafter, the intentional operation is also expressed as a first operation, and the operation determined by the operation determination unit 4 is also expressed as a second operation. The first operation and the second operation include the same operation.

<Controlled device 102>
The determination result receiving unit 7 receives the gesture of the user 1 from the sensor unit 100 (determination result transmitting unit 6).

  The device control unit 8 determines an operation corresponding to the gesture of the user 1 with reference to an operation criterion stored in advance. The device control unit 8 controls the operation of the control target device main body 9 based on the determined operation.

<Processing of Biopotential Input Interface System 1000>
FIG. 10 shows a flowchart of processing of the biopotential input interface system 1000. As shown in FIG. 2A, an example in which the sensor unit 100 is worn on the forearm of the user 1 will be described.

(S101)
The bioelectric potential measurement unit 2 measures myoelectric potential as the bioelectric potential of the user 1 using a plurality of measurement electrodes. FIG. 11 shows an example of myoelectric potential measured by the bioelectric potential measuring unit 2. The vertical axis in FIG. 11 is the magnitude of the potential (μV), and the numbers on the horizontal axis mean each measurement electrode. The example of the myoelectric potential shown in FIG. 11 was measured by the bioelectric potential measuring unit 2 using eight measurement electrodes. The potentials 1 to 7 shown on the horizontal axis mean potentials measured by the biopotential measurement unit 2 and each of the first measurement electrode to the seventh measurement electrode. Hereinafter, the potentials 1 to 7 shown in FIG.

(S102)
The motion detection unit 3 detects the bioelectric potential (muscle potential) of the user 1 having a potential equal to or higher than a predetermined threshold. The predetermined threshold is a magnitude of a potential that is larger than a normal operation potential and is included in a potential range when the user 1 performs an intentional operation. It can be set based on the potential measured in advance according to the part where the user 1 is wearing the sensor unit 100 or the user 1 who is wearing the sensor unit 100.

  For example, the motion detection unit 3 acquires a time at which at least one myoelectric potential of the plurality of measured myoelectric potentials has a potential equal to or higher than a predetermined threshold. Note that the acquired time may have a certain width. Using the acquired time, the myoelectric potential measured by the bioelectric potential measuring unit 2 is determined as the bioelectric potential corresponding to the operation (gesture) of the user 1.

  In the example shown in FIG. 11, when the predetermined threshold is a potential indicating 200 on the vertical axis, the potentials of the channels 5 to 7 exceed the threshold. Therefore, the motion detection unit 3 determines the potentials of the channels 1 to 7 at the time having the potential exceeding the threshold as the biopotential corresponding to the intentional motion (gesture) of the user 1.

  When the motion detection unit 3 cannot detect the bioelectric potential corresponding to the intentional motion of the user 1, the process returns to S101. When the motion detection unit 3 detects a bioelectric potential corresponding to the intentional motion of the user 1, the process proceeds to S103.

(S103)
The operation determination unit 4 determines an operation corresponding to the bioelectric potential detected by the operation detection unit 3 with reference to the reference held by the determination reference accumulation unit 5. FIG. 12 shows an example of information that is stored in the determination criterion storage unit 5 and that associates an action with a bioelectric potential. The hand motion shown in FIG. 12 is the magnitude of the potential of each channel when the hand is moved to a stone, the thumb is bent and the hand is opened.

  For example, the operation determination unit 4 determines an operation corresponding to the potential detected by the operation detection unit 3 shown in FIG. 11 with reference to the reference stored in the determination reference storage unit 5 shown in FIG. At this time, the operation determination unit 4 determines that the potential illustrated in FIG. 11 corresponds to the operation a.

  The determination result transmission unit 6 transmits the operation determined by the operation determination unit 4 to the control target device 102.

(S104)
The determination result receiving unit 7 receives the operation determined by the operation determining unit 4 from the determination result transmitting unit 6.

  The device control unit 8 determines an operation corresponding to the gesture of the user 1 with reference to an operation criterion stored in advance. The device control unit 8 controls the operation of the control target device main body 9 based on the determined operation.

  In FIG. 13, the example of the operation reference | standard which the apparatus control part 8 hold | maintains beforehand is shown. The example illustrated in FIG. 13 is information in which an operation and an operation of the control target device 102 are associated with each other. Examples of operations of the control target device 102 include turning on the power of the control target device 102, turning off the power, adjusting the volume (increasing or decreasing the volume), and executing the set operation (forward, return, zoom in, zoom out) Etc.).

<Modification 1 of the processing flow of FIG. 10>
In S <b> 102, the motion determination unit 4 may determine the myoelectric potential after a predetermined time from the acquired time as a bioelectric potential corresponding to the motion (gesture) of the user 1. A predetermined time is stored in advance in a storage unit included in the motion detection unit 3.

  Alternatively, the operation determination unit 4 may hold a predetermined time in advance, acquire the time from the operation detection unit 3, and acquire the bioelectric potential corresponding to the determination operation.

  At this time, the intentional motion (also referred to as the first motion) detected by the motion detection unit 3 is different from the motion (also denoted as the second motion) determined by the motion determination unit 4 in S103. The user 1 performs a first operation for starting processing of the biopotential input interface and a second operation for operation input of the control target device 102.

<Modification 2 of the processing flow of FIG. 10>
In the example illustrated in FIG. 10, the biopotential input interface system 1000 obtains an operation of the control target device from the biopotential of the user 1 using the reference in FIGS. 12 and 13.

  The criterion storage unit 5 may hold information in which the bioelectric potential of the user illustrated in FIG. 14 and the operation of the control target device 102 are associated with each other. The operation determination unit 4 determines the operation of the control target device 102 from the user's biopotential by referring to information in which the user's biopotential is associated with the operation of the control target device 102. The sensor unit 100 (determination result transmission unit 6) transmits the operation of the control target device 102 to the control target device 102.

  The device control unit 8 controls the operation of the control target device based on the operation of the control target device 102 received from the sensor unit 100.

<Variation 1 of the biopotential input interface system>
FIG. 15 shows the configuration of the biopotential input interface system 1001. A biopotential input interface system 1001 illustrated in FIG. 15 includes a sensor unit 100, a recognition unit 101, and a control target device 102. The sensor unit 100 and the recognition unit 101 are also referred to as a recognition device.
The sensor unit 100, the recognition unit 101, and the control target device 102 are connected by wire or wireless, and information is transmitted and received.

  A difference from the biopotential input interface system of the first embodiment is that a recognition unit 101 including an operation detection unit 3, an operation determination unit 4, a determination reference accumulation unit 5, and a determination result transmission unit 6 is added and included in the recognition unit 101. That is, the configuration is excluded from the sensor unit 100.

  The sensor unit 100A includes a bioelectric potential measurement unit 2 and a measurement potential transmission unit 2A. The recognition unit 101 includes a measurement potential reception unit 2B, an operation detection unit 3, an operation determination unit 4, a determination reference accumulation unit 5, and a determination result transmission unit 6. The control target device 102 includes a device control unit 8 and a control target device main body 9.

  The measurement potential transmission unit 2A included in the sensor unit 100 transmits the measured bioelectric potential to the measurement reception unit 2B included in the recognition unit 101.

  The biopotential input interface system 1001 shown in FIG. 15 performs the same processing as the flowchart shown in FIG.

<Hardware configuration>
FIG. 16 shows a hardware configuration of the sensor unit 100 shown in FIG.

  The sensor unit 100 includes a living body measurement unit 210, a signal processing unit 220, a transmission unit 230, and a battery unit 202. The biological measurement unit 210, the signal processing unit 220, and the transmission unit 230 are connected via the bus 201.

  The biometric measurement unit 210 corresponds to the biopotential measurement unit 2, the measurement electrode 114, the reference electrode, and the ground electrode. The biological measurement unit 210 includes a ground 211, a reference electrode 212, measurement electrodes 213a to 213h, a biological amplifier 214, and an AD conversion unit 215. The potential measured using the ground 211, the reference electrode 212, and the measurement electrodes 213a to 213h is amplified by the biological amplifier 214. The AD converter 215 converts the potential amplified by the biological amplifier 214.

  The signal processing unit 220 includes a CPU 221, a RAM 222, a program 223 included in the RAM, and a ROM 224. The signal processing unit 220 corresponds to the motion detection unit 3, the motion determination unit 4, and the determination reference accumulation unit 5.

  The transmission unit 230 includes an antenna 232a and a transmission control unit 231. The transmission unit 230 corresponds to the determination result transmission unit 6.

(Embodiment 2)
In FIG. 17, the biopotential input interface system 1002 includes a sensor unit 100 and a control target device 102. The sensor unit 100 includes a bioelectric potential measurement unit 2, an operation detection unit 3, an operation determination unit 4, a determination reference accumulation unit 5, and a determination result transmission unit 6. The control target device 102 includes a determination result receiving unit 7, a device control unit 8, a control target device main body 9, a screen transition control unit 10, a display control unit 11, and a display unit 12. The determination result receiving unit 7, the device control unit 8, the screen transition control unit 10, and the display control unit 11 are also referred to as a device control device.

A biopotential input interface system 1002 illustrated in FIG. 17 is different from the biopotential input interface system 1000 illustrated in FIG. 9 in that a control target device 102 includes a screen transition control unit 10, a display control unit 11, and a display unit 12. In addition,
The display unit 12 displays the association relationship between the user's operation (gesture) and operation. Details will be described later. Further, the display unit 12 may be formed integrally with the control target device main body 9 and may display a result of operating the control target device 102 or the like.

  The screen transition control unit 10 determines display screen transition control information of the display unit 10 based on operation information corresponding to the user's operation. The display control unit 11 controls the display of the display unit 12 according to the control information determined by the screen transition control unit 10.

  FIGS. 18A to 18C show examples of user operations (gestures) and operation associations displayed on the display unit 12 and video information (contents).

  The display section 12 shown in FIG. 18A displays video information 121a and 121b. Video information including the video information 121a and 121b is collectively referred to as video information 121. Examples of the video information 121a and 121b are moving image information, image information, or operation information of the control target device main body. Further, the display unit 12 displays a frame 122 and an arrow 123 indicating the moving direction of the frame 122 so as to surround a plurality of pieces of video information. The display unit 12 shows a determination criterion 124 indicating an example of an operation corresponding to the operation of the user 1. The determination criterion 124 corresponds to the operation reference held by the device control unit 8 or the reference operation stored in the determination reference storage unit 5 of the second modification of the first embodiment.

  In the state illustrated in FIG. 18, when the operation of the user 1 corresponding to the operation of Zoom in is recognized, the screen transition control unit 10 determines to transition to the display screen illustrated in FIG. The display controller 12 is controlled by the display controller 11, and the display transitions to the display shown in FIG.

  For example, the video information 121a on the display unit 12 indicates information on a cat (cat), and the video information 121b indicates information on a dog (dog). In the lower right part of each video information 121, an operation corresponding to the determination operation of each video information 121 is displayed. For example, when the user's action corresponding to C shown in the cat video information 121a is recognized by the recognizing unit, the screen transition control unit 10 and the display control unit 11 perform the operation as shown in FIG. Then, the display of the display unit 12 is changed.

  FIGS. 19A to 19C show an example of assignment of operations related to the video information 121 determination method. As shown in FIGS. 19A and 19B, the operation may be determined in accordance with the position of the video information 121 on the display screen. As shown in FIG. The operation may be determined accordingly.

  FIG. 20 shows an example of the operation for determining the video information 121. For example, in order to determine the video information of a cat, the operation shown next to the letter C may be performed. For example, only when the determination operation can be performed on the display unit 12, by displaying the operation corresponding to the determination operation, it is possible to reduce erroneous determination operations.

  The display screen transition unit 10 may determine a determination criterion used by the operation determination unit 4 according to the display screen of the display screen 12. The information 124 on the display screen shown in FIGS. 18A to 18C is different. For example, the criterion 124 for displaying the display screen shown in FIG. 18A does not include an operation corresponding to the operation for determining the video information 121. The determination criterion 124 for displaying the display screen shown in FIG. 18B includes operations for determining the video information 121 (C, D, L, S, B, H), whereas Zoom in Operations are not included. The determination criterion 124 at the time of displaying the display screen shown in FIG. 18C includes only a return operation (returning to the state shown in FIG. 18B).

  As described above, by using the determination criterion 124 including only the operation corresponding to the possible operation according to the display screen displayed on the display unit 12, it is possible to reduce malfunctions of the operation of the control target device at the time of erroneous determination. For example, other processing when an operation that cannot be controlled with respect to the control target device is determined may be difficult. However, if the operation cannot be controlled with respect to the control target device, even if the operation proceeds as an incorrect operation, the user may perform an operation of returning to the previous operation.

  FIG. 21 shows a flowchart of the display screen transition process of the display unit 12.

(S201)
The screen transition control unit 10 determines.

The screen transition control unit 10 determines the determination criterion corresponding to the screen displayed corresponding to the operation corresponding to the bioelectric potential measured last time based on the information on the correspondence between the screen information corresponding to the operation and the determination criterion. By transmitting to the operation determination unit 4 (or the determination criterion storage unit 5), the determination criterion (also referred to as gesture DB) is switched (S202).
The operation determination unit 4 determines the operation corresponding to the bioelectric potential of the user with reference to the switched determination criterion.

(S203)
The device control unit 8 determines an operation corresponding to the operation determined by the operation determination unit 4. The screen transition control unit 10 receives an operation corresponding to the user's operation from the device control unit 8, and determines a screen to be displayed corresponding to the operation. The display control unit 11 changes the screen based on the determined screen.

  Note that when the operation determination unit 4 refers to the operation standard and determines an operation from the bioelectric potential of the user, the operation determination unit 4 determines an operation corresponding to the user's operation.

<Modification 1>
22A and 22B show an example of changing the operation speed on the selection screen of the video information 121. FIG. In the example shown in FIG. 22A, the user 1 is operating with only the index finger raised. In the example shown in FIG. 22B, the user 1 is operating with the index and middle fingers raised. 22A and 22B, the user 1 is performing an operation of scrolling the video information 121 up, down, right, or left.

  At this time, the moving speed of the screen when the operation shown in FIG. 22B is performed is made faster than the moving speed of the screen when the operation shown in FIG. 22A is performed.

  When the finger or arm is moved up, down, right, or left, the operation with the index finger and the middle finger raised is more unnatural than the operation with only the index finger raised. It is considered that the normal operation is to move the fingers or arms with all fingers raised or in the state shown in FIG.

  Therefore, when the user 1 dares to perform the operation with the index finger and middle finger raised, it is considered that the user 1 understands the operation of the biopotential input interface system. If you understand (skilled) the operation of the system, you need to move quickly and smoothly. Therefore, when a relatively unnatural movement is made, an operation corresponding to the movement of the user 1 is made faster or larger than when a natural movement is made.

  For example, according to the user's action, when the number of fingers to be raised is larger than a predetermined number, the operation may be made faster or larger. Alternatively, when the number of fingers to be raised is less than a predetermined number, the operation may be made faster or larger.

  Further, for example, when the user operates during the transition of the display screen, the operation is performed faster or larger than when the user operates after the display screen is displayed.

<Modification 2>
The motion detection unit 3 is composed of an acceleration sensor. The acceleration sensor is attached to the user 1. It is desirable to arrange in the vicinity of the biopotential measurement unit 2 or the measurement electrode. User actions can be acquired from different viewpoints of biopotential and acceleration.

  For example, the case where the user 1 performs the operation (a stone shape) shown in FIG. 13 will be described as an example. It is considered that the operation when the user 1 performs the operation a (stone shape) can be divided into two operations. The first operation is an operation of swinging down from the top of the fingertip to the bottom of the fingertip. The second operation is an operation of moving the fingertip into a stone shape while swinging down the arm.

  When the user 1 wears the sensor unit 100 on the forearm of the arm and measures the myoelectricity by the sensor unit 100, the first operation is less likely to be detected because there is little movement of the forearm muscle. Therefore, when the sensor unit 100 includes the motion detection unit 3 configured by an acceleration sensor, the motion detection unit 3 can detect the first motion, and the motion determination unit 4 can determine the second motion. Normally, the user 1 can perform biopotential input by performing the first operation and the second operation as a series of movements.

  For example, it may be determined whether the motion detection unit 3 is configured by an acceleration sensor or a biopotential sensor according to a user's motion corresponding to the operation.

<DB creation method>
FIG. 23 shows a method for creating the determination standard (DB) shown in FIGS. 18A to 18C by the determination standard creation unit. The determination reference creation unit may be included in the configuration of the biopotential input interface system.

(S301)
The type of user action (gesture) is determined. For example, the operations are C, D, L, S, B, and H shown in the video information 121 of FIG.

(S302)
A bioelectric potential (for example, myoelectric potential) when each operation (gesture) is performed is acquired.

(303)
The screen displayed on the display unit 12 is associated with the operation determined in S301.

(S304)
For each operation (gesture), a rule (for example, a linear discrimination coefficient) is extracted from the acquired bioelectric potential information.

(S305)
For each screen displayed on the display unit 12, rules (for example, linear discrimination coefficients) for discriminating bioelectric potentials corresponding to a plurality of operations are accumulated in the criterion storage unit 5.

<Specific measurement results>
FIG. 24 shows an example of myoelectric potential measured by the sensor unit 100. The vertical axis indicates the magnitude of the potential, and the horizontal axis indicates time. The myoelectric potential shown in FIG. 24 includes the potential in the time range of the normal state A in which the user is not operating, the potential in the time range of the operation (starting point) B in which the user is intentionally operating, and the recognition target space. Potential in the C time range. The fact that the user is not operating means that the user is not performing an operation for operating the control target device.

  In the time range of the normal state A in which the user is not operating as shown in FIG. 24, the time change of the myoelectric potential is small.

  In the time range of the operation (starting point) B shown in FIG. 24, the time change of the myoelectric potential is large and a very large potential is observed as compared with the myoelectric potential in the normal state A. When such a potential is measured, the motion detection unit 3 detects that there is a user's intentional motion.

  Based on the potential in the time range of the recognition target section C shown in FIG. 24, the user's action is detected.

  FIG. 25 shows a flowchart of processing for recognizing the user's operation from the potential shown in FIG.

(S401)
The bioelectric potential measurement unit 2 measures the bioelectric potential of the user 1.

(S402)
The motion detection unit 3 refers to the detection standard and detects an intentional motion corresponding to the bioelectric potential of the user 1.

(S403)
If the motion detection unit 3 detects an intentional motion, the process proceeds to S404. At this time, the time when the motion detection unit 3 detects the intentional motion is transmitted to the motion determination unit 4. If the motion detection unit 3 has not detected an intentional motion, the process returns to S401.

(S404)
The motion determination unit 4 determines an analysis interval within a predetermined time range from the time when the motion detection unit 3 detects an intentional motion. In the example shown in FIG. 24, the range of the predetermined time is a time range (a time range of the recognition target space C) several seconds after the time when the intentional motion is detected.

(S405)
The action determination unit 4 extracts the feature amount of the potential in the determined analysis section. The feature amount includes information such as the magnitude of the potential in the analysis section and the temporal change of the potential. The operation determination unit 4 extracts information corresponding to the determination criterion information stored in the determination criterion storage unit 5 as a feature amount.

(S406)
The operation determination unit 4 refers to the determination criterion and determines an operation corresponding to the feature amount of the analysis section.

(S407)
The determination result transmission unit 6 transmits the operation determined by the operation determination unit 4 to the control target device 102.

  In S406, the motion determination unit 4 may determine an operation corresponding to the feature amount of the analysis section using the operation criterion. At that time, the determination result transmission unit 6 transmits the operation determined by the operation determination unit 4 to the control target device 102.

  According to the myoelectric input interface device, the gesture characteristics are utilized, the movements of up, down, left, and right are recognized as they are, and a finger character gesture is used to select a plurality of icons displayed on the screen. By using together, it becomes a simpler operation. The present invention can be applied to operations that require selection from a plurality of icons among device operations, specifically, operations on televisions, smartphone operations, operation panels of household devices, and the like.

Wear multiple electrodes on the circumference of the user's forearm,
A biopotential measurement unit that measures the potential difference between the electrodes generated in conjunction with the movement of the hand;
An intentional motion detection unit that detects a user's control intention from the time change of the potential difference measured by the biopotential measurement unit;
A criterion storage for storing the correspondence between the biopotential pattern generated by the user's control operation and the operation;
When the intentional motion is detected by the intentional motion detection unit, an operation determination unit that determines the user's motion according to the criterion stored in the determination criterion storage unit;
A myoelectric input interface device including a determination result transmission unit configured to transmit a user operation determined by the operation determination unit to the outside for device control of a control target.

A myoelectric input interface device;
A determination result receiving unit that receives a user operation result output from the myoelectric input interface device;
A screen transition control unit that calculates a transition state of a screen to be displayed to the user based on the determination result received by the determination result receiving unit;
The screen transition control unit, after switching the criterion stored in the criterion storage unit, a display control unit for controlling display content based on the output of the screen transition control unit;
A myoelectric input interface system comprising: a display unit that displays information to a user under the control of the display control unit.

  The determination reference accumulation unit includes a gesture for moving a finger up, down, left, and right. When the operation determination unit determines up, down, left, and right, the display control unit performs display control to scroll the entire screen up, down, left, and right. Myoelectric input interface system.

  A plurality of options are displayed on the screen controlled by the screen control unit, each option is associated with a predetermined gesture, and when a predetermined gesture is determined by the motion determination unit, an association is performed. EMG input interface system where selected options are selected.

  Multiple options are displayed on the screen controlled by the screen control unit, and gestures associated with each option are indicated by characters and images, and the user responds to each option according to the displayed characters and images. EMG input interface system to input gestures made.

  A myoelectric input interface system in which a plurality of options are displayed on the screen controlled by the screen control unit, and numerals are displayed near each option, and the user inputs the displayed numerals with a gesture.

  A myoelectric input interface system in which a plurality of options are displayed on the screen controlled by the screen control unit, characters are shown near each option, and the user inputs the displayed characters with a gesture.

  A plurality of choices are displayed on the screen controlled by the screen control unit, and characters related to the choices are displayed near each choice, and the user inputs the indicated characters with a gesture. .

  A plurality of choices are displayed on the screen controlled by the screen control unit, the first letter of the choice is shown near each choice, and the myoelectric input interface system in which the user inputs the indicated letter with a gesture .

  A plurality of choices are displayed on the screen controlled by the screen control unit, and finger gesture images are shown near each choice. Interface system.

A biopotential measurement step for measuring the potential difference between the electrodes generated in conjunction with the movement of the hand;
An intentional motion detection step of detecting the user's control intention from the time change of the potential difference measured by the biopotential measurement unit;
A criterion accumulation step for storing the correspondence between the biopotential pattern generated by the user's control action and the action;
In the intentional motion detection unit, when an intentional motion is detected, an operation determination step of determining a user's operation according to the criterion stored in the determination criterion accumulation unit;
A determination result transmission step of transmitting the user's operation determined by the operation determination unit to the outside for controlling a device to be controlled;

A biopotential measurement step for measuring the potential difference between the electrodes generated in conjunction with the movement of the hand;
An intentional motion detection step of detecting the user's control intention from the time change of the potential difference measured by the biopotential measurement unit;
A criterion accumulation step for storing the correspondence between the biopotential pattern generated by the user's control action and the action;
In the intentional motion detection unit, when an intentional motion is detected, an operation determination step of determining a user's operation according to the criterion stored in the determination criterion accumulation unit;
A computer program for executing a determination result transmission step of transmitting a user operation determined by the operation determination unit to the outside for controlling a device to be controlled.

DESCRIPTION OF SYMBOLS 1 User 2 Bioelectric potential measurement part 3 Operation | movement detection part 4 Operation | movement determination part 5 Judgment reference | standard accumulation | storage part 6 Judgment result transmission part 7 Judgment result reception part 8 Device control part 9 Control object apparatus main body 100, 100A Sensor part 101 Recognition part 102 Control object machine

Claims (10)

A biopotential measurement unit that measures a user's biopotential using a plurality of measurement electrodes;
An operation detection unit that detects an operation corresponding to the first operation of the user from the bioelectric potential of the user with reference to a detection standard including a voltage of a first threshold;
When the motion detection unit detects a motion corresponding to the first motion, the user motion corresponding to the measured user biopotential is referred to with reference to a determination criterion that correlates the user motion and the biopotential. An operation determining unit for determining;
A device that determines an operation corresponding to the determined user's operation with reference to an operation standard that associates the operation of the control target device with the user's operation, and transmits the determined operation to the control target device main body. A control unit;
Biopotential input interface system. further,
A display unit;
According to the operation determined by the device control unit, a screen transition control unit that determines a screen to be displayed on the display unit,
For each screen displayed on the display unit, a determination criterion storage unit that stores the determination criterion,
The operation determination unit corresponds to the screen determined by the screen transition control unit among the determination criteria stored in the determination criterion storage unit for the user's operation subsequent to the operation determined by the device control unit. Determining the user action corresponding to the measured bioelectric potential of the user with reference to a determination criterion to
The biopotential input interface system according to claim 1. A biopotential measurement unit that measures a user's biopotential using a plurality of measurement electrodes;
An operation detection unit configured by an acceleration sensor and detecting an operation corresponding to the first operation of the user when a change equal to or greater than a predetermined threshold is detected;
When the motion detection unit detects a motion corresponding to the first motion, the user motion corresponding to the measured user biopotential is referred to with reference to a determination criterion that correlates the user motion and the biopotential. An operation determining unit for determining;
A device that determines an operation corresponding to the determined user's operation with reference to an operation standard that associates the operation of the control target device with the user's operation, and transmits the determined operation to the control target device main body. A control unit;
Biopotential input interface system. The device controller is
When the user's action is an action in a state where the number of fingers that are standing at the time of the user's action is large, the speed of the determined operation is increased.
The biopotential input interface system according to claim 1 or 3. The device controller is
If the user's action is an action with a large number of fingers that are raised during the user's action, the speed of the determined operation is reduced.
The biopotential input interface system according to claim 1 or 3. A biopotential measurement step for measuring a user's biopotential using a plurality of measurement electrodes;
An operation detecting step of detecting an operation corresponding to the first operation of the user from the bioelectric potential of the user with reference to a detection standard including a voltage of the first threshold;
When an operation corresponding to the first operation is detected in the operation detection step, the user operation corresponding to the measured user bioelectric potential is referred to with reference to a determination criterion that associates the user operation with the bioelectric potential. An operation determining step for determining;
A device that determines an operation corresponding to the determined user's operation with reference to an operation standard that associates the operation of the control target device with the user's operation, and transmits the determined operation to the control target device main body. A control step,
Biopotential input method. further,
In accordance with the operation determined by the device control unit, including a screen transition control step for determining a screen to be displayed on the display unit,
Of the determination criteria stored in the determination reference storage unit that stores the determination reference for each screen displayed on the display unit with respect to the user's operation subsequent to the operation determined by the device control unit, the screen transition With reference to a determination criterion corresponding to the screen determined by the control unit, the user action corresponding to the measured bioelectric potential of the user is determined.
The biopotential input method according to claim 6. A biopotential measurement step for measuring a user's biopotential using a plurality of measurement electrodes;
An operation detection step of detecting an operation corresponding to the first operation of the user when an acceleration sensor mounted on the user detects a change greater than or equal to a predetermined threshold;
When the motion detection step detects a motion corresponding to the first motion, the user motion corresponding to the measured user biopotential is referred to with reference to a determination criterion that correlates the user motion with the biopotential. An operation determining step for determining;
A device that determines an operation corresponding to the determined user's operation with reference to an operation standard that associates the operation of the control target device with the user's operation, and transmits the determined operation to the control target device main body. A control step,
Biopotential input method. The device control step includes
When the user's action is an action in a state where the number of fingers that are standing at the time of the user's action is large, the speed of the determined operation is increased.
The biopotential input method according to claim 6 or 8. The device control step includes
If the user's action is an action with a large number of fingers that are raised during the user's action, the speed of the determined operation is reduced.
The biopotential input method according to claim 6 or 8.

Images