KR100504217B1 - Human-Machine Interface System and Method based on EMG - Google Patents

Abstract

The present invention relates to a human-machine interface device and method for measuring and analyzing EMG signals obtained from various muscles in which a user can actively move, to grasp a user's command, and to operate various mechanical devices according to the user's command. will be.

A human-machine interface device using an EMG signal according to the present invention includes: a bipolar electrode mounted on a user muscle to obtain an EMG signal of a user; Preprocessing means for filtering a user's EMG signal obtained from the bipolar electrode and converting the EMG signal into a digital value; A memory for storing driving commands of a peripheral device to be controlled and movement of a user; A microprocessor that recognizes a user's movement by using the EMG signal passing through the preprocessor, and obtains and outputs a driving command corresponding to the user's movement from the memory; And a control signal transmission unit for wirelessly transmitting a peripheral device driving control signal according to the driving command output from the microprocessor.

Description

Human-Machine Interface System and Method based on EMG Signal

The present invention relates to a human-machine interface device and method using an EMG signal, and more particularly, to determine the user's command by measuring and interpreting EMG signals obtained from various muscles in which the user can actively move. A human-machine interface device and method for operating various machinery in accordance with the instructions of the present invention.

The EMG signal is one of the biological signals that can be measured when a specific muscle contracts, and has an advantage of being easily measured without harming the user. At present, many technologies for accurately measuring EMG, pattern learning and classification method using EMG signals, virtual reality technology using EMG signals, and techniques for controlling arm prosthetic devices are disclosed (Korean Patent Publication No. 1989-016950, 1999-066563, 1999-066564, and 2002-077346.

However, an interface technology for wirelessly measuring and interpreting EMG signals to grasp a user's command and controlling the devices using the identified user's command has not been disclosed.

The present invention has been made in order to solve the problems of the prior art as described above, measuring the left, right, front, back movement of the user's arm or the movement of the hand grip, stretch, rotate through the EMG signal, measured EMG A human-machine interface device and method for recognizing a user's command from a signal and adjusting various peripheral devices.

In the human-machine interface device using the EMG signal according to the present invention for achieving the above object, in the human-machine interface device for recognizing the movement of the user muscle and outputs a peripheral drive control signal,

A bipolar electrode mounted on the user muscle to obtain an EMG signal of the user;

Preprocessing means for filtering a user's EMG signal obtained from the bipolar electrode and converting the EMG signal into a digital value;

A memory for matching and storing a driving command of the peripheral device and a user's movement;

A microprocessor that recognizes a user's movement by using the EMG signal that has passed through the preprocessor, and obtains and outputs the driving command corresponding to the user's movement from the memory;

And a control signal transmission unit for wirelessly transmitting a peripheral device driving control signal according to the driving command output from the microprocessor.

In addition, the human-machine interface method using the EMG signal according to the present invention, in the human-machine interface method for outputting a peripheral drive control signal by recognizing the movement of the user muscle,

A data storage step of storing movement data of a user according to a peripheral device driving command;

A signal acquisition step of acquiring an EMG signal of a user after the data storage step;

A motion recognition step of recognizing a user's movement by using the EMG signal obtained in the signal acquisition step;

A command search step of searching for a peripheral device driving command corresponding to the user's movement recognized in the motion recognition step;

And a control signal transmission step of generating a peripheral device driving control signal according to the peripheral device driving command found in the command searching step and transmitting the wirelessly.

According to the present invention, there is also provided a computer-readable recording medium having recorded thereon a program for executing the human-machine interface method using the EMG signal as described above with respect to the human-machine interface device.

Hereinafter, a human-machine interface device and method using EMG signals according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a human-machine interface device using EMG signals according to an embodiment of the present invention.

This means that the pair of bipolar electrodes 101a, 101b, 111a, 111b, 121a, 121b, 131a, and 131b measuring the EMG signal in the muscle of the user and the EMG signal measured by each pair of bipolar electrodes are about 1500V. Four differential amplifiers 102, 112, 122, and 132 differentially amplified to / V magnitude, and four power amplifiers that remove 60 Hz of power amplified signal from each of the differential amplifiers 102, 112, 122, and 132. Four bandpasses passing only signals in the required band of 30 Hz to 500 Hz in the signal from which power noise is removed from the notch filters 103, 113, 123, and 133 and the respective notch filters 103, 113, 123, and 133. A four-channel A / D converter 141 for converting the analog signal passed through the filters 104, 114, 124, and 134, the respective bandpass filters 104, 114, 124, and 134 into a digital signal, A microprocessor 142 for acquiring a user's command using the EMG signal digitally converted by the A / D converter 141 and the microprocessor 142. The memory 143 stores the pattern recognition information according to the feature set of the EMG signal learned by the user under control, and provides the microprocessor 142 with the memory 143 and the user input value. A key input unit 144, a display unit 145 for displaying the information required by the user under the control of the microprocessor 142, and a peripheral device command interface for signal processing to deliver the command identified by the microprocessor 142 to the peripheral device And a wireless transmitter 147 for wirelessly transmitting the user command signaled by the peripheral device command interface unit 146 to the peripheral device, and a power supply unit (not shown) for supplying power required for each component. In this embodiment, four pairs of bipolar electrodes are attached to the muscle of the user, but the present invention is not limited thereto. Is the number of bipolar electrodes and a differential amplifier for processing the electromyogram signal obtained from the bipolar electrode, notch filters, bandpass filters, A / D converter may increase or decrease.

The human-machine interface device using the EMG signal configured as described above operates as follows.

2 is a schematic operation flowchart illustrating a human-machine interface method using an EMG signal according to the present invention.

First, a pair of bipolar electrodes are attached to measure the EMG signal along the upper and lower muscles of the user's arm. 3 is a diagram illustrating the upper and lower muscles to which the bipolar electrode is attached to measure the EMG signal. The biceps can be attached to the upper biceps and triceps, and the lower biceps include autologous muscle contractions, autologous muscles, finger muscles, and twisted muscles.

In the present embodiment, four pairs of bipolar electrodes 101a, 101b, 111a, 111b, 121a, 121b, 131a, and 131b are attached to the user's lower extremity muscles, namely, the autologous muscle contraction muscle, the muscle muscle muscle, and the finger muscle and the torso. The movement of the arm motion is measured.

By attaching four pairs of bipolar electrodes 101a, 101b, 111a, 111b, 121a, 121b, 131a, and 131b around the user's lower abdominal muscles, the left, right, front, back and hand grips, straightening, EMG signals for various arm motions such as rotation are obtained (S21).

In this case, the EMG signal obtained from any pair of bipolar electrodes is signal-processed by the differential amplifier, notch filter, and bandpass filter connected to the pair of bipolar electrodes, and preprocessed by the A / D converter (S22). In other words, the EMG signals obtained by the first dipole electrodes 101a and 101b are transmitted by the first differential amplifier 102, the first notch filter 103, the first band pass filter 104, and the A / D converter 141. Preprocessed. The EMG signals obtained by the second dipole electrodes 102a and 102b are preprocessed by the second differential amplifier 112, the second notch filter 113, the second band pass filter 114, and the A / D converter 141. . The EMG signals obtained by the third dipole electrodes 121a and 121b are preprocessed by the third differential amplifier 122, the third notch filter 123, the third band pass filter 124, and the A / D converter 141. . Finally, the EMG signals obtained by the fourth dipole electrodes 131a and 131b are obtained by the fourth differential amplifier 132, the fourth notch filter 133, the fourth band pass filter 134, and the A / D converter 141. Preprocessed.

Each of the differential amplifiers 102, 112, 122, and 132 differentially amplifies the EMG signal of the arm muscle obtained from a pair of bipolar electrodes, and each of the notch filters 103, 113, 123, and 133 is differentially amplified. Blocks 60Hz power supply noise from the EMG signal. Each of the band pass filters 104, 114, 124, and 134 removes low frequency noise and high frequency noise, which are a kind of motion noise. The A / D converter 141 converts the analog EMG signal from which the noise is removed from the band pass filter into a digital signal and provides the digital signal to the microprocessor 142.

The microprocessor 142 extracts the features of the pre-processed EMG signal (S23), recognizes the pattern (S24), and transfers the peripheral device command interface unit 146 to drive various peripheral devices according to the recognized pattern (S25). ). A process of recognizing a command for driving a peripheral device by extracting a feature from a preprocessed EMG signal and recognizing a pattern will be described below.

4 is a flowchart illustrating a method of driving a peripheral device using an EMG signal of a microprocessor according to an embodiment of the present invention.

First, when the EMG-based interface device is powered on, the microprocessor recognizes a peripheral device to be controlled by the user, and senses the mode of the peripheral device (S401). For example, when controlling the television among home appliances, when the power of the EMG-based interface device is initially turned on, the EMG-based interface device according to the present invention operates in an initial mode to learn the user's movement. After the EMG-based interface device according to the present invention learns the movement of the user, the EMG-based interface device is switched to the execution mode and operates in the execution mode at the next power-on.

If the mode of the peripheral device detected in step S401 is the initial mode (S402), the microprocessor guides the movement of the arm to learn the user's movement (S403). That is, the user may select a command by himself or herself from among various commands for driving the television. For example, if the user selects six instructions, such as 'channel up', 'channel down', 'volume up', 'volume down', 'power on', 'power off', etc., the microprocessor The corresponding motion data should be received and stored.

To this end, the microprocessor displays the six instructions selected by the user in the initial mode and the motion data input request according to each instruction on the screen, and waits for the movement data according to each instruction to be input from the user (S403). When motion data according to any one command is input (S404), the EMG signal is processed by signal processing to extract a feature (S405), and the pattern feature is extracted and stored in the memory (S406). ).

For example, in the initial mode, the microprocessor inquires on the screen which motion to set the 'channel up' command and waits for input from the user. At this time, when the user attempts to set the motion for the 'channel up' command to the 'handhold' motion, and repeatedly performs the handhold motion, four pairs of bipolar electrodes sense the EMG signal of the user's arm motion. The EMG signals detected by the four pairs of bipolar electrodes are preprocessed through the differential amplifier, notch filter, bandpass filter, and A / D converter as described above and input to the microprocessor. The microprocessor then extracts the features needed to recognize the pattern from the preprocessed EMG signal.

At this time, the features extracted by the microprocessor include an absolute integral value (IAV), a dispersion value (VAR), a zero-crossing coefficient (ZC), and the like, and the microprocessor extracts a feature for each EMG signal input from each bipolar electrode.

Integral Absolute Value (IAV) is a value obtained by integrating the absolute value of the EMG signal acquired for a predetermined time, and is defined as in Equation 1 below.

Here, j is a channel number, x _j is an EMG signal measured in channel j, i is an order of samples, Δt is a sampling time interval, and N is the number of samples.

Variance (VAR) is a feature indicating how far the EMG signal is distributed from the average value, and is expressed as in Equation 2 below.

here, And x _j is an average value.

Zero Crossing (ZC) is the number of times the EMG signal passes the zero point, and is expressed as in Equation 3 below.

here, to be.

After extracting the features in step S405, the microprocessor learns a pattern recognizer (FMMNN) using the feature sets of the extracted EMG signals and stores the learned information in a memory (S406). The user's motion data for one television command is learned through the above-described steps S403 to S406.

Next, it is determined whether all user motion data for all the television commands selected by the user are collected (S407). If all the data has been collected, the process proceeds to step S401 after switching to the execution mode (S408). If all the data have not been collected, steps S403 to S406 are repeated for other television commands.

An example of arm and hand movements for the television command selected by the user in the initial mode of the microprocessor is summarized in Table 1.

TABLE 1

On the other hand, if the mode of the peripheral device is the execution mode in step S402, the microprocessor waits for data input for driving the television (S409). When the EMG signal is input from the user in the air (S410), the features (absolute integral value, variance value, zero crossing) are extracted by applying Equations 1 to 3 to the input EMG signal (S411).

The microprocessor applies the corresponding features to the pattern recognizer, classifies the pattern, searches the memory, and recognizes a television command corresponding to the user's movement (S412). Then, the microprocessor transmits the recognized television command to the peripheral device, that is, the television, through the peripheral device command interface unit and the wireless transmitter (S413).

Steps S409 to S413 are repeatedly performed until the operation of the human-machine interface device according to the present invention is terminated (S414).

For example, in the run mode, when the user takes the action of 'move to the left of the wrist' corresponding to 'channel down', the microprocessor detects the movement of the user by extracting features of the EMG signal corresponding to the user's movement. After recognizing a television command corresponding to the movement, a control signal corresponding to a channel down is output to the television through a wireless transmitter. In addition, when the user takes 'move right to wrist' or 'move down' corresponding to 'channel up' or 'volume down', the microprocessor detects the user's movement and sends the control signal corresponding to the instruction. Transmit to television. In addition, if the user's movement is 'move up the wrist' control signal corresponding to 'volume up', if the user's movement is 'holding' or 'rotating the wrist' it corresponds to 'power off' or 'power on' Output the control signal wirelessly.

Although the technical spirit of the present invention has been described above with reference to the accompanying drawings, it is intended to exemplarily describe the best embodiment of the present invention, but not to limit the present invention. In addition, it is obvious that any person skilled in the art may make various modifications and imitations without departing from the scope of the technical idea of the present invention.

According to the present invention as described above, there is an effect that can easily adjust a variety of peripheral devices (electric appliances, computers, robot devices, etc.) wirelessly from the movement of the user's arm or hand.

1 is a block diagram showing a human-machine interface device using an EMG signal according to an embodiment of the present invention,

2 is a schematic flowchart illustrating a human-machine interface method using an EMG signal according to an embodiment of the present invention;

3 shows candidate muscle positions for measuring EMG signals;

4 is an operation flowchart illustrating a control method of a microprocessor according to an exemplary embodiment of the present invention.

<Brief description of symbols for the main parts of the drawings>

101a, 101b, 111a, 111b, 121a, 121b, 131a, 131b; Bipolar electrodes

102, 112, 122, 132; Differential amplifiers

103, 113, 123, 133; Notch filters

104, 114, 124, 134; Bandpass Filters

141; A / D converter 142; Microprocessor

143; Memory 144; Key input section

145; Display unit 146; Peripheral Command Interface Section

147; Wireless transmitter

Claims (13)

In the human-machine interface device for recognizing the movement of the user muscles and outputs a drive control signal to the peripheral device, A bipolar electrode mounted on the user muscle to obtain an EMG signal of the user; Preprocessing means for filtering a user's EMG signal obtained from the bipolar electrode and converting the EMG signal into a digital value; A memory for matching and storing a driving command of the peripheral device and a user's movement; It operates in the initial mode and the execution mode, and during the initial mode operation, the driving commands required for the control of the peripheral device are sequentially displayed to request the user's movement, and the user's movement is learned using the EMG signal passed through the preprocessing means. Match the driving command and the user's movement to the memory, recognize the user's movement using the EMG signal passed through the pre-processing means during the execution mode operation, and apply the recognized user's movement to the memory A microprocessor recognizing a peripheral device driving command corresponding to the movement of the user; And a control signal transmitter for wirelessly transmitting peripheral drive control signals according to the drive command output from the microprocessor. The method of claim 1, wherein the pretreatment means, A differential amplifier for differentially amplifying the EMG signal obtained from the bipolar electrode, a notch filter for blocking power supply noise from the EMG signal passing through the differential amplifier, and a bandpass filter for removing motion noise from the EMG signal passing through the notch filter And an analog-to-digital converter for converting the EMG signal passed through the band pass filter into a digital value and providing the same to the microprocessor. According to claim 1, wherein the bipolar electrode is made of a plurality of bipolar electrodes mounted on the upper or lower muscles of the user's arm, The preprocessor passes a plurality of differential amplifiers for differentially amplifying the EMG signals obtained from the bipolar electrodes, a plurality of notch filters for blocking power supply noise from the EMG signals passed through the respective differential amplifiers, and the respective notch filters. And a plurality of bandpass filters for removing motion noise from one EMG signal, and a plurality of channels of analog-to-digital converters that convert the EMG signals passed through the respective bandpass filters into digital values and provide them to the microprocessor. Human-machine interface device using an EMG signal, characterized in that. delete The human-machine interface device of claim 1, wherein the microprocessor uses a feature value obtained from the EMG signal of the user to learn and recognize the movement of the user. 6. The human-machine interface device of claim 5, wherein the feature value is at least one of an absolute integral value, a variance value, and a zero crossing. The method of claim 1, wherein the control signal transmitting unit, And a peripheral device command interface unit for outputting the peripheral device driving control signal according to the driving command, and a wireless transmission unit for wirelessly transmitting the peripheral device driving control signal. delete In the human-machine interface method for recognizing the movement of the user muscle and outputs a peripheral drive control signal, An initial mode step of learning a user's movement according to a driving command required for controlling the peripheral device; a recognition of the user's movement after the initial mode step, and a search of a driving command corresponding to the recognized user's motion to drive the control signal to the peripheral device; Outputting a run mode step; The initial mode step may include: a motion request step of sequentially displaying the driving command to a user and requesting a motion according to the peripheral device driving command; and a first EMG signal obtaining an EMG signal according to the user's motion after the motion request step; A first movement recognition step of recognizing the movement of the user by using the acquisition, the EMG signal obtained in the first EMG signal acquisition step, and a storing step of matching and storing the driving command and the movement of the user; , The execution mode step may include a second EMG signal acquisition step of acquiring an EMG signal according to a user's movement, and a second motion recognition step of recognizing a user's movement using the EMG signal acquired in the second EMG signal acquisition step. And a command search step for searching for a driving command corresponding to the movement of the user recognized in the second movement recognition step, and a control signal for generating a peripheral device driving control signal according to the driving command found in the command searching step and transmitting it wirelessly. Human-machine interface method using an EMG signal comprising a transmission step. 10. The human-machine interface method of claim 9, wherein the motion recognition step uses a feature value obtained from an EMG signal according to the movement of the user. The human-machine interface method of claim 10, wherein the feature value is at least one of an absolute integral value, a variance value, and a zero crossing. 10. The method of claim 9, wherein the initial mode step further comprises a command selection request step of requesting a user to select a peripheral drive command to be set. And a step of requesting movement of the peripheral device selected by the user after the command selection requesting step. A computer-readable recording medium recording a program for executing a human-machine interface method for recognizing movement of a user muscle and outputting a peripheral drive control signal to a human-machine interface device, The human-machine interface method, An initial mode step of learning a user's movement according to a driving command required for controlling the peripheral device; a recognition of the user's movement after the initial mode step, and a search of a driving command corresponding to the recognized user's motion to drive the control signal to the peripheral device; Outputting a run mode step; The initial mode step may include: a motion request step of sequentially displaying the driving command to a user and requesting a motion according to the peripheral device driving command; and a first EMG signal obtaining an EMG signal according to the user's motion after the motion request step; A first movement recognition step of recognizing the movement of the user by using the acquisition, the EMG signal obtained in the first EMG signal acquisition step, and a storing step of matching and storing the driving command and the movement of the user; , The execution mode step may include a second EMG signal acquisition step of acquiring an EMG signal according to a user's movement, and a second motion recognition step of recognizing a user's movement using the EMG signal acquired in the second EMG signal acquisition step. And a command search step for searching for a driving command corresponding to the movement of the user recognized in the second movement recognition step, and a control signal for generating a peripheral device driving control signal according to the driving command found in the command searching step and transmitting it wirelessly. A computer-readable recording medium comprising a transmission step.