KR101520462B1

KR101520462B1 - Apparatus for interface with disabled upper limbs

Info

Publication number: KR101520462B1
Application number: KR1020130147207A
Authority: KR
Inventors: 이응혁; 엄수홍; 장문석; 이원영; 송기선
Original assignee: 한국산업기술대학교산학협력단
Priority date: 2013-11-29
Filing date: 2013-11-29
Publication date: 2015-05-18

Abstract

The present invention relates to an interface device for an upper limb disabled person, which can improve the accessibility of an information technology (IT) device of a handicapped person to replace a minute hand movements. The present invention relates to an interface device for an upper limb disabled person, An electromyogram signal measuring part for measuring; An accelerometer for detecting an arm motion state signal of the upper limb disabled person; A processor for recognizing an operation for controlling the control object based on the measured EMG and arm movement state signals and generating a control object control signal according to the recognized operation result; And a communication module that converts the control target control signal generated by the processor into an interface signal and transmits the interface signal to a control object.

Description

[0001] The present invention relates to an interface device for an upper limb,

The present invention relates to an interface device for a disabled person, and more particularly, to an interface device for a disabled person who is capable of improving accessibility to an IT (Information Technology) device of a disabled person so as to replace a minute hand movements.

With the advent of the smart era in which smart devices are activated, information technology devices such as smart phones and smart pads have appeared. Most of these IT devices are composed of touch input, so it requires fine manipulation. However, accessibility to IT devices such as smart devices is inadequate in cases such as those with disabilities where the upper extremity is severed due to an accident, These problems deepen the information gap between people with disabilities and non-disabled people through the reality of popularization. In order to improve this, an interface for improving the accessibility of IT devices is required instead of the minute hand movements.

Examples of assistive devices for improving the accessibility of IT devices for people with disabilities include ancillary devices such as an ocular mouse, an integral mouse, a big keyboard, and a foot switch. However, since the main use of these devices is the computer, portability is inconvenient, and it is not applicable to smart phones or smart pads. Therefore, the development of interfaces that can be applied to such smart phones and pads is required in the times.

Conventional technologies for improving the accessibility of IT devices such as smart phones and smart pads are disclosed in [Non-Patent Document 1] SH Lee, W Lee, JS Kong and EH Lee, "Gesture-based interface for the mobile use of the upper extremity disabilities ", 12th Rehabilitation Engineering & Assistive Technology Society of Korea, Jeonju, Korea, Nov, 2012, pp. 283-285. KS Kim, YH Han, WB Jung, YH Lee, JH Kang, HH Choi and CW Mun, "Portable Biosignal Measurement System Technologies", 10th Korea Game Society, Korea, Oct, 2010, pp. There is .65-73.

[Non-Patent Document 1] and [Non-Patent Document 2] describe a method of using a mouse as recognition of the movement of the neck and an interface of manipulating the game by the movement of a hand using an EMG signal of the arm .

SH Lee, WY Lee, JS Kong and EH Lee, "Gesture-based interface for the upper extremity disabilities", 12th Rehabilitation Engineering and Assistive Technology Society of Korea, Jeonju, Korea, Nov, 285. K. Kim, Y. Han, W. Jung, Y. Lee, J.H. Kang, H.H Choi and C. W Mun, "Portable Biosignal Measurement System Technologies", 10th Korea Game Society, Korea, Oct, 2010, pp.65-73.

However, the conventional method of recognizing the movement of the neck and using it as a mouse has a disadvantage in that it is not good at the time of use and it is inconvenient to use because of increased fatigue of the neck when used for a long time.

In addition, in the case of an interface using an EMG signal, it is difficult to apply it to a smartphone and it is difficult to use for a long time because it is operated only by the movement of the wrist.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an interface device for an upper limb disabled person who can improve the accessibility of information technology (IT) .

Another object of the present invention is to provide an interface device for a disabled person who provides a multi-sensor interface that is similar to a smartphone usage method of an ordinary person and considers ease of use, for improving the accessibility of an IT device, .

Another object of the present invention is to provide an interface device for a disabled person who provides a sensing device in which a gyro sensor, a motion recognition sensor, and a bio-signal detector, which are fused with a gyro sensor and an acceleration sensor, will be.

According to another aspect of the present invention, there is provided an interface device for a disabled person, comprising: an electromyogram signal measuring unit attached to an arm of a person with upper extremity and measuring an electromyogram, which is an action potential generated when the muscle contracts; An accelerometer for detecting an arm motion state signal of the upper limb disabled person; A processor for recognizing an operation for controlling the control object based on the measured EMG and arm movement state signals and generating a control object control signal according to the recognized operation result; And a communication module that converts the control object control signal generated by the processor into an interface signal and transmits the interface signal to a control object.

The EMG signal measuring unit includes an EMG detecting electrode attached to an arm of a person with upper limb and detecting an activity potential; A preamplifier for pre-amplifying the minute action potentials detected by the electromyogram detecting electrodes and outputting the resultant signals as a living body signal; A differential amplifier for amplifying the bio-signal amplified by the preamplifier according to a predetermined gain; A band-pass filter for filtering the biomedical signal output from the differential amplifier to a set band; A notch filter for removing noise contained in the bio-signal output from the band-pass filter; And an analog / digital converter for converting the analog electromyogram signal output from the notch filter into a digital electromyogram signal.

In the above, the EMG detecting electrode is attached to at least two of the chin-lateral carpometacarpal and lateral carpometacarpal muscles of the upper hand disabled person.

The processor includes a feature extraction unit for extracting features for pattern classification from the bio-signal output from the electromyogram signal measuring unit; A preprocessor for preprocessing the arm motion state signal output from the accelerometer by a moving average filter; A pitch angle / roll angle estimating unit for estimating a pitch angle and a roll angle in the data formatted through the preprocessing unit; A pitch angle / roll angle estimated by the pitch angle / roll angle estimating unit, and recognizes the muscle motion of the arm as the feature extraction value of the feature extracting unit, And an operation recognition unit for generating a control object control signal.

Wherein the feature extracting unit extracts features by calculating consecutively acquired electromyogram signals with absolute integral values.

The operation recognition unit recognizes the keyboard direction key operation by the front, back, left, and right tilt operations of the arm, recognizes the click and double click operations of the mouse with the feature extraction value, and recognizes the keyboard direction key operation recognition and the mouse operation recognition And recognizes a call receiving and a disconnecting operation by a combination of the telephone number and the telephone number.

According to the present invention, there is an advantage that the accessibility of IT (Information Technology) equipment of the upper handed person is improved, so that a minute hand operation can be substituted.

In addition, according to the present invention, it is possible to provide a multi-sensor interface that is similar to the smartphone usage method of a general person and improves the usability in order to improve the accessibility of the IT devices to the persons with disabilities whose upper limbs are severed or difficult to perform fine upper limbs.

1 is a block diagram of an interface device for a disabled person according to a preferred embodiment of the present invention;
FIG. 2 is a block diagram of an embodiment of the electromyogram signal measuring unit of FIG. 1,
FIG. 3 is a block diagram of an embodiment of the processor of FIG. 1,
Figs. 4A to 4C are diagrams illustrating an example of the operation configuration for control object control,
Figs. 5A to 5C are diagrams illustrating positions of electrodes attached to respective channels,
6A to 6C are diagrams showing results of electromyogram signals in which absolute integration values are performed,
7 is an exemplary view of angle data of the acceleration sensor according to the movement of the arm;

Hereinafter, an interface device for a disabled person according to a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an interface device for a disabled person according to a preferred embodiment of the present invention.

An interface device for a disabled person according to a preferred embodiment of the present invention includes an EMG signal measuring unit 10, an accelerometer 20, a processor 30, a communication module 40, and a power source unit 50.

The EMG signal measuring unit 10 is attached to an arm of a disabled person and measures the EMG, which is an action potential generated when the muscle contracts.

2, the electromyogram signal measuring unit 10 includes an electromyogram detecting electrode 11 attached to an arm of a person with upper hand and detecting an action potential; A preamplifier 12 for preamplifying the minute action potentials detected by the electromyogram detecting electrode 11 and outputting the resultant signals as a living body signal; A differential amplifier 13 for amplifying the bio-signal amplified by the preamplifier 12 according to a predetermined gain; A band-pass filter (14) for filtering the biomedical signal output from the differential amplifier (13) into a preset band; A notch filter (15) for removing noise contained in the bio-signal output from the band-pass filter (14); And an analog / digital converter (16) for converting the analog electromyogram signal output from the notch filter (15) into a digital electromyogram signal.

Here, it is preferable that at least two electromyographic detection electrodes 11 are attached to the chin-lateral carpometacarpal and lateral carpometacarpal muscles of the upper handed person.

The accelerometer 20 serves to detect an arm motion state signal of a disabled person. The accelerometer 20 preferably uses an acceleration sensor or a gyro sensor.

The processor 30 recognizes an operation for controlling the control object based on the measured electromyogram and the arm motion state signal, and generates a control object control signal according to the recognized operation result.

As shown in FIG. 3, the processor 30 includes a feature extraction unit 31 for extracting features for pattern classification from the bio-signal output from the electromyogram signal measuring unit 10; A preprocessor 32 for preprocessing the arm motion state signal output from the accelerometer 20 by a moving average filter; A pitch angle / roll angle estimation unit 33 for estimating a pitch angle and a roll angle from the data formatted through the preprocessing unit 32; The upper and lower arms and the left and right arms are determined by the pitch angle / roll angle estimated by the pitch angle / roll angle estimating unit 33, and the muscle motion of the arm is recognized as the feature extraction value of the feature extracting unit 31 And an operation recognition unit 34 for generating a control object control signal in accordance with the recognized operation result.

The communication module 40 converts the control target control signal generated in the processor 30 into an interface signal and transmits the interface signal to the control object. Preferably, the communication module 40 uses a wireless communication module. Particularly, it is more preferable to use a Bluetooth module among various wireless communication modules.

The power source unit 50 serves to supply driving power to the processor 30 and a power source using a power source of the built-in battery.

The operation of the above-configured interface device for a disabled person according to the present invention will now be described in detail.

First, the electromyogram detecting electrode 11 is attached to the arm of the upper hand for the purpose of EMG measurement, and the 3-axis acceleration sensor is mounted at an appropriate position of the arm of the upper hand.

Here, the electromyographic detection electrode 11 uses a 2-pole type electrode, and uses at least two. It is desirable to select a position at which the electromyogram detection electrode 11 can be accurately sensed when the user lifts the hand of the user. In the present invention, as shown in FIG. 5A and FIG. 5B, the electrode of channel 1 adheres to the lateral carpal flexor and the electrode of channel 2 attaches to the lateral carpal flexor, as shown in FIG. 5C.

In general, electromyogram (EMG) measures the activity potential generated when a muscle contracts, and has an amplitude of several mV and a frequency characteristic of 0 to 500 Hz. Electromyogram (EMG) signals are measured using an electrode, usually non-invasive and invasive. In the present invention, an EMG signal was obtained using a non-invasive bipolar snap electrode for detecting surface EMG.

When an upper limb disabled person moves his or her arm to operate the control target device, an electromyogram (EMG) signal is detected by the electromyogram detecting electrode 11. As shown in FIG. 2, the electromyogram signal thus detected amplifies a fine signal through the preamplifier 12 and differentially amplifies the differential amplifier 13 by a gain of 100. The EMG signal amplified through the differential amplifier 13 is band-filtered by the band filter 14 whose band is set to 10 Hz to 500 Hz, and only the effective frequency of the EMG signal passes. The notch filter 15 removes the 60 Hz phase noise included in the electromyogram signal that has passed through the bandpass filter 14. The noisy EMG signal is converted into a digital signal through the A / D converter 16, and the converted digital signal is transmitted to the processor 30 by the bio-signal (EMG signal) of the disabled person.

Next, when the upper limb disabled person moves his or her arm to operate the control target device, the accelerometer 20 detects the arm movement state signal using the gyro sensor or the acceleration sensor and transmits it to the processor 30.

The feature extraction unit 31 of the processor 30 extracts features for EMG signal recognition. In EMG signal recognition, there are a feature set based on the amplitude of a signal and a method of extracting from a frequency domain. Based on the magnitude of the signal, there are absolute integral (IAV), variance (VAR), and Williamson amplitude (WAMP). In the present invention, a simple absolute absolute value (IAV) is used to design a recognizer for a small terminal.

The absolute integration value (IAV) is a feature extraction method for pattern classification that is robust to noise in continuously acquired electromyogram signals and uses the following equation (1).

The extracted feature value is processed and transmitted to the motion recognition unit 34. [

Next, the preprocessing unit 32 preprocesses the inputted arm motion state signal into the formatted data.

Methods of preprocessing signals of acceleration sensors include discretization, smoothing, resampling, and complex techniques. The double smoothing technique includes data with information among the input data and external environmental noise. In this case, it is a method of reducing noise by flattening the data with information and environmental noise. These techniques include moving average, weighted moving average, least square method, and envelope detection.

In the present invention, a simple moving average (SMA) is used because it is simple to recognize the motion and to be applied to a small terminal.

The moving average filter (SMA) is expressed by the following equation (2).

The data formatted through the moving average filter is input to a pitch angle / roll angle estimating unit 33.

The pitch angle / roll angle estimator 33 estimates the pitch angle and the roll angle using Equation (3) as follows based on the input formatted data.

∠Pitch

Next, the motion recognition unit 34 recognizes the motion of the hand with the feature value extracted by the feature extraction unit 31. [ In order to accurately determine the intention of the user, the data converted into [Equation 1] are successively acquired, threshold values preset for distinguishing the hand motion are compared, and if the extracted feature value exceeds the threshold value Only distinguish the motion of the hand.

The pitch angle and the roll angle estimated by the pitch angle / roll angle estimator 33 are used to determine the upper (upper), lower (lower), left, and right states of the arm. It is classified.

For example, the object of the interface to be implemented in the present invention is a cutter at the lower hand or a handicapped person with difficulty in finely manipulating the hand, and the interface type is a bracelet to recognize the motion of the arm and implement the keyboard and mouse functions of the smartphone and the pad It is.

To accomplish this, we constructed nine behaviors, which are comfortable for users and similar to those of ordinary people.

That is, as shown in FIG. 4A, for the same operation as the keyboard direction key, the front, back, left, and right tilting movements of the arm are constituted by double-clicking the mouse and entering ) Or ESP for the same operation. In addition, for the user to receive or hang up a telephone call using the smartphone, the operation for the call function as shown in FIG. 4C is configured by utilizing the tilt of the hand as shown in FIG. 4A and the operation using the double click as shown in FIG. 4B.

In addition, the motion recognizing unit 34 recognizes the motion of the hand using the acceleration and the EMG signal composed of two channels using the acceleration. .

Thereafter, a control target control signal according to the operation recognition result is generated and transmitted to the communication module 40.

The communication module 40 converts the transmitted control target signal into a radio signal and transmits it to the IT device to be controlled. IT devices such as smart phones and smart pads provide a variety of interfaces. As a product for using the interface provided by such IT devices, it is preferable to interwork with IT devices using a Bluetooth type HID (Human Interface Device) module such as a Bluetooth earphone or a Bluetooth keyboard. The Bluetooth HID module allows you to use PCs as well as smart devices without the need for a separate application. HID module and smart device's HID protocol. In case of keyboard input, 12Byte packet should be transmitted. In case of mouse input, 8Byte packet should be sent.

Therefore, disabled persons with upper limbs who are less able to use hands or hands with less hands can operate IT devices such as smart phones and pads.

The inventors performed two similar experiments for comparison with the gesture-based interface of a similar prior art throat.

In the first experiment, we analyzed the EMG signal, roll, and pitch data for the basic operation of the proposed nine operations to verify the performance of the module. As shown in FIG. 6A, by the mouse operation, the characteristic of the chin radiograph flexor was detected in the ESC operation, and the characteristic of the flexor flexor flexor was detected in the operation of Enter as in FIG. 6B. In Fig. 6c, the reaction characteristics were detected in both of the carpal flexors of the chest and the lateral side.

In Fig. 7 (a) and (b), the change of the roll axis angle was detected in the form of tilting back and forth, and in the cases of (c) and (d), the angle change of the pitch axis was detected in the tilting of the arm.

Therefore, it can be seen that the characteristic of the corresponding operation is detected with respect to the mouse operation and the keyboard operation as a result of the analysis of the detected graph.

In the second experiment, five male adults in their twenties were used for the verification of the actual user, and 10 minutes were used after explaining the nine motions, and the keyboard and mouse motions were presented at least ten times each. And the operation of disconnecting, the experiment assistant made a telephone call arbitrarily and tried to confirm whether or not it succeeded through a few attempts.

The results of the second experiment are shown in [Table 1].

action Total attempts Total Success Success rate (%) Up 50 49 98 Down 51 48 94.1 Left 54 52 96.3 Right 52 50 96.2 Enter 50 48 96 Double-Click 51 49 96.1 ESC 53 48 90.5 Get a call 12 10 83.3 Hang up 14 10 85.4

In the present invention, to improve the accessibility of the IT device to the upper handed person, the acceleration sensor and the EMG signal are used to recognize the operation of the arm, and an interface for using a smartphone or a smart pad is proposed. In order to perform the keyboard operation, the motion of the arm was configured using two axes of the acceleration sensor, and the movement of the hand was performed using the EMG signals of the lateral carpal and flexor carpi ulnar muscle Respectively. And the operation of receiving and disconnecting the call is composed of a mixture of keyboard operation and mouse operation.

Experimental results show that the average success rate of keyboard operation was 94.7%, the average operation success rate of mouse operation was 90.2%, the average success rate of call receiving and hanging operation was 84.5%, and the average success rate was 92.9%. The recognition success rate using the accelerometer was 96.1% and the EMG signal recognition rate was 90.3%.

Although the present invention has been described in detail with reference to the above embodiments, it is needless to say that the present invention is not limited to the above-described embodiments, and various modifications may be made without departing from the spirit of the present invention.

The present invention is applied to a technique for an IT device interface of a disabled person.

10: EMG signal measuring unit
20: Accelerometer
30: Processor
31: Feature extraction unit
32:
33: Pitch angle / roll angle estimating unit
34:
40: Communication module

Claims

An EMG signal measuring unit attached to an arm of a person with upper limb and measuring an EMG, which is an action potential generated when the muscle contracts;
An accelerometer for detecting an arm motion state signal of the upper limb disabled person;
A processor for recognizing an operation for controlling the control object based on the measured EMG and arm movement state signals and generating a control object control signal according to the recognized operation result; And
And a communication module for converting the control object control signal generated by the processor into an interface signal and transmitting the interface signal to a control object,
The processor includes a feature extraction unit for extracting a feature for pattern classification from the bio-signal output from the electromyogram signal measurement unit; A preprocessor for preprocessing the arm motion state signal output from the accelerometer by a moving average filter; A pitch angle / roll angle estimating unit for estimating a pitch angle and a roll angle in the data formatted through the preprocessing unit; A pitch angle / roll angle estimated by the pitch angle / roll angle estimating unit, and recognizes the muscle motion of the arm as the feature extraction value of the feature extracting unit, And an operation recognition unit for generating a control object control signal.

The electromyographic signal measuring unit according to claim 1, wherein the electromyogram signal measuring unit comprises: an electromyogram detecting electrode attached to an arm of a person with upper hand and detecting an action potential; A preamplifier for pre-amplifying the minute action potentials detected by the electromyogram detecting electrodes and outputting the resultant signals as a living body signal; A differential amplifier for amplifying the bio-signal amplified by the preamplifier according to a predetermined gain; A band-pass filter for filtering the biomedical signal output from the differential amplifier to a set band; A notch filter for removing noise contained in the bio-signal output from the band-pass filter; And an analog / digital converter for converting the analog EMG signal output from the notch filter into a digital EMG signal.

[3] The interface device of claim 2, wherein the electromyographic detection electrode is attached to at least two of the lateral and lateral carpal flexors of the arm of the upper limb.

delete

2. The interface device according to claim 1, wherein the feature extracting unit extracts a characteristic by continuously calculating an electromyogram signal with absolute integration values.

The method as claimed in claim 1, wherein the motion recognition unit recognizes a keyboard direction key operation by forward, backward, left, and right tilting of the arm, recognizes a click and double click operation of the mouse with the feature extraction value, And recognizes a call receiving and a disconnecting operation by a combination of the mouse operation recognition.