JP2659653B2 - Robot hand control device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a robot hand control device for controlling the operation of a robot hand based on myoelectric signals detected from skin surface electrodes.

[0002]

2. Description of the Related Art Conventionally, the idea of controlling a robot hand or an artificial hand as an artificial hand by an electromyographic signal detected from an electrode or the like provided on the skin surface has been proposed by N.I. Wine
r in 1948 (Reference 1. N. Win)
er, Cybernetics (1948)), and subsequently a number of devices have been developed in practice. In view of the myoelectric prosthesis controlled by this myoelectric signal, it is a method of controlling the operation of the prosthesis by the myoelectric potential of the stump of the arm, usually through a few pairs of skin surface electrodes from the stump, 1
After detecting a myoelectric potential of about 00 to 1000 microvolts and amplifying it with an amplifier, it performs electrical preprocessing such as rectification and smoothing, and recognizes and identifies the processed signal.
It controls the fingers of the prosthetic hand or the operation of the indirect joint, and is widely used worldwide.

[0003] Conventional myoelectric prostheses are broadly classified into two types for the purpose of operation control, regardless of the research level or the practical level.

[0004] The first is to control the forearm hand device as an object to be controlled from the detected myoelectric signal.
Usually, the function is limited to one degree of freedom, and it is intended to open and close the operation of the finger of the prosthetic hand (hand tool).

[0005] As the control method with one degree of freedom, two types of an on-off control method and a proportional control method are representative.

In the on / off control method, a difference signal is formed from two signals that are rectified and smoothed by removing an in-phase noise signal from myoelectric signals detected from two electrodes of a residual flexor and an extensor at a stump. Open and close the prosthesis and fingers (Reference 2. Edited by the Japanese Society of Orthopedic Surgery and the Japanese Society of Rehabilitation Medicine, Checkpoint for Prosthetics and Orthotics (1987), pp.
59-68).

In the proportional control method, the opening and closing speed of the artificial hand and the gripping force are adjusted by utilizing a neurophysiologically well-known fact that an approximated proportional relationship is established between the integrated myoelectric signal and muscle tone. How to do it. This proportional control method uses pulse width modulation.If the myoelectric signal is weak, the prosthetic hand opening / closing operation motor is turned on for a short time, and if the muscle contraction increases, the on time is long and the pulse width is large. That is, the rotational speed of the motor is further increased, and the opening / closing speed of the artificial hand is increased.

In both the on-off control method and the proportional control method, a threshold value is set for the rectified and smoothed myoelectric signal and switching is performed. In addition, since the wearer's physical condition and the level change of the myoelectric signal with the passage of time after the disconnection cannot be handled, the wearer may not be able to move as intended.

Normally, in the training process until the wearing of the myoelectric prosthesis, the wrist joint of the phantom limb is bent. For example, in the unilateral amputation, both the unaffected upper limb and the amputated limb are put forward, and the wrist joint is bent. Then, while allowing the person to be wearing the myoelectricity detector to observe, training is performed until the myoelectric signal suitable for the level recognized by the myoelectric identification circuit of the artificial hand is output. However, a person who intends to wear a myoelectric hand that cannot generate the myoelectric signals of the two channels of flexor and extensor muscles even after training cannot be separately subjected to myoelectric prosthesis mounting. In other words, this is even a matter of life and death for forearm amputees.

The second method is to emphasize how many degrees of freedom are simultaneously controlled based on the detected myoelectric signals. The second method aims at controlling the upper arm prosthesis for a high amputee such as a scapulothoracic amputee. Is what you do.

In one of the myoelectric identification methods for multi-degree-of-freedom control, a myoelectric signal detected from the multiple electrodes is smoothed and rectified, and a time-averaged signal is input to a perceptron at regular intervals. There is an example of learning by giving an action as a teacher signal (Reference 3 Ryoji Suzuki, Tatsumi Suematsu, Learning and Identification of Myoelectric Current Pattern Using Link-8, Medical Electronics and Biotechnology, No. 7)
Vol. 1, No. 1969, pp. 47-48). In this example, the skin surface electrodes were mounted immediately above three different muscles, enabling seven types of motion to be identified.
Due to the property of perceptrons that pattern separation that cannot be performed by linear separation cannot be performed, there has been a problem that operation discrimination cannot be practically performed when the mounting position of the electrode or the type of operation increases.

As another example of a myoelectric identification method for multi-degree-of-freedom control, a skin surface electrode is mounted immediately above four different muscles and rectified and smoothed as described above. Example of attempting to identify five types of shoulder motions using a first-order linear discriminant function for myoelectric signals (Reference 4, Kazuo Tani, et al., 7, Myoelectric Pattern Discrimination Learning for Prosthetic Hand Control, Biomechanism 5, The University of Tokyo Press (1980), pp88
However, if the number of learning myoelectric patterns increases, it takes time to converge the discriminant function, and linear separation cannot be performed in many cases. In addition, both the learning pattern and the recognition pattern
If there is noise, there is a problem that learning does not converge or the operation cannot be recognized.

Further, the two methods have a common problem that multi-channel identification requires mounting of multi-channel electrodes.

As a third conventional technique of the myoelectric identification method for multi-degree-of-freedom control, the fact that the myoelectric signals from the same site exhibit different frequency characteristics due to the operation makes use of the fact that the forearm's lateral hand is A band filter is applied to the one-channel myoelectric signal detected from the root flexor muscle, and four types of forearm motions are discriminated by a first-order and second-order linear discriminant function on the frequency spectrum of the ten-divided myoelectric signal. (Reference 5 Hisashi Sakakibara and three others, control of multifunctional forearm prosthesis using frequency information of myoelectric signal, biomechanism 4 (19
78), pp 131-138).

However, since this uses a linear discriminant function, when creating a myoelectric pattern for learning the discriminant function, the hand motion normally performed in a cooperative motion is decomposed into a simple motion. However, there is a problem that the subject must be trained to perform hand movements. However, in the present example, since the frequency characteristic of the myoelectric signal uses the characteristic that the muscle contraction force is weak and the change due to muscle fatigue is small, there is an advantage that the reproducibility in the motion identification is good.

As described above, both the one-degree-of-freedom control method for controlling the finger opening / closing operation and the multi-degree-of-freedom control method for controlling the upper arm or the forearm can be used for the conventional electromyography without learning ability. In the method operated by the identification method,
The wearer himself can generate a myoelectric signal suitable for the recognition and identification device, so that the body image that he has already formed, that is, the muscle of each part of the body according to his own intention, acquired by growth. The ability to unconsciously cooperate must be reconfigured.

In a conventional linear discriminant function or perceptron method in which a learning function is provided in the process of recognizing and identifying myoelectric signals and converting them into hand movement patterns,
As the number of motion patterns to be identified increases, the probability that the patterns cannot be linearly separated increases.In learning the EMG signals with known motion patterns, learning often does not converge, especially when power saving data contains noise. It was not practical because it did not converge forever (Reference 4 Kazuo Tani and 7 others, myoelectric pattern discrimination learning for prosthetic hand control, biomechanism 5, University of Tokyo Press (1980), pp. 88-95).

Further, even if the learning is converged, if noise such as an artifact of a myoelectric signal is mixed, there is a problem that the identification is impossible.

Further, if the number of operation patterns for recognition and identification increases, the number of electrodes to be mounted is too large to be practical.

In addition, it is not possible to separate and recognize and identify the movement of each finger other than the opening and closing of the finger movement by the conventional myoelectric signal identification.

As means for solving such a problem, F
A myoelectric control learning type robot hand that recognizes an FT processed myoelectric pattern by a neural network (back propagation method) having a non-linear separation capability of the pattern has been proposed (Hiraiwa et al., “Electromyographic control learning type”). Robot Hand ", Japanese Patent Application No. 1-114215. Recognition by the neural network is to perform binary discrimination of which finger is bent or not to be bent for each finger in a steady state, and to perform dynamic and continuous operation of each finger at a continuous finger bending angle. Was not something to do as recognition.

Therefore, the generation of a teacher signal pattern used for learning of the neural network is detected by detecting the user's finger motion continuously and in time series, and the dynamic and continuous motion of each finger is calculated as follows.
A learning-type electromyogram pattern recognition robot hand that recognizes continuous finger bending angles has been proposed (Hiraiwa et al., “
Learning type electromyogram pattern recognition robot hand ”, Heihei 3-
274167). At the stage where the recognition unit of the myoelectric control learning type robot hand learns the relationship between the user's myoelectricity and the finger bending angle, in the first embodiment described in the application, the user generates a teacher signal to the recognition unit. Glove-like finger bending angle detection sensor on the hand, and a glove-like sensor that is not normally required when operating the robot hand with myoelectric signals should be prepared for the learning stage of the recognition unit. There is a disadvantage that waste is large.

Further, in the glove-shaped sensor, since the thickness of the finger and the interval length between the joints differ for each user, an error cannot be avoided in the value of the finger bending angle detected as the teacher signal. Furthermore, if a glove-like sensor is used, for example, a data glove (trade name) that detects the deflection of the optical fiber attached to the glove from the amount of transmitted light of the optical fiber,
The system that detects the finger bending joint angle from the strain gauge attached to the glove has a force on the deflection of the fiber and the strain gauge.Therefore, when the user wears the glove, the robot uses myoelectric signals without wearing the glove. In the case of controlling the hand, there is a problem that an error in theoretically recognizing the finger bending joint angle cannot be avoided by the force required for the deflection.

In the second embodiment described in the above-mentioned application,
In order to generate a teacher signal to the recognition unit at the learning stage of the recognition unit, the user's finger image must be image-processed and the joint bending angle must be detected. was there.

[0025]

SUMMARY OF THE INVENTION The present invention has been made in consideration of the above problems, and has as its object to provide a robot hand control device capable of achieving the following objects.

That is, a neural network, which is a processing mechanism that is strong in learning and noise, is introduced into the recognition unit so that the prosthetic wearer does not need to reconstruct the body image of the prosthesis wearer or to perform myoelectric generation training. Other types of motions can be identified from myoelectric signals detected from a small number of skin surface electrodes, and there are few malfunctions against myoelectric signal artifacts due to body movement of the wearer or displacement of the skin surface electrodes. Is a robot hand control device that operates by recognizing and identifying myoelectric signals detected from skin surface electrodes.

More specifically, the dynamic and continuous operation of each finger is performed as recognition of a continuous finger bending angle from skin surface electromyogram signals. The finger bending pattern stored in a bending teacher signal pattern storage unit in time series in advance, and the stored finger bending pattern is visually presented to the user by a finger image or an actual image by computer graphics (hereinafter simply abbreviated as CG), By giving the same signal as the finger bending pattern presented to the user as a teacher signal at the same time to the recognition unit by the neural network, in the learning stage of the recognition unit,
An object of the present invention is to provide a learning-type electromyographic pattern recognition robot hand control device that does not require a glove-like finger bending angle detection sensor or a finger bending angle detection device by image processing.

[0028]

Means for Solving the Problems To achieve the above object, a first invention of the present application provides a skin surface electrode provided on a skin surface, and a frequency analysis for performing frequency analysis of a myoelectric signal detected from the skin surface electrode. Means, finger bending teacher pattern storage means for storing the finger bending teacher pattern in time series, display means for displaying and outputting the finger bending teacher pattern stored in the finger bending teacher pattern storage means, a plurality of units and weights. A neural network having a learning function by a linked link, and a myoelectric signal subjected to frequency analysis by the frequency analysis means, and a finger bending teacher pattern output from the finger bending teacher pattern storage means. As input, learning for measuring the coincidence of these inputs is performed, and based on the myoelectric signal frequency-analyzed by the frequency analyzing means based on the learning result. Recognition means for outputting drive data of the bot hand, robot hand control means for outputting an operation command signal based on the drive data output from the recognition means, and based on the operation command signal output from the robot hand control means The gist of the present invention is to have an operating robot hand. Further, the second invention of the present application includes a skin surface electrode provided on the skin surface, a frequency analysis means for performing frequency analysis of a myoelectric signal detected from the skin surface electrode, and a finger bending teacher pattern stored in time series. A finger bending teacher pattern storing means, a reproducing means for reproducing a video tape in which a finger bending teacher pattern and a synchronization audio timing signal are recorded in advance, a display means for displaying a teacher pattern image reproduced by the reproducing means, A microphone for detecting a synchronization audio timing signal reproduced as an audio signal by the reproduction means, a neural network having a learning function by a plurality of units and weighted links, and a myoelectric signal frequency-analyzed by the frequency analysis means; Output from the finger bending teacher pattern storage means in synchronization with the signal and the synchronization audio timing signal detected by the microphone. Using the bending teacher pattern as an input of the plurality of units, learning to measure the coincidence of these inputs is performed, and drive data of the robot hand is output based on the myoelectric signal frequency-analyzed by the frequency analyzing means based on the learning result. Recognition means to
The gist comprises a robot hand control means for outputting an operation command signal based on the drive data output from the recognition means, and a robot hand which operates based on the operation command signal output from the robot hand control means. .

[0029]

With the above-described configuration, the robot hand control device of the present invention enables a healthy person or a forearm amputee to transmit a myoelectric signal generated in the movement of a hand intended by himself / herself to the vicinity of a muscle involved in the movement. It is detected by the attached skin surface electrode, and the detected myoelectric signal is subjected to, for example, fast Fourier transform to analyze the frequency. The frequency-analyzed linear non-separable pattern is supervisedly learned a plurality of times by the neural network of the recognizing means, and the corresponding driving amount of the robot hand, for example, the bending angle of the finger is recognized, identified and operated.

In the process of learning the relationship between the finger pattern and the bending angle of the finger in the neural network, the teacher values used in the learning process of the neural network are stored in the teacher value storage means in a time-series manner. On the basis of the stored finger bending teacher pattern as a teacher value, the finger image or actual image by, for example, CG, and furthermore, the finger bending teacher pattern is visually presented on a video tape recorded in advance, and is recognized by the neural network. Since the same signal as the finger bending teacher pattern visually presented to the user is given to the means as a teacher signal in synchronization, the glove-like finger bending angle detection sensor or the finger bending angle by image processing is used in the learning step of the recognition means. No detection device is required. Therefore, since a glove-like finger bending angle detection sensor is unnecessary, it is possible to provide a prosthetic hand by the myoelectric pattern learning of the present invention even for a two-hand amputee.

At this time, the robot hand that operates based on the operation command signal may be an actual mechanical robot hand or a virtual robot hand using computer graphics.

[0032]

An embodiment according to the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing a configuration of a robot hand control device according to the present invention. First, the present embodiment will be described by roughly dividing into a learning mode (A) of the recognition unit and a recognition mode (B) of recognizing an intended finger bending angle from the generated electromyogram.

First, the learning mode (A) of the recognition unit will be described. In the present embodiment, an example will be described in which the skin surface electromyogram of two channels is detected, and the bending angles of the first and second joints of the fingers of one hand, that is, the finger bending angles of a total of 10 joints are recognized.

First, the skin surface electrodes 1, 3 are mounted near the total finger extensor of the wrist, and the skin surface electrodes 5, 7 are mounted near the superficial flexor of the wrist. After the electrodes are mounted, when the power of the robot hand control device is turned on, the finger bending teacher pattern stored in the finger bending teacher pattern storage unit 37 is transferred to the display monitor 59 via the finger bending teacher value presentation interface unit 31. Presented as a computer graphics image.

FIG. 12 is an example of a finger bending teacher pattern presented on the display monitor 59 by computer graphics. The finger by the computer graphics is shown in an oblique projection view of the finger, and at the same time, the finger bending joint angle is displayed as a bar graph on the lower side of the screen. At this time, if the user is a healthy person, he or she actually moves his / her finger by mimicking the finger bending operation of the image while watching the finger bending operation presented on the display monitor 59. Then, the movement of the finger is imaged in the head, or the healthy arm on the opposite side to which the skin surface electrode is attached and the hand on the cut side which does not actually exist are moved symmetrically on the image. During this operation, FIG.
The signals a ₁ and a ₂ detected by the skin surface electrodes 1, 3, 5, and 7 are amplified by the amplifiers 9 and 11, and become a signal c through the low-cut filters 13 and 15 as a time-series myoelectric signal b. Furthermore, the high cut filter 17,
A signal d is obtained through the signal 19.

The time-series myoelectric signal d is taken into the FFTs 21 and 23 in the state where each finger is moving, and is converted into a pattern e obtained by analyzing the signal d detected in each state by ３ octave frequency. You.

FIG. 9 is an example of a pattern e at a certain moment during the movement of the hand. The signal component level of the pattern e is converted into a numerical value matrix f for each bandwidth by the matrix pattern conversion means. FIG. 9 shows an example, and one horizontal row next to input in the above figure corresponds to one operation pattern.

The numeric matrix pattern f
Is input to At the same time, an operation corresponding to the pattern of f is given to the recognition unit 29 as teacher signal data g.

Here, the correspondence between the numerical matrix pattern f input to the recognition unit 29 and the timing of the teacher signal data g will be described with reference to FIGS. Here, a frame length of 50 with respect to the time-series myoelectric signal is used.
0 msec, FFT with a frame cycle of 250 msec,
On the other hand, an example will be described in which sampling of the finger bending angle teacher value is performed at 32 points per second. The time-series myoelectric signal is filtered at a high cut frequency of 3000 Hz and a low cut frequency of 50 Hz, and the FFT performs 1/3 octave analysis on 10 bands (center frequencies 63, 80, 100, 125, 160, 2).
(00, 250, 315, 400, 500 Hz).

First, as shown in FIG. 10, under the above-described conditions, the recognizing section 29 recognizes four patterns per second. Time-series EMG signals and teacher signals are
As shown in FIG. 10, both observation windows are opened so as to coincide with each other, and each is provided to the recognition unit 29. The time-series myoelectric signals obtained by opening the observation window of 500 msec as shown in FIG. 10 and taking in the FFTs 21 and 23 have the FFT patterns as shown in FIGS.

Further, this is converted by the matrix pattern conversion unit 27 into a numerical pattern in a horizontal row next to the input in FIG. In the case of the myoelectric EMG (1), the finger bending angle teacher value corresponding to the time is Teach in FIG.
er (1) corresponds. In FIG. 10, EMG (2), EM
In the case of G (3), each Teacher
(2) and Teacher (3) correspond. At this time,
The finger bending angle teacher value of each finger and each joint is converted into a finger image by CG in the finger bending angle teacher value presentation interface unit.

Since the sampling rate of the finger bending angle teacher value is different from the sampling rate of the recognition unit 29, the finger bending angle teacher value is time-averaged so that the two sampling rates match. Now, the processing is shown in FIG.
Shown in

As shown in FIG. 11, a time window of the same length as the time-series myoelectric signal to be subjected to the FFT processing is opened for the finger bending angle teaching value sampled at 32 points per second, and 16 point signals are taken. , 1/16 and time averaged. The processing of the finger bending angle teacher value is performed by the finger bending angle teacher value presentation interface unit 31 using the synchronization signals j, k,
The processing is performed while synchronizing the processing of the myoelectric signal based on l and m as shown in FIG. As shown in FIG. 10, in FIG.
The finger bending teacher value at the time of t (1) in FIG. 5 is given to the recognition unit 29 of t (1), and the corresponding value at the time of t (2) is given to t (2). Shall be given. FIG. 9 shows g obtained by converting the finger bending angle teacher value by the finger bending angle teacher value presentation interface unit 31 and the corresponding input / output pattern for learning of the recognition unit 29 using the FFT-valued numerical matrix EMG pattern f. . Now, the Teache of FIG.
The next row and column of r is the converted finger bending angle teaching value pattern. In this example, the conversion is from the finger bending angle of 0 degree to 1
Twenty degrees was converted proportionally from the value -0.5 to +0.5.

The learning of the neural network by the recognition unit 29 is performed as shown in FIG.
This is performed as shown in the flowchart of FIG.

First, in step 110, the weight between each unit of the neural network of the recognition unit 7 and the offset of each unit are initialized with random numbers. Next, in step 120, the user repeats for several minutes a hand motion imitating the CG image as the finger bending teacher pattern presented on the screen of the display monitor 59, and in step 130, the processed myoelectric signal and the finger bending The input / output pattern pair of the angle teacher value is taken into the input / output pattern storage unit 39 of the recognizing unit 29, and after the end of the hand operation in step 140, in step 150,
A pair of input / output patterns is extracted from the input / output pattern storage unit 39, and learning of the neural network is performed by, for example, a back propagation method (literature, supervised by Kaoru Nakano, "Neurocomputer", Technical Review, p47, 1989). In step 160, the weight of each unit of the neural network is updated, and the offset of each unit is updated. More specifically, learning is performed by sequentially giving the input / output pattern pairs shown in FIG. 9 from top to bottom to the neural network, and when the last input / output pair comes, learning is repeated again from the first pair. (Step 170).

When the number of times of learning exceeds the designated number of times, the learning is terminated (steps 190 and 200). Above,
The learning mode (A) ends.

Next, the recognition mode (B) of the neural network will be described. The recognition mode (B) is a mode in which a skin surface electrode is mounted and a robot hand or a virtual CG hand is controlled.

When the user is a healthy person, the hand on the electrode mounting side is moved, and when the user is an amputee, the hand is moved in the head by the phantom limb.

This operation or the time-series myoelectric signal detected in the image is discretized and frequency-analyzed for each FFT time window opened on the time axis under the same conditions as in the learning mode, and the matrix pattern is analyzed. Pattern e
Is entered as

The pattern e is converted as a pattern f by the matrix pattern converter 27 in the same manner as in the learning mode (A).

The pattern f is based on the pattern already learned in the learning mode (A), and the robot hand controller 3 sets the corresponding finger bending angle value as the finger bending angle data h.
Output to 3.

The robot hand controller 33 outputs an operation command signal i to the robot hand 35,
5 reproduces the bending operation of the finger based on the operation command signal i.

FIG. 15 shows the electromyogram of the finger bending angle of the first finger of the forefinger when the two-channel electromyogram pattern of the finger bending motion of the unlearned hand is recognized by the learned recognition unit 29. Finger bending recognition value by pattern and at the same time glove-shaped finger bending joint angle detection device (product name: Data Glove)
1 shows an example of implementation of recognition in which true values of finger bending angles measured in a time series are shown in time series. The finger bending angle recognized from the myoelectric signal according to the present invention and the true finger bending angle are in good agreement, indicating the effectiveness of the present invention.

FIG. 11 shows the electromyogram pattern of the finger bending angle of the first finger of the index finger when the two-channel electromyogram pattern of the finger bending motion of the unlearned hand is recognized by the learned recognition unit. 1 shows an example of recognition execution showing a correlation between a finger bending recognition value according to the present invention and a true value of a finger bending angle measured by a data glove at the same time. The correlation between the two is as shown in the figure, indicating the effectiveness in this embodiment.

FIG. 17 shows the finger bending angles of the first and second joints of each finger when a two-channel myoelectric pattern of a finger bending operation of an unlearned hand is recognized by the learned recognition unit. Of the finger bending recognition value based on the myoelectric pattern and the true value of the finger bending angle measured by the data glove at the same time
5 shows an ot Mean Square Error, and shows the effectiveness in the present embodiment.

The neural network is composed of 20 units of the input layer, 20 units of the intermediate layer, and 10 units of the output layer.
Unit ignites. FIG. 3 shows the structure of the nerve cell unit of the neural network and the input / output function characteristics of the unit. The output of the output layer corresponds to the finger bending angles of the first and second joints of each finger, and each of the five fingers has two joints for a total of 10 joints.

In the present embodiment, an example is shown in which the myoelectric signal to be detected is two channels and is applied to a finger, but the following modifications are possible.

That is, the number of pairs of skin surface electrodes is increased, the signal from each electrode is subjected to FFT processing, converted into a level numerical pattern for each band, and the number of units in the input layer of the neural network is increased. Inputting patterns from not only two pairs of two-channel electrodes but also several pairs of electrodes at the same time, various sizes and shapes of electrodes, F
Changing the bandwidth and the number of bands for FT processing, FFT
Instead of using a multiband bandpass filter, applying it to the motion of other parts of the human body, and the ability to separate non-separable linear patterns other than back-propagation neural networks by pattern attributes. An object of the present invention is to provide a device in which a neural network having a learning mode (A) is omitted by fixing a weight and an offset of a neural network of a recognition unit suitable for an unspecified number of users in advance and using a neural network. .

Further, the above can be similarly carried out independently of each other, and is not limited to the above embodiment, and various changes can be made without departing from the gist of the invention.

Next, a second embodiment according to the present invention will be described. The second embodiment differs from the first embodiment in that the finger bending angle teacher value presenting interface unit 31 instead of displaying the finger bending teacher pattern image by CG as shown in FIG. By reproducing the video tape 55 pre-recorded with the finger bending teacher pattern and the synchronization audio timing signal on the VCR 57, the synchronized finger bending angle teacher value is displayed while the finger bending teacher pattern is presented to the user. This is a part given to the recognition unit 29 in the teacher value presentation interface unit 31.

In this embodiment, as shown in FIG. 19, in order to synchronize the signals of the finger bending teacher pattern presented on the television monitor 59 and the finger bending angle teacher pattern presented on the recognition unit, the finger bending is performed. In the audio track of the teacher pattern recording film 55, a pure tone of a certain frequency is recorded at the finger bending teacher pattern image reproduction start point.
When the film is played, the pure tone is played from the speaker of the television monitor 59 and detected by the microphone 53 of the apparatus. The sound detected by the microphone 53 is detected by the band-pass filter 51 that passes only the frequency of the pure sound, and the detection signal is full-wave rectified and smoothed, and is sent to the signal control unit 25 as a trigger signal.

The transmission start timing of the signal stored in the finger bending teacher pattern storage unit 37 and sent to the finger bending teacher value presentation interface unit 31 is controlled by the trigger signal sent to the signal control unit 25, and the television monitor 59
Is synchronized with the finger bending operation pattern presented as the teacher value to the recognition unit 29.

In the learning of the recognition unit 29, the user moves his / her finger in accordance with the CG of the finger bending operation pattern presented on the television monitor 59 or the actual finger image, and after the learning is completed, the television monitor 59, the VTR 57 and the video film. 5
5, the robot hand 35
Alternatively, it controls a virtual CG hand.

As described above, in this embodiment, the neural circuit having the ability to identify the linear non-separable pattern obtained by performing the FFT processing on the myoelectric signal from the skin surface electrode by the attribute of the pattern and the learning function. The network provides "supervised learning" for each type of hand movement. Next, after this "supervised learning", the neural network recognizes the myoelectricity from the robot hand operator wearing the skin surface electrode and controls the robot hand.

Therefore, due to the automatic extraction of the pattern features of the neural network, the high discrimination of similar patterns, and the high tolerance, when the electrode is not located directly above the muscle directly involved in the hand movement in the learning mode. However, learning converges even when there are many learning patterns, and even when noise is mixed in the learning patterns. Similarly, in the recognition mode, if electrode misalignment occurs, noise or artifacts are mixed in the myoelectric signal, or when several types of muscle groups move in coordination, Even when the electrodes are not located directly above the muscles directly involved in the movement of the skin, the myoelectric potentials generated from several muscle groups simultaneously simultaneously nonlinearly interfere, mix, and crosstalk to reach the skin surface Since the finger bending angle of the corresponding hand can be recognized from the surface EMG,
It becomes possible to control the finger bending angle of the robot hand by discriminating the finger bending angle from the skin surface myoelectricity, which was conventionally impossible.

Further, in the conventional method, a glove-like finger bending joint angle sensor and a finger bending angle detecting device by image processing are required for learning of the recognition unit, but are unnecessary.

Further, with respect to a change in physical condition or a change in the generated myoelectricity due to a change in age, the use of the robot hand of the present invention is interrupted before or during use, and the learning is interrupted. By performing the mode, the device can be used in a state more suitable for the generated myoelectricity.

On the other hand, as an application field, there is a use in a welfare field as a power prosthesis such as a myoelectric prosthesis for a forearm amputee, a prosthesis for another amputee, and a prosthesis. In addition, control of the robot hand of a remote control robot for extreme work, such as work in outer space, inside the containment vessel of a nuclear power plant, deep sea, fire scene, culture vessel of toxic microorganisms, using a healthy person as the pilot, remote surgery Operation by the doctor of the robot hand at the time,
Operation of micro robot hand by doctor at the operation of micro organs such as blood vessels and eyeballs, manned operation of robot hand for handling micro devices such as LSI devices or unmanned telephone station, unmanned radio wave relay facility, tunnel, unmanned substation Manned operation of a robotic hand of a remote-controlled robot that performs maintenance management of facilities that require maintenance management, etc., furthermore, an input device to an aircraft pilot's control device, application to manipulation in a virtual space created by a computer CG, computer An input device that substitutes for a keyboard or mouse, an input device that acts as a substitute for the keyboard of a piano or musical instrument, an input device for machine translation of sign language, and a machine that recognizes characters written by pen input when pen input is performed by character recognition. Character recognition rate by simultaneously recognizing myoelectric patterns Auxiliary enhance device, etc. are conceivable.

[0070]

As described above, the robot hand control device of the present invention provides a neural network which has learning properties and is strong in noise in recognizing and identifying myoelectric signals detected from skin surface electrodes. Since it is used, an effect such as operation is obtained even with a myoelectric signal detected from a small number of skin surface electrodes.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of one embodiment according to a robot hand control device of the present invention.

FIG. 2 is a block diagram illustrating a configuration example of a recognition unit illustrated in FIG. 1 using a neural network.

FIG. 3 is a diagram showing an input / output relationship of a neural cell unit of a recognition unit and a structure of a neural network unit.

FIG. 4 is an FFT of an EMG signal in a state where all five fingers are bent.
It is a figure showing a myoelectric pattern obtained by processing.

FIG. 5 shows an FFT of an EMG signal in a state where only the index finger is bent.
It is a figure showing a myoelectric pattern obtained by processing.

FIG. 6 is a diagram showing a myoelectric pattern obtained by performing FFT processing on the myoelectric signal in a state where only the middle finger is bent.

FIG. 7 is a diagram showing an EMG pattern obtained by performing FFT processing on an EMG signal in a state where only the thumb is bent.

FIG. 8 is a diagram showing a myoelectric pattern obtained by performing FFT processing on myoelectric signals in a state where all fingers are opened.

FIG. 9 is a diagram illustrating an input pattern, which is a signal component level for each bandwidth of a myoelectric signal subjected to frequency analysis, and a learning pattern of a recognition unit including a finger bending angle teacher pattern;

FIG. 10 is a diagram illustrating a timing chart of an input / output pattern in a learning mode of the recognition unit.

FIG. 11 is a diagram illustrating a relationship between sampling of a finger bending angle teacher value and a processing procedure of time averaging in a learning mode of a recognition unit.

FIG. 12 is a diagram showing a CG image of a finger as a finger bending teacher pattern presented on the screen.

FIG. 13 is a diagram illustrating a flowchart in a learning mode of the recognition unit.

FIG. 14 is a diagram illustrating a flowchart of a recognition unit in a recognition mode.

FIG. 15 shows a finger bending recognition value based on the myoelectric pattern of the finger bending angle of the first finger of the index finger when a two-channel electromyographic pattern of a finger bending motion of an unlearned hand is recognized by a learned recognition unit. FIG. 7 is a diagram showing recognition results in which true values of finger bending angles measured by a data glove are shown in time series.

FIG. 16 shows a finger bending recognition value based on a myoelectric pattern of the finger bending angle of the first finger of the index finger when a two-channel myoelectric pattern of a finger bending operation of an unlearned hand is recognized by a learned recognition unit. FIG. 9 is a diagram illustrating a recognition result indicating a correlation between the data and a true value of a finger bending angle measured by a data glove.

FIG. 17 is a graph showing muscle bending of the finger bending angles of the first and second joints of each finger when a two-channel myoelectric pattern of a finger bending motion of an unlearned hand is recognized by a learned recognition unit; Root Mean Squ of the finger bending recognition value by the electric pattern and the true value of the finger bending angle measured by the data glove
It is a figure which shows are Error.

FIG. 18 shows a second example of the robot hand control device according to the present invention.
FIG. 3 is a block diagram showing a schematic configuration of the embodiment.

FIG. 19 is a diagram showing a process of processing a synchronization signal tone in the second embodiment shown in FIG. 18;

[Explanation of symbols]

 Reference Signs List 1 skin surface electrode 3 skin surface electrode 5 skin surface electrode 7 skin surface electrode 9 amplifier 11 amplifier 13 low cut filter 15 low cut filter 17 high cut filter 19 high cut filter 21 fast Fourier transform section 23 fast Fourier transform section 25 signal control section 27 matrix pattern transform Unit 29 recognition unit 31 finger bending angle teacher value presentation interface unit 33 robot hand control unit 35 robot hand 37 finger bending teacher pattern storage unit 39 input / output pattern storage unit 41 recognition input pattern 43 output pattern 45 learning teacher signal pattern 47 neural circuit Input layer of network 49 Intermediate layer of neural network 51 Output layer of neural network 59 Display monitor a1 Detection signal of skin surface electrode a2 Detection signal of skin surface electrode b Myoelectric signal c Myoelectric signal (low frequency component) E) Myoelectric signal (after FFT conversion) f Numerical matrix pattern of myoelectric signal g Finger bending angle teacher signal pattern h Finger bending angle operation data i Robot hand finger bending angle operation Command signal j Timing signal k Timing signal m Timing signal n Timing signal p Finger bending teacher pattern signal q Finger bending teacher pattern CG image signal

Claims (2)

(57) [Claims] 1. A skin surface electrode provided on a skin surface, a frequency analysis means for performing frequency analysis of a myoelectric signal detected from the skin surface electrode, and a finger bending teacher for storing a finger bending teacher pattern in time series A pattern storage means, a display means for displaying and outputting the finger bending teacher pattern stored in the finger bending teacher pattern storage means, a neural network having a learning function by a plurality of units and weighted links, The myoelectric signal subjected to the frequency analysis by the means and the finger bending teacher pattern output from the finger bending teacher pattern storage means are input to the plurality of units, and learning is performed to measure the coincidence of these inputs. Recognition means for outputting drive data of the robot hand based on the myoelectric signal frequency-analyzed by the frequency analysis means based on A robot hand control means for outputting an operation command signal based on drive data output from the recognition means; and a robot hand operating based on the operation command signal output from the robot hand control means. Hand control device. 2. A skin surface electrode provided on a skin surface, frequency analysis means for performing frequency analysis of a myoelectric signal detected from the skin surface electrode, and a finger bending teacher for storing a finger bending teacher pattern in time series. Pattern storage means; reproducing means for reproducing a videotape in which a finger bending teacher pattern and a synchronization audio timing signal are recorded in advance; display means for displaying a teacher pattern image reproduced by the reproducing means; A microphone for detecting a synchronization audio timing signal reproduced as a signal; a neural network having a learning function by a plurality of units and weighted links; a myoelectric signal frequency-analyzed by the frequency analysis means; Finger bending teaching output from the finger bending teaching pattern storage means in synchronization with the synchronization audio timing signal detected in step (b). Using the master pattern as an input of the plurality of units, learning to measure the coincidence of these inputs is performed, and driving data of the robot hand is output based on the myoelectric signal frequency-analyzed by the frequency analysis unit based on the learning result. Recognition means; robot hand control means for outputting an operation command signal based on drive data output from the recognition means; and a robot hand operating based on the operation command signal output from the robot hand control means. A robot hand control device characterized by the above-mentioned.