JP2010125287A - Digital joint angle estimating device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a digital joint angle estimating device which can respond to the time change of the relation between biological signals and the digital joint angles and can estimate several digital joint angles continuously and in real time. <P>SOLUTION: The digital joint angle estimating device 10 receives multiple surface myoelectric signals when digital joints move, extracts feature values of the surface myoelectric signals by an integral treatment section 14 and one-dimensionally aligns the feature values which are normalized by a normalization treatment section 15 into a feature value vector. A neural network 16 receives feed forward components which are one-dimensionally aligned feature vectors in chronological order from past to present and input vectors which comprise feedback components of output vectors in chronological order which are estimated until now for each of operation periods and estimates the digital joint angles. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a finger joint angle estimation device.

Conventionally, a method for identifying a finger shape using a surface myoelectric signal in a human robot system has been proposed (Non-Patent Document 1). Non-Patent Document 2 proposes a method in which the relationship between the surface myoelectric signal and the finger joint angle is modeled by a model formula, function approximation is performed, and parameter estimation is performed from learning data by the least square method. Patent Document 1 proposes a method for estimating the angle of a plurality of fingers by associating a surface myoelectric signal with a finger joint angle on a one-to-one basis.
JP-A-5-111881 Osamu Fukuda, Satoshi Tsuji, Toshio Tsuji, “Control of Electric Prosthetic Hand with Raw EMG Signal”, Proceedings of Society of Instrument and Control Engineers, Vol.40, No.11, pp. 1124-1131, 2004. Toru Kitamura, Nobuyoshi Kajiuchi, Takayuki Koizumi, “Manipulation of Robot Hands Based on Motion Estimation Using Myoelectric Signals”, Transactions of the Japan Society of Mechanical Engineers (C), Vol.73, No.735, pp.152-158, 2007

  In Non-Patent Document 1, discrete pattern identification is performed for the joint angle, and the finger joint angle is not estimated. Non-Patent Document 2 performs a single angle estimation at a specific finger joint, and does not specifically propose the possibility of application to a large number of finger joints. In Patent Document 1, only a one-to-one correspondence between a surface myoelectric signal and a finger joint angle is performed, and the possibility of correspondence to time conversion is specific in the relationship between the surface myoelectric signal and the finger joint angle. Not proposed.

  Conventionally, it is possible to cope with a temporal change in the relationship between a biological signal such as a surface myoelectric signal and a finger joint angle, and the finger joint angle is estimated for a plurality of finger joints, for example, five finger joints. None of the angle estimation devices have been proposed.

  An object of the present invention is to provide a finger joint angle estimation device that can cope with a temporal change in the relationship between a biological signal and a finger joint angle and can continuously estimate a plurality of finger joint angles in real time. is there.

  In order to solve the above-described problems, the invention of claim 1 inputs one or more types of biological signals when each finger joint is moving, extracts the feature amount of each biological signal, and extracts the feature A feature amount extraction unit that arranges the amounts in a one-dimensional manner to obtain a feature amount vector, and estimates an angle of each finger joint (hereinafter referred to as a finger joint angle) based on an input vector for each calculation cycle. A neural network means for outputting a joint angle as an output vector, and the input vector is a feedforward component in which time-series feature quantity vectors from the past to the present extracted by the feature quantity extraction means are arranged one-dimensionally; Output feedback is arranged in one dimension, and the feedback component is a time system that the neural network means has estimated up to the present time for each calculation cycle. It is intended to be subject matter of the finger joint angle estimating apparatus according to claim which is composed of the output vector.

In this specification, the biological signal is a signal of physiological information that occurs during physical activity, and refers to a signal that can be sensed by a detection means such as a sensor.
According to a second aspect of the present invention, in the first aspect, the biological signal is a surface myoelectric signal.

  According to a third aspect of the present invention, in the second aspect, the feature amount extraction unit integrates the surface myoelectric signal by shifting an integration interval for each calculation period, and calculates an integral value obtained by the integration as a feature amount. Is extracted as

According to a fourth aspect of the present invention, in the second aspect, the feature amount extraction means extracts a section sum for each predetermined bandwidth of the power spectrum of the surface myoelectric signal as a feature amount.
According to a fifth aspect of the present invention, in the second aspect, the feature amount extraction unit extracts an average frequency or a center frequency of a power spectrum of the surface myoelectric signal as a feature amount.

  According to a sixth aspect of the present invention, in the first aspect, the biological signal is an electroencephalogram, and the feature amount extraction unit is an event-related synchronization based on an increase / decrease change in at least one of the alpha wave and the beta wave of the electroencephalogram. And event-related desynchronization are extracted as feature quantities.

  According to a seventh aspect of the present invention, in the first aspect, the biological signal is an MRI signal of a brain region, and the feature amount extraction unit is activated from a change in reduced hemoglobin in a blood flow in a region that controls exercise. A place is extracted as a feature amount.

  The invention according to claim 8 is the signal according to claim 11, wherein the biological signal is a signal obtained by detecting near infrared light transmitted through a muscle related to a head or a finger joint. Or the change of the hemoglobin oxygenation state in the blood flowing through the muscle tissue related to the finger joint is extracted as a feature amount.

The ninth aspect of the present invention is the method according to the first aspect, wherein the biological signal is a magnetoencephalogram, and the feature amount extraction means extracts a change in the magnetic field of the magnetoencephalogram as a feature amount.
The tenth aspect of the present invention is the biometric signal according to the first aspect, wherein the biological signal is a signal that captures movement of a tendon due to movement of a finger joint, and the feature amount extraction unit is configured to detect changes in movement of the tendon Is extracted as

  The invention according to claim 11 is the signal according to claim 1, wherein the biological signal is a signal that captures a change in muscle bulge due to movement of a finger joint, and the feature amount extraction unit is configured to detect a change in the muscle bulge. This is extracted as a feature quantity.

  The invention of claim 12 is the signal according to claim 1, wherein the biological signal is a signal for detecting a nerve signal transmitted from the brain to the muscle cell of the finger by movement of a finger joint. A change in a nerve signal that changes in accordance with the movement of a finger joint is extracted as a feature amount.

  According to the first to twelfth aspects of the present invention, a finger joint angle that can cope with a temporal change in the relationship between the biological signal and the finger joint angle and can continuously estimate a plurality of finger joint angles in real time. An estimation device can be provided.

  Hereinafter, an embodiment in which the finger joint angle estimation device of the present invention is embodied as a finger joint angle estimation device will be described with reference to FIGS. FIG. 1 is a schematic block diagram of a finger joint angle estimation device.

  The finger joint angle estimation device 10 includes a computer and includes an A / D 11, a rectification processing unit 12, a smoothing processing unit 13, an integration processing unit 14, a normalization processing unit 15, a neural network 16, and a conversion processing unit 19. In FIG. 1, the rectification processing unit 12, the smoothing processing unit 13, the integration processing unit 14, the normalization processing unit 15, the neural network 16, and the conversion processing unit 17 are functional blocks of a central processing unit provided in the computer. It does not indicate a hardware configuration.

  The finger joint angle estimation device 10 inputs surface myoelectric signals through a plurality of electrodes 20 and A / D 11 attached to a human forearm Z. A signal input from each electrode 20 is referred to as a channel signal. The A / D 11 converts the surface myoelectric signal (analog signal) acquired at a common sampling frequency into a digital signal for each channel.

  The rectification processing unit 12 full-wave rectifies the digital signal from the A / D 11 for each channel. The smoothing processing unit 13 is composed of a low-pass filter, and smoothes the signal subjected to full-wave rectification by the rectification processing unit 12 by cutting off a predetermined frequency (for example, 50 [Hz]) or more for each channel.

  The integration processing unit 14 integrates the amplitude of the integration interval with respect to each channel signal input from the smoothing processing unit 13 and shifts the integration interval for each calculation period to obtain an integration value (FIG. 9B). reference). This integral value corresponds to the feature amount.

  The normalization processing unit 15 normalizes each channel signal integrated by the integration processing unit 14. In the present embodiment, the normalization processing is to make the amplitude of each channel signal in the range of 0 to 1, but is not limited to this range. Here, the rectification processing unit 12, the smoothing processing unit 13, the integration processing unit 14, and the normalization processing unit 15 correspond to feature quantity extraction means.

  The neural network 16 is composed of a three-layer perceptron neural network using a sigmoid function as an element transfer function. As shown in FIG. 2, the three-layer perceptron neural network includes an input layer 16a, an intermediate layer 16b, and an output layer 16c. Learning of the network in the neural network 16 is performed by a general back propagation method so that each finger joint angle of five fingers can be estimated. As the feedback component B described later, a past teacher signal is used at the time of learning, and a value actually output by the network in the past is used at the time of real-time finger joint angle estimation.

The units constituting the input layer 16a of the neural network 16 are prepared by the number for inputting the input vector I _k (see FIG. 5) composed of the feed forward component A and the feedback component B. Also, the unit forming the output layer 16c outputs the finger joint angle of each finger of the five fingers, which are estimated as the output vector O _k obtained by arranging one-dimensionally. Incidentally, in the output vector O _k, the finger joint angle of each finger is normalized to range (0, 1). When the unit of the output layer 16c has five fingers, in this embodiment, for example, since one finger has three joints, the bending angle (finger joint angle) of these three joints, and the inside and outside In order to detect the rolling joint angle (finger joint angle), twenty pieces are prepared so as to obtain an estimated value of each joint angle for each finger. However, the present invention is not limited to this value.

Figure 2, as shown in FIG. 3, the feedforward component ^A, A 0, ......, consisting ^{A n.} A ⁰ is composed of a feature quantity vector in which integrated values (feature quantities) F _k obtained by normalizing the current (that is, latest) channel signals, which are input from the normalization processing unit 15, are arranged one-dimensionally.

^N of A ⁿ means a calculation cycle before n (n is a positive integer) step from the present. Therefore, ^An is a feature quantity vector in which integration values (feature quantities) obtained by normalizing each channel signal and arranged in one dimension are input from the normalization processing unit 15 before n steps. More specifically, as shown in FIG. 3A, the feedforward component A at the current time t is characterized by the normalization processing unit 15 at time t + 1 (see FIG. 3B) of the next calculation cycle. The quantity vector is input, A ⁰ is updated, and the content of A ⁰ at the current time t is updated as the content of A ¹ .

Thereafter, A ^{2 to An} are updated in the same manner. The contents of A ⁿ at time t of the operation cycle, the time time t + 1 of the calculation cycle, is discarded. Hereinafter, as shown in FIG. 3 (c), at the next time t + 2 or later ^likewise, the contents of A 0 to A ⁿ are updated. As described above, the feedforward component A is a one-dimensional arrangement of normalized time-series feature vectors from the past to the present.

As shown in FIGS. 2 and 4, the feedback component B is composed of B ¹ ,..., B ^h . B ¹ represents fed back input from the output layer 16c, one step before the current calculation cycle, consisting of each joint angle of each finger from the output vector O _k obtained by arranging one-dimensionally. B ^h in B ^h means a calculation cycle before the current h (h is a positive integer) step. Thus, B ^h is fed back from the output layer 16c, the front h steps of the current calculation cycle, consisting of each joint angle of each finger from the output vector O _k obtained by arranging one-dimensionally.

More specifically, as shown in FIG. 4 (a), the feedback component B of the current time t, at time t + 1 of the next operation cycle (see FIG. 4 (b)), the output from the output layer 16c vector O _k Is fed back, B ¹ is updated, and the content of B ¹ at the current time t is updated as the content of B ² . Thereafter, B ^{3 to} B ^h are updated in the same manner. The content of B ^h of the time t of the operation cycle, the time time t + 1 of the calculation cycle, is discarded. Thereafter, as shown in FIG. 4C, the contents of B ^{1 to} B ^h are similarly updated after the next time t + 2. As described above, the feedback component B becomes a time-series output vector estimated to date.

Conversion processing unit 17 receives the output vector O _k of the neural network 16 is estimated, and treated denormalization, converted to each finger joint angle N of each finger. Expression (1) is an expression in the case of performing denormalization processing.

Here, MIN is the minimum value of the finger joint angle, and MAX is the maximum value of the finger joint angle.
The finger joint angle estimation device 10 outputs the result to the output device 30 after performing inverse normalization processing and converting into an angle according to the equation (1). The output device 30 is configured by, for example, a display display or a printer.

(Operation of the embodiment)
Now, the operation of the finger joint angle estimation device 10 configured as described above will be described with reference to FIGS.

  A surface myoelectric signal input as a biological signal to the finger joint angle estimation device 10 via the electrode 20 is acquired at a predetermined sampling frequency. The sampling frequency is, for example, 1 [kHz], but is not limited to this value. FIG. 6 is an example of a chart of A / D converted channel signals. In the example of FIG. 6, the sampling frequency is 1 [kHz]. This signal is a signal measured in the vicinity of muscles involved in thumb flexion when gripped from the weak state and returned to the weak state after several seconds.

  Next, the rectification processing unit 12 performs full-wave rectification on each input channel signal. FIG. 7 is an example of a chart obtained by full-wave rectifying the channel signal of FIG. Next, the smoothing processing unit 13 smoothes each channel signal. FIG. 8 is an example of a chart obtained by smoothing the channel signal of FIG.

  Next, the integration processing unit 14 integrates the amplitude of the integration interval for each channel signal input from the smoothing processing unit 13 and shifts the integration interval for each calculation cycle to obtain an integration value. FIG. 9A is the same as the smoothed channel signal of FIG. 8, and FIG. 9B is a chart of an example of the integrated channel signal. The normalization processing unit 15 normalizes each channel signal output from the integration processing unit 14 to be in the range of 0-1. FIG. 10 is a chart of channel signals obtained by normalizing the signals shown in FIG.

6 to 9, the vertical axis represents amplitude, and the horizontal axis represents time. In FIG. 10, the vertical axis is the amplitude at normalization, and the horizontal axis is the time axis.
Neural network 16, the feed forward component A, by inputting to the input layer 16a of the input vector I _k configured for each calculation period in feedback component B, and the finger joint angle of five fingers being normalized estimation, and outputs the output vector _{O k} from the output layer 16c. The output vector _Ok is denormalized by the conversion processing unit 17 to perform angle conversion, and output to the output device 30.

  In the following, a data glove (not shown) for measuring the finger joint angle is attached to the hand so that the angle of each finger joint of the five fingers can be individually detected, and the forearm Z FIG. 11A to FIG. 11D show the results of estimating the finger joint angle by attaching the eight electrodes 20 and inputting the surface myoelectric signal to the finger joint angle estimating device 10.

  In each figure, the vertical axis is the angle, the horizontal axis is the time [ms], the dotted line is a chart of actual values detected and measured by the data glove, and the solid line is a chart of estimated values estimated by the finger joint angle estimation device 10. It is.

  11A is a chart of the angle of the index finger inward and outward, FIG. 11B is a chart of the bending angle of the first joint of the index finger, and FIG. 11C is a bending angle of the second joint of the index finger. FIG. 11D shows a chart of the bending angle of the third joint of the index finger. As shown in each figure, it can be seen that the finger joint angles estimated by the finger joint angle estimation device 10 almost coincide with the measured values, and each joint angle is obtained almost accurately in real time. . Although not shown, the finger joint angles of the fingers other than the index finger were also obtained almost accurately.

  As a comparative example, FIG. 13 shows a conceptual diagram of a conventional neural network when the output of the neural network 16 is not used as a feedback component and a biological signal before the current time is not used.

  In the case of this comparative example, a data glove (not shown) is attached to the hand so that the angles of the finger joints of the five fingers can be individually detected, and eight electrodes are provided on the forearm Z. A result of estimating the finger joint angle by inputting the surface myoelectric signal to the finger joint angle estimating device 10 with 20 is shown in FIGS. 14A and 14B, the vertical axis represents the angle, the horizontal axis represents time [ms], the dotted line is a chart of actual measurement values detected and measured by the data glove, and the solid line is the finger joint angle estimation device 10. It is a chart of the estimated value estimated by this.

  FIG. 14A is a chart of the angle of inward and outward rotation of the index finger, and FIG. 14B is a chart of the bending angle of the first joint of the index finger. As shown in the figure, the estimated finger joint angle is a value far from the actually measured value.

According to the embodiment described in detail above, the following effects can be obtained.
(1) The finger joint angle estimation device 10 of the present embodiment inputs a plurality of surface myoelectric signals (biological signals) when each finger joint is moving, and extracts the feature amount of each surface myoelectric signal. The rectification processing unit 12, the smoothing processing unit 13, the integration processing unit 14, and the normalization processing unit 15 (feature amount extraction means) are arranged so that the extracted feature amounts are arranged one-dimensionally into a feature amount vector. Also, the finger joint angle estimation device 10 estimates each finger joint angle based on the input vector I _k for each calculation cycle, and outputs the estimated finger joint angle as an output vector (neural network means). ). The input vector I _k is a feed in which time series feature vectors from the past to the present extracted by the rectification processing unit 12, the smoothing processing unit 13, the integration processing unit 14, and the normalization processing unit 15 are arranged in one dimension. and forward component a, is composed are arranged in a feedback component B is one-dimensional, the feedback component B, the neural network 16, is constituted by the output vector O _k time series was estimated to date for each calculation cycle Yes. As a result, according to the finger joint angle estimation device 10 of the present embodiment, it is possible to cope with a temporal change in the relationship between the surface myoelectric signal and the finger joint angle, and the finger joint angles of the five fingers are continuously determined. It can be estimated accurately in real time.

  (2) In the finger joint angle estimation device 10 of the present embodiment, the rectification processing unit 12, the smoothing processing unit 13, the integration processing unit 14, and the normalization processing unit set the integration interval for the surface myoelectric signal for each calculation cycle. The integration value obtained by integrating is extracted as a feature value. As a result, an integral value that is a feature amount of the surface myoelectric signal can be acquired for each calculation cycle.

In addition, you may change embodiment of this invention as follows.
In the above-described embodiment, when the finger joint angle is estimated, the integral value obtained from the surface myoelectric signal (sEMG) measured in the forearm portion is used as the feature value as the biometric signal. The quantity is not limited to the integral value.

For example, the power spectrum of the surface myoelectric signal may be used.
In this case, when there are a plurality of measurement sites in the forearm, the channel number of the measurement site is set to d. Then, when the power spectrum obtained for each channel and for each sampling period is vectorized by obtaining a section sum for each predetermined bandwidth, the power spectrum (vector) sampled in a certain channel at one time is expressed by the equation It can be represented by the vector F _k ^d of (2). The predetermined bandwidth is not limited and may be determined as appropriate. The sum of the sections of the power spectrum is a feature amount.

In Equation (2), T is a transposed matrix, k is a rank in time series, d is a channel number, and p is the number of sections divided by a predetermined bandwidth of frequency.
The power spectrum can be obtained by Fourier transforming the full-wave rectified surface myoelectric signal. In addition, the calculation method of a power spectrum is not limited, but the calculation method by Fourier transformation is common.

  FIG. 12 shows the power spectrum after Fourier transform of the surface myoelectric signal, where the horizontal axis represents frequency and the vertical axis represents power spectrum intensity. The power spectrum includes information that is difficult to obtain compared to the case where the integrated value is used as a feature amount, for example, the effect of muscle fatigue can be reflected.

  ○ Since the power spectrum is a waveform, quantitative evaluation may be difficult. In this case, an average frequency (Mean power frequency: hereinafter referred to as MNF) or a central frequency (Median power frequency: hereinafter referred to as MDF) may be used as the feature quantity of the surface myoelectric signal.

MNF is a mathematically more straightforward amount, but MDF has the advantage of being less susceptible to noise contained in high-frequency components.
The definition formulas of MNF and MDF are shown by the following formulas (3) and (4).

Here, P (x) is a power spectrum. f is a frequency.
In addition, the biological signal is not limited to the surface myoelectric signal. Since the surface myoelectric signal (sEMG) is generated based on a nerve signal emitted from the brain, the brain information that is the basis of the surface myoelectric signal (sEMG) can also be usefully used for estimating the finger joint angle. The surface myoelectric signal (sEMG) is the sum of the action potentials of the muscle fibers and has a great significance in the amplitude of the signal. However, with regard to brain information, the signal processing method to be recommended differs depending on each signal.
In the following, the brain information includes brain waves (EEG), MRI signals for forming functional magnetic resonance images, near infrared spectroscopy (NIRS), and magnetoencephalogram (Magneto-Encephalo-Graphy: MEG). Can do.

(1) Electroencephalogram (EEG)
EEG is a signal that can be measured by an old non-invasive measurement technique. This is a signal obtained by measuring the action potential of nerve cells in the brain on the scalp surface. Here, event-related synchronization and event-related desynchronization due to increase / decrease change of at least one of α wave and β wave can be used as the feature amount. For example, there is a phenomenon called event-related desynchronization in which the α wave weakens when the motor area is activated, and this also occurs when imagining without actually exercising. There is also event-related synchronization when the α wave becomes large. Further, EEG has an advantage of higher time resolution than fMRI described later. What is important in EEG is the phenomenon of event-related desynchronization. This phenomenon changes a specific frequency component of an electroencephalogram at a specific site. Therefore, in order to extract an element related to motion from an electroencephalogram, signal processing is performed so that a power spectrum and frequency component distribution can be obtained by a frequency analysis method such as fast Fourier transform (FFT) or wavelet transform. In this embodiment, the signal processing is performed by a computer, and the signal processing result is used as a feedforward component. Then, the feedforward component and the feedback component are input as input vectors to the neural network 16 of the embodiment.

(2) Functional magnetic resonance imaging (fMRI)
It is a method for estimating a brain region where a greater amount of neural activity occurs from a change in reduced hemoglobin in the bloodstream, and is measured by MRI, and an MRI signal obtained from the brain region is used as a biological signal. Here, in the brain region, the location activated from the change in reduced hemoglobin in the blood flow at the region that controls exercise is used as a feature amount.

  Although the temporal resolution is inferior to the above-mentioned EEG, there is an advantage that the spatial resolution is extremely high. Measuring instruments are large and expensive, but they are the most powerful non-invasive method for reading nerve information in detail and converting it into machine control signals.

  In this embodiment, the MRI signal having the feature amount is used as a feedforward component. Then, the feedforward component and the feedback component are input as input vectors to the neural network 16 of the embodiment.

(3) Near infrared spectroscopy (NIRS)
In near-infrared spectroscopy (NIRS), near-infrared light (wavelength 700 nm to 1000 nm) that is highly permeable to biological tissues such as the head and muscles is irradiated from the outside, and light transmitted through the tissue is analyzed. Therefore, a device for examining the oxygenated state of hemoglobin in the blood flowing through the tissue from the outside is used. The biological signal is a signal detected by the apparatus. In addition, a change in the hemoglobin oxygenation state in the blood flowing through the tissue is used as a feature amount.

  In this modification, a local change in blood flow that occurs transiently when the brain is activated is detected by an optical sensor. It has intermediate characteristics between EEG and fMRI. Although the measurement system has the same scale as the EEG, it is non-contact, and unlike the EEG, there is an advantage that a terminal for attaching a biological signal to the body is not required and there is no troublesome installation of the terminal. In this embodiment, a signal having a feature value detected by the device is used as a feedforward component. Then, the feedforward component and the feedback component are input as input vectors to the neural network 16 of the embodiment.

(4) Magnetoencephalography (MEG)
A magnetoencephalogram (MEG) is similar to the EEG, but detects a change in a magnetic field caused by an active site in the brain as a feature quantity. Electrical signals are easy to get on noise, but magnetic fields are hard to get on.

  In this embodiment, a sensor (that is, a pickup coil) that detects the magnetic field is arranged around the subject's head, and a signal having a characteristic amount is generated from the change in the magnetic field and included in the signal. The change in the magnetic field is the feedforward component. Then, the feedforward component and the feedback component are input as input vectors to the neural network 16 of the embodiment.

  The detections in (2) to (4) basically detect the distribution of active sites in the brain. fMRI and NIRS can see changes in blood flow in the brain, and MEG can see the distribution of magnetic fields in the brain, and it can see which part of the nerve cell is activated. Therefore, in medical knowledge and pre-measurement, it is assumed that the finger movement is associated with the active site in the brain at that time.

○ As another modification, the following may be used.
The joint is driven by the movement of the tendon caused by the contraction of the muscle and the generation of tension. Therefore, a bicep or the like accompanying muscle contraction can be effectively used for estimation of the finger joint angle.

(1) Movement of tendon If you move your finger while looking inside the wrist, you can see how different tendons are moving depending on the joint to be moved. Since the movement of the tendon and the bending of the joint are directly related, it is possible to estimate the rotation angle of the joint from the amount of movement of the tendon.

  Thus, in order to detect the movement of the tendon, that is, to obtain a biological signal, a sensor that detects a pressure distribution in a two-dimensional plane, for example, a distributed pressure sensor can be used. Alternatively, using a video camera, the movement of the tendon is imaged, and an image signal including change information such as a shadow accompanying the movement of the tendon is acquired as a biological signal.

  Here, a change in tendon movement may be used as a feature amount based on a detection signal of the distributed pressure sensor. In this case, the detection signal having the feature amount is used as a feedforward component. Then, the feedforward component and the feedback component are input as input vectors to the neural network 16 of the embodiment.

  Alternatively, in the image signal imaged by the video camera, the change of the tendon may be obtained as a feature amount by taking the difference between the image frames obtained by imaging the movement of the tendon. In this case, an image signal having a feature amount in the portion with the change becomes a biological signal. Similarly, in this case, the biological signal having the feature amount is used as a feedforward component. Then, the feedforward component and the feedback component are input as input vectors to the neural network 16 of the embodiment.

(2) Muscular bulging Since tendon movement and muscular bulge are closely related, muscular bulge also uses a video camera, and an image signal containing bulge changes corresponding to changes in the finger joints is used as a biological signal. Acquired and used to estimate the finger joint angle. In this case, in the image signal imaged by the video camera, the change may be obtained as a feature amount by taking a difference between image frames obtained by imaging the change in the bulge of the muscle. In this case, an image signal having a feature amount in the portion with the change becomes a biological signal. Similarly, in this case, the biological signal having the feature amount is used as a feedforward component. Then, the feedforward component and the feedback component are input as input vectors to the neural network 16 of the embodiment.

(3) Neural signal The nerve transmits a weak electric signal. Since the signal for generating sEMG is also an electric signal, if a signal transmitted from the brain to the muscle cell when the finger joint is changed is detected, it becomes a useful signal as a biological signal. Specifically, a nerve signal can be detected by attaching a terminal to the motor nerve. A signal transmitted through a nerve is closer to a discrete digital signal than a continuous analog signal. By measuring signals transmitted through a plurality of nerves that control the muscle group of the forearm, and measuring how much time signals are transmitted by which nerves, a biological signal that can reflect movement is obtained. And in the signal which detected the nerve signal transmitted from the brain to the muscle cell of the said finger | toe by the motion of a finger joint, the change of the nerve signal becomes a feature-value.

  In this case, a biological signal having the feature amount is used as a feedforward component. Then, the feedforward component and the feedback component are input as input vectors to the neural network 16 of the embodiment.

In the above-described embodiment, the finger joint angle is estimated. However, the present invention is not limited to the finger joint angle, and it is of course possible to estimate the toe joint angle.
In the above embodiment, the neural network 16 is configured by a three-layer perceptron neural network, but may be configured by a three-layer perceptron neural network. Further, the neural network 16 is not limited to a perceptron, and may be constituted by other neural networks.

  ○ In the above embodiment, since one finger has three joints, in order to detect the bending angle (finger joint angle) of these joints, and further the angle of inner and outer rotation (finger joint angle) That is, in order to estimate each joint angle, the number of units of the output layer 16c of the neural network 16 is 20. However, the number of units of the output layer 16c may be reduced according to the joint angle required for estimation.

Further, in order to estimate other joint angles such as wrist joints in addition to the joint angles of the five fingers, the number of units of the output layer 16c may be increased.
In the above embodiment, a plurality of biological signals (surface myoelectric signals) are acquired, but a single surface myoelectric signal may be used.

  In the above-described embodiment, a single type of biological signal, that is, a surface myoelectric signal is used, but a plurality of types of the above-described various types of biological signals can be used in combination.

1 is a schematic block diagram of a finger joint angle estimation device according to an embodiment that embodies the present invention. Explanatory drawing of the structure of the neural network of an estimation part. (A)-(c) is explanatory drawing of the feedforward component A. FIG. (A)-(c) is explanatory drawing of the feedback component B. FIG. Explanatory drawing of an input vector. The chart of the channel signal after A / D conversion. A chart of channel signals subjected to full-wave rectification. The chart of the smoothed channel signal. (A) is the same channel signal chart as in FIG. 8, and (b) is a channel signal chart after integration processing. Chart of normalized channel signal. (A) is a chart of a finger joint angle obtained by detecting and measuring the inversion of the index finger with a data glove, and a chart of a finger joint angle estimated by the finger joint angle estimation device. Chart of detected finger joint angle and chart of finger joint angle estimated by finger joint angle estimation device, (c) shows chart of finger joint angle and finger joint angle obtained by detecting and measuring second joint angle of index finger with data glove. The chart of the finger joint angle estimated by the estimation apparatus, (d) is a chart of the finger joint angle obtained by detecting and measuring the angle of the third joint of the index finger with the data glove and the chart of the finger joint angle estimated by the finger joint angle estimation apparatus. The chart of the power spectrum after carrying out the Fourier transform of the surface myoelectric signal. The conceptual diagram when not using the output of the neural network 16 of a comparative example as a feedback component. (A) is a chart of the finger joint angle obtained by detecting and measuring the inversion / inversion of the index finger in the comparative example with a data glove and a chart of the finger joint angle estimated by the finger joint angle estimation device, and (b) is the first joint of the index finger in the comparative example. The chart of the finger joint angle estimated by the finger joint angle estimation device and the chart of the finger joint angle detected and measured by the data glove.

Explanation of symbols

10: Finger joint angle estimation device (finger joint angle estimation device),
12 ... Rectification processing unit, 13 ... Smoothing processing unit, 14 ... Integration processing unit,
15... Normalization processing unit (which constitutes a feature quantity extracting means together with the smoothing processing unit 13 and the integration processing unit 14),
16. Neural network (neural network means),
17 ... Conversion processing unit, 20 ... Electrode, 30 ... Output device.

Claims (12)

Feature quantity extraction means for inputting one or a plurality of types of biological signals when each finger joint is moving, extracting the feature quantity of each biological signal, and arranging the extracted feature quantities in a one-dimensional manner as a feature quantity vector When,
Neural network means for estimating an angle of each finger joint (hereinafter referred to as a finger joint angle) based on an input vector for each calculation cycle, and outputting the estimated finger joint angle as an output vector;
The input vector is composed of a feedforward component in which time-series feature amount vectors from the past to the present extracted by the feature amount extraction unit are arranged one-dimensionally, and an output feedback component,
The finger joint angle estimation device according to claim 1, wherein the feedback component is composed of the time-series output vectors that the neural network means has estimated to the present for each calculation cycle.   The finger joint angle estimation apparatus according to claim 1, wherein the biological signal is a surface myoelectric signal.   3. The feature amount extraction unit is configured to integrate the surface myoelectric signal by shifting an integration interval for each calculation cycle, and extract an integration value obtained by the integration as a feature amount. Finger joint angle estimation device.   The finger joint angle estimation apparatus according to claim 2, wherein the feature amount extraction unit extracts a section sum for each predetermined bandwidth of the power spectrum of the surface myoelectric signal as a feature amount.   The finger joint angle estimation device according to claim 2, wherein the feature amount extraction unit extracts an average frequency or a center frequency of a power spectrum of the surface myoelectric signal as a feature amount. The biological signal is an electroencephalogram,
2. The feature amount extraction unit extracts event-related synchronization and event-related desynchronization due to increase / decrease change of at least one of the α wave and β wave of the electroencephalogram as feature amounts. The described finger joint angle estimation device. The biological signal is an MRI signal of a brain region,
The finger joint angle estimation device according to claim 1, wherein the feature amount extraction unit extracts a location activated from a change in reduced hemoglobin in the bloodstream in a region that controls exercise as a feature amount. The biological signal is a signal obtained by detecting near-infrared light transmitted through muscles related to the head or finger joints,
The finger joint angle estimation device according to claim 1, wherein the feature amount extraction unit extracts, as a feature amount, a change in a hemoglobin oxygenation state in blood flowing through a muscle tissue related to a head or a finger joint. . The biological signal is a magnetoencephalogram,
The finger joint angle estimation apparatus according to claim 1, wherein the feature amount extraction unit extracts a change in the magnetic field of the magnetoencephalogram as a feature amount. The biological signal is a signal that captures the movement of a tendon by the movement of a finger joint,
The finger joint angle estimation apparatus according to claim 1, wherein the feature amount extraction unit extracts a change in movement of the tendon as a feature amount. The biological signal is a signal that captures a change in muscle bulge caused by movement of a finger joint,
The finger joint angle estimation device according to claim 1, wherein the feature amount extraction unit extracts a change in the bulge of the muscle as a feature amount. The biological signal is a signal for detecting a nerve signal transmitted from the brain to a muscle cell of the finger by movement of a finger joint,
The finger joint angle estimation apparatus according to claim 1, wherein the feature amount extraction unit extracts a change in a nerve signal that changes according to the movement of the finger joint as a feature amount.

Images