CN105892676A - Human-machine interaction device, system and method of vascular intervention operation wire feeder - Google Patents
Human-machine interaction device, system and method of vascular intervention operation wire feeder Download PDFInfo
- Publication number
- CN105892676A CN105892676A CN201610266180.7A CN201610266180A CN105892676A CN 105892676 A CN105892676 A CN 105892676A CN 201610266180 A CN201610266180 A CN 201610266180A CN 105892676 A CN105892676 A CN 105892676A
- Authority
- CN
- China
- Prior art keywords
- electromyographic signal
- wire feeder
- wire
- seal wire
- neutral net
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 230000003993 interaction Effects 0.000 title claims abstract description 18
- 230000002792 vascular Effects 0.000 title abstract description 4
- 230000009471 action Effects 0.000 claims abstract description 28
- 230000033001 locomotion Effects 0.000 claims description 40
- 210000004204 blood vessel Anatomy 0.000 claims description 30
- 230000007935 neutral effect Effects 0.000 claims description 30
- 239000010410 layer Substances 0.000 claims description 28
- 210000003205 muscle Anatomy 0.000 claims description 28
- 238000012549 training Methods 0.000 claims description 16
- 230000000875 corresponding effect Effects 0.000 claims description 14
- 230000003183 myoelectrical effect Effects 0.000 claims description 14
- 210000003813 thumb Anatomy 0.000 claims description 14
- 238000001914 filtration Methods 0.000 claims description 13
- 230000003321 amplification Effects 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 8
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 7
- 230000004118 muscle contraction Effects 0.000 claims description 7
- 238000013528 artificial neural network Methods 0.000 claims description 6
- 239000000284 extract Substances 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 claims description 5
- 230000002452 interceptive effect Effects 0.000 claims description 4
- 238000012360 testing method Methods 0.000 claims description 4
- 230000002596 correlated effect Effects 0.000 claims description 3
- 230000001537 neural effect Effects 0.000 claims description 3
- 230000000644 propagated effect Effects 0.000 claims description 3
- 230000003068 static effect Effects 0.000 claims description 3
- 238000011478 gradient descent method Methods 0.000 claims description 2
- 239000011229 interlayer Substances 0.000 claims description 2
- 238000012546 transfer Methods 0.000 claims description 2
- 238000007781 pre-processing Methods 0.000 claims 1
- 230000008859 change Effects 0.000 abstract description 8
- 238000002567 electromyography Methods 0.000 abstract 1
- 210000002364 input neuron Anatomy 0.000 description 7
- 230000007246 mechanism Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 210000002569 neuron Anatomy 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- LFQSCWFLJHTTHZ-UHFFFAOYSA-N Ethanol Chemical compound CCO LFQSCWFLJHTTHZ-UHFFFAOYSA-N 0.000 description 4
- 241001269238 Data Species 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000008602 contraction Effects 0.000 description 2
- 230000005611 electricity Effects 0.000 description 2
- 210000003811 finger Anatomy 0.000 description 2
- 210000005224 forefinger Anatomy 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 208000024172 Cardiovascular disease Diseases 0.000 description 1
- 208000005189 Embolism Diseases 0.000 description 1
- 208000007536 Thrombosis Diseases 0.000 description 1
- 230000036982 action potential Effects 0.000 description 1
- 238000002583 angiography Methods 0.000 description 1
- 238000005311 autocorrelation function Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 210000004027 cell Anatomy 0.000 description 1
- 208000026106 cerebrovascular disease Diseases 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002526 effect on cardiovascular system Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 210000003414 extremity Anatomy 0.000 description 1
- 210000000245 forearm Anatomy 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000013152 interventional procedure Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003387 muscular Effects 0.000 description 1
- 210000004218 nerve net Anatomy 0.000 description 1
- 210000000653 nervous system Anatomy 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/25—Bioelectric electrodes therefor
- A61B5/279—Bioelectric electrodes therefor specially adapted for particular uses
- A61B5/296—Bioelectric electrodes therefor specially adapted for particular uses for electromyography [EMG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/389—Electromyography [EMG]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Medical Informatics (AREA)
- Heart & Thoracic Surgery (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Pathology (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Molecular Biology (AREA)
- Human Computer Interaction (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Dermatology (AREA)
- Neurology (AREA)
- Neurosurgery (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention discloses a human-machine interaction system, device and method of a vascular intervention operation wire feeder based on surface electromyography. The system comprises a driving guide wire, a first electrode plate, an arm, a second electrode plate, an electromyographic signal acquisition channel, an electromyographic signal acquisition device, a driven guide wire, a wire feeder and a control device, wherein the first electrode plate is pasted on the abductor pollicis brevis of a user; the second electrode plate is pasted on the musculus biceps brachii of the user; the electromyographic signal acquisition device is used for acquiring electromyographic signals; the control device generates control instructions according to the electromyographic signals; the wire feeder controls a radial stepping motor and an axial stepping motor to execute actions according to the received control instructions; the radial stepping motor is used for rotating the driven guide wire; and the axial stepping motor is used for pushing the driven guide wire. In the invention, the problems such as low human-machine interaction transparency, low operation efficiency and change of natural operation habits of a doctor of a vascular intervention robot are solved.
Description
Technical field
The present invention relates to signal transacting and area of pattern recognition, particularly relate to a kind of based on surface myoelectric
The human-computer interaction device of blood vessel intervention operation wire feeder, system and method.
Background technology
At present, blood vessel intervention operation robot has become the important of auxiliary medical treatment cardiovascular and cerebrovascular disease
Instrument.Robot assisted blood vessel intervention operation refers to that doctor is at digital subtraction angiography imaging (DSA)
Under the guiding of system, it is (a kind of with rigidity that manipulation control stick or operation handle control robot propelling movement seal wire
Soft silk) in human vas move, focus is treated, reaches embolism deformity blood vessel, dissolving
The purposes such as thrombus, expansion narrow blood vessel.By master & slave control, blood vessel intervention operation robot can be big
Amplitude ground reduces the X-ray radiation dosage that doctor is subject to.
But, in whole interventional procedure, doctor needs to be controlled by control stick or operation handle
Robot processed performs the operation, and this revolutionizes tradition and gets involved the natural mode of operation of operation.Blood vessel is situated between
Enter that robot is primarily present man-machine interaction poor transparency, procedure efficiency is low, change doctor naturally performs the operation
The problems such as operating habit, Man machine interaction has become as restriction blood vessel and gets involved the main of robot development
One of bottleneck.Doctor, in long-term vascular intervention operation practice, defines good operation skill,
Can be good at manipulating intervention apparatus, and the introducing of robot by the power sense of touch on " feel " finger
Cut off the direct thoughts and feelings that intervention apparatus is manipulated by doctor physically.How by the real-time letter in art
Breath perception, improves the transparency of intervention apparatus operation, is that can blood vessels present be got involved robot and be broken through and face
One major issue of bed application bottleneck.
For the problems referred to above, in Chinese invention patent CN 104899594 disclosed in 9 days September in 2015
A, discloses hand motion recognition method based on Decomposition Surface EMG, utilizes sEMG signal
Decompose the moving cell action potential information obtained, hand motion is identified, is effectively increased list
Passage sEMG discrimination.In Chinese invention patent CN disclosed in 19 days Augusts in 2015
104850231 A disclose a kind of man-machine interface system merged based on surface myoelectric and muscle signals,
Surface myoelectric and muscle signals can be carried out fusing and decoding, the intention of operator is converted into external setting
Standby control instruction.In Chinese invention patent CN 105326500 A disclosed in 17 days February in 2016
Disclose a kind of action identification method based on surface electromyogram signal and equipment, surface myoelectric can be believed
Number frequency combine with amplitude Characteristics and identify corresponding limb action.
But, although these methods have carried out a lot of process to electromyographic signal, all reach good knowledge
Other effect, but be not applied in blood vessel intervention operation, the most not by surface electromyogram signal and blood
Pipe is got involved the operation naturally of doctor in operation and is connected, and prior art does not also take into account this point.
Summary of the invention
In order to solve above-mentioned problems of the prior art, the invention provides a kind of based on surface flesh
The man-machine interactive system of the blood vessel intervention operation wire feeder of electricity, it utilizes surface electromyogram signal as doctor
Raw Man Machine Interface in blood vessel intervention operation.
According to an aspect of the present invention, it provides a kind of blood vessel intervention operation wire feed based on surface myoelectric
The man-machine interactive system of mechanism, this system includes: actively seal wire, the first electrode slice, the first electrode are even
Wiring, arm, the second electrode connecting line, the second electrode slice, electromyographic signal collection passage, myoelectricity letter
Number Acquisition Instrument, driven seal wire, wire feeder and control device;Wherein,
Described active seal wire is for user operation;
Described first electrode slice is used for being attached at user's short abductor muscle of thumb, and by the first electrode connecting line even
It is connected on the first electromyographic signal collection passage of electromyographic signal collection instrument;
Described second electrode slice is used for being attached at user's bicipital muscle of arm, is connected by the second electrode connecting line
To the second electromyographic signal collection passage of electromyographic signal collection instrument;
Described electromyographic signal collection instrument is for gathering the first electromyographic signal collection passage and the second myoelectricity letter
Electromyographic signal on number acquisition channel;
Described control device produces control instruction according to described electromyographic signal, and is sent to wire feeder;
Described wire feeder includes driven seal wire, radial stepping motor and axial stepper motor;Described footpath
Being used for rotating described driven seal wire to stepper motor, described axial stepper motor is used for pushing described driven
Seal wire, described wire feeder controls radial stepping motor and axial stepping according to the control instruction received
Motor execution action.
According to a second aspect of the present invention, it provides a kind of blood vessel intervention operation based on surface myoelectric and send
The human-computer interaction device of silk mechanism, comprising:
Electromyographic signal collection module, electromyographic signal pretreatment module, electromyographic signal characteristic extracting module,
Action recognition module and control module, wherein:
Described electromyographic signal collection module is used for gathering doctor and carries out operation during blood vessel intervention operation
The actively electromyographic signal of hand contraction of muscle during seal wire;
Described electromyographic signal pretreatment module is for carrying out differential amplification and bar to the electromyographic signal gathered
Special Butterworth filtering;
Described electromyographic signal characteristic extracting module is for extracting the feature of the electromyographic signal through pretreatment;
Described action recognition module is used for training BP neutral net, and utilizes the BP trained neural
Hand motion when network carries out described operation to doctor is identified;
Described control module, for the action gone out according to described BP neural network recognization, sends corresponding
Control instruction is to wire feeder, to control the corresponding actions of driven seal wire.
According to a third aspect of the present invention, it is provided that a kind of blood vessel intervention operation wire feed based on surface myoelectric
The man-machine interaction method of mechanism, it comprises the following steps:
Step 1: be attached at short abductor muscle of thumb by three piece of first electrode slice, is attached to three piece of second electrode slice
At the bicipital muscle of arm, three piece of first electrode slice connect respectively the positive pole in electromyographic signal collection instrument first passage,
Negative pole and ground, three piece of second electrode slice just connects in electromyographic signal collection instrument second channel respectively
Pole, negative pole and ground;
Step: 2: operator carries out rotating, stop and pushing three kinds of actions with operation by human hand actively seal wire
Each a period of time, first passage described in electromyographic signal collection instrument acquisition operations person and the flesh on second channel
The signal of telecommunication;
Step 3: the forward part of described electromyographic signal is gathered data and is correlated with as training data, extraction
Feature, and train BP neutral net;
Step 4: the rear section of described electromyographic signal gathers the BP neutral net that data test trains,
When described BP neutral net rate of accuracy reached is to predetermined threshold, enters next step, otherwise return step
3;
Step 5: operator with operation by human hand actively seal wire electromyographic signal collection instrument gather first passage and
Feature extraction is carried out after electromyographic signal in second channel, then as the input of BP neutral net,
The actuating signal of described operator is identified by BP neutral net according to described input, and exports control
System instruction is to wire feeder;
Step 6: when wire feeder receives rotation instruction, wire feeder controls radial stepping motor
Motion, it is achieved the rotation of driven seal wire;When wire feeder receives propelling movement instruction, wire feeder
Control the motion of axial stepper motor, it is achieved the propelling movement of driven seal wire;Refer to when wire feeder receives stopping
After order, the radial stepping motor of wire feeder control and the equal stop motion of axial stepper motor.
According to a fourth aspect of the present invention, it is provided that a kind of blood vessel intervention operation wire feed based on surface myoelectric
The man-machine interaction method of mechanism, it comprises the following steps:
Gather hand contraction of muscle when operating active seal wire during doctor carries out blood vessel intervention operation
Electromyographic signal;
The electromyographic signal gathered is carried out differential amplification and Butterworth filtering;
Extract the feature through the electromyographic signal pre-processed;
Utilize the features training BP neutral net extracted, and utilize the BP neutral net trained
Hand motion when doctor carries out described operation is identified;
The action gone out according to described BP neural network recognization, sends corresponding control instruction to wire-feed motor
Structure, to control the corresponding actions of driven seal wire.
Compared with prior art, the present invention obtains and provides the benefit that: getting involved hand by obtaining doctor
Afterturn seal wire and propelling movement or hand muscle signal of telecommunication when withdrawing seal wire during art, filtering process after warp
The training pattern that BP neural metwork training obtains, can be used to identify the surgical action of doctor.Action
Again control command is sent to wire feeder after identification, it is achieved Man Machine Interface.This man-machine interaction connects
Mouth solve blood vessel get involved robot man-machine interaction poor transparency, procedure efficiency is low, change doctor from
The problems such as right operation technique custom.
Accompanying drawing explanation
Fig. 1 is the method flow diagram that man-machine interaction according to an embodiment of the invention realizes;
Fig. 2 is BP neural network structure schematic diagram according to an embodiment of the invention;
Fig. 3 is Man Machine Interface schematic diagram according to an embodiment of the invention.
Detailed description of the invention
For making the object, technical solutions and advantages of the present invention clearer, below in conjunction with concrete real
Execute example, and referring to the drawings, the present invention is described in more detail.
The invention provides the man-machine friendship of a kind of blood vessel intervention operation wire feeder based on surface myoelectric
Interface mutually.During blood vessel intervention operation, mainly include afterturn when doctor operates seal wire or conduit and push away
Send two kinds of actions.During afterturn seal wire, doctor's forefinger and thumb pinch seal wire, and two fingers slide relatively,
Under the driving of frictional force, seal wire rotates and searches out optimal bifurcated blood vessel entrance.Propelling movement is led
During silk, doctor clamps seal wire in the same way, and whole hand pushes forward or withdraws backward, real
Show the axially-movable of seal wire.
In order to achieve the above object, propose a kind of blood vessel based on surface myoelectric according to the present invention to get involved
The Man Machine Interface of operation wire feeder, this interface includes: electromyographic signal collection module, and myoelectricity is believed
Number pretreatment module, electromyographic signal characteristic extracting module, action recognition module, control module, wherein:
Described electromyographic signal collection module is the 8 passage electromyographic signals developed voluntarily by the Chinese Academy of Sciences
Acquisition Instrument gathers electromyographic signal when short abductor muscle of thumb and bicipital muscle of arm contraction.During doctor's afterturn seal wire, hand
The short abductor muscle of thumb contraction in portion and diastole, the thumb of forefinger produces relative motion, and frictional force drives seal wire revolves
Turn;When doctor pushes seal wire, the bicipital muscle of arm of hand shrinks and diastole, and muscular force drives forearm motion,
Thus drive seal wire push forward or withdraw backward.Before gathering electromyographic signal, the hair of skin surface
Must remove clean, then wipe with alcohol stranding, after alcohol volatilizees, electrode slice is attached to short abductor muscle of thumb and the upper arm
Bicipital surface, and connect two acquisition channels of electromyographic signal collection instrument, the collection of electromyographic signal
Frequency is 1024Hz,.
Described electromyographic signal processing module mainly includes differential amplification and Butterworth filtering.Surface flesh
The signal of telecommunication substantially nervous system stimulates produced a kind of ultra-weak electronic signal during contraction of muscle, not
Amplified primary signal is generally between 50 μ v to 20-30mv.Original surface electromyogram signal exists
After collecting system, it is necessary first to carry out differential amplification to eliminate the impact from dc source noise.
Electromyographic signal limiting frequency scope is generally between 10-500Hz.Wherein Butterworth filtering is mainly
High frequency in electromyographic signal, low frequency, 50Hz power frequency component are filtered, including bandpass filtering and trap
Filtering.Bandpass filtering is to eliminate the frequency range of the interference fringe bandpass filter of motion artifacts and high-frequency noise
20-500Hz can be set to.It addition, for the Hz noise eliminating 50Hz, in addition it is also necessary to carry out 50Hz
Notch filter.In the range of the main frequency range of electromyographic signal concentrates on 50-150Hz.
Described electromyographic signal characteristic extracting module uses the surface electromyogram signal of two passages gathered
Extract the relevant feature parameters of signal, including: integrated absolute (IAV), difference absolute value DAMV,
Zero passage is counted (ZC), mean square deviation (VAR), median frequency (MDF), average frequency (MF).Six spies
Levy the definition of parameter and extract as follows:
(1) integrated absolute (IAV)
Integrated absolute (IAV) is to be completely converted on the occasion of later feature by the amplitude of signal, it
Being the most intuitively reacting of contraction of muscle strength, integrated absolute (IAV) is the biggest, and the receipts of muscle are described
Contracting strength is the biggest.The method that integrated absolute (IAV) generally utilizes sliding window to average obtains,
Its mathematic(al) representation can be write as:
Wherein xjRepresenting the amplitude of the jth time point of electromyographic signal, N represents the length of sliding window.
(2) difference absolute value (DAMV)
Difference absolute value (DAMV) first passes through in the electromyographic signal time series calculating length-specific
The difference absolute value of each two point of proximity, is averaging the most again.The size of this feature value indicates myoelectricity
The vibrant characteristic of signal, this value is the biggest, illustrates that signal shakes the strongest, otherwise the faintest.It
Mathematic(al) representation be represented by:
(3) zero passage counts (ZC)
Electromyographic signal time series that what zero passage counted that (ZC) describe is is in the change procedure of amplitude, positive and negative
The number of times of value alternately change, or the number of times through 0 coordinate.This feature value from the angle of time domain to signal
Frequency domain character estimated, be used for the severe degree that waveform changes in amplitude is described, reflect
The variation tendency of signal, its computing formula is represented by:
Wherein xiData dot values for surface electromyogram signal sampling.
(4) mean square deviation (VAR)
Mean square deviation (VAR) is that the energy size utilizing electromyographic signal is as characteristic value.Generally, mean square deviation
(VAR) it is to utilize statistical feature, electromyographic signal time series is deducted electromyographic signal average, asks
Square, then average.Owing to electromyographic signal has eliminated DC component in pretreatment, therefore its
Average can be considered 0, then the calculation of mean square deviation (VAR) is represented by:
It is the measurement of signal power, and wherein N is the sampling number often organized.
(5) median frequency (MDF)
The speed of median frequency (MDF) and contraction of muscle have much relations, median frequency (MDF) be by
The power spectrum of signal is divided into two-part intermediate frequency, and its mathematic(al) representation is:
Wherein S (f) represents the power spectral density of electromyographic signal, and f represents signal frequency.
For discrete random sequence x (n), its autopower spectral density SxF () is represented by:
Wherein TsFor sampling time interval, RxM () is the auto-correlation function of signal x (n).
(6) average frequency (MF)
The average frequency (MF) of electromyographic signal is represented by:
In the present invention, the electromyographic signal of two passages all comprises this 6 features, to the feature extracted
Parameter builds the characteristic vector of multi-parameter, refers to that each action two paths of signals totally 12 parameters build one
Individual characteristic vector, this feature vector dimension is 12 dimensions.
Described action recognition module is for identifying the electromyographic signal after feature extraction, and output is corresponding
Action.For afterturn, static, three actions of propelling movement, each action gathers 30 seconds, each passage
There are 30720 sampled points.Wherein, first 20 seconds sampled datas are for backpropagation (BP) nerve net
The training of network, rear 10 second datas are for the test of network.
After being extracted the feature of electromyographic signal, in addition it is also necessary to the weights of training network and threshold value, and to net
The performance of network is tested.Before training network, the identical parameters of different actions must be done horizontal normalizing
Change processes, and the identical parameters being the multiple different actions to same operator makees lateral comparison normalizing
Change.Normalization is that a kind of dimensionless processes means, makes the absolute value of physical system numerical value become certain phase
Relation to value.
The network structure selected is three layers: input layer, intermediate layer and output layer.Input layer
Number by the characteristic value used number determine, when each passage use single IAV, DAMV,
When ZC, VAR, MF, MDF etc. input as feature, then the input neuron number of network is 2,
When using m feature as input, then the input neuron number of network is 2m.The present invention is real
In example, input layer node is 12.The number of intermediate layer neuron is adjusted according to the convergence situation of network
Joint, in present example, intermediate layer node is 10.The number of the neuron of output layer is 3, and defeated
Go out value between zero and one, in order to distinguish 3 kinds of different motion states, afterturn, static, push can
To represent with three binary codes, it is [100], [010], [001] respectively.
The training process of BP neutral net is as follows: forward-propagating be input signal from input layer through centre
Layer is transmitted to output layer, if output layer has obtained desired output, then learning algorithm terminates;Otherwise, turn
To backpropagation.The learning algorithm of network includes that propagated forward is for calculating the output of network;Reversely pass
Broadcast employing gradient descent method, be used for adjusting the weights of each interlayer.The basic thought of algorithm is that gradient declines
Method.It uses gradient search technology, the error of real output value Yu desired output to making network
Mean-square value is minimum.Neutral net not only has the features such as self study, self-organizing and parallel processing, also
There is the strongest fault-tolerant ability and associative ability, therefore neutral net has pattern and knows inputoutput data
Other ability.
Described control module is used for sending control command to wire feeder, the motion shape of control wire feeder
State.When network is output as [1 0 0], control program sends a command to wire feeder, wire feeder
Radial stepping motor moves, it is achieved the rotation of seal wire;When network is output as [0 0 1], control program
Send a command to wire feeder, the axial stepper motor motion of wire feeder, it is achieved the propelling movement of seal wire;
When network is output as [0 1 0], control program sends a command to wire feeder, the radial direction of wire feeder
Stepper motor and the equal stop motion of axial stepper motor.
Fig. 1 is Man Machine Interface flowchart, original surface according to an embodiment of the invention
Electromyographic signal is after collecting system, it is necessary first to carries out differential amplification and makes an uproar from dc source to eliminate
The impact of sound, then carries out bandpass filtering and leads to filter with elimination motion artifacts and the interference of high-frequency noise, band
The frequency range of ripple device can be set to 20-500Hz.It addition, for the Hz noise eliminating 50Hz,
Also need to carry out the notch filter of 50Hz, it is ensured that the quality of signal.Then filtered signal is entered
Row feature extraction, including: integrated absolute (IAV), difference absolute value (DAMV), zero crossing
Number (ZC), mean square deviation (VAR), median frequency (MDF), average frequency (MF).Further according to accordingly
Features training BP neutral net, the network trained just can be used to identify the hand motion of doctor,
Including: afterturn, stop, pushing, and export corresponding binary code: [1 0 0], [0 1 0], [0
0 1].Then corresponding control command is sent in the control chip in slave computer, controls corresponding
Motor movement or stopping.
Fig. 2 is BP neural network structure schematic diagram according to an embodiment of the invention, establishes one
Three layers of BP neutral net realize the mapping of electromyographic signal and doctor's hand motion, have selected non-linear
Tansig function and linear purelin function pass respectively as intermediate layer transfer function and the output layer of network
Defeated function.Such as the network structure of Fig. 2, the output of its network is represented by:
Wherein, p=[p1, p2..., pl] be the input vector of network, i.e. the electromyographic signal feature of short abductor muscle of thumb
Electromyographic signal feature with the bicipital muscle of arm;L is the input neuron number of network, binWith boutIt is respectively
Network intermediate layer threshold vector and output layer threshold vector, WinWith WoutIt is respectively network input layer weights
Matrix and output layer weight matrix, State=[State1, State2, State3] it is expressed as the identification of motion state
Binary code.Concrete determination methods: output vector State and the motion state binary representation of standardInner product, inner product the maximum is i.e. right
The motion state answered.
It can be seen that number l of input neuron depends not only on the number of electromyographic signal passage, also
Depend on the number of selected feature.If selected Characteristic Number is m, simultaneously the port number of electromyographic signal
For k, then the number of input neuron is l=km.The number of output layer neuron is 3.Intermediate layer
The number of neuron is uncertain, needs to enter according to the quantity of input neuron and the performance of network
Row constantly regulation.In an embodiment of the present invention, selected Characteristic Number is 6, the passage of electromyographic signal
Number is 2, and the number of input neuron is 12, and intermediate layer neuron number is 10.
Fig. 3 is man-machine interactive system schematic diagram, this man-machine interface system according to an embodiment of the invention
Including: actively seal wire the 1, first electrode slice the 2, first electrode connecting line 3, arm the 4, second electrode
Connecting line the 5, second electrode slice 6, electromyographic signal collection passage 7, electromyographic signal collection instrument 8, connection
Line 9, computer 10, display 11, driven seal wire 12, radial stepping motor 13, axial stepping
Motor 14, wire feeder 15.
Wherein, actively seal wire 1, user is sent by the described active seal wire 1 of operation and operates life accordingly
Order;Preferably, described operation includes rotating, pushing and stop operation;
First electrode slice 2, it is used for being attached at user's short abductor muscle of thumb, and it passes through the first electrode connecting line
3 are connected on the first electromyographic signal collection passage 7 of electromyographic signal collection instrument 8;
Second electrode slice 6, it is used for being attached at user's bicipital muscle of arm, by the second electrode connecting line 5
It is connected on the second electromyographic signal collection passage 7 of electromyographic signal collection instrument 8;
Electromyographic signal collection instrument 8, for the first electromyographic signal collection passage and the second electromyographic signal collection
Electromyographic signal on passage;
Computer 10, for producing control instruction according to described electromyographic signal, and is sent to wire feeder
15;
Wire feeder 15, it includes driven seal wire 12, radial stepping motor 13 and axial stepper motor
14;Wherein, described radial stepping motor 13 rotates described driven lead according to the control instruction that receives
Silk 12, described axial stepper motor 14 pushes described driven seal wire 12 according to the control instruction received.
Controlling wire feeder drives guide wire motion to comprise the following steps:
Step 1: remove short abductor muscle of thumb and the chaeta of the bicipital muscle of arm on operator's arm 6, and use alcohol
Wiping;
Step 2: after alcohol volatilization completely, first electrode slice the 2, second electrode slice 5 is pasted respectively
At short abductor muscle of thumb and the bicipital muscle of arm, the first electrode slice 2 includes three pieces, and the second electrode slice 5 includes three
Block, every cube electrode sheet 2,5 is separated by 1-2cm, and three cube electrode sheets in the first electrode slice 2 pass through first
Electrode connecting line 3 is respectively connecting to the positive pole of a passage 7 of electromyographic signal collection instrument 8, negative pole and ground,
Three cube electrode sheets in second electrode slice 5 are connected electromyographic signal respectively and adopt by the second electrode connecting line 5
The collection positive pole of another passage 7 of instrument 8, negative pole and ground;
Step 3: operator rotates with arm 4 operation actively seal wire 1, stop, pushing each 30 seconds,
After each action completes, have a rest 5 minutes and continue to gather next action, and open computer 10 simultaneously
Middle pc control procedure, the electromyographic signal data of each muscle of acquisition operations person;
Step 4: in computer-controlled program, by 20 second datas before the electromyographic signal of two passages
As training data, extract correlated characteristic, and train BP neutral net;
Step 5: the BP neutral net trained with the data test of latter 10 seconds, when rate of accuracy reached arrives
When 90%, enter next step, otherwise return step 3;
Step 6: connect wire feeder 15, operator operates actively seal wire 1, including rotating, stopping,
Pushing three actions, electromyographic signal collection instrument 8 carries out feature extraction after gathering electromyographic signal, then makees
For the input of BP neutral net, actuating signal is identified by network, and exports control instruction to sending
Silk mechanism 15;When network is output as [1 0 0], control program sends a command to wire feeder 15, send
The radial stepping motor 13 of silk mechanism 15 moves, it is achieved the rotation of driven seal wire 12;When network is defeated
When going out for [0 0 1], control program sends a command to wire feeder 15, the axial step of wire feeder 15
Enter motor 14 to move, it is achieved the propelling movement of driven seal wire 12;When network is output as [0 1 0], control
Program sends a command to wire feeder 15, the radial stepping motor 13 of wire feeder and axial stepping electricity
The equal stop motion of machine 14.
Computer 20 processor is Intel i7-2600, RAM 4.00GB;Display 21 is 22inch
Widescreen.
Particular embodiments described above, is carried out the purpose of the present invention, technical scheme and beneficial effect
Further describe, be it should be understood that the foregoing is only the present invention specific embodiment and
, be not limited to the present invention, all within the spirit and principles in the present invention, that is done any repaiies
Change, equivalent, improvement etc., should be included within the scope of the present invention.
Claims (13)
1. a man-machine interactive system for blood vessel intervention operation wire feeder based on surface myoelectric, its
Being characterised by, this system includes: actively seal wire, the first electrode slice, the first electrode connecting line, arm,
Second electrode connecting line, the second electrode slice, electromyographic signal collection passage, electromyographic signal collection instrument, from
Dynamic seal wire, wire feeder and control device;Wherein,
Described active seal wire is for user operation;
Described first electrode slice is used for being attached at user's short abductor muscle of thumb, and by the first electrode connecting line even
It is connected on the first electromyographic signal collection passage of electromyographic signal collection instrument;
Described second electrode slice is used for being attached at user's bicipital muscle of arm, is connected by the second electrode connecting line
To the second electromyographic signal collection passage of electromyographic signal collection instrument;
Described electromyographic signal collection instrument is for gathering the first electromyographic signal collection passage and the second myoelectricity letter
Electromyographic signal on number acquisition channel;
Described control device produces control instruction according to described electromyographic signal, and is sent to wire feeder;
Described wire feeder includes driven seal wire, radial stepping motor and axial stepper motor;Described footpath
Being used for rotating described driven seal wire to stepper motor, described axial stepper motor is used for pushing described driven
Seal wire, described wire feeder controls radial stepping motor and axial stepping according to the control instruction received
Motor execution action.
2. the system as claimed in claim 1, wherein, described control instruction rotates, pushes and stop
Only instruction;Described wire feeder rotates described driven according to rotating the instruction described radial stepping motor of control
Seal wire;Described wire feeder according to described push instruction control described axial stepper motor push described from
Dynamic seal wire, described wire feeder stops described axial stepper motor and described footpath according to described halt instruction
Operate to stepper motor.
3. system as claimed in claim 2, wherein, user's operation bag to described active seal wire
Include afterturn, propelling movement and static, respectively corresponding rotation, propelling movement and halt instruction.
4. the system as described in any one of claim 1-3, wherein, described control device receives institute
Carry out feature extraction after stating electromyographic signal, utilize the features training BP neutral net extracted, and profit
User operation is identified by the BP neutral net obtained with training.
5. system as claimed in claim 4, wherein, described feature includes integrated absolute, difference
Point absolute value, zero passage are counted, the group of one or more in mean square deviation, median frequency and average frequency
Close.
6. the system as described in claim 4 or 5, wherein, described control device is to receiving flesh
The signal of telecommunication extracts feature after pre-processing, and described pretreatment includes differential amplification and Butterworth filtering.
7. a human-computer interaction device for blood vessel intervention operation wire feeder based on surface myoelectric, its
Including:
Electromyographic signal collection module, electromyographic signal pretreatment module, electromyographic signal characteristic extracting module,
Action recognition module and control module, wherein:
Described electromyographic signal collection module is used for gathering doctor and carries out operation during blood vessel intervention operation
The actively electromyographic signal of hand contraction of muscle during seal wire;
Described electromyographic signal pretreatment module is for carrying out differential amplification and bar to the electromyographic signal gathered
Special Butterworth filtering;
Described electromyographic signal characteristic extracting module is for extracting the feature of the electromyographic signal through pretreatment;
Described action recognition module is used for training BP neutral net, and utilizes the BP trained neural
Hand motion when network carries out described operation to doctor is identified;
Described control module, for the action gone out according to described BP neural network recognization, sends corresponding
Control instruction is to wire feeder, to control the corresponding actions of driven seal wire.
Human-computer interaction device the most according to claim 7, it is characterised in that described myoelectricity is believed
Number include the electromyographic signal of short abductor muscle of thumb and the bicipital muscle of arm.
Human-computer interaction device the most according to claim 7, it is characterised in that described difference is put
It is used for greatly amplifying electromyographic signal, and removes the impact of dc source noise;The filtering of described Butterworth is used
In the high frequency in electromyographic signal, low frequency, 50Hz power frequency component are filtered, electromyographic signal limiting frequency
Scope is between 10-500Hz.
Device the most according to claim 7, it is characterised in that
The learning algorithm of BP neutral net includes propagated forward and backpropagation;Described propagated forward is used
Calculate the output of BP neutral net;Described backpropagation uses gradient descent method, is used for adjusting BP
The weights of each interlayer in neutral net.
11. devices according to claim 7, it is characterised in that described BP neutral net bag
Including input layer, intermediate layer and output layer, learning algorithm uses non-linear tansig function and linear purelin
Function is respectively as intermediate layer and the transfer function of output layer.
The man-machine interaction method of 12. 1 kinds of blood vessel intervention operation wire feeders based on surface myoelectric, its
Comprise the following steps:
Step 1: be attached at short abductor muscle of thumb by three piece of first electrode slice, is attached to three piece of second electrode slice
At the bicipital muscle of arm, three piece of first electrode slice connect respectively the positive pole in electromyographic signal collection instrument first passage,
Negative pole and ground, three piece of second electrode slice just connects in electromyographic signal collection instrument second channel respectively
Pole, negative pole and ground;
Step: 2: operator carries out rotating, stop and pushing three kinds of actions with operation by human hand actively seal wire
Each a period of time, first passage described in electromyographic signal collection instrument acquisition operations person and the flesh on second channel
The signal of telecommunication;
Step 3: the forward part of described electromyographic signal is gathered data and is correlated with as training data, extraction
Feature, and train BP neutral net;
Step 4: the rear section of described electromyographic signal gathers the BP neutral net that data test trains,
When described BP neutral net rate of accuracy reached is to predetermined threshold, enters next step, otherwise return step
3:
Step 5: operator with operation by human hand actively seal wire electromyographic signal collection instrument gather first passage and
Feature extraction is carried out after electromyographic signal in second channel, then as the input of BP neutral net,
The actuating signal of described operator is identified by BP neutral net according to described input, and exports control
System instruction is to wire feeder;
Step 6: when wire feeder receives rotation instruction, wire feeder controls radial stepping motor
Motion, it is achieved the rotation of driven seal wire;When wire feeder receives propelling movement instruction, wire feeder control
Make the motion of axial stepper motor, it is achieved the propelling movement of driven seal wire;When wire feeder receives halt instruction
After, the radial stepping motor of wire feeder control and the equal stop motion of axial stepper motor.
The man-machine interaction method of 13. 1 kinds of blood vessel intervention operation wire feeders based on surface myoelectric, its
Comprise the following steps:
Gather hand contraction of muscle when operating active seal wire during doctor carries out blood vessel intervention operation
Electromyographic signal;
The electromyographic signal gathered is carried out differential amplification and Butterworth filtering;
Extract the feature through the electromyographic signal pre-processed;
Utilize the features training BP neutral net extracted, and utilize the BP neutral net trained
Hand motion when doctor carries out described operation is identified;
The action gone out according to described BP neural network recognization, sends corresponding control instruction to wire-feed motor
Structure, to control the corresponding actions of driven seal wire.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610266180.7A CN105892676B (en) | 2016-04-26 | 2016-04-26 | Human-computer interaction device, system and method of wire feeding mechanism for vascular intervention operation |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610266180.7A CN105892676B (en) | 2016-04-26 | 2016-04-26 | Human-computer interaction device, system and method of wire feeding mechanism for vascular intervention operation |
Publications (2)
Publication Number | Publication Date |
---|---|
CN105892676A true CN105892676A (en) | 2016-08-24 |
CN105892676B CN105892676B (en) | 2021-12-24 |
Family
ID=56705493
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201610266180.7A Active CN105892676B (en) | 2016-04-26 | 2016-04-26 | Human-computer interaction device, system and method of wire feeding mechanism for vascular intervention operation |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN105892676B (en) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108305522A (en) * | 2018-04-09 | 2018-07-20 | 西南石油大学 | A kind of training equipment for blood vessel intervention operation operation guide |
CN108388855A (en) * | 2018-02-11 | 2018-08-10 | 宁波工程学院 | A kind of pin electrode EMG Signal Decomposition Based method |
CN110151177A (en) * | 2019-05-28 | 2019-08-23 | 长春理工大学 | Drop foot detection device and detection method based on surface electromyogram signal |
CN112017516A (en) * | 2020-08-26 | 2020-12-01 | 北京理工大学 | Remote vascular intervention operation training system |
CN112807001A (en) * | 2019-11-15 | 2021-05-18 | 上海中研久弋科技有限公司 | Multi-modal intent recognition and motion prediction method, system, terminal, and medium |
CN113545855A (en) * | 2021-05-31 | 2021-10-26 | 中国科学院自动化研究所 | Force detection system and method applied to vascular interventional operation |
CN112807001B (en) * | 2019-11-15 | 2024-06-04 | 上海中研久弋科技有限公司 | Multi-step intention recognition and motion prediction method, system, terminal and medium |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103861290A (en) * | 2014-02-27 | 2014-06-18 | 常州大学 | Myoelectricity remote control toy aircraft and control method thereof |
CN105511600A (en) * | 2015-07-31 | 2016-04-20 | 华南理工大学 | Multi-media man-machine interaction platform based on mixed reality |
-
2016
- 2016-04-26 CN CN201610266180.7A patent/CN105892676B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103861290A (en) * | 2014-02-27 | 2014-06-18 | 常州大学 | Myoelectricity remote control toy aircraft and control method thereof |
CN105511600A (en) * | 2015-07-31 | 2016-04-20 | 华南理工大学 | Multi-media man-machine interaction platform based on mixed reality |
Non-Patent Citations (3)
Title |
---|
丁其川 等: "基于表面肌电的运动意图识别方法研究及应用综述", 《自动化学报》 * |
奉振球 等: "微创血管介入手术机器人系统的设计与控制", 《PROCEEDINGS OF THE 32ND CHINESE CONTROL CONFERENCE》 * |
张旭: "基于表面肌电信号的人体动作识别与交互", 《中国优秀硕士论文全文数据库 信息科技辑》 * |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108388855A (en) * | 2018-02-11 | 2018-08-10 | 宁波工程学院 | A kind of pin electrode EMG Signal Decomposition Based method |
CN108388855B (en) * | 2018-02-11 | 2022-02-11 | 宁波工程学院 | Needle electrode electromyographic signal decomposition method |
CN108305522A (en) * | 2018-04-09 | 2018-07-20 | 西南石油大学 | A kind of training equipment for blood vessel intervention operation operation guide |
CN108305522B (en) * | 2018-04-09 | 2023-09-01 | 西南石油大学 | Training equipment for guiding vascular interventional operation |
CN110151177A (en) * | 2019-05-28 | 2019-08-23 | 长春理工大学 | Drop foot detection device and detection method based on surface electromyogram signal |
CN112807001A (en) * | 2019-11-15 | 2021-05-18 | 上海中研久弋科技有限公司 | Multi-modal intent recognition and motion prediction method, system, terminal, and medium |
CN112807001B (en) * | 2019-11-15 | 2024-06-04 | 上海中研久弋科技有限公司 | Multi-step intention recognition and motion prediction method, system, terminal and medium |
CN112017516A (en) * | 2020-08-26 | 2020-12-01 | 北京理工大学 | Remote vascular intervention operation training system |
CN113545855A (en) * | 2021-05-31 | 2021-10-26 | 中国科学院自动化研究所 | Force detection system and method applied to vascular interventional operation |
CN113545855B (en) * | 2021-05-31 | 2022-12-06 | 中国科学院自动化研究所 | Force detection system and method applied to vascular interventional operation |
Also Published As
Publication number | Publication date |
---|---|
CN105892676B (en) | 2021-12-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Alazrai et al. | A deep learning framework for decoding motor imagery tasks of the same hand using EEG signals | |
CN104382595B (en) | Upper limb rehabilitation system and method based on myoelectric signal and virtual reality interaction technology | |
CN107378944B (en) | Multidimensional surface electromyographic signal artificial hand control method based on principal component analysis method | |
CN105892676A (en) | Human-machine interaction device, system and method of vascular intervention operation wire feeder | |
CN104107134B (en) | Upper limbs training method and system based on EMG feedback | |
Wang et al. | Deep back propagation–long short-term memory network based upper-limb sEMG signal classification for automated rehabilitation | |
CN104000586A (en) | Stroke patient rehabilitation training system and method based on brain myoelectricity and virtual scene | |
Too et al. | A detail study of wavelet families for EMG pattern recognition | |
CN104207793A (en) | Gripping function evaluating and training system | |
CN111584030A (en) | Idea control intelligent rehabilitation system based on deep learning and complex network and application | |
Ye et al. | Optimal feature selection for EMG-based finger force estimation using LightGBM model | |
CN111584031B (en) | Brain-controlled intelligent limb rehabilitation system based on portable electroencephalogram acquisition equipment and application | |
CN108743223A (en) | A kind of the intelligent exoskeleton manipulators in rehabilitation and method automatically controlled based on strong side biology | |
Li et al. | EEG signal classification method based on feature priority analysis and CNN | |
CN112732090B (en) | Muscle cooperation-based user-independent real-time gesture recognition method | |
Li et al. | Wireless sEMG-based identification in a virtual reality environment | |
CN114206292A (en) | Hand function rehabilitation device with intention perception function | |
Guo et al. | Lw‐CNN‐Based Myoelectric Signal Recognition and Real‐Time Control of Robotic Arm for Upper‐Limb Rehabilitation | |
Yu et al. | Wrist torque estimation via electromyographic motor unit decomposition and image reconstruction | |
Guo et al. | Towards semi-supervised myoelectric finger motion recognition based on spatial motor units activation | |
CN113730190A (en) | Upper limb rehabilitation robot system with three-dimensional space motion | |
Amrani et al. | Artificial neural networks based myoelectric control system for automatic assistance in hand rehabilitation | |
CN112998725A (en) | Rehabilitation method and system of brain-computer interface technology based on motion observation | |
CN110705656A (en) | Facial action recognition method based on EEG sensor | |
CN116562392A (en) | Vascular intervention operation action sensing and learning system, method and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
TA01 | Transfer of patent application right |
Effective date of registration: 20190524 Address after: 100095 Building 1, Unit 1, 101A, No. 14, No. 3 Courtyard, Gaolizhang Road, Haidian District, Beijing Applicant after: Beijing Zhongke Hongtai Medical Technology Co., Ltd. Address before: 100190 Zhongguancun East Road, Haidian District, Haidian District, Beijing Applicant before: Institute of Automation, Chinese Academy of Sciences |
|
TA01 | Transfer of patent application right | ||
GR01 | Patent grant | ||
GR01 | Patent grant |