CN108646726A - The wheelchair control system of wheelchair control method and combination voice based on brain wave - Google Patents
The wheelchair control system of wheelchair control method and combination voice based on brain wave Download PDFInfo
- Publication number
- CN108646726A CN108646726A CN201810290117.6A CN201810290117A CN108646726A CN 108646726 A CN108646726 A CN 108646726A CN 201810290117 A CN201810290117 A CN 201810290117A CN 108646726 A CN108646726 A CN 108646726A
- Authority
- CN
- China
- Prior art keywords
- wheelchair
- module
- brain wave
- signal
- voice
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/0088—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS], computer integrated manufacturing [CIM]
- G05B19/4185—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS], computer integrated manufacturing [CIM] characterised by the network communication
- G05B19/41855—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS], computer integrated manufacturing [CIM] characterised by the network communication by local area network [LAN], network structure
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS], computer integrated manufacturing [CIM]
- G05B19/4189—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS], computer integrated manufacturing [CIM] characterised by the transport system
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0238—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using obstacle or wall sensors
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0242—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using non-visible light signals, e.g. IR or UV signals
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D13/00—Control of linear speed; Control of angular speed; Control of acceleration or deceleration, e.g. of a prime mover
- G05D13/62—Control of linear speed; Control of angular speed; Control of acceleration or deceleration, e.g. of a prime mover characterised by the use of electric means, e.g. use of a tachometric dynamo, use of a transducer converting an electric value into a displacement
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/20—Pc systems
- G05B2219/26—Pc applications
- G05B2219/2637—Vehicle, car, auto, wheelchair
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Abstract
The invention discloses the wheelchair control systems of wheelchair control method and combination voice based on brain wave, and the brain wave of acquisition by being filtered, being converted into wheelchair control signal controling wheelchair action after feature extraction and Modulation recognition by wheelchair control method；Wheelchair control system includes wheelchair traveling motor, processor module, voice module, motor drive module, obstacle detection module, road conditions detection module, the first communication module group；Further include brain wave collector and terminating machine control device；Brain wave detection device includes acquiring brain waves processing module and the second communication module；Terminal device includes mobile phone and server.The invention enables wheelchair users can pass through brain wave, voice or manual mode controling wheelchair action, the wheelchair control of wheelchair user can be realized and be converted into voice messaging, and barrier and road conditions detection are carried out in wheelchair traveling process and is reported, to improve the automating of wheelchair control, intelligent, safety and reliability.
Description
Technical field
The invention belongs to the means of transport fields suitable for patient or disabled person, more particularly to the wheelchair control based on brain wave
Method processed and the wheelchair control system for combining voice.
Background technology
Wheelchair is the important trip tool of handicapped the elderly, patient or disabled person, for above-mentioned crowd, very much
In the case of, there is only walking disorders, while hand can not well control the action of wheelchair, or simple use
The action of hand controling wheelchair is very painstaking.So, to realize that travel requirement just needs the help by other people, to drop
Low trip degree of freedom.
Invention content
The technical problem to be solved by the present invention is to provide the wheelchair control method based on brain wave and combine voice
Wheelchair control system, using Internet of Things and intelligent control technology, wheelchair user is controlled by brain wave, voice or manual mode
Wheelchair is advanced, while can be realized the wheelchair control of wheelchair user and be converted into voice messaging, and in wheelchair traveling process
It middle progress barrier and road conditions detection and reports, to improve the automating of wheelchair control, intelligent, safety and reliable
Property.
In order to solve the above technical problems, the present invention is based on the technical solutions of the wheelchair control method of brain wave to be：
Wheelchair control method based on brain wave, includes the following steps：
S1：The eeg signal for acquiring wheelchair user, filtering out will be described after ambient noise and electric power signal interference
Eeg signal is converted to brain wave data signal；
S2：Using average filter algorithm to through step S1, treated that brain wave data signal is filtered；
S3：Feature extraction is carried out to brain wave data signal after step S2 processing using period map method；
S4：Using linear discriminant analysis to carrying out Modulation recognition through step S3 treated brain wave data signals；
S5：It will treated that brain wave data signal is converted to wheelchair control signal through step S4.
In the above-mentioned technical solutions, the original eeg signal for acquiring wheelchair user, filters out ambient noise and electricity
Force signal interferes, convenient for being converted into being further processed after brain wave data signal；Using average value filtering algorithm to brain
Wave data signal is filtered, and can improve calculation amount and EMS memory occupation, and when sampling number is larger, smoothness is good；Using line
Property discriminant analysis to brain wave data signal carry out Modulation recognition, according to different movements or consciousness can be electrical activity of brain production
Relationship between type and characteristic signal of the characteristic of raw different responses to determine movement or consciousness；Just eeg data is believed later
Number it is converted into wheelchair control signal, to make wheelchair take action according to the movement or consciousness of user.The technical program profit
With the eeg signal of wheelchair user can controling wheelchair action, improve the automation and intelligence to wheelchair control.
Preferably, according to the gait of march of the focus controling wheelchair of the wheelchair user, this mode can be more
It is accurate and quickly the speed of wheelchair is controlled according to the idea of wheelchair user, improve using brain wave to wheelchair into
The intelligence of row control and control effect and comfort level.
In order to solve the above technical problems, technical solution used by the wheelchair control system of brain wave combination voice of the present invention
It is：
The wheelchair control system of brain wave combination voice, including the wheelchair traveling motor, the processor die that are set on wheelchair
Block and the voice module being connect respectively with the processor module, motor drive module, obstacle detection module, road conditions detect mould
Block and the first communication module group, the motor drive module are connected with the wheelchair traveling motor simultaneously, the wheelchair row
Stepper motor is connect with wheelchair road wheel simultaneously；Further include brain wave collector and terminating machine control device；The brain wave
Harvester include acquiring brain waves module acquiring brain waves module 8 and with the acquiring brain waves module acquiring brain waves mould
The second communication module that block 8 connects；The terminal device includes mobile phone and the server that is connect with the mobile phone wireless, described
Application software for wheelchair control is installed, the server is used to be used to control by what the application software received on mobile phone
The voice messaging of wheelchair is then transmit to the mobile phone into processing of racking and forms specific instruction on the mobile phone；Described first
Communication module group can respectively be communicated wirelessly with second communication module and the mobile phone.
In the above-mentioned technical solutions, can acquire wheelchair by acquiring brain waves module acquiring brain waves module 8 first makes
The eeg signal of user filters out ambient noise and electric power signal interference, eeg signal is then converted to brain wave
Data-signal, brain wave data signal is transmitted to the first communication module group by the second communication module, and then is transferred to processor
Module, processor module is filtered brain wave data signal using average filter algorithm, using period map method to brain wave
Data-signal carries out feature extraction, carries out Modulation recognition to brain wave data signal using linear discriminant analysis, finally by brain electricity
Wave data-signal is converted into corresponding module control signal and is transmitted to motor drive module or voice module, and motor drives mould
Root tuber is according to the working order of the control signal controling wheelchair traveling motor received, the road wheel of wheelchair traveling motor controling wheelchair
Action, that is, the action of controling wheelchair just realizes in this way to the control of the eeg signal of wheelchair, and voice module is adopted
The control signal received is converted into voice signal with phonetic synthesis and is reported, is also equivalent to wheelchair user couple
The control idea of wheelchair is converted into voice messaging, also allows for other people and understands the idea of wheelchair user, especially for action
With speech with difficult wheelchair user；Secondly, voice module is also integrated with speech identifying function, and wheelchair user can be straight
It connects to voice module, the voice signal of wheelchair user is converted to command adapted thereto by voice module, then these instruction quilts
It is transmitted to processor module, processor module is converted into corresponding control signal after carrying out verification and analyzing processing to these instructions
And it is transmitted to motor drive module, by the working order of motor drive module controling wheelchair traveling motor；Again, wheelchair user
The wheelchair control application software of mobile phone can be opened and then say wheelchair control voice command, wheelchair control voice command is transmitted
Local handset is returned after to server into processing of racking and generates spy corresponding with wheelchair control voice command in local handset
Fixed instruction, these specific instructions are transferred to the first communication module group, and then are transferred to processor module, processor module pair
These specific instructions are converted into corresponding control signal after carrying out verification and analyzing processing and are transmitted to motor drive module, by electricity
The working order of machine drive module controling wheelchair traveling motor；Wheelchair user can also by press different mobile phone keys come
The instruction of corresponding wheelchair is triggered, likewise, the instruction of these wheelchairs is transferred to server into returning to local handset after processing of racking
And corresponding specific instruction is generated in local handset, these specific instructions are transferred to the first communication module group, and then are passed
Processor module is transported to, processor module is converted into corresponding control letter after carrying out verification and analyzing processing to these specific instructions
Number and be transmitted to motor drive module, by the working order of motor drive module controling wheelchair traveling motor；Finally, in view of wheelchair
Various barriers and complex road condition may be faced in traveling process, the technical program is provided with obstacle detection module and road conditions
Detection module sends a signal to processor module when the detection of obstacle detection module takes turns to when chair direction of travel has barrier, handles
Device module sends out control signal to motor drive module and voice module, and then voice module carries out barrier report；Work as road conditions
When poor, road conditions detection module sends a signal to processor module, and processor module sends out control signal to voice module, in turn
Voice module carries out road conditions prompting.The technical program combination Internet of Things and intelligent control technology, wheelchair user can pass through brain
Electric wave, voice or the action of modes controling wheelchair such as manually, for the inconvenient crowd that takes action or talk all than more easily making
With, and when certain control modes failures wherein or mistake, other control modes can also be used to carry out
Again it controls or corrects, while the wheelchair control of wheelchair user can be realized and be converted into voice messaging, and can be
Carry out barrier and road conditions detection in wheelchair traveling process and report, to improve the automating of wheelchair control, it is intelligent with
And safety；The technical program is based on modular thought, by entire wheelchair intelligent control on the basis of system function is planned
System is divided into several modules, and each module is mutual indepedent, clear in structure, and program is brief and concise, changes and safeguards convenient for the later stage, can
Production cost is reduced, production efficiency is improved.
As being further improved for technical solution of the present invention, the acquiring brain waves module acquiring brain waves module 8 includes
Brain wave sensor, the brain wave sensor are connect with second communication module, and brain wave sensor can detect pole
Faint EEG signals indicate convenient for the accurate idea for obtaining user, while can filter out ambient noise and electric power letter
Number interference, EEG signals are further processed convenient for processor module.
As being further improved for technical solution of the present invention, the obstacle detection module includes sequentially connected infrared ray two
Pole pipe, infrared receiving tube, integrated chip, voltage comparator and LED light tube, obstacle detection module pass through with processor module
Conventional means is connected to realize that the information data contact of the two, obstacle detection module use the infrared barrier of anti-interference type high-performance
Detection module, obstacle detection module is hindered to export low and high level signal according to there is clear, processor module hinders by detecting
The low and high level signal that detection module exports is hindered to send out control signal automatically, the travel condition of one side controling wheelchair is another
Aspect control voice module sends out prompting, to further increase the safety in utilization of wheelchair.
As being further improved for technical solution of the present invention, the road conditions detection module includes connecting with the processor module
The shock sensor connect, the shock sensor export low and high level signal according to pavement behavior, are transmitted to after analog-to-digital conversion
Processor module, processor module detect whether road conditions environment causes wheelchair largely by the way that failing edge detection interruption is arranged
Vibrations, and send out after vibrations reache a certain level control signal control voice module and send out prompting, further improve wheel
The safety in utilization of chair.
To sum up, the present invention is based on the wheelchair control systems of the wheelchair control method of brain wave and combination voice to improve
The automating of wheelchair control, intelligent, safety and reliability help handicapped crowd to realize that " body is dynamic with brain, says away just
Walk " target；And it changes and safeguards convenient for the later stage, production cost can be reduced, improve production efficiency.
Description of the drawings
Fig. 1 is that people normally blinks brain wave signal.
Fig. 2 is that people normally grits one's teeth brain wave signal.
Fig. 3 be people continuously blink in 0.5 second twice caused by brain wave signal.
Fig. 4 is focus brain wave signal.
Fig. 5 is motor-drive circuit figure.
Fig. 6 is the logi function chart of wheelchair.
Fig. 7 is the schematic diagram of wheelchair control system of the present invention.
In figure：1, processor module；2, voice module；3, motor drive module；4, obstacle detection module；5, road conditions detection
Module；6, the first communication module group；7, the second communication module；8, acquiring brain waves module；9, mobile phone；10, server.
Specific implementation mode
The present invention provides the wheelchair control methods based on brain wave, include the following steps：
S1：Acquire wheelchair user eeg signal, filter out ambient noise and electric power signal interference after by brain electricity
Wave signal is converted to brain wave data signal；In the signal of potentials extraction, 12 kinds of different EEG signals are shared, including
The original E.E.Gs of RAW WAVE, Delta, Theta, Low Alpha, High Alpha, Low Beta, High Beta, Low
Gamma, Mid Gamma, Noise, Attention, Mediation, in this 12 kinds of different eeg signals, with RAW
The brain wave signal of the channel WAVE, Attention acquisition handles the first choice of signal for E.E.G as reference signal；
S2：Using average filter algorithm to through step S1, treated that brain wave data signal is filtered；
Sampled value C, accumulator S, average value A, sampling number N are defined,
Initialize A=initial values, S=A*N, S=S-A+C, average value A=S/N；
The thought of this algorithm is average value filtering, improves operand and EMS memory occupation, smoothness is good when N values are larger；
S3：Feature extraction is carried out to brain wave data signal after step S2 processing using period map method；
Feature is extracted with period map method, is not required to calculate auto-correlation function, and is directly obtained by Fourier transformation.
The N point gathered data xN (n) of random signal x (n) are considered as a finite energy signal, and the Fourier transformation of xN (n) is directly taken to obtain xN
(ejw), then take square of its amplitude, and divided by N be spaced and take on w as the estimation to x (n) real powers spectrum P (ejw)
Value, obtains
Pper (k)=1/N | XN (k) | 2
Wherein Pper (k) is expressed as the power spectrum estimated with periodic method；
It gets electroencephalogram power spectrum density and finds characteristic value and analyzed, different EEG signals are indicated by feature vector
Corresponding different conditions；
S4：Using linear discriminant analysis to carrying out Modulation recognition through step S3 treated brain wave data signals；
Characteristic signal classification be according to different movements or consciousness can make electrical activity of brain generate the characteristic of different responses come
Determine the relationship between movement or the type and characteristic signal of consciousness；Make similar sample as poly- as possible using linear discriminant analysis
It gathers together, inhomogeneity sample as far as possible, by given test data, determines projecting direction W and threshold value y0, that is, determines line
Property discriminant function tests test data then according to this linear discriminant function, finally obtains the class of test data
Not；
Wherein, the method for determination of W is：
Different categories of samples mean vector mi
Dispersion matrix between sample classes Si and total within class scatter matrix Sw
Sw=S1+S2
Matrix between samples Sb=(m1-m2) (m1-m2) T
In the one-dimensional space in the projected, Different categories of samples mean value mi '=WTmi；It is discrete in within-class scatter and total class
Spend Si '=WTSiWSWW；Between-class scatter Sb '=WTSbW.
Two critical natures of linear discriminant analysis are：
I. as intensive as possible inside Different categories of samples, i.e., total within-cluster variance is the smaller the better.
II. as far as possible far away, i.e., between-class scatter is the bigger the better Different categories of samples.
Criterion function is determined according to this property, according to making criterion function obtain maximum value, can find out W:
W=Sw-1 (m1-m2)
The method of determination of threshold value y0 is：
Y0=(m1 '+m2 ')/2
The decision rule of linear discriminant analysis
For the sample vector x of some unknown classification, if y=WTx>Y0, then x ∈ w1；Otherwise x ∈ w2；It carries out again
Normalized is completed at the same time sample classification it may determine that go out the classification of test data；
S5：It will treated that brain wave data signal is converted to wheelchair control signal through step S4.
In above-mentioned wheelchair control method, according to the gait of march of the focus controling wheelchair of wheelchair user, focus
Height can directly be obtained from wheelchair control signal.
The above-mentioned wheelchair control method based on brain wave can controling wheelchair row using the eeg signal of wheelchair user
It is dynamic, for example, normal signal of blinking will not generate wheelchair control signal to wheelchair, in a short time, such as continuously blinked in 0.5s
Eye then can open the switch of wheelchair advance with controling wheelchair direction of travel by gritting one's teeth twice；In wheelchair traveling process, root
According to the speed that the focus controling wheelchair of user is advanced, focus is higher, and gait of march is faster；People normally blinks brain wave signal
As shown in Figure 1.People normally grit one's teeth brain wave signal as shown in Fig. 2, people continuously blink in 0.5 second twice caused by brain wave signal
As shown in Figure 3；Focus brain wave signal is as shown in Figure 4.
As shown in fig. 7, the wheelchair control system that brain wave of the present invention and voice combine includes wheelchair traveling motor, processor
Module 1 and the voice module 2 being connect respectively with processor module 1, motor drive module 3, obstacle detection module 4, road conditions inspection
Module 5 and the first communication module group 6, above-mentioned component or module is surveyed to may be contained on wheelchair, motor drive module 3 simultaneously with
Wheelchair traveling motor is connected, and wheelchair traveling motor is connect with wheelchair road wheel simultaneously；Further include brain wave collector and
Terminating machine control device；Brain wave collector include acquiring brain waves module 8 and connect with acquiring brain waves module 8
Two communication modules 7；Terminal device includes mobile phone 9 and the server 10 that is wirelessly connected with mobile phone 9, is equipped with and is used on mobile phone 9
The application software of wheelchair control, server 10 are used to locate the voice messaging for controling wheelchair that application software receives into racking
Reason is then transmit to mobile phone 9 and forms specific instruction on mobile phone 9；First communication module group 6 can be with the second communication module 7
It is communicated wirelessly respectively with mobile phone 9.
The wheelchair control system of brain wave combination voice of the present invention is specifically described below by specific embodiment：
The wheelchair control system of the brain wave combination voice includes wheelchair traveling motor, the processor die being set on wheelchair
Block 1 and the voice module 2 being connect respectively with processor module 1, motor drive module 3, obstacle detection module 4, road conditions detection
Module 5 and the first communication module group 6, motor drive module 3 are connected with wheelchair traveling motor simultaneously, and wheelchair traveling motor is same
When connect with wheelchair road wheel.
Wherein voice module 2 is for carrying out phonetic synthesis, speech recognition, voice broadcast and receiving wheelchair voice control letter
Breath uses XFS5152 voice synthetic modules, it can be achieved that Chinese, English phonetic synthesis, are also innovatively integrated with speech recognition
Function supports the identification of 30 order words；Processor module 1 is communicated by serial ports with voice module 2.
Wheelchair traveling motor includes two direct current generators, is connect respectively with two road wheels of wheelchair, for driving wheelchair
Road wheel is run；Motor drive module 3 is used for the operating of controling wheelchair traveling motor, has been internally integrated ambipolar H- bridges electricity
Road generates enable signal by internal logic；The input quantity of H- bridge circuits can be used for that motor rotation direction, enable signal is arranged
It can be used for pulse-width adjustment (PWM)；Each motor needs 3 controls signal ENA/B, IN1, IN2, and wherein ENA/B is enabled letter
Number, IN1, IN2 are that direction of motor rotation controls signal, and when IN1, IN2 are respectively 1,0, motor rotates forward, conversely, motor reversal；Choosing
With the connection ENA/B pins of PWM all the way, the rotating speed of motor can be adjusted by adjusting the duty ratio of PWM；Processor module 1 is sent out
Corresponding pwm waveform is sent to control 3 chip of motor drive module, driving chip is controlled by the closure to relevant pins again
Make the movement of two direct current generators；Motor-drive circuit figure is as shown in figure 5, the logi function chart of wheelchair is as shown in Figure 6.
4 module of obstacle detection module uses the infrared obstacle detection of anti-interference type high-performance, wraps for carrying out obstacle detection
Sequentially connected infrared diode, infrared receiving tube, integrated chip, voltage comparator and LED light tube are included, when front has
When barrier, infrared diode sends out signal, is received back through infrared receiving tube, and integrated chip amplification, voltage comparator compares
Afterwards, light the LED light tube in module, and export low level, processor module 1 by detect low level signal input come to
Motor drive module 3 and voice module 2 export corresponding control signal to the traveling of controling wheelchair and carry out voice reminder.
Road conditions detection module 5 is for carrying out road conditions detection, using SW-1801P shock sensors, when road conditions are steady,
Vibration switch is in off-state, and output end exports high level, and vibrations indicator light does not work；Lead to sensor vibration when road conditions are bad
When, vibration switch transient switching, vibrations indicator light is bright, output end exports low level signal；Processor module 1 is declined by being arranged
It is interrupted along detection to detect whether road conditions environment causes wheelchair largely to shake, when vibration signal generation, processor module 1
Corresponding control signal is exported to voice module 2, voice module 2 prompts user to need to pay attention to current road conditions, takes in a reef.
Brain wave collector includes acquiring brain waves module 8 and the second communication being connect with acquiring brain waves module 8
Module 7.
Acquiring brain waves module 8 selects ThinkGear TGAM brains for brain wave to be acquired and pre-processed
Radio wave sensor can detect the original EEG signals of 512Hz from brain, pre-process, that is, filter to original EEG signals
Brain wave data signal is converted to after other noises and electric power signal interference around falling, brain wave data signal is by the second communication mould
Block 7 is transmitted to the first communication module group 6, and then is transferred to processor module 1；Due to the ad hoc network characteristic of bluetooth, in this reality
It applies in example, the communication work between the first communication module group 6 and the second communication module 7, the first communication is realized using Bluetooth technology
Module group 6 includes bluetooth module, and bluetooth module uses bluetooth skill as main equipment connection processing device module 1, the second communication module 7
Art, as slave connection acquiring brain waves module 8, the bluetooth module and second in the first communication module group 6 communicates mould
Block 7 forms piconet (Piconet)；Acquiring brain waves module 8 and the second communication module 7 can be integrated in an acquiring brain waves
Cap.
Terminal device includes mobile phone 9 and the server 10 that is wirelessly connected with mobile phone 9, and mobile phone 9 is for receiving wheelchair voice
It controls information and wheelchair is manually controlled, the application software for wheelchair control is installed, in application software on mobile phone 9
Interface, server 10 are used to the voice messaging for controling wheelchair that mobile phone 9 receives being then transmit to mobile phone 9 into processing of racking
And form specific instruction on mobile phone 9；These specific instructions are transferred to the first communication module group 6, and then are transferred to processing
Device module 1；In the present embodiment, it is communicated using WiFi between the first communication module group 6 and mobile phone 9, the first communication module
Group 6 further includes WiFi modules, and WiFi module is selected by serial ports connection processing device module 1, WIFI module based on UART interface
WIFI wireless network modules can realize user's serial data to the exchange between wireless network；By serial ports WiFi module,
Traditional serial equipment can access wireless network；It is communicated between WiFi module and mobile phone 9, uses ASCII character, mobile phone 9
Coding is become ASCII character to be emitted, UART-WIFI modules can receive ASCII character and generate interruption to processor module 1；
This WIFI module can be arranged as ordinary router by terminal and Modify password, can also be repaiied by configuration software
Change and be arranged, there is the security performance that comparison is pretty good.
For the wheelchair control system of the brain wave combination voice, can acquire wheelchair by acquiring brain waves module 8 makes
The original EEG signals of 512Hz of user filter out ambient noise and electric power signal interference, are then converted to eeg signal
Brain wave data signal, brain wave data signal is transmitted to the first communication module group 6 by the second communication module 7, and then is transmitted
To processor module 1, TGAM chips send 513 packets about each second, and transmission is surrounded by parcel and big two kinds of packet：Parcel
Format is 04 80 02 High of AA AA, Low, (the AA AA 04 80 02 of front are frame heads to Check Sum, rear three words
Section be data and verification and) High and Low constitute initial data raw_data, Check Sum be verification and.
One group of initial data is got, it first should be first to verifying and testing.Formula：
Sum=((0x80+0x02+High+Low) ^0xFFFFFFFF)s ＆0xFF
If sum and Check Sum are equal, that illustrates that this packet is correct, then goes to calculate raw_data again,
Otherwise directly ignore this packet.
Parcel analytic approach parses initial data：
Raw_data=(High<<8)|Low；
if(raw_data>32768) { raw_data=65536；}
Processor module 1 is used as the information processing platform, is handled brain wave data signal using processor module 1,
MSP430F5438A may be used in its model, is arranged and integrates peripheral hardware：Analog comparator, DMA, hardware multiplier, SVS (power supply electricity
Press monitoring module), 12 DAC.
Processing is carried out using processor module 1 to brain wave data signal to include the following steps：
(1) brain wave data signal is filtered using average filter algorithm；
Sampled value C, accumulator S, average value A, sampling number N are defined,
Initialize A=initial values, S=A*N, S=S-A+C, average value A=S/N；
The thought of this algorithm is average value filtering, improves operand and EMS memory occupation, smoothness is good when N values are larger；
(2) use period map method to carrying out feature extraction through step (1) treated brain wave data signal；
Feature is extracted with period map method, is not required to calculate auto-correlation function, and is directly obtained by Fourier transformation.
The N point gathered data xN (n) of random signal x (n) are considered as a finite energy signal, and the Fourier transformation of xN (n) is directly taken to obtain xN
(ejw), then take square of its amplitude, and divided by N be spaced and take on w as the estimation to x (n) real powers spectrum P (ejw)
Value, obtains
Pper (k)=1/N | XN (k) | 2
Wherein Pper (k) is expressed as the power spectrum estimated with periodic method；
It gets electroencephalogram power spectrum density and finds characteristic value and analyzed, different EEG signals are indicated by feature vector
Corresponding different conditions；
(3) use linear discriminant analysis to carrying out Modulation recognition through step (2) treated brain wave data signal；
Characteristic signal classification be according to different movements or consciousness can make electrical activity of brain generate the characteristic of different responses come
Determine the relationship between movement or the type and characteristic signal of consciousness；Make similar sample as poly- as possible using linear discriminant analysis
It gathers together, inhomogeneity sample as far as possible, by given test data, determines projecting direction W and threshold value y0, that is, determines line
Property discriminant function tests test data then according to this linear discriminant function, finally obtains the class of test data
Not；
Wherein, the method for determination of W is：
Different categories of samples mean vector mi
Dispersion matrix between sample classes Si and total within class scatter matrix Sw
Sw=S1+S2
Matrix between samples Sb=(m1-m2) (m1-m2) T
In the one-dimensional space in the projected, Different categories of samples mean value mi '=WTmi；It is discrete in within-class scatter and total class
Spend Si '=WTSiWSWW；Between-class scatter Sb '=WTSbW.
Two critical natures of linear discriminant analysis are：
I. as intensive as possible inside Different categories of samples, i.e., total within-cluster variance is the smaller the better.
II. as far as possible far away, i.e., between-class scatter is the bigger the better Different categories of samples.
Criterion function is determined according to this property, according to making criterion function obtain maximum value, can find out W:
W=Sw-1 (m1-m2)
The method of determination of threshold value y0 is：
Y0=(m1 '+m2 ')/2
The decision rule of linear discriminant analysis
For the sample vector x of some unknown classification, if y=WTx>Y0, then x ∈ w1；Otherwise x ∈ w2；It carries out again
Normalized is completed at the same time sample classification it may determine that go out the classification of test data；
(4) will through step (3), treated that brain wave data signal is converted to module control signal, the module control signal
As wheelchair control signal, for the corresponding module in controling wheelchair control system.
Module control signal is transferred to motor drive module 3 or voice module 2, and motor drive module 3 is according to reception
The working order of the control signal controling wheelchair traveling motor arrived, wheelchair traveling motor are then connected with the road wheel of wheelchair, in this way
Just the eeg signal control to wheelchair is realized, for example, normal signal of blinking will not control signal to wheelchair generation module,
In a short time, such as in 0.5s continuously blinking twice then can be with controling wheelchair direction of travel, before opening wheelchair by gritting one's teeth
Into switch；In wheelchair traveling process, according to the speed that the focus controling wheelchair of user is advanced, focus is higher, row
It is faster into speed.
The control signal received is converted to voice signal and is reported by voice module 2 using phonetic synthesis, also
It is equivalent to and wheelchair user is converted into voice messaging to the control idea of wheelchair, also allow for other people and understand wheelchair user's
Idea, especially for action and speech with difficult wheelchair user.
Voice module 2 is also integrated with speech identifying function, and wheelchair user can directly pronounce to voice module 2, such as
User sends out the instructions such as "ON", "front", "rear", "left", "right", " fast ", " slow ", " stopping ", and voice module 2 is by wheelchair user
Voice signal be converted to command adapted thereto, then these instructions are transferred to processor module 1, and processor module 1 refers to these
Order is converted into corresponding control signal after carrying out verification and analyzing processing and is transmitted to motor drive module 3, and mould is driven by motor
The working order of 3 controling wheelchair traveling motor of block.
Wheelchair user can open the wheelchair control application software of mobile phone 9 and then say wheelchair control voice command, take turns
Chair control voice order is transferred to server 10 into return local handset 9 after processing of racking and generates and take turns in local handset 9
The corresponding specific instruction of chair control voice order, these specific instructions are transferred to the first communication module group 6, and then are transmitted
To processor module 1, processor module 1 is converted into corresponding control letter after carrying out verification and analyzing processing to these specific instructions
Number and be transmitted to motor drive module 3, by the working order of 3 controling wheelchair traveling motor of motor drive module；Wheelchair user
Corresponding wheelchair instruction can also be triggered by pressing different 9 buttons of mobile phone, likewise, the instruction of these wheelchairs is transferred to
Server 10 into return local handset 9 after processing of racking and generates corresponding specific instruction in local handset 9, these are specific
Instruction is transferred to the first communication module group 6, and then is transferred to processor module 1, and processor module 1 is to these specific fingers
Order is converted into corresponding control signal after carrying out verification and analyzing processing and is transmitted to motor drive module 3, and mould is driven by motor
The working order of 3 controling wheelchair traveling motor of block.
Processor module 1 is sent a signal to when chair direction of travel has barrier when the detection of obstacle detection module 4 takes turns to, is located
Reason device module 1 sends out control signal to motor drive module 3 and voice module 2, and then voice module 2 carries out barrier report；
When road conditions are poor, road conditions detection module 5 sends a signal to processor module 1, and processor module 1 sends out control to voice module 2
Signal processed, and then voice module 2 carries out road conditions prompting.
To ensure the uniqueness of user, security of system is improved, the wheelchair control application software in the present embodiment uses people
Face identification carries out system login, specifically uses regional characteristics analysis algorithm, by computer image processing technology and biology
Principle of Statistics is blended in one, extracts 21 characteristic points of portrait from video using computer image processing technology, passes through life
The statistical principle of object carries out analysis founding mathematical models and obtains skin detection.Meanwhile it is special using built face
The image surface for levying template and the people of measured carries out signature analysis, provides a similar value according to the result of analysis, passes through this value
It can be determined whether as same people；Wheelchair control application software in the present embodiment is developed using the prior art.
The either connection or communication between module of the various pieces of system in above-mentioned specific implementation mode and embodiment
The prior art, such as electrical connection or serial, parallel communication may be used in mode unless otherwise indicated, for above-mentioned specific implementation
Some technologies that the progress of mode may use, such as analog-to-digital conversion or digital-to-analogue conversion etc. can equally utilize the prior art,
It no longer elaborates in a specific embodiment.
Further instruction, but this hair have been carried out to the present invention above in conjunction with the drawings and specific embodiments and embodiment
It is bright to be not limited to the above-mentioned specific embodiments and embodiment, within the knowledge of a person skilled in the art, also
It can make a variety of changes without departing from the purpose of the present invention.
Claims (6)
1. the wheelchair control method based on brain wave, which is characterized in that include the following steps：
S1：Acquire wheelchair user eeg signal, filter out ambient noise and electric power signal interference after by the brain electricity
Wave signal is converted to brain wave data signal；
S2：Using average filter algorithm to through step S1, treated that brain wave data signal is filtered；
S3：Feature extraction is carried out to brain wave data signal after step S2 processing using period map method；
S4：Using linear discriminant analysis to carrying out Modulation recognition through step S3 treated brain wave data signals；
S5：It will treated that brain wave data signal is converted to wheelchair control signal through step S4.
2. the wheelchair control method according to claim 1 based on brain wave, which is characterized in that used according to the wheelchair
The gait of march of the focus controling wheelchair of person.
3. the wheelchair control system of brain wave combination voice, which is characterized in that including be set on wheelchair wheelchair traveling motor,
Processor module (1) and the voice module (2) being connect respectively with the processor module (1), motor drive module (3), barrier
Hinder detection module (4), road conditions detection module (5) and the first communication module group (6), the motor drive module (3) and meanwhile with
The wheelchair traveling motor is connected, and the wheelchair traveling motor is connect with wheelchair road wheel simultaneously；
Further include brain wave collector and terminating machine control device；
The brain wave collector includes acquiring brain waves module (8) and is connect with the acquiring brain waves module (8)
Second communication module (7)；
The terminal device includes mobile phone (9) and the server (10) that is wirelessly connected with the mobile phone (9), the mobile phone (9)
On application software for wheelchair control is installed, the server (10) is used to be used to control by what the mobile phone (9) received
The voice messaging of wheelchair is then transmit to the mobile phone (9) into processing of racking and forms specific instruction on the mobile phone (9)；
The first communication module group (6) can carry out channel radio respectively with second communication module (7) and the mobile phone (9)
News.
4. the wheelchair control system of brain wave combination voice according to claim 3, which is characterized in that the brain wave is adopted
It includes brain wave sensor to collect module (8), and the brain wave sensor is connect with second communication module (7).
5. the wheelchair control system of brain wave combination voice according to claim 3, which is characterized in that the obstacle detection
Module (4) includes that sequentially connected infrared diode, infrared receiving tube, integrated chip, voltage comparator and LED shine
Pipe.
6. the wheelchair control system of brain wave combination voice according to claim 3, which is characterized in that the road conditions detection
Module (5) includes the shock sensor being connect with the processor module (1), and the shock sensor is exported according to pavement behavior
Low and high level signal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810290117.6A CN108646726A (en) | 2018-04-03 | 2018-04-03 | The wheelchair control system of wheelchair control method and combination voice based on brain wave |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810290117.6A CN108646726A (en) | 2018-04-03 | 2018-04-03 | The wheelchair control system of wheelchair control method and combination voice based on brain wave |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108646726A true CN108646726A (en) | 2018-10-12 |
Family
ID=63745562
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810290117.6A Pending CN108646726A (en) | 2018-04-03 | 2018-04-03 | The wheelchair control system of wheelchair control method and combination voice based on brain wave |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108646726A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110012104A (en) * | 2019-04-12 | 2019-07-12 | 深圳市班玛智行科技有限公司 | A kind of social intercourse system and method based on artificial intelligence and technology of Internet of things |
| CN110051351A (en) * | 2019-03-28 | 2019-07-26 | 深圳市宏智力科技有限公司 | It grits one's teeth the control method and device of signal acquiring method and electronic equipment |
| CN110162182A (en) * | 2019-05-28 | 2019-08-23 | 深圳市宏智力科技有限公司 | Brain electric control module device and its method for controlling controlled plant |
| CN112518743A (en) * | 2020-11-20 | 2021-03-19 | 南京航空航天大学 | Multi-mode neural decoding control system and method for on-orbit operation of space manipulator |
| CN113031766A (en) * | 2021-03-15 | 2021-06-25 | 哈尔滨工业大学 | Method for decoding Chinese pronunciation through electroencephalogram |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101776981A (en) * | 2010-01-07 | 2010-07-14 | 天津大学 | Method for controlling mouse by jointing brain electricity and myoelectricity |
| CN101897640A (en) * | 2010-08-10 | 2010-12-01 | 北京师范大学 | Novel movement imagery electroencephalogram control-based intelligent wheelchair system |
| CN102117119A (en) * | 2011-01-28 | 2011-07-06 | 国家康复辅具研究中心 | Method for realizing mouse control based on temporalis surface electromyogram signals |
| CN102419588A (en) * | 2011-12-28 | 2012-04-18 | 许冰 | Method and device for controlling target based on brain electrical signal and motion signal |
| CN103099693A (en) * | 2013-02-07 | 2013-05-15 | 王禹 | Handicapped-helping control system based on electroencephalogram/voice instructions |
| CN103110486A (en) * | 2013-03-06 | 2013-05-22 | 胡三清 | Wheel chair and method controlled by tooth |
| CN104133470A (en) * | 2013-05-03 | 2014-11-05 | 徐燕锋 | Multi-mode brain machine interface wheelchair control system |
| CN104622649A (en) * | 2015-03-05 | 2015-05-20 | 中南大学 | Wheelchair system on basis of Bluetooth myoelectric collectors and method for controlling wheelchair system |
| CN104997582A (en) * | 2015-07-30 | 2015-10-28 | 沈阳工业大学 | Device and method for controlling intelligent artificial limb based on eye and lower jaw electromyographic signals |
| CN106726200A (en) * | 2017-02-07 | 2017-05-31 | 德阳力久云智知识产权运营有限公司 | A kind of intelligent wheel chair based on the control of people's brain wave |
| CN107007407A (en) * | 2017-04-12 | 2017-08-04 | 华南理工大学 | Wheelchair control system based on eye electricity |
| CN107260420A (en) * | 2017-07-03 | 2017-10-20 | 南京邮电大学 | Intelligent wheel chair human-computer interactive control system and method based on eye motion recognition |
-
2018
- 2018-04-03 CN CN201810290117.6A patent/CN108646726A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101776981A (en) * | 2010-01-07 | 2010-07-14 | 天津大学 | Method for controlling mouse by jointing brain electricity and myoelectricity |
| CN101897640A (en) * | 2010-08-10 | 2010-12-01 | 北京师范大学 | Novel movement imagery electroencephalogram control-based intelligent wheelchair system |
| CN102117119A (en) * | 2011-01-28 | 2011-07-06 | 国家康复辅具研究中心 | Method for realizing mouse control based on temporalis surface electromyogram signals |
| CN102419588A (en) * | 2011-12-28 | 2012-04-18 | 许冰 | Method and device for controlling target based on brain electrical signal and motion signal |
| CN103099693A (en) * | 2013-02-07 | 2013-05-15 | 王禹 | Handicapped-helping control system based on electroencephalogram/voice instructions |
| CN103110486A (en) * | 2013-03-06 | 2013-05-22 | 胡三清 | Wheel chair and method controlled by tooth |
| CN104133470A (en) * | 2013-05-03 | 2014-11-05 | 徐燕锋 | Multi-mode brain machine interface wheelchair control system |
| CN104622649A (en) * | 2015-03-05 | 2015-05-20 | 中南大学 | Wheelchair system on basis of Bluetooth myoelectric collectors and method for controlling wheelchair system |
| CN104997582A (en) * | 2015-07-30 | 2015-10-28 | 沈阳工业大学 | Device and method for controlling intelligent artificial limb based on eye and lower jaw electromyographic signals |
| CN106726200A (en) * | 2017-02-07 | 2017-05-31 | 德阳力久云智知识产权运营有限公司 | A kind of intelligent wheel chair based on the control of people's brain wave |
| CN107007407A (en) * | 2017-04-12 | 2017-08-04 | 华南理工大学 | Wheelchair control system based on eye electricity |
| CN107260420A (en) * | 2017-07-03 | 2017-10-20 | 南京邮电大学 | Intelligent wheel chair human-computer interactive control system and method based on eye motion recognition |
Non-Patent Citations (1)
| Title |
|---|
| 谷湘琼等: "意念控制的智能残疾轮椅设计", 《计算机科学与技术》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110051351A (en) * | 2019-03-28 | 2019-07-26 | 深圳市宏智力科技有限公司 | It grits one's teeth the control method and device of signal acquiring method and electronic equipment |
| CN110012104A (en) * | 2019-04-12 | 2019-07-12 | 深圳市班玛智行科技有限公司 | A kind of social intercourse system and method based on artificial intelligence and technology of Internet of things |
| CN110162182A (en) * | 2019-05-28 | 2019-08-23 | 深圳市宏智力科技有限公司 | Brain electric control module device and its method for controlling controlled plant |
| CN110162182B (en) * | 2019-05-28 | 2022-09-06 | 深圳市宏智力科技有限公司 | Electroencephalogram control module device and method for controlling controlled equipment by electroencephalogram control module device |
| CN112518743A (en) * | 2020-11-20 | 2021-03-19 | 南京航空航天大学 | Multi-mode neural decoding control system and method for on-orbit operation of space manipulator |
| CN112518743B (en) * | 2020-11-20 | 2022-04-01 | 南京航空航天大学 | Multi-mode neural decoding control system and method for on-orbit operation of space manipulator |
| CN113031766A (en) * | 2021-03-15 | 2021-06-25 | 哈尔滨工业大学 | Method for decoding Chinese pronunciation through electroencephalogram |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108646726A (en) | The wheelchair control system of wheelchair control method and combination voice based on brain wave | |
| Gao et al. | A recurrence network-based convolutional neural network for fatigue driving detection from EEG | |
| CN103294199B (en) | A kind of unvoiced information identifying system based on face's muscle signals | |
| Yang et al. | A hardware-efficient scalable spike sorting neural signal processor module for implantable high-channel-count brain machine interfaces | |
| Lan et al. | Channel selection and feature projection for cognitive load estimation using ambulatory EEG | |
| CN107981997B (en) | A kind of method for controlling intelligent wheelchair and system based on human brain motion intention | |
| CN111616682B (en) | Epileptic seizure early warning system based on portable electroencephalogram acquisition equipment and application | |
| CN110495893B (en) | System and method for multi-level dynamic fusion recognition of continuous brain and muscle electricity of motor intention | |
| CN110013249A (en) | A kind of Portable adjustable wears seizure monitoring instrument | |
| CN107212883B (en) | A kind of mechanical arm writing device and control method based on brain electric control | |
| CN106127191B (en) | Brain electricity classification method based on WAVELET PACKET DECOMPOSITION and logistic regression | |
| CN102096468A (en) | Brain-computer interface (BCI)-based home appliance remote control device and method | |
| CN111513735A (en) | Major depressive disorder identification system based on brain-computer interface and deep learning and application | |
| CN108113670A (en) | A kind of UAV system and control method of the control of multi-channel type induced brain wave | |
| CN108363493A (en) | User characteristics method for establishing model, system and storage medium based on brain-computer interface | |
| EP3912014A1 (en) | Brain-computer interface system, system for brain activity analysis, and method of analysis | |
| CN108958482A (en) | A kind of similitude action recognition device and method based on convolutional neural networks | |
| CN105786189A (en) | Finger independent action recognition method and system based on MMG signal | |
| CN109770900A (en) | Brain-computer interface based on convolutional neural networks instructs delivery method, system, device | |
| CN107480716A (en) | A kind of combination EOG and video pan signal recognition method and system | |
| CN109717866A (en) | A kind of disturbance of consciousness diagnostic method based on EEG signals | |
| CN111616681A (en) | Anesthesia state monitoring system based on portable electroencephalogram acquisition equipment and deep learning | |
| CN111671420A (en) | Method for extracting features from resting electroencephalogram data and terminal equipment | |
| CN109481164A (en) | A kind of electric wheelchair control system based on EEG signals | |
| Milan et al. | Adaptive brain interfaces for physically-disabled people |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181012 |