CN108803880A - Control device based on brain signal and method - Google Patents
Control device based on brain signal and method Download PDFInfo
- Publication number
- CN108803880A CN108803880A CN201810681319.3A CN201810681319A CN108803880A CN 108803880 A CN108803880 A CN 108803880A CN 201810681319 A CN201810681319 A CN 201810681319A CN 108803880 A CN108803880 A CN 108803880A
- Authority
- CN
- China
- Prior art keywords
- user
- control
- data
- signal
- brain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Neurology (AREA)
- Computational Linguistics (AREA)
- Neurosurgery (AREA)
- Dermatology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The present invention provides a kind of control device and method based on brain signal, the control device include：Detector is configured to respond to the brain activity of user, acquires and send the brain signal of user；Processor, it is configured as being communicated the brain signal of the user to receive the detector acquisition with the detector, image reconstruction process is carried out to the brain signal of reception, and the information obtained is identified based on the image data to reconstruction, to generate the control instruction for control targe equipment；Communication unit is configured as the control signal including the control instruction being sent to target device.The present invention is based on wearable micro MR imaging apparatus and image reconstruction algorithms, and the brain signal of user to be acquired and be handled respectively, realize the control to target device by brain signal, the demand for meeting the brain signal control of various terminal equipment and magnanimity type, provides better user experience.
Description
Technical field
The present invention relates to brain-machine interaction technical fields, more particularly, be related to a kind of control device based on brain signal and
Method.
Background technology
In recent years, the various intelligent terminal extensive uses such as intelligent appliance, smart phone, computer and wearable device
In people's lives, user usually uses hand channel to be realized to intelligent terminal by the input mechanisms such as button, membrane keyboard (interface portion)
Operation and control.But when the both hands of user are occupied by other operations, such as when doing housework, give birth to children, driving,
Intelligent terminal can not be operated, therefore, user has stronger demand to realize the operation and control to intelligent terminal in all cases.Needle
To such demand, brain-machine interaction technology is increasingly risen.
Brain signal control method is roughly divided into three kinds to existing brain-machine interaction method in other words, and a kind of rely on extracts brain signal
Frequency range data, the state for analyzing human brain includes attention, Depth of sleep etc., and then reaches feedback training and user is helped to enter certain
Kind state of a control；One is the relevant error current potentials by extracting brain signal, and user is made to face a series of options whether selected,
Whether brain signal analysis can select a certain option with automatic decision user；One is the user of multi collect is done a certain control
Brain signal when (imagining that a water droplet moves up under such as relaxation state) saves as standard brain signal, and implements brain letter in user
Number control when the brain signal of extraction is compared with standard signal, to judge whether to implement corresponding instruction (such as control targe
It moves up).
Still with the growth of intelligent terminal and IoT equipment (Internet of Things) explosion type, either type
Quantity, they are more diversified and intelligent for the demand of control mode, while control mode being needed to meet personalization
There is universality as possible simultaneously.The mode in the prior art controlled based on brain signal has larger limitation, cannot
Meet the brain signal demand for control of various intelligent terminal magnanimity types.
Invention content
In view of the deficiencies of the prior art, the present invention provides a kind of control devices based on brain signal and method to realize pair
The control of the brain signal of various intelligent terminals and magnanimity type.
According to an aspect of the present invention, a kind of control device based on brain signal is provided, including：Detector is configured
For the brain activity in response to user, the brain signal of user is acquired and sent；Processor is configured as carrying out with the detector
Communication carries out image reconstruction process to receive the brain signal of the user of the detector acquisition, to the brain signal of reception, and is based on
The information obtained is identified to the image data of reconstruction, to generate the control instruction for control targe equipment；Communication unit,
It is configured as the control signal including the control instruction being sent to target device.
Preferably, the processor is configured as：It is rebuild from the brain signal of reception and obtains image data, and utilize word
And/or algorithm for pattern recognition the image data of reconstruction is parsed with identify including in word, symbol and figure at least
One information；Target device is determined according to the information of identification, and generates the control instruction for controlling the target device.
Preferably, the control signal includes infrared control signal, wifi network message signals, mobile network's message signals
At least one of with Bluetooth signal.
Preferably, the control device further includes：Memory, be configured as storage user brain activity data and with
The corresponding preliminary instruction of brain activity data of user.
Preferably, the brain activity data of the user include lteral data, icon data and mark data and user
At least one of customized code data, number data and image data.
Preferably, the processor is additionally configured to：Under simulated training pattern, according to the number stored in the memory
Simulated training is carried out according to user, and the simulated training result of user is judged and fed back.
Preferably, the simulated training result to user judge and feed back including：According to being deposited in the memory
The preliminary instruction of storage judges whether the control instruction that simulated training parses is right instructions, wherein when simulated training parses
When the control instruction gone out matches with preliminary instruction, judge that control instruction for right instructions, marks the simulated training of user to pass through,
When the control instruction that simulated training parses is mismatched with preliminary instruction, control instruction is judged for false command, marks user
Simulated training result do not pass through, and image reconstruction algorithm and word and/or algorithm for pattern recognition are optimized.
Preferably, the detector is wearable micro MR imaging apparatus.
According to another aspect of the present invention, a kind of control method based on brain signal is provided, the control method includes
Following steps：In response to the brain activity of user, the brain signal of user is acquired and sent；The brain signal of the user of acquisition is received,
Image reconstruction process is carried out to the brain signal of reception, and the information obtained is identified based on the image data to reconstruction, with production
The raw control instruction for control targe equipment；Control signal including the control instruction is sent to target device.
Preferably, the brain signal of the user for receiving acquisition carries out image reconstruction process, and base to the brain signal of reception
The step of information obtained generates the control instruction for control targe equipment, which is identified, in the image data to reconstruction includes：
It is rebuild from the brain signal of reception and obtains image data, and utilize the image data of word and/or algorithm for pattern recognition to reconstruction
It is parsed to identify including the information of at least one in word, symbol and figure；Target is determined according to the information of identification
Equipment, and generate the control instruction for controlling the target device.
Preferably, the control signal includes infrared control signal, wifi network message signals, mobile network's message signals
At least one of with Bluetooth signal.
Preferably, the control method further includes：Under simulated training pattern, the data according to storage carry out mould to user
Quasi- training, and the simulated training result of user is judged and fed back, wherein the data of the storage include the brain of user
Activity data and preliminary instruction corresponding with the brain activity data of user.
Preferably, the brain activity data of the user include lteral data, icon data and mark data and user
At least one of customized code data, number data and image data.
Preferably, the simulated training result to user judges and includes the step of feedback：According to the first of storage
Step instructs to judge whether the control instruction that simulated training parses is right instructions, wherein when the control that simulated training parses
When system instruction matches with preliminary instruction, judge that control instruction for right instructions, marks the simulated training of user to pass through, works as simulation
When the control instruction that training parses is mismatched with preliminary instruction, judge that control instruction for false command, marks the simulation of user
Training result does not pass through, and is optimized to image reconstruction algorithm and word and/or algorithm for pattern recognition.
Preferably, the brain activity in response to user acquires and is by can the step of sending the brain signal of user
What the micro MR imaging apparatus of wearing was realized.
According to another aspect of the present invention, a kind of terminal device that can carry out brain signal control, the terminal are provided
Equipment includes：Communication unit is configured as receiving the brain signal of user from external brain signal acquisition device；Processor is configured
To be communicated with the communication unit to receive the brain signal of user, and to the brain signal of reception carry out image reconstruction process with
The control instruction for control terminal equipment is obtained, and corresponding control signal is generated to execute correspondence according to the control instruction
Operation.
Preferably, the terminal device further includes：Memory, be configured as storage user brain activity data and with
The corresponding preliminary instruction of brain activity data of user, the processor is under simulated training pattern, according in the memory
The data of storage carry out simulated training to user, and the simulated training result of user is judged and fed back, wherein the use
The brain activity data at family include lteral data, icon data and mark data and user-defined code data, number
At least one of data and image data.
According to another aspect of the present invention, a kind of brain signal harvester for carrying out brain signal control, institute are provided
Stating brain signal harvester includes：Detector acquires the brain signal of user in response to the brain activity of user；Processor, by with
Be set to and communicated with the detector to receive the brain signal of user, and to the brain signal of reception carry out image reconstruction process with
The control instruction for control terminal equipment is obtained, generate corresponding control signal according to the control instruction is set with control terminal
For the corresponding operation of execution, and
Under simulated training pattern, the data according to storage carry out simulated training to user, and to the simulated training of user
As a result judged and fed back, wherein the data of storage include the brain activity data of user and the brain activity with user
The corresponding preliminary instruction of data.
According to another aspect of the invention, a kind of brain signal processing unit for carrying out brain signal control, institute are provided
Stating brain signal acquisition device includes：Communication unit is configured as receiving the brain signal of user from external brain signal acquisition device；Place
Device is managed, is configured as being communicated with the communication unit to receive the brain signal of user, and figure is carried out to the brain signal of reception
Corresponding control letter is generated as rebuilding processing to obtain the control instruction for control terminal equipment, and according to the control instruction
Number corresponding operation is executed with control terminal equipment, and
Under simulated training pattern, the data according to storage carry out simulated training to user, and to the simulated training of user
As a result judged and fed back, wherein the data of storage include the brain activity data of user and the brain activity with user
The corresponding preliminary instruction of data.
Description of the drawings
Carry out the detailed description of the present invention below in conjunction with attached drawing, features described above of the invention and other purposes, feature and
Advantage will become apparent, wherein：
Fig. 1 shows the block diagram of the control device according to the ... of the embodiment of the present invention based on brain signal；
Fig. 2 shows the users of exemplary embodiment according to the present invention to directly control what mobile phone was made a phone call by brain signal
Schematic diagram；
Fig. 3 shows the flow chart of the control method according to the ... of the embodiment of the present invention based on brain signal；
Fig. 4 shows a kind of end that can carry out brain signal control according to the ... of the embodiment of the present invention
The block diagram of end equipment；
Fig. 5 shows a kind of frame for carrying out the brain signal harvester of brain signal control according to the ... of the embodiment of the present invention
Figure；
Fig. 6 shows a kind of frame for carrying out the brain signal processing unit of brain signal control according to the ... of the embodiment of the present invention
Figure.
Specific implementation mode
Hereinafter, the embodiment of the present invention is described in detail with reference to the attached drawings.Wherein, identical label always shows identical portion
Part.
In the following, it is described that relevant technical terms define：
1, deep neural network
Deep neural network (Deep Neural Networks, DNN) is the basis of deep learning, deep neural network
(DNN) neuronal structure connected entirely is refered in particular to, and does not include convolution unit or temporal association.
2, depth generator network
Depth generator network (Deep Generator Network, DGN) is a kind of algorithm by pre-training, can be with
From middle establishment image true to nature is originally inputted, basically, DGN is to put details after the completion on the image, it is made to have seen
Come more natural.
3, KNN algorithms
K arest neighbors (K-Nearest Neighbor, KNN) algorithm is one of simplest machine learning algorithm.The algorithm
Thinking be：If big in the K in feature space most like samples (i.e. closest in feature space) of a sample
Majority belongs to some classification, then the sample also belongs to this classification, and the neighbours selected in KNN algorithms are correctly to have divided
The object of class, and the algorithm only determines sample to be divided on determining class decision according to the classification of one or several closest samples
Classification belonging to this.
Fig. 1 shows the frame of the control device according to the ... of the embodiment of the present invention based on brain signal
Figure.
As shown in Figure 1, the control device 100 based on brain signal includes detector 101, processor 102, communication unit 103
With memory 104.Wherein, detector 101 is configured to respond to the brain activity of user, acquires and send the brain letter of user
Number.Processor 102 is configured as being communicated the brain signal of the user to receive detector acquisition with detector, to the brain of reception
Signal carries out image reconstruction process, and the information obtained is identified based on the image data to reconstruction, to generate for controlling
The control instruction of target device.Communication unit 103 is configured as the control signal including control instruction being sent to target device.
Memory 104 is configured as the brain activity data of storage user and preliminary finger corresponding with the brain activity data of user
It enables.According to an embodiment of the invention, specifically, detector 101 is configured as wearable micro MR imaging apparatus
MRI, MRI machine are detected the brain activity operation of user, and the brain signal for acquiring the user obtained is MRI image.Processing
Device 102 rebuilds the brain signal received from detector 101 to obtain image data, and is identified using word and/or figure
Algorithm parses the image data of reconstruction to identify including the information of at least one in word, symbol and figure, then
Target device is determined according to the information of identification, and generates the control instruction for control targe equipment.Communication unit 103 is to target
Equipment sends the control signal including control instruction corresponding with target device.
According to an embodiment of the invention, processor 102 extracts the brain signal of the collected user of MRI machine, i.e. MRI image
Data, and data processing is carried out to the MRI image of extraction.Assuming that processor 102 is based on depth image algorithm for reconstructing to MRI image
Image procossing is carried out, then detailed process is as follows：First the brain signal of the user of acquisition is decoded and obtains deep neural network
(DNN) feature, then the pixel value of to arbitrarily inputting image carry out constantly iteration optimization, make the DNN features of the image
Constantly the DNN features obtained are decoded to complete the reconstruction to MRI image close to brain signal.Wherein, constantly iteration optimization
Process is executed in the input space for the depth generator network (DNN) being introduced into.Then, after processor 102 is to reconstruction
Image carries out denoising, and word and/or algorithm for pattern recognition is used to parse to identify including text the image after denoising
The information of at least one in word, symbol and figure determines target device further according to the information of identification, and generates for controlling mesh
The control instruction of marking device.Assuming that using KNN algorithms to parse the image after denoising to identify graphical information, or adopt
The image after denoising is parsed to identify word and/or symbolic information with Text region algorithm, is obtained after image analysis
Control instruction can be digital command, letter instruction, punctuate instruction, icon instruction, mark instruction or it is user-defined
Instruction etc..It should be understood that the above-mentioned citing for word and/or algorithm for pattern recognition is only illustrative examples, the present invention can be used
Word and/or algorithm for pattern recognition it is without being limited thereto.Finally, target device is determined according to obtained control instruction and controlled logical
Believe that unit 103 sends control signal corresponding with target device to target device.Wherein, control signal includes infrared control letter
Number, at least one of wifi network message signals, mobile network's message signals and Bluetooth signal.Communication unit 103 sends control
The mode of signal processed is the mode of wireless transmission.According to an embodiment of the invention, it is assumed that determining target device is intelligent terminal
Equipment, for example, smart mobile phone or intelligent TV set etc., then communication unit 103 sends the control signal including control instruction
To the intelligent Target equipment, which has reception and handles the message of control instruction, and the control to being received
Instruction carries out command analysis and order executes.Assuming that the target device determined is unintelligent terminal equipment, for example, air-conditioning or hanging
Lamp etc., then processor 102 need to be converted to control instruction into the instruction of the control interface suitable for the target device, communication unit
Control signal including the instruction is sent to target device by 103.It should be understood that the above-mentioned citing for target device is only example
Property citing, the adoptable target device of the present invention is without being limited thereto.
According to an embodiment of the invention, the control signal including control instruction is sent to target device by communication unit 103,
Target device executes relevant operation according to the control signal of reception.For example, target device executes the operation of output character symbol,
In, letter symbol includes but not limited to phonetic letter typewriting, the direct output of various spoken and written languages, or closer to further,
Word output content as control instruction, the user terminal either receives control signal target device utilize itself or remotely
The word control instruction is decomposed into the instruction that target device can execute by semantic service device by semantic analysis, then is set by target
It is standby to execute relevant operation.In another example control targe equipment executes the operation of mobile target, wherein mobile target includes but unlimited
Direction and position in mobile mouse or focal position, mobile remote control toy direction and the virtual target in position, moving game
And direction and the position etc. for moving sweeping robot.For another example control targe equipment executes call operation, beats on or off
Closed operation, air-conditioning etc. is opened and is adjusted operation etc..Below with reference to the specific embodiment of different target devices
The control device 100 according to the ... of the embodiment of the present invention based on brain signal is described in detail.
According to an embodiment of the invention, it is assumed that determining target device is smart mobile phone, then the control dress based on brain signal
Setting 100, to directly control the process that smart mobile phone is made a phone call realizing user by brain signal as shown in Figure 2.Specifically, false
If user imagines literal order " Call 10010 ", then the brain signal of MRI machine detection user obtains corresponding MRI image number
According to.Processor 102 is Cloud Server, and Cloud Server extracts MRI image data and based on depth image algorithm for reconstructing to extraction
MRI image rebuild, denoising and parsing, is shown that control instruction is " Call 10010 ", be can determine by the control instruction obtained
Target device is smart mobile phone, and generates corresponding control signal.Communication unit 103 sends control signals to smart mobile phone,
At this point, communication unit 103 can be the integrated device inside Cloud Server.Smart mobile phone, which receives the control comprising control instruction, to be believed
Number and carry out command analysis to control instruction and executed automatically to control smart mobile phone to dial the operation of " 10010 ".According to the present invention
Embodiment, the control device 100 based on brain signal come realize user by brain signal directly control mobile phone send message mistake
Cheng Wei：Assuming that user imagines the literal order of " sending out wechat to Xiao Wang " and " be shown in " at 9 points, then MRI machine detects the brain signal of user
Obtain MRI image data corresponding with word.Processor 102 extracts MRI image data and is based on depth image algorithm for reconstructing
The MRI image of extraction is rebuild, denoising and parsing, show that control instruction is " sending out wechat to Xiao Wang " and " be shown in " at 9 points, and root
Determine that target device is the wechat in smart mobile phone and smart mobile phone according to the control instruction obtained, communication unit 103 will include control
The control signal of system instruction is sent to smart mobile phone.Smart mobile phone reception includes the control signal of control instruction and to control instruction
It carries out command analysis and executes corresponding operation, wherein it includes the wechat software opened in smart mobile phone to execute corresponding operation,
Xiao Wang is found in contact person, and is opened the dialog box between Xiao Wang and sent message " be shown in " at 9 points to Xiao Wang.It should be understood that above-mentioned
It is only illustrative examples to directly control the citing that intelligent terminal is operated by brain signal for user, and the present invention can adopt
The operation that intelligent terminal carries out is without being limited thereto.
According to an embodiment of the invention, it is assumed that determining target device is intelligent TV set, the then control based on brain signal
Device 100 come realize user by brain signal directly control intelligent TV set play self-defined station symbol TV programme process such as
Under：User imagines or recalls self-defined station symbol, and the brain signal of MRI machine detection user obtains corresponding with self-defined station symbol
MRI image data.Processor 102 is extracted MRI image data and is carried out to the MRI image of extraction based on depth image algorithm for reconstructing
It rebuilds, and denoising and KNN algorithm process is carried out to the image data of the self-defined station symbol after reconstruction, to parse and self-defined
Corresponding control instruction is marked, and determines that target device is intelligent TV set according to control instruction.Communication unit 103 will include control
The control signal of system instruction is sent to intelligent TV set.Intelligent TV set carries out command analysis to control instruction and executes corresponding
Operation, wherein execute corresponding operation and specifically include the TV opened intelligent TV set and select to play User Defined station symbol
Program.
According to an embodiment of the invention, it is assumed that determining target device is player, then the control device based on brain signal
100 directly control realizing user by brain signal player process it is as follows：User imagines or recalls related to player
The controls such as control mark, such as switch, broadcastings, pause, F.F., rewind, mute, volume increase, volume reduction, closing, recording
The brain signal of system mark, MRI machine detection user obtains corresponding MRI image data.Processor 102 extracts MRI image data
And the MRI image of extraction is rebuild based on depth image algorithm for reconstructing to obtain the control mark imagined with user or recalled
Know corresponding image data, processor 102 carries out denoising and KNN algorithm process, solution to the image data after the reconstruction that obtains again
Be precipitated corresponding control instruction, and determine that target device is player according to control instruction, at this point, processor 102 also need to by
Control instruction is converted to the control signal of the control interface suitable for player, for example, the infrared control signal of control player or
Person's wifi network message signals etc..Communication unit 103 believes the infrared control signal for controlling player or wifi network message
Number it is sent to player.Player receives control signal and executes specific operation according to the control signal of reception, for example controls
Player executes the operation etc. of the operation or F.F. that play.
According to an embodiment of the invention, it is assumed that determining target device is movable object, the then control based on brain signal
Device 100 directly controls the target realizing user by brain signal, and to carry out mobile process as follows：User imagines or recalls
Move relevant direction symbol, such as the arrow of " ↑ ", " ↓ ", " ← ", " → " with the target, respectively represent it is upward, downward, to
It is left, move right, for example, TV main interface focus is mobile, text input when cursor is mobile, target in game or virtual environment
Movement, remote-control toy direction movement etc., MRI machine detection user brain signal obtain corresponding MRI image data.Processing
Device 102 extract MRI image data simultaneously the MRI image of extraction is rebuild based on depth image algorithm for reconstructing, and to reconstruction after
Direction symbol image data carry out denoising and KNN algorithm process, parse control instruction corresponding with direction symbol, and according to
Control instruction determines target device, at this point, processor 102 also needs to be converted to control instruction suitable for setting where the target
The control signal of standby control interface, such as the infrared control signal of control remote-control toy or the indigo plant for controlling game execution apparatus
Tooth signal etc., wherein game execution apparatus includes mobile phone, computer, Pad, PSP etc..It should be understood that above-mentioned for game execution apparatus
Citing be only illustrative examples, the adoptable game execution apparatus of the present invention is without being limited thereto.Communication unit 103 will include control
The control signal of instruction is sent to equipment where the target.Equipment receives control signal and the control according to reception where the target
Signal executes specific control moving operation, for example target person executes the operation moved up in control game, that is, advance behaviour
Make.It should be understood that the above-mentioned citing for directly controlling movable object progress moving operation by brain signal for user is only example
Property citing, the adoptable movable object of the present invention is without being limited thereto.
According to an embodiment of the invention, memory 104 stores the brain activity data of user and lives with the brain of user
The dynamic corresponding preliminary instruction of data, wherein the brain activity data of user include lteral data, icon data and mark data with
And at least one of user-defined code data, number data and image data.For example, lteral data can choose use
Language known to family, common application scenarios have word input and all data that can use verbal description.Icon data can be with
Use directional icon known to user.Mark data can choose identifier more common at present, can also selected text number
According to, such as broadcasting, tentative, F.F., rewind etc., user-defined identification setting can also be carried out to common control object, for example want
Intelligent control set-top box can be set as TV station's number or station symbol known to user to open the mark number of the TV station
According to.In addition, for the special screne of certain secrecy, user can use customized code data, number data and/or picture
Data, for example, the image data etc. of the code data of Intelligent door lock switch and intelligent terminal screen lock and unlock.Memory
104 can be the mass storage of any suitable type, which, which provides, may need any class operated
The information of type, memory 104 can be volatibility or non-volatile, magnetic, semiconductor, belt, optics, can be removed, it is non-can
Including but not limited to ROM, removal or other types of storage device or tangible (i.e. non-transient) computer-readable medium dodge
It deposits, dynamic ram and static state RAM.
According to an embodiment of the invention, processor 102 also simulates user according to the data stored in memory 104
Training, and the simulated training result of user is judged and fed back.Specifically, under simulated training pattern, user recall and/
Or the brain activity data that store in imagination memory 104, processor 102 to the brain signal of acquisition carry out image reconstruction process with
And the processing of word and/or algorithm for pattern recognition obtains control instruction.Processor 102 is by the control instruction obtained and memory
The preliminary instruction stored in 104 is matched, and when control instruction matches with preliminary instruction, judges control instruction correctly to refer to
It enables, the simulated training of user is marked to pass through.Conversely, when control instruction and preliminary instruction mismatch, judge control instruction for mistake
Accidentally instruct, the simulated training result of user marked not pass through, and to image reconstruction algorithm and word and/or algorithm for pattern recognition into
Row optimization.According to an embodiment of the invention, it is assumed that under simulated training pattern, a certain instruction of user's selection is trained, for example,
Common instruction, the user couple such as selection " opening air-conditioning " or " opening TV " remember with the relevant image data of the instruction
And it selects to enter simulated training pattern.When user recalls the instruction, detector 101 operates simultaneously in response to the brain activity of user
Acquire the brain signal of user, processor 102 extracts brain signal and processing obtains control instruction, and according to control instruction come
It determines target device and controls signal accordingly.Communication unit 103 sends control signals to target device, to complete user
Training to the instruction.Processor 102 extracts the preliminary instruction in memory 104, and by the control instruction obtained and deposits preliminary
Instruction is matched, and judges whether the training result of user is correct, and feed back judging result to processor 102.Wherein, work as simulation
When the control instruction that training parses matches with preliminary instruction, judge that control instruction for right instructions, marks the simulation of user
Training passes through, and when control instruction that simulated training parses is mismatched with preliminary instruction, judges control instruction for false command,
The simulated training result of label user does not pass through, and is optimized to image reconstruction algorithm and word and/or algorithm for pattern recognition.
In addition, processor 102 also by mistake training result feed back to memory 104 and by mistake training result and relevant instruction
It carries out visualization to show, to reinforce memory of the user to related preliminary instruction.
Fig. 3 shows the flow chart of the control method according to the ... of the embodiment of the present invention based on brain signal.
As shown in figure 3, in step S301, in response to the brain activity of user, the brain signal of user is acquired and sent.According to
The embodiment of the present invention specifically acquires the brain signal that the brain activity of user generates, wherein detector by detector
For wearable micro MR imaging apparatus (MRI machine), MRI machine is detected simultaneously the brain activity operation of user
Acquisition obtains the brain signal of user, i.e. MRI image.
Next, in step S302, the brain signal of the user of acquisition is received, the brain signal of reception is carried out at image reconstruction
Reason, and the information obtained is identified based on the image data to reconstruction, to generate the control instruction for control targe equipment.
Specifically, after reconstruction obtains image data from the brain signal of reception, and utilization word and/or algorithm for pattern recognition are to reconstruction
Image data carries out data parsing to identify including the information of at least one in word, symbol and figure.Here, can pass through
Depth image algorithm for reconstructing carries out image reconstruction to the brain signal of reception.Then, target device is determined further according to the information of identification,
And generate the control instruction for being used for control targe equipment.Wherein, data parsing is carried out to the image data after reconstruction, including to figure
As data progress denoising, and word and/or algorithm for pattern recognition is used to parse to obtain the image after denoising
Corresponding control instruction.Finally, target device and and mesh that the brain signal of user to be controlled are determined further according to control instruction
The corresponding control signal of marking device.Here, control signal includes infrared control signal, wifi network message signals, mobile network
At least one of message signals and Bluetooth signal.
Control signal including control instruction is sent to target device by step S303.Specifically, it is obtained according to step S302
The control instruction and target device arrived generates corresponding control signal, and the control signal including the control instruction is sent to mesh
Marking device.Wherein, when determining target device is intelligent terminal, the control signal including the control instruction is sent to
Intelligent terminal is converted to the control instruction suitable for non-intelligence when determining target device is unintelligent terminal equipment
The control instruction of the control interface of energy terminal device, and the control signal including the control instruction is sent to unintelligent terminal and is set
It is standby.
According to an embodiment of the invention, the control method based on brain signal can also be under simulated training pattern, and foundation is deposited
The data of storage carry out simulated training to user, and the simulated training result of user is judged and fed back.Wherein, the data of storage
Brain activity data including user and preliminary instruction corresponding with the brain activity data of user.Specifically, it is instructed in simulation
Practice under pattern, some brain activity data that user recalls and/or the imagination stores simultaneously are executed such as above-mentioned steps S301 to step
It is operated in S303, preliminary instruction then, then according to storage judges whether the control instruction that simulated training parses is correct
Instruction.Wherein, when the control instruction that simulated training parses matches with preliminary instruction, control instruction is judged correctly to refer to
It enables, the simulated training of user is marked to pass through, when the control instruction that simulated training parses is mismatched with preliminary instruction, judge to control
System instruction is false command, marks the simulated training result of user not pass through, and to image reconstruction algorithm and word and/or figure
Recognizer optimizes.
Fig. 4 shows a kind of end that can carry out brain signal control according to the ... of the embodiment of the present invention
The block diagram of end equipment.
As shown in figure 4, a kind of terminal device 400 that can carry out brain signal control includes communication unit 401, processor
402 and memory 403.Wherein, communication unit 401 is configured as receiving the brain signal of user from external brain signal acquisition device.
Processor 402 be configured as being communicated with, communication unit 401 built with receiving the brain signal of user to the brain signal of reception into
Row image reconstruction process generates corresponding control letter to obtain the control instruction for control terminal equipment according to control instruction
Number execute corresponding operation.Memory 403 is configured as the brain activity data of storage user and lives with the brain of user
The corresponding preliminary instruction of dynamic data.Specifically, under simulated training pattern, processor 402 is according to the number stored in memory 403
Simulated training is carried out according to user, and the simulated training result of user is judged and fed back, wherein the user's of storage is big
Brain activity data include lteral data, icon data and mark data and user-defined code data, number data and
At least one of image data.
Fig. 5 shows a kind of frame for carrying out the brain signal harvester of brain signal control according to the ... of the embodiment of the present invention
Figure.
As shown in figure 5, the brain signal harvester 500 for carrying out brain signal control includes detector 501 and processor
502.Wherein, detector 501 acquires the brain signal of user in response to the brain activity of user.Processor 502 and detector 501
It is communicated to receive the brain signal of user, and image reconstruction process is carried out to the brain signal of reception and is used for control terminal to obtain
The control instruction of equipment generates corresponding control signal further according to control instruction and executes corresponding operation with control terminal equipment,
And processor 502, under simulated training pattern, the data according to storage carry out simulated training to user, and to the simulation of user
Training result is judged and is fed back, wherein the data of storage include the brain activity data of user and the brain with user
The corresponding preliminary instruction of activity data.
Fig. 6 shows a kind of frame for carrying out the brain signal processing unit of brain signal control according to the ... of the embodiment of the present invention
Figure.
As shown in fig. 6, the brain signal processing unit 600 for carrying out brain signal control includes communication unit 601 and processing
Device 602.Wherein, communication unit 601 is configured as receiving the brain signal of user from external brain signal acquisition device.Processor 602
It is configured as being communicated with communication unit 601 to receive the brain signal of user, and image reconstruction is carried out to the brain signal of reception
Processing generates corresponding control signal to control end to obtain the control instruction for control terminal equipment according to control instruction
End equipment executes corresponding operation and processor 602 also under simulated training pattern, and the data according to storage carry out user
Simulated training, and the simulated training result of user is judged and fed back, wherein the data of storage include that the brain of user is lived
Dynamic data and preliminary instruction corresponding with the brain activity data of user.
Control device and method according to an embodiment of the invention based on brain signal can by image reconstruction algorithm with
And realize that the control of diversified brain signal operates using wearable micro MR imaging apparatus, it realizes and is based on
The popularity that brain signal is controlled meets the demand of the brain signal control of various terminal equipment and magnanimity type, provides
Better user experience.
Although the present invention, art technology has been shown and described with reference to the certain exemplary embodiments of the present invention
Personnel will be understood that, can be into the case where not departing from the spirit and scope of the present invention limited by claim and its equivalent
Various changes on row various forms and details.
Claims (19)
1. a kind of control device based on brain signal, which is characterized in that the control device includes：
Detector is configured to respond to the brain activity of user, acquires and send the brain signal of user；
Processor is configured as being communicated the brain signal of the user to receive the detector acquisition with the detector, right
The brain signal of reception carries out image reconstruction process, and the information obtained is identified based on the image data to reconstruction, to generate
Control instruction for control targe equipment；
Communication unit is configured as the control signal including the control instruction being sent to target device.
2. control device as described in claim 1, which is characterized in that the processor is configured as：
It is rebuild from the brain signal of reception and obtains image data, and utilize the image of word and/or algorithm for pattern recognition to reconstruction
Data are parsed to identify including the information of at least one in word, symbol and figure；
Target device is determined according to the information of identification, and generates the control instruction for controlling the target device.
3. control device as described in claim 1, which is characterized in that the control signal includes infrared control signal, wifi
At least one of network message signal, mobile network's message signals and Bluetooth signal.
4. control device as claimed in claim 2, which is characterized in that the control device further includes：
Memory is configured as the brain activity data of storage user and preliminary finger corresponding with the brain activity data of user
It enables.
5. control device as claimed in claim 4, which is characterized in that the brain activity data of the user include word number
According at least one of, icon data and mark data and user-defined code data, number data and image data.
6. control device as claimed in claim 4, which is characterized in that the processor is additionally configured to：
Under simulated training pattern, simulated training is carried out to user according to the data stored in the memory, and to user's
Simulated training result is judged and is fed back.
7. control device as claimed in claim 6, which is characterized in that the simulated training result to user carry out judge and
Feedback includes：
Judge whether the control instruction that simulated training parses is correctly to refer to according to the preliminary instruction stored in the memory
It enables, wherein when control instruction that simulated training parses matches with preliminary instruction, judge control instruction for right instructions,
The simulated training of label user passes through, and when the control instruction that simulated training parses is mismatched with preliminary instruction, judges to control
Instruction is false command, and the simulated training result of user is marked not pass through, and is known to image reconstruction algorithm and word and/or figure
Other algorithm optimizes.
8. control device as described in claim 1, which is characterized in that the detector be wearable micro magnetic resonance at
As equipment.
9. a kind of control method based on brain signal, which is characterized in that the control method includes the following steps：
In response to the brain activity of user, the brain signal of user is acquired and sent；
The brain signal for receiving the user of acquisition carries out image reconstruction process to the brain signal of reception, and based on the image to reconstruction
The information obtained is identified in data, to generate the control instruction for control targe equipment；
Control signal including the control instruction is sent to target device.
10. control method as claimed in claim 9, which is characterized in that the brain signal of the user for receiving acquisition, to receiving
Brain signal carry out image reconstruction process, and the information that obtains is identified based on the image data to reconstruction and generates for controlling
The step of control instruction of target device includes：
It is rebuild from the brain signal of reception and obtains image data, and utilize the image of word and/or algorithm for pattern recognition to reconstruction
Data are parsed to identify including the information of at least one in word, symbol and figure；
Target device is determined according to the information of identification, and generates the control instruction for controlling the target device.
11. control method as claimed in claim 9, which is characterized in that the control signal includes infrared control signal, wifi
At least one of network message signal, mobile network's message signals and Bluetooth signal.
12. control method as claimed in claim 9, which is characterized in that the control method further includes：
Under simulated training pattern, the data according to storage carry out simulated training to user, and to the simulated training result of user
Judged and fed back, wherein the data of the storage include the brain activity data of user and the brain activity with user
The corresponding preliminary instruction of data.
13. control method as claimed in claim 12, which is characterized in that the brain activity data of the user include word number
According at least one of, icon data and mark data and user-defined code data, number data and image data.
14. control method as claimed in claim 13, which is characterized in that the simulated training result to user judges
Include with the step of feedback：
Judge whether the control instruction that simulated training parses is right instructions according to the preliminary instruction of storage, wherein work as mould
When the control instruction that quasi- training parses matches with preliminary instruction, judge that control instruction for right instructions, marks the mould of user
Quasi- training passes through, and when the control instruction that simulated training parses is mismatched with preliminary instruction, judges that control instruction refers to for mistake
It enables, the simulated training result of user is marked not pass through, and image reconstruction algorithm and word and/or algorithm for pattern recognition are carried out excellent
Change.
15. control method as claimed in claim 9, which is characterized in that the brain activity in response to user, acquisition are concurrent
The step of brain signal for sending user, is realized by wearable micro MR imaging apparatus.
16. a kind of terminal device that can carry out brain signal control, which is characterized in that the terminal device includes：
Communication unit is configured as receiving the brain signal of user from external brain signal acquisition device；
Processor is configured as being communicated with the communication unit to receive the brain signal of user, and to the brain signal of reception
Image reconstruction process is carried out to obtain the control instruction for control terminal equipment, and generate accordingly according to the control instruction
Signal is controlled to execute corresponding operation.
17. terminal device as claimed in claim 16, which is characterized in that the terminal device further includes：
Memory is configured as the brain activity data of storage user and preliminary finger corresponding with the brain activity data of user
It enabling, the processor carries out simulated training under simulated training pattern, according to the data stored in the memory to user, and
The simulated training result of user is judged and fed back, wherein the brain activity data of the user include lteral data, figure
Mark at least one of data and mark data and user-defined code data, number data and image data.
18. a kind of brain signal harvester for carrying out brain signal control, which is characterized in that the brain signal harvester packet
It includes：
Detector acquires the brain signal of user in response to the brain activity of user；
Processor is configured as being communicated with the detector to receive the brain signal of user, and to the brain signal of reception into
Row image reconstruction process generates corresponding control to obtain the control instruction for control terminal equipment, according to the control instruction
Signal executes corresponding operation with control terminal equipment, and
Under simulated training pattern, the data according to storage carry out simulated training to user, and to the simulated training result of user
Judged and fed back, wherein the data of storage include the brain activity data of user and the brain activity data with user
Corresponding preliminary instruction.
19. a kind of brain signal processing unit for carrying out brain signal control, which is characterized in that the brain signal acquisition device packet
It includes：
Communication unit is configured as receiving the brain signal of user from external brain signal acquisition device；
Processor is configured as being communicated with the communication unit to receive the brain signal of user, and to the brain signal of reception
Image reconstruction process is carried out to obtain the control instruction for control terminal equipment, and generate accordingly according to the control instruction
It controls signal and corresponding operation is executed with control terminal equipment, and
Under simulated training pattern, the data according to storage carry out simulated training to user, and to the simulated training result of user
Judged and fed back, wherein the data of storage include the brain activity data of user and the brain activity data with user
Corresponding preliminary instruction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810681319.3A CN108803880A (en) | 2018-06-27 | 2018-06-27 | Control device based on brain signal and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810681319.3A CN108803880A (en) | 2018-06-27 | 2018-06-27 | Control device based on brain signal and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108803880A true CN108803880A (en) | 2018-11-13 |
Family
ID=64072034
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810681319.3A Pending CN108803880A (en) | 2018-06-27 | 2018-06-27 | Control device based on brain signal and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108803880A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109491509A (en) * | 2018-12-13 | 2019-03-19 | 深圳壹账通智能科技有限公司 | A kind of signal processing method, equipment and computer readable storage medium |
| CN110531791A (en) * | 2019-08-25 | 2019-12-03 | 西北工业大学 | The machine integrated target detection unmanned vehicle of multiple instruction set hypencephalon |
| CN113190122A (en) * | 2021-05-31 | 2021-07-30 | 江苏集萃脑机融合智能技术研究所有限公司 | Intelligent device and method based on brain signals, intelligent system and application |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103034778A (en) * | 2012-09-28 | 2013-04-10 | 中国科学院自动化研究所 | Method for extracting brain function network of individual based on analysis of multiple tested brain function data |
| CN105054928A (en) * | 2015-07-17 | 2015-11-18 | 张洪振 | Emotion display equipment based on BCI (brain-computer interface) device electroencephalogram acquisition and analysis |
| CN105446492A (en) * | 2015-12-29 | 2016-03-30 | 武汉华星光电技术有限公司 | Information interaction system based on brainwave sensing headset and intelligent wearable apparatus |
| CN106445155A (en) * | 2016-09-29 | 2017-02-22 | 珠海市魅族科技有限公司 | Controlling method and virtual reality equipment based on electroencephalogram eeg |
| CN107045593A (en) * | 2017-04-17 | 2017-08-15 | 浙江大学 | Brain function experimental duties stimulating system with Real-time Feedback and task more New function |
-
2018
- 2018-06-27 CN CN201810681319.3A patent/CN108803880A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103034778A (en) * | 2012-09-28 | 2013-04-10 | 中国科学院自动化研究所 | Method for extracting brain function network of individual based on analysis of multiple tested brain function data |
| CN105054928A (en) * | 2015-07-17 | 2015-11-18 | 张洪振 | Emotion display equipment based on BCI (brain-computer interface) device electroencephalogram acquisition and analysis |
| CN105446492A (en) * | 2015-12-29 | 2016-03-30 | 武汉华星光电技术有限公司 | Information interaction system based on brainwave sensing headset and intelligent wearable apparatus |
| CN106445155A (en) * | 2016-09-29 | 2017-02-22 | 珠海市魅族科技有限公司 | Controlling method and virtual reality equipment based on electroencephalogram eeg |
| CN107045593A (en) * | 2017-04-17 | 2017-08-15 | 浙江大学 | Brain function experimental duties stimulating system with Real-time Feedback and task more New function |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109491509A (en) * | 2018-12-13 | 2019-03-19 | 深圳壹账通智能科技有限公司 | A kind of signal processing method, equipment and computer readable storage medium |
| CN110531791A (en) * | 2019-08-25 | 2019-12-03 | 西北工业大学 | The machine integrated target detection unmanned vehicle of multiple instruction set hypencephalon |
| CN113190122A (en) * | 2021-05-31 | 2021-07-30 | 江苏集萃脑机融合智能技术研究所有限公司 | Intelligent device and method based on brain signals, intelligent system and application |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021082749A1 (en) | Action identification method based on artificial intelligence and related apparatus | |
| CN105426857B (en) | Human face recognition model training method and device | |
| CN108803880A (en) | Control device based on brain signal and method | |
| CN107688802A (en) | A kind of easy programming method and device based on image recognition | |
| CN111616941B (en) | Mode selection method, device, mobile terminal and computer readable storage medium | |
| CN108352159A (en) | The electronic equipment and method of voice for identification | |
| CN102789313A (en) | User interaction system and method | |
| CN108703824B (en) | Bionic hand control system and control method based on myoelectricity bracelet | |
| CN106225174A (en) | Air conditioner control method and system and air conditioner | |
| CN107229230A (en) | Terminal and its control method, intelligent domestic system and computer-readable recording medium | |
| US11517500B2 (en) | Massage apparatus | |
| CN107507620A (en) | Voice broadcast sound setting method and device, mobile terminal and storage medium | |
| CN107248948A (en) | Send message treatment method and system | |
| US20190294954A1 (en) | Artificial intelligence server | |
| CN106713811A (en) | Video communication method and device | |
| CN104574930B (en) | Configuration method, mobile terminal and the household electric appliances of mobile terminal home control function | |
| CN112036261A (en) | Gesture recognition method and device, storage medium and electronic device | |
| CN105807640A (en) | Terminal control method and terminal control system | |
| CN109308159A (en) | Smart machine control method, device, system, electronic equipment and storage medium | |
| CN110349577B (en) | Man-machine interaction method and device, storage medium and electronic equipment | |
| CN103297530B (en) | A kind of method of control intelligent home device | |
| CN113537122A (en) | Motion recognition method and device, storage medium and electronic equipment | |
| CN109508088A (en) | One kind is based on electromyography signal Sign Language Recognition translation armlet and sign Language Recognition Method | |
| US20190163436A1 (en) | Electronic device and method for controlling the same | |
| DE112018007850B4 (en) | VOICE RECOGNITION SYSTEM AND OPERATING METHOD OF A VOICE RECOGNITION SYSTEM |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181113 |