CN109620257A - State of mind intervention and regulating system and its working method based on biofeedback - Google Patents
State of mind intervention and regulating system and its working method based on biofeedback Download PDFInfo
- Publication number
- CN109620257A CN109620257A CN201811431066.0A CN201811431066A CN109620257A CN 109620257 A CN109620257 A CN 109620257A CN 201811431066 A CN201811431066 A CN 201811431066A CN 109620257 A CN109620257 A CN 109620257A
- Authority
- CN
- China
- Prior art keywords
- state
- mind
- module
- projection
- subject
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Psychiatry (AREA)
- Engineering & Computer Science (AREA)
- Pathology (AREA)
- Biomedical Technology (AREA)
- Hospice & Palliative Care (AREA)
- Psychology (AREA)
- Social Psychology (AREA)
- Physics & Mathematics (AREA)
- Developmental Disabilities (AREA)
- Biophysics (AREA)
- Child & Adolescent Psychology (AREA)
- Educational Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The present invention provides a kind of state of mind intervention and regulating system based on biofeedback, it is characterized by comprising: projection module, audio playing module, physiological signal collection module and state of mind evaluation module, and the state of mind assessment result for being obtained according to state of mind evaluation module adjusts scene source file and audio source file, and feeds back to the state of mind of projection module and audio playing module and intervene adjustment module.The system monitors analysis subject's state of mind in the case where subject receives stimulation state, and adjusts stimulation content and intensity according to the state of mind, so that the mood on subject influences, and then the state of mind is intervened and adjusted, it realizes simplicity, there is good real-time, it is high-efficient.It is another object of the present invention to provide the working method of a kind of above-mentioned state of mind intervention and regulating system, which realizes simplicity, and the state of mind, which is adjusted, has good real-time.
Description
Technical field
The present invention relates to state of mind interventions and regulation technology field, are based on biofeedback more specifically to one kind
State of mind intervention and regulating system and its working method.
Background technique
Biofeedback technology is using modern physiological scientific instrument, instead by itself of people's body physiological or pathological information
Feedback, makes subject after special training, carries out conscious " idea " control and psychological training.With biofeedback technology
Development, the training for Feeling control are possibly realized.There is within nearly 2 years the real-time functional mri fMRI of research and utilization
Feedback technique carries out the training of Feeling control, and subject passes through the brain activity situation for intuitively observing itself, to complete to adjust
Control process effectively confirms the feasibility of system.But the pump back test cost based on fMRI is high；It is used as a dynamic simultaneously
Process, the time-varying characteristics of Feeling control are Milliseconds, and the feedback system carried out using fMRI, real-time is not high, affects tune
Control efficiency.
In addition, existing biofeedback technology mostly uses the method interacted with display screen, for example, by game or video with
Biofeedback technology combines, and to adjust the physiology and psychological condition of people, this and display screen interaction is very stiff, people
It cannot be immersed in stimulation atmosphere well, therefore intervene general with regulating effect.
Existing state of mind assessment passes through the physiological signal of acquisition human body, such as brain electricity, skin electricity, myoelectricity, heart rate mostly
Signal analyzes the mood of people, to realize the assessment to the state of mind.But the existing spiritual shape based on brain electricity
State assessment system is to acquire EEG signals by electrode, is then counted with Short Time Fourier Transform, principal component analysis scheduling algorithm
According to processing, data characteristics is extracted, mood is identified in different frequency range, but the feature that the algorithm extracts is known applied to mood
Effect is bad when other.
Summary of the invention
To overcome shortcoming and deficiency of the prior art, it is an object of the present invention to provide one kind to be based on biofeedback
State of mind intervention and regulating system.The system monitors analysis subject's state of mind in the case where subject receives stimulation state,
And stimulation content and intensity is adjusted according to the state of mind, so that the mood on subject influences, so to the state of mind into
Row is intervened and is adjusted, and realizes simplicity, has good real-time, high-efficient.It is another object of the present invention to provide in one kind
The working method of state of mind intervention and regulating system is stated, which realizes that simplicity, the state of mind are adjusted with good real
Shi Xing.
In order to achieve the above object, the technical scheme is that: a kind of essence based on biofeedback
Refreshing state intervention and regulating system, it is characterised in that: include:
For scene source file to be projected into the projection module of image；
Audio playing module for playing out audio source file corresponding with projection image；
For acquiring the physiological signal collection module of subject's physiological signal；
For carrying out Emotion identification and state of mind assessment to the collected physiological signal of physiological signal collection module, obtain
The state of mind evaluation module of state of mind assessment result；
And the state of mind assessment result for being obtained according to state of mind evaluation module adjust scene source file and
Audio source file, and feed back to the state of mind of projection module and audio playing module and intervene adjustment module.
It preferably, further include cloud platform for storing and managing state of mind assessment result and physiological signal, cloud platform
It is connect with state of mind evaluation module signal.
A kind of working method of above-mentioned state of mind intervention and regulating system, characterized by the following steps:
Scene source file is projected into image and watched for subject by S1 step, projection module, carries out visual stimulus to subject；
Audio playing module plays out audio source file corresponding with projection image simultaneously, carries out auditory stimulation to subject；
S2 step, physiological signal collection module acquire the physiological signal of subject；Physiological signal includes EEG signals, heart rate letter
Number, any one of electromyography signal, skin electric signal, facial expression signal and voice signal or two or more；
S3 step, state of mind evaluation module pre-processes physiological signal, feature extraction and Emotion identification, to subject
The state of mind assessed, obtain state of mind assessment result；
S4 step, the state of mind intervenes adjustment module and adjusts scene source file according to state of mind assessment result, and feeds back and arrive
Projection module, to adjust the projection image of projection module；Audio source file is adjusted simultaneously, and is fed back and arrived audio playing module, with
Play music corresponding with projection image.
Preferably, in the S1 step, scene source file is projected into image and watched for subject by projection module, to subject
Carry out visual stimulus, refer to: projection module is holographic projection module, and scene source file is projected into image and supplied by holographic projection module
Subject's viewing carries out visual stimulus to subject.
Preferably, in the S1 step, audio playing module plays out audio source file corresponding with projection image,
Carry out auditory stimulation to subject, refer to: audio playing module is loudspeaker or In-Ear Headphones or bone conduction earphone；Loudspeaker
Or In-Ear Headphones or bone conduction earphone play out audio source file corresponding with projection image, listen subject
Feel stimulation.
Preferably, in the S2 step, physiological signal collection module acquires the physiological signal of subject, refers to: physiological signal
The EEG signals of acquisition module acquisition subject；
S3 step, state of mind evaluation module pre-processes physiological signal, feature extraction and Emotion identification, to subject
The state of mind assessed, obtain state of mind assessment result, refer to: state of mind evaluation module carries out EEG signals pre-
Processing, and feature extraction and Emotion identification are carried out using Dynamic Graph convolutional neural networks algorithm combination width learning system, to quilt
The state of mind of examination person is assessed, and state of mind assessment result is obtained.
Preferably, the state of mind evaluation module pre-processes EEG signals, and using Dynamic Graph convolution mind
Feature extraction and Emotion identification are carried out through network algorithm combination width learning system, the state of mind of subject is assessed,
State of mind assessment result is obtained, is referred to: using in independent composition analysis algorithm and Principal Component Analysis Algorithm removal EEG signals
Eye electricity, electrocardio, electromagnetic interference artefact with realize pretreatment；EEG signals are extracted using Dynamic Graph convolutional neural networks algorithm
Feature, EEG signals are mapped to feature space, then in feature space using width learning system be used as classifier, general
EEG signals carry out mood Classification and Identification, obtain the intensity value of each mood classification；It is commented according to the intensity value of each mood classification
Estimate the state of mind of subject.
Preferably, the mood classification includes tired, depressed, dejected and boring；According to the intensity value of each mood classification
The state of mind for assessing subject, refers to: tired normal value, depressed normal value, dejected normal value and boring normal value are set, point
Do not judge the ratio of fatigue strength values and tired normal value, the ratio of depressed intensity value and depressed normal value, dejected intensity value with
The ratio of dejected normal value and boring intensity value assess the state of mind of subject with boring normal value.
Preferably, in the S4 step, the state of mind intervenes adjustment module and adjusts scene source according to state of mind assessment result
File, and feed back and arrive projection module, to adjust the projection image of projection module；Audio source file is adjusted simultaneously, and is fed back and arrived sound
Frequency playing module is referred to playing music corresponding with projection image: using one of the following two kinds scheme:
One, the state of mind is intervened adjustment module and is intervened in adjustment module according to state of mind assessment result from the state of mind
Projection scene library recalls corresponding scene source file, and scene source file is sent to projection module, and projection module is to receiving
Scene source file projected to adjust projection image；The state of mind is intervened adjustment module and is adjusted from state of mind intervention simultaneously
Broadcasting audio repository in module recalls audio source file corresponding with projection image, and audio source file is sent to audio and is broadcast
Amplification module, audio playing module plays out the audio source file received so that audio playing module plays and projection image
Corresponding music；
Two, the state of mind is intervened adjustment module and is instructed according to the corresponding projection signal of state of mind assessment result generation,
And it is sent to projection module；Projection module recalls corresponding according to projection scene library of projection signal's instruction from projection module
Scene source file is projected to adjust projection image；The state of mind is intervened adjustment module and is generated and projection signal's instruction phase simultaneously
Corresponding audio signal instruction, and it is sent to audio playing module；Audio playing module is instructed from audio according to audio signal and is broadcast
Broadcasting audio repository in amplification module recalls corresponding audio source file and plays out so that audio playing module plays and projection
The corresponding music of image.
Compared with prior art, the invention has the advantages that with the utility model has the advantages that
1, the physiological signal that the present invention acquires subject in the case where carrying out visual stimulus and auditory stimulation state to subject is commented
Estimate the state of mind of subject, and adjusts projection image and music in real time to adjust stimulation content and intensity, thus in real time to quilt
The mood of examination person is influenced, and then the state of mind is intervened and adjusted；It realizes simplicity, subject's Feeling control is carried out
Real-time monitoring and training integrate state of mind assessment and intervene adjusting, have good real-time, high-efficient；
2, traditional emotional distress is mostly using picture or video, and subject is not easy to rapidly enter state, Er Qieke
Absent-minded situation can be will appear, so that the EEG signals inaccuracy of acquisition；The present invention designs line holographic projections associated with mood
Scene, using the interference of line holographic projections technology and diffraction principle record and the true 3-D image of reconstructed object, and from human eye
Object Depth sense is taken in psychological hint physiologically, more life-like stereoscopic three-dimensional image is generated, people is allowed to have more
The experience of immersion, it is also more accurate and strong to the stimulation of mood；
3, the present invention is in state of mind evaluation module, using Dynamic Graph convolutional neural networks algorithm (Dynamical
Graph Convolutional Neural Networks, DGCNN) and width learning system (Broad Learning
System, BLS), i.e. the algorithm of DGCNN+BLS realizes the assessment of the state of mind；Dynamic Graph convolutional neural networks algorithm is volume
The extension of product neural network algorithm (CNN) on the diagram；The research object of traditional CNN is mainly in rule space structure
Data, and DGCNN uses the thought of map, realizes the method for carrying out deep learning to the data of Anomalistic space structure；Dynamically
Picture scroll product neural network algorithm may be implemented to learn end to end, learn spy complicated out automatically from original EEG signals
Sign；Width learning system is a kind of incremental learning system for not needing depth structure, it can be used as a classifier to carry out
The classification and identification of mood, effect is good and speed is very fast；Therefore this deep learning+width study algorithm is to the state of mind
Assessment it is more accurate；
4, the present invention not only can play audio using loudspeaker and In-Ear Headphones, can also be broadcast with bone conduction earphone
Playback frequency；Bone conduction earphone converts sound to the mechanical oscillation signal of different frequency, passes through the skull of people, human body and neural group
It knits to transmit；The sound conduction mode of sound wave is generated by vibrating diaphragm relative to tradition, osteoacusis eliminates many sound wave transmitting
The step of；And clearly sound-reducing can be realized in a noisy environment, compared to traditional In-Ear Headphones and loudspeaker
With outstanding audio result of broadcast, and sound wave will not influence other people because of spreading in air；
5, state of mind evaluation module ties collected physiological signal and the assessment of the obtained state of mind in the present invention
Fruit is sent to cloud platform, facilitates data storage, processing and the data analysis operation for realizing cloud platform, is convenient for model correction.
Detailed description of the invention
Fig. 1 is the block diagram of present system；
Fig. 2 is the block diagram of holographic projection module in present system；
Fig. 3 is the block diagram of present system sound intermediate frequency playing module；
Fig. 4 is the block diagram of physiological signal collection module in present system；
Fig. 5 is the block diagram of state of mind evaluation module in present system；
Fig. 6 is the block diagram that the state of mind intervenes adjustment module in present system.
Specific embodiment
The present invention is described in further detail with specific embodiment with reference to the accompanying drawing.
Embodiment
State of mind intervention and regulating system of the present embodiment based on biofeedback, structure are as shown in Figure 1, comprising:
For scene source file to be projected into the projection module of image；
Audio playing module for playing out audio source file corresponding with projection image；
For acquiring the physiological signal collection module of subject's physiological signal；
For carrying out Emotion identification and state of mind assessment to the collected physiological signal of physiological signal collection module, obtain
The state of mind evaluation module of state of mind assessment result；
And the state of mind assessment result for being obtained according to state of mind evaluation module adjust scene source file and
Audio source file, and feed back to the state of mind of projection module and audio playing module and intervene adjustment module.
It is also preferable to include the cloud platform for storing and managing state of mind assessment result and physiological signal, cloud platform and essence
Refreshing state estimation module signal connection.
The working method of state of mind intervention and regulating system includes the following steps:
Scene source file is projected into image and watched for subject by S1 step, projection module, carries out visual stimulus to subject；
Audio playing module plays out audio source file corresponding with projection image simultaneously, carries out auditory stimulation to subject；
S2 step, physiological signal collection module acquire the physiological signal of subject；Physiological signal includes EEG signals, heart rate letter
Number, any one of electromyography signal, skin electric signal, facial expression signal and voice signal or two or more；
S3 step, state of mind evaluation module pre-processes physiological signal, feature extraction and Emotion identification, to subject
The state of mind assessed, obtain state of mind assessment result；
S4 step, the state of mind intervenes adjustment module and adjusts scene source file according to state of mind assessment result, and feeds back and arrive
Projection module, to adjust the projection image of projection module；Audio source file is adjusted simultaneously, and is fed back and arrived audio playing module, with
Play music corresponding with projection image.
The physiological signal that the present invention acquires subject in the case where carrying out visual stimulus and auditory stimulation state to subject is assessed
The state of mind of subject adjusts projection image and music to adjust stimulation content and intensity, thus to the mood of subject into
Row influences, and then the state of mind of subject is intervened and adjusted；It realizes simplicity, subject's Feeling control is carried out real-time
Monitoring and training have good real-time, high-efficient.
By subject overcome it is probably high for, the state of mind first intervenes adjustment module and selects 10 floors from projection scene library
High scene source file, is sent to projection module, and some easily music is selected to be sent to audio playing module, projective module
Block is projected out the high image of 10 floors, stimulates the mood of subject, and physiological signal collection module acquires the physiology letter of people
Number, such as brain electricity, myoelectricity, heart rate, mood analysis is carried out by state of mind evaluation module and the state of mind is assessed.For a period of time
Afterwards, when subject gradually gets used to the height of 10 floors, state of mind evaluation module can detect that subject is no longer so frightened,
After state of mind intervention adjustment module receives this state of mind assessment result, adjustment projection image, such as 20 layers of selection
The high scene source file in building, is sent to projection module, so that the state of mind of subject is intervened and be adjusted, helps to be tested
Person gradually overcomes probably high.
Preferable scheme is that: in S1 step, scene source file is projected into image and watched for subject by projection module, to subject
Person carries out visual stimulus, and refer to: projection module is holographic projection module, and scene source file is projected into image by holographic projection module
It is watched for subject, visual stimulus is carried out to subject.
Holographic projection module mainly projects the source file in holographic scene library, generates 3 D stereo line holographic projections
Image.As shown in Fig. 2, scene source file is played by display screen first, the broadcasting content of display screen by tower-shaped glass into
After row reflection, three-dimensional 3D hologram projection image is formed.The selection of the scene source file of line holographic projections is by the state of mind
Intervene adjustment module to complete.
Audio playing module plays out audio source file corresponding with projection image, carries out sense of hearing thorn to subject
Swash, refer to: audio playing module is loudspeaker or In-Ear Headphones or bone conduction earphone；Loudspeaker or In-Ear Headphones or bone pass
Guide lug machine plays out audio source file corresponding with projection image, carries out auditory stimulation to subject, as shown in Figure 3.
Bone conduction earphone is the mechanical oscillation signal for converting sound to different frequency, passes through the skull, human body and mind of people
It is transmitted through organizing.The sound conduction mode of sound wave is generated by vibrating diaphragm relative to tradition, bone conduction earphone eliminates many
The step of sound wave transmits, and clearly sound-reducing can be realized in a noisy environment, and sound wave will not be because of in sky
It is spread in gas and influences other people.When in use, subject can according to specific circumstances unrestricted choice play audio mode,
To there is better experience effect.
In S2 step, physiological signal collection module acquires the physiological signal of subject, and refer to: physiological signal is preferably brain telecommunications
Number, physiological signal collection module acquires the EEG signals of subject.Physiological signal collection module is as shown in Figure 4, it is preferred to use intelligence
The wearable devices such as energy headband, intelligent helmet, can be used STM32 chip and are controlled and handled, first wear wearable device
On subject head, make electrode that forehead skin be steadily contacted, then carries out Conduction choice, the brain electricity of one lead of acquisition to multi-lead
Collected EEG signals are carried out preposition amplification by signal, to obtain carrying out postposition amplification and filter later than stronger signal
Wave operation, finally exports EEG signals.
S3 step, state of mind evaluation module pre-processes physiological signal, feature extraction and Emotion identification, to subject
The state of mind assessed, obtain state of mind assessment result, refer to: state of mind evaluation module carries out EEG signals pre-
Processing, and feature extraction and Emotion identification are carried out using Dynamic Graph convolutional neural networks algorithm combination width learning system, to quilt
The state of mind of examination person is assessed, and obtains state of mind assessment result, as shown in Figure 5.
It further says, the state of mind evaluation module pre-processes EEG signals, and uses dynamic picture scroll
Product neural network algorithm combination width learning system carries out feature extraction and Emotion identification, comments the state of mind of subject
Estimate, obtain state of mind assessment result, refer to: using independent composition analysis algorithm and Principal Component Analysis Algorithm removal EEG signals
In eye electricity, electrocardio, electromagnetic interference artefact with realize pretreatment；Brain telecommunications is extracted using Dynamic Graph convolutional neural networks algorithm
Number feature, EEG signals are mapped to feature space, then in feature space using width learning system as classifier,
EEG signals are subjected to mood Classification and Identification, obtain the intensity value of each mood classification；According to the intensity value of each mood classification
Assess the state of mind of subject.
Mood classification includes tired, depressed, dejected and boring；Assess subject's according to the intensity value of each mood classification
The state of mind refers to: setting tired normal value, depressed normal value, dejected normal value and boring normal value, judgement fatigue is strong respectively
Angle value and the ratio of tired normal value, the ratio of depressed intensity value and depressed normal value, dejected intensity value and dejected normal value
Ratio and boring intensity value assess the state of mind of subject with boring normal value.
In S4 step, the state of mind intervenes adjustment module and adjusts scene source file according to state of mind assessment result, and feeds back
To projection module, to adjust the projection image of projection module；Audio source file is adjusted simultaneously, and is fed back and arrived audio playing module,
To play music corresponding with projection image, refer to: using one of the following two kinds scheme:
One, the state of mind is intervened adjustment module and is intervened in adjustment module according to state of mind assessment result from the state of mind
Projection scene library recalls corresponding scene source file, and scene source file is sent to projection module, and projection module is to receiving
Scene source file projected to adjust projection image；The state of mind is intervened adjustment module and is adjusted from state of mind intervention simultaneously
Broadcasting audio repository in module recalls audio source file corresponding with projection image, and audio source file is sent to audio and is broadcast
Amplification module, audio playing module plays out the audio source file received so that audio playing module plays and projection image
Corresponding music, as shown in Figure 6；
Two, the state of mind is intervened adjustment module and is instructed according to the corresponding projection signal of state of mind assessment result generation,
And it is sent to projection module；Projection module recalls corresponding according to projection scene library of projection signal's instruction from projection module
Scene source file is projected to adjust projection image；The state of mind is intervened adjustment module and is generated and projection signal's instruction phase simultaneously
Corresponding audio signal instruction, and it is sent to audio playing module；Audio playing module is instructed from audio according to audio signal and is broadcast
Broadcasting audio repository in amplification module recalls corresponding audio source file and plays out so that audio playing module plays and projection
The corresponding music of image.
State of mind evaluation module realizes that signal is connect with cloud platform by WiFi wireless communication module.WiFi wireless communication
Module setting can turn WiFi module etc. with serial ports WiFi module TLN13UA06, M02SPI.State of mind evaluation module will acquire
To physiological signal and the obtained state of mind assessment result cloud platform is sent to by WiFi wireless communication module, facilitate reality
Data storage, processing and the data analysis operation of existing cloud platform.
The invention has the advantages that
1, the physiological signal that the present invention acquires subject in the case where carrying out visual stimulus and auditory stimulation state to subject is commented
Estimate the state of mind of subject, adjusts projection image and music to adjust stimulation content and intensity, thus to the spirit of subject
State is intervened and is adjusted；It realizes simplicity, real-time monitoring and training is carried out to subject's Feeling control, have good real-time
Property, it is high-efficient；
2, traditional emotional distress is mostly using picture or video, and subject is not easy to rapidly enter state, Er Qieke
Absent-minded situation can be will appear, so that the EEG signals inaccuracy of acquisition；The present invention designs line holographic projections associated with mood
Scene, using the interference of line holographic projections technology and diffraction principle record and the true 3-D image of reconstructed object, and from human eye
Object Depth sense is taken in psychological hint physiologically, more life-like stereoscopic three-dimensional image is generated, people is allowed to have more
The experience of immersion, it is also more accurate and strong to the stimulation of mood；
3, the present invention is in state of mind evaluation module, using Dynamic Graph convolutional neural networks algorithm (Dynamical
Graph Convolutional Neural Networks, DGCNN) and width learning system (Broad Learning
System, BLS), i.e. the algorithm of DGCNN+BLS realizes the assessment of the state of mind；Dynamic Graph convolutional neural networks algorithm is volume
The extension of product neural network algorithm (CNN) on the diagram；The research object of traditional CNN is mainly in rule space structure
Data, and DGCNN uses the thought of map, realizes the method for carrying out deep learning to the data of Anomalistic space structure；Dynamically
Picture scroll product neural network algorithm may be implemented to learn end to end, learn spy complicated out automatically from original EEG signals
Sign；Width learning system is a kind of incremental learning system for not needing depth structure, it can be used as a classifier to carry out
The classification and identification of mood, effect is good and speed is very fast；Therefore this deep learning+width study algorithm is to the state of mind
Assessment it is more accurate；
4, the present invention not only can play audio using loudspeaker and In-Ear Headphones, can also be broadcast with bone conduction earphone
Playback frequency；Bone conduction earphone converts sound to the mechanical oscillation signal of different frequency, passes through the skull of people, human body and neural group
It knits to transmit；The sound conduction mode of sound wave is generated by vibrating diaphragm relative to tradition, osteoacusis eliminates many sound wave transmitting
The step of；And clearly sound-reducing can be realized in a noisy environment, compared to traditional In-Ear Headphones and loudspeaker
With outstanding audio result of broadcast, and sound wave will not influence other people because of spreading in air；
5, state of mind evaluation module ties collected physiological signal and the assessment of the obtained state of mind in the present invention
Fruit is sent to cloud platform, facilitates data storage, processing and the data analysis operation for realizing cloud platform, is convenient for model correction.
The above embodiment is a preferred embodiment of the present invention, but embodiments of the present invention are not by above-described embodiment
Limitation, other any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present invention,
It should be equivalent substitute mode, be included within the scope of the present invention.
Claims (9)
1. a kind of state of mind intervention and regulating system based on biofeedback, it is characterised in that: include:
For scene source file to be projected into the projection module of image；
Audio playing module for playing out audio source file corresponding with projection image；
For acquiring the physiological signal collection module of subject's physiological signal；
For carrying out Emotion identification and state of mind assessment to the collected physiological signal of physiological signal collection module, spirit is obtained
The state of mind evaluation module of condition evaluation results；
And state of mind assessment result for being obtained according to state of mind evaluation module adjusts scene source file and audio
Source file, and feed back to the state of mind of projection module and audio playing module and intervene adjustment module.
2. the state of mind intervention and regulating system according to claim 1 based on biofeedback, it is characterised in that: also wrap
The cloud platform for storing and managing state of mind assessment result and physiological signal is included, cloud platform and state of mind evaluation module are believed
Number connection.
3. a kind of work side of the state of mind intervention and regulating system according to claim 1 or 2 based on biofeedback
Method, characterized by the following steps:
Scene source file is projected into image and watched for subject by S1 step, projection module, carries out visual stimulus to subject；Simultaneously
Audio playing module plays out audio source file corresponding with projection image, carries out auditory stimulation to subject；
S2 step, physiological signal collection module acquire the physiological signal of subject；Physiological signal include EEG signals, heart rate signal,
Any one of electromyography signal, skin electric signal, facial expression signal and voice signal or two or more；
S3 step, state of mind evaluation module pre-processes physiological signal, feature extraction and Emotion identification, to the essence of subject
Refreshing state is assessed, and state of mind assessment result is obtained；
S4 step, the state of mind intervenes adjustment module and adjusts scene source file according to state of mind assessment result, and feeds back to projection
Module, to adjust the projection image of projection module；Audio source file is adjusted simultaneously, and feeds back and arrives audio playing module, to play
Music corresponding with projection image.
4. working method according to claim 3, it is characterised in that: in S1 step, projection module is by scene source file
It projects into image to watch for subject, visual stimulus is carried out to subject, refer to: projection module is holographic projection module, holographic
Scene source file is projected into image and watched for subject by projection module, carries out visual stimulus to subject.
5. working method according to claim 3, it is characterised in that: in the S1 step, audio playing module will be with projection
The corresponding audio source file of image plays out, and carries out auditory stimulation to subject, refer to: audio playing module is loudspeaker
Or In-Ear Headphones or bone conduction earphone；Loudspeaker or In-Ear Headphones or bone conduction earphone will sounds corresponding with projection image
Frequency source file plays out, and carries out auditory stimulation to subject.
6. working method according to claim 3, it is characterised in that: in the S2 step, the acquisition of physiological signal collection module
The physiological signal of subject, refers to: the EEG signals of physiological signal collection module acquisition subject；
S3 step, state of mind evaluation module pre-processes physiological signal, feature extraction and Emotion identification, to the essence of subject
Refreshing state is assessed, and is obtained state of mind assessment result, is referred to: state of mind evaluation module locates EEG signals in advance
Reason, and feature extraction and Emotion identification are carried out using Dynamic Graph convolutional neural networks algorithm combination width learning system, to subject
The state of mind of person is assessed, and state of mind assessment result is obtained.
7. working method according to claim 6, it is characterised in that: the state of mind evaluation module is to EEG signals
It is pre-processed, and feature extraction and mood knowledge is carried out using Dynamic Graph convolutional neural networks algorithm combination width learning system
Not, the state of mind of subject is assessed, obtains state of mind assessment result, refers to: using independent composition analysis algorithm
The eye electricity in EEG signals, electrocardio, electromagnetic interference artefact are removed with Principal Component Analysis Algorithm to realize pretreatment；Using Dynamic Graph
Convolutional neural networks algorithm extracts the features of EEG signals, EEG signals is mapped to feature space, then in feature space
It is middle that EEG signals progress mood Classification and Identification is obtained as classifier by the strong of each mood classification using width learning system
Angle value；The state of mind of subject is assessed according to the intensity value of each mood classification.
8. working method according to claim 7, it is characterised in that: the mood classification include it is tired, depressed, dejected and
It is boring；The state of mind that subject is assessed according to the intensity value of each mood classification, refers to: setting tired normal value, depression just
Constant value, dejected normal value and boring normal value, judge respectively the ratio of fatigue strength values and tired normal value, depressed intensity value with
The ratio of depressed normal value, the ratio of dejected intensity value and dejected normal value and boring intensity value assess quilt with boring normal value
The state of mind of examination person.
9. working method according to claim 3, it is characterised in that: in the S4 step, the state of mind intervenes adjustment module
Scene source file is adjusted according to state of mind assessment result, and feeds back and arrives projection module, to adjust the projection image of projection module；
Audio source file is adjusted simultaneously, and feeds back and arrives audio playing module, to play music corresponding with projection image, is referred to: being adopted
With one of the following two kinds scheme:
One, state of mind intervention adjustment module intervenes the projection in adjustment module according to state of mind assessment result from the state of mind
Scene library recalls corresponding scene source file, and scene source file is sent to projection module, and projection module is to the field received
Scape source file is projected to adjust projection image；The state of mind intervenes adjustment module and intervenes adjustment module from the state of mind simultaneously
In broadcasting audio repository recall audio source file corresponding with projection image, and audio source file is sent to audio and plays mould
Block, audio playing module plays out the audio source file received so that audio playing module broadcasting is opposite with projection image
The music answered；
Two, the state of mind is intervened adjustment module and is instructed according to the corresponding projection signal of state of mind assessment result generation, concurrently
It send to projection module；Projection module instructs the projection scene library from projection module to recall corresponding scene according to projection signal
Source file is projected to adjust projection image；It is corresponding with projection signal's instruction to intervene adjustment module generation for the state of mind simultaneously
Audio signal instruction, and be sent to audio playing module；Audio playing module instructs from audio according to audio signal and plays mould
Broadcasting audio repository in block recalls corresponding audio source file and plays out so that audio playing module plays and projection image
Corresponding music.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811431066.0A CN109620257B (en) | 2018-11-28 | 2018-11-28 | Mental state intervention and regulation system based on biofeedback and working method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811431066.0A CN109620257B (en) | 2018-11-28 | 2018-11-28 | Mental state intervention and regulation system based on biofeedback and working method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109620257A true CN109620257A (en) | 2019-04-16 |
| CN109620257B CN109620257B (en) | 2020-12-22 |
Family
ID=66069854
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811431066.0A Active CN109620257B (en) | 2018-11-28 | 2018-11-28 | Mental state intervention and regulation system based on biofeedback and working method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109620257B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110110776A (en) * | 2019-04-28 | 2019-08-09 | 大连理工大学 | A kind of tranquillization state plural number fMRI data ICA-CNN taxonomy model of patient and Healthy People |
| CN110742603A (en) * | 2019-10-31 | 2020-02-04 | 华南理工大学 | Brain wave audible mental state detection method and system for realizing same |
| CN110947076A (en) * | 2019-11-27 | 2020-04-03 | 华南理工大学 | Intelligent brain wave music wearable device capable of adjusting mental state |
| CN111345784A (en) * | 2020-05-09 | 2020-06-30 | 中国人民解放军海军特色医学中心 | Wearable protective equipment based on individual intelligent bone conduction auditory feedback technology |
| CN112515680A (en) * | 2019-09-19 | 2021-03-19 | 中国科学院半导体研究所 | Wearable brain electrical fatigue monitoring system |
| CN112618913A (en) * | 2021-01-06 | 2021-04-09 | 中国科学院心理研究所 | Multi-mode-based self-adaptive emotion adjusting system and method |
| CN113368365A (en) * | 2021-05-21 | 2021-09-10 | 苏州声动医疗科技有限公司 | Sound vibration regulation and control equipment and method for brain function monitoring, headrest and head-mounted equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120049998A1 (en) * | 2010-08-24 | 2012-03-01 | Lg Electronics Inc. | Mobile terminal and method of controlling operation of the mobile terminal |
| CN105920735A (en) * | 2016-06-15 | 2016-09-07 | 贾东明 | Physical and psychological recovery feedback training device |
| CN107106047A (en) * | 2015-05-27 | 2017-08-29 | 默林数字综合贸易有限责任公司 | Biofeedback virtual reality system and method |
| US20170358229A1 (en) * | 2010-12-22 | 2017-12-14 | Brightstar Learning | Monotonous game-like task to promote effortless automatic recognition of sight words |
| CN107714057A (en) * | 2017-10-01 | 2018-02-23 | 南京邮电大学盐城大数据研究院有限公司 | A kind of three classification Emotion identification model methods based on convolutional neural networks |
| WO2018063521A1 (en) * | 2016-09-29 | 2018-04-05 | Intel Corporation | Methods and apparatus for identifying potentially seizure-inducing virtual reality content |
| WO2018129211A1 (en) * | 2017-01-04 | 2018-07-12 | StoryUp, Inc. | System and method for modifying biometric activity using virtual reality therapy |
-
2018
- 2018-11-28 CN CN201811431066.0A patent/CN109620257B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120049998A1 (en) * | 2010-08-24 | 2012-03-01 | Lg Electronics Inc. | Mobile terminal and method of controlling operation of the mobile terminal |
| US20170358229A1 (en) * | 2010-12-22 | 2017-12-14 | Brightstar Learning | Monotonous game-like task to promote effortless automatic recognition of sight words |
| CN107106047A (en) * | 2015-05-27 | 2017-08-29 | 默林数字综合贸易有限责任公司 | Biofeedback virtual reality system and method |
| CN105920735A (en) * | 2016-06-15 | 2016-09-07 | 贾东明 | Physical and psychological recovery feedback training device |
| WO2018063521A1 (en) * | 2016-09-29 | 2018-04-05 | Intel Corporation | Methods and apparatus for identifying potentially seizure-inducing virtual reality content |
| WO2018129211A1 (en) * | 2017-01-04 | 2018-07-12 | StoryUp, Inc. | System and method for modifying biometric activity using virtual reality therapy |
| CN107714057A (en) * | 2017-10-01 | 2018-02-23 | 南京邮电大学盐城大数据研究院有限公司 | A kind of three classification Emotion identification model methods based on convolutional neural networks |
Non-Patent Citations (3)
| Title |
|---|
| C. L. PHILIP CHEN: "Broad learning system: A new learning paradigm and system without going deep", 《 2017 32ND YOUTH ACADEMIC ANNUAL CONFERENCE OF CHINESE ASSOCIATION OF AUTOMATION (YAC)》 * |
| T. SONG: "EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks", 《IEEE TRANSACTIONS ON AFFECTIVE COMPUTING》 * |
| VIKAS CHAUHAN: "on the construction of hierarchical broad learning neural network: alternative way of deep learning", 《IEEE SYMPOSIUM SERIES ON COMPUTATIONAL INTERLLIGENCE SSCI 2018》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110110776A (en) * | 2019-04-28 | 2019-08-09 | 大连理工大学 | A kind of tranquillization state plural number fMRI data ICA-CNN taxonomy model of patient and Healthy People |
| CN112515680A (en) * | 2019-09-19 | 2021-03-19 | 中国科学院半导体研究所 | Wearable brain electrical fatigue monitoring system |
| CN112515680B (en) * | 2019-09-19 | 2023-03-31 | 中国科学院半导体研究所 | Wearable brain electrical fatigue monitoring system |
| CN110742603A (en) * | 2019-10-31 | 2020-02-04 | 华南理工大学 | Brain wave audible mental state detection method and system for realizing same |
| CN110947076A (en) * | 2019-11-27 | 2020-04-03 | 华南理工大学 | Intelligent brain wave music wearable device capable of adjusting mental state |
| CN111345784A (en) * | 2020-05-09 | 2020-06-30 | 中国人民解放军海军特色医学中心 | Wearable protective equipment based on individual intelligent bone conduction auditory feedback technology |
| CN112618913A (en) * | 2021-01-06 | 2021-04-09 | 中国科学院心理研究所 | Multi-mode-based self-adaptive emotion adjusting system and method |
| CN113368365A (en) * | 2021-05-21 | 2021-09-10 | 苏州声动医疗科技有限公司 | Sound vibration regulation and control equipment and method for brain function monitoring, headrest and head-mounted equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109620257B (en) | 2020-12-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109620257A (en) | State of mind intervention and regulating system and its working method based on biofeedback | |
| CN109585021B (en) | Mental state evaluation method based on holographic projection technology | |
| CN109350051A (en) | The head wearable device and its working method with adjusting are assessed for the state of mind | |
| US11185257B2 (en) | Hearing assistance device with brain computer interface | |
| CN109992113B (en) | MI-BCI system based on multi-scene induction and control method thereof | |
| US20200268272A1 (en) | Headgear with displaceable sensors for electrophysiology measurement and training | |
| WO2021103829A1 (en) | Personalized mental state adjustment system and method based on brainwave music | |
| EP3439327A1 (en) | Ear-worn electronic device for conducting and monitoring mental exercises | |
| CN110947076B (en) | Intelligent brain wave music wearable device capable of adjusting mental state | |
| CA2993636C (en) | Devices, systems and methods for monitoring brain activity and for enabling brain to rebalance | |
| CN108428475A (en) | Biofeedback training system based on human body physiological data monitoring and virtual reality | |
| JP5574407B2 (en) | Facial motion estimation apparatus and facial motion estimation method | |
| CN110742603A (en) | Brain wave audible mental state detection method and system for realizing same | |
| US11843919B2 (en) | Improving musical perception of a recipient of an auditory device | |
| CN107184205B (en) | Automatic knowledge memory traction method based on memory scale and induction capture of brain | |
| KR20150137453A (en) | Mobile device and control method using brainwave | |
| CN109567936B (en) | Brain-computer interface system based on auditory attention and multi-focus electrophysiology and implementation method | |
| CN108784692A (en) | A kind of Feeling control training system and method based on individual brain electricity difference | |
| CN115334969A (en) | Heart monitoring system with wireless earplugs | |
| CN208521618U (en) | Biofeedback training system based on human body physiological data monitoring and virtual reality | |
| WO2020139108A1 (en) | Method for conducting cognitive examinations using a neuroimaging system and a feedback mechanism | |
| WO2020260942A1 (en) | Assessing responses to sensory events and performing treatment actions based thereon | |
| CN219439147U (en) | Virtual reality system for emotion detection | |
| CN1554454A (en) | Multiple perception music therapeutic instrument and brain electric signal analysis method | |
| Jaimovich et al. | Contagion of physiological correlates of emotion between performer and audience: An exploratory study |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |