CN113406663B - Visual imaging system and method based on P300 and associated imaging - Google Patents
Visual imaging system and method based on P300 and associated imaging Download PDFInfo
- Publication number
- CN113406663B CN113406663B CN202110670573.5A CN202110670573A CN113406663B CN 113406663 B CN113406663 B CN 113406663B CN 202110670573 A CN202110670573 A CN 202110670573A CN 113406663 B CN113406663 B CN 113406663B
- Authority
- CN
- China
- Prior art keywords
- speckle
- random
- random speckle
- imaging
- optical image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 72
- 230000000007 visual effect Effects 0.000 title claims abstract description 32
- 238000000034 method Methods 0.000 title claims description 10
- 230000003287 optical effect Effects 0.000 claims abstract description 49
- 210000004556 brain Anatomy 0.000 claims abstract description 22
- 239000011159 matrix material Substances 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 239000005338 frosted glass Substances 0.000 claims description 6
- 230000002596 correlated effect Effects 0.000 claims 5
- 230000000875 corresponding effect Effects 0.000 claims 2
- 230000001276 controlling effect Effects 0.000 claims 1
- 238000012512 characterization method Methods 0.000 abstract description 3
- 238000001514 detection method Methods 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 6
- 230000000638 stimulation Effects 0.000 description 6
- 210000001525 retina Anatomy 0.000 description 5
- 238000011160 research Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 230000000763 evoking effect Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 230000002269 spontaneous effect Effects 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 208000014644 Brain disease Diseases 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000001747 exhibiting effect Effects 0.000 description 1
- VZCCETWTMQHEPK-QNEBEIHSSA-N gamma-linolenic acid Chemical compound CCCCC\C=C/C\C=C/C\C=C/CCCCC(O)=O VZCCETWTMQHEPK-QNEBEIHSSA-N 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 210000000239 visual pathway Anatomy 0.000 description 1
- 230000004400 visual pathway Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/4802—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Electromagnetism (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
- Eye Examination Apparatus (AREA)
Abstract
The invention relates to the technical field of imaging detection, and discloses a visual imaging system and a visual imaging method based on P300 and associated imaging, wherein the system comprises the following steps: the device comprises an active random light source, a random speckle generating module and a control module, wherein the active random light source is used for generating dynamic random speckle to irradiate the surface of an object to be imaged, exposing the surface of the object to be imaged, and controlling reflected photons of the object to be imaged after exposure to enter human eyes in a time interval for generating random speckle modulation so as to generate optical images, and each random speckle corresponds to one optical image; the electroencephalogram equipment is used for acquiring P300 signals generated by the human brain according to the optical image; the data acquisition card is used for acquiring and storing P300 signals corresponding to each random speckle according to the time sequence; and the computing device is used for carrying out cross-correlation operation on the random speckle and the P300 signal so as to obtain the associated imaging. The invention can realize imaging characterization of the visual information of human eyes.
Description
Technical Field
The invention relates to the technical field of imaging detection, in particular to a visual imaging system and method based on P300 and associated imaging.
Background
The brain is the center of advanced neural activity in the human body, has hundreds of millions of neurons, and communicates and processes human information through interconnections. The brain electrical signals can be divided into: an electroencephalogram signal and a spontaneous electroencephalogram signal are induced. The evoked electroencephalogram signal is an electroencephalogram activity formed by potential change of the brain through some external stimulus; spontaneous brain electrical signals refer to brain electrical activity that the brain spontaneously produces without external special stimulation. The P300 event-related potential is one of evoked electroencephalogram signals, and a forward peak (a wave exhibiting an upward trend with respect to the baseline) appears in the range of about 300 milliseconds after the occurrence of the small probability stimulus. Because of the individual variability, the time of occurrence of P300 varies, and fig. 1 shows a waveform of P300 about 450 milliseconds after the stimulation. The P300 potential, which is an endogenous component, is not affected by the physical characteristics of stimulation, is related to the psychological activities of perception or cognition, and is closely related to the processing processes of attention, memory, intelligence and the like. The brain-computer interface based on P300 has the advantages that a user can obtain higher identification accuracy without complex training, and the brain-computer interface has stable timeliness and high time precision characteristics.
The acquisition of visual information captured by the human eye has a positive role in scientific research, medical treatment and life. In scientific research, the method can be used as a novel technical means for researching the visual working process of human eyes, researching the information communication mode of human eyes and brains and the like, and has obvious positive effects on the biological development of the human eyes at the front. The composition can be used for research of retina of human eyes, diagnosis of visual pathway, and the like in medical treatment, and has auxiliary effect on diagnosis of brain diseases. In life, the visual enhancement device can assist in use, enhance user experience, improve man-machine interaction effect and even directly enhance user experience and effect. And can further assist the daily life of the disabled.
However, in scientific research, medical treatment and life at present, it is often difficult to obtain visual information captured by human eyes, that is, imaging and characterization of the visual information of the human eyes cannot be achieved.
Disclosure of Invention
The invention provides a visual imaging system based on P300 and associated imaging, which solves the problem that the existing imaging technology is difficult to realize imaging characterization on visual information of human eyes.
The invention discloses a visual imaging system based on P300 and associated imaging, which comprises:
the device comprises an active random light source, a random speckle generating module and a control module, wherein the active random light source is used for generating dynamic random speckle to irradiate the surface of an object to be imaged, exposing the surface of the object to be imaged, and controlling reflected photons of the object to be imaged after exposure to enter human eyes in a time interval for generating random speckle modulation so as to generate optical images, and each random speckle corresponds to one optical image;
the electroencephalogram equipment is used for acquiring P300 signals generated by the human brain according to the optical image;
the data acquisition card is used for acquiring and storing P300 signals corresponding to each random speckle according to the time sequence;
and the computing device is used for carrying out cross-correlation operation on the random speckle and the P300 signal so as to obtain the associated imaging.
Wherein the optical image is expressed by the following calculation:
P(x,y,t)＝P 0 (x,y,t)+N(x,y,t)＝S(x,y,t)×O(x,y)+N(x,y,t)；
wherein P (x, y, t) is the optical image, S (x, y, t) is dynamic random speckle, P 0 (x, y, t) is an optical image obtained under the irradiation of dynamic random speckle S (x, y, t), O (x, y) is an image of an object to be imaged, N (x, y, t) is noise light, wherein x and y are respectively the abscissa and ordinate of the optical image, and t is the time of random speckle generation.
Wherein the computing device performs a cross-correlation operation on the random speckle and the P300 signal by the following formula:
I(x,y)＝<S(x,y,t)D(t)>；
wherein, the formula represents cross-correlation operation, I (x, y) is associated imaging, S (x, y, t) is dynamic random speckle, D (t) is a distinguishing signal of the P300 signal, when the amplitude of the P300 signal is greater than a threshold value, D (t) =1, otherwise D (t) =0.
Wherein the active random light source generates dynamic random speckles which are distributed according to a sparse matrix.
Wherein the active random light source comprises: a laser and a rotatable frosted glass, wherein laser light emitted by the laser generates pseudo-thermal light through the rotating frosted glass.
Wherein the active random light source is a projector.
The invention also provides a visual imaging method based on the P300 and the associated imaging, which comprises the following steps:
s1: the method comprises the steps that dynamic random speckles generated by an active random light source are irradiated on the surface of an object to be imaged for exposure, and reflected photons of the object to be imaged after exposure are controlled to enter human eyes in a time interval of random speckle modulation so as to generate optical images, wherein each random speckle corresponds to one optical image;
s2: acquiring a P300 signal generated by the brain according to the optical image by adopting brain electrical equipment;
s3: collecting and storing P300 signals corresponding to each random speckle according to time sequence by using a data collecting card;
s4: the computing device carries out cross-correlation operation on the speckle and the P300 signal to obtain associated imaging.
Wherein, in the step S1, the optical image is expressed by the following calculation method:
P(x,y,t)＝P 0 (x,y,t)+N(x,y,t)＝S(x,y,t)×O(x,y)+N(x,y,t)；
wherein P (x, y, t) is the optical image, S (x, y, t) is dynamic random speckle, P 0 (x, y, t) is an optical image obtained under the irradiation of dynamic random speckle S (x, y, t), O (x, y) is an image of an object to be imaged, N (x, y, t) is noise light, wherein x and y are respectively the abscissa and ordinate of the optical image, and t is the time of random speckle generation.
The cross-correlation operation in the step S4 is as follows:
I(x,y)＝<S(x,y,t)D(t)>；
wherein, the formula represents cross-correlation operation, I (x, y) is associated imaging, S (x, y, t) is dynamic random speckle, D (t) is a distinguishing signal of the P300 signal, when the amplitude of the P300 signal is greater than a threshold value, D (t) =1, otherwise D (t) =0.
Wherein the active random light source generates dynamic random speckles which are distributed according to a sparse matrix.
In the visual imaging system and method based on P300 and associated imaging, the active random light source generates dynamic random speckle to irradiate on the surface of an object to be imaged, the electroencephalogram equipment acquires P300 signals generated after the brain of a person is irradiated by the random speckle to stimulate an optical image of the object to be imaged, the acquisition card acquires P300 signals corresponding to each speckle in time sequence, and the computing device carries out associated operation on the speckle and the P300 signals to obtain associated imaging, and the associated imaging is imaging representation of visual information of human eyes, so that the imaging representation of the visual information of the human eyes is realized.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are only some embodiments of the invention, and that other drawings can be obtained according to these drawings without inventive faculty for a person skilled in the art.
Fig. 1 is a P300 waveform of the human brain about 450 milliseconds after stimulation occurs.
FIG. 2 is a schematic block diagram of a visual imaging system based on P300 and associated imaging according to an embodiment of the invention;
FIG. 3 is an original image of an object to be imaged in an embodiment of the present invention;
FIG. 4 is an image of an object to be imaged under a single speckle in an embodiment of the invention;
FIG. 5 is a schematic diagram of an imaging range of a human eye according to an embodiment of the present invention;
FIG. 6 is a graph of simulation results of imaging on the retina of a human eye in an embodiment of the invention;
fig. 7 is a diagram of a correlation imaging simulation result based on P300 of an object to be imaged in an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The visual imaging system based on P300 and associated imaging of the present embodiment is shown in fig. 2, and includes: an active random light source 1, an electroencephalogram device 3, a data acquisition card 4 and a computing device 5. The active random light source 1 generates dynamic random speckles S (x, y, t) to irradiate on the surface of the object 2 to be imaged, the image of the object 2 to be imaged is O (x, y), the surface of the object 2 to be imaged is exposed, and reflected photons of the object 2 to be imaged after exposure are controlled to enter human eyes in a time interval for generating random speckle modulation so as to generate optical images P (x, y, t), wherein each random speckle corresponds to one optical image, and the optical image is an optical image imaged by retina of the human eyes. The electroencephalogram device 3 is used for acquiring P300 signals generated by the human brain from the optical image P (x, y, t). The data acquisition card 4 acquires and stores P300 signals corresponding to each random speckle according to a time sequence, and the computing device 5 carries out association operation on the speckle and the P300 signals to obtain association imaging.
In operation of the system, the active random light source 1 can control the duration of the random speckle illumination to achieve that reflected photons enter the human eye during the time interval that the random speckle modulation is generated (i.e., the time window of exposure is less than the time interval of dynamic random speckle modulation). The active random light source 1 performs multiple exposure, and the time window of each exposure is smaller than the time interval of dynamic random speckle modulation, so that the imaging result of the variable speckle is avoided, and a plurality of different speckles are overlapped. The data acquisition card 4 acquires and stores the P300 signal for a plurality of times and sends the P300 signal to the computing device 5, and then the computing device 5 is adopted to carry out association operation on the speckle and the P300 signal to obtain association imaging. The electroencephalogram device 3 may specifically be a brain-computer interface device, for example: emotiv EPOC X is used for acquiring brain-computer signals, the brain-computer interface equipment is also integrated with the data acquisition card 4, and the computing device 5 can be a computer or a singlechip and the like.
In the visual imaging system based on P300 and associated imaging in this embodiment, the active random light source 1 generates dynamic random speckle to irradiate on the surface of the object 2 to be imaged, the electroencephalogram device 3 collects P300 signals generated after the human brain is stimulated by the optical image of the object to be imaged irradiated by the random speckle, the data collection card 4 collects P300 signals corresponding to each speckle in time sequence, the computing device 5 carries out associated operation on the speckle and the P300 signals to obtain associated imaging, and the associated imaging is imaging representation of human eye visual information, so that imaging representation of the human eye visual information is realized.
Specifically, the optical image P (x, y, t) is expressed by the following calculation:
P(x,y,t)＝P 0 (x,y,t)+N(x,y,t)＝S(x,y,t)×O(x,y)+N(x,y,t)；
wherein S (x, y, t) is dynamic random speckle, P 0 (x, y, t) is an optical image obtained under the irradiation of dynamic random speckle S (x, y, t), O (x, y) is an image of an object to be imaged, N (x, y, t) is noise light, wherein x and y are respectively the abscissa and ordinate of the optical image, and t is the time of random speckle generation.
The computing means 5 performs a cross-correlation operation on the random speckle and the P300 signal by the following formula:
I(x,y)＝<S(x,y,t)D(t)>；
wherein, the formula represents cross-correlation operation, I (x, y) is associated imaging, S (x, y, t) is dynamic random speckle, D (t) is a distinguishing signal of the P300 signal, when the amplitude of the P300 signal is greater than a threshold value, D (t) =1, otherwise D (t) =0. The threshold value varies from person to person, and the specific values of the threshold values of different persons may be different, and in this embodiment, the average value+2×standard deviation of the electroencephalogram signal when the person is not stimulated may be selected. The specific operation mode is as follows: and multiplying and integrating S (x, y, t) and D (t). The random speckles are different in shape, each speckle is hit on an object, the total reflected light intensity is different, the reflected light intensity signal can be used as a stimulation signal for generating a P300 signal, when the total light intensity of P (x, y, t) is larger than the light intensity of the 80 th percentile of the statistical distribution of the total light intensity, a P300 signal with larger amplitude is generated, if the total light intensity is larger than the light intensity of the 80 th percentile of the statistical distribution of the total light intensity, D (t) =1 is obtained, otherwise D (t) =0 is obtained, and the more obvious the variation of the total light intensity is, the more the P300 signal can be excited, and the stronger the generated P300 signal is.
Specifically, the data acquisition card 4 generally performs the discrimination of whether the P300 signal exists in the signal by adopting an algorithm (such as a support vector machine, a linear discrimination model, a long-short time sequence network, etc.), if the discrimination result is that the P300 signal exists (i.e. when the amplitude of the P300 signal is greater than a threshold value), the output is 1, and if the discrimination result is that the P300 signal does not exist, the output is 0.
The active random light source 1 generates dynamic random speckles, which are speckles distributed in a sparse matrix. The more sparse the speckle, the more spatially distributed the speckle light spots are, so that the light spots striking the object are sometimes absent, and the probability of meeting the condition of the total intensity distribution of the light intensity of the P (x, y, t) is high, so that the brain is stimulated to generate P300 signals.
The active random light source 1 includes: the laser device and the rotatable frosted glass, laser emitted by the laser device generates pseudo-thermal light through the rotating frosted glass, and the pseudo-thermal light is random speckle. The active random light source 1 may also be a projector, using which random speckle is projected.
The invention also provides a visual imaging method based on the P300 and the associated imaging, which comprises the following steps:
step S1: the dynamic random speckle generated by the active random light source 1 irradiates the surface of the object 2 to be imaged for exposure, and the reflected photons of the object to be imaged after exposure are controlled to enter the human eye in the time interval of generating random speckle modulation so as to generate optical images, wherein each random speckle corresponds to one optical image.
Step S2: and acquiring a P300 signal generated by the human brain according to the optical image by adopting brain electrical equipment.
Step S3: the P300 signal corresponding to each random speckle is collected and stored by a data collection card 4 according to time sequence; and repeating the exposure and the acquisition for a plurality of times, namely repeating the steps S1 to S3, wherein the exposure time window is smaller than the time interval of dynamic random speckle modulation.
Step S4: the correlation imaging is obtained by performing a cross-correlation operation on the speckle and the P300 signal using the computing device 5.
In step S1, an optical image P (x, y, t) is obtained through the retina of the human eye, i.e. a calculation between the random speckle S (x, y, t) and the image O (x, y) of the object 2 to be imaged is achieved, the optical image P (x, y, t) being expressed by the following calculation means:
P(x,y,t)＝P 0 (x,y,t)+N(x,y,t)＝S(x,y,t)*O(x,y)+N(x,y,t)；
in the above formula, S (x, y, t) is dynamic random speckle, P 0 (x, y, t) is an optical image obtained under the irradiation of the dynamic random speckle S (x, y, t), O (x, y) is an image of the object 2 to be imaged, and N (x, y, t) is noise light.
The cross-correlation operation is: i (x, y) = < S (x, y, t) D (t) >, D (t) is a distinguishing signal of the P300 signal, I (x, y) is a correlation imaging, and the specific operation mode is: and multiplying and integrating S (x, y, t) and D (t). The random speckles are different in shape, each speckle is striking on an object, so that the reflected total light intensity is different, the reflected light intensity signal can be used as a stimulation signal for generating a P300 signal, when the amplitude of the P300 signal is greater than a threshold value, D (t) =1, otherwise D (t) =0, specifically, when the total light intensity of P (x, y, t) is greater than the light intensity of the 80 th percentile of the statistical distribution of the total light intensity, a P300 signal with a larger amplitude is generated, if the threshold value is exceeded, D (t) =1, otherwise D (t) =0, and the more obvious the difference of the change of the total light intensity is, the P300 signal can be excited, and the generated P300 signal is stronger. In general, the data acquisition card 4 adopts an algorithm (such as a support vector machine, a linear recognition model, a long-term and short-term time sequence network, etc.) to realize whether the P300 signal exists in the signals, and outputs 1 if the P300 signal exists as a recognition result, and outputs 0 if the P300 signal does not exist as a recognition result.
Referring to fig. 3, referring to fig. 4, for the original image of the object 2 to be imaged, the image of the object 2 to be imaged under random speckle, that is, the single Zhang Guangxue image obtained by dynamic random speckle S (x, y, t), it can be found that under the irradiation of random speckle, the object is partially lightened, preferably random speckle distributed in sparse mode is used, the more sparse speckle is used, and the more dispersed speckle light spots are spatially distributed, so that the light spots striking the object sometimes have no time, and the probability satisfies the condition of the total intensity distribution of the light intensity of P (x, y, t) so as to generate a P300 signal by the brain. According to the concept of a sparse matrix, when the ratio of the occupied area of light spots of random speckle on an object to the total area of the surface of the object is less than or equal to 0.05, the random speckle distributed according to the sparse matrix is obtained.
Referring to fig. 5-7, fig. 5 is a view of the area of the object 2 to be imaged seen by the human eye, which images the aligned portion, as shown in fig. 6, when illuminated with uniform illumination, is a graph of the result of simulating the image of the object seen by the human eye, i.e., the image on the retina of the human eye. Fig. 7 is an imaging representation of human eye vision.
The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.
Claims (8)
1. A visual imaging system based on P300 and associated imaging, comprising:
the device comprises an active random light source, a random speckle generating module and a control module, wherein the active random light source is used for generating dynamic random speckle to irradiate the surface of an object to be imaged, exposing the surface of the object to be imaged, and controlling reflected photons of the object to be imaged after exposure to enter human eyes in a time interval for generating random speckle modulation so as to generate optical images, and each random speckle corresponds to one optical image;
the electroencephalogram equipment is used for acquiring P300 signals generated by the human brain according to the optical image;
the data acquisition card is used for acquiring and storing P300 signals corresponding to each random speckle according to the time sequence;
a computing device for performing a cross-correlation operation on the random speckle and the P300 signal to obtain a correlated image, the computing device performing a cross-correlation operation on the random speckle and the P300 signal by the following formula:
I(x,y)＝<S(x,y,t)D(t)>；
wherein, I (x, y) is correlation imaging, S (x, y, t) is dynamic random speckle, D (t) is a distinguishing signal of the P300 signal, wherein x and y are respectively the abscissa of the optical image, t is the time of random speckle generation, D (t) =1 when the amplitude of the P300 signal is greater than a threshold value, otherwise D (t) =0.
2. The P300 and correlated imaging based visual imaging system of claim 1, wherein said optical image is expressed by the following computational manner:
P(x,y,t)＝P 0 (x,y,t)+N(x,y,t)＝S(x,y,t)×O(x,y)+N(x,y,t)；
wherein P (x, y, t) is the optical image, S (x, y, t) is dynamic random speckle, P 0 (x, y, t) is an optical image obtained under the irradiation of dynamic random speckle S (x, y, t), O (x, y) is an image of an object to be imaged, N (x, y, t) is noise light, wherein x and y are respectively the abscissa and ordinate of the optical image, and t is the time of random speckle generation.
3. The P300 and correlated imaging based visual imaging system of claim 1, wherein said active random light source produces dynamic random speckle as speckle distributed in a sparse matrix.
4. A visual imaging system based on P300 and associated imaging as set forth in any of claims 1-3, wherein said active random light source comprises: a laser and a rotatable frosted glass, wherein laser light emitted by the laser generates pseudo-thermal light through the rotating frosted glass.
5. A visual imaging system based on P300 and associated imaging as claimed in any of claims 1 to 3, wherein said active random light source is a projector.
6. A visual imaging method based on P300 and associated imaging, comprising the steps of:
s1: the method comprises the steps that dynamic random speckles generated by an active random light source are irradiated on the surface of an object to be imaged for exposure, and reflected photons of the object to be imaged after exposure are controlled to enter human eyes in a time interval of random speckle modulation so as to generate optical images, wherein each random speckle corresponds to one optical image;
s2: acquiring a P300 signal generated by the brain according to the optical image by adopting brain electrical equipment;
s3: collecting and storing P300 signals corresponding to each random speckle according to time sequence by using a data collecting card;
s4: the computing device performs cross-correlation operation on the speckle and the P300 signal to obtain a correlated image, and performs cross-correlation operation on the random speckle and the P300 signal through the following formula:
I(x,y)＝<S(x,y,t)D(t)>；
wherein, I (x, y) is correlation imaging, S (x, y, t) is dynamic random speckle, D (t) is a distinguishing signal of the P300 signal, wherein x and y are respectively the abscissa of the optical image, t is the time of random speckle generation, D (t) =1 when the amplitude of the P300 signal is greater than a threshold value, otherwise D (t) =0.
7. The visual imaging method based on P300 and associated imaging according to claim 6, wherein in said step S1, the optical image is expressed by the following calculation:
P(x,y,t)＝P 0 (x,y,t)+N(x,y,t)＝S(x,y,t)×O(x,y)+N(x,y,t)；
wherein P (x, y, t) is the optical image, S (x, y, t) is dynamic random speckle, P 0 (x, y, t) is an optical image obtained under the irradiation of dynamic random speckle S (x, y, t), O (x, y) is an image of an object to be imaged, N (x, y, t) is noise light, wherein x and y are respectively the abscissa and ordinate of the optical image, and t is the time of random speckle generation.
8. The P300 and correlated imaging based visual imaging method of claim 7, wherein said active random light source produces dynamic random speckle as speckle distributed in a sparse matrix.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110670573.5A CN113406663B (en) | 2021-06-17 | 2021-06-17 | Visual imaging system and method based on P300 and associated imaging |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110670573.5A CN113406663B (en) | 2021-06-17 | 2021-06-17 | Visual imaging system and method based on P300 and associated imaging |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113406663A CN113406663A (en) | 2021-09-17 |
| CN113406663B true CN113406663B (en) | 2024-03-12 |
Family
ID=77684609
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110670573.5A Active CN113406663B (en) | 2021-06-17 | 2021-06-17 | Visual imaging system and method based on P300 and associated imaging |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113406663B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006295253A (en) * | 2005-04-05 | 2006-10-26 | Hiroshima Univ | Visual information processing apparatus and visual information processing method thereof |
| JP2017076193A (en) * | 2015-10-13 | 2017-04-20 | 株式会社国際電気通信基礎技術研究所 | Brain activity analysis device, brain activity analysis method and brain activity analysis program |
| KR20180136663A (en) * | 2017-06-15 | 2018-12-26 | 단국대학교 천안캠퍼스 산학협력단 | Imaging device for multi-spectral laser image having speckle reducer and imaging method thereof |
| CN110236482A (en) * | 2019-05-31 | 2019-09-17 | 中国科学院苏州生物医学工程技术研究所 | Integrated eye brain visual performance imaging system |
| CN110353671A (en) * | 2019-07-09 | 2019-10-22 | 杭州绎杰检测科技有限公司 | A kind of visual fixations location measurement method based on video modulation and EEG signals |
| CN110708471A (en) * | 2019-09-27 | 2020-01-17 | 西安交通大学 | CCD self-correlation imaging system and method based on active illumination |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9588490B2 (en) * | 2014-10-21 | 2017-03-07 | City University Of Hong Kong | Neural control holography |
-
2021
- 2021-06-17 CN CN202110670573.5A patent/CN113406663B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006295253A (en) * | 2005-04-05 | 2006-10-26 | Hiroshima Univ | Visual information processing apparatus and visual information processing method thereof |
| JP2017076193A (en) * | 2015-10-13 | 2017-04-20 | 株式会社国際電気通信基礎技術研究所 | Brain activity analysis device, brain activity analysis method and brain activity analysis program |
| KR20180136663A (en) * | 2017-06-15 | 2018-12-26 | 단국대학교 천안캠퍼스 산학협력단 | Imaging device for multi-spectral laser image having speckle reducer and imaging method thereof |
| CN110236482A (en) * | 2019-05-31 | 2019-09-17 | 中国科学院苏州生物医学工程技术研究所 | Integrated eye brain visual performance imaging system |
| CN110353671A (en) * | 2019-07-09 | 2019-10-22 | 杭州绎杰检测科技有限公司 | A kind of visual fixations location measurement method based on video modulation and EEG signals |
| CN110708471A (en) * | 2019-09-27 | 2020-01-17 | 西安交通大学 | CCD self-correlation imaging system and method based on active illumination |
Non-Patent Citations (2)
| Title |
|---|
| 吴义根 等.人脑初级视皮层内事件相关型视觉刺激fMRI BOLD响应特性初步研究.波谱学杂志.2003,第20卷(第3期),第209-213. * |
| 用于人眼视网膜成像照明的激光消散斑技术研究;李抄;姜宝光;夏明亮;程少园;宣丽;;光学学报;第28卷(第12期);第2245-2249页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113406663A (en) | 2021-09-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110765920B (en) | Motor imagery classification method based on convolutional neural network | |
| Ngai et al. | Emotion recognition based on convolutional neural networks and heterogeneous bio-signal data sources | |
| Petrantonakis et al. | Emotion recognition from brain signals using hybrid adaptive filtering and higher order crossings analysis | |
| Saganowski et al. | Emotion recognition using wearables: A systematic literature review-work-in-progress | |
| CN104871160B (en) | System and method for feeling and recognizing anatomy | |
| Becedas | Brain–machine interfaces: basis and advances | |
| Ahmadian et al. | Constrained blind source extraction of readiness potentials from EEG | |
| Brunet et al. | Human visual cortical gamma reflects natural image structure | |
| US10290139B2 (en) | Method for expressing social presence of virtual avatar, using change in pupil size according to heartbeats, and system employing same | |
| KR102712314B1 (en) | Apparatus for controlling customized driving based on brain wave and method thereof | |
| Joyce et al. | Early selection of diagnostic facial information in the human visual cortex | |
| Cotrina et al. | A SSVEP-BCI setup based on depth-of-field | |
| López et al. | Comparison of classification techniques for the control of EOG-based HCIs | |
| Gurariy et al. | Using high-density electroencephalography to explore spatiotemporal representations of object categories in visual cortex | |
| CN113406663B (en) | Visual imaging system and method based on P300 and associated imaging | |
| Fickling et al. | Distant sensor prediction of event-related potentials | |
| Zakrzewski et al. | VR-oriented EEG signal classification of motor imagery tasks | |
| Prasad et al. | SSVEP signal detection for BCI application | |
| Li et al. | Single trial EEG classification applied to a face recognition experiment using different feature extraction methods | |
| Tahira et al. | Eeg based mental stress detection using deep learning techniques | |
| JARILLO-SILVA et al. | Emotion classification from EEG signals using wearable sensors: pilot test | |
| CN108451528A (en) | Change the method and system for inferring electroencephalogram frequency spectrum based on pupil | |
| Bentley et al. | An offline auditory P300 brain-computer interface using principal and independent component analysis techniques for functional electrical stimulation application | |
| Zhang | Computer Vision Overview | |
| Guo et al. | Phosphene object perception employs holistic processing during early visual processing stage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |