CN105468738A - Image retrieval method based on combination of eye movement and electroencephalogram - Google Patents
Image retrieval method based on combination of eye movement and electroencephalogram Download PDFInfo
- Publication number
- CN105468738A CN105468738A CN201510822183.XA CN201510822183A CN105468738A CN 105468738 A CN105468738 A CN 105468738A CN 201510822183 A CN201510822183 A CN 201510822183A CN 105468738 A CN105468738 A CN 105468738A
- Authority
- CN
- China
- Prior art keywords
- image
- retrieved
- eye movement
- eye
- zone
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/53—Querying
- G06F16/532—Query formulation, e.g. graphical querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Dermatology (AREA)
- General Health & Medical Sciences (AREA)
- Neurology (AREA)
- Neurosurgery (AREA)
- Mathematical Physics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Library & Information Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an image retrieval method based on the combination of eye movement and electroencephalogram, and mainly solves the problems of large retrieval amount and low retrieval efficiency in the prior art. The image retrieval method comprises the following implementation steps: 1) observing an image to be retrieved by a tested person, and extracting the eye movement data and the electroencephalogram data of the tested person by an eye tracker and an electroencephalograph; 2) combining eye movement data with electroencephalogram data, and extracting the hot area graph of the image to be retrieved; 3) extracting a target image to be retrieved from the generated hot area graph; and 4) in an image library, carrying out image retrieval on the target image to be retrieved by full search or an image retrieval way based on a SIFT (Scale Invariant Feature Transform) algorithm. The image retrieval method based on the combination of the eye movement and the electroencephalogram carries out the image retrieval on the target image in the image library, lowers a retrieval amount, improves retrieval efficiency and can be used for object recognition, target identification and image tracking.
Description
Technical field
The invention belongs to technical field of image processing, further relate to a kind of image search method, can be used for object identification, target identification, image tracing.
Background technology
At present, image retrieval technologies is mainly divided into following two classes:
One class is text based image retrieval technologies, namely carrys out retrieving images by the author of image, the information such as title and creative time.This method has a lot of defect: one is that image itself comprises quantity of information too greatly, and word is not enough to contain so many content; Two are mark large nuber of images is also a huge task.
Another kind of is CBIR technology, namely the technology such as image procossing, pattern-recognition is adopted, extract the low-level image features such as color, shape and the texture in view data, based on these low-level image features, when carrying out image retrieval, the retrieving images given to a view picture extracts proper vector, sets up feature database, similarity matching algorithm is utilized to calculate the proper vector of query instance image and the similarity of proper vector in feature database, according to the size output result for retrieval of similarity.But developing rapidly along with infotech, image information is expanded day by day, if picture in its entirety is applied in image retrieval, not only can brings the tediously long complexity of computation process, increase volumes of searches, and recall precision is reduced greatly.
Summary of the invention
The present invention is directed to the deficiency that above-mentioned prior art exists, propose and a kind ofly to move and the image search method that combines of brain electricity based on eye, effectively to reduce volumes of searches, improve recall precision.
Technical scheme of the present invention is achieved in that
One. know-why
Eye movement technique, is by the record to eye movement, therefrom extracts the data such as blinkpunkt, fixation time, twitching of the eyelid, thus the inherent cognitive process that research is individual.Brain power technology then mainly provides the information of the brain course of work by indexs such as the spatial frequencys of wave amplitude, latent period and current potential or electric current.Brain electricity directly can reflect neural electrical activity, and have high resolution, therefore it is subject to the attention of researcher, becomes a kind of Cognitive Neuroscience means of comparative maturity.
Although eye movement technique effectively can infer individual inherent cognitive process, but can not directly illustration information processing physiological mechanism, and brain electricity index can reflect the physiology course of Information procession in brain, combine dynamic for eye with brain power technology, not only bring the breakthrough in scientific research, and enrich the achievement in research of cognitive science.
Because human brain is not swallow when experiencing environmental stimuli, but have subjective hobby, thus produce the otherness of attention level.The present invention moves tracer technique by eye and brain electricity can record out this otherness intuitively, thus is extracted in vision attention district and retrieve.This methods combining visual information intuitively, region of interest region is extracted from original image, and set up feature database by the low-level image feature extracting region of interest image, similarity matching algorithm is utilized to calculate the proper vector of query instance image and the similarity of proper vector in feature database, according to the size output result for retrieval of similarity.
Two. technical scheme
According to above-mentioned principle, the present invention provides the following two kinds technical scheme:
Technical scheme 1: a kind ofly to move and the image search method that combines of brain electricity based on eye, comprising:
(1) original image to be retrieved is inputted;
(2) eye is moved equipment and brain electricity equipment connect one by test-run a machine simultaneously, original image to be retrieved is presented on by test-run a machine, allows subject observe these pictures, and record eye movement data and the eeg data of subject simultaneously;
(3) generate the hot-zone figure of image to be retrieved according to the eye movement data recorded and eeg data, hot-zone figure represents area-of-interest with the rgb color of 0-255, and includes temporal information and the coordinate information of set time section fixation point;
(4) from the hot-zone figure generated, target image to be retrieved is extracted;
(4a) by Matlab software, all coordinates in the figure of hot-zone are showed in former figure;
(4b) centered by the coordinate of each fixation point, obtain the rectangular block of pixels containing 100 pixels, extract this block of pixels, as target image to be detected;
(5) in image library, retrieve target image to be retrieved by full way of search, the image according to retrieving calculates recall ratio.Recall ratio is in an image retrieval procedure, and the associated picture quantity retrieved accounts for the ratio of all associated picture quantity.
Technical scheme 2: a kind ofly to move and the image search method that combines of brain electricity based on eye, comprising:
1) original image to be retrieved is inputted;
2) eye is moved equipment and brain electricity equipment connect one by test-run a machine simultaneously, original image to be retrieved is presented on by test-run a machine, allows subject observe these pictures, and record eye movement data and the eeg data of subject simultaneously;
3) generate the hot-zone figure of image to be retrieved according to the eye movement data recorded and eeg data, hot-zone figure represents area-of-interest with the rgb color of 0-255, and includes temporal information and the coordinate information of set time section fixation point;
4) from the hot-zone figure generated, target image to be retrieved is extracted;
4a) by Matlab software, all coordinates in the figure of hot-zone are showed in former figure to be retrieved;
4b) calculate the barycenter of these fixation point coordinate, centered by this barycenter, generate the rectangle frame comprising all fixation point, this rectangle frame is extracted, as target image to be retrieved;
5) in image library, with the image search method based on scale invariant feature conversion SIFT algorithm, target image to be retrieved is retrieved, and calculate recall ratio.
The present invention compared with prior art has the following advantages:
The first, the present invention combines by eye movement data and eeg data the area-of-interest extracting subject, avoids and retrieves picture in its entirety, effectively reduce volumes of searches.
The second, the present invention for clue is retrieved with the area-of-interest of subject, makes retrieval more press close to the intention of people, effectively raises recall precision.
Accompanying drawing explanation
Fig. 1 is realization flow figure of the present invention;
Fig. 2 is experimental facilities connection layout of the present invention;
Fig. 3 is full way of search sub-process figure in the present invention;
Fig. 4 is the image retrieval sub-process figure based on scale invariant feature conversion SIFT algorithm in the present invention;
Fig. 5 is the picture to be retrieved used during the present invention tests;
Fig. 6 is the hot-zone figure extracted during the present invention tests;
Fig. 7 is the target image for entirely searching for extracted during the present invention tests;
Fig. 8 is by the result that full way of search retrieves during the present invention tests;
Fig. 9 is the target image for the retrieval of scale invariant feature conversion SIFT algorithm extracted during the present invention tests;
Figure 10 is the scale invariant feature conversion SIFT feature point extracted during the present invention tests;
Figure 11 is the coupling that the present invention tests mesoscale invariant features conversion SIFT feature point;
Figure 12 is the image searching result based on scale invariant feature conversion SIFT algorithm during the present invention tests;
Embodiment
Below in conjunction with accompanying drawing, step of the present invention is described in further detail.
Embodiment 1: the image retrieval of full way of search
Step 1, builds experiment porch.
With reference to Fig. 2, experimental facilities of the present invention comprises the CRT monitor of EyeLink1000 eye tracker, Netroscancurry7 electroencephalograph and 19 inches, eye tracker and electroencephalograph are connected over the display simultaneously, pass through uniting and adjustment, enable eye tracker and electroencephalograph synchronous recording data, and respectively eye tracker and electroencephalograph are calibrated.
Step 2, gathers eye movement data and eeg data.
Subject is sitting in a sound insulation and in quiet room, by the EEG signals of electroencephalograph record subject, sample frequency is set to 1000Hz;
In experimentation, require that lower jaw is placed in jaw holder by subject, fixing distance between subject's eyes and screen is 60cm, and by the eye movement of eye tracker record subject, sample frequency is 500Hz, and namely every 2ms records once eye and moves position;
Presented by retrieving image over the display, spatial resolution is 1280 × 1024, and refreshing frequency is 85Hz, and subject selects interested image content to carry out concentrating viewing voluntarily, and eye tracker and electroencephalograph gather eye movement data and the eeg data of subject respectively.
Step 3, generates the hot-zone figure of image to be retrieved according to the eye movement data recorded and eeg data.
(3a) according to the change of the eeg data of subject when seeing oneself part interested, the time period of this change is obtained;
(3b) filter out the eye movement data of this time period, and this eye movement data is shown as the form of form;
(3c) eye movement data obtained is input in DataViewer software, generates the hot-zone figure of respective image.
Described DataViewer software is the eye movement data analysis software that EyeLink1000 eye tracker carries, and can generate the hot-zone figure of corresponding picture according to eye movement data.
Step 4, extracts target image to be retrieved from the hot-zone figure generated.
(4a) by Matlab software, all coordinates in the figure of hot-zone are showed in former figure;
(4b) centered by the coordinate of each fixation point, obtain the rectangular block of pixels containing 100 pixels, extract this block of pixels, as target image to be detected.
Described Matlab software is the business mathematics software of U.S. MathWorks Company, can carry out matrix operation, draw function and the function such as data, implementation algorithm.
Step 5, based on full-search algorithm, retrieves target image to be detected.
Full-search algorithm also claims exhaustive search algorithm, is to mate position candidate all possible in hunting zone, therefrom finds out the position of mating most.
With reference to Fig. 3, the concrete implementation step of the image retrieval based on full-search algorithm is as follows:
(5a) read the image in picture library, this Iamge Segmentation is become and block of pixels to be searched multiple block of pixels of a size, has lap between each block of pixels, to increase search accuracy rate;
(5b) from image to be searched from the block of pixels of first, upper left, calculate the Euclidean distance between two block of pixels, Euclidean distance is less, and the similarity of two targets is higher:
Because the pixel extracted is fast less, range estimation value is set to zero, and when the Euclidean distance between appearance two block of pixels is zero, then the match is successful, starts counting simultaneously, if not zero, then do not count, according to continuing traversal from top left to bottom right;
Piece image is after overmatching, if the counting that Euclidean distance is zero is more than or equal to 1, then judges that this figure is the similar pictures of image to be searched; If the counting that Euclidean distance is zero is less than 1, then judge that this figure is not the similar image of image to be searched, proceed search;
(5c) according to counting size, descending sort is carried out to retrieving images, counts larger, illustrate that the content of two width figure is more similar, and show front 9 images the most similar.
Embodiment 2: based on the image retrieval of scale invariant feature conversion SIFT algorithm.
Step one, builds experiment porch.
Step 2, gathers eye movement data and eeg data.
Step 3, generates the hot-zone figure of image to be retrieved according to the eye movement data recorded and eeg data.
The realization of above-mentioned steps one to step 3 is identical with the step 1-step 3 of embodiment 1.
Step 4, extracts target image to be retrieved from the hot-zone figure generated.
4a) by Matlab software, all coordinates in the figure of hot-zone are showed in former figure;
4b) calculate the barycenter of these fixation point coordinate, centered by this barycenter, generate the rectangle frame comprising all fixation point, this rectangle frame is extracted, as target image to be retrieved;
Step 5, based on the image search method of scale invariant feature conversion SIFT algorithm.
Scale invariant feature conversion SIFT algorithm is a kind of algorithm extracting local feature, from image, mainly extract the proper vector had nothing to do to scaling, rotation, brightness change; See Lowe, D.:Objectrecognitionfromlocalscale-invariantfeatures.In: Proc.ICCV.
With reference to Fig. 4, the realization that this step carries out image retrieval based on scale invariant feature conversion SIFT algorithm is as follows:
5a) use scale invariant feature conversion SIFT algorithm to extract the scale invariant feature conversion SIFT feature point of target image to be retrieved, and generate corresponding scale invariant feature conversion SIFT feature vector;
5b) read the image in picture library, use scale invariant feature conversion SIFT algorithm to extract the scale invariant feature conversion SIFT feature point of retrieving images, and generate corresponding scale invariant feature conversion SIFT feature vector;
5c) after the scale invariant feature conversion SIFT feature vector of target image to be retrieved and retrieving images generates, calculate the Euclidean distance between unique point proper vector in two width images, measure the similarity in this two width image between unique point by Euclidean distance:
For certain unique point in target image to be retrieved, calculate the Euclidean distance between all unique points and this unique point in retrieving images, find unique point nearest with this unique point Euclidean distance in retrieving images and secondary near unique point, if recently Euclidean distance is less than the threshold value 0.6 of setting except the value of nearly Euclidean distance gained in proper order, then can judge that minimum a pair unique point of Euclidean distance is as the unique point matched, and start counting, until unique points all in target image to be retrieved all mates end;
5d) according to counting size, descending sort is carried out to retrieving images, counts larger, illustrate that two width figure are more similar, and show front 9 images the most similar.
Effect of the present invention is further illustrated by following experiment:
1, experiment condition
Experimental Hardware equipment: Netroscancurry7 electroencephalograph, EyeLink1000 eye tracker, the CRT monitor of 19 inches;
Experiment software platform: the MATLABR2012a under Windows7 operating system;
2, experiment content and result
Experiment one, retrieves 98 pictures in picture library by full way of search, and calculates recall ratio.
The first step, carries out data acquisition to Fig. 5 picture to be retrieved, and extract hot-zone figure according to the data gathered, as shown in Figure 6, from the figure Fig. 6 of hot-zone, extract target image, result as shown in Figure 7 for result.
Second step, in image library, retrieve the target image Fig. 7 in the first step with all direction search method, the parts of images retrieved as shown in Figure 8.
Fig. 8 shows, in all direction search method, recall ratio can be adopted to show the achievement of image retrieval.Recall ratio is in an image retrieval procedure, and the associated picture quantity retrieved accounts for the ratio of all associated picture quantity.In this experiment one, retrieve 98 images of picture library, retrieve 78 images similar to target image, the recall ratio obtained is 79.59%, and recall ratio is higher, can realize the retrieval to target image.
Experiment two, retrieves 98 pictures in picture library with the image search method based on scale invariant feature conversion SIFT algorithm, and calculates recall ratio.
The first step, carries out data acquisition with the present invention to Fig. 5, and extract hot-zone figure according to the data gathered, result as shown in Figure 6, extracts target image from the figure of hot-zone, and result as shown in Figure 9.
Second step, extract the scale invariant feature conversion SIFT feature point of target image Fig. 9 to be retrieved and retrieving images, result as shown in Figure 10, wherein Figure 10 (a) is the scale invariant feature conversion SIFT feature point of target image, and Figure 10 (b) is searching image scale invariant feature conversion SIFT feature point.
3rd step, mate the target image to be retrieved extracted in Figure 10 and retrieving images scale invariant feature conversion SIFT feature point, result as shown in figure 11.
4th step, in image library, change SIFT feature Point matching to the scale invariant feature that every width image all carries out as shown in figure 11, the result retrieval according to coupling goes out the image similar to target image, and the parts of images retrieved as shown in figure 12.
Figure 12 shows, in experiment two, retrieves 98 images of picture library, and retrieve 50 images similar to target image, the recall ratio obtained is about 50%, can realize the retrieval to target image.
To sum up, the present invention adopts and eye is moved the method combined with brain electricity and can be applicable in image retrieval.
Claims (6)
1. move the image search method combined with brain electricity based on eye, comprising:
(1) original image to be retrieved is inputted;
(2) eye is moved equipment and brain electricity equipment connect one by test-run a machine simultaneously, original image to be retrieved is presented on by test-run a machine, allows subject observe these pictures, and record eye movement data and the eeg data of subject simultaneously;
(3) generate the hot-zone figure of image to be retrieved according to the eye movement data recorded and eeg data, hot-zone figure represents area-of-interest with the rgb color of 0-255, and includes temporal information and the coordinate information of set time section fixation point;
(4) from the hot-zone figure generated, target image to be retrieved is extracted;
(4a) by Matlab software, all coordinates in the figure of hot-zone are showed in former figure;
(4b) centered by the coordinate of each fixation point, obtain the rectangular block of pixels containing 100 pixels, extract this block of pixels, as target image to be detected;
(5) in image library, retrieve target image to be retrieved by full way of search, the image according to retrieving calculates recall ratio.Recall ratio is in an image retrieval procedure, and the associated picture quantity retrieved accounts for the ratio of all associated picture quantity.
2. according to claim 1ly to move and the image search method that combines of brain electricity based on eye, its feature in: generate hot-zone figure according to eye movement data and eeg data in step (3), carry out as follows:
(3a) according to the brain wave patterns change of subject when seeing oneself part interested, the time period of this change is obtained;
(3b) filter out the eye movement data of this time period, and this eye movement data is shown as the form of form;
(3c) eye movement data obtained is input in DataViewer software, generates the hot-zone figure of respective image.
3. according to claim 1ly move the image search method combined with EEG based on eye, its feature in: based on the image retrieval of full-search algorithm in step (5), carry out as follows:
(5a) read the image in picture library, this Iamge Segmentation is become and block of pixels to be searched multiple block of pixels of a size;
(5b) from the block of pixels of first, searching image upper left, the Euclidean distance between itself and block of pixels to be searched is calculated;
If when the Euclidean distance (5c) between two block of pixels is zero, then the match is successful, and start counting, until all block of pixels traversals terminated;
(5d) in image library, according to the similar image that the result retrieval of coupling goes out, and recall ratio is calculated.
4. move the image search method combined with brain electricity based on eye, comprising:
1) original image to be retrieved is inputted;
2) eye is moved equipment and brain electricity equipment connect one by test-run a machine simultaneously, original image to be retrieved is presented on by test-run a machine, allows subject observe these pictures, and record eye movement data and the eeg data of subject simultaneously;
3) generate the hot-zone figure of image to be retrieved according to the eye movement data recorded and eeg data, hot-zone figure represents area-of-interest with the rgb color of 0-255, and includes temporal information and the coordinate information of set time section fixation point;
4) from the hot-zone figure generated, target image to be retrieved is extracted;
4a) by Matlab software, all coordinates in the figure of hot-zone are showed in former figure to be retrieved;
4b) calculate the barycenter of these fixation point coordinate, centered by this barycenter, generate the rectangle frame comprising all fixation point, this rectangle frame is extracted, as target image to be retrieved;
5) in image library, with the image search method based on scale invariant feature conversion SIFT algorithm, target image to be retrieved is retrieved, and calculate recall ratio.
5. according to claim 4ly move and the image search method that combines of brain electricity based on eye, its feature is in step 3) in generate hot-zone figure according to eye movement data and eeg data, carry out as follows:
3a) according to the brain wave patterns change of subject when seeing oneself part interested, obtain the time period of this change;
3b) filter out the eye movement data of this time period, and this eye movement data is shown as the form of form;
3c) eye movement data obtained is input in DataViewer software, generates the hot-zone figure of respective image.
6. according to claim 4ly to move and the image search method that combines of brain electricity based on eye, it is characterized in that: step 5) in based on the image retrieval of scale invariant feature conversion SIFT algorithm, carry out as follows:
5a) extraction step 4) in the SIFT feature point of image to be searched and searching image, and generate corresponding SIFT feature vector;
5b) by step 4) each unique point in the target to be searched image that extracts mates with all unique points in searching image, calculate the Euclidean distance between unique point, if recently Euclidean distance is less than the threshold value 0.6 of setting except the value of nearly Euclidean distance gained in proper order, then can judge that minimum a pair unique point of Euclidean distance is as the unique point matched, and start counting, until unique points all in target image to be retrieved all mates end;
5c) in image library, obtain the image retrieved according to the result of Feature Points Matching, and calculate recall ratio.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510822183.XA CN105468738A (en) | 2015-11-24 | 2015-11-24 | Image retrieval method based on combination of eye movement and electroencephalogram |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510822183.XA CN105468738A (en) | 2015-11-24 | 2015-11-24 | Image retrieval method based on combination of eye movement and electroencephalogram |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105468738A true CN105468738A (en) | 2016-04-06 |
Family
ID=55606439
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510822183.XA Pending CN105468738A (en) | 2015-11-24 | 2015-11-24 | Image retrieval method based on combination of eye movement and electroencephalogram |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105468738A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108459705A (en) * | 2017-12-21 | 2018-08-28 | 北京数字新思科技有限公司 | A kind of method, apparatus and computer readable storage medium of test image attraction |
| CN108958489A (en) * | 2018-07-20 | 2018-12-07 | 东南大学 | A kind of interesting image regions Rapid Detection method based on brain electricity and eye tracker |
| CN109828664A (en) * | 2019-01-15 | 2019-05-31 | 西安交通大学 | Steady State Visual Evoked Potential brain-machine interface method based on sense feedback dynamic adjustment |
| CN110244839A (en) * | 2019-05-20 | 2019-09-17 | 联想（上海）信息技术有限公司 | Control method, electronic equipment and storage medium |
| CN110687929A (en) * | 2019-10-10 | 2020-01-14 | 辽宁科技大学 | Aircraft three-dimensional space target searching system based on monocular vision and motor imagery |
| CN111158471A (en) * | 2019-12-18 | 2020-05-15 | 浙江大学 | Human-computer interaction method based on eye movement and brain-computer interface technology |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101714153A (en) * | 2009-11-16 | 2010-05-26 | 杭州电子科技大学 | Visual perception based interactive mammography image searth method |
| CN103473740A (en) * | 2013-08-31 | 2013-12-25 | 西安电子科技大学 | Sparse representation and low-rank double restraints-based nonlocal denoising method |
| CN103631941A (en) * | 2013-12-11 | 2014-03-12 | 北京师范大学 | Electroencephalogram-based target image retrieval system |
| CN104063872A (en) * | 2014-07-04 | 2014-09-24 | 西安电子科技大学 | Method for detecting salient regions in sequence images based on improved visual attention model |
| US9841811B2 (en) * | 2008-11-03 | 2017-12-12 | Bruce Reiner | Visually directed human-computer interaction for medical applications |
-
2015
- 2015-11-24 CN CN201510822183.XA patent/CN105468738A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9841811B2 (en) * | 2008-11-03 | 2017-12-12 | Bruce Reiner | Visually directed human-computer interaction for medical applications |
| CN101714153A (en) * | 2009-11-16 | 2010-05-26 | 杭州电子科技大学 | Visual perception based interactive mammography image searth method |
| CN103473740A (en) * | 2013-08-31 | 2013-12-25 | 西安电子科技大学 | Sparse representation and low-rank double restraints-based nonlocal denoising method |
| CN103631941A (en) * | 2013-12-11 | 2014-03-12 | 北京师范大学 | Electroencephalogram-based target image retrieval system |
| CN104063872A (en) * | 2014-07-04 | 2014-09-24 | 西安电子科技大学 | Method for detecting salient regions in sequence images based on improved visual attention model |
Non-Patent Citations (1)
| Title |
|---|
| 刘俊: ""基于眼动追踪的隐式相关反馈方法研究"", 《中国优秀硕士学位论文全文数据库-信息科技辑》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108459705A (en) * | 2017-12-21 | 2018-08-28 | 北京数字新思科技有限公司 | A kind of method, apparatus and computer readable storage medium of test image attraction |
| CN108958489A (en) * | 2018-07-20 | 2018-12-07 | 东南大学 | A kind of interesting image regions Rapid Detection method based on brain electricity and eye tracker |
| CN109828664A (en) * | 2019-01-15 | 2019-05-31 | 西安交通大学 | Steady State Visual Evoked Potential brain-machine interface method based on sense feedback dynamic adjustment |
| CN110244839A (en) * | 2019-05-20 | 2019-09-17 | 联想（上海）信息技术有限公司 | Control method, electronic equipment and storage medium |
| CN110244839B (en) * | 2019-05-20 | 2022-11-18 | 联想（上海）信息技术有限公司 | Control method, electronic device, and storage medium |
| CN110687929A (en) * | 2019-10-10 | 2020-01-14 | 辽宁科技大学 | Aircraft three-dimensional space target searching system based on monocular vision and motor imagery |
| CN110687929B (en) * | 2019-10-10 | 2022-08-12 | 辽宁科技大学 | Aircraft three-dimensional space target searching system based on monocular vision and motor imagery |
| CN111158471A (en) * | 2019-12-18 | 2020-05-15 | 浙江大学 | Human-computer interaction method based on eye movement and brain-computer interface technology |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105468738A (en) | Image retrieval method based on combination of eye movement and electroencephalogram | |
| Li et al. | Towards reading hidden emotions: A comparative study of spontaneous micro-expression spotting and recognition methods | |
| Zhang et al. | Multimodal spontaneous emotion corpus for human behavior analysis | |
| Abdat et al. | Human-computer interaction using emotion recognition from facial expression | |
| Li et al. | Delving into egocentric actions | |
| Ofli et al. | Berkeley mhad: A comprehensive multimodal human action database | |
| US20110150283A1 (en) | Apparatus and method for providing advertising content | |
| Agarwal et al. | 3D local ternary co-occurrence patterns for natural, texture, face and bio medical image retrieval | |
| Loureiro et al. | Using a skeleton gait energy image for pathological gait classification | |
| Yang et al. | Depth sequential information entropy maps and multi-label subspace learning for human action recognition | |
| CN113435236A (en) | Home old man posture detection method, system, storage medium, equipment and application | |
| Sun et al. | Automatic visual mimicry expression analysis in interpersonal interaction | |
| Tran et al. | Adaptive pulsatile plane for robust noncontact heart rate monitoring | |
| Assens et al. | Scanpath and saliency prediction on 360 degree images | |
| Wu et al. | Appearance-based gaze block estimation via CNN classification | |
| CN111443804A (en) | Method and system for describing fixation point track based on video analysis | |
| Pang et al. | Analysis of computer vision applied in martial arts | |
| Sun et al. | Contrast-phys+: Unsupervised and weakly-supervised video-based remote physiological measurement via spatiotemporal contrast | |
| Singh et al. | A robust, real-time camera-based eye gaze tracking system to analyze users’ visual attention using deep learning | |
| Zhang et al. | Physical activity recognition based on motion in images acquired by a wearable camera | |
| Alshawi et al. | Understanding spatial correlation in eye-fixation maps for visual attention in videos | |
| US20140016859A1 (en) | Systems, methods, and media for optical recognition | |
| CN109359543B (en) | Portrait retrieval method and device based on skeletonization | |
| Zhang et al. | An approach of region of interest detection based on visual attention and gaze tracking | |
| Ghalleb et al. | Contribution to the fusion of soft facial and body biometrics for remote people identification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20160406 |
|
| WD01 | Invention patent application deemed withdrawn after publication |