CN106022384B - Image attention target semanteme dividing method based on fMRI visual performance data DeconvNet - Google Patents
Image attention target semanteme dividing method based on fMRI visual performance data DeconvNet Download PDFInfo
- Publication number
- CN106022384B CN106022384B CN201610361766.1A CN201610361766A CN106022384B CN 106022384 B CN106022384 B CN 106022384B CN 201610361766 A CN201610361766 A CN 201610361766A CN 106022384 B CN106022384 B CN 106022384B
- Authority
- CN
- China
- Prior art keywords
- performance data
- visual performance
- image
- semantic segmentation
- fmri
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/2163—Partitioning the feature space
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/29—Graphical models, e.g. Bayesian networks
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The present invention relates to a kind of image attention target semanteme dividing methods based on fMRI visual performance data DeconvNet, by being tested the collected fMRI visual performance data under natural scene image stimulation, training depth convolutional neural networks model maps that on concern target category label and carries out model optimization；Construction and the symmetrical depth network model of depth convolutional neural networks after optimization, using fMRI visual performance data and the corresponding semantic segmentation result optimizing model parameter of stimulating image, the mapping of fMRI visual performance data to semantic segmentation result pixel-by-pixel is obtained；Acquisition subject watches the fMRI visual performance data of test image, determines subject concern target category and pixel-by-pixel semantic segmentation as a result, being partitioned into concern target area and respective objects semanteme.The fMRI visual performance data that the present invention is caused when watching natural scene image to subject parse, and extract all target categories in stimulating image and obtain semantic segmentation as a result, improving the accuracy of concern Objective extraction.
Description
Technical field
The present invention relates to human-computer interaction fMRI visual performance technical field of data processing, in particular to a kind of to be regarded based on fMRI
Feel the image attention target semanteme dividing method of performance data DeconvNet.
Background technique
Acquisition of vision information is that the mankind obtain the most important mode of external information, and deciphering method is also grinding for Neuscience
Study carefully emphasis.For a long time, numerous studies personnel fit extension human vision function with computer mould from a variety of different angle trials
Energy.And in neuroscience field, there is one there is the problem of very big attraction always, that is, why human brain can be used seldom
Energy complete the such as advanced visual task such as object identification, scene understanding.In recent years, neuroimaging technology achieves considerable
Progress, functional mri (functional Magnetic Resonance Imaging, fMRI) with its non-intruding,
Major nerve imaging methods of the features such as spatial and temporal resolution is good as research brain Vision information processing mechanism.For system research
Human brain visual performance activity understands treatment mechanism of the human brain for visual information, the fMRI signal solution of human brain visual performance brain area
Analysis technology achieves significant progress, these researchs are also referred to as the research of visual information encoding and decoding technique.The coding of visual information
Technology is a kind of technology that visual cognition forward direction calculates, and by establishing vision computation model i.e. visual coding model, prediction is any
The response for the brain visual performance that visual stimulus can cause.And vision decoding technique is then the brain function activity letter by measuring
Number recover the information such as the classification, scene, details of visual stimulus.
2001, Haxby et al. proved the classification information of sensation target in veutro temporal lobe (ventral temporal
Lobe it) expresses in a distributed manner, can accurately differentiate plurality of target classification using the voxel activation pattern of the brain area.2003,
Cox et al. application multi-voxel proton method for classifying modes carries out the classification of ten kinds of classification objects.2010, Chen et al. proposed to be based on skin
The feature selection approach of layer surface searchlight (Searchlight) is classified the musical instrument, chair and canoe of rotation.2012
Year, Connolly et al. studies the expression of human brain biological species, is classified to different primates, birds, insect.Although
Existing research has been able to stimulate caused fMRI visual performance data to parse its affiliated class the image of a certain classification
Not, however the extraction for being tested target semanteme of interest, but without corresponding research achievement.
Summary of the invention
In order to overcome the shortcomings in the prior art, the present invention provides a kind of based on fMRI visual performance data DeconvNet's
Image attention target semanteme dividing method, the fMRI visual performance data caused when can watch natural scene image to subject
It is parsed, extract all target categories in stimulating image and obtains semantic segmentation as a result, further improving for human brain
The analytic ability of visual performance.
According to design scheme provided by the present invention, a kind of image pass based on fMRI visual performance data DeconvNet
Poster justice dividing method is gazed at, is comprised the following steps:
The fMRI visual performance data of step 1, acquisition subject under natural scene image stimulation, training one is schemed by stimulation
Picture is to the depth convolutional neural networks model of fMRI visual performance data and one by fMRI visual performance data to concern target
The depth convolutional neural networks model that training obtains is mapped on Linear Mapping model by the Linear Mapping model of classification, is carried out
Network model optimization；
Step 2 constructs one and the symmetrical deconvolution depth network of depth convolutional neural networks after network model optimization
Model, using fMRI visual performance data and the corresponding semantic segmentation result of stimulating image to deconvolution depth network model into
Row optimization processing obtains fMRI visual performance data to the mapping of semantic segmentation result pixel-by-pixel, obtains semantic segmentation depth net
Network model；
FMRI visual performance data when step 3, acquisition subject viewing test image, pass through semantic segmentation depth network mould
Type obtains image pixel by pixel semantic segmentation result；
Step 4, the Linear Mapping model by paying close attention to target category, obtain the target category of subject image attention；
Step 5, the subject image according to obtained in the image pixel by pixel semantic segmentation result and step 4 that step 3 obtains close
The target category of note, target area and the respective objects for being partitioned into subject concern are semantic.
Above-mentioned, deconvolution depth network model includes warp lamination, anti-pond layer in step 2, by according to collected
FMRI visual performance data are tested as input, the corresponding concern target semanteme segmentation result of stimulating image is as output, optimization
Training pattern parameter, training learn the deconvolution core of each layer, and anti-pond layer uses the behaviour of pond layer in depth convolutional neural networks
It is up-sampled, training obtains fMRI visual performance data to the deconvolution depth network model of image, semantic segmentation result.
Beneficial effects of the present invention:
The present invention passes through building convolutional neural networks modeling natural scene image reflecting to fMRI visual performance data
Relationship is penetrated, is divided using with its symmetrical deconvolution network structure training from fMRI visual performance data to concern target category semanteme
The depth network model of result is cut, obtaining includes each target category semantic segmentation knot trained in category set in image
Fruit pays close attention to target to extract in image, the fMRI visual performance caused when can watch natural scene image to subject
Data are parsed, and are extracted all target categories in stimulating image and are obtained semantic segmentation as a result, improving concern Objective extraction
Accuracy, further promoted to the analytic ability of human brain visual performance.
Detailed description of the invention:
Fig. 1 is flow diagram of the invention.
Specific embodiment:
The image, semantic segmentation result of label is provided in conventional images data set (such as Pascal VOC data set), in addition also
Image, semantic segmentation result can be obtained by way of handmarking；When can satisfy network training by both modes pair
The needs of image, semantic segmentation result.
The present invention is described in further detail with technical solution with reference to the accompanying drawing, and detailed by preferred embodiment
Describe bright embodiments of the present invention in detail, but embodiments of the present invention are not limited to this.
Embodiment one, a kind of shown in Figure 1, image attention target based on fMRI visual performance data DeconvNet
Semantic segmentation method, comprises the following steps:
The fMRI visual performance data of step 1, acquisition subject under natural scene image stimulation, training one is schemed by stimulation
Picture is to the depth convolutional neural networks model of fMRI visual performance data and one by fMRI visual performance data to concern target
The depth convolutional neural networks model that training obtains is mapped on Linear Mapping model by the Linear Mapping model of classification, is carried out
Network model optimization；
Step 2 constructs one and the symmetrical deconvolution depth network of depth convolutional neural networks after network model optimization
Model, using fMRI visual performance data and the corresponding semantic segmentation result of stimulating image to deconvolution depth network model into
Row optimization processing obtains fMRI visual performance data to the mapping of semantic segmentation result pixel-by-pixel, obtains semantic segmentation depth net
Network model；
FMRI visual performance data when step 3, acquisition subject viewing test image, pass through semantic segmentation depth network mould
Type obtains image pixel by pixel semantic segmentation result；
Step 4, the Linear Mapping model by paying close attention to target category, obtain the target category of subject image attention；
Step 5, the subject image according to obtained in the image pixel by pixel semantic segmentation result and step 4 that step 3 obtains close
The target category of note, target area and the respective objects for being partitioned into subject concern are semantic.
Embodiment two, a kind of shown in Figure 1, image attention target based on fMRI visual performance data DeconvNet
Semantic segmentation method, comprises the following steps:
The fMRI visual performance data of step 1, acquisition subject under natural scene image stimulation, training one is schemed by stimulation
Picture is to the depth convolutional neural networks model of fMRI visual performance data and one by fMRI visual performance data to concern target
The depth convolutional neural networks model that training obtains is mapped on Linear Mapping model by the Linear Mapping model of classification, is carried out
Network model optimization；
Step 2 constructs one and the symmetrical deconvolution depth network of depth convolutional neural networks after network model optimization
Model, using fMRI visual performance data and the corresponding semantic segmentation result of stimulating image to deconvolution depth network model into
Row optimization processing obtains fMRI visual performance data to the mapping of semantic segmentation result pixel-by-pixel, obtains semantic segmentation depth net
Network model, wherein deconvolution depth network model includes warp lamination, anti-pond layer, by being regarded according to collected subject fMRI
Performance data is felt as input, and the corresponding concern target semanteme segmentation result of stimulating image is as output, optimization training pattern ginseng
Number, training learn the deconvolution core of each layer, and anti-pond layer adopt using the operation of pond layer in depth convolutional neural networks
Sample, training obtain fMRI visual performance data to the deconvolution depth network model of image, semantic segmentation result；
FMRI visual performance data when step 3, acquisition subject viewing test image, pass through semantic segmentation depth network mould
Type obtains image pixel by pixel semantic segmentation result；
Step 4, the Linear Mapping model by paying close attention to target category, obtain the target category of subject image attention；
Step 5, the subject image according to obtained in the image pixel by pixel semantic segmentation result and step 4 that step 3 obtains close
The target category of note, target area and the respective objects for being partitioned into subject concern are semantic.
Innovative proposition of the invention constructs convolutional neural networks modeling natural scene image to fMRI visual performance number
According to mapping relations, and using and its symmetrical network structure training from fMRI visual performance data to image, semantic segmentation result
Depth network model, obtaining includes each target category semantic segmentation result in trained category set in image.The party
The fMRI visual performance data that method is caused when can watch natural scene image to subject parse, and extract stimulating image
In all target categories and obtain semantic segmentation as a result, extracting subject target of interest in the picture, greatly improve concern
The accuracy of Objective extraction further promotes the analytic ability for human brain visual performance, for the brain of view-based access control model function parsing
Machine interactive application research provides further technical support.
The invention is not limited to above-mentioned specific embodiment, those skilled in the art can also make a variety of variations accordingly,
But it is any all to cover within the scope of the claims with equivalent or similar variation of the invention.
Claims (2)
1. a kind of image attention target semanteme dividing method based on fMRI visual performance data DeconvNet, it is characterised in that:
It comprises the following steps:
Step 1, acquisition subject natural scene image stimulation under fMRI visual performance data, training one by stimulating image to
The depth convolutional neural networks model of fMRI visual performance data and one are by fMRI visual performance data to concern target category
Linear Mapping model, the obtained depth convolutional neural networks model of training is mapped on Linear Mapping model, network is carried out
Model optimization；
Step 2 constructs one and the symmetrical deconvolution depth network model of depth convolutional neural networks after network model optimization,
Deconvolution depth network model is carried out using fMRI visual performance data and stimulating image corresponding semantic segmentation result excellent
Change processing obtains fMRI visual performance data to the mapping of semantic segmentation result pixel-by-pixel, obtains semantic segmentation depth network mould
Type；
FMRI visual performance data when step 3, acquisition subject viewing test image, by semantic segmentation depth network model,
Obtain image pixel by pixel semantic segmentation result；
Step 4, the Linear Mapping model by paying close attention to target category, obtain the target category of subject image attention；
Step 5, the subject image attention according to obtained in the image pixel by pixel semantic segmentation result and step 4 that step 3 obtains
Target category, target area and the respective objects for being partitioned into subject concern are semantic.
2. the image attention target semantic segmentation according to claim 1 based on fMRI visual performance data DeconvNet
Method, it is characterised in that: deconvolution depth network model includes warp lamination, anti-pond layer in step 2, by according to collected
FMRI visual performance data are tested as input, the corresponding concern target semanteme segmentation result of stimulating image is as output, optimization
Training pattern parameter, training learn the deconvolution core of each layer, and anti-pond layer uses the behaviour of pond layer in depth convolutional neural networks
It is up-sampled, training obtains fMRI visual performance data to the deconvolution depth network model of image, semantic segmentation result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610361766.1A CN106022384B (en) | 2016-05-27 | 2016-05-27 | Image attention target semanteme dividing method based on fMRI visual performance data DeconvNet |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610361766.1A CN106022384B (en) | 2016-05-27 | 2016-05-27 | Image attention target semanteme dividing method based on fMRI visual performance data DeconvNet |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106022384A CN106022384A (en) | 2016-10-12 |
| CN106022384B true CN106022384B (en) | 2019-07-12 |
Family
ID=57093546
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610361766.1A Active CN106022384B (en) | 2016-05-27 | 2016-05-27 | Image attention target semanteme dividing method based on fMRI visual performance data DeconvNet |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106022384B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107066916B (en) * | 2016-10-26 | 2020-02-07 | 中国科学院自动化研究所 | Scene semantic segmentation method based on deconvolution neural network |
| CN106780498A (en) * | 2016-11-30 | 2017-05-31 | 南京信息工程大学 | Based on point depth convolutional network epithelium and matrix organization's automatic division method pixel-by-pixel |
| CN106971155B (en) * | 2017-03-21 | 2020-03-24 | 电子科技大学 | Unmanned vehicle lane scene segmentation method based on height information |
| CN107731011B (en) * | 2017-10-27 | 2021-01-19 | 中国科学院深圳先进技术研究院 | Port berthing monitoring method and system and electronic equipment |
| CN110276762A (en) * | 2018-03-15 | 2019-09-24 | 北京大学 | A kind of full-automatic bearing calibration of respiratory movement of the diffusion-weighted Abdominal MRI imaging of more b values |
| CN108898606B (en) * | 2018-06-20 | 2021-06-15 | 中南民族大学 | Method, system, device and storage medium for automatic segmentation of medical images |
| CN108961269B (en) * | 2018-06-22 | 2022-04-08 | 深源恒际科技有限公司 | Pig weight measuring and calculating method and system based on image |
| CN110569880A (en) * | 2019-08-09 | 2019-12-13 | 天津大学 | Method for decoding visual stimulation by using artificial neural network model |
| CN111404274B (en) * | 2020-04-29 | 2023-06-06 | 平顶山天安煤业股份有限公司 | Transmission system displacement on-line monitoring and early warning system |
| CN112686098B (en) * | 2020-12-15 | 2023-05-26 | 中国人民解放军战略支援部队信息工程大学 | Shape-Resnet-based method and system for processing signals of middle and high-grade vision areas of brain |
| CN113419537A (en) * | 2021-07-08 | 2021-09-21 | 西安理工大学 | Brain-computer fusion control method and system for autonomous movement of mobile robot |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103778240B (en) * | 2014-02-10 | 2017-04-26 | 中国人民解放军信息工程大学 | Image retrieval method based on functional magnetic resonance imaging and image dictionary sparse decomposition |
| CN104598575B (en) * | 2015-01-14 | 2017-09-01 | 中国人民解放军信息工程大学 | The brain-machine interaction image indexing system being imaged based on real-time functional magnetic resonance |
-
2016
- 2016-05-27 CN CN201610361766.1A patent/CN106022384B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN106022384A (en) | 2016-10-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106022384B (en) | Image attention target semanteme dividing method based on fMRI visual performance data DeconvNet | |
| Davison et al. | Objective classes for micro-facial expression recognition | |
| CN106569604B (en) | Audiovisual bimodal semantic matches and semantic mismatch collaboration stimulation brain-machine interface method | |
| CN109815785A (en) | A kind of face Emotion identification method based on double-current convolutional neural networks | |
| Lin et al. | Mind reader: Reconstructing complex images from brain activities | |
| CN106056602B (en) | FMRI visual performance datum target extracting method based on CNN | |
| Zou et al. | Do game data generalize well for remote sensing image segmentation? | |
| CN113782190B (en) | Image processing method based on multistage space-time characteristics and mixed attention network | |
| CN104598575B (en) | The brain-machine interaction image indexing system being imaged based on real-time functional magnetic resonance | |
| Yohanandan et al. | Saliency preservation in low-resolution grayscale images | |
| Du et al. | fmri brain decoding and its applications in brain–computer interface: A survey | |
| Zhang et al. | Cross-cultural consistency and diversity in intrinsic functional organization of Broca's Region | |
| CN110321827A (en) | A kind of pain level appraisal procedure based on face pain expression video | |
| De Venuto et al. | A single-trial P300 detector based on symbolized EEG and autoencoded-(1D) CNN to improve ITR performance in BCIs | |
| CN109770896A (en) | Dreamland image reproducing method, device and storage medium, server | |
| CN108985301A (en) | A kind of hyperspectral image classification method of the sub- dictionary learning of multiple features class | |
| Lee et al. | Automatic object detection algorithm-based braille image generation system for the recognition of real-life obstacles for visually impaired people | |
| Zhao et al. | Attention optimized deep generative adversarial network for removing uneven dense haze | |
| Jain | How AI could lead to a better understanding of the brain | |
| CN108985332A (en) | Natural image random forest imaging method based on action potential granting interval | |
| Han et al. | Constructing a forest color palette and the effects of the color patch index on human eye recognition accuracy | |
| Duan et al. | Fusion of infrared and visible images using fast global smoothing decomposition and target-enhanced parallel gaussian fuzzy logic | |
| Gui et al. | Utilizing wavelet deep learning network to classify different states of task-fMRI for verifying activation regions | |
| Santos-Mayo et al. | A Model of the Early Visual System Based on Parallel Spike-Sequence Detection, Showing Orientation Selectivity | |
| CN108921052A (en) | Character picture naive Bayesian method for reconstructing based on local field potentials amplitude Characteristics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |