CN109902714A - A kind of multi-modality medical image search method based on more figure regularization depth Hash - Google Patents
A kind of multi-modality medical image search method based on more figure regularization depth Hash Download PDFInfo
- Publication number
- CN109902714A CN109902714A CN201910048281.0A CN201910048281A CN109902714A CN 109902714 A CN109902714 A CN 109902714A CN 201910048281 A CN201910048281 A CN 201910048281A CN 109902714 A CN109902714 A CN 109902714A
- Authority
- CN
- China
- Prior art keywords
- medical image
- data
- depth
- matrix
- modality medical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 239000011159 matrix material Substances 0.000 claims abstract description 49
- 239000000284 extract Substances 0.000 claims abstract description 7
- 230000001174 ascending effect Effects 0.000 claims abstract description 6
- 230000000007 visual effect Effects 0.000 claims description 38
- 238000012549 training Methods 0.000 claims description 27
- 230000003044 adaptive effect Effects 0.000 claims description 19
- 230000004927 fusion Effects 0.000 claims description 10
- 241001269238 Data Species 0.000 claims description 5
- 238000013527 convolutional neural network Methods 0.000 claims description 4
- HUTDUHSNJYTCAR-UHFFFAOYSA-N ancymidol Chemical compound C1=CC(OC)=CC=C1C(O)(C=1C=NC=NC=1)C1CC1 HUTDUHSNJYTCAR-UHFFFAOYSA-N 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 238000013139 quantization Methods 0.000 claims description 3
- 235000013399 edible fruits Nutrition 0.000 claims description 2
- 239000003814 drug Substances 0.000 claims 2
- 238000005481 NMR spectroscopy Methods 0.000 abstract 1
- 238000003745 diagnosis Methods 0.000 abstract 1
- 238000005259 measurement Methods 0.000 abstract 1
- 238000002604 ultrasonography Methods 0.000 abstract 1
- 238000012360 testing method Methods 0.000 description 11
- 230000002902 bimodal effect Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000013507 mapping Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000005065 mining Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 210000004556 brain Anatomy 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 206010012289 Dementia Diseases 0.000 description 1
- 208000032612 Glial tumor Diseases 0.000 description 1
- 206010018338 Glioma Diseases 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Landscapes
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
A kind of more figure regularization depth Hash multi-modality medical image search methods are claimed in the present invention, extract the feature of multi-modality medical image group simultaneously particular by multichannel depth model；Multiple figure regularization matrix are constructed according to the feature of multi-modality medical image group is corresponding；Multiple figure regularization matrix are merged, and is adaptively limited Boltzmann machine using mode and learns to obtain the Hash codes of multi-modality medical image group；Single modal data Hash codes are found out at a distance from multi-modality medical image group Hash codes by Hamming distance measurement and are sorted in ascending order, and selection returns to user apart from the smallest n group multi-modality medical image, to realize multi-modality medical image retrieval.The realization of this method can help doctor in ultrasound image, controversial issue text, in the multi-modality medical images such as nuclear magnetic resonance image, the data of other multiple modalities are found rapidly by the data of a certain mode, facilitate the medical diagnosis of doctor, the workload of doctor is reduced, working efficiency is improved.
Description
Technical field
The invention belongs to technical field of medical image processing more particularly to more figure regularization depth hash methods to realize multimode
State medical image retrieval.
Background technique
The input data according to a certain mode that multi-modality medical image retrieval technique refers to is from multi-modality medical image library
Retrieve the medical image of the same mode and different modalities that match.Mainly there are three modules for existing multi-modal retrieval technology:
Text based image retrieval technologies, text based video retrieval technology, the text retrieval technique based on image.It is existing more
Mode retrieval technique is mostly retrieved mutually between both modalities which, however growing multi-modality medical image makes the prior art
It is unable to satisfy the demand that user retrieves mutually between any modal data.
Cross-module state Hash searching algorithm becomes research hotspot in recent years, and achieves preferable effect.However it still deposits
In some technological deficiencies: (1) existing method mostly learns Hash codes by extracting the manual feature of data, compared to passing through
Learn Hash codes according to the immanent structure feature of different data, manual extraction feature learning to Hash codes have to retrieval precision
Larger impact；(2) existing most cross-module state hash algorithms based on deep learning also all only two modal datas it
Between realize mutually retrieval；(3) existing method is not all considered in data in manifold when realizing mapping of the data to Hash codes
Structure, so that the Hash codes learnt also fail to keep the local manifolds structure of data, to affect retrieval precision.
For above-mentioned Railway Project, although many scholars have put into a large amount of time and efforts and gone to study, do not have still
The multi-modal retrieval method of one realization self-adapting data mode occurs.The principle of RBM, the hidden layer knot that can be learnt
Hash codes of the fruit directly as data；And manifold structure keep addition, can while data are mapped to Hash codes in keep
The local manifolds structure of data.
Problem to be solved by this invention is that manual feature is unable to satisfy high-precision Search Requirement, cross-module state hash algorithm
Mostly bimodal is retrieved mutually, data to the mapping between Hash codes are not able to maintain data local manifolds structure etc. no
Foot.The present invention extracts the manual feature that data depth feature replaces data with depth model, and avoiding manual feature can not be very
The problem of mining data immanent structure got well, to substantially increase retrieval precision in Hash retrieval；It is breathed out using adaptive RBM
Uncommon algorithm can solve existing multi-modal retrieval mostly and can only realize in the data of two mode the problem of retrieval mutually, can be
Retrieval mutually is realized in any multi-modal data；It is kept using manifold structure, it can be in the mapping process of data to Hash codes
The local manifolds structure for keeping data well, to further promote retrieval precision.
Summary of the invention
Present invention seek to address that the above problem of the prior art.Propose it is a kind of promoted retrieval precision based on more figure canonicals
Change the multi-modality medical image search method of depth Hash.Technical scheme is as follows:
A kind of multi-modality medical image search method based on more figure regularization depth Hash comprising following steps:
Step 1, the depth characteristic that multi-modality medical image is extracted using multichannel depth model, and to depth characteristic standard
Change；
The multiple neighbour's figure matrixes of feature construction of step 2, the multiple and different modal datas extracted according to step 1, to protect
The local manifolds structure of data is held, and constructs a label matrix；
The multiple neighbour's figure matrixes and label matrix of building are fused into a figure matrix by step 3；
Step 4 combines fused figure matrix to learn to obtain multimode using the adaptive limited Boltzmann machine RBM of mode
The common Hash codes of state medical image；
Modal data to be retrieved is generated Hash codes by the adaptive RBM of depth channel and mode by step 5；
Step 6, data and the multi-modality medical image library that a certain mode to be retrieved is calculated using Hamming distance measure
The distance between and ascending sort, user will be returned to apart from the smallest closest multi-modality medical image of n group.
Further, the step 1 extracts the depth characteristic of multi-modality medical image using multichannel depth model, specifically
It include: adaptively to determine the number of channels of depth model according to data group mode quantity first, then multiple depth channels are connected
As a whole to same classification layer, the whole multichannel depth model of training, training complete to take reciprocal the of each channel
Depth characteristic of the two layers of result as corresponding modal data.
Further, the step 1 uses depth characteristic after the depth characteristic for extracting multi-modality medical image group
Z-score standardization, the feature after quantization will obey standardized normal distribution, and the quantitative formula of feature is as follows:
Wherein μ indicates the mean value of some feature, and δ indicates the standard deviation of feature.
Further, step 2 neighbour schemes in building, and figure is considered as the set of n vector to describe the geometry of data
Structure, wherein the corresponding data point of each vector, the length of each vector are ρ, indicate ρ and the data point arest neighbors
Data point, multiple neighbour's matrixDistance metric mode using Gauss thermonuclear distance or manhatton distance
Or Chebyshev's distance, m indicate the mode quantity of multi-modality medical image, i indicates neighbour's matrix of a certain modal data building
The i-th row, j indicate jth column；Indicate neighbour's matrix of a certain modal data building.Neighbour's figure has been constructed according to depth characteristic
Afterwards, an additional label neighbour is constructed according to label to scheme.Further, described close according to label one additional label of building
Neighbour's figure, specifically includes: a n dimension matrix is constructed according to label, building rule is as follows:
xiExpression respectively indicates one group of image of multi-modal data, xjIndicate any one group in remaining n-1 group image, a table
Show xiWith xjThe number of same label.After having constructed m+1 matrix, more figure regularization matrix are carried out using following formula and are merged:
Wherein μ indicates the weight coefficient of each matrix when fusion.
Further, the step 4 learns to obtain multi-modal using the adaptive RBM of mode in conjunction with fused figure matrix
The common Hash codes of medical image, specifically include:
The adaptive RBM of mode is improved by raw Gaussian RBM, and visual layer number is adaptive according to data modality quantity
It determines, is all connected to same hidden layer；Manifold is added when visual layers and hidden layer are generated by conditional probability simultaneously and keeps square
Battle array, so that the hidden layer Hash codes generated are able to maintain the local neighbor structure of data；The following institute of the energy function of improved RBM model
Show:
Wherein U indicates the energy function of entire improved RBM model；Represent the 1st, 2 ..., M visual
The a certain node of layer, hiIndicate a certain node of hidden layer.M indicates mode quantity, N1,N2Respectively indicate the node of each visual layers
Quantity and hidden node quantity；The parameter sets of θ expression RBM, the biasing a comprising visual layers, the biasing b of hidden layer, and it is visual
Connection weight w between layer and hidden layer.Indicate the biasing of m-th of visual layers, r-th of node, bsIndicate s-th of node of hidden layer
Biasing,Indicate the connection weight of r-th node and s-th of node of hidden layer of m-th of visual layers；Indicate m-th of visual layers
R-th of node, hsIndicate s-th of node of hidden layer；Indicate that the normal distribution standard of m-th of visual layers, r-th of node is poor,
It for positive value, does not train generally, takes definite value 1；λ indicates regularization weight parameter, the flatness that control hidden layer indicates.hisIt indicates
S-th of node of hidden layer, hjsExpression is obtained according to fusion matrix and hisThe node of neighbour.M indicates m-th of visual layers, r table
Show that r-th of node of a certain visual layers, s indicate s-th of node of hidden layer.
Further, the step 5 is by depth channel and the adaptive RBM of mode by a certain mode number to be retrieved
According to Hash codes are generated, specifically include: the corresponding mode is special when the medical image of a certain mode to be retrieved is put into its training
Sign is extracted in channel, takes convolutional neural networks number of plies value second from the bottom as its characteristic value；The identical molds obtained when according to training
The characteristic value that the mean μ and standard deviation δ of the eigenmatrix of state data standardize；Using standardized characteristic value as having trained
At the input of RBM visual layers, with the visual layers of mode when paying attention to corresponding training, other visual layers input rule identical as characteristic value
The null matrix of lattice does the result obtained after matrix multiple with connection weight and sign function sign is taken to obtain the Hash of data to be retrieved
Code.
Further, the step 6 calculates the data and multimode of a certain mode to be retrieved using Hamming distance measure
The distance between state medical image library and ascending sort, will return to use apart from the smallest closest multi-modality medical image of n group
Family, wherein Hamming distance measure formulas is as follows:
Wherein k indicates the length of Hash codes, hr(x) the r Hash codes of sample x, h are indicatedr(y) r of sample y is indicated
Position Hash codes,Indicate XOR operation.
It advantages of the present invention and has the beneficial effect that:
1, the invention avoids manual feature can not well mining data immanent structure the problem of, propose in step 1
Depth characteristic based on deep learning extracts structure and solves the problems, such as that traditional hash method precision based on manual feature is not high.
2, the proposition of the adaptive RBM visual layers of the feature extraction structure of the adaptive mode of step 1 and step 4 solves
The problem of existing most methods can only mutually be retrieved between two mode is realized between any multi-modality medical image mutually
Mutually retrieve.
3, step 2, the RBM model for combining more figure regularizations in 3,4 are solved and must be wanted in Hash mapping process
The problem of data local manifolds structure, further increase retrieval precision.
Detailed description of the invention
Fig. 1 is that the present invention provides the operational flowchart of preferred embodiment；
Fig. 2 be in the present invention multi-modality medical image group to Hash codes map overall model figure；
Fig. 3 is the adaptive RBM illustraton of model of mode in the present invention；
Fig. 4 is description of test and actual retrieval result (bimodal data set) in the present invention；
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, detailed
Carefully describe.Described embodiment is only a part of the embodiments of the present invention.
The technical solution that the present invention solves above-mentioned technical problem is:
Select different types of depth model for extracting its depth according to the different types of data of multi-modality medical image
Recognition with Recurrent Neural Network may be selected in feature, such as image data selection convolutional neural networks, text, for extracting the mould of depth characteristic
Type is not key task of the invention.
Because the numerical value dimension of each feature is inconsistent, and the adaptive RBM model of mode for being subsequently used for Hash codes study is wanted
It asks visual layers to meet Gaussian Profile, after the depth characteristic for extracting multi-modality medical image group, feature is marked using Z-score
Standardization, the feature after quantization will obey standardized normal distribution.The quantitative formula of feature is as follows:
Wherein μ and δ respectively indicates the standard deviation of the mean value of a feature.
Neighbour's figure is constructed according to primitive character, for keeping the local neighbor structure of data while Hash codes learn.
Neighbour schemes in building, and the mode of distance metric has Gauss thermonuclear distance, manhatton distance, Chebyshev's distance etc., measures herein
The selection of mode is not key task of the invention.After having constructed neighbour's figure according to depth characteristic, a volume is constructed according to label
Outer neighbour's figure, influence of the label to the inherent manifold structure of data is great, schemes in fusion in subsequent neighbour, label neighbour figure
Proportion is larger.
Real value data can be mapped as two-value data by original Gauss RBM model, using this characteristic, by its visual number of plies
Amount is adaptively adjusted according to multi-modality medical image mode quantity, and is connected to the same hidden layer simultaneously, and in its energy function
Similar data are generated similar Hash codes for constraining by a upper plus figure regularization term, to keep the inherent manifold knot of data
Structure, this is beneficial to improve retrieval precision.
Existing multi-modal hash method spininess generates manual feature for Hash codes, and mostly in both modalities which data
Between mutual retrieval.This greatly reduces retrieval precision, and is unable to satisfy user and mutually examines between any modal data
The demand of rope.One kind proposed by the present invention can greatly improve retrieval based on more figure regularization depth Hash multi-modal retrieval methods
Precision, and it is able to satisfy the demand that user retrieves mutually between any data modality.
The following detailed description of technical solution of the present invention:
Step 1: multi-modality medical image group depth characteristic is extracted
The Hash codes of multi-modality medical image group in order to obtain first extract depth characteristic to data.First according to data group
Mode quantity adaptively determines the number of channels of depth model, then multiple depth channels are connected to same classification layer as one
It is whole, as shown in Fig. 3.The whole multichannel depth model of training, training are completed to take the layer result second from the bottom in each channel
Depth characteristic as corresponding modal data.
Step 2: constructing and merges how close adjacent figure matrix
In order to keep data local neighbor structure while obtaining data group Hash codes, according to the depth of different modalities data
Spend feature construction neighbour figure.Figure is considered as the set of n vector to describe the geometry of data, wherein each vector is corresponding
One data point, the length of each vector are ρ, indicate the ρ data points with the data point arest neighbors.Multiple neighbour's matrix squares
Battle arrayDistance metric mode there are many selection, such as Gauss thermonuclear distances, manhatton distance, Chebyshev's distance
Deng；Wherein m indicates the mode quantity of multi-modality medical image.A n is constructed further accordance with label and ties up matrix, and building rule is as follows:
It is fused into a matrix after the completion of the building of neighbour's matrix, fusion rule is as follows:
Wherein fusion coefficients are learnt in the form of traversal to one group of suitable parameter.
Step 3: depth characteristic standardization
Due to following two, we must standardize obtained depth characteristic:
1. different features often has different dimensions, such situation influences whether subsequent data analysis and Hash codes
Study as a result, needing to carry out data normalization processing to eliminate the dimension impact between index.
2. Gauss Bernoulli Jacob is limited Boltzmann machine and assumes that data input Gaussian distributed, however original feature square
Battle array in data and do not meet Gaussian Profile.
In conjunction with two above reason, the primitive character matrix that we obtain is standardized using Z-score, the spy after standardization
Sign will obey standardized normal distribution.The quantitative formula of feature is as follows:
Wherein μ and δ respectively indicates the standard deviation of the mean value of a feature.
Step 4: Hash codes study
Fused figure matrix is combined to learn to obtain the common Hash of multi-modality medical image using the adaptive RBM of mode
Code.The adaptive RBM of mode is improved by raw Gaussian RBM, and visual layer number is adaptively determined according to data modality quantity,
It is all connected to same hidden layer；Manifold holding matrix is added when visual layers and hidden layer are generated by conditional probability simultaneously, makes
The hidden layer Hash codes that must be generated are able to maintain the local neighbor structure of data.It is after depth characteristic is standardized, it is defeated by mode
Enter into visual layers, in conjunction with fused matrix, utilizes the multi input contrast divergence algorithm training adaptive RBM of mode.It has trained
At the shared Hash codes of available multi-modal medical data.
Step 5: n (n takes 3 in the present invention) group neighbour's figure of image to be retrieved is returned
An image for taking one of mode is concentrated from the multi-modal medical data of test, is obtained using the model that training is completed
Its depth characteristic is obtained, then obtains its Hash codes.By the shared Kazakhstan of obtained test image Hash codes and obtained multi-modal data
Uncommon code carries out distance metric.The distance between sample and ascending sort are calculated using Hamming distance measure, it will be apart from minimum
N Hash codes take out, and its corresponding n group image is returned into user.
Specific experiment verifying
It is real as the method validation proposed using bimodal brain medical data collection for the validity of verification method
It tests.Data set includes 323 groups of totally 10 classification bimodal brain images in total, and a mode is MRI modal data, another mode
It is PET mode.Will wherein 290 groups of images be used as training group, residue 32 groups of images as test set.
Specific experiment step:
1, picture depth feature is extracted using convolutional neural networks.The section of two full articulamentums of convolutional network layer second from the bottom
Point number is set as 256, and shares the same classification layer, and bimodal training data is input to binary channels depth network
In, utilize classification task training overall network.It is right that 290 groups of training datas and 32 groups of test datas are put into its after the completion of training
Answer in the depth channel of mode, depth characteristic of the numerical value as data of layer second from the bottom will be obtained at this time, respectively obtain two
The eigenmatrix of a (256 × 290) and two (256 × 32).
2, two neighbour's matrixes (290 × 290) are constructed using obtained training set depth characteristic matrix, further according to training number
A label matrix is constructed according to the label of collection.Three matrixes, which are merged, using above-mentioned fusion rule obtains a fusion matrix.
3, the 4 depth characteristic matrixes standardized according to above-mentioned normalisation rule.Pay attention to test set eigenmatrix
Mean value when standardization is the mean value and standard deviation of the test set eigenmatrix of its corresponding mode with standard deviation.
4, the input of Boltzmann machine (RBM), knot are limited using the depth characteristic matrix after standardization as double visual layers
Close the double visual layers RBM of fusion matrix training.The eigenmatrix of training set and test set is inputted after the completion of training, with connection weight
It does and obtains the Hash codes of training set and test set after taking sign function (sign) after matrix multiplication.
5, take the MRI modality images that two labels are Glioma and Dementia as image to be retrieved respectively, its is right
The Hash codes answered calculate Hamming distance with training set Hash codes and sort, and take first 3 apart from the smallest Hash codes, then return to it
Corresponding 3 groups of bimodal images, shown in obtained result figure 4.
6, all test sets of MRI mode are taken as image to be retrieved, takes preceding 10 to return the result calculating mean accuracy
(mAP), the result obtained is 0.6765.
The feasibility of above-mentioned provable this method of experimental result.
In conclusion innovation and advantage of the invention:
1, avoid manual feature can not well mining data immanent structure the problem of, solve tradition based on manual feature
The not high problem of hash method precision.
2, it solves the problems, such as that existing most methods can only mutually be retrieved between two mode, realizes any multi-modal doctor
It learns and is retrieved mutually between image.
3, it solves the problems, such as that data local manifolds structure must be wanted in Hash mapping process, further increases retrieval essence
Degree.
One kind proposed by the present invention passes through depth based on more figure regularization depth Hash multi-modality medical image search methods
Feature replaces traditional-handwork feature, can improve Hash retrieval precision.
One kind proposed by the present invention passes through manifold based on more figure regularization depth Hash multi-modality medical image search methods
Hold mode keeps the local manifolds structure of data while to obtain Hash codes, can be further improved retrieval precision.
A kind of realized based on more figure regularization depth Hash multi-modality medical image search methods proposed by the present invention is appointed
The mutual retrieval of meaning modal data, greatly meets user demand.
Proposed by the present invention a kind of based on more figure regularization depth Hash multi-modality medical image search methods, step is clear
It is clear, it is with strong points.
The above embodiment is interpreted as being merely to illustrate the present invention rather than limit the scope of the invention.?
After the content for having read record of the invention, technical staff can be made various changes or modifications the present invention, these equivalent changes
Change and modification equally falls into the scope of the claims in the present invention.
Claims (8)
1. a kind of multi-modality medical image search method based on more figure regularization depth Hash, which is characterized in that including following
Step:
Step 1, the depth characteristic that multi-modality medical image is extracted using multichannel depth model, and depth characteristic is standardized；
The multiple neighbour's figure matrixes of feature construction of step 2, the multiple and different modal datas extracted according to step 1, to keep number
According to local manifolds structure, and construct a label matrix；
The multiple neighbour's figure matrixes and label matrix of building are fused into a figure matrix by step 3；
Step 4 combines fused figure matrix to learn to obtain multi-modal doctor using the adaptive limited Boltzmann machine RBM of mode
Learn the common Hash codes of image；
Modal data to be retrieved is generated Hash codes by the adaptive RBM of depth channel and mode by step 5；
Step 6 is calculated between the data and multi-modality medical image library of a certain mode to be retrieved using Hamming distance measure
Distance and ascending sort, user will be returned to apart from the smallest closest multi-modality medical image of n group.
2. a kind of multi-modality medical image search method based on more figure regularization depth Hash according to claim 1,
It is characterized in that, the step 1 extracts the depth characteristic of multi-modality medical image using multichannel depth model, specifically include:
The number of channels of depth model is adaptively determined according to data group mode quantity first, then multiple depth channels are connected to same
Classify layer as a whole, the whole multichannel depth model of training, training is completed to take the layer knot second from the bottom in each channel
Depth characteristic of the fruit as corresponding modal data.
3. a kind of multi-modality medical image search method based on more figure regularization depth Hash according to claim 2,
It is characterized in that, the step 1 uses Z- after the depth characteristic for extracting multi-modality medical image group, by depth characteristic
Score standardization, the feature after quantization will obey standardized normal distribution, and the quantitative formula of feature is as follows:
Wherein μ indicates the mean value of some feature, and δ indicates the standard deviation of feature.
4. a kind of multi-modality medical image search method based on more figure regularization depth Hash according to claim 3,
It is characterized in that, step 2 neighbour schemes in building, figure is considered as and gathering for n vector describes the geometry of data,
In the corresponding data point of each vector, the length of each vector is ρ, indicates the ρ data with the data point arest neighbors
Point, multiple neighbour's matrixDistance metric mode using Gauss thermonuclear distance or manhatton distance or cut ratio
Husband's distance is avenged, m indicates the mode quantity of multi-modality medical image, and i indicates the i-th of neighbour's matrix of a certain modal data building
Row, j indicate jth column；The neighbour's matrix for indicating a certain modal data building, after having constructed neighbour's figure according to depth characteristic, root
An additional label neighbour figure is constructed according to label.
5. a kind of multi-modality medical image search method based on more figure regularization depth Hash according to claim 4,
Scheme it is characterized in that, the step 3 constructs an additional label neighbour according to label, specifically include: according to label building one
A n ties up matrix, and building rule is as follows:
xiIndicate one group of image of multi-modal data, xjIndicate that any one group in remaining n-1 group image, a indicate xiWith xjIt is identical
The number of label after having constructed m+1 matrix, carries out more figure regularization matrix using following formula and merges:
Wherein μ indicates that the weight coefficient of each matrix when fusion, Ψ indicate fused matrix.
6. a kind of multi-modality medical image search method based on more figure regularization depth Hash according to claim 5,
It is characterized in that, the step 4 combines fused figure matrix to learn to obtain multi-modal medicine figure using the adaptive RBM of mode
As common Hash codes, specifically include:
The adaptive RBM of mode is improved by raw Gaussian RBM, and visual layer number is adaptively true according to data modality quantity
It is fixed, it is all connected to same hidden layer；Manifold holding matrix is added when visual layers and hidden layer are generated by conditional probability simultaneously,
So that the hidden layer Hash codes generated are able to maintain the local neighbor structure of data；The energy function of improved RBM model is as follows:
Wherein U indicates the energy function of entire improved RBM model；fi 1,fi 2,...,fi MRepresent the 1st, 2 ..., M visual layers
A certain node, hiIndicate that a certain node of hidden layer, M indicate mode quantity, N1,N2Respectively indicate the number of nodes of each visual layers
Amount and hidden node quantity；θ indicates the parameter sets of RBM, the biasing a comprising visual layers, the biasing b and visual layers of hidden layer
Connection weight w between hidden layer,Indicate the biasing of m-th of visual layers, r-th of node, bsIndicate the inclined of s-th of node of hidden layer
It sets,Indicate the connection weight of r-th node and s-th of node of hidden layer of m-th of visual layers；Indicate m-th of visual layers
R-th of node, hsIndicate s-th of node of hidden layer；It indicates that the normal distribution standard of m-th of visual layers, r-th of node is poor, is
Positive value is not trained generally, and definite value 1 is taken；λ indicates regularization weight parameter, the flatness that control hidden layer indicates, hisIndicate hidden
S-th of node of layer, hjsExpression is obtained according to fusion matrix and hisThe node of neighbour, m indicate m-th of visual layers, and r is indicated
R-th of node, the s of a certain visual layers indicate s-th of node of hidden layer.
7. a kind of multi-modality medical image search method based on more figure regularization depth Hash according to claim 6,
It is characterized in that, the step 5 is generated a certain modal data to be retrieved by the adaptive RBM of depth channel and mode
Hash codes specifically include: the corresponding modal characteristics extract when the medical image of a certain mode to be retrieved is put into its training
In channel, take convolutional neural networks number of plies value second from the bottom as its characteristic value；The same modality data obtained when according to training
Eigenmatrix mean μ and the characteristic value that standardizes of standard deviation δ；Standardized characteristic value is completed as training
The input of RBM visual layers pays attention to corresponding to visual layers when training with mode, other visual layers inputs and characteristic value same size
Null matrix does the result obtained after matrix multiple with connection weight and sign function sign is taken to obtain the Hash codes of data to be retrieved.
8. a kind of multi-modality medical image search method based on more figure regularization depth Hash according to claim 7,
It is characterized in that, the step 6 calculates the data and multi-modal medicine of a certain mode to be retrieved using Hamming distance measure
The distance between image library and ascending sort, will return to user apart from the smallest closest multi-modality medical image of n group, wherein
Hamming distance measure formulas is as follows:
Wherein k indicates the length of Hash codes, hr(x) the r Hash codes of sample x, h are indicatedr(y) the r Kazakhstan of sample y are indicated
Uncommon code,Indicate XOR operation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910048281.0A CN109902714B (en) | 2019-01-18 | 2019-01-18 | Multi-modal medical image retrieval method based on multi-graph regularization depth hashing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910048281.0A CN109902714B (en) | 2019-01-18 | 2019-01-18 | Multi-modal medical image retrieval method based on multi-graph regularization depth hashing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109902714A true CN109902714A (en) | 2019-06-18 |
| CN109902714B CN109902714B (en) | 2022-05-03 |
Family
ID=66943792
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910048281.0A Active CN109902714B (en) | 2019-01-18 | 2019-01-18 | Multi-modal medical image retrieval method based on multi-graph regularization depth hashing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109902714B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110444277A (en) * | 2019-07-19 | 2019-11-12 | 重庆邮电大学 | It is a kind of based on generating multipair anti-multi-modal brain MRI image bi-directional conversion method more |
| CN110990596A (en) * | 2019-12-04 | 2020-04-10 | 山东师范大学 | Multi-mode hash retrieval method and system based on self-adaptive quantization |
| CN111428072A (en) * | 2020-03-31 | 2020-07-17 | 南方科技大学 | Ophthalmologic multimodal image retrieval method, apparatus, server and storage medium |
| CN111666442A (en) * | 2020-06-02 | 2020-09-15 | 腾讯科技（深圳）有限公司 | Image retrieval method and device and computer equipment |
| CN111881979A (en) * | 2020-07-28 | 2020-11-03 | 复旦大学 | Multi-modal data annotation device and computer-readable storage medium containing program |
| CN112800260A (en) * | 2021-04-09 | 2021-05-14 | 北京邮电大学 | Multi-label image retrieval method and device based on deep hash energy model |
| CN113658683A (en) * | 2021-08-05 | 2021-11-16 | 重庆金山医疗技术研究院有限公司 | Disease diagnosis system and data recommendation method |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102968626A (en) * | 2012-12-19 | 2013-03-13 | 中国电子科技集团公司第三研究所 | Human face image matching method |
| US20150178786A1 (en) * | 2012-12-25 | 2015-06-25 | Catharina A.J. Claessens | Pictollage: Image-Based Contextual Advertising Through Programmatically Composed Collages |
| CN104899253A (en) * | 2015-05-13 | 2015-09-09 | 复旦大学 | Cross-modality image-label relevance learning method facing social image |
| CN105701800A (en) * | 2015-12-31 | 2016-06-22 | 上海交通大学 | Multi-mode image matching method |
| WO2016095487A1 (en) * | 2014-12-17 | 2016-06-23 | 中山大学 | Human-computer interaction-based method for parsing high-level semantics of image |
| CN106484782A (en) * | 2016-09-18 | 2017-03-08 | 重庆邮电大学 | A kind of large-scale medical image retrieval based on the study of multinuclear Hash |
| CN106777318A (en) * | 2017-01-05 | 2017-05-31 | 西安电子科技大学 | Matrix decomposition cross-module state Hash search method based on coorinated training |
| CN109166615A (en) * | 2018-07-11 | 2019-01-08 | 重庆邮电大学 | A kind of medicine CT image storage and retrieval method of random forest Hash |
-
2019
- 2019-01-18 CN CN201910048281.0A patent/CN109902714B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102968626A (en) * | 2012-12-19 | 2013-03-13 | 中国电子科技集团公司第三研究所 | Human face image matching method |
| US20150178786A1 (en) * | 2012-12-25 | 2015-06-25 | Catharina A.J. Claessens | Pictollage: Image-Based Contextual Advertising Through Programmatically Composed Collages |
| WO2016095487A1 (en) * | 2014-12-17 | 2016-06-23 | 中山大学 | Human-computer interaction-based method for parsing high-level semantics of image |
| CN104899253A (en) * | 2015-05-13 | 2015-09-09 | 复旦大学 | Cross-modality image-label relevance learning method facing social image |
| CN105701800A (en) * | 2015-12-31 | 2016-06-22 | 上海交通大学 | Multi-mode image matching method |
| CN106484782A (en) * | 2016-09-18 | 2017-03-08 | 重庆邮电大学 | A kind of large-scale medical image retrieval based on the study of multinuclear Hash |
| CN106777318A (en) * | 2017-01-05 | 2017-05-31 | 西安电子科技大学 | Matrix decomposition cross-module state Hash search method based on coorinated training |
| CN109166615A (en) * | 2018-07-11 | 2019-01-08 | 重庆邮电大学 | A kind of medicine CT image storage and retrieval method of random forest Hash |
Non-Patent Citations (7)
| Title |
|---|
| MICHAEL M.BRONSTEIN 等: "Data fusion through cross-modality metric learning using similarity-sensitive hashing", 《2010 IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
| XIAOLONG GONG 等: "Deep Semantic Correlation Learning Based Hashing for Multimedia Cross-Modal Retrieval", 《2018 IEEE INTERNATIONAL CONFERENCE ON DATA MINING》 * |
| 刘建伟 等: "半监督学习方法", 《计算机学报》 * |
| 孙韶言: "基于深度学习表征的图像检索技术", 《中国优秀博士学位论文全文数据库 信息科技辑》 * |
| 曾宪华 等: "基于多特征多核哈希学习的大规模图像检索", 《中国科学:信息科学》 * |
| 樊花 等: "基于哈希方法的跨模态检索研究进展", 《数据通信》 * |
| 陈思: "哈希码学习及其在图像检索中的应用研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110444277A (en) * | 2019-07-19 | 2019-11-12 | 重庆邮电大学 | It is a kind of based on generating multipair anti-multi-modal brain MRI image bi-directional conversion method more |
| CN110444277B (en) * | 2019-07-19 | 2023-03-28 | 重庆邮电大学 | Multi-mode brain MRI image bidirectional conversion method based on multi-generation and multi-confrontation |
| CN110990596A (en) * | 2019-12-04 | 2020-04-10 | 山东师范大学 | Multi-mode hash retrieval method and system based on self-adaptive quantization |
| CN110990596B (en) * | 2019-12-04 | 2020-09-25 | 山东师范大学 | Multi-mode hash retrieval method and system based on self-adaptive quantization |
| CN111428072A (en) * | 2020-03-31 | 2020-07-17 | 南方科技大学 | Ophthalmologic multimodal image retrieval method, apparatus, server and storage medium |
| CN111666442A (en) * | 2020-06-02 | 2020-09-15 | 腾讯科技（深圳）有限公司 | Image retrieval method and device and computer equipment |
| CN111666442B (en) * | 2020-06-02 | 2023-04-18 | 腾讯科技（深圳）有限公司 | Image retrieval method and device and computer equipment |
| CN111881979A (en) * | 2020-07-28 | 2020-11-03 | 复旦大学 | Multi-modal data annotation device and computer-readable storage medium containing program |
| CN112800260A (en) * | 2021-04-09 | 2021-05-14 | 北京邮电大学 | Multi-label image retrieval method and device based on deep hash energy model |
| CN112800260B (en) * | 2021-04-09 | 2021-08-20 | 北京邮电大学 | Multi-label image retrieval method and device based on deep hash energy model |
| CN113658683A (en) * | 2021-08-05 | 2021-11-16 | 重庆金山医疗技术研究院有限公司 | Disease diagnosis system and data recommendation method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109902714B (en) | 2022-05-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109902714A (en) | A kind of multi-modality medical image search method based on more figure regularization depth Hash | |
| CN110147457B (en) | Image-text matching method, device, storage medium and equipment | |
| CN110059198B (en) | Discrete hash retrieval method of cross-modal data based on similarity maintenance | |
| CN101916382B (en) | Method for recognizing image of plant leaf | |
| CN111428071B (en) | Zero-sample cross-modal retrieval method based on multi-modal feature synthesis | |
| CN111612066A (en) | Remote sensing image classification method based on depth fusion convolutional neural network | |
| CN108596203B (en) | Optimization method of parallel pooling layer for pantograph carbon slide plate surface abrasion detection model | |
| CN108629367A (en) | A method of clothes Attribute Recognition precision is enhanced based on depth network | |
| CN110059206A (en) | A kind of extensive hashing image search method based on depth representative learning | |
| CN104573669A (en) | Image object detection method | |
| CN106383891A (en) | Deep hash-based medical image distributed retrieval method | |
| CN107220373A (en) | A kind of Lung neoplasm CT image Hash search methods based on medical science sign and convolutional neural networks | |
| CN112732921B (en) | False user comment detection method and system | |
| CN110188827A (en) | A kind of scene recognition method based on convolutional neural networks and recurrence autocoder model | |
| CN113032613B (en) | Three-dimensional model retrieval method based on interactive attention convolution neural network | |
| CN111862260B (en) | Cross-domain dual generation type countermeasure network-based bias eliminating method and device | |
| CN106250918A (en) | A kind of mixed Gauss model matching process based on the soil-shifting distance improved | |
| CN112182275A (en) | Trademark approximate retrieval system and method based on multi-dimensional feature fusion | |
| CN112035689A (en) | Zero sample image hash retrieval method based on vision-to-semantic network | |
| CN116206158A (en) | Scene image classification method and system based on double hypergraph neural network | |
| Matrone et al. | Bubblex: An explainable deep learning framework for point-cloud classification | |
| Li et al. | Zero-shot predicate prediction for scene graph parsing | |
| CN113076490A (en) | Case-related microblog object-level emotion classification method based on mixed node graph | |
| CN117556076A (en) | Pathological image cross-modal retrieval method and system based on multi-modal characterization learning | |
| CN117273134A (en) | Zero-sample knowledge graph completion method based on pre-training language model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240123 Address after: Room 801, 85 Kefeng Road, Huangpu District, Guangzhou City, Guangdong Province Patentee after: Guangzhou Dayu Chuangfu Technology Co.,Ltd. Country or region after: China Address before: 400065 Chongwen Road, Nanshan Street, Nanan District, Chongqing Patentee before: CHONGQING University OF POSTS AND TELECOMMUNICATIONS Country or region before: China |