CN109840290A - A kind of skin lens image search method based on end-to-end depth Hash - Google Patents
A kind of skin lens image search method based on end-to-end depth Hash Download PDFInfo
- Publication number
- CN109840290A CN109840290A CN201910062340.XA CN201910062340A CN109840290A CN 109840290 A CN109840290 A CN 109840290A CN 201910062340 A CN201910062340 A CN 201910062340A CN 109840290 A CN109840290 A CN 109840290A
- Authority
- CN
- China
- Prior art keywords
- network
- image
- layer
- lens image
- skin
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A kind of skin lens image search method based on end-to-end depth Hash of the present invention, including step are as follows: step 1: establish skin lens image database；Step 2: end-to-end depth Hash network model design；Step 3: network training；Step 4: extracting depth Hash codes, constructs searching database；Step 5: retrieved skin mirror image.Advantage is: devising Res-DenseNet50 depth hash data structure, improves the fusion faculty between high-level characteristic and low-level feature, avoids loss of the information between the layers in transmittance process.The high-level characteristic extracted has better separability, to have higher retrieval rate.Realize the search method based on end-to-end depth Hash.The present invention directly learns original image, and can directly obtain the corresponding depth Hash codes of input picture from the layer second from the bottom of network, simplifies the process of skin lens image retrieval, avoids the accumulated error in conventional retrieval process between the step of front and back.
Description
Technical field
The invention belongs to skin lens image process fields, and in particular to a kind of dermoscopy figure based on end-to-end depth Hash
As search method.
Background technique
Various skin diseases endanger people's health in life, and dermoscopy diagnosis is a kind of for skin disease
The non-invasive Micro Imaging Analytical Technique of disease.The diagnostic method of previous dermatologist mainly passes through skin sem observation skin lesion area
Domain, then experience and subjective visual evaluation are relied on to make diagnostic result.This mode for relying on eye-observation easily causes vision
Fatigue, and diagnostic result has subjectivity, and repeatability is poor.And skin lens image aided diagnosis technique can be from dermoscopy
Skin lesion target is automatically extracted in image and skin lesion type is identified, so that doctor be assisted to make correct diagnosis, this mode
Have the advantages that objective repeatable.Skin lens image retrieval is in the important research in skin lens image computer-aided diagnosis
Hold, it can rapidly and accurately retrieve similar image from database, these similar images have the information made a definite diagnosis, can
Reference frame is provided to carry out clinical diagnosis for doctor.
It is main or based on image segmentation and classification at present to the research of skin lens image, dermoscopy figure in contrast
As the research achievement of retrieval aspect is also fewer.Existing skin lens image search method mainly includes based on traditional low-level feature
Skin lens image search method and skin lens image search method based on deep learning.Skin based on traditional low-level feature
Mirror image search method is first to carry out the processing such as hair, segmentation to image, then be partitioned into skin lesion region, then to skin lesion region
The low-level features such as some colors, texture, boundary are extracted, are finally retrieved using the method for Hash mapping.Since this method mentions
The feature taken is not strong to the descriptive power in skin lesion region, and can have accumulated error between multiple steps, therefore retrieves accurately
Rate is lower.And the skin lens image search method based on deep learning is to be extracted using deep learning method to skin lens image
High-level semantics features, and image retrieval is carried out using depth Hash end to end, using the extracted feature of deep learning method
There is stronger descriptive power than traditional low-level feature, better search result usually can be obtained.Due to being currently directed to skin
The research of mirror image retrieval be not also it is very deep, the accuracy rate of existing skin lens image search method be not it is very high, for
There is also very big rooms for promotion for the research of skin lens image retrieval.
Summary of the invention
Purpose:
The purpose of the present invention is to provide a kind of skin lens image search methods based on end-to-end depth Hash, it passes through
The convolutional neural networks model of end-to-end depth Hash can directly extract the corresponding Hash codes of skin lens image, the net of design
Network model not only increases the descriptive power of feature, is omitted the front and back dependence in conventional retrieval between each step, energy
Enough obtain more accurate search result.
Technical solution:
The present invention combines residual error network and DenseNet Network Theory, devises end-to-end depth Hash network mould
Type.It directly can extract depth Hash codes to original skin lens image, then realize dermoscopy by building hash table data structure
The quick-searching of image obtains the one group image most like with image to be retrieved.Specific technical solution is as follows:
The present invention is a kind of skin lens image search method based on end-to-end depth Hash, it the following steps are included:
Step 1: skin lens image database is established
The present invention constructs searching database for the skin lens image of yellow, includes N kind common skin diseases.We will adopt
The resolution sizes of the skin lens image collected are uniformly scaled K × K, construct skin lens image database with this.
Since skin lens image database is usually smaller, first database is expanded.It will be in every kind of skin disease
Rotation (90 °, 180 °, 270 °) and lateral mirror image progress data extending that library image has carried out three angles are built, can be obtained most
Whole search library data set.
Step 2: end-to-end depth Hash network model design
The present invention uses for reference the thought of DenseNet on the basis of residual error network ResNet50, and devising one has 51 layers
Convolutional neural networks Res-DenseNet50.The network includes 4 parts: 1 convolutional layer, 4 residual error groups, 1 average value
Pond layer and 2 full articulamentums.
For convolutional layer, the size of characteristic pattern can indicate that the first two value is indicated with width × height × depth
Bulk, the last one value indicate number of channels.Convolutional layer is represented with Conv#, Maxp# represents maximum value pond layer, FC# generation
The full articulamentum of table, then designed network structure detail is as follows:
1) the 1st layer of network is convolutional layer Conv1 first, having a size of 7 × 7 × 64, for mapping an image to higher-dimension sky
Between, characteristic pattern number is 64.The convolutional layer is connect with 4 groups of residual blocks.
2) it mutually contacts after level 1 volume lamination with 4 groups of residual blocks.With the increase of network depth, network can encounter gradient and disappear
It becomes estranged network degenerate problem.Residual error structure is output it to be added with input, is described as follows with formula (1):
F (x)=H (x)+x (1)
Wherein x be input, H (x) be original structure mapping function, F (x) be will export and input be added after residual error agllutination
The mapping function of structure, so as to form residual error mapping.In residual error network structure, network learning procedure is mainly to input and output
Between residual error portion learnt, which makes network be easier to train, and avoid gradient disappearance problem to a certain degree,
It can obtain better result.
In the network structure that the present invention designs, the back Conv1 is by 4 groups of residual block tandems.In view of with network depth
The increase of degree, residual error network still can encounter gradient and disappear and network degenerate problem, and therefore, we use for reference DenseNet thought,
Preceding 3 groups of outputting and inputting for residual block are merged, so that the input of a certain layer residual error structure does not depend solely on
Close to the output of residual error structure, may also rely on the output apart from farther residual block so that the high-level characteristic in network and
Low-level feature is merged, and further avoids gradient disappearance, mitigates over-fitting.For the 2nd group and the 3rd group of residual block, by
Half is reduced in wide after residual block and height in characteristic pattern, therefore, is respectively added in the forward direction line that connection is output and input
Enter the maximum value pond layer that one layer of step-length is 2, respectively Maxp1 and Maxp2, dimensionality reduction is carried out to input data.
The structural parameters of every group of residual block are as follows.
1st group include 3 residual block Conv2-x, each residual error block structure be Conv (1 × 1 × 64) → Conv (3 × 3 ×
64)→Conv(1×1×256)。
2nd group includes 4 residual block Conv3-x, and each residual error block structure is Conv (1 × 1 × 128) → Conv (3 × 3
×128)→Conv(1×1×512)。
3rd group includes 6 residual block Conv4-x, and each residual error block structure is Conv (1 × 1 × 256) → Conv (3 × 3
×256)→Conv(1×1×1024)。
4th group includes 3 residual block Conv5-x, and each residual error block structure is Conv (1 × 1 × 512) → Conv (3 × 3
×512)→Conv(1×1×2048)。
3) it is 1 average value pond layer Average pool that the 4th group of residual block is latter linked, is used to carry out characteristic
Dimensionality reduction.
4) network is finally two full articulamentum FC1 and FC2.
The purpose of full articulamentum FC1 is therefore the neuron in order to be corresponding binary code by the Feature Mapping of preceding layer
Number corresponds to number of encoding bits b, and the activation primitive of each neuron uses sigmoid function:
Wherein, x is the input of neuron, and S (x) is activation primitive output.
This function can limit the value of neuron between zero and one, convenient for the value of this layer is finally quantified as 0 or 1, obtain
To binary system Hash codes.
Full articulamentum FC2 is final task layer, and output neuron number is the skin disease type N that tranining database includes,
Allow network by the training of more classification tasks.Loss function used by training is softmax function, such as formula (3):
In formula, xiFor i-th of output of network, xjFor j-th of output of network, SiFor the class probability of the i-th class.
By formula (3) can calculate every one kind class probability Si, therefore, loss can be calculated by formula (4) and formula (5)
Loss。
In formula, N is classification number, SiFor the class probability of the i-th class, yiIt is the true tag of input picture, i.e., in addition to corresponding class
Other position is 1, remaining N-1 value is all that 0, R (W) is the regularization loss item being added, WK, lFor network final task layer l
Weight between k-th of neuron of a neuron and layer second from the bottom.
Step 3: network training
We use the skin lens image database in step 1, by the more classification tasks of image to the convolution designed in step 2
Neural network Res-DenseNet50 is trained.Deep learning network needs big-sample data to be trained, but medical image
Sample set it is usually smaller, therefore, generally use the mode of transfer learning.The convolutional neural networks Res- that the present invention designs
DenseNet50 therefore can train present networks using the method for transfer learning by means of the frame of Resnet50.
Herein using the network model in step 2, on the skin lens image database in step 1, by more points of image
Generic task is to being trained.Training method is based on transfer training, and using the good parameter of Imagenet pre-training, gradient decline is used
Stochastic gradient descent method, batch are set as 64, and loss function uses softmax function.Initial learning rate is set as 0.001, the
20,30,40 10 times of wheel decaying, weight decaying are set as 0.001, train 50 wheels altogether.All parameters needed for network are finally obtained, i.e.,
Weight and biasing in network model.
Step 4: extracting depth Hash codes, constructs searching database
Using network model trained in step 3, the image of skin lens image database in step 1 is input to net
Network model, then by network layer second from the bottom, i.e., the b place value of full articulamentum FC1 extracts and quantifies by binaryzation formula, formula
Such as formula (6).
In formula, siFor the activation primitive value of i-th of neuron of this layer, HiBinary code after quantifying for i-th of neuron.
Then the b binary code is the corresponding depth Hash codes of input picture, these Hash codes are pressed Hash table structure
Construct the searching database of skin lens image.An array is created, element subscript is the code value of Hash codes, and each element is one
A linked list head, what is stored in chained list is the skin lens image serial number that Hash codes are the element subscript in searching database.Hash table
Data structure can increase substantially recall precision, when retrieving image, can directly retrieve one group without traversing entire database
Serial number of the similar skin lens image in searching database, i.e. search result.
In addition, also the feature vector of 2048 dimensions of network layer third from the bottom is extracted as the high-level characteristic of input picture
Come, constructs a high dimensional feature database.After retrieving one group of similar skin lens image according to Hash codes, with 2048 high levels
Feature can calculate to obtain the identical image of this group of Hash codes and image similarity to be retrieved, and sequence can obtain final search result, with this
To improve search result.
Step 5: retrieved skin mirror image
Skin lens image to be retrieved is input in trained network, is respectively obtained from layer second from the bottom and third layer
Corresponding b bit depth Hash codes and 2048 high-level characteristics.Then it is retrieved in searching database according to Hash codes, directly
The skin lens image that identical Hash codes can be obtained using the code value of the Hash codes as the linked list head of element subscript is returned as preliminary
Search result.The Euclidean distance in image and preliminary search result to be retrieved between image is calculated further according to high-level characteristic, is obtained
Similitude simultaneously sorts, and final search result can be obtained.
Euclidean distance formula is as follows:
A and B is the feature vector that two n are in formula, and d (A, B) is the Euclidean distance of A and B, aiAnd biFor i-th in A and B
A element.
By above step, we can relatively accurately be retrieved in skin lens image database and image to be retrieved
The similar skin lens image made a definite diagnosis and its relevant information are objectively suggested and are referred to provide for dermatologist,
Improve the accuracy rate of diagnosis.
The beneficial effects of the present invention are:
(1) residual error network is combined with DenseNet thought, devises Res-DenseNet50 depth hash data structure, changes
The fusion faculty being apt between high-level characteristic and low-level feature avoids information transmittance process between the layers to a certain extent
In loss.The high-level characteristic extracted has better separability, to have higher retrieval rate.
(2) existing residual error structural framing is cleverly utilized, so as to complete network training with transfer learning, solves
Sample deficiency problem.
(3) search method based on end-to-end depth Hash is realized.Method is not only able to extract target area end to end
The feature in domain, while area's another characteristic between target and background can be also extracted, more global semantic informations are contained, thus
Further improve retrieval performance.In addition, the present invention directly learns original image, and from the layer second from the bottom of network
The corresponding depth Hash codes of input picture can be directly obtained, the process of skin lens image retrieval is simplified, avoids conventional retrieval
Accumulated error in process between the step of front and back.While improving retrieval rate, recall precision is also improved.
Detailed description of the invention
Fig. 1 is implementation flow chart of the invention.
Fig. 2 (a) is the exemplary diagram of mole.
Fig. 2 (b) is the exemplary diagram of seborrheic keratosis.
Fig. 2 (c) is the exemplary diagram of psoriasis.
Fig. 2 (d) is the exemplary diagram of eczema.
Fig. 3 is convolutional neural networks model Res-DenseNet50 structure chart proposed by the present invention.
Fig. 4 is Hash table structural schematic diagram.
Specific embodiment
Technical solution for a better understanding of the present invention, with reference to the accompanying drawing and this hair is discussed in detail in specific embodiment
It is bright.
Embodiment 1: the skin lens image retrieval example of mole, seborrheic keratosis, psoriasis and eczema, specific implementation
Process is as shown in Figure 1.
Step 1: skin lens image database is established
The present invention constructs searching database for the skin disease of yellow, includes four kinds of common skin diseases, is pigment respectively
Mole, seborrheic keratosis, psoriasis and eczema.By collected skin lens image, resolution sizes are uniformly scaled 224 ×
224, skin lens image database is constructed, four kinds of dermopathic exemplary diagram such as Fig. 2 (a), (b), (c), (d) are shown.Every kind of skin
Disease has 700 skin lens images, i.e. data set totally 2800 images.Since database is smaller, it is expanded.Every kind
Skin disease image, 500 therein for search library building, 200 as test.And to 500 in every kind of skin disease
Rotation (90 °, 180 °, 270 °) and lateral mirror image progress data extending that library image has carried out three angles are built, is obtained 16000
Image is opened as search library data set.
Step 2: end-to-end depth Hash network model design
The present invention uses for reference the thought of DenseNet on the basis of residual error network ResNet50, and devising one has 51 layers
Convolutional neural networks Res-DenseNet50.The network includes 4 parts: 1 convolutional layer, 4 groups of residual blocks, 1 average value
Pond layer and 2 full articulamentums.
For convolutional layer, the size of characteristic pattern can indicate that the first two value is indicated with width × height × depth
Bulk, the last one value indicate number of channels.Convolutional layer is represented with Conv#, Maxp# represents maximum value pond layer, FC# generation
The full articulamentum of table, then designed network structure detail is as follows:
1) the 1st layer of network is convolutional layer Conv1 first, having a size of 7 × 7 × 64, for mapping an image to higher-dimension sky
Between, which is connected by 4 groups of residual blocks.
2) in the network structure that the present invention designs, the back Conv1 is by 4 groups of residual block tandems.In view of with network
The increase of depth, residual error network still can encounter gradient and disappear and network degenerate problem, and therefore, we use for reference DenseNet think of
Think, preceding 3 groups of outputting and inputting for residual block are merged, so that the input of a certain layer residual error structure not only relies only on
In the output close to residual error structure, the output apart from farther residual block may also rely on, so that high-level characteristic and low layer are special
Sign is merged, and further avoids gradient disappearance, mitigates over-fitting.For the 2nd group and the 3rd group of residual block, due to feature
Figure reduces half in wide after residual block and height, therefore, one layer of step-length is respectively added in the forward direction line output and input
For 2 maximum value pond layer, respectively Maxp1 and Maxp2 carry out dimensionality reduction to input data.
The structural parameters of every group of residual block are as shown in table 1.
14 residual error group structural parameters of table
3) it is 1 average value pond layer Average pool that the 4th group of residual block is latter linked, is used to carry out characteristic
Dimensionality reduction extracts main feature.
4) network is finally two full articulamentum FC1 and FC2.
The purpose of full articulamentum FC1 is therefore the neuron in order to be corresponding binary code by the Feature Mapping of preceding layer
Number corresponds to number of encoding bits b, and the activation primitive of each neuron uses sigmoid function:
Wherein, x is the input of neuron, and S (x) is activation primitive output.
This function can limit the value of neuron between zero and one, convenient for the value of this layer is finally quantified as 0 or 1, obtain
To binary system Hash codes.
Full articulamentum FC2 is final task layer, and output neuron number is the skin disease type N that tranining database includes,
Allow network by the training of more classification tasks.Loss function used by training is softmax function, such as formula (9):
In formula, xiFor i-th of output of network, xjFor j-th of output of network, SiFor the class probability of the i-th class.
By formula (9) can calculate every one kind class probability Si, therefore, damage can be calculated by formula (10) and formula (11)
Lose Loss.
In formula, N is classification number, SiFor the class probability of the i-th class, y is the true tag of input picture, i.e., in addition to corresponding class
Other position is 1, remaining N-1 value is all that 0, R (W) is the regularization loss item being added, WK, lFor network final task layer l
Weight between k-th of neuron of a neuron and layer second from the bottom.
Network structure is as shown in Figure 3.
Step 3: network training
We use the skin lens image database in step 1, by the more classification tasks of image to the network designed in step 2
Model Res-DenseNet50 is trained.Deep learning network needs big-sample data to be trained, but the sample of medical image
This collection is usually smaller, therefore, generallys use the mode of transfer learning.The network model Res-DenseNet50 that the present invention designs is borrowed
The frame of Resnet50 has been helped, therefore present networks can have been trained using the method for transfer learning.
Herein using the network model in step 2, on the skin lens image database in step 1, by more points of image
Generic task is to being trained.Training method is based on transfer training, and using the good parameter of Imagenet pre-training, gradient decline is used
Stochastic gradient descent method, batch are set as 64, and loss function uses softmax function.Initial learning rate is set as 0.001, the
20,30,40 10 times of wheel decaying, weight decaying are set as 0.001, train 50 wheels altogether.All parameters needed for network are finally obtained, i.e.,
Network model.
Step 4: extracting depth Hash codes, constructs searching database
Using trained network model, the image of skin lens image database in step 1 is input to network model,
Again by network layer second from the bottom, i.e., 8 place values of full articulamentum FC1 extract and carry out binaryzation, binaryzation formula are as follows:
In formula, siFor the activation primitive value of i-th of neuron of this layer, HiBinary code after quantifying for i-th of neuron.
Then 8 binary codes are the corresponding depth Hash codes of input picture, these Hash codes are pressed Hash table structure
The searching database of skin lens image is constructed, and by the depth Hash codes extracted by Hash table structure building skin lens image
Searching database, Hash table structure are as shown in Figure 4.Concordance list is an array, and element subscript is the code value of Hash codes, Mei Geyuan
Element is a linked list head, and what is stored in chained list is the skin lens image serial number that Hash codes are the element subscript in searching database.
Hash table data structure can increase substantially recall precision, when retrieving image, can directly retrieve without traversing entire database
To serial number of one group of similar skin lens image in searching database, i.e. search result.
In addition, also the feature vector of 2048 dimensions of network layer third from the bottom is extracted as the high-level characteristic of input picture
Come, constructs a high dimensional feature database.After retrieving one group of similar skin lens image according to Hash codes, with 2048 high levels
Feature can calculate to obtain the identical image of this group of Hash codes and image similarity to be retrieved, and sequence can obtain final search result, with this
To improve search result.
Step 5: retrieved skin mirror image
Skin lens image to be retrieved is input in trained network, is respectively obtained from layer second from the bottom and third layer
Corresponding 8 bit depth Hash codes and 2048 high-level characteristics.Then it is retrieved in searching database according to Hash codes, directly
The skin lens image that identical Hash codes can be obtained using the code value of the Hash codes as the linked list head of element subscript is returned as preliminary
Search result.The Euclidean distance in image and preliminary search result to be retrieved between image is calculated further according to high-level characteristic, is obtained
Similitude simultaneously sorts, and final search result can be obtained.
Euclidean distance formula is as follows:
A and B is the feature vector that two n are in formula, and d (A, B) is the Euclidean distance of A and B, aiAnd biFor i-th in A and B
A element.
To 4 kinds of skin diseases in database, totally 800 images are retrieved, and each retrieval is most like by preceding ten
For skin lens image as search result, table 2 gives every kind of dermopathic retrieval rate and average retrieval rate, can see
Average Accuracy has reached 71.94% out, and effect is satisfactory.
Retrieval performance comparison before and after 2 residual error network improvement of table
Claims (5)
1. a kind of skin lens image search method based on end-to-end depth Hash, which is characterized in that it the following steps are included:
Step 1: skin lens image database is established
Searching database is constructed for the skin lens image of yellow, comprising common skin diseases by collected skin lens image
Resolution sizes are uniformly scaled K × K, construct skin lens image database with this；
Step 2: end-to-end depth Hash network model design
Design one has 51 layers of convolutional neural networks Res-DenseNet50；The network includes 4 parts: 1 convolutional layer,
4 residual error groups, 1 average value pond layer and 2 full articulamentums；
For convolutional layer, the size of characteristic pattern is indicated with width × height × depth, the first two value representation space size,
The last one value indicates number of channels；Convolutional layer is represented with Conv#, Maxp# represents maximum value pond layer, and FC# represents full connection
Layer, then designed network structure detail is as follows:
1) the 1st layer of network is convolutional layer Conv1 first, special for mapping an image to higher dimensional space having a size of 7 × 7 × 64
Levying figure number is 64；The convolutional layer is connect with 4 groups of residual blocks；
2) it mutually contacts after level 1 volume lamination with 4 groups of residual blocks；Residual error structure is output it to be added with input, with formula (1)
It is described as follows:
F (x)=H (x)+x (1)
Wherein x be input, H (x) be original structure mapping function, F (x) be will export with input be added after residual error block structure
Mapping function, so as to form residual error mapping；
The back Conv1 is by 4 groups of residual block tandems；Preceding 3 groups of outputting and inputting for residual block are merged, so that network
In high-level characteristic and low-level feature merged；For the 2nd group and the 3rd group of residual block, since characteristic pattern is passing through residual error
Wide and height reduces half after block, therefore, the maximum that one layer of step-length is 2 is respectively added in the forward direction line that connection is output and input
It is worth pond layer, respectively Maxp1 and Maxp2, dimensionality reduction is carried out to input data；
The structural parameters of every group of residual block are as follows；
1st group includes 3 residual block Conv2-x, and each residual error block structure is Conv (1 × 1 × 64) → Conv (3 × 3 × 64)
→Conv(1×1×256)；
2nd group include 4 residual block Conv3-x, each residual error block structure be Conv (1 × 1 × 128) → Conv (3 × 3 ×
128)→Conv(1×1×512)；
3rd group include 6 residual block Conv4-x, each residual error block structure be Conv (1 × 1 × 256) → Conv (3 × 3 ×
256)→Conv(1×1×1024)；
4th group include 3 residual block Conv5-x, each residual error block structure be Conv (1 × 1 × 512) → Conv (3 × 3 ×
512)→Conv(1×1×2048)；
3) it is 1 average value pond layer Average pool that the 4th group of residual block is latter linked, for dropping to characteristic
Dimension；
4) network is finally two full articulamentum FC1 and FC2；
The purpose of full articulamentum FC1 is therefore the neuron number in order to be corresponding binary code by the Feature Mapping of preceding layer
Number of encoding bits b is corresponded to, the activation primitive of each neuron uses sigmoid function:
Wherein, x is the input of neuron, and S (x) is activation primitive output；
The value limitation of neuron between zero and one, convenient for the value of this layer is finally quantified as 0 or 1, is obtained binary system by this function
Hash codes；
Full articulamentum FC2 is final task layer, and output neuron number is the skin disease type that tranining database includes, and makes network
By the training of more classification tasks；Loss function used by training is softmax function, such as formula (3):
In formula, xiFor i-th of output of network, xjFor j-th of output of network, SiFor the class probability of the i-th class；
By formula (3) calculate every one kind class probability Si, therefore, loss Loss is calculated by formula (4) and formula (5)；
In formula, N is classification number, SiFor the class probability of the i-th class, yiIt is the true tag of input picture, i.e., in addition to corresponding classification
Position is 1, remaining N-1 value is all that 0, R (W) is the regularization loss item being added, WK, lFor first of mind of network final task layer
Through the weight between member and k-th of neuron of layer second from the bottom；
Step 3: network training
With the skin lens image database in step 1, by the more classification tasks of image to the convolutional neural networks designed in step 2
Res-DenseNet50 is trained；
Step 4: extracting depth Hash codes, constructs searching database
Using network model trained in step 3, the image of skin lens image database in step 1 is input to network mould
Type, then by network layer second from the bottom, i.e., the b place value of full articulamentum FC1 extracts and quantifies by binaryzation formula, formula such as formula
(6)；
In formula, siFor the activation primitive value of i-th of neuron of this layer, HiBinary code after quantifying for i-th of neuron；
The feature vector of 2048 dimensions of network layer third from the bottom is extracted as the high-level characteristic of input picture, constructs one
High dimensional feature database；After retrieving one group of similar skin lens image according to Hash codes, calculated with 2048 high-level characteristics
The identical image of this group of Hash codes and image similarity to be retrieved, sort to obtain final search result, improves search result with this；
Step 5: retrieved skin mirror image
Skin lens image to be retrieved is input in trained network, respectively obtains correspondence from layer second from the bottom and third layer
B bit depth Hash codes and 2048 high-level characteristics；Then it is retrieved in searching database according to Hash codes, is directly returned
Using the code value of the Hash codes as the linked list head of element subscript, the skin lens image of identical Hash codes is obtained as preliminary search knot
Fruit；The Euclidean distance in image and preliminary search result to be retrieved between image is calculated further according to high-level characteristic, obtains similitude
And sort, obtain final search result；
Euclidean distance formula is as follows:
A and B is the feature vector that two n are in formula, and d (A, B) is the Euclidean distance of A and B, aiAnd biFor i-th yuan in A and B
Element.
2. a kind of skin lens image search method based on end-to-end depth Hash according to claim 1, feature exist
In: common skin diseases include: mole, seborrheic keratosis, psoriasis and eczema.
3. a kind of skin lens image search method based on end-to-end depth Hash according to claim 1, feature exist
In: first database is expanded, by every kind of skin disease build library image carried out being rotated by 90 ° of three angles, 180 °,
270 ° and lateral mirror image progress data extending, obtain final search library data set.
4. a kind of skin lens image search method based on end-to-end depth Hash according to claim 1, feature exist
In: using the network model in step 2, on the skin lens image database in step 1, by the more classification tasks of image into
Row training；Training method is based on transfer training, and using the good parameter of Imagenet pre-training, gradient decline is using under stochastic gradient
Drop method, batch are set as 64, and loss function uses softmax function；Initial learning rate is set as 0.001, declines in the 20th, 30,40 wheels
Subtract 10 times, weight decaying is set as 0.001, trains 50 wheels altogether；All parameters needed for network are finally obtained, i.e., in network model
Weight and biasing.
5. a kind of skin lens image search method based on end-to-end depth Hash according to claim 1, feature exist
In: b binary codes are the corresponding depth Hash codes of input picture, these Hash codes are constructed dermoscopy by Hash table structure
The searching database of image；An array is created, element subscript is the code value of Hash codes, and each element is a linked list head,
What is stored in chained list is the skin lens image serial number that Hash codes are the element subscript in searching database；Hash table data structure is big
Amplitude improves recall precision, when retrieving image, can directly retrieve one group of similar dermoscopy figure without traversing entire database
As the serial number in searching database, i.e. search result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910062340.XA CN109840290B (en) | 2019-01-23 | 2019-01-23 | End-to-end depth hash-based dermoscope image retrieval method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910062340.XA CN109840290B (en) | 2019-01-23 | 2019-01-23 | End-to-end depth hash-based dermoscope image retrieval method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109840290A true CN109840290A (en) | 2019-06-04 |
| CN109840290B CN109840290B (en) | 2020-09-22 |
Family
ID=66884006
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910062340.XA Active CN109840290B (en) | 2019-01-23 | 2019-01-23 | End-to-end depth hash-based dermoscope image retrieval method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109840290B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110633385A (en) * | 2019-09-24 | 2019-12-31 | 天津医科大学 | Retrieval and compression method of medical image |
| CN110674673A (en) * | 2019-07-31 | 2020-01-10 | 国家计算机网络与信息安全管理中心 | Key video frame extraction method, device and storage medium |
| CN111583256A (en) * | 2020-05-21 | 2020-08-25 | 北京航空航天大学 | Dermatoscope image classification method based on rotating mean value operation |
| CN112084362A (en) * | 2020-08-07 | 2020-12-15 | 北京航空航天大学 | Image hash retrieval method based on hierarchical feature complementation |
| CN112948611A (en) * | 2021-03-01 | 2021-06-11 | 北京航空航天大学 | Skin mirror image retrieval method based on Cauchy rotation loss resistant function |
| CN114376526A (en) * | 2022-01-12 | 2022-04-22 | 广东药科大学 | Skin state analysis method and skin care mirror |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160358043A1 (en) * | 2015-06-05 | 2016-12-08 | At&T Intellectual Property I, L.P. | Hash codes for images |
| US20170293838A1 (en) * | 2016-04-06 | 2017-10-12 | Nec Laboratories America, Inc. | Deep high-order exemplar learning for hashing and fast information retrieval |
| CN107943938A (en) * | 2017-11-23 | 2018-04-20 | 清华大学 | A kind of large-scale image similar to search method and system quantified based on depth product |
| US20180276528A1 (en) * | 2015-12-03 | 2018-09-27 | Sun Yat-Sen University | Image Retrieval Method Based on Variable-Length Deep Hash Learning |
| CN108932314A (en) * | 2018-06-21 | 2018-12-04 | 南京农业大学 | A kind of chrysanthemum image content retrieval method based on the study of depth Hash |
| CN109064405A (en) * | 2018-08-23 | 2018-12-21 | 武汉嫦娥医学抗衰机器人股份有限公司 | A kind of multi-scale image super-resolution method based on dual path network |
| CN109165306A (en) * | 2018-08-09 | 2019-01-08 | 长沙理工大学 | Image search method based on the study of multitask Hash |
| CN109241313A (en) * | 2018-08-14 | 2019-01-18 | 大连大学 | A kind of image search method based on the study of high-order depth Hash |
-
2019
- 2019-01-23 CN CN201910062340.XA patent/CN109840290B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160358043A1 (en) * | 2015-06-05 | 2016-12-08 | At&T Intellectual Property I, L.P. | Hash codes for images |
| US20180276528A1 (en) * | 2015-12-03 | 2018-09-27 | Sun Yat-Sen University | Image Retrieval Method Based on Variable-Length Deep Hash Learning |
| US20170293838A1 (en) * | 2016-04-06 | 2017-10-12 | Nec Laboratories America, Inc. | Deep high-order exemplar learning for hashing and fast information retrieval |
| CN107943938A (en) * | 2017-11-23 | 2018-04-20 | 清华大学 | A kind of large-scale image similar to search method and system quantified based on depth product |
| CN108932314A (en) * | 2018-06-21 | 2018-12-04 | 南京农业大学 | A kind of chrysanthemum image content retrieval method based on the study of depth Hash |
| CN109165306A (en) * | 2018-08-09 | 2019-01-08 | 长沙理工大学 | Image search method based on the study of multitask Hash |
| CN109241313A (en) * | 2018-08-14 | 2019-01-18 | 大连大学 | A kind of image search method based on the study of high-order depth Hash |
| CN109064405A (en) * | 2018-08-23 | 2018-12-21 | 武汉嫦娥医学抗衰机器人股份有限公司 | A kind of multi-scale image super-resolution method based on dual path network |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110674673A (en) * | 2019-07-31 | 2020-01-10 | 国家计算机网络与信息安全管理中心 | Key video frame extraction method, device and storage medium |
| CN110633385A (en) * | 2019-09-24 | 2019-12-31 | 天津医科大学 | Retrieval and compression method of medical image |
| CN111583256A (en) * | 2020-05-21 | 2020-08-25 | 北京航空航天大学 | Dermatoscope image classification method based on rotating mean value operation |
| CN111583256B (en) * | 2020-05-21 | 2022-11-04 | 北京航空航天大学 | Dermatoscope image classification method based on rotating mean value operation |
| CN112084362A (en) * | 2020-08-07 | 2020-12-15 | 北京航空航天大学 | Image hash retrieval method based on hierarchical feature complementation |
| CN112084362B (en) * | 2020-08-07 | 2022-06-14 | 北京航空航天大学 | Image hash retrieval method based on hierarchical feature complementation |
| CN112948611A (en) * | 2021-03-01 | 2021-06-11 | 北京航空航天大学 | Skin mirror image retrieval method based on Cauchy rotation loss resistant function |
| CN112948611B (en) * | 2021-03-01 | 2023-06-23 | 北京航空航天大学 | Dermatological image retrieval method based on cauchy anti-rotation loss function |
| CN114376526A (en) * | 2022-01-12 | 2022-04-22 | 广东药科大学 | Skin state analysis method and skin care mirror |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109840290B (en) | 2020-09-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109840290A (en) | A kind of skin lens image search method based on end-to-end depth Hash | |
| CN111476292B (en) | Small sample element learning training method for medical image classification processing artificial intelligence | |
| CN108734208B (en) | Multi-source heterogeneous data fusion system based on multi-mode deep migration learning mechanism | |
| CN106651830A (en) | Image quality test method based on parallel convolutional neural network | |
| CN107609503A (en) | Intelligent cancerous tumor cell identifying system and method, cloud platform, server, computer | |
| CN109977955A (en) | A kind of precancerous lesions of uterine cervix knowledge method for distinguishing based on deep learning | |
| CN109711426A (en) | A kind of pathological picture sorter and method based on GAN and transfer learning | |
| CN106408001A (en) | Rapid area-of-interest detection method based on depth kernelized hashing | |
| CN111767952B (en) | Interpretable lung nodule benign and malignant classification method | |
| CN106909938A (en) | Viewing angle independence Activity recognition method based on deep learning network | |
| CN115147600A (en) | GBM multi-mode MR image segmentation method based on classifier weight converter | |
| CN104408697B (en) | Image Super-resolution Reconstruction method based on genetic algorithm and canonical prior model | |
| CN113724206A (en) | Fundus image blood vessel segmentation method and system based on self-supervision learning | |
| CN110517272A (en) | Blood cell segmentation method based on deep learning | |
| Qin et al. | Convolutional neural networks and hash learning for feature extraction and of fast retrieval of pulmonary nodules | |
| CN115661165A (en) | Glioma fusion segmentation system and method based on attention enhancement coding and decoding network | |
| Ma et al. | Retinal vessel segmentation based on generative adversarial network and dilated convolution | |
| CN116758090A (en) | Medical image segmentation method based on multi-scale subtraction | |
| Rashad et al. | Content-based medical image retrieval based on deep features expansion | |
| CN108960005A (en) | The foundation and display methods, system of subjects visual label in a kind of intelligent vision Internet of Things | |
| CN112948611B (en) | Dermatological image retrieval method based on cauchy anti-rotation loss function | |
| CN116310335A (en) | Method for segmenting pterygium focus area based on Vision Transformer | |
| Yang et al. | Automatic brain tumor segmentation using cascaded FCN with DenseCRF and K-means | |
| CN115565671A (en) | Atrial fibrillation auxiliary analysis method based on cross-model mutual teaching semi-supervision | |
| CN115937590A (en) | Skin disease image classification method with CNN and Transformer fused in parallel |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |