CN113705630A - Skin lesion image classification method - Google Patents

Skin lesion image classification method Download PDF

Info

Publication number
CN113705630A
CN113705630A CN202110911205.5A CN202110911205A CN113705630A CN 113705630 A CN113705630 A CN 113705630A CN 202110911205 A CN202110911205 A CN 202110911205A CN 113705630 A CN113705630 A CN 113705630A
Authority
CN
China
Prior art keywords
lesion
image block
skin lesion
data set
sample image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110911205.5A
Other languages
Chinese (zh)
Other versions
CN113705630B (en
Inventor
王玉峰
万承北
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Posts and Telecommunications
Original Assignee
Nanjing University of Posts and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Posts and Telecommunications filed Critical Nanjing University of Posts and Telecommunications
Priority to CN202110911205.5A priority Critical patent/CN113705630B/en
Publication of CN113705630A publication Critical patent/CN113705630A/en
Application granted granted Critical
Publication of CN113705630B publication Critical patent/CN113705630B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20132—Image cropping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30088—Skin; Dermal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Evolutionary Biology (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a skin lesion image classification method, which comprises the following steps: performing center cutting on a skin lesion image to be classified to obtain a first image block; extracting the features of the first image block by using an upper branch network in the neural network model to obtain a first feature vector; obtaining coordinates of a lesion area according to the first feature vector; cutting the first image block according to the coordinates of the lesion area to obtain a second image block; extracting the features of the second image block by using a lower branch network in the neural network model to obtain a second feature vector; and performing feature fusion on the first feature vector and the second feature vector to obtain skin lesion category prediction probability, and obtaining the skin lesion category of the skin lesion image to be classified according to the skin lesion category prediction probability. The method can rapidly, objectively and accurately judge the skin lesion type of the skin lesion image.

Description

Skin lesion image classification method
Technical Field
The invention relates to a skin lesion image classification method, and belongs to the technical field of image processing.
Background
Skin cancer has been one of the major life-threatening cancers in humans, and skin melanoma is one of the major categories of skin cancer. Since skin melanoma looks very much like a mole at an early stage, it is difficult for an ordinary person to recognize it by the naked eye. Skin lesions such as skin melanoma can be identified by processing skin images, currently, most hospitals amplify local skin by using a skin mirror to eliminate interference around the skin, then obtain skin images, and judge skin lesion types according to the skin images by professional dermatologists.
The appearance of the deep neural network provides hopes for accurately and quickly judging the skin lesion type, but the skin lesion area of the skin image in the existing skin lesion data set is relatively small, the skin lesion area is different in size and the appearance difference characteristic is not obvious, and meanwhile, the data set distribution in the real world often has unbalanced characteristics, so that the efficiency and the accuracy of the skin lesion image classification system based on the neural network are seriously influenced by the characteristics.
Disclosure of Invention
In order to improve the efficiency and the accuracy of a skin lesion image classification system based on a neural network, the invention provides a skin lesion image classification method, which predicts the probability of skin lesions by using a neural network model based on a multi-scale double-layer attention mechanism, finally obtains accurate and reliable skin lesion classes according to the probability of skin lesions, and improves the efficiency and the accuracy of skin lesion image classification.
In order to solve the technical problems, the invention adopts the following technical means:
the invention provides a skin lesion image classification method, which comprises the following steps:
performing center cutting on a skin lesion image to be classified to obtain a first image block;
extracting features of the first image block by adopting an upper branch network of a trained neural network model to obtain a first feature vector, wherein the neural network model adopts a neural network model based on a multi-scale double-layer attention mechanism;
carrying out lesion area positioning according to the first feature vector by using a lesion positioning structure in the upper branch network;
cutting the first image block according to the located lesion area to obtain a second image block;
extracting the features of the second image block by using a lower branch network of the trained neural network model to obtain a second feature vector;
performing feature fusion on the first feature vector and the second feature vector by using a feature fusion structure in the lower branch network to obtain a fusion vector;
processing the fusion vector by utilizing a softmax activation function in an output layer of the lower branch network to obtain the skin lesion category prediction probability;
and classifying the skin lesion images to be classified according to the skin lesion category prediction probability.
Furthermore, the upper branch network of the neural network model comprises a feature extraction structure, an auxiliary output layer, a lesion location structure and a cutting scaling structure, and the lower branch network of the neural network model comprises a feature extraction structure, an auxiliary output layer, a feature fusion structure and an output layer, wherein the feature extraction structure comprises a convolution layer and a plurality of attention residual error unit learning structures ARL, and the lesion location structure comprises a hidden layer and an output layer.
Further, the method for obtaining the first feature vector comprises:
inputting the first image block into the convolution layer of the feature extraction structure of the upper branch network, and obtaining a middle vector X through a Relu nonlinear activation function1；
Learning structure ARL versus intermediate vector X using several attention residual units1Performing convolution, normalization and downsampling processing to obtain an output vector y;
processing the output vector y by utilizing a global average pooling layer to obtain correspondence of the first image blockFirst feature vector F of1。
Further, the method for locating the lesion area according to the first feature vector by using the lesion locating structure in the upper branch network comprises the following steps:
the first feature vector F1Inputting the state of the hidden layer into a hidden layer of a lesion positioning structure, and obtaining a hidden layer state g by utilizing a Relu nonlinear activation function:
g＝Relu(U3F1+b3) (1)
wherein ,U3Parameter matrix as a hidden layer, b3A bias term that is a hidden layer;
according to the hidden layer state g, acquiring the coordinates of a lesion area of the first image block by using a sigmoid nonlinear activation function in an output layer of the lesion positioning structure, wherein the expression of the coordinates of the lesion area is as follows:
[tx，ty，tl]＝n*sigmoid(U4g+b4) (2)
wherein ,txAbscissa representing center point of lesion region, tyOrdinate, t, representing the center point of the lesionlRepresenting the radius of the lesion, n being the side length of the first image block, U4Is a parameter matrix of the output layer, b4Is the bias term for the output layer.
Further, the method for acquiring the second image block includes:
obtaining the vertex coordinates of the clipping area in the first image block according to the coordinates of the lesion area: the coordinate of the upper left corner of the cutting area is (t)x(tl)，ty(tl)) The coordinate of the lower left corner of the clipping region is (t)x(tl)，ty(br)) The coordinate of the upper right corner of the cutting area is (t)x(br)，ty(tl)) The coordinate of the lower right corner of the cutting area is (t)x(br)，ty(br)), wherein ,tx(tl)＝tx-tl，ty(tl)＝ty-tl，tx(br)＝tx+tl，ty(br)＝ty+tl；
Cutting the first image block according to the vertex coordinates of the cutting area to obtain a cutting image corresponding to the first image block;
scaling the cut image according to the side length of the first image block to obtain a second image block:
Figure BDA0003203683630000041
wherein ,
Figure BDA0003203683630000042
representing the pixel values in the ith row and jth column in the second image block,
Figure BDA0003203683630000043
denotes a pixel value on the ith row and the w column in the clipped image, and α ═ h- [ i/λ]，β＝w-[j/λ]，
Figure BDA0003203683630000044
[·]For the rounding function, {. cndot.x(tl)，tx(br)]，w∈[ty(tl)，ty(br)]，i，j∈{1，2，…，n}。
Further, the training method of the neural network model comprises the following steps:
obtaining a skin lesion dataset comprising a plurality of sample images under a plurality of skin lesion categories;
performing center cutting on each sample image in the skin lesion data set to obtain a first sample image block, and forming a preprocessed skin lesion data set by using all the first sample image blocks;
dividing the preprocessed skin lesion data set into a plurality of category data sets according to skin lesion categories, and decomposing each category data set into a plurality of sub-category data sets according to image correlation;
performing feature extraction on each first sample image block in each sub-data set by using a feature extraction structure of an upper branch network to obtain a first feature vector, and obtaining a first lesion category prediction probability by using an auxiliary output layer;
positioning a lesion area of each first sample image block according to the first feature vector by using a lesion positioning structure of the upper branch network;
cutting each first sample image block according to the located lesion area to obtain a second sample image block;
performing feature extraction on each second sample image block by using a feature extraction structure of the lower branch network to obtain a second feature vector, and obtaining a second lesion category prediction probability by using an auxiliary output layer;
performing feature fusion on the first feature vector and the second feature vector by using a feature fusion structure to obtain a fusion vector;
processing the fusion vector by utilizing a softmax activation function in an output layer to obtain the skin lesion class prediction probability;
and performing parameter training on the neural network model by using the permutation loss function and the weighting loss function based on the first lesion category prediction probability, the second lesion category prediction probability and the skin lesion category prediction probability, and obtaining the trained neural network model through iterative convergence.
Further, assuming that there are d category datasets in the preprocessed skin lesion dataset, and Z is 1, 2, …, d, the method for decomposing the Z-th category dataset into a plurality of sub-category datasets according to the image correlation is as follows:
(1) carrying out gray processing on each first sample image block in the Z-th category data set to obtain a gray image corresponding to the first sample image block;
(2) randomly selecting a first sample image block from the Z category data set as an initial clustering center cZ1；
(3) Calculating each first sample image block to an initial clustering center c in the Z category data set according to the gray level image corresponding to the first sample image blockZ1And calculating the probability of each first sample image block being selected as the next cluster center according to the distance, the calculation formula is as follows:
Figure BDA0003203683630000061
wherein ,zkRepresents the kth first sample image block, P (Z), in the Z category datasetk) Denotes zkProbability of being selected as next cluster center, D (z)k) Denotes zkTo the initial cluster center cZ1K is 1, 2, …, K is the number of the first sample image blocks in the Z-th class data set;
(4) in [0, 1]]Internally randomly generating a random number, and when the random number belongs to the interval
Figure BDA0003203683630000062
Then, the (r + 1) th first sample image block in the Z-th class data set is selected as a second clustering center point cZ2Wherein r is 1, 2, …, K-1;
(5) repeating the step (4) until N clustering centers are selected from the Z category data set: cZ1，cZ2，…，cZN；
(6) Calculating the Hamming distance from each first sample image block in the Z-th category data set to N clustering centers, and dividing each first sample image block in the Z-th category data set into the N clustering centers according to the principle of closeness to obtain N clusters;
(7) and recalculating the clustering centers of the N clusters, wherein the calculation formula is as follows:
Figure BDA0003203683630000063
wherein ,
Figure BDA0003203683630000064
represents the cluster center of the v-th cluster at the p +1 th iterative clustering,
Figure BDA0003203683630000065
represents the v-th cluster at the p-th iterative clustering,
Figure BDA0003203683630000066
representing the number of samples of the v-th cluster at the p-th iterative clustering, v being 1, 2, …, N;
(8) repeating the steps (6) and (7) until the cluster center of the two continuous iterative clusters in each cluster meets the requirement
Figure BDA0003203683630000071
And acquiring N clusters of the final cluster, wherein one cluster represents one subcategory data set.
Further, the method for performing parameter training on the neural network model by using the permutation loss function and the weighting loss function comprises the following steps:
extracting a first probability p of the sample image from the first lesion class prediction probability, the second lesion class prediction probability and the skin lesion class prediction probability according to the real skin lesion class of the sample image in the skin lesion data set1Second probability p2And a third probability p3；
Network parameters of the fixed lesion localization structure according to the first probability p1Second probability p2And a third probability p3And optimizing other network parameters in the neural network model by using a weighted loss function, wherein the expression of the weighted loss function is as follows:
Figure BDA0003203683630000072
where LF denotes the weighted loss function and H is the number of sample images in the skin lesion dataset, ρZRepresenting the number of first sample image blocks in the Z-th category data set, γ being an artificially set hyper-parameter, Z being 1, 2, …, d, d being the number of category data sets in the skin lesion data set;
other network parameters in the fixed neural network model, using the permutation loss function Lrank(p1，p2) Optimization of network parameters of lesion localization structures, Lrank(p1，p2) The expression of (a) is as follows:
Lrank(p1，p2)＝max(0，p1-p2+margin) (7)
wherein margin is a preset decimal.
Further, the first lesion class prediction probability is calculated as follows:
P1＝softmax(U2F1+b2) (8)
wherein ,U2To assist the parameter matrix of the output layer, F1Representing a first feature vector, b2Is a bias term for the auxiliary output layer.
The following advantages can be obtained by adopting the technical means:
the invention provides a skin lesion image classification method, which is characterized in that a neural network model based on a multi-scale double-layer attention mechanism is utilized to process a skin lesion image to be classified, and the skin lesion class probability of the skin lesion image to be classified is predicted, so that the skin lesion class of the skin lesion image to be classified is obtained. Before the classification is started, the method performs center cutting on the skin lesion image to be classified, enlarges the skin lesion area in the image and unifies the image size, thereby being beneficial to subsequent image processing and feature recognition. In the process of image processing by using the neural network module, the method of the invention utilizes the feature extraction structure consisting of the attention residual learning block and the lesion positioning structure based on the attention mechanism to ensure that the neural network is highly concentrated in the skin lesion area during feature extraction, thereby greatly reducing the influence on the prediction of the neural network due to the undersize skin lesion area in the skin lesion image to be classified and improving the classification accuracy of the skin lesion image.
In the model training process, the method not only unifies the size of the sample images in the skin lesion data set, but also performs class decomposition on various classes in the skin lesion data set, divides the classes into a plurality of subclass data sets, forms new data distribution, achieves the effect of extracting fine-grained information hidden in each class of the images, and improves the model training effect; in addition, the invention also solves the problem of data imbalance possibly caused by class decomposition by using a weighting loss function, and greatly improves the sensitivity and specificity of the skin lesion image classification method.
The method of the invention does not depend on manual operation, improves the efficiency of classifying the skin lesion images, and can rapidly, objectively and accurately judge the skin lesion categories of the skin lesion images.
Drawings
FIG. 1 is a flowchart illustrating steps of a method for classifying skin lesion images according to the present invention;
FIG. 2 is a network structure diagram of a neural network model according to an embodiment of the present invention;
FIG. 3 is a diagram of an attention residual error unit learning structure ARL according to an embodiment of the present invention;
FIG. 4 is a flow chart of a training process of a neural network model according to an embodiment of the present invention;
FIG. 5 is an exploded flow chart of a skin lesion data set according to an embodiment of the present invention.
Detailed Description
The technical scheme of the invention is further explained by combining the accompanying drawings as follows:
the invention provides a skin lesion image classification method, as shown in fig. 1, which specifically comprises the following steps:
and A, performing center cutting on the skin lesion image to be classified to obtain a first image block, wherein the size of the first image block is n x 3, n is the side length of the first image block, and 3 represents three channels of the RGB image.
And step B, extracting the features of the first image block by adopting an upper branch network of the trained neural network model to obtain a first feature vector, wherein the neural network model adopts a neural network model based on a multi-scale double-layer attention mechanism.
And C, positioning the lesion area by using the lesion positioning structure in the upper branch network according to the first feature vector.
And D, cutting the first image block according to the positioned lesion area to obtain a second image block.
And E, extracting the features of the second image block by using the trained lower branch network of the neural network model to obtain a second feature vector.
And F, performing feature fusion on the first feature vector and the second feature vector by using a feature fusion structure in the lower branch network to obtain a fusion vector.
And G, processing the fusion vector by utilizing a softmax activation function in an output layer of the lower branch network, and acquiring the skin lesion class prediction probability.
And H, classifying the skin lesion images to be classified according to the skin lesion category prediction probability.
In the embodiment of the present invention, a Network structure of a neural Network model is shown in fig. 2, the neural Network model may be divided into an upper branch Network and a lower branch Network, the upper branch Network mainly includes a feature extraction structure, an auxiliary output layer, a Lesion Location Network (LLN) and a clipping scaling structure, and the lower branch Network mainly includes a feature extraction structure, an auxiliary output layer, a feature fusion structure and an output layer; wherein, the feature extraction structure comprises a convolution layer, a plurality of attention Residual error unit learning structures ARL (attention Residual learning) and a global Average pooling layer GAP (global Average pooling layer); the auxiliary output layer is a full connecting layer; the pathological change positioning structure comprises a hidden layer and an output layer, wherein the hidden layer and the output layer are all full-connection layers.
In the embodiment of the present invention, the specific operation of step B is as follows:
step B01, inputting the first image block with dimension n X3 into the convolution layer of the feature extraction structure of the upper branch network, and obtaining the intermediate vector X through the Relu nonlinear activation function1Intermediate vector X1Has a dimension of n1*n1*D1，n1Representing the intermediate vector X1Side length of (D)1The number of convolution kernels representing the convolution layer of the feature extraction structure.
Step B02, connecting a plurality of attention residual error unit learning structures ARL in the feature extraction structure in sequence, and connecting the intermediate vector X1After the ARL is input, a plurality of attention residual error unit learning structures ARL are utilized to learn the intermediate vector X1The output vector y can be obtained by performing processing such as convolution, normalization, and downsampling.
As shown in fig. 3, the attention residual error unit learning structure ARL includes 1 × 1 convolution layer, batch layer, nonlinear active layer, 3 × 3 convolution layer, batch layer, nonlinear active layer, 1 × 1 convolution layer, batch layer, nonlinear active layer, and down-sampling layer (1 × 1 convolution layer).
Taking the first ARL as an example, the intermediate vector X1Inputting the vector Q into the first ARL, firstly performing convolution processing on a plurality of convolution layers, outputting the vector Q after passing through a third layer 1 x 1 convolution layer in the first ARL, wherein the dimension of the vector Q is n '. times.n '. times.D ', n ' represents the side length of the vector Q, and D ' represents the number of convolution kernels of the third layer 1 x 1; then the first ARL carries out normalization processing on the vector Q to obtain a vector matrix M [ Q ]](ii) a Intermediate vector X1Passing through a downsampling layer (1X 1 convolutional layer), downsampling X1Is the same as the vector Q, and down-sampled X1And M [ Q ]]Pixel level multiplication is carried out; finally, X is1Q and M [ Q ]]·X1Performing pixel-level addition to obtain an output vector y of the first ARL1＝X1+Q+μ·M[Q]·X1Where μ is a parameter for automatic neural network learning, y1The dimension of (c) is still n '. cndot.Dd'.
Intermediate vector X1After the last ARL, an output vector y with dimension n is obtained2*n2*D，n2The side length of the output vector y is indicated and D indicates the number of the third layer 1 x 1 convolution kernels in the final ARL structure.
Step B03, utilizing the global average pooling layer GAP to process the output vector y to obtain a first feature vector F of the first image block1，F1Dimension of (D) is 1 x D.
In the embodiment of the present invention, the specific operation of step C is as follows:
step C01, the first feature vector F1Inputting the state of the hidden layer into a hidden layer of a lesion positioning structure, and obtaining a hidden layer state g by utilizing a Relu nonlinear activation function:
g＝Relu(U3F1+b3) (9)
wherein ,U3Parameter matrix as a hidden layer, b3Biasing terms for hidden layers.
Step C02, according to the hidden layer state g, obtaining the coordinates of the lesion area of the first image block by using a sigmoid nonlinear activation function in the output layer of the lesion positioning structure, wherein the lesion area adopts a circular area in the method, and the expression of the coordinates of the lesion area is as follows:
[tx，ty，tl]＝n*sigmoid(U4g+b4) (10)
wherein ,txAbscissa representing center point of lesion region, tyOrdinate, t, representing the center point of the lesionlRepresenting the radius of the lesion, n being the side length of the first image block, U4Is a parameter matrix of the output layer, b4Is the bias term for the output layer.
In the formula (10), the values obtained by the sigmoid nonlinear activation function are all between 0 and 1, and in order to obtain a real coordinate value, the values obtained by the sigmoid nonlinear activation function need to be amplified, so that sigmoid (U) needs to be controlled4g+b4) Multiplied by n.
In the embodiment of the present invention, the specific operation of step D is as follows:
step D01, the clipping region in the present invention is rectangular, and 4 vertex coordinates of the clipping region in the first image block can be obtained according to the coordinates of the lesion region: the coordinate of the upper left corner of the cutting area is (t)x(tl)，ty(tl)) The coordinate of the lower left corner of the clipping region is (t)x(tl)，ty(br)) The coordinate of the upper right corner of the cutting area is (t)x(br)，ty(tl)) The coordinate of the lower right corner of the cutting area is (t)x(br)，ty(br)), wherein ,tx(tl)＝tx-tl，ty(tl)＝ty-tl，tx(br)＝tx+tl，ty(br)＝ty+tl。
D02, cutting the first image block according to the vertex coordinates of the cutting area to obtain a cutting image X corresponding to the first image blockatt。
Step D03, scaling the cropped image according to the side length of the first image blockProcessing to obtain a second image block XampSecond image block XampFor a three-dimensional vector of n x 3, the scaling process is expressed as follows:
Figure BDA0003203683630000131
wherein ,
Figure BDA0003203683630000132
representing the pixel values in the ith row and jth column in the second image block,
Figure BDA0003203683630000133
denotes a pixel value on the ith row and the w column in the clipped image, and α ═ h- [ i/λ]，β＝w-[j/λ]，
Figure BDA0003203683630000134
[·]For the rounding function, {. cndot.x(tl)，tx(br)]，w∈[ty(tl)，ty(br)]，i，j∈{1，2，…，n}。
In step E of the invention, a second image block X of dimension n X3 is appliedampInputting the feature extraction structure of the lower branch network, and utilizing the convolution layer, a plurality of attention residual error unit learning structures ARL and a global average pooling layer GAP in the feature extraction structure to perform X pairampExtracting the features to obtain a second feature vector F2，F2Dimension of (D) is 1 x D. The specific operation of step E is identical to step B.
In step F, a first feature vector F is added1And a second feature vector F2Splicing in a concat mode, and obtaining a fusion vector F after splicing, wherein F is ═ F1；F2]Is a 1 x 2D three-dimensional vector.
In step G, inputting the fusion vector F into an output layer of the lower branch network, processing the fusion vector F through a softmax activation function, and outputting the skin lesion class prediction probability P3，P3Is 1 × s, s is the number of subclasses in the neural network model, at P3Comprises sNumerical values, each numerical value representing the probability that the skin lesion image to be classified belongs to 1 subclass.
In step H, the probability P is predicted according to the skin lesion class3And classifying the skin lesion images to be classified to obtain the skin lesion categories of the skin lesion images to be classified. Specifically, the skin lesion class prediction probability P is set3The prediction probabilities of all the subclasses belonging to the same class are added to obtain the real probability of the skin lesion image to be classified corresponding to each class, and the class corresponding to the maximum value is taken as the final skin lesion class.
It is assumed that the skin lesion image to be classified may belong to two categories: class a and class B, the neural network model decomposes class B into three subclasses, class B1, class B2, class B3, after the skin lesion image to be classified is input into the neural network model, the model outputs a 1 × 4 vector [0.4, 0.1, 0.4, 0.1], four numbers in the vector are prediction probabilities of class a, class B1, class B2, class B3, since class B is decomposed, the last three probability values in the vector need to be added to obtain the probability of class B, i.e., 0.1+0.4+0.1 ═ 0.6, to obtain the true probability of the skin lesion image to be classified [0.4, 0.6], 0.4 and 0.6 are prediction probabilities of class a and class B, and since the prediction probability of class B is greater than the prediction probability of class a, the skin lesion image to be classified is considered as class B.
In the method of the present invention, a neural network model needs to be trained, as shown in fig. 4, the training method includes the following steps:
step 1, obtaining a skin lesion data set with d types of samples, wherein each skin lesion type in the skin lesion data set comprises a plurality of sample images.
Because the sample images are different in size and the lesion area in the sample images is relatively small, the method carries out data preprocessing on the skin lesion data set, and comprises the following specific operations:
step 101, performing center clipping on all sample images in the skin lesion data set according to a preset size to obtain a corresponding first sample image block, wherein the preset size is n × 3, n is the side length of the first sample image block, and 3 represents three channels of an RGB image.
And 102, utilizing all the first sample image blocks to form a preprocessed skin lesion data set.
And 2, in order to conveniently extract the hidden fine-grained information in each category of the image, carrying out category decomposition processing on the preprocessed skin lesion data set, dividing the preprocessed skin lesion data set into a plurality of category data sets according to the category of the skin lesion, and decomposing each category data set into a plurality of sub-category data sets according to the image correlation.
As shown in FIG. 5, the specific operation of decomposing the Z-th category dataset into a plurality of sub-category datasets is as follows:
step 201, performing gray processing on each first sample image block in the Z-th category data set to obtain a gray image corresponding to the first sample image block, and taking each pixel point in the gray image as a unit, so that the gray image can be represented as a 2-dimensional data matrix an×n：
Figure BDA0003203683630000151
Where nxn denotes the number of rows x columns, a, of the matrix AijRepresentation matrix An×nAnd (3) pixel point values of ith row and j column, wherein Z is 1, 2, … and d.
Step 201, randomly selecting a first sample image block from the Z-th category data set as an initial clustering center cZ1。
Step 203, calculating each first sample image block to the initial clustering center c in the Z-th class data set by using the gray scale image corresponding to the first sample image blockZ1Distance D (z)k)2The calculation formula is as follows:
Figure BDA0003203683630000152
wherein ,zkFirst sample graph representing the kth in the Z category datasetImage block, D (z)k) Denotes zkTo the initial cluster center cZ1A distance ofi，jRepresenting the initial cluster center cZ1Pixel point value of ith row and j column, bi，jDenotes zkThe pixel point value K in the ith row and j column is 1, 2, …, K is the number of the first sample image blocks in the Z-th category data set.
From all first sample image blocks to cZ1Calculates the probability that each first sample image block is selected as the next cluster center, and the calculation formula is as follows:
Figure BDA0003203683630000161
wherein ,P(zk) Denotes zkProbability of being selected as the next cluster center.
Step 204, at [0, 1]]Internally randomly generating a random number, and when the random number belongs to the interval
Figure BDA0003203683630000162
Then, the (r + 1) th first sample image block in the Z-th class data set is selected as a second clustering center point cZ2Wherein r is 1, 2, …, K-1.
Step 205, repeating step 204 until N clustering centers are selected from the Z category data set: c. CZ1，cZ2，...，cZNAnd N is the preset number of cluster centers, and the selected N value ensures that the number of the first sample image blocks in each sub-class after decomposition is approximately the same.
Step 206, calculating the hamming distance | Z from each first sample image block to the N clustering centers in the Z-th category data set in sequencek-Czv||1And v is 1, 2, …, N, dividing each first sample image block in the Z-th class data set into N clusters according to the principle of proximity, and obtaining N clusters, which are respectively marked as N clusters
Figure BDA0003203683630000163
p denotes the p-th iterative clustering.
Step 207, according to the N clusters in step 206, recalculating the cluster centers of the N clusters, wherein the calculation formula is as follows:
Figure BDA0003203683630000164
wherein ,
Figure BDA0003203683630000165
represents the cluster center of the v-th cluster at the p +1 th iterative clustering,
Figure BDA0003203683630000166
represents the v-th cluster at the p-th iterative clustering,
Figure BDA0003203683630000167
representing the number of samples of the v-th cluster at the p-th iterative clustering.
Step 208, repeating steps 206 and 207, continuously updating the first sample image block and the clustering center in the cluster until the distance between the new center position and the old center position of the cluster meets the requirement, namely the clustering center of the two continuous iterative clustering in each cluster meets the requirement
Figure BDA0003203683630000168
At this time, the classification result is not changed any more, iteration is finished, and N clusters of the final cluster are obtained, wherein one cluster represents one subclass data set.
Performing clustering decomposition on each category data set in the skin lesion data set according to the steps, extracting corresponding first sample image blocks, and storing the first sample image blocks in the same folder; the folder of each subclass data set is named in a class name _ N mode, which indicates N corresponding subclasses in the original class, and accordingly each class label of a new data set formed by the subclass data sets is the name of the corresponding folder, and the new data set contains s subclass data sets after decomposition.
Step 3, performing feature extraction on each first sample image block in each sub-data set by using a feature extraction structure of the upper branch networkExtracting and performing global average pooling to obtain a first feature vector F1The concrete operation is the same as that of step B, F1And D is the number of the last convolution kernels in the feature extraction structure.
The auxiliary output layer of the upper branch network receiving F1Outputting the prediction probability P of the first lesion class through the softmax activation function1：
P1＝softmax(U2F1+b2) (16)
wherein ,U2To assist the parameter matrix of the output layer, F1Representing a first feature vector, b2To assist the bias term of the output layer, P1Dimension 1 × s.
And 4, positioning a lesion area of each first sample image block according to the first feature vector by using a lesion positioning structure of the upper branch network, wherein the specific operation is consistent with that in the step C.
And 5, cutting each first sample image block according to the positioned lesion area to obtain a second sample image block, wherein the specific operation is consistent with that in the step D.
Step 6, performing feature extraction and global average pooling on the skin disease picture of each second sample image block by using a feature extraction structure of the lower branch network to obtain a second feature vector F2The specific operation is the same as step E.
Receiving a second feature vector F using an auxiliary output layer2Calculating a second lesion class prediction probability P using the softmax activation function2，P2Dimension (d) is 1 × s.
Step 7, performing feature fusion on the first feature vector and the second feature vector by using the feature fusion structure to obtain a fusion vector, specifically, F1，F2Splicing in a concat mode, and fusing a vector F ═ F1；F2]。
Step 8, processing the fusion vector by utilizing a softmax activation function in the output layer to obtain the skin lesion class prediction probability P corresponding to each sample image3，P3Dimension (d) is 1 × s. Comparison P3And selecting the category corresponding to the maximum value as the classification result of the sample image according to the prediction probability values corresponding to different categories.
Step 9, based on the first lesion category prediction probability, the second lesion category prediction probability and the skin lesion category prediction probability, performing parameter training on the neural network model by using the permutation loss function and the weighting loss function, and obtaining the trained neural network model through iterative convergence, wherein the method specifically comprises the following steps:
step 901, obtaining a first probability p of the sample image from a first lesion class prediction probability, a second lesion class prediction probability and a skin lesion class prediction probability according to the real skin lesion class of each sample image in the skin lesion class data set1Second probability p2And a third probability p3。
P1、P2 and P3All dimensions of (1 x s) include the prediction probability values corresponding to the s sub-classes. Respectively adding P1、P2 and P3Adding the prediction probability values of different subclasses in the same class to obtain the prediction probability values corresponding to different classes; from P, on the basis of the correct label of the sample image (the expert judged category of the real skin lesion of this image)1、P2 and P3Finding out the predicted probability value corresponding to the real skin lesion category of the sample image block, and recording as the first probability p corresponding to the correct label1Second probability p2And a third probability p3。
Step 902, fix the network parameters of the lesion localization structure according to the first probability p1Second probability p2And a third probability p3And optimizing other network parameters in the neural network model by using a weighted loss function, wherein the expression of the weighted loss function is as follows:
Figure BDA0003203683630000191
where LF denotes the weighted loss function and H is the number of sample images in the skin lesion dataset, ρZTo representThe number of the first sample image blocks in the Z-th category data set, gamma is a manually set hyper-parameter, and Z is 1, 2, …, d, d is the number of category data sets in the skin lesion data set;
step 903, fixing other network parameters in the neural network model by using the permutation loss function Lrank(p1，p2) Optimization of network parameters of lesion localization structures, Lrank(p1，p2) The expression of (a) is as follows:
Lrank(p1，p2)＝max(0，p1-p2+margin) (18)
wherein margin is a preset decimal close to 0.
In the method, the deep neural network is applied to the field of classification of the skin lesion images, the skin lesion types are quickly and accurately judged through the trained neural network model, in the specific implementation process, the method enlarges the skin lesion areas through center cutting and type decomposition, unifies the sizes of the skin lesion images, and achieves the effect of extracting fine-grained information hidden in each type of the images; the invention utilizes the feature extraction structure formed by the attention residual learning blocks and the lesion positioning structure based on the attention mechanism to lead the network to be highly concentrated in the skin lesion area during feature extraction, thereby greatly reducing the influence on neural network detection caused by the undersize skin lesion area in the skin lesion image to be classified, solving the problem of data imbalance possibly caused by class decomposition by utilizing the weighting loss function, and greatly improving the sensitivity and specificity of the skin lesion image classification method.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.

Claims (9)

1. A skin lesion image classification method is characterized by comprising the following steps:
performing center cutting on a skin lesion image to be classified to obtain a first image block;
extracting features of the first image block by adopting an upper branch network of a trained neural network model to obtain a first feature vector, wherein the neural network model adopts a neural network model based on a multi-scale double-layer attention mechanism;
carrying out lesion area positioning according to the first feature vector by using a lesion positioning structure in the upper branch network;
cutting the first image block according to the located lesion area to obtain a second image block;
extracting the features of the second image block by using a lower branch network of the trained neural network model to obtain a second feature vector;
performing feature fusion on the first feature vector and the second feature vector by using a feature fusion structure in the lower branch network to obtain a fusion vector;
processing the fusion vector by utilizing a softmax activation function in an output layer of the lower branch network to obtain the skin lesion category prediction probability;
and classifying the skin lesion images to be classified according to the skin lesion category prediction probability.
2. The method for classifying skin lesion images according to claim 1, wherein an upper branch network of the neural network model comprises a feature extraction structure, an auxiliary output layer, a lesion localization structure and a clipping scaling structure, and a lower branch network of the neural network model comprises a feature extraction structure, an auxiliary output layer, a feature fusion structure and an output layer, wherein the feature extraction structure comprises a convolution layer and a plurality of attention residual error unit Learning Structures (ARLs), and the lesion localization structure comprises a hidden layer and an output layer.
3. The method for classifying skin lesion images according to claim 1 or 2, wherein the method for obtaining the first feature vector comprises:
inputting the first image block into a convolution layer of a feature extraction structure of an upper branch network, passing throughRelu nonlinear activation function obtains intermediate vector X1；
Learning structure ARL versus intermediate vector X using several attention residual units1Performing convolution, normalization and downsampling processing to obtain an output vector y;
processing the output vector y by using the global average pooling layer to obtain a first feature vector F corresponding to the first image block1。
4. The method for classifying skin lesion images according to claim 1, wherein the method for locating lesion regions according to the first feature vector by using the lesion locating structure in the upper branch network comprises:
the first feature vector F1Inputting the state of the hidden layer into a hidden layer of a lesion positioning structure, and obtaining a hidden layer state g by utilizing a Relu nonlinear activation function:
g＝Relu(U3F1+b3)
wherein ,U3Parameter matrix as a hidden layer, b3A bias term that is a hidden layer;
according to the hidden layer state g, acquiring the coordinates of a lesion area of the first image block by using a sigmoid nonlinear activation function in an output layer of the lesion positioning structure, wherein the expression of the coordinates of the lesion area is as follows:
[tx，ty，tl]＝n*sigmoid(U4g+b4)
wherein ,txAbscissa representing center point of lesion region, tyOrdinate, t, representing the center point of the lesionlRepresenting the radius of the lesion, n being the side length of the first image block, U4Is a parameter matrix of the output layer, b4Is the bias term for the output layer.
5. The method for classifying skin lesion images according to claim 4, wherein the method for obtaining the second image block comprises:
obtaining the vertex coordinates of the clipping area in the first image block according to the coordinates of the lesion area: the coordinate of the upper left corner of the cutting area is (t)x(tl)，ty(tl)) The coordinate of the lower left corner of the clipping region is (t)x(tl)，ty(br)) The coordinate of the upper right corner of the cutting area is (t)x(br)，ty(tl)) The coordinate of the lower right corner of the cutting area is (t)x(br)，ty(br)), wherein ,tx(tl)＝tx-tl，ty(tl)＝ty-tl，tx(br)＝tx+tl，ty(br)＝ty+tl；
Cutting the first image block according to the vertex coordinates of the cutting area to obtain a cutting image corresponding to the first image block;
scaling the cut image according to the side length of the first image block to obtain a second image block:
Figure FDA0003203683620000031
wherein ,
Figure FDA0003203683620000032
representing the pixel values in the ith row and jth column in the second image block,
Figure FDA0003203683620000033
denotes a pixel value on the ith row and the w column in the clipped image, and α ═ h- [ i/λ]，β＝w-[j/λ]，
Figure FDA0003203683620000034
[·]For the rounding function, {. cndot.x(tl)，tx(br)]，w∈[ty(tl)，ty(br)]，i，j∈{1，2，…，n}。
6. The method for classifying skin lesion images according to claim 1 or 2, wherein the training method of the neural network model comprises the following steps:
obtaining a skin lesion dataset comprising a plurality of sample images under a plurality of skin lesion categories;
performing center cutting on each sample image in the skin lesion data set to obtain a first sample image block, and forming a preprocessed skin lesion data set by using all the first sample image blocks;
dividing the preprocessed skin lesion data set into a plurality of category data sets according to skin lesion categories, and decomposing each category data set into a plurality of sub-category data sets according to image correlation;
performing feature extraction on each first sample image block in each sub-data set by using a feature extraction structure of an upper branch network to obtain a first feature vector, and obtaining a first lesion category prediction probability by using an auxiliary output layer;
positioning a lesion area of each first sample image block according to the first feature vector by using a lesion positioning structure of the upper branch network;
cutting each first sample image block according to the located lesion area to obtain a second sample image block;
performing feature extraction on each second sample image block by using a feature extraction structure of the lower branch network to obtain a second feature vector, and obtaining a second lesion category prediction probability by using an auxiliary output layer;
performing feature fusion on the first feature vector and the second feature vector by using a feature fusion structure to obtain a fusion vector;
processing the fusion vector by utilizing a softmax activation function in an output layer to obtain the skin lesion class prediction probability;
and performing parameter training on the neural network model by using the permutation loss function and the weighting loss function based on the first lesion category prediction probability, the second lesion category prediction probability and the skin lesion category prediction probability, and obtaining the trained neural network model through iterative convergence.
7. The method of claim 6, wherein if the pre-processed skin lesion data set has d category data sets, Z is 1, 2, …, d, then the method of decomposing the Z-th category data set into a plurality of sub-category data sets according to the image correlation comprises:
(1) carrying out gray processing on each first sample image block in the Z-th category data set to obtain a gray image corresponding to the first sample image block;
(2) randomly selecting a first sample image block from the Z category data set as an initial clustering center cZ1；
(3) Calculating each first sample image block to an initial clustering center c in the Z category data set according to the gray level image corresponding to the first sample image blockZ1And calculating the probability of each first sample image block being selected as the next cluster center according to the distance, the calculation formula is as follows:
Figure FDA0003203683620000051
wherein ,zkRepresents the kth first sample image block, P (Z), in the Z category datasetk) Denotes zkProbability of being selected as next cluster center, D (z)k) Denotes zkTo the initial cluster center cZ1K is 1, 2, …, K is the number of the first sample image blocks in the Z-th class data set;
(4) in [0, 1]]Internally randomly generating a random number, and when the random number belongs to the interval
Figure FDA0003203683620000052
Then, the (r + 1) th first sample image block in the Z-th class data set is selected as a second clustering center point cZ2Wherein r is 1, 2, …, K-1;
(5) repeating the step (4) until N clustering centers are selected from the Z category data set: c. CZ1，cZ2，...，cZN；
(6) Calculating the Hamming distance from each first sample image block in the Z-th category data set to N clustering centers, and dividing each first sample image block in the Z-th category data set into the N clustering centers according to the principle of closeness to obtain N clusters;
(7) and recalculating the clustering centers of the N clusters, wherein the calculation formula is as follows:
Figure FDA0003203683620000061
wherein ,
Figure FDA0003203683620000062
represents the cluster center of the v-th cluster at the p +1 th iterative clustering,
Figure FDA0003203683620000063
represents the v-th cluster at the p-th iterative clustering,
Figure FDA0003203683620000064
representing the number of samples of the v-th cluster at the p-th iterative clustering, v being 1, 2, …, N;
(8) repeating the steps (6) and (7) until the cluster center of the two continuous iterative clusters in each cluster meets the requirement
Figure FDA0003203683620000065
And acquiring N clusters of the final cluster, wherein one cluster represents one subcategory data set.
8. The method for classifying skin lesion images according to claim 6, wherein the method for performing parameter training on the neural network model by using the permutation loss function and the weighting loss function comprises:
extracting a first probability p of the sample image from the first lesion class prediction probability, the second lesion class prediction probability and the skin lesion class prediction probability according to the real skin lesion class of the sample image in the skin lesion data set1Second probability p2And a third probability p3；
Network parameters of the fixed lesion localization structure according toFirst probability p1Second probability p2And a third probability p3And optimizing other network parameters in the neural network model by using a weighted loss function, wherein the expression of the weighted loss function is as follows:
Figure FDA0003203683620000066
where LF denotes the weighted loss function and H is the number of sample images in the skin lesion dataset, ρZRepresenting the number of first sample image blocks in the Z-th category data set, γ being an artificially set hyper-parameter, Z being 1, 2, …, d, d being the number of category data sets in the skin lesion data set;
other network parameters in the fixed neural network model, using the permutation loss function Lrank(p1，p2) Optimization of network parameters of lesion localization structures, Lrank(p1，p2) The expression of (a) is as follows:
Lrank(p1，p2)＝max(0，p1-p2+margin)
wherein margin is a preset decimal.
9. The method for classifying skin lesion images according to claim 6 or 8, wherein the prediction probability of the first lesion class is calculated as follows:
P1＝softmax(U2F1+b2)
wherein ,U2To assist the parameter matrix of the output layer, F1Representing a first feature vector, b2Is a bias term for the auxiliary output layer.
CN202110911205.5A 2021-08-10 2021-08-10 Skin lesion image classification method Active CN113705630B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110911205.5A CN113705630B (en) 2021-08-10 2021-08-10 Skin lesion image classification method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110911205.5A CN113705630B (en) 2021-08-10 2021-08-10 Skin lesion image classification method

Publications (2)

Publication Number Publication Date
CN113705630A true CN113705630A (en) 2021-11-26
CN113705630B CN113705630B (en) 2023-10-13

Family

ID=78651972

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110911205.5A Active CN113705630B (en) 2021-08-10 2021-08-10 Skin lesion image classification method

Country Status (1)

Country Link
CN (1) CN113705630B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116050503A (en) * 2023-02-15 2023-05-02 哈尔滨工业大学 Generalized neural network forward training method

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010133938A1 (en) * 2009-05-22 2010-11-25 Nokia Corporation Method and apparatus for performing feature extraction using local primitive code
CN111178432A (en) * 2019-12-30 2020-05-19 武汉科技大学 Weak supervision fine-grained image classification method of multi-branch neural network model
CN111444960A (en) * 2020-03-26 2020-07-24 上海交通大学 Skin disease image classification system based on multi-mode data input
WO2021082480A1 (en) * 2019-10-29 2021-05-06 华为技术有限公司 Image classification method and related device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010133938A1 (en) * 2009-05-22 2010-11-25 Nokia Corporation Method and apparatus for performing feature extraction using local primitive code
WO2021082480A1 (en) * 2019-10-29 2021-05-06 华为技术有限公司 Image classification method and related device
CN111178432A (en) * 2019-12-30 2020-05-19 武汉科技大学 Weak supervision fine-grained image classification method of multi-branch neural network model
CN111444960A (en) * 2020-03-26 2020-07-24 上海交通大学 Skin disease image classification system based on multi-mode data input

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116050503A (en) * 2023-02-15 2023-05-02 哈尔滨工业大学 Generalized neural network forward training method
CN116050503B (en) * 2023-02-15 2023-11-10 哈尔滨工业大学 Generalized neural network forward training method

Also Published As

Publication number Publication date
CN113705630B (en) 2023-10-13

Similar Documents

Publication Publication Date Title
US10984289B2 (en) License plate recognition method, device thereof, and user equipment
Yuan et al. Factorization-based texture segmentation
CN110532920B (en) Face recognition method for small-quantity data set based on FaceNet method
WO2020248581A1 (en) Graph data identification method and apparatus, computer device, and storage medium
US20220157048A1 (en) Method and System for Classification and Visualisation of 3D Images
CN112288011B (en) Image matching method based on self-attention deep neural network
CN112801169B (en) Camouflage target detection method, system, device and storage medium based on improved YOLO algorithm
CN111738091A (en) Posture estimation and human body analysis system based on multi-task deep learning
CN112750148B (en) Multi-scale target perception tracking method based on twin network
CN111783779B (en) Image processing method, apparatus and computer readable storage medium
CN111476806B (en) Image processing method, image processing device, computer equipment and storage medium
CN112861970B (en) Fine-grained image classification method based on feature fusion
CN113361495A (en) Face image similarity calculation method, device, equipment and storage medium
CN110490262B (en) Image processing model generation method, image processing device and electronic equipment
CN111967464A (en) Weak supervision target positioning method based on deep learning
CN111860823A (en) Neural network training method, neural network training device, neural network image processing method, neural network image processing device, neural network image processing equipment and storage medium
CN113705630A (en) Skin lesion image classification method
ul Haq et al. Neural network based adaboosting approach for hyperspectral data classification
CN114119669A (en) Image matching target tracking method and system based on Shuffle attention
CN116563649B (en) Tensor mapping network-based hyperspectral image lightweight classification method and device
CN116758419A (en) Multi-scale target detection method, device and equipment for remote sensing image
CN115719414A (en) Target detection and accurate positioning method based on arbitrary quadrilateral regression
Maithil et al. Semantic Segmentation of Urban Area Satellite Imagery Using DensePlusU-Net
JP7441656B2 (en) Recognition device, learning device, recognition system, recognition method, and recognition program
Grana et al. Further results on AMM for endmember induction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant