CN110796115A - Image detection method and device, electronic equipment and readable storage medium - Google Patents
Image detection method and device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN110796115A CN110796115A CN201911087220.1A CN201911087220A CN110796115A CN 110796115 A CN110796115 A CN 110796115A CN 201911087220 A CN201911087220 A CN 201911087220A CN 110796115 A CN110796115 A CN 110796115A
- Authority
- CN
- China
- Prior art keywords
- blackhead
- size
- image
- detection
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The embodiment of the application provides an image detection method, an image detection device, an electronic device and a readable storage medium, and the method and the device can be used for carrying out size conversion on a first image of a blackhead to be detected as far as possible by carrying out different-size reduction and improvement on a residual error network structure of a blackhead detection model, so as to obtain corresponding second images with different sizes, greatly improve the blackhead detection speed and reduce the performance loss on the premise of not reducing the detection precision. And then, respectively outputting the blackhead heat map corresponding to each size by the residual error network structure corresponding to each size, and then carrying out weighting processing to obtain a blackhead detection result. Therefore, accuracy of blackhead detection can be improved under a complex background or a variable background, various skin environment changes can be effectively adapted, multilayer semantic information of a blackhead detection model is enriched by adopting a residual network structure with rich expression capacity and corresponding to different sizes, and detection accuracy is improved.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to an image detection method, an image detection apparatus, an electronic device, and a readable storage medium.
Background
The existing blackhead detection method generally adopts a traditional detection method, for example, gradient, brightness, edge, shape, color, size information and the like of an image are used as detection results, and blackheads are stripped from background information of the image according to a responding blackhead detection rule. The disadvantage of this approach is that a more sophisticated blackhead detection rule cannot be customized to adapt to all skin environment changes, and thus blackheads and other imperfections (e.g., spots, shadows, etc.) cannot be well distinguished.
In the conventional scheme, the blackhead detection scheme adopting deep learning has large performance loss and limited information expression capability, which may result in low detection accuracy of the trained blackhead detection model.
Disclosure of Invention
In view of the above, an object of the present application is to provide an image detection method, an image detection apparatus, an electronic device, and a readable storage medium, which can improve accuracy of black head detection in a complex background or a variable background, effectively adapt to changes of various skin environments, improve detection accuracy, greatly improve black head detection speed without reducing detection accuracy, and reduce performance loss.
According to an aspect of the present application, there is provided an image detection method applied to an electronic device, the method including:
acquiring a first image of a blackhead to be detected;
performing size conversion on the first image to obtain corresponding second images with different sizes;
inputting the second image of each size into a pre-trained blackhead detection model, and outputting a blackhead heat map corresponding to each size through a residual error network structure corresponding to each size in the blackhead detection model;
and carrying out weighting processing on the blackhead heat map corresponding to the second image of each size to obtain a blackhead detection result.
In one possible embodiment, the step of outputting the blackhead heatmap corresponding to each size through the residual network structure corresponding to each size in the blackhead detection model respectively comprises:
outputting a first feature map, a second feature map and a third feature map corresponding to the first size, the second size and the third size respectively through residual error network structures corresponding to the first size, the second size and the third size;
the first feature map is subjected to up-sampling and then is fused with the second feature map, and the fused fourth feature map is subjected to convolution operation to obtain a first blackhead heat map;
the fourth feature map is subjected to up-sampling and then is fused with the second feature map, and the fused fifth feature map is subjected to convolution operation to obtain a second blackhead heat map;
and upsampling the third feature map to obtain a third blackhead heat map.
In a possible implementation, the blackhead detection model is trained by the following steps:
acquiring a training sample set and a test sample set, wherein the training sample set comprises a plurality of nose area samples marked with blackhead areas, and the test sample set comprises a plurality of nose area samples not marked with the blackhead areas;
training a convolutional neural network model to be trained according to the training sample set to obtain a trained convolutional neural network model;
and testing the trained convolutional neural network model according to the test sample set, and obtaining a blackhead detection model obtained by training when a test result meets a preset condition.
In a possible implementation manner, before the step of training the convolutional neural network model to be trained according to the training sample set to obtain the trained convolutional neural network model, the method further includes:
and performing sample gain on the training sample set, adjusting the size of each nose region sample in the training sample set after the sample gain to be a set size, and training the convolutional neural network model to be trained according to the adjusted training sample set.
In a possible implementation manner, the step of training the convolutional neural network model to be trained according to the training sample set to obtain a trained convolutional neural network model includes:
for each nose area sample in the training sample set, after size conversion is carried out on the nose area sample, a blackhead heat map corresponding to the nose area sample of each size is output through a residual error network structure corresponding to each size in the convolutional neural network model;
calculating a cross entropy loss function value between the blackhead heat map corresponding to the nose area sample of each size and the blackhead area marked on the nose area sample;
performing back propagation training according to each calculated cross entropy loss function value, and calculating the gradient of the network parameter of the residual error network structure corresponding to each size;
and according to the calculated gradient, updating the network parameters of the residual error network structure corresponding to each size by adopting a random gradient descent method, and returning to the step of outputting the blackhead heat map corresponding to the nose area sample of each size through the residual error network structure corresponding to each size in the convolutional neural network model respectively until the convolutional neural network model meets the training termination condition, so as to obtain the trained convolutional neural network model.
In a possible implementation manner, the step of testing the trained convolutional neural network model according to the test sample set and obtaining a blackhead detection model obtained by training when a test result satisfies a preset condition includes:
inputting each nose area sample in the test sample set into the trained convolutional neural network model to obtain a blackhead detection result corresponding to each nose area sample;
counting the prediction accuracy of the blackhead detection result corresponding to each nose area sample;
when the prediction accuracy is larger than the set accuracy, judging that the test result meets a preset condition, and obtaining a blackhead detection model obtained through training;
and when the predicted accuracy is not greater than the set accuracy, recording the times of the current iteration test, performing weighted calculation according to the weights of the times of the current iteration test to obtain the weight of the next iteration test, and performing the iteration test on each nose area sample in the test sample set according to the weight of the next iteration test.
In a possible implementation manner, the step of performing weighting processing on the blackhead heat map corresponding to the second image of each size to obtain a blackhead detection result includes:
the method comprises the steps of up-sampling a blackhead heat map corresponding to a second image with the size lower than that of a first image into the size of the blackhead heat map of the first image, and carrying out weighted average calculation on the up-sampled blackhead heat map and the blackhead heat map of the second image with the same size as that of the first image to obtain a weighted blackhead heat map;
searching a predicted value of each unit area in the weighted blackhead heatmap, and determining the unit area as a blackhead area if the predicted value is greater than a set prediction threshold.
According to another aspect of the present application, there is provided an image detection apparatus applied to an electronic device, the apparatus including:
the acquisition module is used for acquiring a first image of a blackhead to be detected;
the size conversion module is used for carrying out size conversion on the first image to obtain corresponding second images with different sizes;
the heat map output module is used for inputting the second images of each size into a pre-trained blackhead detection model and outputting the blackhead heat map corresponding to each size through a residual error network structure corresponding to each size in the blackhead detection model;
and the blackhead detection module is used for weighting the blackhead heat map corresponding to the second image of each size to obtain a blackhead detection result.
According to another aspect of the present application, an electronic device is provided, which includes a machine-readable storage medium storing machine-executable instructions and a processor, and when the processor executes the machine-executable instructions, the electronic device implements the foregoing image detection method.
According to another aspect of the present application, there is provided a readable storage medium having stored therein machine executable instructions which, when executed, implement the aforementioned image detection method.
Based on any one of the above aspects, the residual error network structure of the blackhead detection model is subjected to different-size reduction and improvement, so that the size of the first image of the blackhead to be detected can be converted as much as possible to obtain the corresponding second images with different sizes, the blackhead detection speed is greatly increased on the premise of not reducing the detection precision, and the performance loss is reduced. And then, respectively outputting the blackhead heat map corresponding to each size by the residual error network structure corresponding to each size, and then carrying out weighting processing to obtain a blackhead detection result. Therefore, accuracy of blackhead detection can be improved under a complex background or a variable background, various skin environment changes can be effectively adapted, multilayer semantic information of a blackhead detection model is enriched by adopting a residual network structure with rich expression capacity and corresponding to different sizes, and detection accuracy is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
FIG. 1 is a schematic flow chart illustrating an image detection method provided by an embodiment of the present application;
FIG. 2 shows a flow diagram of the substeps of step S120 shown in FIG. 1;
FIG. 3 shows a flow diagram of the sub-steps of step S140 shown in FIG. 1;
FIG. 4 is a schematic diagram illustrating functional modules of an image detection apparatus provided in an embodiment of the present application;
fig. 5 shows a schematic block diagram of a structure of an electronic device for implementing the image detection method according to an embodiment of the present application.
Detailed Description
In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for illustrative and descriptive purposes only and are not used to limit the scope of protection of the present application. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this application illustrate operations implemented according to some of the embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.
In addition, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
Fig. 1 shows a flowchart of an image detection method provided in an embodiment of the present application, and it should be understood that, in other embodiments, the order of some steps in the image detection method of the present application may be interchanged according to actual needs, or some steps in the image detection method may be omitted or deleted. The detailed steps of the image detection method are described below.
Step S110, a first image of the blackhead to be detected is obtained.
In this embodiment, the first image may be the previously stored image captured in the past, or may also be an image captured with respect to the current scene, which is not limited in this embodiment.
Step S120, performing size conversion on the first image to obtain corresponding second images with different sizes.
Step S130, inputting the second image of each size into a pre-trained blackhead detection model, and outputting a blackhead heat map corresponding to each size through a residual error network structure corresponding to each size in the blackhead detection model.
Step S140, performing weighting processing on the blackhead heatmap corresponding to the second image of each size to obtain a blackhead detection result.
In this embodiment, for the blackhead detection model, for different sizes including the residual error structure network with corresponding size, the residual error network structure of the blackhead detection model is subjected to different size reduction and improvement, so that the size of the first image to be detected of the blackhead can be converted as much as possible to obtain corresponding second images with different sizes, and the blackhead detection speed is greatly increased and the performance loss is reduced on the premise of not reducing the detection precision. And then, respectively outputting the blackhead heat map corresponding to each size by the residual error network structure corresponding to each size, and then carrying out weighting processing to obtain a blackhead detection result. Therefore, accuracy of blackhead detection can be improved under a complex background or a variable background, various skin environment changes can be effectively adapted, multilayer semantic information of a blackhead detection model is enriched by adopting a residual network structure with rich expression capacity and corresponding to different sizes, and detection accuracy is improved.
In one possible implementation, for step S130, in the process of performing size conversion on the first image, the size of the transformation and the number of the second images may be selected according to actual requirements to adapt to an actual scene. For example, referring to fig. 2 in combination, step S130 may be implemented by the following sub-steps:
and a substep S131, outputting a first feature map, a second feature map and a third feature map corresponding to the first size, the second size and the third size respectively through a residual error network structure corresponding to the first size, the second size and the third size.
And a substep S132, performing upsampling on the first feature map, fusing the upsampled first feature map with the second feature map, and performing convolution operation on the fused fourth feature map to obtain a first blackhead heat map.
And a substep S133, performing upsampling on the fourth feature map, fusing the upsampled fourth feature map with the second feature map, and performing convolution operation on the fused fifth feature map to obtain a second blackhead heat map.
And a substep S134, performing upsampling on the third feature map to obtain a third blackhead heat map.
In this embodiment, the sizes of the foregoing conversion may include a first size, a second size, and a third size, which are sequentially increased, but generally do not exceed the size of the first image at maximum in order to reduce the amount of calculation. For example, taking the size of the first image as m as an example, the aforementioned converted sizes may include that the first size, the second size, and the third size may be 1/4m, 1/2m, respectively.
Correspondingly, the blackhead detection model may include residual network structures B1, B2, B3 corresponding to the first size, the second size and the third size, since the size of the input image corresponding to each of the residual network structures B1, B2 and B3 is 1/4, 1/2 and 1 of the first image, therefore, at the final stage of decreasing the respective residual network structures B1, B2, and B3, the blackhead detection model can finally obtain the corresponding first feature map f1, second feature map f2, and third feature map f3, the feature sizes are related to the decreasing network levels, for example, four network levels, the sizes of the first feature map f1, the second feature map f2 and the third feature map f3 are 1/4 of the corresponding second image, i.e., 1/16, 1/8 and 1/4 of the first image.
On the basis, f1 can be upsampled and then fused with f2, so that a fused fourth feature map f4 can be obtained, and by applying f4 to a convolution operation (for example, 1 × 1 convolution), a first blackhead heat map h1 with the size of 1/8 can be obtained as the supervisory information.
Then, by upsampling f4 and fusing with f2, a fused fifth feature map f5 can be obtained, and by applying a convolution operation (e.g., 1 × 1 convolution) to f5, a second blackhead heat map h2 of 1/4 size can be obtained as the supervisory information.
Next, by upsampling f3, a third blackhead heatmap h3 of size 1/2 may be obtained.
As such, the first blackhead heatmap h1, the second blackhead heatmap h2, and the third blackhead heatmap h3 may serve as blackhead heatmaps for the first size, the second size, and the third size.
Therefore, the residual error network structure of the blackhead detection model is subjected to different-size reduction and improvement, so that the size of the first image of the blackhead to be detected can be converted as much as possible, corresponding second images with different sizes are obtained, the blackhead detection speed is greatly improved on the premise of not reducing the detection precision, and the performance loss is reduced.
In a possible implementation manner, in order to avoid performance loss caused by the existing ensemble training scheme and improve the blackhead speed, the blackhead detection model can be obtained by training in the following manner:
first, a training sample set and a test sample set are obtained. The training sample set can include a plurality of nose area samples marked with blackhead areas, and the test sample set includes a plurality of nose area samples not marked with blackhead areas. For example, a plurality of image data sets including faces may be acquired, a nose region image of each image is cut out, a blackhead region in the nose region image is labeled to obtain a training sample set, and a part of the nose region image is allocated as a test sample set.
And then, training the convolutional neural network model to be trained according to the training sample set to obtain the trained convolutional neural network model.
And finally, testing the trained convolutional neural network model according to the test sample set, and obtaining the blackhead detection model obtained by training when the test result meets the preset condition.
Therefore, blackhead detection can be performed on the same convolutional neural network model from training to testing instead of being formed by different steps in a set connection mode, detection precision loss is avoided, and detection speed is improved.
In a possible implementation manner, since the number of the training samples may directly reflect the detection accuracy of the blackhead detection model obtained by subsequent training, before the specific training, in order to expand the training sample set, this embodiment may further perform sample gain on the training sample set, and adjust the size of each nose region sample in the training sample set after the sample gain to a set size, so as to train the convolutional neural network model to be trained according to the adjusted training sample set.
For example, the training sample set may be subjected to sample gain in any manner including clipping, horizontal or vertical flipping, and the like, then each pixel point of each nose region sample in the training sample set may be normalized to be between 0 and 1, and finally the size of each nose region sample in the training sample set is scaled to 512 × 512.
In one possible implementation, the trained network parameters, such as learning rate, weight attenuation coefficient, momentum, etc., may be configured in advance for the convolutional neural network model to be trained (e.g., learning rate initialized to 0.001, weight attenuation coefficient 0.0005, momentum 0.99, etc.). In the process of training the convolutional neural network model to be trained according to the training sample set to obtain the trained convolutional neural network model, the size of each nose region sample in the training sample set can be converted, and then the blackhead heatmap corresponding to the nose region sample of each size is output through the residual error network structure corresponding to each size in the convolutional neural network model. In the specific size conversion process in this step and the process of outputting the blackhead heatmap corresponding to the nose area sample of each size, reference is made to the foregoing detailed description of step S130, and details are not repeated here.
Then, cross entropy loss function values between the blackhead heat maps corresponding to the nose area samples of each size and the blackhead areas marked on the nose area samples can be calculated, so that back propagation training is carried out according to each cross entropy loss function value obtained through calculation, after the gradient of the network parameters of the residual network structures corresponding to each size is calculated, the network parameters of the residual network structures corresponding to each size are updated by a random gradient descent method according to the calculated gradient, the step of outputting the blackhead heat maps corresponding to the nose area samples of each size through the residual network structures corresponding to each size in the convolutional neural network model is returned, and the convolutional neural network model after training is obtained until the convolutional neural network model meets the training termination condition.
The training termination condition may include at least one of the following conditions:
1) the iterative training times reach the set times; 2) the cross entropy loss function value is lower than a set threshold; 3) the cross entropy loss function value does not decrease.
In the condition 1), in order to save the operation amount, the maximum value of the iteration times may be set, and if the iteration times reaches the set times, the iteration of the iteration cycle may be stopped, and the finally obtained convolutional neural network model is used as the blackhead detection model. In the condition 2), if the cross entropy loss function value is lower than the set threshold, which indicates that the current blackhead detection model can substantially meet the condition, the iteration can be stopped. In condition 3), the cross entropy loss function value no longer decreases, indicating that the optimal blackhead detection model has been formed, and the iteration can be stopped.
It should be noted that the above-mentioned iteration stop conditions may be used in combination or alternatively, for example, the iteration may be stopped when the cross entropy loss function value does not decrease any more, or the iteration may be stopped when the number of iterations reaches a set number, or the iteration may be stopped when the cross entropy loss function value does not decrease any more. Alternatively, the iteration may also be stopped when the cross entropy loss function value is below a set threshold and the cross entropy loss function value no longer drops.
In addition, in the practical implementation process, the training termination condition may not be limited to the above example, and a person skilled in the art may design a training termination condition different from the above example according to the practical requirement.
In a possible implementation manner, in the process of testing the trained convolutional neural network model according to the test sample set and obtaining the blackhead detection model obtained by training when the test result meets the preset condition, firstly, each nose area sample in the test sample set can be input into the trained convolutional neural network model to obtain a blackhead detection result corresponding to each nose area sample, then, the prediction accuracy of the blackhead detection result corresponding to each nose area sample can be counted, and when the prediction accuracy is greater than the set accuracy, the test result is judged to meet the preset condition, and the blackhead detection model obtained by training is obtained.
And when the predicted accuracy is not greater than the set accuracy, recording the times of the current iteration test, performing weighted calculation according to the weights of the times of the current iteration test to obtain the weight of the next iteration test, and performing the iteration test on each nose area sample in the test sample set according to the weight of the next iteration test. For example, there are many weights in the blackhead detection model, and assuming that W, the weights of different iteration times may be weighted as the iteration times overlap, that is, W (T +1) ═ 0.9 × W (T) +0.1 × W (T +1), where T refers to the iteration times, and the test is performed by updating the weights through such weights, so that the test result may be more stable.
In one possible implementation, for step S140, after obtaining the blackhead heat maps corresponding to the second images of each size, the blackhead heat maps are different in size in order to obtain blackhead detection information of different scales. Based on this, the final blackhead detection result can be determined by performing weighted calculation on the blackhead heat map corresponding to the second image of each size. For example, referring to fig. 3 in combination, step S140 may be implemented by the following sub-steps:
in the substep S141, the blackhead heat map corresponding to the second image with the size lower than that of the first image is up-sampled to the size of the blackhead heat map of the first image, and the weighted average calculation is performed on the up-sampled blackhead heat map of the second image with the same size as that of the first image, so as to obtain a weighted blackhead heat map.
In this embodiment, taking the first blackhead heat map h1, the second blackhead heat map h2, and the third blackhead heat map h3 corresponding to the first size, the second size, and the third size as examples, the first blackhead heat map h1 and the second blackhead heat map h2 may be upsampled to the size of the third blackhead heat map h3, and then the upsampled first blackhead heat map h1, second blackhead heat map h2, and third blackhead heat map h3 may be subjected to weighted average calculation to obtain a weighted blackhead heat map.
And a substep S142 of searching a predicted value for each unit region in the weighted blackhead heatmap, and determining the unit region as the blackhead region if the predicted value is greater than a set prediction threshold.
In this embodiment, the weighted blackhead heatmap may be understood as the final output of the blackhead detection model, i.e., the predicted value of each unit area is predicted. Thus, the predicted value for each unit area in the weighted blackhead heatmap can be searched, and if the predicted value is greater than the set prediction threshold, the unit area is determined as a blackhead area, otherwise, the unit area is determined as a non-blackhead area.
Therefore, the accuracy of blackhead detection can be improved under a complex background or a variable background, and the blackhead detection method is effectively suitable for various skin environment changes.
Based on the same inventive concept, please refer to fig. 4, which shows a schematic diagram of functional modules of the image detection apparatus 110 provided in the embodiment of the present application, and the embodiment can divide the functional modules of the image detection apparatus 110 according to the above method embodiment. For example, the functional blocks may be divided for the respective functions, or two or more functions may be integrated into one processing block. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. It should be noted that, in the embodiment of the present application, the division of the module is schematic, and is only one logic function division, and there may be another division manner in actual implementation. For example, in the case of dividing each functional module according to each function, the image detection apparatus 110 shown in fig. 4 is only an apparatus diagram. The image detection apparatus 110 may include an acquisition module 111, a size conversion module 112, a heat map output module 113, and a blackhead detection module 114, and the functions of the functional modules of the image detection apparatus 110 are described in detail below.
The acquiring module 111 is configured to acquire a first image of a blackhead to be detected. It is understood that the obtaining module 111 can be used to execute the step S110, and for the detailed implementation of the obtaining module 111, reference can be made to the content related to the step S110.
And a size conversion module 112, configured to perform size conversion on the first image to obtain corresponding second images with different sizes. It is understood that the size conversion module 112 can be used to perform the above step S120, and for the detailed implementation of the size conversion module 112, reference can be made to the above contents related to the step S120.
And the heat map output module 113 is configured to input the second image of each size into a pre-trained blackhead detection model, and output the blackhead heat map corresponding to each size through a residual network structure corresponding to each size in the blackhead detection model. It is understood that the heatmap output module 113 may be configured to perform the step S130, and the detailed implementation of the heatmap output module 113 may refer to the content related to the step S130.
And the blackhead detection module 114 is configured to perform weighting processing on the blackhead heatmap corresponding to the second image of each size to obtain a blackhead detection result. It is understood that the blackhead detecting module 114 may be configured to perform the step S140, and for the detailed implementation of the blackhead detecting module 114, reference may be made to the content related to the step S140.
In one possible implementation, the heat map output module 113 may output the blackhead heat map corresponding to each size by:
outputting a first feature map, a second feature map and a third feature map corresponding to the first size, the second size and the third size respectively through residual error network structures corresponding to the first size, the second size and the third size;
the first feature map is subjected to up-sampling and then is fused with the second feature map, and the fused fourth feature map is subjected to convolution operation to obtain a first blackhead heat map;
the fourth feature map is subjected to up-sampling and then is fused with the second feature map, and the fused fifth feature map is subjected to convolution operation to obtain a second blackhead heat map;
and upsampling the third feature map to obtain a third blackhead heat map.
In one possible implementation, the blackhead detection model can be trained by the following steps:
acquiring a training sample set and a test sample set, wherein the training sample set comprises a plurality of nose area samples marked with blackhead areas, and the test sample set comprises a plurality of nose area samples not marked with the blackhead areas;
training the convolutional neural network model to be trained according to the training sample set to obtain the trained convolutional neural network model;
and testing the trained convolutional neural network model according to the test sample set, and obtaining the blackhead detection model obtained by training when the test result meets the preset condition.
In a possible implementation manner, the training sample set may further be subjected to sample gain, and the size of each nose region sample in the training sample set after the sample gain is adjusted to a set size, so as to train the convolutional neural network model to be trained according to the adjusted training sample set.
In a possible implementation manner, the convolutional neural network model to be trained is trained according to the training sample set, and the manner of obtaining the trained convolutional neural network model may be:
for each nose area sample in the training sample set, after size conversion is carried out on the nose area sample, a blackhead heat map corresponding to the nose area sample of each size is output through a residual error network structure corresponding to each size in the convolutional neural network model;
calculating a cross entropy loss function value between the blackhead heat map corresponding to the nose area sample of each size and the blackhead area marked on the nose area sample;
performing back propagation training according to each calculated cross entropy loss function value, and calculating the gradient of the network parameter of the residual error network structure corresponding to each size;
and according to the calculated gradient, updating the network parameters of the residual error network structure corresponding to each size by adopting a random gradient descent method, and returning to the step of outputting the blackhead heat map corresponding to the nose area sample of each size through the residual error network structure corresponding to each size in the convolutional neural network model respectively until the convolutional neural network model meets the training termination condition, thereby obtaining the trained convolutional neural network model.
In a possible implementation manner, the method of testing the trained convolutional neural network model according to the test sample set and obtaining the blackhead detection model obtained by training when the test result satisfies the preset condition may be:
inputting each nose area sample in the test sample set into the trained convolutional neural network model to obtain a blackhead detection result corresponding to each nose area sample;
counting the prediction accuracy of the blackhead detection result corresponding to each nose area sample;
when the prediction accuracy is larger than the set accuracy, judging that the test result meets a preset condition, and obtaining a blackhead detection model obtained through training;
and when the predicted accuracy is not greater than the set accuracy, recording the times of the current iteration test, performing weighted calculation according to the weights of the times of the current iteration test to obtain the weight of the next iteration test, and performing the iteration test on each nose area sample in the test sample set according to the weight of the next iteration test.
In one possible implementation, the blackhead detection module 114 may perform weighting processing on the blackhead heatmap corresponding to the second image of each size to obtain the blackhead detection result by:
the method comprises the steps that a blackhead heat map corresponding to a second image with the size lower than that of a first image is up-sampled to be the size of the blackhead heat map of the first image, and weighted average calculation is carried out on the up-sampled blackhead heat map and the blackhead heat map of the second image with the same size as that of the first image to obtain a weighted blackhead heat map;
searching a predicted value of each unit area in the weighted blackhead heatmap, and determining the unit area as the blackhead area if the predicted value is greater than a set prediction threshold value.
Based on the same inventive concept, please refer to fig. 5, which illustrates a schematic block diagram of an electronic device 100 for executing the image detection method provided in the embodiment of the present application, where the electronic device 100 may include an image detection apparatus 110, a machine-readable storage medium 120, and a processor 130.
In this embodiment, the machine-readable storage medium 120 and the processor 130 are both located in the electronic device 100 and are separately located. However, it should be understood that the machine-readable storage medium 120 may also be separate from the electronic device 100 and accessible by the processor 130 through a bus interface. Alternatively, the machine-readable storage medium 120 may be integrated into the processor 130, e.g., may be a cache and/or general purpose registers.
The processor 130 is a control center of the electronic device 100, connects various parts of the entire electronic device 100 using various interfaces and lines, performs various functions of the electronic device 100 and processes data by running or executing software programs and/or modules stored in the machine-readable storage medium 120 and calling data stored in the machine-readable storage medium 120, thereby performing overall monitoring of the electronic device 100. Alternatively, processor 130 may include one or more processing cores; for example, the processor 130 may integrate an application processor, which primarily handles operating systems, user interfaces, applications, etc., and a modem processor, which primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor.
The processor 130 may be a general-purpose Central Processing Unit (CPU), a microprocessor, an Application-Specific Integrated Circuit (ASIC), or one or more Integrated circuits for controlling the execution of the image detection method provided by the above-mentioned method embodiments.
The machine-readable storage medium 120 may be, but is not limited to, a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, an Electrically Erasable programmable Read-Only MEMory (EEPROM), a compact disc Read-Only MEMory (CD-ROM) or other optical disk storage, optical disk storage (including compact disc, laser disk, optical disk, digital versatile disk, blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The machine-readable storage medium 120 may be self-contained and coupled to the processor 130 via a communication bus. The machine-readable storage medium 120 may also be integrated with the processor. The machine-readable storage medium 120 is used for storing machine-executable instructions for performing aspects of the present application. The processor 130 is configured to execute machine executable instructions stored in the machine readable storage medium 120 to implement the image detection method provided by the foregoing method embodiment.
The image detection apparatus 110 may include, for example, the functional modules (e.g., the acquisition module 111, the size conversion module 112, the heat map output module 113, and the blackhead detection module 114) described in fig. 4, and may be stored in the machine-readable storage medium 120 in the form of software program codes, and the processor 130 may implement the image detection method provided by the foregoing method embodiment by executing the functional modules of the image detection apparatus 110.
Since the electronic device 100 provided in the embodiment of the present application is another implementation form of the method embodiment executed by the electronic device 100, and the electronic device 100 can be used to execute the image detection method provided in the method embodiment, the technical effect obtained by the electronic device 100 can refer to the method embodiment, and is not described herein again.
Further, the present application also provides a readable storage medium containing computer executable instructions, and when executed, the computer executable instructions may be used to implement the image detection method provided by the foregoing method embodiment.
Of course, the storage medium provided in the embodiments of the present application and containing computer-executable instructions is not limited to the above method operations, and may also perform related operations in the image detection method provided in any embodiment of the present application.
Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the present application has been described in connection with various embodiments, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed application, from a review of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
The above description is only for various embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present application, and all such changes or substitutions are included in the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. An image detection method applied to an electronic device, the method comprising:
acquiring a first image of a blackhead to be detected;
performing size conversion on the first image to obtain corresponding second images with different sizes;
inputting the second image of each size into a pre-trained blackhead detection model, and outputting a blackhead heat map corresponding to each size through a residual error network structure corresponding to each size in the blackhead detection model;
and carrying out weighting processing on the blackhead heat map corresponding to the second image of each size to obtain a blackhead detection result.
2. The image detection method according to claim 1, wherein the sizes include a first size, a second size, and a third size, and the first size, the second size, and the third size sequentially increase, and the step of outputting the blackhead heatmap corresponding to each size through the residual network structure corresponding to each size in the blackhead detection model respectively includes:
outputting a first feature map, a second feature map and a third feature map corresponding to the first size, the second size and the third size respectively through residual error network structures corresponding to the first size, the second size and the third size;
the first feature map is subjected to up-sampling and then is fused with the second feature map, and the fused fourth feature map is subjected to convolution operation to obtain a first blackhead heat map;
the fourth feature map is subjected to up-sampling and then is fused with the second feature map, and the fused fifth feature map is subjected to convolution operation to obtain a second blackhead heat map;
and upsampling the third feature map to obtain a third blackhead heat map.
3. The image detection method according to claim 1, wherein the blackhead detection model is trained by the following steps:
acquiring a training sample set and a test sample set, wherein the training sample set comprises a plurality of nose area samples marked with blackhead areas, and the test sample set comprises a plurality of nose area samples not marked with the blackhead areas;
training a convolutional neural network model to be trained according to the training sample set to obtain a trained convolutional neural network model;
and testing the trained convolutional neural network model according to the test sample set, and obtaining a blackhead detection model obtained by training when a test result meets a preset condition.
4. The image detection method according to claim 3, wherein before the step of training the convolutional neural network model to be trained according to the training sample set to obtain the trained convolutional neural network model, the method further comprises:
and performing sample gain on the training sample set, adjusting the size of each nose region sample in the training sample set after the sample gain to be a set size, and training the convolutional neural network model to be trained according to the adjusted training sample set.
5. The image detection method according to claim 3, wherein the step of training the convolutional neural network model to be trained according to the training sample set to obtain the trained convolutional neural network model comprises:
for each nose area sample in the training sample set, after size conversion is carried out on the nose area sample, a blackhead heat map corresponding to the nose area sample of each size is output through a residual error network structure corresponding to each size in the convolutional neural network model;
calculating a cross entropy loss function value between the blackhead heat map corresponding to the nose area sample of each size and the blackhead area marked on the nose area sample;
performing back propagation training according to each calculated cross entropy loss function value, and calculating the gradient of the network parameter of the residual error network structure corresponding to each size;
and according to the calculated gradient, updating the network parameters of the residual error network structure corresponding to each size by adopting a random gradient descent method, and returning to the step of outputting the blackhead heat map corresponding to the nose area sample of each size through the residual error network structure corresponding to each size in the convolutional neural network model respectively until the convolutional neural network model meets the training termination condition, so as to obtain the trained convolutional neural network model.
6. The image detection method according to claim 3, wherein the step of testing the trained convolutional neural network model according to the test sample set and obtaining a blackhead detection model obtained by training when a test result meets a preset condition includes:
inputting each nose area sample in the test sample set into the trained convolutional neural network model to obtain a blackhead detection result corresponding to each nose area sample;
counting the prediction accuracy of the blackhead detection result corresponding to each nose area sample;
when the prediction accuracy is larger than the set accuracy, judging that the test result meets a preset condition, and obtaining a blackhead detection model obtained through training;
and when the predicted accuracy is not greater than the set accuracy, recording the times of the current iteration test, performing weighted calculation according to the weights of the times of the current iteration test to obtain the weight of the next iteration test, and performing the iteration test on each nose area sample in the test sample set according to the weight of the next iteration test.
7. The image detection method according to any one of claims 1 to 6, wherein the step of weighting the blackhead heat map corresponding to the second image of each size to obtain a blackhead detection result includes:
the method comprises the steps of up-sampling a blackhead heat map corresponding to a second image with the size lower than that of a first image into the size of the blackhead heat map of the first image, and carrying out weighted average calculation on the up-sampled blackhead heat map and the blackhead heat map of the second image with the same size as that of the first image to obtain a weighted blackhead heat map;
searching a predicted value of each unit area in the weighted blackhead heatmap, and determining the unit area as a blackhead area if the predicted value is greater than a set prediction threshold.
8. An image detection apparatus, applied to an electronic device, the apparatus comprising:
the acquisition module is used for acquiring a first image of a blackhead to be detected;
the size conversion module is used for carrying out size conversion on the first image to obtain corresponding second images with different sizes;
the heat map output module is used for inputting the second images of each size into a pre-trained blackhead detection model and outputting the blackhead heat map corresponding to each size through a residual error network structure corresponding to each size in the blackhead detection model;
and the blackhead detection module is used for weighting the blackhead heat map corresponding to the second image of each size to obtain a blackhead detection result.
9. An electronic device comprising a machine-readable storage medium having stored thereon machine-executable instructions and a processor, wherein the processor, when executing the machine-executable instructions, implements the image detection method of any one of claims 1-7.
10. A readable storage medium having stored therein machine executable instructions which when executed perform the image detection method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911087220.1A CN110796115B (en) | 2019-11-08 | 2019-11-08 | Image detection method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911087220.1A CN110796115B (en) | 2019-11-08 | 2019-11-08 | Image detection method and device, electronic equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110796115A true CN110796115A (en) | 2020-02-14 |
| CN110796115B CN110796115B (en) | 2022-12-23 |
Family
ID=69443494
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911087220.1A Active CN110796115B (en) | 2019-11-08 | 2019-11-08 | Image detection method and device, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110796115B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111311613A (en) * | 2020-03-03 | 2020-06-19 | 北京推想科技有限公司 | Image segmentation model training method, image segmentation method and device |
| CN113435233A (en) * | 2020-03-23 | 2021-09-24 | 北京金山云网络技术有限公司 | Pornographic image identification method and system and electronic equipment |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108229455A (en) * | 2017-02-23 | 2018-06-29 | 北京市商汤科技开发有限公司 | Object detecting method, the training method of neural network, device and electronic equipment |
| CN108921825A (en) * | 2018-06-12 | 2018-11-30 | 北京羽医甘蓝信息技术有限公司 | The method and device of the facial skin points shape defect of detection based on deep learning |
| CN109344821A (en) * | 2018-08-30 | 2019-02-15 | 西安电子科技大学 | Small target detecting method based on Fusion Features and deep learning |
| CN109447247A (en) * | 2018-10-23 | 2019-03-08 | 北京旷视科技有限公司 | Obtain the method and device of convolutional neural networks |
| CN109544516A (en) * | 2018-11-05 | 2019-03-29 | 深圳和而泰数据资源与云技术有限公司 | Image detecting method and device |
| CN109785344A (en) * | 2019-01-22 | 2019-05-21 | 成都大学 | The remote sensing image segmentation method of binary channel residual error network based on feature recalibration |
| CN109816695A (en) * | 2019-01-31 | 2019-05-28 | 中国人民解放军国防科技大学 | Target detection and tracking method for infrared small unmanned aerial vehicle under complex background |
| CN109948551A (en) * | 2019-03-20 | 2019-06-28 | 合肥黎曼信息科技有限公司 | A kind of ordered categorization method of skin blackhead detection |
| CN110110798A (en) * | 2019-05-13 | 2019-08-09 | 广东工业大学 | A kind of weld joint recognition method based on Mask-RCNN network |
| CN110287806A (en) * | 2019-05-30 | 2019-09-27 | 华南师范大学 | A kind of traffic sign recognition method based on improvement SSD network |
| CN110321896A (en) * | 2019-04-30 | 2019-10-11 | 深圳市四季宏胜科技有限公司 | Blackhead recognition methods, device and computer readable storage medium |
-
2019
- 2019-11-08 CN CN201911087220.1A patent/CN110796115B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108229455A (en) * | 2017-02-23 | 2018-06-29 | 北京市商汤科技开发有限公司 | Object detecting method, the training method of neural network, device and electronic equipment |
| CN108921825A (en) * | 2018-06-12 | 2018-11-30 | 北京羽医甘蓝信息技术有限公司 | The method and device of the facial skin points shape defect of detection based on deep learning |
| CN109344821A (en) * | 2018-08-30 | 2019-02-15 | 西安电子科技大学 | Small target detecting method based on Fusion Features and deep learning |
| CN109447247A (en) * | 2018-10-23 | 2019-03-08 | 北京旷视科技有限公司 | Obtain the method and device of convolutional neural networks |
| CN109544516A (en) * | 2018-11-05 | 2019-03-29 | 深圳和而泰数据资源与云技术有限公司 | Image detecting method and device |
| CN109785344A (en) * | 2019-01-22 | 2019-05-21 | 成都大学 | The remote sensing image segmentation method of binary channel residual error network based on feature recalibration |
| CN109816695A (en) * | 2019-01-31 | 2019-05-28 | 中国人民解放军国防科技大学 | Target detection and tracking method for infrared small unmanned aerial vehicle under complex background |
| CN109948551A (en) * | 2019-03-20 | 2019-06-28 | 合肥黎曼信息科技有限公司 | A kind of ordered categorization method of skin blackhead detection |
| CN110321896A (en) * | 2019-04-30 | 2019-10-11 | 深圳市四季宏胜科技有限公司 | Blackhead recognition methods, device and computer readable storage medium |
| CN110110798A (en) * | 2019-05-13 | 2019-08-09 | 广东工业大学 | A kind of weld joint recognition method based on Mask-RCNN network |
| CN110287806A (en) * | 2019-05-30 | 2019-09-27 | 华南师范大学 | A kind of traffic sign recognition method based on improvement SSD network |
Non-Patent Citations (2)
| Title |
|---|
| TSUNG-YI LIN，PIOTR DOLLA&ACUTER，ROSS GIRSHICK，KAIMING HE: "Feature Pyramid Networks for Object Detection", 《ARXIV:1612.03144V2 [CS.CV]》 * |
| YANBEI CHEN，XIATIAN ZHU，SHAOGANG GONG: "Person Re-Identification by Deep Learning Multi-Scale Representations", 《2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111311613A (en) * | 2020-03-03 | 2020-06-19 | 北京推想科技有限公司 | Image segmentation model training method, image segmentation method and device |
| CN113435233A (en) * | 2020-03-23 | 2021-09-24 | 北京金山云网络技术有限公司 | Pornographic image identification method and system and electronic equipment |
| CN113435233B (en) * | 2020-03-23 | 2024-03-05 | 北京金山云网络技术有限公司 | Pornographic image recognition method and system and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110796115B (en) | 2022-12-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112801164A (en) | Training method, device and equipment of target detection model and storage medium | |
| CN111488985B (en) | Deep neural network model compression training method, device, equipment and medium | |
| CN111914843B (en) | Character detection method, system, equipment and storage medium | |
| CN112857268B (en) | Object area measuring method, device, electronic equipment and storage medium | |
| CN113313083B (en) | Text detection method and device | |
| US20220406090A1 (en) | Face parsing method and related devices | |
| CN108875519B (en) | Object detection method, device and system and storage medium | |
| CN110796115B (en) | Image detection method and device, electronic equipment and readable storage medium | |
| JP2023527615A (en) | Target object detection model training method, target object detection method, device, electronic device, storage medium and computer program | |
| CN114580263A (en) | Knowledge graph-based information system fault prediction method and related equipment | |
| CN115631112B (en) | Building contour correction method and device based on deep learning | |
| CN114202648B (en) | Text image correction method, training device, electronic equipment and medium | |
| CN109492697B (en) | Picture detection network training method and picture detection network training device | |
| CN113269280B (en) | Text detection method and device, electronic equipment and computer readable storage medium | |
| CN111597845A (en) | Two-dimensional code detection method, device and equipment and readable storage medium | |
| CN112633065A (en) | Face detection method, system, storage medium and terminal based on data enhancement | |
| CN110610185B (en) | Method, device and equipment for detecting salient object of image | |
| CN115331077B (en) | Training method of feature extraction model, target classification method, device and equipment | |
| CN117078942B (en) | Context-aware refereed image segmentation method, system, device and storage medium | |
| CN114842474B (en) | Character recognition method, device, electronic equipment and medium | |
| CN116188917B (en) | Defect data generation model training method, defect data generation method and device | |
| CN117376977B (en) | Mobile phone 5G wireless signal testing system, method, equipment and medium | |
| CN112949526B (en) | Face detection method and device | |
| CN113837948B (en) | Image generation method, apparatus and storage medium | |
| CN111311616B (en) | Method and apparatus for segmenting an image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20211209 Address after: 361100 568, No. 942, tonglong Second Road, torch high tech Zone (Xiang'an) Industrial Zone, Xiang'an District, Xiamen City, Fujian Province Applicant after: Xiamen Meitu Yifu Technology Co.,Ltd. Address before: B1f-089, Zone C, Huaxun building, software park, torch high tech Zone, Xiamen City, Fujian Province Applicant before: XIAMEN HOME MEITU TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |