CN117611580B

CN117611580B - Flaw detection method, flaw detection device, computer equipment and storage medium

Info

Publication number: CN117611580B
Application number: CN202410074197.7A
Authority: CN
Inventors: 王念欧; 郦轲; 刘文华; 万进
Original assignee: Shenzhen Accompany Technology Co Ltd
Current assignee: Shenzhen Accompany Technology Co Ltd
Priority date: 2024-01-18
Filing date: 2024-01-18
Publication date: 2024-05-24
Anticipated expiration: 2044-01-18
Also published as: CN117611580A

Abstract

The application relates to a flaw detection method, a flaw detection device, computer equipment and a storage medium. The method comprises the following steps: performing feature extraction on the image to be detected through a first weight matrix to obtain a flaw number convolution feature map, and performing feature extraction on the image to be detected through a second weight matrix to obtain a flaw level convolution feature map; carrying out global maximum pooling treatment on the flaw number convolution feature images, and carrying out global average pooling treatment on the flaw level convolution feature images; flattening and straightening the flaw quantity pooling feature map and the flaw grade pooling feature map respectively; converting the flaw number feature vector into a flaw number probability distribution feature vector, and converting the flaw grade feature vector into a flaw grade probability distribution feature vector; and determining a flaw detection result of the target part according to the flaw number probability distribution feature vector and the flaw grade probability distribution feature vector. The accuracy of counting and classifying flaws can be improved.

Description

Flaw detection method, flaw detection device, computer equipment and storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a flaw detection method, a flaw detection device, a computer device, and a storage medium.

Background

With the increasing attention of the current society to skin health conditions, the detection and analysis method for skin conditions also makes great progress in the development process. Acne is a common skin problem, also known as acne or comedo, and usually occurs in puberty. It is caused by excessive secretion of grease from hair follicle, blockage of follicular orifice by cutin, and bacterial infection. Acne is commonly present on the face, chest and back and can have an impact on the appearance and mental health of teenagers.

Deep learning is currently commonly used in the industry to grade acne severity, but ambiguity between labels is often not considered in grading acne severity. Since the appearance of skin problems such as acne is very similar and the severity is very similar, the current counting and grading of acne is extremely inaccurate.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a flaw detection method, apparatus, computer device, and computer-readable storage medium that can improve the accuracy of counting and classifying acne.

In a first aspect, the present application provides a flaw detection method. The method comprises the following steps:

acquiring an image to be detected; the image to be detected is an image shot aiming at a target part of a target object;

Performing feature extraction on the image to be detected through a first weight matrix to obtain a flaw number convolution feature map, and performing feature extraction on the image to be detected through a second weight matrix to obtain a flaw level convolution feature map;

Performing global maximum pooling treatment on the flaw number convolution feature images to obtain flaw number pooling feature images, and performing global average pooling treatment on the flaw level convolution feature images to obtain flaw level pooling feature images;

flattening and straightening the flaw number pooling feature map and the flaw grade pooling feature map respectively to obtain flaw number feature vectors and flaw grade feature vectors;

converting the flaw number feature vector into a flaw number probability distribution feature vector, and converting the flaw grade feature vector into a flaw grade probability distribution feature vector;

And determining a flaw detection result of the target part according to the flaw number probability distribution feature vector and the flaw grade probability distribution feature vector.

In a second aspect, the application further provides a flaw detection device. The device comprises:

The acquisition module is used for acquiring the image to be detected; the image to be detected is an image shot aiming at a target part of a target object;

The feature extraction module is used for carrying out feature extraction on the image to be detected through the first weight matrix to obtain a flaw number convolution feature map, and carrying out feature extraction on the image to be detected through the second weight matrix to obtain a flaw level convolution feature map;

The pooling processing module is used for carrying out global maximum pooling processing on the flaw number convolution feature images to obtain flaw number pooling feature images, and carrying out global average pooling processing on the flaw level convolution feature images to obtain flaw level pooling feature images;

The flattening and straightening module is used for respectively flattening and straightening the flaw number pooling feature map and the flaw grade pooling feature map to obtain flaw number feature vectors and flaw grade feature vectors;

The conversion module is used for converting the flaw number feature vector into a flaw number probability distribution feature vector and converting the flaw grade feature vector into a flaw grade probability distribution feature vector;

And the determining module is used for determining the flaw detection result of the target part according to the flaw number probability distribution feature vector and the flaw grade probability distribution feature vector.

In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor which when executing the computer program performs the steps of:

In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:

The flaw detection method, the flaw detection device, the computer equipment and the storage medium are used for acquiring an image to be detected; the image to be detected is an image shot aiming at a target part of a target object; performing feature extraction on the image to be detected through a first weight matrix to obtain a flaw number convolution feature map, and performing feature extraction on the image to be detected through a second weight matrix to obtain a flaw level convolution feature map; performing global maximum pooling treatment on the flaw number convolution feature images to obtain flaw number pooling feature images, and performing global average pooling treatment on the flaw level convolution feature images to obtain flaw level pooling feature images; flattening and straightening the flaw quantity pooling feature map and the flaw grade pooling feature map respectively to obtain flaw quantity feature vectors and flaw grade feature vectors; converting the flaw number feature vector into a flaw number probability distribution feature vector, and converting the flaw grade feature vector into a flaw grade probability distribution feature vector; and determining a flaw detection result of the target part according to the flaw number probability distribution feature vector and the flaw grade probability distribution feature vector. Thereby avoiding adverse effects caused by ambiguity between labels to a certain extent and improving the accuracy of counting and grading acnes.

Drawings

FIG. 1 is a diagram of an application environment of a flaw detection method according to an embodiment;

FIG. 2 is a flow chart of a flaw detection method according to an embodiment;

FIG. 3 is a process diagram of a flaw detection model in one embodiment;

FIG. 4 is a schematic diagram of a residual module of a flaw detection model in one embodiment;

FIG. 5 is a schematic diagram of a defect number branching module of the defect detection model in one embodiment;

FIG. 6 is a schematic diagram of a defect level branching module of the defect detection model in one embodiment;

FIG. 7 is a flow chart of a flaw detection method according to another embodiment;

FIG. 8 is a block diagram of a flaw detection device according to an embodiment;

FIG. 9 is a block diagram of a flaw detection device according to an embodiment;

Fig. 10 is an internal structural view of a computer device in one embodiment.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

The image processing method provided by the embodiment of the application can be applied to an application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server. The terminal 102 generates a flaw detection request, and then sends the flaw detection request to the server 104, so that the server 104 determines a flaw detection result of the target portion. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices, and portable wearable devices, where the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart vehicle devices, and the like. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers.

In one embodiment, as shown in fig. 2, a flaw detection method is provided, and the method is applied to the server in fig. 1 for illustration, and includes the following steps:

step 202, obtaining an image to be detected; the image to be detected is an image taken for a target portion of a target object.

The image to be detected, the target object and the target part are not particularly limited, and can be set according to actual needs. Optionally, the target object is a human body to be detected, the target part is a face of the human body to be detected, and the image to be detected is a face image of the human body to be detected. The image to be detected may be obtained by shooting through shooting equipment, or may be obtained by wearing an acquisition device such as an intelligent mask, which is not limited herein.

Specifically, whether the face of the human body to be detected is photographed by a photographing device or the face of the human body to be detected is collected by wearing a collecting device such as an intelligent mask, the size of the finally obtained image is not fixed in many cases, but the size of the flaw detection model to be input is fixed, and it is very important to maintain the original shape and proportion of the image. The shape of the image may be distorted, distorted or otherwise lost if deformed. Therefore, it is necessary to ensure that excessive stretching, compression, or shape deformation of the image is not caused when filling is performed. The application is not limited to a specific filling mode, and can be set according to actual needs.

Optionally, the padding operation is performed on the acquired image, which specifically includes: assuming that the acquired image is I and its size is w×h (width and height), it is first necessary to fill it to the target size W1×h1, and first the filling amount in the width and height is calculated by the following calculation formula.

Pading_h = W1 - W

Pading_w = H1 - H1

The padding is then distributed evenly to the four sides (up, down, left, right) of the image I, i.e. the padding matrix P,And finally obtaining a filled image which is the image to be detected.

And 204, performing feature extraction on the image to be detected through a first weight matrix to obtain a flaw number convolution feature map, and performing feature extraction on the image to be detected through a second weight matrix to obtain a flaw level convolution feature map.

The flaws in the present application refer to flaws related to skin problems, and it is easy to understand that the specific type of flaws is not limited, and can be set according to actual needs. Optionally, the flaws in the application refer to acne, and the flaw number convolution feature map is a feature map obtained by performing downsampling on the flaw number convolution feature map, and the flaw grade pooling feature map is a feature map obtained by performing downsampling on the flaw grade convolution feature map.

Specifically, referring to fig. 3, efficientne-b4 (a convolutional neural network) is selected as a backbone network to perform feature extraction on an image to be detected, so as to obtain an initial feature map, and a corresponding linear transformation and nonlinear transformation (i.e., a first weight matrix) are performed on the initial feature map through a flaw number branching module, so as to obtain a feature map for detecting the flaw number, i.e., a flaw number convolutional feature map. And similarly, performing corresponding linear transformation and nonlinear transformation (namely a second weight matrix) on the initial feature map through a flaw grade branching module to obtain a feature map for detecting flaw grades, namely a flaw grade convolution feature map. The application does not limit the specific structures of the flaw number branch modules and the flaw grade branch modules, and can be set according to actual needs.

Optionally, referring to fig. 4 and 5, in the defect number branching module, an input layer is sequentially included: and receiving an initial feature map extracted from the input backbone network. CBG module: the convolution layer (3*3)/batch normalization/GRelu (generalized rectifying linear unit), which is an extended form of an activation function (activation function), combines the characteristics of ReLU and Sigmoid or other activation functions for nonlinear transformation of neural networks. A residual module (convolution 1*1), a global max pooling module, and an acne count classification layer associated with a single sample acne count maximum. Referring to fig. 4 and 6, in the defect level branching module, an input layer: and receiving an initial feature map extracted from the input backbone network. CBG module: convolution layer (7*7), residual module (convolution 5*5), global average pooling module, residual module (convolution 3*3), acne grade classification layer: the output is 4 categories (mild/moderate/severe/very severe). Among them, CBG is collectively called Convolution (convolution) Batch Normailzation (batch normalization) Generailzed RECTIFIED LINEAR Unit (generalized rectification linear Unit).

And 206, performing global maximum pooling treatment on the flaw number convolution feature images to obtain flaw number pooling feature images, and performing global average pooling treatment on the flaw level convolution feature images to obtain flaw level pooling feature images.

The defect number pooling feature map is a feature map obtained after global maximum pooling treatment is carried out on the defect number convolution feature map, and the defect level pooling feature map is a feature map obtained after global average pooling treatment is carried out on the defect level convolution feature map.

In particular, global maximization refers to global maximization, an operation commonly used in deep learning networks to extract features and reduce data dimensions. In the global maximization process, for each feature map, the maximum value of the feature map is selected as an output. Global maximization is typically used to extract image features, helps preserve the most salient features in the image, and can reduce the number of parameters, reducing the risk of overfitting. In the present application, the number of acnes is the result of one locally significant feature, and global max pooling extracts the maximum on each feature channel as the pooling result. This operation helps to highlight the most prominent, important features in the image. For example, the defect number convolution feature map is downsampled, the maximum value on each feature channel is extracted in the defect number convolution feature map, and the final pooling result (i.e., the defect number pooling feature map) is obtained based on the maximum value extracted on each feature channel extracted in the defect number convolution feature map. The global average pooling process refers to a common pooling method in a neural network, which performs average pooling on the whole feature map, and takes all values in the feature map as output values. This pooling approach can help reduce the size of the feature map, reduce the number of parameters, and can extract the overall features of the overall feature map. Global averaging pooling is typically used at the last layer of the convolutional neural network to convert the feature map into a fixed length feature vector for classification or other tasks. In the present application, acne grade classification is an overall assessment of the overall face image globally, with global averaging pooling helping to extract global information of images or features. This helps the model learn the overall structure and characteristics of the image. And (3) downsampling the flaw-level convolution feature map, extracting an average value on each feature channel in the flaw-level convolution feature map, and obtaining a final pooling result (namely the flaw-level pooling feature map) according to the average value extracted on each feature channel extracted in the flaw-level convolution feature map.

And step 208, flattening and straightening the flaw number pooled feature map and the flaw grade pooled feature map respectively to obtain flaw number feature vectors and flaw grade feature vectors.

Specifically, a flattening operation is applied to the defect-number pooling feature map, converting it into a one-dimensional vector. In the Python deep learning framework, the defect number pooling feature map may be flattened using a corresponding function. After flattening treatment, the obtained one-dimensional vector is the feature vector of the defect number, and can be used for subsequent classification, regression or other tasks. Similarly, a flattening operation is applied to the defect level pooling feature map, converting it into a one-dimensional vector. In the Python deep learning framework, the flaw level pooling feature map may be flattened using a corresponding function. After flattening treatment, the obtained one-dimensional vector is a flaw grade characteristic vector, and can be used for subsequent classification, regression or other tasks. After the corresponding flaw number feature vectors and flaw grade feature vectors are obtained through flattening treatment, an output layer (full-connection layer) is needed to be connected in consideration of the image classification task in the application, and the flattened flaw number feature vectors and flaw grade feature vectors are mapped to specific output through the connected output layer. It will be appreciated how the output layer is specifically accessed and the specific output results mapped out are elaborated in the following steps. Step 210, converting the defect number feature vector into a defect number probability distribution feature vector, and converting the defect level feature vector into a defect level probability distribution feature vector.

Specifically, since the defect number classification category has 80 classification categories, a total of 80 from 1 to 80, respectively. Therefore, it is first necessary to map the defect number feature vector to 80 activation values correspondingly, specifically by mapping the defect number feature vector to 80 activation values through one output layer. This output layer may be a fully connected layer in a neural network that receives as input the fault number feature vector, learns to derive weights and bias parameters, and maps the input to a vector of 80 activation values. The 80 activation values are then respectively subjected to an exponential operation. An exponential operation can be used to map any real number to a non-negative real number, and by taking the exponent from each activation value, a corresponding positive real number is obtained. And finally, carrying out normalization processing on the 80 activation values after the exponential operation to obtain a flaw number probability distribution feature vector. Each element in the defect number probability distribution feature vector represents the probability of the corresponding category, and can be used to represent the probability distribution of the defect number feature vector under 80 categories.

Since the flaw-level classification category has 4 classification categories, a total of 4 classification categories of mild, moderate, and very severe, respectively. Therefore, it is first necessary to map the defect level feature vector into 4 activation values correspondingly, specifically by mapping the defect level feature vector into 4 activation values through one output layer. This output layer may be a fully connected layer in a neural network that receives the fault level feature vector as input, learns to derive weights and bias parameters, and maps the input to a vector of 4 activation values. The 4 activation values are then respectively subjected to an exponential operation. An exponential operation can be used to map any real number to a non-negative real number, and by taking the exponent from each activation value, a corresponding positive real number is obtained. And finally, carrying out normalization processing on the 4 activation values after the exponential operation to obtain a flaw level probability distribution feature vector. Each element in the fault level probability distribution feature vector represents the probability of the corresponding category and can be used to represent the probability distribution of the fault level feature vector under 4 categories.

Step 212, determining the flaw detection result of the target portion according to the flaw number probability distribution feature vector and the flaw level probability distribution feature vector.

Specifically, the flaw detection results regarding the target portion include a flaw number detection result and a flaw level detection result. The determining process of the flaw number detection result comprises the following steps: because each element in the defect number probability distribution feature vector represents the probability of being classified into the corresponding defect number classification category, the defect number classification category with the highest probability is directly selected from all elements in the defect number probability distribution feature vector, and the defect number corresponding to the defect number classification category is the defect number detection result. The determination process of the flaw grade detection result comprises the following steps: because each element in the flaw level probability distribution feature vector represents the probability of being classified into the corresponding flaw level classification category, the flaw level classification category with the highest probability is directly selected from all elements in the flaw level probability distribution feature vector, and the flaw level corresponding to the flaw level classification category is the flaw level detection result. However, only one flaw level detection result has a certain error, so that the flaw number detection result calculated in the previous step needs to be used for converting the flaw number detection result into another flaw level detection result in order to overcome the error, and thus, two flaw level detection results are comprehensively utilized, and finally, a more accurate flaw level detection result is obtained.

In one embodiment, for one of a plurality of defect number classification categories, a first weight value and a first bias value corresponding to the targeted defect number classification category are obtained; multiplying the flaw number feature vector by a first weight value to obtain a first target value; adding the first target value and the first bias value to obtain an activation value corresponding to the aimed flaw number classification category; and obtaining the probability distribution feature vector of the flaw number according to the activation value corresponding to each flaw number classification category.

Specifically, for one of a plurality of flaw number classification categories, a first weight value and a first bias value corresponding to the flaw number classification category for which it is intended are acquired. This means that a neural network needs to be trained first to learn the characteristics of the different defect number classification categories. During the training process, the neural network learns the weights and bias values corresponding to each category. And multiplying the flaw number feature vector by a first weight value to obtain a first target value. And adding the first target value and the first bias value to obtain an activation value corresponding to the category of the defect number classification. The first target value is added to the bias value learned before to obtain an activation value that represents the final score of the defect number classification. And obtaining the probability distribution feature vector of the flaw number according to the activation value corresponding to each flaw number classification category. According to the activation value corresponding to each flaw number classification, the probability distribution of each classification can be calculated. This probability distribution can be used to determine the likelihood of an overall quantitative classification.

Since the structure and the activation function of the output layer can be adjusted, multiple categories can be classified efficiently, and multi-classification tasks can be processed better. And by calculating the activation value and converting the activation value into probability distribution, the output of the model can be more intuitively understood, so that more references are provided for decision making. Such probability outputs may more clearly indicate the likelihood of classifying each flaw number than simply a binary classification result.

In one embodiment, the activation value corresponding to each flaw number classification category is converted into the probability that the image to be detected is classified into each flaw number classification category; and obtaining the probability distribution feature vector of the flaw number according to the probability that the image to be detected is classified into each flaw number classification category.

Specifically, the output activation value of the neural network is first converted into a probability of classifying a class corresponding to each flaw number by using a softmax function (activation function). The softmax function converts a set of real numbers into a probability distribution such that the sum of the probabilities for all classes is 1. The probability corresponding to each flaw number classification category can be visually represented. And then sequentially forming a feature vector from the probability value of each flaw number classification category, wherein the feature vector represents the probability distribution condition of the image to be detected classified into each flaw number classification category. By observing the feature vector, the probability distribution condition of the neural network for classifying the number of flaws of the image to be detected can be intuitively known.

As the probability distribution feature vector can provide more visual and rich information, the output result of the neural network can be better understood, and thus, decisions and judgments can be better made. Meanwhile, the probability distribution characteristic vector can be input into other models or systems for further processing and decision making. Therefore, the method is helpful for improving understanding and utilization of the defect number classification result.

In one embodiment, the conversion of the activation value corresponding to each flaw number classification category to the probability of classifying the image to be detected into each flaw number classification category is implemented by the following calculation formula:

Wherein Zi is the activation value corresponding to the i-th flaw number classification category, and n is the total number of flaw number classification categories.

Specifically, the number of flaws is 80, and the number of flaws is 1,2,3, 80. Zi is the activation value corresponding to the i-th type flaw number classification category, for example, Z1 is the activation value corresponding to the 1-th type flaw number classification category (i.e. the activation value corresponding to 1 flaw number), n is the total number of flaw number classification categories, i.e. 80, and finally Z1 is the activation value corresponding to the 1-th type flaw number classification category and n is 80, substituted intoThe obtained P1, P1 is the probability that the image to be detected is classified into the first type of flaw number classification category (i.e. the flaw number of the image to be detected is 1). And the probability that the image to be detected is classified into each flaw number classification category can be obtained by the similar method.

Since the probability calculation provides a quantification method for flaw number classification of the image to be detected. In practical application, the method can be used for accurately determining the flaw number classification of the image to be detected, finely dividing the flaw number range and providing more accurate information for subsequent processing and analysis.

In one embodiment, for one of a plurality of flaw level classification categories, a second weight value and a second bias value corresponding to the flaw level classification category being targeted are obtained; multiplying the flaw grade characteristic vector by a second weight value to obtain a second target value; adding the second target value and the second bias value to obtain an activation value corresponding to the aimed flaw level classification category; and obtaining the probability distribution feature vector of the flaw grade according to the activation value corresponding to each flaw grade classification category.

Specifically, for one of a plurality of flaw class classification categories, a second weight value and a second bias value corresponding to the flaw class classification category for which is aimed are obtained. This means that a neural network needs to be trained first to learn the characteristics of the different flaw class classification categories. During the training process, the neural network learns the weights and bias values corresponding to each category. And multiplying the flaw grade characteristic vector by the first weight value to obtain a second target value. And adding the second target value and the second bias value to obtain an activation value corresponding to the aimed flaw grade classification category. The second target value is added to the previously learned bias value to obtain an activation value that represents the final score for the fault class classification. And obtaining the probability distribution feature vector of the flaw grade according to the activation value corresponding to each flaw grade classification category. According to the activation value corresponding to each flaw grade classification, the probability distribution of each classification can be calculated. This probability distribution can be used to determine the likelihood of overall rank classification.

In one embodiment, the activation value corresponding to each flaw level classification category is converted into the probability that the image to be detected is classified into each flaw level classification category; and obtaining the probability distribution feature vector of the flaw grade according to the probability of classifying the image to be detected into each flaw grade classification category.

Specifically, the output activation value of the neural network is first converted into a probability corresponding to each flaw class classification category by using a softmax function (activation function). The softmax function converts a set of real numbers into a probability distribution such that the sum of the probabilities for all classes is 1. Thus, the probability corresponding to each flaw grade classification category can be visually represented. And then sequentially forming a feature vector from the probability value of each flaw level classification category, wherein the feature vector represents the probability distribution condition of the image to be detected classified into each flaw level classification category. By observing the feature vector, the probability distribution condition of the neural network for classifying the flaw level of the image to be detected can be intuitively known.

In one embodiment, the conversion of the activation value corresponding to each flaw level classification category into the probability of classifying the image to be detected into each flaw level classification category is achieved by the following calculation formula:

Wherein Xi is the activation value corresponding to the i-th flaw class classification class, and n is the total number of flaw class classification classes.

Specifically, there are 4 flaw classification categories, the flaw classification categories being mild, moderate, severe and very severe. Xi is the activation value corresponding to the i-th flaw class classification category, for example, X1 is the activation value corresponding to the 1-th flaw class classification category (i.e. the activation value corresponding to the flaw class is mild), n is the total number of flaw class classification categories, i.e. 4, and finally, substituting X1 is the activation value corresponding to the 1-th flaw class classification category and n is 4The obtained P1, P1 is the probability that the image to be detected is classified into the first type of flaw class classification class (i.e. the flaw class of the image to be detected is mild). And the probability that the image to be detected is classified into each flaw level classification category can be obtained by the similar method.

Since the probability calculation provides a quantification method for flaw grade classification of the image to be detected. In practical application, the method can be used for accurately determining the flaw grade classification of the image to be detected, finely dividing the grade range of flaws and providing more accurate information for subsequent processing and analysis.

In one embodiment, determining a defect number classification category to which the maximum probability of the image to be detected is classified according to the defect number probability distribution feature vector; taking the flaw number classification category to which the maximum probability of the image to be detected is classified as a flaw number detection result; and determining a flaw level detection result according to the flaw number probability distribution feature vector and the flaw level probability distribution feature vector.

The accuracy and precision of flaws on the target part can be improved by comprehensively utilizing the flaw number detection result and the flaw grade detection result. The defect number classification category and the defect grade classification category with the highest probability are directly selected, so that the influence of subjective factors can be reduced to a certain extent, and the objectivity of the result is improved. When a certain error exists in the flaw level detection result, the accuracy of the flaw level detection result can be improved by converting the flaw number detection result, so that the actual requirement is better met. And the results obtained by a plurality of steps are comprehensively utilized, so that more accurate flaw detection data can be provided.

In one embodiment, according to the feature vector of the probability distribution of the flaw level, determining a flaw level classification class to which the maximum probability of the image to be detected is classified, and taking the flaw level classification class to which the maximum probability of the image to be detected is classified as a flaw level output result of the image to be detected; determining a plurality of flaw levels corresponding to the plurality of flaw level classification categories; determining the probability of classifying the image to be detected into each flaw level according to the probability of classifying the image to be detected into each flaw number classification type; determining a flaw grade corresponding to the maximum probability according to the probability that the image to be detected is classified into each flaw grade, and taking the flaw grade corresponding to the maximum probability as a flaw grade mapping result; and determining a flaw level detection result according to the flaw level output result and the flaw level mapping result.

Specifically, it can be understood that the flaw level detection results in the present application include a flaw level output result and a flaw level map result. The determining process of the flaw grade output result comprises the following steps: because each element in the flaw level probability distribution feature vector represents the probability of being classified into the corresponding flaw level classification category, the flaw level classification category with the highest probability is directly selected from all elements in the flaw level probability distribution feature vector, and the flaw level corresponding to the flaw level classification category is the flaw level output result. The determining process of the flaw grade mapping result comprises the following steps: and mapping and converting the flaw number detection result determined in the previous step into another flaw grade detection result by utilizing the flaw number detection result determined in the previous step, so that two flaw grade detection results are comprehensively utilized, and finally, a more accurate flaw grade detection result is obtained. It is easy to understand that the specific mapping conversion process is not limited, and may be set according to actual needs.

Optionally, firstly, the defect number classification category is mapped to the defect grade classification category (mild, moderate, severe and very severe), and summing operation is performed on each interval corresponding to the defect number classification category. For example, the slightly corresponding flaw number classification class is 1-5 flaws; the flaw number classification category corresponding to the moderate degree is 6-20; the number of the defects corresponding to the severity is classified into 21-50; the very serious corresponding defect number classification category is 51-80 defects. Performing softmax normalization operation on the activation value corresponding to each flaw number classification category to convert the activation value into a flaw number probability distribution feature vector, wherein the sum of the distributions is 1, and finally, taking the flaw grade classification category corresponding to the sum of the largest element as a flaw grade mapping result according to the sum (probability) of the elements corresponding to the four flaw grade classification categories of mild, moderate, severe and very severe in the flaw number probability distribution feature vector

In one embodiment, a sample image set is obtained, and a flaw number real label and a flaw grade real label corresponding to each sample image in the sample image set are obtained; detecting each sample image in the sample image set through a flaw detection model to obtain flaw number output labels, flaw grade mapping labels and flaw grade output labels corresponding to each sample image; determining flaw number detection loss according to the flaw number real labels and the flaw number output labels, determining flaw level output loss according to the flaw level real labels and the flaw level output labels, and determining flaw level mapping loss according to the flaw level real labels and the flaw level mapping labels; performing weighted summation processing on the flaw number detection loss, the flaw grade output loss and the flaw grade mapping loss to obtain a flaw detection total loss; and adjusting model parameters of the flaw detection model through total flaw detection loss until training is stopped when training stopping conditions are reached, and obtaining the trained flaw detection model.

Specifically, for the flaw number real label and flaw level real label corresponding to each sample image in the sample image set, the following manner may be obtained: the method comprises the steps that a person to be detected wears an intelligent mask to collect face images, face images are cut and aligned through a face detection model, sample images are obtained, flaw grades of each sample image are marked (mild/moderate/severe/very serious) to obtain flaw grade real labels, flaw positions are marked through a target detection marking tool, and quantity statistics is carried out on marked sample images to generate flaw quantity real labels.

In one embodiment, determining the defect number detection loss according to the defect number real tag and the defect number output tag is achieved by the following calculation formula:

wherein Loss1 is flaw number detection Loss, zi is flaw number real label corresponding to the input ith sample image, cj is flaw number output label obtained after the input ith sample image is processed by the flaw detection model, Pi is the probability of classifying the input ith sample image into the flaw number classification category corresponding to the flaw number output label, and Z is the total number of flaw number classification categories.

According to the flaw level real label and the flaw level output label, determining flaw level output loss, and realizing by the following calculation formula:

Wherein Loss2 is flaw grade output Loss, qi is a flaw grade real label corresponding to the input ith sample image, yj is a flaw grade output label obtained by processing the input ith sample image by a flaw detection model, Pi is the probability of classifying the input ith sample image into the flaw grade classification category corresponding to the flaw grade output label, and Q is the total number of flaw grade classification categories.

According to the flaw level real label and the flaw level mapping label, determining flaw level mapping loss, and realizing by the following calculation formula:

Wherein Loss3 is flaw grade mapping Loss, qi is a flaw grade real label corresponding to the input ith sample image, tj is a flaw grade mapping label obtained by processing the input ith sample image by a flaw detection model, Pi is the probability of classifying the input ith sample image into the flaw grade classification category corresponding to the flaw grade mapping label, and Q is the total number of flaw grade classification categories.

In one embodiment, the defect number detection loss, the defect level output loss, and the defect level map loss are weighted and summed to obtain a total defect detection loss.

Specifically, the present application does not specifically limit the weight parameters involved in the weighted summation processing, and may be set as needed. Optionally, the total defect detection loss=0.4×defect number detection loss+0.3×defect level mapping loss+0.3×defect level output loss.

In one embodiment, as shown in fig. 7, fig. 7 is a flowchart of an image processing method in another embodiment, including the following steps:

step 702, obtaining an image to be detected; the image to be detected is an image shot aiming at a target part of a target object;

Step 704, performing feature extraction on the image to be detected through a first weight matrix to obtain a flaw number convolution feature map, and performing feature extraction on the image to be detected through a second weight matrix to obtain a flaw level convolution feature map;

Step 706, performing global maximum pooling processing on the flaw number convolution feature images to obtain flaw number pooling feature images, and performing global average pooling processing on the flaw level convolution feature images to obtain flaw level pooling feature images;

step 708, flattening and straightening the flaw number pooling feature map and the flaw grade pooling feature map respectively to obtain flaw number feature vectors and flaw grade feature vectors;

step 710, converting the defect number feature vector into a defect number probability distribution feature vector, and converting the defect level feature vector into a defect level probability distribution feature vector;

Step 712, determining the defect number classification category to which the maximum probability of the image to be detected is classified according to the defect number probability distribution feature vector;

Taking the flaw number classification category to which the maximum probability of the image to be detected is classified as a flaw number detection result;

According to the flaw level probability distribution feature vector, determining a flaw level classification category to which the maximum probability of the image to be detected is classified, and taking the flaw level classification category to which the maximum probability of the image to be detected is classified as a flaw level output result of the image to be detected;

determining a plurality of flaw levels corresponding to the plurality of flaw level classification categories;

determining the probability of classifying the image to be detected into each flaw level according to the probability of classifying the image to be detected into each flaw number classification type;

Determining a flaw grade corresponding to the maximum probability according to the probability that the image to be detected is classified into each flaw grade, and taking the flaw grade corresponding to the maximum probability as a flaw grade mapping result;

And determining a flaw level detection result according to the flaw level output result and the flaw level mapping result.

It should be understood that, although the steps in the flowcharts related to the above embodiments are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides a flaw detection device for realizing the flaw detection method. The implementation of the solution provided by the device is similar to that described in the above method, so the specific limitation of one or more embodiments of the flaw detection device provided below may be referred to above for limitation of the flaw detection method, and will not be repeated here.

In one embodiment, as shown in fig. 8, there is provided a flaw detection apparatus 800 including: an acquisition module 802, a feature extraction module 804, a pooling module 806, a flattening and straightening module 808, and a conversion module 810, and a determination module 812, wherein:

an acquisition module 802, configured to acquire an image to be detected; the image to be detected is an image shot aiming at a target part of a target object;

the feature extraction module 804 is configured to perform feature extraction on the image to be detected through a first weight matrix to obtain a flaw number convolution feature map, and perform feature extraction on the image to be detected through a second weight matrix to obtain a flaw level convolution feature map;

The pooling processing module 806 is configured to perform global maximum pooling processing on the defect number convolution feature map to obtain a defect number pooling feature map, and perform global average pooling processing on the defect level convolution feature map to obtain a defect level pooling feature map;

The flattening and straightening module 808 is configured to perform flattening and straightening processing on the defect number pooling feature map and the defect level pooling feature map respectively, so as to obtain a defect number feature vector and a defect level feature vector;

the conversion module 810 is configured to convert the defect number feature vector into a defect number probability distribution feature vector, and convert the defect level feature vector into a defect level probability distribution feature vector;

the determining module 812 is configured to determine a flaw detection result of the target portion according to the flaw number probability distribution feature vector and the flaw level probability distribution feature vector.

In one embodiment, the conversion module 804 is configured to obtain, for one of the defect number classification categories, a first weight value and a first bias value corresponding to the targeted defect number classification category; multiplying the flaw number feature vector by a first weight value to obtain a first target value; adding the first target value and the first bias value to obtain an activation value corresponding to the aimed flaw number classification category; and obtaining the probability distribution feature vector of the flaw number according to the activation value corresponding to each flaw number classification category.

In one embodiment, the conversion module 804 is configured to convert the activation value corresponding to each of the defect number classification categories into a probability that the image to be detected is classified into each of the defect number classification categories; and obtaining the probability distribution feature vector of the flaw number according to the probability that the image to be detected is classified into each flaw number classification category.

In one embodiment, the conversion module 804 is configured to convert the activation value corresponding to each flaw number classification category into a probability that the image to be detected is classified into each flaw number classification category, which is implemented by the following calculation formula:

In one embodiment, the conversion module 804 is configured to obtain, for one of the plurality of defect level classification categories, a second weight value and a second bias value corresponding to the defect level classification category to which the defect level classification category is directed; multiplying the flaw grade characteristic vector by a second weight value to obtain a second target value; adding the second target value and the second bias value to obtain an activation value corresponding to the aimed flaw level classification category; and obtaining the probability distribution feature vector of the flaw grade according to the activation value corresponding to each flaw grade classification category.

In one embodiment, the conversion module 804 is configured to convert the activation value corresponding to each of the defect level classification categories into a probability that the image to be detected is classified into each of the defect level classification categories; and obtaining the probability distribution feature vector of the flaw grade according to the probability of classifying the image to be detected into each flaw grade classification category.

In one embodiment, the conversion module 804 is configured to convert the activation value corresponding to each flaw level classification category into a probability that the image to be detected is classified into each flaw level classification category, which is implemented by the following calculation formula:

In one embodiment, the determining module 812 is configured to determine, according to the feature vector of the probability distribution of the number of flaws, a flaw number classification category to which the maximum probability of the image to be detected is classified; taking the flaw number classification category to which the maximum probability of the image to be detected is classified as a flaw number detection result; and determining a flaw level detection result according to the flaw number probability distribution feature vector and the flaw level probability distribution feature vector.

In one embodiment, the determining module 812 is configured to determine, according to the feature vector of the probability distribution of the flaw level, a flaw level classification class to which the maximum probability of the image to be detected is classified, and take the flaw level classification class to which the maximum probability of the image to be detected is classified as the flaw level output result of the image to be detected; determining a plurality of flaw levels corresponding to the plurality of flaw level classification categories; determining the probability of classifying the image to be detected into each flaw level according to the probability of classifying the image to be detected into each flaw number classification type; determining a flaw grade corresponding to the maximum probability according to the probability that the image to be detected is classified into each flaw grade, and taking the flaw grade corresponding to the maximum probability as a flaw grade mapping result; and determining a flaw level detection result according to the flaw level output result and the flaw level mapping result.

In one embodiment, the flaw detection apparatus 800 further includes a training module 814, configured to obtain a sample image set, and a flaw number real tag and a flaw level real tag corresponding to each sample image in the sample image set; detecting each sample image in the sample image set through a flaw detection model to obtain flaw number output labels, flaw grade mapping labels and flaw grade output labels corresponding to each sample image; determining flaw number detection loss according to the flaw number real labels and the flaw number output labels, determining flaw level output loss according to the flaw level real labels and the flaw level output labels, and determining flaw level mapping loss according to the flaw level real labels and the flaw level mapping labels; performing weighted summation processing on the flaw number detection loss, the flaw grade output loss and the flaw grade mapping loss to obtain a flaw detection total loss; and adjusting model parameters of the flaw detection model through total flaw detection loss until training is stopped when training stopping conditions are reached, and obtaining the trained flaw detection model.

In one embodiment, the training module 814 is further configured to determine the defect number detection loss according to the defect number real tag and the defect number output tag, which is implemented by the following calculation formula:

In one embodiment, the training module 814 is further configured to determine the flaw level output loss according to the flaw level real tag and the flaw level output tag, which is implemented by the following calculation formula:

In one embodiment, the training module 814 is further configured to determine a defect level mapping loss according to the defect level real label and the defect level mapping label, which is implemented by the following calculation formula:

In another embodiment, as shown in fig. 9, fig. 9 is a block diagram of a flaw detection apparatus 800 according to an embodiment, including: acquisition module 802, feature extraction module 804, pooling processing module 806, flattening and straightening module 808, conversion module 810, and determination module 812, further include training module 814.

The respective modules in the flaw detection apparatus described above may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 10. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing data related to flaw detection. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a flaw detection method.

It will be appreciated by those skilled in the art that the structure shown in FIG. 10 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, carries out the steps of the method embodiments described above.

It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region.

Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magneto-resistive random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (PHASE CHANGE Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as Static Random access memory (Static Random access memory AccessMemory, SRAM) or dynamic Random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The foregoing examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.

Claims

1. A method of flaw detection, the method comprising:

Acquiring an image to be detected and a flaw detection model; the image to be detected is an image shot aiming at a target part of a target object, the flaw detection model comprises a main network, a flaw number branch module and a flaw grade branch module, and the flaw number branch module comprises a first input layer, a first convolution layer, a batch normalization module, a generalized rectification linear unit, a first residual error module, a global maximum pooling module and an acne counting classification layer; the flaw grade branching module comprises a second input layer, a second convolution layer, a second residual error module, a global average pooling module, a third residual error module and an acne grade classification layer;

acquiring a first weight matrix and a second weight matrix;

Extracting features of the image to be detected through the backbone network to obtain an initial feature map;

Performing linear transformation and nonlinear transformation corresponding to the first weight matrix on the initial feature map through the flaw number branch module to obtain a flaw number convolution feature map, and determining a flaw number pooling feature map according to the maximum feature value of the flaw number convolution feature map on each feature channel;

Performing linear transformation and nonlinear transformation corresponding to the second weight matrix on the initial feature map through the flaw grade branching module to obtain a flaw grade convolution feature map, and determining a flaw grade pooling feature map according to average feature values of the flaw grade convolution feature map on each feature channel;

2. The method of claim 1, wherein said converting said defect number feature vector into a defect number probability distribution feature vector comprises:

for one of a plurality of flaw number classification categories, acquiring a first weight value and a first bias value corresponding to the flaw number classification category to which the flaw number classification category is directed;

Multiplying the flaw number feature vector by the first weight value to obtain a first target value;

Adding the first target value and the first bias value to obtain an activation value corresponding to the aimed flaw number classification category;

and obtaining the probability distribution feature vector of the flaw number according to the activation value corresponding to each flaw number classification category.

3. The method according to claim 2, wherein the obtaining the feature vector of the probability distribution of the number of flaws according to the respective activation value of each of the classification categories of the number of flaws comprises:

converting the activation value corresponding to each flaw number classification category into the probability that the image to be detected is classified into each flaw number classification category;

and obtaining the probability distribution feature vector of the flaw number according to the probability that the image to be detected is classified into each flaw number classification category.

4. A method according to claim 3, wherein the conversion of the respective activation value of each flaw number classification category into the probability of classifying the image to be detected into each flaw number classification category is achieved by the following calculation formula:

；

5. The method of claim 1, wherein said converting said flaw level feature vector into a flaw level probability distribution feature vector comprises:

For one of a plurality of flaw class classification categories, acquiring a second weight value and a second bias value corresponding to the flaw class classification category to which the flaw class classification category is directed;

Multiplying the flaw grade characteristic vector by the second weight value to obtain a second target value;

adding the second target value and the second bias value to obtain an activation value corresponding to the aimed flaw grade classification category;

And obtaining the probability distribution feature vector of the flaw grade according to the activation value corresponding to each flaw grade classification category.

6. The method of claim 5, wherein the obtaining the feature vector of the probability distribution of the defect level according to the respective activation value of each defect level classification category comprises:

Converting the activation value corresponding to each flaw grade classification category into the probability that the image to be detected is classified into each flaw grade classification category;

And obtaining the probability distribution feature vector of the flaw grade according to the probability of classifying the image to be detected into each flaw grade classification category.

7. The method of claim 6, wherein the converting the activation value corresponding to each of the defect level classification categories into the probability that the image to be detected is classified into each of the defect level classification categories is performed by the following calculation formula:

；

8. The method of claim 1, wherein the flaw detection results include a flaw number detection result and a flaw level detection result; determining a flaw detection result of the image to be detected according to the flaw number probability distribution feature vector and the flaw level probability distribution feature vector, including:

determining the flaw number classification category to which the maximum probability of the image to be detected is classified according to the flaw number probability distribution feature vector;

taking the flaw number classification category to which the maximum probability of the image to be detected is classified as a flaw number detection result; and determining a flaw grade detection result according to the flaw number probability distribution feature vector and the flaw grade probability distribution feature vector.

9. The method of claim 8, wherein determining a defect level detection result from the defect number probability distribution feature vector and the defect level probability distribution feature vector comprises:

determining a flaw grade classification category to which the maximum probability of the image to be detected is classified according to the flaw grade probability distribution feature vector, and taking the flaw grade classification category to which the maximum probability of the image to be detected is classified as a flaw grade output result of the image to be detected;

Determining the probability of classifying the image to be detected into each flaw level according to the probability of classifying the image to be detected into each flaw number classification category;

determining a flaw level corresponding to the maximum probability according to the probability that the image to be detected is classified into each flaw level, and taking the flaw level corresponding to the maximum probability as a flaw level mapping result;

10. The method according to any one of claims 1 to 9, wherein the flaw level detection result of the target portion is output by a flaw detection model;

the training step of the flaw detection model comprises the following steps:

Acquiring a sample image set, wherein each sample image in the sample image set corresponds to a flaw number real label and a flaw grade real label;

Detecting each sample image in the sample image set through the flaw detection model to obtain flaw number output labels, flaw grade mapping labels and flaw grade output labels corresponding to each sample image;

determining flaw number detection loss according to the flaw number real labels and the flaw number output labels, determining flaw grade output loss according to the flaw grade real labels and the flaw grade output labels, and determining flaw grade mapping loss according to the flaw grade real labels and the flaw grade mapping labels;

Performing weighted summation processing on the flaw number detection loss, the flaw grade output loss and the flaw grade mapping loss to obtain flaw detection total loss;

And adjusting model parameters of the flaw detection model through the total flaw detection loss until training is stopped when the training stopping condition is reached, so as to obtain a trained flaw detection model.

11. The method of claim 10, wherein determining the defect number detection loss from the defect number genuine tag and the defect number output tag is achieved by the following calculation formula:

；

Wherein Xi is an activation value corresponding to the class classification category of the ith flaw, loss1 is flaw number detection Loss, mi is a flaw number real label corresponding to the input ith sample image, cj is a flaw number output label obtained after the input ith sample image is processed by a flaw detection model, Pi is the probability of classifying the input ith sample image into the flaw number classification category corresponding to the flaw number output label, and Z is the total number of flaw number classification categories.

12. The method of claim 10, wherein determining a defect level output loss from the defect level genuine label and the defect level output label is accomplished by a calculation formula:

；

Wherein Xi is an activation value corresponding to the i-th type flaw level classification category, loss2 is flaw level output Loss, qi is a flaw level real label corresponding to the input i-th sample image, yj is a flaw level output label obtained after the input i-th sample image is processed by a flaw detection model, Pi is the probability of classifying the input ith sample image into the flaw grade classification category corresponding to the flaw grade output label, and Q is the total number of flaw grade classification categories.

13. The method of claim 10, wherein determining a defect level mapping loss from the defect level genuine label and the defect level mapped label is accomplished by a calculation formula:

；

Wherein Xi is an activation value corresponding to the class i flaw level classification class, loss3 is flaw level mapping Loss, qi is a flaw level real label corresponding to the input ith sample image, tj is a flaw level mapping label obtained after the input ith sample image is processed by a flaw detection model, Pi is the probability of classifying the input ith sample image into the flaw grade classification category corresponding to the flaw grade mapping label, and Q is the total number of flaw grade classification categories.

14. A flaw detection device, the device comprising:

The acquisition module is used for acquiring the image to be detected and the flaw detection model; the image to be detected is an image shot aiming at a target part of a target object, the flaw detection model comprises a main network, a flaw number branch module and a flaw grade branch module, and the flaw number branch module comprises a first input layer, a first convolution layer, a batch normalization module, a generalized rectification linear unit, a first residual error module, a global maximum pooling module and an acne counting classification layer; the flaw grade branching module comprises a second input layer, a second convolution layer, a second residual error module, a global average pooling module, a third residual error module and an acne grade classification layer;

the feature extraction module is used for acquiring a first weight matrix and a second weight matrix;

Performing linear transformation and nonlinear transformation corresponding to the first weight matrix on the initial feature map through the flaw number branching module to obtain a flaw number convolution feature map, and performing linear transformation and nonlinear transformation corresponding to the second weight matrix on the initial feature map to obtain a flaw level convolution feature map;

The pooling processing module is used for determining a flaw number pooling feature map according to the maximum feature value of the flaw number convolution feature map on each feature channel through the flaw level branching module, and determining a flaw level pooling feature map according to the average feature value of the flaw level convolution feature map on each feature channel;

15. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 13 when the computer program is executed.

16. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 13.