CN113239878B - Image classification method, device, equipment and medium - Google Patents
Image classification method, device, equipment and medium Download PDFInfo
- Publication number
- CN113239878B CN113239878B CN202110611214.2A CN202110611214A CN113239878B CN 113239878 B CN113239878 B CN 113239878B CN 202110611214 A CN202110611214 A CN 202110611214A CN 113239878 B CN113239878 B CN 113239878B
- Authority
- CN
- China
- Prior art keywords
- image
- loss value
- classification
- decoupling
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an image classification method, a device, equipment and a medium, wherein the image classification method is characterized in that an initial normal image and an initial defect image are input into a preset classification model containing initial parameters, so that decoupling recovery processing is carried out on the initial normal image and the initial defect image through the preset classification model, and an image recovery loss value, an image decoupling loss value, an image classification loss value and a characteristic distinguishing loss value are determined; determining a total loss value of a preset classification model according to the image restoration loss value, the image decoupling loss value, the image classification loss value and the characteristic distinguishing loss value; and when the total loss value does not reach the preset convergence condition, updating initial parameters of the iterative preset classification model, and recording the converged preset classification model as an image classification model until the total loss value reaches the preset convergence condition. The invention improves the accuracy of image classification.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image classification method, apparatus, device, and medium.
Background
With the development of scientific technology, the more computer vision technology is applied to various fields to achieve technical effects such as image classification, but there is an unbalanced image classification problem with only fine differences in image classification tasks, since the differences of fine features are extremely difficult to distinguish, and since there are few sample images of fine features only; in the prior art, images are generally classified through classification models such as a deep learning model, but the deep learning model needs a large amount of labeling data for training, but sample images with fine features are fewer, labeling of the fine features is often carried out by means of advanced doctors, labeling standards of each doctor are not uniform, and further the image classification difficulty based on deep learning is high, so that the image classification accuracy is low.
Disclosure of Invention
The embodiment of the invention provides an image classification method, device, equipment and medium, which are used for solving the problem of low image classification accuracy.
An image classification method, comprising:
receiving an image classification instruction containing an image to be classified;
inputting the image to be classified into an image classification model to carry out image classification on the image to be classified through the image classification model to obtain an image classification result; the image classification model is obtained by training according to an image classification model training method; the image classification model training method comprises the following steps:
Acquiring a preset image sample set; the preset image sample set comprises at least one sample classification image group; one of the sample classified image groups includes an initial normal image and an initial defect image;
inputting the initial normal image and the initial defect image into a preset classification model containing initial parameters, so as to perform decoupling recovery processing on the initial normal image and the initial defect image through the preset classification model, and determining an image recovery loss value, an image decoupling loss value, an image classification loss value and a characteristic distinguishing loss value of the preset classification model in the decoupling recovery processing process;
determining a total loss value of the preset classification model according to the image restoration loss value, the image decoupling loss value, the image classification loss value and the characteristic distinguishing loss value;
and updating and iterating initial parameters of the preset classification model when the total loss value does not reach a preset convergence condition, and recording the preset classification model after convergence as the image classification model when the total loss value reaches the preset convergence condition.
An image classification apparatus comprising:
the instruction receiving module is used for receiving an image classification instruction containing an image to be classified;
The image classification module is used for inputting the images to be classified into an image classification model so as to carry out image classification on the images to be classified through the image classification model to obtain an image classification result; the image classification model is obtained by training according to an image classification model training method; the image classification model training method comprises the following steps:
acquiring a preset image sample set; the preset image sample set comprises at least one sample classification image group; one of the sample classified image groups includes an initial normal image and an initial defect image;
inputting the initial normal image and the initial defect image into a preset classification model containing initial parameters, so as to perform decoupling recovery processing on the initial normal image and the initial defect image through the preset classification model, and determining an image recovery loss value, an image decoupling loss value, an image classification loss value and a characteristic distinguishing loss value of the preset classification model in the decoupling recovery processing process;
determining a total loss value of the preset classification model according to the image restoration loss value, the image decoupling loss value, the image classification loss value and the characteristic distinguishing loss value;
And updating and iterating initial parameters of the preset classification model when the total loss value does not reach a preset convergence condition, and recording the preset classification model after convergence as the image classification model when the total loss value reaches the preset convergence condition.
A computer device comprising a memory, a processor and computer readable instructions stored in the memory and executable on the processor, the processor implementing the image classification method described above when executing the computer readable instructions.
A computer readable storage medium storing computer readable instructions which when executed by a processor implement the above-described image classification method.
The image classification method, the device, the equipment and the medium are characterized in that the image classification method receives an image classification instruction containing an image to be classified; inputting the image to be classified into an image classification model to carry out image classification on the image to be classified through the image classification model to obtain an image classification result; the image classification model is obtained by training according to an image classification model training method; the image classification model training method comprises the following steps: acquiring a preset image sample set; the preset image sample set comprises at least one sample classification image group; one of the sample classified image groups includes an initial normal image and an initial defect image; inputting the initial normal image and the initial defect image into a preset classification model containing initial parameters, so as to perform decoupling recovery processing on the initial normal image and the initial defect image through the preset classification model, and determining an image recovery loss value, an image decoupling loss value, an image classification loss value and a characteristic distinguishing loss value of the preset classification model in the decoupling recovery processing process; determining a total loss value of the preset classification model according to the image restoration loss value, the image decoupling loss value, the image classification loss value and the characteristic distinguishing loss value; and updating and iterating initial parameters of the preset classification model when the total loss value does not reach a preset convergence condition, and recording the preset classification model after convergence as an image classification model when the total loss value reaches the preset convergence condition.
Under the condition that the quantity of the initial normal images and the quantity of the initial defect images are extremely unbalanced (namely, under the condition that the initial defect images are extremely absent), the image classification model is obtained through training under the combined action of the decoupling recovery processing method and four groups of loss values (namely, the image recovery loss value, the image decoupling loss value, the image classification loss value and the characteristic distinguishing loss value), so that the accuracy of the image classification model for classifying the images to be classified of the fine characteristics is higher.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic view of an application environment of an image classification method according to an embodiment of the invention;
FIG. 2 is a flow chart of a method of classifying images according to an embodiment of the present invention;
FIG. 3 is a flowchart of an image classification model training method in an image classification method according to an embodiment of the invention;
FIG. 4 is a flowchart of step S02 in the image classification method according to an embodiment of the present invention;
FIG. 5 is a schematic block diagram of an image classification apparatus according to an embodiment of the present invention;
FIG. 6 is a schematic block diagram of an image classification module in an image classification apparatus according to an embodiment of the invention;
FIG. 7 is a schematic diagram of a computer device in accordance with an embodiment of the invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The image classification method provided by the embodiment of the invention can be applied to an application environment shown in fig. 1. Specifically, the image classification method is applied to an image classification system, the image classification system comprises a client and a server as shown in fig. 1, and the client and the server communicate through a network to solve the problem of low image classification accuracy. The client is also called a client, and refers to a program corresponding to the server for providing local service for the client. The client may be installed on, but is not limited to, various personal computers, notebook computers, smartphones, tablet computers, and portable wearable devices. The server may be implemented as a stand-alone server or as a server cluster composed of a plurality of servers.
In one embodiment, as shown in fig. 2, an image classification method is provided, and the method is applied to the server in fig. 1, and the method includes the following steps:
s10: receiving an image classification instruction containing an image to be classified;
it will be appreciated that the image classification instructions may be sent by the user via the mobile terminal or may be automatically generated after the user transmits the image to be classified. The image to be classified refers to an image to be classified, in this embodiment, the image to be classified is an image including defect features, for example, in an automobile detection scene, the image to be classified may be an image of an automobile device with damage or defect; still alternatively, in a medical scenario, the image to be classified may be a face image containing a disorder.
S20: inputting the image to be classified into an image classification model to carry out image classification on the image to be classified through the image classification model to obtain an image classification result; the image classification model is obtained by training according to an image classification model training method;
specifically, after receiving an image classification instruction containing an image to be classified, inputting the image to be classified into an image classification model, and further performing image classification on the image to be classified through the image classification model to obtain an image classification result.
As shown in fig. 3, the image classification model training method includes the following steps:
s01: acquiring a preset image sample set; the preset image sample set comprises at least one sample classification image group; one of the sample classified image groups includes an initial normal image and an initial defect image;
it can be understood that the preset image sample set can be a set of different sample images collected under different scenes, so that the different sample images form a plurality of sample classification image groups; wherein a sample classification image group comprises at least one initial normal image and at least one initial defect image; for example, in a medical scenario, the initial normal image may be a normal face image and the initial defect image may be a face image containing a disorder; in the automobile detection scene, the initial normal image may be a normal automobile equipment image, and the initial defect image may be an automobile equipment image with a damage or defect. Further, in the present embodiment, the number of initial defect images is far smaller than the number of initial normal images, and thus the initial defect images in the image groups may be the same for different samples.
S02: inputting the initial normal image and the initial defect image into a preset classification model containing initial parameters, so as to perform decoupling recovery processing on the initial normal image and the initial defect image through the preset classification model, and determining an image recovery loss value, an image decoupling loss value, an image classification loss value and a characteristic distinguishing loss value of the preset classification model in the decoupling recovery processing process;
it may be appreciated that the preset classification model may be a model constructed based on a deep learning technique, for example, the preset classification model may be a model capable of classifying images with large differences or more obvious differences in the prior art. The decoupling recovery processing comprises image decoupling processing and image recovery processing; the image decoupling processing refers to decomposing an image into two images (i.e., decomposing one image into one base image and one feature image in this embodiment), and the loss value generated in the image decoupling processing is the image decoupling loss value; the image restoration process refers to an operation of restoring the two decomposed images to the original image, and a loss value generated in the image restoration process is an image restoration loss value (for example, the two images may not be completely restored to the original image). Further, the preset classification model further comprises image classification of the initial normal image, the initial defect image or the image generated in the decoupling recovery process, and the loss value generated in the image classification process is the image classification loss value. The feature discrimination loss value refers to the feature discrimination between feature images (a first feature image, a second feature image, a third feature image, and a fourth feature image as indicated in the following description) generated during the decoupling recovery processing.
S03: determining a total loss value of the preset classification model according to the image restoration loss value, the image decoupling loss value, the image classification loss value and the characteristic distinguishing loss value;
specifically, after determining an image restoration loss value, an image decoupling loss value, an image classification loss value, and a feature discrimination loss value of a preset classification model in the decoupling restoration process, the sum of the image restoration loss value, the image decoupling loss value, the image classification loss value, and the feature discrimination loss value is recorded as a total loss value.
Further, the total loss value may be determined according to the following expression:
L＝λ 1 L rec +λ 2 L dec +λ 3 L cls +λ 4 L dis
wherein L is the total loss value; l (L) rec Restoring a loss value for the image; lambda (lambda) 1 A loss weight corresponding to the image restoration loss value; l (L) dec Decoupling the loss value for the image; lambda (lambda) 2 Loss weights corresponding to image decoupling loss values; l (L) cls Classifying a loss value for the image; lambda (lambda) 3 Loss weights corresponding to image decoupling loss values; l (L) dis Distinguishing a loss value for the feature; lambda (lambda) 4 Loss weights corresponding to the feature discrimination loss values. Lambda (lambda) 1 、λ 2 、λ 3 And lambda is 4 The sum is 1.
S04: and updating and iterating initial parameters of the preset classification model when the total loss value does not reach a preset convergence condition, and recording the preset classification model after convergence as an image classification model when the total loss value reaches the preset convergence condition.
It is to be understood that the convergence condition may be a condition that the total loss value is smaller than the set threshold, that is, training is stopped when the total loss value is smaller than the set threshold; the convergence condition may be a condition that the total loss value is small after 10000 times of calculation and does not drop any more, that is, when the total loss value is small after 10000 times of calculation and does not drop, training is stopped, and the preset classification model after convergence is recorded as an image classification model.
Further, after determining the total loss value of the preset classification model according to the image recovery loss value, the image decoupling loss value and the image classification loss value, when the total loss value does not reach the preset convergence condition, adjusting the initial parameters of the preset classification model according to the total loss value, and re-inputting the sample classification image group into the preset classification model after the initial parameters are adjusted, so that when the total loss value corresponding to the sample classification image group reaches the preset convergence condition, another sample classification image group in the preset image sample group is selected, the steps are executed, the total loss value corresponding to the sample classification image group is obtained, and when the total loss value does not reach the preset convergence condition, the initial parameters of the preset classification model are adjusted again according to the total loss value, so that the total loss value corresponding to the sample classification image group reaches the preset convergence condition.
Therefore, after training the preset classification model through all sample classification image groups in the preset image sample set, the result output by the preset classification model can be continuously and accurately drawn close to the result, the recognition accuracy is higher and higher, and the preset classification model after convergence is recorded as the image classification model until the total loss value corresponding to all sample classification image groups reaches the preset convergence condition.
In this embodiment, under the condition that the number of the initial normal image and the number of the initial defect image are extremely unbalanced (i.e., under the condition that the initial defect image is extremely absent), an image classification model is obtained through training under the combined action of a decoupling recovery processing method and four groups of loss values (i.e., an image recovery loss value, an image decoupling loss value, an image classification loss value and a feature discrimination loss value), so that the accuracy of performing image classification on the image with fine features by the image classification model is higher.
In an embodiment, as shown in fig. 4, in step S02, that is, in the decoupling recovery processing of the initial normal image and the initial defect image by the preset classification model, the method includes:
S21: performing image decoupling processing on the initial normal image to obtain a first basic image and a first characteristic image; meanwhile, performing image decoupling processing on the initial defect image to obtain a second basic image and a second characteristic image;
it can be understood that, assuming that the application scenario of the embodiment is a medical scenario, the initial normal image is a normal face image, and the initial defect image is a defect face image including a disease, in this embodiment, after performing image decoupling processing on the initial normal image (for example, performing downsampling processing on the initial normal image through a preset convolution kernel group, etc.), the obtained first base image is a base face image that does not include disease features, the first feature image is a disease feature face image that includes disease features, and, illustratively, after performing image decoupling processing on the initial normal image, a feature map with a size of (m+1) xk×k is obtained, where m×k×k is the first feature image, and 1×k×k is the first base image. And similarly, after the image decoupling processing is carried out on the initial defect image, the obtained second basic image is the basic face image which does not contain disease features, and the second feature image is the disease feature face image which contains the disease features. It should be noted that, the original source of the initial normal image and the initial defect image are different, for example, in a medical scene, the initial normal image and the initial defect image are face images from different individuals; in an automotive inspection scenario, the initial normal image and the initial defect image are device images from the same device of different vehicles.
S22: performing image restoration processing on the first basic image and the first characteristic image to obtain a restored normal image; performing image restoration processing on the second basic image and the second characteristic image to obtain a restoration defect image;
it can be understood that after the image decoupling process is performed on the initial normal image and the initial defect image, whether the image decoupling process in the preset classification model is accurate needs to be verified, so that the verification can be performed by an image restoration process method, that is, the image restoration process is performed on the first base image and the first feature image, for example, in the above description, mxkxk is the first feature image, 1 xkxk is the first base image, and further, by performing the upsampling process on the first base image and the first feature image through the preset convolution kernel group, a feature image of (m+1) xkxk can be obtained, at this time, the feature image is the restoration normal image, wherein a restoration loss value exists between the restoration normal image and the initial normal image, that is, the restoration normal image can be transformed into the initial normal image through a restoration loss function; similarly, the up-sampling process is performed on the second base image and the second feature image through the preset convolution kernel group, that is, after the image recovery process, a feature map of (m+1) x K may be obtained, which is the recovery defect image.
S23: generating a transformation defect image according to the first basic image and the second characteristic image; generating a transformation normal image according to the first characteristic image and the second basic image;
it will be appreciated that, assuming that in a medical scene, the initial normal image is a normal face image, and further the first base image is a face image that does not contain disease features; the initial defect image is a face image containing disease features, and the second feature image is a face image containing disease features, so that a new image containing disease features can be generated according to the first basic image and the second feature image, namely, the defect image is transformed, and data for training a preset classification model can be expanded. Similarly, the initial normal image is a normal face image, and the first feature image is a face image containing disease features, but at the moment, the initial normal image is a normal face image, so that the first feature image also does not contain disease features; the initial defect image is a face image containing disease features, and the second basic image is a face image not containing disease features, so that a new image not containing disease features can be generated according to the first feature image and the second basic image, namely, a normal image is transformed. Therefore, the problem that the number of initial defect images is extremely short can be solved, the preset image sample set can be subjected to data expansion through the generated transformation defect images and the transformation normal images, and the accuracy of training the preset classification model is improved.
S24: performing image decoupling processing on the transformed normal image to obtain a third basic image and a third characteristic image; and performing image decoupling processing on the transformation defect image to obtain a fourth basic image and a fourth characteristic image.
It can be understood that, assuming that the application scenario of the present embodiment is a medical scenario, a normal image is transformed into a normal face image, and a defective image is transformed into a defective face image including a disease, in this embodiment, after performing image decoupling processing on the transformed normal image (for example, performing downsampling processing on the transformed normal image through a preset convolution kernel group, etc.), the obtained third base image is a base face image that does not include disease features, and the third feature image is a disease feature face image that includes disease features, and, illustratively, after performing image decoupling processing on the transformed normal image, a feature map with a size of (m+1) xk×k is obtained, where m×k×k is the third feature image, and 1×k×k is the third base image. And similarly, after image decoupling processing is performed on the transformed defect image, the obtained fourth basic image is a basic face image which does not contain disease features, and the fourth feature image is a disease feature face image which contains disease features.
In an embodiment, in step S02, that is, the determining the image restoration loss value of the preset classification model during the decoupling restoration process includes:
determining a normal recovery loss value according to the initial normal image and the recovery normal image; simultaneously determining a defect recovery loss value according to the initial defect image and the recovery defect image;
and determining the image recovery loss value according to the recovery loss value and the defect recovery loss value.
Specifically, performing image decoupling processing on the initial normal image to obtain a first basic image and a first characteristic image; meanwhile, performing image decoupling processing on the initial defect image to obtain a second basic image and a second characteristic image, and determining a normal recovery loss value according to the initial normal image and the recovery normal image through a cross entropy loss function, a Perceptual loss function and the like; and determining a defect recovery loss value according to the initial defect image and the recovery defect image, and further determining the image recovery loss value by the sum of the recovery loss value and the defect recovery loss value.
Further, the image restoration loss value may be determined according to the following expression:
L rec ＝|X A -X A-rec | 2 +X B -X B-rec | 2
wherein L is rec Restoring a loss value for the image; x is X A Is an initial defect image; x is X A-rec To restore the defective image; i X A -X A-rec | 2 Recovering a loss value for the defect; x is X B Is an initial normal image; x is X B-rec To restore normal images; i X B -X B-rec | 2 The loss value is recovered normally.
In an embodiment, the determining the image decoupling loss value of the preset classification model in the decoupling recovery process includes:
determining a first base decoupling loss value according to the first base image and the fourth base image; simultaneously determining a second base decoupling loss value according to the second base image and the third base image;
determining a first symptom decoupling loss value according to the first feature image and the third feature image; determining a second symptom decoupling loss value according to the second feature image and the fourth feature image;
and recording the sum of the first basic decoupling loss value, the second basic decoupling loss value, the first symptom decoupling loss value and the second symptom decoupling loss value as the image decoupling loss value.
It can be understood that, in this embodiment, the decoupling loss values generated by the image decoupling process in the above steps are determined, so that the image decoupling process is performed on the initial normal image to obtain a first base image and a first feature image; meanwhile, performing image decoupling processing on the initial defect image to obtain a second basic image and a second characteristic image, and performing image decoupling processing on the transformation normal image to obtain a third basic image and a third characteristic image; after image decoupling processing is carried out on the transformation defect image to obtain a fourth basic image and a fourth characteristic image, a first basic decoupling loss value is determined according to the first basic image and the fourth basic image through a cross entropy loss function, a Perceptual loss function and the like; simultaneously determining a second base decoupling loss value according to the second base image and the third base image; determining a first symptom decoupling loss value according to the first feature image and the third feature image; and determining a second symptom decoupling loss value according to the second characteristic image and the fourth characteristic image, and further recording the sum of the first basic decoupling loss value, the second basic decoupling loss value, the first symptom decoupling loss value and the second symptom decoupling loss value as the image decoupling loss value.
Further, the image decoupling loss value may be determined according to the following expression:
L dec-b ＝|X B-b -X C-b | 2 +|X A-b -X D-b | 2
L dec-f ＝|X A-f -X C-f | 2 +|X B-f -X D-f | 2
L dec ＝L dec-b +L dec-f
wherein L is dec Decoupling the loss value for the image; l (L) dec-b Summing the first base decoupling loss value and the second base decoupling loss value; x is X B-b Is a first base image; x is X C-b Is a fourth base image; i X B-b -X C-b | 2 Decoupling the loss value for the first basis; x is X A-b Is a second base image; x is X D-b Is a third base image; i X A-b -X D-b | 2 Decoupling the loss value for the second basis; l (L) dec-f Is the sum of the first and second symptom decoupling loss values；X A-f Is a second feature image; x is X C-f Is a fourth feature image; i X A-f -X C-f | 2 Decoupling the loss value for the first condition; x is X B-f Is a first feature image; x is X D-f Is a third feature image; i X B-f -X D-f | 2 The loss value is decoupled for the second condition.
In an embodiment, the determining the image classification loss value of the preset classification model in the decoupling recovery process includes:
determining a first classification loss value for performing image classification on the initial normal image and the initial defect image;
it can be understood that the first classification loss value is a classification loss value when the preset classification model performs image classification on the initial normal image and the initial defect image.
Determining a second classification loss value for performing image classification on the restored normal image and the restored defect image;
It can be understood that the second classification loss value is a classification loss value when the preset classification model performs image classification on the restored normal image and the restored defect image.
Determining a third classification loss value for performing image classification on the transformed normal image and the transformed defective image;
it can be understood that the third classification loss value is a classification loss value when the preset classification model performs image classification on the transformed normal image and the transformed defect image.
And recording the sum of the first classification loss value, the second classification loss value and the third classification loss value as the image classification loss value.
Specifically, performing image restoration processing on the first basic image and the first characteristic image to obtain a restored normal image; performing image restoration processing on the second basic image and the second characteristic image to obtain a restored defect image, and generating a transformation defect image according to the first basic image and the second characteristic image; after generating a transformed normal image from the first feature image and the second base image, determining a first classification loss value for performing image classification on the initial normal image and the initial defect image by, for example, a cross entropy loss function, a Perceptual loss function, or the like; determining a second classification loss value for performing image classification on the restored normal image and the restored defect image; and determining a third classification loss value for classifying the transformed normal image and the transformed defect image, and recording the sum of the first classification loss value, the second classification loss value and the third classification loss value as an image classification loss value.
In an embodiment, determining the feature discrimination loss value of the preset classification model in the decoupling recovery process includes:
and determining the feature distinguishing loss value according to the first feature image, the second feature image, the third feature image and the fourth feature image.
Specifically, the feature discrimination loss value may be determined by the following expression:
L dis ＝min(X B-f +m-X A-f ,0)+min(X D-f +m-X C-f ,0)
wherein L is dis Distinguishing a loss value for the feature; x is X B-f Is a first feature image; x is X A-f Is a second feature image; x is X D-f Is a third feature image; x is X C-f Is a fourth feature image; min () is a minimum function; m is used for spacing normal images and defect images, and the first characteristic image is obtained by performing image decoupling processing on an initial normal image; the second feature image is obtained by performing image decoupling processing on the initial defect image, so that the first feature image and the second feature image are separated by m in this embodiment.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.
In one embodiment, an image classification apparatus is provided, which corresponds to the image classification method in the above embodiment one by one. As shown in fig. 5, the image classification apparatus includes an instruction receiving module 10 and an image classification module 20. The functional modules are described in detail as follows:
An instruction receiving module 10 for receiving an image classification instruction containing an image to be classified;
the image classification module 20 is configured to input the image to be classified into an image classification model, so as to perform image classification on the image to be classified through the image classification model, thereby obtaining an image classification result; the image classification model is obtained by training according to an image classification model training method; the image classification model training method comprises the following steps:
acquiring a preset image sample set; the preset image sample set comprises at least one sample classification image group; one of the sample classified image groups includes an initial normal image and an initial defect image;
inputting the initial normal image and the initial defect image into a preset classification model containing initial parameters, so as to perform decoupling recovery processing on the initial normal image and the initial defect image through the preset classification model, and determining an image recovery loss value, an image decoupling loss value, an image classification loss value and a characteristic distinguishing loss value of the preset classification model in the decoupling recovery processing process;
determining a total loss value of the preset classification model according to the image restoration loss value, the image decoupling loss value, the image classification loss value and the characteristic distinguishing loss value;
And updating and iterating initial parameters of the preset classification model when the total loss value does not reach a preset convergence condition, and recording the preset classification model after convergence as the image classification model when the total loss value reaches the preset convergence condition.
Preferably, as shown in fig. 6, the image classification module 20 includes:
a first decoupling processing unit 21, configured to perform decoupling processing on the initial normal image, so as to obtain a first base image and a first feature image; meanwhile, decoupling is carried out on the initial defect image to obtain a second basic image and a second characteristic image;
an image restoration unit 22, configured to perform image restoration on the first base image and the first feature image, so as to obtain a restored normal image; performing image restoration on the second basic image and the second characteristic image to obtain a restored defect image;
an image conversion generation unit 23 for generating a transformation defect image from the first base image and the second feature image; generating a transformation normal image according to the first characteristic image and the second basic image;
a second decoupling processing unit 24, configured to perform decoupling processing on the transformed normal image, so as to obtain a third base image and a third feature image; and decoupling the transformation defect image to obtain a fourth basic image and a fourth characteristic image.
Preferably, the image classification module 20 further comprises:
a recovery loss value determining unit, configured to determine a normal recovery loss value according to the initial normal image and the recovery normal image; simultaneously determining a defect recovery loss value according to the initial defect image and the recovery defect image;
and the image recovery loss value determining unit is used for determining the image recovery loss value according to the recovery loss value and the defect recovery loss value.
Preferably, the image classification module 20 further comprises:
a first decoupling loss value determining unit, configured to determine a first base decoupling loss value according to the first base image and the fourth base image; simultaneously determining a second base decoupling loss value according to the second base image and the third base image;
the second decoupling loss value determining unit is used for determining a first symptom decoupling loss value according to the first characteristic image and the third characteristic image; determining a second symptom decoupling loss value according to the second feature image and the fourth feature image;
and the third decoupling loss value determining unit is used for recording the sum of the first basic decoupling loss value, the second basic decoupling loss value, the first symptom decoupling loss value and the second symptom decoupling loss value as the image decoupling loss value.
Preferably, the image classification module 20 further comprises:
a first classification loss value determining unit configured to determine a first classification loss value for performing image classification on the initial normal image and the initial defect image;
a second classification loss value determining unit configured to determine a second classification loss value for performing image classification on the restored normal image and the restored defective image;
a third classification loss value determining unit configured to determine a third classification loss value for performing image classification on the transformed normal image and the transformed defective image;
an image classification loss value determination unit configured to record, as the image classification loss value, a sum of the first classification loss value, the second classification loss value, and the third classification loss value.
For specific limitations of the image classification apparatus, reference may be made to the above limitations of the image classification method, and no further description is given here. The respective modules in the above-described image classification apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a readable storage medium, an internal memory. The readable storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for the execution of an operating system and computer-readable instructions in a readable storage medium. The database of the computer device is used to store the image classification method in the above embodiment. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer readable instructions when executed by a processor implement a method of image classification. The readable storage medium provided by the present embodiment includes a nonvolatile readable storage medium and a volatile readable storage medium.
In one embodiment, a computer device is provided that includes a memory, a processor, and computer readable instructions stored on the memory and executable on the processor, when executing the computer readable instructions, implementing the image classification method of the above embodiments.
In one embodiment, a computer readable storage medium having computer readable instructions stored thereon which when executed by a processor implement the image classification method of the above embodiments is provided.
Those skilled in the art will appreciate that implementing all or part of the above described embodiment methods may be accomplished by way of computer readable instructions, stored in a non-transitory computer readable storage medium or volatile computer readable storage medium, which when executed, may comprise the above described embodiment methods. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (10)
1. An image classification method, comprising:
receiving an image classification instruction containing an image to be classified;
inputting the image to be classified into an image classification model to carry out image classification on the image to be classified through the image classification model to obtain an image classification result; the image classification model is obtained by training according to an image classification model training method; the image classification model training method comprises the following steps:
Acquiring a preset image sample set; the preset image sample set comprises at least one sample classification image group; one of the sample classified image groups includes an initial normal image and an initial defect image;
inputting the initial normal image and the initial defect image into a preset classification model containing initial parameters, so as to perform decoupling recovery processing on the initial normal image and the initial defect image through the preset classification model, and determining an image recovery loss value, an image decoupling loss value, an image classification loss value and a characteristic distinguishing loss value of the preset classification model in the decoupling recovery processing process;
determining a total loss value of the preset classification model according to the image restoration loss value, the image decoupling loss value, the image classification loss value and the characteristic distinguishing loss value;
and updating and iterating initial parameters of the preset classification model when the total loss value does not reach a preset convergence condition, and recording the preset classification model after convergence as the image classification model when the total loss value reaches the preset convergence condition.
2. The image classification method according to claim 1, wherein the performing the decoupling recovery processing on the initial normal image and the initial defect image by the preset classification model includes:
Performing image decoupling processing on the initial normal image to obtain a first basic image and a first characteristic image; meanwhile, performing image decoupling processing on the initial defect image to obtain a second basic image and a second characteristic image;
performing image restoration processing on the first basic image and the first characteristic image to obtain a restored normal image; performing image restoration processing on the second basic image and the second characteristic image to obtain a restoration defect image;
generating a transformation defect image according to the first basic image and the second characteristic image; generating a transformation normal image according to the first characteristic image and the second basic image;
performing image decoupling processing on the transformed normal image to obtain a third basic image and a third characteristic image; and performing image decoupling processing on the transformation defect image to obtain a fourth basic image and a fourth characteristic image.
3. The image classification method according to claim 2, wherein the determining an image restoration loss value of a preset classification model during the decoupling restoration process includes:
determining a normal recovery loss value according to the initial normal image and the recovery normal image; simultaneously determining a defect recovery loss value according to the initial defect image and the recovery defect image;
And determining the image recovery loss value according to the recovery loss value and the defect recovery loss value.
4. The image classification method according to claim 2, wherein the determining an image decoupling loss value of a preset classification model during the decoupling restoration process includes:
determining a first base decoupling loss value according to the first base image and the fourth base image; simultaneously determining a second base decoupling loss value according to the second base image and the third base image;
determining a first symptom decoupling loss value according to the first feature image and the third feature image; determining a second symptom decoupling loss value according to the second feature image and the fourth feature image;
and recording the sum of the first basic decoupling loss value, the second basic decoupling loss value, the first symptom decoupling loss value and the second symptom decoupling loss value as the image decoupling loss value.
5. The image classification method according to claim 2, wherein the determining an image classification loss value of a preset classification model during the decoupling recovery process includes:
determining a first classification loss value for performing image classification on the initial normal image and the initial defect image;
Determining a second classification loss value for performing image classification on the restored normal image and the restored defect image;
determining a third classification loss value for performing image classification on the transformed normal image and the transformed defective image;
and recording the sum of the first classification loss value, the second classification loss value and the third classification loss value as the image classification loss value.
6. An image classification apparatus, comprising:
the instruction receiving module is used for receiving an image classification instruction containing an image to be classified;
the image classification module is used for inputting the images to be classified into an image classification model so as to carry out image classification on the images to be classified through the image classification model to obtain an image classification result; the image classification model is obtained by training according to an image classification model training method; the image classification model training method comprises the following steps:
acquiring a preset image sample set; the preset image sample set comprises at least one sample classification image group; one of the sample classified image groups includes an initial normal image and an initial defect image;
inputting the initial normal image and the initial defect image into a preset classification model containing initial parameters, so as to perform decoupling recovery processing on the initial normal image and the initial defect image through the preset classification model, and determining an image recovery loss value, an image decoupling loss value, an image classification loss value and a characteristic distinguishing loss value of the preset classification model in the decoupling recovery processing process;
Determining a total loss value of the preset classification model according to the image restoration loss value, the image decoupling loss value, the image classification loss value and the characteristic distinguishing loss value;
and updating and iterating initial parameters of the preset classification model when the total loss value does not reach a preset convergence condition, and recording the preset classification model after convergence as the image classification model when the total loss value reaches the preset convergence condition.
7. The image classification apparatus of claim 6, wherein the image classification module comprises:
the first decoupling processing unit is used for performing decoupling processing on the initial normal image to obtain a first basic image and a first characteristic image; meanwhile, decoupling is carried out on the initial defect image to obtain a second basic image and a second characteristic image;
the image restoration unit is used for carrying out image restoration on the first basic image and the first characteristic image to obtain a restored normal image; performing image restoration on the second basic image and the second characteristic image to obtain a restored defect image;
an image conversion generating unit configured to generate a transformation defect image from the first base image and the second feature image; generating a transformation normal image according to the first characteristic image and the second basic image;
The second decoupling processing unit is used for performing decoupling processing on the transformation normal image to obtain a third basic image and a third characteristic image; and decoupling the transformation defect image to obtain a fourth basic image and a fourth characteristic image.
8. The image classification apparatus of claim 7, wherein the image classification module further comprises:
a recovery loss value determining unit, configured to determine a normal recovery loss value according to the initial normal image and the recovery normal image; simultaneously determining a defect recovery loss value according to the initial defect image and the recovery defect image;
and the image recovery loss value determining unit is used for determining the image recovery loss value according to the recovery loss value and the defect recovery loss value.
9. A computer device comprising a memory, a processor, and computer readable instructions stored in the memory and executable on the processor, wherein the processor, when executing the computer readable instructions, implements the image classification method of any of claims 1 to 5.
10. A computer readable storage medium storing computer readable instructions which, when executed by a processor, implement the image classification method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110611214.2A CN113239878B (en) | 2021-06-01 | 2021-06-01 | Image classification method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110611214.2A CN113239878B (en) | 2021-06-01 | 2021-06-01 | Image classification method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113239878A CN113239878A (en) | 2021-08-10 |
| CN113239878B true CN113239878B (en) | 2023-09-05 |
Family
ID=77136330
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110611214.2A Active CN113239878B (en) | 2021-06-01 | 2021-06-01 | Image classification method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113239878B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111160406A (en) * | 2019-12-10 | 2020-05-15 | 北京达佳互联信息技术有限公司 | Training method of image classification model, and image classification method and device |
| CN111160411A (en) * | 2019-12-11 | 2020-05-15 | 东软集团股份有限公司 | Classification model training method, image processing method, device, medium, and apparatus |
| CN111476268A (en) * | 2020-03-04 | 2020-07-31 | 中国平安人寿保险股份有限公司 | Method, device, equipment and medium for training reproduction recognition model and image recognition |
-
2021
- 2021-06-01 CN CN202110611214.2A patent/CN113239878B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111160406A (en) * | 2019-12-10 | 2020-05-15 | 北京达佳互联信息技术有限公司 | Training method of image classification model, and image classification method and device |
| CN111160411A (en) * | 2019-12-11 | 2020-05-15 | 东软集团股份有限公司 | Classification model training method, image processing method, device, medium, and apparatus |
| CN111476268A (en) * | 2020-03-04 | 2020-07-31 | 中国平安人寿保险股份有限公司 | Method, device, equipment and medium for training reproduction recognition model and image recognition |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113239878A (en) | 2021-08-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111667011B (en) | Damage detection model training and vehicle damage detection method, device, equipment and medium | |
| CN110569721B (en) | Recognition model training method, image recognition method, device, equipment and medium | |
| CN109886077B (en) | Image recognition method and device, computer equipment and storage medium | |
| CN113807353B (en) | Image conversion model training method, device, equipment and storage medium | |
| CN113593611A (en) | Voice classification network training method and device, computing equipment and storage medium | |
| CN111666990A (en) | Vehicle damage characteristic detection method and device, computer equipment and storage medium | |
| CN111062324A (en) | Face detection method and device, computer equipment and storage medium | |
| CN111666931B (en) | Mixed convolution text image recognition method, device, equipment and storage medium | |
| CN112132279A (en) | Convolutional neural network model compression method, device, equipment and storage medium | |
| CN111832581A (en) | Lung feature recognition method and device, computer equipment and storage medium | |
| CN111860582B (en) | Image classification model construction method and device, computer equipment and storage medium | |
| CN113421276A (en) | Image processing method, device and storage medium | |
| CN112241646A (en) | Lane line recognition method and device, computer equipment and storage medium | |
| JP2023526899A (en) | Methods, devices, media and program products for generating image inpainting models | |
| CN113192175A (en) | Model training method and device, computer equipment and readable storage medium | |
| CN117033039A (en) | Fault detection method, device, computer equipment and storage medium | |
| CN115223012A (en) | Method, device, computer equipment and medium for restoring unmasked face | |
| CN110956195A (en) | Image matching method and device, computer equipment and storage medium | |
| CN111401335B (en) | Key point detection method and device and storage medium | |
| CN112861950A (en) | Method and device for identifying copied image, electronic equipment and storage medium | |
| CN113239878B (en) | Image classification method, device, equipment and medium | |
| CN114549849A (en) | Image recognition method and device, computer equipment and storage medium | |
| CN113780131B (en) | Text image orientation recognition method, text content recognition method, device and equipment | |
| CN111191062A (en) | Picture processing method and device, computer equipment and storage medium | |
| CN118154897A (en) | Training method of feature extractor, face recognition method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |