CN115018857A - Image segmentation method, image segmentation device, computer-readable storage medium and computer equipment - Google Patents
Image segmentation method, image segmentation device, computer-readable storage medium and computer equipment Download PDFInfo
- Publication number
- CN115018857A CN115018857A CN202210953897.4A CN202210953897A CN115018857A CN 115018857 A CN115018857 A CN 115018857A CN 202210953897 A CN202210953897 A CN 202210953897A CN 115018857 A CN115018857 A CN 115018857A
- Authority
- CN
- China
- Prior art keywords
- image segmentation
- image
- loss
- value
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
An image segmentation method, an image segmentation device, a computer-readable storage medium and a computer device are provided, wherein the method comprises the following steps: carrying out pixel normalization processing on the training sample image; inputting the training sample image subjected to the normalization processing into an image segmentation model for image segmentation processing; inputting the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into a target loss function for loss value calculation, wherein the target loss function is used for performing cross loss calculation, mean absolute error loss calculation, loss calculation in pixel classes and loss calculation between pixel classes; when the calculated loss value is larger than a preset value, optimizing parameters of the image segmentation model according to the loss value; and when the calculated loss value is less than or equal to a preset value, inputting the image to be detected into the image segmentation model to obtain an image segmentation result of the image to be detected. The image segmentation precision and the image segmentation efficiency of the invention are superior to those of the prior art.
Description
Technical Field
The present invention relates to the field of image processing, and in particular, to an image segmentation method, an image segmentation apparatus, a computer-readable storage medium, and a computer device.
Background
In industrial visual defect detection, deep learning has been widely used. Wherein the deep learning based semantic segmentation model exhibits more advantages over the conventional threshold segmentation in defective pixel segmentation. However, the types of defects in industrial scenes are various, and some common semantic segmentation loss functions cannot be well applied to some special scenes for industrial detection.
In image segmentation scenarios, the general task is to classify pixels, so the penalty for pixel classification usually employs a cross-entropy penalty function. The cross entropy loss function is defined as:
The cross entropy loss function is better in performance under ideal conditions of balanced category number, noiseless data and the like, but in a defect detection scene, scenes of uneven defect type distribution, small defect area, low-contrast scratch and the like often exist, and image labeling of industrial defect detection is often calculated on defect pixel labeling noise, for example, when labeling is not uniform due to unclear defect outline, model training is performed by adopting the cross entropy loss function, and the obtained model is not ideal in image segmentation effect.
Disclosure of Invention
In view of the above situation, it is desirable to provide an image segmentation method, an image segmentation apparatus, a computer-readable storage medium, and a computer device, which are directed to the problem in the prior art that the image segmentation effect is not ideal.
The invention provides an image segmentation method, which comprises the following steps:
carrying out pixel normalization processing on the training sample image;
inputting the training sample image subjected to the normalization processing into an image segmentation model for image segmentation processing;
inputting the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into a target loss function for loss value calculation, wherein the target loss function is used for performing cross loss calculation, mean absolute error loss calculation, loss calculation in pixel classes and loss calculation among pixel classes;
when the calculated loss value is larger than a preset value, optimizing parameters of the image segmentation model according to the loss value, and returning to the step of inputting the training sample image subjected to the normalization processing into the image segmentation model for image segmentation processing;
and when the calculated loss value is less than or equal to the preset value, inputting the image to be detected into the image segmentation model to obtain an image segmentation result of the image to be detected.
Further, in the image segmentation method, the step of inputting the prediction probability output by the image segmentation model and the true segmentation value of the training sample image into the target loss function to calculate the loss value includes:
counting the number of defective pixels and the number of non-defective pixels in the training sample image, and screening out the non-defective pixels with the probability lower than a threshold value by a preset number according to the prediction probability output by the image segmentation model;
and taking the defective pixels in the training sample image and the screened non-defective images as input pixels, and inputting the prediction probability of the input pixels and the segmentation true value of the training sample image into a target loss function for loss value calculation.
Further, in the image segmentation method, the preset number ish n Is composed of
And the training sample image is a non-defective pixel image, wherein p is the number of defective pixels in the training sample image, and n is the number of non-defective pixels in the training sample image.
Further, the image segmentation method described above, wherein the step of inputting the prediction probability output by the image segmentation model and the true segmentation value of the training sample image into the objective loss function to calculate the loss value further includes:
constructing an initial loss function according to the cross entropy function and the average absolute error loss function;
determining a loss function corresponding to each pixel category according to the pixels of each pixel category and the initial loss function;
and introducing a mean value to the loss function corresponding to each pixel type to calculate to obtain a target loss function.
Further, in the image segmentation method, the objective loss function is:
wherein, P t Predicting the probability, S, for the model of pixel t n Is a set of pixels of the nth pixel class,
the number of pixels in the pixel set of the nth pixel class is N, the total number of the pixel classes is N, and lambda is a coefficient for balancing cross entropy and average absolute error loss.
Further, in the image segmentation method, λ is 2.
Further, in the image segmentation method, the image segmentation model adopts a UNet image segmentation network or an FCN image segmentation network.
The present invention also provides an image segmentation apparatus, comprising:
the normalization processing module is used for carrying out pixel normalization processing on the training sample image;
the first image segmentation module is used for inputting the training sample image subjected to the normalization processing into an image segmentation model for image segmentation processing;
the calculation module is used for inputting the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into a target loss function for loss value calculation, wherein the target loss function is used for performing cross loss calculation, mean absolute error loss calculation, loss calculation in pixel classes and loss calculation between pixel classes;
the model optimization module is used for optimizing parameters of the image segmentation model according to the loss value when the calculated loss value is larger than a preset value, and returning to execute the step of inputting the training sample image subjected to the normalization processing into the image segmentation model for image segmentation processing;
and the second image segmentation module is used for inputting the image to be detected into the image segmentation model when the calculated loss value is less than or equal to the preset value, so as to obtain the image segmentation result of the image to be detected.
Further, the image segmentation apparatus further includes a target loss function construction module, where the target loss function construction module is configured to:
constructing an initial loss function according to the cross entropy function and the average absolute error loss function;
determining a loss function corresponding to each pixel category according to the pixels of each pixel category and the initial loss function;
and introducing a mean value to the loss function corresponding to each pixel type to calculate to obtain a target loss function.
The invention also provides a computer device comprising a memory and a processor, the memory storing a program which, when executed by the processor, implements any of the methods described above.
The invention also provides a computer readable storage medium having a program stored thereon, which when executed by a processor implements any of the methods described above.
In the invention, an image is input into an image segmentation model for image segmentation processing, a prediction probability is output, the prediction probability and a segmentation true value of the image are input into a target loss function for loss value calculation, parameters of the image segmentation model are optimized according to the calculated loss value, and the optimization of the image segmentation model is completed until the loss value is lower than a preset value. The target loss function definition defined in the invention can unify cross entropy, average absolute error, loss calculation in pixel categories, loss among pixel categories and the like, and improve the image segmentation precision.
Drawings
FIG. 1 is a flowchart of an image segmentation method according to a first embodiment of the present invention;
FIG. 2 is a flowchart of an image segmentation method according to a second embodiment of the present invention;
FIG. 3 is a block diagram of an image segmentation apparatus according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer device in an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
These and other aspects of embodiments of the invention will be apparent with reference to the following description and attached drawings. In the description and drawings, particular embodiments of the invention have been disclosed in detail as being indicative of some of the ways in which the principles of the embodiments of the invention may be practiced, but it is understood that the scope of the embodiments of the invention is not limited correspondingly. On the contrary, the embodiments of the invention include all changes, modifications and equivalents coming within the spirit and terms of the claims appended hereto.
Referring to fig. 1, an image segmentation method according to a first embodiment of the present invention includes steps S11-S15.
In step S11, the training sample image is subjected to pixel normalization processing.
In step S12, the training sample image after the normalization processing is input to the image segmentation model to be subjected to image segmentation processing.
In this embodiment, the image segmentation model is trained by using a training sample data set and a loss function. The training sample data set comprises a plurality of training sample images, and each training sample image needs to be preprocessed. The preprocessing process mainly comprises normalization processing, namely dividing the pixel value of the training sample image by 255 to normalize the pixel value to be between 0 and 1.
The image segmentation model can adopt a deep learning model, such as UNet, FCN and other segmentation networks. The image segmentation model is used for predicting the pixel type of each pixel, namely calculating the probability that the pixel is consistent with the labeling result, and determining the pixel type of the pixel according to the probability.
And step S13, inputting the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into a target loss function for loss value calculation, wherein the target loss function is used for performing cross entropy loss calculation, mean absolute error loss calculation, loss calculation in pixel class and loss calculation between pixel classes.
And inputting the output of the image segmentation model and a segmentation true value (Ground Truth) of the training sample image into a loss function calculation module to obtain a loss value. The output of the image segmentation model is the class prediction probability of the pixel, i.e. the probability of being consistent with the labeling result. The true segmentation value is the pixel type labeled by each pixel.
The pixel type is a defect type of the image, such as crack, slip line, scratch, bump, and the like.
When loss calculation is performed by the loss function according to input information, the following three calculation parts are mainly involved:
calculating cross entropy loss;
calculating the average absolute error loss;
loss calculations within pixel classes and loss calculations between pixel classes.
The cross entropy function used for cross entropy loss calculation may be:
The cross entropy function is defined as standard cross entropy, and the standard cross entropy function is a standard loss function in image segmentation and is suitable for most scenes with larger sizes.
In order to improve the class data labeling noise scene, average Absolute Error loss calculation and an average Absolute Error loss function (MAE) are introduced, so that the noise scene can be effectively improved.
The mean absolute error loss function is more robust in noisy data sets than the cross entropy loss function, which is defined as follows:
the gradient of cross entropy is more negative for negative samples, while the mean absolute error is unity. Therefore, if there are many noisy samples in the sample, then using a cross-entropy model will try to fit the noisy data, where it is preferable to use MAE as the loss function.
In an industrial defect detection scene, the number of different types of defects is often unbalanced, and in order to improve the problem of class imbalance, in this embodiment, a method of averaging all pixels is not adopted, but loss averaging is performed inside each pixel class first, and then averaging between the pixel classes is performed. This way the class imbalance situation is greatly alleviated.
And step S14, when the calculated loss value is larger than a preset value, optimizing the parameters of the image segmentation model according to the loss value, and returning to the step of inputting the training sample image after the normalization processing into the image segmentation model for image segmentation processing.
And step S15, when the calculated loss value is less than or equal to the preset value, inputting the image to be detected into the image segmentation model to obtain the image segmentation result of the image to be detected.
The smaller the calculated loss value is, the more accurate the image segmentation of the image segmentation model is, and the better the segmentation effect is. When the calculated loss value is less than or equal to the preset value, it can be said that the image segmentation model is trained. The trained model can be used for image segmentation of the image to be detected.
In this embodiment, an image is input into an image segmentation model to perform image segmentation processing, a prediction probability is output, the prediction probability and a true segmentation value of the image are input into a target loss function to perform loss value calculation, and parameters of the image segmentation model are optimized according to the calculated loss value until the loss value is lower than a preset value, so that the optimization of the image segmentation model is completed. The target loss function defined in this embodiment may unify cross entropy, average absolute error, loss calculation in pixel categories, loss between pixel categories, and the like, and improve image segmentation accuracy, and the target loss function in this embodiment does not require complex coefficient search, and simple coefficients have already reached the improvement of data set accuracy.
Referring to FIG. 2, an image segmentation method according to a second embodiment of the present invention includes steps S21-S28.
In step S21, the training sample image is subjected to pixel normalization processing.
The normalization process mainly adjusts the size of the characteristic value to a similar range, so that the model training is stable in convergence. In specific implementation, the pixel value of the training sample image can be divided by 255, so that the pixel value is normalized to be between 0 and 1.
In step S22, the training sample image after the normalization processing is input to the image segmentation model to be subjected to image segmentation processing.
The image segmentation model can adopt a deep learning model, such as UNet, FCN and other segmentation networks. The image segmentation model is used for predicting the pixel category of each pixel, namely calculating the probability that the pixel is consistent with the labeling result, and determining the pixel category of the pixel according to the probability.
Step S23, an objective loss function is constructed.
Specifically, the step of constructing the target loss function includes:
s231, constructing an initial loss function according to the cross entropy function and the average absolute error loss function;
s232, determining a loss function corresponding to each pixel type according to the pixels of each pixel type and the initial loss function;
and S233, calculating the average value of the loss functions corresponding to the pixel types to obtain a target loss function.
The target loss function is:
wherein, P t Predicting the probability, S, for the model of pixel t n Is a set of pixels of the nth pixel class,
the number of pixels in the pixel set of the nth pixel class is N, the total number of the pixel classes is N, and lambda is a coefficient for balancing cross entropy and average absolute error loss.
The objective loss function in this embodiment mainly includes the following three components.
1. Cross entropy part
The adopted cross entropy formula is
，P t Is the prediction probability. This section is defined as the standard cross entropy, which is the standard loss function in image segmentation that applies to larger and more scenes.
2. Mean absolute error part
In order to improve class data labeling noise scenes, an average absolute error part is introduced, and the average absolute error is verified, so that the noise scenes can be effectively improved. In this embodiment, the loss function of the average absolute error part may be:
this portion is superimposed with the cross entropy, where λ is the coefficient that balances the cross entropy and the mean absolute error loss, reducing the effect of noisy data. In this embodiment, λ =2, and a good segmentation effect can be achieved for most data sets by using the coefficient.
3. Loss per class and mean between classes part:
in an industrial defect detection scenario, the number of different defect types is often unbalanced, and in order to improve the problem of class imbalance, in this embodiment, a method of averaging all pixels is not adopted, but loss averaging is performed inside each class first, and then inter-class averaging is performed. This way the class imbalance situation is greatly alleviated. In that
In this section, the penalty for each pixel class is obtained. After the loss of each pixel class is obtained, the overall average is obtained, which is equivalent to equalizing the weight of each class. The method can effectively relieve the conditions of small defect area and unbalanced defect types in industrial defect detection.
And step S24, counting the number of defective pixels and the number of non-defective pixels in the training sample image, and screening out the non-defective pixels with the probability lower than a threshold value according to the prediction probability output by the image segmentation model.
Step S25, using the defective pixels in the training sample image and the screened non-defective images as input pixels, and inputting the prediction probability of the input pixels and the true segmentation value of the training sample image into a target loss function for loss value calculation.
For the current training sample image, firstly, screening the defective pixels and the pixels with low confidence level, and taking the pixels of the screened training sample image as the pixel sample for calculating loss. Counting the number p of the existing marked defective pixels and the number n of the non-defective pixels, and screening the number with lower probability as
P defective pixels and screened
One non-defective pixel, which is a pixel in a single image for which loss is calculated, is denoted as a pixel set S.
And inputting the prediction probability of each pixel in the pixel set S and the corresponding segmentation true value into the target loss function for loss value calculation.
In step S26, it is determined whether the calculated loss value is less than or equal to the predetermined value, if not, step S27 is performed, and if so, step S28 is performed.
And step S27, optimizing the parameters of the image segmentation model according to the loss values, and returning to the step S22.
And step S28, inputting the image to be detected into the image segmentation model to obtain the image segmentation result of the image to be detected.
The smaller the calculated loss value is, the more accurate the image segmentation of the image segmentation model is, and the better the segmentation effect is. And when the calculated loss value is larger than the preset value, adjusting the model parameters, and returning to the step of carrying out image segmentation and loss value calculation again. When the calculated loss value is less than or equal to the preset value, the image segmentation model can be proved to be trained, and the trained model can be used for image segmentation of the image to be detected.
To verify the effect of the image segmentation method in this embodiment, the following experiment was performed:
the defect detection data sets of a plurality of products are adopted, 4 data sets are provided, each data set comprises 100 sample images, the number of pixel categories is from 2 to 5, and the number of the sample images in the pixel categories is unbalanced. The 4 data sets are respectively subjected to training set division and test set division, wherein the proportion of the training set is 80%, the proportion of the verification set is 10%, the proportion of the test set is 10%, the batch size is unified into 16, the size of the image training set is 448 x 448, the model backbone network can comprise UNET, FCN and other common backbone networks, and DDRNet-slim is adopted in the calculation efficiency considered in the scheme. DDRNet is a high-speed real-time image segmentation backbone networkThe calculation efficiency is higher, and the accuracy statistical mode adopts the mode of testing the set defect mIOU, namely, the intersection/union of the positions of the detected defective pixel and the marked pixel. Based on the training set, the loss function and the conventional standard cross entropy loss function L in the embodiment are respectively utilized CE Loss function L 1 And a loss function L 2 The image segmentation model is trained, and the trained image segmentation model is tested by using a test set, wherein the test result is shown in table 1. Wherein substrate-1, substrate-2, epi-1, and epi-2 represent four data sets. L represents a loss function in the present embodiment;
，L CE representing a standard cross entropy loss function; l1 is
，L 1 Representing the introduction of an average absolute error loss function in a standard cross entropy loss function; l2 is
Equivalent to a weighted cross entropy loss function.
TABLE 1
The experimental result shows that after the average absolute error loss, the intra-class and inter-class average losses are introduced in the loss value calculation process, the image segmentation model has the highest accuracy in the test set and the best segmentation effect.
Referring to fig. 3, an image segmentation apparatus according to a third embodiment of the present invention includes:
a normalization processing module 31, configured to perform pixel normalization processing on the training sample image;
the first image segmentation module 32 is configured to input the training sample image after the normalization processing into an image segmentation model for image segmentation processing;
a calculating module 33, configured to input the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into a target loss function for performing loss value calculation, where the target loss function is used to perform cross loss calculation, mean absolute error loss calculation, and intra-pixel loss calculation and inter-pixel loss calculation;
the model optimization module 34 is configured to, when the calculated loss value is greater than a preset value, optimize parameters of the image segmentation model according to the loss value, and return to the step of inputting the training sample image after the normalization processing into the image segmentation model for image segmentation processing;
and the second image segmentation module 35 is configured to, when the calculated loss value is less than or equal to the preset value, input the image to be detected into the image segmentation model, so as to obtain an image segmentation result of the image to be detected.
Further, the image segmentation apparatus further includes a target loss function construction module, where the target loss function construction module is configured to:
constructing an initial loss function according to the cross entropy function and the average absolute error loss function;
determining a loss function corresponding to each pixel category according to the pixels of each pixel category and the initial loss function;
and introducing a mean value to the loss function corresponding to each pixel type to calculate to obtain a target loss function.
The image segmentation apparatus provided in the embodiment of the present invention has the same implementation principle and technical effect as those of the foregoing method embodiments, and for brief description, reference may be made to corresponding contents in the foregoing method embodiments for the sake of brevity.
Referring to fig. 4, a computer device according to an embodiment of the present invention is shown, which includes a processor 10, a memory 20, and a computer program 30 stored in the memory and executable on the processor, and the processor 10 implements the image segmentation method as described above when executing the computer program 30.
The computer device may be, but is not limited to, a personal computer, a server, and the like. The processor 10 may be, in some embodiments, a Central Processing Unit (CPU), controller, microcontroller, microprocessor or other data Processing chip for executing program codes stored in the memory 20 or Processing data.
The memory 20 includes at least one type of readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, and the like. The memory 20 may in some embodiments be an internal storage unit of the computer device, for example a hard disk of the computer device. The memory 20 may also be an external storage device of the computer device in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the computer device. Further, the memory 20 may also include both an internal storage unit and an external storage device of the computer apparatus. The memory 20 may be used not only to store application software installed in the computer device, various types of data, and the like, but also to temporarily store data that has been output or is to be output.
Optionally, the computer device may further comprise a user interface, which may comprise a Display (Display), an input unit such as a Keyboard (Keyboard), a network interface, a communication bus, etc., and the optional user interface may also comprise a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the computer device and for displaying a visualized user interface. The network interface may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), typically used to establish a communication link between the device and other electronic devices. The communication bus is used to enable connection communication between these components.
It should be noted that the configuration shown in fig. 4 does not constitute a limitation of the computer device, and in other embodiments, the computer device may include fewer or more components than those shown, or some components may be combined, or a different arrangement of components may be used.
The invention also proposes a computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, implements the image segmentation method as described above.
Those of skill in the art will understand that the logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be viewed as implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus (e.g., a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or execute the instructions). For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following technologies, which are well known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. An image segmentation method, comprising:
carrying out pixel normalization processing on the training sample image;
inputting the training sample image subjected to the normalization processing into an image segmentation model for image segmentation processing;
inputting the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into a target loss function for loss value calculation, wherein the target loss function is used for performing cross loss calculation, mean absolute error loss calculation, loss calculation in pixel classes and loss calculation between pixel classes;
when the calculated loss value is larger than a preset value, optimizing parameters of the image segmentation model according to the loss value, and returning to the step of inputting the training sample image subjected to the normalization processing into the image segmentation model for image segmentation processing;
and when the calculated loss value is less than or equal to the preset value, inputting the image to be detected into the image segmentation model to obtain an image segmentation result of the image to be detected.
2. The image segmentation method as claimed in claim 1, wherein the step of inputting the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into an objective loss function for loss value calculation comprises:
counting the number of defective pixels and the number of non-defective pixels in the training sample image, and screening out the non-defective pixels with the probability lower than a threshold value by a preset number according to the prediction probability output by the image segmentation model;
and taking the defective pixels in the training sample image and the screened non-defective images as input pixels, and inputting the prediction probability of the input pixels and the segmentation true value of the training sample image into a target loss function for loss value calculation.
3. The image segmentation method according to claim 2, wherein the preset numberh n Is composed of
Wherein p is the number of defective pixels in the training sample image, and n is a non-defective pixel in the training sample imageThe number of (2).
4. The image segmentation method according to claim 1, wherein the step of inputting the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into the target loss function for loss value calculation is preceded by the step of:
constructing an initial loss function according to the cross entropy function and the average absolute error loss function;
determining a loss function corresponding to each pixel category according to the pixels of each pixel category and the initial loss function;
and introducing a mean value to the loss function corresponding to each pixel type to calculate to obtain a target loss function.
5. The image segmentation method of claim 4, wherein the objective loss function is:
wherein L is an objective loss function, P t Predicting the probability, S, for the model of pixel t n Is a set of pixels of the nth pixel class,
the number of pixels in the pixel set of the nth pixel class is N, the total number of the pixel classes is N, and lambda is a coefficient for balancing cross entropy and average absolute error loss.
6. An image segmentation method as claimed in claim 5, characterized in that λ takes the value 2.
7. The image segmentation method according to claim 1, wherein the image segmentation model employs a UNet image segmentation network or an FCN image segmentation network.
8. An image segmentation apparatus, comprising:
the normalization processing module is used for carrying out pixel normalization processing on the training sample image;
the first image segmentation module is used for inputting the training sample image subjected to the normalization processing into an image segmentation model for image segmentation processing;
the calculation module is used for inputting the prediction probability output by the image segmentation model and the segmentation true value of the training sample image into a target loss function for loss value calculation, wherein the target loss function is used for performing cross loss calculation, mean absolute error loss calculation, loss calculation in pixel classes and loss calculation between pixel classes;
the model optimization module is used for optimizing parameters of the image segmentation model according to the loss value when the calculated loss value is larger than a preset value, and returning to execute the step of inputting the training sample image subjected to the normalization processing into the image segmentation model for image segmentation processing;
and the second image segmentation module is used for inputting the image to be detected into the image segmentation model when the calculated loss value is less than or equal to the preset value, so as to obtain the image segmentation result of the image to be detected.
9. A computer device comprising a memory and a processor, the memory storing a program that, when executed by the processor, implements the method of any of claims 1-7.
10. A computer-readable storage medium, on which a program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210953897.4A CN115018857B (en) | 2022-08-10 | 2022-08-10 | Image segmentation method, image segmentation device, computer-readable storage medium and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210953897.4A CN115018857B (en) | 2022-08-10 | 2022-08-10 | Image segmentation method, image segmentation device, computer-readable storage medium and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115018857A true CN115018857A (en) | 2022-09-06 |
| CN115018857B CN115018857B (en) | 2022-11-11 |
Family
ID=83065502
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210953897.4A Active CN115018857B (en) | 2022-08-10 | 2022-08-10 | Image segmentation method, image segmentation device, computer-readable storage medium and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115018857B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115239719A (en) * | 2022-09-22 | 2022-10-25 | 南昌昂坤半导体设备有限公司 | Defect detection method, system, electronic device and storage medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107403430A (en) * | 2017-06-15 | 2017-11-28 | 中山大学 | A kind of RGBD image, semantics dividing method |
| CN108921839A (en) * | 2018-07-02 | 2018-11-30 | 北京百度网讯科技有限公司 | Continuous casting billet quality detection method, device, electronic equipment and storage medium |
| CN108961238A (en) * | 2018-07-02 | 2018-12-07 | 北京百度网讯科技有限公司 | Display screen quality determining method, device, electronic equipment and storage medium |
| US20200126236A1 (en) * | 2018-10-22 | 2020-04-23 | The Board Of Trustees Of The Leland Stanford Junior University | Systems and Methods for Image Segmentation using IOU Loss Functions |
| CN111179272A (en) * | 2019-12-10 | 2020-05-19 | 中国科学院深圳先进技术研究院 | Rapid semantic segmentation method for road scene |
| CN111260665A (en) * | 2020-01-17 | 2020-06-09 | 北京达佳互联信息技术有限公司 | Image segmentation model training method and device |
| CN111667027A (en) * | 2020-07-03 | 2020-09-15 | 腾讯科技（深圳）有限公司 | Multi-modal image segmentation model training method, image processing method and device |
| CN112419248A (en) * | 2020-11-13 | 2021-02-26 | 复旦大学 | Ear sclerosis focus detection and diagnosis system based on small target detection neural network |
| CN112613515A (en) * | 2020-11-23 | 2021-04-06 | 上海眼控科技股份有限公司 | Semantic segmentation method and device, computer equipment and storage medium |
| CN113468977A (en) * | 2021-06-10 | 2021-10-01 | 汉王科技股份有限公司 | Text line language identification method and device and electronic equipment |
| CN113538480A (en) * | 2020-12-15 | 2021-10-22 | 腾讯科技（深圳）有限公司 | Image segmentation processing method and device, computer equipment and storage medium |
| CN113763387A (en) * | 2021-07-22 | 2021-12-07 | 深圳大学 | Placenta ultramicro blood vessel segmentation method, storage medium and terminal equipment |
-
2022
- 2022-08-10 CN CN202210953897.4A patent/CN115018857B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107403430A (en) * | 2017-06-15 | 2017-11-28 | 中山大学 | A kind of RGBD image, semantics dividing method |
| CN108921839A (en) * | 2018-07-02 | 2018-11-30 | 北京百度网讯科技有限公司 | Continuous casting billet quality detection method, device, electronic equipment and storage medium |
| CN108961238A (en) * | 2018-07-02 | 2018-12-07 | 北京百度网讯科技有限公司 | Display screen quality determining method, device, electronic equipment and storage medium |
| US20200126236A1 (en) * | 2018-10-22 | 2020-04-23 | The Board Of Trustees Of The Leland Stanford Junior University | Systems and Methods for Image Segmentation using IOU Loss Functions |
| CN111179272A (en) * | 2019-12-10 | 2020-05-19 | 中国科学院深圳先进技术研究院 | Rapid semantic segmentation method for road scene |
| CN111260665A (en) * | 2020-01-17 | 2020-06-09 | 北京达佳互联信息技术有限公司 | Image segmentation model training method and device |
| CN111667027A (en) * | 2020-07-03 | 2020-09-15 | 腾讯科技（深圳）有限公司 | Multi-modal image segmentation model training method, image processing method and device |
| CN112419248A (en) * | 2020-11-13 | 2021-02-26 | 复旦大学 | Ear sclerosis focus detection and diagnosis system based on small target detection neural network |
| CN112613515A (en) * | 2020-11-23 | 2021-04-06 | 上海眼控科技股份有限公司 | Semantic segmentation method and device, computer equipment and storage medium |
| CN113538480A (en) * | 2020-12-15 | 2021-10-22 | 腾讯科技（深圳）有限公司 | Image segmentation processing method and device, computer equipment and storage medium |
| CN113468977A (en) * | 2021-06-10 | 2021-10-01 | 汉王科技股份有限公司 | Text line language identification method and device and electronic equipment |
| CN113763387A (en) * | 2021-07-22 | 2021-12-07 | 深圳大学 | Placenta ultramicro blood vessel segmentation method, storage medium and terminal equipment |
Non-Patent Citations (1)
| Title |
|---|
| 朱勇等: "《3D SE-ResNet：一种从CT图像中自动分割COVID-19肺部感染模型》", 《中南民族大学学报（自然科学版）》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115239719A (en) * | 2022-09-22 | 2022-10-25 | 南昌昂坤半导体设备有限公司 | Defect detection method, system, electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115018857B (en) | 2022-11-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108921206B (en) | Image classification method and device, electronic equipment and storage medium | |
| CN108304775B (en) | Remote sensing image recognition method and device, storage medium and electronic equipment | |
| CN108171203B (en) | Method and device for identifying vehicle | |
| US20190130233A1 (en) | Method and apparatus for improved segmentation and recognition of images | |
| WO2022213465A1 (en) | Neural network-based image recognition method and apparatus, electronic device, and medium | |
| CN111275660B (en) | Flat panel display defect detection method and device | |
| CN115240011A (en) | Image classification method and device, computer readable storage medium and computer equipment | |
| CN110874604A (en) | Model training method and terminal equipment | |
| CN115018857B (en) | Image segmentation method, image segmentation device, computer-readable storage medium and computer equipment | |
| CN110059666A (en) | A kind of attention detection method and device | |
| CN116894985B (en) | Semi-supervised image classification method and semi-supervised image classification system | |
| CN111723815A (en) | Model training method, image processing method, device, computer system, and medium | |
| CN117147561B (en) | Surface quality detection method and system for metal zipper | |
| CN111681228A (en) | Flaw detection model, training method, detection method, apparatus, device, and medium | |
| CN110135428B (en) | Image segmentation processing method and device | |
| US9424484B2 (en) | Feature interpolation | |
| CN110751170A (en) | Panel quality detection method, system, terminal device and computer readable medium | |
| CN110796129A (en) | Text line region detection method and device | |
| CN106407991B (en) | Image attribute recognition method and system and related network training method and system | |
| CN111784053A (en) | Transaction risk detection method, device and readable storage medium | |
| CN116468479A (en) | Method for determining page quality evaluation dimension, and page quality evaluation method and device | |
| CN116977271A (en) | Defect detection method, model training method, device and electronic equipment | |
| CN111091163A (en) | Minimum distance classification method and device, computer equipment and storage medium | |
| CN115880533A (en) | Bridge apparent crack identification method based on adaptive subset search and deep learning | |
| CN115512203A (en) | Information detection method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |