CN115240023A - Training method of defect detection network, defect detection method and related equipment - Google Patents
Training method of defect detection network, defect detection method and related equipment Download PDFInfo
- Publication number
- CN115240023A CN115240023A CN202210665933.7A CN202210665933A CN115240023A CN 115240023 A CN115240023 A CN 115240023A CN 202210665933 A CN202210665933 A CN 202210665933A CN 115240023 A CN115240023 A CN 115240023A
- Authority
- CN
- China
- Prior art keywords
- image
- defect detection
- sample
- defect
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a training method of a defect detection network, a defect detection method and related equipment, wherein the method comprises the following steps: acquiring a training sample image and a defect sample image, wherein the training sample image is an image without defects, and the defect sample image is an image generated by adding defects into the training sample image; inputting the defect sample image into a defect detection network to obtain a reconstructed sample image, wherein the reconstructed sample image is an image without defects; processing the reconstructed sample image and the defect sample image by adopting a defect detection network to obtain a sample defect detection image, wherein the sample defect detection image is obtained by predicting defects in the defect sample image; and training the defect detection network based on the sample defect detection image, the reconstructed sample image and the training sample image to obtain the trained defect detection network. Through the mode, the method and the device can realize unsupervised learning, and improve the accuracy of defect detection.
Description
Technical Field
The present application relates to the field of defect detection technologies, and in particular, to a training method for a defect detection network, a defect detection method, and a related device.
Background
At present, the demand for defect detection of some equipment or articles is rapidly increased, the manual inspection scheme is limited by the influence of factors such as terrain, weather and the like, and the efficiency is low; the defect detection method based on the traditional image processing is greatly influenced by illumination change, has poor generalization and has lower detection accuracy on smaller defects.
Disclosure of Invention
The application provides a training method of a defect detection network, a defect detection method and related equipment, which can realize unsupervised learning and improve the accuracy of defect detection.
In order to solve the technical problem, the technical scheme adopted by the application is as follows: a training method of a defect detection network is provided, and the method comprises the following steps: acquiring a training sample image and a defect sample image, wherein the training sample image is an image without a defect, and the defect sample image is an image generated by adding a defect into the training sample image; inputting the defect sample image into a defect detection network to obtain a reconstructed sample image, wherein the reconstructed sample image is an image without defects; processing the reconstructed sample image and the defect sample image by adopting a defect detection network to obtain a sample defect detection image, wherein the sample defect detection image is obtained by predicting defects in the defect sample image; and training the defect detection network based on the sample defect detection image, the reconstructed sample image and the training sample image to obtain the trained defect detection network.
In order to solve the technical problem, the technical scheme adopted by the application is as follows: there is provided a defect detection method, the method comprising: acquiring an image to be detected, wherein the image to be detected is an image containing defects; inputting the image to be detected into the trained defect detection network to obtain a defect detection image; the trained defect detection network is obtained by the training method of the defect detection network in the technical scheme.
In order to solve the above technical problem, another technical solution adopted by the present application is: there is provided a defect detecting apparatus comprising a memory and a processor connected to each other, wherein the memory is used for storing a computer program, and the computer program, when being executed by the processor, is used for implementing a training method of a defect detecting network in the above technical solution or a defect detecting method in the above technical solution.
In order to solve the above technical problem, another technical solution adopted by the present application is: there is provided a computer readable storage medium for storing a computer program, which when executed by a processor is configured to implement the method for training a defect detection network according to the above technical solution or the method for detecting a defect according to the above technical solution.
Through the scheme, the beneficial effects of the application are that: firstly, acquiring a training sample image without defects; then adding defects into the training sample image to obtain a defect sample image; then, inputting the defect sample image into a defect detection network to obtain a reconstructed sample image without defects; processing the reconstructed sample image and the defect sample image by adopting a defect detection network to obtain a sample defect detection image, wherein the sample defect detection image is obtained by predicting defects in the defect sample image; secondly, training the defect detection network by using the sample defect detection image, the reconstructed sample image and the training sample image to obtain a trained defect detection network; according to the scheme, the training of the defect detection network can be realized only by adopting the training sample image without the defect, the image containing the defect does not need to be obtained in advance, the unsupervised training is realized, the manpower and material resources required for collecting the image containing the defect and marking the defect can be effectively reduced, the model training cost is reduced, and the detection accuracy is higher; compared with a detection scheme which needs to acquire a defective image in advance, the defect detection method can detect the defect under the condition of lacking the image with the defect, has higher generalization, is easy to capture the image without the defect due to a large number of images without the defect in practical application, and is simple to realize.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts. Wherein:
FIG. 1 is a schematic flowchart illustrating an embodiment of a method for training a defect detection network provided in the present application;
FIG. 2 is a schematic illustration of an original image provided herein;
FIG. 3 is a schematic diagram of a sample segmentation image corresponding to FIG. 2;
FIG. 4 is a schematic illustration of a rectified image corresponding to region D of FIG. 3;
FIG. 5 is a schematic flowchart of another embodiment of a method for training a defect detection network provided in the present application;
FIG. 6 is a schematic diagram of a defect detection network provided herein;
FIG. 7 is a schematic flowchart of an embodiment of a defect detection method provided in the present application;
FIG. 8 is a schematic illustration of a defect detection image provided herein;
FIG. 9 is a schematic structural diagram of an embodiment of a defect detection apparatus provided in the present application;
FIG. 10 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided in the present application.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be noted that the following examples are only illustrative of the present application, and do not limit the scope of the present application. Likewise, the following examples are only some examples and not all examples of the present application, and all other examples obtained by a person of ordinary skill in the art without any inventive work are within the scope of the present application.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the specification. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
It should be noted that the terms "first", "second" and "third" in this application are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or to imply that the number of indicated technical features is high. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of the feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless explicitly specifically limited otherwise. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
Referring to fig. 1, fig. 1 is a schematic flowchart illustrating an embodiment of a method for training a defect detection network according to the present application, where the method includes:
s11: and acquiring a training sample image and a defect sample image.
Shooting a target scene by adopting camera equipment or an unmanned aerial vehicle to obtain a training sample image, wherein the training sample image is an image without defects; alternatively, training sample images are obtained from a database of images. Then, noise or defects are added into the training sample image to obtain a defect sample image, namely the defect sample image is an image generated by adding defects into the training sample image, and the defects can be block defects, line defects or planar defects.
In a specific embodiment, the inclination angle of the target object in the training sample image is smaller than a preset angle, and the preset angle may be an angle value falling around 0 °; however, due to the influence of the shooting angle, the inclination angle of the target object may be larger than the preset angle, and therefore, the image where the target object is located needs to be corrected, so that the inclination angle of the target object in the corrected image is smaller than the preset angle.
Further, acquiring an original image, wherein the original image is an image without defects, the original image can be an infrared thermal imaging image or a color image, and the original image comprises a target object, and the target object can be a photovoltaic panel, a pipeline, a display panel or the like; carrying out target detection on the original image to determine the position of a target object in the original image; and correcting the original image based on the position of the target object to obtain a training sample image. It is understood that the solution adopted for the correction is a common correction solution in the related art, such as: and transforming the original image into a training sample image through perspective transformation, or inputting the original image into a pre-trained correction model to obtain the training sample image.
S12: and inputting the defect sample image into a defect detection network to obtain a reconstructed sample image.
Firstly, acquiring a defect detection network, wherein the defect detection network is a model based on a neural network; after the defect sample image is obtained, inputting the defect sample image into a defect detection network which is constructed in advance, so that the defect detection network carries out reconstruction processing on the defect sample image, the defect in the defect sample image is removed, and a reconstructed sample image is obtained, namely the reconstructed sample image is an image without defects.
S13: and processing the reconstructed sample image and the defect sample image by adopting a defect detection network to obtain a sample defect detection image.
After the reconstructed sample image is obtained, a defect detection network is adopted to carry out defect detection processing on the reconstructed sample image and the defect sample image, and a sample defect detection image is generated, namely the sample defect detection image is obtained by predicting defects in the defect sample image. Specifically, the sample defect detection image is a mask image, which includes a region where the defect is located (referred to as a defective region) and a non-defective region, and the pixel value of the defective region may be set to a first preset value, and the pixel value of the non-defective region may be set to a second preset value. It can be understood that the first preset value and the second preset value can be set according to specific application needs or experience, such as: the first preset value is 255, and the second preset value is 0; or the first preset value is 0, and the second preset value is 255; or the first preset value is 1, and the second preset value is 0.
Furthermore, the reconstructed sample image and the defect sample image can be spliced, and the image generated after splicing is subjected to prediction processing to obtain a sample defect detection image. Or, carrying out feature extraction processing on the sample defect image to obtain defect features; carrying out feature extraction processing on the reconstructed sample image to obtain a reconstructed feature; splicing the defect characteristics and the reconstruction characteristics to obtain splicing characteristics; and performing prediction processing on the splicing characteristics to obtain a sample defect detection image.
S14: and training the defect detection network based on the sample defect detection image, the reconstructed sample image and the training sample image to obtain the trained defect detection network.
After a sample defect detection image is obtained, calculating a loss value (recorded as a current loss value) of a current defect detection network by using the sample defect detection image, a reconstructed sample image and a training sample image; then, calculating the current loss value or the current accumulated training times, and determining whether to finish the training of the defect detection network; if the defect detection network training is judged to be finished, obtaining the trained defect detection network; if the defect detection network is judged not to be trained, the step returns to the step S11, and the training of the defect detection network is continued.
In a specific embodiment, taking a target object as a photovoltaic panel as an example, hot spots are main defects affecting normal operation of the photovoltaic panel, and in order to eliminate interference of non-photovoltaic panel areas and correctly detect the hot spots, an original image is input into an image segmentation network to obtain a sample segmentation image, wherein the sample segmentation image includes the photovoltaic panel area and the non-photovoltaic panel area, and the photovoltaic panel area is an area where the photovoltaic panel is located. Specifically, as shown in fig. 3, the photovoltaic panel region is composed of four boundary lines, and the connection point of the four boundary lines (i.e., the vertex of the photovoltaic panel region) can be used as the position of the photovoltaic panel; the segmentation model may be a common model for implementing a segmentation function, such as a net Network, a High Resolution Network (HRNet), or a Full Convolution Network (FCN). It is understood that a target detection model or other methods may also be used to determine the position of the photovoltaic panel, and is not limited herein.
And then, based on the position of the photovoltaic panel, carrying out rotation correction on the original image to obtain a training sample image, wherein the inclination angle of the photovoltaic panel in the training sample image is smaller than a preset angle. For example, an unmanned aerial vehicle is used for shooting a photovoltaic panel to obtain an original image shown in fig. 2, wherein the original image is an infrared thermal imaging image; inputting the original image into a Unet network to obtain a sample segmentation image shown in FIG. 3, wherein a region D surrounded by a dotted frame is a photovoltaic panel region; after the photovoltaic panel in the region D is corrected, an infrared thermal imaging image of the photovoltaic panel at the normal viewing angle shown in fig. 4 can be obtained; and inputting the corrected image into a defect detection network to obtain a sample defect detection image.
The embodiment provides a training method of an unsupervised defect detection network, which is characterized in that a defect is automatically generated on a training sample image to obtain a defect sample image; the defect detection network is trained by utilizing the defect sample image and the training sample image, the defect detection network can be trained only by acquiring the training sample image without the defect without acquiring the image containing the defect in advance, the unsupervised training is realized, the manpower and material resources required for collecting the image containing the defect and marking the defect can be effectively reduced, and the model training cost is reduced; compared with a detection scheme which needs to acquire a defective image in advance, the defect detection can be carried out under the condition of lacking the defective image, the generalization is higher, and the implementation difficulty is lower because the number of the images which do not contain the defect is larger in practical application; in addition, the defect detection network based on the neural network is adopted to replace the traditional image processing technology to identify the defects, and the detection accuracy is higher.
Referring to fig. 5, fig. 5 is a schematic flowchart illustrating a training method of a defect detection network according to another embodiment of the present application, the method including:
s51: training sample images are acquired.
The training sample image is an image that does not contain defects.
S52: and randomly generating a mask image, and synthesizing the mask image and the training sample image to obtain a defect sample image.
And synthesizing the training sample image with the normal visual angle and the randomly generated mask image to obtain a synthesized defect sample image with the defects, wherein the defect sample image is an image generated by adding the defects into the training sample image.
In a specific embodiment, as shown in fig. 6, the defect detection network includes a reconstruction subnetwork, a splicing subnetwork, and a discrimination subnetwork, where "Anomaly generation" is an Anomaly generation module, and is used to generate a Mask image Mask; the following describes the processing procedure of the defect detection network.
S53: inputting the defect sample image into a reconstruction sub-network to obtain a reconstructed sample image; splicing the reconstructed sample image and the defect sample image by adopting a splicing subnetwork to obtain a sample spliced image; and inputting the sample splicing image into a discrimination subnetwork to obtain a first detection result image.
As shown in fig. 6, the following scheme is adopted to generate the first detection result image Mask _ pred:
1) Merging the training sample image I and the Mask image Mask to obtain a defect sample image I a 。
The dimension of the training sample image I is the same as that of the Mask image Mask.
2) Defective sample image I a Sent into a reconstruction sub-network to reconstruct an image containing no defects (i.e. a reconstructed sample image I) r )。
The reconstruction sub-network comprises a first encoder and a first decoder, and the first decoder is used for the defect sample image I a Coding is carried out to obtain first coding information; decoding the first coding information by adopting a first decoder to obtain a reconstructed sample image I r Reconstructing a sample image I r For images that do not contain defects, the first encoder and the first decoder may be Dense Network (densenert) based encoders and decoders.
3) Reconstruction of sample images I with stitching sub-network pairs r And a defect sample image I a And splicing in the channel direction to obtain a sample spliced image.
For example, assume that sample image I is reconstructed r Dimension of (d) is mxnxh, defect sample image I a Is m × n × h, the dimension of the sample mosaic image is m × n × (2 × h).
4) And inputting the sample splicing image into a discrimination subnetwork to obtain a first detection result image Mask _ pred.
The judgment subnetwork comprises a second encoder and a second decoder, and the second encoder is adopted to encode the sample splicing image to obtain second encoding information; decoding the second coding information by adopting a second decoder to obtain a first detection result image Mask _ pred; the second encoder and the second decoder may be a densinet based encoder and decoder.
S54: and generating a sample defect detection image based on the first detection result image.
As shown in fig. 6, performing average pooling on the first detection result image Mask _ pred to obtain a second detection result image; performing maximum pooling on the second detection result image to obtain a sample defect detection image I m Sample Defect detection image I m Containing defective sample image I a Information of the defect.
S55: calculating the loss between the training sample image and the reconstructed sample image to obtain a first loss value; and calculating the loss between the sample defect detection image and the mask image to obtain a second loss value.
The loss between the training sample image and the reconstructed sample image is calculated by using a loss calculation method in the related art, such as: structural Similarity (SSIM), L1 loss function, L2 loss function, or Peak Signal to Noise Ratio (PSNR). Similarly, the loss between the sample defect inspection image and the mask image is calculated by using a loss calculation method in the related art, such as: a Focal Loss (Focal Loss) is used.
S56: and generating a current loss value based on the first loss value and the second loss value.
After the loss between the training sample image and the reconstructed sample image (namely, the first loss value) and the loss between the sample defect detection image and the mask image (namely, the second loss value) are obtained, the first loss value and the second loss value are processed to generate a current loss value.
In one embodiment, as shown in FIG. 6, the loss function used to calculate the current loss value is as follows:
L＝L rec +L focal
wherein L is rec Is a first loss value, L focal Is the second loss value.
It will be appreciated that, in addition to directly superimposing the first loss value with the second loss value to generate the current loss value, other reasonable schemes may be employed, such as: the first loss value and the second loss value are subjected to weighted summation, and the weighting coefficient can be set according to the specific application requirement; alternatively, the first loss value is multiplied by the second loss value.
S57: and judging whether the defect detection network meets a preset training ending condition or not based on the current loss value.
After the current loss value is calculated, whether the defect detection network meets a preset training end condition can be judged to determine whether the defect detection network is converged; if the defect detection network does not meet the preset training end condition, adjusting the model parameters of the defect detection network, returning to the step of acquiring the training sample image, namely returning to execute S51 until the defect detection network meets the preset training end condition. It can be understood that the scheme adopted for adjusting the model parameters is similar to that in the related art, and is not described herein again, for example: and back-propagating the current loss value to the defect detection network to adjust the model parameters.
Further, the preset training end condition includes: the loss value is converged, namely the difference value between the last loss value and the current loss value is smaller than a set value; judging whether the current loss value is smaller than a preset loss value, wherein the preset loss value is a preset loss threshold value, and if the current loss value is smaller than the preset loss value, determining that a preset training end condition is reached; training times reach a set value (for example: 10000 times of training); or the accuracy obtained when the test set is used for testing reaches a set condition (for example, the accuracy exceeds a preset accuracy), and the like.
S58: and generating a trained defect detection network.
And if the defect detection network is judged to meet the preset training end condition, the accuracy of the current defect detection network is shown to meet the requirement, the training of the defect detection network is ended at the moment, and the current defect detection network is recorded as the trained defect detection network.
In the embodiment, a training sample image and a randomly generated mask image are superposed to add defects in the training sample image to obtain a defective sample image; then, inputting the defect sample image into a defect detection network to obtain a reconstructed sample image; then, calculating a current loss value by using the sample defect detection image, the training sample image and the reconstructed sample image; when the defect detection network meets the preset training end condition, ending training the defect detection network; the embodiment adopts an unsupervised learning mode, does not need to collect images with defects in advance, can train the defect detection network by only using normal images (namely images without defects), and has the advantages of simple realization and higher detection accuracy.
Referring to fig. 7, fig. 7 is a schematic flowchart illustrating a defect detection method according to an embodiment of the present disclosure, the method including:
s71: and acquiring an image to be detected.
The target scene can be shot by adopting the camera equipment or the unmanned aerial vehicle, and the image to be detected containing the defects is obtained.
S72: and inputting the image to be detected into the trained defect detection network to obtain a defect detection image.
The image to be detected is input into the trained defect detection network, so that a defect detection image can be obtained, the defect detection image comprises the defects in the image to be detected, and the trained defect detection network is obtained by the training method of the defect detection network in the embodiment.
In a specific embodiment, use the infrared thermal imaging image of waiting to detect the image as the photovoltaic board as an example, the defect is the hot spot, in order to discern the hot spot on the photovoltaic board, can adopt following scheme:
1) And inputting the image to be detected into a reconstruction sub-network to obtain a current reconstruction image.
The reconstruction sub-network comprises a first encoder and a first decoder, and the first encoder is adopted to encode the image to be detected to obtain third encoding information; and decoding the third coding information by adopting a first decoder to obtain the current reconstructed image.
2) And splicing the current reconstructed image and the image to be detected in the channel direction by adopting a splicing sub-network to obtain a spliced image.
3) And inputting the spliced image into a discrimination subnetwork to obtain a third detection result image.
The judgment subnetwork comprises a second encoder and a second decoder, and the second encoder is adopted to encode the spliced image to obtain fourth encoding information; and decoding the fourth coded information by adopting a second decoder to obtain a third detection result image.
4) And generating a defect detection image based on the third detection result image.
Carrying out average pooling on the third detection result image to obtain a fourth detection result image; performing maximum pooling on the fourth detection result image to obtain a defect detection image; for example, as shown in fig. 8, B is a hot spot, and gfb _ dzrb is an identifier of the hot spot.
Understandably, the image to be detected can be input into an image segmentation network to obtain a segmentation image, wherein the segmentation image comprises a photovoltaic panel area and a non-photovoltaic panel area, and the photovoltaic panel area is an area where the photovoltaic panel is located; based on the photovoltaic panel area, the image to be detected is corrected, the corrected image is input into the reconstruction subnetwork, and the subsequent processing process is similar to the above embodiment and is not described again.
The embodiment provides a method for detecting the hot spots of the infrared thermal imaging image of the photovoltaic panel based on an unsupervised defect detection network, and the hot spots of the photovoltaic panel are identified by adopting a deep learning technology, so that the method has better generalization capability and is simpler to realize; moreover, due to the adoption of an unsupervised learning mode, the infrared thermal imaging image of the photovoltaic panel with the hot spots does not need to be collected in advance, the infrared thermal imaging device can still be normally used in the application scene lacking the infrared thermal imaging image of the photovoltaic panel with the hot spots, and the application range is wide.
Referring to fig. 9, fig. 9 is a schematic structural diagram of an embodiment of a defect detection apparatus provided in the present application, the defect detection apparatus 90 includes a memory 91 and a processor 92 connected to each other, the memory 91 is used for storing a computer program, and the computer program is used for implementing a training method of a defect detection network in the above-mentioned embodiment or a defect detection method in the above-mentioned embodiment when being executed by the processor 92.
Referring to fig. 10, fig. 10 is a schematic structural diagram of an embodiment of a computer-readable storage medium 100 provided in the present application, where the computer-readable storage medium 100 is used to store a computer program 101, and the computer program 101 is used to implement the training method of the defect detection network in the above-mentioned embodiment or the defect detection method in the above-mentioned embodiment when being executed by a processor.
The computer-readable storage medium 100 may be a server, a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of modules or units is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
If the technical scheme of the application relates to personal information, a product applying the technical scheme of the application clearly informs personal information processing rules before processing the personal information, and obtains personal independent consent. If the technical scheme of the application relates to sensitive personal information, before the sensitive personal information is processed, a product applying the technical scheme of the application obtains individual consent and simultaneously meets the requirement of 'explicit consent'. For example, at a personal information collection device such as a camera, a clear and significant identifier is set to inform that the personal information collection range is entered, the personal information is collected, and if the person voluntarily enters the collection range, the person is regarded as agreeing to collect the personal information; or on the device for processing the personal information, under the condition of informing the personal information processing rule by using obvious identification/information, obtaining personal authorization by modes of popping window information or asking a person to upload personal information of the person by himself, and the like; the personal information processing rule may include information such as a personal information processor, a personal information processing purpose, a processing method, and a type of personal information to be processed.
The above are only examples of the present application, and not intended to limit the scope of the present application, and all equivalent structures or equivalent processes performed by the present application and the contents of the attached drawings, which are directly or indirectly applied to other related technical fields, are also included in the scope of the present application.
Claims (10)
1. A method for training a defect detection network, comprising:
acquiring a training sample image and a defect sample image, wherein the training sample image is an image without a defect, and the defect sample image is an image generated by adding a defect into the training sample image;
inputting the defect sample image into a defect detection network to obtain a reconstructed sample image, wherein the reconstructed sample image is an image without defects;
processing the reconstructed sample image and the defect sample image by using the defect detection network to obtain a sample defect detection image, wherein the sample defect detection image is obtained by predicting defects in the defect sample image;
and training the defect detection network based on the sample defect detection image, the reconstructed sample image and the training sample image to obtain the trained defect detection network.
2. The method of claim 1, wherein the defect detection network comprises a reconstruction sub-network, a stitching sub-network, and a discrimination sub-network, and the step of inputting the defect sample image into the defect detection network to obtain the reconstructed sample image comprises:
inputting the defect sample image into the reconstruction sub-network to obtain a reconstruction sample image;
the step of processing the reconstructed sample image and the defect sample image by using the defect detection network to obtain a sample defect detection image comprises the following steps:
splicing the reconstructed sample image and the defect sample image by adopting the splicing sub-network to obtain a sample spliced image;
inputting the sample splicing image into the discrimination subnetwork to obtain a first detection result image;
and generating the sample defect detection image based on the first detection result image.
3. The method for training the defect detection network of claim 2, wherein the step of generating the sample defect detection image based on the first detection result image comprises:
performing average pooling on the first detection result image to obtain a second detection result image;
and performing maximum pooling on the second detection result image to obtain the sample defect detection image.
4. The method for training the defect detection network of claim 1, wherein the step of obtaining the training sample image and the defect sample image is preceded by:
randomly generating a mask image, wherein the mask image is an image containing defects;
and synthesizing the mask image and the training sample image to obtain the defect sample image.
5. The method according to claim 4, wherein the step of training the defect detection network based on the sample defect detection image, the reconstructed sample image, and the training sample image to obtain a trained defect detection network comprises:
calculating the loss between the training sample image and the reconstruction sample image to obtain a first loss value;
calculating the loss between the sample defect detection image and the mask image to obtain a second loss value;
generating a current loss value based on the first loss value and the second loss value;
judging whether the defect detection network meets a preset training end condition or not based on the current loss value;
if not, returning to the step of obtaining the training sample image until the defect detection network meets the preset training end condition to obtain the trained defect detection network.
6. The method for training a defect detection network of claim 1, wherein the step of obtaining a training sample image comprises:
acquiring an original image, and determining the position of a target object in the original image, wherein the original image is an image without defects;
and correcting the original image based on the position to obtain the training sample image.
7. The method according to claim 6, wherein the target object is a photovoltaic panel, an inclination angle of the photovoltaic panel in the training sample image is smaller than a preset angle, and the step of determining the position of the target object in the original image comprises:
inputting the original image into an image segmentation network to obtain a sample segmentation image;
the sample segmentation image comprises a photovoltaic panel area and a non-photovoltaic panel area, and the photovoltaic panel area is the area where the photovoltaic panel is located.
8. A method of defect detection, comprising:
acquiring an image to be detected, wherein the image to be detected is an image containing defects;
inputting the image to be detected into a trained defect detection network to obtain a defect detection image;
wherein the trained defect detection network is obtained by the training method of the defect detection network according to any one of claims 1 to 7.
9. A defect detection apparatus, comprising a memory and a processor connected to each other, wherein the memory is configured to store a computer program, which when executed by the processor is configured to implement the training method of the defect detection network according to any one of claims 1 to 7 or the defect detection method according to claim 8.
10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, is configured to implement the method for training a defect detection network according to any one of claims 1 to 7 or the method for defect detection according to claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210665933.7A CN115240023A (en) | 2022-06-13 | 2022-06-13 | Training method of defect detection network, defect detection method and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210665933.7A CN115240023A (en) | 2022-06-13 | 2022-06-13 | Training method of defect detection network, defect detection method and related equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115240023A true CN115240023A (en) | 2022-10-25 |
Family
ID=83670225
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210665933.7A Pending CN115240023A (en) | 2022-06-13 | 2022-06-13 | Training method of defect detection network, defect detection method and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115240023A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115439721A (en) * | 2022-11-08 | 2022-12-06 | 南方电网数字电网研究院有限公司 | Method and device for training classification model of few abnormal sample defects of power equipment |
| CN116030038A (en) * | 2023-02-23 | 2023-04-28 | 季华实验室 | Unsupervised OLED defect detection method based on defect generation |
| CN116188917A (en) * | 2023-04-24 | 2023-05-30 | 苏州苏映视图像软件科技有限公司 | Defect data generation model training method, defect data generation method and device |
-
2022
- 2022-06-13 CN CN202210665933.7A patent/CN115240023A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115439721A (en) * | 2022-11-08 | 2022-12-06 | 南方电网数字电网研究院有限公司 | Method and device for training classification model of few abnormal sample defects of power equipment |
| CN116030038A (en) * | 2023-02-23 | 2023-04-28 | 季华实验室 | Unsupervised OLED defect detection method based on defect generation |
| CN116188917A (en) * | 2023-04-24 | 2023-05-30 | 苏州苏映视图像软件科技有限公司 | Defect data generation model training method, defect data generation method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115240023A (en) | Training method of defect detection network, defect detection method and related equipment | |
| CN111784633B (en) | Insulator defect automatic detection algorithm for electric power inspection video | |
| CN110751630B (en) | Power transmission line foreign matter detection method and device based on deep learning and medium | |
| CN103581660B (en) | Line pair based full field sharpness test method and system | |
| CN111383209A (en) | Unsupervised flaw detection method based on full convolution self-encoder network | |
| JP2003501850A (en) | Method and apparatus for estimating digital image quality without using reference image | |
| Ghosh et al. | Quantitative evaluation of image mosaicing in multiple scene categories | |
| CN106683040B (en) | Infrared panoramic image splicing method based on NCC algorithm | |
| CN103093458A (en) | Detecting method and detecting device for key frame | |
| CN111696049A (en) | Deep learning-based underwater distorted image reconstruction method | |
| Li et al. | Image quality assessment using deep convolutional networks | |
| Babu et al. | An efficient image dahazing using Googlenet based convolution neural networks | |
| CN115272340B (en) | Industrial product defect detection method and device | |
| KR101559724B1 (en) | Method and Apparatus for Detecting the Bad Pixels in Sensor Array and Concealing the Error | |
| CN116664446A (en) | Lightweight dim light image enhancement method based on residual error dense block | |
| CN115393743A (en) | Vehicle detection method based on double-branch encoding and decoding network, unmanned aerial vehicle and medium | |
| CN113628125B (en) | Method for enhancing multiple infrared images based on space parallax priori network | |
| CN112950592B (en) | Non-reference light field image quality evaluation method based on high-dimensional discrete cosine transform | |
| CN113160104A (en) | Image fusion method based on dense connection network | |
| CN113887489A (en) | Carriage crowd counting method based on position enhancement and multi-scale fusion network | |
| Anitha et al. | Quality assessment of resultant images after processing | |
| CN117041531B (en) | Mobile phone camera focusing detection method and system based on image quality evaluation | |
| CN117893989B (en) | Sequential picture tracing method and system based on panoramic automobile data recorder | |
| CN113902739B (en) | NUT wire clamp defect identification method, device and equipment and readable storage medium | |
| CN116485802B (en) | Insulator flashover defect detection method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |