CN114627122A - Defect detection method and device - Google Patents
Defect detection method and device Download PDFInfo
- Publication number
- CN114627122A CN114627122A CN202210526865.6A CN202210526865A CN114627122A CN 114627122 A CN114627122 A CN 114627122A CN 202210526865 A CN202210526865 A CN 202210526865A CN 114627122 A CN114627122 A CN 114627122A
- Authority
- CN
- China
- Prior art keywords
- defect
- appearance
- sample
- image
- defect detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Abstract
The invention provides a defect detection method and a device, comprising the following steps: acquiring an EL image of a unit to be detected of each unit cell in a target photovoltaic assembly, and acquiring an appearance image of the unit to be detected of each unit cell; inputting all EL images of the units to be detected into a first defect detection model, determining a first defect type and first position information output by the first defect detection model, inputting all appearance images of the units to be detected into a second defect detection model, and determining a second defect type and second position information output by the second defect detection model. According to the defect detection method and device provided by the invention, the defect detection is carried out on the EL image and the appearance image of each cell on the photovoltaic module through the defect detection model, so that more defect types can be detected while the defect detection accuracy and the recognition efficiency are improved.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a defect detection method.
Background
The production process of the solar cell is complex, and the defects caused by thermal stress, mechanical stress and the like in the production and installation processes of the component directly influence the conversion efficiency and the service life of the solar cell, so that the detection of the defects of the component is an essential link in the production process of the photovoltaic component.
With the development of deep learning, more and more defect detection methods based on deep learning are proposed and applied, for example, a multi-spectral line convolution neural network detection method, or a multi-feature fusion detection method based on a convolution neural network, or a detection scheme of a solar cell module which generates a countermeasure network and combines the convolution neural network, so as to detect defects on the surface of a solar cell.
The existing detection technology cannot give consideration to both precision and efficiency.
Disclosure of Invention
The invention provides a defect detection method and a defect detection device, which are used for solving the defect that the precision and the efficiency cannot be considered simultaneously in the prior art and improving the precision and the identification efficiency of defect detection.
The invention provides a defect detection method, which comprises the following steps:
acquiring an EL image of a unit to be detected of each unit cell in a target photovoltaic assembly, and acquiring an appearance image of the unit to be detected of each unit cell;
inputting all EL images of the units to be detected into a first defect detection model, determining a first defect type and first position information output by the first defect detection model, inputting all appearance images of the units to be detected into a second defect detection model, and determining a second defect type and second position information output by the second defect detection model;
the first defect detection model is obtained after the sample unit EL image with the EL defect label is trained, and the second defect detection model is obtained after the sample unit appearance image with the appearance defect label is trained.
According to the defect detection method provided by the present invention, the first defect detection model and the second defect detection model are both constructed based on the YOLOv5 network, and before inputting all the unit EL images to be detected into the first defect detection model, the method further includes:
acquiring a plurality of sample unit EL images;
determining an EL defect label corresponding to each sample unit EL image, wherein the EL defect label comprises an EL defect category label and an EL position information label;
combining each sample unit EL image and the corresponding EL defect label into an EL training sample to obtain a plurality of EL training samples;
training the first defect detection model using the plurality of EL training samples;
before the inputting all appearance images of the unit to be detected to the second defect detection model, the method further comprises:
acquiring a plurality of sample unit appearance images;
determining an appearance defect label corresponding to each sample unit appearance image, wherein the appearance defect label comprises an appearance defect category label and an appearance position information label;
combining the appearance image of each sample unit and the appearance defect label corresponding to the appearance image of each sample unit into an appearance training sample to obtain a plurality of appearance training samples;
training the second defect detection model using the plurality of appearance training samples.
According to a defect detection method provided by the present invention, the acquiring a plurality of sample unit EL images includes:
acquiring a sample initial EL image of a sample photovoltaic assembly;
determining a sliding window specification and a sliding window step length according to the size of each sample unit cell in the sample photovoltaic assembly in the sample initial EL image;
and segmenting the sample initial EL image based on the sliding window specification and the sliding window step length to determine a sample unit EL image of each sample unit cell.
According to a defect detection method provided by the invention, the acquiring of the appearance images of the plurality of sample units comprises the following steps:
obtaining a sample initial appearance image of the sample photovoltaic assembly;
carrying out average segmentation on the sample initial appearance image to obtain a plurality of sample appearance sub-images;
inputting the plurality of sample appearance sub-images into an image segmentation model, and acquiring a sample unit appearance image of each sample unit cell output by the image segmentation model; the image segmentation model is obtained by training a sample appearance image with an image segmentation label.
According to the defect detection method provided by the invention, the first defect type comprises EL defect types corresponding to all EL defects in EL images of units to be detected, and the first position information comprises target frame position information of each EL defect;
after the determining the first defect type and the first position information output by the first defect detection model, further comprising:
determining a first serial number of a unit cell where each EL defect is located;
determining coordinate information and area information of each EL defect on the target photovoltaic module according to the first serial number and the first position information;
the second defect type comprises appearance defect types corresponding to all appearance defects in the appearance images of the units to be detected, and the second position information comprises the position information of a target frame of each appearance defect;
after the determining the second defect type and the second position information output by the second defect detection model, further comprising:
determining a second serial number of each unit cell where the appearance defect is located;
and determining coordinate information and area information of each appearance defect on the target photovoltaic module according to the second serial number and the second position information.
According to a defect detection method provided by the present invention, the training of the first defect detection model by using the plurality of EL training samples includes:
inputting any EL training sample into the first defect detection model, and outputting a predicted EL defect type and predicted EL position information corresponding to the EL training sample;
calculating an EL loss value according to the predicted EL defect type and the predicted EL position information corresponding to any EL training sample and the EL defect label in any EL training sample by using a preset loss function;
if the EL loss value is smaller than a first preset threshold value, the first defect detection model is trained;
the training the second defect detection model using the plurality of appearance training samples includes:
inputting any appearance training sample into the second defect detection model, and outputting the predicted appearance defect type and the predicted appearance position information corresponding to the any appearance training sample;
calculating an appearance loss value according to the predicted appearance defect type and the predicted appearance position information corresponding to any appearance training sample and an appearance defect label in any appearance training sample by using a preset loss function;
and if the appearance loss value is smaller than a second preset threshold value, finishing the training of the second defect detection model.
According to a defect detection method provided by the present invention, the segmenting the sample initial EL image based on the sliding window specification and the sliding window step length to determine the sample unit EL image of each sample unit cell includes:
based on the sliding window specification and the sliding window step length, segmenting the sample initial EL image to obtain a sample initial EL subimage of each unit cell;
removing gray level self-interference of each sample initial EL sub-image, and obtaining a plurality of sample interference-removed EL images;
and performing noise reduction processing on each sample interference-removed EL image to obtain a plurality of sample unit EL images.
The present invention also provides a defect detecting apparatus, comprising:
the device comprises an acquisition module, a detection module and a display module, wherein the acquisition module is used for acquiring an EL image of a unit to be detected of each unit cell in a target photovoltaic assembly and acquiring an appearance image of the unit to be detected of each unit cell;
the detection module is used for inputting all the EL images of the units to be detected into a first defect detection model, determining a first defect type and first position information output by the first defect detection model, inputting all the appearance images of the units to be detected into a second defect detection model, and determining a second defect type and second position information output by the second defect detection model;
the first defect detection model is obtained after the sample unit EL image with the EL defect label is trained, and the second defect detection model is obtained after the sample unit appearance image with the appearance defect label is trained.
The present invention also provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the above-mentioned defect detection methods.
The invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a defect detection method as described in any one of the above.
The invention also provides a computer program product comprising a computer program which, when executed by a processor, implements a method of defect detection as described in any one of the above.
According to the defect detection method and device provided by the invention, the defect detection is carried out on the EL image and the appearance image of each cell on the photovoltaic module through the defect detection model, so that more defect types can be detected while the defect detection accuracy and the recognition efficiency are improved.
Drawings
In order to more clearly illustrate the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic flow chart of a defect detection method provided by the present invention;
FIG. 2 is a diagram of a segmentation result output by the image segmentation model provided by the present invention;
FIG. 3 is a schematic spacing diagram of an appearance image of a photovoltaic module provided by the present invention;
FIG. 4 is a schematic structural diagram of a defect detection apparatus provided in the present invention;
fig. 5 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The machine vision detection has the advantages of high production efficiency, high automation level, good detection rate, strong adaptability to special industrial environments and the like. Therefore, the defect detection technology based on machine vision is studied for detecting the surface defects of the object.
Although the multi-spectral line convolution neural network can detect the cell on the surface of the solar cell, the requirement for intelligent detection of Electro Luminescence (EL) is not met.
The surface defect detection of the solar cell based on the convolution neural network and the multi-feature fusion is high in parameter quantity and low in detection efficiency.
The detection scheme of the solar cell module combining the generation countermeasure Network and the Convolutional Neural Network is that firstly, the generation countermeasure Network (GAN) is used for generating an image close to a real defect so as to expand a data set, and finally, the Convolutional Neural Network (CNN) is used for realizing classification detection of the defect. However, the deep learning method-based photovoltaic module cell defect positioning is less in research and lacks of completeness of research.
The existing deep learning-based method has the problem that the efficiency of defect detection of the solar cell is influenced due to the large parameter quantity of the convolutional neural network; most of solar cell defect detection technologies based on deep learning do not have the function of defect positioning, so that the completeness of research is deficient. Meanwhile, the recognition capability of the illumination non-uniform image and the fuzzy image with poor quality is weak, and the robustness of the model is poor.
The defect detection method and apparatus provided by the embodiment of the invention are described below with reference to fig. 1 to 5.
Fig. 1 is a schematic flow chart of a defect detection method provided by the present invention, as shown in fig. 1, including but not limited to the following steps:
first, in step S1, a cell-under-test EL image of each unit cell in the target photovoltaic module is acquired, and a cell-under-test appearance image of each unit cell is acquired.
The target photovoltaic module is shot by utilizing image acquisition equipment in an actual industrial field, and a target EL image and a target appearance image of the target photovoltaic module are obtained. The image acquisition equipment can be arranged beside each photovoltaic assembly and is used for acquiring an EL image and an appearance image of the target photovoltaic assembly in real time; and the target photovoltaic module can be brought to an environment with sufficient light and favorable for shooting, and the picture quality is improved.
By cropping the target EL image and the target appearance image, the unit-under-test EL image of each unit cell and the unit-under-test appearance image of each unit cell can be obtained.
Further, in step S2, inputting all the unit-under-test EL images to a first defect detection model, determining a first defect type and first position information output by the first defect detection model, and inputting all the unit-under-test appearance images to a second defect detection model, determining a second defect type and second position information output by the second defect detection model; the first defect detection model is obtained after the sample unit EL image with the EL defect label is trained, and the second defect detection model is obtained after the sample unit appearance image with the appearance defect label is trained.
The first defect detection model and the second defect detection model are both constructed based on a single-stage target detection algorithm (YOLO series).
After any unit to be detected EL image is input into the first defect detection model, the EL defect type corresponding to the unit to be detected EL image output by the first defect detection model and the position information of the target frame for detecting the defect on the unit to be detected EL image can be obtained. And taking the defect types corresponding to all the EL images of the unit to be detected as first defect types, and taking all the position information of the target frame as first position information.
The EL defect categories may include: insufficient solder, black spots, broken grids, overwelding, scratches, broken pieces, hidden cracks of wire mounting, hidden cracks of fork shape, hidden cracks of net shape, hidden cracks of penetration, hidden cracks of line shape, short circuit, no power supply, bright spots, poor splicing and the like.
After the appearance image of any unit to be detected is input into the second defect detection model, the appearance defect category corresponding to the appearance image of the unit to be detected output by the second defect detection model and the position information of the target frame for detecting the defect on the appearance image of the unit to be detected can be obtained. And taking the defect types corresponding to all the appearance images of the units to be detected as second defect types, and taking all the position information of the target frame as second position information.
The appearance defect categories may include: foreign matter, dirt, fragments, white spots, bad spacing and the like.
According to the defect detection method provided by the invention, the defect detection is carried out on the EL image and the appearance image of each cell on the photovoltaic module through the defect detection model, so that more defect types can be detected while the defect detection accuracy and the recognition efficiency are improved.
Optionally, the first defect type includes EL defect types corresponding to EL defects in all EL images of the unit to be tested, and the first position information includes target frame position information of each EL defect;
after the determining the first defect type and the first position information output by the first defect detection model, further comprising:
determining a first serial number of a unit cell where each EL defect is located;
determining coordinate information and area information of each EL defect on the target photovoltaic module according to the first serial number and the first position information;
the second defect type comprises appearance defect types corresponding to all appearance defects in the appearance images of the units to be detected, and the second position information comprises the position information of a target frame of each appearance defect;
after the determining the second defect type and the second position information output by the second defect detection model, further comprising:
determining a second serial number of each unit cell where the appearance defect is located;
and determining coordinate information and area information of each appearance defect on the target photovoltaic module according to the second serial number and the second position information.
The serial numbers of all the unit cells where the EL defects are located are first serial numbers, the first serial numbers and a target frame in the first position information are mapped to a target EL image, and the coordinate position and the ratio area of each EL defect on the target photovoltaic module on the target EL image are determined; the serial numbers of the unit cells where all the appearance defects are located are second serial numbers, the second serial numbers and a target frame in second position information are mapped to a target appearance image, and the coordinate position and the proportion area of each appearance defect on the target photovoltaic module on the target appearance image are determined; and finally obtaining a defect detection result of the target photovoltaic module.
According to the defect detection method provided by the invention, the area ratio of each defect can be calculated according to the detection result, and a basis can be further provided for subsequently judging the component grade.
Optionally, the acquiring a plurality of sample unit EL images includes:
acquiring a sample initial EL image of a sample photovoltaic assembly;
determining a sliding window specification and a sliding window step length according to the size of each sample unit cell in the sample photovoltaic assembly in the sample initial EL image;
and segmenting the sample initial EL image based on the sliding window specification and the sliding window step length to determine a sample unit EL image of each sample unit cell.
Optionally, the segmenting the sample initial EL image based on the sliding window specification and the sliding window step size to determine the sample unit EL image of each sample unit cell slice includes:
based on the sliding window specification and the sliding window step length, segmenting the sample initial EL image to obtain a sample initial EL subimage of each unit cell;
removing gray level self-interference of each sample initial EL sub-image, and obtaining a plurality of sample interference-removed EL images;
and performing noise reduction processing on each sample interference-removed EL image to obtain a plurality of sample unit EL images.
According to the characteristics of a sample initial EL image of a sample photovoltaic module, the sliding window specification and the sliding window step length are determined according to the pixel size of a unit cell piece by adopting equal pixel area, the number of the unit cell pieces of the sample initial EL image in the transverse direction and the longitudinal direction is cut one by one from left to right according to a sliding window method, and the cutting is executed according to the sequence from top to bottom, in order to avoid the influence of uncertain spacing distance between the half-sheet assemblies on the left side and the right side, the right half-sheet assembly is cut according to a method from right to left, the cut image is arranged in the reverse sequence, the sequence from left to right is obtained, and the sequence from left to right can be obtained
A sample initial EL sub-image of the cell slice with the sequence.
Carrying out gray level histogram detection on each sample initial EL sub-image, and removing the gray level self-carried interference of the image according to the detection result to obtain a plurality of sample interference-removed EL images; and carrying out bilateral filtering on each sample interference-removing EL sub-image, removing noise in the sample interference-removing EL sub-image, and obtaining a plurality of sample unit EL images.
According to the defect detection method provided by the invention, the accuracy of small target detection is improved based on small picture detection after image segmentation, the parameter quantity is small, the detection efficiency is high, the deployment to a mobile terminal is facilitated, and the model has light weight.
Optionally, the obtaining a plurality of sample cell appearance images includes:
acquiring a sample initial appearance image of the sample photovoltaic module;
carrying out average segmentation on the sample initial appearance image to obtain a plurality of sample appearance sub-images;
inputting the plurality of sample appearance sub-images into an image segmentation model, and acquiring a sample unit appearance image of each sample unit cell output by the image segmentation model; the image segmentation model is obtained by training a sample appearance image with an image segmentation label.
The image segmentation model may be based on U2-NET network construction.
The method comprises the steps of averagely dividing a sample initial appearance image into 6 sample appearance sub-images according to the pixel area, further dividing each unit cell by using an image division model on the basis of the sample appearance sub-images, further carrying out contour detection by using a find contacts function in OpenCV, filtering the contour by adopting a certain method, further extracting the vertex coordinates of the unit cells, and sequencing according to the coordinates of the unit cells to obtain a sample unit appearance image of each unit cell with a sequence. The pixel coordinates can be used for calculating the distance between the battery pieces so as to judge whether the poor spacing defect exists.
Fig. 2 is a schematic diagram of a segmentation result output by the image segmentation model provided by the present invention, and as shown in fig. 2, the image segmentation model segments an input sample appearance sub-image into a plurality of sample cell appearance images according to the boundaries of the cell slices.
According to the defect detection method provided by the invention, the problems of larger image resolution and smaller defect target can be solved by segmenting and cutting the image, after cutting, the relative proportion of the target defect in the image can be greatly increased, the characteristic attenuation and even loss during deep learning down-sampling can be avoided, and the accuracy and recall rate of the detection model can be improved.
Optionally, the first defect detection model and the second defect detection model are both constructed based on YOLOv5 network, and before the inputting all the EL images of the unit under test into the first defect detection model, the method further includes:
acquiring a plurality of sample unit EL images;
determining an EL defect label corresponding to each sample unit EL image, wherein the EL defect label comprises an EL defect category label and an EL position information label;
combining each sample unit EL image and the corresponding EL defect label into an EL training sample to obtain a plurality of EL training samples;
training the first defect detection model using the plurality of EL training samples;
before the inputting all appearance images of the unit to be detected to the second defect detection model, the method further comprises:
acquiring a plurality of sample unit appearance images;
determining an appearance defect label corresponding to each sample unit appearance image, wherein the appearance defect label comprises an appearance defect category label and an appearance position information label;
combining the appearance image of each sample unit and the appearance defect label corresponding to the appearance image of each sample unit into an appearance training sample to obtain a plurality of appearance training samples;
training the second defect detection model using the plurality of appearance training samples.
Since the defect type and location information corresponding to each EL training sample and appearance training sample are known and have been labeled by defect labels.
The sample unit EL image and the sample appearance image after cutting are labeled with the defect type and the defect position by using a label tool.
The EL defect categories may include: insufficient solder, black spots, broken grids, overwelding, scratches, broken pieces, hidden cracks of wire mounting, hidden cracks of fork shape, hidden cracks of net shape, hidden cracks of penetration, hidden cracks of line shape, short circuit, no power supply, bright spots, poor splicing and the like.
The appearance defect categories may include: foreign matter, dirt, fragments, white spots, bad spacing and the like.
Fig. 3 is a schematic pitch diagram of an appearance image of a photovoltaic module according to the present invention, as shown in fig. 3, the photovoltaic module includes a unit cell sheet and a bus bar, and the pitch includes: busbar-to-cell spacing, half-cell spacing, and string spacing.
According to the defect detection method provided by the invention, the YOLOv5 target detection model is used for detecting the defect flaws of the EL image and the appearance image, the defect detection method has certain detection capability under the conditions of poor illumination and fuzzy image, the model has robustness, end-to-end real-time online detection is realized, and the flaw detection can be accurately and efficiently carried out on the photovoltaic assembly.
Optionally, the training the first defect detection model with the plurality of EL training samples includes:
inputting any EL training sample into the first defect detection model, and outputting a predicted EL defect type and predicted EL position information corresponding to the EL training sample;
calculating an EL loss value according to the predicted EL defect type and the predicted EL position information corresponding to any EL training sample and the EL defect label in any EL training sample by using a preset loss function;
if the EL loss value is smaller than a first preset threshold value, the first defect detection model is trained;
the training the second defect detection model using the plurality of appearance training samples includes:
inputting any appearance training sample into the second defect detection model, and outputting the predicted appearance defect type and the predicted appearance position information corresponding to the any appearance training sample;
calculating an appearance loss value according to the predicted appearance defect type and the predicted appearance position information corresponding to any appearance training sample and an appearance defect label in any appearance training sample by using a preset loss function;
and if the appearance loss value is smaller than a second preset threshold value, finishing the training of the second defect detection model.
The plurality of EL training samples may be divided into a training set, a validation set, and a test set to implement training of the first defect detection model.
And sequentially inputting the plurality of EL training samples into the first defect detection model, namely simultaneously inputting the sample unit EL image in each EL training sample and the EL defect label corresponding to the sample unit EL image into the first defect detection model, adjusting the model parameters in the first defect detection model according to each output result of the first defect detection model, and finally finishing the training process of the first defect detection model.
The plurality of appearance training samples can be divided into a training set, a verification set and a test set, so that the second defect detection model can be trained.
And sequentially inputting the plurality of appearance training samples into a second defect detection model, namely simultaneously inputting the sample unit appearance images and the appearance defect labels corresponding to the sample unit appearance images in each appearance training sample into the second defect detection model, adjusting model parameters in the second defect detection model according to each output result of the second defect detection model, and finally finishing the training process of the second defect detection model.
The following describes the defect detection apparatus provided by the present invention, and the defect detection apparatus described below and the defect detection method described above can be referred to correspondingly.
Fig. 4 is a schematic structural diagram of a defect detection apparatus provided by the present invention, as shown in fig. 4, including:
an obtaining module 401, configured to obtain an EL image of a unit to be detected of each unit cell in a target photovoltaic assembly, and obtain an appearance image of the unit to be detected of each unit cell;
a detection module 402, configured to input all EL images of the unit to be detected to a first defect detection model, determine a first defect type and first position information output by the first defect detection model, input all appearance images of the unit to be detected to a second defect detection model, and determine a second defect type and second position information output by the second defect detection model; the first defect detection model is obtained after the sample unit EL image with the EL defect label is trained, and the second defect detection model is obtained after the sample unit appearance image with the appearance defect label is trained.
First, the obtaining module 401 obtains an image of a unit to be measured EL of each unit cell in the target photovoltaic module, and obtains an appearance image of the unit to be measured of each unit cell.
The target photovoltaic module is shot by utilizing image acquisition equipment in an actual industrial field, and a target EL image and a target appearance image of the target photovoltaic module are obtained. The image acquisition equipment can be arranged beside each photovoltaic module and is used for acquiring an EL image and an appearance image of the target photovoltaic module in real time; and the target photovoltaic module can be brought to an environment with sufficient light and favorable for shooting, so that the picture quality is improved.
By cropping the target EL image and the target appearance image, the unit-under-test EL image of each unit cell and the unit-under-test appearance image of each unit cell can be obtained.
Further, the detection module 402 inputs all EL images of the unit to be detected to a first defect detection model, determines a first defect type and first position information output by the first defect detection model, and inputs all appearance images of the unit to be detected to a second defect detection model, determines a second defect type and second position information output by the second defect detection model; the first defect detection model is obtained by training sample unit EL images with EL defect labels, and the second defect detection model is obtained by training sample unit appearance images with appearance defect labels.
The first defect detection model and the second defect detection model are both constructed based on a single-stage target detection algorithm (YOLO series).
After any unit to be detected EL image is input into the first defect detection model, the EL defect type corresponding to the unit to be detected EL image output by the first defect detection model and the position information of the target frame for detecting the defect on the unit to be detected EL image can be obtained. And taking the defect types corresponding to all the EL images of the unit to be detected as first defect types, and taking all the position information of the target frame as first position information.
The EL defect categories may include: insufficient solder, black spots, broken grids, overwelding, scratches, broken pieces, hidden cracks of wire mounting, hidden cracks of fork shape, hidden cracks of net shape, hidden cracks of penetration, hidden cracks of line shape, short circuit, no power supply, bright spots, poor splicing and the like.
After the appearance image of any unit to be detected is input into the second defect detection model, the appearance defect category corresponding to the appearance image of the unit to be detected output by the second defect detection model and the position information of the target frame for detecting the defect on the appearance image of the unit to be detected can be obtained. And taking the defect types corresponding to all the appearance images of the units to be detected as second defect types, and taking all the position information of the target frame as second position information.
The appearance defect categories may include: foreign matter, dirt, fragments, white spots, bad spacing and the like.
According to the defect detection device provided by the invention, the defect detection is carried out on the EL image and the appearance image of each cell on the photovoltaic module through the defect detection model, so that more defect types can be detected while the defect detection accuracy and the recognition efficiency are improved.
Fig. 5 is a schematic structural diagram of an electronic device provided in the present invention, and as shown in fig. 5, the electronic device may include: a processor (processor)510, a communication Interface (Communications Interface)520, a memory (memory)530 and a communication bus 540, wherein the processor 510, the communication Interface 520 and the memory 530 communicate with each other via the communication bus 540. Processor 510 may call logic instructions in memory 530 to perform a defect detection method comprising: acquiring an EL image of a unit to be detected of each unit cell in a target photovoltaic assembly, and acquiring an appearance image of the unit to be detected of each unit cell; inputting all EL images of the units to be detected into a first defect detection model, determining a first defect type and first position information output by the first defect detection model, inputting all appearance images of the units to be detected into a second defect detection model, and determining a second defect type and second position information output by the second defect detection model; the first defect detection model is obtained after the sample unit EL image with the EL defect label is trained, and the second defect detection model is obtained after the sample unit appearance image with the appearance defect label is trained.
Furthermore, the logic instructions in the memory 530 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being storable on a non-transitory computer-readable storage medium, the computer program, when executed by a processor, being capable of executing the defect detection method provided by the above methods, the method comprising: acquiring an EL image of a unit to be detected of each unit cell in a target photovoltaic module, and acquiring an appearance image of the unit to be detected of each unit cell; inputting all EL images of the units to be detected into a first defect detection model, determining a first defect type and first position information output by the first defect detection model, inputting all appearance images of the units to be detected into a second defect detection model, and determining a second defect type and second position information output by the second defect detection model; the first defect detection model is obtained after the sample unit EL image with the EL defect label is trained, and the second defect detection model is obtained after the sample unit appearance image with the appearance defect label is trained.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which when executed by a processor, implements a method for defect detection provided by the above methods, the method comprising: acquiring an EL image of a unit to be detected of each unit cell in a target photovoltaic assembly, and acquiring an appearance image of the unit to be detected of each unit cell; inputting all EL images of the units to be detected into a first defect detection model, determining a first defect type and first position information output by the first defect detection model, inputting all appearance images of the units to be detected into a second defect detection model, and determining a second defect type and second position information output by the second defect detection model; the first defect detection model is obtained after the sample unit EL image with the EL defect label is trained, and the second defect detection model is obtained after the sample unit appearance image with the appearance defect label is trained.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method of defect detection, comprising:
acquiring an EL image of a unit to be detected of each unit cell in a target photovoltaic assembly, and acquiring an appearance image of the unit to be detected of each unit cell;
inputting all EL images of the units to be detected into a first defect detection model, determining a first defect type and first position information output by the first defect detection model, inputting all appearance images of the units to be detected into a second defect detection model, and determining a second defect type and second position information output by the second defect detection model;
the first defect detection model is obtained after the sample unit EL image with the EL defect label is trained, and the second defect detection model is obtained after the sample unit appearance image with the appearance defect label is trained.
2. The method of claim 1, wherein the first defect detection model and the second defect detection model are both constructed based on a YOLOv5 network, and before the inputting all the EL images of the unit under test into the first defect detection model, the method further comprises:
acquiring a plurality of sample unit EL images;
determining an EL defect label corresponding to each sample unit EL image, wherein the EL defect label comprises an EL defect category label and an EL position information label;
combining each sample unit EL image and the corresponding EL defect label into an EL training sample to obtain a plurality of EL training samples;
training the first defect detection model using the plurality of EL training samples;
before the inputting all appearance images of the unit to be detected to the second defect detection model, the method further comprises:
acquiring a plurality of sample unit appearance images;
determining an appearance defect label corresponding to each sample unit appearance image, wherein the appearance defect label comprises an appearance defect category label and an appearance position information label;
combining the appearance image of each sample unit and the appearance defect label corresponding to the appearance image of each sample unit into an appearance training sample to obtain a plurality of appearance training samples;
training the second defect detection model using the plurality of appearance training samples.
3. The defect detection method of claim 2, wherein said obtaining a plurality of sample unit EL images comprises:
acquiring a sample initial EL image of a sample photovoltaic assembly;
determining a sliding window specification and a sliding window step length according to the size of each sample unit cell in the sample photovoltaic assembly in the sample initial EL image;
and segmenting the sample initial EL image based on the sliding window specification and the sliding window step length to determine a sample unit EL image of each sample unit cell.
4. The defect detection method of claim 3, wherein said obtaining a plurality of sample cell appearance images comprises:
obtaining a sample initial appearance image of the sample photovoltaic assembly;
carrying out average segmentation on the sample initial appearance image to obtain a plurality of sample appearance sub-images;
inputting the plurality of sample appearance sub-images into an image segmentation model, and acquiring a sample unit appearance image of each sample unit cell output by the image segmentation model; the image segmentation model is obtained by training a sample appearance image with an image segmentation label.
5. The defect detection method according to claim 1, wherein the first defect category includes EL defect categories corresponding to EL defects in all EL images of the unit under test, and the first position information includes target frame position information of each EL defect;
after the determining the first defect type and the first position information output by the first defect detection model, further comprising:
determining a first serial number of a unit cell where each EL defect is located;
determining coordinate information and area information of each EL defect on the target photovoltaic module according to the first serial number and the first position information;
the second defect type comprises appearance defect types corresponding to all appearance defects in the appearance images of the units to be detected, and the second position information comprises the position information of a target frame of each appearance defect;
after the determining the second defect type and the second position information output by the second defect detection model, further comprising:
determining a second serial number of each unit cell where the appearance defect is located;
and determining coordinate information and area information of each appearance defect on the target photovoltaic module according to the second serial number and the second position information.
6. The defect detection method of claim 2, wherein said training the first defect detection model with the plurality of EL training samples comprises:
inputting any EL training sample into the first defect detection model, and outputting a predicted EL defect type and predicted EL position information corresponding to the EL training sample;
calculating an EL loss value according to the predicted EL defect type and the predicted EL position information corresponding to any EL training sample and the EL defect label in any EL training sample by using a preset loss function;
if the EL loss value is smaller than a first preset threshold value, the first defect detection model is trained;
the training the second defect detection model using the plurality of appearance training samples comprises:
inputting any appearance training sample into the second defect detection model, and outputting the predicted appearance defect type and the predicted appearance position information corresponding to the any appearance training sample;
calculating an appearance loss value according to the predicted appearance defect type and the predicted appearance position information corresponding to any appearance training sample and an appearance defect label in any appearance training sample by using a preset loss function;
and if the appearance loss value is smaller than a second preset threshold value, finishing the training of the second defect detection model.
7. The defect detection method of claim 3, wherein the segmenting the sample initial EL image based on the sliding window specification and the sliding window step size to determine the sample unit EL image of each sample unit cell comprises:
based on the sliding window specification and the sliding window step length, segmenting the sample initial EL image to obtain a sample initial EL subimage of each unit cell;
removing gray self-interference of each sample initial EL sub-image to obtain a plurality of sample interference-removed EL images;
and carrying out noise reduction processing on each sample interference-removed EL image to obtain a plurality of sample unit EL images.
8. A defect detection apparatus, comprising:
the device comprises an acquisition module, a detection module and a display module, wherein the acquisition module is used for acquiring an EL image of a unit to be detected of each unit cell in a target photovoltaic assembly and acquiring an appearance image of the unit to be detected of each unit cell;
the detection module is used for inputting all the EL images of the units to be detected into a first defect detection model, determining a first defect type and first position information output by the first defect detection model, inputting all the appearance images of the units to be detected into a second defect detection model, and determining a second defect type and second position information output by the second defect detection model;
the first defect detection model is obtained after the sample unit EL image with the EL defect label is trained, and the second defect detection model is obtained after the sample unit appearance image with the appearance defect label is trained.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the defect detection method according to any one of claims 1 to 7 when executing the program.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the defect detection method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210526865.6A CN114627122A (en) | 2022-05-16 | 2022-05-16 | Defect detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210526865.6A CN114627122A (en) | 2022-05-16 | 2022-05-16 | Defect detection method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114627122A true CN114627122A (en) | 2022-06-14 |
Family
ID=81907080
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210526865.6A Pending CN114627122A (en) | 2022-05-16 | 2022-05-16 | Defect detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114627122A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115661466A (en) * | 2022-12-14 | 2023-01-31 | 尚特杰电力科技有限公司 | Photovoltaic panel positioning method and device based on deep learning image segmentation |
| CN116503412A (en) * | 2023-06-29 | 2023-07-28 | 宁德时代新能源科技股份有限公司 | Appearance defect detection method, apparatus, computer device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110473806A (en) * | 2019-07-13 | 2019-11-19 | 河北工业大学 | The deep learning identification of photovoltaic cell sorting and control method and device |
| CN111275679A (en) * | 2020-01-17 | 2020-06-12 | 同济大学 | Solar cell defect detection system and method based on image |
| CN113962929A (en) * | 2021-09-07 | 2022-01-21 | 湖南大学 | Photovoltaic cell assembly defect detection method and system and photovoltaic cell assembly production line |
| CN113989241A (en) * | 2021-10-29 | 2022-01-28 | 南京埃斯顿机器人工程有限公司 | Photovoltaic module EL defect detection method based on image processing and deep learning fusion |
| CN114399484A (en) * | 2022-01-11 | 2022-04-26 | 华电电力科学研究院有限公司 | Photovoltaic module defect detection method, device, equipment and medium |
-
2022
- 2022-05-16 CN CN202210526865.6A patent/CN114627122A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110473806A (en) * | 2019-07-13 | 2019-11-19 | 河北工业大学 | The deep learning identification of photovoltaic cell sorting and control method and device |
| CN111275679A (en) * | 2020-01-17 | 2020-06-12 | 同济大学 | Solar cell defect detection system and method based on image |
| CN113962929A (en) * | 2021-09-07 | 2022-01-21 | 湖南大学 | Photovoltaic cell assembly defect detection method and system and photovoltaic cell assembly production line |
| CN113989241A (en) * | 2021-10-29 | 2022-01-28 | 南京埃斯顿机器人工程有限公司 | Photovoltaic module EL defect detection method based on image processing and deep learning fusion |
| CN114399484A (en) * | 2022-01-11 | 2022-04-26 | 华电电力科学研究院有限公司 | Photovoltaic module defect detection method, device, equipment and medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115661466A (en) * | 2022-12-14 | 2023-01-31 | 尚特杰电力科技有限公司 | Photovoltaic panel positioning method and device based on deep learning image segmentation |
| CN116503412A (en) * | 2023-06-29 | 2023-07-28 | 宁德时代新能源科技股份有限公司 | Appearance defect detection method, apparatus, computer device and storage medium |
| CN116503412B (en) * | 2023-06-29 | 2023-12-08 | 宁德时代新能源科技股份有限公司 | Appearance defect detection method, apparatus, computer device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111862064B (en) | Silver wire surface flaw identification method based on deep learning | |
| CN110148130B (en) | Method and device for detecting part defects | |
| CN114627122A (en) | Defect detection method and device | |
| CN111275679B (en) | Image-based solar cell defect detection system and method | |
| CN108492291B (en) | CNN segmentation-based solar photovoltaic silicon wafer defect detection system and method | |
| CN113592828B (en) | Nondestructive testing method and system based on industrial endoscope | |
| CN113506286A (en) | Microwave chip defect detection method based on small sample data set of YOLOv5 algorithm | |
| CN112862770A (en) | Defect analysis and diagnosis system, method and device based on artificial intelligence | |
| CN102901735B (en) | System for carrying out automatic detections upon workpiece defect, cracking, and deformation by using computer | |
| CN115294031A (en) | Photovoltaic module fault image identification method based on infrared thermal imaging analysis | |
| CN111626249A (en) | Method and device for identifying geometric figure in topic image and computer storage medium | |
| CN113470005A (en) | Welding spot detection device and welding spot detection method for cylindrical battery cap | |
| CN113538603A (en) | Optical detection method and system based on array product and readable storage medium | |
| CN113487563B (en) | EL image-based self-adaptive detection method for hidden cracks of photovoltaic module | |
| US20230222643A1 (en) | Semantic deep learning and rule optimization for surface corrosion detection and evaluation | |
| CN115866502A (en) | Microphone part surface defect online detection process | |
| CN116952958B (en) | Defect detection method, device, electronic equipment and storage medium | |
| CN112561875A (en) | Photovoltaic cell panel coarse grid detection method based on artificial intelligence | |
| CN115953387A (en) | Radiographic image weld defect detection method based on deep learning | |
| CN115619796A (en) | Method and device for obtaining photovoltaic module template and nonvolatile storage medium | |
| CN112750113B (en) | Glass bottle defect detection method and device based on deep learning and linear detection | |
| CN113658096A (en) | Method and device for detecting plate abnormity | |
| CN116758060B (en) | Vertical basket of flowers visual detection system of battery piece | |
| CN117269168B (en) | New energy automobile precision part surface defect detection device and detection method | |
| CN117197131B (en) | Method and apparatus for conveyor belt tear identification and computing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220614 |