CN115984158A - Defect analysis method and device, electronic equipment and computer readable storage medium - Google Patents
Defect analysis method and device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN115984158A CN115984158A CN202111187191.3A CN202111187191A CN115984158A CN 115984158 A CN115984158 A CN 115984158A CN 202111187191 A CN202111187191 A CN 202111187191A CN 115984158 A CN115984158 A CN 115984158A
- Authority
- CN
- China
- Prior art keywords
- defect
- target
- defects
- determining
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Image Analysis (AREA)
Abstract
The embodiment of the application discloses a defect analysis method, a defect analysis device, electronic equipment and a computer readable storage medium, wherein the method comprises the following steps: acquiring image information of a target panel; determining the position of the defect in the target panel according to the image information; determining a target defect type corresponding to the defect; and determining the influence level of the defects on the target panel according to the positions and the types of the target defects. By adopting the embodiment of the application, the efficiency of defect classification is improved, and after the position of the defect is determined, the influence level of the defect on the panel is determined by combining the type of the defect, namely, the damage condition of the panel caused by the defect is determined, and the efficiency of panel quality inspection is improved.
Description
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to a defect analysis method and device, electronic equipment and a computer-readable storage medium.
Background
With the development of computer technology, computer technology plays a greater and greater role in the production process of products, and particularly in the panel generation process, the computer technology promotes the rapid and efficient production of panel products.
At present, the detection of the panel defects mainly adopts a manual visual detection method, but the method depends on the subjective judgment of quality inspection personnel, and the detection work lasting for a long time can affect the detection accuracy along with the reduction of the attention of the personnel, and the defect types need to be manually classified after the defects are found, so that the efficiency of positioning the defect types is low.
Disclosure of Invention
The embodiment of the application provides a defect analysis method, a defect analysis device, electronic equipment and a computer-readable storage medium, which can improve the efficiency of defect type positioning.
In a first aspect, an embodiment of the present application provides a defect analysis method, including:
acquiring image information of a target panel;
determining the position of the defect in the target panel according to the image information;
determining a target defect type corresponding to the defect;
and determining the influence level of the defects on the target panel according to the positions and the types of the target defects.
In a second aspect, an embodiment of the present application further provides a defect analysis apparatus, including:
the acquisition module is used for acquiring the image information of the target panel;
the determining module is used for determining the position of the defect in the target panel according to the image information;
the identification module is used for determining a target defect type corresponding to the defect;
and the analysis module is used for determining the influence level of the defects on the target panel according to the positions and the types of the target defects.
In a third aspect, an embodiment of the present application further provides an electronic device, where the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the defect analysis method when executing the computer program.
In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps in the defect analysis method are implemented.
According to the embodiment of the application, the influence level of the defect is analyzed and determined by determining the position and the type of the defect and the position and the type of the defect, so that the damage condition of the panel caused by the defect is obtained, and the accuracy of the quality inspection of the panel is improved.
Drawings
In order to more clearly illustrate the technical solutions in the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic view of a scene of a defect analysis method provided in an embodiment of the present application;
FIG. 2 is a schematic flow chart of a defect analysis method provided in an embodiment of the present application;
fig. 3 is a schematic structural diagram of a defect analysis apparatus provided in an embodiment of the present application;
fig. 4 is a flow chart illustrating a configuration of a defect analysis apparatus according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the present application will be described clearly and completely with reference to the drawings in the present application, and it should be apparent that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the application provides a defect analysis method, a defect analysis device, electronic equipment and a computer-readable storage medium. Specifically, the embodiment of the present application provides a defect analysis apparatus suitable for an electronic device, where the electronic device may be a terminal or a server. The terminal may be a video camera, a high definition camera, or An Optical Inspection (AOI) camera, which is an apparatus having an image acquisition function. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, middleware service, a domain name service, a security service, a Content Delivery Network (CDN), a big data and artificial intelligence platform, and the server may be directly or indirectly connected through wired or wireless communication.
For example, please refer to fig. 1, in the embodiment of the present application, a defect analysis method executed by a terminal and a server is taken as an example, wherein other devices may be added on the basis of the terminal and the server to assist in completing the defect analysis method, and the types of the other devices are not limited herein; the terminal and the server are connected through a network, for example, a wired or wireless network connection, and the specific implementation process is as follows:
the terminal device A starts an image acquisition function to acquire image information of a panel, wherein the panel comprises a TFT-LCD panel or an integrated circuit panel and the like, and then the terminal device A transmits the image information of the panel to a server B;
after receiving the panel image information transmitted by the terminal equipment A, the server B identifies the image information, identifies defects contained in the panel image, marks positions of the defects, extracts characteristics of the defects to obtain defect characteristic information corresponding to the defects, predicts the types of the defects according to the defect characteristic information, predicts the probability that the defects belong to at least one preset defect type, and determines the type of the defects from the at least one preset defect type according to the probability;
and then, the server B determines the influence level of the defects on the panel according to the positions and types of the defects, namely determines the damage degree of the panel caused by the defects, and realizes the positioning of the positions and types of the defects and the analysis of the influence level. The server B also transmits information such as the position, the type, the influence level and the like of the defect to the terminal equipment A, so that the terminal equipment A can mark and display the position of the defect on the panel at the display end, and mark the defect type corresponding to the defect and display the influence level of the defect on the current panel.
The defect analysis method provided by the embodiment of the application relates to machine learning in the field of artificial intelligence. The embodiment of the application can improve the accuracy of defect detection and defect type positioning in the panel and improve the accuracy of defect analysis.
Among them, artificial Intelligence (AI) is a theory, method, technique and application system that simulates, extends and expands human Intelligence using a digital computer or a machine controlled by a digital computer, senses the environment, acquires knowledge and uses the knowledge to obtain the best result. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making. The artificial intelligence technology is a comprehensive subject, and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, a machine learning/deep learning direction and the like.
Machine Learning (ML) is a multi-domain cross subject, and relates to multiple subjects such as probability theory, statistics, approximation theory, convex analysis and algorithm complexity theory. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formula learning.
The embodiment of the application provides a defect analysis method, which specifically comprises the following steps:
acquiring image information of a target panel;
the target panel is a panel to be analyzed for defects, and in the embodiment of the present application, the panel includes a TFT-LCD panel, an integrated circuit panel, or the like. The image information of the panel can be acquired after being acquired according to the design of terminals with acquisition functions such as a high-definition camera and the like.
Determining the position of the defect in the target panel according to the image information;
the method comprises the steps of identifying whether the panel comprises the defects or not through identifying the defects in the image information, carrying out focusing and labeling on the positions of the defects after the defects are identified, and obtaining the positions of the defects in the panel.
And determining the target defect type corresponding to the defect.
The method and the device facilitate the subsequent determination of the influence of the defects on the target panel according to the types of the defects by determining the types of the defects.
And determining the influence level of the defects on the target panel according to the positions and the types of the target defects.
The influence level of the defects on the panel, namely the damage degree of the panel caused by the defects, can be comprehensively determined by determining the defect positions and the defect types, so that the defects in the panel can be analyzed.
According to the embodiment of the application, the influence level of the defects is analyzed and determined through the determination of the positions and the types of the defects and the positions and the types of the defects, the damage condition of the panel caused by the defects is determined, and the accuracy of the quality inspection of the panel is improved.
The following are detailed below. It should be noted that the description sequence of the following embodiments is not intended to limit the priority sequence of the embodiments.
Referring to fig. 2, fig. 2 is a schematic flow chart illustrating a defect analysis method according to an embodiment of the present disclosure. The specific flow of the defect analysis method can be as follows:
101. image information of a target panel is acquired.
The target panel is a panel to be subjected to defect analysis, and in the embodiment of the present application, the panel includes a TFT-LCD panel, an integrated circuit panel, or the like. The image information of the panel can be acquired after being acquired according to the design of terminals with acquisition functions, such as a high-definition camera.
The panel image information is acquired, so that the defects in the panel can be conveniently detected and identified.
102. The location of the defect in the target panel is determined from the image information.
The method comprises the steps of identifying whether the panel comprises the defects or not through identifying the defects in the image information, carrying out focusing and labeling on the positions of the defects after the defects are identified, and obtaining the positions of the defects in the panel. And when the defect does not exist, the returned defect position is null.
For example, in this embodiment of the present application, whether the image includes the defect is detected by using a deep learning method, and the detected defect is labeled, that is, optionally, in some embodiments, before the step "determining the position of the defect in the target panel according to the image information", the method further includes:
performing feature extraction on the image information through the trained defect detection model to obtain image feature information;
predicting a defect detection result corresponding to the image characteristic information through the trained defect detection model, wherein the defect detection result comprises defects and no defects;
determining a location of a defect in the target panel based on the image information, comprising:
and when the defect detection result comprises defects, determining the positions of the defects in the target panel according to the image information and the trained defect detection model.
Whether the image information has the defects or not is detected through the defect detection model, the efficiency and the accuracy of defect detection can be improved, and after the panel is detected to contain the defects, the defect detection model is used for positioning and marking the defects, so that the positions of the defects can be accurately positioned.
Optionally, in some embodiments, before the step "performing feature extraction on the image information through the trained defect detection model to obtain the image feature information", the method further includes:
obtaining model training data, wherein the model training data comprises at least one sample original image;
dividing the original sample image into a training set and a checking set, wherein the training set and the checking set respectively comprise at least one original sample image marked with a defective position;
training a preset defect detection model based on a training set;
and verifying the training result of the preset defect detection model by using the verification set to obtain the trained defect detection model.
And training and verifying a preset defect model according to the original image data of the sample marked with the defect position, so that the defect detection model obtained after training has the capabilities of defect detection and defect positioning.
The method comprises the steps of extracting image characteristic information corresponding to a panel image through a trained defect detection model, detecting whether the panel image has defects according to the image characteristic information to obtain a defect detection result of the panel image, wherein the defect detection result comprises probabilities corresponding to the defects and the non-defects, and determining whether the panel comprises the defects according to probability conditions, wherein when the defect detection result is the defects, the defect detection result also comprises defective position information, and the position information can comprise coordinate position information or pixel position information.
For example, in the embodiment of the present application, the training process of the defect detection model includes: collecting a plurality of sample data, dividing the sample data into a training set and a verification set, wherein the training set and the verification set respectively comprise at least one sample image marked with a defect position, predicting the position of the defect in the sample image by using a model, predicting the prediction probability of the defect belonging to each region in a panel, selecting the region with the maximum prediction probability as the prediction position of the defect position of the sample according to the probability value, and then converging the training result by comparing the prediction position with the actual position of the defect of the sample; after the model is trained, the model is verified according to the verification set, and the model is optimized according to the verification result, so that the accuracy of model prediction is improved.
In the embodiment of the application, the defect detection model comprises a trained real-time object detection algorithm model (YOLOV 3), wherein a network structure of the YOLOV3 algorithm can comprise a combination of a batch normalization layer and a convolution layer, a network loss function and a performance evaluation index of the YOLOV3 algorithm are determined according to a generalized cross-over ratio, an anchor frame is carried out on a panel surface defect sample data set according to a clustering algorithm (such as K-means + +), a defect prediction model is established according to the network structure, the network loss function, the performance evaluation index and the anchor frame, and the defect prediction model is trained according to a sample marked with a defect position.
103. And determining the target defect type corresponding to the defect.
The method and the device facilitate the subsequent determination of the influence of the defects on the target panel according to the types of the defects by determining the types of the defects.
In the embodiment of the present application, the features of the defect may be extracted through a neural network model, and the category to which the defect belongs is determined, that is, optionally, in some embodiments, the step "determining the target defect type corresponding to the defect" includes:
performing feature extraction on the defects to obtain defect feature information;
predicting the actual probability that the defect belongs to each preset defect type in at least one preset defect type according to the defect characteristic information;
and determining a target defect type corresponding to the defect from at least one preset defect type according to the actual probability.
The defect characteristic information corresponding to the defect is obtained by extracting the characteristic information of the defect, and the category of the defect can be conveniently predicted according to the defect characteristic information in the follow-up process. In the embodiment of the present application, the image feature extraction is further preceded by preprocessing of the image, where the preprocessing of the image includes scaling, cropping, flipping, and normalizing the image.
According to the defect characteristic information, the probability that the defects belong to each preset defect type can be predicted. In the embodiment of the present application, the defect types include black dots, sand, scratch, and the like.
And selecting a defect type corresponding to the defect according to the actual probability, wherein in the embodiment of the application, the defect type corresponding to the maximum actual probability value is taken as the defect type corresponding to the current defect.
In the embodiment of the present application, in order to improve accuracy of extracting defect feature information, feature extraction may be performed on a defect through a preset defect classification model, that is, optionally, in some embodiments, the step "performing feature extraction on a defect to obtain defect feature information" includes:
and performing feature extraction on the defects according to a preset defect classification model to obtain defect feature information.
Correspondingly, the step of predicting the actual probability that the defect belongs to each preset defect type in the at least one preset defect type according to the defect feature information includes:
and predicting the defect characteristic information by using a preset defect classification model to obtain the actual probability that the defect belongs to each preset defect type in at least one preset defect type.
The accuracy of defect feature extraction can be improved through the defect classification model for extracting the defect features.
The defect characteristic information is input into the defect classification model, so that the types of the defects can be predicted, and the probability values of the types of the defects are obtained. The accuracy of defect classification can be improved by predicting the defect types through the defect classification model.
In the embodiment of the present application, the preset defect classification model is obtained by training according to a plurality of sample data labeled with defect types, and specifically the following steps are performed: the method comprises the steps of firstly collecting image samples corresponding to a plurality of panels with defects to obtain a trained sample set, then labeling the defects in the image samples to label the types corresponding to the defects, then dividing the sample set labeled with the defect types into a training set and a check set, training the model through the training set, and verifying the training result through the check set to obtain a preset defect classification model, wherein the training model can be converged according to a predicted value (a type value with a higher predicted probability value) and a true value (a true type corresponding to the defects in the samples) during training to obtain the trained preset defect classification model. For example, in the embodiment of the present application, the defect classification model includes a residual error network (resnet 50), the resnet50 network model is trained through the preprocessed panel image, so as to obtain the trained resnet50 network model, and the defect is classified according to the trained resnet50 network model, for example, a defect image (such as a target image frame) is input into the resnet50 network model, so as to obtain the probability of the defect for each type.
In the embodiment of the application, the defect images with the low confidence degrees corresponding to the classification errors or the classification probabilities are used as sample data to perform repeated training, and the defect classification model is subjected to fine tuning and optimization.
In this embodiment of the present application, before performing feature extraction on a defect, scaling may be performed on position information of the defect to improve the effectiveness of defect information extraction, that is, optionally, in some embodiments, the method further includes:
determining a first image frame of the defect according to the position;
acquiring a second image frame of the circuit in the first image frame when the circuit is not defective;
and zooming the first image frame according to the circuit structure of the second image frame to obtain a defective target image frame.
For example, in the embodiment of the present application, for a periodic circuit, image information of other circuits that are periodically arranged with a current circuit can be obtained according to a periodic arrangement manner of the circuit, and the obtained image information is used as a second image frame corresponding to the first image frame, where the periodic circuit refers to a circuit structure in which circuits in a panel are arranged according to periodic, repetitive, and regular features, and the two circuits that are periodically arranged have the same structure in circuit arrangement.
In the embodiment of the present application, after the first image frame is compared with the second image frame, the difference points of the two image frames can be determined, the difference points are marked, and the positions of the difference points are obtained, so that the scaled target image frame can be obtained.
The image frame corresponding to the defect is scaled to maximize the information reflecting the defect in the image frame, for example, the image frame is narrowed to the edge of the defect to increase the effective proportion of the defect feature information.
Therefore, the step of "extracting the features of the defect to obtain the defect feature information" based on the scaled target image frame includes:
and performing feature extraction on the image information in the target image frame to obtain defect feature information.
By extracting the features of the image information of the target image frame obtained after zooming, the effective proportion of the defect feature information is improved, and the accuracy of defect classification is further improved.
104. And determining the influence level of the defects on the target panel according to the positions and the types of the target defects.
Wherein different types of defects, and different locations of defects, have different levels of impact on the panel, i.e., different levels of fatality to the panel. Therefore, the influence level of the defects on the panel can be comprehensively judged according to the positions where the defects occur and the types of the defects.
In this embodiment of the present application, the location of the defect includes a component where the defect is located (i.e., a damaged component), for example, the defect is on a certain component, so that the impact level of the defect may be analyzed according to the damaged component, that is, optionally, in some embodiments, the step "determining the impact level of the defect on the target panel according to the location and the type of the target defect" includes:
and determining the influence level of the defects on the target panel according to the damaged components where the defects are located and the target defect type.
The determination of the damaged components not only positions the position of the defect, but also positions the components directly affected by the defect, and the influence level of the defect can be obtained through the performance analysis of the damaged components, so that the influence level of the defect on the panel can be determined through the types of the damaged components and the defect.
In this embodiment, the component (i.e., the damaged component) in which the defect is located may be determined according to the position of the defect and the position of the component, that is, optionally, the method further includes:
determining the position coordinates of the target image frame as defect position coordinates of the defect;
acquiring device position coordinates of the components;
and determining the damaged component intersected with the defect according to the defect position coordinate and the device position coordinate.
And comparing the position coordinates of the defect and the position coordinates of the component to determine the component related to the defect.
In the embodiment of the application, the position coordinates of the reduced target image frame of the first image frame are used as the defect position coordinates, so that the accuracy of acquiring the defect position coordinates can be improved, and the accuracy of positioning the damaged component is improved.
In the embodiment of the present application, the position coordinates include pixel coordinates or distance coordinates of relative positions on the panel.
Wherein, in this application embodiment, for the accuracy that improves impaired components and parts are confirmed, can cut apart the components and parts in the panel alone to circuit in the panel, cut apart the components and parts in the panel, confirm the position coordinate of each components and parts through the components and parts after cutting apart, wherein, in this application embodiment, the segmentation mode of components and parts includes in the panel:
obtaining a circuit distribution template, wherein the circuit distribution template comprises a circuit in a panel;
matching the panel with the template, and determining a matching position matched with the panel in the template;
segmenting the circuit area at the matched position to obtain a circuit segmentation image corresponding to the panel;
since the template does not include defects, the circuit division image obtained by division also does not include defects, and therefore, the position coordinates of each component in the panel circuit are relatively accurately obtained from the circuit division image. The circuit distribution template is a basic circuit template containing panel circuits and is a set of a plurality of panel circuit structures, and the template is manufactured uniformly before the panel is manufactured, so that the template can be directly obtained from a storage space.
In the process of matching the panel and the template, the panel image and the template can be correspondingly scaled and aligned so as to improve the matching and consistency of the circuit segmentation image and the panel.
In this embodiment, because the positions and types of the defects have different impact levels on the panel, a one-to-one correspondence relationship among the positions, types, and impact levels of the defects may be established, and then the impact levels of the defects on the panel may be determined according to the correspondence relationship, that is, optionally, in some embodiments, the step "determining the impact levels of the defects on the target panel according to the damaged component where the defect is located and the target defect type" includes:
acquiring a mapping relation set, wherein the mapping relation set comprises mapping relations among preset damaged components, preset defect types and preset influence levels;
and determining the influence level of the defects on the target panel according to the mapping relation set, the damaged components where the defects are located and the target defect type.
The mapping relationship set may be formulated according to the past damage experience or the corresponding parameter requirement, and in the embodiment of the present application, the establishment of the mapping relationship is not limited.
And determining the influence level of the defects on the panel according to the positions of the defects and the types of the defects through the mapping relation set, so that the defect analysis efficiency is improved.
Different mapping relation sets can be set aiming at different types of panels, and analysis and judgment of different panel defects are achieved.
In this embodiment of the present application, after determining the impact level of the defect on the panel according to the position and the type of the defect, a subsequent execution flow of the panel may also be determined according to the impact level, that is, optionally, in some embodiments, after "determining the impact level of the defect on the target panel according to the damaged component where the defect is located and the target defect type", the method further includes:
and when the influence level is higher than or equal to the preset safety level, performing secondary detection treatment on the defects, wherein the secondary detection treatment comprises manual screening.
Wherein, in this application embodiment, through the artifical screening of the secondary of the higher defect of the influence grade to the panel, can improve defect analysis detection's accuracy, avoid extravagant panel material because of the false retrieval, simultaneously, the testing result of artifical screening also can be regarded as defect analysis's result feedback, is adjusting defect analysis according to the feedback after, can improve defect analysis's accuracy.
In the embodiment of the application, after the defects are analyzed, the defects with uncertain defect analysis results can be manually screened, so that the accuracy of defect detection is improved, and the occurrence of false detection is avoided.
In the embodiment of the application, the preset security level can be set empirically according to past historical data and can also be set according to actual needs of users, and the setting mode and specific numerical values of the preset security level are not limited in the embodiment of the application.
When the influence level of the defect on the panel is lower than the preset safety level, the panel can be put into the next production flow, for example, the panel is packaged and encapsulated, and the subsequent flow of the panel is not limited in the embodiment of the present application.
According to the method and the device, the obtained defects are subjected to feature extraction to obtain the defect feature information corresponding to the defects, the defect types corresponding to the defects are predicted according to the defect feature information, the defect classification efficiency is improved, after the positions of the defects are determined, the types of the defects are combined to determine the influence levels of the defects on the panel, namely the damage condition of the panel caused by the defects is determined, and the panel quality inspection efficiency is improved.
In order to better implement the defect analysis method of the present application, the present application also provides a defect analysis device based on the defect analysis method. The terms are the same as those in the defect analysis method, and details of implementation can be referred to the description in the method embodiment.
Referring to fig. 3, fig. 3 is a schematic structural diagram of a defect analysis apparatus provided in the present application, where the defect analysis apparatus may include an obtaining module 201, a determining module 202, an identifying module 203, and an analyzing module 204, which may specifically be as follows:
an obtaining module 201, configured to obtain image information of a target panel.
And a determining module 202 for determining the position of the defect in the target panel according to the image information.
Optionally, in some embodiments of the present invention, before the determining module 202, the apparatus further includes:
the first extraction unit is used for extracting the characteristics of the image information through the trained defect detection model to obtain the image characteristic information;
the first prediction unit is used for predicting a defect detection result corresponding to the image characteristic information through the trained defect detection model, and the defect detection result comprises defects and no defects;
a determining module 202, comprising:
and a first determining unit for determining a position of the defect in the target panel based on the image information and the trained defect detection model when the defect detection result includes the defect.
Optionally, in some embodiments of the present invention, before the first extracting unit, the apparatus further includes:
the device comprises a first acquisition unit, a second acquisition unit and a third acquisition unit, wherein the first acquisition unit is used for acquiring model training data, and the model training data comprises at least one sample original image;
the dividing unit is used for dividing the original sample image into a training set and a check set, wherein the training set and the check set respectively comprise at least one original sample image marked with a defective position;
the training unit is used for training a preset defect detection model based on a training set;
and the verification unit is used for verifying the training result of the preset defect detection model by using the check set to obtain the trained defect detection model.
And the identifying module 203 is used for determining a target defect type corresponding to the defect.
Optionally, in some embodiments of the present invention, the identifying module 203 includes:
the second extraction unit is used for extracting the characteristics of the defects to obtain defect characteristic information;
the second prediction unit is used for predicting the actual probability that the defect belongs to each preset defect type in at least one preset defect type according to the defect characteristic information;
and the selecting unit is used for determining a target defect type corresponding to the defect from at least one preset defect type according to the actual probability.
Optionally, in some embodiments of the present invention, the second extraction unit includes:
the first extraction subunit is used for carrying out feature extraction on the defects according to a preset defect classification model to obtain defect feature information;
the prediction unit includes:
and the predicting subunit is used for predicting the defect characteristic information by using a preset defect classification model to obtain the actual probability that the defect belongs to each preset defect type in at least one preset defect type.
Optionally, in some embodiments of the present invention, the apparatus further includes:
a second determining unit for determining a first image frame of the defect according to the position;
a second acquiring unit, configured to acquire a second image frame of the first image frame when the circuit is defect-free;
the zooming unit is used for zooming the first image frame according to the circuit structure of the second image frame to obtain a defective target image frame;
the second extraction unit includes:
and the second extraction subunit is used for performing feature extraction on the image information in the target image frame to obtain defect feature information.
And the analysis module 204 is used for determining the influence level of the defect on the target panel according to the position and the target defect type.
Optionally, in some embodiments of the present invention, the target panel includes at least one component, the position includes a damaged component where the defect is located, and the analysis module 204 includes:
and the analysis unit is used for determining the influence level of the defect on the target panel according to the damaged component where the defect is located and the target defect type.
Optionally, in some embodiments of the present invention, the analysis unit includes:
the acquiring subunit is used for acquiring a mapping relation set, wherein the mapping relation set comprises mapping relations among preset damaged components, preset defect types and preset influence levels;
and the analysis subunit is used for determining the influence level of the defects on the target panel according to the mapping relation set, the damaged components where the defects are located and the target defect type.
Optionally, in some embodiments of the present invention, the analysis module 204 is further specifically configured to:
and when the influence level is higher than or equal to the preset safety level, performing secondary detection treatment on the defects, wherein the secondary detection treatment comprises manual screening.
Referring to fig. 4, fig. 4 is a flow chart illustrating a construction of a defect analysis apparatus according to an embodiment of the present application, which specifically includes:
111. a data acquisition module: collecting panel image information to be subjected to defect analysis, and collecting sample panel image information;
112. a data labeling module: marking the position and the type of the defect in the sample panel image;
113. a defect detection model training module: training the model according to the sample panel image marked with the defect position to obtain a defect detection model;
114. a defect classification model training module: training the model according to the sample panel image marked with the defect type to obtain a defect classification model;
115. the template drawing manufacturing module: manufacturing a circuit distribution template;
116. a template graph segmentation module: dividing the circuit distribution template according to the panel to be analyzed for the defect to obtain a circuit distribution diagram corresponding to the panel to be analyzed for the defect;
117. a defect business rule configuration module: establishing a mapping relation among defect positions, types and damage degrees, and establishing a mapping relation set;
118. a system integration module: integrating all modules to obtain a defect analysis device;
119. a test module: testing the whole system;
120. a commissioning module: performing test operation on the defect analysis device, for example, inputting a panel image to obtain an analysis result;
121. an optimization module: and analyzing the analysis result, determining a panel image with wrong classification and a panel image with low classification confidence coefficient, inputting the panel image with wrong classification and low classification confidence coefficient into a defect analysis device, and optimizing the defect analysis device.
In the embodiment of the present application, after the defect analysis device is optimized, the defect analysis device can be operated online.
In the embodiment of the application, firstly, the obtaining module 201 obtains the panel image information to be subjected to defect analysis, secondly, the determining module 202 determines the position of the defect in the panel image, secondly, the extracting module 203 identifies the type of the defect to determine the type of the defect, and finally, the analyzing module 204 determines the influence level of the defect on the panel according to the position of the defect and the type of the defect.
According to the method and the device, the obtained defects are subjected to feature extraction to obtain the defect feature information corresponding to the defects, the defect types corresponding to the defects are predicted according to the defect feature information, the defect classification efficiency is improved, after the positions of the defects are determined, the damage conditions of the panel caused by the defects are determined according to the defect types, and the panel quality inspection efficiency is improved.
In addition, an embodiment of the present application further provides an electronic device, as shown in fig. 5, which shows a schematic structural diagram of the electronic device related to the present application, and specifically:
the electronic device may include components such as a processor 401 of one or more processing cores, memory 402 of one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that the electronic device configuration shown in fig. 5 does not constitute a limitation of the electronic device and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components. Wherein:
the processor 401 is a control center of the electronic device, connects various parts of the whole electronic device by various interfaces and lines, performs various functions of the electronic device and processes data by running or executing software programs and/or modules stored in the memory 402 and calling data stored in the memory 402, thereby performing overall monitoring of the electronic device. Optionally, processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 401.
The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by operating the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to use of the electronic device, and the like. Further, the memory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 access to the memory 402.
The electronic device further comprises a power supply 403 for supplying power to the various components, and preferably, the power supply 403 is logically connected to the processor 401 through a power management system, so that functions of managing charging, discharging, and power consumption are realized through the power management system. The power supply 403 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.
The electronic device may further include an input unit 404, and the input unit 404 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
Although not shown, the electronic device may further include a display unit and the like, which are not described in detail herein. Specifically, in this embodiment, the processor 401 in the electronic device loads an executable file corresponding to a process of one or more application programs into the memory 402 according to the following instructions, and the processor 401 runs the application programs stored in the memory 402, thereby implementing any of the steps in the defect analysis method provided by the present application.
According to the method and the device, the obtained defects are subjected to feature extraction to obtain the defect feature information corresponding to the defects, the defect types corresponding to the defects are predicted according to the defect feature information, the defect classification efficiency is improved, the types of the defects are combined after the positions of the defects are determined, the influence levels of the defects on the panel are determined, and the panel quality inspection efficiency is improved.
It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.
To this end, the present application provides a computer-readable storage medium (referred to as a storage medium for short) having a plurality of instructions (where a set of the plurality of instructions may constitute a computer program) stored thereon, where the instructions can be loaded by a processor to execute steps in any one of the defect analysis methods provided in the present application.
Wherein the storage medium may include: read Only Memory (ROM), random Access Memory (RAM), magnetic or optical disks, and the like.
Since the instructions stored in the storage medium can execute any of the steps in the defect analysis provided by the present application, the beneficial effects that can be achieved by any of the defect analysis methods provided by the present application can be achieved, which are detailed in the foregoing embodiments and will not be described herein again.
The defect analysis method, apparatus, electronic device and computer-readable storage medium provided in the present application are described in detail above, and a specific example is applied in the present application to illustrate the principles and embodiments of the present invention, and the description of the above embodiment is only used to help understanding the method and the core idea of the present invention; meanwhile, for those skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (12)
1. A method of defect analysis, comprising:
acquiring image information of a target panel;
determining the position of a defect in the target panel according to the image information;
determining a target defect type corresponding to the defect;
and determining the influence level of the defect on the target panel according to the position and the target defect type.
2. The method of claim 1, wherein the determining the target defect type corresponding to the defect comprises:
extracting the characteristics of the defects to obtain defect characteristic information;
predicting the actual probability that the defect belongs to each preset defect type in at least one preset defect type according to the defect characteristic information;
and determining a target defect type corresponding to the defect from the at least one preset defect type according to the actual probability.
3. The method of claim 2, wherein the extracting the features of the defect to obtain defect feature information comprises:
performing feature extraction on the defects according to a preset defect classification model to obtain defect feature information;
predicting the actual probability that the defect belongs to each preset defect type in at least one preset defect type according to the defect feature information comprises the following steps:
and predicting the defect characteristic information by using the preset defect classification model to obtain the actual probability that the defect belongs to each preset defect type in at least one preset defect type.
4. The method of any of claims 1-3, wherein prior to determining the location of the defect in the target panel from the image information, the method further comprises:
performing feature extraction on the image information through a trained defect detection model to obtain image feature information;
predicting a defect detection result corresponding to the image characteristic information through the trained defect detection model, wherein the defect detection result comprises defects and no defects;
the determining the position of the defect in the target panel according to the image information comprises:
when the defect detection result comprises a defect, determining the position of the defect in the target panel according to the image information and the trained defect detection model.
5. The method of claim 4, wherein before the feature extraction of the image information by the trained defect detection model to obtain image feature information, the method further comprises:
obtaining model training data, wherein the model training data comprises at least one sample original image;
dividing the sample original image into a training set and a check set, wherein the training set and the check set respectively comprise at least one sample original image marked with a defective position;
training a preset defect detection model based on the training set;
and verifying the training result of the preset defect detection model by using the verification set to obtain the trained defect detection model.
6. The method of claim 2, wherein prior to determining the target defect type corresponding to the defect, the method further comprises:
determining a first image frame of the defect according to the position;
acquiring a second image frame of the circuit in the first image frame when the circuit is not defective;
zooming the first image frame according to the circuit structure of the second image frame to obtain a defective target image frame;
the feature extraction of the defect to obtain defect feature information includes:
and performing feature extraction on the image information in the target image frame to obtain defect feature information.
7. The method of claim 1, wherein the target panel includes at least one component, the location includes a damaged component where the defect is located, and determining the level of impact of the defect on the target panel based on the location and the target defect type comprises:
and determining the influence level of the defects on the target panel according to the damaged components where the defects are located and the target defect type.
8. The method of claim 7, wherein determining the level of impact of the defect on the target panel according to the damaged component on which the defect is located and the target defect type comprises:
acquiring a mapping relation set, wherein the mapping relation set comprises mapping relations among preset damaged components, preset defect types and preset influence levels;
and determining the influence level of the defects on the target panel according to the mapping relation set, the damaged component where the defects are located and the target defect type.
9. The method of claim 1, wherein after determining the level of impact of the defect on the target panel based on the location and the target defect type, the method further comprises:
and when the influence grade is higher than or equal to a preset safety grade, carrying out secondary detection treatment on the defects, wherein the secondary detection treatment comprises manual screening.
10. A defect analysis apparatus, comprising:
the acquisition module is used for acquiring the image information of the target panel;
the determining module is used for determining the position of the defect in the target panel according to the image information;
the identification module is used for determining a target defect type corresponding to the defect;
and the analysis module is used for determining the influence level of the defects on the target panel according to the positions and the target defect types.
11. An electronic device, characterized in that the electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps in the defect analysis method according to any one of claims 1-9 when executing the computer program.
12. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the defect analysis method according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111187191.3A CN115984158A (en) | 2021-10-12 | 2021-10-12 | Defect analysis method and device, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111187191.3A CN115984158A (en) | 2021-10-12 | 2021-10-12 | Defect analysis method and device, electronic equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115984158A true CN115984158A (en) | 2023-04-18 |
Family
ID=85964740
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111187191.3A Pending CN115984158A (en) | 2021-10-12 | 2021-10-12 | Defect analysis method and device, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115984158A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117670876A (en) * | 2024-01-31 | 2024-03-08 | 成都数之联科技股份有限公司 | Panel defect severity level judging method, system, equipment and storage medium |
| CN117764908A (en) * | 2023-08-17 | 2024-03-26 | 上海感图网络科技有限公司 | Method, device, equipment and storage medium for displaying defect information of NG image |
| CN117764908B (en) * | 2023-08-17 | 2024-06-07 | 上海感图网络科技有限公司 | Method, device, equipment and storage medium for displaying defect information of NG image |
-
2021
- 2021-10-12 CN CN202111187191.3A patent/CN115984158A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117764908A (en) * | 2023-08-17 | 2024-03-26 | 上海感图网络科技有限公司 | Method, device, equipment and storage medium for displaying defect information of NG image |
| CN117764908B (en) * | 2023-08-17 | 2024-06-07 | 上海感图网络科技有限公司 | Method, device, equipment and storage medium for displaying defect information of NG image |
| CN117670876A (en) * | 2024-01-31 | 2024-03-08 | 成都数之联科技股份有限公司 | Panel defect severity level judging method, system, equipment and storage medium |
| CN117670876B (en) * | 2024-01-31 | 2024-05-03 | 成都数之联科技股份有限公司 | Panel defect severity level judging method, system, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111353413B (en) | Low-missing-report-rate defect identification method for power transmission equipment | |
| KR102321765B1 (en) | Display device quality inspection method, device, electronic device and storage medium | |
| US20200357109A1 (en) | Method for detecting display screen quality, apparatus, electronic device and storage medium | |
| WO2017084186A1 (en) | System and method for automatic monitoring and intelligent analysis of flexible circuit board manufacturing process | |
| CN111460250B (en) | Image data cleaning method, image data cleaning device, image data cleaning medium, and electronic apparatus | |
| CN110598620B (en) | Deep neural network model-based recommendation method and device | |
| CN111401418A (en) | Employee dressing specification detection method based on improved Faster r-cnn | |
| CN111900694B (en) | Relay protection equipment information acquisition method and system based on automatic identification | |
| CN111626177B (en) | PCB element identification method and device | |
| CN111126820A (en) | Electricity stealing prevention method and system | |
| CN110794360A (en) | Method and system for predicting fault of intelligent electric energy meter based on machine learning | |
| CN114037645A (en) | Coating defect detection method and device for pole piece, electronic equipment and readable medium | |
| CN112419268A (en) | Method, device, equipment and medium for detecting image defects of power transmission line | |
| CN114399484A (en) | Photovoltaic module defect detection method, device, equipment and medium | |
| CN115984158A (en) | Defect analysis method and device, electronic equipment and computer readable storage medium | |
| CN112967248A (en) | Method, apparatus, medium, and program product for generating defect image sample | |
| CN112668365A (en) | Material warehousing identification method, device, equipment and storage medium | |
| CN116630989A (en) | Visual fault detection method and system for intelligent ammeter, electronic equipment and storage medium | |
| CN116308679A (en) | Supply chain abnormal order processing method, device, equipment and storage medium | |
| CN113963011A (en) | Image recognition method and device, electronic equipment and storage medium | |
| CN111859370A (en) | Method, apparatus, electronic device and computer-readable storage medium for identifying service | |
| CN117173448B (en) | Method and device for intelligently controlling and early warning progress of foundation engineering | |
| CN117094967A (en) | Detection method, device, equipment and medium for PCB defect detection | |
| CN118037707A (en) | Wafer defect detection method and device based on neural network model | |
| CN115439674A (en) | Intelligent image labeling method and device based on power image knowledge graph |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |