CN116380931A

CN116380931A - Defect judging device, defect detecting system and defect detecting method

Info

Publication number: CN116380931A
Application number: CN202310183774.1A
Authority: CN
Inventors: 金昱; 张洁; 沈崴; 陆唯佳; 葛欢; 吴少刚; 刘鹏; 孙林; 顾周顺
Original assignee: United Automotive Electronic Systems Co Ltd
Current assignee: United Automotive Electronic Systems Co Ltd
Priority date: 2023-02-28
Filing date: 2023-02-28
Publication date: 2023-07-04

Abstract

The invention provides a defect judging device, a defect detecting system and a defect detecting method, wherein a rechecking device receives first main data and first images output by an AOI device and provides second main data and second images with standard formats for an AI computing center, so that the AI computing center can use a set of processing systems facing to AOI devices of different models, and therefore, the AI computing center does not need to be matched with different AI computing centers, and the manufacturing cost is reduced.

Description

Defect judging device, defect detecting system and defect detecting method

Technical Field

The present invention relates to the field of automatic detection technologies, and in particular, to a defect determining apparatus, a defect detecting system, and a defect detecting method.

Background

With the continuous development of science, technology and economy, the demands of people for electronic products are also increasing, and the application of PCB (printed circuit board ) products is also increasing.

The processing of PCB products typically involves a reflow soldering process for the chip components and a selective wave soldering process for the discrete package components. After welding, in order to ensure the quality of the PCB product, the welded PCB product needs to be detected. The existing detection process is usually to detect through AOI (Automated Optical Inspection, automatic optical detection) equipment firstly so as to obtain main data and images of welding spots suspected to be problematic in PCB products; then, the detection result of the AOI device is judged manually to confirm whether the defect (NG) is a defect or a qualified defect (OK).

The detection process occupies a large amount of human resources, and the production cost is increased. Therefore, how to efficiently and reliably reduce the occupation of human resources is a problem that needs to be solved by those skilled in the art.

Disclosure of Invention

The invention aims to provide defect judging equipment, a defect detecting system and a defect detecting method, so as to solve the problems that in the prior art, a large amount of human resources are occupied for detecting PCB products, and the production cost is increased.

In order to solve the above technical problems, the present invention provides a defect judging apparatus including:

the rechecking device is used for forming second main data and a second image of the welding spot suspected to be problematic according to first main data and a first image of the welding spot suspected to be problematic, which are obtained by detecting a product to be detected through the AOI device, wherein the formats of the second main data and the second image are standard formats; the method comprises the steps of,

and the AI computing center is used for judging whether the welding spot suspected to be problematic is a defect or qualified according to the second main data and the second image.

Optionally, in the defect determining device, the second image includes a labeling area, and the labeling area labels an area that needs to be determined by the AI computing center.

Optionally, in the defect determining apparatus, the second image is a png format image of not more than 600×600 pixels.

Optionally, in the defect determining apparatus, the second main data includes: the method comprises the steps of uniquely encoding the suspected problematic welding spot, the position of the suspected problematic welding spot on a product to be detected, the program name of a detection library used by the corresponding AOI equipment, the reason considered by the corresponding AOI equipment, the type of a camera used by the corresponding rechecking equipment, the number of the camera used by the corresponding rechecking equipment, the parameters of the camera used by the corresponding rechecking equipment and the coordinates of the suspected problematic welding spot.

Optionally, in the defect judging device, the rechecking device further outputs main data of the product to be detected, where the main data of the product to be detected includes: the method comprises the steps of detecting the number of a production line of a product to be detected, the tracing ID of the product, the model number of the product, the detection time of the AOI equipment, the detection time of the rechecking equipment, the total number of welding spots of the product, the total number of suspected problematic welding spots of the product and the corresponding detection program library name used by the AOI equipment.

Optionally, in the defect judging device, the AI computation center performs training based on a training image with fine labels, wherein the fine labels are implemented based on priori knowledge.

Optionally, in the defect judging device, for the training image of abnormal solder filling, the training image includes: a pin annotation image, a bonding pad annotation image and a filling abnormal region annotation image; for the training image of a pin-out board, the training image comprises: the pins label the image.

Optionally, in the defect judging device, for the training image of abnormal solder filling, the training image further includes: a pixel-level labeling image of the pin, a pixel-level labeling image of the bonding pad and a pixel-level labeling image of the filling abnormal region; for the training image of a pin-out board, the training image further comprises: the pixel level of the pin annotates the image.

Optionally, in the defect judging device, the AI computing center adopts a semantic segmentation network model.

Optionally, in the defect determining device, the AI computing force center is further configured to output a feature table and/or a thermodynamic diagram corresponding to the suspected problematic solder joint, where the feature table and/or the thermodynamic diagram includes information of a pin, a pad, and/or a filling abnormal region.

The present invention also provides a defect detection system including: an AOI device and a defect judging device as described above; the AOI equipment is used for detecting a product to be detected to obtain first main data and a first image of a welding spot suspected to be problematic on the product to be detected, and providing the first main data and the first image to the defect judging equipment; the defect judging device is used for judging whether the welding spot suspected to be problematic is defective or qualified according to the first main data and the first image.

The invention also provides a defect detection method, which comprises the following steps:

detecting a product to be detected by adopting AOI equipment to obtain first main data and a first image of a welding spot suspected to be problematic on the product to be detected;

forming second main data and a second image of the welding spot suspected to be problematic according to the first main data and the first image by using a rechecking device, wherein the formats of the second main data and the second image are standard formats; the method comprises the steps of,

and judging the welding spot suspected to be problematic as a defect or qualified according to the second main data and the second image by adopting an AI computing center.

Optionally, in the defect detection method, the AI computing force center further outputs a feature table and/or a thermodynamic diagram corresponding to the suspected problematic welding spot.

In the defect judging device, the defect detecting system and the defect detecting method provided by the invention, the first main data and the first image output by the AOI device are received through the rechecking device, and the second main data and the second image with standard formats are provided for the AI computing center, so that one set of processing system can be used for the AI computing center facing the AOI devices with different models, and therefore, the AI computing center does not need to be matched with different AI computing centers, and the manufacturing cost is reduced.

Furthermore, in the defect judging equipment, the defect detecting system and the defect detecting method provided by the invention, the device not only can face AOI equipment of various types, but also has extremely strong applicability; when the defect problem is found, the method can be very conveniently positioned and traced so as to intercept the defect product from flowing into the market and facilitate the investigation and the solution of the problem; in addition, the final judgment result is also convenient for understanding and rechecking manually through the characteristic table and/or the thermodynamic diagram. Therefore, the defect judging equipment, the defect detecting system and the defect detecting method provided by the invention are powerful in function and friendly to use.

Drawings

FIG. 1 is a block diagram of a defect detection system according to an embodiment of the present invention.

Fig. 2 is a schematic diagram of a second image of an embodiment of the present invention.

Fig. 3 is a schematic diagram of second main data of an embodiment of the present invention.

Fig. 4 is a schematic diagram of main data of a product to be inspected according to an embodiment of the present invention.

Fig. 5 is a schematic view of an SP image of an embodiment of the present invention.

Fig. 6 is a pin annotation image of the SP image shown in fig. 5.

Fig. 7 is a pad marking image of the SP image shown in fig. 5.

Fig. 8 is a filling abnormality region labeling image of the SP image shown in fig. 5.

Figure 9 is a schematic representation of a WP image of an embodiment of the invention.

Fig. 10 is a pin signature image of the WP image shown in fig. 9.

Fig. 11 is a schematic view of an SP image of an embodiment of the present invention.

Fig. 12 is a schematic diagram of pixel level labeling of the SP-image shown in fig. 11.

Fig. 13 is a schematic view of an SP image of an embodiment of the present invention.

Fig. 14 is a schematic diagram of pixel level labeling of the SP-image shown in fig. 13.

Fig. 15 is a schematic view of an SP image of an embodiment of the present invention.

FIG. 16 is a schematic illustration of pixel level labeling of the SP image shown in FIG. 15.

Figure 17 is a schematic representation of a WP image of an embodiment of the invention.

Fig. 18 is a schematic illustration of pixel level labeling of the WP image shown in fig. 17.

Fig. 19 is a schematic diagram of a feature table of an embodiment of the present invention.

Fig. 20 is a schematic diagram of a general data table of an AI computing center according to an embodiment of the invention.

FIG. 21 is a schematic diagram of a thermodynamic diagram of an embodiment of the invention.

FIG. 22 is a schematic diagram of another thermodynamic diagram of an embodiment of the present invention.

Fig. 23 is a flow chart of a defect detection method according to an embodiment of the invention.

Wherein reference numerals are as follows:

100-a defect detection system; 110-AOI device; 120-defect judging device; 122-rechecking equipment; 124-AI calculation center;

200-a second image; 210-a labeling area;

B100—Pin; b102-bonding pads and B104-filling abnormal areas; b106-pin;

p100-pin; p102-the boundary of the pad and the filled anomaly region; p104-pin; p106-the boundary of the pad and the filled anomaly region; p108-pin; p110-the boundary of the pad and the filled anomaly region; p112-pin.

Detailed Description

The defect judging device, the defect detecting system and the defect detecting method according to the present invention will be described in further detail with reference to the accompanying drawings and the specific embodiments. Advantages and features of the invention will become more apparent from the following description and from the claims. It should be noted that the drawings are in a very simplified form and are all to a non-precise scale, merely for convenience and clarity in aiding in the description of embodiments of the invention.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Unless defined otherwise herein, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this invention belongs. The terms first, second and the like in the description and in the claims, are not used for any order, quantity or importance, but are used for distinguishing between different elements. Likewise, the terms "a" or "an" and the like do not denote a limitation of quantity, but rather denote the presence of at least one. "plurality" or "plurality" means two or more. Unless otherwise indicated, the terms "front," "rear," "lower," and/or "upper" and the like are merely for convenience of description and are not limited to one location or one spatial orientation. The word "comprising" or "comprises", and the like, means that elements or items appearing before "comprising" or "comprising" are encompassed by the element or item recited after "comprising" or "comprising" and equivalents thereof, and that other elements or items are not excluded. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.

In order to ensure the quality and reliability of the PCB product, the welded PCB product needs to be detected. In the prior art, after the AOI equipment is detected, judgment is performed manually, so that a large amount of human resources can be occupied, and the production cost is increased. If the AI (Artificial Intelligence ) device is adopted to replace manual judgment, the AOI devices of different models are faced, the output detection data and images are different, and the corresponding AI devices have different processing requirements, so that different AI devices are required to adapt, and the cost is increased.

For this reason, the present invention provides a defect detection system, please refer to fig. 1, which is a block diagram of a defect detection system according to an embodiment of the present invention. As shown in fig. 1, in an embodiment of the present application, the defect detection system 100 includes: an AOI device 110 and a defect judgment device 120 connected to the AOI device 110. The AOI device 110 is configured to detect a product to be detected to obtain first main data and a first image of a weld spot suspected to be problematic on the product to be detected, and provide the first main data and the first image to the defect determining device 120.

In the embodiment of the present application, the defect determining apparatus 120 includes: a rechecking device 122, where the rechecking device 122 is configured to form second main data and a second image of a suspected problematic welding spot on the product to be detected according to the first main data and the first image output by the AOI device 110, where a format of the second main data and a format of the second image are standard formats; and an AI (Artificial Intelligence ) computing center 124, where the AI computing center 124 is configured to determine, according to the second main data and the second image, that the welding spot suspected of being problematic is a defect (NG) or a PASS (OK or PASS).

In this embodiment of the present application, the review device 122 receives the first main data and the first image output by the AOI device 110, and provides the AI computing center 124 with the second main data and the second image having standard formats, so that, facing AOI devices 110 of different manufacturers and models, the AI computing center 124 may use a set of processing systems, so that no adaptation with different AI computing centers 124 is required, and manufacturing costs are reduced.

Specifically, the rechecking device 122 may locate a suspected problematic solder joint according to the first main data and the first image output by the AOI device 110, and form, for the located suspected problematic solder joint, second main data and a second image of the suspected problematic solder joint.

Wherein the review device 122 may obtain the second image having a standard format by converting the format of the first image. Preferably, the rechecking device 122 uses a high-definition camera to capture the located welding spots suspected of being problematic, so as to form the second image with a standard format. Thus, the second image has higher definition than the first image, which is more advantageous for the AI computing force center 124 to determine the suspected problematic weld spot. Further, the second image provided by the rechecking device 122 to the AI computing center 124 may be a color image, so that the second image has more information, for example, whether there is a problem of abnormal solder filling can be more clearly displayed, thereby being more beneficial to the determination of the AI computing center 124 and improving the accuracy of the determination of the AI computing center 124. Still further, the review device 122 may capture the located suspected problematic weld points from different angles, thereby forming a plurality of the second images, and provide a plurality of the second images to the AI computing force center 124. The second images captured from different angles contain more information, so that the determination of the welding points suspected to be problematic by the AI computing force center 124 can be further simplified, and the accuracy of the determination can be improved.

Please refer to fig. 2, which is a schematic diagram of a second image according to an embodiment of the present invention. As shown in fig. 2, in the embodiment of the present application, the second image 200 includes a labeling area 210, where the labeling area 210 labels an area that needs to be determined by the AI computing force center 124. Generally, the second image 200 includes an area that needs to be determined by the AI computing force center 124 and its surrounding area, so that the integrity of the view can be ensured. In the embodiment of the present application, the area that needs to be determined by the AI computing center 124 is marked to form the marked area 210, so that the processing of the AI computing center 124 on the second image 200 can be simplified, the processing capacity of the AI computing center 124 is reduced, and the determination speed of the AI computing center 124 is improved. Preferably, the second image 200 is a png format image of not more than 600×600 pixels.

In the embodiment of the present application, the second main data may be obtained by converting the format of the first main data, so as to form the second main data with a standard format, so that the AI computing force center 124 has wider applicability. In particular, the AI computing center 124 may directly determine based on the received second main data and the second image, so that the AI computing center 124 may be simplified and the determination speed of the AI computing center 124 may be increased. The AI computing center 124 has a higher complexity and cost, where providing the second primary data and the second image in a standard format by the review device 122 can greatly simplify the complexity and cost of the AI computing center 124, reduce overall costs, and increase overall revenue.

Further, the review device 122 may also form the second main data having a standard format through the second graphic located and photographed; alternatively, the review device 122 may combine the received first main data, the first image, and the captured second image to form the second main data having a standard format, so that information included in the second main data may be added to facilitate the determination of the suspected problematic welding spot by the AI computing force center 124.

Please refer to fig. 3, which is a schematic diagram of the second main data according to an embodiment of the present invention. As shown in fig. 3, in the embodiment of the present application, the second main data includes: the unique code res_id of the suspected problematic solder joint, the location id_commands on the product to be tested (here, the Number of the solder joint location where the suspected problematic solder joint is located may be adopted), the corresponding detection library program name libs used by the AOI device 110, the corresponding cause mac_name of the defect considered by the AOI device 110, the Camera Type used by the recheck device 122, the Camera Number used by the recheck device 122, the Camera parameter used by the recheck device 122, and the coordinates of the suspected problematic solder joint. The coordinates of the welding spot suspected of being problematic may be represented by four parameters, i.e., an abscissa x_mittelp, an ordinate y_mittelp, a first-order derivative dx of the abscissa, and a first-order derivative dy of the ordinate, respectively. Here, the second main data mainly includes the identity information, the position information and the obtaining conditions of the suspected problematic welding spot, so that it is convenient for the AI computing center 124 to trace back and compare the difference between the risk part and the normal part when the AI computing center is truly defective or qualified and the AI computing center is judged to be suspected to be in error in the future.

Preferably, the rechecking device 122 also outputs the main data of the product to be tested. Please refer to fig. 3, which is a schematic diagram of main data of a product to be tested according to an embodiment of the present invention. As shown in fig. 3, the main data of the product to be detected includes: the Line number line_no of the product to be detected, the product traceability ID pcba code, the product model pgm_id_org, the corresponding detection time t_ AOI _start of the AOI device 110, the detection time t_rep_start of the recheck device 122, the total number of welding spots no_total of the product, the total number of suspected problematic welding spots no_comp of the product, and the corresponding detection program library name si_version used by the AOI device 110. Here, the main data of the product to be detected mainly includes identity information, production and detection information of the product to be detected, so that if the suspected problematic welding spots on the product to be detected are judged to be defects, tracing can be timely and conveniently realized, outflow of bad products is reduced/avoided, and production reliability is improved.

In this embodiment, after the review device 122 outputs the second main data and the second image with the standard format, the AI computing center 124 may determine that the suspected problematic welding spot is defective or qualified according to the second main data and the second image.

In order to improve the accuracy of the determination of the AI computing force center 124, in the embodiment of the present application, the AI computing force center 124 performs training based on a training image with fine labels, where the fine labels are implemented based on a priori knowledge. The inventor finds that the number of samples in a sample set is small when an AI network is trained due to (good/bad) sample imbalance generated in actual production, and the distribution cannot well cover the actual working condition, so that a hotspot graph of network detection lacks stability. Based on the method, prior knowledge of experts is introduced, a hotspot graph of the network is stabilized, and robustness and generalization capability of the network are improved. The training images are finely marked by introducing priori knowledge of experts, and the model training is performed by adopting the finely marked training images, so that the reliability of the trained model can be improved, and the judgment accuracy of the AI computing center 124 is improved.

Preferably, for the training image (may be referred to as SP image) of solder filling anomaly, the fine annotation includes: pins (Pin), pads (Pad) and filling abnormal areas (Hole); for the training image of a pin-out template (which may be referred to as a WP image), the fine annotation includes: pins (Pin). The number of the SP images and the number of the WP images are multiple, for example, the number of the SP images can be 10-20, and the number of the WP images can be 40-60. Correspondingly, for each SP image, the fine annotation comprises: pins (Pin), pads (Pad) and filling abnormal areas (Hole); for each of the WP images, the fine annotation comprises: pins (Pin).

Specifically, please refer to fig. 5 to 10, wherein fig. 5 is a schematic diagram of an SP image according to an embodiment of the present invention; FIG. 6 is a pin annotation image of the SP image shown in FIG. 5; FIG. 7 is a pad label image of the SP image shown in FIG. 5; FIG. 8 is a filling anomaly region annotation image of the SP image shown in FIG. 5; FIG. 9 is a schematic representation of a WP image of an embodiment of the invention; fig. 10 is a pin signature image of the WP image shown in fig. 9. As shown in fig. 5 to 8, in the embodiment of the present application, the pin B100, the pad B102 and the filling anomaly region B104 are respectively marked for the training image of the solder filling anomaly, and preferably, the marked pin B100, pad B102 and filling anomaly region B104 form independent marked images. As shown in fig. 9-10, pin B106 is labeled for the training image of the pin-out-plate, and a separate labeled image is formed.

Preferably, the noted pin B100, the noted pad B102, the noted filling anomaly region B104, and the noted pin B106 are subjected to image processing to form a pixel level annotation of the pin B100, a pixel level annotation of the pad B102, a pixel level annotation of the filling anomaly region B104, and a pixel level annotation of the pin B106.

Specifically, please refer to fig. 11 to 18, wherein fig. 11 is a schematic diagram of an SP image according to an embodiment of the present invention; FIG. 12 is a schematic illustration of pixel level labeling of the SP image shown in FIG. 11; FIG. 13 is a schematic view of an SP image of an embodiment of the present invention; FIG. 14 is a schematic illustration of pixel level labeling of the SP image shown in FIG. 13; FIG. 15 is a schematic view of an SP image of an embodiment of the present invention; FIG. 16 is a schematic illustration of pixel level labeling of the SP image shown in FIG. 15; FIG. 17 is a schematic illustration of a WP image of an embodiment of the invention; fig. 18 is a schematic illustration of pixel level labeling of the WP image shown in fig. 17.

As shown in fig. 11 and 12, for the training image of a solder filling anomaly, a pixel level annotation image of the leads, pads and filling anomaly areas is formed, wherein the leads P100 and the pad and filling anomaly area boundaries P102 are shown in fig. 12. As shown in fig. 13 and 14, for the training image of a solder filling anomaly, a pixel level annotation image of the leads, pads and filling anomaly areas is formed, wherein the leads P104 and the pad and filling anomaly area boundary P106 are shown in fig. 14. As shown in fig. 15 and 16, for the training image of a solder filling anomaly, a pixel level annotation image of the leads, pads and filling anomaly areas is formed, wherein the leads P108 and the pad and filling anomaly area boundary P110 are shown in fig. 16. As shown in fig. 17 and 18, for the training image of a pin but not a board, a pixel level annotation image of the pin is formed, wherein pin P112 is shown in fig. 18.

In the embodiment of the present application, the AI computation center 124 employs a semantic segmentation network model. Specifically, the semantic segmentation network model is trained through the labeling image and the pixel-level labeling image, so that the AI computing center 124 can accurately obtain the pins, the welding pads and/or the abnormal filling areas in the second image, and further judge whether the abnormal filling of the soldering tin exists or whether the soldering tin is not excessively filled in the pins, so that a conclusion that the suspected problematic soldering points are defects or qualified is obtained.

Preferably, the fine mark further includes a manual feature, for example, the manual feature includes the number of pixels of the leads, the number of pixels of the pad and the filling anomaly region boundary, the divergence angle of the pad and the filling anomaly region boundary, and the like. Further, the semantic segmentation network model may be trained by using the manual features, so that the AI computing center 124 can accurately determine whether there is abnormal solder filling or no pin but no board, so as to more accurately obtain a conclusion that the suspected problematic solder joint is defective or qualified.

In this embodiment of the present application, by introducing priori knowledge of an expert, a fine labeled training image is formed to train the AI computing center 124, so that the AI computing center 124 can efficiently and accurately determine the welding spot suspected to be problematic, and a defect or a qualified conclusion is obtained.

Preferably, in the embodiment of the present application, the AI computing force center 124 determines the determination of the weld spot suspected of being problematic and outputs the conclusion, and also outputs the feature data, where the feature data is presented in the form of a feature table and/or a thermodynamic diagram.

As shown in fig. 19, a schematic diagram of a feature table is provided. In an embodiment of the present application, the feature data may specifically include: the number of pixels in the solder filling area SP_hole_edge PixelTot, the number of pixels in the periphery of the inner ring 1 SP_ring_maxGradLoc_low, the number of pixels in the periphery of the inner ring 2 SP_ring_maxGradLoc_high, the minimum number of pixels in the small Pin type WP_small_pin_low, the maximum number of pixels in the small Pin type WP_small_pin_high, the minimum number of pixels in the large Pin type WP_large_pin_low, the maximum number of pixels in the large Pin type WP_large_pin_high, the maximum number of pixels in the mini Pin type WP_tiny_pin_high, the minimum number of pixels in the mini Pin type WP_pin_high, the minimum number of pixels in the large Pin type WP_large Pin_pin_high, the maximum number of pixels in the large Pin type WP_pin_pin_low, the confidence value in the small Pin type determination value WP_small_ footprint, pin, the confidence of pixels in the confidence determination of the small Pin type WP_pin_low, the confidence determination of the gray scale of the noise, or the confidence determination of the gray scale of the noise (or the confidence ratio of the noise.

After the AI computing center 124 determines the welding spot suspected of being problematic, some or all parameters in the feature table shown in fig. 19 may be output at the same time, so that an expert may perform analysis when needed later. For example, the result determined by the AI computing center 124 is manually checked or checked again, and when the result of the manual check or check is different from the result determined by the AI computing center 124, the expert may analyze according to the parameter output by the AI computing center 124, so as to facilitate the expert to make the determination.

Further, the AI computing center 124 may also output general data for the AI computing center 124. Fig. 20 is a schematic diagram of a general data table of an AI computing center according to an embodiment of the invention. As shown in fig. 20, the general data may specifically include: line number line_no of the product to be detected, AI algorithm version number model, AI algorithm configuration version number config, inferred data set version number segData, and training data set version number clsData. By outputting the general data of the AI computing center 124, the state of the AI computing center 124 can be known, so that when the result of manual spot check or rechecking is different from the determination result of the AI computing center 124, the expert is convenient to analyze and find out the reasons of different generation, thereby being beneficial to the improvement and upgrading of subsequent equipment.

Referring to fig. 21 and 22, in the embodiment of the present application, the feature data may also be presented in the form of a thermodynamic diagram. As shown in fig. 21 and 22, in the thermodynamic diagram, the determination of the AI computation center 124 for the suspected problematic solder joints and corresponding feature data are presented, and further, corresponding pins, pads, and/or filling abnormal regions are presented. Thus, when the determination result of the AI computing center 124 needs to be analyzed, a corresponding analysis can be made very conveniently.

Further, an embodiment of the present application further provides a defect detection method, as shown in fig. 23, where the defect detection method includes:

step S10: detecting a product to be detected by adopting AOI equipment to obtain first main data and a first image of a welding spot suspected to be problematic on the product to be detected;

step S12: forming second main data and a second image of the welding spot suspected to be problematic according to the first main data and the first image by using a rechecking device, wherein the formats of the second main data and the second image are standard formats; the method comprises the steps of,

step S14: and judging the welding spot suspected to be problematic as a defect or qualified according to the second main data and the second image by adopting an AI computing center.

Further, the AI computing force center outputs the characteristics and/or thermodynamic diagrams corresponding to the welding spots suspected to be problematic.

In the defect judging device, the defect detecting system and the defect detecting method provided by the embodiment of the application, the first main data and the first image output by the AOI device are received through the rechecking device, and the second main data and the second image with standard formats are provided for the AI computing center, so that the AI computing center can use a set of processing system in the face of AOI devices of different manufacturers and models, and therefore, the AI computing center does not need to be out of fit with different AI computing centers, and the manufacturing cost is reduced.

Here, the judgment of the welding spot suspected to be problematic is realized through the AI calculation center so as to obtain a judgment conclusion that the welding spot is defective or qualified, thereby greatly saving labor, correspondingly greatly reducing labor cost and improving production efficiency. Through measurement and calculation, the labor consumption can be reduced by more than 50%, and the cost can be directly reduced by tens of millions.

Furthermore, in the defect judging device, the defect detecting system and the defect detecting method provided by the embodiment of the application, the device not only can face AOI devices of various types, but also has extremely strong applicability; when the defect problem is found, the method can be very conveniently positioned and traced so as to intercept the defect product from flowing into the market and facilitate the investigation and the solution of the problem; in addition, the final judgment result is also convenient for understanding and rechecking manually through the characteristic table and/or the thermodynamic diagram. Therefore, the defect judging equipment, the defect detecting system and the defect detecting method provided by the embodiment of the application are powerful in function and friendly to use.

Furthermore, in other implementations of the present application, various combinations may be made to form different specific implementations, according to the claims and the embodiments described above, which are not listed herein, and one of ordinary skill in the art may make many variations on the disclosed disclosure without undue burden.

The above description is only illustrative of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention, and any alterations and modifications made by those skilled in the art based on the above disclosure shall fall within the scope of the appended claims.

Claims

1. A defect judgment apparatus, characterized in that the defect judgment apparatus comprises:

2. The defect review device of claim 1 wherein the second image includes a label area that labels the area for which the AI computation center review is desired.

3. The defect judging apparatus of claim 1, wherein the second image is a png format image of not more than 600 x 600 pixels.

4. The defect judgment apparatus of claim 1, wherein the second main data includes: the method comprises the steps of uniquely encoding the suspected problematic welding spot, the position of the suspected problematic welding spot on a product to be detected, the program name of a detection library used by the corresponding AOI equipment, the reason considered by the corresponding AOI equipment, the type of a camera used by the corresponding rechecking equipment, the number of the camera used by the corresponding rechecking equipment, the parameters of the camera used by the corresponding rechecking equipment and the coordinates of the suspected problematic welding spot.

5. The defect judgment apparatus of claim 1, wherein the review apparatus further outputs main data of the product to be inspected, the main data of the product to be inspected including: the method comprises the steps of detecting the number of a production line of a product to be detected, the tracing ID of the product, the model number of the product, the detection time of the AOI equipment, the detection time of the rechecking equipment, the total number of welding spots of the product, the total number of suspected problematic welding spots of the product and the corresponding detection program library name used by the AOI equipment.

6. The defect review device of claim 1 wherein the AI computation center is trained based on a fine annotation training image, wherein the fine annotation is implemented based on a priori knowledge.

7. The defect determining apparatus of claim 6, wherein for the training image of solder filling anomalies, the training image comprises: a pin annotation image, a bonding pad annotation image and a filling abnormal region annotation image; for the training image of a pin-out board, the training image comprises: the pins label the image.

8. The defect determining apparatus of claim 7, wherein for the training image of solder filling anomalies, the training image further comprises: a pixel-level labeling image of the pin, a pixel-level labeling image of the bonding pad and a pixel-level labeling image of the filling abnormal region; for the training image of a pin-out board, the training image further comprises: the pixel level of the pin annotates the image.

9. The defect review device of claim 8 wherein the AI computation center employs a semantic segmentation network model.

10. The defect determining apparatus according to any one of claims 1 to 9, wherein the AI computation center is further configured to output a feature table and/or a thermodynamic diagram corresponding to the suspected problematic solder joint, where the feature table and/or thermodynamic diagram includes information of a pin, a pad, and/or a filling abnormal region.

11. A defect detection system, the defect detection system comprising: AOI device according to any one of claims 1 to 10; the AOI equipment is used for detecting a product to be detected to obtain first main data and a first image of a welding spot suspected to be problematic on the product to be detected, and providing the first main data and the first image to the defect judging equipment; the defect judging device is used for judging whether the welding spot suspected to be problematic is defective or qualified according to the first main data and the first image.

12. A defect detection method, characterized in that the defect detection method comprises:

13. The defect detection method of claim 12, wherein the AI computation center further outputs a feature table and/or a thermodynamic diagram corresponding to the suspected problematic weld spot.