CN116228741A

CN116228741A - PCBA (printed circuit board assembly) component defect detection method and device

Info

Publication number: CN116228741A
Application number: CN202310380298.2A
Authority: CN
Inventors: 周华锋; 石智伟
Original assignee: Guangdong University of Technology
Current assignee: Guangdong University of Technology
Priority date: 2023-04-10
Filing date: 2023-04-10
Publication date: 2023-06-06

Abstract

The application discloses a PCBA component defect detection method and device, wherein the method comprises the following steps: acquiring PCBA images to be detected by an image acquisition device to obtain component images to be detected; performing target detection on the component image to be detected by adopting a preset YOLO detection model to obtain a target component image, wherein the target component image comprises component categories and component center coordinates; carrying out preset angle recognition on text characters in the target component image by adopting a preset text recognition model to obtain component text information, wherein the component text information comprises text content and text center coordinates; and performing defect comparison detection according to the reference sample information, the target component image and the component text information to obtain a component defect detection result. The method and the device can solve the technical problems that the prior art is either too complex to operate or low in adaptation degree and cannot meet the defect detection requirement of the PCB component.

Description

PCBA (printed circuit board assembly) component defect detection method and device

Technical Field

The application relates to the technical field of image processing, in particular to a method and a device for detecting defects of PCBA components.

Background

PCB (Printed Circuit Board), a printed wiring board, is one of the important components of the electronics industry. PCBs are classified into two types, namely, PCB bare boards and PCB assembly boards, the latter being commonly referred to as assembled Circuit boards (Printed Circuit BoardAssembly, PCBA). At present, the printed wiring board has been extremely widely applied to the production and manufacture of electronic products. Therefore, the quality detection and monitoring of the PCB is very important to ensure that the PCB can be used normally, the quality of the PCB often directly influences the performance of the assembled product, and the defects in the manufacturing process are timely found to be the guarantee that the product can be used safely and normally.

In the past, the PCB defect detection adopts the method of manual visual inspection or ICT electrical inspection to detect components and parts, and electronic equipment is continuously reduced nowadays, so that the PCB requirement of mechanical manufacturing is larger and more complex, the functions are more and the size is smaller, and the 2 methods are not suitable for the development requirement of modern industrial production for various reasons.

Most of the existing effective technologies are automatic optical detection methods based on machine vision, but the existing automatic optical detection methods based on machine vision still have various technical problems, such as high cost of related optical instruments and equipment, complex operation, or high time cost of a method for calculating similarity by matching feature points, and low adaptation of certain methods, so that not only can incorrect components be detected, but also defect detection of various components cannot be adapted, and therefore, the existing automatic optical detection methods based on machine vision cannot be really adapted to the defect detection scene of the PCB components in the current PCBA image.

Disclosure of Invention

The application provides a PCBA component defect detection method and device, which are used for solving the technical problems that the prior art is either too complex to operate or low in adaptation degree and cannot meet the requirements of PCB component defect detection.

In view of the foregoing, a first aspect of the present application provides a method for detecting defects of a PCBA component, including:

acquiring PCBA images to be detected by an image acquisition device to obtain component images to be detected;

performing target detection on the component image to be detected by adopting a preset YOLO detection model to obtain a target component image, wherein the target component image comprises a component category and a component center coordinate;

performing preset angle recognition on text characters in the target component image by adopting a preset text recognition model to obtain component text information, wherein the component text information comprises text content and text center coordinates;

and performing defect comparison detection according to the reference sample information, the target component image and the component text information to obtain a component defect detection result.

Preferably, the image collecting device collects the PCBA image to be detected to obtain the component image to be detected, and then the method further includes:

And sequentially carrying out preprocessing operation and labeling operation on the to-be-detected component image, wherein the preprocessing operation comprises cutting, affine transformation, median filtering and sharpening.

Preferably, the detecting the object of the component image to be detected by using a preset YOLO detection model to obtain an object component image, where the object component image includes a component category and a component center coordinate, and the detecting includes:

constructing an initial YOLO detection model based on an upsampling mechanism, a SE attention mechanism and a small target detection mechanism, wherein the initial YOLO detection model comprises a plurality of SPD convolution kernels;

performing target detection training on the initial YOLO detection model by adopting a multi-scale training mode and a weighted image training strategy to obtain a preset YOLO detection model;

and carrying out local target detection on the discrete components in the component image to be detected by adopting a preset YOLO detection model to obtain a target component image, wherein the target component image comprises component categories and component center coordinates.

Preferably, the performing target detection on the component image to be detected by using a preset YOLO detection model to obtain a target component image, and then further includes:

And judging missing of the components and short circuit of tin balls according to the target component image, and obtaining a preliminary defect detection result.

Preferably, the performing preset angle recognition on text characters in the target component image by using a preset text recognition model to obtain component text information includes:

performing peer-to-peer and peer-to-peer object detection on text characters in the object component image based on an improved DBNet algorithm in a preset text recognition model to obtain a component text region image, wherein the component text region image comprises a text detection box and text center coordinates;

after the component text area image is adjusted to a preset angle, a preset SVTR algorithm in the preset text recognition model is adopted to recognize text characters in the component text area image, so that component text information is obtained, and the component text information comprises text content and text center coordinates.

Preferably, the method for detecting the text characters in the target component image by using the improved DBNet algorithm based on the preset text recognition model includes:

Text detection is carried out on text characters in the target component image according to a DBNet algorithm, so that a plurality of text detection boxes are obtained;

analyzing whether the text detection boxes of the same line are on the same component or not based on a preset gray level fluctuation principle, and merging and sorting the text detection boxes of the same line and the same component to obtain a component text region image.

Preferably, the performing defect contrast detection according to the reference sample information, the target component image and the component text information to obtain a component defect detection result includes:

extracting reference sample information based on the defect-free PCBA image to form a reference sample information list;

and respectively carrying out multiple defect comparison detection on the reference sample information and the target component image and the component text information according to a comparison threshold value to obtain a component defect detection result.

The second aspect of the present application provides a PCBA component defect detection apparatus, comprising:

the image acquisition unit is used for acquiring PCBA images to be detected through the image acquisition device to obtain images of components to be detected;

the target detection unit is used for carrying out target detection on the component image to be detected by adopting a preset YOLO detection model to obtain a target component image, wherein the target component image comprises a component category and a component center coordinate;

The text recognition unit is used for carrying out preset angle recognition on text characters in the target component image by adopting a preset text recognition model to obtain component text information, wherein the component text information comprises text content and text center coordinates;

and the defect detection unit is used for carrying out defect comparison detection according to the reference sample information, the target component image and the component text information to obtain a component defect detection result.

Preferably, the target detection unit includes:

a model construction subunit configured to construct an initial YOLO detection model based on an upsampling mechanism, a SE attention mechanism, and a small target detection mechanism, the initial YOLO detection model including a plurality of SPD convolution kernels;

the model training subunit is used for carrying out target detection training on the initial YOLO detection model by adopting a multi-scale training mode and a weighted image training strategy to obtain a preset YOLO detection model;

and the target detection subunit is used for carrying out local target detection on the discrete components in the component image to be detected by adopting a preset YOLO detection model to obtain a target component image, wherein the target component image comprises component categories and component center coordinates.

Preferably, the text recognition unit includes:

the region detection subunit is used for carrying out the same-row and same-component target detection on text characters in the target component image based on an improved DBNet algorithm in a preset text recognition model to obtain a component text region image, wherein the component text region image comprises a text detection box and text center coordinates;

and the text recognition subunit is used for recognizing text characters in the text region image of the component by adopting a preset SVTR algorithm in the preset text recognition model after the text region image of the component is adjusted to a preset angle, so as to obtain the text information of the component, wherein the text information of the component comprises text content and text center coordinates.

From the above technical solutions, the embodiments of the present application have the following advantages:

in the application, a method for detecting defects of PCBA components is provided, which comprises the following steps: acquiring PCBA images to be detected by an image acquisition device to obtain component images to be detected; performing target detection on the component image to be detected by adopting a preset YOLO detection model to obtain a target component image, wherein the target component image comprises component categories and component center coordinates; carrying out preset angle recognition on text characters in the target component image by adopting a preset text recognition model to obtain component text information, wherein the component text information comprises text content and text center coordinates; and performing defect comparison detection according to the reference sample information, the target component image and the component text information to obtain a component defect detection result.

According to the PCBA component defect detection method, the PCBA image is analyzed and identified through the detection model, the defect detection of the component can be completed without complex instruments and equipment, and the method is simple and easy to operate; in addition, in the detection process, not only the component area is focused, but also the text information on the components is identified and analyzed, and the electrical property of each component can be clearly grasped, so that a plurality of different defect types can be judged in a targeted manner, and the defect detection method can be suitable for defect detection of a plurality of different components. Therefore, the method and the device can solve the technical problems that the prior art is either too complex to operate or low in adaptation degree and cannot meet the defect detection requirement of the PCB component.

Drawings

Fig. 1 is a schematic flow chart of a method for detecting defects of a PCBA component according to an embodiment of the present application;

fig. 2 is another schematic flow chart of a PCBA component defect detection method according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of a PCBA component defect detection apparatus according to an embodiment of the present application;

fig. 4 is an exemplary image one of a target component obtained by target detection according to an embodiment of the present application;

fig. 5 is an exemplary image two of a target component obtained by target detection according to an embodiment of the present application;

Fig. 6 is an exemplary diagram of a detection segment of a same line text on the same component provided in the embodiment of the present application;

fig. 7 is an exemplary diagram of a gray area between two text detection boxes on the same component according to an embodiment of the present application;

fig. 8 is an exemplary diagram of gray scale areas between two text detection boxes on different components according to an embodiment of the present application;

fig. 9 is an exemplary diagram of a text detection box on the same component in the same row provided in the embodiment of the present application;

FIG. 10 is a diagram of an example of results of merging peer text on a peer device according to an embodiment of the present application;

FIG. 11 is a diagram of a comparison example before and after threshold segmentation according to an embodiment of the present application;

FIG. 12 is a diagram illustrating multiple lines of text analysis on the same component provided in an embodiment of the present application;

fig. 13 is a diagram showing a structural example of a PCBA component defect detection system according to an application example of the present application.

Detailed Description

In order to make the present application solution better understood by those skilled in the art, the following description will clearly and completely describe the technical solution in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.

For ease of understanding, referring to fig. 1, an embodiment one of a method for detecting defects of a PCBA component provided in the present application includes:

and step 101, acquiring a PCBA image to be detected through an image acquisition device to obtain an image of the component to be detected.

The image acquisition device can be selected according to actual conditions, the method is not limited in this embodiment, the PCBA image acquisition system mainly comprises an industrial camera, an industrial lens, a light source, an image acquisition bracket and a computer, the Hokkawamor MV-CH120-10GM industrial camera and the MVL-KF1228-12MP industrial lens are adopted, the camera adopts CMOS sensing, the frame rate of 9.4fps is high, the real-time requirement is met, the 4096×3000 high-definition resolution is suitable for shooting components with the minimum size of 0.5cm×1cm of the research components, and the high-precision requirement is met; the interface is GigE, and the camera can be connected by inserting a network cable, so that the camera is convenient and quick; the annular light source is used as the light source in combination with the industrial lens of MVL-KF1228-12MP, so that the brightness of image shooting can be effectively improved to meet the hardware requirement of the invention; the computer is equipped with GPU3080TI, 12G is shipped.

The PCBA component shooting image can be directly checked through MVS software, and object distance, resolution, light intensity, affine transformation coefficient and the like are adjusted, so that texts in components can be clearly shot without overexposure, and a 4090 multiplied by 3000 resolution image, namely a component image to be detected, is obtained.

And 102, carrying out target detection on the component image to be detected by adopting a preset YOLO detection model to obtain a target component image, wherein the target component image comprises a component category and a component center coordinate.

The preset YOLO detection model in the embodiment can frame the PCB components in the component image to be detected, and give out category calibration to the framed target components. In the target detection process, the central coordinates of the components are selected for research and analysis, so that complexity is reduced, and compared with the process of taking the four corner coordinates of the detection frame, the risk of detection errors can be reduced to a certain extent by taking the central coordinates. In addition, the detection model is adopted to directly detect the target, the operation is simple and easy to perform, complicated instrument equipment is not needed, and the input cost of the instrument can be reduced to a certain extent.

And 103, carrying out preset angle recognition on text characters in the target component image by adopting a preset text recognition model to obtain component text information, wherein the component text information comprises text content and text center coordinates.

The preset text recognition model is mainly used for recognizing text characters on components in the target component image, attribute data or model information of some components are marked on the components generally, and the character content on each PCB component has obvious difference. In this embodiment, besides detecting a specific component area, the text information on the component is analyzed, because the category or the position of the component can be determined according to the text character information of different components, and based on this, whether the component is stuck back, neglected loading, misplacement, tin bead short circuit, polarity error and other conditions can be detected. That is, the method of the embodiment can be suitable for more component defect detection scenes, and the adaptation degree is higher.

And 104, performing defect comparison detection according to the reference sample information, the target component image and the component text information to obtain a component defect detection result.

It can be understood that the reference sample information is related information of normal components in the conventional PCBA image without defects, the types of the information are consistent with or correspond to the target component image and the component text information researched by the embodiment, and defect detection of information comparison can be performed, so that a component defect detection result is obtained. The reference sample information may be obtained by the method as mentioned in the present embodiment, or may be information obtained by other means, as long as the information corresponds to and can be compared reliably, which is not limited herein.

According to the PCBA component defect detection method, the PCBA image is analyzed and identified through the detection model, the defect detection of the component can be completed without complex instruments and equipment, and the method is simple and easy to operate; in addition, in the detection process, not only the component area is focused, but also the text information on the components is identified and analyzed, and the electrical property of each component can be clearly grasped, so that a plurality of different defect types can be judged in a targeted manner, and the defect detection method can be suitable for defect detection of a plurality of different components. Therefore, the embodiment of the application can solve the technical problems that the prior art is either too complex to operate or low in adaptation degree and cannot meet the defect detection requirement of the PCB component.

For ease of understanding, referring to fig. 2, the present application provides a second embodiment of a method for detecting defects of a PCBA component, including:

step 201, acquiring a PCBA image to be detected through an image acquisition device to obtain a component image to be detected.

Step 202, carrying out pretreatment operation and labeling operation on the component image to be detected in sequence, wherein the pretreatment operation comprises cutting, affine transformation, median filtering and sharpening.

The pretreatment aims are mainly three, one is to adjust the image size, so that the subsequent model processing analysis is facilitated; the other is data enhancement; and finally, the image quality is improved. For example, the cropping operation summarized in this embodiment is to unify the image sizes, and affine transformation is used to enhance the image dataset. It will be appreciated that the enhanced graphic dataset is primarily for the model training phase, not some single target image detection phase.

Cutting the acquired PCBA component 4090×3000 resolution image, cutting the interested PCBA detection area, cutting the size of the resolution of the cut image to be about 640×640 as much as possible, and cutting the image as much as possible, wherein the component is complete and free as much as possible, and the incomplete label can influence the detection precision; the dataset collated 6362 sheets.

The affine transformation in the embodiment is mainly embodied on the expansion of the data set, and because the PCBA sample types are unbalanced in the embodiment, a series of preprocessing means are adopted to properly expand the data set, so that the subsequent model training is facilitated. The main methods include horizontal overturn, vertical overturn according to probability, random cutting, random rotation and other rotation modes, and other methods can be added, such as standardization, addition of three-channel images and the like, and the method is not limited herein.

Compared with other detection objects, the PCBA component has the characteristics of small volume, high density, large quantity and the like, for example, a pcb board with the area of 40 multiplied by 60cm, wherein hundreds of components of the millboard can exist, and the area of the small components is only 0.5cm multiplied by 0.5cm. In addition, in this embodiment, it is necessary to extract text information to detect defects of components, and in order to improve the detection reliability, it is necessary to ensure the quality of PCBA images. The preprocessing means for improving the image quality in this embodiment includes contrast enhancement, sharpening, black-and-white contrast, median filtering, gamma conversion, and the like, and other preprocessing means may be added as needed, which is not limited herein.

And marking the image of the component to be detected, wherein the marking comprises marking the PCB component, the tin ball short circuit and the missing component. The data set can identify up to 25 types at present, including capacitors, resistors, chips, diodes, detection points, jumpers, inductors, triodes, transformers, patch diodes, tantalum electrolytic capacitors, aluminum electrolytic capacitors, cement resistors, metal film resistors, test points, chip inductors, series rectifier diodes, safety capacitors, schottky diodes, ceramic capacitors, fuses, sockets, stacked inductors, tin ball shorting and device neglected loading, and the contents cover most conventional components; the dataset collates 6362 pictures with a total number of labels up to 58480. Besides the component type and the defect type, the labels can be direction labels, namely, the distribution angles of components or text characters in the image, such as 0 degree, 90 degrees, 180 degrees, 270 degrees and the like, and the direction labels are used as references to form a data set marked by the direction.

If in the model training stage, a large number of component images need to be processed, a training set, a verification set and a test set need to be divided, the dividing ratio given by the embodiment is 6:2:2, only one reference is used here, and other dividing ratios can be designed according to actual conditions. In addition, other preprocessing means are required for the data enhancement method in which affine transformation may not be applied to the data set formed by the direction labels, and the method is not particularly limited herein.

Step 203, constructing an initial YOLO detection model based on the upsampling mechanism, the SE attention mechanism and the small object detection mechanism, wherein the initial YOLO detection model comprises a plurality of SPD convolution kernels.

And 204, performing target detection training on the initial YOLO detection model by adopting a multi-scale training mode and a weighted image training strategy to obtain a preset YOLO detection model.

And 205, carrying out local target detection on discrete components in the component image to be detected by adopting a preset YOLO detection model to obtain a target component image, wherein the target component image comprises component categories and component center coordinates.

In this embodiment, some improvements are made on the original YOLOv7 detection model, where a small target detection layer formed by a small target detection mechanism can splice a shallow feature map and a deep feature map, and the size of an anchor of the small target detection layer can count the average size of a small target candidate frame, so as to obtain a comprehensive size. Because the downsampling multiple in the original YOLOv7 model is large, the feature information of a small target is difficult to learn by a feature map, and small-sized components in the PCBA image cannot be accurately detected, so that a small target detection layer is required to be added for adjustment and perfection.

The up-sampling mechanism is added to avoid the problem caused by the down-sampling mechanism in the original model, the down-sampling mechanism and the multiple are too fast, so that the small target information is lost, and the up-sampling can compensate the problem. The optimal up-sampling mode can be obtained through learning by adopting an SPD convolution kernel to replace a nearest neighbor interpolation method. In addition, limited visual information processing resources can be reasonably utilized by the SE attention mechanism, the special part of characteristic information is focused, and the detection accuracy is improved.

The method comprises the steps of introducing a multi-scale training and a weighted image strategy, wherein the multi-scale training is to set several pictures with different scales, randomly selecting one scale training at intervals of certain iteration times during training, generating several feature images with different scales during testing, and selecting a candidate frame closest to a certain fixed size (namely the input size of a detection head) as a subsequent input, wherein each candidate frame also has different scales on the different feature images. Feature maps, which are tens of times smaller than the original image, are often generated when the network is downsampled, so that feature descriptions of small objects are not easily captured by the detection network. The images with larger and more sizes are input for training through multi-scale training, so that the robustness of the detection model to the size of the object can be improved to a certain extent.

The weighting image strategy is mainly used for solving the problem of class unbalance, for example, the data sets of the components have 25 classes, but the class differences among the components are obvious, the training can lead to more biased classes of network weights and influence the accuracy of small classes. I.e. the order in which each picture is taken is determined based on its category weight and the weight that is the sample.

The local detection is used for replacing global detection, the local image of the acquired component replaces the global image, for example, an image with 4096 multiplied by 3000 resolution can be divided into a plurality of component local images, and the problem that the detection precision of a small target is low due to the fact that the resolution of the global detection image and the resolution of the small target component are too large is solved. The local detection only collects images with components instead of the whole PCBA, does not collect components outside the background and the data set, and avoids the influence of irrelevant data on the identification effect. In addition, the phenomenon of ghosting appears due to the fact that edge information is disordered and fuzzy caused by local image splicing, accuracy is low, calculation amount of splicing large-sized images is large, time is spent, and quality analysis, maximum value suppression and other operations are needed to be carried out on the images.

In addition, in this embodiment, conv with a step length of 2 in the Head module of the preset YOLO detection model is replaced by SPD-Conv, and Strided Convolution convolution step length or pooling layer in the conventional convolution layer can cause the characteristic trouble of losing fine-grained information and learning deficiency, while SPD-Conv can realize convolution without step length so as not to lose learnable information during downsampling. CBS represents a new introduced modified convolution module of YOLOV 7. Their convolution kernels and step sizes consist of a Conv convolution layer, a batch regularization layer (batch normalization) and a Silu activation function layer. The SPD-Conv module consists of an SPD layer and a non-strided convolution layer. The image of the target component obtained by performing target detection using the preset YOLO detection model of the present embodiment is shown in fig. 4 and 5.

And 206, judging missing of the components and short circuit of the tin beads according to the target component image, and obtaining a preliminary defect detection result.

Referring to fig. 4 and 5, the problems of missing components and short circuit of tin beads can be primarily determined according to the detected target component image, namely, the components are not welded on a board, and the welding parts of the tin beads bulge or adhere to cause short circuit and the like; other defects are currently undetectable, and are therefore preliminary defect detection results.

Step 207, performing peer-to-peer and peer-to-peer object detection on text characters in the object component image based on an improved DBNet algorithm in a preset text recognition model to obtain a component text region image, wherein the component text region image comprises a text detection box and text center coordinates.

Further, step 207 comprises:

analyzing whether the text detection boxes in the same line are on the same component or not based on a preset gray level fluctuation principle, and merging and sorting the text detection boxes in the same line and the same component to obtain a component text region image.

When the original DBNet algorithm performs target detection, texts with the same row or the same column but with a slightly larger distance are treated as two sections, and some texts can only be detected as one section, if the texts are detected as two sections, referring to FIG. 6, the central coordinates of the texts can be directly changed, so that the errors of subsequent comparison detection are caused. Therefore, the embodiment improves the DBNet algorithm, judges whether the detected texts are in the same line or the same column and whether different text detection boxes are located on the same component, if not, the text detection boxes are not needed to be combined, and if so, the text detection boxes are combined to form a finished text detection box in the same line or the same column. The method specifically judges whether different text detection boxes are on the same component or not, can analyze and judge according to the principle that obvious gray level fluctuation does not exist between texts on the same component, and has obvious gray level fluctuation necessarily.

The gray scale fluctuation judging method mainly comprises the following steps: firstly, obtaining the center coordinates of all text detection boxes, and setting two thresholds sigma with fixed values _x Sum sigma _y The method comprises the steps of carrying out a first treatment on the surface of the Then, judging that if the ordinate or the abscissa of the two different texts is consistent and the absolute value of the subtraction of the abscissa or the ordinate is smaller than at least one of the two thresholds, performing image clipping according to the coordinates to obtain an image area between the two texts, and referring to fig. 7 and 8; finally, carrying out threshold calculation on the image area obtained by cutting, if the calculated image gray level distribution is uniform, judging that the two texts belong to the same component, and combining the two text detection boxes; if not, it is stated that the two texts are not on the same component.

Referring to fig. 6, the component is labeled a, and the same row of components "P", "CS24", "16", and "48" are all divided into two sections during detection; referring to fig. 9, all the peer texts are successfully detected on the b component. This is due to the unequal spacing of the text in the same row on the component, and the same may be the case for other reasons, such as illumination in the environment, etc. Regardless of how the center coordinates of the texts eventually cause a relatively obvious difference, the improved method provided by the embodiment can judge the gray scale area between the texts "P" and "CS24", so that the two text detection boxes can be combined to obtain the detection effect shown in fig. 10.

And step 208, after the component text area image is adjusted to a preset angle, identifying text characters in the component text area image by adopting a preset SVTR algorithm in a preset text identification model, so as to obtain component text information, wherein the component text information comprises text content and text center coordinates.

The text region images of the components may be distributed at different angles, so that in order to facilitate subsequent comparison and detection, in this embodiment, all the texts are adjusted to the same preset angle, and the texts already at the preset angle are not processed. The preset angle can be selected according to the actual situation, and an example is given in this embodiment, where the preset angle is selected to be 0 degrees.

The SVTR algorithm is preset to replace a converters structure with an RNN structure, so that the context information of the text image can be more effectively mined, and the recognition accuracy is improved. It can be understood that all the text region images of the components need to be subjected to preset angle adjustment, and the size is unified through padding processing, so that the unified recognition of the models is facilitated.

The text content can reflect the specific model or attribute information of the component, and the connection requirement and the position requirement of the component can be clarified according to the information, so that more kinds of defects of the component can be detected based on the text content.

The data sets for text region image formation may be divided into text angle data sets and text data sets according to an angle processing procedure, and some text image processing operations may be performed on these data sets, so as to improve text recognition reliability, such as scaling or threshold segmentation.

The scaling method mainly comprises the following steps: 1. respectively solving the scaling of the image length and width and the target size, and finding out the minimum scaling factor; 2. scaling the image according to the minimum scaling factor of the previous step; 3. padding the scaled image to a target size; the image size is (640, 360), and the network input size is (192 ), the length-width scaling is calculated first, 192/640=0.3; 192 ≡360=0.533, so 0.3 is selected as the scaling ratio multiplied by the original length and width, and the image is scaled (192, 108). The next step is to fill 108 to 128 so that it can be divided by 32, resulting in the final picture size (192, 128). It should be noted that, although the image input size satisfies the model input requirement after the image is subjected to the adaptive Padding processing, the edge of the image generates a gray edge, which is an invalid region, and affects not only the algorithm processing speed but also the detection accuracy. Therefore, the embodiment improves the packing algorithm 1. The scaling ratio of the image length and width to the target size is respectively calculated, and the minimum scaling factor is found; 2. scaling the image according to the minimum scaling factor of the previous step; 3. for pixel values that do not reach a multiple of 32, if the value that decreases to a multiple of 32 is less than the value that increases to a multiple of 32, then the edges are clipped uniformly to the target pixel, otherwise Padding to the target pixel. The original image size is 640×360, after scaling (192, 108), 108 does not reach the multiple of 32, so the pixel point where 108 reaches the nearest multiple of 32 is calculated to be 96, only 12 pixels are needed, and the pixels from 108Padding to 128 are needed to be 20 pixels, so the image is cut to the edge uniformly (192, 96), and the improved algorithm of the embodiment removes part of irrelevant information, so that the main information occupies more proportion of the image, and the improvement of the algorithm feature extraction capability is facilitated.

The threshold segmentation is to maximize the gray level difference between the text and the background, so that the text direction classifier can better acquire text edge information. In this embodiment, a global-based threshold segmentation algorithm is selected, and in order to avoid inaccurate segmentation points selected by global threshold segmentation, please refer to fig. 11, an Otsu threshold segmentation algorithm is adopted, and the principle is that different thresholds are traversed according to the distribution of gray values on an image, intra-class variances between a background and a foreground corresponding to the different thresholds are calculated, and when the intra-class variances obtain maximum values, the corresponding thresholds are the optimal segmentation thresholds.

Step 209, extracting reference sample information based on the defect-free PCBA image to form a reference sample information list.

The reference sample information in this embodiment, also referred to as golden sample information, is a comparative reference, and is information corresponding to the information in this embodiment extracted by the defect-free PCBA image. The coordinate standard can be compared with the information of the sample to be detected, so that the defect of the component is detected. The reference information list updates the information content and the information category according to the comparison detection requirement.

And 210, performing multiple defect comparison detection on the reference sample information and the target component image and the component text information according to the comparison threshold value to obtain a component defect detection result.

The comparison threshold values in this embodiment are multiple, and specifically include a confidence threshold value, a component center coordinate matching threshold value of a normal component and a component to be tested, a component center coordinate and text center coordinate matching threshold value, and a text threshold value. In addition, the method also comprises three identification parameters of component category, text angle judgment and text content judgment.

And comparing the information of the PCBA board to be detected formed by the target component image and the component text information with the reference sample information, directly obtaining a result if one item of the information is not matched in the gradual comparison process, and terminating the flow, for example, judging that the component is wrongly installed if the category is not matched in the component category analysis, and not needing to carry out subsequent detection. The specific judgment is as follows:

1. comparing whether the confidence coefficient of the component to be tested is smaller than a threshold value, if so, continuing to carry out subsequent judgment, and if so, judging that the data is invalid, and ending the flow;

2. comparing whether the difference between the center coordinates of the to-be-detected component and the center coordinates of the normal component is smaller than a threshold value, if so, continuing to carry out subsequent judgment, and if the difference is larger than or equal to the threshold value, judging that the data is invalid, and ending the flow;

3. judging whether the categories of the components are consistent or not, if not, judging that the components are wrongly installed;

4. Judging whether the component has a text or not, if so, entering a 5 th point, and if so, entering a 6 th point;

5. judging whether the to-be-tested component and the reference component at the same position have no text, if not, the defect type is component misloading;

5. if the text exists, judging whether the difference between the text center coordinates and the component center coordinates is smaller than a threshold value, if so, judging that the data is invalid, and ending the flow;

6. judging whether the angles of the texts of the components are consistent or not, if not, judging that the components are reversely attached, if so, entering a 7 th point;

7. and judging whether the text content identification of the component is smaller than a text threshold value. For example, the threshold value is set to 80%, the normal text is "12345", the defective text is detected to be "12346", the threshold value condition is met or more, and the device is error-free. If the detection result is smaller than the threshold value, judging that the component is wrongly mounted.

It should be noted that, for the above coordinate contrast analysis process, a specific text coordinate contrast analysis scheme is provided in this embodiment, and the specific description is as follows:

1) Counting text coordinate information of all components and the number of the components, and setting the total number of the components as N;

2) Setting the number of single-row text components and non-text components as T;

3) Because a single line of text can be matched directly to the component. Therefore, text coordinates on a plurality of lines of text components are screened out from text coordinate information of all components to form a data set n;

4) Calculating the number K of the components with a plurality of lines of texts by using a K-means++ clustering algorithm, wherein the number of the components with the lines of texts is equal to the total number N of the components minus the number T of the components with the single line of texts and the number T of the components without the texts, so that K is equal to N-T;

5) Selecting K initial clustering centers C from a data set n _i (i.ltoreq.1.ltoreq.k), and the sample points farther from the other cluster centers are more likely to be selected as the next cluster center, the remaining data objects are calculated together with the cluster center C _i Finding the nearest cluster center C from the target data object _i And assign data objects to cluster centers C _i In the corresponding cluster. Then calculating the average value of the data objects in each cluster as a new cluster center, and carrying out the next iteration until the cluster center is not changed or the maximum iteration times are reached;

6) K multi-line text clusters are obtained through calculation, and the clusters can be matched with the components.

For the example of the multi-line text, referring to fig. 12, the "1002" on the resistor belongs to the single-line text, and the center coordinate of the resistor corresponds to the center coordinate of the resistor. The "106", "16K" and "847" on the tantalum electrolytic capacitor belong to a plurality of lines of text, which cannot directly correspond to the central coordinates of the tantalum electrolytic capacitor, and the K-means clustering algorithm of the embodiment is used to detect whether the plurality of lines of text are on the component, and the analysis shows that the "106", "16K" and "847" correspond to the central coordinates of the tantalum electrolytic capacitor, i.e. the text are located on the same component.

In this embodiment, mainly for defect detection of some components with text characters, if the components have no text characters, for example, an aluminum electrolytic capacitor, the aluminum spot electrolytic capacitor is divided into an aluminum electrolytic capacitor with text and an aluminum electrolytic capacitor without text, and for the aluminum electrolytic capacitor without text, although the aluminum electrolytic capacitors with different specifications are different in style, the area with a small block of surface and different in color represents a negative electrode, so that polarity detection can be performed to determine whether the components are reversely assembled. Summarizing in the embodiment, if the comparison detection stage is carried out, and the component class is an aluminum electrolytic capacitor without text, intercepting an image according to the PCBA component target detection result, carrying out threshold segmentation on the image, and then carrying out affine transformation to improve the skew problem during interception; and then, calculating characteristic points for the images of the to-be-detected and reference samples simultaneously, and calculating characteristic point matching scores, if the characteristic point matching scores are higher than a preset value, the patch is proved to be correct, otherwise, the aluminum electrolytic capacitor reverse installation can be deduced.

It can be understood that for the capacitor, the chip, the diode, the detection point, the jumper, the inductor, the triode, the transformer, the cement resistor, the metal film resistor, the test point, the chip inductor, the series rectifier diode, the safety capacitor, the schottky diode, the ceramic capacitor, the fuse, the socket and the stacked inductor, which are all components without polarity requirements and text, the wrong installation problem of the device can be judged directly through the detection type.

In order to facilitate understanding, the method in this embodiment is applied to an actual system, please refer to fig. 13, which mainly includes an image acquisition module, a light source illumination module, a motion control module and a computer design module, and the image acquisition module is used to acquire an image of the moving PCBA, and transmit the image to the computer for image processing, so as to obtain a PCB component defect detection result.

For ease of understanding, referring to fig. 3, the present application further provides an embodiment of a PCBA component defect detection apparatus, including:

the image acquisition unit 301 is configured to acquire an image of the PCBA to be detected by using the image acquisition device, so as to obtain an image of the component to be detected;

the target detection unit 302 is configured to perform target detection on a component image to be detected by using a preset YOLO detection model, so as to obtain a target component image, where the target component image includes a component category and a component center coordinate;

a text recognition unit 303, configured to perform preset angle recognition on text characters in the target component image by using a preset text recognition model, so as to obtain component text information, where the component text information includes text content and text center coordinates;

and the defect detection unit 304 is configured to perform defect contrast detection according to the reference sample information, the target component image, and the component text information, so as to obtain a component defect detection result.

Further, the target detection unit 302 includes:

a model construction subunit 3021 for constructing an initial YOLO detection model based on the upsampling mechanism, the SE attention mechanism, and the small target detection mechanism, the initial YOLO detection model comprising a plurality of SPD convolution kernels;

a model training subunit 3022, configured to perform target detection training on the initial YOLO detection model by using a multi-scale training manner and a weighted image training strategy, so as to obtain a preset YOLO detection model;

and the target detection subunit 3023 is configured to perform local target detection on the discrete components in the component image to be detected by using a preset YOLO detection model, so as to obtain a target component image, where the target component image includes a component category and a component center coordinate.

Further, the text recognition unit 303 includes:

the region detection subunit 3031 is configured to perform peer-to-peer and peer-to-peer target detection on text characters in a target component image based on an improved DBNet algorithm in a preset text recognition model, so as to obtain a component text region image, where the component text region image includes a text detection box and a text center coordinate;

and the text recognition subunit 3032 is configured to recognize text characters in the component text region image by using a preset SVTR algorithm in a preset text recognition model after adjusting the component text region image to a preset angle, so as to obtain component text information, where the component text information includes text content and text center coordinates.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions to execute all or part of the steps of the methods described in the embodiments of the present application by a computer device (which may be a personal computer, a server, or a network device, etc.). And the aforementioned storage medium includes: u disk, mobile hard disk, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.

The above embodiments are merely for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.

Claims

1. A PCBA component defect detection method is characterized by comprising the following steps:

2. The method for detecting defects of PCBA components according to claim 1, wherein the step of acquiring the PCBA image to be detected by the image acquisition device to obtain the component image to be detected further comprises:

3. The PCBA component defect detection method according to claim 1, wherein the performing target detection on the component image to be detected using a preset YOLO detection model to obtain a target component image, the target component image including a component category and a component center coordinate, includes:

4. The PCBA component defect detection method according to claim 1, wherein the performing target detection on the component image to be detected using a preset YOLO detection model to obtain a target component image, and further comprising:

5. The PCBA component defect detection method according to claim 1, wherein the performing preset angle recognition on text characters in the target component image using a preset text recognition model to obtain component text information comprises:

6. The PCBA component defect detection method according to claim 5, wherein the performing object detection of the same line and the same component on text characters in the object component image based on the modified DBNet algorithm in the preset text recognition model to obtain a component text area image comprises:

7. The PCBA component defect detection method according to claim 1, wherein the performing defect contrast detection according to the reference sample information, the target component image, and the component text information to obtain a component defect detection result includes:

8. A PCBA components and parts defect detection device, characterized by, include:

9. The PCBA component defect inspection apparatus of claim 7, wherein the target inspection unit comprises:

10. The PCBA component defect detection apparatus as recited in claim 7, wherein the text recognition unit comprises: