CN115601341A

CN115601341A - Method, system, equipment, medium and product for detecting defects of PCBA (printed circuit board assembly) board

Info

Publication number: CN115601341A
Application number: CN202211317805.XA
Authority: CN
Inventors: 马彩丰; 危进伟
Original assignee: Shanghai Wingtech Information Technology Co Ltd
Current assignee: Shanghai Wingtech Information Technology Co Ltd
Priority date: 2022-10-26
Filing date: 2022-10-26
Publication date: 2023-01-13

Abstract

The present disclosure relates to a method, system, device, medium and product for detecting defects of a PCBA board, comprising: inputting a PCBA image to be detected into a PCBA defect detection model, and outputting a defect type and a defect area, wherein the PCBA defect detection model is pre-trained by taking a PCBA image data set as a sample and marking the PCBA image in the PCBA image data set with the defect area and the defect type as a label, the PCBA image data set comprises a real good PCBA image, a real bad PCBA image and an augmented PCBA image obtained through data augmentation, and the data augmentation at least comprises one of image enhancement augmentation, simulated image augmentation and fused image augmentation; and screening the defect type and the defect area based on a preset threshold value, and outputting a defect result of the PCBA to be detected. According to the method and the device, the PCBA image data set has a large amount of sample data through data augmentation, and therefore the prediction accuracy of the PCBA defect detection model is improved.

Description

Method, system, equipment, medium and product for detecting defects of PCBA (printed circuit board assembly) board

Technical Field

The present disclosure relates to the field of PCBA board inspection technologies, and in particular, to a method, a system, a device, a medium, and a product for inspecting defects of a PCBA board.

Background

The Printed circuit board assembly (PCB) refers to a Printed Circuit Board (PCB) containing components and parts after an empty Printed Circuit Board (PCB) is subjected to SMT. In the actual production of the existing manufacturer, the quality of the PCBA needs to be detected in order to ensure the product quality, and then whether the quality of the circuit board is in a problem or not and whether the position of the mounted component is correct or not are judged. Surface Mounted Technology (SMT), which is a short of a series of process flows processed on the basis of a PCB, is a technology and process widely used in the electronic assembly industry, one of the current detection technologies for PCBA is detection by manual visual inspection, and because human visual angles are limited (for example, desoldering between small-sized patch elements cannot be effectively observed, polarities are opposite, and the like), the defects of low detection speed, low accuracy and poor detection effect exist, and thus the usage of the method is gradually reduced. The other way is to judge through an AOI system (automatic optical detection, which is a device for detecting common defects encountered in welding production based on an optical principle); although the AOI system adopts an optical principle, uses an optical lens to replace human eyes, and performs image amplification in the shooting process, so as to obtain a relatively clear device image, the current AOI system has a defect that the method for judging whether a detection point is faulty is manually based on a standard digital image stored in the AOI system to compare and judge with an actually detected image, that is, manual visual comparison and detection are also needed, so that the defects of low detection speed, missing detection and low accuracy exist. The PCBA appearance is inspected through a convolutional neural network, and due to the fact that the number of defective components in the production process is small, the form of a defective sample is too single, the accuracy of the obtained detection model is insufficient, and therefore in actual production, accurate prediction cannot be achieved through the detection model, and the defects of missing detection and low accuracy still exist.

Disclosure of Invention

To overcome the problems in the related art, the present disclosure provides a PCBA board defect detection method, system, device, medium, and product.

According to a first aspect of the embodiments of the present disclosure, there is provided a PCBA board defect detection method, including:

inputting a PCBA image to be detected into a PCBA defect detection model, and outputting a defect type and a defect area, wherein the PCBA defect detection model is pre-trained by taking a PCBA image data set as a sample and marking the PCBA image in the PCBA image data set with the defect area and the defect type as a label, the PCBA image data set comprises a real good PCBA image, a real bad PCBA image and an augmented PCBA image obtained through data augmentation, and the data augmentation at least comprises one of image enhancement augmentation, simulated image augmentation and fused image augmentation;

and screening the defect type and the defect area based on a preset threshold value, and outputting a defect result of the PCBA to be detected.

In some embodiments, the screening the defect type and the defect area based on a preset threshold value, and outputting a defect result of the PCBA to be detected includes:

when the defect type is a defect affecting the function of the PCBA to be detected, outputting a defect result of the PCBA to be detected as a defective product;

and when the defect type is a defect which does not influence the function of the PCBA to be detected and the defect area does not exceed the preset threshold value, outputting a defect result of the PCBA to be detected as a good product.

In some embodiments, the image enhancement augmentation comprises:

and the image enhancement and the augmentation are completed by one mode at least comprising image zooming, gaussian noise adding, polarity device overturning and PCBA image rotation on the real good product PCBA image and the real bad PCBA image.

In some embodiments, the analog image augmentation comprises:

acquiring the defect area of the real poor PCBA board image, and drawing the defect area to form a defect image set;

and after the Gaussian blur processing is carried out on the defect image set, randomly pasting the defect image set to a good image of the same type as the real and defective PCBA image to form a simulated defective image set.

Further, the fused image augmentation includes:

placing the real good PCBA board image, the real bad PCBA board image and the simulation bad product image set under the same folder;

randomly acquiring PCBA (printed circuit board assembly) plate images of a preset number, wherein the PCBA plate images of the preset number at least comprise a bad PCBA plate image;

preprocessing the PCBA plate images of the preset number;

and merging and outputting the preprocessed PCBA images with the preset number into a PCBA image.

Further, the preprocessing includes at least one of flipping, scaling, and color gamut changing.

According to a second aspect of the embodiments of the present disclosure, there is provided a PCBA board defect detection system, including:

the detection module is used for inputting a PCBA image to be detected into a PCBA defect detection model and outputting a defect type and a defect area, wherein the PCBA defect detection model is pre-trained by taking a PCBA image data set as a sample and taking a PCBA image in the PCBA image data set marked with the defect area and the defect type as a label, the PCBA image data set comprises a real good PCBA image, a real bad PCBA image and an augmented PCBA image obtained through data augmentation, and the data augmentation at least comprises one of image enhancement augmentation, simulated image augmentation and fused image data augmentation;

and the screening module screens the defect type and the defect area based on a preset threshold value and outputs a defect result of the PCBA to be detected.

Embodiments of the third aspect of the present application provide an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by the processor to implement the steps of the PCBA board defect detection method provided by embodiments of the first aspect of the present application described above.

An embodiment of a fourth aspect of the present application provides a non-transitory computer-readable storage medium, where instructions, when executed by a processor of a mobile terminal, enable the mobile terminal to perform the steps of the PCBA board defect detection method provided in the above-described embodiment of the first aspect of the present application.

An embodiment of a fifth aspect of the present application provides a computer program product, wherein when the instructions of the computer program product are executed by a processor of a mobile terminal, the mobile terminal is enabled to execute the steps of implementing the PCBA board defect detection method provided by the embodiments of the first aspect of the present application.

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects: this application obtains the image of a large amount of yields and bad PCBA board through carrying out data amplification to real yields PCBA board image and real bad PCBA board image. Therefore, through simulation and training of different defect modes of images of the poor PCBA, the robustness of defect types which are not collected in the model detection production process is improved, the detection rate of defective components is improved, and the prediction accuracy of the PCBA defect detection model is improved. Therefore, the missing detection of the original manual detection or AOI equipment method is compensated, and the detection accuracy and efficiency are improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

FIG. 1 is a flow chart illustrating a method of PCBA board defect detection, in accordance with an exemplary embodiment.

FIG. 2 is a block diagram illustrating a PCBA board defect detection system in accordance with an exemplary embodiment.

Fig. 3 is an internal block diagram of an electronic device shown in accordance with an example embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.

Fig. 1 is a flow chart illustrating a method of defect detection for a PCBA board, according to an exemplary embodiment, as shown in fig. 1, including the following steps.

In step S101, inputting a PCBA board image to be detected into a PCBA board defect detection model, and outputting a defect type and a defect area, wherein the PCBA board defect detection model is pre-trained by using a PCBA board image data set as a sample, and using the PCBA board image in the PCBA board image data set to mark a defect area and a defect type as a label, the PCBA board image data set includes a real good PCBA board image, a real bad PCBA board image, and an augmented PCBA board image obtained through data augmentation, and the data augmentation at least includes one of image augmentation, simulated image augmentation, and fused image augmentation.

Specifically, during actual production, the image of the PCBA can be acquired by photographing or monitoring the PCBA to be detected, and the defect type and the defect area of the PCBA can be acquired by inputting the image of the PCBA to be detected into the defect detection model of the PCBA, wherein the defect type is divided into 7 items including 'broken filament, polar opening, continuous tin, less tin, damage and dirt' and 'polar identifier' in general. In some embodiments, the PCBA board defect detection model is a PCBA board image dataset pre-trained by using a PCBA board image dataset as a sample, and using defects of a PCBA board image labeled with a defect area and a defect type in the PCBA board image dataset as labels, wherein the PCBA board image dataset comprises a real good PCBA board image and a real bad PCBA board image, and an augmented PCBA board image obtained by data augmentation, and the data augmentation at least comprises one of image augmentation, simulated image augmentation, and fused image augmentation.

Specifically, real bad PCBA and real good product PCBA images are collected through normal production, and because of the fact that in collected real image samples, the number of defective components is small, the form of defective samples is too single, and therefore training samples are insufficient, the obtained PCBA defect detection model is inaccurate, therefore, the training samples are increased through an image augmentation mode, and an image augmentation (image augmentation) technology is used for generating similar but different training samples through a series of random changes of the training images, and accordingly the scale of a training data set is enlarged. The method comprises the steps of obtaining an augmented PCBA image by carrying out at least one image augmentation mode of image enhancement augmentation, simulated image augmentation and fused image augmentation on images of real poor PCBA and real good PCBA, so as to obtain a PCBA image data set, and finally marking defect positions in the PCBA image data set and adding defect type labels by using an open source marking tool on a visual interface by taking the PCBA image data set as a sample. FPN network training parameter initialization is done using weights trained by imagenet (a large set of source data for the field of computer vision deep learning), FPN: a Feature Pyramid Network (FPN) is a network structure for continuously extracting image features of different receptive field ranges from low to high resolution and fusing features, and is often used for image detection and image segmentation, and the FPN model is trained by dividing a training set and a verification set for a PCBA sample defect area image sample set, and iterating the model according to the output of the model and a loss function between labels until the PCBA plate defect detection model meets the production detection requirements. The method and the device use the trained weight provided by the imagenet official as the FPN network training parameter initialization, accelerate the network convergence, adjust the network parameters more quickly and achieve the detection purpose in shorter time.

In some embodiments, the image enhancement augmentation comprises:

and finishing the image enhancement and augmentation by at least one mode of image scaling, gaussian noise addition, polar device overturning and PCBA image rotation on the real good PCBA image and the real bad PCBA image.

Specifically, image zooming (zooming ratio: 0.9-1.2) is adopted, gaussian noise is added, the polarity identification piece is subjected to mirror image inversion, the image rotation angle is (-3 degrees, and if the angle is too large, the image rotation angle is a defective piece), and therefore PCBA images subjected to image enhancement are obtained.

In some embodiments, the analog image augmentation comprises:

acquiring the defect area of the real poor PCBA image, and drawing the defect area to form a defect image set;

and after the Gaussian blur processing is carried out on the defect image set, the defect image set is randomly pasted to a good image of the same type as the real poor PCBA image to form a simulated poor product image set.

Specifically, the method comprises the following steps:

s1: cutting out a defect area of a real poor PCBA board image, drawing a plurality of defect areas with different forms by using traditional drawing software, and storing the defect areas as a defect image set;

s2: selecting a plurality of good component images which can generate corresponding types of defects to cut the position area blocks which can generate the defects to obtain a series of sub-block images;

s3: respectively carrying out template matching on the images of the same type of good products by utilizing the sub-block images to find out position coordinates on the images of the same type and form a position coordinate list;

s4: and (3) after the defective image sub-blocks are subjected to Gaussian blur processing, randomly pasting the defective image sub-blocks to the good component image set from any positions in the position coordinate list in the step (3) to form a simulated defective image set. The number, angles and positions of the defective image sub-blocks pasted on different good image are random, the maximum number capable of being pasted on one image depends on the number of the sub-block images, and the angles are between 0 and 360 degrees.

In some embodiments, the fused image augmentation comprises:

preprocessing the PCBA plate images of the preset number;

Specifically, the following steps are performed in combination with specific embodiments:

s21: putting the real good PCBA board image, the real bad PCBA board image and the simulation bad PCBA board image set under a folder:

s22: randomly reading 4 images, wherein the 4 images can be randomly good images or defective images but must contain at least one defective image;

s23: preprocessing 4 images, and placing the images at the upper, lower, left and right positions of a specified image canvas by a fixed zoom factor;

s24: and combining 4 images for step 3 and outputting the combined images as a PCBA board image.

In some embodiments, the pre-processing comprises at least one of flipping, scaling, and gamut changing.

In step S102, the defect type and the defect area are screened based on a preset threshold, and a defect result of the PCBA to be detected is output.

Specifically, the PCBA image to be detected is input into the PCBA defect detection model, defect types and defect areas are output, and the output defect types and defect areas are screened by the preset threshold value which needs to be set as some defect types do not affect the actual functions of the PCBA, so that the defect result of the PCBA to be detected is determined.

and when the defect type is a defect which does not affect the function of the PCBA to be detected and the defect area does not exceed the preset threshold value, outputting a defect result of the PCBA to be detected as a good product.

Specifically, in combination with the specific embodiment, for example, when the output defect type is continuous tin, the defective component is directly output regardless of the area size of the defect region, and if the polarity identifiers are not consistent, the defective component is also determined, and the defective component is directly output. When the defect type is a defect type which does not affect the function of the PCBA board, such as damage, little tin, broken filament, dirt and the like, a preset threshold value is set, when the area and the length of the defect area are smaller than the preset threshold value, the defect area is output as a good product, and otherwise, the defect area is output as a defective product.

FIG. 2 is a block diagram illustrating a PCBA board defect detection system in accordance with an exemplary embodiment. Referring to fig. 2, the apparatus includes a detection module 201 and a screening module 202.

The detection module 201 inputs a PCBA image to be detected into a PCBA defect detection model, and outputs a defect type and a defect area, wherein the PCBA defect detection model is pre-trained by taking a PCBA image data set as a sample and taking a PCBA image in the PCBA image data set marked with the defect area and the defect type as a label, the PCBA image data set comprises a real good PCBA image, a real bad PCBA image and an augmented PCBA image obtained through data augmentation, and the data augmentation at least comprises one of image augmentation, simulated image augmentation and fused image data augmentation;

and the screening module 202 is used for screening the defect type and the defect area based on a preset threshold value and outputting a defect result of the PCBA to be detected.

With regard to the system in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

In one embodiment, an electronic device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 3. The electronic device comprises a processor, a memory, a communication interface, a display screen and an input device which are connected through a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic equipment comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, near Field Communication (NFC) or other technologies. The computer program is executed by a processor to implement a method of detecting defects in a PCBA board. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on a shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the configuration shown in fig. 3 is a block diagram of only a portion of the configuration associated with the present application, and is not intended to limit the computing device to which the present application may be applied, and that a particular computing device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, the PCBA board defect detection system provided by the present application may be implemented in the form of a computer program that is executable on an electronic device such as that shown in fig. 3. The memory of the electronic device may store various program modules that make up the PCBA board defect detection system.

At least one instruction, at least one program, a set of codes, or a set of instructions is stored in a memory in the electronic device, and the instruction, the program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the PCBA board defect detection method according to any one of the above embodiments. For example, a method for detecting defects of a PCBA board includes: inputting a PCBA image to be detected into a PCBA defect detection model, and outputting a defect type and a defect area, wherein the PCBA defect detection model is pre-trained by taking a PCBA image data set as a sample and marking the PCBA image in the PCBA image data set with the defect area and the defect type as a label, the PCBA image data set comprises a real good PCBA image, a real bad PCBA image and an augmented PCBA image obtained through data augmentation, and the data augmentation at least comprises one of image enhancement augmentation, simulated image augmentation and fused image augmentation; and screening the defect type and the defect area based on a preset threshold value, and outputting a defect result of the PCBA to be detected.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, performs the steps of: inputting a PCBA image to be detected into a PCBA defect detection model, and outputting a defect type and a defect area, wherein the PCBA defect detection model is pre-trained by taking a PCBA image data set as a sample and marking the PCBA image in the PCBA image data set with the defect area and the defect type as a label, the PCBA image data set comprises a real good PCBA image, a real bad PCBA image and an augmented PCBA image obtained through data augmentation, and the data augmentation at least comprises one of image enhancement augmentation, simulated image augmentation and fused image augmentation; and screening the defect type and the defect area based on a preset threshold value, and outputting a defect result of the PCBA to be detected.

In one embodiment, a computer program product is provided, the instructions in which, when executed by a processor of a mobile terminal, enable the mobile terminal to perform the steps of: inputting a PCBA (printed circuit board assembly) image to be detected into a PCBA defect detection model, and outputting a defect type and a defect area, wherein the PCBA defect detection model is pre-trained by taking a PCBA image data set as a sample and defects of a defect area and a defect type marked on the PCBA image in the PCBA image data set as labels, the PCBA image data set comprises a real good PCBA image, a real bad PCBA image and an augmented PCBA image obtained through data augmentation, and the data augmentation at least comprises one of image enhancement augmentation, simulated image augmentation and fused image augmentation; and screening the defect type and the defect area based on a preset threshold value, and outputting a defect result of the PCBA to be detected.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by a computer program, which may be stored in a non-volatile computer-readable storage medium, and the computer program may include the processes of the embodiments of the methods described above when executed. Any reference to memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), and the like.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, however, as long as there is no contradiction between the combinations of the technical features, the scope of the present description should be considered as being described in the present specification.

The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not to be construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, and these are all within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A PCBA board defect detection method is characterized by comprising the following steps:

inputting a PCBA (printed circuit board assembly) image to be detected into a PCBA defect detection model, and outputting a defect type and a defect area, wherein the PCBA defect detection model is pre-trained by taking a PCBA image data set as a sample and defects of a defect area and a defect type marked on the PCBA image in the PCBA image data set as labels, the PCBA image data set comprises a real good PCBA image, a real bad PCBA image and an augmented PCBA image obtained through data augmentation, and the data augmentation at least comprises one of image enhancement augmentation, simulated image augmentation and fused image augmentation;

2. The PCBA defect detection method according to claim 1, wherein the step of screening the defect type and the defect area based on a preset threshold value and outputting a PCBA defect result to be detected comprises the following steps:

3. The PCBA board defect detection method of claim 1, wherein the image enhancement augmentation comprises:

4. The PCBA board defect detection method according to claim 1, wherein the simulated image augmentation comprises:

5. A PCBA plate defect detection method as in any of claims 3-4, wherein the fused image augmentation comprises:

placing the real good PCBA board image, the real defective PCBA board image and the simulated defective product image set under the same folder;

randomly acquiring PCBA (printed circuit board assembly) board images of a preset number, wherein the PCBA board images of the preset number at least comprise a bad PCBA board image;

preprocessing the PCBA plate images of the preset number;

and merging and outputting the preprocessed PCBA board images of the preset number into one PCBA board image.

6. A method of detecting defects in a PCBA board as claimed in claim 5, comprising: the preprocessing at least comprises one of turning, scaling and color gamut changing.

7. A PCBA board defect detection system, comprising:

the detection module inputs a PCBA image to be detected into a PCBA defect detection model and outputs a defect type and a defect area, wherein the PCBA defect detection model is pre-trained by taking a PCBA image data set as a sample and taking a PCBA image marked with the defect area and the defect type in the PCBA image data set as a label, the PCBA image data set comprises a real good PCBA image, a real bad PCBA image and an augmented PCBA image obtained through data augmentation, and the data augmentation at least comprises one of image enhancement augmentation, simulated image augmentation and fused image data augmentation;

8. An electronic device comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, set of codes, or set of instructions, the instruction, the program, the set of codes, or the set of instructions being loaded and executed by the processor to implement a PCBA board defect detection method in accordance with any of claims 1-6.

9. A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of a mobile terminal, enable the mobile terminal to perform the PCBA board defect detection method of any one of claims 1-6.

10. A computer program product, characterized in that the instructions in the computer program product, when executed by a processor of a mobile terminal, enable the mobile terminal to perform the PCBA board defect detection method according to any of the claims 1-6.