CN114240877A - Method and device for detecting welding quality - Google Patents
Method and device for detecting welding quality Download PDFInfo
- Publication number
- CN114240877A CN114240877A CN202111532686.5A CN202111532686A CN114240877A CN 114240877 A CN114240877 A CN 114240877A CN 202111532686 A CN202111532686 A CN 202111532686A CN 114240877 A CN114240877 A CN 114240877A
- Authority
- CN
- China
- Prior art keywords
- image
- region
- area
- chip
- detection result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/001—Industrial image inspection using an image reference approach
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30152—Solder
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The embodiment of the invention provides a method and a device for detecting welding quality, wherein the method comprises the following steps: determining a second image where each chip is located from the first image; the first image is obtained by shooting from the back of the substrate through a vision system; the front surface of the substrate is provided with a chip which is installed through a flip-chip process; the visual system is provided with shooting equipment and a light source which is coaxial with the shooting equipment; performing region segmentation on the second image to obtain at least one region; wherein, each pixel point in the same region has similarity; determining a characteristic vector of each region, and inputting the characteristic vector of each region into a multilayer perceptron model to obtain a welding detection result of each region; the multi-layer perceptron model is obtained by pre-training.
Description
Technical Field
The invention relates to the field of image processing, in particular to a method and a device for detecting welding quality.
Background
With the continuous development of economy, electronic products become essential things in people's lives, and among them, MiniLED (micro-scale LED array) panels are also applied to more and more electronic products. In the process of producing the MiniLED panel, one of the steps is to solder the chip on the bonding pad, and the problem of void solder and cold solder may occur during the soldering process.
At present, X-ray imaging is generally used for detecting the empty solder joint and the cold solder joint, but the X-ray can cause workers to be irradiated to influence the bodies, so that the X-ray detection device cannot be used on a production line in a large scale.
In summary, how to accurately determine the chip bonding condition without harming the health of the staff is a technical problem that needs to be solved at present.
Disclosure of Invention
The embodiment of the invention provides a method and a device for detecting welding quality, which are used for solving the problem that in the prior art, X-ray imaging is used for detecting the welding condition, so that workers are radiated, and the health of the workers is damaged.
In a first aspect, an embodiment of the present invention provides a method for detecting welding quality, where the method includes: determining a second image where the chip is located from the first image; the first image is obtained by shooting from the back of the substrate through a vision system; the front surface of the substrate is provided with a chip mounted by a flip-chip process; the visual system is provided with shooting equipment and a light source which is coaxial with the shooting equipment; performing region segmentation on the second image to obtain at least one region; wherein, each pixel point in the same region has similarity; determining a characteristic vector of each region, and inputting the characteristic vector of each region into a multilayer perceptron model to obtain a welding detection result of each region; the multilayer perceptron model is obtained through pre-training.
In the technical scheme, the welding condition of the chip can be shot under the flip-chip condition by shooting from the back surface of the substrate through the vision system; and determining a second image from the first image according to the position of the chip, so that the image area needing to be processed subsequently can be reduced, the detection speed is accelerated, then the second image is subjected to area segmentation according to similar pixel points, and the feature vector of the segmented area is input into the multilayer sensor model, so that the accuracy of the identification of the multilayer sensor model can be improved.
Optionally, the determining a second image where the chip is located from the first image includes: determining an initial position from the first image by using the boundary shape of the chip bonding pad on the substrate as a reference through a shape matching algorithm; and dividing an image in the set area into a second image in which the chip is positioned by taking the initial position as a center.
According to the technical scheme, the second image where the chip is located is found in the first image according to the boundary shape of the chip bonding pad on the substrate, the area where the chip is located can be rapidly located, the data volume of the subsequent image needing to be processed is also reduced, and therefore the speed of detecting the welding quality is improved.
Optionally, before performing region segmentation on the second image, the method further includes: performing brightness equalization processing on the second image; and performing gray stretching processing on the second image after the brightness equalization processing.
According to the technical scheme, the brightness equalization processing is carried out on the second image, then the gray stretching processing is carried out on the second image, the darker area in the second image can be darker, the brighter area is brighter, the brightness of the second image can be seen more clearly, the second image can be conveniently divided into the initial areas, and the result of detecting the welding quality can be accurately obtained.
Optionally, performing brightness equalization processing on the second image includes: determining too bright and too dark regions in the second image; the over-bright area is an area with the brightness value of the pixel higher than a first threshold value; the too-dark area is an area where the brightness value of the pixel is lower than a second threshold value; setting the brightness value of each pixel in the over-bright area and the brightness value of each pixel in the over-dark area according to the average brightness of each pixel in the balanced area; the equalization area is an area in the second image except the over-bright area and the over-dark area; and adjusting the brightness of each pixel point in the second image to enable the average brightness of the second image to be a set value.
In the technical scheme, the average brightness of the balanced area is assigned to the over-dark area and the over-bright area, so that the interference of the over-dark area and the over-bright area on the subsequent gray scale stretching treatment effect can be avoided, further, the average brightness of the second image is normalized before the gray scale stretching, the gray scale stretching effect is better, and the result of detecting the welding quality can be accurately obtained.
Optionally, performing region segmentation on the second image to obtain at least one region, including: performing region segmentation on the second image through a region growing algorithm to obtain each initial region; for any initial region, if an excessively dark region exists in the initial region, deleting the excessively dark region to obtain at least one region; the too-dark area is an area where the luminance value of the pixel in the second image is lower than a second threshold.
In the technical scheme, the too-dark area represents the damage of the bonding pad, the second image is divided into the initial areas according to the area growing algorithm, and the too-dark area in each area is deleted, so that the accuracy of determining the empty welding area and the dummy welding area can be improved. Therefore, the result of detecting the welding quality can be accurately obtained.
Optionally, the welding detection result of the region is a null weld or a cold weld, and after the welding detection result of the region is obtained, the method further includes: determining whether the welding detection result of the chip is the cold joint according to the area of each area belonging to the cold joint detection result; or, determining whether the welding detection result of the chip is empty welding according to the area of each region belonging to the empty welding detection result; or determining whether the welding detection result of the chip is a fault according to the area of each region of the false welding detection result, the area of each region of the false welding detection result and the area of the over-dark region in the second picture.
In the above technical solution, the welding detection result of the second image is determined according to the determined empty welding and insufficient welding areas of the regions, and the second image corresponds to the image of the pad, so that the welding result of the chip can be obtained more accurately according to the empty welding and insufficient welding areas of the regions. The method not only detects the empty soldering, the empty soldering and the damage of the chip, but also combines the areas of the empty soldering, the empty soldering and the damage of each area, and the four conditions can more comprehensively judge the soldering condition of the chip.
Optionally, the method further includes: determining whether the welding detection result of the chip is pad damage or not according to the area of the over-dark area in the second picture; the too-dark area is an area where the luminance value of the pixel in the second image is lower than a second threshold.
According to the technical scheme, the chip welding detection result can be determined according to the area of the over-dark area, and the subsequent cold joint, empty joint and damaged areas of each area can be conveniently combined to carry out more accurate detection on the welding condition of the chip.
Optionally, the feature vector of the region includes: energy, autocorrelation, homogeneity, contrast, entropy, anisotropy, mean gray level, standard gray level difference, difference between maximum and minimum gray levels, approximated image plane gray level deviation, fuzzy entropy, fuzzy perimeter.
In the technical scheme, the welding condition of the region can be comprehensively analyzed according to the determined characteristic vector, and the characteristic vector is subsequently input into the multilayer sensor model, so that a chip welding detection result can be obtained.
In a second aspect, an embodiment of the present invention further provides an apparatus for detecting welding quality, including a determining unit, configured to determine, from a first image, a second image where a chip is located; the first image is obtained by shooting from the back of the substrate through a vision system; the front surface of the substrate is provided with a chip mounted by a flip-chip process; the visual system is provided with shooting equipment and a light source which is coaxial with the shooting equipment; the processing unit is used for carrying out region segmentation on the second image to obtain at least one region; wherein, each pixel point in the same region has similarity; determining a characteristic vector of each region, and inputting the characteristic vector of each region into a multilayer perceptron model to obtain a welding detection result of each region; the multilayer perceptron model is obtained through pre-training.
Optionally, the determining unit is specifically configured to determine the initial position from the first image by using a shape matching algorithm based on a boundary shape of the chip pad on the substrate; and dividing an image in the set area into a second image in which the chip is positioned by taking the initial position as a center.
Optionally, the processing unit is specifically configured to perform brightness equalization processing on the second image; and performing gray stretching processing on the second image after the brightness equalization processing.
Optionally, the processing unit is specifically configured to determine an excessively bright region and an excessively dark region in the second image; the over-bright area is an area with the brightness value of the pixel higher than a first threshold value; the too-dark area is an area where the brightness value of the pixel is lower than a second threshold value; setting the brightness value of each pixel in the over-bright area and the brightness value of each pixel in the over-dark area according to the average brightness of each pixel in the balanced area; the equalization area is an area in the second image except the over-bright area and the over-dark area; and adjusting the brightness of each pixel point in the second image to enable the average brightness of the second image to be a set value.
Optionally, the processing unit is specifically configured to perform region segmentation on the second image through a region growing algorithm to obtain each initial region; for any initial region, if an excessively dark region exists in the initial region, deleting the excessively dark region to obtain at least one region; the too-dark area is an area where the luminance value of the pixel in the second image is lower than a second threshold.
Optionally, the processing unit is specifically configured to determine whether the soldering detection result of the chip is a cold joint or not according to the area of each region belonging to the cold joint detection result; determining whether the welding detection result of the chip is empty welding or not according to the area of each region belonging to the empty welding detection result; and determining whether the welding detection result of the chip is a fault according to the area of each region of the cold joint detection result, the area of each region of the cold joint detection result and the area of the over-dark region in each second picture.
Optionally, the processing unit is specifically configured to determine whether the welding detection result of the chip is pad damage according to the area of the excessively dark area in each second picture; the too-dark area is an area where the luminance value of the pixel in the second image is lower than a second threshold.
Optionally, the processing unit is specifically configured to use energy, autocorrelation, homogeneity, contrast, entropy, anisotropy, a mean value of gray scale, a standard difference of gray scale, a difference between a maximum value and a minimum value of gray scale, an approximate image plane gray scale value deviation, a fuzzy entropy, and a fuzzy perimeter.
In a third aspect, the present invention further provides a computing device, including at least one processor and at least one memory, where the memory stores a computer program, and when the program is executed by the processor, the processor is caused to execute a method for detecting welding quality according to any of the first aspect.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which stores a program, and when the program runs on a computer, the program causes the computer to implement a method for detecting welding quality according to any one of the above first aspects.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a schematic diagram of a possible application scenario provided in an embodiment of the present invention;
FIG. 2 is a flowchart of a method for detecting a quality of a weld according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a vision system according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a first image according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a boundary shape of a six-chip pad on a substrate according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating a second image according to an embodiment of the present invention;
fig. 7 is a schematic diagram illustrating a comparison between a second image and a second image after brightness equalization according to an embodiment of the present invention;
FIG. 8 is a schematic diagram illustrating a comparison between a second image and a segmented second image according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of a sample image according to an embodiment of the present invention;
FIG. 10 is a schematic structural diagram of an apparatus for inspecting welding quality according to an embodiment of the present invention;
fig. 11 is a schematic structural diagram of a computing device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic diagram of a possible application scenario provided in the embodiment of the present invention. The application scenario may be a MiniLED panel as an example, and fig. 1 includes a MiniLED panel 100 and a chip 101. Generally, in the process of producing the MiniLED panel 100, there is a step of soldering the chip 101 on the pad, in the soldering process, a poor soldering condition of the chip 101 may occur, and a serious poor soldering condition may be detected when the MiniLED panel 100 is subjected to an appearance inspection and a bright panel inspection before shipment and is classified as a defective product, which needs to be returned to a factory for repair or scrapping, but some empty soldering or cold soldering conditions are not easily detected. The empty soldering or the false soldering is a slight poor soldering, and can be temporarily lightened when the appearance inspection and the bright panel inspection are carried out before the factory leaves, so that the product can be normally delivered as a qualified product, but after the factory leaves, the MiniLED panel 100 with the empty soldering or the false soldering is likely to be desoldered due to vibration in the process of being packaged, transported and unpacked to reach a customer, so that the MiniLED panel 100 becomes a defective product and needs to be returned to a factory for repair or even scrapped. Under a possible condition, not only appearance inspection and bright panel inspection are carried out to MiniLED panel 100 before dispatching from the factory, still carry out infrared imaging's detection, can detect whether there is empty solder or rosin joint's problem MiniLED panel 100 according to infrared imaging, but because infrared imaging's detection method need heat the solder joint, then gather the thermal imaging of solder joint in succession, it is longer to detect the time of infrared imaging once like this, and efficiency is lower, and it has the risk of destroying the chip to heat the solder joint, lead to originally being the MiniLED panel of certified products to become the defective products because of the improper solder joint heating.
As shown in fig. 2, a flowchart of a method for detecting the welding quality is provided for the embodiment of the present invention. The method comprises the following steps:
In the embodiment of the invention, the MiniLED panel is provided with the chip mounted by the flip-chip process, and the chip mounted by the flip-chip process has a structure that the front surface of the chip faces downwards to the substrate, so that a welding point is below the chip in the welding process, and the welding point cannot be directly observed by using a Charge Coupled Device (CCD) camera.
Fig. 3 is a schematic structural diagram of a vision system according to an embodiment of the present invention. The vision system comprises shooting equipment and a light source which is coaxial with the shooting equipment. Specifically, the photographing apparatus may be a CCD camera having an industrial lens, and the light source may be a highly uniform coaxial light source, wherein the CCD camera functions to convert an optical signal into an electrical signal, the industrial lens functions to image a target on a photosensitive surface of the image sensor, and the highly uniform coaxial light source functions to highlight a portion of the chip where the surface is not flat in a pad image, so that interference caused by surface reflection can be overcome. Because the area of the MiniLED panel is large, the vision system cannot completely and clearly shoot the MiniLED panel completely with one image, so the vision system can shoot the MiniLED panel according to a preset sequence, and can also shoot the MiniLED panel according to specific conditions, which is not limited herein. For example, the vision system may start at the top left of the MiniLED panel, assuming each captured image is 900 x 900 in size, then move from left to right, then down by a set step, then left to right, and so on through the entire MiniLED panel. In this case, the vision system may take at least one first image. Fig. 4 is a schematic diagram of a first image according to an embodiment of the present invention.
Because chips do not exist in all positions in the first image, the areas where the chips exist are screened out, so that the data volume of the subsequent image needing to be processed can be reduced, and the efficiency of subsequent image processing is greatly improved. Fig. 5 is a schematic structural diagram of a boundary shape of a six-chip pad on a substrate according to an embodiment of the present invention. Since the chip is soldered to the pad, the location of the chip can be found from the first image by a shape matching algorithm if the shape of the boundary of the chip pad on the substrate is taken as a reference. Specifically, the boundary shape of the matched chip bonding pad on the substrate is obtained through a shape matching algorithm, the midpoint of the boundary between two bonding pads in the boundary shape is used as the center, and then a plurality of second images are cut out from the first image according to the preset width and height, wherein each second image only comprises one chip. Illustratively, if the size of the first image is 900 × 900, the shape matching algorithm starts from the top left of the first image, sets the detection window to be 30 × 30, moves from left to right according to the set step size of the detection window, then moves downwards by one step size, and then moves from left to right, and so on until the first image is traversed, if an area with the similarity of the boundary shape of the chip pad on the substrate reaching the set threshold is detected, the area is cut from the first image according to the preset width and height to obtain a second image. Fig. 6 is a schematic diagram of a second image according to an embodiment of the present invention.
As can be seen from step 201, in order to perform a complete inspection on the MiniLED panel, a plurality of first images need to be captured, and a plurality of chips are usually disposed in the first images. Therefore, the detection is performed for each chip of the MiniLED panel, that is, for each second image, and the following description will be given by taking one second image as an example.
In the embodiment of the present invention, in order to perform region segmentation on the second image more accurately, luminance equalization processing and gray scale stretching processing need to be performed on the second image first, so that darker places in the second image are darker, and brighter places in the second image are brighter.
First, performing brightness equalization processing on the second image, specifically, dividing a region in the second image, where a brightness value of a pixel in the second image is lower than a second threshold, into an overly dark region, and dividing a region in the second image, where a brightness value of a pixel in the second image is higher than a first threshold, into an overly bright region, where the first threshold and the second threshold may be preset, or may be determined according to a specific situation, which is not limited herein. The too-dark area is generally a damaged area of the bonding pad, and the too-bright area may be an area where interference elements such as dust and broken filaments exist on the chip. After identifying the too dark regions, the contour of each of the too dark regions is recorded, and the total area of the too dark regions is counted. One possible implementation manner is that when the ratio of the area of the excessively dark region to the area of the second image is greater than or equal to the first area threshold, the pad breakage is directly determined, and the following detection is not performed any more, so that the welding detection efficiency can be improved. And when the ratio of the area of the over-dark area to the area of the second image is smaller than the first area threshold value, continuing to perform subsequent detection.
Since the subsequent second image needs to make the darker area and the lighter area of the second image darker and brighter according to the gray stretching process, in order to prevent the too-dark area and the too-bright area from interfering with the result of the gray stretching process, the average brightness of the equalized area excluding the too-dark area and the too-bright area in the second image needs to be calculated first, and then the calculated average brightness is assigned to the too-dark area and the too-bright area, that is, the brightness of the too-dark area of the second image is adjusted to be consistent with the average brightness of the equalized area, and the brightness of the too-bright area is adjusted to be consistent with the average brightness of the equalized area, so that the subsequent result of the gray stretching process is not interfered. For example, if the second image is dark as a whole, the excessively dark area in the second image needs to be adjusted to have the same average brightness of the equalized area according to the brightness equalization processing, as shown in fig. 7, which is a schematic diagram illustrating a comparison between the second image and the second image after the brightness equalization processing according to the embodiment of the present invention. As can be seen from the figure, in the same region in the second image, the too-dark region in the brightness-processed second image is brighter than the too-dark region in the original second image, so that the too-dark region in the processed second image does not interfere with the subsequent gray stretching result.
Then, in order to make the effect of gray stretching better, the average value of the brightness of each pixel point in the second image needs to be adjusted to a set value, and the set value may be preset, or may be determined according to specific situations, and generally the average value of the brightness values is taken as a median of 8 bit depths, namely 128 as an example. And then performing gray scale stretching processing on the second image. For example, the gray scale values in the designated range are drawn in advance, and assuming that the designated range is [ first gray scale, second gray scale ], the range to be drawn is [ third gray scale, fourth gray scale ], there are three cases:
the first case is that if the target gray level is less than the first gray level, the reassigned target gray level is calculated according to equation 1.
Re-assigned target gray-scale (target gray-scale) third gray-scale/first gray-scale formula 1
As can be seen from equation 1, when the target gray scale is smaller than the first gray scale, the reassigned target gray scale is smaller than the third gray scale. For example, if the specified range is [100, 150] and the range to be stretched is [50, 200], then if the target gray scale is 80, the reassigned target gray scale will be less than 50 since 80<100, which will make the originally darker areas of the second image darker after gray scale stretching.
The second case is that when the target gradation is equal to or greater than the first gradation and equal to or less than the second gradation, the re-assigned target gradation is calculated according to equation 2.
The reassigned target gray level ═ target gray level-first gray level (fourth gray level-third gray level)/(second gray level-first gray level) + fourth gray level formula 2
According to the formula 2, when the target gray scale is greater than or equal to the first gray scale and less than or equal to the second gray scale, the re-assigned target gray scale is greater than or equal to the third gray scale and less than or equal to the fourth gray scale. For example, if the specified range is [100, 150], the range to be stretched is [50, 200], then if the target gray scale is 120, the reassigned target gray scale is in the [50, 200] range since 100<120<150, which causes the slightly brighter region in the second image to become brighter after gray scale stretching.
The third case is that when the target gray is greater than the second gray, the re-assigned target gray is calculated according to equation 3.
The reassigned target gray level ═ target gray level-second gray level (255-fourth gray level)/(255-second gray level) ] + fourth gray level formula 3
As can be seen from equation 3, when the target gray scale is greater than the second gray scale, the reassigned target gray scale is greater than the fourth gray scale. For example, if the specified range is [100, 150] and the range to be stretched is [50, 200], then if the target gray scale is 180, the reassigned target gray scale will be greater than 200 since 180>150, which will cause the originally brighter region in the second image to become brighter after gray scale stretching.
By the above three formulas, the darker area in the second image after the gray stretching becomes darker, and the lighter area becomes brighter.
And then, using a region growing algorithm to divide the second image after the gray stretching into a region with rapid change and a region with smooth change. The region with rapid change has a high probability of being a region with good welding, and the region with smooth change has a high probability of being a region with poor welding. For example, a group of growing points is determined in the second image, wherein the determination of the growing points may be set in advance, or may be determined according to specific situations, which is not limited herein. And merging the adjacent pixel points or the areas with the brightness difference of the growing points meeting the first threshold value with the growing points to form new growing points, and repeating the operation until the brightness difference of the growing points does not meet the first threshold value, and stopping merging. Through the merging of the growing points, a region composed of a plurality of growing points can be obtained, as shown in fig. 8, which is a schematic diagram comparing the second image and the segmented second image in the embodiment of the present invention. The outline of the area in the box is an area with gentle change, and the rest areas are areas with sharp change. The area outline may be a regular shape or an irregular shape, which is not limited herein. Since the too-dark area in the second image is a damaged area, one way to achieve this is to mark the too-dark area after determining the area of the too-dark area in the second image, so that after the area segmentation of the second image, the too-dark area in the second image can be removed according to the mark of the too-dark area in the second image, and a second image with the too-dark area removed is obtained, wherein the second image is divided into an area with rapid change and an area with smooth change.
In the embodiment of the present invention, a 12-dimensional feature vector of each region in the second image determined in step 202 is calculated, where the 12-dimensional feature vector includes energy, autocorrelation, homogeneity, contrast, entropy, anisotropy, mean gray level, standard gray level difference, difference between maximum gray level and minimum gray level, deviation of gray level of an approximate image plane, fuzzy entropy, and fuzzy perimeter. Specifically, the energy is the sum of the squares of all pixel grays. Autocorrelation is a measure of how a pixel is correlated with neighboring pixels in the entire image. Homogeneity is a measure of how closely the elements are distributed to the diagonals in the co-occurrence matrix of the metric image. Contrast is the brightness contrast between a pixel and an adjacent pixel in an image. The entropy reflects how much the average amount of information in the image is. Anisotropy is a characteristic of differences in gray level presentation of image pixels in various directions. The gray average is the average of the gray values of all pixels of the image. The gray scale standard deviation is the standard deviation of the gray scale values of all pixels of the image. The difference between the maximum and minimum gray values is the difference between the gray value of the brightest point and the gray value of the darkest point in all the pixels of the image. The approximated image plane gray value deviation is the deviation between the gray value of the computed image and the gray value approximation. The blur entropy is a measure of how close an image approaches a pure dark black/pure white image. The blur perimeter is a perimeter calculated by regarding an image as a set of blur regions.
According to the 12-dimensional feature vector, the image information of each region can be comprehensively analyzed, and the welding condition of the region can be conveniently determined according to the 12-dimensional feature vector.
And step 204, inputting the feature vectors of the regions into a multilayer perceptron model to obtain welding detection results of the regions.
In the embodiment of the invention, the multilayer sensor model takes an MLP model as an example, 12-dimensional feature vectors of the region are input into the MLP model, and the welding detection result of the region is obtained. Specifically, the MLP model needs to be trained first, then the 12-dimensional feature vectors of the region are input into the MLP model, and then the MLP model outputs the welding detection result of the region.
The specific step of training the MLP model is to collect sample images in advance, such as several images of good products, several images of defects including solder joints, and several images of defects including solder joints. Fig. 9 is a schematic diagram of a sample image according to an embodiment of the present invention. And inputting the images into an MLP model so as to train the MLP model, so that the MLP model can well distinguish empty solder, cold solder and solder. Since the images are marked before the images are input, that is, the MLP model knows whether the input images are qualified or faulty or empty, the MLP model can be used for training different images specifically, so as to find the difference between the faulty or empty welding and the good welding.
The 12-dimensional feature vectors of the region are input into the MLP model, and then the MLP model outputs the welding detection result of the region. Specifically, the 12-dimensional feature vectors of the region are input into the MLP model, and the null solder reliability and the cold solder reliability are obtained.
One way to achieve this is to set a priority between determining that the solder joint, and the solder joint are good, first determine whether the reliability of the solder joint is greater than a solder joint reliability threshold, if so, determine that the area is a solder joint, if not, continue to determine whether the reliability of the solder joint is greater than the solder joint reliability threshold, if so, determine that the area is a solder joint, and if not, determine that the area is a solder joint.
After the welding condition of each region is determined, the welding condition of the second image can be determined according to the welding condition of each region. Specifically, the total area of the void solder, and the total area of the pad damage in each region are counted, wherein the total area of the pad damage is determined in step 202, which is not described herein again. And judging that the welding condition of the second image has a priority relation. The priority determination comprises the following steps:
first, according to the above step 202, the area ratio threshold of the pad damage to the second image is set as the first threshold, the area ratio threshold of the dummy solder to the second image is set as the second threshold, and the area ratio threshold of the dummy solder to the second image is set as the third threshold. The weight values of the dummy solder, the dummy solder and the pad damage are set, which may be preset or determined according to actual situations, and are not limited herein. And setting the unknown fault threshold value as a fourth threshold value.
Secondly, judging whether the total area of the damaged bonding pad is larger than or equal to a first threshold value, if so, judging that the bonding pad is damaged, and if not, judging that the bonding pad is damaged;
judging whether the total area of the cold joint is larger than or equal to a second threshold value, if so, judging that the second image is the cold joint, and if not, judging that the second image is the cold joint;
judging whether the total area of the empty welding is larger than or equal to a third threshold value, if so, judging that the second image is empty welding, and if not, judging that the second image is empty welding;
and judging whether the total area of the insufficient solder joints, the second weight and the pad breakage and the third weight are larger than or equal to a fourth threshold value, and if so, judging that the second image is an unknown fault. If not, the second image is judged to be good in welding.
Since each second image corresponds to one chip, the bonding condition of the second image is the bonding condition of each chip, and then, operations such as repairing can be performed specifically according to the bonding condition of the corresponding chip.
As can be seen from the above steps 201 to 204, the second image where each chip is located can be determined from the first image through the shape matching algorithm, so that the area to be processed is subsequently reduced, the detection speed is increased, and the accuracy of subsequent determination of area solder joint or cold joint can be improved by using the gray stretching process and removing the damaged area after segmentation of the second image.
Based on the same technical concept, the embodiment of the invention also provides a device for detecting the welding quality, and the device can execute the method in the method embodiment. Referring to fig. 10, a structure of an apparatus for detecting solder quality according to an embodiment of the present invention may include a determining unit 1001 configured to determine a second image where each chip is located from a first image; the first image is obtained by shooting from the back of the substrate through a vision system; the front surface of the substrate is provided with a chip mounted by a flip-chip process; the visual system is provided with shooting equipment and a light source which is coaxial with the shooting equipment. The processing unit 1002 is configured to perform region segmentation on the second image to obtain at least one region; wherein, each pixel point in the same region has similarity; determining a characteristic vector of each region, and inputting the characteristic vector of each region into a multilayer perceptron model to obtain a welding detection result of each region; the multilayer perceptron model is obtained through pre-training.
Optionally, the determining unit 1001 is specifically configured to determine the initial position from the first image by using a shape matching algorithm with reference to a boundary shape of the chip pad on the substrate; and dividing an image in the set area into a second image in which the chip is positioned by taking the initial position as a center.
Optionally, the processing unit 1002 is specifically configured to perform brightness equalization processing on the second image; and performing gray stretching processing on the second image after the brightness equalization processing.
Optionally, the processing unit 1002 is specifically configured to determine an excessively bright region and an excessively dark region in the second image; the over-bright area is an area with the brightness value of the pixel higher than a first threshold value; the too-dark area is an area where the brightness value of the pixel is lower than a second threshold value; setting the brightness value of each pixel in the over-bright area and the brightness value of each pixel in the over-dark area according to the average brightness of each pixel in the balanced area; the equalization area is an area in the second image except the over-bright area and the over-dark area; and adjusting the brightness of each pixel point in the second image to enable the average brightness of the second image to be a set value.
Optionally, the processing unit 1002 is specifically configured to perform region segmentation on the second image through a region growing algorithm to obtain each initial region; for any initial region, if an excessively dark region exists in the initial region, deleting the excessively dark region to obtain at least one region; the too-dark area is an area where the luminance value of the pixel in the second image is lower than a second threshold.
Optionally, the processing unit 1002 is specifically configured to determine whether the welding detection result of the chip is a cold joint or not according to the area of each region belonging to the cold joint detection result; determining whether the welding detection result of the chip is empty welding or not according to the area of each region belonging to the empty welding detection result; and determining whether the welding detection result of the chip is a fault according to the area of each region of the cold joint detection result, the area of each region of the cold joint detection result and the area of the over-dark region in each second picture.
Optionally, the processing unit 1002 is specifically configured to determine whether the welding detection result of the chip is pad damage according to the area of the excessively dark area in each second picture; the too-dark area is an area where the luminance value of the pixel in the second image is lower than a second threshold.
Optionally, the processing unit 1002 is specifically configured to use energy, autocorrelation, homogeneity, contrast, entropy, anisotropy, a gray mean, a gray standard deviation, a difference between a maximum gray value and a minimum gray value, an approximate image plane gray value deviation, a fuzzy entropy, and a fuzzy perimeter.
Based on the same technical concept, the embodiment of the present application further provides a computing device, as shown in fig. 11, including at least one processor 1101 and a memory 1102 connected to the at least one processor, where a specific connection medium between the processor 1101 and the memory 1102 is not limited in the embodiment of the present application, and the processor 1101 and the memory 1102 are connected through a bus in fig. 11 as an example. The bus may be divided into an address bus, a data bus, a control bus, etc.
In the embodiment of the present application, the memory 1102 stores instructions executable by the at least one processor 1101, and the at least one processor 1101 may execute the steps included in the method for detecting welding quality by executing the instructions stored in the memory 1102.
The processor 1101 is a control center of the computing device, and may connect various parts of the computing device by using various interfaces and lines, and implement data processing by executing or executing instructions stored in the memory 1102 and calling data stored in the memory 1102. Optionally, the processor 1101 may include one or more processing units, and the processor 1101 may integrate an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface, an application program, and the like, and the modem processor mainly processes an issued instruction. It will be appreciated that the modem processor described above may not be integrated into the processor 1101. In some embodiments, the processor 1101 and the memory 1102 may be implemented on the same chip, or in some embodiments, they may be implemented separately on separate chips.
The processor 1101 may be a general purpose processor such as a Central Processing Unit (CPU), a digital signal processor, an Application Specific Integrated Circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof, configured to implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present Application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the disclosed method in connection with the method embodiments for detecting weld quality may be embodied directly in a hardware processor, or in a combination of hardware and software modules within the processor.
Based on the same technical concept, embodiments of the present application also provide a computer-readable storage medium storing a computer program executable by a computing device, wherein when the program is run on the computing device, the computer program causes the computing device to execute the steps of the method for detecting welding quality.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (11)
1. A method of detecting weld quality, the method comprising:
determining a second image where the chip is located from the first image; the first image is obtained by shooting from the back of the substrate through a vision system; the front surface of the substrate is provided with a chip mounted by a flip-chip process; the visual system is provided with shooting equipment and a light source which is coaxial with the shooting equipment;
performing region segmentation on the second image to obtain at least one region; wherein, each pixel point in the same region has similarity;
determining a characteristic vector of each region, and inputting the characteristic vector of each region into a multilayer perceptron model to obtain a welding detection result of each region; the multilayer perceptron model is obtained through pre-training.
2. The method of claim 1, wherein determining the second image in which the chip is located from the first image comprises:
determining an initial position from the first image by a shape matching algorithm by taking the boundary shape of the chip bonding pad on the substrate as a reference; and dividing an image in a set area into the second image of the chip by taking the initial position as a center.
3. The method of claim 1, wherein prior to performing region segmentation on the second image, further comprising:
performing brightness equalization processing on the second image;
and performing gray stretching processing on the second image after the brightness equalization processing.
4. The method of claim 3, wherein performing a brightness equalization process on the second image comprises:
determining too bright and too dark regions in the second image; the over-bright area is an area with the brightness value of the pixel higher than a first threshold value; the too-dark area is an area where the brightness value of the pixel is lower than a second threshold value;
setting the brightness value of each pixel in the over-bright area and the brightness value of each pixel in the over-dark area according to the average brightness of each pixel in the balanced area; the equalization area is an area in the second image except the over-bright area and the over-dark area;
and adjusting the brightness of each pixel point in the second image to enable the average brightness of the second image to be a set value.
5. The method of claim 1, wherein performing region segmentation on the second image to obtain at least one region comprises:
performing region segmentation on the second image through a region growing algorithm to obtain each initial region;
for any initial region, if an excessively dark region exists in the initial region, deleting the excessively dark region to obtain at least one region; the too-dark area is an area where the luminance value of the pixel in the second image is lower than a second threshold.
6. The method of any one of claims 1 to 5, wherein the welding detection result of the region is a null weld or a cold weld, and after obtaining the welding detection result of the region, the method further comprises:
determining whether the welding detection result of the chip is the cold joint according to the area of each area belonging to the cold joint detection result; or the like, or, alternatively,
determining whether the welding detection result of the chip is empty welding according to the area of each region belonging to the empty welding detection result; or the like, or, alternatively,
and determining whether the welding detection result of the chip fails according to the area of each region of the cold joint detection result, the area of each region of the cold joint detection result and the area of the over-dark region in the second image.
7. The method of any of claims 1 to 5, further comprising:
determining whether the welding detection result of the chip is pad damage or not according to the area of the over-dark area in the second picture; the too-dark area is an area where the luminance value of the pixel in the second image is lower than a second threshold.
8. The method of any of claims 1 to 5, wherein the feature vectors of the regions comprise:
energy, autocorrelation, homogeneity, contrast, entropy, anisotropy, mean gray level, standard gray level difference, difference between maximum and minimum gray levels, approximated image plane gray level deviation, fuzzy entropy, fuzzy perimeter.
9. An apparatus for testing weld quality, comprising:
the determining unit is used for determining a second image where the chip is located from the first image; the first image is obtained by shooting from the back of the substrate through a vision system; the front surface of the substrate is provided with a chip mounted by a flip-chip process; the visual system is provided with shooting equipment and a light source which is coaxial with the shooting equipment;
the processing unit is used for carrying out region segmentation on the second image to obtain at least one region; wherein, each pixel point in the same region has similarity; determining a characteristic vector of each region, and inputting the characteristic vector of each region into a multilayer perceptron model to obtain a welding detection result of each region; the multilayer perceptron model is obtained through pre-training.
10. A computing device comprising at least one processor and at least one memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the method of any of claims 1 to 8.
11. A computer-readable storage medium, characterized in that the storage medium stores a program which, when run on a computer, causes the computer to carry out the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111532686.5A CN114240877A (en) | 2021-12-15 | 2021-12-15 | Method and device for detecting welding quality |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111532686.5A CN114240877A (en) | 2021-12-15 | 2021-12-15 | Method and device for detecting welding quality |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114240877A true CN114240877A (en) | 2022-03-25 |
Family
ID=80756142
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111532686.5A Pending CN114240877A (en) | 2021-12-15 | 2021-12-15 | Method and device for detecting welding quality |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114240877A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116309568A (en) * | 2023-05-18 | 2023-06-23 | 深圳恒邦新创科技有限公司 | Chip soldering leg welding quality detection method and system |
| CN116385476A (en) * | 2023-06-05 | 2023-07-04 | 青岛星跃铁塔有限公司 | Iron tower quality analysis method based on visual detection |
-
2021
- 2021-12-15 CN CN202111532686.5A patent/CN114240877A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116309568A (en) * | 2023-05-18 | 2023-06-23 | 深圳恒邦新创科技有限公司 | Chip soldering leg welding quality detection method and system |
| CN116385476A (en) * | 2023-06-05 | 2023-07-04 | 青岛星跃铁塔有限公司 | Iron tower quality analysis method based on visual detection |
| CN116385476B (en) * | 2023-06-05 | 2023-08-18 | 青岛星跃铁塔有限公司 | Iron tower quality analysis method based on visual detection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111325713B (en) | Neural network-based wood defect detection method, system and storage medium | |
| CN110659660B (en) | Automatic optical detection classification equipment using deep learning system and training equipment thereof | |
| US11017259B2 (en) | Defect inspection method, defect inspection device and defect inspection system | |
| JP7325952B2 (en) | Display device defect detection system and method | |
| KR102171491B1 (en) | Method for sorting products using deep learning | |
| CA2150524C (en) | Method and system for inspecting packages | |
| CN114240877A (en) | Method and device for detecting welding quality | |
| CN106501272B (en) | Machine vision soldering tin positioning detection system | |
| CN116559183B (en) | Method and system for improving defect judging efficiency | |
| US20240062363A1 (en) | Grading cosmetic appearance of a test object | |
| CN110517265A (en) | A kind of detection method of surface defects of products, device and storage medium | |
| CN112884743B (en) | Detection method and device, detection equipment and storage medium | |
| CN113176270B (en) | Dimming method, device and equipment | |
| CN113240673B (en) | Defect detection method, defect detection device, electronic equipment and storage medium | |
| CN116503414B (en) | Screen defect detection method, device, computer equipment and storage medium | |
| CN116843650A (en) | SMT welding defect detection method and system integrating AOI detection and deep learning | |
| CN116543247A (en) | Data set manufacturing method and verification system based on photometric stereo surface reconstruction | |
| CN114612418A (en) | Method, device and system for detecting surface defects of mouse shell and electronic equipment | |
| CN107248151A (en) | A kind of LCD panel intelligent detecting method and system based on machine vision | |
| CN110310235B (en) | Fundus image processing method, device and equipment and storage medium | |
| CN113267506A (en) | Wood board AI visual defect detection device, method, equipment and medium | |
| JP2022018507A (en) | Image processing apparatus for subject determination, and image processing method | |
| CN112989964B (en) | Detection device, detection system and detection method for performance of eyeball tracking module | |
| TWI822454B (en) | Defect-detection system and method for training with blended defects | |
| JP5778685B2 (en) | System and method for alignment and inspection of ball grid array devices |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 230013 Southwest District of Hefei City, Anhui Province Applicant after: Hefei xinyihua Intelligent Machine Co.,Ltd. Address before: 230013 Southwest District of Hefei City, Anhui Province Applicant before: HEFEI SINEVA INTELLIGENT MACHINE Co.,Ltd. |