CN115880288B - Detection method, system and computer equipment for electronic element welding - Google Patents

Detection method, system and computer equipment for electronic element welding Download PDF

Info

Publication number
CN115880288B
CN115880288B CN202310143229.XA CN202310143229A CN115880288B CN 115880288 B CN115880288 B CN 115880288B CN 202310143229 A CN202310143229 A CN 202310143229A CN 115880288 B CN115880288 B CN 115880288B
Authority
CN
China
Prior art keywords
welding
image
detected
detection
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310143229.XA
Other languages
Chinese (zh)
Other versions
CN115880288A (en
Inventor
姜锐
沈自全
刘青伟
张�杰
程云
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Zhaoxing Botuo Technology Co ltd
Original Assignee
Shenzhen Zhaoxing Botuo Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Zhaoxing Botuo Technology Co ltd filed Critical Shenzhen Zhaoxing Botuo Technology Co ltd
Priority to CN202310143229.XA priority Critical patent/CN115880288B/en
Publication of CN115880288A publication Critical patent/CN115880288A/en
Application granted granted Critical
Publication of CN115880288B publication Critical patent/CN115880288B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Image Analysis (AREA)

Abstract

The invention relates to the technical field of automatic detection, solves the technical problems of high detection cost and low efficiency, in particular to a detection method, a detection system and computer equipment for electronic element welding, which comprise the following steps: acquiring a standard welding image and a welding image to be detected, which are acquired by image acquisition equipment; calculating a matching coefficient of the standard welding image and the welding image to be detected and the area of the difference area; performing preliminary detection on the electronic component welding products to be detected according to a preset matching coefficient threshold value and a preset difference area threshold value; and carrying out layer-by-layer searching and comparing on a plurality of characteristic images with different resolutions of the welding image to be detected according to invariant moment characteristics of the image, and obtaining the welding area to be detected which is accurately matched with the standard welding image. According to the invention, a two-stage detection mode is adopted, products with obvious welding defects are screened, and then the improved convolutional neural network is utilized to carry out secondary detection on products without obvious welding defects, so that the detection efficiency is improved, and the cost is low.

Description

Detection method, system and computer equipment for electronic element welding
Technical Field
The present invention relates to the field of automatic detection technologies, and in particular, to a method and a system for detecting electronic component welding, and a computer device.
Background
When electronic components are welded, whether manual production or mechanized production is performed, welding defects such as lack of welding, less tin, askew adhesion, bridging and the like can occur with a certain probability, and the defects can seriously affect the reliability of the electronic products, so that the detection of the welding quality of the electronic components is particularly important.
At present, the basic idea of automatically detecting the welding quality of an electronic component by adopting a visual detection technology is that an image acquisition device is adopted to acquire an image of the welding component, denoising and segmentation processing is carried out on the image of the welding component to obtain an image to be detected only containing a welding area, and then the image to be detected is input into a detection model for characteristic analysis so as to judge whether the image has defects and determine the types of the defects.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a detection method, a detection system and computer equipment for electronic element welding, solves the technical problems of high acquisition cost of high-resolution welding images and lower detection efficiency due to low transmission speed, and achieves the aim of improving the detection efficiency and accuracy of the welding quality of electronic elements on the premise of not increasing the detection cost.
In order to solve the technical problems, the invention provides the following technical scheme: a detection method for welding electronic components comprises the following steps:
s1, acquiring a standard welding image acquired by image acquisition equipment and a to-be-detected welding image of an electronic element welding piece to be detected, wherein the standard welding image refers to an image which is qualified in welding quality and only contains an electronic element and a welding area thereof;
s2, calculating a matching coefficient r of the standard welding image and the welding image to be detected and a difference area Sc;
s3, performing preliminary detection on the electronic element welding product to be detected according to a preset matching coefficient threshold Tr and a difference area threshold Ts;
if |r| < Tr and |Sc| < Ts, judging that the product is not qualified and ending the detection, otherwise, executing the step S4;
s4, carrying out layer-by-layer searching and comparing on a plurality of characteristic images with different resolutions of the welding image to be detected according to invariant moment characteristics of the image to obtain a welding area to be detected which is accurately matched with the standard welding image;
s5, carrying out image segmentation on the welding image to be detected according to the welding area to be detected to obtain a target image to be detected;
S6, preprocessing the target image to be detected to obtain a preprocessed image with distinct welding spot contours and electronic element supporting bodies;
s7, extracting a welding spot morphology line to be detected from the preprocessed image according to a segmentation threshold value, and calculating a morphology difference value X between the welding spot morphology line to be detected and a preset standard welding spot morphology line;
s8, performing secondary detection according to the comparison result of the form difference value X and a preset detection threshold Tx, and judging whether welding defects exist or not;
and if the form difference value X exceeds the range of the preset detection threshold Tx, judging that the product is a disqualified product, otherwise, judging that the product is a qualified product.
Further, the step S2 specifically includes the following steps:
s21, moving the standard welding image in the welding image to be detected, and calculating a matching coefficient r between the standard welding image and the welding image;
s22, regarding the position where the maximum value of the matching coefficient appears as the rough registration of the standard welding image and the welding image to be detected;
s23, performing binarization processing on the standard welding image and the welding image to be detected after rough registration, and performing differential operation to obtain a difference area Sc.
Further, the step S4 specifically includes the following steps:
S41, inputting the welding image to be detected into an improved convolutional neural network, and performing convolutional downsampling operation by adopting convolutional kernels with different sizes to obtain a plurality of characteristic images with different resolutions;
s42, forming a plurality of feature images with different resolutions into a pyramid shape, wherein the higher the hierarchy is, the lower the resolution of the feature images is;
s43, carrying out exhaustive comparison on the characteristic image with low resolution of the top layer according to the welding spot area of the standard welding image to obtain a coarse positioning area;
and S44, searching the characteristic images layer by layer in the sequence from top to bottom, and searching only the position near the coarse positioning area of the upper layer until reaching the bottommost layer to obtain a welding area to be detected which is accurately matched with the standard welding image.
Further, the step S6 specifically includes the following steps:
s61, carrying out graying treatment on the target image to be detected to obtain a gray image;
s62, denoising the gray level image by adopting non-local mean filtering to obtain a denoised image;
and S63, sharpening the edge part of the denoising image by using the Laplacian operator to obtain a preprocessed image with distinct welding spot contours and support layers.
Further, in step S8, a formula for calculating the morphology difference value X between the to-be-detected welding spot morphology line and the preset standard welding spot morphology line is as follows:
wherein,representing the corresponding +.f. on the to-be-detected welding spot morphology line and the standard welding spot morphology line>Height difference of high points, +.>And the number of corresponding high points on the welding spot morphology line to be detected and the standard welding spot morphology line is represented.
Further, the step S5 is configured to:
and carrying out fusion reconstruction on a plurality of feature images with different resolutions by adopting a multi-scale feature fusion strategy to obtain a high-resolution image, and dividing the high-resolution image according to a welding area to be detected to obtain a target image to be detected, wherein the target image only comprises the welding area.
Further, after the step S8, the method further includes:
and S9, outputting the detection result information and generating an instruction for recalling the unqualified products of the electronic component welding.
The invention also provides a technical scheme that: a system for implementing the method for detecting electronic component soldering described above, comprising:
the image acquisition module is used for acquiring a standard welding image acquired by the image acquisition equipment and a welding image to be detected of the electronic element welding piece to be detected, wherein the standard welding image refers to an image which is qualified in welding quality and only contains the electronic element and a welding area thereof;
The first calculation module is used for calculating a matching coefficient r of the standard welding image and the welding image to be detected and a difference area Sc;
the primary detection module is used for carrying out primary detection on the electronic element welding product to be detected according to a preset matching coefficient threshold Tr and a difference area threshold Ts;
the welding area positioning module to be detected is used for carrying out layer-by-layer searching and comparing on a plurality of characteristic images with different resolutions of the welding image to be detected according to invariant moment characteristics of the image to obtain a welding area to be detected which is accurately matched with the standard welding image;
the to-be-detected target image determining module is used for carrying out image segmentation on the to-be-detected welding image according to the to-be-detected welding area to obtain a to-be-detected target image;
the image preprocessing module is used for preprocessing the target image to be detected to obtain a preprocessed image with distinct welding spot outline and electronic element support layers;
the second calculation module is used for extracting a welding spot morphology line to be detected from the preprocessed image according to a segmentation threshold value and calculating a morphology difference value X between the welding spot morphology line to be detected and a preset standard welding spot morphology line;
The secondary detection module is used for carrying out secondary detection according to the comparison result of the morphological difference value X and a preset detection threshold Tx, and judging whether welding defects exist or not.
Further, the method further comprises the following steps:
the detection result output module is used for outputting detection result information and generating an instruction for recalling unqualified products of electronic element welding;
the high-resolution image fusion module is used for carrying out fusion reconstruction on a plurality of feature images with different resolutions by adopting a multi-scale feature fusion strategy so as to obtain a high-resolution image.
The invention also provides computer equipment, which comprises a processor and a memory, wherein the memory is used for storing a computer program, and the computer program is executed by the processor to realize the method for detecting the welding of the electronic component.
By means of the technical scheme, the invention provides a detection method, a detection system and computer equipment for electronic element welding, which at least have the following beneficial effects:
1. according to the invention, the obtained welding image to be detected and the standard welding image are subjected to rough registration and differential operation according to the welding defect type of the electronic element, whether the product to be detected has a lack of welding defect or not is judged by combining a preset threshold value, a product with obvious welding defect is screened out, and an improved convolutional neural network is adopted to carry out secondary detection on the product without obvious welding defect, so that the detection efficiency of the welding product of the electronic element is improved.
2. According to the application, the improved convolutional neural network is adopted to extract a plurality of characteristic images with different resolutions of the to-be-detected welding image without obvious defects, a layer-by-layer searching and exhaustive comparison strategy is used according to invariant moment characteristics of the images, and the to-be-detected welding image matched with the standard welding image is accurately positioned to be subjected to image segmentation according to the to-be-detected welding area, so that the to-be-detected target image only comprising the welding area is obtained, the calculation amount of subsequent image processing can be reduced, the welding quality detection efficiency is further improved, and the system maintenance cost is also reduced.
3. According to the application, the multi-scale feature fusion strategy is adopted to accurately position a plurality of feature images with different resolutions of the welding area to be detected, which are matched with the standard welding image, to perform feature fusion reconstruction, and the fusion reconstructed high-resolution image is segmented according to the welding area to be detected, so that the target image to be detected, which only comprises the welding area, is obtained, and the aims of improving the welding quality detection efficiency and accuracy of the electronic element under the premise of not increasing the detection cost are achieved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 is a flow chart of a method for detecting soldering of an electronic component according to a first embodiment of the invention;
FIG. 2 is a diagram showing a comparison between a standard welding image and a welding image to be detected according to a first embodiment of the present invention;
FIG. 3 is a flowchart of searching a welding area to be detected according to a first embodiment of the present invention;
FIG. 4 is a flowchart of preprocessing an image of a target to be detected according to a first embodiment of the present invention;
FIG. 5 is a schematic block diagram of a detection system for electronic component soldering in accordance with a first embodiment of the present invention;
FIG. 6 is a flowchart of a method for detecting soldering of an electronic component according to a second embodiment of the present invention;
FIG. 7 is a block diagram of a convolutional neural network modified in accordance with a second embodiment of the present invention;
FIG. 8 is a schematic diagram of a multi-scale image search and fusion structure in a second embodiment of the present invention;
FIG. 9 is a schematic block diagram of a system for detecting electronic component soldering in accordance with a second embodiment of the present invention;
fig. 10 is a block diagram showing an internal structure of a computer device according to an embodiment of the present invention.
In the figure: 101. an image acquisition module; 102. a first computing module; 103. a preliminary detection module; 104. a welding area positioning module to be detected; 105. the target image determining module is used for determining a target image to be detected; 106. an image preprocessing module; 107. a second computing module; 108. a secondary detection module; 109. the detection result output module; 110. and a high-resolution image fusion module.
Detailed Description
In order that the above-recited objects, features and advantages of the present application will become more readily apparent, a more particular description of the application will be rendered by reference to the appended drawings and appended detailed description. Therefore, the realization process of how to apply the technical means to solve the technical problems and achieve the technical effects can be fully understood and implemented.
Example 1
Referring to fig. 1-4, a specific implementation manner of the present embodiment is shown, in the present embodiment, by performing coarse registration and differential operation on an obtained to-be-detected welding image and a standard welding image according to a welding defect type of an electronic component, and determining whether a to-be-detected welding product has a lack of welding defect by combining a preset threshold, screening a product with an obvious welding defect, and performing secondary detection on a product without the obvious welding defect by using an improved convolutional neural network, thereby improving the detection efficiency of the welding product of the electronic component.
As shown in fig. 1, a method for detecting soldering of an electronic component includes the steps of:
s1, acquiring a standard welding image acquired by image acquisition equipment and a welding image to be detected of an electronic element welding piece to be detected.
The method comprises the steps that a standard welding image and a to-be-detected welding image of an electronic element welding piece to be detected are obtained by using an industrial area array camera of a kilomega Ethernet (MV-CA 060-10 GC), wherein the standard welding image refers to an image which is qualified in welding quality and only comprises the electronic element and a welding area thereof by a professional; in addition, the highest resolution of the camera is 3072×2048, the maximum frame rate is 17FPS, the exposure time is 27 μs to 2.5s, and two linear LED light sources are used to ensure the quality of the acquired image.
S2, calculating a matching coefficient r of the standard welding image and the welding image to be detected and a difference area Sc. The method specifically comprises the following steps:
s21, moving the standard welding image in the welding image to be detected, and calculating a matching coefficient r between the standard welding image and the welding image, wherein the calculation formula of the matching coefficient r is as follows:
in the above-mentioned method, the step of,representing pixels on a standard welding image, < >>Represents the gray-scale average of a standard welding image,representing pixels on the welding image to be detected, < >>And representing the gray average value of the welding image to be detected.
S22, taking the position where the maximum value of the matching coefficient appears as a standard welding image and a welding image to be detected to finish rough registration.
The absolute value of the matching coefficient r is in the range of 0,1, and the larger the value is, the higher the registration accuracy of the standard welding image and the welding image to be detected is, and the lower the registration accuracy is. For this reason, in this embodiment, the position where the maximum value of the matching coefficient occurs is regarded as the position where the rough registration of the standard welding image and the welding image to be detected is completed.
S23, performing binarization processing on the standard welding image after rough registration and the welding image to be detected, and performing differential operation to obtain a difference region area Sc.
The following will further describe the comparison schematic diagram of the standard welding image and the welding image to be detected shown in fig. 2, wherein in fig. 2, the picture a represents the standard welding image and the picture B represents the welding image to be detected;
firstly, moving a picture A in a picture B, calculating a matching coefficient between the picture A and the picture B according to a calculation formula of the matching coefficient r, and when the matching coefficient r has a maximum value, taking the position of the maximum value as the position of the standard welding image and the welding image to be detected for finishing rough registration; and then, performing binarization processing on the pictures A and B after coarse registration, and performing differential operation on the corresponding positions of the picture B and the picture A to obtain a difference region area Sc between the two.
S3, performing preliminary detection on the electronic element welding product to be detected according to a preset matching coefficient threshold Tr and a difference area threshold Ts;
in this embodiment, the matching coefficient threshold Tr and the difference area threshold Ts are set to 0.45 and 0.08 respectively according to the characteristics of the electronic components to be welded, if |r| < 0.45 and |sc| < 0.08, it is determined that the electronic components are not in the welding position, and the detection is finished, otherwise, step S4 is executed to perform secondary detection on the product to be welded, and finally it is determined whether the welding defect exists.
It is noted that the matching coefficient threshold value Tr and the difference area threshold value Ts are closely related to the category of the electronic component to be soldered, and for this purpose, the matching coefficient threshold value Tr and the difference area threshold value Ts may be set to other values.
And S4, carrying out layer-by-layer searching and comparing on a plurality of characteristic images with different resolutions of the welding image to be detected according to invariant moment characteristics of the image, and obtaining a welding area to be detected which is accurately matched with the standard welding image.
The following will further describe the process of searching for a welding area to be detected with reference to the flowchart shown in fig. 3, which specifically includes the following steps:
s41, inputting a welding image to be detected into an improved convolutional neural network, and performing convolutional downsampling operation by adopting convolution kernels with different sizes to obtain a plurality of characteristic images with different resolutions;
in this embodiment, the improved convolutional neural network includes an input layer, ten convolutional layers and a downsampling layer, a full connection layer and an output layer, when extracting a feature image, a to-be-detected welding image enters the convolutional layers through the input layer, four feature images are generated through convolutional downsampling operations performed through the first to third convolutional layers, and then, high-frequency, medium-frequency and low-frequency feature information in the image are respectively extracted from each feature image through three paths of different frequency bands, so that four feature images with different resolutions are obtained; the high-frequency band path part uses three convolution layers, the medium-frequency band path part uses two convolution layers, the convolution kernels of the convolution layers used in the high-frequency band and the medium-frequency band are 3 multiplied by 3, and the low-frequency band path part uses one convolution layer with the convolution kernel of 5 multiplied by 5.
The process of extracting a plurality of different resolution feature images from a low resolution welding image to be detected can be expressed as the following expression:
in the above-mentioned method, the step of,representing waitingDetecting welding images +.>Respectively representing the weight and bias of the filter, the sign +.>Representing convolution operations +.>The activation function ReLU used on the output feature image is represented.
S42, forming a plurality of feature images with different resolutions into a pyramid shape, wherein the higher the level is, the lower the resolution of the feature images is;
s43, performing exhaustive comparison on the top-layer low-resolution characteristic image according to the welding spot area of the standard welding image to obtain a coarse positioning area;
and S44, searching the characteristic images layer by layer in the sequence from top to bottom, and searching only the position near the coarse positioning area of the upper layer until the position of the bottom layer is reached to obtain a welding area to be detected which is accurately matched with the standard welding image.
Exemplary as a further explanation, assume that the welding area of the standard welding image in the present embodimentAnd->Layer search area->Is +.>And->) In the process of searching and positioning, selecting a welding areaAnd->Layer search area->The region with the greatest degree of matching is taken as the matching region of the next layer of characteristic image, namely +. >Search area of layer image->The selection range is as follows:
in the above-mentioned method, the step of,expressed as +.>Is a rectangular neighborhood of the center coordinates,respectively indicate->At->Length of direction, ++>Respectively represent rectangular neighborhood in->Coefficient of variation of the directional length.
By repeating the process, the characteristic images with different resolutions extracted from the welding images to be detected are searched layer by layer, the searching subgraph generated by each candidate position is compared with the standard welding image in an exhaustive mode, the searching subgraph generated by the top-layer characteristic image is small in size, the searching subgraph generated by the top-layer characteristic image is small, and the welding area to be detected of the welding images to be detected can be positioned conveniently, quickly and accurately, and therefore the welding quality detection efficiency of electronic elements is improved.
And S5, performing image segmentation on the welding image to be detected according to the welding area to be detected to obtain a target image to be detected.
The to-be-detected welding image is subjected to image segmentation according to the accurately positioned to-be-detected welding area matched with the standard welding image, so that the to-be-detected target image only comprising the welding area is obtained, the calculated amount of subsequent image processing can be reduced, and the welding quality detection efficiency is improved.
S6, preprocessing the target image to be detected to obtain a preprocessed image with the welding spot outline and the electronic element support body being distinct in hierarchy.
Because the contrast ratio of the electronic element and the supporting body is not strong, the characteristic information of the welding spots is difficult to extract, and therefore, a series of pretreatment is needed to be adopted for eliminating external noise and interference on the welding part image before the welding quality detection is carried out, and the surface characteristics of the welding spots are highlighted so as to facilitate the subsequent extraction of the characteristics of the welding spots. As shown in fig. 4, the specific process of pretreatment includes the following steps:
s61, carrying out graying treatment on the target image to be detected to obtain a gray image, and improving the conditions of dark image and poor contrast.
It should be noted that, the target image to be detected includes three channels of red, green and blue, and the data describing the color of each channel has a value of 0-255, and the three components need to be processed respectively subsequently, so that the operation amount is greatly increased, and the processing efficiency is reduced; in the welding quality detection process, only the outline of the welding spot and the position information of the component are required to be extracted, the color of the welding spot is not required to be concerned, and therefore, the gray scale processing is required to be carried out on the welding part image, the component values of the red, green and blue channels of the gray scale image are equal, the original data are simplified, and the subsequent operation efficiency is improved.
S62, denoising the gray level image by adopting non-local mean filtering to obtain a denoised image;
the uneven reflection of the surface layer of the electronic element support body and the influence of the surrounding environment and the image acquisition equipment in the acquisition process of the welding image to be detected lead to various noises and distortions of the welding image to be detected, therefore, the noise generated by the dust of the surrounding environment and the characteristics of the electronic element support body can be filtered out by adopting non-local mean value filtering to process the gray image, but in the denoising process, the outline of a welding spot is weakened by different degrees, which is an important characteristic of welding quality detection, and the denoising image is required to be sharpened.
And S63, sharpening the edge part of the denoising image by using the Laplacian operator to obtain a preprocessed image with distinct welding spot contours and support layers.
The image to be detected is more blurred than the original image after the denoising treatment, and the welding characteristics are not convenient to extract later, so that the edge part of the denoising image, the welding area morphology and the gray jump part are required to be sharpened by adopting a Laplacian operator, and a preprocessed image with distinct welding spot contours and support layers is obtained.
By carrying out a series of pretreatment on the welding part image, the outline of the welding spot is not lost, and the contrast ratio of the electronic element and the supporting body is enhanced, so that a data foundation is laid for the subsequent welding quality detection of the electronic element.
S7, extracting a welding spot morphology line to be detected from the preprocessed image according to the segmentation threshold value, and calculating a morphology difference value X between the welding spot morphology line to be detected and a preset standard welding spot morphology line.
In this embodiment, the preprocessed image is divided into two major classes, namely a foreground and a background, the variance values of the two parts are calculated through global search, so that the gray value with the largest variance value between the two classes is used as the optimal segmentation threshold, and finally the morphological line of the welding spot to be detected is extracted from the preprocessed image according to the segmentation threshold.
Calculating a morphological difference value X between a to-be-detected welding spot morphological line and a preset standard welding spot morphological line according to a calculation formula, wherein the formula of the morphological difference value X is as follows:
wherein,representing the corresponding +.f. on the to-be-detected welding spot morphology line and the standard welding spot morphology line>Height difference of high points, +.>And the number of corresponding high points on the welding spot morphology line to be detected and the standard welding spot morphology line is represented.
S8, performing secondary detection according to a comparison result of the form difference value X and a preset detection threshold Tx, and judging whether welding defects exist or not;
If the form difference value X exceeds the range of the preset detection threshold Tx, judging as a disqualified product, otherwise judging as a qualified product.
And S9, outputting the detection result information and generating an instruction for recalling the unqualified products of the electronic component welding.
So that workers recall unqualified electronic products according to recall instructions, and the product qualification rate is improved, thereby more meeting the actual production requirements of modern enterprises.
Referring to fig. 5, the present embodiment further provides a system for implementing the method for detecting electronic component soldering, including:
an image acquisition module 101, configured to acquire a standard welding image acquired by an image acquisition device and a to-be-detected welding image of an electronic component welding member to be detected, where the standard welding image refers to an image that is qualified in welding quality and includes only an electronic component and a welding area thereof;
the first calculating module 102 is configured to calculate a matching coefficient r and a difference area Sc of the standard welding image and the welding image to be detected;
the preliminary detection module 103 is configured to perform preliminary detection on the electronic component welding product to be detected according to a preset matching coefficient threshold Tr and a difference area threshold Ts;
the to-be-detected welding area positioning module 104 is configured to search and compare a plurality of feature images with different resolutions of the to-be-detected welding image layer by layer according to invariant moment features of the image, so as to obtain a to-be-detected welding area that is accurately matched with the standard welding image;
The target image to be detected determining module 105 is configured to perform image segmentation on the welding image to be detected according to the welding area to be detected, so as to obtain a target image to be detected;
the image preprocessing module 106 is used for preprocessing the target image to be detected to obtain a preprocessed image with distinct welding spot contours and electronic element supporting body layers;
a second calculation module 107, configured to extract a morphological line of a to-be-detected welding spot from the preprocessed image according to the segmentation threshold, and calculate a morphological difference value X between the morphological line of the to-be-detected welding spot and a preset standard welding spot morphological line;
the secondary detection module 108 is configured to perform secondary detection on the welded product to be detected according to a comparison result of the morphological difference value X and the preset detection threshold Tx, and determine whether a welding defect exists;
and the detection result output module 109 is used for outputting detection result information and generating an instruction for recalling the unqualified products of the electronic component welding.
According to the embodiment, firstly, rough registration and difference operation are carried out on an obtained to-be-detected welding image and a standard welding image according to the welding defect type of an electronic element, whether to-be-detected welding products have a lack welding defect or not is judged by combining a preset threshold value, products with obvious welding defects are screened out, then, a plurality of characteristic images with different resolutions are extracted from the to-be-detected welding image by adopting an improved convolution neural network, a to-be-detected welding area matched with the standard welding image is positioned by using a layer-by-layer searching strategy, and then, the to-be-detected welding image is segmented according to the positioned to-be-detected welding area to obtain a to-be-detected target image, so that the calculation amount of subsequent image processing can be reduced, the detection efficiency of a system is improved, finally, the to-be-detected target image is preprocessed, secondary detection is carried out according to a form difference value, the welding quality detection efficiency of the electronic element is greatly improved, and the hardware cost is also reduced.
Example two
The implementation manner provided in this embodiment is made on the basis of the first embodiment, and the same parts can solve the same technical problems and have the same beneficial effects, so that the same technical problems can be referred to each other, and detailed description will not be expanded in this embodiment.
Referring to fig. 6-8, a specific implementation manner of the present embodiment is shown, in the present embodiment, a plurality of feature images with different resolutions are extracted from an acquired low-resolution welding image to be detected, and fusion reconstruction is performed on the feature images with different resolutions to obtain a high-resolution image, so that the purpose of improving the welding quality detection accuracy of an electronic component is achieved without increasing the detection cost.
As shown in fig. 6, a method for detecting soldering of an electronic component includes the steps of:
s1, acquiring a standard welding image acquired by an image acquisition device and a welding image to be detected of an electronic component welding piece to be detected, wherein the standard welding image refers to an image which is qualified in welding quality and only contains the electronic component and a welding area thereof.
S2, calculating a matching coefficient r of the standard welding image and the welding image to be detected and a difference area Sc.
S3, performing preliminary detection on the electronic element welding product to be detected according to a preset matching coefficient threshold Tr and a difference area threshold Ts;
if |r| < Tr and |Sc| < Ts, judging as a defective product and ending the detection, otherwise, executing step S4.
And S4, carrying out layer-by-layer searching and comparing on a plurality of characteristic images with different resolutions of the welding image to be detected according to invariant moment characteristics of the image, and obtaining a welding area to be detected which is accurately matched with the standard welding image.
S51, performing fusion reconstruction on a plurality of feature images with different resolutions by adopting a multi-scale feature fusion strategy so as to obtain a high-resolution image.
As shown in fig. 7, in the present embodiment, the improved convolutional neural network includes one input layer, ten convolutional layers and a downsampling layer, one full connection layer, and one output layer. Firstly, introducing an improved convolutional neural network into a welding image to be detected through an input layer, performing convolutional downsampling operation through a first convolutional layer to a third convolutional layer to generate four characteristic images, respectively extracting high-frequency, medium-frequency and low-frequency characteristic information in the image through three different frequency band channels of each characteristic image to obtain four characteristic images with different resolutions, wherein three convolutional layers are used for a high-frequency band channel part, two convolutional layers are used for a medium-frequency band channel part, the convolutional kernels of the convolutional layers used for the high-frequency band and the medium-frequency band are 3 multiplied by 3, and one convolutional layer with the convolutional kernel of 5 multiplied by 5 is used for a low-frequency band channel part; then, a multi-scale image searching and fusing unit (shown in fig. 8) is adopted to search the extracted characteristic images with different resolutions layer by layer, and after the welding area to be detected of the welding image to be detected is accurately positioned, a nonlinear mapping mode is adopted to fuse and reconstruct the characteristic images with different resolutions so as to obtain a high-resolution image.
The following will further describe a specific process of fusion reconstruction of high resolution images in conjunction with the multi-scale image search and fusion structure diagram of fig. 8:
firstly, carrying out maximum pooling downsampling on a primary characteristic image and carrying out 1X 1 convolution kernel calculation;
step two, extracting the characteristics of the two-level characteristic map after the two-level characteristic map is subjected to 1X 1 convolution kernel to reduce channel dimension simplification calculation, and performing transverse linking operation;
thirdly, fusing the feature images obtained in the first step and the second step, and performing linear interpolation twice up-sampling operation on the fused result;
fourthly, fusing the three feature images with different resolutions and the same size obtained in the first step to the third step to strengthen semantic information of the feature images, and calculating a fusion result through a 3X 3 convolution kernel to eliminate an aliasing effect generated by multiple times of fusion;
and fifthly, repeating the first step to the fourth step until feature extraction is completed, and obtaining a high-resolution image.
It should be noted that, in fig. 8, the feature images with one-level to four-level different resolutions correspond to Conv1 to conv4_x in the Resnet101 neural network respectively, so that semantic information extracted from the images with different resolutions is effectively utilized to generate a final high-resolution image, accuracy of a welding quality detection result is improved, and therefore misjudgment rate is reduced.
S52, dividing the high-resolution image according to the welding area to be detected to obtain a target image to be detected, wherein the target image only comprises the welding area.
After the high-resolution image is obtained, the fused high-resolution image is segmented according to the welding area to be detected and the segmentation threshold value, so that the target image to be detected which only comprises the welding area is obtained, and the aim of improving the welding quality detection accuracy of the electronic element is fulfilled on the premise of not increasing the detection cost.
S6, preprocessing the target image to be detected to obtain a preprocessed image with the welding spot outline and the electronic element support body being distinct in hierarchy.
S7, extracting a welding spot morphology line to be detected from the preprocessed image according to the segmentation threshold value, and calculating a morphology difference value X between the welding spot morphology line to be detected and a preset standard welding spot morphology line;
s8, performing secondary detection according to a comparison result of the form difference value X and a preset detection threshold Tx, and judging whether welding defects exist or not;
if the form difference value X exceeds the range of the preset detection threshold Tx, judging as a disqualified product, otherwise judging as a qualified product.
And S9, outputting the detection result information and generating an instruction for recalling the unqualified products of the electronic component welding.
Referring to fig. 9, the present embodiment further provides a system for implementing the method for detecting electronic component soldering, including:
an image acquisition module 101, configured to acquire a standard welding image acquired by an image acquisition device and a to-be-detected welding image of an electronic component welding member to be detected, where the standard welding image refers to an image that is qualified in welding quality and includes only an electronic component and a welding area thereof;
the first calculation module 102 is configured to calculate a matching coefficient r and a difference area Sc of the standard welding image and the welding image to be detected according to a calculation formula;
the preliminary detection module 103 is configured to perform preliminary detection on the electronic component welding product to be detected according to a preset matching coefficient threshold Tr and a difference area threshold Ts;
the to-be-detected welding area positioning module 104 is configured to search and compare a plurality of feature images with different resolutions of the to-be-detected welding image layer by layer according to invariant moment features of the image, so as to obtain a to-be-detected welding area that is accurately matched with the standard welding image;
the high-resolution image fusion module 110 is configured to perform fusion reconstruction on a plurality of feature images with different resolutions by adopting a multi-scale feature fusion strategy so as to obtain a high-resolution image;
The target image to be detected determining module 105 is configured to perform image segmentation on the high-resolution image according to the welding area to be detected to obtain a target image to be detected;
the image preprocessing module 106 is used for preprocessing the target image to be detected to obtain a preprocessed image with distinct welding spot contours and electronic element supporting body layers;
a second calculation module 107, configured to extract a morphological line of a to-be-detected welding spot from the preprocessed image according to the segmentation threshold, and calculate a morphological difference value X between the morphological line of the to-be-detected welding spot and a preset standard welding spot morphological line;
the secondary detection module 108 is configured to perform secondary detection according to a comparison result of the morphology difference value X and a preset detection threshold Tx, and determine whether a welding defect exists;
and the detection result output module 109 is used for outputting detection result information and generating an instruction for recalling the unqualified products of the electronic component welding.
According to the embodiment, the improved convolutional neural network is adopted to extract a plurality of characteristic images with different resolutions from the welding image to be detected, after the layer-by-layer searching strategy is used to locate the welding area to be detected matched with the standard welding image, the multi-scale characteristic fusion strategy is adopted to conduct characteristic fusion reconstruction on the characteristic images with different resolutions so as to obtain a high-resolution image, the fusion reconstructed high-resolution image is segmented according to the welding area to be detected, the target image to be detected only comprising the welding area is obtained, and the purpose of improving the welding quality detection efficiency and accuracy of the electronic element under the premise of not increasing the detection cost is achieved.
The present embodiment also provides a computer apparatus, whose internal structure will be described below with reference to fig. 10, including a processor, a memory, a network interface, a display screen, and an input device connected through a system bus.
The processor is a control center of the computer device, utilizes various interfaces and lines to connect various parts of the whole device, and realizes various functions of the computer device by running or executing computer readable instructions and/or modules stored in the memory and calling data stored in the memory; the system can be a central processing unit CPU, other general processors, a digital signal processor DSP, an application specific integrated circuit ASIC, an off-the-shelf programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components and the like; wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory is used for storing computer readable instructions and/or modules, and mainly comprises a storage medium and an internal memory, wherein the storage medium can be a nonvolatile storage medium or a volatile storage medium, the storage medium stores an operating system, and can also store computer readable instructions, and when the computer readable instructions are executed by the processor, the processor can realize a detection method for welding electronic elements. For example, steps S1 to S9 shown in fig. 1 and 6 and other extensions of the method and extensions of the relevant steps. Alternatively, the processor executes the computer readable instructions to implement the functions of each module/unit of the electronic component soldering detection system in the above embodiment, such as the functions of the modules 101 to 109 shown in fig. 5, and the description thereof is omitted herein for avoiding repetition.
The network interface is used for communicating with an external server through network connection; the display screen can be a liquid crystal display screen or an electronic ink display screen; the input device can be a touch layer covered on the display screen, can also be a key, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse, etc.
It should be noted that the memory may be integrated into the processor or may be separate from the processor, and the structure shown in fig. 10 is merely a schematic block diagram of a portion of the structure related to the present application, and does not constitute a limitation of the computer device to which the present application is applied, and a specific computer device may include more or less components than those shown in the drawings, or may combine some components, or employ different component arrangements.
The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments. From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, or may be implemented by hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present application.
The foregoing embodiments have been presented in a detail description of the invention, and are presented herein with a particular application to the understanding of the principles and embodiments of the invention, the foregoing embodiments being merely intended to facilitate an understanding of the method of the invention and its core concepts; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.

Claims (8)

1. The method for detecting the welding of the electronic element is characterized by comprising the following steps:
s1, acquiring a standard welding image acquired by image acquisition equipment and a to-be-detected welding image of an electronic element welding piece to be detected, wherein the standard welding image refers to an image which is qualified in welding quality and only contains an electronic element and a welding area thereof;
s2, calculating a matching coefficient r of the standard welding image and the welding image to be detected and a difference area Sc;
s3, performing preliminary detection on the electronic element welding product to be detected according to a preset matching coefficient threshold Tr and a difference area threshold Ts;
if |r| < Tr and |Sc| < Ts, judging that the product is not qualified and ending the detection, otherwise, executing the step S4;
S4, carrying out layer-by-layer searching and comparing on a plurality of characteristic images with different resolutions of the welding image to be detected according to invariant moment characteristics of the image to obtain a welding area to be detected which is accurately matched with the standard welding image; the step S4 specifically includes the following steps:
s41, inputting the welding image to be detected into an improved convolutional neural network, and performing convolutional downsampling operation by adopting convolutional kernels with different sizes to obtain a plurality of characteristic images with different resolutions;
s42, forming a plurality of feature images with different resolutions into a pyramid shape, wherein the higher the hierarchy is, the lower the resolution of the feature images is;
s43, carrying out exhaustive comparison on the characteristic image with low resolution of the top layer according to the welding spot area of the standard welding image to obtain a coarse positioning area;
s44, searching the characteristic images layer by layer in the sequence from top to bottom, and searching only the position near the coarse positioning area of the upper layer until the position of the bottom layer is reached to obtain a welding area to be detected, which is accurately matched with the standard welding image;
s5, carrying out image segmentation on the welding image to be detected according to the welding area to be detected to obtain a target image to be detected;
S6, preprocessing the target image to be detected to obtain a preprocessed image with distinct welding spot contours and electronic element supporting bodies;
s7, extracting a welding spot morphology line to be detected from the preprocessed image according to a segmentation threshold value, and calculating a morphology difference value X between the welding spot morphology line to be detected and a preset standard welding spot morphology line; in step S7, the formula for calculating the morphology difference value X is as follows:
wherein,representing the corresponding +.f. on the to-be-detected welding spot morphology line and the standard welding spot morphology line>Height difference of high points, +.>Representing the number of corresponding high points on the form line of the welding spot to be detected and the form line of the standard welding spot;
s8, performing secondary detection according to the comparison result of the form difference value X and a preset detection threshold Tx, and judging whether welding defects exist or not;
and if the form difference value X exceeds the range of the preset detection threshold Tx, judging that the product is a disqualified product, otherwise, judging that the product is a qualified product.
2. The method for inspecting soldering of electronic components according to claim 1, wherein the step S2 specifically comprises the steps of:
s21, moving the standard welding image in the welding image to be detected, and calculating a matching coefficient r between the standard welding image and the welding image;
S22, regarding the position where the maximum value of the matching coefficient appears as the rough registration of the standard welding image and the welding image to be detected;
s23, performing binarization processing on the standard welding image and the welding image to be detected after rough registration, and performing differential operation to obtain a difference area Sc.
3. The method for inspecting soldering of electronic components according to claim 1, wherein the step S6 specifically comprises the steps of:
s61, carrying out graying treatment on the target image to be detected to obtain a gray image;
s62, denoising the gray level image by adopting non-local mean filtering to obtain a denoised image;
and S63, sharpening the edge part of the denoising image by using the Laplacian operator to obtain a preprocessed image with distinct welding spot contours and support layers.
4. The method for inspecting soldering of electronic components according to claim 1, wherein the step S5 is configured to:
and carrying out fusion reconstruction on a plurality of feature images with different resolutions by adopting a multi-scale feature fusion strategy to obtain a high-resolution image, and dividing the high-resolution image according to a welding area to be detected to obtain a target image to be detected, wherein the target image only comprises the welding area.
5. The method for inspecting soldering of electronic components according to claim 1, wherein step S8 further comprises:
and S9, outputting the detection result information and generating an instruction for recalling the unqualified products of the electronic component welding.
6. A system for implementing a method for inspecting electronic component bonding according to any of the preceding claims 1-5, comprising:
the image acquisition module (101) is used for acquiring a standard welding image acquired by the image acquisition equipment and a to-be-detected welding image of the to-be-detected electronic element welding piece, wherein the standard welding image refers to an image which is qualified in welding quality and only comprises the electronic element and a welding area thereof;
the first calculation module (102) is used for calculating a matching coefficient r and a difference area Sc of the standard welding image and the welding image to be detected;
the primary detection module (103), the primary detection module (103) is used for carrying out primary detection on the electronic component welding product to be detected according to a preset matching coefficient threshold Tr and a difference area threshold Ts;
the welding area positioning module (104) to be detected is used for carrying out layer-by-layer searching and comparing on a plurality of characteristic images with different resolutions of the welding image to be detected according to invariant moment characteristics of the image, so as to obtain a welding area to be detected which is accurately matched with the standard welding image;
The detection target image determining module (105), wherein the detection target image determining module (105) is used for carrying out image segmentation on the welding image to be detected according to the welding area to be detected to obtain a target image to be detected;
the image preprocessing module (106) is used for preprocessing the target image to be detected to obtain a preprocessed image with distinct welding spot outline and electronic element support layers;
the second calculation module (107) is used for extracting a welding spot morphology line to be detected from the preprocessed image according to a segmentation threshold value, and calculating a morphology difference value X between the welding spot morphology line to be detected and a preset standard welding spot morphology line;
the secondary detection module (108), the secondary detection module (108) is used for carrying out secondary detection according to the comparison result of the morphological difference value X and a preset detection threshold Tx, and judging whether welding defects exist or not.
7. The system of claim 6, further comprising:
the detection result output module (109), the detection result output module (109) is used for outputting detection result information and generating an instruction for recalling unqualified products of electronic element welding;
The high-resolution image fusion module (110) is used for carrying out fusion reconstruction on a plurality of feature images with different resolutions by adopting a multi-scale feature fusion strategy so as to obtain a high-resolution image.
8. A computer device, characterized by comprising a processor and a memory for storing a computer program which, when executed by the processor, implements the method of detecting electronic component soldering according to any one of claims 1 to 5.
CN202310143229.XA 2023-02-21 2023-02-21 Detection method, system and computer equipment for electronic element welding Active CN115880288B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310143229.XA CN115880288B (en) 2023-02-21 2023-02-21 Detection method, system and computer equipment for electronic element welding

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310143229.XA CN115880288B (en) 2023-02-21 2023-02-21 Detection method, system and computer equipment for electronic element welding

Publications (2)

Publication Number Publication Date
CN115880288A CN115880288A (en) 2023-03-31
CN115880288B true CN115880288B (en) 2023-11-14

Family

ID=85761408

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310143229.XA Active CN115880288B (en) 2023-02-21 2023-02-21 Detection method, system and computer equipment for electronic element welding

Country Status (1)

Country Link
CN (1) CN115880288B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117938626A (en) * 2024-01-24 2024-04-26 南京脉期尔电气有限公司 Network warning system triggered by applying power-on detection
CN117746077B (en) * 2024-02-20 2024-05-10 深圳超盈智能科技有限公司 Chip defect detection method, device, equipment and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110517265A (en) * 2019-09-04 2019-11-29 北京理工大学 A kind of detection method of surface defects of products, device and storage medium
CN112116557A (en) * 2020-08-12 2020-12-22 西安交通大学 Radiographic image weld area defect detection method, storage medium and equipment
CN113362326A (en) * 2021-07-26 2021-09-07 广东奥普特科技股份有限公司 Method and device for detecting welding spot defects of battery
CN113379678A (en) * 2021-05-14 2021-09-10 珠海格力智能装备有限公司 Circuit board detection method and device, electronic equipment and storage medium
CN113888746A (en) * 2020-07-01 2022-01-04 上海电力大学 Welding seam image mode recognition method based on combination of Hu invariant moment and SVM
CN114240939A (en) * 2022-02-24 2022-03-25 苏州浪潮智能科技有限公司 Method, system, equipment and medium for detecting appearance defects of mainboard components
CN114693633A (en) * 2022-03-28 2022-07-01 深圳市深视智能科技有限公司 Welding defect detection method, device and system

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110517265A (en) * 2019-09-04 2019-11-29 北京理工大学 A kind of detection method of surface defects of products, device and storage medium
CN113888746A (en) * 2020-07-01 2022-01-04 上海电力大学 Welding seam image mode recognition method based on combination of Hu invariant moment and SVM
CN112116557A (en) * 2020-08-12 2020-12-22 西安交通大学 Radiographic image weld area defect detection method, storage medium and equipment
CN113379678A (en) * 2021-05-14 2021-09-10 珠海格力智能装备有限公司 Circuit board detection method and device, electronic equipment and storage medium
CN113362326A (en) * 2021-07-26 2021-09-07 广东奥普特科技股份有限公司 Method and device for detecting welding spot defects of battery
CN114240939A (en) * 2022-02-24 2022-03-25 苏州浪潮智能科技有限公司 Method, system, equipment and medium for detecting appearance defects of mainboard components
CN114693633A (en) * 2022-03-28 2022-07-01 深圳市深视智能科技有限公司 Welding defect detection method, device and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
硅电池片自动串焊表面缺陷在线视觉检测研究;陈智强;楼佩煌;钱晓明;;机电工程(第07期);全文 *

Also Published As

Publication number Publication date
CN115880288A (en) 2023-03-31

Similar Documents

Publication Publication Date Title
US11244432B2 (en) Image filtering based on image gradients
CN115908269B (en) Visual defect detection method, visual defect detection device, storage medium and computer equipment
CN115880288B (en) Detection method, system and computer equipment for electronic element welding
CN111915704A (en) Apple hierarchical identification method based on deep learning
CN106846339A (en) Image detection method and device
FR2705479A1 (en) Method and apparatus for pattern recognition.
CN114742799B (en) Industrial scene unknown type defect segmentation method based on self-supervision heterogeneous network
CN115830004A (en) Surface defect detection method, device, computer equipment and storage medium
CN114494223A (en) PCB defect detection model training method and device, electronic equipment and storage medium
CN113609984A (en) Pointer instrument reading identification method and device and electronic equipment
Ma et al. An automatic detection method of Mura defects for liquid crystal display
CN117197527A (en) Defect detection classification method and device for glass substrate of liquid crystal display screen
CN117152165A (en) Photosensitive chip defect detection method and device, storage medium and electronic equipment
CN116342519A (en) Image processing method based on machine learning
CN114792310A (en) Mura defect detection method for edge blurring in LCD screen
CN112833821B (en) Differential geometric three-dimensional micro-vision detection system and method for high-density IC welding spots
CN113674260A (en) SMT welding spot defect detection method
CN111415365B (en) Image detection method and device
CN117557565A (en) Detection method and device for lithium battery pole piece
CN116665146A (en) Vehicle thatch cover detection method and device, electronic equipment and storage medium
CN115619678A (en) Image deformation correction method and device, computer equipment and storage medium
Golodov et al. Weld segmentation and defect detection in radiographic images of pipe welds
CN116612146B (en) Image processing method, device, electronic equipment and computer storage medium
CN112666733B (en) LCD gray scale picture offline defect detection method, electronic equipment and storage medium
CN118275449B (en) Copper strip surface defect detection method, device and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant