CN117274258A

CN117274258A - Method, system, equipment and storage medium for detecting defects of main board image

Info

Publication number: CN117274258A
Application number: CN202311554529.3A
Authority: CN
Inventors: 钟煌
Original assignee: Shenzhen Elsky Ipc Technology Co ltd
Current assignee: Shenzhen Elsky Ipc Technology Co ltd
Priority date: 2023-11-21
Filing date: 2023-11-21
Publication date: 2023-12-22

Abstract

The invention relates to the field of image processing, and discloses a defect detection method, system, equipment and storage medium for a main board image, which are used for realizing intelligent main board defect analysis and improving the defect detection accuracy of the main board. The method comprises the following steps: acquiring multiple angles and multiple light source images of a target main board to obtain multiple initial main board images, and performing image denoising processing to obtain multiple target main board images; performing feature decomposition to obtain a multi-layer feature map; carrying out weak detail information enhancement processing to obtain a multi-layer detail enhancement image; performing image fusion on the multi-layer detail enhancement images of each target main board image to generate a fusion main board image; image blocking is carried out on the fusion main board image to obtain a plurality of target block images, and sub-image superposition is carried out on the plurality of target block images to obtain a plurality of target sub-images; and respectively inputting the multiple target sub-images into a main board defect detection model to detect main board defects, and obtaining target main board defect detection results.

Description

Method, system, equipment and storage medium for detecting defects of main board image

Technical Field

The present invention relates to the field of image processing, and in particular, to a method, a system, an apparatus, and a storage medium for detecting defects of a motherboard image.

Background

With the continuous development of artificial intelligence technology, the field of image processing has made remarkable progress. In the field of manufacturing, etc., it is becoming particularly critical to rapidly and accurately detect product quality. The main board is used as a core component of the electronic product, and the quality of the main board has important influence on the performance and stability of the whole product.

However, due to the complex structure of the motherboard, the detection of defects usually requires highly trained professionals, and the manual detection is time-consuming and laborious, and is prone to omission, particularly, there is a certain limitation in processing complex images and detecting fine defects, and the traditional single-angle image acquisition is prone to neglecting some fine defects, i.e. the accuracy of the existing scheme is low.

Disclosure of Invention

The invention provides a defect detection method, system, equipment and storage medium for a main board image, which are used for realizing intelligent main board defect analysis and improving the defect detection accuracy of the main board.

The first aspect of the present invention provides a method for detecting a defect of a motherboard image, the method comprising:

acquiring multiple angles and multiple light source images of a target main board to be detected to obtain multiple initial main board images, and performing image denoising processing on the multiple initial main board images to obtain multiple target main board images;

Respectively carrying out feature decomposition on each target main board image through a preset multi-layer decomposition model to obtain a multi-layer feature map of each target main board image;

carrying out weak detail information enhancement processing on the multilayer feature images through a preset pulse coupling neural network to obtain multilayer detail enhancement images of each target main board image;

image fusion is carried out on the multi-layer detail enhancement images of each target main board image by adopting a fusion rule with obvious region energy and contrast, and a corresponding fusion main board image is generated;

image blocking is carried out on the fusion main board image to obtain a plurality of target block images, and sub-image superposition is carried out on the plurality of target block images to obtain a plurality of target sub-images;

and respectively inputting the target sub-images into a preset main board defect detection model to detect main board defects, so as to obtain target main board defect detection results.

With reference to the first aspect, in a first implementation manner of the first aspect of the present invention, the performing multi-angle and multi-light source image acquisition on the target motherboard to be detected to obtain a plurality of initial motherboard images, and performing image denoising processing on the plurality of initial motherboard images to obtain a plurality of target motherboard images includes:

Acquiring multi-angle and multi-light source images of a target main board to be detected through a plurality of preset optical image sensors to obtain a plurality of initial main board images;

acquiring angle information of each initial main board image, and calculating a Gaussian kernel of each initial main board image according to the angle information;

performing Gaussian filtering operation on the plurality of initial main board images according to the Gaussian kernel to obtain a plurality of denoising main board images;

and performing distortion correction and offset calibration on the plurality of denoising mainboard images to obtain a plurality of target mainboard images.

With reference to the first aspect, in a second implementation manner of the first aspect of the present invention, the performing feature decomposition on each target motherboard image through a preset multi-layer decomposition model to obtain a multi-layer feature map of each target motherboard image includes:

inputting each target main board image into a preset multi-layer analysis model respectively, and carrying out feature decomposition on each target main board image through a gradient filter in the multi-layer analysis model to obtain a first layer feature map and a basic layer map;

performing feature decomposition on each target main board image through a gradient bilateral filter in the multi-layer decomposition model to obtain a second-layer feature map and a third-layer feature map;

Performing differential decomposition on the first layer of feature images and the target main board image to obtain a first thin structural feature image and a first thick structural feature image, performing differential decomposition on the second layer of feature images and the target main board image to obtain a second thin structural feature image and a second thick structural feature image, and performing differential decomposition on the third layer of feature images and the target main board image to obtain a third thin structural feature image and a third thick structural feature image;

and carrying out feature map fusion on the first fine structure feature map and the first coarse structure feature map, carrying out feature map fusion on the second fine structure feature map and the second coarse structure feature map, and carrying out feature map fusion on the third fine structure feature map and the third coarse structure feature map to obtain a multi-layer feature map of each target main board image.

With reference to the first aspect, in a third implementation manner of the first aspect of the present invention, the performing weak detail information enhancement processing on the multi-layer feature map through a preset pulse coupled neural network to obtain a multi-layer detail enhanced image of each target motherboard image includes:

inputting the multilayer characteristic diagram into a preset pulse coupling neural network, wherein the pulse coupling neural network comprises a receiving domain, a modulation domain and a pulse generator;

Local feature scanning is carried out on the multilayer feature map through a receiving domain in the pulse coupling neural network, so that corresponding local feature information is obtained;

adjusting the activation degree of the neurons according to the local characteristic information through a modulation domain in the pulse coupling neural network to obtain activation degree information;

generating a target pulse signal according to the local characteristic information of the receiving domain and the activation degree information of the modulation domain by a pulse generator in the pulse coupling neural network;

based on the target pulse signals propagated through the connection between neurons, generating weak responses to detail information in the multi-layer feature map and outputting multi-layer detail enhancement images of each target main board image.

With reference to the first aspect, in a fourth implementation manner of the first aspect of the present invention, the performing image fusion on the multi-layer detail enhancement image of each target main board image by using a fusion rule with significant region energy and contrast, to generate a corresponding fused main board image includes:

acquiring component information of the target main board, and determining a corresponding target local window according to the component information;

according to the target local window, calculating the pixel intensity variation degree of the multi-layer detail enhancement image of each target main board image to obtain a target area energy index of each target main board image;

According to the target local window, gray level change degrees are respectively carried out on the multi-layer detail enhancement images of each target main board image, and a target contrast significant index of each target main board image is obtained;

determining first fusion weights of a plurality of target pixel points in each target main board image according to the target area energy index, and determining second fusion weights of a plurality of target pixel points in each target main board image according to the target contrast significant index;

and carrying out weighted average and image fusion on a plurality of target pixel points in each target main board image according to the first fusion weight and the second fusion weight, and generating a corresponding fusion main board image.

With reference to the first aspect, in a fifth implementation manner of the first aspect of the present invention, the performing image blocking on the fused motherboard image to obtain a plurality of target block images, and performing sub-image superposition on the plurality of target block images to obtain a plurality of target sub-images, includes:

image blocking is carried out on the fusion main board image to obtain a plurality of target block images, and a plurality of target pixel areas of each target block image are obtained;

acquiring pixel density data of a plurality of target pixel areas in each target block image, and counting the maximum pixel density and the minimum pixel density of the plurality of target pixel areas;

According to the maximum pixel density and the minimum pixel density, carrying out normalization processing on the pixel density data of the plurality of target pixel areas to obtain normalized pixel density of each target pixel area;

calculating an image stacking weight value among the plurality of target block images based on the normalized pixel density of each target pixel region;

and according to the image overlapping weight value, carrying out sub-image overlapping on the plurality of target block images to obtain a plurality of target sub-images.

With reference to the first aspect, in a sixth implementation manner of the first aspect of the present invention, the inputting the plurality of target sub-images into a preset main board defect detection model to perform main board defect detection, to obtain a target main board defect detection result, includes:

inputting the target sub-images into a preset main board defect detection model respectively, wherein the main board defect detection model comprises a plurality of convolution networks and a full-connection network;

carrying out main board feature extraction on each target sub-image through the plurality of convolution networks to obtain a convolution feature image of each target sub-image;

inputting the convolution feature images of each target sub-image into the fully connected network to detect defects, and obtaining an initial main board defect detection result of each target sub-image;

And carrying out result fusion output on the initial main board defect detection result of each target sub-image to obtain a target main board defect detection result.

A second aspect of the present invention provides a defect detection system for a motherboard image, the defect detection system for a motherboard image including:

the acquisition module is used for acquiring multi-angle and multi-light source images of the target main board to be detected to obtain a plurality of initial main board images, and carrying out image denoising processing on the initial main board images to obtain a plurality of target main board images;

the decomposition module is used for respectively carrying out characteristic decomposition on each target main board image through a preset multi-layer decomposition model to obtain a multi-layer characteristic diagram of each target main board image;

the enhancement module is used for carrying out weak detail information enhancement processing on the multi-layer feature images through a preset pulse coupling neural network to obtain multi-layer detail enhancement images of each target main board image;

the fusion module is used for carrying out image fusion on the multi-layer detail enhancement images of each target main board image by adopting a fusion rule with obvious zone energy and contrast ratio to generate a corresponding fusion main board image;

the superposition module is used for performing image blocking on the fusion main board image to obtain a plurality of target block images, and performing sub-image superposition on the plurality of target block images to obtain a plurality of target sub-images;

And the detection module is used for respectively inputting the target sub-images into a preset main board defect detection model to detect main board defects, so as to obtain target main board defect detection results.

A third aspect of the present invention provides a defect detecting apparatus for a motherboard image, comprising: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to cause the defect detection device of the motherboard image to execute the defect detection method of the motherboard image described above.

A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored therein that, when executed on a computer, cause the computer to perform the above-described defect detection method of a motherboard image.

In the technical scheme provided by the invention, multi-angle and multi-light source image acquisition is carried out on a target main board to obtain a plurality of initial main board images, and image denoising processing is carried out to obtain a plurality of target main board images; performing feature decomposition to obtain a multi-layer feature map; carrying out weak detail information enhancement processing to obtain a multi-layer detail enhancement image; performing image fusion on the multi-layer detail enhancement images of each target main board image to generate a fusion main board image; image blocking is carried out on the fusion main board image to obtain a plurality of target block images, and sub-image superposition is carried out on the plurality of target block images to obtain a plurality of target sub-images; the invention uses a plurality of optical image sensors to collect multi-angle and multi-light source images, the system can obtain high-quality and omnibearing image information of the target mainboard, which is helpful to improve the accuracy and reliability of subsequent processing. By adopting the Gaussian filtering and other image denoising processing methods, noise in the image can be effectively reduced, the definition and quality of the image are improved, and the accurate detection of the defects of the main board by subsequent processing is facilitated. And carrying out feature decomposition on the target main board image by using a preset multi-layer decomposition model to obtain a multi-layer feature map. This helps to better understand the structure and details of the image, providing a more informative input for subsequent detail enhancement and defect detection. The multi-layer feature map is subjected to weak detail information enhancement through the pulse coupling neural network, so that weak details in an image can be captured and emphasized better, and the sensitivity and the detection effect on the main board defects are improved. The multi-layer detail enhancement images are fused by adopting a fusion rule with obvious region energy and contrast, so that local information of the images can be better combined, a fusion main board image with higher contrast and definition can be generated, and the visual detection effect of defects can be improved. By means of image blocking and sub-image superposition, the system can generate a plurality of target sub-images, so that defect detection is finer and more accurate. This approach helps to reduce false and missed detection situations. And the system can comprehensively utilize detection results of different sub-images by adopting a plurality of convolution networks and full-connection networks to detect the defects of the main board and fusing and outputting the results, so that the accuracy and the robustness of the overall defect detection are improved.

Drawings

FIG. 1 is a schematic diagram of an embodiment of a method for detecting defects in a motherboard image according to an embodiment of the present invention;

FIG. 2 is a flow chart of feature decomposition in an embodiment of the invention;

FIG. 3 is a flowchart of weak detail information enhancement processing in an embodiment of the present invention;

FIG. 4 is a flow chart of image fusion in an embodiment of the invention;

FIG. 5 is a schematic diagram of an embodiment of a defect detection system for a motherboard image according to an embodiment of the present invention;

fig. 6 is a schematic diagram of an embodiment of a defect detection apparatus for a motherboard image according to an embodiment of the present invention.

Detailed Description

The embodiment of the invention provides a defect detection method, system, equipment and storage medium for a main board image, which are used for realizing intelligent main board defect analysis and improving the defect detection accuracy of the main board. The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.

For easy understanding, the following describes a specific flow of an embodiment of the present invention, referring to fig. 1, and an embodiment of a method for detecting defects of a motherboard image in an embodiment of the present invention includes:

s101, performing multi-angle and multi-light source image acquisition on a target main board to be detected to obtain a plurality of initial main board images, and performing image denoising processing on the plurality of initial main board images to obtain a plurality of target main board images;

it can be understood that the execution subject of the present invention may be a defect detection system of a motherboard image, and may also be a terminal or a server, which is not limited herein. The embodiment of the invention is described by taking a server as an execution main body as an example.

Specifically, a plurality of preset optical image sensors are adopted to collect multi-angle and multi-light source images of a target main board to be detected. The purpose of this step is to acquire images of the target motherboard at a number of different angles and under the light source to ensure that all details of the motherboard are covered. For example, a motherboard contains complex circuit components and connections that exhibit different characteristics at different angles and light sources. And acquiring angle information for each initial main board image, and calculating corresponding Gaussian kernels according to the angle information. These gaussian kernels are important tools for image denoising. The size and shape of the Gaussian kernel can be adjusted according to the angle information so as to better adapt to the image characteristics under different angles. For example, if a portion of the motherboard is more susceptible to light at a certain angle, the gaussian kernel may be tuned to better handle this situation. And performing Gaussian filtering operation on the plurality of initial main board images by using the calculated Gaussian kernel. Gaussian filtering is a common technique for denoising that can help eliminate noise and interference in an image, thereby improving the quality of the image. By applying the corresponding Gaussian kernel on each image, noise in the image can be effectively reduced, and the image is clearer. And performing distortion correction and offset calibration. This is to ensure that the images at different angles and light sources are consistent in scale and position for subsequent processing and analysis. These calibration operations can correct for distortions and offsets caused by different angle shots and light source conditions to align all images under the same coordinate system. For example, suppose that it is necessary to detect a micro solder joint defect on a motherboard. These defects are easier to observe at different angles. By capturing the main plate image from different angles using multiple optical sensors, then applying gaussian filtering to remove noise, and finally through distortion correction and offset calibration, it can be ensured that the images at different angles are all aligned in the same coordinate system. In this way, weld defects can be detected more accurately, as they are not affected by angle and light source variations.

S102, respectively carrying out feature decomposition on each target main board image through a preset multi-layer decomposition model to obtain a multi-layer feature map of each target main board image;

specifically, each target motherboard image is input into a preset multi-layer analysis model. This model uses gradient filters to perform feature decomposition on each image, resulting in a multi-layered feature map. The feature maps include a first layer feature map and a base layer feature map, a second layer feature map, and a third layer feature map. Gradient filters help capture details and structural features in the image. And performing differential decomposition on the first layer of feature images and the target main board image to obtain a first thin structure feature image and a first thick structure feature image. This process helps to separate out the detailed and coarse structural parts of the image. Similarly, the second-layer feature map and the third-layer feature map are also subjected to differential decomposition to obtain a second fine-structure feature map and a second coarse-structure feature map, and a third fine-structure feature map and a third coarse-structure feature map, respectively. As the feature map of each image is decomposed into fine and coarse structural parts, feature map fusion is then performed. The fine and coarse structural feature maps of the respective layers are combined to generate a multi-layer feature map for each target motherboard image. This process may use various image processing techniques, such as weighted averaging or convolution operations, to obtain a multi-layer feature map that contains the different features and details of the original image. For example, fine defects on a printed circuit board (target motherboard) include solder joint quality problems or component damage. And decomposing each target mainboard image by the server through a multi-layer decomposition model to obtain images containing different layers of features. At the first layer, gradient filtering captures edges and fine detail of the image, thereby generating a first layer feature map. The differentiation process separates these detailed and coarse structures, enabling the server to focus on subtle changes and defects. At the second and third levels, the server continues to analyze higher level features such as the arrangement and connection of elements. The feature maps are fused together to generate a multi-layer feature map that integrates various details and structural features of the image. These multi-layer feature maps can be used for defect detection because they capture various aspects of the target motherboard image. By comparing these feature maps with known normal image features, the server detects potential defects, thereby improving the accuracy and efficiency of the detection.

S103, carrying out weak detail information enhancement processing on the multilayer feature images through a preset pulse coupling neural network to obtain multilayer detail enhancement images of each target main board image;

the multi-layer characteristic diagram is input into a preset pulse coupling neural network. This neural network comprises a receiving domain, a modulation domain and a pulse generator. These components work cooperatively to detail the multi-layer feature map. And carrying out local feature scanning on the multi-layer feature map through a receiving domain in the pulse coupling neural network. Neural networks may focus on local features in the image, such as edges, textures, or other important details. Local feature scanning helps to capture important parts of the image in order to better enhance the details of these areas. And adjusting the activation degree of the neurons according to the local characteristic information through a modulation domain in the pulse coupling neural network. This adjustment process can help the neural network to focus more on and highlight local features with important information, thereby improving the perceptibility of those features. A target pulse signal is generated based on local characteristic information of the receiving domain and activation degree information of the modulation domain by a pulse generator in the pulse coupled neural network. These pulse signals may be considered as feedback mechanisms of the network for adjusting the activity of neurons to enhance specific details. Based on the propagation of the target impulse signal through the connections between neurons, a weak response to detailed information in the multi-layer feature map is generated. This weak response reflects the network's perception of detailed information of the image. This output may be considered as a multi-layer detail enhanced image, which contains enhanced local feature information.

S104, performing image fusion on the multi-layer detail enhancement images of each target main board image by adopting a fusion rule with obvious region energy and contrast, and generating a corresponding fusion main board image;

specifically, the server obtains component information on the target motherboard, which may include the location, shape, and distribution of the elements. This information will be used to determine the target local window, which is a small region containing the target components, for calculating the region energy and contrast saliency index. And according to the target local window, the server calculates the pixel intensity variation degree of the multi-layer detail enhancement image of each target main board image respectively so as to obtain the target area energy index. This step helps the server to know the intensity variation of the pixels in the target area, i.e. the local energy distribution of the image. Meanwhile, the server calculates the gray level change degree of the multi-layer detail enhancement image of each target main board image according to the target local window so as to obtain a target contrast significant index. This index helps to measure the contrast of pixels within a local window, i.e. the local contrast distribution of the image. The server uses the target area energy index to determine a first fusion weight of a plurality of target pixel points in each target main board image. These weights reflect the energy distribution of each pixel within the target local window, helping to determine which pixels are more area-energy. Likewise, the server uses the target contrast saliency index to determine a second fusion weight for the plurality of target pixels in each target motherboard image. These weights reflect the contrast distribution of each pixel within the target local window, helping to determine which pixels are more contrast-salient. And according to the first fusion weight and the second fusion weight, the server performs weighted average and image fusion on a plurality of target pixel points in each target main board image. This fusion process will take into account the region energy and contrast saliency of the pixels to generate a corresponding fused tile image.

S105, performing image blocking on the fusion main board image to obtain a plurality of target block images, and performing sub-image superposition on the plurality of target block images to obtain a plurality of target sub-images;

specifically, image blocking is performed on the fusion main board image, and a plurality of target block images are generated. Each tile contains a portion of the motherboard image, including different elements or specific areas. This blocking process helps to more finely process the image information of different areas. In each target tile image, the server obtains a plurality of target pixel regions. These regions correspond to different elements or features of interest. By defining the target areas explicitly, the server further processes the image information within these areas. For each target tile image, the server obtains pixel density data for the target pixel region. The pixel density represents the number or distribution of pixels within each region. And counting the maximum and minimum pixel density values of each target pixel region to know the pixel distribution range in the region. And according to the maximum pixel density value and the minimum pixel density value, the server performs normalization processing on the pixel density data of each target pixel area. This step facilitates mapping pixel density values of different regions to similar scales for subsequent processing. Based on the normalized pixel density of each target pixel region, the server calculates an image overlay weight between the plurality of target tile images. These weights reflect the extent to which each segmented image contributes in the final sub-image. Generally, areas with higher pixel density will have higher weights to ensure that the information of these areas is more pronounced. And according to the calculated image superposition weighted values, sub-image superposition is carried out on the plurality of target block images, so that a plurality of target sub-images are generated. The sub-images represent information of different areas of the main board and are overlapped according to pixel density data and weights, so that the final sub-images contain key information of each area of the main board.

S106, inputting the target sub-images into a preset main board defect detection model to detect main board defects, and obtaining target main board defect detection results.

Specifically, a plurality of target sub-images are respectively input into a preset main board defect detection model. This model includes a plurality of convolutional networks and a fully-connected network. Convolutional networks focus on extracting features from input images, while fully connected networks are used to perform defect detection tasks. And respectively extracting the characteristics of each target sub-image through a plurality of convolution networks. The convolutional network functions to detect local features and patterns in the input image. The convolution network converts the image information into a convolution feature map through the operations of layer-by-layer convolution and pooling, wherein the convolution feature map contains feature information of different layers in the image. And respectively inputting the convolution characteristic images of each target sub-image into a fully-connected network to detect defects. The fully connected network is a deep learning model for converting the convolution features into final defect detection results. Fully connected networks are able to understand the high-level abstract features in the convolution signature and associate them with specific defect patterns. Each target sub-image generates an initial motherboard defect detection result. These initial results are binary classification (defective/non-defective) or degree of defective. However, the detection results of different sub-images may have some differences. In order to improve accuracy, the initial motherboard defect detection results need to be fused. This fusion process may include statistical methods, machine learning methods, or deep learning methods. The method aims at comprehensively considering the results of all the sub-images to obtain the final target main board defect detection result. For example, the convolution network first extracts features from each input image, such as the shape, color, and texture of the weld spot. These features are passed into the fully connected network in the form of a convolved feature map. The fully connected network accepts the convolution feature maps and then combines them with pre-trained model weights to perform weld defect detection. Each sub-image produces an initial test result indicating whether the spot is defective and the type and extent of the defect. The initial detection results are subjected to result fusion according to a statistical or deep learning method. This may be a voting mechanism or by using a deep learning technique, the information of each sub-image is comprehensively considered, and a final target motherboard defect detection result is generated.

In the embodiment of the invention, multi-angle and multi-light source image acquisition is carried out on a target main board to obtain a plurality of initial main board images, and image denoising processing is carried out to obtain a plurality of target main board images; performing feature decomposition to obtain a multi-layer feature map; carrying out weak detail information enhancement processing to obtain a multi-layer detail enhancement image; performing image fusion on the multi-layer detail enhancement images of each target main board image to generate a fusion main board image; image blocking is carried out on the fusion main board image to obtain a plurality of target block images, and sub-image superposition is carried out on the plurality of target block images to obtain a plurality of target sub-images; the invention uses a plurality of optical image sensors to collect multi-angle and multi-light source images, the system can obtain high-quality and omnibearing image information of the target mainboard, which is helpful to improve the accuracy and reliability of subsequent processing. By adopting the Gaussian filtering and other image denoising processing methods, noise in the image can be effectively reduced, the definition and quality of the image are improved, and the accurate detection of the defects of the main board by subsequent processing is facilitated. And carrying out feature decomposition on the target main board image by using a preset multi-layer decomposition model to obtain a multi-layer feature map. This helps to better understand the structure and details of the image, providing a more informative input for subsequent detail enhancement and defect detection. The multi-layer feature map is subjected to weak detail information enhancement through the pulse coupling neural network, so that weak details in an image can be captured and emphasized better, and the sensitivity and the detection effect on the main board defects are improved. The multi-layer detail enhancement images are fused by adopting a fusion rule with obvious region energy and contrast, so that local information of the images can be better combined, a fusion main board image with higher contrast and definition can be generated, and the visual detection effect of defects can be improved. By means of image blocking and sub-image superposition, the system can generate a plurality of target sub-images, so that defect detection is finer and more accurate. This approach helps to reduce false and missed detection situations. And the system can comprehensively utilize detection results of different sub-images by adopting a plurality of convolution networks and full-connection networks to detect the defects of the main board and fusing and outputting the results, so that the accuracy and the robustness of the overall defect detection are improved.

In a specific embodiment, the process of executing step S101 may specifically include the following steps:

(1) Acquiring multi-angle and multi-light source images of a target main board to be detected through a plurality of preset optical image sensors to obtain a plurality of initial main board images;

(2) Acquiring angle information of each initial main board image, and calculating a Gaussian kernel of each initial main board image according to the angle information;

(3) Carrying out Gaussian filtering operation on a plurality of initial main board images according to the Gaussian kernel to obtain a plurality of denoising main board images;

(4) And performing distortion correction and offset calibration on the plurality of denoising mainboard images to obtain a plurality of target mainboard images.

Specifically, image acquisition of multiple angles and multiple light sources is performed on a target main board to be detected through multiple preset optical image sensors. This ensures that multiple initial motherboard images are obtained from different perspectives and illumination conditions, providing more comprehensive information for subsequent processing and analysis. Then, the angle information thereof is extracted for each obtained initial main board image. The angle information may be a photographing angle of the camera or a position angle with respect to the main board. This information facilitates subsequent processing to understand the spatial relationship of the images. Then, a gaussian filter operation is performed on each initial main plate image by calculating a gaussian kernel from the angle information. Gaussian filtering is a technique for removing noise from an image and smoothing the image by blurring details in the image using gaussian kernels to remove noise. The result of the gaussian filtering is a plurality of denoised motherboard images that retain the critical features of the motherboard while removing noise. These denoised images are more suitable for subsequent analysis and processing. After obtaining a plurality of denoised main board images, distortion correction and offset calibration are performed. This procedure is used to correct image distortions and offsets due to camera angle or motherboard position to ensure that the image accurately reflects the actual state of the motherboard. Through the series of image processing steps, the server finally obtains a plurality of target main board images, and the images have high-quality characteristics and information through denoising, distortion correction and offset calibration. These target motherboard images may be used for subsequent analysis, detection, and quality control tasks.

In a specific embodiment, as shown in fig. 2, the process of executing step S102 may specifically include the following steps:

s201, inputting each target main board image into a preset multi-layer analysis model, and carrying out feature decomposition on each target main board image through a gradient filter in the multi-layer analysis model to obtain a first layer feature map and a basic layer map;

s202, carrying out feature decomposition on each target main board image through a gradient bilateral filter in a multi-layer decomposition model to obtain a second-layer feature map and a third-layer feature map;

s203, performing differential decomposition on the first layer of feature images and the target main board image to obtain a first thin structural feature image and a first thick structural feature image, performing differential decomposition on the second layer of feature images and the target main board image to obtain a second thin structural feature image and a second thick structural feature image, and performing differential decomposition on the third layer of feature images and the target main board image to obtain a third thin structural feature image and a third thick structural feature image;

s204, carrying out feature map fusion on the first fine structure feature map and the first coarse structure feature map, carrying out feature map fusion on the second fine structure feature map and the second coarse structure feature map, and carrying out feature map fusion on the third fine structure feature map and the third coarse structure feature map to obtain a multi-layer feature map of each target main board image.

Specifically, each target motherboard image is input into a preset multi-layer analysis model. The main task of this model is to subject the input image to a multi-level feature decomposition in order to analyze the different parts and details of the image. In the first hierarchy, the image is feature decomposed using gradient filters. Gradient filters help capture important features such as edges and contours in the image. This step generates a first layer feature map and a base layer map. The first layer feature map contains edge and texture information of the image, while the base layer map contains some of the basic features of the original image. Next, in a second level, the image is feature decomposed using a gradient bilateral filter. The gradient bilateral filter combines the similarity of spatial information and pixel values to more finely decompose image features. This step generates a second layer of feature maps that contain more rich feature information. In the third level, the image is further decomposed, again using a gradient bilateral filter, to generate a third level feature map. The feature map of this hierarchy is more detailed, including finer details and structure in the image. And performing differential decomposition on the first layer of feature images and the original target main board image to obtain a first fine structure feature image and a first coarse structure feature image. This step helps to separate the fine texture and structural features of the image. Similarly, in the second level and the third level, the target main board image is also subjected to differential decomposition, so as to obtain a second fine structural feature map and a second coarse structural feature map, and a third fine structural feature map and a third coarse structural feature map. These fine structure feature maps contain more subtle and complex feature information in the image. And carrying out feature map fusion on the feature maps of different levels of each target main board image. This process may employ various techniques, such as weighted averaging or deep learning, to synthesize different fine and coarse structural features to generate a final multi-layer feature map.

In a specific embodiment, as shown in fig. 3, the process of executing step S103 may specifically include the following steps:

s301, inputting a multilayer characteristic diagram into a preset pulse coupling neural network, wherein the pulse coupling neural network comprises a receiving domain, a modulation domain and a pulse generator;

s302, carrying out local feature scanning on the multilayer feature map through a receiving domain in the pulse coupling neural network to obtain corresponding local feature information;

s303, adjusting the activation degree of the neurons according to the local characteristic information through a modulation domain in the pulse coupling neural network to obtain activation degree information;

s304, generating a target pulse signal according to local characteristic information of a receiving domain and activation degree information of a modulation domain by a pulse generator in a pulse coupling neural network;

s305, generating weak response to detail information in the multilayer feature map based on the connection propagation of the target pulse signals among neurons, and outputting multilayer detail enhancement images of each target mainboard image.

Specifically, the multi-layer feature map is input into a preset pulse coupled neural network. This neural network includes three key components: a receive domain, a modulation domain, and a pulse generator. These components work cooperatively to process the input multi-layer feature map and enhance the detail information of the image. And carrying out local feature scanning on the multi-layer feature map through a receiving domain in the pulse coupling neural network. The task of the receiving domain is to detect local features in the image at each location. This may include textures, edges or other local structures. Through scanning, the network obtains corresponding local characteristic information. Then, the activation degree of the neurons is adjusted according to the local characteristic information by pulse coupling the modulation domain in the neural network. This step helps to determine which local features are interesting and their importance in the image. By means of the activation level information, the network can better focus on key details. Then, a target pulse signal is generated by a pulse generator in the pulse coupled neural network according to the local characteristic information of the receiving domain and the activation degree information of the modulation domain. These target pulse signals are the outputs of the neural network reflecting the key detail information detected in the multi-layer feature map. Based on the target pulse signal, a weak response to detail information in the multi-layer feature map is generated through connection propagation between neurons. This weak response may be added back to the original image, thereby generating a multi-layer detail enhanced image for each target motherboard image. This image contains enhanced detail information that makes defects and features on the motherboard easier to detect and analyze. For example, assume that a server acquires a multi-layer feature map and then inputs it into a pulse coupled neural network. The neural network scans local features of the image, adjusts the activation level according to its importance, and generates a target pulse signal. These pulse signals propagate to the connections between neurons, generating a weak response that reflects the tiny details in the image. These weak responses are superimposed on the original image, creating an enhanced image, making the weld defects more visible, facilitating detection and repair. This approach improves the efficiency and accuracy of manufacturing quality control.

In a specific embodiment, as shown in fig. 4, the process of executing step S104 may specifically include the following steps:

s401, acquiring component information of a target main board, and determining a corresponding target local window according to the component information;

s402, respectively calculating the pixel intensity variation degree of the multi-layer detail enhancement image of each target main board image according to the target local window to obtain a target area energy index of each target main board image;

s403, respectively carrying out gray level change degree on the multi-layer detail enhancement images of each target main board image according to the target local window to obtain a target contrast significant index of each target main board image;

s404, determining first fusion weights of a plurality of target pixel points in each target main board image according to the target area energy index, and determining second fusion weights of a plurality of target pixel points in each target main board image according to the target contrast significant index;

s405, performing weighted average and image fusion on a plurality of target pixel points in each target main board image according to the first fusion weight and the second fusion weight, and generating a corresponding fusion main board image.

Specifically, component information of the target motherboard is acquired, which may include the position, size, shape, and the like of the component. Such information may be obtained through computer vision techniques or other sensors. Based on the component information, a corresponding target local window is determined. This window should include the target component and be large enough to capture the surrounding environment. The local window will be used as a region of interest in subsequent calculations. And respectively calculating the pixel intensity variation degree of the multi-layer detail enhancement image of each target main board image according to the target local window. This involves analyzing the intensity values of the pixels to determine the energy distribution in the region. This calculation will produce a target region energy indicator for quantifying intensity variations within the region. And meanwhile, calculating the gray level change degree of the multi-layer detail enhancement image of each target main board image. This step is used to evaluate the degree of gray scale variation between pixels within a local window. This will generate a target contrast saliency index for representing the contrast within the region. And determining first fusion weights of a plurality of target pixel points in each target main board image according to the target area energy index. This weight may be assigned according to the distribution of energy to highlight the high energy portions within the region. And meanwhile, determining second fusion weights of a plurality of target pixel points in each target main board image according to the target contrast significant index. This weight may be assigned according to a distribution of contrast to emphasize the high contrast portions within the region. And carrying out weighted average and image fusion on a plurality of target pixel points in each target main board image according to the first fusion weight and the second fusion weight. This will generate a corresponding fused motherboard image in which the high energy and high contrast regions will be more prominent, helping to detect and analyze features and defects of the target motherboard. For example, assume that a server obtains component information of a circuit board, including solder joint positions. The server determines a target local window around each weld spot to analyze the quality of the weld spot. The server obtains the target area energy index and the target contrast significant index by calculating the pixel intensity change and the gray level change. Based on these metrics, the server generates a fused image to highlight high quality welds and to help detect problem welds. This approach helps to improve the quality and reliability of circuit board manufacture.

In a specific embodiment, the process of executing step S105 may specifically include the following steps:

(1) Image blocking is carried out on the fusion main board image to obtain a plurality of target block images, and a plurality of target pixel areas of each target block image are obtained;

(2) Acquiring pixel density data of a plurality of target pixel areas in each target block image, and counting the maximum pixel density and the minimum pixel density of the plurality of target pixel areas;

(3) According to the maximum pixel density and the minimum pixel density, carrying out normalization processing on pixel density data of a plurality of target pixel areas to obtain normalized pixel density of each target pixel area;

(4) Calculating an image stacking weight value among a plurality of target block images based on the normalized pixel density of each target pixel region;

(5) And according to the image superposition weighted value, sub-image superposition is carried out on the plurality of target block images, so as to obtain a plurality of target sub-images.

Specifically, the fusion master board image is divided into a plurality of target block images, which can be realized by an image block technique. Each target tile image will contain a region of interest for further analysis. And identifying and extracting a plurality of target pixel areas in each target block image. This may be achieved by a target detection algorithm or by identification of pixel areas based on characteristics of color, shape, etc. Next, pixel density data of each region is calculated. Pixel density refers to the pixel value of each pixel, typically representing color or brightness. By counting the pixel density data, the maximum pixel density and the minimum pixel density of each target pixel region can be found. And carrying out normalization processing on the pixel density data of the plurality of target pixel areas according to the maximum pixel density and the minimum pixel density. Normalization may map pixel density values between different regions to the same scale for comparison and analysis. One common normalization method is to scale the pixel density values to between 0 and 1. Based on the normalized pixel density of each target pixel region, an image stack weight between a plurality of target tile images may be calculated. These weights may be calculated from the differences in normalized pixel density in order to more emphasize areas with high pixel density. These weights will be used to control the contribution of the different tile images. And according to the image superposition weighted value, carrying out sub-image superposition on the plurality of target block images. This will generate a plurality of target sub-images, with each segmented image superimposed according to its contribution. These sub-images will highlight features of different areas and help to better understand and analyze the characteristics and defects of the target motherboard. For example, assume that the server divides the fused motherboard image into a plurality of tile images, and then identifies the location of each solder joint using computer vision techniques. The server extracts the target pixel area around each pad and calculates pixel density data. By comparing the pixel densities of the different solder joint areas, the server determines the difference in solder joint quality. The server superimposes these segmented images together using the image overlay weights to generate sub-images that highlight quality differences, helping the server to better detect and analyze problems with the welds. The method can improve the control and production efficiency of the quality of the welding spots of the circuit board.

In a specific embodiment, the process of executing step S106 may specifically include the following steps:

(1) Respectively inputting a plurality of target sub-images into a preset main board defect detection model, wherein the main board defect detection model comprises a plurality of convolution networks and a full connection network;

(2) Carrying out main board feature extraction on each target sub-image through a plurality of convolution networks to obtain a convolution feature image of each target sub-image;

(3) Inputting the convolution feature images of each target sub-image into a fully-connected network to detect defects, and obtaining an initial main board defect detection result of each target sub-image;

(4) And carrying out result fusion output on the initial main board defect detection result of each target sub-image to obtain a target main board defect detection result.

Specifically, a motherboard defect detection model is prepared, which is typically a deep learning model including a plurality of Convolutional Neural Networks (CNNs) and fully-connected networks. This model is pre-trained to enable features to be extracted from the input image and defect detection. And respectively inputting the multiple target sub-images into a main board defect detection model. Each sub-image represents a different portion of the main panel or images of a plurality of different perspectives. These sub-images will be subject to motherboard feature extraction through a number of convolutional networks through the forward propagation process of the model. Each convolution network performs feature extraction on the sub-images to generate a convolution feature map. These convolution feature maps will contain rich information about the sub-images, including edges, textures, and other features. The convolution feature map of each target sub-image is input into a fully connected network for defect detection. The fully connected network will learn how to translate the convolution characteristics into defect detection results. This step will generate an initial motherboard defect detection result for each sub-image. And carrying out result fusion on the initial main board defect detection result of each target sub-image. The fusion of results may employ various techniques such as voting, weighted averaging, or other ensemble learning methods. This will produce a final target motherboard defect detection result to determine any defects or problems on the motherboard. For example, assume that a server first takes a plurality of target main board images of different perspectives, dividing them into a plurality of sub-images. The server feeds these sub-images into a pre-trained motherboard defect detection model. The convolved feature map of each sub-image is passed to a fully connected network that will generate an initial defect detection result. The server combines the detection results of all the sub-images together by adopting a result fusion method, such as a voting method, so as to determine the defect condition on the whole target main board. The method can improve the detection and control of the quality of the target main board in the electronic manufacturing process.

The method for detecting defects of a motherboard image in the embodiment of the present invention is described above, and the system for detecting defects of a motherboard image in the embodiment of the present invention is described below, referring to fig. 5, where an embodiment of the system for detecting defects of a motherboard image in the embodiment of the present invention includes:

the acquisition module 501 is configured to acquire multiple angles and multiple light source images of a target motherboard to be detected, obtain multiple initial motherboard images, and perform image denoising processing on the multiple initial motherboard images to obtain multiple target motherboard images;

the decomposition module 502 is configured to perform feature decomposition on each target motherboard image through a preset multi-layer decomposition model, so as to obtain a multi-layer feature map of each target motherboard image;

the enhancement module 503 is configured to perform weak detail information enhancement processing on the multi-layer feature map through a preset pulse coupled neural network, so as to obtain a multi-layer detail enhancement image of each target motherboard image;

the fusion module 504 is configured to perform image fusion on the multi-layer detail enhancement images of each target main board image by using a fusion rule with significant region energy and contrast, so as to generate a corresponding fusion main board image;

the superposition module 505 is configured to perform image blocking on the fused motherboard image to obtain a plurality of target block images, and perform sub-image superposition on the plurality of target block images to obtain a plurality of target sub-images;

And the detection module 506 is configured to input the plurality of target sub-images into a preset main board defect detection model to detect main board defects, so as to obtain a target main board defect detection result.

Through the cooperation of the components, multi-angle and multi-light source image acquisition is carried out on the target main board, a plurality of initial main board images are obtained, image denoising processing is carried out, and a plurality of target main board images are obtained; performing feature decomposition to obtain a multi-layer feature map; carrying out weak detail information enhancement processing to obtain a multi-layer detail enhancement image; performing image fusion on the multi-layer detail enhancement images of each target main board image to generate a fusion main board image; image blocking is carried out on the fusion main board image to obtain a plurality of target block images, and sub-image superposition is carried out on the plurality of target block images to obtain a plurality of target sub-images; the invention uses a plurality of optical image sensors to collect multi-angle and multi-light source images, the system can obtain high-quality and omnibearing image information of the target mainboard, which is helpful to improve the accuracy and reliability of subsequent processing. By adopting the Gaussian filtering and other image denoising processing methods, noise in the image can be effectively reduced, the definition and quality of the image are improved, and the accurate detection of the defects of the main board by subsequent processing is facilitated. And carrying out feature decomposition on the target main board image by using a preset multi-layer decomposition model to obtain a multi-layer feature map. This helps to better understand the structure and details of the image, providing a more informative input for subsequent detail enhancement and defect detection. The multi-layer feature map is subjected to weak detail information enhancement through the pulse coupling neural network, so that weak details in an image can be captured and emphasized better, and the sensitivity and the detection effect on the main board defects are improved. The multi-layer detail enhancement images are fused by adopting a fusion rule with obvious region energy and contrast, so that local information of the images can be better combined, a fusion main board image with higher contrast and definition can be generated, and the visual detection effect of defects can be improved. By means of image blocking and sub-image superposition, the system can generate a plurality of target sub-images, so that defect detection is finer and more accurate. This approach helps to reduce false and missed detection situations. And the system can comprehensively utilize detection results of different sub-images by adopting a plurality of convolution networks and full-connection networks to detect the defects of the main board and fusing and outputting the results, so that the accuracy and the robustness of the overall defect detection are improved.

The above-described system for detecting defects in a motherboard image in an embodiment of the present invention is described in detail in fig. 5 from the point of view of a modularized functional entity, and the following describes the device for detecting defects in a motherboard image in an embodiment of the present invention from the point of view of hardware processing.

Fig. 6 is a schematic structural diagram of a defect detection apparatus for a motherboard image, where the defect detection apparatus 600 for a motherboard image may have a relatively large difference due to different configurations or performances, and may include one or more processors (central processing units, CPU) 610 (e.g., one or more processors) and a memory 620, and one or more storage media 630 (e.g., one or more mass storage devices) storing applications 633 or data 632. Wherein the memory 620 and the storage medium 630 may be transitory or persistent storage. The program stored in the storage medium 630 may include one or more modules (not shown), each of which may include a series of instruction operations in the defect detection device 600 for a motherboard image. Still further, the processor 610 may be configured to communicate with the storage medium 630 to execute a series of instruction operations in the storage medium 630 on the defect detection device 600 of the motherboard image.

The defect detection device 600 of the motherboard image may also include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input/output interfaces 660, and/or one or more operating systems 631, such as Windows Server, mac OS X, unix, linux, freeBSD, and the like. It will be appreciated by those skilled in the art that the configuration of the defect detection device of the motherboard image shown in fig. 6 does not constitute a limitation of the defect detection device of the motherboard image, and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.

The present invention also provides a device for detecting defects of a motherboard image, where the device for detecting defects of a motherboard image includes a memory and a processor, and the memory stores computer readable instructions, which when executed by the processor, cause the processor to execute the steps of the method for detecting defects of a motherboard image in the above embodiments.

The present invention also provides a computer readable storage medium, which may be a non-volatile computer readable storage medium, and may also be a volatile computer readable storage medium, where instructions are stored in the computer readable storage medium, when the instructions are executed on a computer, cause the computer to perform the steps of the defect detection method for the motherboard image.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. The defect detection method of the main board image is characterized by comprising the following steps of:

2. The method for detecting defects of a motherboard image according to claim 1, wherein the performing multi-angle and multi-light source image acquisition on the target motherboard to be detected to obtain a plurality of initial motherboard images, and performing image denoising processing on the plurality of initial motherboard images to obtain a plurality of target motherboard images includes:

3. The method for detecting defects of a motherboard image according to claim 1, wherein the performing feature decomposition on each target motherboard image by a preset multi-layer decomposition model to obtain a multi-layer feature map of each target motherboard image includes:

4. The method for detecting defects of a motherboard image according to claim 1, wherein the performing weak detail information enhancement processing on the multi-layer feature map by using a preset pulse coupled neural network to obtain multi-layer detail enhancement images of each target motherboard image comprises:

5. The method for detecting defects of a motherboard image according to claim 1, wherein the performing image fusion on the multi-layer detail enhancement image of each target motherboard image by using a fusion rule with significant region energy and contrast to generate a corresponding fused motherboard image comprises:

6. The method for detecting defects of a motherboard image according to claim 1, wherein the performing image blocking on the fused motherboard image to obtain a plurality of target block images, and performing sub-image superposition on the plurality of target block images to obtain a plurality of target sub-images, includes:

7. The method for detecting defects of a motherboard image according to claim 1, wherein inputting the plurality of target sub-images into a preset motherboard defect detection model respectively to detect the motherboard defects, and obtaining a target motherboard defect detection result comprises:

8. A defect detection system for a motherboard image, the defect detection system comprising:

9. A defect detecting apparatus of a main board image, characterized in that the defect detecting apparatus of a main board image comprises: a memory and at least one processor, the memory having instructions stored therein;

the at least one processor invokes the instructions in the memory to cause the defect detection device of the motherboard image to perform the defect detection method of the motherboard image as recited in any of claims 1-7.

10. A computer readable storage medium having instructions stored thereon, wherein the instructions when executed by a processor implement the method of defect detection of a motherboard image according to any of claims 1-7.