CN114240939B - Method, system, equipment and medium for detecting appearance defects of mainboard components - Google Patents

Method, system, equipment and medium for detecting appearance defects of mainboard components Download PDF

Info

Publication number
CN114240939B
CN114240939B CN202210171790.4A CN202210171790A CN114240939B CN 114240939 B CN114240939 B CN 114240939B CN 202210171790 A CN202210171790 A CN 202210171790A CN 114240939 B CN114240939 B CN 114240939B
Authority
CN
China
Prior art keywords
image
detected
main board
mainboard
pixels
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210171790.4A
Other languages
Chinese (zh)
Other versions
CN114240939A (en
Inventor
华逢友
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Inspur Intelligent Technology Co Ltd
Original Assignee
Suzhou Inspur Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Inspur Intelligent Technology Co Ltd filed Critical Suzhou Inspur Intelligent Technology Co Ltd
Priority to CN202210171790.4A priority Critical patent/CN114240939B/en
Publication of CN114240939A publication Critical patent/CN114240939A/en
Application granted granted Critical
Publication of CN114240939B publication Critical patent/CN114240939B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • G06T7/001Industrial image inspection using an image reference approach
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • G06T2207/30141Printed circuit board [PCB]
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a method, a system, equipment and a medium for detecting appearance defects of main board components, wherein the method comprises the following steps: acquiring a high-definition image of a mainboard, preprocessing the high-definition image to obtain an image to be detected, comparing the image to be detected of the mainboard through a standard template image corresponding to the mainboard, and storing the image to be detected of the mainboard and a comparison result thereof; responding to the stored images to be detected of the main board and the comparison results thereof to be accumulated to a preset number, and training an image recognition model through the stored images to be detected of the main board and the comparison results thereof; and carrying out defect detection on the acquired image to be detected of the mainboard through the trained image recognition model. By the method, the system, the equipment and the medium for detecting the appearance defects of the mainboard components, the problem that deep learning model reasoning cannot be used due to the fact that defect samples are difficult to collect in the mainboard appearance defect detection process can be effectively solved.

Description

Method, system, equipment and medium for detecting appearance defects of mainboard components
Technical Field
The invention belongs to the field of computers, and particularly relates to a method, a system, equipment and a medium for detecting appearance defects of a mainboard component.
Background
With the development of Artificial Intelligence (AI) technology in recent years, especially in the field of deep learning, more and more new technologies are applied to production and life of people. Currently, AI has achieved technology ground in a plurality of fields such as finance, medical treatment, industrial manufacturing and security, and application scenarios are increasingly abundant, which leads to a profound revolution in various industries. The future development of AI is the combination of technology and industry, realizes the AI technology to enable various industries, solves pain points, creates value, reduces cost and improves efficiency.
The current major outbreaks of artificial intelligence are caused by deep learning. Deep learning is to use a deep neural network to automatically learn the characteristics of an object, and then the deep neural network has the capability of identifying the object. Image recognition has been increasingly applied as an important direction of deep learning applications such as CNN (Convolutional Neural Networks), RNN (Recurrent Neural Networks). The method is more mature in the fields of intelligent driving, face recognition, medical image recognition, industrial quality inspection and the like.
In the actual factory production, quality detection plays an important role as the guarantee of product quality.
At present, the surface defect detection of most of main board components is basically carried out by naked eyes of workers or by means of naked eyes of a magnifier, the condition of missed detection and error detection is not only low in efficiency but also easy to occur, and meanwhile, the workers use eyes for a long time, fatigue damage is easy to cause, and the damage is irreversible.
Based on the reasons, a plurality of quality inspectors must be equipped on each production line to detect and rotate regularly, so that the quality of the product is ensured, and the damage to eyes is reduced, thereby not only reducing the production efficiency, but also needing to invest a large amount of manpower and material resources, lowering the product profit and influencing the product competitiveness. Meanwhile, due to the fact that eyes are used excessively, the loss rate of quality inspection workers is high, and a factory is in a state of always calling for workers.
Aiming at the problems, the invention provides a method for combining image registration, template matching and deep learning, which is a method for detecting defects by registration and template matching in the early period, automatically labeling the defects in the problems, training a model when the number of defect samples reaches a certain number, and detecting by using the deep learning model after training. The method overcomes the defects that defect samples need to be collected from the beginning to perform model training when deep learning is directly used for defect detection in the prior art. In normal production products, the number of defect products is relatively very small, all kinds of defect samples need to be collected, the method is very difficult to achieve in reality, and a plurality of model methods using deep learning are not practical in actual production.
Therefore, a detection method which is more convenient for detection is needed.
Disclosure of Invention
In order to solve the above problems, the present invention provides a method for detecting an appearance defect of a motherboard device, including:
acquiring a high-definition image of a mainboard, preprocessing the high-definition image to obtain an image to be detected, comparing the image to be detected of the mainboard through a standard template image corresponding to the mainboard, and storing the image to be detected of the mainboard and a comparison result thereof;
responding to the stored images to be detected of the mainboard and the comparison results thereof accumulated to a preset number, and training an image recognition model through the stored images to be detected of the mainboard and the comparison results thereof; and
and carrying out defect detection on the acquired image to be detected of the mainboard through the trained image recognition model.
In some embodiments of the present invention, obtaining a high-definition image of a motherboard and preprocessing the high-definition image to obtain an image to be detected, and comparing the image to be detected of the motherboard with a standard motherboard image corresponding to the motherboard includes:
carrying out bit-wise subtraction on the image to be detected of the main board and the pixel corresponding to the standard template image corresponding to the main board to obtain a corresponding difference value, and judging whether the difference value is greater than a first threshold value or not;
and in response to the difference value being larger than a first threshold value, removing the isolated pixels of which the difference value is larger than the first threshold value through a median filtering algorithm, and automatically marking the area of which the difference value is larger than the first threshold value as a defect area.
In some embodiments of the present invention, obtaining a high-definition image of a motherboard and preprocessing the high-definition image to obtain an image to be detected, and comparing the image to be detected of the motherboard with a standard motherboard image corresponding to the motherboard further includes:
generating a corresponding abstract picture by the image to be detected of the main board and the standard template image corresponding to the main board according to the difference of pixel values between adjacent images;
carrying out bit-wise subtraction on the generated abstract picture of the image to be detected of the main board and the generated standard template image corresponding to the main board to obtain a difference value of corresponding pixel bits;
removing the pixels with the difference value not being the first preset value through a median filtering algorithm, and judging whether the area of a region formed by the pixels with the difference value not being the first preset value is larger than a second preset value or not;
automatically marking the region as a defective region in response to the area of the region being greater than a second predetermined value.
In some embodiments of the present invention, generating the corresponding abstract picture according to the difference between the pixel values of the adjacent images of the main board and the standard template image corresponding to the main board includes:
acquiring RGB channel values of corresponding pixels, and calculating Euclidean distances between the RGB channel values of the pixels and the RGB channel values of the adjacent pixels according to the RGB channel values of the pixels;
judging Euclidean distances of RGB channel values of the pixels and the pixels between adjacent pixels, dividing the pixels with the same Euclidean distances or the difference smaller than a second threshold value into the same area, and uniformly coloring the color of the area according to the color of the most pixels in the area.
In some embodiments of the present invention, obtaining a high-definition image of a motherboard and preprocessing the high-definition image to obtain an image to be detected, and comparing the image to be detected of the motherboard with a standard motherboard image corresponding to the motherboard further includes:
generating a standard vector diagram of the main board according to the standard template image corresponding to the main board and the design drawing of the main board;
generating a vector diagram of the main board from the image to be detected of the main board according to the design drawing of the main board;
scaling the vector diagram and the standard vector diagram of the main board to a preset size, carrying out bitwise subtraction on the vector diagram of the main board scaled to the preset size and the standard vector diagram of the main board, and simultaneously judging whether the bitwise subtraction value of each bitwise pixel is a first preset value or not;
and in response to the bit-wise subtraction value of the pixel being a first preset value, marking the component area to which the pixel belongs as a defect area according to the design drawing of the mainboard.
In some embodiments of the present invention, training an image recognition model according to the saved image to be detected of the motherboard and the comparison result thereof includes:
decomposing the stored image to be detected of the main board into a plurality of local images to be detected, and associating the comparison result of the images to be detected with the corresponding local images to be detected;
and respectively and independently training corresponding image recognition models according to the local images to be detected and the corresponding comparison results thereof.
In some embodiments of the present invention, the performing defect detection on the to-be-detected image of the acquired motherboard by using the trained image recognition model includes:
preprocessing the high-definition image of the mainboard according to the acquired high-definition image of the mainboard in the preprocessing mode of the image to be detected of the trained image recognition model to obtain the image to be detected;
decomposing the image to be detected into a plurality of local images to be detected according to the decomposition mode and respectively inputting the local images to be detected into the image recognition model for detection;
and marking the defect detection result of the main board as a defect in response to the fact that the detection result of any one of the image recognition models is abnormal.
In another aspect of the present invention, a system for detecting an appearance defect of a motherboard device is further provided, including:
the image registration detection module is configured to acquire a high-definition image of a main board and preprocess the high-definition image to obtain an image to be detected, compare the image to be detected of the main board with a standard template image corresponding to the main board, and store the image to be detected of the main board and a comparison result thereof;
the model training module is configured for responding to the stored images to be detected of the mainboard and the comparison results thereof, accumulating the images to be detected of the mainboard and the comparison results thereof to a preset number, and training an image recognition model through the stored images to be detected of the mainboard and the comparison results thereof; and
and the model detection module is configured and used for carrying out defect detection on the acquired to-be-detected image of the mainboard through the trained image recognition model.
Yet another aspect of the present invention is a computer apparatus comprising:
at least one processor; and
a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of the method of any one of the above embodiments.
Yet another aspect of the present invention further provides a computer-readable storage medium, which stores a computer program, and the computer program realizes the steps of the method of any one of the above embodiments when executed by a processor.
According to the method, the system, the equipment and the medium for detecting the appearance defects of the mainboard components, the problem that deep learning model reasoning cannot be used due to the fact that defect samples are difficult to collect in the mainboard appearance defect detection process can be effectively solved, and due to the fact that the sizes of the mainboards are different and the types of the mainboard components are different in factory production line production, training of models is completed by collecting defect sample pictures of the components, and then defect detection is not practical. The method comprises the steps of firstly using a template for comparison to collect defect samples, collecting defect sample pictures while detecting defects, and then carrying out model training along with the increase of the sample pictures. The detection of any mainboard product can be used immediately under the condition of no defect sample, and the model training can be carried out along with the increase of the using time, and the using process can be simplified by introducing the model at the later stage.
The method and the device effectively solve the problem that the appearance defect detection of the factory mainboard is difficult to use deep learning model reasoning, accelerate the detection speed, save manpower and material resources and improve the production efficiency.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a method for detecting an appearance defect of a motherboard component according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a system for detecting an appearance defect of a motherboard component according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a computer device according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two entities with the same name but different names or different parameters, and it should be noted that "first" and "second" are merely for convenience of description and should not be construed as limitations of the embodiments of the present invention, and they are not described in any more detail in the following embodiments.
As shown in fig. 1, the present invention provides a method for detecting an appearance defect of a motherboard device, including:
s1, acquiring a high-definition image of a mainboard, preprocessing the high-definition image to obtain an image to be detected, comparing the image to be detected of the mainboard through a standard template image corresponding to the mainboard, and storing the image to be detected of the mainboard and a comparison result thereof;
s2, responding to the stored images to be detected of the main board and the comparison results thereof, accumulating to a preset number, and training an image recognition model through the stored images to be detected of the main board and the comparison results thereof; and
and S3, carrying out defect detection on the acquired to-be-detected image of the mainboard through the trained image recognition model.
In the embodiment of the present invention, when step S1 of the present invention is implemented, a high-definition camera is used to photograph a newly produced main board on a product line to form a high-definition image of the main board, and a series of processing manners such as rotational translation and the like are performed on the high-definition image of the main board to correct the main board in the high-definition image, and pixels such as an environment unrelated to the main board in the high-definition image are removed to generate an image to be detected of the main board.
And further detecting the difference, namely the defect, between the main board and the standard template in a mode of registering, matching and comparing the image to be detected of the main board with the standard template image corresponding to the main board, automatically marking the difference area as a defect area, and simultaneously saving the marked image to be detected of the main board and the detection result (if the marked area exists, the detection result is the defect main board, otherwise, the detection result is the normal main board) as a training sample.
In step S2, when the number of the defective motherboards and the normal motherboards automatically labeled in the manner of step S1 reaches a certain number, for example, 2000, or more, an artificial intelligent image recognition model is trained by the above-mentioned sample automatically labeled with the images to be detected of the plurality of motherboards.
In step S3, the main board on the subsequent production line is rapidly detected through the image recognition model trained in step S2. When the mainboard on the production line is detected by using the image recognition model, the picture of the mainboard shot by the industrial camera still needs to be preprocessed to form an image to be detected.
In some embodiments of the present invention, obtaining a high-definition image of a motherboard and preprocessing the high-definition image to obtain an image to be detected, and comparing the image to be detected of the motherboard with a standard motherboard image corresponding to the motherboard includes:
carrying out bit-wise subtraction on the image to be detected of the main board and the pixel corresponding to the standard template image corresponding to the main board to obtain a corresponding difference value, and judging whether the difference value is greater than a first threshold value or not;
and in response to the difference value being larger than a first threshold value, removing the isolated pixels of which the difference value is larger than the first threshold value through a median filtering algorithm, and automatically marking the area of which the difference value is larger than the first threshold value as a defect area.
In this embodiment, when the high-definition image of the acquired motherboard (the motherboard manufactured on the production line) is registered and compared with the image of the standard template of the motherboard of this type, the difference between the two images is calculated by subtracting the two images in a pixel-by-pixel manner. Specifically, three channel pixel values of R, G and B of the two frames of images are subtracted, the obtained maximum difference value is compared with a set threshold value, a pixel with large change is reserved, the reserved isolated pixel is removed through median filtering, and an area with obvious color change and large change range is used as a change area between the two frames of images, namely an abnormal area. For example, if the values of the RGB three-channel pixels are subtracted and then are all greater than 5, the value of the low pixel point is considered to be changed greatly, and if the values of the RGB three-channel pixels of the plurality of pixel points in the pixel range are subtracted and then are all greater than 5, and the area formed by the plurality of pixel points is converted according to the size of the motherboard and then is greater than the area of the minimum component on the motherboard, the area is considered to belong to an abnormal area.
And further automatically marking the area with obvious color change and large change range by adopting a general marking tool, and marking the main board as a defect sample.
It should be noted that, in the above embodiment, the image boundary of the acquired main board needs to be adjusted, and the size of the image to be detected, which is to be compared with the standard template, of the main board and the pixel content of the main board in the image are similar to the standard template. However, since the captured picture is taken by an industrial camera, the focusing and lighting conditions of the camera affect the content of the main board in the imaged picture. For example, in some cases, distances of the same device may be different in two captured main board images, where the difference does not refer to a real distance, but the number of rows or columns of pixels between two devices in two frame images is different, so that when comparing the to-be-detected image of the main board with the standard template, differences generated by bit subtraction due to differences of several rows or several different pixel differences need to be processed. To solve this problem, in some embodiments of the present invention, the difference is formed by ignoring, i.e. if the difference forms a difference of 1 row of pixels or 1 column of pixels, i.e. it cannot form a range of the area of the minimum device, the pixels of the difference can be completely ignored (there may be 1 row of pixels or two rows of pixels, i.e. according to the shape and size of the corresponding device), i.e. the difference resulting from the bitwise subtraction is not treated as a defect, but directly ignored.
In some embodiments of the present invention, obtaining a high definition image of a motherboard and preprocessing the high definition image to obtain an image to be detected, and comparing the image to be detected of the motherboard with a standard template image corresponding to the motherboard further includes:
generating a corresponding abstract picture by the image to be detected of the main board and the standard template image corresponding to the main board according to the difference of pixel values between adjacent images;
carrying out bit-by-bit subtraction on the generated abstract picture of the image to be detected of the main board and the generated standard template image corresponding to the main board to obtain a difference value of corresponding pixel bits;
removing the pixels with the difference value not being the first preset value through a median filtering algorithm, and judging whether the area of a region formed by the pixels with the difference value not being the first preset value is larger than a second preset value or not;
automatically marking the region as a defective region in response to the area of the region being greater than a second predetermined value.
In this embodiment, when the acquired high-definition image of the motherboard (the motherboard manufactured on the production line) is registered and compared with the image of the standard template of the motherboard of this type, the high-definition image of the motherboard and the standard image of the motherboard are made into abstract pieces for comparison. The method includes the steps that a mainboard image originally representing a real object is rendered into an abstract picture through the difference of pixels of adjacent high-definition images of the collected mainboard, for example, a chip on the mainboard is usually packaged into black, relevant information about the model of the chip with lighter color is engraved on the outer surface of a packaging layer of the chip through laser, when the image of the mainboard is abstracted in the mode, the color of an area where the chip is located is coated into the color of the chip, namely, the most colors in the appearance colors of different devices on the mainboard are used as the abstract colors of the devices according to the appearance colors of the different devices, for example, the chips are processed, if the chips are black, the black is used as the abstract colors of the area where the chip is located on the mainboard, therefore, the area can hardly be recognized after abstraction without referring to a design drawing, the abstract colors of the area are abstracted according to the most pixels of the area, the information of the corresponding devices can not be recognized in the abstracted image of the mainboard to be detected, the different devices represent different color areas, and the image to be detected at the moment is completely abstracted by a plurality of color spots and does not have the information of the real components on the mainboard.
Similarly, the image of the standard template used for comparing with the image to be detected of the main board is also subjected to the abstraction processing, so as to form an abstracted picture of the standard template with different color areas.
Further, subtracting the abstract picture of the image to be detected of the main board from the abstract picture of the standard template of the main board according to the position to obtain a corresponding difference value, judging whether the difference value is 0, if the difference value is 0, indicating that the pixel in the area belongs to a normal pixel point, and if not, considering that the pixel point is an abnormal pixel point. And meanwhile, removing the pixel points with the difference value not being 0 through a median filtering algorithm, and judging the area formed by the abnormal pixel points. The area where the abnormal pixel points are located is larger than the area of the mainboard minimum component and the area of the abstracted mainboard after the conversion of the image to be detected, and then the area is considered to be abnormal.
In some embodiments of the present invention, when the above bit-wise subtraction is performed, since the camera may have a difference (a template that is earlier) from the color of the standard template after abstracting the high-definition image of the captured main board due to the light environment or due to the influence of the service life of the camera, the bit-wise subtraction is not 0, and therefore, pixels within a certain range of pixel difference values may be considered as normal, that is, considered as 0.
In addition, in some cases, the abstract picture image of the latest normal image to be detected can be dynamically replaced as the abstract picture image of the standard template according to a certain time.
It should be noted that, in the abstracted main board, the image to be detected may also have a condition of a misalignment of 1 pixel or several pixels when two frames of pictures are compared in the above embodiment, and the corresponding scheme in the above embodiment may still be adopted to be ignored. And the situation that the difference value of smaller ranges or isolated pixels is different due to bit-wise subtraction brought by other characters or patterns on the component can be effectively avoided by adopting an abstract mode.
In some embodiments of the present invention, generating the corresponding abstract picture according to the difference between the pixel values of the adjacent images of the main board and the standard template image corresponding to the main board includes:
acquiring RGB channel values of corresponding pixels, and calculating Euclidean distances of the RGB channel values of the pixels and the adjacent pixels according to the RGB channel values of the pixels;
judging Euclidean distances of RGB channel values of the pixels and the pixels between adjacent pixels, dividing the pixels with the same Euclidean distances or the difference smaller than a second threshold value into the same area, and uniformly coloring the color of the area according to the color of the most pixels in the area.
In this embodiment, when abstracting the image to be detected of the motherboard and the standard template image corresponding to the motherboard, what is needed is to determine the range of the abstracted colors or what colors are divided into one region, that is, the region corresponding to the component. For example, when an image regarded as green by naked eyes is represented in the form of an RGB channel, if a color variation range is ignored, combinations of green can be more than ten thousands, that is, (0, 255, 0) in RGB represents green, (150, 222, 159) also represents green (light green), and especially when colors carried by a certain component on a mainboard are gradually varied, color values actually shot by a camera at the same place are different. Therefore, the difference of colors cannot be accurately determined by the single contrast of the values on the RGB channels. Therefore, the embodiment of the invention adopts a mode of calculating the Euclidean distance, namely, the numerical values on the three channels are uniformly calculated.
Specifically, the euclidean distance between a certain pixel and 8 surrounding neighboring pixels is calculated for each pixel, for example, if the RGB values of the certain pixel and the surrounding pixels are the same (up, down, left, right, and obliquely up, down (including 4 pixels in total), the euclidean distance between the certain pixel and the surrounding pixels is 0, and the calculation method can be regarded as:
Figure 803204DEST_PATH_IMAGE001
x1, y1, z1 respectively represent the values of the three RGB channels of the first pixel (current pixel), x2, y2, z2 represent the values of the three RGB channels of the second pixel (the other 8 pixels adjacent to the current pixel), and d represents the euclidean distance between two pixels on the three RGB color channels. The euclidean distance is calculated by comparing the value of any one pixel with the values of the other 8 pixels adjacent thereto, and the pixels having the euclidean distance smaller than 20 are divided into the same region.
Further, after dividing a specific pixel into corresponding regions, the RGB pixel values of the pixels of the region are counted, the square root is calculated for the sum of the pixel values of the same RBG or the pixels of three channels of a single pixel, and the similar pixels are taken as the pixels of the same color. And counting the number of pixel points of each color classification, taking the color with the largest number of pixel points as the main color of the region, and unifying the colors of other pixels in the region into the main color so as to finish abstract color representation of the region.
In some embodiments of the present invention, obtaining a high-definition image of a motherboard and preprocessing the high-definition image to obtain an image to be detected, and comparing the image to be detected of the motherboard with a standard motherboard image corresponding to the motherboard further includes:
generating a standard vector diagram of the main board according to the standard template image corresponding to the main board and the design drawing of the main board;
generating a vector diagram of the main board from the image to be detected of the main board according to the design drawing of the main board;
scaling the vector diagram of the main board and the standard vector diagram to a preset size, carrying out bitwise subtraction on the vector diagram of the main board scaled to the preset size and the standard vector diagram of the main board, and simultaneously judging whether the bitwise subtraction value of each bitwise pixel is a first preset value or not;
and in response to the bit-wise subtraction value of the pixel being a first preset value, marking the component area to which the pixel belongs as a defect area according to the design drawing of the mainboard.
In this embodiment, in order to solve the influence of the bit-wise subtraction on the calculation result caused by the displacement of a plurality of pixels of the main board position in the main board high-definition image due to the imaging of the camera, a method for converting the image to be detected of the main board into a vector diagram is provided. The method comprises the steps of determining the distance between corresponding components according to the size of a design drawing on the basis of the design imaging resolution and the design drawing, estimating the position of the corresponding component on an image to be detected of a mainboard, which is converted into the resolution of the image to be detected of the mainboard, on the basis of the resolution of the image to be detected of the mainboard, which is shot by a camera, and then abstracting the color of the corresponding area on the image to be detected according to the processing mode of the color on the mainboard in the embodiment (for example, the color of the area is classified according to RGB channel values of adjacent pixels, and the color with the largest number of pixel points is selected as the abstract color of the area).
Likewise, the standard template image for the motherboard is still processed and converted into a corresponding vector image in the same manner as described above.
Furthermore, after the image to be detected of the main board and the standard template image are both converted into vector diagrams, the image to be detected and the standard template image can be converted into images with corresponding resolution ratios according to the computing power, and then bitwise subtraction is carried out. Due to the characteristics of the vector image, even if the resolution ratio of the vector image is reduced, the distribution of components of the mainboard can still be represented, the distribution of the components of the mainboard cannot be influenced no matter how many times the design drawing of the circuit board is amplified in 3D software, and operations such as bitwise subtraction of the image to be detected and the like consume extremely high computing resources, so that the efficiency of registration and comparison can be effectively improved through the method.
In some embodiments of the present invention, when generating the vector diagram for the image to be detected of the motherboard, the motherboard design file may be exported into a design diagram with the same size as the image to be detected of the motherboard, and all colors on the design diagram are removed, and under the condition of the same size, the corresponding component unit on the design diagram is colored according to the colors on the image to be detected of the motherboard, and the vector diagram for the image to be detected of the motherboard is generated by using the colored design diagram as the vector diagram for the image to be detected of the motherboard. And compared with the vector diagram of the standard template after being reduced to a proper resolution.
It should be noted that, if the design file of the motherboard is obtained and the corresponding device in the design file can be inserted into the real appearance picture of the device, the vector diagram of the corresponding standard template can be directly derived through the design file, and a manner of manually selecting a qualified motherboard as the standard template of the motherboard to regenerate the vector diagram of the corresponding standard template is not needed at the beginning, thereby completely realizing automation.
In some embodiments of the present invention, training an image recognition model according to the saved image to be detected of the motherboard and the comparison result thereof includes:
decomposing the stored image to be detected of the main board into a plurality of local images to be detected, and associating the comparison result of the images to be detected with the corresponding local images to be detected;
and respectively and independently training corresponding image recognition models according to the local images to be detected and the corresponding comparison results.
In some embodiments of the present invention, the performing defect detection on the acquired to-be-detected image of the motherboard by using the trained image recognition model includes:
preprocessing the high-definition image of the mainboard into an image to be detected according to the acquired high-definition image of the mainboard in a preprocessing mode of the image to be detected for training the image recognition model;
decomposing the image to be detected into a plurality of local images to be detected according to the decomposition mode and respectively inputting the images to be detected into the plurality of image recognition models for detection;
and marking the defect detection result of the main board as a defect in response to the fact that the detection result of any one of the image recognition models is abnormal.
In this embodiment, it should be noted that resolution of the output image of the industrial camera adopted by the present invention is different according to different sizes of the main boards, in the actual production, the circuit board on the production line may be the main board of the server, and the main board of the server is different in size due to different versions, therefore, in order to clearly present the appearance layout of some large main boards, a higher resolution must be used, and therefore, in order to ensure image quality, it is preferable to process images with a resolution specification of 5000 × 5000 pixels on the main board of the server. However, no matter what kind of image recognition model is, the calculation of enough large pixels cannot be supported for image recognition, generally, most common image recognition engines support data with a size of 640 × 640 convolution, and for recognition of a server motherboard, such a small resolution ratio can hardly recognize corresponding small components, such as some smaller patch capacitors or some other smaller components such as some solder joints, which are already eliminated during data preprocessing of the image recognition algorithm model or when the image recognition algorithm model passes through the first convolution layer.
Therefore, in order to improve the accuracy of the image recognition model, the image to be detected is split into a plurality of small images to respectively and correspondingly train a plurality of models, for example, for the image to be detected with 5000 × 5000 pixels, the image to be detected is split into 49 blocks with 7 × 7, 49 image recognition models are respectively trained, and then the image is recognized through 49 models.
And when the image recognition model is used for detection, when a plurality of recognition models exist, the detection result of the main board is abnormal as long as 1 detection result is abnormal.
In addition, it should be noted that in some small-sized motherboards or circuit boards, for example, the resolution of the images to be detected can be completely reduced, and the identification can be quickly completed through an image identification model.
In addition, when the image recognition model is trained, a plurality of image recognition models may be trained after splitting the to-be-detected image of the motherboard, or the to-be-detected image of the motherboard may be trained after being subjected to abstraction processing and the like according to the above embodiment, and when the corresponding image recognition model is used for recognition, the to-be-detected image of the motherboard to be detected is subjected to the same abstraction processing and the like in the above manner, and then is recognized by the image recognition model.
As shown in fig. 2, another aspect of the present invention further provides a system for detecting an appearance defect of a motherboard component, including:
the image registration detection module 1 is configured to acquire a high-definition image of a main board and preprocess the high-definition image to obtain an image to be detected, compare the image to be detected of the main board with a standard template image corresponding to the main board, and store the image to be detected of the main board and a comparison result thereof;
the model training module 2 is configured to respond to the stored images to be detected of the main board and the comparison results thereof, accumulate the images to be detected of the main board and the comparison results thereof to a preset number, and train an image recognition model through the stored images to be detected of the main board and the comparison results thereof; and
and the model detection module 3 is configured to perform defect detection on the acquired to-be-detected image of the main board through the trained image recognition model.
As shown in fig. 3, another aspect of the present invention also provides a computer device, including:
at least one processor 21; and
a memory 22, said memory 22 storing computer instructions 23 executable on said processor 21, said instructions 23 when executed by said processor 21 implementing the steps of the method of any of the above embodiments.
As shown in fig. 4, a further aspect of the present invention also provides a computer-readable storage medium 401, where the computer-readable storage medium 401 stores a computer program 402, and the computer program 402 implements the steps of the method according to any one of the above embodiments when being executed by a processor.
According to the method, the system, the equipment and the medium for detecting the appearance defects of the mainboard components, the problem that deep learning model reasoning cannot be used due to the fact that defect samples are difficult to collect in the mainboard appearance defect detection process can be effectively solved, and due to the fact that the sizes of the mainboards are different and the types of the mainboard components are different in factory production line production, training of models is completed by collecting defect sample pictures of the components, and then defect detection is not practical. The method comprises the steps of firstly using a template for comparison to collect defect samples, collecting defect sample pictures while detecting defects, and then carrying out model training along with the increase of the sample pictures. The detection of any mainboard product can be used immediately under the condition of no defect sample, and the model training can be carried out along with the increase of the using time, and the using process can be simplified by introducing the model at the later stage.
The method and the device effectively solve the problem that the appearance defect detection of the factory mainboard is difficult to use deep learning model reasoning, accelerate the detection speed, save manpower and material resources and improve the production efficiency.
In addition, as can be seen from the exemplary embodiment of the present invention, the present invention can have a completely automatic implementation capability, and in some cases, only at the beginning of the design of the motherboard, the real object image of the corresponding component is added to the design file and exported to the corresponding picture, so that a completely automatic detection process can be implemented without human intervention.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of an embodiment of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.

Claims (8)

1. A method for detecting appearance defects of a mainboard component is characterized by comprising the following steps:
acquiring a high-definition image of a mainboard, preprocessing the high-definition image to obtain an image to be detected, comparing the image to be detected of the mainboard through a standard template image corresponding to the mainboard, and storing the image to be detected of the mainboard and a comparison result thereof;
responding to the stored images to be detected of the main board and comparison results thereof are accumulated to a preset number, and training an image recognition model through the stored images to be detected of the main board and comparison results thereof; and
carrying out defect detection on the acquired image to be detected of the mainboard through the trained image recognition model;
the method comprises the following steps of obtaining a high-definition image of a main board, preprocessing the high-definition image to obtain an image to be detected, and comparing the image to be detected of the main board through a standard template image corresponding to the main board, wherein the steps of:
generating a corresponding abstract picture by the image to be detected of the main board and the standard template image corresponding to the main board according to the difference of pixel values between adjacent images;
carrying out bit-wise subtraction on the generated abstract picture of the image to be detected of the main board and the generated standard template image corresponding to the main board to obtain a difference value of corresponding pixel bits;
removing the pixels with the difference value not being the first preset value through a median filtering algorithm, and judging whether the area of a region formed by the pixels with the difference value not being the first preset value is larger than a second preset value or not;
automatically marking the region as a defective region in response to the area of the region being greater than a second predetermined value; or
Generating a standard vector diagram of the main board according to the standard template image corresponding to the main board and the design drawing of the main board;
scaling the vector diagram and the standard vector diagram of the main board to a preset size, carrying out bitwise subtraction on the vector diagram of the main board scaled to the preset size and the standard vector diagram of the main board, and simultaneously judging whether the bitwise subtraction value of each bitwise pixel is a first preset value or not;
and in response to the bit-wise subtraction value of the pixel being a first preset value, marking the component area to which the pixel belongs as a defect area according to the design drawing of the mainboard.
2. The method according to claim 1, wherein the obtaining of the high-definition image of the main board and the preprocessing of the high-definition image are performed to obtain an image to be detected, and the comparing of the image to be detected of the main board with the standard template image corresponding to the main board comprises:
carrying out bit-wise subtraction on the image to be detected of the main board and the pixel corresponding to the standard template image corresponding to the main board to obtain a corresponding difference value, and judging whether the difference value is greater than a first threshold value or not;
and in response to the difference value being larger than a first threshold value, removing the isolated pixels with the difference value being larger than the first threshold value through a median filtering algorithm, and automatically marking the area with the difference value being larger than the first threshold value as a defect area.
3. The method according to claim 1, wherein the generating of the corresponding abstracted picture by the image to be detected of the main board and the standard template image corresponding to the main board according to the difference of the pixel values between the adjacent main boards comprises:
acquiring RGB channel values of corresponding pixels, and calculating Euclidean distances between the RGB channel values of the pixels and the RGB channel values of the adjacent pixels according to the RGB channel values of the pixels;
judging Euclidean distances of RGB channel values of the pixels and the pixels between adjacent pixels, dividing the pixels with the same Euclidean distances or the difference smaller than a second threshold value into the same area, and uniformly coloring the color of the area according to the color of the most pixels in the area.
4. The method according to claim 1, wherein the training of the image recognition model through the saved images to be detected of the main board and the comparison result thereof comprises:
decomposing the stored image to be detected of the main board into a plurality of local images to be detected, and associating the comparison result of the images to be detected with the corresponding local images to be detected;
and respectively and independently training corresponding image recognition models according to the local image to be detected and the corresponding comparison result.
5. The method according to claim 4, wherein the defect detection of the acquired image to be detected of the main board through the trained image recognition model comprises:
preprocessing the high-definition image of the mainboard according to the preprocessing mode of the image to be detected of the trained image recognition model of the acquired high-definition image of the mainboard to obtain the image to be detected;
decomposing the image to be detected into a plurality of local images to be detected according to the decomposition mode and respectively inputting the images to be detected into the image recognition model for detection;
and in response to the fact that the detection result of any one image recognition model is abnormal, marking the defect detection result of the mainboard as a defect.
6. The utility model provides a mainboard components and parts appearance defect detecting system which characterized in that includes:
the image registration detection module is configured to acquire a high-definition image of a main board and preprocess the high-definition image to obtain an image to be detected, compare the image to be detected of the main board with a standard template image corresponding to the main board, and store the image to be detected of the main board and a comparison result thereof;
the model training module is configured for responding to the stored images to be detected of the mainboard and the comparison results thereof, accumulating the images to be detected of the mainboard and the comparison results thereof to a preset number, and training an image recognition model through the stored images to be detected of the mainboard and the comparison results thereof; and
the model detection module is configured to perform defect detection on the acquired to-be-detected image of the mainboard through the trained image recognition model;
the method comprises the following steps of obtaining a high-definition image of a main board, preprocessing the high-definition image to obtain an image to be detected, and comparing the image to be detected of the main board through a standard template image corresponding to the main board, wherein the steps of:
generating a corresponding abstract picture by the image to be detected of the main board and the standard template image corresponding to the main board according to the difference of pixel values between adjacent images;
carrying out bit-wise subtraction on the generated abstract picture of the image to be detected of the main board and the generated standard template image corresponding to the main board to obtain a difference value of corresponding pixel bits;
removing the pixels of which the difference values are not the first preset value through a median filtering algorithm, and judging whether the area of a region formed by the pixels of which the difference values are not the first preset value is larger than a second preset value or not;
automatically marking the region as a defective region in response to the area of the region being greater than a second predetermined value; or
Generating a standard vector diagram of the main board according to the standard template image corresponding to the main board and the design drawing of the main board;
scaling the vector diagram and the standard vector diagram of the main board to a preset size, carrying out bitwise subtraction on the vector diagram of the main board scaled to the preset size and the standard vector diagram of the main board, and simultaneously judging whether the bitwise subtraction value of each bitwise pixel is a first preset value or not;
and in response to the bit-wise subtraction value of the pixel being a first preset value, marking the component area to which the pixel belongs as a defect area according to a design drawing of the mainboard.
7. A computer device, comprising:
at least one processor; and
a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of the method of any one of claims 1 to 5.
8. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 5.
CN202210171790.4A 2022-02-24 2022-02-24 Method, system, equipment and medium for detecting appearance defects of mainboard components Active CN114240939B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210171790.4A CN114240939B (en) 2022-02-24 2022-02-24 Method, system, equipment and medium for detecting appearance defects of mainboard components

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210171790.4A CN114240939B (en) 2022-02-24 2022-02-24 Method, system, equipment and medium for detecting appearance defects of mainboard components

Publications (2)

Publication Number Publication Date
CN114240939A CN114240939A (en) 2022-03-25
CN114240939B true CN114240939B (en) 2023-04-07

Family

ID=80748036

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210171790.4A Active CN114240939B (en) 2022-02-24 2022-02-24 Method, system, equipment and medium for detecting appearance defects of mainboard components

Country Status (1)

Country Link
CN (1) CN114240939B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114782437B (en) * 2022-06-20 2022-09-02 西南石油大学 Computer mainboard quality detection method and system based on artificial intelligence
CN115880288B (en) * 2023-02-21 2023-11-14 深圳市兆兴博拓科技股份有限公司 Detection method, system and computer equipment for electronic element welding
CN116075148B (en) * 2023-03-14 2023-06-20 四川易景智能终端有限公司 PCBA board production line intelligent supervision system based on artificial intelligence
CN115965646B (en) * 2023-03-16 2023-07-04 深圳思谋信息科技有限公司 Region dividing method, device, computer equipment and computer readable storage medium
CN117129480B (en) * 2023-10-25 2024-02-13 深圳市吉方工控有限公司 Intelligent detection method and device for computer main board components based on machine vision
CN117274258A (en) * 2023-11-21 2023-12-22 深圳市研盛芯控电子技术有限公司 Method, system, equipment and storage medium for detecting defects of main board image
CN117974662A (en) * 2024-03-29 2024-05-03 东莞市昌盛电子制品有限公司 Chip detection method, electronic equipment and storage medium

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140126839A1 (en) * 2012-11-08 2014-05-08 Sharp Laboratories Of America, Inc. Defect detection using joint alignment and defect extraction
CN105160654A (en) * 2015-07-09 2015-12-16 浙江工商大学 Towel label defect detecting method based on feature point extraction
CN112070747A (en) * 2020-09-09 2020-12-11 深兰人工智能芯片研究院(江苏)有限公司 LED lamp bead defect detection method and device

Also Published As

Publication number Publication date
CN114240939A (en) 2022-03-25

Similar Documents

Publication Publication Date Title
CN114240939B (en) Method, system, equipment and medium for detecting appearance defects of mainboard components
CN111179251B (en) Defect detection system and method based on twin neural network and by utilizing template comparison
CN111612763B (en) Mobile phone screen defect detection method, device and system, computer equipment and medium
CN110060237B (en) Fault detection method, device, equipment and system
CN108562589A (en) A method of magnetic circuit material surface defect is detected
CN109767445B (en) High-precision PCB defect intelligent detection method
WO2022236876A1 (en) Cellophane defect recognition method, system and apparatus, and storage medium
CN111815564B (en) Method and device for detecting silk ingots and silk ingot sorting system
CN111738994B (en) Lightweight PCB defect detection method
CN111445459A (en) Image defect detection method and system based on depth twin network
CN110533654A (en) The method for detecting abnormality and device of components
CN113989794B (en) License plate detection and recognition method
CN115661161B (en) Defect detection method, device, storage medium, apparatus and program product for parts
CN113780484B (en) Industrial product defect detection method and device
CN115578585A (en) Industrial image anomaly detection method, system, computer device and storage medium
CN111882547A (en) PCB missing part detection method based on neural network
CN114187247A (en) Ampoule bottle printing character defect detection method based on image registration
CN117078677A (en) Defect detection method and system for starting sheet
CN105354833A (en) Shadow detection method and apparatus
CN112861861B (en) Method and device for recognizing nixie tube text and electronic equipment
CN111935480B (en) Detection method for image acquisition device and related device
CN115410184A (en) Target detection license plate recognition method based on deep neural network
CN114708247A (en) Cigarette case packaging defect identification method and device based on deep learning
CN115330688A (en) Image anomaly detection method considering tag uncertainty
CN113592789A (en) Dim light image identification method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant