TWI712991B - Defect detection method and device with both automatic optical detection and artificial intelligence detection functions - Google Patents

Abstract

A defect detection method with both automatic optical detection and artificial intelligence detection functions and its device It is mainly based on first taking the image of the object to be measured, and then using the individual strengths and weaknesses of Automated Optical Inspection (AOI) and Artificial Intelligence (AI) detectors. Its pre- and post-procedures are combined to establish methods to complement its shortcomings, so that the missed detection rate can still be maintained at the lowest level, and it has the advantage of maintaining the lowest false detection rate.

Description

Defect detection method with both automatic optical detection and artificial intelligence detection function and Its device

The present invention relates to a defect detection method and device with both automatic optical detection and artificial intelligence detection functions; applied to optical detection, especially a method of through automatic optical inspection (Automated Optical Inspection, referred to as AOI) and artificial intelligence (Artificial Intelligence, Abbreviated as AI) The two detectors can have multiple and repeatable combinations before and after the detector to establish a method and device for detecting.

Press; At present, there are two types of general defect detection methods. One is Automated Optical Inspection (AOI), which is a high-speed and high-precision optical image inspection system that uses machine vision to cooperate with the geometric/grayscale value conditions set by the engineer as the inspection Standard technology, as an improvement to the shortcomings of human use of optical instruments for detection; the other is the use of artificial intelligence (Artificial Intelligence, abbreviated as AI) detection, which means that the machine uses deep learning to present image interpretation similar to human intelligence to achieve The purpose of the test.

The calculation method of the former Automated Optical Inspection (AOI) is the following: 1. Binarization is the simplest method of image segmentation. Binarization can convert grayscale images. To form a binary image, set the grayscale of pixels greater than a certain critical grayscale value to the maximum grayscale value, and set the grayscale of the pixel less than this value to the minimum grayscale value; 2. Histogram Histogram statistics is a graphical representation of the data distribution. It is a two-dimensional statistical chart. Its two coordinates are the measurement of a statistical sample and a certain attribute corresponding to the sample; 3. Edge detection (Edge detection) is the use of image processing and computer vision capabilities to identify points with obvious brightness changes in digital images. Significant changes in image attributes usually reflect the characteristics of the attributes. These detection features include (i) the difference in depth Continuous, (ii) discontinuous surface orientation, (iii) changes in material properties, and (iv) changes in scene lighting; fourth, object segmentation, the object is segmented through automatic analysis of video content. There are many analysis methods. For example, you can use the image with the mean value removed to retain the image texture information and reduce the characteristics of light source changes and shadows affecting the background, or use the set grayscale threshold or the grayscale differential threshold to segment the object. The purpose of dividing objects quickly and effectively.

Automated Optical Inspection (AOI) can set the required tightness by itself, so this method has the advantage of low missed detection rate, but at the same time, it will also cause over kill or error. The lack of detection rate is too high. Therefore, in the current general conventional technology, it is impossible to have the basic shortcomings that it is impossible to balance the two in the automatic optical inspection (Automated Optical Inspection, referred to as AOI) method.

In addition, the detection method of Artificial Intelligence (AI) mainly uses deep learning, and its calculation method adopts: 1. Object Detection, using Convolutional Neural Network (Convolutional Neural Network) Neural Network (CNN for short), classifies images (Image classification), and uses Neural Network (NN for short) to solve another problem. Classification with localization is to put the subject matter into the bounding box ( bounding box, detect all multiple objects and localize them; second, the classifier, through the linear combination of features to gather objects with similar characteristics, and make classification decisions. The characteristics of the object are usually described as characteristic values, while In the vector, it is described as a feature vector. There are generative models and discriminative models. The former uses the principle of modelized conditional probability, and the latter discriminates models (discriminative models). This method is to try to maximize the output value of a training set.

Artificial Intelligence (AI) has deep learning and imitates human intelligence. The advantage of using the above method is that it adapts better to the environment and the overall accuracy is higher. The disadvantage is that if you use deep learning only In the object detection method, the missed detection rate cannot be as low as the AOI, which will lead to the risk of defective products entering the market. However, simply using the classifier in deep learning can not locate the position of the defect, and often the defect part accounts for the overall image The ratio of the area is too small, resulting in poor accuracy. Therefore, there are many difficulties and problems that must be overcome.

Therefore, the inventor of the present case has, after years of painstaking research and development, in view of the deficiencies of Xi’s technology, proposes a defect detection method and device with both automatic optical detection and artificial intelligence detection functions in this case as described below.

In view of the shortcomings caused by the above-mentioned conventional technology, the main purpose of a defect detection method and device with both automatic optical detection and artificial intelligence detection functions of the present invention is to apply traditional automated optical inspection (Automated Optical Inspection, referred to as AOI) and Artificial Intelligence (AI) detectors have individual strengths and weaknesses. Through the combined process of the two, they complement their shortcomings, so that the missed detection rate can still be kept to the lowest, and the false detection rate is also the lowest. Complete effect requirements.

Another purpose of the present invention is a defect detection method and device with both automatic optical detection and artificial intelligence detection functions. Inspection, abbreviated as AOI) and artificial intelligence (Artificial Intelligence, abbreviated as AI) detectors are combined with multiple and repeatable procedures to establish various inspection methods, which have a wider range of applications.

In order to achieve the above and other objectives, the present invention is a defect detection method and device with both automatic optical inspection and artificial intelligence detection functions, which are suitable for optical inspection, especially a method through automated optical inspection (AOI) and Artificial Intelligence (AI) detectors can be multiple and repeatable combinations before and after two detectors to establish a method and device for detecting.

The present invention is a defect detection method and device with both automatic optical detection and artificial intelligence detection functions. The implementation method includes: firstly, a camera fixture and a movable mechanism combined with a light source device are used to apply multiple lighting methods to Provide different light source control methods to switch each group of light sources, edit the intensity and angle of the light source, and combine with the camera's repeat shooting function to take images, and then use Automated Optical Inspection (AOI) and Artificial Intelligence (Artificial Intelligence, Abbreviated as AI) The individual strengths and weaknesses of the two detectors can be detected through multiple and repeatable combinations of the preceding and following procedures.

When the detection object is mainly used for the detection of relatively simple image background, such as the detection of the frame defect of a smart phone, the object to be measured is first captured, and then the automatic optical detection AOI module is used to detect the information after the object is captured. Perform automated optical inspection AOI algorithm to set and find the ROI (Region of Interest) and store it in the database, then read the ROI from the database and input artificial intelligence The AI classifier is detected for detection, and the artificial intelligence AI detector can filter various types that should not be classified as defects, such as dust, to obtain classification results, and then output the classification results into the database.

When the detection object is mainly used in the image background is relatively complicated, but the defect area needs to be accurately defined, such as the detection of wood grain surfaces, first take an image of the object to be measured, and then use an artificial intelligence AI detector to take the object. Apply an object detection algorithm to find out the bounding box of the flaw, output it to the database after the flaw is classified, and then read the image in the bounding box from the database for automation The optical inspection AOI module divides the object, and then outputs the result of the division into the database.

The present invention is a defect detection method and device with both automatic optical detection and artificial intelligence detection functions. The device mainly includes a loading and collecting platform for placing the object to be tested; a light source device, which is set on the loading The upper periphery of the collection platform is equipped with various sets of light source fixtures such as coaxial light source, line light source, backlight source, ring light source, spherical light source, etc., combined with a movable mechanism such as a mechanical arm and a light source photography control module for multiple lighting The method is to provide different light source control methods to switch each group of light sources, edit the intensity and angle of the light sources; a camera unit, the camera unit and the light source device are staggered on the upper periphery of the loading and collection platform, including a surface camera fixture, a line Camera jigs and other application camera jigs combined with the camera's repeat shooting function; movable mechanism, which can be a lifting mechanism conveying mechanism or a mechanical arm, etc.; automatic optical inspection AOI module, which is connected to the camera unit and The database is used to receive the information of the object to be measured after the image is taken or reconnected, and to perform automatic optical detection AOI algorithm, etc.; the artificial intelligence AI detector is connected to the camera unit and the database to receive the object to be measured for retrieval After the image or subsequent information and apply object detection algorithms, etc.; the database is linked to the automated optical inspection AOI module and artificial intelligence AI detector to retrieve the object under test. The result of image detection is stored in the database, and information is provided for connection detection or other applications.

Therefore, in the present invention, a defect detection method and device with both automatic optical detection and artificial intelligence detection functions are mainly combined with a light source device by a camera fixture and a movable mechanism. After applying multiple lighting methods to provide different light source control methods, switch each group of light sources, edit the intensity and angle of the light source, and combine with the camera's repeat shooting function to achieve rapid data collection, and then use automated optical detection (Automated The individual strengths and weaknesses of Optical Inspection (AOI) and Artificial Intelligence (AI) detectors are tested through a combination of procedures before and after it is established to complement their shortcomings, so that the missed detection rate can still be maintained Under the premise of the lowest, it also has the advantage of maintaining the lowest false detection rate, so that the detection object is mainly used for the detection of relatively simple image background, such as the detection of smartphone frame defects, the automatic optical inspection AOI module performs first The optical detection AOI algorithm can ensure that the missed detection rate is low enough to prevent defective products from entering the market, and the artificial intelligence detector AI in the next step of the detection process can effectively reduce the false detection rate and avoid unnecessary losses. A detection process that takes into account the advantages of both, can make the entire product detection process have the functions of low missed detection rate and low false detection rate at the same time, and when the detection object is mainly used in the image background, it is relatively complex but the defect area needs When accurately defining the detection of wood grain surfaces, for example, the artificial intelligence AI detector has a high background adaptability through the detection process, and classifies and outputs the bounding box of the defect and the defect. This is an automated optical inspection AOI module If the automatic optical inspection AOI module algorithm cannot be used directly, let the artificial intelligence AI detector find out the bounding box of the defect and then output the database and then read the data by the automatic optical inspection AOI module Object segmentation, because the automated optical inspection AOI module uses the AOI algorithm to accurately segment the defect area and calculate the defect area, which makes up for the lack of the AI algorithm used by the artificial intelligence AI detector to accurately calculate the area. Advantages, thus becoming an effective creator of the invention.

1: Loading the collection platform

2: Light source device

20: Coaxial light source

21: Line light source

3: camera unit

4: Automated optical inspection AOI module

5: Movable mechanism

6: Artificial intelligence AI detector

7: Database

The first figure is a flowchart of the implementation method of the present invention.

The second figure is a flow chart when the detection object of the present invention is mainly used for relatively simple detection of the image background.

The third figure is a flow chart of the present invention when the detection object is mainly used for the detection of relatively complex image background.

The fourth figure is a side view of the structure of the device of the present invention.

The fifth figure is a block diagram of the present invention.

The following is a specific example to illustrate the implementation of the present invention. Those familiar with the art can easily understand the other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied by other different specific examples. The details in this specification can also be based on different viewpoints and applications, and various modifications and changes can be made without departing from the spirit of the present invention.

First of all, please refer to the first, second, and third figures, as shown in the remaining figures, the present invention is a defect detection method and device with both automatic optical detection and artificial intelligence detection functions, and its implementation method The main system includes: first image acquisition, which means that the camera unit 3 is used to image the object to be measured, and the camera unit 3 or a movable mechanism 5 such as a robotic arm is combined with the light source device 2 and the light source photography control module 4 to apply multiple numbers. Sub-lighting method to provide different light source control methods to switch each group of light sources, edit the intensity and angle of the light source, and combine with the camera's repeat shooting function to achieve the function of rapid image acquisition; adjust the inspection sequence, according to the required inspection object, and automatic optics Detect the individual strengths and weaknesses of AOI module 4 and artificial intelligence AI detector 6, choose this automated optical inspection AOI The module 4 and the artificial intelligence AI detector 6 can be combined with multiple and repeatable procedures before and after the detection to establish a method for detection; the first detection is performed, and the first detection is performed according to the order of adjusting the detection order.

The first inspection data is input into database 7, which refers to the output of the first detected data made by the automated optical inspection AOI module 4 or artificial intelligence AI detector 6 and stored in the database 7, for subsequent inspection or use Storage: The data extracted from the database 7 for connection detection means that the automated optical inspection AOI module 4 or artificial intelligence AI detector 6 extracts the data that needs to be detected from the database 7 for the use of connection data detection.

The data after the continuous detection is then input into the database 7, which means that the data continuously detected by the automated optical detection AOI module 4 or artificial intelligence AI detector 6 is output and stored in the database 7 for use or storage for subsequent detection.

Among them (as shown in the second figure), when the detection object is mainly used for the detection of relatively simple image background, such as the detection of the frame defect of a smartphone, the object to be measured is first captured; then the AOI module is automatically detected by optical 4 Perform automatic optical detection AOI algorithm for the information detection after the object is taken to set and find the detection range (Region of Interest, ROI) and output it to the database 7; then from the database The detection range selected by reading the circle in 7 is input to the artificial intelligence AI detector 6 for detection, and the artificial intelligence AI detector 6 can filter various types that should not be classified as defects, such as dust, to obtain classification results , And then output the classification result into the database 7.

Among them (as shown in the third figure), when the detection object is mainly used in the image background is relatively complex, but the defect area needs to be accurately defined, such as the detection of wood grain surfaces, first take an image of the object to be measured; then use The artificial intelligence AI detector 6 performs object detection on the object after taking an image The algorithm finds out the bounding box of the flaw, and outputs it and stores it in the database 7 after the flaw is classified; then reads the image in the bounding box from the database 7 to automatically detect the AOI module 4 for the object Divide, and then output the result of the division into the database 7.

Please refer to the fourth and fifth pictures with the remaining pictures; the present invention is a defect detection method and device with both automatic optical detection and artificial intelligence detection functions. The concept of this method is designed to provide: load collection Platform 1, for placing the object to be tested; light source device 2, the light source device 2 is installed on the upper periphery of the loading and collecting platform 1, equipped with coaxial light source 20, line light source 21, backlight source, ring light source, spherical light source and other groups Light source fixture, combined with a movable mechanism 5 such as a robotic arm and a light source photography control module 4 to provide multiple lighting methods to provide different light source control methods to switch each group of light sources, and edit the intensity and angle of the light source; camera Unit 3, the camera unit 3 and the light source device 2 are interleaved on the upper periphery of the loading and collection platform 1, including camera fixtures, line camera fixtures and other application camera fixtures combined with the camera's repeat shooting function to achieve rapid collection Data function; movable mechanism 5, the movable mechanism 5 can be a lifting mechanism conveying mechanism or a mechanical arm, etc.; automatic optical inspection AOI module 4, which is connected to the camera unit 3 and database 7 to receive the test After the object is captured or re-connected, it is used for automatic optical detection AOI algorithm, etc.; the smart AI detector 6 is built into the camera unit 3 and the database 7 to receive the object to be tested for after the capture Or follow the information and apply the object detection algorithm, etc.; the database 7 is connected to the automated optical inspection AOI module 4 and artificial intelligence AI detector 6 respectively to retrieve the object under test. The results of image detection are stored in the database 7 and information is provided to Connection detection or other application terminal use.

Therefore, in the present invention, a defect detection method and device with both automatic optical detection and artificial intelligence detection functions are mainly implemented by using the camera fixture 3 and the movable mechanism 5 in combination with the light source device 2 to perform multiple lighting methods. To provide different light source control methods to switch each group of light sources, edit the intensity and angle of the light source, and combine with the camera's repeat shooting function to achieve rapid data collection, and then use the automatic optical inspection AOI module 4 and artificial intelligence AI detector 6 The individual strengths and weaknesses of the individual can be tested through the combination of pre- and post-procedures and established methods to complement their shortcomings, so that the missed detection rate can still be kept to a minimum, and the false detection rate can be maintained at the same time. The object is mainly used for the detection of relatively simple image background, such as the detection of flaws in the frame of a smartphone. The automatic optical inspection AOI algorithm carried out by the automatic optical inspection AOI module 4 can ensure a sufficiently low rate of missed inspections and avoid defective products The artificial intelligence AI detector 6, which is the next step in the inspection process, can effectively reduce the false detection rate and avoid unnecessary losses. It is a inspection process that takes into account the advantages of both, which can make the entire product inspection process at the same time It has the function of low missed detection rate and low false detection rate; and when the detection object is mainly used in the image background is relatively complex but the defect area needs to be accurately defined, such as the detection of wood grain surfaces, it can be detected by the artificial intelligence The detection process of the device 6 has high background adaptability, and it classifies and outputs the bounding box of the flaw and the flaw. This is something that the automatic optical inspection AOI module 4 cannot directly use the automatic optical inspection AOI module algorithm. The artificial intelligence AI detector 6 finds the bounding box of the defect and then outputs the database 7 and then the automatic optical inspection AOI module 4 reads the data and performs object segmentation, because the automatic optical inspection AOI module 4 uses The AOI algorithm can accurately segment the defect area and calculate the defect area, which complements the advantage of the AI algorithm used by the artificial intelligence AI detector 6 that cannot accurately calculate the area, thus becoming an effective creator of the present invention. .

Claims (4)

A defect detection method with both automatic optical detection and artificial intelligence detection functions. The main system is: first image acquisition, which means that the camera unit is used to image the object to be measured, and the camera unit or movable mechanism is used to combine the light source device and the light source photography control module , Apply a variety of lighting methods to provide different light source control methods to switch each group of light sources, edit the intensity and angle of the light source, and combine the camera to repeat the shooting function; adjust the inspection sequence, and select the automatic optical inspection AOI module according to your needs Pre- and post-detection procedures with artificial intelligence AI detectors; for the first test, the first test is performed according to the sequence of adjusting the test sequence; the first test data is entered into the database, which refers to the automated optical inspection AOI module or artificial intelligence The data output by the AI detector for the first detection is stored in the database for subsequent use or storage; the data extracted from the database for continuous detection refers to the automated optical detection AOI module or artificial intelligence AI detection The device extracts the data that needs to be inspected from the database for subsequent data inspection use; the data after the continued inspection is entered into the database, which refers to the data continuously detected by the automatic optical inspection AOI module or artificial intelligence AI detector Export and store in the database for subsequent testing or storage. For example, the defect detection method with both automatic optical detection and artificial intelligence detection functions described in the scope of patent application 1, wherein the detection may include: first taking an image of the object to be tested; Then use the automatic optical detection AOI module to detect the information after the object is captured and perform the automatic optical detection AOI algorithm to set the detection range (Region of Interest, ROI) for finding the circle and output it to the database. Middle; then read the circled detection range from the database and input it into the artificial intelligence AI detector for detection, and the artificial intelligence AI detector can filter out various types that should not be classified as defects to obtain classification results , And then output the classification result into the database. For example, the defect detection method with both automatic optical detection and artificial intelligence detection functions described in the scope of patent application 1, wherein the detection may include: first taking an image of the object to be tested; then using an artificial intelligence AI detector to take an image of the object Afterwards, an object detection algorithm is applied to find the bounding box of the defect, and after the defect is classified, it is output and stored in the database; then the image in the bounding box is read from the database to The automatic optical inspection AOI module performs object segmentation, and then outputs the result of the segmentation and stores it in the database. A defect detection device with both automated optical detection and artificial intelligence detection functions. It includes: a loading and collecting platform for placing the object to be tested; a light source device, which is arranged on the upper periphery of the loading and collecting platform and has a coaxial Light source, line light source, backlight source, ring light source, spherical light source and other five groups of light source fixtures, combined with a movable mechanism and light source photography control module to provide multiple lighting methods to provide different light source control methods switch each Group of light sources, editable light source intensity and angle; camera unit, the camera unit and the light source device are interlaced on the upper periphery of the loading and collection platform, including two application camera fixtures: camera fixtures and line camera fixtures And combined with the camera repeat shooting function; Movable mechanism, the movable mechanism can be two kinds of movable mechanisms, such as a lifting conveying mechanism or a robotic arm; an automatic optical inspection AOI module, which is connected to the camera unit and the database to receive the object to be measured for image acquisition Or follow the information and perform automatic optical detection AOI algorithm; artificial intelligence AI detector is connected to the camera unit and database to receive the information of the object to be measured for acquisition or subsequent information and perform object detection The object detection algorithm; the database is linked to the automated optical inspection AOI module and the artificial intelligence AI detector respectively to store the result of the image acquisition and inspection of the object to be tested into the database unit, and Provide information for connection detection or other applications.

Images