TW202127371A - Image-based defect detection method and computer readable medium thereof - Google Patents

Abstract

The present invention relates to an image-based defect detection method, comprising: obtaining at least an image of an object to be detected, extracting a plurality f target defect sub-areas from the image, determining defect type of the plurality of defect sub-areas by a first processing method, generating at least a target defect area from the plurality of defect sub-areas by a second image processing method, determining a first defect level of the target defect area according to a first rule, and storing the first defect level. The present invention uses the first processing method to determine the defect type of the plurality of defect sub-areas extracted from the image of the object to be detected , uses the second processing method to generate at least an object defect area from the plurality of target defect sub-areas, and determines the first defect level according to a first rule so as to implement the defect determination of the object to be detected.

Description

Image-based defect detection method and computer readable storage medium

The invention relates to an image-based defect detection method and a computer-readable storage medium.

The current product manufacturing industry is developing towards high precision and high quality, making precision parts prone to collisions, crushing, scratches and other defects after processing, and the size of the defects reaches the micron level. At present, a large amount of inspection manpower is required to detect the above defects. However, the inspectors need to train for a long time to accurately distinguish the defect types and grades, and the inspectors are susceptible to subjective consciousness, emotions, eyesight, and fatigue, resulting in unstable judgment accuracy. Although the use of automatic optical inspection equipment (Automated Optical Inspection, AOI) to take images and detect defects based on image processing technology can reduce a lot of labor input, traditional machine vision appearance inspection based on image processing technology is very Defects, it is impossible to give an accurate classification and grade judgment.

In view of the above, it is necessary to propose an image-based defect detection method and computer-readable storage medium to accurately determine the appearance of the product.

The first aspect of the present application provides an image-based defect detection method, which is suitable for detecting the appearance defect of an object to be tested, and the image defect detection method includes: Acquire at least one image of the object to be measured; Extracting multiple target defect sub-regions from the image; Judging the defect types of the multiple target defect sub-regions by using the first processing method; Generating at least one target defect area from the plurality of target defect sub-areas by using the second processing method; Judging the first defect level of the target defect area according to the first criterion; The first defect level is stored.

Preferably, the method further includes: Judging whether the first defect level meets a preset condition; When the first defect level does not meet a preset condition, judging the second defect level of the target defect area according to a second criterion; The second defect level is stored.

Preferably, the method further comprises: judging whether the target defect area has a defect according to the first defect level and/or the second defect level.

Preferably, the preset condition is that the first defect level belongs to a preset level.

Preferably, the step of judging the second flaw level of the target flaw area according to the second criterion further includes: Extracting a plurality of first feature values of the target defect area; Converting the plurality of first characteristic values into second characteristic values in a preset format; A third processing method is used to process the second characteristic value to obtain the second defect level.

Preferably, the first feature value can be any combination of size, grayscale, texture, position, and direction, the preset format is an image format, and the second feature value is converted from the first feature value. Composition feature map.

Preferably, the third processing method is a deep learning algorithm.

Preferably, the step of judging the first flaw level of the target flaw area according to the first criterion includes: Calculating the area size of the target defect area; The first flaw level is given according to the area size of the target flaw area.

Preferably, the step of judging the first flaw level of the target flaw area according to the first criterion includes: Judging the attention level of the target flaw area according to the flaw type of the target flaw area; Calculating the defect value of the target defect area according to the attention level; The first defect level of the target defect area is obtained according to the defect value and at least one preset threshold value.

Preferably, the step of extracting multiple target defect sub-regions from the image further includes: Pre-processing the image to extract multiple predicted defect positions; Frame selection of multiple defect sub-regions according to the multiple predicted defect positions; Selecting multiple target defect sub-areas from the multiple defect sub-areas according to the size.

Preferably, the step of pre-processing the image to extract multiple predicted defect positions further includes: Extracting multiple regions of interest from the image; Extracting the plurality of predicted defect positions from the plurality of regions of interest by using a fourth processing method; At least two adjacent ones of the plurality of predicted defect positions are aggregated.

Preferably, the fourth processing method is a semantic segmentation algorithm.

Preferably, the step of judging the defect types of the plurality of target defect sub-regions by using the first processing method further includes: The convolutional neural network model is used to determine the defect types of the multiple target defect sub-regions.

Preferably, the step of generating at least one target defect area from the plurality of target defect sub-areas by using the second processing method further includes: According to the defect types and positions of the multiple target defect sub-areas, the target defect area is generated by grouping one or more of the target defect sub-areas with the same type and adjacent positions.

A second aspect of the present application provides a computer-readable storage medium on which a computer program is stored, wherein the computer program is characterized in that the image-based defect detection method is implemented when the computer program is executed by a processor.

The present invention uses the first processing method to determine the defect types of the multiple target defect sub-regions extracted from the image of the object to be measured, and uses the second processing method to generate at least one target defect from the multiple target defect sub-regions Area, and judging the first defect level of the target defect area according to the first criterion, so as to realize the accurate judgment of the defect of the object to be measured.

In order to be able to understand the above objectives, features and advantages of the present invention more clearly, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the application and the features in the embodiments can be combined with each other if there is no conflict.

In the following description, many specific details are explained in order to fully understand the present invention. The described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments, and are not intended to limit the present invention.

Preferably, the appearance defect detection method based on deep learning of the present invention is applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculation and/or information processing in accordance with pre-set or stored instructions. Its hardware includes, but is not limited to, a microprocessor and a dedicated integrated circuit (ASIC) , Field-Programmable Gate Array (FPGA), Digital Signal Processor (DSP), embedded devices, etc.

The electronic device may be a computing device such as a desktop computer, a notebook computer, a tablet computer, and a cloud server. The device can interact with the user through a keyboard, a mouse, a remote control, a touch pad, or a voice control device.

Figures 1A and 1B illustrate that the image of the object to be tested may contain multiple flaws. The existing image processing method combined with the intelligent algorithm can be used to extract the relevant information of the flaw and make a more accurate judgment. Through the related information of the defect, the corresponding defect level can be given according to different defect types. Since there may be various types and shapes of defects in the object to be tested, the characteristics of each type of defect in the image are not the same. A single method cannot accurately determine all possible defects or concerns. The present invention first processes the image in small areas to find the possible positions of the flaws, classifies the small areas with flaws, and then aggregates the small flaws that may actually belong to the same flaw into a large flaw after the classification. All types of large defects are classified. If this type of defect is more complicated and cannot be simply judged, you can use other judgment criteria to make a second judgment for the large defect with doubts about the defect level.

Please refer to FIG. 2, which shows a flowchart of an image-based defect detection method in an embodiment of the present invention. In this embodiment, after the defect area in the image is extracted, the defect area is first classified to obtain the defect type, and then the first criterion is used to initially judge the level of the defect area according to the defect type, wherein the first criterion only uses The known feature value of the image determines the level of the defect area.

Referring to FIG. 2, the image-based defect detection method specifically includes the following steps.

Step S21: Acquire at least one image of the object to be measured.

In this embodiment, acquiring at least one image of the object to be measured includes: acquiring at least one image of the object to be measured taken by a camera, where the camera may be a line scan camera or an area scan camera. In this embodiment, the object to be measured is a device such as a mobile phone or a tablet computer. In another embodiment, acquiring at least one image of the object to be measured includes: receiving at least one image of the object to be measured transmitted by the server. In other embodiments, at least one image of the object to be measured can be obtained from a local database. In this embodiment, the image may include a complete or partial image of the object to be measured. The image can be of any resolution, or it can be high-sampled or low-sampled, depending on actual needs.

Step S22: Extract multiple target defect sub-regions from the image.

In one embodiment, the step of extracting multiple target defect sub-regions from the image further includes: pre-processing the image to extract multiple predicted defect locations, and frame selection of multiple defect sub-regions according to the multiple predicted defect locations, Select multiple target defect sub-areas from multiple defect sub-areas according to the size. In an embodiment, pre-processing the image to extract multiple predicted defect locations further includes: extracting multiple regions of interest from the image, and extracting multiple predicted defect locations from the multiple regions of interest using the fourth processing method, Aggregating at least two adjacent ones of the multiple predicted defect positions to obtain multiple target defect sub-regions. In this embodiment, the fourth processing algorithm is a semantic segmentation algorithm.

In an embodiment, multiple regions of interest can be extracted from the image according to a Region of Interest (ROI) algorithm. Use semantic segmentation algorithm to predict and process multiple regions of interest and output background primitive points and flawed primitive points in multiple regions of interest, and binarize the background primitive points and flawed primitive points according to the binarization Separate the defect image element points in the multiple regions of interest, obtain multiple predicted defect locations according to the separated defect image element points in the multiple interest areas, and aggregate at least adjacent ones of the multiple predicted defect locations Two get multiple target defect sub-regions.

In one embodiment, binarizing the background image element point and the defect image element point and separating the defect image element points in the multiple interest regions according to the binarized image element points includes: The gray level of the image element point is set to 0 or 255 to binarize the gray level of the image element points in the multiple regions of interest. A pixel point with a gray value of 255 is used as a defective pixel point, and a pixel point with a gray value of 0 is used as a background pixel point. In a specific embodiment, the gray levels of the pixel points in multiple regions of interest can be grouped by the k-means clustering method to obtain two groups, and then the gray level binary values of the pixel points in the two groups can be obtained. And the gray values of the binarized pixel points in each group are the same. Then compare the gray values of the pixel points in the multiple regions of interest with the preset threshold, set the gray value of the pixel points greater than the preset threshold to 255, and set the pixel points not to be greater than the preset threshold The gray value of is set to 0. Among them, the preset threshold can be set according to the needs of the user.

In one embodiment, obtaining multiple predicted defect locations according to the separated defect image element points in the multiple interest regions includes: filtering out non-defect image element points in the multiple interest areas, and correcting the defects in the multiple interest areas. The pixel points are clustered to obtain multiple defect blocks, a rectangular area is selected as the defect area of each defect block through the boundary box of each defect block, and the coordinates of the defect area of each defect block are determined, and each defect block is represented by the defect map The element points are clustered, and a plurality of predicted defect locations are obtained according to the coordinates of the defect area of each defect block, wherein each predicted defect location corresponds to the defect area of a defect block.

In one embodiment, selecting multiple defect sub-regions based on multiple predicted defect position frames includes: selecting multiple defect sub-regions based on coordinate boxes of the defect area. In this embodiment, selecting multiple defect sub-areas according to the coordinate frame of each defect area includes: establishing a Cartesian coordinate system in the image with the point on the upper left corner of the image as the origin, where the X direction of the Cartesian coordinate system represents The width of the image, the Y direction of the Cartesian coordinate system represents the height of the image. In the Cartesian coordinate system, the x coordinate corresponding to the leftmost primitive point of each defective block is regarded as the left boundary of the defective block, and the x coordinate corresponding to the rightmost primitive point of each defective block is regarded as the defective block The y coordinate corresponding to the uppermost primitive point of each defective block is regarded as the upper boundary of the defective block, and the y coordinate corresponding to the lowermost primitive point of each defective block is regarded as the y coordinate of the defective block Lower border. According to the left boundary, right boundary, upper boundary, and lower boundary box, a rectangular area is selected as the coordinates of the flaw area of the flaw block, and multiple flaw sub-areas are selected according to the coordinate box of the flaw area.

In one embodiment, selecting multiple target defect sub-areas from the multiple defect sub-areas according to the size includes: sorting the multiple defect sub-areas according to the size, and selecting the first preset number of defect sub-areas with the highest ranking. The area serves as the target defect sub-area. Sort the defect areas except the first preset number of defect sub-areas from the multiple defect sub-areas according to the sum of width and height, and select the sum of width and height to be within the preset range and sort The first second preset number of defect areas are used as target defect areas. In this embodiment, the first preset quantity, the second preset quantity, and the preset range can be set according to the needs of the user.

In step S23, the first processing method is used to determine the defect types of the multiple target defect sub-regions.

In one embodiment, using the first processing method to determine the defect types of multiple target defect sub-regions is to use a convolutional neural network model to determine the defect types of multiple target defect sub-regions.

In an embodiment, the defect type of the target defect sub-region includes: a scratch type, a scratch type, a bruise type, and a stain type. In an embodiment, the convolutional neural network model includes, but is not limited to: a support vector machine (Support Vector Machine, SVM) model. The multiple target defect sub-regions are used as the input of the convolutional neural network model, and the defect type is output after the convolutional neural network model is calculated.

In one embodiment, the training process of the convolutional neural network model includes:

1) Obtain the defect data of the image of the positive sample and the defect data of the image of the negative sample, and mark the defect type of the image of the positive sample so that the defect data of the image of the positive sample carries the defect type label.

For example, select 1000 defect data corresponding to the scratch type, scratch type, bruise type, and stain type, and mark the type of each defect data. You can use "1" as the data label of the scratch type and "2" as the label For the data label of the scratch type, use "3" as the data label of the scratch type, and use "4" as the data label of the stain type.

2) Randomly divide the defect data of the positive sample and the defect data of the negative sample into a training set with a first preset ratio and a validation set with a second preset ratio, use the training set to train the convolutional neural network model, and use the verification set to verify The accuracy of the trained convolutional neural network model.

First distribute the training samples in the training set of different defect types to different folders. For example, distribute the training samples of the scratch type to the first folder, distribute the training samples of the scratch type to the second folder, distribute the training samples of the bruise type to the third folder, and distribute the training samples of the stain type Go to the fourth folder. Then extract the training samples of the first preset ratio (for example, 70%) from different folders as the total training samples for training the convolutional neural network model, and take the remaining second presets from different folders. Set a proportion (for example, 30%) of the training samples as the total test samples to verify the accuracy of the trained convolutional neural network model.

3) If the accuracy rate is greater than or equal to the preset accuracy rate, the training ends, and the trained convolutional neural network model is used as the classifier to identify the defect type of the target defect sub-region. If the accuracy rate is less than the preset accuracy rate, increase the number of positive samples and the number of negative samples to retrain the convolutional neural network model until the accuracy rate is greater than or equal to the preset accuracy rate.

Step S24, using a second processing method to generate at least one target defect area from a plurality of target defect sub-areas.

In this embodiment, using the second processing method to generate at least one target defect area from a plurality of target defect sub-areas includes: according to the types and positions of the multiple target defect sub-areas, aggregating one or more targets with the same type and adjacent positions. The defect sub-area produces the target defect area.

For example, when the type of blemish in the image is a scratch type, one scratch may include a plurality of consecutive and adjacent small scratches. Please refer to FIG. 3A, which is a schematic diagram of abrasion in an embodiment of the present invention. As shown in FIG. 3A, the first scratch 30 includes a first area 301, a second area 302, a third area 303, and a fourth area 304. The second scratch 31 includes a fifth area 311 and a sixth area 312. In order to avoid mistaking the first area 301, the second area 302, the third area 303, the fourth area 304, the fifth area 311 and the sixth area 312 as six scratches, the first area 301 needs to be The second area 302, the third area 303, and the fourth area 304 are clustered to obtain the target defect area, that is, the first scratch 30. The fifth area 311 and the sixth area 312 are clustered to obtain the target defect area, which becomes the second scratch 31.

For another example, when the type of blemish in the image is a scratch type, one scratch may include multiple consecutive and adjacent small scratches. Please refer to FIG. 3B, which is a schematic diagram of scratches in an embodiment of the present invention. As shown in FIG. 3B, the scratch 20 includes a first long strip 21, a second long strip 22, a third long strip 23 and a fourth long strip 24. In order to avoid mistaking the first strip 21, the second strip 22, the third strip 23 and the fourth strip 24 as four scratches, the first strip 21 and the second strip 22. The third strip 23 and the fourth strip 24 are fitted to obtain the target defect area, which becomes the scratch 20.

Step S25, judging the first defect level of the target defect area according to the first criterion.

In one embodiment, judging the first flaw level of the target flaw area according to the first criterion includes: calculating the area size of the target flaw area, and determining the first flaw level according to the area size of the target flaw area. In a specific embodiment, after the area size of the target defect area is calculated, the defect level relationship table is searched according to the area size of the target defect area to determine the first defect level corresponding to the area size, wherein the defect level relationship table includes multiple The area size of the target defect area and multiple first defect levels, and the corresponding relationship between the area size of the multiple target defect areas and the multiple first defect levels is defined.

In another embodiment, judging the first flaw level of the target flaw area according to the first criterion includes: determining the attention level of the target flaw area according to the flaw type of the target flaw area, and calculating the flaw value of the target flaw area according to the attention level, and According to the defect value and at least one preset threshold value, the first defect level of the target defect area is obtained. In a specific embodiment, judging the attention level of the target defect area includes: searching the attention level relation table according to the defect type of the target defect area to determine the attention level of the target defect area corresponding to the defect type, where the attention level relation table includes multiple The defect type and multiple attention levels of a target defect area are defined, and the corresponding relationship between multiple defect types and attention levels is defined. In a specific embodiment, calculating the defect value of the target defect area according to the attention level includes: searching the calculation rule relationship table according to the attention level to determine the calculation rule of the defect value corresponding to the attention level, and calculating the attention level of the target defect area according to the calculation rule The corresponding defect value. Among them, the calculation rule relationship table defines the correspondence between multiple attention levels of the target defect area and multiple calculation rules. In this embodiment, the calculation rule includes calculating the defect value according to the area of the target defect area, and calculating the defect value according to the sum of the length and width of the target defect area.

In step S26, the first defect level is stored.

Please refer to FIG. 4, which shows a flowchart of an image-based defect detection method in another embodiment of the present invention. In this embodiment, after the defect area is extracted, the defect area is first classified to obtain the defect type, and then the first criterion is used to make a preliminary judgment according to the defect type. If the preliminary judgment result is not accurate, the first criterion is used for the defect area. The second criterion is to make a second judgment. In this embodiment, the first criterion only uses the known feature values of the image, while the second criterion uses the known feature values for further processing, and the calculation amount of the first criterion is smaller than the second criterion, thereby improving the computational efficiency and Judgment accuracy.

Referring to FIG. 4, the image-based defect detection method specifically includes the following steps.

Step S31: Acquire at least one image of the object to be measured.

Step S32: Extract multiple target defect sub-regions from the image.

In one embodiment, the step of extracting multiple target defect sub-regions from the image further includes: pre-processing the image to extract multiple predicted defect locations, and frame selection of multiple defect sub-regions according to the multiple predicted defect locations, Select multiple target defect sub-areas from multiple defect sub-areas according to the size. In an embodiment, pre-processing the image to extract multiple predicted defect locations further includes: extracting multiple regions of interest from the image, and extracting multiple predicted defect locations from the multiple regions of interest using the fourth processing method, Aggregating at least two adjacent ones of the multiple predicted defect positions to obtain multiple target defect sub-regions. In one embodiment, the fourth processing algorithm is a semantic segmentation algorithm.

In a specific implementation, multiple regions of interest can be extracted from the image according to the region of interest algorithm. Use semantic segmentation algorithm to predict and process multiple regions of interest and output background primitive points and flawed primitive points in multiple regions of interest, and binarize the background primitive points and flawed primitive points according to the binarization Separate the defect image element points in the multiple regions of interest, obtain multiple predicted defect locations according to the separated defect image element points in the multiple interest areas, and aggregate at least adjacent ones of the multiple predicted defect locations Two get multiple target defect sub-regions.

In one embodiment, binarizing the background image element point and the defect image element point and separating the defect image element points in the multiple interest regions according to the binarized image element points includes: Set the gray level of the pixel point to 0 or 255 to binarize the gray level of the pixel points in multiple regions of interest. Use the pixel point with a gray value of 255 as the defective pixel point, and set the gray value The primitive point that is 0 is used as the background primitive point. In a specific embodiment, the gray levels of the primitive points in the multiple regions of interest are grouped by the k-means clustering method to obtain two groups. Binarize the gray values of the pixel points in the two groups, and the gray values of the binarized pixel points in each group are the same, and then convert the gray values of the pixel points in multiple regions of interest Comparing with the preset threshold, the gray value of the image element point greater than the preset threshold is set to 255, and the gray value of the image element point not greater than the preset threshold is set to 0. Among them, the preset threshold can be set according to the needs of the user.

In one embodiment, obtaining multiple predicted defect locations according to the separated defect image element points in the multiple interest regions includes: filtering out non-defect image element points in the multiple interest areas, and correcting the defects in the multiple interest areas. The pixel points are clustered to obtain multiple defect blocks, a rectangular area is selected as the defect area of each defect block through the boundary box of each defect block, and the coordinates of the defect area of each defect block are determined, and each defect block is represented by the defect map The element points are clustered, and a plurality of predicted defect locations are obtained according to the coordinates of the defect area of each defect block, wherein each predicted defect location corresponds to the defect area of a defect block.

In one embodiment, selecting multiple defect sub-regions based on multiple predicted defect position frames includes: selecting multiple defect sub-regions based on coordinate boxes of the defect area. In this embodiment, selecting multiple defect sub-areas according to the coordinate frame of each defect area includes: establishing a Cartesian coordinate system in the image with the point on the upper left corner of the image as the origin, where the X direction of the Cartesian coordinate system represents The width of the image, the Y direction of the Cartesian coordinate system represents the height of the image. In the Cartesian coordinate system, the x coordinate corresponding to the leftmost primitive point of each defective block is regarded as the left boundary of the defective block, and the x coordinate corresponding to the rightmost primitive point of each defective block is regarded as the defective block The y coordinate corresponding to the uppermost primitive point of each defective block is regarded as the upper boundary of the defective block, and the y coordinate corresponding to the lowermost primitive point of each defective block is regarded as the y coordinate of the defective block Lower border. According to the left boundary, right boundary, upper boundary, and lower boundary box, a rectangular area is selected as the coordinates of the flaw area of the flaw block, and multiple flaw sub-areas are selected according to the coordinate box of the flaw area.

In one embodiment, selecting multiple target defect sub-areas from the multiple defect sub-areas according to the size includes: sorting the multiple defect sub-areas according to the size, and selecting the first preset number of defect sub-areas with the highest ranking. The area serves as the target defect sub-area. Then sort the defect areas except the first preset number of defect sub-areas in the multiple defect sub-areas according to the size of the sum of width and height, and select the sum of width and height to be within the preset range and The second preset number of defect areas ranked at the top are used as the target defect areas. In this embodiment, the first preset quantity, the second preset quantity, and the preset range can be set according to the needs of the user.

In step S33, the first processing method is used to determine the defect types of the multiple target defect sub-regions.

In one embodiment, using the first processing method to determine the defect types of multiple target defect sub-regions is to use a convolutional neural network model to determine the types of multiple target defect sub-regions.

Step S34, using a second processing method to generate at least one target defect area from a plurality of target defect sub-areas.

In one embodiment, using the second processing method to generate at least one target defect area from a plurality of target defect sub-areas includes: according to the types and positions of the multiple target defect sub-areas, one or more of the same type and adjacent positions are aggregated The target defect sub-area generates a target defect area.

Step S35: Determine the first defect level of the target defect area according to the first criterion.

In one embodiment, judging the first flaw level of the target flaw area according to the first criterion includes: calculating the area size of the target flaw area, and determining the first flaw level according to the area size of the target flaw area. In a specific embodiment, after calculating the area size of the target defect area, the defect level relationship table is searched according to the area size of the target defect area to determine the first defect level corresponding to the area size. Wherein, the defect level relationship table includes the area sizes of multiple target defect areas and multiple first defect levels, and defines the correspondence between the area sizes of multiple target defect areas and the multiple first defect levels.

In step S36, the first defect level is stored.

Step S37: Determine whether the first flaw level meets the preset condition, and when the first flaw level does not meet the preset condition, determine the second flaw level of the target flaw area according to the second criterion, and store the second flaw level.

In one embodiment, the preset condition is that the first defect level belongs to the preset level. That is, when the first defect level does not belong to the preset level, the second defect level of the target defect area is determined according to the second criterion.

In one embodiment, judging the second flaw level of the target flaw area according to the second criterion includes: extracting a plurality of first characteristic values of the target flaw area, and converting the plurality of first characteristic values into second characteristic values in a preset format , And use the third processing method to process the second feature value to obtain the second defect level. In one embodiment, the first feature value is any combination of size, grayscale, texture, position, and direction, the preset format is an image format, and the second feature value is a feature map formed by the conversion of the first feature value. In this embodiment, the third processing method is a deep learning algorithm.

In one embodiment, the flaw detection method further includes: judging whether the target flaw area has a flaw according to the first flaw level and/or the second flaw level.

Please refer to FIG. 5, which shows a flowchart of an image-based defect detection method in another embodiment of the present invention. The method includes the following steps.

Step S41: Acquire at least one image of the object to be measured.

In one embodiment, the method of acquiring at least one image of the object to be measured is to acquire at least one image of the object to be measured taken by a camera, where the camera may be a line scan camera or an area scan camera. In this embodiment, the object to be measured is a device such as a mobile phone or a tablet computer. In another embodiment, the method of acquiring at least one image of the object to be measured is to receive at least one image of the object to be measured sent by the server. In other embodiments, at least one image of the object to be measured can be obtained from a local database. In this embodiment, the image may include a complete or partial image of the object to be measured. The image can be of any resolution, or it can be high-sampled or low-sampled, depending on actual needs.

Step S42, pre-processing the image.

In one embodiment, pre-processing the image includes: extracting multiple regions of interest from the image according to the region of interest algorithm, using the semantic segmentation algorithm to predict and process the multiple regions of interest, and outputting information in the multiple regions of interest. The background image element point and the defect image element point are binarized, and the defect image element points in the multiple regions of interest are separated according to the binarized image element point.

In other embodiments, pre-processing the image further includes: filtering and denoising the image.

Step S43: Extract multiple defect sub-regions from the image.

In one embodiment, extracting multiple defect sub-regions from the image includes: clustering defect primitive points in multiple regions of interest to obtain multiple defect blocks, and selecting a rectangular area through the bounding box of each defect block As the defect area of the defect block, and determine the coordinates of the defect area of each defect block. Each of the defect blocks is obtained by clustering the defect image element points, and multiple defect sub-areas are obtained according to the coordinates of the defect area of each defect block.

Step S44, extracting the target defect sub-areas from the multiple defect sub-areas.

In one embodiment, extracting the target flaw sub-regions from the multiple flaw sub-regions includes: sorting the multiple flaw sub-regions according to the size, and selecting the first preset number of flaw sub-regions with the highest ranking as the target flaw sub-regions area. Sort the defect areas except the first preset number of defect sub-areas from the multiple defect sub-areas according to the sum of width and height, and select the sum of width and height to be within the preset range and sort The first second preset number of defect areas are used as target defect areas. In this embodiment, the first preset quantity, the second preset quantity, and the preset range can be set according to the needs of the user.

In step S45, the first processing method is used to determine the defect types of the multiple target defect sub-regions.

In one embodiment, using the first processing method to determine the defect type of the multiple target defect sub-regions includes: dividing the image into equal parts to obtain multiple image blocks of a preset size and the coordinates of each image block, and Each image block is associated with a target flaw sub-region to be predicted, and the image block associated with the target flaw sub-region is used to determine the flaw type of the multiple target flaw sub-regions using a convolutional neural network model.

In an embodiment, the defect type of the target defect sub-region includes: a scratch type, a scratch type, a bruise type, and a stain type. In this embodiment, the convolutional neural network model includes, but is not limited to: a support vector machine model. The multiple target defect sub-regions are used as the input of the convolutional neural network model, and the defect type is output after the convolutional neural network model is calculated.

Step S46, generating at least one target defect area from the target defect sub-area according to the second processing method.

In an embodiment, generating at least one target defect area from the target defect sub-area according to the second processing method includes: according to the types and positions of the multiple target defect sub-areas, aggregating one or more target defects of the same type and adjacent in position. The sub-area produces the target defect area.

Step S47, judging the attention level of the target flaw area according to the type of the flaw area.

In one embodiment, judging the attention level of the target flaw area according to the type of the flaw area includes: using the type and coordinates of the target flaw area to determine the aspect ratio of the target flaw area, and determining the target flaw area according to the aspect ratio of the target flaw area Level of attention. Different types of defects may have different distribution tendencies. The attention level can be determined according to the type and location of the target defect area. For example, the higher the attention level, the higher the degree of conformity between the location of the target defect area and the type.

Step S48, according to the attention level of the target flaw area, determine the first flaw level of the target flaw area according to the first criterion.

In one embodiment, judging the first flaw level of the target flaw area according to the first criterion according to the attention level of the target flaw area includes: calculating the area size of the target flaw area, and determining the first flaw level according to the area size of the target flaw area.

In step S49, it is judged whether the first defect level of the target defect area meets the preset condition, and when the first defect level does not meet the preset condition, the second defect level of the destination area is judged according to the second criterion.

In one embodiment, the preset condition is that the first defect level belongs to the preset level. In one embodiment, judging the second flaw level of the target flaw area according to the second criterion includes: extracting a plurality of first characteristic values of the target flaw area, and converting the plurality of first characteristic values into second characteristic values in a preset format , And use the third processing method to process the second feature value to obtain the second defect level. In a specific embodiment, the first feature value is any combination of size, grayscale, texture, position, and direction, the preset format is an image format, and the second feature value is a feature map formed by the conversion of the first feature value. In one embodiment, the third processing method is a deep learning algorithm.

Step S50, judging whether there is a defect in the target defect area according to the first defect level and/or the second defect level.

In one embodiment, if the first defect level belongs to the preset level, determine whether there is a defect in the target defect area according to the first defect level, and if the first defect level does not belong to the preset level, then judge according to the second defect level Whether there are defects in the target defect area. In other implementation manners, if the first defect level does not belong to the preset level, it can also be comprehensively judged whether there is a defect in the target defect area based on the first defect level and the second defect level.

Step S51, storing the first defect level and the second defect level.

Please refer to FIG. 6, which is a schematic diagram of the defect detection system 1 in an embodiment of the present invention. The flaw detection system 1 includes a calculation unit 11 and a storage unit 12. The calculation unit 11 can execute the detection program 121 in the storage unit 12. The computing unit 11 can obtain the image of the object to be tested from the defect detection system 1 or a remote storage unit, or use a photographing unit set in the defect detection system 1 or a remote, or from a remote server or Database acquisition. In this embodiment, the calculation unit 11 may include multiple calculation sub-units, and different sections of the detection program 121 may be executed by different calculation sub-units. In this embodiment, the computing unit 11 may also cooperate with a remote computing unit to execute part of the detection program 121 section respectively. The detection result can be stored in the defect detection system 1 or a remote storage unit, or output to a remote server or database.

Please refer to FIG. 7, which shows a functional module diagram of the defect detection system 1 in an embodiment of the present invention. The defect detection system 1 includes one or more modules, and one or more modules run in the computing unit 11. In this embodiment, the defect detection system 1 includes an image acquisition module 101, a defect extraction module 102, a first processing module 103, a second processing module 104, a first judgment module 105, and a storage module 106. In this embodiment, the image acquisition module 101, the defect extraction module 102, the first processing module 103, the second processing module 104, the first judgment module 105, and the storage module 106 are stored in the storage unit 12, and Called by the calculation unit 11 for execution. The module referred to in the present invention refers to a series of computer program instruction segments that can complete specific functions, and is more suitable for describing the execution process of software in the defect detection system 1 than a program. In other embodiments, the image acquisition module 101, the defect extraction module 102, the first processing module 103, the second processing module 104, the first judgment module 105, and the storage module 106 are embedded or cured in The program segment or code in the calculation unit 11.

In this embodiment, the image acquisition module 101 is used to acquire at least one image of the object to be measured. The image acquisition module 101 may, for example, acquire at least one image of the object to be measured taken by a camera, where the camera may be a line scan camera or an area scan camera. In this embodiment, the object to be measured is a device such as a mobile phone or a tablet computer. In another embodiment, the image acquisition module 101 receives at least one image of the object to be measured sent by the server. In other embodiments, at least one image of the object to be measured can be obtained from a local database. In this embodiment, the image may include a complete or partial image of the object to be measured. The image can be of any resolution, or it can be high-sampled or low-sampled, depending on actual needs.

In this embodiment, the defect extraction module 102 is used to extract multiple target defect sub-regions from the image. In one embodiment, the defect extraction module 102 may also perform pre-processing on the image to extract multiple predicted defect positions, frame multiple defect sub-regions according to the multiple predicted defect positions, and select multiple defect sub-regions according to the size. Select multiple target defect sub-regions in the region. In one embodiment, the defect extraction module 102 can also extract multiple regions of interest from the image, extract multiple predicted defect locations from the multiple regions of interest using the fourth processing method, and aggregate the adjacent multiple predicted defect locations. At least two of them get multiple target defect sub-regions. In one embodiment, the fourth processing algorithm is a semantic segmentation algorithm.

In a specific implementation, the defect extraction module 102 may extract multiple regions of interest from the image according to a Region of Interest (ROI) algorithm. The defect extraction module 102 uses a semantic segmentation algorithm to perform prediction processing on multiple regions of interest and outputs background primitive points and defect primitive points in multiple regions of interest, and binarizes the background primitive points and defect primitive points And according to the binarized image element points, the defect image element points in the multiple interest areas are separated, multiple predicted defect locations are obtained according to the separated defect image element points in the multiple interest areas, and multiple predicted defect locations are aggregated At least two adjacent ones of the two obtain multiple target defect sub-regions.

In an embodiment, the defect extraction module 102 can set the gray levels of the pixel points in the multiple interest areas to 0 or 255 to binarize the gray levels of the pixel points in the multiple interest areas. A pixel point with a degree value of 255 is used as a defect pixel point, and a pixel point with a gray value of 0 is used as a background pixel point. In a specific embodiment, the defect extraction module 102 may use the k-means clustering method to group the gray levels of the primitive points in the multiple regions of interest to obtain two groups. The grayscale is binarized, and the grayscale value of the binarized pixel points in each group is the same, and then the grayscale values of the pixel points in multiple regions of interest are compared with the preset threshold, and the pixel The gray value of the points that is greater than the preset threshold is set to 255, and the gray value of the pixel points that is not greater than the preset threshold is set to 0. Among them, the preset threshold can be set according to the needs of the user.

In one embodiment, the defect extraction module 102 can filter out non-defective primitive points in multiple regions of interest, and cluster the defective primitive points in multiple regions of interest to obtain multiple defective blocks. A rectangular area is selected as the defect area of each defect block through the boundary box of each defect block, and the coordinates of the defect area of each defect block are determined, wherein each defect block is obtained by clustering the defect image element points. A plurality of predicted defect positions are obtained according to the coordinates of the defect area of each defect block, wherein each predicted defect position corresponds to the defect area of a defect block.

In one embodiment, the defect extraction module 102 can select a plurality of defect sub-areas according to the coordinate frame of the defect area. In an embodiment, the defect extraction module 102 can establish a Cartesian coordinate system in the image with the point at the upper left corner of the image as the origin, where the X direction of the Cartesian coordinate system represents the width of the image, and the Cartesian coordinate system The Y direction indicates the height of the image. In the Cartesian coordinate system, the x coordinate corresponding to the leftmost primitive point of each defective block is regarded as the left boundary of the defective block, and the x coordinate corresponding to the rightmost primitive point of each defective block is regarded as the defective block The y coordinate corresponding to the uppermost primitive point of each defective block is regarded as the upper boundary of the defective block, and the y coordinate corresponding to the lowermost primitive point of each defective block is regarded as the y coordinate of the defective block Lower border. According to the left boundary, right boundary, upper boundary, and lower boundary box, a rectangular area is selected as the coordinates of the flaw area of the flaw block, and multiple flaw sub-areas are selected according to the coordinate box of the flaw area.

In an embodiment, the defect extraction module 102 may sort a plurality of defect sub-areas according to the size, and select the first predetermined number of defect sub-areas with the highest ranking as the target defect sub-areas. Then sort the defect areas except the first preset number of defect sub-areas in the multiple defect sub-areas according to the sum of width and height, and select the sum of width and height to be within the preset range and The second preset number of defect areas ranked at the top are used as the target defect areas. In this embodiment, the first preset quantity, the second preset quantity, and the preset range can be set according to the needs of the user.

The first processing module 103 is used for judging the defect types of multiple target defect sub-regions by using the first processing method. In one embodiment, the first processing method is to use a convolutional neural network model to determine the defect types of multiple target defect sub-regions. In an embodiment, the defect type of the target defect sub-region includes: a scratch type, a scratch type, a bruise type, and a stain type.

The second processing module 104 is used for generating at least one target defect area from a plurality of target defect sub-areas using the second processing method. In one embodiment, the second processing module 104 aggregates one or more target defect sub-areas of the same type and adjacent positions to generate a target defect area according to the types and positions of the multiple target defect sub-areas.

The first judging module 105 is used for judging the first defect level of the target defect area according to the first criterion. In one embodiment, the first judgment module 105 may calculate the area size of the target defect area, and determine the first defect level according to the area size of the target defect area. In a specific embodiment, after the area size of the target defect area is calculated, the defect level relationship table is searched according to the area size of the target defect area to determine the first defect level corresponding to the area size, wherein the defect level relationship table includes multiple The area size of the target defect area and multiple first defect levels, and the corresponding relationship between the area size of the multiple target defect areas and the multiple first defect levels is defined.

In another embodiment, the first judgment module 105 judges the attention level of the target defect area according to the defect type of the target defect area, calculates the defect value of the target defect area according to the attention level, and then according to the defect value and at least one preset Threshold to get the first flaw level of the target flaw area. In a specific embodiment, the first judging module 105 searches the attention level relation table according to the defect type of the target defect area to determine the attention level of the target defect area corresponding to the defect type, wherein the attention level relation table includes multiple The defect type and multiple attention levels of the target defect area, and the corresponding relationship between multiple defect types and attention levels are defined. In a specific embodiment, the first judgment module 105 determines the calculation rule of the defect value corresponding to the attention level according to the attention level search calculation rule relationship table, and calculates the defect value corresponding to the attention level of the target defect area according to the calculation rule. . Among them, the calculation rule relationship table defines the correspondence between multiple attention levels of the target defect area and multiple calculation rules. In this embodiment, the calculation rule includes calculating the defect value according to the area of the target defect area, and calculating the defect value according to the sum of the length and width of the target defect area.

The storage module 106 is used to store the first defect level.

Please refer to FIG. 8, which shows a functional module diagram of the defect detection system 1 in another embodiment of the present invention. The defect detection system 1 includes one or more modules, and one or more modules run in the computing unit 11. In this embodiment, the defect detection system 1 includes an image acquisition module 201, a defect extraction module 202, a first processing module 203, a second processing module 204, a first judgment module 205, a storage module 206, and a The second judgment module 207 and the third judgment module 208.

The image acquisition module 201 is used to acquire at least one image of the object to be measured.

The defect extraction module 202 is used to extract multiple target defect sub-regions from the image.

In one embodiment, the defect extraction module 202 can perform pre-processing on the image to extract multiple predicted defect positions, frame multiple defect sub-regions according to the multiple predicted defect positions, and select multiple defect sub-regions according to the size. Select multiple target defect sub-regions. In one embodiment, the defect extraction module 202 can extract multiple regions of interest from the image, use the fourth processing method to extract multiple predicted defect locations from the multiple regions of interest, and aggregate adjacent ones of the multiple predicted defect locations. At least two get multiple target defect sub-regions. In one embodiment, the fourth processing algorithm is a semantic segmentation algorithm.

In one embodiment, the defect extraction module 202 can be used to extract multiple regions of interest from the image according to the region of interest algorithm, use the semantic segmentation algorithm to predict the multiple regions of interest, and output the background in the multiple regions of interest. Primitive points and defective primitive points, binarize the background primitive points and defective primitive points, and separate the defective primitive points in multiple regions of interest according to the binarized primitive points. According to the separated multiple The defect pixel points in each region of interest obtain multiple predicted defect locations, and at least two adjacent ones of the multiple predicted defect locations are aggregated to obtain multiple target defect sub-regions.

In one embodiment, the defect extraction module 202 can be used to filter out non-defective primitive points in multiple regions of interest, cluster the defective primitive points in multiple regions of interest to obtain multiple defective blocks, and pass each The boundary box of the defect block selects a rectangular area as the defect area of the defect block and determines the coordinates of the defect area of each defect block, wherein each defect block is obtained by clustering the defect image element points. And obtaining a plurality of predicted defect positions according to the coordinates of the defect area of each defect block, wherein each predicted defect position corresponds to the defect area of one defect block.

In one embodiment, the defect extraction module 202 can be used to select multiple defect sub-areas according to the coordinate frame of the defect area. In this embodiment, the defect extraction module 202 can be used to establish a Cartesian coordinate system in the image with the point at the upper left corner of the image as the origin, where the X direction of the Cartesian coordinate system represents the width of the image, and the Cartesian coordinate system The Y direction represents the height of the image; in the Cartesian coordinate system, the x coordinate corresponding to the leftmost pixel point of each defective block is regarded as the left boundary of the defective block, and the rightmost pixel point of each defective block The corresponding x coordinate is taken as the right boundary of the defect block, the y coordinate corresponding to the uppermost primitive point of each defect block is regarded as the upper boundary of the defect block, and the lowermost primitive point of each defect block is located The corresponding y coordinate serves as the lower boundary of the defect block. According to the left boundary, right boundary, upper boundary, and lower boundary box, a rectangular area is selected as the coordinates of the flaw area of the flaw block, and multiple flaw sub-areas are selected according to the coordinate box of the flaw area.

In one embodiment, the defect extraction module 202 can be used to sort a plurality of defect sub-areas according to the size, and select the first predetermined number of defect sub-areas with the highest ranking as the target defect sub-areas. Then sort the defect areas except the first preset number of defect sub-areas in the multiple defect sub-areas according to the sum of width and height, and select the sum of width and height to be within the preset range and The second preset number of defect areas ranked at the top are used as the target defect areas. In this embodiment, the first preset quantity, the second preset quantity, and the preset range can be set according to the needs of the user.

The first processing module 203 is used for judging the defect types of multiple target defect sub-regions by using the first processing method. In one embodiment, the first processing module 203 uses a convolutional neural network model to determine the types of multiple target defect sub-regions.

The second processing module 20 is used for generating at least one target defect area from a plurality of target defect sub-areas using the second processing method. In an embodiment, the second processing module 204 can aggregate one or more target defect sub-areas of the same type and adjacent positions to generate the target defect area according to the types and positions of the multiple target defect sub-areas.

The first judging module 205 is used for judging the first defect level of the target defect area according to the first criterion. In one embodiment, the first judgment module 205 can calculate the area size of the target defect area, and determine the first defect level according to the area size of the target defect area. In a specific embodiment, after the area size of the target defect area is calculated, the defect level relationship table is searched according to the area size of the target defect area to determine the first defect level corresponding to the area size, wherein the defect level relationship table includes multiple The area size of the target defect area and multiple first defect levels, and the corresponding relationship between the area size of the multiple target defect areas and the multiple first defect levels is defined.

The storage module 206 is used to store the first defect level.

The second judgment module 207 is used to judge whether the first defect level meets the preset condition, and when the first defect level does not meet the preset condition, judge the second defect level of the target defect area according to the second criterion. In one embodiment, the preset condition is that the first defect level belongs to the preset level. That is, when the first defect level does not belong to the preset level, the second judgment module 207 judges the second defect level of the target defect area according to the second criterion.

The storage module 206 is also used to store the second defect level.

In one embodiment, the second judging module 207 can extract a plurality of first feature values of the target defect area, convert the plurality of first feature values into second feature values in a preset format, and use a third processing method to process The second characteristic value is used to obtain the second defect level. In one embodiment, the first feature value is any combination of size, grayscale, texture, position, and direction, the preset format is an image format, and the second feature value is a feature map formed by the conversion of the first feature value. In one embodiment, the third processing method is a deep learning algorithm.

In one embodiment, the third judgment module 208 is used to judge whether the target defect area has a defect according to the first defect level and/or the second defect level.

In the several embodiments provided by the present invention, it should be understood that the disclosed electronic device and method may be implemented in other ways. For example, the electronic device embodiments described above are merely illustrative. For example, the division of modules is only a logical function division, and there may be other division methods in actual implementation.

In addition, the functional modules in the various embodiments of the present invention may be integrated in the same processing module, or each module may exist alone physically, or two or more modules may be integrated in the same module. The above-mentioned integrated modules can be implemented either in the form of hardware or in the form of hardware plus software functional modules.

For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the scope of the appended invention patent application rather than the above description, so it is intended to fall within The meaning of the equivalent elements and all changes within the scope of the patent application for invention are included in the present invention. Any reference signs in the scope of the patent application for invention shall not be regarded as limiting the scope of the patent application for the invention involved. In addition, it is obvious that the word "include" does not exclude other modules or steps, and the singular does not exclude the plural. Multiple modules or electronic devices stated in the scope of an electronic device invention patent application can also be implemented by the same module or electronic device through software or hardware. Words such as first and second are used to denote names, but do not denote any specific order.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be Modifications or equivalent replacements are made without departing from the spirit and scope of the technical solution of the present invention.

S21~S26: steps 20: scratch 21: The first strip 22: The second long strip 23: The third bar 24: The fourth bar 30: first scratch 301: The first area 302: The second area 303: The third area 304: The fourth area 31: The second scrape 311: Fifth Region 312: The sixth area S31~S37: steps S41~S49, S50~S51: steps 1: Defect detection system 11: Computing unit 12: storage unit 121: detection program 101: Image acquisition module 102: Defect Extraction Module 103: The first processing module 104: The second processing module 105: The first judgment module 106: storage module 201: Image acquisition module 202: Defect Extraction Module 203: The first processing module 204: The second processing module 205: The first judgment module 206: Storage Module 207: Second Judgment Module 208: Third Judgment Module

FIG. 1A and FIG. 1B are schematic diagrams of the image of the object to be measured including multiple defects in an embodiment of the present invention. Fig. 2 is a flowchart of an image-based defect detection method in an embodiment of the present invention. FIG. 3A is a schematic diagram of a scratch in an embodiment of the present invention, and FIG. 3B is a schematic diagram of a scratch in an embodiment of the present invention. FIG. 4 is a flowchart of an image-based defect detection method in another embodiment of the present invention. Fig. 5 is a flowchart of an image-based defect detection method in another embodiment of the present invention. Fig. 6 is a schematic diagram of a defect detection system in an embodiment of the present invention. FIG. 7 is a functional module diagram of a defect detection system in an embodiment of the present invention. FIG. 8 is a functional module diagram of a defect detection system in another embodiment of the present invention.

S21~S26: steps

Claims (15)

An image-based defect detection method is suitable for detecting the appearance defect of an object to be tested. The improvement lies in that the image defect detection method includes: Acquire at least one image of the object to be measured; Extracting multiple target defect sub-regions from the image; Judging the defect types of the multiple target defect sub-regions by using the first processing method; Generating at least one target defect area from the plurality of target defect sub-areas by using the second processing method; Judging the first defect level of the target defect area according to the first criterion; The first defect level is stored. The image-based defect detection method according to claim 1, wherein the method further includes: Judging whether the first defect level meets a preset condition; When the first defect level does not meet a preset condition, judging the second defect level of the target defect area according to a second criterion; The second defect level is stored. The image-based defect detection method according to claim 2, wherein the method further comprises: judging whether the target defect area has a defect according to the first defect level and/or the second defect level. The image-based defect detection method according to claim 2, wherein the preset condition is that the first defect level belongs to a preset level. The image-based defect detection method according to claim 2, wherein the step of judging the second defect level of the target defect area according to a second criterion further includes: Extracting a plurality of first feature values of the target defect area; Converting the plurality of first characteristic values into second characteristic values in a preset format; A third processing method is used to process the second characteristic value to obtain the second defect level. The image-based defect detection method according to claim 5, wherein the first feature value can be any combination of size, grayscale, texture, position, and direction, and the preset format is an image format, so The second feature value is a feature map formed by the conversion of the first feature value. The image-based defect detection method according to claim 5, wherein the third processing method is a deep learning algorithm. The image-based defect detection method according to claim 1, wherein the step of judging the first defect level of the target defect area according to a first criterion comprises: Calculating the area size of the target defect area; The first flaw level is given according to the area size of the target flaw area. The image-based defect detection method according to claim 1, wherein the step of judging the first defect level of the target defect area according to a first criterion comprises: Judging the attention level of the target flaw area according to the flaw type of the target flaw area; Calculating the defect value of the target defect area according to the attention level; The first defect level of the target defect area is obtained according to the defect value and at least one preset threshold value. The image-based defect detection method according to claim 1, wherein the step of extracting multiple target defect sub-regions from the image further comprises: Pre-processing the image to extract multiple predicted defect positions; Frame selection of multiple defect sub-regions according to the multiple predicted defect positions; Selecting multiple target defect sub-areas from the multiple defect sub-areas according to the size. The image-based defect detection method according to claim 1, wherein the step of pre-processing the image to extract multiple predicted defect positions further includes: Extracting multiple regions of interest from the image; Extracting the plurality of predicted defect positions from the plurality of regions of interest by using a fourth processing method; At least two adjacent ones of the plurality of predicted defect positions are aggregated. The image-based defect detection method according to claim 11, wherein the fourth processing method is a semantic segmentation algorithm. The image-based defect detection method according to claim 1, wherein the step of determining the defect type of the plurality of target defect sub-regions by using the first processing method further includes: The convolutional neural network model is used to determine the defect types of the multiple target defect sub-regions. The image-based defect detection method according to claim 1, wherein the step of generating at least one target defect area from the plurality of target defect sub-areas by using the second processing method further comprises: According to the defect types and positions of the multiple target defect sub-areas, the target defect area is generated by grouping one or more of the target defect sub-areas with the same type and adjacent positions. A computer-readable storage medium, which is improved in that a computer program is stored thereon, and the computer program can be executed by a processor, and implements the image-based defect detection method according to any one of claims 1 to 14 .

Images