CN113763355A - Defect detection method and device, electronic equipment and storage medium - Google Patents
Defect detection method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113763355A CN113763355A CN202111045437.3A CN202111045437A CN113763355A CN 113763355 A CN113763355 A CN 113763355A CN 202111045437 A CN202111045437 A CN 202111045437A CN 113763355 A CN113763355 A CN 113763355A
- Authority
- CN
- China
- Prior art keywords
- image
- defect
- depth
- defect detection
- depth information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 290
- 238000001514 detection method Methods 0.000 title claims abstract description 139
- 238000000034 method Methods 0.000 claims abstract description 43
- 238000012549 training Methods 0.000 claims description 23
- 238000012545 processing Methods 0.000 abstract description 10
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000007849 functional defect Effects 0.000 description 6
- 238000002372 labelling Methods 0.000 description 6
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 6
- 230000001680 brushing effect Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000002950 deficient Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 238000004026 adhesive bonding Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000003292 glue Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000000344 soap Substances 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/0008—Industrial image inspection checking presence/absence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The application provides a defect detection method, a defect detection device, an electronic device and a storage medium. The method comprises the following steps: acquiring an image to be identified and a depth image of an object to be detected; the positions of pixel points occupied by the object to be detected in the image to be identified and the depth image are the same; inputting the image to be identified into a defect detection model to obtain a defect area output by the defect detection model; determining depth information corresponding to pixel points in the defect area according to the depth image; and obtaining a defect detection result according to the depth information. According to the method and the device, the image of the object to be detected is acquired, the image is subjected to defect identification, after the defect area is determined, the depth information is obtained by using the depth map corresponding to the defect area, and finally whether the object is the target defect or not is determined according to the depth information. According to the defect detection method and device, the defect detection is achieved by processing the image, the target defect is determined by combining the depth information, and the defect detection efficiency is improved.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a defect detection method and apparatus, an electronic device, and a storage medium.
Background
At present, the shell of the electronic product needs to be subjected to multiple quality checks before the electronic product is delivered out of a factory. Taking the mobile phone middle frame as an example, before shipping, quality inspection needs to be performed on whether the appearance of the mobile phone is defective or not.
The mobile phone middle frame can generate defects due to process reasons when being sealed, the defects are in shapes of scratches, hollows, cracks and the like when viewed from the appearance of an image, and the defects are defined as appearance defects. The detection of the appearance defect can be realized by brushing soapy water, and if bubbles emerge, the defect is defined as a functional defect. For the quality detection of the middle frame glue brushing, the functional defects in the appearance defects are emphasized, and the appearance defects do not leak air bubbles when being brushed with soap water, so that the appearance defects do not influence the gluing quality of the middle frame. However, the detection method is time-consuming, labor-consuming and high in labor cost.
Disclosure of Invention
An object of the embodiments of the present application is to provide a defect detection method, a defect detection apparatus, an electronic device, and a storage medium, so as to improve the efficiency of detecting the defects of the housing of the electronic product.
In a first aspect, an embodiment of the present application provides a defect detection method, including: acquiring an image to be identified and a depth image of an object to be detected; the positions of pixel points occupied by the object to be detected in the image to be identified and the depth image are the same; inputting the image to be identified into a defect detection model to obtain a defect area output by the defect detection model; determining depth information corresponding to pixel points in the defect area according to the depth image; and obtaining a defect detection result according to the depth information.
According to the method and the device, the image of the object to be detected is acquired, the image is subjected to defect identification, after the defect area is determined, the depth information is obtained by using the depth map corresponding to the defect area, and finally whether the object is the target defect or not is determined according to the depth information. Above-mentioned operation can be realized through the automation, compares in the mode through the soapy water of brush, and the efficiency of this application embodiment detection defect is higher.
In any embodiment, the obtaining a defect detection result according to the depth information includes: setting the gray value of the pixel point with the depth information larger than the preset depth as a first gray value, and setting the gray value of the pixel point with the depth information smaller than or equal to the preset depth as a second gray value; and acquiring a first quantity corresponding to the pixel points of the first gray value, and if the first quantity is greater than a first preset quantity, determining the defect area as a target defect.
According to the method and the device, the image of the defect area is binarized according to the depth information, and whether the image is the target defect or not is determined according to the size of the area with the deeper defect depth, so that the accuracy of target defect detection is improved.
In any embodiment, the obtaining a defect detection result according to the depth information includes: acquiring the maximum depth value of the pixel points in the defect area according to the depth information; and if the maximum depth value is larger than a first preset threshold value, determining the defect area as a target defect.
According to the defect detection method and device, whether the maximum depth value of the pixel point in the defect area is larger than the first preset threshold value or not is judged, if yes, the target defect is determined, and therefore the defect detection efficiency is improved.
In any embodiment, the obtaining a defect detection result according to the depth information includes: determining the average depth corresponding to the defect area according to the depth information; and if the average depth is larger than a second preset threshold value, determining the defect area as a target defect.
According to the defect detection method and device, the average depth of the defect area is calculated, and if the average depth is larger than the second preset threshold value, the defect area reflects defects seriously on the whole, so that the defect area is determined to be a target defect, and the defect detection accuracy is improved.
In any embodiment, the acquiring the image to be recognized and the depth image of the object to be detected includes: acquiring an original image and an original depth image; segmenting the original image according to a preset size to obtain a plurality of original image blocks; inputting the plurality of original image blocks into an object recognition model to obtain a recognition result corresponding to each original image block output by the object recognition model; the identification result corresponding to the original image block forms the image to be identified; dividing the original depth image according to the identification result corresponding to each original image block to obtain a depth image block corresponding to each original image block; all the blocks of depth images constitute the depth image.
According to the image processing method and device, the original image is divided into the plurality of original image blocks, then the object detection is carried out on each original image block, the image to be recognized is obtained, and the image processing efficiency is improved.
In any embodiment, inputting the image to be recognized into a defect detection model, and obtaining a defect region output by the defect detection model, includes: and inputting the identification result corresponding to each original image block into the defect detection model to obtain a defect area corresponding to each identification result block output by the defect detection model.
According to the defect detection method and device, the identification result corresponding to the original image block is input into the defect detection model for defect detection, and the defect detection efficiency is improved.
In any embodiment, the method further comprises: acquiring a training image and a mask image with defects marked; inputting the training image into a model to be trained to obtain a prediction result output by the model to be trained; and optimizing the internal parameters of the model to be trained according to the prediction result and the mask image to obtain the trained defect detection model.
According to the defect detection method and device, the defect detection model is trained through the training image and the mask image, so that the defect area in the image to be recognized can be recognized quickly and accurately through the defect detection model, and the defect detection efficiency is improved.
In a second aspect, an embodiment of the present application provides a defect detecting apparatus, including: the image acquisition module is used for acquiring an image to be identified and a depth image of an object to be detected; the positions of pixel points occupied by the object to be detected in the image to be identified and the depth image are the same; the defect area identification module is used for inputting the image to be identified into a defect detection model to obtain a defect area output by the defect detection model; the depth information counting module is used for determining depth information corresponding to pixel points in the defect area according to the depth image; and the defect detection module is used for obtaining a defect detection result according to the depth information.
In a third aspect, an embodiment of the present application provides an electronic device, including: the system comprises a processor, a memory and a bus, wherein the processor and the memory are communicated with each other through the bus; the memory stores program instructions executable by the processor, the processor being capable of performing the method of the first aspect when invoked by the program instructions.
In a fourth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium, including: the non-transitory computer readable storage medium stores computer instructions that cause the computer to perform the method of the first aspect.
Additional features and advantages of the present application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the embodiments of the present application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 is a schematic flow chart of a defect detection method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a defect detection model training process according to an embodiment of the present disclosure;
FIG. 3 is a schematic flow chart of another defect detection method according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a defect detection apparatus according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
The inventor researches and finds that the method is low in efficiency by brushing soapy water on the shell of the electronic equipment to be detected to judge whether bubbles are generated or not so as to determine whether the appearance defect is a functional defect or not. Therefore, in order to solve the technical problem, embodiments of the present application provide a defect detection method, which determines a defect region from an image to be identified by using a defect detection model and obtains a defect detection result in combination with a depth image, and in this method, a result can be obtained by processing an acquired image without brushing soapy water on a shell, thereby greatly improving the efficiency of defect detection.
The defect detection method provided by the embodiment of the application can be applied to shell defect detection of electronic products, for example: the defect detection method comprises the following steps of detecting defects of a middle frame of the mobile phone, detecting defects of a middle frame of the smart watch, detecting defects of a middle frame of the tablet personal computer, detecting defects of a back plate of the mobile phone and the like. In addition, the defect detection of the outer shell of other products except the electronic product, for example, the defect detection of a certain food packaging box, and the like can be performed. For convenience of description, the embodiment of the present application takes defect detection of a middle frame of a mobile phone as an example for expansion description.
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
The model training method and the defect detection method provided by the embodiment of the application can be applied to terminal equipment (also called electronic equipment) and a server; the terminal device may be a smart phone, a tablet computer, a Personal Digital Assistant (PDA), or the like; the server may specifically be an application server, and may also be a Web server. In addition, both the model training method and the detection method can be executed by the same terminal device, and can also be executed by different terminal devices.
For convenience of understanding, in the technical solution provided in the embodiment of the present application, an application scenario of the model training method and the detection method provided in the embodiment of the present application is described below by taking a terminal device as an execution subject.
Fig. 1 is a schematic flow chart of a defect detection method provided in an embodiment of the present application, and as shown in fig. 1, the method includes:
step 101: acquiring an image to be identified and a depth image of an object to be detected; the positions of pixel points occupied by the object to be detected in the image to be identified and the depth image are the same;
step 102: inputting the image to be identified into a defect detection model to obtain a defect area output by the defect detection model;
step 103: determining depth information corresponding to pixel points in the defect area according to the depth image;
step 104: and obtaining a defect detection result according to the depth information.
In step 101, the image to be recognized includes an object to be detected, the depth image includes depth information of the object to be detected, and pixel points in the image to be detected and the depth image correspond to each other one to one, that is, positions of the pixel points occupied by the object to be detected in the image to be recognized and the depth image are the same. The image to be recognized and the depth image can be obtained by shooting an object to be detected by a high-definition laser line scanning camera, and the acquired image to be recognized and the acquired depth image are sent to the terminal equipment by the high-definition laser line scanning camera. It can be understood that the object to be detected may be a middle frame of a mobile phone. The image to be recognized and the depth image can be collected by different image collecting devices, but the positions of pixel points of the object to be detected in the image to be recognized and the depth image are the same.
In step 102, the defect detection model is pre-trained, and is capable of analyzing the input image to be recognized, so as to recognize the defect region on the object to be detected in the image to be recognized. It can be understood that the defect area is an image output by the defect detection model and obtained by labeling the defect with a labeling frame. The defect detection model may be constructed by using a neural network, a convolutional neural network, an SSD network, a YOLO network, or the like, and may also be constructed by using other networks, which is not specifically limited in this embodiment of the present application.
In step 103, since the depth image and the image to be identified correspond to each other, after the defect region is obtained, the defect region may be mapped into the depth image, a pixel point region corresponding to the defect region is determined in the depth image, and depth information corresponding to the pixel point region is obtained.
In step 104, after obtaining the depth information of the defect area, the terminal device obtains a defect detection result according to the depth information. It is understood that the defect detection result includes whether the defect type corresponding to the defect area is a functional defect or not.
According to the method and the device, the image of the object to be detected is acquired, the image is subjected to defect identification, after the defect area is determined, the depth information is obtained by using the depth map corresponding to the defect area, and finally whether the object is the target defect or not is determined according to the depth information. Above-mentioned operation can be realized through the automation, compares in the mode through the soapy water of brush, and the efficiency of this application embodiment detection defect is higher.
In another embodiment, fig. 2 is a schematic diagram of a defect detection model training process provided in the embodiment of the present application, as shown in fig. 2, including:
step 201: acquiring a training image and a mask image with defects marked; the training image comprises a mobile phone middle frame and a marking frame corresponding to the defect in the mobile phone middle frame. It can be understood that the labeling frame may be manually labeled, or may be an image of a mobile phone middle frame containing a defect labeling frame downloaded from the internet. The resolution of the mask image is the same as that of the training image, the mask image is a binary image, the gray value of a pixel point in the mask image, which corresponds to the area in the defect labeling frame, is 255, and the gray values of other areas are 0. Of course, the gray value of the pixel point corresponding to the region in the defect labeling frame in the mask image may also be 0, and the gray values of the other regions are 255.
Step 202: inputting the training image into a model to be trained to obtain a prediction result output by the model to be trained;
step 203: and optimizing the internal parameters of the model to be trained according to the prediction result and the mask image to obtain the trained defect detection model. And constructing a loss function according to the prediction result and the mask image, and optimizing internal parameters of the model to be trained by using the loss function.
According to the defect detection method and device, the defect detection model is trained through the training image and the mask image, so that the defect area in the image to be recognized can be recognized quickly and accurately through the defect detection model, and the defect detection efficiency is improved.
On the basis of the above embodiment, the obtaining a defect detection result according to the depth information includes:
setting the gray value of the pixel point with the depth information larger than the preset depth as a first gray value, and setting the gray value of the pixel point with the depth information smaller than or equal to the preset depth as a second gray value;
and acquiring a first quantity corresponding to the pixel points of the first gray value, and if the first quantity is greater than a first preset quantity, determining the defect area as a target defect.
In a specific implementation process, after the depth information corresponding to the pixel point region in the depth image is obtained, binarization is performed on the pixel point region, specifically, a preset depth may be preset, the gray value of the pixel point of which the depth information is greater than the preset depth is set to be a first gray value, for example, 0, and the gray value of the pixel point of which the depth information is less than or equal to the preset depth is set to be a second gray value, for example, 255. Counting the number of pixel points with the gray value of 0, recording the number as a first number, and if the first number is larger than a first preset number, indicating that the defect is large, so that the defect area is determined to be the target defect. It is understood that the target defect is understood that the defect region is the defect to be detected in the embodiment of the present application, that is, the target defect is the functional defect.
According to the method and the device, the image of the defect area is binarized according to the depth information, and whether the image is the target defect or not is determined according to the size of the area with the deeper defect depth, so that the accuracy of target defect detection is improved.
In another embodiment, when the terminal device obtains the first number corresponding to the pixel point of the first gray value, the number of the pixel points of the first gray value adjacent to the pixel points of the other first gray values may be counted. In other words, if the gray value of a pixel is the first gray value, but the gray values of the four pixels, i.e., the upper, lower, left and right pixels, of the pixel are not the first gray value, then the pixel is not counted.
In another embodiment, the terminal device may further obtain the defect detection result according to the depth information by:
after the terminal equipment acquires the depth information corresponding to the defect area, counting the number of pixel points in the defect area, wherein the depth information is greater than the preset depth, and if the number exceeds a certain number, indicating that the defect area is large, and considering the defect as a target defect.
On the basis of the above embodiment, the obtaining a defect detection result according to the depth information includes:
acquiring the maximum depth value of the pixel points in the defect area according to the depth information;
and if the maximum depth value is larger than a first preset threshold value, determining the defect area as a target defect.
In a specific implementation process, because depths of functional defects are generally continuous, that is, differences of depth information of adjacent pixels corresponding to a defect region are not very different, in the embodiment of the present application, a maximum depth value of a pixel in the defect region is obtained, and whether the maximum depth value is greater than a first preset threshold is determined, if so, the defect is deeper, and the defect region is determined to be a target defect.
According to the defect detection method and device, whether the maximum depth value of the pixel point in the defect area is larger than the first preset threshold value or not is judged, if yes, the target defect is determined, and therefore the defect detection efficiency is improved.
In another embodiment, it may also be determined whether the defective area is a target defect by:
and calculating to obtain the average depth corresponding to the defect region according to the depth information of each pixel point in the defect region, if the average depth is greater than a second preset threshold, indicating that the defect region is relatively deep, and determining the defect region as a target defect. It is understood that the second preset threshold is preset according to actual conditions, and this is not specifically limited in this embodiment of the present application.
According to the defect detection method and device, the average depth of the defect area is calculated, and if the average depth is larger than the second preset threshold value, the defect area reflects defects seriously on the whole, so that the defect area is determined to be a target defect, and the defect detection accuracy is improved.
On the basis of the foregoing embodiment, fig. 3 is a schematic flow chart of another defect detection method provided in the embodiment of the present application, and as shown in fig. 3, the method includes:
step 301: acquiring an original image and an original depth image;
step 302: segmenting the original image according to a preset size to obtain a plurality of original image blocks;
step 303: inputting the plurality of original image blocks into an object recognition model to obtain a recognition result corresponding to each original image block output by the object recognition model; the identification result corresponding to the original image block forms the image to be identified; the object recognition model is obtained by pre-training, and can be constructed by adopting a neural network, an SSD network, a YOLO network and the like. The training process is similar to the training mode of the defect detection model, and the difference is the training sample, and the training sample of the object recognition model is an image containing the mobile phone middle frame and a mask image containing the mobile phone middle frame.
Step 304: dividing the original depth image according to the identification result corresponding to each original image block to obtain a depth image block corresponding to each original image block; all the depth image blocks form the depth image;
step 305: inputting the image to be identified into a defect detection model to obtain a defect area output by the defect detection model;
step 306: determining depth information corresponding to pixel points in the defect area according to the depth image;
step 307: and obtaining a defect detection result according to the depth information.
In a specific implementation process, steps 305 to 307 are the same as steps 102 to 104 in the above embodiment, and therefore, for the description of steps 305 to 307, refer to steps 102 to 104, which is not described again in this embodiment of the present application.
In step 301, the original image includes an object to be detected, the original depth image includes depth information of the object to be detected, and pixel points in the original image and the original depth image correspond to each other one to one, that is, positions of the pixel points occupied by the object to be detected in the original image and the original depth image are the same. The original image and the original depth image can be obtained by shooting an object to be detected by a high-definition laser line scanning camera, and the acquired original image and the acquired original depth image are sent to the terminal equipment by the high-definition laser line scanning camera. Of course, if the terminal device is provided with an image acquisition device, the terminal device may also acquire an image of the object to be detected. It can be understood that the object to be detected may be a middle frame of a mobile phone. The original image and the original depth image can also be acquired by different image acquisition devices, but the positions of pixel points of the object to be detected in the original image and the original depth image are ensured to be the same. In addition, the original image is different from the image to be recognized in that the original image includes other backgrounds besides the object to be detected. Similarly, the original depth image is different from the depth image in that the original depth image includes depth information of other backgrounds in addition to the depth information of the object to be detected.
In step 302, since the resolution of the original image and the original depth image is high, if the original image and the original depth image are directly processed, a large memory is occupied, and the processing efficiency is low. Therefore, in order to improve the processing efficiency, the original image may be divided into a plurality of original image blocks in a preset size. Taking the original image with a resolution of 800 × 20000 as an example, the original image can be divided into 10 original image blocks according to a preset size of 800 × 2000.
In step 303, in order to remove the interference of the background in the original image and further improve the efficiency of image processing, object identification is performed on each original image block, specifically, each original image block is input into an object identification model, and the object identification model performs object identification on each original image block, so as to obtain an identification result of each original image block. It can be understood that the recognition result of each original image block refers to a region obtained by the object recognition model recognizing the region of interest in each original image block and dividing the region of interest from the corresponding original image block. And splicing the identification results corresponding to all the original image blocks to obtain the image to be identified.
In step 304, since the original depth image corresponds to the original image, after the recognition result is obtained, the original depth image is segmented according to the recognition result corresponding to each original image block, so as to obtain the depth image.
According to the image processing method and device, the original image is divided into the plurality of original image blocks, then the object detection is carried out on each original image block, the image to be recognized is obtained, and the image processing efficiency is improved.
On the basis of the above embodiment, inputting the image to be recognized into a defect detection model to obtain a defect region output by the defect detection model, including:
and inputting the identification result corresponding to each original image block into the defect detection model to obtain a defect area corresponding to each identification result block output by the defect detection model.
In a specific implementation process, after obtaining the identification result corresponding to the original image block, the identification result is input into the defect detection model, and the defect detection model detects each identification result to obtain a defect area corresponding to each identification result.
According to the defect detection method and device, the identification result corresponding to the original image block is input into the defect detection model for defect detection, and the defect detection efficiency is improved.
Fig. 4 is a schematic structural diagram of a defect detection apparatus provided in an embodiment of the present application, where the apparatus may be a module, a program segment, or a code on an electronic device. It should be understood that the apparatus corresponds to the above-mentioned embodiment of the method of fig. 1, and can perform various steps related to the embodiment of the method of fig. 1, and the specific functions of the apparatus can be referred to the description above, and the detailed description is appropriately omitted here to avoid redundancy. The device comprises: an image acquisition module 401, a defect region identification module 402, a depth information statistics module 403, and a defect detection module 404, wherein:
the image acquisition module 401 is configured to acquire an image to be recognized and a depth image of an object to be detected; the positions of pixel points occupied by the object to be detected in the image to be identified and the depth image are the same;
the defect area identification module 402 is configured to input the image to be identified into a defect detection model, and obtain a defect area output by the defect detection model;
the depth information statistics module 403 is configured to determine depth information corresponding to a pixel point in the defect region according to the depth image;
the defect detection module 404 is configured to obtain a defect detection result according to the depth information.
On the basis of the foregoing embodiment, the defect detecting module 404 is specifically configured to:
setting the gray value of the pixel point with the depth information larger than the preset depth as a first gray value, and setting the gray value of the pixel point with the depth information smaller than or equal to the preset depth as a second gray value;
and acquiring a first quantity corresponding to the pixel points of the first gray value, and if the first quantity is greater than a first preset quantity, determining the defect area as a target defect.
On the basis of the foregoing embodiment, the defect detecting module 404 is specifically configured to:
acquiring the maximum depth value of the pixel points in the defect area according to the depth information;
and if the maximum depth value is larger than a first preset threshold value, determining the defect area as a target defect.
On the basis of the foregoing embodiment, the defect detecting module 404 is specifically configured to:
determining the average depth corresponding to the defect area according to the depth information;
and if the average depth is larger than a second preset threshold value, determining the defect area as a target defect.
On the basis of the foregoing embodiment, the image acquisition module 401 is specifically configured to:
acquiring an original image and an original depth image;
segmenting the original image according to a preset size to obtain a plurality of original image blocks;
inputting the plurality of original image blocks into an object recognition model to obtain a recognition result corresponding to each original image block output by the object recognition model; the identification result corresponding to the original image block forms the image to be identified;
dividing the original depth image according to the identification result corresponding to each original image block to obtain a depth image block corresponding to each original image block; all the blocks of depth images constitute the depth image.
On the basis of the foregoing embodiment, the defective area identifying module 402 is specifically configured to:
and inputting the identification result corresponding to each original image block into the defect detection model to obtain a defect area corresponding to each identification result block output by the defect detection model.
On the basis of the above embodiment, the apparatus further includes a model training module for:
acquiring a training image and a mask image with defects marked;
inputting the training image into a model to be trained to obtain a prediction result output by the model to be trained;
and optimizing the internal parameters of the model to be trained according to the prediction result and the mask image to obtain the trained defect detection model.
Fig. 5 is a schematic structural diagram of an entity of an electronic device provided in an embodiment of the present application, and as shown in fig. 5, the electronic device includes: a processor (processor)501, a memory (memory)502, and a bus 503; wherein the content of the first and second substances,
the processor 501 and the memory 502 are communicated with each other through the bus 503;
the processor 501 is configured to call program instructions in the memory 502 to perform the methods provided by the above-mentioned method embodiments, for example, including: acquiring an image to be identified and a depth image of an object to be detected; the positions of pixel points occupied by the object to be detected in the image to be identified and the depth image are the same; inputting the image to be identified into a defect detection model to obtain a defect area output by the defect detection model; determining depth information corresponding to pixel points in the defect area according to the depth image; and obtaining a defect detection result according to the depth information.
The processor 501 may be an integrated circuit chip having signal processing capabilities. The Processor 501 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. Which may implement or perform the various methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The Memory 502 may include, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read Only Memory (PROM), Erasable Read Only Memory (EPROM), Electrically Erasable Read Only Memory (EEPROM), and the like.
The present embodiment discloses a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the method provided by the above-mentioned method embodiments, for example, comprising: acquiring an image to be identified and a depth image of an object to be detected; the positions of pixel points occupied by the object to be detected in the image to be identified and the depth image are the same; inputting the image to be identified into a defect detection model to obtain a defect area output by the defect detection model; determining depth information corresponding to pixel points in the defect area according to the depth image; and obtaining a defect detection result according to the depth information.
The present embodiments provide a non-transitory computer-readable storage medium storing computer instructions that cause the computer to perform the methods provided by the above method embodiments, for example, including: acquiring an image to be identified and a depth image of an object to be detected; the positions of pixel points occupied by the object to be detected in the image to be identified and the depth image are the same; inputting the image to be identified into a defect detection model to obtain a defect area output by the defect detection model; determining depth information corresponding to pixel points in the defect area according to the depth image; and obtaining a defect detection result according to the depth information.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
In addition, units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
Furthermore, the functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. A method of defect detection, comprising:
acquiring an image to be identified and a depth image of an object to be detected; the positions of pixel points occupied by the object to be detected in the image to be identified and the depth image are the same;
inputting the image to be identified into a defect detection model to obtain a defect area output by the defect detection model;
determining depth information corresponding to pixel points in the defect area according to the depth image;
and obtaining a defect detection result according to the depth information.
2. The method of claim 1, wherein obtaining the defect detection result according to the depth information comprises:
setting the gray value of the pixel point with the depth information larger than the preset depth as a first gray value, and setting the gray value of the pixel point with the depth information smaller than or equal to the preset depth as a second gray value;
and acquiring a first quantity corresponding to the pixel points of the first gray value, and if the first quantity is greater than a first preset quantity, determining the defect area as a target defect.
3. The method of claim 1, wherein obtaining the defect detection result according to the depth information comprises:
acquiring the maximum depth value of the pixel points in the defect area according to the depth information;
and if the maximum depth value is larger than a first preset threshold value, determining the defect area as a target defect.
4. The method of claim 1, wherein obtaining the defect detection result according to the depth information comprises:
determining the average depth corresponding to the defect area according to the depth information;
and if the average depth is larger than a second preset threshold value, determining the defect area as a target defect.
5. The method according to claim 1, wherein the acquiring the image to be recognized and the depth image of the object to be detected comprises:
acquiring an original image and an original depth image;
segmenting the original image according to a preset size to obtain a plurality of original image blocks;
inputting the plurality of original image blocks into an object recognition model to obtain a recognition result corresponding to each original image block output by the object recognition model; the identification result corresponding to the original image block forms the image to be identified;
dividing the original depth image according to the identification result corresponding to each original image block to obtain a depth image block corresponding to each original image block; all the blocks of depth images constitute the depth image.
6. The method of claim 5, wherein inputting the image to be recognized into a defect detection model, and obtaining the defect region output by the defect detection model comprises:
and inputting the identification result corresponding to each original image block into the defect detection model to obtain a defect area corresponding to each identification result block output by the defect detection model.
7. The method according to any one of claims 1-6, further comprising:
acquiring a training image and a mask image with defects marked;
inputting the training image into a model to be trained to obtain a prediction result output by the model to be trained;
and optimizing the internal parameters of the model to be trained according to the prediction result and the mask image to obtain the trained defect detection model.
8. A defect detection apparatus, comprising:
the image acquisition module is used for acquiring an image to be identified and a depth image of an object to be detected; the positions of pixel points occupied by the object to be detected in the image to be identified and the depth image are the same;
the defect area identification module is used for inputting the image to be identified into a defect detection model to obtain a defect area output by the defect detection model;
the depth information counting module is used for determining depth information corresponding to pixel points in the defect area according to the depth image;
and the defect detection module is used for obtaining a defect detection result according to the depth information.
9. An electronic device, comprising: a processor, a memory, and a bus, wherein,
the processor and the memory are communicated with each other through the bus;
the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of any one of claims 1-7.
10. A non-transitory computer-readable storage medium storing computer instructions which, when executed by a computer, cause the computer to perform the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111045437.3A CN113763355A (en) | 2021-09-07 | 2021-09-07 | Defect detection method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111045437.3A CN113763355A (en) | 2021-09-07 | 2021-09-07 | Defect detection method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113763355A true CN113763355A (en) | 2021-12-07 |
Family
ID=78793657
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111045437.3A Pending CN113763355A (en) | 2021-09-07 | 2021-09-07 | Defect detection method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113763355A (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114202528A (en) * | 2021-12-13 | 2022-03-18 | 北京创源微致软件有限公司 | Product detection method and device |
| CN114399467A (en) * | 2021-12-15 | 2022-04-26 | 北京德风新征程科技有限公司 | Case shell detection method and device, electronic equipment and computer readable medium |
| CN114494142A (en) * | 2021-12-28 | 2022-05-13 | 深圳科瑞技术股份有限公司 | Mobile terminal middle frame defect detection method and device based on deep learning |
| CN114663404A (en) * | 2022-03-25 | 2022-06-24 | 广州中科云图智能科技有限公司 | Line defect identification method and device, electronic equipment and storage medium |
| CN114782445A (en) * | 2022-06-22 | 2022-07-22 | 深圳思谋信息科技有限公司 | Object defect detection method and device, computer equipment and storage medium |
| CN114972892A (en) * | 2022-07-12 | 2022-08-30 | 山东嘉通专用汽车制造有限公司 | Automobile brake pad defect classification method based on computer vision |
| CN115222739A (en) * | 2022-09-20 | 2022-10-21 | 成都数之联科技股份有限公司 | Defect labeling method, device, storage medium, equipment and computer program product |
| CN116051542A (en) * | 2023-03-06 | 2023-05-02 | 深圳市深视智能科技有限公司 | Defect detection method and defect detection device |
| CN116206111A (en) * | 2023-03-07 | 2023-06-02 | 广州市易鸿智能装备有限公司 | Defect identification method and device, electronic equipment and storage medium |
| CN116310424A (en) * | 2023-05-17 | 2023-06-23 | 青岛创新奇智科技集团股份有限公司 | Equipment quality assessment method, device, terminal and medium based on image recognition |
| CN116773546A (en) * | 2023-06-20 | 2023-09-19 | 上海感图网络科技有限公司 | Copper plating plate stacking defect detection method, copper plating plate stacking defect detection device and storage medium |
| CN116797590A (en) * | 2023-07-03 | 2023-09-22 | 深圳市拓有软件技术有限公司 | Mura defect detection method and system based on machine vision |
| CN116952958A (en) * | 2023-09-18 | 2023-10-27 | 杭州百子尖科技股份有限公司 | Defect detection method, device, electronic equipment and storage medium |
| CN116993727A (en) * | 2023-09-26 | 2023-11-03 | 宁德思客琦智能装备有限公司 | Detection method and device, electronic equipment and computer readable medium |
| CN117058155A (en) * | 2023-10-13 | 2023-11-14 | 西安空天机电智能制造有限公司 | 3DP metal printing powder spreading defect detection method, device, equipment and medium |
| CN117078666A (en) * | 2023-10-13 | 2023-11-17 | 东声(苏州)智能科技有限公司 | Two-dimensional and three-dimensional combined defect detection method, device, medium and equipment |
| CN117078665A (en) * | 2023-10-13 | 2023-11-17 | 东声(苏州)智能科技有限公司 | Product surface defect detection method and device, storage medium and electronic equipment |
| CN117495829A (en) * | 2023-11-15 | 2024-02-02 | 广东昭明电子集团股份有限公司 | Intelligent watch hardware quality detection method |
| CN117871538A (en) * | 2024-03-07 | 2024-04-12 | 江苏时代新能源科技有限公司 | Defect detection system, defect detection method and related equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101694472A (en) * | 2009-10-23 | 2010-04-14 | 郭震 | Defect hole image recognition method |
| CN103077526A (en) * | 2013-02-01 | 2013-05-01 | 苏州华兴致远电子科技有限公司 | Train abnormality detection method and system with deep detection function |
| US20180322623A1 (en) * | 2017-05-08 | 2018-11-08 | Aquifi, Inc. | Systems and methods for inspection and defect detection using 3-d scanning |
| CN109978865A (en) * | 2019-03-28 | 2019-07-05 | 中核建中核燃料元件有限公司 | A kind of method, apparatus for the detection of nuclear fuel rod face of weld |
| CN110136130A (en) * | 2019-05-23 | 2019-08-16 | 北京阿丘机器人科技有限公司 | A kind of method and device of testing product defect |
| CN111044522A (en) * | 2019-12-14 | 2020-04-21 | 中国科学院深圳先进技术研究院 | Defect detection method and device and terminal equipment |
| CN111982911A (en) * | 2020-07-10 | 2020-11-24 | 深圳先进技术研究院 | Method and device for detecting defects of circuit board, terminal equipment and storage medium |
| CN112532882A (en) * | 2020-11-26 | 2021-03-19 | 维沃移动通信有限公司 | Image display method and device |
-
2021
- 2021-09-07 CN CN202111045437.3A patent/CN113763355A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101694472A (en) * | 2009-10-23 | 2010-04-14 | 郭震 | Defect hole image recognition method |
| CN103077526A (en) * | 2013-02-01 | 2013-05-01 | 苏州华兴致远电子科技有限公司 | Train abnormality detection method and system with deep detection function |
| US20180322623A1 (en) * | 2017-05-08 | 2018-11-08 | Aquifi, Inc. | Systems and methods for inspection and defect detection using 3-d scanning |
| CN109978865A (en) * | 2019-03-28 | 2019-07-05 | 中核建中核燃料元件有限公司 | A kind of method, apparatus for the detection of nuclear fuel rod face of weld |
| CN110136130A (en) * | 2019-05-23 | 2019-08-16 | 北京阿丘机器人科技有限公司 | A kind of method and device of testing product defect |
| CN111044522A (en) * | 2019-12-14 | 2020-04-21 | 中国科学院深圳先进技术研究院 | Defect detection method and device and terminal equipment |
| CN111982911A (en) * | 2020-07-10 | 2020-11-24 | 深圳先进技术研究院 | Method and device for detecting defects of circuit board, terminal equipment and storage medium |
| CN112532882A (en) * | 2020-11-26 | 2021-03-19 | 维沃移动通信有限公司 | Image display method and device |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114202528A (en) * | 2021-12-13 | 2022-03-18 | 北京创源微致软件有限公司 | Product detection method and device |
| CN114399467A (en) * | 2021-12-15 | 2022-04-26 | 北京德风新征程科技有限公司 | Case shell detection method and device, electronic equipment and computer readable medium |
| CN114494142A (en) * | 2021-12-28 | 2022-05-13 | 深圳科瑞技术股份有限公司 | Mobile terminal middle frame defect detection method and device based on deep learning |
| CN114663404A (en) * | 2022-03-25 | 2022-06-24 | 广州中科云图智能科技有限公司 | Line defect identification method and device, electronic equipment and storage medium |
| CN114782445A (en) * | 2022-06-22 | 2022-07-22 | 深圳思谋信息科技有限公司 | Object defect detection method and device, computer equipment and storage medium |
| CN114972892A (en) * | 2022-07-12 | 2022-08-30 | 山东嘉通专用汽车制造有限公司 | Automobile brake pad defect classification method based on computer vision |
| CN115222739A (en) * | 2022-09-20 | 2022-10-21 | 成都数之联科技股份有限公司 | Defect labeling method, device, storage medium, equipment and computer program product |
| CN116051542A (en) * | 2023-03-06 | 2023-05-02 | 深圳市深视智能科技有限公司 | Defect detection method and defect detection device |
| CN116206111A (en) * | 2023-03-07 | 2023-06-02 | 广州市易鸿智能装备有限公司 | Defect identification method and device, electronic equipment and storage medium |
| CN116206111B (en) * | 2023-03-07 | 2024-02-02 | 广州市易鸿智能装备有限公司 | Defect identification method and device, electronic equipment and storage medium |
| CN116310424A (en) * | 2023-05-17 | 2023-06-23 | 青岛创新奇智科技集团股份有限公司 | Equipment quality assessment method, device, terminal and medium based on image recognition |
| CN116310424B (en) * | 2023-05-17 | 2023-08-18 | 青岛创新奇智科技集团股份有限公司 | Equipment quality assessment method, device, terminal and medium based on image recognition |
| CN116773546A (en) * | 2023-06-20 | 2023-09-19 | 上海感图网络科技有限公司 | Copper plating plate stacking defect detection method, copper plating plate stacking defect detection device and storage medium |
| CN116773546B (en) * | 2023-06-20 | 2024-03-22 | 上海感图网络科技有限公司 | Copper plating plate stacking defect detection method, copper plating plate stacking defect detection device and storage medium |
| CN116797590A (en) * | 2023-07-03 | 2023-09-22 | 深圳市拓有软件技术有限公司 | Mura defect detection method and system based on machine vision |
| CN116952958A (en) * | 2023-09-18 | 2023-10-27 | 杭州百子尖科技股份有限公司 | Defect detection method, device, electronic equipment and storage medium |
| CN116952958B (en) * | 2023-09-18 | 2023-12-29 | 杭州百子尖科技股份有限公司 | Defect detection method, device, electronic equipment and storage medium |
| CN116993727A (en) * | 2023-09-26 | 2023-11-03 | 宁德思客琦智能装备有限公司 | Detection method and device, electronic equipment and computer readable medium |
| CN116993727B (en) * | 2023-09-26 | 2024-03-08 | 宁德思客琦智能装备有限公司 | Detection method and device, electronic equipment and computer readable medium |
| CN117078665A (en) * | 2023-10-13 | 2023-11-17 | 东声(苏州)智能科技有限公司 | Product surface defect detection method and device, storage medium and electronic equipment |
| CN117078666A (en) * | 2023-10-13 | 2023-11-17 | 东声(苏州)智能科技有限公司 | Two-dimensional and three-dimensional combined defect detection method, device, medium and equipment |
| CN117058155B (en) * | 2023-10-13 | 2024-03-12 | 西安空天机电智能制造有限公司 | 3DP metal printing powder spreading defect detection method, device, equipment and medium |
| CN117058155A (en) * | 2023-10-13 | 2023-11-14 | 西安空天机电智能制造有限公司 | 3DP metal printing powder spreading defect detection method, device, equipment and medium |
| CN117078666B (en) * | 2023-10-13 | 2024-04-09 | 东声(苏州)智能科技有限公司 | Two-dimensional and three-dimensional combined defect detection method, device, medium and equipment |
| CN117078665B (en) * | 2023-10-13 | 2024-04-09 | 东声(苏州)智能科技有限公司 | Product surface defect detection method and device, storage medium and electronic equipment |
| CN117495829A (en) * | 2023-11-15 | 2024-02-02 | 广东昭明电子集团股份有限公司 | Intelligent watch hardware quality detection method |
| CN117495829B (en) * | 2023-11-15 | 2024-04-30 | 广东昭明电子集团股份有限公司 | Intelligent watch hardware quality detection method |
| CN117871538A (en) * | 2024-03-07 | 2024-04-12 | 江苏时代新能源科技有限公司 | Defect detection system, defect detection method and related equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113763355A (en) | Defect detection method and device, electronic equipment and storage medium | |
| CN111612763B (en) | Mobile phone screen defect detection method, device and system, computer equipment and medium | |
| CN111753692B (en) | Target object extraction method, product detection method, device, computer and medium | |
| CN113379680A (en) | Defect detection method, defect detection device, electronic equipment and computer readable storage medium | |
| CN115018840B (en) | Method, system and device for detecting cracks of precision casting | |
| CN109598298B (en) | Image object recognition method and system | |
| CN114972191A (en) | Method and device for detecting farmland change | |
| CN114978037B (en) | Solar cell performance data monitoring method and system | |
| CN116433666B (en) | Board card line defect online identification method, system, electronic equipment and storage medium | |
| CN111340796A (en) | Defect detection method and device, electronic equipment and storage medium | |
| CN116168351B (en) | Inspection method and device for power equipment | |
| CN117147561B (en) | Surface quality detection method and system for metal zipper | |
| CN110796039B (en) | Face flaw detection method and device, electronic equipment and storage medium | |
| CN115471476A (en) | Method, device, equipment and medium for detecting component defects | |
| CN116309532A (en) | Method, device, equipment and medium for detecting quality of target object | |
| CN114708247A (en) | Cigarette case packaging defect identification method and device based on deep learning | |
| CN111178445A (en) | Image processing method and device | |
| CN113283439B (en) | Intelligent counting method, device and system based on image recognition | |
| CN113191977A (en) | Image enhancement system for target detection and identification under severe environment condition | |
| CN114596243A (en) | Defect detection method, device, equipment and computer readable storage medium | |
| CN115937555A (en) | Industrial defect detection algorithm based on standardized flow model | |
| CN116977249A (en) | Defect detection method, model training method and device | |
| CN115984185A (en) | Paper towel package defect detection method, device and system and storage medium | |
| CN113870754B (en) | Method and system for judging defects of panel detection electronic signals | |
| CN115639578A (en) | Beidou navigation positioning monitoring processing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |